Commit Graph

119 Commits

Author SHA1 Message Date
Ludovico Magnocavallo 980011806c
fix permadiff in cloud nat module (#2301) 2024-05-23 08:38:03 +02:00
Ludovico Magnocavallo be9214f99a
add support for tenant factory CI/CD (#2297) 2024-05-21 10:39:47 +02:00
simonebruzzechesse 79af34b69e
Add wif permissions to bootstrap tf SA (#2290)
* add wif permissions to bootstrap tf SA
2024-05-20 18:15:23 +02:00
Simone Ruffilli 21f3b733ab
FAST: Cleanup/harmonization of Simple and NVA net stages (#2287)
Cleanup/harmonization of Simple and NVA net stages
2024-05-16 16:49:15 +03:00
Simone Ruffilli 887c7e7926
Unify VPN and Peering FAST stages (#2284)
* Unify VPN and Peering FAST stages
2024-05-16 12:18:32 +03:00
Ludovico Magnocavallo 7a5dd4e6db
FAST: add top-level folders and restructure teams/tenants in resman (#2254)
* remove teams and tenants from resman

* move fast features to stage 1, fix test inventories

* folders

* fix factory, add top level folder resources to outputs

* tfdoc

* stage 0 log sink defs

* tfdoc

* enable toc in resman readme

* simple tenants

* fast compatibility automation and logging

* testing fast-compatible tenants

* testing fast-compatible tenants

* tfdoc

* remove mt stages

* remove tests, fix links

* disable tflint

* fast tests

* make organization conditional in resman

* check names tool

* export real prefix to tfvars, prevent destroy errors

* prefix validation

* fix billing account export format

* tfdoc

* root node folder

* resman changes

* tenant resman roles

* first apply of tenant resman

* tenant log sinks in stage 1

* fix test vars

* tfdoc

* tenant vpc-sc access policy

* fix tests expected values

* tenant CI/CD

* identity providers

* wif

* tfdoc

* add comments to identity locals

* full-feature tenant resman apply

* tenant billing IAM

* stage test

* fix CI/CD comments

* tenant net stage verified

* tenant sec stage verified

* fix test

* README work

* tfdoc

* README

* README rewording

* README rewording

* tfdoc

* FAST excalidraw

* review comments

* diagram review changes

* add iam log sink for tenants

* remove redundant try from security stage

* Implement tflint-fast in Python driven by tftest.yaml files

* tflint

* test ci changes

* revert linting changes

* disable tflint for fast

* Create junit-style report for FAST tflint

* Remove junit-reporter

* YAPF tflint-fast.py

* Output tflint FAST to job summary

* Step summary

* Disable step_summary as output is not useful

* ignore tflint warning

* re-enable tflint on FAST

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-05-15 09:17:13 +00:00
Simone Ruffilli 9a26fe8635
Add support for reserved_internal_range in net-vpc (#2275)
Adds support for reserved_internal_range to net-vpc
2024-05-14 22:19:45 +03:00
Wiktor Niesiobędzki af253c9702
Fix 0-bootstrap iam_by_principals not taking into account all principals (#2267)
* Fix 0-bootstrap iam_by_principals not taking into account all principals
* Add test-case for iam_by_principals for 0-bootstrap stage

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2024-05-12 21:02:04 +02:00
Julio Castillo d838c4ac47
Make Simple NVA route IAP traffic through NIC 0 (#2262) 2024-05-09 18:29:25 +02:00
Julio Castillo c58850c096
Add Hybrid NAT support (#2261)
* Updates to support hybid NAT

* Fix readme

* Fix variable order
2024-05-09 13:24:41 +00:00
Ludovico Magnocavallo c9503d5ac5
Remove data source from folder module (#2260)
* remove data source from folder module

* fix fast tfdoc

* fix locals type error

* fix folder test

* fix fast test
2024-05-09 13:09:54 +00:00
Julio Castillo 94c32c1d71
Misc FAST fixes (#2253)
* Misc FAST fixes

* Fix readme

* Fix FAST nva bgp tests
2024-05-02 06:56:26 +00:00
Julio Castillo 99129d54a3
Update FAST logging (#2235)
* Update FAST logging

* Fix readme

* Fix tests
2024-04-25 08:31:51 +02:00
Wiktor Niesiobędzki 024d3255e6
Generalization of tflint call for FAST stages (#2225)
* Generalization of tflint call for FAST

* Fix tfvars path

* Fix tfvars path - depending where the file is

* Fix regex

* Reeanble linting

* Align test directory to stage name

* Align all fast stages to use tftest
2024-04-18 21:04:24 +02:00
Julio Castillo 3af7e257d2
Add tflint to pipelines (#2220)
* Fix terraform_deprecated_index

https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_index.md

* Fix terraform_deprecated_interpolation

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_interpolation.md

* Fix more indexing

* Remove unused variable

* Enable TFLint for modules

* Add tflint config file

* Fix chdir

* Lint modules

* TFLint fixes

* TFLint

* Fixes binauthz README

* Fixes DNS response policy tests. Restores MIG outputs.

* Fixes other DNS response policy tests.

* Update tests for fast 2-e

* Moar fixed tests

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
2024-04-17 10:23:48 +02:00
Ludovico Magnocavallo 198d90c6fc
Remove data source from net-vpc module (#2216)
* remove data source from net-vpc module

* fix test inventories

* remove data source, fix fast inventories
2024-04-16 14:11:12 +03:00
Julio Castillo a74a106f8b
Add new org policies to FAST (#2215)
* Add new org policies to FAST

* Fix tests
2024-04-15 15:29:24 +02:00
Ludovico Magnocavallo 9cace5272f
Add support for quotas to project module (#2210)
* add support for quotas to project module

* tfdoc

* better outputs

* Ensure keys in `quota_configs` are unique

* update fast tests

* Make quota E2E testable

* Remove quota project caveat

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-04-10 17:03:04 +00:00
Ludovico Magnocavallo 8511170412
FAST security stage refactor (#2203)
* first working change, missing docs and tests

* fixes

* tests

* boilerplate

* tfdoc
2024-04-07 20:14:39 -07:00
Julio Castillo a9ac0f40cd
Add variable to resman to control top-level folder IAM (#2196) 2024-04-04 10:26:35 +02:00
Ludovico Magnocavallo 33ffe2daa6
Allow r/o project factory SAs access to folder-level IAM (#2195)
* allow project factory SAs read access to folder-level IAM

* tfdoc
2024-04-03 21:51:46 +02:00
Elia b80132a618
FAST GCVE stage (#2191)
* GCVE stage prerequisites

* fix gcve prereq

* gcve stage first deploy

* Updated readme filex

* docs updated

* some fixes after testing

* updated readme files

* elia fix

* gcve net admin custom role

* gcve net admin custom role

* elia fix

* ven peering deploy

* elia fix

* added blueprint and stage tests

* Edits to Readme files

* typo in outputs

* clean-up

* gcve stage tests fix

* readme fix

* fix sorting

* fix copyrights and readme file

* fix test

* fix copyright

* fixed gcve feature flag cond.

* removed validation

* fixed typo

* fixed typo

* fixed gcve tests

* fixed typo

* fixed typo

* fixed sorting

* fixed sorting

---------

Co-authored-by: Konrad Schieban <kschieban@google.com>
2024-04-03 17:25:12 +02:00
Julio Castillo 759e85d6af
Logging updates (#2139)
* Add support for logging settings to the organization module

* Enable log analytics on FAST-managed logging buckets

* Fix README

* Fix tests
2024-03-08 10:07:12 +01:00
Ludovico Magnocavallo 3972eb6df4
Align resource names in FAST networking stages (#2115)
* stage c nva

* fix tests

* remove moved blocks from net c stage

* simplify subnet naming in stage 2 net e

* address most renames in stage 2 e

* address most renames in stage 2 e

* address most renames in stage 2 e

* complete renames in stage 2 e

* use non-regional names in subnets

* use non-regional names in subnets

* use non-regional names in subnets
2024-02-29 07:45:19 +01:00
Ludovico Magnocavallo dbabfb9ae0
Add support for billing budgets to project factory (#2112)
* align factory variable name in project factory module

* tested

* align fast stage
2024-02-27 18:13:49 +00:00
Ludovico Magnocavallo eb23bb62d2
Support domainless orgs in FAST (#2086)
* bootstrap

* align org policies to domainless enforced ones

* fix #2073

* fix tests

* fix team admin attribute in resman stage
2024-02-19 08:29:37 +00:00
simonebruzzechesse 946ae148f7
Add workforce_identity_federation in 0-bootstrap (#2077)
* add workforce_identity_federation in 0-bootstrap
* update tests
2024-02-15 00:10:24 +01:00
Ludovico Magnocavallo 71a64487d5
Extend FAST to support different principal types (#2064)
* add doc draft

* typos

* typo

* typo

* typos

* rewording

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* move iam variables to a separate file

* move billing-account module to iam_principals

* move data-catalog-policy-tag module to iam_principals

* move dataplex-datascan module to iam_principals

* move dataproc module to iam_principals

* move folder module to iam_principals

* copyright

* move organization module to iam_principals

* move project module to iam_principals

* move source-repository module to iam_principals

* update blueprints for iam_principals interface

* FAST bootstrap

* module READMEs fixes

* FAST bootstrap

* FAST networking stages

* FAST security stage

* FAST gke stage

* FAST multitenant bootstrap stage

* FAST multitenant resman stage

* tfdoc

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* fix module test

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Rename iam_principals to iam_by_principals

* Update IAM template to include iam_by_principals

* Update Resman README

* Fix ADR link format

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2024-02-12 14:35:30 +01:00
Wiktor Niesiobędzki c58e61e98e Introduce variable to disable imports, fix tests 2024-02-07 17:25:11 +01:00
Julio Castillo e219d92217
Enable additional recommended org policies (#2050)
* Enable additional recommended org policies

Fixes #2047
Fixes #2048
Fixes #2049

* Fix tests
2024-02-05 10:46:37 +01:00
Ludovico Magnocavallo 5448ab64c4
Leverage net-vpc module for DNS logging in FAST (#2041)
* revert #2023

* leverage net vpc module for dns logging in fast
2024-02-03 08:16:00 +01:00
Julio Castillo 13636ba07b
Make Cloud NAT creation optional in FAST net stages. (#2038)
* Make Cloud NAT creation optional in FAST net stages.

Fixes #2021

* Update READMEs
2024-02-02 10:58:16 +01:00
Julio Castillo 4c68c016a9
Add DNS query logging to FAST net stages (#2033)
* Add DNS query logging to FAST net stages

Fixes #2020

* Update readmes

* Add variable to toggle DNS logging

* Extend DNS logging toggle to other net stages
2024-01-31 13:44:51 +01:00
Julio Castillo da95434308
logging for default ingress rules in FAST (#2030)
* Add default ingress deny rule with logging to FAST net stages.

Fixes #2024

* Allow firewall factory to omit rules key

* Fix tests

* Fix fast tests

* fix fast tests
2024-01-30 16:53:01 +00:00
Ludovico Magnocavallo 99228363b2
enforce trusted image projects constraint in stage 0 (#2014) 2024-01-26 10:14:44 +00:00
Ludovico Magnocavallo 6d9b6403dd
add support for essential contacts to FAST (#2010) 2024-01-25 12:20:14 +01:00
Ludovico Magnocavallo c5416f3af1
Tighten up security of automation project (CSPR-related) (#2009)
* enforce compute/iam policies on the automation project

* tests
2024-01-24 18:40:36 +00:00
Ludovico Magnocavallo 070584ae74
Checklist attribution bucket (#2000) 2024-01-23 11:32:14 +00:00
Ludovico Magnocavallo 4b911a6047
update checklist parsing for top-level key (#1997) 2024-01-23 07:34:03 +01:00
Ludovico Magnocavallo a8c84357f4
Integrate checklist data in FAST (#1969)
* add locals for additive and authoritative org iam roles

* first shot at IAM and logging location

* tfdoc

* use locals for locations

* fix file parsing, resman stubs

* initial resman implementation

* remove unneeded code

* fix data file

* replace dumb yamldecode

* fix wrong type in organization additive bindings try

* simplify logging local

* Use check asserts for version and org id

* Checks on checklist for resman

* refactor checks, ignore checklist files on wrong org id

* stage 0 tests

* fix checklist checks

* stage 1 tests

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-01-18 05:45:29 +01:00
Ludovico Magnocavallo 9d6e61428b
(WIP) Read-only service accounts for automation and CI/CD (#1899)
* add design doc for the new CI/CD sa

* describe the actual implementation

* specify which files will need to be changed

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Fix typo

* stage 0 read-only service accounts

* stage 0 IAM map

* linting

* cicd read-only service accounts

* tweak workflow templates

* roles and github workflow fixes

* tfdoc

* Ad-hoc custom role factory for FAST bootstrap

* use factory variable for custom roles data path

* custom roles factory in org/project modules

* tfdoc

* rename custom roles factory variable, fix gitlab template

* gitlab workflow fixes

* fix merge

* output plan results on failed assertion

* update stage 0 expected values

* data platform branch

* gke

* networking

* security

* project factory

* outputs

* workflow templates

* resman apply fixes

* tfdoc

* fix stage 1 test fixture

* fix gh workflow

* read-only resman sa roles

* fix test

* read-only resman sa roles

* read-only resman sa roles

* read-only resman sa roles

* read-only resman sa roles

* fix test variables

* rename wif principal attribute names

* rename wif principal variables

* multitenant stages

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-12-27 11:33:16 +00:00
simonebruzzechesse c9a8d777ba
Add kernels.googleusercontent.com zone in dns response policy (#1940)
* Add kernels.googleusercontent.com zone in dns response policy
* update fast tests
2023-12-20 11:18:11 +01:00
Simone Ruffilli 6d89b88149
versions.tf maintenance + copyright notice bump (#1782)
* Bump copyright notice to 2023

* Delete versions.tf on blueprints

* Pin provider to major version 5

* Remove comment

* Fix lint

* fix bq-ml blueprint readme

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-20 18:17:47 +02:00
Ludovico Magnocavallo e0d84fb10b
add sink for workspace logs (#1780) 2023-10-19 14:51:01 +00:00
Ludovico Magnocavallo e7e188818a
Add service usage consumer role to IaC SAs, refactor delegated grants in FAST (#1773)
* add serviceusage role to iac sas, refactor delegated grants

* fix test

* tfdoc
2023-10-18 12:18:31 +00:00
Ludovico Magnocavallo 252127bde5
Billing account module (#1743)
* initial untested draft

* readme and tests

* folder module tfdoc

* remove redundant billing cost manager role in fast stage 0

* fix FAST test
2023-10-15 15:02:50 +00:00
Julio Castillo 4b15605711
Fix dnssec keys lookup (#1728)
* Fix dnssec keys lookup

* Fix DNS examples

* Fix FAST and blueprints resource counts
2023-10-03 21:37:21 +02:00
Julio Castillo 1dfa72cadf
Define and adopt standard IP ranges for FAST networking (#1697)
* Define and adopt standard IP ranges for FAST networking

This PR documents and adopts a consistent IP address plan for FAST
networking stages

Fixes #1644

* Fix documented aggregated ranges for FAST

* Fix tests

* Fix ip ranges in documentation

* Fix NVA stages README
2023-09-21 14:27:53 +00:00
simonebruzzechesse 77c1e69666
New phpIPAM serverless third parties solution in blueprints (#1642)
* Added new phpIPAM serverless third parties solution in blueprints
* added jit to iap.googleapis.com service in project module
* updated tests
2023-09-07 15:30:22 +02:00
Luca Prete 12e78af055
Fix project factory blueprint and fast stage (#1654) 2023-09-07 12:48:39 +00:00