3972eb6df4
Align resource names in FAST networking stages ( #2115 )
...
* stage c nva
* fix tests
* remove moved blocks from net c stage
* simplify subnet naming in stage 2 net e
* address most renames in stage 2 e
* address most renames in stage 2 e
* address most renames in stage 2 e
* complete renames in stage 2 e
* use non-regional names in subnets
* use non-regional names in subnets
* use non-regional names in subnets
2024-02-29 07:45:19 +01:00
dbabfb9ae0
Add support for billing budgets to project factory ( #2112 )
...
* align factory variable name in project factory module
* tested
* align fast stage
2024-02-27 18:13:49 +00:00
eb23bb62d2
Support domainless orgs in FAST ( #2086 )
...
* bootstrap
* align org policies to domainless enforced ones
* fix #2073
* fix tests
* fix team admin attribute in resman stage
2024-02-19 08:29:37 +00:00
946ae148f7
Add workforce_identity_federation in 0-bootstrap ( #2077 )
...
* add workforce_identity_federation in 0-bootstrap
* update tests
2024-02-15 00:10:24 +01:00
71a64487d5
Extend FAST to support different principal types ( #2064 )
...
* add doc draft
* typos
* typo
* typo
* typos
* rewording
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* move iam variables to a separate file
* move billing-account module to iam_principals
* move data-catalog-policy-tag module to iam_principals
* move dataplex-datascan module to iam_principals
* move dataproc module to iam_principals
* move folder module to iam_principals
* copyright
* move organization module to iam_principals
* move project module to iam_principals
* move source-repository module to iam_principals
* update blueprints for iam_principals interface
* FAST bootstrap
* module READMEs fixes
* FAST bootstrap
* FAST networking stages
* FAST security stage
* FAST gke stage
* FAST multitenant bootstrap stage
* FAST multitenant resman stage
* tfdoc
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* fix module test
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Rename iam_principals to iam_by_principals
* Update IAM template to include iam_by_principals
* Update Resman README
* Fix ADR link format
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2024-02-12 14:35:30 +01:00
c58e61e98e
Introduce variable to disable imports, fix tests
2024-02-07 17:25:11 +01:00
e219d92217
Enable additional recommended org policies ( #2050 )
...
* Enable additional recommended org policies
Fixes #2047
Fixes #2048
Fixes #2049
* Fix tests
2024-02-05 10:46:37 +01:00
5448ab64c4
Leverage net-vpc module for DNS logging in FAST ( #2041 )
...
* revert #2023
* leverage net vpc module for dns logging in fast
2024-02-03 08:16:00 +01:00
13636ba07b
Make Cloud NAT creation optional in FAST net stages. ( #2038 )
...
* Make Cloud NAT creation optional in FAST net stages.
Fixes #2021
* Update READMEs
2024-02-02 10:58:16 +01:00
4c68c016a9
Add DNS query logging to FAST net stages ( #2033 )
...
* Add DNS query logging to FAST net stages
Fixes #2020
* Update readmes
* Add variable to toggle DNS logging
* Extend DNS logging toggle to other net stages
2024-01-31 13:44:51 +01:00
da95434308
logging for default ingress rules in FAST ( #2030 )
...
* Add default ingress deny rule with logging to FAST net stages.
Fixes #2024
* Allow firewall factory to omit rules key
* Fix tests
* Fix fast tests
* fix fast tests
2024-01-30 16:53:01 +00:00
99228363b2
enforce trusted image projects constraint in stage 0 ( #2014 )
2024-01-26 10:14:44 +00:00
6d9b6403dd
add support for essential contacts to FAST ( #2010 )
2024-01-25 12:20:14 +01:00
c5416f3af1
Tighten up security of automation project (CSPR-related) ( #2009 )
...
* enforce compute/iam policies on the automation project
* tests
2024-01-24 18:40:36 +00:00
070584ae74
Checklist attribution bucket ( #2000 )
2024-01-23 11:32:14 +00:00
4b911a6047
update checklist parsing for top-level key ( #1997 )
2024-01-23 07:34:03 +01:00
a8c84357f4
Integrate checklist data in FAST ( #1969 )
...
* add locals for additive and authoritative org iam roles
* first shot at IAM and logging location
* tfdoc
* use locals for locations
* fix file parsing, resman stubs
* initial resman implementation
* remove unneeded code
* fix data file
* replace dumb yamldecode
* fix wrong type in organization additive bindings try
* simplify logging local
* Use check asserts for version and org id
* Checks on checklist for resman
* refactor checks, ignore checklist files on wrong org id
* stage 0 tests
* fix checklist checks
* stage 1 tests
---------
Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-01-18 05:45:29 +01:00
9d6e61428b
(WIP) Read-only service accounts for automation and CI/CD ( #1899 )
...
* add design doc for the new CI/CD sa
* describe the actual implementation
* specify which files will need to be changed
* Update 0-cicd-plan-sa.md
* Update 0-cicd-plan-sa.md
* Update 0-cicd-plan-sa.md
* Update 0-cicd-plan-sa.md
* Update 0-cicd-plan-sa.md
* Update 0-cicd-plan-sa.md
* Update 0-cicd-plan-sa.md
* Fix typo
* stage 0 read-only service accounts
* stage 0 IAM map
* linting
* cicd read-only service accounts
* tweak workflow templates
* roles and github workflow fixes
* tfdoc
* Ad-hoc custom role factory for FAST bootstrap
* use factory variable for custom roles data path
* custom roles factory in org/project modules
* tfdoc
* rename custom roles factory variable, fix gitlab template
* gitlab workflow fixes
* fix merge
* output plan results on failed assertion
* update stage 0 expected values
* data platform branch
* gke
* networking
* security
* project factory
* outputs
* workflow templates
* resman apply fixes
* tfdoc
* fix stage 1 test fixture
* fix gh workflow
* read-only resman sa roles
* fix test
* read-only resman sa roles
* read-only resman sa roles
* read-only resman sa roles
* read-only resman sa roles
* fix test variables
* rename wif principal attribute names
* rename wif principal variables
* multitenant stages
---------
Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-12-27 11:33:16 +00:00
c9a8d777ba
Add kernels.googleusercontent.com zone in dns response policy ( #1940 )
...
* Add kernels.googleusercontent.com zone in dns response policy
* update fast tests
2023-12-20 11:18:11 +01:00
6d89b88149
versions.tf maintenance + copyright notice bump ( #1782 )
...
* Bump copyright notice to 2023
* Delete versions.tf on blueprints
* Pin provider to major version 5
* Remove comment
* Fix lint
* fix bq-ml blueprint readme
---------
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-20 18:17:47 +02:00
e0d84fb10b
add sink for workspace logs ( #1780 )
2023-10-19 14:51:01 +00:00
e7e188818a
Add service usage consumer role to IaC SAs, refactor delegated grants in FAST ( #1773 )
...
* add serviceusage role to iac sas, refactor delegated grants
* fix test
* tfdoc
2023-10-18 12:18:31 +00:00
252127bde5
Billing account module ( #1743 )
...
* initial untested draft
* readme and tests
* folder module tfdoc
* remove redundant billing cost manager role in fast stage 0
* fix FAST test
2023-10-15 15:02:50 +00:00
4b15605711
Fix dnssec keys lookup ( #1728 )
...
* Fix dnssec keys lookup
* Fix DNS examples
* Fix FAST and blueprints resource counts
2023-10-03 21:37:21 +02:00
1dfa72cadf
Define and adopt standard IP ranges for FAST networking ( #1697 )
...
* Define and adopt standard IP ranges for FAST networking
This PR documents and adopts a consistent IP address plan for FAST
networking stages
Fixes #1644
* Fix documented aggregated ranges for FAST
* Fix tests
* Fix ip ranges in documentation
* Fix NVA stages README
2023-09-21 14:27:53 +00:00
77c1e69666
New phpIPAM serverless third parties solution in blueprints ( #1642 )
...
* Added new phpIPAM serverless third parties solution in blueprints
* added jit to iap.googleapis.com service in project module
* updated tests
2023-09-07 15:30:22 +02:00
12e78af055
Fix project factory blueprint and fast stage ( #1654 )
2023-09-07 12:48:39 +00:00
50a449965f
Fix: align stage-2-e-nva-bgp to the latest APIs
2023-08-23 13:34:11 +02:00
819894d2ba
IAM interface refactor ( #1595 )
...
* IAM modules refactor proposal
* policy
* subheading
* Update 20230816-iam-refactor.md
* log Julio's +1
* data-catalog-policy-tag
* dataproc
* dataproc
* folder
* folder
* folder
* folder
* project
* better filtering in test examples
* project
* folder
* folder
* organization
* fix variable descriptions
* kms
* net-vpc
* dataplex-datascan
* modules/iam-service-account
* modules/source-repository/
* blueprints/cloud-operations/vm-migration/
* blueprints/third-party-solutions/wordpress
* dataplex-datascan
* blueprints/cloud-operations/workload-identity-federation
* blueprints/data-solutions/cloudsql-multiregion/
* blueprints/data-solutions/composer-2
* Update 20230816-iam-refactor.md
* Update 20230816-iam-refactor.md
* capture discussion in architectural doc
* update variable names and refactor proposal
* project
* blueprints first round
* folder
* organization
* data-catalog-policy-tag
* re-enable folder inventory
* project module style fix
* dataproc
* source-repository
* source-repository tests
* dataplex-datascan
* dataplex-datascan tests
* net-vpc
* net-vpc test examples
* iam-service-account
* iam-service-account test examples
* kms
* boilerplate
* tfdoc
* fix module tests
* more blueprint fixes
* fix typo in data blueprints
* incomplete refactor of data platform foundations
* tfdoc
* data platform foundation
* refactor data platform foundation iam locals
* remove redundant example test
* shielded folder fix
* fix typo
* project factory
* project factory outputs
* tfdoc
* test workflow: less verbose tests, fix tf version
* re-enable -vv, shorter traceback, fix action version
* ignore github extension warning, re-enable action version
* fast bootstrap IAM, untested
* bootstrap stage IAM fixes
* stage 0 tests
* fast stage 1
* tenant stage 1
* minor changes to fast stage 0 and 1
* fast security stage
* fast mt stage 0
* fast mt stage 0
* fast pf
2023-08-20 09:44:20 +02:00
79373721df
Remove firewall policy management from resource management modules ( #1581 )
...
* rename firewall policy module, fix outputs
* add TOC to firewall policy module
* don't depend policy on parent id
* remove firewall policy from resource management modules
* remove factory conditionals
* fast net a and b
* fast stages
* fast tfdoc
* fast tfdoc
* remove unused test
* fix shielded folder blueprint
* fix shielded folder blueprint
2023-08-09 11:23:07 +00:00
cacb0c02e2
Refactoring of dns module
2023-07-19 12:57:44 +02:00
623c886e95
Peering dashboard ( #1492 )
...
* Adding dashboard to monitor VPC and VPC peering group quotas
* Adding 1 ressource to the tests (dashboard)
* Adding dashboard and tests for other networking architecture
* Update test
2023-07-05 18:25:31 +02:00
0fe3f165ed
Add VPN monitoring alerts to 2-networking and VPN usage chart
...
The Fast stage 2-networking-* currently adds a monitoring dashboard
for VPN metrics. This change adds an additional chart to monitor the
usage of the VPN bandwidth.
This change also adds the following monitoring alerts:
* VPN tunnel established
*
[VPN bandwidth](https://cloud.google.com/network-connectivity/docs/vpn/how-to/viewing-logs-metrics#define-bandwidth-alerts )
To configure the alerts, there is a new `alert_config` variable with
defined default values.
The alerts are created in the stage `b` by default. In the stages a,
c, d, and e, the alerts are created if the user creates the On-prem
VPN.
To disable the creation of alerts, add the following to
`terraform.tfvars`:
```
alert_config = {
vpn_tunnel_established = null
vpn_tunnel_bandwidth = null
}
```
2023-06-06 13:49:21 +01:00
efb0ebe689
Switch FAST networking stages to network policies for Google domains ( #1352 )
...
* peering stage implementation
* vpn stage implementation
* tfdoc
* tests
* add most supported google domains
* align all net stages
* add support for factory to DNS response policy module
* use dns policy factory in network stages
* boilerplate
2023-05-04 07:38:40 +02:00
c819305a42
Migrate apigee tests
2023-04-21 17:51:19 +02:00
a9cba47ce8
Add FAST stage 2-networking-e-nva-bgp (NVA+NCC)
...
Co-authored-by: Luca Prete <lucaprete@google.com>
Co-authored-by: Simone Bruzzechesse <bruzzechesse@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>
2023-04-04 20:41:04 +02:00
3d41d01efc
FAST plugin system ( #1266 )
...
* plugin folder, gitignore, serverless connector example
* add support to fast plugin variables and outputs to tfdoc
* rename folder, READMEs
* add variable description
* show diffs
* check documentation, use multiple files
* debug check doc
* try a different glob
* debug tfdoc names
* more debug
* and even more debug
* fix gitignore
* fix links
* support extra files in tests
* fix fixture, switch stage 2 peering to new tests
* tfdoc
* Allow globs in extra files
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2023-03-24 12:28:32 +00:00
5fb17cb3ac
Widen scope for prod project factory SA to dev ( #1263 )
...
* restrict storage role on outputs bucket for stage SAs
* grant prod project factory SA authority over prod and dev org policies
* network stages delegated grants on dev to prod pf SA
* security grants to prod pf SA on dev
* tfdoc
* tests
2023-03-17 16:24:55 +00:00
be06554bba
Simplify VPN implementation in FAST networking stages ( #1228 )
...
* peering stage
* fix link, toc
* vpn stage
* fix link
* nva stage
* fix examples and test
* separate envs stage
* tfdoc
2023-03-09 17:57:44 +01:00
e33caf0059
Fix tests
2023-03-07 17:52:00 +01:00
a5e905cb80
Update remaining org policies
2023-02-21 15:49:16 +01:00
8708f490ce
Allow configuring regions from tfvars in FAST networking stages ( #1137 )
...
* configurable regions
* vpn, tests
* tfdoc
* separate envs
* nva
* test resources
* add new custom role for tenant network service accounts
* allow setting firewall policy name in networking stages
* fix stage links script
* set custom role to tenant networking service account
* rename tenant stage 1 provider files
* remove extra file
* fix peering and vpn
* tfdoc
* fix variable order
* tests
2023-02-08 09:59:43 +01:00
5453c585e0
FAST multitenant bootstrap and resource management, rename org-level FAST stages ( #1052 )
...
* rename stages
* remove support for external org billing, rename output files
* resman: make groups optional, align on new billing account variable
* bootstrap: multitenant outputs
* tenant bootstrap stage, untested
* fix folder name
* fix stage 0 output names
* optional creation for tag keys in organization module
* single tenant bootstrap minus tag
* rename output files, add tenant tag key
* fix organization module tag values output
* test skipping creation for tags in organization module
* single tenant bootstrap plan working
* multitenant bootstrap
* tfdoc
* fix check links error messages
* fix links
* tfdoc
* fix links
* rename fast tests, fix bootstrap tests
* multitenant stages have their own folder, simplify stage numbering
* stage renumbering
* wip
* rename tests
* exclude fast providers in fixture
* stage 0 tests
* stage 1 tests
* network stages tests
* stage tests
* tfdoc
* fix links
* tfdoc
* multitenant tests
* remove local files
* stage links command
* fix links script, TODO
* wip
* wip single tenant bootstrap
* working tenant bootstrap
* update gitignore
* remove local files
* tfdoc
* remove local files
* allow tests for tenant bootstrap stage
* tenant bootstrap proxies stage 1 tfvars
* stage 2 and 3 service accounts and IAM in tenant bootstrap
* wip
* wip
* wip
* drop multitenant bootstrap
* tfdoc
* add missing stage 2 SAs, fix org-level IAM condition
* wip
* wip
* optional tag value creation in organization module
* stage 1 working
* linting
* linting
* READMEs
* wip
* Make stage-links script work in old macos bash
* stage links command help
* fix output file names
* diagrams
* fix svg
* stage 0 skeleton and diagram
* test svg
* test svg
* test diagram
* diagram
* readme
* fix stage links script
* stage 0 readme
* README changes
* stage readmes
* fix outputs order
* fix link
* fix tests
* stage 1 test
* skip stage example
* boilerplate
* fix tftest skip
* default bootstrap stage log sinks to log buckets
* add logging to tenant bootstrap
* move iam variables out of tenant config
* fix cicd, reintroduce missing variable
* use optional in stage 1 cicd variable
* rename extras stage
* rename and move identity providers local, use optional for cicd variable
* tfdoc
* add support for wif pool and providers, ci/cd
* tfdoc
* fix links
* better handling of modules repository
* add missing role on logging project
* fix cicd pools in locals, test cicd
* fix workflow extension
* fix module source replacement
* allow tenant bootstrap cicd sa to impersonate resman sa
* tenant workflow templates fix for no providers file
* fix output files, push github workflow template to new repository
* remove try from outpout files
* align stage 1 cicd internals to stage 0
* tfdoc
* tests
* fix tests
* tests
* improve variable descriptions
* use optional in fast features
* actually create tenant log sinks, and allow the resman sa to do it
* test
* tests
* aaaand tests again
* fast features tenant override
* fast features tenant override
* fix wording
* add missing comment
* configure pf service accounts
* add missing comment
* tfdoc
* tests
* IAM docs
* update copyright
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2023-02-04 15:00:45 +01:00
edd3a82453
Include cloudbuild API in project module ( #1116 )
...
* Include cloudbuild API in project module
* Increase number of resources
2023-01-27 21:38:01 +01:00
09ad53000e
Remove recursive_e2e_plan_runner
2022-12-18 14:00:20 +01:00
be0e807435
Bring back `tests` key in test yaml spec
2022-12-06 00:06:29 +01:00
589f7a5c2f
Simplify yaml test spec
2022-12-06 00:06:29 +01:00
34f01762c3
Simplify fast bootstrap test
2022-12-06 00:06:29 +01:00
2af4a826fa
Initial FAST bootstrap fixture
2022-12-06 00:06:29 +01:00
b4d3aa2055
Migrate organizations tests
2022-12-06 00:06:29 +01:00
Ludovico Magnocavallo
simonebruzzechesse
Wiktor Niesiobędzki
Julio Castillo
Simone Ruffilli
Luca Prete
Miren Esnaola
Aurélien Legrand
Ana Fernandez del Alamo
Ayman Farhat