Commit Graph

683 Commits

Author SHA1 Message Date
Simone Ruffilli b015380028 Fix allow-nat-ranges priority 2023-10-25 14:05:15 +02:00
Simone Ruffilli a3290f2204
FAST: Add access transparency logs to the default sinks (#1810)
* Adds access transparency logs to the default sinks
2023-10-24 20:09:00 +00:00
Simone Ruffilli 1836c68990
Hierarchical rules update (#1809) 2023-10-24 19:46:04 +00:00
Simone Ruffilli 1378214af5
FAST: removed references to kms_defaults (#1811) 2023-10-24 21:18:08 +02:00
Ludovico Magnocavallo 4647b07665
less verbose project factory stage outputs (#1802) 2023-10-24 09:03:35 +02:00
Ludovico Magnocavallo a93f08e833
improve usage of optionals in FAST stage 2 VPN variables (#1797) 2023-10-23 15:23:30 +02:00
Ludovico Magnocavallo 4690bf206a
Update README.md 2023-10-21 18:59:17 +02:00
Simone Ruffilli 3e16c6a959
FAST: adds support to uploading a wif provider pubkey (#1788) 2023-10-21 16:52:19 +00:00
Simone Ruffilli 6d89b88149
versions.tf maintenance + copyright notice bump (#1782)
* Bump copyright notice to 2023

* Delete versions.tf on blueprints

* Pin provider to major version 5

* Remove comment

* Fix lint

* fix bq-ml blueprint readme

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-20 18:17:47 +02:00
Ludovico Magnocavallo e0d84fb10b
add sink for workspace logs (#1780) 2023-10-19 14:51:01 +00:00
Ludovico Magnocavallo 77a4696aa6
Add gcp org policy constraints file to bootstrap stage (#1775)
* add gcp org policy constraints file to bootstrap

* make the org policy factories more resilient
2023-10-18 18:21:16 +00:00
Ludovico Magnocavallo b0c552cff5
Update IAM.md 2023-10-18 19:59:07 +02:00
Ludovico Magnocavallo e34cb20dc6
Update IAM.md 2023-10-18 19:58:18 +02:00
Ludovico Magnocavallo f4c8786677
Update IAM.md 2023-10-18 19:57:46 +02:00
Ludovico Magnocavallo 94ae8634fc
Update IAM.md 2023-10-18 19:57:03 +02:00
Ludovico Magnocavallo e41cc4ec36
Update IAM.md 2023-10-18 19:56:40 +02:00
Ludovico Magnocavallo 6252198961
Update IAM.md 2023-10-18 19:56:20 +02:00
Ludovico Magnocavallo e7e188818a
Add service usage consumer role to IaC SAs, refactor delegated grants in FAST (#1773)
* add serviceusage role to iac sas, refactor delegated grants

* fix test

* tfdoc
2023-10-18 12:18:31 +00:00
Luca Prete 6c48512f7e
[#1764] net-lb-int: add support for dual stack and multiple forwarding rules 2023-10-17 09:30:34 +00:00
Ludovico Magnocavallo 6fd58e33c9
Add support for psa peered domains to fast stages (#1760)
* add support for psa peered domains

* tfdoc
2023-10-16 06:57:18 +00:00
Ludovico Magnocavallo 28e19ab180
Minor edits to FAST network stage READMEs (#1759)
* PSA section

* VPC description, ranges
2023-10-15 16:14:48 +00:00
Ludovico Magnocavallo 252127bde5
Billing account module (#1743)
* initial untested draft

* readme and tests

* folder module tfdoc

* remove redundant billing cost manager role in fast stage 0

* fix FAST test
2023-10-15 15:02:50 +00:00
Ludovico Magnocavallo 2afdc5a8e1
Update COMPANION.md 2023-10-08 08:47:35 +02:00
Julio Castillo dfc5023e0b
Make deletion protection consistent across all modules (#1735)
* Expose deletion_protection in GKE modules

* Make deletion protection consistent across all modules

* Add deletion_protection option to blueprints

* Fix blueprints tests

* Fix types

* Update READMEs

* Fix dp readme

* Fix cmek blueprint default deletion_protection

* Fix blueprints tests
2023-10-05 17:31:07 +02:00
Alejandro Leal 81c6959617 Update to lint.sh and wording to some tf
fast/stages-multitenant/0-bootstrap-tenant/identity-providers.tf
fast/stages/0-bootstrap/identity-providers.tf
tools/lint.sh
2023-10-05 00:17:20 -04:00
Alejandro Leal 8c4cd8548c
Update README.md 2023-10-04 14:04:04 -04:00
lcaggio 6889f02954
Fix data platform roles (#1725)
* Fix Data Platform roles

* Fix README

* Fix blueprint tests

* Update cleanup dp steps

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-10-04 07:31:40 +02:00
Ludovico Magnocavallo 789328ff5a
Bump provider versions to v5.0.0 (#1724)
* bump provider versions to 5.0.0

* fix cloud run, logging and vpc-sc

* Fix secret manager

* fix gke nodepool

* fix gke multitenant stage and blueprint

* Moving alloydb module to experimental.

* Add project to bare resources in examples

* tfdoc

* fix svpc blueprint test

* Revert "fix svpc blueprint test"

This reverts commit 14f02659098070136e64ead600580dd52c23c339.

* Fix GKE peering project

* Disable tests in alloydb module

* Bring back secret ids in secret manager tests

* Remove duplicate key

* last push

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-03 12:15:36 +00:00
Ludovico Magnocavallo 2ee8f57769
FAST: add example of custom org policy condition to bootstrap README (#1718)
* add oslogin constraint condition example to bootstrap

* add oslogin constraint condition example to bootstrap

* add oslogin constraint condition example to bootstrap
2023-09-30 10:22:56 +02:00
Julio Castillo 9082bbcc48
Fix indentation in FAST hierarchical firewall rules (#1715)
Fixes #1712
2023-09-29 13:37:41 +00:00
lcaggio e4a25d7c99
Fix tenant folder tag (#1711) 2023-09-28 23:48:14 +02:00
Julio Castillo b2d27b5f12 Update bootstrap and destroy roles 2023-09-28 11:41:56 +02:00
Julio Castillo 30772d921c
Update README.md 2023-09-28 10:59:54 +02:00
Ludovico Magnocavallo fcc1aa87c4
fix latest commit 2023-09-28 10:58:31 +02:00
Ludovico Magnocavallo 76b4605326
add missing roles for initial bootstrap 2023-09-28 10:57:46 +02:00
Ludovico Magnocavallo fb08e1b01e
Only apply org policies when bootstrap user is not set (#1707)
* only apply org policies when bootstrap user is not set

* Add Org Policy Admin to bootstrap roles

* Fix cleanup doc

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-09-27 23:24:40 +02:00
giterinhub 22186ff884
Update README.md
Changed aopproach to approach
2023-09-27 13:59:19 +02:00
Julio Castillo 1dfa72cadf
Define and adopt standard IP ranges for FAST networking (#1697)
* Define and adopt standard IP ranges for FAST networking

This PR documents and adopts a consistent IP address plan for FAST
networking stages

Fixes #1644

* Fix documented aggregated ranges for FAST

* Fix tests

* Fix ip ranges in documentation

* Fix NVA stages README
2023-09-21 14:27:53 +00:00
Ludovico Magnocavallo f628cdbc06
FAST: move organization policies to stage 0 (#1698)
* design doc

* Update 0-org-policies.md

* moved org policies to stage 0, wip

* stage0

* stage 0

* export tag keys and values from stage 0

* rename factory variable

* change org policy outputs

* stage 1

* Update 0-org-policies.md

* make org policy variable not nullable, README changes

* use optionals for tag names

* better factory variable name

* README changes

* ADR
2023-09-21 14:03:21 +00:00
Ludovico Magnocavallo 82fcd5a7d3
rename FAST globals output file (#1695) 2023-09-20 10:36:06 +02:00
Julio Castillo ad14a7d415 Update READMEs 2023-09-17 00:21:36 +02:00
Julio Castillo 960e015b42 Fix FAST tests 2023-09-17 00:21:36 +02:00
Julio Castillo 121598dbea Move FAST security delegated admins to iam_bindings_additive 2023-09-17 00:21:36 +02:00
Julio Castillo 9c878dc9cf Fix tests for new KMS IAM interface 2023-09-17 00:21:36 +02:00
Ludovico Magnocavallo d3d77d17fb
fix psa routing variable in FAST net stages (#1685) 2023-09-16 10:31:02 +02:00
Oliver Frolovs 6eb862a775
GKE cluster modules: add optional kube state metrics (#1682)
* `gke-cluster-standard`: add optional kube state metrics
* `gke-cluster-autopilot`: add optional kube state metrics

* FAST: add kube state metrics support for GKE
* blueprints/gke: add kube state metrics support

* Bump up the provider version to `v4.82.0`
2023-09-15 12:18:45 +01:00
Julio Castillo f3be29cbc9 Fix tests 2023-09-15 00:27:55 +02:00
Oliver Frolovs b3dc91b5cd
Upgrades to `monitoring_config` in `gke-cluster-*`, docs update, and cosmetics fixes to GKE cluster modules (#1680)
* gke-cluster-standard: upgrade `monitoring_config` to use object style. Add tests.
* gke-cluster-standard: update docs

* gke-cluster-autopilot: move gateway_api_config block (cosmetic change)
* gke-cluster-autopilot: update docs and fix typos

* Update blueprints due to `monitoring_config` changes in `gke-cluster-standard`.
* Update FAST due to `monitoring_config` changes in `gke-cluster-standard`.
* Update docs for affected blueprints and FAST stages
2023-09-14 23:25:57 +01:00
Julio Castillo 8d7772761c Fix FAST readmes 2023-09-14 13:10:16 +02:00
Julio Castillo c1be435b09 Fix range names definition of GKE clusters
Fixes #1677
2023-09-14 12:51:43 +02:00
Julio Castillo 949e98d375 Increase size of pod range for default GKE subnets in FAST
Related to the issues reported in #1644
2023-09-11 10:28:42 +02:00
Ludovico Magnocavallo 3915a016c9
Align pf stage sample data to new format (#1664)
* align pf stage sample data to new format

* boilerplate
2023-09-09 10:04:19 +02:00
Luca Prete fcefadbd8e
[#1661] Make FAST stage 1 resman tf destroy more reliable
Co-authored-by: Luca Prete <lucaprete@google.com>
2023-09-08 10:09:31 +00:00
Ludovico Magnocavallo e14789ecb0
link project factory documentation from FAST stage (#1659) 2023-09-08 07:14:16 +00:00
Ludovico Magnocavallo ec3b705f53
Change type of `iam_bindings` variable to allow multiple conditional bindings (#1658)
* modules

* fast

* dns readme
2023-09-08 08:56:31 +02:00
Luca Prete 12e78af055
Fix project factory blueprint and fast stage (#1654) 2023-09-07 12:48:39 +00:00
Oliver Frolovs 988fd2ee05
gke-cluster-standard: change logging configuration (#1638)
* Update logging configuration of this module to use object interface in harmony with `gke-cluster-autopilot` module.
* Update blueprints that use this module.
* Add "WORKLOADS" log source to logging configuration of the blueprints where the README files say so.
* Update FAST stage 3 because it uses this module.
2023-08-31 12:49:15 +01:00
Julio Castillo 804e7c961e Silence FAST tests warnings
- Fix pytest PytestUnraisableExceptionWarning
- Remove incorrect print
- Use tfvars for some examples in READMEs
2023-08-28 18:40:41 +02:00
Luca Prete c63884d52e
Remove unused ASN numbers in CloudNAT to avoid FAST provider errors 2023-08-28 15:32:30 +00:00
Julio Castillo b88e4c6f6e Fix syntax error in FAST nva 2023-08-28 16:28:01 +02:00
Julio Castillo b701d55b1f Fix tests 2023-08-28 16:00:48 +02:00
Julio Castillo 5e9829373c Fix FAST hfw policies 2023-08-28 16:00:48 +02:00
Luca Prete 4c64c15871
Revert "Remove unused ASN numbers from CloudNAT to avoid provider errors" (#1626)
This reverts commit 311bed8e83.
2023-08-28 09:33:52 +02:00
Julio Castillo 1adfb9fb32 Fix role name for delegated grants in FAST bootstrap
Fixes issue behind #1621
2023-08-24 19:13:42 +02:00
Luca Prete 50a449965f
Fix: align stage-2-e-nva-bgp to the latest APIs 2023-08-23 13:34:11 +02:00
Luca Prete 8ca60881f1
Fix: use existing variable to optionally name fw policies (#1610) 2023-08-22 08:55:56 +02:00
Ludovico Magnocavallo 819894d2ba
IAM interface refactor (#1595)
* IAM modules refactor proposal

* policy

* subheading

* Update 20230816-iam-refactor.md

* log Julio's +1

* data-catalog-policy-tag

* dataproc

* dataproc

* folder

* folder

* folder

* folder

* project

* better filtering in test examples

* project

* folder

* folder

* organization

* fix variable descriptions

* kms

* net-vpc

* dataplex-datascan

* modules/iam-service-account

* modules/source-repository/

* blueprints/cloud-operations/vm-migration/

* blueprints/third-party-solutions/wordpress

* dataplex-datascan

* blueprints/cloud-operations/workload-identity-federation

* blueprints/data-solutions/cloudsql-multiregion/

* blueprints/data-solutions/composer-2

* Update 20230816-iam-refactor.md

* Update 20230816-iam-refactor.md

* capture discussion in architectural doc

* update variable names and refactor proposal

* project

* blueprints first round

* folder

* organization

* data-catalog-policy-tag

* re-enable folder inventory

* project module style fix

* dataproc

* source-repository

* source-repository tests

* dataplex-datascan

* dataplex-datascan tests

* net-vpc

* net-vpc test examples

* iam-service-account

* iam-service-account test examples

* kms

* boilerplate

* tfdoc

* fix module tests

* more blueprint fixes

* fix typo in data blueprints

* incomplete refactor of data platform foundations

* tfdoc

* data platform foundation

* refactor data platform foundation iam locals

* remove redundant example test

* shielded folder fix

* fix typo

* project factory

* project factory outputs

* tfdoc

* test workflow: less verbose tests, fix tf version

* re-enable -vv, shorter traceback, fix action version

* ignore github extension warning, re-enable action version

* fast bootstrap IAM, untested

* bootstrap stage IAM fixes

* stage 0 tests

* fast stage 1

* tenant stage 1

* minor changes to fast stage 0 and 1

* fast security stage

* fast mt stage 0

* fast mt stage 0

* fast pf
2023-08-20 09:44:20 +02:00
lcaggio 6eeba5e599
[Data Platform] Update README.md (#1601)
Fix hardcoded path in readme.
2023-08-18 18:27:43 +02:00
Alejandro Leal ea0de3adbb Fixing some typos 2023-08-18 05:51:00 +00:00
Stefan Moser dcb3c32761
fix null object exception in bootstrap output when using cloudsource repos (#1597) 2023-08-17 09:03:23 +00:00
Ludovico Magnocavallo 2423fd40c1
Fix FAST CI/CD for Gitlab (#1593)
* fix cicd (multitenant untested)

* tfdoc

* rename allowed_audiences to audiences, align multitenant
2023-08-15 12:59:31 +02:00
Ludovico Magnocavallo c5a77ebfe3
fix module path for teams cicd (#1583) 2023-08-09 21:41:56 +00:00
Matt 9600047a32
Enable team CI/CD impersonation (#1579) 2023-08-09 08:46:24 -04:00
Ludovico Magnocavallo 79373721df
Remove firewall policy management from resource management modules (#1581)
* rename firewall policy module, fix outputs

* add TOC to firewall policy module

* don't depend policy on parent id

* remove firewall policy from resource management modules

* remove factory conditionals

* fast net a and b

* fast stages

* fast tfdoc

* fast tfdoc

* remove unused test

* fix shielded folder blueprint

* fix shielded folder blueprint
2023-08-09 11:23:07 +00:00
Ludovico Magnocavallo 9c75aa469c
More module descriptions (#1572)
* bigquery dataset

* data catalog policy tag

* net-address

* fix data catalog callers

* bigquery dataset views

* fix data catalog callers

* logging bucket

* net vpn ha
2023-08-06 09:25:45 +00:00
Luca Prete 311bed8e83
Remove unused ASN numbers from CloudNAT to avoid provider errors 2023-08-04 08:02:11 +00:00
Luca Prete 47daeaafe1
Update FAST CI/CD workflows so it can work with ID_TOKEN and Gitlab 15+ 2023-08-03 16:09:45 +00:00
Ludovico Magnocavallo b524aa137c
Peering module refactor (#1547)
* refactor net-vpc-peering module

* hub and spoke peering blueprint

* fast stages

* boilerplate

* fast tfdoc

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-07-29 21:33:57 +02:00
Ludovico Magnocavallo c918cfc800
Update README.md 2023-07-27 13:40:26 +02:00
Ludovico Magnocavallo ea800fa475
fix stage links for GKE stage (#1514) 2023-07-20 10:48:45 +00:00
Miren Esnaola cacb0c02e2 Refactoring of dns module 2023-07-19 12:57:44 +02:00
Ludovico Magnocavallo aa1a79632b
Create 0-org-policies.md 2023-07-15 18:40:18 +02:00
Ludovico Magnocavallo 4ad55923b7
Update 0-bootstram-user-iam.md 2023-07-13 19:33:01 +02:00
Ludovico Magnocavallo fbbe668015
Document architectural decisions (#1506)
* add architectural decisions log and first decision

* add header

* typo
2023-07-13 16:15:32 +02:00
Natalia Strelkova e00d3bcba4
README: audit logs on org level go to a logging bucket, not bigquery 2023-07-10 16:42:01 +02:00
Ludovico Magnocavallo 154df17951
FAST: initial implementation of lightweight tenants (#1470)
* initial import

* fixes

* fixes

* fixes

* red SA roles

* red SA roles

* org-level custom roles var, tenants IAM config

* tfdoc

* allow core SA to write output files to tenant bucket

* README

* implement comments on PR

* show tenant org example

* update example
2023-07-07 08:40:37 +02:00
Aurélien Legrand 623c886e95
Peering dashboard (#1492)
* Adding dashboard to monitor VPC and VPC peering group quotas

* Adding 1 ressource to the tests (dashboard)

* Adding dashboard and tests for other networking architecture

* Update test
2023-07-05 18:25:31 +02:00
Julio Castillo d49a5c0fbb Fix primary gke/dp ranges in FAST subnets 2023-06-30 19:28:21 +02:00
Roberto Jung Drebes 772cf813fc
FAST: short_name_is_prefix for multi-tenant (#1478)
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-06-30 09:49:25 +02:00
Alejandro Leal 43b3490ef1 Updating a few files for typos
fast/stages/3-data-platform/dev/README.md
fast/stages/3-data-platform/dev/outputs.tf
CHANGELOG.md
blueprints/data-solutions/data-platform-minimal/README.md
blueprints/data-solutions/data-platform-minimal/outputs.tf
blueprints/data-solutions/data-platform-foundations/README.md
2023-06-29 21:47:17 -04:00
Arvind Ganesh d3e4864b57 Making the changes as suggested in https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/1477#issuecomment-1612846907 2023-06-29 12:24:29 -04:00
Arvind Ganesh 0b19a16593 Changing the IP ranges in all networking stages 2023-06-28 14:45:33 -04:00
Arvind Ganesh f75bc321b9 Changing the IP range of pods from 100.64.48.0/20 to 100.65.16.0/20 as there is an overlap in 100.64.0.0/16 range with dev-gke-nodes-ew1.yaml 2023-06-28 14:15:35 -04:00
Julio Castillo d6aea3ff5f Remove unneeded file from resman stage 2023-06-27 09:54:46 +02:00
Ludovico Magnocavallo 638841c8d1
Rename network load balancer modules (#1466)
* update LB modules to new names

* update LB modules names

* update test paths
2023-06-26 07:50:10 +00:00
Albert Lloveras 7cacc46b4b
fixup(project-factory): Use the correct KMS Service Agents attribute … (#1446)
* fixup(project-factory): Use the correct KMS Service Agents attribute name

* Add new KMS bindings to tests

* Update test resource counts

* Update README.md resource count
2023-06-19 23:53:08 +00:00
Keith Harvey a68a3b55cb
Bump TF version in all workflow templates to coincide with module requirements (#1445)
* Resman - bump GH TF version to coincide with module requirements (#1)

Bootstrap was bumped in #1414

* Bump TF version in all workflow files

* bump TF version in missed workflow file
2023-06-16 07:39:28 +00:00
Ludovico Magnocavallo 815728aca6
fix repo names check (#1443) 2023-06-15 16:08:57 +00:00
Julio Castillo e900e9c951 Make internal/external addresses optional in compute-vm
Fixes 1431
2023-06-08 14:00:10 +02:00
Wiktor Niesiobędzki 6b4bca10bd Use RFC6598 addresses for pods and subnets
10.128.0.0/9 is public network.

Closes: #1424
2023-06-08 07:56:31 +02:00