Commit Graph

67 Commits

Author SHA1 Message Date
Ludovico Magnocavallo e3809e6735
Add main project factory service account (#2353)
* add main project factory service account

* add main project factory service account
2024-06-10 12:23:30 +02:00
Ludovico Magnocavallo b13b6032d3
Remove support for source repositories from FAST CI/CD (#2352)
* stage 0

* stage 1

* stage 1 mt

* remove unused locals from resman

* remove unused locals from resman

* tfdoc
2024-06-10 09:02:55 +00:00
Ludo 26357d9b0c
update resman IAM 2024-05-31 15:31:51 +02:00
Ludovico Magnocavallo 5cae3fd266
add support for iam to builtin tags (#2334) 2024-05-31 12:57:13 +00:00
Ludovico Magnocavallo d736a06c0b
Fix resman top-level folders variable types (#2333)
* fix resman top-level folders variable types

* pass logging settings to underlying module

* fix logging settings
2024-05-31 14:45:31 +02:00
Simone Ruffilli 11050c46cf
FAST MT: Readme updates and more prefix validation (#2305)
This change documents the process of deploying FAST on a tenant-factory bootstrapped tenant.

It also fixes changes the validation logic for prefix as follows:

- 0-bootstrap: 9 chars or less
- 1-resman/1-tenant-factory: 9 chars or less if ran at org-level, else 11
- else 11

It also uniforms across all stages the variables.tf and variables-fast.tf breakdown.
2024-05-24 12:01:55 +02:00
Peter Brumblay a04d63e991
Avoid unnecessary terraform plans for closed (unmerged) PRs (#2294)
* Avoid unnecessary terraform plans for closed (unmerged) PRs

* fix: also update templates copied to avoid Windows symlink problems.

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2024-05-21 13:03:07 +00:00
Ludovico Magnocavallo 8014121be1
switch stages 0-1s to excalidraw diagrams (#2286) 2024-05-17 09:10:13 +00:00
Ludovico Magnocavallo 7a5dd4e6db
FAST: add top-level folders and restructure teams/tenants in resman (#2254)
* remove teams and tenants from resman

* move fast features to stage 1, fix test inventories

* folders

* fix factory, add top level folder resources to outputs

* tfdoc

* stage 0 log sink defs

* tfdoc

* enable toc in resman readme

* simple tenants

* fast compatibility automation and logging

* testing fast-compatible tenants

* testing fast-compatible tenants

* tfdoc

* remove mt stages

* remove tests, fix links

* disable tflint

* fast tests

* make organization conditional in resman

* check names tool

* export real prefix to tfvars, prevent destroy errors

* prefix validation

* fix billing account export format

* tfdoc

* root node folder

* resman changes

* tenant resman roles

* first apply of tenant resman

* tenant log sinks in stage 1

* fix test vars

* tfdoc

* tenant vpc-sc access policy

* fix tests expected values

* tenant CI/CD

* identity providers

* wif

* tfdoc

* add comments to identity locals

* full-feature tenant resman apply

* tenant billing IAM

* stage test

* fix CI/CD comments

* tenant net stage verified

* tenant sec stage verified

* fix test

* README work

* tfdoc

* README

* README rewording

* README rewording

* tfdoc

* FAST excalidraw

* review comments

* diagram review changes

* add iam log sink for tenants

* remove redundant try from security stage

* Implement tflint-fast in Python driven by tftest.yaml files

* tflint

* test ci changes

* revert linting changes

* disable tflint for fast

* Create junit-style report for FAST tflint

* Remove junit-reporter

* YAPF tflint-fast.py

* Output tflint FAST to job summary

* Step summary

* Disable step_summary as output is not useful

* ignore tflint warning

* re-enable tflint on FAST

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-05-15 09:17:13 +00:00
Julio Castillo 94c32c1d71
Misc FAST fixes (#2253)
* Misc FAST fixes

* Fix readme

* Fix FAST nva bgp tests
2024-05-02 06:56:26 +00:00
Luca Prete dd213ea9a9
Fix permissions for branch network dev - read sa (#2233)
Co-authored-by: Luca Prete <lucaprete@google.com>
2024-04-23 15:19:38 +02:00
Julio Castillo 3af7e257d2
Add tflint to pipelines (#2220)
* Fix terraform_deprecated_index

https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_index.md

* Fix terraform_deprecated_interpolation

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_interpolation.md

* Fix more indexing

* Remove unused variable

* Enable TFLint for modules

* Add tflint config file

* Fix chdir

* Lint modules

* TFLint fixes

* TFLint

* Fixes binauthz README

* Fixes DNS response policy tests. Restores MIG outputs.

* Fixes other DNS response policy tests.

* Update tests for fast 2-e

* Moar fixed tests

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
2024-04-17 10:23:48 +02:00
Julio Castillo b4b926a56e
Remove unused variables/locals from FAST (#2219)
* Remove unused variables/locals from FAST

* Fix readmes
2024-04-16 14:14:24 +00:00
Timothy Jabez 215c5cec26
Update the description and README for the tags variable (#2206)
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2024-04-10 15:08:59 +02:00
Ludovico Magnocavallo 8511170412
FAST security stage refactor (#2203)
* first working change, missing docs and tests

* fixes

* tests

* boilerplate

* tfdoc
2024-04-07 20:14:39 -07:00
Julio Castillo a9ac0f40cd
Add variable to resman to control top-level folder IAM (#2196) 2024-04-04 10:26:35 +02:00
Ludovico Magnocavallo 33ffe2daa6
Allow r/o project factory SAs access to folder-level IAM (#2195)
* allow project factory SAs read access to folder-level IAM

* tfdoc
2024-04-03 21:51:46 +02:00
Elia b80132a618
FAST GCVE stage (#2191)
* GCVE stage prerequisites

* fix gcve prereq

* gcve stage first deploy

* Updated readme filex

* docs updated

* some fixes after testing

* updated readme files

* elia fix

* gcve net admin custom role

* gcve net admin custom role

* elia fix

* ven peering deploy

* elia fix

* added blueprint and stage tests

* Edits to Readme files

* typo in outputs

* clean-up

* gcve stage tests fix

* readme fix

* fix sorting

* fix copyrights and readme file

* fix test

* fix copyright

* fixed gcve feature flag cond.

* removed validation

* fixed typo

* fixed typo

* fixed gcve tests

* fixed typo

* fixed typo

* fixed sorting

* fixed sorting

---------

Co-authored-by: Konrad Schieban <kschieban@google.com>
2024-04-03 17:25:12 +02:00
Ludovico Magnocavallo f25701d79c
update org viewer custom role and cicd templates (#2178) 2024-03-27 09:06:25 +01:00
Ludovico Magnocavallo ecf5400fe3
update FAST state IAM files (#2136) 2024-03-07 00:08:09 +01:00
Ludovico Magnocavallo eb23bb62d2
Support domainless orgs in FAST (#2086)
* bootstrap

* align org policies to domainless enforced ones

* fix #2073

* fix tests

* fix team admin attribute in resman stage
2024-02-19 08:29:37 +00:00
Ludovico Magnocavallo 71a64487d5
Extend FAST to support different principal types (#2064)
* add doc draft

* typos

* typo

* typo

* typos

* rewording

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* move iam variables to a separate file

* move billing-account module to iam_principals

* move data-catalog-policy-tag module to iam_principals

* move dataplex-datascan module to iam_principals

* move dataproc module to iam_principals

* move folder module to iam_principals

* copyright

* move organization module to iam_principals

* move project module to iam_principals

* move source-repository module to iam_principals

* update blueprints for iam_principals interface

* FAST bootstrap

* module READMEs fixes

* FAST bootstrap

* FAST networking stages

* FAST security stage

* FAST gke stage

* FAST multitenant bootstrap stage

* FAST multitenant resman stage

* tfdoc

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* fix module test

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Rename iam_principals to iam_by_principals

* Update IAM template to include iam_by_principals

* Update Resman README

* Fix ADR link format

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2024-02-12 14:35:30 +01:00
Ludovico Magnocavallo 6d9b6403dd
add support for essential contacts to FAST (#2010) 2024-01-25 12:20:14 +01:00
Ludovico Magnocavallo 4b911a6047
update checklist parsing for top-level key (#1997) 2024-01-23 07:34:03 +01:00
Ludovico Magnocavallo a8c84357f4
Integrate checklist data in FAST (#1969)
* add locals for additive and authoritative org iam roles

* first shot at IAM and logging location

* tfdoc

* use locals for locations

* fix file parsing, resman stubs

* initial resman implementation

* remove unneeded code

* fix data file

* replace dumb yamldecode

* fix wrong type in organization additive bindings try

* simplify logging local

* Use check asserts for version and org id

* Checks on checklist for resman

* refactor checks, ignore checklist files on wrong org id

* stage 0 tests

* fix checklist checks

* stage 1 tests

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-01-18 05:45:29 +01:00
Ludovico Magnocavallo 9d6e61428b
(WIP) Read-only service accounts for automation and CI/CD (#1899)
* add design doc for the new CI/CD sa

* describe the actual implementation

* specify which files will need to be changed

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Fix typo

* stage 0 read-only service accounts

* stage 0 IAM map

* linting

* cicd read-only service accounts

* tweak workflow templates

* roles and github workflow fixes

* tfdoc

* Ad-hoc custom role factory for FAST bootstrap

* use factory variable for custom roles data path

* custom roles factory in org/project modules

* tfdoc

* rename custom roles factory variable, fix gitlab template

* gitlab workflow fixes

* fix merge

* output plan results on failed assertion

* update stage 0 expected values

* data platform branch

* gke

* networking

* security

* project factory

* outputs

* workflow templates

* resman apply fixes

* tfdoc

* fix stage 1 test fixture

* fix gh workflow

* read-only resman sa roles

* fix test

* read-only resman sa roles

* read-only resman sa roles

* read-only resman sa roles

* read-only resman sa roles

* fix test variables

* rename wif principal attribute names

* rename wif principal variables

* multitenant stages

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-12-27 11:33:16 +00:00
Ludovico Magnocavallo a2263da1f3
fix GitHub CI/CD provider (#1945) 2023-12-21 17:10:50 +00:00
Julio Castillo b6e0557bbb
Simplify organization tags.tf locals (#1932)
* Simplify organization tags.tf locals

* Fix boilerplate

* Override github provider version for tests
2023-12-18 16:09:22 +00:00
ibrahimparvez2 21297f28a6
Patch Github actions ci google-github-actions/auth@v0 --> v2 (#1900)
* MInor patch auth

* Minor update auth
2023-12-04 12:16:02 +00:00
Simone Ruffilli 6d89b88149
versions.tf maintenance + copyright notice bump (#1782)
* Bump copyright notice to 2023

* Delete versions.tf on blueprints

* Pin provider to major version 5

* Remove comment

* Fix lint

* fix bq-ml blueprint readme

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-20 18:17:47 +02:00
Ludovico Magnocavallo b0c552cff5
Update IAM.md 2023-10-18 19:59:07 +02:00
Ludovico Magnocavallo e34cb20dc6
Update IAM.md 2023-10-18 19:58:18 +02:00
Ludovico Magnocavallo f4c8786677
Update IAM.md 2023-10-18 19:57:46 +02:00
Ludovico Magnocavallo e7e188818a
Add service usage consumer role to IaC SAs, refactor delegated grants in FAST (#1773)
* add serviceusage role to iac sas, refactor delegated grants

* fix test

* tfdoc
2023-10-18 12:18:31 +00:00
lcaggio e4a25d7c99
Fix tenant folder tag (#1711) 2023-09-28 23:48:14 +02:00
Ludovico Magnocavallo f628cdbc06
FAST: move organization policies to stage 0 (#1698)
* design doc

* Update 0-org-policies.md

* moved org policies to stage 0, wip

* stage0

* stage 0

* export tag keys and values from stage 0

* rename factory variable

* change org policy outputs

* stage 1

* Update 0-org-policies.md

* make org policy variable not nullable, README changes

* use optionals for tag names

* better factory variable name

* README changes

* ADR
2023-09-21 14:03:21 +00:00
Ludovico Magnocavallo 82fcd5a7d3
rename FAST globals output file (#1695) 2023-09-20 10:36:06 +02:00
Luca Prete fcefadbd8e
[#1661] Make FAST stage 1 resman tf destroy more reliable
Co-authored-by: Luca Prete <lucaprete@google.com>
2023-09-08 10:09:31 +00:00
Julio Castillo 804e7c961e Silence FAST tests warnings
- Fix pytest PytestUnraisableExceptionWarning
- Remove incorrect print
- Use tfvars for some examples in READMEs
2023-08-28 18:40:41 +02:00
Ludovico Magnocavallo 819894d2ba
IAM interface refactor (#1595)
* IAM modules refactor proposal

* policy

* subheading

* Update 20230816-iam-refactor.md

* log Julio's +1

* data-catalog-policy-tag

* dataproc

* dataproc

* folder

* folder

* folder

* folder

* project

* better filtering in test examples

* project

* folder

* folder

* organization

* fix variable descriptions

* kms

* net-vpc

* dataplex-datascan

* modules/iam-service-account

* modules/source-repository/

* blueprints/cloud-operations/vm-migration/

* blueprints/third-party-solutions/wordpress

* dataplex-datascan

* blueprints/cloud-operations/workload-identity-federation

* blueprints/data-solutions/cloudsql-multiregion/

* blueprints/data-solutions/composer-2

* Update 20230816-iam-refactor.md

* Update 20230816-iam-refactor.md

* capture discussion in architectural doc

* update variable names and refactor proposal

* project

* blueprints first round

* folder

* organization

* data-catalog-policy-tag

* re-enable folder inventory

* project module style fix

* dataproc

* source-repository

* source-repository tests

* dataplex-datascan

* dataplex-datascan tests

* net-vpc

* net-vpc test examples

* iam-service-account

* iam-service-account test examples

* kms

* boilerplate

* tfdoc

* fix module tests

* more blueprint fixes

* fix typo in data blueprints

* incomplete refactor of data platform foundations

* tfdoc

* data platform foundation

* refactor data platform foundation iam locals

* remove redundant example test

* shielded folder fix

* fix typo

* project factory

* project factory outputs

* tfdoc

* test workflow: less verbose tests, fix tf version

* re-enable -vv, shorter traceback, fix action version

* ignore github extension warning, re-enable action version

* fast bootstrap IAM, untested

* bootstrap stage IAM fixes

* stage 0 tests

* fast stage 1

* tenant stage 1

* minor changes to fast stage 0 and 1

* fast security stage

* fast mt stage 0

* fast mt stage 0

* fast pf
2023-08-20 09:44:20 +02:00
Ludovico Magnocavallo 2423fd40c1
Fix FAST CI/CD for Gitlab (#1593)
* fix cicd (multitenant untested)

* tfdoc

* rename allowed_audiences to audiences, align multitenant
2023-08-15 12:59:31 +02:00
Ludovico Magnocavallo c5a77ebfe3
fix module path for teams cicd (#1583) 2023-08-09 21:41:56 +00:00
Matt 9600047a32
Enable team CI/CD impersonation (#1579) 2023-08-09 08:46:24 -04:00
Luca Prete 47daeaafe1
Update FAST CI/CD workflows so it can work with ID_TOKEN and Gitlab 15+ 2023-08-03 16:09:45 +00:00
Ludovico Magnocavallo 154df17951
FAST: initial implementation of lightweight tenants (#1470)
* initial import

* fixes

* fixes

* fixes

* red SA roles

* red SA roles

* org-level custom roles var, tenants IAM config

* tfdoc

* allow core SA to write output files to tenant bucket

* README

* implement comments on PR

* show tenant org example

* update example
2023-07-07 08:40:37 +02:00
Julio Castillo d6aea3ff5f Remove unneeded file from resman stage 2023-06-27 09:54:46 +02:00
Keith Harvey a68a3b55cb
Bump TF version in all workflow templates to coincide with module requirements (#1445)
* Resman - bump GH TF version to coincide with module requirements (#1)

Bootstrap was bumped in #1414

* Bump TF version in all workflow files

* bump TF version in missed workflow file
2023-06-16 07:39:28 +00:00
Ludovico Magnocavallo c024eca320
Add custom tag support to FAST (#1426)
* initial implementation of custom tags

* depend org policies on tags

* fix test

* integrate default and custom org policy tags
2023-06-07 22:10:27 +00:00
Ludovico Magnocavallo 7bd6e5d57b
Small fixes (#1425)
* fix serverless connector plugin outputs

* add internal and lb to allowed ingress org policy

* add validation condition on cloud run ingress settings

* tfdoc

* plugin tfdoc

* allow disabling googleapis routes with a single instruction in net-vpc

* fix variable def

* fix variable description

* fix cr variable validation

* fix usage of   create_googleapis_routes in examples and stages
2023-06-07 17:37:46 +00:00
Ludovico Magnocavallo 4aa99ea829
allow setting identities in egress policies (#1394) 2023-05-24 12:05:16 +02:00