Commit Graph

21 Commits

Author SHA1 Message Date
Ludovico Magnocavallo 44ae2671b0
CI/CD support for Source Repository and Cloud Build (#669)
* add id to outputs

* initial cloud build implementation for stage 0

* comments

* stage 0

* stage 1, untested

* add support for IAM and CB triggers to source repository module

* refactor stage 0 to use sourcerepo module

* refactor stage 1 to use sourcerepo module

* file descriptions

* fix gitlab pipeline
2022-06-08 11:34:08 +02:00
Ludovico Magnocavallo eec0fd2fdf
FAST: allow changing tag names from variables in resman (#628) 2022-04-13 10:22:33 +02:00
Ludovico Magnocavallo 725f7effce
Initial MVP for CI/CD (#608)
* preliminary support for wif in stage 0

* IAM wif role

* IAM wif role TODO

* add support for external SA IAM to SA module

* add name output to SA module

* separate cicd SA

* tfdoc

* GITLAB principal (untested)

* make GCS name output static

* outputs bucket

* fix stage 1 test

* tweak outputs

* tfdoc

* move wif_pool to automation variable

* add support for top-level and repository providers

* add missing boilerplate

* fix branchless principal

* initial workflow

* symlink provider template in stages

* remove service accounts from stage 0 cicd tfvars

* add cicd interface variable to resman stage

* fix cicd variable in resman stage

* better condition on outputs_location

* fix last change

* change outputs_location type

* revert outputs_location change

* split outputs in stage 0

* update ci/cd temporary notes

* rename additive IAM resource in SA module

* split outputs in stage 1

* remove unused locals

* fix stage 1 tests

* tfdoc

* Upload action files to outputs_bucket

* Fix tests and README

* rename template, streamline outputs

* local templates and gcs output for all stage 2

* add workflows to local output files

* Use lowercase WIF providers everywhere

* Bring back suffix for workflow files

* Remove unused files

* Update READMEs

* preliminary CI/CD implementation for stage 1

* fix stage 1

* stage 1 cicd

* tfdoc

* fix tests

* readme and links for cicd and wif

* refactor wif providers

* refactor cicd for stage 1

* fix stage 1

* wif org policies

* split identity provider configuration from cicd

* add type attribute to cicd repositories

* valid cicd repositories have a workflow template

* refactor stage 01

* fix stage 01 tests

* minimal CI/CD documentation

* better check_links error reporting

* fix links

* Added Gitlab specific configurations

Set the default issuer_uri for Gitlab. Added allowed audiences to OIDC configuration.

* Fixed TF formatting in identity providers.

* Changing identity provider audience to null

Changing identity provider audience to default to null.

* add instructions for renaming workflows

* address Julio's comments

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: alexmeissner <alexmeissner@google.com>
2022-04-12 08:17:27 +02:00
Ludovico Magnocavallo 474bcbdd0e
Use tags and tag-based IAM conditions in FAST (#553)
* organization module

* folder module

* project module

* fix project binding

* environment tags

* use id instead of name for references

* environment bindings

* conditional org policy admin binding via tags

* rename pf service accounts and buckets

* update IAM docs

* kms module

* compute-vm

* fix compute-vm

* tfdoc
2022-02-20 11:26:30 +01:00
Julio Castillo 73c2233b9b Small fixes to DP 2022-02-16 18:52:17 +01:00
Ludovico Magnocavallo c5fa5d62e4 fix remaining issues, align variables 2022-02-16 14:12:39 +01:00
Ludovico Magnocavallo 9c736ea337
Merge branch 'master' into lcaggioni/fast-data-platform 2022-02-16 11:47:41 +00:00
Julio Castillo 49f36157d5 Fix typo in stage1 outputs 2022-02-16 10:57:54 +01:00
Julio Castillo 0544456462 Use local for tfvar outputs 2022-02-16 09:07:07 +01:00
Ludovico Magnocavallo 5c46d351d5 02-networking-vpn 2022-02-16 07:52:20 +01:00
Ludovico Magnocavallo cd52f7c34c stage 01 2022-02-16 00:35:50 +01:00
Lorenzo Caggioni 8906252528 Fix 2022-02-14 23:54:53 +01:00
Lorenzo Caggioni 7252e7ec01 Fix review comments. 2022-02-14 16:54:42 +01:00
lcaggio bf64a3dfda
Add Data Platform to FAST (#510)
* Import Fast from dev repository.
>
>
Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>

* Import Fast from dev repository.
>
>
Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>

* merge tools changes

* Import Fast from dev repository.
>
>
Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>

* add bolierplate to validate_schema

Co-authored-by: Julio Castillo <juliocc@users.noreply.github.com>

* stage 02-security

* Import Fast from dev repository.

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>

* Copy FAST top level README

* Copy FAST top level README

* TODO list

* TODO list

* fix linting action to account for fast

* remove providers file

* add missing boilerplate

* update factory README

* align examples tfdoc

* fast readmes tfdoc

* disable markdown link check

* really disable markdown link check

* update TODO

* switch to local module refs in stage0

* replace module refs in 02-sec

* Import Fast from dev repository.
>
>
Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>

* merge tools changes

* Import Fast from dev repository.
>
>
Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>

* add bolierplate to validate_schema

Co-authored-by: Julio Castillo <juliocc@users.noreply.github.com>

* Import Fast from dev repository.
>
>
Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>

* stage 02-security

* Import Fast from dev repository.

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>

* Copy FAST top level README

* Copy FAST top level README

* TODO list

* TODO list

* fix linting action to account for fast

* remove providers file

* add missing boilerplate

* update factory README

* align examples tfdoc

* fast readmes tfdoc

* disable markdown link check

* really disable markdown link check

* update TODO

* switch to local module refs in stage0

* replace module refs in 02-sec

* Move first draft to fast branch

* Fix roles and variables. Add e2e DAG example!

* Fix example

* Fix KMS

* First draft: README

* Update README

* Add DLP, update README

* Update Readme

* README

* Add todos

* Merge master

* Merge master

* Merge master

* Fix and test KMS, Fix and test existing prj (it works also with single prj), Update README

* Fix READM and Demo

* add  on TF files

* Remove block comments

* simplify service_encryption_keys logic

* fix README

* Fix TODOs

* fix tfdoc description

* fix demo README

* fix sample files

* rename tf files

* Fix outputs file name, fix README, remove dependeces on composer resource

* Add test.

* Fix README.

* Initial README update

* README review

* Fix issues & readme

* Fix README

* Fix README

* Fix test error

* Fix test error

* Add datacatalog

* Fix test, for real? :-)

* fix readme

* support policy_boolean

* split Cloud NAT flag

* Fix README.

* Fix Shared VPC, first try :-)

* Fix tests and resource name

* fix tests

* fix tests

* README refactor

* Fix secondary range logic

* First commit

* Replace existing data platform

* Fix secondary range logic

* Fix README

* Replace DP example tests with the new one.

* Fix test module location.

* Fix test module location, for real.

* Support DataPlatform project in VPC-SC

* Fix VPC-SC

* Add TODO, VPC-SC

* Possible improvement to handle VPC-SC perimeter projects with folder as variable

* Add TODO

* Fix module path

* Initial fix for KMS

* Add PubSub encryption

* Fix secondary range logic

* First commit

* Support DataPlatform project in VPC-SC

* Fix VPC-SC

* Add TODO, VPC-SC

* Possible improvement to handle VPC-SC perimeter projects with folder as variable

* Add TODO

* Fix module path

* Initial fix for KMS

* Update READMEs

* Update README

* Fix composer roles and README.

* Fix test.

* Fixes.

* Add DLP documentation link.

* Temp commit with errors

* Refactor variables

* Fix secondary range logic

* First commit

* Support DataPlatform project in VPC-SC

* Fix VPC-SC

* Add TODO, VPC-SC

* Possible improvement to handle VPC-SC perimeter projects with folder as variable

* Add TODO

* Fix module path

* Initial fix for KMS

* rebase

* rebase

* rebase

* Rebase

* rebase

* Update READMEs

* Fixes.

* Fix new variables

* Fix misconfiguration and tests.

* Fix secondary range logic

* First commit

* Support DataPlatform project in VPC-SC

* Fix VPC-SC

* Add TODO, VPC-SC

* Possible improvement to handle VPC-SC perimeter projects with folder as variable

* Add TODO

* Fix module path

* Initial fix for KMS

* rebase

* rebase

* rebase

* Rebase

* rebase

* Update READMEs

* Fixes.

* Rebase - Fix secondary range logic

* Rebase - First commit

* Support DataPlatform project in VPC-SC

* Fix VPC-SC

* Possible improvement to handle VPC-SC perimeter projects with folder as variable

* Initial fix for KMS

* Fix secondary range logic

* First commit

* Support DataPlatform project in VPC-SC

* Fix VPC-SC

* Fix module path

* Initial fix for KMS

* Update READMEs

* Fixes.

* Fix new variables

* Revert VPC-SC logic

* Fix variable typos

* README fixes

* Fix Project Name logic

* Fix Linting

* READEME

* update READEME

* update READEME

* update README

* mandatory project creation, refactor

* formatting

* add TODO for service accounts descriptive name

* use project module to assign shared vpc roles

* Fix shared-vpc-project module

* Fix vpc name and tests

* README

* update to newer version

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>
Co-authored-by: Julio Castillo <juliocc@users.noreply.github.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2022-02-11 17:32:16 +01:00
Ludovico Magnocavallo d711a93397
fix previous merge (#513) 2022-02-06 18:34:32 +01:00
Ludovico Magnocavallo c4533503a6
Fast: use pathexpand for output location (#512)
* use pathexpand for output location

* tfdoc
2022-02-06 18:24:46 +01:00
Simone Ruffilli 1aa5d5d93f
Misc documentation fixes (#493) 2022-02-06 10:50:23 +01:00
Julio Castillo ed4fc477a7 Update resman outputs and projects 2022-02-04 16:21:00 +01:00
Simone Ruffilli 0cef15301b
Rename 02-networking to 02-networking-vpn (#492) 2022-02-04 11:03:37 +01:00
Simone Ruffilli 8272694360 02-net-nva: README.md updates 2022-02-04 07:42:56 +01:00
Ludovico Magnocavallo e866db2c88 Merge Fabric FAST (#435)
Co-authored-by: Simone Ruffilli <sruffilli@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2022-01-19 14:19:39 +01:00