Commit Graph

110 Commits

Author SHA1 Message Date
Ludovico Magnocavallo 525684faf3
Define service attachment interface for lb modules and implement in internal LBs (#2122)
* add service attachment support to lb app int module

* allow direct referencing of self managed ig in ilb module

* add service attachment support to net-ilb-int

* add service attachments example to net-lb-int

* fix resource name in net-lb-ext

* rename fwd rules resource in test inventories

* add toc to net-lb-int
2024-03-02 18:36:29 +00:00
Ludovico Magnocavallo 6941313c7d
Factories refactor (#1843)
* factories refactor doc

* Adds file schema and filesystem organization

* Update 20231106-factories.md

* move factories out of blueprints and create new factories  README

* align factory in billing-account module

* align factory in dataplex-datascan module

* align factory in billing-account module

* align factory in net-firewall-policy module

* align factory in dns-response-policy module

* align factory in net-vpc-firewall module

* align factory in net-vpc module

* align factory variable names in FAST

* remove decentralized firewall blueprint

* bump terraform version

* bump module versions

* update top-level READMEs

* move project factory to modules

* fix variable names and tests

* tfdoc

* remove changelog link

* add project factory to top-level README

* fix cludrun eventarc diff

* fix README

* fix cludrun eventarc diff

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
2024-02-26 10:16:52 +00:00
simonebruzzechesse c50b732c79
Allow granting network user role on host project from project module and factory (#1930)
* Update shared vpc config for project factory and project module for more granular Shared VPC configuration

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-12-15 14:39:21 +01:00
Ludovico Magnocavallo f548b65b1c
Add support for subnet-level service network user grants to project module, improve docs (#1907)
* improve project factory example

* light refactor of project modules shared vpc internals and docs

* add support for subnet-level grants on host project
2023-12-07 09:07:48 +00:00
Luca Prete 642f65b841
F5 blueprint (#1787)
---------

Co-authored-by: Luca Prete <lucaprete@google.com>
2023-11-22 19:48:13 +01:00
Francesco Spinelli 82c74e4ab6
Dataproc module bug fix (#1848)
* bug fix

* bug fix

* fix dinamic for_each

---------

Co-authored-by: Francesco Spinelli <francesco.spinelli@nttdata.com>
2023-11-09 15:48:29 +00:00
Ludovico Magnocavallo 03937f2b6e
Support multilevel data and allow overriding project id in project factory (#1851) 2023-11-09 08:29:46 +00:00
Ludovico Magnocavallo de0325b3a3
Avoid map-related casting errors in project factory (#1836)
* try to repro pf example error

* repro

* repro

* pf fix

* remove extra file

* FAST stage
2023-11-02 08:24:50 +01:00
Simone Ruffilli 6d89b88149
versions.tf maintenance + copyright notice bump (#1782)
* Bump copyright notice to 2023

* Delete versions.tf on blueprints

* Pin provider to major version 5

* Remove comment

* Fix lint

* fix bq-ml blueprint readme

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-20 18:17:47 +02:00
Ludovico Magnocavallo aa5d883deb
add support for org policies to project factory (#1722) 2023-10-02 14:13:56 +00:00
Pierre Formont e141b86f79
merge labels from data_merges in project factory (#1662)
* merge labels from data_merges in project factory

* fix project factory test for labels
2023-09-08 10:27:46 +00:00
Luca Prete 12e78af055
Fix project factory blueprint and fast stage (#1654) 2023-09-07 12:48:39 +00:00
Ludovico Magnocavallo 819894d2ba
IAM interface refactor (#1595)
* IAM modules refactor proposal

* policy

* subheading

* Update 20230816-iam-refactor.md

* log Julio's +1

* data-catalog-policy-tag

* dataproc

* dataproc

* folder

* folder

* folder

* folder

* project

* better filtering in test examples

* project

* folder

* folder

* organization

* fix variable descriptions

* kms

* net-vpc

* dataplex-datascan

* modules/iam-service-account

* modules/source-repository/

* blueprints/cloud-operations/vm-migration/

* blueprints/third-party-solutions/wordpress

* dataplex-datascan

* blueprints/cloud-operations/workload-identity-federation

* blueprints/data-solutions/cloudsql-multiregion/

* blueprints/data-solutions/composer-2

* Update 20230816-iam-refactor.md

* Update 20230816-iam-refactor.md

* capture discussion in architectural doc

* update variable names and refactor proposal

* project

* blueprints first round

* folder

* organization

* data-catalog-policy-tag

* re-enable folder inventory

* project module style fix

* dataproc

* source-repository

* source-repository tests

* dataplex-datascan

* dataplex-datascan tests

* net-vpc

* net-vpc test examples

* iam-service-account

* iam-service-account test examples

* kms

* boilerplate

* tfdoc

* fix module tests

* more blueprint fixes

* fix typo in data blueprints

* incomplete refactor of data platform foundations

* tfdoc

* data platform foundation

* refactor data platform foundation iam locals

* remove redundant example test

* shielded folder fix

* fix typo

* project factory

* project factory outputs

* tfdoc

* test workflow: less verbose tests, fix tf version

* re-enable -vv, shorter traceback, fix action version

* ignore github extension warning, re-enable action version

* fast bootstrap IAM, untested

* bootstrap stage IAM fixes

* stage 0 tests

* fast stage 1

* tenant stage 1

* minor changes to fast stage 0 and 1

* fast security stage

* fast mt stage 0

* fast mt stage 0

* fast pf
2023-08-20 09:44:20 +02:00
Ludovico Magnocavallo 79373721df
Remove firewall policy management from resource management modules (#1581)
* rename firewall policy module, fix outputs

* add TOC to firewall policy module

* don't depend policy on parent id

* remove firewall policy from resource management modules

* remove factory conditionals

* fast net a and b

* fast stages

* fast tfdoc

* fast tfdoc

* remove unused test

* fix shielded folder blueprint

* fix shielded folder blueprint
2023-08-09 11:23:07 +00:00
Wiktor Niesiobędzki 4998f1d376
Grant IAM rights to service identities in host project (#1542)
* [module/project] Grant IAM rights to service identities based on used services in host project
* [blueprints/factories/project-factory] enable granting IAM permissions in host VPC for service identities directly or by specifying services in use
2023-07-29 20:07:21 +02:00
Miren Esnaola cacb0c02e2 Refactoring of dns module 2023-07-19 12:57:44 +02:00
Albert Lloveras 7cacc46b4b
fixup(project-factory): Use the correct KMS Service Agents attribute … (#1446)
* fixup(project-factory): Use the correct KMS Service Agents attribute name

* Add new KMS bindings to tests

* Update test resource counts

* Update README.md resource count
2023-06-19 23:53:08 +00:00
Julio Castillo c819305a42 Migrate apigee tests 2023-04-21 17:51:19 +02:00
Julio Castillo 6be11dc013 Add empty test in blueprints folder 2023-02-25 19:26:11 +01:00
Julio Castillo be60753246 Fix failing tests 2023-02-25 19:26:11 +01:00
Julio Castillo edce6edd28 Update factories and apigee tests 2023-02-25 19:26:11 +01:00
Julio Castillo b186365cab Update data solutions tests 2023-02-25 19:26:11 +01:00
Julio Castillo 3c68780537 Update gke blueprints tests 2023-02-25 19:26:11 +01:00
Julio Castillo a25f83656f Update cloud operations tests 2023-02-25 19:26:11 +01:00
Julio Castillo d71d0c5c22 Update networking bluprints tests 2023-02-25 19:26:11 +01:00
Julio Castillo daccd803ed Update decentralized-firewall tests 2023-02-25 19:26:11 +01:00
Julio Castillo 3c88315f6e Update tests for hub and spoke blueprints 2023-02-25 19:26:11 +01:00
Julio Castillo 7c17af66aa Update asset-inventory-feed-remediation tests 2023-02-25 19:26:11 +01:00
Julio Castillo f777e97cef Update tests terraform_enterprise_wif 2023-02-25 19:26:11 +01:00
Julio Castillo 3e4b6c0639 Update iam-delegated-role-grants tests 2023-02-25 19:26:11 +01:00
Julio Castillo d114a81b08 Update packer_image_builder tests 2023-02-25 19:26:11 +01:00
Julio Castillo 23c564cb74 Update unmanaged-instances-healthcheck tests 2023-02-25 19:26:11 +01:00
Julio Castillo 99e69e319f Update data platform tests 2023-02-25 19:26:11 +01:00
Julio Castillo 71e96dfdee Allow blueprints to be tested from README files without examples. 2023-02-25 19:26:11 +01:00
lcaggio 47855cb682
Merge branch 'master' into lcaggio/dp-projectcreate 2023-02-23 11:54:48 +01:00
lcaggio 2108b4650d Fix Tests, rely on iam additive. 2023-02-22 01:36:01 +01:00
Julio Castillo a5e905cb80 Update remaining org policies 2023-02-21 15:49:16 +01:00
lcaggio f4c1fa6c20 Fix tests. 2023-02-20 00:56:32 +01:00
Aleksandr Averbukh 4ad30b812b Fix tests for tf-cloud-dynamic-credentials blueprint 2023-02-19 01:14:24 +01:00
Aleksandr Averbukh befc73ec12 feat: Update TFC+WIF blueprint with TFC Dynamic Credentials feature. 2023-02-19 00:51:49 +01:00
Julio Castillo d7b88b7d0e Fix bq factory docs 2023-02-08 18:22:10 +01:00
lcaggio dd77c2fe79
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-07 07:58:00 +01:00
lcaggio d68bb4b320 Fixes 2023-02-06 21:33:04 +01:00
lcaggio 1e4499c8ab Fixes 2023-02-06 14:58:40 +01:00
Ayman Farhat 02d8d8367a
[Feature] Update data platform blue print with Dataflow Flex template (#1105)
* Add initial dataflow template code + TF infra

* Refactor the datapipeline DAG to use flex template operator, cleanup code

* Remove unneeded bash scripts, update README with manual examples

* Refactor datapipeline_dc_tags.py and include new Flex template

* Update docs to reflect changes

* Remove sub-dependencies and keep apache beam

* Add missing license headers and update tests

* Set resouces to 291 in tests

* Update outputs via tfdoc

* Update with outputs order and tfdoc

* Correct number of resources

* Fix to add region into command from var

* Enable service account impersonation for running builds

* Update example dataflow run command to use orchestrator SA

* Remove hard coded values in example

* Keep original airflow files, add new which use Flex template as example

* Update tests and doc

* Fix number of resources in plan

* Run tfdoc remove files section in README

* Fix number of modules in tfdoc

* Update number of resources

* Add missin service account

* Update DF demo README

* Quick rename

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-02-06 07:35:40 +01:00
lcaggio 39ffdcf184
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-05 22:45:14 +01:00
lcaggio 47c75f8862 Implement project_config variable 2023-02-05 21:12:46 +01:00
lcaggio 38fc9e8b9b First commit 2023-02-05 07:52:33 +01:00
lcaggio f00d4d3edd
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-03 22:17:30 +01:00
javiergp ce1f86d20b
Vertex Pipelines MLOps framework blueprint (#1038)
* First release of the MLOps blueprint
2023-02-02 19:13:13 +01:00