6941313c7d
Factories refactor ( #1843 )
...
* factories refactor doc
* Adds file schema and filesystem organization
* Update 20231106-factories.md
* move factories out of blueprints and create new factories README
* align factory in billing-account module
* align factory in dataplex-datascan module
* align factory in billing-account module
* align factory in net-firewall-policy module
* align factory in dns-response-policy module
* align factory in net-vpc-firewall module
* align factory in net-vpc module
* align factory variable names in FAST
* remove decentralized firewall blueprint
* bump terraform version
* bump module versions
* update top-level READMEs
* move project factory to modules
* fix variable names and tests
* tfdoc
* remove changelog link
* add project factory to top-level README
* fix cludrun eventarc diff
* fix README
* fix cludrun eventarc diff
---------
Co-authored-by: Simone Ruffilli <sruffilli@google.com>
2024-02-26 10:16:52 +00:00
d11c380aec
Format python files in blueprints ( #2079 )
...
* format python files in blueprints
* update check on blueprints python code
* update python linter in CI workflow
2024-02-15 09:37:49 +01:00
b902b1dab9
Fix non-empty plan after apply
2024-02-13 07:40:31 +01:00
71a64487d5
Extend FAST to support different principal types ( #2064 )
...
* add doc draft
* typos
* typo
* typo
* typos
* rewording
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* move iam variables to a separate file
* move billing-account module to iam_principals
* move data-catalog-policy-tag module to iam_principals
* move dataplex-datascan module to iam_principals
* move dataproc module to iam_principals
* move folder module to iam_principals
* copyright
* move organization module to iam_principals
* move project module to iam_principals
* move source-repository module to iam_principals
* update blueprints for iam_principals interface
* FAST bootstrap
* module READMEs fixes
* FAST bootstrap
* FAST networking stages
* FAST security stage
* FAST gke stage
* FAST multitenant bootstrap stage
* FAST multitenant resman stage
* tfdoc
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* fix module test
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Rename iam_principals to iam_by_principals
* Update IAM template to include iam_by_principals
* Update Resman README
* Fix ADR link format
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2024-02-12 14:35:30 +01:00
3b4b355800
Ensure data platform service accounts meet FAST requirements ( #2016 )
...
* use 9-chars prefix in data platform example test
* use 9-chars prefix in data platform example test
2024-01-28 14:00:32 +01:00
208902c8da
Fix Data platform foundation ( #1992 )
...
* FAST + Minimal DP
* Fix tests
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2024-01-20 08:49:46 +01:00
6c258e6562
Fix description
2023-12-29 12:09:16 +01:00
1eea077460
Add service account email to outputs to manage its permissions
2023-12-29 12:09:16 +01:00
724dfda09c
Provider doesn't set defaults on config
...
Can't provide just one size (like `web_server` or `triggerrer`) because
of no defaults are taken:
module.composer.google_composer_environment.env: Modifying... [id=***]
╷
│ Error: googleapi: Error 400: Found 6 problems:
│ 1) You have to specify Scheduler CPUs not lower than 0.5.
│ 2) You have to specify number of schedulers larger than 0.
│ 3) You have to specify Web Server CPUs not lower than 0.5.
│ 4) You have to specify Worker CPUs not lower than 0.5.
│ 5) You have to specify minimum number of workers larger than 0.
│ 6) Triggerer memory must be between 1.00GB and 6.50GB for given vCpu
So provide the defaults as set workloads_config == null
2023-12-29 12:09:16 +01:00
bba814c091
Custom role factories for organization and project modules ( #1912 )
...
* backport custom role factories
* backport from fast ci/cd branch
* indent
* tfdoc
* fix module tests
2023-12-11 14:16:39 +00:00
886734e1e9
Add trigger configuration for Composer ( #1916 )
2023-12-11 12:54:49 +01:00
98accdb3ad
Added PSC support to CloudSQL Module ( #1874 )
...
* Added Feature
* Added PSC to CloudSQL module
* Added psc to read replica
* Changed variables
* Updated README
* Ran fmt
* Removed old variables
* Fix README
* Fixed blueprints
* Fix README
* Fixed output
* Added more outputs and bug fixes
* Changed variable structure
* Bug fix
* Added PSC example.
2023-11-24 15:47:45 +01:00
9ff67fdaf0
Add DLP Service Agent role
2023-11-20 15:34:27 +01:00
543ea6e7f3
Fix/dlpagent ( #1868 )
...
Create DLP Service Account on service activation.
2023-11-20 14:11:01 +01:00
63f1dfb6de
end-to-end tests for Vertex blueprint
2023-11-15 11:04:12 +00:00
1c2f1c7b0d
Sql user features ( #1856 )
...
* added user type feature
* fix readme
* fix comment
* fix blueprint cloudsql users value + minor fix
* readme fix
* variables fix
* local var fix
* fix for in local var
* fix on readme
* fix intentations var in readme
* fix blueprint user quote
---------
Co-authored-by: Francesco Spinelli <francesco.spinelli@nttdata.com>
2023-11-13 10:27:14 +01:00
d07f8fd33d
Added CMEK for Secret auto managed ( #1739 )
...
Allow to specify custom KMS keys for Secret Manager secrets
2023-11-10 16:45:47 +01:00
a0ae43fc6f
[Minimal Data Platform] Fix Landing and curated IAM ( #1832 )
...
* Fix IAM on Minimal DP
* fix
2023-11-01 17:53:06 +01:00
4decc641bb
Stop wrapping yamldecode with try() ( #1812 )
2023-10-25 16:16:05 +02:00
6d89b88149
versions.tf maintenance + copyright notice bump ( #1782 )
...
* Bump copyright notice to 2023
* Delete versions.tf on blueprints
* Pin provider to major version 5
* Remove comment
* Fix lint
* fix bq-ml blueprint readme
---------
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-20 18:17:47 +02:00
6c48512f7e
[ #1764 ] net-lb-int: add support for dual stack and multiple forwarding rules
2023-10-17 09:30:34 +00:00
dfc5023e0b
Make deletion protection consistent across all modules ( #1735 )
...
* Expose deletion_protection in GKE modules
* Make deletion protection consistent across all modules
* Add deletion_protection option to blueprints
* Fix blueprints tests
* Fix types
* Update READMEs
* Fix dp readme
* Fix cmek blueprint default deletion_protection
* Fix blueprints tests
2023-10-05 17:31:07 +02:00
de883957fe
Minimal Data Platform - Fix ( #1730 )
...
* Fix
* Fix README
* Fix data platform minimal tests
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-04 12:15:51 +02:00
6889f02954
Fix data platform roles ( #1725 )
...
* Fix Data Platform roles
* Fix README
* Fix blueprint tests
* Update cleanup dp steps
---------
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-10-04 07:31:40 +02:00
789328ff5a
Bump provider versions to v5.0.0 ( #1724 )
...
* bump provider versions to 5.0.0
* fix cloud run, logging and vpc-sc
* Fix secret manager
* fix gke nodepool
* fix gke multitenant stage and blueprint
* Moving alloydb module to experimental.
* Add project to bare resources in examples
* tfdoc
* fix svpc blueprint test
* Revert "fix svpc blueprint test"
This reverts commit 14f02659098070136e64ead600580dd52c23c339.
* Fix GKE peering project
* Disable tests in alloydb module
* Bring back secret ids in secret manager tests
* Remove duplicate key
* last push
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-03 12:15:36 +00:00
df5daab6cc
Allow using no service account in compute-vm ( #1692 )
...
* module and tests
* align blueprints
* tfdoc
* remove stale variables
* fix blueprint
* variable description
2023-09-19 16:56:51 +00:00
ad14a7d415
Update READMEs
2023-09-17 00:21:36 +02:00
9c878dc9cf
Fix tests for new KMS IAM interface
2023-09-17 00:21:36 +02:00
da883bab8c
Update kms module key-level IAM
2023-09-17 00:21:36 +02:00
6eb862a775
GKE cluster modules: add optional kube state metrics ( #1682 )
...
* `gke-cluster-standard`: add optional kube state metrics
* `gke-cluster-autopilot`: add optional kube state metrics
* FAST: add kube state metrics support for GKE
* blueprints/gke: add kube state metrics support
* Bump up the provider version to `v4.82.0`
2023-09-15 12:18:45 +01:00
79723f9ce1
Add lineage on Minimal Data Platform blueprint ( #1679 )
...
Add lineage on Minimal Data Platform blueprint
2023-09-14 17:52:19 +02:00
0e7cfc8b29
add AIRFLOW_VAR_ prefix to environment variables in data-platform blueprints ( #1651 )
...
* add AIRFLOW_VAR_ prefix to env vars in minimal blueprint
* update DAGs for env vars in minimal blueprint
* add AIRFLOW_VAR_ prefix to env vars in foundation blueprint
* update DAGs for env vars in foundation blueprint
* apply tf linting
---------
Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-09-08 07:38:28 +00:00
77c1e69666
New phpIPAM serverless third parties solution in blueprints ( #1642 )
...
* Added new phpIPAM serverless third parties solution in blueprints
* added jit to iap.googleapis.com service in project module
* updated tests
2023-09-07 15:30:22 +02:00
3d4cc7164a
Bump provider version to 4.80.0
2023-09-05 09:48:15 +02:00
819894d2ba
IAM interface refactor ( #1595 )
...
* IAM modules refactor proposal
* policy
* subheading
* Update 20230816-iam-refactor.md
* log Julio's +1
* data-catalog-policy-tag
* dataproc
* dataproc
* folder
* folder
* folder
* folder
* project
* better filtering in test examples
* project
* folder
* folder
* organization
* fix variable descriptions
* kms
* net-vpc
* dataplex-datascan
* modules/iam-service-account
* modules/source-repository/
* blueprints/cloud-operations/vm-migration/
* blueprints/third-party-solutions/wordpress
* dataplex-datascan
* blueprints/cloud-operations/workload-identity-federation
* blueprints/data-solutions/cloudsql-multiregion/
* blueprints/data-solutions/composer-2
* Update 20230816-iam-refactor.md
* Update 20230816-iam-refactor.md
* capture discussion in architectural doc
* update variable names and refactor proposal
* project
* blueprints first round
* folder
* organization
* data-catalog-policy-tag
* re-enable folder inventory
* project module style fix
* dataproc
* source-repository
* source-repository tests
* dataplex-datascan
* dataplex-datascan tests
* net-vpc
* net-vpc test examples
* iam-service-account
* iam-service-account test examples
* kms
* boilerplate
* tfdoc
* fix module tests
* more blueprint fixes
* fix typo in data blueprints
* incomplete refactor of data platform foundations
* tfdoc
* data platform foundation
* refactor data platform foundation iam locals
* remove redundant example test
* shielded folder fix
* fix typo
* project factory
* project factory outputs
* tfdoc
* test workflow: less verbose tests, fix tf version
* re-enable -vv, shorter traceback, fix action version
* ignore github extension warning, re-enable action version
* fast bootstrap IAM, untested
* bootstrap stage IAM fixes
* stage 0 tests
* fast stage 1
* tenant stage 1
* minor changes to fast stage 0 and 1
* fast security stage
* fast mt stage 0
* fast mt stage 0
* fast pf
2023-08-20 09:44:20 +02:00
ea0de3adbb
Fixing some typos
2023-08-18 05:51:00 +00:00
79373721df
Remove firewall policy management from resource management modules ( #1581 )
...
* rename firewall policy module, fix outputs
* add TOC to firewall policy module
* don't depend policy on parent id
* remove firewall policy from resource management modules
* remove factory conditionals
* fast net a and b
* fast stages
* fast tfdoc
* fast tfdoc
* remove unused test
* fix shielded folder blueprint
* fix shielded folder blueprint
2023-08-09 11:23:07 +00:00
dd3a298892
Moved allow_net_admin to enable_features flag. Bumped provider version to 4.76
2023-08-07 14:27:20 +01:00
182ce27143
Add information about required groups
2023-08-06 20:27:59 +02:00
9c75aa469c
More module descriptions ( #1572 )
...
* bigquery dataset
* data catalog policy tag
* net-address
* fix data catalog callers
* bigquery dataset views
* fix data catalog callers
* logging bucket
* net vpn ha
2023-08-06 09:25:45 +00:00
a2eddd0be0
Fixed Cloud Build default bucket name and vpc/subnet names ( #1548 )
2023-07-31 12:52:36 +02:00
5a534edf5c
update and refactor artifact registry module ( #1536 )
2023-07-28 09:54:36 +00:00
009c6d0f93
Make demo pipeline append into BQ tables ( #1533 )
2023-07-27 11:38:00 -04:00
0b224a7f55
Return only bucket name of composer, not full url to dags folder
2023-07-10 11:20:51 +02:00
d9f7cef1e7
allow using a separate resource for boot disk ( #1496 )
2023-07-07 15:40:13 +00:00
d3739cc0ff
Update outputs.tf
2023-06-29 22:36:27 -04:00
43b3490ef1
Updating a few files for typos
...
fast/stages/3-data-platform/dev/README.md
fast/stages/3-data-platform/dev/outputs.tf
CHANGELOG.md
blueprints/data-solutions/data-platform-minimal/README.md
blueprints/data-solutions/data-platform-minimal/outputs.tf
blueprints/data-solutions/data-platform-foundations/README.md
2023-06-29 21:47:17 -04:00
b203198441
Merge branch 'master' into kthhrv-patch-1
2023-06-29 10:10:14 +01:00
912497a158
use key/value and update docs
2023-06-29 09:57:23 +01:00
f15e2f3509
default to empty map
2023-06-29 09:55:10 +01:00
Ludovico Magnocavallo
simonebruzzechesse
Wiktor Niesiobędzki
lcaggio
luigi-bitonti
ddaluka
Francesco Spinelli
Simone Ruffilli
Luca Prete
Julio Castillo
Oliver Frolovs
Pierre Formont
Alejandro Leal
Eunan Hardy
javiergp
Daniel De Leo
Alejandro Leal
Keith Harvey