d07f8fd33d
Added CMEK for Secret auto managed ( #1739 )
...
Allow to specify custom KMS keys for Secret Manager secrets
2023-11-10 16:45:47 +01:00
a0ae43fc6f
[Minimal Data Platform] Fix Landing and curated IAM ( #1832 )
...
* Fix IAM on Minimal DP
* fix
2023-11-01 17:53:06 +01:00
4decc641bb
Stop wrapping yamldecode with try() ( #1812 )
2023-10-25 16:16:05 +02:00
6d89b88149
versions.tf maintenance + copyright notice bump ( #1782 )
...
* Bump copyright notice to 2023
* Delete versions.tf on blueprints
* Pin provider to major version 5
* Remove comment
* Fix lint
* fix bq-ml blueprint readme
---------
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-20 18:17:47 +02:00
6c48512f7e
[ #1764 ] net-lb-int: add support for dual stack and multiple forwarding rules
2023-10-17 09:30:34 +00:00
dfc5023e0b
Make deletion protection consistent across all modules ( #1735 )
...
* Expose deletion_protection in GKE modules
* Make deletion protection consistent across all modules
* Add deletion_protection option to blueprints
* Fix blueprints tests
* Fix types
* Update READMEs
* Fix dp readme
* Fix cmek blueprint default deletion_protection
* Fix blueprints tests
2023-10-05 17:31:07 +02:00
de883957fe
Minimal Data Platform - Fix ( #1730 )
...
* Fix
* Fix README
* Fix data platform minimal tests
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-04 12:15:51 +02:00
6889f02954
Fix data platform roles ( #1725 )
...
* Fix Data Platform roles
* Fix README
* Fix blueprint tests
* Update cleanup dp steps
---------
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-10-04 07:31:40 +02:00
789328ff5a
Bump provider versions to v5.0.0 ( #1724 )
...
* bump provider versions to 5.0.0
* fix cloud run, logging and vpc-sc
* Fix secret manager
* fix gke nodepool
* fix gke multitenant stage and blueprint
* Moving alloydb module to experimental.
* Add project to bare resources in examples
* tfdoc
* fix svpc blueprint test
* Revert "fix svpc blueprint test"
This reverts commit 14f02659098070136e64ead600580dd52c23c339.
* Fix GKE peering project
* Disable tests in alloydb module
* Bring back secret ids in secret manager tests
* Remove duplicate key
* last push
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-03 12:15:36 +00:00
df5daab6cc
Allow using no service account in compute-vm ( #1692 )
...
* module and tests
* align blueprints
* tfdoc
* remove stale variables
* fix blueprint
* variable description
2023-09-19 16:56:51 +00:00
ad14a7d415
Update READMEs
2023-09-17 00:21:36 +02:00
9c878dc9cf
Fix tests for new KMS IAM interface
2023-09-17 00:21:36 +02:00
da883bab8c
Update kms module key-level IAM
2023-09-17 00:21:36 +02:00
6eb862a775
GKE cluster modules: add optional kube state metrics ( #1682 )
...
* `gke-cluster-standard`: add optional kube state metrics
* `gke-cluster-autopilot`: add optional kube state metrics
* FAST: add kube state metrics support for GKE
* blueprints/gke: add kube state metrics support
* Bump up the provider version to `v4.82.0`
2023-09-15 12:18:45 +01:00
79723f9ce1
Add lineage on Minimal Data Platform blueprint ( #1679 )
...
Add lineage on Minimal Data Platform blueprint
2023-09-14 17:52:19 +02:00
0e7cfc8b29
add AIRFLOW_VAR_ prefix to environment variables in data-platform blueprints ( #1651 )
...
* add AIRFLOW_VAR_ prefix to env vars in minimal blueprint
* update DAGs for env vars in minimal blueprint
* add AIRFLOW_VAR_ prefix to env vars in foundation blueprint
* update DAGs for env vars in foundation blueprint
* apply tf linting
---------
Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-09-08 07:38:28 +00:00
77c1e69666
New phpIPAM serverless third parties solution in blueprints ( #1642 )
...
* Added new phpIPAM serverless third parties solution in blueprints
* added jit to iap.googleapis.com service in project module
* updated tests
2023-09-07 15:30:22 +02:00
3d4cc7164a
Bump provider version to 4.80.0
2023-09-05 09:48:15 +02:00
819894d2ba
IAM interface refactor ( #1595 )
...
* IAM modules refactor proposal
* policy
* subheading
* Update 20230816-iam-refactor.md
* log Julio's +1
* data-catalog-policy-tag
* dataproc
* dataproc
* folder
* folder
* folder
* folder
* project
* better filtering in test examples
* project
* folder
* folder
* organization
* fix variable descriptions
* kms
* net-vpc
* dataplex-datascan
* modules/iam-service-account
* modules/source-repository/
* blueprints/cloud-operations/vm-migration/
* blueprints/third-party-solutions/wordpress
* dataplex-datascan
* blueprints/cloud-operations/workload-identity-federation
* blueprints/data-solutions/cloudsql-multiregion/
* blueprints/data-solutions/composer-2
* Update 20230816-iam-refactor.md
* Update 20230816-iam-refactor.md
* capture discussion in architectural doc
* update variable names and refactor proposal
* project
* blueprints first round
* folder
* organization
* data-catalog-policy-tag
* re-enable folder inventory
* project module style fix
* dataproc
* source-repository
* source-repository tests
* dataplex-datascan
* dataplex-datascan tests
* net-vpc
* net-vpc test examples
* iam-service-account
* iam-service-account test examples
* kms
* boilerplate
* tfdoc
* fix module tests
* more blueprint fixes
* fix typo in data blueprints
* incomplete refactor of data platform foundations
* tfdoc
* data platform foundation
* refactor data platform foundation iam locals
* remove redundant example test
* shielded folder fix
* fix typo
* project factory
* project factory outputs
* tfdoc
* test workflow: less verbose tests, fix tf version
* re-enable -vv, shorter traceback, fix action version
* ignore github extension warning, re-enable action version
* fast bootstrap IAM, untested
* bootstrap stage IAM fixes
* stage 0 tests
* fast stage 1
* tenant stage 1
* minor changes to fast stage 0 and 1
* fast security stage
* fast mt stage 0
* fast mt stage 0
* fast pf
2023-08-20 09:44:20 +02:00
ea0de3adbb
Fixing some typos
2023-08-18 05:51:00 +00:00
79373721df
Remove firewall policy management from resource management modules ( #1581 )
...
* rename firewall policy module, fix outputs
* add TOC to firewall policy module
* don't depend policy on parent id
* remove firewall policy from resource management modules
* remove factory conditionals
* fast net a and b
* fast stages
* fast tfdoc
* fast tfdoc
* remove unused test
* fix shielded folder blueprint
* fix shielded folder blueprint
2023-08-09 11:23:07 +00:00
dd3a298892
Moved allow_net_admin to enable_features flag. Bumped provider version to 4.76
2023-08-07 14:27:20 +01:00
182ce27143
Add information about required groups
2023-08-06 20:27:59 +02:00
9c75aa469c
More module descriptions ( #1572 )
...
* bigquery dataset
* data catalog policy tag
* net-address
* fix data catalog callers
* bigquery dataset views
* fix data catalog callers
* logging bucket
* net vpn ha
2023-08-06 09:25:45 +00:00
a2eddd0be0
Fixed Cloud Build default bucket name and vpc/subnet names ( #1548 )
2023-07-31 12:52:36 +02:00
5a534edf5c
update and refactor artifact registry module ( #1536 )
2023-07-28 09:54:36 +00:00
009c6d0f93
Make demo pipeline append into BQ tables ( #1533 )
2023-07-27 11:38:00 -04:00
0b224a7f55
Return only bucket name of composer, not full url to dags folder
2023-07-10 11:20:51 +02:00
d9f7cef1e7
allow using a separate resource for boot disk ( #1496 )
2023-07-07 15:40:13 +00:00
d3739cc0ff
Update outputs.tf
2023-06-29 22:36:27 -04:00
43b3490ef1
Updating a few files for typos
...
fast/stages/3-data-platform/dev/README.md
fast/stages/3-data-platform/dev/outputs.tf
CHANGELOG.md
blueprints/data-solutions/data-platform-minimal/README.md
blueprints/data-solutions/data-platform-minimal/outputs.tf
blueprints/data-solutions/data-platform-foundations/README.md
2023-06-29 21:47:17 -04:00
b203198441
Merge branch 'master' into kthhrv-patch-1
2023-06-29 10:10:14 +01:00
912497a158
use key/value and update docs
2023-06-29 09:57:23 +01:00
f15e2f3509
default to empty map
2023-06-29 09:55:10 +01:00
e3e05540de
change var name, drop default, update docs
2023-06-29 09:45:29 +01:00
026071209c
Minimal Data Platform - Shared VPC ( #1475 )
...
* Fix
* Fix dataproc vpc links
* Add missing networkUser role.
* Fix README.
2023-06-28 21:58:03 +02:00
52ad1fee34
more linting
2023-06-28 14:07:36 +01:00
f9742c5405
lint
2023-06-28 14:06:05 +01:00
9e8323bd7d
Update 02-composer.tf
2023-06-28 13:56:35 +01:00
b30efa8bfe
Update variables.tf
2023-06-28 13:56:00 +01:00
7cd9083fd7
data-platform-minimal - support web_server_network_access_control
2023-06-28 13:28:28 +01:00
099ad03910
Improve Minimal Data Platform Blueprint ( #1473 )
...
* Add SA to access to Curated resources
* Add BQ dataset in the landing project
* Provide example to move data from landing to curated using BQ engine
* Improve diagram
2023-06-28 09:05:48 +02:00
638841c8d1
Rename network load balancer modules ( #1466 )
...
* update LB modules to new names
* update LB modules names
* update test paths
2023-06-26 07:50:10 +00:00
45adcf1187
First commit.
2023-06-23 15:52:08 +02:00
15ae95df90
Fixing typos
...
- modules/organization/variables.tf
- modules/organization/README.md
- blueprints/data-solutions/shielded-folder/main.tf
2023-06-23 00:14:02 -04:00
adc1ff3fca
match readme groups with variables file ( #1455 )
...
I faced error during my deployment even though I created data-engineers group. Then I found out that variables file uses gcp-data-engineers as default. The same is valid for data-security.
2023-06-21 11:51:32 +02:00
261ad646a8
Improve Minimal Data Platform blueprint ( #1451 )
2023-06-20 16:47:15 +00:00
dc964411e0
data-platform-minimal - 02-processing.tf typo ( #1454 )
...
error is
```
│ Error: Unsupported attribute
│
│ on ../../../../blueprints/data-solutions/data-platform-minimal/02-processing.tf line 46, in locals:
│ 46: ? var.network_config.subnet_self_links.processingestration
│ ├────────────────
│ │ var.network_config.subnet_self_links is object with 2 attributes
│
│ This object does not have an attribute named "processingestration".
╵
```
I'm guessing this should be `processing_transformation` the other option is `processing_composer`
2023-06-20 15:26:10 +02:00
a37b99501e
fix typo data-platform-minimal ( #1453 )
...
Fix `container-engine` service agent typo.
2023-06-20 13:12:00 +02:00
26e5662e84
Some fixes
2023-06-16 00:01:07 +00:00
luigi-bitonti
lcaggio
Simone Ruffilli
Luca Prete
Julio Castillo
Ludovico Magnocavallo
Oliver Frolovs
Pierre Formont
simonebruzzechesse
Alejandro Leal
Eunan Hardy
Wiktor Niesiobędzki
javiergp
Daniel De Leo
Alejandro Leal
Keith Harvey
Canburak Tümer