c8f25512eb
Fix variabler to support existing projects.
2023-02-20 00:39:37 +01:00
ad6667a8fa
First commit
2023-02-19 22:37:32 +01:00
eac2065ae2
Update on docs wording
2023-02-16 17:04:05 +01:00
fcdadf521d
Fix spelling mistake
2023-02-16 15:48:12 +01:00
a853dc4fe2
update Data Platform blueprint README with more example Dataflow commands
2023-02-16 15:45:20 +01:00
ebc4bc51a5
Workaround to mitigate issue 9164
2023-02-13 15:25:24 +01:00
08f2540872
Bump provider version
2023-02-07 15:59:29 +01:00
d78815d3d7
Fix README
2023-02-07 13:57:26 +01:00
71469679e3
Add variables validation
2023-02-07 13:53:54 +01:00
dd77c2fe79
Merge branch 'master' into lcaggio/shieldedfolder
2023-02-07 07:58:00 +01:00
307cf32cf9
Fix
2023-02-06 21:34:29 +01:00
d68bb4b320
Fixes
2023-02-06 21:33:04 +01:00
1e4499c8ab
Fixes
2023-02-06 14:58:40 +01:00
02d8d8367a
[Feature] Update data platform blue print with Dataflow Flex template ( #1105 )
...
* Add initial dataflow template code + TF infra
* Refactor the datapipeline DAG to use flex template operator, cleanup code
* Remove unneeded bash scripts, update README with manual examples
* Refactor datapipeline_dc_tags.py and include new Flex template
* Update docs to reflect changes
* Remove sub-dependencies and keep apache beam
* Add missing license headers and update tests
* Set resouces to 291 in tests
* Update outputs via tfdoc
* Update with outputs order and tfdoc
* Correct number of resources
* Fix to add region into command from var
* Enable service account impersonation for running builds
* Update example dataflow run command to use orchestrator SA
* Remove hard coded values in example
* Keep original airflow files, add new which use Flex template as example
* Update tests and doc
* Fix number of resources in plan
* Run tfdoc remove files section in README
* Fix number of modules in tfdoc
* Update number of resources
* Add missin service account
* Update DF demo README
* Quick rename
---------
Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-02-06 07:35:40 +01:00
39ffdcf184
Merge branch 'master' into lcaggio/shieldedfolder
2023-02-05 22:45:14 +01:00
f49e5ccec7
Fix readmes and variables.
2023-02-05 22:44:31 +01:00
47c75f8862
Implement project_config variable
2023-02-05 21:12:46 +01:00
38fc9e8b9b
First commit
2023-02-05 07:52:33 +01:00
ce1cc5f75a
Merge branch 'master' into lcaggio/shieldedfolder
2023-02-04 19:42:19 +01:00
5453c585e0
FAST multitenant bootstrap and resource management, rename org-level FAST stages ( #1052 )
...
* rename stages
* remove support for external org billing, rename output files
* resman: make groups optional, align on new billing account variable
* bootstrap: multitenant outputs
* tenant bootstrap stage, untested
* fix folder name
* fix stage 0 output names
* optional creation for tag keys in organization module
* single tenant bootstrap minus tag
* rename output files, add tenant tag key
* fix organization module tag values output
* test skipping creation for tags in organization module
* single tenant bootstrap plan working
* multitenant bootstrap
* tfdoc
* fix check links error messages
* fix links
* tfdoc
* fix links
* rename fast tests, fix bootstrap tests
* multitenant stages have their own folder, simplify stage numbering
* stage renumbering
* wip
* rename tests
* exclude fast providers in fixture
* stage 0 tests
* stage 1 tests
* network stages tests
* stage tests
* tfdoc
* fix links
* tfdoc
* multitenant tests
* remove local files
* stage links command
* fix links script, TODO
* wip
* wip single tenant bootstrap
* working tenant bootstrap
* update gitignore
* remove local files
* tfdoc
* remove local files
* allow tests for tenant bootstrap stage
* tenant bootstrap proxies stage 1 tfvars
* stage 2 and 3 service accounts and IAM in tenant bootstrap
* wip
* wip
* wip
* drop multitenant bootstrap
* tfdoc
* add missing stage 2 SAs, fix org-level IAM condition
* wip
* wip
* optional tag value creation in organization module
* stage 1 working
* linting
* linting
* READMEs
* wip
* Make stage-links script work in old macos bash
* stage links command help
* fix output file names
* diagrams
* fix svg
* stage 0 skeleton and diagram
* test svg
* test svg
* test diagram
* diagram
* readme
* fix stage links script
* stage 0 readme
* README changes
* stage readmes
* fix outputs order
* fix link
* fix tests
* stage 1 test
* skip stage example
* boilerplate
* fix tftest skip
* default bootstrap stage log sinks to log buckets
* add logging to tenant bootstrap
* move iam variables out of tenant config
* fix cicd, reintroduce missing variable
* use optional in stage 1 cicd variable
* rename extras stage
* rename and move identity providers local, use optional for cicd variable
* tfdoc
* add support for wif pool and providers, ci/cd
* tfdoc
* fix links
* better handling of modules repository
* add missing role on logging project
* fix cicd pools in locals, test cicd
* fix workflow extension
* fix module source replacement
* allow tenant bootstrap cicd sa to impersonate resman sa
* tenant workflow templates fix for no providers file
* fix output files, push github workflow template to new repository
* remove try from outpout files
* align stage 1 cicd internals to stage 0
* tfdoc
* tests
* fix tests
* tests
* improve variable descriptions
* use optional in fast features
* actually create tenant log sinks, and allow the resman sa to do it
* test
* tests
* aaaand tests again
* fast features tenant override
* fast features tenant override
* fix wording
* add missing comment
* configure pf service accounts
* add missing comment
* tfdoc
* tests
* IAM docs
* update copyright
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2023-02-04 15:00:45 +01:00
45a74f37f9
Mention policies in the costomization section.
2023-02-04 06:24:35 +01:00
f00d4d3edd
Merge branch 'master' into lcaggio/shieldedfolder
2023-02-03 22:17:30 +01:00
ce1f86d20b
Vertex Pipelines MLOps framework blueprint ( #1038 )
...
* First release of the MLOps blueprint
2023-02-02 19:13:13 +01:00
30f2b49c9c
README Typos
2023-02-01 17:34:31 +01:00
e3daaf2503
Update READMEs
2023-02-01 13:02:07 +01:00
f9e3eb7d5a
Update copyright
2023-02-01 09:30:28 +01:00
840fc86b3e
Fix groups. Fix tests.
2023-02-01 08:55:33 +01:00
053d4151de
Chane feature name, improve README
2023-02-01 00:50:22 +01:00
3995ec1550
Merge branch 'master' into lcaggio/shieldedfolder
2023-02-01 00:28:01 +01:00
f36b004664
Make VPCSC optional; ingress policy
2023-02-01 00:26:57 +01:00
fb04e78829
Fix variables
2023-01-31 22:35:48 +01:00
0e5f2f71a2
Fix Linting
2023-01-30 23:57:20 +01:00
a166c612d5
Merge branch 'master' into lcaggio/shieldedfolder
2023-01-30 23:52:46 +01:00
6a39d80487
Fix README
2023-01-30 23:49:47 +01:00
66a10d76c3
Add test
2023-01-30 23:47:33 +01:00
a808ea3293
Draft README
2023-01-30 23:32:13 +01:00
1a657b31d3
Bump beta provider to 4.48
...
This is the first version that supports `gateway_api_config` block
2023-01-29 15:50:24 +01:00
edd3a82453
Include cloudbuild API in project module ( #1116 )
...
* Include cloudbuild API in project module
* Increase number of resources
2023-01-27 21:38:01 +01:00
9a46f06dd0
update readme
2023-01-27 10:05:19 +01:00
1189e38788
Fix
2023-01-25 18:30:21 +01:00
0ce110f170
Make KMS and Log sink optionals
2023-01-25 18:22:43 +01:00
440e9c59b9
Update
2023-01-25 12:32:10 +01:00
4007d42705
Add KMS and Log.
2023-01-21 01:08:51 +01:00
dcbfdd9c91
Merge branch 'master' into lcaggio/shieldedfolder
2023-01-19 10:59:33 +01:00
3c88aace1a
Fix dataflow shared-vpc support
2023-01-19 01:04:11 +01:00
6477d0790d
Remove temp file.
2023-01-19 00:39:55 +01:00
07a7be29e3
Add shared-vpc support on Data Playgroud blueprint
2023-01-19 00:33:31 +01:00
b9a4c27a03
VPC-SC
2023-01-18 15:50:49 +01:00
191e56fa80
Merge branch 'master' into lcaggio/shieldedfolder
2023-01-17 15:51:55 +01:00
84be665172
First commit.
2023-01-17 08:49:04 +01:00
02938bdfa1
Fix Table in readme.
2023-01-16 15:58:11 +01:00
1672cfe9c1
Fix README.
2023-01-12 12:45:39 +01:00
50a52ae01a
Implement PR comments.
2023-01-12 12:41:00 +01:00
07a975f84e
Fix example test.
2023-01-11 12:37:08 +01:00
72abd47520
Fix documentation.
2023-01-11 08:47:42 +01:00
c172c7bb2e
Remove plg project and update to composer 2
2023-01-11 00:31:58 +01:00
488b6d4dc0
Remove plg project, composer 2
2023-01-11 00:31:52 +01:00
66a402083b
Allow setting no ranges in firewall module custom rules ( #1073 )
...
* allow setting no ranges in custom firewall rules
* fix blueprint
* fix example
* fix example
2022-12-23 09:03:31 +01:00
7df527e1ef
Bump versions EVERYWHERE
2022-12-23 00:06:42 +01:00
e56762b4d3
Merge branch 'master' into lcaggio/fix-20221123
2022-11-25 14:54:47 +01:00
612d7283e8
Enforce nonempty descriptions ending in a colon
2022-11-24 18:56:01 +01:00
d1cc0519d8
Merge branch 'master' into lcaggio/fix-20221123
2022-11-24 16:07:33 +01:00
6b01b8641c
Fix encryption.
2022-11-24 15:25:29 +01:00
e4fc47a063
Normalize prefix handling in blueprints ( #1003 )
2022-11-23 11:09:00 +01:00
826ff14ef9
Sort variables and outputs
2022-11-21 13:17:55 +01:00
7be3288cc7
Add tests for factory examples
2022-11-18 17:48:52 +01:00
fae5654e33
Refactor VPC firewall module for Terraform 1.3 ( #949 )
...
* module and module tests/examples
* align blueprints and fast
* fix null ranges
* make ports optional
* tfdoc
* make rules optional defaulting to all protocols
* review comments
* last round of comments
* invert precedence of template variables
* add option to disable all default rules
* add option to disable all default rules
* split egress/ingress
* tests
* fix tests
2022-11-04 13:56:07 +01:00
0563260678
Refactor ILB module for Terraform 1.3 ( #941 )
...
* wip
* wip
* wip
* backport health check to compute-mig
* align blueprints and fast
* fix blueprints
* tfdoc
2022-11-02 18:05:20 +01:00
8c51a2a386
Enable org policy service and add README notice to modules ( #936 )
...
* enable org policy service and add README notice to modules
* fix tests
2022-11-01 14:25:07 +01:00
ad420138ad
Fix readme examples
2022-10-28 17:49:44 +02:00
b021d84633
Fix formatting for gcloud dataflow job launch command ( #924 )
...
This fix is to prevent errors similar to "ERROR: (gcloud.dataflow.jobs.run) unrecognized arguments: df-loading@tf-pso-workshop-test.iam.gserviceaccount.com" when copy pasting the example code. Line 8 misses a space between the statement and line break, leading to missing a space between parameters when evaluated by the gcloud command.
2022-10-27 14:07:24 +00:00
ed9fd6b08d
Align documentation, move glb blueprint ( #921 )
...
* aling documentation, move glb blueprint
* modules README, sort modules
* reorder modules
* fix bp links
* fix moved blueprint test
* use a single cloud shell image in the repo
2022-10-26 14:31:04 +02:00
1a3bb25917
Update provider version (needed for dns logging support).
2022-10-25 12:15:02 +02:00
13ae26deb2
Merge branch 'master' into lcaggio/fix-pipeline
2022-10-25 09:08:30 +02:00
ef29b3ba12
Fix template path.
2022-10-25 09:06:58 +02:00
7561f7db69
Remove data platform file wrongly committed.
2022-10-25 07:42:05 +02:00
aa6cba24a3
Organize png and tpl in subfolders
2022-10-25 07:39:29 +02:00
c61313b37c
Move images and templates to subfolder.
2022-10-25 07:39:16 +02:00
f9795a38d0
Fix DataPlatform file wrongly submitted.
2022-10-25 06:25:36 +02:00
35f39369c5
Merge branch 'master' into lcaggio/dp-dataaccess
2022-10-24 23:00:47 +02:00
9c985fbbb5
Fix regexp
2022-10-24 22:59:53 +02:00
529dde5a46
Fix CloudSQL blueprint
2022-10-24 22:59:35 +02:00
a988775c14
Do not modify in parameters in format_* functions
2022-10-20 18:56:54 +02:00
4fa1dc431d
Refactor net-vpc module for Terraform 1.3 ( #880 )
...
* module tests pass
* doc examples
* refactor blueprints
* fast stages
* fix comment typo
* fix module factory test
2022-10-14 11:02:33 +02:00
ce148d68f4
Update roles
2022-10-13 06:36:20 +02:00
e8056577ce
Refactor GKE nodepool and blueprints ( #875 )
...
* first shot, untested
* example tests working
* module tests
* work on gke blueprints
* multitenant fleet doc examples
* fix gke hub doc examples
* blueprint tests
* move master range to vpc config
* fast stage 3 gke test
* tfdoc
* bump provider versions
* and bump provider again to latest
2022-10-12 12:59:36 +02:00
9c2b850a0a
Update README.md
2022-10-11 15:22:46 +02:00
6abb7ae0bc
Update README.md
2022-10-11 15:21:30 +02:00
d8f0f7040f
Revert "Fix cloud shell links"
...
This reverts commit 75a6dab413
2022-10-11 15:16:38 +02:00
75a6dab413
Fix cloud shell links
2022-10-11 14:53:15 +02:00
5dbc61cb8c
temporary revert to Terraform 1.3.1 to support Cloud Shell
2022-10-10 09:19:55 +00:00
a0171b2c49
Bump terraform required version ( #864 )
...
* bump terraform required version
* fix test
* debug test
* debug test
* disable test
* fix CI file, bump terraform action version
2022-10-07 12:51:56 +02:00
894647ff9e
Leverage new shared VPC project config defaults across the repo
2022-10-07 08:55:47 +02:00
f7ef127149
first commit.
2022-09-28 15:14:36 +02:00
ac835b6d50
Update terraform version
2022-09-28 11:28:05 +02:00
3a2b2d443b
Remove env_variables in the default value.
2022-09-27 15:32:30 +02:00
a2598991fc
Implements review comments.
2022-09-27 12:44:54 +02:00
78026a4d5a
Improve composer config variable.
2022-09-21 15:00:52 +02:00
3937e46da9
Merge branch 'master' into lcaggio/composer-2
2022-09-20 17:26:47 +02:00
ee309ecc06
Update firewall rules.
2022-09-20 16:57:35 +02:00
0704a0ad30
Fixes based on comments.
2022-09-20 12:26:57 +02:00
a88447d10f
Merge branch 'master' into lcaggio/composer-2
2022-09-20 11:49:20 +02:00
f28dc5cb04
Fix tests
2022-09-20 10:51:11 +02:00
6c42fa3504
Add articatregistry service
2022-09-20 10:35:06 +02:00
df8bcc1bb2
Fix resource dependencies.
2022-09-19 12:00:42 +02:00
5968661e8d
First commit
2022-09-19 10:34:46 +02:00
7e9173d35d
Create fully recursive e2e test runner
2022-09-12 13:58:11 +02:00
0ee32be2fa
More updates 2
2022-09-09 16:40:37 +02:00
da2f601c9a
More updates
2022-09-09 16:39:01 +02:00
269894e92b
Update all internal links examples -> blueprints
2022-09-09 16:39:01 +02:00
a248602827
Rename examples folder to blueprints
2022-09-09 16:38:43 +02:00
lcaggio
Ayman Farhat
Julio Castillo
Ayman Farhat
Ludovico Magnocavallo
javiergp
Wiktor Niesiobędzki
Israel Herraiz
Sebastian Kunze
Natalia Strelkova