Commit Graph

634 Commits

Author SHA1 Message Date
lcaggio 47855cb682
Merge branch 'master' into lcaggio/dp-projectcreate 2023-02-23 11:54:48 +01:00
lcaggio e39be7b01d Fix 2023-02-22 13:02:29 +01:00
lcaggio 2108b4650d Fix Tests, rely on iam additive. 2023-02-22 01:36:01 +01:00
Julio Castillo a5e905cb80 Update remaining org policies 2023-02-21 15:49:16 +01:00
Julio Castillo d3bcf625f9 Update yaml org policies 2023-02-21 15:49:16 +01:00
Julio Castillo 6b767c9035 Simplify org policies data model in resman modules. 2023-02-21 15:49:16 +01:00
Julio Castillo 3085922cea Fix tests 2023-02-20 12:19:36 +01:00
lcaggio f4c1fa6c20 Fix tests. 2023-02-20 00:56:32 +01:00
Aleksandr Averbukh 4ad30b812b Fix tests for tf-cloud-dynamic-credentials blueprint 2023-02-19 01:14:24 +01:00
Aleksandr Averbukh befc73ec12 feat: Update TFC+WIF blueprint with TFC Dynamic Credentials feature. 2023-02-19 00:51:49 +01:00
Wiktor Niesiobędzki e64e8db20d Allow additive IAM grants by robots name
Regreatablly member name will be known after apply, hence changes in the
tests
2023-02-16 14:39:21 +01:00
Julio Castillo d6e3716815 Fix gke-cluster dns config feature
Fixes #1141
2023-02-10 11:14:07 +01:00
Julio Castillo d7b88b7d0e Fix bq factory docs 2023-02-08 18:22:10 +01:00
Ludovico Magnocavallo 8708f490ce
Allow configuring regions from tfvars in FAST networking stages (#1137)
* configurable regions

* vpn, tests

* tfdoc

* separate envs

* nva

* test resources

* add new custom role for tenant network service accounts

* allow setting firewall policy name in networking stages

* fix stage links script

* set custom role to tenant networking service account

* rename tenant stage 1 provider files

* remove extra file

* fix peering and vpn

* tfdoc

* fix variable order

* tests
2023-02-08 09:59:43 +01:00
Julio Castillo bb565b72f1 Add Disk examples 2023-02-07 15:59:29 +01:00
Julio Castillo 1784dd8cad Remove unneeded __init__.py 2023-02-07 15:59:29 +01:00
Julio Castillo 04602353ce New compute-vm examples and tests 2023-02-07 15:59:29 +01:00
lcaggio dd77c2fe79
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-07 07:58:00 +01:00
lcaggio d68bb4b320 Fixes 2023-02-06 21:33:04 +01:00
lcaggio 1e4499c8ab Fixes 2023-02-06 14:58:40 +01:00
Ayman Farhat 02d8d8367a
[Feature] Update data platform blue print with Dataflow Flex template (#1105)
* Add initial dataflow template code + TF infra

* Refactor the datapipeline DAG to use flex template operator, cleanup code

* Remove unneeded bash scripts, update README with manual examples

* Refactor datapipeline_dc_tags.py and include new Flex template

* Update docs to reflect changes

* Remove sub-dependencies and keep apache beam

* Add missing license headers and update tests

* Set resouces to 291 in tests

* Update outputs via tfdoc

* Update with outputs order and tfdoc

* Correct number of resources

* Fix to add region into command from var

* Enable service account impersonation for running builds

* Update example dataflow run command to use orchestrator SA

* Remove hard coded values in example

* Keep original airflow files, add new which use Flex template as example

* Update tests and doc

* Fix number of resources in plan

* Run tfdoc remove files section in README

* Fix number of modules in tfdoc

* Update number of resources

* Add missin service account

* Update DF demo README

* Quick rename

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-02-06 07:35:40 +01:00
lcaggio 39ffdcf184
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-05 22:45:14 +01:00
lcaggio 47c75f8862 Implement project_config variable 2023-02-05 21:12:46 +01:00
lcaggio 38fc9e8b9b First commit 2023-02-05 07:52:33 +01:00
lcaggio ce1cc5f75a
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-04 19:42:19 +01:00
Ludovico Magnocavallo 5453c585e0
FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052)
* rename stages

* remove support for external org billing, rename output files

* resman: make groups optional, align on new billing account variable

* bootstrap: multitenant outputs

* tenant bootstrap stage, untested

* fix folder name

* fix stage 0 output names

* optional creation for tag keys in organization module

* single tenant bootstrap minus tag

* rename output files, add tenant tag key

* fix organization module tag values output

* test skipping creation for tags in organization module

* single tenant bootstrap plan working

* multitenant bootstrap

* tfdoc

* fix check links error messages

* fix links

* tfdoc

* fix links

* rename fast tests, fix bootstrap tests

* multitenant stages have their own folder, simplify stage numbering

* stage renumbering

* wip

* rename tests

* exclude fast providers in fixture

* stage 0 tests

* stage 1 tests

* network stages tests

* stage tests

* tfdoc

* fix links

* tfdoc

* multitenant tests

* remove local files

* stage links command

* fix links script, TODO

* wip

* wip single tenant bootstrap

* working tenant bootstrap

* update gitignore

* remove local files

* tfdoc

* remove local files

* allow tests for tenant bootstrap stage

* tenant bootstrap proxies stage 1 tfvars

* stage 2 and 3 service accounts and IAM in tenant bootstrap

* wip

* wip

* wip

* drop multitenant bootstrap

* tfdoc

* add missing stage 2 SAs, fix org-level IAM condition

* wip

* wip

* optional tag value creation in organization module

* stage 1 working

* linting

* linting

* READMEs

* wip

* Make stage-links script work in old macos bash

* stage links command help

* fix output file names

* diagrams

* fix svg

* stage 0 skeleton and diagram

* test svg

* test svg

* test diagram

* diagram

* readme

* fix stage links script

* stage 0 readme

* README changes

* stage readmes

* fix outputs order

* fix link

* fix tests

* stage 1 test

* skip stage example

* boilerplate

* fix tftest skip

* default bootstrap stage log sinks to log buckets

* add logging to tenant bootstrap

* move iam variables out of tenant config

* fix cicd, reintroduce missing variable

* use optional in stage 1 cicd variable

* rename extras stage

* rename and move identity providers local, use optional for cicd variable

* tfdoc

* add support for wif pool and providers, ci/cd

* tfdoc

* fix links

* better handling of modules repository

* add missing role on logging project

* fix cicd pools in locals, test cicd

* fix workflow extension

* fix module source replacement

* allow tenant bootstrap cicd sa to impersonate resman sa

* tenant workflow templates fix for no providers file

* fix output files, push github workflow template to new repository

* remove try from outpout files

* align stage 1 cicd internals to stage 0

* tfdoc

* tests

* fix tests

* tests

* improve variable descriptions

* use optional in fast features

* actually create tenant log sinks, and allow the resman sa to do it

* test

* tests

* aaaand tests again

* fast features tenant override

* fast features tenant override

* fix wording

* add missing comment

* configure pf service accounts

* add missing comment

* tfdoc

* tests

* IAM docs

* update copyright

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-02-04 15:00:45 +01:00
lcaggio f00d4d3edd
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-03 22:17:30 +01:00
javiergp ce1f86d20b
Vertex Pipelines MLOps framework blueprint (#1038)
* First release of the MLOps blueprint
2023-02-02 19:13:13 +01:00
lcaggio f9e3eb7d5a Update copyright 2023-02-01 09:30:28 +01:00
Julio Castillo 351698873b Fix test names and update to yaml-based tests 2023-02-01 09:03:53 +01:00
lcaggio 840fc86b3e Fix groups. Fix tests. 2023-02-01 08:55:33 +01:00
lcaggio 3995ec1550
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-01 00:28:01 +01:00
lcaggio f36b004664 Make VPCSC optional; ingress policy 2023-02-01 00:26:57 +01:00
lcaggio fb04e78829 Fix variables 2023-01-31 22:35:48 +01:00
Julio Castillo 1204b0077e
Merge branch 'master' into mci_gateway 2023-01-31 11:54:31 +00:00
Miren Esnaola 1a2745b632 In the apigee module now both the /22 and /28 peering IP ranges are passed at instance creation 2023-01-31 11:32:53 +01:00
lcaggio 2584eb149c Fix test 2023-01-31 00:02:30 +01:00
lcaggio a166c612d5 Merge branch 'master' into lcaggio/shieldedfolder 2023-01-30 23:52:46 +01:00
lcaggio 66a10d76c3 Add test 2023-01-30 23:47:33 +01:00
Wiktor Niesiobędzki db8382fc02 Fix tests after version bump 2023-01-29 15:50:24 +01:00
Wiktor Niesiobędzki 2976df596e Fix tests (add JIT-ed SA for multiclusteringress.googleapis.com) 2023-01-29 15:50:24 +01:00
Ayman Farhat edd3a82453
Include cloudbuild API in project module (#1116)
* Include cloudbuild API in project module

* Increase number of resources
2023-01-27 21:38:01 +01:00
fdhaussy a291dca63b chore: linting issue in tfvars file 2023-01-27 16:22:34 +01:00
fdhaussy bed7e05d09 add support for deployment_type and api_proxy_type in google_apigee_environment resource 2023-01-27 15:37:57 +01:00
Julio Castillo d537897cbf Remove trailing whitespaces 2023-01-26 17:08:20 +00:00
Julio Diez 4d6561712e Fix tftest and format 2023-01-26 17:56:20 +01:00
Julio Castillo 551d45760f
Merge branch 'master' into google_gkehub_feature_membership 2023-01-23 09:11:49 +01:00
apichick 6a633390b2
Merge branch 'master' into hybrid-gke-updates 2023-01-20 13:18:19 +01:00
Ludovico Magnocavallo 050daa65f2
Merge branch 'master' into hybrid-gke-updates 2023-01-20 07:48:49 +01:00
Miren Esnaola 8945165bc3 Improvements in apigee hybrid-gke: now using workload identity and GLB 2023-01-19 22:11:44 +01:00
Julio Castillo 13352779ac Fix nodepool test 2023-01-19 18:55:30 +01:00
Julio Castillo 1820269680 Add inventories to gke-nodepool examples 2023-01-19 18:19:34 +01:00
Julio Castillo 9c9aafb3f1 Update gke-cluster tests 2023-01-19 17:47:14 +01:00
Julio Castillo 1e0d7776e1 Update DNS tests 2023-01-19 17:47:14 +01:00
Julio Castillo a12089ef8c Move VPC factory and route tests to examples. 2023-01-19 17:47:14 +01:00
Julio Castillo 2aad7845a4 Allow dashes and underscores in tftest file ids 2023-01-19 17:47:14 +01:00
Julio Castillo a0cb67e1f4 Add inventories to gcs examples 2023-01-19 17:47:14 +01:00
Julio Castillo 12f07ebeac Extend net-vpc README with more tested examples 2023-01-19 17:47:14 +01:00
Julio Castillo fd19e4a923 add inventories net-vpc examples 2023-01-19 17:47:14 +01:00
Julio Castillo 4897aa7109 bump test suite versions 2023-01-19 17:47:14 +01:00
Julio Castillo de704110c9 Update api-gateway tests 2023-01-19 17:47:14 +01:00
Wiktor Niesiobędzki b38ef22572 Even more test fixes 2023-01-19 09:37:48 +01:00
Wiktor Niesiobędzki 10e462d594 Fix tests for servicemesh 2023-01-19 09:37:48 +01:00
lcaggio 07a7be29e3 Add shared-vpc support on Data Playgroud blueprint 2023-01-19 00:33:31 +01:00
Miren Esnaola 7d9dc6c28e Added tags to gke-cluster module 2023-01-13 12:58:40 +01:00
lcaggio 488b6d4dc0 Remove plg project, composer 2 2023-01-11 00:31:52 +01:00
Ludovico Magnocavallo f7350be743
Merge branch 'master' into apigee-hybrid-gke 2023-01-05 09:11:39 +01:00
Miren Esnaola f6413747b7 Fixes in Apigee Bigquery Analytics blueprint 2023-01-04 17:30:07 +01:00
apichick 9774e866e3
Merge branch 'master' into apigee-hybrid-gke 2023-01-04 11:47:40 +01:00
Miren Esnaola 33f5607ff1 Apigee hybrid on GKE 2023-01-04 10:53:49 +01:00
Taneli Leppä fbf42197a0
Merge branch 'master' into stateful-disks-deletion-rule-fix 2023-01-04 08:39:43 +01:00
Taneli Leppä ca4cb3b3b9 Fixed delete_rule in compute-mig module for stateful disks. 2023-01-03 21:53:31 +01:00
Julio Castillo 2e35bff903 Fix tests for latest provider version 2023-01-03 16:52:31 +01:00
Julio Castillo 1114c6a8a4 Fix org module tests 2023-01-03 16:52:31 +01:00
Julio Castillo c8bfe892a6 Update folder tests 2023-01-03 16:52:31 +01:00
Julio Castillo 065b1471a8 Reorder org policy rules 2023-01-03 16:52:31 +01:00
Israel Herraiz 20579394b8 Add cluster replicas to Bigtable module.
This adds the possiblity to define several clusters in a Bigtable instance,
which enables replication in Bigtable.

Some of the configurations options are moved inside a `map` that defines the
properties of each cluster.

These changes alter the interface of the module, so any previous code using this
module will have to adapt to the new options interface.
2022-12-28 19:53:17 +01:00
Miren Esnaola 0235690aaa Moved apigee bigquery analytics blueprint, added apigee network patterns 2022-12-23 14:38:15 +01:00
Ludovico Magnocavallo 66a402083b
Allow setting no ranges in firewall module custom rules (#1073)
* allow setting no ranges in custom firewall rules

* fix blueprint

* fix example

* fix example
2022-12-23 09:03:31 +01:00
Israel Herraiz 5fc5165adf Fix modules tests after version bump 2022-12-23 00:33:53 +01:00
Israel Herraiz f0395f9a8d
Merge branch 'master' into fix_column_families 2022-12-22 17:22:12 +01:00
Israel Herraiz 8427bd3ba6 Fix linting 2022-12-22 16:50:37 +01:00
Israel Herraiz 9152d5a0f5 Fix default values for test 2022-12-22 16:49:05 +01:00
Ludovico Magnocavallo 082c63dfc5
refactor GCS module and tests (#1066) 2022-12-22 12:27:09 +01:00
Ludovico Magnocavallo b438e0c37a
Allow using dynamically generated address in LB modules NEGs (#1061)
* test

* fix #1055 for net-ilb-l7

* fix glb module
2022-12-21 17:04:56 +01:00
Ludo 0d7243979b
revert last commit 2022-12-21 13:12:15 +01:00
Ludo 7c95b7ce43
fix #1055 for net-ilb-l7 2022-12-21 13:07:36 +01:00
Ludo aaf5a9013b
fix test 2022-12-20 07:55:03 +01:00
Julio Castillo 4f7cb39a06 Simplify readme discovery 2022-12-18 14:00:20 +01:00
Julio Castillo df03f14103 Fix SA tests 2022-12-18 14:00:20 +01:00
Julio Castillo 09ad53000e Remove recursive_e2e_plan_runner 2022-12-18 14:00:20 +01:00
Julio Castillo b2232e4f00 Remove service account fixture 2022-12-18 14:00:20 +01:00
Julio Castillo 126227502e Fix tests paths for examples 2022-12-18 14:00:20 +01:00
Julio Castillo 4b77e484b1 Migrate project module to new tests 2022-12-18 14:00:19 +01:00
Julio Castillo e700a27079 Enforce terraform fmt in examples 2022-12-18 14:00:19 +01:00
Julio Castillo 0faf8ae1f1 Fix net-vpc tests 2022-12-18 14:00:19 +01:00
Julio Castillo 8598816980 Remove organization fixture 2022-12-18 14:00:19 +01:00
Julio Castillo 4e95645e69 Silence linting 2022-12-18 14:00:19 +01:00
Julio Castillo 8accf2ed91 Fix typo 2022-12-18 14:00:19 +01:00
Julio Castillo 513fdfe91a Make directive for supporting files in examples more explicit 2022-12-18 14:00:19 +01:00