Commit Graph

846 Commits

Author SHA1 Message Date
Wiktor Niesiobędzki 86cf5dfc84 Fix reporting non-empty plan after apply 2023-11-17 11:47:44 +00:00
Wiktor Niesiobędzki 6ca688d694 terraform fmt 2023-11-16 11:16:26 +00:00
Wiktor Niesiobędzki 985569fa83 Allow running E2E tests that are skipped in examples 2023-11-15 09:33:03 +00:00
Wiktor Niesiobędzki 1498696b6b JIT service account for storage 2023-11-15 09:33:03 +00:00
Wiktor Niesiobędzki 1fbd018f5f E2E tests for GCS 2023-11-15 09:33:03 +00:00
Wiktor Niesiobędzki 03bf0b15b3
Organization module end-to-end tests (#1860)
* added tag serial to mark tests to be run serially
* always run tests using loadgroup distribution to make use of serial tag
* added end-to-end tests for organization, not adding to custom constraints as the name has to be unique
* fixed granting custom roles created in the same module call
2023-11-14 18:54:59 +01:00
Francesco Spinelli 82c74e4ab6
Dataproc module bug fix (#1848)
* bug fix

* bug fix

* fix dinamic for_each

---------

Co-authored-by: Francesco Spinelli <francesco.spinelli@nttdata.com>
2023-11-09 15:48:29 +00:00
Ludovico Magnocavallo 03937f2b6e
Support multilevel data and allow overriding project id in project factory (#1851) 2023-11-09 08:29:46 +00:00
Wiktor Niesiobędzki ecaa253594
Merge branch 'master' into wiktorn-provider-5.4.0 2023-11-04 08:31:56 +01:00
Wiktor Niesiobędzki aec7fc87c2 Trim down inventories for Cloud SQL 2023-11-04 07:31:33 +00:00
Wiktor Niesiobędzki fe485414e6
Add end-to-end tests for project module (#1823)
* Add end-to-end tests for project module
* Add inventory to data tests
* Add files to end-to-end test cases
* Review fixes - use named groups

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-11-03 18:04:19 +01:00
Ludovico Magnocavallo de0325b3a3
Avoid map-related casting errors in project factory (#1836)
* try to repro pf example error

* repro

* repro

* pf fix

* remove extra file

* FAST stage
2023-11-02 08:24:50 +01:00
cmalpe b8bb000073
Merge branch 'master' into cmalpe/kms-import-job 2023-10-30 20:32:50 +05:30
Luca Prete 7c6726e79b
[net-address] enable ipv6 (#1821)
---------

Co-authored-by: Luca Prete <lucaprete@google.com>
2023-10-28 15:36:30 +02:00
Ludovico Magnocavallo b29987bb61
Merge branch 'master' into cmalpe/kms-import-job 2023-10-28 01:29:28 +02:00
apichick 022b9f5060 Added iam_bindings and iam_bindings_additive to apigee module 2023-10-27 18:22:07 +02:00
Wiktor Niesiobędzki 5b17c11d01
Merge branch 'master' into fix/ai-models-support 2023-10-27 10:22:47 +02:00
cmalpe b0e5231f50 Merge branch 'master' into cmalpe/kms-import-job 2023-10-27 07:21:36 +00:00
Chaitanya Malpe 8a76b10161 added test for import job 2023-10-27 11:53:35 +05:30
Ludovico Magnocavallo d0b1ced280
fix logic for default source range in firewall ingress rules (#1815) 2023-10-26 15:25:36 +00:00
Ewa Wojtach 6fc960ea0b empty gpu sharing config fix 2023-10-26 07:23:13 +02:00
Ewa Wojtach 33ce0e1db5
AI models support (#1750)
* nodepool config

* added gpu driver configuration

* documentation update

* regenerated docs

* review comments

* review comments

* blocks structure

* documentation update

* test fix and doc update

* review comments

* doc

* Extend inventory

* Update README

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-25 09:42:36 +00:00
Luca Prete a23b3d62ae
net-lb-ext: add support for multiple forwarding rules (IPs) and dual-stack (IPv4/IPv6) 2023-10-21 18:19:18 +02:00
Simone Ruffilli 6d89b88149
versions.tf maintenance + copyright notice bump (#1782)
* Bump copyright notice to 2023

* Delete versions.tf on blueprints

* Pin provider to major version 5

* Remove comment

* Fix lint

* fix bq-ml blueprint readme

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-20 18:17:47 +02:00
Wiktor Niesiobędzki d07daf966a
End-to-end tests for terraform modules (#1751)
Add end-to-end tests (apply, plan, destroy) for examples.

When run, `tests/examples_e2e`:
1. Create an environment for tests to run (folder, project vpc network) 
2. For each marked example (with `e2e` tftest directive), run apply, plan, destroy
3. Verify:
* no failure in apply
* empty plan after apply
* no failure during destroy
4. When all tests are done, destroy test environment

More details in `tests/examples_e2e/README.md`
2023-10-20 09:59:52 +02:00
luigi-bitonti 4e439720aa
Added ProtectedApplication feature to GKE Backup (#1774)
* Added ProtectedApplication feature to GKE Backup

* Fixed location name and added example

* Modified test module

* Changed test

* Changed test

* Changed test

* Restore old "all_namespaces" logic

* Bug fix

* Ran fmt on README example

* Modified variable structure

* Fix test

* Fix
2023-10-19 19:54:22 +02:00
Ludovico Magnocavallo e0d84fb10b
add sink for workspace logs (#1780) 2023-10-19 14:51:01 +00:00
Ludovico Magnocavallo e7e188818a
Add service usage consumer role to IaC SAs, refactor delegated grants in FAST (#1773)
* add serviceusage role to iac sas, refactor delegated grants

* fix test

* tfdoc
2023-10-18 12:18:31 +00:00
Ludovico Magnocavallo 02ccc576f5
fix resource manager tag bindings in compute-vm module (#1771) 2023-10-18 09:24:00 +00:00
Wiktor Niesiobędzki c21fa4558f
Remove incompatible balancing_mode (#1769)
## net-lb-int
* Fix error on apply of example:
```
Error creating RegionBackendService: googleapi: Error 400: Invalid value for field 'resource.backends[0].balancingMode': 'UTILIZATION'. Balancing mode must be CONNECTION for an INTERNAL backend service., invalid
```
* remove unused `balancing_mode` variable, as only one value is possible anyhow

## net-lb-ext
* update in the `backends` description

## net-lb-proxy-int
* update in the `backends` description

## net-lb-app-int
* added validation of `balancing_mode`
* fixed other validations

## net-lb-app-ext
* added validation of `balancing_mode`
* fixed other validations
* removed validation for `locality_lb_policy` as this variable is not used in this module

Closes: #1767
2023-10-18 08:11:32 +02:00
Luca Prete 6c48512f7e
[#1764] net-lb-int: add support for dual stack and multiple forwarding rules 2023-10-17 09:30:34 +00:00
jeroenmonteban f464557525
Add autoclass to GCS (#1757)
* Add autoclass to GCS

* Fix linting

* Make autoclass block dynamic

* Fix syntax

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-16 07:45:10 +00:00
Ludovico Magnocavallo 252127bde5
Billing account module (#1743)
* initial untested draft

* readme and tests

* folder module tfdoc

* remove redundant billing cost manager role in fast stage 0

* fix FAST test
2023-10-15 15:02:50 +00:00
Julio Castillo 9ab3b49f69
Add PSA peered domains support to `net-vpc` (#1741)
* Add PSA peered domains support to `net-vpc`

* Fix tests
2023-10-06 15:31:32 +00:00
Julio Castillo dfc5023e0b
Make deletion protection consistent across all modules (#1735)
* Expose deletion_protection in GKE modules

* Make deletion protection consistent across all modules

* Add deletion_protection option to blueprints

* Fix blueprints tests

* Fix types

* Update READMEs

* Fix dp readme

* Fix cmek blueprint default deletion_protection

* Fix blueprints tests
2023-10-05 17:31:07 +02:00
Luca Prete 20dd3df27c
Support multiple protocols (L3_DEFAULT) through net-ilb-int 2023-10-04 08:30:11 +00:00
apichick e7c31cb83a
Merge branch 'master' into gcs-iam 2023-10-04 08:15:00 +02:00
Miren Esnaola c10eb72a8f Changes in IAM to GCS module 2023-10-04 08:13:45 +02:00
Julio Castillo 4b15605711
Fix dnssec keys lookup (#1728)
* Fix dnssec keys lookup

* Fix DNS examples

* Fix FAST and blueprints resource counts
2023-10-03 21:37:21 +02:00
Ludovico Magnocavallo 789328ff5a
Bump provider versions to v5.0.0 (#1724)
* bump provider versions to 5.0.0

* fix cloud run, logging and vpc-sc

* Fix secret manager

* fix gke nodepool

* fix gke multitenant stage and blueprint

* Moving alloydb module to experimental.

* Add project to bare resources in examples

* tfdoc

* fix svpc blueprint test

* Revert "fix svpc blueprint test"

This reverts commit 14f02659098070136e64ead600580dd52c23c339.

* Fix GKE peering project

* Disable tests in alloydb module

* Bring back secret ids in secret manager tests

* Remove duplicate key

* last push

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-03 12:15:36 +00:00
Ludovico Magnocavallo aa5d883deb
add support for org policies to project factory (#1722) 2023-10-02 14:13:56 +00:00
Anmol Sachdeva 06d4bf5788 fix: apigee unit tests 2023-09-29 16:40:48 +00:00
luigi-bitonti be5d9b8fe0
Add bug fix in bucket local variable (#1709)
Fix bucket name reference in Cloud Functions if bucket is created and prefix is used.
2023-09-28 10:17:53 +00:00
Wiktor Niesiobędzki 71def9e6dd Allow partial specification for deeply nested inventories
For objects that are not simple attributes of the resource (i.e.
objects, lists) allow specifing subset of values that needs to be
verified.

In case of lists, count of list objects must match, but if values are
objects / dicts, they can be specified as empty dicts which should pass
all the checks, as no keys are specified.
2023-09-26 11:22:35 +02:00
Wiktor Niesiobędzki 1c2f0c6b14 Tests for secrets for CF 2023-09-26 11:22:35 +02:00
Julio Castillo 1dfa72cadf
Define and adopt standard IP ranges for FAST networking (#1697)
* Define and adopt standard IP ranges for FAST networking

This PR documents and adopts a consistent IP address plan for FAST
networking stages

Fixes #1644

* Fix documented aggregated ranges for FAST

* Fix tests

* Fix ip ranges in documentation

* Fix NVA stages README
2023-09-21 14:27:53 +00:00
Jason Steenblik 79b0dc9751
Add deletion_protection_enabled attribute to cloudsql-instance to enable the GCP native deletion protection flag (#1696) 2023-09-20 15:09:38 +02:00
Ludovico Magnocavallo df5daab6cc
Allow using no service account in compute-vm (#1692)
* module and tests

* align blueprints

* tfdoc

* remove stale variables

* fix blueprint

* variable description
2023-09-19 16:56:51 +00:00
Ludovico Magnocavallo 96c28e605b
fix repd disk attachment in compute-vm module (#1688) 2023-09-18 13:02:28 +00:00
Julio Castillo 9d61c6e26d Update IAM for pubsub topics and subscriptions 2023-09-17 00:21:36 +02:00
Julio Castillo da883bab8c Update kms module key-level IAM 2023-09-17 00:21:36 +02:00
Julio Castillo 4d5df5aeb2 Fix tests 2023-09-15 14:57:32 +02:00
Ludovico Magnocavallo bd7b9b4dd1
Merge branch 'master' into jccb/fix-subnet-iam-bindings 2023-09-15 14:31:28 +02:00
Julio Castillo 616c6fbc48 Fix subnet iam_bindings to use arbitrary keys 2023-09-15 14:30:27 +02:00
Oliver Frolovs 6eb862a775
GKE cluster modules: add optional kube state metrics (#1682)
* `gke-cluster-standard`: add optional kube state metrics
* `gke-cluster-autopilot`: add optional kube state metrics

* FAST: add kube state metrics support for GKE
* blueprints/gke: add kube state metrics support

* Bump up the provider version to `v4.82.0`
2023-09-15 12:18:45 +01:00
Oliver Frolovs b3dc91b5cd
Upgrades to `monitoring_config` in `gke-cluster-*`, docs update, and cosmetics fixes to GKE cluster modules (#1680)
* gke-cluster-standard: upgrade `monitoring_config` to use object style. Add tests.
* gke-cluster-standard: update docs

* gke-cluster-autopilot: move gateway_api_config block (cosmetic change)
* gke-cluster-autopilot: update docs and fix typos

* Update blueprints due to `monitoring_config` changes in `gke-cluster-standard`.
* Update FAST due to `monitoring_config` changes in `gke-cluster-standard`.
* Update docs for affected blueprints and FAST stages
2023-09-14 23:25:57 +01:00
Oliver Frolovs 05c0195a06
GKE Autopilot module: add network tags (#1675)
* gke-cluster-autopilot: add support for network tags

* gke-cluster-autopilot: add validation for network tags

* gke-cluster-autopilot: expand README and fix some typos

* gke-cluster-autopilot: fix Cloud DNS section in README

Removed a reference to Standard clusters and updated the section to include a warning because the new versions of Autopilot clusters can only use Cloud DNS and it is pre-configured by default so the example in the README does not apply to them.

* gke-cluster-autopilot: cosmetic fixes in README

* gke-cluster-autopilot: rollback validation on network tags var

* gke-cluster-autopilot: rollback docs string update for network tags var

* gke-cluster-autopilot: rollback some updates to README

* gke-cluster-autopilot: remove dead code

* gke-cluster-autopilot: add a tftest for network tags

* gke-cluster-autopilot: fix a tftest
2023-09-14 09:34:51 +00:00
apichick 008abe4ddc
Merge branch 'master' into apigee-module-fix 2023-09-13 15:25:51 +02:00
Miren Esnaola fdb133f1de Fixed, added back environments to each instance, that way we can also create instances for already existing environments 2023-09-13 14:43:37 +02:00
Dave Gulli 09e6e8f4d2 created passing test, fixed logic 2023-09-13 16:49:14 +10:00
Dave Gulli f95e3ad9a2 fixed 2023-09-12 17:17:10 +10:00
Dave Gulli 3479c40735 fixing tests 2023-09-12 15:53:28 +10:00
Dave Gulli 7d714f1d15 fixed a few missing things 2023-09-12 15:05:39 +10:00
Dave Gulli 5f2bc7fa92 refactored to recommended method 2023-09-12 14:58:49 +10:00
Dave Gulli db5030e93f adding test 2023-09-11 21:08:55 +10:00
Dave Gulli 8c264da200 small commits to add variables, append tests 2023-09-11 20:14:17 +10:00
Pierre Formont e141b86f79
merge labels from data_merges in project factory (#1662)
* merge labels from data_merges in project factory

* fix project factory test for labels
2023-09-08 10:27:46 +00:00
Ludovico Magnocavallo ec3b705f53
Change type of `iam_bindings` variable to allow multiple conditional bindings (#1658)
* modules

* fast

* dns readme
2023-09-08 08:56:31 +02:00
Julio Castillo a6230d2b11
Merge branch 'master' into jccb/apigee-fixes 2023-09-07 16:34:45 +02:00
simonebruzzechesse 77c1e69666
New phpIPAM serverless third parties solution in blueprints (#1642)
* Added new phpIPAM serverless third parties solution in blueprints
* added jit to iap.googleapis.com service in project module
* updated tests
2023-09-07 15:30:22 +02:00
Luca Prete 12e78af055
Fix project factory blueprint and fast stage (#1654) 2023-09-07 12:48:39 +00:00
g-greatdevaks b98537689c add tests and update readme for apigee psc peering 2023-09-07 16:55:16 +05:30
Julio Castillo 46f4d22c87 Examples and tests 2023-09-07 11:31:40 +02:00
Julio Castillo 24e591f85a Fix tests using provider 4.80 2023-09-05 11:28:12 +02:00
Oliver Frolovs 9f23d504ec
gke-cluster-autopilot: add monitoring configuration (#1646)
* gke-cluster-autopilot: add monitoring configuration block (monitoring_config)
2023-09-04 16:43:59 +01:00
Oliver Frolovs 988fd2ee05
gke-cluster-standard: change logging configuration (#1638)
* Update logging configuration of this module to use object interface in harmony with `gke-cluster-autopilot` module.
* Update blueprints that use this module.
* Add "WORKLOADS" log source to logging configuration of the blueprints where the README files say so.
* Update FAST stage 3 because it uses this module.
2023-08-31 12:49:15 +01:00
Oliver Frolovs 80e85ad343
gke-cluster-autopilot: add logging configuration (#1625)
Although one cannot disable Cloud Logging and Cloud Monitoring integration in GKE Autopilot clusters, one has some flexibility over which control plane components' logs should be ingested. This commit adds a new variable and a block to the cluster resource to facilitate that.

* gke-cluster-autopilot: update logging configuration and add an example to module README
2023-08-31 12:06:57 +01:00
Julio Castillo 3d487c89c3
Merge branch 'master' into jccb/fast-tests 2023-08-29 07:26:43 +02:00
LudovicEmo c558d9b753
Vpc sc allow null for identity type (#1632)
* vpc-sc : allow null for identity_type

* cloudsql-instance : fmt

* vpc-sc: tfdoc

* modules/vpc-sc: update the test result for the identity_type field

* cloudsql-instance: remove empty line
2023-08-29 04:28:57 +02:00
Julio Castillo 804e7c961e Silence FAST tests warnings
- Fix pytest PytestUnraisableExceptionWarning
- Remove incorrect print
- Use tfvars for some examples in READMEs
2023-08-28 18:40:41 +02:00
Luca Prete 50a449965f
Fix: align stage-2-e-nva-bgp to the latest APIs 2023-08-23 13:34:11 +02:00
Ludovico Magnocavallo 95a0c76b18
Merge branch 'master' into ehorning/support-gcs-object-upload 2023-08-22 09:48:32 +02:00
Ludovico Magnocavallo ff8eef6a6f
use cloud run bindings for cf v2 invoker role, refactor iam handling in cf v2 and cloud run (#1609) 2023-08-22 07:23:49 +00:00
Erin Horning 33d51dbee4
Merge branch 'master' into ehorning/support-gcs-object-upload 2023-08-21 16:36:35 -06:00
Ludovico Magnocavallo 5cb4accbd0
Merge branch 'master' into elia-gcve 2023-08-20 10:19:14 +02:00
Ludovico Magnocavallo 819894d2ba
IAM interface refactor (#1595)
* IAM modules refactor proposal

* policy

* subheading

* Update 20230816-iam-refactor.md

* log Julio's +1

* data-catalog-policy-tag

* dataproc

* dataproc

* folder

* folder

* folder

* folder

* project

* better filtering in test examples

* project

* folder

* folder

* organization

* fix variable descriptions

* kms

* net-vpc

* dataplex-datascan

* modules/iam-service-account

* modules/source-repository/

* blueprints/cloud-operations/vm-migration/

* blueprints/third-party-solutions/wordpress

* dataplex-datascan

* blueprints/cloud-operations/workload-identity-federation

* blueprints/data-solutions/cloudsql-multiregion/

* blueprints/data-solutions/composer-2

* Update 20230816-iam-refactor.md

* Update 20230816-iam-refactor.md

* capture discussion in architectural doc

* update variable names and refactor proposal

* project

* blueprints first round

* folder

* organization

* data-catalog-policy-tag

* re-enable folder inventory

* project module style fix

* dataproc

* source-repository

* source-repository tests

* dataplex-datascan

* dataplex-datascan tests

* net-vpc

* net-vpc test examples

* iam-service-account

* iam-service-account test examples

* kms

* boilerplate

* tfdoc

* fix module tests

* more blueprint fixes

* fix typo in data blueprints

* incomplete refactor of data platform foundations

* tfdoc

* data platform foundation

* refactor data platform foundation iam locals

* remove redundant example test

* shielded folder fix

* fix typo

* project factory

* project factory outputs

* tfdoc

* test workflow: less verbose tests, fix tf version

* re-enable -vv, shorter traceback, fix action version

* ignore github extension warning, re-enable action version

* fast bootstrap IAM, untested

* bootstrap stage IAM fixes

* stage 0 tests

* fast stage 1

* tenant stage 1

* minor changes to fast stage 0 and 1

* fast security stage

* fast mt stage 0

* fast mt stage 0

* fast pf
2023-08-20 09:44:20 +02:00
Ludovico Magnocavallo 3a8071d93c
Merge branch 'master' into elia-gcve 2023-08-20 08:18:53 +02:00
eliamaldini e762e93677 fixed variable names 2023-08-18 15:37:32 +02:00
eliamaldini 5a298780c1 changed variable name 2023-08-18 11:19:35 +02:00
eliamaldini 8e9d544be6 fixed description 2023-08-17 12:04:48 +02:00
Ludovico Magnocavallo def2f476d1
Add support for conditions to `iam_members` module variables (#1594)
* project

* data-catalog-policy-tag

* dataproc

* folder

* iam-service-account

* kms

* net-vpc

* organization

* source-repository

* dataplex-datascan
2023-08-15 16:28:23 +02:00
Ludovico Magnocavallo 5689aacac2
Merge branch 'master' into elia-gcve 2023-08-14 11:56:47 +02:00
Ludovico Magnocavallo adf2621727
Add new `iam_members` variable to IAM additive module interfaces (#1589)
* resource management modules

* data catalog policy

* dataproc

* service account

* kms

* net-vpc

* source repository

* dataplex datascan

* service account module variable order
2023-08-14 09:54:50 +00:00
eliamaldini a509756f1b GCVE module first release 2023-08-14 11:48:27 +02:00
Ludovico Magnocavallo 841459a059
Fix factory rules key in net firewall policy module (#1587)
* fix factory rules key in net firewall policy

* fix test
2023-08-14 07:52:36 +02:00
Erin Horning 2fa6369ba0 Merge branch 'master' of https://github.com/GoogleCloudPlatform/cloud-foundation-fabric into ehorning/support-gcs-object-upload 2023-08-11 11:19:21 -06:00
Sam Bentley b1679ad21a
Fix: Instance level stateful disk config (#1578)
* update doco

* fix bug in TF code

* change instance name in README to fix test

* revert disk name

* Update stateful.yaml

* fix examples and tests

---------

Co-authored-by: Julio Castillo <juliocc@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-08-11 15:25:17 +00:00
Ludovico Magnocavallo f2e26b6a2c
Merge branch 'master' into jccb/test-print-failed-inventory 2023-08-11 12:10:31 +02:00
Julio Castillo 4d6574e0a1 Print inventory path when a test fails 2023-08-11 12:07:07 +02:00
Erin Horning df1fded669 add support for object upload to gcs module 2023-08-10 21:43:53 -06:00