Commit Graph

876 Commits

Author SHA1 Message Date
Thinh Ha 82f11e7ca5
add analytics hub module (#2087) 2024-02-19 15:55:00 +00:00
Ludovico Magnocavallo eb23bb62d2
Support domainless orgs in FAST (#2086)
* bootstrap

* align org policies to domainless enforced ones

* fix #2073

* fix tests

* fix team admin attribute in resman stage
2024-02-19 08:29:37 +00:00
Wiktor Niesiobędzki bee3072568
Add support for Cloud Run v2 jobs (#1954)
Add support for Cloud Run v2 jobs

* create a separate file for service creation (service.tf) and job
  (job.tf) - for easy comparison
* add E2E tests where possibile
* remove default value for input variable `region`
* fix subnet range VPC Access Connector example
* add creation of service account for audit logs call (trigger requires
  service account)
* use provided trigger service account email in
  `local.trigger_sa_email`, so explicitly provided SA is passed to
  trigger
* set default value for vpc_connector_create.throughput.max, to match
  what is set by GCP API, as provider uses wrong default of 300 which
  results in perma-diff
* create inventory fiels for all examples

Global changes
* (tests) add input variable `project_number`, to allow assigning IAM permissions to Service Accounts in fixtures
* (tests) fix not outputting the path, when object is not found in inventory
* (tests) fix `create_e2e_sandbox.sh` - now it properly finds root of the repo

Secret Manager
* added `version_versions` output, to allow specifying versions in other modules. `versions` is sensitive and it makes it unsuitable for `for_each` values

New test fixtures
* `pubsub.tf` - creating one topic
* `secret-credential.tf` - creating Secret Manager `credential` secret
* `shared-vpc.tf` - creating two projects (host and service), and vpc in host project
* `vpc-connector.tf` - creating VPC Access Connector instance
2024-02-18 14:57:34 +01:00
Ludovico Magnocavallo 91615e0140
VPC-SC module factories (#2081)
* factory untested

* factory example test
2024-02-17 08:02:16 +01:00
simonebruzzechesse 946ae148f7
Add workforce_identity_federation in 0-bootstrap (#2077)
* add workforce_identity_federation in 0-bootstrap
* update tests
2024-02-15 00:10:24 +01:00
Wiktor Niesiobędzki 72183be254 Use less conflicting IP 2024-02-13 07:40:31 +01:00
Ludovico Magnocavallo 71a64487d5
Extend FAST to support different principal types (#2064)
* add doc draft

* typos

* typo

* typo

* typos

* rewording

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* move iam variables to a separate file

* move billing-account module to iam_principals

* move data-catalog-policy-tag module to iam_principals

* move dataplex-datascan module to iam_principals

* move dataproc module to iam_principals

* move folder module to iam_principals

* copyright

* move organization module to iam_principals

* move project module to iam_principals

* move source-repository module to iam_principals

* update blueprints for iam_principals interface

* FAST bootstrap

* module READMEs fixes

* FAST bootstrap

* FAST networking stages

* FAST security stage

* FAST gke stage

* FAST multitenant bootstrap stage

* FAST multitenant resman stage

* tfdoc

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* fix module test

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Rename iam_principals to iam_by_principals

* Update IAM template to include iam_by_principals

* Update Resman README

* Fix ADR link format

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2024-02-12 14:35:30 +01:00
Wiktor Niesiobędzki c58e61e98e Introduce variable to disable imports, fix tests 2024-02-07 17:25:11 +01:00
Wiktor Niesiobędzki 2c9eb5795b Bum terraform to version 1.7.0 2024-02-07 17:25:11 +01:00
Julio Castillo e219d92217
Enable additional recommended org policies (#2050)
* Enable additional recommended org policies

Fixes #2047
Fixes #2048
Fixes #2049

* Fix tests
2024-02-05 10:46:37 +01:00
Ludovico Magnocavallo 5448ab64c4
Leverage net-vpc module for DNS logging in FAST (#2041)
* revert #2023

* leverage net vpc module for dns logging in fast
2024-02-03 08:16:00 +01:00
Julio Castillo 13636ba07b
Make Cloud NAT creation optional in FAST net stages. (#2038)
* Make Cloud NAT creation optional in FAST net stages.

Fixes #2021

* Update READMEs
2024-02-02 10:58:16 +01:00
Julio Castillo 4c68c016a9
Add DNS query logging to FAST net stages (#2033)
* Add DNS query logging to FAST net stages

Fixes #2020

* Update readmes

* Add variable to toggle DNS logging

* Extend DNS logging toggle to other net stages
2024-01-31 13:44:51 +01:00
Ludovico Magnocavallo 01c7f806ce
Selectively enable logging in FAST and firewall policy module rules (#2032)
* use logging in firewall policy module examples

* enable logging for selected hierarchical firewall rules
2024-01-31 09:50:35 +01:00
Julio Castillo da95434308
logging for default ingress rules in FAST (#2030)
* Add default ingress deny rule with logging to FAST net stages.

Fixes #2024

* Allow firewall factory to omit rules key

* Fix tests

* Fix fast tests

* fix fast tests
2024-01-30 16:53:01 +00:00
Ludovico Magnocavallo bf93b6fb4e
fix typo in logging sinks interface (#2015) 2024-01-28 10:27:28 +01:00
Ludovico Magnocavallo 99228363b2
enforce trusted image projects constraint in stage 0 (#2014) 2024-01-26 10:14:44 +00:00
Ludovico Magnocavallo 6d9b6403dd
add support for essential contacts to FAST (#2010) 2024-01-25 12:20:14 +01:00
Ludovico Magnocavallo c5416f3af1
Tighten up security of automation project (CSPR-related) (#2009)
* enforce compute/iam policies on the automation project

* tests
2024-01-24 18:40:36 +00:00
Wiktor Niesiobędzki 526185fd1f
Remove default region for Cloud Function and Cloud Run (#2004)
Remove default region for Cloud Function and Cloud Run
2024-01-24 10:23:40 +00:00
Wiktor Niesiobędzki 277777d1c7
Fix DNS E2E test + add one to net-lb-app-int-cross-region (#1993)
* Fix DNS E2E test + add one to net-lb-app-int-cross-region

* Update README.md

* Fix inventory for tests

* Fix tests

* Fix number of resources

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2024-01-23 16:34:45 +01:00
Ludovico Magnocavallo 070584ae74
Checklist attribution bucket (#2000) 2024-01-23 11:32:14 +00:00
Ludovico Magnocavallo 4b911a6047
update checklist parsing for top-level key (#1997) 2024-01-23 07:34:03 +01:00
Ludovico Magnocavallo 37dc48bca4
fix factory type for firewall rule ports (#1996) 2024-01-21 12:38:24 +01:00
dibaskar-google 4ed738688a
dns reponse policy e2e changes (#1994)
dns reponse policy e2e changes
2024-01-20 19:47:02 +01:00
Ludovico Magnocavallo 11d7edac64
Add example to FAST GKE stage, streamline GKE Hub module variables and usage (#1977)
* implement optionals in gke-hub module

* simplify gke hub module call in mc mesh blueprint

* simplify gke hub module call and variables in multitenant blueprint

* gke hub inventory

* provide cluster and fleet examples in stage
2024-01-20 10:06:38 +00:00
apichick 228a4a82c3 Fixed README and test for dns module 2024-01-18 19:04:03 +01:00
apichick 4bf394a916 Added health checked targets for geo routing policy in dns module
Added health checked targets for geo routing policy in dns module
2024-01-18 16:39:56 +01:00
Ludovico Magnocavallo a8c84357f4
Integrate checklist data in FAST (#1969)
* add locals for additive and authoritative org iam roles

* first shot at IAM and logging location

* tfdoc

* use locals for locations

* fix file parsing, resman stubs

* initial resman implementation

* remove unneeded code

* fix data file

* replace dumb yamldecode

* fix wrong type in organization additive bindings try

* simplify logging local

* Use check asserts for version and org id

* Checks on checklist for resman

* refactor checks, ignore checklist files on wrong org id

* stage 0 tests

* fix checklist checks

* stage 1 tests

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-01-18 05:45:29 +01:00
Wiktor Niesiobędzki a34cdd5597
Better error reporting when missing setup for E2E tests (#1985) 2024-01-17 23:34:20 +03:00
Luca Prete bc506795f2
compute-vm: add resource manager tags support for instance templates (#1982) 2024-01-16 17:40:15 +00:00
apichick 5ba54aeaf7 Added Cross-region internal application load balancer module 2024-01-16 17:54:50 +01:00
Wiktor Niesiobędzki af41e0fe38 Use fixtures in net-lb-ext 2024-01-06 17:09:49 +01:00
Wiktor Niesiobędzki b92135a56d Remove null address for PSC
null address results in following error:
Error creating GlobalAddress: googleapi: Error 400: Invalid value for field 'resource.address': ''. The field is not a valid IP address or does not match the given prefix length, invalid
2024-01-06 14:02:52 +01:00
Wiktor Niesiobędzki 3470661993 IPv6 address reservation not supported for ILB
Error creating Address: googleapi: Error 400: Invalid value for field 'resource.purpose': 'SHARED_LOADBALANCER_VIP'. Shared LoadBalancer VIP IPv6 address reservation is not supported., invalid
2024-01-06 14:02:52 +01:00
Wiktor Niesiobędzki 8afdf66a09 Add end-to-end tests for net-address 2024-01-06 14:02:52 +01:00
Wiktor Niesiobędzki 6e24898850 Add proxy networks to e2e harness 2024-01-06 12:05:53 +01:00
Wiktor Niesiobędzki a6e377fad2 Rename compute-mig-bc.tf to compute-vm-group-bc.tf 2024-01-06 12:05:53 +01:00
Julio Castillo 8beb621e07
New module for external regional application load balancer (#1892)
* Initial version of regional external application load balancer.

* Fix tests

* Remove unsupported features in regional app lbs

* update readme with fixtures

* Add ssl-certificate fixture

* Switch examples to regions b c

* Remove redundant NEG examples

* Update README

* Update versions.tf

* Add missing boilerplate
2024-01-05 16:59:27 +01:00
Stefano Tribioli a4def10c19 Add PNA support to Service Directory module
Endpoints in Service Directory can be *associated* with a
VPC. In this case, they can be used by supported Google
Cloud products to send requests directly to resources inside
a VPC. This feature is called Private Network Access.

The `google_service_directory_endpoint` resource supports
this configuration with a new argument `network`.
Unfortunately, this argument has an unusual format: it
is similar to a standard VPC ID, but instead of the project ID,
it expects the project number.
2024-01-05 15:05:32 +00:00
Julio Castillo c13a192755
Use zones b and c for MIG fixture (#1961) 2024-01-05 15:02:12 +00:00
Andy Bubune Amewuda 3edacd0aba Add e2e test for net_lb_app_ext module 2024-01-05 10:02:23 +01:00
Julio Castillo fde7b76036
Allow per-module terraform fixtures (#1914)
* Allow terraform fixtures for examples

* Allow defining multiple fixtures, and named fixtures under tests/fixtures/

* Enable e2e for wiktorn

* Fix prepare_files call for e2e

* Move fixture to separate file, fix test

* Revert shallow-copying symlinks, performane penalty - 20%

* Update tfdoc.py to list used fixtures

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2023-12-29 09:43:44 +00:00
andybubu 2ad109ae23
Fix variable region (#1953)
This PR changes variable region's default value in example tests to real region value.

Some of the modules parse the region name to decide whether to create regional or zonal resources.
2023-12-28 15:04:15 +01:00
Ludovico Magnocavallo 9d6e61428b
(WIP) Read-only service accounts for automation and CI/CD (#1899)
* add design doc for the new CI/CD sa

* describe the actual implementation

* specify which files will need to be changed

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Update 0-cicd-plan-sa.md

* Fix typo

* stage 0 read-only service accounts

* stage 0 IAM map

* linting

* cicd read-only service accounts

* tweak workflow templates

* roles and github workflow fixes

* tfdoc

* Ad-hoc custom role factory for FAST bootstrap

* use factory variable for custom roles data path

* custom roles factory in org/project modules

* tfdoc

* rename custom roles factory variable, fix gitlab template

* gitlab workflow fixes

* fix merge

* output plan results on failed assertion

* update stage 0 expected values

* data platform branch

* gke

* networking

* security

* project factory

* outputs

* workflow templates

* resman apply fixes

* tfdoc

* fix stage 1 test fixture

* fix gh workflow

* read-only resman sa roles

* fix test

* read-only resman sa roles

* read-only resman sa roles

* read-only resman sa roles

* read-only resman sa roles

* fix test variables

* rename wif principal attribute names

* rename wif principal variables

* multitenant stages

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-12-27 11:33:16 +00:00
dibaskar-google 969111f0cf
dns e2e tests (#1944) 2023-12-23 10:29:32 +00:00
Luca Prete 44b1115b9f
Fix GCVE network policy (#1948)
Co-authored-by: Luca Prete <lucaprete@google.com>
2023-12-22 10:29:43 +00:00
Luca Prete 06b2a97291
GCVE: add network policy configuration 2023-12-22 10:02:12 +00:00
Luca Prete c4123044b7
Use new resources in GCVE module, bump up provider versions (#1941) 2023-12-21 13:23:38 +00:00
simonebruzzechesse c9a8d777ba
Add kernels.googleusercontent.com zone in dns response policy (#1940)
* Add kernels.googleusercontent.com zone in dns response policy
* update fast tests
2023-12-20 11:18:11 +01:00