zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' of https://github.com/GoogleCloudPlatform/cloud-foundation-fabric into vpc-sc

This commit is contained in:
Maurizio Noseda Pedraglio 2022-11-10 12:54:39 +01:00
parent 2bc60e6dee b7bfcf3575
commit 55d544b172
181 changed files with 5359 additions and 3700 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.github/workflows/linting.yml vendored
View File

@ -53,12 +53,12 @@ jobs:
run: |
terraform fmt -recursive -check -diff $GITHUB_WORKSPACE
- name: Check documentation (fabric)
- name: Check documentation
id: documentation-fabric
run: |
python3 tools/check_documentation.py examples modules fast
python3 tools/check_documentation.py modules fast blueprints
- name: Check documentation links (fabric)
- name: Check documentation links
id: documentation-links-fabric
run: |
python3 tools/check_links.py .

35
CHANGELOG.md
View File

@ -5,9 +5,18 @@ All notable changes to this project will be documented in this file.
## [Unreleased]
<!-- None < 2022-09-09 18:02:15+00:00 -->
- [[#939](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/939)] Temporarily duplicate cloud armor example ([ludoo](https://github.com/ludoo)) <!-- 2022-11-02 09:36:04+00:00 -->
### BLUEPRINTS
- [[#952](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/952)] Remove duplicate GLB+CA blueprint folder ([ludoo](https://github.com/ludoo)) <!-- 2022-11-07 12:46:22+00:00 -->
- [[#949](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/949)] **incompatible change:** Refactor VPC firewall module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-04 12:56:08+00:00 -->
- [[#945](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/945)] Org policy factory ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 11:30:58+00:00 -->
- [[#941](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/941)] **incompatible change:** Refactor ILB module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-02 17:05:21+00:00 -->
- [[#936](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/936)] Enable org policy service and add README notice to modules ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 13:25:08+00:00 -->
- [[#931](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/931)] **incompatible change:** Refactor compute-mig module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 08:39:00+00:00 -->
- [[#932](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/932)] feat(project-factory): introduce additive iam bindings to project-fac… ([Malet](https://github.com/Malet)) <!-- 2022-10-31 17:24:25+00:00 -->
- [[#925](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/925)] Network dashboard: update main.tf and README following #922 ([brianhmj](https://github.com/brianhmj)) <!-- 2022-10-28 15:49:12+00:00 -->
- [[#924](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/924)] Fix formatting for gcloud dataflow job launch command ([aymanfarhat](https://github.com/aymanfarhat)) <!-- 2022-10-27 14:07:25+00:00 -->
- [[#921](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/921)] Align documentation, move glb blueprint ([ludoo](https://github.com/ludoo)) <!-- 2022-10-26 12:31:04+00:00 -->
- [[#915](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/915)] TFE OIDC with GCP WIF blueprint added ([averbuks](https://github.com/averbuks)) <!-- 2022-10-25 19:06:43+00:00 -->
@ -43,6 +52,9 @@ All notable changes to this project will be documented in this file.
### DOCUMENTATION
- [[#961](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/961)] Remove extra file from root ([ludoo](https://github.com/ludoo)) <!-- 2022-11-09 07:53:11+00:00 -->
- [[#943](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/943)] Update bootstrap README.md with unique project id requirements ([KPRepos](https://github.com/KPRepos)) <!-- 2022-11-03 22:22:22+00:00 -->
- [[#937](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/937)] Fix typos in blueprints README.md ([kumar-dhanagopal](https://github.com/kumar-dhanagopal)) <!-- 2022-11-02 07:39:26+00:00 -->
- [[#921](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/921)] Align documentation, move glb blueprint ([ludoo](https://github.com/ludoo)) <!-- 2022-10-26 12:31:04+00:00 -->
- [[#898](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/898)] Update FAST bootstrap README.md ([juliocc](https://github.com/juliocc)) <!-- 2022-10-19 15:15:36+00:00 -->
- [[#878](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/878)] chore: update cft and fabric ([bharathkkb](https://github.com/bharathkkb)) <!-- 2022-10-12 15:38:06+00:00 -->
@ -51,6 +63,15 @@ All notable changes to this project will be documented in this file.
### FAST
- [[#956](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/956)] FAST: bootstrap and extra stage CI/CD improvements and fixes ([ludoo](https://github.com/ludoo)) <!-- 2022-11-08 08:38:16+00:00 -->
- [[#949](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/949)] **incompatible change:** Refactor VPC firewall module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-04 12:56:08+00:00 -->
- [[#943](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/943)] Update bootstrap README.md with unique project id requirements ([KPRepos](https://github.com/KPRepos)) <!-- 2022-11-03 22:22:22+00:00 -->
- [[#948](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/948)] Use display_name instead of description for FAST service accounts ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 16:22:18+00:00 -->
- [[#947](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/947)] Use org policy factory for resman stage ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 14:04:08+00:00 -->
- [[#941](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/941)] **incompatible change:** Refactor ILB module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-02 17:05:21+00:00 -->
- [[#935](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/935)] FAST: enable org policy API, fix run.allowedIngress value ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 08:52:03+00:00 -->
- [[#931](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/931)] **incompatible change:** Refactor compute-mig module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 08:39:00+00:00 -->
- [[#930](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/930)] **incompatible change:** Update organization/folder/project modules to use new org policies API and tf1.3 optionals ([juliocc](https://github.com/juliocc)) <!-- 2022-10-28 16:21:06+00:00 -->
- [[#911](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/911)] FAST: Additional PGA DNS records ([sruffilli](https://github.com/sruffilli)) <!-- 2022-10-25 12:28:29+00:00 -->
- [[#903](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/903)] Initial replacement for CI/CD stage ([ludoo](https://github.com/ludoo)) <!-- 2022-10-23 17:52:46+00:00 -->
- [[#898](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/898)] Update FAST bootstrap README.md ([juliocc](https://github.com/juliocc)) <!-- 2022-10-19 15:15:36+00:00 -->
@ -71,6 +92,18 @@ All notable changes to this project will be documented in this file.
### MODULES
- [[#958](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/958)] Add support for org policy custom constraints ([averbuks](https://github.com/averbuks)) <!-- 2022-11-09 09:07:46+00:00 -->
- [[#960](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/960)] Fix README typo in firewall module ([valeriobponza](https://github.com/valeriobponza)) <!-- 2022-11-08 23:25:34+00:00 -->
- [[#953](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/953)] Added IAM Additive and converted some outputs to static ([muresan](https://github.com/muresan)) <!-- 2022-11-07 13:20:17+00:00 -->
- [[#951](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/951)] cloud-functions v2 - fix reference to bucket_name ([wiktorn](https://github.com/wiktorn)) <!-- 2022-11-06 07:32:39+00:00 -->
- [[#949](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/949)] **incompatible change:** Refactor VPC firewall module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-04 12:56:08+00:00 -->
- [[#946](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/946)] **incompatible change:** Deprecate organization-policy module ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 11:56:12+00:00 -->
- [[#945](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/945)] Org policy factory ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 11:30:58+00:00 -->
- [[#941](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/941)] **incompatible change:** Refactor ILB module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-02 17:05:21+00:00 -->
- [[#940](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/940)] Ensure the implementation of org policies is consistent ([juliocc](https://github.com/juliocc)) <!-- 2022-11-02 09:55:21+00:00 -->
- [[#936](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/936)] Enable org policy service and add README notice to modules ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 13:25:08+00:00 -->
- [[#931](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/931)] **incompatible change:** Refactor compute-mig module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 08:39:00+00:00 -->
- [[#930](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/930)] **incompatible change:** Update organization/folder/project modules to use new org policies API and tf1.3 optionals ([juliocc](https://github.com/juliocc)) <!-- 2022-10-28 16:21:06+00:00 -->
- [[#926](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/926)] Fix backwards compatibility for vpc subnet descriptions ([ludoo](https://github.com/ludoo)) <!-- 2022-10-28 06:13:04+00:00 -->
- [[#927](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/927)] Add support for deployment type and api proxy type for Apigee org ([kmucha555](https://github.com/kmucha555)) <!-- 2022-10-27 19:56:41+00:00 -->
- [[#923](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/923)] Fix service account creation error in gke nodepool module ([ludoo](https://github.com/ludoo)) <!-- 2022-10-27 15:12:05+00:00 -->
@ -111,6 +144,8 @@ All notable changes to this project will be documented in this file.
### TOOLS
- [[#950](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/950)] Add a pytest fixture to convert tfvars to yaml ([ludoo](https://github.com/ludoo)) <!-- 2022-11-04 17:37:24+00:00 -->
- [[#942](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/942)] Bump tftest and improve dns tests ([juliocc](https://github.com/juliocc)) <!-- 2022-11-02 19:38:01+00:00 -->
- [[#919](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/919)] Rename workflow names ([juliocc](https://github.com/juliocc)) <!-- 2022-10-25 15:22:51+00:00 -->
- [[#902](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/902)] Bring back sorted variables check ([juliocc](https://github.com/juliocc)) <!-- 2022-10-20 17:08:17+00:00 -->
- [[#887](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/887)] Disable parallel execution of tests and plugin cache ([ludoo](https://github.com/ludoo)) <!-- 2022-10-14 17:52:38+00:00 -->

2
README.md
View File

@ -29,7 +29,7 @@ The current list of modules supports most of the core foundational and networkin
Currently available modules:
- **foundational** - [billing budget](./modules/billing-budget), [Cloud Identity group](./modules/cloud-identity-group/), [folder](./modules/folder), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [organization](./modules/organization), [organization-policy](./modules/organization-policy), [project](./modules/project), [projects-data-source](./modules/projects-data-source)
- **foundational** - [billing budget](./modules/billing-budget), [Cloud Identity group](./modules/cloud-identity-group/), [folder](./modules/folder), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [organization](./modules/organization), [project](./modules/project), [projects-data-source](./modules/projects-data-source)
- **networking** - [DNS](./modules/dns), [Cloud Endpoints](./modules/endpoints), [address reservation](./modules/net-address), [NAT](./modules/net-cloudnat), [Global Load Balancer (classic)](./modules/net-glb/), [L4 ILB](./modules/net-ilb), [L7 ILB](./modules/net-ilb-l7), [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC peering](./modules/net-vpc-peering), [VPN dynamic](./modules/net-vpn-dynamic), [HA VPN](./modules/net-vpn-ha), [VPN static](./modules/net-vpn-static), [Service Directory](./modules/service-directory)
- **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid), [GKE cluster](./modules/gke-cluster), [GKE hub](./modules/gke-hub), [GKE nodepool](./modules/gke-nodepool)
- **data** - [BigQuery dataset](./modules/bigquery-dataset), [Bigtable instance](./modules/bigtable-instance), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag), [Datafusion](./modules/datafusion), [GCS](./modules/gcs), [Pub/Sub](./modules/pubsub)

4
blueprints/README.md
View File

@ -1,12 +1,12 @@
# Terraform end-to-end blueprints for Google Cloud
This section **[networking blueprints](./networking/)** that implement core patterns or features, **[data solutions blueprints](./data-solutions/)** that demonstrate how to integrate data services in complete scenarios, **[cloud operations blueprints](./cloud-operations/)** that leverage specific products to meet specific operational needs, **[GKE](./gke/)** and **[Serverless](./serverless/)** blueprints, and **[factories](./factories/)** that implement resource factories for the repetitive creation of specific resources.
This section provides **[networking blueprints](./networking/)** that implement core patterns or features, **[data solutions blueprints](./data-solutions/)** that demonstrate how to integrate data services in complete scenarios, **[cloud operations blueprints](./cloud-operations/)** that leverage specific products to meet specific operational needs, **[GKE](./gke/)** and **[Serverless](./serverless/)** blueprints, and **[factories](./factories/)** that implement resource factories for the repetitive creation of specific resources.
Currently available blueprints:
- **cloud operations** - [Active Directory Federation Services](./cloud-operations/adfs), [Cloud Asset Inventory feeds for resource change tracking and remediation](./cloud-operations/asset-inventory-feed-remediation), [Fine-grained Cloud DNS IAM via Service Directory](./cloud-operations/dns-fine-grained-iam), [Cloud DNS & Shared VPC design](./cloud-operations/dns-shared-vpc), [Delegated Role Grants](./cloud-operations/iam-delegated-role-grants), [Networking Dashboard](./cloud-operations/network-dashboard), [Managing on-prem service account keys by uploading public keys](./cloud-operations/onprem-sa-key-management), [Compute Image builder with Hashicorp Packer](./cloud-operations/packer-image-builder), [Packer example](./cloud-operations/packer-image-builder/packer), [Compute Engine quota monitoring](./cloud-operations/quota-monitoring), [Scheduled Cloud Asset Inventory Export to Bigquery](./cloud-operations/scheduled-asset-inventory-export-bq), [Configuring workload identity federation for Terraform Cloud/Enterprise workflow](./cloud-operations/terraform-enterprise-wif), [TCP healthcheck and restart for unmanaged GCE instances](./cloud-operations/unmanaged-instances-healthcheck), [Migrate for Compute Engine (v5) blueprints](./cloud-operations/vm-migration), [Configuring workload identity federation to access Google Cloud resources from apps running on Azure](./cloud-operations/workload-identity-federation)
- **data solutions** - [GCE and GCS CMEK via centralized Cloud KMS](./data-solutions/cmek-via-centralized-kms), [Cloud Composer version 2 private instance, supporting Shared VPC and external CMEK key](./data-solutions/composer-2), [Cloud SQL instance with multi-region read replicas](./data-solutions/cloudsql-multiregion), [Data Platform](./data-solutions/data-platform-foundations), [Spinning up a foundation data pipeline on Google Cloud using Cloud Storage, Dataflow and BigQuery](./data-solutions/gcs-to-bq-with-least-privileges), [#SQL Server Always On Groups blueprint](./data-solutions/sqlserver-alwayson), [Data Playground](./data-solutions/data-playground)
- **factories** - [[The why and the how of Resource Factories](./factories), [Google Cloud Identity Group Factory](./factories/cloud-identity-group-factory), [Google Cloud BQ Factory](./factories/bigquery-factory), [Google Cloud VPC Firewall Factory](./factories/net-vpc-firewall-yaml), [Minimal Project Factory](./factories/project-factory)
- **factories** - [The why and the how of Resource Factories](./factories), [Google Cloud Identity Group Factory](./factories/cloud-identity-group-factory), [Google Cloud BQ Factory](./factories/bigquery-factory), [Google Cloud VPC Firewall Factory](./factories/net-vpc-firewall-yaml), [Minimal Project Factory](./factories/project-factory)
- **GKE** - [Binary Authorization Pipeline Blueprint](./gke/binauthz), [Storage API](./gke/binauthz/image), [Multi-cluster mesh on GKE (fleet API)](./gke/multi-cluster-mesh-gke-fleet-api), [GKE Multitenant Blueprint](./gke/multitenant-fleet), [Shared VPC with GKE support](./networking/shared-vpc-gke/)
- **networking** - [Decentralized firewall management](./networking/decentralized-firewall), [Decentralized firewall validator](./networking/decentralized-firewall/validator), [Network filtering with Squid](./networking/filtering-proxy), [HTTP Load Balancer with Cloud Armor](./networking/glb-and-armor), [Hub and Spoke via VPN](./networking/hub-and-spoke-vpn), [Hub and Spoke via VPC Peering](./networking/hub-and-spoke-peering), [Internal Load Balancer as Next Hop](./networking/ilb-next-hop), [Nginx-based reverse proxy cluster](./networking/nginx-reverse-proxy-cluster), [On-prem DNS and Google Private Access](./networking/onprem-google-access-dns), [Calling a private Cloud Function from On-premises](./networking/private-cloud-function-from-onprem), [Hybrid connectivity to on-premise services through PSC](./networking/psc-hybrid), [PSC Producer](./networking/psc-hybrid/psc-producer), [PSC Consumer](./networking/psc-hybrid/psc-consumer), [Shared VPC with optional GKE cluster](./networking/shared-vpc-gke)
- **serverless** - [Creating multi-region deployments for API Gateway](./serverless/api-gateway)

3
blueprints/cloud-operations/network-dashboard/README.md
View File

@ -29,6 +29,8 @@ Clone this repository, then go through the following steps to create resources:
- `terraform init`
- `terraform apply`
Note: Org level viewing permission is required for some metrics such as firewall policies.
Once the resources are deployed, go to the following page to see the dashboard: https://console.cloud.google.com/monitoring/dashboards?project=<YOUR-MONITORING-PROJECT> (or <YOUR-METRICS-PROJECT> if populated)
A dashboard called "quotas-utilization" should be created.
@ -78,6 +80,7 @@ In a future release, we could support:
- Dynamic routes calculation for VPCs/PPGs with "global routing" set to OFF
- Static routes calculation for projects/PPGs with "custom routes importing/exporting" set to OFF
- Calculations for cross Organization peering groups
- Support different scopes (reduced and fine-grained)
If you are interested in this and/or would like to contribute, please contact legranda@google.com.
<!-- BEGIN TFDOC -->

133
blueprints/cloud-operations/network-dashboard/cloud-function/metrics/subnets.py
View File

@ -25,7 +25,6 @@ import ipaddress
def get_all_subnets(config):
'''
Returns a dictionary with subnet level informations (such as IP utilization)
Parameters:
config (dict): The dict containing config like clients and limits
Returns:
@ -83,19 +82,16 @@ def get_all_subnets(config):
return subnet_dict
def compute_subnet_utilization(config, all_subnets_dict):
def compute_subnet_utilization_vms(config, read_mask, all_subnets_dict):
'''
Counts resources (VMs, ILBs, reserved IPs) using private IPs in the different subnets.
Counts VMs using private IPs in the different subnets.
Parameters:
config (dict): Dict containing config like clients and limits
read_mask (FieldMask): read_mask to get additional metadata from Cloud Asset Inventory
all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
Returns:
None
'''
read_mask = field_mask_pb2.FieldMask()
read_mask.FromJsonString('name,versionedResources')
response_vm = config["clients"]["asset_client"].search_all_resources(
request={
"scope": f"organizations/{config['organization']}",
@ -123,6 +119,17 @@ def compute_subnet_utilization(config, all_subnets_dict):
all_subnets_dict[project_id][f"{subnet_region}/{subnet_name}"][
'used_ip_addresses'] += 1
def compute_subnet_utilization_ilbs(config, read_mask, all_subnets_dict):
'''
Counts ILBs using private IPs in the different subnets.
Parameters:
config (dict): Dict containing config like clients and limits
read_mask (FieldMask): read_mask to get additional metadata from Cloud Asset Inventory
all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
Returns:
None
'''
response_ilb = config["clients"]["asset_client"].search_all_resources(
request={
"scope": f"organizations/{config['organization']}",
@ -131,7 +138,6 @@ def compute_subnet_utilization(config, all_subnets_dict):
"page_size": config["page_size"],
})
# Counting IP addresses for GCE Internal Load Balancers
for asset in response_ilb:
internal = False
psc = False
@ -139,9 +145,11 @@ def compute_subnet_utilization(config, all_subnets_dict):
subnet_name = ''
subnet_region = ''
address = ''
network = ''
for versioned in asset.versioned_resources:
for field_name, field_value in versioned.resource.items():
if 'loadBalancingScheme' in field_name and field_value in ['INTERNAL', 'INTERNAL_MANAGED']:
if 'loadBalancingScheme' in field_name and field_value in [
'INTERNAL', 'INTERNAL_MANAGED']:
internal = True
# We want to count only accepted PSC endpoint Forwarding Rule
# If the PSC endpoint Forwarding Rule is pending, we will count it in the reserved IP addresses
@ -151,6 +159,7 @@ def compute_subnet_utilization(config, all_subnets_dict):
address = field_value
elif field_name == 'network':
project_id = field_value.split('/')[6]
network = field_value.split('/')[-1]
elif 'subnetwork' in field_name:
subnet_name = field_value.split('/')[-1]
subnet_region = field_value.split('/')[-3]
@ -163,9 +172,21 @@ def compute_subnet_utilization(config, all_subnets_dict):
# We need to find the correct subnet with IP address matching
ip_address = ipaddress.ip_address(address)
for subnet_key, subnet_dict in all_subnets_dict[project_id].items():
if ip_address in ipaddress.ip_network(subnet_dict['ip_cidr_range']):
all_subnets_dict[project_id][subnet_key]['used_ip_addresses'] += 1
if subnet_dict["network_name"] == network:
if ip_address in ipaddress.ip_network(subnet_dict['ip_cidr_range']):
all_subnets_dict[project_id][subnet_key]['used_ip_addresses'] += 1
def compute_subnet_utilization_addresses(config, read_mask, all_subnets_dict):
'''
Counts reserved IP addresses in the different subnets.
Parameters:
config (dict): Dict containing config like clients and limits
read_mask (FieldMask): read_mask to get additional metadata from Cloud Asset Inventory
all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
Returns:
None
'''
response_reserved_ips = config["clients"][
"asset_client"].search_all_resources(
request={
@ -185,8 +206,11 @@ def compute_subnet_utilization(config, all_subnets_dict):
subnet_region = ""
address = ""
prefixLength = ""
address_name = ""
for versioned in asset.versioned_resources:
for field_name, field_value in versioned.resource.items():
if field_name == 'name':
address_name = field_value
if field_name == 'purpose':
purpose = field_value
elif field_name == 'region':
@ -214,9 +238,90 @@ def compute_subnet_utilization(config, all_subnets_dict):
'used_ip_addresses'] += 1
# PSA Range for Cloud SQL, MemoryStore, etc.
elif purpose == "VPC_PEERING":
# TODO: PSA range to be handled later
# print("PSA range to be handled later:", address, prefixLength, network_name)
continue
ip_range = f"{address}/{int(prefixLength)}"
net = ipaddress.ip_network(ip_range)
# Note that 4 IP addresses are reserved by GCP in all subnets
# Source: https://cloud.google.com/vpc/docs/subnets#reserved_ip_addresses_in_every_subnet
total_ip_addresses = int(net.num_addresses) - 4
all_subnets_dict[project_id][f"psa/{address_name}"] = {
'name': f"psa/{address_name}",
'region': subnet_region,
'ip_cidr_range': ip_range,
'total_ip_addresses': total_ip_addresses,
'used_ip_addresses': 0,
'network_name': network_name
}
def compute_subnet_utilization_redis(config, read_mask, all_subnets_dict):
'''
Counts Redis (Memorystore) instances using private IPs in the different subnets.
Parameters:
config (dict): Dict containing config like clients and limits
read_mask (FieldMask): read_mask to get additional metadata from Cloud Asset Inventory
all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
Returns:
None
'''
response_redis = config["clients"]["asset_client"].search_all_resources(
request={
"scope": f"organizations/{config['organization']}",
"asset_types": ["redis.googleapis.com/Instance"],
"read_mask": read_mask,
"page_size": config["page_size"],
})
for asset in response_redis:
ip_range = ""
connect_mode = ""
network_name = ""
project_id = ""
region = ""
for versioned in asset.versioned_resources:
for field_name, field_value in versioned.resource.items():
if field_name == 'locationId':
region = field_value[0:-2]
if field_name == 'authorizedNetwork':
network_name = field_value.split('/')[-1]
project_id = field_value.split('/')[1]
if field_name == 'reservedIpRange':
ip_range = field_value
if field_name == 'connectMode':
connect_mode = field_value
# Only handling PSA for Redis for now
if connect_mode == "PRIVATE_SERVICE_ACCESS":
redis_ip_range = ipaddress.ip_network(ip_range)
for subnet_key, subnet_dict in all_subnets_dict[project_id].items():
if subnet_dict["network_name"] == network_name:
# Reddis instance asset doesn't contain the subnet information in Asset Inventory
# We need to find the correct subnet range with IP address matching to compute the utilization
if redis_ip_range.overlaps(
ipaddress.ip_network(subnet_dict['ip_cidr_range'])):
all_subnets_dict[project_id][subnet_key][
'used_ip_addresses'] += redis_ip_range.num_addresses
all_subnets_dict[project_id][subnet_key]['region'] = region
def compute_subnet_utilization(config, all_subnets_dict):
'''
Counts resources (VMs, ILBs, reserved IPs) using private IPs in the different subnets.
Parameters:
config (dict): Dict containing config like clients and limits
all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
Returns:
None
'''
read_mask = field_mask_pb2.FieldMask()
read_mask.FromJsonString('name,versionedResources')
compute_subnet_utilization_vms(config, read_mask, all_subnets_dict)
compute_subnet_utilization_ilbs(config, read_mask, all_subnets_dict)
compute_subnet_utilization_addresses(config, read_mask, all_subnets_dict)
# TODO: Other PSA services such as FileStore, Cloud SQL
compute_subnet_utilization_redis(config, read_mask, all_subnets_dict)
# TODO: Handle secondary ranges and count GKE pods
def get_subnets(config, metrics_dict):

9
blueprints/cloud-operations/network-dashboard/main.tf
View File

@ -21,7 +21,6 @@ locals {
folder_ids = toset(var.monitored_folders_list)
folders = join(",", local.folder_ids)
monitoring_project = var.monitoring_project_id == "" ? module.project-monitoring[0].project_id : var.monitoring_project_id
metrics_project = var.metrics_project_id == "" ? (var.monitoring_project_id == "" ? module.project-monitoring[0].project_id : var.monitoring_project_id) : var.metrics_project_id
}
@ -52,7 +51,6 @@ module "service-account-function" {
# Required IAM permissions for this service account are:
# 1) compute.networkViewer on projects to be monitored (I gave it at organization level for now for simplicity)
# 2) monitoring viewer on the projects to be monitored (I gave it at organization level for now for simplicity)
# 3) if you dont have permission to create service account and assign permission at organization Level, move these 3 roles to project level.
iam_organization_roles = {
"${var.organization_id}" = [
@ -179,9 +177,6 @@ module "cloud-function" {
# Internal only doesn't seem to work with CFv2:
ingress_settings = var.cf_version == "V2" ? "ALLOW_ALL" : "ALLOW_INTERNAL_ONLY"
trigger_config = {
event = "google.pubsub.topic.publish"
resource = module.pubsub.topic.id
@ -195,5 +190,9 @@ module "cloud-function" {
resource "google_monitoring_dashboard" "dashboard" {
dashboard_json = file("${path.module}/dashboards/quotas-utilization.json")
<<<<<<< HEAD
project = local.metrics_project
=======
project = local.monitoring_project
>>>>>>> b7bfcf3575cda18a2fdd2862c72e33c1648c0aa4
}

2
modules/organization-policy/versions.tf → blueprints/cloud-operations/network-dashboard/versions.tf
View File

@ -25,5 +25,3 @@ terraform {
}
}
}

13
blueprints/cloud-operations/packer-image-builder/main.tf
View File

@ -67,17 +67,16 @@ module "firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc.name
custom_rules = {
ingress_rules = {
image-builder-ingress-builder-vm = {
description = "Allow image builder vm ingress traffic"
direction = "INGRESS"
action = "allow"
sources = []
ranges = var.packer_source_cidrs
source_ranges = var.packer_source_cidrs
targets = [module.service-account-image-builder-vm.email]
use_service_accounts = true
rules = [{ protocol = "tcp", ports = [22, 5985, 5986] }]
extra_attributes = {}
rules = [{
protocol = "tcp"
ports = [22, 5985, 5986]
}]
}
}
}

22
blueprints/cloud-operations/terraform-enterprise-wif/gcp-workload-identity-provider/README.md
View File

@ -12,22 +12,22 @@ The codebase provisions the following list of resources:
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [billing_account](variables.tf#L16) | Billing account id used as default for new projects. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L38) | Existing project id. | <code>string</code> | ✓ | |
| [tfe_organization_id](variables.tf#L43) | | <code></code> | ✓ | |
| [tfe_workspace_id](variables.tf#L48) | | <code></code> | ✓ | |
| [issuer_uri](variables.tf#L65) | Terraform Enterprise uri. Replace the uri if a self hosted instance is used. | <code>string</code> | | <code>&#34;https:&#47;&#47;app.terraform.io&#47;&#34;</code> |
| [project_id](variables.tf#L43) | Existing project id. | <code>string</code> | ✓ | |
| [tfe_organization_id](variables.tf#L48) | TFE organization id. | <code>string</code> | ✓ | |
| [tfe_workspace_id](variables.tf#L53) | TFE workspace id. | <code>string</code> | ✓ | |
| [issuer_uri](variables.tf#L21) | Terraform Enterprise uri. Replace the uri if a self hosted instance is used. | <code>string</code> | | <code>&#34;https:&#47;&#47;app.terraform.io&#47;&#34;</code> |
| [parent](variables.tf#L27) | Parent folder or organization in 'folders/folder_id' or 'organizations/org_id' format. | <code>string</code> | | <code>null</code> |
| [project_create](variables.tf#L21) | Create project instead of using an existing one. | <code>bool</code> | | <code>true</code> |
| [workload_identity_pool_id](variables.tf#L53) | Workload identity pool id. | <code>string</code> | | <code>&#34;tfe-pool&#34;</code> |
| [workload_identity_pool_provider_id](variables.tf#L59) | Workload identity pool provider id. | <code>string</code> | | <code>&#34;tfe-provider&#34;</code> |
| [project_create](variables.tf#L37) | Create project instead of using an existing one. | <code>bool</code> | | <code>true</code> |
| [workload_identity_pool_id](variables.tf#L58) | Workload identity pool id. | <code>string</code> | | <code>&#34;tfe-pool&#34;</code> |
| [workload_identity_pool_provider_id](variables.tf#L64) | Workload identity pool provider id. | <code>string</code> | | <code>&#34;tfe-provider&#34;</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| [impersonate_service_account_email](outputs.tf#L31) | | |
| [project_id](outputs.tf#L16) | | |
| [workload_identity_audience](outputs.tf#L26) | | |
| [workload_identity_pool_provider_id](outputs.tf#L21) | GCP workload identity pool provider ID. | |
| [impersonate_service_account_email](outputs.tf#L16) | Service account to be impersonated by workload identity. | |
| [project_id](outputs.tf#L21) | GCP Project ID. | |
| [workload_identity_audience](outputs.tf#L26) | TFC Workload Identity Audience. | |
| [workload_identity_pool_provider_id](outputs.tf#L31) | GCP workload identity pool provider ID. | |
<!-- END TFDOC -->

16
blueprints/cloud-operations/terraform-enterprise-wif/gcp-workload-identity-provider/outputs.tf
View File

@ -13,22 +13,22 @@
# limitations under the License.
output "impersonate_service_account_email" {
description = "Service account to be impersonated by workload identity."
value = module.sa-tfe.email
}
output "project_id" {
description = "GCP Project ID."
value = module.project.project_id
}
output "workload_identity_pool_provider_id" {
description = "GCP workload identity pool provider ID."
value = google_iam_workload_identity_pool_provider.tfe-pool-provider.name
}
output "workload_identity_audience" {
description = "TFC Workload Identity Audience."
value = "//iam.googleapis.com/${google_iam_workload_identity_pool_provider.tfe-pool-provider.name}"
}
output "impersonate_service_account_email" {
description = "Service account to be impersonated by workload identity."
value = module.sa-tfe.email
output "workload_identity_pool_provider_id" {
description = "GCP workload identity pool provider ID."
value = google_iam_workload_identity_pool_provider.tfe-pool-provider.name
}

19
blueprints/cloud-operations/terraform-enterprise-wif/gcp-workload-identity-provider/variables.tf
View File

@ -18,10 +18,10 @@ variable "billing_account" {
type = string
}
variable "project_create" {
description = "Create project instead of using an existing one."
type = bool
default = true
variable "issuer_uri" {
description = "Terraform Enterprise uri. Replace the uri if a self hosted instance is used."
type = string
default = "https://app.terraform.io/"
}
variable "parent" {
@ -34,6 +34,11 @@ variable "parent" {
}
}
variable "project_create" {
description = "Create project instead of using an existing one."
type = bool
default = true
}
variable "project_id" {
description = "Existing project id."
@ -61,9 +66,3 @@ variable "workload_identity_pool_provider_id" {
type = string
default = "tfe-provider"
}
variable "issuer_uri" {
description = "Terraform Enterprise uri. Replace the uri if a self hosted instance is used."
type = string
default = "https://app.terraform.io/"
}

7
blueprints/cloud-operations/terraform-enterprise-wif/tfc-workflow-using-wif/README.md
View File

@ -5,15 +5,14 @@ This terraform code is a part of [GCP Workload Identity Federation for Terraform
The codebase provisions the following list of resources:
- GCS Bucket
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [impersonate_service_account_email](variables.tf#L26) | | <code></code> | ✓ | |
| [project_id](variables.tf#L16) | | <code></code> | ✓ | |
| [workload_identity_pool_provider_id](variables.tf#L21) | GCP workload identity pool provider ID. | <code>string</code> | ✓ | |
| [impersonate_service_account_email](variables.tf#L21) | Service account to be impersonated by workload identity. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L16) | GCP project ID. | <code>string</code> | ✓ | |
| [workload_identity_pool_provider_id](variables.tf#L26) | GCP workload identity pool provider ID. | <code>string</code> | ✓ | |
<!-- END TFDOC -->

10
blueprints/cloud-operations/terraform-enterprise-wif/tfc-workflow-using-wif/variables.tf
View File

@ -18,12 +18,12 @@ variable "project_id" {
type = string
}
variable "workload_identity_pool_provider_id" {
description = "GCP workload identity pool provider ID."
type = string
}
variable "impersonate_service_account_email" {
description = "Service account to be impersonated by workload identity."
type = string
}
variable "workload_identity_pool_provider_id" {
description = "GCP workload identity pool provider ID."
type = string
}

23
blueprints/cloud-operations/vm-migration/single-project/main.tf
View File

@ -66,24 +66,7 @@ module "landing-vpc" {
}
module "landing-vpc-firewall" {
source = "../../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
custom_rules = {
allow-ssh = {
description = "Allow SSH from IAP"
direction = "INGRESS"
action = "allow"
sources = []
ranges = ["35.235.240.0/20"]
targets = []
use_service_accounts = false
rules = [{ protocol = "tcp", ports = ["22"] }]
extra_attributes = {}
}
}
source = "../../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-vpc.name
}

12
blueprints/data-solutions/cloudsql-multiregion/main.tf
View File

@ -128,11 +128,13 @@ module "vpc" {
}
module "firewall" {
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.project.project_id
network = module.vpc.0.name
admin_ranges = ["10.0.0.0/20"]
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.project.project_id
network = module.vpc.0.name
default_rules_config = {
admin_ranges = ["10.0.0.0/20"]
}
}
module "nat" {

10
blueprints/data-solutions/cmek-via-centralized-kms/main.tf
View File

@ -59,10 +59,12 @@ module "vpc" {
}
module "vpc-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project-service.project_id
network = module.vpc.name
admin_ranges = [var.vpc_ip_cidr_range]
source = "../../../modules/net-vpc-firewall"
project_id = module.project-service.project_id
network = module.vpc.name
default_rules_config = {
admin_ranges = [var.vpc_ip_cidr_range]
}
}
###############################################################################

12
blueprints/data-solutions/data-platform-foundations/02-load.tf
View File

@ -118,11 +118,13 @@ module "load-vpc" {
}
module "load-vpc-firewall" {
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.load-project.project_id
network = module.load-vpc.0.name
admin_ranges = ["10.10.0.0/24"]
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.load-project.project_id
network = module.load-vpc.0.name
default_rules_config = {
admin_ranges = ["10.10.0.0/24"]
}
}
module "load-nat" {

19
blueprints/data-solutions/data-platform-foundations/03-orchestration.tf
View File

@ -67,8 +67,10 @@ module "orch-project" {
"roles/storage.objectViewer" = [module.load-sa-df-0.iam_email]
}
oslogin = false
policy_boolean = {
"constraints/compute.requireOsLogin" = false
org_policies = {
"constraints/compute.requireOsLogin" = {
enforce = false
}
}
services = concat(var.project_services, [
"artifactregistry.googleapis.com",
@ -82,6 +84,7 @@ module "orch-project" {
"container.googleapis.com",
"containerregistry.googleapis.com",
"dataflow.googleapis.com",
"orgpolicy.googleapis.com",
"pubsub.googleapis.com",
"servicenetworking.googleapis.com",
"storage.googleapis.com",
@ -130,11 +133,13 @@ module "orch-vpc" {
}
module "orch-vpc-firewall" {
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.orch-project.project_id
network = module.orch-vpc.0.name
admin_ranges = ["10.10.0.0/24"]
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.orch-project.project_id
network = module.orch-vpc.0.name
default_rules_config = {
admin_ranges = ["10.10.0.0/24"]
}
}
module "orch-nat" {

12
blueprints/data-solutions/data-platform-foundations/04-transformation.tf
View File

@ -142,11 +142,13 @@ module "transf-vpc" {
}
module "transf-vpc-firewall" {
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.transf-project.project_id
network = module.transf-vpc.0.name
admin_ranges = ["10.10.0.0/24"]
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.transf-project.project_id
network = module.transf-vpc.0.name
default_rules_config = {
admin_ranges = ["10.10.0.0/24"]
}
}
module "transf-nat" {

11
blueprints/data-solutions/data-platform-foundations/README.md
View File

@ -160,9 +160,10 @@ You can find more details and best practices on using DLP to De-identification a
[Data Catalog](https://cloud.google.com/data-catalog) helps you to document your data entry at scale. Data Catalog relies on [tags](https://cloud.google.com/data-catalog/docs/tags-and-tag-templates#tags) and [tag template](https://cloud.google.com/data-catalog/docs/tags-and-tag-templates#tag-templates) to manage metadata for all data entries in a unified and centralized service. To implement [column-level security](https://cloud.google.com/bigquery/docs/column-level-security-intro) on BigQuery, we suggest to use `Tags` and `Tag templates`.
The default configuration will implement 3 tags:
- `3_Confidential`: policy tag for columns that include very sensitive information, such as credit card numbers.
- `2_Private`: policy tag for columns that include sensitive personal identifiable information (PII) information, such as a person's first name.
- `1_Sensitive`: policy tag for columns that include data that cannot be made public, such as the credit limit.
- `3_Confidential`: policy tag for columns that include very sensitive information, such as credit card numbers.
- `2_Private`: policy tag for columns that include sensitive personal identifiable information (PII) information, such as a person's first name.
- `1_Sensitive`: policy tag for columns that include data that cannot be made public, such as the credit limit.
Anything that is not tagged is available to all users who have access to the data warehouse.
@ -222,7 +223,7 @@ module "data-platform" {
prefix = "myprefix"
}
# tftest modules=42 resources=315
# tftest modules=42 resources=316
```
## Customizations
@ -238,7 +239,7 @@ To do this, you need to remove IAM binging at project-level for the `data-analys
## Demo pipeline
The application layer is out of scope of this script. As a demo purpuse only, several Cloud Composer DAGs are provided. Demos will import data from the `drop off` area to the `Data Warehouse Confidential` dataset suing different features.
The application layer is out of scope of this script. As a demo purpuse only, several Cloud Composer DAGs are provided. Demos will import data from the `drop off` area to the `Data Warehouse Confidential` dataset suing different features.
You can find examples in the `[demo](./demo)` folder.
<!-- BEGIN TFDOC -->

32
blueprints/data-solutions/data-playground/main.tf
View File

@ -35,13 +35,16 @@ module "project" {
"dataflow.googleapis.com",
"ml.googleapis.com",
"notebooks.googleapis.com",
"orgpolicy.googleapis.com",
"servicenetworking.googleapis.com",
"stackdriver.googleapis.com",
"storage.googleapis.com",
"storage-component.googleapis.com"
]
policy_boolean = {
# "constraints/compute.requireOsLogin" = false
org_policies = {
# "constraints/compute.requireOsLogin" = {
# enforce = false
# }
# Example of applying a project wide policy, mainly useful for Composer
}
service_encryption_key_ids = {
@ -69,22 +72,19 @@ module "vpc" {
}
module "vpc-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc.name
admin_ranges = [var.vpc_config.ip_cidr_range]
custom_rules = {
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc.name
default_rules_config = {
admin_ranges = [var.vpc_config.ip_cidr_range]
}
ingress_rules = {
#TODO Remove and rely on 'ssh' tag once terraform-provider-google/issues/9273 is fixed
("${var.prefix}-iap") = {
description = "Enable SSH from IAP on Notebooks."
direction = "INGRESS"
action = "allow"
sources = []
ranges = ["35.235.240.0/20"]
targets = ["notebook-instance"]
use_service_accounts = false
rules = [{ protocol = "tcp", ports = [22] }]
extra_attributes = {}
description = "Enable SSH from IAP on Notebooks."
source_ranges = ["35.235.240.0/20"]
targets = ["notebook-instance"]
rules = [{ protocol = "tcp", ports = [22] }]
}
}
}

12
blueprints/data-solutions/gcs-to-bq-with-least-privileges/vpc.tf
View File

@ -27,11 +27,13 @@ module "vpc" {
}
module "vpc-firewall" {
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.project.project_id
network = module.vpc[0].name
admin_ranges = [var.vpc_subnet_range]
source = "../../../modules/net-vpc-firewall"
count = local.use_shared_vpc ? 0 : 1
project_id = module.project.project_id
network = module.vpc[0].name
default_rules_config = {
admin_ranges = [var.vpc_subnet_range]
}
}
module "nat" {

67
blueprints/data-solutions/sqlserver-alwayson/main.tf
View File

@ -13,25 +13,54 @@
# limitations under the License.
locals {
prefix = var.prefix != "" ? format("%s-", var.prefix) : ""
vpc_project = var.shared_vpc_project_id != null ? var.shared_vpc_project_id : module.project.project_id
network = module.vpc.self_link
subnetwork = var.project_create != null ? module.vpc.subnet_self_links[format("%s/%s", var.region, var.subnetwork)] : data.google_compute_subnetwork.subnetwork[0].self_link
node_base = format("%s%s", local.prefix, var.node_name)
node_prefix = length(local.node_base) > 12 ? substr(local.node_base, 0, 12) : local.node_base
node_netbios_names = [for idx in range(1, 3) : format("%s-%02d", local.node_prefix, idx)]
witness_name = format("%s%s", local.prefix, var.witness_name)
witness_netbios_name = length(local.witness_name) > 15 ? substr(local.witness_name, 0, 15) : local.witness_name
zones = var.project_create == null ? data.google_compute_zones.zones[0].names : formatlist("${var.region}-%s", ["a", "b", "c"])
node_zones = merge({ for idx, node_name in local.node_netbios_names : node_name => local.zones[idx] },
{ (local.witness_netbios_name) = local.zones[length(local.zones) - 1] })
cluster_full_name = format("%s%s", local.prefix, var.cluster_name)
cluster_netbios_name = length(local.cluster_full_name) > 15 ? substr(local.cluster_full_name, 0, 15) : local.cluster_full_name
ad_user_password_secret = format("%s%s-password", local.prefix, var.cluster_name)
ad_user_password_secret = "${local.cluster_full_name}-password"
cluster_full_name = "${local.prefix}${var.cluster_name}"
cluster_netbios_name = (
length(local.cluster_full_name) > 15
? substr(local.cluster_full_name, 0, 15)
: local.cluster_full_name
)
network = module.vpc.self_link
node_base = "${local.prefix}${var.node_name}"
node_prefix = (
length(local.node_base) > 12
? substr(local.node_base, 0, 12)
: local.node_base
)
node_netbios_names = [
for idx in range(1, 3) : format("%s-%02d", local.node_prefix, idx)
]
node_zones = merge(
{
for idx, node_name in local.node_netbios_names :
node_name => local.zones[idx]
},
{
(local.witness_netbios_name) = local.zones[length(local.zones) - 1]
}
)
prefix = var.prefix != "" ? "${var.prefix}-" : ""
subnetwork = (
var.project_create != null
? module.vpc.subnet_self_links["${var.region}/${var.subnetwork}"]
: data.google_compute_subnetwork.subnetwork[0].self_link
)
vpc_project = (
var.shared_vpc_project_id != null
? var.shared_vpc_project_id
: module.project.project_id
)
witness_name = "${local.prefix}${var.witness_name}"
witness_netbios_name = (
length(local.witness_name) > 15
? substr(local.witness_name, 0, 15)
: local.witness_name
)
zones = (
var.project_create == null
? data.google_compute_zones.zones[0].names
: formatlist("${var.region}-%s", ["a", "b", "c"])
)
}
module "project" {

133
blueprints/data-solutions/sqlserver-alwayson/vpc.tf
View File

@ -15,26 +15,36 @@
# tfdoc:file:description Creates the VPC and manages the firewall rules and ILB.
locals {
listeners = { for aog in var.always_on_groups : format("%slb-%s", local.prefix, aog) => {
region = var.region
subnetwork = local.subnetwork
internal_addresses = merge(
local.listeners,
local.node_ips,
{
"${local.prefix}cluster" = {
region = var.region
subnetwork = local.subnetwork
}
(local.witness_netbios_name) = {
region = var.region
subnetwork = local.subnetwork
}
}
)
internal_address_ips = {
for k, v in module.ip-addresses.internal_addresses :
k => v.address
}
node_ips = { for node_name in local.node_netbios_names : node_name => {
region = var.region
subnetwork = local.subnetwork
}
}
internal_addresses = merge({
format("%scluster", local.prefix) = {
listeners = {
for aog in var.always_on_groups : "${local.prefix}lb-${aog}" => {
region = var.region
subnetwork = local.subnetwork
}
(local.witness_netbios_name) = {
}
node_ips = {
for node_name in local.node_netbios_names : node_name => {
region = var.region
subnetwork = local.subnetwork
}
}, local.listeners, local.node_ips)
}
}
data "google_compute_zones" "zones" {
@ -50,7 +60,6 @@ data "google_compute_subnetwork" "subnetwork" {
region = var.region
}
# Create VPC if required
module "vpc" {
source = "../../../modules/net-vpc"
@ -66,108 +75,82 @@ module "vpc" {
vpc_create = var.project_create != null ? true : false
}
# Firewall rules required for WSFC nodes
module "firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = local.vpc_project
network = local.network
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
custom_rules = {
format("%sallow-all-between-wsfc-nodes", local.prefix) = {
source = "../../../modules/net-vpc-firewall"
project_id = local.vpc_project
network = local.network
default_rules_config = {
disabled = true
}
ingress_rules = {
"${local.prefix}allow-all-between-wsfc-nodes" = {
description = "Allow all between WSFC nodes"
direction = "INGRESS"
action = "allow"
sources = [module.compute-service-account.email]
targets = [module.compute-service-account.email]
ranges = []
use_service_accounts = true
rules = [
{ protocol = "tcp", ports = [] },
{ protocol = "udp", ports = [] },
{ protocol = "icmp", ports = [] }
{ protocol = "tcp" },
{ protocol = "udp" },
{ protocol = "icmp" }
]
extra_attributes = {}
}
format("%sallow-all-between-wsfc-witness", local.prefix) = {
"${local.prefix}allow-all-between-wsfc-witness" = {
description = "Allow all between WSFC witness nodes"
direction = "INGRESS"
action = "allow"
sources = [module.compute-service-account.email]
targets = [module.witness-service-account.email]
ranges = []
use_service_accounts = true
rules = [
{ protocol = "tcp", ports = [] },
{ protocol = "udp", ports = [] },
{ protocol = "icmp", ports = [] }
{ protocol = "tcp" },
{ protocol = "udp" },
{ protocol = "icmp" }
]
extra_attributes = {}
}
format("%sallow-sql-to-wsfc-nodes", local.prefix) = {
"${local.prefix}allow-sql-to-wsfc-nodes" = {
description = "Allow SQL connections to WSFC nodes"
direction = "INGRESS"
action = "allow"
sources = []
targets = [module.compute-service-account.email]
ranges = var.sql_client_cidrs
use_service_accounts = true
rules = [
{ protocol = "tcp", ports = [1433] },
]
extra_attributes = {}
}
format("%sallow-health-check-to-wsfc-nodes", local.prefix) = {
"${local.prefix}allow-health-check-to-wsfc-nodes" = {
description = "Allow health checks to WSFC nodes"
direction = "INGRESS"
action = "allow"
sources = []
targets = [module.compute-service-account.email]
ranges = var.health_check_ranges
use_service_accounts = true
rules = [
{ protocol = "tcp", ports = [] },
{ protocol = "tcp" }
]
extra_attributes = {}
}
}
}
# IP Address reservation for cluster and listener
module "ip-addresses" {
source = "../../../modules/net-address"
project_id = local.vpc_project
source = "../../../modules/net-address"
project_id = local.vpc_project
internal_addresses = local.internal_addresses
}
# L4 Internal Load Balancer for SQL Listener
module "listener-ilb" {
source = "../../../modules/net-ilb"
for_each = toset(var.always_on_groups)
project_id = var.project_id
region = var.region
name = format("%s-%s-ilb", var.prefix, each.value)
service_label = format("%s-%s-ilb", var.prefix, each.value)
address = module.ip-addresses.internal_addresses[format("%slb-%s", local.prefix, each.value)].address
network = local.network
subnetwork = local.subnetwork
source = "../../../modules/net-ilb"
for_each = toset(var.always_on_groups)
project_id = var.project_id
region = var.region
name = "${var.prefix}-${each.value}-ilb"
service_label = "${var.prefix}-${each.value}-ilb"
address = local.internal_address_ips["${local.prefix}lb-${each.value}"]
vpc_config = {
network = local.network
subnetwork = local.subnetwork
}
backends = [for k, node in module.nodes : {
failover = false
group = node.group.self_link
balancing_mode = "CONNECTION"
group = node.group.self_link
}]
health_check_config = {
type = "tcp",
check = { port = var.health_check_port },
config = var.health_check_config,
logging = true
enable_logging = true
tcp = {
port = var.health_check_port
}
}
}

49
blueprints/factories/project-factory/README.md
View File

@ -68,13 +68,13 @@ module "projects" {
iam = try(each.value.iam, {})
kms_service_agents = try(each.value.kms, {})
labels = try(each.value.labels, {})
org_policies = try(each.value.org_policies, null)
org_policies = try(each.value.org_policies, {})
service_accounts = try(each.value.service_accounts, {})
services = try(each.value.services, [])
service_identities_iam = try(each.value.service_identities_iam, {})
vpc = try(each.value.vpc, null)
}
# tftest modules=7 resources=27
# tftest modules=7 resources=29
```
### Projects configuration
@ -153,16 +153,16 @@ labels:
environment: prod
# [opt] Org policy overrides defined at project level
org_policies:
policy_boolean:
constraints/compute.disableGuestAttributesAccess: true
policy_list:
constraints/compute.trustedImageProjects:
inherit_from_parent: null
status: true
suggested_value: null
org_policies:
constraints/compute.disableGuestAttributesAccess:
enforce: true
constraints/compute.trustedImageProjects:
allow:
values:
- projects/fast-prod-iac-core-0
- projects/fast-dev-iac-core-0
constraints/compute.vmExternalIpAccess:
deny:
all: true
# [opt] Service account to create for the project and their roles on the project
# in name => [roles] format
@ -221,23 +221,28 @@ vpc:
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [billing_account_id](variables.tf#L17) | Billing account id. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L119) | Project id. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L157) | Project id. | <code>string</code> | ✓ | |
| [billing_alert](variables.tf#L22) | Billing alert configuration. | <code title="object&#40;&#123;&#10; amount &#61; number&#10; thresholds &#61; object&#40;&#123;&#10; current &#61; list&#40;number&#41;&#10; forecasted &#61; list&#40;number&#41;&#10; &#125;&#41;&#10; credit_treatment &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [defaults](variables.tf#L35) | Project factory default values. | <code title="object&#40;&#123;&#10; billing_account_id &#61; string&#10; billing_alert &#61; object&#40;&#123;&#10; amount &#61; number&#10; thresholds &#61; object&#40;&#123;&#10; current &#61; list&#40;number&#41;&#10; forecasted &#61; list&#40;number&#41;&#10; &#125;&#41;&#10; credit_treatment &#61; string&#10; &#125;&#41;&#10; environment_dns_zone &#61; string&#10; essential_contacts &#61; list&#40;string&#41;&#10; labels &#61; map&#40;string&#41;&#10; notification_channels &#61; list&#40;string&#41;&#10; shared_vpc_self_link &#61; string&#10; vpc_host_project &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [dns_zones](variables.tf#L57) | DNS private zones to create as child of var.defaults.environment_dns_zone. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [essential_contacts](variables.tf#L63) | Email contacts to be used for billing and GCP notifications. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [folder_id](variables.tf#L69) | Folder ID for the folder where the project will be created. | <code>string</code> | | <code>null</code> |
| [group_iam](variables.tf#L75) | Custom IAM settings in group => [role] format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam](variables.tf#L81) | Custom IAM settings in role => [principal] format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [kms_service_agents](variables.tf#L87) | KMS IAM configuration in as service => [key]. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [labels](variables.tf#L93) | Labels to be assigned at project level. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> |
| [org_policies](variables.tf#L99) | Org-policy overrides at project level. | <code title="object&#40;&#123;&#10; policy_boolean &#61; map&#40;bool&#41;&#10; policy_list &#61; map&#40;object&#40;&#123;&#10; inherit_from_parent &#61; bool&#10; suggested_value &#61; string&#10; status &#61; bool&#10; values &#61; list&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [prefix](variables.tf#L113) | Prefix used for the project id. | <code>string</code> | | <code>null</code> |
| [service_accounts](variables.tf#L124) | Service accounts to be created, and roles assigned them on the project. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_accounts_iam](variables.tf#L130) | IAM bindings on service account resources. Format is KEY => {ROLE => [MEMBERS]} | <code>map&#40;map&#40;list&#40;string&#41;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_identities_iam](variables.tf#L144) | Custom IAM settings for service identities in service => [role] format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [services](variables.tf#L137) | Services to be enabled for the project. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [vpc](variables.tf#L151) | VPC configuration for the project. | <code title="object&#40;&#123;&#10; host_project &#61; string&#10; gke_setup &#61; object&#40;&#123;&#10; enable_security_admin &#61; bool&#10; enable_host_service_agent &#61; bool&#10; &#125;&#41;&#10; subnets_iam &#61; map&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [group_iam_additive](variables.tf#L81) | Custom additive IAM settings in group => [role] format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam](variables.tf#L87) | Custom IAM settings in role => [principal] format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam_additive](variables.tf#L93) | Custom additive IAM settings in role => [principal] format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [kms_service_agents](variables.tf#L99) | KMS IAM configuration in as service => [key]. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [labels](variables.tf#L105) | Labels to be assigned at project level. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> |
| [org_policies](variables.tf#L111) | Org-policy overrides at project level. | <code title="map&#40;object&#40;&#123;&#10; inherit_from_parent &#61; optional&#40;bool&#41; &#35; for list policies only.&#10; reset &#61; optional&#40;bool&#41;&#10; allow &#61; optional&#40;object&#40;&#123;&#10; all &#61; optional&#40;bool&#41;&#10; values &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#10; deny &#61; optional&#40;object&#40;&#123;&#10; all &#61; optional&#40;bool&#41;&#10; values &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#10; enforce &#61; optional&#40;bool, true&#41; &#35; for boolean policies only.&#10; rules &#61; optional&#40;list&#40;object&#40;&#123;&#10; allow &#61; optional&#40;object&#40;&#123;&#10; all &#61; optional&#40;bool&#41;&#10; values &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#10; deny &#61; optional&#40;object&#40;&#123;&#10; all &#61; optional&#40;bool&#41;&#10; values &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#10; enforce &#61; optional&#40;bool, true&#41; &#35; for boolean policies only.&#10; condition &#61; object&#40;&#123;&#10; description &#61; optional&#40;string&#41;&#10; expression &#61; optional&#40;string&#41;&#10; location &#61; optional&#40;string&#41;&#10; title &#61; optional&#40;string&#41;&#10; &#125;&#41;&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [prefix](variables.tf#L151) | Prefix used for the project id. | <code>string</code> | | <code>null</code> |
| [service_accounts](variables.tf#L162) | Service accounts to be created, and roles assigned them on the project. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_accounts_additive](variables.tf#L168) | Service accounts to be created, and roles assigned them on the project additively. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_accounts_iam](variables.tf#L174) | IAM bindings on service account resources. Format is KEY => {ROLE => [MEMBERS]} | <code>map&#40;map&#40;list&#40;string&#41;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_accounts_iam_additive](variables.tf#L181) | IAM additive bindings on service account resources. Format is KEY => {ROLE => [MEMBERS]} | <code>map&#40;map&#40;list&#40;string&#41;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_identities_iam](variables.tf#L195) | Custom IAM settings for service identities in service => [role] format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_identities_iam_additive](variables.tf#L202) | Custom additive IAM settings for service identities in service => [role] format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [services](variables.tf#L188) | Services to be enabled for the project. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [vpc](variables.tf#L209) | VPC configuration for the project. | <code title="object&#40;&#123;&#10; host_project &#61; string&#10; gke_setup &#61; object&#40;&#123;&#10; enable_security_admin &#61; bool&#10; enable_host_service_agent &#61; bool&#10; &#125;&#41;&#10; subnets_iam &#61; map&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
## Outputs

48
blueprints/factories/project-factory/main.tf
View File

@ -21,7 +21,14 @@ locals {
"group:${k}" if try(index(v, r), null) != null
]
}
_group_iam_bindings = distinct(flatten(values(var.group_iam)))
_group_iam_additive = {
for r in local._group_iam_additive_bindings : r => [
for k, v in var.group_iam_additive :
"group:${k}" if try(index(v, r), null) != null
]
}
_group_iam_bindings = distinct(flatten(values(var.group_iam)))
_group_iam_additive_bindings = distinct(flatten(values(var.group_iam_additive)))
_project_id = (
var.prefix == null || var.prefix == ""
? var.project_id
@ -37,9 +44,20 @@ locals {
_service_accounts_iam_bindings = distinct(flatten(
values(var.service_accounts)
))
_service_accounts_iam_additive = {
for r in local._service_accounts_iam_additive_bindings : r => [
for k, v in var.service_accounts_additive :
module.service-accounts[k].iam_email
if try(index(v, r), null) != null
]
}
_service_accounts_iam_additive_bindings = distinct(flatten(
values(var.service_accounts_additive)
))
_services = concat([
"billingbudgets.googleapis.com",
"essentialcontacts.googleapis.com"
"essentialcontacts.googleapis.com",
"orgpolicy.googleapis.com",
],
length(var.dns_zones) > 0 ? ["dns.googleapis.com"] : [],
try(var.vpc.gke_setup, null) != null ? ["container.googleapis.com"] : [],
@ -53,6 +71,14 @@ locals {
if contains(roles, role)
]
}
_service_identities_roles_additive = distinct(flatten(values(var.service_identities_iam_additive)))
_service_identities_iam_additive = {
for role in local._service_identities_roles_additive : role => [
for service, roles in var.service_identities_iam_additive :
"serviceAccount:${module.project.service_accounts.robots[service]}"
if contains(roles, role)
]
}
_vpc_subnet_bindings = (
local.vpc.subnets_iam == null || local.vpc.host_project == null
? []
@ -91,6 +117,20 @@ locals {
try(local._service_identities_iam[role], []),
)
}
iam_additive = {
for role in distinct(concat(
keys(var.iam_additive),
keys(local._group_iam_additive),
keys(local._service_accounts_iam_additive),
keys(local._service_identities_iam_additive),
)) :
role => concat(
try(var.iam_additive[role], []),
try(local._group_iam_additive[role], []),
try(local._service_accounts_iam_additive[role], []),
try(local._service_identities_iam_additive[role], []),
)
}
labels = merge(
coalesce(var.labels, {}), coalesce(try(var.defaults.labels, {}), {})
)
@ -147,10 +187,10 @@ module "project" {
prefix = var.prefix
contacts = { for c in local.essential_contacts : c => ["ALL"] }
iam = local.iam
iam_additive = local.iam_additive
labels = local.labels
org_policies = try(var.org_policies, {})
parent = var.folder_id
policy_boolean = try(var.org_policies.policy_boolean, {})
policy_list = try(var.org_policies.policy_list, {})
service_encryption_key_ids = var.kms_service_agents
services = local.services
shared_vpc_service_config = var.vpc == null ? null : {

14
blueprints/factories/project-factory/sample-data/projects/project.yaml
View File

@ -48,15 +48,15 @@ labels:
# [opt] Org policy overrides defined at project level
org_policies:
policy_boolean:
constraints/compute.disableGuestAttributesAccess: true
policy_list:
constraints/compute.trustedImageProjects:
inherit_from_parent: null
status: true
suggested_value: null
constraints/compute.disableGuestAttributesAccess:
enforce: true
constraints/compute.trustedImageProjects:
allow:
values:
- projects/fast-dev-iac-core-0
constraints/compute.vmExternalIpAccess:
deny:
all: true
# [opt] Service account to create for the project and their roles on the project
# in name => [roles] format

79
blueprints/factories/project-factory/variables.tf
View File

@ -78,12 +78,24 @@ variable "group_iam" {
default = {}
}
variable "group_iam_additive" {
description = "Custom additive IAM settings in group => [role] format."
type = map(list(string))
default = {}
}
variable "iam" {
description = "Custom IAM settings in role => [principal] format."
type = map(list(string))
default = {}
}
variable "iam_additive" {
description = "Custom additive IAM settings in role => [principal] format."
type = map(list(string))
default = {}
}
variable "kms_service_agents" {
description = "KMS IAM configuration in as service => [key]."
type = map(list(string))
@ -98,16 +110,42 @@ variable "labels" {
variable "org_policies" {
description = "Org-policy overrides at project level."
type = object({
policy_boolean = map(bool)
policy_list = map(object({
inherit_from_parent = bool
suggested_value = string
status = bool
values = list(string)
type = map(object({
inherit_from_parent = optional(bool) # for list policies only.
reset = optional(bool)
# default (unconditional) values
allow = optional(object({
all = optional(bool)
values = optional(list(string))
}))
})
default = null
deny = optional(object({
all = optional(bool)
values = optional(list(string))
}))
enforce = optional(bool, true) # for boolean policies only.
# conditional values
rules = optional(list(object({
allow = optional(object({
all = optional(bool)
values = optional(list(string))
}))
deny = optional(object({
all = optional(bool)
values = optional(list(string))
}))
enforce = optional(bool, true) # for boolean policies only.
condition = object({
description = optional(string)
expression = optional(string)
location = optional(string)
title = optional(string)
})
})), [])
}))
default = {}
nullable = false
}
variable "prefix" {
@ -127,6 +165,12 @@ variable "service_accounts" {
default = {}
}
variable "service_accounts_additive" {
description = "Service accounts to be created, and roles assigned them on the project additively."
type = map(list(string))
default = {}
}
variable "service_accounts_iam" {
description = "IAM bindings on service account resources. Format is KEY => {ROLE => [MEMBERS]}"
type = map(map(list(string)))
@ -134,6 +178,13 @@ variable "service_accounts_iam" {
nullable = false
}
variable "service_accounts_iam_additive" {
description = "IAM additive bindings on service account resources. Format is KEY => {ROLE => [MEMBERS]}"
type = map(map(list(string)))
default = {}
nullable = false
}
variable "services" {
description = "Services to be enabled for the project."
type = list(string)
@ -148,6 +199,13 @@ variable "service_identities_iam" {
nullable = false
}
variable "service_identities_iam_additive" {
description = "Custom additive IAM settings for service identities in service => [role] format."
type = map(list(string))
default = {}
nullable = false
}
variable "vpc" {
description = "VPC configuration for the project."
type = object({
@ -160,6 +218,3 @@ variable "vpc" {
})
default = null
}

58
blueprints/gke/multi-cluster-mesh-gke-fleet-api/vpc.tf
View File

@ -40,36 +40,36 @@ module "firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.host_project.project_id
network = module.svpc.name
custom_rules = merge({ allow-mesh = {
description = "Allow "
direction = "INGRESS"
action = "allow"
sources = []
ranges = [for k, v in var.clusters_config : v.pods_cidr_block]
targets = [for k, v in var.clusters_config : "${k}-node"]
use_service_accounts = false
rules = [{ protocol = "tcp", ports = null },
{ protocol = "udp", ports = null },
{ protocol = "icmp", ports = null },
{ protocol = "esp", ports = null },
{ protocol = "ah", ports = null },
{ protocol = "sctp", ports = null }]
extra_attributes = {
priority = 900
}
} },
{ for k, v in var.clusters_config : "allow-${k}-istio" => {
description = "Allow "
direction = "INGRESS"
action = "allow"
sources = []
ranges = [v.master_cidr_block]
targets = ["${k}-node"]
use_service_accounts = false
rules = [{ protocol = "tcp", ports = [8080, 15014, 15017] }]
extra_attributes = {
priority = 1000
ingress_rules = merge(
{
allow-mesh = {
description = "Allow mesh."
priority = 900
source_ranges = [
for k, v in var.clusters_config : v.pods_cidr_block
]
targets = [
for k, v in var.clusters_config : "${k}-node"
]
rules = [
{ protocol = "tcp" },
{ protocol = "udp" },
{ protocol = "icmp" },
{ protocol = "esp" },
{ protocol = "ah" },
{ protocol = "sctp" }
]
}
},
{
for k, v in var.clusters_config : "allow-${k}-istio" => {
description = "Allow istio."
source_ranges = [v.master_cidr_block]
targets = ["${k}-node"]
rules = [{
protocol = "tcp"
ports = [8080, 15014, 15017]
}]
}
}
)

90
blueprints/networking/filtering-proxy/main.tf
View File

@ -74,17 +74,18 @@ module "firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project-host.project_id
network = module.vpc.name
custom_rules = {
ingress_rules = {
allow-ingress-squid = {
description = "Allow squid ingress traffic"
direction = "INGRESS"
action = "allow"
sources = []
ranges = [var.cidrs.apps, "35.191.0.0/16", "130.211.0.0/22"]
description = "Allow squid ingress traffic"
source_ranges = [
var.cidrs.apps, "35.191.0.0/16", "130.211.0.0/22"
]
targets = [module.service-account-squid.email]
use_service_accounts = true
rules = [{ protocol = "tcp", ports = [3128] }]
extra_attributes = {}
rules = [{
protocol = "tcp"
ports = [3128]
}]
}
}
}
@ -165,33 +166,31 @@ module "squid-vm" {
}
module "squid-mig" {
count = var.mig ? 1 : 0
source = "../../../modules/compute-mig"
project_id = module.project-host.project_id
location = "${var.region}-b"
name = "squid-mig"
target_size = 1
autoscaler_config = {
max_replicas = 10
min_replicas = 1
cooldown_period = 30
cpu_utilization_target = 0.65
load_balancing_utilization_target = null
metric = null
count = var.mig ? 1 : 0
source = "../../../modules/compute-mig"
project_id = module.project-host.project_id
location = "${var.region}-b"
name = "squid-mig"
instance_template = module.squid-vm.template.self_link
target_size = 1
auto_healing_policies = {
initial_delay_sec = 60
}
default_version = {
instance_template = module.squid-vm.template.self_link
name = "default"
autoscaler_config = {
max_replicas = 10
min_replicas = 1
cooldown_period = 30
scaling_signals = {
cpu_utilization = {
target = 0.65
}
}
}
health_check_config = {
type = "tcp"
check = { port = 3128 }
config = {}
logging = true
}
auto_healing_policies = {
health_check = module.squid-mig.0.health_check.self_link
initial_delay_sec = 60
enable_logging = true
tcp = {
port = 3128
}
}
}
@ -201,20 +200,20 @@ module "squid-ilb" {
project_id = module.project-host.project_id
region = var.region
name = "squid-ilb"
service_label = "squid-ilb"
network = module.vpc.self_link
subnetwork = module.vpc.subnet_self_links["${var.region}/proxy"]
ports = [3128]
service_label = "squid-ilb"
vpc_config = {
network = module.vpc.self_link
subnetwork = module.vpc.subnet_self_links["${var.region}/proxy"]
}
backends = [{
failover = false
group = module.squid-mig.0.group_manager.instance_group
balancing_mode = "CONNECTION"
group = module.squid-mig.0.group_manager.instance_group
}]
health_check_config = {
type = "tcp"
check = { port = 3128 }
config = {}
logging = true
enable_logging = true
tcp = {
port = 3128
}
}
}
@ -226,13 +225,10 @@ module "folder-apps" {
source = "../../../modules/folder"
parent = var.root_node
name = "apps"
policy_list = {
org_policies = {
# prevent VMs with public IPs in the apps folder
"constraints/compute.vmExternalIpAccess" = {
inherit_from_parent = false
suggested_value = null
status = false
values = []
deny = { all = true }
}
}
}

56
blueprints/networking/glb-and-armor/main.tf
View File

@ -153,22 +153,20 @@ module "vm_siege" {
}
module "mig_ew1" {
source = "../../../modules/compute-mig"
project_id = module.project.project_id
location = "europe-west1"
name = "${local.prefix}europe-west1-mig"
regional = true
default_version = {
instance_template = module.instance_template_ew1.template.self_link
name = "default"
}
source = "../../../modules/compute-mig"
project_id = module.project.project_id
location = "europe-west1"
name = "${local.prefix}europe-west1-mig"
instance_template = module.instance_template_ew1.template.self_link
autoscaler_config = {
max_replicas = 5
min_replicas = 1
cooldown_period = 45
cpu_utilization_target = 0.8
load_balancing_utilization_target = null
metric = null
max_replicas = 5
min_replicas = 1
cooldown_period = 45
scaling_signals = {
cpu_utilization = {
target = 0.65
}
}
}
named_ports = {
http = 80
@ -179,22 +177,20 @@ module "mig_ew1" {
}
module "mig_ue1" {
source = "../../../modules/compute-mig"
project_id = module.project.project_id
location = "us-east1"
name = "${local.prefix}us-east1-mig"
regional = true
default_version = {
instance_template = module.instance_template_ue1.template.self_link
name = "default"
}
source = "../../../modules/compute-mig"
project_id = module.project.project_id
location = "us-east1"
name = "${local.prefix}us-east1-mig"
instance_template = module.instance_template_ue1.template.self_link
autoscaler_config = {
max_replicas = 5
min_replicas = 1
cooldown_period = 45
cpu_utilization_target = 0.8
load_balancing_utilization_target = null
metric = null
max_replicas = 5
min_replicas = 1
cooldown_period = 45
scaling_signals = {
cpu_utilization = {
target = 0.65
}
}
}
named_ports = {
http = 80

30
blueprints/networking/hub-and-spoke-peering/main.tf
View File

@ -69,10 +69,12 @@ module "nat-hub" {
}
module "vpc-hub-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.vpc-hub.name
admin_ranges = values(var.ip_ranges)
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.vpc-hub.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
}
}
################################################################################
@ -93,10 +95,12 @@ module "vpc-spoke-1" {
}
module "vpc-spoke-1-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc-spoke-1.name
admin_ranges = values(var.ip_ranges)
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc-spoke-1.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
}
}
module "nat-spoke-1" {
@ -138,10 +142,12 @@ module "vpc-spoke-2" {
}
module "vpc-spoke-2-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc-spoke-2.name
admin_ranges = values(var.ip_ranges)
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc-spoke-2.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
}
}
module "nat-spoke-2" {

10
blueprints/networking/hub-and-spoke-vpn/net-dev.tf
View File

@ -39,10 +39,12 @@ module "dev-vpc" {
}
module "dev-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.dev-vpc.name
admin_ranges = values(var.ip_ranges)
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.dev-vpc.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
}
}
module "dev-dns-peering" {

10
blueprints/networking/hub-and-spoke-vpn/net-landing.tf
View File

@ -39,10 +39,12 @@ module "landing-vpc" {
}
module "landing-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.landing-vpc.name
admin_ranges = values(var.ip_ranges)
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.landing-vpc.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
}
}
module "landing-dns-zone" {

10
blueprints/networking/hub-and-spoke-vpn/net-prod.tf
View File

@ -39,10 +39,12 @@ module "prod-vpc" {
}
module "prod-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.prod-vpc.name
admin_ranges = values(var.ip_ranges)
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.prod-vpc.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
}
}
module "prod-dns-peering" {

48
blueprints/networking/ilb-next-hop/gateways.tf
View File

@ -62,22 +62,22 @@ module "ilb-left" {
project_id = module.project.project_id
region = var.region
name = "${local.prefix}ilb-left"
network = module.vpc-left.self_link
subnetwork = values(module.vpc-left.subnet_self_links)[0]
address = local.addresses.ilb-left
ports = null
backend_config = {
session_affinity = var.ilb_session_affinity
timeout_sec = null
connection_draining_timeout_sec = null
vpc_config = {
network = module.vpc-left.self_link
subnetwork = values(module.vpc-left.subnet_self_links)[0]
}
address = local.addresses.ilb-left
backend_service_config = {
session_affinity = var.ilb_session_affinity
}
backends = [for z, mod in module.gw : {
failover = false
group = mod.group.self_link
balancing_mode = "CONNECTION"
group = mod.group.self_link
}]
health_check_config = {
type = "tcp", check = { port = 22 }, config = {}, logging = true
enable_logging = true
tcp = {
port = 22
}
}
}
@ -86,21 +86,21 @@ module "ilb-right" {
project_id = module.project.project_id
region = var.region
name = "${local.prefix}ilb-right"
network = module.vpc-right.self_link
subnetwork = values(module.vpc-right.subnet_self_links)[0]
address = local.addresses.ilb-right
ports = null
backend_config = {
session_affinity = var.ilb_session_affinity
timeout_sec = null
connection_draining_timeout_sec = null
vpc_config = {
network = module.vpc-right.self_link
subnetwork = values(module.vpc-right.subnet_self_links)[0]
}
address = local.addresses.ilb-right
backend_service_config = {
session_affinity = var.ilb_session_affinity
}
backends = [for z, mod in module.gw : {
failover = false
group = mod.group.self_link
balancing_mode = "CONNECTION"
group = mod.group.self_link
}]
health_check_config = {
type = "tcp", check = { port = 22 }, config = {}, logging = true
enable_logging = true
tcp = {
port = 22
}
}
}

12
blueprints/networking/ilb-next-hop/vpc-left.tf
View File

@ -35,11 +35,13 @@ module "vpc-left" {
}
module "firewall-left" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc-left.name
admin_ranges = values(var.ip_ranges)
ssh_source_ranges = ["35.235.240.0/20", "35.191.0.0/16", "130.211.0.0/22"]
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc-left.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
ssh_ranges = ["35.235.240.0/20", "35.191.0.0/16", "130.211.0.0/22"]
}
}
module "nat-left" {

12
blueprints/networking/ilb-next-hop/vpc-right.tf
View File

@ -46,11 +46,13 @@ module "vpc-right" {
}
module "firewall-right" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc-right.name
admin_ranges = values(var.ip_ranges)
ssh_source_ranges = ["35.235.240.0/20", "35.191.0.0/16", "130.211.0.0/22"]
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc-right.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
ssh_ranges = ["35.235.240.0/20", "35.191.0.0/16", "130.211.0.0/22"]
}
}
module "nat-right" {

18
blueprints/networking/nginx-reverse-proxy-cluster/main.tf
View File

@ -161,28 +161,22 @@ module "firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project.project_id
network = module.vpc.name
custom_rules = {
ingress_rules = {
format("%sallow-http-to-proxy-cluster", var.prefix) = {
description = "Allow Nginx HTTP(S) ingress traffic"
direction = "INGRESS"
action = "allow"
sources = []
ranges = [var.cidrs[var.subnetwork], "35.191.0.0/16", "130.211.0.0/22"]
description = "Allow Nginx HTTP(S) ingress traffic"
source_ranges = [
var.cidrs[var.subnetwork], "35.191.0.0/16", "130.211.0.0/22"
]
targets = [module.service-account-proxy.email]
use_service_accounts = true
rules = [{ protocol = "tcp", ports = [80, 443] }]
extra_attributes = {}
}
format("%sallow-iap-ssh", var.prefix) = {
description = "Allow Nginx SSH traffic from IAP"
direction = "INGRESS"
action = "allow"
sources = []
ranges = ["35.235.240.0/20"]
source_ranges = ["35.235.240.0/20"]
targets = [module.service-account-proxy.email]
use_service_accounts = true
rules = [{ protocol = "tcp", ports = [22] }]
extra_attributes = {}
}
}
}

12
blueprints/networking/onprem-google-access-dns/main.tf
View File

@ -69,11 +69,13 @@ module "vpc" {
}
module "vpc-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.vpc.name
admin_ranges = values(var.ip_ranges)
ssh_source_ranges = var.ssh_source_ranges
source = "../../../modules/net-vpc-firewall"
project_id = var.project_id
network = module.vpc.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
ssh_ranges = var.ssh_source_ranges
}
}
module "vpn1" {

10
blueprints/networking/shared-vpc-gke/main.tf
View File

@ -130,10 +130,12 @@ module "vpc-shared" {
}
module "vpc-shared-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.project-host.project_id
network = module.vpc-shared.name
admin_ranges = values(var.ip_ranges)
source = "../../../modules/net-vpc-firewall"
project_id = module.project-host.project_id
network = module.vpc-shared.name
default_rules_config = {
admin_ranges = values(var.ip_ranges)
}
}
module "nat" {

25
fast/assets/schemas/firewall_rules.schema.yaml
View File

@ -12,18 +12,23 @@
# See the License for the specific language governing permissions and
# limitations under the License.
map(include('firewall_rule'))
egress: map(include('firewall_rule'), required=False)
ingress: map(include('firewall_rule'), required=False)
---
firewall_rule:
description: str()
direction: enum("INGRESS", "EGRESS")
action: enum("allow", "deny")
sources: list(str())
ranges: list(str())
targets: list(str())
use_service_accounts: bool()
rules: list(include('rule'))
deny: bool(required=False)
description: str(required=False)
destination_ranges: list(str(), required=False)
disabled: bool(required=False)
# enable_logging:
# include_metadata: bool(required=False)
priority: int(required=False)
source_ranges: list(str(), required=False)
sources: list(str(), required=False)
targets: list(str(), required=False)
use_service_accounts: bool(required=False)
rules: list(include('rule'), required=False)
---
rule:
protocol: enum("tcp", "udp", "all")
protocol: str()
ports: list(num())

2
fast/assets/templates/workflow-github.yaml
View File

@ -30,7 +30,7 @@ env:
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
TF_PROVIDERS_FILE: ${tf_providers_file}
TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
TF_VERSION: 1.1.7
TF_VERSION: 1.3.2
jobs:
fast-pr:

2
fast/assets/templates/workflow-sourcerepo.yaml
View File

@ -95,4 +95,4 @@ substitutions:
_FAST_OUTPUTS_BUCKET: ${outputs_bucket}
_TF_PROVIDERS_FILE: ${tf_providers_file}
_TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
_TF_VERSION: 1.1.7
_TF_VERSION: 1.3.2

23
fast/extras/00-cicd-github/README.md
View File

@ -8,11 +8,16 @@ This stage is designed for quick repository creation in a GitHub organization, a
Initial file population of repositories is controlled via the `populate_from` attribute, and needs a bit of care:
- never run this stage gain with the same variables used for population once the repository starts being used, as **Terraform will manage file state and revert any changes at each apply**, which is probably not what you want.
- be mindful when enabling initial population of the modules repository, as the number of resulting files to manage is very close to the GitHub hourly limit for their API
- never run this stage with the same variables used for population once the repository starts being used, as **Terraform will manage file state and revert any changes at each apply**, which is probably not what you want.
- initial population of the modules repository is discouraged, as the number of resulting files Terraform needs to manage is very close to the GitHub hourly limit for their API, it's much easier to populate modules via regular git commands
The scenario for which this stage has been designed is one-shot creation and/or population of stage repositories, running it multiple times with different variables and Terraform states if incremental creation is needed for subsequent FAST stages (e.g. GKE, data platform, etc.).
Once initial population is done, you need to manually push to the repository
- the `.tfvars` file with custom variable values for your stages
- the workflow configuration file generated by FAST stages
## GitHub provider credentials
A [GitHub token](https://github.com/settings/tokens) is needed to authenticate against their API. The token needs organization-level permissions, like shown in this screenshot:
@ -77,7 +82,8 @@ When initial population is configured for a repository, this stage also adds a s
| name | description | resources |
|---|---|---|
| [cicd-versions.tf](./cicd-versions.tf) | Provider version. | |
| [main.tf](./main.tf) | Module-level locals and resources. | <code>github_actions_secret</code> · <code>github_repository</code> · <code>github_repository_file</code> · <code>tls_private_key</code> |
| [main.tf](./main.tf) | Module-level locals and resources. | <code>github_actions_secret</code> · <code>github_repository</code> · <code>github_repository_deploy_key</code> · <code>github_repository_file</code> · <code>tls_private_key</code> |
| [outputs.tf](./outputs.tf) | Module outputs. | |
| [providers.tf](./providers.tf) | Provider configuration. | |
| [variables.tf](./variables.tf) | Module variables. | |
@ -85,8 +91,15 @@ When initial population is configured for a repository, this stage also adds a s
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [organization](variables.tf#L28) | GitHub organization. | <code>string</code> | ✓ | |
| [organization](variables.tf#L34) | GitHub organization. | <code>string</code> | ✓ | |
| [commmit_config](variables.tf#L17) | Configure commit metadata. | <code title="object&#40;&#123;&#10; author &#61; optional&#40;string, &#34;FAST loader&#34;&#41;&#10; email &#61; optional&#40;string, &#34;fast-loader&#64;fast.gcp.tf&#34;&#41;&#10; message &#61; optional&#40;string, &#34;FAST initial loading&#34;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [repositories](variables.tf#L33) | Repositories to create. | <code title="map&#40;object&#40;&#123;&#10; create_options &#61; optional&#40;object&#40;&#123;&#10; allow &#61; optional&#40;object&#40;&#123;&#10; auto_merge &#61; optional&#40;bool&#41;&#10; merge_commit &#61; optional&#40;bool&#41;&#10; rebase_merge &#61; optional&#40;bool&#41;&#10; squash_merge &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; auto_init &#61; optional&#40;bool&#41;&#10; description &#61; optional&#40;string&#41;&#10; features &#61; optional&#40;object&#40;&#123;&#10; issues &#61; optional&#40;bool&#41;&#10; projects &#61; optional&#40;bool&#41;&#10; wiki &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; templates &#61; optional&#40;object&#40;&#123;&#10; gitignore &#61; optional&#40;string, &#34;Terraform&#34;&#41;&#10; license &#61; optional&#40;string&#41;&#10; repository &#61; optional&#40;object&#40;&#123;&#10; name &#61; string&#10; owner &#61; string&#10; &#125;&#41;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; visibility &#61; optional&#40;string, &#34;private&#34;&#41;&#10; &#125;&#41;&#41;&#10; has_modules &#61; optional&#40;bool, false&#41;&#10; populate_from &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [modules_ref](variables.tf#L28) | Optional git ref used in module sources. | <code>string</code> | | <code>null</code> |
| [repositories](variables.tf#L39) | Repositories to create. | <code title="map&#40;object&#40;&#123;&#10; create_options &#61; optional&#40;object&#40;&#123;&#10; allow &#61; optional&#40;object&#40;&#123;&#10; auto_merge &#61; optional&#40;bool&#41;&#10; merge_commit &#61; optional&#40;bool&#41;&#10; rebase_merge &#61; optional&#40;bool&#41;&#10; squash_merge &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; auto_init &#61; optional&#40;bool&#41;&#10; description &#61; optional&#40;string&#41;&#10; features &#61; optional&#40;object&#40;&#123;&#10; issues &#61; optional&#40;bool&#41;&#10; projects &#61; optional&#40;bool&#41;&#10; wiki &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; templates &#61; optional&#40;object&#40;&#123;&#10; gitignore &#61; optional&#40;string, &#34;Terraform&#34;&#41;&#10; license &#61; optional&#40;string&#41;&#10; repository &#61; optional&#40;object&#40;&#123;&#10; name &#61; string&#10; owner &#61; string&#10; &#125;&#41;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; visibility &#61; optional&#40;string, &#34;private&#34;&#41;&#10; &#125;&#41;&#41;&#10; has_modules &#61; optional&#40;bool, false&#41;&#10; populate_from &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| [clone](outputs.tf#L17) | Clone repository commands. | |
<!-- END TFDOC -->

45
fast/extras/00-cicd-github/main.tf
View File

@ -30,6 +30,7 @@ locals {
}
] if v.populate_from != null
])
modules_ref = var.modules_ref == null ? "" : "?ref=${var.modules_ref}"
modules_repository = (
length(local._modules_repository) > 0
? local._modules_repository.0
@ -39,13 +40,24 @@ locals {
for k, v in var.repositories :
k => v.create_options == null ? k : github_repository.default[k].name
}
repository_files = {
for k in local._repository_files :
"${k.repository}/${k.name}" => k
if !endswith(k.name, ".tf") || (
!startswith(k.name, "0") && k.name != "globals.tf"
)
}
repository_files = merge(
{
for k in local._repository_files :
"${k.repository}/${k.name}" => k
if !endswith(k.name, ".tf") || (
!startswith(k.name, "0") && k.name != "globals.tf"
)
},
{
for k, v in var.repositories :
"${k}/templates/providers.tf.tpl" => {
repository = k
file = "../../assets/templates/providers.tf.tpl"
name = "templates/providers.tf.tpl"
}
if v.populate_from != null
}
)
}
resource "github_repository" "default" {
@ -88,15 +100,20 @@ resource "tls_private_key" "default" {
algorithm = "ED25519"
}
resource "github_repository_deploy_key" "exdefaultample_repository_deploy_key" {
count = local.modules_repository == null ? 0 : 1
title = "Modules repository access"
repository = local.modules_repository
key = tls_private_key.default.0.public_key_openssh
read_only = true
}
resource "github_actions_secret" "default" {
for_each = local.modules_repository == null ? {} : {
for k, v in local.repositories :
k => v if(
k != local.modules_repository &&
var.repositories[k].populate_from != null
)
k => v if k != local.modules_repository
}
repository = local.repositories[local.modules_repository]
repository = each.key
secret_name = "CICD_MODULES_KEY"
plaintext_value = tls_private_key.default.0.private_key_openssh
}
@ -112,8 +129,8 @@ resource "github_repository_file" "default" {
endswith(each.value.name, ".tf") && local.modules_repository != null
? replace(
file(each.value.file),
"/source\\s*=\\s*\"../../../",
"source = \"git@github.com:${var.organization}/${local.modules_repository}.git/"
"/source\\s*=\\s*\"../../../modules/([^/\"]+)\"/",
"source = \"git@github.com:${var.organization}/${local.modules_repository}.git//$1${local.modules_ref}\"" # "
)
: file(each.value.file)
)

9
modules/organization-policy/outputs.tf → fast/extras/00-cicd-github/outputs.tf
View File

@ -14,7 +14,10 @@
* limitations under the License.
*/
output "policies" {
description = "Organization policies."
value = google_org_policy_policy.primary
output "clone" {
description = "Clone repository commands."
value = {
for k, v in var.repositories :
k => "git clone git@github.com:${var.organization}/${k}.git"
}
}

6
fast/extras/00-cicd-github/variables.tf
View File

@ -25,6 +25,12 @@ variable "commmit_config" {
nullable = false
}
variable "modules_ref" {
description = "Optional git ref used in module sources."
type = string
default = null
}
variable "organization" {
description = "GitHub organization."
type = string

1
fast/stages/00-bootstrap/README.md
View File

@ -262,6 +262,7 @@ terraform init
terraform apply \
-var bootstrap_user=$(gcloud config list --format 'value(core.account)')
```
> If you see an error related to project name already exists, please make sure the project name is unique or the project was not deleted recently
Once the initial `apply` completes successfully, configure a remote backend using the new GCS bucket, and impersonation on the automation service account for this stage. To do this you can use the generated `providers.tf` file if you have configured output files as described above, or extract its contents from Terraform's output, then migrate state with `terraform init`:

31
fast/stages/00-bootstrap/automation.tf
View File

@ -72,6 +72,7 @@ module "automation-project" {
"essentialcontacts.googleapis.com",
"iam.googleapis.com",
"iamcredentials.googleapis.com",
"orgpolicy.googleapis.com",
"pubsub.googleapis.com",
"servicenetworking.googleapis.com",
"serviceusage.googleapis.com",
@ -110,11 +111,11 @@ module "automation-tf-bootstrap-gcs" {
}
module "automation-tf-bootstrap-sa" {
source = "../../../modules/iam-service-account"
project_id = module.automation-project.project_id
name = "bootstrap-0"
description = "Terraform organization bootstrap service account."
prefix = local.prefix
source = "../../../modules/iam-service-account"
project_id = module.automation-project.project_id
name = "bootstrap-0"
display_name = "Terraform organization bootstrap service account."
prefix = local.prefix
# allow SA used by CI/CD workflow to impersonate this SA
iam = {
"roles/iam.serviceAccountTokenCreator" = compact([
@ -143,11 +144,11 @@ module "automation-tf-cicd-gcs" {
}
module "automation-tf-cicd-provisioning-sa" {
source = "../../../modules/iam-service-account"
project_id = module.automation-project.project_id
name = "cicd-0"
description = "Terraform stage 1 CICD service account."
prefix = local.prefix
source = "../../../modules/iam-service-account"
project_id = module.automation-project.project_id
name = "cicd-0"
display_name = "Terraform stage 1 CICD service account."
prefix = local.prefix
# allow SA used by CI/CD workflow to impersonate this SA
iam = {
"roles/iam.serviceAccountTokenCreator" = compact([
@ -176,11 +177,11 @@ module "automation-tf-resman-gcs" {
}
module "automation-tf-resman-sa" {
source = "../../../modules/iam-service-account"
project_id = module.automation-project.project_id
name = "resman-0"
description = "Terraform stage 1 resman service account."
prefix = local.prefix
source = "../../../modules/iam-service-account"
project_id = module.automation-project.project_id
name = "resman-0"
display_name = "Terraform stage 1 resman service account."
prefix = local.prefix
# allow SA used by CI/CD workflow to impersonate this SA
iam = {
"roles/iam.serviceAccountTokenCreator" = compact([

12
fast/stages/00-bootstrap/cicd.tf
View File

@ -87,12 +87,12 @@ module "automation-tf-cicd-repo" {
# SAs used by CI/CD workflows to impersonate automation SAs
module "automation-tf-cicd-sa" {
source = "../../../modules/iam-service-account"
for_each = local.cicd_repositories
project_id = module.automation-project.project_id
name = "${each.key}-1"
description = "Terraform CI/CD ${each.key} service account."
prefix = local.prefix
source = "../../../modules/iam-service-account"
for_each = local.cicd_repositories
project_id = module.automation-project.project_id
name = "${each.key}-1"
display_name = "Terraform CI/CD ${each.key} service account."
prefix = local.prefix
iam = (
each.value.type == "sourcerepo"
# used directly from the cloud build trigger for source repos

14
fast/stages/00-bootstrap/outputs-files.tf
View File

@ -19,27 +19,27 @@
resource "local_file" "providers" {
for_each = var.outputs_location == null ? {} : local.providers
file_permission = "0644"
filename = "${pathexpand(var.outputs_location)}/providers/${each.key}-providers.tf"
content = each.value
filename = "${try(pathexpand(var.outputs_location), "")}/providers/${each.key}-providers.tf"
content = try(each.value, null)
}
resource "local_file" "tfvars" {
for_each = var.outputs_location == null ? {} : { 1 = 1 }
file_permission = "0644"
filename = "${pathexpand(var.outputs_location)}/tfvars/00-bootstrap.auto.tfvars.json"
filename = "${try(pathexpand(var.outputs_location), "")}/tfvars/00-bootstrap.auto.tfvars.json"
content = jsonencode(local.tfvars)
}
resource "local_file" "tfvars_globals" {
for_each = var.outputs_location == null ? {} : { 1 = 1 }
file_permission = "0644"
filename = "${pathexpand(var.outputs_location)}/tfvars/globals.auto.tfvars.json"
filename = "${try(pathexpand(var.outputs_location), "")}/tfvars/globals.auto.tfvars.json"
content = jsonencode(local.tfvars_globals)
}
resource "local_file" "workflows" {
for_each = local.cicd_workflows
for_each = var.outputs_location == null ? {} : local.cicd_workflows
file_permission = "0644"
filename = "${pathexpand(var.outputs_location)}/workflows/${each.key}-workflow.yaml"
content = each.value
filename = "${try(pathexpand(var.outputs_location), "")}/workflows/${each.key}-workflow.yaml"
content = try(each.value, null)
}

19
fast/stages/01-resman/README.md
View File

@ -182,17 +182,18 @@ Due to its simplicity, this stage lends itself easily to customizations: adding
|---|---|:---:|:---:|:---:|:---:|
| [automation](variables.tf#L20) | Automation resources created by the bootstrap stage. | <code title="object&#40;&#123;&#10; outputs_bucket &#61; string&#10; project_id &#61; string&#10; project_number &#61; string&#10; federated_identity_pool &#61; string&#10; federated_identity_providers &#61; map&#40;object&#40;&#123;&#10; issuer &#61; string&#10; issuer_uri &#61; string&#10; name &#61; string&#10; principal_tpl &#61; string&#10; principalset_tpl &#61; string&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| [billing_account](variables.tf#L38) | Billing account id and organization id ('nnnnnnnn' or null). | <code title="object&#40;&#123;&#10; id &#61; string&#10; organization_id &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| [organization](variables.tf#L191) | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; number&#10; customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| [prefix](variables.tf#L215) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
| [organization](variables.tf#L197) | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; number&#10; customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| [prefix](variables.tf#L221) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
| [cicd_repositories](variables.tf#L47) | CI/CD repository configuration. Identity providers reference keys in the `automation.federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | <code title="object&#40;&#123;&#10; data_platform_dev &#61; object&#40;&#123;&#10; branch &#61; string&#10; identity_provider &#61; string&#10; name &#61; string&#10; type &#61; string&#10; &#125;&#41;&#10; data_platform_prod &#61; object&#40;&#123;&#10; branch &#61; string&#10; identity_provider &#61; string&#10; name &#61; string&#10; type &#61; string&#10; &#125;&#41;&#10; gke_dev &#61; object&#40;&#123;&#10; branch &#61; string&#10; identity_provider &#61; string&#10; name &#61; string&#10; type &#61; string&#10; &#125;&#41;&#10; gke_prod &#61; object&#40;&#123;&#10; branch &#61; string&#10; identity_provider &#61; string&#10; name &#61; string&#10; type &#61; string&#10; &#125;&#41;&#10; networking &#61; object&#40;&#123;&#10; branch &#61; string&#10; identity_provider &#61; string&#10; name &#61; string&#10; type &#61; string&#10; &#125;&#41;&#10; project_factory_dev &#61; object&#40;&#123;&#10; branch &#61; string&#10; identity_provider &#61; string&#10; name &#61; string&#10; type &#61; string&#10; &#125;&#41;&#10; project_factory_prod &#61; object&#40;&#123;&#10; branch &#61; string&#10; identity_provider &#61; string&#10; name &#61; string&#10; type &#61; string&#10; &#125;&#41;&#10; security &#61; object&#40;&#123;&#10; branch &#61; string&#10; identity_provider &#61; string&#10; name &#61; string&#10; type &#61; string&#10; &#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| [custom_roles](variables.tf#L129) | Custom roles defined at the org level, in key => id format. | <code title="object&#40;&#123;&#10; service_project_network_admin &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | <code>00-bootstrap</code> |
| [fast_features](variables.tf#L138) | Selective control for top-level FAST features. | <code title="object&#40;&#123;&#10; data_platform &#61; bool&#10; gke &#61; bool&#10; project_factory &#61; bool&#10; sandbox &#61; bool&#10; teams &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; data_platform &#61; true&#10; gke &#61; true&#10; project_factory &#61; true&#10; sandbox &#61; true&#10; teams &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
| [groups](variables.tf#L158) | Group names to grant organization-level permissions. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; gcp-billing-admins &#61; &#34;gcp-billing-admins&#34;,&#10; gcp-devops &#61; &#34;gcp-devops&#34;,&#10; gcp-network-admins &#61; &#34;gcp-network-admins&#34;&#10; gcp-organization-admins &#61; &#34;gcp-organization-admins&#34;&#10; gcp-security-admins &#61; &#34;gcp-security-admins&#34;&#10; gcp-support &#61; &#34;gcp-support&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
| [locations](variables.tf#L173) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object&#40;&#123;&#10; bq &#61; string&#10; gcs &#61; string&#10; logging &#61; string&#10; pubsub &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; bq &#61; &#34;EU&#34;&#10; gcs &#61; &#34;EU&#34;&#10; logging &#61; &#34;global&#34;&#10; pubsub &#61; &#91;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
| [organization_policy_configs](variables.tf#L201) | Organization policies customization. | <code title="object&#40;&#123;&#10; allowed_policy_member_domains &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| [outputs_location](variables.tf#L209) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable | <code>string</code> | | <code>null</code> | |
| [tag_names](variables.tf#L226) | Customized names for resource management tags. | <code title="object&#40;&#123;&#10; context &#61; string&#10; environment &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; context &#61; &#34;context&#34;&#10; environment &#61; &#34;environment&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [team_folders](variables.tf#L243) | Team folders to be created. Format is described in a code comment. | <code title="map&#40;object&#40;&#123;&#10; descriptive_name &#61; string&#10; group_iam &#61; map&#40;list&#40;string&#41;&#41;&#10; impersonation_groups &#61; list&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>null</code> | |
| [data_dir](variables.tf#L138) | Relative path for the folder storing configuration data. | <code>string</code> | | <code>&#34;data&#34;</code> | |
| [fast_features](variables.tf#L144) | Selective control for top-level FAST features. | <code title="object&#40;&#123;&#10; data_platform &#61; bool&#10; gke &#61; bool&#10; project_factory &#61; bool&#10; sandbox &#61; bool&#10; teams &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; data_platform &#61; true&#10; gke &#61; true&#10; project_factory &#61; true&#10; sandbox &#61; true&#10; teams &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
| [groups](variables.tf#L164) | Group names to grant organization-level permissions. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; gcp-billing-admins &#61; &#34;gcp-billing-admins&#34;,&#10; gcp-devops &#61; &#34;gcp-devops&#34;,&#10; gcp-network-admins &#61; &#34;gcp-network-admins&#34;&#10; gcp-organization-admins &#61; &#34;gcp-organization-admins&#34;&#10; gcp-security-admins &#61; &#34;gcp-security-admins&#34;&#10; gcp-support &#61; &#34;gcp-support&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
| [locations](variables.tf#L179) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object&#40;&#123;&#10; bq &#61; string&#10; gcs &#61; string&#10; logging &#61; string&#10; pubsub &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; bq &#61; &#34;EU&#34;&#10; gcs &#61; &#34;EU&#34;&#10; logging &#61; &#34;global&#34;&#10; pubsub &#61; &#91;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
| [organization_policy_configs](variables.tf#L207) | Organization policies customization. | <code title="object&#40;&#123;&#10; allowed_policy_member_domains &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| [outputs_location](variables.tf#L215) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable | <code>string</code> | | <code>null</code> | |
| [tag_names](variables.tf#L232) | Customized names for resource management tags. | <code title="object&#40;&#123;&#10; context &#61; string&#10; environment &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; context &#61; &#34;context&#34;&#10; environment &#61; &#34;environment&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [team_folders](variables.tf#L249) | Team folders to be created. Format is described in a code comment. | <code title="map&#40;object&#40;&#123;&#10; descriptive_name &#61; string&#10; group_iam &#61; map&#40;list&#40;string&#41;&#41;&#10; impersonation_groups &#61; list&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>null</code> | |
## Outputs

24
fast/stages/01-resman/branch-data-platform.tf
View File

@ -77,12 +77,12 @@ module "branch-dp-prod-folder" {
# automation service accounts and buckets
module "branch-dp-dev-sa" {
source = "../../../modules/iam-service-account"
count = var.fast_features.data_platform ? 1 : 0
project_id = var.automation.project_id
name = "dev-resman-dp-0"
description = "Terraform data platform development service account."
prefix = var.prefix
source = "../../../modules/iam-service-account"
count = var.fast_features.data_platform ? 1 : 0
project_id = var.automation.project_id
name = "dev-resman-dp-0"
display_name = "Terraform data platform development service account."
prefix = var.prefix
iam = {
"roles/iam.serviceAccountTokenCreator" = compact([
try(module.branch-dp-dev-sa-cicd.0.iam_email, null)
@ -94,12 +94,12 @@ module "branch-dp-dev-sa" {
}
module "branch-dp-prod-sa" {
source = "../../../modules/iam-service-account"
count = var.fast_features.data_platform ? 1 : 0
project_id = var.automation.project_id
name = "prod-resman-dp-0"
description = "Terraform data platform production service account."
prefix = var.prefix
source = "../../../modules/iam-service-account"
count = var.fast_features.data_platform ? 1 : 0
project_id = var.automation.project_id
name = "prod-resman-dp-0"
display_name = "Terraform data platform production service account."
prefix = var.prefix
iam = {
"roles/iam.serviceAccountTokenCreator" = compact([
try(module.branch-dp-prod-sa-cicd.0.iam_email, null)

24
fast/stages/01-resman/branch-gke.tf
View File

@ -69,12 +69,12 @@ module "branch-gke-prod-folder" {
}
module "branch-gke-dev-sa" {
source = "../../../modules/iam-service-account"
count = var.fast_features.gke ? 1 : 0
project_id = var.automation.project_id
name = "dev-resman-gke-0"
description = "Terraform gke multitenant dev service account."
prefix = var.prefix
source = "../../../modules/iam-service-account"
count = var.fast_features.gke ? 1 : 0
project_id = var.automation.project_id
name = "dev-resman-gke-0"
display_name = "Terraform gke multitenant dev service account."
prefix = var.prefix
iam = {
"roles/iam.serviceAccountTokenCreator" = concat(
["group:${local.groups.gcp-devops}"],
@ -89,12 +89,12 @@ module "branch-gke-dev-sa" {
}
module "branch-gke-prod-sa" {
source = "../../../modules/iam-service-account"
count = var.fast_features.gke ? 1 : 0
project_id = var.automation.project_id
name = "prod-resman-gke-0"
description = "Terraform gke multitenant prod service account."
prefix = var.prefix
source = "../../../modules/iam-service-account"
count = var.fast_features.gke ? 1 : 0
project_id = var.automation.project_id
name = "prod-resman-gke-0"
display_name = "Terraform gke multitenant prod service account."
prefix = var.prefix
iam = {
"roles/iam.serviceAccountTokenCreator" = concat(
["group:${local.groups.gcp-devops}"],

10
fast/stages/01-resman/branch-networking.tf
View File

@ -86,11 +86,11 @@ module "branch-network-dev-folder" {
# automation service account and bucket
module "branch-network-sa" {
source = "../../../modules/iam-service-account"
project_id = var.automation.project_id
name = "prod-resman-net-0"
description = "Terraform resman networking service account."
prefix = var.prefix
source = "../../../modules/iam-service-account"
project_id = var.automation.project_id
name = "prod-resman-net-0"
display_name = "Terraform resman networking service account."
prefix = var.prefix
iam = {
"roles/iam.serviceAccountTokenCreator" = compact([
try(module.branch-network-sa-cicd.0.iam_email, null)

8
fast/stages/01-resman/branch-project-factory.tf
View File

@ -22,8 +22,8 @@ module "branch-pf-dev-sa" {
project_id = var.automation.project_id
name = "dev-resman-pf-0"
# naming: environment in description
description = "Terraform project factory development service account."
prefix = var.prefix
display_name = "Terraform project factory development service account."
prefix = var.prefix
iam = {
"roles/iam.serviceAccountTokenCreator" = compact([
try(module.branch-pf-dev-sa-cicd.0.iam_email, null)
@ -40,8 +40,8 @@ module "branch-pf-prod-sa" {
project_id = var.automation.project_id
name = "prod-resman-pf-0"
# naming: environment in description
description = "Terraform project factory production service account."
prefix = var.prefix
display_name = "Terraform project factory production service account."
prefix = var.prefix
iam = {
"roles/iam.serviceAccountTokenCreator" = compact([
try(module.branch-pf-prod-sa-cicd.0.iam_email, null)

25
fast/stages/01-resman/branch-sandbox.tf
View File

@ -32,16 +32,9 @@ module "branch-sandbox-folder" {
"roles/resourcemanager.folderAdmin" = [module.branch-sandbox-sa.0.iam_email]
"roles/resourcemanager.projectCreator" = [module.branch-sandbox-sa.0.iam_email]
}
policy_boolean = {
"constraints/sql.restrictPublicIp" = false
}
policy_list = {
"constraints/compute.vmExternalIpAccess" = {
inherit_from_parent = false
suggested_value = null
status = true
values = []
}
org_policies = {
"constraints/sql.restrictPublicIp" = { enforce = false }
"constraints/compute.vmExternalIpAccess" = { allow = { all = true } }
}
tag_bindings = {
context = try(
@ -75,10 +68,10 @@ moved {
}
module "branch-sandbox-sa" {
source = "../../../modules/iam-service-account"
count = var.fast_features.sandbox ? 1 : 0
project_id = var.automation.project_id
name = "dev-resman-sbox-0"
description = "Terraform resman sandbox service account."
prefix = var.prefix
source = "../../../modules/iam-service-account"
count = var.fast_features.sandbox ? 1 : 0
project_id = var.automation.project_id
name = "dev-resman-sbox-0"
display_name = "Terraform resman sandbox service account."
prefix = var.prefix
}

10
fast/stages/01-resman/branch-security.tf
View File

@ -49,11 +49,11 @@ module "branch-security-folder" {
# automation service account and bucket
module "branch-security-sa" {
source = "../../../modules/iam-service-account"
project_id = var.automation.project_id
name = "prod-resman-sec-0"
description = "Terraform resman security service account."
prefix = var.prefix
source = "../../../modules/iam-service-account"
project_id = var.automation.project_id
name = "prod-resman-sec-0"
display_name = "Terraform resman security service account."
prefix = var.prefix
iam = {
"roles/iam.serviceAccountTokenCreator" = compact([
try(module.branch-security-sa-cicd.0.iam_email, null)

24
fast/stages/01-resman/branch-teams.tf
View File

@ -40,12 +40,12 @@ module "branch-teams-folder" {
}
module "branch-teams-sa" {
source = "../../../modules/iam-service-account"
count = var.fast_features.teams ? 1 : 0
project_id = var.automation.project_id
name = "prod-resman-teams-0"
description = "Terraform resman teams service account."
prefix = var.prefix
source = "../../../modules/iam-service-account"
count = var.fast_features.teams ? 1 : 0
project_id = var.automation.project_id
name = "prod-resman-teams-0"
display_name = "Terraform resman teams service account."
prefix = var.prefix
iam_storage_roles = {
(var.automation.outputs_bucket) = ["roles/storage.admin"]
}
@ -83,12 +83,12 @@ module "branch-teams-team-folder" {
}
module "branch-teams-team-sa" {
source = "../../../modules/iam-service-account"
for_each = var.fast_features.teams ? coalesce(var.team_folders, {}) : {}
project_id = var.automation.project_id
name = "prod-teams-${each.key}-0"
description = "Terraform team ${each.key} service account."
prefix = var.prefix
source = "../../../modules/iam-service-account"
for_each = var.fast_features.teams ? coalesce(var.team_folders, {}) : {}
project_id = var.automation.project_id
name = "prod-teams-${each.key}-0"
display_name = "Terraform team ${each.key} service account."
prefix = var.prefix
iam = {
"roles/iam.serviceAccountTokenCreator" = (
each.value.impersonation_groups == null

16
fast/stages/01-resman/cicd-data-platform.tf
View File

@ -95,10 +95,10 @@ module "branch-dp-dev-sa-cicd" {
? { 0 = local.cicd_repositories.data_platform_dev }
: {}
)
project_id = var.automation.project_id
name = "dev-resman-dp-1"
description = "Terraform CI/CD data platform development service account."
prefix = var.prefix
project_id = var.automation.project_id
name = "dev-resman-dp-1"
display_name = "Terraform CI/CD data platform development service account."
prefix = var.prefix
iam = (
each.value.type == "sourcerepo"
# used directly from the cloud build trigger for source repos
@ -138,10 +138,10 @@ module "branch-dp-prod-sa-cicd" {
? { 0 = local.cicd_repositories.data_platform_prod }
: {}
)
project_id = var.automation.project_id
name = "prod-resman-dp-1"
description = "Terraform CI/CD data platform production service account."
prefix = var.prefix
project_id = var.automation.project_id
name = "prod-resman-dp-1"
display_name = "Terraform CI/CD data platform production service account."
prefix = var.prefix
iam = (
each.value.type == "sourcerepo"
# used directly from the cloud build trigger for source repos

16
fast/stages/01-resman/cicd-gke.tf
View File

@ -95,10 +95,10 @@ module "branch-gke-dev-sa-cicd" {
? { 0 = local.cicd_repositories.gke_dev }
: {}
)
project_id = var.automation.project_id
name = "dev-resman-gke-1"
description = "Terraform CI/CD GKE development service account."
prefix = var.prefix
project_id = var.automation.project_id
name = "dev-resman-gke-1"
display_name = "Terraform CI/CD GKE development service account."
prefix = var.prefix
iam = (
each.value.type == "sourcerepo"
# used directly from the cloud build trigger for source repos
@ -138,10 +138,10 @@ module "branch-gke-prod-sa-cicd" {
? { 0 = local.cicd_repositories.gke_prod }
: {}
)
project_id = var.automation.project_id
name = "prod-resman-gke-1"
description = "Terraform CI/CD GKE production service account."
prefix = var.prefix
project_id = var.automation.project_id
name = "prod-resman-gke-1"
display_name = "Terraform CI/CD GKE production service account."
prefix = var.prefix
iam = (
each.value.type == "sourcerepo"
# used directly from the cloud build trigger for source repos

8
fast/stages/01-resman/cicd-networking.tf
View File

@ -57,10 +57,10 @@ module "branch-network-sa-cicd" {
? { 0 = local.cicd_repositories.networking }
: {}
)
project_id = var.automation.project_id
name = "prod-resman-net-1"
description = "Terraform CI/CD stage 2 networking service account."
prefix = var.prefix
project_id = var.automation.project_id
name = "prod-resman-net-1"
display_name = "Terraform CI/CD stage 2 networking service account."
prefix = var.prefix
iam = (
each.value.type == "sourcerepo"
# used directly from the cloud build trigger for source repos

16
fast/stages/01-resman/cicd-project-factory.tf
View File

@ -106,10 +106,10 @@ module "branch-pf-dev-sa-cicd" {
? { 0 = local.cicd_repositories.project_factory_dev }
: {}
)
project_id = var.automation.project_id
name = "dev-pf-resman-pf-1"
description = "Terraform CI/CD project factory development service account."
prefix = var.prefix
project_id = var.automation.project_id
name = "dev-pf-resman-pf-1"
display_name = "Terraform CI/CD project factory development service account."
prefix = var.prefix
iam = (
each.value.type == "sourcerepo"
# used directly from the cloud build trigger for source repos
@ -154,10 +154,10 @@ module "branch-pf-prod-sa-cicd" {
? { 0 = local.cicd_repositories.project_factory_prod }
: {}
)
project_id = var.automation.project_id
name = "prod-pf-resman-pf-1"
description = "Terraform CI/CD project factory production service account."
prefix = var.prefix
project_id = var.automation.project_id
name = "prod-pf-resman-pf-1"
display_name = "Terraform CI/CD project factory production service account."
prefix = var.prefix
iam = (
each.value.type == "sourcerepo"
# used directly from the cloud build trigger for source repos

8
fast/stages/01-resman/cicd-security.tf
View File

@ -57,10 +57,10 @@ module "branch-security-sa-cicd" {
? { 0 = local.cicd_repositories.security }
: {}
)
project_id = var.automation.project_id
name = "prod-resman-sec-1"
description = "Terraform CI/CD stage 2 security service account."
prefix = var.prefix
project_id = var.automation.project_id
name = "prod-resman-sec-1"
display_name = "Terraform CI/CD stage 2 security service account."
prefix = var.prefix
iam = (
each.value.type == "sourcerepo"
# used directly from the cloud build trigger for source repos

73
fast/stages/01-resman/data/org-policies/compute.yaml Normal file
View File

@ -0,0 +1,73 @@
# skip boilerplate check
#
# sample subset of useful organization policies, edit to suit requirements
compute.disableGuestAttributesAccess:
enforce: true
compute.requireOsLogin:
enforce: true
compute.restrictLoadBalancerCreationForTypes:
allow:
values:
- in:INTERNAL
compute.skipDefaultNetworkCreation:
enforce: true
compute.vmExternalIpAccess:
deny:
all: true
# compute.disableInternetNetworkEndpointGroup:
# enforce: true
# compute.disableNestedVirtualization:
# enforce: true
# compute.disableSerialPortAccess:
# enforce: true
# compute.restrictCloudNATUsage:
# deny:
# all: true
# compute.restrictDedicatedInterconnectUsage:
# deny:
# all: true
# compute.restrictPartnerInterconnectUsage:
# deny:
# all: true
# compute.restrictProtocolForwardingCreationForTypes:
# deny:
# all: true
# compute.restrictSharedVpcHostProjects:
# deny:
# all: true
# compute.restrictSharedVpcSubnetworks:
# deny:
# all: true
# compute.restrictVpcPeering:
# deny:
# all: true
# compute.restrictVpnPeerIPs:
# deny:
# all: true
# compute.restrictXpnProjectLienRemoval:
# enforce: true
# compute.setNewProjectDefaultToZonalDNSOnly:
# enforce: true
# compute.vmCanIpForward:
# deny:
# all: true

12
fast/stages/01-resman/data/org-policies/iam.yaml Normal file
View File

@ -0,0 +1,12 @@
# skip boilerplate check
#
# sample subset of useful organization policies, edit to suit requirements
iam.automaticIamGrantsForDefaultServiceAccounts:
enforce: true
iam.disableServiceAccountKeyCreation:
enforce: true
iam.disableServiceAccountKeyUpload:
enforce: true

26
fast/stages/01-resman/data/org-policies/serverless.yaml Normal file
View File

@ -0,0 +1,26 @@
# skip boilerplate check
#
# sample subset of useful organization policies, edit to suit requirements
run.allowedIngress:
allow:
values:
- is:internal
# run.allowedVPCEgress:
# allow:
# values:
# - is:private-ranges-only
# cloudfunctions.allowedIngressSettings:
# allow:
# values:
# - is:ALLOW_INTERNAL_ONLY
# cloudfunctions.allowedVpcConnectorEgressSettings:
# allow:
# values:
# - is:PRIVATE_RANGES_ONLY
# cloudfunctions.requireVPCConnector:
# enforce: true

9
fast/stages/01-resman/data/org-policies/sql.yaml Normal file
View File

@ -0,0 +1,9 @@
# skip boilerplate check
#
# sample subset of useful organization policies, edit to suit requirements
sql.restrictAuthorizedNetworks:
enforce: true
sql.restrictPublicIp:
enforce: true

6
fast/stages/01-resman/data/org-policies/storage.yaml Normal file
View File

@ -0,0 +1,6 @@
# skip boilerplate check
#
# sample subset of useful organization policies, edit to suit requirements
storage.uniformBucketLevelAccess:
enforce: true

100
fast/stages/01-resman/organization.tf
View File

@ -18,18 +18,11 @@
locals {
list_allow = {
inherit_from_parent = false
suggested_value = null
status = true
values = []
}
list_deny = {
inherit_from_parent = false
suggested_value = null
status = false
values = []
}
all_drs_domains = concat(
[var.organization.customer_id],
try(local.policy_configs.allowed_policy_member_domains, [])
)
policy_configs = (
var.organization_policy_configs == null
? {}
@ -73,76 +66,25 @@ module "organization" {
)
} : {}
)
# sample subset of useful organization policies, edit to suit requirements
policy_boolean = {
# "constraints/cloudfunctions.requireVPCConnector" = true
# "constraints/compute.disableGuestAttributesAccess" = true
# "constraints/compute.disableInternetNetworkEndpointGroup" = true
# "constraints/compute.disableNestedVirtualization" = true
# "constraints/compute.disableSerialPortAccess" = true
"constraints/compute.requireOsLogin" = true
# "constraints/compute.restrictXpnProjectLienRemoval" = true
"constraints/compute.skipDefaultNetworkCreation" = true
# "constraints/compute.setNewProjectDefaultToZonalDNSOnly" = true
"constraints/iam.automaticIamGrantsForDefaultServiceAccounts" = true
"constraints/iam.disableServiceAccountKeyCreation" = true
# "constraints/iam.disableServiceAccountKeyUpload" = true
"constraints/sql.restrictPublicIp" = true
"constraints/sql.restrictAuthorizedNetworks" = true
"constraints/storage.uniformBucketLevelAccess" = true
}
policy_list = {
# "constraints/cloudfunctions.allowedIngressSettings" = merge(
# local.list_allow, { values = ["is:ALLOW_INTERNAL_ONLY"] }
# )
# "constraints/cloudfunctions.allowedVpcConnectorEgressSettings" = merge(
# local.list_allow, { values = ["is:PRIVATE_RANGES_ONLY"] }
# )
"constraints/compute.restrictLoadBalancerCreationForTypes" = merge(
local.list_allow, { values = ["in:INTERNAL"] }
)
"constraints/compute.vmExternalIpAccess" = local.list_deny
"constraints/iam.allowedPolicyMemberDomains" = merge(
local.list_allow, {
values = concat(
[var.organization.customer_id],
try(local.policy_configs.allowed_policy_member_domains, [])
)
})
"constraints/run.allowedIngress" = merge(
local.list_allow, { values = ["is:internal"] }
)
# "constraints/run.allowedVPCEgress" = merge(
# local.list_allow, { values = ["is:private-ranges-only"] }
# )
# "constraints/compute.restrictCloudNATUsage" = local.list_deny
# "constraints/compute.restrictDedicatedInterconnectUsage" = local.list_deny
# "constraints/compute.restrictPartnerInterconnectUsage" = local.list_deny
# "constraints/compute.restrictProtocolForwardingCreationForTypes" = local.list_deny
# "constraints/compute.restrictSharedVpcHostProjects" = local.list_deny
# "constraints/compute.restrictSharedVpcSubnetworks" = local.list_deny
# "constraints/compute.restrictVpcPeering" = local.list_deny
# "constraints/compute.restrictVpnPeerIPs" = local.list_deny
# "constraints/compute.vmCanIpForward" = local.list_deny
# "constraints/gcp.resourceLocations" = {
# inherit_from_parent = false
# suggested_value = null
# status = true
# values = local.allowed_regions
org_policies = {
"iam.allowedPolicyMemberDomains" = { allow = { values = local.all_drs_domains } }
#"gcp.resourceLocations" = {
# allow = { values = local.allowed_regions }
# }
# "iam.workloadIdentityPoolProviders" = {
# allow = {
# values = [
# for k, v in coalesce(var.automation.federated_identity_providers, {}) :
# v.issuer_uri
# ]
# }
# }
# https://cloud.google.com/iam/docs/manage-workload-identity-pools-providers#restrict
# "constraints/iam.workloadIdentityPoolProviders" = merge(
# local.list_allow, { values = [
# for k, v in coalesce(var.automation.federated_identity_providers, {}) :
# v.issuer_uri
# ] }
# )
# "constraints/iam.workloadIdentityPoolAwsAccounts" = merge(
# local.list_allow, { values = [
#
# ] }
# )
}
org_policies_data_path = "${var.data_dir}/org-policies"
tags = {
(var.tag_names.context) = {
description = "Resource management context."

6
fast/stages/01-resman/variables.tf
View File

@ -135,6 +135,12 @@ variable "custom_roles" {
default = null
}
variable "data_dir" {
description = "Relative path for the folder storing configuration data."
type = string
default = "data"
}
variable "fast_features" {
# tfdoc:variable:source 00-bootstrap
description = "Selective control for top-level FAST features."

44
fast/stages/02-networking-nva/data/firewall-rules/dev/rules.yaml
View File

@ -1,27 +1,21 @@
# skip boilerplate check
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
direction: INGRESS
action: allow
sources:
- composer-worker
targets:
- composer-worker
use_service_accounts: false
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
direction: INGRESS
action: allow
sources:
- dataflow
targets:
- dataflow
use_service_accounts: false
rules:
- protocol: tcp
ports: [12345, 12346]
ingress:
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
sources:
- composer-worker
targets:
- composer-worker
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
sources:
- dataflow
targets:
- dataflow
rules:
- protocol: tcp
ports: [12345, 12346]

44
fast/stages/02-networking-nva/data/firewall-rules/landing-trusted/rules.yaml
View File

@ -1,29 +1,19 @@
# skip boilerplate check
allow-hc-nva-ssh-trusted:
description: "Allow traffic from Google healthchecks to NVA appliances"
direction: INGRESS
action: allow
sources: []
ranges:
- $healthchecks
targets: []
use_service_accounts: false
rules:
- protocol: tcp
ports:
- 22
allow-onprem-probes-trusted-example:
description: "Allow traffic from onprem probes"
direction: INGRESS
action: allow
sources: []
ranges:
- $onprem_probes
targets: []
use_service_accounts: false
rules:
- protocol: tcp
ports:
- 12345
ingress:
allow-hc-nva-ssh-trusted:
description: "Allow traffic from Google healthchecks to NVA appliances"
source_ranges:
- healthchecks
rules:
- protocol: tcp
ports:
- 22
allow-onprem-probes-trusted-example:
description: "Allow traffic from onprem probes"
source_ranges:
- onprem_probes
rules:
- protocol: tcp
ports:
- 12345

22
fast/stages/02-networking-nva/data/firewall-rules/landing-untrusted/rules.yaml
View File

@ -1,15 +1,11 @@
# skip boilerplate check
allow-hc-nva-ssh-untrusted:
description: "Allow traffic from Google healthchecks to NVA appliances"
direction: INGRESS
action: allow
sources: []
ranges:
- $healthchecks
targets: []
use_service_accounts: false
rules:
- protocol: tcp
ports:
- 22
ingress:
allow-hc-nva-ssh-untrusted:
description: "Allow traffic from Google healthchecks to NVA appliances"
source_ranges:
- healthchecks
rules:
- protocol: tcp
ports:
- 22

38
fast/stages/02-networking-nva/landing.tf
View File

@ -57,15 +57,16 @@ module "landing-untrusted-vpc" {
}
module "landing-untrusted-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-untrusted-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/landing-untrusted"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-untrusted-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/landing-untrusted"
}
}
# NAT
@ -123,13 +124,14 @@ module "landing-trusted-vpc" {
}
module "landing-trusted-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-trusted-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/landing-trusted"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-trusted-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/landing-trusted"
}
}

80
fast/stages/02-networking-nva/nva.tf
View File

@ -15,7 +15,7 @@
*/
locals {
# routing_config should be aligned to the NVA network interfaces - i.e.
# routing_config should be aligned to the NVA network interfaces - i.e.
# local.routing_config[0] sets up the first interface, and so on.
routing_config = [
{
@ -94,27 +94,21 @@ module "nva-template" {
}
module "nva-mig" {
for_each = local.nva_locality
source = "../../../modules/compute-mig"
project_id = module.landing-project.project_id
regional = true
location = each.value.region
name = "nva-cos-${each.value.trigram}-${each.value.zone}"
target_size = 1
# FIXME: cycle
# auto_healing_policies = {
# health_check = module.nva-mig[each.key].health_check.self_link
# initial_delay_sec = 30
# }
health_check_config = {
type = "tcp"
check = { port = 22 }
config = {}
logging = true
for_each = local.nva_locality
source = "../../../modules/compute-mig"
project_id = module.landing-project.project_id
location = each.value.region
name = "nva-cos-${each.value.trigram}-${each.value.zone}"
instance_template = module.nva-template[each.key].template.self_link
target_size = 1
auto_healing_policies = {
initial_delay_sec = 30
}
default_version = {
instance_template = module.nva-template[each.key].template.self_link
name = "default"
health_check_config = {
enable_logging = true
tcp = {
port = 22
}
}
}
@ -126,16 +120,20 @@ module "ilb-nva-untrusted" {
name = "nva-untrusted-${each.value.0}"
service_label = var.prefix
global_access = true
network = module.landing-untrusted-vpc.self_link
subnetwork = module.landing-untrusted-vpc.subnet_self_links["${each.key}/landing-untrusted-default-${each.value.0}"]
backends = [for key, _ in local.nva_locality :
{
failover = false
group = module.nva-mig[key].group_manager.instance_group
balancing_mode = "CONNECTION"
} if local.nva_locality[key].region == each.key]
vpc_config = {
network = module.landing-untrusted-vpc.self_link
subnetwork = module.landing-untrusted-vpc.subnet_self_links["${each.key}/landing-untrusted-default-${each.value.0}"]
}
backends = [
for key, _ in local.nva_locality : {
group = module.nva-mig[key].group_manager.instance_group
} if local.nva_locality[key].region == each.key
]
health_check_config = {
type = "tcp", check = { port = 22 }, config = {}, logging = false
enable_logging = true
tcp = {
port = 22
}
}
}
@ -148,16 +146,20 @@ module "ilb-nva-trusted" {
name = "nva-trusted-${each.value.0}"
service_label = var.prefix
global_access = true
network = module.landing-trusted-vpc.self_link
subnetwork = module.landing-trusted-vpc.subnet_self_links["${each.key}/landing-trusted-default-${each.value.0}"]
backends = [for key, _ in local.nva_locality :
{
failover = false
group = module.nva-mig[key].group_manager.instance_group
balancing_mode = "CONNECTION"
} if local.nva_locality[key].region == each.key]
vpc_config = {
network = module.landing-trusted-vpc.self_link
subnetwork = module.landing-trusted-vpc.subnet_self_links["${each.key}/landing-trusted-default-${each.value.0}"]
}
backends = [
for key, _ in local.nva_locality : {
group = module.nva-mig[key].group_manager.instance_group
} if local.nva_locality[key].region == each.key
]
health_check_config = {
type = "tcp", check = { port = 22 }, config = {}, logging = false
enable_logging = true
tcp = {
port = 22
}
}
}

19
fast/stages/02-networking-nva/spoke-dev.tf
View File

@ -97,15 +97,16 @@ module "dev-spoke-vpc" {
}
module "dev-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/dev"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/dev"
}
}
module "peering-dev" {

19
fast/stages/02-networking-nva/spoke-prod.tf
View File

@ -97,15 +97,16 @@ module "prod-spoke-vpc" {
}
module "prod-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/prod"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/prod"
}
}
module "peering-prod" {

44
fast/stages/02-networking-peering/data/firewall-rules/dev/rules.yaml
View File

@ -1,27 +1,21 @@
# skip boilerplate check
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
direction: INGRESS
action: allow
sources:
- composer-worker
targets:
- composer-worker
use_service_accounts: false
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
direction: INGRESS
action: allow
sources:
- dataflow
targets:
- dataflow
use_service_accounts: false
rules:
- protocol: tcp
ports: [12345, 12346]
ingress:
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
sources:
- composer-worker
targets:
- composer-worker
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
sources:
- dataflow
targets:
- dataflow
rules:
- protocol: tcp
ports: [12345, 12346]

22
fast/stages/02-networking-peering/data/firewall-rules/landing/rules.yaml
View File

@ -1,15 +1,11 @@
# skip boilerplate check
allow-onprem-probes-example:
description: "Allow traffic from onprem probes"
direction: INGRESS
action: allow
sources: []
ranges:
- $onprem_probes
targets: []
use_service_accounts: false
rules:
- protocol: tcp
ports:
- 12345
ingress:
allow-onprem-probes-example:
description: "Allow traffic from onprem probes"
source_ranges:
- onprem_probes
rules:
- protocol: tcp
ports:
- 12345

19
fast/stages/02-networking-peering/landing.tf
View File

@ -67,15 +67,16 @@ module "landing-vpc" {
}
module "landing-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/landing"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/landing"
}
}
module "landing-nat-ew1" {

19
fast/stages/02-networking-peering/spoke-dev.tf
View File

@ -67,15 +67,16 @@ module "dev-spoke-vpc" {
}
module "dev-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/dev"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/dev"
}
}
module "dev-spoke-cloudnat" {

19
fast/stages/02-networking-peering/spoke-prod.tf
View File

@ -67,15 +67,16 @@ module "prod-spoke-vpc" {
}
module "prod-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/prod"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/prod"
}
}
module "prod-spoke-cloudnat" {

40
fast/stages/02-networking-separate-envs/data/firewall-rules/dev/rules.yaml
View File

@ -1,27 +1,17 @@
# skip boilerplate check
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
direction: INGRESS
action: allow
sources: []
ranges: ["0.0.0.0/0"]
targets:
- composer-worker
use_service_accounts: false
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
direction: INGRESS
action: allow
sources: []
ranges: ["0.0.0.0/0"]
targets:
- dataflow
use_service_accounts: false
rules:
- protocol: tcp
ports: [12345, 12346]
ingress:
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
targets:
- composer-worker
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
targets:
- dataflow
rules:
- protocol: tcp
ports: [12345, 12346]

19
fast/stages/02-networking-separate-envs/spoke-dev.tf
View File

@ -66,15 +66,16 @@ module "dev-spoke-vpc" {
}
module "dev-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/dev"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/dev"
}
}
module "dev-spoke-cloudnat" {

19
fast/stages/02-networking-separate-envs/spoke-prod.tf
View File

@ -66,15 +66,16 @@ module "prod-spoke-vpc" {
}
module "prod-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/prod"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/prod"
}
}
module "prod-spoke-cloudnat" {

44
fast/stages/02-networking-vpn/data/firewall-rules/dev/rules.yaml
View File

@ -1,27 +1,21 @@
# skip boilerplate check
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
direction: INGRESS
action: allow
sources:
- composer-worker
targets:
- composer-worker
use_service_accounts: false
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
direction: INGRESS
action: allow
sources:
- dataflow
targets:
- dataflow
use_service_accounts: false
rules:
- protocol: tcp
ports: [12345, 12346]
ingress:
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
sources:
- composer-worker
targets:
- composer-worker
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
sources:
- dataflow
targets:
- dataflow
rules:
- protocol: tcp
ports: [12345, 12346]

22
fast/stages/02-networking-vpn/data/firewall-rules/landing/rules.yaml
View File

@ -1,15 +1,11 @@
# skip boilerplate check
allow-onprem-probes-example:
description: "Allow traffic from onprem probes"
direction: INGRESS
action: allow
sources: []
ranges:
- $onprem_probes
targets: []
use_service_accounts: false
rules:
- protocol: tcp
ports:
- 12345
ingress:
allow-onprem-probes-example:
description: "Allow traffic from onprem probes"
source_ranges:
- onprem_probes
rules:
- protocol: tcp
ports:
- 12345

19
fast/stages/02-networking-vpn/landing.tf
View File

@ -67,15 +67,16 @@ module "landing-vpc" {
}
module "landing-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/landing"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/landing"
}
}
module "landing-nat-ew1" {

19
fast/stages/02-networking-vpn/spoke-dev.tf
View File

@ -67,15 +67,16 @@ module "dev-spoke-vpc" {
}
module "dev-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/dev"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/dev"
}
}
module "dev-spoke-cloudnat" {

19
fast/stages/02-networking-vpn/spoke-prod.tf
View File

@ -67,15 +67,16 @@ module "prod-spoke-vpc" {
}
module "prod-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/prod"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
rules_folder = "${var.data_dir}/firewall-rules/prod"
}
}
module "prod-spoke-cloudnat" {

14
fast/stages/03-project-factory/dev/data/projects/project.yaml.sample
View File

@ -48,15 +48,15 @@ labels:
# [opt] Org policy overrides defined at project level
org_policies:
policy_boolean:
constraints/compute.disableGuestAttributesAccess: true
policy_list:
constraints/compute.trustedImageProjects:
inherit_from_parent: null
status: true
suggested_value: null
constraints/compute.disableGuestAttributesAccess:
enforce: true
constraints/compute.trustedImageProjects:
allow:
values:
- projects/fast-dev-iac-core-0
constraints/compute.vmExternalIpAccess:
deny:
all: true
# [opt] Service account to create for the project and their roles on the project
# in name => [roles] format

1
modules/README.md
View File

@ -36,7 +36,6 @@ These modules are used in the examples included in this repository. If you are u
- [service accounts](./iam-service-account)
- [logging bucket](./logging-bucket)
- [organization](./organization)
- [organization-policy](./organization-policy)
- [project](./project)
- [projects-data-source](./projects-data-source)

2
modules/cloud-function/main.tf
View File

@ -141,7 +141,7 @@ resource "google_cloudfunctions2_function" "function" {
environment_variables = var.environment_variables
source {
storage_source {
bucket = google_storage_bucket.bucket[0].name
bucket = local.bucket
object = google_storage_bucket_object.bundle.name
}
}

248
modules/compute-mig/README.md
View File

@ -2,7 +2,7 @@
This module allows creating a managed instance group supporting one or more application versions via instance templates. Optionally, a health check and an autoscaler can be created, and the managed instance group can be configured to be stateful.
This module can be coupled with the [`compute-vm`](../compute-vm) module which can manage instance templates, and the [`net-ilb`](../net-ilb) module to assign the MIG to a backend wired to an Internal Load Balancer. The first use case is shown in the examples below.
This module can be coupled with the [`compute-vm`](../compute-vm) module which can manage instance templates, and the [`net-ilb`](../net-ilb) module to assign the MIG to a backend wired to an Internal Load Balancer. The first use case is shown in the examples below.
Stateful disks can be created directly, as shown in the last example below.
@ -39,15 +39,12 @@ module "nginx-template" {
}
module "nginx-mig" {
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 2
default_version = {
instance_template = module.nginx-template.template.self_link
name = "default"
}
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 2
instance_template = module.nginx-template.template.self_link
}
# tftest modules=2 resources=2
```
@ -85,20 +82,18 @@ module "nginx-template" {
}
module "nginx-mig" {
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
default_version = {
instance_template = module.nginx-template.template.self_link
name = "default"
}
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
instance_template = module.nginx-template.template.self_link
versions = {
canary = {
instance_template = module.nginx-template.template.self_link
target_type = "fixed"
target_size = 1
target_size = {
fixed = 1
}
}
}
}
@ -138,24 +133,20 @@ module "nginx-template" {
}
module "nginx-mig" {
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
default_version = {
instance_template = module.nginx-template.template.self_link
name = "default"
}
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
instance_template = module.nginx-template.template.self_link
auto_healing_policies = {
health_check = module.nginx-mig.health_check.self_link
initial_delay_sec = 30
}
health_check_config = {
type = "http"
check = { port = 80 }
config = {}
logging = true
enable_logging = true
http = {
port = 80
}
}
}
# tftest modules=2 resources=3
@ -194,22 +185,21 @@ module "nginx-template" {
}
module "nginx-mig" {
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
default_version = {
instance_template = module.nginx-template.template.self_link
name = "default"
}
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
instance_template = module.nginx-template.template.self_link
autoscaler_config = {
max_replicas = 3
min_replicas = 1
cooldown_period = 30
cpu_utilization_target = 0.65
load_balancing_utilization_target = null
metric = null
max_replicas = 3
min_replicas = 1
cooldown_period = 30
scaling_signals = {
cpu_utilization = {
target = 0.65
}
}
}
}
# tftest modules=2 resources=3
@ -246,23 +236,19 @@ module "nginx-template" {
}
module "nginx-mig" {
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
default_version = {
instance_template = module.nginx-template.template.self_link
name = "default"
}
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
instance_template = module.nginx-template.template.self_link
update_policy = {
type = "PROACTIVE"
minimal_action = "REPLACE"
type = "PROACTIVE"
min_ready_sec = 30
max_surge_type = "fixed"
max_surge = 1
max_unavailable_type = null
max_unavailable = null
max_surge = {
fixed = 1
}
}
}
# tftest modules=2 resources=2
@ -270,7 +256,7 @@ module "nginx-mig" {
### Stateful MIGs - MIG Config
Stateful MIGs have some limitations documented [here](https://cloud.google.com/compute/docs/instance-groups/configuring-stateful-migs#limitations). Enforcement of these requirements is the responsibility of users of this module.
Stateful MIGs have some limitations documented [here](https://cloud.google.com/compute/docs/instance-groups/configuring-stateful-migs#limitations). Enforcement of these requirements is the responsibility of users of this module.
You can configure a disk defined in the instance template to be stateful for all instances in the MIG by configuring in the MIG's stateful policy, using the `stateful_disk_mig` variable. Alternatively, you can also configure stateful persistent disks individually per instance of the MIG by setting the `stateful_disk_instance` variable. A discussion on these scenarios can be found in the [docs](https://cloud.google.com/compute/docs/instance-groups/configuring-stateful-disks-in-migs).
@ -278,7 +264,6 @@ An example using only the configuration at the MIG level can be seen below.
Note that when referencing the stateful disk, you use `device_name` and not `disk_name`.
```hcl
module "cos-nginx" {
source = "./fabric/modules/cloud-config-container/nginx"
@ -319,40 +304,33 @@ module "nginx-template" {
}
module "nginx-mig" {
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
default_version = {
instance_template = module.nginx-template.template.self_link
name = "default"
}
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
instance_template = module.nginx-template.template.self_link
autoscaler_config = {
max_replicas = 3
min_replicas = 1
cooldown_period = 30
cpu_utilization_target = 0.65
load_balancing_utilization_target = null
metric = null
}
stateful_config = {
per_instance_config = {},
mig_config = {
stateful_disks = {
repd-1 = {
delete_rule = "NEVER"
}
max_replicas = 3
min_replicas = 1
cooldown_period = 30
scaling_signals = {
cpu_utilization = {
target = 0.65
}
}
}
stateful_disks = {
repd-1 = null
}
}
# tftest modules=2 resources=3
```
### Stateful MIGs - Instance Config
Here is an example defining the stateful config at the instance level.
Here is an example defining the stateful config at the instance level.
Note that you will need to know the instance name in order to use this configuration.
@ -396,46 +374,36 @@ module "nginx-template" {
}
module "nginx-mig" {
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
default_version = {
instance_template = module.nginx-template.template.self_link
name = "default"
}
source = "./fabric/modules/compute-mig"
project_id = "my-project"
location = "europe-west1-b"
name = "mig-test"
target_size = 3
instance_template = module.nginx-template.template.self_link
autoscaler_config = {
max_replicas = 3
min_replicas = 1
cooldown_period = 30
cpu_utilization_target = 0.65
load_balancing_utilization_target = null
metric = null
max_replicas = 3
min_replicas = 1
cooldown_period = 30
scaling_signals = {
cpu_utilization = {
target = 0.65
}
}
}
stateful_config = {
per_instance_config = {
# note that this needs to be the name of an existing instance within the Managed Instance Group
instance-1 = {
stateful_disks = {
# name needs to match a MIG instance name
instance-1 = {
minimal_action = "NONE",
most_disruptive_allowed_action = "REPLACE"
preserved_state = {
disks = {
persistent-disk-1 = {
source = "test-disk",
mode = "READ_ONLY",
delete_rule= "NEVER",
},
},
}
}
metadata = {
foo = "bar"
},
update_config = {
minimal_action = "NONE",
most_disruptive_allowed_action = "REPLACE",
remove_instance_state_on_destroy = false,
},
},
},
mig_config = {
stateful_disks = {
}
}
}
}
@ -449,21 +417,25 @@ module "nginx-mig" {
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [default_version](variables.tf#L45) | Default application version template. Additional versions can be specified via the `versions` variable. | <code title="object&#40;&#123;&#10; instance_template &#61; string&#10; name &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | |
| [location](variables.tf#L64) | Compute zone, or region if `regional` is set to true. | <code>string</code> | ✓ | |
| [name](variables.tf#L68) | Managed group name. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L79) | Project id. | <code>string</code> | ✓ | |
| [auto_healing_policies](variables.tf#L17) | Auto-healing policies for this group. | <code title="object&#40;&#123;&#10; health_check &#61; string&#10; initial_delay_sec &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [autoscaler_config](variables.tf#L26) | Optional autoscaler configuration. Only one of 'cpu_utilization_target' 'load_balancing_utilization_target' or 'metric' can be not null. | <code title="object&#40;&#123;&#10; max_replicas &#61; number&#10; min_replicas &#61; number&#10; cooldown_period &#61; number&#10; cpu_utilization_target &#61; number&#10; load_balancing_utilization_target &#61; number&#10; metric &#61; object&#40;&#123;&#10; name &#61; string&#10; single_instance_assignment &#61; number&#10; target &#61; number&#10; type &#61; string &#35; GAUGE, DELTA_PER_SECOND, DELTA_PER_MINUTE&#10; filter &#61; string&#10; &#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [health_check_config](variables.tf#L53) | Optional auto-created health check configuration, use the output self-link to set it in the auto healing policy. Refer to examples for usage. | <code title="object&#40;&#123;&#10; type &#61; string &#35; http https tcp ssl http2&#10; check &#61; map&#40;any&#41; &#35; actual health check block attributes&#10; config &#61; map&#40;number&#41; &#35; interval, thresholds, timeout&#10; logging &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [named_ports](variables.tf#L73) | Named ports. | <code>map&#40;number&#41;</code> | | <code>null</code> |
| [regional](variables.tf#L84) | Use regional instance group. When set, `location` should be set to the region. | <code>bool</code> | | <code>false</code> |
| [stateful_config](variables.tf#L90) | Stateful configuration can be done by individual instances or for all instances in the MIG. They key in per_instance_config is the name of the specific instance. The key of the stateful_disks is the 'device_name' field of the resource. Please note that device_name is defined at the OS mount level, unlike the disk name. | <code title="object&#40;&#123;&#10; per_instance_config &#61; map&#40;object&#40;&#123;&#10; stateful_disks &#61; map&#40;object&#40;&#123;&#10; source &#61; string&#10; mode &#61; string &#35; READ_WRITE &#124; READ_ONLY &#10; delete_rule &#61; string &#35; NEVER &#124; ON_PERMANENT_INSTANCE_DELETION&#10; &#125;&#41;&#41;&#10; metadata &#61; map&#40;string&#41;&#10; update_config &#61; object&#40;&#123;&#10; minimal_action &#61; string &#35; NONE &#124; REPLACE &#124; RESTART &#124; REFRESH&#10; most_disruptive_allowed_action &#61; string &#35; REPLACE &#124; RESTART &#124; REFRESH &#124; NONE&#10; remove_instance_state_on_destroy &#61; bool&#10; &#125;&#41;&#10; &#125;&#41;&#41;&#10;&#10;&#10; mig_config &#61; object&#40;&#123;&#10; stateful_disks &#61; map&#40;object&#40;&#123;&#10; delete_rule &#61; string &#35; NEVER &#124; ON_PERMANENT_INSTANCE_DELETION&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#10;&#10;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [target_pools](variables.tf#L121) | Optional list of URLs for target pools to which new instances in the group are added. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [target_size](variables.tf#L127) | Group target size, leave null when using an autoscaler. | <code>number</code> | | <code>null</code> |
| [update_policy](variables.tf#L133) | Update policy. Type can be 'OPPORTUNISTIC' or 'PROACTIVE', action 'REPLACE' or 'restart', surge type 'fixed' or 'percent'. | <code title="object&#40;&#123;&#10; instance_redistribution_type &#61; optional&#40;string, &#34;PROACTIVE&#34;&#41; &#35; NONE &#124; PROACTIVE. The attribute is ignored if regional is set to false.&#10; max_surge_type &#61; string &#35; fixed &#124; percent&#10; max_surge &#61; number&#10; max_unavailable_type &#61; string&#10; max_unavailable &#61; number&#10; minimal_action &#61; string &#35; REPLACE &#124; RESTART&#10; min_ready_sec &#61; number&#10; type &#61; string &#35; OPPORTUNISTIC &#124; PROACTIVE&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [versions](variables.tf#L148) | Additional application versions, target_type is either 'fixed' or 'percent'. | <code title="map&#40;object&#40;&#123;&#10; instance_template &#61; string&#10; target_type &#61; string &#35; fixed &#124; percent&#10; target_size &#61; number&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>null</code> |
| [wait_for_instances](variables.tf#L158) | Wait for all instances to be created/updated before returning. | <code>bool</code> | | <code>null</code> |
| [instance_template](variables.tf#L174) | Instance template for the default version. | <code>string</code> | ✓ | |
| [location](variables.tf#L179) | Compute zone or region. | <code>string</code> | ✓ | |
| [name](variables.tf#L184) | Managed group name. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L195) | Project id. | <code>string</code> | ✓ | |
| [all_instances_config](variables.tf#L17) | Metadata and labels set to all instances in the group. | <code title="object&#40;&#123;&#10; labels &#61; optional&#40;map&#40;string&#41;&#41;&#10; metadata &#61; optional&#40;map&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [auto_healing_policies](variables.tf#L26) | Auto-healing policies for this group. | <code title="object&#40;&#123;&#10; health_check &#61; optional&#40;string&#41;&#10; initial_delay_sec &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [autoscaler_config](variables.tf#L35) | Optional autoscaler configuration. | <code title="object&#40;&#123;&#10; max_replicas &#61; number&#10; min_replicas &#61; number&#10; cooldown_period &#61; optional&#40;number&#41;&#10; mode &#61; optional&#40;string&#41; &#35; OFF, ONLY_UP, ON&#10; scaling_control &#61; optional&#40;object&#40;&#123;&#10; down &#61; optional&#40;object&#40;&#123;&#10; max_replicas_fixed &#61; optional&#40;number&#41;&#10; max_replicas_percent &#61; optional&#40;number&#41;&#10; time_window_sec &#61; optional&#40;number&#41;&#10; &#125;&#41;&#41;&#10; in &#61; optional&#40;object&#40;&#123;&#10; max_replicas_fixed &#61; optional&#40;number&#41;&#10; max_replicas_percent &#61; optional&#40;number&#41;&#10; time_window_sec &#61; optional&#40;number&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; scaling_signals &#61; optional&#40;object&#40;&#123;&#10; cpu_utilization &#61; optional&#40;object&#40;&#123;&#10; target &#61; number&#10; optimize_availability &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; load_balancing_utilization &#61; optional&#40;object&#40;&#123;&#10; target &#61; number&#10; &#125;&#41;&#41;&#10; metrics &#61; optional&#40;list&#40;object&#40;&#123;&#10; name &#61; string&#10; type &#61; string &#35; GAUGE, DELTA_PER_SECOND, DELTA_PER_MINUTE&#10; target_value &#61; number&#10; single_instance_assignment &#61; optional&#40;number&#41;&#10; time_series_filter &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#41;&#10; schedules &#61; optional&#40;list&#40;object&#40;&#123;&#10; duration_sec &#61; number&#10; name &#61; string&#10; min_required_replicas &#61; number&#10; cron_schedule &#61; string&#10; description &#61; optional&#40;bool&#41;&#10; timezone &#61; optional&#40;string&#41;&#10; disabled &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [default_version_name](variables.tf#L83) | Name used for the default version. | <code>string</code> | | <code>&#34;default&#34;</code> |
| [description](variables.tf#L89) | Optional description used for all resources managed by this module. | <code>string</code> | | <code>&#34;Terraform managed.&#34;</code> |
| [distribution_policy](variables.tf#L95) | DIstribution policy for regional MIG. | <code title="object&#40;&#123;&#10; target_shape &#61; optional&#40;string&#41;&#10; zones &#61; optional&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [health_check_config](variables.tf#L104) | Optional auto-created health check configuration, use the output self-link to set it in the auto healing policy. Refer to examples for usage. | <code title="object&#40;&#123;&#10; check_interval_sec &#61; optional&#40;number&#41;&#10; description &#61; optional&#40;string, &#34;Terraform managed.&#34;&#41;&#10; enable_logging &#61; optional&#40;bool, false&#41;&#10; healthy_threshold &#61; optional&#40;number&#41;&#10; timeout_sec &#61; optional&#40;number&#41;&#10; unhealthy_threshold &#61; optional&#40;number&#41;&#10; grpc &#61; optional&#40;object&#40;&#123;&#10; port &#61; optional&#40;number&#41;&#10; port_name &#61; optional&#40;string&#41;&#10; port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10; service_name &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; http &#61; optional&#40;object&#40;&#123;&#10; host &#61; optional&#40;string&#41;&#10; port &#61; optional&#40;number&#41;&#10; port_name &#61; optional&#40;string&#41;&#10; port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10; proxy_header &#61; optional&#40;string&#41;&#10; request_path &#61; optional&#40;string&#41;&#10; response &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; http2 &#61; optional&#40;object&#40;&#123;&#10; host &#61; optional&#40;string&#41;&#10; port &#61; optional&#40;number&#41;&#10; port_name &#61; optional&#40;string&#41;&#10; port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10; proxy_header &#61; optional&#40;string&#41;&#10; request_path &#61; optional&#40;string&#41;&#10; response &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; https &#61; optional&#40;object&#40;&#123;&#10; host &#61; optional&#40;string&#41;&#10; port &#61; optional&#40;number&#41;&#10; port_name &#61; optional&#40;string&#41;&#10; port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10; proxy_header &#61; optional&#40;string&#41;&#10; request_path &#61; optional&#40;string&#41;&#10; response &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; tcp &#61; optional&#40;object&#40;&#123;&#10; port &#61; optional&#40;number&#41;&#10; port_name &#61; optional&#40;string&#41;&#10; port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10; proxy_header &#61; optional&#40;string&#41;&#10; request &#61; optional&#40;string&#41;&#10; response &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; ssl &#61; optional&#40;object&#40;&#123;&#10; port &#61; optional&#40;number&#41;&#10; port_name &#61; optional&#40;string&#41;&#10; port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10; proxy_header &#61; optional&#40;string&#41;&#10; request &#61; optional&#40;string&#41;&#10; response &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [named_ports](variables.tf#L189) | Named ports. | <code>map&#40;number&#41;</code> | | <code>null</code> |
| [stateful_config](variables.tf#L207) | Stateful configuration for individual instances. | <code title="map&#40;object&#40;&#123;&#10; minimal_action &#61; optional&#40;string&#41;&#10; most_disruptive_action &#61; optional&#40;string&#41;&#10; remove_state_on_destroy &#61; optional&#40;bool&#41;&#10; preserved_state &#61; optional&#40;object&#40;&#123;&#10; disks &#61; optional&#40;map&#40;object&#40;&#123;&#10; source &#61; string&#10; delete_on_instance_deletion &#61; optional&#40;bool&#41;&#10; read_only &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#41;&#10; metadata &#61; optional&#40;map&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [stateful_disks](variables.tf#L200) | Stateful disk configuration applied at the MIG level to all instances, in device name => on permanent instance delete rule as boolean. | <code>map&#40;bool&#41;</code> | | <code>&#123;&#125;</code> |
| [target_pools](variables.tf#L226) | Optional list of URLs for target pools to which new instances in the group are added. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [target_size](variables.tf#L232) | Group target size, leave null when using an autoscaler. | <code>number</code> | | <code>null</code> |
| [update_policy](variables.tf#L238) | Update policy. Minimal action and type are required. | <code title="object&#40;&#123;&#10; minimal_action &#61; string&#10; type &#61; string&#10; max_surge &#61; optional&#40;object&#40;&#123;&#10; fixed &#61; optional&#40;number&#41;&#10; percent &#61; optional&#40;number&#41;&#10; &#125;&#41;&#41;&#10; max_unavailable &#61; optional&#40;object&#40;&#123;&#10; fixed &#61; optional&#40;number&#41;&#10; percent &#61; optional&#40;number&#41;&#10; &#125;&#41;&#41;&#10; min_ready_sec &#61; optional&#40;number&#41;&#10; most_disruptive_action &#61; optional&#40;string&#41;&#10; regional_redistribution_type &#61; optional&#40;string&#41;&#10; replacement_method &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [versions](variables.tf#L259) | Additional application versions, target_size is optional. | <code title="map&#40;object&#40;&#123;&#10; instance_template &#61; string&#10; target_size &#61; optional&#40;object&#40;&#123;&#10; fixed &#61; optional&#40;number&#41;&#10; percent &#61; optional&#40;number&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [wait_for_instances](variables.tf#L272) | Wait for all instances to be created/updated before returning. | <code title="object&#40;&#123;&#10; enabled &#61; bool&#10; status &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
## Outputs

229
modules/compute-mig/autoscaler.tf Normal file
View File

@ -0,0 +1,229 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Autoscaler resource.
locals {
as_enabled = true
as_scaling = try(var.autoscaler_config.scaling_control, null)
as_signals = try(var.autoscaler_config.scaling_signals, null)
}
resource "google_compute_autoscaler" "default" {
provider = google-beta
count = local.is_regional || var.autoscaler_config == null ? 0 : 1
project = var.project_id
name = var.name
zone = var.location
description = var.description
target = google_compute_instance_group_manager.default.0.id
autoscaling_policy {
max_replicas = var.autoscaler_config.max_replicas
min_replicas = var.autoscaler_config.min_replicas
cooldown_period = var.autoscaler_config.cooldown_period
dynamic "scale_down_control" {
for_each = local.as_scaling.down == null ? [] : [""]
content {
time_window_sec = local.as_scaling.down.time_window_sec
dynamic "max_scaled_down_replicas" {
for_each = (
local.as_scaling.down.max_replicas_fixed == null &&
local.as_scaling.down.max_replicas_percent == null
? []
: [""]
)
content {
fixed = local.as_scaling.down.max_replicas_fixed
percent = local.as_scaling.down.max_replicas_percent
}
}
}
}
dynamic "scale_in_control" {
for_each = local.as_scaling.in == null ? [] : [""]
content {
time_window_sec = local.as_scaling.in.time_window_sec
dynamic "max_scaled_in_replicas" {
for_each = (
local.as_scaling.in.max_replicas_fixed == null &&
local.as_scaling.in.max_replicas_percent == null
? []
: [""]
)
content {
fixed = local.as_scaling.in.max_replicas_fixed
percent = local.as_scaling.in.max_replicas_percent
}
}
}
}
dynamic "cpu_utilization" {
for_each = local.as_signals.cpu_utilization == null ? [] : [""]
content {
target = local.as_signals.cpu_utilization.target
predictive_method = (
local.as_signals.cpu_utilization.optimize_availability == true
? "OPTIMIZE_AVAILABILITY"
: null
)
}
}
dynamic "load_balancing_utilization" {
for_each = local.as_signals.load_balancing_utilization == null ? [] : [""]
content {
target = local.as_signals.load_balancing_utilization.target
}
}
dynamic "metric" {
for_each = toset(
local.as_signals.metrics == null ? [] : local.as_signals.metrics
)
content {
name = metric.value.name
type = metric.value.type
target = metric.value.target_value
single_instance_assignment = metric.value.single_instance_assignment
filter = metric.value.time_series_filter
}
}
dynamic "scaling_schedules" {
for_each = toset(
local.as_signals.schedules == null ? [] : local.as_signals.schedules
)
iterator = schedule
content {
duration_sec = schedule.value.duration_sec
min_required_replicas = schedule.value.min_required_replicas
name = schedule.value.name
schedule = schedule.value.cron_schedule
description = schedule.value.description
disabled = schedule.value.disabled
time_zone = schedule.value.timezone
}
}
}
}
resource "google_compute_region_autoscaler" "default" {
provider = google-beta
count = local.is_regional && var.autoscaler_config != null ? 1 : 0
project = var.project_id
name = var.name
region = var.location
description = var.description
target = google_compute_region_instance_group_manager.default.0.id
autoscaling_policy {
max_replicas = var.autoscaler_config.max_replicas
min_replicas = var.autoscaler_config.min_replicas
cooldown_period = var.autoscaler_config.cooldown_period
dynamic "scale_down_control" {
for_each = local.as_scaling.down == null ? [] : [""]
content {
time_window_sec = local.as_scaling.down.time_window_sec
dynamic "max_scaled_down_replicas" {
for_each = (
local.as_scaling.down.max_replicas_fixed == null &&
local.as_scaling.down.max_replicas_percent == null
? []
: [""]
)
content {
fixed = local.as_scaling.down.max_replicas_fixed
percent = local.as_scaling.down.max_replicas_percent
}
}
}
}
dynamic "scale_in_control" {
for_each = local.as_scaling.in == null ? [] : [""]
content {
time_window_sec = local.as_scaling.in.time_window_sec
dynamic "max_scaled_in_replicas" {
for_each = (
local.as_scaling.in.max_replicas_fixed == null &&
local.as_scaling.in.max_replicas_percent == null
? []
: [""]
)
content {
fixed = local.as_scaling.in.max_replicas_fixed
percent = local.as_scaling.in.max_replicas_percent
}
}
}
}
dynamic "cpu_utilization" {
for_each = local.as_signals.cpu_utilization == null ? [] : [""]
content {
target = local.as_signals.cpu_utilization.target
predictive_method = (
local.as_signals.cpu_utilization.optimize_availability == true
? "OPTIMIZE_AVAILABILITY"
: null
)
}
}
dynamic "load_balancing_utilization" {
for_each = local.as_signals.load_balancing_utilization == null ? [] : [""]
content {
target = local.as_signals.load_balancing_utilization.target
}
}
dynamic "metric" {
for_each = toset(
local.as_signals.metrics == null ? [] : local.as_signals.metrics
)
content {
name = metric.value.name
type = metric.value.type
target = metric.value.target_value
single_instance_assignment = metric.value.single_instance_assignment
filter = metric.value.time_series_filter
}
}
dynamic "scaling_schedules" {
for_each = toset(
local.as_signals.schedules == null ? [] : local.as_signals.schedules
)
iterator = schedule
content {
duration_sec = schedule.value.duration_sec
min_required_replicas = schedule.value.min_required_replicas
name = schedule.value.name
schedule = schedule.cron_schedule
description = schedule.value.description
disabled = schedule.value.disabled
time_zone = schedule.value.timezone
}
}
}
}

119
modules/compute-mig/health-check.tf Normal file
View File

@ -0,0 +1,119 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Health check resource.
locals {
hc = var.health_check_config
hc_grpc = try(local.hc.grpc, null) != null
hc_http = try(local.hc.http, null) != null
hc_http2 = try(local.hc.http2, null) != null
hc_https = try(local.hc.https, null) != null
hc_ssl = try(local.hc.ssl, null) != null
hc_tcp = try(local.hc.tcp, null) != null
}
resource "google_compute_health_check" "default" {
provider = google-beta
count = local.hc != null ? 1 : 0
project = var.project_id
name = var.name
description = local.hc.description
check_interval_sec = local.hc.check_interval_sec
healthy_threshold = local.hc.healthy_threshold
timeout_sec = local.hc.timeout_sec
unhealthy_threshold = local.hc.unhealthy_threshold
dynamic "grpc_health_check" {
for_each = local.hc_grpc ? [""] : []
content {
port = local.hc.grpc.port
port_name = local.hc.grpc.port_name
port_specification = local.hc.grpc.port_specification
grpc_service_name = local.hc.grpc.service_name
}
}
dynamic "http_health_check" {
for_each = local.hc_http ? [""] : []
content {
host = local.hc.http.host
port = local.hc.http.port
port_name = local.hc.http.port_name
port_specification = local.hc.http.port_specification
proxy_header = local.hc.http.proxy_header
request_path = local.hc.http.request_path
response = local.hc.http.response
}
}
dynamic "http2_health_check" {
for_each = local.hc_http2 ? [""] : []
content {
host = local.hc.http.host
port = local.hc.http.port
port_name = local.hc.http.port_name
port_specification = local.hc.http.port_specification
proxy_header = local.hc.http.proxy_header
request_path = local.hc.http.request_path
response = local.hc.http.response
}
}
dynamic "https_health_check" {
for_each = local.hc_https ? [""] : []
content {
host = local.hc.http.host
port = local.hc.http.port
port_name = local.hc.http.port_name
port_specification = local.hc.http.port_specification
proxy_header = local.hc.http.proxy_header
request_path = local.hc.http.request_path
response = local.hc.http.response
}
}
dynamic "ssl_health_check" {
for_each = local.hc_ssl ? [""] : []
content {
port = local.hc.tcp.port
port_name = local.hc.tcp.port_name
port_specification = local.hc.tcp.port_specification
proxy_header = local.hc.tcp.proxy_header
request = local.hc.tcp.request
response = local.hc.tcp.response
}
}
dynamic "tcp_health_check" {
for_each = local.hc_tcp ? [""] : []
content {
port = local.hc.tcp.port
port_name = local.hc.tcp.port_name
port_specification = local.hc.tcp.port_specification
proxy_header = local.hc.tcp.proxy_header
request = local.hc.tcp.request
response = local.hc.tcp.response
}
}
dynamic "log_config" {
for_each = try(local.hc.enable_logging, null) == true ? [""] : []
content {
enable = true
}
}
}

519
modules/compute-mig/main.tf
View File

@ -14,105 +14,50 @@
* limitations under the License.
*/
resource "google_compute_autoscaler" "default" {
provider = google-beta
count = var.regional || var.autoscaler_config == null ? 0 : 1
project = var.project_id
name = var.name
description = "Terraform managed."
zone = var.location
target = google_compute_instance_group_manager.default.0.id
autoscaling_policy {
max_replicas = var.autoscaler_config.max_replicas
min_replicas = var.autoscaler_config.min_replicas
cooldown_period = var.autoscaler_config.cooldown_period
dynamic "cpu_utilization" {
for_each = (
var.autoscaler_config.cpu_utilization_target == null ? [] : [""]
)
content {
target = var.autoscaler_config.cpu_utilization_target
}
}
dynamic "load_balancing_utilization" {
for_each = (
var.autoscaler_config.load_balancing_utilization_target == null ? [] : [""]
)
content {
target = var.autoscaler_config.load_balancing_utilization_target
}
}
dynamic "metric" {
for_each = (
var.autoscaler_config.metric == null
? []
: [var.autoscaler_config.metric]
)
iterator = config
content {
name = config.value.name
single_instance_assignment = config.value.single_instance_assignment
target = config.value.target
type = config.value.type
filter = config.value.filter
}
}
}
locals {
health_check = (
try(var.auto_healing_policies.health_check, null) == null
? try(google_compute_health_check.default.0.self_link, null)
: try(var.auto_healing_policies.health_check, null)
)
instance_group_manager = (
local.is_regional ?
google_compute_region_instance_group_manager.default :
google_compute_instance_group_manager.default
)
is_regional = length(split("-", var.location)) == 2
}
resource "google_compute_instance_group_manager" "default" {
provider = google-beta
count = var.regional ? 0 : 1
project = var.project_id
zone = var.location
name = var.name
base_instance_name = var.name
description = "Terraform-managed."
target_size = var.target_size
target_pools = var.target_pools
wait_for_instances = var.wait_for_instances
provider = google-beta
count = local.is_regional ? 0 : 1
project = var.project_id
zone = var.location
name = var.name
base_instance_name = var.name
description = var.description
target_size = var.target_size
target_pools = var.target_pools
wait_for_instances = try(var.wait_for_instances.enabled, null)
wait_for_instances_status = try(var.wait_for_instances.status, null)
dynamic "all_instances_config" {
for_each = var.all_instances_config == null ? [] : [""]
content {
labels = try(var.all_instances_config.labels, null)
metadata = try(var.all_instances_config.metadata, null)
}
}
dynamic "auto_healing_policies" {
for_each = var.auto_healing_policies == null ? [] : [var.auto_healing_policies]
for_each = var.auto_healing_policies == null ? [] : [""]
iterator = config
content {
health_check = config.value.health_check
initial_delay_sec = config.value.initial_delay_sec
}
}
dynamic "stateful_disk" {
for_each = try(var.stateful_config.mig_config.stateful_disks, {})
iterator = config
content {
device_name = config.key
delete_rule = config.value.delete_rule
}
}
dynamic "update_policy" {
for_each = var.update_policy == null ? [] : [var.update_policy]
iterator = config
content {
type = config.value.type
minimal_action = config.value.minimal_action
min_ready_sec = config.value.min_ready_sec
max_surge_fixed = (
config.value.max_surge_type == "fixed" ? config.value.max_surge : null
)
max_surge_percent = (
config.value.max_surge_type == "percent" ? config.value.max_surge : null
)
max_unavailable_fixed = (
config.value.max_unavailable_type == "fixed" ? config.value.max_unavailable : null
)
max_unavailable_percent = (
config.value.max_unavailable_type == "percent" ? config.value.max_unavailable : null
)
health_check = local.health_check
initial_delay_sec = var.auto_healing_policies.initial_delay_sec
}
}
dynamic "named_port" {
for_each = var.named_ports == null ? {} : var.named_ports
iterator = config
@ -121,167 +66,88 @@ resource "google_compute_instance_group_manager" "default" {
port = config.value
}
}
version {
instance_template = var.default_version.instance_template
name = var.default_version.name
dynamic "stateful_disk" {
for_each = var.stateful_disks
content {
device_name = stateful_disk.key
delete_rule = stateful_disk.value
}
}
dynamic "update_policy" {
for_each = var.update_policy == null ? [] : [var.update_policy]
iterator = p
content {
minimal_action = p.value.minimal_action
type = p.value.type
max_surge_fixed = try(p.value.max_surge.fixed, null)
max_surge_percent = try(p.value.max_surge.percent, null)
max_unavailable_fixed = try(p.value.max_unavailable.fixed, null)
max_unavailable_percent = try(p.value.max_unavailable.percent, null)
min_ready_sec = p.value.min_ready_sec
most_disruptive_allowed_action = p.value.most_disruptive_action
replacement_method = p.value.replacement_method
}
}
version {
instance_template = var.instance_template
name = var.default_version_name
}
dynamic "version" {
for_each = var.versions == null ? {} : var.versions
iterator = version
for_each = var.versions
content {
name = version.key
instance_template = version.value.instance_template
target_size {
fixed = (
version.value.target_type == "fixed" ? version.value.target_size : null
)
percent = (
version.value.target_type == "percent" ? version.value.target_size : null
)
dynamic "target_size" {
for_each = version.value.target_size == null ? [] : [""]
content {
fixed = version.value.target_size.fixed
percent = version.value.target_size.percent
}
}
}
}
}
locals {
instance_group_manager = (
var.regional ?
google_compute_region_instance_group_manager.default :
google_compute_instance_group_manager.default
)
}
resource "google_compute_per_instance_config" "default" {
for_each = try(var.stateful_config.per_instance_config, {})
#for_each = var.stateful_config && var.stateful_config.per_instance_config == null ? {} : length(var.stateful_config.per_instance_config)
zone = var.location
# terraform error, solved with locals
#instance_group_manager = var.regional ? google_compute_region_instance_group_manager.default : google_compute_instance_group_manager.default
instance_group_manager = local.instance_group_manager[0].id
name = each.key
project = var.project_id
minimal_action = try(each.value.update_config.minimal_action, null)
most_disruptive_allowed_action = try(each.value.update_config.most_disruptive_allowed_action, null)
remove_instance_state_on_destroy = try(each.value.update_config.remove_instance_state_on_destroy, null)
preserved_state {
metadata = each.value.metadata
dynamic "disk" {
for_each = try(each.value.stateful_disks, {})
#for_each = var.stateful_config.mig_config.stateful_disks == null ? {} : var.stateful_config.mig_config.stateful_disks
iterator = config
content {
device_name = config.key
source = config.value.source
mode = config.value.mode
delete_rule = config.value.delete_rule
}
}
}
}
resource "google_compute_region_autoscaler" "default" {
provider = google-beta
count = var.regional && var.autoscaler_config != null ? 1 : 0
project = var.project_id
name = var.name
description = "Terraform managed."
region = var.location
target = google_compute_region_instance_group_manager.default.0.id
autoscaling_policy {
max_replicas = var.autoscaler_config.max_replicas
min_replicas = var.autoscaler_config.min_replicas
cooldown_period = var.autoscaler_config.cooldown_period
dynamic "cpu_utilization" {
for_each = (
var.autoscaler_config.cpu_utilization_target == null ? [] : [""]
)
content {
target = var.autoscaler_config.cpu_utilization_target
}
}
dynamic "load_balancing_utilization" {
for_each = (
var.autoscaler_config.load_balancing_utilization_target == null ? [] : [""]
)
content {
target = var.autoscaler_config.load_balancing_utilization_target
}
}
dynamic "metric" {
for_each = (
var.autoscaler_config.metric == null
? []
: [var.autoscaler_config.metric]
)
iterator = config
content {
name = config.value.name
single_instance_assignment = config.value.single_instance_assignment
target = config.value.target
type = config.value.type
filter = config.value.filter
}
}
}
}
resource "google_compute_region_instance_group_manager" "default" {
provider = google-beta
count = var.regional ? 1 : 0
count = local.is_regional ? 1 : 0
project = var.project_id
region = var.location
name = var.name
base_instance_name = var.name
description = "Terraform-managed."
target_size = var.target_size
target_pools = var.target_pools
wait_for_instances = var.wait_for_instances
dynamic "auto_healing_policies" {
for_each = var.auto_healing_policies == null ? [] : [var.auto_healing_policies]
iterator = config
description = var.description
distribution_policy_target_shape = try(
var.distribution_policy.target_shape, null
)
distribution_policy_zones = try(
var.distribution_policy.zones, null
)
target_size = var.target_size
target_pools = var.target_pools
wait_for_instances = try(var.wait_for_instances.enabled, null)
wait_for_instances_status = try(var.wait_for_instances.status, null)
dynamic "all_instances_config" {
for_each = var.all_instances_config == null ? [] : [""]
content {
health_check = config.value.health_check
initial_delay_sec = config.value.initial_delay_sec
}
}
dynamic "stateful_disk" {
for_each = try(var.stateful_config.mig_config.stateful_disks, {})
iterator = config
content {
device_name = config.key
delete_rule = config.value.delete_rule
labels = try(var.all_instances_config.labels, null)
metadata = try(var.all_instances_config.metadata, null)
}
}
dynamic "update_policy" {
for_each = var.update_policy == null ? [] : [var.update_policy]
dynamic "auto_healing_policies" {
for_each = var.auto_healing_policies == null ? [] : [""]
iterator = config
content {
instance_redistribution_type = config.value.instance_redistribution_type
type = config.value.type
minimal_action = config.value.minimal_action
min_ready_sec = config.value.min_ready_sec
max_surge_fixed = (
config.value.max_surge_type == "fixed" ? config.value.max_surge : null
)
max_surge_percent = (
config.value.max_surge_type == "percent" ? config.value.max_surge : null
)
max_unavailable_fixed = (
config.value.max_unavailable_type == "fixed" ? config.value.max_unavailable : null
)
max_unavailable_percent = (
config.value.max_unavailable_type == "percent" ? config.value.max_unavailable : null
)
health_check = local.health_check
initial_delay_sec = var.auto_healing_policies.initial_delay_sec
}
}
dynamic "named_port" {
for_each = var.named_ports == null ? {} : var.named_ports
iterator = config
@ -290,172 +156,49 @@ resource "google_compute_region_instance_group_manager" "default" {
port = config.value
}
}
version {
instance_template = var.default_version.instance_template
name = var.default_version.name
dynamic "stateful_disk" {
for_each = var.stateful_disks
content {
device_name = stateful_disk.key
delete_rule = stateful_disk.value
}
}
dynamic "update_policy" {
for_each = var.update_policy == null ? [] : [var.update_policy]
iterator = p
content {
minimal_action = p.value.minimal_action
type = p.value.type
instance_redistribution_type = p.value.regional_redistribution_type
max_surge_fixed = try(p.value.max_surge.fixed, null)
max_surge_percent = try(p.value.max_surge.percent, null)
max_unavailable_fixed = try(p.value.max_unavailable.fixed, null)
max_unavailable_percent = try(p.value.max_unavailable.percent, null)
min_ready_sec = p.value.min_ready_sec
most_disruptive_allowed_action = p.value.most_disruptive_action
replacement_method = p.value.replacement_method
}
}
version {
instance_template = var.instance_template
name = var.default_version_name
}
dynamic "version" {
for_each = var.versions == null ? {} : var.versions
iterator = version
for_each = var.versions
content {
name = version.key
instance_template = version.value.instance_template
target_size {
fixed = (
version.value.target_type == "fixed" ? version.value.target_size : null
)
percent = (
version.value.target_type == "percent" ? version.value.target_size : null
)
dynamic "target_size" {
for_each = version.value.target_size == null ? [] : [""]
content {
fixed = version.value.target_size.fixed
percent = version.value.target_size.percent
}
}
}
}
}
resource "google_compute_health_check" "http" {
provider = google-beta
count = try(var.health_check_config.type, null) == "http" ? 1 : 0
project = var.project_id
name = var.name
description = "Terraform managed."
check_interval_sec = try(var.health_check_config.config.check_interval_sec, null)
healthy_threshold = try(var.health_check_config.config.healthy_threshold, null)
timeout_sec = try(var.health_check_config.config.timeout_sec, null)
unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
http_health_check {
host = try(var.health_check_config.check.host, null)
port = try(var.health_check_config.check.port, null)
port_name = try(var.health_check_config.check.port_name, null)
port_specification = try(var.health_check_config.check.port_specification, null)
proxy_header = try(var.health_check_config.check.proxy_header, null)
request_path = try(var.health_check_config.check.request_path, null)
response = try(var.health_check_config.check.response, null)
}
dynamic "log_config" {
for_each = try(var.health_check_config.logging, false) ? [""] : []
content {
enable = true
}
}
}
resource "google_compute_health_check" "https" {
provider = google-beta
count = try(var.health_check_config.type, null) == "https" ? 1 : 0
project = var.project_id
name = var.name
description = "Terraform managed."
check_interval_sec = try(var.health_check_config.config.check_interval_sec, null)
healthy_threshold = try(var.health_check_config.config.healthy_threshold, null)
timeout_sec = try(var.health_check_config.config.timeout_sec, null)
unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
https_health_check {
host = try(var.health_check_config.check.host, null)
port = try(var.health_check_config.check.port, null)
port_name = try(var.health_check_config.check.port_name, null)
port_specification = try(var.health_check_config.check.port_specification, null)
proxy_header = try(var.health_check_config.check.proxy_header, null)
request_path = try(var.health_check_config.check.request_path, null)
response = try(var.health_check_config.check.response, null)
}
dynamic "log_config" {
for_each = try(var.health_check_config.logging, false) ? [""] : []
content {
enable = true
}
}
}
resource "google_compute_health_check" "tcp" {
provider = google-beta
count = try(var.health_check_config.type, null) == "tcp" ? 1 : 0
project = var.project_id
name = var.name
description = "Terraform managed."
check_interval_sec = try(var.health_check_config.config.check_interval_sec, null)
healthy_threshold = try(var.health_check_config.config.healthy_threshold, null)
timeout_sec = try(var.health_check_config.config.timeout_sec, null)
unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
tcp_health_check {
port = try(var.health_check_config.check.port, null)
port_name = try(var.health_check_config.check.port_name, null)
port_specification = try(var.health_check_config.check.port_specification, null)
proxy_header = try(var.health_check_config.check.proxy_header, null)
request = try(var.health_check_config.check.request, null)
response = try(var.health_check_config.check.response, null)
}
dynamic "log_config" {
for_each = try(var.health_check_config.logging, false) ? [""] : []
content {
enable = true
}
}
}
resource "google_compute_health_check" "ssl" {
provider = google-beta
count = try(var.health_check_config.type, null) == "ssl" ? 1 : 0
project = var.project_id
name = var.name
description = "Terraform managed."
check_interval_sec = try(var.health_check_config.config.check_interval_sec, null)
healthy_threshold = try(var.health_check_config.config.healthy_threshold, null)
timeout_sec = try(var.health_check_config.config.timeout_sec, null)
unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
ssl_health_check {
port = try(var.health_check_config.check.port, null)
port_name = try(var.health_check_config.check.port_name, null)
port_specification = try(var.health_check_config.check.port_specification, null)
proxy_header = try(var.health_check_config.check.proxy_header, null)
request = try(var.health_check_config.check.request, null)
response = try(var.health_check_config.check.response, null)
}
dynamic "log_config" {
for_each = try(var.health_check_config.logging, false) ? [""] : []
content {
enable = true
}
}
}
resource "google_compute_health_check" "http2" {
provider = google-beta
count = try(var.health_check_config.type, null) == "http2" ? 1 : 0
project = var.project_id
name = var.name
description = "Terraform managed."
check_interval_sec = try(var.health_check_config.config.check_interval_sec, null)
healthy_threshold = try(var.health_check_config.config.healthy_threshold, null)
timeout_sec = try(var.health_check_config.config.timeout_sec, null)
unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
http2_health_check {
host = try(var.health_check_config.check.host, null)
port = try(var.health_check_config.check.port, null)
port_name = try(var.health_check_config.check.port_name, null)
port_specification = try(var.health_check_config.check.port_specification, null)
proxy_header = try(var.health_check_config.check.proxy_header, null)
request_path = try(var.health_check_config.check.request_path, null)
response = try(var.health_check_config.check.response, null)
}
dynamic "log_config" {
for_each = try(var.health_check_config.logging, false) ? [""] : []
content {
enable = true
}
}
}

Some files were not shown because too many files have changed in this diff Show More