Merge pull request #3 from maunope/vpc-sc

Merging updated from VPC-SC branch

CHANGELOG.md

@@ -5,9 +5,15 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 <!-- None < 2022-09-09 18:02:15+00:00 -->
+- [[#939](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/939)] Temporarily duplicate cloud armor example ([ludoo](https://github.com/ludoo)) <!-- 2022-11-02 09:36:04+00:00 -->
 
 ### BLUEPRINTS
 
+- [[#952](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/952)] Remove duplicate GLB+CA blueprint folder ([ludoo](https://github.com/ludoo)) <!-- 2022-11-07 12:46:22+00:00 -->
+- [[#949](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/949)] **incompatible change:** Refactor VPC firewall module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-04 12:56:08+00:00 -->
+- [[#945](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/945)] Org policy factory ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 11:30:58+00:00 -->
+- [[#941](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/941)] **incompatible change:** Refactor ILB module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-02 17:05:21+00:00 -->
+- [[#936](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/936)] Enable org policy service and add README notice to modules ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 13:25:08+00:00 -->
 - [[#931](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/931)] **incompatible change:** Refactor compute-mig module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 08:39:00+00:00 -->
 - [[#932](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/932)] feat(project-factory): introduce additive iam bindings to project-fac… ([Malet](https://github.com/Malet)) <!-- 2022-10-31 17:24:25+00:00 -->
 - [[#925](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/925)] Network dashboard: update main.tf and README following #922 ([brianhmj](https://github.com/brianhmj)) <!-- 2022-10-28 15:49:12+00:00 -->
@@ -46,6 +52,9 @@ All notable changes to this project will be documented in this file.
 
 ### DOCUMENTATION
 
+- [[#961](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/961)] Remove extra file from root ([ludoo](https://github.com/ludoo)) <!-- 2022-11-09 07:53:11+00:00 -->
+- [[#943](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/943)] Update bootstrap README.md with unique project id requirements ([KPRepos](https://github.com/KPRepos)) <!-- 2022-11-03 22:22:22+00:00 -->
+- [[#937](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/937)] Fix typos in blueprints README.md ([kumar-dhanagopal](https://github.com/kumar-dhanagopal)) <!-- 2022-11-02 07:39:26+00:00 -->
 - [[#921](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/921)] Align documentation, move glb blueprint ([ludoo](https://github.com/ludoo)) <!-- 2022-10-26 12:31:04+00:00 -->
 - [[#898](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/898)] Update FAST bootstrap README.md ([juliocc](https://github.com/juliocc)) <!-- 2022-10-19 15:15:36+00:00 -->
 - [[#878](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/878)] chore: update cft and fabric ([bharathkkb](https://github.com/bharathkkb)) <!-- 2022-10-12 15:38:06+00:00 -->
@@ -54,6 +63,12 @@ All notable changes to this project will be documented in this file.
 
 ### FAST
 
+- [[#956](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/956)] FAST: bootstrap and extra stage CI/CD improvements and fixes ([ludoo](https://github.com/ludoo)) <!-- 2022-11-08 08:38:16+00:00 -->
+- [[#949](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/949)] **incompatible change:** Refactor VPC firewall module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-04 12:56:08+00:00 -->
+- [[#943](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/943)] Update bootstrap README.md with unique project id requirements ([KPRepos](https://github.com/KPRepos)) <!-- 2022-11-03 22:22:22+00:00 -->
+- [[#948](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/948)] Use display_name instead of description for FAST service accounts ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 16:22:18+00:00 -->
+- [[#947](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/947)] Use org policy factory for resman stage ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 14:04:08+00:00 -->
+- [[#941](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/941)] **incompatible change:** Refactor ILB module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-02 17:05:21+00:00 -->
 - [[#935](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/935)] FAST: enable org policy API, fix run.allowedIngress value ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 08:52:03+00:00 -->
 - [[#931](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/931)] **incompatible change:** Refactor compute-mig module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 08:39:00+00:00 -->
 - [[#930](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/930)] **incompatible change:** Update organization/folder/project modules to use new org policies API and tf1.3 optionals ([juliocc](https://github.com/juliocc)) <!-- 2022-10-28 16:21:06+00:00 -->
@@ -77,6 +92,16 @@ All notable changes to this project will be documented in this file.
 
 ### MODULES
 
+- [[#958](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/958)] Add support for org policy custom constraints ([averbuks](https://github.com/averbuks)) <!-- 2022-11-09 09:07:46+00:00 -->
+- [[#960](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/960)] Fix README typo in firewall module ([valeriobponza](https://github.com/valeriobponza)) <!-- 2022-11-08 23:25:34+00:00 -->
+- [[#953](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/953)] Added IAM Additive and converted some outputs to static ([muresan](https://github.com/muresan)) <!-- 2022-11-07 13:20:17+00:00 -->
+- [[#951](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/951)] cloud-functions v2 - fix reference to bucket_name ([wiktorn](https://github.com/wiktorn)) <!-- 2022-11-06 07:32:39+00:00 -->
+- [[#949](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/949)] **incompatible change:** Refactor VPC firewall module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-04 12:56:08+00:00 -->
+- [[#946](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/946)] **incompatible change:** Deprecate organization-policy module ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 11:56:12+00:00 -->
+- [[#945](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/945)] Org policy factory ([juliocc](https://github.com/juliocc)) <!-- 2022-11-03 11:30:58+00:00 -->
+- [[#941](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/941)] **incompatible change:** Refactor ILB module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-02 17:05:21+00:00 -->
+- [[#940](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/940)] Ensure the implementation of org policies is consistent ([juliocc](https://github.com/juliocc)) <!-- 2022-11-02 09:55:21+00:00 -->
+- [[#936](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/936)] Enable org policy service and add README notice to modules ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 13:25:08+00:00 -->
 - [[#931](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/931)] **incompatible change:** Refactor compute-mig module for Terraform 1.3 ([ludoo](https://github.com/ludoo)) <!-- 2022-11-01 08:39:00+00:00 -->
 - [[#930](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/930)] **incompatible change:** Update organization/folder/project modules to use new org policies API and tf1.3 optionals ([juliocc](https://github.com/juliocc)) <!-- 2022-10-28 16:21:06+00:00 -->
 - [[#926](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/926)] Fix backwards compatibility for vpc subnet descriptions ([ludoo](https://github.com/ludoo)) <!-- 2022-10-28 06:13:04+00:00 -->
@@ -119,6 +144,8 @@ All notable changes to this project will be documented in this file.
 
 ### TOOLS
 
+- [[#950](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/950)] Add a pytest fixture to convert tfvars to yaml ([ludoo](https://github.com/ludoo)) <!-- 2022-11-04 17:37:24+00:00 -->
+- [[#942](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/942)] Bump tftest and improve dns tests ([juliocc](https://github.com/juliocc)) <!-- 2022-11-02 19:38:01+00:00 -->
 - [[#919](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/919)] Rename workflow names ([juliocc](https://github.com/juliocc)) <!-- 2022-10-25 15:22:51+00:00 -->
 - [[#902](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/902)] Bring back sorted variables check ([juliocc](https://github.com/juliocc)) <!-- 2022-10-20 17:08:17+00:00 -->
 - [[#887](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/887)] Disable parallel execution of tests and plugin cache ([ludoo](https://github.com/ludoo)) <!-- 2022-10-14 17:52:38+00:00 -->

README.md

@@ -29,7 +29,7 @@ The current list of modules supports most of the core foundational and networkin
 
 Currently available modules:
 
-- **foundational** - [billing budget](./modules/billing-budget), [Cloud Identity group](./modules/cloud-identity-group/), [folder](./modules/folder), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [organization](./modules/organization), [organization-policy](./modules/organization-policy), [project](./modules/project), [projects-data-source](./modules/projects-data-source)
+- **foundational** - [billing budget](./modules/billing-budget), [Cloud Identity group](./modules/cloud-identity-group/), [folder](./modules/folder), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [organization](./modules/organization), [project](./modules/project), [projects-data-source](./modules/projects-data-source)
 - **networking** - [DNS](./modules/dns), [Cloud Endpoints](./modules/endpoints), [address reservation](./modules/net-address), [NAT](./modules/net-cloudnat), [Global Load Balancer (classic)](./modules/net-glb/), [L4 ILB](./modules/net-ilb), [L7 ILB](./modules/net-ilb-l7), [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC peering](./modules/net-vpc-peering), [VPN dynamic](./modules/net-vpn-dynamic), [HA VPN](./modules/net-vpn-ha), [VPN static](./modules/net-vpn-static), [Service Directory](./modules/service-directory)
 - **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid), [GKE cluster](./modules/gke-cluster), [GKE hub](./modules/gke-hub), [GKE nodepool](./modules/gke-nodepool)
 - **data** - [BigQuery dataset](./modules/bigquery-dataset), [Bigtable instance](./modules/bigtable-instance), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag), [Datafusion](./modules/datafusion), [GCS](./modules/gcs), [Pub/Sub](./modules/pubsub)

blueprints/cloud-operations/network-dashboard/README.md

@@ -29,11 +29,11 @@ Clone this repository, then go through the following steps to create resources:
 
 Note: Org level viewing permission is required for some metrics such as firewall policies.
 
-Once the resources are deployed, go to the following page to see the dashboard: https://console.cloud.google.com/monitoring/dashboards?project=<YOUR-MONITORING-PROJECT>.
+Once the resources are deployed, go to the following page to see the dashboard: https://console.cloud.google.com/monitoring/dashboards?project=<YOUR-MONITORING-PROJECT> (or <YOUR-METRICS-PROJECT> if populated)
 A dashboard called "quotas-utilization" should be created.
 
 The Cloud Function runs every 10 minutes by default so you should start getting some data points after a few minutes.
-You can use the metric explorer to view the data points for the different custom metrics created: https://console.cloud.google.com/monitoring/metrics-explorer?project=<YOUR-MONITORING-PROJECT>.
+You can use the metric explorer to view the data points for the different custom metrics created: https://console.cloud.google.com/monitoring/metrics-explorer?project=<YOUR-MONITORING-PROJECT> (or <YOUR-METRICS-PROJECT> if populated).
 You can change this frequency by modifying the "schedule_cron" variable in variables.tf.
 
 Note that some charts in the dashboard align values over 1h so you might need to wait 1h to see charts on the dashboard views.
@@ -70,7 +70,6 @@ Note that metrics are created in the cloud-function/metrics.yaml file. You can a
 - The CF assumes custom routes importing/exporting is ON, this impacts static and dynamic routes usage calculation
 - The CF assumes all networks in peering groups have the same global routing and custom routes sharing configuration
 
-
 ## Next steps and ideas
 In a future release, we could support:
 - Google managed VPCs that are peered with PSA (such as Cloud SQL or Memorystore)
@@ -88,13 +87,15 @@ If you are interested in this and/or would like to contribute, please contact le
 |---|---|:---:|:---:|:---:|
 | [billing_account](variables.tf#L17) | The ID of the billing account to associate this project with | <code></code> | ✓ |  |
 | [monitored_projects_list](variables.tf#L36) | ID of the projects to be monitored (where limits and quotas data will be pulled) | <code>list&#40;string&#41;</code> | ✓ |  |
-| [organization_id](variables.tf#L47) | The organization id for the associated services | <code></code> | ✓ |  |
+| [organization_id](variables.tf#L54) | The organization id for the associated services | <code></code> | ✓ |  |
-| [prefix](variables.tf#L51) | Customer name to use as prefix for monitoring project | <code></code> | ✓ |  |
+| [prefix](variables.tf#L58) | Customer name to use as prefix for monitoring project | <code></code> | ✓ |  |
 | [cf_version](variables.tf#L21) | Cloud Function version 2nd Gen or 1st Gen. Possible options: 'V1' or 'V2'.Use CFv2 if your Cloud Function timeouts after 9 minutes. By default it is using CFv1. | <code></code> |  | <code>V1</code> |
+| [metrics_project_id](variables.tf#L46) | Optional, populate to write metrics and deploy the dashboard in a separated project | <code></code> |  |  |
 | [monitored_folders_list](variables.tf#L30) | ID of the projects to be monitored (where limits and quotas data will be pulled) | <code>list&#40;string&#41;</code> |  | <code>&#91;&#93;</code> |
-| [monitoring_project_id](variables.tf#L41) | Monitoring project where the dashboard will be created and the solution deployed; a project will be created if set to empty string | <code></code> |  |  |
+| [monitoring_project_id](variables.tf#L41) | Monitoring project where the dashboard will be created and the solution deployed; a project will be created if set to empty string, if metrics_project_id is provided, metrics and dashboard will be deployed there  | <code></code> |  |  |
-| [project_monitoring_services](variables.tf#L55) | Service APIs enabled in the monitoring project if it will be created. | <code></code> |  | <code title="&#91;&#10;  &#34;artifactregistry.googleapis.com&#34;,&#10;  &#34;cloudasset.googleapis.com&#34;,&#10;  &#34;cloudbilling.googleapis.com&#34;,&#10;  &#34;cloudbuild.googleapis.com&#34;,&#10;  &#34;cloudresourcemanager.googleapis.com&#34;,&#10;  &#34;cloudscheduler.googleapis.com&#34;,&#10;  &#34;compute.googleapis.com&#34;,&#10;  &#34;cloudfunctions.googleapis.com&#34;,&#10;  &#34;iam.googleapis.com&#34;,&#10;  &#34;iamcredentials.googleapis.com&#34;,&#10;  &#34;logging.googleapis.com&#34;,&#10;  &#34;monitoring.googleapis.com&#34;,&#10;  &#34;run.googleapis.com&#34;,&#10;  &#34;serviceusage.googleapis.com&#34;&#10;&#93;">&#91;&#8230;&#93;</code> |
+| [project_monitoring_services](variables.tf#L63) | Service APIs enabled in the monitoring project if it will be created. | <code></code> |  | <code title="&#91;&#10;  &#34;artifactregistry.googleapis.com&#34;,&#10;  &#34;cloudasset.googleapis.com&#34;,&#10;  &#34;cloudbilling.googleapis.com&#34;,&#10;  &#34;cloudbuild.googleapis.com&#34;,&#10;  &#34;cloudfunctions.googleapis.com&#34;,&#10;  &#34;cloudresourcemanager.googleapis.com&#34;,&#10;  &#34;cloudscheduler.googleapis.com&#34;,&#10;  &#34;compute.googleapis.com&#34;,&#10;  &#34;iam.googleapis.com&#34;,&#10;  &#34;iamcredentials.googleapis.com&#34;,&#10;  &#34;logging.googleapis.com&#34;,&#10;  &#34;monitoring.googleapis.com&#34;,&#10;  &#34;pubsub.googleapis.com&#34;,&#10;  &#34;run.googleapis.com&#34;,&#10;  &#34;servicenetworking.googleapis.com&#34;,&#10;  &#34;serviceusage.googleapis.com&#34;,&#10;  &#34;storage-component.googleapis.com&#34;&#10;&#93;">&#91;&#8230;&#93;</code> |
-| [region](variables.tf#L75) | Region used to deploy the cloud functions and scheduler | <code></code> |  | <code>europe-west1</code> |
+| [region](variables.tf#L88) | Region used to deploy the cloud functions and scheduler | <code></code> |  | <code>europe-west1</code> |
-| [schedule_cron](variables.tf#L80) | Cron format schedule to run the Cloud Function. Default is every 10 minutes. | <code></code> |  | <code>&#42;&#47;10 &#42; &#42; &#42; &#42;</code> |
+| [schedule_cron](variables.tf#L93) | Cron format schedule to run the Cloud Function. Default is every 10 minutes. | <code></code> |  | <code>&#42;&#47;10 &#42; &#42; &#42; &#42;</code> |
+| [vpc_connector_name](variables.tf#L99) | Serverless VPC connection name for the Cloud Function | <code></code> |  |  |
 
 <!-- END TFDOC -->

blueprints/cloud-operations/network-dashboard/cloud-function/metrics/subnets.py

@@ -25,7 +25,6 @@ import ipaddress
 def get_all_subnets(config):
   '''
     Returns a dictionary with subnet level informations (such as IP utilization)
-
       Parameters:
         config (dict): The dict containing config like clients and limits
       Returns:
@@ -83,19 +82,16 @@ def get_all_subnets(config):
   return subnet_dict
 
 
-def compute_subnet_utilization(config, all_subnets_dict):
+def compute_subnet_utilization_vms(config, read_mask, all_subnets_dict):
   '''
-    Counts resources (VMs, ILBs, reserved IPs) using private IPs in the different subnets.
+    Counts VMs using private IPs in the different subnets.
       Parameters:
         config (dict): Dict containing config like clients and limits
+        read_mask (FieldMask): read_mask to get additional metadata from Cloud Asset Inventory
         all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
-
       Returns:
         None
   '''
-  read_mask = field_mask_pb2.FieldMask()
-  read_mask.FromJsonString('name,versionedResources')
-
   response_vm = config["clients"]["asset_client"].search_all_resources(
       request={
           "scope": f"organizations/{config['organization']}",
@@ -123,6 +119,17 @@ def compute_subnet_utilization(config, all_subnets_dict):
           all_subnets_dict[project_id][f"{subnet_region}/{subnet_name}"][
               'used_ip_addresses'] += 1
 
+
+def compute_subnet_utilization_ilbs(config, read_mask, all_subnets_dict):
+  '''
+    Counts ILBs using private IPs in the different subnets.
+      Parameters:
+        config (dict): Dict containing config like clients and limits
+        read_mask (FieldMask): read_mask to get additional metadata from Cloud Asset Inventory
+        all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
+      Returns:
+        None
+  '''
   response_ilb = config["clients"]["asset_client"].search_all_resources(
       request={
           "scope": f"organizations/{config['organization']}",
@@ -131,7 +138,6 @@ def compute_subnet_utilization(config, all_subnets_dict):
           "page_size": config["page_size"],
       })
 
-  # Counting IP addresses for GCE Internal Load Balancers
   for asset in response_ilb:
     internal = False
     psc = False
@@ -139,9 +145,11 @@ def compute_subnet_utilization(config, all_subnets_dict):
     subnet_name = ''
     subnet_region = ''
     address = ''
+    network = ''
     for versioned in asset.versioned_resources:
       for field_name, field_value in versioned.resource.items():
-        if 'loadBalancingScheme' in field_name and field_value in ['INTERNAL', 'INTERNAL_MANAGED']:
+        if 'loadBalancingScheme' in field_name and field_value in [
+            'INTERNAL', 'INTERNAL_MANAGED']:
           internal = True
         # We want to count only accepted PSC endpoint Forwarding Rule
         # If the PSC endpoint Forwarding Rule is pending, we will count it in the reserved IP addresses
@@ -151,6 +159,7 @@ def compute_subnet_utilization(config, all_subnets_dict):
           address = field_value
         elif field_name == 'network':
           project_id = field_value.split('/')[6]
+          network = field_value.split('/')[-1]
         elif 'subnetwork' in field_name:
           subnet_name = field_value.split('/')[-1]
           subnet_region = field_value.split('/')[-3]
@@ -163,9 +172,21 @@ def compute_subnet_utilization(config, all_subnets_dict):
       # We need to find the correct subnet with IP address matching
       ip_address = ipaddress.ip_address(address)
       for subnet_key, subnet_dict in all_subnets_dict[project_id].items():
-        if ip_address in ipaddress.ip_network(subnet_dict['ip_cidr_range']):
+        if subnet_dict["network_name"] == network:
-          all_subnets_dict[project_id][subnet_key]['used_ip_addresses'] += 1
+          if ip_address in ipaddress.ip_network(subnet_dict['ip_cidr_range']):
+            all_subnets_dict[project_id][subnet_key]['used_ip_addresses'] += 1
 
+
+def compute_subnet_utilization_addresses(config, read_mask, all_subnets_dict):
+  '''
+    Counts reserved IP addresses in the different subnets.
+      Parameters:
+        config (dict): Dict containing config like clients and limits
+        read_mask (FieldMask): read_mask to get additional metadata from Cloud Asset Inventory
+        all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
+      Returns:
+        None
+  '''
   response_reserved_ips = config["clients"][
       "asset_client"].search_all_resources(
           request={
@@ -185,8 +206,11 @@ def compute_subnet_utilization(config, all_subnets_dict):
     subnet_region = ""
     address = ""
     prefixLength = ""
+    address_name = ""
     for versioned in asset.versioned_resources:
       for field_name, field_value in versioned.resource.items():
+        if field_name == 'name':
+          address_name = field_value
         if field_name == 'purpose':
           purpose = field_value
         elif field_name == 'region':
@@ -214,9 +238,90 @@ def compute_subnet_utilization(config, all_subnets_dict):
           'used_ip_addresses'] += 1
     # PSA Range for Cloud SQL, MemoryStore, etc.
     elif purpose == "VPC_PEERING":
-      # TODO: PSA range to be handled later
+      ip_range = f"{address}/{int(prefixLength)}"
-      # print("PSA range to be handled later:", address, prefixLength, network_name)
+      net = ipaddress.ip_network(ip_range)
-      continue
+      # Note that 4 IP addresses are reserved by GCP in all subnets
+      # Source: https://cloud.google.com/vpc/docs/subnets#reserved_ip_addresses_in_every_subnet
+      total_ip_addresses = int(net.num_addresses) - 4
+      all_subnets_dict[project_id][f"psa/{address_name}"] = {
+          'name': f"psa/{address_name}",
+          'region': subnet_region,
+          'ip_cidr_range': ip_range,
+          'total_ip_addresses': total_ip_addresses,
+          'used_ip_addresses': 0,
+          'network_name': network_name
+      }
+
+
+def compute_subnet_utilization_redis(config, read_mask, all_subnets_dict):
+  '''
+    Counts Redis (Memorystore) instances using private IPs in the different subnets.
+      Parameters:
+        config (dict): Dict containing config like clients and limits
+        read_mask (FieldMask): read_mask to get additional metadata from Cloud Asset Inventory
+        all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
+      Returns:
+        None
+  '''
+  response_redis = config["clients"]["asset_client"].search_all_resources(
+      request={
+          "scope": f"organizations/{config['organization']}",
+          "asset_types": ["redis.googleapis.com/Instance"],
+          "read_mask": read_mask,
+          "page_size": config["page_size"],
+      })
+
+  for asset in response_redis:
+    ip_range = ""
+    connect_mode = ""
+    network_name = ""
+    project_id = ""
+    region = ""
+    for versioned in asset.versioned_resources:
+      for field_name, field_value in versioned.resource.items():
+        if field_name == 'locationId':
+          region = field_value[0:-2]
+        if field_name == 'authorizedNetwork':
+          network_name = field_value.split('/')[-1]
+          project_id = field_value.split('/')[1]
+        if field_name == 'reservedIpRange':
+          ip_range = field_value
+        if field_name == 'connectMode':
+          connect_mode = field_value
+
+    # Only handling PSA for Redis for now
+    if connect_mode == "PRIVATE_SERVICE_ACCESS":
+      redis_ip_range = ipaddress.ip_network(ip_range)
+      for subnet_key, subnet_dict in all_subnets_dict[project_id].items():
+        if subnet_dict["network_name"] == network_name:
+          # Reddis instance asset doesn't contain the subnet information in Asset Inventory
+          # We need to find the correct subnet range with IP address matching to compute the utilization
+          if redis_ip_range.overlaps(
+              ipaddress.ip_network(subnet_dict['ip_cidr_range'])):
+            all_subnets_dict[project_id][subnet_key][
+                'used_ip_addresses'] += redis_ip_range.num_addresses
+            all_subnets_dict[project_id][subnet_key]['region'] = region
+
+
+def compute_subnet_utilization(config, all_subnets_dict):
+  '''
+    Counts resources (VMs, ILBs, reserved IPs) using private IPs in the different subnets.
+      Parameters:
+        config (dict): Dict containing config like clients and limits
+        all_subnets_dict (dict): Dict containing the information for each subnets in the GCP organization
+      Returns:
+        None
+  '''
+  read_mask = field_mask_pb2.FieldMask()
+  read_mask.FromJsonString('name,versionedResources')
+
+  compute_subnet_utilization_vms(config, read_mask, all_subnets_dict)
+  compute_subnet_utilization_ilbs(config, read_mask, all_subnets_dict)
+  compute_subnet_utilization_addresses(config, read_mask, all_subnets_dict)
+  # TODO: Other PSA services such as FileStore, Cloud SQL
+  compute_subnet_utilization_redis(config, read_mask, all_subnets_dict)
+
+  # TODO: Handle secondary ranges and count GKE pods
 
 
 def get_subnets(config, metrics_dict):

blueprints/cloud-operations/network-dashboard/main.tf

@@ -21,6 +21,7 @@ locals {
   folder_ids         = toset(var.monitored_folders_list)
   folders            = join(",", local.folder_ids)
   monitoring_project = var.monitoring_project_id == "" ? module.project-monitoring[0].project_id : var.monitoring_project_id
+  metrics_project = var.metrics_project_id == "" ? (var.monitoring_project_id == "" ? module.project-monitoring[0].project_id : var.monitoring_project_id) : var.metrics_project_id
 }
 
 ################################################
@@ -60,7 +61,7 @@ module "service-account-function" {
   }
 
   iam_project_roles = {
-    "${local.monitoring_project}" = [
+    "${local.metrics_project}" = [
       "roles/monitoring.metricWriter",
     ]
   }
@@ -141,6 +142,13 @@ module "cloud-function" {
     lifecycle_delete_age = null
   }
   region = var.region
+  vpc_connector = (var.vpc_connector_name != "" ?
+    {
+      create          = false
+      name            = var.vpc_connector_name
+      egress_settings = "ALL_TRAFFIC"
+  } : null)
+
 
   bundle_config = {
     source_dir  = "cloud-function"
@@ -160,7 +168,7 @@ module "cloud-function" {
   environment_variables = {
     MONITORED_PROJECTS_LIST = local.projects
     MONITORED_FOLDERS_LIST  = local.folders
-    MONITORING_PROJECT_ID   = local.monitoring_project
+    MONITORING_PROJECT_ID   = local.metrics_project
     ORGANIZATION_ID         = var.organization_id
     CF_VERSION              = var.cf_version
   }
@@ -182,5 +190,9 @@ module "cloud-function" {
 
 resource "google_monitoring_dashboard" "dashboard" {
   dashboard_json = file("${path.module}/dashboards/quotas-utilization.json")
+<<<<<<< HEAD
+  project        = local.metrics_project
+=======
   project        = local.monitoring_project
+>>>>>>> b7bfcf3575cda18a2fdd2862c72e33c1648c0aa4
 }

blueprints/cloud-operations/network-dashboard/variables.tf

@@ -39,10 +39,17 @@ variable "monitored_projects_list" {
 }
 
 variable "monitoring_project_id" {
-  description = "Monitoring project where the dashboard will be created and the solution deployed; a project will be created if set to empty string"
+  description = "Monitoring project where the dashboard will be created and the solution deployed; a project will be created if set to empty string, if metrics_project_id is provided, metrics and dashboard will be deployed there "
   default     = ""
 }
 
+variable "metrics_project_id" {
+  description = "Optional, populate to write metrics and deploy the dashboard in a separated project"
+  default     = ""
+}
+
+
+
 
 variable "organization_id" {
   description = "The organization id for the associated services"
@@ -88,3 +95,13 @@ variable "schedule_cron" {
   description = "Cron format schedule to run the Cloud Function. Default is every 10 minutes."
   default     = "*/10 * * * *"
 }
+
+
+variable "vpc_connector_name" {
+  description = "Serverless VPC connection name for the Cloud Function"
+  default     = ""
+}
+
+
+
+

blueprints/cloud-operations/packer-image-builder/main.tf

@@ -67,17 +67,16 @@ module "firewall" {
   source     = "../../../modules/net-vpc-firewall"
   project_id = module.project.project_id
   network    = module.vpc.name
-  custom_rules = {
+  ingress_rules = {
     image-builder-ingress-builder-vm = {
       description          = "Allow image builder vm ingress traffic"
-      direction            = "INGRESS"
+      source_ranges        = var.packer_source_cidrs
-      action               = "allow"
-      sources              = []
-      ranges               = var.packer_source_cidrs
       targets              = [module.service-account-image-builder-vm.email]
       use_service_accounts = true
-      rules                = [{ protocol = "tcp", ports = [22, 5985, 5986] }]
+      rules = [{
-      extra_attributes     = {}
+        protocol = "tcp"
+        ports    = [22, 5985, 5986]
+      }]
     }
   }
 }

blueprints/cloud-operations/vm-migration/single-project/main.tf

@@ -66,24 +66,7 @@ module "landing-vpc" {
 }
 
 module "landing-vpc-firewall" {
-  source              = "../../../../modules/net-vpc-firewall"
+  source     = "../../../../modules/net-vpc-firewall"
-  project_id          = module.landing-project.project_id
+  project_id = module.landing-project.project_id
-  network             = module.landing-vpc.name
+  network    = module.landing-vpc.name
-  admin_ranges        = []
-  http_source_ranges  = []
-  https_source_ranges = []
-  ssh_source_ranges   = []
-  custom_rules = {
-    allow-ssh = {
-      description          = "Allow SSH from IAP"
-      direction            = "INGRESS"
-      action               = "allow"
-      sources              = []
-      ranges               = ["35.235.240.0/20"]
-      targets              = []
-      use_service_accounts = false
-      rules                = [{ protocol = "tcp", ports = ["22"] }]
-      extra_attributes     = {}
-    }
-  }
 }

blueprints/data-solutions/cloudsql-multiregion/main.tf

@@ -128,11 +128,13 @@ module "vpc" {
 }
 
 module "firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  count        = local.use_shared_vpc ? 0 : 1
+  count      = local.use_shared_vpc ? 0 : 1
-  project_id   = module.project.project_id
+  project_id = module.project.project_id
-  network      = module.vpc.0.name
+  network    = module.vpc.0.name
-  admin_ranges = ["10.0.0.0/20"]
+  default_rules_config = {
+    admin_ranges = ["10.0.0.0/20"]
+  }
 }
 
 module "nat" {

blueprints/data-solutions/cmek-via-centralized-kms/main.tf

@@ -59,10 +59,12 @@ module "vpc" {
 }
 
 module "vpc-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id   = module.project-service.project_id
+  project_id = module.project-service.project_id
-  network      = module.vpc.name
+  network    = module.vpc.name
-  admin_ranges = [var.vpc_ip_cidr_range]
+  default_rules_config = {
+    admin_ranges = [var.vpc_ip_cidr_range]
+  }
 }
 
 ###############################################################################

blueprints/data-solutions/data-platform-foundations/02-load.tf

@@ -118,11 +118,13 @@ module "load-vpc" {
 }
 
 module "load-vpc-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  count        = local.use_shared_vpc ? 0 : 1
+  count      = local.use_shared_vpc ? 0 : 1
-  project_id   = module.load-project.project_id
+  project_id = module.load-project.project_id
-  network      = module.load-vpc.0.name
+  network    = module.load-vpc.0.name
-  admin_ranges = ["10.10.0.0/24"]
+  default_rules_config = {
+    admin_ranges = ["10.10.0.0/24"]
+  }
 }
 
 module "load-nat" {

blueprints/data-solutions/data-platform-foundations/03-orchestration.tf

@@ -133,11 +133,13 @@ module "orch-vpc" {
 }
 
 module "orch-vpc-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  count        = local.use_shared_vpc ? 0 : 1
+  count      = local.use_shared_vpc ? 0 : 1
-  project_id   = module.orch-project.project_id
+  project_id = module.orch-project.project_id
-  network      = module.orch-vpc.0.name
+  network    = module.orch-vpc.0.name
-  admin_ranges = ["10.10.0.0/24"]
+  default_rules_config = {
+    admin_ranges = ["10.10.0.0/24"]
+  }
 }
 
 module "orch-nat" {

blueprints/data-solutions/data-platform-foundations/04-transformation.tf

@@ -142,11 +142,13 @@ module "transf-vpc" {
 }
 
 module "transf-vpc-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  count        = local.use_shared_vpc ? 0 : 1
+  count      = local.use_shared_vpc ? 0 : 1
-  project_id   = module.transf-project.project_id
+  project_id = module.transf-project.project_id
-  network      = module.transf-vpc.0.name
+  network    = module.transf-vpc.0.name
-  admin_ranges = ["10.10.0.0/24"]
+  default_rules_config = {
+    admin_ranges = ["10.10.0.0/24"]
+  }
 }
 
 module "transf-nat" {

blueprints/data-solutions/data-playground/main.tf

@@ -72,22 +72,19 @@ module "vpc" {
 }
 
 module "vpc-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id   = module.project.project_id
+  project_id = module.project.project_id
-  network      = module.vpc.name
+  network    = module.vpc.name
-  admin_ranges = [var.vpc_config.ip_cidr_range]
+  default_rules_config = {
-  custom_rules = {
+    admin_ranges = [var.vpc_config.ip_cidr_range]
+  }
+  ingress_rules = {
     #TODO Remove and rely on 'ssh' tag once terraform-provider-google/issues/9273 is fixed
     ("${var.prefix}-iap") = {
-      description          = "Enable SSH from IAP on Notebooks."
+      description   = "Enable SSH from IAP on Notebooks."
-      direction            = "INGRESS"
+      source_ranges = ["35.235.240.0/20"]
-      action               = "allow"
+      targets       = ["notebook-instance"]
-      sources              = []
+      rules         = [{ protocol = "tcp", ports = [22] }]
-      ranges               = ["35.235.240.0/20"]
-      targets              = ["notebook-instance"]
-      use_service_accounts = false
-      rules                = [{ protocol = "tcp", ports = [22] }]
-      extra_attributes     = {}
     }
   }
 }

blueprints/data-solutions/gcs-to-bq-with-least-privileges/vpc.tf

@@ -27,11 +27,13 @@ module "vpc" {
 }
 
 module "vpc-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  count        = local.use_shared_vpc ? 0 : 1
+  count      = local.use_shared_vpc ? 0 : 1
-  project_id   = module.project.project_id
+  project_id = module.project.project_id
-  network      = module.vpc[0].name
+  network    = module.vpc[0].name
-  admin_ranges = [var.vpc_subnet_range]
+  default_rules_config = {
+    admin_ranges = [var.vpc_subnet_range]
+  }
 }
 
 module "nat" {

blueprints/data-solutions/sqlserver-alwayson/main.tf

@@ -13,25 +13,54 @@
 # limitations under the License.
 
 locals {
-  prefix      = var.prefix != "" ? format("%s-", var.prefix) : ""
+  ad_user_password_secret = "${local.cluster_full_name}-password"
-  vpc_project = var.shared_vpc_project_id != null ? var.shared_vpc_project_id : module.project.project_id
+  cluster_full_name       = "${local.prefix}${var.cluster_name}"
-
+  cluster_netbios_name = (
-  network    = module.vpc.self_link
+    length(local.cluster_full_name) > 15
-  subnetwork = var.project_create != null ? module.vpc.subnet_self_links[format("%s/%s", var.region, var.subnetwork)] : data.google_compute_subnetwork.subnetwork[0].self_link
+    ? substr(local.cluster_full_name, 0, 15)
-
+    : local.cluster_full_name
-  node_base            = format("%s%s", local.prefix, var.node_name)
+  )
-  node_prefix          = length(local.node_base) > 12 ? substr(local.node_base, 0, 12) : local.node_base
+  network   = module.vpc.self_link
-  node_netbios_names   = [for idx in range(1, 3) : format("%s-%02d", local.node_prefix, idx)]
+  node_base = "${local.prefix}${var.node_name}"
-  witness_name         = format("%s%s", local.prefix, var.witness_name)
+  node_prefix = (
-  witness_netbios_name = length(local.witness_name) > 15 ? substr(local.witness_name, 0, 15) : local.witness_name
+    length(local.node_base) > 12
-  zones                = var.project_create == null ? data.google_compute_zones.zones[0].names : formatlist("${var.region}-%s", ["a", "b", "c"])
+    ? substr(local.node_base, 0, 12)
-  node_zones = merge({ for idx, node_name in local.node_netbios_names : node_name => local.zones[idx] },
+    : local.node_base
-  { (local.witness_netbios_name) = local.zones[length(local.zones) - 1] })
+  )
-
+  node_netbios_names = [
-  cluster_full_name    = format("%s%s", local.prefix, var.cluster_name)
+    for idx in range(1, 3) : format("%s-%02d", local.node_prefix, idx)
-  cluster_netbios_name = length(local.cluster_full_name) > 15 ? substr(local.cluster_full_name, 0, 15) : local.cluster_full_name
+  ]
-
+  node_zones = merge(
-  ad_user_password_secret = format("%s%s-password", local.prefix, var.cluster_name)
+    {
+      for idx, node_name in local.node_netbios_names :
+      node_name => local.zones[idx]
+    },
+    {
+      (local.witness_netbios_name) = local.zones[length(local.zones) - 1]
+    }
+  )
+  prefix = var.prefix != "" ? "${var.prefix}-" : ""
+  subnetwork = (
+    var.project_create != null
+    ? module.vpc.subnet_self_links["${var.region}/${var.subnetwork}"]
+    : data.google_compute_subnetwork.subnetwork[0].self_link
+  )
+  vpc_project = (
+    var.shared_vpc_project_id != null
+    ? var.shared_vpc_project_id
+    : module.project.project_id
+  )
+  witness_name = "${local.prefix}${var.witness_name}"
+  witness_netbios_name = (
+    length(local.witness_name) > 15
+    ? substr(local.witness_name, 0, 15)
+    : local.witness_name
+  )
+  zones = (
+    var.project_create == null
+    ? data.google_compute_zones.zones[0].names
+    : formatlist("${var.region}-%s", ["a", "b", "c"])
+  )
 }
 
 module "project" {

blueprints/data-solutions/sqlserver-alwayson/vpc.tf

@@ -15,26 +15,36 @@
 # tfdoc:file:description Creates the VPC and manages the firewall rules and ILB.
 
 locals {
-  listeners = { for aog in var.always_on_groups : format("%slb-%s", local.prefix, aog) => {
+  internal_addresses = merge(
-    region     = var.region
+    local.listeners,
-    subnetwork = local.subnetwork
+    local.node_ips,
+    {
+      "${local.prefix}cluster" = {
+        region     = var.region
+        subnetwork = local.subnetwork
+      }
+      (local.witness_netbios_name) = {
+        region     = var.region
+        subnetwork = local.subnetwork
+      }
     }
+  )
+  internal_address_ips = {
+    for k, v in module.ip-addresses.internal_addresses :
+    k => v.address
   }
-  node_ips = { for node_name in local.node_netbios_names : node_name => {
+  listeners = {
-    region     = var.region
+    for aog in var.always_on_groups : "${local.prefix}lb-${aog}" => {
-    subnetwork = local.subnetwork
-    }
-  }
-  internal_addresses = merge({
-    format("%scluster", local.prefix) = {
       region     = var.region
       subnetwork = local.subnetwork
     }
-    (local.witness_netbios_name) = {
+  }
+  node_ips = {
+    for node_name in local.node_netbios_names : node_name => {
       region     = var.region
       subnetwork = local.subnetwork
     }
-  }, local.listeners, local.node_ips)
+  }
 }
 
 data "google_compute_zones" "zones" {
@@ -50,7 +60,6 @@ data "google_compute_subnetwork" "subnetwork" {
   region  = var.region
 }
 
-# Create VPC if required
 module "vpc" {
   source = "../../../modules/net-vpc"
 
@@ -66,108 +75,82 @@ module "vpc" {
   vpc_create = var.project_create != null ? true : false
 }
 
-# Firewall rules required for WSFC nodes
 module "firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = local.vpc_project
+  project_id = local.vpc_project
-  network             = local.network
+  network    = local.network
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  ingress_rules = {
-  custom_rules = {
+    "${local.prefix}allow-all-between-wsfc-nodes" = {
-    format("%sallow-all-between-wsfc-nodes", local.prefix) = {
       description          = "Allow all between WSFC nodes"
-      direction            = "INGRESS"
-      action               = "allow"
       sources              = [module.compute-service-account.email]
       targets              = [module.compute-service-account.email]
-      ranges               = []
       use_service_accounts = true
       rules = [
-        { protocol = "tcp", ports = [] },
+        { protocol = "tcp" },
-        { protocol = "udp", ports = [] },
+        { protocol = "udp" },
-        { protocol = "icmp", ports = [] }
+        { protocol = "icmp" }
       ]
-      extra_attributes = {}
     }
-    format("%sallow-all-between-wsfc-witness", local.prefix) = {
+    "${local.prefix}allow-all-between-wsfc-witness" = {
       description          = "Allow all between WSFC witness nodes"
-      direction            = "INGRESS"
-      action               = "allow"
       sources              = [module.compute-service-account.email]
       targets              = [module.witness-service-account.email]
-      ranges               = []
       use_service_accounts = true
       rules = [
-        { protocol = "tcp", ports = [] },
+        { protocol = "tcp" },
-        { protocol = "udp", ports = [] },
+        { protocol = "udp" },
-        { protocol = "icmp", ports = [] }
+        { protocol = "icmp" }
       ]
-      extra_attributes = {}
     }
-    format("%sallow-sql-to-wsfc-nodes", local.prefix) = {
+    "${local.prefix}allow-sql-to-wsfc-nodes" = {
       description          = "Allow SQL connections to WSFC nodes"
-      direction            = "INGRESS"
-      action               = "allow"
-      sources              = []
       targets              = [module.compute-service-account.email]
       ranges               = var.sql_client_cidrs
       use_service_accounts = true
       rules = [
         { protocol = "tcp", ports = [1433] },
       ]
-      extra_attributes = {}
     }
-    format("%sallow-health-check-to-wsfc-nodes", local.prefix) = {
+    "${local.prefix}allow-health-check-to-wsfc-nodes" = {
       description          = "Allow health checks to WSFC nodes"
-      direction            = "INGRESS"
-      action               = "allow"
-      sources              = []
       targets              = [module.compute-service-account.email]
       ranges               = var.health_check_ranges
       use_service_accounts = true
       rules = [
-        { protocol = "tcp", ports = [] },
+        { protocol = "tcp" }
       ]
-      extra_attributes = {}
     }
   }
 }
 
-# IP Address reservation for cluster and listener
 module "ip-addresses" {
-  source     = "../../../modules/net-address"
+  source             = "../../../modules/net-address"
-  project_id = local.vpc_project
+  project_id         = local.vpc_project
-
   internal_addresses = local.internal_addresses
 }
 
-# L4 Internal Load Balancer for SQL Listener
 module "listener-ilb" {
-  source   = "../../../modules/net-ilb"
+  source        = "../../../modules/net-ilb"
-  for_each = toset(var.always_on_groups)
+  for_each      = toset(var.always_on_groups)
-
+  project_id    = var.project_id
-  project_id = var.project_id
+  region        = var.region
-  region     = var.region
+  name          = "${var.prefix}-${each.value}-ilb"
-
+  service_label = "${var.prefix}-${each.value}-ilb"
-  name          = format("%s-%s-ilb", var.prefix, each.value)
+  address       = local.internal_address_ips["${local.prefix}lb-${each.value}"]
-  service_label = format("%s-%s-ilb", var.prefix, each.value)
+  vpc_config = {
-
+    network    = local.network
-  address    = module.ip-addresses.internal_addresses[format("%slb-%s", local.prefix, each.value)].address
+    subnetwork = local.subnetwork
-  network    = local.network
+  }
-  subnetwork = local.subnetwork
-
   backends = [for k, node in module.nodes : {
-    failover       = false
+    group = node.group.self_link
-    group          = node.group.self_link
-    balancing_mode = "CONNECTION"
   }]
-
   health_check_config = {
-    type    = "tcp",
+    enable_logging = true
-    check   = { port = var.health_check_port },
+    tcp = {
-    config  = var.health_check_config,
+      port = var.health_check_port
-    logging = true
+    }
   }
 }

blueprints/gke/multi-cluster-mesh-gke-fleet-api/vpc.tf

@@ -40,36 +40,36 @@ module "firewall" {
   source     = "../../../modules/net-vpc-firewall"
   project_id = module.host_project.project_id
   network    = module.svpc.name
-  custom_rules = merge({ allow-mesh = {
+  ingress_rules = merge(
-    description          = "Allow "
+    {
-    direction            = "INGRESS"
+      allow-mesh = {
-    action               = "allow"
+        description = "Allow mesh."
-    sources              = []
+        priority    = 900
-    ranges               = [for k, v in var.clusters_config : v.pods_cidr_block]
+        source_ranges = [
-    targets              = [for k, v in var.clusters_config : "${k}-node"]
+          for k, v in var.clusters_config : v.pods_cidr_block
-    use_service_accounts = false
+        ]
-    rules = [{ protocol = "tcp", ports = null },
+        targets = [
-      { protocol = "udp", ports = null },
+          for k, v in var.clusters_config : "${k}-node"
-      { protocol = "icmp", ports = null },
+        ]
-      { protocol = "esp", ports = null },
+        rules = [
-      { protocol = "ah", ports = null },
+          { protocol = "tcp" },
-    { protocol = "sctp", ports = null }]
+          { protocol = "udp" },
-    extra_attributes = {
+          { protocol = "icmp" },
-      priority = 900
+          { protocol = "esp" },
-    }
+          { protocol = "ah" },
-    } },
+          { protocol = "sctp" }
-    { for k, v in var.clusters_config : "allow-${k}-istio" => {
+        ]
-      description          = "Allow "
-      direction            = "INGRESS"
-      action               = "allow"
-      sources              = []
-      ranges               = [v.master_cidr_block]
-      targets              = ["${k}-node"]
-      use_service_accounts = false
-      rules                = [{ protocol = "tcp", ports = [8080, 15014, 15017] }]
-      extra_attributes = {
-        priority = 1000
       }
+    },
+    {
+      for k, v in var.clusters_config : "allow-${k}-istio" => {
+        description   = "Allow istio."
+        source_ranges = [v.master_cidr_block]
+        targets       = ["${k}-node"]
+        rules = [{
+          protocol = "tcp"
+          ports    = [8080, 15014, 15017]
+        }]
       }
     }
   )

blueprints/networking/filtering-proxy/main.tf

@@ -74,17 +74,18 @@ module "firewall" {
   source     = "../../../modules/net-vpc-firewall"
   project_id = module.project-host.project_id
   network    = module.vpc.name
-  custom_rules = {
+  ingress_rules = {
     allow-ingress-squid = {
-      description          = "Allow squid ingress traffic"
+      description = "Allow squid ingress traffic"
-      direction            = "INGRESS"
+      source_ranges = [
-      action               = "allow"
+        var.cidrs.apps, "35.191.0.0/16", "130.211.0.0/22"
-      sources              = []
+      ]
-      ranges               = [var.cidrs.apps, "35.191.0.0/16", "130.211.0.0/22"]
       targets              = [module.service-account-squid.email]
       use_service_accounts = true
-      rules                = [{ protocol = "tcp", ports = [3128] }]
+      rules = [{
-      extra_attributes     = {}
+        protocol = "tcp"
+        ports    = [3128]
+      }]
     }
   }
 }
@@ -199,20 +200,20 @@ module "squid-ilb" {
   project_id    = module.project-host.project_id
   region        = var.region
   name          = "squid-ilb"
-  service_label = "squid-ilb"
-  network       = module.vpc.self_link
-  subnetwork    = module.vpc.subnet_self_links["${var.region}/proxy"]
   ports         = [3128]
+  service_label = "squid-ilb"
+  vpc_config = {
+    network    = module.vpc.self_link
+    subnetwork = module.vpc.subnet_self_links["${var.region}/proxy"]
+  }
   backends = [{
-    failover       = false
+    group = module.squid-mig.0.group_manager.instance_group
-    group          = module.squid-mig.0.group_manager.instance_group
-    balancing_mode = "CONNECTION"
   }]
   health_check_config = {
-    type    = "tcp"
+    enable_logging = true
-    check   = { port = 3128 }
+    tcp = {
-    config  = {}
+      port = 3128
-    logging = true
+    }
   }
 }
 

blueprints/networking/hub-and-spoke-peering/main.tf

@@ -69,10 +69,12 @@ module "nat-hub" {
 }
 
 module "vpc-hub-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id   = var.project_id
+  project_id = var.project_id
-  network      = module.vpc-hub.name
+  network    = module.vpc-hub.name
-  admin_ranges = values(var.ip_ranges)
+  default_rules_config = {
+    admin_ranges = values(var.ip_ranges)
+  }
 }
 
 ################################################################################
@@ -93,10 +95,12 @@ module "vpc-spoke-1" {
 }
 
 module "vpc-spoke-1-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id   = module.project.project_id
+  project_id = module.project.project_id
-  network      = module.vpc-spoke-1.name
+  network    = module.vpc-spoke-1.name
-  admin_ranges = values(var.ip_ranges)
+  default_rules_config = {
+    admin_ranges = values(var.ip_ranges)
+  }
 }
 
 module "nat-spoke-1" {
@@ -138,10 +142,12 @@ module "vpc-spoke-2" {
 }
 
 module "vpc-spoke-2-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id   = module.project.project_id
+  project_id = module.project.project_id
-  network      = module.vpc-spoke-2.name
+  network    = module.vpc-spoke-2.name
-  admin_ranges = values(var.ip_ranges)
+  default_rules_config = {
+    admin_ranges = values(var.ip_ranges)
+  }
 }
 
 module "nat-spoke-2" {

blueprints/networking/hub-and-spoke-vpn/net-dev.tf

@@ -39,10 +39,12 @@ module "dev-vpc" {
 }
 
 module "dev-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id   = var.project_id
+  project_id = var.project_id
-  network      = module.dev-vpc.name
+  network    = module.dev-vpc.name
-  admin_ranges = values(var.ip_ranges)
+  default_rules_config = {
+    admin_ranges = values(var.ip_ranges)
+  }
 }
 
 module "dev-dns-peering" {

blueprints/networking/hub-and-spoke-vpn/net-landing.tf

@@ -39,10 +39,12 @@ module "landing-vpc" {
 }
 
 module "landing-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id   = var.project_id
+  project_id = var.project_id
-  network      = module.landing-vpc.name
+  network    = module.landing-vpc.name
-  admin_ranges = values(var.ip_ranges)
+  default_rules_config = {
+    admin_ranges = values(var.ip_ranges)
+  }
 }
 
 module "landing-dns-zone" {

blueprints/networking/hub-and-spoke-vpn/net-prod.tf

@@ -39,10 +39,12 @@ module "prod-vpc" {
 }
 
 module "prod-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id   = var.project_id
+  project_id = var.project_id
-  network      = module.prod-vpc.name
+  network    = module.prod-vpc.name
-  admin_ranges = values(var.ip_ranges)
+  default_rules_config = {
+    admin_ranges = values(var.ip_ranges)
+  }
 }
 
 module "prod-dns-peering" {

blueprints/networking/ilb-next-hop/gateways.tf

@@ -62,22 +62,22 @@ module "ilb-left" {
   project_id = module.project.project_id
   region     = var.region
   name       = "${local.prefix}ilb-left"
-  network    = module.vpc-left.self_link
+  vpc_config = {
-  subnetwork = values(module.vpc-left.subnet_self_links)[0]
+    network    = module.vpc-left.self_link
-  address    = local.addresses.ilb-left
+    subnetwork = values(module.vpc-left.subnet_self_links)[0]
-  ports      = null
+  }
-  backend_config = {
+  address = local.addresses.ilb-left
-    session_affinity                = var.ilb_session_affinity
+  backend_service_config = {
-    timeout_sec                     = null
+    session_affinity = var.ilb_session_affinity
-    connection_draining_timeout_sec = null
   }
   backends = [for z, mod in module.gw : {
-    failover       = false
+    group = mod.group.self_link
-    group          = mod.group.self_link
-    balancing_mode = "CONNECTION"
   }]
   health_check_config = {
-    type = "tcp", check = { port = 22 }, config = {}, logging = true
+    enable_logging = true
+    tcp = {
+      port = 22
+    }
   }
 }
 
@@ -86,21 +86,21 @@ module "ilb-right" {
   project_id = module.project.project_id
   region     = var.region
   name       = "${local.prefix}ilb-right"
-  network    = module.vpc-right.self_link
+  vpc_config = {
-  subnetwork = values(module.vpc-right.subnet_self_links)[0]
+    network    = module.vpc-right.self_link
-  address    = local.addresses.ilb-right
+    subnetwork = values(module.vpc-right.subnet_self_links)[0]
-  ports      = null
+  }
-  backend_config = {
+  address = local.addresses.ilb-right
-    session_affinity                = var.ilb_session_affinity
+  backend_service_config = {
-    timeout_sec                     = null
+    session_affinity = var.ilb_session_affinity
-    connection_draining_timeout_sec = null
   }
   backends = [for z, mod in module.gw : {
-    failover       = false
+    group = mod.group.self_link
-    group          = mod.group.self_link
-    balancing_mode = "CONNECTION"
   }]
   health_check_config = {
-    type = "tcp", check = { port = 22 }, config = {}, logging = true
+    enable_logging = true
+    tcp = {
+      port = 22
+    }
   }
 }

blueprints/networking/ilb-next-hop/vpc-left.tf

@@ -35,11 +35,13 @@ module "vpc-left" {
 }
 
 module "firewall-left" {
-  source            = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id        = module.project.project_id
+  project_id = module.project.project_id
-  network           = module.vpc-left.name
+  network    = module.vpc-left.name
-  admin_ranges      = values(var.ip_ranges)
+  default_rules_config = {
-  ssh_source_ranges = ["35.235.240.0/20", "35.191.0.0/16", "130.211.0.0/22"]
+    admin_ranges = values(var.ip_ranges)
+    ssh_ranges   = ["35.235.240.0/20", "35.191.0.0/16", "130.211.0.0/22"]
+  }
 }
 
 module "nat-left" {

blueprints/networking/ilb-next-hop/vpc-right.tf

@@ -46,11 +46,13 @@ module "vpc-right" {
 }
 
 module "firewall-right" {
-  source            = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id        = module.project.project_id
+  project_id = module.project.project_id
-  network           = module.vpc-right.name
+  network    = module.vpc-right.name
-  admin_ranges      = values(var.ip_ranges)
+  default_rules_config = {
-  ssh_source_ranges = ["35.235.240.0/20", "35.191.0.0/16", "130.211.0.0/22"]
+    admin_ranges = values(var.ip_ranges)
+    ssh_ranges   = ["35.235.240.0/20", "35.191.0.0/16", "130.211.0.0/22"]
+  }
 }
 
 module "nat-right" {

blueprints/networking/nginx-reverse-proxy-cluster/main.tf

@@ -161,28 +161,22 @@ module "firewall" {
   source     = "../../../modules/net-vpc-firewall"
   project_id = module.project.project_id
   network    = module.vpc.name
-  custom_rules = {
+  ingress_rules = {
     format("%sallow-http-to-proxy-cluster", var.prefix) = {
-      description          = "Allow Nginx HTTP(S) ingress traffic"
+      description = "Allow Nginx HTTP(S) ingress traffic"
-      direction            = "INGRESS"
+      source_ranges = [
-      action               = "allow"
+        var.cidrs[var.subnetwork], "35.191.0.0/16", "130.211.0.0/22"
-      sources              = []
+      ]
-      ranges               = [var.cidrs[var.subnetwork], "35.191.0.0/16", "130.211.0.0/22"]
       targets              = [module.service-account-proxy.email]
       use_service_accounts = true
       rules                = [{ protocol = "tcp", ports = [80, 443] }]
-      extra_attributes     = {}
     }
     format("%sallow-iap-ssh", var.prefix) = {
       description          = "Allow Nginx SSH traffic from IAP"
-      direction            = "INGRESS"
+      source_ranges        = ["35.235.240.0/20"]
-      action               = "allow"
-      sources              = []
-      ranges               = ["35.235.240.0/20"]
       targets              = [module.service-account-proxy.email]
       use_service_accounts = true
       rules                = [{ protocol = "tcp", ports = [22] }]
-      extra_attributes     = {}
     }
   }
 }

blueprints/networking/onprem-google-access-dns/main.tf

@@ -69,11 +69,13 @@ module "vpc" {
 }
 
 module "vpc-firewall" {
-  source            = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id        = var.project_id
+  project_id = var.project_id
-  network           = module.vpc.name
+  network    = module.vpc.name
-  admin_ranges      = values(var.ip_ranges)
+  default_rules_config = {
-  ssh_source_ranges = var.ssh_source_ranges
+    admin_ranges = values(var.ip_ranges)
+    ssh_ranges   = var.ssh_source_ranges
+  }
 }
 
 module "vpn1" {

blueprints/networking/shared-vpc-gke/main.tf

@@ -130,10 +130,12 @@ module "vpc-shared" {
 }
 
 module "vpc-shared-firewall" {
-  source       = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id   = module.project-host.project_id
+  project_id = module.project-host.project_id
-  network      = module.vpc-shared.name
+  network    = module.vpc-shared.name
-  admin_ranges = values(var.ip_ranges)
+  default_rules_config = {
+    admin_ranges = values(var.ip_ranges)
+  }
 }
 
 module "nat" {

fast/assets/schemas/firewall_rules.schema.yaml

@@ -12,18 +12,23 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-map(include('firewall_rule'))
+egress: map(include('firewall_rule'), required=False)
+ingress: map(include('firewall_rule'), required=False)
 ---
 firewall_rule:
-  description: str()
+  deny: bool(required=False)
-  direction: enum("INGRESS", "EGRESS")
+  description: str(required=False)
-  action: enum("allow", "deny")
+  destination_ranges: list(str(), required=False)
-  sources: list(str())
+  disabled: bool(required=False)
-  ranges: list(str())
+  # enable_logging:
-  targets: list(str())
+  #   include_metadata: bool(required=False)
-  use_service_accounts: bool()
+  priority: int(required=False)
-  rules: list(include('rule'))
+  source_ranges: list(str(), required=False)
+  sources: list(str(), required=False)
+  targets: list(str(), required=False)
+  use_service_accounts: bool(required=False)
+  rules: list(include('rule'), required=False)
 ---
 rule:
-  protocol: enum("tcp", "udp", "all")
+  protocol: str()
   ports: list(num())

fast/assets/templates/workflow-github.yaml

@@ -30,7 +30,7 @@ env:
   SSH_AUTH_SOCK: /tmp/ssh_agent.sock
   TF_PROVIDERS_FILE: ${tf_providers_file}
   TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n    ", tf_var_files)}
-  TF_VERSION: 1.1.7
+  TF_VERSION: 1.3.2
 
 jobs:
   fast-pr:

fast/assets/templates/workflow-sourcerepo.yaml

@@ -95,4 +95,4 @@ substitutions:
   _FAST_OUTPUTS_BUCKET: ${outputs_bucket}
   _TF_PROVIDERS_FILE: ${tf_providers_file}
   _TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n    ", tf_var_files)}
-  _TF_VERSION: 1.1.7
+  _TF_VERSION: 1.3.2

fast/extras/00-cicd-github/README.md

@@ -8,11 +8,16 @@ This stage is designed for quick repository creation in a GitHub organization, a
 
 Initial file population of repositories is controlled via the `populate_from` attribute, and needs a bit of care:
 
-- never run this stage gain with the same variables used for population once the repository starts being used,  as **Terraform will manage file state and revert any changes at each apply**, which is probably not what you want.
+- never run this stage with the same variables used for population once the repository starts being used, as **Terraform will manage file state and revert any changes at each apply**, which is probably not what you want.
-- be mindful when enabling initial population of the modules repository, as the number of resulting files to manage is very close to the GitHub hourly limit for their API
+- initial population of the modules repository is discouraged, as the number of resulting files Terraform needs to manage is very close to the GitHub hourly limit for their API, it's much easier to populate modules via regular git commands
 
 The scenario for which this stage has been designed is one-shot creation and/or population of stage repositories, running it multiple times with different variables and Terraform states if incremental creation is needed for subsequent FAST stages (e.g. GKE, data platform, etc.).
 
+Once initial population is done, you need to manually push to the repository
+
+- the `.tfvars` file with custom variable values for your stages
+- the workflow configuration file generated by FAST stages
+
 ## GitHub provider credentials
 
 A [GitHub token](https://github.com/settings/tokens) is needed to authenticate against their API. The token needs organization-level permissions, like shown in this screenshot:
@@ -77,7 +82,8 @@ When initial population is configured for a repository, this stage also adds a s
 | name | description | resources |
 |---|---|---|
 | [cicd-versions.tf](./cicd-versions.tf) | Provider version. |  |
-| [main.tf](./main.tf) | Module-level locals and resources. | <code>github_actions_secret</code> · <code>github_repository</code> · <code>github_repository_file</code> · <code>tls_private_key</code> |
+| [main.tf](./main.tf) | Module-level locals and resources. | <code>github_actions_secret</code> · <code>github_repository</code> · <code>github_repository_deploy_key</code> · <code>github_repository_file</code> · <code>tls_private_key</code> |
+| [outputs.tf](./outputs.tf) | Module outputs. |  |
 | [providers.tf](./providers.tf) | Provider configuration. |  |
 | [variables.tf](./variables.tf) | Module variables. |  |
 
@@ -85,8 +91,15 @@ When initial population is configured for a repository, this stage also adds a s
 
 | name | description | type | required | default |
 |---|---|:---:|:---:|:---:|
-| [organization](variables.tf#L28) | GitHub organization. | <code>string</code> | ✓ |  |
+| [organization](variables.tf#L34) | GitHub organization. | <code>string</code> | ✓ |  |
 | [commmit_config](variables.tf#L17) | Configure commit metadata. | <code title="object&#40;&#123;&#10;  author  &#61; optional&#40;string, &#34;FAST loader&#34;&#41;&#10;  email   &#61; optional&#40;string, &#34;fast-loader&#64;fast.gcp.tf&#34;&#41;&#10;  message &#61; optional&#40;string, &#34;FAST initial loading&#34;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [repositories](variables.tf#L33) | Repositories to create. | <code title="map&#40;object&#40;&#123;&#10;  create_options &#61; optional&#40;object&#40;&#123;&#10;    allow &#61; optional&#40;object&#40;&#123;&#10;      auto_merge   &#61; optional&#40;bool&#41;&#10;      merge_commit &#61; optional&#40;bool&#41;&#10;      rebase_merge &#61; optional&#40;bool&#41;&#10;      squash_merge &#61; optional&#40;bool&#41;&#10;    &#125;&#41;&#41;&#10;    auto_init   &#61; optional&#40;bool&#41;&#10;    description &#61; optional&#40;string&#41;&#10;    features &#61; optional&#40;object&#40;&#123;&#10;      issues   &#61; optional&#40;bool&#41;&#10;      projects &#61; optional&#40;bool&#41;&#10;      wiki     &#61; optional&#40;bool&#41;&#10;    &#125;&#41;&#41;&#10;    templates &#61; optional&#40;object&#40;&#123;&#10;      gitignore &#61; optional&#40;string, &#34;Terraform&#34;&#41;&#10;      license   &#61; optional&#40;string&#41;&#10;      repository &#61; optional&#40;object&#40;&#123;&#10;        name  &#61; string&#10;        owner &#61; string&#10;      &#125;&#41;&#41;&#10;    &#125;&#41;, &#123;&#125;&#41;&#10;    visibility &#61; optional&#40;string, &#34;private&#34;&#41;&#10;  &#125;&#41;&#41;&#10;  has_modules   &#61; optional&#40;bool, false&#41;&#10;  populate_from &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [modules_ref](variables.tf#L28) | Optional git ref used in module sources. | <code>string</code> |  | <code>null</code> |
+| [repositories](variables.tf#L39) | Repositories to create. | <code title="map&#40;object&#40;&#123;&#10;  create_options &#61; optional&#40;object&#40;&#123;&#10;    allow &#61; optional&#40;object&#40;&#123;&#10;      auto_merge   &#61; optional&#40;bool&#41;&#10;      merge_commit &#61; optional&#40;bool&#41;&#10;      rebase_merge &#61; optional&#40;bool&#41;&#10;      squash_merge &#61; optional&#40;bool&#41;&#10;    &#125;&#41;&#41;&#10;    auto_init   &#61; optional&#40;bool&#41;&#10;    description &#61; optional&#40;string&#41;&#10;    features &#61; optional&#40;object&#40;&#123;&#10;      issues   &#61; optional&#40;bool&#41;&#10;      projects &#61; optional&#40;bool&#41;&#10;      wiki     &#61; optional&#40;bool&#41;&#10;    &#125;&#41;&#41;&#10;    templates &#61; optional&#40;object&#40;&#123;&#10;      gitignore &#61; optional&#40;string, &#34;Terraform&#34;&#41;&#10;      license   &#61; optional&#40;string&#41;&#10;      repository &#61; optional&#40;object&#40;&#123;&#10;        name  &#61; string&#10;        owner &#61; string&#10;      &#125;&#41;&#41;&#10;    &#125;&#41;, &#123;&#125;&#41;&#10;    visibility &#61; optional&#40;string, &#34;private&#34;&#41;&#10;  &#125;&#41;&#41;&#10;  has_modules   &#61; optional&#40;bool, false&#41;&#10;  populate_from &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+
+## Outputs
+
+| name | description | sensitive |
+|---|---|:---:|
+| [clone](outputs.tf#L17) | Clone repository commands. |  |
 
 <!-- END TFDOC -->

fast/extras/00-cicd-github/main.tf

@@ -30,6 +30,7 @@ locals {
       }
     ] if v.populate_from != null
   ])
+  modules_ref = var.modules_ref == null ? "" : "?ref=${var.modules_ref}"
   modules_repository = (
     length(local._modules_repository) > 0
     ? local._modules_repository.0
@@ -39,13 +40,24 @@ locals {
     for k, v in var.repositories :
     k => v.create_options == null ? k : github_repository.default[k].name
   }
-  repository_files = {
+  repository_files = merge(
-    for k in local._repository_files :
+    {
-    "${k.repository}/${k.name}" => k
+      for k in local._repository_files :
-    if !endswith(k.name, ".tf") || (
+      "${k.repository}/${k.name}" => k
-      !startswith(k.name, "0") && k.name != "globals.tf"
+      if !endswith(k.name, ".tf") || (
-    )
+        !startswith(k.name, "0") && k.name != "globals.tf"
-  }
+      )
+    },
+    {
+      for k, v in var.repositories :
+      "${k}/templates/providers.tf.tpl" => {
+        repository = k
+        file       = "../../assets/templates/providers.tf.tpl"
+        name       = "templates/providers.tf.tpl"
+      }
+      if v.populate_from != null
+    }
+  )
 }
 
 resource "github_repository" "default" {
@@ -88,15 +100,20 @@ resource "tls_private_key" "default" {
   algorithm = "ED25519"
 }
 
+resource "github_repository_deploy_key" "exdefaultample_repository_deploy_key" {
+  count      = local.modules_repository == null ? 0 : 1
+  title      = "Modules repository access"
+  repository = local.modules_repository
+  key        = tls_private_key.default.0.public_key_openssh
+  read_only  = true
+}
+
 resource "github_actions_secret" "default" {
   for_each = local.modules_repository == null ? {} : {
     for k, v in local.repositories :
-    k => v if(
+    k => v if k != local.modules_repository
-      k != local.modules_repository &&
-      var.repositories[k].populate_from != null
-    )
   }
-  repository      = local.repositories[local.modules_repository]
+  repository      = each.key
   secret_name     = "CICD_MODULES_KEY"
   plaintext_value = tls_private_key.default.0.private_key_openssh
 }
@@ -112,8 +129,8 @@ resource "github_repository_file" "default" {
     endswith(each.value.name, ".tf") && local.modules_repository != null
     ? replace(
       file(each.value.file),
-      "/source\\s*=\\s*\"../../../",
+      "/source\\s*=\\s*\"../../../modules/([^/\"]+)\"/",
-      "source = \"git@github.com:${var.organization}/${local.modules_repository}.git/"
+      "source = \"git@github.com:${var.organization}/${local.modules_repository}.git//$1${local.modules_ref}\"" # "
     )
     : file(each.value.file)
   )

fast/extras/00-cicd-github/outputs.tf

@@ -14,7 +14,10 @@
  * limitations under the License.
  */
 
-output "policies" {
+output "clone" {
-  description = "Organization policies."
+  description = "Clone repository commands."
-  value       = google_org_policy_policy.primary
+  value = {
+    for k, v in var.repositories :
+    k => "git clone git@github.com:${var.organization}/${k}.git"
+  }
 }

fast/extras/00-cicd-github/variables.tf

@@ -25,6 +25,12 @@ variable "commmit_config" {
   nullable = false
 }
 
+variable "modules_ref" {
+  description = "Optional git ref used in module sources."
+  type        = string
+  default     = null
+}
+
 variable "organization" {
   description = "GitHub organization."
   type        = string

fast/stages/00-bootstrap/README.md

@@ -262,6 +262,7 @@ terraform init
 terraform apply \
   -var bootstrap_user=$(gcloud config list --format 'value(core.account)')
 ```
+> If you see an error related to project name already exists, please make sure the project name is unique or the project was not deleted recently
 
 Once the initial `apply` completes successfully, configure a remote backend using the new GCS bucket, and impersonation on the automation service account for this stage. To do this you can use the generated `providers.tf` file if you have configured output files as described above, or extract its contents from Terraform's output, then migrate state with `terraform init`:
 

fast/stages/00-bootstrap/automation.tf

@@ -111,11 +111,11 @@ module "automation-tf-bootstrap-gcs" {
 }
 
 module "automation-tf-bootstrap-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  project_id  = module.automation-project.project_id
+  project_id   = module.automation-project.project_id
-  name        = "bootstrap-0"
+  name         = "bootstrap-0"
-  description = "Terraform organization bootstrap service account."
+  display_name = "Terraform organization bootstrap service account."
-  prefix      = local.prefix
+  prefix       = local.prefix
   # allow SA used by CI/CD workflow to impersonate this SA
   iam = {
     "roles/iam.serviceAccountTokenCreator" = compact([
@@ -144,11 +144,11 @@ module "automation-tf-cicd-gcs" {
 }
 
 module "automation-tf-cicd-provisioning-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  project_id  = module.automation-project.project_id
+  project_id   = module.automation-project.project_id
-  name        = "cicd-0"
+  name         = "cicd-0"
-  description = "Terraform stage 1 CICD service account."
+  display_name = "Terraform stage 1 CICD service account."
-  prefix      = local.prefix
+  prefix       = local.prefix
   # allow SA used by CI/CD workflow to impersonate this SA
   iam = {
     "roles/iam.serviceAccountTokenCreator" = compact([
@@ -177,11 +177,11 @@ module "automation-tf-resman-gcs" {
 }
 
 module "automation-tf-resman-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  project_id  = module.automation-project.project_id
+  project_id   = module.automation-project.project_id
-  name        = "resman-0"
+  name         = "resman-0"
-  description = "Terraform stage 1 resman service account."
+  display_name = "Terraform stage 1 resman service account."
-  prefix      = local.prefix
+  prefix       = local.prefix
   # allow SA used by CI/CD workflow to impersonate this SA
   iam = {
     "roles/iam.serviceAccountTokenCreator" = compact([

fast/stages/00-bootstrap/cicd.tf

@@ -87,12 +87,12 @@ module "automation-tf-cicd-repo" {
 # SAs used by CI/CD workflows to impersonate automation SAs
 
 module "automation-tf-cicd-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  for_each    = local.cicd_repositories
+  for_each     = local.cicd_repositories
-  project_id  = module.automation-project.project_id
+  project_id   = module.automation-project.project_id
-  name        = "${each.key}-1"
+  name         = "${each.key}-1"
-  description = "Terraform CI/CD ${each.key} service account."
+  display_name = "Terraform CI/CD ${each.key} service account."
-  prefix      = local.prefix
+  prefix       = local.prefix
   iam = (
     each.value.type == "sourcerepo"
     # used directly from the cloud build trigger for source repos

fast/stages/00-bootstrap/outputs-files.tf

@@ -19,27 +19,27 @@
 resource "local_file" "providers" {
   for_each        = var.outputs_location == null ? {} : local.providers
   file_permission = "0644"
-  filename        = "${pathexpand(var.outputs_location)}/providers/${each.key}-providers.tf"
+  filename        = "${try(pathexpand(var.outputs_location), "")}/providers/${each.key}-providers.tf"
-  content         = each.value
+  content         = try(each.value, null)
 }
 
 resource "local_file" "tfvars" {
   for_each        = var.outputs_location == null ? {} : { 1 = 1 }
   file_permission = "0644"
-  filename        = "${pathexpand(var.outputs_location)}/tfvars/00-bootstrap.auto.tfvars.json"
+  filename        = "${try(pathexpand(var.outputs_location), "")}/tfvars/00-bootstrap.auto.tfvars.json"
   content         = jsonencode(local.tfvars)
 }
 
 resource "local_file" "tfvars_globals" {
   for_each        = var.outputs_location == null ? {} : { 1 = 1 }
   file_permission = "0644"
-  filename        = "${pathexpand(var.outputs_location)}/tfvars/globals.auto.tfvars.json"
+  filename        = "${try(pathexpand(var.outputs_location), "")}/tfvars/globals.auto.tfvars.json"
   content         = jsonencode(local.tfvars_globals)
 }
 
 resource "local_file" "workflows" {
-  for_each        = local.cicd_workflows
+  for_each        = var.outputs_location == null ? {} : local.cicd_workflows
   file_permission = "0644"
-  filename        = "${pathexpand(var.outputs_location)}/workflows/${each.key}-workflow.yaml"
+  filename        = "${try(pathexpand(var.outputs_location), "")}/workflows/${each.key}-workflow.yaml"
-  content         = each.value
+  content         = try(each.value, null)
 }

fast/stages/01-resman/README.md

@@ -182,17 +182,18 @@ Due to its simplicity, this stage lends itself easily to customizations: adding
 |---|---|:---:|:---:|:---:|:---:|
 | [automation](variables.tf#L20) | Automation resources created by the bootstrap stage. | <code title="object&#40;&#123;&#10;  outputs_bucket          &#61; string&#10;  project_id              &#61; string&#10;  project_number          &#61; string&#10;  federated_identity_pool &#61; string&#10;  federated_identity_providers &#61; map&#40;object&#40;&#123;&#10;    issuer           &#61; string&#10;    issuer_uri       &#61; string&#10;    name             &#61; string&#10;    principal_tpl    &#61; string&#10;    principalset_tpl &#61; string&#10;  &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  | <code>00-bootstrap</code> |
 | [billing_account](variables.tf#L38) | Billing account id and organization id ('nnnnnnnn' or null). | <code title="object&#40;&#123;&#10;  id              &#61; string&#10;  organization_id &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  | <code>00-bootstrap</code> |
-| [organization](variables.tf#L191) | Organization details. | <code title="object&#40;&#123;&#10;  domain      &#61; string&#10;  id          &#61; number&#10;  customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  | <code>00-bootstrap</code> |
+| [organization](variables.tf#L197) | Organization details. | <code title="object&#40;&#123;&#10;  domain      &#61; string&#10;  id          &#61; number&#10;  customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  | <code>00-bootstrap</code> |
-| [prefix](variables.tf#L215) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ |  | <code>00-bootstrap</code> |
+| [prefix](variables.tf#L221) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ |  | <code>00-bootstrap</code> |
 | [cicd_repositories](variables.tf#L47) | CI/CD repository configuration. Identity providers reference keys in the `automation.federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | <code title="object&#40;&#123;&#10;  data_platform_dev &#61; object&#40;&#123;&#10;    branch            &#61; string&#10;    identity_provider &#61; string&#10;    name              &#61; string&#10;    type              &#61; string&#10;  &#125;&#41;&#10;  data_platform_prod &#61; object&#40;&#123;&#10;    branch            &#61; string&#10;    identity_provider &#61; string&#10;    name              &#61; string&#10;    type              &#61; string&#10;  &#125;&#41;&#10;  gke_dev &#61; object&#40;&#123;&#10;    branch            &#61; string&#10;    identity_provider &#61; string&#10;    name              &#61; string&#10;    type              &#61; string&#10;  &#125;&#41;&#10;  gke_prod &#61; object&#40;&#123;&#10;    branch            &#61; string&#10;    identity_provider &#61; string&#10;    name              &#61; string&#10;    type              &#61; string&#10;  &#125;&#41;&#10;  networking &#61; object&#40;&#123;&#10;    branch            &#61; string&#10;    identity_provider &#61; string&#10;    name              &#61; string&#10;    type              &#61; string&#10;  &#125;&#41;&#10;  project_factory_dev &#61; object&#40;&#123;&#10;    branch            &#61; string&#10;    identity_provider &#61; string&#10;    name              &#61; string&#10;    type              &#61; string&#10;  &#125;&#41;&#10;  project_factory_prod &#61; object&#40;&#123;&#10;    branch            &#61; string&#10;    identity_provider &#61; string&#10;    name              &#61; string&#10;    type              &#61; string&#10;  &#125;&#41;&#10;  security &#61; object&#40;&#123;&#10;    branch            &#61; string&#10;    identity_provider &#61; string&#10;    name              &#61; string&#10;    type              &#61; string&#10;  &#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |  |
 | [custom_roles](variables.tf#L129) | Custom roles defined at the org level, in key => id format. | <code title="object&#40;&#123;&#10;  service_project_network_admin &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> | <code>00-bootstrap</code> |
-| [fast_features](variables.tf#L138) | Selective control for top-level FAST features. | <code title="object&#40;&#123;&#10;  data_platform   &#61; bool&#10;  gke             &#61; bool&#10;  project_factory &#61; bool&#10;  sandbox         &#61; bool&#10;  teams           &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  data_platform   &#61; true&#10;  gke             &#61; true&#10;  project_factory &#61; true&#10;  sandbox         &#61; true&#10;  teams           &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
+| [data_dir](variables.tf#L138) | Relative path for the folder storing configuration data. | <code>string</code> |  | <code>&#34;data&#34;</code> |  |
-| [groups](variables.tf#L158) | Group names to grant organization-level permissions. | <code>map&#40;string&#41;</code> |  | <code title="&#123;&#10;  gcp-billing-admins      &#61; &#34;gcp-billing-admins&#34;,&#10;  gcp-devops              &#61; &#34;gcp-devops&#34;,&#10;  gcp-network-admins      &#61; &#34;gcp-network-admins&#34;&#10;  gcp-organization-admins &#61; &#34;gcp-organization-admins&#34;&#10;  gcp-security-admins     &#61; &#34;gcp-security-admins&#34;&#10;  gcp-support             &#61; &#34;gcp-support&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
+| [fast_features](variables.tf#L144) | Selective control for top-level FAST features. | <code title="object&#40;&#123;&#10;  data_platform   &#61; bool&#10;  gke             &#61; bool&#10;  project_factory &#61; bool&#10;  sandbox         &#61; bool&#10;  teams           &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  data_platform   &#61; true&#10;  gke             &#61; true&#10;  project_factory &#61; true&#10;  sandbox         &#61; true&#10;  teams           &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
-| [locations](variables.tf#L173) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object&#40;&#123;&#10;  bq      &#61; string&#10;  gcs     &#61; string&#10;  logging &#61; string&#10;  pubsub  &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  bq      &#61; &#34;EU&#34;&#10;  gcs     &#61; &#34;EU&#34;&#10;  logging &#61; &#34;global&#34;&#10;  pubsub  &#61; &#91;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
+| [groups](variables.tf#L164) | Group names to grant organization-level permissions. | <code>map&#40;string&#41;</code> |  | <code title="&#123;&#10;  gcp-billing-admins      &#61; &#34;gcp-billing-admins&#34;,&#10;  gcp-devops              &#61; &#34;gcp-devops&#34;,&#10;  gcp-network-admins      &#61; &#34;gcp-network-admins&#34;&#10;  gcp-organization-admins &#61; &#34;gcp-organization-admins&#34;&#10;  gcp-security-admins     &#61; &#34;gcp-security-admins&#34;&#10;  gcp-support             &#61; &#34;gcp-support&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
-| [organization_policy_configs](variables.tf#L201) | Organization policies customization. | <code title="object&#40;&#123;&#10;  allowed_policy_member_domains &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |  |
+| [locations](variables.tf#L179) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object&#40;&#123;&#10;  bq      &#61; string&#10;  gcs     &#61; string&#10;  logging &#61; string&#10;  pubsub  &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  bq      &#61; &#34;EU&#34;&#10;  gcs     &#61; &#34;EU&#34;&#10;  logging &#61; &#34;global&#34;&#10;  pubsub  &#61; &#91;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
-| [outputs_location](variables.tf#L209) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable | <code>string</code> |  | <code>null</code> |  |
+| [organization_policy_configs](variables.tf#L207) | Organization policies customization. | <code title="object&#40;&#123;&#10;  allowed_policy_member_domains &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |  |
-| [tag_names](variables.tf#L226) | Customized names for resource management tags. | <code title="object&#40;&#123;&#10;  context     &#61; string&#10;  environment &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  context     &#61; &#34;context&#34;&#10;  environment &#61; &#34;environment&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |  |
+| [outputs_location](variables.tf#L215) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable | <code>string</code> |  | <code>null</code> |  |
-| [team_folders](variables.tf#L243) | Team folders to be created. Format is described in a code comment. | <code title="map&#40;object&#40;&#123;&#10;  descriptive_name     &#61; string&#10;  group_iam            &#61; map&#40;list&#40;string&#41;&#41;&#10;  impersonation_groups &#61; list&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>null</code> |  |
+| [tag_names](variables.tf#L232) | Customized names for resource management tags. | <code title="object&#40;&#123;&#10;  context     &#61; string&#10;  environment &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  context     &#61; &#34;context&#34;&#10;  environment &#61; &#34;environment&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |  |
+| [team_folders](variables.tf#L249) | Team folders to be created. Format is described in a code comment. | <code title="map&#40;object&#40;&#123;&#10;  descriptive_name     &#61; string&#10;  group_iam            &#61; map&#40;list&#40;string&#41;&#41;&#10;  impersonation_groups &#61; list&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>null</code> |  |
 
 ## Outputs
 

fast/stages/01-resman/branch-data-platform.tf

@@ -77,12 +77,12 @@ module "branch-dp-prod-folder" {
 # automation service accounts and buckets
 
 module "branch-dp-dev-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  count       = var.fast_features.data_platform ? 1 : 0
+  count        = var.fast_features.data_platform ? 1 : 0
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "dev-resman-dp-0"
+  name         = "dev-resman-dp-0"
-  description = "Terraform data platform development service account."
+  display_name = "Terraform data platform development service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = {
     "roles/iam.serviceAccountTokenCreator" = compact([
       try(module.branch-dp-dev-sa-cicd.0.iam_email, null)
@@ -94,12 +94,12 @@ module "branch-dp-dev-sa" {
 }
 
 module "branch-dp-prod-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  count       = var.fast_features.data_platform ? 1 : 0
+  count        = var.fast_features.data_platform ? 1 : 0
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-resman-dp-0"
+  name         = "prod-resman-dp-0"
-  description = "Terraform data platform production service account."
+  display_name = "Terraform data platform production service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = {
     "roles/iam.serviceAccountTokenCreator" = compact([
       try(module.branch-dp-prod-sa-cicd.0.iam_email, null)

fast/stages/01-resman/branch-gke.tf

@@ -69,12 +69,12 @@ module "branch-gke-prod-folder" {
 }
 
 module "branch-gke-dev-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  count       = var.fast_features.gke ? 1 : 0
+  count        = var.fast_features.gke ? 1 : 0
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "dev-resman-gke-0"
+  name         = "dev-resman-gke-0"
-  description = "Terraform gke multitenant dev service account."
+  display_name = "Terraform gke multitenant dev service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = {
     "roles/iam.serviceAccountTokenCreator" = concat(
       ["group:${local.groups.gcp-devops}"],
@@ -89,12 +89,12 @@ module "branch-gke-dev-sa" {
 }
 
 module "branch-gke-prod-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  count       = var.fast_features.gke ? 1 : 0
+  count        = var.fast_features.gke ? 1 : 0
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-resman-gke-0"
+  name         = "prod-resman-gke-0"
-  description = "Terraform gke multitenant prod service account."
+  display_name = "Terraform gke multitenant prod service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = {
     "roles/iam.serviceAccountTokenCreator" = concat(
       ["group:${local.groups.gcp-devops}"],

fast/stages/01-resman/branch-networking.tf

@@ -86,11 +86,11 @@ module "branch-network-dev-folder" {
 # automation service account and bucket
 
 module "branch-network-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-resman-net-0"
+  name         = "prod-resman-net-0"
-  description = "Terraform resman networking service account."
+  display_name = "Terraform resman networking service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = {
     "roles/iam.serviceAccountTokenCreator" = compact([
       try(module.branch-network-sa-cicd.0.iam_email, null)

fast/stages/01-resman/branch-project-factory.tf

@@ -22,8 +22,8 @@ module "branch-pf-dev-sa" {
   project_id = var.automation.project_id
   name       = "dev-resman-pf-0"
   # naming: environment in description
-  description = "Terraform project factory development service account."
+  display_name = "Terraform project factory development service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = {
     "roles/iam.serviceAccountTokenCreator" = compact([
       try(module.branch-pf-dev-sa-cicd.0.iam_email, null)
@@ -40,8 +40,8 @@ module "branch-pf-prod-sa" {
   project_id = var.automation.project_id
   name       = "prod-resman-pf-0"
   # naming: environment in description
-  description = "Terraform project factory production service account."
+  display_name = "Terraform project factory production service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = {
     "roles/iam.serviceAccountTokenCreator" = compact([
       try(module.branch-pf-prod-sa-cicd.0.iam_email, null)

fast/stages/01-resman/branch-sandbox.tf

@@ -68,10 +68,10 @@ moved {
 }
 
 module "branch-sandbox-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  count       = var.fast_features.sandbox ? 1 : 0
+  count        = var.fast_features.sandbox ? 1 : 0
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "dev-resman-sbox-0"
+  name         = "dev-resman-sbox-0"
-  description = "Terraform resman sandbox service account."
+  display_name = "Terraform resman sandbox service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
 }

fast/stages/01-resman/branch-security.tf

@@ -49,11 +49,11 @@ module "branch-security-folder" {
 # automation service account and bucket
 
 module "branch-security-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-resman-sec-0"
+  name         = "prod-resman-sec-0"
-  description = "Terraform resman security service account."
+  display_name = "Terraform resman security service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = {
     "roles/iam.serviceAccountTokenCreator" = compact([
       try(module.branch-security-sa-cicd.0.iam_email, null)

fast/stages/01-resman/branch-teams.tf

@@ -40,12 +40,12 @@ module "branch-teams-folder" {
 }
 
 module "branch-teams-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  count       = var.fast_features.teams ? 1 : 0
+  count        = var.fast_features.teams ? 1 : 0
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-resman-teams-0"
+  name         = "prod-resman-teams-0"
-  description = "Terraform resman teams service account."
+  display_name = "Terraform resman teams service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam_storage_roles = {
     (var.automation.outputs_bucket) = ["roles/storage.admin"]
   }
@@ -83,12 +83,12 @@ module "branch-teams-team-folder" {
 }
 
 module "branch-teams-team-sa" {
-  source      = "../../../modules/iam-service-account"
+  source       = "../../../modules/iam-service-account"
-  for_each    = var.fast_features.teams ? coalesce(var.team_folders, {}) : {}
+  for_each     = var.fast_features.teams ? coalesce(var.team_folders, {}) : {}
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-teams-${each.key}-0"
+  name         = "prod-teams-${each.key}-0"
-  description = "Terraform team ${each.key} service account."
+  display_name = "Terraform team ${each.key} service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = {
     "roles/iam.serviceAccountTokenCreator" = (
       each.value.impersonation_groups == null

fast/stages/01-resman/cicd-data-platform.tf

@@ -95,10 +95,10 @@ module "branch-dp-dev-sa-cicd" {
     ? { 0 = local.cicd_repositories.data_platform_dev }
     : {}
   )
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "dev-resman-dp-1"
+  name         = "dev-resman-dp-1"
-  description = "Terraform CI/CD data platform development service account."
+  display_name = "Terraform CI/CD data platform development service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = (
     each.value.type == "sourcerepo"
     # used directly from the cloud build trigger for source repos
@@ -138,10 +138,10 @@ module "branch-dp-prod-sa-cicd" {
     ? { 0 = local.cicd_repositories.data_platform_prod }
     : {}
   )
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-resman-dp-1"
+  name         = "prod-resman-dp-1"
-  description = "Terraform CI/CD data platform production service account."
+  display_name = "Terraform CI/CD data platform production service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = (
     each.value.type == "sourcerepo"
     # used directly from the cloud build trigger for source repos

fast/stages/01-resman/cicd-gke.tf

@@ -95,10 +95,10 @@ module "branch-gke-dev-sa-cicd" {
     ? { 0 = local.cicd_repositories.gke_dev }
     : {}
   )
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "dev-resman-gke-1"
+  name         = "dev-resman-gke-1"
-  description = "Terraform CI/CD GKE development service account."
+  display_name = "Terraform CI/CD GKE development service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = (
     each.value.type == "sourcerepo"
     # used directly from the cloud build trigger for source repos
@@ -138,10 +138,10 @@ module "branch-gke-prod-sa-cicd" {
     ? { 0 = local.cicd_repositories.gke_prod }
     : {}
   )
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-resman-gke-1"
+  name         = "prod-resman-gke-1"
-  description = "Terraform CI/CD GKE production service account."
+  display_name = "Terraform CI/CD GKE production service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = (
     each.value.type == "sourcerepo"
     # used directly from the cloud build trigger for source repos

fast/stages/01-resman/cicd-networking.tf

@@ -57,10 +57,10 @@ module "branch-network-sa-cicd" {
     ? { 0 = local.cicd_repositories.networking }
     : {}
   )
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-resman-net-1"
+  name         = "prod-resman-net-1"
-  description = "Terraform CI/CD stage 2 networking service account."
+  display_name = "Terraform CI/CD stage 2 networking service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = (
     each.value.type == "sourcerepo"
     # used directly from the cloud build trigger for source repos

fast/stages/01-resman/cicd-project-factory.tf

@@ -106,10 +106,10 @@ module "branch-pf-dev-sa-cicd" {
     ? { 0 = local.cicd_repositories.project_factory_dev }
     : {}
   )
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "dev-pf-resman-pf-1"
+  name         = "dev-pf-resman-pf-1"
-  description = "Terraform CI/CD project factory development service account."
+  display_name = "Terraform CI/CD project factory development service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = (
     each.value.type == "sourcerepo"
     # used directly from the cloud build trigger for source repos
@@ -154,10 +154,10 @@ module "branch-pf-prod-sa-cicd" {
     ? { 0 = local.cicd_repositories.project_factory_prod }
     : {}
   )
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-pf-resman-pf-1"
+  name         = "prod-pf-resman-pf-1"
-  description = "Terraform CI/CD project factory production service account."
+  display_name = "Terraform CI/CD project factory production service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = (
     each.value.type == "sourcerepo"
     # used directly from the cloud build trigger for source repos

fast/stages/01-resman/cicd-security.tf

@@ -57,10 +57,10 @@ module "branch-security-sa-cicd" {
     ? { 0 = local.cicd_repositories.security }
     : {}
   )
-  project_id  = var.automation.project_id
+  project_id   = var.automation.project_id
-  name        = "prod-resman-sec-1"
+  name         = "prod-resman-sec-1"
-  description = "Terraform CI/CD stage 2 security service account."
+  display_name = "Terraform CI/CD stage 2 security service account."
-  prefix      = var.prefix
+  prefix       = var.prefix
   iam = (
     each.value.type == "sourcerepo"
     # used directly from the cloud build trigger for source repos

fast/stages/01-resman/data/org-policies/compute.yaml

@@ -0,0 +1,73 @@
+# skip boilerplate check
+#
+# sample subset of useful organization policies, edit to suit requirements
+
+compute.disableGuestAttributesAccess:
+  enforce: true
+
+compute.requireOsLogin:
+  enforce: true
+
+compute.restrictLoadBalancerCreationForTypes:
+  allow:
+    values:
+    - in:INTERNAL
+
+compute.skipDefaultNetworkCreation:
+  enforce: true
+
+compute.vmExternalIpAccess:
+  deny:
+    all: true
+
+
+# compute.disableInternetNetworkEndpointGroup:
+#   enforce: true
+
+# compute.disableNestedVirtualization:
+#   enforce: true
+
+# compute.disableSerialPortAccess:
+#   enforce: true
+
+# compute.restrictCloudNATUsage:
+#   deny:
+#     all: true
+
+# compute.restrictDedicatedInterconnectUsage:
+#   deny:
+#     all: true
+
+# compute.restrictPartnerInterconnectUsage:
+#   deny:
+#     all: true
+
+# compute.restrictProtocolForwardingCreationForTypes:
+#   deny:
+#     all: true
+
+# compute.restrictSharedVpcHostProjects:
+#   deny:
+#     all: true
+
+# compute.restrictSharedVpcSubnetworks:
+#   deny:
+#     all: true
+
+# compute.restrictVpcPeering:
+#   deny:
+#     all: true
+
+# compute.restrictVpnPeerIPs:
+#   deny:
+#     all: true
+
+# compute.restrictXpnProjectLienRemoval:
+#   enforce: true
+
+# compute.setNewProjectDefaultToZonalDNSOnly:
+#   enforce: true
+
+# compute.vmCanIpForward:
+#   deny:
+#     all: true

fast/stages/01-resman/data/org-policies/iam.yaml

@@ -0,0 +1,12 @@
+# skip boilerplate check
+#
+# sample subset of useful organization policies, edit to suit requirements
+
+iam.automaticIamGrantsForDefaultServiceAccounts:
+  enforce: true
+
+iam.disableServiceAccountKeyCreation:
+  enforce: true
+
+iam.disableServiceAccountKeyUpload:
+  enforce: true

fast/stages/01-resman/data/org-policies/serverless.yaml

@@ -0,0 +1,26 @@
+# skip boilerplate check
+#
+# sample subset of useful organization policies, edit to suit requirements
+
+run.allowedIngress:
+  allow:
+    values:
+    - is:internal
+
+# run.allowedVPCEgress:
+#   allow:
+#     values:
+#       - is:private-ranges-only
+
+# cloudfunctions.allowedIngressSettings:
+#   allow:
+#     values:
+#     - is:ALLOW_INTERNAL_ONLY
+
+# cloudfunctions.allowedVpcConnectorEgressSettings:
+#   allow:
+#     values:
+#     - is:PRIVATE_RANGES_ONLY
+
+# cloudfunctions.requireVPCConnector:
+#   enforce: true

fast/stages/01-resman/data/org-policies/sql.yaml

@@ -0,0 +1,9 @@
+# skip boilerplate check
+#
+# sample subset of useful organization policies, edit to suit requirements
+
+sql.restrictAuthorizedNetworks:
+  enforce: true
+
+sql.restrictPublicIp:
+  enforce: true

fast/stages/01-resman/data/org-policies/storage.yaml

@@ -0,0 +1,6 @@
+# skip boilerplate check
+#
+# sample subset of useful organization policies, edit to suit requirements
+
+storage.uniformBucketLevelAccess:
+  enforce: true

fast/stages/01-resman/organization.tf

@@ -66,44 +66,12 @@ module "organization" {
       )
     } : {}
   )
-  # sample subset of useful organization policies, edit to suit requirements
 
+  # sample subset of useful organization policies, edit to suit requirements
   org_policies = {
-    "compute.disableGuestAttributesAccess"            = { enforce = true }
+    "iam.allowedPolicyMemberDomains" = { allow = { values = local.all_drs_domains } }
-    "compute.requireOsLogin"                          = { enforce = true }
+
-    "compute.restrictLoadBalancerCreationForTypes"    = { allow = { values = ["in:INTERNAL"] } }
+    #"gcp.resourceLocations" = {
-    "compute.skipDefaultNetworkCreation"              = { enforce = true }
-    "compute.vmExternalIpAccess"                      = { deny = { all = true } }
-    "iam.allowedPolicyMemberDomains"                  = { allow = { values = local.all_drs_domains } }
-    "iam.automaticIamGrantsForDefaultServiceAccounts" = { enforce = true }
-    "iam.disableServiceAccountKeyCreation"            = { enforce = true }
-    "iam.disableServiceAccountKeyUpload"              = { enforce = true }
-    "run.allowedIngress"                              = { allow = { values = ["is:internal"] } }
-    "sql.restrictAuthorizedNetworks"                  = { enforce = true }
-    "sql.restrictPublicIp"                            = { enforce = true }
-    "storage.uniformBucketLevelAccess"                = { enforce = true }
-    # "cloudfunctions.allowedIngressSettings" = {
-    #   allow = { values = ["is:ALLOW_INTERNAL_ONLY"] }
-    # }
-    # "cloudfunctions.allowedVpcConnectorEgressSettings" = {
-    #   allow = { values = ["is:PRIVATE_RANGES_ONLY"] }
-    # }
-    # "cloudfunctions.requireVPCConnector"              = { enforce = true }
-    # "compute.disableInternetNetworkEndpointGroup"     = { enforce = true }
-    # "compute.disableNestedVirtualization"             = { enforce = true }
-    # "compute.disableSerialPortAccess"                 = { enforce = true }
-    # "compute.restrictCloudNATUsage"                      = { deny = { all = true }}
-    # "compute.restrictDedicatedInterconnectUsage"         = { deny = { all = true }}
-    # "compute.restrictPartnerInterconnectUsage"           = { deny = { all = true }}
-    # "compute.restrictProtocolForwardingCreationForTypes" = { deny = { all = true }}
-    # "compute.restrictSharedVpcHostProjects"              = { deny = { all = true }}
-    # "compute.restrictSharedVpcSubnetworks"               = { deny = { all = true }}
-    # "compute.restrictVpcPeering" = { deny = { all = true }}
-    # "compute.restrictVpnPeerIPs" = { deny = { all = true }}
-    # "compute.restrictXpnProjectLienRemoval"           = { enforce = true }
-    # "compute.setNewProjectDefaultToZonalDNSOnly"      = { enforce = true }
-    # "compute.vmCanIpForward"     = { deny = { all = true }}
-    # "gcp.resourceLocations" = {
     #   allow = { values = local.allowed_regions }
     # }
     # "iam.workloadIdentityPoolProviders" = {
@@ -114,8 +82,9 @@ module "organization" {
     #     ]
     #   }
     # }
-    # "run.allowedVPCEgress" = { allow = { values = ["is:private-ranges-only"] }  }
   }
+  org_policies_data_path = "${var.data_dir}/org-policies"
+
   tags = {
     (var.tag_names.context) = {
       description = "Resource management context."

fast/stages/01-resman/variables.tf

@@ -135,6 +135,12 @@ variable "custom_roles" {
   default = null
 }
 
+variable "data_dir" {
+  description = "Relative path for the folder storing configuration data."
+  type        = string
+  default     = "data"
+}
+
 variable "fast_features" {
   # tfdoc:variable:source 00-bootstrap
   description = "Selective control for top-level FAST features."

fast/stages/02-networking-nva/data/firewall-rules/dev/rules.yaml

@@ -1,27 +1,21 @@
 # skip boilerplate check
 
-ingress-allow-composer-nodes:
+ingress:
-  description: "Allow traffic to Composer nodes."
+  ingress-allow-composer-nodes:
-  direction: INGRESS
+    description: "Allow traffic to Composer nodes."
-  action: allow
+    sources:
-  sources:
+      - composer-worker
-    - composer-worker
+    targets:
-  targets:
+      - composer-worker
-    - composer-worker
+    rules:
-  use_service_accounts: false
+      - protocol: tcp
-  rules:
+        ports: [80, 443, 3306, 3307]
-    - protocol: tcp
+  ingress-allow-dataflow-load:
-      ports: [80, 443, 3306, 3307]
+    description: "Allow traffic to Dataflow nodes."
-
+    sources:
-ingress-allow-dataflow-load:
+      - dataflow
-  description: "Allow traffic to Dataflow nodes."
+    targets:
-  direction: INGRESS
+      - dataflow
-  action: allow
+    rules:
-  sources:
+      - protocol: tcp
-    - dataflow
+        ports: [12345, 12346]
-  targets:
-    - dataflow
-  use_service_accounts: false
-  rules:
-    - protocol: tcp
-      ports: [12345, 12346]

fast/stages/02-networking-nva/data/firewall-rules/landing-trusted/rules.yaml

@@ -1,29 +1,19 @@
 # skip boilerplate check
 
-allow-hc-nva-ssh-trusted:
+ingress:
-  description: "Allow traffic from Google healthchecks to NVA appliances"
+  allow-hc-nva-ssh-trusted:
-  direction: INGRESS
+    description: "Allow traffic from Google healthchecks to NVA appliances"
-  action: allow
+    source_ranges:
-  sources: []
+      - healthchecks
-  ranges:
+    rules:
-    - $healthchecks
+      - protocol: tcp
-  targets: []
+        ports:
-  use_service_accounts: false
+          - 22
-  rules:
+  allow-onprem-probes-trusted-example:
-    - protocol: tcp
+    description: "Allow traffic from onprem probes"
-      ports:
+    source_ranges:
-        - 22
+      - onprem_probes
-
+    rules:
-allow-onprem-probes-trusted-example:
+      - protocol: tcp
-  description: "Allow traffic from onprem probes"
+        ports:
-  direction: INGRESS
+          - 12345
-  action: allow
-  sources: []
-  ranges:
-    - $onprem_probes
-  targets: []
-  use_service_accounts: false
-  rules:
-    - protocol: tcp
-      ports:
-        - 12345

fast/stages/02-networking-nva/data/firewall-rules/landing-untrusted/rules.yaml

@@ -1,15 +1,11 @@
 # skip boilerplate check
 
-allow-hc-nva-ssh-untrusted:
+ingress:
-  description: "Allow traffic from Google healthchecks to NVA appliances"
+  allow-hc-nva-ssh-untrusted:
-  direction: INGRESS
+    description: "Allow traffic from Google healthchecks to NVA appliances"
-  action: allow
+    source_ranges:
-  sources: []
+      - healthchecks
-  ranges:
+    rules:
-    - $healthchecks
+      - protocol: tcp
-  targets: []
+        ports:
-  use_service_accounts: false
+          - 22
-  rules:
-    - protocol: tcp
-      ports:
-        - 22

fast/stages/02-networking-nva/landing.tf

@@ -57,15 +57,16 @@ module "landing-untrusted-vpc" {
 }
 
 module "landing-untrusted-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.landing-project.project_id
+  project_id = module.landing-project.project_id
-  network             = module.landing-untrusted-vpc.name
+  network    = module.landing-untrusted-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/landing-untrusted"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/landing-untrusted"
+  }
 }
 
 # NAT
@@ -123,13 +124,14 @@ module "landing-trusted-vpc" {
 }
 
 module "landing-trusted-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.landing-project.project_id
+  project_id = module.landing-project.project_id
-  network             = module.landing-trusted-vpc.name
+  network    = module.landing-trusted-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/landing-trusted"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/landing-trusted"
+  }
 }

fast/stages/02-networking-nva/nva.tf

@@ -120,16 +120,20 @@ module "ilb-nva-untrusted" {
   name          = "nva-untrusted-${each.value.0}"
   service_label = var.prefix
   global_access = true
-  network       = module.landing-untrusted-vpc.self_link
+  vpc_config = {
-  subnetwork    = module.landing-untrusted-vpc.subnet_self_links["${each.key}/landing-untrusted-default-${each.value.0}"]
+    network    = module.landing-untrusted-vpc.self_link
-  backends = [for key, _ in local.nva_locality :
+    subnetwork = module.landing-untrusted-vpc.subnet_self_links["${each.key}/landing-untrusted-default-${each.value.0}"]
-    {
+  }
-      failover       = false
+  backends = [
-      group          = module.nva-mig[key].group_manager.instance_group
+    for key, _ in local.nva_locality : {
-      balancing_mode = "CONNECTION"
+      group = module.nva-mig[key].group_manager.instance_group
-  } if local.nva_locality[key].region == each.key]
+    } if local.nva_locality[key].region == each.key
+  ]
   health_check_config = {
-    type = "tcp", check = { port = 22 }, config = {}, logging = false
+    enable_logging = true
+    tcp = {
+      port = 22
+    }
   }
 }
 
@@ -142,16 +146,20 @@ module "ilb-nva-trusted" {
   name          = "nva-trusted-${each.value.0}"
   service_label = var.prefix
   global_access = true
-  network       = module.landing-trusted-vpc.self_link
+  vpc_config = {
-  subnetwork    = module.landing-trusted-vpc.subnet_self_links["${each.key}/landing-trusted-default-${each.value.0}"]
+    network    = module.landing-trusted-vpc.self_link
-  backends = [for key, _ in local.nva_locality :
+    subnetwork = module.landing-trusted-vpc.subnet_self_links["${each.key}/landing-trusted-default-${each.value.0}"]
-    {
+  }
-      failover       = false
+  backends = [
-      group          = module.nva-mig[key].group_manager.instance_group
+    for key, _ in local.nva_locality : {
-      balancing_mode = "CONNECTION"
+      group = module.nva-mig[key].group_manager.instance_group
-  } if local.nva_locality[key].region == each.key]
+    } if local.nva_locality[key].region == each.key
+  ]
   health_check_config = {
-    type = "tcp", check = { port = 22 }, config = {}, logging = false
+    enable_logging = true
+    tcp = {
+      port = 22
+    }
   }
 }
 

fast/stages/02-networking-nva/spoke-dev.tf

@@ -97,15 +97,16 @@ module "dev-spoke-vpc" {
 }
 
 module "dev-spoke-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.dev-spoke-project.project_id
+  project_id = module.dev-spoke-project.project_id
-  network             = module.dev-spoke-vpc.name
+  network    = module.dev-spoke-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/dev"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/dev"
+  }
 }
 
 module "peering-dev" {

fast/stages/02-networking-nva/spoke-prod.tf

@@ -97,15 +97,16 @@ module "prod-spoke-vpc" {
 }
 
 module "prod-spoke-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.prod-spoke-project.project_id
+  project_id = module.prod-spoke-project.project_id
-  network             = module.prod-spoke-vpc.name
+  network    = module.prod-spoke-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/prod"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/prod"
+  }
 }
 
 module "peering-prod" {

fast/stages/02-networking-peering/data/firewall-rules/dev/rules.yaml

@@ -1,27 +1,21 @@
 # skip boilerplate check
 
-ingress-allow-composer-nodes:
+ingress:
-  description: "Allow traffic to Composer nodes."
+  ingress-allow-composer-nodes:
-  direction: INGRESS
+    description: "Allow traffic to Composer nodes."
-  action: allow
+    sources:
-  sources:
+      - composer-worker
-    - composer-worker
+    targets:
-  targets:
+      - composer-worker
-    - composer-worker
+    rules:
-  use_service_accounts: false
+      - protocol: tcp
-  rules:
+        ports: [80, 443, 3306, 3307]
-    - protocol: tcp
+  ingress-allow-dataflow-load:
-      ports: [80, 443, 3306, 3307]
+    description: "Allow traffic to Dataflow nodes."
-
+    sources:
-ingress-allow-dataflow-load:
+      - dataflow
-  description: "Allow traffic to Dataflow nodes."
+    targets:
-  direction: INGRESS
+      - dataflow
-  action: allow
+    rules:
-  sources:
+      - protocol: tcp
-    - dataflow
+        ports: [12345, 12346]
-  targets:
-    - dataflow
-  use_service_accounts: false
-  rules:
-    - protocol: tcp
-      ports: [12345, 12346]

fast/stages/02-networking-peering/data/firewall-rules/landing/rules.yaml

@@ -1,15 +1,11 @@
 # skip boilerplate check
 
-allow-onprem-probes-example:
+ingress:
-  description: "Allow traffic from onprem probes"
+  allow-onprem-probes-example:
-  direction: INGRESS
+    description: "Allow traffic from onprem probes"
-  action: allow
+    source_ranges:
-  sources: []
+      - onprem_probes
-  ranges:
+    rules:
-    - $onprem_probes
+      - protocol: tcp
-  targets: []
+        ports:
-  use_service_accounts: false
+          - 12345
-  rules:
-    - protocol: tcp
-      ports:
-        - 12345

fast/stages/02-networking-peering/landing.tf

@@ -67,15 +67,16 @@ module "landing-vpc" {
 }
 
 module "landing-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.landing-project.project_id
+  project_id = module.landing-project.project_id
-  network             = module.landing-vpc.name
+  network    = module.landing-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/landing"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/landing"
+  }
 }
 
 module "landing-nat-ew1" {

fast/stages/02-networking-peering/spoke-dev.tf

@@ -67,15 +67,16 @@ module "dev-spoke-vpc" {
 }
 
 module "dev-spoke-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.dev-spoke-project.project_id
+  project_id = module.dev-spoke-project.project_id
-  network             = module.dev-spoke-vpc.name
+  network    = module.dev-spoke-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/dev"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/dev"
+  }
 }
 
 module "dev-spoke-cloudnat" {

fast/stages/02-networking-peering/spoke-prod.tf

@@ -67,15 +67,16 @@ module "prod-spoke-vpc" {
 }
 
 module "prod-spoke-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.prod-spoke-project.project_id
+  project_id = module.prod-spoke-project.project_id
-  network             = module.prod-spoke-vpc.name
+  network    = module.prod-spoke-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/prod"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/prod"
+  }
 }
 
 module "prod-spoke-cloudnat" {

fast/stages/02-networking-separate-envs/data/firewall-rules/dev/rules.yaml

@@ -1,27 +1,17 @@
 # skip boilerplate check
 
-ingress-allow-composer-nodes:
+ingress:
-  description: "Allow traffic to Composer nodes."
+  ingress-allow-composer-nodes:
-  direction: INGRESS
+    description: "Allow traffic to Composer nodes."
-  action: allow
+    targets:
-  sources: []
+      - composer-worker
-  ranges: ["0.0.0.0/0"]
+    rules:
-  targets:
+      - protocol: tcp
-    - composer-worker
+        ports: [80, 443, 3306, 3307]
-  use_service_accounts: false
+  ingress-allow-dataflow-load:
-  rules:
+    description: "Allow traffic to Dataflow nodes."
-    - protocol: tcp
+    targets:
-      ports: [80, 443, 3306, 3307]
+      - dataflow
-
+    rules:
-ingress-allow-dataflow-load:
+      - protocol: tcp
-  description: "Allow traffic to Dataflow nodes."
+        ports: [12345, 12346]
-  direction: INGRESS
-  action: allow
-  sources: []
-  ranges: ["0.0.0.0/0"]
-  targets:
-    - dataflow
-  use_service_accounts: false
-  rules:
-    - protocol: tcp
-      ports: [12345, 12346]

fast/stages/02-networking-separate-envs/spoke-dev.tf

@@ -66,15 +66,16 @@ module "dev-spoke-vpc" {
 }
 
 module "dev-spoke-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.dev-spoke-project.project_id
+  project_id = module.dev-spoke-project.project_id
-  network             = module.dev-spoke-vpc.name
+  network    = module.dev-spoke-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/dev"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/dev"
+  }
 }
 
 module "dev-spoke-cloudnat" {

fast/stages/02-networking-separate-envs/spoke-prod.tf

@@ -66,15 +66,16 @@ module "prod-spoke-vpc" {
 }
 
 module "prod-spoke-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.prod-spoke-project.project_id
+  project_id = module.prod-spoke-project.project_id
-  network             = module.prod-spoke-vpc.name
+  network    = module.prod-spoke-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/prod"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/prod"
+  }
 }
 
 module "prod-spoke-cloudnat" {

fast/stages/02-networking-vpn/data/firewall-rules/dev/rules.yaml

@@ -1,27 +1,21 @@
 # skip boilerplate check
 
-ingress-allow-composer-nodes:
+ingress:
-  description: "Allow traffic to Composer nodes."
+  ingress-allow-composer-nodes:
-  direction: INGRESS
+    description: "Allow traffic to Composer nodes."
-  action: allow
+    sources:
-  sources:
+      - composer-worker
-    - composer-worker
+    targets:
-  targets:
+      - composer-worker
-    - composer-worker
+    rules:
-  use_service_accounts: false
+      - protocol: tcp
-  rules:
+        ports: [80, 443, 3306, 3307]
-    - protocol: tcp
+  ingress-allow-dataflow-load:
-      ports: [80, 443, 3306, 3307]
+    description: "Allow traffic to Dataflow nodes."
-
+    sources:
-ingress-allow-dataflow-load:
+      - dataflow
-  description: "Allow traffic to Dataflow nodes."
+    targets:
-  direction: INGRESS
+      - dataflow
-  action: allow
+    rules:
-  sources:
+      - protocol: tcp
-    - dataflow
+        ports: [12345, 12346]
-  targets:
-    - dataflow
-  use_service_accounts: false
-  rules:
-    - protocol: tcp
-      ports: [12345, 12346]

fast/stages/02-networking-vpn/data/firewall-rules/landing/rules.yaml

@@ -1,15 +1,11 @@
 # skip boilerplate check
 
-allow-onprem-probes-example:
+ingress:
-  description: "Allow traffic from onprem probes"
+  allow-onprem-probes-example:
-  direction: INGRESS
+    description: "Allow traffic from onprem probes"
-  action: allow
+    source_ranges:
-  sources: []
+      - onprem_probes
-  ranges:
+    rules:
-    - $onprem_probes
+      - protocol: tcp
-  targets: []
+        ports:
-  use_service_accounts: false
+          - 12345
-  rules:
-    - protocol: tcp
-      ports:
-        - 12345

fast/stages/02-networking-vpn/landing.tf

@@ -67,15 +67,16 @@ module "landing-vpc" {
 }
 
 module "landing-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.landing-project.project_id
+  project_id = module.landing-project.project_id
-  network             = module.landing-vpc.name
+  network    = module.landing-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/landing"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/landing"
+  }
 }
 
 module "landing-nat-ew1" {

fast/stages/02-networking-vpn/spoke-dev.tf

@@ -67,15 +67,16 @@ module "dev-spoke-vpc" {
 }
 
 module "dev-spoke-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.dev-spoke-project.project_id
+  project_id = module.dev-spoke-project.project_id
-  network             = module.dev-spoke-vpc.name
+  network    = module.dev-spoke-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/dev"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/dev"
+  }
 }
 
 module "dev-spoke-cloudnat" {

fast/stages/02-networking-vpn/spoke-prod.tf

@@ -67,15 +67,16 @@ module "prod-spoke-vpc" {
 }
 
 module "prod-spoke-firewall" {
-  source              = "../../../modules/net-vpc-firewall"
+  source     = "../../../modules/net-vpc-firewall"
-  project_id          = module.prod-spoke-project.project_id
+  project_id = module.prod-spoke-project.project_id
-  network             = module.prod-spoke-vpc.name
+  network    = module.prod-spoke-vpc.name
-  admin_ranges        = []
+  default_rules_config = {
-  http_source_ranges  = []
+    disabled = true
-  https_source_ranges = []
+  }
-  ssh_source_ranges   = []
+  factories_config = {
-  data_folder         = "${var.data_dir}/firewall-rules/prod"
+    cidr_tpl_file = "${var.data_dir}/cidrs.yaml"
-  cidr_template_file  = "${var.data_dir}/cidrs.yaml"
+    rules_folder  = "${var.data_dir}/firewall-rules/prod"
+  }
 }
 
 module "prod-spoke-cloudnat" {

modules/README.md

@@ -36,7 +36,6 @@ These modules are used in the examples included in this repository. If you are u
 - [service accounts](./iam-service-account)
 - [logging bucket](./logging-bucket)
 - [organization](./organization)
-- [organization-policy](./organization-policy)
 - [project](./project)
 - [projects-data-source](./projects-data-source)
 

modules/cloud-function/main.tf

@@ -141,7 +141,7 @@ resource "google_cloudfunctions2_function" "function" {
     environment_variables = var.environment_variables
     source {
       storage_source {
-        bucket = google_storage_bucket.bucket[0].name
+        bucket = local.bucket
         object = google_storage_bucket_object.bundle.name
       }
     }

modules/compute-mig/README.md

@@ -417,25 +417,25 @@ module "nginx-mig" {
 
 | name | description | type | required | default |
 |---|---|:---:|:---:|:---:|
-| [instance_template](variables.tf#L150) | Instance template for the default version. | <code>string</code> | ✓ |  |
+| [instance_template](variables.tf#L174) | Instance template for the default version. | <code>string</code> | ✓ |  |
-| [location](variables.tf#L155) | Compute zone or region. | <code>string</code> | ✓ |  |
+| [location](variables.tf#L179) | Compute zone or region. | <code>string</code> | ✓ |  |
-| [name](variables.tf#L160) | Managed group name. | <code>string</code> | ✓ |  |
+| [name](variables.tf#L184) | Managed group name. | <code>string</code> | ✓ |  |
-| [project_id](variables.tf#L171) | Project id. | <code>string</code> | ✓ |  |
+| [project_id](variables.tf#L195) | Project id. | <code>string</code> | ✓ |  |
 | [all_instances_config](variables.tf#L17) | Metadata and labels set to all instances in the group. | <code title="object&#40;&#123;&#10;  labels   &#61; optional&#40;map&#40;string&#41;&#41;&#10;  metadata &#61; optional&#40;map&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
 | [auto_healing_policies](variables.tf#L26) | Auto-healing policies for this group. | <code title="object&#40;&#123;&#10;  health_check      &#61; optional&#40;string&#41;&#10;  initial_delay_sec &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
 | [autoscaler_config](variables.tf#L35) | Optional autoscaler configuration. | <code title="object&#40;&#123;&#10;  max_replicas    &#61; number&#10;  min_replicas    &#61; number&#10;  cooldown_period &#61; optional&#40;number&#41;&#10;  mode            &#61; optional&#40;string&#41; &#35; OFF, ONLY_UP, ON&#10;  scaling_control &#61; optional&#40;object&#40;&#123;&#10;    down &#61; optional&#40;object&#40;&#123;&#10;      max_replicas_fixed   &#61; optional&#40;number&#41;&#10;      max_replicas_percent &#61; optional&#40;number&#41;&#10;      time_window_sec      &#61; optional&#40;number&#41;&#10;    &#125;&#41;&#41;&#10;    in &#61; optional&#40;object&#40;&#123;&#10;      max_replicas_fixed   &#61; optional&#40;number&#41;&#10;      max_replicas_percent &#61; optional&#40;number&#41;&#10;      time_window_sec      &#61; optional&#40;number&#41;&#10;    &#125;&#41;&#41;&#10;  &#125;&#41;, &#123;&#125;&#41;&#10;  scaling_signals &#61; optional&#40;object&#40;&#123;&#10;    cpu_utilization &#61; optional&#40;object&#40;&#123;&#10;      target                &#61; number&#10;      optimize_availability &#61; optional&#40;bool&#41;&#10;    &#125;&#41;&#41;&#10;    load_balancing_utilization &#61; optional&#40;object&#40;&#123;&#10;      target &#61; number&#10;    &#125;&#41;&#41;&#10;    metrics &#61; optional&#40;list&#40;object&#40;&#123;&#10;      name                       &#61; string&#10;      type                       &#61; string &#35; GAUGE, DELTA_PER_SECOND, DELTA_PER_MINUTE&#10;      target_value               &#61; number&#10;      single_instance_assignment &#61; optional&#40;number&#41;&#10;      time_series_filter         &#61; optional&#40;string&#41;&#10;    &#125;&#41;&#41;&#41;&#10;    schedules &#61; optional&#40;list&#40;object&#40;&#123;&#10;      duration_sec          &#61; number&#10;      name                  &#61; string&#10;      min_required_replicas &#61; number&#10;      cron_schedule         &#61; string&#10;      description           &#61; optional&#40;bool&#41;&#10;      timezone              &#61; optional&#40;string&#41;&#10;      disabled              &#61; optional&#40;bool&#41;&#10;    &#125;&#41;&#41;&#41;&#10;  &#125;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
 | [default_version_name](variables.tf#L83) | Name used for the default version. | <code>string</code> |  | <code>&#34;default&#34;</code> |
 | [description](variables.tf#L89) | Optional description used for all resources managed by this module. | <code>string</code> |  | <code>&#34;Terraform managed.&#34;</code> |
 | [distribution_policy](variables.tf#L95) | DIstribution policy for regional MIG. | <code title="object&#40;&#123;&#10;  target_shape &#61; optional&#40;string&#41;&#10;  zones        &#61; optional&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
-| [health_check_config](variables.tf#L104) | Optional auto-created health check configuration, use the output self-link to set it in the auto healing policy. Refer to examples for usage. | <code title="object&#40;&#123;&#10;  check_interval_sec  &#61; optional&#40;number&#41;&#10;  description         &#61; optional&#40;string, &#34;Terraform managed.&#34;&#41;&#10;  enable_logging      &#61; optional&#40;bool, false&#41;&#10;  healthy_threshold   &#61; optional&#40;number&#41;&#10;  timeout_sec         &#61; optional&#40;number&#41;&#10;  unhealthy_threshold &#61; optional&#40;number&#41;&#10;  grpc &#61; optional&#40;object&#40;&#123;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    service_name       &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  http &#61; optional&#40;object&#40;&#123;&#10;    host               &#61; optional&#40;string&#41;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request_path       &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;    use_protocol       &#61; optional&#40;string, &#34;http&#34;&#41; &#35; http http2 https&#10;  &#125;&#41;&#41;&#10;  tcp &#61; optional&#40;object&#40;&#123;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request            &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;    use_ssl            &#61; optional&#40;bool, false&#41;&#10;  &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [health_check_config](variables.tf#L104) | Optional auto-created health check configuration, use the output self-link to set it in the auto healing policy. Refer to examples for usage. | <code title="object&#40;&#123;&#10;  check_interval_sec  &#61; optional&#40;number&#41;&#10;  description         &#61; optional&#40;string, &#34;Terraform managed.&#34;&#41;&#10;  enable_logging      &#61; optional&#40;bool, false&#41;&#10;  healthy_threshold   &#61; optional&#40;number&#41;&#10;  timeout_sec         &#61; optional&#40;number&#41;&#10;  unhealthy_threshold &#61; optional&#40;number&#41;&#10;  grpc &#61; optional&#40;object&#40;&#123;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    service_name       &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  http &#61; optional&#40;object&#40;&#123;&#10;    host               &#61; optional&#40;string&#41;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request_path       &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  http2 &#61; optional&#40;object&#40;&#123;&#10;    host               &#61; optional&#40;string&#41;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request_path       &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  https &#61; optional&#40;object&#40;&#123;&#10;    host               &#61; optional&#40;string&#41;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request_path       &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  tcp &#61; optional&#40;object&#40;&#123;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request            &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  ssl &#61; optional&#40;object&#40;&#123;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request            &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
-| [named_ports](variables.tf#L165) | Named ports. | <code>map&#40;number&#41;</code> |  | <code>null</code> |
+| [named_ports](variables.tf#L189) | Named ports. | <code>map&#40;number&#41;</code> |  | <code>null</code> |
-| [stateful_config](variables.tf#L183) | Stateful configuration for individual instances. | <code title="map&#40;object&#40;&#123;&#10;  minimal_action          &#61; optional&#40;string&#41;&#10;  most_disruptive_action  &#61; optional&#40;string&#41;&#10;  remove_state_on_destroy &#61; optional&#40;bool&#41;&#10;  preserved_state &#61; optional&#40;object&#40;&#123;&#10;    disks &#61; optional&#40;map&#40;object&#40;&#123;&#10;      source                      &#61; string&#10;      delete_on_instance_deletion &#61; optional&#40;bool&#41;&#10;      read_only                   &#61; optional&#40;bool&#41;&#10;    &#125;&#41;&#41;&#41;&#10;    metadata &#61; optional&#40;map&#40;string&#41;&#41;&#10;  &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [stateful_config](variables.tf#L207) | Stateful configuration for individual instances. | <code title="map&#40;object&#40;&#123;&#10;  minimal_action          &#61; optional&#40;string&#41;&#10;  most_disruptive_action  &#61; optional&#40;string&#41;&#10;  remove_state_on_destroy &#61; optional&#40;bool&#41;&#10;  preserved_state &#61; optional&#40;object&#40;&#123;&#10;    disks &#61; optional&#40;map&#40;object&#40;&#123;&#10;      source                      &#61; string&#10;      delete_on_instance_deletion &#61; optional&#40;bool&#41;&#10;      read_only                   &#61; optional&#40;bool&#41;&#10;    &#125;&#41;&#41;&#41;&#10;    metadata &#61; optional&#40;map&#40;string&#41;&#41;&#10;  &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [stateful_disks](variables.tf#L176) | Stateful disk configuration applied at the MIG level to all instances, in device name => on permanent instance delete rule as boolean. | <code>map&#40;bool&#41;</code> |  | <code>&#123;&#125;</code> |
+| [stateful_disks](variables.tf#L200) | Stateful disk configuration applied at the MIG level to all instances, in device name => on permanent instance delete rule as boolean. | <code>map&#40;bool&#41;</code> |  | <code>&#123;&#125;</code> |
-| [target_pools](variables.tf#L202) | Optional list of URLs for target pools to which new instances in the group are added. | <code>list&#40;string&#41;</code> |  | <code>&#91;&#93;</code> |
+| [target_pools](variables.tf#L226) | Optional list of URLs for target pools to which new instances in the group are added. | <code>list&#40;string&#41;</code> |  | <code>&#91;&#93;</code> |
-| [target_size](variables.tf#L208) | Group target size, leave null when using an autoscaler. | <code>number</code> |  | <code>null</code> |
+| [target_size](variables.tf#L232) | Group target size, leave null when using an autoscaler. | <code>number</code> |  | <code>null</code> |
-| [update_policy](variables.tf#L214) | Update policy. Minimal action and type are required. | <code title="object&#40;&#123;&#10;  minimal_action &#61; string&#10;  type           &#61; string&#10;  max_surge &#61; optional&#40;object&#40;&#123;&#10;    fixed   &#61; optional&#40;number&#41;&#10;    percent &#61; optional&#40;number&#41;&#10;  &#125;&#41;&#41;&#10;  max_unavailable &#61; optional&#40;object&#40;&#123;&#10;    fixed   &#61; optional&#40;number&#41;&#10;    percent &#61; optional&#40;number&#41;&#10;  &#125;&#41;&#41;&#10;  min_ready_sec                &#61; optional&#40;number&#41;&#10;  most_disruptive_action       &#61; optional&#40;string&#41;&#10;  regional_redistribution_type &#61; optional&#40;string&#41;&#10;  replacement_method           &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [update_policy](variables.tf#L238) | Update policy. Minimal action and type are required. | <code title="object&#40;&#123;&#10;  minimal_action &#61; string&#10;  type           &#61; string&#10;  max_surge &#61; optional&#40;object&#40;&#123;&#10;    fixed   &#61; optional&#40;number&#41;&#10;    percent &#61; optional&#40;number&#41;&#10;  &#125;&#41;&#41;&#10;  max_unavailable &#61; optional&#40;object&#40;&#123;&#10;    fixed   &#61; optional&#40;number&#41;&#10;    percent &#61; optional&#40;number&#41;&#10;  &#125;&#41;&#41;&#10;  min_ready_sec                &#61; optional&#40;number&#41;&#10;  most_disruptive_action       &#61; optional&#40;string&#41;&#10;  regional_redistribution_type &#61; optional&#40;string&#41;&#10;  replacement_method           &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
-| [versions](variables.tf#L235) | Additional application versions, target_size is optional. | <code title="map&#40;object&#40;&#123;&#10;  instance_template &#61; string&#10;  target_size &#61; optional&#40;object&#40;&#123;&#10;    fixed   &#61; optional&#40;number&#41;&#10;    percent &#61; optional&#40;number&#41;&#10;  &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [versions](variables.tf#L259) | Additional application versions, target_size is optional. | <code title="map&#40;object&#40;&#123;&#10;  instance_template &#61; string&#10;  target_size &#61; optional&#40;object&#40;&#123;&#10;    fixed   &#61; optional&#40;number&#41;&#10;    percent &#61; optional&#40;number&#41;&#10;  &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [wait_for_instances](variables.tf#L248) | Wait for all instances to be created/updated before returning. | <code title="object&#40;&#123;&#10;  enabled &#61; bool&#10;  status  &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [wait_for_instances](variables.tf#L272) | Wait for all instances to be created/updated before returning. | <code title="object&#40;&#123;&#10;  enabled &#61; bool&#10;  status  &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
 
 ## Outputs
 

modules/compute-mig/health-check.tf

@@ -17,25 +17,16 @@
 # tfdoc:file:description Health check resource.
 
 locals {
-  hc      = var.health_check_config
+  hc       = var.health_check_config
-  hc_grpc = try(local.hc.grpc, null) != null
+  hc_grpc  = try(local.hc.grpc, null) != null
-  hc_http = (
+  hc_http  = try(local.hc.http, null) != null
-    try(local.hc.http, null) != null &&
+  hc_http2 = try(local.hc.http2, null) != null
-    lower(try(local.hc.http.use_protocol, "")) == "http"
+  hc_https = try(local.hc.https, null) != null
-  )
+  hc_ssl   = try(local.hc.ssl, null) != null
-  hc_http2 = (
+  hc_tcp   = try(local.hc.tcp, null) != null
-    try(local.hc.http, null) != null &&
-    lower(try(local.hc.http.use_protocol, "")) == "http2"
-  )
-  hc_https = (
-    try(local.hc.http, null) != null &&
-    lower(try(local.hc.http.use_protocol, "")) == "https"
-  )
-  hc_ssl = try(local.hc.tcp.use_ssl, null) == true
-  hc_tcp = try(local.hc.tcp, null) != null && !local.hc_ssl
 }
 
-resource "google_compute_health_check" "autohealing" {
+resource "google_compute_health_check" "default" {
   provider            = google-beta
   count               = local.hc != null ? 1 : 0
   project             = var.project_id

modules/compute-mig/main.tf

@@ -17,7 +17,7 @@
 locals {
   health_check = (
     try(var.auto_healing_policies.health_check, null) == null
-    ? try(google_compute_health_check.autohealing.0.self_link, null)
+    ? try(google_compute_health_check.default.0.self_link, null)
     : try(var.auto_healing_policies.health_check, null)
   )
   instance_group_manager = (

modules/compute-mig/outputs.tf

@@ -37,6 +37,6 @@ output "health_check" {
   value = (
     var.health_check_config == null
     ? null
-    : google_compute_health_check.autohealing.0
+    : google_compute_health_check.default.0
   )
 }

modules/compute-mig/variables.tf

@@ -124,7 +124,24 @@ variable "health_check_config" {
       proxy_header       = optional(string)
       request_path       = optional(string)
       response           = optional(string)
-      use_protocol       = optional(string, "http") # http http2 https
+    }))
+    http2 = optional(object({
+      host               = optional(string)
+      port               = optional(number)
+      port_name          = optional(string)
+      port_specification = optional(string) # USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT
+      proxy_header       = optional(string)
+      request_path       = optional(string)
+      response           = optional(string)
+    }))
+    https = optional(object({
+      host               = optional(string)
+      port               = optional(number)
+      port_name          = optional(string)
+      port_specification = optional(string) # USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT
+      proxy_header       = optional(string)
+      request_path       = optional(string)
+      response           = optional(string)
     }))
     tcp = optional(object({
       port               = optional(number)
@@ -133,7 +150,14 @@ variable "health_check_config" {
       proxy_header       = optional(string)
       request            = optional(string)
       response           = optional(string)
-      use_ssl            = optional(bool, false)
+    }))
+    ssl = optional(object({
+      port               = optional(number)
+      port_name          = optional(string)
+      port_specification = optional(string) # USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT
+      proxy_header       = optional(string)
+      request            = optional(string)
+      response           = optional(string)
     }))
   })
   default = null

modules/folder/README.md

@@ -75,6 +75,10 @@ module "folder" {
 # tftest modules=1 resources=8
 ```
 
+### Organization policy factory
+
+See the [organization policy factory in the project module](../project#organization-policy-factory).
+
 ### Firewall policy factory
 
 In the same way as for the [organization](../organization) module, the in-built factory allows you to define a single policy, using one file for rules, and an optional file for CIDR range substitution variables. Remember that non-absolute paths are relative to the root module (the folder where you run `terraform`).
@@ -311,8 +315,9 @@ module "folder" {
 | [logging_sinks](variables.tf#L105) | Logging sinks to create for this folder. | <code title="map&#40;object&#40;&#123;&#10;  destination      &#61; string&#10;  type             &#61; string&#10;  filter           &#61; string&#10;  include_children &#61; bool&#10;  exclusions &#61; map&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
 | [name](variables.tf#L126) | Folder name. | <code>string</code> |  | <code>null</code> |
 | [org_policies](variables.tf#L132) | Organization policies applied to this folder keyed by policy name. | <code title="map&#40;object&#40;&#123;&#10;  inherit_from_parent &#61; optional&#40;bool&#41; &#35; for list policies only.&#10;  reset               &#61; optional&#40;bool&#41;&#10;  allow &#61; optional&#40;object&#40;&#123;&#10;    all    &#61; optional&#40;bool&#41;&#10;    values &#61; optional&#40;list&#40;string&#41;&#41;&#10;  &#125;&#41;&#41;&#10;  deny &#61; optional&#40;object&#40;&#123;&#10;    all    &#61; optional&#40;bool&#41;&#10;    values &#61; optional&#40;list&#40;string&#41;&#41;&#10;  &#125;&#41;&#41;&#10;  enforce &#61; optional&#40;bool, true&#41; &#35; for boolean policies only.&#10;  rules &#61; optional&#40;list&#40;object&#40;&#123;&#10;    allow &#61; optional&#40;object&#40;&#123;&#10;      all    &#61; optional&#40;bool&#41;&#10;      values &#61; optional&#40;list&#40;string&#41;&#41;&#10;    &#125;&#41;&#41;&#10;    deny &#61; optional&#40;object&#40;&#123;&#10;      all    &#61; optional&#40;bool&#41;&#10;      values &#61; optional&#40;list&#40;string&#41;&#41;&#10;    &#125;&#41;&#41;&#10;    enforce &#61; optional&#40;bool, true&#41; &#35; for boolean policies only.&#10;    condition &#61; object&#40;&#123;&#10;      description &#61; optional&#40;string&#41;&#10;      expression  &#61; optional&#40;string&#41;&#10;      location    &#61; optional&#40;string&#41;&#10;      title       &#61; optional&#40;string&#41;&#10;    &#125;&#41;&#10;  &#125;&#41;&#41;, &#91;&#93;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [parent](variables.tf#L172) | Parent in folders/folder_id or organizations/org_id format. | <code>string</code> |  | <code>null</code> |
+| [org_policies_data_path](variables.tf#L172) | Path containing org policies in YAML format. | <code>string</code> |  | <code>null</code> |
-| [tag_bindings](variables.tf#L182) | Tag bindings for this folder, in key => tag value id format. | <code>map&#40;string&#41;</code> |  | <code>null</code> |
+| [parent](variables.tf#L178) | Parent in folders/folder_id or organizations/org_id format. | <code>string</code> |  | <code>null</code> |
+| [tag_bindings](variables.tf#L188) | Tag bindings for this folder, in key => tag value id format. | <code>map&#40;string&#41;</code> |  | <code>null</code> |
 
 ## Outputs
 

modules/folder/organization-policies.tf

@@ -17,9 +17,61 @@
 # tfdoc:file:description Folder-level organization policies.
 
 locals {
+  _factory_data_raw = (
+    var.org_policies_data_path == null
+    ? tomap({})
+    : merge([
+      for f in fileset(var.org_policies_data_path, "*.yaml") :
+      yamldecode(file("${var.org_policies_data_path}/${f}"))
+    ]...)
+  )
+
+  # simulate applying defaults to data coming from yaml files
+  _factory_data = {
+    for k, v in local._factory_data_raw :
+    k => {
+      inherit_from_parent = try(v.inherit_from_parent, null)
+      reset               = try(v.reset, null)
+      allow = can(v.allow) ? {
+        all    = try(v.allow.all, null)
+        values = try(v.allow.values, null)
+      } : null
+      deny = can(v.deny) ? {
+        all    = try(v.deny.all, null)
+        values = try(v.deny.values, null)
+      } : null
+      enforce = try(v.enforce, true)
+
+      rules = [
+        for r in try(v.rules, []) : {
+          allow = can(r.allow) ? {
+            all    = try(r.allow.all, null)
+            values = try(r.allow.values, null)
+          } : null
+          deny = can(r.deny) ? {
+            all    = try(r.deny.all, null)
+            values = try(r.deny.values, null)
+          } : null
+          enforce = try(r.enforce, true)
+          condition = {
+            description = try(r.condition.description, null)
+            expression  = try(r.condition.expression, null)
+            location    = try(r.condition.location, null)
+            title       = try(r.condition.title, null)
+          }
+        }
+      ]
+    }
+  }
+
+  _org_policies = merge(local._factory_data, var.org_policies)
+
   org_policies = {
-    for k, v in var.org_policies :
+    for k, v in local._org_policies :
     k => merge(v, {
+      name   = "${local.folder.name}/policies/${k}"
+      parent = local.folder.name
+
       is_boolean_policy = v.allow == null && v.deny == null
       has_values = (
         length(coalesce(try(v.allow.values, []), [])) > 0 ||
@@ -40,8 +92,8 @@ locals {
 
 resource "google_org_policy_policy" "default" {
   for_each = local.org_policies
-  name     = "${local.folder.name}/policies/${each.key}"
+  name     = each.value.name
-  parent   = local.folder.name
+  parent   = each.value.parent
 
   spec {
     inherit_from_parent = each.value.inherit_from_parent

modules/folder/variables.tf

@@ -169,6 +169,12 @@ variable "org_policies" {
   nullable = false
 }
 
+variable "org_policies_data_path" {
+  description = "Path containing org policies in YAML format."
+  type        = string
+  default     = null
+}
+
 variable "parent" {
   description = "Parent in folders/folder_id or organizations/org_id format."
   type        = string

modules/gke-hub/README.md

@@ -178,50 +178,28 @@ module "firewall" {
   source     = "./fabric/modules/net-vpc-firewall"
   project_id = module.project.project_id
   network    = module.vpc.name
-  custom_rules = { 
+  ingress_rules = {
     allow-mesh = {
-      description          = "Allow mesh"
+      description   = "Allow mesh"
-      direction            = "INGRESS"
+      priority      = 900
-      action               = "allow"
+      source_ranges = ["10.1.0.0/16", "10.3.0.0/16"]
-      sources              = []
+      targets       = ["cluster-1-node", "cluster-2-node"]
-      ranges               = ["10.1.0.0/16", "10.3.0.0/16"]
-      targets              = ["cluster-1-node", "cluster-2-node"]
-      use_service_accounts = false
-      rules = [{ protocol = "tcp", ports = null },
-        { protocol = "udp", ports = null },
-        { protocol = "icmp", ports = null },
-        { protocol = "esp", ports = null },
-        { protocol = "ah", ports = null },
-      { protocol = "sctp", ports = null }]
-      extra_attributes = {
-        priority = 900
-      }
     },
     "allow-cluster-1-istio" = {
-      description          = "Allow istio sidecar injection, istioctl version and istioctl ps"
+      description   = "Allow istio sidecar injection, istioctl version and istioctl ps"
-      direction            = "INGRESS"
+      source_ranges = ["192.168.1.0/28"]
-      action               = "allow"
+      targets       = ["cluster-1-node"]
-      sources              = []
+      rules = [
-      ranges               = [ "192.168.1.0/28" ]
+        { protocol = "tcp", ports = [8080, 15014, 15017] }
-      targets              = ["cluster-1-node"]
+      ]
-      use_service_accounts = false
-      rules                = [{ protocol = "tcp", ports = [8080, 15014, 15017] }]
-      extra_attributes = {
-        priority = 1000
-      }
     },
     "allow-cluster-2-istio" = {
-      description          = "Allow istio sidecar injection, istioctl version and istioctl ps"
+      description   = "Allow istio sidecar injection, istioctl version and istioctl ps"
-      direction            = "INGRESS"
+      source_ranges = ["192.168.2.0/28"]
-      action               = "allow"
+      targets       = ["cluster-2-node"]
-      sources              = []
+      rules = [
-      ranges               = [ "192.168.2.0/28" ]
+        { protocol = "tcp", ports = [8080, 15014, 15017] }
-      targets              = ["cluster-2-node"]
+      ]
-      use_service_accounts = false
-      rules                = [{ protocol = "tcp", ports = [8080, 15014, 15017] }]
-      extra_attributes = {
-        priority = 1000
-      }
     }
   }
 }

modules/iam-service-account/README.md

@@ -43,21 +43,22 @@ module "myproject-default-service-accounts" {
 
 | name | description | type | required | default |
 |---|---|:---:|:---:|:---:|
-| [name](variables.tf#L84) | Name of the service account to create. | <code>string</code> | ✓ |  |
+| [name](variables.tf#L91) | Name of the service account to create. | <code>string</code> | ✓ |  |
-| [project_id](variables.tf#L95) | Project id where service account will be created. | <code>string</code> | ✓ |  |
+| [project_id](variables.tf#L102) | Project id where service account will be created. | <code>string</code> | ✓ |  |
 | [description](variables.tf#L17) | Optional description. | <code>string</code> |  | <code>null</code> |
 | [display_name](variables.tf#L23) | Display name of the service account to create. | <code>string</code> |  | <code>&#34;Terraform-managed.&#34;</code> |
 | [generate_key](variables.tf#L29) | Generate a key for service account. | <code>bool</code> |  | <code>false</code> |
 | [iam](variables.tf#L35) | IAM bindings on the service account in {ROLE => [MEMBERS]} format. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_billing_roles](variables.tf#L42) | Billing account roles granted to this service account, by billing account id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_additive](variables.tf#L42) | IAM additive bindings on the service account in {ROLE => [MEMBERS]} format. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_folder_roles](variables.tf#L49) | Folder roles granted to this service account, by folder id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_billing_roles](variables.tf#L49) | Billing account roles granted to this service account, by billing account id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_organization_roles](variables.tf#L56) | Organization roles granted to this service account, by organization id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_folder_roles](variables.tf#L56) | Folder roles granted to this service account, by folder id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_project_roles](variables.tf#L63) | Project roles granted to this service account, by project id. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_organization_roles](variables.tf#L63) | Organization roles granted to this service account, by organization id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_sa_roles](variables.tf#L70) | Service account roles granted to this service account, by service account name. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_project_roles](variables.tf#L70) | Project roles granted to this service account, by project id. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_storage_roles](variables.tf#L77) | Storage roles granted to this service account, by bucket name. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_sa_roles](variables.tf#L77) | Service account roles granted to this service account, by service account name. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [prefix](variables.tf#L89) | Prefix applied to service account names. | <code>string</code> |  | <code>null</code> |
+| [iam_storage_roles](variables.tf#L84) | Storage roles granted to this service account, by bucket name. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [public_keys_directory](variables.tf#L100) | Path to public keys data files to upload to the service account (should have `.pem` extension). | <code>string</code> |  | <code>&#34;&#34;</code> |
+| [prefix](variables.tf#L96) | Prefix applied to service account names. | <code>string</code> |  | <code>null</code> |
-| [service_account_create](variables.tf#L106) | Create service account. When set to false, uses a data source to reference an existing service account. | <code>bool</code> |  | <code>true</code> |
+| [public_keys_directory](variables.tf#L107) | Path to public keys data files to upload to the service account (should have `.pem` extension). | <code>string</code> |  | <code>&#34;&#34;</code> |
+| [service_account_create](variables.tf#L113) | Create service account. When set to false, uses a data source to reference an existing service account. | <code>bool</code> |  | <code>true</code> |
 
 ## Outputs
 
@@ -66,9 +67,9 @@ module "myproject-default-service-accounts" {
 | [email](outputs.tf#L17) | Service account email. |  |
 | [iam_email](outputs.tf#L25) | IAM-format service account email. |  |
 | [id](outputs.tf#L33) | Service account id. |  |
-| [key](outputs.tf#L41) | Service account key. | ✓ |
+| [key](outputs.tf#L42) | Service account key. | ✓ |
-| [name](outputs.tf#L47) | Service account name. |  |
+| [name](outputs.tf#L48) | Service account name. |  |
-| [service_account](outputs.tf#L52) | Service account resource. |  |
+| [service_account](outputs.tf#L57) | Service account resource. |  |
-| [service_account_credentials](outputs.tf#L57) | Service account json credential templates for uploaded public keys data. |  |
+| [service_account_credentials](outputs.tf#L62) | Service account json credential templates for uploaded public keys data. |  |
 
 <!-- END TFDOC -->

modules/iam-service-account/iam.tf

@@ -17,6 +17,15 @@
 # tfdoc:file:description IAM bindings.
 
 locals {
+  _iam_additive_pairs = flatten([
+    for role, members in var.iam_additive : [
+      for member in members : { role = role, member = member }
+    ]
+  ])
+  iam_additive = {
+    for pair in local._iam_additive_pairs :
+    "${pair.role}-${pair.member}" => pair
+  }
   iam_billing_pairs = flatten([
     for entity, roles in var.iam_billing_roles : [
       for role in roles : [
@@ -61,6 +70,13 @@ locals {
   ])
 }
 
+resource "google_service_account_iam_member" "roles" {
+  for_each           = local.iam_additive
+  service_account_id = local.service_account.name
+  role               = each.value.role
+  member             = each.value.member
+}
+
 resource "google_service_account_iam_binding" "roles" {
   for_each           = var.iam
   service_account_id = local.service_account.name

modules/iam-service-account/main.tf

@@ -21,7 +21,7 @@ locals {
     ? google_service_account_key.key["1"]
     : map("", null)
   , {})
-  prefix                = var.prefix != null ? "${var.prefix}-" : ""
+  prefix                = var.prefix == null || var.prefix == "" ? "" : "${var.prefix}-"
   resource_email_static = "${local.prefix}${var.name}@${var.project_id}.iam.gserviceaccount.com"
   resource_iam_email = (
     local.service_account != null
@@ -29,6 +29,7 @@ locals {
     : local.resource_iam_email_static
   )
   resource_iam_email_static = "serviceAccount:${local.resource_email_static}"
+  service_account_id_static = "projects/${var.project_id}/serviceAccounts/${local.resource_email_static}"
   service_account = (
     var.service_account_create
     ? try(google_service_account.service_account.0, null)

modules/iam-service-account/outputs.tf

@@ -32,9 +32,10 @@ output "iam_email" {
 
 output "id" {
   description = "Service account id."
-  value       = local.service_account.id
+  value       = local.service_account_id_static
   depends_on = [
-    local.service_account
+    data.google_service_account.service_account,
+    google_service_account.service_account
   ]
 }
 
@@ -46,7 +47,11 @@ output "key" {
 
 output "name" {
   description = "Service account name."
-  value       = local.service_account.name
+  value       = local.service_account_id_static
+  depends_on = [
+    data.google_service_account.service_account,
+    google_service_account.service_account
+  ]
 }
 
 output "service_account" {

modules/iam-service-account/variables.tf

@@ -39,6 +39,13 @@ variable "iam" {
   nullable    = false
 }
 
+variable "iam_additive" {
+  description = "IAM additive bindings on the service account in {ROLE => [MEMBERS]} format."
+  type        = map(list(string))
+  default     = {}
+  nullable    = false
+}
+
 variable "iam_billing_roles" {
   description = "Billing account roles granted to this service account, by billing account id. Non-authoritative."
   type        = map(list(string))

modules/net-ilb/README.md

@@ -23,11 +23,13 @@ module "ilb" {
   region        = "europe-west1"
   name          = "ilb-test"
   service_label = "ilb-test"
-  network       = var.vpc.self_link
+  vpc_config = {
-  subnetwork    = var.subnet.self_link
+    network    = var.vpc.self_link
+    subnetwork = var.subnet.self_link
+  }
   group_configs = {
     my-group = {
-      zone = "europe-west1-b", named_ports = null
+      zone = "europe-west1-b"
       instances = [
         "instance-1-self-link",
         "instance-2-self-link"
@@ -35,12 +37,12 @@ module "ilb" {
     }
   }
   backends = [{
-    failover       = false
     group          = module.ilb.groups.my-group.self_link
-    balancing_mode = "CONNECTION"
   }]
   health_check_config = {
-    type = "http", check = { port = 80 }, config = {}, logging = true
+    http = {
+      port = 80
+    }
   }
 }
 # tftest modules=1 resources=4
@@ -91,18 +93,21 @@ module "ilb" {
   region        = "europe-west1"
   name          = "ilb-test"
   service_label = "ilb-test"
-  network       = var.vpc.self_link
+  vpc_config = {
-  subnetwork    = var.subnet.self_link
+    network    = var.vpc.self_link
+    subnetwork = var.subnet.self_link
+    }
   ports         = [80]
   backends = [
     for z, mod in module.instance-group : {
-      failover       = false
       group          = mod.group.self_link
-      balancing_mode = "CONNECTION"
+      balancing_mode = "UTILIZATION"
     }
   ]
   health_check_config = {
-    type = "http", check = { port = 80 }, config = {}, logging = true
+    http = {
+      port = 80
+    }
   }
 }
 # tftest modules=3 resources=7
@@ -113,37 +118,36 @@ module "ilb" {
 
 | name | description | type | required | default |
 |---|---|:---:|:---:|:---:|
-| [backends](variables.tf#L33) | Load balancer backends, balancing mode is one of 'CONNECTION' or 'UTILIZATION'. | <code title="list&#40;object&#40;&#123;&#10;  failover       &#61; bool&#10;  group          &#61; string&#10;  balancing_mode &#61; string&#10;&#125;&#41;&#41;">list&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | ✓ |  |
+| [name](variables.tf#L184) | Name used for all resources. | <code>string</code> | ✓ |  |
-| [name](variables.tf#L98) | Name used for all resources. | <code>string</code> | ✓ |  |
+| [project_id](variables.tf#L195) | Project id where resources will be created. | <code>string</code> | ✓ |  |
-| [network](variables.tf#L103) | Network used for resources. | <code>string</code> | ✓ |  |
+| [region](variables.tf#L206) | GCP region. | <code>string</code> | ✓ |  |
-| [project_id](variables.tf#L114) | Project id where resources will be created. | <code>string</code> | ✓ |  |
+| [vpc_config](variables.tf#L217) | VPC-level configuration. | <code title="object&#40;&#123;&#10;  network    &#61; string&#10;  subnetwork &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  |
-| [region](variables.tf#L125) | GCP region. | <code>string</code> | ✓ |  |
-| [subnetwork](variables.tf#L136) | Subnetwork used for the forwarding rule. | <code>string</code> | ✓ |  |
 | [address](variables.tf#L17) | Optional IP address used for the forwarding rule. | <code>string</code> |  | <code>null</code> |
-| [backend_config](variables.tf#L23) | Optional backend configuration. | <code title="object&#40;&#123;&#10;  session_affinity                &#61; string&#10;  timeout_sec                     &#61; number&#10;  connection_draining_timeout_sec &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [backend_service_config](variables.tf#L23) | Backend service level configuration. | <code title="object&#40;&#123;&#10;  connection_draining_timeout_sec &#61; optional&#40;number&#41;&#10;  connection_tracking &#61; optional&#40;object&#40;&#123;&#10;    idle_timeout_sec          &#61; optional&#40;number&#41;&#10;    persist_conn_on_unhealthy &#61; optional&#40;string&#41;&#10;    track_per_session         &#61; optional&#40;bool&#41;&#10;  &#125;&#41;&#41;&#10;  enable_subsetting &#61; optional&#40;bool&#41;&#10;  failover_config &#61; optional&#40;object&#40;&#123;&#10;    disable_conn_drain        &#61; optional&#40;bool&#41;&#10;    drop_traffic_if_unhealthy &#61; optional&#40;bool&#41;&#10;    ratio                     &#61; optional&#40;number&#41;&#10;  &#125;&#41;&#41;&#10;  log_sample_rate  &#61; optional&#40;number&#41;&#10;  session_affinity &#61; optional&#40;string&#41;&#10;  timeout_sec      &#61; optional&#40;number&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [failover_config](variables.tf#L42) | Optional failover configuration. | <code title="object&#40;&#123;&#10;  disable_connection_drain  &#61; bool&#10;  drop_traffic_if_unhealthy &#61; bool&#10;  ratio                     &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [backends](variables.tf#L56) | Load balancer backends, balancing mode is one of 'CONNECTION' or 'UTILIZATION'. | <code title="list&#40;object&#40;&#123;&#10;  group          &#61; string&#10;  balancing_mode &#61; optional&#40;string, &#34;CONNECTION&#34;&#41;&#10;  description    &#61; optional&#40;string, &#34;Terraform managed.&#34;&#41;&#10;  failover       &#61; optional&#40;bool, false&#41;&#10;&#125;&#41;&#41;">list&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#91;&#93;</code> |
-| [global_access](variables.tf#L52) | Global access, defaults to false if not set. | <code>bool</code> |  | <code>null</code> |
+| [description](variables.tf#L75) | Optional description used for resources. | <code>string</code> |  | <code>&#34;Terraform managed.&#34;</code> |
-| [group_configs](variables.tf#L58) | Optional unmanaged groups to create. Can be referenced in backends via outputs. | <code title="map&#40;object&#40;&#123;&#10;  instances   &#61; list&#40;string&#41;&#10;  named_ports &#61; map&#40;number&#41;&#10;  zone        &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [global_access](variables.tf#L81) | Global access, defaults to false if not set. | <code>bool</code> |  | <code>null</code> |
-| [health_check](variables.tf#L68) | Name of existing health check to use, disables auto-created health check. | <code>string</code> |  | <code>null</code> |
+| [group_configs](variables.tf#L87) | Optional unmanaged groups to create. Can be referenced in backends via outputs. | <code title="map&#40;object&#40;&#123;&#10;  zone        &#61; string&#10;  instances   &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10;  named_ports &#61; optional&#40;map&#40;number&#41;, &#123;&#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [health_check_config](variables.tf#L74) | Configuration of the auto-created helth check. | <code title="object&#40;&#123;&#10;  type    &#61; string      &#35; http https tcp ssl http2&#10;  check   &#61; map&#40;any&#41;    &#35; actual health check block attributes&#10;  config  &#61; map&#40;number&#41; &#35; interval, thresholds, timeout&#10;  logging &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  type &#61; &#34;http&#34;&#10;  check &#61; &#123;&#10;    port_specification &#61; &#34;USE_SERVING_PORT&#34;&#10;  &#125;&#10;  config  &#61; &#123;&#125;&#10;  logging &#61; false&#10;&#125;">&#123;&#8230;&#125;</code> |
+| [health_check](variables.tf#L98) | Name of existing health check to use, disables auto-created health check. | <code>string</code> |  | <code>null</code> |
-| [labels](variables.tf#L92) | Labels set on resources. | <code>map&#40;string&#41;</code> |  | <code>&#123;&#125;</code> |
+| [health_check_config](variables.tf#L104) | Optional auto-created health check configuration, use the output self-link to set it in the auto healing policy. Refer to examples for usage. | <code title="object&#40;&#123;&#10;  check_interval_sec  &#61; optional&#40;number&#41;&#10;  description         &#61; optional&#40;string, &#34;Terraform managed.&#34;&#41;&#10;  enable_logging      &#61; optional&#40;bool, false&#41;&#10;  healthy_threshold   &#61; optional&#40;number&#41;&#10;  timeout_sec         &#61; optional&#40;number&#41;&#10;  unhealthy_threshold &#61; optional&#40;number&#41;&#10;  grpc &#61; optional&#40;object&#40;&#123;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    service_name       &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  http &#61; optional&#40;object&#40;&#123;&#10;    host               &#61; optional&#40;string&#41;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request_path       &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  http2 &#61; optional&#40;object&#40;&#123;&#10;    host               &#61; optional&#40;string&#41;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request_path       &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  https &#61; optional&#40;object&#40;&#123;&#10;    host               &#61; optional&#40;string&#41;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request_path       &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  tcp &#61; optional&#40;object&#40;&#123;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request            &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;  ssl &#61; optional&#40;object&#40;&#123;&#10;    port               &#61; optional&#40;number&#41;&#10;    port_name          &#61; optional&#40;string&#41;&#10;    port_specification &#61; optional&#40;string&#41; &#35; USE_FIXED_PORT USE_NAMED_PORT USE_SERVING_PORT&#10;    proxy_header       &#61; optional&#40;string&#41;&#10;    request            &#61; optional&#40;string&#41;&#10;    response           &#61; optional&#40;string&#41;&#10;  &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  tcp &#61; &#123;&#10;    port_specification &#61; &#34;USE_SERVING_PORT&#34;&#10;  &#125;&#10;&#125;">&#123;&#8230;&#125;</code> |
-| [ports](variables.tf#L108) | Comma-separated ports, leave null to use all ports. | <code>list&#40;string&#41;</code> |  | <code>null</code> |
+| [labels](variables.tf#L178) | Labels set on resources. | <code>map&#40;string&#41;</code> |  | <code>&#123;&#125;</code> |
-| [protocol](variables.tf#L119) | IP protocol used, defaults to TCP. | <code>string</code> |  | <code>&#34;TCP&#34;</code> |
+| [ports](variables.tf#L189) | Comma-separated ports, leave null to use all ports. | <code>list&#40;string&#41;</code> |  | <code>null</code> |
-| [service_label](variables.tf#L130) | Optional prefix of the fully qualified forwarding rule name. | <code>string</code> |  | <code>null</code> |
+| [protocol](variables.tf#L200) | IP protocol used, defaults to TCP. | <code>string</code> |  | <code>&#34;TCP&#34;</code> |
+| [service_label](variables.tf#L211) | Optional prefix of the fully qualified forwarding rule name. | <code>string</code> |  | <code>null</code> |
 
 ## Outputs
 
 | name | description | sensitive |
 |---|---|:---:|
-| [backend](outputs.tf#L17) | Backend resource. |  |
+| [backend_service](outputs.tf#L17) | Backend resource. |  |
-| [backend_id](outputs.tf#L22) | Backend id. |  |
+| [backend_service_id](outputs.tf#L22) | Backend id. |  |
-| [backend_self_link](outputs.tf#L27) | Backend self link. |  |
+| [backend_service_self_link](outputs.tf#L27) | Backend self link. |  |
 | [forwarding_rule](outputs.tf#L32) | Forwarding rule resource. |  |
 | [forwarding_rule_address](outputs.tf#L37) | Forwarding rule address. |  |
 | [forwarding_rule_id](outputs.tf#L42) | Forwarding rule id. |  |
 | [forwarding_rule_self_link](outputs.tf#L47) | Forwarding rule self link. |  |
-| [group_self_links](outputs.tf#L52) | Optional unmanaged instance group self links. |  |
+| [group_self_links](outputs.tf#L57) | Optional unmanaged instance group self links. |  |
-| [groups](outputs.tf#L59) | Optional unmanaged instance group resources. |  |
+| [groups](outputs.tf#L52) | Optional unmanaged instance group resources. |  |
 | [health_check](outputs.tf#L64) | Auto-created health-check resource. |  |
 | [health_check_self_id](outputs.tf#L69) | Auto-created health-check self id. |  |
 | [health_check_self_link](outputs.tf#L74) | Auto-created health-check self link. |  |

modules/net-ilb/groups.tf

@@ -0,0 +1,33 @@
+/**
+ * Copyright 2022 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+# tfdoc:file:description Optional instance group resources.
+
+resource "google_compute_instance_group" "unmanaged" {
+  for_each    = var.group_configs
+  project     = var.project_id
+  zone        = each.value.zone
+  name        = each.key
+  description = "Terraform-managed."
+  instances   = each.value.instances
+  dynamic "named_port" {
+    for_each = each.value.named_ports
+    content {
+      name = named_port.key
+      port = named_port.value
+    }
+  }
+}

modules/net-ilb/health-check.tf

@@ -0,0 +1,119 @@
+/**
+ * Copyright 2022 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+# tfdoc:file:description Health check resource.
+
+locals {
+  hc       = var.health_check_config
+  hc_grpc  = try(local.hc.grpc, null) != null
+  hc_http  = try(local.hc.http, null) != null
+  hc_http2 = try(local.hc.http2, null) != null
+  hc_https = try(local.hc.https, null) != null
+  hc_ssl   = try(local.hc.ssl, null) != null
+  hc_tcp   = try(local.hc.tcp, null) != null
+}
+
+resource "google_compute_health_check" "default" {
+  provider            = google-beta
+  count               = local.hc != null ? 1 : 0
+  project             = var.project_id
+  name                = var.name
+  description         = local.hc.description
+  check_interval_sec  = local.hc.check_interval_sec
+  healthy_threshold   = local.hc.healthy_threshold
+  timeout_sec         = local.hc.timeout_sec
+  unhealthy_threshold = local.hc.unhealthy_threshold
+
+  dynamic "grpc_health_check" {
+    for_each = local.hc_grpc ? [""] : []
+    content {
+      port               = local.hc.grpc.port
+      port_name          = local.hc.grpc.port_name
+      port_specification = local.hc.grpc.port_specification
+      grpc_service_name  = local.hc.grpc.service_name
+    }
+  }
+
+  dynamic "http_health_check" {
+    for_each = local.hc_http ? [""] : []
+    content {
+      host               = local.hc.http.host
+      port               = local.hc.http.port
+      port_name          = local.hc.http.port_name
+      port_specification = local.hc.http.port_specification
+      proxy_header       = local.hc.http.proxy_header
+      request_path       = local.hc.http.request_path
+      response           = local.hc.http.response
+    }
+  }
+
+  dynamic "http2_health_check" {
+    for_each = local.hc_http2 ? [""] : []
+    content {
+      host               = local.hc.http.host
+      port               = local.hc.http.port
+      port_name          = local.hc.http.port_name
+      port_specification = local.hc.http.port_specification
+      proxy_header       = local.hc.http.proxy_header
+      request_path       = local.hc.http.request_path
+      response           = local.hc.http.response
+    }
+  }
+
+  dynamic "https_health_check" {
+    for_each = local.hc_https ? [""] : []
+    content {
+      host               = local.hc.http.host
+      port               = local.hc.http.port
+      port_name          = local.hc.http.port_name
+      port_specification = local.hc.http.port_specification
+      proxy_header       = local.hc.http.proxy_header
+      request_path       = local.hc.http.request_path
+      response           = local.hc.http.response
+    }
+  }
+
+  dynamic "ssl_health_check" {
+    for_each = local.hc_ssl ? [""] : []
+    content {
+      port               = local.hc.tcp.port
+      port_name          = local.hc.tcp.port_name
+      port_specification = local.hc.tcp.port_specification
+      proxy_header       = local.hc.tcp.proxy_header
+      request            = local.hc.tcp.request
+      response           = local.hc.tcp.response
+    }
+  }
+
+  dynamic "tcp_health_check" {
+    for_each = local.hc_tcp ? [""] : []
+    content {
+      port               = local.hc.tcp.port
+      port_name          = local.hc.tcp.port_name
+      port_specification = local.hc.tcp.port_specification
+      proxy_header       = local.hc.tcp.proxy_header
+      request            = local.hc.tcp.request
+      response           = local.hc.tcp.response
+    }
+  }
+
+  dynamic "log_config" {
+    for_each = try(local.hc.enable_logging, null) == true ? [""] : []
+    content {
+      enable = true
+    }
+  }
+}

modules/net-ilb/main.tf

@@ -16,252 +16,100 @@
 
 
 locals {
+  bs_conntrack = var.backend_service_config.connection_tracking
+  bs_failover  = var.backend_service_config.failover_config
   health_check = (
     var.health_check != null
     ? var.health_check
-    : try(local.health_check_resource.self_link, null)
+    : google_compute_health_check.default.0.self_link
   )
-  health_check_resource = try(
-    google_compute_health_check.http.0,
-    google_compute_health_check.https.0,
-    google_compute_health_check.tcp.0,
-    google_compute_health_check.ssl.0,
-    google_compute_health_check.http2.0,
-    {}
-  )
-  health_check_type = try(var.health_check_config.type, null)
 }
 
 resource "google_compute_forwarding_rule" "default" {
-  provider              = google-beta
+  provider    = google-beta
-  project               = var.project_id
+  project     = var.project_id
-  name                  = var.name
+  region      = var.region
-  description           = "Terraform managed."
+  name        = var.name
+  description = var.description
+  ip_address  = var.address
+  ip_protocol = var.protocol # TCP | UDP
+  backend_service = (
+    google_compute_region_backend_service.default.self_link
+  )
   load_balancing_scheme = "INTERNAL"
-  region                = var.region
+  network               = var.vpc_config.network
-  network               = var.network
+  ports                 = var.ports # "nnnnn" or "nnnnn,nnnnn,nnnnn" max 5
-  subnetwork            = var.subnetwork
+  subnetwork            = var.vpc_config.subnetwork
-  ip_address            = var.address
-  ip_protocol           = var.protocol # TCP | UDP
-  ports                 = var.ports    # "nnnnn" or "nnnnn,nnnnn,nnnnn" max 5
-  service_label         = var.service_label
-  all_ports             = var.ports == null ? true : null
   allow_global_access   = var.global_access
-  backend_service       = google_compute_region_backend_service.default.self_link
+  labels                = var.labels
+  all_ports             = var.ports == null ? true : null
+  service_label         = var.service_label
   # is_mirroring_collector = false
-  labels = var.labels
 }
 
 resource "google_compute_region_backend_service" "default" {
-  provider              = google-beta
+  provider                        = google-beta
-  project               = var.project_id
+  project                         = var.project_id
-  name                  = var.name
+  region                          = var.region
-  description           = "Terraform managed."
+  name                            = var.name
-  load_balancing_scheme = "INTERNAL"
+  description                     = var.description
-  region                = var.region
+  load_balancing_scheme           = "INTERNAL"
-  network               = var.network
+  protocol                        = var.protocol
-  health_checks         = [local.health_check]
+  network                         = var.vpc_config.network
-  protocol              = var.protocol
+  health_checks                   = [local.health_check]
-
+  connection_draining_timeout_sec = var.backend_service_config.connection_draining_timeout_sec
-  session_affinity                = try(var.backend_config.session_affinity, null)
+  session_affinity                = var.backend_service_config.session_affinity
-  timeout_sec                     = try(var.backend_config.timeout_sec, null)
+  timeout_sec                     = var.backend_service_config.timeout_sec
-  connection_draining_timeout_sec = try(var.backend_config.connection_draining_timeout_sec, null)
 
   dynamic "backend" {
     for_each = { for b in var.backends : b.group => b }
-    iterator = backend
     content {
       balancing_mode = backend.value.balancing_mode
-      description    = "Terraform managed."
+      description    = backend.value.description
       failover       = backend.value.failover
       group          = backend.key
     }
   }
 
+  dynamic "connection_tracking_policy" {
+    for_each = local.bs_conntrack == null ? [] : [""]
+    content {
+      connection_persistence_on_unhealthy_backends = (
+        local.bs_conntrack.persist_conn_on_unhealthy != null
+        ? local.bs_conntrack.persist_conn_on_unhealthy
+        : null
+      )
+      idle_timeout_sec = local.bs_conntrack.idle_timeout_sec
+      tracking_mode = (
+        local.bs_conntrack.track_per_session != null
+        ? local.bs_conntrack.track_per_session
+        : null
+      )
+    }
+  }
+
   dynamic "failover_policy" {
-    for_each = var.failover_config == null ? [] : [var.failover_config]
+    for_each = local.bs_failover == null ? [] : [""]
-    iterator = config
     content {
-      disable_connection_drain_on_failover = config.value.disable_connection_drain
+      disable_connection_drain_on_failover = local.bs_failover.disable_conn_drain
-      drop_traffic_if_unhealthy            = config.value.drop_traffic_if_unhealthy
+      drop_traffic_if_unhealthy            = local.bs_failover.drop_traffic_if_unhealthy
-      failover_ratio                       = config.value.ratio
+      failover_ratio                       = local.bs_failover.ratio
     }
   }
 
-}
-
-resource "google_compute_instance_group" "unmanaged" {
-  for_each    = var.group_configs
-  project     = var.project_id
-  zone        = each.value.zone
-  name        = each.key
-  description = "Terraform-managed."
-  instances   = each.value.instances
-  dynamic "named_port" {
-    for_each = each.value.named_ports != null ? each.value.named_ports : {}
-    iterator = config
-    content {
-      name = config.key
-      port = config.value
-    }
-  }
-}
-
-resource "google_compute_health_check" "http" {
-  provider = google-beta
-  count = (
-    var.health_check == null && local.health_check_type == "http" ? 1 : 0
-  )
-  project     = var.project_id
-  name        = var.name
-  description = "Terraform managed."
-
-  check_interval_sec  = try(var.health_check_config.config.check_interval_sec, null)
-  healthy_threshold   = try(var.health_check_config.config.healthy_threshold, null)
-  timeout_sec         = try(var.health_check_config.config.timeout_sec, null)
-  unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
-
-  http_health_check {
-    host               = try(var.health_check_config.check.host, null)
-    port               = try(var.health_check_config.check.port, null)
-    port_name          = try(var.health_check_config.check.port_name, null)
-    port_specification = try(var.health_check_config.check.port_specification, null)
-    proxy_header       = try(var.health_check_config.check.proxy_header, null)
-    request_path       = try(var.health_check_config.check.request_path, null)
-    response           = try(var.health_check_config.check.response, null)
-  }
-
   dynamic "log_config" {
-    for_each = try(var.health_check_config.logging, false) ? [""] : []
+    for_each = var.backend_service_config.log_sample_rate == null ? [] : [""]
     content {
-      enable = true
+      enable      = true
+      sample_rate = var.backend_service_config.log_sample_rate
     }
   }
-}
 
-resource "google_compute_health_check" "https" {
+  dynamic "subsetting" {
-  provider = google-beta
+    for_each = var.backend_service_config.enable_subsetting == true ? [""] : []
-  count = (
-    var.health_check == null && local.health_check_type == "https" ? 1 : 0
-  )
-  project     = var.project_id
-  name        = var.name
-  description = "Terraform managed."
-
-  check_interval_sec  = try(var.health_check_config.config.check_interval_sec, null)
-  healthy_threshold   = try(var.health_check_config.config.healthy_threshold, null)
-  timeout_sec         = try(var.health_check_config.config.timeout_sec, null)
-  unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
-
-  https_health_check {
-    host               = try(var.health_check_config.check.host, null)
-    port               = try(var.health_check_config.check.port, null)
-    port_name          = try(var.health_check_config.check.port_name, null)
-    port_specification = try(var.health_check_config.check.port_specification, null)
-    proxy_header       = try(var.health_check_config.check.proxy_header, null)
-    request_path       = try(var.health_check_config.check.request_path, null)
-    response           = try(var.health_check_config.check.response, null)
-  }
-
-  dynamic "log_config" {
-    for_each = try(var.health_check_config.logging, false) ? [""] : []
     content {
-      enable = true
+      policy = "CONSISTENT_HASH_SUBSETTING"
     }
   }
+
 }
-
-resource "google_compute_health_check" "tcp" {
-  provider = google-beta
-  count = (
-    var.health_check == null && local.health_check_type == "tcp" ? 1 : 0
-  )
-  project     = var.project_id
-  name        = var.name
-  description = "Terraform managed."
-
-  check_interval_sec  = try(var.health_check_config.config.check_interval_sec, null)
-  healthy_threshold   = try(var.health_check_config.config.healthy_threshold, null)
-  timeout_sec         = try(var.health_check_config.config.timeout_sec, null)
-  unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
-
-  tcp_health_check {
-    port               = try(var.health_check_config.check.port, null)
-    port_name          = try(var.health_check_config.check.port_name, null)
-    port_specification = try(var.health_check_config.check.port_specification, null)
-    proxy_header       = try(var.health_check_config.check.proxy_header, null)
-    request            = try(var.health_check_config.check.request, null)
-    response           = try(var.health_check_config.check.response, null)
-  }
-
-  dynamic "log_config" {
-    for_each = try(var.health_check_config.logging, false) ? [""] : []
-    content {
-      enable = true
-    }
-  }
-}
-
-resource "google_compute_health_check" "ssl" {
-  provider = google-beta
-  count = (
-    var.health_check == null && local.health_check_type == "ssl" ? 1 : 0
-  )
-  project     = var.project_id
-  name        = var.name
-  description = "Terraform managed."
-
-  check_interval_sec  = try(var.health_check_config.config.check_interval_sec, null)
-  healthy_threshold   = try(var.health_check_config.config.healthy_threshold, null)
-  timeout_sec         = try(var.health_check_config.config.timeout_sec, null)
-  unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
-
-  ssl_health_check {
-    port               = try(var.health_check_config.check.port, null)
-    port_name          = try(var.health_check_config.check.port_name, null)
-    port_specification = try(var.health_check_config.check.port_specification, null)
-    proxy_header       = try(var.health_check_config.check.proxy_header, null)
-    request            = try(var.health_check_config.check.request, null)
-    response           = try(var.health_check_config.check.response, null)
-  }
-
-  dynamic "log_config" {
-    for_each = try(var.health_check_config.logging, false) ? [""] : []
-    content {
-      enable = true
-    }
-  }
-}
-
-resource "google_compute_health_check" "http2" {
-  provider = google-beta
-  count = (
-    var.health_check == null && local.health_check_type == "http2" ? 1 : 0
-  )
-  project     = var.project_id
-  name        = var.name
-  description = "Terraform managed."
-
-  check_interval_sec  = try(var.health_check_config.config.check_interval_sec, null)
-  healthy_threshold   = try(var.health_check_config.config.healthy_threshold, null)
-  timeout_sec         = try(var.health_check_config.config.timeout_sec, null)
-  unhealthy_threshold = try(var.health_check_config.config.unhealthy_threshold, null)
-
-  http2_health_check {
-    host               = try(var.health_check_config.check.host, null)
-    port               = try(var.health_check_config.check.port, null)
-    port_name          = try(var.health_check_config.check.port_name, null)
-    port_specification = try(var.health_check_config.check.port_specification, null)
-    proxy_header       = try(var.health_check_config.check.proxy_header, null)
-    request_path       = try(var.health_check_config.check.request_path, null)
-    response           = try(var.health_check_config.check.response, null)
-  }
-
-  dynamic "log_config" {
-    for_each = try(var.health_check_config.logging, false) ? [""] : []
-    content {
-      enable = true
-    }
-  }
-}
-