zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Factories refactor (#1843) 

* factories refactor doc

* Adds file schema and filesystem organization

* Update 20231106-factories.md

* move factories out of blueprints and create new factories  README

* align factory in billing-account module

* align factory in dataplex-datascan module

* align factory in billing-account module

* align factory in net-firewall-policy module

* align factory in dns-response-policy module

* align factory in net-vpc-firewall module

* align factory in net-vpc module

* align factory variable names in FAST

* remove decentralized firewall blueprint

* bump terraform version

* bump module versions

* update top-level READMEs

* move project factory to modules

* fix variable names and tests

* tfdoc

* remove changelog link

* add project factory to top-level README

* fix cludrun eventarc diff

* fix README

* fix cludrun eventarc diff

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
This commit is contained in:
Ludovico Magnocavallo 2024-02-26 11:16:52 +01:00 committed by GitHub
parent 8e86f0e108
commit 6941313c7d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
188 changed files with 917 additions and 2292 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/linting.yml vendored
View File

@ -34,7 +34,7 @@ jobs:
- name: Set up Terraform
uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.7.0
terraform_version: 1.7.4
- name: Install dependencies
run: |

2
.github/workflows/tests.yml vendored
View File

@ -25,7 +25,7 @@ env:
PYTEST_ADDOPTS: "--color=yes"
PYTHON_VERSION: "3.10"
TF_PLUGIN_CACHE_DIR: "/home/runner/.terraform.d/plugin-cache"
TF_VERSION: 1.7.0
TF_VERSION: 1.7.4
TFTEST_COPY: 1
jobs:

2
CHANGELOG.md
View File

@ -1653,7 +1653,7 @@ All notable changes to this project will be documented in this file.
- add support for VPC-SC perimeters in Data Foundation end to end example
- fix `vpc-sc` module
- new networking example showing how to use [Private Service Connect to call a Cloud Function from on-premises](./blueprints/networking/private-cloud-function-from-onprem/)
- new networking example showing how to organize [decentralized firewall](./blueprints/networking/decentralized-firewall/) management on GCP
- new networking example showing how to organize decentralized firewall management on GCP
## [5.0.0] - 2021-06-17

1
README.md
View File

@ -30,6 +30,7 @@ The current list of modules supports most of the core foundational and networkin
Currently available modules:
- **foundational** - [billing account](./modules/billing-account), [Cloud Identity group](./modules/cloud-identity-group/), [folder](./modules/folder), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [organization](./modules/organization), [project](./modules/project), [projects-data-source](./modules/projects-data-source)
- **process factories** - [project factory](./modules/project-factory/README.md)
- **networking** - [DNS](./modules/dns), [DNS Response Policy](./modules/dns-response-policy/), [Cloud Endpoints](./modules/endpoints), [address reservation](./modules/net-address), [NAT](./modules/net-cloudnat), [VLAN Attachment](./modules/net-vlan-attachment/), [External Application LB](./modules/net-lb-app-ext/), [External Passthrough Network LB](./modules/net-lb-ext), [External Regional Application Load Balancer](./modules/net-lb-app-ext-regional/), [Firewall policy](./modules/net-firewall-policy), [Internal Application LB](./modules/net-lb-app-int), [Cross-region Internal Application LB](./modules/net-lb-app-int-cross-region), [Internal Passthrough Network LB](./modules/net-lb-int), [Internal Proxy Network LB](./modules/net-lb-proxy-int), [IPSec over Interconnect](./modules/net-ipsec-over-interconnect), [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC peering](./modules/net-vpc-peering), [VPN dynamic](./modules/net-vpn-dynamic), [HA VPN](./modules/net-vpn-ha), [VPN static](./modules/net-vpn-static), [Service Directory](./modules/service-directory), [Secure Web Proxy](./modules/net-swp)
- **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid), [GKE cluster](./modules/gke-cluster-standard), [GKE hub](./modules/gke-hub), [GKE nodepool](./modules/gke-nodepool), [GCVE private cloud](./modules/gcve-private-cloud)
- **data** - <!-- [AlloyDB instance](./modules/alloydb-instance), --> [Analytics Hub](./modules/analytics-hub), [BigQuery dataset](./modules/bigquery-dataset), [Bigtable instance](./modules/bigtable-instance), [Dataplex](./modules/dataplex), [Dataplex DataScan](./modules/dataplex-datascan/), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag), [Data Catalog Tag](./modules/data-catalog-tag), [Data Catalog Tag Template](./modules/data-catalog-tag-template), [Datafusion](./modules/datafusion), [Dataproc](./modules/dataproc), [GCS](./modules/gcs), [Pub/Sub](./modules/pubsub), [Dataform Repository](./modules/dataform-repository/)

4
blueprints/README.md
View File

@ -7,9 +7,9 @@ Currently available blueprints:
- **apigee** - [Apigee Hybrid on GKE](./apigee/hybrid-gke/), [Apigee X analytics in BigQuery](./apigee/bigquery-analytics), [Apigee network patterns](./apigee/network-patterns/)
- **cloud operations** - [Active Directory Federation Services](./cloud-operations/adfs), [Cloud Asset Inventory feeds for resource change tracking and remediation](./cloud-operations/asset-inventory-feed-remediation), [Fine-grained Cloud DNS IAM via Service Directory](./cloud-operations/dns-fine-grained-iam), [Cloud DNS & Shared VPC design](./cloud-operations/dns-shared-vpc), [Delegated Role Grants](./cloud-operations/iam-delegated-role-grants), [Network Quota Monitoring](./cloud-operations/network-quota-monitoring), [Managing on-prem service account keys by uploading public keys](./cloud-operations/onprem-sa-key-management), [Compute Image builder with Hashicorp Packer](./cloud-operations/packer-image-builder), [Packer example](./cloud-operations/packer-image-builder/packer), [Compute Engine quota monitoring](./cloud-operations/compute-quota-monitoring), [Scheduled Cloud Asset Inventory Export to Bigquery](./cloud-operations/scheduled-asset-inventory-export-bq), [Configuring workload identity federation with Terraform Cloud/Enterprise workflows](./cloud-operations/terraform-cloud-dynamic-credentials), [TCP healthcheck and restart for unmanaged GCE instances](./cloud-operations/unmanaged-instances-healthcheck), [Migrate for Compute Engine (v5) blueprints](./cloud-operations/vm-migration), [Configuring workload identity federation to access Google Cloud resources from apps running on Azure](./cloud-operations/workload-identity-federation)
- **data solutions** - [GCE and GCS CMEK via centralized Cloud KMS](./data-solutions/cmek-via-centralized-kms), [Cloud Composer version 2 private instance, supporting Shared VPC and external CMEK key](./data-solutions/composer-2), [Cloud SQL instance with multi-region read replicas](./data-solutions/cloudsql-multiregion), [Data Platform](./data-solutions/data-platform-foundations), [Minimal Data Platform](./data-solutions/data-platform-minimal), [Spinning up a foundation data pipeline on Google Cloud using Cloud Storage, Dataflow and BigQuery](./data-solutions/gcs-to-bq-with-least-privileges), [#SQL Server Always On Groups blueprint](./data-solutions/sqlserver-alwayson), [Data Playground](./data-solutions/data-playground), [MLOps with Vertex AI](./data-solutions/vertex-mlops), [Shielded Folder](./data-solutions/shielded-folder), [BigQuery ML and Vertex AI Pipeline](./data-solutions/bq-ml)
- **factories** - [The why and the how of Resource Factories](./factories), [Google Cloud Identity Group Factory](./factories/cloud-identity-group-factory), [Google Cloud BQ Factory](./factories/bigquery-factory), [Google Cloud VPC Firewall Factory](./factories/net-vpc-firewall-yaml), [Minimal Project Factory](./factories/project-factory)
- **factories** - [Fabric resource factories](./factories)
- **GKE** - [Binary Authorization Pipeline Blueprint](./gke/binauthz), [Storage API](./gke/binauthz/image), [Multi-cluster mesh on GKE (fleet API)](./gke/multi-cluster-mesh-gke-fleet-api), [GKE Multitenant Blueprint](./gke/multitenant-fleet), [Shared VPC with GKE support](./networking/shared-vpc-gke/), [GKE Autopilot](./gke/autopilot)
- **networking** - [Calling a private Cloud Function from On-premises](./networking/private-cloud-function-from-onprem), [Decentralized firewall management](./networking/decentralized-firewall), [Decentralized firewall validator](./networking/decentralized-firewall/validator), [HA VPN over Interconnect](./networking/ha-vpn-over-interconnect/), [GLB and multi-regional daisy-chaining through hybrid NEGs](./networking/glb-hybrid-neg-internal), [Hybrid connectivity to on-premise services through PSC](./networking/psc-hybrid), [HTTP Load Balancer with Cloud Armor](./networking/glb-and-armor), [Hub and Spoke via VPN](./networking/hub-and-spoke-vpn), [Hub and Spoke via VPC Peering](./networking/hub-and-spoke-peering), [Internal Load Balancer as Next Hop](./networking/ilb-next-hop), On-prem DNS and Google Private Access, [PSC Producer](./networking/psc-hybrid/psc-producer), [PSC Consumer](./networking/psc-hybrid/psc-consumer), [Shared VPC with optional GKE cluster](./networking/shared-vpc-gke), [VPC Connectivity Lab](./networking/vpc-connectivity-lab/)
- **networking** - [Calling a private Cloud Function from On-premises](./networking/private-cloud-function-from-onprem), [HA VPN over Interconnect](./networking/ha-vpn-over-interconnect/), [GLB and multi-regional daisy-chaining through hybrid NEGs](./networking/glb-hybrid-neg-internal), [Hybrid connectivity to on-premise services through PSC](./networking/psc-hybrid), [HTTP Load Balancer with Cloud Armor](./networking/glb-and-armor), [Hub and Spoke via VPN](./networking/hub-and-spoke-vpn), [Hub and Spoke via VPC Peering](./networking/hub-and-spoke-peering), [Internal Load Balancer as Next Hop](./networking/ilb-next-hop), On-prem DNS and Google Private Access, [PSC Producer](./networking/psc-hybrid/psc-producer), [PSC Consumer](./networking/psc-hybrid/psc-consumer), [Shared VPC with optional GKE cluster](./networking/shared-vpc-gke), [VPC Connectivity Lab](./networking/vpc-connectivity-lab/)
- **serverless** - [Cloud Run series](./serverless/cloud-run-explore)
- **third party solutions** - [OpenShift on GCP user-provisioned infrastructure](./third-party-solutions/openshift), [Wordpress deployment on Cloud Run](./third-party-solutions/wordpress/cloudrun)

2
blueprints/data-solutions/shielded-folder/main.tf
View File

@ -94,7 +94,7 @@ module "firewall-policy" {
source = "../../../modules/net-firewall-policy"
name = "default"
parent_id = module.folder.id
rules_factory_config = var.data_dir == null ? {} : {
factories_config = var.data_dir == null ? {} : {
cidr_file_path = "${var.data_dir}/firewall-policies/cidrs.yaml"
ingress_rules_file_path = "${var.data_dir}/firewall-policies/hierarchical-ingress-rules.yaml"
}

99
blueprints/factories/README.md
View File

@ -1,44 +1,79 @@
# The why and the how of Resource Factories
# Resource Factories
Terraform modules can be designed - where it makes sense - to implement a resource factory, which is a configuration-driven approach to resource creation meant to:
This README explains the rationale and high level approach for resource factories, a pattern that is widely used in this repository across modules and in the FAST framework. It also collects pointers to all the different factories implemented in modules to simplify discovery.
- accelerate and rationalize the repetitive creation of common resources, such as firewall rules and subnets
- enable teams without Terraform specific knowledge to leverage IaC via human-friendly and machine-parseable YAML files
- make it simple to implement specific requirements and best practices (e.g. "always enable PGA for GCP subnets", or "only allow using regions `europe-west1` and `europe-west3`")
- codify and centralise business logics and policies (e.g. labels and naming conventions)
- allow to easily parse and understand sets of specific resources, for documentation purposes
<!-- BEGIN TOC -->
- [The why](#the-why)
- [The how](#the-how)
- [Factory implementations](#factory-implementations)
- [Module-level factory interfaces](#module-level-factory-interfaces)
- [Standalone factories](#standalone-factories)
<!-- END TOC -->
Generally speaking, the configurations for a resource factory consists in one or more YaML files, optionally grouped in folders, that describe resources following a well defined, validable schema, such as in the example below for the subnet factory of the [`net-vpc`](../../modules/net-vpc) module, which allows for the massive creation of subnets for a given VPC.
## The why
```yaml
region: europe-west3
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-prod-a, vpc-alpha
secondary_ip_ranges:
secondary-range-a: 192.168.0.0/24
secondary-range-b: 192.168.1.0/24
```
Managing large sets of uniform resources with Terraform usually involves different teams collaborating on the same codebase, complex authorization processes and checks managed via CI/CD approvals, or even integrating with external systems that manage digital workflows.
Terraform natively supports YaML, JSON and CSV parsing - however Fabric has decided to embrace YaML for the following reasons:
Factories are a way to simplify all above use cases, by moving repetitive resource definitions out of the Terraform codebase and into sets of files that leverage different formats.
- YaML is easier to parse for a human, and allows for comments and nested, complex structures
- JSON and CSV can't include comments, which can be used to document configurations, but are often useful to bridge from other systems in automated pipelines
- JSON is more verbose (reads: longer) and harder to parse visually for humans
- CSV isn't often expressive enough (e.g. dit doesn't allow for nested structures)
Using factories, repetive resource creation and management becomes easier
If needed, converting factories to consume JSON is a matter of switching from `yamldecode()` to `jsondecode()` in the right place on each module.
- for humans who have no direct experience with Terraform, by exposing filesystem hierarchies and YAML-based configuration data
- for connected systems, by accepting well know data exchange formats like JSON or CSV
- for external code that needs to enforce checks or policies, by eliminating the need to parse HCL code or Terraform outputs
- to implement authorization processes or workwflows in CI/CD, by removing the dependency on Terraform and HCL knowledge for the teams involved
## Resource factories in Fabric
## The how
### Fabric Modules
Fabric resource-level factories can be broadly split into two different types
- [folder](../../modules/folder/README.md#firewall-policy-factory) and [organization](../../modules/organization/README.md#firewall-policy-factory) implement factories for [hierarchical firewall policies](https://cloud.google.com/vpc/docs/firewall-policies)
- [net-vpc](../../modules/net-vpc/README.md#subnet-factory) for subnets creation
- [net-vpc-firewall](../../modules/net-vpc-firewall/README.md#rules-factory) for massive rules creation
- simple factories that manage one simple resource type (firewalls, VPC-SC policies)
- complex factories that manage a set of connected resources to implement a complex flow that is usually perceived as a single unit (project creation)
### Dedicated Factories
The first factory type is implemented at the module level, where one module exposes one or more factories for some of the resources that depend on the main module resource (e.g. firewall rules for a VPC). The main goal with this approach is to simplify resource management at scale by removing the dependency on Terraform and HCL.
- [cloud-identity-group-factory](cloud-identity-group-factory/README.md) for Cloud Identity group
- [net-vpc-firewall-yaml](net-vpc-firewall-yaml/README.md) for VPC firewall rules across different projects/VPCs
- [project-factory](project-factory/README.md) for projects
These factories are often designed as module-level interfaces which are then exposed by any module that manages a specific type of resource. All these factories leverage a single `factory_configs` variable, that allows passing in the paths for all the different factories supported in the module.
The second factory type is implemented as a standalone module that internally references other modules, and implements complex management of different resource sets as part of a single process implemented via the factory. The typical example is the project factory, that brings together the project, service accounts, and billing accounts modules to cover all the main aspects of project creation as a single unit.
## Factory implementations
### Module-level factory interfaces
- **BigQuery Analicts Hub rules**
- `analytics-hub`
- **billing budgets**
- `billing-account`
- **Data Catalog tags**
- `data-catalog-tag`
- **Data Catalog tag templates**
- `data-catalog-tag-template`
- **Dataplex Datascan rules**
- `dataplex-datascan`
- **firewall policy rules**
- `net-firewall-policy`
- **hierarchical firewall policies**
- `folder`
- `project`
- **IAM custom roles**
- `organization`
- `project`
- **organization policies**
- `organization`
- `folder`
- `project`
- **organization policy custom constraints**
- `organization`
- **DNS response policy rules**
- `dns-response-policy`
- **VPC firewall rules**
- `net-vpc-firewall`
- **VPC subnets**
- `net-vpc`
- **VPC-SC access levels and policies**
- `vpc-sc`
### Standalone factories
- **projects**
- `project-factory`

201
blueprints/factories/net-vpc-firewall-yaml/README.md
View File

@ -1,201 +0,0 @@
# Google Cloud VPC Firewall Factory
This module allows creation and management of different types of firewall rules by defining them in well formatted `yaml` files.
Yaml abstraction for FW rules can simplify users onboarding and also makes rules definition simpler and clearer comparing to HCL.
Nested folder structure for yaml configurations is optionally supported, which allows better and structured code management for multiple teams and environments.
## Example
### Terraform code
```hcl
module "prod-firewall" {
source = "./fabric/blueprints/factories/net-vpc-firewall-yaml"
project_id = "my-prod-project"
network = "my-prod-network"
config_directories = [
"./firewall/prod",
"./firewall/common"
]
log_config = {
metadata = "INCLUDE_ALL_METADATA"
}
}
module "dev-firewall" {
source = "./fabric/blueprints/factories/net-vpc-firewall-yaml"
project_id = "my-dev-project"
network = "my-dev-network"
config_directories = [
"./firewall/dev",
"./firewall/common"
]
}
# tftest modules=2 resources=16 files=common,dev,prod inventory=example.yaml
```
```yaml
# tftest-file id=common path=firewall/common/common.yaml
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
# allow ingress from GCLB to all instances in the network
lb-health-checks:
allow:
- ports: []
protocol: tcp
direction: INGRESS
priority: 1001
source_ranges:
- 35.191.0.0/16
- 130.211.0.0/22
# deny all egress
deny-all:
deny:
- ports: []
protocol: all
direction: EGRESS
priority: 65535
destination_ranges:
- 0.0.0.0/0
```
```yaml
# tftest-file id=dev path=firewall/dev/app.yaml
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
# Myapp egress
web-app-dev-egress:
allow:
- ports: [443]
protocol: tcp
direction: EGRESS
destination_ranges:
- 192.168.0.0/24
target_service_accounts:
- myapp@myproject-dev.iam.gserviceaccount.com
# Myapp ingress
web-app-dev-ingress:
allow:
- ports: [1234]
protocol: tcp
direction: INGRESS
source_service_accounts:
- frontend-sa@myproject-dev.iam.gserviceaccount.com
target_service_accounts:
- web-app-a@myproject-dev.iam.gserviceaccount.com
```
```yaml
# tftest-file id=prod path=firewall/prod/app.yaml
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
# Myapp egress
web-app-prod-egress:
allow:
- ports: [443]
protocol: tcp
direction: EGRESS
destination_ranges:
- 192.168.10.0/24
target_service_accounts:
- myapp@myproject-prod.iam.gserviceaccount.com
# Myapp ingress
web-app-prod-ingress:
allow:
- ports: [1234]
protocol: tcp
direction: INGRESS
source_service_accounts:
- frontend-sa@myproject-prod.iam.gserviceaccount.com
target_service_accounts:
- web-app-a@myproject-prod.iam.gserviceaccount.com
```
### Configuration Structure
```bash
├── common
│ ├── default-egress.yaml
│   ├── lb-rules.yaml
│   └── iap-ingress.yaml
├── dev
│   ├── team-a
│   │   ├── databases.yaml
│   │   └── webb-app-a.yaml
│   └── team-b
│   ├── backend.yaml
│   └── frontend.yaml
└── prod
├── team-a
│   ├── databases.yaml
│   └── webb-app-a.yaml
└── team-b
├── backend.yaml
└── frontend.yaml
```
### Rule definition format and structure
Firewall rules configuration should be placed in a set of yaml files in a folder/s. Firewall rule entry structure is following:
```yaml
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
rule-name: # descriptive name, naming convention is adjusted by the module
allow: # `allow` or `deny`
- ports: ['443', '80'] # ports for a specific protocol, keep empty list `[]` for all ports
protocol: tcp # protocol, put `all` for any protocol
direction: EGRESS # EGRESS or INGRESS
disabled: false # `false` or `true`, FW rule is disabled when `true`, default value is `false`
priority: 1000 # rule priority value, default value is 1000
source_ranges: # list of source ranges, should be specified only for `INGRESS` rule
- 0.0.0.0/0
destination_ranges: # list of destination ranges, should be specified only for `EGRESS` rule
- 0.0.0.0/0
source_tags: ['some-tag'] # list of source tags, should be specified only for `INGRESS` rule
source_service_accounts: # list of source service accounts, should be specified only for `INGRESS` rule, cannot be specified together with `source_tags` or `target_tags`
- myapp@myproject-id.iam.gserviceaccount.com
target_tags: ['some-tag'] # list of target tags
target_service_accounts: # list of target service accounts, , cannot be specified together with `source_tags` or `target_tags`
- myapp@myproject-id.iam.gserviceaccount.com
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [config_directories](variables.tf#L17) | List of paths to folders where firewall configs are stored in yaml format. Folder may include subfolders with configuration files. Files suffix must be `.yaml`. | <code>list&#40;string&#41;</code> | ✓ | |
| [network](variables.tf#L30) | Name of the network this set of firewall rules applies to. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L35) | Project Id. | <code>string</code> | ✓ | |
| [log_config](variables.tf#L22) | Log configuration. Possible values for `metadata` are `EXCLUDE_ALL_METADATA` and `INCLUDE_ALL_METADATA`. Set to `null` for disabling firewall logging. | <code title="object&#40;&#123;&#10; metadata &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| [egress_allow_rules](outputs.tf#L17) | Egress rules with allow blocks. | |
| [egress_deny_rules](outputs.tf#L25) | Egress rules with allow blocks. | |
| [ingress_allow_rules](outputs.tf#L33) | Ingress rules with allow blocks. | |
| [ingress_deny_rules](outputs.tf#L41) | Ingress rules with deny blocks. | |
<!-- END TFDOC -->

113
blueprints/factories/net-vpc-firewall-yaml/main.tf
View File

@ -1,113 +0,0 @@
/**
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
firewall_rule_files = flatten(
[
for config_path in var.config_directories :
concat(
[
for config_file in fileset(config_path, "**/*.yaml") :
"${config_path}/${config_file}"
]
)
]
)
firewall_rules = merge(
[
for config_file in local.firewall_rule_files :
yamldecode(file(config_file))
]...
)
}
resource "time_static" "timestamp" {
for_each = local.firewall_rules
triggers = {
name = md5(jsonencode(each.value))
}
}
resource "google_compute_firewall" "rules" {
for_each = local.firewall_rules
project = var.project_id
name = format(
"fwr-%s-%s-%s-%s",
var.network,
(try(each.value.target_service_accounts, null) != null ? "sac" : try(each.value.target_tags, null) != null ? "vpc" : "all"),
substr(lower(each.value.direction), 0, 1),
each.key
)
description = format(
"%s rule in network %s for %s created at %s",
each.value.direction,
var.network,
each.key,
time_static.timestamp[each.key].rfc3339
)
network = var.network
direction = each.value.direction
priority = try(each.value.priority, 1000)
disabled = try(each.value.disabled, null)
source_ranges = try(each.value.source_ranges, each.value.direction == "INGRESS" ? [] : null)
source_tags = try(each.value.source_tags, null)
source_service_accounts = try(each.value.source_service_accounts, null)
destination_ranges = try(each.value.destination_ranges, each.value.direction == "EGRESS" ? [] : null)
target_tags = try(each.value.target_tags, null)
target_service_accounts = try(each.value.target_service_accounts, null)
dynamic "allow" {
for_each = { for block in try(each.value.allow, []) :
"${block.protocol}-${join("-", block.ports)}" => {
ports = [for port in block.ports : tostring(port)]
protocol = block.protocol
}
}
content {
protocol = allow.value.protocol
ports = allow.value.ports
}
}
dynamic "deny" {
for_each = { for block in try(each.value.deny, []) :
"${block.protocol}-${join("-", block.ports)}" => {
ports = [for port in block.ports : tostring(port)]
protocol = block.protocol
}
}
content {
protocol = deny.value.protocol
ports = deny.value.ports
}
}
dynamic "log_config" {
for_each = var.log_config != null ? [""] : []
content {
metadata = var.log_config.metadata
}
}
lifecycle {
create_before_destroy = true
}
}

47
blueprints/factories/net-vpc-firewall-yaml/outputs.tf
View File

@ -1,47 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "egress_allow_rules" {
description = "Egress rules with allow blocks."
value = [
for rule in google_compute_firewall.rules :
rule if rule.direction == "EGRESS" && length(rule.allow) > 0
]
}
output "egress_deny_rules" {
description = "Egress rules with allow blocks."
value = [
for rule in google_compute_firewall.rules :
rule if rule.direction == "EGRESS" && length(rule.deny) > 0
]
}
output "ingress_allow_rules" {
description = "Ingress rules with allow blocks."
value = [
for rule in google_compute_firewall.rules :
rule if rule.direction == "INGRESS" && length(rule.allow) > 0
]
}
output "ingress_deny_rules" {
description = "Ingress rules with deny blocks."
value = [
for rule in google_compute_firewall.rules :
rule if rule.direction == "INGRESS" && length(rule.deny) > 0
]
}

38
blueprints/factories/net-vpc-firewall-yaml/variables.tf
View File

@ -1,38 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "config_directories" {
description = "List of paths to folders where firewall configs are stored in yaml format. Folder may include subfolders with configuration files. Files suffix must be `.yaml`."
type = list(string)
}
variable "log_config" {
description = "Log configuration. Possible values for `metadata` are `EXCLUDE_ALL_METADATA` and `INCLUDE_ALL_METADATA`. Set to `null` for disabling firewall logging."
type = object({
metadata = string
})
default = null
}
variable "network" {
description = "Name of the network this set of firewall rules applies to."
type = string
}
variable "project_id" {
description = "Project Id."
type = string
}

6
blueprints/gke/patterns/autopilot-cluster/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
blueprints/gke/patterns/batch/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
blueprints/gke/patterns/kafka/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
blueprints/gke/patterns/mysql/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
blueprints/gke/patterns/redis-cluster/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
blueprints/networking/README.md
View File

@ -18,12 +18,6 @@ They are meant to be used as minimal but complete starting points to create actu
<br clear="left">
### Decentralized firewall management
<a href="./decentralized-firewall/" title="Decentralized firewall management"><img src="./decentralized-firewall/diagram.png" align="left" width="280px"></a> This [blueprint](./decentralized-firewall/) shows how a decentralized firewall management can be organized using the [firewall factory](../factories/net-vpc-firewall-yaml/).
<br clear="left">
### GLB and multi-regional daisy-chaining through hybrid NEGs
<a href="./glb-hybrid-neg-internal/" title="XGLB and multi-regional daisy-chaining through hybrid NEGs"><img src="./glb-hybrid-neg-internal/diagram.png" align="left" width="280px"></a> This [blueprint](./glb-hybrid-neg-internal/) shows the experimental use of hybrid NEGs behind external Global Load Balancers (GLBs) to connect to GCP instances living in spoke VPCs and behind Network Virtual Appliances (NVAs).

55
blueprints/networking/decentralized-firewall/README.md
View File

@ -1,55 +0,0 @@
# Decentralized firewall management
This example shows how a decentralized firewall management can be organized using the [firewall factory](../../factories/net-vpc-firewall-yaml/README.md).
This approach is a good fit when Shared VPCs are used across multiple application/infrastructure teams. A central repository keeps environment/team
specific folders with firewall definitions in `yaml` format.
In the current blueprint multiple teams can define their [VPC Firewall Rules](https://cloud.google.com/vpc/docs/firewalls)
for [dev](./firewall/dev) and [prod](./firewall/prod) environments using team specific subfolders. Rules defined in the
[common](./firewall/common) folder are applied to both dev and prod environments.
> **_NOTE:_** Common rules are meant to be used for situations where [hierarchical rules](https://cloud.google.com/vpc/docs/firewall-policies)
do not map precisely to requirements (e.g. SA, etc.)
This is the high level diagram:
![High-level diagram](diagram.png "High-level diagram")
The rules can be validated either using an automated process or a manual process (or a combination of
the two). There is an blueprint of a YAML-based validator using [Yamale](https://github.com/23andMe/Yamale)
in the [`validator/`](validator/) subdirectory, which can be integrated as part of a CI/CD pipeline.
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [billing_account_id](variables.tf#L15) | Billing account id used as default for new projects. | <code>string</code> | ✓ | |
| [prefix](variables.tf#L29) | Prefix used for resource names. | <code>string</code> | ✓ | |
| [root_node](variables.tf#L54) | Hierarchy node where projects will be created, 'organizations/org_id' or 'folders/folder_id'. | <code>string</code> | ✓ | |
| [ip_ranges](variables.tf#L20) | Subnet IP CIDR ranges. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; prod &#61; &#34;10.0.16.0&#47;24&#34;&#10; dev &#61; &#34;10.0.32.0&#47;24&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
| [project_services](variables.tf#L38) | Service APIs enabled by default in new projects. | <code>list&#40;string&#41;</code> | | <code title="&#91;&#10; &#34;container.googleapis.com&#34;,&#10; &#34;dns.googleapis.com&#34;,&#10; &#34;stackdriver.googleapis.com&#34;,&#10;&#93;">&#91;&#8230;&#93;</code> |
| [region](variables.tf#L48) | Region used. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| [fw_rules](outputs.tf#L15) | Firewall rules. | |
| [projects](outputs.tf#L33) | Project ids. | |
| [vpc](outputs.tf#L41) | Shared VPCs. | |
<!-- END TFDOC -->
## Test
```hcl
module "test" {
source = "./fabric/blueprints/networking/decentralized-firewall"
billing_account_id = "ABCDE-12345-ABCDE"
prefix = "prefix"
root_node = "organizations/0123456789"
}
# tftest modules=9 resources=54
```

20
blueprints/networking/decentralized-firewall/backend.tf.sample
View File

@ -1,20 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
backend "gcs" {
bucket = ""
}
}

BIN
blueprints/networking/decentralized-firewall/diagram.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 290 KiB

47
blueprints/networking/decentralized-firewall/firewall/common/common-egress.yaml
View File

@ -1,47 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
# Deny all egress (egress traffic is allowed by default)
deny-all:
deny:
- ports: []
protocol: all
direction: EGRESS
priority: 65535
destination_ranges:
- 0.0.0.0/0
# Allow access to GCP APIs via Private Google Access
# https://cloud.google.com/vpc/docs/access-apis-external-ip#config
gcp-pga-apis:
allow:
- ports: [443]
protocol: tcp
direction: EGRESS
priority: 500
destination_ranges:
- 199.36.153.8/30
# Allow egress to internal networks
internal-egress:
allow:
- ports: []
protocol: tcp
direction: EGRESS
destination_ranges:
- 10.0.0.0/16

27
blueprints/networking/decentralized-firewall/firewall/common/iap-access.yaml
View File

@ -1,27 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
# Access via SSH from IAP to all instancess https://cloud.google.com/iap/docs/using-tcp-forwarding#create-firewall-rule
iap-ssh-access:
allow:
- ports: [22]
protocol: tcp
direction: INGRESS
priority: 1001
source_ranges:
- 35.235.240.0/20

28
blueprints/networking/decentralized-firewall/firewall/common/lb-access.yaml
View File

@ -1,28 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
# Access from GCP LBs https://cloud.google.com/load-balancing/docs/https/#firewall_rules
lb-health-checks:
allow:
- ports: []
protocol: tcp
direction: INGRESS
priority: 1001
source_ranges:
- 35.191.0.0/16
- 130.211.0.0/22

35
blueprints/networking/decentralized-firewall/firewall/dev/app-1/app1-rules.yaml
View File

@ -1,35 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
# Allow traffic from the frontend VMs
app1-backend:
allow:
- ports: ["443", "80"]
protocol: tcp
direction: INGRESS
source_tags: ["app1-frontend"]
target_tags: ["app1-backend"]
# Allow traffic to MySQL Servers from App1 backend
app1-db:
allow:
- ports: ["3306"]
protocol: tcp
direction: INGRESS
source_tags: ["app1-backend"]
target_tags: ["mysql-server"]

35
blueprints/networking/decentralized-firewall/firewall/dev/app-2/app2-rules.yaml
View File

@ -1,35 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
# Allow traffic from app1 frontend
app2-backend:
allow:
- ports: ["443", "80"]
protocol: tcp
direction: INGRESS
source_tags: ["app1-frontend"]
target_tags: ["app2-backend"]
# Allow traffic to MySQL servers from App2 backend
app2-db:
allow:
- ports: ["3306"]
protocol: tcp
direction: INGRESS
source_tags: ["app2-backend"]
target_tags: ["mysql-server"]

35
blueprints/networking/decentralized-firewall/firewall/prod/app-1/app1-rules.yaml
View File

@ -1,35 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
# Terraform will be unable to decode this file if it does not contain valid YAML
# You can retain `---` (start of the document) to indicate an empty document.
# Allow traffic from the frontend VMs
app1-backend:
allow:
- ports: ["443", "80"]
protocol: tcp
direction: INGRESS
source_tags: ["app1-frontend"]
target_tags: ["app1-backend"]
# Allow traffic to MySQL Servers from App1 backend
app1-db:
allow:
- ports: ["3306"]
protocol: tcp
direction: INGRESS
source_tags: ["app1-backend"]
target_tags: ["mysql-server"]

138
blueprints/networking/decentralized-firewall/main.tf
View File

@ -1,138 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
###############################################################################
# Shared VPC Host projects #
###############################################################################
module "project-host-prod" {
source = "../../../modules/project"
parent = var.root_node
billing_account = var.billing_account_id
prefix = var.prefix
name = "prod-host"
services = var.project_services
shared_vpc_host_config = {
enabled = true
}
}
module "project-host-dev" {
source = "../../../modules/project"
parent = var.root_node
billing_account = var.billing_account_id
prefix = var.prefix
name = "dev-host"
services = var.project_services
shared_vpc_host_config = {
enabled = true
}
}
################################################################################
# Networking #
################################################################################
module "vpc-prod" {
source = "../../../modules/net-vpc"
project_id = module.project-host-prod.project_id
name = "prod-vpc"
subnets = [
{
ip_cidr_range = var.ip_ranges.prod
name = "prod"
region = var.region
}
]
}
module "vpc-dev" {
source = "../../../modules/net-vpc"
project_id = module.project-host-dev.project_id
name = "dev-vpc"
subnets = [
{
ip_cidr_range = var.ip_ranges.dev
name = "dev"
region = var.region
}
]
}
###############################################################################
# Private Google Access DNS #
###############################################################################
module "dns-api-prod" {
source = "../../../modules/dns"
project_id = module.project-host-prod.project_id
name = "googleapis"
zone_config = {
domain = "googleapis.com."
private = {
client_networks = [module.vpc-prod.self_link]
}
}
recordsets = {
"CNAME *" = { records = ["private.googleapis.com."] }
}
}
module "dns-api-dev" {
source = "../../../modules/dns"
project_id = module.project-host-dev.project_id
name = "googleapis"
zone_config = {
domain = "googleapis.com."
private = {
client_networks = [module.vpc-dev.self_link]
}
}
recordsets = {
"CNAME *" = { records = ["private.googleapis.com."] }
}
}
###############################################################################
# Distributed Firewall #
###############################################################################
module "vpc-firewall-prod" {
source = "../../factories/net-vpc-firewall-yaml"
project_id = module.project-host-prod.project_id
network = module.vpc-prod.name
config_directories = [
"${path.module}/firewall/common",
"${path.module}/firewall/prod"
]
# Enable Firewall Logging for the production fwl rules
log_config = {
metadata = "INCLUDE_ALL_METADATA"
}
}
module "vpc-firewall-dev" {
source = "../../factories/net-vpc-firewall-yaml"
project_id = module.project-host-dev.project_id
network = module.vpc-dev.name
config_directories = [
"${path.module}/firewall/common",
"${path.module}/firewall/dev"
]
}

53
blueprints/networking/decentralized-firewall/outputs.tf
View File

@ -1,53 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
output "fw_rules" {
description = "Firewall rules."
value = {
prod = {
ingress_allow_rules = module.vpc-firewall-prod.ingress_allow_rules
ingress_deny_rules = module.vpc-firewall-prod.ingress_deny_rules
egress_allow_rules = module.vpc-firewall-prod.egress_allow_rules
egress_deny_rules = module.vpc-firewall-prod.egress_deny_rules
}
dev = {
ingress_allow_rules = module.vpc-firewall-dev.ingress_allow_rules
ingress_deny_rules = module.vpc-firewall-dev.ingress_deny_rules
egress_allow_rules = module.vpc-firewall-dev.egress_allow_rules
egress_deny_rules = module.vpc-firewall-dev.egress_deny_rules
}
}
}
output "projects" {
description = "Project ids."
value = {
prod-host = module.project-host-prod.project_id
dev-host = module.project-host-dev.project_id
}
}
output "vpc" {
description = "Shared VPCs."
value = {
prod = {
name = module.vpc-prod.name
subnets = module.vpc-prod.subnet_ips
}
dev = {
name = module.vpc-dev.name
subnets = module.vpc-dev.subnet_ips
}
}
}

29
blueprints/networking/decentralized-firewall/validator/Dockerfile
View File

@ -1,29 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM python:3.9-slim
RUN mkdir /validator
COPY requirements.txt /validator/requirements.txt
RUN pip install -r /validator/requirements.txt
COPY validator.py /validator/validator.py
RUN mkdir /schemas
COPY firewallSchema.yaml /schemas/firewallSchema.yaml
COPY firewallSchemaAutoApprove.yaml /schemas/firewallAutoApprove.yaml
COPY firewallSchemaSettings.yaml /schemas/firewallSchemaSettings.yaml
RUN mkdir /rules
CMD ["/rules/**/*.yaml"]
ENTRYPOINT ["python3", "/validator/validator.py"]

80
blueprints/networking/decentralized-firewall/validator/README.md
View File

@ -1,80 +0,0 @@
# Decentralized firewall validator
The decentralized firewall validator is a Python scripts that utilizes [Yamale](https://github.com/23andMe/Yamale) schema
validation library to validate the configured firewall rules.
## Configuring schemas
There are three configuration files:
- [firewallSchema.yaml](firewallSchema.yaml), where the basic validation schema is configured
- [firewallSchemaAutoApprove.yaml](firewallSchemaAutoApprove.yaml), where the a different schema for auto-approval
can be configured (in case more validation is required than what is available in the schema settings)
- [firewallSchemaSettings.yaml](firewallSchemaSettings.yaml), configures list of allowed and approved
source and destination ranges, ports, network tags and service accounts.
## Building the container
You can build the container like this:
```sh
docker build -t eu.gcr.io/YOUR-PROJECT/firewall-validator:latest .
docker push eu.gcr.io/YOUR-PROJECT/firewall-validator:latest
```
## Running the validator
Example:
```sh
docker run -v $(pwd)/firewall:/rules/ -t eu.gcr.io/YOUR-PROJECT/firewall-validator:latest
```
Output is JSON with keys `ok` and `errors` (if any were found).
## Using as a GitHub action
An `action.yml` is provided for this validator to be used as a GitHub action.
Example of being used in a pipeline:
```yaml
- uses: actions/checkout@v2
- name: Get changed files
if: ${{ github.event_name == 'pull_request' }}
id: changed-files
uses: tj-actions/changed-files@v1.1.2
- uses: ./.github/actions/validate-firewall
if: ${{ github.event_name == 'pull_request' }}
id: validation
with:
files: ${{ steps.changed-files.outputs.all_modified_files }}
- uses: actions/github-script@v3
if: ${{ github.event_name == 'pull_request' && steps.validation.outputs.ok != 'true' }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
var comments = [];
var errors = JSON.parse(process.env.ERRORS);
for (const filename in errors) {
var fn = filename.replace('/github/workspace/', '');
comments.push({
path: fn,
body: "```\n" + errors[filename].join("\n") + "\n```\n",
position: 1,
});
}
github.pulls.createReview({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number,
event: "REQUEST_CHANGES",
body: "Firewall rule validation failed.",
comments: comments,
});
core.setFailed("Firewall validation failed");
env:
ERRORS: '${{ steps.validation.outputs.errors }}'
```

44
blueprints/networking/decentralized-firewall/validator/action.yml
View File

@ -1,44 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
name: "Validate firewall rules"
description: "Validate firewall rule YAML files"
inputs:
files:
description: "Files to scan (supports wildcards)"
required: false
default: "/github/workspace/firewall/**/*.yaml"
mode:
description: "Mode (validate or approve)"
required: false
default: "validate"
schema:
description: "Schema"
required: false
default: "/schemas/firewallSchema.yaml"
outputs:
ok:
description: "Validation successful"
errors:
description: "Validation results"
runs:
using: "docker"
image: "Dockerfile"
args:
- ${{ inputs.files }}
- "--mode"
- ${{ inputs.mode }}
- "--schema"
- ${{ inputs.schema }}
- "--github"

32
blueprints/networking/decentralized-firewall/validator/firewallSchema.yaml
View File

@ -1,32 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
map(include('rule'), key=str(min=3, max=30))
---
rule:
disabled: bool(required=False)
deny: list(include('trafficSpec'), required=False)
allow: list(include('trafficSpec'), required=False)
direction: enum('ingress', 'INGRESS', 'egress', 'EGRESS')
priority: int(min=1, max=65535, required=False)
destination_ranges: list(netmask(type='destination'), max=256, required=False)
source_ranges: list(netmask(type='source'), max=256, required=False)
source_tags: list(networktag(), max=30, required=False)
target_tags: list(networktag(), max=70, required=False)
source_service_accounts: list(serviceaccount(), max=10, required=False)
target_service_accounts: list(serviceaccount(), max=10, required=False)
---
trafficSpec:
ports: list(networkports())
protocol: enum('all', 'tcp', 'udp', 'icmp', 'esp', 'ah', 'ipip', 'sctp')

42
blueprints/networking/decentralized-firewall/validator/firewallSchemaAutoApprove.yaml
View File

@ -1,42 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
map(include('ingress'), include('egress'), key=str(min=3, max=30))
---
ingress:
disabled: bool(required=False)
deny: list(include('trafficSpec'), required=False)
allow: list(include('trafficSpec'), required=False)
direction: enum('ingress', 'INGRESS')
priority: int(min=1, max=65535, required=False)
source_ranges: list(netmask(type='source'), max=256, required=False)
source_tags: list(networktag(), max=30, required=False)
target_tags: list(networktag(), max=70, required=False)
source_service_accounts: list(serviceaccount(), max=10, required=False)
target_service_accounts: list(serviceaccount(), max=10, required=False)
---
egress:
disabled: bool(required=False)
deny: list(include('trafficSpec'), required=False)
allow: list(include('trafficSpec'), required=False)
direction: enum('egress', 'EGRESS')
priority: int(min=1, max=65535, required=False)
destination_ranges: list(netmask(type='destination'), max=256, required=False)
source_tags: list(networktag(), max=30, required=False)
target_tags: list(networktag(), max=70, required=False)
source_service_accounts: list(serviceaccount(), max=10, required=False)
target_service_accounts: list(serviceaccount(), max=10, required=False)
---
trafficSpec:
ports: list()
protocol: enum('all', 'tcp', 'udp', 'icmp', 'esp', 'ah', 'ipip', 'sctp')

49
blueprints/networking/decentralized-firewall/validator/firewallSchemaSettings.yaml
View File

@ -1,49 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
allowedPorts:
- ports: 22 # SSH
approved: false
- ports: 80 # HTTP
approved: true
- ports: 443 # HTTPS
approved: true
- ports: 3306 # MySQL
approved: false
- ports: 8000-8999
approved: true
allowedSourceRanges:
- cidr: 10.0.0.0/8 # Example on-premise range
approved: true
- cidr: 35.191.0.0/16 # Load balancing & health checks
approved: true
- cidr: 130.211.0.0/22 # Load balancing & health checks
approved: false
- cidr: 35.235.240.0/20 # IAP source range
approved: true
allowedDestinationRanges:
- cidr: 10.0.0.0/8
approved: true
- cidr: 0.0.0.0/0
approved: false
allowedNetworkTags:
- tag: "*"
approved: true
allowedServiceAccounts:
- serviceAccount: "*"
approved: true

16
blueprints/networking/decentralized-firewall/validator/requirements.txt
View File

@ -1,16 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
yamale~=3.0.0
PyYAML~=5.4.0
click~=7.1.0

256
blueprints/networking/decentralized-firewall/validator/validator.py
View File

@ -1,256 +0,0 @@
#!/usr/bin/env python3
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import glob
import ipaddress
import json
import sys
import click
import yaml
import yamale
from fnmatch import fnmatch
from types import SimpleNamespace
from yamale.validators import DefaultValidators, Validator
class Netmask(Validator):
""" Custom netmask validator """
tag = 'netmask'
settings = {}
mode = None
_type = None
def __init__(self, *args, **kwargs):
self._type = kwargs.pop('type', 'source-or-dest')
super().__init__(*args, **kwargs)
def fail(self, value):
dir_str = 'source or destination'
mode_str = 'allowed'
if self._type == 'source':
dir_str = 'source'
elif self._type == 'destination':
dir_str = 'destination'
if self.mode == 'approve':
mode_str = 'automatically approved'
return '\'%s\' is not an %s %s network.' % (value, mode_str, dir_str)
def _is_valid(self, value):
is_ok = False
network = ipaddress.ip_network(value)
if self._type == 'source' or self._type == 'source-or-dest':
for ip_range in self.settings['allowedSourceRanges']:
allowed_network = ipaddress.ip_network(ip_range['cidr'])
if network.subnet_of(allowed_network):
if self.mode != 'approve' or ip_range['approved']:
is_ok = True
break
if self._type == 'destination' or self._type == 'source-or-dest':
for ip_range in self.settings['allowedDestinationRanges']:
allowed_network = ipaddress.ip_network(ip_range['cidr'])
if network.subnet_of(allowed_network):
if self.mode != 'approve' or ip_range['approved']:
is_ok = True
break
return is_ok
class NetworkTag(Validator):
""" Custom network tag validator """
tag = 'networktag'
settings = {}
mode = None
_type = None
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def fail(self, value):
mode_str = 'allowed'
if self.mode == 'approve':
mode_str = 'automatically approved'
return '\'%s\' is not an %s network tag.' % (value, mode_str)
def _is_valid(self, value):
is_ok = False
for tag in self.settings['allowedNetworkTags']:
if fnmatch(value, tag['tag']):
if self.mode != 'approve' or tag['approved']:
is_ok = True
break
return is_ok
class ServiceAccount(Validator):
""" Custom service account validator """
tag = 'serviceaccount'
settings = {}
mode = None
_type = None
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def fail(self, value):
mode_str = 'allowed'
if self.mode == 'approve':
mode_str = 'automatically approved'
return '\'%s\' is not an %s service account.' % (value, mode_str)
def _is_valid(self, value):
is_ok = False
for sa in self.settings['allowedServiceAccounts']:
if fnmatch(value, sa['serviceAccount']):
if self.mode != 'approve' or sa['approved']:
is_ok = True
break
return is_ok
class NetworkPorts(Validator):
""" Custom ports validator """
tag = 'networkports'
settings = {}
mode = None
_type = None
allowed_port_map = []
approved_port_map = []
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
for port in self.settings['allowedPorts']:
ports = self._process_port_definition(port['ports'])
self.allowed_port_map.extend(ports)
if port['approved']:
self.approved_port_map.extend(ports)
def _process_port_definition(self, port_definition):
ports = []
if not isinstance(port_definition, int) and '-' in port_definition:
start, end = port_definition.split('-', 2)
for port in range(int(start), int(end) + 1):
ports.append(int(port))
else:
ports.append(int(port_definition))
return ports
def fail(self, value):
mode_str = 'allowed'
if self.mode == 'approve':
mode_str = 'automatically approved'
return '\'%s\' is not an %s IP port.' % (value, mode_str)
def _is_valid(self, value):
ports = self._process_port_definition(value)
is_ok = True
for port in ports:
if self.mode == 'approve' and port not in self.approved_port_map:
is_ok = False
break
elif port not in self.allowed_port_map:
is_ok = False
break
return is_ok
class FirewallValidator:
schema = None
settings = None
validators = None
def __init__(self, settings, mode):
self.settings = settings
self.validators = DefaultValidators.copy()
Netmask.settings = self.settings
Netmask.mode = mode
self.validators[Netmask.tag] = Netmask
NetworkTag.settings = self.settings
NetworkTag.mode = mode
self.validators[NetworkTag.tag] = NetworkTag
ServiceAccount.settings = self.settings
ServiceAccount.mode = mode
self.validators[ServiceAccount.tag] = ServiceAccount
NetworkPorts.settings = self.settings
NetworkPorts.mode = mode
self.validators[NetworkPorts.tag] = NetworkPorts
def set_schema_from_file(self, schema):
self.schema = yamale.make_schema(path=schema, validators=self.validators)
def set_schema_from_string(self, schema):
self.schema = yamale.make_schema(content=schema, validators=self.validators)
def validate_file(self, file):
print('Validating %s...' % (file), file=sys.stderr)
data = yamale.make_data(file)
yamale.validate(self.schema, data)
@click.command()
@click.argument('files')
@click.option('--schema', default='/schemas/firewallSchema.yaml',
help='YAML schema file')
@click.option('--settings', default='/schemas/firewallSchemaSettings.yaml',
help='schema configuration file')
@click.option('--mode', default='validate',
help='select mode (validate or approve)')
@click.option('--github', is_flag=True, default=False,
help='output GitHub action compatible variables')
def main(**kwargs):
args = SimpleNamespace(**kwargs)
files = [args.files]
if '*' in args.files:
files = glob.glob(args.files, recursive=True)
print('Arguments: %s' % (str(sys.argv)), file=sys.stderr)
f = open(args.settings)
settings = yaml.load(f, Loader=yaml.SafeLoader)
firewall_validator = FirewallValidator(settings, args.mode)
firewall_validator.set_schema_from_file(args.schema)
output = {'ok': True, 'errors': {}}
for file in files:
try:
firewall_validator.validate_file(file)
except yamale.yamale_error.YamaleError as e:
if file not in output['errors']:
output['errors'][file] = []
output['ok'] = False
for result in e.results:
for err in result.errors:
output['errors'][file].append(err)
if args.github:
print('::set-output name=ok::%s' % ('true' if output['ok'] else 'false'))
print('::set-output name=errors::%s' % (json.dumps(output['errors'])))
print(json.dumps(output), file=sys.stderr)
else:
print(json.dumps(output))
if not output['ok'] and not args.github:
sys.exit(1)
if __name__ == '__main__':
main()

57
blueprints/networking/decentralized-firewall/variables.tf
View File

@ -1,57 +0,0 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
variable "billing_account_id" {
description = "Billing account id used as default for new projects."
type = string
}
variable "ip_ranges" {
description = "Subnet IP CIDR ranges."
type = map(string)
default = {
prod = "10.0.16.0/24"
dev = "10.0.32.0/24"
}
}
variable "prefix" {
description = "Prefix used for resource names."
type = string
validation {
condition = var.prefix != ""
error_message = "Prefix cannot be empty."
}
}
variable "project_services" {
description = "Service APIs enabled by default in new projects."
type = list(string)
default = [
"container.googleapis.com",
"dns.googleapis.com",
"stackdriver.googleapis.com",
]
}
variable "region" {
description = "Region used."
type = string
default = "europe-west1"
}
variable "root_node" {
description = "Hierarchy node where projects will be created, 'organizations/org_id' or 'folders/folder_id'."
type = string
}

6
default-versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

4
fast/stages/2-networking-a-peering/dns-landing.tf
View File

@ -82,8 +82,10 @@ module "landing-dns-policy-googleapis" {
source = "../../../modules/dns-response-policy"
project_id = module.landing-project.project_id
name = "googleapis"
factories_config = {
rules = var.factories_config.dns_policy_rules_file
}
networks = {
landing = module.landing-vpc.self_link
}
rules_file = var.factories_config.dns_policy_rules_file
}

2
fast/stages/2-networking-a-peering/main.tf
View File

@ -60,7 +60,7 @@ module "firewall-policy-default" {
source = "../../../modules/net-firewall-policy"
name = var.factories_config.firewall_policy_name
parent_id = module.folder.id
rules_factory_config = {
factories_config = {
cidr_file_path = "${var.factories_config.data_dir}/cidrs.yaml"
ingress_rules_file_path = "${var.factories_config.data_dir}/hierarchical-ingress-rules.yaml"
}

4
fast/stages/2-networking-b-vpn/dns-landing.tf
View File

@ -82,8 +82,10 @@ module "landing-dns-policy-googleapis" {
source = "../../../modules/dns-response-policy"
project_id = module.landing-project.project_id
name = "googleapis"
factories_config = {
rules = var.factories_config.dns_policy_rules_file
}
networks = {
landing = module.landing-vpc.self_link
}
rules_file = var.factories_config.dns_policy_rules_file
}

2
fast/stages/2-networking-b-vpn/main.tf
View File

@ -60,7 +60,7 @@ module "firewall-policy-default" {
source = "../../../modules/net-firewall-policy"
name = var.factories_config.firewall_policy_name
parent_id = module.folder.id
rules_factory_config = {
factories_config = {
cidr_file_path = "${var.factories_config.data_dir}/cidrs.yaml"
ingress_rules_file_path = "${var.factories_config.data_dir}/hierarchical-ingress-rules.yaml"
}

4
fast/stages/2-networking-c-nva/dns-landing.tf
View File

@ -91,9 +91,11 @@ module "landing-dns-policy-googleapis" {
source = "../../../modules/dns-response-policy"
project_id = module.landing-project.project_id
name = "googleapis"
factories_config = {
rules = var.factories_config.dns_policy_rules_file
}
networks = {
landing-trusted = module.landing-trusted-vpc.self_link
landing-untrusted = module.landing-untrusted-vpc.self_link
}
rules_file = var.factories_config.dns_policy_rules_file
}

2
fast/stages/2-networking-c-nva/main.tf
View File

@ -61,7 +61,7 @@ module "firewall-policy-default" {
source = "../../../modules/net-firewall-policy"
name = var.factories_config.firewall_policy_name
parent_id = module.folder.id
rules_factory_config = {
factories_config = {
cidr_file_path = "${var.factories_config.data_dir}/cidrs.yaml"
ingress_rules_file_path = "${var.factories_config.data_dir}/hierarchical-ingress-rules.yaml"
}

4
fast/stages/2-networking-d-separate-envs/dns-dev.tf
View File

@ -77,8 +77,10 @@ module "dev-dns-policy-googleapis" {
source = "../../../modules/dns-response-policy"
project_id = module.dev-spoke-project.project_id
name = "googleapis"
factories_config = {
rules = var.factories_config.dns_policy_rules_file
}
networks = {
dev = module.dev-spoke-vpc.self_link
}
rules_file = var.factories_config.dns_policy_rules_file
}

4
fast/stages/2-networking-d-separate-envs/dns-prod.tf
View File

@ -77,8 +77,10 @@ module "prod-dns-policy-googleapis" {
source = "../../../modules/dns-response-policy"
project_id = module.prod-spoke-project.project_id
name = "googleapis"
factories_config = {
rules = var.factories_config.dns_policy_rules_file
}
networks = {
prod = module.prod-spoke-vpc.self_link
}
rules_file = var.factories_config.dns_policy_rules_file
}

2
fast/stages/2-networking-d-separate-envs/main.tf
View File

@ -56,7 +56,7 @@ module "firewall-policy-default" {
source = "../../../modules/net-firewall-policy"
name = var.factories_config.firewall_policy_name
parent_id = module.folder.id
rules_factory_config = {
factories_config = {
cidr_file_path = "${var.factories_config.data_dir}/cidrs.yaml"
ingress_rules_file_path = "${var.factories_config.data_dir}/hierarchical-ingress-rules.yaml"
}

4
fast/stages/2-networking-e-nva-bgp/dns-landing.tf
View File

@ -91,9 +91,11 @@ module "landing-dns-policy-googleapis" {
source = "../../../modules/dns-response-policy"
project_id = module.landing-project.project_id
name = "googleapis"
factories_config = {
rules = var.factories_config.dns_policy_rules_file
}
networks = {
landing-trusted = module.landing-trusted-vpc.self_link
landing-untrusted = module.landing-untrusted-vpc.self_link
}
rules_file = var.factories_config.dns_policy_rules_file
}

2
fast/stages/2-networking-e-nva-bgp/main.tf
View File

@ -61,7 +61,7 @@ module "firewall-policy-default" {
source = "../../../modules/net-firewall-policy"
name = var.factories_config.firewall_policy_name
parent_id = module.folder.id
rules_factory_config = {
factories_config = {
cidr_file_path = "${var.factories_config.data_dir}/cidrs.yaml"
ingress_rules_file_path = "${var.factories_config.data_dir}/hierarchical-ingress-rules.yaml"
}

6
fast/stages/3-project-factory/dev/README.md
View File

@ -1,7 +1,7 @@
# Project factory
The Project Factory (or PF) builds on top of your foundations to create and set up projects (and related resources) to be used for your workloads.
It is organized in folders representing environments (e.g., "dev", "prod"), each implemented by a stand-alone terraform [resource factory](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c).
It is organized in folders representing environments (e.g., "dev", "prod"), each implemented by a stand-alone terraform [process factory](../../../../blueprints/factories/README.md).
## Design overview and choices
@ -13,7 +13,7 @@ A single factory creates projects in a well-defined context, according to your r
Projects for each environment across different teams are created by dedicated service accounts, as exemplified in the diagram above. While there's no intrinsic limitation regarding where the project factory can create a projects, the IAM bindings for the service account effectively enforce boundaries (e.g., the production service account shouldn't be able to create or have any access to the development projects, and vice versa).
The project factory exposes all the features of the underlying [project module](../../../../modules/project/), including Shared VPC service project attachment, VPC SC perimeter membership, etc.
The project factory stage lightly wraps the underlying [project-factory module](../../../../modules/project-factory/), including Shared VPC service project attachment, VPC SC perimeter membership, etc.
## How to run this stage
@ -55,7 +55,7 @@ gcloud alpha storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/2-security.auto.
If you're not using FAST, refer to the [Variables](#variables) table at the bottom of this document for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning.
Besides the values above, the project factory is driven by data files which closely follow the variables exposed by the [project module](../../../../modules/project/), with one file per project. Please refer to the underlying [project factory blueprint](../../../../blueprints/factories/project-factory/) documentation for details on the format.
Besides the values above, the project factory is driven by YAML data files, with one file per project. Please refer to the underlying [project factory module](../../../../modules/project-factory/) documentation for details on the format.
Once the configuration is complete, run the project factory with:

2
fast/stages/3-project-factory/dev/main.tf
View File

@ -17,7 +17,7 @@
# tfdoc:file:description Project factory.
module "projects" {
source = "../../../../blueprints/factories/project-factory"
source = "../../../../modules/project-factory"
data_defaults = {
billing_account = var.billing_account.id
# more defaults are available, check the project factory variables

4
modules/README.md
View File

@ -39,6 +39,10 @@ These modules are used in the examples included in this repository. If you are u
- [Project](./project)
- [Projects (data source)](./projects-data-source)
## Process factories
- [Project factory](./project-factory/)
## Networking modules
- [Address reservation](./net-address)

102
modules/__docs/20231106-factories.md Normal file
View File

@ -0,0 +1,102 @@
# Factories Refactor and Plan Forward
**authors:** [Ludo](https://github.com/ludoo)
**last modified:** February 16, 2024
## Status
Under discussion.
## Context
Factories evolved progressively in Fabric, from the original firewall factory module, to a semi-standardized approach to management of repeated resources. This progression happened piecemeal and it's now time to define a clear strategy for factories in both Fabric and FAST, so that we can remove guesswork from new developments and provide a predictive approach to users.
The remainder of this section provides a summary of the current status.
### Modules
Several modules implement factories for repeated resources which are typically dependent from the main resource managed in the module:
- `billing-account` provides a factory for billing alert rules tied to the billing account
- `dns-response-policy` provides a factory for rules in within the policy
- `net-firewall-policy` provides a factory for rules within the policy
- `net-vpc` provides a factory for subnets in the VPC
- `net-vpc-firewall` provides a factory for VPC firewall rules
- `organization` and `folder` provide a factory for hierarchical firewall rules within their policy
- `organization`, `folder` and `project` provide a factory for organization policies
The common pattern for modules is management of *multiple resources* typically dependent from the single *main resource* managed by the module.
### Blueprints
The `factories` folder in blueprints contains a collection of factories with a fuzzier approach
- `bigquery-factory` manages tables and views for 1-n datasets by wrapping the `bigquery-dataset` module via simple locals
- `cloud-identity-group-factory` manages Cloud Identity group members for 1-n groups by wrapping the `cloud-identity-group` via simple locals
- `net-vpc-firewall-yaml` is the original factory module managing VPC firewall rules, superseded by the factory in the `net-vpc-firewall` module
- `project-factory` combines the project, service account, and (planned) billing account and VPC modules to implement end-to-end project creation and configuration
There's no clear common pattern for these factories, where some could be moved to the respective module and the project factory combines a collection of modules to implement a process.
### FAST
FAST currently leverages module-level factories (organization policies, subnets, firewalls, etc.), and also provides the project factory as a dedicated level 3 stage by wrapping the relevant blueprint and localizing a few variables for the environment (`prefix`, `labels`).
## Proposal
While the current approach is reasonably clear in regards to modules, it has never been formalized in a set of guidelines that can help authors define when and how new factories would made sense.
On top of this, the `factories` blueprints folder contains code that that should really be moved to module-level factories, and the project factory which could/should be published directly as a FAST stage, since those are consumable as standalone modules.
This proposal aims at addressing the above problems.
### Module-level factory approach
The current approach for module-level factories can be summarized in a single principle:
> factories implemented in modules manage multiple resources which depend from one single main resource (or a small set of main resources) which are the main driver of the module.
For example, the module managing a firewall policy exposes a factory for its rules, or the module managing a VPC exposes a factory for its subnets. But the project module would not expose a projects factory, as one project maps to a single module invocation.
The proposal on factory modules then is to:
- align all factory variables to the same standard, outlined below
- move the groups and bigquery factories from blueprints to the respective modules
- eventually add more factories when it makes sense to do so (e.g. for KMS keys, service accounts, etc.)
The variable interface for module-level factories should use a single top-level `factory_configs` variable, whose type is an object with one or more attributes which are named according to the specific factory. This will allow composing multiple factory configurations into a single variable in FAST stages, by avoiding name overlaps. An example:
```hcl
variable "factory_configs" {
description = "Path to folder containing budget alerts data files."
type = object({
budgets_data_path = optional(string, "data/billing-budgets")
})
nullable = false
default = {}
}
```
### Blueprint factories
The `factories` folder in blueprints will be emptied, and a single README left in it pointing to all the module-level and FAST stage factories available.
As outlined above, the existing factories will be moved to modules (bigquery and groups), FAST (project factory), or deleted (firewall rules).
### FAST factories
The only change for FAST factories will be moving the project factory from blueprints to the stage folder, and updating the path used for the environment-level wrapping stage.
### File schema and filesystem organization
Factory files schema must mimick and implement the variable interface for the module, including optionals and validation - which are implemented in code and checks.
With notable exceptions (currently only the `cidrs.yaml` file consumed by firewall factories), the following convention for files/directory is proposed:
- Factories should consume directories (vs single files)
- All files should contain a dictionary of resources or a single resource
- If the factory accepts one resource per file (e.g. VPC subnets), the file name should be used for the resource name and the YAML should allow defining a `name:` override
- Files in a directory should be parsed together and flattened into a single dictionary
This allows developers to implement multiple resources in a single file or to use one file per resource, as they see fit.

0
modules/__experimental/alloydb-instance/README.md → modules/__experimental_deprecated/alloydb-instance/README.md
View File

0
modules/__experimental/alloydb-instance/main.tf → modules/__experimental_deprecated/alloydb-instance/main.tf
View File

0
modules/__experimental/alloydb-instance/outputs.tf → modules/__experimental_deprecated/alloydb-instance/outputs.tf
View File

0
modules/__experimental/alloydb-instance/variables.tf → modules/__experimental_deprecated/alloydb-instance/variables.tf
View File

6
modules/__experimental/alloydb-instance/versions.tf → modules/__experimental_deprecated/alloydb-instance/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
blueprints/factories/bigquery-factory/README.md → modules/__experimental_deprecated/bigquery-factory/README.md
View File

@ -11,6 +11,7 @@ You can create as many files as you like, the code will loop through it and crea
### Terraform code
In this section we show how to create tables and views from a file structure similar to the one shown below.
```bash
bigquery
│
@ -53,12 +54,12 @@ With this file structure, we can use the factory as follows:
```hcl
module "bq" {
source = "./fabric/blueprints/factories/bigquery-factory"
source = "./fabric/modules/__experimental_deprecated/bigquery-factory"
project_id = var.project_id
tables_path = "bigquery/tables"
views_path = "bigquery/views"
}
# tftest modules=2 resources=3 files=table,view inventory=simple.yaml
# tftest modules=2 resources=3 files=table,view
```
<!-- BEGIN TFDOC -->
@ -76,4 +77,3 @@ module "bq" {
- [ ] add external table support
- [ ] add materialized view support

0
blueprints/factories/bigquery-factory/main.tf → modules/__experimental_deprecated/bigquery-factory/main.tf
View File

0
blueprints/factories/bigquery-factory/variables.tf → modules/__experimental_deprecated/bigquery-factory/variables.tf
View File

4
blueprints/factories/cloud-identity-group-factory/README.md → modules/__experimental_deprecated/cloud-identity-group-factory/README.md
View File

@ -10,11 +10,11 @@ Yaml abstraction for Groups can simplify groups creation and members management.
```hcl
module "groups" {
source = "./fabric/blueprints/factories/cloud-identity-group-factory"
source = "./fabric/modules/__experimental_deprecated/cloud-identity-group-factory"
customer_id = "customers/C0xxxxxxx"
data_dir = "data"
}
# tftest modules=2 resources=3 files=group1 inventory=example.yaml
# tftest modules=2 resources=3 files=group1
```
```yaml

0
blueprints/factories/cloud-identity-group-factory/main.tf → modules/__experimental_deprecated/cloud-identity-group-factory/main.tf
View File

0
blueprints/factories/cloud-identity-group-factory/outputs.tf → modules/__experimental_deprecated/cloud-identity-group-factory/outputs.tf
View File

0
blueprints/factories/cloud-identity-group-factory/variables.tf → modules/__experimental_deprecated/cloud-identity-group-factory/variables.tf
View File

0
modules/__experimental/net-dns-policy-address/README.md → modules/__experimental_deprecated/net-dns-policy-address/README.md
View File

0
modules/__experimental/net-dns-policy-address/main.tf → modules/__experimental_deprecated/net-dns-policy-address/main.tf
View File

0
modules/__experimental/net-dns-policy-address/outputs.tf → modules/__experimental_deprecated/net-dns-policy-address/outputs.tf
View File

0
modules/__experimental/net-dns-policy-address/variables.tf → modules/__experimental_deprecated/net-dns-policy-address/variables.tf
View File

0
modules/__experimental/net-neg/README.md → modules/__experimental_deprecated/net-neg/README.md
View File

0
modules/__experimental/net-neg/main.tf → modules/__experimental_deprecated/net-neg/main.tf
View File

0
modules/__experimental/net-neg/outputs.tf → modules/__experimental_deprecated/net-neg/outputs.tf
View File

0
modules/__experimental/net-neg/variables.tf → modules/__experimental_deprecated/net-neg/variables.tf
View File

6
modules/__experimental/net-neg/versions.tf → modules/__experimental_deprecated/net-neg/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/__experimental/project-iam-magic/versions.tf → modules/__experimental_deprecated/project-iam-magic/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

5
modules/analytics-hub/main.tf
View File

@ -18,9 +18,10 @@ locals {
prefix = var.prefix == null || var.prefix == "" ? "" : "${var.prefix}_"
_factory_listings = {
for f in try(fileset(var.factories_config.listings, "*.yaml"), []) :
trimsuffix(f, ".yaml") => yamldecode(file("${var.factories_config.listings}/${f}"))
trimsuffix(f, ".yaml") => yamldecode(
file("${var.factories_config.listings}/${f}")
)
}
factory_listings = merge(local._factory_listings, var.listings)
}

6
modules/analytics-hub/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/api-gateway/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/apigee/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/artifact-registry/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/bigquery-dataset/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/bigtable-instance/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

8
modules/billing-account/README.md
View File

@ -260,16 +260,16 @@ update_rules:
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [id](variables.tf#L131) | Billing account id. | <code>string</code> | ✓ | |
| [id](variables.tf#L130) | Billing account id. | <code>string</code> | ✓ | |
| [budget_notification_channels](variables.tf#L17) | Notification channels used by budget alerts. | <code title="map&#40;object&#40;&#123;&#10; project_id &#61; string&#10; type &#61; string&#10; description &#61; optional&#40;string&#41;&#10; display_name &#61; optional&#40;string&#41;&#10; enabled &#61; optional&#40;bool, true&#41;&#10; force_delete &#61; optional&#40;bool&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;&#41;&#10; sensitive_labels &#61; optional&#40;list&#40;object&#40;&#123;&#10; auth_token &#61; optional&#40;string&#41;&#10; password &#61; optional&#40;string&#41;&#10; service_key &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#41;&#10; user_labels &#61; optional&#40;map&#40;string&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [budgets](variables.tf#L47) | Billing budgets. Notification channels are either keys in corresponding variable, or external ids. | <code title="map&#40;object&#40;&#123;&#10; amount &#61; object&#40;&#123;&#10; currency_code &#61; optional&#40;string&#41;&#10; nanos &#61; optional&#40;number&#41;&#10; units &#61; optional&#40;number&#41;&#10; use_last_period &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#10; display_name &#61; optional&#40;string&#41;&#10; filter &#61; optional&#40;object&#40;&#123;&#10; credit_types_treatment &#61; optional&#40;object&#40;&#123;&#10; exclude_all &#61; optional&#40;bool&#41;&#10; include_specified &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#10; label &#61; optional&#40;object&#40;&#123;&#10; key &#61; string&#10; value &#61; string&#10; &#125;&#41;&#41;&#10; period &#61; optional&#40;object&#40;&#123;&#10; calendar &#61; optional&#40;string&#41;&#10; custom &#61; optional&#40;object&#40;&#123;&#10; start_date &#61; object&#40;&#123;&#10; day &#61; number&#10; month &#61; number&#10; year &#61; number&#10; &#125;&#41;&#10; end_date &#61; optional&#40;object&#40;&#123;&#10; day &#61; number&#10; month &#61; number&#10; year &#61; number&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#10; projects &#61; optional&#40;list&#40;string&#41;&#41;&#10; resource_ancestors &#61; optional&#40;list&#40;string&#41;&#41;&#10; services &#61; optional&#40;list&#40;string&#41;&#41;&#10; subaccounts &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#10; threshold_rules &#61; optional&#40;list&#40;object&#40;&#123;&#10; percent &#61; number&#10; forecasted_spend &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; update_rules &#61; optional&#40;map&#40;object&#40;&#123;&#10; disable_default_iam_recipients &#61; optional&#40;bool&#41;&#10; monitoring_notification_channels &#61; optional&#40;list&#40;string&#41;&#41;&#10; pubsub_topic &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [factory_config](variables.tf#L121) | Path to folder containing budget alerts data files. | <code title="object&#40;&#123;&#10; budgets_data_path &#61; optional&#40;string, &#34;data&#47;billing-budgets&#34;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [factories_config](variables.tf#L121) | Path to folder containing budget alerts data files. | <code title="object&#40;&#123;&#10; budgets_data_path &#61; optional&#40;string, &#34;data&#47;billing-budgets&#34;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam](variables-iam.tf#L17) | IAM bindings in {ROLE => [MEMBERS]} format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam_bindings](variables-iam.tf#L24) | Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. | <code title="map&#40;object&#40;&#123;&#10; members &#61; list&#40;string&#41;&#10; role &#61; string&#10; condition &#61; optional&#40;object&#40;&#123;&#10; expression &#61; string&#10; title &#61; string&#10; description &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam_bindings_additive](variables-iam.tf#L39) | Individual additive IAM bindings. Keys are arbitrary. | <code title="map&#40;object&#40;&#123;&#10; member &#61; string&#10; role &#61; string&#10; condition &#61; optional&#40;object&#40;&#123;&#10; expression &#61; string&#10; title &#61; string&#10; description &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam_by_principals](variables-iam.tf#L54) | Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the `iam` variable. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [logging_sinks](variables.tf#L136) | Logging sinks to create for the organization. | <code title="map&#40;object&#40;&#123;&#10; destination &#61; string&#10; type &#61; string&#10; bq_partitioned_table &#61; optional&#40;bool, false&#41;&#10; description &#61; optional&#40;string&#41;&#10; disabled &#61; optional&#40;bool, false&#41;&#10; exclusions &#61; optional&#40;map&#40;object&#40;&#123;&#10; filter &#61; string&#10; description &#61; optional&#40;string&#41;&#10; disabled &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10; filter &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [projects](variables.tf#L169) | Projects associated with this billing account. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [logging_sinks](variables.tf#L135) | Logging sinks to create for the organization. | <code title="map&#40;object&#40;&#123;&#10; destination &#61; string&#10; type &#61; string&#10; bq_partitioned_table &#61; optional&#40;bool, false&#41;&#10; description &#61; optional&#40;string&#41;&#10; disabled &#61; optional&#40;bool, false&#41;&#10; exclusions &#61; optional&#40;map&#40;object&#40;&#123;&#10; filter &#61; string&#10; description &#61; optional&#40;string&#41;&#10; disabled &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10; filter &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [projects](variables.tf#L168) | Projects associated with this billing account. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
## Outputs

2
modules/billing-account/factory.tf
View File

@ -19,7 +19,7 @@ locals {
for f in fileset("${local._factory_path}", "**/*.yaml") :
trimsuffix(f, ".yaml") => yamldecode(file("${local._factory_path}/${f}"))
}
_factory_path = var.factory_config.budgets_data_path
_factory_path = try(pathexpand(var.factories_config.budgets_data_path), "")
factory_budgets = {
for k, v in local._factory_data : k => merge(v, {
amount = merge(

3
modules/billing-account/variables.tf
View File

@ -118,8 +118,7 @@ variable "budgets" {
}
}
variable "factory_config" {
# TODO: align all other factory variable names
variable "factories_config" {
description = "Path to folder containing budget alerts data files."
type = object({
budgets_data_path = optional(string, "data/billing-budgets")

6
modules/billing-account/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/binauthz/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/__need_fixing/onprem/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/__need_fixing/squid/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/coredns/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/cos-generic-metadata/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/envoy-sni-dyn-fwd-proxy/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/envoy-traffic-director/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/mysql/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/nginx-tls/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/nginx/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-config-container/simple-nva/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-function-v1/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-function-v2/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

6
modules/cloud-identity-group/versions.tf
View File

@ -13,15 +13,15 @@
# limitations under the License.
terraform {
required_version = ">= 1.7.0"
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 5.11.0, < 6.0.0" # tftest
version = ">= 5.12.0, < 6.0.0" # tftest
}
}
}

Some files were not shown because too many files have changed in this diff Show More