Merge branch 'master' into ludo-fix-example-project-creation

This commit is contained in:
Ludovico Magnocavallo 2022-01-14 08:31:31 +01:00 committed by GitHub
commit 4c60112bdb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
73 changed files with 2471 additions and 1507 deletions

View File

@ -2,6 +2,10 @@
All notable changes to this project will be documented in this file.
## Unreleased
- new `net-xlb` module for Global External Load balancer
## [12.0.0] - 2022-01-11
- new repo structure. All end-to-end examples moved to the top level `examples` folder

View File

@ -4,11 +4,11 @@ This section contains **[foundational examples](./foundations/)** that bootstrap
Currently available examples:
- **cloud operations** - [Resource tracking and remediation via Cloud Asset feeds](./cloud-operations/asset-inventory-feed-remediation), [Granular Cloud DNS IAM via Service Directory](./cloud-operations/dns-fine-grained-iam), [Granular Cloud DNS IAM for Shared VPC](./cloud-operations/dns-shared-vpc), [Compute Engine quota monitoring](./cloud-operations/quota-monitoring), [Scheduled Cloud Asset Inventory Export to Bigquery](./cloud-operations/scheduled-asset-inventory-export-bq), [Packer image builder](./cloud-operations/packer-image-builder), [On-prem SA key management](./cloud-operations/onprem-sa-key-management)
- **data solutions** - [GCE/GCS CMEK via centralized Cloud KMS](./data-solutions/cmek-via-centralized-kms/), [Cloud Storage to Bigquery with Cloud Dataflow](./data-solutions/gcs-to-bq-with-dataflow/)
- **factories** - [The why and the how of resource factories](./factories/README.md)
- **foundations** - [single level hierarchy](./foundations/environments/) (environments), [multiple level hierarchy](./foundations/business-units/) (business units + environments)
- **networking** - [hub and spoke via peering](./networking/hub-and-spoke-peering/), [hub and spoke via VPN](./networking/hub-and-spoke-vpn/), [DNS and Google Private Access for on-premises](./networking/onprem-google-access-dns/), [Shared VPC with GKE support](./networking/shared-vpc-gke/), [ILB as next hop](./networking/ilb-next-hop), [PSC for on-premises Cloud Function invocation](./networking/private-cloud-function-from-onprem/), [decentralized firewall](./networking/decentralized-firewall)
- **data solutions** - [GCE/GCS CMEK via centralized Cloud KMS](./data-solutions/cmek-via-centralized-kms/), [Cloud Storage to Bigquery with Cloud Dataflow](./data-solutions/gcs-to-bq-with-dataflow/)
- **cloud operations** - [Resource tracking and remediation via Cloud Asset feeds](./cloud-operations/asset-inventory-feed-remediation), [Granular Cloud DNS IAM via Service Directory](./cloud-operations/dns-fine-grained-iam), [Granular Cloud DNS IAM for Shared VPC](./cloud-operations/dns-shared-vpc), [Compute Engine quota monitoring](./cloud-operations/quota-monitoring), [Scheduled Cloud Asset Inventory Export to Bigquery](./cloud-operations/scheduled-asset-inventory-export-bq), [Packer image builder](./cloud-operations/packer-image-builder), [On-prem SA key management](./cloud-operations/onprem-sa-key-management)
- **third party solutions** - [OpenShift cluster on Shared VPC](./third-party-solutions/openshift)
- **factories** - [Example environments](./factories/example-environments), [Hierarchical Firewall Policies](./factories/firewall-hierarchical-policies), [VPC Firewall Rules](./factories/firewall-vpc-rules), [Subnets](./factories/subnets)
For more information see the README files in the [foundations](./foundations/), [networking](./networking/), [data solutions](./data-solutions/), [cloud operations](./cloud-operations/) and [factories](./factories/) folders.

View File

@ -1,25 +1,16 @@
# Resource Factories
# The why and the how of Resource Factories
This set of modules creates specialized resource factories, made of two distinct components:
Terraform modules can be designed - where it makes sense - to implement a resource factory, which is a configuration-driven approach to resource creation meant to:
- a module, which implements the factory logic in Terraform syntax, and
- a set of directories, which hold the configuration for the factory in YAML syntax.
- accelerate and rationalize the repetitive creation of common resources, such as firewall rules and subnets
- enable teams without Terraform specific knowledge to leverage IaC via human-friendly and machine-parseable YAML files
- make it simple to implement specific requirements and best practices (e.g. "always enable PGA for GCP subnets", or "only allow using regions `europe-west1` and `europe-west3`")
- codify and centralise business logics and policies (e.g. labels and naming conventions)
- allow to easily parse and understand sets of specific resources, for documentation purposes
## Available modules
- [Hierarchical Firewall policies](./firewall-hierarchical-policies)
- [VPC Firewall rules](./firewall-vpc-rules)
- [Subnets](./subnets)
## Using the modules
Each module specialises on a single resource type, and comes with a `README.md` file which describes the module interface, and the directory/file structure each module requires.
All modules consume specialized `yaml` configurations - located on a well-defined directory structure that carries metadata. Let's observe an example which leverages the `subnet` module, taken from the [Example environments](example-environments/) directory:
Generally speaking, the configurations for a resource factory consists in one or more YaML files, optionally grouped in folders, that describe resources following a well defined, validable schema, such as in the example below for the subnet factory of the [`net-vpc`](../../modules/net-vpc) module, which allows for the massive creation of subnets for a given VPC.
```yaml
# ../example-environments/prod/conf/project-prod-a/vpc-alpha/subnet-alpha-a.yaml
region: europe-west3
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-prod-a, vpc-alpha
@ -28,21 +19,7 @@ secondary_ip_ranges:
secondary-range-b: 192.168.1.0/24
```
This configuration creates the `subnet-alpha-a` subnet, located in VPC `vpc-alpha`, inside project `project-prod-a`.
## Rationale
This approach is based on modules implementing the factory logic using Terraform code, and a set of directories having a well-defined, semantic structure holding the configuration for the resources in YaML syntax.
Resource factories are designed to:
- accelerate and rationalize the repetitive creation of common resources, such as firewall rules and subnets
- enable teams without Terraform specific knowledge to leverage IaC via human-friendly and machine-parseable YAML files
- make it simple to implement specific requirements and best practices (e.g. "always enable PGA for GCP subnets", or "only allow using regions `europe-west1` and `europe-west3`")
- codify and centralise business logics and policies (e.g. labels and naming conventions)
- allow to easily parse and understand sets of specific resources, for documentation purposes
Terraform natively supports YaML, JSON and CSV parsing - however we've decided to embrace YaML for the following reasons:
Terraform natively supports YaML, JSON and CSV parsing - however Fabric has decided to embrace YaML for the following reasons:
- YaML is easier to parse for a human, and allows for comments and nested, complex structures
- JSON and CSV can't include comments, which can be used to document configurations, but are often useful to bridge from other systems in automated pipelines
@ -50,3 +27,16 @@ Terraform natively supports YaML, JSON and CSV parsing - however we've decided t
- CSV isn't often expressive enough (e.g. dit oesn't allow for nested structures)
If needed, converting factories to consume JSON is a matter of switching from `yamldecode()` to `jsondecode()` in the right place on each module.
## Resource factories in Fabric
### Fabric Modules
- [folder](../../modules/folder/README.md#firewall-policy-factory) and [organization](../../modules/organization/README.md#firewall-policy-factory) implement factories for [hierarchical firewall policies](https://cloud.google.com/vpc/docs/firewall-policies)
- [net-vpc](../../modules/net-vpc/README.md#subnet-factory) for subnets creation
- [net-vpc-firewall](../../modules/net-vpc-firewall/README.md#rules-factory) for massive rules creation
### Dedicated Factories
A dedicated resource factory for the massive creation of VPC firewall rules across different projects/VPCs is also provided as a reference: [net-vpc-firewall-yaml](net-vpc-firewall-yaml/README.md)

View File

@ -1,43 +0,0 @@
# Resource Factories
The examples in this folder are derived from actual production use cases, and show how to use a factory module, and how to structure a codebase for multiple environments.
## Resource Factories usage - Managing subnets
At the top level of this directory, besides the `README.md` your're reading now, you'll find
- `dev/`, a directory which holds all configurations for the *development* environment
- `prod/`, a directory which holds all configurations for the *production* environment
and on each directory, `main.tf`, a simple terraform file which consumes the [`subnets`](../subnets/) module and the configurations.
Each environment directory structure is meant to mimic your GCP resource structure
```
.
├── dev # Environment
│ ├── conf # Configuration directory
│ │ ├── project-dev-a # Project id
│ │ │ └── vpc-alpha # VPC name
│ │ │ ├── subnet-alpha-a.yaml # Subnet name (one file per subnet)
│ │ │ └── subnet-alpha-b.yaml
│ │ └── project-dev-b
│ │ ├── vpc-beta
│ │ │ └── subnet-beta-a.yaml
│ │ └── vpc-gamma
│ │ └── subnet-gamma-a.yaml
│ └── main.tf
└── prod
├── conf
│ └── project-prod-a
│ └── vpc-alpha
│ ├── subnet-alpha-a.yaml
│ └── subnet-alpha-b.yaml
└── main.tf
```
Since this resource factory only creates subnets, projects and VPCs are expected to exist.
In this example, each environment is implemented as a distinct factory, and each has its own `main.tf` file (and hence a dedicated state).
Another option you might want to consider, in line with the CI/CD pipeline or processes you have in place, might be to leverage a single `main.tf` consuming both environment directories.

View File

@ -1,8 +0,0 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-dev-a, vpc-alpha
secondary_ip_ranges:
secondary-range-a: 192.168.0.0/24
secondary-range-b: 192.168.1.0/24

View File

@ -1,6 +0,0 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.1.0/24
description: Sample Subnet in project project-dev-a, vpc-alpha
private_ip_google_access: false

View File

@ -1,7 +0,0 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.0.2.0/24
description: Sample Subnet in project project-dev-b, vpc-beta
iam_users: ["sruffilli@google.com"]
iam_groups: []

View File

@ -1,5 +0,0 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.0.3.0/24
description: Sample Subnet in project project-dev-b, vpc-gamma

View File

@ -1,8 +0,0 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-prod-a, vpc-alpha
secondary_ip_ranges:
secondary-range-a: 192.168.0.0/24
secondary-range-b: 192.168.1.0/24

View File

@ -1,6 +0,0 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.1.0/24
description: Sample Subnet in project project-prod-a, vpc-alpha
private_ip_google_access: false

View File

@ -1,21 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "subnets" {
source = "../../subnets"
config_folder = "conf"
}

View File

@ -1,29 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_version = ">= 1.0.0"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 4.0.0"
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 4.0.0"
}
}
}

View File

@ -1,168 +0,0 @@
# Google Cloud Resource Factories - Hierarchical Firewall Policies
This module implements a resource factory which allows the creation and management of [hierarchical firewall policies](https://cloud.google.com/vpc/docs/firewall-policies) through properly formatted `yaml` files.
`yaml` configurations are stored on a well-defined folder structure, whose entry point can be customized, and which allows for simple grouping of policies by Organization ID.
This module also allows defining custom template variables, to centralize common CIDRs or Service Account lists, which enables re-using them across different policies.
## Example
### Terraform code
```hcl
module "hierarchical" {
source = "./factories/hierarchical-firewall-policies"
config_folder = "firewall/hierarchical"
templates_folder = "firewall/templates"
}
# tftest:skip
```
### Configuration Structure
The naming convention for the `config_folder` variable requires
- the first directory layer to be named after the organization ID we're creating the policies for
- each file to be named either `$folder_id-$description.yaml` (e.g. `1234567890-sharedinfra.yaml`) for policies applying to regular folders or `org.yaml` for the root folder.
Organizations and folders should exist prior to running this module, or set as an explicit dependency to this module, leveraging `depends_on`.
The optional `templates_folder` variable can have two files.
- `cidrs.yaml` - a YAML map defining lists of CIDRs
- `service_accounts.yaml` - a YAML map defining lists of Service Accounts
Examples for both files are shown in the following section.
```bash
└── firewall
├── hierarchical
│ ├── 31415926535
│ │ ├── 1234567890-sharedinfra.yaml # Maps to folders/1234567890
│ │ └── org.yaml # Maps to organizations/31415926535
│ └── 27182818284
│ └── 1234567891-sharedinfra.yaml # Maps to folders/1234567891
└── templates
├── cidrs.yaml
└── service_accounts.yaml
```
### Hierarchical firewall policies format and structure
The following syntax applies for both `$folder_id-$description.yaml` and for `org.yaml` files, with the former applying at the `$folder_id` level and the latter at the Organization level.
Each file can contain an arbitrary number of policies.
```yaml
# Policy name
allow-icmp:
# Description
description: Sample policy
# Direction {INGRESS, EGRESS}
direction: INGRESS
# Action {allow, deny}
action: allow
# Priority (must be unique on a node)
priority: 1000
# List of CIDRs this rule applies to (for INGRESS rules)
source_ranges:
- 0.0.0.0/0
# List of CIDRs this rule applies to (for EGRESS rules)
destination_ranges:
- 0.0.0.0/0
# List of ports this rule applies to (empty array means all ports)
ports:
tcp: []
udp: []
icmp: []
# List of VPCs this rule applies to - a null value implies all VPCs
target_resources: null
# Opt - List of target Service Accounts this rule applies to
target_service_accounts:
- example-service-account@foobar.iam.gserviceaccount.com
# Opt - Whether to enable logs - defaults to false
enable_logging: true
```
A sample configuration file might look like the following one:
```yaml
allow-icmp:
description: Enable ICMP for all hosts
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- 0.0.0.0/0
ports:
icmp: []
target_resources: null
enable_logging: false
allow-ssh-from-onprem:
description: Enable SSH for on prem hosts
direction: INGRESS
action: allow
priority: 1001
source_ranges:
- $onprem
ports:
tcp: ["22"]
target_resources: null
enable_logging: false
allow-443-from-clients:
description: Enable HTTPS for web clients
direction: INGRESS
action: allow
priority: 1001
source_ranges:
- $web_clients
ports:
tcp: ["443"]
target_resources: null
target_service_accounts:
- $web_frontends
enable_logging: false
```
with `firewall/templates/cidrs.yaml` defined as follows:
```yaml
onprem:
- 10.0.0.0/8
- 192.168.0.0/16
web_clients:
- 172.16.0.0/16
- 10.0.10.0/24
- 10.0.250.0/24
```
and `firewall/templates/service_accounts.yaml`:
```yaml
web_frontends:
- web-frontends@project-wf1.iam.gserviceaccount.com
- web-frontends@project-wf2.iam.gserviceaccount.com
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| config_folder | Relative path of the folder containing the hierarchical firewall configuration | <code>string</code> | ✓ | |
| templates_folder | Relative path of the folder containing the cidr/service account templates | <code>string</code> | ✓ | |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| hierarchical-firewall-rules | Generated Hierarchical Firewall Rules | |
<!-- END TFDOC -->

View File

@ -1,99 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
cidrs = try({
for name, cidrs in yamldecode(file("${var.templates_folder}/cidrs.yaml")) :
name => cidrs
}, {})
service_accounts = try({ for name, service_accounts in yamldecode(file("${var.templates_folder}/service_accounts.yaml")) :
name => service_accounts
}, {})
org_paths = {
for file in fileset(var.config_folder, "**/*.yaml") :
file => split("/", file)[1] == "org.yaml"
? "organizations/${split("/", file)[0]}"
: "folders/${split("-", split("/", file)[1])[0]}"
}
rules = flatten([
for file in fileset(var.config_folder, "**/*.yaml") : [
for key, ruleset in yamldecode(file("${var.config_folder}/${file}")) :
merge(ruleset, {
parent_id = local.org_paths[file]
name = "${replace(local.org_paths[file], "/", "-")}-${key}"
source_ranges = try(ruleset.source_ranges, null) == null ? null : flatten(
[for cidr in ruleset.source_ranges :
can(regex("^\\$", cidr))
? local.cidrs[trimprefix(cidr, "$")]
: [cidr]
])
destination_ranges = try(ruleset.destination_ranges, null) == null ? null : flatten(
[for cidr in ruleset.destination_ranges :
can(regex("^\\$", cidr))
? local.cidrs[trimprefix(cidr, "$")]
: [cidr]
])
target_service_accounts = try(ruleset.target_service_accounts, null) == null ? null : flatten(
[for service_account in ruleset.target_service_accounts :
can(regex("^\\$", service_account))
? local.service_accounts[trimprefix(service_account, "$")]
: [service_account]
])
})
]
])
}
resource "google_compute_firewall_policy" "default" {
for_each = { for rule in local.rules : rule.parent_id => rule.name... }
short_name = replace("hierarchical-fw-policy-${each.key}", "/", "-")
description = replace("hierarchical-fw-policy-${each.key}", "/", "-")
parent = each.key
}
resource "google_compute_firewall_policy_rule" "default" {
for_each = { for rule in local.rules : "${rule.parent_id}-${rule.name}" => rule }
firewall_policy = google_compute_firewall_policy.default[each.value.parent_id].id
action = each.value.action
direction = each.value.direction
priority = each.value.priority
target_resources = each.value.target_resources
target_service_accounts = each.value.target_service_accounts
enable_logging = try(each.value.enable_logging, false)
# preview = each.value.preview
match {
src_ip_ranges = each.value.direction == "INGRESS" ? each.value.source_ranges : null
dest_ip_ranges = each.value.direction == "EGRESS" ? each.value.destination_ranges : null
dynamic "layer4_configs" {
for_each = each.value.ports
iterator = port
content {
ip_protocol = port.key
ports = port.value
}
}
}
}
resource "google_compute_firewall_policy_association" "default" {
for_each = { for rule in local.rules : rule.parent_id => rule.name... }
name = replace("hierarchical-fw-policy-${each.key}", "/", "-")
attachment_target = google_compute_firewall_policy.default[each.key].parent
firewall_policy = google_compute_firewall_policy.default[each.key].id
}

View File

@ -1,26 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "hierarchical-firewall-rules" {
description = "Generated Hierarchical Firewall Rules"
value = {
for k, v in google_compute_firewall_policy_rule.default :
k => {
parent_id = split("-", k)[0]
id = v.id
}
}
}

View File

@ -1,25 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "config_folder" {
description = "Relative path of the folder containing the hierarchical firewall configuration"
type = string
}
variable "templates_folder" {
description = "Relative path of the folder containing the cidr/service account templates"
type = string
}

View File

@ -1,6 +0,0 @@
# Google Cloud VPC Firewall Factories
This collection of modules implement two different metodologies for the creation of VPC firewall rules, both based on leveraging well-defined `yaml` configuration files.
- The [flat module](flat/) delegates the definition of all firewall rules metadata (project, network amongst other) to the individual `yaml` configuration. This module allows for maximum flexibility, and a custom logical grouping of resources which can be trasversal to the traditional resource hierarchy, and could be useful in scenarios where networking is not managed centrally by a single team.
- The [nested module](nested/) requires and enforces a semantical folder structure that carries some of the rules metadata (project and network), and leaves the rest to each `yaml` configuration. This solution allows for the definition of a resource hierarchy that is aligned with the organisational resource structure.

View File

@ -1,29 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_version = ">= 1.0.0"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 4.0.0"
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 4.0.0"
}
}
}

View File

@ -1,144 +0,0 @@
# Google Cloud VPC Firewall Factory - Nested hierarchy
This module implements a resource factory which allows the creation and management of [VPC firewall rules](https://cloud.google.com/vpc/docs/firewalls) via properly formatted `yaml` files.
`yaml` configurations are stored in a well-defined folder structure, whose entry point can be customized, and which represents and forces the resource hierarchy a firewall rule belongs to (Project > VPC > Firewall Rule).
This module also allows for the definition of template variables, allowing to centralize common CIDRs or Service Account lists, which enables re-using them across different policies.
## Example
### Terraform code
```hcl
module "vpc-firewall" {
source = "./factories/firewall-vpc-rules/nested"
config_folder = "firewall/vpc"
templates_folder = "firewall/templates"
}
# tftest:skip
```
### Configuration Structure
The naming convention for the `config_folder` folder **requires**
- the first directory layer to be named after the project ID which contains the VPC we're creating the firewall rules for
- the second directory layer to be named after the VPC we're creating the firewall rules for
- `yaml` files contained in the "VPC" directory can be arbitrarily named, to allow for an easier logical grouping.
Projects and VPCs should exist prior to running this module, or set as an explicit dependency to this module, leveraging `depends_on`.
The optional `templates_folder` folder can have two files.
- `cidrs.yaml` - a YAML map defining lists of CIDRs
- `service_accounts.yaml` - a YAML map definint lists of Service Accounts
```bash
└── firewall
├── vpc
│ ├── project-resource-factory-dev
│ │ └── vpc-resource-factory-dev-one
│ │ │ ├── frontend.yaml
│ │ │ └── backend.yaml
│ │ └── vpc-resource-factory-dev-two
│ │ ├── foo.yaml
│ │ └── bar.yaml
│ └── project-resource-factory-prod
│ │ └── vpc-resource-factory-prod-alpha
│ │ ├── lorem.yaml
│ │ └── ipsum.yaml
└── templates
├── cidrs.yaml
└── service_accounts.yaml
```
### Rule definition format and structure
Firewall rules configuration should be placed in a set of yaml files in a folder/s. Firewall rule entry structure is following:
```yaml
rule-name: # descriptive name, naming convention is adjusted by the module
description: "Allow icmp" # rule description
action: allow # `allow` or `deny`
direction: INGRESS # EGRESS or INGRESS
ports:
icmp: [] # {tcp, udp, icmp, all}: [ports], use [] for any port
priority: 1000 # rule priority value, default value is 1000
source_ranges: # list of source ranges
- 0.0.0.0/0
destination_ranges: # list of destination ranges
- 0.0.0.0/0
source_tags: ['some-tag'] # list of source tags
source_service_accounts: # list of source service accounts
- myapp@myproject-id.iam.gserviceaccount.com
target_tags: ['some-tag'] # list of target tags
target_service_accounts: # list of target service accounts
- myapp@myproject-id.iam.gserviceaccount.com
enable_logging: true # `false` or `true`, logging is enabled when `true`
```
A sample configuration file might look like the following one:
```yaml
allow-healthchecks:
description: "Allow traffic from healthcheck"
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- $healthcheck
ports:
tcp: ["80"]
enable_logging: false
allow-http:
description: "Allow traffic to LB backend"
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- 0.0.0.0/0
target_service_accounts:
- $web_frontends
ports:
tcp: ["80", "443"]
enable_logging: false
```
with `firewall/templates/cidrs.yaml` defined as follows:
```yaml
healthcheck:
- 35.191.0.0/16
- 130.211.0.0/22
```
and `firewall/templates/service_accounts.yaml`:
```yaml
web_frontends:
- web-frontends@project-wf1.iam.gserviceaccount.com
- web-frontends@project-wf2.iam.gserviceaccount.com
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| config_folder | Relative path of the folder containing the hierarchical firewall configuration | <code>string</code> | ✓ | |
| templates_folder | Relative path of the folder containing the cidr/service account templates | <code>string</code> | ✓ | |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| vpc-firewall-rules | Generated VPC Firewall Rules | |
<!-- END TFDOC -->

View File

@ -1,143 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
cidrs = try({ for name, cidrs in yamldecode(file("${var.templates_folder}/cidrs.yaml")) :
name => cidrs
}, {})
service_accounts = try({ for name, service_accounts in yamldecode(file("${var.templates_folder}/service_accounts.yaml")) :
name => service_accounts
}, {})
rules = flatten([
for file in fileset(var.config_folder, "**/*.yaml") : [
for key, ruleset in yamldecode(file("${var.config_folder}/${file}")) :
merge(ruleset, {
project_id = split("/", file)[0]
network = split("/", file)[1]
name = "${key}-${split("/", file)[1]}"
source_ranges = try(ruleset.source_ranges, null) == null ? null : flatten(
[for cidr in ruleset.source_ranges :
can(regex("^\\$", cidr))
? local.cidrs[trimprefix(cidr, "$")]
: [cidr]
])
destination_ranges = try(ruleset.destination_ranges, null) == null ? null : flatten(
[for cidr in ruleset.destination_ranges :
can(regex("^\\$", cidr))
? local.cidrs[trimprefix(cidr, "$")]
: [cidr]
])
source_service_accounts = try(ruleset.source_service_accounts, null) == null ? null : flatten(
[for service_account in ruleset.source_service_accounts :
can(regex("^\\$", service_account))
? local.service_accounts[trimprefix(service_account, "$")]
: [service_account]
])
target_service_accounts = try(ruleset.target_service_accounts, null) == null ? null : flatten(
[for service_account in ruleset.target_service_accounts :
can(regex("^\\$", service_account))
? local.service_accounts[trimprefix(service_account, "$")]
: [service_account]
])
})
]
])
rules_allow = [for item in local.rules : item if item.action == "allow"]
rules_deny = [for item in local.rules : item if item.action == "deny"]
}
resource "google_compute_firewall" "rules-allow" {
for_each = { for rule in local.rules_allow : "${rule.network}-${rule.name}" => rule }
project = each.value.project_id
name = each.value.name
description = each.value.description
network = each.value.network
direction = each.value.direction
priority = each.value.priority
source_ranges = try(each.value.source_ranges, each.value.direction == "INGRESS" ? [] : null)
source_tags = try(each.value.source_tags, null)
source_service_accounts = try(each.value.source_service_accounts, null)
destination_ranges = try(each.value.destination_ranges, each.value.direction == "EGRESS" ? [] : null)
target_tags = try(each.value.target_tags, null)
target_service_accounts = try(each.value.target_service_accounts, null)
dynamic "allow" {
for_each = { for proto, ports in try(each.value.ports, []) :
"${proto}-${join("-", ports)}" => {
ports = [for port in ports : tostring(port)]
protocol = proto
}
}
content {
protocol = allow.value.protocol
ports = allow.value.ports
}
}
dynamic "log_config" {
for_each = (each.value.enable_logging == null) || (each.value.enable_logging == false) ? [] : [""]
content {
metadata = "INCLUDE_ALL_METADATA"
}
}
}
resource "google_compute_firewall" "rules-deny" {
for_each = { for rule in local.rules_deny : "${rule.network}-${rule.name}" => rule }
project = each.value.project_id
name = each.value.name
description = each.value.description
network = each.value.network
direction = each.value.direction
priority = each.value.priority
source_ranges = try(each.value.source_ranges, each.value.direction == "INGRESS" ? [] : null)
source_tags = try(each.value.source_tags, null)
source_service_accounts = try(each.value.source_service_accounts, null)
destination_ranges = try(each.value.destination_ranges, each.value.direction == "EGRESS" ? [] : null)
target_tags = try(each.value.target_tags, null)
target_service_accounts = try(each.value.target_service_accounts, null)
dynamic "deny" {
for_each = { for proto, ports in try(each.value.ports, []) :
"${proto}-${join("-", ports)}" => {
ports = [for port in ports : tostring(port)]
protocol = proto
}
}
content {
protocol = deny.value.protocol
ports = deny.value.ports
}
}
dynamic "log_config" {
for_each = (each.value.enable_logging == null) || (each.value.enable_logging == false) ? [] : [""]
content {
metadata = "INCLUDE_ALL_METADATA"
}
}
}

View File

@ -1,20 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "vpc-firewall-rules" {
description = "Generated VPC Firewall Rules"
value = merge(google_compute_firewall.rules-allow, google_compute_firewall.rules-deny)
}

View File

@ -1,25 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "config_folder" {
description = "Relative path of the folder containing the hierarchical firewall configuration"
type = string
}
variable "templates_folder" {
description = "Relative path of the folder containing the cidr/service account templates"
type = string
}

View File

@ -1,29 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_version = ">= 1.0.0"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 4.0.0"
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 4.0.0"
}
}
}

View File

@ -1,4 +1,4 @@
# Google Cloud VPC Firewall Factory - Flat hierarchy
# Google Cloud VPC Firewall Factory
This module allows creation and management of different types of firewall rules by defining them in well formatted `yaml` files.
@ -140,21 +140,21 @@ web-app-a-ingress:
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| config_directories | List of paths to folders where firewall configs are stored in yaml format. Folder may include subfolders with configuration files. Files suffix must be `.yaml` | <code>list&#40;string&#41;</code> | ✓ | |
| network | Name of the network this set of firewall rules applies to. | <code>string</code> | ✓ | |
| project_id | Project Id. | <code>string</code> | ✓ | |
| log_config | Log configuration. Possible values for `metadata` are `EXCLUDE_ALL_METADATA` and `INCLUDE_ALL_METADATA`. Set to `null` for disabling firewall logging. | <code title="object&#40;&#123;&#10; metadata &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| name | description | type | required | default |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------: | :------: | :---------------: |
| config_directories | List of paths to folders where firewall configs are stored in yaml format. Folder may include subfolders with configuration files. Files suffix must be `.yaml` | <code>list&#40;string&#41;</code> | | |
| network | Name of the network this set of firewall rules applies to. | <code>string</code> | | |
| project_id | Project Id. | <code>string</code> | | |
| log_config | Log configuration. Possible values for `metadata` are `EXCLUDE_ALL_METADATA` and `INCLUDE_ALL_METADATA`. Set to `null` for disabling firewall logging. | <code title="object&#40;&#123;&#10; metadata &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| egress_allow_rules | Egress rules with allow blocks. | |
| egress_deny_rules | Egress rules with allow blocks. | |
| ingress_allow_rules | Ingress rules with allow blocks. | |
| ingress_deny_rules | Ingress rules with deny blocks. | |
| name | description | sensitive |
| ------------------- | -------------------------------- | :-------: |
| egress_allow_rules | Egress rules with allow blocks. | |
| egress_deny_rules | Egress rules with allow blocks. | |
| ingress_allow_rules | Ingress rules with allow blocks. | |
| ingress_deny_rules | Ingress rules with deny blocks. | |
<!-- END TFDOC -->

View File

@ -1,71 +0,0 @@
# Google Cloud Resource Factories - VPC Subnets
This module implements a resource factory which allows the creation and management of subnets through properly formatted `yaml` files.
`yaml` configurations are stored on a well-defined folder structure, whose entry point can be customized, and which allows for simple grouping of subnets by Project > VPC.
## Example
### Terraform code
```hcl
module "subnets" {
source = "./factories/subnets"
config_folder = "subnets"
}
# tftest:skip
```
### Configuration Structure
The directory structure implies the project and the VPC each subnet belongs to.
Per the structure below, a subnet named `subnet-a` (after filename `subnet-a.yaml`) will be created on VPC `vpc-alpha-one` which belongs to project `project-alpha`.
Projects and VPCs should exist prior to running this module, or set as an explicit dependency to this module, leveraging `depends_on`.
```bash
└── subnets
├── project-alpha
│ ├── vpc-alpha-one
│ │ ├── subnet-a.yaml
│ │ └── subnet-b.yaml
│ └── vpc-alpha-two
│ └── subnet-c.yaml
└── project-bravo
└── vpc-bravo-one
└── subnet-d.yaml
```
### Subnet definition format and structure
```yaml
region: europe-west1 # Region where the subnet will be creted
description: Sample description # Description
ip_cidr_range: 10.0.0.0/24 # Primary IP range for the subnet
private_ip_google_access: false # Opt- Enables PGA. Defaults to true
iam_users: ["foobar@example.com"] # Opt- Users to grant compute/networkUser to
iam_groups: ["lorem@example.com"] # Opt- Groups to grant compute/networkUser to
iam_service_accounts: ["foobar@project-id.iam.gserviceaccount.com"]
# Opt- SAs to grant compute/networkUser to
secondary_ip_ranges: # Opt- List of secondary IP ranges
- secondary-range-a: 192.168.0.0/24
# Secondary ranges in name: cidr format
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| config_folder | Relative path of the folder containing the subnet configuration | <code>string</code> | ✓ | |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| subnet | Generated subnets | |
<!-- END TFDOC -->

View File

@ -1,72 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
_data = {
for f in fileset(var.config_folder, "**/*.yaml") :
trimsuffix(split("/", f)[2], ".yaml") => merge(
yamldecode(file("${var.config_folder}/${f}")),
{
project_id = split("/", f)[0]
network = split("/", f)[1]
}
)
}
data = {
for k, v in local._data : k => merge(v,
{
network_users : concat(
formatlist("group:%s", try(v.iam_groups, [])),
formatlist("user:%s", try(v.iam_users, [])),
formatlist("serviceAccount:%s", try(v.iam_service_accounts, []))
)
}
)
}
}
resource "google_compute_subnetwork" "default" {
for_each = local.data
project = each.value.project_id
network = each.value.network
name = each.key
region = each.value.region
description = each.value.description
ip_cidr_range = each.value.ip_cidr_range
private_ip_google_access = try(each.value.private_ip_google_access, true)
dynamic "secondary_ip_range" {
for_each = try(each.value.secondary_ip_ranges, [])
iterator = secondary_range
content {
range_name = secondary_range.key
ip_cidr_range = secondary_range.value
}
}
}
resource "google_compute_subnetwork_iam_binding" "default" {
for_each = {
for k, v in local.data : k => v if length(v.network_users) > 0
}
project = each.value.project_id
subnetwork = google_compute_subnetwork.default[each.key].name
region = each.value.region
role = "roles/compute.networkUser"
members = each.value.network_users
}

View File

@ -1,30 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "subnet" {
description = "Generated subnets"
value = {
for k, v in google_compute_subnetwork.default :
k => {
id = v.id
network = v.network
project = v.project
range = v.ip_cidr_range
region = v.region
self_link = v.self_link
}
}
}

View File

@ -1,20 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "config_folder" {
description = "Relative path of the folder containing the subnet configuration"
type = string
}

View File

@ -1,29 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_version = ">= 1.0.0"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 4.0.0"
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 4.0.0"
}
}
}

View File

@ -46,5 +46,5 @@ It is meant to be used as a starting point for most Shared VPC configurations, a
### Decentralized firewall management
<a href="./decentralized-firewall/" title="Decentralized firewall management"><img src="./decentralized-firewall/diagram.png" align="left" width="280px"></a> This [example](./decentralized-firewall/) shows how a decentralized firewall management can be organized using the [firewall factory](../factories/firewall-vpc-rules/).
<a href="./decentralized-firewall/" title="Decentralized firewall management"><img src="./decentralized-firewall/diagram.png" align="left" width="280px"></a> This [example](./decentralized-firewall/) shows how a decentralized firewall management can be organized using the [firewall factory](../factories/net-vpc-firewall-yaml/).
<br clear="left">

View File

@ -1,6 +1,6 @@
# Decentralized firewall management
This sample shows how a decentralized firewall management can be organized using the [firewall factory](../../factories/firewall-vpc-rules/).
This sample shows how a decentralized firewall management can be organized using the [firewall factory](../../factories/net-vpc-firewall-yaml/README.md).
This approach is a good fit when Shared VPCs are used across multiple application/infrastructure teams. A central repository keeps environment/team
specific folders with firewall definitions in `yaml` format.
@ -25,22 +25,22 @@ in the [`validator/`](validator/) subdirectory, which can be integrated as part
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| billing_account_id | Billing account id used as default for new projects. | <code>string</code> | ✓ | |
| prefix | Prefix used for resources that need unique names. | <code>string</code> | ✓ | |
| root_node | Hierarchy node where projects will be created, 'organizations/org_id' or 'folders/folder_id'. | <code>string</code> | ✓ | |
| ip_ranges | Subnet IP CIDR ranges. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; prod &#61; &#34;10.0.16.0&#47;24&#34;&#10; dev &#61; &#34;10.0.32.0&#47;24&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
| project_services | Service APIs enabled by default in new projects. | <code>list&#40;string&#41;</code> | | <code title="&#91;&#10; &#34;container.googleapis.com&#34;,&#10; &#34;dns.googleapis.com&#34;,&#10; &#34;stackdriver.googleapis.com&#34;,&#10;&#93;">&#91;&#8230;&#93;</code> |
| region | Region used. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
| name | description | type | required | default |
| ------------------ | --------------------------------------------------------------------------------------------- | :-------------------------------: | :------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| billing_account_id | Billing account id used as default for new projects. | <code>string</code> | | |
| prefix | Prefix used for resources that need unique names. | <code>string</code> | | |
| root_node | Hierarchy node where projects will be created, 'organizations/org_id' or 'folders/folder_id'. | <code>string</code> | | |
| ip_ranges | Subnet IP CIDR ranges. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; prod &#61; &#34;10.0.16.0&#47;24&#34;&#10; dev &#61; &#34;10.0.32.0&#47;24&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
| project_services | Service APIs enabled by default in new projects. | <code>list&#40;string&#41;</code> | | <code title="&#91;&#10; &#34;container.googleapis.com&#34;,&#10; &#34;dns.googleapis.com&#34;,&#10; &#34;stackdriver.googleapis.com&#34;,&#10;&#93;">&#91;&#8230;&#93;</code> |
| region | Region used. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| fw_rules | Firewall rules. | |
| projects | Project ids. | |
| vpc | Shared VPCs. | |
| name | description | sensitive |
| -------- | --------------- | :-------: |
| fw_rules | Firewall rules. | |
| projects | Project ids. | |
| vpc | Shared VPCs. | |
<!-- END TFDOC -->

View File

@ -109,7 +109,7 @@ module "dns-api-dev" {
###############################################################################
module "vpc-firewall-prod" {
source = "../../factories/firewall-vpc-rules/flat"
source = "../../factories/net-vpc-firewall-yaml"
project_id = module.project-host-prod.project_id
network = module.vpc-prod.name
@ -125,7 +125,7 @@ module "vpc-firewall-prod" {
}
module "vpc-firewall-dev" {
source = "../../factories/firewall-vpc-rules/flat"
source = "../../factories/net-vpc-firewall-yaml"
project_id = module.project-host-dev.project_id
network = module.vpc-dev.name

451
modules/net-xlb/README.md Normal file
View File

@ -0,0 +1,451 @@
# External Global (HTTP/S) Load Balancer Module
The module allows managing External Global HTTP/HTTPS Load Balancers (XGLB), integrating the global forwarding rule, the url-map, the backends (supporting buckets and groups), and optional health checks and SSL certificates (both managed and unmanaged). It's designed to be a simple match for the [`gcs`](../gcs) and the [`compute-vm`](../compute-vm) modules, which can be used to manage GCS buckets, instance templates and instance groups.
## Examples
### GCS Bucket Minimal Example
This is a minimal example, which creates a global HTTP load balancer, pointing the path `/` to an existing GCS bucket called `my_test_bucket`.
```hcl
module "xlb" {
source = "./modules/net-xlb"
name = "xlb-test"
project_id = var.project_id
backend_services_config = {
my-bucket-backend = {
bucket_config = {
bucket_name = "my_test_bucket"
options = null
}
group_config = null
enable_cdn = false
cdn_config = null
}
}
}
# tftest:modules=1:resources=4
```
### Group Backend Service Minimal Example
A very similar coniguration also applies to GCE instance groups:
```hcl
module "xlb" {
source = "./modules/net-xlb"
name = "xlb-test"
project_id = var.project_id
backend_services_config = {
my-group-backend = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_test_group"
options = null
}
],
health_checks = []
log_config = null
options = null
}
}
}
}
# tftest:modules=1:resources=5
```
### Health Checks For Group Backend Services
Group backend services support health checks.
If no health checks are specified, a default HTTP health check is created and associated to each group backend service. The default health check configuration can be modified through the `health_checks_config_defaults` variable.
Alternatively, one or more health checks can be either contextually created or attached, if existing. If the id of the health checks specified is equal to one of the keys of the `health_checks_config` variable, the health check is contextually created; otherwise, the health check id is used as is, assuming an health check alredy exists.
For example, to contextually create a health check and attach it to the backend service:
```hcl
module "xlb" {
source = "./modules/net-xlb"
name = "xlb-test"
project_id = var.project_id
backend_services_config = {
my-group-backend = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_test_group"
options = null
}
],
health_checks = ["hc_1"]
log_config = null
options = null
}
}
}
health_checks_config = {
hc_1 = {
type = "http"
logging = true
options = {
timeout_sec = 40
}
check = {
port_specification = "USE_SERVING_PORT"
}
}
}
}
# tftest:modules=1:resources=5
```
### Mixing Backends
Backends can be multiple, group and bucket backends can be mixed and group backends support multiple groups.
```hcl
module "xlb" {
source = "./modules/net-xlb"
name = "xlb-test"
project_id = var.project_id
backend_services_config = {
my-group-backend = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_test_group"
options = null
}
],
health_checks = []
log_config = null
options = null
}
},
another-group-backend = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_other_test_group"
options = null
}
],
health_checks = []
log_config = null
options = null
}
},
a-bucket-backend = {
bucket_config = {
bucket_name = "my_test_bucket"
options = null
}
group_config = null
enable_cdn = false
cdn_config = null
}
}
}
# tftest:modules=1:resources=7
```
### Url-map
The url-map can be customized with lots of different configurations. This includes leveraging multiple backends in different parts of the configuration.
Given its complexity, it's left to the user passing the right data structure.
For simplicity, *if no configurations are given* the first backend service defined (in alphabetical order, with priority to bucket backend services, if any) is used as the *default_service*, thus answering to the root (*/*) path.
Backend services can be specified as needed in the url-map configuration, referencing the id used to declare them in the backend services map. If a corresponding backend service is found, their object id is automatically used; otherwise, it is assumed that the string passed is the id of an already existing backend and it is given to the provider as it was passed.
In this example, we're using one backend service as the default backend
```hcl
module "xlb" {
source = "./modules/net-xlb"
name = "xlb-test"
project_id = var.project_id
url_map_config = {
default_service = "my-group-backend"
default_route_action = null
default_url_redirect = null
tests = null
header_action = null
host_rules = []
path_matchers = [
{
name = "my-example-page"
path_rules = [
{
paths = ["/my-example-page"]
service = "another-group-backend"
}
]
}
]
}
backend_services_config = {
my-group-backend = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_test_group"
options = null
}
],
health_checks = []
log_config = null
options = null
}
},
my-example-page = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_other_test_group"
options = null
}
],
health_checks = []
log_config = null
options = null
}
}
}
}
# tftest:modules=1:resources=6
```
### Reserve a static IP address
Optionally, a static IP address can be reserved:
```hcl
module "xlb" {
source = "./modules/net-xlb"
name = "xlb-test"
project_id = var.project_id
reserve_ip_address = true
backend_services_config = {
my-group-backend = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_test_group"
options = null
}
],
health_checks = []
log_config = null
options = null
}
}
}
}
# tftest:modules=1:resources=6
```
### HTTPS And SSL Certificates
HTTPS is disabled by default but it can be optionally enabled.
The module supports both managed and unmanaged certificates, and they can be either created contextually with other resources or attached, if already existing.
If no `ssl_certificates_config` variable is specified, a managed certificate for the domain *example.com* is automatically created.
```hcl
module "xlb" {
source = "./modules/net-xlb"
name = "xlb-test"
project_id = var.project_id
https = true
backend_services_config = {
my-group-backend = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_test_group"
options = null
}
],
health_checks = []
log_config = null
options = null
}
}
}
}
# tftest:modules=1:resources=6
```
Otherwise, SSL certificates can be explicitely defined. In this case, they'll need to be referenced from the `target_proxy_https_config.ssl_certificates` variable.
If the ids specified in the `target_proxy_https_config` variable are not found in the `ssl_certificates_config` map, they are used as is, assuming the ssl certificates already exist.
```hcl
module "xlb" {
source = "./modules/net-xlb"
name = "xlb-test"
project_id = var.project_id
https = true
ssl_certificates_config = {
my-domain = {
domains = [
"my-domain.com"
],
unmanaged_config = null
}
}
target_proxy_https_config = {
ssl_certificates = [
"my-domain",
"an-existing-cert"
]
}
backend_services_config = {
my-group-backend = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_test_group"
options = null
}
]
health_checks = []
log_config = null
options = null
}
}
}
}
# tftest:modules=1:resources=6
```
Using unamanged certificates is also possible. Here is an example:
```hcl
module "xlb" {
source = "./modules/net-xlb"
name = "xlb-test"
project_id = var.project_id
https = true
ssl_certificates_config = {
my-domain = {
domains = [
"my-domain.com"
],
unmanaged_config = {
tls_private_key = nonsensitive(tls_private_key.self_signed_key.private_key_pem)
tls_self_signed_cert = nonsensitive(tls_self_signed_cert.self_signed_cert.cert_pem)
}
}
}
target_proxy_https_config = {
ssl_certificates = [
"my-domain"
]
}
backend_services_config = {
my-group-backend = {
bucket_config = null
enable_cdn = false
cdn_config = null
group_config = {
backends = [
{
group = "my_test_group"
options = null
}
],
health_checks = []
log_config = null
options = null
}
}
}
}
resource "tls_private_key" "self_signed_key" {
algorithm = "RSA"
rsa_bits = 2048
}
resource "tls_self_signed_cert" "self_signed_cert" {
key_algorithm = tls_private_key.self_signed_key.algorithm
private_key_pem = tls_private_key.self_signed_key.private_key_pem
validity_period_hours = 12
early_renewal_hours = 3
dns_names = ["example.com"]
allowed_uses = [
"key_encipherment",
"digital_signature",
"server_auth"
]
subject {
common_name = "example.com"
organization = "My Test Org"
}
}
# tftest:modules=1:resources=6
```
## Components And Files Mapping
An External Global Load Balancer is made of multiple components, that change depending on the configurations. Sometimes, it may be tricky to understand what they are, and how they relate to each other. Following, we provide a very brief overview to become more familiar with them.
* The global load balancer ([global_forwarding_rule.tf](global_forwarding_rule.tf)) binds a frontend public Virtual IP (VIP) (ip_address.tf)[ip_address.tf] to an HTTP(S) target proxy ((target_proxy.tf)[target_proxy.tf]).
* If the target proxy ((target_proxy.tf)[target_proxy.tf]) is HTTPS, it requires one or more -managed or unmanaged- SSL certificates ((ssl_certificates.tf)[ssl_certificates.tf]).
* Target proxies ((target_proxy.tf)[target_proxy.tf]) leverage url-maps ((url_map.tf)[url_map.tf]): set of L7 rules, which create a mapping between specific hostnames, URIs (and more) to one or more backends services ((backend_services.tf)[backend_services.tf]).
* Backends services ((backend_services.tf)[backend_services.tf]) can either link to a bucket or -one or multiple- groups, which can be GCE instance groups or NEGs. It is assumed in this module that buckets and groups are previously created through other modules, and passed in as input variables.
* Backends services ((backend_services.tf)[backend_services.tf]) support one or more health checks ((health_cheks.tf)[health_checks.tf]), used to verify that the backend is indeed healthy, so that traffic can be forwarded to it. Health checks currently supported in this module are HTTP, HTTPS, HTTP2, SSL, TCP.

View File

@ -0,0 +1,212 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
backend_services_bucket = {
for k, v in coalesce(var.backend_services_config, {}) :
k => v if v.bucket_config != null
}
backend_services_group = {
for k, v in coalesce(var.backend_services_config, {}) :
k => v if v.group_config != null
}
}
resource "google_compute_backend_bucket" "bucket" {
for_each = local.backend_services_bucket
name = "${var.name}-${each.key}"
description = "Terraform managed."
project = var.project_id
bucket_name = try(each.value.bucket_config.bucket_name, null)
custom_response_headers = try(each.value.bucket_config.options.custom_response_headers, null)
enable_cdn = try(each.value.enable_cdn, null)
dynamic "cdn_policy" {
for_each = try(each.value.cdn_policy, null) == null ? [] : [each.value.cdn_policy]
content {
cache_mode = try(cdn_policy.value.cache_mode, null)
client_ttl = try(cdn_policy.value.client_ttl, null)
default_ttl = try(cdn_policy.value.default_ttl, null)
max_ttl = try(cdn_policy.value.max_ttl, null)
negative_caching = try(cdn_policy.value.negative_caching, null)
serve_while_stale = try(cdn_policy.value.serve_while_stale, null)
signed_url_cache_max_age_sec = try(cdn_policy.value.signed_url_cache_max_age_sec, null)
dynamic "negative_caching_policy" {
for_each = try(cdn_policy.value.negative_caching_policy, null) == null ? [] : [cdn_policy.value.negative_caching_policy]
iterator = ncp
content {
code = ncp.value.code
ttl = ncp.value.ttl
}
}
}
}
}
resource "google_compute_backend_service" "group" {
for_each = local.backend_services_group
name = "${var.name}-${each.key}"
project = var.project_id
description = "Terraform managed."
affinity_cookie_ttl_sec = try(each.value.group_config.options.affinity_cookie_ttl_sec, null)
enable_cdn = try(each.value.enable_cdn, null)
custom_request_headers = try(each.value.group_config.options.custom_request_headers, null)
custom_response_headers = try(each.value.group_config.options.custom_response_headers, null)
connection_draining_timeout_sec = try(each.value.group_config.options.connection_draining_timeout_sec, null)
load_balancing_scheme = try(each.value.group_config.options.load_balancing_scheme, null)
locality_lb_policy = try(each.value.group_config.options.locality_lb_policy, null)
port_name = try(each.value.group_config.options.port_name, null)
protocol = try(each.value.group_config.options.protocol, null)
security_policy = try(each.value.group_config.options.security_policy, null)
session_affinity = try(each.value.group_config.options.session_affinity, null)
timeout_sec = try(each.value.group_config.options.timeout_sec, null)
# If no health checks are defined, use the default one.
# Otherwise, look in the health_checks_config map.
# Otherwise, use the health_check id as is (already existing).
health_checks = (
try(each.value.group_config.health_checks, null) == null
|| length(try(each.value.group_config.health_checks, [])) == 0
? try(
[google_compute_health_check.health_check["default"].self_link],
null
)
: [
for hc in each.value.group_config.health_checks :
try(
google_compute_health_check.health_check[hc].id,
hc
)
]
)
dynamic "backend" {
for_each = try(each.value.group_config.backends, [])
content {
balancing_mode = try(backend.value.options.balancing_mode, null)
capacity_scaler = try(backend.value.options.capacity_scaler, null)
group = try(backend.value.group, null)
max_connections = try(backend.value.options.max_connections, null)
max_connections_per_instance = try(backend.value.options.max_connections_per_instance, null)
max_connections_per_endpoint = try(backend.value.options.max_connections_per_endpoint, null)
max_rate = try(backend.value.options.max_rate, null)
max_rate_per_instance = try(backend.value.options.max_rate_per_instance, null)
max_rate_per_endpoint = try(backend.value.options.max_rate_per_endpoint, null)
max_utilization = try(backend.value.options.max_utilization, null)
}
}
dynamic "circuit_breakers" {
for_each = (
try(each.value.group_config.options.circuit_breakers, null) == null
? []
: [each.value.group_config.options.circuit_breakers]
)
iterator = cb
content {
max_requests_per_connection = try(cb.value.max_requests_per_connection, null)
max_connections = try(cb.value.max_connections, null)
max_pending_requests = try(cb.value.max_pending_requests, null)
max_requests = try(cb.value.max_requests, null)
max_retries = try(cb.value.max_retries, null)
}
}
dynamic "consistent_hash" {
for_each = (
try(each.value.group_config.options.consistent_hash, null) == null
? []
: [each.value.group_config.options.consistent_hash]
)
content {
http_header_name = try(consistent_hash.value.http_header_name, null)
minimum_ring_size = try(consistent_hash.value.minimum_ring_size, null)
dynamic "http_cookie" {
for_each = try(consistent_hash.value.http_cookie, null) == null ? [] : [consistent_hash.value.http_cookie]
content {
name = try(http_cookie.value.name, null)
path = try(http_cookie.value.path, null)
dynamic "ttl" {
for_each = try(consistent_hash.value.ttl, null) == null ? [] : [consistent_hash.value.ttl]
content {
seconds = try(ttl.value.seconds, null) # Must be from 0 to 315,576,000,000 inclusive
nanos = try(ttl.value.nanos, null) # Must be from 0 to 999,999,999 inclusive
}
}
}
}
}
}
dynamic "cdn_policy" {
for_each = (
try(each.value.cdn_policy, null) == null
? []
: [each.value.cdn_policy]
)
iterator = cdn_policy
content {
signed_url_cache_max_age_sec = try(cdn_policy.value.signed_url_cache_max_age_sec, null)
default_ttl = try(cdn_policy.value.default_ttl, null)
max_ttl = try(cdn_policy.value.max_ttl, null)
client_ttl = try(cdn_policy.value.client_ttl, null)
negative_caching = try(cdn_policy.value.negative_caching, null)
cache_mode = try(cdn_policy.value.cache_mode, null)
serve_while_stale = try(cdn_policy.value.serve_while_stale, null)
dynamic "negative_caching_policy" {
for_each = (
try(cdn_policy.value.negative_caching_policy, null) == null
? []
: [cdn_policy.value.negative_caching_policy]
)
iterator = ncp
content {
code = try(ncp.value.code, null)
ttl = try(ncp.value.ttl, null)
}
}
}
}
dynamic "iap" {
for_each = (
try(each.value.group_config.options.iap, null) == null
? []
: [each.value.group_config.options.iap]
)
content {
oauth2_client_id = try(iap.value.oauth2_client_id, null)
oauth2_client_secret = try(iap.value.oauth2_client_secret, null) # sensitive
oauth2_client_secret_sha256 = try(iap.value.oauth2_client_secret_sha256, null) # sensitive
}
}
dynamic "log_config" {
for_each = (
try(each.value.group_config.log_config, null) == null
? []
: [each.value.group_config.log_config]
)
content {
enable = try(log_config.value.enable, null)
sample_rate = try(log_config.value.sample_rate, null)
}
}
}

View File

@ -0,0 +1,41 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
ip_address = (
var.reserve_ip_address
? google_compute_global_address.static_ip.0.id
: null
)
target = (
var.https
? google_compute_target_https_proxy.https.0.id
: google_compute_target_http_proxy.http.0.id
)
}
resource "google_compute_global_forwarding_rule" "forwarding_rule" {
provider = google-beta
name = var.name
project = var.project_id
description = "Terraform managed."
ip_protocol = var.global_forwarding_rule_config.ip_protocol
load_balancing_scheme = var.global_forwarding_rule_config.load_balancing_scheme
port_range = var.global_forwarding_rule_config.port_range
target = local.target
ip_address = local.ip_address
}

View File

@ -0,0 +1,142 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
# Get group backend services without health checks defined
_backends_without_hcs = [
for k, v in coalesce(var.backend_services_config, {}) :
v if(
v.group_config != null
&& (
try(v.group_config.health_checks, null) == null
|| length(try(v.group_config.health_checks, [])) == 0
)
)
]
# If at least one group backend service without
# HC is defined, create also a default HC
health_checks_config = (
length(local._backends_without_hcs) > 0
? merge(
{ default = var.health_checks_config_defaults },
coalesce(var.health_checks_config, {})
)
: coalesce(var.health_checks_config, {})
)
}
resource "google_compute_health_check" "health_check" {
for_each = local.health_checks_config
provider = google-beta
name = "${var.name}-${each.key}"
project = var.project_id
description = "Terraform managed."
check_interval_sec = try(each.value.options.check_interval_sec, null)
healthy_threshold = try(each.value.options.healthy_threshold, null)
timeout_sec = try(each.value.options.timeout_sec, null)
unhealthy_threshold = try(each.value.options.unhealthy_threshold, null)
dynamic "http_health_check" {
for_each = (
try(each.value.type, null) == "http" || try(each.value.type, null) == null
? { 1 = 1 }
: {}
)
content {
host = try(each.value.check.host, null)
port = try(each.value.check.port, null)
port_name = try(each.value.check.port_name, null)
port_specification = try(each.value.check.port_specification, null)
proxy_header = try(each.value.check.proxy_header, null)
request_path = try(each.value.check.request_path, null)
response = try(each.value.check.response, null)
}
}
dynamic "https_health_check" {
for_each = (
try(each.value.type, null) == "https" || try(each.value.type, null) == null
? { 1 = 1 }
: {}
)
content {
host = try(each.value.check.host, null)
port = try(each.value.check.port, null)
port_name = try(each.value.check.port_name, null)
port_specification = try(each.value.check.port_specification, null)
proxy_header = try(each.value.check.proxy_header, null)
request_path = try(each.value.check.request_path, null)
response = try(each.value.check.response, null)
}
}
dynamic "tcp_health_check" {
for_each = (
try(each.value.type, null) == "tcp" || try(each.value.type, null) == null
? { 1 = 1 }
: {}
)
content {
port = try(each.value.check.port, null)
port_name = try(each.value.check.port_name, null)
port_specification = try(each.value.check.port_specification, null)
proxy_header = try(each.value.check.proxy_header, null)
request = try(each.value.check.request, null)
response = try(each.value.check.response, null)
}
}
dynamic "ssl_health_check" {
for_each = (
try(each.value.type, null) == "ssl" || try(each.value.type, null) == null
? { 1 = 1 }
: {}
)
content {
port = try(each.value.check.port, null)
port_name = try(each.value.check.port_name, null)
port_specification = try(each.value.check.port_specification, null)
proxy_header = try(each.value.check.proxy_header, null)
request = try(each.value.check.request, null)
response = try(each.value.check.response, null)
}
}
dynamic "http2_health_check" {
for_each = (
try(each.value.type, null) == "http2" || try(each.value.type, null) == null
? { 1 = 1 }
: {}
)
content {
host = try(each.value.check.host, null)
port = try(each.value.check.port, null)
port_name = try(each.value.check.port_name, null)
port_specification = try(each.value.check.port_specification, null)
proxy_header = try(each.value.check.proxy_header, null)
request_path = try(each.value.check.request_path, null)
response = try(each.value.check.response, null)
}
}
dynamic "log_config" {
for_each = try(each.value.logging, false) ? { 0 = 0 } : {}
content {
enable = true
}
}
}

View File

@ -14,8 +14,10 @@
* limitations under the License.
*/
module "subnets" {
source = "../../subnets"
config_folder = "conf"
resource "google_compute_global_address" "static_ip" {
count = var.reserve_ip_address ? 1 : 0
provider = google-beta
name = var.name
project = var.project_id
description = "Terraform managed."
}

View File

@ -0,0 +1,60 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "health_checks" {
description = "Health-check resources."
value = try(google_compute_health_check.health_check, [])
}
output "backend_services" {
description = "Backend service resources."
value = {
bucket = try(google_compute_backend_bucket.bucket, [])
group = try(google_compute_backend_service.group, [])
}
}
output "url_map" {
description = "The url-map."
value = google_compute_url_map.url_map
}
output "ssl_certificates" {
description = "The SSL certificate."
value = try(
google_compute_managed_ssl_certificate.managed,
google_compute_ssl_certificate.unmanaged,
null
)
}
output "ip_address" {
description = "The reserved global IP address."
value = try(google_compute_global_address.static_ip, null)
}
output "target_proxy" {
description = "The target proxy."
value = try(
google_compute_target_http_proxy.http,
google_compute_target_https_proxy.https
)
}
output "global_forwarding_rule" {
description = "The global forwarding rule."
value = google_compute_global_forwarding_rule.forwarding_rule
}

View File

@ -0,0 +1,51 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
managed = (
var.https
? {
for k, v in coalesce(var.ssl_certificates_config, {}) :
k => v if v.unmanaged_config == null
}
: {}
)
unmanaged = (
var.https
? {
for k, v in coalesce(var.ssl_certificates_config, {}) :
k => v if v.unmanaged_config != null
}
: {}
)
}
resource "google_compute_managed_ssl_certificate" "managed" {
for_each = local.managed
project = var.project_id
name = "${var.name}-${each.key}"
managed {
domains = try(each.value.domains, null)
}
}
resource "google_compute_ssl_certificate" "unmanaged" {
for_each = local.unmanaged
project = var.project_id
name = "${var.name}-${each.key}"
certificate = try(each.value.unmanaged_config.tls_self_signed_cert, null)
private_key = try(each.value.unmanaged_config.tls_private_key, null)
}

View File

@ -0,0 +1,50 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
# If no SSL certificates are defined, use the default one.
# Otherwise, look in the ssl_certificates_config map.
# Otherwise, use the SSL certificate id as is (already existing).
ssl_certificates = (
var.https == true
? [
for cert in try(var.target_proxy_https_config.ssl_certificates, ["default"]) :
try(
google_compute_managed_ssl_certificate.managed[cert].id,
google_compute_ssl_certificate.unmanaged[cert].id,
cert
)
]
: []
)
}
resource "google_compute_target_http_proxy" "http" {
count = var.https ? 0 : 1
name = var.name
project = var.project_id
description = "Terraform managed."
url_map = google_compute_url_map.url_map.id
}
resource "google_compute_target_https_proxy" "https" {
count = var.https ? 1 : 0
name = var.name
project = var.project_id
description = "Terraform managed."
url_map = google_compute_url_map.url_map.id
ssl_certificates = local.ssl_certificates
}

1180
modules/net-xlb/url_map.tf Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,218 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "name" {
description = "Load balancer name."
type = string
}
variable "project_id" {
description = "Project id."
type = string
}
variable "health_checks_config_defaults" {
description = "Auto-created health check default configuration."
type = object({
type = string # http https tcp ssl http2
check = map(any) # actual health check block attributes
options = map(number) # interval, thresholds, timeout
logging = bool
})
default = {
type = "http"
logging = false
options = {}
check = {
port_specification = "USE_SERVING_PORT"
}
}
}
variable "health_checks_config" {
description = "Custom health checks configuration."
type = map(object({
type = string # http https tcp ssl http2
check = map(any) # actual health check block attributes
options = map(number) # interval, thresholds, timeout
logging = bool
}))
default = {}
}
variable "backend_services_config" {
description = "The backends services configuration."
type = map(object({
enable_cdn = bool
cdn_config = object({
cache_mode = string
client_ttl = number
default_ttl = number
max_ttl = number
negative_caching = bool
negative_caching_policy = map(number)
serve_while_stale = bool
signed_url_cache_max_age_sec = string
})
bucket_config = object({
bucket_name = string
options = object({
custom_response_headers = list(string)
})
})
group_config = object({
backends = list(object({
group = string # IG or NEG FQDN address
options = object({
balancing_mode = string # Can be UTILIZATION, RATE, CONNECTION
capacity_scaler = number # Valid range is [0.0,1.0]
max_connections = number
max_connections_per_instance = number
max_connections_per_endpoint = number
max_rate = number
max_rate_per_instance = number
max_rate_per_endpoint = number
max_utilization = number
})
}))
# Optional health check ids for backend service groups.
# Will lookup for ids in health_chacks_config first,
# then will use the id as is. If no ids are defined
# at all (null, []) health_checks_config_defaults is used
health_checks = list(string)
log_config = object({
enable = bool
sample_rate = number # must be in [0, 1]
})
options = object({
affinity_cookie_ttl_sec = number
custom_request_headers = list(string)
custom_response_headers = list(string)
connection_draining_timeout_sec = number
load_balancing_scheme = string # only EXTERNAL (default) makes sense here
locality_lb_policy = string
port_name = string
protocol = string
security_policy = string
session_affinity = string
timeout_sec = number
circuits_breakers = object({
max_requests_per_connection = number # Set to 1 to disable keep-alive
max_connections = number # Defaults to 1024
max_pending_requests = number # Defaults to 1024
max_requests = number # Defaults to 1024
max_retries = number # Defaults to 3
})
consistent_hash = object({
http_header_name = string
minimum_ring_size = string
http_cookie = object({
name = string
path = string
ttl = object({
seconds = number
nanos = number
})
})
})
iap = object({
oauth2_client_id = string
oauth2_client_secret = string
oauth2_client_secret_sha256 = string
})
})
})
}))
default = {}
}
variable "url_map_config" {
description = "The url-map configuration."
type = object({
default_service = string
default_route_action = any
default_url_redirect = map(any)
header_action = any
host_rules = list(any)
path_matchers = list(any)
tests = list(map(string))
})
default = null
}
variable "ssl_certificates_config" {
description = "The SSL certificate configuration."
type = map(object({
domains = list(string)
# If unmanaged_config is null, the certificate will be managed
unmanaged_config = object({
tls_private_key = string
tls_self_signed_cert = string
})
}))
default = {
default = {
domains = ["example.com"],
unmanaged_config = null
}
}
}
variable "target_proxy_https_config" {
description = "The HTTPS target proxy configuration."
type = object({
ssl_certificates = list(string)
})
default = null
}
variable "global_forwarding_rule_config" {
description = "Global forwarding rule configurations."
type = object({
ip_protocol = string
ip_version = string
load_balancing_scheme = string
port_range = string
})
default = {
load_balancing_scheme = "EXTERNAL"
ip_protocol = "TCP"
ip_version = "IPV4"
port_range = "80" # 80, 8080, 443
}
}
variable "https" {
description = "Whether to enable HTTPS."
type = bool
default = false
}
variable "reserve_ip_address" {
description = "Whether to reserve a static global IP address."
type = bool
default = false
}

View File

@ -1,13 +0,0 @@
# skip boilerplate check
allow-ssh-from-onprem:
description: Enable SSH for onprem ranges
direction: INGRESS
action: allow
priority: 1001
source_ranges:
- $example
ports:
tcp: ["22"]
target_resources: null
enable_logging: false

View File

@ -1,13 +0,0 @@
# skip boilerplate check
allow-icmp:
description: Enable ICMP for all hosts
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- 0.0.0.0/0
ports:
icmp: []
target_resources: null
enable_logging: false

View File

@ -1,10 +0,0 @@
# skip boilerplate check
example:
- 10.0.0.0/24
- 10.0.10.0/24
- 192.168.1.1/32
healthcheck:
- 35.191.0.0/16
- 130.211.0.0/22

View File

@ -1,4 +0,0 @@
# skip boilerplate check
example:
- example-service-account@resource-factory-playground.iam.gserviceaccount.com

View File

@ -1,21 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "hierarchical-firewall-rules" {
source = "../../../../../examples/factories/firewall-hierarchical-policies/"
config_folder = "conf/rules"
templates_folder = "conf/templates"
}

View File

@ -1,47 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
def test_firewall(plan_runner):
"Test hierarchical firewall rules from conf/rules"
_, resources = plan_runner()
assert len(resources) == 6
assert set(r["type"] for r in resources) == set([
"google_compute_firewall_policy_rule", "google_compute_firewall_policy_association", "google_compute_firewall_policy"
])
rule_ssh = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy_rule"
and r["values"]["priority"] == 1001]
rule_icmp = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy_rule"
and r["values"]["priority"] == 1000]
association_org = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy_association"
and r["values"]["attachment_target"] == "organizations/1234567890"]
association_folder = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy_association"
and r["values"]["attachment_target"] == "folders/0987654321"]
policies_org = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy"
and r["values"]["parent"] == "organizations/1234567890"]
policies_folder = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy"
and r["values"]["parent"] == "folders/0987654321"]
assert set(rule_ssh[0]["match"][0]["src_ip_ranges"]) == set(
["10.0.0.0/24", "10.0.10.0/24", "192.168.1.1/32"])
assert rule_icmp[0]["match"][0]["layer4_configs"][0]["ip_protocol"] == "icmp"
assert association_org[0]["name"] == "hierarchical-fw-policy-organizations-1234567890"
assert association_folder[0]["name"] == "hierarchical-fw-policy-folders-0987654321"
assert policies_org[0]["short_name"] == "hierarchical-fw-policy-organizations-1234567890"
assert policies_folder[0]["short_name"] == "hierarchical-fw-policy-folders-0987654321"

View File

@ -15,7 +15,7 @@
*/
module "firewall" {
source = "../../../../../../examples/factories/firewall-vpc-rules/flat"
source = "../../../../../examples/factories/net-vpc-firewall-yaml"
project_id = "my-project"
network = "my-network"
config_directories = [

View File

@ -1,13 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

View File

@ -1,8 +0,0 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-a, vpc-a
secondary_ip_ranges:
secondary-range-a: 192.168.0.0/24
secondary-range-b: 192.168.1.0/24

View File

@ -1,6 +0,0 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.1.0/24
description: Sample Subnet in project project-a, vpc-a
private_ip_google_access: false

View File

@ -1,7 +0,0 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.0.2.0/24
description: Sample Subnet in project project-a, vpc-b
iam_users: ["sruffilli@google.com"]
iam_groups: []

View File

@ -1,7 +0,0 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 172.16.0.0/24
description: Sample Subnet in project project-b, vpc-x
iam_users: ["sruffilli@google.com"]
iam_groups: []

View File

@ -1,20 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "subnets" {
source = "../../../../../examples/factories/subnets"
config_folder = "conf"
}

View File

@ -1,60 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
def test_firewall(plan_runner):
"Test hierarchical firewall rules from conf/rules"
_, resources = plan_runner()
assert len(resources) == 6
assert set(r["type"] for r in resources) == set(
["google_compute_subnetwork", "google_compute_subnetwork_iam_binding"])
subnets = [
r["values"] for r in resources
if r["type"] == "google_compute_subnetwork"
]
iam_bindings = [
r["values"] for r in resources
if r["type"] == "google_compute_subnetwork_iam_binding"
]
subnet_a_a = [
s for s in subnets if s["project"] == "project-a"
and s["network"] == "vpc-a" and s["name"] == "subnet-a"
][0]
assert subnet_a_a["ip_cidr_range"] == "10.0.0.0/24"
assert subnet_a_a["private_ip_google_access"] == True
assert subnet_a_a["region"] == "europe-west1"
assert subnet_a_a["secondary_ip_range"] == [{
"ip_cidr_range":
"192.168.0.0/24",
"range_name":
"secondary-range-a"
}, {
"ip_cidr_range":
"192.168.1.0/24",
"range_name":
"secondary-range-b"
}]
subnet_a_b = [
s for s in subnets if s["project"] == "project-a"
and s["network"] == "vpc-a" and s["name"] == "subnet-b"
][0]
assert subnet_a_b["private_ip_google_access"] == False
iam_binding_b_alpha = [
b for b in iam_bindings if b["project"] == "project-b"
][0]
assert set(iam_binding_b_alpha["members"]) == set(
["user:sruffilli@google.com"])
assert iam_binding_b_alpha["role"] == "roles/compute.networkUser"
assert iam_binding_b_alpha["subnetwork"] == "subnet-alpha"

View File

@ -1,13 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

View File

@ -1,13 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

View File

@ -1,13 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

View File

@ -1,25 +0,0 @@
# skip boilerplate check
allow-healthchecks:
description: "Allow traffic from healthcheck"
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- $healthcheck
ports:
tcp: ["80"]
enable_logging: false
allow-http:
description: "Allow traffic to LB backend"
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- 0.0.0.0/0
target_service_accounts:
- example-service-account@resource-factory-playground.iam.gserviceaccount.com
ports:
tcp: ["80", "443"]
enable_logging: true

View File

@ -1,10 +0,0 @@
# skip boilerplate check
example:
- 10.0.0.0/24
- 10.0.10.0/24
- 192.168.1.1/32
healthcheck:
- 35.191.0.0/16
- 130.211.0.0/22

View File

@ -1,4 +0,0 @@
# skip boilerplate check
couchbase:
- example-service-account@resource-factory-playground.iam.gserviceaccount.com

View File

@ -1,21 +0,0 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "vpc-firewall-rules" {
source = "../../../../../../examples/factories/firewall-vpc-rules/nested"
config_folder = "conf/rules"
templates_folder = "conf/templates"
}

View File

@ -1,38 +0,0 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
def test_firewall(plan_runner):
"Test hierarchical firewall rules from conf/rules"
_, resources = plan_runner()
assert len(resources) == 2
assert set(r["type"]
for r in resources) == set(["google_compute_firewall"])
rule_hc = [
r["values"] for r in resources
if r["values"]["name"] == "allow-healthchecks-vpc-a"
][0]
rule_be = [
r["values"] for r in resources
if r["values"]["description"] == "Allow traffic to LB backend"
][0]
assert set(rule_hc["source_ranges"]) == set(
["130.211.0.0/22", "35.191.0.0/16"])
assert rule_hc["direction"] == "INGRESS"
assert rule_hc["network"] == "vpc-a"
assert rule_hc["priority"] == 1000
assert rule_hc["project"] == "resource-factory-playground"
assert rule_hc["allow"][0] == {'ports': ['80'], 'protocol': 'tcp'}
assert rule_be["log_config"][0] == {'metadata': 'INCLUDE_ALL_METADATA'}