zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #324 from terraform-google-modules/sruffilli/factories 

Added resource factories
This commit is contained in:
sruffilli 2021-10-15 12:56:43 +02:00 committed by GitHub
parent f5353d2d2c 2bb947292e
commit 403d7034a0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
60 changed files with 1442 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.ci/cloudbuild.lint.yaml
View File

@ -43,12 +43,14 @@ steps:
args:
[
"/workspace/tools/check_documentation.py",
"modules",
"cloud-operations",
"data-solutions",
"data-solutions/data-platform-foundations",
"factories",
"factories/firewall-vpc-rules",
"foundations",
"networking",
"modules",
"networking"
]
substitutions:

1
.ci/cloudbuild.test.environments.yaml
View File

@ -32,6 +32,7 @@ steps:
- -vv
- tests/cloud_operations
- tests/data_solutions
- tests/factories
- tests/foundations
- tests/networking
env:

1
CHANGELOG.md
View File

@ -4,6 +4,7 @@ All notable changes to this project will be documented in this file.
## [Unreleased]
- new `factories` top-level folder with initial `subnets`, `firewall-hierarchical-policies`, `firewall-vpc-rules` and `example-environments` examples
- new cloud operations example showing how to deploy infrastructure for [Compute Engine image builder based on Hashicorp Packer](./cloud-operations/packer-image-builder)
- **incompatible change** the format of the `records` variable in the `dns` module has changed, to better support dynamic values
- new `naming-convention` module

5
README.md
View File

@ -12,7 +12,7 @@ Both the examples and modules require some measure of Terraform skills to be use
## End-to-end examples
The examples in this repository are split in several main sections: **foundational examples** that bootstrap the organizational hierarchy and automation prerequisites, **networking examples** that implement core patterns or features, **data solutions examples** that demonstrate how to integrate data services in complete scenarios, and **cloud operations examples** that leverage specific products to meet specific operational needs.
The examples in this repository are split in several main sections: **foundational examples** that bootstrap the organizational hierarchy and automation prerequisites, **networking examples** that implement core patterns or features, **data solutions examples** that demonstrate how to integrate data services in complete scenarios, **cloud operations examples** that leverage specific products to meet specific operational needs and **factories** that implement resource factories for the repetitive creation of specific resources.
Currently available examples:
@ -21,8 +21,9 @@ Currently available examples:
- **data solutions** - [GCE/GCS CMEK via centralized Cloud KMS](./data-solutions/cmek-via-centralized-kms/), [Cloud Storage to Bigquery with Cloud Dataflow](./data-solutions/gcs-to-bq-with-dataflow/)
- **cloud operations** - [Resource tracking and remediation via Cloud Asset feeds](.//cloud-operations/asset-inventory-feed-remediation), [Granular Cloud DNS IAM via Service Directory](./cloud-operations/dns-fine-grained-iam), [Granular Cloud DNS IAM for Shared VPC](./cloud-operations/dns-shared-vpc), [Compute Engine quota monitoring](./cloud-operations/quota-monitoring), [Scheduled Cloud Asset Inventory Export to Bigquery](./cloud-operations/scheduled-asset-inventory-export-bq), [Packer image builder](./cloud-operations/packer-image-builder)
- **third party solutions** - [OpenShift cluster on Shared VPC](./third-party-solutions/openshift)
- **factories** - [Example environments](./factories/example-environments), [Hierarchical Firewall Policies](./factories/firewall-hierarchical-policies), [VPC Firewall Rules](./factories/firewall-vpc-rules), [Subnets](./factories/subnets)
For more information see the README files in the [foundations](./foundations/), [networking](./networking/), [data solutions](./data-solutions/) and [cloud operations](./cloud-operations/) folders.
For more information see the README files in the [foundations](./foundations/), [networking](./networking/), [data solutions](./data-solutions/), [cloud operations](./cloud-operations/) and [factories](./factories/) folders.
## Modules

67
factories/README.md Normal file
View File

@ -0,0 +1,67 @@
# Resource Factories
This set of modules creates specialized resource factories, made of two distinct components:
- a module, which implements the factory logic in Terraform syntax, and
- a set of directories, which hold the configuration for the factory in YAML syntax.
## Available modules
- [Hierarchical Firewall policies](./firewall-hierarchical-policies)
- [VPC Firewall rules](./firewall-vpc-rules)
- [Subnets](./subnets)
## Using the modules
Each module specialises on a single resource type, and comes with a `README.md` file which describes the module interface, and the directory/file structure each module requires.
All modules consume specialized `yaml` configurations - located on a well-defined directory structure that carries metadata. Let's observe an example from the [Example environments](example-environments/) directory:
```yaml
# ../example-environments/prod/conf/project-prod-a/vpc-alpha/subnet-alpha-a.yaml
region: europe-west3
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-prod-a, vpc-alpha
secondary_ip_ranges:
secondary-range-a: 192.168.0.0/24
secondary-range-b: 192.168.1.0/24
```
This configuration creates the `subnet-alpha-a` subnet, located in VPC `vpc-alpha`, inside project `project-prod-a`.
All modules consume specialized `yaml` configurations - located on a well-defined directory structure that carries metadata. Let's observe an example from the [Example environments](example-environments/) directory:
```yaml
# ../example-environments/prod/conf/project-prod-a/vpc-alpha/subnet-alpha-a.yaml
region: europe-west3
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-prod-a, vpc-alpha
secondary_ip_ranges:
secondary-range-a: 192.168.0.0/24
secondary-range-b: 192.168.1.0/24
```
This configuration creates the `subnet-alpha-a` subnet, located in VPC `vpc-alpha`, inside project `project-prod-a`.
## Rationale
This approach is based on modules implementing the factory logic using Terraform code, and a set of directories having a well-defined, semantic structure holding the configuration for the resources in YaML syntax.
Resource factories are designed to:
- accelerate and rationalize the repetitive creation of common resources, such as firewall rules and subnets
- enable teams without Terraform specific knowledge to leverage IaC via human-friendly and machine-parseable YAML files
- make it simple to implement specific requirements and best practices (e.g. "always enable PGA for GCP subnets", or "only allow using regions `europe-west1` and `europe-west3`")
- codify and centralise business logics and policies (e.g. labels and naming conventions)
- allow to easily parse and understand sets of specific resources, for documentation purposes
Terraform natively supports YaML, JSON and CSV parsing - however we've decided to embrace YaML for the following reasons:
- YaML is easier to parse for a human, and allows for comments and nested, complex structures
- JSON and CSV can't include comments, which can be used to document configurations, but are often useful to bridge from other systems in automated pipelines
- JSON is more verbose (reads: longer) and harder to parse visually for humans
- CSV isn't often expressive enough (e.g. dit oesn't allow for nested structures)
If needed, converting factories to consume JSON is a matter of switching from `yamldecode()` to `jsondecode()` in the right place on each module.

43
factories/example-environments/README.md Normal file
View File

@ -0,0 +1,43 @@
# Resource Factories
The examples in this folder are derived from actual production use cases, and show how to use a factory module, and how to structure a codebase for multiple environments.
## Resource Factories usage - Managing subnets
At the top level of this directory, besides the `README.md` your're reading now, you'll find
- `dev/`, a directory which holds all configurations for the *development* environment
- `prod/`, a directory which holds all configurations for the *production* environment
and on each directory, `main.tf`, a simple terraform file which consumes the [`subnets`](../subnets/) module and the configurations.
Each environment directory structure is meant to mimic your GCP resource structure
```
.
├── dev # Environment
│ ├── conf # Configuration directory
│ │ ├── project-dev-a # Project id
│ │ │ └── vpc-alpha # VPC name
│ │ │ ├── subnet-alpha-a.yaml # Subnet name (one file per subnet)
│ │ │ └── subnet-alpha-b.yaml
│ │ └── project-dev-b
│ │ ├── vpc-beta
│ │ │ └── subnet-beta-a.yaml
│ │ └── vpc-gamma
│ │ └── subnet-gamma-a.yaml
│ └── main.tf
└── prod
├── conf
│ └── project-prod-a
│ └── vpc-alpha
│ ├── subnet-alpha-a.yaml
│ └── subnet-alpha-b.yaml
└── main.tf
```
Since this resource factory only creates subnets, projects and VPCs are expected to exist.
In this example, each environment is implemented as a distinct factory, and each has its own `main.tf` file (and hence a dedicated state).
Another option you might want to consider, in line with the CI/CD pipeline or processes you have in place, might be to leverage a single `main.tf` consuming both environment directories.

8
factories/example-environments/dev/conf/project-dev-a/vpc-alpha/subnet-alpha-a.yaml Normal file
View File

@ -0,0 +1,8 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-dev-a, vpc-alpha
secondary_ip_ranges:
secondary-range-a: 192.168.0.0/24
secondary-range-b: 192.168.1.0/24

6
factories/example-environments/dev/conf/project-dev-a/vpc-alpha/subnet-alpha-b.yaml Normal file
View File

@ -0,0 +1,6 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.1.0/24
description: Sample Subnet in project project-dev-a, vpc-alpha
private_ip_google_access: false

7
factories/example-environments/dev/conf/project-dev-b/vpc-beta/subnet-beta-a.yaml Normal file
View File

@ -0,0 +1,7 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.0.2.0/24
description: Sample Subnet in project project-dev-b, vpc-beta
iam_users: ["sruffilli@google.com"]
iam_groups: []

5
factories/example-environments/dev/conf/project-dev-b/vpc-gamma/subnet-gamma-a.yaml Normal file
View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.0.3.0/24
description: Sample Subnet in project project-dev-b, vpc-gamma

21
factories/example-environments/dev/main.tf Normal file
View File

@ -0,0 +1,21 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "subnets" {
source = "../../subnets"
config_folder = "conf"
}

8
factories/example-environments/prod/conf/project-prod-a/vpc-alpha/subnet-alpha-a.yaml Normal file
View File

@ -0,0 +1,8 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-prod-a, vpc-alpha
secondary_ip_ranges:
secondary-range-a: 192.168.0.0/24
secondary-range-b: 192.168.1.0/24

6
factories/example-environments/prod/conf/project-prod-a/vpc-alpha/subnet-alpha-b.yaml Normal file
View File

@ -0,0 +1,6 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.1.0/24
description: Sample Subnet in project project-prod-a, vpc-alpha
private_ip_google_access: false

21
factories/example-environments/prod/main.tf Normal file
View File

@ -0,0 +1,21 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "subnets" {
source = "../../subnets"
config_folder = "conf"
}

164
factories/firewall-hierarchical-policies/README.md Normal file
View File

@ -0,0 +1,164 @@
# Google Cloud Resource Factories - Hierarchical Firewall Policies
This module implements a resource factory which allows the creation and management of [hierarchical firewall policies](https://cloud.google.com/vpc/docs/firewall-policies) through properly formatted `yaml` files.
`yaml` configurations are stored on a well-defined folder structure, whose entry point can be customized, and which allows for simple grouping of policies by Organization ID.
This module also allows defining custom template variables, to centralize common CIDRs or Service Account lists, which enables re-using them across different policies.
## Example
### Terraform code
```hcl
module "hierarchical" {
source = "./modules/resource-factories/hierarchical-firewall"
config_folder = "firewall/hierarchical"
templates_folder = "firewall/templates"
}
# tftest:skip
```
### Configuration Structure
The naming convention for the `config_folder` variable requires
- the first directory layer to be named after the organization ID we're creating the policies for
- each file to be named either `$folder_id-$description.yaml` (e.g. `1234567890-sharedinfra.yaml`) for policies applying to regular folders or `org.yaml` for the root folder.
Organizations and folders should exist prior to running this module, or set as an explicit dependency to this module, leveraging `depends_on`.
The optional `templates_folder` variable can have two files.
- `cidrs.yaml` - a YAML map defining lists of CIDRs
- `service_accounts.yaml` - a YAML map defining lists of Service Accounts
Examples for both files are shown in the following section.
```bash
└── firewall
├── hierarchical
│ ├── 31415926535
│ │ ├── 1234567890-sharedinfra.yaml # Maps to folders/1234567890
│ │ └── org.yaml # Maps to organizations/31415926535
│ └── 27182818284
│ └── 1234567891-sharedinfra.yaml # Maps to folders/1234567891
└── templates
├── cidrs.yaml
└── service_accounts.yaml
```
### Hierarchical firewall policies format and structure
The following syntax applies for both `$folder_id-$description.yaml` and for `org.yaml` files, with the former applying at the `$folder_id` level and the latter at the Organization level.
Each file can contain an arbitrary number of policies.
```yaml
# Policy name
allow-icmp:
# Description
description: Sample policy
# Direction {INGRESS, EGRESS}
direction: INGRESS
# Action {allow, deny}
action: allow
# Priority (must be unique on a node)
priority: 1000
# List of CIDRs this rule applies to (for INGRESS rules)
source_ranges:
- 0.0.0.0/0
# List of CIDRs this rule applies to (for EGRESS rules)
destination_ranges:
- 0.0.0.0/0
# List of ports this rule applies to (empty array means all ports)
ports:
tcp: []
udp: []
icmp: []
# List of VPCs this rule applies to - a null value implies all VPCs
target_resources: null
# Opt - List of target Service Accounts this rule applies to
target_service_accounts:
- example-service-account@foobar.iam.gserviceaccount.com
# Opt - Whether to enable logs - defaults to false
enable_logging: true
```
A sample configuration file might look like the following one:
```yaml
allow-icmp:
description: Enable ICMP for all hosts
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- 0.0.0.0/0
ports:
icmp: []
target_resources: null
enable_logging: false
allow-ssh-from-onprem:
description: Enable SSH for on prem hosts
direction: INGRESS
action: allow
priority: 1001
source_ranges:
- $onprem
ports:
tcp: ["22"]
target_resources: null
enable_logging: false
allow-443-from-clients:
description: Enable HTTPS for web clients
direction: INGRESS
action: allow
priority: 1001
source_ranges:
- $web_clients
ports:
tcp: ["443"]
target_resources: null
target_service_accounts:
- $web_frontends
enable_logging: false
```
with `firewall/templates/cidrs.yaml` defined as follows:
```yaml
onprem:
- 10.0.0.0/8
- 192.168.0.0/16
web_clients:
- 172.16.0.0/16
- 10.0.10.0/24
- 10.0.250.0/24
```
and `firewall/templates/service_accounts.yaml`:
```yaml
web_frontends:
- web-frontends@project-wf1.iam.gserviceaccount.com
- web-frontends@project-wf2.iam.gserviceaccount.com
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---: |:---:|:---:|
| config_folder | Relative path of the folder containing the hierarchical firewall configuration | <code title="">string</code> | ✓ | |
| templates_folder | Relative path of the folder containing the cidr/service account templates | <code title="">string</code> | ✓ | |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| hierarchical-firewall-rules | Generated Hierarchical Firewall Rules | |
<!-- END TFDOC -->

98
factories/firewall-hierarchical-policies/main.tf Normal file
View File

@ -0,0 +1,98 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
cidrs = try({ for name, cidrs in yamldecode(file("${var.templates_folder}/cidrs.yaml")) :
name => cidrs
}, {})
service_accounts = try({ for name, service_accounts in yamldecode(file("${var.templates_folder}/service_accounts.yaml")) :
name => service_accounts
}, {})
org_paths = {
for file in fileset(var.config_folder, "**/*.yaml") :
file => split("/", file)[1] == "org.yaml"
? "organizations/${split("/", file)[0]}"
: "folders/${split("-", split("/", file)[1])[0]}"
}
rules = flatten([
for file in fileset(var.config_folder, "**/*.yaml") : [
for key, ruleset in yamldecode(file("${var.config_folder}/${file}")) :
merge(ruleset, {
parent_id = local.org_paths[file]
name = "${replace(local.org_paths[file], "/", "-")}-${key}"
source_ranges = try(ruleset.source_ranges, null) == null ? null : flatten(
[for cidr in ruleset.source_ranges :
can(regex("^\\$", cidr))
? local.cidrs[trimprefix(cidr, "$")]
: [cidr]
])
destination_ranges = try(ruleset.destination_ranges, null) == null ? null : flatten(
[for cidr in ruleset.destination_ranges :
can(regex("^\\$", cidr))
? local.cidrs[trimprefix(cidr, "$")]
: [cidr]
])
target_service_accounts = try(ruleset.target_service_accounts, null) == null ? null : flatten(
[for service_account in ruleset.target_service_accounts :
can(regex("^\\$", service_account))
? local.service_accounts[trimprefix(service_account, "$")]
: [service_account]
])
})
]
])
}
resource "google_compute_firewall_policy" "default" {
for_each = { for rule in local.rules : rule.parent_id => rule.name... }
short_name = replace("hierarchical-fw-policy-${each.key}", "/", "-")
description = replace("hierarchical-fw-policy-${each.key}", "/", "-")
parent = each.key
}
resource "google_compute_firewall_policy_rule" "default" {
for_each = { for rule in local.rules : "${rule.parent_id}-${rule.name}" => rule }
firewall_policy = google_compute_firewall_policy.default[each.value.parent_id].id
action = each.value.action
direction = each.value.direction
priority = each.value.priority
target_resources = each.value.target_resources
target_service_accounts = each.value.target_service_accounts
enable_logging = try(each.value.enable_logging, false)
# preview = each.value.preview
match {
src_ip_ranges = each.value.direction == "INGRESS" ? each.value.source_ranges : null
dest_ip_ranges = each.value.direction == "EGRESS" ? each.value.destination_ranges : null
dynamic "layer4_configs" {
for_each = each.value.ports
iterator = port
content {
ip_protocol = port.key
ports = port.value
}
}
}
}
resource "google_compute_firewall_policy_association" "default" {
for_each = { for rule in local.rules : rule.parent_id => rule.name... }
name = replace("hierarchical-fw-policy-${each.key}", "/", "-")
attachment_target = google_compute_firewall_policy.default[each.key].parent
firewall_policy = google_compute_firewall_policy.default[each.key].id
}

26
factories/firewall-hierarchical-policies/outputs.tf Normal file
View File

@ -0,0 +1,26 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "hierarchical-firewall-rules" {
description = "Generated Hierarchical Firewall Rules"
value = {
for k, v in google_compute_firewall_policy_rule.default :
k => {
parent_id = split("-", k)[0]
id = v.id
}
}
}

25
factories/firewall-hierarchical-policies/variables.tf Normal file
View File

@ -0,0 +1,25 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "config_folder" {
description = "Relative path of the folder containing the hierarchical firewall configuration"
type = string
}
variable "templates_folder" {
description = "Relative path of the folder containing the cidr/service account templates"
type = string
}

6
factories/firewall-vpc-rules/README.md Normal file
View File

@ -0,0 +1,6 @@
# Google Cloud VPC Firewall Factories
This collection of modules implement two different metodologies for the creation of VPC firewall rules, both based on leveraging well-defined `yaml` configuration files.
- The [flat module](flat/) delegates the definition of all firewall rules metadata (project, network amongst other) to the individual `yaml` configuration. This module allows for maximum flexibility, and a custom logical grouping of resources which can be trasversal to the traditional resource hierarchy, and could be useful in scenarios where networking is not managed centrally by a single team.
- The [nested module](nested/) requires and enforces a semantical folder structure that carries some of the rules metadata (project and network), and leaves the rest to each `yaml` configuration. This solution allows for the definition of a resource hierarchy that is aligned with the organisational resource structure.

4
modules/net-vpc-firewall-yaml/README.md → factories/firewall-vpc-rules/flat/README.md
View File

@ -1,10 +1,10 @@
# Google Cloud VPC Firewall - Yaml
# Google Cloud VPC Firewall Factory - Flat hierarchy
This module allows creation and management of different types of firewall rules by defining them in well formatted `yaml` files.
Yaml abstraction for FW rules can simplify users onboarding and also makes rules definition simpler and clearer comparing to HCL.
Nested folder structure for yaml configurations is supported, which allows better and structured code management for multiple teams and environments.
Nested folder structure for yaml configurations is optionally supported, which allows better and structured code management for multiple teams and environments.
## Example

0
modules/net-vpc-firewall-yaml/main.tf → factories/firewall-vpc-rules/flat/main.tf
View File

0
modules/net-vpc-firewall-yaml/outputs.tf → factories/firewall-vpc-rules/flat/outputs.tf
View File

0
modules/net-vpc-firewall-yaml/variables.tf → factories/firewall-vpc-rules/flat/variables.tf
View File

0
modules/net-vpc-firewall-yaml/versions.tf → factories/firewall-vpc-rules/flat/versions.tf
View File

140
factories/firewall-vpc-rules/nested/README.md Normal file
View File

@ -0,0 +1,140 @@
# Google Cloud VPC Firewall Factory - Nested hierarchy
This module implements a resource factory which allows the creation and management of [VPC firewall rules](https://cloud.google.com/vpc/docs/firewalls) via properly formatted `yaml` files.
`yaml` configurations are stored in a well-defined folder structure, whose entry point can be customized, and which represents and forces the resource hierarchy a firewall rule belongs to (Project > VPC > Firewall Rule).
This module also allows for the definition of template variables, allowing to centralize common CIDRs or Service Account lists, which enables re-using them across different policies.
## Example
### Terraform code
```hcl
module "vpc-firewall" {
source = "../../cloud-foundation-fabric/modules/resource-factories/vpc-firewall"
config_folder = "firewall/vpc"
templates_folder = "firewall/templates"
}
# tftest:skip
```
### Configuration Structure
The naming convention for the `config_folder` folder **requires**
- the first directory layer to be named after the project ID which contains the VPC we're creating the firewall rules for
- the second directory layer to be named after the VPC we're creating the firewall rules for
- `yaml` files contained in the "VPC" directory can be arbitrarily named, to allow for an easier logical grouping.
Projects and VPCs should exist prior to running this module, or set as an explicit dependency to this module, leveraging `depends_on`.
The optional `templates_folder` folder can have two files.
- `cidrs.yaml` - a YAML map defining lists of CIDRs
- `service_accounts.yaml` - a YAML map definint lists of Service Accounts
```bash
└── firewall
├── vpc
│ ├── project-resource-factory-dev
│ │ └── vpc-resource-factory-dev-one
│ │ │ ├── frontend.yaml
│ │ │ └── backend.yaml
│ │ └── vpc-resource-factory-dev-two
│ │ ├── foo.yaml
│ │ └── bar.yaml
│ └── project-resource-factory-prod
│ │ └── vpc-resource-factory-prod-alpha
│ │ ├── lorem.yaml
│ │ └── ipsum.yaml
└── templates
├── cidrs.yaml
└── service_accounts.yaml
```
### Rule definition format and structure
Firewall rules configuration should be placed in a set of yaml files in a folder/s. Firewall rule entry structure is following:
```yaml
rule-name: # descriptive name, naming convention is adjusted by the module
description: "Allow icmp" # rule description
action: allow # `allow` or `deny`
direction: INGRESS # EGRESS or INGRESS
ports:
icmp: [] # {tcp, udp, icmp, all}: [ports], use [] for any port
priority: 1000 # rule priority value, default value is 1000
source_ranges: # list of source ranges
- 0.0.0.0/0
destination_ranges: # list of destination ranges
- 0.0.0.0/0
source_tags: ['some-tag'] # list of source tags
source_service_accounts: # list of source service accounts
- myapp@myproject-id.iam.gserviceaccount.com
target_tags: ['some-tag'] # list of target tags
target_service_accounts: # list of target service accounts
- myapp@myproject-id.iam.gserviceaccount.com
enable_logging: true # `false` or `true`, logging is enabled when `true`
```
A sample configuration file might look like the following one:
```yaml
allow-healthchecks:
description: "Allow traffic from healthcheck"
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- $healthcheck
ports:
tcp: ["80"]
enable_logging: false
allow-http:
description: "Allow traffic to LB backend"
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- 0.0.0.0/0
target_service_accounts:
- $web_frontends
ports:
tcp: ["80", "443"]
enable_logging: false
```
with `firewall/templates/cidrs.yaml` defined as follows:
```yaml
healthcheck:
- 35.191.0.0/16
- 130.211.0.0/22
```
and `firewall/templates/service_accounts.yaml`:
```yaml
web_frontends:
- web-frontends@project-wf1.iam.gserviceaccount.com
- web-frontends@project-wf2.iam.gserviceaccount.com
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---: |:---:|:---:|
| config_folder | Relative path of the folder containing the hierarchical firewall configuration | <code title="">string</code> | ✓ | |
| templates_folder | Relative path of the folder containing the cidr/service account templates | <code title="">string</code> | ✓ | |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| vpc-firewall-rules | Generated VPC Firewall Rules | |
<!-- END TFDOC -->

143
factories/firewall-vpc-rules/nested/main.tf Normal file
View File

@ -0,0 +1,143 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
cidrs = try({ for name, cidrs in yamldecode(file("${var.templates_folder}/cidrs.yaml")) :
name => cidrs
}, {})
service_accounts = try({ for name, service_accounts in yamldecode(file("${var.templates_folder}/service_accounts.yaml")) :
name => service_accounts
}, {})
rules = flatten([
for file in fileset(var.config_folder, "**/*.yaml") : [
for key, ruleset in yamldecode(file("${var.config_folder}/${file}")) :
merge(ruleset, {
project_id = split("/", file)[0]
network = split("/", file)[1]
name = "${key}-${split("/", file)[1]}"
source_ranges = try(ruleset.source_ranges, null) == null ? null : flatten(
[for cidr in ruleset.source_ranges :
can(regex("^\\$", cidr))
? local.cidrs[trimprefix(cidr, "$")]
: [cidr]
])
destination_ranges = try(ruleset.destination_ranges, null) == null ? null : flatten(
[for cidr in ruleset.destination_ranges :
can(regex("^\\$", cidr))
? local.cidrs[trimprefix(cidr, "$")]
: [cidr]
])
source_service_accounts = try(ruleset.source_service_accounts, null) == null ? null : flatten(
[for service_account in ruleset.source_service_accounts :
can(regex("^\\$", service_account))
? local.service_accounts[trimprefix(service_account, "$")]
: [service_account]
])
target_service_accounts = try(ruleset.target_service_accounts, null) == null ? null : flatten(
[for service_account in ruleset.target_service_accounts :
can(regex("^\\$", service_account))
? local.service_accounts[trimprefix(service_account, "$")]
: [service_account]
])
})
]
])
rules_allow = [for item in local.rules : item if item.action == "allow"]
rules_deny = [for item in local.rules : item if item.action == "deny"]
}
resource "google_compute_firewall" "rules-allow" {
for_each = { for rule in local.rules_allow : "${rule.network}-${rule.name}" => rule }
project = each.value.project_id
name = each.value.name
description = each.value.description
network = each.value.network
direction = each.value.direction
priority = each.value.priority
source_ranges = try(each.value.source_ranges, each.value.direction == "INGRESS" ? [] : null)
source_tags = try(each.value.source_tags, null)
source_service_accounts = try(each.value.source_service_accounts, null)
destination_ranges = try(each.value.destination_ranges, each.value.direction == "EGRESS" ? [] : null)
target_tags = try(each.value.target_tags, null)
target_service_accounts = try(each.value.target_service_accounts, null)
dynamic "allow" {
for_each = { for proto, ports in try(each.value.ports, []) :
"${proto}-${join("-", ports)}" => {
ports = [for port in ports : tostring(port)]
protocol = proto
}
}
content {
protocol = allow.value.protocol
ports = allow.value.ports
}
}
dynamic "log_config" {
for_each = (each.value.enable_logging == null) || (each.value.enable_logging == false) ? [] : [""]
content {
metadata = "INCLUDE_ALL_METADATA"
}
}
}
resource "google_compute_firewall" "rules-deny" {
for_each = { for rule in local.rules_deny : "${rule.network}-${rule.name}" => rule }
project = each.value.project_id
name = each.value.name
description = each.value.description
network = each.value.network
direction = each.value.direction
priority = each.value.priority
source_ranges = try(each.value.source_ranges, each.value.direction == "INGRESS" ? [] : null)
source_tags = try(each.value.source_tags, null)
source_service_accounts = try(each.value.source_service_accounts, null)
destination_ranges = try(each.value.destination_ranges, each.value.direction == "EGRESS" ? [] : null)
target_tags = try(each.value.target_tags, null)
target_service_accounts = try(each.value.target_service_accounts, null)
dynamic "deny" {
for_each = { for proto, ports in try(each.value.ports, []) :
"${proto}-${join("-", ports)}" => {
ports = [for port in ports : tostring(port)]
protocol = proto
}
}
content {
protocol = deny.value.protocol
ports = deny.value.ports
}
}
dynamic "log_config" {
for_each = (each.value.enable_logging == null) || (each.value.enable_logging == false) ? [] : [""]
content {
metadata = "INCLUDE_ALL_METADATA"
}
}
}

20
factories/firewall-vpc-rules/nested/outputs.tf Normal file
View File

@ -0,0 +1,20 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "vpc-firewall-rules" {
description = "Generated VPC Firewall Rules"
value = merge(google_compute_firewall.rules-allow, google_compute_firewall.rules-deny)
}

25
factories/firewall-vpc-rules/nested/variables.tf Normal file
View File

@ -0,0 +1,25 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "config_folder" {
description = "Relative path of the folder containing the hierarchical firewall configuration"
type = string
}
variable "templates_folder" {
description = "Relative path of the folder containing the cidr/service account templates"
type = string
}

68
factories/subnets/README.md Normal file
View File

@ -0,0 +1,68 @@
# Google Cloud Resource Factories - VPC Subnets
This module implements a resource factory which allows the creation and management of subnets through properly formatted `yaml` files.
`yaml` configurations are stored on a well-defined folder structure, whose entry point can be customized, and which allows for simple grouping of subnets by Project > VPC.
## Example
### Terraform code
```hcl
module "subnets" {
source = "./modules/resource-factories/subnets"
config_folder = "subnets"
}
# tftest:skip
```
### Configuration Structure
The directory structure implies the project and the VPC each subnet belongs to.
Per the structure below, a subnet named `subnet-a` (after filename `subnet-a.yaml`) will be created on VPC `vpc-alpha-one` which belongs to project `project-alpha`.
Projects and VPCs should exist prior to running this module, or set as an explicit dependency to this module, leveraging `depends_on`.
```bash
└── subnets
├── project-alpha
│ ├── vpc-alpha-one
│ │ ├── subnet-a.yaml
│ │ └── subnet-b.yaml
│ └── vpc-alpha-two
│ └── subnet-c.yaml
└── project-bravo
└── vpc-bravo-one
└── subnet-d.yaml
```
### Subnet definition format and structure
```yaml
region: europe-west1 # Region where the subnet will be creted
description: Sample description # Description
ip_cidr_range: 10.0.0.0/24 # Primary IP range for the subnet
private_ip_google_access: false # Opt- Enables PGA. Defaults to true
iam_users: ["foobar@example.com"] # Opt- Users to grant compute/networkUser to
iam_groups: ["lorem@example.com"] # Opt- Groups to grant compute/networkUser to
iam_service_accounts: ["foobar@project-id.iam.gserviceaccount.com"]
# Opt- SAs to grant compute/networkUser to
secondary_ip_ranges: # Opt- List of secondary IP ranges
- name: secondary-range-a # Name for the secondary range
ip_cidr_range: 192.168.0.0/24 # IP range for the secondary range
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---: |:---:|:---:|
| config_folder | Relative path of the folder containing the subnet configuration | <code title="">string</code> | ✓ | |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| subnet | Generated subnets | |
<!-- END TFDOC -->

72
factories/subnets/main.tf Normal file
View File

@ -0,0 +1,72 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
_data = {
for f in fileset(var.config_folder, "**/*.yaml") :
trimsuffix(split("/", f)[2], ".yaml") => merge(
yamldecode(file("${var.config_folder}/${f}")),
{
project_id = split("/", f)[0]
network = split("/", f)[1]
}
)
}
data = {
for k, v in local._data : k => merge(v,
{
network_users : concat(
formatlist("group:%s", try(v.iam_groups, [])),
formatlist("user:%s", try(v.iam_users, [])),
formatlist("serviceAccount:%s", try(v.iam_service_accounts, []))
)
}
)
}
}
resource "google_compute_subnetwork" "default" {
for_each = local.data
project = each.value.project_id
network = each.value.network
name = each.key
region = each.value.region
description = each.value.description
ip_cidr_range = each.value.ip_cidr_range
private_ip_google_access = try(each.value.private_ip_google_access, true)
dynamic "secondary_ip_range" {
for_each = try(each.value.secondary_ip_ranges, [])
iterator = secondary_range
content {
range_name = secondary_range.key
ip_cidr_range = secondary_range.value
}
}
}
resource "google_compute_subnetwork_iam_binding" "default" {
for_each = {
for k, v in local.data : k => v if length(v.network_users) > 0
}
project = each.value.project_id
subnetwork = google_compute_subnetwork.default[each.key].name
region = each.value.region
role = "roles/compute.networkUser"
members = each.value.network_users
}

28
factories/subnets/outputs.tf Normal file
View File

@ -0,0 +1,28 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "subnet" {
description = "Generated subnets"
value = {
for k, v in google_compute_subnetwork.default :
k => {
network = v.network
project = v.project
range = v.ip_cidr_range
region = v.region
}
}
}

20
factories/subnets/variables.tf Normal file
View File

@ -0,0 +1,20 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "config_folder" {
description = "Relative path of the folder containing the subnet configuration"
type = string
}

4
networking/decentralized-firewall/main.tf
View File

@ -109,7 +109,7 @@ module "dns-api-dev" {
###############################################################################
module "vpc-firewall-prod" {
source = "../../modules/net-vpc-firewall-yaml"
source = "../../factories/firewall-vpc-rules/flat"
project_id = module.project-host-prod.project_id
network = module.vpc-prod.name
@ -125,7 +125,7 @@ module "vpc-firewall-prod" {
}
module "vpc-firewall-dev" {
source = "../../modules/net-vpc-firewall-yaml"
source = "../../factories/firewall-vpc-rules/flat"
project_id = module.project-host-dev.project_id
network = module.vpc-dev.name

0
tests/modules/net_vpc_firewall_yaml/__init__.py → tests/factories/__init__.py
View File

13
tests/factories/firewall_hierarchical_policies/__init__.py Normal file
View File

@ -0,0 +1,13 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

13
tests/factories/firewall_hierarchical_policies/fixture/conf/rules/1234567890/0987654321-foobar.yaml Normal file
View File

@ -0,0 +1,13 @@
# skip boilerplate check
allow-ssh-from-onprem:
description: Enable SSH for onprem ranges
direction: INGRESS
action: allow
priority: 1001
source_ranges:
- $example
ports:
tcp: ["22"]
target_resources: null
enable_logging: false

13
tests/factories/firewall_hierarchical_policies/fixture/conf/rules/1234567890/org.yaml Normal file
View File

@ -0,0 +1,13 @@
# skip boilerplate check
allow-icmp:
description: Enable ICMP for all hosts
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- 0.0.0.0/0
ports:
icmp: []
target_resources: null
enable_logging: false

10
tests/factories/firewall_hierarchical_policies/fixture/conf/templates/cidrs.yaml Normal file
View File

@ -0,0 +1,10 @@
# skip boilerplate check
example:
- 10.0.0.0/24
- 10.0.10.0/24
- 192.168.1.1/32
healthcheck:
- 35.191.0.0/16
- 130.211.0.0/22

4
tests/factories/firewall_hierarchical_policies/fixture/conf/templates/service_accounts.yaml Normal file
View File

@ -0,0 +1,4 @@
# skip boilerplate check
example:
- example-service-account@resource-factory-playground.iam.gserviceaccount.com

21
tests/factories/firewall_hierarchical_policies/fixture/main.tf Normal file
View File

@ -0,0 +1,21 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "hierarchical-firewall-rules" {
source = "../../../../factories/firewall-hierarchical-policies/"
config_folder = "conf/rules"
templates_folder = "conf/templates"
}

55
tests/factories/firewall_hierarchical_policies/test_plan.py Normal file
View File

@ -0,0 +1,55 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import pytest
FIXTURES_DIR = os.path.join(os.path.dirname(__file__), "fixture")
def test_firewall(plan_runner):
"Test hierarchical firewall rules from conf/rules"
_, resources = plan_runner(FIXTURES_DIR)
assert len(resources) == 6
assert set(r["type"] for r in resources) == set([
"google_compute_firewall_policy_rule", "google_compute_firewall_policy_association", "google_compute_firewall_policy"
])
rule_ssh = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy_rule"
and r["values"]["priority"] == 1001]
rule_icmp = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy_rule"
and r["values"]["priority"] == 1000]
association_org = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy_association"
and r["values"]["attachment_target"] == "organizations/1234567890"]
association_folder = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy_association"
and r["values"]["attachment_target"] == "folders/0987654321"]
policies_org = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy"
and r["values"]["parent"] == "organizations/1234567890"]
policies_folder = [r["values"] for r in resources if r["type"] ==
"google_compute_firewall_policy"
and r["values"]["parent"] == "folders/0987654321"]
assert set(rule_ssh[0]["match"][0]["src_ip_ranges"]) == set(
["10.0.0.0/24", "10.0.10.0/24", "192.168.1.1/32"])
assert rule_icmp[0]["match"][0]["layer4_configs"][0]["ip_protocol"] == "icmp"
assert association_org[0]["name"] == "hierarchical-fw-policy-organizations-1234567890"
assert association_folder[0]["name"] == "hierarchical-fw-policy-folders-0987654321"
assert policies_org[0]["short_name"] == "hierarchical-fw-policy-organizations-1234567890"
assert policies_folder[0]["short_name"] == "hierarchical-fw-policy-folders-0987654321"

13
tests/factories/subnets/__init__.py Normal file
View File

@ -0,0 +1,13 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

8
tests/factories/subnets/fixture/conf/project-a/vpc-a/subnet-a.yaml Normal file
View File

@ -0,0 +1,8 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.0.0.0/24
description: Sample Subnet in project project-a, vpc-a
secondary_ip_ranges:
secondary-range-a: 192.168.0.0/24
secondary-range-b: 192.168.1.0/24

6
tests/factories/subnets/fixture/conf/project-a/vpc-a/subnet-b.yaml Normal file
View File

@ -0,0 +1,6 @@
# skip boilerplate check
region: europe-west3
ip_cidr_range: 10.0.1.0/24
description: Sample Subnet in project project-a, vpc-a
private_ip_google_access: false

7
tests/factories/subnets/fixture/conf/project-a/vpc-b/subnet-one.yaml Normal file
View File

@ -0,0 +1,7 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.0.2.0/24
description: Sample Subnet in project project-a, vpc-b
iam_users: ["sruffilli@google.com"]
iam_groups: []

7
tests/factories/subnets/fixture/conf/project-b/vpc-x/subnet-alpha.yaml Normal file
View File

@ -0,0 +1,7 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 172.16.0.0/24
description: Sample Subnet in project project-b, vpc-x
iam_users: ["sruffilli@google.com"]
iam_groups: []

20
tests/factories/subnets/fixture/main.tf Normal file
View File

@ -0,0 +1,20 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "subnets" {
source = "../../../../factories/subnets"
config_folder = "conf"
}

67
tests/factories/subnets/test_plan.py Normal file
View File

@ -0,0 +1,67 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import pytest
FIXTURES_DIR = os.path.join(os.path.dirname(__file__), "fixture")
def test_firewall(plan_runner):
"Test hierarchical firewall rules from conf/rules"
_, resources = plan_runner(FIXTURES_DIR)
assert len(resources) == 6
assert set(r["type"] for r in resources) == set(
["google_compute_subnetwork", "google_compute_subnetwork_iam_binding"])
subnets = [
r["values"] for r in resources
if r["type"] == "google_compute_subnetwork"
]
iam_bindings = [
r["values"] for r in resources
if r["type"] == "google_compute_subnetwork_iam_binding"
]
subnet_a_a = [
s for s in subnets if s["project"] == "project-a"
and s["network"] == "vpc-a" and s["name"] == "subnet-a"
][0]
assert subnet_a_a["ip_cidr_range"] == "10.0.0.0/24"
assert subnet_a_a["private_ip_google_access"] == True
assert subnet_a_a["region"] == "europe-west1"
assert subnet_a_a["secondary_ip_range"] == [{
"ip_cidr_range":
"192.168.0.0/24",
"range_name":
"secondary-range-a"
}, {
"ip_cidr_range":
"192.168.1.0/24",
"range_name":
"secondary-range-b"
}]
subnet_a_b = [
s for s in subnets if s["project"] == "project-a"
and s["network"] == "vpc-a" and s["name"] == "subnet-b"
][0]
assert subnet_a_b["private_ip_google_access"] == False
iam_binding_b_alpha = [
b for b in iam_bindings if b["project"] == "project-b"
][0]
assert set(iam_binding_b_alpha["members"]) == set(
["user:sruffilli@google.com"])
assert iam_binding_b_alpha["role"] == "roles/compute.networkUser"
assert iam_binding_b_alpha["subnetwork"] == "subnet-alpha"

13
tests/factories/vpc_firewall/__init__.py Normal file
View File

@ -0,0 +1,13 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

13
tests/factories/vpc_firewall/flat/__init__.py Normal file
View File

@ -0,0 +1,13 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

2
tests/modules/net_vpc_firewall_yaml/fixture/main.tf → tests/factories/vpc_firewall/flat/fixture/main.tf
View File

@ -15,7 +15,7 @@
*/
module "firewall" {
source = "../../../../modules/net-vpc-firewall-yaml"
source = "../../../../../factories/firewall-vpc-rules/flat"
project_id = "my-project"
network = "my-network"
config_directories = [

0
tests/modules/net_vpc_firewall_yaml/fixture/rules/common.yaml → tests/factories/vpc_firewall/flat/fixture/rules/common.yaml
View File

0
tests/modules/net_vpc_firewall_yaml/fixture/variables.tf → tests/factories/vpc_firewall/flat/fixture/variables.tf
View File

0
tests/modules/net_vpc_firewall_yaml/test_plan.py → tests/factories/vpc_firewall/flat/test_plan.py
View File

13
tests/factories/vpc_firewall/nested/__init__.py Normal file
View File

@ -0,0 +1,13 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

25
tests/factories/vpc_firewall/nested/fixture/conf/rules/resource-factory-playground/vpc-a/ingress-loadbalancers.yaml Normal file
View File

@ -0,0 +1,25 @@
# skip boilerplate check
allow-healthchecks:
description: "Allow traffic from healthcheck"
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- $healthcheck
ports:
tcp: ["80"]
enable_logging: false
allow-http:
description: "Allow traffic to LB backend"
direction: INGRESS
action: allow
priority: 1000
source_ranges:
- 0.0.0.0/0
target_service_accounts:
- example-service-account@resource-factory-playground.iam.gserviceaccount.com
ports:
tcp: ["80", "443"]
enable_logging: true

10
tests/factories/vpc_firewall/nested/fixture/conf/templates/cidrs.yaml Normal file
View File

@ -0,0 +1,10 @@
# skip boilerplate check
example:
- 10.0.0.0/24
- 10.0.10.0/24
- 192.168.1.1/32
healthcheck:
- 35.191.0.0/16
- 130.211.0.0/22

4
tests/factories/vpc_firewall/nested/fixture/conf/templates/service_accounts.yaml Normal file
View File

@ -0,0 +1,4 @@
# skip boilerplate check
couchbase:
- example-service-account@resource-factory-playground.iam.gserviceaccount.com

21
tests/factories/vpc_firewall/nested/fixture/main.tf Normal file
View File

@ -0,0 +1,21 @@
/**
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "vpc-firewall-rules" {
source = "../../../../../factories/firewall-vpc-rules/nested"
config_folder = "conf/rules"
templates_folder = "conf/templates"
}

45
tests/factories/vpc_firewall/nested/test_plan.py Normal file
View File

@ -0,0 +1,45 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import pytest
FIXTURES_DIR = os.path.join(os.path.dirname(__file__), "fixture")
def test_firewall(plan_runner):
"Test hierarchical firewall rules from conf/rules"
_, resources = plan_runner(FIXTURES_DIR)
assert len(resources) == 2
assert set(r["type"]
for r in resources) == set(["google_compute_firewall"])
rule_hc = [
r["values"] for r in resources
if r["values"]["name"] == "allow-healthchecks-vpc-a"
][0]
rule_be = [
r["values"] for r in resources
if r["values"]["description"] == "Allow traffic to LB backend"
][0]
assert set(rule_hc["source_ranges"]) == set(
["130.211.0.0/22", "35.191.0.0/16"])
assert rule_hc["direction"] == "INGRESS"
assert rule_hc["network"] == "vpc-a"
assert rule_hc["priority"] == 1000
assert rule_hc["project"] == "resource-factory-playground"
assert rule_hc["allow"][0] == {'ports': ['80'], 'protocol': 'tcp'}
assert rule_be["log_config"][0] == {'metadata': 'INCLUDE_ALL_METADATA'}