zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into lcaggio-patch-2

This commit is contained in:
Ludovico Magnocavallo 2022-01-19 15:32:17 +01:00 committed by GitHub
parent 9b254242ac 337528066b
commit 87385fd2a6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
124 changed files with 15562 additions and 149 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
.github/workflows/linting.yml vendored
View File

@ -16,6 +16,7 @@ name: "Linting"
on:
pull_request:
branches:
- fast-dev
- master
tags:
- ci
@ -51,23 +52,22 @@ jobs:
run: |
terraform fmt -recursive -check -diff $GITHUB_WORKSPACE
- name: Check documentation
id: documentation
- name: Check documentation (fabric)
id: documentation-fabric
run: |
python3 tools/check_documentation.py \
cloud-operations \
data-solutions \
factories \
foundations \
modules \
networking
python3 tools/check_documentation.py examples modules
markdown-link-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
- uses: gaurav-nelson/github-action-markdown-link-check@v1
with:
use-quiet-mode: 'yes'
use-verbose-mode: 'yes'
config-file: '.github/workflows/markdown-link-check.json'
- name: Check documentation (fast)
id: documentation-fast
run: |
python3 tools/check_documentation.py --files --show-extra fast
# markdown-link-check:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@master
# - uses: gaurav-nelson/github-action-markdown-link-check@v1
# with:
# use-quiet-mode: "yes"
# use-verbose-mode: "yes"
# config-file: ".github/workflows/markdown-link-check.json"

72
.github/workflows/tests.yml vendored
View File

@ -18,6 +18,7 @@ on:
- cron: "45 2 * * *"
pull_request:
branches:
- fast-dev
- master
tags:
- ci
@ -27,6 +28,38 @@ env:
PYTEST_ADDOPTS: "--color=yes"
jobs:
doc-examples:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.9"
- name: Set up Terraform
uses: hashicorp/setup-terraform@v1
with:
terraform_version: 1.1.3
terraform_wrapper: false
- name: Set environment
run: |
echo "TF_PLUGIN_CACHE_DIR=${{ github.workspace }}/.terraform.d/plugin-cache-${GITHUB_JOB}" >> $GITHUB_ENV
echo "GOOGLE_APPLICATION_CREDENTIALS=${GITHUB_WORKSPACE}/.github/workflows/fake-key.json" >> $GITHUB_ENV
mkdir --parents ${{ github.workspace }}/.terraform.d/plugin-cache-${GITHUB_JOB}
terraform -chdir=tests providers lock
- name: Install dependencies
run: |
pip install -r tests/requirements.txt
- name: Run tests on documentation examples
id: doc-examples
run: |
pytest -n 4 -vv tests/doc_examples
examples:
runs-on: ubuntu-latest
steps:
@ -40,12 +73,15 @@ jobs:
- name: Set up Terraform
uses: hashicorp/setup-terraform@v1
with:
terraform_version: 1.0.9
terraform_version: 1.1.3
terraform_wrapper: false
- name: Set environment
run: |
echo "TF_PLUGIN_CACHE_DIR=${{ github.workspace }}/.terraform.d/plugin-cache-${GITHUB_JOB}" >> $GITHUB_ENV
echo "GOOGLE_APPLICATION_CREDENTIALS=${GITHUB_WORKSPACE}/.github/workflows/fake-key.json" >> $GITHUB_ENV
mkdir --parents ${{ github.workspace }}/.terraform.d/plugin-cache-${GITHUB_JOB}
terraform -chdir=tests providers lock
- name: Install dependencies
run: |
@ -56,35 +92,6 @@ jobs:
run: |
pytest -n 4 -vv tests/examples
module-examples:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.9"
- name: Set up Terraform
uses: hashicorp/setup-terraform@v1
with:
terraform_version: 1.0.9
terraform_wrapper: false
- name: Set environment
run: |
echo "GOOGLE_APPLICATION_CREDENTIALS=${GITHUB_WORKSPACE}/.github/workflows/fake-key.json" >> $GITHUB_ENV
- name: Install dependencies
run: |
pip install -r tests/requirements.txt
- name: Run tests examples
id: test-examples
run: |
pytest -n 4 -vv tests/modules/examples
modules:
runs-on: ubuntu-latest
steps:
@ -98,12 +105,15 @@ jobs:
- name: Set up Terraform
uses: hashicorp/setup-terraform@v1
with:
terraform_version: 1.0.9
terraform_version: 1.1.3
terraform_wrapper: false
- name: Set environment
run: |
echo "TF_PLUGIN_CACHE_DIR=${{ github.workspace }}/.terraform.d/plugin-cache-${GITHUB_JOB}" >> $GITHUB_ENV
echo "GOOGLE_APPLICATION_CREDENTIALS=${GITHUB_WORKSPACE}/.github/workflows/fake-key.json" >> $GITHUB_ENV
mkdir --parents ${{ github.workspace }}/.terraform.d/plugin-cache-${GITHUB_JOB}
terraform -chdir=tests providers lock
- name: Install dependencies
run: |

5
.gitignore vendored
View File

@ -19,3 +19,8 @@ bundle.zip
**/packer_cache
**/*.pkrvars.hcl
fixture_*
fast/configs
fast/stages/**/providers.tf
fast/stages/**/terraform.tfvars
fast/stages/**/terraform.tfvars.json
fast/stages/**/terraform-*.auto.tfvars.json

14
README.md
View File

@ -6,17 +6,17 @@
This repository provides **end-to-end examples** and a **suite of Terraform modules** for Google Cloud, which support different use cases:
- starter kits used to bootstrap real-world cloud foundations, and reference examples used to deep dive on network patterns or product features
- composable modules that support quick prototyping and testing
- a comprehensive source of lean modules that lend themselves well to changes
- organization-wide [landing zone blueprint](fast/) used to bootstrap real-world cloud foundations
- reference [examples](./examples/) used to deep dive on network patterns or product features
- a comprehensive source of lean [modules](./modules/dns) that lend themselves well to changes
The whole repository is meant to be cloned as a single unit, and then forked into separate owned repositories to seed production usage, or used as-is and periodically updated as a complete toolkit for prototyping. You can read more on this approach in our [manifesto](./MANIFESTO.md).
Both the examples and modules require some measure of Terraform skills to be used effectively. If you are looking for a feature-rich black box to manage project or product creation with minimal specific skills, you might be better served by the [Cloud Foundation Toolkit](https://registry.terraform.io/modules/terraform-google-modules) suite of modules.
## End-to-end examples
## Organization blueprint (Fabric FAST)
The [examples](./examples/) in this repository are split in several main sections: **[foundational examples](./examples/foundations/)** that bootstrap the organizational hierarchy and automation prerequisites, **[networking examples](./examples/networking/)** that implement core patterns or features, **[data solutions examples](./examples/data-solutions/)** that demonstrate how to integrate data services in complete scenarios, **[cloud operations examples](./examples/cloud-operations/)** that leverage specific products to meet specific operational needs and **[factories](./examples/factories/)** that implement resource factories for the repetitive creation of specific resources.
Setting up a production-ready GCP organization is often a time-consuming process. Fabric [FAST](fast/) aims to speed up this process via two complementary goals. On the one hand, FAST provides a design of a GCP organization that includes the typical elements required by enterprise customers. Secondly, we provide a reference implementation of the FAST design using Terraform.
## Modules
@ -37,3 +37,7 @@ Currently available modules:
- **serverless** - [Cloud Function](./modules/cloud-function), [Cloud Run](./modules/cloud-run)
For more information and usage examples see each module's README file.
## End-to-end examples
The [examples](./examples/) in this repository are split in several main sections: **[foundational examples](./examples/foundations/)** that bootstrap the organizational hierarchy and automation prerequisites, **[networking examples](./examples/networking/)** that implement core patterns or features, **[data solutions examples](./examples/data-solutions/)** that demonstrate how to integrate data services in complete scenarios, **[cloud operations examples](./examples/cloud-operations/)** that leverage specific products to meet specific operational needs and **[factories](./examples/factories/)** that implement resource factories for the repetitive creation of specific resources.

4
examples/data-solutions/gcs-to-bq-with-dataflow/README.md
View File

@ -113,14 +113,15 @@ You can check data imported into Google BigQuery from the Google Cloud Console U
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| prefix | Unique prefix used for resource names. Not used for project if 'project_create' is null. | <code>string</code> | ✓ | |
| project_id | Project id, references existing project if `project_create` is null. | <code>string</code> | ✓ | |
| prefix | Unique prefix used for resource names. Not used for project if 'project_create' is null. | <code>string</code> | | <code>null</code> |
| project_create | Provide values if project creation is needed, uses existing project if null. Parent is in 'folders/nnn' or 'organizations/nnn' format | <code title="object&#40;&#123;&#10; billing_account_id &#61; string&#10; parent &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| region | The region where resources will be deployed. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
| vpc_subnet_range | Ip range used for the VPC subnet created for the example. | <code>string</code> | | <code>&#34;10.0.0.0&#47;20&#34;</code> |
@ -139,3 +140,4 @@ You can check data imported into Google BigQuery from the Google Cloud Console U

26
examples/factories/net-vpc-firewall-yaml/README.md
View File

@ -136,25 +136,27 @@ web-app-a-ingress:
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
| ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------: | :------: | :---------------: |
| config_directories | List of paths to folders where firewall configs are stored in yaml format. Folder may include subfolders with configuration files. Files suffix must be `.yaml` | <code>list&#40;string&#41;</code> | ✓ | |
| network | Name of the network this set of firewall rules applies to. | <code>string</code> | ✓ | |
| project_id | Project Id. | <code>string</code> | ✓ | |
| log_config | Log configuration. Possible values for `metadata` are `EXCLUDE_ALL_METADATA` and `INCLUDE_ALL_METADATA`. Set to `null` for disabling firewall logging. | <code title="object&#40;&#123;&#10; metadata &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| config_directories | List of paths to folders where firewall configs are stored in yaml format. Folder may include subfolders with configuration files. Files suffix must be `.yaml` | <code>list&#40;string&#41;</code> | ✓ | |
| network | Name of the network this set of firewall rules applies to. | <code>string</code> | ✓ | |
| project_id | Project Id. | <code>string</code> | ✓ | |
| log_config | Log configuration. Possible values for `metadata` are `EXCLUDE_ALL_METADATA` and `INCLUDE_ALL_METADATA`. Set to `null` for disabling firewall logging. | <code title="object&#40;&#123;&#10; metadata &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
## Outputs
| name | description | sensitive |
| ------------------- | -------------------------------- | :-------: |
| egress_allow_rules | Egress rules with allow blocks. | |
| egress_deny_rules | Egress rules with allow blocks. | |
| ingress_allow_rules | Ingress rules with allow blocks. | |
| ingress_deny_rules | Ingress rules with deny blocks. | |
| name | description | sensitive |
|---|---|:---:|
| egress_allow_rules | Egress rules with allow blocks. | |
| egress_deny_rules | Egress rules with allow blocks. | |
| ingress_allow_rules | Ingress rules with allow blocks. | |
| ingress_deny_rules | Ingress rules with deny blocks. | |
<!-- END TFDOC -->

8
examples/factories/project-factory/main.tf
View File

@ -101,7 +101,7 @@ locals {
module "billing-alert" {
for_each = local.billing_alert == null ? {} : { 1 = 1 }
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/billing-budget?ref=v12.0.0"
source = "../../../modules/billing-budget"
billing_account = local.billing_account_id
name = "${module.project.project_id} budget"
amount = local.billing_alert.amount
@ -116,7 +116,7 @@ module "billing-alert" {
}
module "dns" {
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/dns?ref=v12.0.0"
source = "../../../modules/dns"
for_each = toset(var.dns_zones)
project_id = module.project.project_id
type = "private"
@ -126,7 +126,7 @@ module "dns" {
}
module "project" {
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/project?ref=v12.0.0"
source = "../../../modules/project"
billing_account = local.billing_account_id
name = var.project_id
contacts = { for c in local.essential_contacts : c => ["ALL"] }
@ -144,7 +144,7 @@ module "project" {
}
module "service-accounts" {
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/iam-service-account?ref=v12.0.0"
source = "../../../modules/iam-service-account"
for_each = var.service_accounts
name = each.key
project_id = module.project.project_id

28
examples/networking/decentralized-firewall/README.md
View File

@ -21,26 +21,28 @@ the two). There is an example of a YAML-based validator using [Yamale](https://g
in the [`validator/`](validator/) subdirectory, which can be integrated as part of a CI/CD pipeline.
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
| ------------------ | --------------------------------------------------------------------------------------------- | :-------------------------------: | :------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| billing_account_id | Billing account id used as default for new projects. | <code>string</code> | ✓ | |
| prefix | Prefix used for resources that need unique names. | <code>string</code> | ✓ | |
| root_node | Hierarchy node where projects will be created, 'organizations/org_id' or 'folders/folder_id'. | <code>string</code> | ✓ | |
| ip_ranges | Subnet IP CIDR ranges. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; prod &#61; &#34;10.0.16.0&#47;24&#34;&#10; dev &#61; &#34;10.0.32.0&#47;24&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
| project_services | Service APIs enabled by default in new projects. | <code>list&#40;string&#41;</code> | | <code title="&#91;&#10; &#34;container.googleapis.com&#34;,&#10; &#34;dns.googleapis.com&#34;,&#10; &#34;stackdriver.googleapis.com&#34;,&#10;&#93;">&#91;&#8230;&#93;</code> |
| region | Region used. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| billing_account_id | Billing account id used as default for new projects. | <code>string</code> | ✓ | |
| prefix | Prefix used for resources that need unique names. | <code>string</code> | ✓ | |
| root_node | Hierarchy node where projects will be created, 'organizations/org_id' or 'folders/folder_id'. | <code>string</code> | ✓ | |
| ip_ranges | Subnet IP CIDR ranges. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; prod &#61; &#34;10.0.16.0&#47;24&#34;&#10; dev &#61; &#34;10.0.32.0&#47;24&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
| project_services | Service APIs enabled by default in new projects. | <code>list&#40;string&#41;</code> | | <code title="&#91;&#10; &#34;container.googleapis.com&#34;,&#10; &#34;dns.googleapis.com&#34;,&#10; &#34;stackdriver.googleapis.com&#34;,&#10;&#93;">&#91;&#8230;&#93;</code> |
| region | Region used. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
## Outputs
| name | description | sensitive |
| -------- | --------------- | :-------: |
| fw_rules | Firewall rules. | |
| projects | Project ids. | |
| vpc | Shared VPCs. | |
| name | description | sensitive |
|---|---|:---:|
| fw_rules | Firewall rules. | |
| projects | Project ids. | |
| vpc | Shared VPCs. | |
<!-- END TFDOC -->

87
fast/README.md Normal file
View File

@ -0,0 +1,87 @@
# Fabric FAST
Setting up a production-ready GCP organization is often a time-consuming process. Fabric FAST aims to speed up this process via two complementary goals. On the one hand, FAST provides a design of a GCP organization that includes the typical elements required by enterprise customers. Secondly, we provide a reference implementation of the FAST design using Terraform.
Note that while our implementation is necessarily influenced (and constrained) by the way Terraform works, the design we put forward only refers to GCP constructs and features. In other words, while we use Terraform for our reference implementation, in theory, the FAST design can be implemented using any other tool (e.g., Pulumi, bash scripts, or even calling the relevant APIs directly).
Fabric FAST comes from engineers in Google Cloud's Professional Services Organization, with a combined experience of decades solving the typical technical problems faced by GCP customers. While every GCP user has specific requirements, many common issues arise repeatedly. Solving those issues correctly from the beginning is key to a robust and scalable GCP setup. It's those common issues and their solutions that Fabric FAST aims to collect and present coherently.
Fabric FAST was initially conceived to help enterprises quickly set up a GCP organization following battle-tested and widely-used patterns. Despite its origin in enterprise environments, FAST includes many customization points making it an ideal blueprint for organizations of all sizes, ranging from startups to the largest companies.
## Guiding principles
### Contracts and stages
FAST uses the concept of stages, which individually perform precise tasks but, taken together, build a functional, ready-to-use GCP organization. More importantly, stages are modeled around the security boundaries that typically appear in mature organizations. This arrangement allows delegating ownership of each stage to the team responsible for the types of resources it manages. For example, as its name suggests, the networking stage sets up all the networking elements and is usually the responsibility of a dedicated networking team within the organization.
From the perspective of FAST's overall design, stages also work as contacts or interfaces, defining a set of pre-requisites and inputs required to perform their designed task and generating outputs needed by other stages lower in the chain. The diagram below shows the relationships between stages.
<p align="center">
<img src="stages.svg" alt="Stages diagram">
</p>
### Security-first design
Security was, from the beginning, one of the most critical elements in the design of Fabric FAST. Many of FAST's design decisions aim to build the foundations of a secure organization. In fact, the first two stages deal mainly with the organization-wide security setup.
FAST also aims to minimize the number of permissions granted to principals according to the security-first approach previously mentioned. We achieve this through the meticulous use of groups, service accounts, custom roles, and [Cloud IAM Conditions](https://cloud.google.com/iam/docs/conditions-overview), among other things.
### Extensive use of factories
A resource factory consumes a simple representation of a resource (e.g., in YAML) and deploys it (e.g., using Terraform). Used correctly, factories can help decrease the management overhead of large-scale infrastructure deployments. See "[Resource Factories: A descriptive approach to Terraform](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c)" for more details and the rationale behind factories.
FAST uses YAML-based factories to deploy subnets and firewall rules and, as its name suggests, in the [project factory](./stages/03-project-factory/) stage.
## High level design
As mentioned before, fast relies on multiple stages to progressively bring up your GCP organization(s). In this section we briefly describe each stage.
### Organizational level (00-01)
- [Bootstrap](stages/00-bootstrap/README.md)<br/>
Enables critical organization-level functionality that depends on broad permissions. It has two primary purposes. The first is to bootstrap the resources needed to automate this and the following stages (service accounts, GCS buckets). And secondly, it applies the minimum amount of configuration needed at the organization level, to avoid the need for broad permissions later on, and to implement a minimum of security features like sinks and exports from the start.
- [Resource Management](stages/01-resman/README.md)<br/>
Creates the base resource hierarchy (folders) and the automation resources required to delegate each part of the hierarchy to separate stages. This stage also configures organization-level policies and any exceptions needed by different branches of the resource hierarchy.
### Shared resources (02)
- [Security](stages/02-security/README.md)<br/>
Manages centralized security configurations in a separate stage, typically owned by the security team. This stage implements VPC Security Controls via separate perimeters for environments and central services, and creates projects to host centralized KMS keys used by the whole organization. It's intentionally easy to extend to include other security-related resources, like Secret Manager.
- [Networking](stages/02-networking/README.md)<br/>
Manages centralized network resources in a separate stage, and is typically owned by the networking team. This stage implements a hub-and-spoke design, includes connectivity via VPN to on-premises, and YAML-based factories for firewall rules (hierarchical and VPC-level) and subnets.
### Environment-level resources (03)
- [Project Factory](03-projectfactory/prod/README.md)<br/>
YAML-based factory to create and configure application- or team-level projects. Configuration includes VPC-level settings for Shared VPC, service-level configuration for CMEK encryption via centralized keys, and service account creation for workloads and applications. This stage is meant to be used once per environment.
- Data Platform (in development)
- GKE Multitenant (in development)
- GCE Migration (in development)
Please refer to the READMEs of each stage for further details.
## Implementation
There are many decisions and tasks required to convert an empty GCP organization to one that can host production environments safely. Arguably, FAST could expose those decisions as configuration options to allow for different outcomes. However, supporting all the possible combinations is almost impossible and leads to code which is hard to maintain efficiently.
Instead, FAST aims to leverage different reference architectures as “pluggable modules”, and then have a small set of variables covering only the essential options of each stage. While we could expose every option of the underlying resources as stage-level variables, we prefer to provide the basic implementation and encourage users to modify the codebase if additional (or different) behavior is needed.
Since we expect users to customize FAST to their specific needs, we strive to make its code easy to understand and modify. Root-level modules (i.e., stages) should be low in complexity, which among other things, means:
- Code should avoid magic and be as explicit as possible.
- We hide advanced features and complexity behind modules.
- We prefer as little indirection as possible.
- We favor flat over nested.
We also recognize that FAST users don't need all of its features. Therefore, you don't need to use our project factory or our GKE implementation if you don't want to. Instead, remove those stages or pieces of code and keep what suits you.
Those familiar with Python will note that FAST follows many of the maxims in the [Zen of Python](https://www.python.org/dev/peps/pep-0020/#id2).
## Roadmap
Besides the features already described, FAST roadmap includes:
- Stage to deploy environment-specific multitenant GKE clusters following Google's best practices
- Stage to deploy a fully featured data platform
- Reference implementation to use FAST in CI/CD pipelines
- Static policy enforcement

23
fast/TODO.md Normal file
View File

@ -0,0 +1,23 @@
TODO before merging
- [x] fix tests
- [x] fix linting errors
- [x] fast-specific .gitignore
- [x] YAML samples thingy
- [ ] stages README
- [ ] fabric top-level README
- [x] proper docstring on new tools
- [x] validate_schema.py
- [x] tfutils.py (deleted, it's an empty shell)
- [x] check_boilerplate.py
- [x] check_documentation.py
- [x] remove GKE branch from resman and update diagram
- [x] remove GKE branch from resman and update diagram
- [x] modify github actions for different fast tfdoc usage
- [x] add roadmap to top-level fast README
- [x] update modules references to local paths
- [x] stage 00 (ludo)
- [x] stage 01 (julio)
- [x] stage 02-net (simo)
- [x] stage 02-sec (ludo)
- [x] stage 03-pf (simo)

29
fast/assets/schemas/firewall_rules.schema.yaml Normal file
View File

@ -0,0 +1,29 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
map(include('firewall_rule'))
---
firewall_rule:
description: str()
direction: enum("INGRESS", "EGRESS")
action: enum("allow", "deny")
sources: list(str())
ranges: list(str())
targets: list(str())
use_service_accounts: bool()
rules: list(include('rule'))
---
rule:
protocol: enum("tcp", "udp", "all")
ports: list(num())

25
fast/assets/schemas/hierarchical_rules.schema.yaml Normal file
View File

@ -0,0 +1,25 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
map(include('hierarchical_rule'))
---
hierarchical_rule:
description: str()
direction: enum("INGRESS", "EGRESS")
action: enum("allow", "deny")
priority: int()
ranges: list(str())
target_resources: any(null(), list(str()))
ports: map(list(str(), required=False))
enable_logging: bool()

57
fast/assets/schemas/project.schema.yaml Normal file
View File

@ -0,0 +1,57 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
billing_account_id: str(matches='[A-F0-9]{6}-[A-F0-9]{6}-[A-F0-9]{6}', required=False)
billing_alert: any(include('billing_alert'), null(), required=False) # If set to null, use defaults
dns_zones: list(str(), required=False)
essential_contacts: list(str(), required=False) # Also used for billing alerts
folder_id: str(matches='(organizations/|folders/)[0-9]*$')
group_iam: map(list(str()), key=str(), required=False)
iam: map(list(str()), key=str(), required=False)
kms_service_agents: map(list(str()), key=str(), required=False)
labels: map(str(), key=str(), required=False)
org_policies: include('org_policies', required=False)
secrets: map(list(str()), key=str(), required=False)
service_accounts: map(list(str()), required=False)
services: list(str(matches='^[a-z-]*\.googleapis\.com$'), required=False)
services_iam: map(list(str()), key=str(), required=False)
vpc: include('vpc', required=False)
---
billing_alert:
amount: int()
thresholds: include('billing_alert_thresholds')
credit_treatment: enum("INCLUDE_ALL_CREDITS", "EXCLUDE_ALL_CREDITS")
---
billing_alert_thresholds:
current: list(num(min=0, max=1))
forecasted: list(num(min=0, max=1))
---
gke_setup:
enable_security_admin: bool(required=False)
enable_host_service_agent: bool(required=False)
---
org_policies:
policy_boolean: map(bool(), key=str(matches='^constraints/[A-z\.]*$'), required=False)
policy_list: map(include('policy_list'), key=str(matches='^constraints/[A-z\.]*$'), required=False)
---
policy_list:
inherit_from_parent: any(bool(), null())
suggested_value: any(str(), null())
status: any(bool(), null())
values: list(str())
---
vpc:
host_project: str(matches='[a-z]([-a-z0-9]*[a-z0-9])?', min=6, max=30)
gke_setup: include('gke_setup', required=False)
subnets_iam: map(list(str()), key=str(matches='^[a-z0-9-]*/[a-z0-9-]*$'), required=False)

28
fast/assets/schemas/project_defaults.schema.yaml Normal file
View File

@ -0,0 +1,28 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
billing_account_id: str(matches='[A-F0-9]{6}-[A-F0-9]{6}-[A-F0-9]{6}', required=False)
billing_alert: any(include('billing_alert'), null(), required=False)
essential_contacts: list(str(), required=False)
labels: map(str(), key=str(), required=False)
notification_channels: list(str(), required=False)
---
billing_alert:
amount: int()
thresholds: include('billing_alert_thresholds')
credit_treatment: enum('INCLUDE_ALL_CREDITS', 'EXCLUDE_ALL_CREDITS')
---
billing_alert_thresholds:
current: list(num(min=0, max=1))
forecasted: list(num(min=0, max=1))

29
fast/assets/schemas/subnet.schema.yaml Normal file
View File

@ -0,0 +1,29 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
region: str()
description: str()
ip_cidr_range: str()
# optional attributes
private_ip_google_access: bool(required=False) # defaults to true
iam_users: list(str(), required=False)
iam_groups: list(str(), required=False)
iam_service_accounts: list(str(), required=False)
secondary_ip_ranges: list(map(str()), key=str(), required=False)
flow_logs: any(include('flow_logs'), required=False)
---
flow_logs:
- aggregation_interval: enum('INTERVAL_5_SEC', 'INTERVAL_30_SEC', 'INTERVAL_1_MIN', 'INTERVAL_5_MIN', 'INTERVAL_10_MIN', 'INTERVAL_15_MIN', required=False)
- flow_sampling: num(min=0, max=1, required=False)
- metadata: enum('EXCLUDE_ALL_METADATA', 'INCLUDE_ALL_METADATA', 'CUSTOM_METADATA', required=False)

30
fast/assets/templates/providers.tpl Normal file
View File

@ -0,0 +1,30 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
terraform {
backend "gcs" {
bucket = "${bucket}"
impersonate_service_account = "${sa}"
}
}
provider "google" {
impersonate_service_account = "${sa}"
}
provider "google-beta" {
impersonate_service_account = "${sa}"
}
# end provider.tf for ${name}

BIN
fast/stages.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 272 KiB

1074
fast/stages.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 177 KiB

310
fast/stages/00-bootstrap/README.md Normal file
View File

@ -0,0 +1,310 @@
# Organization bootstrap
The primary purpose of this stage is to enable critical organization-level functionality that depends on broad administrative permissions, and prepare the prerequisites needed to enable automation in this and future stages.
It is intentionally simple, to minimize usage of administrative-level permissions and enable simple auditing and troubleshooting, and only deals with three sets of resources:
- project, service accounts, and GCS buckets for automation
- projects, BQ datasets, and sinks for audit log and billing exports
- IAM bindings on the organization
Use the following diagram as a simple high level reference for the following sections, which describe the stage and its possible customizations in detail.
<p align="center">
<img src="diagram.svg" alt="Organization-level diagram">
</p>
## Design overview and choices
As mentioned above, this stage only does the bare minimum required to bootstrap automation, and ensure that base audit and billing exports are in place from the start to provide some measure of accountability, even before the security configurations are applied in a later stage.
It also sets up organization-level IAM bindings so the Organization Administrator role is only used here, trading off some design freedom for ease of auditing and troubleshooting, and reducing the risk of costly security mistakes down the line. The only exception to this rule is for the [Resource Management stage](../01-resman) service account, described below.
### User groups
User groups are important, not only here but throughout the whole automation process. They provide a stable frame of reference that allows decoupling the final set of permissions for each group, from the stage where entities and resources are created and their IAM bindings defined. For example, the final set of roles for the networking group is contributed by this stage at the organization level (XPN Admin, Cloud Asset Viewer, etc.), and by the Resource Management stage at the folder level.
We have standardized the initial set of groups on those outlined in the [GCP Enterprise Setup Checklist](https://cloud.google.com/docs/enterprise/setup-checklist) to simplify adoption. They provide a comprehensive and flexible starting point that can suit most users. Adding new groups, or deviating from the initial setup is possible and reasonably simple, and it's briefly outlined in the customization section below.
### Organization-level IAM
The service account used in the [Resource Management stage](../01-resman) needs to be able to grant specific roles at the organizational level (`roles/billing.user`, `roles/compute.xpnAdmin`, etc.), to enable specific functionality for subsequent stages that deal with network or security resources, or billing-related activities.
In order to be able to assign those roles without having the full authority of the Organization Admin role, this stage defines a custom role that only allows setting IAM policies on the organization, and grants it via a [delegated role grant](https://cloud.google.com/iam/docs/setting-limits-on-granting-roles) that only allows it to be used to grant a limited subset of roles.
In this way, the Resource Management service account can effectively act as an Organization Admin, but only to grant the roles it effectively needs to control.
One consequence of the above setup, is the need to configure IAM bindings as non-authoritative for the roles included in the IAM condition, since those same roles are effectively under the control of two stages: this one and Resource Management. Using authoritative bindings for these roles (instead of non-authoritative ones) would generate potential conflicts, where each stage could try to overwrite and negate the bindings applied by the other at each `apply` cycle.
### Automation project and resources
One other design choice worth mentioning here is using a single automation project for all foundational stages. We trade off some complexity on the API side (single source for usage quota, multiple service activation) for increased flexibility and simpler operations, while still effectively providing the same degree of separation via resource-level IAM.
### Billing account
We support three use cases in regards to billing:
- the billing account is part of this same organization, IAM bindings will be set at the organization level
- the billing account is part of a different organization, billing IAM bindings will be set at the organization level in the billing account owning organization
- the billing account is not considered part of an organization (even though it might be), billing IAM bindings are set on the billing account itself
For same-organization billing, we configure a custom organization role that can set IAM bindings, via a delegated role grant to limit its scope to the relevant roles.
For details on configuring the different billing account modes, refer to the [How to run this stage](#how-to-run-this-stage) section below.
### Naming
We are intentionally not supporting random prefix/suffixes for names, as that is an antipattern typically only used in development. It does not map to our customer's actual production usage, where they always adopt a fixed naming convention.
What is implemented here is a fairly common convention, composed of tokens ordered by relative importance:
- a static prefix (e.g. `myco` or `myco-gcp`)
- an environment identifier (e.g. `prod`)
- a team/owner identifier (e.g. `sec` for Security)
- a context identifier (e.g. `core` or `kms`)
- an arbitrary identifier used to distinguish similar resources (e.g. `0`, `1`)
Tokens are joined by a `-` character, making it easy to separate the individual tokens visually, and to programmatically split them in billing exports to derive initial high-level groupings for cost attribution.
The convention is used in its full form only for specific resources with globally unique names (projects, GCS buckets). Other resources adopt a shorter version for legibility, as the full context can always be derived from their project.
The [Customizations](#names-and-naming-convention) section on names below explains how to configure tokens, or implement a different naming convention.
## How to run this stage
This stage has straightforward initial requirements, as it is designed to work on newly created GCP organizations. Four steps are needed to bring up this stage:
- an Organization Admin self-assigns the required roles listed below
- the same administrator runs the first `init/apply` sequence passing a special variable to `apply`
- the providers configuration file is derived from the Terraform output or linked from the generated file
- a second `init` is run to migrate state, and from then on, the stage is run via impersonation
### Prerequisites
The roles that the Organization Admin used in the first `apply` needs to self-grant are:
- Billing Account Administrator (`roles/billing.admin`)
either on the organization or the billing account (see the following section for details)
- Logging Admin (`roles/logging.admin`)
- Organization Role Administrator (`roles/iam.organizationRoleAdmin`)
- Organization Administrator (`roles/resourcemanager.organizationAdmin`)
- Project Creator (`roles/resourcemanager.projectCreator`)
To quickly self-grant the above roles, run the following code snippet as the initial Organization Admin:
```bash
export BOOTSTRAP_ORG_ID=123456
export BOOTSTRAP_USER=$(gcloud config list --format 'value(core.account)')
export BOOTSTRAP_ROLES=(roles/billing.admin roles/logging.admin roles/iam.organizationRoleAdmin roles/resourcemanager.projectCreator)
for role in $BOOTSTRAP_ROLES; do
gcloud organizations add-iam-policy-binding $BOOTSTRAP_ORG_ID \
--member user:$BOOTSTRAP_USER --role $role
done
```
#### Billing account in a different organization
If you are using a billing account belonging to a different organization (e.g. in multiple organization setups), some initial configurations are needed to ensure the identities running this stage can assign billing-related roles.
If the billing organization is managed by another version of this stage, we leverage the `organizationIamAdmin` role created there, to allow restricted granting of billing roles at the organization level.
If that's not the case, an equivalent role needs to exist, or the predefined `resourcemanager.organizationAdmin` role can be used if not managed authoritatively. The role name then needs to be manually changed in the `billing.tf` file, in the `google_organization_iam_binding` resource.
The identity applying this stage for the first time also needs two roles in billing organization, they can be removed after the first `apply` completes successfully:
```bash
export BILLING_ORG_ID=789012
export BILLING_ROLES=(roles/billing.admin roles/resourcemanager.organizationAdmin)
for role in $BILLING_ROLES; do
gcloud organizations add-iam-policy-binding $BILLING_ORG_ID \
--member user:$BOOTSTRAP_USER --role $role
done
```
#### Standalone billing account
If you are using a standalone billing account, the identity applying this stage for the first time needs to be a billing account administrator:
```bash
export BILLING_ACCOUNT_ID=ABCD-01234-ABCD
gcloud beta billing accounts add-iam-policy-binding $BILLING_ACCOUNT \
--member user:$BOOTSTRAP_USER --role roles/billing.admin
```
#### Groups
Before the first run, the following IAM groups must exist to allow IAM bindings to be created (actual names are flexible, see the [Customization](#customizations) section):
- gcp-billing-admins
- gcp-devops
- gcp-network-admins
- gcp-organization-admins
- gcp-security-admins
- gcp-support
#### Configure variables
Then make sure you have configured the correct values for the following variables by editing providing a `terraform.tfvars` file:
- `billing_account`
an object containing the id of your billing account, derived from the Cloud Console UI or by running `gcloud beta billing accounts list`, and the id of the organization owning it, or `null` to use the billing account in isolation
- `groups`
the name mappings for your groups, if you're following the default convention you can leave this to the provided default
- `organization.id`, `organization.domain`, `organization.customer_id`
the id, domain and customer id of your organization, derived from the Cloud Console UI or by running `gcloud organizations list`
- `prefix`
the fixed prefix used in your naming convention
### Output files and cross-stage variables
At any time during the life of this stage, you can configure it to automatically generate provider configurations and variable files for the following, to simplify exchanging inputs and outputs between stages and avoid having to edit files manually.
Automatic generation of files is disabled by default. To enable the mechanism, set the `outputs_location` variable to a valid path on a local filesystem, e.g.
```hcl
outputs_location = "../../configs"
```
Once the variable is set, `apply` will generate and manage providers and variables files, including the initial one used for this stage after the first run. You can then link these files in the relevant stages, instead of manually transfering outputs from one stage, to Terraform variables in another.
Below is the outline of the output files generated by this stage:
```bash
[path specified in outputs_location]
├── 00-bootstrap
│   ├── providers.tf
├── 01-resman
│   ├── providers.tf
│   ├── terraform-bootstrap.auto.tfvars.json
├── 02-networking
│   ├── providers.tf
│   ├── terraform-bootstrap.auto.tfvars.json
├── 02-security
│   ├── providers.tf
│   ├── terraform-bootstrap.auto.tfvars.json
├── 03-project-factory-dev
│   └── terraform-bootstrap.auto.tfvars.json
├── 03-project-factory-prod
│   └── terraform-bootstrap.auto.tfvars.json
```
### Running the stage
The first `apply` run as a user needs a special runtime variable, so that the user roles are preserved when setting IAM bindings:
```bash
terraform init
terraform apply \
-var bootstrap_user=$(gcloud config list --format 'value(core.account)')
```
Once the initial `apply` completes successfully, configure a remote backend using the new GCS bucket, and impersonation on the automation service account for this stage. To do this, you can use the generated `providers.tf` file if you have configured output files as described above, or extract its contents from Terraform's output, then migrate state with `terraform init`:
```bash
# if using output files via the outputs_location variable
ln -s [path set in outputs_location]/00-bootstrap/* ./
# or from outputs if not using output files
terraform output -json providers | jq -r '.["00-bootstrap"]' \
> providers.tf
# migrate state to GCS bucket configured in providers file
terraform init -migrate-state
```
## Customizations
Most variables (e.g. `billing_account` and `organization`) are only used to input actual values and should be self-explanatory. The only meaningful customizations that apply here are groups, and IAM roles.
### Group names
As we mentioned above, groups reflect the convention used in the [GCP Enterprise Setup Checklist](https://cloud.google.com/docs/enterprise/setup-checklist), with an added level of indirection: the `groups` variable maps logical names to actual names, so that you don't need to delve into the code if your group names do not comply with the checklist convention.
For example, if your network admins team is called `net-rockstars@example.com`, simply set that name in the variable, minus the domain which is interpolated internally with the organization domain:
```hcl
variable "groups" {
description = "Group names to grant organization-level permissions."
type = map(string)
default = {
gcp-network-admins = "net-rockstars"
# [...]
}
}
```
If your groups layout differs substantially from the checklist, define all relevant groups in the `groups` variable, then rearrange IAM roles in the code to match your setup.
### IAM
One other area where we directly support customizations is IAM. The code here, as in all stages, follows a simple pattern derived from best practices:
- operational roles for humans are assigned to groups
- any other principal is a service account
In code, the distinction above reflects on how IAM bindings are specified in the underlying module variables:
- group roles "for humans" always use `iam_groups` variables
- service account roles always use `iam` variables
This makes it easy to tweak user roles by adding mappings to the `iam_groups` variables of the relevant resources, without having to understand and deal with the details of service account roles.
In those cases where roles need to be assigned to end-user service accounts (e.g. an application or pipeline service account), we offer a stage-level `iam` variable that allows pinpointing individual role/members pairs, without having to touch the code internals, to avoid the risk of breaking a critical role for a robot account. The variable can also be used to assign roles to specific users or to groups external to the organization, e.g. to support external suppliers.
The one exception to this convention is for roles which are part of the delegated grant condition described above, and which can then be assigned from other stages. In this case, use the `iam_additive` variable as they are implemented with non-authoritative resources. Using non-authoritative bindings ensure that re-executing this stage will not override any bindings set in downstream stages.
### Names and naming convention
Configuring the individual tokens for the naming convention described above, has varying degrees of complexity:
- the static prefix can be set via the `prefix` variable once
- the environment identifier is set to `prod` as resources here influence production and are considered as such, and can be changed in `main.tf` locals
All other tokens are set directly in resource names, as providing abstractions to manage them would have added too much complexity to the code, making it less readable and more fragile.
If a different convention is needed, identify names via search/grep (e.g. with `^\s+name\s+=\s+"`) and change them in an editor: it should take a couple of minutes at most, as there's just a handful of modules and resources to change.
Names used in internal references (e.g. `module.foo-prod.id`) are only used by Terraform and do not influence resource naming, so they are best left untouched to avoid having to debug complex errors.
<!-- BEGIN TFDOC -->
## Files
| name | description | modules | resources |
|---|---|---|---|
| [automation.tf](./automation.tf) | Automation project and resources. | <code>gcs</code> · <code>iam-service-account</code> · <code>project</code> | |
| [billing.tf](./billing.tf) | Billing export project and dataset. | <code>bigquery-dataset</code> · <code>organization</code> · <code>project</code> | <code>google_billing_account_iam_member</code> · <code>google_organization_iam_binding</code> |
| [log-export.tf](./log-export.tf) | Audit log project and sink. | <code>bigquery-dataset</code> · <code>gcs</code> · <code>logging-bucket</code> · <code>project</code> · <code>pubsub</code> | |
| [main.tf](./main.tf) | Module-level locals and resources. | | |
| [organization.tf](./organization.tf) | Organization-level IAM and org policies. | <code>organization</code> | <code>google_organization_iam_binding</code> |
| [outputs.tf](./outputs.tf) | Module outputs. | | <code>local_file</code> |
| [variables.tf](./variables.tf) | Module variables. | | |
## Variables
| name | description | type | required | default | producer |
|---|---|:---:|:---:|:---:|:---:|
| billing_account | Billing account id and organization id ('nnnnnnnn' or null). | <code title="object&#40;&#123;&#10; id &#61; string&#10; organization_id &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | |
| organization | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; number&#10; customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | |
| prefix | Prefix used for resources that need unique names. | <code>string</code> | ✓ | | |
| bootstrap_user | Email of the nominal user running this stage for the first time. | <code>string</code> | | <code>null</code> | |
| groups | Group names to grant organization-level permissions. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; gcp-billing-admins &#61; &#34;gcp-billing-admins&#34;,&#10; gcp-devops &#61; &#34;gcp-devops&#34;,&#10; gcp-network-admins &#61; &#34;gcp-network-admins&#34;&#10; gcp-organization-admins &#61; &#34;gcp-organization-admins&#34;&#10; gcp-security-admins &#61; &#34;gcp-security-admins&#34;&#10; gcp-support &#61; &#34;gcp-support&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| iam | Organization-level custom IAM settings in role => [principal] format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| iam_additive | Organization-level custom IAM settings in role => [principal] format for non-authoritative bindings. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| log_sinks | Org-level log sinks, in name => {type, filter} format. | <code title="map&#40;object&#40;&#123;&#10; filter &#61; string&#10; type &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; audit-logs &#61; &#123;&#10; filter &#61; &#34;logName:&#92;&#34;&#47;logs&#47;cloudaudit.googleapis.com&#37;2Factivity&#92;&#34; OR logName:&#92;&#34;&#47;logs&#47;cloudaudit.googleapis.com&#37;2Fsystem_event&#92;&#34;&#34;&#10; type &#61; &#34;bigquery&#34;&#10; &#125;&#10; vpc-sc &#61; &#123;&#10; filter &#61; &#34;protoPayload.metadata.&#64;type&#61;&#92;&#34;type.googleapis.com&#47;google.cloud.audit.VpcServiceControlAuditMetadata&#92;&#34;&#34;&#10; type &#61; &#34;bigquery&#34;&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| outputs_location | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | <code>string</code> | | <code>null</code> | |
## Outputs
| name | description | sensitive | consumers |
|---|---|:---:|---|
| billing_dataset | BigQuery dataset prepared for billing export. | | |
| project_ids | Projects created by this stage. | | |
| providers | Terraform provider files for this stage and dependent stages. | ✓ | <code>stage-01</code> |
| tfvars | Terraform variable files for the following stages. | ✓ | |
<!-- END TFDOC -->

107
fast/stages/00-bootstrap/automation.tf Normal file
View File

@ -0,0 +1,107 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Automation project and resources.
module "automation-project" {
source = "../../../modules/project"
billing_account = var.billing_account.id
name = "iac-core-0"
parent = "organizations/${var.organization.id}"
prefix = local.prefix
# human (groups) IAM bindings
group_iam = {
(local.groups.gcp-devops) = [
"roles/iam.serviceAccountAdmin",
"roles/iam.serviceAccountTokenCreator",
]
(local.groups.gcp-organization-admins) = [
"roles/iam.serviceAccountTokenCreator",
]
}
# machine (service accounts) IAM bindings
iam = {
"roles/owner" = [module.automation-tf-bootstrap-sa.iam_email]
"roles/iam.serviceAccountAdmin" = [
module.automation-tf-bootstrap-sa.iam_email,
module.automation-tf-resman-sa.iam_email
]
"roles/storage.admin" = [
module.automation-tf-bootstrap-sa.iam_email,
module.automation-tf-resman-sa.iam_email
]
}
services = [
"accesscontextmanager.googleapis.com",
"bigquery.googleapis.com",
"bigqueryreservation.googleapis.com",
"bigquerystorage.googleapis.com",
"billingbudgets.googleapis.com",
"cloudbilling.googleapis.com",
"cloudkms.googleapis.com",
"cloudresourcemanager.googleapis.com",
"compute.googleapis.com",
"essentialcontacts.googleapis.com",
"iam.googleapis.com",
"pubsub.googleapis.com",
"servicenetworking.googleapis.com",
"serviceusage.googleapis.com",
"stackdriver.googleapis.com",
"storage-component.googleapis.com",
"storage.googleapis.com",
]
}
# this stage's bucket and service account
module "automation-tf-bootstrap-gcs" {
source = "../../../modules/gcs"
project_id = module.automation-project.project_id
name = "iac-core-bootstrap-0"
prefix = local.prefix
versioning = true
depends_on = [module.organization]
}
module "automation-tf-bootstrap-sa" {
source = "../../../modules/iam-service-account"
project_id = module.automation-project.project_id
name = "bootstrap-0"
description = "Terraform organization bootstrap service account."
prefix = local.prefix
}
# resource hierarchy stage's bucket and service account
module "automation-tf-resman-gcs" {
source = "../../../modules/gcs"
project_id = module.automation-project.project_id
name = "iac-core-resman-0"
prefix = local.prefix
versioning = true
iam = {
"roles/storage.objectAdmin" = [module.automation-tf-resman-sa.iam_email]
}
depends_on = [module.organization]
}
module "automation-tf-resman-sa" {
source = "../../../modules/iam-service-account"
project_id = module.automation-project.project_id
name = "resman-0"
description = "Terraform organization bootstrap service account."
prefix = local.prefix
}

102
fast/stages/00-bootstrap/billing.tf Normal file
View File

@ -0,0 +1,102 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Billing export project and dataset.
locals {
# used here for convenience, in organization.tf members are explicit
billing_ext_admins = [
local.groups_iam.gcp-organization-admins,
module.automation-tf-bootstrap-sa.iam_email,
module.automation-tf-resman-sa.iam_email
]
}
# billing account in same org (IAM is in the organization.tf file)
module "billing-export-project" {
source = "../../../modules/project"
count = local.billing_org ? 1 : 0
billing_account = var.billing_account.id
name = "billing-export-0"
parent = "organizations/${var.organization.id}"
prefix = local.prefix
iam = {
"roles/owner" = [module.automation-tf-bootstrap-sa.iam_email]
}
services = [
# "cloudresourcemanager.googleapis.com",
# "iam.googleapis.com",
# "serviceusage.googleapis.com",
"bigquery.googleapis.com",
"bigquerydatatransfer.googleapis.com",
"storage.googleapis.com"
]
}
module "billing-export-dataset" {
source = "../../../modules/bigquery-dataset"
count = local.billing_org ? 1 : 0
project_id = module.billing-export-project.0.project_id
id = "billing_export"
friendly_name = "Billing export."
}
# billing account in a different org
module "billing-organization-ext" {
source = "../../../modules/organization"
count = local.billing_org_ext ? 1 : 0
organization_id = "organizations/${var.billing_account.organization_id}"
iam_additive = {
"roles/billing.admin" = local.billing_ext_admins
}
}
resource "google_organization_iam_binding" "billing_org_ext_admin_delegated" {
# refer to organization.tf for the explanation of how this binding works
count = local.billing_org_ext ? 1 : 0
org_id = var.billing_account.organization_id
# if the billing org does not have our custom role, user the predefined one
# role = "roles/resourcemanager.organizationAdmin"
role = "organizations/${var.billing_account.organization_id}/roles/organizationIamAdmin"
members = [module.automation-tf-resman-sa.iam_email]
condition {
title = "automation_sa_delegated_grants"
description = "Automation service account delegated grants"
expression = format(
"api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
join(",", formatlist("'%s'", [
"roles/billing.costsManager",
"roles/billing.user",
]
))
)
}
depends_on = [module.billing-organization-ext]
}
# standalone billing account
resource "google_billing_account_iam_member" "billing_ext_admin" {
for_each = toset(
local.billing_ext ? local.billing_ext_admins : []
)
billing_account_id = var.billing_account.id
role = "roles/billing.admin"
member = each.key
}

BIN
fast/stages/00-bootstrap/diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 42 KiB

807
fast/stages/00-bootstrap/diagram.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 158 KiB

73
fast/stages/00-bootstrap/log-export.tf Normal file
View File

@ -0,0 +1,73 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Audit log project and sink.
locals {
log_types = toset([for k, v in var.log_sinks : v.type])
}
module "log-export-project" {
source = "../../../modules/project"
name = "audit-logs-0"
parent = "organizations/${var.organization.id}"
prefix = local.prefix
billing_account = var.billing_account.id
iam = {
"roles/owner" = [module.automation-tf-bootstrap-sa.iam_email]
}
services = [
# "cloudresourcemanager.googleapis.com",
# "iam.googleapis.com",
# "serviceusage.googleapis.com",
"bigquery.googleapis.com",
"storage.googleapis.com",
"stackdriver.googleapis.com"
]
}
# one log export per type, with conditionals to skip those not needed
module "log-export-dataset" {
source = "../../../modules/bigquery-dataset"
count = contains(local.log_types, "bigquery") ? 1 : 0
project_id = module.log-export-project.project_id
id = "audit_export"
friendly_name = "Audit logs export."
}
module "log-export-gcs" {
source = "../../../modules/gcs"
count = contains(local.log_types, "storage") ? 1 : 0
project_id = module.log-export-project.project_id
name = "audit-logs-0"
prefix = local.prefix
}
module "log-export-logbucket" {
source = "../../../modules/logging-bucket"
count = contains(local.log_types, "logging") ? 1 : 0
parent_type = "project"
parent = module.log-export-project.project_id
id = "audit-logs-0"
}
module "log-export-pubsub" {
source = "../../../modules/pubsub"
for_each = toset([for k, v in var.log_sinks : k if v == "pubsub"])
project_id = module.log-export-project.project_id
name = "audit-logs-${each.key}"
}

32
fast/stages/00-bootstrap/main.tf Normal file
View File

@ -0,0 +1,32 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
groups = {
for k, v in var.groups :
k => "${v}@${var.organization.domain}"
}
groups_iam = {
for k, v in local.groups :
k => "group:${v}"
}
# convenience flags that express where billing account resides
billing_ext = var.billing_account.organization_id == null
billing_org = var.billing_account.organization_id == var.organization.id
billing_org_ext = !local.billing_ext && !local.billing_org
# naming: environment used in most resource names
prefix = join("-", compact([var.prefix, "prod"]))
}

205
fast/stages/00-bootstrap/organization.tf Normal file
View File

@ -0,0 +1,205 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Organization-level IAM and org policies.
locals {
# organization authoritative IAM bindings, in an easy to edit format before
# they are combined with var.iam a bit further in locals
_iam = {
"roles/browser" = [
"domain:${var.organization.domain}"
]
"roles/logging.admin" = [
module.automation-tf-bootstrap-sa.iam_email
]
"roles/owner" = local._iam_bootstrap_user
"roles/resourcemanager.folderAdmin" = [
module.automation-tf-resman-sa.iam_email
]
"roles/resourcemanager.organizationAdmin" = concat(
[module.automation-tf-bootstrap-sa.iam_email],
local._iam_bootstrap_user
)
"roles/resourcemanager.organizationViewer" = [
"domain:${var.organization.domain}"
]
}
# organization additive IAM bindings, in an easy to edit format before
# they are combined with var.iam_additive a bit further in locals
_iam_additive = merge(
{
"roles/accesscontextmanager.policyAdmin" = [
local.groups_iam.gcp-security-admins
]
"roles/compute.orgFirewallPolicyAdmin" = [
local.groups_iam.gcp-network-admins
]
"roles/compute.xpnAdmin" = [
local.groups_iam.gcp-network-admins
]
# use additive to support cross-org roles for billing
"roles/iam.organizationRoleAdmin" = [
local.groups_iam.gcp-security-admins,
module.automation-tf-bootstrap-sa.iam_email
]
"roles/orgpolicy.policyAdmin" = [
module.automation-tf-resman-sa.iam_email,
local.groups_iam.gcp-security-admins
]
},
local.billing_org ? {
"roles/billing.admin" = [
local.groups_iam.gcp-organization-admins,
module.automation-tf-bootstrap-sa.iam_email,
module.automation-tf-resman-sa.iam_email
]
} : {}
)
_iam_bootstrap_user = (
var.bootstrap_user == null ? [] : ["user:${var.bootstrap_user}"]
)
_log_sink_destinations = {
bigquery = try(module.log-export-dataset.0.id, null),
logging = try(module.log-export-logbucket.0.id, null),
storage = try(module.log-export-gcs.0.name, null)
}
iam = {
for role in local.iam_roles : role => distinct(concat(
try(sort(local._iam[role]), []),
try(sort(var.iam[role]), [])
))
}
iam_additive = {
for role in local.iam_roles_additive : role => distinct(concat(
try(sort(local._iam_additive[role]), []),
try(sort(var.iam_additive[role]), [])
))
}
iam_roles = distinct(concat(
keys(local._iam), keys(var.iam)
))
iam_roles_additive = distinct(concat(
keys(local._iam_additive), keys(var.iam_additive)
))
log_sink_destinations = {
for k, v in var.log_sinks : k => (
v.type == "pubsub"
? module.log-export-pubsub[k]
: local._log_sink_destinations[v.type]
)
}
}
module "organization" {
source = "../../../modules/organization"
organization_id = "organizations/${var.organization.id}"
# human (groups) IAM bindings
group_iam = {
(local.groups.gcp-organization-admins) = [
"roles/cloudasset.owner",
"roles/cloudsupport.admin",
"roles/compute.osAdminLogin",
"roles/compute.osLoginExternalUser",
"roles/owner",
"roles/resourcemanager.folderAdmin",
"roles/resourcemanager.organizationAdmin",
"roles/resourcemanager.projectCreator",
],
(local.groups.gcp-network-admins) = [
"roles/cloudasset.owner",
"roles/cloudsupport.techSupportEditor",
]
(local.groups.gcp-security-admins) = [
"roles/cloudasset.owner",
"roles/cloudsupport.techSupportEditor",
"roles/iam.securityReviewer",
"roles/logging.admin",
"roles/securitycenter.admin",
],
(local.groups.gcp-support) = [
"roles/cloudsupport.techSupportEditor",
"roles/logging.viewer",
"roles/monitoring.viewer",
]
}
# machine (service accounts) IAM bindings
iam = local.iam
# additive bindings, used for roles co-managed by different stages
iam_additive = local.iam_additive
custom_roles = {
# this is needed for use in additive IAM bindings, to avoid conflicts
"organizationIamAdmin" = [
"resourcemanager.organizations.get",
"resourcemanager.organizations.getIamPolicy",
"resourcemanager.organizations.setIamPolicy"
]
"xpnServiceAdmin" = [
"compute.globalOperations.get",
"compute.organizations.disableXpnResource",
"compute.organizations.enableXpnResource",
"compute.projects.get",
"compute.subnetworks.getIamPolicy",
"compute.subnetworks.setIamPolicy",
"dns.networks.bindPrivateDNSZone",
"resourcemanager.projects.get",
]
}
logging_sinks = {
for name, attrs in var.log_sinks : name => {
bq_partitioned_table = attrs.type == "bigquery"
destination = local.log_sink_destinations[name]
exclusions = {}
filter = attrs.filter
iam = true
include_children = true
type = attrs.type
}
}
}
# assign the custom restricted Organization Admin role to the relevant service
# accounts, with a condition that only enables granting specific roles;
# these roles use additive bindings everywhere to avoid conflicts / permadiffs
resource "google_organization_iam_binding" "org_admin_delegated" {
org_id = var.organization.id
count = local.billing_org ? 1 : 0
role = module.organization.custom_role_id.organizationIamAdmin
members = [module.automation-tf-resman-sa.iam_email]
condition {
title = "automation_sa_delegated_grants"
description = "Automation service account delegated grants"
expression = format(
"api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
join(",", formatlist("'%s'", concat(
[
"roles/accesscontextmanager.policyAdmin",
"roles/compute.orgFirewallPolicyAdmin",
"roles/compute.xpnAdmin",
"roles/orgpolicy.policyAdmin",
module.organization.custom_role_id.xpnServiceAdmin
],
local.billing_org ? [
"roles/billing.admin",
"roles/billing.costsManager",
"roles/billing.user",
] : []
)))
)
}
depends_on = [module.organization]
}

113
fast/stages/00-bootstrap/outputs.tf Normal file
View File

@ -0,0 +1,113 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
providers = {
"00-bootstrap" = templatefile("${path.module}/../../assets/templates/providers.tpl", {
bucket = module.automation-tf-bootstrap-gcs.name
name = "bootstrap"
sa = module.automation-tf-bootstrap-sa.email
})
"01-resman" = templatefile("${path.module}/../../assets/templates/providers.tpl", {
bucket = module.automation-tf-resman-gcs.name
name = "resman"
sa = module.automation-tf-resman-sa.email
})
}
tfvars = {
"01-resman" = jsonencode({
automation_project_id = module.automation-project.project_id
billing_account = var.billing_account
custom_roles = module.organization.custom_role_id
groups = var.groups
organization = var.organization
prefix = var.prefix
})
"02-networking" = jsonencode({
billing_account_id = var.billing_account.id
organization = var.organization
prefix = var.prefix
})
"02-security" = jsonencode({
billing_account_id = var.billing_account.id
organization = var.organization
prefix = var.prefix
})
"03-gke-multitenant-dev" = jsonencode({
billing_account_id = var.billing_account.id
prefix = var.prefix
})
"03-gke-multitenant-prod" = jsonencode({
billing_account_id = var.billing_account.id
prefix = var.prefix
})
"03-project-factory-dev" = jsonencode({
billing_account_id = var.billing_account.id
prefix = var.prefix
})
"03-project-factory-prod" = jsonencode({
billing_account_id = var.billing_account.id
prefix = var.prefix
})
}
}
# optionally generate providers and tfvars files for subsequent stages
resource "local_file" "providers" {
for_each = var.outputs_location == null ? {} : local.providers
filename = "${var.outputs_location}/${each.key}/providers.tf"
content = each.value
}
resource "local_file" "tfvars" {
for_each = var.outputs_location == null ? {} : local.tfvars
filename = "${var.outputs_location}/${each.key}/terraform-bootstrap.auto.tfvars.json"
content = each.value
}
# outputs
output "billing_dataset" {
description = "BigQuery dataset prepared for billing export."
value = try(module.billing-export-dataset.0.id, null)
}
output "project_ids" {
description = "Projects created by this stage."
value = {
automation = module.automation-project.project_id
billing-export = try(module.billing-export-project.0.project_id, null)
log-export = module.log-export-project.project_id
}
}
# ready to use provider configurations for subsequent stages when not using files
output "providers" {
# tfdoc:output:consumers stage-01
description = "Terraform provider files for this stage and dependent stages."
sensitive = true
value = local.providers
}
# ready to use variable values for subsequent stages
output "tfvars" {
description = "Terraform variable files for the following stages."
sensitive = true
value = local.tfvars
}

100
fast/stages/00-bootstrap/variables.tf Normal file
View File

@ -0,0 +1,100 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "billing_account" {
description = "Billing account id and organization id ('nnnnnnnn' or null)."
type = object({
id = string
organization_id = number
})
}
variable "bootstrap_user" {
description = "Email of the nominal user running this stage for the first time."
type = string
default = null
}
variable "groups" {
# https://cloud.google.com/docs/enterprise/setup-checklist
description = "Group names to grant organization-level permissions."
type = map(string)
default = {
gcp-billing-admins = "gcp-billing-admins",
gcp-devops = "gcp-devops",
gcp-network-admins = "gcp-network-admins"
gcp-organization-admins = "gcp-organization-admins"
gcp-security-admins = "gcp-security-admins"
gcp-support = "gcp-support"
}
}
variable "iam" {
description = "Organization-level custom IAM settings in role => [principal] format."
type = map(list(string))
default = {}
}
variable "iam_additive" {
description = "Organization-level custom IAM settings in role => [principal] format for non-authoritative bindings."
type = map(list(string))
default = {}
}
variable "log_sinks" {
description = "Org-level log sinks, in name => {type, filter} format."
type = map(object({
filter = string
type = string
}))
default = {
audit-logs = {
filter = "logName:\"/logs/cloudaudit.googleapis.com%2Factivity\" OR logName:\"/logs/cloudaudit.googleapis.com%2Fsystem_event\""
type = "bigquery"
}
vpc-sc = {
filter = "protoPayload.metadata.@type=\"type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata\""
type = "bigquery"
}
}
validation {
condition = alltrue([
for k, v in var.log_sinks :
contains(["bigquery", "logging", "pubsub", "storage"], v.type)
])
error_message = "Type must be one of 'bigquery', 'logging', 'pubsub', 'storage'."
}
}
variable "organization" {
description = "Organization details."
type = object({
domain = string
id = number
customer_id = string
})
}
variable "outputs_location" {
description = "Path where providers and tfvars files for the following stages are written. Leave empty to disable."
type = string
default = null
}
variable "prefix" {
description = "Prefix used for resources that need unique names."
type = string
}

196
fast/stages/01-resman/README.md Normal file
View File

@ -0,0 +1,196 @@
# Resource hierarchy
This stage performs two important tasks:
- create the top-level hierarchy of folders, and the associated resources used later on to automate each part of the hierarchy (eg. Networking)
- set organization policies on the organization, and any exception required on specific folders
The code is intentionally simple, as it's intended to provide a generic initial setup (Networking, Security, etc.), and then allow easy customizations to complete the implementation of the intended hierarchy design.
The following diagram is a high level reference of the resources created and managed here:
<p align="center">
<img src="diagram.svg" alt="Resource-management diagram">
</p>
## Design overview and choices
Despite its simplicity, this stage implements the basics of a design that we've seen working well for a variety of customers, where the hierarchy is laid out following two conceptually different approaches:
- core or shared resources are grouped in hierarchy branches that map to their type or purpose (e.g. Networking)
- team or application resources are grouped in lower level hierarchy branches that map to management or operational considerations (e.g. which team manages a set of applications, or owns a subset of company data, etc.)
This split approach usually represents well functional and operational patterns, where core resources are centrally managed by individual teams (e.g. networking, security, fleets of similar VMS, etc.), while teams need more granularity to access managed services used by the applications they maintain.
The approach also adapts to different high level requirements:
- it can be used either for single organizations containing multiple environments, or with multiple organizations dedicated to specific environments (e.g. prod/nonprod), as the environment split is implemented at the project or lower folder level
- it adapts to complex scenarios, with different countries or corporate entities using the same GCP organization, as core services are typically shared, and/or an extra layer on top can be used as a drop-in to implement the country/entity separation
Additionally, a few critical benefits are directly provided by this design:
- core services are clearly separated, with very few touchpoints where IAM and security policies need to be applied (typically their top-level folder)
- adding a new set of core services (e.g. shared GKE clusters) is a trivial operation that does not break the existing design
- grouping application resources and services using teams or business logic is a flexible approach, which maps well to typical operational or budget requirements
- automation stages (e.g. Networking) can be segregated in a simple and effective way, by creating the required service accounts and buckets for each stage here, and applying a handful of IAM roles to the relevant folder
For a discussion on naming, please refer to the [Bootstrap stage documentation](../00-bootstrap/README.md#naming), as the same approach is shared by all stages.
## How to run this stage
This stage is meant to be executed after the [bootstrap](../00-bootstrap) stage has run, as it leverages the automation service account and bucket created there. The relevant user groups must also exist, but that's one of the requirements for the previous stage too, so if you ran that successfully, you're good to go.
It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the bootstrap stage for the actual roles needed.
Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration.
### Providers configuration
The default way of making sure you have the right permissions, is to use the identity of the service account pre-created for this stage during bootstrap, and that you are a member of the group that can impersonate it via provider-level configuration (`gcp-devops` or `organization-admins`).
To simplify setup, the previous stage pre-configures a valid providers file in its output, and optionally writes it to a local file if the `outputs_location` variable is set to a valid path.
If you have set a valid value for `outputs_location` in the bootstrap stage, simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
```bash
# `outputs_location` is set to `../../configs/example`
ln -s ../../configs/example/01-resman/providers.tf
```
If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage's outputs:
```bash
cd ../00-bootstrap
terraform output -json providers | jq -r '.["01-resman"]' \
> ../01-resman/providers.tf
```
### Variable configuration
There are two broad sets of variables you will need to fill in:
- variables shared by other stages (org id, billing account id, etc.), or derived from a resource managed by a different stage (folder id, automation project id, etc.)
- variables specific to resources managed by this stage
To avoid the tedious job of filling in the first group of variable with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files.
If you configured a valid path for `outputs_location` in the bootstrap stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's outputs folder (under the path you specified), where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is avalaible:
```bash
# `outputs_location` is set to `../../configs/example`
ln -s ../../configs/example/01-resman/terraform-bootstrap.auto.tfvars.json
```
A second set of variables is specific to this stage, they are all optional so if you need to customize them, create an extra `terraform.tfvars` file.
Refer to the [Variables](#variables) table at the bottom of this document, for a full list of variables, their origin (e.g. a stage or specific to this one), and descriptions explaining their meaning. The sections below also describe some of the possible customizations. For billing configurations, refer to the [Bootstrap documentation on billing](../00-bootstrap/README.md#billing-account) as the `billing_account` variable is identical across all stages.
Once done, you can run this stage:
```bash
terraform init
terraform apply
```
## Customizations
### Team folders
This stage provides a single built-in customization that offers a minimal (but usable) implementation of the "application" or "business" grouping for resources discussed above. The `team_folders` variable allows you to specify a map of team name and groups, that will result in folders, automation service accounts, and IAM policies applied.
Consider the following example
```hcl
team_folders = {
team-a = {
descriptive_name = "Team A"
group_iam = {
"team-a@gcp-pso-italy.net" = [
"roles/viewer"
]
}
impersonation_groups = ["team-a-admins@gcp-pso-italy.net"]
}
}
```
This will result in
- a "Team A" folder under the "Teams" folder
- one GCS bucket in the automation project
- one service account in the automation project with the correct IAM policies on the folder and bucket
- a IAM policy on the folder that assigns `roles/viewer` to the `team-a` group
- a IAM policy on the service account that allows `team-a` to impersonate it
This allows to centralize the minimum set of resources to delegate control of each team's folder to a pipeline, and/or to the team group. This can be used as a starting point for scenarios that implement more complex requirements (e.g. environment folders per team, etc.).
### Organization policies
Organization policies are laid out in an explicit manner in the `organization.tf` file, so it's fairly easy to add or remove specific policies.
For policies where additional data is needed, a root-level `organization_policy_configs` variable allows passing in specific data. Its built-in use to add additional organizations to the [Domain Restricted Sharing](https://cloud.google.com/resource-manager/docs/organization-policy/restricting-domains) policy, can be taken as an example on how to leverage it for additional customizations.
### IAM
IAM roles can be easily edited in the relevant `branch-xxx.tf` file, following the best practice outlined in the [bootstrap stage](../00-bootstrap#customizations) documentation of separating user-level and service-account level IAM policies in modules' `iam_groups`, `iam`, and `iam_additive` variables.
### Additional folders
Due to its simplicity, this stage lends itself easily to customizations: adding a new top-level branch (e.g. for shared GKE clusters) is as easy as cloning one of the `branch-xxx.tf` files, and changing names.
<!-- BEGIN TFDOC -->
## Files
| name | description | modules | resources |
|---|---|---|---|
| [billing.tf](./billing.tf) | Billing resources for external billing use cases. | <code>organization</code> | <code>google_billing_account_iam_member</code> |
| [branch-networking.tf](./branch-networking.tf) | Networking stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
| [branch-sandbox.tf](./branch-sandbox.tf) | Sandbox stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
| [branch-security.tf](./branch-security.tf) | Security stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
| [branch-teams.tf](./branch-teams.tf) | Team stages resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
| [main.tf](./main.tf) | Module-level locals and resources. | | |
| [organization.tf](./organization.tf) | Organization policies. | <code>organization</code> | |
| [outputs.tf](./outputs.tf) | Module outputs. | | <code>local_file</code> |
| [variables.tf](./variables.tf) | Module variables. | | |
## Variables
| name | description | type | required | default | producer |
|---|---|:---:|:---:|:---:|:---:|
| automation_project_id | Project id for the automation project created by the bootstrap stage. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
| billing_account | Billing account id and organization id ('nnnnnnnn' or null). | <code title="object&#40;&#123;&#10; id &#61; string&#10; organization_id &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| organization | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; number&#10; customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| prefix | Prefix used for resources that need unique names. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
| custom_roles | Custom roles defined at the org level, in key => id format. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>00-bootstrap</code> |
| groups | Group names to grant organization-level permissions. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; gcp-billing-admins &#61; &#34;gcp-billing-admins&#34;,&#10; gcp-devops &#61; &#34;gcp-devops&#34;,&#10; gcp-network-admins &#61; &#34;gcp-network-admins&#34;&#10; gcp-organization-admins &#61; &#34;gcp-organization-admins&#34;&#10; gcp-security-admins &#61; &#34;gcp-security-admins&#34;&#10; gcp-support &#61; &#34;gcp-support&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>00-bootstrap</code> |
| organization_policy_configs | Organization policies customization. | <code title="object&#40;&#123;&#10; allowed_policy_member_domains &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| outputs_location | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | <code>string</code> | | <code>null</code> | |
| team_folders | Team folders to be created. Format is described in a code comment. | <code title="map&#40;object&#40;&#123;&#10; descriptive_name &#61; string&#10; group_iam &#61; map&#40;list&#40;string&#41;&#41;&#10; impersonation_groups &#61; list&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>null</code> | |
## Outputs
| name | description | sensitive | consumers |
|---|---|:---:|---|
| networking | Data for the networking stage. | | <code>02-networking</code> |
| project_factories | Data for the project factories stage. | | <code>xx-teams</code> |
| providers | Terraform provider files for this stage and dependent stages. | ✓ | <code>02-networking</code> · <code>02-security</code> · <code>xx-sandbox</code> · <code>xx-teams</code> |
| sandbox | Data for the sandbox stage. | | <code>xx-sandbox</code> |
| security | Data for the networking stage. | | <code>02-security</code> |
| teams | Data for the teams stage. | | |
| tfvars | Terraform variable files for the following stages. | ✓ | |
<!-- END TFDOC -->

56
fast/stages/01-resman/billing.tf Normal file
View File

@ -0,0 +1,56 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Billing resources for external billing use cases.
locals {
# used here for convenience, in organization.tf members are explicit
billing_ext_users = concat(
[
module.branch-network-sa.iam_email,
module.branch-security-sa.iam_email,
],
# enable if individual teams can create their own projects
# [
# for k, v in module.branch-teams-team-sa : v.iam_email
# ],
local.branch_teams_pf_sa_iam_emails
)
}
# billing account in same org (resources is in the organization.tf file)
# billing account in a different org
module "billing-organization-ext" {
source = "../../../modules/organization"
count = local.billing_org_ext ? 1 : 0
organization_id = "organizations/${var.billing_account.organization_id}"
iam_additive = {
"roles/billing.user" = local.billing_ext_users
}
}
# standalone billing account
resource "google_billing_account_iam_member" "billing_ext_admin" {
for_each = toset(
local.billing_ext ? local.billing_ext_users : []
)
billing_account_id = var.billing_account.id
role = "roles/billing.user"
member = each.key
}

59
fast/stages/01-resman/branch-networking.tf Normal file
View File

@ -0,0 +1,59 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Networking stage resources.
module "branch-network-folder" {
source = "../../../modules/folder"
parent = "organizations/${var.organization.id}"
name = "Networking"
group_iam = {
(local.groups.gcp-network-admins) = [
# add any needed roles for resources/services not managed via Terraform,
# or replace editor with ~viewer if no broad resource management needed
# e.g.
# "roles/compute.networkAdmin",
# "roles/dns.admin",
# "roles/compute.securityAdmin",
"roles/editor",
]
}
iam = {
"roles/logging.admin" = [module.branch-network-sa.iam_email]
"roles/owner" = [module.branch-network-sa.iam_email]
"roles/resourcemanager.folderAdmin" = [module.branch-network-sa.iam_email]
"roles/resourcemanager.projectCreator" = [module.branch-network-sa.iam_email]
}
}
module "branch-network-sa" {
source = "../../../modules/iam-service-account"
project_id = var.automation_project_id
name = "resman-networking-0"
description = "Terraform resman networking service account."
prefix = local.prefixes.prod
}
module "branch-network-gcs" {
source = "../../../modules/gcs"
project_id = var.automation_project_id
name = "resman-networking-0"
prefix = local.prefixes.prod
versioning = true
iam = {
"roles/storage.objectAdmin" = [module.branch-network-sa.iam_email]
}
}

59
fast/stages/01-resman/branch-sandbox.tf Normal file
View File

@ -0,0 +1,59 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Sandbox stage resources.
module "branch-sandbox-folder" {
source = "../../../modules/folder"
parent = "organizations/${var.organization.id}"
name = "Sandbox"
iam = {
"roles/logging.admin" = [module.branch-sandbox-sa.iam_email]
"roles/owner" = [module.branch-sandbox-sa.iam_email]
"roles/resourcemanager.folderAdmin" = [module.branch-sandbox-sa.iam_email]
"roles/resourcemanager.projectCreator" = [module.branch-sandbox-sa.iam_email]
}
policy_boolean = {
"constraints/sql.restrictPublicIp" = false
}
policy_list = {
"constraints/compute.vmExternalIpAccess" = {
inherit_from_parent = false
suggested_value = null
status = true
values = []
}
}
}
module "branch-sandbox-gcs" {
source = "../../../modules/gcs"
project_id = var.automation_project_id
name = "resman-sandbox-0"
prefix = local.prefixes.dev
versioning = true
iam = {
"roles/storage.objectAdmin" = [module.branch-sandbox-sa.iam_email]
}
}
module "branch-sandbox-sa" {
source = "../../../modules/iam-service-account"
project_id = var.automation_project_id
name = "resman-sandbox-0"
description = "Terraform resman sandbox service account."
prefix = local.prefixes.dev
}

61
fast/stages/01-resman/branch-security.tf Normal file
View File

@ -0,0 +1,61 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Security stage resources.
module "branch-security-folder" {
source = "../../../modules/folder"
parent = "organizations/${var.organization.id}"
name = "Security"
group_iam = {
(local.groups.gcp-security-admins) = [
# add any needed roles for resources/services not managed via Terraform,
# e.g.
# "roles/bigquery.admin",
# "roles/cloudasset.owner",
# "roles/cloudkms.admin",
# "roles/logging.admin",
# "roles/secretmanager.admin",
# "roles/storage.admin",
"roles/viewer"
]
}
iam = {
"roles/logging.admin" = [module.branch-security-sa.iam_email]
"roles/owner" = [module.branch-security-sa.iam_email]
"roles/resourcemanager.folderAdmin" = [module.branch-security-sa.iam_email]
"roles/resourcemanager.projectCreator" = [module.branch-security-sa.iam_email]
}
}
module "branch-security-sa" {
source = "../../../modules/iam-service-account"
project_id = var.automation_project_id
name = "resman-security-0"
description = "Terraform resman security service account."
prefix = local.prefixes.prod
}
module "branch-security-gcs" {
source = "../../../modules/gcs"
project_id = var.automation_project_id
name = "resman-security-0"
prefix = local.prefixes.prod
versioning = true
iam = {
"roles/storage.objectAdmin" = [module.branch-security-sa.iam_email]
}
}

165
fast/stages/01-resman/branch-teams.tf Normal file
View File

@ -0,0 +1,165 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Team stages resources.
# top-level teams folder and service account
module "branch-teams-folder" {
source = "../../../modules/folder"
parent = "organizations/${var.organization.id}"
name = "Teams"
}
module "branch-teams-prod-sa" {
source = "../../../modules/iam-service-account"
project_id = var.automation_project_id
name = "resman-teams-0"
description = "Terraform resman production service account."
prefix = local.prefixes.prod
}
# Team-level folders, service accounts and buckets for each individual team
module "branch-teams-team-folder" {
source = "../../../modules/folder"
for_each = coalesce(var.team_folders, {})
parent = module.branch-teams-folder.id
name = each.value.descriptive_name
group_iam = each.value.group_iam == null ? {} : each.value.group_iam
}
module "branch-teams-team-sa" {
source = "../../../modules/iam-service-account"
for_each = coalesce(var.team_folders, {})
project_id = var.automation_project_id
name = "teams-${each.key}-0"
description = "Terraform team ${each.key} service account."
prefix = local.prefixes.prod
iam = {
"roles/iam.serviceAccountTokenCreator" = (
each.value.impersonation_groups == null
? []
: [for g in each.value.impersonation_groups : "group:${g}"]
)
}
}
module "branch-teams-team-gcs" {
source = "../../../modules/gcs"
for_each = coalesce(var.team_folders, {})
project_id = var.automation_project_id
name = "teams-${each.key}-0"
prefix = local.prefixes.prod
versioning = true
iam = {
"roles/storage.objectAdmin" = [module.branch-teams-team-sa[each.key].iam_email]
}
}
# environment: development folder and project factory automation resources
module "branch-teams-team-dev-folder" {
source = "../../../modules/folder"
for_each = coalesce(var.team_folders, {})
parent = module.branch-teams-team-folder[each.key].id
# naming: environment descriptive name
name = "${module.branch-teams-team-folder[each.key].name} - Development"
# environment-wide human permissions on the whole teams environment
group_iam = {}
iam = {
# remove owner here and at project level if SA does not manage project resources
"roles/owner" = [
module.branch-teams-dev-projectfactory-sa.iam_email
]
"roles/logging.admin" = [
module.branch-teams-dev-projectfactory-sa.iam_email
]
"roles/resourcemanager.folderAdmin" = [
module.branch-teams-dev-projectfactory-sa.iam_email
]
"roles/resourcemanager.projectCreator" = [
module.branch-teams-dev-projectfactory-sa.iam_email
]
}
}
module "branch-teams-dev-projectfactory-sa" {
source = "../../../modules/iam-service-account"
project_id = var.automation_project_id
name = "resman-pf-0"
# naming: environment in description
description = "Terraform project factory development service account."
prefix = local.prefixes.dev
}
module "branch-teams-dev-projectfactory-gcs" {
source = "../../../modules/gcs"
project_id = var.automation_project_id
name = "resman-pf-0"
prefix = local.prefixes.dev
versioning = true
iam = {
"roles/storage.objectAdmin" = [module.branch-teams-dev-projectfactory-sa.iam_email]
}
}
# environment: production folder and project factory automation resources
module "branch-teams-team-prod-folder" {
source = "../../../modules/folder"
for_each = coalesce(var.team_folders, {})
parent = module.branch-teams-team-folder[each.key].id
# naming: environment descriptive name
name = "${module.branch-teams-team-folder[each.key].name} - Production"
# environment-wide human permissions on the whole teams environment
group_iam = {}
iam = {
# remove owner here and at project level if SA does not manage project resources
"roles/owner" = [
module.branch-teams-prod-projectfactory-sa.iam_email
]
"roles/logging.admin" = [
module.branch-teams-prod-projectfactory-sa.iam_email
]
"roles/resourcemanager.folderAdmin" = [
module.branch-teams-prod-projectfactory-sa.iam_email
]
"roles/resourcemanager.projectCreator" = [
module.branch-teams-prod-projectfactory-sa.iam_email
]
}
}
module "branch-teams-prod-projectfactory-sa" {
source = "../../../modules/iam-service-account"
project_id = var.automation_project_id
name = "resman-pf-0"
# naming: environment in description
description = "Terraform project factory production service account."
prefix = local.prefixes.prod
}
module "branch-teams-prod-projectfactory-gcs" {
source = "../../../modules/gcs"
project_id = var.automation_project_id
name = "resman-pf-0"
prefix = local.prefixes.prod
versioning = true
iam = {
"roles/storage.objectAdmin" = [module.branch-teams-prod-projectfactory-sa.iam_email]
}
}

BIN
fast/stages/01-resman/diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 228 KiB

1340
fast/stages/01-resman/diagram.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 435 KiB

35
fast/stages/01-resman/main.tf Normal file
View File

@ -0,0 +1,35 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
# convenience flags that express where billing account resides
billing_ext = var.billing_account.organization_id == null
billing_org = var.billing_account.organization_id == var.organization.id
billing_org_ext = !local.billing_ext && !local.billing_org
groups = {
for k, v in var.groups :
k => "${v}@${var.organization.domain}"
}
groups_iam = {
for k, v in local.groups :
k => "group:${v}"
}
# naming: environment names
prefixes = {
dev = "${var.prefix}-dev"
prod = "${var.prefix}-prod"
}
}

136
fast/stages/01-resman/organization.tf Normal file
View File

@ -0,0 +1,136 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Organization policies.
locals {
# set to the empty list if you remove the teams branch
branch_teams_pf_sa_iam_emails = [
module.branch-teams-dev-projectfactory-sa.iam_email,
module.branch-teams-prod-projectfactory-sa.iam_email
]
list_deny = {
inherit_from_parent = false
suggested_value = null
status = false
values = []
}
policy_configs = (
var.organization_policy_configs == null
? {}
: var.organization_policy_configs
)
}
module "organization" {
source = "../../../modules/organization"
organization_id = "organizations/${var.organization.id}"
# IAM additive bindings, granted via the restricted Organization Admin custom
# role assigned in stage 00; they need to be additive to avoid conflicts
iam_additive = merge(
{
(var.custom_roles.xpnServiceAdmin) = concat(
local.branch_teams_pf_sa_iam_emails
)
"roles/accesscontextmanager.policyAdmin" = [
module.branch-security-sa.iam_email
]
"roles/billing.costsManager" = concat(
local.branch_teams_pf_sa_iam_emails
),
"roles/compute.orgFirewallPolicyAdmin" = [
module.branch-network-sa.iam_email
]
"roles/compute.xpnAdmin" = [
module.branch-network-sa.iam_email
]
"roles/orgpolicy.policyAdmin" = local.branch_teams_pf_sa_iam_emails
},
local.billing_org ? {
"roles/billing.user" = concat(
[
module.branch-network-sa.iam_email,
module.branch-security-sa.iam_email,
],
# enable if individual teams can create their own projects
# [
# for k, v in module.branch-teams-team-sa : v.iam_email
# ],
local.branch_teams_pf_sa_iam_emails
)
} : {}
)
# sample subset of useful organization policies, edit to suit requirements
policy_boolean = {
"constraints/cloudfunctions.requireVPCConnector" = true
"constraints/compute.disableGuestAttributesAccess" = true
"constraints/compute.disableInternetNetworkEndpointGroup" = true
"constraints/compute.disableNestedVirtualization" = true
"constraints/compute.disableSerialPortAccess" = true
"constraints/compute.requireOsLogin" = true
"constraints/compute.restrictXpnProjectLienRemoval" = true
"constraints/compute.skipDefaultNetworkCreation" = true
"constraints/iam.automaticIamGrantsForDefaultServiceAccounts" = true
"constraints/iam.disableServiceAccountKeyCreation" = true
"constraints/iam.disableServiceAccountKeyUpload" = true
"constraints/sql.restrictPublicIp" = true
"constraints/sql.restrictAuthorizedNetworks" = true
"constraints/storage.uniformBucketLevelAccess" = true
}
policy_list = {
"constraints/cloudfunctions.allowedIngressSettings" = merge(
local.list_deny, { values = ["ALLOW_INTERNAL_ONLY"] }
)
"constraints/cloudfunctions.allowedVpcConnectorEgressSettings" = merge(
local.list_deny, { values = ["PRIVATE_RANGES_ONLY"] }
)
"constraints/compute.restrictLoadBalancerCreationForTypes" = merge(
local.list_deny, { values = ["in:INTERNAL"] }
)
"constraints/compute.vmExternalIpAccess" = local.list_deny
"constraints/iam.allowedPolicyMemberDomains" = {
inherit_from_parent = false
suggested_value = null
status = true
values = concat(
[var.organization.customer_id],
try(local.policy_configs.allowed_policy_member_domains, [])
)
}
"constraints/run.allowedIngress" = merge(
local.list_deny, { values = ["internal"] }
)
"constraints/run.allowedVPCEgress" = merge(
local.list_deny, { values = ["private-ranges-only"] }
)
# "constraints/compute.restrictCloudNATUsage" = local.list_deny
# "constraints/compute.restrictDedicatedInterconnectUsage" = local.list_deny
# "constraints/compute.restrictPartnerInterconnectUsage" = local.list_deny
# "constraints/compute.restrictProtocolForwardingCreationForTypes" = local.list_deny
# "constraints/compute.restrictSharedVpcHostProjects" = local.list_deny
# "constraints/compute.restrictSharedVpcSubnetworks" = local.list_deny
# "constraints/compute.restrictVpcPeering" = local.list_deny
# "constraints/compute.restrictVpnPeerIPs" = local.list_deny
# "constraints/compute.vmCanIpForward" = local.list_deny
# "constraints/gcp.resourceLocations" = {
# inherit_from_parent = false
# suggested_value = null
# status = true
# values = local.allowed_regions
# }
}
}

150
fast/stages/01-resman/outputs.tf Normal file
View File

@ -0,0 +1,150 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
_project_factory_sas = {
dev = module.branch-teams-dev-projectfactory-sa.iam_email
prod = module.branch-teams-prod-projectfactory-sa.iam_email
}
providers = {
"02-networking" = templatefile("${path.module}/../../assets/templates/providers.tpl", {
bucket = module.branch-network-gcs.name
name = "networking"
sa = module.branch-network-sa.email
})
"02-security" = templatefile("${path.module}/../../assets/templates/providers.tpl", {
bucket = module.branch-security-gcs.name
name = "security"
sa = module.branch-security-sa.email
})
"99-sandbox" = templatefile("${path.module}/../../assets/templates/providers.tpl", {
bucket = module.branch-sandbox-gcs.name
name = "sandbox"
sa = module.branch-sandbox-sa.email
})
"03-project-factory-dev" = templatefile("${path.module}/../../assets/templates/providers.tpl", {
bucket = module.branch-teams-dev-projectfactory-gcs.name
name = "team-dev"
sa = module.branch-teams-dev-projectfactory-sa.email
})
"03-project-factory-prod" = templatefile("${path.module}/../../assets/templates/providers.tpl", {
bucket = module.branch-teams-prod-projectfactory-gcs.name
name = "team-prod"
sa = module.branch-teams-prod-projectfactory-sa.email
})
}
tfvars = {
"02-networking" = jsonencode({
folder_id = module.branch-network-folder.id
project_factory_sa = local._project_factory_sas
})
"02-security" = jsonencode({
folder_id = module.branch-security-folder.id
kms_restricted_admins = {
for k, v in local._project_factory_sas : k => [v]
}
})
}
}
# optionally generate providers and tfvars files for subsequent stages
resource "local_file" "providers" {
for_each = var.outputs_location == null ? {} : local.providers
filename = "${var.outputs_location}/${each.key}/providers.tf"
content = each.value
}
resource "local_file" "tfvars" {
for_each = var.outputs_location == null ? {} : local.tfvars
filename = "${var.outputs_location}/${each.key}/terraform-resman.auto.tfvars.json"
content = each.value
}
# outputs
output "networking" {
# tfdoc:output:consumers 02-networking
description = "Data for the networking stage."
value = {
folder = module.branch-network-folder.id
gcs_bucket = module.branch-network-gcs.name
service_account = module.branch-network-sa.iam_email
}
}
output "project_factories" {
# tfdoc:output:consumers xx-teams
description = "Data for the project factories stage."
value = {
dev = {
bucket = module.branch-teams-dev-projectfactory-gcs.name
sa = module.branch-teams-dev-projectfactory-sa.email
}
prod = {
bucket = module.branch-teams-prod-projectfactory-gcs.name
sa = module.branch-teams-prod-projectfactory-sa.email
}
}
}
# ready to use provider configurations for subsequent stages
output "providers" {
# tfdoc:output:consumers 02-networking 02-security xx-sandbox xx-teams
description = "Terraform provider files for this stage and dependent stages."
sensitive = true
value = local.providers
}
output "sandbox" {
# tfdoc:output:consumers xx-sandbox
description = "Data for the sandbox stage."
value = {
folder = module.branch-sandbox-folder.id
gcs_bucket = module.branch-sandbox-gcs.name
service_account = module.branch-sandbox-sa.email
}
}
output "security" {
# tfdoc:output:consumers 02-security
description = "Data for the networking stage."
value = {
folder = module.branch-security-folder.id
gcs_bucket = module.branch-security-gcs.name
service_account = module.branch-security-sa.iam_email
}
}
output "teams" {
description = "Data for the teams stage."
value = {
for k, v in module.branch-teams-team-folder : k => {
folder = v.id
gcs_bucket = module.branch-teams-team-gcs[k].name
service_account = module.branch-teams-team-sa[k].email
}
}
}
# ready to use variable values for subsequent stages
output "tfvars" {
description = "Terraform variable files for the following stages."
sensitive = true
value = local.tfvars
}

104
fast/stages/01-resman/variables.tf Normal file
View File

@ -0,0 +1,104 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# defaults for variables marked with global tfdoc annotations, can be set via
# the tfvars file generated in stage 00 and stored in its outputs
variable "billing_account" {
# tfdoc:variable:source 00-bootstrap
description = "Billing account id and organization id ('nnnnnnnn' or null)."
type = object({
id = string
organization_id = number
})
}
variable "automation_project_id" {
# tfdoc:variable:source 00-bootstrap
description = "Project id for the automation project created by the bootstrap stage."
type = string
}
variable "custom_roles" {
# tfdoc:variable:source 00-bootstrap
description = "Custom roles defined at the org level, in key => id format."
type = map(string)
default = {}
}
variable "groups" {
# tfdoc:variable:source 00-bootstrap
description = "Group names to grant organization-level permissions."
type = map(string)
# https://cloud.google.com/docs/enterprise/setup-checklist
default = {
gcp-billing-admins = "gcp-billing-admins",
gcp-devops = "gcp-devops",
gcp-network-admins = "gcp-network-admins"
gcp-organization-admins = "gcp-organization-admins"
gcp-security-admins = "gcp-security-admins"
gcp-support = "gcp-support"
}
}
variable "organization" {
# tfdoc:variable:source 00-bootstrap
description = "Organization details."
type = object({
domain = string
id = number
customer_id = string
})
}
variable "organization_policy_configs" {
description = "Organization policies customization."
type = object({
allowed_policy_member_domains = list(string)
})
default = null
}
variable "outputs_location" {
description = "Path where providers and tfvars files for the following stages are written. Leave empty to disable."
type = string
default = null
}
variable "prefix" {
# tfdoc:variable:source 00-bootstrap
description = "Prefix used for resources that need unique names."
type = string
}
variable "team_folders" {
description = "Team folders to be created. Format is described in a code comment."
type = map(object({
descriptive_name = string
group_iam = map(list(string))
impersonation_groups = list(string)
}))
default = null
# default = {
# team-a = {
# descriptive_name = "Team A"
# group_iam = {
# team-a-group = [roles/owner, roles/projectCreator]
# }
# impersonation_groups = ["team-a-admins@example.com"]
# }
# }
}

338
fast/stages/02-networking/README.md Normal file
View File

@ -0,0 +1,338 @@
# Networking
This stage sets up the shared network infrastructure for the whole organization. It adopts the common “hub and spoke” reference design, which is well suited to multiple scenarios, and offers several advantages versus other designs:
- the “hub” VPC centralizes external connectivity to on-prem or other cloud environments, and is ready to host cross-environment services like CI/CD, code repositories, and monitoring probes
- the “spoke” VPCs allow partitioning workloads (e.g. by environment like in this setup), while still retaining controlled access to central connectivity and services
- Shared VPC in both hub and spokes splits management of network resources in specific (host) projects, while still allowing them to be consumed from workload (service) projects
- the design also lends itself to easy DNS centralization, both from on-prem to cloud and from cloud to on-prem
Connectivity between hub and spokes is established here via [VPN HA](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) tunnels, which offer easy interoperability with some key GCP features (GKE, services leveraging Service Networking like Cloud SQL, etc.), allowing clear partitioning of quota and limits between environments, and fine-grained control of routing. Different ways of implementing connectivity, and their respective pros and cons, are discussed below.
The following diagram illustrates the high-level design, and should be used as a reference for the following sections. The final number of subnets, and their IP addressing design will of course depend on customer-specific requirements, and can be easily changed via variables or external data files without having to edit the actual code.
<p align="center">
<img src="diagram.svg" alt="Networking diagram">
</p>
## Design overview and choices
### VPC design
The hub/landing VPC hosts external connectivity and shared services for spoke VPCs, which are connected to it via VPN HA tunnels. Spokes are used here to partition environments, which is a fairly common pattern:
- one spoke VPC for the production environment
- one spoke VPC for the development environment
Each VPC is created into its own project, and each project is configured as a Shared VPC host, so that network-related resources and access configurations via IAM are kept separate for each VPC.
The design easily lends itself to implementing additional environments, or adopting a different logical mapping for spokes (e.g. one spoke for each company entity, etc.). Adding spokes is a trivial operation, does not increase the design complexity, and is explained in operational terms in the following sections.
In multi-organization scenarios, where production and non-production resources use different Cloud Identity and GCP organizations, the hub/landing VPC is usually part of the production organization, and establishes connections with production spokes in its same organization, and non-production spokes in a different organization.
An additional VPC is also deployed by default with the provided code, disconnected from the other VPCs and hosting a single VM that emulates on-prem for testing purposes, via a Docker network and containers for VPN, DNS, HTTP, etc.
### External connectivity
External connectivity to on-prem is implemented here via VPN HA (two tunnels per region), as this is the minimum common denominator often used directly, or as a stop-gap solution to validate routing and transfer data, while waiting for [interconnects](https://cloud.google.com/network-connectivity/docs/interconnect) to be provisioned.
Connectivity to additional on-prem sites or other cloud providers should be implemented in a similar fashion, via VPN tunnels or interconnects in the landing VPC sharing the same regional router.
### Internal connectivity
As mentioned initially, there are of course other ways to implement internal connectivity other than VPN HA. These can be easily retrofitted with minimal code changes, but introduce additional considerations for service interoperability, quotas and management.
This is a summary of the main options:
- [VPN HA](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) (implemented here)
- Pros: simple compatibility with GCP services that leverage peering internally, better control on routes, avoids peering groups shared quotas and limits
- Cons: additional cost, marginal increase in latency, requires multiple tunnels for full bandwidth
- [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering)
- Pros: no additional costs, full bandwidth with no configurations, no extra latency
- Cons: no transitivity (e.g. to GKE masters, Cloud SQL, etc.), no selective exchange of routes, several quotas and limits shared between VPCs in a peering group
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic)
- Pros: additional security features (e.g. IPS), potentially better integration with on-prem systems by using the same vendor
- Cons: complex HA/failover setup, limited by VM bandwidth and scale, additional costs for VMs and licenses, out of band management of a critical cloud component
### IP ranges, subnetting, routing
Minimizing the number of routes (and subnets) in use on the cloud environment is an important consideration, as it simplifies management and avoids hitting [Cloud Router](https://cloud.google.com/network-connectivity/docs/router/quotas) and [VPC](https://cloud.google.com/vpc/docs/quota) quotas and limits. For this reason, we recommend careful planning of the IP space used in your cloud environment, to be able to use large IP CIDR blocks in routes whenever possible.
This stage uses a dedicated /16 block (which should of course be sized to your needs) for each region in each VPC, and subnets created in each VPC derive their ranges from the relevant block.
Spoke VPCs also define and reserve two "special" CIDR ranges dedicated to [PSA (Private Service Access)](https://cloud.google.com/vpc/docs/private-services-access) and [Internal HTTPs Load Balancers (L7ILB)](https://cloud.google.com/load-balancing/docs/l7-internal).
Routes in GCP are either automatically created for VPC subnets, manually created via static routes, or dynamically programmed by [Cloud Routers](https://cloud.google.com/network-connectivity/docs/router#docs) via BGP sessions, which can be configured to advertise VPC ranges, and/or custom ranges via custom advertisements.
In this setup, the Cloud Routers are configured so as to exclude the default advertisement of VPC ranges, and they only advertise their respective aggregate ranges via custom advertisements. This greatly simplifies the routing configuration, and more importantly it allows to avoid quota or limit issues by keeping the number of routes small, instead of making it proportional to the subnets and secondary ranges in the VPCs.
The high-level routing plan implemented in this architecture is as follows:
| source | target | advertisement |
| ----------- | ----------- | ------------------------------ |
| VPC landing | onprem | GCP aggregate |
| VPC landing | onprem | Cloud DNS forwarders |
| VPC landing | onprem | Google private/restricted APIs |
| VPC landing | spokes | RFC1918 |
| VPC spoke | VPC landing | spoke aggregate |
| onprem | VC landing | onprem aggregates |
As is evident from the table above, the hub/landing VPC acts as the route concentrator for the whole GCP network, implementing a full line of sight between environments, and between GCP and on-prem. While advertisements can be adjusted to selectively exchange routes (e.g. to isolate the production and the development environment), we recommend using [Firewall](#firewall) policies or rules to achieve the desired isolation.
### Internet egress
The path of least resistance for Internet egress is using Cloud NAT, and that is what's implemented in this setup, with a NAT gateway configured for each VPC.
Several other scenarios are possible of course, with varying degrees of complexity:
- a forward proxy, with optional URL filters
- a default route to on-prem to leverage existing egress infrastructure
- a full-fledged perimeter firewall to control egress and implement additional security features like IPS
Future pluggable modules will allow to easily experiment, or deploy the above scenarios.
### VPC and Hierarchical Firewall
The GCP Firewall is a stateful, distributed feature that allows the creation of L4 policies, either via VPC-level rules or more recently via hierarchical policies applied on the resource hierarchy (organization, folders).
The current setup adopts both firewall types, and uses hierarchical rules on the Networking folder for common ingress rules (egress is open by default), e.g. from health check or IAP forwarders ranges, and VPC rules for the environment or workload-level ingress.
Rules and policies are defined in simple YAML files, described below.
### DNS
DNS often goes hand in hand with networking, especially on GCP where Cloud DNS zones and policies are associated at the VPC level. This setup implements both DNS flows:
- on-prem to cloud via private zones for cloud-managed domains, and an [inbound policy](https://cloud.google.com/dns/docs/server-policies-overview#dns-server-policy-in) used as forwarding target or via delegation (requires some extra configuration) from on-prem DNS resolvers
- cloud to on-prem via forwarding zones for the on-prem managed domains
DNS configuration is further centralized by leveraging peering zones, so that
- the hub/landing Cloud DNS hosts configurations for on-prem forwarding and Google API domains, with the spokes consuming them via DNS peering zones
- the spokes Cloud DNS host configurations for the environment-specific domains, with the hub/landing VPC acting as consumer via DNS peering
To complete the configuration, the 35.199.192.0/19 range should be routed on the VPN tunnels from on-prem, and the following names configured for DNS forwarding to cloud:
- `private.googleapis.com`
- `restricted.googleapis.com`
- `gcp.example.com` (used as a placeholder)
From cloud, the `example.com` domain (used as a placeholder) is forwarded to on-prem.
This configuration is battle-tested, and flexible enough to lend itself to simple modifications without subverting its design, for example by forwarding and peering root zones to bypass Cloud DNS external resolution.
## How to run this stage
This stage is meant to be executed after the [resman](../01-resman) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured in the [bootstrap](../00-boostrap) stage.
It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the previous stages for the environmental requirements.
Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration.
### Providers configuration
The default way of making sure you have the right permissions, is to use the identity of the service account pre-created for this stage during the [resource management](./01-resman) stage, and that you are a member of the group that can impersonate it via provider-level configuration (`gcp-devops` or `organization-admins`).
To simplify setup, the previous stage pre-configures a valid providers file in its output, and optionally writes it to a local file if the `outputs_location` variable is set to a valid path.
If you have set a valid value for `outputs_location` in the bootstrap stage, simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
```bash
# `outputs_location` is set to `../../configs/example`
ln -s ../../configs/example/02-networking/providers.tf
```
If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage's outputs:
```bash
cd ../00-bootstrap
terraform output -json providers | jq -r '.["02-networking"]' \
> ../02-networking/providers.tf
```
### Variable configuration
There are two broad sets of variables you will need to fill in:
- variables shared by other stages (org id, billing account id, etc.), or derived from a resource managed by a different stage (folder id, automation project id, etc.)
- variables specific to resources managed by this stage
To avoid the tedious job of filling in the first group of variables with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files.
If you have set a valid value for `outputs_location` in the bootstrap and in the resman stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's folder in the path you specified, where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is available:
```bash
# `outputs_location` is set to `../../configs/example`
ln -s ../../configs/example/02-networking/terraform-bootstrap.auto.tfvars.json
ln -s ../../configs/example/02-networking/terraform-resman.auto.tfvars.json
```
Please refer to the [Variables](#variables) table below for a map of the variable origins, and to the sections below on how to adapt this stage to your networking configuration.
### VPCs
VPCs are defined in separate files, one for `landing` and one for each of `prod` and `dev`.
Each file contains the same resources, described in the following paragraphs.
The **project** ([`project`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/project)) contains the VPC, and enables the required APIs and sets itself as a "[host project](https://cloud.google.com/vpc/docs/shared-vpc)".
The **VPC** ([`net-vpc`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc)) manages the DNS inbound policy (for Landing), explicit routes for `{private,restricted}.googleapis.com`, and its **subnets**. Subnets are created leveraging a "resource factory" paradigm, where the configuration is separated from the module that implements it, and stored in a well-structured file. To add a new subnet, simply create a new file in the `data_folder` directory defined in the module, following the examples found in the [Fabric `net-vpc` documentation](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc#subnet-factory). Sample subnets are shipped in [data/subnets](./data/subnets), and can be easily customised to fit your needs.
Subnets for [L7 ILBs](https://cloud.google.com/load-balancing/docs/l7-internal/proxy-only-subnets) are handled differently, and defined in variable `l7ilb_subnets`, while ranges for [PSA](https://cloud.google.com/vpc/docs/configure-private-services-access#allocating-range) are configured by variable `psa_ranges` - such variables are consumed by spoke VPCs.
**Cloud NAT** ([`net-cloudnat`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-cloudnat)) manages the networking infrastructure required to enable internet egress.
### VPNs
#### External
Connectivity to on-prem is implemented with VPN HA ([`net-vpn`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpn)) and defined in [`vpn-onprem.tf`](./vpn-onprem.tf). The file provisionally implements a single logical connection between onprem and landing at `europe-west1`, and the relevant parameters for its configuration are found in variable `vpn_onprem_configs`.
#### Internal
VPNs ([`net-vpn`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpn)) used to interconnect landing and spokes are managed by `vpn-spoke-*.tf` files, each implementing both sides of the VPN connection. Per-gateway configurations (e.g. BGP advertisements and session ranges) are controlled by variable `vpn_onprem_configs`. VPN gateways and IKE secrets are automatically generated and configured.
### Routing and BGP
Each VPC network ([`net-vpc`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc)) manages a separate routing table, which can define static routes (e.g. to private.googleapis.com) and receives dynamic routes from BGP sessions established with neighbor networks (e.g. landing receives routes from onprem and spokes, and spokes receive RFC1918 from landing).
Static routes are defined in `vpc-*.tf` files, in the `routes` section of each `net-vpc` module.
BGP sessions for landing-spoke are configured through variable `vpn_spoke_configs`, while the ones for landing-onprem use variable `vpn_onprem_configs`
### Firewall
**VPC firewall rules** ([`net-vpc-firewall`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc-firewall)) are defined per-vpc on each `vpc-*.tf` file and leverage a resource factory to massively create rules.
To add a new firewall rule, create a new file or edit an existing one in the `data_folder` directory defined in the module `net-vpc-firewall`, following the examples of the "[Rules factory](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc-firewall#rules-factory)" section of the module documentation. Sample firewall rules are shipped in [data/firewall-rules/landing](./data/firewall-rules/landing) and can be easily customised.
**Hierarchical firewall policies** ([`folder`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/folder)) are defined in `main.tf`, and managed through a policy factory implemented by the `folder` module, which applies the defined hierarchical to the `Networking` folder, which contains all the core networking infrastructure. Policies are defined in the `rules_file` file - to define a new one simply use the instructions found on "[Firewall policy factory](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/organization#firewall-policy-factory)". Sample hierarchical firewall policies are shipped in [data/hierarchical-policy-rules.yaml](./data/hierarchical-policy-rules.yaml) and can be easily customised.
### DNS architecture
The DNS ([`dns`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/dns)) infrastructure is defined in [`dns.tf`](dns.tf).
Cloud DNS manages onprem forwarding, the main GCP zone (in this example `gcp.example.com`) and is peered to environment-specific zones (i.e. `dev.gcp.example.com` and `prod.gcp.example.com`).
#### Cloud environment
Per the section above Landing acts as the source of truth for DNS within the Cloud environment. Resources defined in the spoke VPCs consume the Landing DNS infrastructure through DNS peering (e.g. `prod-landing-root-dns-peering`).
Spokes can optionally define private zones (e.g. `prod-dns-private-zone`) - granting visibility to the Landing VPC ensures that the whole cloud environment can query such zones.
#### Cloud to on-prem
Leveraging the forwarding zones defined on Landing (e.g. `onprem-example-dns-forwarding` and `reverse-10-dns-forwarding`), the cloud environment can resolve `in-addr.arpa.` and `onprem.example.com.` using the on-premises DNS infrastructure. Onprem resolvers IPs are set in variable `dns.onprem`.
DNS queries sent to the on-premises infrastructure come from the `35.199.192.0/19` source range, which is only accessible from within a VPC or networks connected to one.
#### On-prem to cloud
The [Inbound DNS Policy](https://cloud.google.com/dns/docs/server-policies-overview#dns-server-policy-in) defined in module `landing-vpc` ([`landing.tf`](./landing.tf)) automatically reserves the first available IP address on each created subnet (typically the third one in a CIDR) to expose the Cloud DNS service so that it can be consumed from outside of GCP.
### Private Google Access
[Private Google Access](https://cloud.google.com/vpc/docs/private-google-access) (or PGA) enables VMs and on-prem systems to consume Google APIs from within the Google network, and is already fully configured on this environment.
For PGA to work:
- Private Google Access should be enabled on the subnet. \
Subnets created by the `net-vpc` module are PGA-enabled by default.
- 199.36.153.4/30 (`restricted.googleapis.com`) and 199.36.153.8/30 (`private.googleapis.com`) should be routed from on-prem to VPC, and from there to the `default-internet-gateway`. \
Per variable `vpn_onprem_configs` such ranges are advertised to onprem - furthermore every VPC (e.g. see `landing-vpc` in [`landing.tf`](./landing.tf)) has explicit routes set in case the `0.0.0.0/0` route is changed.
- A private DNS zone for `googleapis.com` should be created and configured per [this article](https://cloud.google.com/vpc/docs/configure-private-google-access-hybrid#config-domain), as implemented in module `googleapis-private-zone` in [`dns.tf`](./dns.tf)
### Preliminar activities
Before running `terraform apply` on this stage, make sure to adapt all of `variables.tf` to your needs, to update all reference to regions (e.g. `europe-west1` or `ew1`) in the whole directory to match your preferences.
If you're not using FAST, you'll also need to create a `providers.tf` file to configure the GCS backend and the service account to use to run the deployment.
You're now ready to run `terraform init` and `apply`.
### Post-deployment activities
- On-prem routers should be configured to advertise all relevant CIDRs to the GCP environments. To avoid hitting GCP quotas, we recomment aggregating routes as much as possible.
- On-prem routers should accept BGP sessions from their cloud peers.
- On-prem DNS servers should have forward zones for GCP-managed ones.
## Customizations
### Adding an environment
To create a new environment (e.g. `staging`), a few changes are required.
Create a `vpc-spoke-staging.tf` file by copying `vpc-spoke-prod.tf` file,
and adapt the new file by replacing the value "prod" with the value "staging".
Running `diff vpc-spoke-dev.tf vpc-spoke-prod.tf` can help to see how environment files differ.
The new VPC requires a set of dedicated CIDRs, one per region, added to variable `custom_adv` (for example as `spoke_staging_ew1` and `spoke_staging_ew4`).
>`custom_adv` is a map that "resolves" CIDR names to actual addresses, and will be used later to configure routing.
>
Variables managing L7 Interal Load Balancers (`l7ilb_subnets`) and Private Service Access (`psa_ranges`) should also be adapted, and subnets and firewall rules for the new spoke should be added as described above.
VPN HA connectivity (see also [VPNs](#vpns)) to `landing` is managed by the `vpn-spoke-*.tf` files.
Copy `vpn-spoke-prod.tf` to `vpn-spoke-staging.tf` - replace "prod" with "staging" where relevant.
VPN configuration also controls BGP advertisements, which requires the following variable changes:
- `router_configs` to configure the new routers (one per region) created for the `staging` VPC
- `vpn_onprem_configs` to configure the new advertisments to on-premises for the new CIDRs
- `vpn_spoke_configs` to configure the new advertisements to `landing` for the new VPC - new keys (one per region) should be added, such as e.g. `staging-ew1` and `staging-ew4`
DNS configurations are centralised in the `dns.tf` file. Spokes delegate DNS resolution to Landing through DNS peering, and optionally define a private zone (e.g. `staging.gcp.example.com`) which the landing peers to. To configure DNS for a new environment, copy all the `prod-*` modules in the `dns.tf` file to `staging-*`, and update their content accordingly. Don't forget to add a peering zone from Landing to the newly created environment private zone.
<!-- BEGIN TFDOC -->
## Files
| name | description | modules | resources |
|---|---|---|---|
| [dns-dev.tf](./dns-dev.tf) | Development spoke DNS zones and peerings setup. | <code>dns</code> | |
| [dns-landing.tf](./dns-landing.tf) | Landing DNS zones and peerings setup. | <code>dns</code> | |
| [dns-prod.tf](./dns-prod.tf) | Production spoke DNS zones and peerings setup. | <code>dns</code> | |
| [main.tf](./main.tf) | Networking folder and hierarchical policy. | <code>folder</code> | |
| [monitoring.tf](./monitoring.tf) | Network monitoring dashboards. | | <code>google_monitoring_dashboard</code> |
| [outputs.tf](./outputs.tf) | Module outputs. | | <code>local_file</code> |
| [test-resources.tf](./test-resources.tf) | temporary instances for testing | <code>compute-vm</code> | |
| [variables.tf](./variables.tf) | Module variables. | | |
| [vpc-landing.tf](./vpc-landing.tf) | Landing VPC and related resources. | <code>net-cloudnat</code> · <code>net-vpc</code> · <code>net-vpc-firewall</code> · <code>project</code> | |
| [vpc-spoke-dev.tf](./vpc-spoke-dev.tf) | Dev spoke VPC and related resources. | <code>net-address</code> · <code>net-cloudnat</code> · <code>net-vpc</code> · <code>net-vpc-firewall</code> · <code>project</code> | |
| [vpc-spoke-prod.tf](./vpc-spoke-prod.tf) | Production spoke VPC and related resources. | <code>net-address</code> · <code>net-cloudnat</code> · <code>net-vpc</code> · <code>net-vpc-firewall</code> · <code>project</code> | |
| [vpn-onprem.tf](./vpn-onprem.tf) | VPN between landing and onprem. | <code>net-vpn-ha</code> | |
| [vpn-spoke-dev.tf](./vpn-spoke-dev.tf) | VPN between landing and development spoke. | <code>net-vpn-ha</code> | |
| [vpn-spoke-prod.tf](./vpn-spoke-prod.tf) | VPN between landing and production spoke. | <code>net-vpn-ha</code> | |
## Variables
| name | description | type | required | default | producer |
|---|---|:---:|:---:|:---:|:---:|
| billing_account_id | Billing account id. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
| organization | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; number&#10; customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| prefix | Prefix used for resources that need unique names. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
| custom_adv | Custom advertisement definitions in name => range format. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; cloud_dns &#61; &#34;35.199.192.0&#47;19&#34;&#10; googleapis_private &#61; &#34;199.36.153.8&#47;30&#34;&#10; googleapis_restricted &#61; &#34;199.36.153.4&#47;30&#34;&#10; rfc_1918_10 &#61; &#34;10.0.0.0&#47;8&#34;&#10; rfc_1918_172 &#61; &#34;172.16.0.0&#47;16&#34;&#10; rfc_1918_192 &#61; &#34;192.168.0.0&#47;16&#34;&#10; landing_ew1 &#61; &#34;10.128.0.0&#47;16&#34;&#10; landing_ew4 &#61; &#34;10.129.0.0&#47;16&#34;&#10; spoke_prod_ew1 &#61; &#34;10.136.0.0&#47;16&#34;&#10; spoke_prod_ew4 &#61; &#34;10.137.0.0&#47;16&#34;&#10; spoke_dev_ew1 &#61; &#34;10.144.0.0&#47;16&#34;&#10; spoke_dev_ew4 &#61; &#34;10.145.0.0&#47;16&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| data_dir | Relative path for the folder storing configuration data for network resources. | <code>string</code> | | <code>&#34;data&#34;</code> | |
| dns | Onprem DNS resolvers | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code title="&#123;&#10; onprem &#61; &#91;&#34;10.0.200.3&#34;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| folder_id | Folder to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created. | <code>string</code> | | <code>null</code> | <code>01-resman</code> |
| gke | | <code title="map&#40;object&#40;&#123;&#10; folder_id &#61; string&#10; sa &#61; string&#10; gcs &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | <code>01-resman</code> |
| l7ilb_subnets | Subnets used for L7 ILBs. | <code title="map&#40;list&#40;object&#40;&#123;&#10; ip_cidr_range &#61; string&#10; region &#61; string&#10;&#125;&#41;&#41;&#41;">map&#40;list&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;&#41;</code> | | <code title="&#123;&#10; prod &#61; &#91;&#10; &#123; ip_cidr_range &#61; &#34;10.136.240.0&#47;24&#34;, region &#61; &#34;europe-west1&#34; &#125;,&#10; &#123; ip_cidr_range &#61; &#34;10.137.240.0&#47;24&#34;, region &#61; &#34;europe-west4&#34; &#125;&#10; &#93;&#10; dev &#61; &#91;&#10; &#123; ip_cidr_range &#61; &#34;10.144.240.0&#47;24&#34;, region &#61; &#34;europe-west1&#34; &#125;,&#10; &#123; ip_cidr_range &#61; &#34;10.145.240.0&#47;24&#34;, region &#61; &#34;europe-west4&#34; &#125;&#10; &#93;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| outputs_location | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | <code>string</code> | | <code>null</code> | |
| project_factory_sa | IAM emails for project factory service accounts | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>01-resman</code> |
| psa_ranges | IP ranges used for Private Service Access (e.g. CloudSQL). | <code>map&#40;map&#40;string&#41;&#41;</code> | | <code title="&#123;&#10; prod &#61; &#123;&#10; cloudsql-mysql &#61; &#34;10.136.250.0&#47;24&#34;&#10; cloudsql-sqlserver &#61; &#34;10.136.251.0&#47;24&#34;&#10; &#125;&#10; dev &#61; &#123;&#10; cloudsql-mysql &#61; &#34;10.144.250.0&#47;24&#34;&#10; cloudsql-sqlserver &#61; &#34;10.144.251.0&#47;24&#34;&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| router_configs | Configurations for CRs and onprem routers. | <code title="map&#40;object&#40;&#123;&#10; adv &#61; object&#40;&#123;&#10; custom &#61; list&#40;string&#41;&#10; default &#61; bool&#10; &#125;&#41;&#10; asn &#61; number&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; onprem-ew1 &#61; &#123;&#10; asn &#61; &#34;65534&#34;&#10; adv &#61; null&#10; &#125;&#10; landing-ew1 &#61; &#123; asn &#61; &#34;64512&#34;, adv &#61; null &#125;&#10; landing-ew4 &#61; &#123; asn &#61; &#34;64512&#34;, adv &#61; null &#125;&#10; spoke-dev-ew1 &#61; &#123; asn &#61; &#34;64513&#34;, adv &#61; null &#125;&#10; spoke-dev-ew4 &#61; &#123; asn &#61; &#34;64513&#34;, adv &#61; null &#125;&#10; spoke-prod-ew1 &#61; &#123; asn &#61; &#34;64514&#34;, adv &#61; null &#125;&#10; spoke-prod-ew4 &#61; &#123; asn &#61; &#34;64514&#34;, adv &#61; null &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| vpn_onprem_configs | VPN gateway configuration for onprem interconnection. | <code title="map&#40;object&#40;&#123;&#10; adv &#61; object&#40;&#123;&#10; default &#61; bool&#10; custom &#61; list&#40;string&#41;&#10; &#125;&#41;&#10; session_range &#61; string&#10; peer &#61; object&#40;&#123;&#10; address &#61; string&#10; asn &#61; number&#10; secret_id &#61; string&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; landing-ew1 &#61; &#123;&#10; adv &#61; &#123;&#10; default &#61; false&#10; custom &#61; &#91;&#10; &#34;cloud_dns&#34;,&#10; &#34;googleapis_restricted&#34;,&#10; &#34;googleapis_private&#34;,&#10; &#34;landing_ew1&#34;,&#10; &#34;landing_ew4&#34;,&#10; &#34;spoke_prod_ew1&#34;,&#10; &#34;spoke_prod_ew4&#34;,&#10; &#34;spoke_dev_ew1&#34;,&#10; &#34;spoke_dev_ew4&#34;&#10; &#93;&#10; &#125;&#10; session_range &#61; &#34;169.254.1.0&#47;29&#34;&#10; peer &#61; &#123;&#10; address &#61; &#34;8.8.8.8&#34;&#10; asn &#61; 65534&#10; secret_id &#61; &#34;foobar&#34;&#10; &#125;&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| vpn_spoke_configs | VPN gateway configuration for spokes. | <code title="map&#40;object&#40;&#123;&#10; adv &#61; object&#40;&#123;&#10; default &#61; bool&#10; custom &#61; list&#40;string&#41;&#10; &#125;&#41;&#10; session_range &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; landing-ew1 &#61; &#123;&#10; adv &#61; &#123;&#10; default &#61; false&#10; custom &#61; &#91;&#34;rfc_1918_10&#34;, &#34;rfc_1918_172&#34;, &#34;rfc_1918_192&#34;&#93;&#10; &#125;&#10; session_range &#61; null &#35; values for the landing router are pulled from the spoke range&#10; &#125;&#10; landing-ew4 &#61; &#123;&#10; adv &#61; &#123;&#10; default &#61; false&#10; custom &#61; &#91;&#34;rfc_1918_10&#34;, &#34;rfc_1918_172&#34;, &#34;rfc_1918_192&#34;&#93;&#10; &#125;&#10; session_range &#61; null &#35; values for the landing router are pulled from the spoke range&#10; &#125;&#10; dev-ew1 &#61; &#123;&#10; adv &#61; &#123;&#10; default &#61; false&#10; custom &#61; &#91;&#34;spoke_dev_ew1&#34;, &#34;spoke_dev_ew4&#34;&#93;&#10; &#125;&#10; session_range &#61; &#34;169.254.0.0&#47;27&#34; &#35; resize according to required number of tunnels&#10; &#125;&#10; prod-ew1 &#61; &#123;&#10; adv &#61; &#123;&#10; default &#61; false&#10; custom &#61; &#91;&#34;spoke_prod_ew1&#34;, &#34;spoke_prod_ew4&#34;&#93;&#10; &#125;&#10; session_range &#61; &#34;169.254.0.64&#47;27&#34; &#35; resize according to required number of tunnels&#10; &#125;&#10; prod-ew4 &#61; &#123;&#10; adv &#61; &#123;&#10; default &#61; false&#10; custom &#61; &#91;&#34;spoke_prod_ew1&#34;, &#34;spoke_prod_ew4&#34;&#93;&#10; &#125;&#10; session_range &#61; &#34;169.254.0.96&#47;27&#34; &#35; resize according to required number of tunnels&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | |
## Outputs
| name | description | sensitive | consumers |
|---|---|:---:|---|
| cloud_dns_inbound_policy | IP Addresses for Cloud DNS inbound policy. | | |
| project_ids | Network project ids. | | |
| project_numbers | Network project numbers. | | |
| shared_vpc_host_projects | Shared VPC host projects. | | |
| shared_vpc_self_links | Shared VPC host projects. | | |
| tfvars | Network-related variables used in other stages. | ✓ | |
| vpn_gateway_endpoints | External IP Addresses for the GCP VPN gateways. | | |
<!-- END TFDOC -->

15
fast/stages/02-networking/data/cidrs.yaml Normal file
View File

@ -0,0 +1,15 @@
# skip boilerplate check
healthchecks:
- 35.191.0.0/16
- 130.211.0.0/22
- 209.85.152.0/22
- 209.85.204.0/22
rfc1918:
- 10.0.0.0/8
- 172.16.0.0/16
- 192.168.0.0/16
onprem_probes:
- 10.255.255.254/32

68
fast/stages/02-networking/data/dashboards/firewall_insights.json Normal file
View File

@ -0,0 +1,68 @@
{
"displayName": "Firewall Insights Monitoring",
"gridLayout": {
"columns": "2",
"widgets": [
{
"title": "Subnet Firewall Hit Counts",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"firewallinsights.googleapis.com/subnet/firewall_hit_count\" resource.type=\"gce_subnetwork\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "VM Firewall Hit Counts",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"firewallinsights.googleapis.com/vm/firewall_hit_count\" resource.type=\"gce_instance\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
}
]
}
}

248
fast/stages/02-networking/data/dashboards/vpn.json Normal file
View File

@ -0,0 +1,248 @@
{
"displayName": "VPN Monitoring",
"gridLayout": {
"columns": "2",
"widgets": [
{
"title": "Number of connections",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_MEAN"
},
"filter": "metric.type=\"vpn.googleapis.com/gateway/connections\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Tunnel established",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_MEAN"
},
"filter": "metric.type=\"vpn.googleapis.com/tunnel_established\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Received bytes",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/received_bytes_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "By"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Sent bytes",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/sent_bytes_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "By"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Received packets",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/received_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "{packets}"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Sent packets",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/sent_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "{packets}"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Incoming packets dropped",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/dropped_received_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Outgoing packets dropped",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/dropped_sent_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
}
]
}
}

15
fast/stages/02-networking/data/firewall-rules/landing/rules.yaml Normal file
View File

@ -0,0 +1,15 @@
# skip boilerplate check
allow-onprem-probes-example:
description: "Allow traffic from onprem probes"
direction: INGRESS
action: allow
sources: []
ranges:
- $onprem_probes
targets: []
use_service_accounts: false
rules:
- protocol: tcp
ports:
- 12345

49
fast/stages/02-networking/data/hierarchical-policy-rules.yaml Normal file
View File

@ -0,0 +1,49 @@
# skip boilerplate check
allow-admins:
description: Access from the admin subnet to all subnets
direction: INGRESS
action: allow
priority: 1000
ranges:
- $rfc1918
ports:
all: []
target_resources: null
enable_logging: false
allow-healthchecks:
description: Enable HTTP and HTTPS healthchecks
direction: INGRESS
action: allow
priority: 1001
ranges:
- $healthchecks
ports:
tcp: ["80", "443"]
target_resources: null
enable_logging: false
allow-ssh-from-iap:
description: Enable SSH from IAP
direction: INGRESS
action: allow
priority: 1002
ranges:
- 35.235.240.0/20
ports:
tcp: ["22"]
target_resources: null
enable_logging: false
allow-icmp:
description: Enable ICMP
direction: INGRESS
action: allow
priority: 1003
ranges:
- 0.0.0.0/0
ports:
icmp: []
target_resources: null
enable_logging: false

5
fast/stages/02-networking/data/subnets/dev/dev-default-ew1.yaml Normal file
View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.144.0.0/24
description: Default subnet for dev

5
fast/stages/02-networking/data/subnets/landing/landing-default-ew1.yaml Normal file
View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.128.0.0/24
description: Default subnet for landing

5
fast/stages/02-networking/data/subnets/prod/prod-default-ew1.yaml Normal file
View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.136.0.0/24
description: Default subnet for prod

BIN
fast/stages/02-networking/diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 138 KiB

2788
fast/stages/02-networking/diagram.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 357 KiB

53
fast/stages/02-networking/dns-dev.tf Normal file
View File

@ -0,0 +1,53 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Development spoke DNS zones and peerings setup.
# GCP-specific environment zone
module "dev-dns-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "dev-gcp-example-com"
domain = "dev.gcp.example.com."
client_networks = [module.dev-spoke-vpc.self_link]
recordsets = {
"A localhost" = { type = "A", ttl = 300, records = ["127.0.0.1"] }
}
}
# root zone peering to landing to centralize configuration; remove if unneeded
module "dev-landing-root-dns-peering" {
source = "../../../modules/dns"
project_id = module.dev-spoke-project.project_id
type = "peering"
name = "dev-root-dns-peering"
domain = "."
client_networks = [module.dev-spoke-vpc.self_link]
peer_network = module.landing-vpc.self_link
}
module "dev-reverse-10-dns-peering" {
source = "../../../modules/dns"
project_id = module.dev-spoke-project.project_id
type = "peering"
name = "dev-reverse-10-dns-peering"
domain = "10.in-addr.arpa."
client_networks = [module.dev-spoke-vpc.self_link]
peer_network = module.landing-vpc.self_link
}

93
fast/stages/02-networking/dns-landing.tf Normal file
View File

@ -0,0 +1,93 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Landing DNS zones and peerings setup.
# forwarding to on-prem DNS resolvers
module "onprem-example-dns-forwarding" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "forwarding"
name = "example-com"
domain = "onprem.example.com."
client_networks = [module.landing-vpc.self_link]
forwarders = { for ip in var.dns.onprem : ip => null }
}
module "reverse-10-dns-forwarding" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "forwarding"
name = "root-reverse-10"
domain = "10.in-addr.arpa."
client_networks = [module.landing-vpc.self_link]
forwarders = { for ip in var.dns.onprem : ip => null }
}
module "gcp-example-dns-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "gcp-example-com"
domain = "gcp.example.com."
client_networks = [module.landing-vpc.self_link]
recordsets = {
"A localhost" = { type = "A", ttl = 300, records = ["127.0.0.1"] }
}
}
# GCP-specific DNS zones peered to the environment spoke that holds the config
module "prod-gcp-example-dns-peering" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "peering"
name = "prod-root-dns-peering"
domain = "prod.gcp.example.com."
client_networks = [module.landing-vpc.self_link]
peer_network = module.prod-spoke-vpc.self_link
}
module "dev-gcp-example-dns-peering" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "peering"
name = "dev-root-dns-peering"
domain = "dev.gcp.example.com."
client_networks = [module.landing-vpc.self_link]
peer_network = module.dev-spoke-vpc.self_link
}
# Google API zone to trigger Private Access
module "googleapis-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "googleapis-com"
domain = "googleapis.com."
client_networks = [module.landing-vpc.self_link]
recordsets = {
"A private" = { type = "A", ttl = 300, records = [
"199.36.153.8", "199.36.153.9", "199.36.153.10", "199.36.153.11"
] }
"A restricted" = { type = "A", ttl = 300, records = [
"199.36.153.4", "199.36.153.5", "199.36.153.6", "199.36.153.7"
] }
"CNAME *" = { type = "CNAME", ttl = 300, records = ["private.googleapis.com."] }
}
}

53
fast/stages/02-networking/dns-prod.tf Normal file
View File

@ -0,0 +1,53 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Production spoke DNS zones and peerings setup.
# GCP-specific environment zone
module "prod-dns-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "prod-gcp-example-com"
domain = "prod.gcp.example.com."
client_networks = [module.prod-spoke-vpc.self_link]
recordsets = {
"A localhost" = { type = "A", ttl = 300, records = ["127.0.0.1"] }
}
}
# root zone peering to landing to centralize configuration; remove if unneeded
module "prod-landing-root-dns-peering" {
source = "../../../modules/dns"
project_id = module.prod-spoke-project.project_id
type = "peering"
name = "prod-root-dns-peering"
domain = "."
client_networks = [module.prod-spoke-vpc.self_link]
peer_network = module.landing-vpc.self_link
}
module "prod-reverse-10-dns-peering" {
source = "../../../modules/dns"
project_id = module.prod-spoke-project.project_id
type = "peering"
name = "prod-reverse-10-dns-peering"
domain = "10.in-addr.arpa."
client_networks = [module.prod-spoke-vpc.self_link]
peer_network = module.landing-vpc.self_link
}

72
fast/stages/02-networking/main.tf Normal file
View File

@ -0,0 +1,72 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Networking folder and hierarchical policy.
locals {
# define the structures used for BGP peers in the VPN resources
bgp_peer_options = {
for k, v in var.vpn_spoke_configs :
k => var.vpn_spoke_configs[k].adv == null ? null : {
advertise_groups = []
advertise_ip_ranges = {
for adv in(var.vpn_spoke_configs[k].adv == null ? [] : var.vpn_spoke_configs[k].adv.custom) :
var.custom_adv[adv] => adv
}
advertise_mode = try(var.vpn_spoke_configs[k].adv.default, false) ? "DEFAULT" : "CUSTOM"
route_priority = null
}
}
bgp_peer_options_onprem = {
for k, v in var.vpn_onprem_configs :
k => var.vpn_onprem_configs[k].adv == null ? null : {
advertise_groups = []
advertise_ip_ranges = {
for adv in(var.vpn_onprem_configs[k].adv == null ? [] : var.vpn_onprem_configs[k].adv.custom) :
var.custom_adv[adv] => adv
}
advertise_mode = try(var.vpn_onprem_configs[k].adv.default, false) ? "DEFAULT" : "CUSTOM"
route_priority = null
}
}
l7ilb_subnets = { for env, v in var.l7ilb_subnets : env => [
for s in v : merge(s, {
active = true
name = "${env}-l7ilb-${s.region}"
})]
}
region_trigram = {
europe-west1 = "ew1"
europe-west3 = "ew3"
}
}
module "folder" {
source = "../../../modules/folder"
parent = "organizations/${var.organization.id}"
name = "Networking"
folder_create = var.folder_id == null
id = var.folder_id
firewall_policy_factory = {
cidr_file = "${var.data_dir}/cidrs.yaml"
policy_name = null
rules_file = "${var.data_dir}/hierarchical-policy-rules.yaml"
}
firewall_policy_association = {
factory-policy = "factory"
}
}

32
fast/stages/02-networking/monitoring.tf Normal file
View File

@ -0,0 +1,32 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Network monitoring dashboards.
locals {
dashboard_path = "${var.data_dir}/dashboards"
dashboard_files = fileset(local.dashboard_path, "*.json")
dashboards = {
for filename in local.dashboard_files :
filename => "${local.dashboard_path}/${filename}"
}
}
resource "google_monitoring_dashboard" "dashboard" {
for_each = local.dashboards
project = module.landing-project.project_id
dashboard_json = file(each.value)
}

95
fast/stages/02-networking/outputs.tf Normal file
View File

@ -0,0 +1,95 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# optionally generate providers and tfvars files for subsequent stages
locals {
tfvars = {
"03-project-factory-dev" = jsonencode({
environment_dns_zone = module.dev-dns-private-zone.domain
shared_vpc_self_link = module.dev-spoke-vpc.self_link
vpc_host_project = module.dev-spoke-project.project_id
})
"03-project-factory-prod" = jsonencode({
environment_dns_zone = module.prod-dns-private-zone.domain
shared_vpc_self_link = module.prod-spoke-vpc.self_link
vpc_host_project = module.prod-spoke-project.project_id
})
}
}
resource "local_file" "tfvars" {
for_each = var.outputs_location == null ? {} : local.tfvars
filename = "${var.outputs_location}/${each.key}/terraform-networking.auto.tfvars.json"
content = each.value
}
# outputs
output "cloud_dns_inbound_policy" {
description = "IP Addresses for Cloud DNS inbound policy."
value = [for s in module.landing-vpc.subnets : cidrhost(s.ip_cidr_range, 2)]
}
output "project_ids" {
description = "Network project ids."
value = {
dev = module.dev-spoke-project.project_id
landing = module.landing-project.project_id
prod = module.prod-spoke-project.project_id
}
}
output "project_numbers" {
description = "Network project numbers."
value = {
dev = "projects/${module.dev-spoke-project.number}"
landing = "projects/${module.landing-project.number}"
prod = "projects/${module.prod-spoke-project.number}"
}
}
output "shared_vpc_host_projects" {
description = "Shared VPC host projects."
value = {
landing = module.landing-project.project_id
dev = module.dev-spoke-project.project_id
prod = module.prod-spoke-project.project_id
}
}
output "shared_vpc_self_links" {
description = "Shared VPC host projects."
value = {
landing = module.landing-vpc.self_link
dev = module.dev-spoke-vpc.self_link
prod = module.prod-spoke-vpc.self_link
}
}
output "vpn_gateway_endpoints" {
description = "External IP Addresses for the GCP VPN gateways."
value = {
onprem-ew1 = { for v in module.landing-to-onprem-ew1-vpn.gateway.vpn_interfaces : v.id => v.ip_address }
}
}
output "tfvars" {
description = "Network-related variables used in other stages."
sensitive = true
value = local.tfvars
}

100
fast/stages/02-networking/test-resources.tf Normal file
View File

@ -0,0 +1,100 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description temporary instances for testing
module "test-vm-landing-0" {
source = "../../../modules/compute-vm"
project_id = module.landing-project.project_id
zone = "europe-west1-b"
name = "test-vm-1"
network_interfaces = [{
network = module.landing-vpc.self_link
subnetwork = module.landing-vpc.subnet_self_links["europe-west1/landing-default-ew1"]
alias_ips = {}
nat = false
addresses = null
}]
tags = ["ssh"]
service_account_create = true
boot_disk = {
image = "projects/debian-cloud/global/images/family/debian-10"
type = "pd-balanced"
size = 10
}
metadata = {
startup-script = <<EOF
apt update
apt install iputils-ping bind9-dnsutils
EOF
}
}
module "test-vm-dev-0" {
source = "../../../modules/compute-vm"
project_id = module.dev-spoke-project.project_id
zone = "europe-west1-b"
name = "test-vm-0"
network_interfaces = [{
network = module.dev-spoke-vpc.self_link
# change the subnet name to match the values you are actually using
subnetwork = module.dev-spoke-vpc.subnet_self_links["europe-west1/dev-default-ew1"]
alias_ips = {}
nat = false
addresses = null
}]
tags = ["ssh"]
service_account_create = true
boot_disk = {
image = "projects/debian-cloud/global/images/family/debian-10"
type = "pd-balanced"
size = 10
}
metadata = {
startup-script = <<EOF
apt update
apt install iputils-ping bind9-dnsutils
EOF
}
}
module "test-vm-prod-0" {
source = "../../../modules/compute-vm"
project_id = module.prod-spoke-project.project_id
zone = "europe-west1-b"
name = "test-vm-0"
network_interfaces = [{
network = module.prod-spoke-vpc.self_link
# change the subnet name to match the values you are actually using
subnetwork = module.prod-spoke-vpc.subnet_self_links["europe-west1/prod-default-ew1"]
alias_ips = {}
nat = false
addresses = null
}]
tags = ["ssh"]
service_account_create = true
boot_disk = {
image = "projects/debian-cloud/global/images/family/debian-10"
type = "pd-balanced"
size = 10
}
metadata = {
startup-script = <<EOF
apt update
apt install iputils-ping bind9-dnsutils
EOF
}
}

253
fast/stages/02-networking/variables.tf Normal file
View File

@ -0,0 +1,253 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "billing_account_id" {
# tfdoc:variable:source 00-bootstrap
description = "Billing account id."
type = string
}
variable "custom_adv" {
description = "Custom advertisement definitions in name => range format."
type = map(string)
default = {
cloud_dns = "35.199.192.0/19"
googleapis_private = "199.36.153.8/30"
googleapis_restricted = "199.36.153.4/30"
rfc_1918_10 = "10.0.0.0/8"
rfc_1918_172 = "172.16.0.0/16"
rfc_1918_192 = "192.168.0.0/16"
landing_ew1 = "10.128.0.0/16"
landing_ew4 = "10.129.0.0/16"
spoke_prod_ew1 = "10.136.0.0/16"
spoke_prod_ew4 = "10.137.0.0/16"
spoke_dev_ew1 = "10.144.0.0/16"
spoke_dev_ew4 = "10.145.0.0/16"
}
}
variable "data_dir" {
description = "Relative path for the folder storing configuration data for network resources."
type = string
default = "data"
}
variable "dns" {
description = "Onprem DNS resolvers"
type = map(list(string))
default = {
onprem = ["10.0.200.3"]
}
}
variable "folder_id" {
# tfdoc:variable:source 01-resman
description = "Folder to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created."
type = string
default = null
validation {
condition = (
var.folder_id == null ||
can(regex("folders/[0-9]{8,}", var.folder_id))
)
error_message = "Invalid folder_id. Should be in 'folders/nnnnnnnnnnn' format."
}
}
variable "gke" {
#tfdoc:variable:source 01-resman
description = ""
type = map(object({
folder_id = string
sa = string
gcs = string
}))
default = {}
}
variable "l7ilb_subnets" {
description = "Subnets used for L7 ILBs."
type = map(list(object({
ip_cidr_range = string
region = string
})))
default = {
prod = [
{ ip_cidr_range = "10.136.240.0/24", region = "europe-west1" },
{ ip_cidr_range = "10.137.240.0/24", region = "europe-west4" }
]
dev = [
{ ip_cidr_range = "10.144.240.0/24", region = "europe-west1" },
{ ip_cidr_range = "10.145.240.0/24", region = "europe-west4" }
]
}
}
variable "organization" {
# tfdoc:variable:source 00-bootstrap
description = "Organization details."
type = object({
domain = string
id = number
customer_id = string
})
}
variable "outputs_location" {
description = "Path where providers and tfvars files for the following stages are written. Leave empty to disable."
type = string
default = null
}
variable "prefix" {
# tfdoc:variable:source 00-bootstrap
description = "Prefix used for resources that need unique names."
type = string
}
variable "project_factory_sa" {
# tfdoc:variable:source 01-resman
description = "IAM emails for project factory service accounts"
type = map(string)
default = {}
}
variable "psa_ranges" {
description = "IP ranges used for Private Service Access (e.g. CloudSQL)."
type = map(map(string))
default = {
prod = {
cloudsql-mysql = "10.136.250.0/24"
cloudsql-sqlserver = "10.136.251.0/24"
}
dev = {
cloudsql-mysql = "10.144.250.0/24"
cloudsql-sqlserver = "10.144.251.0/24"
}
}
}
variable "router_configs" {
description = "Configurations for CRs and onprem routers."
type = map(object({
adv = object({
custom = list(string)
default = bool
})
asn = number
}))
default = {
onprem-ew1 = {
asn = "65534"
adv = null
# adv = { default = false, custom = [] }
}
landing-ew1 = { asn = "64512", adv = null }
landing-ew4 = { asn = "64512", adv = null }
spoke-dev-ew1 = { asn = "64513", adv = null }
spoke-dev-ew4 = { asn = "64513", adv = null }
spoke-prod-ew1 = { asn = "64514", adv = null }
spoke-prod-ew4 = { asn = "64514", adv = null }
}
}
variable "vpn_onprem_configs" {
description = "VPN gateway configuration for onprem interconnection."
type = map(object({
adv = object({
default = bool
custom = list(string)
})
session_range = string
peer = object({
address = string
asn = number
secret_id = string
})
}))
default = {
landing-ew1 = {
adv = {
default = false
custom = [
"cloud_dns",
"googleapis_restricted",
"googleapis_private",
"landing_ew1",
"landing_ew4",
"spoke_prod_ew1",
"spoke_prod_ew4",
"spoke_dev_ew1",
"spoke_dev_ew4"
]
}
session_range = "169.254.1.0/29"
peer = {
address = "8.8.8.8"
asn = 65534
secret_id = "foobar"
}
}
}
}
variable "vpn_spoke_configs" {
description = "VPN gateway configuration for spokes."
type = map(object({
adv = object({
default = bool
custom = list(string)
})
session_range = string
}))
default = {
landing-ew1 = {
adv = {
default = false
custom = ["rfc_1918_10", "rfc_1918_172", "rfc_1918_192"]
}
session_range = null # values for the landing router are pulled from the spoke range
}
landing-ew4 = {
adv = {
default = false
custom = ["rfc_1918_10", "rfc_1918_172", "rfc_1918_192"]
}
session_range = null # values for the landing router are pulled from the spoke range
}
dev-ew1 = {
adv = {
default = false
custom = ["spoke_dev_ew1", "spoke_dev_ew4"]
}
session_range = "169.254.0.0/27" # resize according to required number of tunnels
}
prod-ew1 = {
adv = {
default = false
custom = ["spoke_prod_ew1", "spoke_prod_ew4"]
}
session_range = "169.254.0.64/27" # resize according to required number of tunnels
}
prod-ew4 = {
adv = {
default = false
custom = ["spoke_prod_ew1", "spoke_prod_ew4"]
}
session_range = "169.254.0.96/27" # resize according to required number of tunnels
}
}
}

93
fast/stages/02-networking/vpc-landing.tf Normal file
View File

@ -0,0 +1,93 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Landing VPC and related resources.
module "landing-project" {
source = "../../../modules/project"
billing_account = var.billing_account_id
name = "prod-net-landing-0"
parent = var.folder_id
prefix = var.prefix
service_config = {
disable_on_destroy = false
disable_dependent_services = false
}
services = [
"compute.googleapis.com",
"dns.googleapis.com",
"iap.googleapis.com",
"networkmanagement.googleapis.com",
"stackdriver.googleapis.com"
]
shared_vpc_host_config = {
enabled = true
service_projects = []
}
}
module "landing-vpc" {
source = "../../../modules/net-vpc"
project_id = module.landing-project.project_id
name = "prod-landing-0"
mtu = 1500
dns_policy = {
inbound = true
logging = false
outbound = null
}
# set explicit routes for googleapis in case the default route is deleted
routes = {
private-googleapis = {
dest_range = "199.36.153.8/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
restricted-googleapis = {
dest_range = "199.36.153.4/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
}
data_folder = "${var.data_dir}/subnets/landing"
}
module "landing-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/landing"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
}
module "landing-nat-ew1" {
source = "../../../modules/net-cloudnat"
project_id = module.landing-project.project_id
region = "europe-west1"
name = "ew1"
router_create = true
router_name = "prod-nat-ew1"
router_network = module.landing-vpc.name
router_asn = 4200001024
}

105
fast/stages/02-networking/vpc-spoke-dev.tf Normal file
View File

@ -0,0 +1,105 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Dev spoke VPC and related resources.
module "dev-spoke-project" {
source = "../../../modules/project"
billing_account = var.billing_account_id
name = "dev-net-spoke-0"
parent = var.folder_id
prefix = var.prefix
service_config = {
disable_on_destroy = false
disable_dependent_services = false
}
services = [
"compute.googleapis.com",
"dns.googleapis.com",
"iap.googleapis.com",
"networkmanagement.googleapis.com",
"servicenetworking.googleapis.com",
]
shared_vpc_host_config = {
enabled = true
service_projects = []
}
metric_scopes = [module.landing-project.project_id]
iam = {
"roles/dns.admin" = [var.project_factory_sa.dev]
}
}
module "dev-spoke-vpc" {
source = "../../../modules/net-vpc"
project_id = module.dev-spoke-project.project_id
name = "dev-spoke-0"
mtu = 1500
data_folder = "${var.data_dir}/subnets/dev"
subnets_l7ilb = local.l7ilb_subnets.dev
# set explicit routes for googleapis in case the default route is deleted
routes = {
private-googleapis = {
dest_range = "199.36.153.8/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
restricted-googleapis = {
dest_range = "199.36.153.4/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
}
}
module "dev-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/dev"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
}
module "dev-spoke-cloudnat" {
for_each = toset(values(module.dev-spoke-vpc.subnet_regions))
source = "../../../modules/net-cloudnat"
project_id = module.dev-spoke-project.project_id
region = each.value
name = "dev-nat-${local.region_trigram[each.value]}"
router_create = true
router_network = module.dev-spoke-vpc.name
router_asn = 4200001024
logging_filter = "ERRORS_ONLY"
}
module "dev-spoke-psa-addresses" {
source = "../../../modules/net-address"
project_id = module.dev-spoke-project.project_id
psa_addresses = { for r, v in var.psa_ranges.dev : r => {
address = cidrhost(v, 0)
network = module.dev-spoke-vpc.self_link
prefix_length = split("/", v)[1]
}
}
}

105
fast/stages/02-networking/vpc-spoke-prod.tf Normal file
View File

@ -0,0 +1,105 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Production spoke VPC and related resources.
module "prod-spoke-project" {
source = "../../../modules/project"
billing_account = var.billing_account_id
name = "prod-net-spoke-0"
parent = var.folder_id
prefix = var.prefix
service_config = {
disable_on_destroy = false
disable_dependent_services = false
}
services = [
"compute.googleapis.com",
"dns.googleapis.com",
"iap.googleapis.com",
"networkmanagement.googleapis.com",
"servicenetworking.googleapis.com",
]
shared_vpc_host_config = {
enabled = true
service_projects = []
}
metric_scopes = [module.landing-project.project_id]
iam = {
"roles/dns.admin" = [var.project_factory_sa.prod]
}
}
module "prod-spoke-vpc" {
source = "../../../modules/net-vpc"
project_id = module.prod-spoke-project.project_id
name = "prod-spoke-0"
mtu = 1500
data_folder = "${var.data_dir}/subnets/prod"
subnets_l7ilb = local.l7ilb_subnets.prod
# set explicit routes for googleapis in case the default route is deleted
routes = {
private-googleapis = {
dest_range = "199.36.153.8/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
restricted-googleapis = {
dest_range = "199.36.153.4/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
}
}
module "prod-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/prod"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
}
module "prod-spoke-cloudnat" {
for_each = toset(values(module.prod-spoke-vpc.subnet_regions))
source = "../../../modules/net-cloudnat"
project_id = module.prod-spoke-project.project_id
region = each.value
name = "prod-nat-${local.region_trigram[each.value]}"
router_create = true
router_network = module.prod-spoke-vpc.name
router_asn = 4200001024
logging_filter = "ERRORS_ONLY"
}
module "prod-spoke-psa-addresses" {
source = "../../../modules/net-address"
project_id = module.prod-spoke-project.project_id
psa_addresses = { for r, v in var.psa_ranges.prod : r => {
address = cidrhost(v, 0)
network = module.prod-spoke-vpc.self_link
prefix_length = split("/", v)[1]
}
}
}

50
fast/stages/02-networking/vpn-onprem.tf Normal file
View File

@ -0,0 +1,50 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description VPN between landing and onprem.
module "landing-to-onprem-ew1-vpn" {
source = "../../../modules/net-vpn-ha"
project_id = module.landing-project.project_id
network = module.landing-vpc.self_link
region = "europe-west1"
name = "vpn-to-onprem-ew1"
router_create = true
router_name = "dev-spoke-vpn-ew1"
router_asn = var.router_configs.landing-ew1.asn
peer_external_gateway = {
redundancy_type = "SINGLE_IP_INTERNALLY_REDUNDANT"
interfaces = [{
id = 0
# on-prem router ip address
ip_address = var.vpn_onprem_configs.landing-ew1.peer.address
}]
}
tunnels = { for t in range(2) : "remote-${t}" => {
bgp_peer = {
address = cidrhost(var.vpn_onprem_configs.landing-ew1.session_range, 1 + (t * 4))
asn = var.vpn_onprem_configs.landing-ew1.peer.asn
}
bgp_peer_options = local.bgp_peer_options_onprem["landing-ew1"]
bgp_session_range = "${cidrhost(var.vpn_onprem_configs.landing-ew1.session_range, 2 + (t * 4))}/30"
ike_version = 2
peer_external_gateway_interface = 0
router = null
shared_secret = var.vpn_onprem_configs.landing-ew1.peer.secret_id
vpn_gateway_interface = t
}
}
}

73
fast/stages/02-networking/vpn-spoke-dev.tf Normal file
View File

@ -0,0 +1,73 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description VPN between landing and development spoke.
module "landing-to-dev-ew1-vpn" {
source = "../../../modules/net-vpn-ha"
project_id = module.landing-project.project_id
network = module.landing-vpc.self_link
region = "europe-west1"
name = "vpn-to-dev-ew1"
# The router used for this VPN is managed in vpn-prod.tf
router_create = false
router_name = "landing-vpn-ew1"
router_asn = var.router_configs.landing-ew1.asn
peer_gcp_gateway = module.dev-to-landing-ew1-vpn.self_link
tunnels = { for t in range(2) : "tunnel-${t}" => {
bgp_peer = {
address = cidrhost(var.vpn_spoke_configs.dev-ew1.session_range, 1 + (t * 4))
asn = var.router_configs.spoke-dev-ew1.asn
}
bgp_peer_options = local.bgp_peer_options["landing-ew1"]
bgp_session_range = "${cidrhost(var.vpn_spoke_configs.dev-ew1.session_range, 2 + (t * 4))}/30"
ike_version = 2
peer_external_gateway_interface = null
router = null
shared_secret = null
vpn_gateway_interface = t
}
}
depends_on = [
module.landing-to-prod-ew1-vpn.router
]
}
module "dev-to-landing-ew1-vpn" {
source = "../../../modules/net-vpn-ha"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.self_link
region = "europe-west1"
name = "vpn-to-landing-ew1"
router_create = true
router_name = "dev-spoke-vpn-ew1"
router_asn = var.router_configs.spoke-dev-ew1.asn
peer_gcp_gateway = module.landing-to-dev-ew1-vpn.self_link
tunnels = { for t in range(2) : "tunnel-${t}" => {
bgp_peer = {
address = cidrhost(var.vpn_spoke_configs.dev-ew1.session_range, 2 + (t * 4))
asn = var.router_configs.landing-ew1.asn
}
bgp_peer_options = local.bgp_peer_options["dev-ew1"]
bgp_session_range = "${cidrhost(var.vpn_spoke_configs.dev-ew1.session_range, 1 + (t * 4))}/30"
ike_version = 2
peer_external_gateway_interface = null
router = null
shared_secret = module.landing-to-dev-ew1-vpn.random_secret
vpn_gateway_interface = t
}
}
}

121
fast/stages/02-networking/vpn-spoke-prod.tf Normal file
View File

@ -0,0 +1,121 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description VPN between landing and production spoke.
module "landing-to-prod-ew1-vpn" {
source = "../../../modules/net-vpn-ha"
project_id = module.landing-project.project_id
network = module.landing-vpc.self_link
region = "europe-west1"
name = "vpn-to-prod-ew1"
router_create = true
router_name = "landing-vpn-ew1"
router_asn = var.router_configs.landing-ew1.asn
peer_gcp_gateway = module.prod-to-landing-ew1-vpn.self_link
tunnels = { for t in range(2) : "tunnel-${t}" => {
bgp_peer = {
address = cidrhost(var.vpn_spoke_configs.prod-ew1.session_range, 1 + (t * 4))
asn = var.router_configs.spoke-prod-ew1.asn
}
bgp_peer_options = local.bgp_peer_options["landing-ew1"]
bgp_session_range = "${cidrhost(var.vpn_spoke_configs.prod-ew1.session_range, 2 + (t * 4))}/30"
ike_version = 2
peer_external_gateway_interface = null
router = null
shared_secret = null
vpn_gateway_interface = t
}
}
}
module "prod-to-landing-ew1-vpn" {
source = "../../../modules/net-vpn-ha"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.self_link
region = "europe-west1"
name = "vpn-to-landing-ew1"
router_create = true
router_name = "prod-spoke-vpn-ew1"
router_asn = var.router_configs.spoke-prod-ew1.asn
peer_gcp_gateway = module.landing-to-prod-ew1-vpn.self_link
tunnels = { for t in range(2) : "tunnel-${t}" => {
bgp_peer = {
address = cidrhost(var.vpn_spoke_configs.prod-ew1.session_range, 2 + (t * 4))
asn = var.router_configs.landing-ew1.asn
}
bgp_peer_options = local.bgp_peer_options["prod-ew1"]
bgp_session_range = "${cidrhost(var.vpn_spoke_configs.prod-ew1.session_range, 1 + (t * 4))}/30"
ike_version = 2
peer_external_gateway_interface = null
router = null
shared_secret = module.landing-to-prod-ew1-vpn.random_secret
vpn_gateway_interface = t
}
}
}
module "landing-to-prod-ew4-vpn" {
source = "../../../modules/net-vpn-ha"
project_id = module.landing-project.project_id
network = module.landing-vpc.self_link
region = "europe-west1"
name = "vpn-to-prod-ew4"
router_create = true
router_name = "landing-vpn-ew4"
router_asn = var.router_configs.landing-ew4.asn
peer_gcp_gateway = module.prod-to-landing-ew4-vpn.self_link
tunnels = { for t in range(2) : "tunnel-${t}" => {
bgp_peer = {
address = cidrhost(var.vpn_spoke_configs.prod-ew4.session_range, 1 + (t * 4))
asn = var.router_configs.spoke-prod-ew4.asn
}
bgp_peer_options = local.bgp_peer_options["landing-ew4"]
bgp_session_range = "${cidrhost(var.vpn_spoke_configs.prod-ew4.session_range, 2 + (t * 4))}/30"
ike_version = 2
peer_external_gateway_interface = null
router = null
shared_secret = null
vpn_gateway_interface = t
}
}
}
module "prod-to-landing-ew4-vpn" {
source = "../../../modules/net-vpn-ha"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.self_link
region = "europe-west1"
name = "vpn-to-landing-ew4"
router_create = true
router_name = "prod-spoke-vpn-ew4"
router_asn = var.router_configs.spoke-prod-ew4.asn
peer_gcp_gateway = module.landing-to-prod-ew4-vpn.self_link
tunnels = { for t in range(2) : "tunnel-${t}" => {
bgp_peer = {
address = cidrhost(var.vpn_spoke_configs.prod-ew4.session_range, 2 + (t * 4))
asn = var.router_configs.landing-ew4.asn
}
bgp_peer_options = local.bgp_peer_options["prod-ew4"]
bgp_session_range = "${cidrhost(var.vpn_spoke_configs.prod-ew4.session_range, 1 + (t * 4))}/30"
ike_version = 2
peer_external_gateway_interface = null
router = null
shared_secret = module.landing-to-prod-ew4-vpn.random_secret
vpn_gateway_interface = t
}
}
}

323
fast/stages/02-security/README.md Normal file
View File

@ -0,0 +1,323 @@
# Shared security resources
This stage sets up security resources and configurations which impact the whole organization, or are shared across the hierarchy to other projects and teams.
The design of this stage is fairly general, and provides a reference example for [Cloud KMS](https://cloud.google.com/security-key-management) and a [VPC Service Controls](https://cloud.google.com/vpc-service-controls) configuration that sets up three perimeters (landing, development, production), their related bridge perimeters, and provides variables to configure their resources, access levels, and directional policies.
Expanding this stage to include other security-related services like Secret Manager, is fairly simple by using the provided implementation for Cloud KMS, and leveraging the broad permissions on the top-level Security folder of the automation service account used.
The following diagram illustrates the high-level design of created resources and a schema of the VPC SC design, which can be adapted to specific requirements via variables:
<p align="center">
<img src="diagram.svg" alt="Security diagram">
</p>
## Design overview and choices
Project-level security resources are grouped into two separate projects, one per environment. This setup matches requirements we frequently observe in real life and provides enough separation without needlessly complicating operations.
Cloud KMS is configured and designed mainly to encrypt GCP resources with a [Customer-managed encryption key](https://cloud.google.com/kms/docs/cmek) but it may be used to create cryptokeys used to [encrypt application data](https://cloud.google.com/kms/docs/encrypting-application-data) too.
IAM for management-related operations is already assigned at the folder level to the security team by the previous stage, but more granularity can be added here at the project level, to grant control of separate services across environments to different actors.
### Cloud KMS
A reference Cloud KMS implementation is part of this stage, to provide a simple way of managing centralized keys, that are then shared and consumed widely across the organization to enable customer-managed encryption. The implementation is also easy to clone and modify to support other services like Secret Manager.
The Cloud KMS configuration allows defining keys by name (typically matching the downstream service that uses them) in different locations, either based on a common default or a per-key setting. It then takes care internally of provisioning the relevant keyrings and creating keys in the appropriate location.
IAM roles on keys can be configured at the logical level for all locations where a logical key is created. Their management can also be delegated via [delegated role grants](https://cloud.google.com/iam/docs/setting-limits-on-granting-roles) exposed through a simple variable, to allow other identities to set IAM policies on keys. This is particularly useful in setups like project factories, making it possible to configure IAM bindings during project creation for team groups or service agent accounts (compute, storage, etc.).
### VPC Service Controls
This stage also provisions the VPC Service Controls configuration on demand for the whole organization, implementing the straightforward design illustrated above:
- one perimeter for each environment
- one perimeter for centralized services and the landing VPC
- bridge perimeters to connect the landing perimeter to each environment
The VPC SC configuration is set to dry-run mode, but switching to enforced mode is a simple operation involving modifying a few lines of code highlighted by ad-hoc comments. Variables are designed to enable easy centralized management of VPC Service Controls, including access levels and [ingress/egress rules](https://cloud.google.com/vpc-service-controls/docs/ingress-egress-rules) as described below.
Some care needs to be taken with project membership in perimeters, which can only be implemented here instead of being delegated (all or partially) to different stages, until the [Google Provider feature request](https://github.com/hashicorp/terraform-provider-google/issues/7270) allowing using project-level association for both enforced and dry-run modes is implemented.
## How to run this stage
This stage is meant to be executed after the [resource management](../01-resman) stage has run, as it leverages the folder and automation resources created there. The relevant user groups must also exist, but that's one of the requirements for the previous stages too, so if you ran those successfully, you're good to go.
It's possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the bootstrap stage for the required roles.
Before running this stage, you need to ensure you have the correct credentials and permissions, and customize variables by assigning values that match your configuration.
### Providers configuration
The default way of making sure you have the correct permissions is to use the identity of the service account pre-created for this stage during bootstrap, and that you are a member of the group that can impersonate it via provider-level configuration (`gcp-devops` or `organization-admins`).
To simplify setup, the previous stage pre-configures a valid providers file in its output, and optionally writes it to a local file if the `outputs_location` variable is set to a valid path.
If you have set a valid value for `outputs_location` in the resource management stage, simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
```bash
# `outputs_location` is set to `../../configs/example`
ln -s ../../configs/example/02-security/providers.tf
```
If you have not configured `outputs_location` in resource management, you can derive the providers file from that stage's outputs:
```bash
cd ../01-resman
terraform output -json providers | jq -r '.["02-security"]' \
> ../02-security/providers.tf
```
### Variable configuration
There are two broad sets of variables you will need to fill in:
- variables shared by other stages (organization id, billing account id, etc.), or derived from a resource managed by a different stage (folder id, automation project id, etc.)
- variables specific to resources managed by this stage
To avoid the tedious job of filling in the first group of variables with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files.
If you configured a valid path for `outputs_location` in the previous stages, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's output folder (under the path you specified), where the `*` above is set to the name of the stage that produced it. For this stage, two `.tfvars` files are available:
```bash
# `outputs_location` is set to `../../configs/example`
ln -s ../../configs/example/02-security/terraform-bootstrap.auto.tfvars.json
ln -s ../../configs/example/02-security/terraform-resman.auto.tfvars.json
```
A second set of optional variables is specific to this stage. If you need to customize them, create an extra `terraform.tfvars` file.
Refer to the [Variables](#variables) table at the bottom of this document, for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning. The sections below also describe some of the possible customizations.
Once done, you can run this stage:
```bash
terraform init
terraform apply
```
## Customizations
### KMS keys
Cloud KMS configuration is split in two variables:
- `kms_defaults` configures the locations and rotation period, used for keys that don't specifically configure them
- `kms_keys` configures the actual keys to create, and also allows configuring their IAM bindings and labels, and overriding locations and rotation period. When configuring locations for a key, please consider the limitations each cloud product may have.
The additional `kms_restricted_admins` variable allows granting `roles/cloudkms.admin` to specified principals, restricted via [delegated role grants](https://cloud.google.com/iam/docs/setting-limits-on-granting-roles) so that it only allows granting the roles needed for encryption/decryption on keys. This allows safe delegation of key management to subsequent Terraform stages like the Project Factory, for example to grant usage access on relevant keys to the service agent accounts for compute, storage, etc.
To support these scenarios, key IAM bindings are configured by default to be additive, to enable other stages or Terraform configuration to safely co-manage bindings on the same keys. If this is not desired, follow the comments in the `core-dev.tf` and `core-prod.tf` files to switch to authoritative bindings on keys.
An example of how to configure keys:
```hcl
# terraform.tfvars
kms_defaults = {
locations = ["europe-west1", "europe-west3", "global"]
rotation_period = "7776000s"
}
kms_keys = {
compute = {
iam = {
"roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
"user:user1@example.com"
]
}
labels = { service = "compute" }
locations = null
rotation_period = null
}
storage = {
iam = null
labels = { service = "compute" }
locations = ["europe"]
rotation_period = null
}
}
```
The script will create one keyring for each specified location and keys on each keyring.
### VPC Service Controls configuration
A set of variables allows configuring the VPC SC perimeters described above:
- `vpc_sc_perimeter_projects` configures project membership in the three regular perimeters
- `vpc_sc_access_levels` configures access levels, which can then be associated to perimeters by key using the `vpc_sc_perimeter_access_levels`
- `vpc_sc_egress_policies` configures directional egress policies, which can then be associated to perimeters by key using the `vpc_sc_perimeter_egress_policies`
- `vpc_sc_ingress_policies` configures directional ingress policies, which can then be associated to perimeters by key using the `vpc_sc_perimeter_ingress_policies`
This allows configuring VPC SC in a fairly flexible and concise way, without repeating similar definitions. Bridges perimeters configuration will be computed automatically to allow communication between regular perimeters: `landing <-> prod` and `landing <-> dev`.
#### Dry-run vs. enforced
The VPC SC configuration is set up by default in dry-run mode to allow easy experimentation, and detecting violations before enforcement. Once everything is set up correctly, switching to enforced mode needs to be done in code, by swapping the contents of the `spec` and `status` attributes for perimeters in the `vpc-sc.tf` file. The effort involved is minimal (2 lines of code per perimeter), and comments help identify the correct lines.
#### Perimeter resources
Projects are added to perimeters via the `vpc_sc_perimeter_projects`, and that's currently the only way of doing it without generating permadiffs or conflicts, because of the way the Terraform provider is implemented.
Once the Google Terraform Provider [implements support for dry-run mode in the additive resource](https://github.com/hashicorp/terraform-provider-google/issues/7270), it will be possible to concurrently manage perimeter resources both here and in subsequent Terraform configurations, for example to allow the Project Factory to add a project to a perimeter during the creation process.
Bridge perimeters are auto-populated with all projects configured for the connected regular perimeters.
An example of adding projects to perimeters using project numbers:
```hcl
# terraform.tfvars
vpc_sc_perimeter_projects = {
dev = ["projects/12345678", "projects/12345679"]
landing = ["projects/12345670"]
prod = ["projects/12345674", "projects/12345675"]
}
```
#### Access levels
Below an example for an access level that allows unconditional ingress from a set of IP CIDR ranges can be configured once, and enabled on selected perimeters:
```hcl
# terraform.tfvars
vpc_sc_access_levels = {
on-prem = {
conditions = [{
ip_subnetworks = ["10.0.0.0/24", "10.0.0.1/24"],
combining_function = null, members = null, negate = null,
regions = null, required_access_levels = null
}]
}
}
vpc_sc_perimeter_access_levels = {
dev = null
landing = ["on-prem"]
prod = ["on-prem"]
}
```
#### Ingress and Egress policies
The same applies to Ingress and Egress policies, as shown in the examples below referencing the automation service account for this stage.
Below you can find an ingress policy configuration that allows applying Terraform from outside the perimeter, useful when bringing up this stage to avoid generating violations:
```hcl
# terraform.tfvars
vpc_sc_ingress_policies = {
iac = {
ingress_from = {
identities = [
"serviceAccount:xxx-prod-resman-security-0@xxx-prod-iac-core-0.iam.gserviceaccount.com"
]
source_access_levels = ["*"]
identity_type = null
source_resources = null
}
ingress_to = {
operations = [{ method_selectors = [], service_name = "*" }]
resources = ["*"]
}
}
}
vpc_sc_perimeter_ingress_policies = {
dev = ["iac"]
landing = ["iac"]
prod = ["iac"]
}
```
Below you can find an egress policy that allows writing Terraform state to the automation bucket, useful once Terraform starts running inside the perimeter in a pipeline:
```hcl
# terraform.tfvars
vpc_sc_egress_policies = {
iac-gcs = {
egress_from = {
identity_type = null
identities = [
"serviceAccount:xxx-prod-resman-security-0@xxx-prod-iac-core-0.iam.gserviceaccount.com"
]
}
egress_to = {
operations = [{
method_selectors = ["*"], service_name = "storage.googleapis.com"
}]
resources = ["projects/123456782"]
}
}
}
vpc_sc_perimeter_ingress_policies = {
dev = ["iac-gcs"]
landing = ["iac-gcs"]
prod = ["iac-gcs"]
}
```
## Notes
Some references that might be useful in setting up this stage:
- [VPC SC CSCC requirements](https://cloud.google.com/security-command-center/docs/troubleshooting).
<!-- BEGIN TFDOC -->
## Files
| name | description | modules | resources |
|---|---|---|---|
| [core-dev.tf](./core-dev.tf) | None | <code>kms</code> · <code>project</code> | <code>google_project_iam_member</code> |
| [core-prod.tf](./core-prod.tf) | None | <code>kms</code> · <code>project</code> | <code>google_project_iam_member</code> |
| [main.tf](./main.tf) | Module-level locals and resources. | | |
| [outputs.tf](./outputs.tf) | Module outputs. | | <code>local_file</code> |
| [variables.tf](./variables.tf) | Module variables. | | |
| [vpc-sc.tf](./vpc-sc.tf) | None | <code>vpc-sc</code> | |
## Variables
| name | description | type | required | default | producer |
|---|---|:---:|:---:|:---:|:---:|
| billing_account_id | Billing account id. | <code>string</code> | ✓ | | <code>bootstrap</code> |
| folder_id | Folder to be used for the networking resources in folders/nnnn format. | <code>string</code> | ✓ | | <code>resman</code> |
| organization | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; number&#10; customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>bootstrap</code> |
| prefix | Prefix used for resources that need unique names. | <code>string</code> | ✓ | | |
| groups | Group names to grant organization-level permissions. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; gcp-billing-admins &#61; &#34;gcp-billing-admins&#34;,&#10; gcp-devops &#61; &#34;gcp-devops&#34;,&#10; gcp-network-admins &#61; &#34;gcp-network-admins&#34;&#10; gcp-organization-admins &#61; &#34;gcp-organization-admins&#34;&#10; gcp-security-admins &#61; &#34;gcp-security-admins&#34;&#10; gcp-support &#61; &#34;gcp-support&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | <code>bootstrap</code> |
| kms_defaults | Defaults used for KMS keys. | <code title="object&#40;&#123;&#10; locations &#61; list&#40;string&#41;&#10; rotation_period &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; locations &#61; &#91;&#34;europe&#34;, &#34;europe-west1&#34;, &#34;europe-west3&#34;, &#34;global&#34;&#93;&#10; rotation_period &#61; &#34;7776000s&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| kms_keys | KMS keys to create, keyed by name. Null attributes will be interpolated with defaults. | <code title="map&#40;object&#40;&#123;&#10; iam &#61; map&#40;list&#40;string&#41;&#41;&#10; labels &#61; map&#40;string&#41;&#10; locations &#61; list&#40;string&#41;&#10; rotation_period &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| kms_restricted_admins | Map of environment => [identities] who can assign the encrypt/decrypt roles on keys. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| outputs_location | Path where providers, tfvars files, and lists for the following stages are written. Leave empty to disable. | <code>string</code> | | <code>null</code> | |
| vpc_sc_access_levels | VPC SC access level definitions. | <code title="map&#40;object&#40;&#123;&#10; combining_function &#61; string&#10; conditions &#61; list&#40;object&#40;&#123;&#10; ip_subnetworks &#61; list&#40;string&#41;&#10; members &#61; list&#40;string&#41;&#10; negate &#61; bool&#10; regions &#61; list&#40;string&#41;&#10; required_access_levels &#61; list&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| vpc_sc_egress_policies | VPC SC egress policy defnitions. | <code title="map&#40;object&#40;&#123;&#10; egress_from &#61; object&#40;&#123;&#10; identity_type &#61; string&#10; identities &#61; list&#40;string&#41;&#10; &#125;&#41;&#10; egress_to &#61; object&#40;&#123;&#10; operations &#61; list&#40;object&#40;&#123;&#10; method_selectors &#61; list&#40;string&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;&#10; resources &#61; list&#40;string&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| vpc_sc_ingress_policies | VPC SC ingress policy defnitions. | <code title="map&#40;object&#40;&#123;&#10; ingress_from &#61; object&#40;&#123;&#10; identity_type &#61; string&#10; identities &#61; list&#40;string&#41;&#10; source_access_levels &#61; list&#40;string&#41;&#10; source_resources &#61; list&#40;string&#41;&#10; &#125;&#41;&#10; ingress_to &#61; object&#40;&#123;&#10; operations &#61; list&#40;object&#40;&#123;&#10; method_selectors &#61; list&#40;string&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;&#10; resources &#61; list&#40;string&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | |
| vpc_sc_perimeter_access_levels | VPC SC perimeter access_levels. | <code title="object&#40;&#123;&#10; dev &#61; list&#40;string&#41;&#10; landing &#61; list&#40;string&#41;&#10; prod &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| vpc_sc_perimeter_egress_policies | VPC SC egress policies per perimeter, values reference keys defined in the `vpc_sc_ingress_policies` variable. | <code title="object&#40;&#123;&#10; dev &#61; list&#40;string&#41;&#10; landing &#61; list&#40;string&#41;&#10; prod &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| vpc_sc_perimeter_ingress_policies | VPC SC ingress policies per perimeter, values reference keys defined in the `vpc_sc_ingress_policies` variable. | <code title="object&#40;&#123;&#10; dev &#61; list&#40;string&#41;&#10; landing &#61; list&#40;string&#41;&#10; prod &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| vpc_sc_perimeter_projects | VPC SC perimeter resources. | <code title="object&#40;&#123;&#10; dev &#61; list&#40;string&#41;&#10; landing &#61; list&#40;string&#41;&#10; prod &#61; list&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
## Outputs
| name | description | sensitive | consumers |
|---|---|:---:|---|
| stage_perimeter_projects | Security project numbers. They can be added to perimeter resources. | | |
<!-- END TFDOC -->

64
fast/stages/02-security/core-dev.tf Normal file
View File

@ -0,0 +1,64 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "dev-sec-project" {
source = "../../../modules/project"
name = "dev-sec-core-0"
parent = var.folder_id
prefix = var.prefix
billing_account = var.billing_account_id
iam = {
"roles/cloudkms.viewer" = try(var.kms_restricted_admins.dev, [])
}
labels = { environment = "dev", team = "security" }
services = local.project_services
}
module "dev-sec-kms" {
for_each = toset(local.kms_locations)
source = "../../../modules/kms"
project_id = module.dev-sec-project.project_id
keyring = {
location = each.key
name = "dev-${each.key}"
}
# rename to `key_iam` to switch to authoritative bindings
key_iam_additive = {
for k, v in local.kms_locations_keys[each.key] : k => v.iam
}
keys = local.kms_locations_keys[each.key]
}
# TODO(ludo): add support for conditions to Fabric modules
resource "google_project_iam_member" "dev_key_admin_delegated" {
for_each = toset(try(var.kms_restricted_admins.dev, []))
project = module.dev-sec-project.project_id
role = "roles/cloudkms.admin"
member = each.key
condition {
title = "kms_sa_delegated_grants"
description = "Automation service account delegated grants"
expression = format(
"api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
join(",", formatlist("'%s'", [
"roles/cloudkms.cryptoKeyEncrypterDecrypter",
"roles/cloudkms.cryptoKeyEncrypterDecrypterViaDelegation"
]))
)
}
depends_on = [module.dev-sec-project]
}

64
fast/stages/02-security/core-prod.tf Normal file
View File

@ -0,0 +1,64 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "prod-sec-project" {
source = "../../../modules/project"
name = "prod-sec-core-0"
parent = var.folder_id
prefix = var.prefix
billing_account = var.billing_account_id
iam = {
"roles/cloudkms.viewer" = try(var.kms_restricted_admins.prod, [])
}
labels = { environment = "prod", team = "security" }
services = local.project_services
}
module "prod-sec-kms" {
for_each = toset(local.kms_locations)
source = "../../../modules/kms"
project_id = module.prod-sec-project.project_id
keyring = {
location = each.key
name = "prod-${each.key}"
}
# rename to `key_iam` to switch to authoritative bindings
key_iam_additive = {
for k, v in local.kms_locations_keys[each.key] : k => v.iam
}
keys = local.kms_locations_keys[each.key]
}
# TODO(ludo): add support for conditions to Fabric modules
resource "google_project_iam_member" "prod_key_admin_delegated" {
for_each = toset(try(var.kms_restricted_admins.prod, []))
project = module.prod-sec-project.project_id
role = "roles/cloudkms.admin"
member = each.key
condition {
title = "kms_sa_delegated_grants"
description = "Automation service account delegated grants"
expression = format(
"api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
join(",", formatlist("'%s'", [
"roles/cloudkms.cryptoKeyEncrypterDecrypter",
"roles/cloudkms.cryptoKeyEncrypterDecrypterViaDelegation"
]))
)
}
depends_on = [module.prod-sec-project]
}

BIN
fast/stages/02-security/diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 91 KiB

1157
fast/stages/02-security/diagram.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 430 KiB

47
fast/stages/02-security/main.tf Normal file
View File

@ -0,0 +1,47 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
kms_keys = {
for k, v in var.kms_keys : k => {
iam = coalesce(v.iam, {})
labels = coalesce(v.labels, {})
locations = (
v.locations == null
? var.kms_defaults.locations
: v.locations
)
rotation_period = (
v.rotation_period == null
? var.kms_defaults.rotation_period
: v.rotation_period
)
}
}
kms_locations = distinct(flatten([
for k, v in local.kms_keys : v.locations
]))
kms_locations_keys = {
for loc in local.kms_locations : loc => {
for k, v in local.kms_keys : k => v if contains(v.locations, loc)
}
}
project_services = [
"cloudkms.googleapis.com",
"secretmanager.googleapis.com",
"stackdriver.googleapis.com"
]
}

43
fast/stages/02-security/outputs.tf Normal file
View File

@ -0,0 +1,43 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# optionally generate files for subsequent stages
resource "local_file" "dev_sec_kms" {
for_each = var.outputs_location == null ? {} : { 1 = 1 }
filename = "${var.outputs_location}/yamls/02-security-kms-dev-keys.yaml"
content = yamlencode({
for k, m in module.dev-sec-kms : k => m.key_ids
})
}
resource "local_file" "prod_sec_kms" {
for_each = var.outputs_location == null ? {} : { 1 = 1 }
filename = "${var.outputs_location}/yamls/02-security-kms-prod-keys.yaml"
content = yamlencode({
for k, m in module.prod-sec-kms : k => m.key_ids
})
}
# outputs
output "stage_perimeter_projects" {
description = "Security project numbers. They can be added to perimeter resources."
value = {
dev = ["projects/${module.dev-sec-project.number}"]
prod = ["projects/${module.prod-sec-project.number}"]
}
}

185
fast/stages/02-security/variables.tf Normal file
View File

@ -0,0 +1,185 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "billing_account_id" {
# tfdoc:variable:source bootstrap
description = "Billing account id."
type = string
}
variable "folder_id" {
# tfdoc:variable:source resman
description = "Folder to be used for the networking resources in folders/nnnn format."
type = string
}
variable "groups" {
# tfdoc:variable:source bootstrap
description = "Group names to grant organization-level permissions."
type = map(string)
# https://cloud.google.com/docs/enterprise/setup-checklist
default = {
gcp-billing-admins = "gcp-billing-admins",
gcp-devops = "gcp-devops",
gcp-network-admins = "gcp-network-admins"
gcp-organization-admins = "gcp-organization-admins"
gcp-security-admins = "gcp-security-admins"
gcp-support = "gcp-support"
}
}
variable "kms_defaults" {
description = "Defaults used for KMS keys."
type = object({
locations = list(string)
rotation_period = string
})
default = {
locations = ["europe", "europe-west1", "europe-west3", "global"]
rotation_period = "7776000s"
}
}
variable "kms_keys" {
description = "KMS keys to create, keyed by name. Null attributes will be interpolated with defaults."
type = map(object({
iam = map(list(string))
labels = map(string)
locations = list(string)
rotation_period = string
}))
default = {}
}
variable "kms_restricted_admins" {
description = "Map of environment => [identities] who can assign the encrypt/decrypt roles on keys."
type = map(list(string))
default = {}
}
variable "organization" {
# tfdoc:variable:source bootstrap
description = "Organization details."
type = object({
domain = string
id = number
customer_id = string
})
}
variable "outputs_location" {
description = "Path where providers, tfvars files, and lists for the following stages are written. Leave empty to disable."
type = string
default = null
}
variable "prefix" {
description = "Prefix used for resources that need unique names."
type = string
}
variable "vpc_sc_access_levels" {
description = "VPC SC access level definitions."
type = map(object({
combining_function = string
conditions = list(object({
ip_subnetworks = list(string)
members = list(string)
negate = bool
regions = list(string)
required_access_levels = list(string)
}))
}))
default = {}
}
variable "vpc_sc_egress_policies" {
description = "VPC SC egress policy defnitions."
type = map(object({
egress_from = object({
identity_type = string
identities = list(string)
})
egress_to = object({
operations = list(object({
method_selectors = list(string)
service_name = string
}))
resources = list(string)
})
}))
default = {}
}
variable "vpc_sc_ingress_policies" {
description = "VPC SC ingress policy defnitions."
type = map(object({
ingress_from = object({
identity_type = string
identities = list(string)
source_access_levels = list(string)
source_resources = list(string)
})
ingress_to = object({
operations = list(object({
method_selectors = list(string)
service_name = string
}))
resources = list(string)
})
}))
default = {}
}
variable "vpc_sc_perimeter_access_levels" {
description = "VPC SC perimeter access_levels."
type = object({
dev = list(string)
landing = list(string)
prod = list(string)
})
default = null
}
variable "vpc_sc_perimeter_egress_policies" {
description = "VPC SC egress policies per perimeter, values reference keys defined in the `vpc_sc_ingress_policies` variable."
type = object({
dev = list(string)
landing = list(string)
prod = list(string)
})
default = null
}
variable "vpc_sc_perimeter_ingress_policies" {
description = "VPC SC ingress policies per perimeter, values reference keys defined in the `vpc_sc_ingress_policies` variable."
type = object({
dev = list(string)
landing = list(string)
prod = list(string)
})
default = null
}
variable "vpc_sc_perimeter_projects" {
description = "VPC SC perimeter resources."
type = object({
dev = list(string)
landing = list(string)
prod = list(string)
})
default = null
}

88
fast/stages/02-security/vpc-sc-restricted-services.yaml Normal file
View File

@ -0,0 +1,88 @@
# skip boilerplate check
- accessapproval.googleapis.com
- adsdatahub.googleapis.com
- aiplatform.googleapis.com
- alpha-documentai.googleapis.com
- apigee.googleapis.com
- apigeeconnect.googleapis.com
- artifactregistry.googleapis.com
- assuredworkloads.googleapis.com
- automl.googleapis.com
- bigquery.googleapis.com
- bigquerydatatransfer.googleapis.com
- bigtable.googleapis.com
- binaryauthorization.googleapis.com
- cloudasset.googleapis.com
- cloudbuild.googleapis.com
- cloudfunctions.googleapis.com
- cloudkms.googleapis.com
- cloudprofiler.googleapis.com
- cloudresourcemanager.googleapis.com
- cloudsearch.googleapis.com
- cloudtrace.googleapis.com
- composer.googleapis.com
- compute.googleapis.com
- connectgateway.googleapis.com
- contactcenterinsights.googleapis.com
- container.googleapis.com
- containeranalysis.googleapis.com
- containerregistry.googleapis.com
- containerthreatdetection.googleapis.com
- datacatalog.googleapis.com
- dataflow.googleapis.com
- datafusion.googleapis.com
- dataproc.googleapis.com
- datastream.googleapis.com
- dialogflow.googleapis.com
- dlp.googleapis.com
- dns.googleapis.com
- documentai.googleapis.com
- eventarc.googleapis.com
- file.googleapis.com
- gameservices.googleapis.com
- gkeconnect.googleapis.com
- gkehub.googleapis.com
- healthcare.googleapis.com
- iam.googleapis.com
- iaptunnel.googleapis.com
- language.googleapis.com
- lifesciences.googleapis.com
- logging.googleapis.com
- managedidentities.googleapis.com
- memcache.googleapis.com
- meshca.googleapis.com
- metastore.googleapis.com
- ml.googleapis.com
- monitoring.googleapis.com
- networkconnectivity.googleapis.com
- networkmanagement.googleapis.com
- networksecurity.googleapis.com
- networkservices.googleapis.com
- notebooks.googleapis.com
- opsconfigmonitoring.googleapis.com
- osconfig.googleapis.com
- oslogin.googleapis.com
- privateca.googleapis.com
- pubsub.googleapis.com
- pubsublite.googleapis.com
- recaptchaenterprise.googleapis.com
- recommender.googleapis.com
- redis.googleapis.com
- run.googleapis.com
- secretmanager.googleapis.com
- servicecontrol.googleapis.com
- servicedirectory.googleapis.com
- spanner.googleapis.com
- speakerid.googleapis.com
- speech.googleapis.com
- sqladmin.googleapis.com
- storage.googleapis.com
- storagetransfer.googleapis.com
- texttospeech.googleapis.com
- tpu.googleapis.com
- trafficdirector.googleapis.com
- transcoder.googleapis.com
- translate.googleapis.com
- videointelligence.googleapis.com
- vision.googleapis.com
- vpcaccess.googleapis.com

167
fast/stages/02-security/vpc-sc.tf Normal file
View File

@ -0,0 +1,167 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
# compute the number of projects in each perimeter to detect which to create
vpc_sc_counts = {
for k in ["dev", "landing", "prod"] : k => length(
coalesce(try(var.vpc_sc_perimeter_projects[k], null), [])
)
}
# dereference perimeter egress policy names to the actual objects
vpc_sc_perimeter_egress_policies = {
for k, v in coalesce(var.vpc_sc_perimeter_egress_policies, {}) :
k => [
for i in coalesce(v, []) : var.vpc_sc_egress_policies[i]
if lookup(var.vpc_sc_egress_policies, i, null) != null
]
}
# dereference perimeter ingress policy names to the actual objects
vpc_sc_perimeter_ingress_policies = {
for k, v in coalesce(var.vpc_sc_perimeter_ingress_policies, {}) :
k => [
for i in coalesce(v, []) : var.vpc_sc_ingress_policies[i]
if lookup(var.vpc_sc_ingress_policies, i, null) != null
]
}
# get the list of restricted services from the yaml file
vpcsc_restricted_services = yamldecode(
file("${path.module}/vpc-sc-restricted-services.yaml")
)
}
module "vpc-sc" {
source = "../../../modules/vpc-sc"
# only enable if we have projects defined for perimeters
count = anytrue([for k, v in local.vpc_sc_counts : v > 0]) ? 1 : 0
access_policy = null
access_policy_create = {
parent = "organizations/${var.organization.id}"
title = "default"
}
access_levels = coalesce(try(var.vpc_sc_access_levels, null), {})
# bridge type perimeters
service_perimeters_bridge = merge(
# landing to dev, only we have projects in landing and dev perimeters
local.vpc_sc_counts.landing * local.vpc_sc_counts.dev == 0 ? {} : {
landing_to_dev = {
status_resources = null
spec_resources = concat(
var.vpc_sc_perimeter_projects.landing,
var.vpc_sc_perimeter_projects.dev
)
use_explicit_dry_run_spec = true
}
},
# landing to prod, only we have projects in landing and prod perimeters
local.vpc_sc_counts.landing * local.vpc_sc_counts.prod == 0 ? {} : {
landing_to_prod = {
status_resources = null
spec_resources = concat(
var.vpc_sc_perimeter_projects.landing,
var.vpc_sc_perimeter_projects.prod
)
# set to null and switch spec and status above to enforce
use_explicit_dry_run_spec = true
}
}
)
# regular type perimeters
service_perimeters_regular = merge(
# dev if we have projects in var.vpc_sc_perimeter_projects.dev
local.vpc_sc_counts.dev == 0 ? {} : {
dev = {
spec = {
access_levels = coalesce(
try(var.vpc_sc_perimeter_access_levels.dev, null), []
)
resources = var.vpc_sc_perimeter_projects.dev
restricted_services = local.vpcsc_restricted_services
egress_policies = try(
local.vpc_sc_perimeter_egress_policies.dev, null
)
ingress_policies = try(
local.vpc_sc_perimeter_ingress_policies.dev, null
)
# replace with commented block to enable vpc restrictions
vpc_accessible_services = null
# vpc_accessible_services = {
# allowed_services = ["RESTRICTED-SERVICES"]
# enable_restriction = true
# }
}
status = null
# set to null and switch spec and status above to enforce
use_explicit_dry_run_spec = true
}
},
# prod if we have projects in var.vpc_sc_perimeter_projects.prod
local.vpc_sc_counts.prod == 0 ? {} : {
prod = {
spec = {
access_levels = coalesce(
try(var.vpc_sc_perimeter_access_levels.prod, null), []
)
# combine the security project, and any specified in the variable
resources = var.vpc_sc_perimeter_projects.prod
restricted_services = local.vpcsc_restricted_services
egress_policies = try(
local.vpc_sc_perimeter_egress_policies.prod, null
)
ingress_policies = try(
local.vpc_sc_perimeter_ingress_policies.prod, null
)
# replace with commented block to enable vpc restrictions
vpc_accessible_services = null
# vpc_accessible_services = {
# allowed_services = ["RESTRICTED-SERVICES"]
# enable_restriction = true
# }
}
status = null
# set to null and switch spec and status above to enforce
use_explicit_dry_run_spec = true
}
},
# prod if we have projects in var.vpc_sc_perimeter_projects.prod
local.vpc_sc_counts.landing == 0 ? {} : {
landing = {
spec = {
access_levels = coalesce(
try(var.vpc_sc_perimeter_access_levels.landing, null), []
)
resources = var.vpc_sc_perimeter_projects.landing
restricted_services = local.vpcsc_restricted_services
egress_policies = try(
local.vpc_sc_perimeter_egress_policies.landing, null
)
ingress_policies = try(
local.vpc_sc_perimeter_ingress_policies.landing, null
)
# replace with commented block to enable vpc restrictions
vpc_accessible_services = null
# vpc_accessible_services = {
# allowed_services = ["RESTRICTED-SERVICES"]
# enable_restriction = true
# }
}
status = null
# set to null and switch spec and status above to enforce
use_explicit_dry_run_spec = true
}
}
)
}

6
fast/stages/03-project-factory/README.md Normal file
View File

@ -0,0 +1,6 @@
# Project factory
The Project Factory (PF) builds on top of your foundations to create and set up projects (and related resources) to be used for your workloads.
It is organized in folders representing environments (e.g. "dev", "prod"), each implemented by a stand-alone terraform [resource factory](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c).
This directory contains a single project factory ([`prod/`](./prod/)) as an example - to implement multiple environments (e.g. "prod" and "dev") you'll need to copy the `prod` folder into one folder per environment, then customize each one following the instructions found in [`prod/README.md`](./prod/README.md).

131
fast/stages/03-project-factory/prod/README.md Normal file
View File

@ -0,0 +1,131 @@
# Project factory
The Project Factory (or PF) builds on top of your foundations to create and set up projects (and related resources) to be used for your workloads.
It is organized in folders representing environments (e.g., "dev", "prod"), each implemented by a stand-alone terraform [resource factory](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c).
## Design overview and choices
<p align="center">
<img src="diagram.svg" alt="Project factory diagram">
</p>
A single factory creates projects in a well-defined context, according to your resource management structure. For example, in the diagram above, each Team is structured to have specific folders projects for a given environment, such as Production and Development, per the resource management structure configured in stage `01-resman`.
Projects for each environment across different teams are created by dedicated service accounts, as exemplified in the diagram above. While there's no intrinsic limitation regarding where the project factory can create a projects, the IAM bindings for the service account effectively enforce boundaries (e.g., the production service account shouldn't be able to create or have any access to the development projects, and vice versa).
The project factory takes care of the following activities:
* Project creation
* API/Services enablement
* Service accounts creation
* IAM roles assignment for groups and service accounts
* KMS keys roles assignment
* Shared VPC attachment and subnets IAM binding
* DNS zones creation and visibility configuration
* Project-level org policies definition
* Billing setup (billing account attachment and budget configuration)
* Essential contacts definition (for [budget alerts](https://cloud.google.com/billing/docs/how-to/budgets) and [important notifications](https://cloud.google.com/resource-manager/docs/managing-notification-contacts?hl=en))
## How to run this stage
This stage is meant to be executed after "foundational stages" (i.e., stages [`00-bootstrap`](../../00-bootstrap), [`01-resman`](../../01-resman), [`02-networking`](../../02-networking) and [`02-security`](../../02-security)) have been run.
It's of course possible to run this stage in isolation, by making sure the architectural prerequisites are satisfied (e.g., networking), and that the Service Account running the stage is granted the roles/permissions below:
* One service account per environment, each with appropriate permissions
* at the organization level a custom role for networking operations including the following permissions
* `"compute.organizations.enableXpnResource"`,
* `"compute.organizations.disableXpnResource"`,
* `"compute.subnetworks.setIamPolicy"`,
* `"dns.networks.bindPrivateDNSZone"`
* and role `"roles/orgpolicy.policyAdmin"`
* on each folder where projects are created
* `"roles/logging.admin"`
* `"roles/owner"`
* `"roles/resourcemanager.folderAdmin"`
* `"roles/resourcemanager.projectCreator"`
* on the host project for the Shared VPC
* `"roles/browser"`
* `"roles/compute.viewer"`
* `"roles/dns.admin"`
* If networking is used (e.g., for VMs, GKE Clusters or AppEngine flex), VPC Host projects and their subnets should exist when creating projects
* If per-environment DNS sub-zones are required, one "root" zone per environment should exist when creating projects (e.g., prod.gcp.example.com.)
### Providers configuration
If you're running this on top of Fast, you should run the following commands to create the providers file, and populate the required variables from the previous stage.
```bash
# Variable `outputs_location` is set to `../../configs/example` in stage 01-resman
$ cd fabric-fast/stages/03-project-factory/prod
ln -s ../../../configs/example/03-project-factory-prod/providers.tf
```
### Variable configuration
There are two broad sets of variables you will need to fill in:
- variables shared by other stages (org id, billing account id, etc.), or derived from a resource managed by a different stage (folder id, automation project id, etc.)
- variables specific to resources managed by this stage
To avoid the tedious job of filling in the first group of variables with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files.
If you configured a valid path for `outputs_location` in the bootstrap and networking stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's outputs folder (under the path you specified), where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is available:
```bash
# Variable `outputs_location` is set to `../../configs/example` in stages 01-bootstrap and 02-networking
ln -s ../../../configs/example/03-project-factory-prod/terraform-bootstrap.auto.tfvars.json
ln -s ../../../configs/example/03-project-factory-prod/terraform-networking.auto.tfvars.json
```
If you're not using Fast, refer to the [Variables](#variables) table at the bottom of this document for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning.
Besides the values above, a project factory takes 2 additional inputs:
* `data/defaults.yaml`, manually configured by adapting the [`prod/data/defaults.yaml.sample`](./prod/data/defaults.yaml.sample), which defines per-environment default values e.g., for billing alerts and labels.
* `data/projects/*.yaml`, one file per project (optionally grouped in folders), which configures each project. A [`prod/data/projects/project.yaml.sample`](./prod/data/projects/project.yaml.sample) is provided as reference and documentation for the schema. Projects will be named after the filename, e.g., `fast-prod-lab0.yaml` will create project `fast-prod-lab0`.
Once the configuration is complete, run the project factory by running
```bash
terraform init
terraform apply
```
<!-- BEGIN TFDOC -->
## Files
| name | description | modules | resources |
|---|---|---|---|
| [main.tf](./main.tf) | Project factory. | <code>project-factory</code> | |
| [outputs.tf](./outputs.tf) | Module outputs. | | |
| [variables.tf](./variables.tf) | Module variables. | | |
## Variables
| name | description | type | required | default | producer |
|---|---|:---:|:---:|:---:|:---:|
| billing_account_id | Billing account id. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
| shared_vpc_self_link | Self link for the shared VPC. | <code>string</code> | ✓ | | <code>02-networking</code> |
| vpc_host_project | Host project for the shared VPC. | <code>string</code> | ✓ | | <code>02-networking</code> |
| data_dir | Relative path for the folder storing configuration data. | <code>string</code> | | <code>&#34;data&#47;projects&#34;</code> | |
| defaults_file | Relative path for the file storing the project factory configuration. | <code>string</code> | | <code>&#34;data&#47;defaults.yaml&#34;</code> | |
| environment_dns_zone | DNS zone suffix for environment. | <code>string</code> | | <code>null</code> | <code>02-networking</code> |
## Outputs
| name | description | sensitive | consumers |
|---|---|:---:|---|
| projects | Created projects and service accounts. | | |
<!-- END TFDOC -->

24
fast/stages/03-project-factory/prod/data/defaults.yaml Normal file
View File

@ -0,0 +1,24 @@
# skip boilerplate check
billing_account_id: 012345-67890A-BCDEF0
# [opt] Setup for billing alerts
billing_alert:
amount: 1000
thresholds:
current: [0.5, 0.8]
forecasted: [0.5, 0.8]
credit_treatment: INCLUDE_ALL_CREDITS
# [opt] Contacts for billing alerts and important notifications
essential_contacts: ["team-contacts@example.com"]
# [opt] Labels set for all projects
labels:
environment: prod
department: accounting
application: example-app
foo: bar
# [opt] Additional notification channels for billing
notification_channels: []

100
fast/stages/03-project-factory/prod/data/projects/project.yaml Normal file
View File

@ -0,0 +1,100 @@
# skip boilerplate check
# [opt] Billing account id - overrides default if set
billing_account_id: 012345-67890A-BCDEF0
# [opt] Billing alerts config - overrides default if set
billing_alert:
amount: 10
thresholds:
current:
- 0.5
- 0.8
forecasted: []
credit_treatment: INCLUDE_ALL_CREDITS
# [opt] DNS zones to be created as children of the environment_dns_zone defined in defaults
dns_zones:
- lorem
- ipsum
# [opt] Contacts for billing alerts and important notifications
essential_contacts:
- team-a-contacts@example.com
# Folder the project will be created as children of
folder_id: folders/012345678901
# [opt] Authoritative IAM bindings in group => [roles] format
group_iam:
test-team-foobar@fast-lab-0.gcp-pso-italy.net:
- roles/compute.admin
# [opt] Authoritative IAM bindings in role => [principals] format
# Generally used to grant roles to service accounts external to the project
iam:
roles/compute.admin:
- serviceAccount:service-account
# [opt] Service robots and keys they will be assigned as cryptoKeyEncrypterDecrypter
# in service => [keys] format
kms_service_agents:
compute: [key1, key2]
storage: [key1, key2]
# [opt] Labels for the project - merged with the ones defined in defaults
labels:
environment: prod
# [opt] Org policy overrides defined at project level
org_policies:
policy_boolean:
constraints/compute.disableGuestAttributesAccess: true
policy_list:
constraints/compute.trustedImageProjects:
inherit_from_parent: null
status: true
suggested_value: null
values:
- projects/fast-prod-iac-core-0
# [opt] Service account to create for the project and their roles on the project
# in name => [roles] format
service_accounts:
another-service-account:
- roles/compute.admin
my-service-account:
- roles/compute.admin
# [opt] APIs to enable on the project.
services:
- storage.googleapis.com
- stackdriver.googleapis.com
- compute.googleapis.com
# [opt] Roles to assign to the robots service accounts in robot => [roles] format
services_iam:
compute:
- roles/storage.objectViewer
# [opt] VPC setup.
# If set enables the `compute.googleapis.com` service and configures
# service project attachment
vpc:
# [opt] If set, enables the container API
gke_setup:
# Grants "roles/container.hostServiceAgentUser" to the container robot if set
enable_host_service_agent: false
# Grants "roles/compute.securityAdmin" to the container robot if set
enable_security_admin: true
# Host project the project will be service project of
host_project: fast-prod-net-spoke-0
# [opt] Subnets in the host project where principals will be granted networkUser
# in region/subnet-name => [principals]
subnets_iam:
europe-west1/prod-default-ew1:
- user:foobar@example.com
- serviceAccount:service-account1

BIN
fast/stages/03-project-factory/prod/diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 56 KiB

1530
fast/stages/03-project-factory/prod/diagram.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 590 KiB

56
fast/stages/03-project-factory/prod/main.tf Normal file
View File

@ -0,0 +1,56 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Project factory.
locals {
_defaults = yamldecode(file(var.defaults_file))
_defaults_net = {
billing_account_id = var.billing_account_id
environment_dns_zone = var.environment_dns_zone
shared_vpc_self_link = var.shared_vpc_self_link
vpc_host_project = var.vpc_host_project
}
defaults = merge(local._defaults, local._defaults_net)
projects = {
for f in fileset("${var.data_dir}", "**/*.yaml") :
trimsuffix(f, ".yaml") => yamldecode(file("${var.data_dir}/${f}"))
}
}
module "projects" {
source = "../../../../examples/factories/project-factory"
for_each = local.projects
defaults = local.defaults
project_id = each.key
billing_account_id = try(each.value.billing_account_id, null)
billing_alert = try(each.value.billing_alert, null)
dns_zones = try(each.value.dns_zones, [])
essential_contacts = try(each.value.essential_contacts, [])
folder_id = each.value.folder_id
group_iam = try(each.value.group_iam, {})
iam = try(each.value.iam, {})
kms_service_agents = try(each.value.kms, {})
labels = try(each.value.labels, {})
org_policies = try(each.value.org_policies, null)
service_accounts = try(each.value.service_accounts, {})
services = try(each.value.services, [])
services_iam = try(each.value.services_iam, {})
vpc = try(each.value.vpc, null)
}

20
fast/stages/03-project-factory/prod/outputs.tf Normal file
View File

@ -0,0 +1,20 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "projects" {
description = "Created projects and service accounts."
value = module.projects
}

54
fast/stages/03-project-factory/prod/variables.tf Normal file
View File

@ -0,0 +1,54 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#TODO: tfdoc annotations
variable "billing_account_id" {
# tfdoc:variable:source 00-bootstrap
description = "Billing account id."
type = string
}
variable "data_dir" {
description = "Relative path for the folder storing configuration data."
type = string
default = "data/projects"
}
variable "environment_dns_zone" {
# tfdoc:variable:source 02-networking
description = "DNS zone suffix for environment."
type = string
default = null
}
variable "defaults_file" {
description = "Relative path for the file storing the project factory configuration."
type = string
default = "data/defaults.yaml"
}
variable "shared_vpc_self_link" {
# tfdoc:variable:source 02-networking
description = "Self link for the shared VPC."
type = string
}
variable "vpc_host_project" {
# tfdoc:variable:source 02-networking
description = "Host project for the shared VPC."
type = string
}

29
fast/stages/README.md Normal file
View File

@ -0,0 +1,29 @@
# Fast stages
Each of the folders contained here is a separate "stage", or Terraform root module.
They are designed to be combined together, each stage leveraging the previous stage's resources and providing outputs to the following stages, but they can also be run in isolation if their specific functionality is all that is needed (e.g. only bring up a hub and spoke VPC in an existing environment).
Refer to each stage's documentation for a detailed description of its purpose, the architectural choices made in its design, and how it can be configured and wired together to terraform a whole GCP organization. The following is a brief overview of each stage.
## Organizational level (00-01)
- [Bootstrap](stages/00-bootstrap/README.md)
Enables critical organization-level functionality that depends on broad permissions. It has two primary purposes. The first is to bootstrap the resources needed for automation of this and the following stages (service accounts, GCS buckets). And secondly, it applies the minimum amount of configuration needed at the organization level, to avoid the need of broad permissions later on, and to implement a minimum of security features like sinks and exports from the start.
- [Resource Management](stages/01-resman/README.md)
Creates the base resource hierarchy (folders) and the automation resources required later to delegate deployment of each part of the hierarchy to separate stages. This stage also configures organization-level policies and any exceptions needed by different branches of the resource hierarchy.
## Shared resources (02)
- [Security](stages/02-security/README.md)
Manages centralized security configurations in a separate stage, and is typically owned by the security team. This stage implements VPC Security Controls via separate perimeters for environments and central services, and creates projects to host centralized KMS keys used by the whole organization. It's meant to be easily extended to include other security-related resources which are required, like Secret Manager.
- [Networking](stages/02-security/README.md)
Manages centralized network resources in a separate stage, and is typically owned by the networking team. This stage implements a hub-and-spoke design, and includes connectivity via VPN to on-premises, and YAML-based factories for firewall rules (hierarchical and VPC-level) and subnets.
## Environment-level resources (03)
- [Project Factory](stages/03-project-factory/README.md)
YAML-based fatory to create and configure application or team-level projects. Configuration includes VPC-level settings for Shared VPC, service-level configuration for CMEK encryption via centralized keys, and service account creation for workloads and applications. This stage is meant to be used once per environment.
- Data Platform (in development)
- GKE Multitenant (in development)
- GCE Migration (in development)

BIN
stages.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 39 KiB

13
tests/conftest.py
View File

@ -38,12 +38,15 @@ def _plan_runner():
fixture_parent = os.path.dirname(fixture_path)
fixture_prefix = os.path.basename(fixture_path) + "_"
tf_lock = os.path.join(BASEDIR, 'tests/.terraform.lock.hcl')
tf_lock_use = os.path.isfile(tf_lock)
with tempfile.TemporaryDirectory(prefix=fixture_prefix,
dir=fixture_parent) as tmp_path:
# copy fixture to a temporary directory so we can execute
# multiple tests in parallel
shutil.copytree(fixture_path, tmp_path, dirs_exist_ok=True)
if tf_lock_use:
os.symlink(tf_lock, os.path.join(tmp_path, '.terraform.lock.hcl'))
tf = tftest.TerraformTest(tmp_path, BASEDIR,
os.environ.get('TERRAFORM', 'terraform'))
tf.setup(upgrade=True)
@ -52,7 +55,7 @@ def _plan_runner():
return run_plan
@pytest.fixture(scope='session')
@ pytest.fixture(scope='session')
def plan_runner(_plan_runner):
"Returns a function to run Terraform plan on a module fixture."
@ -66,7 +69,7 @@ def plan_runner(_plan_runner):
return run_plan
@pytest.fixture(scope='session')
@ pytest.fixture(scope='session')
def e2e_plan_runner(_plan_runner):
"Returns a function to run Terraform plan on an end-to-end fixture."
@ -88,7 +91,7 @@ def e2e_plan_runner(_plan_runner):
return run_plan
@pytest.fixture(scope='session')
@ pytest.fixture(scope='session')
def example_plan_runner(_plan_runner):
"Returns a function to run Terraform plan on documentation examples."
@ -104,7 +107,7 @@ def example_plan_runner(_plan_runner):
return run_plan
@pytest.fixture(scope='session')
@ pytest.fixture(scope='session')
def apply_runner():
"Returns a function to run Terraform apply on a fixture."

2
tests/modules/examples/conftest.py → tests/doc_examples/conftest.py
View File

@ -17,7 +17,7 @@ from pathlib import Path
import marko
MODULES_PATH = Path(__file__).parents[3] / 'modules/'
MODULES_PATH = Path(__file__).parents[2] / 'modules/'
print(MODULES_PATH)

12
tests/modules/examples/test_plan.py → tests/doc_examples/test_plan.py
View File

@ -12,21 +12,19 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import tftest
import re
import tempfile
from pathlib import Path
import marko
MODULES_PATH = Path(__file__, '../../../../modules/').resolve()
VARIABLES_PATH = Path(__file__, '../variables.tf').resolve()
BASE_PATH = Path(__file__).parent
EXPECTED_RESOURCES_RE = re.compile(r'# tftest:modules=(\d+):resources=(\d+)')
def test_example(example_plan_runner, tmp_path, example):
(tmp_path / 'modules').symlink_to(MODULES_PATH)
(tmp_path / 'variables.tf').symlink_to(VARIABLES_PATH)
(tmp_path / 'modules').symlink_to(
Path(BASE_PATH, '../../modules/').resolve())
(tmp_path / 'variables.tf').symlink_to(
Path(BASE_PATH, 'variables.tf').resolve())
(tmp_path / 'main.tf').write_text(example)
match = EXPECTED_RESOURCES_RE.search(example)

0
tests/modules/examples/variables.tf → tests/doc_examples/variables.tf
View File

0
tests/examples/factories/net-vpc-firewall-yaml/__init__.py → tests/examples/factories/net_vpc_firewall_yaml/__init__.py
View File

0
tests/examples/factories/net-vpc-firewall-yaml/fixture/main.tf → tests/examples/factories/net_vpc_firewall_yaml/fixture/main.tf
View File

0
tests/examples/factories/net-vpc-firewall-yaml/fixture/rules/common.yaml → tests/examples/factories/net_vpc_firewall_yaml/fixture/rules/common.yaml
View File

0
tests/examples/factories/net-vpc-firewall-yaml/fixture/variables.tf → tests/examples/factories/net_vpc_firewall_yaml/fixture/variables.tf
View File

0
tests/examples/factories/net-vpc-firewall-yaml/test_plan.py → tests/examples/factories/net_vpc_firewall_yaml/test_plan.py
View File

34
tests/fast/README.md Normal file
View File

@ -0,0 +1,34 @@
# Fabric FAST
Setting up a production-ready GCP organization is often a time-consuming process. Fabric FAST aims to speed up this process via two complementary goals. On the one hand, FAST provides a design of a GCP organization that includes the typical elements required by enterprise customers. Secondly, we provide a reference implementation of the FAST design using Terraform.
Note that while our implementation is necessarily influenced (and constrained) by the way Terraform works, the design we put forward only refers to GCP constructs and features. In other words, while we use Terraform for our reference implementation, in theory, the FAST design can be implemented using any other tool (e.g., Pulumi, bash scripts, or even calling the relevant APIs directly).
Fabric FAST comes from engineers in Google Cloud's Professional Services Organization, with a combined experience of decades solving the typical technical problems faced by GCP customers. While every GCP user has specific requirements, many common issues arise repeatedly. Solving those issues correctly from the beginning is key to a robust and scalable GCP setup. It's those common issues and their solutions that Fabric FAST aims to collect and present coherently.
Fabric FAST was initially conceived to help enterprises quickly set up a GCP organization following battle-tested and widely-used patterns. Despite its origin in enterprise environments, FAST includes many customization points making it an ideal blueprint for organizations of all sizes, ranging from startups to the largest companies.
## Guiding principles
### Contracts and stages
FAST uses the concept of stages, which individually perform precise tasks but, taken together, build a functional, ready-to-use GCP organization. More importantly, stages are modeled around the security boundaries that typically appear in mature organizations. This arrangement allows delegating ownership of each stage to the team responsible for the types of resources it manages. For example, as its name suggests, the networking stage sets up all the networking elements and is usually the responsibility of a dedicated networking team within the organization.
From the perspective of FAST's overall design, stages also work as contacts or interfaces, defining a set of pre-requisites and inputs required to perform their designed task and generating outputs needed by other stages lower in the chain.
### Security-first design
Security was, from the beginning, one of the most critical elements in the design of Fabric FAST. Many of FAST's design decisions aim to build the foundations of a secure organization. In fact, the first two stages deal mainly with the organization-wide security setup.
FAST also aims to minimize the number of permissions granted to principals according to the security-first approach previously mentioned. We achieve this through the meticulous use of groups, service accounts, custom roles, and [Cloud IAM Conditions](https://cloud.google.com/iam/docs/conditions-overview), among other things.
### Extensive use of factories
A resource factory consumes a simple representation of a resource (e.g., in YAML) and deploys it (e.g., using Terraform). Used correctly, factories can help decrease the management overhead of large-scale infrastructure deployments. See "[Resource Factories: A descriptive approach to Terraform](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c)" for more details and the rationale behind factories.
FAST uses YAML-based factories to deploy subnets and firewall rules and, as its name suggests, in the [project factory](./stages/03-project-factory/) stage.
## High level design
TBD
## Implementation
TBD

13
tests/fast/__init__.py Normal file
View File

@ -0,0 +1,13 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

48
tests/fast/conftest.py Normal file
View File

@ -0,0 +1,48 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"Shared fixtures"
import inspect
import os
import types
import pytest
import tftest
BASEDIR = os.path.dirname(os.path.dirname(__file__))
@pytest.fixture(scope='session')
def fast_e2e_plan_runner(_plan_runner):
"Plan runner for end-to-end root module, returns modules and resources."
def run_plan(fixture_path=None, targets=None, refresh=True,
include_bare_resources=False, compute_sums=True, **tf_vars):
"Runs Terraform plan on a root module using defaults, returns data."
plan = _plan_runner(fixture_path, targets=targets, refresh=refresh,
**tf_vars)
root_module = plan.root_module['child_modules'][0]
modules = {
m['address'].removeprefix(root_module['address'])[1:]: m['resources']
for m in root_module['child_modules']
}
resources = [r for m in modules.values() for r in m]
if include_bare_resources:
bare_resources = root_module['resources']
resources.extend(bare_resources)
if compute_sums:
return len(modules), len(resources), {k: len(v) for k, v in modules.items()}
return modules, resources
return run_plan

13
tests/fast/stages/__init__.py Normal file
View File

@ -0,0 +1,13 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

13
tests/fast/stages/s00_bootstrap/__init__.py Normal file
View File

@ -0,0 +1,13 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

Some files were not shown because too many files have changed in this diff Show More