FAST: Separate network environment (#566)

This commit is contained in:
Simone Ruffilli 2022-10-10 11:50:07 +02:00 committed by GitHub
parent 22fa031399
commit 24c3ffe66b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
27 changed files with 3880 additions and 0 deletions

View File

@ -0,0 +1 @@
ludo-*

View File

@ -0,0 +1,16 @@
# IAM bindings reference
Legend: <code>+</code> additive, <code></code> conditional.
## Project <i>dev-net-spoke-0</i>
| members | roles |
|---|---|
|<b>dev-resman-pf-0</b><br><small><i>serviceAccount</i></small>|[roles/resourcemanager.projectIamAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectIamAdmin) <code></code><br>[roles/dns.admin](https://cloud.google.com/iam/docs/understanding-roles#dns.admin) |
|<b>prod-resman-pf-0</b><br><small><i>serviceAccount</i></small>|organizations/[org_id #0]/roles/serviceProjectNetworkAdmin |
## Project <i>prod-net-spoke-0</i>
| members | roles |
|---|---|
|<b>prod-resman-pf-0</b><br><small><i>serviceAccount</i></small>|[roles/resourcemanager.projectIamAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectIamAdmin) <code></code><br>organizations/[org_id #0]/roles/serviceProjectNetworkAdmin <br>[roles/dns.admin](https://cloud.google.com/iam/docs/understanding-roles#dns.admin) |

View File

@ -0,0 +1,262 @@
# Networking
This stage sets up the shared network infrastructure for the whole organization. It implements a single shared VPC per environment, where each environment is independently connected to the on-premise environment, to maintain a fully separated routing domain on GCP.
While no communication between environment is implemented on this design, that can be achieved with a number of different options:
- [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering) - which is not recommended as it would effectively create a full line of sight between workloads belonging to different environments
- [VPN HA](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) tunnels between environments, exchanging a subset of well-defined routes.
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) connecting the different environments, allowing the use of NVAs to enforce networking policies.
The following diagram illustrates the high-level design, and should be used as a reference for the following sections. The final number of subnets, and their IP addressing design will of course depend on customer-specific requirements, and can be easily changed via variables or external data files without having to edit the actual code.
<p align="center">
<img src="diagram.svg" alt="Networking diagram">
</p>
## Design overview and choices
### VPC design
This architecture creates one VPC for each environment, each in its respective project. Each VPC hosts external connectivity and shared services solely serving its own environment.
As each environment is fully independent, this design trivialises the creation of new environments.
### External connectivity
External connectivity to on-prem is implemented here via HA VPN (two tunnels per region), as this is the minimum common denominator often used directly, or as a stop-gap solution to validate routing and transfer data, while waiting for [interconnects](https://cloud.google.com/network-connectivity/docs/interconnect) to be provisioned.
Connectivity to additional on-prem sites or other cloud providers should be implemented in a similar fashion, via VPN tunnels or interconnects on each of the environment VPCs, sharing the same regional router.
### IP ranges, subnetting, routing
Minimizing the number of routes (and subnets) in use on the cloud environment is an important consideration, as it simplifies management and avoids hitting [Cloud Router](https://cloud.google.com/network-connectivity/docs/router/quotas) and [VPC](https://cloud.google.com/vpc/docs/quota) quotas and limits. For this reason, we recommend careful planning of the IP space used in your cloud environment, to be able to use large IP CIDR blocks in routes whenever possible.
This stage uses a dedicated /16 block (which should of course be sized to your needs) shared by all regions and environments, and subnets created in each VPC derive their ranges from their relevant block.
Each VPC also defines and reserves two "special" CIDR ranges dedicated to [PSA (Private Service Access)](https://cloud.google.com/vpc/docs/private-services-access) and [Internal HTTPs Load Balancers (L7ILB)](https://cloud.google.com/load-balancing/docs/l7-internal).
Routes in GCP are either automatically created for VPC subnets, manually created via static routes, or dynamically programmed by [Cloud Routers](https://cloud.google.com/network-connectivity/docs/router#docs) via BGP sessions, which can be configured to advertise VPC ranges, and/or custom ranges via custom advertisements.
In this setup:
- routes between multiple subnets within the same VPC are automatically programmed by GCP
- on-premises is connected to each environment VPC and dynamically exchanges BGP routes with GCP using HA VPN
### Internet egress
The path of least resistance for Internet egress is using Cloud NAT, and that is what's implemented in this setup, with a NAT gateway configured for each VPC.
Several other scenarios are possible of course, with varying degrees of complexity:
- a forward proxy, with optional URL filters
- a default route to on-prem to leverage existing egress infrastructure
- a full-fledged perimeter firewall to control egress and implement additional security features like IPS
Future pluggable modules will allow to easily experiment, or deploy the above scenarios.
### VPC and Hierarchical Firewall
The GCP Firewall is a stateful, distributed feature that allows the creation of L4 policies, either via VPC-level rules or more recently via hierarchical policies applied on the resource hierarchy (organization, folders).
The current setup adopts both firewall types, and uses hierarchical rules on the Networking folder for common ingress rules (egress is open by default), e.g. from health check or IAP forwarders ranges, and VPC rules for the environment or workload-level ingress.
Rules and policies are defined in simple YAML files, described below.
### DNS
DNS often goes hand in hand with networking, especially on GCP where Cloud DNS zones and policies are associated at the VPC level. This setup implements both DNS flows:
- on-prem to cloud via private zones for cloud-managed domains, and an [inbound policy](https://cloud.google.com/dns/docs/server-policies-overview#dns-server-policy-in) used as forwarding target or via delegation (requires some extra configuration) from on-prem DNS resolvers
- cloud to on-prem via forwarding zones for the on-prem managed domains
To complete the configuration, the 35.199.192.0/19 range should be routed on the VPN tunnels from on-prem, and the following names configured for DNS forwarding to cloud:
- `private.googleapis.com`
- `restricted.googleapis.com`
- `gcp.example.com` (used as a placeholder)
From cloud, the `example.com` domain (used as a placeholder) is forwarded to on-prem.
This configuration is battle-tested, and flexible enough to lend itself to simple modifications without subverting its design, for example by forwarding and peering root zones to bypass Cloud DNS external resolution.
## How to run this stage
This stage is meant to be executed after the [resman](../01-resman) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured in the [bootstrap](../00-bootstrap) stage.
It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the previous stages for the environmental requirements.
Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration.
### Providers configuration
The default way of making sure you have the right permissions, is to use the identity of the service account pre-created for this stage during the [resource management](../01-resman) stage, and that you are a member of the group that can impersonate it via provider-level configuration (`gcp-devops` or `organization-admins`).
To simplify setup, the previous stage pre-configures a valid providers file in its output, and optionally writes it to a local file if the `outputs_location` variable is set to a valid path.
If you have set a valid value for `outputs_location` in the bootstrap stage, simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
```bash
# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/providers/02-networking-providers.tf .
```
If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage's outputs:
```bash
cd ../01-resman
terraform output -json providers | jq -r '.["02-networking"]' \
> ../02-networking/providers.tf
```
### Variable configuration
There are two broad sets of variables you will need to fill in:
- variables shared by other stages (org id, billing account id, etc.), or derived from a resource managed by a different stage (folder id, automation project id, etc.)
- variables specific to resources managed by this stage
To avoid the tedious job of filling in the first group of variables with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files.
If you have set a valid value for `outputs_location` in the bootstrap and in the resman stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's folder in the path you specified, where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is available:
```bash
# `outputs_location` is set to `~/fast-config`
ln -s ../../configs/example/02-networking/terraform-bootstrap.auto.tfvars.json
ln -s ../../configs/example/02-networking/terraform-resman.auto.tfvars.json
# also copy the tfvars file used for the bootstrap stage
cp ../00-bootstrap/terraform.tfvars .
```
A second set of variables is specific to this stage, they are all optional so if you need to customize them, add them to the file copied from bootstrap.
Please refer to the [Variables](#variables) table below for a map of the variable origins, and to the sections below on how to adapt this stage to your networking configuration.
### VPCs
VPCs are defined in separate files, one for each of `prod` and `dev`.
Each file contains the same resources, described in the following paragraphs.
The **project** ([`project`](../../../modules/project)) contains the VPC, and enables the required APIs and sets itself as a "[host project](https://cloud.google.com/vpc/docs/shared-vpc)".
The **VPC** ([`net-vpc`](../../../modules/net-vpc)) manages the DNS inbound policy, explicit routes for `{private,restricted}.googleapis.com`, and its **subnets**. Subnets are created leveraging a "resource factory" paradigm, where the configuration is separated from the module that implements it, and stored in a well-structured file. To add a new subnet, simply create a new file in the `data_folder` directory defined in the module, following the examples found in the [Fabric `net-vpc` documentation](../../../modules/net-vpc#subnet-factory). Sample subnets are shipped in [data/subnets](./data/subnets), and can be easily customised to fit your needs.
Subnets for [L7 ILBs](https://cloud.google.com/load-balancing/docs/l7-internal/proxy-only-subnets) are handled differently, and defined in variable `l7ilb_subnets`, while ranges for [PSA](https://cloud.google.com/vpc/docs/configure-private-services-access#allocating-range) are configured by variable `psa_ranges` - such variables are consumed by spoke VPCs.
**Cloud NAT** ([`net-cloudnat`](../../../modules/net-cloudnat)) manages the networking infrastructure required to enable internet egress.
### VPNs
Connectivity to on-prem is implemented with HA VPN ([`net-vpn`](../../../modules/net-vpn-ha)) and defined in `vpn-onprem-{dev,prod}.tf`. The files provisionally implement each a single logical connection between onprem and environment at `europe-west1`, and the relevant parameters for its configuration are found in variable `vpn_onprem_configs`.
### Routing and BGP
Each VPC network ([`net-vpc`](../../../modules/net-vpc)) manages a separate routing table, which can define static routes (e.g. to private.googleapis.com) and receives dynamic routes from BGP sessions established with neighbor networks (e.g. from onprem).
Static routes are defined in `net-*.tf` files, in the `routes` section of each `net-vpc` module.
### Firewall
**VPC firewall rules** ([`net-vpc-firewall`](../../../modules/net-vpc-firewall)) are defined per-vpc on each `net-*.tf` file and leverage a resource factory to massively create rules.
To add a new firewall rule, create a new file or edit an existing one in the `data_folder` directory defined in the module `net-vpc-firewall`, following the examples of the "[Rules factory](../../../modules/net-vpc-firewall#rules-factory)" section of the module documentation. Sample firewall rules are shipped in [data/firewall-rules/dev](./data/firewall-rules/dev) and can be easily customised.
**Hierarchical firewall policies** ([`folder`](../../../modules/folder)) are defined in `main.tf`, and managed through a policy factory implemented by the `folder` module, which applies the defined hierarchical to the `Networking` folder, which contains all the core networking infrastructure. Policies are defined in the `rules_file` file - to define a new one simply use the instructions found on "[Firewall policy factory](../../../modules/organization#firewall-policy-factory)". Sample hierarchical firewall policies are shipped in [data/hierarchical-policy-rules.yaml](./data/hierarchical-policy-rules.yaml) and can be easily customised.
### DNS architecture
The DNS ([`dns`](../../../modules/dns)) infrastructure is defined in the respective `dns-xxx.tf` files.
Cloud DNS manages onprem forwarding and environment-specific zones (i.e. `dev.gcp.example.com` and `prod.gcp.example.com`).
#### Cloud to on-prem
Leveraging the forwarding zones defined on each environment, the cloud environment can resolve `in-addr.arpa.` and `onprem.example.com.` using the on-premises DNS infrastructure. Onprem resolvers IPs are set in variable `dns`.
DNS queries sent to the on-premises infrastructure come from the `35.199.192.0/19` source range, which is only accessible from within a VPC or networks connected to one.
When implementing this architecture, make sure you'll be able to route packets coming from the /19 range to the right environment (route to prod requests coming from prod and to dev for requests coming from dev). As an alternative, consider leveraging self-managed DNS resolvers (e.g. CoreDNS forwarders) on each environment.
#### On-prem to cloud
The [Inbound DNS Policy](https://cloud.google.com/dns/docs/server-policies-overview#dns-server-policy-in) defined on eachVPC automatically reserves the first available IP address on each created subnet (typically the third one in a CIDR) to expose the Cloud DNS service so that it can be consumed from outside of GCP.
### Private Google Access
[Private Google Access](https://cloud.google.com/vpc/docs/private-google-access) (or PGA) enables VMs and on-prem systems to consume Google APIs from within the Google network, and is already fully configured on this environment.
For PGA to work:
- Private Google Access should be enabled on the subnet. \
Subnets created by the `net-vpc` module are PGA-enabled by default.
- 199.36.153.4/30 (`restricted.googleapis.com`) and 199.36.153.8/30 (`private.googleapis.com`) should be routed from on-prem to VPC, and from there to the `default-internet-gateway`. \
Per variable `vpn_onprem_configs` such ranges are advertised to onprem - furthermore every VPC has explicit routes set in case the `0.0.0.0/0` route is changed.
- A private DNS zone for `googleapis.com` should be created and configured per [this article](https://cloud.google.com/vpc/docs/configure-private-google-access-hybrid#config-domain), as implemented in module `googleapis-private-zone` in `dns-xxx.tf`
### Preliminar activities
Before running `terraform apply` on this stage, make sure to adapt all of `variables.tf` to your needs, to update all reference to regions (e.g. `europe-west1` or `ew1`) in the whole directory to match your preferences.
If you're not using FAST, you'll also need to create a `providers.tf` file to configure the GCS backend and the service account to use to run the deployment.
You're now ready to run `terraform init` and `apply`.
### Post-deployment activities
- On-prem routers should be configured to advertise all relevant CIDRs to the GCP environments. To avoid hitting GCP quotas, we recomment aggregating routes as much as possible.
- On-prem routers should accept BGP sessions from their cloud peers.
- On-prem DNS servers should have forward zones for GCP-managed ones.
<!-- TFDOC OPTS files:1 show_extra:1 -->
<!-- BEGIN TFDOC -->
## Files
| name | description | modules | resources |
|---|---|---|---|
| [dns-dev.tf](./dns-dev.tf) | Development spoke DNS zones and peerings setup. | <code>dns</code> | |
| [dns-prod.tf](./dns-prod.tf) | Production spoke DNS zones and peerings setup. | <code>dns</code> | |
| [main.tf](./main.tf) | Networking folder and hierarchical policy. | <code>folder</code> | |
| [monitoring.tf](./monitoring.tf) | Network monitoring dashboards. | | <code>google_monitoring_dashboard</code> |
| [outputs.tf](./outputs.tf) | Module outputs. | | <code>google_storage_bucket_object</code> · <code>local_file</code> |
| [spoke-dev.tf](./spoke-dev.tf) | Dev spoke VPC and related resources. | <code>net-cloudnat</code> · <code>net-vpc</code> · <code>net-vpc-firewall</code> · <code>project</code> | <code>google_project_iam_binding</code> |
| [spoke-prod.tf](./spoke-prod.tf) | Production spoke VPC and related resources. | <code>net-cloudnat</code> · <code>net-vpc</code> · <code>net-vpc-firewall</code> · <code>project</code> | <code>google_project_iam_binding</code> |
| [test-resources.tf](./test-resources.tf) | Temporary instances for testing | <code>compute-vm</code> | |
| [variables.tf](./variables.tf) | Module variables. | | |
| [vpn-onprem-dev.tf](./vpn-onprem-dev.tf) | VPN between dev and onprem. | <code>net-vpn-ha</code> | |
| [vpn-onprem-prod.tf](./vpn-onprem-prod.tf) | VPN between prod and onprem. | <code>net-vpn-ha</code> | |
## Variables
| name | description | type | required | default | producer |
|---|---|:---:|:---:|:---:|:---:|
| [automation](variables.tf#L17) | Automation resources created by the bootstrap stage. | <code title="object&#40;&#123;&#10; outputs_bucket &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| [billing_account](variables.tf#L25) | Billing account id and organization id ('nnnnnnnn' or null). | <code title="object&#40;&#123;&#10; id &#61; string&#10; organization_id &#61; number&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| [folder_ids](variables.tf#L74) | Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created. | <code title="object&#40;&#123;&#10; networking &#61; string&#10; networking-dev &#61; string&#10; networking-prod &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>01-resman</code> |
| [organization](variables.tf#L102) | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; number&#10; customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>00-bootstrap</code> |
| [prefix](variables.tf#L118) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
| [custom_adv](variables.tf#L34) | Custom advertisement definitions in name => range format. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; cloud_dns &#61; &#34;35.199.192.0&#47;19&#34;&#10; gcp_all &#61; &#34;10.128.0.0&#47;16&#34;&#10; gcp_dev &#61; &#34;10.128.32.0&#47;19&#34;&#10; gcp_prod &#61; &#34;10.128.64.0&#47;19&#34;&#10; googleapis_private &#61; &#34;199.36.153.8&#47;30&#34;&#10; googleapis_restricted &#61; &#34;199.36.153.4&#47;30&#34;&#10; rfc_1918_10 &#61; &#34;10.0.0.0&#47;8&#34;&#10; rfc_1918_172 &#61; &#34;172.16.0.0&#47;12&#34;&#10; rfc_1918_192 &#61; &#34;192.168.0.0&#47;16&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [custom_roles](variables.tf#L50) | Custom roles defined at the org level, in key => id format. | <code title="object&#40;&#123;&#10; service_project_network_admin &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | <code>00-bootstrap</code> |
| [data_dir](variables.tf#L59) | Relative path for the folder storing configuration data for network resources. | <code>string</code> | | <code>&#34;data&#34;</code> | |
| [dns](variables.tf#L65) | Onprem DNS resolvers. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code title="&#123;&#10; prod &#61; &#91;&#34;10.0.1.1&#34;&#93;&#10; dev &#61; &#91;&#34;10.0.2.1&#34;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [l7ilb_subnets](variables.tf#L84) | Subnets used for L7 ILBs. | <code title="map&#40;list&#40;object&#40;&#123;&#10; ip_cidr_range &#61; string&#10; region &#61; string&#10;&#125;&#41;&#41;&#41;">map&#40;list&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;&#41;</code> | | <code title="&#123;&#10; prod &#61; &#91;&#10; &#123; ip_cidr_range &#61; &#34;10.128.92.0&#47;24&#34;, region &#61; &#34;europe-west1&#34; &#125;,&#10; &#123; ip_cidr_range &#61; &#34;10.128.93.0&#47;24&#34;, region &#61; &#34;europe-west4&#34; &#125;&#10; &#93;&#10; dev &#61; &#91;&#10; &#123; ip_cidr_range &#61; &#34;10.128.60.0&#47;24&#34;, region &#61; &#34;europe-west1&#34; &#125;,&#10; &#123; ip_cidr_range &#61; &#34;10.128.61.0&#47;24&#34;, region &#61; &#34;europe-west4&#34; &#125;&#10; &#93;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [outputs_location](variables.tf#L112) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | <code>string</code> | | <code>null</code> | |
| [psa_ranges](variables.tf#L129) | IP ranges used for Private Service Access (e.g. CloudSQL). | <code title="object&#40;&#123;&#10; dev &#61; object&#40;&#123;&#10; ranges &#61; map&#40;string&#41;&#10; routes &#61; object&#40;&#123;&#10; export &#61; bool&#10; import &#61; bool&#10; &#125;&#41;&#10; &#125;&#41;&#10; prod &#61; object&#40;&#123;&#10; ranges &#61; map&#40;string&#41;&#10; routes &#61; object&#40;&#123;&#10; export &#61; bool&#10; import &#61; bool&#10; &#125;&#41;&#10; &#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| [router_onprem_configs](variables.tf#L166) | Configurations for routers used for onprem connectivity. | <code title="map&#40;object&#40;&#123;&#10; adv &#61; object&#40;&#123;&#10; custom &#61; list&#40;string&#41;&#10; default &#61; bool&#10; &#125;&#41;&#10; asn &#61; number&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; prod-ew1 &#61; &#123;&#10; asn &#61; &#34;65533&#34;&#10; adv &#61; null&#10; &#125;&#10; dev-ew1 &#61; &#123;&#10; asn &#61; &#34;65534&#34;&#10; adv &#61; null&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [service_accounts](variables.tf#L189) | Automation service accounts in name => email format. | <code title="object&#40;&#123;&#10; data-platform-dev &#61; string&#10; data-platform-prod &#61; string&#10; project-factory-dev &#61; string&#10; project-factory-prod &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | <code>01-resman</code> |
| [vpn_onprem_configs](variables.tf#L201) | VPN gateway configuration for onprem interconnection. | <code title="map&#40;object&#40;&#123;&#10; adv &#61; object&#40;&#123;&#10; default &#61; bool&#10; custom &#61; list&#40;string&#41;&#10; &#125;&#41;&#10; peer_external_gateway &#61; object&#40;&#123;&#10; redundancy_type &#61; string&#10; interfaces &#61; list&#40;object&#40;&#123;&#10; id &#61; number&#10; ip_address &#61; string&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#10; tunnels &#61; list&#40;object&#40;&#123;&#10; peer_asn &#61; number&#10; peer_external_gateway_interface &#61; number&#10; secret &#61; string&#10; session_range &#61; string&#10; vpn_gateway_interface &#61; number&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; dev-ew1 &#61; &#123;&#10; adv &#61; &#123;&#10; default &#61; false&#10; custom &#61; &#91;&#10; &#34;cloud_dns&#34;, &#34;googleapis_private&#34;, &#34;googleapis_restricted&#34;, &#34;gcp_dev&#34;&#10; &#93;&#10; &#125;&#10; peer_external_gateway &#61; &#123;&#10; redundancy_type &#61; &#34;SINGLE_IP_INTERNALLY_REDUNDANT&#34;&#10; interfaces &#61; &#91;&#10; &#123; id &#61; 0, ip_address &#61; &#34;8.8.8.8&#34; &#125;,&#10; &#93;&#10; &#125;&#10; tunnels &#61; &#91;&#10; &#123;&#10; peer_asn &#61; 65544&#10; peer_external_gateway_interface &#61; 0&#10; secret &#61; &#34;foobar&#34;&#10; session_range &#61; &#34;169.254.1.0&#47;30&#34;&#10; vpn_gateway_interface &#61; 0&#10; &#125;,&#10; &#123;&#10; peer_asn &#61; 65544&#10; peer_external_gateway_interface &#61; 0&#10; secret &#61; &#34;foobar&#34;&#10; session_range &#61; &#34;169.254.1.4&#47;30&#34;&#10; vpn_gateway_interface &#61; 1&#10; &#125;&#10; &#93;&#10; &#125;&#10; prod-ew1 &#61; &#123;&#10; adv &#61; &#123;&#10; default &#61; false&#10; custom &#61; &#91;&#10; &#34;cloud_dns&#34;, &#34;googleapis_private&#34;, &#34;googleapis_restricted&#34;, &#34;gcp_prod&#34;&#10; &#93;&#10; &#125;&#10; peer_external_gateway &#61; &#123;&#10; redundancy_type &#61; &#34;SINGLE_IP_INTERNALLY_REDUNDANT&#34;&#10; interfaces &#61; &#91;&#10; &#123; id &#61; 0, ip_address &#61; &#34;8.8.8.8&#34; &#125;,&#10; &#93;&#10; &#125;&#10; tunnels &#61; &#91;&#10; &#123;&#10; peer_asn &#61; 65543&#10; peer_external_gateway_interface &#61; 0&#10; secret &#61; &#34;foobar&#34;&#10; session_range &#61; &#34;169.254.1.0&#47;30&#34;&#10; vpn_gateway_interface &#61; 0&#10; &#125;,&#10; &#123;&#10; peer_asn &#61; 65543&#10; peer_external_gateway_interface &#61; 0&#10; secret &#61; &#34;foobar&#34;&#10; session_range &#61; &#34;169.254.1.4&#47;30&#34;&#10; vpn_gateway_interface &#61; 1&#10; &#125;&#10; &#93;&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | |
## Outputs
| name | description | sensitive | consumers |
|---|---|:---:|---|
| [dev_cloud_dns_inbound_policy](outputs.tf#L59) | IP Addresses for Cloud DNS inbound policy for the dev environment. | | |
| [host_project_ids](outputs.tf#L69) | Network project ids. | | |
| [host_project_numbers](outputs.tf#L74) | Network project numbers. | | |
| [prod_cloud_dns_inbound_policy](outputs.tf#L64) | IP Addresses for Cloud DNS inbound policy for the prod environment. | | |
| [shared_vpc_self_links](outputs.tf#L79) | Shared VPC host projects. | | |
| [tfvars](outputs.tf#L98) | Terraform variables file for the following stages. | ✓ | |
| [vpn_gateway_endpoints](outputs.tf#L84) | External IP Addresses for the GCP VPN gateways. | | |
<!-- END TFDOC -->

View File

@ -0,0 +1,15 @@
# skip boilerplate check
healthchecks:
- 35.191.0.0/16
- 130.211.0.0/22
- 209.85.152.0/22
- 209.85.204.0/22
rfc1918:
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
onprem_probes:
- 10.255.255.254/32

View File

@ -0,0 +1,68 @@
{
"displayName": "Firewall Insights Monitoring",
"gridLayout": {
"columns": "2",
"widgets": [
{
"title": "Subnet Firewall Hit Counts",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"firewallinsights.googleapis.com/subnet/firewall_hit_count\" resource.type=\"gce_subnetwork\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "VM Firewall Hit Counts",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"firewallinsights.googleapis.com/vm/firewall_hit_count\" resource.type=\"gce_instance\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
}
]
}
}

View File

@ -0,0 +1,248 @@
{
"displayName": "VPN Monitoring",
"gridLayout": {
"columns": "2",
"widgets": [
{
"title": "Number of connections",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_MEAN"
},
"filter": "metric.type=\"vpn.googleapis.com/gateway/connections\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Tunnel established",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_MEAN"
},
"filter": "metric.type=\"vpn.googleapis.com/tunnel_established\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Received bytes",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/received_bytes_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "By"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Sent bytes",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/sent_bytes_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "By"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Received packets",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/received_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "{packets}"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Sent packets",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/sent_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "{packets}"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Incoming packets dropped",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/dropped_received_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Outgoing packets dropped",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/dropped_sent_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
}
]
}
}

View File

@ -0,0 +1,27 @@
# skip boilerplate check
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
direction: INGRESS
action: allow
sources: []
ranges: ["0.0.0.0/0"]
targets:
- composer-worker
use_service_accounts: false
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
direction: INGRESS
action: allow
sources: []
ranges: ["0.0.0.0/0"]
targets:
- dataflow
use_service_accounts: false
rules:
- protocol: tcp
ports: [12345, 12346]

View File

@ -0,0 +1,49 @@
# skip boilerplate check
allow-admins:
description: Access from the admin subnet to all subnets
direction: INGRESS
action: allow
priority: 1000
ranges:
- $rfc1918
ports:
all: []
target_resources: null
enable_logging: false
allow-healthchecks:
description: Enable HTTP and HTTPS healthchecks
direction: INGRESS
action: allow
priority: 1001
ranges:
- $healthchecks
ports:
tcp: ["80", "443"]
target_resources: null
enable_logging: false
allow-ssh-from-iap:
description: Enable SSH from IAP
direction: INGRESS
action: allow
priority: 1002
ranges:
- 35.235.240.0/20
ports:
tcp: ["22"]
target_resources: null
enable_logging: false
allow-icmp:
description: Enable ICMP
direction: INGRESS
action: allow
priority: 1003
ranges:
- 0.0.0.0/0
ports:
icmp: []
target_resources: null
enable_logging: false

View File

@ -0,0 +1,8 @@
# skip boilerplate check
region: europe-west1
description: Default subnet for dev Data Platform
ip_cidr_range: 10.128.48.0/24
secondary_ip_range:
pods: 100.128.48.0/20
services: 100.255.48.0/24

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.128.32.0/24
description: Default subnet for dev

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.128.64.0/24
description: Default subnet for prod

Binary file not shown.

After

Width:  |  Height:  |  Size: 123 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 256 KiB

View File

@ -0,0 +1,69 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Development spoke DNS zones and peerings setup.
# GCP-specific environment zone
module "dev-dns-private-zone" {
source = "../../../modules/dns"
project_id = module.dev-spoke-project.project_id
type = "private"
name = "dev-gcp-example-com"
domain = "dev.gcp.example.com."
client_networks = [module.dev-spoke-vpc.self_link]
recordsets = {
"A localhost" = { type = "A", ttl = 300, records = ["127.0.0.1"] }
}
}
module "dev-onprem-example-dns-forwarding" {
source = "../../../modules/dns"
project_id = module.dev-spoke-project.project_id
type = "forwarding"
name = "example-com"
domain = "onprem.example.com."
client_networks = [module.dev-spoke-vpc.self_link]
forwarders = { for ip in var.dns.dev : ip => null }
}
module "dev-reverse-10-dns-forwarding" {
source = "../../../modules/dns"
project_id = module.dev-spoke-project.project_id
type = "forwarding"
name = "root-reverse-10"
domain = "10.in-addr.arpa."
client_networks = [module.dev-spoke-vpc.self_link]
forwarders = { for ip in var.dns.dev : ip => null }
}
module "dev-googleapis-private-zone" {
source = "../../../modules/dns"
project_id = module.dev-spoke-project.project_id
type = "private"
name = "googleapis-com"
domain = "googleapis.com."
client_networks = [module.dev-spoke-vpc.self_link]
recordsets = {
"A private" = { type = "A", ttl = 300, records = [
"199.36.153.8", "199.36.153.9", "199.36.153.10", "199.36.153.11"
] }
"A restricted" = { type = "A", ttl = 300, records = [
"199.36.153.4", "199.36.153.5", "199.36.153.6", "199.36.153.7"
] }
"CNAME *" = { type = "CNAME", ttl = 300, records = ["private.googleapis.com."] }
}
}

View File

@ -0,0 +1,70 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Production spoke DNS zones and peerings setup.
# GCP-specific environment zone
module "prod-dns-private-zone" {
source = "../../../modules/dns"
project_id = module.prod-spoke-project.project_id
type = "private"
name = "prod-gcp-example-com"
domain = "prod.gcp.example.com."
client_networks = [module.prod-spoke-vpc.self_link]
recordsets = {
"A localhost" = { type = "A", ttl = 300, records = ["127.0.0.1"] }
}
}
module "prod-onprem-example-dns-forwarding" {
source = "../../../modules/dns"
project_id = module.prod-spoke-project.project_id
type = "forwarding"
name = "example-com"
domain = "onprem.example.com."
client_networks = [module.prod-spoke-vpc.self_link]
forwarders = { for ip in var.dns.prod : ip => null }
}
module "prod-reverse-10-dns-forwarding" {
source = "../../../modules/dns"
project_id = module.prod-spoke-project.project_id
type = "forwarding"
name = "root-reverse-10"
domain = "10.in-addr.arpa."
client_networks = [module.prod-spoke-vpc.self_link]
forwarders = { for ip in var.dns.prod : ip => null }
}
module "prod-googleapis-private-zone" {
source = "../../../modules/dns"
project_id = module.prod-spoke-project.project_id
type = "private"
name = "googleapis-com"
domain = "googleapis.com."
client_networks = [module.prod-spoke-vpc.self_link]
recordsets = {
"A private" = { type = "A", ttl = 300, records = [
"199.36.153.8", "199.36.153.9", "199.36.153.10", "199.36.153.11"
] }
"A restricted" = { type = "A", ttl = 300, records = [
"199.36.153.4", "199.36.153.5", "199.36.153.6", "199.36.153.7"
] }
"CNAME *" = { type = "CNAME", ttl = 300, records = ["private.googleapis.com."] }
}
}

View File

@ -0,0 +1,58 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Networking folder and hierarchical policy.
locals {
custom_roles = coalesce(var.custom_roles, {})
l7ilb_subnets = {
for env, v in var.l7ilb_subnets : env => [
for s in v : merge(s, {
active = true
name = "${env}-l7ilb-${s.region}"
})]
}
region_trigram = {
europe-west1 = "ew1"
europe-west3 = "ew3"
}
stage3_sas_delegated_grants = [
"roles/composer.sharedVpcAgent",
"roles/compute.networkUser",
"roles/container.hostServiceAgentUser",
"roles/vpcaccess.user",
]
service_accounts = {
for k, v in coalesce(var.service_accounts, {}) : k => "serviceAccount:${v}" if v != null
}
}
module "folder" {
source = "../../../modules/folder"
parent = "organizations/${var.organization.id}"
name = "Networking"
folder_create = var.folder_ids.networking == null
id = var.folder_ids.networking
firewall_policy_factory = {
cidr_file = "${var.data_dir}/cidrs.yaml"
policy_name = null
rules_file = "${var.data_dir}/hierarchical-policy-rules.yaml"
}
firewall_policy_association = {
factory-policy = "factory"
}
}

View File

@ -0,0 +1,38 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Network monitoring dashboards.
locals {
dashboard_path = "${var.data_dir}/dashboards"
dashboard_files = fileset(local.dashboard_path, "*.json")
dashboards = {
for filename in local.dashboard_files :
filename => "${local.dashboard_path}/${filename}"
}
}
resource "google_monitoring_dashboard" "dev-dashboard" {
for_each = local.dashboards
project = module.dev-spoke-project.project_id
dashboard_json = file(each.value)
}
resource "google_monitoring_dashboard" "prod-dashboard" {
for_each = local.dashboards
project = module.prod-spoke-project.project_id
dashboard_json = file(each.value)
}

View File

@ -0,0 +1,102 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
host_project_ids = {
dev-spoke-0 = module.dev-spoke-project.project_id
prod-spoke-0 = module.prod-spoke-project.project_id
}
host_project_numbers = {
dev-spoke-0 = module.dev-spoke-project.number
prod-spoke-0 = module.prod-spoke-project.number
}
subnet_self_links = {
dev-spoke-0 = module.dev-spoke-vpc.subnet_self_links
prod-spoke-0 = module.prod-spoke-vpc.subnet_self_links
}
tfvars = {
host_project_ids = local.host_project_ids
host_project_numbers = local.host_project_numbers
subnet_self_links = local.subnet_self_links
vpc_self_links = local.vpc_self_links
}
vpc_self_links = {
dev-spoke-0 = module.dev-spoke-vpc.self_link
prod-spoke-0 = module.prod-spoke-vpc.self_link
}
}
# generate tfvars file for subsequent stages
resource "local_file" "tfvars" {
for_each = var.outputs_location == null ? {} : { 1 = 1 }
file_permission = "0644"
filename = "${pathexpand(var.outputs_location)}/tfvars/02-networking.auto.tfvars.json"
content = jsonencode(local.tfvars)
}
resource "google_storage_bucket_object" "tfvars" {
bucket = var.automation.outputs_bucket
name = "tfvars/02-networking.auto.tfvars.json"
content = jsonencode(local.tfvars)
}
# outputs
output "dev_cloud_dns_inbound_policy" {
description = "IP Addresses for Cloud DNS inbound policy for the dev environment."
value = [for s in module.dev-spoke-vpc.subnets : cidrhost(s.ip_cidr_range, 2)]
}
output "prod_cloud_dns_inbound_policy" {
description = "IP Addresses for Cloud DNS inbound policy for the prod environment."
value = [for s in module.prod-spoke-vpc.subnets : cidrhost(s.ip_cidr_range, 2)]
}
output "host_project_ids" {
description = "Network project ids."
value = local.host_project_ids
}
output "host_project_numbers" {
description = "Network project numbers."
value = local.host_project_numbers
}
output "shared_vpc_self_links" {
description = "Shared VPC host projects."
value = local.vpc_self_links
}
output "vpn_gateway_endpoints" {
description = "External IP Addresses for the GCP VPN gateways."
value = local.enable_onprem_vpn == false ? null : {
dev-onprem-ew1 = {
for v in module.dev-to-onprem-ew1-vpn[0].gateway.vpn_interfaces :
v.id => v.ip_address
}
prod-onprem-ew1 = {
for v in module.prod-to-onprem-ew1-vpn[0].gateway.vpn_interfaces :
v.id => v.ip_address
}
}
}
output "tfvars" {
description = "Terraform variables file for the following stages."
sensitive = true
value = local.tfvars
}

View File

@ -0,0 +1,112 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Dev spoke VPC and related resources.
module "dev-spoke-project" {
source = "../../../modules/project"
billing_account = var.billing_account.id
name = "dev-net-spoke-0"
parent = var.folder_ids.networking-dev
prefix = var.prefix
services = [
"container.googleapis.com",
"compute.googleapis.com",
"dns.googleapis.com",
"iap.googleapis.com",
"networkmanagement.googleapis.com",
"servicenetworking.googleapis.com",
"stackdriver.googleapis.com",
]
shared_vpc_host_config = {
enabled = true
service_projects = []
}
iam = {
"roles/dns.admin" = compact([
try(local.service_accounts.project-factory-dev, null)
])
}
}
module "dev-spoke-vpc" {
source = "../../../modules/net-vpc"
project_id = module.dev-spoke-project.project_id
name = "dev-spoke-0"
mtu = 1500
data_folder = "${var.data_dir}/subnets/dev"
psa_config = try(var.psa_ranges.dev, null)
subnets_proxy_only = local.l7ilb_subnets.dev
# set explicit routes for googleapis in case the default route is deleted
routes = {
private-googleapis = {
dest_range = "199.36.153.8/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
restricted-googleapis = {
dest_range = "199.36.153.4/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
}
}
module "dev-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/dev"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
}
module "dev-spoke-cloudnat" {
for_each = toset(values(module.dev-spoke-vpc.subnet_regions))
source = "../../../modules/net-cloudnat"
project_id = module.dev-spoke-project.project_id
region = each.value
name = "dev-nat-${local.region_trigram[each.value]}"
router_create = true
router_network = module.dev-spoke-vpc.name
router_asn = 4200001024
logging_filter = "ERRORS_ONLY"
}
# Create delegated grants for stage3 service accounts
resource "google_project_iam_binding" "dev_spoke_project_iam_delegated" {
project = module.dev-spoke-project.project_id
role = "roles/resourcemanager.projectIamAdmin"
members = compact([
try(local.service_accounts.data-platform-dev, null),
try(local.service_accounts.project-factory-dev, null),
])
condition {
title = "dev_stage3_sa_delegated_grants"
description = "Development host project delegated grants."
expression = format(
"api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
join(",", formatlist("'%s'", local.stage3_sas_delegated_grants))
)
}
}

View File

@ -0,0 +1,112 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Production spoke VPC and related resources.
module "prod-spoke-project" {
source = "../../../modules/project"
billing_account = var.billing_account.id
name = "prod-net-spoke-0"
parent = var.folder_ids.networking-prod
prefix = var.prefix
services = [
"container.googleapis.com",
"compute.googleapis.com",
"dns.googleapis.com",
"iap.googleapis.com",
"networkmanagement.googleapis.com",
"servicenetworking.googleapis.com",
"stackdriver.googleapis.com",
]
shared_vpc_host_config = {
enabled = true
service_projects = []
}
iam = {
"roles/dns.admin" = compact([
try(local.service_accounts.project-factory-prod, null)
])
}
}
module "prod-spoke-vpc" {
source = "../../../modules/net-vpc"
project_id = module.prod-spoke-project.project_id
name = "prod-spoke-0"
mtu = 1500
data_folder = "${var.data_dir}/subnets/prod"
psa_config = try(var.psa_ranges.prod, null)
subnets_proxy_only = local.l7ilb_subnets.prod
# set explicit routes for googleapis in case the default route is deleted
routes = {
private-googleapis = {
dest_range = "199.36.153.8/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
restricted-googleapis = {
dest_range = "199.36.153.4/30"
priority = 1000
tags = []
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
}
}
module "prod-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
admin_ranges = []
http_source_ranges = []
https_source_ranges = []
ssh_source_ranges = []
data_folder = "${var.data_dir}/firewall-rules/prod"
cidr_template_file = "${var.data_dir}/cidrs.yaml"
}
module "prod-spoke-cloudnat" {
for_each = toset(values(module.prod-spoke-vpc.subnet_regions))
source = "../../../modules/net-cloudnat"
project_id = module.prod-spoke-project.project_id
region = each.value
name = "prod-nat-${local.region_trigram[each.value]}"
router_create = true
router_network = module.prod-spoke-vpc.name
router_asn = 4200001024
logging_filter = "ERRORS_ONLY"
}
# Create delegated grants for stage3 service accounts
resource "google_project_iam_binding" "prod_spoke_project_iam_delegated" {
project = module.prod-spoke-project.project_id
role = "roles/resourcemanager.projectIamAdmin"
members = compact([
try(local.service_accounts.data-platform-prod, null),
try(local.service_accounts.project-factory-prod, null),
])
condition {
title = "prod_stage3_sa_delegated_grants"
description = "Production host project delegated grants."
expression = format(
"api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
join(",", formatlist("'%s'", local.stage3_sas_delegated_grants))
)
}
}

View File

@ -0,0 +1,83 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Temporary instances for testing
# module "test-vm-dev-0" {
# source = "../../../modules/compute-vm"
# project_id = module.dev-spoke-project.project_id
# zone = "europe-west1-b"
# name = "test-vm-0"
# network_interfaces = [{
# network = module.dev-spoke-vpc.self_link
# # change the subnet name to match the values you are actually using
# subnetwork = module.dev-spoke-vpc.subnet_self_links["europe-west1/dev-default-ew1"]
# alias_ips = {}
# nat = false
# addresses = null
# }]
# tags = ["ssh"]
# service_account_create = true
# boot_disk = {
# image = "projects/debian-cloud/global/images/family/debian-10"
# type = "pd-balanced"
# size = 10
# }
# options = {
# allow_stopping_for_update = true
# deletion_protection = false
# spot = true
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }
# module "test-vm-prod-0" {
# source = "../../../modules/compute-vm"
# project_id = module.prod-spoke-project.project_id
# zone = "europe-west1-b"
# name = "test-vm-0"
# network_interfaces = [{
# network = module.prod-spoke-vpc.self_link
# # change the subnet name to match the values you are actually using
# subnetwork = module.prod-spoke-vpc.subnet_self_links["europe-west1/prod-default-ew1"]
# alias_ips = {}
# nat = false
# addresses = null
# }]
# tags = ["ssh"]
# service_account_create = true
# boot_disk = {
# image = "projects/debian-cloud/global/images/family/debian-10"
# type = "pd-balanced"
# size = 10
# }
# options = {
# allow_stopping_for_update = true
# deletion_protection = false
# spot = true
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }

View File

@ -0,0 +1,285 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "automation" {
# tfdoc:variable:source 00-bootstrap
description = "Automation resources created by the bootstrap stage."
type = object({
outputs_bucket = string
})
}
variable "billing_account" {
# tfdoc:variable:source 00-bootstrap
description = "Billing account id and organization id ('nnnnnnnn' or null)."
type = object({
id = string
organization_id = number
})
}
variable "custom_adv" {
description = "Custom advertisement definitions in name => range format."
type = map(string)
default = {
cloud_dns = "35.199.192.0/19"
gcp_all = "10.128.0.0/16"
gcp_dev = "10.128.32.0/19"
gcp_prod = "10.128.64.0/19"
googleapis_private = "199.36.153.8/30"
googleapis_restricted = "199.36.153.4/30"
rfc_1918_10 = "10.0.0.0/8"
rfc_1918_172 = "172.16.0.0/12"
rfc_1918_192 = "192.168.0.0/16"
}
}
variable "custom_roles" {
# tfdoc:variable:source 00-bootstrap
description = "Custom roles defined at the org level, in key => id format."
type = object({
service_project_network_admin = string
})
default = null
}
variable "data_dir" {
description = "Relative path for the folder storing configuration data for network resources."
type = string
default = "data"
}
variable "dns" {
description = "Onprem DNS resolvers."
type = map(list(string))
default = {
prod = ["10.0.1.1"]
dev = ["10.0.2.1"]
}
}
variable "folder_ids" {
# tfdoc:variable:source 01-resman
description = "Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created."
type = object({
networking = string
networking-dev = string
networking-prod = string
})
}
variable "l7ilb_subnets" {
description = "Subnets used for L7 ILBs."
type = map(list(object({
ip_cidr_range = string
region = string
})))
default = {
prod = [
{ ip_cidr_range = "10.128.92.0/24", region = "europe-west1" },
{ ip_cidr_range = "10.128.93.0/24", region = "europe-west4" }
]
dev = [
{ ip_cidr_range = "10.128.60.0/24", region = "europe-west1" },
{ ip_cidr_range = "10.128.61.0/24", region = "europe-west4" }
]
}
}
variable "organization" {
# tfdoc:variable:source 00-bootstrap
description = "Organization details."
type = object({
domain = string
id = number
customer_id = string
})
}
variable "outputs_location" {
description = "Path where providers and tfvars files for the following stages are written. Leave empty to disable."
type = string
default = null
}
variable "prefix" {
# tfdoc:variable:source 00-bootstrap
description = "Prefix used for resources that need unique names. Use 9 characters or less."
type = string
validation {
condition = try(length(var.prefix), 0) < 10
error_message = "Use a maximum of 9 characters for prefix."
}
}
variable "psa_ranges" {
description = "IP ranges used for Private Service Access (e.g. CloudSQL)."
type = object({
dev = object({
ranges = map(string)
routes = object({
export = bool
import = bool
})
})
prod = object({
ranges = map(string)
routes = object({
export = bool
import = bool
})
})
})
default = null
# default = {
# dev = {
# ranges = {
# cloudsql-mysql = "10.128.62.0/24"
# cloudsql-sqlserver = "10.128.63.0/24"
# }
# routes = null
# }
# prod = {
# ranges = {
# cloudsql-mysql = "10.128.94.0/24"
# cloudsql-sqlserver = "10.128.95.0/24"
# }
# routes = null
# }
# }
}
variable "router_onprem_configs" {
description = "Configurations for routers used for onprem connectivity."
type = map(object({
adv = object({
custom = list(string)
default = bool
})
asn = number
}))
default = {
prod-ew1 = {
asn = "65533"
adv = null
# adv = { default = false, custom = [] }
}
dev-ew1 = {
asn = "65534"
adv = null
# adv = { default = false, custom = [] }
}
}
}
variable "service_accounts" {
# tfdoc:variable:source 01-resman
description = "Automation service accounts in name => email format."
type = object({
data-platform-dev = string
data-platform-prod = string
project-factory-dev = string
project-factory-prod = string
})
default = null
}
variable "vpn_onprem_configs" {
description = "VPN gateway configuration for onprem interconnection."
type = map(object({
adv = object({
default = bool
custom = list(string)
})
peer_external_gateway = object({
redundancy_type = string
interfaces = list(object({
id = number
ip_address = string
}))
})
tunnels = list(object({
peer_asn = number
peer_external_gateway_interface = number
secret = string
session_range = string
vpn_gateway_interface = number
}))
}))
default = {
dev-ew1 = {
adv = {
default = false
custom = [
"cloud_dns", "googleapis_private", "googleapis_restricted", "gcp_dev"
]
}
peer_external_gateway = {
redundancy_type = "SINGLE_IP_INTERNALLY_REDUNDANT"
interfaces = [
{ id = 0, ip_address = "8.8.8.8" },
]
}
tunnels = [
{
peer_asn = 65544
peer_external_gateway_interface = 0
secret = "foobar"
session_range = "169.254.1.0/30"
vpn_gateway_interface = 0
},
{
peer_asn = 65544
peer_external_gateway_interface = 0
secret = "foobar"
session_range = "169.254.1.4/30"
vpn_gateway_interface = 1
}
]
}
prod-ew1 = {
adv = {
default = false
custom = [
"cloud_dns", "googleapis_private", "googleapis_restricted", "gcp_prod"
]
}
peer_external_gateway = {
redundancy_type = "SINGLE_IP_INTERNALLY_REDUNDANT"
interfaces = [
{ id = 0, ip_address = "8.8.8.8" },
]
}
tunnels = [
{
peer_asn = 65543
peer_external_gateway_interface = 0
secret = "foobar"
session_range = "169.254.1.0/30"
vpn_gateway_interface = 0
},
{
peer_asn = 65543
peer_external_gateway_interface = 0
secret = "foobar"
session_range = "169.254.1.4/30"
vpn_gateway_interface = 1
}
]
}
}
}

View File

@ -0,0 +1,62 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description VPN between dev and onprem.
locals {
enable_onprem_vpn = var.vpn_onprem_configs != null
bgp_peer_options_onprem = local.enable_onprem_vpn == false ? null : {
for k, v in var.vpn_onprem_configs :
k => v.adv == null ? null : {
advertise_groups = []
advertise_ip_ranges = {
for adv in(v.adv == null ? [] : v.adv.custom) :
var.custom_adv[adv] => adv
}
advertise_mode = try(v.adv.default, false) ? "DEFAULT" : "CUSTOM"
route_priority = null
}
}
}
module "dev-to-onprem-ew1-vpn" {
count = local.enable_onprem_vpn ? 1 : 0
source = "../../../modules/net-vpn-ha"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.self_link
region = "europe-west1"
name = "vpn-to-onprem-ew1"
router_create = true
router_name = "dev-onprem-vpn-ew1"
router_asn = var.router_onprem_configs.dev-ew1.asn
peer_external_gateway = var.vpn_onprem_configs.dev-ew1.peer_external_gateway
tunnels = {
for t in var.vpn_onprem_configs.dev-ew1.tunnels :
"remote-${t.vpn_gateway_interface}-${t.peer_external_gateway_interface}" => {
bgp_peer = {
address = cidrhost(t.session_range, 1)
asn = t.peer_asn
}
bgp_peer_options = local.bgp_peer_options_onprem.dev-ew1
bgp_session_range = "${cidrhost(t.session_range, 2)}/30"
ike_version = 2
peer_external_gateway_interface = t.peer_external_gateway_interface
router = null
shared_secret = t.secret
vpn_gateway_interface = t.vpn_gateway_interface
}
}
}

View File

@ -0,0 +1,46 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description VPN between prod and onprem.
module "prod-to-onprem-ew1-vpn" {
count = local.enable_onprem_vpn ? 1 : 0
source = "../../../modules/net-vpn-ha"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.self_link
region = "europe-west1"
name = "vpn-to-onprem-ew1"
router_create = true
router_name = "prod-onprem-vpn-ew1"
router_asn = var.router_onprem_configs.prod-ew1.asn
peer_external_gateway = var.vpn_onprem_configs.prod-ew1.peer_external_gateway
tunnels = {
for t in var.vpn_onprem_configs.prod-ew1.tunnels :
"remote-${t.vpn_gateway_interface}-${t.peer_external_gateway_interface}" => {
bgp_peer = {
address = cidrhost(t.session_range, 1)
asn = t.peer_asn
}
bgp_peer_options = local.bgp_peer_options_onprem.prod-ew1
bgp_session_range = "${cidrhost(t.session_range, 2)}/30"
ike_version = 2
peer_external_gateway_interface = t.peer_external_gateway_interface
router = null
shared_secret = t.secret
vpn_gateway_interface = t.vpn_gateway_interface
}
}
}

View File

@ -0,0 +1,13 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

View File

@ -0,0 +1,47 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "stage" {
source = "../../../../../fast/stages/02-networking-separate-envs"
data_dir = "../../../../../fast/stages/02-networking-separate-envs/data/"
automation = {
outputs_bucket = "test"
}
billing_account = {
id = "000000-111111-222222"
organization_id = 123456789012
}
custom_roles = {
service_project_network_admin = "organizations/123456789012/roles/foo"
}
folder_ids = {
networking = null
networking-dev = null
networking-prod = null
}
service_accounts = {
data-platform-dev = "string"
data-platform-prod = "string"
project-factory-dev = "string"
project-factory-prod = "string"
}
organization = {
domain = "fast.example.com"
id = 123456789012
customer_id = "C00000000"
}
prefix = "fast2"
}

View File

@ -0,0 +1,20 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
def test_counts(recursive_e2e_plan_runner):
"Test stage."
num_modules, num_resources = recursive_e2e_plan_runner()
# TODO: to re-enable per-module resource count check print _, then test
assert num_modules > 0 and num_resources > 0