Add FAST stage 2-networking-e-nva-bgp (NVA+NCC)

Co-authored-by: Luca Prete <lucaprete@google.com>
Co-authored-by: Simone Bruzzechesse <bruzzechesse@google.com>
Co-authored-by: Simone Ruffilli <sruffilli@google.com>
This commit is contained in:
Luca Prete 2023-04-04 20:41:04 +02:00 committed by GitHub
parent 70aefc2ddb
commit a9cba47ce8
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
46 changed files with 3027 additions and 10 deletions

View File

@ -72,13 +72,13 @@ As mentioned initially, there are of course other ways to implement internal con
This is a summary of the main options:
- [HA VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) (implemented by [2-networking-vpn](../2-networking-b-vpn/))
- [HA VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) (implemented by [2-networking-b-vpn](../2-networking-b-vpn/))
- Pros: simple compatibility with GCP services that leverage peering internally, better control on routes, avoids peering groups shared quotas and limits
- Cons: additional cost, marginal increase in latency, requires multiple tunnels for full bandwidth
- [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering) (implemented here)
- Pros: no additional costs, full bandwidth with no configurations, no extra latency, total environment isolation
- Cons: no transitivity (e.g. to GKE masters, Cloud SQL, etc.), no selective exchange of routes, several quotas and limits shared between VPCs in a peering group
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) (implemented by [2-networking-nva](../2-networking-c-nva/))
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) (implemented by [2-networking-c-nva](../2-networking-c-nva/) and [2-networking-e-nva-bgp](../2-networking-e-nva-bgp/))
- Pros: additional security features (e.g. IPS), potentially better integration with on-prem systems by using the same vendor
- Cons: complex HA/failover setup, limited by VM bandwidth and scale, additional costs for VMs and licenses, out of band management of a critical cloud component

View File

@ -73,10 +73,10 @@ This is a summary of the main options:
- [HA VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) (implemented here)
- Pros: simple compatibility with GCP services that leverage peering internally, better control on routes, avoids peering groups shared quotas and limits
- Cons: additional cost, marginal increase in latency, requires multiple tunnels for full bandwidth
- [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering) (implemented by [2-networking-peering](../2-networking-a-peering/))
- [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering) (implemented by [2-networking-a-peering](../2-networking-a-peering/))
- Pros: no additional costs, full bandwidth with no configurations, no extra latency
- Cons: no transitivity (e.g. to GKE masters, Cloud SQL, etc.), no selective exchange of routes, several quotas and limits shared between VPCs in a peering group
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) (implemented by [2-networking-nva](../2-networking-c-nva/))
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) (implemented by [2-networking-c-nva](../2-networking-c-nva/) and [2-networking-e-nva-bgp](../2-networking-e-nva-bgp/))
- Pros: additional security features (e.g. IPS), potentially better integration with on-prem systems by using the same vendor
- Cons: complex HA/failover setup, limited by VM bandwidth and scale, additional costs for VMs and licenses, out of band management of a critical cloud component

View File

@ -110,7 +110,7 @@ This is an options summary:
- [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering) (used here to connect the trusted landing VPC with the spokes, also used by [02-networking-vpn](../2-networking-b-vpn/))
- Pros: no additional costs, full bandwidth with no configurations, no extra latency
- Cons: no transitivity (e.g. to GKE masters, Cloud SQL, etc.), no selective exchange of routes, several quotas and limits shared between VPCs in a peering group
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) (used here to connect the trusted landing and untrusted VPCs)
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) (used here to connect the trusted landing and untrusted VPCs) and multi-NIC appliances with NCC/BGP support implemented [here](../2-networking-e-nva-bgp/)
- Pros: provides additional security features (e.g. IPS), potentially better integration with on-prem systems by using the same vendor
- Cons: complex HA/failover setup, limited by VM bandwidth and scale, additional costs for VMs and licenses, out of band management of a critical cloud component
- [HA VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies)

View File

@ -6,7 +6,7 @@ While no communication between environment is implemented on this design, that c
- [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering) - which is not recommended as it would effectively create a full line of sight between workloads belonging to different environments
- [VPN HA](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) tunnels between environments, exchanging a subset of well-defined routes.
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) connecting the different environments, allowing the use of NVAs to enforce networking policies.
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) implemented in [2-networking-c-nva](../2-networking-c-nva) and [2-networking-e-nva-bgp](../2-networking-e-nva-bgp) connecting the different environments, allowing the use of NVAs to enforce networking policies.
The following diagram illustrates the high-level design, and should be used as a reference for the following sections. The final number of subnets, and their IP addressing design will of course depend on customer-specific requirements, and can be easily changed via variables or external data files without having to edit the actual code.

View File

@ -0,0 +1,540 @@
# Networking with Network Virtual Appliance
This stage sets up the shared network infrastructure for the whole organization.
It is designed for those who would like to leverage Network Virtual Appliances (NVAs) between trusted and untrusted areas of the network, for example for Intrusion Prevention System (IPS) purposes.
We use Network Connectivity Center Router Appliance (NCC-RA) and BGP appliances (for the sake of the demo [FRRouting](https://frrouting.org/)) to avoid different limitations that static routes bring (more [here](https://medium.com/google-cloud/gcp-routing-adventures-vol-2-enterprise-multi-regional-deployments-in-google-cloud-3968e9591d59)). The goals of this design include:
- Avoid using network tags to route traffic.
- Route traffic symmetrically to prevent breaking stateful NVAs.
- Avoid unnecessary NAT traffic at the NVAs.
- Avoid cross-regional traffic unless absolutely necessary for disaster recovery.
- Automatically send all traffic through the cross-regional NVAs if the ones in-region fail.
- Keep the trusted hub VPC unique, rather than having one per region.
It adopts the common “hub and spoke” reference design, which is well suited for multiple scenarios, and it offers several advantages versus other designs:
- the "trusted hub" VPC centralizes the external connectivity towards trusted network resources (e.g. on-prem, other cloud environments and the spokes), and it is ready to host cross-environment services like CI/CD, code repositories, and monitoring probes
- the "spoke" VPCs allow partitioning workloads (e.g. by environment like in this setup), while still retaining controlled access to central connectivity and services
- Shared VPCs -both in hub and spokes- split the management of the network resources into specific (host) projects, while still allowing them to be consumed from the workload (service) projects
- the design facilitates DNS centralization
Connectivity between the hub and the spokes is established via [VPC network peerings](https://cloud.google.com/vpc/docs/vpc-peering), which offer uncapped bandwidth, lower latencies, at no additional costs and with a very low management overhead. Different ways of implementing connectivity, and related some pros and cons, are discussed below.
The diagram shows the high-level design and it should be used as a reference throughout the following sections.
The final number of subnets, and their IP addressing will depend on the user-specific requirements. It can be easily changed via variables or external data files, without any need to edit the code.
<p align="center">
<img src="diagram.svg" alt="Networking diagram">
</p>
## Table of contents
- [Networking with Network Virtual Appliance](#networking-with-network-virtual-appliance)
- [Table of contents](#table-of-contents)
- [Design overview and choices](#design-overview-and-choices)
- [Multi-regional deployment](#multi-regional-deployment)
- [VPC design](#vpc-design)
- [External connectivity](#external-connectivity)
- [Internal connectivity](#internal-connectivity)
- [IP ranges, subnetting, routing](#ip-ranges-subnetting-routing)
- [Internet egress](#internet-egress)
- [VPC and Hierarchical Firewall](#vpc-and-hierarchical-firewall)
- [DNS](#dns)
- [Stage structure and files layout](#stage-structure-and-files-layout)
- [VPCs](#vpcs)
- [VPNs](#vpns)
- [Routing and BGP](#routing-and-bgp)
- [Firewall](#firewall)
- [DNS architecture](#dns-architecture)
- [Cloud environment](#cloud-environment)
- [Cloud to on-prem](#cloud-to-on-prem)
- [On-prem to cloud](#on-prem-to-cloud)
- [How to run this stage](#how-to-run-this-stage)
- [Provider and Terraform variables](#provider-and-terraform-variables)
- [Impersonating the automation service account](#impersonating-the-automation-service-account)
- [Variable configuration](#variable-configuration)
- [Using delayed billing association for projects](#using-delayed-billing-association-for-projects)
- [Running the stage](#running-the-stage)
- [Post-deployment activities](#post-deployment-activities)
- [Private Google Access](#private-google-access)
- [Customizations](#customizations)
- [Changing default regions](#changing-default-regions)
- [Configuring the VPNs to on prem](#configuring-the-vpns-to-on-prem)
- [Adding an environment](#adding-an-environment)
- [Files](#files)
- [Variables](#variables)
- [Outputs](#outputs)
## Design overview and choices
### Multi-regional deployment
The stage deploys the the infrastructure in two regions. By default, europe-west1 and europe-west4. Regional resources include NVAs and test VMs. This provides enough redundancy to be resilient to regional failures.
In case of a regional failure, the corresponding dynamic routes are withdrawn and traffic will failover in the secondary region.
### VPC design
The "landing zone" is divided into two VPC networks:
- the trusted VPC: the connectivity hub towards other trusted networks
- the untrusted VPC: the connectivity hub towards any other untrusted network
### NCC, NVAs and BGP sessions
The VPCs connect through two sets of sample NVA machines: one per region, each containing two instances. The appliances run [Contrainer-Optimized OS](https://cloud.google.com/container-optimized-os/docs) and a container with [FRRouting](https://frrouting.org/).
We levarage NCC-RA to allow the NVAs to establish BGP sessions with Cloud Routers in the untrusted and in the trusted VPCs. This allows Cloud Routers to advertise routes to the NVAs, and the NVAs to announce routes to the Cloud Router, so it can program them in the VPC.
Specifically, each NVA establishes two BGP sessions (for redundancy) with the the Cloud Router deployed in the VPC and in the subnet where the interface of that VM is attached to.
**Cloud Routers in the untrusted VPC advertise the default route (0.0.0.0/0) to the NVAs**. The NVAs advertise the route to the Cloud Routers in the trusted VPC. These dynamic routes are then imported through VPC peerings in the spokes.
**Cloud Routers in the trusted hub advertis to the NVAs** all the subnets of the trusted VPCs. This includes the regional subnets and the cross-regional subnets. The NVAs manipulate the route costs (MED) before advertising them to the Cloud Routers in the untrusted VPC. This is done to guarantee symmetric traffic paths (more [here](https://medium.com/google-cloud/gcp-routing-adventures-vol-2-enterprise-multi-regional-deployments-in-google-cloud-3968e9591d59)).
NVAs establish **extra BGP sessions with both cross-regional NVAs**. In this case, the NVAs advertise the regional trusted routes only. This allows cross-spoke (environment) traffic to remain also symmetric (more [here](https://medium.com/google-cloud/gcp-routing-adventures-vol-2-enterprise-multi-regional-deployments-in-google-cloud-3968e9591d59)). We set these routes to be exchanged at a lower cost than the one set for the other routes.
Following the majority of real-life deployments, **we assume appliances to be stateful and not able to synchronize sessions between multiple NVAs within the same regional cluster**. For this reason, within each regional cluster, NVAs announce the same routes with different MED costs (1 point of difference between the primary and the secondary). This will cause traffic to go deterministically through one applaiance at the time within each region. You can change this default behavior modifying the cost settings in the [NVAs BGP configuration file](./data/bgp-config.tftpl).
By default, the design assumes that:
- on-premise networks (and related resources) are considered trusted. As such, the VPNs connecting with on-premises are terminated in GCP, in the trusted VPC
- the public Internet is considered untrusted. As such [Cloud NAT](https://cloud.google.com/nat/docs/overview) has been deployed in the untrusted landing VPC only. Also, the default route is set to carry traffic from the trusted VPCs, through the NVAs, to the untrusted VPC.
- cross-spoke (environment) traffic and traffic from any untrusted network to any trusted network (and vice versa) pass through the NVAs.
- any traffic from a trusted network to an untrusted network (e.g. Internet) is natted by the NVAs. Users can configure further exclusions.
The trusted landing VPC acts as a hub: it bridges internal resources with the outside world and it hosts the shared services consumed by the spoke VPCs, connected to the hub thorugh VPC network peerings. Spokes are used to partition the environments. By default:
- one spoke VPC hosts the development environment resources
- one spoke VPC hosts the production environment resources
Each virtual network is a [shared VPC](https://cloud.google.com/vpc/docs/shared-vpc): shared VPCs are managed in dedicated *host projects* and shared with other *service projects* that consume the network resources.
Shared VPCs let organization administrators delegate administrative responsibilities, such as creating and managing instances, to Service Project Admins while maintaining centralized control over network resources like subnets, routes, and firewalls.
Users can easily extend the design to host additional environments, or adopt different logical mappings for the spokes (for example, in order to create a new spoke for each company entity). Adding spokes is trivial and it does not increase the design complexity. The steps to add more spokes are provided in the following sections.
In multi-organization scenarios, where production and non-production resources use different Cloud Identity and GCP organizations, the hub/landing VPC is usually part of the production organization. It establishes connections with the production spokes within the same organization, and with non-production spokes in a different organization.
### External connectivity
External connectivity to on-prem is implemented leveraging [Cloud HA VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) (two tunnels per region). This is what users normally deploy as a final solution, or to validate routing and to transfer data, while waiting for [interconnects](https://cloud.google.com/network-connectivity/docs/interconnect) to be provisioned.
Connectivity to additional on-prem sites or to other cloud providers should be implemented in a similar fashion, via VPN tunnels or interconnects, in the landing VPC (either trusted or untrusted, depending by the nature of the peers), sharing the same regional routers.
### Internal connectivity
Internal connectivity (e.g. between the trusted landing VPC and the spokes) is realized with VPC network peerings. As mentioned, there are other ways to implement connectivity. These can be easily retrofitted with minimal code changes, although they introduce additional considerations on service interoperability, quotas and management.
This is an options summary:
- [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering) (used here to connect the trusted landing VPC with the spokes, also used by [02-networking-vpn](../2-networking-b-vpn/))
- Pros: no additional costs, full bandwidth with no configurations, no extra latency
- Cons: no transitivity (e.g. to GKE masters, Cloud SQL, etc.), no selective exchange of routes, several quotas and limits shared between VPCs in a peering group
- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) (used here to connect the trusted landing and untrusted VPCs)
- Pros: provides additional security features (e.g. IPS), potentially better integration with on-prem systems by using the same vendor
- Cons: complex HA/failover setup, limited by VM bandwidth and scale, additional costs for VMs and licenses, out of band management of a critical cloud component
- [HA VPN](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies)
- Pros: simple compatibility with GCP services that leverage peering internally, better control on routes, avoids peering groups shared quotas and limits
- Cons: additional costs, marginal increase in latency, requires multiple tunnels for full bandwidth
### IP ranges, subnetting, routing
Minimizing the number of routes (and subnets) in the cloud environment is important, as it simplifies management and it avoids hitting [Cloud Router](https://cloud.google.com/network-connectivity/docs/router/quotas) and [VPC](https://cloud.google.com/vpc/docs/quota) quotas and limits. For this reason, we recommend to carefully plan the IP space used in your cloud environment. This allows the use of larger IP CIDR blocks in routes, whenever possible.
This stage uses a dedicated /16 block (10.128.0.0/16), which should be sized to the own needs. The subnets created in each VPC derive from this range.
The /16 block is evenly split in eight, smaller /19 blocks, assigned to different areas of the GCP network: *landing untrusted europe-west1*, *landing untrusted europe-west4*, *landing trusted europe-west1*, *landing untrusted europe-west4*, *development europe-west1*, *development europe-west4*, *production europe-west1*, *production europe-west4*.
The first /24 range in every area is allocated for a default subnet, which can be removed or modified as needed.
Spoke VPCs also define and reserve three "special" CIDR ranges, derived from the respective /19, dedicated to
- [PSA (Private Service Access)](https://cloud.google.com/vpc/docs/private-services-access):
- The second-last /24 range is used for PSA (CloudSQL, Postrgres)
- The third-last /24 range is used for PSA (CloudSQL, MySQL)
- [Internal HTTPs Load Balancers (L7ILB)](https://cloud.google.com/load-balancing/docs/l7-internal):
- The last /24 range
This is a summary of the subnets allocated by default in this setup:
| name | description | CIDR |
|---|---|---|
| landing-trusted-default-ew1 | Trusted landing subnet - europe-west1 | 10.128.64.0/24 |
| landing-trusted-default-ew4 | Trusted landing subnet - europe-west4 | 10.128.96.0/24 |
| landing-untrusted-default-ew1 | Untrusted landing subnet - europe-west1 | 10.128.0.0/24 |
| landing-untrusted-default-ew4 | Untrusted landing subnet - europe-west4 | 10.128.32.0/24 |
| dev-default-ew1 | Dev spoke subnet - europe-west1 | 10.128.128.0/24 |
| dev-default-ew1 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west1 | 10.128.157.0/24 |
| dev-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west1 | 10.128.158.0/24 |
| dev-default-ew1 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west1 | 10.128.92.0/24 |
| dev-default-ew4 | Dev spoke subnet - europe-west4 | 10.128.160.0/24 |
| dev-default-ew4 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west4 | 10.128.189.0/24 |
| dev-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west4 | 10.128.190.0/24 |
| dev-default-ew4 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west4 | 10.128.93.0/24 |
| prod-default-ew1 | Prod spoke subnet - europe-west1 | 10.128.192.0/24 |
| prod-default-ew1 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west1 | 10.128.221.0/24 |
| prod-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west1 | 10.128.253.0/24 |
| prod-default-ew1 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west1 | 10.128.60.0/24 |
| prod-default-ew4 | Prod spoke subnet - europe-west4 | 10.128.224.0/24 |
| prod-default-ew4 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west4 | 10.128.222.0/24 |
| prod-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west4 | 10.128.254.0/24 |
| prod-default-ew4 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west4 | 10.128.61.0/24 |
These subnets are advertised to on-premises as a whole /16 range (10.128.0.0/16).
Routes in GCP are either automatically created (for example, when a subnet is added to a VPC), manually created via static routes, dynamically exchanged through VPC peerings, or dynamically programmed by [Cloud Routers](https://cloud.google.com/network-connectivity/docs/router#docs) when a BGP session is established. BGP sessions can be configured to advertise VPC ranges, and/or custom ranges via custom advertisements.
In this setup:
- routes between multiple subnets within the same VPC are automatically exchanged by GCP
- the spokes and the trusted landing VPC exchange dynamic routes through VPC peerings
- on-premises is connected to the trusted landing VPC and it dynamically exchanges BGP routes with GCP (with the trusted VPC) using HA VPN
- the NVAs exchange dynamic routes using BGP with Cloud Routers in the untrusted VPC, Cloud Routers in the trusted VPC and cross-regional NVAs. This allows VMs in different environments and different regions to communicate.
The Cloud Routers (connected to the VPN gateways in the trusted VPC) are configured to exclude the default advertisement of VPC ranges and they only advertise their respective aggregate ranges, via custom advertisements. This greatly simplifies the routing configuration and avoids quota or limit issues, by keeping the number of routes small, instead of making it proportional to the subnets and to the secondary ranges in the VPCs.
### Internet egress
In this setup, Internet egress is realized through [Cloud NAT](https://cloud.google.com/nat/docs/overview), deployed in the untrusted landing VPC. This allows instances in all other VPCs to reach the Internet, passing through the NVAs (being the public Internet considered untrusted).
Several other scenarios are possible, with various degrees of complexity:
- deploy Cloud NAT in every VPC
- add forwarding proxies, with optional URL filters
- send Internet traffic to on-premises, so the existing egress infrastructure can be leveraged
Future pluggable modules will allow users to easily experiment with the above scenarios.
### VPC and Hierarchical Firewall
The GCP Firewall is a stateful, distributed feature that allows the creation of L4 policies, either via VPC-level rules or -more recently- via hierarchical policies, applied on the resource hierarchy (organization, folders).
The current setup adopts both firewall types. Hierarchical firewall rules are applied in the networking folder for common ingress rules (egress is open by default): for example, it allows the health checks and the IAP forwarders traffic to reach the VMs.
Rules and policies are defined in simple YAML files, described below.
### DNS
DNS goes hand in hand with networking, especially on GCP where Cloud DNS zones and policies are associated at the VPC level. This setup implements both DNS flows:
- on-prem to cloud via private zones for cloud-managed domains, and an [inbound policy](https://cloud.google.com/dns/docs/server-policies-overview#dns-server-policy-in) used as forwarding target or via delegation (requires some extra configuration) from on-prem DNS resolvers
- cloud to on-prem via forwarding zones for the on-prem managed domains
DNS configuration is further centralized by leveraging peering zones, so that
- the hub/landing Cloud DNS hosts configurations for on-prem forwarding, Google API domains, and the top-level private zone/s (e.g. gcp.example.com)
- the spokes Cloud DNS host configurations for the environment-specific domains (e.g. prod.gcp.example.com), which are bound to the hub/landing leveraging [cross-project binding](https://cloud.google.com/dns/docs/zones/zones-overview#cross-project_binding); a peering zone for the `.` (root) zone is then created on each spoke, delegating all DNS resolution to hub/landing.
- Private Google Access is enabled for a selection of the [supported domains](https://cloud.google.com/vpc/docs/configure-private-google-access#domain-options), namely
- `private.googleapis.com`
- `restricted.googleapis.com`
- `gcr.io`
- `packages.cloud.google.com`
- `pkg.dev`
- `pki.goog`
To complete the configuration, the 35.199.192.0/19 range should be routed to the VPN tunnels from on-premises, and the following names should be configured for DNS forwarding to cloud:
- `private.googleapis.com`
- `restricted.googleapis.com`
- `gcp.example.com` (used as a placeholder)
In GCP, a forwarding zone in the landing project is configured to forward queries to the placeholder domain `onprem.example.com` to on-premises.
This configuration is battle-tested, and flexible enough to lend itself to simple modifications without subverting its design.
## Stage structure and files layout
### VPCs
VPCs are defined in separate files, one for `landing` (trusted and untrusted), one for `prod` and one for `dev`.
These files contain different resources:
- **project** ([`projects`](../../../modules/project)): the "[host projects](https://cloud.google.com/vpc/docs/shared-vpc)" containing the VPCs and enabling the required APIs.
- **VPCs** ([`net-vpc`](../../../modules/net-vpc)): manage the subnets, the explicit routes for `{private,restricted}.googleapis.com` and the DNS inbound policy (for the trusted landing VPC). Subnets are created leveraging "resource factories": the configuration is separated from the module that implements it, and stored in a well-structured file. To add a new subnet, simply create a new file in the `data_folder` directory defined in the module, following the examples found in the [Fabric `net-vpc` documentation](../../../modules/net-vpc#subnet-factory). Sample subnets are shipped in [data/subnets](./data/subnets) and can be easily customized to fit users' needs.
Subnets for [L7 ILBs](https://cloud.google.com/load-balancing/docs/l7-internal/proxy-only-subnets) are handled differently, and defined in variable `l7ilb_subnets`, while ranges for [PSA](https://cloud.google.com/vpc/docs/configure-private-services-access#allocating-range) are configured by variable `psa_ranges` - such variables are consumed by spoke VPCs.
- **Cloud NAT** ([`net-cloudnat`](../../../modules/net-cloudnat)) (in the untrusted landing VPC only): it manages the networking infrastructure required to enable the Internet egress.
### VPNs
The connectivity between on-premises and GCP (the trusted landing VPC) is implemented with Cloud HA VPN ([`net-vpn`](../../../modules/net-vpn-ha)) and defined in [`vpn-onprem.tf`](./vpn-onprem.tf). The file implements a single logical connection between on-premises and the trusted landing VPC, both in `europe-west1` and `europe-west4`. The relevant parameters for its configuration are found in the variables `vpn_onprem_primary_config` and `vpn_onprem_secondary_config`.
### Routing and BGP
Each VPC network ([`net-vpc`](../../../modules/net-vpc)) manages a separate routing table, where you can define static routes (e.g. to private.googleapis.com) and receives dynamic routes through VPC peering and BGP sessions established with the neighbor networks (e.g. NCC routers, routers on-premises).
NCC/Cloud Router BGP settings are defined in `ncc.tf`.
NVA BGP settings are defined in the [bpg-config.tftpl template file](./data/bgp-config.tftpl).
The variable `ncc_asn` allows to change the Autonomous System Number (ASN) assigned to the untrusted VPC Cloud Routers, to the trusted VPC Cloud Routers and to the NVAs.
BGP sessions for trusted landing to on-premises are configured through the variable `vpn_onprem_configs`.
### Firewall
**VPC firewall rules** ([`net-vpc-firewall`](../../../modules/net-vpc-firewall)) are defined per-vpc on each `vpc-*.tf` file and leverage a resource factory to massively create rules.
To add a new firewall rule, create a new file or edit an existing one in the `data_folder` directory defined in the module `net-vpc-firewall`, following the examples of the "[Rules factory](../../../modules/net-vpc-firewall#rules-factory)" section of the module documentation. Sample firewall rules are shipped in [data/firewall-rules/landing-untrusted](./data/firewall-rules/landing-untrusted) and in [data/firewall-rules/landing-trusted](./data/firewall-rules/landing-trusted), and can be easily customized.
**Hierarchical firewall policies** ([`folder`](../../../modules/folder)) are defined in `main.tf`, and managed through a policy factory implemented by the `folder` module, which applies the defined hierarchical to the `Networking` folder, which contains all the core networking infrastructure. Policies are defined in the `rules_file` file - to define a new one simply use the instructions found on "[Firewall policy factory](../../../modules/organization#firewall-policy-factory)". Sample hierarchical firewall policies are shipped in [data/hierarchical-policy-rules.yaml](./data/hierarchical-policy-rules.yaml) and can be easily customized.
### DNS architecture
The DNS ([`dns`](../../../modules/dns)) infrastructure is defined in [`dns-*.tf`] files.
Cloud DNS manages onprem forwarding, the main GCP zone (in this example `gcp.example.com`) and environment-specific zones (i.e. `dev.gcp.example.com` and `prod.gcp.example.com`).
#### Cloud environment
The root DNS zone defined in the landing project acts as the source of truth for DNS within the Cloud environment. The resources defined in the spoke VPCs consume the landing DNS infrastructure through DNS peering (e.g. `prod-landing-root-dns-peering`).
The spokes can optionally define private zones (e.g. `prod-dns-private-zone`). Granting visibility both to the trusted and untrusted landing VPCs ensures that the whole cloud environment can query such zones.
#### Cloud to on-prem
Leveraging the forwarding zone defined in the landing project (e.g. `onprem-example-dns-forwarding` and `reverse-10-dns-forwarding`), the cloud environment can resolve `in-addr.arpa.` and `onprem.example.com.` using the on-premise DNS infrastructure. On-premise resolver IPs are set in the variable `dns.onprem`.
DNS queries sent to the on-premise infrastructure come from the `35.199.192.0/19` source range.
#### On-prem to cloud
The [Inbound DNS Policy](https://cloud.google.com/dns/docs/server-policies-overview#dns-server-policy-in) defined in the *trusted landing VPC module* ([`landing.tf`](./landing.tf)) automatically reserves the first available IP address on each subnet (typically the third one in a CIDR) to expose the Cloud DNS service, so that it can be consumed from outside of GCP.
## How to run this stage
This stage is meant to be executed after the [resource management](../1-resman) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured in the [bootstrap](../0-bootstrap) stage.
It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the previous stages for the environmental requirements.
Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration.
### Provider and Terraform variables
As all other FAST stages, the [mechanism used to pass variable values and pre-built provider files from one stage to the next](../0-bootstrap/README.md#output-files-and-cross-stage-variables) is also leveraged here.
The commands to link or copy the provider and terraform variable files can be easily derived from the `stage-links.sh` script in the FAST root folder, passing it a single argument with the local output files folder (if configured) or the GCS output bucket in the automation project (derived from stage 0 outputs). The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run.
```bash
../../stage-links.sh ~/fast-config
# copy and paste the following commands for '2-networking-a-peering'
ln -s ~/fast-config/providers/2-networking-providers.tf ./
ln -s ~/fast-config/tfvars/globals.auto.tfvars.json ./
ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json ./
ln -s ~/fast-config/tfvars/1-resman.auto.tfvars.json ./
```
```bash
../../stage-links.sh gs://xxx-prod-iac-core-outputs-0
# copy and paste the following commands for '2-networking-a-peering'
gcloud alpha storage cp gs://xxx-prod-iac-core-outputs-0/providers/2-networking-providers.tf ./
gcloud alpha storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/globals.auto.tfvars.json ./
gcloud alpha storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./
gcloud alpha storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./
```
### Impersonating the automation service account
The preconfigured provider file uses impersonation to run with this stage's automation service account's credentials. The `gcp-devops` and `organization-admins` groups have the necessary IAM bindings in place to do that, so make sure the current user is a member of one of those groups.
### Variable configuration
Variables in this stage -- like most other FAST stages -- are broadly divided into three separate sets:
- variables which refer to global values for the whole organization (org id, billing account id, prefix, etc.), which are pre-populated via the `globals.auto.tfvars.json` file linked or copied above
- variables which refer to resources managed by previous stage, which are prepopulated here via the `0-bootstrap.auto.tfvars.json` and `1-resman.auto.tfvars.json` files linked or copied above
- and finally variables that optionally control this stage's behaviour and customizations, and can to be set in a custom `terraform.tfvars` file
The latter set is explained in the [Customization](#customizations) sections below, and the full list can be found in the [Variables](#variables) table at the bottom of this document.
Note that the `outputs_location` variable is disabled by default, you need to explicitly set it in your `terraform.tfvars` file if you want output files to be generated by this stage. This is a sample `terraform.tfvars` that configures it, refer to the [bootstrap stage documentation](../0-bootstrap/README.md#output-files-and-cross-stage-variables) for more details:
```hcl
outputs_location = "~/fast-config"
```
### Using delayed billing association for projects
This configuration is possible but unsupported and only exists for development purposes, use at your own risk:
- temporarily switch `billing_account.id` to `null` in `globals.auto.tfvars.json`
- for each project resources in the project modules used in this stage (`dev-spoke-project`, `landing-project`, `prod-spoke-project`)
- apply using `-target`, for example
`terraform apply -target 'module.landing-project.google_project.project[0]'`
- untaint the project resource after applying, for example
`terraform untaint 'module.landing-project.google_project.project[0]'`
- go through the process to associate the billing account with the two projects
- switch `billing_account.id` back to the real billing account id
- resume applying normally
### Running the stage
Once provider and variable values are in place and the correct user is configured, the stage can be run:
```bash
terraform init
terraform apply
```
### Post-deployment activities
- On-prem routers should be configured to advertise all relevant CIDRs to the GCP environments. To avoid hitting GCP quotas, we recomment aggregating routes as much as possible.
- On-prem routers should accept BGP sessions from their cloud peers.
- On-prem DNS servers should have forward zones for GCP-managed ones.
#### Private Google Access
[Private Google Access](https://cloud.google.com/vpc/docs/private-google-access) (or PGA) enables VMs and on-prem systems to consume Google APIs from within the Google network, and is already fully configured on this environment.
For PGA to work:
- Private Google Access should be enabled on the subnet. \
Subnets created by the `net-vpc` module are PGA-enabled by default.
- 199.36.153.4/30 (`restricted.googleapis.com`) and 199.36.153.8/30 (`private.googleapis.com`) should be routed from on-prem to VPC, and from there to the `default-internet-gateway`. \
Per variable `vpn_onprem_configs` such ranges are advertised to onprem - furthermore every VPC (e.g. see `landing-vpc` in [`landing.tf`](./landing.tf)) has explicit routes set in case the `0.0.0.0/0` route is changed.
- A private DNS zone for `googleapis.com` should be created and configured per [this article](https://cloud.google.com/vpc/docs/configure-private-google-access-hybrid#config-domain), as implemented in module `googleapis-private-zone` in [`dns-landing.tf`](./dns-landing.tf)
## Customizations
### Changing default regions
Regions are defined via the `regions` variable which sets up a mapping between the `regions.primary` and `regions.secondary` logical names and actual GCP region names. If you need to change regions from the defaults:
- change the values of the mappings in the `regions` variable to the regions you are going to use
- change the regions in the factory subnet files in the `data` folder
### Configuring the VPNs to on prem
This stage includes basic support for an HA VPN connecting the landing zone in the primary region to on prem. Configuration is via the `vpn_onprem_primary_config` and `vpn_onprem_secondary_config` variables, that closely mirrors the variables defined in the [`net-vpn-ha`](../../../modules/net-vpn-ha/).
Support for the onprem VPNs is disabled by default so that no resources are created, this is an example of how to configure one variable to enable the VPN in the primary region:
```hcl
vpn_onprem_primary_config = {
peer_external_gateways = {
default = {
redundancy_type = "SINGLE_IP_INTERNALLY_REDUNDANT"
interfaces = ["8.8.8.8"]
}
}
router_config = {
asn = 65501
custom_advertise = {
all_subnets = false
ip_ranges = {
"10.1.0.0/16" = "gcp"
"35.199.192.0/19" = "gcp-dns"
"199.36.153.4/30" = "gcp-restricted"
}
}
}
tunnels = {
"0" = {
bgp_peer = {
address = "169.254.1.1"
asn = 65500
}
bgp_session_range = "169.254.1.2/30"
peer_external_gateway_interface = 0
shared_secret = "foo"
vpn_gateway_interface = 0
}
"1" = {
bgp_peer = {
address = "169.254.2.1"
asn = 64513
}
bgp_session_range = "169.254.2.2/30"
peer_external_gateway_interface = 1
shared_secret = "foo"
vpn_gateway_interface = 1
}
}
}
# tftest skip
```
### Adding an environment
To create a new environment (e.g. `staging`), a few changes are required:
Create a `spoke-staging.tf` file by copying `spoke-prod.tf` file.
Adapt the new file by replacing the value "prod" with the value "staging".
Running `diff spoke-dev.tf spoke-prod.tf` can help to see how environment files differ.
The new VPC requires a set of dedicated CIDRs, one per region, added to variable `gcp_ranges` (for example as `spoke_staging_ew1` and `spoke_staging_ew4`).
`gcp_ranges` is a map that "resolves" CIDR names to the actual addresses, and will be used later to configure routing.
Variables managing L7 Internal Load Balancers (`l7ilb_subnets`) and Private Service Access (`psa_ranges`) should also be adapted, and subnets and firewall rules for the new spoke should be added, as described above.
Configure the NVAs deployed updating the sample BGP [config file](./data/bgp-config.tftpl).
DNS configurations are centralised in the `dns-*.tf` files. Spokes delegate DNS resolution to Landing through DNS peering, and optionally define a private zone (e.g. `dev.gcp.example.com`) which the landing peers to. To configure DNS for a new environment, copy one of the other environments DNS files [e.g. (dns-dev.tf)](dns-dev.tf) into a new `dns-*.tf` file suffixed with the environment name (e.g. `dns-staging.tf`), and update its content accordingly. Don't forget to add a peering zone from the landing to the newly created environment private zone.
<!-- TFDOC OPTS files:1 show_extra:1 -->
<!-- BEGIN TFDOC -->
## Files
| name | description | modules | resources |
|---|---|---|---|
| [dns-dev.tf](./dns-dev.tf) | Development spoke DNS zones and peerings setup. | <code>dns</code> | |
| [dns-landing.tf](./dns-landing.tf) | Landing DNS zones and peerings setup. | <code>dns</code> | |
| [dns-prod.tf](./dns-prod.tf) | Production spoke DNS zones and peerings setup. | <code>dns</code> | |
| [landing.tf](./landing.tf) | Landing VPC and related resources. | <code>net-cloudnat</code> · <code>net-vpc</code> · <code>net-vpc-firewall</code> · <code>project</code> | |
| [main.tf](./main.tf) | Networking folder and hierarchical policy. | <code>folder</code> | |
| [monitoring.tf](./monitoring.tf) | Network monitoring dashboards. | | <code>google_monitoring_dashboard</code> |
| [ncc.tf](./ncc.tf) | None | <code>ncc-spoke-ra</code> | |
| [nva.tf](./nva.tf) | None | <code>compute-vm</code> · <code>simple-nva</code> | <code>google_compute_address</code> |
| [outputs.tf](./outputs.tf) | Module outputs. | | <code>google_storage_bucket_object</code> · <code>local_file</code> |
| [regions.tf](./regions.tf) | Compute short names for regions. | | |
| [spoke-dev.tf](./spoke-dev.tf) | Dev spoke VPC and related resources. | <code>net-vpc</code> · <code>net-vpc-firewall</code> · <code>net-vpc-peering</code> · <code>project</code> | <code>google_project_iam_binding</code> |
| [spoke-prod.tf](./spoke-prod.tf) | Production spoke VPC and related resources. | <code>net-vpc</code> · <code>net-vpc-firewall</code> · <code>net-vpc-peering</code> · <code>project</code> | <code>google_project_iam_binding</code> |
| [test-resources.tf](./test-resources.tf) | temporary instances for testing | <code>compute-vm</code> | |
| [variables.tf](./variables.tf) | Module variables. | | |
| [vpn-onprem.tf](./vpn-onprem.tf) | VPN between landing and onprem. | <code>net-vpn-ha</code> | |
## Variables
| name | description | type | required | default | producer |
|---|---|:---:|:---:|:---:|:---:|
| [automation](variables.tf#L17) | Automation resources created by the bootstrap stage. | <code title="object&#40;&#123;&#10; outputs_bucket &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>0-bootstrap</code> |
| [billing_account](variables.tf#L25) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | <code title="object&#40;&#123;&#10; id &#61; string&#10; is_org_level &#61; optional&#40;bool, true&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>0-bootstrap</code> |
| [folder_ids](variables.tf#L75) | Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created. | <code title="object&#40;&#123;&#10; networking &#61; string&#10; networking-dev &#61; string&#10; networking-prod &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>1-resman</code> |
| [organization](variables.tf#L119) | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; number&#10; customer_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>0-bootstrap</code> |
| [prefix](variables.tf#L135) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | <code>0-bootstrap</code> |
| [custom_roles](variables.tf#L38) | Custom roles defined at the org level, in key => id format. | <code title="object&#40;&#123;&#10; service_project_network_admin &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | <code>0-bootstrap</code> |
| [dns](variables.tf#L47) | Onprem DNS resolvers. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code title="&#123;&#10; onprem &#61; &#91;&#34;10.0.200.3&#34;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [factories_config](variables.tf#L55) | Configuration for network resource factories. | <code title="object&#40;&#123;&#10; data_dir &#61; optional&#40;string, &#34;data&#34;&#41;&#10; firewall_policy_name &#61; optional&#40;string, &#34;factory&#34;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; data_dir &#61; &#34;data&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [gcp_ranges](variables.tf#L85) | GCP address ranges in name => range format. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; gcp_dev_primary &#61; &#34;10.128.128.0&#47;19&#34;&#10; gcp_dev_secondary &#61; &#34;10.128.160.0&#47;19&#34;&#10; gcp_landing_trusted_primary &#61; &#34;10.128.64.0&#47;19&#34;&#10; gcp_landing_trusted_secondary &#61; &#34;10.128.96.0&#47;19&#34;&#10; gcp_landing_untrusted_primary &#61; &#34;10.128.0.0&#47;19&#34;&#10; gcp_landing_untrusted_secondary &#61; &#34;10.128.32.0&#47;19&#34;&#10; gcp_prod_primary &#61; &#34;10.128.192.0&#47;19&#34;&#10; gcp_prod_secondary &#61; &#34;10.128.224.0&#47;19&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [ncc_asn](variables.tf#L100) | The NCC Cloud Routers ASN configuration. | <code>map&#40;number&#41;</code> | | <code title="&#123;&#10; nva_primary &#61; 64513&#10; nva_secondary &#61; 64514&#10; trusted &#61; 64515&#10; untrusted &#61; 64512&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [onprem_cidr](variables.tf#L111) | Onprem addresses in name => range format. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; main &#61; &#34;10.0.0.0&#47;24&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [outputs_location](variables.tf#L129) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | <code>string</code> | | <code>null</code> | |
| [psa_ranges](variables.tf#L146) | IP ranges used for Private Service Access (e.g. CloudSQL). Ranges is in name => range format. | <code title="object&#40;&#123;&#10; dev &#61; object&#40;&#123;&#10; ranges &#61; map&#40;string&#41;&#10; routes &#61; object&#40;&#123;&#10; export &#61; bool&#10; import &#61; bool&#10; &#125;&#41;&#10; &#125;&#41;&#10; prod &#61; object&#40;&#123;&#10; ranges &#61; map&#40;string&#41;&#10; routes &#61; object&#40;&#123;&#10; export &#61; bool&#10; import &#61; bool&#10; &#125;&#41;&#10; &#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| [regions](variables.tf#L167) | Region definitions. | <code title="object&#40;&#123;&#10; primary &#61; string&#10; secondary &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; primary &#61; &#34;europe-west1&#34;&#10; secondary &#61; &#34;europe-west4&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | |
| [service_accounts](variables.tf#L179) | Automation service accounts in name => email format. | <code title="object&#40;&#123;&#10; data-platform-dev &#61; string&#10; data-platform-prod &#61; string&#10; gke-dev &#61; string&#10; gke-prod &#61; string&#10; project-factory-dev &#61; string&#10; project-factory-prod &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | <code>1-resman</code> |
| [vpn_onprem_primary_config](variables.tf#L193) | VPN gateway configuration for onprem interconnection in the primary region. | <code title="object&#40;&#123;&#10; peer_external_gateways &#61; map&#40;object&#40;&#123;&#10; redundancy_type &#61; string&#10; interfaces &#61; list&#40;string&#41;&#10; &#125;&#41;&#41;&#10; router_config &#61; object&#40;&#123;&#10; create &#61; optional&#40;bool, true&#41;&#10; asn &#61; number&#10; name &#61; optional&#40;string&#41;&#10; keepalive &#61; optional&#40;number&#41;&#10; custom_advertise &#61; optional&#40;object&#40;&#123;&#10; all_subnets &#61; bool&#10; ip_ranges &#61; map&#40;string&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#10; tunnels &#61; map&#40;object&#40;&#123;&#10; bgp_peer &#61; object&#40;&#123;&#10; address &#61; string&#10; asn &#61; number&#10; route_priority &#61; optional&#40;number, 1000&#41;&#10; custom_advertise &#61; optional&#40;object&#40;&#123;&#10; all_subnets &#61; bool&#10; all_vpc_subnets &#61; bool&#10; all_peer_vpc_subnets &#61; bool&#10; ip_ranges &#61; map&#40;string&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#10; bgp_session_range &#61; string&#10; ike_version &#61; optional&#40;number, 2&#41;&#10; peer_external_gateway_interface &#61; optional&#40;number&#41;&#10; peer_gateway &#61; optional&#40;string, &#34;default&#34;&#41;&#10; router &#61; optional&#40;string&#41;&#10; shared_secret &#61; optional&#40;string&#41;&#10; vpn_gateway_interface &#61; number&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| [vpn_onprem_secondary_config](variables.tf#L236) | VPN gateway configuration for onprem interconnection in the secondary region. | <code title="object&#40;&#123;&#10; peer_external_gateways &#61; map&#40;object&#40;&#123;&#10; redundancy_type &#61; string&#10; interfaces &#61; list&#40;string&#41;&#10; &#125;&#41;&#41;&#10; router_config &#61; object&#40;&#123;&#10; create &#61; optional&#40;bool, true&#41;&#10; asn &#61; number&#10; name &#61; optional&#40;string&#41;&#10; keepalive &#61; optional&#40;number&#41;&#10; custom_advertise &#61; optional&#40;object&#40;&#123;&#10; all_subnets &#61; bool&#10; ip_ranges &#61; map&#40;string&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#10; tunnels &#61; map&#40;object&#40;&#123;&#10; bgp_peer &#61; object&#40;&#123;&#10; address &#61; string&#10; asn &#61; number&#10; route_priority &#61; optional&#40;number, 1000&#41;&#10; custom_advertise &#61; optional&#40;object&#40;&#123;&#10; all_subnets &#61; bool&#10; all_vpc_subnets &#61; bool&#10; all_peer_vpc_subnets &#61; bool&#10; ip_ranges &#61; map&#40;string&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#10; bgp_session_range &#61; string&#10; ike_version &#61; optional&#40;number, 2&#41;&#10; peer_external_gateway_interface &#61; optional&#40;number&#41;&#10; peer_gateway &#61; optional&#40;string, &#34;default&#34;&#41;&#10; router &#61; optional&#40;string&#41;&#10; shared_secret &#61; optional&#40;string&#41;&#10; vpn_gateway_interface &#61; number&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | |
| [zones](variables.tf#L279) | Zones in which NVAs are deployed. | <code>list&#40;string&#41;</code> | | <code>&#91;&#34;b&#34;, &#34;c&#34;&#93;</code> | |
## Outputs
| name | description | sensitive | consumers |
|---|---|:---:|---|
| [host_project_ids](outputs.tf#L58) | Network project ids. | | |
| [host_project_numbers](outputs.tf#L63) | Network project numbers. | | |
| [shared_vpc_self_links](outputs.tf#L68) | Shared VPC host projects. | | |
| [tfvars](outputs.tf#L73) | Terraform variables file for the following stages. | ✓ | |
| [vpn_gateway_endpoints](outputs.tf#L79) | External IP Addresses for the GCP VPN gateways. | | |
<!-- END TFDOC -->

View File

@ -0,0 +1,76 @@
# NVAs configuration template
log syslog informational
no ipv6 forwarding
service integrated-vtysh-config
interface lo
ip address ${ip_untrusted}/32
ip prefix-list DEFAULT seq 10 permit 0.0.0.0/0
!
ip prefix-list PRIMARY seq 10 permit ${gcp_landing_trusted_primary}
ip prefix-list PRIMARY seq 20 permit ${gcp_dev_primary}
ip prefix-list PRIMARY seq 30 permit ${gcp_prod_primary}
!
ip prefix-list SECONDARY seq 10 permit ${gcp_landing_trusted_secondary}
ip prefix-list SECONDARY seq 20 permit ${gcp_dev_secondary}
ip prefix-list SECONDARY seq 30 permit ${gcp_prod_secondary}
route-map TO-UNTRUSTED permit 10
match ip address prefix-list PRIMARY
set metric ${cost_primary}
!
route-map TO-UNTRUSTED permit 20
match ip address prefix-list SECONDARY
set metric ${cost_secondary}
!
route-map TO-TRUSTED permit 10
match ip address prefix-list DEFAULT
set metric 100
!
route-map TO-NVA permit 10
match ip address prefix-list ${announce-to-nva}
set metric 50
router bgp ${asn_nva}
bgp router-id ${ip_untrusted}
bgp bestpath as-path ignore
bgp disable-ebgp-connected-route-check
bgp timers 20 60
!
no bgp ebgp-requires-policy
no bgp network import-check
!
neighbor ${ip_neighbor_untrusted_0} remote-as ${asn_untrusted}
neighbor ${ip_neighbor_untrusted_1} remote-as ${asn_untrusted}
!
neighbor ${ip_neighbor_trusted_0} remote-as ${asn_trusted}
neighbor ${ip_neighbor_trusted_0} update-source ${ip_trusted}
neighbor ${ip_neighbor_trusted_1} remote-as ${asn_trusted}
neighbor ${ip_neighbor_trusted_1} update-source ${ip_trusted}
!
neighbor ${ip_neighbor_cross_region_nva_0} remote-as ${asn_nva_cross_region}
neighbor ${ip_neighbor_cross_region_nva_0} ebgp-multihop 2
neighbor ${ip_neighbor_cross_region_nva_1} remote-as ${asn_nva_cross_region}
neighbor ${ip_neighbor_cross_region_nva_1} ebgp-multihop 2
!
address-family ipv4 unicast
neighbor ${ip_neighbor_untrusted_0} route-map TO-UNTRUSTED out
neighbor ${ip_neighbor_untrusted_0} soft-reconfiguration inbound
!
neighbor ${ip_neighbor_untrusted_1} route-map TO-UNTRUSTED out
neighbor ${ip_neighbor_untrusted_1} soft-reconfiguration inbound
!
neighbor ${ip_neighbor_trusted_0} route-map TO-TRUSTED out
neighbor ${ip_neighbor_trusted_0} soft-reconfiguration inbound
!
neighbor ${ip_neighbor_trusted_1} route-map TO-TRUSTED out
neighbor ${ip_neighbor_trusted_1} soft-reconfiguration inbound
!
neighbor ${ip_neighbor_cross_region_nva_0} route-map TO-NVA out
neighbor ${ip_neighbor_cross_region_nva_0} soft-reconfiguration inbound
!
neighbor ${ip_neighbor_cross_region_nva_1} route-map TO-NVA out
neighbor ${ip_neighbor_cross_region_nva_1} soft-reconfiguration inbound
exit-address-family

View File

@ -0,0 +1,27 @@
# skip boilerplate check
healthchecks:
- 35.191.0.0/16
- 130.211.0.0/22
- 209.85.152.0/22
- 209.85.204.0/22
ncc_cloud_routers_trusted:
- 10.128.64.201/32
- 10.128.64.202/32
- 10.128.96.201/32
- 10.128.96.202/32
ncc_cloud_routers_untrusted:
- 10.128.0.201/32
- 10.128.0.202/32
- 10.128.32.201/32
- 10.128.32.202/32
rfc1918:
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
onprem_probes:
- 10.255.255.254/32

View File

@ -0,0 +1,68 @@
{
"displayName": "Firewall Insights Monitoring",
"gridLayout": {
"columns": "2",
"widgets": [
{
"title": "Subnet Firewall Hit Counts",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"firewallinsights.googleapis.com/subnet/firewall_hit_count\" resource.type=\"gce_subnetwork\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "VM Firewall Hit Counts",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"firewallinsights.googleapis.com/vm/firewall_hit_count\" resource.type=\"gce_instance\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
}
]
}
}

View File

@ -0,0 +1,248 @@
{
"displayName": "VPN Monitoring",
"gridLayout": {
"columns": "2",
"widgets": [
{
"title": "Number of connections",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_MEAN"
},
"filter": "metric.type=\"vpn.googleapis.com/gateway/connections\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Tunnel established",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_MEAN"
},
"filter": "metric.type=\"vpn.googleapis.com/tunnel_established\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Received bytes",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/received_bytes_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "By"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Sent bytes",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/sent_bytes_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "By"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Received packets",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/received_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "{packets}"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Cloud VPN Gateway - Sent packets",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/sent_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "{packets}"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Incoming packets dropped",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/dropped_received_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
},
{
"title": "Outgoing packets dropped",
"xyChart": {
"chartOptions": {
"mode": "COLOR"
},
"dataSets": [
{
"minAlignmentPeriod": "60s",
"plotType": "LINE",
"targetAxis": "Y1",
"timeSeriesQuery": {
"timeSeriesFilter": {
"aggregation": {
"perSeriesAligner": "ALIGN_RATE"
},
"filter": "metric.type=\"vpn.googleapis.com/network/dropped_sent_packets_count\" resource.type=\"vpn_gateway\"",
"secondaryAggregation": {}
},
"unitOverride": "1"
}
}
],
"timeshiftDuration": "0s",
"yAxis": {
"label": "y1Axis",
"scale": "LINEAR"
}
}
}
]
}
}

View File

@ -0,0 +1,21 @@
# skip boilerplate check
ingress:
ingress-allow-composer-nodes:
description: "Allow traffic to Composer nodes."
sources:
- composer-worker
targets:
- composer-worker
rules:
- protocol: tcp
ports: [80, 443, 3306, 3307]
ingress-allow-dataflow-load:
description: "Allow traffic to Dataflow nodes."
sources:
- dataflow
targets:
- dataflow
rules:
- protocol: tcp
ports: [12345, 12346]

View File

@ -0,0 +1,31 @@
# skip boilerplate check
ingress:
allow-hc-nva-ssh-trusted:
description: "Allow traffic from Google healthchecks to NVA appliances"
source_ranges:
- healthchecks
rules:
- protocol: tcp
ports:
- 22
allow-onprem-probes-trusted-example:
description: "Allow traffic from onprem probes"
source_ranges:
- onprem_probes
rules:
- protocol: tcp
ports:
- 12345
# This is not really needed, but it's good to have it
# in place if the more generic hierarchical firewall policies
# get deleted
allow-ncc-nva-bgp-trusted:
description: "Allow BGP traffic from NCC Cloud Routers to NVAs"
source_ranges:
- ncc_cloud_routers_trusted
targets: ["nva"]
rules:
- protocol: tcp
ports:
- 179

View File

@ -0,0 +1,31 @@
# skip boilerplate check
ingress:
allow-hc-nva-ssh-untrusted:
description: "Allow traffic from Google healthchecks to NVA appliances"
source_ranges:
- healthchecks
rules:
- protocol: tcp
ports:
- 22
# these are not really needed, but it's good to have them
# in place if the more generic hierarchical firewall policies
# get deleted
allow-ncc-nva-bgp-untrusted:
description: "Allow BGP traffic from NCC Cloud Routers to NVAs"
source_ranges:
- ncc_cloud_routers_untrusted
targets: ["nva"]
rules:
- protocol: tcp
ports:
- 179
allow-nva-nva-bgp-untrusted:
description: "Allow BGP traffic from cross-regional NVAs"
sources: ["nva"]
targets: ["nva"]
rules:
- protocol: tcp
ports:
- 179

View File

@ -0,0 +1,49 @@
# skip boilerplate check
allow-admins:
description: Access from the admin subnet to all subnets
direction: INGRESS
action: allow
priority: 1000
ranges:
- $rfc1918
ports:
all: []
target_resources: null
enable_logging: false
allow-healthchecks:
description: Enable HTTP and HTTPS healthchecks
direction: INGRESS
action: allow
priority: 1001
ranges:
- $healthchecks
ports:
tcp: ["80", "443"]
target_resources: null
enable_logging: false
allow-ssh-from-iap:
description: Enable SSH from IAP
direction: INGRESS
action: allow
priority: 1002
ranges:
- 35.235.240.0/20
ports:
tcp: ["22"]
target_resources: null
enable_logging: false
allow-icmp:
description: Enable ICMP
direction: INGRESS
action: allow
priority: 1003
ranges:
- 0.0.0.0/0
ports:
icmp: []
target_resources: null
enable_logging: false

View File

@ -0,0 +1,8 @@
# skip boilerplate check
region: europe-west1
description: Default subnet for dev Data Platform
ip_cidr_range: 10.128.48.0/24
secondary_ip_ranges:
pods: 100.128.48.0/20
services: 100.255.48.0/24

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.128.128.0/24
description: Default europe-west1 subnet for dev

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.128.160.0/24
description: Default europe-west4 subnet for dev

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.128.64.0/24
description: Default europe-west1 subnet for landing trusted

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.128.96.0/24
description: Default europe-west4 subnet for landing trusted

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.128.0.0/24
description: Default europe-west1 subnet for landing untrusted

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.128.32.0/24
description: Default europe-west4 subnet for landing untrusted

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west1
ip_cidr_range: 10.128.192.0/24
description: Default europe-west1 subnet for prod

View File

@ -0,0 +1,5 @@
# skip boilerplate check
region: europe-west4
ip_cidr_range: 10.128.224.0/24
description: Default europe-west4 subnet for prod

Binary file not shown.

After

Width:  |  Height:  |  Size: 80 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 470 KiB

View File

@ -0,0 +1,53 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Development spoke DNS zones and peerings setup.
# GCP-specific environment zone
module "dev-dns-private-zone" {
source = "../../../modules/dns"
project_id = module.dev-spoke-project.project_id
type = "private"
name = "dev-gcp-example-com"
domain = "dev.gcp.example.com."
client_networks = [module.landing-trusted-vpc.self_link, module.landing-untrusted-vpc.self_link]
recordsets = {
"A localhost" = { records = ["127.0.0.1"] }
}
}
# root zone peering to landing to centralize configuration; remove if unneeded
module "dev-landing-root-dns-peering" {
source = "../../../modules/dns"
project_id = module.dev-spoke-project.project_id
type = "peering"
name = "dev-root-dns-peering"
domain = "."
client_networks = [module.dev-spoke-vpc.self_link]
peer_network = module.landing-trusted-vpc.self_link
}
module "dev-reverse-10-dns-peering" {
source = "../../../modules/dns"
project_id = module.dev-spoke-project.project_id
type = "peering"
name = "dev-reverse-10-dns-peering"
domain = "10.in-addr.arpa."
client_networks = [module.dev-spoke-vpc.self_link]
peer_network = module.landing-trusted-vpc.self_link
}

View File

@ -0,0 +1,155 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Landing DNS zones and peerings setup.
# forwarding to on-prem DNS resolvers
module "onprem-example-dns-forwarding" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "forwarding"
name = "example-com"
domain = "onprem.example.com."
client_networks = [
module.landing-untrusted-vpc.self_link,
module.landing-trusted-vpc.self_link
]
forwarders = { for ip in var.dns.onprem : ip => null }
}
module "reverse-10-dns-forwarding" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "forwarding"
name = "root-reverse-10"
domain = "10.in-addr.arpa."
client_networks = [
module.landing-untrusted-vpc.self_link,
module.landing-trusted-vpc.self_link
]
forwarders = { for ip in var.dns.onprem : ip => null }
}
module "gcp-example-dns-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "gcp-example-com"
domain = "gcp.example.com."
client_networks = [
module.landing-untrusted-vpc.self_link,
module.landing-trusted-vpc.self_link
]
recordsets = {
"A localhost" = { records = ["127.0.0.1"] }
}
}
# Google APIs
module "googleapis-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "googleapis-com"
domain = "googleapis.com."
client_networks = [
module.landing-untrusted-vpc.self_link,
module.landing-trusted-vpc.self_link
]
recordsets = {
"A private" = { records = [
"199.36.153.8", "199.36.153.9", "199.36.153.10", "199.36.153.11"
] }
"A restricted" = { records = [
"199.36.153.4", "199.36.153.5", "199.36.153.6", "199.36.153.7"
] }
"CNAME *" = { records = ["private.googleapis.com."] }
}
}
module "gcrio-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "gcr-io"
domain = "gcr.io."
client_networks = [
module.landing-untrusted-vpc.self_link,
module.landing-trusted-vpc.self_link
]
recordsets = {
"A gcr.io." = { ttl = 300, records = [
"199.36.153.8", "199.36.153.9", "199.36.153.10", "199.36.153.11"
] }
"CNAME *" = { ttl = 300, records = ["private.googleapis.com."] }
}
}
module "packages-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "packages-cloud"
domain = "packages.cloud.google.com."
client_networks = [
module.landing-untrusted-vpc.self_link,
module.landing-trusted-vpc.self_link
]
recordsets = {
"A packages.cloud.google.com." = { ttl = 300, records = [
"199.36.153.8", "199.36.153.9", "199.36.153.10", "199.36.153.11"
] }
"CNAME *" = { ttl = 300, records = ["private.googleapis.com."] }
}
}
module "pkgdev-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "pkg-dev"
domain = "pkg.dev."
client_networks = [
module.landing-untrusted-vpc.self_link,
module.landing-trusted-vpc.self_link
]
recordsets = {
"A pkg.dev." = { ttl = 300, records = [
"199.36.153.8", "199.36.153.9", "199.36.153.10", "199.36.153.11"
] }
"CNAME *" = { ttl = 300, records = ["private.googleapis.com."] }
}
}
module "pkigoog-private-zone" {
source = "../../../modules/dns"
project_id = module.landing-project.project_id
type = "private"
name = "pki-goog"
domain = "pki.goog."
client_networks = [
module.landing-untrusted-vpc.self_link,
module.landing-trusted-vpc.self_link
]
recordsets = {
"A pki.goog." = { ttl = 300, records = [
"199.36.153.8", "199.36.153.9", "199.36.153.10", "199.36.153.11"
] }
"CNAME *" = { ttl = 300, records = ["private.googleapis.com."] }
}
}

View File

@ -0,0 +1,53 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Production spoke DNS zones and peerings setup.
# GCP-specific environment zone
module "prod-dns-private-zone" {
source = "../../../modules/dns"
project_id = module.prod-spoke-project.project_id
type = "private"
name = "prod-gcp-example-com"
domain = "prod.gcp.example.com."
client_networks = [module.landing-trusted-vpc.self_link, module.landing-untrusted-vpc.self_link]
recordsets = {
"A localhost" = { records = ["127.0.0.1"] }
}
}
# root zone peering to landing to centralize configuration; remove if unneeded
module "prod-landing-root-dns-peering" {
source = "../../../modules/dns"
project_id = module.prod-spoke-project.project_id
type = "peering"
name = "prod-root-dns-peering"
domain = "."
client_networks = [module.prod-spoke-vpc.self_link]
peer_network = module.landing-trusted-vpc.self_link
}
module "prod-reverse-10-dns-peering" {
source = "../../../modules/dns"
project_id = module.prod-spoke-project.project_id
type = "peering"
name = "prod-reverse-10-dns-peering"
domain = "10.in-addr.arpa."
client_networks = [module.prod-spoke-vpc.self_link]
peer_network = module.landing-trusted-vpc.self_link
}

View File

@ -0,0 +1,145 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Landing VPC and related resources.
module "landing-project" {
source = "../../../modules/project"
billing_account = var.billing_account.id
name = "prod-net-landing-0"
parent = var.folder_ids.networking-prod
prefix = var.prefix
services = [
"compute.googleapis.com",
"dns.googleapis.com",
"iap.googleapis.com",
"networkconnectivity.googleapis.com",
"networkmanagement.googleapis.com",
"stackdriver.googleapis.com"
]
shared_vpc_host_config = {
enabled = true
}
iam = {
"roles/dns.admin" = compact([
try(local.service_accounts.project-factory-prod, null)
])
(local.custom_roles.service_project_network_admin) = compact([
try(local.service_accounts.project-factory-prod, null)
])
}
}
# Untrusted VPC
module "landing-untrusted-vpc" {
source = "../../../modules/net-vpc"
project_id = module.landing-project.project_id
name = "prod-untrusted-landing-0"
mtu = 1500
dns_policy = {
inbound = false
logging = false
}
data_folder = "${var.factories_config.data_dir}/subnets/landing-untrusted"
}
module "landing-untrusted-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-untrusted-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.factories_config.data_dir}/cidrs.yaml"
rules_folder = "${var.factories_config.data_dir}/firewall-rules/landing-untrusted"
}
}
# NAT
moved {
from = module.landing-nat-ew1
to = module.landing-nat-primary
}
module "landing-nat-primary" {
source = "../../../modules/net-cloudnat"
project_id = module.landing-project.project_id
region = var.regions.primary
name = local.region_shortnames[var.regions.primary]
router_create = true
router_name = "prod-nat-${local.region_shortnames[var.regions.primary]}"
router_network = module.landing-untrusted-vpc.name
router_asn = 4200001024
}
moved {
from = module.landing-nat-ew4
to = module.landing-nat-secondary
}
module "landing-nat-secondary" {
source = "../../../modules/net-cloudnat"
project_id = module.landing-project.project_id
region = var.regions.secondary
name = local.region_shortnames[var.regions.secondary]
router_create = true
router_name = "prod-nat-${local.region_shortnames[var.regions.secondary]}"
router_network = module.landing-untrusted-vpc.name
router_asn = 4200001024
}
# Trusted VPC
module "landing-trusted-vpc" {
source = "../../../modules/net-vpc"
project_id = module.landing-project.project_id
name = "prod-trusted-landing-0"
delete_default_routes_on_create = true
mtu = 1500
data_folder = "${var.factories_config.data_dir}/subnets/landing-trusted"
dns_policy = {
inbound = true
}
# Set explicit routes for googleapis in case the default route is deleted
routes = {
private-googleapis = {
dest_range = "199.36.153.8/30"
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
restricted-googleapis = {
dest_range = "199.36.153.4/30"
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
}
}
module "landing-trusted-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.landing-project.project_id
network = module.landing-trusted-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.factories_config.data_dir}/cidrs.yaml"
rules_folder = "${var.factories_config.data_dir}/firewall-rules/landing-trusted"
}
}

View File

@ -0,0 +1,57 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Networking folder and hierarchical policy.
locals {
custom_roles = coalesce(var.custom_roles, {})
# combine all regions from variables and subnets
regions = distinct(concat(
values(var.regions),
values(module.dev-spoke-vpc.subnet_regions),
values(module.landing-trusted-vpc.subnet_regions),
values(module.landing-untrusted-vpc.subnet_regions),
values(module.prod-spoke-vpc.subnet_regions),
))
service_accounts = {
for k, v in coalesce(var.service_accounts, {}) :
k => "serviceAccount:${v}" if v != null
}
stage3_sas_delegated_grants = [
"roles/composer.sharedVpcAgent",
"roles/compute.networkUser",
"roles/compute.networkViewer",
"roles/container.hostServiceAgentUser",
"roles/multiclusterservicediscovery.serviceAgent",
"roles/vpcaccess.user",
]
}
module "folder" {
source = "../../../modules/folder"
parent = "organizations/${var.organization.id}"
name = "Networking"
folder_create = var.folder_ids.networking == null
id = var.folder_ids.networking
firewall_policy_factory = {
cidr_file = "${var.factories_config.data_dir}/cidrs.yaml"
policy_name = var.factories_config.firewall_policy_name
rules_file = "${var.factories_config.data_dir}/hierarchical-policy-rules.yaml"
}
firewall_policy_association = {
factory-policy = var.factories_config.firewall_policy_name
}
}

View File

@ -0,0 +1,32 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Network monitoring dashboards.
locals {
dashboard_path = "${var.factories_config.data_dir}/dashboards"
dashboard_files = fileset(local.dashboard_path, "*.json")
dashboards = {
for filename in local.dashboard_files :
filename => "${local.dashboard_path}/${filename}"
}
}
resource "google_monitoring_dashboard" "dashboard" {
for_each = local.dashboards
project = module.landing-project.project_id
dashboard_json = file(each.value)
}

View File

@ -0,0 +1,108 @@
/**
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "spokes-trusted" {
for_each = var.regions
source = "../../../modules/ncc-spoke-ra"
name = "prod-spoke-trusted-${local.region_shortnames[each.value]}"
project_id = module.landing-project.project_id
region = each.value
hub = {
create = true,
name = "prod-hub-trusted-${each.value}"
}
router_appliances = [
for key, config in local.nva_configs :
{
internal_ip = module.nva[key].internal_ips[1]
vm_self_link = module.nva[key].self_link
} if config.region == each.value
]
router_config = {
asn = var.ncc_asn.trusted
ip_interface0 = cidrhost(module.landing-trusted-vpc.subnet_ips["${each.value}/landing-trusted-default-${local.region_shortnames[each.value]}"], 201)
ip_interface1 = cidrhost(module.landing-trusted-vpc.subnet_ips["${each.value}/landing-trusted-default-${local.region_shortnames[each.value]}"], 202)
peer_asn = (
each.key == "primary"
? var.ncc_asn.nva_primary
: var.ncc_asn.nva_secondary
)
ruotes_priority = 100
custom_advertise = {
all_subnets = false
ip_ranges = {
"${var.gcp_ranges.gcp_landing_trusted_primary}" = "GCP landing trusted primary."
"${var.gcp_ranges.gcp_landing_trusted_secondary}" = "GCP landing trusted secondary."
"${var.gcp_ranges.gcp_dev_primary}" = "GCP dev primary.",
"${var.gcp_ranges.gcp_dev_secondary}" = "GCP dev secondary.",
"${var.gcp_ranges.gcp_prod_primary}" = "GCP prod primary.",
"${var.gcp_ranges.gcp_prod_secondary}" = "GCP prod secondary.",
}
}
}
vpc_config = {
network_name = module.landing-trusted-vpc.self_link
subnet_self_link = module.landing-trusted-vpc.subnet_self_links["${each.value}/landing-trusted-default-${local.region_shortnames[each.value]}"]
}
}
module "spokes-untrusted" {
for_each = var.regions
source = "../../../modules/ncc-spoke-ra"
name = "prod-spoke-untrusted-${local.region_shortnames[each.value]}"
project_id = module.landing-project.project_id
region = each.value
hub = {
create = true,
name = "prod-hub-untrusted-${each.value}"
}
router_appliances = [
for key, config in local.nva_configs :
{
internal_ip = module.nva[key].internal_ips[0]
vm_self_link = module.nva[key].self_link
} if config.region == each.value
]
router_config = {
asn = var.ncc_asn.untrusted
ip_interface0 = cidrhost(module.landing-untrusted-vpc.subnet_ips["${each.value}/landing-untrusted-default-${local.region_shortnames[each.value]}"], 201)
ip_interface1 = cidrhost(module.landing-untrusted-vpc.subnet_ips["${each.value}/landing-untrusted-default-${local.region_shortnames[each.value]}"], 202)
peer_asn = (
each.key == "primary"
? var.ncc_asn.nva_primary
: var.ncc_asn.nva_secondary
)
ruotes_priority = 100
custom_advertise = {
all_subnets = false
ip_ranges = { "0.0.0.0/0" = "Deafult route." }
}
}
vpc_config = {
network_name = module.landing-untrusted-vpc.self_link
subnet_self_link = module.landing-untrusted-vpc.subnet_self_links["${each.value}/landing-untrusted-default-${local.region_shortnames[each.value]}"]
}
}

View File

@ -0,0 +1,187 @@
/**
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
_nva_zones = ["b", "c"]
# The configurations used to create the NVA VMs.
#
# Rendered as following:
# nva_configs = {
# primary-b = {...}
# primary-c = {...}
# secondary-b = {...}
# secondary-c = {...}
# }
nva_configs = {
for v in setproduct(keys(var.regions), local._nva_zones) :
join("-", v) => {
# Each NVA announces its trusted regional subnets
announce-to-nva = upper(v.0)
# NVAs in each region have their own ASN
# and peer with cross-regional NVAs.
asn_nva = (
v.0 == "primary"
? var.ncc_asn.nva_primary
: var.ncc_asn.nva_secondary
)
asn_nva_cross_region = (
v.0 == "primary"
? var.ncc_asn.nva_secondary
: var.ncc_asn.nva_primary
)
asn_trusted = var.ncc_asn.trusted
asn_untrusted = var.ncc_asn.untrusted
# To guarantee traffic to remain symmetric,
# NVAs need to advertise cross-region routes with a higher cost (10100)
cost_primary = v.0 == "primary" ? "100" : "10100"
cost_secondary = v.0 == "primary" ? "10100" : "100"
gcp_dev_primary = var.gcp_ranges.gcp_dev_primary
gcp_dev_secondary = var.gcp_ranges.gcp_dev_secondary
gcp_landing_trusted_primary = var.gcp_ranges.gcp_landing_trusted_primary
gcp_landing_trusted_secondary = var.gcp_ranges.gcp_landing_trusted_secondary
gcp_landing_untrusted_primary = var.gcp_ranges.gcp_landing_untrusted_primary
gcp_landing_untrusted_secondary = var.gcp_ranges.gcp_landing_untrusted_secondary
gcp_prod_primary = var.gcp_ranges.gcp_prod_primary
gcp_prod_secondary = var.gcp_ranges.gcp_prod_secondary
# The IPs of cross-region NVA VMs in the untrusted VPC (x.y.w.z)
ip_neighbor_cross_region_nva_0 = cidrhost(module.landing-untrusted-vpc.subnet_ips["${local._regions_cross[v.0]}/landing-untrusted-default-${local.region_shortnames[local._regions_cross[v.0]]}"], 101)
ip_neighbor_cross_region_nva_1 = cidrhost(module.landing-untrusted-vpc.subnet_ips["${local._regions_cross[v.0]}/landing-untrusted-default-${local.region_shortnames[local._regions_cross[v.0]]}"], 102)
# The Cloud router IPs (x.y.w.z) in the untrusted
# and in the trusted VPCs, where the NVA connects to
ip_neighbor_trusted_0 = cidrhost(module.landing-trusted-vpc.subnet_ips["${var.regions[v.0]}/landing-trusted-default-${local.region_shortnames[var.regions[v.0]]}"], 201)
ip_neighbor_trusted_1 = cidrhost(module.landing-trusted-vpc.subnet_ips["${var.regions[v.0]}/landing-trusted-default-${local.region_shortnames[var.regions[v.0]]}"], 202)
ip_neighbor_untrusted_0 = cidrhost(module.landing-untrusted-vpc.subnet_ips["${var.regions[v.0]}/landing-untrusted-default-${local.region_shortnames[var.regions[v.0]]}"], 201)
ip_neighbor_untrusted_1 = cidrhost(module.landing-untrusted-vpc.subnet_ips["${var.regions[v.0]}/landing-untrusted-default-${local.region_shortnames[var.regions[v.0]]}"], 202)
# The IPs to assign to the NVA NICs
# in the trusted and in the untrusted VPCs.
ip_trusted = cidrhost(module.landing-trusted-vpc.subnet_ips["${var.regions[v.0]}/landing-trusted-default-${local.region_shortnames[var.regions[v.0]]}"], 101 + index(var.zones, v.1))
ip_untrusted = cidrhost(module.landing-untrusted-vpc.subnet_ips["${var.regions[v.0]}/landing-untrusted-default-${local.region_shortnames[var.regions[v.0]]}"], 101 + index(var.zones, v.1))
# Either primary or secondary
name = v.0
# The name of the region where the NVA lives.
# For example, europe-west1 or europe-west4
region = var.regions[v.0]
# the short name for the region. For example, ew1 or ew4
shortname = local.region_shortnames[var.regions[v.0]]
# The zone where the NVA lives. For example, b or c
zone = v.1
}
}
# The routing_config should be aligned to the NVA NICs.
# For example:
# local.routing_config[0] configures eth0;
# local.routing_config[0] configures eth1.
routing_config = [
{
enable_masquerading = true
name = "untrusted"
routes = [
var.gcp_ranges.gcp_landing_untrusted_primary,
var.gcp_ranges.gcp_landing_untrusted_secondary
]
},
{
name = "trusted"
routes = [
var.gcp_ranges.gcp_landing_trusted_primary,
var.gcp_ranges.gcp_landing_trusted_secondary
]
}
]
}
module "nva-bgp-cloud-config" {
for_each = local.nva_configs
source = "../../../modules/cloud-config-container/simple-nva"
enable_health_checks = true
network_interfaces = local.routing_config
frr_config = {
config_file = templatefile("data/bgp-config.tftpl", each.value)
daemons_enabled = ["bgpd"]
}
}
resource "google_compute_address" "nva_static_ip_trusted" {
for_each = local.nva_configs
name = "nva-ip-trusted-${each.value.shortname}-${each.value.zone}"
project = module.landing-project.project_id
subnetwork = module.landing-trusted-vpc.subnet_self_links["${each.value.region}/landing-trusted-default-${each.value.shortname}"]
address_type = "INTERNAL"
address = each.value.ip_trusted
region = each.value.region
}
resource "google_compute_address" "nva_static_ip_untrusted" {
for_each = local.nva_configs
name = "nva-ip-untrusted-${each.value.shortname}-${each.value.zone}"
project = module.landing-project.project_id
subnetwork = module.landing-untrusted-vpc.subnet_self_links["${each.value.region}/landing-untrusted-default-${each.value.shortname}"]
address_type = "INTERNAL"
address = each.value.ip_untrusted
region = each.value.region
}
module "nva" {
for_each = local.nva_configs
source = "../../../modules/compute-vm"
project_id = module.landing-project.project_id
name = "nva-${each.value.shortname}-${each.value.zone}"
instance_type = "e2-standard-2"
can_ip_forward = true
zone = "${each.value.region}-${each.value.zone}"
tags = ["nva"]
network_interfaces = [
{
network = module.landing-untrusted-vpc.self_link
subnetwork = module.landing-untrusted-vpc.subnet_self_links["${each.value.region}/landing-untrusted-default-${each.value.shortname}"]
nat = false
addresses = {
external = null
internal = google_compute_address.nva_static_ip_untrusted[each.key].address
}
},
{
network = module.landing-trusted-vpc.self_link
subnetwork = module.landing-trusted-vpc.subnet_self_links["${each.value.region}/landing-trusted-default-${each.value.shortname}"]
nat = false
addresses = {
external = null
internal = google_compute_address.nva_static_ip_trusted[each.key].address
}
}
]
boot_disk = {
initialize_params = {
image = "projects/cos-cloud/global/images/family/cos-stable"
size = 10
type = "pd-balanced"
}
}
options = {
allow_stopping_for_update = true
deletion_protection = false
termination_action = "STOP"
}
metadata = {
user-data = module.nva-bgp-cloud-config[each.key].cloud_config
}
}

View File

@ -0,0 +1,91 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
host_project_ids = {
dev-spoke-0 = module.dev-spoke-project.project_id
prod-landing = module.landing-project.project_id
prod-spoke-0 = module.prod-spoke-project.project_id
}
host_project_numbers = {
dev-spoke-0 = module.dev-spoke-project.number
prod-landing = module.landing-project.number
prod-spoke-0 = module.prod-spoke-project.number
}
tfvars = {
host_project_ids = local.host_project_ids
host_project_numbers = local.host_project_numbers
vpc_self_links = local.vpc_self_links
}
vpc_self_links = {
prod-landing-trusted = module.landing-trusted-vpc.self_link
prod-landing-untrusted = module.landing-untrusted-vpc.self_link
dev-spoke-0 = module.dev-spoke-vpc.self_link
prod-spoke-0 = module.prod-spoke-vpc.self_link
}
}
# generate tfvars file for subsequent stages
resource "local_file" "tfvars" {
for_each = var.outputs_location == null ? {} : { 1 = 1 }
file_permission = "0644"
filename = "${try(pathexpand(var.outputs_location), "")}/tfvars/2-networking.auto.tfvars.json"
content = jsonencode(local.tfvars)
}
resource "google_storage_bucket_object" "tfvars" {
bucket = var.automation.outputs_bucket
name = "tfvars/2-networking.auto.tfvars.json"
content = jsonencode(local.tfvars)
}
# outputs
output "host_project_ids" {
description = "Network project ids."
value = local.host_project_ids
}
output "host_project_numbers" {
description = "Network project numbers."
value = local.host_project_numbers
}
output "shared_vpc_self_links" {
description = "Shared VPC host projects."
value = local.vpc_self_links
}
output "tfvars" {
description = "Terraform variables file for the following stages."
sensitive = true
value = local.tfvars
}
output "vpn_gateway_endpoints" {
description = "External IP Addresses for the GCP VPN gateways."
value = {
onprem-primary = var.vpn_onprem_primary_config == null ? {} : {
for v in module.landing-to-onprem-primary-vpn.0.gateway.vpn_interfaces :
v.id => v.ip_address
}
onprem-secondary = var.vpn_onprem_secondary_config == null ? {} : {
for v in module.landing-to-onprem-secondary-vpn.0.gateway.vpn_interfaces :
v.id => v.ip_address
}
}
}

View File

@ -0,0 +1,46 @@
/**
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Compute short names for regions.
locals {
# only map when the first character would not work
_region_cardinal = {
southeast = "se"
}
_regions_cross = {
primary = var.regions["secondary"]
secondary = var.regions["primary"]
}
# only map when the first character would not work
_region_geo = {
australia = "o"
}
# split in [geo, cardinal, number] tokens
_region_tokens = {
for v in local.regions : v => regexall("(?:[a-z]+)|(?:[0-9]+)", v)
}
region_shortnames = {
for k, v in local._region_tokens : k => join("", [
# first token via geo alias map or first character
lookup(local._region_geo, v.0, substr(v.0, 0, 1)),
# first token via cardinal alias map or first character
lookup(local._region_cardinal, v.1, substr(v.1, 0, 1)),
# region number as is
v.2
])
}
}

View File

@ -0,0 +1,112 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Dev spoke VPC and related resources.
module "dev-spoke-project" {
source = "../../../modules/project"
billing_account = var.billing_account.id
name = "dev-net-spoke-0"
parent = var.folder_ids.networking-dev
prefix = var.prefix
services = [
"compute.googleapis.com",
"dns.googleapis.com",
"iap.googleapis.com",
"networkmanagement.googleapis.com",
"servicenetworking.googleapis.com",
"stackdriver.googleapis.com",
"vpcaccess.googleapis.com"
]
shared_vpc_host_config = {
enabled = true
}
metric_scopes = [module.landing-project.project_id]
iam = {
"roles/dns.admin" = compact([
try(local.service_accounts.gke-dev, null),
try(local.service_accounts.project-factory-dev, null),
try(local.service_accounts.project-factory-prod, null),
])
}
}
module "dev-spoke-vpc" {
source = "../../../modules/net-vpc"
project_id = module.dev-spoke-project.project_id
name = "dev-spoke-0"
mtu = 1500
data_folder = "${var.factories_config.data_dir}/subnets/dev"
delete_default_routes_on_create = true
psa_config = try(var.psa_ranges.dev, null)
# Set explicit routes for googleapis; send everything else to NVAs
routes = {
private-googleapis = {
dest_range = "199.36.153.8/30"
priority = 999
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
restricted-googleapis = {
dest_range = "199.36.153.4/30"
priority = 999
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
}
}
module "dev-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.dev-spoke-project.project_id
network = module.dev-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.factories_config.data_dir}/cidrs.yaml"
rules_folder = "${var.factories_config.data_dir}/firewall-rules/dev"
}
}
module "peering-dev" {
source = "../../../modules/net-vpc-peering"
prefix = "dev-peering-0"
local_network = module.dev-spoke-vpc.self_link
peer_network = module.landing-trusted-vpc.self_link
export_local_custom_routes = true
export_peer_custom_routes = true
}
# Create delegated grants for stage3 service accounts
resource "google_project_iam_binding" "dev_spoke_project_iam_delegated" {
project = module.dev-spoke-project.project_id
role = "roles/resourcemanager.projectIamAdmin"
members = compact([
try(local.service_accounts.data-platform-dev, null),
try(local.service_accounts.project-factory-dev, null),
try(local.service_accounts.project-factory-prod, null),
try(local.service_accounts.gke-dev, null),
])
condition {
title = "dev_stage3_sa_delegated_grants"
description = "Development host project delegated grants."
expression = format(
"api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
join(",", formatlist("'%s'", local.stage3_sas_delegated_grants))
)
}
}

View File

@ -0,0 +1,110 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Production spoke VPC and related resources.
module "prod-spoke-project" {
source = "../../../modules/project"
billing_account = var.billing_account.id
name = "prod-net-spoke-0"
parent = var.folder_ids.networking-prod
prefix = var.prefix
services = [
"compute.googleapis.com",
"dns.googleapis.com",
"iap.googleapis.com",
"networkmanagement.googleapis.com",
"servicenetworking.googleapis.com",
"stackdriver.googleapis.com",
"vpcaccess.googleapis.com"
]
shared_vpc_host_config = {
enabled = true
}
metric_scopes = [module.landing-project.project_id]
iam = {
"roles/dns.admin" = compact([
try(local.service_accounts.gke-prod, null),
try(local.service_accounts.project-factory-prod, null),
])
}
}
module "prod-spoke-vpc" {
source = "../../../modules/net-vpc"
project_id = module.prod-spoke-project.project_id
name = "prod-spoke-0"
mtu = 1500
data_folder = "${var.factories_config.data_dir}/subnets/prod"
delete_default_routes_on_create = true
psa_config = try(var.psa_ranges.prod, null)
# Set explicit routes for googleapis; send everything else to NVAs
routes = {
private-googleapis = {
dest_range = "199.36.153.8/30"
priority = 999
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
restricted-googleapis = {
dest_range = "199.36.153.4/30"
priority = 999
next_hop_type = "gateway"
next_hop = "default-internet-gateway"
}
}
}
module "prod-spoke-firewall" {
source = "../../../modules/net-vpc-firewall"
project_id = module.prod-spoke-project.project_id
network = module.prod-spoke-vpc.name
default_rules_config = {
disabled = true
}
factories_config = {
cidr_tpl_file = "${var.factories_config.data_dir}/cidrs.yaml"
rules_folder = "${var.factories_config.data_dir}/firewall-rules/prod"
}
}
module "peering-prod" {
source = "../../../modules/net-vpc-peering"
prefix = "prod-peering-0"
local_network = module.prod-spoke-vpc.self_link
peer_network = module.landing-trusted-vpc.self_link
export_local_custom_routes = true
export_peer_custom_routes = true
}
# Create delegated grants for stage3 service accounts
resource "google_project_iam_binding" "prod_spoke_project_iam_delegated" {
project = module.prod-spoke-project.project_id
role = "roles/resourcemanager.projectIamAdmin"
members = compact([
try(local.service_accounts.data-platform-prod, null),
try(local.service_accounts.project-factory-prod, null),
try(local.service_accounts.gke-prod, null),
])
condition {
title = "prod_stage3_sa_delegated_grants"
description = "Production host project delegated grants."
expression = format(
"api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
join(",", formatlist("'%s'", local.stage3_sas_delegated_grants))
)
}
}

View File

@ -0,0 +1,245 @@
/**
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description temporary instances for testing
# # Untrusted (Landing)
# module "test-vm-landing-untrusted-primary-0" {
# source = "../../../modules/compute-vm"
# project_id = module.landing-project.project_id
# zone = "${var.regions.primary}-b"
# name = "test-vm-lnd-unt-primary-0"
# network_interfaces = [{
# network = module.landing-untrusted-vpc.self_link
# subnetwork = module.landing-untrusted-vpc.subnet_self_links["${var.regions.primary}/landing-untrusted-default-${local.region_shortnames[var.regions.primary]}"]
# }]
# tags = ["primary", "ssh"]
# boot_disk = {
# initialize_params = {
# image = "projects/debian-cloud/global/images/family/debian-11"
# }
# }
# options = {
# spot = true
# termination_action = "STOP"
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }
# module "test-vm-landing-untrusted-secondary-0" {
# source = "../../../modules/compute-vm"
# project_id = module.landing-project.project_id
# zone = "${var.regions.secondary}-a"
# name = "test-vm-lnd-unt-secondary-0"
# network_interfaces = [{
# network = module.landing-untrusted-vpc.self_link
# subnetwork = module.landing-untrusted-vpc.subnet_self_links["${var.regions.secondary}/landing-untrusted-default-${local.region_shortnames[var.regions.secondary]}"]
# }]
# tags = ["secondary", "ssh"]
# boot_disk = {
# initialize_params = {
# image = "projects/debian-cloud/global/images/family/debian-11"
# }
# }
# options = {
# spot = true
# termination_action = "STOP"
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }
# # Trusted (hub)
# module "test-vm-landing-trusted-primary-0" {
# source = "../../../modules/compute-vm"
# project_id = module.landing-project.project_id
# zone = "${var.regions.primary}-b"
# name = "test-vm-lnd-tru-primary-0"
# network_interfaces = [{
# network = module.landing-trusted-vpc.self_link
# subnetwork = module.landing-trusted-vpc.subnet_self_links["${var.regions.primary}/landing-trusted-default-${local.region_shortnames[var.regions.primary]}"]
# }]
# tags = ["primary", "ssh"]
# boot_disk = {
# initialize_params = {
# image = "projects/debian-cloud/global/images/family/debian-11"
# }
# }
# options = {
# spot = true
# termination_action = "STOP"
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }
# module "test-vm-landing-trusted-secondary-0" {
# source = "../../../modules/compute-vm"
# project_id = module.landing-project.project_id
# zone = "${var.regions.secondary}-a"
# name = "test-vm-lnd-tru-secondary-0"
# network_interfaces = [{
# network = module.landing-trusted-vpc.self_link
# subnetwork = module.landing-trusted-vpc.subnet_self_links["${var.regions.secondary}/landing-trusted-default-${local.region_shortnames[var.regions.secondary]}"]
# }]
# tags = ["secondary", "ssh"]
# boot_disk = {
# initialize_params = {
# image = "projects/debian-cloud/global/images/family/debian-11"
# }
# }
# options = {
# spot = true
# termination_action = "STOP"
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }
# # Dev spoke
# module "test-vm-dev-primary-0" {
# source = "../../../modules/compute-vm"
# project_id = module.dev-spoke-project.project_id
# zone = "${var.regions.primary}-b"
# name = "test-vm-dev-primary-0"
# network_interfaces = [{
# network = module.dev-spoke-vpc.self_link
# # change the subnet name to match the values you are actually using
# subnetwork = module.dev-spoke-vpc.subnet_self_links["${var.regions.primary}/dev-default-${local.region_shortnames[var.regions.primary]}"]
# }]
# tags = ["primary", "ssh"]
# boot_disk = {
# initialize_params = {
# image = "projects/debian-cloud/global/images/family/debian-11"
# }
# }
# options = {
# spot = true
# termination_action = "STOP"
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }
# module "test-vm-dev-secondary-0" {
# source = "../../../modules/compute-vm"
# project_id = module.dev-spoke-project.project_id
# zone = "${var.regions.secondary}-a"
# name = "test-vm-dev-secondary-0"
# network_interfaces = [{
# network = module.dev-spoke-vpc.self_link
# # change the subnet name to match the values you are actually using
# subnetwork = module.dev-spoke-vpc.subnet_self_links["${var.regions.secondary}/dev-default-${local.region_shortnames[var.regions.secondary]}"]
# }]
# tags = ["secondary", "ssh"]
# boot_disk = {
# initialize_params = {
# image = "projects/debian-cloud/global/images/family/debian-11"
# }
# }
# options = {
# spot = true
# termination_action = "STOP"
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }
# # Prod spoke
# module "test-vm-prod-primary-0" {
# source = "../../../modules/compute-vm"
# project_id = module.prod-spoke-project.project_id
# zone = "${var.regions.primary}-b"
# name = "test-vm-prod-primary-0"
# network_interfaces = [{
# network = module.prod-spoke-vpc.self_link
# # change the subnet name to match the values you are actually using
# subnetwork = module.prod-spoke-vpc.subnet_self_links["${var.regions.primary}/prod-default-${local.region_shortnames[var.regions.primary]}"]
# }]
# tags = ["primary", "ssh"]
# boot_disk = {
# initialize_params = {
# image = "projects/debian-cloud/global/images/family/debian-11"
# }
# }
# options = {
# spot = true
# termination_action = "STOP"
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }
# module "test-vm-prod-secondary-0" {
# source = "../../../modules/compute-vm"
# project_id = module.prod-spoke-project.project_id
# zone = "${var.regions.secondary}-a"
# name = "test-vm-prod-secondary-0"
# network_interfaces = [{
# network = module.prod-spoke-vpc.self_link
# # change the subnet name to match the values you are actually using
# subnetwork = module.prod-spoke-vpc.subnet_self_links["${var.regions.secondary}/prod-default-${local.region_shortnames[var.regions.secondary]}"]
# }]
# tags = ["secondary", "ssh"]
# boot_disk = {
# initialize_params = {
# image = "projects/debian-cloud/global/images/family/debian-11"
# }
# }
# options = {
# spot = true
# termination_action = "STOP"
# }
# metadata = {
# startup-script = <<EOF
# apt update
# apt install iputils-ping bind9-dnsutils
# EOF
# }
# }

View File

@ -0,0 +1,283 @@
/**
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "automation" {
# tfdoc:variable:source 0-bootstrap
description = "Automation resources created by the bootstrap stage."
type = object({
outputs_bucket = string
})
}
variable "billing_account" {
# tfdoc:variable:source 0-bootstrap
description = "Billing account id. If billing account is not part of the same org set `is_org_level` to false."
type = object({
id = string
is_org_level = optional(bool, true)
})
validation {
condition = var.billing_account.is_org_level != null
error_message = "Invalid `null` value for `billing_account.is_org_level`."
}
}
variable "custom_roles" {
# tfdoc:variable:source 0-bootstrap
description = "Custom roles defined at the org level, in key => id format."
type = object({
service_project_network_admin = string
})
default = null
}
variable "dns" {
description = "Onprem DNS resolvers."
type = map(list(string))
default = {
onprem = ["10.0.200.3"]
}
}
variable "factories_config" {
description = "Configuration for network resource factories."
type = object({
data_dir = optional(string, "data")
firewall_policy_name = optional(string, "factory")
})
default = {
data_dir = "data"
}
nullable = false
validation {
condition = var.factories_config.data_dir != null
error_message = "Data folder needs to be non-null."
}
validation {
condition = var.factories_config.firewall_policy_name != null
error_message = "Firewall policy name needs to be non-null."
}
}
variable "folder_ids" {
# tfdoc:variable:source 1-resman
description = "Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created."
type = object({
networking = string
networking-dev = string
networking-prod = string
})
}
variable "gcp_ranges" {
description = "GCP address ranges in name => range format."
type = map(string)
default = {
gcp_dev_primary = "10.128.128.0/19"
gcp_dev_secondary = "10.128.160.0/19"
gcp_landing_trusted_primary = "10.128.64.0/19"
gcp_landing_trusted_secondary = "10.128.96.0/19"
gcp_landing_untrusted_primary = "10.128.0.0/19"
gcp_landing_untrusted_secondary = "10.128.32.0/19"
gcp_prod_primary = "10.128.192.0/19"
gcp_prod_secondary = "10.128.224.0/19"
}
}
variable "ncc_asn" {
description = "The NCC Cloud Routers ASN configuration."
type = map(number)
default = {
nva_primary = 64513
nva_secondary = 64514
trusted = 64515
untrusted = 64512
}
}
variable "onprem_cidr" {
description = "Onprem addresses in name => range format."
type = map(string)
default = {
main = "10.0.0.0/24"
}
}
variable "organization" {
# tfdoc:variable:source 0-bootstrap
description = "Organization details."
type = object({
domain = string
id = number
customer_id = string
})
}
variable "outputs_location" {
description = "Path where providers and tfvars files for the following stages are written. Leave empty to disable."
type = string
default = null
}
variable "prefix" {
# tfdoc:variable:source 0-bootstrap
description = "Prefix used for resources that need unique names. Use 9 characters or less."
type = string
validation {
condition = try(length(var.prefix), 0) < 10
error_message = "Use a maximum of 9 characters for prefix."
}
}
variable "psa_ranges" {
description = "IP ranges used for Private Service Access (e.g. CloudSQL). Ranges is in name => range format."
type = object({
dev = object({
ranges = map(string)
routes = object({
export = bool
import = bool
})
})
prod = object({
ranges = map(string)
routes = object({
export = bool
import = bool
})
})
})
default = null
}
variable "regions" {
description = "Region definitions."
type = object({
primary = string
secondary = string
})
default = {
primary = "europe-west1"
secondary = "europe-west4"
}
}
variable "service_accounts" {
# tfdoc:variable:source 1-resman
description = "Automation service accounts in name => email format."
type = object({
data-platform-dev = string
data-platform-prod = string
gke-dev = string
gke-prod = string
project-factory-dev = string
project-factory-prod = string
})
default = null
}
variable "vpn_onprem_primary_config" {
description = "VPN gateway configuration for onprem interconnection in the primary region."
type = object({
peer_external_gateways = map(object({
redundancy_type = string
interfaces = list(string)
}))
router_config = object({
create = optional(bool, true)
asn = number
name = optional(string)
keepalive = optional(number)
custom_advertise = optional(object({
all_subnets = bool
ip_ranges = map(string)
}))
})
tunnels = map(object({
bgp_peer = object({
address = string
asn = number
route_priority = optional(number, 1000)
custom_advertise = optional(object({
all_subnets = bool
all_vpc_subnets = bool
all_peer_vpc_subnets = bool
ip_ranges = map(string)
}))
})
# each BGP session on the same Cloud Router must use a unique /30 CIDR
# from the 169.254.0.0/16 block.
bgp_session_range = string
ike_version = optional(number, 2)
peer_external_gateway_interface = optional(number)
peer_gateway = optional(string, "default")
router = optional(string)
shared_secret = optional(string)
vpn_gateway_interface = number
}))
})
default = null
}
variable "vpn_onprem_secondary_config" {
description = "VPN gateway configuration for onprem interconnection in the secondary region."
type = object({
peer_external_gateways = map(object({
redundancy_type = string
interfaces = list(string)
}))
router_config = object({
create = optional(bool, true)
asn = number
name = optional(string)
keepalive = optional(number)
custom_advertise = optional(object({
all_subnets = bool
ip_ranges = map(string)
}))
})
tunnels = map(object({
bgp_peer = object({
address = string
asn = number
route_priority = optional(number, 1000)
custom_advertise = optional(object({
all_subnets = bool
all_vpc_subnets = bool
all_peer_vpc_subnets = bool
ip_ranges = map(string)
}))
})
# each BGP session on the same Cloud Router must use a unique /30 CIDR
# from the 169.254.0.0/16 block.
bgp_session_range = string
ike_version = optional(number, 2)
peer_external_gateway_interface = optional(number)
peer_gateway = optional(string, "default")
router = optional(string)
shared_secret = optional(string)
vpn_gateway_interface = number
}))
})
default = null
}
variable "zones" {
description = "Zones in which NVAs are deployed."
type = list(string)
default = ["b", "c"]
}

View File

@ -0,0 +1,56 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description VPN between landing and onprem.
locals {
onprem_peer_gateways = {
primary = try(
var.vpn_onprem_primary_config.peer_external_gateways, {}
)
secondary = try(
var.vpn_onprem_secondary_config.peer_external_gateways, {}
)
}
}
module "landing-to-onprem-primary-vpn" {
count = var.vpn_onprem_primary_config == null ? 0 : 1
source = "../../../modules/net-vpn-ha"
project_id = module.landing-project.project_id
network = module.landing-trusted-vpc.self_link
region = var.regions.primary
name = "vpn-to-onprem-${local.region_shortnames[var.regions.primary]}"
router_config = try(var.vpn_onprem_primary_config.router_config, {})
peer_gateways = {
for k, v in local.onprem_peer_gateways.primary : k => { external = v }
}
tunnels = try(var.vpn_onprem_primary_config.tunnels, {})
}
module "landing-to-onprem-secondary-vpn" {
count = var.vpn_onprem_secondary_config == null ? 0 : 1
source = "../../../modules/net-vpn-ha"
project_id = module.landing-project.project_id
network = module.landing-trusted-vpc.self_link
region = var.regions.secondary
name = "vpn-to-onprem-${local.region_shortnames[var.regions.secondary]}"
router_config = try(var.vpn_onprem_secondary_config.router_config, {})
peer_gateways = {
for k, v in local.onprem_peer_gateways.secondary : k => { external = v }
}
tunnels = try(var.vpn_onprem_secondary_config.tunnels, {})
}

View File

@ -28,7 +28,7 @@ The project factory takes care of the following activities:
## How to run this stage
This stage is meant to be executed after "foundational stages" (i.e., stages [`00-bootstrap`](../../0-bootstrap), [`01-resman`](../../1-resman), 02-networking (either [VPN](../../2-networking-b-vpn) or [NVA](../../2-networking-c-nva)) and [`02-security`](../../2-security)) have been run.
This stage is meant to be executed after "foundational stages" (i.e., stages [`00-bootstrap`](../../0-bootstrap), [`01-resman`](../../1-resman), 02-networking (either [VPN](../../2-networking-b-vpn), [NVA](../../2-networking-c-nva), [NVA with BGP support](../../2-networking-e-nva-bgp)) and [`02-security`](../../2-security)) have been run.
It's of course possible to run this stage in isolation, by making sure the architectural prerequisites are satisfied (e.g., networking), and that the Service Account running the stage is granted the roles/permissions below:

View File

@ -176,9 +176,10 @@ In this stage, we will deploy one of the 3 available Hub&Spoke networking topolo
1. VPC Peering
2. HA VPN
3. Multi-NIC appliances (NVA)
4. Multi-NIC appliances (NVA) with NCC / BGP support
```bash
# move to the 02-networking-XXX directory (where XXX should be one of vpn|peering|nva)
# move to the 02-networking-XXX directory (where XXX should be one of a-peering|b-vpn|c-nva|d-separate-envs|e-nva-bgp)
cd $FAST_PWD/2-networking-XXX
# setup providers and variables from previous stages

View File

@ -37,8 +37,8 @@ Implemented via separate stages that configure separate FAST-enabled hierarchie
- [Security](2-security/README.md)
Manages centralized security configurations in a separate stage, and is typically owned by the security team. This stage implements VPC Security Controls via separate perimeters for environments and central services, and creates projects to host centralized KMS keys used by the whole organization. It's meant to be easily extended to include other security-related resources which are required, like Secret Manager.\
Exports: KMS key ids
- Networking ([Peering](2-networking-a-peering/README.md)/[VPN](2-networking-b-vpn/README.md)/[NVA](2-networking-c-nva/README.md)/[Separate environments](2-networking-d-separate-envs/README.md))
Manages centralized network resources in a separate stage, and is typically owned by the networking team. This stage implements a hub-and-spoke design, and includes connectivity via VPN to on-premises, and YAML-based factories for firewall rules (hierarchical and VPC-level) and subnets. It's currently available in four flavors: [spokes connected via VPC peering](2-networking-a-peering/README.md), [spokes connected via VPN](2-networking-b-vpn/README.md), [and spokes connected via appliances](2-networking-c-nva/README.md), and [separated network environments](2-networking-d-separate-envs/README.md).\
- Networking ([Peering](2-networking-a-peering/README.md)/[VPN](2-networking-b-vpn/README.md)/[NVA](2-networking-c-nva/README.md)/[NVA with BGP support](2-networking-e-nva-bgp/README.md)/[Separate environments](2-networking-d-separate-envs/README.md))
Manages centralized network resources in a separate stage, and is typically owned by the networking team. This stage implements a hub-and-spoke design, and includes connectivity via VPN to on-premises, and YAML-based factories for firewall rules (hierarchical and VPC-level) and subnets. It's currently available in four flavors: [spokes connected via VPC peering](2-networking-a-peering/README.md), [spokes connected via VPN](2-networking-b-vpn/README.md), [spokes connected via appliances](2-networking-c-nva/README.md), [spokes connected via appliances leveraging NCC and BGP](2-networking-e-nva-bgp/README.md) and [separated network environments](2-networking-d-separate-envs/README.md).\
Exports: host project ids and numbers, vpc self links
## Environment-level resources (3)

View File

@ -0,0 +1,106 @@
automation = {
outputs_bucket = "test"
}
billing_account = {
id = "000000-111111-222222"
}
custom_roles = {
service_project_network_admin = "organizations/123456789012/roles/foo"
}
folder_ids = {
networking = null
networking-dev = null
networking-prod = null
}
service_accounts = {
data-platform-dev = "string"
data-platform-prod = "string"
gke-dev = "string"
gke-prod = "string"
project-factory-dev = "string"
project-factory-prod = "string"
}
organization = {
domain = "fast.example.com"
id = 123456789012
customer_id = "C00000000"
}
prefix = "fast2"
vpn_onprem_primary_config = {
peer_external_gateways = {
default = {
redundancy_type = "SINGLE_IP_INTERNALLY_REDUNDANT"
interfaces = ["8.8.8.8"]
}
}
router_config = {
asn = 65501
custom_advertise = {
all_subnets = false
ip_ranges = {
"10.1.0.0/16" = "gcp"
"35.199.192.0/19" = "gcp-dns"
"199.36.153.4/30" = "gcp-restricted"
}
}
}
tunnels = {
"0" = {
bgp_peer = {
address = "169.254.1.1"
asn = 65500
}
bgp_session_range = "169.254.1.2/30"
shared_secret = "foo"
vpn_gateway_interface = 0
}
"1" = {
bgp_peer = {
address = "169.254.2.1"
asn = 64513
}
bgp_session_range = "169.254.2.2/30"
shared_secret = "foo"
vpn_gateway_interface = 1
}
}
}
vpn_onprem_secondary_config = {
peer_external_gateways = {
default = {
redundancy_type = "SINGLE_IP_INTERNALLY_REDUNDANT"
interfaces = ["8.8.4.4"]
}
}
router_config = {
asn = 65501
custom_advertise = {
all_subnets = false
ip_ranges = {
"10.1.0.0/16" = "gcp"
"35.199.192.0/19" = "gcp-dns"
"199.36.153.4/30" = "gcp-restricted"
}
}
}
tunnels = {
"0" = {
bgp_peer = {
address = "169.254.1.1"
asn = 65500
}
bgp_session_range = "169.254.3.2/30"
shared_secret = "foo"
vpn_gateway_interface = 0
}
"1" = {
bgp_peer = {
address = "169.254.2.1"
asn = 64513
}
bgp_session_range = "169.254.4.2/30"
shared_secret = "foo"
vpn_gateway_interface = 1
}
}
}

View File

@ -0,0 +1,17 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
counts:
modules: 39
resources: 181

View File

@ -0,0 +1,20 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
module: fast/stages/2-networking-e-nva-bgp
common_tfvars:
- common.tfvars
tests:
stage: