Define and adopt standard IP ranges for FAST networking (#1697)
* Define and adopt standard IP ranges for FAST networking This PR documents and adopts a consistent IP address plan for FAST networking stages Fixes #1644 * Fix documented aggregated ranges for FAST * Fix tests * Fix ip ranges in documentation * Fix NVA stages README
This commit is contained in:
Julio Castillo
GitHub
parent
f628cdbc06
commit
1dfa72cadf
|
|
@ -0,0 +1,39 @@
|
|||
# IP ranges for network stages
|
||||
|
||||
**authors:** [Ludo](https://github.com/ludoo), [Roberto](https://github.com/drebes), [Julio](https://github.com/jccb) \
|
||||
**date:** Sept 20, 2023
|
||||
|
||||
## Status
|
||||
|
||||
Implemented
|
||||
|
||||
## Context
|
||||
|
||||
Adding or changing subnets to networking stages is a mistake-prone process because there is no clear IP plan. The problem was made worse when we began supporting GKE, which requires secondary ranges and a large number of IP addresses for pods and services.
|
||||
|
||||
This was not an issue when there were only a few networking stages, but as FAST expands, it becomes more difficult to keep track of IP ranges for different regions and environments.
|
||||
|
||||
## Decision
|
||||
|
||||
We adopted an IP plan based on regions and environments with the following key points:
|
||||
- Large ranges for the 3 environments we have out of the box (landing, dev, prod)
|
||||
- Support for 2 regions
|
||||
- Leave enough space to easily grow either the number of environments or regions
|
||||
- Allocate large blocks from the CG-NAT range to use as secondary ranges, primarily for GKE pods and services.
|
||||
|
||||
The following table summarizes the agreed IP plan:
|
||||
|
||||
| | aggregate | landing | dev | prod |
|
||||
|----------------------------|--------------:|-------------------------------------------------------------------:|--------------:|--------------:|
|
||||
| Region 1, primary ranges | 10.64.0.0/12 | 10.64.0.0/16<br>Trusted: 10.64.0.0/17<br>Untrusted: 10.64.128.0/17 | 10.68.0.0/16 | 10.72.0.0/16 |
|
||||
| Region 2, primary ranges | 10.80.0.0/12 | 10.80.0.0/16<br>Trusted: 10.80.0.0/17<br>Untrusted: 10.80.128.0/17 | 10.84.0.0/16 | 10.88.0.0/16 |
|
||||
| Region 1, secondary ranges | 100.64.0.0/12 | 100.64.0.0/14 | 100.68.0.0/14 | 100.72.0.0/14 |
|
||||
| Region 2, secondary ranges | 100.80.0.0/12 | 100.80.0.0/14 | 100.84.0.0/16 | 100.88.0.0/14 |
|
||||
|
||||
To allocate additional secondary ranges for GKE clusters:
|
||||
- For the pods range, use the next available /16 in the secondary range of its region/environment pair.
|
||||
- For the service range, use the next available /24 in the last /16 of its region/environment pair.
|
||||
|
||||
## Consequences
|
||||
|
||||
Default subnets for networking stages were updated to reflect to new ranges.
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
region: europe-west1
|
||||
description: Default subnet for dev Data Platform
|
||||
ip_cidr_range: 10.127.48.0/24
|
||||
ip_cidr_range: 10.68.2.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.64.0.0/16
|
||||
services: 100.64.1.0/24
|
||||
pods: 100.69.0.0/16
|
||||
services: 100.71.2.0/24
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.32.0/24
|
||||
ip_cidr_range: 10.68.0.0/24
|
||||
description: Default subnet for dev
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
region: europe-west1
|
||||
description: Default subnet for prod gke nodes
|
||||
ip_cidr_range: 10.127.49.0/24
|
||||
ip_cidr_range: 10.68.1.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.65.0.0/16
|
||||
services: 100.65.1.0/24
|
||||
pods: 100.68.0.0/16
|
||||
services: 100.71.1.0/24
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.0.0/24
|
||||
ip_cidr_range: 10.64.0.0/24
|
||||
description: Default subnet for landing
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.64.0/24
|
||||
ip_cidr_range: 10.72.0.0/24
|
||||
description: Default subnet for prod
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
region: europe-west1
|
||||
description: Default subnet for dev Data Platform
|
||||
ip_cidr_range: 10.127.48.0/24
|
||||
ip_cidr_range: 10.68.2.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.64.0.0/16
|
||||
services: 100.64.1.0/24
|
||||
pods: 100.69.0.0/16
|
||||
services: 100.71.2.0/24
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.32.0/24
|
||||
ip_cidr_range: 10.68.0.0/24
|
||||
description: Default subnet for dev
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
region: europe-west1
|
||||
description: Default subnet for prod gke nodes
|
||||
ip_cidr_range: 10.127.49.0/24
|
||||
ip_cidr_range: 10.68.1.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.65.0.0/16
|
||||
services: 100.65.1.0/24
|
||||
pods: 100.68.0.0/16
|
||||
services: 100.71.1.0/24
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.0.0/24
|
||||
ip_cidr_range: 10.64.0.0/24
|
||||
description: Default subnet for landing
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.64.0/24
|
||||
ip_cidr_range: 10.72.0.0/24
|
||||
description: Default subnet for prod
|
||||
|
|
|
|||
|
|
@ -121,13 +121,13 @@ This is an options summary:
|
|||
|
||||
Minimizing the number of routes (and subnets) in the cloud environment is important, as it simplifies management and it avoids hitting [Cloud Router](https://cloud.google.com/network-connectivity/docs/router/quotas) and [VPC](https://cloud.google.com/vpc/docs/quota) quotas and limits. For this reason, we recommend to carefully plan the IP space used in your cloud environment. This allows the use of larger IP CIDR blocks in routes, whenever possible.
|
||||
|
||||
This stage uses a dedicated /16 block (10.128.0.0/16), which should be sized to the own needs. The subnets created in each VPC derive from this range.
|
||||
This stage uses a dedicated /11 block (10.64.0.0/11), which should be sized to the own needs. The subnets created in each VPC derive from this range.
|
||||
|
||||
The /16 block is evenly split in eight, smaller /19 blocks, assigned to different areas of the GCP network: *landing untrusted europe-west1*, *landing untrusted europe-west4*, *landing trusted europe-west1*, *landing untrusted europe-west4*, *development europe-west1*, *development europe-west4*, *production europe-west1*, *production europe-west4*.
|
||||
The /11 block is evenly split in eight, smaller /16 blocks, assigned to different areas of the GCP network: *landing untrusted europe-west1*, *landing untrusted europe-west4*, *landing trusted europe-west1*, *landing untrusted europe-west4*, *development europe-west1*, *development europe-west4*, *production europe-west1*, *production europe-west4*.
|
||||
|
||||
The first /24 range in every area is allocated for a default subnet, which can be removed or modified as needed.
|
||||
|
||||
Spoke VPCs also define and reserve three "special" CIDR ranges, derived from the respective /19, dedicated to
|
||||
Spoke VPCs also define and reserve three "special" CIDR ranges, derived from their respective /16, dedicated to
|
||||
|
||||
- [PSA (Private Service Access)](https://cloud.google.com/vpc/docs/private-services-access):
|
||||
|
||||
|
|
@ -147,24 +147,24 @@ This is a summary of the subnets allocated by default in this setup:
|
|||
| landing-trusted-default-ew4 | Trusted landing subnet - europe-west4 | 10.128.96.0/24 |
|
||||
| landing-untrusted-default-ew1 | Untrusted landing subnet - europe-west1 | 10.128.0.0/24 |
|
||||
| landing-untrusted-default-ew4 | Untrusted landing subnet - europe-west4 | 10.128.32.0/24 |
|
||||
| dev-default-ew1 | Dev spoke subnet - europe-west1 | 10.128.128.0/24 |
|
||||
| dev-default-ew1 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west1 | 10.128.157.0/24 |
|
||||
| dev-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west1 | 10.128.158.0/24 |
|
||||
| dev-default-ew1 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west1 | 10.128.92.0/24 |
|
||||
| dev-default-ew4 | Dev spoke subnet - europe-west4 | 10.128.160.0/24 |
|
||||
| dev-default-ew4 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west4 | 10.128.189.0/24 |
|
||||
| dev-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west4 | 10.128.190.0/24 |
|
||||
| dev-default-ew4 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west4 | 10.128.93.0/24 |
|
||||
| prod-default-ew1 | Prod spoke subnet - europe-west1 | 10.128.192.0/24 |
|
||||
| prod-default-ew1 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west1 | 10.128.221.0/24 |
|
||||
| prod-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west1 | 10.128.253.0/24 |
|
||||
| prod-default-ew1 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west1 | 10.128.60.0/24 |
|
||||
| prod-default-ew4 | Prod spoke subnet - europe-west4 | 10.128.224.0/24 |
|
||||
| prod-default-ew4 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west4 | 10.128.222.0/24 |
|
||||
| prod-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west4 | 10.128.254.0/24 |
|
||||
| prod-default-ew4 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west4 | 10.128.61.0/24 |
|
||||
| dev-default-ew1 | Dev spoke subnet - europe-west1 | 10.68.0.0/24 |
|
||||
| dev-default-ew1 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west1 | 10.68.253.0/24 |
|
||||
| dev-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west1 | 10.68.254.0/24 |
|
||||
| dev-default-ew1 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west1 | 10.68.255.0/24 |
|
||||
| dev-default-ew4 | Dev spoke subnet - europe-west4 | 10.84.0.0/24 |
|
||||
| dev-default-ew4 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west4 | 10.84.253.0/24 |
|
||||
| dev-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west4 | 10.84.254.0/24 |
|
||||
| dev-default-ew4 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west4 | 10.84.255.0/24 |
|
||||
| prod-default-ew1 | Prod spoke subnet - europe-west1 | 10.72.0.0/24 |
|
||||
| prod-default-ew1 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west1 | 10.72.253.0/24 |
|
||||
| prod-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west1 | 10.72.254.0/24 |
|
||||
| prod-default-ew1 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west1 | 10.72.255.0/24 |
|
||||
| prod-default-ew4 | Prod spoke subnet - europe-west4 | 10.88.0.0/24 |
|
||||
| prod-default-ew4 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west4 | 10.88.253.0/24 |
|
||||
| prod-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west4 | 10.88.254.0/24 |
|
||||
| prod-default-ew4 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west4 | 10.88.255.0/24 |
|
||||
|
||||
These subnets are advertised to on-premises as a whole /16 range (10.128.0.0/16).
|
||||
These subnets can advertised to on-premises as an aggregate /11 range (10.64.0.0/11). Refer to the `var.vpn_onprem_primary_config.router_config` and `var.vpn_onprem_secondary_config.router_config` variables to configure it.
|
||||
|
||||
Routes in GCP are either automatically created (for example, when a subnet is added to a VPC), manually created via static routes, dynamically exchanged through VPC peerings, or dynamically programmed by [Cloud Routers](https://cloud.google.com/network-connectivity/docs/router#docs) when a BGP session is established. BGP sessions can be configured to advertise VPC ranges, and/or custom ranges via custom advertisements.
|
||||
|
||||
|
|
@ -485,7 +485,7 @@ DNS configurations are centralised in the `dns-*.tf` files. Spokes delegate DNS
|
|||
| [custom_roles](variables.tf#L63) | Custom roles defined at the org level, in key => id format. | <code title="object({ service_project_network_admin = string })">object({…})</code> | | <code>null</code> | <code>0-bootstrap</code> |
|
||||
| [dns](variables.tf#L72) | Onprem DNS resolvers. | <code>map(list(string))</code> | | <code title="{ onprem = ["10.0.200.3"] }">{…}</code> | |
|
||||
| [factories_config](variables.tf#L80) | Configuration for network resource factories. | <code title="object({ data_dir = optional(string, "data") dns_policy_rules_file = optional(string, "data/dns-policy-rules.yaml") firewall_policy_name = optional(string, "net-default") })">object({…})</code> | | <code title="{ data_dir = "data" }">{…}</code> | |
|
||||
| [gcp_ranges](variables.tf#L111) | GCP address ranges in name => range format. | <code>map(string)</code> | | <code title="{ gcp_dev_primary = "10.128.128.0/19" gcp_dev_secondary = "10.128.160.0/19" gcp_landing_trusted_primary = "10.128.64.0/19" gcp_landing_trusted_secondary = "10.128.96.0/19" gcp_landing_untrusted_primary = "10.128.0.0/19" gcp_landing_untrusted_secondary = "10.128.32.0/19" gcp_prod_primary = "10.128.192.0/19" gcp_prod_secondary = "10.128.224.0/19" }">{…}</code> | |
|
||||
| [gcp_ranges](variables.tf#L111) | GCP address ranges in name => range format. | <code>map(string)</code> | | <code title="{ gcp_dev_primary = "10.68.0.0/16" gcp_dev_secondary = "10.84.0.0/16" gcp_landing_trusted_primary = "10.64.0.0/17" gcp_landing_trusted_secondary = "10.80.0.0/17" gcp_landing_untrusted_primary = "10.64.127.0/17" gcp_landing_untrusted_secondary = "10.80.127.0/17" gcp_prod_primary = "10.72.0.0/16" gcp_prod_secondary = "10.88.0.0/16" }">{…}</code> | |
|
||||
| [onprem_cidr](variables.tf#L126) | Onprem addresses in name => range format. | <code>map(string)</code> | | <code title="{ main = "10.0.0.0/24" }">{…}</code> | |
|
||||
| [outputs_location](variables.tf#L144) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | <code>string</code> | | <code>null</code> | |
|
||||
| [psa_ranges](variables.tf#L161) | IP ranges used for Private Service Access (e.g. CloudSQL). Ranges is in name => range format. | <code title="object({ dev = object({ ranges = map(string) export_routes = optional(bool, false) import_routes = optional(bool, false) }) prod = object({ ranges = map(string) export_routes = optional(bool, false) import_routes = optional(bool, false) }) })">object({…})</code> | | <code>null</code> | |
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
region: europe-west1
|
||||
description: Default subnet for dev Data Platform
|
||||
ip_cidr_range: 10.127.48.0/24
|
||||
ip_cidr_range: 10.68.2.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.64.0.0/16
|
||||
services: 100.64.1.0/24
|
||||
pods: 100.69.0.0/16
|
||||
services: 100.71.2.0/24
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.128.0/24
|
||||
ip_cidr_range: 10.68.0.0/24
|
||||
description: Default europe-west1 subnet for dev
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west4
|
||||
ip_cidr_range: 10.128.160.0/24
|
||||
ip_cidr_range: 10.84.0.0/24
|
||||
description: Default europe-west4 subnet for dev
|
||||
|
|
|
|||
|
|
@ -0,0 +1,8 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
description: Default subnet for prod gke nodes
|
||||
ip_cidr_range: 10.68.1.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.68.0.0/16
|
||||
services: 100.71.1.0/24
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.64.0/24
|
||||
ip_cidr_range: 10.64.0.0/24
|
||||
description: Default europe-west1 subnet for landing trusted
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west4
|
||||
ip_cidr_range: 10.128.96.0/24
|
||||
ip_cidr_range: 10.80.0.0/24
|
||||
description: Default europe-west4 subnet for landing trusted
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.0.0/24
|
||||
ip_cidr_range: 10.64.128.0/24
|
||||
description: Default europe-west1 subnet for landing untrusted
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west4
|
||||
ip_cidr_range: 10.128.32.0/24
|
||||
ip_cidr_range: 10.80.128.0/24
|
||||
description: Default europe-west4 subnet for landing untrusted
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.192.0/24
|
||||
ip_cidr_range: 10.72.0.0/24
|
||||
description: Default europe-west1 subnet for prod
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west4
|
||||
ip_cidr_range: 10.128.224.0/24
|
||||
ip_cidr_range: 10.88.0.0/24
|
||||
description: Default europe-west4 subnet for prod
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
|
|
@ -112,14 +112,14 @@ variable "gcp_ranges" {
|
|||
description = "GCP address ranges in name => range format."
|
||||
type = map(string)
|
||||
default = {
|
||||
gcp_dev_primary = "10.128.128.0/19"
|
||||
gcp_dev_secondary = "10.128.160.0/19"
|
||||
gcp_landing_trusted_primary = "10.128.64.0/19"
|
||||
gcp_landing_trusted_secondary = "10.128.96.0/19"
|
||||
gcp_landing_untrusted_primary = "10.128.0.0/19"
|
||||
gcp_landing_untrusted_secondary = "10.128.32.0/19"
|
||||
gcp_prod_primary = "10.128.192.0/19"
|
||||
gcp_prod_secondary = "10.128.224.0/19"
|
||||
gcp_dev_primary = "10.68.0.0/16"
|
||||
gcp_dev_secondary = "10.84.0.0/16"
|
||||
gcp_landing_trusted_primary = "10.64.0.0/17"
|
||||
gcp_landing_trusted_secondary = "10.80.0.0/17"
|
||||
gcp_landing_untrusted_primary = "10.64.127.0/17"
|
||||
gcp_landing_untrusted_secondary = "10.80.127.0/17"
|
||||
gcp_prod_primary = "10.72.0.0/16"
|
||||
gcp_prod_secondary = "10.88.0.0/16"
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
region: europe-west1
|
||||
description: Default subnet for dev Data Platform
|
||||
ip_cidr_range: 10.127.48.0/24
|
||||
ip_cidr_range: 10.68.2.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.64.0.0/16
|
||||
services: 100.64.1.0/24
|
||||
pods: 100.69.0.0/16
|
||||
services: 100.71.2.0/24
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.32.0/24
|
||||
ip_cidr_range: 10.68.0.0/24
|
||||
description: Default subnet for dev
|
||||
|
|
|
|||
|
|
@ -0,0 +1,8 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
description: Default subnet for prod gke nodes
|
||||
ip_cidr_range: 10.68.1.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.68.0.0/16
|
||||
services: 100.71.1.0/24
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.64.0/24
|
||||
ip_cidr_range: 10.72.0.0/24
|
||||
description: Default subnet for prod
|
||||
|
|
|
|||
|
|
@ -144,13 +144,13 @@ This is an options summary:
|
|||
|
||||
Minimizing the number of routes (and subnets) in the cloud environment is important, as it simplifies management and it avoids hitting [Cloud Router](https://cloud.google.com/network-connectivity/docs/router/quotas) and [VPC](https://cloud.google.com/vpc/docs/quota) quotas and limits. For this reason, we recommend to carefully plan the IP space used in your cloud environment. This allows the use of larger IP CIDR blocks in routes, whenever possible.
|
||||
|
||||
This stage uses a dedicated /16 block (10.128.0.0/16), which should be sized to the own needs. The subnets created in each VPC derive from this range.
|
||||
This stage uses a dedicated /11 block (10.64.0.0/11), which should be sized to the own needs. The subnets created in each VPC derive from this range.
|
||||
|
||||
The /16 block is evenly split in eight, smaller /19 blocks, assigned to different areas of the GCP network: *landing untrusted europe-west1*, *landing untrusted europe-west4*, *landing trusted europe-west1*, *landing untrusted europe-west4*, *development europe-west1*, *development europe-west4*, *production europe-west1*, *production europe-west4*.
|
||||
The /11 block is evenly split in eight, smaller /16 blocks, assigned to different areas of the GCP network: *landing untrusted europe-west1*, *landing untrusted europe-west4*, *landing trusted europe-west1*, *landing untrusted europe-west4*, *development europe-west1*, *development europe-west4*, *production europe-west1*, *production europe-west4*.
|
||||
|
||||
The first /24 range in every area is allocated for a default subnet, which can be removed or modified as needed.
|
||||
|
||||
Spoke VPCs also define and reserve three "special" CIDR ranges, derived from the respective /19, dedicated to
|
||||
Spoke VPCs also define and reserve three "special" CIDR ranges, derived from their respective /16, dedicated to
|
||||
|
||||
- [PSA (Private Service Access)](https://cloud.google.com/vpc/docs/private-services-access):
|
||||
|
||||
|
|
@ -170,24 +170,24 @@ This is a summary of the subnets allocated by default in this setup:
|
|||
| landing-trusted-default-ew4 | Trusted landing subnet - europe-west4 | 10.128.96.0/24 |
|
||||
| landing-untrusted-default-ew1 | Untrusted landing subnet - europe-west1 | 10.128.0.0/24 |
|
||||
| landing-untrusted-default-ew4 | Untrusted landing subnet - europe-west4 | 10.128.32.0/24 |
|
||||
| dev-default-ew1 | Dev spoke subnet - europe-west1 | 10.128.128.0/24 |
|
||||
| dev-default-ew1 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west1 | 10.128.157.0/24 |
|
||||
| dev-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west1 | 10.128.158.0/24 |
|
||||
| dev-default-ew1 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west1 | 10.128.92.0/24 |
|
||||
| dev-default-ew4 | Dev spoke subnet - europe-west4 | 10.128.160.0/24 |
|
||||
| dev-default-ew4 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west4 | 10.128.189.0/24 |
|
||||
| dev-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west4 | 10.128.190.0/24 |
|
||||
| dev-default-ew4 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west4 | 10.128.93.0/24 |
|
||||
| prod-default-ew1 | Prod spoke subnet - europe-west1 | 10.128.192.0/24 |
|
||||
| prod-default-ew1 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west1 | 10.128.221.0/24 |
|
||||
| prod-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west1 | 10.128.253.0/24 |
|
||||
| prod-default-ew1 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west1 | 10.128.60.0/24 |
|
||||
| prod-default-ew4 | Prod spoke subnet - europe-west4 | 10.128.224.0/24 |
|
||||
| prod-default-ew4 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west4 | 10.128.222.0/24 |
|
||||
| prod-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west4 | 10.128.254.0/24 |
|
||||
| prod-default-ew4 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west4 | 10.128.61.0/24 |
|
||||
| dev-default-ew1 | Dev spoke subnet - europe-west1 | 10.68.0.0/24 |
|
||||
| dev-default-ew1 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west1 | 10.68.253.0/24 |
|
||||
| dev-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west1 | 10.68.254.0/24 |
|
||||
| dev-default-ew1 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west1 | 10.68.255.0/24 |
|
||||
| dev-default-ew4 | Dev spoke subnet - europe-west4 | 10.84.0.0/24 |
|
||||
| dev-default-ew4 (PSA MySQL) | PSA subnet for MySQL in dev spoke - europe-west4 | 10.84.253.0/24 |
|
||||
| dev-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in dev spoke - europe-west4 | 10.84.254.0/24 |
|
||||
| dev-default-ew4 (L7 ILB) | L7 ILB subnet for dev spoke - europe-west4 | 10.84.255.0/24 |
|
||||
| prod-default-ew1 | Prod spoke subnet - europe-west1 | 10.72.0.0/24 |
|
||||
| prod-default-ew1 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west1 | 10.72.253.0/24 |
|
||||
| prod-default-ew1 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west1 | 10.72.254.0/24 |
|
||||
| prod-default-ew1 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west1 | 10.72.255.0/24 |
|
||||
| prod-default-ew4 | Prod spoke subnet - europe-west4 | 10.88.0.0/24 |
|
||||
| prod-default-ew4 (PSA MySQL) | PSA subnet for MySQL in prod spoke - europe-west4 | 10.88.253.0/24 |
|
||||
| prod-default-ew4 (PSA SQL Server) | PSA subnet for Postgres in prod spoke - europe-west4 | 10.88.254.0/24 |
|
||||
| prod-default-ew4 (L7 ILB) | L7 ILB subnet for prod spoke - europe-west4 | 10.88.255.0/24 |
|
||||
|
||||
These subnets are advertised to on-premises as a whole /16 range (10.128.0.0/16).
|
||||
These subnets can advertised to on-premises as an aggregate /11 range (10.64.0.0/11). Refer to the `var.vpn_onprem_primary_config.router_config` and `var.vpn_onprem_secondary_config.router_config` variables to configure it.
|
||||
|
||||
Routes in GCP are either automatically created (for example, when a subnet is added to a VPC), manually created via static routes, dynamically exchanged through VPC peerings, or dynamically programmed by [Cloud Routers](https://cloud.google.com/network-connectivity/docs/router#docs) when a BGP session is established. BGP sessions can be configured to advertise VPC ranges, and/or custom ranges via custom advertisements.
|
||||
|
||||
|
|
@ -511,7 +511,7 @@ DNS configurations are centralised in the `dns-*.tf` files. Spokes delegate DNS
|
|||
| [custom_roles](variables.tf#L63) | Custom roles defined at the org level, in key => id format. | <code title="object({ service_project_network_admin = string })">object({…})</code> | | <code>null</code> | <code>0-bootstrap</code> |
|
||||
| [dns](variables.tf#L72) | Onprem DNS resolvers. | <code>map(list(string))</code> | | <code title="{ onprem = ["10.0.200.3"] }">{…}</code> | |
|
||||
| [factories_config](variables.tf#L80) | Configuration for network resource factories. | <code title="object({ data_dir = optional(string, "data") dns_policy_rules_file = optional(string, "data/dns-policy-rules.yaml") firewall_policy_name = optional(string, "net-default") })">object({…})</code> | | <code title="{ data_dir = "data" }">{…}</code> | |
|
||||
| [gcp_ranges](variables.tf#L111) | GCP address ranges in name => range format. | <code>map(string)</code> | | <code title="{ gcp_dev_primary = "10.128.128.0/19" gcp_dev_secondary = "10.128.160.0/19" gcp_landing_trusted_primary = "10.128.64.0/19" gcp_landing_trusted_secondary = "10.128.96.0/19" gcp_landing_untrusted_primary = "10.128.0.0/19" gcp_landing_untrusted_secondary = "10.128.32.0/19" gcp_prod_primary = "10.128.192.0/19" gcp_prod_secondary = "10.128.224.0/19" }">{…}</code> | |
|
||||
| [gcp_ranges](variables.tf#L111) | GCP address ranges in name => range format. | <code>map(string)</code> | | <code title="{ gcp_dev_primary = "10.68.0.0/16" gcp_dev_secondary = "10.84.0.0/16" gcp_landing_trusted_primary = "10.64.0.0/17" gcp_landing_trusted_secondary = "10.80.0.0/17" gcp_landing_untrusted_primary = "10.64.127.0/17" gcp_landing_untrusted_secondary = "10.80.127.0/17" gcp_prod_primary = "10.72.0.0/16" gcp_prod_secondary = "10.88.0.0/16" }">{…}</code> | |
|
||||
| [ncc_asn](variables.tf#L126) | The NCC Cloud Routers ASN configuration. | <code>map(number)</code> | | <code title="{ nva_primary = 64513 nva_secondary = 64514 trusted = 64515 untrusted = 64512 }">{…}</code> | |
|
||||
| [onprem_cidr](variables.tf#L137) | Onprem addresses in name => range format. | <code>map(string)</code> | | <code title="{ main = "10.0.0.0/24" }">{…}</code> | |
|
||||
| [outputs_location](variables.tf#L155) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | <code>string</code> | | <code>null</code> | |
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
region: europe-west1
|
||||
description: Default subnet for dev Data Platform
|
||||
ip_cidr_range: 10.127.48.0/24
|
||||
ip_cidr_range: 10.68.2.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.64.0.0/16
|
||||
services: 100.64.1.0/24
|
||||
pods: 100.69.0.0/16
|
||||
services: 100.71.2.0/24
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.128.0/24
|
||||
ip_cidr_range: 10.68.0.0/24
|
||||
description: Default europe-west1 subnet for dev
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west4
|
||||
ip_cidr_range: 10.128.160.0/24
|
||||
ip_cidr_range: 10.84.0.0/24
|
||||
description: Default europe-west4 subnet for dev
|
||||
|
|
|
|||
|
|
@ -0,0 +1,8 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
description: Default subnet for prod gke nodes
|
||||
ip_cidr_range: 10.68.1.0/24
|
||||
secondary_ip_ranges:
|
||||
pods: 100.68.0.0/16
|
||||
services: 100.71.1.0/24
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.64.0/24
|
||||
ip_cidr_range: 10.64.0.0/24
|
||||
description: Default europe-west1 subnet for landing trusted
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west4
|
||||
ip_cidr_range: 10.128.96.0/24
|
||||
ip_cidr_range: 10.80.0.0/24
|
||||
description: Default europe-west4 subnet for landing trusted
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.0.0/24
|
||||
ip_cidr_range: 10.64.128.0/24
|
||||
description: Default europe-west1 subnet for landing untrusted
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west4
|
||||
ip_cidr_range: 10.128.32.0/24
|
||||
ip_cidr_range: 10.80.128.0/24
|
||||
description: Default europe-west4 subnet for landing untrusted
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west1
|
||||
ip_cidr_range: 10.128.192.0/24
|
||||
ip_cidr_range: 10.72.0.0/24
|
||||
description: Default europe-west1 subnet for prod
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
# skip boilerplate check
|
||||
|
||||
region: europe-west4
|
||||
ip_cidr_range: 10.128.224.0/24
|
||||
ip_cidr_range: 10.88.0.0/24
|
||||
description: Default europe-west4 subnet for prod
|
||||
|
|
|
|||
|
|
@ -112,14 +112,14 @@ variable "gcp_ranges" {
|
|||
description = "GCP address ranges in name => range format."
|
||||
type = map(string)
|
||||
default = {
|
||||
gcp_dev_primary = "10.128.128.0/19"
|
||||
gcp_dev_secondary = "10.128.160.0/19"
|
||||
gcp_landing_trusted_primary = "10.128.64.0/19"
|
||||
gcp_landing_trusted_secondary = "10.128.96.0/19"
|
||||
gcp_landing_untrusted_primary = "10.128.0.0/19"
|
||||
gcp_landing_untrusted_secondary = "10.128.32.0/19"
|
||||
gcp_prod_primary = "10.128.192.0/19"
|
||||
gcp_prod_secondary = "10.128.224.0/19"
|
||||
gcp_dev_primary = "10.68.0.0/16"
|
||||
gcp_dev_secondary = "10.84.0.0/16"
|
||||
gcp_landing_trusted_primary = "10.64.0.0/17"
|
||||
gcp_landing_trusted_secondary = "10.80.0.0/17"
|
||||
gcp_landing_untrusted_primary = "10.64.127.0/17"
|
||||
gcp_landing_untrusted_secondary = "10.80.127.0/17"
|
||||
gcp_prod_primary = "10.72.0.0/16"
|
||||
gcp_prod_secondary = "10.88.0.0/16"
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -14,4 +14,4 @@
|
|||
|
||||
counts:
|
||||
modules: 42
|
||||
resources: 200
|
||||
resources: 201
|
||||
|
|
|
|||
|
|
@ -14,4 +14,4 @@
|
|||
|
||||
counts:
|
||||
modules: 21
|
||||
resources: 170
|
||||
resources: 171
|
||||
|
|
|
|||
|
|
@ -14,4 +14,4 @@
|
|||
|
||||
counts:
|
||||
modules: 36
|
||||
resources: 211
|
||||
resources: 212
|
||||
|
|
|
|||
Loading…
Reference in New Issue