diff --git a/.gitignore b/.gitignore index 38524a05..b0522066 100644 --- a/.gitignore +++ b/.gitignore @@ -21,7 +21,8 @@ bundle.zip **/*.pkrvars.hcl fixture_* fast/configs -fast/stages/**/providers.tf +fast/stages/**/*providers.tf fast/stages/**/terraform.tfvars fast/stages/**/terraform.tfvars.json fast/stages/**/terraform-*.auto.tfvars.json +fast/stages/**/0*.auto.tfvars* diff --git a/fast/stages/00-bootstrap/README.md b/fast/stages/00-bootstrap/README.md index 0add953b..b5331188 100644 --- a/fast/stages/00-bootstrap/README.md +++ b/fast/stages/00-bootstrap/README.md @@ -192,7 +192,7 @@ outputs_location = "../../fast-config" ### Output files and cross-stage variables -At any time during the life of this stage, you can configure it to automatically generate provider configurations and variable files for the following, to simplify exchanging inputs and outputs between stages and avoid having to edit files manually. +At any time during the life of this stage, you can configure it to automatically generate provider configurations and variable files consumed by the following stages, to simplify passing outputs to input variables by not having to edit files manually. Automatic generation of files is disabled by default. To enable the mechanism, set the `outputs_location` variable to a valid path on a local filesystem, e.g. @@ -202,27 +202,23 @@ outputs_location = "../../config" Once the variable is set, `apply` will generate and manage providers and variables files, including the initial one used for this stage after the first run. You can then link these files in the relevant stages, instead of manually transfering outputs from one stage, to Terraform variables in another. -Below is the outline of the output files generated by this stage: +Below is the outline of the output files generated by all stages: ```bash [path specified in outputs_location] -├── 00-bootstrap -│   ├── providers.tf -├── 01-resman -│   ├── providers.tf -│   ├── terraform-bootstrap.auto.tfvars.json -├── 02-networking -│   ├── terraform-bootstrap.auto.tfvars.json -├── 02-security -│   ├── terraform-bootstrap.auto.tfvars.json -├── 03-gke-multitenant-dev -│   └── terraform-bootstrap.auto.tfvars.json -├── 03-gke-multitenant-prod -│   └── terraform-bootstrap.auto.tfvars.json -├── 03-project-factory-dev -│   └── terraform-bootstrap.auto.tfvars.json -├── 03-project-factory-prod -│   └── terraform-bootstrap.auto.tfvars.json +├── providers +│   ├── 00-bootstrap-providers.tf +│   ├── 01-resman-providers.tf +│   ├── 02-networking-providers.tf +│   ├── 02-security-providers.tf +│   ├── 03-project-factory-dev-providers.tf +│   ├── 03-project-factory-prod-providers.tf +│   └── 99-sandbox-providers.tf +└── tfvars + ├── 00-bootstrap.auto.tfvars.json + ├── 01-resman.auto.tfvars.json + ├── 02-networking.auto.tfvars.json + └── 02-security.auto.tfvars.json ``` ### Running the stage @@ -241,7 +237,7 @@ Once the initial `apply` completes successfully, configure a remote backend usin ```bash # if using output files via the outputs_location and set to `../../config` -ln -s ../../config/00-bootstrap/* ./ +ln -s ../../config/providers/00-bootstrap* ./ # or from outputs if not using output files terraform output -json providers | jq -r '.["00-bootstrap"]' \ > providers.tf @@ -350,9 +346,10 @@ Names used in internal references (e.g. `module.foo-prod.id`) are only used by T | name | description | sensitive | consumers | |---|---|:---:|---| -| [billing_dataset](outputs.tf#L89) | BigQuery dataset prepared for billing export. | | | -| [project_ids](outputs.tf#L94) | Projects created by this stage. | | | -| [providers](outputs.tf#L105) | Terraform provider files for this stage and dependent stages. | ✓ | stage-01 | -| [tfvars](outputs.tf#L114) | Terraform variable files for the following stages. | ✓ | | +| [billing_dataset](outputs.tf#L58) | BigQuery dataset prepared for billing export. | | | +| [custom_roles](outputs.tf#L63) | Organization-level custom roles. | | | +| [project_ids](outputs.tf#L68) | Projects created by this stage. | | | +| [providers](outputs.tf#L79) | Terraform provider files for this stage and dependent stages. | ✓ | stage-01 | +| [tfvars](outputs.tf#L88) | Terraform variable files for the following stages. | ✓ | | diff --git a/fast/stages/00-bootstrap/outputs.tf b/fast/stages/00-bootstrap/outputs.tf index d074d91f..46677b6a 100644 --- a/fast/stages/00-bootstrap/outputs.tf +++ b/fast/stages/00-bootstrap/outputs.tf @@ -15,7 +15,7 @@ */ locals { - _custom_roles = { + custom_roles = { for k, v in var.custom_role_names : k => module.organization.custom_role_id[v] } @@ -32,56 +32,25 @@ locals { }) } tfvars = { - "01-resman" = jsonencode({ - automation_project_id = module.automation-project.project_id - billing_account = var.billing_account - custom_roles = local._custom_roles - groups = var.groups - organization = var.organization - prefix = var.prefix - }) - "02-networking" = jsonencode({ - billing_account_id = var.billing_account.id - custom_roles = local._custom_roles - organization = var.organization - prefix = var.prefix - }) - "02-security" = jsonencode({ - billing_account_id = var.billing_account.id - organization = var.organization - prefix = var.prefix - }) - "03-gke-multitenant-dev" = jsonencode({ - billing_account_id = var.billing_account.id - prefix = var.prefix - }) - "03-gke-multitenant-prod" = jsonencode({ - billing_account_id = var.billing_account.id - prefix = var.prefix - }) - "03-project-factory-dev" = jsonencode({ - billing_account_id = var.billing_account.id - prefix = var.prefix - }) - "03-project-factory-prod" = jsonencode({ - billing_account_id = var.billing_account.id - prefix = var.prefix - }) + automation_project_id = module.automation-project.project_id + custom_roles = local.custom_roles } } # optionally generate providers and tfvars files for subsequent stages resource "local_file" "providers" { - for_each = var.outputs_location == null ? {} : local.providers - filename = "${pathexpand(var.outputs_location)}/${each.key}/providers.tf" - content = each.value + for_each = var.outputs_location == null ? {} : local.providers + file_permission = "0644" + filename = "${pathexpand(var.outputs_location)}/providers/${each.key}-providers.tf" + content = each.value } resource "local_file" "tfvars" { - for_each = var.outputs_location == null ? {} : local.tfvars - filename = "${pathexpand(var.outputs_location)}/${each.key}/terraform-bootstrap.auto.tfvars.json" - content = each.value + for_each = var.outputs_location == null ? {} : { 1 = 1 } + file_permission = "0644" + filename = "${pathexpand(var.outputs_location)}/tfvars/00-bootstrap.auto.tfvars.json" + content = jsonencode(local.tfvars) } # outputs @@ -91,6 +60,11 @@ output "billing_dataset" { value = try(module.billing-export-dataset.0.id, null) } +output "custom_roles" { + description = "Organization-level custom roles." + value = local.custom_roles +} + output "project_ids" { description = "Projects created by this stage." value = { diff --git a/fast/stages/01-resman/README.md b/fast/stages/01-resman/README.md index 8ff41f7d..ec7d8109 100644 --- a/fast/stages/01-resman/README.md +++ b/fast/stages/01-resman/README.md @@ -50,11 +50,11 @@ The default way of making sure you have the right permissions, is to use the ide To simplify setup, the previous stage pre-configures a valid providers file in its output, and optionally writes it to a local file if the `outputs_location` variable is set to a valid path. -If you have set a valid value for `outputs_location` in the bootstrap stage, simply link the relevant `providers.tf` file from this stage's folder in the path you specified: +If you have set a valid value for `outputs_location` in the bootstrap stage (see the [bootstrap stage README](../00-bootstrap/#output-files-and-cross-stage-variables) for more details), simply link the relevant `providers.tf` file from this stage's folder in the path you specified: ```bash -# `outputs_location` is set to `../../config` -ln -s ../../config/01-resman/providers.tf +# `outputs_location` is set to `~/config` +ln -s ~/config/providers/01-resman* ./ ``` If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage's outputs: @@ -76,14 +76,16 @@ There are two broad sets of variables you will need to fill in: To avoid the tedious job of filling in the first group of variable with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files. -If you configured a valid path for `outputs_location` in the bootstrap stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's outputs folder (under the path you specified), where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is avalaible: +If you configured a valid path for `outputs_location` in the bootstrap stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's outputs folder. For this stage, you need the `.tfvars` file compiled manually for the bootstrap stage, and the one generated by it: ```bash -# `outputs_location` is set to `../../config` -ln -s ../../config/01-resman/terraform-bootstrap.auto.tfvars.json +# `outputs_location` is set to `~/config` +ln -s ../../config/tfvars/00*.json ./ +# also copy the tfvars file used for the bootstrap stage +cp ../00-bootstrap/terraform.tfvars ./ ``` -A second set of variables is specific to this stage, they are all optional so if you need to customize them, create an extra `terraform.tfvars` file. +A second set of variables is specific to this stage, they are all optional so if you need to customize them, create an extra `terraform.tfvars` file or add them to the file copied from bootstrap. Refer to the [Variables](#variables) table at the bottom of this document, for a full list of variables, their origin (e.g. a stage or specific to this one), and descriptions explaining their meaning. The sections below also describe some of the possible customizations. For billing configurations, refer to the [Bootstrap documentation on billing](../00-bootstrap/README.md#billing-account) as the `billing_account` variable is identical across all stages. @@ -163,8 +165,8 @@ Due to its simplicity, this stage lends itself easily to customizations: adding | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| -| [automation_project_id](variables.tf#L29) | Project id for the automation project created by the bootstrap stage. | string | ✓ | | 00-bootstrap | -| [billing_account](variables.tf#L20) | Billing account id and organization id ('nnnnnnnn' or null). | object({…}) | ✓ | | 00-bootstrap | +| [automation_project_id](variables.tf#L20) | Project id for the automation project created by the bootstrap stage. | string | ✓ | | 00-bootstrap | +| [billing_account](variables.tf#L26) | Billing account id and organization id ('nnnnnnnn' or null). | object({…}) | ✓ | | 00-bootstrap | | [organization](variables.tf#L57) | Organization details. | object({…}) | ✓ | | 00-bootstrap | | [prefix](variables.tf#L81) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 00-bootstrap | | [custom_roles](variables.tf#L35) | Custom roles defined at the org level, in key => id format. | map(string) | | {} | 00-bootstrap | @@ -177,12 +179,12 @@ Due to its simplicity, this stage lends itself easily to customizations: adding | name | description | sensitive | consumers | |---|---|:---:|---| -| [networking](outputs.tf#L83) | Data for the networking stage. | | 02-networking | -| [project_factories](outputs.tf#L93) | Data for the project factories stage. | | xx-teams | -| [providers](outputs.tf#L110) | Terraform provider files for this stage and dependent stages. | ✓ | 02-networking · 02-security · xx-sandbox · xx-teams | -| [sandbox](outputs.tf#L117) | Data for the sandbox stage. | | xx-sandbox | -| [security](outputs.tf#L127) | Data for the networking stage. | | 02-security | -| [teams](outputs.tf#L137) | Data for the teams stage. | | | -| [tfvars](outputs.tf#L150) | Terraform variable files for the following stages. | ✓ | | +| [networking](outputs.tf#L101) | Data for the networking stage. | | | +| [project_factories](outputs.tf#L110) | Data for the project factories stage. | | | +| [providers](outputs.tf#L126) | Terraform provider files for this stage and dependent stages. | ✓ | 02-networking · 02-security · xx-sandbox · xx-teams | +| [sandbox](outputs.tf#L133) | Data for the sandbox stage. | | xx-sandbox | +| [security](outputs.tf#L143) | Data for the networking stage. | | 02-security | +| [teams](outputs.tf#L153) | Data for the teams stage. | | | +| [tfvars](outputs.tf#L166) | Terraform variable files for the following stages. | ✓ | | diff --git a/fast/stages/01-resman/outputs.tf b/fast/stages/01-resman/outputs.tf index 23056119..e2f7e0b5 100644 --- a/fast/stages/01-resman/outputs.tf +++ b/fast/stages/01-resman/outputs.tf @@ -15,10 +15,25 @@ */ locals { - _project_factory_sas = { - dev = module.branch-teams-dev-projectfactory-sa.iam_email - prod = module.branch-teams-prod-projectfactory-sa.iam_email - } + folder_ids = merge( + { + networking = module.branch-network-folder.id + networking-dev = module.branch-network-dev-folder.id + networking-prod = module.branch-network-prod-folder.id + sandbox = module.branch-sandbox-folder.id + security = module.branch-security-folder.id + teams = module.branch-teams-folder.id + }, + { + for k, v in module.branch-teams-team-folder : "team-${k}" => v.id + }, + { + for k, v in module.branch-teams-team-dev-folder : "team-${k}-dev" => v.id + }, + { + for k, v in module.branch-teams-team-prod-folder : "team-${k}-prod" => v.id + } + ) providers = { "02-networking" = templatefile("${path.module}/../../assets/templates/providers.tpl", { bucket = module.branch-network-gcs.name @@ -46,42 +61,44 @@ locals { sa = module.branch-sandbox-sa.email }) } + service_accounts = merge( + { + networking = module.branch-network-sa.email + project-factory-dev = module.branch-teams-dev-projectfactory-sa.email + project-factory-prod = module.branch-teams-prod-projectfactory-sa.email + sandbox = module.branch-sandbox-sa.email + security = module.branch-security-sa.email + teams = module.branch-teams-prod-sa.email + }, + { + for k, v in module.branch-teams-team-sa : "team-${k}" => v.email + }, + ) tfvars = { - "02-networking" = jsonencode({ - folder_ids = { - networking = module.branch-network-folder.id - networking-dev = module.branch-network-dev-folder.id - networking-prod = module.branch-network-prod-folder.id - } - project_factory_sa = local._project_factory_sas - }) - "02-security" = jsonencode({ - folder_id = module.branch-security-folder.id - kms_restricted_admins = { - for k, v in local._project_factory_sas : k => [v] - } - }) + folder_ids = local.folder_ids + service_accounts = local.service_accounts } } # optionally generate providers and tfvars files for subsequent stages resource "local_file" "providers" { - for_each = var.outputs_location == null ? {} : local.providers - filename = "${pathexpand(var.outputs_location)}/${each.key}/providers.tf" - content = each.value + for_each = var.outputs_location == null ? {} : local.providers + file_permission = "0644" + filename = "${pathexpand(var.outputs_location)}/providers/${each.key}-providers.tf" + content = each.value } resource "local_file" "tfvars" { - for_each = var.outputs_location == null ? {} : local.tfvars - filename = "${pathexpand(var.outputs_location)}/${each.key}/terraform-resman.auto.tfvars.json" - content = each.value + for_each = var.outputs_location == null ? {} : { 1 = 1 } + file_permission = "0644" + filename = "${pathexpand(var.outputs_location)}/tfvars/01-resman.auto.tfvars.json" + content = jsonencode(local.tfvars) } # outputs output "networking" { - # tfdoc:output:consumers 02-networking description = "Data for the networking stage." value = { folder = module.branch-network-folder.id @@ -91,7 +108,6 @@ output "networking" { } output "project_factories" { - # tfdoc:output:consumers xx-teams description = "Data for the project factories stage." value = { dev = { diff --git a/fast/stages/01-resman/variables.tf b/fast/stages/01-resman/variables.tf index 93398a2e..7c8a584d 100644 --- a/fast/stages/01-resman/variables.tf +++ b/fast/stages/01-resman/variables.tf @@ -17,6 +17,12 @@ # defaults for variables marked with global tfdoc annotations, can be set via # the tfvars file generated in stage 00 and stored in its outputs +variable "automation_project_id" { + # tfdoc:variable:source 00-bootstrap + description = "Project id for the automation project created by the bootstrap stage." + type = string +} + variable "billing_account" { # tfdoc:variable:source 00-bootstrap description = "Billing account id and organization id ('nnnnnnnn' or null)." @@ -26,12 +32,6 @@ variable "billing_account" { }) } -variable "automation_project_id" { - # tfdoc:variable:source 00-bootstrap - description = "Project id for the automation project created by the bootstrap stage." - type = string -} - variable "custom_roles" { # tfdoc:variable:source 00-bootstrap description = "Custom roles defined at the org level, in key => id format." diff --git a/fast/stages/02-networking-nva/README.md b/fast/stages/02-networking-nva/README.md index ef61af08..30ddb0f6 100644 --- a/fast/stages/02-networking-nva/README.md +++ b/fast/stages/02-networking-nva/README.md @@ -363,30 +363,30 @@ Don't forget to add a peering zone in the landing project and point it to the ne | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| -| [billing_account_id](variables.tf#L17) | Billing account id. | string | ✓ | | 00-bootstrap | -| [folder_ids](variables.tf#L59) | Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created. | map(string) | ✓ | | 01-resman | -| [organization](variables.tf#L91) | Organization details. | object({…}) | ✓ | | 00-bootstrap | -| [prefix](variables.tf#L107) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 00-bootstrap | -| [custom_adv](variables.tf#L23) | Custom advertisement definitions in name => range format. | map(string) | | {…} | | -| [data_dir](variables.tf#L45) | Relative path for the folder storing configuration data for network resources. | string | | "data" | | -| [dns](variables.tf#L51) | Onprem DNS resolvers | map(list(string)) | | {…} | | -| [l7ilb_subnets](variables.tf#L65) | Subnets used for L7 ILBs. | map(list(object({…}))) | | {…} | | -| [onprem_cidr](variables.tf#L83) | Onprem addresses in name => range format. | map(string) | | {…} | | -| [outputs_location](variables.tf#L101) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | string | | null | | -| [project_factory_sa](variables.tf#L118) | IAM emails for project factory service accounts | map(string) | | {} | 01-resman | -| [psa_ranges](variables.tf#L125) | IP ranges used for Private Service Access (e.g. CloudSQL). | map(map(string)) | | {…} | | -| [router_configs](variables.tf#L144) | Configurations for CRs and onprem routers. | map(object({…})) | | {…} | | -| [vpn_onprem_configs](variables.tf#L167) | VPN gateway configuration for onprem interconnection. | map(object({…})) | | {…} | | +| [billing_account](variables.tf#L17) | Billing account id and organization id ('nnnnnnnn' or null). | object({…}) | ✓ | | 00-bootstrap | +| [folder_ids](variables.tf#L71) | Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created. | object({…}) | ✓ | | 01-resman | +| [organization](variables.tf#L107) | Organization details. | object({…}) | ✓ | | 00-bootstrap | +| [prefix](variables.tf#L123) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 00-bootstrap | +| [custom_adv](variables.tf#L26) | Custom advertisement definitions in name => range format. | map(string) | | {…} | | +| [custom_roles](variables.tf#L48) | Custom roles defined at the org level, in key => id format. | object({…}) | | null | 00-bootstrap | +| [data_dir](variables.tf#L57) | Relative path for the folder storing configuration data for network resources. | string | | "data" | | +| [dns](variables.tf#L63) | Onprem DNS resolvers | map(list(string)) | | {…} | | +| [l7ilb_subnets](variables.tf#L81) | Subnets used for L7 ILBs. | map(list(object({…}))) | | {…} | | +| [onprem_cidr](variables.tf#L99) | Onprem addresses in name => range format. | map(string) | | {…} | | +| [outputs_location](variables.tf#L117) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | string | | null | | +| [psa_ranges](variables.tf#L134) | IP ranges used for Private Service Access (e.g. CloudSQL). | map(map(string)) | | {…} | | +| [router_configs](variables.tf#L153) | Configurations for CRs and onprem routers. | map(object({…})) | | {…} | | +| [service_accounts](variables.tf#L176) | Automation service accounts in name => email format. | object({…}) | | null | 01-resman | +| [vpn_onprem_configs](variables.tf#L186) | VPN gateway configuration for onprem interconnection. | map(object({…})) | | {…} | | ## Outputs | name | description | sensitive | consumers | |---|---|:---:|---| -| [project_ids](outputs.tf#L42) | Network project ids. | | | -| [project_numbers](outputs.tf#L51) | Network project numbers. | | | -| [shared_vpc_host_projects](outputs.tf#L60) | Shared VPC host projects. | | | -| [shared_vpc_self_links](outputs.tf#L69) | Shared VPC host projects. | | | -| [tfvars](outputs.tf#L93) | Network-related variables used in other stages. | ✓ | | -| [vpn_gateway_endpoints](outputs.tf#L79) | External IP Addresses for the GCP VPN gateways. | | | +| [host_project_ids](outputs.tf#L52) | Network project ids. | | | +| [host_project_numbers](outputs.tf#L57) | Network project numbers. | | | +| [shared_vpc_self_links](outputs.tf#L62) | Shared VPC host projects. | | | +| [tfvars](outputs.tf#L81) | Terraform variables file for the following stages. | ✓ | | +| [vpn_gateway_endpoints](outputs.tf#L67) | External IP Addresses for the GCP VPN gateways. | | | diff --git a/fast/stages/02-networking-nva/landing.tf b/fast/stages/02-networking-nva/landing.tf index ed983da0..af3e56bc 100644 --- a/fast/stages/02-networking-nva/landing.tf +++ b/fast/stages/02-networking-nva/landing.tf @@ -18,7 +18,7 @@ module "landing-project" { source = "../../../modules/project" - billing_account = var.billing_account_id + billing_account = var.billing_account.id name = "prod-net-landing-0" parent = var.folder_ids.networking-prod prefix = var.prefix @@ -37,6 +37,13 @@ module "landing-project" { enabled = true service_projects = [] } + metric_scopes = [module.landing-project.project_id] + iam = { + "roles/dns.admin" = [local.service_accounts.project-factory-prod] + (local.custom_roles.service_project_network_admin) = [ + local.service_accounts.project-factory-prod + ] + } } # Untrusted VPC diff --git a/fast/stages/02-networking-nva/main.tf b/fast/stages/02-networking-nva/main.tf index 932191dc..0c8f230c 100644 --- a/fast/stages/02-networking-nva/main.tf +++ b/fast/stages/02-networking-nva/main.tf @@ -17,12 +17,16 @@ # tfdoc:file:description Networking folder and hierarchical policy. locals { + custom_roles = coalesce(var.custom_roles, {}) l7ilb_subnets = { for env, v in var.l7ilb_subnets : env => [ for s in v : merge(s, { active = true name = "${env}-l7ilb-${s.region}" })] } + service_accounts = { + for k, v in coalesce(var.service_accounts, {}) : k => "serviceAccount:${v}" + } } module "folder" { diff --git a/fast/stages/02-networking-nva/outputs.tf b/fast/stages/02-networking-nva/outputs.tf index cf39b1df..0b93bf4b 100644 --- a/fast/stages/02-networking-nva/outputs.tf +++ b/fast/stages/02-networking-nva/outputs.tf @@ -14,66 +14,54 @@ * limitations under the License. */ -# Optionally, generate providers and tfvars files for subsequent stages - locals { + host_project_ids = { + dev-spoke-0 = module.dev-spoke-project.project_id + prod-landing = module.landing-project.project_id + prod-spoke-0 = module.prod-spoke-project.project_id + } + host_project_numbers = { + dev-spoke-0 = module.dev-spoke-project.number + prod-landing = module.landing-project.number + prod-spoke-0 = module.prod-spoke-project.number + } tfvars = { - "03-project-factory-dev" = jsonencode({ - environment_dns_zone = module.dev-dns-private-zone.domain - shared_vpc_self_link = module.dev-spoke-vpc.self_link - vpc_host_project = module.dev-spoke-project.project_id - }) - "03-project-factory-prod" = jsonencode({ - environment_dns_zone = module.prod-dns-private-zone.domain - shared_vpc_self_link = module.prod-spoke-vpc.self_link - vpc_host_project = module.prod-spoke-project.project_id - }) + host_project_ids = local.host_project_ids + host_project_numbers = local.host_project_numbers + vpc_self_links = local.vpc_self_links + } + vpc_self_links = { + prod-landing-trusted = module.landing-trusted-vpc.self_link + prod-landing-untrusted = module.landing-untrusted-vpc.self_link + dev-spoke-0 = module.dev-spoke-vpc.self_link + prod-spoke-0 = module.prod-spoke-vpc.self_link } } +# optionally generate tfvars file for subsequent stages + resource "local_file" "tfvars" { - for_each = var.outputs_location == null ? {} : local.tfvars - filename = "${pathexpand(var.outputs_location)}/${each.key}/terraform-networking.auto.tfvars.json" - content = each.value + for_each = var.outputs_location == null ? {} : { 1 = 1 } + file_permission = "0644" + filename = "${pathexpand(var.outputs_location)}/tfvars/02-networking.auto.tfvars.json" + content = jsonencode(local.tfvars) } -# Outputs +# outputs -output "project_ids" { +output "host_project_ids" { description = "Network project ids." - value = { - dev = module.dev-spoke-project.project_id - landing = module.landing-project.project_id - prod = module.prod-spoke-project.project_id - } + value = local.host_project_ids } -output "project_numbers" { +output "host_project_numbers" { description = "Network project numbers." - value = { - dev = "projects/${module.dev-spoke-project.number}" - landing = "projects/${module.landing-project.number}" - prod = "projects/${module.prod-spoke-project.number}" - } -} - -output "shared_vpc_host_projects" { - description = "Shared VPC host projects." - value = { - dev = module.dev-spoke-project.project_id - landing = module.landing-project.project_id - prod = module.prod-spoke-project.project_id - } + value = local.host_project_numbers } output "shared_vpc_self_links" { description = "Shared VPC host projects." - value = { - dev = module.dev-spoke-vpc.self_link - landing-trusted = module.landing-trusted-vpc.self_link - landing-untrusted = module.landing-untrusted-vpc.self_link - prod = module.prod-spoke-vpc.self_link - } + value = local.vpc_self_links } output "vpn_gateway_endpoints" { @@ -91,7 +79,7 @@ output "vpn_gateway_endpoints" { } output "tfvars" { - description = "Network-related variables used in other stages." + description = "Terraform variables file for the following stages." sensitive = true value = local.tfvars } diff --git a/fast/stages/02-networking-nva/spoke-dev.tf b/fast/stages/02-networking-nva/spoke-dev.tf index ced773f4..c3387b22 100644 --- a/fast/stages/02-networking-nva/spoke-dev.tf +++ b/fast/stages/02-networking-nva/spoke-dev.tf @@ -18,7 +18,7 @@ module "dev-spoke-project" { source = "../../../modules/project" - billing_account = var.billing_account_id + billing_account = var.billing_account.id name = "dev-net-spoke-0" parent = var.folder_ids.networking-dev prefix = var.prefix @@ -39,7 +39,10 @@ module "dev-spoke-project" { } metric_scopes = [module.landing-project.project_id] iam = { - "roles/dns.admin" = [var.project_factory_sa.dev] + "roles/dns.admin" = [local.service_accounts.project-factory-dev] + (local.custom_roles.service_project_network_admin) = [ + local.service_accounts.project-factory-dev + ] } } diff --git a/fast/stages/02-networking-nva/spoke-prod.tf b/fast/stages/02-networking-nva/spoke-prod.tf index 8ce177af..2fb5e761 100644 --- a/fast/stages/02-networking-nva/spoke-prod.tf +++ b/fast/stages/02-networking-nva/spoke-prod.tf @@ -18,7 +18,7 @@ module "prod-spoke-project" { source = "../../../modules/project" - billing_account = var.billing_account_id + billing_account = var.billing_account.id name = "prod-net-spoke-0" parent = var.folder_ids.networking-prod prefix = var.prefix @@ -39,7 +39,10 @@ module "prod-spoke-project" { } metric_scopes = [module.landing-project.project_id] iam = { - "roles/dns.admin" = [var.project_factory_sa.prod] + "roles/dns.admin" = [local.service_accounts.project-factory-prod] + (local.custom_roles.service_project_network_admin) = [ + local.service_accounts.project-factory-prod + ] } } diff --git a/fast/stages/02-networking-nva/variables.tf b/fast/stages/02-networking-nva/variables.tf index 8fca0ba7..4fdc992b 100644 --- a/fast/stages/02-networking-nva/variables.tf +++ b/fast/stages/02-networking-nva/variables.tf @@ -14,10 +14,13 @@ * limitations under the License. */ -variable "billing_account_id" { +variable "billing_account" { # tfdoc:variable:source 00-bootstrap - description = "Billing account id." - type = string + description = "Billing account id and organization id ('nnnnnnnn' or null)." + type = object({ + id = string + organization_id = number + }) } variable "custom_adv" { @@ -42,6 +45,15 @@ variable "custom_adv" { } } +variable "custom_roles" { + # tfdoc:variable:source 00-bootstrap + description = "Custom roles defined at the org level, in key => id format." + type = object({ + service_project_network_admin = string + }) + default = null +} + variable "data_dir" { description = "Relative path for the folder storing configuration data for network resources." type = string @@ -59,7 +71,11 @@ variable "dns" { variable "folder_ids" { # tfdoc:variable:source 01-resman description = "Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created." - type = map(string) + type = object({ + networking = string + networking-dev = string + networking-prod = string + }) } variable "l7ilb_subnets" { @@ -115,13 +131,6 @@ variable "prefix" { } } -variable "project_factory_sa" { - # tfdoc:variable:source 01-resman - description = "IAM emails for project factory service accounts" - type = map(string) - default = {} -} - variable "psa_ranges" { description = "IP ranges used for Private Service Access (e.g. CloudSQL)." type = map(map(string)) @@ -164,6 +173,16 @@ variable "router_configs" { } } +variable "service_accounts" { + # tfdoc:variable:source 01-resman + description = "Automation service accounts in name => email format." + type = object({ + project-factory-dev = string + project-factory-prod = string + }) + default = null +} + variable "vpn_onprem_configs" { description = "VPN gateway configuration for onprem interconnection." type = map(object({ diff --git a/fast/stages/02-networking-vpn/README.md b/fast/stages/02-networking-vpn/README.md index f8cc210c..48bb9b8c 100644 --- a/fast/stages/02-networking-vpn/README.md +++ b/fast/stages/02-networking-vpn/README.md @@ -308,32 +308,31 @@ DNS configurations are centralised in the `dns.tf` file. Spokes delegate DNS res | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| -| [billing_account_id](variables.tf#L17) | Billing account id. | string | ✓ | | 00-bootstrap | -| [folder_ids](variables.tf#L61) | Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created. | map(string) | ✓ | | 01-resman | -| [organization](variables.tf#L85) | Organization details. | object({…}) | ✓ | | 00-bootstrap | -| [prefix](variables.tf#L101) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 00-bootstrap | -| [custom_adv](variables.tf#L23) | Custom advertisement definitions in name => range format. | map(string) | | {…} | | -| [custom_roles](variables.tf#L40) | Custom roles defined at the org level, in key => id format. | map(string) | | {} | 00-bootstrap | -| [data_dir](variables.tf#L47) | Relative path for the folder storing configuration data for network resources. | string | | "data" | | -| [dns](variables.tf#L53) | Onprem DNS resolvers. | map(list(string)) | | {…} | | -| [l7ilb_subnets](variables.tf#L67) | Subnets used for L7 ILBs. | map(list(object({…}))) | | {…} | | -| [outputs_location](variables.tf#L95) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | string | | null | | -| [project_factory_sa](variables.tf#L112) | IAM emails for project factory service accounts. | map(string) | | {} | 01-resman | -| [psa_ranges](variables.tf#L119) | IP ranges used for Private Service Access (e.g. CloudSQL). | map(map(string)) | | {…} | | -| [router_configs](variables.tf#L134) | Configurations for CRs and onprem routers. | map(object({…})) | | {…} | | -| [vpn_onprem_configs](variables.tf#L158) | VPN gateway configuration for onprem interconnection. | map(object({…})) | | {…} | | -| [vpn_spoke_configs](variables.tf#L214) | VPN gateway configuration for spokes. | map(object({…})) | | {…} | | +| [billing_account](variables.tf#L17) | Billing account id and organization id ('nnnnnnnn' or null). | object({…}) | ✓ | | 00-bootstrap | +| [folder_ids](variables.tf#L66) | Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created. | object({…}) | ✓ | | 01-resman | +| [organization](variables.tf#L94) | Organization details. | object({…}) | ✓ | | 00-bootstrap | +| [prefix](variables.tf#L110) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 00-bootstrap | +| [custom_adv](variables.tf#L26) | Custom advertisement definitions in name => range format. | map(string) | | {…} | | +| [custom_roles](variables.tf#L43) | Custom roles defined at the org level, in key => id format. | object({…}) | | null | 00-bootstrap | +| [data_dir](variables.tf#L52) | Relative path for the folder storing configuration data for network resources. | string | | "data" | | +| [dns](variables.tf#L58) | Onprem DNS resolvers. | map(list(string)) | | {…} | | +| [l7ilb_subnets](variables.tf#L76) | Subnets used for L7 ILBs. | map(list(object({…}))) | | {…} | | +| [outputs_location](variables.tf#L104) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | string | | null | | +| [psa_ranges](variables.tf#L121) | IP ranges used for Private Service Access (e.g. CloudSQL). | map(map(string)) | | {…} | | +| [router_configs](variables.tf#L136) | Configurations for CRs and onprem routers. | map(object({…})) | | {…} | | +| [service_accounts](variables.tf#L160) | Automation service accounts in name => email format. | object({…}) | | null | 01-resman | +| [vpn_onprem_configs](variables.tf#L170) | VPN gateway configuration for onprem interconnection. | map(object({…})) | | {…} | | +| [vpn_spoke_configs](variables.tf#L226) | VPN gateway configuration for spokes. | map(object({…})) | | {…} | | ## Outputs | name | description | sensitive | consumers | |---|---|:---:|---| -| [cloud_dns_inbound_policy](outputs.tf#L41) | IP Addresses for Cloud DNS inbound policy. | | | -| [project_ids](outputs.tf#L46) | Network project ids. | | | -| [project_numbers](outputs.tf#L55) | Network project numbers. | | | -| [shared_vpc_host_projects](outputs.tf#L64) | Shared VPC host projects. | | | -| [shared_vpc_self_links](outputs.tf#L74) | Shared VPC host projects. | | | -| [tfvars](outputs.tf#L91) | Network-related variables used in other stages. | ✓ | | -| [vpn_gateway_endpoints](outputs.tf#L84) | External IP Addresses for the GCP VPN gateways. | | | +| [cloud_dns_inbound_policy](outputs.tf#L51) | IP Addresses for Cloud DNS inbound policy. | | | +| [host_project_ids](outputs.tf#L56) | Network project ids. | | | +| [host_project_numbers](outputs.tf#L61) | Network project numbers. | | | +| [shared_vpc_self_links](outputs.tf#L66) | Shared VPC host projects. | | | +| [tfvars](outputs.tf#L81) | Terraform variables file for the following stages. | ✓ | | +| [vpn_gateway_endpoints](outputs.tf#L71) | External IP Addresses for the GCP VPN gateways. | | | diff --git a/fast/stages/02-networking-vpn/landing.tf b/fast/stages/02-networking-vpn/landing.tf index e2b6c45a..042c95cb 100644 --- a/fast/stages/02-networking-vpn/landing.tf +++ b/fast/stages/02-networking-vpn/landing.tf @@ -18,7 +18,7 @@ module "landing-project" { source = "../../../modules/project" - billing_account = var.billing_account_id + billing_account = var.billing_account.id name = "prod-net-landing-0" parent = var.folder_ids.networking-prod prefix = var.prefix @@ -37,6 +37,13 @@ module "landing-project" { enabled = true service_projects = [] } + metric_scopes = [module.landing-project.project_id] + iam = { + "roles/dns.admin" = [local.service_accounts.project-factory-prod] + (local.custom_roles.service_project_network_admin) = [ + local.service_accounts.project-factory-prod + ] + } } module "landing-vpc" { diff --git a/fast/stages/02-networking-vpn/main.tf b/fast/stages/02-networking-vpn/main.tf index fcca8867..964b7dfc 100644 --- a/fast/stages/02-networking-vpn/main.tf +++ b/fast/stages/02-networking-vpn/main.tf @@ -30,6 +30,7 @@ locals { route_priority = null } } + custom_roles = coalesce(var.custom_roles, {}) l7ilb_subnets = { for env, v in var.l7ilb_subnets : env => [ for s in v : merge(s, { @@ -47,6 +48,9 @@ locals { "roles/container.hostServiceAgentUser", "roles/vpcaccess.user", ] + service_accounts = { + for k, v in coalesce(var.service_accounts, {}) : k => "serviceAccount:${v}" + } } module "folder" { diff --git a/fast/stages/02-networking-vpn/outputs.tf b/fast/stages/02-networking-vpn/outputs.tf index 15b4c49c..7b401dbb 100644 --- a/fast/stages/02-networking-vpn/outputs.tf +++ b/fast/stages/02-networking-vpn/outputs.tf @@ -13,27 +13,37 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -# optionally generate providers and tfvars files for subsequent stages locals { + host_project_ids = { + dev-spoke-0 = module.dev-spoke-project.project_id + prod-landing = module.landing-project.project_id + prod-spoke-0 = module.prod-spoke-project.project_id + } + host_project_numbers = { + dev-spoke-0 = module.dev-spoke-project.number + prod-landing = module.landing-project.number + prod-spoke-0 = module.prod-spoke-project.number + } tfvars = { - "03-project-factory-dev" = jsonencode({ - environment_dns_zone = module.dev-dns-private-zone.domain - shared_vpc_self_link = module.dev-spoke-vpc.self_link - vpc_host_project = module.dev-spoke-project.project_id - }) - "03-project-factory-prod" = jsonencode({ - environment_dns_zone = module.prod-dns-private-zone.domain - shared_vpc_self_link = module.prod-spoke-vpc.self_link - vpc_host_project = module.prod-spoke-project.project_id - }) + host_project_ids = local.host_project_ids + host_project_numbers = local.host_project_numbers + vpc_self_links = local.vpc_self_links + } + vpc_self_links = { + prod-landing = module.landing-vpc.self_link + dev-spoke-0 = module.dev-spoke-vpc.self_link + prod-spoke-0 = module.prod-spoke-vpc.self_link } } +# optionally generate tfvars file for subsequent stages + resource "local_file" "tfvars" { - for_each = var.outputs_location == null ? {} : local.tfvars - filename = "${pathexpand(var.outputs_location)}/${each.key}/terraform-networking.auto.tfvars.json" - content = each.value + for_each = var.outputs_location == null ? {} : { 1 = 1 } + file_permission = "0644" + filename = "${pathexpand(var.outputs_location)}/tfvars/02-networking.auto.tfvars.json" + content = jsonencode(loca.tfvars) } # outputs @@ -43,53 +53,33 @@ output "cloud_dns_inbound_policy" { value = [for s in module.landing-vpc.subnets : cidrhost(s.ip_cidr_range, 2)] } -output "project_ids" { +output "host_project_ids" { description = "Network project ids." - value = { - dev = module.dev-spoke-project.project_id - landing = module.landing-project.project_id - prod = module.prod-spoke-project.project_id - } + value = local.host_project_ids } -output "project_numbers" { +output "host_project_numbers" { description = "Network project numbers." - value = { - dev = "projects/${module.dev-spoke-project.number}" - landing = "projects/${module.landing-project.number}" - prod = "projects/${module.prod-spoke-project.number}" - } + value = local.host_project_numbers } -output "shared_vpc_host_projects" { - description = "Shared VPC host projects." - value = { - landing = module.landing-project.project_id - dev = module.dev-spoke-project.project_id - prod = module.prod-spoke-project.project_id - } -} - - output "shared_vpc_self_links" { description = "Shared VPC host projects." - value = { - landing = module.landing-vpc.self_link - dev = module.dev-spoke-vpc.self_link - prod = module.prod-spoke-vpc.self_link - } + value = local.vpc_self_links } - output "vpn_gateway_endpoints" { description = "External IP Addresses for the GCP VPN gateways." value = { - onprem-ew1 = { for v in module.landing-to-onprem-ew1-vpn.gateway.vpn_interfaces : v.id => v.ip_address } + onprem-ew1 = { + for v in module.landing-to-onprem-ew1-vpn.gateway.vpn_interfaces : + v.id => v.ip_address + } } } output "tfvars" { - description = "Network-related variables used in other stages." + description = "Terraform variables file for the following stages." sensitive = true value = local.tfvars } diff --git a/fast/stages/02-networking-vpn/spoke-dev.tf b/fast/stages/02-networking-vpn/spoke-dev.tf index 255142bf..2ae4dc04 100644 --- a/fast/stages/02-networking-vpn/spoke-dev.tf +++ b/fast/stages/02-networking-vpn/spoke-dev.tf @@ -18,7 +18,7 @@ module "dev-spoke-project" { source = "../../../modules/project" - billing_account = var.billing_account_id + billing_account = var.billing_account.id name = "dev-net-spoke-0" parent = var.folder_ids.networking-dev prefix = var.prefix @@ -39,9 +39,9 @@ module "dev-spoke-project" { } metric_scopes = [module.landing-project.project_id] iam = { - "roles/dns.admin" = [var.project_factory_sa.dev] - (var.custom_roles.service_project_network_admin) = [ - var.project_factory_sa.prod + "roles/dns.admin" = [local.service_accounts.project-factory-dev] + (local.custom_roles.service_project_network_admin) = [ + local.service_accounts.project-factory-dev ] } } @@ -102,7 +102,7 @@ resource "google_project_iam_binding" "dev_spoke_project_iam_delegated" { project = module.dev-spoke-project.project_id role = "roles/resourcemanager.projectIamAdmin" members = [ - var.project_factory_sa.dev + local.service_accounts.project-factory-dev ] condition { title = "dev_stage3_sa_delegated_grants" diff --git a/fast/stages/02-networking-vpn/spoke-prod.tf b/fast/stages/02-networking-vpn/spoke-prod.tf index 0ce38dc7..c24a4608 100644 --- a/fast/stages/02-networking-vpn/spoke-prod.tf +++ b/fast/stages/02-networking-vpn/spoke-prod.tf @@ -18,7 +18,7 @@ module "prod-spoke-project" { source = "../../../modules/project" - billing_account = var.billing_account_id + billing_account = var.billing_account.id name = "prod-net-spoke-0" parent = var.folder_ids.networking-prod prefix = var.prefix @@ -39,9 +39,9 @@ module "prod-spoke-project" { } metric_scopes = [module.landing-project.project_id] iam = { - "roles/dns.admin" = [var.project_factory_sa.prod] - (var.custom_roles.service_project_network_admin) = [ - var.project_factory_sa.prod + "roles/dns.admin" = [local.service_accounts.project-factory-prod] + (local.custom_roles.service_project_network_admin) = [ + local.service_accounts.project-factory-prod ] } } @@ -102,7 +102,7 @@ resource "google_project_iam_binding" "prod_spoke_project_iam_delegated" { project = module.prod-spoke-project.project_id role = "roles/resourcemanager.projectIamAdmin" members = [ - var.project_factory_sa.prod + local.service_accounts.project-factory-prod ] condition { title = "prod_stage3_sa_delegated_grants" diff --git a/fast/stages/02-networking-vpn/variables.tf b/fast/stages/02-networking-vpn/variables.tf index 92c264fc..e68a4374 100644 --- a/fast/stages/02-networking-vpn/variables.tf +++ b/fast/stages/02-networking-vpn/variables.tf @@ -14,10 +14,13 @@ * limitations under the License. */ -variable "billing_account_id" { +variable "billing_account" { # tfdoc:variable:source 00-bootstrap - description = "Billing account id." - type = string + description = "Billing account id and organization id ('nnnnnnnn' or null)." + type = object({ + id = string + organization_id = number + }) } variable "custom_adv" { @@ -40,8 +43,10 @@ variable "custom_adv" { variable "custom_roles" { # tfdoc:variable:source 00-bootstrap description = "Custom roles defined at the org level, in key => id format." - type = map(string) - default = {} + type = object({ + service_project_network_admin = string + }) + default = null } variable "data_dir" { @@ -61,7 +66,11 @@ variable "dns" { variable "folder_ids" { # tfdoc:variable:source 01-resman description = "Folders to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created." - type = map(string) + type = object({ + networking = string + networking-dev = string + networking-prod = string + }) } variable "l7ilb_subnets" { @@ -109,13 +118,6 @@ variable "prefix" { } } -variable "project_factory_sa" { - # tfdoc:variable:source 01-resman - description = "IAM emails for project factory service accounts." - type = map(string) - default = {} -} - variable "psa_ranges" { description = "IP ranges used for Private Service Access (e.g. CloudSQL)." type = map(map(string)) @@ -155,6 +157,16 @@ variable "router_configs" { } } +variable "service_accounts" { + # tfdoc:variable:source 01-resman + description = "Automation service accounts in name => email format." + type = object({ + project-factory-dev = string + project-factory-prod = string + }) + default = null +} + variable "vpn_onprem_configs" { description = "VPN gateway configuration for onprem interconnection." type = map(object({ diff --git a/fast/stages/02-security/README.md b/fast/stages/02-security/README.md index 613b1c49..c36e159b 100644 --- a/fast/stages/02-security/README.md +++ b/fast/stages/02-security/README.md @@ -285,27 +285,29 @@ Some references that might be useful in setting up this stage: | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| -| [billing_account_id](variables.tf#L17) | Billing account id. | string | ✓ | | bootstrap | -| [folder_id](variables.tf#L23) | Folder to be used for the networking resources in folders/nnnn format. | string | ✓ | | resman | -| [organization](variables.tf#L73) | Organization details. | object({…}) | ✓ | | bootstrap | -| [prefix](variables.tf#L89) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 00-bootstrap | -| [groups](variables.tf#L29) | Group names to grant organization-level permissions. | map(string) | | {…} | bootstrap | -| [kms_defaults](variables.tf#L44) | Defaults used for KMS keys. | object({…}) | | {…} | | -| [kms_keys](variables.tf#L56) | KMS keys to create, keyed by name. Null attributes will be interpolated with defaults. | map(object({…})) | | {} | | -| [kms_restricted_admins](variables.tf#L67) | Map of environment => [identities] who can assign the encrypt/decrypt roles on keys. | map(list(string)) | | {} | | -| [outputs_location](variables.tf#L83) | Path where providers, tfvars files, and lists for the following stages are written. Leave empty to disable. | string | | null | | -| [vpc_sc_access_levels](variables.tf#L100) | VPC SC access level definitions. | map(object({…})) | | {} | | -| [vpc_sc_egress_policies](variables.tf#L115) | VPC SC egress policy defnitions. | map(object({…})) | | {} | | -| [vpc_sc_ingress_policies](variables.tf#L133) | VPC SC ingress policy defnitions. | map(object({…})) | | {} | | -| [vpc_sc_perimeter_access_levels](variables.tf#L153) | VPC SC perimeter access_levels. | object({…}) | | null | | -| [vpc_sc_perimeter_egress_policies](variables.tf#L163) | VPC SC egress policies per perimeter, values reference keys defined in the `vpc_sc_ingress_policies` variable. | object({…}) | | null | | -| [vpc_sc_perimeter_ingress_policies](variables.tf#L173) | VPC SC ingress policies per perimeter, values reference keys defined in the `vpc_sc_ingress_policies` variable. | object({…}) | | null | | -| [vpc_sc_perimeter_projects](variables.tf#L183) | VPC SC perimeter resources. | object({…}) | | null | | +| [billing_account](variables.tf#L17) | Billing account id and organization id ('nnnnnnnn' or null). | object({…}) | ✓ | | 00-bootstrap | +| [folder_ids](variables.tf#L26) | Folder name => id mappings, the 'security' folder name must exist. | object({…}) | ✓ | | 01-resman | +| [organization](variables.tf#L81) | Organization details. | object({…}) | ✓ | | 00-bootstrap | +| [prefix](variables.tf#L97) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 00-bootstrap | +| [service_accounts](variables.tf#L72) | Automation service accounts that can assign the encrypt/decrypt roles on keys. | object({…}) | ✓ | | 01-resman | +| [groups](variables.tf#L34) | Group names to grant organization-level permissions. | map(string) | | {…} | 00-bootstrap | +| [kms_defaults](variables.tf#L49) | Defaults used for KMS keys. | object({…}) | | {…} | | +| [kms_keys](variables.tf#L61) | KMS keys to create, keyed by name. Null attributes will be interpolated with defaults. | map(object({…})) | | {} | | +| [outputs_location](variables.tf#L91) | Path where providers, tfvars files, and lists for the following stages are written. Leave empty to disable. | string | | null | | +| [vpc_sc_access_levels](variables.tf#L108) | VPC SC access level definitions. | map(object({…})) | | {} | | +| [vpc_sc_egress_policies](variables.tf#L123) | VPC SC egress policy defnitions. | map(object({…})) | | {} | | +| [vpc_sc_ingress_policies](variables.tf#L141) | VPC SC ingress policy defnitions. | map(object({…})) | | {} | | +| [vpc_sc_perimeter_access_levels](variables.tf#L161) | VPC SC perimeter access_levels. | object({…}) | | null | | +| [vpc_sc_perimeter_egress_policies](variables.tf#L171) | VPC SC egress policies per perimeter, values reference keys defined in the `vpc_sc_ingress_policies` variable. | object({…}) | | null | | +| [vpc_sc_perimeter_ingress_policies](variables.tf#L181) | VPC SC ingress policies per perimeter, values reference keys defined in the `vpc_sc_ingress_policies` variable. | object({…}) | | null | | +| [vpc_sc_perimeter_projects](variables.tf#L191) | VPC SC perimeter resources. | object({…}) | | null | | ## Outputs | name | description | sensitive | consumers | |---|---|:---:|---| -| [stage_perimeter_projects](outputs.tf#L37) | Security project numbers. They can be added to perimeter resources. | | | +| [kms_keys](outputs.tf#L53) | KMS key ids. | | | +| [stage_perimeter_projects](outputs.tf#L58) | Security project numbers. They can be added to perimeter resources. | | | +| [tfvars](outputs.tf#L68) | Terraform variable files for the following stages. | ✓ | | diff --git a/fast/stages/02-security/core-dev.tf b/fast/stages/02-security/core-dev.tf index db12bdee..92fcaec0 100644 --- a/fast/stages/02-security/core-dev.tf +++ b/fast/stages/02-security/core-dev.tf @@ -14,14 +14,20 @@ * limitations under the License. */ +locals { + dev_kms_restricted_admins = [ + "serviceAccount:${var.service_accounts.project-factory-dev}" + ] +} + module "dev-sec-project" { source = "../../../modules/project" name = "dev-sec-core-0" - parent = var.folder_id + parent = var.folder_ids.security prefix = var.prefix - billing_account = var.billing_account_id + billing_account = var.billing_account.id iam = { - "roles/cloudkms.viewer" = try(var.kms_restricted_admins.dev, []) + "roles/cloudkms.viewer" = local.dev_kms_restricted_admins } labels = { environment = "dev", team = "security" } services = local.project_services @@ -46,7 +52,7 @@ module "dev-sec-kms" { # TODO(ludo): grant delegated role at key instead of project level resource "google_project_iam_member" "dev_key_admin_delegated" { - for_each = toset(try(var.kms_restricted_admins.dev, [])) + for_each = toset(local.dev_kms_restricted_admins) project = module.dev-sec-project.project_id role = "roles/cloudkms.admin" member = each.key diff --git a/fast/stages/02-security/core-prod.tf b/fast/stages/02-security/core-prod.tf index 4154d050..d00c724d 100644 --- a/fast/stages/02-security/core-prod.tf +++ b/fast/stages/02-security/core-prod.tf @@ -14,14 +14,20 @@ * limitations under the License. */ +locals { + prod_kms_restricted_admins = [ + "serviceAccount:${var.service_accounts.project-factory-prod}" + ] +} + module "prod-sec-project" { source = "../../../modules/project" name = "prod-sec-core-0" - parent = var.folder_id + parent = var.folder_ids.security prefix = var.prefix - billing_account = var.billing_account_id + billing_account = var.billing_account.id iam = { - "roles/cloudkms.viewer" = try(var.kms_restricted_admins.prod, []) + "roles/cloudkms.viewer" = local.prod_kms_restricted_admins } labels = { environment = "prod", team = "security" } services = local.project_services @@ -45,7 +51,7 @@ module "prod-sec-kms" { # TODO(ludo): add support for conditions to Fabric modules resource "google_project_iam_member" "prod_key_admin_delegated" { - for_each = toset(try(var.kms_restricted_admins.prod, [])) + for_each = toset(local.prod_kms_restricted_admins) project = module.prod-sec-project.project_id role = "roles/cloudkms.admin" member = each.key diff --git a/fast/stages/02-security/outputs.tf b/fast/stages/02-security/outputs.tf index 15c75a2a..ee2ac15e 100644 --- a/fast/stages/02-security/outputs.tf +++ b/fast/stages/02-security/outputs.tf @@ -14,26 +14,47 @@ * limitations under the License. */ -# optionally generate files for subsequent stages - -resource "local_file" "dev_sec_kms" { - for_each = var.outputs_location == null ? {} : { 1 = 1 } - filename = "${pathexpand(var.outputs_location)}/yamls/02-security-kms-dev-keys.yaml" - content = yamlencode({ - for k, m in module.dev-sec-kms : k => m.key_ids - }) +locals { + _output_kms_keys = concat( + flatten([ + for location, mod in module.dev-sec-kms : [ + for name, id in mod.key_ids : { + key = "dev-${name}:${location}" + id = id + } + ] + ]), + flatten([ + for location, mod in module.prod-sec-kms : [ + for name, id in mod.key_ids : { + key = "prod-${name}:${location}" + id = id + } + ] + ]) + ) + output_kms_keys = { for k in local._output_kms_keys : k.key => k.id } + tfvars = { + kms_keys = local.output_kms_keys + } } -resource "local_file" "prod_sec_kms" { - for_each = var.outputs_location == null ? {} : { 1 = 1 } - filename = "${pathexpand(var.outputs_location)}/yamls/02-security-kms-prod-keys.yaml" - content = yamlencode({ - for k, m in module.prod-sec-kms : k => m.key_ids - }) +# optionally generate files for subsequent stages + +resource "local_file" "tfvars" { + for_each = var.outputs_location == null ? {} : { 1 = 1 } + file_permission = "0644" + filename = "${pathexpand(var.outputs_location)}/tfvars/02-security.auto.tfvars.json" + content = jsonencode(local.tfvars) } # outputs +output "kms_keys" { + description = "KMS key ids." + value = local.output_kms_keys +} + output "stage_perimeter_projects" { description = "Security project numbers. They can be added to perimeter resources." value = { @@ -41,3 +62,11 @@ output "stage_perimeter_projects" { prod = ["projects/${module.prod-sec-project.number}"] } } + +# ready to use variable values for subsequent stages + +output "tfvars" { + description = "Terraform variable files for the following stages." + sensitive = true + value = local.tfvars +} diff --git a/fast/stages/02-security/variables.tf b/fast/stages/02-security/variables.tf index 00cbe4dc..8ff52ffd 100644 --- a/fast/stages/02-security/variables.tf +++ b/fast/stages/02-security/variables.tf @@ -14,20 +14,25 @@ * limitations under the License. */ -variable "billing_account_id" { - # tfdoc:variable:source bootstrap - description = "Billing account id." - type = string +variable "billing_account" { + # tfdoc:variable:source 00-bootstrap + description = "Billing account id and organization id ('nnnnnnnn' or null)." + type = object({ + id = string + organization_id = number + }) } -variable "folder_id" { - # tfdoc:variable:source resman - description = "Folder to be used for the networking resources in folders/nnnn format." - type = string +variable "folder_ids" { + # tfdoc:variable:source 01-resman + description = "Folder name => id mappings, the 'security' folder name must exist." + type = object({ + security = string + }) } variable "groups" { - # tfdoc:variable:source bootstrap + # tfdoc:variable:source 00-bootstrap description = "Group names to grant organization-level permissions." type = map(string) # https://cloud.google.com/docs/enterprise/setup-checklist @@ -64,14 +69,17 @@ variable "kms_keys" { default = {} } -variable "kms_restricted_admins" { - description = "Map of environment => [identities] who can assign the encrypt/decrypt roles on keys." - type = map(list(string)) - default = {} +variable "service_accounts" { + # tfdoc:variable:source 01-resman + description = "Automation service accounts that can assign the encrypt/decrypt roles on keys." + type = object({ + project-factory-dev = string + project-factory-prod = string + }) } variable "organization" { - # tfdoc:variable:source bootstrap + # tfdoc:variable:source 00-bootstrap description = "Organization details." type = object({ domain = string diff --git a/fast/stages/03-project-factory/dev/README.md b/fast/stages/03-project-factory/dev/README.md index 2971cd05..b90e845c 100644 --- a/fast/stages/03-project-factory/dev/README.md +++ b/fast/stages/03-project-factory/dev/README.md @@ -107,13 +107,13 @@ terraform apply | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| -| [billing_account_id](variables.tf#L19) | Billing account id. | string | ✓ | | 00-bootstrap | -| [prefix](variables.tf#L44) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 00-bootstrap | -| [data_dir](variables.tf#L25) | Relative path for the folder storing configuration data. | string | | "data/projects" | | -| [defaults_file](variables.tf#L38) | Relative path for the file storing the project factory configuration. | string | | "data/defaults.yaml" | | -| [environment_dns_zone](variables.tf#L31) | DNS zone suffix for environment. | string | | null | 02-networking | -| [shared_vpc_self_link](variables.tf#L55) | Self link for the shared VPC. | string | | null | 02-networking | -| [vpc_host_project](variables.tf#L62) | Host project for the shared VPC. | string | | null | 02-networking | +| [billing_account](variables.tf#L19) | Billing account id and organization id ('nnnnnnnn' or null). | object({…}) | ✓ | | 00-bootstrap | +| [prefix](variables.tf#L47) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 00-bootstrap | +| [data_dir](variables.tf#L28) | Relative path for the folder storing configuration data. | string | | "data/projects" | | +| [defaults_file](variables.tf#L41) | Relative path for the file storing the project factory configuration. | string | | "data/defaults.yaml" | | +| [environment_dns_zone](variables.tf#L34) | DNS zone suffix for environment. | string | | null | 02-networking | +| [shared_vpc_self_links](variables.tf#L58) | Self link for the shared VPC. | object({…}) | | null | 02-networking | +| [vpc_host_project_ids](variables.tf#L67) | Host project for the shared VPC. | object({…}) | | null | 02-networking | ## Outputs diff --git a/fast/stages/03-project-factory/dev/main.tf b/fast/stages/03-project-factory/dev/main.tf index ac562fe9..db95c7c2 100644 --- a/fast/stages/03-project-factory/dev/main.tf +++ b/fast/stages/03-project-factory/dev/main.tf @@ -20,10 +20,10 @@ locals { _defaults = yamldecode(file(var.defaults_file)) _defaults_net = { - billing_account_id = var.billing_account_id + billing_account_id = var.billing_account.id environment_dns_zone = var.environment_dns_zone - shared_vpc_self_link = var.shared_vpc_self_link - vpc_host_project = var.vpc_host_project + shared_vpc_self_link = try(var.shared_vpc_self_links["dev:spoke-0"], null) + vpc_host_project = try(var.vpc_host_project_ids["dev:spoke-0"], null) } defaults = merge(local._defaults, local._defaults_net) projects = { diff --git a/fast/stages/03-project-factory/dev/variables.tf b/fast/stages/03-project-factory/dev/variables.tf index a580260c..3773902d 100644 --- a/fast/stages/03-project-factory/dev/variables.tf +++ b/fast/stages/03-project-factory/dev/variables.tf @@ -16,10 +16,13 @@ #TODO: tfdoc annotations -variable "billing_account_id" { +variable "billing_account" { # tfdoc:variable:source 00-bootstrap - description = "Billing account id." - type = string + description = "Billing account id and organization id ('nnnnnnnn' or null)." + type = object({ + id = string + organization_id = number + }) } variable "data_dir" { @@ -52,16 +55,20 @@ variable "prefix" { } } -variable "shared_vpc_self_link" { +variable "shared_vpc_self_links" { # tfdoc:variable:source 02-networking description = "Self link for the shared VPC." - type = string - default = null + type = object({ + dev-spoke-0 = string + }) + default = null } -variable "vpc_host_project" { +variable "vpc_host_project_ids" { # tfdoc:variable:source 02-networking description = "Host project for the shared VPC." - type = string - default = null + type = object({ + dev-spoke-0 = string + }) + default = null } diff --git a/fast/stages/README.md b/fast/stages/README.md index 4bb70e0a..93965031 100644 --- a/fast/stages/README.md +++ b/fast/stages/README.md @@ -2,23 +2,38 @@ Each of the folders contained here is a separate "stage", or Terraform root module. -They are designed to be combined together, each stage leveraging the previous stage's resources and providing outputs to the following stages, but they can also be run in isolation if their specific functionality is all that is needed (e.g. only bring up a hub and spoke VPC in an existing environment). +Each stage can be run in isolation (for example to only bring up a hub and spoke VPC in an existing environment), but when combined together they form a modular setup that allows top-down configuration of a whole GCP organization. + +When combined together, each stage is designed to leverage the previous stage's resources and to provide outputs to the following stages via predefined contracts, that regulate what is exchanged. + +This has two important consequences + +- any stage can be swapped out and replaced by different code as long as it respects the contract by providing a predefined set of outputs and optionally accepting a predefined set of variables +- data flow between stages can be partially automated, reducing the effort and pain required to compile variables by hand + +One important assumption is that the flow of data is always forward looking, so no stage should depend on outputs generated further down the chain. This greatly simplifies both the logic and the implementation, and allows stages to be effectively independent. + +To achieve this, we rely on specific GCP functionality like [delegated role grants](https://medium.com/google-cloud/managing-gcp-service-usage-through-delegated-role-grants-a843610f2226) that allow controlled delegation of responsibilities, for example to allow managing IAM bindings at the organization level only for specific roles. Refer to each stage's documentation for a detailed description of its purpose, the architectural choices made in its design, and how it can be configured and wired together to terraform a whole GCP organization. The following is a brief overview of each stage. ## Organizational level (00-01) - [Bootstrap](00-bootstrap/README.md) - Enables critical organization-level functionality that depends on broad permissions. It has two primary purposes. The first is to bootstrap the resources needed for automation of this and the following stages (service accounts, GCS buckets). And secondly, it applies the minimum amount of configuration needed at the organization level, to avoid the need of broad permissions later on, and to implement a minimum of security features like sinks and exports from the start. + Enables critical organization-level functionality that depends on broad permissions. It has two primary purposes. The first is to bootstrap the resources needed for automation of this and the following stages (service accounts, GCS buckets). And secondly, it applies the minimum amount of configuration needed at the organization level, to avoid the need of broad permissions later on, and to implement a minimum of security features like sinks and exports from the start.\ + Exports: automation project id, organization-level custom roles - [Resource Management](01-resman/README.md) - Creates the base resource hierarchy (folders) and the automation resources required later to delegate deployment of each part of the hierarchy to separate stages. This stage also configures organization-level policies and any exceptions needed by different branches of the resource hierarchy. + Creates the base resource hierarchy (folders) and the automation resources required later to delegate deployment of each part of the hierarchy to separate stages. This stage also configures organization-level policies and any exceptions needed by different branches of the resource hierarchy.\ + Exports: folder ids, automation service account emails ## Shared resources (02) - [Security](02-security/README.md) - Manages centralized security configurations in a separate stage, and is typically owned by the security team. This stage implements VPC Security Controls via separate perimeters for environments and central services, and creates projects to host centralized KMS keys used by the whole organization. It's meant to be easily extended to include other security-related resources which are required, like Secret Manager. + Manages centralized security configurations in a separate stage, and is typically owned by the security team. This stage implements VPC Security Controls via separate perimeters for environments and central services, and creates projects to host centralized KMS keys used by the whole organization. It's meant to be easily extended to include other security-related resources which are required, like Secret Manager.\ + Exports: KMS key ids - Networking ([VPN](02-networking-vpn/README.md)/[NVA](02-networking-nva/README.md)) - Manages centralized network resources in a separate stage, and is typically owned by the networking team. This stage implements a hub-and-spoke design, and includes connectivity via VPN to on-premises, and YAML-based factories for firewall rules (hierarchical and VPC-level) and subnets. It's currently available in two versions: [spokes connected via VPN](02-networking-vpn/README.md), [and spokes connected via appliances](02-networking-nva/README.md). + Manages centralized network resources in a separate stage, and is typically owned by the networking team. This stage implements a hub-and-spoke design, and includes connectivity via VPN to on-premises, and YAML-based factories for firewall rules (hierarchical and VPC-level) and subnets. It's currently available in two versions: [spokes connected via VPN](02-networking-vpn/README.md), [and spokes connected via appliances](02-networking-nva/README.md).\ + Exports: host project ids and numbers, vpc self links ## Environment-level resources (03)