diff --git a/fast/stages/00-bootstrap/README.md b/fast/stages/00-bootstrap/README.md index 3842aa07..a9b804e9 100644 --- a/fast/stages/00-bootstrap/README.md +++ b/fast/stages/00-bootstrap/README.md @@ -54,6 +54,11 @@ For same-organization billing, we configure a custom organization role that can For details on configuring the different billing account modes, refer to the [How to run this stage](#how-to-run-this-stage) section below. +### Organization-level logging +We create organization-level log sinks early in the bootstrap process to ensure a proper audit trail is in place from the very beginning. By default, we provide log filters to capture [Cloud Audit Logs](https://cloud.google.com/logging/docs/audit) and [VPC Service Controls violations](https://cloud.google.com/vpc-service-controls/docs/troubleshooting#vpc-sc-errors) into a Bigquery dataset in the top-level audit project. + +The [Customizations](#log-sinks-and-log-destinations) section explains how to change the logs captured and their destination. + ### Naming We are intentionally not supporting random prefix/suffixes for names, as that is an antipattern typically only used in development. It does not map to our customer's actual production usage, where they always adopt a fixed naming convention. @@ -278,6 +283,17 @@ In those cases where roles need to be assigned to end-user service accounts (e.g The one exception to this convention is for roles which are part of the delegated grant condition described above, and which can then be assigned from other stages. In this case, use the `iam_additive` variable as they are implemented with non-authoritative resources. Using non-authoritative bindings ensure that re-executing this stage will not override any bindings set in downstream stages. +### Log sinks and log destinations + +You can customize organization-level logs through the `log_sinks` variable in two ways: + +* creating additional log sinks to capture more logs +* changing the destination of captured logs + +By default, all logs are exported to Bigquery, but FAST can create sinks to Cloud Logging Buckets, GCS, or PubSub. + +If you need to capture additional logs, please refer to GCP's documentation on [scenarios for exporting logging data](https://cloud.google.com/architecture/exporting-stackdriver-logging-for-security-and-access-analytics), where you can find ready-made filter expressions for different use cases. + ### Names and naming convention Configuring the individual tokens for the naming convention described above, has varying degrees of complexity: @@ -311,22 +327,23 @@ Names used in internal references (e.g. `module.foo-prod.id`) are only used by T | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| | [billing_account](variables.tf#L17) | Billing account id and organization id ('nnnnnnnn' or null). | object({…}) | ✓ | | | -| [organization](variables.tf#L82) | Organization details. | object({…}) | ✓ | | | -| [prefix](variables.tf#L97) | Prefix used for resources that need unique names. | string | ✓ | | | +| [organization](variables.tf#L96) | Organization details. | object({…}) | ✓ | | | +| [prefix](variables.tf#L111) | Prefix used for resources that need unique names. | string | ✓ | | | | [bootstrap_user](variables.tf#L25) | Email of the nominal user running this stage for the first time. | string | | null | | -| [groups](variables.tf#L31) | Group names to grant organization-level permissions. | map(string) | | {…} | | -| [iam](variables.tf#L45) | Organization-level custom IAM settings in role => [principal] format. | map(list(string)) | | {} | | -| [iam_additive](variables.tf#L51) | Organization-level custom IAM settings in role => [principal] format for non-authoritative bindings. | map(list(string)) | | {} | | -| [log_sinks](variables.tf#L57) | Org-level log sinks, in name => {type, filter} format. | map(object({…})) | | {…} | | -| [outputs_location](variables.tf#L91) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | string | | null | | +| [custom_role_names](variables.tf#L31) | Names of custom roles defined at the org level. | object({…}) | | {…} | | +| [groups](variables.tf#L43) | Group names to grant organization-level permissions. | map(string) | | {…} | | +| [iam](variables.tf#L57) | Organization-level custom IAM settings in role => [principal] format. | map(list(string)) | | {} | | +| [iam_additive](variables.tf#L63) | Organization-level custom IAM settings in role => [principal] format for non-authoritative bindings. | map(list(string)) | | {} | | +| [log_sinks](variables.tf#L71) | Org-level log sinks, in name => {type, filter} format. | map(object({…})) | | {…} | | +| [outputs_location](variables.tf#L105) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | string | | null | | ## Outputs | name | description | sensitive | consumers | |---|---|:---:|---| -| [billing_dataset](outputs.tf#L85) | BigQuery dataset prepared for billing export. | | | -| [project_ids](outputs.tf#L90) | Projects created by this stage. | | | -| [providers](outputs.tf#L101) | Terraform provider files for this stage and dependent stages. | ✓ | stage-01 | -| [tfvars](outputs.tf#L110) | Terraform variable files for the following stages. | ✓ | | +| [billing_dataset](outputs.tf#L89) | BigQuery dataset prepared for billing export. | | | +| [project_ids](outputs.tf#L94) | Projects created by this stage. | | | +| [providers](outputs.tf#L105) | Terraform provider files for this stage and dependent stages. | ✓ | stage-01 | +| [tfvars](outputs.tf#L114) | Terraform variable files for the following stages. | ✓ | | diff --git a/fast/stages/00-bootstrap/billing.tf b/fast/stages/00-bootstrap/billing.tf index e6abc31d..e8227041 100644 --- a/fast/stages/00-bootstrap/billing.tf +++ b/fast/stages/00-bootstrap/billing.tf @@ -73,7 +73,10 @@ resource "google_organization_iam_binding" "billing_org_ext_admin_delegated" { org_id = var.billing_account.organization_id # if the billing org does not have our custom role, user the predefined one # role = "roles/resourcemanager.organizationAdmin" - role = "organizations/${var.billing_account.organization_id}/roles/organizationIamAdmin" + role = join("", [ + "organizations/${var.billing_account.organization_id}/", + "roles/${var.custom_role_names.organization_iam_admin}" + ]) members = [module.automation-tf-resman-sa.iam_email] condition { title = "automation_sa_delegated_grants" diff --git a/fast/stages/00-bootstrap/organization.tf b/fast/stages/00-bootstrap/organization.tf index ea2e4cef..689f378b 100644 --- a/fast/stages/00-bootstrap/organization.tf +++ b/fast/stages/00-bootstrap/organization.tf @@ -147,12 +147,12 @@ module "organization" { iam_additive = local.iam_additive custom_roles = { # this is needed for use in additive IAM bindings, to avoid conflicts - "organizationIamAdmin" = [ + (var.custom_role_names.organization_iam_admin) = [ "resourcemanager.organizations.get", "resourcemanager.organizations.getIamPolicy", "resourcemanager.organizations.setIamPolicy" ] - "serviceProjectNetworkAdmin" = [ + (var.custom_role_names.service_project_network_admin) = [ "compute.globalOperations.get", "compute.organizations.disableXpnResource", "compute.organizations.enableXpnResource", @@ -182,7 +182,7 @@ module "organization" { resource "google_organization_iam_binding" "org_admin_delegated" { org_id = var.organization.id - role = module.organization.custom_role_id.organizationIamAdmin + role = module.organization.custom_role_id[var.custom_role_names.organization_iam_admin] members = [module.automation-tf-resman-sa.iam_email] condition { title = "automation_sa_delegated_grants" diff --git a/fast/stages/00-bootstrap/outputs.tf b/fast/stages/00-bootstrap/outputs.tf index d7924781..d074d91f 100644 --- a/fast/stages/00-bootstrap/outputs.tf +++ b/fast/stages/00-bootstrap/outputs.tf @@ -15,6 +15,10 @@ */ locals { + _custom_roles = { + for k, v in var.custom_role_names : + k => module.organization.custom_role_id[v] + } providers = { "00-bootstrap" = templatefile("${path.module}/../../assets/templates/providers.tpl", { bucket = module.automation-tf-bootstrap-gcs.name @@ -31,14 +35,14 @@ locals { "01-resman" = jsonencode({ automation_project_id = module.automation-project.project_id billing_account = var.billing_account - custom_roles = module.organization.custom_role_id + custom_roles = local._custom_roles groups = var.groups organization = var.organization prefix = var.prefix }) "02-networking" = jsonencode({ billing_account_id = var.billing_account.id - custom_roles = module.organization.custom_role_id + custom_roles = local._custom_roles organization = var.organization prefix = var.prefix }) diff --git a/fast/stages/00-bootstrap/terraform.tfvars.sample b/fast/stages/00-bootstrap/terraform.tfvars.sample index e9b9a90a..7dbe2e6b 100644 --- a/fast/stages/00-bootstrap/terraform.tfvars.sample +++ b/fast/stages/00-bootstrap/terraform.tfvars.sample @@ -1,4 +1,5 @@ # use `gcloud beta billing accounts list` +# if you have too many accounts, check the Cloud Console :) billing_account = { id = "012345-67890A-BCDEF0" organization_id = 1234567890 diff --git a/fast/stages/00-bootstrap/variables.tf b/fast/stages/00-bootstrap/variables.tf index 9f102c77..33709703 100644 --- a/fast/stages/00-bootstrap/variables.tf +++ b/fast/stages/00-bootstrap/variables.tf @@ -28,6 +28,18 @@ variable "bootstrap_user" { default = null } +variable "custom_role_names" { + description = "Names of custom roles defined at the org level." + type = object({ + organization_iam_admin = string + service_project_network_admin = string + }) + default = { + organization_iam_admin = "organizationIamAdmin" + service_project_network_admin = "serviceProjectNetworkAdmin" + } +} + variable "groups" { # https://cloud.google.com/docs/enterprise/setup-checklist description = "Group names to grant organization-level permissions." @@ -54,6 +66,8 @@ variable "iam_additive" { default = {} } +# See https://cloud.google.com/architecture/exporting-stackdriver-logging-for-security-and-access-analytics +# for additional logging filter examples variable "log_sinks" { description = "Org-level log sinks, in name => {type, filter} format." type = map(object({ diff --git a/fast/stages/02-networking-vpn/vpc-spoke-dev.tf b/fast/stages/02-networking-vpn/vpc-spoke-dev.tf index 9b3c0f9e..4a3f0f25 100644 --- a/fast/stages/02-networking-vpn/vpc-spoke-dev.tf +++ b/fast/stages/02-networking-vpn/vpc-spoke-dev.tf @@ -40,7 +40,7 @@ module "dev-spoke-project" { metric_scopes = [module.landing-project.project_id] iam = { "roles/dns.admin" = [var.project_factory_sa.dev] - (var.custom_roles.serviceProjectNetworkAdmin) = [ + (var.custom_roles.service_project_network_admin) = [ var.project_factory_sa.prod ] } diff --git a/fast/stages/02-networking-vpn/vpc-spoke-prod.tf b/fast/stages/02-networking-vpn/vpc-spoke-prod.tf index 7f42ab2c..3be90c2e 100644 --- a/fast/stages/02-networking-vpn/vpc-spoke-prod.tf +++ b/fast/stages/02-networking-vpn/vpc-spoke-prod.tf @@ -40,7 +40,7 @@ module "prod-spoke-project" { metric_scopes = [module.landing-project.project_id] iam = { "roles/dns.admin" = [var.project_factory_sa.prod] - (var.custom_roles.serviceProjectNetworkAdmin) = [ + (var.custom_roles.service_project_network_admin) = [ var.project_factory_sa.prod ] } diff --git a/fast/stages/02-security/vpc-sc.tf b/fast/stages/02-security/vpc-sc.tf index b3125541..49611e0f 100644 --- a/fast/stages/02-security/vpc-sc.tf +++ b/fast/stages/02-security/vpc-sc.tf @@ -15,6 +15,7 @@ */ locals { + _perimeter_names = ["dev", "landing", "prod"] # dereference perimeter egress policy names to the actual objects _vpc_sc_perimeter_egress_policies = { for k, v in coalesce(var.vpc_sc_perimeter_egress_policies, {}) : @@ -33,8 +34,8 @@ locals { } # compute the number of projects in each perimeter to detect which to create vpc_sc_counts = { - for k in ["dev", "landing", "prod"] : k => length( - coalesce(try(var.vpc_sc_perimeter_projects[k], null), []) + for k in local._perimeter_names : k => length( + local.vpc_sc_perimeter_projects[k] ) } # define dry run spec at file level for convenience @@ -42,12 +43,12 @@ locals { # compute perimeter bridge resources (projects) vpc_sc_p_bridge_resources = { landing_to_dev = concat( - var.vpc_sc_perimeter_projects.landing, - var.vpc_sc_perimeter_projects.dev + local.vpc_sc_perimeter_projects.landing, + local.vpc_sc_perimeter_projects.dev ) landing_to_prod = concat( - var.vpc_sc_perimeter_projects.landing, - var.vpc_sc_perimeter_projects.prod + local.vpc_sc_perimeter_projects.landing, + local.vpc_sc_perimeter_projects.prod ) } # computer perimeter regular specs / status @@ -56,7 +57,7 @@ locals { access_levels = coalesce( try(var.vpc_sc_perimeter_access_levels.dev, null), [] ) - resources = var.vpc_sc_perimeter_projects.dev + resources = local.vpc_sc_perimeter_projects.dev restricted_services = local.vpc_sc_restricted_services egress_policies = try( local._vpc_sc_perimeter_egress_policies.dev, null @@ -74,7 +75,7 @@ locals { access_levels = coalesce( try(var.vpc_sc_perimeter_access_levels.landing, null), [] ) - resources = var.vpc_sc_perimeter_projects.landing + resources = local.vpc_sc_perimeter_projects.landing restricted_services = local.vpc_sc_restricted_services egress_policies = try( local._vpc_sc_perimeter_egress_policies.landing, null @@ -93,7 +94,7 @@ locals { try(var.vpc_sc_perimeter_access_levels.prod, null), [] ) # combine the security project, and any specified in the variable - resources = var.vpc_sc_perimeter_projects.prod + resources = local.vpc_sc_perimeter_projects.prod restricted_services = local.vpc_sc_restricted_services egress_policies = try( local._vpc_sc_perimeter_egress_policies.prod, null @@ -108,6 +109,17 @@ locals { # } } } + # account for null values in variable + vpc_sc_perimeter_projects = ( + var.vpc_sc_perimeter_projects == null ? + { + for k in local._perimeter_names : k => [] + } + : { + for k, v in local._perimeter_names : + k => v == null ? [] : v + } + ) # get the list of restricted services from the yaml file vpc_sc_restricted_services = yamldecode( file("${path.module}/vpc-sc-restricted-services.yaml")