cloud-foundation-fabric/modules/project

Project Module

This module implements the creation and management of one GCP project including IAM, organization policies, Shared VPC host or service attachment, service API activation, and tag attachment. It also offers a convenient way to refer to managed service identities (aka robot service accounts) for APIs.

TOC

• TOC
• Basic Project Creation
• IAM
  • Authoritative IAM
  • Additive IAM
  • Additive IAM by Member
  • Service Identities and Authoritative IAM
  • Using Shortcodes for Service Identities in Additive Iam
  • Service Identities Requiring Manual Iam Grants
• Shared VPC
• Organization Policies
  • Organization Policy Factory
• Log Sinks
• Data Access Logs
• Cloud Kms Encryption Keys
• Tags
• Outputs
• Files
• Variables
• Outputs

Basic Project Creation

module "project" {
  source          = "./fabric/modules/project"
  billing_account = "123456-123456-123456"
  name            = "myproject"
  parent          = "folders/1234567890"
  prefix          = "foo"
  services = [
    "container.googleapis.com",
    "stackdriver.googleapis.com"
  ]
}
# tftest modules=1 resources=3 inventory=basic.yaml

IAM

IAM is managed via several variables that implement different levels of control:

• group_iam and iam configure authoritative bindings that manage individual roles exclusively, mapping to the google_project_iam_binding resource
• iam_additive and iam_additive_members configure additive bindings that only manage individual role/member pairs, mapping to the google_project_iam_member resource
• iam_policy which controls the entire IAM policy for the project, where any binding created outside this module (eg in the console) will be removed at each terraform apply cycle regardless of the role

The authoritative and additive approaches can be used together, provided different roles are managed by each. The IAM policy is incompatible with the other approaches, and must be used with extreme care.

Be mindful about service identity roles when using authoritative IAM, as you might inadvertently remove a role from a service identity or default service account. For example, using roles/editor with iam or group_iam will remove the default permissions for the Cloud Services identity. A simple workaround for these scenarios is described below.

Authoritative IAM

The iam variable is based on role keys and is typically used for service accounts, or where member values can be dynamic and would create potential problems in the underlying for_each cycle.

locals {
  gke_service_account = "my_gke_service_account"
}

module "project" {
  source          = "./fabric/modules/project"
  billing_account = "123456-123456-123456"
  name            = "project-example"
  parent          = "folders/1234567890"
  prefix          = "foo"
  services = [
    "container.googleapis.com",
    "stackdriver.googleapis.com"
  ]
  iam = {
    "roles/container.hostServiceAgentUser" = [
      "serviceAccount:${local.gke_service_account}"
    ]
  }
}
# tftest modules=1 resources=4 inventory=iam-authoritative.yaml

The group_iam variable uses group email addresses as keys and is a convenient way to assign roles to humans following Google's best practices. The end result is readable code that also serves as documentation.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = "123456-123456-123456"
  name            = "project-example"
  parent          = "folders/1234567890"
  prefix          = "foo"
  group_iam = {
    "gcp-security-admins@example.com" = [
      "roles/cloudasset.owner",
      "roles/cloudsupport.techSupportEditor",
      "roles/iam.securityReviewer",
      "roles/logging.admin",
    ]
  }
}
# tftest modules=1 resources=5 inventory=iam-group.yaml

Additive IAM

Additive IAM is typically used where bindings for specific roles are controlled by different modules or in different Terraform stages. One example is when the project is created by one team but a different team manages service account creation for the project, and some of the project-level roles overlap in the two configurations.

module "project" {
  source = "./fabric/modules/project"
  name   = "project-example"
  iam_additive = {
    "roles/viewer" = [
      "group:one@example.org",
      "group:two@xample.org"
    ],
    "roles/storage.objectAdmin" = [
      "group:two@example.org"
    ],
    "roles/owner" = [
      "group:three@example.org"
    ],
  }
}
# tftest modules=1 resources=5 inventory=iam-additive.yaml

Additive IAM by Member

module "project" {
  source = "./fabric/modules/project"
  name   = "project-example"
  iam_additive_members = {
    "user:one@example.org" = ["roles/owner"]
    "user:two@example.org" = ["roles/owner", "roles/editor"]
  }

}
# tftest modules=1 resources=4 inventory=iam-additive-members.yaml

Service Identities and Authoritative IAM

As mentioned above, there are cases where authoritative management of specific IAM roles results in removal of default bindings from service identities. One example is outlined below, with a simple workaround leveraging the service_accounts output to identify the service identity. A full list of service identities and their roles can be found here.

module "project" {
  source = "./fabric/modules/project"
  name   = "project-example"
  group_iam = {
    "foo@example.com" = [
      "roles/editor"
    ]
  }
  iam = {
    "roles/editor" = [
      "serviceAccount:${module.project.service_accounts.cloud_services}"
    ]
  }
}
# tftest modules=1 resources=2

Using Shortcodes for Service Identities in Additive Iam

Most Service Identities contains project number in their e-mail address and this prevents additive IAM to work, as these values are not known at moment of execution of terraform plan (its not an issue for authoritative IAM). To refer current project Service Identities you may use shortcodes for Service Identities similarly as for service_identity_iam when configuring Shared VPC.

module "project" {
  source = "./fabric/modules/project"
  name   = "project-example"

  services = [
    "run.googleapis.com",
    "container.googleapis.com",
  ]

  iam_additive = {
    "roles/editor"                         = ["cloudservices"]
    "roles/vpcaccess.user"                 = ["cloudrun"]
    "roles/container.hostServiceAgentUser" = ["container-engine"]
  }
}
# tftest modules=1 resources=6

Service Identities Requiring Manual Iam Grants

The module will create service identities at project creation instead of creating of them at the time of first use. This allows granting these service identities roles in other projects, something which is usually necessary in a Shared VPC context.

You can grant roles to service identities using the following construct:

module "project" {
  source = "./fabric/modules/project"
  name   = "project-example"
  iam = {
    "roles/apigee.serviceAgent" = [
      "serviceAccount:${module.project.service_accounts.robots.apigee}"
    ]
  }
}
# tftest modules=1 resources=2

This table lists all affected services and roles that you need to grant to service identities

service	service identity	role
apigee.googleapis.com	apigee	roles/apigee.serviceAgent
artifactregistry.googleapis.com	artifactregistry	roles/artifactregistry.serviceAgent
cloudasset.googleapis.com	cloudasset	roles/cloudasset.serviceAgent
cloudbuild.googleapis.com	cloudbuild	roles/cloudbuild.builds.builder
dataplex.googleapis.com	dataplex	roles/dataplex.serviceAgent
gkehub.googleapis.com	fleet	roles/gkehub.serviceAgent
meshconfig.googleapis.com	servicemesh	roles/anthosservicemesh.serviceAgent
multiclusteringress.googleapis.com	multicluster-ingress	roles/multiclusteringress.serviceAgent
pubsub.googleapis.com	pubsub	roles/pubsub.serviceAgent
sqladmin.googleapis.com	sqladmin	roles/cloudsql.serviceAgent

Shared VPC

The module allows managing Shared VPC status for both hosts and service projects, and includes a simple way of assigning Shared VPC roles to service identities.

You can enable Shared VPC Host at the project level and manage project service association independently.

module "host-project" {
  source = "./fabric/modules/project"
  name   = "my-host-project"
  shared_vpc_host_config = {
    enabled = true
  }
}

module "service-project" {
  source = "./fabric/modules/project"
  name   = "my-service-project"
  shared_vpc_service_config = {
    host_project = module.host-project.project_id
    service_identity_iam = {
      "roles/compute.networkUser" = [
        "cloudservices", "container-engine"
      ]
      "roles/vpcaccess.user" = [
        "cloudrun"
      ]
      "roles/container.hostServiceAgentUser" = [
        "container-engine"
      ]
    }
  }
}
# tftest modules=2 resources=8 inventory=shared-vpc.yaml

The module allows also granting necessary permissions in host project to service identities by specifying which services will be used in service project in grant_iam_for_services.

module "host-project" {
  source = "./fabric/modules/project"
  name   = "my-host-project"
  shared_vpc_host_config = {
    enabled = true
  }
}

module "service-project" {
  source = "./fabric/modules/project"
  name   = "my-service-project"
  services = [
    "container.googleapis.com",
  ]
  shared_vpc_service_config = {
    host_project       = module.host-project.project_id
    service_iam_grants = module.service-project.services
  }
}
# tftest modules=2 resources=9 inventory=shared-vpc-auto-grants.yaml

Organization Policies

To manage organization policies, the orgpolicy.googleapis.com service should be enabled in the quota project.

module "project" {
  source          = "./fabric/modules/project"
  billing_account = "123456-123456-123456"
  name            = "project-example"
  parent          = "folders/1234567890"
  prefix          = "foo"
  org_policies = {
    "compute.disableGuestAttributesAccess" = {
      rules = [{ enforce = true }]
    }
    "compute.skipDefaultNetworkCreation" = {
      rules = [{ enforce = true }]
    }
    "iam.disableServiceAccountKeyCreation" = {
      rules = [{ enforce = true }]
    }
    "iam.disableServiceAccountKeyUpload" = {
      rules = [
        {
          condition = {
            expression  = "resource.matchTagId('tagKeys/1234', 'tagValues/1234')"
            title       = "condition"
            description = "test condition"
            location    = "somewhere"
          }
          enforce = true
        },
        {
          enforce = false
        }
      ]
    }
    "iam.allowedPolicyMemberDomains" = {
      rules = [{
        allow = {
          values = ["C0xxxxxxx", "C0yyyyyyy"]
        }
      }]
    }
    "compute.trustedImageProjects" = {
      rules = [{
        allow = {
          values = ["projects/my-project"]
        }
      }]
    }
    "compute.vmExternalIpAccess" = {
      rules = [{ deny = { all = true } }]
    }
  }
}
# tftest modules=1 resources=8 inventory=org-policies.yaml

Organization Policy Factory

Organization policies can be loaded from a directory containing YAML files where each file defines one or more constraints. The structure of the YAML files is exactly the same as the org_policies variable.

Note that constraints defined via org_policies take precedence over those in org_policies_data_path. In other words, if you specify the same constraint in a YAML file and in the org_policies variable, the latter will take priority.

The example below deploys a few organization policies split between two YAML files.

module "project" {
  source                 = "./fabric/modules/project"
  billing_account        = "123456-123456-123456"
  name                   = "project-example"
  parent                 = "folders/1234567890"
  prefix                 = "foo"
  org_policies_data_path = "configs/org-policies/"
}
# tftest modules=1 resources=8 files=boolean,list inventory=org-policies.yaml

# tftest-file id=boolean path=configs/org-policies/boolean.yaml
compute.disableGuestAttributesAccess:
  rules:
  - enforce: true
compute.skipDefaultNetworkCreation:
  rules:
  - enforce: true
iam.disableServiceAccountKeyCreation:
  rules:
  - enforce: true
iam.disableServiceAccountKeyUpload:
  rules:
  - condition:
      description: test condition
      expression: resource.matchTagId('tagKeys/1234', 'tagValues/1234')
      location: somewhere
      title: condition
    enforce: true
  - enforce: false

# tftest-file id=list path=configs/org-policies/list.yaml
compute.trustedImageProjects:
  rules:
  - allow:
      values:
      - projects/my-project
compute.vmExternalIpAccess:
  rules:
  - deny:
      all: true
iam.allowedPolicyMemberDomains:
  rules:
  - allow:
      values:
      - C0xxxxxxx
      - C0yyyyyyy

Log Sinks

module "gcs" {
  source        = "./fabric/modules/gcs"
  project_id    = var.project_id
  name          = "gcs_sink"
  force_destroy = true
}

module "dataset" {
  source     = "./fabric/modules/bigquery-dataset"
  project_id = var.project_id
  id         = "bq_sink"
}

module "pubsub" {
  source     = "./fabric/modules/pubsub"
  project_id = var.project_id
  name       = "pubsub_sink"
}

module "bucket" {
  source      = "./fabric/modules/logging-bucket"
  parent_type = "project"
  parent      = "my-project"
  id          = "bucket"
}

module "project-host" {
  source          = "./fabric/modules/project"
  name            = "my-project"
  billing_account = "123456-123456-123456"
  parent          = "folders/1234567890"
  logging_sinks = {
    warnings = {
      destination = module.gcs.id
      filter      = "severity=WARNING"
      type        = "storage"
    }
    info = {
      destination = module.dataset.id
      filter      = "severity=INFO"
      type        = "bigquery"
    }
    notice = {
      destination = module.pubsub.id
      filter      = "severity=NOTICE"
      type        = "pubsub"
    }
    debug = {
      destination = module.bucket.id
      filter      = "severity=DEBUG"
      exclusions = {
        no-compute = "logName:compute"
      }
      type = "logging"
    }
  }
  logging_exclusions = {
    no-gce-instances = "resource.type=gce_instance"
  }
}
# tftest modules=5 resources=14 inventory=logging.yaml

Data Access Logs

Activation of data access logs can be controlled via the logging_data_access variable. If the iam_bindings_authoritative variable is used to set a resource-level IAM policy, the data access log configuration will also be authoritative as part of the policy.

This example shows how to set a non-authoritative access log configuration:

module "project" {
  source          = "./fabric/modules/project"
  name            = "my-project"
  billing_account = "123456-123456-123456"
  parent          = "folders/1234567890"
  logging_data_access = {
    allServices = {
      # logs for principals listed here will be excluded
      ADMIN_READ = ["group:organization-admins@example.org"]
    }
    "storage.googleapis.com" = {
      DATA_READ  = []
      DATA_WRITE = []
    }
  }
}
# tftest modules=1 resources=3 inventory=logging-data-access.yaml

While this sets an authoritative policies that has exclusive control of both IAM bindings for all roles and data access log configuration, and should be used with extreme care:

module "project" {
  source          = "./fabric/modules/project"
  name            = "my-project"
  billing_account = "123456-123456-123456"
  parent          = "folders/1234567890"
  iam_policy = {
    "roles/owner"                             = ["group:org-admins@example.com"]
    "roles/resourcemanager.folderAdmin"       = ["group:org-admins@example.com"]
    "roles/resourcemanager.organizationAdmin" = ["group:org-admins@example.com"]
    "roles/resourcemanager.projectCreator"    = ["group:org-admins@example.com"]
  }
  logging_data_access = {
    allServices = {
      ADMIN_READ = ["group:organization-admins@example.org"]
    }
    "storage.googleapis.com" = {
      DATA_READ  = []
      DATA_WRITE = []
    }
  }
}
# tftest modules=1 resources=2 inventory=iam-policy.yaml

Cloud Kms Encryption Keys

The module offers a simple, centralized way to assign roles/cloudkms.cryptoKeyEncrypterDecrypter to service identities.

module "project" {
  source = "./fabric/modules/project"
  name   = "my-project"
  prefix = "foo"
  services = [
    "compute.googleapis.com",
    "storage.googleapis.com"
  ]
  service_encryption_key_ids = {
    compute = [
      "projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce",
      "projects/kms-central-prj/locations/europe-west4/keyRings/my-keyring/cryptoKeys/europe4-gce"
    ]
    storage = [
      "projects/kms-central-prj/locations/europe/keyRings/my-keyring/cryptoKeys/europe-gcs"
    ]
  }
}
# tftest modules=1 resources=7

Tags

Refer to the Creating and managing tags documentation for details on usage.

module "org" {
  source          = "./fabric/modules/organization"
  organization_id = var.organization_id
  tags = {
    environment = {
      description = "Environment specification."
      iam         = null
      values = {
        dev  = null
        prod = null
      }
    }
  }
}

module "project" {
  source = "./fabric/modules/project"
  name   = "test-project"
  tag_bindings = {
    env-prod = module.org.tag_values["environment/prod"].id
    foo      = "tagValues/12345678"
  }
}
# tftest modules=2 resources=6

Outputs

Most of this module's outputs depend on its resources, to allow Terraform to compute all dependencies required for the project to be correctly configured. This allows you to reference outputs like project_id in other modules or resources without having to worry about setting depends_on blocks manually.

One non-obvious output is service_accounts, which offers a simple way to discover service identities and default service accounts, and guarantees that service identities that require an API call to trigger creation (like GCS or BigQuery) exist before use.

module "project" {
  source = "./fabric/modules/project"
  name   = "project-example"
  services = [
    "compute.googleapis.com"
  ]
}

output "compute_robot" {
  value = module.project.service_accounts.robots.compute
}
# tftest modules=1 resources=2 inventory:outputs.yaml

Files

name	description	resources
iam.tf	Generic and OSLogin-specific IAM bindings and roles.	google_project_iam_binding · google_project_iam_custom_role · google_project_iam_member · google_project_iam_policy
logging.tf	Log sinks and supporting resources.	google_bigquery_dataset_iam_member · google_logging_project_exclusion · google_logging_project_sink · google_project_iam_audit_config · google_project_iam_member · google_pubsub_topic_iam_member · google_storage_bucket_iam_member
main.tf	Module-level locals and resources.	google_compute_project_metadata_item · google_essential_contacts_contact · google_monitoring_monitored_project · google_project · google_project_service · google_resource_manager_lien
organization-policies.tf	Project-level organization policies.	google_org_policy_policy
outputs.tf	Module outputs.
service-accounts.tf	Service identities and supporting resources.	google_kms_crypto_key_iam_member · google_project_default_service_accounts · google_project_iam_member · google_project_service_identity
shared-vpc.tf	Shared VPC project-level configuration.	google_compute_shared_vpc_host_project · google_compute_shared_vpc_service_project · google_project_iam_member
tags.tf	None	google_tags_tag_binding
variables.tf	Module variables.
versions.tf	Version pins.
vpc-sc.tf	VPC-SC project-level perimeter configuration.	google_access_context_manager_service_perimeter_resource

Variables

name	description	type	required	default
name	Project name and id suffix.	string	✓
auto_create_network	Whether to create the default network for the project.	bool		false
billing_account	Billing account id.	string		null
contacts	List of essential contacts for this resource. Must be in the form EMAIL -> [NOTIFICATION_TYPES]. Valid notification types are ALL, SUSPENSION, SECURITY, TECHNICAL, BILLING, LEGAL, PRODUCT_UPDATES.	map(list(string))		{}
custom_roles	Map of role name => list of permissions to create in this project.	map(list(string))		{}
default_service_account	Project default service account setting: can be one of delete, deprivilege, disable, or keep.	string		"keep"
descriptive_name	Name of the project name. Used for project name instead of name variable.	string		null
group_iam	Authoritative IAM binding for organization groups, in {GROUP_EMAIL => [ROLES]} format. Group emails need to be static. Can be used in combination with the iam variable.	map(list(string))		{}
iam	IAM bindings in {ROLE => [MEMBERS]} format.	map(list(string))		{}
iam_additive	IAM additive bindings in {ROLE => [MEMBERS]} format.	map(list(string))		{}
iam_additive_members	IAM additive bindings in {MEMBERS => [ROLE]} format. This might break if members are dynamic values.	map(list(string))		{}
iam_policy	IAM authoritative policy in {ROLE => [MEMBERS]} format. Roles and members not explicitly listed will be cleared, use with extreme caution.	map(list(string))		null
labels	Resource labels.	map(string)		{}
lien_reason	If non-empty, creates a project lien with this description.	string		""
logging_data_access	Control activation of data access logs. Format is service => { log type => [exempted members]}. The special 'allServices' key denotes configuration for all services.	map(map(list(string)))		{}
logging_exclusions	Logging exclusions for this project in the form {NAME -> FILTER}.	map(string)		{}
logging_sinks	Logging sinks to create for this project.	map(object({…}))		{}
metric_scopes	List of projects that will act as metric scopes for this project.	list(string)		[]
org_policies	Organization policies applied to this project keyed by policy name.	map(object({…}))		{}
org_policies_data_path	Path containing org policies in YAML format.	string		null
oslogin	Enable OS Login.	bool		false
oslogin_admins	List of IAM-style identities that will be granted roles necessary for OS Login administrators.	list(string)		[]
oslogin_users	List of IAM-style identities that will be granted roles necessary for OS Login users.	list(string)		[]
parent	Parent folder or organization in 'folders/folder_id' or 'organizations/org_id' format.	string		null
prefix	Optional prefix used to generate project id and name.	string		null
project_create	Create project. When set to false, uses a data source to reference existing project.	bool		true
service_config	Configure service API activation.	object({…})		{…}
service_encryption_key_ids	Cloud KMS encryption key in {SERVICE => [KEY_URL]} format.	map(list(string))		{}
service_perimeter_bridges	Name of VPC-SC Bridge perimeters to add project into. See comment in the variables file for format.	list(string)		null
service_perimeter_standard	Name of VPC-SC Standard perimeter to add project into. See comment in the variables file for format.	string		null
services	Service APIs to enable.	list(string)		[]
shared_vpc_host_config	Configures this project as a Shared VPC host project (mutually exclusive with shared_vpc_service_project).	object({…})		null
shared_vpc_service_config	Configures this project as a Shared VPC service project (mutually exclusive with shared_vpc_host_config).	object({…})		{…}
skip_delete	Allows the underlying resources to be destroyed without destroying the project itself.	bool		false
tag_bindings	Tag bindings for this project, in key => tag value id format.	map(string)		null

Outputs

name	description	sensitive
custom_roles	Ids of the created custom roles.
id	Project id.
name	Project name.
number	Project number.
project_id	Project id.
service_accounts	Product robot service accounts in project.
services	Service APIs to enabled in the project.
sink_writer_identities	Writer identities created for each sink.