Regreatablly member name will be known after apply, hence changes in the tests |
||
---|---|---|
.. | ||
README.md | ||
iam.tf | ||
logging.tf | ||
main.tf | ||
organization-policies.tf | ||
outputs.tf | ||
service-accounts.tf | ||
shared-vpc.tf | ||
tags.tf | ||
variables.tf | ||
versions.tf | ||
vpc-sc.tf |
README.md
Project Module
This module implements the creation and management of one GCP project including IAM, organization policies, Shared VPC host or service attachment, service API activation, and tag attachment. It also offers a convenient way to refer to managed service identities (aka robot service accounts) for APIs.
Basic Project Creation
module "project" {
source = "./fabric/modules/project"
billing_account = "123456-123456-123456"
name = "myproject"
parent = "folders/1234567890"
prefix = "foo"
services = [
"container.googleapis.com",
"stackdriver.googleapis.com"
]
}
# tftest modules=1 resources=3 inventory=basic.yaml
IAM Examples
IAM is managed via several variables that implement different levels of control:
group_iam
andiam
configure authoritative bindings that manage individual roles exclusively, mapping to thegoogle_project_iam_binding
resourceiam_additive
andiam_additive_members
configure additive bindings that only manage individual role/member pairs, mapping to thegoogle_project_iam_member
resource
Be mindful about service identity roles when using authoritative IAM, as you might inadvertently remove a role from a service identity or default service account. For example, using roles/editor
with iam
or group_iam
will remove the default permissions for the Cloud Services identity. A simple workaround for these scenarios is described below.
Authoritative IAM
The iam
variable is based on role keys and is typically used for service accounts, or where member values can be dynamic and would create potential problems in the underlying for_each
cycle.
locals {
gke_service_account = "my_gke_service_account"
}
module "project" {
source = "./fabric/modules/project"
billing_account = "123456-123456-123456"
name = "project-example"
parent = "folders/1234567890"
prefix = "foo"
services = [
"container.googleapis.com",
"stackdriver.googleapis.com"
]
iam = {
"roles/container.hostServiceAgentUser" = [
"serviceAccount:${local.gke_service_account}"
]
}
}
# tftest modules=1 resources=4 inventory=iam-authoritative.yaml
The group_iam
variable uses group email addresses as keys and is a convenient way to assign roles to humans following Google's best practices. The end result is readable code that also serves as documentation.
module "project" {
source = "./fabric/modules/project"
billing_account = "123456-123456-123456"
name = "project-example"
parent = "folders/1234567890"
prefix = "foo"
group_iam = {
"gcp-security-admins@example.com" = [
"roles/cloudasset.owner",
"roles/cloudsupport.techSupportEditor",
"roles/iam.securityReviewer",
"roles/logging.admin",
]
}
}
# tftest modules=1 resources=5 inventory=iam-group.yaml
Additive IAM
Additive IAM is typically used where bindings for specific roles are controlled by different modules or in different Terraform stages. One example is when the project is created by one team but a different team manages service account creation for the project, and some of the project-level roles overlap in the two configurations.
module "project" {
source = "./fabric/modules/project"
name = "project-example"
iam_additive = {
"roles/viewer" = [
"group:one@example.org",
"group:two@xample.org"
],
"roles/storage.objectAdmin" = [
"group:two@example.org"
],
"roles/owner" = [
"group:three@example.org"
],
}
}
# tftest modules=1 resources=5 inventory=iam-additive.yaml
Additive IAM by members
module "project" {
source = "./fabric/modules/project"
name = "project-example"
iam_additive_members = {
"user:one@example.org" = ["roles/owner"]
"user:two@example.org" = ["roles/owner", "roles/editor"]
}
}
# tftest modules=1 resources=4 inventory=iam-additive-members.yaml
Service Identities and authoritative IAM
As mentioned above, there are cases where authoritative management of specific IAM roles results in removal of default bindings from service identities. One example is outlined below, with a simple workaround leveraging the service_accounts
output to identify the service identity. A full list of service identities and their roles can be found here.
module "project" {
source = "./fabric/modules/project"
name = "project-example"
group_iam = {
"foo@example.com" = [
"roles/editor"
]
}
iam = {
"roles/editor" = [
"serviceAccount:${module.project.service_accounts.cloud_services}"
]
}
}
# tftest modules=1 resources=2
Using shortcodes for Service Identities in additive IAM
Most Service Identities contains project number in their e-mail address and this prevents additive IAM to work, as these values are not known at moment of execution of terraform plan
(its not an issue for authoritative IAM). To refer current project Service Identities you may use shortcodes for Service Identities similarly as for service_identity_iam
when configuring Shared VPC.
module "project" {
source = "./fabric/modules/project"
name = "project-example"
services = [
"run.googleapis.com",
"container.googleapis.com",
]
iam_additive = {
"roles/editor" = ["cloudservices"]
"roles/vpcaccess.user" = ["cloudrun"]
"roles/container.hostServiceAgentUser" = ["container-engine"]
}
}
# tftest modules=1 resources=6
Service identities requiring manual IAM grants
The module will create service identities at project creation instead of creating of them at the time of first use. This allows granting these service identities roles in other projects, something which is usually necessary in a Shared VPC context.
You can grant roles to service identities using the following construct:
module "project" {
source = "./fabric/modules/project"
name = "project-example"
iam = {
"roles/apigee.serviceAgent" = [
"serviceAccount:${module.project.service_accounts.robots.apigee}"
]
}
}
# tftest modules=1 resources=2
This table lists all affected services and roles that you need to grant to service identities
service | service identity | role |
---|---|---|
apigee.googleapis.com | apigee | roles/apigee.serviceAgent |
artifactregistry.googleapis.com | artifactregistry | roles/artifactregistry.serviceAgent |
cloudasset.googleapis.com | cloudasset | roles/cloudasset.serviceAgent |
cloudbuild.googleapis.com | cloudbuild | roles/cloudbuild.builds.builder |
gkehub.googleapis.com | fleet | roles/gkehub.serviceAgent |
multiclusteringress.googleapis.com | multicluster-ingress | roles/multiclusteringress.serviceAgent |
pubsub.googleapis.com | pubsub | roles/pubsub.serviceAgent |
sqladmin.googleapis.com | sqladmin | roles/cloudsql.serviceAgent |
Shared VPC
The module allows managing Shared VPC status for both hosts and service projects, and includes a simple way of assigning Shared VPC roles to service identities.
You can enable Shared VPC Host at the project level and manage project service association independently.
module "host-project" {
source = "./fabric/modules/project"
name = "my-host-project"
shared_vpc_host_config = {
enabled = true
}
}
module "service-project" {
source = "./fabric/modules/project"
name = "my-service-project"
shared_vpc_service_config = {
attach = true
host_project = module.host-project.project_id
service_identity_iam = {
"roles/compute.networkUser" = [
"cloudservices", "container-engine"
]
"roles/vpcaccess.user" = [
"cloudrun"
]
"roles/container.hostServiceAgentUser" = [
"container-engine"
]
}
}
}
# tftest modules=2 resources=8 inventory=shared-vpc.yaml
Organization policies
To manage organization policies, the orgpolicy.googleapis.com
service should be enabled in the quota project.
module "project" {
source = "./fabric/modules/project"
billing_account = "123456-123456-123456"
name = "project-example"
parent = "folders/1234567890"
prefix = "foo"
org_policies = {
"compute.disableGuestAttributesAccess" = {
enforce = true
}
"constraints/compute.skipDefaultNetworkCreation" = {
enforce = true
}
"iam.disableServiceAccountKeyCreation" = {
enforce = true
}
"iam.disableServiceAccountKeyUpload" = {
enforce = false
rules = [
{
condition = {
expression = "resource.matchTagId(\"tagKeys/1234\", \"tagValues/1234\")"
title = "condition"
description = "test condition"
location = "somewhere"
}
enforce = true
}
]
}
"constraints/iam.allowedPolicyMemberDomains" = {
allow = {
values = ["C0xxxxxxx", "C0yyyyyyy"]
}
}
"constraints/compute.trustedImageProjects" = {
allow = {
values = ["projects/my-project"]
}
}
"constraints/compute.vmExternalIpAccess" = {
deny = { all = true }
}
}
}
# tftest modules=1 resources=8 inventory=org-policies.yaml
Organization policy factory
Organization policies can be loaded from a directory containing YAML files where each file defines one or more constraints. The structure of the YAML files is exactly the same as the org_policies
variable.
Note that contraints defined via org_policies
take precedence over those in org_policies_data_path
. In other words, if you specify the same contraint in a YAML file and in the org_policies
variable, the latter will take priority.
The example below deploys a few organization policies split between two YAML files.
module "project" {
source = "./fabric/modules/project"
billing_account = "123456-123456-123456"
name = "project-example"
parent = "folders/1234567890"
prefix = "foo"
org_policies_data_path = "configs/org-policies/"
}
# tftest modules=1 resources=8 files=boolean,list inventory=org-policies.yaml
# tftest-file id=boolean path=configs/org-policies/boolean.yaml
compute.disableGuestAttributesAccess:
enforce: true
constraints/compute.skipDefaultNetworkCreation:
enforce: true
iam.disableServiceAccountKeyCreation:
enforce: true
iam.disableServiceAccountKeyUpload:
enforce: false
rules:
- condition:
description: test condition
expression: resource.matchTagId("tagKeys/1234", "tagValues/1234")
location: somewhere
title: condition
enforce: true
# tftest-file id=list path=configs/org-policies/list.yaml
constraints/compute.trustedImageProjects:
allow:
values:
- projects/my-project
constraints/compute.vmExternalIpAccess:
deny:
all: true
constraints/iam.allowedPolicyMemberDomains:
allow:
values:
- C0xxxxxxx
- C0yyyyyyy
Logging Sinks
module "gcs" {
source = "./fabric/modules/gcs"
project_id = var.project_id
name = "gcs_sink"
force_destroy = true
}
module "dataset" {
source = "./fabric/modules/bigquery-dataset"
project_id = var.project_id
id = "bq_sink"
}
module "pubsub" {
source = "./fabric/modules/pubsub"
project_id = var.project_id
name = "pubsub_sink"
}
module "bucket" {
source = "./fabric/modules/logging-bucket"
parent_type = "project"
parent = "my-project"
id = "bucket"
}
module "project-host" {
source = "./fabric/modules/project"
name = "my-project"
billing_account = "123456-123456-123456"
parent = "folders/1234567890"
logging_sinks = {
warnings = {
destination = module.gcs.id
filter = "severity=WARNING"
type = "storage"
}
info = {
destination = module.dataset.id
filter = "severity=INFO"
type = "bigquery"
}
notice = {
destination = module.pubsub.id
filter = "severity=NOTICE"
type = "pubsub"
}
debug = {
destination = module.bucket.id
filter = "severity=DEBUG"
exclusions = {
no-compute = "logName:compute"
}
type = "logging"
}
}
logging_exclusions = {
no-gce-instances = "resource.type=gce_instance"
}
}
# tftest modules=5 resources=14 inventory=logging.yaml
Cloud KMS encryption keys
The module offers a simple, centralized way to assign roles/cloudkms.cryptoKeyEncrypterDecrypter
to service identities.
module "project" {
source = "./fabric/modules/project"
name = "my-project"
prefix = "foo"
services = [
"compute.googleapis.com",
"storage.googleapis.com"
]
service_encryption_key_ids = {
compute = [
"projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce",
"projects/kms-central-prj/locations/europe-west4/keyRings/my-keyring/cryptoKeys/europe4-gce"
]
storage = [
"projects/kms-central-prj/locations/europe/keyRings/my-keyring/cryptoKeys/europe-gcs"
]
}
}
# tftest modules=1 resources=7
Tags
Refer to the Creating and managing tags documentation for details on usage.
module "org" {
source = "./fabric/modules/organization"
organization_id = var.organization_id
tags = {
environment = {
description = "Environment specification."
iam = null
values = {
dev = null
prod = null
}
}
}
}
module "project" {
source = "./fabric/modules/project"
name = "test-project"
tag_bindings = {
env-prod = module.org.tag_values["environment/prod"].id
foo = "tagValues/12345678"
}
}
# tftest modules=2 resources=6
Outputs
Most of this module's outputs depend on its resources, to allow Terraform to compute all dependencies required for the project to be correctly configured. This allows you to reference outputs like project_id
in other modules or resources without having to worry about setting depends_on
blocks manually.
One non-obvious output is service_accounts
, which offers a simple way to discover service identities and default service accounts, and guarantees that service identities that require an API call to trigger creation (like GCS or BigQuery) exist before use.
module "project" {
source = "./fabric/modules/project"
name = "project-example"
services = [
"compute.googleapis.com"
]
}
output "compute_robot" {
value = module.project.service_accounts.robots.compute
}
# tftest modules=1 resources=2 inventory:outputs.yaml
Files
name | description | resources |
---|---|---|
iam.tf | Generic and OSLogin-specific IAM bindings and roles. | google_project_iam_binding · google_project_iam_custom_role · google_project_iam_member |
logging.tf | Log sinks and supporting resources. | google_bigquery_dataset_iam_member · google_logging_project_exclusion · google_logging_project_sink · google_project_iam_member · google_pubsub_topic_iam_member · google_storage_bucket_iam_member |
main.tf | Module-level locals and resources. | google_compute_project_metadata_item · google_essential_contacts_contact · google_monitoring_monitored_project · google_project · google_project_service · google_resource_manager_lien |
organization-policies.tf | Project-level organization policies. | google_org_policy_policy |
outputs.tf | Module outputs. | |
service-accounts.tf | Service identities and supporting resources. | google_kms_crypto_key_iam_member · google_project_default_service_accounts · google_project_iam_member · google_project_service_identity |
shared-vpc.tf | Shared VPC project-level configuration. | google_compute_shared_vpc_host_project · google_compute_shared_vpc_service_project · google_project_iam_member |
tags.tf | None | google_tags_tag_binding |
variables.tf | Module variables. | |
versions.tf | Version pins. | |
vpc-sc.tf | VPC-SC project-level perimeter configuration. | google_access_context_manager_service_perimeter_resource |
Variables
name | description | type | required | default |
---|---|---|---|---|
name | Project name and id suffix. | string |
✓ | |
auto_create_network | Whether to create the default network for the project. | bool |
false |
|
billing_account | Billing account id. | string |
null |
|
contacts | List of essential contacts for this resource. Must be in the form EMAIL -> [NOTIFICATION_TYPES]. Valid notification types are ALL, SUSPENSION, SECURITY, TECHNICAL, BILLING, LEGAL, PRODUCT_UPDATES. | map(list(string)) |
{} |
|
custom_roles | Map of role name => list of permissions to create in this project. | map(list(string)) |
{} |
|
default_service_account | Project default service account setting: can be one of delete , deprivilege , disable , or keep . |
string |
"keep" |
|
descriptive_name | Name of the project name. Used for project name instead of name variable. |
string |
null |
|
group_iam | Authoritative IAM binding for organization groups, in {GROUP_EMAIL => [ROLES]} format. Group emails need to be static. Can be used in combination with the iam variable. |
map(list(string)) |
{} |
|
iam | IAM bindings in {ROLE => [MEMBERS]} format. | map(list(string)) |
{} |
|
iam_additive | IAM additive bindings in {ROLE => [MEMBERS]} format. | map(list(string)) |
{} |
|
iam_additive_members | IAM additive bindings in {MEMBERS => [ROLE]} format. This might break if members are dynamic values. | map(list(string)) |
{} |
|
labels | Resource labels. | map(string) |
{} |
|
lien_reason | If non-empty, creates a project lien with this description. | string |
"" |
|
logging_exclusions | Logging exclusions for this project in the form {NAME -> FILTER}. | map(string) |
{} |
|
logging_sinks | Logging sinks to create for this project. | map(object({…})) |
{} |
|
metric_scopes | List of projects that will act as metric scopes for this project. | list(string) |
[] |
|
org_policies | Organization policies applied to this project keyed by policy name. | map(object({…})) |
{} |
|
org_policies_data_path | Path containing org policies in YAML format. | string |
null |
|
oslogin | Enable OS Login. | bool |
false |
|
oslogin_admins | List of IAM-style identities that will be granted roles necessary for OS Login administrators. | list(string) |
[] |
|
oslogin_users | List of IAM-style identities that will be granted roles necessary for OS Login users. | list(string) |
[] |
|
parent | Parent folder or organization in 'folders/folder_id' or 'organizations/org_id' format. | string |
null |
|
prefix | Optional prefix used to generate project id and name. | string |
null |
|
project_create | Create project. When set to false, uses a data source to reference existing project. | bool |
true |
|
service_config | Configure service API activation. | object({…}) |
{…} |
|
service_encryption_key_ids | Cloud KMS encryption key in {SERVICE => [KEY_URL]} format. | map(list(string)) |
{} |
|
service_perimeter_bridges | Name of VPC-SC Bridge perimeters to add project into. See comment in the variables file for format. | list(string) |
null |
|
service_perimeter_standard | Name of VPC-SC Standard perimeter to add project into. See comment in the variables file for format. | string |
null |
|
services | Service APIs to enable. | list(string) |
[] |
|
shared_vpc_host_config | Configures this project as a Shared VPC host project (mutually exclusive with shared_vpc_service_project). | object({…}) |
null |
|
shared_vpc_service_config | Configures this project as a Shared VPC service project (mutually exclusive with shared_vpc_host_config). | object({…}) |
null |
|
skip_delete | Allows the underlying resources to be destroyed without destroying the project itself. | bool |
false |
|
tag_bindings | Tag bindings for this project, in key => tag value id format. | map(string) |
null |
Outputs
name | description | sensitive |
---|---|---|
custom_roles | Ids of the created custom roles. | |
name | Project name. | |
number | Project number. | |
project_id | Project id. | |
service_accounts | Product robot service accounts in project. | |
sink_writer_identities | Writer identities created for each sink. |