diff --git a/blueprints/serverless/cloud-run-explore/README.md b/blueprints/serverless/cloud-run-explore/README.md new file mode 100644 index 00000000..1002e817 --- /dev/null +++ b/blueprints/serverless/cloud-run-explore/README.md @@ -0,0 +1,214 @@ +# Cloud Run Explore + +## Introduction + +This blueprint contains all the necessary Terraform modules to build and publicly expose a Cloud Run service in a variety of use cases. + +The content of this blueprint corresponds to the chapter '_My serverless "Hello, World! - Exploring Cloud Run_' of the __Serverless Networking Guide__ (to be released soon). This guide is an easy to follow introduction to Cloud Run, where a couple of friendly characters will guide you from the basics to more advanced topics with a very practical approach and in record time! The code here complements this learning and allows you to test the scenarios presented and your knowledge. + +## Architecture + +The following diagram depicts the main components that this blueprint will set up: + +

+ +The following products or features are used to fulfill the different use cases covered in this blueprint (to learn more about them click on the hyperlinks): + +* [Cloud Run](https://cloud.google.com/run/docs/overview/what-is-cloud-run) - Cloud Run is a managed compute platform that lets you run containers directly on top of Google's scalable infrastructure. +* [Cloud Run Ingress Settings](https://cloud.google.com/run/docs/securing/ingress) - feature that restricts network access to your Cloud Run service. At a network level, by default, any resource on the Internet can reach your Cloud Run service on its run.app URL or at a custom domain set up in Cloud Run. You can change this default by specifying a different setting for its ingress. All ingress paths, including the default run.app URL, are subject to your ingress setting. Ingress is set at the service level. The following settings are available: + * __Internal__: Allows requests from VPC networks that are in the same project or VPC Service Controls perimeter as your Cloud Run service. + * __Internal and Cloud Load Balancing__: Allows requests from resources allowed by the more restrictive Internal setting and an External HTTP(S) load balancer. + * __All__ (default): Allows all requests, including requests directly from the Internet to the default run.app URL. +* [Google Cloud Load Balancer](https://cloud.google.com/run/docs/mapping-custom-domains#https-load-balancer) - When an HTTP(S) load balancer is enabled for Cloud Run, you can reach your serverless app through a custom domain mapped to a single dedicated global Anycast IP address that is not shared with other services. +* [Cloud Armor](https://cloud.google.com/armor) - Google Cloud Armor is the web-application firewall (WAF) and DDoS mitigation service that helps users defend their web apps and services at Google scale at the edge of Google’s network. +* [Identity Aware Proxy](https://cloud.google.com/iap/docs/concepts-overview) - IAP lets you establish a central authorization layer for applications accessed by HTTPS, so you can use an application-level access control model instead of relying on network-level firewalls. [External Load Balancing with IAP](https://cloud.google.com/iap/docs/load-balancer-howto) is supported for Cloud Run with Serverless NEGs. +* [Cloud CDN](https://cloud.google.com/cdn) - Configure fast, reliable web and video content delivery with global scale and reach. __Note__: Cloud CDN is not part of this blueprint yet. + +## Prerequisites + +You will need an existing [project](https://cloud.google.com/resource-manager/docs/creating-managing-projects) with [billing enabled](https://cloud.google.com/billing/docs/how-to/modify-project) and a user with the “Project owner” [IAM](https://cloud.google.com/iam) role on that project. __Note__: to grant a user a role, take a look at the [Granting and Revoking Access](https://cloud.google.com/iam/docs/granting-changing-revoking-access#grant-single-role) documentation. + +## Spinning up the architecture + +### General steps + +1. Clone the repo to your local machine or Cloud Shell: +```bash +git clone https://github.com/GoogleCloudPlatform/cloud-foundation-fabric +``` + +2. Change to the directory of the blueprint: +```bash +cd cloud-foundation-fabric/blueprints/serverless/cloud-run-explore +``` +You should see this README and some terraform files. + +3. To deploy a specific use case, you will need to create a file in this directory called `terraform.tfvars` and follow the corresponding instructions to set variables. Sometimes values that are meant to be substituted will be shown inside brackets but you need to omit these brackets. E.g.: +```tfvars +project_id = "[your-project_id]" +``` +may become +```tfvars +project_id = "spiritual-hour-331417" +``` + +Although each use case is somehow built around the previous one they are self-contained so you can deploy any of them at will. + +4. The usual terraform commands will do the work: +```bash +terraform init +terraform plan +terraform apply +``` + +The resource creation will take a few minutes but when it’s complete, you should see an output stating the command completed successfully with a list of the created resources, and some output variables with information to access your service. + +__Congratulations!__ You have successfully deployed the use case you chose based on the variables configuration. + +### Use case 1: Cloud Run service with default URL + +This is the simplest case, the "Hello World" for Cloud Run. A Cloud Run service is deployed with a default URL based in your project, service name and cloud region where it is deployed: + +

+ +In this case the only variable that you need to set in `terraform.tfvars` is the project ID: +```tfvars +project_id = "[your-project-id]" +``` +Alternatively you can pass this value on the command line: +```bash +terraform apply -var project_id="[your-project-id]" +``` + +The default URL is automatically created and shown as a terraform output variable. It will be similar to the one shown in the picture above. Now use your browser to visit it, you should see the following: + +

+ +### Use case 2: Cloud Run service with custom domain + +If you want to use your own custom domain you need a GCLB in front of your Cloud Run app: + +

+ +The following values will need to be set in `terraform.tfvars`, replacing the custom_domain value with your own domain: +```tfvars +project_id = "[your-project-id]" +custom_domain = "cloud-run-explore.example.org" +``` +Since it is an HTTPS connection a Google managed certificate is created, but for it to be provisioned correctly you will need to point to the load balancer IP address with an A DNS record at your registrar: [Use Google-managed SSL certificates | Load Balancing](https://cloud.google.com/load-balancing/docs/ssl-certificates/google-managed-certs#update-dns). The LB IP is shown as a terraform output. + +Be aware that in this case the Cloud Run service can also be reached through the default URL. To limit access only through the custom domain see the next use case. + +### Use case 3: Cloud Run service exposed only via custom domain + +To block access to the default URL, you can configure Ingress Settings so that Internet requests will be accepted only if they come through the Load Balancer: + +

+ +You only need to set one more value in the previous `terraform.tfvars` file: +```tfvars +project_id = "[your-project-id]" +custom_domain = "cloud-run-explore.example.org" +ingress_settings = "internal-and-cloud-load-balancing" +``` + +The default URL is still created but if you try to visit it, you should see a forbidden error: +

+ +### Use case 4: Cloud Run service protected by Cloud Armor + +To use Cloud Armor to protect the Cloud Run service, you need to create a security policy to enforce in the load balancer: +

+ +The code allows to block a list of IPs and a specific URL path. For example, you may want to block access to a login page to external users. To test its behavior, by default all IPs and the path `"/login.html"` are blocked, but you can override any of these settings with your own values: +```tfvars +project_id = "[your-project-id]" +custom_domain = "cloud-run-explore.example.org" +ingress_settings = "internal-and-cloud-load-balancing" +security_policy = { + enabled = true + ip_blacklist = ["79.149.0.0/16"] + path_blocked = "/admin.html" +} +``` + +Note that to avoid users to bypass the Cloud Armor policy you need to block access through the default URL. Ingress settings is configured to do that. + +### Use case 5: Cloud Run service protected by Cloud Armor and Identity-Aware Proxy + +You can enable IAP at the load balancer to control access using identity and context: +

+Use your own email as identity to access the Cloud Run service: + +```tfvars +project_id = "[your-project-id]" +custom_domain = "cloud-run-explore.example.org" +ingress_settings = "internal-and-cloud-load-balancing" +security_policy = { + enabled = true + ip_blacklist = ["79.149.0.0/16"] +} +iap = { + enabled = true + email = "user@example.org" +} +``` +When visiting it you may be redirected to login with Google. You can use an incognito window to test this behavior. + +## Cleaning up your environment + +The easiest way to remove all the deployed resources is to run the following command: +```bash +terraform destroy +``` +The above command will delete the associated resources so there will be no billable charges made afterwards. IAP Brands, though, can only be created once per project and not deleted. Destroying a Terraform-managed IAP Brand will remove it from state but will not delete it from Google Cloud. + + +## Variables + +| name | description | type | required | default | +|---|---|:---:|:---:|:---:| +| [project_id](variables.tf#L55) | Project ID. | string | ✓ | | +| [custom_domain](variables.tf#L17) | Custom domain for the Load Balancer. | string | | null | +| [iap](variables.tf#L23) | Identity-Aware Proxy for Cloud Run in the LB. | object({…}) | | {} | +| [image](variables.tf#L34) | Container image to deploy. | string | | "us-docker.pkg.dev/cloudrun/container/hello" | +| [ingress_settings](variables.tf#L40) | Ingress traffic sources allowed to call the service. | string | | "all" | +| [project_create](variables.tf#L46) | Parameters for the creation of a new project. | object({…}) | | null | +| [region](variables.tf#L60) | Cloud region where resource will be deployed. | string | | "europe-west1" | +| [run_svc_name](variables.tf#L66) | Cloud Run service name. | string | | "hello" | +| [security_policy](variables.tf#L72) | Security policy (Cloud Armor) to enforce in the LB. | object({…}) | | {} | + +## Outputs + +| name | description | sensitive | +|---|---|:---:| +| [custom_domain](outputs.tf#L19) | Custom domain for the Load Balancer. | | +| [default_URL](outputs.tf#L24) | Cloud Run service default URL. | | +| [load_balancer_ip](outputs.tf#L29) | LB IP that forwards to Cloud Run service. | | + + + +## Tests + +```hcl +module "test" { + source = "./fabric/blueprints/serverless/cloud-run-explore" + project_create = { + billing_account_id = "ABCDE-12345-ABCDE" + parent = "organizations/0123456789" + } + project_id = "myproject" + custom_domain = "cloud-run-explore.example.org" + ingress_settings = "internal-and-cloud-load-balancing" + security_policy = { + enabled = true + ip_blacklist = ["79.149.0.0/16"] + } + iap = { + enabled = true + email = "user@example.org" + } +} + +# tftest modules=4 resources=17 +``` \ No newline at end of file diff --git a/blueprints/serverless/cloud-run-explore/images/architecture.png b/blueprints/serverless/cloud-run-explore/images/architecture.png new file mode 100644 index 00000000..46e3a41d Binary files /dev/null and b/blueprints/serverless/cloud-run-explore/images/architecture.png differ diff --git a/blueprints/serverless/cloud-run-explore/images/forbidden.png b/blueprints/serverless/cloud-run-explore/images/forbidden.png new file mode 100644 index 00000000..67d313b8 Binary files /dev/null and b/blueprints/serverless/cloud-run-explore/images/forbidden.png differ diff --git a/blueprints/serverless/cloud-run-explore/images/service-running.png b/blueprints/serverless/cloud-run-explore/images/service-running.png new file mode 100644 index 00000000..2c517a0a Binary files /dev/null and b/blueprints/serverless/cloud-run-explore/images/service-running.png differ diff --git a/blueprints/serverless/cloud-run-explore/images/use-case-1.png b/blueprints/serverless/cloud-run-explore/images/use-case-1.png new file mode 100644 index 00000000..8028c65d Binary files /dev/null and b/blueprints/serverless/cloud-run-explore/images/use-case-1.png differ diff --git a/blueprints/serverless/cloud-run-explore/images/use-case-2.png b/blueprints/serverless/cloud-run-explore/images/use-case-2.png new file mode 100644 index 00000000..a0fafb99 Binary files /dev/null and b/blueprints/serverless/cloud-run-explore/images/use-case-2.png differ diff --git a/blueprints/serverless/cloud-run-explore/images/use-case-3.png b/blueprints/serverless/cloud-run-explore/images/use-case-3.png new file mode 100644 index 00000000..af834a3d Binary files /dev/null and b/blueprints/serverless/cloud-run-explore/images/use-case-3.png differ diff --git a/blueprints/serverless/cloud-run-explore/images/use-case-4.png b/blueprints/serverless/cloud-run-explore/images/use-case-4.png new file mode 100644 index 00000000..113e9206 Binary files /dev/null and b/blueprints/serverless/cloud-run-explore/images/use-case-4.png differ diff --git a/blueprints/serverless/cloud-run-explore/images/use-case-5.png b/blueprints/serverless/cloud-run-explore/images/use-case-5.png new file mode 100644 index 00000000..d4f91878 Binary files /dev/null and b/blueprints/serverless/cloud-run-explore/images/use-case-5.png differ diff --git a/blueprints/serverless/cloud-run-explore/main.tf b/blueprints/serverless/cloud-run-explore/main.tf new file mode 100644 index 00000000..04bb3cc8 --- /dev/null +++ b/blueprints/serverless/cloud-run-explore/main.tf @@ -0,0 +1,187 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +locals { + gclb_create = var.custom_domain == null ? false : true +} + +module "project" { + source = "../../../modules/project" + billing_account = (var.project_create != null + ? var.project_create.billing_account_id + : null + ) + parent = (var.project_create != null + ? var.project_create.parent + : null + ) + name = var.project_id + services = [ + "run.googleapis.com", + "compute.googleapis.com", + "iap.googleapis.com" + ] + project_create = var.project_create != null +} + +# Cloud Run service +module "cloud_run" { + source = "../../../modules/cloud-run" + project_id = module.project.project_id + name = var.run_svc_name + region = var.region + containers = [{ + image = var.image + options = null + ports = null + resources = null + volume_mounts = null + }] + iam = { + "roles/run.invoker" = ["allUsers"] + } + ingress_settings = var.ingress_settings +} + +# Reserved static IP for the Load Balancer +resource "google_compute_global_address" "default" { + count = local.gclb_create ? 1 : 0 + project = module.project.project_id + name = "glb-ip" +} + +# Global L7 HTTPS Load Balancer in front of Cloud Run +module "glb" { + source = "../../../modules/net-glb" + count = local.gclb_create ? 1 : 0 + project_id = module.project.project_id + name = "glb" + address = google_compute_global_address.default[0].address + backend_service_configs = { + default = { + backends = [ + { backend = "neg-0" } + ] + health_checks = [] + port_name = "http" + security_policy = try(google_compute_security_policy.policy[0].name, + null) + iap_config = try({ + oauth2_client_id = google_iap_client.iap_client[0].client_id, + oauth2_client_secret = google_iap_client.iap_client[0].secret + }, null) + } + } + health_check_configs = {} + neg_configs = { + neg-0 = { + cloudrun = { + region = var.region + target_service = { + name = var.run_svc_name + } + } + } + } + protocol = "HTTPS" + ssl_certificates = { + managed_configs = { + default = { + domains = [var.custom_domain] + } + } + } +} + +# Cloud Armor configuration +resource "google_compute_security_policy" "policy" { + count = local.gclb_create && var.security_policy.enabled ? 1 : 0 + name = "cloud-run-policy" + project = module.project.project_id + rule { + action = "deny(403)" + priority = 1000 + match { + versioned_expr = "SRC_IPS_V1" + config { + src_ip_ranges = var.security_policy.ip_blacklist + } + } + description = "Deny access to list of IPs" + } + rule { + action = "deny(403)" + priority = 900 + match { + expr { + expression = "request.path.matches(\"${var.security_policy.path_blocked}\")" + } + } + description = "Deny access to specific URL paths" + } + rule { + action = "allow" + priority = "2147483647" + match { + versioned_expr = "SRC_IPS_V1" + config { + src_ip_ranges = ["*"] + } + } + description = "Default rule" + } +} + +# Identity-Aware Proxy (IAP) or OAuth brand (see OAuth consent screen) +# Note: +# Only "Organization Internal" brands can be created programmatically +# via API. To convert it into an external brand please use the GCP +# Console. +# Brands can only be created once for a Google Cloud project and the +# underlying Google API doesn't support DELETE or PATCH methods. +# Destroying a Terraform-managed Brand will remove it from state but +# will not delete it from Google Cloud. +resource "google_iap_brand" "iap_brand" { + count = local.gclb_create && var.iap.enabled ? 1 : 0 + project = module.project.project_id + # Support email displayed on the OAuth consent screen. The caller must be + # the user with the associated email address, or if a group email is + # specified, the caller can be either a user or a service account which + # is an owner of the specified group in Cloud Identity. + support_email = var.iap.email + application_title = var.iap.app_title +} + +# IAP owned OAuth2 client +# Note: +# Only internal org clients can be created via declarative tools. +# External clients must be manually created via the GCP console. +# Warning: +# All arguments including secret will be stored in the raw state as plain-text. +resource "google_iap_client" "iap_client" { + count = local.gclb_create && var.iap.enabled ? 1 : 0 + display_name = var.iap.oauth2_client_name + brand = google_iap_brand.iap_brand[0].name +} + +# IAM policy for IAP +# For simplicity we use the same email as support_email and authorized member +resource "google_iap_web_iam_member" "iap_iam" { + count = local.gclb_create && var.iap.enabled ? 1 : 0 + project = module.project.project_id + role = "roles/iap.httpsResourceAccessor" + member = "user:${var.iap.email}" +} diff --git a/blueprints/serverless/cloud-run-explore/outputs.tf b/blueprints/serverless/cloud-run-explore/outputs.tf new file mode 100644 index 00000000..2005b13d --- /dev/null +++ b/blueprints/serverless/cloud-run-explore/outputs.tf @@ -0,0 +1,32 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# Custom domain for the Load Balancer. I'd prefer getting the value from the +# SSL certificate but it is not exported as output +output "custom_domain" { + description = "Custom domain for the Load Balancer." + value = local.gclb_create ? var.custom_domain : "none" +} + +output "default_URL" { + description = "Cloud Run service default URL." + value = module.cloud_run.service.status[0].url +} + +output "load_balancer_ip" { + description = "LB IP that forwards to Cloud Run service." + value = local.gclb_create ? module.glb[0].address : "none" +} diff --git a/blueprints/serverless/cloud-run-explore/variables.tf b/blueprints/serverless/cloud-run-explore/variables.tf new file mode 100644 index 00000000..1c3b04a1 --- /dev/null +++ b/blueprints/serverless/cloud-run-explore/variables.tf @@ -0,0 +1,80 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "custom_domain" { + description = "Custom domain for the Load Balancer." + type = string + default = null +} + +variable "iap" { + description = "Identity-Aware Proxy for Cloud Run in the LB." + type = object({ + enabled = optional(bool, false) + app_title = optional(string, "Cloud Run Explore Application") + oauth2_client_name = optional(string, "Test Client") + email = optional(string) + }) + default = {} +} + +variable "image" { + description = "Container image to deploy." + type = string + default = "us-docker.pkg.dev/cloudrun/container/hello" +} + +variable "ingress_settings" { + description = "Ingress traffic sources allowed to call the service." + type = string + default = "all" +} + +variable "project_create" { + description = "Parameters for the creation of a new project." + type = object({ + billing_account_id = string + parent = string + }) + default = null +} + +variable "project_id" { + description = "Project ID." + type = string +} + +variable "region" { + description = "Cloud region where resource will be deployed." + type = string + default = "europe-west1" +} + +variable "run_svc_name" { + description = "Cloud Run service name." + type = string + default = "hello" +} + +variable "security_policy" { + description = "Security policy (Cloud Armor) to enforce in the LB." + type = object({ + enabled = optional(bool, false) + ip_blacklist = optional(list(string), ["*"]) + path_blocked = optional(string, "/login.html") + }) + default = {} +} diff --git a/modules/net-ilb/README.md b/modules/net-ilb/README.md index c284c4c6..48c1d908 100644 --- a/modules/net-ilb/README.md +++ b/modules/net-ilb/README.md @@ -12,6 +12,62 @@ One other issue is a `Provider produced inconsistent final plan` error which is ## Examples +### Reference existing MIGs + +This example shows how to reference existing Managed Infrastructure Groups (MIGs). + +```hcl +module "instance_template" { + source = "./fabric/modules/compute-vm" + project_id = var.project_id + create_template = true + name = "vm-test" + service_account_create = true + zone = "europe-west1-b" + + network_interfaces = [ + { + network = var.vpc.self_link + subnetwork = var.subnet.self_link + } + ] + + tags = [ + "http-server" + ] +} + +module "mig" { + source = "./fabric/modules/compute-mig" + project_id = var.project_id + location = "europe-west1" + name = "mig-test" + target_size = 1 + instance_template = module.instance_template.template.self_link +} + +module "ilb" { + source = "./fabric/modules/net-ilb" + project_id = var.project_id + region = "europe-west1" + name = "ilb-test" + service_label = "ilb-test" + vpc_config = { + network = var.vpc.self_link + subnetwork = var.subnet.self_link + } + backends = [{ + group = module.mig.group_manager.instance_group + }] + health_check_config = { + http = { + port = 80 + } + } +} +# tftest modules=3 resources=6 +``` + ### Externally managed instances This examples shows how to create an ILB by combining externally managed instances (in a custom module or even outside of the current root module) in an unmanaged group. When using internally managed groups, remember to run `terraform apply` each time group instances change. diff --git a/modules/project/README.md b/modules/project/README.md index 3753a5da..fbc4ab29 100644 --- a/modules/project/README.md +++ b/modules/project/README.md @@ -138,6 +138,39 @@ module "project" { # tftest modules=1 resources=2 ``` +### Service identities requiring manual IAM grants + +The module will create service identities at project creation instead of creating of them at the time of first use. This allows granting these service identities roles in other projects, something which is usually necessary in a Shared VPC context. + +You can grant roles to service identities using the following construct: + +```hcl +module "project" { + source = "./fabric/modules/project" + name = "project-example" + iam = { + "roles/apigee.serviceAgent" = [ + "serviceAccount:${module.project.service_accounts.robots.apigee}" + ] + } +} +# tftest modules=1 resources=2 +``` + +This table lists all affected services and roles that you need to grant to service identities + +| service | service identity | role | +|---|---|---| +| apigee.googleapis.com | apigee | roles/apigee.serviceAgent | +| artifactregistry.googleapis.com | artifactregistry | roles/artifactregistry.serviceAgent | +| cloudasset.googleapis.com | cloudasset | roles/cloudasset.serviceAgent | +| cloudbuild.googleapis.com | cloudbuild | roles/cloudbuild.builds.builder | +| gkehub.googleapis.com | fleet | roles/gkehub.serviceAgent | +| multiclusteringress.googleapis.com | multicluster-ingress | roles/multiclusteringress.serviceAgent | +| pubsub.googleapis.com | pubsub | roles/pubsub.serviceAgent | +| sqladmin.googleapis.com | sqladmin | roles/cloudsql.serviceAgent | + + ## Shared VPC The module allows managing Shared VPC status for both hosts and service projects, and includes a simple way of assigning Shared VPC roles to service identities. diff --git a/modules/project/service-accounts.tf b/modules/project/service-accounts.tf index 1979958b..e93978a8 100644 --- a/modules/project/service-accounts.tf +++ b/modules/project/service-accounts.tf @@ -70,16 +70,19 @@ locals { gke-mcs-importer = "${local.project.project_id}.svc.id.goog[gke-mcs/gke-mcs-importer]" } ) + # JIT-ed service accounts are created without default roles granted, these needs to be assigned manually to them + # Roles can be found here: https://cloud.google.com/iam/docs/service-agents + # Remember to update "Service identities requiring manual IAM grants" in README.md when updating this list service_accounts_jit_services = [ - "apigee.googleapis.com", - "artifactregistry.googleapis.com", - "cloudasset.googleapis.com", - "gkehub.googleapis.com", - "multiclusteringress.googleapis.com", - "pubsub.googleapis.com", - "secretmanager.googleapis.com", - "sqladmin.googleapis.com", - "cloudbuild.googleapis.com", + "apigee.googleapis.com", # grant roles/apigee.serviceAgent to apigee + "artifactregistry.googleapis.com", # grant roles/artifactregistry.serviceAgent to artifactregistry + "cloudasset.googleapis.com", # grant roles/cloudasset.serviceAgent to cloudasset + "cloudbuild.googleapis.com", # grant roles/cloudbuild.builds.builder to cloudbuild + "gkehub.googleapis.com", # grant roles/gkehub.serviceAgent to fleet + "multiclusteringress.googleapis.com", # grant roles/multiclusteringress.serviceAgent to multicluster-ingress + "pubsub.googleapis.com", # grant roles/pubsub.serviceAgent to pubsub + "secretmanager.googleapis.com", # no grants needed + "sqladmin.googleapis.com", # grant roles/cloudsql.serviceAgent to sqladmin (TODO: verify) ] service_accounts_cmek_service_keys = distinct(flatten([ for s in keys(var.service_encryption_key_ids) : [