Merge branch 'master' into fast/gke2

2022-08-10 17:18:49 +02:00 · 2022-08-10 17:18:49 +02:00 · ea72c5bc29
parent ac575119c6 c0e17f4732
commit ea72c5bc29
14 changed files with 306 additions and 158 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -41,6 +41,7 @@ All notable changes to this project will be documented in this file.

 ### EXAMPLES

+- [[#771](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/771)] Example of a multi-cluster mesh on GKE configuring managed control pl… ([apichick](https://github.com/apichick)) <!-- 2022-08-08 14:54:03+00:00 -->
 - [[#743](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/743)] Update Readme.md: gcs to bq + cloud armor / glb ([bensadikgoogle](https://github.com/bensadikgoogle)) <!-- 2022-08-01 15:22:04+00:00 -->
 - [[#757](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/757)] Remove key_algorithm from glb/ilb-l7 examples ([ludoo](https://github.com/ludoo)) <!-- 2022-07-25 14:00:14+00:00 -->
 - [[#753](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/753)] Add support for IAM bindings on service accounts to project factory ([ludoo](https://github.com/ludoo)) <!-- 2022-07-21 13:13:40+00:00 -->
@ -52,6 +53,8 @@ All notable changes to this project will be documented in this file.

 ### MODULES

+- [[#773](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/773)] **incompatible change:** Refactor Cloud Run module ([ludoo](https://github.com/ludoo)) <!-- 2022-08-09 12:06:30+00:00 -->
+- [[#754](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/754)] Add support to a public access to cloudsql-instance ([alefmreis](https://github.com/alefmreis)) <!-- 2022-08-09 11:42:42+00:00 -->
 - [[#768](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/768)] Add egress / ingress policy example to VPC SC module ([ludoo](https://github.com/ludoo)) <!-- 2022-08-04 15:00:14+00:00 -->
 - [[#767](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/767)] Allow interpolating SAs in project factory subnet IAM bindings ([ludoo](https://github.com/ludoo)) <!-- 2022-08-04 08:39:28+00:00 -->
 - [[#764](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/pull/764)] Add dependency on shared vpc service project attachment to project module outputs ([apichick](https://github.com/apichick)) <!-- 2022-08-02 16:38:01+00:00 -->
--- a/examples/data-solutions/README.md
+++ b/examples/data-solutions/README.md
@ -36,6 +36,6 @@ This [example](./cloudsql-multiregion/) creates a [Cloud SQL instance](https://c

 ### Data Playground starter with Cloud Vertex AI Notebook and GCS

-<a href="./data-playground/" title="Data Playground starter with Cloud Vertex AI Notebook and GCS"><img src="./data-playground/diagram.png" align="left" width="280px"></a>
-This [example](./data-playground/) creates a [Vertex AI Notebook](https://cloud.google.com/vertex-ai/docs/workbench/introduction) running under a VPC network and a starter GCS bucket to store inputs and outputs of data experiments.
+<a href="./data-playground/" title="Data Playground project with Cloud Vertex AI Notebook, BigQuery and GCS"><img src="./data-playground/diagram.png" align="left" width="280px"></a>
+This [example](./data-playground/) creates a [Vertex AI Notebook](https://cloud.google.com/vertex-ai/docs/workbench/introduction) running on a VPC with a private IP and a dedicated Service Account. A GCS bucket and a BigQuery dataset are created to store inputs and outputs of data experiments.
 <br clear="left">
--- a/examples/data-solutions/data-playground/README.md
+++ b/examples/data-solutions/data-playground/README.md
@ -1,6 +1,6 @@
 # Data Playground

-This example creates a minimum viable template for a data experimentation project with the needed APIs enabled, basic VPC and Firewall set in place, GCS bucket and an AI notebook to get started.
+This example creates a minimum viable architecture for a data experimentation project with the needed APIs enabled, VPC and Firewall set in place, BigQuesy dataset, GCS bucket and an AI notebook to get started.

 This is the high level diagram:

@ -10,34 +10,58 @@ This is the high level diagram:

 This sample creates several distinct groups of resources:

- projects
-    - Service Project configured for GCE instances and GCS buckets
+- project
 - networking
-    - VPC network
-    - One default subnet
+    - VPC network with a default subnet and CloudNat
    - Firewall rules for [SSH access via IAP](https://cloud.google.com/iap/docs/using-tcp-forwarding) and open communication within the VPC
- Vertex AI notebook
-  - One Jupyter lab notebook instance with public access
- GCS
-    - One bucket initial bucket
+- Vertex AI Workbench notebook configured with a private IP and using a dedicated Service Account
+- One GCS bucket
+- One BigQuery dataset

+## Deploy your enviroment
+We assume the identiy running the following steps has the following role:

+- resourcemanager.projectCreator in case a new project will be created.
+- owner on the project in case you use an existing project.
+
+Run Terraform init:
+```
+$ terraform init
+```
+
+Configure the Terraform variable in your terraform.tfvars file. You need to spefify at least the following variables:
+```
+prefix = "prefix"
+project_id      = "data-001"
+```
+
+You can run now:
+```
+$ terraform apply
+```
+
+You can now connect to the Vertex AI notbook to perform your data analysy.
+<!-- BEGIN TFDOC -->

 ## Variables
-| name                      | description                                                                                                                             | type        | required | default      |
-| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -------- | ------------ |
-| project\_id               | Project id, references existing project if \`project\_create\` is null.                                                                 | string      | ✓        |              |
-| location                  | The location where resources will be deployed                                                                                           | string      |          | europe       |
-| region                    | The region where resources will be deployed.                                                                                            | string      |          | europe-west1 |
-| project\_create           | Provide values if project creation is needed, uses existing project if null. Parent format: folders/folder\_id or organizations/org\_id | object({…}) |          | null         |
-| prefix                    | Unique prefix used for resource names. Not used for project if 'project\_create' is null.                                               | string      |          | dp           |
-| service\_encryption\_keys | Cloud KMS to use to encrypt different services. Key location should match service region.                                               | object({…}) |          | null         |
-| vpc\_config               | Parameters to create a simple VPC for the Data Playground                                                                               | object({…}) |          | {...}        |
+
+| name | description | type | required | default |
+|---|---|:---:|:---:|:---:|
+| [prefix](variables.tf#L36) | Unique prefix used for resource names. Not used for project if 'project_create' is null. | <code>string</code> | ✓ |  |
+| [project_id](variables.tf#L22) | Project id, references existing project if `project_create` is null. | <code>string</code> | ✓ |  |
+| [location](variables.tf#L16) | The location where resources will be deployed. | <code>string</code> |  | <code>&#34;EU&#34;</code> |
+| [project_create](variables.tf#L27) | Provide values if project creation is needed, uses existing project if null. Parent format:  folders/folder_id or organizations/org_id | <code title="object&#40;&#123;&#10;  billing_account_id &#61; string&#10;  parent             &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [region](variables.tf#L41) | The region where resources will be deployed. | <code>string</code> |  | <code>&#34;europe-west1&#34;</code> |
+| [vpc_config](variables.tf#L57) | Parameters to create a VPC. | <code title="object&#40;&#123;&#10;  ip_cidr_range &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  ip_cidr_range &#61; &#34;10.0.0.0&#47;20&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |

 ## Outputs
-| Name                                                                                                                                                  | Description             |
-| ----------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
-| bucket  | GCS Bucket URL.         |
-| project | Project id              |
-| vpc                                                                                                                                                   | VPC Network name        |
-| notebook                                                                                                                                              | Vertex AI notebook name |
+
+| name | description | sensitive |
+|---|---|:---:|
+| [bucket](outputs.tf#L15) | GCS Bucket URL. |  |
+| [dataset](outputs.tf#L20) | GCS Bucket URL. |  |
+| [notebook](outputs.tf#L25) | Vertex AI notebook details. |  |
+| [project](outputs.tf#L33) | Project id |  |
+| [vpc](outputs.tf#L38) | VPC Network |  |
+
+<!-- END TFDOC -->
--- a/examples/data-solutions/data-playground/diagram.png
+++ b/examples/data-solutions/data-playground/diagram.png
--- a/examples/data-solutions/data-playground/main.tf
+++ b/examples/data-solutions/data-playground/main.tf
@ -27,25 +27,33 @@ module "project" {
  project_create  = var.project_create != null
  prefix          = var.project_create == null ? null : var.prefix
  services = [
-    "stackdriver.googleapis.com",
-    "compute.googleapis.com",
-    "storage-component.googleapis.com",
-    "storage.googleapis.com",
-    "servicenetworking.googleapis.com",
    "bigquery.googleapis.com",
    "bigquerystorage.googleapis.com",
    "bigqueryreservation.googleapis.com",
+    "composer.googleapis.com",
+    "compute.googleapis.com",
    "dataflow.googleapis.com",
+    "ml.googleapis.com",
    "notebooks.googleapis.com",
-    "composer.googleapis.com"
+    "servicenetworking.googleapis.com",
+    "stackdriver.googleapis.com",
+    "storage.googleapis.com",
+    "storage-component.googleapis.com"
  ]
  policy_boolean = {
    # "constraints/compute.requireOsLogin" = false 
    # Example of applying a project wide policy, mainly useful for Composer
  }
  service_encryption_key_ids = {
+    compute = [try(local.service_encryption_keys.compute, null)]
+    bq      = [try(local.service_encryption_keys.bq, null)]
    storage = [try(local.service_encryption_keys.storage, null)]
  }
+
+  service_config = {
+    disable_on_destroy         = false,
+    disable_dependent_services = false
+  }
 }

 ###############################################################################
@ -55,11 +63,11 @@ module "project" {
 module "vpc" {
  source     = "../../../modules/net-vpc"
  project_id = module.project.project_id
-  name       = var.vpc_config.vpc_name
+  name       = "${var.prefix}-vpc"
  subnets = [
    {
      ip_cidr_range      = var.vpc_config.ip_cidr_range
-      name               = var.vpc_config.subnet_name
+      name               = "${var.prefix}-subnet"
      region             = var.region
      secondary_ip_range = {}
    }
@ -71,27 +79,73 @@ module "vpc-firewall" {
  project_id   = module.project.project_id
  network      = module.vpc.name
  admin_ranges = [var.vpc_config.ip_cidr_range]
+  custom_rules = {
+    #TODO Remove and rely on 'ssh' tag once terraform-provider-google/issues/9273 is fixed
+    ("${var.prefix}-iap") = {
+      description          = "Enable SSH from IAP on Notebooks."
+      direction            = "INGRESS"
+      action               = "allow"
+      sources              = []
+      ranges               = ["35.235.240.0/20"]
+      targets              = ["notebook-instance"]
+      use_service_accounts = false
+      rules                = [{ protocol = "tcp", ports = [22] }]
+      extra_attributes     = {}
+    }
+  }
+}
+
+module "cloudnat" {
+  source         = "../../../modules/net-cloudnat"
+  project_id     = module.project.project_id
+  name           = "${var.prefix}-default"
+  region         = var.region
+  router_network = module.vpc.name
 }

 ###############################################################################
-#                                GCS                                          #
+#                              Storage                                        #
 ###############################################################################

-module "base-gcs-bucket" {
+module "bucket" {
  source         = "../../../modules/gcs"
  project_id     = module.project.project_id
-  prefix         = module.project.project_id
-  name           = "base"
+  prefix         = var.prefix
+  location       = var.location
+  name           = "data"
  encryption_key = try(local.service_encryption_keys.storage, null) # Example assignment of an encryption key
 }

+module "dataset" {
+  source         = "../../../modules/bigquery-dataset"
+  project_id     = module.project.project_id
+  id             = "${var.prefix}_data"
+  encryption_key = try(local.service_encryption_keys.bq, null) # Example assignment of an encryption key
+}
+
 ###############################################################################
 #                         Vertex AI Notebook                                   #
 ###############################################################################
 # TODO: Add encryption_key to Vertex AI notebooks as well
 # TODO: Add shared VPC support
+
+module "service-account-notebook" {
+  source     = "../../../modules/iam-service-account"
+  project_id = module.project.project_id
+  name       = "notebook-sa"
+  iam_project_roles = {
+    (module.project.project_id) = [
+      "roles/bigquery.admin",
+      "roles/bigquery.jobUser",
+      "roles/bigquery.dataEditor",
+      "roles/bigquery.user",
+      "roles/storage.admin",
+    ]
+  }
+}
+
 resource "google_notebooks_instance" "playground" {
-  name         = "data-play-notebook"
+  name         = "${var.prefix}-notebook"
  location     = format("%s-%s", var.region, "b")
  machine_type = "e2-medium"
  project      = module.project.project_id
@ -104,10 +158,17 @@ resource "google_notebooks_instance" "playground" {
  install_gpu_driver = true
  boot_disk_type     = "PD_SSD"
  boot_disk_size_gb  = 110
+  disk_encryption    = try(local.service_encryption_keys.compute != null, false) ? "CMEK" : "GMEK"
+  kms_key            = try(local.service_encryption_keys.compute, null)

-  no_public_ip    = false
+  no_public_ip    = true
  no_proxy_access = false

  network = module.vpc.network.id
-  subnet  = module.vpc.subnets[format("%s/%s", var.region, var.vpc_config.subnet_name)].id
+  subnet  = module.vpc.subnets[format("%s/%s", var.region, "${var.prefix}-subnet")].id
+
+  service_account = module.service-account-notebook.email
+
+  #TODO Uncomment once terraform-provider-google/issues/9273 is fixed
+  # tags = ["ssh"]
 }
--- a/examples/data-solutions/data-playground/outputs.tf
+++ b/examples/data-solutions/data-playground/outputs.tf
@ -14,12 +14,20 @@

 output "bucket" {
  description = "GCS Bucket URL."
-  value       = module.base-gcs-bucket.url
+  value       = module.bucket.url
+}
+
+output "dataset" {
+  description = "GCS Bucket URL."
+  value       = module.dataset.id
 }

 output "notebook" {
-  description = "Vertex AI notebook"
-  value       = resource.google_notebooks_instance.playground.name
+  description = "Vertex AI notebook details."
+  value = {
+    name = resource.google_notebooks_instance.playground.name
+    id   = resource.google_notebooks_instance.playground.id
+  }
 }

 output "project" {
@ -30,4 +38,4 @@ output "project" {
 output "vpc" {
  description = "VPC Network"
  value       = module.vpc.name
-}
+}
--- a/examples/data-solutions/data-playground/variables.tf
+++ b/examples/data-solutions/data-playground/variables.tf
@ -16,7 +16,7 @@
 variable "location" {
  description = "The location where resources will be deployed."
  type        = string
-  default     = "europe"
+  default     = "EU"
 }

 variable "project_id" {
@ -36,7 +36,6 @@ variable "project_create" {
 variable "prefix" {
  description = "Unique prefix used for resource names. Not used for project if 'project_create' is null."
  type        = string
-  default     = "dp"
 }

 variable "region" {
@ -48,21 +47,19 @@ variable "region" {
 variable "service_encryption_keys" { # service encription key
  description = "Cloud KMS to use to encrypt different services. Key location should match service region."
  type = object({
+    bq      = string
+    compute = string
    storage = string
  })
  default = null
 }

 variable "vpc_config" {
-  description = "Parameters to create a simple VPC for the Data Playground"
+  description = "Parameters to create a VPC."
  type = object({
    ip_cidr_range = string
-    subnet_name   = string
-    vpc_name      = string
  })
  default = {
    ip_cidr_range = "10.0.0.0/20"
-    subnet_name   = "default-subnet"
-    vpc_name      = "data-playground-vpc"
  }
-}
+}
--- a/modules/cloud-run/README.md
+++ b/modules/cloud-run/README.md
@ -222,13 +222,13 @@ module "cloud_run" {
 | [prefix](variables.tf#L82) | Optional prefix used for resource names. | <code>string</code> |  | <code>null</code> |
 | [pubsub_triggers](variables.tf#L93) | Eventarc triggers (Pub/Sub). | <code>list&#40;string&#41;</code> |  | <code>null</code> |
 | [region](variables.tf#L99) | Region used for all resources. | <code>string</code> |  | <code>&#34;europe-west1&#34;</code> |
-| [revision_name](variables.tf#L105) | Revision name. | <code>string</code> |  | <code>null</code> |
-| [service_account](variables.tf#L111) | Service account email. Unused if service account is auto-created. | <code>string</code> |  | <code>null</code> |
-| [service_account_create](variables.tf#L117) | Auto-create service account. | <code>bool</code> |  | <code>false</code> |
-| [traffic](variables.tf#L123) | Traffic. | <code>map&#40;number&#41;</code> |  | <code>null</code> |
-| [volumes](variables.tf#L129) | Volumes. | <code title="list&#40;object&#40;&#123;&#10;  name        &#61; string&#10;  secret_name &#61; string&#10;  items &#61; list&#40;object&#40;&#123;&#10;    key  &#61; string&#10;    path &#61; string&#10;  &#125;&#41;&#41;&#10;&#125;&#41;&#41;">list&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>null</code> |
-| [vpc_connector](variables.tf#L142) | VPC connector configuration. Set create to 'true' if a new connecto needs to be created. | <code title="object&#40;&#123;&#10;  create          &#61; bool&#10;  name            &#61; string&#10;  egress_settings &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
-| [vpc_connector_config](variables.tf#L152) | VPC connector network configuration. Must be provided if new VPC connector is being created. | <code title="object&#40;&#123;&#10;  ip_cidr_range &#61; string&#10;  network       &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [revision_annotations](variables.tf#L105) | Configure revision template annotations. | <code title="object&#40;&#123;&#10;  autoscaling &#61; object&#40;&#123;&#10;    max_scale &#61; number&#10;    min_scale &#61; number&#10;  &#125;&#41;&#10;  cloudsql_instances  &#61; list&#40;string&#41;&#10;  vpcaccess_connector &#61; string&#10;  vpcaccess_egress    &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [revision_name](variables.tf#L119) | Revision name. | <code>string</code> |  | <code>null</code> |
+| [service_account](variables.tf#L125) | Service account email. Unused if service account is auto-created. | <code>string</code> |  | <code>null</code> |
+| [service_account_create](variables.tf#L131) | Auto-create service account. | <code>bool</code> |  | <code>false</code> |
+| [traffic](variables.tf#L137) | Traffic. | <code>map&#40;number&#41;</code> |  | <code>null</code> |
+| [volumes](variables.tf#L143) | Volumes. | <code title="list&#40;object&#40;&#123;&#10;  name        &#61; string&#10;  secret_name &#61; string&#10;  items &#61; list&#40;object&#40;&#123;&#10;    key  &#61; string&#10;    path &#61; string&#10;  &#125;&#41;&#41;&#10;&#125;&#41;&#41;">list&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>null</code> |
+| [vpc_connector_create](variables.tf#L156) | Populate this to create a VPC connector. You can then refer to it in the template annotations. | <code title="object&#40;&#123;&#10;  ip_cidr_range &#61; string&#10;  name          &#61; string&#10;  vpc_self_link &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |

 ## Outputs

--- a/modules/cloud-run/main.tf
+++ b/modules/cloud-run/main.tf
@ -15,25 +15,47 @@
 */

 locals {
+  _vpcaccess_annotation = (
+    local.vpc_connector_create
+    ? {
+      "run.googleapis.com/vpc-access-connector" = google_vpc_access_connector.connector.0.id
+    }
+    : (
+      try(var.revision_annotations.vpcaccess_connector, null) == null
+      ? {}
+      : {
+        "run.googleapis.com/vpc-access-connector" = var.revision_annotations.vpcaccess_connector
+      }
+    )
+  )
  annotations = merge(
    var.ingress_settings == null ? {} : {
      "run.googleapis.com/ingress" = var.ingress_settings
    }
  )
-  template_annotations = merge(
-    var.vpc_connector == null ? {} : {
-      "run.googleapis.com/vpc-access-connector" = (
-        try(var.vpc_connector.create, false)
-        ? google_vpc_access_connector.connector.0.id
-        : var.vpc_connector.name
-      )
+  prefix = var.prefix == null ? "" : "${var.prefix}-"
+  revision_annotations = merge(
+    try(var.revision_annotations.autoscaling.max_scale, null) == null ? {} : {
+      "autoscaling.knative.dev/maxScale" = var.revision_annotations.autoscaling.max_scale
+    },
+    try(var.revision_annotations.cloudsql_instances, null) == null ? {} : {
+      "run.googleapis.com/cloudsql-instances" = join(",", coalesce(
+        var.revision_annotations.cloudsql_instances, []
+      ))
+    },
+    local._vpcaccess_annotation,
+    try(var.revision_annotations.autoscaling.max_scale, null) == null ? {} : {
+      "autoscaling.knative.dev/minScale" = var.revision_annotations.autoscaling.min_scale
+    },
+    try(var.revision_annotations.vpcaccess_egress, null) == null ? {} : {
+      "run.googleapis.com/vpc-access-egress" = var.revision_annotations.vpcaccess_egress
    },
-    try(var.vpc_connector.egress_settings, null) == null ? {} : {
-      "run.googleapis.com/vpc-access-egress" = var.vpc_connector.egress_settings
-    }
  )
-  revision_name = try(var.revision_name, null) == null ? null : "${var.name}-${var.revision_name}"
-  prefix        = var.prefix == null ? "" : "${var.prefix}-"
+  revision_name = (
+    try(var.revision_name, null) == null
+    ? null
+    : "${var.name}-${var.revision_name}"
+  )
  service_account_email = (
    var.service_account_create
    ? (
@ -43,15 +65,16 @@ locals {
    )
    : var.service_account
  )
+  vpc_connector_create = var.vpc_connector_create != null
 }

 resource "google_vpc_access_connector" "connector" {
-  count         = try(var.vpc_connector.create, false) ? 1 : 0
+  count         = local.vpc_connector_create ? 1 : 0
  project       = var.project_id
-  name          = var.vpc_connector.name
+  name          = var.vpc_connector_create.name
  region        = var.region
-  ip_cidr_range = var.vpc_connector_config.ip_cidr_range
-  network       = var.vpc_connector_config.network
+  ip_cidr_range = var.vpc_connector_create.ip_cidr_range
+  network       = var.vpc_connector_create.vpc_self_link
 }

 resource "google_cloud_run_service" "service" {
@ -67,14 +90,14 @@ resource "google_cloud_run_service" "service" {
          for i, container in var.containers : i => container
        }
        content {
-          image   = containers.value["image"]
-          command = try(containers.value["options"]["command"], null)
-          args    = try(containers.value["options"]["args"], null)
+          image   = containers.value.image
+          command = try(containers.value.options.command, null)
+          args    = try(containers.value.options.args, null)
          dynamic "env" {
            for_each = (
-              try(containers.value["options"]["env"], null) == null
+              try(containers.value.options.env, null) == null
              ? {}
-              : containers.value["options"]["env"]
+              : containers.value.options.env
            )
            content {
              name  = env.key
@ -83,47 +106,47 @@ resource "google_cloud_run_service" "service" {
          }
          dynamic "env" {
            for_each = (
-              try(containers.value["options"]["env_from"], null) == null
+              try(containers.value.options.env_from, null) == null
              ? {}
-              : containers.value["options"]["env_from"]
+              : containers.value.options.env_from
            )
            content {
              name = env.key
              value_from {
                secret_key_ref {
-                  name = env.value["name"]
-                  key  = env.value["key"]
+                  name = env.value.name
+                  key  = env.value.key
                }
              }
            }
          }
          dynamic "ports" {
            for_each = (
-              containers.value["ports"] == null
+              containers.value.ports == null
              ? {}
              : {
-                for port in containers.value["ports"] :
+                for port in containers.value.ports :
                "${port.name}-${port.container_port}" => port
              }
            )
            content {
-              name           = ports.value["name"]
-              protocol       = ports.value["protocol"]
-              container_port = ports.value["container_port"]
+              name           = ports.value.name
+              protocol       = ports.value.protocol
+              container_port = ports.value.container_port
            }
          }
          dynamic "resources" {
-            for_each = containers.value["resources"] == null ? [] : [""]
+            for_each = containers.value.resources == null ? [] : [""]
            content {
-              limits   = containers.value["resources"]["limits"]
-              requests = containers.value["resources"]["requests"]
+              limits   = containers.value.resources.limits
+              requests = containers.value.resources.requests
            }
          }
          dynamic "volume_mounts" {
            for_each = (
-              containers.value["volume_mounts"] == null
+              containers.value.volume_mounts == null
              ? {}
-              : containers.value["volume_mounts"]
+              : containers.value.volume_mounts
            )
            content {
              name       = volume_mounts.key
@ -136,18 +159,16 @@ resource "google_cloud_run_service" "service" {
      dynamic "volumes" {
        for_each = var.volumes == null ? [] : var.volumes
        content {
-          name = volumes.value["name"]
+          name = volumes.value.name
          secret {
-            secret_name = volumes.value["secret_name"]
+            secret_name = volumes.value.secret_name
            dynamic "items" {
              for_each = (
-                volumes.value["items"] == null
-                ? []
-                : volumes.value["items"]
+                volumes.value.items == null ? [] : volumes.value.items
              )
              content {
-                key  = items.value["key"]
-                path = items.value["path"]
+                key  = items.value.key
+                path = items.value.path
              }
            }
          }
@ -156,7 +177,7 @@ resource "google_cloud_run_service" "service" {
    }
    metadata {
      name        = local.revision_name
-      annotations = local.template_annotations
+      annotations = local.revision_annotations
    }
  }

@ -204,11 +225,11 @@ resource "google_eventarc_trigger" "audit_log_triggers" {
  }
  matching_criteria {
    attribute = "serviceName"
-    value     = each.value["service_name"]
+    value     = each.value.service_name
  }
  matching_criteria {
    attribute = "methodName"
-    value     = each.value["method_name"]
+    value     = each.value.method_name
  }
  destination {
    cloud_run_service {
--- a/modules/cloud-run/variables.tf
+++ b/modules/cloud-run/variables.tf
@ -102,6 +102,20 @@ variable "region" {
  default     = "europe-west1"
 }

+variable "revision_annotations" {
+  description = "Configure revision template annotations."
+  type = object({
+    autoscaling = object({
+      max_scale = number
+      min_scale = number
+    })
+    cloudsql_instances  = list(string)
+    vpcaccess_connector = string
+    vpcaccess_egress    = string
+  })
+  default = null
+}
+
 variable "revision_name" {
  description = "Revision name."
  type        = string
@ -139,21 +153,12 @@ variable "volumes" {
  default = null
 }

-variable "vpc_connector" {
-  description = "VPC connector configuration. Set create to 'true' if a new connecto needs to be created."
-  type = object({
-    create          = bool
-    name            = string
-    egress_settings = string
-  })
-  default = null
-}
-
-variable "vpc_connector_config" {
-  description = "VPC connector network configuration. Must be provided if new VPC connector is being created."
+variable "vpc_connector_create" {
+  description = "Populate this to create a VPC connector. You can then refer to it in the template annotations."
  type = object({
    ip_cidr_range = string
-    network       = string
+    name          = string
+    vpc_self_link = string
  })
  default = null
 }
--- a/tests/examples/data_solutions/data_playground/fixture/main.tf
+++ b/tests/examples/data_solutions/data_playground/fixture/main.tf
@ -17,6 +17,7 @@
 module "test" {
  source     = "../../../../../examples/data-solutions/data-playground/"
  project_id = "sampleproject"
+  prefix     = "tst"
  project_create = {
    billing_account_id = "123456-123456-123456",
    parent             = "folders/467898377"
--- a/tests/examples/data_solutions/data_playground/test_plan.py
+++ b/tests/examples/data_solutions/data_playground/test_plan.py
@ -22,5 +22,5 @@ FIXTURES_DIR = os.path.join(os.path.dirname(__file__), 'fixture')
 def test_resources(e2e_plan_runner):
  "Test that plan works and the numbers of resources is as expected."
  modules, resources = e2e_plan_runner(FIXTURES_DIR)
-  assert len(modules) == 4
-  assert len(resources) == 23
+  assert len(modules) == 7
+  assert len(resources) == 34
--- a/tests/modules/cloud_run/fixture/main.tf
+++ b/tests/modules/cloud_run/fixture/main.tf
@ -12,21 +12,28 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

-variable "vpc_connector" {
-  type    = any
-  default = null
+variable "revision_annotations" {
+  description = "Configure revision template annotations."
+  type        = any
+  default     = null
 }

-variable "vpc_connector_config" {
-  type    = any
-  default = null
+variable "vpc_connector_create" {
+  description = "Populate this to create a VPC connector. You can then refer to it in the template annotations."
+  type        = any
+  default     = null
 }

 module "cloud_run" {
-  source        = "../../../../modules/cloud-run"
-  project_id    = "my-project"
-  name          = "hello"
-  revision_name = "blue"
+  source     = "../../../../modules/cloud-run"
+  project_id = "my-project"
+  name       = "hello"
+  audit_log_triggers = [
+    {
+      "service_name" : "cloudresourcemanager.googleapis.com",
+      "method_name" : "SetIamPolicy"
+    }
+  ]
  containers = [{
    image         = "us-docker.pkg.dev/cloudrun/container/hello"
    options       = null
@ -34,19 +41,14 @@ module "cloud_run" {
    resources     = null
    volume_mounts = null
  }]
-  audit_log_triggers = [
-    {
-      "service_name" : "cloudresourcemanager.googleapis.com",
-      "method_name" : "SetIamPolicy"
-    }
-  ]
+  iam = {
+    "roles/run.invoker" = ["allUsers"]
+  }
  pubsub_triggers = [
    "topic1",
    "topic2"
  ]
-  iam = {
-    "roles/run.invoker" = ["allUsers"]
-  }
-  vpc_connector        = var.vpc_connector
-  vpc_connector_config = var.vpc_connector_config
+  revision_name        = "blue"
+  revision_annotations = var.revision_annotations
+  vpc_connector_create = var.vpc_connector_create
 }
--- a/tests/modules/cloud_run/test_plan.py
+++ b/tests/modules/cloud_run/test_plan.py
@ -57,25 +57,51 @@ def test_pubsub_triggers(resources):
  assert len(pubsub_triggers) == 2


-def test_vpc_connector_none(plan_runner):
-  "Test VPC connector creation."
-  _, resources = plan_runner()
-  assert len(
-      [r for r in resources if r['type'] == 'google_vpc_access_connector']) == 0
+def test_revision_annotations(plan_runner):
+  revision_annotations = '''{
+    autoscaling = null
+    cloudsql_instances = ["a", "b"]
+    vpcaccess_connector = "foo"
+    vpcaccess_egress = "all-traffic"
+  }'''
+  _, resources = plan_runner(revision_annotations=revision_annotations)
+  r = [
+      r['values'] for r in resources if r['type'] == 'google_cloud_run_service'
+  ][0]
+  assert r['template'][0]['metadata'][0]['annotations'] == {
+      'run.googleapis.com/cloudsql-instances': 'a,b',
+      'run.googleapis.com/vpc-access-connector': 'foo',
+      'run.googleapis.com/vpc-access-egress': 'all-traffic'
+  }


-def test_vpc_connector_nocreate(plan_runner):
-  "Test VPC connector creation."
-  _, resources = plan_runner(
-      vpc_connector='{create=false, name="foo", egress_settings=null}')
-  assert len(
-      [r for r in resources if r['type'] == 'google_vpc_access_connector']) == 0
+def test_revision_annotations_autoscaling(plan_runner):
+  revision_annotations = '''{
+    autoscaling = {max_scale = 5, min_scale = 1}
+    cloudsql_instances = null
+    vpcaccess_connector = null
+    vpcaccess_egress = null
+  }'''
+  _, resources = plan_runner(revision_annotations=revision_annotations)
+  r = [
+      r['values'] for r in resources if r['type'] == 'google_cloud_run_service'
+  ][0]
+  assert r['template'][0]['metadata'][0]['annotations'] == {
+      'autoscaling.knative.dev/maxScale': '5',
+      'autoscaling.knative.dev/minScale': '1'
+  }
+
+
+def test_revision_annotations_none(resources):
+  r = [
+      r['values'] for r in resources if r['type'] == 'google_cloud_run_service'
+  ][0]
+  assert r['template'][0]['metadata'][0].get('annotations') is None


 def test_vpc_connector_create(plan_runner):
-  "Test VPC connector creation."
-  _, resources = plan_runner(
-      vpc_connector='{create=true, name="foo", egress_settings=null}',
-      vpc_connector_config='{ip_cidr_range="10.0.0.0/28", network="default"}')
-  assert len(
-      [r for r in resources if r['type'] == 'google_vpc_access_connector']) == 1
+  vpc_connector_create = '''{
+    ip_cidr_range = "10.10.10.0/24", name = "foo", vpc_self_link = "foo-vpc"
+  }'''
+  _, resources = plan_runner(vpc_connector_create=vpc_connector_create)
+  assert any(r['type'] == 'google_vpc_access_connector' for r in resources)