Apigee hybrid on GKE

.gitignore

@@ -45,3 +45,7 @@ blueprints/apigee/bigquery-analytics/deploy-apiproxy.sh
 blueprints/apigee/network-patterns/nb-glb-psc-neg-sb-psc-ilbl7-hybrid-neg/bundle/apiproxy/targets/default.xml
 blueprints/apigee/network-patterns/nb-glb-psc-neg-sb-psc-ilbl7-hybrid-neg/bundle.zip
 blueprints/apigee/network-patterns/nb-glb-psc-neg-sb-psc-ilbl7-hybrid-neg/deploy-apiproxy.sh
+blueprints/apigee/hybrid-gke/apiproxy.zip
+blueprints/apigee/hybrid-gke/deploy-apiproxy.sh
+blueprints/apigee/hybrid-gke/ansible/gssh.sh
+blueprints/apigee/hybrid-gke/ansible/vars/vars.yaml

blueprints/apigee/README.md

@@ -1,7 +1,8 @@
-# Apigee Examples
+# Apigee Blueprints
 
 This repository contains the following Apigee examples:
 
+* [Apigee Hybrid on GKE](./hybrid-gke/README.md)
 * [Apigee BigQuery analytics](./bigquery-analytics/README.md)
 * Apigee network patterns
     * [Apigee X - Northbound GLB with PSC Neg, Southbouth PSC with ILB (L7) and Hybrid NEG

blueprints/apigee/hybrid-gke/README.md

@@ -0,0 +1,59 @@
+# Apigee Hybrid on GKE
+
+This example installs Apigee hybrid in a non-prod environment on a GKE private cluster using Terraform and Ansible.
+The Terraform configuration deploys all the required infrastructure including a management VM used to run an ansible playbook to the actual Apigee Hybrid setup.
+
+The diagram below depicts the architecture.
+
+![Diagram](./diagram.png)
+
+## Running the blueprint
+
+1. Clone this repository or [open it in cloud shell](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2Fterraform-google-modules%2Fcloud-foundation-fabric&cloudshell_print=cloud-shell-readme.txt&cloudshell_working_dir=blueprints%2Fapigee%2Fhybrid), then go through the following steps to create resources:
+
+2. Copy the file [terraform.tfvars.sample](./terraform.tfvars.sample) to a file called ```terraform.tfvars``` and update the values if required.
+
+3. Initialize the terraform configuration
+
+    ```
+    terraform init
+    ```
+
+4. Apply the terraform configuration
+
+    ```
+    terraform apply
+    ```
+
+## Testing the blueprint
+
+2. Deploy an api proxy
+
+    ```
+    ./deploy-apiproxy.sh
+    ```
+
+3. In the console check the IP address that has been allocated to the Apigee ingress gateway and send some traffic to the deployed API proxy.
+
+    ```
+    curl -k -v -H "Host:HOSTNAME" \
+    --resolve HOSTNAME:443:IP_ADDRESS \
+    https://HOSTNAME/httpbin/headers
+    ```
+<!-- BEGIN TFDOC -->
+
+## Variables
+
+| name | description | type | required | default |
+|---|---|:---:|:---:|:---:|
+| [hostname](variables.tf#L43) | Host name. | <code>string</code> | ✓ |  |
+| [project_id](variables.tf#L79) | Project ID. | <code>string</code> | ✓ |  |
+| [cluster_machine_type](variables.tf#L17) | Cluster nachine type. | <code>string</code> |  | <code>&#34;e2-standard-4&#34;</code> |
+| [cluster_network_config](variables.tf#L23) | Cluster network configuration. | <code title="object&#40;&#123;&#10;  nodes_cidr_block              &#61; string&#10;  pods_cidr_block               &#61; string&#10;  services_cidr_block           &#61; string&#10;  master_authorized_cidr_blocks &#61; map&#40;string&#41;&#10;  master_cidr_block             &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  nodes_cidr_block    &#61; &#34;10.0.1.0&#47;24&#34;&#10;  pods_cidr_block     &#61; &#34;172.16.0.0&#47;20&#34;&#10;  services_cidr_block &#61; &#34;192.168.0.0&#47;24&#34;&#10;  master_authorized_cidr_blocks &#61; &#123;&#10;    internal &#61; &#34;10.0.0.0&#47;8&#34;&#10;  &#125;&#10;  master_cidr_block &#61; &#34;10.0.0.0&#47;28&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
+| [mgmt_server_config](variables.tf#L48) | Mgmt server configuration. | <code title="object&#40;&#123;&#10;  disk_size     &#61; number&#10;  disk_type     &#61; string&#10;  image         &#61; string&#10;  instance_type &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code title="&#123;&#10;  disk_size     &#61; 50&#10;  disk_type     &#61; &#34;pd-ssd&#34;&#10;  image         &#61; &#34;projects&#47;ubuntu-os-cloud&#47;global&#47;images&#47;family&#47;ubuntu-2204-lts&#34;&#10;  instance_type &#61; &#34;n1-standard-2&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
+| [mgmt_subnet_cidr_block](variables.tf#L64) | Management subnet CIDR block. | <code>string</code> |  | <code>&#34;10.0.2.0&#47;28&#34;</code> |
+| [project_create](variables.tf#L70) | Parameters for the creation of the new project. | <code title="object&#40;&#123;&#10;  billing_account_id &#61; string&#10;  parent             &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>null</code> |
+| [region](variables.tf#L84) | Region. | <code>string</code> |  | <code>&#34;europe-west1&#34;</code> |
+| [zone](variables.tf#L90) | Zone. | <code>string</code> |  | <code>&#34;europe-west1-c&#34;</code> |
+
+<!-- END TFDOC -->

blueprints/apigee/hybrid-gke/ansible.tf

@@ -0,0 +1,39 @@
+/**
+ * Copyright 2023 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+# tfdoc:file:description Ansible generated files.
+
+resource "local_file" "vars_file" {
+  content = yamlencode({
+    cluster    = module.cluster.name
+    region     = var.region
+    project_id = module.project.project_id
+    envgroup   = local.envgroup
+    env        = local.environment
+    hostname   = var.hostname
+  })
+  filename        = "${path.module}/ansible/vars/vars.yaml"
+  file_permission = "0666"
+}
+
+resource "local_file" "gssh_file" {
+  content = templatefile("${path.module}/templates/gssh.sh.tpl", {
+    project_id = module.project.project_id
+    zone       = var.zone
+  })
+  filename        = "${path.module}/ansible/gssh.sh"
+  file_permission = "0777"
+}

blueprints/apigee/hybrid-gke/ansible/ansible.cfg

@@ -0,0 +1,8 @@
+[defaults]
+inventory = inventory/hosts.ini
+timeout = 900
+
+[ssh_connection]
+pipelining = True
+ssh_executable = ./gssh.sh
+transfer_method = piped

blueprints/apigee/hybrid-gke/ansible/inventory/hosts.ini

@@ -0,0 +1 @@
+mgmt

blueprints/apigee/hybrid-gke/ansible/playbook.yaml

@@ -0,0 +1,26 @@
+# Copyright 2022 Google LLC
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+- hosts: mgmt
+  gather_facts: "no"
+  vars_files:
+    - vars/vars.yaml
+  environment:
+    USE_GKE_GCLOUD_AUTH_PLUGIN: True
+  roles:
+    - role: prerequisites
+      become: yes
+      become_method: sudo
+    - role: apigee-hybrid
+    

blueprints/apigee/hybrid-gke/ansible/roles/apigee-hybrid/tasks/main.yaml

@@ -0,0 +1,143 @@
+# Copyright 2023 Google LLC
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+- name: Get cluster credentials
+  shell: >
+    gcloud container clusters get-credentials {{ cluster }} \
+    --region {{ region }} \
+    --project {{ project_id }} \
+    --internal-ip
+
+- name: Install cert-manager
+  shell: >
+    kubectl apply \
+    --validate=false \
+    -f https://github.com/jetstack/cert-manager/releases/download/v1.7.2/cert-manager.yaml
+
+- name: Wait until pods are ready in cert-manager namespace
+  shell: >
+    kubectl wait --for=condition=ready pods \
+    -l app.kubernetes.io/instance=cert-manager \
+    -n cert-manager \ 
+    --timeout=90s
+
+- name: Fetch apigeectl version
+  uri:
+    url: https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/current-version.txt?ignoreCache=1
+    return_content: yes
+  register: version
+
+- name: Download apigeectl bundle
+  uri:
+    url: https://storage.googleapis.com/apigee-release/hybrid/apigee-hybrid-setup/{{ version.content }}/apigeectl_linux_64.tar.gz
+    dest: "~/apigeectl.tar.gz"
+    status_code: [200, 304]
+
+- name: Extract apigeectl bundle
+  unarchive:
+    src: "~/apigeectl.tar.gz"
+    dest: "~"
+    remote_src: yes    
+
+- name: Move apigeectl folder
+  shell: >
+    mv ~/apigeectl_* ~/apigeectl
+
+- name: Create hybrid-files
+  file:
+    path: "~/hybrid-files/{{ item }}"
+    state: directory
+  with_items:
+    - overrides
+    - certs
+
+- name: Create a symbolic links
+  file:
+    src: ~/apigeectl/{{ item }}
+    dest: "~/hybrid-files/{{ item }}"
+    state: link    
+  with_items:
+    - tools
+    - config
+    - templates
+    - plugins    
+
+- name: Create service accounts
+  shell: >
+    ~/hybrid-files/tools/create-service-account -i {{ project_id }} -e non-prod -d ~/hybrid-files/service-accounts
+
+- name: Create certificates
+  shell: >
+    openssl req  \ 
+    -nodes \
+    -new \
+    -x509 \
+    -keyout ~/hybrid-files/certs/{{ envgroup }}.key \
+    -out ~/hybrid-files/certs/{{ envgroup }}.cert -subj '/CN='{{ hostname }}'' -days 3650
+
+- name: Create overrides.yaml
+  template:
+    src: templates/overrides.yaml.j2
+    dest: ~/hybrid-files/overrides/overrides.yaml
+
+- name: Enable syncronizer access
+  shell: >
+    curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \
+    -H "Content-Type:application/json" \
+    "https://apigee.googleapis.com/v1/organizations/{{ project_id }}:setSyncAuthorization" \
+    -d '{"identities":["'"serviceAccount:apigee-non-prod@{{ project_id }}.iam.gserviceaccount.com"'"]}'
+
+- name: Dry-run (init)
+  shell: >
+    ~/apigeectl/apigeectl init -f overrides/overrides.yaml --dry-run=client  
+  args:
+    chdir: ~/hybrid-files
+
+- name: Install the Apigee deployment services Apigee Deployment Controller and Apigee Admission Webhook.
+  shell: >
+    ~/apigeectl/apigeectl init -f overrides/overrides.yaml 
+  args:
+    chdir: ~/hybrid-files    
+
+- name: Wait until pods are ready in apigee-system namespace
+  shell: >
+    kubectl wait --for=condition=ready pods \
+    -l app=apigee-controller \
+    -n apigee-system \ 
+    --timeout=300s    
+
+- name: Wait until pods are ready in apigee namespace
+  shell: >
+    kubectl wait --for=condition=ready pods \
+    -l app=apigee-ingressgateway-manager \
+    -n apigee \ 
+    --timeout=300s    
+
+- name: Dry-run (apply)
+  shell: >
+    ~/apigeectl/apigeectl apply -f overrides/overrides.yaml --dry-run=client  
+  args:
+    chdir: ~/hybrid-files
+
+- name: Install the Apigee runtime components
+  shell: >
+    ~/apigeectl/apigeectl apply -f overrides/overrides.yaml 
+  args:
+    chdir: ~/hybrid-files        
+
+- name: Check status of the deployment
+  shell: >
+    while [ -n "$(kubectl get pods -n apigee | tail -n +2 | grep -v Running | grep -v Completed)" ]; do sleep 1; done
+  args:
+    chdir: ~/hybrid-files

blueprints/apigee/hybrid-gke/ansible/roles/apigee-hybrid/templates/overrides.yaml.j2

@@ -0,0 +1,63 @@
+gcp:
+  region: {{ region }}
+  projectID: {{ project_id }}
+
+k8sCluster:
+  name: {{ cluster }}
+  region: CLUSTER_LOCATION # Must be the closest Google Cloud region to your cluster.
+org: {{ project_id }}
+
+instanceID: "instance-1"
+
+cassandra:
+  hostNetwork: false
+    # Set to false for single region installations and multi-region installations
+    # with connectivity between pods in different clusters, for example GKE installations.
+    # Set to true  for multi-region installations with no communication between
+    # pods in different clusters, for example GKE On-prem, GKE on AWS, Anthos on bare metal,
+    # AKS, EKS, and OpenShift installations.
+    # See Multi-region deployment: Prerequisites
+
+virtualhosts:
+  - name: {{ envgroup }}
+    selector:
+      app: apigee-ingressgateway
+    sslCertPath: ./certs/{{ envgroup }}.cert
+    sslKeyPath: ./certs/{{ envgroup }}.key
+
+ao:
+  args:
+  # This configuration is introduced in hybrid v1.8
+    disableIstioConfigInAPIServer: true
+
+# This configuration is introduced in hybrid v1.8
+ingressGateways:
+- name: ingress # maximum 17 characters. See Known issue 243167389.
+  replicaCountMin: 2
+  replicaCountMax: 10
+
+envs:
+  - name: {{ env }}
+    serviceAccountPaths:
+      synchronizer: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+      udca: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+      runtime: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+mart:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+connectAgent:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+metrics:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+udca:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+watcher:
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json
+
+logger:
+  enabled: true
+  serviceAccountPath: ./service-accounts/{{ project_id }}-apigee-non-prod.json

blueprints/apigee/hybrid-gke/ansible/roles/prerequisites/tasks/main.yaml

@@ -0,0 +1,37 @@
+# Copyright 2021 Google LLC
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+- name: Download the Google Cloud SDK package repository signing key
+  get_url:
+    url: https://packages.cloud.google.com/apt/doc/apt-key.gpg
+    dest: /usr/share/keyrings/cloud.google.gpg
+
+- name: Add Google Cloud SDK package repository source
+  apt_repository:
+    filename: google-cloud-sdk.list
+    repo: "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main"
+    state: present
+    update_cache: yes
+
+- name: Install dependencies
+  apt:
+    pkg:
+      - kubectl
+      - google-cloud-sdk-gke-gcloud-auth-plugin
+    state: present
+
+- name: Install gke-gcloud-auth-plugin
+  apt:
+    name: google-cloud-sdk-gke-gcloud-auth-plugin
+    state: present

blueprints/apigee/hybrid-gke/apigee.tf

@@ -0,0 +1,46 @@
+/**
+ * Copyright 2023 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+locals {
+  envgroup    = "test"
+  environment = "apis-test"
+}
+
+module "apigee" {
+  source     = "../../../modules/apigee"
+  project_id = module.project.project_id
+  organization = {
+    analytics_region = var.region
+    runtime_type     = "HYBRID"
+  }
+  envgroups = {
+    (local.envgroup) = [var.hostname]
+  }
+  environments = {
+    (local.environment) = {
+      envgroups = [local.envgroup]
+    }
+  }
+}
+
+resource "local_file" "deploy_apiproxy_file" {
+  content = templatefile("${path.module}/templates/deploy-apiproxy.sh.tpl", {
+    org = module.project.project_id
+    env = local.environment
+  })
+  filename        = "${path.module}/deploy-apiproxy.sh"
+  file_permission = "0777"
+}

blueprints/apigee/hybrid-gke/gke.tf

@@ -0,0 +1,82 @@
+/**
+ * Copyright 2023 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+module "cluster" {
+  source     = "../../../modules/gke-cluster"
+  project_id = module.project.project_id
+  name       = "cluster"
+  location   = var.region
+  vpc_config = {
+    network    = module.vpc.self_link
+    subnetwork = module.vpc.subnet_self_links["${var.region}/subnet-apigee"]
+    secondary_range_names = {
+      pods     = "pods"
+      services = "services"
+    }
+    master_authorized_ranges = var.cluster_network_config.master_authorized_cidr_blocks
+    master_ipv4_cidr_block   = var.cluster_network_config.master_cidr_block
+  }
+  max_pods_per_node = 32
+  private_cluster_config = {
+    enable_private_endpoint = true
+    master_global_access    = false
+  }
+  enable_features = {
+    workload_identity = true
+  }
+}
+
+module "apigee-data-nodepool" {
+  source       = "../../../modules/gke-nodepool"
+  project_id   = module.project.project_id
+  cluster_name = module.cluster.name
+  location     = var.region
+  name         = "apigee-data-nodepool"
+  nodepool_config = {
+    autoscaling = {
+      min_node_count = 1
+      max_node_count = 3
+    }
+  }
+  node_config = {
+    machine_type = var.cluster_machine_type
+  }
+  service_account = {
+    create = true
+  }
+  tags = ["node"]
+}
+
+module "apigee-runtime-nodepool" {
+  source       = "../../../modules/gke-nodepool"
+  project_id   = module.project.project_id
+  cluster_name = module.cluster.name
+  location     = var.region
+  name         = "apigee-runtime-nodepool"
+  nodepool_config = {
+    autoscaling = {
+      min_node_count = 1
+      max_node_count = 3
+    }
+  }
+  node_config = {
+    machine_type = var.cluster_machine_type
+  }
+  service_account = {
+    create = true
+  }
+  tags = ["node"]
+}

blueprints/apigee/hybrid-gke/main.tf

@@ -0,0 +1,44 @@
+/**
+ * Copyright 2023 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+module "project" {
+  source = "../../../modules/project"
+  billing_account = (var.project_create != null
+    ? var.project_create.billing_account_id
+    : null
+  )
+  parent = (var.project_create != null
+    ? var.project_create.parent
+    : null
+  )
+  project_create = var.project_create != null
+  name           = var.project_id
+  services = [
+    "apigee.googleapis.com",
+    "apigeeconnect.googleapis.com",
+    "cloudresourcemanager.googleapis.com",
+    "compute.googleapis.com",
+    "container.googleapis.com",
+    "pubsub.googleapis.com"
+  ]
+  iam = {
+    "roles/apigee.admin"                    = [module.mgmt_server.service_account_iam_email]
+    "roles/container.admin"                 = [module.mgmt_server.service_account_iam_email]
+    "roles/resourcemanager.projectIamAdmin" = [module.mgmt_server.service_account_iam_email]
+    "roles/iam.serviceAccountAdmin"         = [module.mgmt_server.service_account_iam_email]
+    "roles/iam.serviceAccountKeyAdmin"      = [module.mgmt_server.service_account_iam_email]
+  }
+}

blueprints/apigee/hybrid-gke/mgmt.tf

@@ -0,0 +1,37 @@
+/**
+ * Copyright 2023 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+# tfdoc:file:description Management server.
+
+module "mgmt_server" {
+  source        = "../../../modules/compute-vm"
+  project_id    = module.project.project_id
+  zone          = var.zone
+  name          = "mgmt"
+  instance_type = var.mgmt_server_config.instance_type
+  network_interfaces = [{
+    network    = module.vpc.self_link
+    subnetwork = module.vpc.subnet_self_links["${var.region}/subnet-mgmt"]
+    nat        = false
+    addresses  = null
+  }]
+  service_account_create = true
+  boot_disk = {
+    image = var.mgmt_server_config.image
+    type  = var.mgmt_server_config.disk_type
+    size  = var.mgmt_server_config.disk_size
+  }
+}

blueprints/apigee/hybrid-gke/templates/deploy-apiproxy.sh.tpl

@@ -0,0 +1,36 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+ 
+#!/bin/bash
+
+ORG_NAME=${org}
+ENV_NAME=${env}
+
+wget https://github.com/apigee/api-platform-samples/raw/master/sample-proxies/apigee-quickstart/httpbin_rev1_2020_02_02.zip -O apiproxy.zip 
+
+export TOKEN=$(gcloud auth print-access-token)
+
+curl -v -X POST \
+-H "Authorization: Bearer $TOKEN" \
+-H "Content-Type:application/octet-stream" \
+-T 'apiproxy.zip' \
+"https://apigee.googleapis.com/v1/organizations/$ORG_NAME/apis?name=httpbin&action=import"
+
+curl -v -X POST \
+-H "Authorization: Bearer $TOKEN" \
+"https://apigee.googleapis.com/v1/organizations/$ORG_NAME/environments/$ENV_NAME/apis/httpbin/revisions/1/deployments"
+
+curl -v \
+-H "Authorization: Bearer $TOKEN" \
+"https://apigee.googleapis.com/v1/organizations/$ORG_NAME/environments/$ENV_NAME/apis/httpbin/revisions/1/deployments"

blueprints/apigee/hybrid-gke/templates/gssh.sh.tpl

@@ -0,0 +1,30 @@
+#!/bin/bash
+#
+# Copyright 2023 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+host="$${@: -2: 1}"
+cmd="$${@: -1: 1}"
+
+gcloud_args="
+--tunnel-through-iap
+--zone=${zone}
+--project=${project_id}
+--quiet
+--no-user-output-enabled
+--
+-C
+"
+
+exec gcloud compute ssh "$host" $gcloud_args "$cmd"

blueprints/apigee/hybrid-gke/terraform.tfvars.sample

@@ -0,0 +1,6 @@
+project_create = {
+  billing_account_id = "12345-12345-12345"
+  parent             = "folders/123456789"
+}
+project_id = "my-project"
+hostname = "test.myorg.org"

blueprints/apigee/hybrid-gke/variables.tf

@@ -0,0 +1,94 @@
+/**
+ * Copyright 2023 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+variable "cluster_machine_type" {
+  description = "Cluster nachine type."
+  type        = string
+  default     = "e2-standard-4"
+}
+
+variable "cluster_network_config" {
+  description = "Cluster network configuration."
+  type = object({
+    nodes_cidr_block              = string
+    pods_cidr_block               = string
+    services_cidr_block           = string
+    master_authorized_cidr_blocks = map(string)
+    master_cidr_block             = string
+  })
+  default = {
+    nodes_cidr_block    = "10.0.1.0/24"
+    pods_cidr_block     = "172.16.0.0/20"
+    services_cidr_block = "192.168.0.0/24"
+    master_authorized_cidr_blocks = {
+      internal = "10.0.0.0/8"
+    }
+    master_cidr_block = "10.0.0.0/28"
+  }
+}
+
+variable "hostname" {
+  description = "Host name."
+  type        = string
+}
+
+variable "mgmt_server_config" {
+  description = "Mgmt server configuration."
+  type = object({
+    disk_size     = number
+    disk_type     = string
+    image         = string
+    instance_type = string
+  })
+  default = {
+    disk_size     = 50
+    disk_type     = "pd-ssd"
+    image         = "projects/ubuntu-os-cloud/global/images/family/ubuntu-2204-lts"
+    instance_type = "n1-standard-2"
+  }
+}
+
+variable "mgmt_subnet_cidr_block" {
+  description = "Management subnet CIDR block."
+  type        = string
+  default     = "10.0.2.0/28"
+}
+
+variable "project_create" {
+  description = "Parameters for the creation of the new project."
+  type = object({
+    billing_account_id = string
+    parent             = string
+  })
+  default = null
+}
+
+variable "project_id" {
+  description = "Project ID."
+  type        = string
+}
+
+variable "region" {
+  description = "Region."
+  type        = string
+  default     = "europe-west1"
+}
+
+variable "zone" {
+  description = "Zone."
+  type        = string
+  default     = "europe-west1-c"
+}

blueprints/apigee/hybrid-gke/vpc.tf

@@ -0,0 +1,63 @@
+/**
+ * Copyright 2023 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+module "vpc" {
+  source     = "../../../modules/net-vpc"
+  project_id = module.project.project_id
+  name       = "vpc"
+  subnets = [
+    {
+      ip_cidr_range = var.cluster_network_config.nodes_cidr_block
+      name          = "subnet-apigee"
+      region        = var.region
+      secondary_ip_ranges = {
+        pods     = var.cluster_network_config.pods_cidr_block
+        services = var.cluster_network_config.services_cidr_block
+      }
+    },
+    {
+      ip_cidr_range = var.mgmt_subnet_cidr_block
+      name          = "subnet-mgmt"
+      region        = var.region
+    }
+  ]
+}
+
+module "firewall" {
+  source     = "../../../modules/net-vpc-firewall"
+  project_id = module.project.project_id
+  network    = module.vpc.name
+  ingress_rules = {
+    # implicit allow action
+    allow-cp = {
+      description = "Allow control plane access to pods."
+      targets     = ["node"]
+      rules = [{
+        protocol = "tcp"
+        ports    = [15017, 9443]
+      }]
+    }
+  }
+}
+
+module "nat" {
+  source         = "../../../modules/net-cloudnat"
+  project_id     = module.project.project_id
+  region         = var.region
+  name           = "nat"
+  router_create  = true
+  router_network = module.vpc.name
+}

tests/blueprints/apigee/hybrid-gke/__init__.py

@@ -0,0 +1,13 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

tests/blueprints/apigee/hybrid-gke/basic.tfvars

@@ -0,0 +1,6 @@
+project_create = {
+  billing_account_id = "12345-12345-12345"
+  parent             = "folders/123456789"
+}
+project_id = "my-project"
+hostname   = "test.myorg.org"

tests/blueprints/apigee/hybrid-gke/basic.yaml

@@ -0,0 +1,17 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+counts:
+  modules: 9
+  resources: 37

tests/blueprints/apigee/hybrid-gke/tftest.yaml

@@ -0,0 +1,18 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+module: blueprints/apigee/hybrid-gke
+
+tests:
+  basic: