cloud-foundation-fabric/modules/gke-cluster-autopilot

GKE Autopilot cluster module

This module offers a way to create and manage Google Kubernetes Engine (GKE) Autopilot clusters. With its sensible default settings based on best practices and authors' experience as Google Cloud practitioners, the module accommodates for many common use cases out-of-the-box, without having to rely on verbose configuration.

• Examples
  • GKE Autopilot cluster
  • Cloud DNS
  • Logging configuration
  • Monitoring configuration
  • Backup for GKE
• Variables
• Outputs

Examples

GKE Autopilot cluster

This example shows how to create a GKE cluster in Autopilot mode.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = "myproject"
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network    = var.vpc.self_link
    subnetwork = var.subnet.self_link
    secondary_range_names = {
      pods     = "pods"
      services = "services"
    }
    master_authorized_ranges = {
      internal-vms = "10.0.0.0/8"
    }
    master_ipv4_cidr_block = "192.168.0.0/28"
  }
  private_cluster_config = {
    enable_private_endpoint = true
    master_global_access    = false
  }
  labels = {
    environment = "dev"
  }
}
# tftest modules=1 resources=1 inventory=basic.yaml

Cloud DNS

[!WARNING] Cloud DNS is the only DNS provider for Autopilot clusters running version 1.25.9-gke.400 and later, and version 1.26.4-gke.500 and later. It is pre-configured for those clusters. The following example only applies to Autopilot clusters running earlier versions.

This example shows how to use Cloud DNS as a Kubernetes DNS provider.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {} # use default names "pods" and "services"
  }
  enable_features = {
    dns = {
      provider = "CLOUD_DNS"
      scope    = "CLUSTER_SCOPE"
      domain   = "gke.local"
    }
  }
}
# tftest modules=1 resources=1 inventory=dns.yaml

Logging configuration

[!NOTE] System and workload logs collection is pre-configured for Autopilot clusters and cannot be disabled.

This example shows how to collect logs for the Kubernetes control plane components. The logs for these components are not collected by default.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {} # use default names "pods" and "services"
  }
  logging_config = {
    enable_api_server_logs         = true
    enable_scheduler_logs          = true
    enable_controller_manager_logs = true
  }
}
# tftest modules=1 resources=1 inventory=logging-config.yaml

Monitoring configuration

[!NOTE] System metrics collection is pre-configured for Autopilot clusters and cannot be disabled.

[!WARNING] GKE workload metrics is deprecated and removed in GKE 1.24 and later. Workload metrics is replaced by Google Cloud Managed Service for Prometheus, which is Google's recommended way to monitor Kubernetes applications by using Cloud Monitoring.

This example shows how to configure collection of Kubernetes control plane metrics. These metrics are optional and are not collected by default.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {} # use default names "pods" and "services"
  }
  monitoring_config = {
    enable_api_server_metrics         = true
    enable_controller_manager_metrics = true
    enable_scheduler_metrics          = true
  }
}
# tftest modules=1 resources=1 inventory=monitoring-config-control-plane.yaml

The next example shows how to configure collection of kube state metrics. These metrics are optional and are not collected by default.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {} # use default names "pods" and "services"
  }
  monitoring_config = {
    enable_daemonset_metrics   = true
    enable_deployment_metrics  = true
    enable_hpa_metrics         = true
    enable_pod_metrics         = true
    enable_statefulset_metrics = true
    enable_storage_metrics     = true
    # Kube state metrics collection requires Google Cloud Managed Service for Prometheus,
    # which is enabled by default.
    # enable_managed_prometheus = true
  }
}
# tftest modules=1 resources=1 inventory=monitoring-config-kube-state.yaml

The control plane metrics and kube state metrics collection can be configured in a single monitoring_config block.

Backup for GKE

[!NOTE] Although Backup for GKE can be enabled as an add-on when configuring your GKE clusters, it is a separate service from GKE.

Backup for GKE is a service for backing up and restoring workloads in GKE clusters. It has two components:

• A Google Cloud API that serves as the control plane for the service.
• A GKE add-on (the Backup for GKE agent) that must be enabled in each cluster for which you wish to perform backup and restore operations.

Backup for GKE is supported in GKE Autopilot clusters with some restrictions.

This example shows how to enable Backup for GKE on a new Autopilot cluster and plan a set of backups.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {}
  }
  backup_configs = {
    enable_backup_agent = true
    backup_plans = {
      "backup-1" = {
        region   = "europe-west-2"
        schedule = "0 9 * * 1"
      }
    }
  }
}
# tftest modules=1 resources=2 inventory=backup.yaml

Variables

name	description	type	required	default
location	Autopilot clusters are always regional.	string	✓
name	Cluster name.	string	✓
project_id	Cluster project ID.	string	✓
vpc_config	VPC-level configuration.	object({…})	✓
backup_configs	Configuration for Backup for GKE.	object({…})		{}
deletion_protection	Whether or not to allow Terraform to destroy the cluster. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the cluster will fail.	bool		true
description	Cluster description.	string		null
enable_addons	Addons enabled in the cluster (true means enabled).	object({…})		{}
enable_features	Enable cluster-level features. Certain features allow configuration.	object({…})		{}
issue_client_certificate	Enable issuing client certificate.	bool		false
labels	Cluster resource labels.	map(string)		null
logging_config	Logging configuration.	object({…})		{}
maintenance_config	Maintenance window configuration.	object({…})		{…}
min_master_version	Minimum version of the master, defaults to the version of the most recent official release.	string		null
monitoring_config	Monitoring configuration. System metrics collection cannot be disabled. Control plane metrics are optional. Kube state metrics are optional. Google Cloud Managed Service for Prometheus is enabled by default.	object({…})		{}
node_config	Configuration for nodes and nodepools.	object({…})		{}
node_locations	Zones in which the cluster's nodes are located.	list(string)		[]
private_cluster_config	Private cluster configuration.	object({…})		null
release_channel	Release channel for GKE upgrades. Clusters created in the Autopilot mode must use a release channel. Choose between "RAPID", "REGULAR", and "STABLE".	string		"REGULAR"

Outputs

name	description	sensitive
ca_certificate	Public certificate of the cluster (base64-encoded).	✓
cluster	Cluster resource.	✓
endpoint	Cluster endpoint.
id	Fully qualified cluster ID.
location	Cluster location.
master_version	Master version.
name	Cluster name.
notifications	GKE Pub/Sub notifications topic.
self_link	Cluster self link.	✓
workload_identity_pool	Workload identity pool.