Separating GKE Standard and Autopilot Modules (#1330)

* separating GKE Standard and Autopilot Modules

* Changes for Updating the terraform and provide versions

* Changes for Autopilot Readme

* Changes for Autopilot Variable

* Changes for Autopilot Readme

* Changes for Autopilot Readme

* Changes for Blueprint

* Changes for Blueprint ReadMe

* Changes for gke-standard-cluster dependency

* Changes for gke-standard-cluster in gke-fleet

* Changes for gke-standard-cluster in cluster-mesh-gke-fleet-api

* python formatting

* python formatting

* python formatting

* GKE module naming convention

* Readme Changes

* test module

* Removing comment code from Autopilot
This commit is contained in:
Avinash Jha 2023-04-21 17:38:13 +05:30 committed by GitHub
parent df8c61fe69
commit e881537f87
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
31 changed files with 908 additions and 142 deletions

View File

@ -161,4 +161,4 @@ Even with all the above points, it may be hard to make a decision. While the mod
* Since modules work well together within their ecosystem, select logical boundaries for using Fabric or CFT. For example use CFT for deploying resources within projects but use Fabric for managing project creation and IAM. * Since modules work well together within their ecosystem, select logical boundaries for using Fabric or CFT. For example use CFT for deploying resources within projects but use Fabric for managing project creation and IAM.
* Use strengths of each collection of modules to your advantage. Empower application teams to define their infrastructure as code using off the shelf CFT modules. Using Fabric, bootstrap your platform team with a collection of tailor built modules for your organization. * Use strengths of each collection of modules to your advantage. Empower application teams to define their infrastructure as code using off the shelf CFT modules. Using Fabric, bootstrap your platform team with a collection of tailor built modules for your organization.
* Lean into module composition and dependency inversion that both Fabric and CFT modules follow. For example, you can create a GKE cluster using either [Fabric](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/gke-cluster#gke-cluster-module) or [CFT](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine) GKE module and then use either [Fabric](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/gke-hub#variables) or [CFT](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/tree/master/modules/fleet-membership) for setting up GKE Hub by passing in outputs from the GKE module. * Lean into module composition and dependency inversion that both Fabric and CFT modules follow. For example, you can create a GKE cluster using either [Fabric](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/gke-cluster-standard#gke-cluster-module) or [CFT](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine) GKE module and then use either [Fabric](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/gke-hub#variables) or [CFT](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/tree/master/modules/fleet-membership) for setting up GKE Hub by passing in outputs from the GKE module.

View File

@ -31,7 +31,7 @@ Currently available modules:
- **foundational** - [billing budget](./modules/billing-budget), [Cloud Identity group](./modules/cloud-identity-group/), [folder](./modules/folder), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [organization](./modules/organization), [project](./modules/project), [projects-data-source](./modules/projects-data-source) - **foundational** - [billing budget](./modules/billing-budget), [Cloud Identity group](./modules/cloud-identity-group/), [folder](./modules/folder), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [organization](./modules/organization), [project](./modules/project), [projects-data-source](./modules/projects-data-source)
- **networking** - [DNS](./modules/dns), [DNS Response Policy](./modules/dns-response-policy/), [Cloud Endpoints](./modules/endpoints), [address reservation](./modules/net-address), [NAT](./modules/net-cloudnat), [Global Load Balancer (classic)](./modules/net-glb/), [L4 ILB](./modules/net-ilb), [L7 ILB](./modules/net-ilb-l7), [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC firewall policy](./modules/net-vpc-firewall-policy), [VPC peering](./modules/net-vpc-peering), [VPN dynamic](./modules/net-vpn-dynamic), [HA VPN](./modules/net-vpn-ha), [VPN static](./modules/net-vpn-static), [Service Directory](./modules/service-directory) - **networking** - [DNS](./modules/dns), [DNS Response Policy](./modules/dns-response-policy/), [Cloud Endpoints](./modules/endpoints), [address reservation](./modules/net-address), [NAT](./modules/net-cloudnat), [Global Load Balancer (classic)](./modules/net-glb/), [L4 ILB](./modules/net-ilb), [L7 ILB](./modules/net-ilb-l7), [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC firewall policy](./modules/net-vpc-firewall-policy), [VPC peering](./modules/net-vpc-peering), [VPN dynamic](./modules/net-vpn-dynamic), [HA VPN](./modules/net-vpn-ha), [VPN static](./modules/net-vpn-static), [Service Directory](./modules/service-directory)
- **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid), [GKE cluster](./modules/gke-cluster), [GKE hub](./modules/gke-hub), [GKE nodepool](./modules/gke-nodepool) - **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid), [GKE cluster](./modules/gke-cluster-standard), [GKE hub](./modules/gke-hub), [GKE nodepool](./modules/gke-nodepool)
- **data** - [BigQuery dataset](./modules/bigquery-dataset), [Bigtable instance](./modules/bigtable-instance), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag), [Datafusion](./modules/datafusion), [Dataproc](./modules/dataproc), [GCS](./modules/gcs), [Pub/Sub](./modules/pubsub) - **data** - [BigQuery dataset](./modules/bigquery-dataset), [Bigtable instance](./modules/bigtable-instance), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag), [Datafusion](./modules/datafusion), [Dataproc](./modules/dataproc), [GCS](./modules/gcs), [Pub/Sub](./modules/pubsub)
- **development** - [API Gateway](./modules/api-gateway), [Apigee](./modules/apigee), [Artifact Registry](./modules/artifact-registry), [Container Registry](./modules/container-registry), [Cloud Source Repository](./modules/source-repository) - **development** - [API Gateway](./modules/api-gateway), [Apigee](./modules/apigee), [Artifact Registry](./modules/artifact-registry), [Container Registry](./modules/container-registry), [Cloud Source Repository](./modules/source-repository)
- **security** - [Binauthz](./modules/binauthz/), [KMS](./modules/kms), [SecretManager](./modules/secret-manager), [VPC Service Control](./modules/vpc-sc) - **security** - [Binauthz](./modules/binauthz/), [KMS](./modules/kms), [SecretManager](./modules/secret-manager), [VPC Service Control](./modules/vpc-sc)

View File

@ -15,7 +15,7 @@
*/ */
module "cluster" { module "cluster" {
source = "../../../modules/gke-cluster" source = "../../../modules/gke-cluster-standard"
project_id = module.project.project_id project_id = module.project.project_id
name = "cluster" name = "cluster"
location = var.region location = var.region

View File

@ -80,8 +80,9 @@ def do_discovery(resources):
resources[result.type][result.id][result.key] = result.data resources[result.type][result.id][result.key] = result.data
else: else:
resources[result.type][result.id] = result.data resources[result.type][result.id] = result.data
LOGGER.info('discovery end {}'.format( LOGGER.info('discovery end {}'.format({
{k: len(v) for k, v in resources.items() if not isinstance(v, str)})) k: len(v) for k, v in resources.items() if not isinstance(v, str)
}))
def do_init(resources, discovery_root, monitoring_project, folders=None, def do_init(resources, discovery_root, monitoring_project, folders=None,

View File

@ -15,7 +15,7 @@
*/ */
module "cluster" { module "cluster" {
source = "../../../modules/gke-cluster" source = "../../../modules/gke-cluster-autopilot"
project_id = module.project.project_id project_id = module.project.project_id
name = "cluster" name = "cluster"
location = var.region location = var.region
@ -29,18 +29,18 @@ module "cluster" {
master_authorized_ranges = var.cluster_network_config.master_authorized_cidr_blocks master_authorized_ranges = var.cluster_network_config.master_authorized_cidr_blocks
master_ipv4_cidr_block = var.cluster_network_config.master_cidr_block master_ipv4_cidr_block = var.cluster_network_config.master_cidr_block
} }
enable_features = { # enable_features = {
autopilot = true # autopilot = true
} # }
monitoring_config = { # monitoring_config = {
enenable_components = ["SYSTEM_COMPONENTS"] # enenable_components = ["SYSTEM_COMPONENTS"]
managed_prometheus = true # managed_prometheus = true
} # }
cluster_autoscaling = { # cluster_autoscaling = {
auto_provisioning_defaults = { # auto_provisioning_defaults = {
service_account = module.node_sa.email # service_account = module.node_sa.email
} # }
} # }
release_channel = "RAPID" release_channel = "RAPID"
depends_on = [ depends_on = [
module.project module.project

View File

@ -83,7 +83,7 @@ module "nat" {
} }
module "cluster" { module "cluster" {
source = "../../../modules/gke-cluster" source = "../../../modules/gke-cluster-standard"
project_id = module.project.project_id project_id = module.project.project_id
name = "${var.prefix}-cluster" name = "${var.prefix}-cluster"
location = var.zone location = var.zone

View File

@ -53,7 +53,7 @@ Once done testing, you can clean up resources by running `terraform destroy`.
| name | description | modules | resources | | name | description | modules | resources |
|---|---|---|---| |---|---|---|---|
| [ansible.tf](./ansible.tf) | Ansible generated files. | | <code>local_file</code> | | [ansible.tf](./ansible.tf) | Ansible generated files. | | <code>local_file</code> |
| [gke.tf](./gke.tf) | GKE cluster and hub resources. | <code>gke-cluster</code> · <code>gke-hub</code> · <code>gke-nodepool</code> | | | [gke.tf](./gke.tf) | GKE cluster and hub resources. | <code>gke-cluster-standard</code> · <code>gke-hub</code> · <code>gke-nodepool</code> | |
| [main.tf](./main.tf) | Project resources. | <code>project</code> | | | [main.tf](./main.tf) | Project resources. | <code>project</code> | |
| [variables.tf](./variables.tf) | Module variables. | | | | [variables.tf](./variables.tf) | Module variables. | | |
| [vm.tf](./vm.tf) | Management server. | <code>compute-vm</code> | | | [vm.tf](./vm.tf) | Management server. | <code>compute-vm</code> | |
@ -75,7 +75,6 @@ Once done testing, you can clean up resources by running `terraform destroy`.
| [region](variables.tf#L99) | Region. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> | | [region](variables.tf#L99) | Region. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
<!-- END TFDOC --> <!-- END TFDOC -->
## Test ## Test
```hcl ```hcl

View File

@ -18,7 +18,7 @@
module "clusters" { module "clusters" {
for_each = var.clusters_config for_each = var.clusters_config
source = "../../../modules/gke-cluster" source = "../../../modules/gke-cluster-standard"
project_id = module.fleet_project.project_id project_id = module.fleet_project.project_id
name = each.key name = each.key
location = var.region location = var.region

View File

@ -234,7 +234,7 @@ module "gke" {
| name | description | modules | | name | description | modules |
|---|---|---| |---|---|---|
| [gke-clusters.tf](./gke-clusters.tf) | GKE clusters. | <code>gke-cluster</code> | | [gke-clusters.tf](./gke-clusters.tf) | GKE clusters. | <code>gke-cluster-standard</code> |
| [gke-hub.tf](./gke-hub.tf) | GKE hub configuration. | <code>gke-hub</code> | | [gke-hub.tf](./gke-hub.tf) | GKE hub configuration. | <code>gke-hub</code> |
| [gke-nodepools.tf](./gke-nodepools.tf) | GKE nodepools. | <code>gke-nodepool</code> | | [gke-nodepools.tf](./gke-nodepools.tf) | GKE nodepools. | <code>gke-nodepool</code> |
| [main.tf](./main.tf) | Project and usage dataset. | <code>bigquery-dataset</code> · <code>project</code> | | [main.tf](./main.tf) | Project and usage dataset. | <code>bigquery-dataset</code> · <code>project</code> |

View File

@ -17,7 +17,7 @@
# tfdoc:file:description GKE clusters. # tfdoc:file:description GKE clusters.
module "gke-cluster" { module "gke-cluster" {
source = "../../../modules/gke-cluster" source = "../../../modules/gke-cluster-standard"
for_each = var.clusters for_each = var.clusters
name = each.key name = each.key
project_id = module.gke-project-0.project_id project_id = module.gke-project-0.project_id

View File

@ -240,7 +240,7 @@ module "service-account-gce" {
################################################################################ ################################################################################
module "cluster-1" { module "cluster-1" {
source = "../../../modules/gke-cluster" source = "../../../modules/gke-cluster-standard"
name = "${var.prefix}-cluster-1" name = "${var.prefix}-cluster-1"
project_id = module.project.project_id project_id = module.project.project_id
location = "${var.region}-b" location = "${var.region}-b"

View File

@ -197,7 +197,7 @@ module "vm-bastion" {
################################################################################ ################################################################################
module "cluster-1" { module "cluster-1" {
source = "../../../modules/gke-cluster" source = "../../../modules/gke-cluster-standard"
count = var.cluster_create ? 1 : 0 count = var.cluster_create ? 1 : 0
name = "cluster-1" name = "cluster-1"
project_id = module.project-svc-gke.project_id project_id = module.project-svc-gke.project_id

View File

@ -63,7 +63,8 @@ These modules are used in the examples included in this repository. If you are u
- [VM/VM group](./compute-vm) - [VM/VM group](./compute-vm)
- [MIG](./compute-mig) - [MIG](./compute-mig)
- [COS container](./cloud-config-container/cos-generic-metadata/) (coredns/mysql/nva/onprem/squid) - [COS container](./cloud-config-container/cos-generic-metadata/) (coredns/mysql/nva/onprem/squid)
- [GKE cluster](./gke-cluster) - [GKE autopilot cluster](./gke-cluster-autopilot)
- [GKE standard cluster](./gke-cluster-standard)
- [GKE hub](./gke-hub) - [GKE hub](./gke-hub)
- [GKE nodepool](./gke-nodepool) - [GKE nodepool](./gke-nodepool)

View File

@ -0,0 +1,132 @@
# GKE cluster Autopilot module
This module allows simplified creation and management of GKE Autopilot clusters. Some sensible defaults are set initially, in order to allow less verbose usage for most use cases.
## Example
### GKE Cluster
```hcl
module "cluster-1" {
source = "./fabric/modules/gke-cluster-autopilot"
project_id = "myproject"
name = "cluster-1"
location = "europe-west1"
vpc_config = {
network = var.vpc.self_link
subnetwork = var.subnet.self_link
secondary_range_names = {
pods = "pods"
services = "services"
}
master_authorized_ranges = {
internal-vms = "10.0.0.0/8"
}
master_ipv4_cidr_block = "192.168.0.0/28"
}
private_cluster_config = {
enable_private_endpoint = true
master_global_access = false
}
labels = {
environment = "dev"
}
}
# tftest modules=1 resources=1 inventory=basic.yaml
```
### Cloud DNS
This example shows how to [use Cloud DNS as a Kubernetes DNS provider](https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-dns) for GKE Standard clusters.
```hcl
module "cluster-1" {
source = "./fabric/modules/gke-cluster-autopilot"
project_id = var.project_id
name = "cluster-1"
location = "europe-west1"
vpc_config = {
network = var.vpc.self_link
subnetwork = var.subnet.self_link
secondary_range_names = { pods = "pods", services = "services" }
}
enable_features = {
dns = {
provider = "CLOUD_DNS"
scope = "CLUSTER_SCOPE"
domain = "gke.local"
}
}
}
# tftest modules=1 resources=1 inventory=dns.yaml
```
### Backup for GKE
This example shows how to [enable the Backup for GKE agent and configure a Backup Plan](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/backup-for-gke) for GKE Standard clusters.
```hcl
module "cluster-1" {
source = "./fabric/modules/gke-cluster-autopilot"
project_id = var.project_id
name = "cluster-1"
location = "europe-west1"
vpc_config = {
network = var.vpc.self_link
subnetwork = var.subnet.self_link
secondary_range_names = { pods = "pods", services = "services" }
}
backup_configs = {
enable_backup_agent = true
backup_plans = {
"backup-1" = {
region = "europe-west-2"
schedule = "0 9 * * 1"
}
}
}
}
# tftest modules=1 resources=2 inventory=backup.yaml
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [location](variables.tf#L106) | Autopilot cluster are always regional. | <code>string</code> | ✓ | |
| [name](variables.tf#L141) | Cluster name. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L167) | Cluster project id. | <code>string</code> | ✓ | |
| [vpc_config](variables.tf#L190) | VPC-level configuration. | <code title="object&#40;&#123;&#10; network &#61; string&#10; subnetwork &#61; string&#10; master_ipv4_cidr_block &#61; optional&#40;string&#41;&#10; secondary_range_blocks &#61; optional&#40;object&#40;&#123;&#10; pods &#61; string&#10; services &#61; string&#10; &#125;&#41;&#41;&#10; secondary_range_names &#61; optional&#40;object&#40;&#123;&#10; pods &#61; string&#10; services &#61; string&#10; &#125;&#41;, &#123; pods &#61; &#34;pods&#34;, services &#61; &#34;services&#34; &#125;&#41;&#10; master_authorized_ranges &#61; optional&#40;map&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | |
| [backup_configs](variables.tf#L17) | Configuration for Backup for GKE. | <code title="object&#40;&#123;&#10; enable_backup_agent &#61; optional&#40;bool, false&#41;&#10; backup_plans &#61; optional&#40;map&#40;object&#40;&#123;&#10; region &#61; string&#10; schedule &#61; string&#10; retention_policy_days &#61; optional&#40;string&#41;&#10; retention_policy_lock &#61; optional&#40;bool, false&#41;&#10; retention_policy_delete_lock_days &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [description](variables.tf#L33) | Cluster description. | <code>string</code> | | <code>null</code> |
| [enable_addons](variables.tf#L39) | Addons enabled in the cluster (true means enabled). | <code title="object&#40;&#123;&#10; cloudrun &#61; optional&#40;bool, false&#41;&#10; config_connector &#61; optional&#40;bool, false&#41;&#10; dns_cache &#61; optional&#40;bool, false&#41;&#10; horizontal_pod_autoscaling &#61; optional&#40;bool, false&#41;&#10; http_load_balancing &#61; optional&#40;bool, false&#41;&#10; istio &#61; optional&#40;object&#40;&#123;&#10; enable_tls &#61; bool&#10; &#125;&#41;&#41;&#10; kalm &#61; optional&#40;bool, false&#41;&#10; network_policy &#61; optional&#40;bool, false&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; horizontal_pod_autoscaling &#61; true&#10; http_load_balancing &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> |
| [enable_features](variables.tf#L60) | Enable cluster-level features. Certain features allow configuration. | <code title="object&#40;&#123;&#10; binary_authorization &#61; optional&#40;bool, false&#41;&#10; dns &#61; optional&#40;object&#40;&#123;&#10; provider &#61; optional&#40;string&#41;&#10; scope &#61; optional&#40;string&#41;&#10; domain &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; database_encryption &#61; optional&#40;object&#40;&#123;&#10; state &#61; string&#10; key_name &#61; string&#10; &#125;&#41;&#41;&#10; gateway_api &#61; optional&#40;bool, false&#41;&#10; groups_for_rbac &#61; optional&#40;string&#41;&#10; l4_ilb_subsetting &#61; optional&#40;bool, false&#41;&#10; mesh_certificates &#61; optional&#40;bool&#41;&#10; pod_security_policy &#61; optional&#40;bool, false&#41;&#10; resource_usage_export &#61; optional&#40;object&#40;&#123;&#10; dataset &#61; string&#10; enable_network_egress_metering &#61; optional&#40;bool&#41;&#10; enable_resource_consumption_metering &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; tpu &#61; optional&#40;bool, false&#41;&#10; upgrade_notifications &#61; optional&#40;object&#40;&#123;&#10; topic_id &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; vertical_pod_autoscaling &#61; optional&#40;bool, false&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10;&#10;&#10;&#125;">&#123;&#8230;&#125;</code> |
| [issue_client_certificate](variables.tf#L94) | Enable issuing client certificate. | <code>bool</code> | | <code>false</code> |
| [labels](variables.tf#L100) | Cluster resource labels. | <code>map&#40;string&#41;</code> | | <code>null</code> |
| [maintenance_config](variables.tf#L112) | Maintenance window configuration. | <code title="object&#40;&#123;&#10; daily_window_start_time &#61; optional&#40;string&#41;&#10; recurring_window &#61; optional&#40;object&#40;&#123;&#10; start_time &#61; string&#10; end_time &#61; string&#10; recurrence &#61; string&#10; &#125;&#41;&#41;&#10; maintenance_exclusions &#61; optional&#40;list&#40;object&#40;&#123;&#10; name &#61; string&#10; start_time &#61; string&#10; end_time &#61; string&#10; scope &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; daily_window_start_time &#61; &#34;03:00&#34;&#10; recurring_window &#61; null&#10; maintenance_exclusion &#61; &#91;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> |
| [min_master_version](variables.tf#L135) | Minimum version of the master, defaults to the version of the most recent official release. | <code>string</code> | | <code>null</code> |
| [node_locations](variables.tf#L146) | Zones in which the cluster's nodes are located. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [private_cluster_config](variables.tf#L153) | Private cluster configuration. | <code title="object&#40;&#123;&#10; enable_private_endpoint &#61; optional&#40;bool&#41;&#10; master_global_access &#61; optional&#40;bool&#41;&#10; peering_config &#61; optional&#40;object&#40;&#123;&#10; export_routes &#61; optional&#40;bool&#41;&#10; import_routes &#61; optional&#40;bool&#41;&#10; project_id &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [release_channel](variables.tf#L172) | Release channel for GKE upgrades. | <code>string</code> | | <code>null</code> |
| [service_account](variables.tf#L178) | The Google Cloud Platform Service Account to be used by the node VMs created by GKE Autopilot. | <code>string</code> | | <code>null</code> |
| [tags](variables.tf#L184) | Network tags applied to nodes. | <code>list&#40;string&#41;</code> | | <code>null</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| [ca_certificate](outputs.tf#L17) | Public certificate of the cluster (base64-encoded). | ✓ |
| [cluster](outputs.tf#L23) | Cluster resource. | ✓ |
| [endpoint](outputs.tf#L29) | Cluster endpoint. | |
| [id](outputs.tf#L34) | Cluster ID. | |
| [location](outputs.tf#L39) | Cluster location. | |
| [master_version](outputs.tf#L44) | Master version. | |
| [name](outputs.tf#L49) | Cluster name. | |
| [notifications](outputs.tf#L54) | GKE PubSub notifications topic. | |
| [self_link](outputs.tf#L59) | Cluster self link. | ✓ |
| [workload_identity_pool](outputs.tf#L65) | Workload identity pool. | |
<!-- END TFDOC -->

View File

@ -0,0 +1,306 @@
/**
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
resource "google_container_cluster" "cluster" {
provider = google-beta
project = var.project_id
name = var.name
description = var.description
location = var.location
node_locations = (
length(var.node_locations) == 0 ? null : var.node_locations
)
min_master_version = var.min_master_version
network = var.vpc_config.network
subnetwork = var.vpc_config.subnetwork
resource_labels = var.labels
enable_l4_ilb_subsetting = var.enable_features.l4_ilb_subsetting
enable_tpu = var.enable_features.tpu
initial_node_count = 1
enable_autopilot = true
addons_config {
http_load_balancing {
disabled = !var.enable_addons.http_load_balancing
}
horizontal_pod_autoscaling {
disabled = !var.enable_addons.horizontal_pod_autoscaling
}
cloudrun_config {
disabled = !var.enable_addons.cloudrun
}
kalm_config {
enabled = var.enable_addons.kalm
}
config_connector_config {
enabled = var.enable_addons.config_connector
}
gke_backup_agent_config {
enabled = var.backup_configs.enable_backup_agent
}
}
dynamic "authenticator_groups_config" {
for_each = var.enable_features.groups_for_rbac != null ? [""] : []
content {
security_group = var.enable_features.groups_for_rbac
}
}
dynamic "binary_authorization" {
for_each = var.enable_features.binary_authorization ? [""] : []
content {
evaluation_mode = "PROJECT_SINGLETON_POLICY_ENFORCE"
}
}
cluster_autoscaling {
dynamic "auto_provisioning_defaults" {
for_each = var.service_account != null ? [""] : []
content {
service_account = var.service_account
}
}
}
dynamic "database_encryption" {
for_each = var.enable_features.database_encryption != null ? [""] : []
content {
state = var.enable_features.database_encryption.state
key_name = var.enable_features.database_encryption.key_name
}
}
dynamic "dns_config" {
for_each = var.enable_features.dns != null ? [""] : []
content {
cluster_dns = var.enable_features.dns.provider
cluster_dns_scope = var.enable_features.dns.scope
cluster_dns_domain = var.enable_features.dns.domain
}
}
dynamic "ip_allocation_policy" {
for_each = var.vpc_config.secondary_range_blocks != null ? [""] : []
content {
cluster_ipv4_cidr_block = var.vpc_config.secondary_range_blocks.pods
services_ipv4_cidr_block = var.vpc_config.secondary_range_blocks.services
}
}
dynamic "ip_allocation_policy" {
for_each = var.vpc_config.secondary_range_names != null ? [""] : []
content {
cluster_secondary_range_name = var.vpc_config.secondary_range_names.pods
services_secondary_range_name = var.vpc_config.secondary_range_names.services
}
}
dynamic "gateway_api_config" {
for_each = var.enable_features.gateway_api ? [""] : []
content {
channel = "CHANNEL_STANDARD"
}
}
maintenance_policy {
dynamic "daily_maintenance_window" {
for_each = (
try(var.maintenance_config.daily_window_start_time, null) != null
? [""]
: []
)
content {
start_time = var.maintenance_config.daily_window_start_time
}
}
dynamic "recurring_window" {
for_each = (
try(var.maintenance_config.recurring_window, null) != null
? [""]
: []
)
content {
start_time = var.maintenance_config.recurring_window.start_time
end_time = var.maintenance_config.recurring_window.end_time
recurrence = var.maintenance_config.recurring_window.recurrence
}
}
dynamic "maintenance_exclusion" {
for_each = (
try(var.maintenance_config.maintenance_exclusions, null) == null
? []
: var.maintenance_config.maintenance_exclusions
)
iterator = exclusion
content {
exclusion_name = exclusion.value.name
start_time = exclusion.value.start_time
end_time = exclusion.value.end_time
}
}
}
master_auth {
client_certificate_config {
issue_client_certificate = var.issue_client_certificate
}
}
dynamic "master_authorized_networks_config" {
for_each = var.vpc_config.master_authorized_ranges != null ? [""] : []
content {
dynamic "cidr_blocks" {
for_each = var.vpc_config.master_authorized_ranges
iterator = range
content {
cidr_block = range.value
display_name = range.key
}
}
}
}
dynamic "mesh_certificates" {
for_each = var.enable_features.mesh_certificates != null ? [""] : []
content {
enable_certificates = var.enable_features.mesh_certificates
}
}
dynamic "notification_config" {
for_each = var.enable_features.upgrade_notifications != null ? [""] : []
content {
pubsub {
enabled = true
topic = (
try(var.enable_features.upgrade_notifications.topic_id, null) != null
? var.enable_features.upgrade_notifications.topic_id
: google_pubsub_topic.notifications[0].id
)
}
}
}
dynamic "private_cluster_config" {
for_each = (
var.private_cluster_config != null ? [""] : []
)
content {
enable_private_nodes = true
enable_private_endpoint = var.private_cluster_config.enable_private_endpoint
master_ipv4_cidr_block = try(var.vpc_config.master_ipv4_cidr_block, null)
master_global_access_config {
enabled = var.private_cluster_config.master_global_access
}
}
}
dynamic "pod_security_policy_config" {
for_each = var.enable_features.pod_security_policy ? [""] : []
content {
enabled = var.enable_features.pod_security_policy
}
}
dynamic "release_channel" {
for_each = var.release_channel != null ? [""] : []
content {
channel = var.release_channel
}
}
dynamic "resource_usage_export_config" {
for_each = (
try(var.enable_features.resource_usage_export.dataset, null) != null
? [""]
: []
)
content {
enable_network_egress_metering = (
var.enable_features.resource_usage_export.enable_network_egress_metering
)
enable_resource_consumption_metering = (
var.enable_features.resource_usage_export.enable_resource_consumption_metering
)
bigquery_destination {
dataset_id = var.enable_features.resource_usage_export.dataset
}
}
}
dynamic "vertical_pod_autoscaling" {
for_each = var.enable_features.vertical_pod_autoscaling ? [""] : []
content {
enabled = var.enable_features.vertical_pod_autoscaling
}
}
}
resource "google_gke_backup_backup_plan" "backup_plan" {
for_each = var.backup_configs.enable_backup_agent ? var.backup_configs.backup_plans : {}
name = each.key
cluster = google_container_cluster.cluster.id
location = each.value.region
project = var.project_id
retention_policy {
backup_delete_lock_days = try(each.value.retention_policy_delete_lock_days)
backup_retain_days = try(each.value.retention_policy_days)
locked = try(each.value.retention_policy_lock)
}
backup_schedule {
cron_schedule = each.value.schedule
}
#TODO add support for configs
backup_config {
include_volume_data = true
include_secrets = true
all_namespaces = true
}
}
resource "google_compute_network_peering_routes_config" "gke_master" {
count = (
try(var.private_cluster_config.peering_config, null) != null ? 1 : 0
)
project = (
try(var.private_cluster_config.peering_config, null) == null
? var.project_id
: var.private_cluster_config.peering_config.project_id
)
peering = try(
google_container_cluster.cluster.private_cluster_config.0.peering_name,
null
)
network = element(reverse(split("/", var.vpc_config.network)), 0)
import_custom_routes = var.private_cluster_config.peering_config.import_routes
export_custom_routes = var.private_cluster_config.peering_config.export_routes
}
resource "google_pubsub_topic" "notifications" {
count = (
try(var.enable_features.upgrade_notifications, null) != null &&
try(var.enable_features.upgrade_notifications.topic_id, null) == null ? 1 : 0
)
project = var.project_id
name = "gke-pubsub-notifications"
labels = {
content = "gke-notifications"
}
}

View File

@ -0,0 +1,207 @@
/**
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "backup_configs" {
description = "Configuration for Backup for GKE."
type = object({
enable_backup_agent = optional(bool, false)
backup_plans = optional(map(object({
region = string
schedule = string
retention_policy_days = optional(string)
retention_policy_lock = optional(bool, false)
retention_policy_delete_lock_days = optional(string)
})), {})
})
default = {}
nullable = false
}
variable "description" {
description = "Cluster description."
type = string
default = null
}
variable "enable_addons" {
description = "Addons enabled in the cluster (true means enabled)."
type = object({
cloudrun = optional(bool, false)
config_connector = optional(bool, false)
dns_cache = optional(bool, false)
horizontal_pod_autoscaling = optional(bool, false)
http_load_balancing = optional(bool, false)
istio = optional(object({
enable_tls = bool
}))
kalm = optional(bool, false)
network_policy = optional(bool, false)
})
default = {
horizontal_pod_autoscaling = true
http_load_balancing = true
}
nullable = false
}
variable "enable_features" {
description = "Enable cluster-level features. Certain features allow configuration."
type = object({
binary_authorization = optional(bool, false)
dns = optional(object({
provider = optional(string)
scope = optional(string)
domain = optional(string)
}))
database_encryption = optional(object({
state = string
key_name = string
}))
gateway_api = optional(bool, false)
groups_for_rbac = optional(string)
l4_ilb_subsetting = optional(bool, false)
mesh_certificates = optional(bool)
pod_security_policy = optional(bool, false)
resource_usage_export = optional(object({
dataset = string
enable_network_egress_metering = optional(bool)
enable_resource_consumption_metering = optional(bool)
}))
tpu = optional(bool, false)
upgrade_notifications = optional(object({
topic_id = optional(string)
}))
vertical_pod_autoscaling = optional(bool, false)
})
default = {
}
}
variable "issue_client_certificate" {
description = "Enable issuing client certificate."
type = bool
default = false
}
variable "labels" {
description = "Cluster resource labels."
type = map(string)
default = null
}
variable "location" {
description = "Autopilot cluster are always regional."
type = string
}
variable "maintenance_config" {
description = "Maintenance window configuration."
type = object({
daily_window_start_time = optional(string)
recurring_window = optional(object({
start_time = string
end_time = string
recurrence = string
}))
maintenance_exclusions = optional(list(object({
name = string
start_time = string
end_time = string
scope = optional(string)
})))
})
default = {
daily_window_start_time = "03:00"
recurring_window = null
maintenance_exclusion = []
}
}
variable "min_master_version" {
description = "Minimum version of the master, defaults to the version of the most recent official release."
type = string
default = null
}
variable "name" {
description = "Cluster name."
type = string
}
variable "node_locations" {
description = "Zones in which the cluster's nodes are located."
type = list(string)
default = []
nullable = false
}
variable "private_cluster_config" {
description = "Private cluster configuration."
type = object({
enable_private_endpoint = optional(bool)
master_global_access = optional(bool)
peering_config = optional(object({
export_routes = optional(bool)
import_routes = optional(bool)
project_id = optional(string)
}))
})
default = null
}
variable "project_id" {
description = "Cluster project id."
type = string
}
variable "release_channel" {
description = "Release channel for GKE upgrades."
type = string
default = null
}
variable "service_account" {
description = "The Google Cloud Platform Service Account to be used by the node VMs created by GKE Autopilot."
type = string
default = null
}
variable "tags" {
description = "Network tags applied to nodes."
type = list(string)
default = null
}
variable "vpc_config" {
description = "VPC-level configuration."
type = object({
network = string
subnetwork = string
master_ipv4_cidr_block = optional(string)
secondary_range_blocks = optional(object({
pods = string
services = string
}))
secondary_range_names = optional(object({
pods = string
services = string
}), { pods = "pods", services = "services" })
master_authorized_ranges = optional(map(string))
})
nullable = false
}

View File

@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
terraform { terraform {
required_version = ">= 1.4.4" required_version = ">= 1.4.4"
required_providers { required_providers {

View File

@ -1,6 +1,6 @@
# GKE cluster module # GKE cluster Standard module
This module allows simplified creation and management of GKE clusters and should be used together with the GKE nodepool module, as the default nodepool is turned off here and cannot be re-enabled. Some sensible defaults are set initially, in order to allow less verbose usage for most use cases. This module allows simplified creation and management of GKE Standard clusters and should be used together with the GKE nodepool module, as the default nodepool is turned off here and cannot be re-enabled. Some sensible defaults are set initially, in order to allow less verbose usage for most use cases.
## Example ## Example
@ -8,7 +8,7 @@ This module allows simplified creation and management of GKE clusters and should
```hcl ```hcl
module "cluster-1" { module "cluster-1" {
source = "./fabric/modules/gke-cluster" source = "./fabric/modules/gke-cluster-standard"
project_id = "myproject" project_id = "myproject"
name = "cluster-1" name = "cluster-1"
location = "europe-west1-b" location = "europe-west1-b"
@ -40,7 +40,7 @@ module "cluster-1" {
```hcl ```hcl
module "cluster-1" { module "cluster-1" {
source = "./fabric/modules/gke-cluster" source = "./fabric/modules/gke-cluster-standard"
project_id = "myproject" project_id = "myproject"
name = "cluster-dataplane-v2" name = "cluster-dataplane-v2"
location = "europe-west1-b" location = "europe-west1-b"
@ -70,32 +70,6 @@ module "cluster-1" {
} }
# tftest modules=1 resources=1 inventory=dataplane-v2.yaml # tftest modules=1 resources=1 inventory=dataplane-v2.yaml
``` ```
### Autopilot Cluster
```hcl
module "cluster-autopilot" {
source = "./fabric/modules/gke-cluster"
project_id = "myproject"
name = "cluster-autopilot"
location = "europe-west1-b"
vpc_config = {
network = var.vpc.self_link
subnetwork = var.subnet.self_link
secondary_range_names = {
pods = "pods"
services = "services"
}
master_authorized_ranges = {
internal-vms = "10.0.0.0/8"
}
master_ipv4_cidr_block = "192.168.0.0/28"
}
enable_features = {
autopilot = true
}
}
# tftest modules=1 resources=1 inventory=autopilot.yaml
```
### Cloud DNS ### Cloud DNS
@ -103,7 +77,7 @@ This example shows how to [use Cloud DNS as a Kubernetes DNS provider](https://c
```hcl ```hcl
module "cluster-1" { module "cluster-1" {
source = "./fabric/modules/gke-cluster" source = "./fabric/modules/gke-cluster-standard"
project_id = var.project_id project_id = var.project_id
name = "cluster-1" name = "cluster-1"
location = "europe-west1-b" location = "europe-west1-b"
@ -130,7 +104,7 @@ This example shows how to [enable the Backup for GKE agent and configure a Backu
```hcl ```hcl
module "cluster-1" { module "cluster-1" {
source = "./fabric/modules/gke-cluster" source = "./fabric/modules/gke-cluster-standard"
project_id = var.project_id project_id = var.project_id
name = "cluster-1" name = "cluster-1"
location = "europe-west1-b" location = "europe-west1-b"
@ -157,26 +131,26 @@ module "cluster-1" {
| name | description | type | required | default | | name | description | type | required | default |
|---|---|:---:|:---:|:---:| |---|---|:---:|:---:|:---:|
| [location](variables.tf#L134) | Cluster zone or region. | <code>string</code> | ✓ | | | [location](variables.tf#L133) | Cluster zone or region. | <code>string</code> | ✓ | |
| [name](variables.tf#L191) | Cluster name. | <code>string</code> | ✓ | | | [name](variables.tf#L190) | Cluster name. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L217) | Cluster project id. | <code>string</code> | ✓ | | | [project_id](variables.tf#L216) | Cluster project id. | <code>string</code> | ✓ | |
| [vpc_config](variables.tf#L234) | VPC-level configuration. | <code title="object&#40;&#123;&#10; network &#61; string&#10; subnetwork &#61; string&#10; master_ipv4_cidr_block &#61; optional&#40;string&#41;&#10; secondary_range_blocks &#61; optional&#40;object&#40;&#123;&#10; pods &#61; string&#10; services &#61; string&#10; &#125;&#41;&#41;&#10; secondary_range_names &#61; optional&#40;object&#40;&#123;&#10; pods &#61; string&#10; services &#61; string&#10; &#125;&#41;, &#123; pods &#61; &#34;pods&#34;, services &#61; &#34;services&#34; &#125;&#41;&#10; master_authorized_ranges &#61; optional&#40;map&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | | [vpc_config](variables.tf#L233) | VPC-level configuration. | <code title="object&#40;&#123;&#10; network &#61; string&#10; subnetwork &#61; string&#10; master_ipv4_cidr_block &#61; optional&#40;string&#41;&#10; secondary_range_blocks &#61; optional&#40;object&#40;&#123;&#10; pods &#61; string&#10; services &#61; string&#10; &#125;&#41;&#41;&#10; secondary_range_names &#61; optional&#40;object&#40;&#123;&#10; pods &#61; string&#10; services &#61; string&#10; &#125;&#41;, &#123; pods &#61; &#34;pods&#34;, services &#61; &#34;services&#34; &#125;&#41;&#10; master_authorized_ranges &#61; optional&#40;map&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | |
| [backup_configs](variables.tf#L17) | Configuration for Backup for GKE. | <code title="object&#40;&#123;&#10; enable_backup_agent &#61; optional&#40;bool, false&#41;&#10; backup_plans &#61; optional&#40;map&#40;object&#40;&#123;&#10; region &#61; string&#10; schedule &#61; string&#10; retention_policy_days &#61; optional&#40;string&#41;&#10; retention_policy_lock &#61; optional&#40;bool, false&#41;&#10; retention_policy_delete_lock_days &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | | [backup_configs](variables.tf#L17) | Configuration for Backup for GKE. | <code title="object&#40;&#123;&#10; enable_backup_agent &#61; optional&#40;bool, false&#41;&#10; backup_plans &#61; optional&#40;map&#40;object&#40;&#123;&#10; region &#61; string&#10; schedule &#61; string&#10; retention_policy_days &#61; optional&#40;string&#41;&#10; retention_policy_lock &#61; optional&#40;bool, false&#41;&#10; retention_policy_delete_lock_days &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [cluster_autoscaling](variables.tf#L33) | Enable and configure limits for Node Auto-Provisioning with Cluster Autoscaler. | <code title="object&#40;&#123;&#10; auto_provisioning_defaults &#61; optional&#40;object&#40;&#123;&#10; boot_disk_kms_key &#61; optional&#40;string&#41;&#10; image_type &#61; optional&#40;string&#41;&#10; oauth_scopes &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_account &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; cpu_limits &#61; optional&#40;object&#40;&#123;&#10; min &#61; number&#10; max &#61; number&#10; &#125;&#41;&#41;&#10; mem_limits &#61; optional&#40;object&#40;&#123;&#10; min &#61; number&#10; max &#61; number&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | | [cluster_autoscaling](variables.tf#L33) | Enable and configure limits for Node Auto-Provisioning with Cluster Autoscaler. | <code title="object&#40;&#123;&#10; auto_provisioning_defaults &#61; optional&#40;object&#40;&#123;&#10; boot_disk_kms_key &#61; optional&#40;string&#41;&#10; image_type &#61; optional&#40;string&#41;&#10; oauth_scopes &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_account &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; cpu_limits &#61; optional&#40;object&#40;&#123;&#10; min &#61; number&#10; max &#61; number&#10; &#125;&#41;&#41;&#10; mem_limits &#61; optional&#40;object&#40;&#123;&#10; min &#61; number&#10; max &#61; number&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [description](variables.tf#L54) | Cluster description. | <code>string</code> | | <code>null</code> | | [description](variables.tf#L54) | Cluster description. | <code>string</code> | | <code>null</code> |
| [enable_addons](variables.tf#L60) | Addons enabled in the cluster (true means enabled). | <code title="object&#40;&#123;&#10; cloudrun &#61; optional&#40;bool, false&#41;&#10; config_connector &#61; optional&#40;bool, false&#41;&#10; dns_cache &#61; optional&#40;bool, false&#41;&#10; gce_persistent_disk_csi_driver &#61; optional&#40;bool, false&#41;&#10; gcp_filestore_csi_driver &#61; optional&#40;bool, false&#41;&#10; horizontal_pod_autoscaling &#61; optional&#40;bool, false&#41;&#10; http_load_balancing &#61; optional&#40;bool, false&#41;&#10; istio &#61; optional&#40;object&#40;&#123;&#10; enable_tls &#61; bool&#10; &#125;&#41;&#41;&#10; kalm &#61; optional&#40;bool, false&#41;&#10; network_policy &#61; optional&#40;bool, false&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; horizontal_pod_autoscaling &#61; true&#10; http_load_balancing &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> | | [enable_addons](variables.tf#L60) | Addons enabled in the cluster (true means enabled). | <code title="object&#40;&#123;&#10; cloudrun &#61; optional&#40;bool, false&#41;&#10; config_connector &#61; optional&#40;bool, false&#41;&#10; dns_cache &#61; optional&#40;bool, false&#41;&#10; gce_persistent_disk_csi_driver &#61; optional&#40;bool, false&#41;&#10; gcp_filestore_csi_driver &#61; optional&#40;bool, false&#41;&#10; horizontal_pod_autoscaling &#61; optional&#40;bool, false&#41;&#10; http_load_balancing &#61; optional&#40;bool, false&#41;&#10; istio &#61; optional&#40;object&#40;&#123;&#10; enable_tls &#61; bool&#10; &#125;&#41;&#41;&#10; kalm &#61; optional&#40;bool, false&#41;&#10; network_policy &#61; optional&#40;bool, false&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; horizontal_pod_autoscaling &#61; true&#10; http_load_balancing &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> |
| [enable_features](variables.tf#L83) | Enable cluster-level features. Certain features allow configuration. | <code title="object&#40;&#123;&#10; autopilot &#61; optional&#40;bool, false&#41;&#10; binary_authorization &#61; optional&#40;bool, false&#41;&#10; dns &#61; optional&#40;object&#40;&#123;&#10; provider &#61; optional&#40;string&#41;&#10; scope &#61; optional&#40;string&#41;&#10; domain &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; database_encryption &#61; optional&#40;object&#40;&#123;&#10; state &#61; string&#10; key_name &#61; string&#10; &#125;&#41;&#41;&#10; dataplane_v2 &#61; optional&#40;bool, false&#41;&#10; gateway_api &#61; optional&#40;bool, false&#41;&#10; groups_for_rbac &#61; optional&#40;string&#41;&#10; intranode_visibility &#61; optional&#40;bool, false&#41;&#10; l4_ilb_subsetting &#61; optional&#40;bool, false&#41;&#10; mesh_certificates &#61; optional&#40;bool&#41;&#10; pod_security_policy &#61; optional&#40;bool, false&#41;&#10; resource_usage_export &#61; optional&#40;object&#40;&#123;&#10; dataset &#61; string&#10; enable_network_egress_metering &#61; optional&#40;bool&#41;&#10; enable_resource_consumption_metering &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; shielded_nodes &#61; optional&#40;bool, false&#41;&#10; tpu &#61; optional&#40;bool, false&#41;&#10; upgrade_notifications &#61; optional&#40;object&#40;&#123;&#10; topic_id &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; vertical_pod_autoscaling &#61; optional&#40;bool, false&#41;&#10; workload_identity &#61; optional&#40;bool, true&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; workload_identity &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> | | [enable_features](variables.tf#L83) | Enable cluster-level features. Certain features allow configuration. | <code title="object&#40;&#123;&#10; binary_authorization &#61; optional&#40;bool, false&#41;&#10; dns &#61; optional&#40;object&#40;&#123;&#10; provider &#61; optional&#40;string&#41;&#10; scope &#61; optional&#40;string&#41;&#10; domain &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; database_encryption &#61; optional&#40;object&#40;&#123;&#10; state &#61; string&#10; key_name &#61; string&#10; &#125;&#41;&#41;&#10; dataplane_v2 &#61; optional&#40;bool, false&#41;&#10; gateway_api &#61; optional&#40;bool, false&#41;&#10; groups_for_rbac &#61; optional&#40;string&#41;&#10; intranode_visibility &#61; optional&#40;bool, false&#41;&#10; l4_ilb_subsetting &#61; optional&#40;bool, false&#41;&#10; mesh_certificates &#61; optional&#40;bool&#41;&#10; pod_security_policy &#61; optional&#40;bool, false&#41;&#10; resource_usage_export &#61; optional&#40;object&#40;&#123;&#10; dataset &#61; string&#10; enable_network_egress_metering &#61; optional&#40;bool&#41;&#10; enable_resource_consumption_metering &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#10; shielded_nodes &#61; optional&#40;bool, false&#41;&#10; tpu &#61; optional&#40;bool, false&#41;&#10; upgrade_notifications &#61; optional&#40;object&#40;&#123;&#10; topic_id &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; vertical_pod_autoscaling &#61; optional&#40;bool, false&#41;&#10; workload_identity &#61; optional&#40;bool, true&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; workload_identity &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> |
| [issue_client_certificate](variables.tf#L122) | Enable issuing client certificate. | <code>bool</code> | | <code>false</code> | | [issue_client_certificate](variables.tf#L121) | Enable issuing client certificate. | <code>bool</code> | | <code>false</code> |
| [labels](variables.tf#L128) | Cluster resource labels. | <code>map&#40;string&#41;</code> | | <code>null</code> | | [labels](variables.tf#L127) | Cluster resource labels. | <code>map&#40;string&#41;</code> | | <code>null</code> |
| [logging_config](variables.tf#L139) | Logging configuration. | <code>list&#40;string&#41;</code> | | <code>&#91;&#34;SYSTEM_COMPONENTS&#34;&#93;</code> | | [logging_config](variables.tf#L138) | Logging configuration. | <code>list&#40;string&#41;</code> | | <code>&#91;&#34;SYSTEM_COMPONENTS&#34;&#93;</code> |
| [maintenance_config](variables.tf#L145) | Maintenance window configuration. | <code title="object&#40;&#123;&#10; daily_window_start_time &#61; optional&#40;string&#41;&#10; recurring_window &#61; optional&#40;object&#40;&#123;&#10; start_time &#61; string&#10; end_time &#61; string&#10; recurrence &#61; string&#10; &#125;&#41;&#41;&#10; maintenance_exclusions &#61; optional&#40;list&#40;object&#40;&#123;&#10; name &#61; string&#10; start_time &#61; string&#10; end_time &#61; string&#10; scope &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; daily_window_start_time &#61; &#34;03:00&#34;&#10; recurring_window &#61; null&#10; maintenance_exclusion &#61; &#91;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> | | [maintenance_config](variables.tf#L144) | Maintenance window configuration. | <code title="object&#40;&#123;&#10; daily_window_start_time &#61; optional&#40;string&#41;&#10; recurring_window &#61; optional&#40;object&#40;&#123;&#10; start_time &#61; string&#10; end_time &#61; string&#10; recurrence &#61; string&#10; &#125;&#41;&#41;&#10; maintenance_exclusions &#61; optional&#40;list&#40;object&#40;&#123;&#10; name &#61; string&#10; start_time &#61; string&#10; end_time &#61; string&#10; scope &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; daily_window_start_time &#61; &#34;03:00&#34;&#10; recurring_window &#61; null&#10; maintenance_exclusion &#61; &#91;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> |
| [max_pods_per_node](variables.tf#L168) | Maximum number of pods per node in this cluster. | <code>number</code> | | <code>110</code> | | [max_pods_per_node](variables.tf#L167) | Maximum number of pods per node in this cluster. | <code>number</code> | | <code>110</code> |
| [min_master_version](variables.tf#L174) | Minimum version of the master, defaults to the version of the most recent official release. | <code>string</code> | | <code>null</code> | | [min_master_version](variables.tf#L173) | Minimum version of the master, defaults to the version of the most recent official release. | <code>string</code> | | <code>null</code> |
| [monitoring_config](variables.tf#L180) | Monitoring components. | <code title="object&#40;&#123;&#10; enable_components &#61; optional&#40;list&#40;string&#41;&#41;&#10; managed_prometheus &#61; optional&#40;bool&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; enable_components &#61; &#91;&#34;SYSTEM_COMPONENTS&#34;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> | | [monitoring_config](variables.tf#L179) | Monitoring components. | <code title="object&#40;&#123;&#10; enable_components &#61; optional&#40;list&#40;string&#41;&#41;&#10; managed_prometheus &#61; optional&#40;bool&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; enable_components &#61; &#91;&#34;SYSTEM_COMPONENTS&#34;&#93;&#10;&#125;">&#123;&#8230;&#125;</code> |
| [node_locations](variables.tf#L196) | Zones in which the cluster's nodes are located. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> | | [node_locations](variables.tf#L195) | Zones in which the cluster's nodes are located. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |
| [private_cluster_config](variables.tf#L203) | Private cluster configuration. | <code title="object&#40;&#123;&#10; enable_private_endpoint &#61; optional&#40;bool&#41;&#10; master_global_access &#61; optional&#40;bool&#41;&#10; peering_config &#61; optional&#40;object&#40;&#123;&#10; export_routes &#61; optional&#40;bool&#41;&#10; import_routes &#61; optional&#40;bool&#41;&#10; project_id &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | | [private_cluster_config](variables.tf#L202) | Private cluster configuration. | <code title="object&#40;&#123;&#10; enable_private_endpoint &#61; optional&#40;bool&#41;&#10; master_global_access &#61; optional&#40;bool&#41;&#10; peering_config &#61; optional&#40;object&#40;&#123;&#10; export_routes &#61; optional&#40;bool&#41;&#10; import_routes &#61; optional&#40;bool&#41;&#10; project_id &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [release_channel](variables.tf#L222) | Release channel for GKE upgrades. | <code>string</code> | | <code>null</code> | | [release_channel](variables.tf#L221) | Release channel for GKE upgrades. | <code>string</code> | | <code>null</code> |
| [tags](variables.tf#L228) | Network tags applied to nodes. | <code>list&#40;string&#41;</code> | | <code>null</code> | | [tags](variables.tf#L227) | Network tags applied to nodes. | <code>list&#40;string&#41;</code> | | <code>null</code> |
## Outputs ## Outputs

View File

@ -15,12 +15,6 @@
*/ */
resource "google_container_cluster" "cluster" { resource "google_container_cluster" "cluster" {
lifecycle {
ignore_changes = [
node_config[0].boot_disk_kms_key,
node_config[0].spot
]
}
provider = google-beta provider = google-beta
project = var.project_id project = var.project_id
name = var.name name = var.name
@ -29,54 +23,39 @@ resource "google_container_cluster" "cluster" {
node_locations = ( node_locations = (
length(var.node_locations) == 0 ? null : var.node_locations length(var.node_locations) == 0 ? null : var.node_locations
) )
min_master_version = var.min_master_version min_master_version = var.min_master_version
network = var.vpc_config.network network = var.vpc_config.network
subnetwork = var.vpc_config.subnetwork subnetwork = var.vpc_config.subnetwork
resource_labels = var.labels resource_labels = var.labels
default_max_pods_per_node = ( default_max_pods_per_node = var.max_pods_per_node
var.enable_features.autopilot ? null : var.max_pods_per_node enable_intranode_visibility = var.enable_features.intranode_visibility
) enable_l4_ilb_subsetting = var.enable_features.l4_ilb_subsetting
enable_intranode_visibility = ( enable_shielded_nodes = var.enable_features.shielded_nodes
var.enable_features.autopilot ? null : var.enable_features.intranode_visibility enable_tpu = var.enable_features.tpu
) initial_node_count = 1
enable_l4_ilb_subsetting = var.enable_features.l4_ilb_subsetting remove_default_node_pool = true
enable_shielded_nodes = (
var.enable_features.autopilot ? null : var.enable_features.shielded_nodes
)
enable_tpu = var.enable_features.tpu
initial_node_count = 1
remove_default_node_pool = var.enable_features.autopilot ? null : true
datapath_provider = ( datapath_provider = (
var.enable_features.dataplane_v2 || var.enable_features.autopilot var.enable_features.dataplane_v2
? "ADVANCED_DATAPATH" ? "ADVANCED_DATAPATH"
: "DATAPATH_PROVIDER_UNSPECIFIED" : "DATAPATH_PROVIDER_UNSPECIFIED"
) )
enable_autopilot = var.enable_features.autopilot ? true : null
# the default nodepool is deleted here, use the gke-nodepool module instead # the default nodepool is deleted here, use the gke-nodepool module instead
# default nodepool configuration based on a shielded_nodes variable # default nodepool configuration based on a shielded_nodes variable
dynamic "node_config" { node_config {
for_each = var.enable_features.autopilot ? [] : [""] dynamic "shielded_instance_config" {
content { for_each = var.enable_features.shielded_nodes ? [""] : []
dynamic "shielded_instance_config" { content {
for_each = var.enable_features.shielded_nodes ? [""] : [] enable_secure_boot = true
content { enable_integrity_monitoring = true
enable_secure_boot = true
enable_integrity_monitoring = true
}
} }
tags = var.tags
} }
tags = var.tags
} }
addons_config { addons_config {
dynamic "dns_cache_config" { dns_cache_config {
for_each = !var.enable_features.autopilot ? [""] : [] enabled = var.enable_addons.dns_cache
content {
enabled = var.enable_addons.dns_cache
}
} }
http_load_balancing { http_load_balancing {
disabled = !var.enable_addons.http_load_balancing disabled = !var.enable_addons.http_load_balancing
@ -84,11 +63,8 @@ resource "google_container_cluster" "cluster" {
horizontal_pod_autoscaling { horizontal_pod_autoscaling {
disabled = !var.enable_addons.horizontal_pod_autoscaling disabled = !var.enable_addons.horizontal_pod_autoscaling
} }
dynamic "network_policy_config" { network_policy_config {
for_each = !var.enable_features.autopilot ? [""] : [] disabled = !var.enable_addons.network_policy
content {
disabled = !var.enable_addons.network_policy
}
} }
cloudrun_config { cloudrun_config {
disabled = !var.enable_addons.cloudrun disabled = !var.enable_addons.cloudrun
@ -100,17 +76,10 @@ resource "google_container_cluster" "cluster" {
) )
} }
gce_persistent_disk_csi_driver_config { gce_persistent_disk_csi_driver_config {
enabled = ( enabled = var.enable_addons.gce_persistent_disk_csi_driver
var.enable_features.autopilot
? true
: var.enable_addons.gce_persistent_disk_csi_driver
)
} }
dynamic "gcp_filestore_csi_driver_config" { gcp_filestore_csi_driver_config {
for_each = !var.enable_features.autopilot ? [""] : [] enabled = var.enable_addons.gcp_filestore_csi_driver
content {
enabled = var.enable_addons.gcp_filestore_csi_driver
}
} }
kalm_config { kalm_config {
enabled = var.enable_addons.kalm enabled = var.enable_addons.kalm
@ -140,7 +109,7 @@ resource "google_container_cluster" "cluster" {
dynamic "cluster_autoscaling" { dynamic "cluster_autoscaling" {
for_each = var.cluster_autoscaling == null ? [] : [""] for_each = var.cluster_autoscaling == null ? [] : [""]
content { content {
enabled = var.enable_features.autopilot ? null : true enabled = true
dynamic "auto_provisioning_defaults" { dynamic "auto_provisioning_defaults" {
for_each = var.cluster_autoscaling.auto_provisioning_defaults != null ? [""] : [] for_each = var.cluster_autoscaling.auto_provisioning_defaults != null ? [""] : []
@ -204,7 +173,7 @@ resource "google_container_cluster" "cluster" {
} }
dynamic "logging_config" { dynamic "logging_config" {
for_each = var.logging_config != null && !var.enable_features.autopilot ? [""] : [] for_each = var.logging_config != null ? [""] : []
content { content {
enable_components = var.logging_config enable_components = var.logging_config
} }
@ -283,7 +252,7 @@ resource "google_container_cluster" "cluster" {
} }
dynamic "monitoring_config" { dynamic "monitoring_config" {
for_each = var.monitoring_config != null && !var.enable_features.autopilot ? [""] : [] for_each = var.monitoring_config != null ? [""] : []
content { content {
enable_components = var.monitoring_config.enable_components enable_components = var.monitoring_config.enable_components
dynamic "managed_prometheus" { dynamic "managed_prometheus" {
@ -379,11 +348,17 @@ resource "google_container_cluster" "cluster" {
} }
dynamic "workload_identity_config" { dynamic "workload_identity_config" {
for_each = (var.enable_features.workload_identity && !var.enable_features.autopilot) ? [""] : [] for_each = var.enable_features.workload_identity ? [""] : []
content { content {
workload_pool = "${var.project_id}.svc.id.goog" workload_pool = "${var.project_id}.svc.id.goog"
} }
} }
lifecycle {
ignore_changes = [
node_config[0].boot_disk_kms_key,
node_config[0].spot
]
}
} }
resource "google_gke_backup_backup_plan" "backup_plan" { resource "google_gke_backup_backup_plan" "backup_plan" {

View File

@ -0,0 +1,71 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "ca_certificate" {
description = "Public certificate of the cluster (base64-encoded)."
value = google_container_cluster.cluster.master_auth.0.cluster_ca_certificate
sensitive = true
}
output "cluster" {
description = "Cluster resource."
sensitive = true
value = google_container_cluster.cluster
}
output "endpoint" {
description = "Cluster endpoint."
value = google_container_cluster.cluster.endpoint
}
output "id" {
description = "Cluster ID."
value = google_container_cluster.cluster.id
}
output "location" {
description = "Cluster location."
value = google_container_cluster.cluster.location
}
output "master_version" {
description = "Master version."
value = google_container_cluster.cluster.master_version
}
output "name" {
description = "Cluster name."
value = google_container_cluster.cluster.name
}
output "notifications" {
description = "GKE PubSub notifications topic."
value = try(google_pubsub_topic.notifications[0].id, null)
}
output "self_link" {
description = "Cluster self link."
sensitive = true
value = google_container_cluster.cluster.self_link
}
output "workload_identity_pool" {
description = "Workload identity pool."
value = "${var.project_id}.svc.id.goog"
depends_on = [
google_container_cluster.cluster
]
}

View File

@ -83,7 +83,6 @@ variable "enable_addons" {
variable "enable_features" { variable "enable_features" {
description = "Enable cluster-level features. Certain features allow configuration." description = "Enable cluster-level features. Certain features allow configuration."
type = object({ type = object({
autopilot = optional(bool, false)
binary_authorization = optional(bool, false) binary_authorization = optional(bool, false)
dns = optional(object({ dns = optional(object({
provider = optional(string) provider = optional(string)

View File

@ -0,0 +1,31 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_version = ">= 1.4.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 4.60.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 4.60.0" # tftest
}
}
}

View File

@ -46,7 +46,7 @@ module "vpc" {
} }
module "cluster_1" { module "cluster_1" {
source = "./fabric/modules/gke-cluster" source = "./fabric/modules/gke-cluster-standard"
project_id = module.project.project_id project_id = module.project.project_id
name = "cluster-1" name = "cluster-1"
location = "europe-west1" location = "europe-west1"
@ -212,7 +212,7 @@ module "firewall" {
} }
module "cluster_1" { module "cluster_1" {
source = "./fabric/modules/gke-cluster" source = "./fabric/modules/gke-cluster-standard"
project_id = module.project.project_id project_id = module.project.project_id
name = "cluster-1" name = "cluster-1"
location = "europe-west1" location = "europe-west1"
@ -253,7 +253,7 @@ module "cluster_1_nodepool" {
} }
module "cluster_2" { module "cluster_2" {
source = "./fabric/modules/gke-cluster" source = "./fabric/modules/gke-cluster-standard"
project_id = module.project.project_id project_id = module.project.project_id
name = "cluster-2" name = "cluster-2"
location = "europe-west4" location = "europe-west4"

View File

@ -0,0 +1,38 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
values:
module.cluster-1.google_container_cluster.cluster:
location: europe-west1
name: cluster-1
module.cluster-1.google_gke_backup_backup_plan.backup_plan["backup-1"]:
backup_config:
- all_namespaces: true
encryption_key: []
include_secrets: true
include_volume_data: true
selected_applications: []
selected_namespaces: []
backup_schedule:
- cron_schedule: 0 9 * * 1
location: europe-west-2
name: backup-1
project: project-id
retention_policy:
- locked: false
counts:
google_container_cluster: 1
google_gke_backup_backup_plan: 1

View File

@ -0,0 +1,28 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
values:
module.cluster-1.google_container_cluster.cluster:
private_cluster_config:
- enable_private_endpoint: true
enable_private_nodes: true
master_global_access_config:
- enabled: false
master_ipv4_cidr_block: 192.168.0.0/28
private_endpoint_subnetwork: null
resource_labels:
environment: dev
counts:
google_container_cluster: 1

View File

@ -13,8 +13,11 @@
# limitations under the License. # limitations under the License.
values: values:
module.cluster-autopilot.google_container_cluster.cluster: module.cluster-1.google_container_cluster.cluster:
enable_autopilot: true dns_config:
- cluster_dns: CLOUD_DNS
cluster_dns_domain: gke.local
cluster_dns_scope: CLUSTER_SCOPE
counts: counts:
google_container_cluster: 1 google_container_cluster: 1