Separating GKE Standard and Autopilot Modules (#1330)
* separating GKE Standard and Autopilot Modules * Changes for Updating the terraform and provide versions * Changes for Autopilot Readme * Changes for Autopilot Variable * Changes for Autopilot Readme * Changes for Autopilot Readme * Changes for Blueprint * Changes for Blueprint ReadMe * Changes for gke-standard-cluster dependency * Changes for gke-standard-cluster in gke-fleet * Changes for gke-standard-cluster in cluster-mesh-gke-fleet-api * python formatting * python formatting * python formatting * GKE module naming convention * Readme Changes * test module * Removing comment code from Autopilot
This commit is contained in:
Avinash Jha
GitHub
parent
df8c61fe69
commit
e881537f87
|
|
@ -161,4 +161,4 @@ Even with all the above points, it may be hard to make a decision. While the mod
|
|||
|
||||
* Since modules work well together within their ecosystem, select logical boundaries for using Fabric or CFT. For example use CFT for deploying resources within projects but use Fabric for managing project creation and IAM.
|
||||
* Use strengths of each collection of modules to your advantage. Empower application teams to define their infrastructure as code using off the shelf CFT modules. Using Fabric, bootstrap your platform team with a collection of tailor built modules for your organization.
|
||||
* Lean into module composition and dependency inversion that both Fabric and CFT modules follow. For example, you can create a GKE cluster using either [Fabric](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/gke-cluster#gke-cluster-module) or [CFT](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine) GKE module and then use either [Fabric](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/gke-hub#variables) or [CFT](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/tree/master/modules/fleet-membership) for setting up GKE Hub by passing in outputs from the GKE module.
|
||||
* Lean into module composition and dependency inversion that both Fabric and CFT modules follow. For example, you can create a GKE cluster using either [Fabric](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/gke-cluster-standard#gke-cluster-module) or [CFT](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine) GKE module and then use either [Fabric](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/modules/gke-hub#variables) or [CFT](https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/tree/master/modules/fleet-membership) for setting up GKE Hub by passing in outputs from the GKE module.
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ Currently available modules:
|
|||
|
||||
- **foundational** - [billing budget](./modules/billing-budget), [Cloud Identity group](./modules/cloud-identity-group/), [folder](./modules/folder), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [organization](./modules/organization), [project](./modules/project), [projects-data-source](./modules/projects-data-source)
|
||||
- **networking** - [DNS](./modules/dns), [DNS Response Policy](./modules/dns-response-policy/), [Cloud Endpoints](./modules/endpoints), [address reservation](./modules/net-address), [NAT](./modules/net-cloudnat), [Global Load Balancer (classic)](./modules/net-glb/), [L4 ILB](./modules/net-ilb), [L7 ILB](./modules/net-ilb-l7), [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC firewall policy](./modules/net-vpc-firewall-policy), [VPC peering](./modules/net-vpc-peering), [VPN dynamic](./modules/net-vpn-dynamic), [HA VPN](./modules/net-vpn-ha), [VPN static](./modules/net-vpn-static), [Service Directory](./modules/service-directory)
|
||||
- **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid), [GKE cluster](./modules/gke-cluster), [GKE hub](./modules/gke-hub), [GKE nodepool](./modules/gke-nodepool)
|
||||
- **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid), [GKE cluster](./modules/gke-cluster-standard), [GKE hub](./modules/gke-hub), [GKE nodepool](./modules/gke-nodepool)
|
||||
- **data** - [BigQuery dataset](./modules/bigquery-dataset), [Bigtable instance](./modules/bigtable-instance), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag), [Datafusion](./modules/datafusion), [Dataproc](./modules/dataproc), [GCS](./modules/gcs), [Pub/Sub](./modules/pubsub)
|
||||
- **development** - [API Gateway](./modules/api-gateway), [Apigee](./modules/apigee), [Artifact Registry](./modules/artifact-registry), [Container Registry](./modules/container-registry), [Cloud Source Repository](./modules/source-repository)
|
||||
- **security** - [Binauthz](./modules/binauthz/), [KMS](./modules/kms), [SecretManager](./modules/secret-manager), [VPC Service Control](./modules/vpc-sc)
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@
|
|||
*/
|
||||
|
||||
module "cluster" {
|
||||
source = "../../../modules/gke-cluster"
|
||||
source = "../../../modules/gke-cluster-standard"
|
||||
project_id = module.project.project_id
|
||||
name = "cluster"
|
||||
location = var.region
|
||||
|
|
|
|||
|
|
@ -80,8 +80,9 @@ def do_discovery(resources):
|
|||
resources[result.type][result.id][result.key] = result.data
|
||||
else:
|
||||
resources[result.type][result.id] = result.data
|
||||
LOGGER.info('discovery end {}'.format(
|
||||
{k: len(v) for k, v in resources.items() if not isinstance(v, str)}))
|
||||
LOGGER.info('discovery end {}'.format({
|
||||
k: len(v) for k, v in resources.items() if not isinstance(v, str)
|
||||
}))
|
||||
|
||||
|
||||
def do_init(resources, discovery_root, monitoring_project, folders=None,
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@
|
|||
*/
|
||||
|
||||
module "cluster" {
|
||||
source = "../../../modules/gke-cluster"
|
||||
source = "../../../modules/gke-cluster-autopilot"
|
||||
project_id = module.project.project_id
|
||||
name = "cluster"
|
||||
location = var.region
|
||||
|
|
@ -29,18 +29,18 @@ module "cluster" {
|
|||
master_authorized_ranges = var.cluster_network_config.master_authorized_cidr_blocks
|
||||
master_ipv4_cidr_block = var.cluster_network_config.master_cidr_block
|
||||
}
|
||||
enable_features = {
|
||||
autopilot = true
|
||||
}
|
||||
monitoring_config = {
|
||||
enenable_components = ["SYSTEM_COMPONENTS"]
|
||||
managed_prometheus = true
|
||||
}
|
||||
cluster_autoscaling = {
|
||||
auto_provisioning_defaults = {
|
||||
service_account = module.node_sa.email
|
||||
}
|
||||
}
|
||||
# enable_features = {
|
||||
# autopilot = true
|
||||
# }
|
||||
# monitoring_config = {
|
||||
# enenable_components = ["SYSTEM_COMPONENTS"]
|
||||
# managed_prometheus = true
|
||||
# }
|
||||
# cluster_autoscaling = {
|
||||
# auto_provisioning_defaults = {
|
||||
# service_account = module.node_sa.email
|
||||
# }
|
||||
# }
|
||||
release_channel = "RAPID"
|
||||
depends_on = [
|
||||
module.project
|
||||
|
|
|
|||
|
|
@ -83,7 +83,7 @@ module "nat" {
|
|||
}
|
||||
|
||||
module "cluster" {
|
||||
source = "../../../modules/gke-cluster"
|
||||
source = "../../../modules/gke-cluster-standard"
|
||||
project_id = module.project.project_id
|
||||
name = "${var.prefix}-cluster"
|
||||
location = var.zone
|
||||
|
|
|
|||
|
|
@ -53,7 +53,7 @@ Once done testing, you can clean up resources by running `terraform destroy`.
|
|||
| name | description | modules | resources |
|
||||
|---|---|---|---|
|
||||
| [ansible.tf](./ansible.tf) | Ansible generated files. | | <code>local_file</code> |
|
||||
| [gke.tf](./gke.tf) | GKE cluster and hub resources. | <code>gke-cluster</code> · <code>gke-hub</code> · <code>gke-nodepool</code> | |
|
||||
| [gke.tf](./gke.tf) | GKE cluster and hub resources. | <code>gke-cluster-standard</code> · <code>gke-hub</code> · <code>gke-nodepool</code> | |
|
||||
| [main.tf](./main.tf) | Project resources. | <code>project</code> | |
|
||||
| [variables.tf](./variables.tf) | Module variables. | | |
|
||||
| [vm.tf](./vm.tf) | Management server. | <code>compute-vm</code> | |
|
||||
|
|
@ -75,7 +75,6 @@ Once done testing, you can clean up resources by running `terraform destroy`.
|
|||
| [region](variables.tf#L99) | Region. | <code>string</code> | | <code>"europe-west1"</code> |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
||||
## Test
|
||||
|
||||
```hcl
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@
|
|||
|
||||
module "clusters" {
|
||||
for_each = var.clusters_config
|
||||
source = "../../../modules/gke-cluster"
|
||||
source = "../../../modules/gke-cluster-standard"
|
||||
project_id = module.fleet_project.project_id
|
||||
name = each.key
|
||||
location = var.region
|
||||
|
|
|
|||
|
|
@ -234,7 +234,7 @@ module "gke" {
|
|||
|
||||
| name | description | modules |
|
||||
|---|---|---|
|
||||
| [gke-clusters.tf](./gke-clusters.tf) | GKE clusters. | <code>gke-cluster</code> |
|
||||
| [gke-clusters.tf](./gke-clusters.tf) | GKE clusters. | <code>gke-cluster-standard</code> |
|
||||
| [gke-hub.tf](./gke-hub.tf) | GKE hub configuration. | <code>gke-hub</code> |
|
||||
| [gke-nodepools.tf](./gke-nodepools.tf) | GKE nodepools. | <code>gke-nodepool</code> |
|
||||
| [main.tf](./main.tf) | Project and usage dataset. | <code>bigquery-dataset</code> · <code>project</code> |
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@
|
|||
# tfdoc:file:description GKE clusters.
|
||||
|
||||
module "gke-cluster" {
|
||||
source = "../../../modules/gke-cluster"
|
||||
source = "../../../modules/gke-cluster-standard"
|
||||
for_each = var.clusters
|
||||
name = each.key
|
||||
project_id = module.gke-project-0.project_id
|
||||
|
|
|
|||
|
|
@ -240,7 +240,7 @@ module "service-account-gce" {
|
|||
################################################################################
|
||||
|
||||
module "cluster-1" {
|
||||
source = "../../../modules/gke-cluster"
|
||||
source = "../../../modules/gke-cluster-standard"
|
||||
name = "${var.prefix}-cluster-1"
|
||||
project_id = module.project.project_id
|
||||
location = "${var.region}-b"
|
||||
|
|
|
|||
|
|
@ -197,7 +197,7 @@ module "vm-bastion" {
|
|||
################################################################################
|
||||
|
||||
module "cluster-1" {
|
||||
source = "../../../modules/gke-cluster"
|
||||
source = "../../../modules/gke-cluster-standard"
|
||||
count = var.cluster_create ? 1 : 0
|
||||
name = "cluster-1"
|
||||
project_id = module.project-svc-gke.project_id
|
||||
|
|
|
|||
|
|
@ -63,7 +63,8 @@ These modules are used in the examples included in this repository. If you are u
|
|||
- [VM/VM group](./compute-vm)
|
||||
- [MIG](./compute-mig)
|
||||
- [COS container](./cloud-config-container/cos-generic-metadata/) (coredns/mysql/nva/onprem/squid)
|
||||
- [GKE cluster](./gke-cluster)
|
||||
- [GKE autopilot cluster](./gke-cluster-autopilot)
|
||||
- [GKE standard cluster](./gke-cluster-standard)
|
||||
- [GKE hub](./gke-hub)
|
||||
- [GKE nodepool](./gke-nodepool)
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,132 @@
|
|||
# GKE cluster Autopilot module
|
||||
|
||||
This module allows simplified creation and management of GKE Autopilot clusters. Some sensible defaults are set initially, in order to allow less verbose usage for most use cases.
|
||||
|
||||
## Example
|
||||
|
||||
### GKE Cluster
|
||||
|
||||
```hcl
|
||||
module "cluster-1" {
|
||||
source = "./fabric/modules/gke-cluster-autopilot"
|
||||
project_id = "myproject"
|
||||
name = "cluster-1"
|
||||
location = "europe-west1"
|
||||
vpc_config = {
|
||||
network = var.vpc.self_link
|
||||
subnetwork = var.subnet.self_link
|
||||
secondary_range_names = {
|
||||
pods = "pods"
|
||||
services = "services"
|
||||
}
|
||||
master_authorized_ranges = {
|
||||
internal-vms = "10.0.0.0/8"
|
||||
}
|
||||
master_ipv4_cidr_block = "192.168.0.0/28"
|
||||
}
|
||||
private_cluster_config = {
|
||||
enable_private_endpoint = true
|
||||
master_global_access = false
|
||||
}
|
||||
labels = {
|
||||
environment = "dev"
|
||||
}
|
||||
}
|
||||
# tftest modules=1 resources=1 inventory=basic.yaml
|
||||
```
|
||||
|
||||
|
||||
### Cloud DNS
|
||||
|
||||
This example shows how to [use Cloud DNS as a Kubernetes DNS provider](https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-dns) for GKE Standard clusters.
|
||||
|
||||
```hcl
|
||||
module "cluster-1" {
|
||||
source = "./fabric/modules/gke-cluster-autopilot"
|
||||
project_id = var.project_id
|
||||
name = "cluster-1"
|
||||
location = "europe-west1"
|
||||
vpc_config = {
|
||||
network = var.vpc.self_link
|
||||
subnetwork = var.subnet.self_link
|
||||
secondary_range_names = { pods = "pods", services = "services" }
|
||||
}
|
||||
enable_features = {
|
||||
dns = {
|
||||
provider = "CLOUD_DNS"
|
||||
scope = "CLUSTER_SCOPE"
|
||||
domain = "gke.local"
|
||||
}
|
||||
}
|
||||
}
|
||||
# tftest modules=1 resources=1 inventory=dns.yaml
|
||||
```
|
||||
|
||||
|
||||
### Backup for GKE
|
||||
|
||||
This example shows how to [enable the Backup for GKE agent and configure a Backup Plan](https://cloud.google.com/kubernetes-engine/docs/add-on/backup-for-gke/concepts/backup-for-gke) for GKE Standard clusters.
|
||||
|
||||
```hcl
|
||||
module "cluster-1" {
|
||||
source = "./fabric/modules/gke-cluster-autopilot"
|
||||
project_id = var.project_id
|
||||
name = "cluster-1"
|
||||
location = "europe-west1"
|
||||
vpc_config = {
|
||||
network = var.vpc.self_link
|
||||
subnetwork = var.subnet.self_link
|
||||
secondary_range_names = { pods = "pods", services = "services" }
|
||||
}
|
||||
backup_configs = {
|
||||
enable_backup_agent = true
|
||||
backup_plans = {
|
||||
"backup-1" = {
|
||||
region = "europe-west-2"
|
||||
schedule = "0 9 * * 1"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
# tftest modules=1 resources=2 inventory=backup.yaml
|
||||
```
|
||||
<!-- BEGIN TFDOC -->
|
||||
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---:|:---:|:---:|
|
||||
| [location](variables.tf#L106) | Autopilot cluster are always regional. | <code>string</code> | ✓ | |
|
||||
| [name](variables.tf#L141) | Cluster name. | <code>string</code> | ✓ | |
|
||||
| [project_id](variables.tf#L167) | Cluster project id. | <code>string</code> | ✓ | |
|
||||
| [vpc_config](variables.tf#L190) | VPC-level configuration. | <code title="object({ network = string subnetwork = string master_ipv4_cidr_block = optional(string) secondary_range_blocks = optional(object({ pods = string services = string })) secondary_range_names = optional(object({ pods = string services = string }), { pods = "pods", services = "services" }) master_authorized_ranges = optional(map(string)) })">object({…})</code> | ✓ | |
|
||||
| [backup_configs](variables.tf#L17) | Configuration for Backup for GKE. | <code title="object({ enable_backup_agent = optional(bool, false) backup_plans = optional(map(object({ region = string schedule = string retention_policy_days = optional(string) retention_policy_lock = optional(bool, false) retention_policy_delete_lock_days = optional(string) })), {}) })">object({…})</code> | | <code>{}</code> |
|
||||
| [description](variables.tf#L33) | Cluster description. | <code>string</code> | | <code>null</code> |
|
||||
| [enable_addons](variables.tf#L39) | Addons enabled in the cluster (true means enabled). | <code title="object({ cloudrun = optional(bool, false) config_connector = optional(bool, false) dns_cache = optional(bool, false) horizontal_pod_autoscaling = optional(bool, false) http_load_balancing = optional(bool, false) istio = optional(object({ enable_tls = bool })) kalm = optional(bool, false) network_policy = optional(bool, false) })">object({…})</code> | | <code title="{ horizontal_pod_autoscaling = true http_load_balancing = true }">{…}</code> |
|
||||
| [enable_features](variables.tf#L60) | Enable cluster-level features. Certain features allow configuration. | <code title="object({ binary_authorization = optional(bool, false) dns = optional(object({ provider = optional(string) scope = optional(string) domain = optional(string) })) database_encryption = optional(object({ state = string key_name = string })) gateway_api = optional(bool, false) groups_for_rbac = optional(string) l4_ilb_subsetting = optional(bool, false) mesh_certificates = optional(bool) pod_security_policy = optional(bool, false) resource_usage_export = optional(object({ dataset = string enable_network_egress_metering = optional(bool) enable_resource_consumption_metering = optional(bool) })) tpu = optional(bool, false) upgrade_notifications = optional(object({ topic_id = optional(string) })) vertical_pod_autoscaling = optional(bool, false) })">object({…})</code> | | <code title="{ }">{…}</code> |
|
||||
| [issue_client_certificate](variables.tf#L94) | Enable issuing client certificate. | <code>bool</code> | | <code>false</code> |
|
||||
| [labels](variables.tf#L100) | Cluster resource labels. | <code>map(string)</code> | | <code>null</code> |
|
||||
| [maintenance_config](variables.tf#L112) | Maintenance window configuration. | <code title="object({ daily_window_start_time = optional(string) recurring_window = optional(object({ start_time = string end_time = string recurrence = string })) maintenance_exclusions = optional(list(object({ name = string start_time = string end_time = string scope = optional(string) }))) })">object({…})</code> | | <code title="{ daily_window_start_time = "03:00" recurring_window = null maintenance_exclusion = [] }">{…}</code> |
|
||||
| [min_master_version](variables.tf#L135) | Minimum version of the master, defaults to the version of the most recent official release. | <code>string</code> | | <code>null</code> |
|
||||
| [node_locations](variables.tf#L146) | Zones in which the cluster's nodes are located. | <code>list(string)</code> | | <code>[]</code> |
|
||||
| [private_cluster_config](variables.tf#L153) | Private cluster configuration. | <code title="object({ enable_private_endpoint = optional(bool) master_global_access = optional(bool) peering_config = optional(object({ export_routes = optional(bool) import_routes = optional(bool) project_id = optional(string) })) })">object({…})</code> | | <code>null</code> |
|
||||
| [release_channel](variables.tf#L172) | Release channel for GKE upgrades. | <code>string</code> | | <code>null</code> |
|
||||
| [service_account](variables.tf#L178) | The Google Cloud Platform Service Account to be used by the node VMs created by GKE Autopilot. | <code>string</code> | | <code>null</code> |
|
||||
| [tags](variables.tf#L184) | Network tags applied to nodes. | <code>list(string)</code> | | <code>null</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| [ca_certificate](outputs.tf#L17) | Public certificate of the cluster (base64-encoded). | ✓ |
|
||||
| [cluster](outputs.tf#L23) | Cluster resource. | ✓ |
|
||||
| [endpoint](outputs.tf#L29) | Cluster endpoint. | |
|
||||
| [id](outputs.tf#L34) | Cluster ID. | |
|
||||
| [location](outputs.tf#L39) | Cluster location. | |
|
||||
| [master_version](outputs.tf#L44) | Master version. | |
|
||||
| [name](outputs.tf#L49) | Cluster name. | |
|
||||
| [notifications](outputs.tf#L54) | GKE PubSub notifications topic. | |
|
||||
| [self_link](outputs.tf#L59) | Cluster self link. | ✓ |
|
||||
| [workload_identity_pool](outputs.tf#L65) | Workload identity pool. | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -0,0 +1,306 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
resource "google_container_cluster" "cluster" {
|
||||
provider = google-beta
|
||||
project = var.project_id
|
||||
name = var.name
|
||||
description = var.description
|
||||
location = var.location
|
||||
node_locations = (
|
||||
length(var.node_locations) == 0 ? null : var.node_locations
|
||||
)
|
||||
min_master_version = var.min_master_version
|
||||
network = var.vpc_config.network
|
||||
subnetwork = var.vpc_config.subnetwork
|
||||
resource_labels = var.labels
|
||||
enable_l4_ilb_subsetting = var.enable_features.l4_ilb_subsetting
|
||||
enable_tpu = var.enable_features.tpu
|
||||
initial_node_count = 1
|
||||
|
||||
enable_autopilot = true
|
||||
|
||||
addons_config {
|
||||
http_load_balancing {
|
||||
disabled = !var.enable_addons.http_load_balancing
|
||||
}
|
||||
horizontal_pod_autoscaling {
|
||||
disabled = !var.enable_addons.horizontal_pod_autoscaling
|
||||
}
|
||||
cloudrun_config {
|
||||
disabled = !var.enable_addons.cloudrun
|
||||
}
|
||||
|
||||
kalm_config {
|
||||
enabled = var.enable_addons.kalm
|
||||
}
|
||||
config_connector_config {
|
||||
enabled = var.enable_addons.config_connector
|
||||
}
|
||||
gke_backup_agent_config {
|
||||
enabled = var.backup_configs.enable_backup_agent
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "authenticator_groups_config" {
|
||||
for_each = var.enable_features.groups_for_rbac != null ? [""] : []
|
||||
content {
|
||||
security_group = var.enable_features.groups_for_rbac
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "binary_authorization" {
|
||||
for_each = var.enable_features.binary_authorization ? [""] : []
|
||||
content {
|
||||
evaluation_mode = "PROJECT_SINGLETON_POLICY_ENFORCE"
|
||||
}
|
||||
}
|
||||
|
||||
cluster_autoscaling {
|
||||
dynamic "auto_provisioning_defaults" {
|
||||
for_each = var.service_account != null ? [""] : []
|
||||
content {
|
||||
service_account = var.service_account
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "database_encryption" {
|
||||
for_each = var.enable_features.database_encryption != null ? [""] : []
|
||||
content {
|
||||
state = var.enable_features.database_encryption.state
|
||||
key_name = var.enable_features.database_encryption.key_name
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "dns_config" {
|
||||
for_each = var.enable_features.dns != null ? [""] : []
|
||||
content {
|
||||
cluster_dns = var.enable_features.dns.provider
|
||||
cluster_dns_scope = var.enable_features.dns.scope
|
||||
cluster_dns_domain = var.enable_features.dns.domain
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "ip_allocation_policy" {
|
||||
for_each = var.vpc_config.secondary_range_blocks != null ? [""] : []
|
||||
content {
|
||||
cluster_ipv4_cidr_block = var.vpc_config.secondary_range_blocks.pods
|
||||
services_ipv4_cidr_block = var.vpc_config.secondary_range_blocks.services
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "ip_allocation_policy" {
|
||||
for_each = var.vpc_config.secondary_range_names != null ? [""] : []
|
||||
content {
|
||||
cluster_secondary_range_name = var.vpc_config.secondary_range_names.pods
|
||||
services_secondary_range_name = var.vpc_config.secondary_range_names.services
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "gateway_api_config" {
|
||||
for_each = var.enable_features.gateway_api ? [""] : []
|
||||
content {
|
||||
channel = "CHANNEL_STANDARD"
|
||||
}
|
||||
}
|
||||
|
||||
maintenance_policy {
|
||||
dynamic "daily_maintenance_window" {
|
||||
for_each = (
|
||||
try(var.maintenance_config.daily_window_start_time, null) != null
|
||||
? [""]
|
||||
: []
|
||||
)
|
||||
content {
|
||||
start_time = var.maintenance_config.daily_window_start_time
|
||||
}
|
||||
}
|
||||
dynamic "recurring_window" {
|
||||
for_each = (
|
||||
try(var.maintenance_config.recurring_window, null) != null
|
||||
? [""]
|
||||
: []
|
||||
)
|
||||
content {
|
||||
start_time = var.maintenance_config.recurring_window.start_time
|
||||
end_time = var.maintenance_config.recurring_window.end_time
|
||||
recurrence = var.maintenance_config.recurring_window.recurrence
|
||||
}
|
||||
}
|
||||
dynamic "maintenance_exclusion" {
|
||||
for_each = (
|
||||
try(var.maintenance_config.maintenance_exclusions, null) == null
|
||||
? []
|
||||
: var.maintenance_config.maintenance_exclusions
|
||||
)
|
||||
iterator = exclusion
|
||||
content {
|
||||
exclusion_name = exclusion.value.name
|
||||
start_time = exclusion.value.start_time
|
||||
end_time = exclusion.value.end_time
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
master_auth {
|
||||
client_certificate_config {
|
||||
issue_client_certificate = var.issue_client_certificate
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "master_authorized_networks_config" {
|
||||
for_each = var.vpc_config.master_authorized_ranges != null ? [""] : []
|
||||
content {
|
||||
dynamic "cidr_blocks" {
|
||||
for_each = var.vpc_config.master_authorized_ranges
|
||||
iterator = range
|
||||
content {
|
||||
cidr_block = range.value
|
||||
display_name = range.key
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "mesh_certificates" {
|
||||
for_each = var.enable_features.mesh_certificates != null ? [""] : []
|
||||
content {
|
||||
enable_certificates = var.enable_features.mesh_certificates
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "notification_config" {
|
||||
for_each = var.enable_features.upgrade_notifications != null ? [""] : []
|
||||
content {
|
||||
pubsub {
|
||||
enabled = true
|
||||
topic = (
|
||||
try(var.enable_features.upgrade_notifications.topic_id, null) != null
|
||||
? var.enable_features.upgrade_notifications.topic_id
|
||||
: google_pubsub_topic.notifications[0].id
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "private_cluster_config" {
|
||||
for_each = (
|
||||
var.private_cluster_config != null ? [""] : []
|
||||
)
|
||||
content {
|
||||
enable_private_nodes = true
|
||||
enable_private_endpoint = var.private_cluster_config.enable_private_endpoint
|
||||
master_ipv4_cidr_block = try(var.vpc_config.master_ipv4_cidr_block, null)
|
||||
master_global_access_config {
|
||||
enabled = var.private_cluster_config.master_global_access
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "pod_security_policy_config" {
|
||||
for_each = var.enable_features.pod_security_policy ? [""] : []
|
||||
content {
|
||||
enabled = var.enable_features.pod_security_policy
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "release_channel" {
|
||||
for_each = var.release_channel != null ? [""] : []
|
||||
content {
|
||||
channel = var.release_channel
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "resource_usage_export_config" {
|
||||
for_each = (
|
||||
try(var.enable_features.resource_usage_export.dataset, null) != null
|
||||
? [""]
|
||||
: []
|
||||
)
|
||||
content {
|
||||
enable_network_egress_metering = (
|
||||
var.enable_features.resource_usage_export.enable_network_egress_metering
|
||||
)
|
||||
enable_resource_consumption_metering = (
|
||||
var.enable_features.resource_usage_export.enable_resource_consumption_metering
|
||||
)
|
||||
bigquery_destination {
|
||||
dataset_id = var.enable_features.resource_usage_export.dataset
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "vertical_pod_autoscaling" {
|
||||
for_each = var.enable_features.vertical_pod_autoscaling ? [""] : []
|
||||
content {
|
||||
enabled = var.enable_features.vertical_pod_autoscaling
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_gke_backup_backup_plan" "backup_plan" {
|
||||
for_each = var.backup_configs.enable_backup_agent ? var.backup_configs.backup_plans : {}
|
||||
name = each.key
|
||||
cluster = google_container_cluster.cluster.id
|
||||
location = each.value.region
|
||||
project = var.project_id
|
||||
retention_policy {
|
||||
backup_delete_lock_days = try(each.value.retention_policy_delete_lock_days)
|
||||
backup_retain_days = try(each.value.retention_policy_days)
|
||||
locked = try(each.value.retention_policy_lock)
|
||||
}
|
||||
backup_schedule {
|
||||
cron_schedule = each.value.schedule
|
||||
}
|
||||
#TODO add support for configs
|
||||
backup_config {
|
||||
include_volume_data = true
|
||||
include_secrets = true
|
||||
all_namespaces = true
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
resource "google_compute_network_peering_routes_config" "gke_master" {
|
||||
count = (
|
||||
try(var.private_cluster_config.peering_config, null) != null ? 1 : 0
|
||||
)
|
||||
project = (
|
||||
try(var.private_cluster_config.peering_config, null) == null
|
||||
? var.project_id
|
||||
: var.private_cluster_config.peering_config.project_id
|
||||
)
|
||||
peering = try(
|
||||
google_container_cluster.cluster.private_cluster_config.0.peering_name,
|
||||
null
|
||||
)
|
||||
network = element(reverse(split("/", var.vpc_config.network)), 0)
|
||||
import_custom_routes = var.private_cluster_config.peering_config.import_routes
|
||||
export_custom_routes = var.private_cluster_config.peering_config.export_routes
|
||||
}
|
||||
|
||||
resource "google_pubsub_topic" "notifications" {
|
||||
count = (
|
||||
try(var.enable_features.upgrade_notifications, null) != null &&
|
||||
try(var.enable_features.upgrade_notifications.topic_id, null) == null ? 1 : 0
|
||||
)
|
||||
project = var.project_id
|
||||
name = "gke-pubsub-notifications"
|
||||
labels = {
|
||||
content = "gke-notifications"
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,207 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
variable "backup_configs" {
|
||||
description = "Configuration for Backup for GKE."
|
||||
type = object({
|
||||
enable_backup_agent = optional(bool, false)
|
||||
backup_plans = optional(map(object({
|
||||
region = string
|
||||
schedule = string
|
||||
retention_policy_days = optional(string)
|
||||
retention_policy_lock = optional(bool, false)
|
||||
retention_policy_delete_lock_days = optional(string)
|
||||
})), {})
|
||||
})
|
||||
default = {}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "description" {
|
||||
description = "Cluster description."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "enable_addons" {
|
||||
description = "Addons enabled in the cluster (true means enabled)."
|
||||
type = object({
|
||||
cloudrun = optional(bool, false)
|
||||
config_connector = optional(bool, false)
|
||||
dns_cache = optional(bool, false)
|
||||
horizontal_pod_autoscaling = optional(bool, false)
|
||||
http_load_balancing = optional(bool, false)
|
||||
istio = optional(object({
|
||||
enable_tls = bool
|
||||
}))
|
||||
kalm = optional(bool, false)
|
||||
network_policy = optional(bool, false)
|
||||
})
|
||||
default = {
|
||||
horizontal_pod_autoscaling = true
|
||||
http_load_balancing = true
|
||||
}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "enable_features" {
|
||||
description = "Enable cluster-level features. Certain features allow configuration."
|
||||
type = object({
|
||||
binary_authorization = optional(bool, false)
|
||||
dns = optional(object({
|
||||
provider = optional(string)
|
||||
scope = optional(string)
|
||||
domain = optional(string)
|
||||
}))
|
||||
database_encryption = optional(object({
|
||||
state = string
|
||||
key_name = string
|
||||
}))
|
||||
gateway_api = optional(bool, false)
|
||||
groups_for_rbac = optional(string)
|
||||
l4_ilb_subsetting = optional(bool, false)
|
||||
mesh_certificates = optional(bool)
|
||||
pod_security_policy = optional(bool, false)
|
||||
resource_usage_export = optional(object({
|
||||
dataset = string
|
||||
enable_network_egress_metering = optional(bool)
|
||||
enable_resource_consumption_metering = optional(bool)
|
||||
}))
|
||||
tpu = optional(bool, false)
|
||||
upgrade_notifications = optional(object({
|
||||
topic_id = optional(string)
|
||||
}))
|
||||
vertical_pod_autoscaling = optional(bool, false)
|
||||
})
|
||||
default = {
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
variable "issue_client_certificate" {
|
||||
description = "Enable issuing client certificate."
|
||||
type = bool
|
||||
default = false
|
||||
}
|
||||
|
||||
variable "labels" {
|
||||
description = "Cluster resource labels."
|
||||
type = map(string)
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "location" {
|
||||
description = "Autopilot cluster are always regional."
|
||||
type = string
|
||||
}
|
||||
|
||||
|
||||
variable "maintenance_config" {
|
||||
description = "Maintenance window configuration."
|
||||
type = object({
|
||||
daily_window_start_time = optional(string)
|
||||
recurring_window = optional(object({
|
||||
start_time = string
|
||||
end_time = string
|
||||
recurrence = string
|
||||
}))
|
||||
maintenance_exclusions = optional(list(object({
|
||||
name = string
|
||||
start_time = string
|
||||
end_time = string
|
||||
scope = optional(string)
|
||||
})))
|
||||
})
|
||||
default = {
|
||||
daily_window_start_time = "03:00"
|
||||
recurring_window = null
|
||||
maintenance_exclusion = []
|
||||
}
|
||||
}
|
||||
|
||||
variable "min_master_version" {
|
||||
description = "Minimum version of the master, defaults to the version of the most recent official release."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "name" {
|
||||
description = "Cluster name."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "node_locations" {
|
||||
description = "Zones in which the cluster's nodes are located."
|
||||
type = list(string)
|
||||
default = []
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "private_cluster_config" {
|
||||
description = "Private cluster configuration."
|
||||
type = object({
|
||||
enable_private_endpoint = optional(bool)
|
||||
master_global_access = optional(bool)
|
||||
peering_config = optional(object({
|
||||
export_routes = optional(bool)
|
||||
import_routes = optional(bool)
|
||||
project_id = optional(string)
|
||||
}))
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "project_id" {
|
||||
description = "Cluster project id."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "release_channel" {
|
||||
description = "Release channel for GKE upgrades."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "service_account" {
|
||||
description = "The Google Cloud Platform Service Account to be used by the node VMs created by GKE Autopilot."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Network tags applied to nodes."
|
||||
type = list(string)
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "vpc_config" {
|
||||
description = "VPC-level configuration."
|
||||
type = object({
|
||||
network = string
|
||||
subnetwork = string
|
||||
master_ipv4_cidr_block = optional(string)
|
||||
secondary_range_blocks = optional(object({
|
||||
pods = string
|
||||
services = string
|
||||
}))
|
||||
secondary_range_names = optional(object({
|
||||
pods = string
|
||||
services = string
|
||||
}), { pods = "pods", services = "services" })
|
||||
master_authorized_ranges = optional(map(string))
|
||||
})
|
||||
nullable = false
|
||||
}
|
||||
|
|
@ -12,6 +12,7 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
|
||||
terraform {
|
||||
required_version = ">= 1.4.4"
|
||||
required_providers {
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
# GKE cluster module
|
||||
# GKE cluster Standard module
|
||||
|
||||
This module allows simplified creation and management of GKE clusters and should be used together with the GKE nodepool module, as the default nodepool is turned off here and cannot be re-enabled. Some sensible defaults are set initially, in order to allow less verbose usage for most use cases.
|
||||
This module allows simplified creation and management of GKE Standard clusters and should be used together with the GKE nodepool module, as the default nodepool is turned off here and cannot be re-enabled. Some sensible defaults are set initially, in order to allow less verbose usage for most use cases.
|
||||
|
||||
## Example
|
||||
|
||||
|
|
@ -8,7 +8,7 @@ This module allows simplified creation and management of GKE clusters and should
|
|||
|
||||
```hcl
|
||||
module "cluster-1" {
|
||||
source = "./fabric/modules/gke-cluster"
|
||||
source = "./fabric/modules/gke-cluster-standard"
|
||||
project_id = "myproject"
|
||||
name = "cluster-1"
|
||||
location = "europe-west1-b"
|
||||
|
|
@ -40,7 +40,7 @@ module "cluster-1" {
|
|||
|
||||
```hcl
|
||||
module "cluster-1" {
|
||||
source = "./fabric/modules/gke-cluster"
|
||||
source = "./fabric/modules/gke-cluster-standard"
|
||||
project_id = "myproject"
|
||||
name = "cluster-dataplane-v2"
|
||||
location = "europe-west1-b"
|
||||
|
|
@ -70,32 +70,6 @@ module "cluster-1" {
|
|||
}
|
||||
# tftest modules=1 resources=1 inventory=dataplane-v2.yaml
|
||||
```
|
||||
### Autopilot Cluster
|
||||
|
||||
```hcl
|
||||
module "cluster-autopilot" {
|
||||
source = "./fabric/modules/gke-cluster"
|
||||
project_id = "myproject"
|
||||
name = "cluster-autopilot"
|
||||
location = "europe-west1-b"
|
||||
vpc_config = {
|
||||
network = var.vpc.self_link
|
||||
subnetwork = var.subnet.self_link
|
||||
secondary_range_names = {
|
||||
pods = "pods"
|
||||
services = "services"
|
||||
}
|
||||
master_authorized_ranges = {
|
||||
internal-vms = "10.0.0.0/8"
|
||||
}
|
||||
master_ipv4_cidr_block = "192.168.0.0/28"
|
||||
}
|
||||
enable_features = {
|
||||
autopilot = true
|
||||
}
|
||||
}
|
||||
# tftest modules=1 resources=1 inventory=autopilot.yaml
|
||||
```
|
||||
|
||||
### Cloud DNS
|
||||
|
||||
|
|
@ -103,7 +77,7 @@ This example shows how to [use Cloud DNS as a Kubernetes DNS provider](https://c
|
|||
|
||||
```hcl
|
||||
module "cluster-1" {
|
||||
source = "./fabric/modules/gke-cluster"
|
||||
source = "./fabric/modules/gke-cluster-standard"
|
||||
project_id = var.project_id
|
||||
name = "cluster-1"
|
||||
location = "europe-west1-b"
|
||||
|
|
@ -130,7 +104,7 @@ This example shows how to [enable the Backup for GKE agent and configure a Backu
|
|||
|
||||
```hcl
|
||||
module "cluster-1" {
|
||||
source = "./fabric/modules/gke-cluster"
|
||||
source = "./fabric/modules/gke-cluster-standard"
|
||||
project_id = var.project_id
|
||||
name = "cluster-1"
|
||||
location = "europe-west1-b"
|
||||
|
|
@ -157,26 +131,26 @@ module "cluster-1" {
|
|||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---:|:---:|:---:|
|
||||
| [location](variables.tf#L134) | Cluster zone or region. | <code>string</code> | ✓ | |
|
||||
| [name](variables.tf#L191) | Cluster name. | <code>string</code> | ✓ | |
|
||||
| [project_id](variables.tf#L217) | Cluster project id. | <code>string</code> | ✓ | |
|
||||
| [vpc_config](variables.tf#L234) | VPC-level configuration. | <code title="object({ network = string subnetwork = string master_ipv4_cidr_block = optional(string) secondary_range_blocks = optional(object({ pods = string services = string })) secondary_range_names = optional(object({ pods = string services = string }), { pods = "pods", services = "services" }) master_authorized_ranges = optional(map(string)) })">object({…})</code> | ✓ | |
|
||||
| [location](variables.tf#L133) | Cluster zone or region. | <code>string</code> | ✓ | |
|
||||
| [name](variables.tf#L190) | Cluster name. | <code>string</code> | ✓ | |
|
||||
| [project_id](variables.tf#L216) | Cluster project id. | <code>string</code> | ✓ | |
|
||||
| [vpc_config](variables.tf#L233) | VPC-level configuration. | <code title="object({ network = string subnetwork = string master_ipv4_cidr_block = optional(string) secondary_range_blocks = optional(object({ pods = string services = string })) secondary_range_names = optional(object({ pods = string services = string }), { pods = "pods", services = "services" }) master_authorized_ranges = optional(map(string)) })">object({…})</code> | ✓ | |
|
||||
| [backup_configs](variables.tf#L17) | Configuration for Backup for GKE. | <code title="object({ enable_backup_agent = optional(bool, false) backup_plans = optional(map(object({ region = string schedule = string retention_policy_days = optional(string) retention_policy_lock = optional(bool, false) retention_policy_delete_lock_days = optional(string) })), {}) })">object({…})</code> | | <code>{}</code> |
|
||||
| [cluster_autoscaling](variables.tf#L33) | Enable and configure limits for Node Auto-Provisioning with Cluster Autoscaler. | <code title="object({ auto_provisioning_defaults = optional(object({ boot_disk_kms_key = optional(string) image_type = optional(string) oauth_scopes = optional(list(string)) service_account = optional(string) })) cpu_limits = optional(object({ min = number max = number })) mem_limits = optional(object({ min = number max = number })) })">object({…})</code> | | <code>null</code> |
|
||||
| [description](variables.tf#L54) | Cluster description. | <code>string</code> | | <code>null</code> |
|
||||
| [enable_addons](variables.tf#L60) | Addons enabled in the cluster (true means enabled). | <code title="object({ cloudrun = optional(bool, false) config_connector = optional(bool, false) dns_cache = optional(bool, false) gce_persistent_disk_csi_driver = optional(bool, false) gcp_filestore_csi_driver = optional(bool, false) horizontal_pod_autoscaling = optional(bool, false) http_load_balancing = optional(bool, false) istio = optional(object({ enable_tls = bool })) kalm = optional(bool, false) network_policy = optional(bool, false) })">object({…})</code> | | <code title="{ horizontal_pod_autoscaling = true http_load_balancing = true }">{…}</code> |
|
||||
| [enable_features](variables.tf#L83) | Enable cluster-level features. Certain features allow configuration. | <code title="object({ autopilot = optional(bool, false) binary_authorization = optional(bool, false) dns = optional(object({ provider = optional(string) scope = optional(string) domain = optional(string) })) database_encryption = optional(object({ state = string key_name = string })) dataplane_v2 = optional(bool, false) gateway_api = optional(bool, false) groups_for_rbac = optional(string) intranode_visibility = optional(bool, false) l4_ilb_subsetting = optional(bool, false) mesh_certificates = optional(bool) pod_security_policy = optional(bool, false) resource_usage_export = optional(object({ dataset = string enable_network_egress_metering = optional(bool) enable_resource_consumption_metering = optional(bool) })) shielded_nodes = optional(bool, false) tpu = optional(bool, false) upgrade_notifications = optional(object({ topic_id = optional(string) })) vertical_pod_autoscaling = optional(bool, false) workload_identity = optional(bool, true) })">object({…})</code> | | <code title="{ workload_identity = true }">{…}</code> |
|
||||
| [issue_client_certificate](variables.tf#L122) | Enable issuing client certificate. | <code>bool</code> | | <code>false</code> |
|
||||
| [labels](variables.tf#L128) | Cluster resource labels. | <code>map(string)</code> | | <code>null</code> |
|
||||
| [logging_config](variables.tf#L139) | Logging configuration. | <code>list(string)</code> | | <code>["SYSTEM_COMPONENTS"]</code> |
|
||||
| [maintenance_config](variables.tf#L145) | Maintenance window configuration. | <code title="object({ daily_window_start_time = optional(string) recurring_window = optional(object({ start_time = string end_time = string recurrence = string })) maintenance_exclusions = optional(list(object({ name = string start_time = string end_time = string scope = optional(string) }))) })">object({…})</code> | | <code title="{ daily_window_start_time = "03:00" recurring_window = null maintenance_exclusion = [] }">{…}</code> |
|
||||
| [max_pods_per_node](variables.tf#L168) | Maximum number of pods per node in this cluster. | <code>number</code> | | <code>110</code> |
|
||||
| [min_master_version](variables.tf#L174) | Minimum version of the master, defaults to the version of the most recent official release. | <code>string</code> | | <code>null</code> |
|
||||
| [monitoring_config](variables.tf#L180) | Monitoring components. | <code title="object({ enable_components = optional(list(string)) managed_prometheus = optional(bool) })">object({…})</code> | | <code title="{ enable_components = ["SYSTEM_COMPONENTS"] }">{…}</code> |
|
||||
| [node_locations](variables.tf#L196) | Zones in which the cluster's nodes are located. | <code>list(string)</code> | | <code>[]</code> |
|
||||
| [private_cluster_config](variables.tf#L203) | Private cluster configuration. | <code title="object({ enable_private_endpoint = optional(bool) master_global_access = optional(bool) peering_config = optional(object({ export_routes = optional(bool) import_routes = optional(bool) project_id = optional(string) })) })">object({…})</code> | | <code>null</code> |
|
||||
| [release_channel](variables.tf#L222) | Release channel for GKE upgrades. | <code>string</code> | | <code>null</code> |
|
||||
| [tags](variables.tf#L228) | Network tags applied to nodes. | <code>list(string)</code> | | <code>null</code> |
|
||||
| [enable_features](variables.tf#L83) | Enable cluster-level features. Certain features allow configuration. | <code title="object({ binary_authorization = optional(bool, false) dns = optional(object({ provider = optional(string) scope = optional(string) domain = optional(string) })) database_encryption = optional(object({ state = string key_name = string })) dataplane_v2 = optional(bool, false) gateway_api = optional(bool, false) groups_for_rbac = optional(string) intranode_visibility = optional(bool, false) l4_ilb_subsetting = optional(bool, false) mesh_certificates = optional(bool) pod_security_policy = optional(bool, false) resource_usage_export = optional(object({ dataset = string enable_network_egress_metering = optional(bool) enable_resource_consumption_metering = optional(bool) })) shielded_nodes = optional(bool, false) tpu = optional(bool, false) upgrade_notifications = optional(object({ topic_id = optional(string) })) vertical_pod_autoscaling = optional(bool, false) workload_identity = optional(bool, true) })">object({…})</code> | | <code title="{ workload_identity = true }">{…}</code> |
|
||||
| [issue_client_certificate](variables.tf#L121) | Enable issuing client certificate. | <code>bool</code> | | <code>false</code> |
|
||||
| [labels](variables.tf#L127) | Cluster resource labels. | <code>map(string)</code> | | <code>null</code> |
|
||||
| [logging_config](variables.tf#L138) | Logging configuration. | <code>list(string)</code> | | <code>["SYSTEM_COMPONENTS"]</code> |
|
||||
| [maintenance_config](variables.tf#L144) | Maintenance window configuration. | <code title="object({ daily_window_start_time = optional(string) recurring_window = optional(object({ start_time = string end_time = string recurrence = string })) maintenance_exclusions = optional(list(object({ name = string start_time = string end_time = string scope = optional(string) }))) })">object({…})</code> | | <code title="{ daily_window_start_time = "03:00" recurring_window = null maintenance_exclusion = [] }">{…}</code> |
|
||||
| [max_pods_per_node](variables.tf#L167) | Maximum number of pods per node in this cluster. | <code>number</code> | | <code>110</code> |
|
||||
| [min_master_version](variables.tf#L173) | Minimum version of the master, defaults to the version of the most recent official release. | <code>string</code> | | <code>null</code> |
|
||||
| [monitoring_config](variables.tf#L179) | Monitoring components. | <code title="object({ enable_components = optional(list(string)) managed_prometheus = optional(bool) })">object({…})</code> | | <code title="{ enable_components = ["SYSTEM_COMPONENTS"] }">{…}</code> |
|
||||
| [node_locations](variables.tf#L195) | Zones in which the cluster's nodes are located. | <code>list(string)</code> | | <code>[]</code> |
|
||||
| [private_cluster_config](variables.tf#L202) | Private cluster configuration. | <code title="object({ enable_private_endpoint = optional(bool) master_global_access = optional(bool) peering_config = optional(object({ export_routes = optional(bool) import_routes = optional(bool) project_id = optional(string) })) })">object({…})</code> | | <code>null</code> |
|
||||
| [release_channel](variables.tf#L221) | Release channel for GKE upgrades. | <code>string</code> | | <code>null</code> |
|
||||
| [tags](variables.tf#L227) | Network tags applied to nodes. | <code>list(string)</code> | | <code>null</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
|
|
@ -15,12 +15,6 @@
|
|||
*/
|
||||
|
||||
resource "google_container_cluster" "cluster" {
|
||||
lifecycle {
|
||||
ignore_changes = [
|
||||
node_config[0].boot_disk_kms_key,
|
||||
node_config[0].spot
|
||||
]
|
||||
}
|
||||
provider = google-beta
|
||||
project = var.project_id
|
||||
name = var.name
|
||||
|
|
@ -29,54 +23,39 @@ resource "google_container_cluster" "cluster" {
|
|||
node_locations = (
|
||||
length(var.node_locations) == 0 ? null : var.node_locations
|
||||
)
|
||||
min_master_version = var.min_master_version
|
||||
network = var.vpc_config.network
|
||||
subnetwork = var.vpc_config.subnetwork
|
||||
resource_labels = var.labels
|
||||
default_max_pods_per_node = (
|
||||
var.enable_features.autopilot ? null : var.max_pods_per_node
|
||||
)
|
||||
enable_intranode_visibility = (
|
||||
var.enable_features.autopilot ? null : var.enable_features.intranode_visibility
|
||||
)
|
||||
enable_l4_ilb_subsetting = var.enable_features.l4_ilb_subsetting
|
||||
enable_shielded_nodes = (
|
||||
var.enable_features.autopilot ? null : var.enable_features.shielded_nodes
|
||||
)
|
||||
enable_tpu = var.enable_features.tpu
|
||||
initial_node_count = 1
|
||||
remove_default_node_pool = var.enable_features.autopilot ? null : true
|
||||
min_master_version = var.min_master_version
|
||||
network = var.vpc_config.network
|
||||
subnetwork = var.vpc_config.subnetwork
|
||||
resource_labels = var.labels
|
||||
default_max_pods_per_node = var.max_pods_per_node
|
||||
enable_intranode_visibility = var.enable_features.intranode_visibility
|
||||
enable_l4_ilb_subsetting = var.enable_features.l4_ilb_subsetting
|
||||
enable_shielded_nodes = var.enable_features.shielded_nodes
|
||||
enable_tpu = var.enable_features.tpu
|
||||
initial_node_count = 1
|
||||
remove_default_node_pool = true
|
||||
datapath_provider = (
|
||||
var.enable_features.dataplane_v2 || var.enable_features.autopilot
|
||||
var.enable_features.dataplane_v2
|
||||
? "ADVANCED_DATAPATH"
|
||||
: "DATAPATH_PROVIDER_UNSPECIFIED"
|
||||
)
|
||||
enable_autopilot = var.enable_features.autopilot ? true : null
|
||||
|
||||
# the default nodepool is deleted here, use the gke-nodepool module instead
|
||||
# default nodepool configuration based on a shielded_nodes variable
|
||||
dynamic "node_config" {
|
||||
for_each = var.enable_features.autopilot ? [] : [""]
|
||||
content {
|
||||
dynamic "shielded_instance_config" {
|
||||
for_each = var.enable_features.shielded_nodes ? [""] : []
|
||||
content {
|
||||
enable_secure_boot = true
|
||||
enable_integrity_monitoring = true
|
||||
}
|
||||
node_config {
|
||||
dynamic "shielded_instance_config" {
|
||||
for_each = var.enable_features.shielded_nodes ? [""] : []
|
||||
content {
|
||||
enable_secure_boot = true
|
||||
enable_integrity_monitoring = true
|
||||
}
|
||||
tags = var.tags
|
||||
}
|
||||
tags = var.tags
|
||||
}
|
||||
|
||||
|
||||
|
||||
addons_config {
|
||||
dynamic "dns_cache_config" {
|
||||
for_each = !var.enable_features.autopilot ? [""] : []
|
||||
content {
|
||||
enabled = var.enable_addons.dns_cache
|
||||
}
|
||||
dns_cache_config {
|
||||
enabled = var.enable_addons.dns_cache
|
||||
}
|
||||
http_load_balancing {
|
||||
disabled = !var.enable_addons.http_load_balancing
|
||||
|
|
@ -84,11 +63,8 @@ resource "google_container_cluster" "cluster" {
|
|||
horizontal_pod_autoscaling {
|
||||
disabled = !var.enable_addons.horizontal_pod_autoscaling
|
||||
}
|
||||
dynamic "network_policy_config" {
|
||||
for_each = !var.enable_features.autopilot ? [""] : []
|
||||
content {
|
||||
disabled = !var.enable_addons.network_policy
|
||||
}
|
||||
network_policy_config {
|
||||
disabled = !var.enable_addons.network_policy
|
||||
}
|
||||
cloudrun_config {
|
||||
disabled = !var.enable_addons.cloudrun
|
||||
|
|
@ -100,17 +76,10 @@ resource "google_container_cluster" "cluster" {
|
|||
)
|
||||
}
|
||||
gce_persistent_disk_csi_driver_config {
|
||||
enabled = (
|
||||
var.enable_features.autopilot
|
||||
? true
|
||||
: var.enable_addons.gce_persistent_disk_csi_driver
|
||||
)
|
||||
enabled = var.enable_addons.gce_persistent_disk_csi_driver
|
||||
}
|
||||
dynamic "gcp_filestore_csi_driver_config" {
|
||||
for_each = !var.enable_features.autopilot ? [""] : []
|
||||
content {
|
||||
enabled = var.enable_addons.gcp_filestore_csi_driver
|
||||
}
|
||||
gcp_filestore_csi_driver_config {
|
||||
enabled = var.enable_addons.gcp_filestore_csi_driver
|
||||
}
|
||||
kalm_config {
|
||||
enabled = var.enable_addons.kalm
|
||||
|
|
@ -140,7 +109,7 @@ resource "google_container_cluster" "cluster" {
|
|||
dynamic "cluster_autoscaling" {
|
||||
for_each = var.cluster_autoscaling == null ? [] : [""]
|
||||
content {
|
||||
enabled = var.enable_features.autopilot ? null : true
|
||||
enabled = true
|
||||
|
||||
dynamic "auto_provisioning_defaults" {
|
||||
for_each = var.cluster_autoscaling.auto_provisioning_defaults != null ? [""] : []
|
||||
|
|
@ -204,7 +173,7 @@ resource "google_container_cluster" "cluster" {
|
|||
}
|
||||
|
||||
dynamic "logging_config" {
|
||||
for_each = var.logging_config != null && !var.enable_features.autopilot ? [""] : []
|
||||
for_each = var.logging_config != null ? [""] : []
|
||||
content {
|
||||
enable_components = var.logging_config
|
||||
}
|
||||
|
|
@ -283,7 +252,7 @@ resource "google_container_cluster" "cluster" {
|
|||
}
|
||||
|
||||
dynamic "monitoring_config" {
|
||||
for_each = var.monitoring_config != null && !var.enable_features.autopilot ? [""] : []
|
||||
for_each = var.monitoring_config != null ? [""] : []
|
||||
content {
|
||||
enable_components = var.monitoring_config.enable_components
|
||||
dynamic "managed_prometheus" {
|
||||
|
|
@ -379,11 +348,17 @@ resource "google_container_cluster" "cluster" {
|
|||
}
|
||||
|
||||
dynamic "workload_identity_config" {
|
||||
for_each = (var.enable_features.workload_identity && !var.enable_features.autopilot) ? [""] : []
|
||||
for_each = var.enable_features.workload_identity ? [""] : []
|
||||
content {
|
||||
workload_pool = "${var.project_id}.svc.id.goog"
|
||||
}
|
||||
}
|
||||
lifecycle {
|
||||
ignore_changes = [
|
||||
node_config[0].boot_disk_kms_key,
|
||||
node_config[0].spot
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_gke_backup_backup_plan" "backup_plan" {
|
||||
|
|
@ -0,0 +1,71 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
output "ca_certificate" {
|
||||
description = "Public certificate of the cluster (base64-encoded)."
|
||||
value = google_container_cluster.cluster.master_auth.0.cluster_ca_certificate
|
||||
sensitive = true
|
||||
}
|
||||
|
||||
output "cluster" {
|
||||
description = "Cluster resource."
|
||||
sensitive = true
|
||||
value = google_container_cluster.cluster
|
||||
}
|
||||
|
||||
output "endpoint" {
|
||||
description = "Cluster endpoint."
|
||||
value = google_container_cluster.cluster.endpoint
|
||||
}
|
||||
|
||||
output "id" {
|
||||
description = "Cluster ID."
|
||||
value = google_container_cluster.cluster.id
|
||||
}
|
||||
|
||||
output "location" {
|
||||
description = "Cluster location."
|
||||
value = google_container_cluster.cluster.location
|
||||
}
|
||||
|
||||
output "master_version" {
|
||||
description = "Master version."
|
||||
value = google_container_cluster.cluster.master_version
|
||||
}
|
||||
|
||||
output "name" {
|
||||
description = "Cluster name."
|
||||
value = google_container_cluster.cluster.name
|
||||
}
|
||||
|
||||
output "notifications" {
|
||||
description = "GKE PubSub notifications topic."
|
||||
value = try(google_pubsub_topic.notifications[0].id, null)
|
||||
}
|
||||
|
||||
output "self_link" {
|
||||
description = "Cluster self link."
|
||||
sensitive = true
|
||||
value = google_container_cluster.cluster.self_link
|
||||
}
|
||||
|
||||
output "workload_identity_pool" {
|
||||
description = "Workload identity pool."
|
||||
value = "${var.project_id}.svc.id.goog"
|
||||
depends_on = [
|
||||
google_container_cluster.cluster
|
||||
]
|
||||
}
|
||||
|
|
@ -83,7 +83,6 @@ variable "enable_addons" {
|
|||
variable "enable_features" {
|
||||
description = "Enable cluster-level features. Certain features allow configuration."
|
||||
type = object({
|
||||
autopilot = optional(bool, false)
|
||||
binary_authorization = optional(bool, false)
|
||||
dns = optional(object({
|
||||
provider = optional(string)
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
|
||||
terraform {
|
||||
required_version = ">= 1.4.4"
|
||||
required_providers {
|
||||
google = {
|
||||
source = "hashicorp/google"
|
||||
version = ">= 4.60.0" # tftest
|
||||
}
|
||||
google-beta = {
|
||||
source = "hashicorp/google-beta"
|
||||
version = ">= 4.60.0" # tftest
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
|
@ -46,7 +46,7 @@ module "vpc" {
|
|||
}
|
||||
|
||||
module "cluster_1" {
|
||||
source = "./fabric/modules/gke-cluster"
|
||||
source = "./fabric/modules/gke-cluster-standard"
|
||||
project_id = module.project.project_id
|
||||
name = "cluster-1"
|
||||
location = "europe-west1"
|
||||
|
|
@ -212,7 +212,7 @@ module "firewall" {
|
|||
}
|
||||
|
||||
module "cluster_1" {
|
||||
source = "./fabric/modules/gke-cluster"
|
||||
source = "./fabric/modules/gke-cluster-standard"
|
||||
project_id = module.project.project_id
|
||||
name = "cluster-1"
|
||||
location = "europe-west1"
|
||||
|
|
@ -253,7 +253,7 @@ module "cluster_1_nodepool" {
|
|||
}
|
||||
|
||||
module "cluster_2" {
|
||||
source = "./fabric/modules/gke-cluster"
|
||||
source = "./fabric/modules/gke-cluster-standard"
|
||||
project_id = module.project.project_id
|
||||
name = "cluster-2"
|
||||
location = "europe-west4"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,38 @@
|
|||
# Copyright 2023 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
values:
|
||||
module.cluster-1.google_container_cluster.cluster:
|
||||
location: europe-west1
|
||||
name: cluster-1
|
||||
|
||||
module.cluster-1.google_gke_backup_backup_plan.backup_plan["backup-1"]:
|
||||
backup_config:
|
||||
- all_namespaces: true
|
||||
encryption_key: []
|
||||
include_secrets: true
|
||||
include_volume_data: true
|
||||
selected_applications: []
|
||||
selected_namespaces: []
|
||||
backup_schedule:
|
||||
- cron_schedule: 0 9 * * 1
|
||||
location: europe-west-2
|
||||
name: backup-1
|
||||
project: project-id
|
||||
retention_policy:
|
||||
- locked: false
|
||||
|
||||
counts:
|
||||
google_container_cluster: 1
|
||||
google_gke_backup_backup_plan: 1
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
# Copyright 2023 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
values:
|
||||
module.cluster-1.google_container_cluster.cluster:
|
||||
private_cluster_config:
|
||||
- enable_private_endpoint: true
|
||||
enable_private_nodes: true
|
||||
master_global_access_config:
|
||||
- enabled: false
|
||||
master_ipv4_cidr_block: 192.168.0.0/28
|
||||
private_endpoint_subnetwork: null
|
||||
resource_labels:
|
||||
environment: dev
|
||||
|
||||
counts:
|
||||
google_container_cluster: 1
|
||||
|
|
@ -13,8 +13,11 @@
|
|||
# limitations under the License.
|
||||
|
||||
values:
|
||||
module.cluster-autopilot.google_container_cluster.cluster:
|
||||
enable_autopilot: true
|
||||
module.cluster-1.google_container_cluster.cluster:
|
||||
dns_config:
|
||||
- cluster_dns: CLOUD_DNS
|
||||
cluster_dns_domain: gke.local
|
||||
cluster_dns_scope: CLUSTER_SCOPE
|
||||
|
||||
counts:
|
||||
google_container_cluster: 1
|
||||
Loading…
Reference in New Issue