diff --git a/.gitignore b/.gitignore
index 543bc870..12a34c55 100644
--- a/.gitignore
+++ b/.gitignore
@@ -57,4 +57,5 @@ blueprints/gke/autopilot/ansible/gssh.sh
blueprints/gke/autopilot/ansible/vars/vars.yaml
blueprints/gke/autopilot/bundle/monitoring/kustomization.yaml
blueprints/gke/autopilot/bundle/locust/kustomization.yaml
-blueprints/gke/autopilot/bundle.tar.gz
\ No newline at end of file
+blueprints/gke/autopilot/bundle.tar.gz
+blueprints/gke/patterns/batch/job-*.yaml
diff --git a/blueprints/gke/patterns/README.md b/blueprints/gke/patterns/README.md
new file mode 100644
index 00000000..65ddbc98
--- /dev/null
+++ b/blueprints/gke/patterns/README.md
@@ -0,0 +1,5 @@
+# GKE Jumpstart Blueprints
+
+This directory includes several blueprints related to Google Kubernetes Engine (GKE), following Google recommendations and best practices. The blueprints in this directory split the deployment process into two stages: an initial infrastructure stage that provisions the cluster, and additional workload stages that deploy specific types of applications/workloads.
+
+As a design rule, all the blueprints in this directory provide sensible defaults for most variables while still providing an enterprise-grade deployment with secure defaults and the ability to use existing resources that are typically found in an enterprise-grade environment.
diff --git a/blueprints/gke/patterns/autopilot-cluster/README.md b/blueprints/gke/patterns/autopilot-cluster/README.md
new file mode 100644
index 00000000..25006ea4
--- /dev/null
+++ b/blueprints/gke/patterns/autopilot-cluster/README.md
@@ -0,0 +1,112 @@
+# GKE Autopilot Cluster Pattern
+
+This blueprint illustrates how to use GKE features to deploy a secure cluster that meets Google's best practices. The cluster deployed by this blueprint can be used to deploy other blueprints such as [Redis](../redis-cluster), [Kafka](../kafka), [Kueue](../batch).
+
+
+- [Design Decisions](#design-decisions)
+- [GKE Onboarding Best Practices](#gke-onboarding-best-practices)
+ - [Environment setup](#environment-setup)
+ - [Cluster configuration](#cluster-configuration)
+ - [Security](#security)
+ - [Networking](#networking)
+ - [Multitenancy](#multitenancy)
+ - [Monitoring](#monitoring)
+ - [Maintenance](#maintenance)
+- [Variables](#variables)
+- [Outputs](#outputs)
+
+
+## Design Decisions
+
+The main purpose of this blueprint is to showcase how to use GKE features to deploy a secure Kubernetes cluster according to Google best practices, including:
+
+- **No public IP addresses** both the control plane and the nodes use private IP addresses. To to simplify the deployment of workloads, we enable [Connect Gateway](https://cloud.google.com/anthos/multicluster-management/gateway) to securely access the control plane even from outside the cluster's VPC. We also use [Remote Repositories](https://cloud.google.com/artifact-registry/docs/repositories/remote-overview) to allow the download of container images by the cluster without requiring Internet egress configured in the clusters's VPC.
+
+- We provide **reasonable but secure defaults** that the user can override. For example, by default we avoid deploying a Cloud NAT gatewayt, but it is possible to enable it with just a few changes to the configuration.
+
+- **Bring your own infrastructure**: that larger organizations might have teams dedicated to the provisioning and management of centralized infrastructure. This blueprint can be deployed to create any required infrastructure (GCP project, VPC, Artifact Registry, etc), or you can leverage existing resources by setting the appropriate variables.
+
+## GKE Onboarding Best Practices
+
+This Terraform blueprint helps you quickly implement most of the [GKE oboarding best practices](https://cloud.google.com/kubernetes-engine/docs/best-practices/onboarding#set-up-terraform) as outlined in the official GKE documentation. In this section we describe the relevant the decisions this blueprint simplifies
+
+
+### Environment setup
+- Set up Terraform: you'll need to install Terraform to use this blueprint. Instructions are [available here](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/getting_started).
+- Terraform state storage: this blueprint doesn't automate this step but can easily be done by specifying a [backend](https://developer.hashicorp.com/terraform/language/settings/backends/gcs).
+- Create a metrics scope using Terraform: if you're creating a new project with this blueprint, you can enable metrics scope using the `metrics_scope` variable in the `project` module. Otherwise, metrics scope setup occurs outside this blueprint's scope.
+- Set up Artifact Registry: by default a remote repository is created to allow downloading container images
+
+### Cluster configuration
+This blueprint by default deploys an Autopilot cluster with private nodes and private control plane. By using Autopilot, Google automatically handles node configuration, scaling, and security
+
+- Choose a mode of operation: this blueprint uses Autopilot clusters
+- Isolate your cluster: this blueprint deploys a private cluster, with private control plane
+- Configure backup for GKE: not configured but can easily be enabled through the `backup_configs` in the `gke-cluster-autopilot` module.
+- Use Container-Optimized OS node images: Autopilot cluster always user COS
+- Enable node auto-provisioning: automatically managed by Autopilot
+- Separate kube-system Pods: automatically managed by Autopilot
+
+### Security
+- Use the security posture dashboard: enabled by default in new clusters
+- Use group authentication: not needed by this blueprint but can be enabled through the `enable_features.groups_for_rbac` variable of the `gke-cluster-autopilot` module.
+- Use RBAC to restrict access to cluster resources: this blueprint deploys the underlying infrastructure, RBAC configuration is out of scope.
+- Enable Shielded GKE Nodes: automatically managed by Autopilot
+- Enable Workload Identity: automatically managed by Autopilot
+- Enable security bulletin notifications: out of scope for this blueprint
+- Use least privilege Google service accounts: this blueprint creates a new service account for the cluster
+- Restrict network access to the control plane and nodes: this blueprint deploys a private cluster
+- Use namespaces to restrict access to cluster resources: this blueprint deploys the underlying infrastructure, namespace handling is left to applications.
+
+### Networking
+- Create a custom mode VPC: this blueprint can optinally deploy a new custom VPC with a single subnet. Otherwise, an existing VPC and subnet can be used.
+- Create a proxy-only subnet: the `vpc_create` variable allows the creation of proxy only subnet, if needed.
+- Configure Shared VPC: by default a new VPC is created within the project, but a Shared VPC can be used when the blueprint handles project creation.
+- Connect the cluster's VPC network to an on-premises network: skipped, out of scope for this blueprint
+- Enable Cloud NAT: the `vpc_create` variable allows the creation of Cloud NAT, if needed.
+- Configure Cloud DNS for GKE: not needed by this blueprint but can be enabled through the `enable_features.dns` variable of the `gke-cluster-autopilot` module.
+- Configure NodeLocal DNSCache: not needed by this blueprint
+- Create firewall rules: only the default rules created by GKE
+
+### Multitenancy
+For simplicity, multi-tenancy is not used in this blueprint.
+
+### Monitoring
+- Configure GKE alert policies: out of scope for this blueprint
+- Enable Google Cloud Managed Service for Prometheus: automatically managed by Autopilot
+- Configure control plane metrics: enabled by default
+- Enable metrics packages: out of scope for this blueprint
+
+### Maintenance
+- Create environments: out of scope for this blueprint
+- Subscribe to Pub/Sub events: out of scope for this blueprint
+- Enroll in release channels: the REGULAR channel is used by default
+- Configure maintenance windows: not configured but can be enabled through the `maintenance_config` in the `gke-cluster-autopilot` module.
+- Set Compute Engine quotas: out of scope for this blueprint
+- Configure cost controls: TBD
+- Configure billing alerts: out of scope for this blueprint
+
+## Variables
+
+| name | description | type | required | default |
+|---|---|:---:|:---:|:---:|
+| [cluster_name](variables.tf#L42) | Name of new or existing cluster. | string | ✓ | |
+| [project_id](variables.tf#L70) | Project id of existing or created project. | string | ✓ | |
+| [region](variables.tf#L75) | Region used for cluster and network resources. | string | ✓ | |
+| [cluster_create](variables.tf#L17) | Cluster configuration for newly created cluster. Set to null to use existing cluster, or create using defaults in new project. | object({…}) | | null |
+| [fleet_project_id](variables.tf#L47) | GKE Fleet project id. If null cluster project will also be used for fleet. | string | | null |
+| [prefix](variables.tf#L53) | Prefix used for resource names. | string | | "jump-0" |
+| [project_create](variables.tf#L60) | Project configuration for newly created project. Leave null to use existing project. Project creation forces VPC and cluster creation. | object({…}) | | null |
+| [registry_create](variables.tf#L80) | Create remote Docker Artifact Registry. | bool | | true |
+| [vpc_create](variables.tf#L86) | Project configuration for newly created VPC. Leave null to use existing VPC, or defaults when project creation is required. | object({…}) | | null |
+
+## Outputs
+
+| name | description | sensitive |
+|---|---|:---:|
+| [created_resources](outputs.tf#L17) | IDs of the resources created, if any. | |
+| [credentials_config](outputs.tf#L44) | Configure how Terraform authenticates to the cluster. | |
+| [fleet_host](outputs.tf#L51) | Fleet Connect Gateway host that can be used to configure the GKE provider. | |
+| [get_credentials](outputs.tf#L56) | Run one of these commands to get cluster credentials. Credentials via fleet allow reaching private clusters without no direct connectivity. | |
+| [region](outputs.tf#L70) | Region used for cluster and network resources. | |
+
diff --git a/blueprints/gke/patterns/autopilot-cluster/cluster.tf b/blueprints/gke/patterns/autopilot-cluster/cluster.tf
new file mode 100644
index 00000000..2ad385cf
--- /dev/null
+++ b/blueprints/gke/patterns/autopilot-cluster/cluster.tf
@@ -0,0 +1,134 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+locals {
+ _cluster_sa = (
+ local.cluster_create
+ ? module.cluster-service-account.0.email
+ : data.google_container_cluster.cluster.0.node_config.0.service_account
+ )
+ cluster_sa = (
+ local._cluster_sa == "default"
+ ? module.project.service_accounts.default.compute
+ : local._cluster_sa
+ )
+ cluster_sa_roles = [
+ "roles/artifactregistry.reader",
+ "roles/logging.logWriter",
+ "roles/monitoring.metricWriter",
+ "roles/monitoring.viewer",
+ "roles/stackdriver.resourceMetadata.writer"
+ ]
+ cluster_vpc = (
+ local.use_shared_vpc || !local.vpc_create
+ # cluster variable configures networking
+ ? {
+ network = try(
+ var.cluster_create.vpc.id, null
+ )
+ secondary_range_names = try(
+ var.cluster_create.vpc.secondary_range_names, null
+ )
+ subnet = try(
+ var.cluster_create.vpc.subnet_id, null
+ )
+ }
+ # VPC creation configures networking
+ : {
+ network = module.vpc.0.id
+ secondary_range_names = { pods = "pods", services = "services" }
+ subnet = values(module.vpc.0.subnet_ids)[0]
+ }
+ )
+}
+
+data "google_container_cluster" "cluster" {
+ count = !local.cluster_create ? 1 : 0
+ project = var.project_id
+ location = var.region
+ name = var.cluster_name
+}
+
+module "cluster-service-account" {
+ source = "../../../../modules/iam-service-account"
+ count = local.cluster_create ? 1 : 0
+ project_id = module.project.project_id
+ name = var.prefix
+}
+
+module "cluster" {
+ source = "../../../../modules/gke-cluster-autopilot"
+ count = local.cluster_create ? 1 : 0
+ project_id = module.project.project_id
+ deletion_protection = var.cluster_create.deletion_protection
+ name = var.cluster_name
+ location = var.region
+ vpc_config = {
+ network = local.cluster_vpc.network
+ subnetwork = local.cluster_vpc.subnet
+ secondary_range_names = local.cluster_vpc.secondary_range_names
+ master_authorized_ranges = var.cluster_create.master_authorized_ranges
+ master_ipv4_cidr_block = var.cluster_create.master_ipv4_cidr_block
+ }
+ private_cluster_config = {
+ enable_private_endpoint = true
+ master_global_access = true
+ }
+ node_config = {
+ service_account = module.cluster-service-account.0.email
+ }
+ labels = var.cluster_create.labels
+ release_channel = var.cluster_create.options.release_channel
+ backup_configs = {
+ enable_backup_agent = var.cluster_create.options.enable_backup_agent
+ }
+ enable_features = {
+ dns = {
+ provider = "CLOUD_DNS"
+ scope = "CLUSTER_SCOPE"
+ domain = "cluster.local"
+ }
+ cost_management = true
+ gateway_api = true
+ }
+ monitoring_config = {
+ enable_api_server_metrics = true
+ enable_controller_manager_metrics = true
+ enable_scheduler_metrics = true
+ }
+ logging_config = {
+ enable_api_server_logs = true
+ enable_scheduler_logs = true
+ enable_controller_manager_logs = true
+ }
+ maintenance_config = {
+ daily_window_start_time = "01:00"
+ }
+}
+
+check "cluster_networking" {
+ assert {
+ condition = (
+ local.use_shared_vpc
+ ? (
+ try(var.cluster_create.vpc.id, null) != null &&
+ try(var.cluster_create.vpc.subnet_id, null) != null
+ )
+ : true
+ )
+ error_message = "Cluster network and subnetwork are required in shared VPC mode."
+ }
+}
diff --git a/blueprints/gke/patterns/autopilot-cluster/main.tf b/blueprints/gke/patterns/autopilot-cluster/main.tf
new file mode 100644
index 00000000..7384457f
--- /dev/null
+++ b/blueprints/gke/patterns/autopilot-cluster/main.tf
@@ -0,0 +1,168 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+locals {
+ cluster_create = var.cluster_create != null || local.vpc_create
+ create_nat = local.vpc_create && try(var.vpc_create.enable_cloud_nat, false) == true
+ vpc_create = (
+ !local.use_shared_vpc && (
+ var.vpc_create != null || var.project_create != null
+ )
+ )
+ fleet_host = join("", [
+ "https://connectgateway.googleapis.com/v1/",
+ "projects/${local.fleet_project.number}/",
+ "locations/global/gkeMemberships/${var.cluster_name}"
+ ])
+ fleet_project = (
+ var.fleet_project_id == null
+ ? {
+ project_id = var.project_id
+ number = module.project.number
+ }
+ : {
+ project_id = var.fleet_project_id
+ number = module.fleet-project.0.number
+ }
+ )
+ proxy_only_subnet = (local.vpc_create && try(var.vpc_create.proxy_only_subnet, null) != null) ? [
+ {
+ ip_cidr_range = var.vpc_create.proxy_only_subnet
+ name = "proxy"
+ region = var.region
+ active = true
+ }
+ ] : null
+ use_shared_vpc = (
+ try(var.project_create.shared_vpc_host, null) != null
+ )
+}
+
+module "project" {
+ source = "../../../../modules/project"
+ parent = try(var.project_create.parent, null)
+ billing_account = try(var.project_create.billing_account, null)
+ name = var.project_id
+ project_create = var.project_create != null
+ services = compact([
+ "anthos.googleapis.com",
+ var.registry_create ? "artifactregistry.googleapis.com" : null,
+ "cloudresourcemanager.googleapis.com",
+ "connectgateway.googleapis.com",
+ "container.googleapis.com",
+ "gkeconnect.googleapis.com",
+ "gkehub.googleapis.com",
+ "stackdriver.googleapis.com"
+ ])
+ shared_vpc_service_config = !local.use_shared_vpc ? null : {
+ attach = true
+ host_project = var.project_create.shared_vpc_host
+ # grant required roles on the host project to service identities
+ service_identity_iam = {
+ "roles/compute.networkUser" = [
+ "cloudservices", "container-engine"
+ ]
+ "roles/container.hostServiceAgentUser" = [
+ "container-engine"
+ ]
+ }
+ }
+ iam_bindings_additive = merge(
+ # allow GKE fleet service identity to manage clusters in this project
+ {
+ gkehub-robot = {
+ role = "roles/gkehub.serviceAgent"
+ member = (
+ var.fleet_project_id == null
+ ? "serviceAccount:${module.project.service_accounts.robots.gkehub}"
+ : "serviceAccount:${module.fleet-project.0.service_accounts.robots.gkehub}"
+ )
+ }
+ },
+ # grant required roles to GKE node service account
+ {
+ for r in local.cluster_sa_roles : "gke-sa-${r}" => {
+ role = r
+ member = "serviceAccount:${local.cluster_sa}"
+ }
+ }
+ )
+}
+
+module "vpc" {
+ source = "../../../../modules/net-vpc"
+ count = local.vpc_create ? 1 : 0
+ project_id = module.project.project_id
+ name = coalesce(
+ try(var.vpc_create.name, null), var.prefix
+ )
+ subnets = [{
+ name = coalesce(
+ try(var.vpc_create.subnet_name, null), "${var.prefix}-default"
+ )
+ region = var.region
+ ip_cidr_range = try(
+ var.vpc_create.primary_range_nodes, "10.0.0.0/24"
+ )
+ secondary_ip_ranges = {
+ pods = try(
+ var.vpc_create.secondary_range_pods, "10.16.0.0/20"
+ )
+ services = try(
+ var.vpc_create.secondary_range_services, "10.32.0.0/24"
+ )
+ }
+ }]
+ subnets_proxy_only = local.proxy_only_subnet
+}
+
+module "fleet-project" {
+ source = "../../../../modules/project"
+ count = var.fleet_project_id == null ? 0 : 1
+ name = var.fleet_project_id
+ project_create = false
+}
+
+module "fleet" {
+ source = "../../../../modules/gke-hub"
+ project_id = local.fleet_project.project_id
+ clusters = {
+ (var.cluster_name) = (
+ var.cluster_create != null
+ ? module.cluster.0.id
+ : "projects/${var.project_id}/locations/${var.region}/clusters/${var.cluster_name}"
+ )
+ }
+}
+
+module "registry" {
+ source = "../../../../modules/artifact-registry"
+ count = var.registry_create ? 1 : 0
+ project_id = module.project.project_id
+ location = var.region
+ name = var.prefix
+ format = { docker = {} }
+ mode = { remote = true }
+}
+
+module "nat" {
+ source = "../../../../modules/net-cloudnat"
+ count = local.create_nat ? 1 : 0
+ project_id = module.project.project_id
+ region = var.region
+ name = "default"
+ router_network = local.cluster_vpc.network
+}
diff --git a/blueprints/gke/patterns/autopilot-cluster/outputs.tf b/blueprints/gke/patterns/autopilot-cluster/outputs.tf
new file mode 100644
index 00000000..b1280275
--- /dev/null
+++ b/blueprints/gke/patterns/autopilot-cluster/outputs.tf
@@ -0,0 +1,73 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+output "created_resources" {
+ description = "IDs of the resources created, if any."
+ value = merge(
+ var.project_create == null ? {} : {
+ project = module.project.project_id
+ },
+ !local.vpc_create ? {} : {
+ subnet_id = one(values(module.vpc.0.subnet_ids))
+ vpc_id = module.vpc.0.id
+ },
+ !var.registry_create ? {} : {
+ registry = module.registry.0.image_path
+ },
+ !local.cluster_create ? {} : {
+ cluster = module.cluster.0.id
+ node_service_account = module.cluster-service-account.0.email
+ },
+ !local.create_nat ? {} : {
+ router = module.nat.0.id
+ cloud_nat = module.nat.0.router.id
+ },
+ local.proxy_only_subnet == null ? {} : {
+ proxy_only_subnet = one(values(module.vpc.0.subnets_proxy_only)).id
+ },
+ )
+}
+
+output "credentials_config" {
+ description = "Configure how Terraform authenticates to the cluster."
+ value = {
+ fleet_host = local.fleet_host
+ }
+}
+
+output "fleet_host" {
+ description = "Fleet Connect Gateway host that can be used to configure the GKE provider."
+ value = local.fleet_host
+}
+
+output "get_credentials" {
+ description = "Run one of these commands to get cluster credentials. Credentials via fleet allow reaching private clusters without no direct connectivity."
+ value = {
+ direct = join("", [
+ "gcloud container clusters get-credentials ${var.cluster_name} ",
+ "--project ${var.project_id} --location ${var.region}"
+ ])
+ fleet = join("", [
+ "gcloud container fleet memberships get-credentials ${var.cluster_name}",
+ " --project ${var.project_id}"
+ ])
+ }
+}
+
+output "region" {
+ description = "Region used for cluster and network resources."
+ value = var.region
+}
diff --git a/blueprints/gke/patterns/autopilot-cluster/variables.tf b/blueprints/gke/patterns/autopilot-cluster/variables.tf
new file mode 100644
index 00000000..1b8a3d07
--- /dev/null
+++ b/blueprints/gke/patterns/autopilot-cluster/variables.tf
@@ -0,0 +1,98 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+variable "cluster_create" {
+ description = "Cluster configuration for newly created cluster. Set to null to use existing cluster, or create using defaults in new project."
+ type = object({
+ deletion_protection = optional(bool, true)
+ labels = optional(map(string))
+ master_authorized_ranges = optional(map(string), {
+ rfc-1918-10-8 = "10.0.0.0/8"
+ })
+ master_ipv4_cidr_block = optional(string, "172.16.255.0/28")
+ vpc = optional(object({
+ id = string
+ subnet_id = string
+ secondary_range_names = optional(object({
+ pods = optional(string, "pods")
+ services = optional(string, "services")
+ }), {})
+ }))
+ options = optional(object({
+ release_channel = optional(string, "REGULAR")
+ enable_backup_agent = optional(bool, false)
+ }), {})
+ })
+ default = null
+}
+
+variable "cluster_name" {
+ description = "Name of new or existing cluster."
+ type = string
+}
+
+variable "fleet_project_id" {
+ description = "GKE Fleet project id. If null cluster project will also be used for fleet."
+ type = string
+ default = null
+}
+
+variable "prefix" {
+ description = "Prefix used for resource names."
+ type = string
+ nullable = false
+ default = "jump-0"
+}
+
+variable "project_create" {
+ description = "Project configuration for newly created project. Leave null to use existing project. Project creation forces VPC and cluster creation."
+ type = object({
+ billing_account = string
+ parent = optional(string)
+ shared_vpc_host = optional(string)
+ })
+ default = null
+}
+
+variable "project_id" {
+ description = "Project id of existing or created project."
+ type = string
+}
+
+variable "region" {
+ description = "Region used for cluster and network resources."
+ type = string
+}
+
+variable "registry_create" {
+ description = "Create remote Docker Artifact Registry."
+ type = bool
+ default = true
+}
+
+variable "vpc_create" {
+ description = "Project configuration for newly created VPC. Leave null to use existing VPC, or defaults when project creation is required."
+ type = object({
+ name = optional(string)
+ subnet_name = optional(string)
+ primary_range_nodes = optional(string, "10.0.0.0/24")
+ secondary_range_pods = optional(string, "10.16.0.0/20")
+ secondary_range_services = optional(string, "10.32.0.0/24")
+ enable_cloud_nat = optional(bool, false)
+ proxy_only_subnet = optional(string)
+ })
+ default = null
+}
diff --git a/blueprints/gke/patterns/autopilot-cluster/versions.tf b/blueprints/gke/patterns/autopilot-cluster/versions.tf
new file mode 100644
index 00000000..3db0e207
--- /dev/null
+++ b/blueprints/gke/patterns/autopilot-cluster/versions.tf
@@ -0,0 +1,27 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+terraform {
+ required_version = ">= 1.7.0"
+ required_providers {
+ google = {
+ source = "hashicorp/google"
+ version = ">= 5.11.0, < 6.0.0" # tftest
+ }
+ google-beta = {
+ source = "hashicorp/google-beta"
+ version = ">= 5.11.0, < 6.0.0" # tftest
+ }
+ }
+}
diff --git a/blueprints/gke/patterns/batch/README.md b/blueprints/gke/patterns/batch/README.md
new file mode 100644
index 00000000..dbd92961
--- /dev/null
+++ b/blueprints/gke/patterns/batch/README.md
@@ -0,0 +1,62 @@
+# Batch Processing on GKE with Kueue
+
+
+- [Introduction](#introduction)
+- [Requirements](#requirements)
+- [Cluster authentication](#cluster-authentication)
+- [Kueue Configuration](#kueue-configuration)
+- [Sample Configuration](#sample-configuration)
+- [Variables](#variables)
+
+
+## Introduction
+
+
+
+
+This blueprint shows how to deploy a batch system using [Kueue](https://kueue.sigs.k8s.io/docs/overview/) to perform job queuing on Google Kubernetes Engine (GKE) using Terraform.
+
+Kueue is a Cloud Native Job scheduler that works with the default Kubernetes scheduler, the Job controller, and the cluster autoscaler to provide an end-to-end batch system. Kueue implements Job queueing, deciding when Jobs should wait and when they should start, based on quotas and a hierarchy for sharing resources fairly among teams.
+
+
+## Requirements
+
+This blueprint assumes the GKE cluster already exists. We recommend using the accompanying [Autopilot Cluster Pattern](../autopilot-cluster) to deploy a cluster according to Google's best practices. Once you have the cluster up-and-running, you can use this blueprint to deploy Kueue in it.
+
+The Kueue manifests use container images hosted by registry.k8s.io, which means that the subnet where the GKE cluster is deployed needs to have Internet connectivity to download the images. If you're using the provided [Autopilot Cluster Pattern](../autopilot-cluster), you can set the `enable_cloud_nat` option of the `vpc_create` variable.
+
+## Cluster authentication
+Once you have a cluster with Internet connectivity, create a `terraform.tfvars` and setup the `credentials_config` variable. We recommend using Anthos Fleet to simplify accessing the control plane.
+
+## Kueue Configuration
+
+Only two variables are available to control Kueue's configuration:
+- `teams_namespaces` which controls the namespaces used by different teams to run jobs.
+- `kueue_namespace` which controls the namepsace to deploy Kueue's own resources.
+
+Any other configuration can be applied by directly modifying the YAML manifests under the [manifest-templates](manifest-templates) directory.
+
+## Sample Configuration
+
+The following template as a starting point for your terraform.tfvars
+```tfvars
+credentials_config = {
+ kubeconfig = {
+ path = "~/.kube/config"
+ }
+}
+teams_namespaces = [
+ "team-a",
+ "team-b"
+]
+```
+
+## Variables
+
+| name | description | type | required | default |
+|---|---|:---:|:---:|:---:|
+| [credentials_config](variables.tf#L17) | Configure how Terraform authenticates to the cluster. | object({…}) | ✓ | |
+| [kueue_namespace](variables.tf#L36) | Namespaces of the teams running jobs in the clusters. | string | | "kueue-system" |
+| [team_namespaces](variables.tf#L43) | Namespaces of the teams running jobs in the clusters. | list(string) | | […] |
+| [templates_path](variables.tf#L53) | Path where manifest templates will be read from. Set to null to use the default manifests. | string | | null |
+
diff --git a/blueprints/gke/patterns/batch/create_jobs.sh b/blueprints/gke/patterns/batch/create_jobs.sh
new file mode 100755
index 00000000..42b379a7
--- /dev/null
+++ b/blueprints/gke/patterns/batch/create_jobs.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FILE_POSITIONS=$(($# - 1))
+SLEEP_TIME=${!#}
+
+while :
+do
+ for i in $(seq 1 $FILE_POSITIONS); do
+ kubectl create -f ${!i}
+ done
+ sleep ${SLEEP_TIME:-10}
+done
diff --git a/blueprints/gke/patterns/batch/job-team-a.yaml b/blueprints/gke/patterns/batch/job-team-a.yaml
new file mode 100755
index 00000000..b0510750
--- /dev/null
+++ b/blueprints/gke/patterns/batch/job-team-a.yaml
@@ -0,0 +1,30 @@
+# skip boilerplate check
+
+apiVersion: batch/v1
+kind: Job
+metadata:
+ namespace: team-a # Job under team-a namespace
+ generateName: sample-job-
+ annotations:
+ kueue.x-k8s.io/queue-name: local-queue # Point to the LocalQueue
+spec:
+ ttlSecondsAfterFinished: 60 # Job will be deleted after 60 seconds
+ parallelism: 3 # This Job will have 3 replicas running at the same time
+ completions: 3 # This Job requires 3 completions
+ suspend: true # Set to true to allow Kueue to control the Job when it starts
+ template:
+ spec:
+ containers:
+ - name: dummy-job
+ image: gcr.io/k8s-staging-perf-tests/sleep:latest
+ args: ["10s"] # Sleep for 10 seconds
+ resources:
+ requests:
+ cpu: "500m"
+ memory: "512Mi"
+ ephemeral-storage: "512Mi"
+ limits:
+ cpu: "500m"
+ memory: "512Mi"
+ ephemeral-storage: "512Mi"
+ restartPolicy: Never
diff --git a/blueprints/gke/patterns/batch/job-team-b.yaml b/blueprints/gke/patterns/batch/job-team-b.yaml
new file mode 100755
index 00000000..57a8e6b7
--- /dev/null
+++ b/blueprints/gke/patterns/batch/job-team-b.yaml
@@ -0,0 +1,30 @@
+# skip boilerplate check
+
+apiVersion: batch/v1
+kind: Job
+metadata:
+ namespace: team-b # Job under team-a namespace
+ generateName: sample-job-
+ annotations:
+ kueue.x-k8s.io/queue-name: local-queue # Point to the LocalQueue
+spec:
+ ttlSecondsAfterFinished: 60 # Job will be deleted after 60 seconds
+ parallelism: 3 # This Job will have 3 replicas running at the same time
+ completions: 3 # This Job requires 3 completions
+ suspend: true # Set to true to allow Kueue to control the Job when it starts
+ template:
+ spec:
+ containers:
+ - name: dummy-job
+ image: gcr.io/k8s-staging-perf-tests/sleep:latest
+ args: ["10s"] # Sleep for 10 seconds
+ resources:
+ requests:
+ cpu: "500m"
+ memory: "512Mi"
+ ephemeral-storage: "512Mi"
+ limits:
+ cpu: "500m"
+ memory: "512Mi"
+ ephemeral-storage: "512Mi"
+ restartPolicy: Never
diff --git a/blueprints/gke/patterns/batch/main.tf b/blueprints/gke/patterns/batch/main.tf
new file mode 100644
index 00000000..341ab961
--- /dev/null
+++ b/blueprints/gke/patterns/batch/main.tf
@@ -0,0 +1,81 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+locals {
+ wl_templates_path = "${path.module}/manifest-templates"
+}
+
+
+resource "kubectl_manifest" "kueue_namespace_manifest" {
+ yaml_body = < matches
+ that of any node on which a pod of the set
+ of pods is running
+ properties:
+ labelSelector:
+ description: A label query over a set
+ of resources, in this case pods.
+ properties:
+ matchExpressions:
+ description: matchExpressions is a
+ list of label selector requirements.
+ The requirements are ANDed.
+ items:
+ description: A label selector requirement
+ is a selector that contains values,
+ a key, and an operator that relates
+ the key and values.
+ properties:
+ key:
+ description: key is the label
+ key that the selector applies
+ to.
+ type: string
+ operator:
+ description: operator represents
+ a key's relationship to a
+ set of values. Valid operators
+ are In, NotIn, Exists and
+ DoesNotExist.
+ type: string
+ values:
+ description: values is an array
+ of string values. If the operator
+ is In or NotIn, the values
+ array must be non-empty. If
+ the operator is Exists or
+ DoesNotExist, the values array
+ must be empty. This array
+ is replaced during a strategic
+ merge patch.
+ items:
+ type: string
+ type: array
+ required:
+ - key
+ - operator
+ type: object
+ type: array
+ matchLabels:
+ additionalProperties:
+ type: string
+ description: matchLabels is a map
+ of {key,value} pairs. A single {key,value}
+ in the matchLabels map is equivalent
+ to an element of matchExpressions,
+ whose key field is "key", the operator
+ is "In", and the values array contains
+ only "value". The requirements are
+ ANDed.
+ type: object
+ type: object
+ x-kubernetes-map-type: atomic
+ namespaceSelector:
+ description: A label query over the set
+ of namespaces that the term applies
+ to. The term is applied to the union
+ of the namespaces selected by this field
+ and the ones listed in the namespaces
+ field. null selector and null or empty
+ namespaces list means "this pod's namespace".
+ An empty selector ({}) matches all namespaces.
+ properties:
+ matchExpressions:
+ description: matchExpressions is a
+ list of label selector requirements.
+ The requirements are ANDed.
+ items:
+ description: A label selector requirement
+ is a selector that contains values,
+ a key, and an operator that relates
+ the key and values.
+ properties:
+ key:
+ description: key is the label
+ key that the selector applies
+ to.
+ type: string
+ operator:
+ description: operator represents
+ a key's relationship to a
+ set of values. Valid operators
+ are In, NotIn, Exists and
+ DoesNotExist.
+ type: string
+ values:
+ description: values is an array
+ of string values. If the operator
+ is In or NotIn, the values
+ array must be non-empty. If
+ the operator is Exists or
+ DoesNotExist, the values array
+ must be empty. This array
+ is replaced during a strategic
+ merge patch.
+ items:
+ type: string
+ type: array
+ required:
+ - key
+ - operator
+ type: object
+ type: array
+ matchLabels:
+ additionalProperties:
+ type: string
+ description: matchLabels is a map
+ of {key,value} pairs. A single {key,value}
+ in the matchLabels map is equivalent
+ to an element of matchExpressions,
+ whose key field is "key", the operator
+ is "In", and the values array contains
+ only "value". The requirements are
+ ANDed.
+ type: object
+ type: object
+ x-kubernetes-map-type: atomic
+ namespaces:
+ description: namespaces specifies a static
+ list of namespace names that the term
+ applies to. The term is applied to the
+ union of the namespaces listed in this
+ field and the ones selected by namespaceSelector.
+ null or empty namespaces list and null
+ namespaceSelector means "this pod's
+ namespace".
+ items:
+ type: string
+ type: array
+ topologyKey:
+ description: This pod should be co-located
+ (affinity) or not co-located (anti-affinity)
+ with the pods matching the labelSelector
+ in the specified namespaces, where co-located
+ is defined as running on a node whose
+ value of the label with key topologyKey
+ matches that of any node on which any
+ of the selected pods is running. Empty
+ topologyKey is not allowed.
+ type: string
+ required:
+ - topologyKey
+ type: object
+ type: array
+ type: object
+ podAntiAffinity:
+ description: Describes pod anti-affinity scheduling
+ rules (e.g. avoid putting this pod in the same
+ node, zone, etc. as some other pod(s)).
+ properties:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ description: The scheduler will prefer to schedule
+ pods to nodes that satisfy the anti-affinity
+ expressions specified by this field, but it
+ may choose a node that violates one or more
+ of the expressions. The node that is most
+ preferred is the one with the greatest sum
+ of weights, i.e. for each node that meets
+ all of the scheduling requirements (resource
+ request, requiredDuringScheduling anti-affinity
+ expressions, etc.), compute a sum by iterating
+ through the elements of this field and adding
+ "weight" to the sum if the node has pods which
+ matches the corresponding podAffinityTerm;
+ the node(s) with the highest sum are the most
+ preferred.
+ items:
+ description: The weights of all of the matched
+ WeightedPodAffinityTerm fields are added
+ per-node to find the most preferred node(s)
+ properties:
+ podAffinityTerm:
+ description: Required. A pod affinity
+ term, associated with the corresponding
+ weight.
+ properties:
+ labelSelector:
+ description: A label query over a
+ set of resources, in this case pods.
+ properties:
+ matchExpressions:
+ description: matchExpressions
+ is a list of label selector
+ requirements. The requirements
+ are ANDed.
+ items:
+ description: A label selector
+ requirement is a selector
+ that contains values, a key,
+ and an operator that relates
+ the key and values.
+ properties:
+ key:
+ description: key is the
+ label key that the selector
+ applies to.
+ type: string
+ operator:
+ description: operator represents
+ a key's relationship to
+ a set of values. Valid
+ operators are In, NotIn,
+ Exists and DoesNotExist.
+ type: string
+ values:
+ description: values is an
+ array of string values.
+ If the operator is In
+ or NotIn, the values array
+ must be non-empty. If
+ the operator is Exists
+ or DoesNotExist, the values
+ array must be empty. This
+ array is replaced during
+ a strategic merge patch.
+ items:
+ type: string
+ type: array
+ required:
+ - key
+ - operator
+ type: object
+ type: array
+ matchLabels:
+ additionalProperties:
+ type: string
+ description: matchLabels is a
+ map of {key,value} pairs. A
+ single {key,value} in the matchLabels
+ map is equivalent to an element
+ of matchExpressions, whose key
+ field is "key", the operator
+ is "In", and the values array
+ contains only "value". The requirements
+ are ANDed.
+ type: object
+ type: object
+ x-kubernetes-map-type: atomic
+ namespaceSelector:
+ description: A label query over the
+ set of namespaces that the term
+ applies to. The term is applied
+ to the union of the namespaces selected
+ by this field and the ones listed
+ in the namespaces field. null selector
+ and null or empty namespaces list
+ means "this pod's namespace". An
+ empty selector ({}) matches all
+ namespaces.
+ properties:
+ matchExpressions:
+ description: matchExpressions
+ is a list of label selector
+ requirements. The requirements
+ are ANDed.
+ items:
+ description: A label selector
+ requirement is a selector
+ that contains values, a key,
+ and an operator that relates
+ the key and values.
+ properties:
+ key:
+ description: key is the
+ label key that the selector
+ applies to.
+ type: string
+ operator:
+ description: operator represents
+ a key's relationship to
+ a set of values. Valid
+ operators are In, NotIn,
+ Exists and DoesNotExist.
+ type: string
+ values:
+ description: values is an
+ array of string values.
+ If the operator is In
+ or NotIn, the values array
+ must be non-empty. If
+ the operator is Exists
+ or DoesNotExist, the values
+ array must be empty. This
+ array is replaced during
+ a strategic merge patch.
+ items:
+ type: string
+ type: array
+ required:
+ - key
+ - operator
+ type: object
+ type: array
+ matchLabels:
+ additionalProperties:
+ type: string
+ description: matchLabels is a
+ map of {key,value} pairs. A
+ single {key,value} in the matchLabels
+ map is equivalent to an element
+ of matchExpressions, whose key
+ field is "key", the operator
+ is "In", and the values array
+ contains only "value". The requirements
+ are ANDed.
+ type: object
+ type: object
+ x-kubernetes-map-type: atomic
+ namespaces:
+ description: namespaces specifies
+ a static list of namespace names
+ that the term applies to. The term
+ is applied to the union of the namespaces
+ listed in this field and the ones
+ selected by namespaceSelector. null
+ or empty namespaces list and null
+ namespaceSelector means "this pod's
+ namespace".
+ items:
+ type: string
+ type: array
+ topologyKey:
+ description: This pod should be co-located
+ (affinity) or not co-located (anti-affinity)
+ with the pods matching the labelSelector
+ in the specified namespaces, where
+ co-located is defined as running
+ on a node whose value of the label
+ with key topologyKey matches that
+ of any node on which any of the
+ selected pods is running. Empty
+ topologyKey is not allowed.
+ type: string
+ required:
+ - topologyKey
+ type: object
+ weight:
+ description: weight associated with matching
+ the corresponding podAffinityTerm, in
+ the range 1-100.
+ format: int32
+ type: integer
+ required:
+ - podAffinityTerm
+ - weight
+ type: object
+ type: array
+ requiredDuringSchedulingIgnoredDuringExecution:
+ description: If the anti-affinity requirements
+ specified by this field are not met at scheduling
+ time, the pod will not be scheduled onto the
+ node. If the anti-affinity requirements specified
+ by this field cease to be met at some point
+ during pod execution (e.g. due to a pod label
+ update), the system may or may not try to
+ eventually evict the pod from its node. When
+ there are multiple elements, the lists of
+ nodes corresponding to each podAffinityTerm
+ are intersected, i.e. all terms must be satisfied.
+ items:
+ description: Defines a set of pods (namely
+ those matching the labelSelector relative
+ to the given namespace(s)) that this pod
+ should be co-located (affinity) or not co-located
+ (anti-affinity) with, where co-located is
+ defined as running on a node whose value
+ of the label with key matches
+ that of any node on which a pod of the set
+ of pods is running
+ properties:
+ labelSelector:
+ description: A label query over a set
+ of resources, in this case pods.
+ properties:
+ matchExpressions:
+ description: matchExpressions is a
+ list of label selector requirements.
+ The requirements are ANDed.
+ items:
+ description: A label selector requirement
+ is a selector that contains values,
+ a key, and an operator that relates
+ the key and values.
+ properties:
+ key:
+ description: key is the label
+ key that the selector applies
+ to.
+ type: string
+ operator:
+ description: operator represents
+ a key's relationship to a
+ set of values. Valid operators
+ are In, NotIn, Exists and
+ DoesNotExist.
+ type: string
+ values:
+ description: values is an array
+ of string values. If the operator
+ is In or NotIn, the values
+ array must be non-empty. If
+ the operator is Exists or
+ DoesNotExist, the values array
+ must be empty. This array
+ is replaced during a strategic
+ merge patch.
+ items:
+ type: string
+ type: array
+ required:
+ - key
+ - operator
+ type: object
+ type: array
+ matchLabels:
+ additionalProperties:
+ type: string
+ description: matchLabels is a map
+ of {key,value} pairs. A single {key,value}
+ in the matchLabels map is equivalent
+ to an element of matchExpressions,
+ whose key field is "key", the operator
+ is "In", and the values array contains
+ only "value". The requirements are
+ ANDed.
+ type: object
+ type: object
+ x-kubernetes-map-type: atomic
+ namespaceSelector:
+ description: A label query over the set
+ of namespaces that the term applies
+ to. The term is applied to the union
+ of the namespaces selected by this field
+ and the ones listed in the namespaces
+ field. null selector and null or empty
+ namespaces list means "this pod's namespace".
+ An empty selector ({}) matches all namespaces.
+ properties:
+ matchExpressions:
+ description: matchExpressions is a
+ list of label selector requirements.
+ The requirements are ANDed.
+ items:
+ description: A label selector requirement
+ is a selector that contains values,
+ a key, and an operator that relates
+ the key and values.
+ properties:
+ key:
+ description: key is the label
+ key that the selector applies
+ to.
+ type: string
+ operator:
+ description: operator represents
+ a key's relationship to a
+ set of values. Valid operators
+ are In, NotIn, Exists and
+ DoesNotExist.
+ type: string
+ values:
+ description: values is an array
+ of string values. If the operator
+ is In or NotIn, the values
+ array must be non-empty. If
+ the operator is Exists or
+ DoesNotExist, the values array
+ must be empty. This array
+ is replaced during a strategic
+ merge patch.
+ items:
+ type: string
+ type: array
+ required:
+ - key
+ - operator
+ type: object
+ type: array
+ matchLabels:
+ additionalProperties:
+ type: string
+ description: matchLabels is a map
+ of {key,value} pairs. A single {key,value}
+ in the matchLabels map is equivalent
+ to an element of matchExpressions,
+ whose key field is "key", the operator
+ is "In", and the values array contains
+ only "value". The requirements are
+ ANDed.
+ type: object
+ type: object
+ x-kubernetes-map-type: atomic
+ namespaces:
+ description: namespaces specifies a static
+ list of namespace names that the term
+ applies to. The term is applied to the
+ union of the namespaces listed in this
+ field and the ones selected by namespaceSelector.
+ null or empty namespaces list and null
+ namespaceSelector means "this pod's
+ namespace".
+ items:
+ type: string
+ type: array
+ topologyKey:
+ description: This pod should be co-located
+ (affinity) or not co-located (anti-affinity)
+ with the pods matching the labelSelector
+ in the specified namespaces, where co-located
+ is defined as running on a node whose
+ value of the label with key topologyKey
+ matches that of any node on which any
+ of the selected pods is running. Empty
+ topologyKey is not allowed.
+ type: string
+ required:
+ - topologyKey
+ type: object
+ type: array
+ type: object
+ type: object
+ automountServiceAccountToken:
+ description: AutomountServiceAccountToken indicates
+ whether a service account token should be automatically
+ mounted.
+ type: boolean
+ containers:
+ description: List of containers belonging to the pod.
+ Containers cannot currently be added or removed. There
+ must be at least one container in a Pod. Cannot be
+ updated.
+ items:
+ description: A single application container that you
+ want to run within a pod.
+ properties:
+ args:
+ description: 'Arguments to the entrypoint. The
+ container image''s CMD is used if this is not
+ provided. Variable references $(VAR_NAME) are
+ expanded using the container''s environment.
+ If a variable cannot be resolved, the reference
+ in the input string will be unchanged. Double
+ $$ are reduced to a single $, which allows for
+ escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)"
+ will produce the string literal "$(VAR_NAME)".
+ Escaped references will never be expanded, regardless
+ of whether the variable exists or not. Cannot
+ be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell'
+ items:
+ type: string
+ type: array
+ command:
+ description: 'Entrypoint array. Not executed within
+ a shell. The container image''s ENTRYPOINT is
+ used if this is not provided. Variable references
+ $(VAR_NAME) are expanded using the container''s
+ environment. If a variable cannot be resolved,
+ the reference in the input string will be unchanged.
+ Double $$ are reduced to a single $, which allows
+ for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)"
+ will produce the string literal "$(VAR_NAME)".
+ Escaped references will never be expanded, regardless
+ of whether the variable exists or not. Cannot
+ be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell'
+ items:
+ type: string
+ type: array
+ env:
+ description: List of environment variables to
+ set in the container. Cannot be updated.
+ items:
+ description: EnvVar represents an environment
+ variable present in a Container.
+ properties:
+ name:
+ description: Name of the environment variable.
+ Must be a C_IDENTIFIER.
+ type: string
+ value:
+ description: 'Variable references $(VAR_NAME)
+ are expanded using the previously defined
+ environment variables in the container
+ and any service environment variables.
+ If a variable cannot be resolved, the
+ reference in the input string will be
+ unchanged. Double $$ are reduced to a
+ single $, which allows for escaping the
+ $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)"
+ will produce the string literal "$(VAR_NAME)".
+ Escaped references will never be expanded,
+ regardless of whether the variable exists
+ or not. Defaults to "".'
+ type: string
+ valueFrom:
+ description: Source for the environment
+ variable's value. Cannot be used if value
+ is not empty.
+ properties:
+ configMapKeyRef:
+ description: Selects a key of a ConfigMap.
+ properties:
+ key:
+ description: The key to select.
+ type: string
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields.
+ apiVersion, kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the
+ ConfigMap or its key must be defined
+ type: boolean
+ required:
+ - key
+ type: object
+ x-kubernetes-map-type: atomic
+ fieldRef:
+ description: 'Selects a field of the
+ pod: supports metadata.name, metadata.namespace,
+ `metadata.labels['''']`, `metadata.annotations['''']`,
+ spec.nodeName, spec.serviceAccountName,
+ status.hostIP, status.podIP, status.podIPs.'
+ properties:
+ apiVersion:
+ description: Version of the schema
+ the FieldPath is written in terms
+ of, defaults to "v1".
+ type: string
+ fieldPath:
+ description: Path of the field to
+ select in the specified API version.
+ type: string
+ required:
+ - fieldPath
+ type: object
+ x-kubernetes-map-type: atomic
+ resourceFieldRef:
+ description: 'Selects a resource of
+ the container: only resources limits
+ and requests (limits.cpu, limits.memory,
+ limits.ephemeral-storage, requests.cpu,
+ requests.memory and requests.ephemeral-storage)
+ are currently supported.'
+ properties:
+ containerName:
+ description: 'Container name: required
+ for volumes, optional for env
+ vars'
+ type: string
+ divisor:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Specifies the output
+ format of the exposed resources,
+ defaults to "1"
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ resource:
+ description: 'Required: resource
+ to select'
+ type: string
+ required:
+ - resource
+ type: object
+ x-kubernetes-map-type: atomic
+ secretKeyRef:
+ description: Selects a key of a secret
+ in the pod's namespace
+ properties:
+ key:
+ description: The key of the secret
+ to select from. Must be a valid
+ secret key.
+ type: string
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields.
+ apiVersion, kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the
+ Secret or its key must be defined
+ type: boolean
+ required:
+ - key
+ type: object
+ x-kubernetes-map-type: atomic
+ type: object
+ required:
+ - name
+ type: object
+ type: array
+ envFrom:
+ description: List of sources to populate environment
+ variables in the container. The keys defined
+ within a source must be a C_IDENTIFIER. All
+ invalid keys will be reported as an event when
+ the container is starting. When a key exists
+ in multiple sources, the value associated with
+ the last source will take precedence. Values
+ defined by an Env with a duplicate key will
+ take precedence. Cannot be updated.
+ items:
+ description: EnvFromSource represents the source
+ of a set of ConfigMaps
+ properties:
+ configMapRef:
+ description: The ConfigMap to select from
+ properties:
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the ConfigMap
+ must be defined
+ type: boolean
+ type: object
+ x-kubernetes-map-type: atomic
+ prefix:
+ description: An optional identifier to prepend
+ to each key in the ConfigMap. Must be
+ a C_IDENTIFIER.
+ type: string
+ secretRef:
+ description: The Secret to select from
+ properties:
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the Secret
+ must be defined
+ type: boolean
+ type: object
+ x-kubernetes-map-type: atomic
+ type: object
+ type: array
+ image:
+ description: 'Container image name. More info:
+ https://kubernetes.io/docs/concepts/containers/images
+ This field is optional to allow higher level
+ config management to default or override container
+ images in workload controllers like Deployments
+ and StatefulSets.'
+ type: string
+ imagePullPolicy:
+ description: 'Image pull policy. One of Always,
+ Never, IfNotPresent. Defaults to Always if :latest
+ tag is specified, or IfNotPresent otherwise.
+ Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images'
+ type: string
+ lifecycle:
+ description: Actions that the management system
+ should take in response to container lifecycle
+ events. Cannot be updated.
+ properties:
+ postStart:
+ description: 'PostStart is called immediately
+ after a container is created. If the handler
+ fails, the container is terminated and restarted
+ according to its restart policy. Other management
+ of the container blocks until the hook completes.
+ More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks'
+ properties:
+ exec:
+ description: Exec specifies the action
+ to take.
+ properties:
+ command:
+ description: Command is the command
+ line to execute inside the container,
+ the working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it
+ is not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to
+ explicitly call out to that shell.
+ Exit status of 0 is treated as live/healthy
+ and non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http
+ request to perform.
+ properties:
+ host:
+ description: Host name to connect
+ to, defaults to the pod IP. You
+ probably want to set "Host" in httpHeaders
+ instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set
+ in the request. HTTP allows repeated
+ headers.
+ items:
+ description: HTTPHeader describes
+ a custom header to be used in
+ HTTP probes
+ properties:
+ name:
+ description: The header field
+ name
+ type: string
+ value:
+ description: The header field
+ value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the
+ HTTP server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ tcpSocket:
+ description: Deprecated. TCPSocket is
+ NOT supported as a LifecycleHandler
+ and kept for the backward compatibility.
+ There are no validation of this field
+ and lifecycle hooks will fail in runtime
+ when tcp handler is specified.
+ properties:
+ host:
+ description: 'Optional: Host name
+ to connect to, defaults to the pod
+ IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ type: object
+ preStop:
+ description: 'PreStop is called immediately
+ before a container is terminated due to
+ an API request or management event such
+ as liveness/startup probe failure, preemption,
+ resource contention, etc. The handler is
+ not called if the container crashes or exits.
+ The Pod''s termination grace period countdown
+ begins before the PreStop hook is executed.
+ Regardless of the outcome of the handler,
+ the container will eventually terminate
+ within the Pod''s termination grace period
+ (unless delayed by finalizers). Other management
+ of the container blocks until the hook completes
+ or until the termination grace period is
+ reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks'
+ properties:
+ exec:
+ description: Exec specifies the action
+ to take.
+ properties:
+ command:
+ description: Command is the command
+ line to execute inside the container,
+ the working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it
+ is not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to
+ explicitly call out to that shell.
+ Exit status of 0 is treated as live/healthy
+ and non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http
+ request to perform.
+ properties:
+ host:
+ description: Host name to connect
+ to, defaults to the pod IP. You
+ probably want to set "Host" in httpHeaders
+ instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set
+ in the request. HTTP allows repeated
+ headers.
+ items:
+ description: HTTPHeader describes
+ a custom header to be used in
+ HTTP probes
+ properties:
+ name:
+ description: The header field
+ name
+ type: string
+ value:
+ description: The header field
+ value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the
+ HTTP server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ tcpSocket:
+ description: Deprecated. TCPSocket is
+ NOT supported as a LifecycleHandler
+ and kept for the backward compatibility.
+ There are no validation of this field
+ and lifecycle hooks will fail in runtime
+ when tcp handler is specified.
+ properties:
+ host:
+ description: 'Optional: Host name
+ to connect to, defaults to the pod
+ IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ type: object
+ type: object
+ livenessProbe:
+ description: 'Periodic probe of container liveness.
+ Container will be restarted if the probe fails.
+ Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ properties:
+ exec:
+ description: Exec specifies the action to
+ take.
+ properties:
+ command:
+ description: Command is the command line
+ to execute inside the container, the
+ working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it is
+ not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to explicitly
+ call out to that shell. Exit status
+ of 0 is treated as live/healthy and
+ non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ failureThreshold:
+ description: Minimum consecutive failures
+ for the probe to be considered failed after
+ having succeeded. Defaults to 3. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ grpc:
+ description: GRPC specifies an action involving
+ a GRPC port. This is a beta field and requires
+ enabling GRPCContainerProbe feature gate.
+ properties:
+ port:
+ description: Port number of the gRPC service.
+ Number must be in the range 1 to 65535.
+ format: int32
+ type: integer
+ service:
+ description: "Service is the name of the
+ service to place in the gRPC HealthCheckRequest
+ (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
+ \n If this is not specified, the default
+ behavior is defined by gRPC."
+ type: string
+ required:
+ - port
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http request
+ to perform.
+ properties:
+ host:
+ description: Host name to connect to,
+ defaults to the pod IP. You probably
+ want to set "Host" in httpHeaders instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set in
+ the request. HTTP allows repeated headers.
+ items:
+ description: HTTPHeader describes a
+ custom header to be used in HTTP probes
+ properties:
+ name:
+ description: The header field name
+ type: string
+ value:
+ description: The header field value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the HTTP
+ server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ initialDelaySeconds:
+ description: 'Number of seconds after the
+ container has started before liveness probes
+ are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ periodSeconds:
+ description: How often (in seconds) to perform
+ the probe. Default to 10 seconds. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ successThreshold:
+ description: Minimum consecutive successes
+ for the probe to be considered successful
+ after having failed. Defaults to 1. Must
+ be 1 for liveness and startup. Minimum value
+ is 1.
+ format: int32
+ type: integer
+ tcpSocket:
+ description: TCPSocket specifies an action
+ involving a TCP port.
+ properties:
+ host:
+ description: 'Optional: Host name to connect
+ to, defaults to the pod IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds
+ the pod needs to terminate gracefully upon
+ probe failure. The grace period is the duration
+ in seconds after the processes running in
+ the pod are sent a termination signal and
+ the time when the processes are forcibly
+ halted with a kill signal. Set this value
+ longer than the expected cleanup time for
+ your process. If this value is nil, the
+ pod's terminationGracePeriodSeconds will
+ be used. Otherwise, this value overrides
+ the value provided by the pod spec. Value
+ must be non-negative integer. The value
+ zero indicates stop immediately via the
+ kill signal (no opportunity to shut down).
+ This is a beta field and requires enabling
+ ProbeTerminationGracePeriod feature gate.
+ Minimum value is 1. spec.terminationGracePeriodSeconds
+ is used if unset.
+ format: int64
+ type: integer
+ timeoutSeconds:
+ description: 'Number of seconds after which
+ the probe times out. Defaults to 1 second.
+ Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ type: object
+ name:
+ description: Name of the container specified as
+ a DNS_LABEL. Each container in a pod must have
+ a unique name (DNS_LABEL). Cannot be updated.
+ type: string
+ ports:
+ description: List of ports to expose from the
+ container. Not specifying a port here DOES NOT
+ prevent that port from being exposed. Any port
+ which is listening on the default "0.0.0.0"
+ address inside a container will be accessible
+ from the network. Modifying this array with
+ strategic merge patch may corrupt the data.
+ For more information See https://github.com/kubernetes/kubernetes/issues/108255.
+ Cannot be updated.
+ items:
+ description: ContainerPort represents a network
+ port in a single container.
+ properties:
+ containerPort:
+ description: Number of port to expose on
+ the pod's IP address. This must be a valid
+ port number, 0 < x < 65536.
+ format: int32
+ type: integer
+ hostIP:
+ description: What host IP to bind the external
+ port to.
+ type: string
+ hostPort:
+ description: Number of port to expose on
+ the host. If specified, this must be a
+ valid port number, 0 < x < 65536. If HostNetwork
+ is specified, this must match ContainerPort.
+ Most containers do not need this.
+ format: int32
+ type: integer
+ name:
+ description: If specified, this must be
+ an IANA_SVC_NAME and unique within the
+ pod. Each named port in a pod must have
+ a unique name. Name for the port that
+ can be referred to by services.
+ type: string
+ protocol:
+ default: TCP
+ description: Protocol for port. Must be
+ UDP, TCP, or SCTP. Defaults to "TCP".
+ type: string
+ required:
+ - containerPort
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - containerPort
+ - protocol
+ x-kubernetes-list-type: map
+ readinessProbe:
+ description: 'Periodic probe of container service
+ readiness. Container will be removed from service
+ endpoints if the probe fails. Cannot be updated.
+ More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ properties:
+ exec:
+ description: Exec specifies the action to
+ take.
+ properties:
+ command:
+ description: Command is the command line
+ to execute inside the container, the
+ working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it is
+ not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to explicitly
+ call out to that shell. Exit status
+ of 0 is treated as live/healthy and
+ non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ failureThreshold:
+ description: Minimum consecutive failures
+ for the probe to be considered failed after
+ having succeeded. Defaults to 3. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ grpc:
+ description: GRPC specifies an action involving
+ a GRPC port. This is a beta field and requires
+ enabling GRPCContainerProbe feature gate.
+ properties:
+ port:
+ description: Port number of the gRPC service.
+ Number must be in the range 1 to 65535.
+ format: int32
+ type: integer
+ service:
+ description: "Service is the name of the
+ service to place in the gRPC HealthCheckRequest
+ (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
+ \n If this is not specified, the default
+ behavior is defined by gRPC."
+ type: string
+ required:
+ - port
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http request
+ to perform.
+ properties:
+ host:
+ description: Host name to connect to,
+ defaults to the pod IP. You probably
+ want to set "Host" in httpHeaders instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set in
+ the request. HTTP allows repeated headers.
+ items:
+ description: HTTPHeader describes a
+ custom header to be used in HTTP probes
+ properties:
+ name:
+ description: The header field name
+ type: string
+ value:
+ description: The header field value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the HTTP
+ server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ initialDelaySeconds:
+ description: 'Number of seconds after the
+ container has started before liveness probes
+ are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ periodSeconds:
+ description: How often (in seconds) to perform
+ the probe. Default to 10 seconds. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ successThreshold:
+ description: Minimum consecutive successes
+ for the probe to be considered successful
+ after having failed. Defaults to 1. Must
+ be 1 for liveness and startup. Minimum value
+ is 1.
+ format: int32
+ type: integer
+ tcpSocket:
+ description: TCPSocket specifies an action
+ involving a TCP port.
+ properties:
+ host:
+ description: 'Optional: Host name to connect
+ to, defaults to the pod IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds
+ the pod needs to terminate gracefully upon
+ probe failure. The grace period is the duration
+ in seconds after the processes running in
+ the pod are sent a termination signal and
+ the time when the processes are forcibly
+ halted with a kill signal. Set this value
+ longer than the expected cleanup time for
+ your process. If this value is nil, the
+ pod's terminationGracePeriodSeconds will
+ be used. Otherwise, this value overrides
+ the value provided by the pod spec. Value
+ must be non-negative integer. The value
+ zero indicates stop immediately via the
+ kill signal (no opportunity to shut down).
+ This is a beta field and requires enabling
+ ProbeTerminationGracePeriod feature gate.
+ Minimum value is 1. spec.terminationGracePeriodSeconds
+ is used if unset.
+ format: int64
+ type: integer
+ timeoutSeconds:
+ description: 'Number of seconds after which
+ the probe times out. Defaults to 1 second.
+ Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ type: object
+ resources:
+ description: 'Compute Resources required by this
+ container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ properties:
+ claims:
+ description: "Claims lists the names of resources,
+ defined in spec.resourceClaims, that are
+ used by this container. \n This is an alpha
+ field and requires enabling the DynamicResourceAllocation
+ feature gate. \n This field is immutable.
+ It can only be set for containers."
+ items:
+ description: ResourceClaim references one
+ entry in PodSpec.ResourceClaims.
+ properties:
+ name:
+ description: Name must match the name
+ of one entry in pod.spec.resourceClaims
+ of the Pod where this field is used.
+ It makes that resource available inside
+ a container.
+ type: string
+ required:
+ - name
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - name
+ x-kubernetes-list-type: map
+ limits:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: 'Limits describes the maximum
+ amount of compute resources allowed. More
+ info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ type: object
+ requests:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: 'Requests describes the minimum
+ amount of compute resources required. If
+ Requests is omitted for a container, it
+ defaults to Limits if that is explicitly
+ specified, otherwise to an implementation-defined
+ value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ type: object
+ type: object
+ securityContext:
+ description: 'SecurityContext defines the security
+ options the container should be run with. If
+ set, the fields of SecurityContext override
+ the equivalent fields of PodSecurityContext.
+ More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/'
+ properties:
+ allowPrivilegeEscalation:
+ description: 'AllowPrivilegeEscalation controls
+ whether a process can gain more privileges
+ than its parent process. This bool directly
+ controls if the no_new_privs flag will be
+ set on the container process. AllowPrivilegeEscalation
+ is true always when the container is: 1)
+ run as Privileged 2) has CAP_SYS_ADMIN Note
+ that this field cannot be set when spec.os.name
+ is windows.'
+ type: boolean
+ capabilities:
+ description: The capabilities to add/drop
+ when running containers. Defaults to the
+ default set of capabilities granted by the
+ container runtime. Note that this field
+ cannot be set when spec.os.name is windows.
+ properties:
+ add:
+ description: Added capabilities
+ items:
+ description: Capability represent POSIX
+ capabilities type
+ type: string
+ type: array
+ drop:
+ description: Removed capabilities
+ items:
+ description: Capability represent POSIX
+ capabilities type
+ type: string
+ type: array
+ type: object
+ privileged:
+ description: Run container in privileged mode.
+ Processes in privileged containers are essentially
+ equivalent to root on the host. Defaults
+ to false. Note that this field cannot be
+ set when spec.os.name is windows.
+ type: boolean
+ procMount:
+ description: procMount denotes the type of
+ proc mount to use for the containers. The
+ default is DefaultProcMount which uses the
+ container runtime defaults for readonly
+ paths and masked paths. This requires the
+ ProcMountType feature flag to be enabled.
+ Note that this field cannot be set when
+ spec.os.name is windows.
+ type: string
+ readOnlyRootFilesystem:
+ description: Whether this container has a
+ read-only root filesystem. Default is false.
+ Note that this field cannot be set when
+ spec.os.name is windows.
+ type: boolean
+ runAsGroup:
+ description: The GID to run the entrypoint
+ of the container process. Uses runtime default
+ if unset. May also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence. Note that this field cannot
+ be set when spec.os.name is windows.
+ format: int64
+ type: integer
+ runAsNonRoot:
+ description: Indicates that the container
+ must run as a non-root user. If true, the
+ Kubelet will validate the image at runtime
+ to ensure that it does not run as UID 0
+ (root) and fail to start the container if
+ it does. If unset or false, no such validation
+ will be performed. May also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence.
+ type: boolean
+ runAsUser:
+ description: The UID to run the entrypoint
+ of the container process. Defaults to user
+ specified in image metadata if unspecified.
+ May also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence. Note that this field cannot
+ be set when spec.os.name is windows.
+ format: int64
+ type: integer
+ seLinuxOptions:
+ description: The SELinux context to be applied
+ to the container. If unspecified, the container
+ runtime will allocate a random SELinux context
+ for each container. May also be set in
+ PodSecurityContext. If set in both SecurityContext
+ and PodSecurityContext, the value specified
+ in SecurityContext takes precedence. Note
+ that this field cannot be set when spec.os.name
+ is windows.
+ properties:
+ level:
+ description: Level is SELinux level label
+ that applies to the container.
+ type: string
+ role:
+ description: Role is a SELinux role label
+ that applies to the container.
+ type: string
+ type:
+ description: Type is a SELinux type label
+ that applies to the container.
+ type: string
+ user:
+ description: User is a SELinux user label
+ that applies to the container.
+ type: string
+ type: object
+ seccompProfile:
+ description: The seccomp options to use by
+ this container. If seccomp options are provided
+ at both the pod & container level, the container
+ options override the pod options. Note that
+ this field cannot be set when spec.os.name
+ is windows.
+ properties:
+ localhostProfile:
+ description: localhostProfile indicates
+ a profile defined in a file on the node
+ should be used. The profile must be
+ preconfigured on the node to work. Must
+ be a descending path, relative to the
+ kubelet's configured seccomp profile
+ location. Must only be set if type is
+ "Localhost".
+ type: string
+ type:
+ description: "type indicates which kind
+ of seccomp profile will be applied.
+ Valid options are: \n Localhost - a
+ profile defined in a file on the node
+ should be used. RuntimeDefault - the
+ container runtime default profile should
+ be used. Unconfined - no profile should
+ be applied."
+ type: string
+ required:
+ - type
+ type: object
+ windowsOptions:
+ description: The Windows specific settings
+ applied to all containers. If unspecified,
+ the options from the PodSecurityContext
+ will be used. If set in both SecurityContext
+ and PodSecurityContext, the value specified
+ in SecurityContext takes precedence. Note
+ that this field cannot be set when spec.os.name
+ is linux.
+ properties:
+ gmsaCredentialSpec:
+ description: GMSACredentialSpec is where
+ the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa)
+ inlines the contents of the GMSA credential
+ spec named by the GMSACredentialSpecName
+ field.
+ type: string
+ gmsaCredentialSpecName:
+ description: GMSACredentialSpecName is
+ the name of the GMSA credential spec
+ to use.
+ type: string
+ hostProcess:
+ description: HostProcess determines if
+ a container should be run as a 'Host
+ Process' container. This field is alpha-level
+ and will only be honored by components
+ that enable the WindowsHostProcessContainers
+ feature flag. Setting this field without
+ the feature flag will result in errors
+ when validating the Pod. All of a Pod's
+ containers must have the same effective
+ HostProcess value (it is not allowed
+ to have a mix of HostProcess containers
+ and non-HostProcess containers). In
+ addition, if HostProcess is true then
+ HostNetwork must also be set to true.
+ type: boolean
+ runAsUserName:
+ description: The UserName in Windows to
+ run the entrypoint of the container
+ process. Defaults to the user specified
+ in image metadata if unspecified. May
+ also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext
+ takes precedence.
+ type: string
+ type: object
+ type: object
+ startupProbe:
+ description: 'StartupProbe indicates that the
+ Pod has successfully initialized. If specified,
+ no other probes are executed until this completes
+ successfully. If this probe fails, the Pod will
+ be restarted, just as if the livenessProbe failed.
+ This can be used to provide different probe
+ parameters at the beginning of a Pod''s lifecycle,
+ when it might take a long time to load data
+ or warm a cache, than during steady-state operation.
+ This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ properties:
+ exec:
+ description: Exec specifies the action to
+ take.
+ properties:
+ command:
+ description: Command is the command line
+ to execute inside the container, the
+ working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it is
+ not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to explicitly
+ call out to that shell. Exit status
+ of 0 is treated as live/healthy and
+ non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ failureThreshold:
+ description: Minimum consecutive failures
+ for the probe to be considered failed after
+ having succeeded. Defaults to 3. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ grpc:
+ description: GRPC specifies an action involving
+ a GRPC port. This is a beta field and requires
+ enabling GRPCContainerProbe feature gate.
+ properties:
+ port:
+ description: Port number of the gRPC service.
+ Number must be in the range 1 to 65535.
+ format: int32
+ type: integer
+ service:
+ description: "Service is the name of the
+ service to place in the gRPC HealthCheckRequest
+ (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
+ \n If this is not specified, the default
+ behavior is defined by gRPC."
+ type: string
+ required:
+ - port
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http request
+ to perform.
+ properties:
+ host:
+ description: Host name to connect to,
+ defaults to the pod IP. You probably
+ want to set "Host" in httpHeaders instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set in
+ the request. HTTP allows repeated headers.
+ items:
+ description: HTTPHeader describes a
+ custom header to be used in HTTP probes
+ properties:
+ name:
+ description: The header field name
+ type: string
+ value:
+ description: The header field value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the HTTP
+ server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ initialDelaySeconds:
+ description: 'Number of seconds after the
+ container has started before liveness probes
+ are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ periodSeconds:
+ description: How often (in seconds) to perform
+ the probe. Default to 10 seconds. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ successThreshold:
+ description: Minimum consecutive successes
+ for the probe to be considered successful
+ after having failed. Defaults to 1. Must
+ be 1 for liveness and startup. Minimum value
+ is 1.
+ format: int32
+ type: integer
+ tcpSocket:
+ description: TCPSocket specifies an action
+ involving a TCP port.
+ properties:
+ host:
+ description: 'Optional: Host name to connect
+ to, defaults to the pod IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds
+ the pod needs to terminate gracefully upon
+ probe failure. The grace period is the duration
+ in seconds after the processes running in
+ the pod are sent a termination signal and
+ the time when the processes are forcibly
+ halted with a kill signal. Set this value
+ longer than the expected cleanup time for
+ your process. If this value is nil, the
+ pod's terminationGracePeriodSeconds will
+ be used. Otherwise, this value overrides
+ the value provided by the pod spec. Value
+ must be non-negative integer. The value
+ zero indicates stop immediately via the
+ kill signal (no opportunity to shut down).
+ This is a beta field and requires enabling
+ ProbeTerminationGracePeriod feature gate.
+ Minimum value is 1. spec.terminationGracePeriodSeconds
+ is used if unset.
+ format: int64
+ type: integer
+ timeoutSeconds:
+ description: 'Number of seconds after which
+ the probe times out. Defaults to 1 second.
+ Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ type: object
+ stdin:
+ description: Whether this container should allocate
+ a buffer for stdin in the container runtime.
+ If this is not set, reads from stdin in the
+ container will always result in EOF. Default
+ is false.
+ type: boolean
+ stdinOnce:
+ description: Whether the container runtime should
+ close the stdin channel after it has been opened
+ by a single attach. When stdin is true the stdin
+ stream will remain open across multiple attach
+ sessions. If stdinOnce is set to true, stdin
+ is opened on container start, is empty until
+ the first client attaches to stdin, and then
+ remains open and accepts data until the client
+ disconnects, at which time stdin is closed and
+ remains closed until the container is restarted.
+ If this flag is false, a container processes
+ that reads from stdin will never receive an
+ EOF. Default is false
+ type: boolean
+ terminationMessagePath:
+ description: 'Optional: Path at which the file
+ to which the container''s termination message
+ will be written is mounted into the container''s
+ filesystem. Message written is intended to be
+ brief final status, such as an assertion failure
+ message. Will be truncated by the node if greater
+ than 4096 bytes. The total message length across
+ all containers will be limited to 12kb. Defaults
+ to /dev/termination-log. Cannot be updated.'
+ type: string
+ terminationMessagePolicy:
+ description: Indicate how the termination message
+ should be populated. File will use the contents
+ of terminationMessagePath to populate the container
+ status message on both success and failure.
+ FallbackToLogsOnError will use the last chunk
+ of container log output if the termination message
+ file is empty and the container exited with
+ an error. The log output is limited to 2048
+ bytes or 80 lines, whichever is smaller. Defaults
+ to File. Cannot be updated.
+ type: string
+ tty:
+ description: Whether this container should allocate
+ a TTY for itself, also requires 'stdin' to be
+ true. Default is false.
+ type: boolean
+ volumeDevices:
+ description: volumeDevices is the list of block
+ devices to be used by the container.
+ items:
+ description: volumeDevice describes a mapping
+ of a raw block device within a container.
+ properties:
+ devicePath:
+ description: devicePath is the path inside
+ of the container that the device will
+ be mapped to.
+ type: string
+ name:
+ description: name must match the name of
+ a persistentVolumeClaim in the pod
+ type: string
+ required:
+ - devicePath
+ - name
+ type: object
+ type: array
+ volumeMounts:
+ description: Pod volumes to mount into the container's
+ filesystem. Cannot be updated.
+ items:
+ description: VolumeMount describes a mounting
+ of a Volume within a container.
+ properties:
+ mountPath:
+ description: Path within the container at
+ which the volume should be mounted. Must
+ not contain ':'.
+ type: string
+ mountPropagation:
+ description: mountPropagation determines
+ how mounts are propagated from the host
+ to container and the other way around.
+ When not set, MountPropagationNone is
+ used. This field is beta in 1.10.
+ type: string
+ name:
+ description: This must match the Name of
+ a Volume.
+ type: string
+ readOnly:
+ description: Mounted read-only if true,
+ read-write otherwise (false or unspecified).
+ Defaults to false.
+ type: boolean
+ subPath:
+ description: Path within the volume from
+ which the container's volume should be
+ mounted. Defaults to "" (volume's root).
+ type: string
+ subPathExpr:
+ description: Expanded path within the volume
+ from which the container's volume should
+ be mounted. Behaves similarly to SubPath
+ but environment variable references $(VAR_NAME)
+ are expanded using the container's environment.
+ Defaults to "" (volume's root). SubPathExpr
+ and SubPath are mutually exclusive.
+ type: string
+ required:
+ - mountPath
+ - name
+ type: object
+ type: array
+ workingDir:
+ description: Container's working directory. If
+ not specified, the container runtime's default
+ will be used, which might be configured in the
+ container image. Cannot be updated.
+ type: string
+ required:
+ - name
+ type: object
+ type: array
+ dnsConfig:
+ description: Specifies the DNS parameters of a pod.
+ Parameters specified here will be merged to the generated
+ DNS configuration based on DNSPolicy.
+ properties:
+ nameservers:
+ description: A list of DNS name server IP addresses.
+ This will be appended to the base nameservers
+ generated from DNSPolicy. Duplicated nameservers
+ will be removed.
+ items:
+ type: string
+ type: array
+ options:
+ description: A list of DNS resolver options. This
+ will be merged with the base options generated
+ from DNSPolicy. Duplicated entries will be removed.
+ Resolution options given in Options will override
+ those that appear in the base DNSPolicy.
+ items:
+ description: PodDNSConfigOption defines DNS resolver
+ options of a pod.
+ properties:
+ name:
+ description: Required.
+ type: string
+ value:
+ type: string
+ type: object
+ type: array
+ searches:
+ description: A list of DNS search domains for host-name
+ lookup. This will be appended to the base search
+ paths generated from DNSPolicy. Duplicated search
+ paths will be removed.
+ items:
+ type: string
+ type: array
+ type: object
+ dnsPolicy:
+ description: Set DNS policy for the pod. Defaults to
+ "ClusterFirst". Valid values are 'ClusterFirstWithHostNet',
+ 'ClusterFirst', 'Default' or 'None'. DNS parameters
+ given in DNSConfig will be merged with the policy
+ selected with DNSPolicy. To have DNS options set along
+ with hostNetwork, you have to specify DNS policy explicitly
+ to 'ClusterFirstWithHostNet'.
+ type: string
+ enableServiceLinks:
+ description: 'EnableServiceLinks indicates whether information
+ about services should be injected into pod''s environment
+ variables, matching the syntax of Docker links. Optional:
+ Defaults to true.'
+ type: boolean
+ ephemeralContainers:
+ description: List of ephemeral containers run in this
+ pod. Ephemeral containers may be run in an existing
+ pod to perform user-initiated actions such as debugging.
+ This list cannot be specified when creating a pod,
+ and it cannot be modified by updating the pod spec.
+ In order to add an ephemeral container to an existing
+ pod, use the pod's ephemeralcontainers subresource.
+ items:
+ description: "An EphemeralContainer is a temporary
+ container that you may add to an existing Pod for
+ user-initiated activities such as debugging. Ephemeral
+ containers have no resource or scheduling guarantees,
+ and they will not be restarted when they exit or
+ when a Pod is removed or restarted. The kubelet
+ may evict a Pod if an ephemeral container causes
+ the Pod to exceed its resource allocation. \n To
+ add an ephemeral container, use the ephemeralcontainers
+ subresource of an existing Pod. Ephemeral containers
+ may not be removed or restarted."
+ properties:
+ args:
+ description: 'Arguments to the entrypoint. The
+ image''s CMD is used if this is not provided.
+ Variable references $(VAR_NAME) are expanded
+ using the container''s environment. If a variable
+ cannot be resolved, the reference in the input
+ string will be unchanged. Double $$ are reduced
+ to a single $, which allows for escaping the
+ $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will
+ produce the string literal "$(VAR_NAME)". Escaped
+ references will never be expanded, regardless
+ of whether the variable exists or not. Cannot
+ be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell'
+ items:
+ type: string
+ type: array
+ command:
+ description: 'Entrypoint array. Not executed within
+ a shell. The image''s ENTRYPOINT is used if
+ this is not provided. Variable references $(VAR_NAME)
+ are expanded using the container''s environment.
+ If a variable cannot be resolved, the reference
+ in the input string will be unchanged. Double
+ $$ are reduced to a single $, which allows for
+ escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)"
+ will produce the string literal "$(VAR_NAME)".
+ Escaped references will never be expanded, regardless
+ of whether the variable exists or not. Cannot
+ be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell'
+ items:
+ type: string
+ type: array
+ env:
+ description: List of environment variables to
+ set in the container. Cannot be updated.
+ items:
+ description: EnvVar represents an environment
+ variable present in a Container.
+ properties:
+ name:
+ description: Name of the environment variable.
+ Must be a C_IDENTIFIER.
+ type: string
+ value:
+ description: 'Variable references $(VAR_NAME)
+ are expanded using the previously defined
+ environment variables in the container
+ and any service environment variables.
+ If a variable cannot be resolved, the
+ reference in the input string will be
+ unchanged. Double $$ are reduced to a
+ single $, which allows for escaping the
+ $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)"
+ will produce the string literal "$(VAR_NAME)".
+ Escaped references will never be expanded,
+ regardless of whether the variable exists
+ or not. Defaults to "".'
+ type: string
+ valueFrom:
+ description: Source for the environment
+ variable's value. Cannot be used if value
+ is not empty.
+ properties:
+ configMapKeyRef:
+ description: Selects a key of a ConfigMap.
+ properties:
+ key:
+ description: The key to select.
+ type: string
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields.
+ apiVersion, kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the
+ ConfigMap or its key must be defined
+ type: boolean
+ required:
+ - key
+ type: object
+ x-kubernetes-map-type: atomic
+ fieldRef:
+ description: 'Selects a field of the
+ pod: supports metadata.name, metadata.namespace,
+ `metadata.labels['''']`, `metadata.annotations['''']`,
+ spec.nodeName, spec.serviceAccountName,
+ status.hostIP, status.podIP, status.podIPs.'
+ properties:
+ apiVersion:
+ description: Version of the schema
+ the FieldPath is written in terms
+ of, defaults to "v1".
+ type: string
+ fieldPath:
+ description: Path of the field to
+ select in the specified API version.
+ type: string
+ required:
+ - fieldPath
+ type: object
+ x-kubernetes-map-type: atomic
+ resourceFieldRef:
+ description: 'Selects a resource of
+ the container: only resources limits
+ and requests (limits.cpu, limits.memory,
+ limits.ephemeral-storage, requests.cpu,
+ requests.memory and requests.ephemeral-storage)
+ are currently supported.'
+ properties:
+ containerName:
+ description: 'Container name: required
+ for volumes, optional for env
+ vars'
+ type: string
+ divisor:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Specifies the output
+ format of the exposed resources,
+ defaults to "1"
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ resource:
+ description: 'Required: resource
+ to select'
+ type: string
+ required:
+ - resource
+ type: object
+ x-kubernetes-map-type: atomic
+ secretKeyRef:
+ description: Selects a key of a secret
+ in the pod's namespace
+ properties:
+ key:
+ description: The key of the secret
+ to select from. Must be a valid
+ secret key.
+ type: string
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields.
+ apiVersion, kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the
+ Secret or its key must be defined
+ type: boolean
+ required:
+ - key
+ type: object
+ x-kubernetes-map-type: atomic
+ type: object
+ required:
+ - name
+ type: object
+ type: array
+ envFrom:
+ description: List of sources to populate environment
+ variables in the container. The keys defined
+ within a source must be a C_IDENTIFIER. All
+ invalid keys will be reported as an event when
+ the container is starting. When a key exists
+ in multiple sources, the value associated with
+ the last source will take precedence. Values
+ defined by an Env with a duplicate key will
+ take precedence. Cannot be updated.
+ items:
+ description: EnvFromSource represents the source
+ of a set of ConfigMaps
+ properties:
+ configMapRef:
+ description: The ConfigMap to select from
+ properties:
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the ConfigMap
+ must be defined
+ type: boolean
+ type: object
+ x-kubernetes-map-type: atomic
+ prefix:
+ description: An optional identifier to prepend
+ to each key in the ConfigMap. Must be
+ a C_IDENTIFIER.
+ type: string
+ secretRef:
+ description: The Secret to select from
+ properties:
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the Secret
+ must be defined
+ type: boolean
+ type: object
+ x-kubernetes-map-type: atomic
+ type: object
+ type: array
+ image:
+ description: 'Container image name. More info:
+ https://kubernetes.io/docs/concepts/containers/images'
+ type: string
+ imagePullPolicy:
+ description: 'Image pull policy. One of Always,
+ Never, IfNotPresent. Defaults to Always if :latest
+ tag is specified, or IfNotPresent otherwise.
+ Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images'
+ type: string
+ lifecycle:
+ description: Lifecycle is not allowed for ephemeral
+ containers.
+ properties:
+ postStart:
+ description: 'PostStart is called immediately
+ after a container is created. If the handler
+ fails, the container is terminated and restarted
+ according to its restart policy. Other management
+ of the container blocks until the hook completes.
+ More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks'
+ properties:
+ exec:
+ description: Exec specifies the action
+ to take.
+ properties:
+ command:
+ description: Command is the command
+ line to execute inside the container,
+ the working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it
+ is not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to
+ explicitly call out to that shell.
+ Exit status of 0 is treated as live/healthy
+ and non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http
+ request to perform.
+ properties:
+ host:
+ description: Host name to connect
+ to, defaults to the pod IP. You
+ probably want to set "Host" in httpHeaders
+ instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set
+ in the request. HTTP allows repeated
+ headers.
+ items:
+ description: HTTPHeader describes
+ a custom header to be used in
+ HTTP probes
+ properties:
+ name:
+ description: The header field
+ name
+ type: string
+ value:
+ description: The header field
+ value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the
+ HTTP server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ tcpSocket:
+ description: Deprecated. TCPSocket is
+ NOT supported as a LifecycleHandler
+ and kept for the backward compatibility.
+ There are no validation of this field
+ and lifecycle hooks will fail in runtime
+ when tcp handler is specified.
+ properties:
+ host:
+ description: 'Optional: Host name
+ to connect to, defaults to the pod
+ IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ type: object
+ preStop:
+ description: 'PreStop is called immediately
+ before a container is terminated due to
+ an API request or management event such
+ as liveness/startup probe failure, preemption,
+ resource contention, etc. The handler is
+ not called if the container crashes or exits.
+ The Pod''s termination grace period countdown
+ begins before the PreStop hook is executed.
+ Regardless of the outcome of the handler,
+ the container will eventually terminate
+ within the Pod''s termination grace period
+ (unless delayed by finalizers). Other management
+ of the container blocks until the hook completes
+ or until the termination grace period is
+ reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks'
+ properties:
+ exec:
+ description: Exec specifies the action
+ to take.
+ properties:
+ command:
+ description: Command is the command
+ line to execute inside the container,
+ the working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it
+ is not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to
+ explicitly call out to that shell.
+ Exit status of 0 is treated as live/healthy
+ and non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http
+ request to perform.
+ properties:
+ host:
+ description: Host name to connect
+ to, defaults to the pod IP. You
+ probably want to set "Host" in httpHeaders
+ instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set
+ in the request. HTTP allows repeated
+ headers.
+ items:
+ description: HTTPHeader describes
+ a custom header to be used in
+ HTTP probes
+ properties:
+ name:
+ description: The header field
+ name
+ type: string
+ value:
+ description: The header field
+ value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the
+ HTTP server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ tcpSocket:
+ description: Deprecated. TCPSocket is
+ NOT supported as a LifecycleHandler
+ and kept for the backward compatibility.
+ There are no validation of this field
+ and lifecycle hooks will fail in runtime
+ when tcp handler is specified.
+ properties:
+ host:
+ description: 'Optional: Host name
+ to connect to, defaults to the pod
+ IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ type: object
+ type: object
+ livenessProbe:
+ description: Probes are not allowed for ephemeral
+ containers.
+ properties:
+ exec:
+ description: Exec specifies the action to
+ take.
+ properties:
+ command:
+ description: Command is the command line
+ to execute inside the container, the
+ working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it is
+ not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to explicitly
+ call out to that shell. Exit status
+ of 0 is treated as live/healthy and
+ non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ failureThreshold:
+ description: Minimum consecutive failures
+ for the probe to be considered failed after
+ having succeeded. Defaults to 3. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ grpc:
+ description: GRPC specifies an action involving
+ a GRPC port. This is a beta field and requires
+ enabling GRPCContainerProbe feature gate.
+ properties:
+ port:
+ description: Port number of the gRPC service.
+ Number must be in the range 1 to 65535.
+ format: int32
+ type: integer
+ service:
+ description: "Service is the name of the
+ service to place in the gRPC HealthCheckRequest
+ (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
+ \n If this is not specified, the default
+ behavior is defined by gRPC."
+ type: string
+ required:
+ - port
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http request
+ to perform.
+ properties:
+ host:
+ description: Host name to connect to,
+ defaults to the pod IP. You probably
+ want to set "Host" in httpHeaders instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set in
+ the request. HTTP allows repeated headers.
+ items:
+ description: HTTPHeader describes a
+ custom header to be used in HTTP probes
+ properties:
+ name:
+ description: The header field name
+ type: string
+ value:
+ description: The header field value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the HTTP
+ server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ initialDelaySeconds:
+ description: 'Number of seconds after the
+ container has started before liveness probes
+ are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ periodSeconds:
+ description: How often (in seconds) to perform
+ the probe. Default to 10 seconds. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ successThreshold:
+ description: Minimum consecutive successes
+ for the probe to be considered successful
+ after having failed. Defaults to 1. Must
+ be 1 for liveness and startup. Minimum value
+ is 1.
+ format: int32
+ type: integer
+ tcpSocket:
+ description: TCPSocket specifies an action
+ involving a TCP port.
+ properties:
+ host:
+ description: 'Optional: Host name to connect
+ to, defaults to the pod IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds
+ the pod needs to terminate gracefully upon
+ probe failure. The grace period is the duration
+ in seconds after the processes running in
+ the pod are sent a termination signal and
+ the time when the processes are forcibly
+ halted with a kill signal. Set this value
+ longer than the expected cleanup time for
+ your process. If this value is nil, the
+ pod's terminationGracePeriodSeconds will
+ be used. Otherwise, this value overrides
+ the value provided by the pod spec. Value
+ must be non-negative integer. The value
+ zero indicates stop immediately via the
+ kill signal (no opportunity to shut down).
+ This is a beta field and requires enabling
+ ProbeTerminationGracePeriod feature gate.
+ Minimum value is 1. spec.terminationGracePeriodSeconds
+ is used if unset.
+ format: int64
+ type: integer
+ timeoutSeconds:
+ description: 'Number of seconds after which
+ the probe times out. Defaults to 1 second.
+ Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ type: object
+ name:
+ description: Name of the ephemeral container specified
+ as a DNS_LABEL. This name must be unique among
+ all containers, init containers and ephemeral
+ containers.
+ type: string
+ ports:
+ description: Ports are not allowed for ephemeral
+ containers.
+ items:
+ description: ContainerPort represents a network
+ port in a single container.
+ properties:
+ containerPort:
+ description: Number of port to expose on
+ the pod's IP address. This must be a valid
+ port number, 0 < x < 65536.
+ format: int32
+ type: integer
+ hostIP:
+ description: What host IP to bind the external
+ port to.
+ type: string
+ hostPort:
+ description: Number of port to expose on
+ the host. If specified, this must be a
+ valid port number, 0 < x < 65536. If HostNetwork
+ is specified, this must match ContainerPort.
+ Most containers do not need this.
+ format: int32
+ type: integer
+ name:
+ description: If specified, this must be
+ an IANA_SVC_NAME and unique within the
+ pod. Each named port in a pod must have
+ a unique name. Name for the port that
+ can be referred to by services.
+ type: string
+ protocol:
+ default: TCP
+ description: Protocol for port. Must be
+ UDP, TCP, or SCTP. Defaults to "TCP".
+ type: string
+ required:
+ - containerPort
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - containerPort
+ - protocol
+ x-kubernetes-list-type: map
+ readinessProbe:
+ description: Probes are not allowed for ephemeral
+ containers.
+ properties:
+ exec:
+ description: Exec specifies the action to
+ take.
+ properties:
+ command:
+ description: Command is the command line
+ to execute inside the container, the
+ working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it is
+ not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to explicitly
+ call out to that shell. Exit status
+ of 0 is treated as live/healthy and
+ non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ failureThreshold:
+ description: Minimum consecutive failures
+ for the probe to be considered failed after
+ having succeeded. Defaults to 3. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ grpc:
+ description: GRPC specifies an action involving
+ a GRPC port. This is a beta field and requires
+ enabling GRPCContainerProbe feature gate.
+ properties:
+ port:
+ description: Port number of the gRPC service.
+ Number must be in the range 1 to 65535.
+ format: int32
+ type: integer
+ service:
+ description: "Service is the name of the
+ service to place in the gRPC HealthCheckRequest
+ (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
+ \n If this is not specified, the default
+ behavior is defined by gRPC."
+ type: string
+ required:
+ - port
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http request
+ to perform.
+ properties:
+ host:
+ description: Host name to connect to,
+ defaults to the pod IP. You probably
+ want to set "Host" in httpHeaders instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set in
+ the request. HTTP allows repeated headers.
+ items:
+ description: HTTPHeader describes a
+ custom header to be used in HTTP probes
+ properties:
+ name:
+ description: The header field name
+ type: string
+ value:
+ description: The header field value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the HTTP
+ server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ initialDelaySeconds:
+ description: 'Number of seconds after the
+ container has started before liveness probes
+ are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ periodSeconds:
+ description: How often (in seconds) to perform
+ the probe. Default to 10 seconds. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ successThreshold:
+ description: Minimum consecutive successes
+ for the probe to be considered successful
+ after having failed. Defaults to 1. Must
+ be 1 for liveness and startup. Minimum value
+ is 1.
+ format: int32
+ type: integer
+ tcpSocket:
+ description: TCPSocket specifies an action
+ involving a TCP port.
+ properties:
+ host:
+ description: 'Optional: Host name to connect
+ to, defaults to the pod IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds
+ the pod needs to terminate gracefully upon
+ probe failure. The grace period is the duration
+ in seconds after the processes running in
+ the pod are sent a termination signal and
+ the time when the processes are forcibly
+ halted with a kill signal. Set this value
+ longer than the expected cleanup time for
+ your process. If this value is nil, the
+ pod's terminationGracePeriodSeconds will
+ be used. Otherwise, this value overrides
+ the value provided by the pod spec. Value
+ must be non-negative integer. The value
+ zero indicates stop immediately via the
+ kill signal (no opportunity to shut down).
+ This is a beta field and requires enabling
+ ProbeTerminationGracePeriod feature gate.
+ Minimum value is 1. spec.terminationGracePeriodSeconds
+ is used if unset.
+ format: int64
+ type: integer
+ timeoutSeconds:
+ description: 'Number of seconds after which
+ the probe times out. Defaults to 1 second.
+ Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ type: object
+ resources:
+ description: Resources are not allowed for ephemeral
+ containers. Ephemeral containers use spare resources
+ already allocated to the pod.
+ properties:
+ claims:
+ description: "Claims lists the names of resources,
+ defined in spec.resourceClaims, that are
+ used by this container. \n This is an alpha
+ field and requires enabling the DynamicResourceAllocation
+ feature gate. \n This field is immutable.
+ It can only be set for containers."
+ items:
+ description: ResourceClaim references one
+ entry in PodSpec.ResourceClaims.
+ properties:
+ name:
+ description: Name must match the name
+ of one entry in pod.spec.resourceClaims
+ of the Pod where this field is used.
+ It makes that resource available inside
+ a container.
+ type: string
+ required:
+ - name
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - name
+ x-kubernetes-list-type: map
+ limits:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: 'Limits describes the maximum
+ amount of compute resources allowed. More
+ info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ type: object
+ requests:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: 'Requests describes the minimum
+ amount of compute resources required. If
+ Requests is omitted for a container, it
+ defaults to Limits if that is explicitly
+ specified, otherwise to an implementation-defined
+ value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ type: object
+ type: object
+ securityContext:
+ description: 'Optional: SecurityContext defines
+ the security options the ephemeral container
+ should be run with. If set, the fields of SecurityContext
+ override the equivalent fields of PodSecurityContext.'
+ properties:
+ allowPrivilegeEscalation:
+ description: 'AllowPrivilegeEscalation controls
+ whether a process can gain more privileges
+ than its parent process. This bool directly
+ controls if the no_new_privs flag will be
+ set on the container process. AllowPrivilegeEscalation
+ is true always when the container is: 1)
+ run as Privileged 2) has CAP_SYS_ADMIN Note
+ that this field cannot be set when spec.os.name
+ is windows.'
+ type: boolean
+ capabilities:
+ description: The capabilities to add/drop
+ when running containers. Defaults to the
+ default set of capabilities granted by the
+ container runtime. Note that this field
+ cannot be set when spec.os.name is windows.
+ properties:
+ add:
+ description: Added capabilities
+ items:
+ description: Capability represent POSIX
+ capabilities type
+ type: string
+ type: array
+ drop:
+ description: Removed capabilities
+ items:
+ description: Capability represent POSIX
+ capabilities type
+ type: string
+ type: array
+ type: object
+ privileged:
+ description: Run container in privileged mode.
+ Processes in privileged containers are essentially
+ equivalent to root on the host. Defaults
+ to false. Note that this field cannot be
+ set when spec.os.name is windows.
+ type: boolean
+ procMount:
+ description: procMount denotes the type of
+ proc mount to use for the containers. The
+ default is DefaultProcMount which uses the
+ container runtime defaults for readonly
+ paths and masked paths. This requires the
+ ProcMountType feature flag to be enabled.
+ Note that this field cannot be set when
+ spec.os.name is windows.
+ type: string
+ readOnlyRootFilesystem:
+ description: Whether this container has a
+ read-only root filesystem. Default is false.
+ Note that this field cannot be set when
+ spec.os.name is windows.
+ type: boolean
+ runAsGroup:
+ description: The GID to run the entrypoint
+ of the container process. Uses runtime default
+ if unset. May also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence. Note that this field cannot
+ be set when spec.os.name is windows.
+ format: int64
+ type: integer
+ runAsNonRoot:
+ description: Indicates that the container
+ must run as a non-root user. If true, the
+ Kubelet will validate the image at runtime
+ to ensure that it does not run as UID 0
+ (root) and fail to start the container if
+ it does. If unset or false, no such validation
+ will be performed. May also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence.
+ type: boolean
+ runAsUser:
+ description: The UID to run the entrypoint
+ of the container process. Defaults to user
+ specified in image metadata if unspecified.
+ May also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence. Note that this field cannot
+ be set when spec.os.name is windows.
+ format: int64
+ type: integer
+ seLinuxOptions:
+ description: The SELinux context to be applied
+ to the container. If unspecified, the container
+ runtime will allocate a random SELinux context
+ for each container. May also be set in
+ PodSecurityContext. If set in both SecurityContext
+ and PodSecurityContext, the value specified
+ in SecurityContext takes precedence. Note
+ that this field cannot be set when spec.os.name
+ is windows.
+ properties:
+ level:
+ description: Level is SELinux level label
+ that applies to the container.
+ type: string
+ role:
+ description: Role is a SELinux role label
+ that applies to the container.
+ type: string
+ type:
+ description: Type is a SELinux type label
+ that applies to the container.
+ type: string
+ user:
+ description: User is a SELinux user label
+ that applies to the container.
+ type: string
+ type: object
+ seccompProfile:
+ description: The seccomp options to use by
+ this container. If seccomp options are provided
+ at both the pod & container level, the container
+ options override the pod options. Note that
+ this field cannot be set when spec.os.name
+ is windows.
+ properties:
+ localhostProfile:
+ description: localhostProfile indicates
+ a profile defined in a file on the node
+ should be used. The profile must be
+ preconfigured on the node to work. Must
+ be a descending path, relative to the
+ kubelet's configured seccomp profile
+ location. Must only be set if type is
+ "Localhost".
+ type: string
+ type:
+ description: "type indicates which kind
+ of seccomp profile will be applied.
+ Valid options are: \n Localhost - a
+ profile defined in a file on the node
+ should be used. RuntimeDefault - the
+ container runtime default profile should
+ be used. Unconfined - no profile should
+ be applied."
+ type: string
+ required:
+ - type
+ type: object
+ windowsOptions:
+ description: The Windows specific settings
+ applied to all containers. If unspecified,
+ the options from the PodSecurityContext
+ will be used. If set in both SecurityContext
+ and PodSecurityContext, the value specified
+ in SecurityContext takes precedence. Note
+ that this field cannot be set when spec.os.name
+ is linux.
+ properties:
+ gmsaCredentialSpec:
+ description: GMSACredentialSpec is where
+ the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa)
+ inlines the contents of the GMSA credential
+ spec named by the GMSACredentialSpecName
+ field.
+ type: string
+ gmsaCredentialSpecName:
+ description: GMSACredentialSpecName is
+ the name of the GMSA credential spec
+ to use.
+ type: string
+ hostProcess:
+ description: HostProcess determines if
+ a container should be run as a 'Host
+ Process' container. This field is alpha-level
+ and will only be honored by components
+ that enable the WindowsHostProcessContainers
+ feature flag. Setting this field without
+ the feature flag will result in errors
+ when validating the Pod. All of a Pod's
+ containers must have the same effective
+ HostProcess value (it is not allowed
+ to have a mix of HostProcess containers
+ and non-HostProcess containers). In
+ addition, if HostProcess is true then
+ HostNetwork must also be set to true.
+ type: boolean
+ runAsUserName:
+ description: The UserName in Windows to
+ run the entrypoint of the container
+ process. Defaults to the user specified
+ in image metadata if unspecified. May
+ also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext
+ takes precedence.
+ type: string
+ type: object
+ type: object
+ startupProbe:
+ description: Probes are not allowed for ephemeral
+ containers.
+ properties:
+ exec:
+ description: Exec specifies the action to
+ take.
+ properties:
+ command:
+ description: Command is the command line
+ to execute inside the container, the
+ working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it is
+ not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to explicitly
+ call out to that shell. Exit status
+ of 0 is treated as live/healthy and
+ non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ failureThreshold:
+ description: Minimum consecutive failures
+ for the probe to be considered failed after
+ having succeeded. Defaults to 3. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ grpc:
+ description: GRPC specifies an action involving
+ a GRPC port. This is a beta field and requires
+ enabling GRPCContainerProbe feature gate.
+ properties:
+ port:
+ description: Port number of the gRPC service.
+ Number must be in the range 1 to 65535.
+ format: int32
+ type: integer
+ service:
+ description: "Service is the name of the
+ service to place in the gRPC HealthCheckRequest
+ (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
+ \n If this is not specified, the default
+ behavior is defined by gRPC."
+ type: string
+ required:
+ - port
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http request
+ to perform.
+ properties:
+ host:
+ description: Host name to connect to,
+ defaults to the pod IP. You probably
+ want to set "Host" in httpHeaders instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set in
+ the request. HTTP allows repeated headers.
+ items:
+ description: HTTPHeader describes a
+ custom header to be used in HTTP probes
+ properties:
+ name:
+ description: The header field name
+ type: string
+ value:
+ description: The header field value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the HTTP
+ server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ initialDelaySeconds:
+ description: 'Number of seconds after the
+ container has started before liveness probes
+ are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ periodSeconds:
+ description: How often (in seconds) to perform
+ the probe. Default to 10 seconds. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ successThreshold:
+ description: Minimum consecutive successes
+ for the probe to be considered successful
+ after having failed. Defaults to 1. Must
+ be 1 for liveness and startup. Minimum value
+ is 1.
+ format: int32
+ type: integer
+ tcpSocket:
+ description: TCPSocket specifies an action
+ involving a TCP port.
+ properties:
+ host:
+ description: 'Optional: Host name to connect
+ to, defaults to the pod IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds
+ the pod needs to terminate gracefully upon
+ probe failure. The grace period is the duration
+ in seconds after the processes running in
+ the pod are sent a termination signal and
+ the time when the processes are forcibly
+ halted with a kill signal. Set this value
+ longer than the expected cleanup time for
+ your process. If this value is nil, the
+ pod's terminationGracePeriodSeconds will
+ be used. Otherwise, this value overrides
+ the value provided by the pod spec. Value
+ must be non-negative integer. The value
+ zero indicates stop immediately via the
+ kill signal (no opportunity to shut down).
+ This is a beta field and requires enabling
+ ProbeTerminationGracePeriod feature gate.
+ Minimum value is 1. spec.terminationGracePeriodSeconds
+ is used if unset.
+ format: int64
+ type: integer
+ timeoutSeconds:
+ description: 'Number of seconds after which
+ the probe times out. Defaults to 1 second.
+ Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ type: object
+ stdin:
+ description: Whether this container should allocate
+ a buffer for stdin in the container runtime.
+ If this is not set, reads from stdin in the
+ container will always result in EOF. Default
+ is false.
+ type: boolean
+ stdinOnce:
+ description: Whether the container runtime should
+ close the stdin channel after it has been opened
+ by a single attach. When stdin is true the stdin
+ stream will remain open across multiple attach
+ sessions. If stdinOnce is set to true, stdin
+ is opened on container start, is empty until
+ the first client attaches to stdin, and then
+ remains open and accepts data until the client
+ disconnects, at which time stdin is closed and
+ remains closed until the container is restarted.
+ If this flag is false, a container processes
+ that reads from stdin will never receive an
+ EOF. Default is false
+ type: boolean
+ targetContainerName:
+ description: "If set, the name of the container
+ from PodSpec that this ephemeral container targets.
+ The ephemeral container will be run in the namespaces
+ (IPC, PID, etc) of this container. If not set
+ then the ephemeral container uses the namespaces
+ configured in the Pod spec. \n The container
+ runtime must implement support for this feature.
+ If the runtime does not support namespace targeting
+ then the result of setting this field is undefined."
+ type: string
+ terminationMessagePath:
+ description: 'Optional: Path at which the file
+ to which the container''s termination message
+ will be written is mounted into the container''s
+ filesystem. Message written is intended to be
+ brief final status, such as an assertion failure
+ message. Will be truncated by the node if greater
+ than 4096 bytes. The total message length across
+ all containers will be limited to 12kb. Defaults
+ to /dev/termination-log. Cannot be updated.'
+ type: string
+ terminationMessagePolicy:
+ description: Indicate how the termination message
+ should be populated. File will use the contents
+ of terminationMessagePath to populate the container
+ status message on both success and failure.
+ FallbackToLogsOnError will use the last chunk
+ of container log output if the termination message
+ file is empty and the container exited with
+ an error. The log output is limited to 2048
+ bytes or 80 lines, whichever is smaller. Defaults
+ to File. Cannot be updated.
+ type: string
+ tty:
+ description: Whether this container should allocate
+ a TTY for itself, also requires 'stdin' to be
+ true. Default is false.
+ type: boolean
+ volumeDevices:
+ description: volumeDevices is the list of block
+ devices to be used by the container.
+ items:
+ description: volumeDevice describes a mapping
+ of a raw block device within a container.
+ properties:
+ devicePath:
+ description: devicePath is the path inside
+ of the container that the device will
+ be mapped to.
+ type: string
+ name:
+ description: name must match the name of
+ a persistentVolumeClaim in the pod
+ type: string
+ required:
+ - devicePath
+ - name
+ type: object
+ type: array
+ volumeMounts:
+ description: Pod volumes to mount into the container's
+ filesystem. Subpath mounts are not allowed for
+ ephemeral containers. Cannot be updated.
+ items:
+ description: VolumeMount describes a mounting
+ of a Volume within a container.
+ properties:
+ mountPath:
+ description: Path within the container at
+ which the volume should be mounted. Must
+ not contain ':'.
+ type: string
+ mountPropagation:
+ description: mountPropagation determines
+ how mounts are propagated from the host
+ to container and the other way around.
+ When not set, MountPropagationNone is
+ used. This field is beta in 1.10.
+ type: string
+ name:
+ description: This must match the Name of
+ a Volume.
+ type: string
+ readOnly:
+ description: Mounted read-only if true,
+ read-write otherwise (false or unspecified).
+ Defaults to false.
+ type: boolean
+ subPath:
+ description: Path within the volume from
+ which the container's volume should be
+ mounted. Defaults to "" (volume's root).
+ type: string
+ subPathExpr:
+ description: Expanded path within the volume
+ from which the container's volume should
+ be mounted. Behaves similarly to SubPath
+ but environment variable references $(VAR_NAME)
+ are expanded using the container's environment.
+ Defaults to "" (volume's root). SubPathExpr
+ and SubPath are mutually exclusive.
+ type: string
+ required:
+ - mountPath
+ - name
+ type: object
+ type: array
+ workingDir:
+ description: Container's working directory. If
+ not specified, the container runtime's default
+ will be used, which might be configured in the
+ container image. Cannot be updated.
+ type: string
+ required:
+ - name
+ type: object
+ type: array
+ hostAliases:
+ description: HostAliases is an optional list of hosts
+ and IPs that will be injected into the pod's hosts
+ file if specified. This is only valid for non-hostNetwork
+ pods.
+ items:
+ description: HostAlias holds the mapping between IP
+ and hostnames that will be injected as an entry
+ in the pod's hosts file.
+ properties:
+ hostnames:
+ description: Hostnames for the above IP address.
+ items:
+ type: string
+ type: array
+ ip:
+ description: IP address of the host file entry.
+ type: string
+ type: object
+ type: array
+ hostIPC:
+ description: 'Use the host''s ipc namespace. Optional:
+ Default to false.'
+ type: boolean
+ hostNetwork:
+ description: Host networking requested for this pod.
+ Use the host's network namespace. If this option is
+ set, the ports that will be used must be specified.
+ Default to false.
+ type: boolean
+ hostPID:
+ description: 'Use the host''s pid namespace. Optional:
+ Default to false.'
+ type: boolean
+ hostUsers:
+ description: 'Use the host''s user namespace. Optional:
+ Default to true. If set to true or not present, the
+ pod will be run in the host user namespace, useful
+ for when the pod needs a feature only available to
+ the host user namespace, such as loading a kernel
+ module with CAP_SYS_MODULE. When set to false, a new
+ userns is created for the pod. Setting false is useful
+ for mitigating container breakout vulnerabilities
+ even allowing users to run their containers as root
+ without actually having root privileges on the host.
+ This field is alpha-level and is only honored by servers
+ that enable the UserNamespacesSupport feature.'
+ type: boolean
+ hostname:
+ description: Specifies the hostname of the Pod If not
+ specified, the pod's hostname will be set to a system-defined
+ value.
+ type: string
+ imagePullSecrets:
+ description: 'ImagePullSecrets is an optional list of
+ references to secrets in the same namespace to use
+ for pulling any of the images used by this PodSpec.
+ If specified, these secrets will be passed to individual
+ puller implementations for them to use. More info:
+ https://kubernetes.io/docs/concepts/containers/images#specifying-imagepullsecrets-on-a-pod'
+ items:
+ description: LocalObjectReference contains enough
+ information to let you locate the referenced object
+ inside the same namespace.
+ properties:
+ name:
+ description: 'Name of the referent. More info:
+ https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion, kind,
+ uid?'
+ type: string
+ type: object
+ x-kubernetes-map-type: atomic
+ type: array
+ initContainers:
+ description: 'List of initialization containers belonging
+ to the pod. Init containers are executed in order
+ prior to containers being started. If any init container
+ fails, the pod is considered to have failed and is
+ handled according to its restartPolicy. The name for
+ an init container or normal container must be unique
+ among all containers. Init containers may not have
+ Lifecycle actions, Readiness probes, Liveness probes,
+ or Startup probes. The resourceRequirements of an
+ init container are taken into account during scheduling
+ by finding the highest request/limit for each resource
+ type, and then using the max of of that value or the
+ sum of the normal containers. Limits are applied to
+ init containers in a similar fashion. Init containers
+ cannot currently be added or removed. Cannot be updated.
+ More info: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/'
+ items:
+ description: A single application container that you
+ want to run within a pod.
+ properties:
+ args:
+ description: 'Arguments to the entrypoint. The
+ container image''s CMD is used if this is not
+ provided. Variable references $(VAR_NAME) are
+ expanded using the container''s environment.
+ If a variable cannot be resolved, the reference
+ in the input string will be unchanged. Double
+ $$ are reduced to a single $, which allows for
+ escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)"
+ will produce the string literal "$(VAR_NAME)".
+ Escaped references will never be expanded, regardless
+ of whether the variable exists or not. Cannot
+ be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell'
+ items:
+ type: string
+ type: array
+ command:
+ description: 'Entrypoint array. Not executed within
+ a shell. The container image''s ENTRYPOINT is
+ used if this is not provided. Variable references
+ $(VAR_NAME) are expanded using the container''s
+ environment. If a variable cannot be resolved,
+ the reference in the input string will be unchanged.
+ Double $$ are reduced to a single $, which allows
+ for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)"
+ will produce the string literal "$(VAR_NAME)".
+ Escaped references will never be expanded, regardless
+ of whether the variable exists or not. Cannot
+ be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell'
+ items:
+ type: string
+ type: array
+ env:
+ description: List of environment variables to
+ set in the container. Cannot be updated.
+ items:
+ description: EnvVar represents an environment
+ variable present in a Container.
+ properties:
+ name:
+ description: Name of the environment variable.
+ Must be a C_IDENTIFIER.
+ type: string
+ value:
+ description: 'Variable references $(VAR_NAME)
+ are expanded using the previously defined
+ environment variables in the container
+ and any service environment variables.
+ If a variable cannot be resolved, the
+ reference in the input string will be
+ unchanged. Double $$ are reduced to a
+ single $, which allows for escaping the
+ $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)"
+ will produce the string literal "$(VAR_NAME)".
+ Escaped references will never be expanded,
+ regardless of whether the variable exists
+ or not. Defaults to "".'
+ type: string
+ valueFrom:
+ description: Source for the environment
+ variable's value. Cannot be used if value
+ is not empty.
+ properties:
+ configMapKeyRef:
+ description: Selects a key of a ConfigMap.
+ properties:
+ key:
+ description: The key to select.
+ type: string
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields.
+ apiVersion, kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the
+ ConfigMap or its key must be defined
+ type: boolean
+ required:
+ - key
+ type: object
+ x-kubernetes-map-type: atomic
+ fieldRef:
+ description: 'Selects a field of the
+ pod: supports metadata.name, metadata.namespace,
+ `metadata.labels['''']`, `metadata.annotations['''']`,
+ spec.nodeName, spec.serviceAccountName,
+ status.hostIP, status.podIP, status.podIPs.'
+ properties:
+ apiVersion:
+ description: Version of the schema
+ the FieldPath is written in terms
+ of, defaults to "v1".
+ type: string
+ fieldPath:
+ description: Path of the field to
+ select in the specified API version.
+ type: string
+ required:
+ - fieldPath
+ type: object
+ x-kubernetes-map-type: atomic
+ resourceFieldRef:
+ description: 'Selects a resource of
+ the container: only resources limits
+ and requests (limits.cpu, limits.memory,
+ limits.ephemeral-storage, requests.cpu,
+ requests.memory and requests.ephemeral-storage)
+ are currently supported.'
+ properties:
+ containerName:
+ description: 'Container name: required
+ for volumes, optional for env
+ vars'
+ type: string
+ divisor:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Specifies the output
+ format of the exposed resources,
+ defaults to "1"
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ resource:
+ description: 'Required: resource
+ to select'
+ type: string
+ required:
+ - resource
+ type: object
+ x-kubernetes-map-type: atomic
+ secretKeyRef:
+ description: Selects a key of a secret
+ in the pod's namespace
+ properties:
+ key:
+ description: The key of the secret
+ to select from. Must be a valid
+ secret key.
+ type: string
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields.
+ apiVersion, kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the
+ Secret or its key must be defined
+ type: boolean
+ required:
+ - key
+ type: object
+ x-kubernetes-map-type: atomic
+ type: object
+ required:
+ - name
+ type: object
+ type: array
+ envFrom:
+ description: List of sources to populate environment
+ variables in the container. The keys defined
+ within a source must be a C_IDENTIFIER. All
+ invalid keys will be reported as an event when
+ the container is starting. When a key exists
+ in multiple sources, the value associated with
+ the last source will take precedence. Values
+ defined by an Env with a duplicate key will
+ take precedence. Cannot be updated.
+ items:
+ description: EnvFromSource represents the source
+ of a set of ConfigMaps
+ properties:
+ configMapRef:
+ description: The ConfigMap to select from
+ properties:
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the ConfigMap
+ must be defined
+ type: boolean
+ type: object
+ x-kubernetes-map-type: atomic
+ prefix:
+ description: An optional identifier to prepend
+ to each key in the ConfigMap. Must be
+ a C_IDENTIFIER.
+ type: string
+ secretRef:
+ description: The Secret to select from
+ properties:
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ optional:
+ description: Specify whether the Secret
+ must be defined
+ type: boolean
+ type: object
+ x-kubernetes-map-type: atomic
+ type: object
+ type: array
+ image:
+ description: 'Container image name. More info:
+ https://kubernetes.io/docs/concepts/containers/images
+ This field is optional to allow higher level
+ config management to default or override container
+ images in workload controllers like Deployments
+ and StatefulSets.'
+ type: string
+ imagePullPolicy:
+ description: 'Image pull policy. One of Always,
+ Never, IfNotPresent. Defaults to Always if :latest
+ tag is specified, or IfNotPresent otherwise.
+ Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images'
+ type: string
+ lifecycle:
+ description: Actions that the management system
+ should take in response to container lifecycle
+ events. Cannot be updated.
+ properties:
+ postStart:
+ description: 'PostStart is called immediately
+ after a container is created. If the handler
+ fails, the container is terminated and restarted
+ according to its restart policy. Other management
+ of the container blocks until the hook completes.
+ More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks'
+ properties:
+ exec:
+ description: Exec specifies the action
+ to take.
+ properties:
+ command:
+ description: Command is the command
+ line to execute inside the container,
+ the working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it
+ is not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to
+ explicitly call out to that shell.
+ Exit status of 0 is treated as live/healthy
+ and non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http
+ request to perform.
+ properties:
+ host:
+ description: Host name to connect
+ to, defaults to the pod IP. You
+ probably want to set "Host" in httpHeaders
+ instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set
+ in the request. HTTP allows repeated
+ headers.
+ items:
+ description: HTTPHeader describes
+ a custom header to be used in
+ HTTP probes
+ properties:
+ name:
+ description: The header field
+ name
+ type: string
+ value:
+ description: The header field
+ value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the
+ HTTP server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ tcpSocket:
+ description: Deprecated. TCPSocket is
+ NOT supported as a LifecycleHandler
+ and kept for the backward compatibility.
+ There are no validation of this field
+ and lifecycle hooks will fail in runtime
+ when tcp handler is specified.
+ properties:
+ host:
+ description: 'Optional: Host name
+ to connect to, defaults to the pod
+ IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ type: object
+ preStop:
+ description: 'PreStop is called immediately
+ before a container is terminated due to
+ an API request or management event such
+ as liveness/startup probe failure, preemption,
+ resource contention, etc. The handler is
+ not called if the container crashes or exits.
+ The Pod''s termination grace period countdown
+ begins before the PreStop hook is executed.
+ Regardless of the outcome of the handler,
+ the container will eventually terminate
+ within the Pod''s termination grace period
+ (unless delayed by finalizers). Other management
+ of the container blocks until the hook completes
+ or until the termination grace period is
+ reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks'
+ properties:
+ exec:
+ description: Exec specifies the action
+ to take.
+ properties:
+ command:
+ description: Command is the command
+ line to execute inside the container,
+ the working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it
+ is not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to
+ explicitly call out to that shell.
+ Exit status of 0 is treated as live/healthy
+ and non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http
+ request to perform.
+ properties:
+ host:
+ description: Host name to connect
+ to, defaults to the pod IP. You
+ probably want to set "Host" in httpHeaders
+ instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set
+ in the request. HTTP allows repeated
+ headers.
+ items:
+ description: HTTPHeader describes
+ a custom header to be used in
+ HTTP probes
+ properties:
+ name:
+ description: The header field
+ name
+ type: string
+ value:
+ description: The header field
+ value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the
+ HTTP server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ tcpSocket:
+ description: Deprecated. TCPSocket is
+ NOT supported as a LifecycleHandler
+ and kept for the backward compatibility.
+ There are no validation of this field
+ and lifecycle hooks will fail in runtime
+ when tcp handler is specified.
+ properties:
+ host:
+ description: 'Optional: Host name
+ to connect to, defaults to the pod
+ IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the
+ port to access on the container.
+ Number must be in the range 1 to
+ 65535. Name must be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ type: object
+ type: object
+ livenessProbe:
+ description: 'Periodic probe of container liveness.
+ Container will be restarted if the probe fails.
+ Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ properties:
+ exec:
+ description: Exec specifies the action to
+ take.
+ properties:
+ command:
+ description: Command is the command line
+ to execute inside the container, the
+ working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it is
+ not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to explicitly
+ call out to that shell. Exit status
+ of 0 is treated as live/healthy and
+ non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ failureThreshold:
+ description: Minimum consecutive failures
+ for the probe to be considered failed after
+ having succeeded. Defaults to 3. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ grpc:
+ description: GRPC specifies an action involving
+ a GRPC port. This is a beta field and requires
+ enabling GRPCContainerProbe feature gate.
+ properties:
+ port:
+ description: Port number of the gRPC service.
+ Number must be in the range 1 to 65535.
+ format: int32
+ type: integer
+ service:
+ description: "Service is the name of the
+ service to place in the gRPC HealthCheckRequest
+ (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
+ \n If this is not specified, the default
+ behavior is defined by gRPC."
+ type: string
+ required:
+ - port
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http request
+ to perform.
+ properties:
+ host:
+ description: Host name to connect to,
+ defaults to the pod IP. You probably
+ want to set "Host" in httpHeaders instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set in
+ the request. HTTP allows repeated headers.
+ items:
+ description: HTTPHeader describes a
+ custom header to be used in HTTP probes
+ properties:
+ name:
+ description: The header field name
+ type: string
+ value:
+ description: The header field value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the HTTP
+ server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ initialDelaySeconds:
+ description: 'Number of seconds after the
+ container has started before liveness probes
+ are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ periodSeconds:
+ description: How often (in seconds) to perform
+ the probe. Default to 10 seconds. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ successThreshold:
+ description: Minimum consecutive successes
+ for the probe to be considered successful
+ after having failed. Defaults to 1. Must
+ be 1 for liveness and startup. Minimum value
+ is 1.
+ format: int32
+ type: integer
+ tcpSocket:
+ description: TCPSocket specifies an action
+ involving a TCP port.
+ properties:
+ host:
+ description: 'Optional: Host name to connect
+ to, defaults to the pod IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds
+ the pod needs to terminate gracefully upon
+ probe failure. The grace period is the duration
+ in seconds after the processes running in
+ the pod are sent a termination signal and
+ the time when the processes are forcibly
+ halted with a kill signal. Set this value
+ longer than the expected cleanup time for
+ your process. If this value is nil, the
+ pod's terminationGracePeriodSeconds will
+ be used. Otherwise, this value overrides
+ the value provided by the pod spec. Value
+ must be non-negative integer. The value
+ zero indicates stop immediately via the
+ kill signal (no opportunity to shut down).
+ This is a beta field and requires enabling
+ ProbeTerminationGracePeriod feature gate.
+ Minimum value is 1. spec.terminationGracePeriodSeconds
+ is used if unset.
+ format: int64
+ type: integer
+ timeoutSeconds:
+ description: 'Number of seconds after which
+ the probe times out. Defaults to 1 second.
+ Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ type: object
+ name:
+ description: Name of the container specified as
+ a DNS_LABEL. Each container in a pod must have
+ a unique name (DNS_LABEL). Cannot be updated.
+ type: string
+ ports:
+ description: List of ports to expose from the
+ container. Not specifying a port here DOES NOT
+ prevent that port from being exposed. Any port
+ which is listening on the default "0.0.0.0"
+ address inside a container will be accessible
+ from the network. Modifying this array with
+ strategic merge patch may corrupt the data.
+ For more information See https://github.com/kubernetes/kubernetes/issues/108255.
+ Cannot be updated.
+ items:
+ description: ContainerPort represents a network
+ port in a single container.
+ properties:
+ containerPort:
+ description: Number of port to expose on
+ the pod's IP address. This must be a valid
+ port number, 0 < x < 65536.
+ format: int32
+ type: integer
+ hostIP:
+ description: What host IP to bind the external
+ port to.
+ type: string
+ hostPort:
+ description: Number of port to expose on
+ the host. If specified, this must be a
+ valid port number, 0 < x < 65536. If HostNetwork
+ is specified, this must match ContainerPort.
+ Most containers do not need this.
+ format: int32
+ type: integer
+ name:
+ description: If specified, this must be
+ an IANA_SVC_NAME and unique within the
+ pod. Each named port in a pod must have
+ a unique name. Name for the port that
+ can be referred to by services.
+ type: string
+ protocol:
+ default: TCP
+ description: Protocol for port. Must be
+ UDP, TCP, or SCTP. Defaults to "TCP".
+ type: string
+ required:
+ - containerPort
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - containerPort
+ - protocol
+ x-kubernetes-list-type: map
+ readinessProbe:
+ description: 'Periodic probe of container service
+ readiness. Container will be removed from service
+ endpoints if the probe fails. Cannot be updated.
+ More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ properties:
+ exec:
+ description: Exec specifies the action to
+ take.
+ properties:
+ command:
+ description: Command is the command line
+ to execute inside the container, the
+ working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it is
+ not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to explicitly
+ call out to that shell. Exit status
+ of 0 is treated as live/healthy and
+ non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ failureThreshold:
+ description: Minimum consecutive failures
+ for the probe to be considered failed after
+ having succeeded. Defaults to 3. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ grpc:
+ description: GRPC specifies an action involving
+ a GRPC port. This is a beta field and requires
+ enabling GRPCContainerProbe feature gate.
+ properties:
+ port:
+ description: Port number of the gRPC service.
+ Number must be in the range 1 to 65535.
+ format: int32
+ type: integer
+ service:
+ description: "Service is the name of the
+ service to place in the gRPC HealthCheckRequest
+ (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
+ \n If this is not specified, the default
+ behavior is defined by gRPC."
+ type: string
+ required:
+ - port
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http request
+ to perform.
+ properties:
+ host:
+ description: Host name to connect to,
+ defaults to the pod IP. You probably
+ want to set "Host" in httpHeaders instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set in
+ the request. HTTP allows repeated headers.
+ items:
+ description: HTTPHeader describes a
+ custom header to be used in HTTP probes
+ properties:
+ name:
+ description: The header field name
+ type: string
+ value:
+ description: The header field value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the HTTP
+ server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ initialDelaySeconds:
+ description: 'Number of seconds after the
+ container has started before liveness probes
+ are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ periodSeconds:
+ description: How often (in seconds) to perform
+ the probe. Default to 10 seconds. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ successThreshold:
+ description: Minimum consecutive successes
+ for the probe to be considered successful
+ after having failed. Defaults to 1. Must
+ be 1 for liveness and startup. Minimum value
+ is 1.
+ format: int32
+ type: integer
+ tcpSocket:
+ description: TCPSocket specifies an action
+ involving a TCP port.
+ properties:
+ host:
+ description: 'Optional: Host name to connect
+ to, defaults to the pod IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds
+ the pod needs to terminate gracefully upon
+ probe failure. The grace period is the duration
+ in seconds after the processes running in
+ the pod are sent a termination signal and
+ the time when the processes are forcibly
+ halted with a kill signal. Set this value
+ longer than the expected cleanup time for
+ your process. If this value is nil, the
+ pod's terminationGracePeriodSeconds will
+ be used. Otherwise, this value overrides
+ the value provided by the pod spec. Value
+ must be non-negative integer. The value
+ zero indicates stop immediately via the
+ kill signal (no opportunity to shut down).
+ This is a beta field and requires enabling
+ ProbeTerminationGracePeriod feature gate.
+ Minimum value is 1. spec.terminationGracePeriodSeconds
+ is used if unset.
+ format: int64
+ type: integer
+ timeoutSeconds:
+ description: 'Number of seconds after which
+ the probe times out. Defaults to 1 second.
+ Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ type: object
+ resources:
+ description: 'Compute Resources required by this
+ container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ properties:
+ claims:
+ description: "Claims lists the names of resources,
+ defined in spec.resourceClaims, that are
+ used by this container. \n This is an alpha
+ field and requires enabling the DynamicResourceAllocation
+ feature gate. \n This field is immutable.
+ It can only be set for containers."
+ items:
+ description: ResourceClaim references one
+ entry in PodSpec.ResourceClaims.
+ properties:
+ name:
+ description: Name must match the name
+ of one entry in pod.spec.resourceClaims
+ of the Pod where this field is used.
+ It makes that resource available inside
+ a container.
+ type: string
+ required:
+ - name
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - name
+ x-kubernetes-list-type: map
+ limits:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: 'Limits describes the maximum
+ amount of compute resources allowed. More
+ info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ type: object
+ requests:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: 'Requests describes the minimum
+ amount of compute resources required. If
+ Requests is omitted for a container, it
+ defaults to Limits if that is explicitly
+ specified, otherwise to an implementation-defined
+ value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ type: object
+ type: object
+ securityContext:
+ description: 'SecurityContext defines the security
+ options the container should be run with. If
+ set, the fields of SecurityContext override
+ the equivalent fields of PodSecurityContext.
+ More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/'
+ properties:
+ allowPrivilegeEscalation:
+ description: 'AllowPrivilegeEscalation controls
+ whether a process can gain more privileges
+ than its parent process. This bool directly
+ controls if the no_new_privs flag will be
+ set on the container process. AllowPrivilegeEscalation
+ is true always when the container is: 1)
+ run as Privileged 2) has CAP_SYS_ADMIN Note
+ that this field cannot be set when spec.os.name
+ is windows.'
+ type: boolean
+ capabilities:
+ description: The capabilities to add/drop
+ when running containers. Defaults to the
+ default set of capabilities granted by the
+ container runtime. Note that this field
+ cannot be set when spec.os.name is windows.
+ properties:
+ add:
+ description: Added capabilities
+ items:
+ description: Capability represent POSIX
+ capabilities type
+ type: string
+ type: array
+ drop:
+ description: Removed capabilities
+ items:
+ description: Capability represent POSIX
+ capabilities type
+ type: string
+ type: array
+ type: object
+ privileged:
+ description: Run container in privileged mode.
+ Processes in privileged containers are essentially
+ equivalent to root on the host. Defaults
+ to false. Note that this field cannot be
+ set when spec.os.name is windows.
+ type: boolean
+ procMount:
+ description: procMount denotes the type of
+ proc mount to use for the containers. The
+ default is DefaultProcMount which uses the
+ container runtime defaults for readonly
+ paths and masked paths. This requires the
+ ProcMountType feature flag to be enabled.
+ Note that this field cannot be set when
+ spec.os.name is windows.
+ type: string
+ readOnlyRootFilesystem:
+ description: Whether this container has a
+ read-only root filesystem. Default is false.
+ Note that this field cannot be set when
+ spec.os.name is windows.
+ type: boolean
+ runAsGroup:
+ description: The GID to run the entrypoint
+ of the container process. Uses runtime default
+ if unset. May also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence. Note that this field cannot
+ be set when spec.os.name is windows.
+ format: int64
+ type: integer
+ runAsNonRoot:
+ description: Indicates that the container
+ must run as a non-root user. If true, the
+ Kubelet will validate the image at runtime
+ to ensure that it does not run as UID 0
+ (root) and fail to start the container if
+ it does. If unset or false, no such validation
+ will be performed. May also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence.
+ type: boolean
+ runAsUser:
+ description: The UID to run the entrypoint
+ of the container process. Defaults to user
+ specified in image metadata if unspecified.
+ May also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence. Note that this field cannot
+ be set when spec.os.name is windows.
+ format: int64
+ type: integer
+ seLinuxOptions:
+ description: The SELinux context to be applied
+ to the container. If unspecified, the container
+ runtime will allocate a random SELinux context
+ for each container. May also be set in
+ PodSecurityContext. If set in both SecurityContext
+ and PodSecurityContext, the value specified
+ in SecurityContext takes precedence. Note
+ that this field cannot be set when spec.os.name
+ is windows.
+ properties:
+ level:
+ description: Level is SELinux level label
+ that applies to the container.
+ type: string
+ role:
+ description: Role is a SELinux role label
+ that applies to the container.
+ type: string
+ type:
+ description: Type is a SELinux type label
+ that applies to the container.
+ type: string
+ user:
+ description: User is a SELinux user label
+ that applies to the container.
+ type: string
+ type: object
+ seccompProfile:
+ description: The seccomp options to use by
+ this container. If seccomp options are provided
+ at both the pod & container level, the container
+ options override the pod options. Note that
+ this field cannot be set when spec.os.name
+ is windows.
+ properties:
+ localhostProfile:
+ description: localhostProfile indicates
+ a profile defined in a file on the node
+ should be used. The profile must be
+ preconfigured on the node to work. Must
+ be a descending path, relative to the
+ kubelet's configured seccomp profile
+ location. Must only be set if type is
+ "Localhost".
+ type: string
+ type:
+ description: "type indicates which kind
+ of seccomp profile will be applied.
+ Valid options are: \n Localhost - a
+ profile defined in a file on the node
+ should be used. RuntimeDefault - the
+ container runtime default profile should
+ be used. Unconfined - no profile should
+ be applied."
+ type: string
+ required:
+ - type
+ type: object
+ windowsOptions:
+ description: The Windows specific settings
+ applied to all containers. If unspecified,
+ the options from the PodSecurityContext
+ will be used. If set in both SecurityContext
+ and PodSecurityContext, the value specified
+ in SecurityContext takes precedence. Note
+ that this field cannot be set when spec.os.name
+ is linux.
+ properties:
+ gmsaCredentialSpec:
+ description: GMSACredentialSpec is where
+ the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa)
+ inlines the contents of the GMSA credential
+ spec named by the GMSACredentialSpecName
+ field.
+ type: string
+ gmsaCredentialSpecName:
+ description: GMSACredentialSpecName is
+ the name of the GMSA credential spec
+ to use.
+ type: string
+ hostProcess:
+ description: HostProcess determines if
+ a container should be run as a 'Host
+ Process' container. This field is alpha-level
+ and will only be honored by components
+ that enable the WindowsHostProcessContainers
+ feature flag. Setting this field without
+ the feature flag will result in errors
+ when validating the Pod. All of a Pod's
+ containers must have the same effective
+ HostProcess value (it is not allowed
+ to have a mix of HostProcess containers
+ and non-HostProcess containers). In
+ addition, if HostProcess is true then
+ HostNetwork must also be set to true.
+ type: boolean
+ runAsUserName:
+ description: The UserName in Windows to
+ run the entrypoint of the container
+ process. Defaults to the user specified
+ in image metadata if unspecified. May
+ also be set in PodSecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext
+ takes precedence.
+ type: string
+ type: object
+ type: object
+ startupProbe:
+ description: 'StartupProbe indicates that the
+ Pod has successfully initialized. If specified,
+ no other probes are executed until this completes
+ successfully. If this probe fails, the Pod will
+ be restarted, just as if the livenessProbe failed.
+ This can be used to provide different probe
+ parameters at the beginning of a Pod''s lifecycle,
+ when it might take a long time to load data
+ or warm a cache, than during steady-state operation.
+ This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ properties:
+ exec:
+ description: Exec specifies the action to
+ take.
+ properties:
+ command:
+ description: Command is the command line
+ to execute inside the container, the
+ working directory for the command is
+ root ('/') in the container's filesystem.
+ The command is simply exec'd, it is
+ not run inside a shell, so traditional
+ shell instructions ('|', etc) won't
+ work. To use a shell, you need to explicitly
+ call out to that shell. Exit status
+ of 0 is treated as live/healthy and
+ non-zero is unhealthy.
+ items:
+ type: string
+ type: array
+ type: object
+ failureThreshold:
+ description: Minimum consecutive failures
+ for the probe to be considered failed after
+ having succeeded. Defaults to 3. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ grpc:
+ description: GRPC specifies an action involving
+ a GRPC port. This is a beta field and requires
+ enabling GRPCContainerProbe feature gate.
+ properties:
+ port:
+ description: Port number of the gRPC service.
+ Number must be in the range 1 to 65535.
+ format: int32
+ type: integer
+ service:
+ description: "Service is the name of the
+ service to place in the gRPC HealthCheckRequest
+ (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md).
+ \n If this is not specified, the default
+ behavior is defined by gRPC."
+ type: string
+ required:
+ - port
+ type: object
+ httpGet:
+ description: HTTPGet specifies the http request
+ to perform.
+ properties:
+ host:
+ description: Host name to connect to,
+ defaults to the pod IP. You probably
+ want to set "Host" in httpHeaders instead.
+ type: string
+ httpHeaders:
+ description: Custom headers to set in
+ the request. HTTP allows repeated headers.
+ items:
+ description: HTTPHeader describes a
+ custom header to be used in HTTP probes
+ properties:
+ name:
+ description: The header field name
+ type: string
+ value:
+ description: The header field value
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ path:
+ description: Path to access on the HTTP
+ server.
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Name or number of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ scheme:
+ description: Scheme to use for connecting
+ to the host. Defaults to HTTP.
+ type: string
+ required:
+ - port
+ type: object
+ initialDelaySeconds:
+ description: 'Number of seconds after the
+ container has started before liveness probes
+ are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ periodSeconds:
+ description: How often (in seconds) to perform
+ the probe. Default to 10 seconds. Minimum
+ value is 1.
+ format: int32
+ type: integer
+ successThreshold:
+ description: Minimum consecutive successes
+ for the probe to be considered successful
+ after having failed. Defaults to 1. Must
+ be 1 for liveness and startup. Minimum value
+ is 1.
+ format: int32
+ type: integer
+ tcpSocket:
+ description: TCPSocket specifies an action
+ involving a TCP port.
+ properties:
+ host:
+ description: 'Optional: Host name to connect
+ to, defaults to the pod IP.'
+ type: string
+ port:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Number or name of the port
+ to access on the container. Number must
+ be in the range 1 to 65535. Name must
+ be an IANA_SVC_NAME.
+ x-kubernetes-int-or-string: true
+ required:
+ - port
+ type: object
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds
+ the pod needs to terminate gracefully upon
+ probe failure. The grace period is the duration
+ in seconds after the processes running in
+ the pod are sent a termination signal and
+ the time when the processes are forcibly
+ halted with a kill signal. Set this value
+ longer than the expected cleanup time for
+ your process. If this value is nil, the
+ pod's terminationGracePeriodSeconds will
+ be used. Otherwise, this value overrides
+ the value provided by the pod spec. Value
+ must be non-negative integer. The value
+ zero indicates stop immediately via the
+ kill signal (no opportunity to shut down).
+ This is a beta field and requires enabling
+ ProbeTerminationGracePeriod feature gate.
+ Minimum value is 1. spec.terminationGracePeriodSeconds
+ is used if unset.
+ format: int64
+ type: integer
+ timeoutSeconds:
+ description: 'Number of seconds after which
+ the probe times out. Defaults to 1 second.
+ Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes'
+ format: int32
+ type: integer
+ type: object
+ stdin:
+ description: Whether this container should allocate
+ a buffer for stdin in the container runtime.
+ If this is not set, reads from stdin in the
+ container will always result in EOF. Default
+ is false.
+ type: boolean
+ stdinOnce:
+ description: Whether the container runtime should
+ close the stdin channel after it has been opened
+ by a single attach. When stdin is true the stdin
+ stream will remain open across multiple attach
+ sessions. If stdinOnce is set to true, stdin
+ is opened on container start, is empty until
+ the first client attaches to stdin, and then
+ remains open and accepts data until the client
+ disconnects, at which time stdin is closed and
+ remains closed until the container is restarted.
+ If this flag is false, a container processes
+ that reads from stdin will never receive an
+ EOF. Default is false
+ type: boolean
+ terminationMessagePath:
+ description: 'Optional: Path at which the file
+ to which the container''s termination message
+ will be written is mounted into the container''s
+ filesystem. Message written is intended to be
+ brief final status, such as an assertion failure
+ message. Will be truncated by the node if greater
+ than 4096 bytes. The total message length across
+ all containers will be limited to 12kb. Defaults
+ to /dev/termination-log. Cannot be updated.'
+ type: string
+ terminationMessagePolicy:
+ description: Indicate how the termination message
+ should be populated. File will use the contents
+ of terminationMessagePath to populate the container
+ status message on both success and failure.
+ FallbackToLogsOnError will use the last chunk
+ of container log output if the termination message
+ file is empty and the container exited with
+ an error. The log output is limited to 2048
+ bytes or 80 lines, whichever is smaller. Defaults
+ to File. Cannot be updated.
+ type: string
+ tty:
+ description: Whether this container should allocate
+ a TTY for itself, also requires 'stdin' to be
+ true. Default is false.
+ type: boolean
+ volumeDevices:
+ description: volumeDevices is the list of block
+ devices to be used by the container.
+ items:
+ description: volumeDevice describes a mapping
+ of a raw block device within a container.
+ properties:
+ devicePath:
+ description: devicePath is the path inside
+ of the container that the device will
+ be mapped to.
+ type: string
+ name:
+ description: name must match the name of
+ a persistentVolumeClaim in the pod
+ type: string
+ required:
+ - devicePath
+ - name
+ type: object
+ type: array
+ volumeMounts:
+ description: Pod volumes to mount into the container's
+ filesystem. Cannot be updated.
+ items:
+ description: VolumeMount describes a mounting
+ of a Volume within a container.
+ properties:
+ mountPath:
+ description: Path within the container at
+ which the volume should be mounted. Must
+ not contain ':'.
+ type: string
+ mountPropagation:
+ description: mountPropagation determines
+ how mounts are propagated from the host
+ to container and the other way around.
+ When not set, MountPropagationNone is
+ used. This field is beta in 1.10.
+ type: string
+ name:
+ description: This must match the Name of
+ a Volume.
+ type: string
+ readOnly:
+ description: Mounted read-only if true,
+ read-write otherwise (false or unspecified).
+ Defaults to false.
+ type: boolean
+ subPath:
+ description: Path within the volume from
+ which the container's volume should be
+ mounted. Defaults to "" (volume's root).
+ type: string
+ subPathExpr:
+ description: Expanded path within the volume
+ from which the container's volume should
+ be mounted. Behaves similarly to SubPath
+ but environment variable references $(VAR_NAME)
+ are expanded using the container's environment.
+ Defaults to "" (volume's root). SubPathExpr
+ and SubPath are mutually exclusive.
+ type: string
+ required:
+ - mountPath
+ - name
+ type: object
+ type: array
+ workingDir:
+ description: Container's working directory. If
+ not specified, the container runtime's default
+ will be used, which might be configured in the
+ container image. Cannot be updated.
+ type: string
+ required:
+ - name
+ type: object
+ type: array
+ nodeName:
+ description: NodeName is a request to schedule this
+ pod onto a specific node. If it is non-empty, the
+ scheduler simply schedules this pod onto that node,
+ assuming that it fits resource requirements.
+ type: string
+ nodeSelector:
+ additionalProperties:
+ type: string
+ description: 'NodeSelector is a selector which must
+ be true for the pod to fit on a node. Selector which
+ must match a node''s labels for the pod to be scheduled
+ on that node. More info: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/'
+ type: object
+ x-kubernetes-map-type: atomic
+ os:
+ description: "Specifies the OS of the containers in
+ the pod. Some pod and container fields are restricted
+ if this is set. \n If the OS field is set to linux,
+ the following fields must be unset: -securityContext.windowsOptions
+ \n If the OS field is set to windows, following fields
+ must be unset: - spec.hostPID - spec.hostIPC - spec.hostUsers
+ - spec.securityContext.seLinuxOptions - spec.securityContext.seccompProfile
+ - spec.securityContext.fsGroup - spec.securityContext.fsGroupChangePolicy
+ - spec.securityContext.sysctls - spec.shareProcessNamespace
+ - spec.securityContext.runAsUser - spec.securityContext.runAsGroup
+ - spec.securityContext.supplementalGroups - spec.containers[*].securityContext.seLinuxOptions
+ - spec.containers[*].securityContext.seccompProfile
+ - spec.containers[*].securityContext.capabilities
+ - spec.containers[*].securityContext.readOnlyRootFilesystem
+ - spec.containers[*].securityContext.privileged -
+ spec.containers[*].securityContext.allowPrivilegeEscalation
+ - spec.containers[*].securityContext.procMount - spec.containers[*].securityContext.runAsUser
+ - spec.containers[*].securityContext.runAsGroup"
+ properties:
+ name:
+ description: 'Name is the name of the operating
+ system. The currently supported values are linux
+ and windows. Additional value may be defined in
+ future and can be one of: https://github.com/opencontainers/runtime-spec/blob/master/config.md#platform-specific-configuration
+ Clients should expect to handle additional values
+ and treat unrecognized values in this field as
+ os: null'
+ type: string
+ required:
+ - name
+ type: object
+ overhead:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: 'Overhead represents the resource overhead
+ associated with running a pod for a given RuntimeClass.
+ This field will be autopopulated at admission time
+ by the RuntimeClass admission controller. If the RuntimeClass
+ admission controller is enabled, overhead must not
+ be set in Pod create requests. The RuntimeClass admission
+ controller will reject Pod create requests which have
+ the overhead already set. If RuntimeClass is configured
+ and selected in the PodSpec, Overhead will be set
+ to the value defined in the corresponding RuntimeClass,
+ otherwise it will remain unset and treated as zero.
+ More info: https://git.k8s.io/enhancements/keps/sig-node/688-pod-overhead/README.md'
+ type: object
+ preemptionPolicy:
+ description: PreemptionPolicy is the Policy for preempting
+ pods with lower priority. One of Never, PreemptLowerPriority.
+ Defaults to PreemptLowerPriority if unset.
+ type: string
+ priority:
+ description: The priority value. Various system components
+ use this field to find the priority of the pod. When
+ Priority Admission Controller is enabled, it prevents
+ users from setting this field. The admission controller
+ populates this field from PriorityClassName. The higher
+ the value, the higher the priority.
+ format: int32
+ type: integer
+ priorityClassName:
+ description: If specified, indicates the pod's priority.
+ "system-node-critical" and "system-cluster-critical"
+ are two special keywords which indicate the highest
+ priorities with the former being the highest priority.
+ Any other name must be defined by creating a PriorityClass
+ object with that name. If not specified, the pod priority
+ will be default or zero if there is no default.
+ type: string
+ readinessGates:
+ description: 'If specified, all readiness gates will
+ be evaluated for pod readiness. A pod is ready when
+ all its containers are ready AND all conditions specified
+ in the readiness gates have status equal to "True"
+ More info: https://git.k8s.io/enhancements/keps/sig-network/580-pod-readiness-gates'
+ items:
+ description: PodReadinessGate contains the reference
+ to a pod condition
+ properties:
+ conditionType:
+ description: ConditionType refers to a condition
+ in the pod's condition list with matching type.
+ type: string
+ required:
+ - conditionType
+ type: object
+ type: array
+ resourceClaims:
+ description: "ResourceClaims defines which ResourceClaims
+ must be allocated and reserved before the Pod is allowed
+ to start. The resources will be made available to
+ those containers which consume them by name. \n This
+ is an alpha field and requires enabling the DynamicResourceAllocation
+ feature gate. \n This field is immutable."
+ items:
+ description: PodResourceClaim references exactly one
+ ResourceClaim through a ClaimSource. It adds a name
+ to it that uniquely identifies the ResourceClaim
+ inside the Pod. Containers that need access to the
+ ResourceClaim reference it with this name.
+ properties:
+ name:
+ description: Name uniquely identifies this resource
+ claim inside the pod. This must be a DNS_LABEL.
+ type: string
+ source:
+ description: Source describes where to find the
+ ResourceClaim.
+ properties:
+ resourceClaimName:
+ description: ResourceClaimName is the name
+ of a ResourceClaim object in the same namespace
+ as this pod.
+ type: string
+ resourceClaimTemplateName:
+ description: "ResourceClaimTemplateName is
+ the name of a ResourceClaimTemplate object
+ in the same namespace as this pod. \n The
+ template will be used to create a new ResourceClaim,
+ which will be bound to this pod. When this
+ pod is deleted, the ResourceClaim will also
+ be deleted. The name of the ResourceClaim
+ will be -, where
+ is the PodResourceClaim.Name.
+ Pod validation will reject the pod if the
+ concatenated name is not valid for a ResourceClaim
+ (e.g. too long). \n An existing ResourceClaim
+ with that name that is not owned by the
+ pod will not be used for the pod to avoid
+ using an unrelated resource by mistake.
+ Scheduling and pod startup are then blocked
+ until the unrelated ResourceClaim is removed.
+ \n This field is immutable and no changes
+ will be made to the corresponding ResourceClaim
+ by the control plane after creating the
+ ResourceClaim."
+ type: string
+ type: object
+ required:
+ - name
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - name
+ x-kubernetes-list-type: map
+ restartPolicy:
+ description: 'Restart policy for all containers within
+ the pod. One of Always, OnFailure, Never. Default
+ to Always. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy'
+ type: string
+ runtimeClassName:
+ description: 'RuntimeClassName refers to a RuntimeClass
+ object in the node.k8s.io group, which should be used
+ to run this pod. If no RuntimeClass resource matches
+ the named class, the pod will not be run. If unset
+ or empty, the "legacy" RuntimeClass will be used,
+ which is an implicit class with an empty definition
+ that uses the default runtime handler. More info:
+ https://git.k8s.io/enhancements/keps/sig-node/585-runtime-class'
+ type: string
+ schedulerName:
+ description: If specified, the pod will be dispatched
+ by specified scheduler. If not specified, the pod
+ will be dispatched by default scheduler.
+ type: string
+ schedulingGates:
+ description: "SchedulingGates is an opaque list of values
+ that if specified will block scheduling the pod. More
+ info: https://git.k8s.io/enhancements/keps/sig-scheduling/3521-pod-scheduling-readiness.
+ \n This is an alpha-level feature enabled by PodSchedulingReadiness
+ feature gate."
+ items:
+ description: PodSchedulingGate is associated to a
+ Pod to guard its scheduling.
+ properties:
+ name:
+ description: Name of the scheduling gate. Each
+ scheduling gate must have a unique name field.
+ type: string
+ required:
+ - name
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - name
+ x-kubernetes-list-type: map
+ securityContext:
+ description: 'SecurityContext holds pod-level security
+ attributes and common container settings. Optional:
+ Defaults to empty. See type description for default
+ values of each field.'
+ properties:
+ fsGroup:
+ description: "A special supplemental group that
+ applies to all containers in a pod. Some volume
+ types allow the Kubelet to change the ownership
+ of that volume to be owned by the pod: \n 1. The
+ owning GID will be the FSGroup 2. The setgid bit
+ is set (new files created in the volume will be
+ owned by FSGroup) 3. The permission bits are OR'd
+ with rw-rw---- \n If unset, the Kubelet will not
+ modify the ownership and permissions of any volume.
+ Note that this field cannot be set when spec.os.name
+ is windows."
+ format: int64
+ type: integer
+ fsGroupChangePolicy:
+ description: 'fsGroupChangePolicy defines behavior
+ of changing ownership and permission of the volume
+ before being exposed inside Pod. This field will
+ only apply to volume types which support fsGroup
+ based ownership(and permissions). It will have
+ no effect on ephemeral volume types such as: secret,
+ configmaps and emptydir. Valid values are "OnRootMismatch"
+ and "Always". If not specified, "Always" is used.
+ Note that this field cannot be set when spec.os.name
+ is windows.'
+ type: string
+ runAsGroup:
+ description: The GID to run the entrypoint of the
+ container process. Uses runtime default if unset.
+ May also be set in SecurityContext. If set in
+ both SecurityContext and PodSecurityContext, the
+ value specified in SecurityContext takes precedence
+ for that container. Note that this field cannot
+ be set when spec.os.name is windows.
+ format: int64
+ type: integer
+ runAsNonRoot:
+ description: Indicates that the container must run
+ as a non-root user. If true, the Kubelet will
+ validate the image at runtime to ensure that it
+ does not run as UID 0 (root) and fail to start
+ the container if it does. If unset or false, no
+ such validation will be performed. May also be
+ set in SecurityContext. If set in both SecurityContext
+ and PodSecurityContext, the value specified in
+ SecurityContext takes precedence.
+ type: boolean
+ runAsUser:
+ description: The UID to run the entrypoint of the
+ container process. Defaults to user specified
+ in image metadata if unspecified. May also be
+ set in SecurityContext. If set in both SecurityContext
+ and PodSecurityContext, the value specified in
+ SecurityContext takes precedence for that container.
+ Note that this field cannot be set when spec.os.name
+ is windows.
+ format: int64
+ type: integer
+ seLinuxOptions:
+ description: The SELinux context to be applied to
+ all containers. If unspecified, the container
+ runtime will allocate a random SELinux context
+ for each container. May also be set in SecurityContext. If
+ set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes precedence
+ for that container. Note that this field cannot
+ be set when spec.os.name is windows.
+ properties:
+ level:
+ description: Level is SELinux level label that
+ applies to the container.
+ type: string
+ role:
+ description: Role is a SELinux role label that
+ applies to the container.
+ type: string
+ type:
+ description: Type is a SELinux type label that
+ applies to the container.
+ type: string
+ user:
+ description: User is a SELinux user label that
+ applies to the container.
+ type: string
+ type: object
+ seccompProfile:
+ description: The seccomp options to use by the containers
+ in this pod. Note that this field cannot be set
+ when spec.os.name is windows.
+ properties:
+ localhostProfile:
+ description: localhostProfile indicates a profile
+ defined in a file on the node should be used.
+ The profile must be preconfigured on the node
+ to work. Must be a descending path, relative
+ to the kubelet's configured seccomp profile
+ location. Must only be set if type is "Localhost".
+ type: string
+ type:
+ description: "type indicates which kind of seccomp
+ profile will be applied. Valid options are:
+ \n Localhost - a profile defined in a file
+ on the node should be used. RuntimeDefault
+ - the container runtime default profile should
+ be used. Unconfined - no profile should be
+ applied."
+ type: string
+ required:
+ - type
+ type: object
+ supplementalGroups:
+ description: A list of groups applied to the first
+ process run in each container, in addition to
+ the container's primary GID, the fsGroup (if specified),
+ and group memberships defined in the container
+ image for the uid of the container process. If
+ unspecified, no additional groups are added to
+ any container. Note that group memberships defined
+ in the container image for the uid of the container
+ process are still effective, even if they are
+ not included in this list. Note that this field
+ cannot be set when spec.os.name is windows.
+ items:
+ format: int64
+ type: integer
+ type: array
+ sysctls:
+ description: Sysctls hold a list of namespaced sysctls
+ used for the pod. Pods with unsupported sysctls
+ (by the container runtime) might fail to launch.
+ Note that this field cannot be set when spec.os.name
+ is windows.
+ items:
+ description: Sysctl defines a kernel parameter
+ to be set
+ properties:
+ name:
+ description: Name of a property to set
+ type: string
+ value:
+ description: Value of a property to set
+ type: string
+ required:
+ - name
+ - value
+ type: object
+ type: array
+ windowsOptions:
+ description: The Windows specific settings applied
+ to all containers. If unspecified, the options
+ within a container's SecurityContext will be used.
+ If set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes precedence.
+ Note that this field cannot be set when spec.os.name
+ is linux.
+ properties:
+ gmsaCredentialSpec:
+ description: GMSACredentialSpec is where the
+ GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa)
+ inlines the contents of the GMSA credential
+ spec named by the GMSACredentialSpecName field.
+ type: string
+ gmsaCredentialSpecName:
+ description: GMSACredentialSpecName is the name
+ of the GMSA credential spec to use.
+ type: string
+ hostProcess:
+ description: HostProcess determines if a container
+ should be run as a 'Host Process' container.
+ This field is alpha-level and will only be
+ honored by components that enable the WindowsHostProcessContainers
+ feature flag. Setting this field without the
+ feature flag will result in errors when validating
+ the Pod. All of a Pod's containers must have
+ the same effective HostProcess value (it is
+ not allowed to have a mix of HostProcess containers
+ and non-HostProcess containers). In addition,
+ if HostProcess is true then HostNetwork must
+ also be set to true.
+ type: boolean
+ runAsUserName:
+ description: The UserName in Windows to run
+ the entrypoint of the container process. Defaults
+ to the user specified in image metadata if
+ unspecified. May also be set in PodSecurityContext.
+ If set in both SecurityContext and PodSecurityContext,
+ the value specified in SecurityContext takes
+ precedence.
+ type: string
+ type: object
+ type: object
+ serviceAccount:
+ description: 'DeprecatedServiceAccount is a depreciated
+ alias for ServiceAccountName. Deprecated: Use serviceAccountName
+ instead.'
+ type: string
+ serviceAccountName:
+ description: 'ServiceAccountName is the name of the
+ ServiceAccount to use to run this pod. More info:
+ https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/'
+ type: string
+ setHostnameAsFQDN:
+ description: If true the pod's hostname will be configured
+ as the pod's FQDN, rather than the leaf name (the
+ default). In Linux containers, this means setting
+ the FQDN in the hostname field of the kernel (the
+ nodename field of struct utsname). In Windows containers,
+ this means setting the registry value of hostname
+ for the registry key HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Services\\Tcpip\\Parameters
+ to FQDN. If a pod does not have FQDN, this has no
+ effect. Default to false.
+ type: boolean
+ shareProcessNamespace:
+ description: 'Share a single process namespace between
+ all of the containers in a pod. When this is set containers
+ will be able to view and signal processes from other
+ containers in the same pod, and the first process
+ in each container will not be assigned PID 1. HostPID
+ and ShareProcessNamespace cannot both be set. Optional:
+ Default to false.'
+ type: boolean
+ subdomain:
+ description: If specified, the fully qualified Pod hostname
+ will be "...svc.". If not specified, the pod will not have
+ a domainname at all.
+ type: string
+ terminationGracePeriodSeconds:
+ description: Optional duration in seconds the pod needs
+ to terminate gracefully. May be decreased in delete
+ request. Value must be non-negative integer. The value
+ zero indicates stop immediately via the kill signal
+ (no opportunity to shut down). If this value is nil,
+ the default grace period will be used instead. The
+ grace period is the duration in seconds after the
+ processes running in the pod are sent a termination
+ signal and the time when the processes are forcibly
+ halted with a kill signal. Set this value longer than
+ the expected cleanup time for your process. Defaults
+ to 30 seconds.
+ format: int64
+ type: integer
+ tolerations:
+ description: If specified, the pod's tolerations.
+ items:
+ description: The pod this Toleration is attached to
+ tolerates any taint that matches the triple
+ using the matching operator .
+ properties:
+ effect:
+ description: Effect indicates the taint effect
+ to match. Empty means match all taint effects.
+ When specified, allowed values are NoSchedule,
+ PreferNoSchedule and NoExecute.
+ type: string
+ key:
+ description: Key is the taint key that the toleration
+ applies to. Empty means match all taint keys.
+ If the key is empty, operator must be Exists;
+ this combination means to match all values and
+ all keys.
+ type: string
+ operator:
+ description: Operator represents a key's relationship
+ to the value. Valid operators are Exists and
+ Equal. Defaults to Equal. Exists is equivalent
+ to wildcard for value, so that a pod can tolerate
+ all taints of a particular category.
+ type: string
+ tolerationSeconds:
+ description: TolerationSeconds represents the
+ period of time the toleration (which must be
+ of effect NoExecute, otherwise this field is
+ ignored) tolerates the taint. By default, it
+ is not set, which means tolerate the taint forever
+ (do not evict). Zero and negative values will
+ be treated as 0 (evict immediately) by the system.
+ format: int64
+ type: integer
+ value:
+ description: Value is the taint value the toleration
+ matches to. If the operator is Exists, the value
+ should be empty, otherwise just a regular string.
+ type: string
+ type: object
+ type: array
+ topologySpreadConstraints:
+ description: TopologySpreadConstraints describes how
+ a group of pods ought to spread across topology domains.
+ Scheduler will schedule pods in a way which abides
+ by the constraints. All topologySpreadConstraints
+ are ANDed.
+ items:
+ description: TopologySpreadConstraint specifies how
+ to spread matching pods among the given topology.
+ properties:
+ labelSelector:
+ description: LabelSelector is used to find matching
+ pods. Pods that match this label selector are
+ counted to determine the number of pods in their
+ corresponding topology domain.
+ properties:
+ matchExpressions:
+ description: matchExpressions is a list of
+ label selector requirements. The requirements
+ are ANDed.
+ items:
+ description: A label selector requirement
+ is a selector that contains values, a
+ key, and an operator that relates the
+ key and values.
+ properties:
+ key:
+ description: key is the label key that
+ the selector applies to.
+ type: string
+ operator:
+ description: operator represents a key's
+ relationship to a set of values. Valid
+ operators are In, NotIn, Exists and
+ DoesNotExist.
+ type: string
+ values:
+ description: values is an array of string
+ values. If the operator is In or NotIn,
+ the values array must be non-empty.
+ If the operator is Exists or DoesNotExist,
+ the values array must be empty. This
+ array is replaced during a strategic
+ merge patch.
+ items:
+ type: string
+ type: array
+ required:
+ - key
+ - operator
+ type: object
+ type: array
+ matchLabels:
+ additionalProperties:
+ type: string
+ description: matchLabels is a map of {key,value}
+ pairs. A single {key,value} in the matchLabels
+ map is equivalent to an element of matchExpressions,
+ whose key field is "key", the operator is
+ "In", and the values array contains only
+ "value". The requirements are ANDed.
+ type: object
+ type: object
+ x-kubernetes-map-type: atomic
+ matchLabelKeys:
+ description: MatchLabelKeys is a set of pod label
+ keys to select the pods over which spreading
+ will be calculated. The keys are used to lookup
+ values from the incoming pod labels, those key-value
+ labels are ANDed with labelSelector to select
+ the group of existing pods over which spreading
+ will be calculated for the incoming pod. Keys
+ that don't exist in the incoming pod labels
+ will be ignored. A null or empty list means
+ only match against labelSelector.
+ items:
+ type: string
+ type: array
+ x-kubernetes-list-type: atomic
+ maxSkew:
+ description: 'MaxSkew describes the degree to
+ which pods may be unevenly distributed. When
+ `whenUnsatisfiable=DoNotSchedule`, it is the
+ maximum permitted difference between the number
+ of matching pods in the target topology and
+ the global minimum. The global minimum is the
+ minimum number of matching pods in an eligible
+ domain or zero if the number of eligible domains
+ is less than MinDomains. For example, in a 3-zone
+ cluster, MaxSkew is set to 1, and pods with
+ the same labelSelector spread as 2/2/1: In this
+ case, the global minimum is 1. | zone1 | zone2
+ | zone3 | | P P | P P | P | - if MaxSkew
+ is 1, incoming pod can only be scheduled to
+ zone3 to become 2/2/2; scheduling it onto zone1(zone2)
+ would make the ActualSkew(3-1) on zone1(zone2)
+ violate MaxSkew(1). - if MaxSkew is 2, incoming
+ pod can be scheduled onto any zone. When `whenUnsatisfiable=ScheduleAnyway`,
+ it is used to give higher precedence to topologies
+ that satisfy it. It''s a required field. Default
+ value is 1 and 0 is not allowed.'
+ format: int32
+ type: integer
+ minDomains:
+ description: "MinDomains indicates a minimum number
+ of eligible domains. When the number of eligible
+ domains with matching topology keys is less
+ than minDomains, Pod Topology Spread treats
+ \"global minimum\" as 0, and then the calculation
+ of Skew is performed. And when the number of
+ eligible domains with matching topology keys
+ equals or greater than minDomains, this value
+ has no effect on scheduling. As a result, when
+ the number of eligible domains is less than
+ minDomains, scheduler won't schedule more than
+ maxSkew Pods to those domains. If value is nil,
+ the constraint behaves as if MinDomains is equal
+ to 1. Valid values are integers greater than
+ 0. When value is not nil, WhenUnsatisfiable
+ must be DoNotSchedule. \n For example, in a
+ 3-zone cluster, MaxSkew is set to 2, MinDomains
+ is set to 5 and pods with the same labelSelector
+ spread as 2/2/2: | zone1 | zone2 | zone3 | |
+ \ P P | P P | P P | The number of domains
+ is less than 5(MinDomains), so \"global minimum\"
+ is treated as 0. In this situation, new pod
+ with the same labelSelector cannot be scheduled,
+ because computed skew will be 3(3 - 0) if new
+ Pod is scheduled to any of the three zones,
+ it will violate MaxSkew. \n This is a beta field
+ and requires the MinDomainsInPodTopologySpread
+ feature gate to be enabled (enabled by default)."
+ format: int32
+ type: integer
+ nodeAffinityPolicy:
+ description: "NodeAffinityPolicy indicates how
+ we will treat Pod's nodeAffinity/nodeSelector
+ when calculating pod topology spread skew. Options
+ are: - Honor: only nodes matching nodeAffinity/nodeSelector
+ are included in the calculations. - Ignore:
+ nodeAffinity/nodeSelector are ignored. All nodes
+ are included in the calculations. \n If this
+ value is nil, the behavior is equivalent to
+ the Honor policy. This is a beta-level feature
+ default enabled by the NodeInclusionPolicyInPodTopologySpread
+ feature flag."
+ type: string
+ nodeTaintsPolicy:
+ description: "NodeTaintsPolicy indicates how we
+ will treat node taints when calculating pod
+ topology spread skew. Options are: - Honor:
+ nodes without taints, along with tainted nodes
+ for which the incoming pod has a toleration,
+ are included. - Ignore: node taints are ignored.
+ All nodes are included. \n If this value is
+ nil, the behavior is equivalent to the Ignore
+ policy. This is a beta-level feature default
+ enabled by the NodeInclusionPolicyInPodTopologySpread
+ feature flag."
+ type: string
+ topologyKey:
+ description: TopologyKey is the key of node labels.
+ Nodes that have a label with this key and identical
+ values are considered to be in the same topology.
+ We consider each as a "bucket",
+ and try to put balanced number of pods into
+ each bucket. We define a domain as a particular
+ instance of a topology. Also, we define an eligible
+ domain as a domain whose nodes meet the requirements
+ of nodeAffinityPolicy and nodeTaintsPolicy.
+ e.g. If TopologyKey is "kubernetes.io/hostname",
+ each Node is a domain of that topology. And,
+ if TopologyKey is "topology.kubernetes.io/zone",
+ each zone is a domain of that topology. It's
+ a required field.
+ type: string
+ whenUnsatisfiable:
+ description: 'WhenUnsatisfiable indicates how
+ to deal with a pod if it doesn''t satisfy the
+ spread constraint. - DoNotSchedule (default)
+ tells the scheduler not to schedule it. - ScheduleAnyway
+ tells the scheduler to schedule the pod in any
+ location, but giving higher precedence to topologies
+ that would help reduce the skew. A constraint
+ is considered "Unsatisfiable" for an incoming
+ pod if and only if every possible node assignment
+ for that pod would violate "MaxSkew" on some
+ topology. For example, in a 3-zone cluster,
+ MaxSkew is set to 1, and pods with the same
+ labelSelector spread as 3/1/1: | zone1 | zone2
+ | zone3 | | P P P | P | P | If WhenUnsatisfiable
+ is set to DoNotSchedule, incoming pod can only
+ be scheduled to zone2(zone3) to become 3/2/1(3/1/2)
+ as ActualSkew(2-1) on zone2(zone3) satisfies
+ MaxSkew(1). In other words, the cluster can
+ still be imbalanced, but scheduler won''t make
+ it *more* imbalanced. It''s a required field.'
+ type: string
+ required:
+ - maxSkew
+ - topologyKey
+ - whenUnsatisfiable
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - topologyKey
+ - whenUnsatisfiable
+ x-kubernetes-list-type: map
+ volumes:
+ description: 'List of volumes that can be mounted by
+ containers belonging to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes'
+ items:
+ description: Volume represents a named volume in a
+ pod that may be accessed by any container in the
+ pod.
+ properties:
+ awsElasticBlockStore:
+ description: 'awsElasticBlockStore represents
+ an AWS Disk resource that is attached to a kubelet''s
+ host machine and then exposed to the pod. More
+ info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore'
+ properties:
+ fsType:
+ description: 'fsType is the filesystem type
+ of the volume that you want to mount. Tip:
+ Ensure that the filesystem type is supported
+ by the host operating system. Examples:
+ "ext4", "xfs", "ntfs". Implicitly inferred
+ to be "ext4" if unspecified. More info:
+ https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
+ TODO: how do we prevent errors in the filesystem
+ from compromising the machine'
+ type: string
+ partition:
+ description: 'partition is the partition in
+ the volume that you want to mount. If omitted,
+ the default is to mount by volume name.
+ Examples: For volume /dev/sda1, you specify
+ the partition as "1". Similarly, the volume
+ partition for /dev/sda is "0" (or you can
+ leave the property empty).'
+ format: int32
+ type: integer
+ readOnly:
+ description: 'readOnly value true will force
+ the readOnly setting in VolumeMounts. More
+ info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore'
+ type: boolean
+ volumeID:
+ description: 'volumeID is unique ID of the
+ persistent disk resource in AWS (Amazon
+ EBS volume). More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore'
+ type: string
+ required:
+ - volumeID
+ type: object
+ azureDisk:
+ description: azureDisk represents an Azure Data
+ Disk mount on the host and bind mount to the
+ pod.
+ properties:
+ cachingMode:
+ description: 'cachingMode is the Host Caching
+ mode: None, Read Only, Read Write.'
+ type: string
+ diskName:
+ description: diskName is the Name of the data
+ disk in the blob storage
+ type: string
+ diskURI:
+ description: diskURI is the URI of data disk
+ in the blob storage
+ type: string
+ fsType:
+ description: fsType is Filesystem type to
+ mount. Must be a filesystem type supported
+ by the host operating system. Ex. "ext4",
+ "xfs", "ntfs". Implicitly inferred to be
+ "ext4" if unspecified.
+ type: string
+ kind:
+ description: 'kind expected values are Shared:
+ multiple blob disks per storage account Dedicated:
+ single blob disk per storage account Managed:
+ azure managed data disk (only in managed
+ availability set). defaults to shared'
+ type: string
+ readOnly:
+ description: readOnly Defaults to false (read/write).
+ ReadOnly here will force the ReadOnly setting
+ in VolumeMounts.
+ type: boolean
+ required:
+ - diskName
+ - diskURI
+ type: object
+ azureFile:
+ description: azureFile represents an Azure File
+ Service mount on the host and bind mount to
+ the pod.
+ properties:
+ readOnly:
+ description: readOnly defaults to false (read/write).
+ ReadOnly here will force the ReadOnly setting
+ in VolumeMounts.
+ type: boolean
+ secretName:
+ description: secretName is the name of secret
+ that contains Azure Storage Account Name
+ and Key
+ type: string
+ shareName:
+ description: shareName is the azure share
+ Name
+ type: string
+ required:
+ - secretName
+ - shareName
+ type: object
+ cephfs:
+ description: cephFS represents a Ceph FS mount
+ on the host that shares a pod's lifetime
+ properties:
+ monitors:
+ description: 'monitors is Required: Monitors
+ is a collection of Ceph monitors More info:
+ https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it'
+ items:
+ type: string
+ type: array
+ path:
+ description: 'path is Optional: Used as the
+ mounted root, rather than the full Ceph
+ tree, default is /'
+ type: string
+ readOnly:
+ description: 'readOnly is Optional: Defaults
+ to false (read/write). ReadOnly here will
+ force the ReadOnly setting in VolumeMounts.
+ More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it'
+ type: boolean
+ secretFile:
+ description: 'secretFile is Optional: SecretFile
+ is the path to key ring for User, default
+ is /etc/ceph/user.secret More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it'
+ type: string
+ secretRef:
+ description: 'secretRef is Optional: SecretRef
+ is reference to the authentication secret
+ for User, default is empty. More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it'
+ properties:
+ name:
+ description: 'Name of the referent. More
+ info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ type: object
+ x-kubernetes-map-type: atomic
+ user:
+ description: 'user is optional: User is the
+ rados user name, default is admin More info:
+ https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it'
+ type: string
+ required:
+ - monitors
+ type: object
+ cinder:
+ description: 'cinder represents a cinder volume
+ attached and mounted on kubelets host machine.
+ More info: https://examples.k8s.io/mysql-cinder-pd/README.md'
+ properties:
+ fsType:
+ description: 'fsType is the filesystem type
+ to mount. Must be a filesystem type supported
+ by the host operating system. Examples:
+ "ext4", "xfs", "ntfs". Implicitly inferred
+ to be "ext4" if unspecified. More info:
+ https://examples.k8s.io/mysql-cinder-pd/README.md'
+ type: string
+ readOnly:
+ description: 'readOnly defaults to false (read/write).
+ ReadOnly here will force the ReadOnly setting
+ in VolumeMounts. More info: https://examples.k8s.io/mysql-cinder-pd/README.md'
+ type: boolean
+ secretRef:
+ description: 'secretRef is optional: points
+ to a secret object containing parameters
+ used to connect to OpenStack.'
+ properties:
+ name:
+ description: 'Name of the referent. More
+ info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ type: object
+ x-kubernetes-map-type: atomic
+ volumeID:
+ description: 'volumeID used to identify the
+ volume in cinder. More info: https://examples.k8s.io/mysql-cinder-pd/README.md'
+ type: string
+ required:
+ - volumeID
+ type: object
+ configMap:
+ description: configMap represents a configMap
+ that should populate this volume
+ properties:
+ defaultMode:
+ description: 'defaultMode is optional: mode
+ bits used to set permissions on created
+ files by default. Must be an octal value
+ between 0000 and 0777 or a decimal value
+ between 0 and 511. YAML accepts both octal
+ and decimal values, JSON requires decimal
+ values for mode bits. Defaults to 0644.
+ Directories within the path are not affected
+ by this setting. This might be in conflict
+ with other options that affect the file
+ mode, like fsGroup, and the result can be
+ other mode bits set.'
+ format: int32
+ type: integer
+ items:
+ description: items if unspecified, each key-value
+ pair in the Data field of the referenced
+ ConfigMap will be projected into the volume
+ as a file whose name is the key and content
+ is the value. If specified, the listed keys
+ will be projected into the specified paths,
+ and unlisted keys will not be present. If
+ a key is specified which is not present
+ in the ConfigMap, the volume setup will
+ error unless it is marked optional. Paths
+ must be relative and may not contain the
+ '..' path or start with '..'.
+ items:
+ description: Maps a string key to a path
+ within a volume.
+ properties:
+ key:
+ description: key is the key to project.
+ type: string
+ mode:
+ description: 'mode is Optional: mode
+ bits used to set permissions on this
+ file. Must be an octal value between
+ 0000 and 0777 or a decimal value between
+ 0 and 511. YAML accepts both octal
+ and decimal values, JSON requires
+ decimal values for mode bits. If not
+ specified, the volume defaultMode
+ will be used. This might be in conflict
+ with other options that affect the
+ file mode, like fsGroup, and the result
+ can be other mode bits set.'
+ format: int32
+ type: integer
+ path:
+ description: path is the relative path
+ of the file to map the key to. May
+ not be an absolute path. May not contain
+ the path element '..'. May not start
+ with the string '..'.
+ type: string
+ required:
+ - key
+ - path
+ type: object
+ type: array
+ name:
+ description: 'Name of the referent. More info:
+ https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ optional:
+ description: optional specify whether the
+ ConfigMap or its keys must be defined
+ type: boolean
+ type: object
+ x-kubernetes-map-type: atomic
+ csi:
+ description: csi (Container Storage Interface)
+ represents ephemeral storage that is handled
+ by certain external CSI drivers (Beta feature).
+ properties:
+ driver:
+ description: driver is the name of the CSI
+ driver that handles this volume. Consult
+ with your admin for the correct name as
+ registered in the cluster.
+ type: string
+ fsType:
+ description: fsType to mount. Ex. "ext4",
+ "xfs", "ntfs". If not provided, the empty
+ value is passed to the associated CSI driver
+ which will determine the default filesystem
+ to apply.
+ type: string
+ nodePublishSecretRef:
+ description: nodePublishSecretRef is a reference
+ to the secret object containing sensitive
+ information to pass to the CSI driver to
+ complete the CSI NodePublishVolume and NodeUnpublishVolume
+ calls. This field is optional, and may
+ be empty if no secret is required. If the
+ secret object contains more than one secret,
+ all secret references are passed.
+ properties:
+ name:
+ description: 'Name of the referent. More
+ info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ type: object
+ x-kubernetes-map-type: atomic
+ readOnly:
+ description: readOnly specifies a read-only
+ configuration for the volume. Defaults to
+ false (read/write).
+ type: boolean
+ volumeAttributes:
+ additionalProperties:
+ type: string
+ description: volumeAttributes stores driver-specific
+ properties that are passed to the CSI driver.
+ Consult your driver's documentation for
+ supported values.
+ type: object
+ required:
+ - driver
+ type: object
+ downwardAPI:
+ description: downwardAPI represents downward API
+ about the pod that should populate this volume
+ properties:
+ defaultMode:
+ description: 'Optional: mode bits to use on
+ created files by default. Must be a Optional:
+ mode bits used to set permissions on created
+ files by default. Must be an octal value
+ between 0000 and 0777 or a decimal value
+ between 0 and 511. YAML accepts both octal
+ and decimal values, JSON requires decimal
+ values for mode bits. Defaults to 0644.
+ Directories within the path are not affected
+ by this setting. This might be in conflict
+ with other options that affect the file
+ mode, like fsGroup, and the result can be
+ other mode bits set.'
+ format: int32
+ type: integer
+ items:
+ description: Items is a list of downward API
+ volume file
+ items:
+ description: DownwardAPIVolumeFile represents
+ information to create the file containing
+ the pod field
+ properties:
+ fieldRef:
+ description: 'Required: Selects a field
+ of the pod: only annotations, labels,
+ name and namespace are supported.'
+ properties:
+ apiVersion:
+ description: Version of the schema
+ the FieldPath is written in terms
+ of, defaults to "v1".
+ type: string
+ fieldPath:
+ description: Path of the field to
+ select in the specified API version.
+ type: string
+ required:
+ - fieldPath
+ type: object
+ x-kubernetes-map-type: atomic
+ mode:
+ description: 'Optional: mode bits used
+ to set permissions on this file, must
+ be an octal value between 0000 and
+ 0777 or a decimal value between 0
+ and 511. YAML accepts both octal and
+ decimal values, JSON requires decimal
+ values for mode bits. If not specified,
+ the volume defaultMode will be used.
+ This might be in conflict with other
+ options that affect the file mode,
+ like fsGroup, and the result can be
+ other mode bits set.'
+ format: int32
+ type: integer
+ path:
+ description: 'Required: Path is the
+ relative path name of the file to
+ be created. Must not be absolute or
+ contain the ''..'' path. Must be utf-8
+ encoded. The first item of the relative
+ path must not start with ''..'''
+ type: string
+ resourceFieldRef:
+ description: 'Selects a resource of
+ the container: only resources limits
+ and requests (limits.cpu, limits.memory,
+ requests.cpu and requests.memory)
+ are currently supported.'
+ properties:
+ containerName:
+ description: 'Container name: required
+ for volumes, optional for env
+ vars'
+ type: string
+ divisor:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Specifies the output
+ format of the exposed resources,
+ defaults to "1"
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ resource:
+ description: 'Required: resource
+ to select'
+ type: string
+ required:
+ - resource
+ type: object
+ x-kubernetes-map-type: atomic
+ required:
+ - path
+ type: object
+ type: array
+ type: object
+ emptyDir:
+ description: 'emptyDir represents a temporary
+ directory that shares a pod''s lifetime. More
+ info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir'
+ properties:
+ medium:
+ description: 'medium represents what type
+ of storage medium should back this directory.
+ The default is "" which means to use the
+ node''s default medium. Must be an empty
+ string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir'
+ type: string
+ sizeLimit:
+ anyOf:
+ - type: integer
+ - type: string
+ description: 'sizeLimit is the total amount
+ of local storage required for this EmptyDir
+ volume. The size limit is also applicable
+ for memory medium. The maximum usage on
+ memory medium EmptyDir would be the minimum
+ value between the SizeLimit specified here
+ and the sum of memory limits of all containers
+ in a pod. The default is nil which means
+ that the limit is undefined. More info:
+ http://kubernetes.io/docs/user-guide/volumes#emptydir'
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ type: object
+ ephemeral:
+ description: "ephemeral represents a volume that
+ is handled by a cluster storage driver. The
+ volume's lifecycle is tied to the pod that defines
+ it - it will be created before the pod starts,
+ and deleted when the pod is removed. \n Use
+ this if: a) the volume is only needed while
+ the pod runs, b) features of normal volumes
+ like restoring from snapshot or capacity tracking
+ are needed, c) the storage driver is specified
+ through a storage class, and d) the storage
+ driver supports dynamic volume provisioning
+ through a PersistentVolumeClaim (see EphemeralVolumeSource
+ for more information on the connection between
+ this volume type and PersistentVolumeClaim).
+ \n Use PersistentVolumeClaim or one of the vendor-specific
+ APIs for volumes that persist for longer than
+ the lifecycle of an individual pod. \n Use CSI
+ for light-weight local ephemeral volumes if
+ the CSI driver is meant to be used that way
+ - see the documentation of the driver for more
+ information. \n A pod can use both types of
+ ephemeral volumes and persistent volumes at
+ the same time."
+ properties:
+ volumeClaimTemplate:
+ description: "Will be used to create a stand-alone
+ PVC to provision the volume. The pod in
+ which this EphemeralVolumeSource is embedded
+ will be the owner of the PVC, i.e. the PVC
+ will be deleted together with the pod. The
+ name of the PVC will be `-` where `` is the name
+ from the `PodSpec.Volumes` array entry.
+ Pod validation will reject the pod if the
+ concatenated name is not valid for a PVC
+ (for example, too long). \n An existing
+ PVC with that name that is not owned by
+ the pod will *not* be used for the pod to
+ avoid using an unrelated volume by mistake.
+ Starting the pod is then blocked until the
+ unrelated PVC is removed. If such a pre-created
+ PVC is meant to be used by the pod, the
+ PVC has to updated with an owner reference
+ to the pod once the pod exists. Normally
+ this should not be necessary, but it may
+ be useful when manually reconstructing a
+ broken cluster. \n This field is read-only
+ and no changes will be made by Kubernetes
+ to the PVC after it has been created. \n
+ Required, must not be nil."
+ properties:
+ metadata:
+ description: May contain labels and annotations
+ that will be copied into the PVC when
+ creating it. No other fields are allowed
+ and will be rejected during validation.
+ type: object
+ spec:
+ description: The specification for the
+ PersistentVolumeClaim. The entire content
+ is copied unchanged into the PVC that
+ gets created from this template. The
+ same fields as in a PersistentVolumeClaim
+ are also valid here.
+ properties:
+ accessModes:
+ description: 'accessModes contains
+ the desired access modes the volume
+ should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1'
+ items:
+ type: string
+ type: array
+ dataSource:
+ description: 'dataSource field can
+ be used to specify either: * An
+ existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot)
+ * An existing PVC (PersistentVolumeClaim)
+ If the provisioner or an external
+ controller can support the specified
+ data source, it will create a new
+ volume based on the contents of
+ the specified data source. When
+ the AnyVolumeDataSource feature
+ gate is enabled, dataSource contents
+ will be copied to dataSourceRef,
+ and dataSourceRef contents will
+ be copied to dataSource when dataSourceRef.namespace
+ is not specified. If the namespace
+ is specified, then dataSourceRef
+ will not be copied to dataSource.'
+ properties:
+ apiGroup:
+ description: APIGroup is the group
+ for the resource being referenced.
+ If APIGroup is not specified,
+ the specified Kind must be in
+ the core API group. For any
+ other third-party types, APIGroup
+ is required.
+ type: string
+ kind:
+ description: Kind is the type
+ of resource being referenced
+ type: string
+ name:
+ description: Name is the name
+ of resource being referenced
+ type: string
+ required:
+ - kind
+ - name
+ type: object
+ x-kubernetes-map-type: atomic
+ dataSourceRef:
+ description: 'dataSourceRef specifies
+ the object from which to populate
+ the volume with data, if a non-empty
+ volume is desired. This may be any
+ object from a non-empty API group
+ (non core object) or a PersistentVolumeClaim
+ object. When this field is specified,
+ volume binding will only succeed
+ if the type of the specified object
+ matches some installed volume populator
+ or dynamic provisioner. This field
+ will replace the functionality of
+ the dataSource field and as such
+ if both fields are non-empty, they
+ must have the same value. For backwards
+ compatibility, when namespace isn''t
+ specified in dataSourceRef, both
+ fields (dataSource and dataSourceRef)
+ will be set to the same value automatically
+ if one of them is empty and the
+ other is non-empty. When namespace
+ is specified in dataSourceRef, dataSource
+ isn''t set to the same value and
+ must be empty. There are three important
+ differences between dataSource and
+ dataSourceRef: * While dataSource
+ only allows two specific types of
+ objects, dataSourceRef allows any
+ non-core object, as well as PersistentVolumeClaim
+ objects. * While dataSource ignores
+ disallowed values (dropping them),
+ dataSourceRef preserves all values,
+ and generates an error if a disallowed
+ value is specified. * While dataSource
+ only allows local objects, dataSourceRef
+ allows objects in any namespaces.
+ (Beta) Using this field requires
+ the AnyVolumeDataSource feature
+ gate to be enabled. (Alpha) Using
+ the namespace field of dataSourceRef
+ requires the CrossNamespaceVolumeDataSource
+ feature gate to be enabled.'
+ properties:
+ apiGroup:
+ description: APIGroup is the group
+ for the resource being referenced.
+ If APIGroup is not specified,
+ the specified Kind must be in
+ the core API group. For any
+ other third-party types, APIGroup
+ is required.
+ type: string
+ kind:
+ description: Kind is the type
+ of resource being referenced
+ type: string
+ name:
+ description: Name is the name
+ of resource being referenced
+ type: string
+ namespace:
+ description: Namespace is the
+ namespace of resource being
+ referenced Note that when a
+ namespace is specified, a gateway.networking.k8s.io/ReferenceGrant
+ object is required in the referent
+ namespace to allow that namespace's
+ owner to accept the reference.
+ See the ReferenceGrant documentation
+ for details. (Alpha) This field
+ requires the CrossNamespaceVolumeDataSource
+ feature gate to be enabled.
+ type: string
+ required:
+ - kind
+ - name
+ type: object
+ resources:
+ description: 'resources represents
+ the minimum resources the volume
+ should have. If RecoverVolumeExpansionFailure
+ feature is enabled users are allowed
+ to specify resource requirements
+ that are lower than previous value
+ but must still be higher than capacity
+ recorded in the status field of
+ the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources'
+ properties:
+ claims:
+ description: "Claims lists the
+ names of resources, defined
+ in spec.resourceClaims, that
+ are used by this container.
+ \n This is an alpha field and
+ requires enabling the DynamicResourceAllocation
+ feature gate. \n This field
+ is immutable. It can only be
+ set for containers."
+ items:
+ description: ResourceClaim references
+ one entry in PodSpec.ResourceClaims.
+ properties:
+ name:
+ description: Name must match
+ the name of one entry
+ in pod.spec.resourceClaims
+ of the Pod where this
+ field is used. It makes
+ that resource available
+ inside a container.
+ type: string
+ required:
+ - name
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - name
+ x-kubernetes-list-type: map
+ limits:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: 'Limits describes
+ the maximum amount of compute
+ resources allowed. More info:
+ https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ type: object
+ requests:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: 'Requests describes
+ the minimum amount of compute
+ resources required. If Requests
+ is omitted for a container,
+ it defaults to Limits if that
+ is explicitly specified, otherwise
+ to an implementation-defined
+ value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
+ type: object
+ type: object
+ selector:
+ description: selector is a label query
+ over volumes to consider for binding.
+ properties:
+ matchExpressions:
+ description: matchExpressions
+ is a list of label selector
+ requirements. The requirements
+ are ANDed.
+ items:
+ description: A label selector
+ requirement is a selector
+ that contains values, a key,
+ and an operator that relates
+ the key and values.
+ properties:
+ key:
+ description: key is the
+ label key that the selector
+ applies to.
+ type: string
+ operator:
+ description: operator represents
+ a key's relationship to
+ a set of values. Valid
+ operators are In, NotIn,
+ Exists and DoesNotExist.
+ type: string
+ values:
+ description: values is an
+ array of string values.
+ If the operator is In
+ or NotIn, the values array
+ must be non-empty. If
+ the operator is Exists
+ or DoesNotExist, the values
+ array must be empty. This
+ array is replaced during
+ a strategic merge patch.
+ items:
+ type: string
+ type: array
+ required:
+ - key
+ - operator
+ type: object
+ type: array
+ matchLabels:
+ additionalProperties:
+ type: string
+ description: matchLabels is a
+ map of {key,value} pairs. A
+ single {key,value} in the matchLabels
+ map is equivalent to an element
+ of matchExpressions, whose key
+ field is "key", the operator
+ is "In", and the values array
+ contains only "value". The requirements
+ are ANDed.
+ type: object
+ type: object
+ x-kubernetes-map-type: atomic
+ storageClassName:
+ description: 'storageClassName is
+ the name of the StorageClass required
+ by the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1'
+ type: string
+ volumeMode:
+ description: volumeMode defines what
+ type of volume is required by the
+ claim. Value of Filesystem is implied
+ when not included in claim spec.
+ type: string
+ volumeName:
+ description: volumeName is the binding
+ reference to the PersistentVolume
+ backing this claim.
+ type: string
+ type: object
+ required:
+ - spec
+ type: object
+ type: object
+ fc:
+ description: fc represents a Fibre Channel resource
+ that is attached to a kubelet's host machine
+ and then exposed to the pod.
+ properties:
+ fsType:
+ description: 'fsType is the filesystem type
+ to mount. Must be a filesystem type supported
+ by the host operating system. Ex. "ext4",
+ "xfs", "ntfs". Implicitly inferred to be
+ "ext4" if unspecified. TODO: how do we prevent
+ errors in the filesystem from compromising
+ the machine'
+ type: string
+ lun:
+ description: 'lun is Optional: FC target lun
+ number'
+ format: int32
+ type: integer
+ readOnly:
+ description: 'readOnly is Optional: Defaults
+ to false (read/write). ReadOnly here will
+ force the ReadOnly setting in VolumeMounts.'
+ type: boolean
+ targetWWNs:
+ description: 'targetWWNs is Optional: FC target
+ worldwide names (WWNs)'
+ items:
+ type: string
+ type: array
+ wwids:
+ description: 'wwids Optional: FC volume world
+ wide identifiers (wwids) Either wwids or
+ combination of targetWWNs and lun must be
+ set, but not both simultaneously.'
+ items:
+ type: string
+ type: array
+ type: object
+ flexVolume:
+ description: flexVolume represents a generic volume
+ resource that is provisioned/attached using
+ an exec based plugin.
+ properties:
+ driver:
+ description: driver is the name of the driver
+ to use for this volume.
+ type: string
+ fsType:
+ description: fsType is the filesystem type
+ to mount. Must be a filesystem type supported
+ by the host operating system. Ex. "ext4",
+ "xfs", "ntfs". The default filesystem depends
+ on FlexVolume script.
+ type: string
+ options:
+ additionalProperties:
+ type: string
+ description: 'options is Optional: this field
+ holds extra command options if any.'
+ type: object
+ readOnly:
+ description: 'readOnly is Optional: defaults
+ to false (read/write). ReadOnly here will
+ force the ReadOnly setting in VolumeMounts.'
+ type: boolean
+ secretRef:
+ description: 'secretRef is Optional: secretRef
+ is reference to the secret object containing
+ sensitive information to pass to the plugin
+ scripts. This may be empty if no secret
+ object is specified. If the secret object
+ contains more than one secret, all secrets
+ are passed to the plugin scripts.'
+ properties:
+ name:
+ description: 'Name of the referent. More
+ info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ type: object
+ x-kubernetes-map-type: atomic
+ required:
+ - driver
+ type: object
+ flocker:
+ description: flocker represents a Flocker volume
+ attached to a kubelet's host machine. This depends
+ on the Flocker control service being running
+ properties:
+ datasetName:
+ description: datasetName is Name of the dataset
+ stored as metadata -> name on the dataset
+ for Flocker should be considered as deprecated
+ type: string
+ datasetUUID:
+ description: datasetUUID is the UUID of the
+ dataset. This is unique identifier of a
+ Flocker dataset
+ type: string
+ type: object
+ gcePersistentDisk:
+ description: 'gcePersistentDisk represents a GCE
+ Disk resource that is attached to a kubelet''s
+ host machine and then exposed to the pod. More
+ info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk'
+ properties:
+ fsType:
+ description: 'fsType is filesystem type of
+ the volume that you want to mount. Tip:
+ Ensure that the filesystem type is supported
+ by the host operating system. Examples:
+ "ext4", "xfs", "ntfs". Implicitly inferred
+ to be "ext4" if unspecified. More info:
+ https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
+ TODO: how do we prevent errors in the filesystem
+ from compromising the machine'
+ type: string
+ partition:
+ description: 'partition is the partition in
+ the volume that you want to mount. If omitted,
+ the default is to mount by volume name.
+ Examples: For volume /dev/sda1, you specify
+ the partition as "1". Similarly, the volume
+ partition for /dev/sda is "0" (or you can
+ leave the property empty). More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk'
+ format: int32
+ type: integer
+ pdName:
+ description: 'pdName is unique name of the
+ PD resource in GCE. Used to identify the
+ disk in GCE. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk'
+ type: string
+ readOnly:
+ description: 'readOnly here will force the
+ ReadOnly setting in VolumeMounts. Defaults
+ to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk'
+ type: boolean
+ required:
+ - pdName
+ type: object
+ gitRepo:
+ description: 'gitRepo represents a git repository
+ at a particular revision. DEPRECATED: GitRepo
+ is deprecated. To provision a container with
+ a git repo, mount an EmptyDir into an InitContainer
+ that clones the repo using git, then mount the
+ EmptyDir into the Pod''s container.'
+ properties:
+ directory:
+ description: directory is the target directory
+ name. Must not contain or start with '..'. If
+ '.' is supplied, the volume directory will
+ be the git repository. Otherwise, if specified,
+ the volume will contain the git repository
+ in the subdirectory with the given name.
+ type: string
+ repository:
+ description: repository is the URL
+ type: string
+ revision:
+ description: revision is the commit hash for
+ the specified revision.
+ type: string
+ required:
+ - repository
+ type: object
+ glusterfs:
+ description: 'glusterfs represents a Glusterfs
+ mount on the host that shares a pod''s lifetime.
+ More info: https://examples.k8s.io/volumes/glusterfs/README.md'
+ properties:
+ endpoints:
+ description: 'endpoints is the endpoint name
+ that details Glusterfs topology. More info:
+ https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod'
+ type: string
+ path:
+ description: 'path is the Glusterfs volume
+ path. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod'
+ type: string
+ readOnly:
+ description: 'readOnly here will force the
+ Glusterfs volume to be mounted with read-only
+ permissions. Defaults to false. More info:
+ https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod'
+ type: boolean
+ required:
+ - endpoints
+ - path
+ type: object
+ hostPath:
+ description: 'hostPath represents a pre-existing
+ file or directory on the host machine that is
+ directly exposed to the container. This is generally
+ used for system agents or other privileged things
+ that are allowed to see the host machine. Most
+ containers will NOT need this. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath
+ --- TODO(jonesdl) We need to restrict who can
+ use host directory mounts and who can/can not
+ mount host directories as read/write.'
+ properties:
+ path:
+ description: 'path of the directory on the
+ host. If the path is a symlink, it will
+ follow the link to the real path. More info:
+ https://kubernetes.io/docs/concepts/storage/volumes#hostpath'
+ type: string
+ type:
+ description: 'type for HostPath Volume Defaults
+ to "" More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath'
+ type: string
+ required:
+ - path
+ type: object
+ iscsi:
+ description: 'iscsi represents an ISCSI Disk resource
+ that is attached to a kubelet''s host machine
+ and then exposed to the pod. More info: https://examples.k8s.io/volumes/iscsi/README.md'
+ properties:
+ chapAuthDiscovery:
+ description: chapAuthDiscovery defines whether
+ support iSCSI Discovery CHAP authentication
+ type: boolean
+ chapAuthSession:
+ description: chapAuthSession defines whether
+ support iSCSI Session CHAP authentication
+ type: boolean
+ fsType:
+ description: 'fsType is the filesystem type
+ of the volume that you want to mount. Tip:
+ Ensure that the filesystem type is supported
+ by the host operating system. Examples:
+ "ext4", "xfs", "ntfs". Implicitly inferred
+ to be "ext4" if unspecified. More info:
+ https://kubernetes.io/docs/concepts/storage/volumes#iscsi
+ TODO: how do we prevent errors in the filesystem
+ from compromising the machine'
+ type: string
+ initiatorName:
+ description: initiatorName is the custom iSCSI
+ Initiator Name. If initiatorName is specified
+ with iscsiInterface simultaneously, new
+ iSCSI interface : will be created for the connection.
+ type: string
+ iqn:
+ description: iqn is the target iSCSI Qualified
+ Name.
+ type: string
+ iscsiInterface:
+ description: iscsiInterface is the interface
+ Name that uses an iSCSI transport. Defaults
+ to 'default' (tcp).
+ type: string
+ lun:
+ description: lun represents iSCSI Target Lun
+ number.
+ format: int32
+ type: integer
+ portals:
+ description: portals is the iSCSI Target Portal
+ List. The portal is either an IP or ip_addr:port
+ if the port is other than default (typically
+ TCP ports 860 and 3260).
+ items:
+ type: string
+ type: array
+ readOnly:
+ description: readOnly here will force the
+ ReadOnly setting in VolumeMounts. Defaults
+ to false.
+ type: boolean
+ secretRef:
+ description: secretRef is the CHAP Secret
+ for iSCSI target and initiator authentication
+ properties:
+ name:
+ description: 'Name of the referent. More
+ info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ type: object
+ x-kubernetes-map-type: atomic
+ targetPortal:
+ description: targetPortal is iSCSI Target
+ Portal. The Portal is either an IP or ip_addr:port
+ if the port is other than default (typically
+ TCP ports 860 and 3260).
+ type: string
+ required:
+ - iqn
+ - lun
+ - targetPortal
+ type: object
+ name:
+ description: 'name of the volume. Must be a DNS_LABEL
+ and unique within the pod. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names'
+ type: string
+ nfs:
+ description: 'nfs represents an NFS mount on the
+ host that shares a pod''s lifetime More info:
+ https://kubernetes.io/docs/concepts/storage/volumes#nfs'
+ properties:
+ path:
+ description: 'path that is exported by the
+ NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs'
+ type: string
+ readOnly:
+ description: 'readOnly here will force the
+ NFS export to be mounted with read-only
+ permissions. Defaults to false. More info:
+ https://kubernetes.io/docs/concepts/storage/volumes#nfs'
+ type: boolean
+ server:
+ description: 'server is the hostname or IP
+ address of the NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs'
+ type: string
+ required:
+ - path
+ - server
+ type: object
+ persistentVolumeClaim:
+ description: 'persistentVolumeClaimVolumeSource
+ represents a reference to a PersistentVolumeClaim
+ in the same namespace. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims'
+ properties:
+ claimName:
+ description: 'claimName is the name of a PersistentVolumeClaim
+ in the same namespace as the pod using this
+ volume. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims'
+ type: string
+ readOnly:
+ description: readOnly Will force the ReadOnly
+ setting in VolumeMounts. Default false.
+ type: boolean
+ required:
+ - claimName
+ type: object
+ photonPersistentDisk:
+ description: photonPersistentDisk represents a
+ PhotonController persistent disk attached and
+ mounted on kubelets host machine
+ properties:
+ fsType:
+ description: fsType is the filesystem type
+ to mount. Must be a filesystem type supported
+ by the host operating system. Ex. "ext4",
+ "xfs", "ntfs". Implicitly inferred to be
+ "ext4" if unspecified.
+ type: string
+ pdID:
+ description: pdID is the ID that identifies
+ Photon Controller persistent disk
+ type: string
+ required:
+ - pdID
+ type: object
+ portworxVolume:
+ description: portworxVolume represents a portworx
+ volume attached and mounted on kubelets host
+ machine
+ properties:
+ fsType:
+ description: fSType represents the filesystem
+ type to mount Must be a filesystem type
+ supported by the host operating system.
+ Ex. "ext4", "xfs". Implicitly inferred to
+ be "ext4" if unspecified.
+ type: string
+ readOnly:
+ description: readOnly defaults to false (read/write).
+ ReadOnly here will force the ReadOnly setting
+ in VolumeMounts.
+ type: boolean
+ volumeID:
+ description: volumeID uniquely identifies
+ a Portworx volume
+ type: string
+ required:
+ - volumeID
+ type: object
+ projected:
+ description: projected items for all in one resources
+ secrets, configmaps, and downward API
+ properties:
+ defaultMode:
+ description: defaultMode are the mode bits
+ used to set permissions on created files
+ by default. Must be an octal value between
+ 0000 and 0777 or a decimal value between
+ 0 and 511. YAML accepts both octal and decimal
+ values, JSON requires decimal values for
+ mode bits. Directories within the path are
+ not affected by this setting. This might
+ be in conflict with other options that affect
+ the file mode, like fsGroup, and the result
+ can be other mode bits set.
+ format: int32
+ type: integer
+ sources:
+ description: sources is the list of volume
+ projections
+ items:
+ description: Projection that may be projected
+ along with other supported volume types
+ properties:
+ configMap:
+ description: configMap information about
+ the configMap data to project
+ properties:
+ items:
+ description: items if unspecified,
+ each key-value pair in the Data
+ field of the referenced ConfigMap
+ will be projected into the volume
+ as a file whose name is the key
+ and content is the value. If specified,
+ the listed keys will be projected
+ into the specified paths, and
+ unlisted keys will not be present.
+ If a key is specified which is
+ not present in the ConfigMap,
+ the volume setup will error unless
+ it is marked optional. Paths must
+ be relative and may not contain
+ the '..' path or start with '..'.
+ items:
+ description: Maps a string key
+ to a path within a volume.
+ properties:
+ key:
+ description: key is the key
+ to project.
+ type: string
+ mode:
+ description: 'mode is Optional:
+ mode bits used to set permissions
+ on this file. Must be an
+ octal value between 0000
+ and 0777 or a decimal value
+ between 0 and 511. YAML
+ accepts both octal and decimal
+ values, JSON requires decimal
+ values for mode bits. If
+ not specified, the volume
+ defaultMode will be used.
+ This might be in conflict
+ with other options that
+ affect the file mode, like
+ fsGroup, and the result
+ can be other mode bits set.'
+ format: int32
+ type: integer
+ path:
+ description: path is the relative
+ path of the file to map
+ the key to. May not be an
+ absolute path. May not contain
+ the path element '..'. May
+ not start with the string
+ '..'.
+ type: string
+ required:
+ - key
+ - path
+ type: object
+ type: array
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields.
+ apiVersion, kind, uid?'
+ type: string
+ optional:
+ description: optional specify whether
+ the ConfigMap or its keys must
+ be defined
+ type: boolean
+ type: object
+ x-kubernetes-map-type: atomic
+ downwardAPI:
+ description: downwardAPI information
+ about the downwardAPI data to project
+ properties:
+ items:
+ description: Items is a list of
+ DownwardAPIVolume file
+ items:
+ description: DownwardAPIVolumeFile
+ represents information to create
+ the file containing the pod
+ field
+ properties:
+ fieldRef:
+ description: 'Required: Selects
+ a field of the pod: only
+ annotations, labels, name
+ and namespace are supported.'
+ properties:
+ apiVersion:
+ description: Version of
+ the schema the FieldPath
+ is written in terms
+ of, defaults to "v1".
+ type: string
+ fieldPath:
+ description: Path of the
+ field to select in the
+ specified API version.
+ type: string
+ required:
+ - fieldPath
+ type: object
+ x-kubernetes-map-type: atomic
+ mode:
+ description: 'Optional: mode
+ bits used to set permissions
+ on this file, must be an
+ octal value between 0000
+ and 0777 or a decimal value
+ between 0 and 511. YAML
+ accepts both octal and decimal
+ values, JSON requires decimal
+ values for mode bits. If
+ not specified, the volume
+ defaultMode will be used.
+ This might be in conflict
+ with other options that
+ affect the file mode, like
+ fsGroup, and the result
+ can be other mode bits set.'
+ format: int32
+ type: integer
+ path:
+ description: 'Required: Path
+ is the relative path name
+ of the file to be created.
+ Must not be absolute or
+ contain the ''..'' path.
+ Must be utf-8 encoded. The
+ first item of the relative
+ path must not start with
+ ''..'''
+ type: string
+ resourceFieldRef:
+ description: 'Selects a resource
+ of the container: only resources
+ limits and requests (limits.cpu,
+ limits.memory, requests.cpu
+ and requests.memory) are
+ currently supported.'
+ properties:
+ containerName:
+ description: 'Container
+ name: required for volumes,
+ optional for env vars'
+ type: string
+ divisor:
+ anyOf:
+ - type: integer
+ - type: string
+ description: Specifies
+ the output format of
+ the exposed resources,
+ defaults to "1"
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ resource:
+ description: 'Required:
+ resource to select'
+ type: string
+ required:
+ - resource
+ type: object
+ x-kubernetes-map-type: atomic
+ required:
+ - path
+ type: object
+ type: array
+ type: object
+ secret:
+ description: secret information about
+ the secret data to project
+ properties:
+ items:
+ description: items if unspecified,
+ each key-value pair in the Data
+ field of the referenced Secret
+ will be projected into the volume
+ as a file whose name is the key
+ and content is the value. If specified,
+ the listed keys will be projected
+ into the specified paths, and
+ unlisted keys will not be present.
+ If a key is specified which is
+ not present in the Secret, the
+ volume setup will error unless
+ it is marked optional. Paths must
+ be relative and may not contain
+ the '..' path or start with '..'.
+ items:
+ description: Maps a string key
+ to a path within a volume.
+ properties:
+ key:
+ description: key is the key
+ to project.
+ type: string
+ mode:
+ description: 'mode is Optional:
+ mode bits used to set permissions
+ on this file. Must be an
+ octal value between 0000
+ and 0777 or a decimal value
+ between 0 and 511. YAML
+ accepts both octal and decimal
+ values, JSON requires decimal
+ values for mode bits. If
+ not specified, the volume
+ defaultMode will be used.
+ This might be in conflict
+ with other options that
+ affect the file mode, like
+ fsGroup, and the result
+ can be other mode bits set.'
+ format: int32
+ type: integer
+ path:
+ description: path is the relative
+ path of the file to map
+ the key to. May not be an
+ absolute path. May not contain
+ the path element '..'. May
+ not start with the string
+ '..'.
+ type: string
+ required:
+ - key
+ - path
+ type: object
+ type: array
+ name:
+ description: 'Name of the referent.
+ More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields.
+ apiVersion, kind, uid?'
+ type: string
+ optional:
+ description: optional field specify
+ whether the Secret or its key
+ must be defined
+ type: boolean
+ type: object
+ x-kubernetes-map-type: atomic
+ serviceAccountToken:
+ description: serviceAccountToken is
+ information about the serviceAccountToken
+ data to project
+ properties:
+ audience:
+ description: audience is the intended
+ audience of the token. A recipient
+ of a token must identify itself
+ with an identifier specified in
+ the audience of the token, and
+ otherwise should reject the token.
+ The audience defaults to the identifier
+ of the apiserver.
+ type: string
+ expirationSeconds:
+ description: expirationSeconds is
+ the requested duration of validity
+ of the service account token.
+ As the token approaches expiration,
+ the kubelet volume plugin will
+ proactively rotate the service
+ account token. The kubelet will
+ start trying to rotate the token
+ if the token is older than 80
+ percent of its time to live or
+ if the token is older than 24
+ hours.Defaults to 1 hour and must
+ be at least 10 minutes.
+ format: int64
+ type: integer
+ path:
+ description: path is the path relative
+ to the mount point of the file
+ to project the token into.
+ type: string
+ required:
+ - path
+ type: object
+ type: object
+ type: array
+ type: object
+ quobyte:
+ description: quobyte represents a Quobyte mount
+ on the host that shares a pod's lifetime
+ properties:
+ group:
+ description: group to map volume access to
+ Default is no group
+ type: string
+ readOnly:
+ description: readOnly here will force the
+ Quobyte volume to be mounted with read-only
+ permissions. Defaults to false.
+ type: boolean
+ registry:
+ description: registry represents a single
+ or multiple Quobyte Registry services specified
+ as a string as host:port pair (multiple
+ entries are separated with commas) which
+ acts as the central registry for volumes
+ type: string
+ tenant:
+ description: tenant owning the given Quobyte
+ volume in the Backend Used with dynamically
+ provisioned Quobyte volumes, value is set
+ by the plugin
+ type: string
+ user:
+ description: user to map volume access to
+ Defaults to serivceaccount user
+ type: string
+ volume:
+ description: volume is a string that references
+ an already created Quobyte volume by name.
+ type: string
+ required:
+ - registry
+ - volume
+ type: object
+ rbd:
+ description: 'rbd represents a Rados Block Device
+ mount on the host that shares a pod''s lifetime.
+ More info: https://examples.k8s.io/volumes/rbd/README.md'
+ properties:
+ fsType:
+ description: 'fsType is the filesystem type
+ of the volume that you want to mount. Tip:
+ Ensure that the filesystem type is supported
+ by the host operating system. Examples:
+ "ext4", "xfs", "ntfs". Implicitly inferred
+ to be "ext4" if unspecified. More info:
+ https://kubernetes.io/docs/concepts/storage/volumes#rbd
+ TODO: how do we prevent errors in the filesystem
+ from compromising the machine'
+ type: string
+ image:
+ description: 'image is the rados image name.
+ More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it'
+ type: string
+ keyring:
+ description: 'keyring is the path to key ring
+ for RBDUser. Default is /etc/ceph/keyring.
+ More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it'
+ type: string
+ monitors:
+ description: 'monitors is a collection of
+ Ceph monitors. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it'
+ items:
+ type: string
+ type: array
+ pool:
+ description: 'pool is the rados pool name.
+ Default is rbd. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it'
+ type: string
+ readOnly:
+ description: 'readOnly here will force the
+ ReadOnly setting in VolumeMounts. Defaults
+ to false. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it'
+ type: boolean
+ secretRef:
+ description: 'secretRef is name of the authentication
+ secret for RBDUser. If provided overrides
+ keyring. Default is nil. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it'
+ properties:
+ name:
+ description: 'Name of the referent. More
+ info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ type: object
+ x-kubernetes-map-type: atomic
+ user:
+ description: 'user is the rados user name.
+ Default is admin. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it'
+ type: string
+ required:
+ - image
+ - monitors
+ type: object
+ scaleIO:
+ description: scaleIO represents a ScaleIO persistent
+ volume attached and mounted on Kubernetes nodes.
+ properties:
+ fsType:
+ description: fsType is the filesystem type
+ to mount. Must be a filesystem type supported
+ by the host operating system. Ex. "ext4",
+ "xfs", "ntfs". Default is "xfs".
+ type: string
+ gateway:
+ description: gateway is the host address of
+ the ScaleIO API Gateway.
+ type: string
+ protectionDomain:
+ description: protectionDomain is the name
+ of the ScaleIO Protection Domain for the
+ configured storage.
+ type: string
+ readOnly:
+ description: readOnly Defaults to false (read/write).
+ ReadOnly here will force the ReadOnly setting
+ in VolumeMounts.
+ type: boolean
+ secretRef:
+ description: secretRef references to the secret
+ for ScaleIO user and other sensitive information.
+ If this is not provided, Login operation
+ will fail.
+ properties:
+ name:
+ description: 'Name of the referent. More
+ info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ type: object
+ x-kubernetes-map-type: atomic
+ sslEnabled:
+ description: sslEnabled Flag enable/disable
+ SSL communication with Gateway, default
+ false
+ type: boolean
+ storageMode:
+ description: storageMode indicates whether
+ the storage for a volume should be ThickProvisioned
+ or ThinProvisioned. Default is ThinProvisioned.
+ type: string
+ storagePool:
+ description: storagePool is the ScaleIO Storage
+ Pool associated with the protection domain.
+ type: string
+ system:
+ description: system is the name of the storage
+ system as configured in ScaleIO.
+ type: string
+ volumeName:
+ description: volumeName is the name of a volume
+ already created in the ScaleIO system that
+ is associated with this volume source.
+ type: string
+ required:
+ - gateway
+ - secretRef
+ - system
+ type: object
+ secret:
+ description: 'secret represents a secret that
+ should populate this volume. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret'
+ properties:
+ defaultMode:
+ description: 'defaultMode is Optional: mode
+ bits used to set permissions on created
+ files by default. Must be an octal value
+ between 0000 and 0777 or a decimal value
+ between 0 and 511. YAML accepts both octal
+ and decimal values, JSON requires decimal
+ values for mode bits. Defaults to 0644.
+ Directories within the path are not affected
+ by this setting. This might be in conflict
+ with other options that affect the file
+ mode, like fsGroup, and the result can be
+ other mode bits set.'
+ format: int32
+ type: integer
+ items:
+ description: items If unspecified, each key-value
+ pair in the Data field of the referenced
+ Secret will be projected into the volume
+ as a file whose name is the key and content
+ is the value. If specified, the listed keys
+ will be projected into the specified paths,
+ and unlisted keys will not be present. If
+ a key is specified which is not present
+ in the Secret, the volume setup will error
+ unless it is marked optional. Paths must
+ be relative and may not contain the '..'
+ path or start with '..'.
+ items:
+ description: Maps a string key to a path
+ within a volume.
+ properties:
+ key:
+ description: key is the key to project.
+ type: string
+ mode:
+ description: 'mode is Optional: mode
+ bits used to set permissions on this
+ file. Must be an octal value between
+ 0000 and 0777 or a decimal value between
+ 0 and 511. YAML accepts both octal
+ and decimal values, JSON requires
+ decimal values for mode bits. If not
+ specified, the volume defaultMode
+ will be used. This might be in conflict
+ with other options that affect the
+ file mode, like fsGroup, and the result
+ can be other mode bits set.'
+ format: int32
+ type: integer
+ path:
+ description: path is the relative path
+ of the file to map the key to. May
+ not be an absolute path. May not contain
+ the path element '..'. May not start
+ with the string '..'.
+ type: string
+ required:
+ - key
+ - path
+ type: object
+ type: array
+ optional:
+ description: optional field specify whether
+ the Secret or its keys must be defined
+ type: boolean
+ secretName:
+ description: 'secretName is the name of the
+ secret in the pod''s namespace to use. More
+ info: https://kubernetes.io/docs/concepts/storage/volumes#secret'
+ type: string
+ type: object
+ storageos:
+ description: storageOS represents a StorageOS
+ volume attached and mounted on Kubernetes nodes.
+ properties:
+ fsType:
+ description: fsType is the filesystem type
+ to mount. Must be a filesystem type supported
+ by the host operating system. Ex. "ext4",
+ "xfs", "ntfs". Implicitly inferred to be
+ "ext4" if unspecified.
+ type: string
+ readOnly:
+ description: readOnly defaults to false (read/write).
+ ReadOnly here will force the ReadOnly setting
+ in VolumeMounts.
+ type: boolean
+ secretRef:
+ description: secretRef specifies the secret
+ to use for obtaining the StorageOS API credentials. If
+ not specified, default values will be attempted.
+ properties:
+ name:
+ description: 'Name of the referent. More
+ info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+ TODO: Add other useful fields. apiVersion,
+ kind, uid?'
+ type: string
+ type: object
+ x-kubernetes-map-type: atomic
+ volumeName:
+ description: volumeName is the human-readable
+ name of the StorageOS volume. Volume names
+ are only unique within a namespace.
+ type: string
+ volumeNamespace:
+ description: volumeNamespace specifies the
+ scope of the volume within StorageOS. If
+ no namespace is specified then the Pod's
+ namespace will be used. This allows the
+ Kubernetes name scoping to be mirrored within
+ StorageOS for tighter integration. Set VolumeName
+ to any name to override the default behaviour.
+ Set to "default" if you are not using namespaces
+ within StorageOS. Namespaces that do not
+ pre-exist within StorageOS will be created.
+ type: string
+ type: object
+ vsphereVolume:
+ description: vsphereVolume represents a vSphere
+ volume attached and mounted on kubelets host
+ machine
+ properties:
+ fsType:
+ description: fsType is filesystem type to
+ mount. Must be a filesystem type supported
+ by the host operating system. Ex. "ext4",
+ "xfs", "ntfs". Implicitly inferred to be
+ "ext4" if unspecified.
+ type: string
+ storagePolicyID:
+ description: storagePolicyID is the storage
+ Policy Based Management (SPBM) profile ID
+ associated with the StoragePolicyName.
+ type: string
+ storagePolicyName:
+ description: storagePolicyName is the storage
+ Policy Based Management (SPBM) profile name.
+ type: string
+ volumePath:
+ description: volumePath is the path that identifies
+ vSphere volume vmdk
+ type: string
+ required:
+ - volumePath
+ type: object
+ required:
+ - name
+ type: object
+ type: array
+ required:
+ - containers
+ type: object
+ type: object
+ required:
+ - count
+ - name
+ - template
+ type: object
+ maxItems: 8
+ minItems: 1
+ type: array
+ x-kubernetes-list-map-keys:
+ - name
+ x-kubernetes-list-type: map
+ priority:
+ description: Priority determines the order of access to the resources
+ managed by the ClusterQueue where the workload is queued. The priority
+ value is populated from PriorityClassName. The higher the value,
+ the higher the priority. If priorityClassName is specified, priority
+ must not be null.
+ format: int32
+ type: integer
+ priorityClassName:
+ description: If specified, indicates the workload's priority. "system-node-critical"
+ and "system-cluster-critical" are two special keywords which indicate
+ the highest priorities with the former being the highest priority.
+ Any other name must be defined by creating a PriorityClass object
+ with that name. If not specified, the workload priority will be
+ default or zero if there is no default.
+ type: string
+ queueName:
+ description: queueName is the name of the LocalQueue the Workload
+ is associated with. queueName cannot be changed while .status.admission
+ is not null.
+ type: string
+ required:
+ - podSets
+ type: object
+ status:
+ description: WorkloadStatus defines the observed state of Workload
+ properties:
+ admission:
+ description: admission holds the parameters of the admission of the
+ workload by a ClusterQueue. admission can be set back to null, but
+ its fields cannot be changed once set.
+ properties:
+ clusterQueue:
+ description: clusterQueue is the name of the ClusterQueue that
+ admitted this workload.
+ type: string
+ podSetAssignments:
+ description: PodSetAssignments hold the admission results for
+ each of the .spec.podSets entries.
+ items:
+ properties:
+ flavors:
+ additionalProperties:
+ description: ResourceFlavorReference is the name of the
+ ResourceFlavor.
+ type: string
+ description: Flavors are the flavors assigned to the workload
+ for each resource.
+ type: object
+ name:
+ default: main
+ description: Name is the name of the podSet. It should match
+ one of the names in .spec.podSets.
+ type: string
+ resourceUsage:
+ additionalProperties:
+ anyOf:
+ - type: integer
+ - type: string
+ pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
+ x-kubernetes-int-or-string: true
+ description: "resourceUsage keeps track of the total resources
+ all the pods in the podset need to run. \n Beside what
+ is provided in podSet's specs, this calculation takes
+ into account the LimitRange defaults and RuntimeClass
+ overheads at the moment of admission."
+ type: object
+ required:
+ - name
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - name
+ x-kubernetes-list-type: map
+ required:
+ - clusterQueue
+ - podSetAssignments
+ type: object
+ conditions:
+ description: "conditions hold the latest available observations of
+ the Workload current state. \n The type of the condition could be:
+ \n - Admitted: the Workload was admitted through a ClusterQueue.
+ - Finished: the associated workload finished running (failed or
+ succeeded). - PodsReady: at least `.spec.podSets[*].count` Pods
+ are ready or have succeeded."
+ items:
+ description: "Condition contains details for one aspect of the current
+ state of this API Resource. --- This struct is intended for direct
+ use as an array at the field path .status.conditions. For example,
+ \n type FooStatus struct{ // Represents the observations of a
+ foo's current state. // Known .status.conditions.type are: \"Available\",
+ \"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge
+ // +listType=map // +listMapKey=type Conditions []metav1.Condition
+ `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\"
+ protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }"
+ properties:
+ lastTransitionTime:
+ description: lastTransitionTime is the last time the condition
+ transitioned from one status to another. This should be when
+ the underlying condition changed. If that is not known, then
+ using the time when the API field changed is acceptable.
+ format: date-time
+ type: string
+ message:
+ description: message is a human readable message indicating
+ details about the transition. This may be an empty string.
+ maxLength: 32768
+ type: string
+ observedGeneration:
+ description: observedGeneration represents the .metadata.generation
+ that the condition was set based upon. For instance, if .metadata.generation
+ is currently 12, but the .status.conditions[x].observedGeneration
+ is 9, the condition is out of date with respect to the current
+ state of the instance.
+ format: int64
+ minimum: 0
+ type: integer
+ reason:
+ description: reason contains a programmatic identifier indicating
+ the reason for the condition's last transition. Producers
+ of specific condition types may define expected values and
+ meanings for this field, and whether the values are considered
+ a guaranteed API. The value should be a CamelCase string.
+ This field may not be empty.
+ maxLength: 1024
+ minLength: 1
+ pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
+ type: string
+ status:
+ description: status of the condition, one of True, False, Unknown.
+ enum:
+ - "True"
+ - "False"
+ - Unknown
+ type: string
+ type:
+ description: type of condition in CamelCase or in foo.example.com/CamelCase.
+ --- Many .condition.type values are consistent across resources
+ like Available, but because arbitrary conditions can be useful
+ (see .node.status.conditions), the ability to deconflict is
+ important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
+ maxLength: 316
+ pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
+ type: string
+ required:
+ - lastTransitionTime
+ - message
+ - reason
+ - status
+ - type
+ type: object
+ type: array
+ x-kubernetes-list-map-keys:
+ - type
+ x-kubernetes-list-type: map
+ type: object
+ type: object
+ served: true
+ storage: true
+ subresources:
+ status: {}
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: kueue-controller-manager
+ namespace: kueue-system
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+ name: kueue-leader-election-role
+ namespace: kueue-system
+rules:
+- apiGroups:
+ - ""
+ resources:
+ - configmaps
+ verbs:
+ - get
+ - list
+ - watch
+ - create
+ - update
+ - patch
+ - delete
+- apiGroups:
+ - coordination.k8s.io
+ resources:
+ - leases
+ verbs:
+ - get
+ - list
+ - watch
+ - create
+ - update
+ - patch
+ - delete
+- apiGroups:
+ - ""
+ resources:
+ - events
+ verbs:
+ - create
+ - patch
+---
+aggregationRule:
+ clusterRoleSelectors:
+ - matchLabels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: kueue-batch-admin-role
+---
+aggregationRule:
+ clusterRoleSelectors:
+ - matchLabels:
+ rbac.kueue.x-k8s.io/batch-user: "true"
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: kueue-batch-user-role
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ name: kueue-clusterqueue-editor-role
+rules:
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - clusterqueues
+ verbs:
+ - create
+ - delete
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - clusterqueues/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ name: kueue-clusterqueue-viewer-role
+rules:
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - clusterqueues
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - clusterqueues/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ rbac.kueue.x-k8s.io/batch-user: "true"
+ name: kueue-job-editor-role
+rules:
+- apiGroups:
+ - batch
+ resources:
+ - jobs
+ verbs:
+ - create
+ - delete
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - batch
+ resources:
+ - jobs/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ rbac.kueue.x-k8s.io/batch-user: "true"
+ name: kueue-job-viewer-role
+rules:
+- apiGroups:
+ - batch
+ resources:
+ - jobs
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - batch
+ resources:
+ - jobs/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ name: kueue-localqueue-editor-role
+rules:
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - localqueues
+ verbs:
+ - create
+ - delete
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - localqueues/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ rbac.kueue.x-k8s.io/batch-user: "true"
+ name: kueue-localqueue-viewer-role
+rules:
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - localqueues
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - localqueues/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ creationTimestamp: null
+ name: kueue-manager-role
+rules:
+- apiGroups:
+ - ""
+ resources:
+ - events
+ verbs:
+ - create
+ - update
+ - watch
+- apiGroups:
+ - ""
+ resources:
+ - limitranges
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - ""
+ resources:
+ - namespaces
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - ""
+ resources:
+ - secrets
+ verbs:
+ - get
+ - list
+ - update
+ - watch
+- apiGroups:
+ - admissionregistration.k8s.io
+ resources:
+ - mutatingwebhookconfigurations
+ verbs:
+ - get
+ - list
+ - update
+ - watch
+- apiGroups:
+ - admissionregistration.k8s.io
+ resources:
+ - validatingwebhookconfigurations
+ verbs:
+ - get
+ - list
+ - update
+ - watch
+- apiGroups:
+ - batch
+ resources:
+ - jobs
+ verbs:
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - batch
+ resources:
+ - jobs/finalizers
+ verbs:
+ - get
+ - patch
+ - update
+- apiGroups:
+ - batch
+ resources:
+ - jobs/status
+ verbs:
+ - get
+- apiGroups:
+ - kubeflow.org
+ resources:
+ - mpijobs
+ verbs:
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - kubeflow.org
+ resources:
+ - mpijobs/status
+ verbs:
+ - get
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - clusterqueues
+ verbs:
+ - create
+ - delete
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - clusterqueues/finalizers
+ verbs:
+ - update
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - clusterqueues/status
+ verbs:
+ - get
+ - patch
+ - update
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - localqueues
+ verbs:
+ - create
+ - delete
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - localqueues/finalizers
+ verbs:
+ - update
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - localqueues/status
+ verbs:
+ - get
+ - patch
+ - update
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - resourceflavors
+ verbs:
+ - delete
+ - get
+ - list
+ - update
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - resourceflavors/finalizers
+ verbs:
+ - update
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - workloads
+ verbs:
+ - create
+ - delete
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - workloads/finalizers
+ verbs:
+ - update
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - workloads/status
+ verbs:
+ - get
+ - patch
+ - update
+- apiGroups:
+ - node.k8s.io
+ resources:
+ - runtimeclasses
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - scheduling.k8s.io
+ resources:
+ - priorityclasses
+ verbs:
+ - get
+ - list
+ - watch
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: kueue-metrics-reader
+rules:
+- nonResourceURLs:
+ - /metrics
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ rbac.kueue.x-k8s.io/batch-user: "true"
+ name: kueue-mpijob-editor-role
+rules:
+- apiGroups:
+ - kubeflow.org
+ resources:
+ - mpijobs
+ verbs:
+ - create
+ - delete
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - kubeflow.org
+ resources:
+ - mpijobs/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ rbac.kueue.x-k8s.io/batch-user: "true"
+ name: kueue-mpijob-viewer-role
+rules:
+- apiGroups:
+ - kubeflow.org
+ resources:
+ - mpijobs
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - kubeflow.org
+ resources:
+ - mpijobs/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ name: kueue-proxy-role
+rules:
+- apiGroups:
+ - authentication.k8s.io
+ resources:
+ - tokenreviews
+ verbs:
+ - create
+- apiGroups:
+ - authorization.k8s.io
+ resources:
+ - subjectaccessreviews
+ verbs:
+ - create
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ name: kueue-resourceflavor-editor-role
+rules:
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - resourceflavors
+ verbs:
+ - create
+ - delete
+ - get
+ - list
+ - patch
+ - update
+ - watch
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ name: kueue-resourceflavor-viewer-role
+rules:
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - resourceflavors
+ verbs:
+ - get
+ - list
+ - watch
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ name: kueue-workload-editor-role
+rules:
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - workloads
+ verbs:
+ - create
+ - delete
+ - get
+ - list
+ - patch
+ - update
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - workloads/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+ labels:
+ rbac.kueue.x-k8s.io/batch-admin: "true"
+ rbac.kueue.x-k8s.io/batch-user: "true"
+ name: kueue-workload-viewer-role
+rules:
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - workloads
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - kueue.x-k8s.io
+ resources:
+ - workloads/status
+ verbs:
+ - get
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+ name: kueue-leader-election-rolebinding
+ namespace: kueue-system
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: Role
+ name: kueue-leader-election-role
+subjects:
+- kind: ServiceAccount
+ name: kueue-controller-manager
+ namespace: kueue-system
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: kueue-manager-rolebinding
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: kueue-manager-role
+subjects:
+- kind: ServiceAccount
+ name: kueue-controller-manager
+ namespace: kueue-system
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+ name: kueue-proxy-rolebinding
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: kueue-proxy-role
+subjects:
+- kind: ServiceAccount
+ name: kueue-controller-manager
+ namespace: kueue-system
+---
+apiVersion: v1
+data:
+ controller_manager_config.yaml: |
+ apiVersion: config.kueue.x-k8s.io/v1beta1
+ kind: Configuration
+ health:
+ healthProbeBindAddress: :8081
+ metrics:
+ bindAddress: :8080
+ webhook:
+ port: 9443
+ leaderElection:
+ leaderElect: true
+ resourceName: c1f6bfd2.kueue.x-k8s.io
+ controller:
+ groupKindConcurrency:
+ Job.batch: 5
+ LocalQueue.kueue.x-k8s.io: 1
+ ClusterQueue.kueue.x-k8s.io: 1
+ ResourceFlavor.kueue.x-k8s.io: 1
+ Workload.kueue.x-k8s.io: 1
+ clientConnection:
+ qps: 50
+ burst: 100
+ #waitForPodsReady:
+ # enable: true
+ #manageJobsWithoutQueueName: true
+ #namespace: ""
+ #internalCertManagement:
+ # enable: false
+ # webhookServiceName: ""
+ # webhookSecretName: ""
+ integrations:
+ frameworks:
+ - "batch/job"
+ # - "kubeflow.org/mpijob"
+kind: ConfigMap
+metadata:
+ name: kueue-manager-config
+ namespace: kueue-system
+---
+apiVersion: v1
+kind: Secret
+metadata:
+ name: kueue-webhook-server-cert
+ namespace: kueue-system
+---
+apiVersion: v1
+kind: Service
+metadata:
+ labels:
+ control-plane: controller-manager
+ name: kueue-controller-manager-metrics-service
+ namespace: kueue-system
+spec:
+ ports:
+ - name: https
+ port: 8443
+ protocol: TCP
+ targetPort: https
+ selector:
+ control-plane: controller-manager
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: kueue-webhook-service
+ namespace: kueue-system
+spec:
+ ports:
+ - port: 443
+ protocol: TCP
+ targetPort: 9443
+ selector:
+ control-plane: controller-manager
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ labels:
+ control-plane: controller-manager
+ name: kueue-controller-manager
+ namespace: kueue-system
+spec:
+ replicas: 1
+ selector:
+ matchLabels:
+ control-plane: controller-manager
+ template:
+ metadata:
+ annotations:
+ kubectl.kubernetes.io/default-container: manager
+ labels:
+ control-plane: controller-manager
+ spec:
+ containers:
+ - args:
+ - --config=controller_manager_config.yaml
+ - --zap-log-level=2
+ command:
+ - /manager
+ image: registry.k8s.io/kueue/kueue:v0.3.0
+ imagePullPolicy: Always
+ livenessProbe:
+ httpGet:
+ path: /healthz
+ port: 8081
+ initialDelaySeconds: 15
+ periodSeconds: 20
+ name: manager
+ ports:
+ - containerPort: 9443
+ name: webhook-server
+ protocol: TCP
+ readinessProbe:
+ httpGet:
+ path: /readyz
+ port: 8081
+ initialDelaySeconds: 5
+ periodSeconds: 10
+ resources:
+ limits:
+ cpu: 500m
+ memory: 512Mi
+ requests:
+ cpu: 500m
+ memory: 512Mi
+ securityContext:
+ allowPrivilegeEscalation: false
+ volumeMounts:
+ - mountPath: /tmp/k8s-webhook-server/serving-certs
+ name: cert
+ readOnly: true
+ - mountPath: /controller_manager_config.yaml
+ name: manager-config
+ subPath: controller_manager_config.yaml
+ - args:
+ - --secure-listen-address=0.0.0.0:8443
+ - --upstream=http://127.0.0.1:8080/
+ - --logtostderr=true
+ - --v=10
+ image: gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0
+ name: kube-rbac-proxy
+ ports:
+ - containerPort: 8443
+ name: https
+ protocol: TCP
+ securityContext:
+ runAsNonRoot: true
+ serviceAccountName: kueue-controller-manager
+ terminationGracePeriodSeconds: 10
+ volumes:
+ - name: cert
+ secret:
+ defaultMode: 420
+ secretName: kueue-webhook-server-cert
+ - configMap:
+ name: kueue-manager-config
+ name: manager-config
+---
+apiVersion: admissionregistration.k8s.io/v1
+kind: MutatingWebhookConfiguration
+metadata:
+ creationTimestamp: null
+ name: kueue-mutating-webhook-configuration
+webhooks:
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /mutate-batch-v1-job
+ failurePolicy: Fail
+ name: mjob.kb.io
+ rules:
+ - apiGroups:
+ - batch
+ apiVersions:
+ - v1
+ operations:
+ - CREATE
+ resources:
+ - jobs
+ sideEffects: None
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /mutate-kueue-x-k8s-io-v1beta1-clusterqueue
+ failurePolicy: Fail
+ name: mclusterqueue.kb.io
+ rules:
+ - apiGroups:
+ - kueue.x-k8s.io
+ apiVersions:
+ - v1beta1
+ operations:
+ - CREATE
+ resources:
+ - clusterqueues
+ sideEffects: None
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /mutate-kueue-x-k8s-io-v1beta1-resourceflavor
+ failurePolicy: Fail
+ name: mresourceflavor.kb.io
+ rules:
+ - apiGroups:
+ - kueue.x-k8s.io
+ apiVersions:
+ - v1beta1
+ operations:
+ - CREATE
+ resources:
+ - resourceflavors
+ sideEffects: None
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /mutate-kueue-x-k8s-io-v1beta1-workload
+ failurePolicy: Fail
+ name: mworkload.kb.io
+ rules:
+ - apiGroups:
+ - kueue.x-k8s.io
+ apiVersions:
+ - v1beta1
+ operations:
+ - CREATE
+ - UPDATE
+ resources:
+ - workloads
+ sideEffects: None
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /mutate-kubeflow-org-v2beta1-mpijob
+ failurePolicy: Fail
+ name: mmpijob.kb.io
+ rules:
+ - apiGroups:
+ - kubeflow.org
+ apiVersions:
+ - v2beta1
+ operations:
+ - CREATE
+ resources:
+ - mpijobs
+ sideEffects: None
+---
+apiVersion: admissionregistration.k8s.io/v1
+kind: ValidatingWebhookConfiguration
+metadata:
+ creationTimestamp: null
+ name: kueue-validating-webhook-configuration
+webhooks:
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /validate-batch-v1-job
+ failurePolicy: Fail
+ name: vjob.kb.io
+ rules:
+ - apiGroups:
+ - batch
+ apiVersions:
+ - v1
+ operations:
+ - UPDATE
+ resources:
+ - jobs
+ sideEffects: None
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /validate-kueue-x-k8s-io-v1beta1-clusterqueue
+ failurePolicy: Fail
+ name: vclusterqueue.kb.io
+ rules:
+ - apiGroups:
+ - kueue.x-k8s.io
+ apiVersions:
+ - v1beta1
+ operations:
+ - CREATE
+ - UPDATE
+ resources:
+ - clusterqueues
+ sideEffects: None
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /validate-kueue-x-k8s-io-v1beta1-localqueue
+ failurePolicy: Fail
+ name: vlocalqueue.kb.io
+ rules:
+ - apiGroups:
+ - kueue.x-k8s.io
+ apiVersions:
+ - v1beta1
+ operations:
+ - CREATE
+ - UPDATE
+ resources:
+ - localqueues
+ sideEffects: None
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /validate-kueue-x-k8s-io-v1beta1-resourceflavor
+ failurePolicy: Fail
+ name: vresourceflavor.kb.io
+ rules:
+ - apiGroups:
+ - kueue.x-k8s.io
+ apiVersions:
+ - v1beta1
+ operations:
+ - CREATE
+ - UPDATE
+ resources:
+ - resourceflavors
+ sideEffects: None
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /validate-kueue-x-k8s-io-v1beta1-workload
+ failurePolicy: Fail
+ name: vworkload.kb.io
+ rules:
+ - apiGroups:
+ - kueue.x-k8s.io
+ apiVersions:
+ - v1beta1
+ operations:
+ - CREATE
+ - UPDATE
+ resources:
+ - workloads
+ - workloads/status
+ sideEffects: None
+- admissionReviewVersions:
+ - v1
+ clientConfig:
+ service:
+ name: kueue-webhook-service
+ namespace: kueue-system
+ path: /validate-kubeflow-org-v2beta1-mpijob
+ failurePolicy: Fail
+ name: vmpijob.kb.io
+ rules:
+ - apiGroups:
+ - kubeflow.org
+ apiVersions:
+ - v2beta1
+ operations:
+ - UPDATE
+ resources:
+ - mpijobs
+ sideEffects: None
diff --git a/blueprints/gke/patterns/batch/manifest-templates/team-resources/job.yaml b/blueprints/gke/patterns/batch/manifest-templates/team-resources/job.yaml
new file mode 100644
index 00000000..b7ad11be
--- /dev/null
+++ b/blueprints/gke/patterns/batch/manifest-templates/team-resources/job.yaml
@@ -0,0 +1,42 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: batch/v1
+kind: Job
+metadata:
+ namespace: ${namespace} # Job under team-a namespace
+ generateName: sample-job-
+ annotations:
+ kueue.x-k8s.io/queue-name: local-queue # Point to the LocalQueue
+spec:
+ ttlSecondsAfterFinished: 60 # Job will be deleted after 60 seconds
+ parallelism: 3 # This Job will have 3 replicas running at the same time
+ completions: 3 # This Job requires 3 completions
+ suspend: true # Set to true to allow Kueue to control the Job when it starts
+ template:
+ spec:
+ containers:
+ - name: dummy-job
+ image: gcr.io/k8s-staging-perf-tests/sleep:latest
+ args: ["10s"] # Sleep for 10 seconds
+ resources:
+ requests:
+ cpu: "500m"
+ memory: "512Mi"
+ ephemeral-storage: "512Mi"
+ limits:
+ cpu: "500m"
+ memory: "512Mi"
+ ephemeral-storage: "512Mi"
+ restartPolicy: Never
diff --git a/blueprints/gke/patterns/batch/manifest-templates/team-resources/local-queue.yaml b/blueprints/gke/patterns/batch/manifest-templates/team-resources/local-queue.yaml
new file mode 100644
index 00000000..1cbd9d0b
--- /dev/null
+++ b/blueprints/gke/patterns/batch/manifest-templates/team-resources/local-queue.yaml
@@ -0,0 +1,21 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: kueue.x-k8s.io/v1beta1
+kind: LocalQueue
+metadata:
+ namespace: team # LocalQueue under team-a namespace
+ name: local-queue
+spec:
+ clusterQueue: cluster-queue # Point to the ClusterQueue
diff --git a/blueprints/gke/patterns/batch/providers.tf b/blueprints/gke/patterns/batch/providers.tf
new file mode 100644
index 00000000..1c1d8a00
--- /dev/null
+++ b/blueprints/gke/patterns/batch/providers.tf
@@ -0,0 +1,36 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+data "google_client_config" "identity" {
+ count = var.credentials_config.fleet_host != null ? 1 : 0
+}
+
+provider "kubectl" {
+ config_path = (
+ var.credentials_config.kubeconfig == null
+ ? null
+ : pathexpand(var.credentials_config.kubeconfig.path)
+ )
+ config_context = try(
+ var.credentials_config.kubeconfig.context, null
+ )
+ host = (
+ var.credentials_config.fleet_host == null
+ ? null
+ : var.credentials_config.fleet_host
+ )
+ token = try(data.google_client_config.identity.0.access_token, null)
+}
diff --git a/blueprints/gke/patterns/batch/tutorial.md b/blueprints/gke/patterns/batch/tutorial.md
new file mode 100644
index 00000000..ec57718b
--- /dev/null
+++ b/blueprints/gke/patterns/batch/tutorial.md
@@ -0,0 +1,215 @@
+# Deploy a batch system using Kueue
+
+This tutorial shows you how to deploy a batch system using Kueue to perform Job queueing on Google Kubernetes Engine (GKE) using Terraform.
+
+Jobs are applications that run to completion, such as machine learning, rendering, simulation, analytics, CI/CD, and similar workloads.
+
+Kueue is a Cloud Native Job scheduler that works with the default Kubernetes scheduler, the Job controller, and the cluster autoscaler to provide an end-to-end batch system. Kueue implements Job queueing, deciding when Jobs should wait and when they should start, based on quotas and a hierarchy for sharing resources fairly among teams.
+
+Kueue has the following characteristics:
+
+* It is optimized for cloud architectures, where resources are heterogeneous, interchangeable, and scalable.
+* It provides a set of APIs to manage elastic quotas and manage Job queueing.
+* It does not re-implement existing functionality such as autoscaling, pod scheduling, or Job lifecycle management.
+* Kueue has built-in support for the Kubernetesbatch/v1.Job API.
+* It can integrate with other job APIs.
+* Kueue refers to jobs defined with any API as Workloads, to avoid the confusion with the specific Kubernetes Job API.
+
+When working with Kueue there are a few concepts that ome needs to be familiar with:
+
+* ResourceFlavour
+
+ An object that you can define to describe what resources are available in a cluster. Typically, it is associated with the characteristics of a group of Nodes: availability, pricing, architecture, models, etc.
+
+* ClusterQueue
+
+ A cluster-scoped resource that governs a pool of resources, defining usage limits and fair sharing rules.
+
+* LocalQueue
+
+ A namespaced resource that groups closely related workloads belonging to a single tenant.
+
+* Workload
+
+ An application that will run to completion. It is the unit of admission in Kueue. Sometimes referred to as job
+
+Kueue refers to jobs defined with any API as Workloads, to avoid the confusion with the specific Kubernetes Job API.
+
+## Objectives
+
+This tutorial is for cluster operators and other users that want to implement a batch system on Kubernetes. In this tutorial, you set up a shared cluster for two tenant teams. Each team has their own namespace where they create Jobs and share the same global resources that are controlled with the corresponding quotas.
+
+In this tutorial we will be doing the following using Terraform code available in a git repository:
+
+1. Create a GKE cluster.
+2. Create a namespace for Kueue (kueue-system).
+3. Create a namespace for each team running batch jobs in the cluster (team-a, team-b).
+4. Install Kueue in the namespace created for it.
+5. Create the ResourceFlavor.
+6. Create the ClusterQueue.
+7. Create a LocalQueue for each of the teams in the corresponding namespace.
+8. Create for each of teams a manifest for a sample job associated with the corresponding LocalQueue.
+
+Estimated time:
+
+
+To get started, click Start.
+
+## select/create a project
+
+
+
+## Create the Autopilot GKE cluster
+
+1. Change to the ```autopilot-cluster``` directory.
+
+ ```bash
+ cd autopilot-cluster
+ ```
+
+2. Create a new file ```terraform.tfvars``` in that directory.
+
+ ```bash
+ touch terraform.tfvars
+ ```
+
+3. Open the file for editing.
+
+4. Paste the following content in the file and update any value as needed.
+
+```hcl
+project_id = ""
+cluster_name = "cluster"
+cluster_create = {
+ deletion_protection = false
+}
+region = "europe-west1"
+vpc_create = {
+ enable_cloud_nat = true
+}
+```
+
+5. Initialize the terraform configuration.
+
+ ```bash
+ terraform init
+ ```
+
+6. Apply the terraform configuration.
+
+ ```bash
+ terraform apply
+ ```
+
+7. Fetch the cluster credentials.
+
+ ```bash
+ gcloud container fleet memberships get-credentials cluster --project ""
+ ```
+
+8. Check the nodes are ready.
+
+ ```bash
+ kubectl get pods -n kube-system
+ ```
+
+## Install Kueue and create associated resources
+
+1. Change to the ```patterns/batch``` directory.
+
+ ```bash
+ cd ../batch
+ ```
+
+2. Create a new file ```terraform.tfvars``` in that directory.
+
+ ```bash
+ touch terraform.tfvars
+ ```
+
+3. Open the file for editing.
+
+4. Paste the following content in the file.
+
+ ```hcl
+ credentials_config = {
+ kubeconfig = {
+ path = "~/.kube/config"
+ }
+ }
+ ```
+
+5. Initialize the terraform configuration.
+
+ ```bash
+ terraform init
+ ```
+
+6. Apply the terraform configuration.
+
+ ```bash
+ terraform apply
+ ```
+
+7. Check that the Kueue pods are ready (Use CTRL+C to exit watching)
+
+ ```bash
+ kubectl get pods -n kueue-system -w
+ ```
+
+8. Check the status of the ClusterQueue
+
+ ```bash
+ kubectl get clusterqueue cluster-queue -o wide -w
+ ```
+
+9. Check the status of the LocalQueue for the teams
+
+ ```bash
+ kubectl get localqueue -n team-a local-queue -o wide -w
+ ```
+
+ ```bash
+ kubectl get localqueue -n team-b local-queue -o wide -w
+ ```
+
+## Run jobs in the cluster
+
+1. Create Jobs for namespace team-a and team-b every 10 seconds associated with the corresponding LocalQueue:
+
+ ```bash
+ ./create_jobs.sh job-team-a.yaml job-team-b.yaml 10
+ ```
+
+ Hit Ctrl-C when you want to stop the creation of jobs
+
+2. Observe the workloads being queued up, admitted in the ClusterQueue, and nodes being brought up with GKE Autopilot.
+
+ ```bash
+ kubectl -n team-a get workloads
+ ```
+
+3. Copy a Job name from the previous step and observe the admission status and events for a Job through the W Workloads API:
+
+ ```bash
+ kubectl -n team-a describe workload JOB_NAME
+ ```
+
+## Destroy resources (optional)
+1. Change to the ```patterns/autopilot-cluster``` directory.
+
+ ```bash
+ cd ../autopilot-cluster
+ ```
+
+2. Destroy the cluster with the following command.
+
+ ```bash
+ terraform destroy
+ ```
+
+## Congratulations
+
+
+
+You’re all set!
diff --git a/blueprints/gke/patterns/batch/variables.tf b/blueprints/gke/patterns/batch/variables.tf
new file mode 100644
index 00000000..a31a08f2
--- /dev/null
+++ b/blueprints/gke/patterns/batch/variables.tf
@@ -0,0 +1,58 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+variable "credentials_config" {
+ description = "Configure how Terraform authenticates to the cluster."
+ type = object({
+ fleet_host = optional(string)
+ kubeconfig = optional(object({
+ context = optional(string)
+ path = optional(string, "~/.kube/config")
+ }))
+ })
+ nullable = false
+ validation {
+ condition = (
+ (var.credentials_config.fleet_host != null) !=
+ (var.credentials_config.kubeconfig != null)
+ )
+ error_message = "Exactly one of fleet host or kubeconfig must be set."
+ }
+}
+
+variable "kueue_namespace" {
+ description = "Namespaces of the teams running jobs in the clusters."
+ type = string
+ nullable = false
+ default = "kueue-system"
+}
+
+variable "team_namespaces" {
+ description = "Namespaces of the teams running jobs in the clusters."
+ type = list(string)
+ nullable = false
+ default = [
+ "team-a",
+ "team-b"
+ ]
+}
+
+variable "templates_path" {
+ description = "Path where manifest templates will be read from. Set to null to use the default manifests."
+ type = string
+ default = null
+}
+
diff --git a/blueprints/gke/patterns/batch/versions.tf b/blueprints/gke/patterns/batch/versions.tf
new file mode 100644
index 00000000..3db0e207
--- /dev/null
+++ b/blueprints/gke/patterns/batch/versions.tf
@@ -0,0 +1,27 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+terraform {
+ required_version = ">= 1.7.0"
+ required_providers {
+ google = {
+ source = "hashicorp/google"
+ version = ">= 5.11.0, < 6.0.0" # tftest
+ }
+ google-beta = {
+ source = "hashicorp/google-beta"
+ version = ">= 5.11.0, < 6.0.0" # tftest
+ }
+ }
+}
diff --git a/blueprints/gke/patterns/kafka/README.md b/blueprints/gke/patterns/kafka/README.md
new file mode 100644
index 00000000..1effcd8e
--- /dev/null
+++ b/blueprints/gke/patterns/kafka/README.md
@@ -0,0 +1,66 @@
+# Highly Available Kafka on GKE
+
+
+- [Introduction](#introduction)
+- [Requirements](#requirements)
+- [Cluster authentication](#cluster-authentication)
+- [Kafka Configuration](#kafka-configuration)
+- [Sample Configuration](#sample-configuration)
+- [Variables](#variables)
+
+
+## Introduction
+
+
+
+
+
+This blueprints shows how to a hihgly available Kakfa instance on GKE using the Strimzi operator.
+
+## Requirements
+
+This blueprint assumes the GKE cluster already exists. We recommend using the accompanying [Autopilot Cluster Pattern](../autopilot-cluster) to deploy a cluster according to Google's best practices. Once you have the cluster up-and-running, you can use this blueprint to deploy Kafka in it.
+
+The Kafka manifests will download , which means that the subnet where the GKE cluster is deployed needs to have Internet connectivity to download the images. If you're using the provided [Autopilot Cluster Pattern](../autopilot-cluster), you can set the `enable_cloud_nat` option of the `vpc_create` variable.
+
+## Cluster authentication
+Once you have a cluster with Internet connectivity, create a `terraform.tfvars` and setup the `credentials_config` variable. We recommend using Anthos Fleet to simplify accessing the control plane.
+
+## Kafka Configuration
+
+This template exposes several variables to configure the Kafka instance:
+- `namespace` which controls the namespace used to deploy the Kafka instance
+- `kafka_config` to customize the configuration of the Kafka instance. The default configuration deploys version 3.6.0 with 3 replicas, with a disk of 10Gi and 4096 MB of RAM.
+- `zookeeper_config` to customize the configuration of the Zookeeper instance. The default configuration deploys 3 replicas, with a disk of 10Gi and 2048 MB of RAM.
+
+Any other configuration can be applied by directly modifying the YAML manifests under the [manifest-templates](manifest-templates) directory.
+
+## Sample Configuration
+
+The following template as a starting point for your terraform.tfvars
+```tfvars
+credentials_config = {
+ kubeconfig = {
+ path = "~/.kube/config"
+ }
+}
+kafka_config = {
+ volume_claim_size = "15Gi"
+ replicas = 4
+}
+
+zookeeper_config = {
+ volume_claim_size = "15Gi"
+}
+```
+
+## Variables
+
+| name | description | type | required | default |
+|---|---|:---:|:---:|:---:|
+| [credentials_config](variables.tf#L17) | Configure how Terraform authenticates to the cluster. | object({…}) | ✓ | |
+| [kafka_config](variables.tf#L36) | Configure Kafka cluster statefulset parameters. | object({…}) | | {} |
+| [namespace](variables.tf#L48) | Namespace used for Redis cluster resources. | string | | "kafka" |
+| [templates_path](variables.tf#L55) | Path where manifest templates will be read from. Set to null to use the default manifests. | string | | null |
+| [zookeeper_config](variables.tf#L61) | Configure Zookeper cluster statefulset parameters. | object({…}) | | {} |
+
diff --git a/blueprints/gke/patterns/kafka/main.tf b/blueprints/gke/patterns/kafka/main.tf
new file mode 100644
index 00000000..a9c0979e
--- /dev/null
+++ b/blueprints/gke/patterns/kafka/main.tf
@@ -0,0 +1,49 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+locals {
+ wl_templates = [
+ for f in fileset(local.wl_templates_path, "*yaml") :
+ "${local.wl_templates_path}/${f}"
+ ]
+ wl_templates_path = (
+ var.templates_path == null
+ ? "${path.module}/manifest-templates"
+ : pathexpand(var.templates_path)
+ )
+}
+
+resource "helm_release" "strimzi-operator" {
+ name = "strimzi-operator"
+ repository = "https://strimzi.io/charts"
+ chart = "strimzi-kafka-operator"
+ namespace = var.namespace
+ create_namespace = true
+}
+
+resource "kubectl_manifest" "kafka-cluster" {
+ for_each = toset(local.wl_templates)
+ yaml_body = templatefile(each.value, {
+ name = "kafka"
+ namespace = var.namespace
+ kafka_config = var.kafka_config
+ zookeeper_config = var.zookeeper_config
+ })
+ timeouts {
+ create = "30m"
+ }
+ depends_on = [helm_release.strimzi-operator]
+}
diff --git a/blueprints/gke/patterns/kafka/manifest-templates/start-cluster.yaml b/blueprints/gke/patterns/kafka/manifest-templates/start-cluster.yaml
new file mode 100644
index 00000000..ce2cd600
--- /dev/null
+++ b/blueprints/gke/patterns/kafka/manifest-templates/start-cluster.yaml
@@ -0,0 +1,148 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: kafka.strimzi.io/v1beta2
+kind: Kafka
+metadata:
+ name: "${name}"
+ namespace: "${namespace}"
+spec:
+ kafka:
+ version: "${kafka_config.version}"
+ replicas: ${kafka_config.replicas}
+ template:
+ pod:
+ tolerations:
+ - key: "app.stateful/component"
+ operator: "Equal"
+ value: "kafka-broker"
+ effect: NoSchedule
+ affinity:
+ nodeAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 1
+ preference:
+ matchExpressions:
+ - key: "app.stateful/component"
+ operator: In
+ values:
+ - "kafka-broker"
+ topologySpreadConstraints:
+ - maxSkew: 1
+ topologyKey: "topology.kubernetes.io/zone"
+ whenUnsatisfiable: DoNotSchedule
+ labelSelector:
+ matchLabels:
+ app.kubernetes.io/name: kafka
+ strimzi.io/cluster: "${name}"
+ strimzi.io/component-type: kafka
+ resources:
+ requests:
+ memory: 5Gi
+ cpu: "1"
+ limits:
+ memory: 5Gi
+ cpu: "2"
+ jvmOptions:
+ -Xms: ${kafka_config.jvm_memory}
+ -Xmx: ${kafka_config.jvm_memory}
+ listeners:
+ - name: plain
+ port: 9092
+ type: internal
+ tls: false
+ - name: tls
+ port: 9093
+ type: internal
+ tls: true
+ config:
+ offsets.topic.replication.factor: 3
+ transaction.state.log.replication.factor: 3
+ transaction.state.log.min.isr: 2
+ default.replication.factor: 3
+ min.insync.replicas: 2
+ inter.broker.protocol.version: "3.4"
+ storage:
+ type: jbod
+ volumes:
+ - id: 0
+ type: persistent-claim
+ size: ${kafka_config.volume_claim_size}
+ class: premium-rwo
+ deleteClaim: false
+ zookeeper:
+ template:
+ pod:
+ affinity:
+ nodeAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 1
+ preference:
+ matchExpressions:
+ - key: "app.stateful/component"
+ operator: In
+ values:
+ - "zookeeper"
+ topologySpreadConstraints:
+ - maxSkew: 1
+ topologyKey: "topology.kubernetes.io/zone"
+ whenUnsatisfiable: DoNotSchedule
+ labelSelector:
+ matchLabels:
+ app.kubernetes.io/name: zookeeper
+ strimzi.io/cluster: "${name}"
+ strimzi.io/component-type: zookeeper
+ replicas: ${zookeeper_config.replicas}
+ resources:
+ requests:
+ memory: 2560Mi
+ cpu: 1000m
+ limits:
+ memory: 2560Mi
+ cpu: 2000m
+ jvmOptions:
+ -Xms: ${zookeeper_config.jvm_memory}
+ -Xmx: ${zookeeper_config.jvm_memory}
+ storage:
+ type: persistent-claim
+ size: ${zookeeper_config.volume_claim_size}
+ class: premium-rwo
+ deleteClaim: false
+ entityOperator:
+ tlsSidecar:
+ resources:
+ requests:
+ cpu: 100m
+ memory: 128Mi
+ limits:
+ cpu: 500m
+ memory: 128Mi
+ topicOperator:
+ resources:
+ requests:
+ cpu: 100m
+ memory: 512Mi
+ limits:
+ cpu: 500m
+ memory: 512Mi
+ userOperator:
+ resources:
+ requests:
+ cpu: 500m
+ ephemeral-storage: 1Gi
+ memory: 2Gi
+ limits:
+ cpu: 500m
+ ephemeral-storage: 1Gi
+ memory: 2Gi
diff --git a/blueprints/gke/patterns/kafka/outputs.tf b/blueprints/gke/patterns/kafka/outputs.tf
new file mode 100644
index 00000000..dd112913
--- /dev/null
+++ b/blueprints/gke/patterns/kafka/outputs.tf
@@ -0,0 +1,15 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
diff --git a/blueprints/gke/patterns/kafka/providers.tf b/blueprints/gke/patterns/kafka/providers.tf
new file mode 100644
index 00000000..f85e96d8
--- /dev/null
+++ b/blueprints/gke/patterns/kafka/providers.tf
@@ -0,0 +1,69 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+data "google_client_config" "identity" {
+ count = var.credentials_config.fleet_host != null ? 1 : 0
+}
+
+# provider "kubernetes" {
+# config_path = (
+# var.credentials_config.kubeconfig == null
+# ? null
+# : pathexpand(var.credentials_config.kubeconfig.path)
+# )
+# config_context = try(
+# var.credentials_config.kubeconfig.context, null
+# )
+# host = (
+# var.credentials_config.fleet_host == null
+# ? null
+# : var.credentials_config.fleet_host
+# )
+# token = try(data.google_client_config.identity.0.access_token, null)
+# }
+
+provider "kubectl" {
+ host = (
+ var.credentials_config.fleet_host == null
+ ? null
+ : var.credentials_config.fleet_host
+ )
+ config_path = (
+ var.credentials_config.kubeconfig == null
+ ? null
+ : pathexpand(var.credentials_config.kubeconfig.path)
+ )
+ token = try(data.google_client_config.identity.0.access_token, null)
+}
+
+provider "helm" {
+ kubernetes {
+ config_path = (
+ var.credentials_config.kubeconfig == null
+ ? null
+ : pathexpand(var.credentials_config.kubeconfig.path)
+ )
+ config_context = try(
+ var.credentials_config.kubeconfig.context, null
+ )
+ host = (
+ var.credentials_config.fleet_host == null
+ ? null
+ : var.credentials_config.fleet_host
+ )
+ token = try(data.google_client_config.identity.0.access_token, null)
+ }
+}
diff --git a/blueprints/gke/patterns/kafka/tutorial.md b/blueprints/gke/patterns/kafka/tutorial.md
new file mode 100644
index 00000000..7100d37e
--- /dev/null
+++ b/blueprints/gke/patterns/kafka/tutorial.md
@@ -0,0 +1,156 @@
+# Deploy Apache Kafka to GKE using Strimzi
+
+The guide shows you how to use the Strimzi operator to deploy Apache Kafka clusters ok GKE.
+
+## Objectives
+
+This tutorial covers the following steps:
+
+- Create a GKE cluster.
+- Deploy and configure the Strimzi operator
+- Configure Apache Kafka using the Strimzi operator
+
+Estimated time:
+
+
+To get started, click Start.
+
+## select/create a project
+
+
+
+## Create the Autopilot GKE cluster
+
+1. Change to the ```autopilot-cluster``` directory.
+
+ ```bash
+ cd autopilot-cluster
+ ```
+
+2. Create a new file ```terraform.tfvars``` in that directory.
+
+ ```bash
+ touch terraform.tfvars
+ ```
+
+3. Open the file for editing.
+
+4. Paste the following content in the file and update any value as needed.
+
+```hcl
+project_id = ""
+cluster_name = "cluster"
+cluster_create = {
+ deletion_protection = false
+}
+region = "europe-west1"
+vpc_create = {
+ enable_cloud_nat = true
+}
+```
+
+5. Initialize the terraform configuration.
+
+ ```bash
+ terraform init
+ ```
+
+6. Apply the terraform configuration.
+
+ ```bash
+ terraform apply
+ ```
+
+7. Fetch the cluster credentials.
+
+ ```bash
+ gcloud container fleet memberships get-credentials cluster --project ""
+ ```
+
+8. Check the nodes are ready.
+
+ ```bash
+ kubectl get pods -n kube-system
+ ```
+
+## Install the Kafka Strimzi operator and create associated resources
+
+1. Change to the ```patterns/batch``` directory.
+
+ ```bash
+ cd ../redis-cluster
+ ```
+
+2. Create a new file ```terraform.tfvars``` in that directory.
+
+ ```bash
+ touch terraform.tfvars
+ ```
+
+3. Open the file for editing.
+
+4. Paste the following content in the file.
+
+ ```hcl
+ credentials_config = {
+ kubeconfig = {
+ path = "~/.kube/config"
+ }
+ }
+ kafka_config = {
+ volume_claim_size = "15Gi"
+ replicas = 4
+ }
+ zookeeper_config = {
+ volume_claim_size = "15Gi"
+ }
+ ```
+
+5. Initialize the terraform configuration.
+
+ ```bash
+ terraform init
+ ```
+
+6. Apply the terraform configuration.
+
+ ```bash
+ terraform apply
+ ```
+
+7. Check that the Redis pods are ready
+
+ ```bash
+ kubectl get pods -n kafka
+ ```
+
+8. Check that the Redis volumes match the number of replicas
+
+ ```bash
+ kubectl get pv
+ ```
+
+8. Confirm the Kafka object is running
+
+ ```bash
+ kubectl get kafka -n kafka
+ ```
+
+## Destroy resources (optional)
+1. Change to the ```patterns/autopilot-cluster``` directory.
+
+ ```bash
+ cd ../autopilot-cluster
+ ```
+
+2. Destroy the cluster with the following command.
+
+ ```bash
+ terraform destroy
+ ```
+
+## Congratulations
+
+
+
+You’re all set!
diff --git a/blueprints/gke/patterns/kafka/variables.tf b/blueprints/gke/patterns/kafka/variables.tf
new file mode 100644
index 00000000..287aa768
--- /dev/null
+++ b/blueprints/gke/patterns/kafka/variables.tf
@@ -0,0 +1,70 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+variable "credentials_config" {
+ description = "Configure how Terraform authenticates to the cluster."
+ type = object({
+ fleet_host = optional(string)
+ kubeconfig = optional(object({
+ context = optional(string)
+ path = optional(string, "~/.kube/config")
+ }))
+ })
+ nullable = false
+ validation {
+ condition = (
+ (var.credentials_config.fleet_host != null) !=
+ (var.credentials_config.kubeconfig != null)
+ )
+ error_message = "Exactly one of fleet host or kubeconfig must be set."
+ }
+}
+
+variable "kafka_config" {
+ description = "Configure Kafka cluster statefulset parameters."
+ type = object({
+ replicas = optional(number, 3)
+ volume_claim_size = optional(string, "10Gi")
+ version = optional(string, "3.6.0")
+ jvm_memory = optional(string, "4096m")
+ })
+ nullable = false
+ default = {}
+}
+
+variable "namespace" {
+ description = "Namespace used for Redis cluster resources."
+ type = string
+ nullable = false
+ default = "kafka"
+}
+
+variable "templates_path" {
+ description = "Path where manifest templates will be read from. Set to null to use the default manifests."
+ type = string
+ default = null
+}
+
+variable "zookeeper_config" {
+ description = "Configure Zookeper cluster statefulset parameters."
+ type = object({
+ replicas = optional(number, 3)
+ volume_claim_size = optional(string, "10Gi")
+ jvm_memory = optional(string, "2048m")
+ })
+ nullable = false
+ default = {}
+}
diff --git a/blueprints/gke/patterns/kafka/versions.tf b/blueprints/gke/patterns/kafka/versions.tf
new file mode 100644
index 00000000..3db0e207
--- /dev/null
+++ b/blueprints/gke/patterns/kafka/versions.tf
@@ -0,0 +1,27 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+terraform {
+ required_version = ">= 1.7.0"
+ required_providers {
+ google = {
+ source = "hashicorp/google"
+ version = ">= 5.11.0, < 6.0.0" # tftest
+ }
+ google-beta = {
+ source = "hashicorp/google-beta"
+ version = ">= 5.11.0, < 6.0.0" # tftest
+ }
+ }
+}
diff --git a/blueprints/gke/patterns/kafka/versions_override.tf b/blueprints/gke/patterns/kafka/versions_override.tf
new file mode 100644
index 00000000..a623e3a9
--- /dev/null
+++ b/blueprints/gke/patterns/kafka/versions_override.tf
@@ -0,0 +1,21 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+terraform {
+ required_providers {
+ kubectl = {
+ source = "gavinbunney/kubectl"
+ }
+ }
+}
diff --git a/blueprints/gke/patterns/redis-cluster/README.md b/blueprints/gke/patterns/redis-cluster/README.md
new file mode 100644
index 00000000..885f4336
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/README.md
@@ -0,0 +1,62 @@
+# Highly Available Redis Cluster on GKE
+
+
+- [Introduction](#introduction)
+- [Requirements](#requirements)
+- [Cluster authentication](#cluster-authentication)
+- [Redis Cluster Configuration](#redis-cluster-configuration)
+- [Sample Configuration](#sample-configuration)
+- [Variables](#variables)
+
+
+## Introduction
+
+
+
+
+This blueprint shows how to deploy a highly available Redis cluster on GKE following Google's recommended practices for creating a stateful application.
+
+## Requirements
+
+This blueprint assumes the GKE cluster already exists. We recommend using the accompanying [Autopilot Cluster Pattern](../autopilot-cluster) to deploy a cluster according to Google's best practices. Once you have the cluster up-and-running, you can use this blueprint to deploy Kueue in it.
+
+## Cluster authentication
+Once you have a cluster with, create a `terraform.tfvars` and setup the `credentials_config` variable. We recommend using Anthos Fleet to simplify accessing the control plane.
+
+## Redis Cluster Configuration
+
+This template exposes several variables to configure the Redis cluster:
+- `namespace` which controls the namespace used to deploy the Redis instances
+- `image` to change the container image used by the Redis cluster. Defaults to `redis:6.2` (i.e. the official Redis image, version 6.2)
+- `stateful_config` to customize the configuration of the Redis' stateful set configuration. The default configuration deploys a 6-node cluster with requests for 1 CPU, 1Gi of RAM and a 10Gi volume.
+
+Any other configuration can be applied by directly modifying the YAML manifests under the [manifest-templates](manifest-templates) directory.
+
+## Sample Configuration
+
+The following template as a starting point for your terraform.tfvars
+```tfvars
+credentials_config = {
+ kubeconfig = {
+ path = "~/.kube/config"
+ }
+}
+statefulset_config = {
+ replicas = 8
+ resource_requests = {
+ cpo = "2"
+ memory = "2Gi"
+ }
+}
+```
+
+## Variables
+
+| name | description | type | required | default |
+|---|---|:---:|:---:|:---:|
+| [credentials_config](variables.tf#L17) | Configure how Terraform authenticates to the cluster. | object({…}) | ✓ | |
+| [image](variables.tf#L36) | Container image to use. | string | | "redis:6.2" |
+| [namespace](variables.tf#L43) | Namespace used for Redis cluster resources. | string | | "redis" |
+| [statefulset_config](variables.tf#L50) | Configure Redis cluster statefulset parameters. | object({…}) | | {} |
+| [templates_path](variables.tf#L68) | Path where manifest templates will be read from. Set to null to use the default manifests. | string | | null |
+
diff --git a/blueprints/gke/patterns/redis-cluster/main.tf b/blueprints/gke/patterns/redis-cluster/main.tf
new file mode 100644
index 00000000..f68a60c9
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/main.tf
@@ -0,0 +1,68 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+locals {
+ wl_templates = [
+ for f in fileset(local.wl_templates_path, "[0-9]*yaml") :
+ "${local.wl_templates_path}/${f}"
+ ]
+ wl_templates_path = (
+ var.templates_path == null
+ ? "${path.module}/manifest-templates"
+ : pathexpand(var.templates_path)
+ )
+}
+
+resource "kubernetes_namespace" "default" {
+ metadata {
+ name = var.namespace
+ }
+}
+
+resource "kubernetes_manifest" "default" {
+ for_each = toset(local.wl_templates)
+ manifest = yamldecode(templatefile(each.value, {
+ image = var.image
+ namespace = kubernetes_namespace.default.metadata.0.name
+ statefulset_config = var.statefulset_config
+ }))
+ dynamic "wait" {
+ for_each = strcontains(each.key, "statefulset") ? [""] : []
+ content {
+ fields = {
+ "status.readyReplicas" = var.statefulset_config.replicas
+ }
+ }
+ }
+ timeouts {
+ create = "30m"
+ }
+}
+
+resource "kubernetes_manifest" "cluster-start" {
+ manifest = yamldecode(templatefile("${local.wl_templates_path}/start-cluster.yaml", {
+ image = var.image
+ namespace = kubernetes_namespace.default.metadata.0.name
+ nodes = [
+ for i in range(var.statefulset_config.replicas) :
+ "redis-${i}.redis-cluster.${var.namespace}.svc.cluster.local"
+ ]
+ }))
+ field_manager {
+ force_conflicts = true
+ }
+ depends_on = [kubernetes_manifest.default]
+}
diff --git a/blueprints/gke/patterns/redis-cluster/manifest-templates/01-configmap-redis-cluster.yaml b/blueprints/gke/patterns/redis-cluster/manifest-templates/01-configmap-redis-cluster.yaml
new file mode 100644
index 00000000..1b6ba08f
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/manifest-templates/01-configmap-redis-cluster.yaml
@@ -0,0 +1,28 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: redis-cluster
+ namespace: ${namespace}
+data:
+ redis.conf: |+
+ cluster-enabled yes
+ cluster-node-timeout 15000
+ cluster-config-file /data/nodes.conf
+ appendonly yes
+ protected-mode no
+ dir /data
+ port 6379
diff --git a/blueprints/gke/patterns/redis-cluster/manifest-templates/02-configmap-redis-cluster-probes.yaml b/blueprints/gke/patterns/redis-cluster/manifest-templates/02-configmap-redis-cluster-probes.yaml
new file mode 100644
index 00000000..040ba803
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/manifest-templates/02-configmap-redis-cluster-probes.yaml
@@ -0,0 +1,46 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: redis-probes
+ namespace: ${namespace}
+data:
+ readiness.sh: |-
+ #!/bin/sh
+
+ pingResponse="$(redis-cli -h localhost ping)"
+ if [ "$?" -eq "124" ]; then
+ echo "PING timed out"
+ exit 1
+ fi
+
+ if [ "$pingResponse" != "PONG"]; then
+ echo "$pingResponse"
+ exit 1
+ fi
+ liveness.sh: |-
+ #!/bin/sh
+
+ pingResponse="$(redis-cli -h localhost ping | head -n1 | awk '{print $1;}')"
+ if [ "$?" -eq "124" ]; then
+ echo "PING timed out"
+ exit 1
+ fi
+
+ if [ "$pingResponse" != "PONG"] && [ "$pingResponse" != "LOADING" ] && [ "$pingResponse" != "MASTERDOWN" ]; then
+ echo "$pingResponse"
+ exit 1
+ fi
diff --git a/blueprints/gke/patterns/redis-cluster/manifest-templates/03-pdb-minavailable.yaml b/blueprints/gke/patterns/redis-cluster/manifest-templates/03-pdb-minavailable.yaml
new file mode 100644
index 00000000..812145fb
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/manifest-templates/03-pdb-minavailable.yaml
@@ -0,0 +1,24 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+ name: redis-pdb
+ namespace: ${namespace}
+spec:
+ minAvailable: 3
+ selector:
+ matchLabels:
+ app: redis
diff --git a/blueprints/gke/patterns/redis-cluster/manifest-templates/04-statefulset-redis-cluster.yaml b/blueprints/gke/patterns/redis-cluster/manifest-templates/04-statefulset-redis-cluster.yaml
new file mode 100644
index 00000000..32467788
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/manifest-templates/04-statefulset-redis-cluster.yaml
@@ -0,0 +1,111 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: redis
+ namespace: ${namespace}
+spec:
+ serviceName: "redis-cluster"
+ replicas: ${statefulset_config.replicas}
+ selector:
+ matchLabels:
+ app: redis
+ template:
+ metadata:
+ labels:
+ app: redis
+ appCluster: redis-cluster
+ spec:
+ terminationGracePeriodSeconds: 20
+ affinity:
+ podAntiAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ podAffinityTerm:
+ labelSelector:
+ matchExpressions:
+ - key: app
+ operator: In
+ values:
+ - redis
+ topologyKey: kubernetes.io/hostname
+ containers:
+ - name: redis
+ image: ${image}
+ command:
+ - "redis-server"
+ args:
+ - "/conf/redis.conf"
+ - "--protected-mode"
+ - "no"
+ resources:
+ requests:
+ cpu: ${statefulset_config.resource_requests.cpu}
+ ephemeral-storage: ${statefulset_config.volume_claim_size}
+ memory: ${statefulset_config.resource_requests.memory}
+ ports:
+ - name: redis
+ containerPort: 6379
+ protocol: "TCP"
+ - name: cluster
+ containerPort: 16379
+ protocol: "TCP"
+ startupProbe:
+ periodSeconds: 5
+ timeoutSeconds: 5
+ successThreshold: 1
+ failureThreshold: 20
+ tcpSocket:
+ port: redis
+ livenessProbe:
+ periodSeconds: 5
+ timeoutSeconds: 5
+ successThreshold: 1
+ failureThreshold: 5
+ exec:
+ command: ["sh", "-c", "/probes/liveness.sh"]
+ readinessProbe:
+ periodSeconds: 5
+ timeoutSeconds: 1
+ successThreshold: 1
+ failureThreshold: 5
+ exec:
+ command: ["sh", "-c", "/probes/readiness.sh"]
+ volumeMounts:
+ - name: conf
+ mountPath: /conf
+ - name: data
+ mountPath: /data
+ - name: probes
+ mountPath: /probes
+ readOnly: true
+ volumes:
+ - name: conf
+ configMap:
+ name: redis-cluster
+ defaultMode: 493
+ - name: probes
+ configMap:
+ name: redis-probes
+ defaultMode: 365
+ volumeClaimTemplates:
+ - metadata:
+ name: data
+ spec:
+ accessModes: [ "ReadWriteOnce" ]
+ resources:
+ requests:
+ storage: ${statefulset_config.volume_claim_size}
diff --git a/blueprints/gke/patterns/redis-cluster/manifest-templates/05-service-redis.yaml b/blueprints/gke/patterns/redis-cluster/manifest-templates/05-service-redis.yaml
new file mode 100644
index 00000000..6c831c6f
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/manifest-templates/05-service-redis.yaml
@@ -0,0 +1,19 @@
+# skip boilerplate check
+
+apiVersion: v1
+kind: Service
+metadata:
+ name: redis-cluster
+ namespace: ${namespace}
+spec:
+ clusterIP: None
+ ports:
+ - name: redis-port
+ port: 6379
+ protocol: TCP
+ targetPort: 6379
+ selector:
+ app: redis
+ appCluster: redis-cluster
+ sessionAffinity: None
+ type: ClusterIP
diff --git a/blueprints/gke/patterns/redis-cluster/manifest-templates/roles.sh b/blueprints/gke/patterns/redis-cluster/manifest-templates/roles.sh
new file mode 100755
index 00000000..2f88062e
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/manifest-templates/roles.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+urls=$(kubectl get pods -l app=redis -o jsonpath='{range.items[*]}{.status.podIP} ')
+command="kubectl exec -it redis-0 -- redis-cli --cluster create --cluster-replicas 1 "
+
+for url in $urls
+do
+ command+=$url":6379 "
+done
+
+echo "Executing command: " $command
+$command
diff --git a/blueprints/gke/patterns/redis-cluster/manifest-templates/start-cluster.yaml b/blueprints/gke/patterns/redis-cluster/manifest-templates/start-cluster.yaml
new file mode 100644
index 00000000..f7defade
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/manifest-templates/start-cluster.yaml
@@ -0,0 +1,56 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: redis-cluster-start
+ namespace: ${namespace}
+spec:
+ suspend: false
+ completions: 1
+ template:
+ spec:
+ restartPolicy: Never
+ volumes:
+ - name: shared-data
+ emptyDir: {}
+
+ initContainers:
+ # we resolve node names in an init container using alpine
+ # because the redis image doesn't include nslookup
+ - name: resolve-node-names
+ image: alpine
+ volumeMounts:
+ - name: shared-data
+ mountPath: /tmp/shared-data
+ command:
+ - /bin/sh
+ - -c
+ - |
+ %{~ for n in nodes ~}
+ echo "$(nslookup ${n} | awk '/^Address: / { print $2 }'):6379" >> /tmp/shared-data/nodes
+ %{~ endfor ~}
+
+ containers:
+ - name: redis-cluster-start
+ image: ${image}
+ volumeMounts:
+ - name: shared-data
+ mountPath: /tmp/shared-data
+ command:
+ - /bin/sh
+ - -c
+ - |
+ redis-cli --cluster-yes --cluster-replicas 1 --cluster create $(cat /tmp/shared-data/nodes)
diff --git a/blueprints/gke/patterns/redis-cluster/providers.tf b/blueprints/gke/patterns/redis-cluster/providers.tf
new file mode 100644
index 00000000..70c82e81
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/providers.tf
@@ -0,0 +1,36 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+data "google_client_config" "identity" {
+ count = var.credentials_config.fleet_host != null ? 1 : 0
+}
+
+provider "kubernetes" {
+ config_path = (
+ var.credentials_config.kubeconfig == null
+ ? null
+ : pathexpand(var.credentials_config.kubeconfig.path)
+ )
+ config_context = try(
+ var.credentials_config.kubeconfig.context, null
+ )
+ host = (
+ var.credentials_config.fleet_host == null
+ ? null
+ : var.credentials_config.fleet_host
+ )
+ token = try(data.google_client_config.identity.0.access_token, null)
+}
diff --git a/blueprints/gke/patterns/redis-cluster/tutorial.md b/blueprints/gke/patterns/redis-cluster/tutorial.md
new file mode 100644
index 00000000..3b5da336
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/tutorial.md
@@ -0,0 +1,155 @@
+# Deploy a Redis cluster on GKE
+
+
+## Objectives
+
+This tutorial covers the following steps:
+
+- Create a GKE cluster.
+- Create a Redis Cluster on GKE.
+- Confirm the redis is up and running.
+- Confirm creation of the volumes for the stateful set.
+- Confirm the Pod Disruption Budget (PDB).
+
+Estimated time:
+
+
+To get started, click Start.
+
+## select/create a project
+
+
+
+## Create the Autopilot GKE cluster
+
+1. Change to the ```autopilot-cluster``` directory.
+
+ ```bash
+ cd autopilot-cluster
+ ```
+
+2. Create a new file ```terraform.tfvars``` in that directory.
+
+ ```bash
+ touch terraform.tfvars
+ ```
+
+3. Open the file for editing.
+
+4. Paste the following content in the file and update any value as needed.
+
+```hcl
+project_id = ""
+cluster_name = "cluster"
+cluster_create = {
+ deletion_protection = false
+}
+region = "europe-west1"
+vpc_create = { }
+```
+
+5. Initialize the terraform configuration.
+
+ ```bash
+ terraform init
+ ```
+
+6. Apply the terraform configuration.
+
+ ```bash
+ terraform apply
+ ```
+
+7. Fetch the cluster credentials.
+
+ ```bash
+ gcloud container fleet memberships get-credentials cluster --project ""
+ ```
+
+8. Check the nodes are ready.
+
+ ```bash
+ kubectl get pods -n kube-system
+ ```
+
+## Install Redis and create associated resources
+
+1. Change to the ```patterns/batch``` directory.
+
+ ```bash
+ cd ../redis-cluster
+ ```
+
+2. Create a new file ```terraform.tfvars``` in that directory.
+
+ ```bash
+ touch terraform.tfvars
+ ```
+
+3. Open the file for editing.
+
+4. Paste the following content in the file.
+
+ ```hcl
+ credentials_config = {
+ kubeconfig = {
+ path = "~/.kube/config"
+ }
+ }
+ statefulset_config = {
+ replicas = 8
+ resource_requests = {
+ cpu = "1"
+ memory = "1.5Gi"
+ }
+ }
+ ```
+
+5. Initialize the terraform configuration.
+
+ ```bash
+ terraform init
+ ```
+
+6. Apply the terraform configuration.
+
+ ```bash
+ terraform apply
+ ```
+
+7. Check that the Redis pods are ready
+
+ ```bash
+ kubectl get pods -n redis
+ ```
+
+8. Check that the Redis volumes match the number of replicas
+
+ ```bash
+ kubectl get pv
+ ```
+
+8. Confirm the Pod Disruption Budget for redis guarantees at least 3 pods are up during a voluntary disruption
+
+ ```bash
+ kubectl describe pdb redis-pdb -n redis
+ ```
+
+## Destroy resources (optional)
+1. Change to the ```patterns/autopilot-cluster``` directory.
+
+ ```bash
+ cd ../autopilot-cluster
+ ```
+
+2. Destroy the cluster with the following command.
+
+ ```bash
+ terraform destroy
+ ```
+
+## Congratulations
+
+
+
+You’re all set!
diff --git a/blueprints/gke/patterns/redis-cluster/variables.tf b/blueprints/gke/patterns/redis-cluster/variables.tf
new file mode 100644
index 00000000..b79e8a5c
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/variables.tf
@@ -0,0 +1,72 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+variable "credentials_config" {
+ description = "Configure how Terraform authenticates to the cluster."
+ type = object({
+ fleet_host = optional(string)
+ kubeconfig = optional(object({
+ context = optional(string)
+ path = optional(string, "~/.kube/config")
+ }))
+ })
+ nullable = false
+ validation {
+ condition = (
+ (var.credentials_config.fleet_host != null) !=
+ (var.credentials_config.kubeconfig != null)
+ )
+ error_message = "Exactly one of fleet host or kubeconfig must be set."
+ }
+}
+
+variable "image" {
+ description = "Container image to use."
+ type = string
+ nullable = false
+ default = "redis:6.2"
+}
+
+variable "namespace" {
+ description = "Namespace used for Redis cluster resources."
+ type = string
+ nullable = false
+ default = "redis"
+}
+
+variable "statefulset_config" {
+ description = "Configure Redis cluster statefulset parameters."
+ type = object({
+ replicas = optional(number, 6)
+ resource_requests = optional(object({
+ cpu = optional(string, "1")
+ memory = optional(string, "1Gi")
+ }), {})
+ volume_claim_size = optional(string, "10Gi")
+ })
+ nullable = false
+ default = {}
+ validation {
+ condition = var.statefulset_config.replicas >= 6
+ error_message = "The minimum number of Redis cluster replicas is 6."
+ }
+}
+
+variable "templates_path" {
+ description = "Path where manifest templates will be read from. Set to null to use the default manifests."
+ type = string
+ default = null
+}
diff --git a/blueprints/gke/patterns/redis-cluster/versions.tf b/blueprints/gke/patterns/redis-cluster/versions.tf
new file mode 100644
index 00000000..3db0e207
--- /dev/null
+++ b/blueprints/gke/patterns/redis-cluster/versions.tf
@@ -0,0 +1,27 @@
+# Copyright 2024 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+terraform {
+ required_version = ">= 1.7.0"
+ required_providers {
+ google = {
+ source = "hashicorp/google"
+ version = ">= 5.11.0, < 6.0.0" # tftest
+ }
+ google-beta = {
+ source = "hashicorp/google-beta"
+ version = ">= 5.11.0, < 6.0.0" # tftest
+ }
+ }
+}