3.1 KiB
Deploy Apache Kafka to GKE using Strimzi
The guide shows you how to use the Strimzi operator to deploy Apache Kafka clusters ok GKE.
Objectives
This tutorial covers the following steps:
- Create a GKE cluster.
- Deploy and configure the Strimzi operator
- Configure Apache Kafka using the Strimzi operator
Estimated time:
To get started, click Start.
select/create a project
Create the Autopilot GKE cluster
-
Change to the
autopilot-cluster
directory.cd autopilot-cluster
-
Create a new file
terraform.tfvars
in that directory.touch terraform.tfvars
-
Open the file for editing.
-
Paste the following content in the file and update any value as needed.
project_id = "<walkthrough-project-name/>"
cluster_name = "cluster"
cluster_create = {
deletion_protection = false
}
region = "europe-west1"
vpc_create = {
enable_cloud_nat = true
}
-
Initialize the terraform configuration.
terraform init
-
Apply the terraform configuration.
terraform apply
-
Fetch the cluster credentials.
gcloud container fleet memberships get-credentials cluster --project "<walkthrough-project-name/>"
-
Check the nodes are ready.
kubectl get pods -n kube-system
Install the Kafka Strimzi operator and create associated resources
-
Change to the
patterns/batch
directory.cd ../redis-cluster
-
Create a new file
terraform.tfvars
in that directory.touch terraform.tfvars
-
Open the file for editing.
-
Paste the following content in the file.
credentials_config = { kubeconfig = { path = "~/.kube/config" } } kafka_config = { volume_claim_size = "15Gi" replicas = 4 } zookeeper_config = { volume_claim_size = "15Gi" }
-
Initialize the terraform configuration.
terraform init
-
Apply the terraform configuration.
terraform apply
-
Check that the Redis pods are ready
kubectl get pods -n kafka
-
Check that the Redis volumes match the number of replicas
kubectl get pv
-
Confirm the Kafka object is running
kubectl get kafka -n kafka
Destroy resources (optional)
-
Change to the
patterns/autopilot-cluster
directory.cd ../autopilot-cluster
-
Destroy the cluster with the following command.
terraform destroy
Congratulations
You’re all set!