zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add GKE Hub module to fabric (#540) 

* GKE Hub initial PR commit

* variable management adjust

* comments, fixes and alphabetically ordered variables

* Update README.md

* Update README.md

* Update README.md

* fix test

* resources vs modules

still needs some love

* remove modules usage

* comments, readme update and output

* adjusting outputs and README

* fix README.md

* fix README

* adjusted based on comments

still need some love in the google_gke_hub_feature_membership variables management

* types and variable management

* optionally enable required api

* Update README.md

* reorder locals and use standard formatting

* Don't enable services from modules

* Use self links for member clusters

* Update readme

* members_clusters back to map

@juliocc let's talk about this cause we saw it together in our call and if I change it to a list than the other resources are not going to work, they need location there too.

* Forcing null feature to false due to a bug in provider

If a block is set to null the provider will crash with a "panic: interface conversion: interface {} is nil, not map[string]interface {}" a PR will follow

* Readme update

* Readme.md update

* Update README.md

* bring back tolist, WIP

* Update main.tf

* Readme.md update

* Update README.md

* Update main.tf

* Update main.tf

* Add id and self_links output to gke-cluster

* Use try and make all member feature blocks dynamic/optional

* Change member clusters to map

* Add gke-hub tests

* Address PR comments

* Update gke-hub readme

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
This commit is contained in:
Daniel Marzini 2022-02-28 12:40:48 +01:00 committed by GitHub
parent fb90500adc
commit e372b50d19
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 572 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
modules/gke-cluster/README.md
View File

@ -114,9 +114,11 @@ module "cluster-1" {
| [ca_certificate](outputs.tf#L17) | Public certificate of the cluster (base64-encoded). | ✓ |
| [cluster](outputs.tf#L23) | Cluster resource. | ✓ |
| [endpoint](outputs.tf#L29) | Cluster endpoint. | |
| [location](outputs.tf#L34) | Cluster location. | |
| [master_version](outputs.tf#L39) | Master version. | |
| [name](outputs.tf#L44) | Cluster name. | |
| [notifications](outputs.tf#L49) | GKE PubSub notifications topic. | |
| [id](outputs.tf#L34) | Cluster ID. | ✓ |
| [location](outputs.tf#L40) | Cluster location. | |
| [master_version](outputs.tf#L45) | Master version. | |
| [name](outputs.tf#L50) | Cluster name. | |
| [notifications](outputs.tf#L55) | GKE PubSub notifications topic. | |
| [self_link](outputs.tf#L60) | Cluster self link. | ✓ |
<!-- END TFDOC -->

12
modules/gke-cluster/outputs.tf
View File

@ -31,6 +31,12 @@ output "endpoint" {
value = google_container_cluster.cluster.endpoint
}
output "id" {
description = "Cluster ID."
sensitive = true
value = google_container_cluster.cluster.id
}
output "location" {
description = "Cluster location."
value = google_container_cluster.cluster.location
@ -50,3 +56,9 @@ output "notifications" {
description = "GKE PubSub notifications topic."
value = var.notification_config ? google_pubsub_topic.notifications[0].id : null
}
output "self_link" {
description = "Cluster self link."
sensitive = true
value = google_container_cluster.cluster.self_link
}

111
modules/gke-hub/README.md Normal file
View File

@ -0,0 +1,111 @@
# GKE hub module
This module allows simplified creation and management of a GKE Hub object and its features for a given set of clusters. The given list of clusters will be registered inside the Hub and all the configured features will be activated.
To use this module you must ensure the following APIs are enabled in the target project:
```
"gkehub.googleapis.com"
"gkeconnect.googleapis.com"
"anthosconfigmanagement.googleapis.com"
"multiclusteringress.googleapis.com"
"multiclusterservicediscovery.googleapis.com"
```
## Full GKE Hub example
```hcl
module "project" {
source = "./modules/project"
billing_account = var.billing_account_id
name = "gkehub-test"
parent = "folders/12345"
services = [
"container.googleapis.com",
"gkehub.googleapis.com",
"gkeconnect.googleapis.com",
"anthosconfigmanagement.googleapis.com",
"multiclusteringress.googleapis.com",
"multiclusterservicediscovery.googleapis.com",
]
}
module "vpc" {
source = "./modules/net-vpc"
project_id = module.project.project_id
name = "network"
subnets = [{
ip_cidr_range = "10.0.0.0/24"
name = "cluster-1"
region = "europe-west1"
secondary_ip_range = {
pods = "10.1.0.0/16"
services = "10.2.0.0/24"
}
}]
}
module "cluster-1" {
source = "./modules/gke-cluster"
project_id = module.project.project_id
name = "cluster-1"
location = "europe-west1-b"
network = module.vpc.self_link
subnetwork = module.vpc.subnet_self_links["europe-west1/cluster-1"]
secondary_range_pods = "pods"
secondary_range_services = "services"
enable_dataplane_v2 = true
master_authorized_ranges = { rfc1918_10_8 = "10.0.0.0/8" }
private_cluster_config = {
enable_private_nodes = true
enable_private_endpoint = true
master_ipv4_cidr_block = "192.168.0.0/28"
master_global_access = false
}
}
module "hub" {
source = "./modules/gke-hub"
project_id = module.project.project_id
member_clusters = {
cluster1 = module.cluster-1.id
}
member_features = {
configmanagement = {
binauthz = true
config_sync = {
gcp_service_account_email = null
https_proxy = null
policy_dir = "configsync"
secret_type = "none"
source_format = "hierarchy"
sync_branch = "main"
sync_repo = "https://github.com/danielmarzini/configsync-platform-example"
sync_rev = null
}
hierarchy_controller = null
policy_controller = null
version = "1.10.2"
}
}
}
# tftest modules=4 resources=13
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [project_id](variables.tf#L75) | GKE hub project ID. | <code>string</code> | ✓ | |
| [features](variables.tf#L17) | GKE hub features to enable. | <code title="object&#40;&#123;&#10; configmanagement &#61; bool&#10; mc_ingress &#61; bool&#10; mc_servicediscovery &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; configmanagement &#61; true&#10; mc_ingress &#61; false&#10; mc_servicediscovery &#61; false&#10;&#125;">&#123;&#8230;&#125;</code> |
| [member_clusters](variables.tf#L32) | List for member cluster self links. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> |
| [member_features](variables.tf#L39) | Member features for each cluster | <code title="object&#40;&#123;&#10; configmanagement &#61; object&#40;&#123;&#10; binauthz &#61; bool&#10; config_sync &#61; object&#40;&#123;&#10; gcp_service_account_email &#61; string&#10; https_proxy &#61; string&#10; policy_dir &#61; string&#10; secret_type &#61; string&#10; source_format &#61; string&#10; sync_branch &#61; string&#10; sync_repo &#61; string&#10; sync_rev &#61; string&#10; &#125;&#41;&#10; hierarchy_controller &#61; object&#40;&#123;&#10; enable_hierarchical_resource_quota &#61; bool&#10; enable_pod_tree_labels &#61; bool&#10; &#125;&#41;&#10; policy_controller &#61; object&#40;&#123;&#10; exemptable_namespaces &#61; list&#40;string&#41;&#10; log_denies_enabled &#61; bool&#10; referential_rules_enabled &#61; bool&#10; template_library_installed &#61; bool&#10; &#125;&#41;&#10; version &#61; string&#10; &#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; configmanagement &#61; null&#10;&#125;">&#123;&#8230;&#125;</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| [cluster_ids](outputs.tf#L17) | | |
<!-- END TFDOC -->

140
modules/gke-hub/main.tf Normal file
View File

@ -0,0 +1,140 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
resource "google_gke_hub_membership" "membership" {
provider = google-beta
for_each = var.member_clusters
membership_id = each.key
project = var.project_id
endpoint {
gke_cluster {
resource_link = each.value
}
}
}
resource "google_gke_hub_feature" "configmanagement" {
provider = google-beta
for_each = var.features.configmanagement ? { 1 = 1 } : {}
project = var.project_id
name = "configmanagement"
location = "global"
}
resource "google_gke_hub_feature" "mci" {
provider = google-beta
for_each = var.features.mc_ingress ? var.member_clusters : {}
project = var.project_id
name = "multiclusteringress"
location = "global"
spec {
multiclusteringress {
config_membership = google_gke_hub_membership.membership[each.key].id
}
}
}
resource "google_gke_hub_feature" "mcs" {
provider = google-beta
for_each = var.features.mc_servicediscovery ? { 1 = 1 } : {}
project = var.project_id
name = "multiclusterservicediscovery"
location = "global"
}
resource "google_gke_hub_feature_membership" "feature_member" {
provider = google-beta
for_each = var.member_clusters
project = var.project_id
location = "global"
feature = google_gke_hub_feature.configmanagement["1"].name
membership = google_gke_hub_membership.membership[each.key].membership_id
dynamic "configmanagement" {
for_each = (
try(var.member_features.configmanagement, null) != null
? [var.member_features.configmanagement]
: []
)
iterator = configmanagement
content {
version = try(configmanagement.value.version, null)
dynamic "config_sync" {
for_each = (
try(configmanagement.value.config_sync, null) != null
? [configmanagement.value.config_sync]
: []
)
iterator = config_sync
content {
git {
https_proxy = try(config_sync.value.https_proxy, null)
sync_repo = try(config_sync.value.sync_repo, null)
sync_branch = try(config_sync.value.sync_branch, null)
sync_rev = try(config_sync.value.sync_rev, null)
secret_type = try(config_sync.value.secret_type, null)
gcp_service_account_email = try(config_sync.value.gcp_service_account_email, null)
policy_dir = try(config_sync.value.policy_dir, null)
}
source_format = try(config_sync.value.source_format, null)
}
}
dynamic "policy_controller" {
for_each = (
try(configmanagement.value.policy_controller, null) != null
? [configmanagement.value.policy_controller]
: []
)
iterator = policy_controller
content {
enabled = true
exemptable_namespaces = try(policy_controller.value.exemptable_namespaces, null)
log_denies_enabled = try(policy_controller.value.log_denies_enabled, null)
referential_rules_enabled = try(policy_controller.value.referential_rules_enabled, null)
template_library_installed = try(policy_controller.value.template_library_installed, null)
}
}
dynamic "binauthz" {
for_each = (
try(configmanagement.value.binauthz, false)
? [1]
: []
)
content {
enabled = true
}
}
dynamic "hierarchy_controller" {
for_each = (
try(var.member_features.hierarchy_controller, null) != null
? [var.member_features.hierarchy_controller]
: []
)
iterator = hierarchy_controller
content {
enabled = true
enable_pod_tree_labels = try(hierarchy_controller.value.enable_pod_tree_labels)
enable_hierarchical_resource_quota = try(hierarchy_controller.value.enable_hierarchical_resource_quota)
}
}
}
}
}

26
modules/gke-hub/outputs.tf Normal file
View File

@ -0,0 +1,26 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "cluster_ids" {
value = var.member_clusters
depends_on = [
google_gke_hub_membership.membership,
google_gke_hub_feature.configmanagement,
google_gke_hub_feature.mci,
google_gke_hub_feature.mcs,
google_gke_hub_feature_membership.feature_member,
]
}

78
modules/gke-hub/variables.tf Normal file
View File

@ -0,0 +1,78 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "features" {
description = "GKE hub features to enable."
type = object({
configmanagement = bool
mc_ingress = bool
mc_servicediscovery = bool
})
default = {
configmanagement = true
mc_ingress = false
mc_servicediscovery = false
}
nullable = false
}
variable "member_clusters" {
description = "List for member cluster self links."
type = map(string)
default = {}
nullable = false
}
variable "member_features" {
description = "Member features for each cluster"
type = object({
configmanagement = object({
binauthz = bool
config_sync = object({
gcp_service_account_email = string
https_proxy = string
policy_dir = string
secret_type = string
source_format = string
sync_branch = string
sync_repo = string
sync_rev = string
})
hierarchy_controller = object({
enable_hierarchical_resource_quota = bool
enable_pod_tree_labels = bool
})
policy_controller = object({
exemptable_namespaces = list(string)
log_denies_enabled = bool
referential_rules_enabled = bool
template_library_installed = bool
})
version = string
})
# mc-ingress = bool
# mc-servicediscovery = bool
})
default = {
configmanagement = null
}
nullable = false
}
variable "project_id" {
description = "GKE hub project ID."
type = string
}

27
modules/gke-hub/versions.tf Normal file
View File

@ -0,0 +1,27 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_version = ">= 1.0.0"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 4.0.0"
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 4.0.0"
}
}
}

13
tests/modules/gke_hub/__init__.py Normal file
View File

@ -0,0 +1,13 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

52
tests/modules/gke_hub/fixture/main.tf Normal file
View File

@ -0,0 +1,52 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
module "hub" {
source = "../../../../modules/gke-hub"
project_id = var.project_id
member_clusters = var.member_clusters
features = {
configmanagement = true
mc_ingress = true
mc_servicediscovery = true
}
member_features = {
configmanagement = {
binauthz = true
config_sync = {
gcp_service_account_email = null
https_proxy = null
policy_dir = "configsync"
secret_type = "none"
source_format = "hierarchy"
sync_branch = "main"
sync_repo = "https://github.com/danielmarzini/configsync-platform-example"
sync_rev = null
}
hierarchy_controller = {
enable_hierarchical_resource_quota = true
enable_pod_tree_labels = true
}
policy_controller = {
exemptable_namespaces = []
log_denies_enabled = true
referential_rules_enabled = true
template_library_installed = true
}
version = "1.10.2"
}
}
}

26
tests/modules/gke_hub/fixture/variables.tf Normal file
View File

@ -0,0 +1,26 @@
/**
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "project_id" {
default = "my-project"
}
variable "member_clusters" {
default = {
mycluster1 = "projects/myproject/locations/europe-west1-b/clusters/mycluster1"
mycluster2 = "projects/myproject/locations/europe-west1-b/clusters/mycluster2"
}
}

81
tests/modules/gke_hub/test_plan.py Normal file
View File

@ -0,0 +1,81 @@
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import pytest
@pytest.fixture
def resources(plan_runner):
_, resources = plan_runner()
return resources
def test_resource_count(resources):
"Test number of resources created."
assert len(resources) == 8
assert sorted(r['address'] for r in resources) == [
'module.hub.google_gke_hub_feature.configmanagement["1"]',
'module.hub.google_gke_hub_feature.mci["mycluster1"]',
'module.hub.google_gke_hub_feature.mci["mycluster2"]',
'module.hub.google_gke_hub_feature.mcs["1"]',
'module.hub.google_gke_hub_feature_membership.feature_member["mycluster1"]',
'module.hub.google_gke_hub_feature_membership.feature_member["mycluster2"]',
'module.hub.google_gke_hub_membership.membership["mycluster1"]',
'module.hub.google_gke_hub_membership.membership["mycluster2"]'
]
def test_configmanagement_setup(resources):
"Test configuration of configmanagement."
resources = {r['address']: r['values'] for r in resources}
expected_repo = 'https://github.com/danielmarzini/configsync-platform-example'
expected_configmanagement = [{
'binauthz': [{
'enabled': True
}],
'config_sync': [{
'git': [{
'gcp_service_account_email': None,
'https_proxy': None,
'policy_dir': 'configsync',
'secret_type': 'none',
'sync_branch': 'main',
'sync_repo': expected_repo,
'sync_rev': None,
'sync_wait_secs': None
}],
'source_format': 'hierarchy'
}],
'hierarchy_controller': [],
'policy_controller': [{
'audit_interval_seconds': None,
'enabled': True,
'exemptable_namespaces': [],
'log_denies_enabled': True,
'referential_rules_enabled': True,
'template_library_installed': True
}],
'version': '1.10.2'
}]
for cluster in ['mycluster1', 'mycluster2']:
membership_key = f'module.hub.google_gke_hub_membership.membership["{cluster}"]'
membership = resources[membership_key]
link = membership['endpoint'][0]['gke_cluster'][0]['resource_link']
assert link == f'projects/myproject/locations/europe-west1-b/clusters/{cluster}'
fm_key = f'module.hub.google_gke_hub_feature_membership.feature_member["{cluster}"]'
fm = resources[fm_key]
assert fm['configmanagement'] == expected_configmanagement