Merge pull request #93 from terraform-google-modules/data-e2e-01
Add e2e example: GCE and GCS CMEK via centralized Cloud KMS
This commit is contained in:
lcaggio
GitHub
commit
0cb973b8bc
|
|
@ -3,6 +3,8 @@
|
|||
All notable changes to this project will be documented in this file.
|
||||
|
||||
## [Unreleased]
|
||||
- new `data-solutions` section
|
||||
- new `cmek-via-centralized-kms` e2e example
|
||||
|
||||
## [1.9.0] - 2020-06-10
|
||||
|
||||
|
|
|
|||
|
|
@ -19,8 +19,9 @@ Currently available examples:
|
|||
|
||||
- **foundations** - [single level hierarchy](./foundations/environments/) (environments), [multiple level hierarchy](./foundations/business-units/) (business units + environments)
|
||||
- **infrastructure** - [hub and spoke via peering](./infrastructure/hub-and-spoke-peering/), [hub and spoke via VPN](./infrastructure/hub-and-spoke-vpn/), [DNS and Google Private Access for on-premises](./infrastructure/onprem-google-access-dns/), [Shared VPC with GKE support](./infrastructure/shared-vpc-gke/)
|
||||
- **data solutions** - [GCE/GCS CMEK via centralized Cloud KMS](./data-solutions/cmek-via-centralized-kms/)
|
||||
|
||||
For more information see the README files in the [foundations](./foundations/) and [infrastructure](./infrastructure/) folders.
|
||||
For more information see the README files in the [foundations](./foundations/), [infrastructure](./infrastructure/) and [data solutions](./data-solutions/) folders.
|
||||
|
||||
## Modules
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,11 @@
|
|||
# GCP Data Services examples
|
||||
|
||||
The examples in this folder implement **typical data service topologies** and **end-to-end scenarios**, that allow testing specific features like Cloud KMS to encrypt your data, or VPC-SC to mitigate data exfiltration.
|
||||
|
||||
They are meant to be used as minimal but complete starting points to create actual infrastructure, and as playgrounds to experiment with specific Google Cloud features.
|
||||
|
||||
## Examples
|
||||
|
||||
### GCE and GCS CMEK via centralized Cloud KMS
|
||||
|
||||
<a href="./cmek-via-centralized-kms/" title="CMEK on Cloud Storage and Compute Engine via centralized Cloud KMS"><img src="./cmek-via-centralized-kms/diagram.png" align="left" width="280px"></a> This [example](./cmek-via-centralized-kms/) implements [CMEK](https://cloud.google.com/kms/docs/cmek) for GCS and GCE, via keys hosted in KMS running in a centralized project. The example shows the basic resources and permissions for the typical use case of application projects implementing encryption at rest via a centrally managed KMS service.
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
# GCE and GCS CMEK via centralized Cloud KMS
|
||||
|
||||
This example creates a sample centralized [Cloud KMS](https://cloud.google.com/kms?hl=it) configuration, and uses it to implement CMEK for [Cloud Storage](https://cloud.google.com/storage/docs/encryption/using-customer-managed-keys) and [Compute Engine](https://cloud.google.com/compute/docs/disks/customer-managed-encryption) in a separate project.
|
||||
|
||||
The example is designed to match real-world use cases with a minimum amount of resources, and be used as a starting point for scenarios where application projects implement CMEK using keys managed by a central team. It also includes the IAM wiring needed to make such scenarios work.
|
||||
|
||||
This is the high level diagram:
|
||||
|
||||
|
||||
|
||||
## Managed resources and services
|
||||
|
||||
This sample creates several distinct groups of resources:
|
||||
|
||||
- projects
|
||||
- Cloud KMS project
|
||||
- Service Project configured for GCE instances and GCS buckets
|
||||
- networking
|
||||
- VPC network
|
||||
- One subnet
|
||||
- Firewall rules for [SSH access via IAP](https://cloud.google.com/iap/docs/using-tcp-forwarding) and open communication within the VPC
|
||||
- IAM
|
||||
- One service account for the GGE instance
|
||||
- KMS
|
||||
- One key ring
|
||||
- One crypto key (Procection level: softwere) for Cloud Engine
|
||||
- One crypto key (Protection level: softwere) for Cloud Storage
|
||||
- GCE
|
||||
- One instance encrypted with a CMEK Cryptokey hosted in Cloud KMS
|
||||
- GCS
|
||||
- One bucket encrypted with a CMEK Cryptokey hosted in Cloud KMS
|
||||
|
||||
<!-- BEGIN TFDOC -->
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---: |:---:|:---:|
|
||||
| billing_account | Billing account id used as default for new projects. | <code title="">string</code> | ✓ | |
|
||||
| root_node | The resource name of the parent Folder or Organization. Must be of the form folders/folder_id or organizations/org_id. | <code title="">string</code> | ✓ | |
|
||||
| *location* | The location where resources will be deployed. | <code title="">string</code> | | <code title="">europe</code> |
|
||||
| *project_kms_name* | Name for the new KMS Project. | <code title="">string</code> | | <code title="">my-project-kms-001</code> |
|
||||
| *project_service_name* | Name for the new Service Project. | <code title="">string</code> | | <code title="">my-project-service-001</code> |
|
||||
| *region* | The region where resources will be deployed. | <code title="">string</code> | | <code title="">europe-west1</code> |
|
||||
| *vpc_ip_cidr_range* | Ip range used in the subnet deployef in the Service Project. | <code title="">string</code> | | <code title="">10.0.0.0/20</code> |
|
||||
| *vpc_name* | Name of the VPC created in the Service Project. | <code title="">string</code> | | <code title="">local</code> |
|
||||
| *vpc_subnet_name* | Name of the subnet created in the Service Project. | <code title="">string</code> | | <code title="">subnet</code> |
|
||||
| *zone* | The zone where resources will be deployed. | <code title="">string</code> | | <code title="">europe-west1-b</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| bucket | GCS Bucket Cloud KMS crypto keys. | |
|
||||
| bucket_keys | GCS Bucket Cloud KMS crypto keys. | |
|
||||
| projects | Project ids. | |
|
||||
| vm | GCE VMs. | |
|
||||
| vm_keys | GCE VM Cloud KMS crypto keys. | |
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
|
||||
terraform {
|
||||
backend "gcs" {
|
||||
bucket = ""
|
||||
}
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 145 KiB |
|
|
@ -0,0 +1,155 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
###############################################################################
|
||||
# Projects #
|
||||
###############################################################################
|
||||
|
||||
module "project-service" {
|
||||
source = "../../modules/project"
|
||||
name = var.project_service_name
|
||||
parent = var.root_node
|
||||
billing_account = var.billing_account
|
||||
services = [
|
||||
"compute.googleapis.com",
|
||||
"servicenetworking.googleapis.com",
|
||||
"storage-component.googleapis.com"
|
||||
]
|
||||
oslogin = true
|
||||
}
|
||||
|
||||
module "project-kms" {
|
||||
source = "../../modules/project"
|
||||
name = var.project_kms_name
|
||||
parent = var.root_node
|
||||
billing_account = var.billing_account
|
||||
services = [
|
||||
"cloudkms.googleapis.com",
|
||||
"servicenetworking.googleapis.com"
|
||||
]
|
||||
oslogin = true
|
||||
}
|
||||
|
||||
###############################################################################
|
||||
# Networking #
|
||||
###############################################################################
|
||||
|
||||
module "vpc" {
|
||||
source = "../../modules/net-vpc"
|
||||
project_id = module.project-service.project_id
|
||||
name = var.vpc_name
|
||||
subnets = [
|
||||
{
|
||||
ip_cidr_range = var.vpc_ip_cidr_range
|
||||
name = var.vpc_subnet_name
|
||||
region = var.region
|
||||
secondary_ip_range = {}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
module "vpc-firewall" {
|
||||
source = "../../modules/net-vpc-firewall"
|
||||
project_id = module.project-service.project_id
|
||||
network = module.vpc.name
|
||||
admin_ranges_enabled = true
|
||||
admin_ranges = [var.vpc_ip_cidr_range]
|
||||
}
|
||||
|
||||
###############################################################################
|
||||
# KMS #
|
||||
###############################################################################
|
||||
|
||||
module "kms" {
|
||||
source = "../../modules/kms"
|
||||
project_id = module.project-kms.project_id
|
||||
keyring = {
|
||||
name = "my-keyring",
|
||||
location = var.location
|
||||
}
|
||||
keys = { key-gce = null, key-gcs = null }
|
||||
key_iam_roles = {
|
||||
key-gce = ["roles/cloudkms.cryptoKeyEncrypterDecrypter"]
|
||||
}
|
||||
key_iam_members = {
|
||||
key-gce = {
|
||||
"roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
|
||||
"serviceAccount:${module.project-service.service_accounts.robots.compute}",
|
||||
]
|
||||
},
|
||||
key-gcs = {
|
||||
"roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
|
||||
"serviceAccount:${module.project-service.service_accounts.robots.storage}",
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
###############################################################################
|
||||
# GCE #
|
||||
###############################################################################
|
||||
|
||||
module "kms_vm_example" {
|
||||
source = "../../modules/compute-vm"
|
||||
project_id = module.project-service.project_id
|
||||
region = var.region
|
||||
zone = var.zone
|
||||
name = "kms-vm"
|
||||
network_interfaces = [{
|
||||
network = module.vpc.self_link,
|
||||
subnetwork = module.vpc.subnet_self_links["${var.region}/subnet"],
|
||||
nat = false,
|
||||
addresses = null
|
||||
}]
|
||||
attached_disks = [
|
||||
{
|
||||
name = "attacheddisk"
|
||||
size = 10
|
||||
image = null
|
||||
options = {
|
||||
auto_delete = true
|
||||
mode = null
|
||||
source = null
|
||||
type = null
|
||||
}
|
||||
}
|
||||
]
|
||||
instance_count = 1
|
||||
boot_disk = {
|
||||
image = "projects/debian-cloud/global/images/family/debian-10"
|
||||
type = "pd-ssd"
|
||||
size = 10
|
||||
encrypt_disk = true
|
||||
}
|
||||
tags = ["ssh"]
|
||||
encryption = {
|
||||
encrypt_boot = true
|
||||
disk_encryption_key_raw = null
|
||||
kms_key_self_link = module.kms.key_self_links.key-gce
|
||||
}
|
||||
}
|
||||
|
||||
###############################################################################
|
||||
# GCS #
|
||||
###############################################################################
|
||||
|
||||
module "kms-gcs" {
|
||||
source = "../../modules/gcs"
|
||||
project_id = module.project-service.project_id
|
||||
prefix = "my-bucket-001"
|
||||
names = ["kms-gcs"]
|
||||
encryption_keys = {
|
||||
kms-gcs = module.kms.keys.key-gce.self_link,
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
output "bucket" {
|
||||
description = "GCS Bucket Cloud KMS crypto keys."
|
||||
value = {
|
||||
for bucket in module.kms-gcs.buckets :
|
||||
bucket.name => bucket.url
|
||||
}
|
||||
}
|
||||
|
||||
output "bucket_keys" {
|
||||
description = "GCS Bucket Cloud KMS crypto keys."
|
||||
value = {
|
||||
for bucket in module.kms-gcs.buckets :
|
||||
bucket.name => bucket.encryption
|
||||
}
|
||||
}
|
||||
|
||||
output "projects" {
|
||||
description = "Project ids."
|
||||
value = {
|
||||
service-project = module.project-service.project_id
|
||||
kms-project = module.project-kms.project_id
|
||||
}
|
||||
}
|
||||
|
||||
output "vm" {
|
||||
description = "GCE VMs."
|
||||
value = {
|
||||
for instance in module.kms_vm_example.instances :
|
||||
instance.name => instance.network_interface.0.network_ip
|
||||
}
|
||||
}
|
||||
|
||||
output "vm_keys" {
|
||||
description = "GCE VM Cloud KMS crypto keys."
|
||||
value = {
|
||||
for instance in module.kms_vm_example.instances :
|
||||
instance.name => instance.boot_disk.0.kms_key_self_link
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
|
||||
variable "billing_account" {
|
||||
description = "Billing account id used as default for new projects."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "location" {
|
||||
description = "The location where resources will be deployed."
|
||||
type = string
|
||||
default = "europe"
|
||||
}
|
||||
|
||||
variable "project_service_name" {
|
||||
description = "Name for the new Service Project."
|
||||
type = string
|
||||
default = "my-project-service-001"
|
||||
}
|
||||
|
||||
variable "project_kms_name" {
|
||||
description = "Name for the new KMS Project."
|
||||
type = string
|
||||
default = "my-project-kms-001"
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "The region where resources will be deployed."
|
||||
type = string
|
||||
default = "europe-west1"
|
||||
}
|
||||
|
||||
variable "root_node" {
|
||||
description = "The resource name of the parent Folder or Organization. Must be of the form folders/folder_id or organizations/org_id."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "vpc_name" {
|
||||
description = "Name of the VPC created in the Service Project."
|
||||
type = string
|
||||
default = "local"
|
||||
}
|
||||
|
||||
variable "vpc_subnet_name" {
|
||||
description = "Name of the subnet created in the Service Project."
|
||||
type = string
|
||||
default = "subnet"
|
||||
}
|
||||
|
||||
variable "vpc_ip_cidr_range" {
|
||||
description = "Ip range used in the subnet deployef in the Service Project."
|
||||
type = string
|
||||
default = "10.0.0.0/20"
|
||||
}
|
||||
|
||||
variable "zone" {
|
||||
description = "The zone where resources will be deployed."
|
||||
type = string
|
||||
default = "europe-west1-b"
|
||||
}
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
terraform {
|
||||
required_version = ">= 0.12.6"
|
||||
}
|
||||
Loading…
Reference in New Issue