Merge pull request #93 from terraform-google-modules/data-e2e-01
Add e2e example: GCE and GCS CMEK via centralized Cloud KMS
This commit is contained in:
commit
0cb973b8bc
|
@ -3,6 +3,8 @@
|
||||||
All notable changes to this project will be documented in this file.
|
All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
- new `data-solutions` section
|
||||||
|
- new `cmek-via-centralized-kms` e2e example
|
||||||
|
|
||||||
## [1.9.0] - 2020-06-10
|
## [1.9.0] - 2020-06-10
|
||||||
|
|
||||||
|
|
|
@ -19,8 +19,9 @@ Currently available examples:
|
||||||
|
|
||||||
- **foundations** - [single level hierarchy](./foundations/environments/) (environments), [multiple level hierarchy](./foundations/business-units/) (business units + environments)
|
- **foundations** - [single level hierarchy](./foundations/environments/) (environments), [multiple level hierarchy](./foundations/business-units/) (business units + environments)
|
||||||
- **infrastructure** - [hub and spoke via peering](./infrastructure/hub-and-spoke-peering/), [hub and spoke via VPN](./infrastructure/hub-and-spoke-vpn/), [DNS and Google Private Access for on-premises](./infrastructure/onprem-google-access-dns/), [Shared VPC with GKE support](./infrastructure/shared-vpc-gke/)
|
- **infrastructure** - [hub and spoke via peering](./infrastructure/hub-and-spoke-peering/), [hub and spoke via VPN](./infrastructure/hub-and-spoke-vpn/), [DNS and Google Private Access for on-premises](./infrastructure/onprem-google-access-dns/), [Shared VPC with GKE support](./infrastructure/shared-vpc-gke/)
|
||||||
|
- **data solutions** - [GCE/GCS CMEK via centralized Cloud KMS](./data-solutions/cmek-via-centralized-kms/)
|
||||||
|
|
||||||
For more information see the README files in the [foundations](./foundations/) and [infrastructure](./infrastructure/) folders.
|
For more information see the README files in the [foundations](./foundations/), [infrastructure](./infrastructure/) and [data solutions](./data-solutions/) folders.
|
||||||
|
|
||||||
## Modules
|
## Modules
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,11 @@
|
||||||
|
# GCP Data Services examples
|
||||||
|
|
||||||
|
The examples in this folder implement **typical data service topologies** and **end-to-end scenarios**, that allow testing specific features like Cloud KMS to encrypt your data, or VPC-SC to mitigate data exfiltration.
|
||||||
|
|
||||||
|
They are meant to be used as minimal but complete starting points to create actual infrastructure, and as playgrounds to experiment with specific Google Cloud features.
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
### GCE and GCS CMEK via centralized Cloud KMS
|
||||||
|
|
||||||
|
<a href="./cmek-via-centralized-kms/" title="CMEK on Cloud Storage and Compute Engine via centralized Cloud KMS"><img src="./cmek-via-centralized-kms/diagram.png" align="left" width="280px"></a> This [example](./cmek-via-centralized-kms/) implements [CMEK](https://cloud.google.com/kms/docs/cmek) for GCS and GCE, via keys hosted in KMS running in a centralized project. The example shows the basic resources and permissions for the typical use case of application projects implementing encryption at rest via a centrally managed KMS service.
|
|
@ -0,0 +1,58 @@
|
||||||
|
# GCE and GCS CMEK via centralized Cloud KMS
|
||||||
|
|
||||||
|
This example creates a sample centralized [Cloud KMS](https://cloud.google.com/kms?hl=it) configuration, and uses it to implement CMEK for [Cloud Storage](https://cloud.google.com/storage/docs/encryption/using-customer-managed-keys) and [Compute Engine](https://cloud.google.com/compute/docs/disks/customer-managed-encryption) in a separate project.
|
||||||
|
|
||||||
|
The example is designed to match real-world use cases with a minimum amount of resources, and be used as a starting point for scenarios where application projects implement CMEK using keys managed by a central team. It also includes the IAM wiring needed to make such scenarios work.
|
||||||
|
|
||||||
|
This is the high level diagram:
|
||||||
|
|
||||||
|
![High-level diagram](diagram.png "High-level diagram")
|
||||||
|
|
||||||
|
## Managed resources and services
|
||||||
|
|
||||||
|
This sample creates several distinct groups of resources:
|
||||||
|
|
||||||
|
- projects
|
||||||
|
- Cloud KMS project
|
||||||
|
- Service Project configured for GCE instances and GCS buckets
|
||||||
|
- networking
|
||||||
|
- VPC network
|
||||||
|
- One subnet
|
||||||
|
- Firewall rules for [SSH access via IAP](https://cloud.google.com/iap/docs/using-tcp-forwarding) and open communication within the VPC
|
||||||
|
- IAM
|
||||||
|
- One service account for the GGE instance
|
||||||
|
- KMS
|
||||||
|
- One key ring
|
||||||
|
- One crypto key (Procection level: softwere) for Cloud Engine
|
||||||
|
- One crypto key (Protection level: softwere) for Cloud Storage
|
||||||
|
- GCE
|
||||||
|
- One instance encrypted with a CMEK Cryptokey hosted in Cloud KMS
|
||||||
|
- GCS
|
||||||
|
- One bucket encrypted with a CMEK Cryptokey hosted in Cloud KMS
|
||||||
|
|
||||||
|
<!-- BEGIN TFDOC -->
|
||||||
|
## Variables
|
||||||
|
|
||||||
|
| name | description | type | required | default |
|
||||||
|
|---|---|:---: |:---:|:---:|
|
||||||
|
| billing_account | Billing account id used as default for new projects. | <code title="">string</code> | ✓ | |
|
||||||
|
| root_node | The resource name of the parent Folder or Organization. Must be of the form folders/folder_id or organizations/org_id. | <code title="">string</code> | ✓ | |
|
||||||
|
| *location* | The location where resources will be deployed. | <code title="">string</code> | | <code title="">europe</code> |
|
||||||
|
| *project_kms_name* | Name for the new KMS Project. | <code title="">string</code> | | <code title="">my-project-kms-001</code> |
|
||||||
|
| *project_service_name* | Name for the new Service Project. | <code title="">string</code> | | <code title="">my-project-service-001</code> |
|
||||||
|
| *region* | The region where resources will be deployed. | <code title="">string</code> | | <code title="">europe-west1</code> |
|
||||||
|
| *vpc_ip_cidr_range* | Ip range used in the subnet deployef in the Service Project. | <code title="">string</code> | | <code title="">10.0.0.0/20</code> |
|
||||||
|
| *vpc_name* | Name of the VPC created in the Service Project. | <code title="">string</code> | | <code title="">local</code> |
|
||||||
|
| *vpc_subnet_name* | Name of the subnet created in the Service Project. | <code title="">string</code> | | <code title="">subnet</code> |
|
||||||
|
| *zone* | The zone where resources will be deployed. | <code title="">string</code> | | <code title="">europe-west1-b</code> |
|
||||||
|
|
||||||
|
## Outputs
|
||||||
|
|
||||||
|
| name | description | sensitive |
|
||||||
|
|---|---|:---:|
|
||||||
|
| bucket | GCS Bucket Cloud KMS crypto keys. | |
|
||||||
|
| bucket_keys | GCS Bucket Cloud KMS crypto keys. | |
|
||||||
|
| projects | Project ids. | |
|
||||||
|
| vm | GCE VMs. | |
|
||||||
|
| vm_keys | GCE VM Cloud KMS crypto keys. | |
|
||||||
|
<!-- END TFDOC -->
|
|
@ -0,0 +1,20 @@
|
||||||
|
# Copyright 2020 Google LLC
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# https://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
|
||||||
|
terraform {
|
||||||
|
backend "gcs" {
|
||||||
|
bucket = ""
|
||||||
|
}
|
||||||
|
}
|
Binary file not shown.
After Width: | Height: | Size: 145 KiB |
|
@ -0,0 +1,155 @@
|
||||||
|
# Copyright 2020 Google LLC
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# https://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
###############################################################################
|
||||||
|
# Projects #
|
||||||
|
###############################################################################
|
||||||
|
|
||||||
|
module "project-service" {
|
||||||
|
source = "../../modules/project"
|
||||||
|
name = var.project_service_name
|
||||||
|
parent = var.root_node
|
||||||
|
billing_account = var.billing_account
|
||||||
|
services = [
|
||||||
|
"compute.googleapis.com",
|
||||||
|
"servicenetworking.googleapis.com",
|
||||||
|
"storage-component.googleapis.com"
|
||||||
|
]
|
||||||
|
oslogin = true
|
||||||
|
}
|
||||||
|
|
||||||
|
module "project-kms" {
|
||||||
|
source = "../../modules/project"
|
||||||
|
name = var.project_kms_name
|
||||||
|
parent = var.root_node
|
||||||
|
billing_account = var.billing_account
|
||||||
|
services = [
|
||||||
|
"cloudkms.googleapis.com",
|
||||||
|
"servicenetworking.googleapis.com"
|
||||||
|
]
|
||||||
|
oslogin = true
|
||||||
|
}
|
||||||
|
|
||||||
|
###############################################################################
|
||||||
|
# Networking #
|
||||||
|
###############################################################################
|
||||||
|
|
||||||
|
module "vpc" {
|
||||||
|
source = "../../modules/net-vpc"
|
||||||
|
project_id = module.project-service.project_id
|
||||||
|
name = var.vpc_name
|
||||||
|
subnets = [
|
||||||
|
{
|
||||||
|
ip_cidr_range = var.vpc_ip_cidr_range
|
||||||
|
name = var.vpc_subnet_name
|
||||||
|
region = var.region
|
||||||
|
secondary_ip_range = {}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
module "vpc-firewall" {
|
||||||
|
source = "../../modules/net-vpc-firewall"
|
||||||
|
project_id = module.project-service.project_id
|
||||||
|
network = module.vpc.name
|
||||||
|
admin_ranges_enabled = true
|
||||||
|
admin_ranges = [var.vpc_ip_cidr_range]
|
||||||
|
}
|
||||||
|
|
||||||
|
###############################################################################
|
||||||
|
# KMS #
|
||||||
|
###############################################################################
|
||||||
|
|
||||||
|
module "kms" {
|
||||||
|
source = "../../modules/kms"
|
||||||
|
project_id = module.project-kms.project_id
|
||||||
|
keyring = {
|
||||||
|
name = "my-keyring",
|
||||||
|
location = var.location
|
||||||
|
}
|
||||||
|
keys = { key-gce = null, key-gcs = null }
|
||||||
|
key_iam_roles = {
|
||||||
|
key-gce = ["roles/cloudkms.cryptoKeyEncrypterDecrypter"]
|
||||||
|
}
|
||||||
|
key_iam_members = {
|
||||||
|
key-gce = {
|
||||||
|
"roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
|
||||||
|
"serviceAccount:${module.project-service.service_accounts.robots.compute}",
|
||||||
|
]
|
||||||
|
},
|
||||||
|
key-gcs = {
|
||||||
|
"roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
|
||||||
|
"serviceAccount:${module.project-service.service_accounts.robots.storage}",
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
###############################################################################
|
||||||
|
# GCE #
|
||||||
|
###############################################################################
|
||||||
|
|
||||||
|
module "kms_vm_example" {
|
||||||
|
source = "../../modules/compute-vm"
|
||||||
|
project_id = module.project-service.project_id
|
||||||
|
region = var.region
|
||||||
|
zone = var.zone
|
||||||
|
name = "kms-vm"
|
||||||
|
network_interfaces = [{
|
||||||
|
network = module.vpc.self_link,
|
||||||
|
subnetwork = module.vpc.subnet_self_links["${var.region}/subnet"],
|
||||||
|
nat = false,
|
||||||
|
addresses = null
|
||||||
|
}]
|
||||||
|
attached_disks = [
|
||||||
|
{
|
||||||
|
name = "attacheddisk"
|
||||||
|
size = 10
|
||||||
|
image = null
|
||||||
|
options = {
|
||||||
|
auto_delete = true
|
||||||
|
mode = null
|
||||||
|
source = null
|
||||||
|
type = null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
instance_count = 1
|
||||||
|
boot_disk = {
|
||||||
|
image = "projects/debian-cloud/global/images/family/debian-10"
|
||||||
|
type = "pd-ssd"
|
||||||
|
size = 10
|
||||||
|
encrypt_disk = true
|
||||||
|
}
|
||||||
|
tags = ["ssh"]
|
||||||
|
encryption = {
|
||||||
|
encrypt_boot = true
|
||||||
|
disk_encryption_key_raw = null
|
||||||
|
kms_key_self_link = module.kms.key_self_links.key-gce
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
###############################################################################
|
||||||
|
# GCS #
|
||||||
|
###############################################################################
|
||||||
|
|
||||||
|
module "kms-gcs" {
|
||||||
|
source = "../../modules/gcs"
|
||||||
|
project_id = module.project-service.project_id
|
||||||
|
prefix = "my-bucket-001"
|
||||||
|
names = ["kms-gcs"]
|
||||||
|
encryption_keys = {
|
||||||
|
kms-gcs = module.kms.keys.key-gce.self_link,
|
||||||
|
}
|
||||||
|
}
|
|
@ -0,0 +1,53 @@
|
||||||
|
# Copyright 2020 Google LLC
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# https://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
output "bucket" {
|
||||||
|
description = "GCS Bucket Cloud KMS crypto keys."
|
||||||
|
value = {
|
||||||
|
for bucket in module.kms-gcs.buckets :
|
||||||
|
bucket.name => bucket.url
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
output "bucket_keys" {
|
||||||
|
description = "GCS Bucket Cloud KMS crypto keys."
|
||||||
|
value = {
|
||||||
|
for bucket in module.kms-gcs.buckets :
|
||||||
|
bucket.name => bucket.encryption
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
output "projects" {
|
||||||
|
description = "Project ids."
|
||||||
|
value = {
|
||||||
|
service-project = module.project-service.project_id
|
||||||
|
kms-project = module.project-kms.project_id
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
output "vm" {
|
||||||
|
description = "GCE VMs."
|
||||||
|
value = {
|
||||||
|
for instance in module.kms_vm_example.instances :
|
||||||
|
instance.name => instance.network_interface.0.network_ip
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
output "vm_keys" {
|
||||||
|
description = "GCE VM Cloud KMS crypto keys."
|
||||||
|
value = {
|
||||||
|
for instance in module.kms_vm_example.instances :
|
||||||
|
instance.name => instance.boot_disk.0.kms_key_self_link
|
||||||
|
}
|
||||||
|
}
|
|
@ -0,0 +1,72 @@
|
||||||
|
# Copyright 2020 Google LLC
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# https://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
|
||||||
|
variable "billing_account" {
|
||||||
|
description = "Billing account id used as default for new projects."
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "location" {
|
||||||
|
description = "The location where resources will be deployed."
|
||||||
|
type = string
|
||||||
|
default = "europe"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "project_service_name" {
|
||||||
|
description = "Name for the new Service Project."
|
||||||
|
type = string
|
||||||
|
default = "my-project-service-001"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "project_kms_name" {
|
||||||
|
description = "Name for the new KMS Project."
|
||||||
|
type = string
|
||||||
|
default = "my-project-kms-001"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "region" {
|
||||||
|
description = "The region where resources will be deployed."
|
||||||
|
type = string
|
||||||
|
default = "europe-west1"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "root_node" {
|
||||||
|
description = "The resource name of the parent Folder or Organization. Must be of the form folders/folder_id or organizations/org_id."
|
||||||
|
type = string
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "vpc_name" {
|
||||||
|
description = "Name of the VPC created in the Service Project."
|
||||||
|
type = string
|
||||||
|
default = "local"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "vpc_subnet_name" {
|
||||||
|
description = "Name of the subnet created in the Service Project."
|
||||||
|
type = string
|
||||||
|
default = "subnet"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "vpc_ip_cidr_range" {
|
||||||
|
description = "Ip range used in the subnet deployef in the Service Project."
|
||||||
|
type = string
|
||||||
|
default = "10.0.0.0/20"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "zone" {
|
||||||
|
description = "The zone where resources will be deployed."
|
||||||
|
type = string
|
||||||
|
default = "europe-west1-b"
|
||||||
|
}
|
|
@ -0,0 +1,17 @@
|
||||||
|
# Copyright 2020 Google LLC
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# https://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
terraform {
|
||||||
|
required_version = ">= 0.12.6"
|
||||||
|
}
|
Loading…
Reference in New Issue