Merge branch 'data-e2e-01' of github.com:terraform-google-modules/cloud-foundation-fabric into data-e2e-01
This commit is contained in:
Ludovico Magnocavallo
commit
a3f5b4eed8
|
|
@ -19,8 +19,9 @@ Currently available examples:
|
|||
|
||||
- **foundations** - [single level hierarchy](./foundations/environments/) (environments), [multiple level hierarchy](./foundations/business-units/) (business units + environments)
|
||||
- **infrastructure** - [hub and spoke via peering](./infrastructure/hub-and-spoke-peering/), [hub and spoke via VPN](./infrastructure/hub-and-spoke-vpn/), [DNS and Google Private Access for on-premises](./infrastructure/onprem-google-access-dns/), [Shared VPC with GKE support](./infrastructure/shared-vpc-gke/)
|
||||
- **data solutions** - [Cloud KMS with Cloud Storage and Compute Engine](./data-solutions/cloud-kms/)
|
||||
|
||||
For more information see the README files in the [foundations](./foundations/) and [infrastructure](./infrastructure/) folders.
|
||||
For more information see the README files in the [foundations](./foundations/), [infrastructure](./infrastructure/) and [data solutions](./data-solutions/) folders.
|
||||
|
||||
## Modules
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,14 @@
|
|||
# GCP Data Services examples
|
||||
|
||||
The examples in this folder implement **typical data servicies topologies**, or **end-to-end scenarios** that allow testing specific features like Cloud KMS to encrypt your data or VPC-SC to mitigate data exfiltration.
|
||||
|
||||
They are meant to be used as minimal but complete starting points to create actual infrastructure, and as playgrounds to experiment with specific Google Cloud features.
|
||||
|
||||
## Examples
|
||||
|
||||
### Cloud KMS with Cloud Storage and Compute Engine
|
||||
|
||||
<a href="./cloud-kms/" title="Cloud KMS with Cloud Storage and Compute Engine"><img src="./cloud-kms/diagram.png" align="left" width="280px"></a> This [example](./cloud-kms/) implements a project to host a centralized Cloud KMS instance and a service project that will use Crypto keys to encrypt resources at rest.
|
||||
|
||||
The sample highlights roles and permissions needed to implement the architecture described.
|
||||
<br clear="left">
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
# Cloud KMS with Cloud Storage and Compute Engine
|
||||
|
||||
This sample creates a sample [Cloud KMS](https://cloud.google.com/kms?hl=it) configuration to be used with [Cloud Storage](https://cloud.google.com/storage/docs/encryption/using-customer-managed-keys) and [Copute Engine](https://cloud.google.com/compute/docs/disks/customer-managed-encryption).
|
||||
Cloud KMS is deployed in a separate project to highlight the IAM binding needed and to mock a more real scenario where you usually have a project to manage keys across all your projects in one single place.
|
||||
|
||||
The sample has been purposefully kept simple so that it can be used as a basis for different and more complex configuration. This is the high level diagram:
|
||||
|
||||
|
||||
|
||||
## Managed resources and services
|
||||
|
||||
This sample creates several distinct groups of resources:
|
||||
|
||||
- projects
|
||||
- Cloud KMS project
|
||||
- Service Project configured for GCE instances and GCS buckets
|
||||
- networking
|
||||
- VPC network
|
||||
- One subnet
|
||||
- Firewall rules for [SSH access via IAP](https://cloud.google.com/iap/docs/using-tcp-forwarding) and open communication within the VPC
|
||||
- IAM
|
||||
- One service account for the GGE instance
|
||||
- KMS
|
||||
- One key ring
|
||||
- One crypto key (Procection level: softwere) for Cloud Engine
|
||||
- One crypto key (Protection level: softwere) for Cloud Storage
|
||||
- GCE
|
||||
- One instance encrypted with a CMEK Cryptokey hosted in Cloud KMS
|
||||
- GCS
|
||||
- One bucket encrypted with a CMEK Cryptokey hosted in Cloud KMS
|
||||
|
||||
<!-- BEGIN TFDOC -->
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---: |:---:|:---:|
|
||||
| billing_account | Billing account id used as default for new projects. | <code title="">string</code> | ✓ | |
|
||||
| projects_parent | The resource name of the parent Folder or Organization. Must be of the form folders/folder_id or organizations/org_id. | <code title="">string</code> | ✓ | |
|
||||
| *project_kms_name* | Name for the new KMS Project. | <code title="">string</code> | | <code title="">my-project-kms-001</code> |
|
||||
| *project_service_name* | Name for the new Service Project. | <code title="">string</code> | | <code title="">my-project-service-001</code> |
|
||||
| *resource_location* | The location where resources will be deployed. | <code title="">string</code> | | <code title="">europe</code> |
|
||||
| *resource_region* | The region where resources will be deployed. | <code title="">string</code> | | <code title="">europe-west1</code> |
|
||||
| *resource_zone* | The zone where resources will be deployed. | <code title="">string</code> | | <code title="">europe-west1-b</code> |
|
||||
| *vpc_ip_cidr_range* | Ip range used in the subnet deployef in the Service Project. | <code title="">string</code> | | <code title="">10.0.0.0/20</code> |
|
||||
| *vpc_name* | Name of the VPC created in the Service Project. | <code title="">string</code> | | <code title="">local</code> |
|
||||
| *vpc_subnet_name* | Name of the subnet created in the Service Project. | <code title="">string</code> | | <code title="">subnet</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| buckets_keys | GCS Buckets Cloud KMS crypto keys. | |
|
||||
| projects | Project ids. | |
|
||||
| vms_keys | GCE VMs Cloud KMS crypto keys. | |
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
|
||||
terraform {
|
||||
backend "gcs" {
|
||||
bucket = ""
|
||||
}
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 143 KiB |
|
|
@ -0,0 +1,156 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
###############################################################################
|
||||
# Projects #
|
||||
###############################################################################
|
||||
|
||||
module "project-service" {
|
||||
source = "../../modules/project"
|
||||
name = var.project_service_name
|
||||
parent = var.projects_parent
|
||||
billing_account = var.billing_account
|
||||
services = [
|
||||
"compute.googleapis.com",
|
||||
"servicenetworking.googleapis.com",
|
||||
"storage-component.googleapis.com"
|
||||
]
|
||||
oslogin = true
|
||||
}
|
||||
|
||||
module "project-kms" {
|
||||
source = "../../modules/project"
|
||||
name = var.project_kms_name
|
||||
parent = var.projects_parent
|
||||
billing_account = var.billing_account
|
||||
services = [
|
||||
"cloudkms.googleapis.com",
|
||||
"servicenetworking.googleapis.com"
|
||||
]
|
||||
oslogin = true
|
||||
}
|
||||
|
||||
###############################################################################
|
||||
# Networking #
|
||||
###############################################################################
|
||||
|
||||
module "vpc" {
|
||||
source = "../../modules/net-vpc"
|
||||
project_id = module.project-service.project_id
|
||||
name = var.vpc_name
|
||||
subnets = [
|
||||
{
|
||||
ip_cidr_range = var.vpc_ip_cidr_range
|
||||
name = var.vpc_subnet_name
|
||||
region = var.resource_region
|
||||
secondary_ip_range = {}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
module "vpc-firewall" {
|
||||
source = "../../modules/net-vpc-firewall"
|
||||
project_id = module.project-service.project_id
|
||||
network = module.vpc.name
|
||||
admin_ranges_enabled = true
|
||||
admin_ranges = [var.vpc_ip_cidr_range]
|
||||
ssh_source_ranges = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
###############################################################################
|
||||
# KMS #
|
||||
###############################################################################
|
||||
|
||||
module "kms" {
|
||||
source = "../../modules/kms"
|
||||
project_id = module.project-kms.project_id
|
||||
keyring = {
|
||||
name = "my-keyring",
|
||||
location = var.resource_location
|
||||
}
|
||||
keys = { key-gce = null, key-gcs = null }
|
||||
key_iam_roles = {
|
||||
key-gce = ["roles/cloudkms.cryptoKeyEncrypterDecrypter"]
|
||||
}
|
||||
key_iam_members = {
|
||||
key-gce = {
|
||||
"roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
|
||||
"serviceAccount:${module.project-service.service_accounts.robots.compute}",
|
||||
]
|
||||
},
|
||||
key-gcs = {
|
||||
"roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
|
||||
"serviceAccount:${module.project-service.service_accounts.robots.storage}",
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
###############################################################################
|
||||
# GCE #
|
||||
###############################################################################
|
||||
|
||||
module "kms_vm_example" {
|
||||
source = "../../modules/compute-vm"
|
||||
project_id = module.project-service.project_id
|
||||
region = var.resource_region
|
||||
zone = var.resource_zone
|
||||
name = "kms-vm"
|
||||
network_interfaces = [{
|
||||
network = module.vpc.self_link,
|
||||
subnetwork = module.vpc.subnet_self_links["${var.resource_region}/subnet"],
|
||||
nat = false,
|
||||
addresses = null
|
||||
}]
|
||||
attached_disks = [
|
||||
{
|
||||
name = "attacheddisk"
|
||||
size = 10
|
||||
image = null
|
||||
options = {
|
||||
auto_delete = true
|
||||
mode = null
|
||||
source = null
|
||||
type = null
|
||||
}
|
||||
}
|
||||
]
|
||||
instance_count = 1
|
||||
boot_disk = {
|
||||
image = "projects/debian-cloud/global/images/family/debian-10"
|
||||
type = "pd-ssd"
|
||||
size = 10
|
||||
encrypt_disk = true
|
||||
}
|
||||
tags = ["ssh"]
|
||||
encryption = {
|
||||
encrypt_boot = true
|
||||
disk_encryption_key_raw = null
|
||||
kms_key_self_link = module.kms.key_self_links.key-gce
|
||||
}
|
||||
}
|
||||
|
||||
###############################################################################
|
||||
# GCS #
|
||||
###############################################################################
|
||||
|
||||
module "kms-gcs" {
|
||||
source = "../../modules/gcs"
|
||||
project_id = module.project-service.project_id
|
||||
prefix = "my-bucket-001"
|
||||
names = ["kms-gcs"]
|
||||
encryption_key = {
|
||||
kms-gcs = module.kms.keys.key-gce.self_link,
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
output "projects" {
|
||||
description = "Project ids."
|
||||
value = {
|
||||
service-project = module.project-service.project_id
|
||||
kms-project = module.project-kms.project_id
|
||||
}
|
||||
}
|
||||
|
||||
output "vms_keys" {
|
||||
description = "GCE VMs Cloud KMS crypto keys."
|
||||
value = {
|
||||
for instance in module.kms_vm_example.instances :
|
||||
instance.name => instance.boot_disk.0.kms_key_self_link
|
||||
}
|
||||
}
|
||||
|
||||
output "buckets_keys" {
|
||||
description = "GCS Buckets Cloud KMS crypto keys."
|
||||
value = {
|
||||
for bucket in module.kms-gcs.buckets :
|
||||
bucket.name => bucket.encryption
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
|
||||
variable "billing_account" {
|
||||
description = "Billing account id used as default for new projects."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "project_service_name" {
|
||||
description = "Name for the new Service Project."
|
||||
type = string
|
||||
default = "my-project-service-001"
|
||||
}
|
||||
|
||||
variable "project_kms_name" {
|
||||
description = "Name for the new KMS Project."
|
||||
type = string
|
||||
default = "my-project-kms-001"
|
||||
}
|
||||
|
||||
variable "projects_parent" {
|
||||
description = "The resource name of the parent Folder or Organization. Must be of the form folders/folder_id or organizations/org_id."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "resource_location" {
|
||||
description = "The location where resources will be deployed."
|
||||
type = string
|
||||
default = "europe"
|
||||
}
|
||||
|
||||
variable "resource_region" {
|
||||
description = "The region where resources will be deployed."
|
||||
type = string
|
||||
default = "europe-west1"
|
||||
}
|
||||
|
||||
variable "resource_zone" {
|
||||
description = "The zone where resources will be deployed."
|
||||
type = string
|
||||
default = "europe-west1-b"
|
||||
}
|
||||
|
||||
variable "vpc_name" {
|
||||
description = "Name of the VPC created in the Service Project."
|
||||
type = string
|
||||
default = "local"
|
||||
}
|
||||
|
||||
variable "vpc_subnet_name" {
|
||||
description = "Name of the subnet created in the Service Project."
|
||||
type = string
|
||||
default = "subnet"
|
||||
}
|
||||
|
||||
variable "vpc_ip_cidr_range" {
|
||||
description = "Ip range used in the subnet deployef in the Service Project."
|
||||
type = string
|
||||
default = "10.0.0.0/20"
|
||||
}
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
terraform {
|
||||
required_version = ">= 0.12"
|
||||
}
|
||||
Loading…
Reference in New Issue