Business Units org-bootstrap example (#8)

* Business Units initial commit

* Update Business-Units example. Log-sink moved to org-level, SA and TF-Buckets moved from wrapper module to main config

* brought README up to date with the section's and environment sample's READMEs

* Add new line at the end of the file

* Add new line at the end of the file

* Streamline README
This commit is contained in:
Aleksandr Averbukh 2019-09-11 14:39:33 +02:00 committed by Ludovico Magnocavallo
parent e553da6464
commit 21ab51d1d6
13 changed files with 662 additions and 0 deletions

View File

@ -0,0 +1,69 @@
# Business-units based organizational sample
This sample creates an organizational layout with two folder levels, where the first level is usually mapped to one business unit or team (infra, data, analytics) and the second level represents enviroments (prod, test). It also sets up all prerequisites for automation (GCS state buckets, service accounts, etc.), and the correct roles on those to enforce separation of duties at the environment level.
This layout is well suited for medium-sized infrastructures managed by different sets of teams, and especially where the foundational infrastructure needs to be managed centrally, as the top-level automation service accounts for each environment allow cross-team management of the base resources (projects, IAM, etc.).
![High-level diagram](diagram.png "High-level diagram")
Refer to the [section-level README](../README.md) for general considerations about this type of samples, and usage instructions.
## Managed resources and services
This sample creates several distinct groups of resources:
- one top-level folder per business unit/team
- one top-level folder for shared services
- one second-level folder for each environment in all the business unit top-level folders
- one project in the shared folder to hold Terraform-related resources
- one project in the shared folder to set up and host centralized audit log exports
- one project in the shared folder to hold services used across environments like GCS, GCR, KMS, Cloud Build, etc.
The number of resources in this sample is kept to a minimum so as to make it generally applicable, more resources can be easily added by leveraging the full array of [Cloud Foundation Toolkit modules](https://github.com/terraform-google-modules), especially in the shared services project.
## Shared services
This sample uses a top-level folder to encapsulate projects that host resources that are not specific to a single environment. If no shared services are needed,the Terraform and audit modules can be easily attached to the root node, and the shared services folder and project removed from `main.tf`.
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Inputs
| Name | Description | Type | Default | Required |
|------|-------------|:----:|:-----:|:-----:|
| audit\_viewers | Audit project viewers, in IAM format. | list | `<list>` | no |
| billing\_account\_id | Billing account id used as default for new projects. | string | n/a | yes |
| business\_unit\_1\_name | Business unit 1 short name. | string | n/a | yes |
| business\_unit\_2\_name | Business unit 2 short name. | string | n/a | yes |
| business\_unit\_3\_name | Business unit 3 short name. | string | n/a | yes |
| environments | Environment short names. | list(string) | n/a | yes |
| gcs\_location | GCS bucket location. | string | `"EU"` | no |
| generate\_service\_account\_keys | Generate and store service account keys in the state file. | string | `"false"` | no |
| organization\_id | Organization id. | string | n/a | yes |
| prefix | Prefix used for resources that need unique names. | string | n/a | yes |
| project\_services | Service APIs enabled by default in new projects. | list | `<list>` | no |
| root\_node | Root node for the new hierarchy, either 'organizations/org_id' or 'folders/folder_id'. | string | n/a | yes |
| shared\_bindings\_members | List of comma-delimited IAM-format members for the additional shared project bindings. | list | `<list>` | no |
| shared\_bindings\_roles | List of roles for additional shared project bindings. | list | `<list>` | no |
| terraform\_owners | Terraform project owners, in IAM format. | list | `<list>` | no |
## Outputs
| Name | Description |
|------|-------------|
| audit\_logs\_bq\_dataset | Bigquery dataset for the audit logs export. |
| audit\_logs\_project | Project that holds the audit logs export resources. |
| bootstrap\_tf\_gcs\_bucket | GCS bucket used for the bootstrap Terraform state. |
| business\_unit\_1\_environment\_folders\_ids | Business unit 1 environment folders. |
| business\_unit\_1\_folder\_id | Business unit 1 top-level folder ID. |
| business\_unit\_2\_environment\_folders\_ids | Business unit 2 environment folders. |
| business\_unit\_2\_folder\_id | Business unit 2 top-level folder ID. |
| business\_unit\_3\_environment\_folders\_ids | Business unit 3 environment folders. |
| business\_unit\_3\_folder\_id | Business unit 3 top-level folder ID. |
| environment\_service\_account\_keys | Service account keys used to run each environment Terraform modules. |
| environment\_service\_accounts | Service accounts used to run each environment Terraform modules. |
| environment\_tf\_gcs\_buckets | GCS buckets used for each environment Terraform state. |
| shared\_folder\_id | Shared folder ID. |
| shared\_resources\_project | Project that holdes resources shared across business units. |
| terraform\_project | Project that holds the base Terraform resources. |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

View File

@ -0,0 +1,22 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
backend "gcs" {
# once initial apply has completed, copy this file to `backend.tf` then
# set the `bucket` value to the `bootstrap_tf_gcs_bucket` output, then
# run apply again to transfer state
bucket = ""
}
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

View File

@ -0,0 +1,195 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# TODO(averbukh): simplify log-sink parameters once https://github.com/terraform-google-modules/terraform-google-log-export/issues/28 is done.
locals {
parent_numeric_id = element(split("/", var.root_node), 1)
log_sink_parent_resource_type = element(split("/", var.root_node), 0) == "organizations" ? "organization" : "folder"
log_sink_name = element(split("/", var.root_node), 0) == "organizations" ? "logs-audit-org-${local.parent_numeric_id}" : "logs-audit-folder-${local.parent_numeric_id}"
}
###############################################################################
# Shared resources folder #
###############################################################################
module "shared-folder" {
source = "terraform-google-modules/folders/google"
version = "2.0.0"
parent = var.root_node
names = ["shared"]
}
###############################################################################
# Terraform top-level resources #
###############################################################################
# Terraform project
module "project-tf" {
source = "terraform-google-modules/project-factory/google//modules/fabric-project"
version = "3.2.0"
parent = module.shared-folder.id
billing_account = var.billing_account_id
prefix = var.prefix
name = "terraform"
lien_reason = "terraform"
owners = var.terraform_owners
activate_apis = var.project_services
}
# Per environment service accounts
module "service-accounts-tf-environments" {
source = "terraform-google-modules/service-accounts/google"
version = "2.0.0"
project_id = module.project-tf.project_id
org_id = var.organization_id
billing_account_id = var.billing_account_id
prefix = var.prefix
names = var.environments
grant_billing_role = true
generate_keys = var.generate_service_account_keys
}
# Bootstrap Terraform state GCS bucket
module "gcs-tf-bootstrap" {
source = "terraform-google-modules/cloud-storage/google"
version = "1.0.0"
project_id = module.project-tf.project_id
prefix = "${var.prefix}-tf"
names = ["tf-bootstrap"]
location = var.gcs_location
}
# Per environment Terraform state GCS buckets
module "gcs-tf-environments" {
source = "terraform-google-modules/cloud-storage/google"
version = "1.0.0"
project_id = module.project-tf.project_id
prefix = "${var.prefix}-tf"
names = var.environments
location = var.gcs_location
set_admin_roles = true
bucket_admins = zipmap(
var.environments,
module.service-accounts-tf-environments.iam_emails_list
)
}
###############################################################################
# Business units #
###############################################################################
# Business unit 1
module "business-unit-1-folders" {
source = "./modules/business-unit-folders"
business_unit_folder_name = var.business_unit_1_name
environments = var.environments
per_folder_admins = module.service-accounts-tf-environments.iam_emails_list
root_node = var.root_node
}
# Business unit 2
module "business-unit-2-folders" {
source = "./modules/business-unit-folders"
business_unit_folder_name = var.business_unit_2_name
environments = var.environments
per_folder_admins = module.service-accounts-tf-environments.iam_emails_list
root_node = var.root_node
}
# Business unit 3
module "business-unit-3-folders" {
source = "./modules/business-unit-folders"
business_unit_folder_name = var.business_unit_3_name
environments = var.environments
per_folder_admins = module.service-accounts-tf-environments.iam_emails_list
root_node = var.root_node
}
###############################################################################
# Audit log exports #
###############################################################################
# Audit logs project
module "project-audit" {
source = "terraform-google-modules/project-factory/google//modules/fabric-project"
version = "3.2.0"
parent = module.shared-folder.id
billing_account = var.billing_account_id
prefix = var.prefix
name = "audit"
lien_reason = "audit"
activate_apis = var.project_services
viewers = var.audit_viewers
}
# Audit logs destination on BigQuery
module "bq-audit-export" {
source = "terraform-google-modules/log-export/google//modules/bigquery"
version = "3.0.0"
project_id = module.project-audit.project_id
dataset_name = "${replace(local.log_sink_name, "-", "_")}"
log_sink_writer_identity = module.log-sink-audit.writer_identity
}
# Audit log sink for root node
module "log-sink-audit" {
source = "terraform-google-modules/log-export/google"
version = "3.0.0"
filter = "logName: \"/logs/cloudaudit.googleapis.com%2Factivity\" OR logName: \"/logs/cloudaudit.googleapis.com%2Fsystem_event\""
log_sink_name = local.log_sink_name
parent_resource_type = local.log_sink_parent_resource_type
parent_resource_id = local.parent_numeric_id
include_children = "true"
unique_writer_identity = "true"
destination_uri = "${module.bq-audit-export.destination_uri}"
}
###############################################################################
# Shared resources (GCR, GCS, KMS, etc.) #
###############################################################################
# Shared resources project
module "project-shared-resources" {
source = "terraform-google-modules/project-factory/google//modules/fabric-project"
version = "3.2.0"
parent = module.shared-folder.id
billing_account = var.billing_account_id
prefix = var.prefix
name = "shared"
lien_reason = "shared"
activate_apis = var.project_services
extra_bindings_roles = var.shared_bindings_roles
extra_bindings_members = var.shared_bindings_members
}
# Add further modules here for resources that are common to all business units
# like GCS buckets (used to hold shared assets), Container Registry, KMS, etc.

View File

@ -0,0 +1,24 @@
# Two-level folders tree
This module is a simple wrapper for the [Cloud Foundation Folder module](https://github.com/terraform-google-modules/terraform-google-folders), that manages one folder and one child folder under it, for each name passed in the `environments` variable. It is meant to be used for organizational layouts where a predefined number of folders representing environments, are created as child folders for business units or teams.
For details on how the IAM variables work, please refer to the [Cloud Foundation Folder module](https://github.com/terraform-google-modules/terraform-google-folders).
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Inputs
| Name | Description | Type | Default | Required |
|------|-------------|:----:|:-----:|:-----:|
| business\_unit\_folder\_name | Business Unit Folder name. | string | n/a | yes |
| environments | Environment short names. | list(string) | n/a | yes |
| per\_folder\_admins | List of IAM-style members per folder who will get extended permissions. | list(string) | `<list>` | no |
| root\_node | Root node for the new hierarchy, either 'organizations/org_id' or 'folders/folder_id'. | string | n/a | yes |
## Outputs
| Name | Description |
|------|-------------|
| business\_unit\_folder\_id | Business Unit Folder ID. |
| environment\_folders\_ids | Environment folders IDs. |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

View File

@ -0,0 +1,44 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
###############################################################################
# Business Unit Folder #
###############################################################################
module "business-unit-folder" {
source = "terraform-google-modules/folders/google"
version = "2.0.0"
parent = var.root_node
names = [var.business_unit_folder_name]
}
###############################################################################
# Environment Folders #
###############################################################################
module "environment-folders" {
source = "terraform-google-modules/folders/google"
version = "2.0.0"
parent = module.business-unit-folder.id
names = var.environments
set_roles = true
per_folder_admins = var.per_folder_admins
folder_admin_roles = [
"roles/resourcemanager.folderViewer",
"roles/resourcemanager.projectCreator",
"roles/owner",
"roles/compute.networkAdmin",
"roles/compute.xpnAdmin"
]
}

View File

@ -0,0 +1,26 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
output "business_unit_folder_id" {
description = "Business Unit Folder ID."
value = module.business-unit-folder.id
}
output "environment_folders_ids" {
description = "Environment folders IDs."
value = module.environment-folders.ids
}
# Add further outputs here for the additional modules that manage shared
# resources, like GCR, GCS buckets, KMS, etc.

View File

@ -0,0 +1,34 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
variable "business_unit_folder_name" {
description = "Business Unit Folder name."
type = string
}
variable "environments" {
description = "Environment short names."
type = list(string)
}
variable "per_folder_admins" {
type = list(string)
description = "List of IAM-style members per folder who will get extended permissions."
default = []
}
variable "root_node" {
description = "Root node for the new hierarchy, either 'organizations/org_id' or 'folders/folder_id'."
type = string
}

View File

@ -0,0 +1,17 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_version = ">= 0.12"
}

View File

@ -0,0 +1,91 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
output "terraform_project" {
description = "Project that holds the base Terraform resources."
value = module.project-tf.project_id
}
output "bootstrap_tf_gcs_bucket" {
description = "GCS bucket used for the bootstrap Terraform state."
value = module.gcs-tf-bootstrap.name
}
output "environment_tf_gcs_buckets" {
description = "GCS buckets used for each environment Terraform state."
value = module.gcs-tf-environments.names
}
output "environment_service_account_keys" {
description = "Service account keys used to run each environment Terraform modules."
sensitive = true
value = module.service-accounts-tf-environments.keys
}
output "environment_service_accounts" {
description = "Service accounts used to run each environment Terraform modules."
value = module.service-accounts-tf-environments.emails
}
output "shared_folder_id" {
description = "Shared folder ID."
value = module.shared-folder.id
}
output "business_unit_1_folder_id" {
description = "Business unit 1 top-level folder ID."
value = module.business-unit-1-folders.business_unit_folder_id
}
output "business_unit_1_environment_folders_ids" {
description = "Business unit 1 environment folders."
value = module.business-unit-1-folders.environment_folders_ids
}
output "business_unit_2_folder_id" {
description = "Business unit 2 top-level folder ID."
value = module.business-unit-2-folders.business_unit_folder_id
}
output "business_unit_2_environment_folders_ids" {
description = "Business unit 2 environment folders."
value = module.business-unit-2-folders.environment_folders_ids
}
output "business_unit_3_folder_id" {
description = "Business unit 3 top-level folder ID."
value = module.business-unit-3-folders.business_unit_folder_id
}
output "business_unit_3_environment_folders_ids" {
description = "Business unit 3 environment folders."
value = module.business-unit-3-folders.environment_folders_ids
}
output "audit_logs_bq_dataset" {
description = "Bigquery dataset for the audit logs export."
value = module.bq-audit-export.resource_name
}
output "audit_logs_project" {
description = "Project that holds the audit logs export resources."
value = module.project-audit.project_id
}
output "shared_resources_project" {
description = "Project that holdes resources shared across business units."
value = module.project-shared-resources.project_id
}
# Add further outputs here for the additional modules that manage shared
# resources, like GCR, GCS buckets, KMS, etc.

View File

@ -0,0 +1,15 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
provider "google" {}

View File

@ -0,0 +1,108 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
variable "audit_viewers" {
description = "Audit project viewers, in IAM format."
default = []
}
variable "billing_account_id" {
description = "Billing account id used as default for new projects."
type = string
}
variable "business_unit_1_name" {
description = "Business unit 1 short name."
type = string
}
variable "business_unit_2_name" {
description = "Business unit 2 short name."
type = string
}
variable "business_unit_3_name" {
description = "Business unit 3 short name."
type = string
}
variable "environments" {
description = "Environment short names."
type = list(string)
}
variable "generate_service_account_keys" {
description = "Generate and store service account keys in the state file."
default = false
}
variable "gcs_location" {
description = "GCS bucket location."
default = "EU"
}
variable "organization_id" {
description = "Organization id."
type = string
}
variable "prefix" {
description = "Prefix used for resources that need unique names."
type = string
}
variable "root_node" {
description = "Root node for the new hierarchy, either 'organizations/org_id' or 'folders/folder_id'."
type = string
}
variable "shared_bindings_members" {
description = "List of comma-delimited IAM-format members for the additional shared project bindings."
# example: ["user:a@example.com,b@example.com", "user:c@example.com"]
default = []
}
variable "shared_bindings_roles" {
description = "List of roles for additional shared project bindings."
# example: ["roles/storage.objectViewer", "roles/storage.admin"]
default = []
}
variable "terraform_owners" {
description = "Terraform project owners, in IAM format."
default = []
}
variable "project_services" {
description = "Service APIs enabled by default in new projects."
default = [
"bigquery-json.googleapis.com",
"bigquerystorage.googleapis.com",
"cloudbilling.googleapis.com",
"cloudresourcemanager.googleapis.com",
"compute.googleapis.com",
"container.googleapis.com",
"containerregistry.googleapis.com",
"deploymentmanager.googleapis.com",
"iam.googleapis.com",
"iamcredentials.googleapis.com",
"logging.googleapis.com",
"oslogin.googleapis.com",
"pubsub.googleapis.com",
"replicapool.googleapis.com",
"replicapoolupdater.googleapis.com",
"resourceviews.googleapis.com",
"serviceusage.googleapis.com",
"storage-api.googleapis.com",
]
}

View File

@ -0,0 +1,17 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_version = ">= 0.12"
}