Organization sample: environments (Almanac bootstrap port)

* add gitignore file

* data and infra skeletons

* org skeleton

* org environments sample skeleton

* Organization teams sample skeleton.

* Organization env sample GCS for tf state.

* org env: service accounts and GCS roles

* org env: folders

* org env: audit export

* org env: shared project

* org env: switch to released 3.1.0 project module version

* rename organization top-level folder to organization-bootstrap

* org env: use folders module with better outputs, module outputs

* org env: switch the service accounts module to v2.0.0.

* Merge ludo's branch from forked repo (#2)

* org env: update gcs, sa, project modules

* Use correct folder ID in format  without  prefix

* org env: update folders module version, improve comments

* org env: initial work on README, diagram, add variable for xpn roles

* org env: update roles in README, backend file

* org env: README changes

* org env: README changes

* org env: README changes

* org env: README changes

* org env: add IAM variables for audit and shared projects

* org env: address README TODOs

* org env: minor README changes

* org env: minor README changes

* org env: minor README changes

* org env: simplify the sample's README by moving general considerations into the section README

* Org examples README changes

* org env: simplify initial state management, add comment for shared folder

* org env: change state instructions to copy and rename instead of renaming backend file

* org env: add a section in the README file to explain shared services, and detail options

* fix comment for shared services project

.gitignore

@@ -0,0 +1,8 @@
+**/.terraform
+**/terraform.tfstate*
+**/terraform.tfvars
+.idea
+.vscode
+backend-config.hcl
+credentials.json
+key.json

organization-bootstrap/README.md

@@ -0,0 +1,40 @@
+# Organization-level bootstrap samples
+
+This set of Terraform root modules is designed with two main purposes in mind: automating the organizational layout, and bootstrapping the initial resources and the corresponding IAM roles, which will then be used to automate the actual infrastructure.
+
+Despite being fairly generic, these modules closely match some of the initial automation stages we have implemented in actual customer engagements, and are purposely kept simple to offer a good starting point for further customizations.
+
+There are several advantages in using an initial stage like the ones provided here:
+
+- automate and parameterize creation of the organizational layout, documenting it through code
+- use a single declarative tool to create and manage both prerequisites and the actual infrastructure, eliminating the need for manual commands, scripts or external tools
+- enforce separation of duties at the environment (or tenant in multi-tenant architectures) level, by automating creation of per-environment Terraform service accounts, their IAM roles, and GCS buckets to hold state
+- decouple and document the use of organization-level permissions from the day to day management of the actual infrastructure, by assigning a minimum but sufficient set of high level IAM roles to Terraform service accounts in an initial stage
+- provide a sane place for the creation and management of shared resources that are not tied to a specific environment
+
+## Operational considerations
+
+This specific type of preliminary automation stage is usually fairly static, only changing when a new environment or shared resource is added or modified, and lends itself well to being applied manually by organization administrators. One secondary advantage of running this initial stage manually, is eliminating the need to create and manage automation credentials that embed sensitive permissions scoped at the organization or root folder level.
+
+### IAM roles
+
+This type of automation stage needs very specific IAM roles on the root node (organization or folder), and additional roles at the organization level if the generated service accounts for automation need to be able to create and manage Shared VPC. The needed roles are:
+
+- on the root node Project Creator, Folder Administrator, Logging Administrator
+- on the billing account or organization Billing Account Administrator
+- on the organization Organization Administrator, if Shared VPC needs to be managed by the automation service accounts
+
+### State
+
+This type of stage creates the prerequisites for Terraform automation including the GCS bucket used for its own remote state, so some care needs to be used when running it for the first time, when its GCS bucket has not yet been created.
+
+After the first successful `terraform apply`, copy the `backend.tf.sample` file
+to `backend.tf`, then set the bucket name to the one shown in the `bootstrap_tf_gcs_bucket` output in the new file. Once that is done, run `terraform apply` again to transfer local state to the remote GCS bucket. From then on, state will be remote.
+
+### Things to be aware of
+
+Using `count` in Terraform resources has the [well-known limitation](https://github.com/hashicorp/terraform/issues/18767) that changing the variable controlling `count` results in the potential unwanted deletion of resources.
+
+These samples use `count` on the `environments` list variable to manage multiples for key resources like service accounts, GCS buckets, and IAM roles. Environment names are usually stable, but care must still be taken in defining the initial list so that names are final, and names for temporary environments (if any are needed) are last so they don't trigger recreation of resources based on the following elements in the list.
+
+This issue will be addressed in a future release of these examples, by replacing `count` with the new `foreach` construct [introduced in Terraform 0.12.6](https://twitter.com/mitchellh/status/1156661893789966336?lang=en) that uses key-based indexing.

organization-bootstrap/environments/README.md

@@ -0,0 +1,63 @@
+# Environment-based organizational sample
+
+This sample creates an organizational layout with a single level, where each  folder is usually mapped to one infrastructure environment (test, dev, etc.). It also sets up all prerequisites for automation (GCS state buckets, service accounts, etc.), and the correct roles on those to enforce separation of duties at the environment level.
+
+This layout is well suited for medium-sized infrastructures managed by a small set of teams, where the complexity in application resource ownership and access roles is mostly dealt with at the project level, and/or in the individual services (GKE, Cloud SQL, etc.). Its simplicity also makes it a good starting point for more complex or specialized layouts.
+
+![High-level diagram](diagram.png "High-level diagram")
+
+Refer to the [section-level README](../README.md) for general considerations about this type of samples, and usage instructions.
+
+## Managed resources and services
+
+This sample creates several distinct groups of resources:
+
+- one folder per environment
+- one top-level project to hold Terraform-related resources
+- one top-level project to set up and host centralized audit log exports (optional)
+- one top-level shared services project
+
+The number of resources in this sample is kept to a minimum so as to make it more generally applicable, further resources can be easily added by leveraging the full array of [Cloud Foundation Toolkit modules](https://github.com/terraform-google-modules), especially in the shared services project.
+
+## Shared services project
+
+This sample contains a single, top-level project used to host services shared across environments (eg GCS, GCR, KMS, Cloud Build, etc.). In our experience, that is enough for many customers, especially those using this organizational layout.
+
+For more complex setups where multiple shared services projects are needed to encapsulate a larger number of resources, shared services should be treated as an extra environment so that they can be managed by a dedicated set of Terraform files, using a separate service account and GCS bucket, with a folder to contain shared projects.
+
+If no shared services are needed, the shared service project module can of course be removed from `main.tf`.
+
+<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|:----:|:-----:|:-----:|
+| audit\_viewers | Audit project viewers, in IAM format. | list | `<list>` | no |
+| billing\_account\_id | Billing account id used as default for new projects. | string | n/a | yes |
+| environments | Environment short names. | list(string) | n/a | yes |
+| gcs\_location | GCS bucket location. | string | `"EU"` | no |
+| generate\_service\_account\_keys | Generate and store service account keys in the state file. | string | `"false"` | no |
+| grant\_xpn\_roles | Grant roles needed for Shared VPC creation to service accounts. | string | `"true"` | no |
+| organization\_id | Organization id. | string | n/a | yes |
+| prefix | Prefix used for resources that need unique names. | string | n/a | yes |
+| project\_services | Service APIs enabled by default in new projects. | list | `<list>` | no |
+| root\_node | Root node for the new hierarchy, either 'organizations/org_id' or 'folders/folder_id'. | string | n/a | yes |
+| shared\_bindings\_members | List of comma-delimited IAM-format members for the additional shared project bindings. | list | `<list>` | no |
+| shared\_bindings\_roles | List of roles for additional shared project bindings. | list | `<list>` | no |
+| terraform\_owners | Terraform project owners, in IAM format. | list | `<list>` | no |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| audit\_logs\_bq\_dataset | Bigquery dataset for the audit logs export. |
+| audit\_logs\_project | Project that holds the audit logs export resources. |
+| bootstrap\_tf\_gcs\_bucket | GCS bucket used for the bootstrap Terraform state. |
+| environment\_folders | Top-level environment folders. |
+| environment\_service\_account\_keys | Service account keys used to run each environment Terraform modules. |
+| environment\_service\_accounts | Service accounts used to run each environment Terraform modules. |
+| environment\_tf\_gcs\_buckets | GCS buckets used for each environment Terraform state. |
+| shared\_resources\_project | Project that holdes resources shared across environments. |
+| terraform\_project | Project that holds the base Terraform resources. |
+
+<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

organization-bootstrap/environments/backend.tf.sample

@@ -0,0 +1,23 @@
+# Copyright 2019 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+terraform {
+  backend "gcs" {
+    # once initial apply has completed, copy this file to `backend.tf` then
+    # set the `bucket` value to the `bootstrap_tf_gcs_bucket` output, then
+    # run apply again to transfer state
+    bucket = ""
+  }
+}

organization-bootstrap/environments/main.tf

@@ -0,0 +1,160 @@
+# Copyright 2019 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+###############################################################################
+#                        Terraform top-level resources                        #
+###############################################################################
+
+# Terraform project
+
+module "project-tf" {
+  source          = "terraform-google-modules/project-factory/google//modules/fabric-project"
+  version         = "3.2.0"
+  parent          = var.root_node
+  billing_account = var.billing_account_id
+  prefix          = var.prefix
+  name            = "terraform"
+  lien_reason     = "terraform"
+  owners          = var.terraform_owners
+  activate_apis   = var.project_services
+}
+
+# per-environment service accounts
+
+module "service-accounts-tf-environments" {
+  source             = "terraform-google-modules/service-accounts/google"
+  version            = "2.0.0"
+  project_id         = module.project-tf.project_id
+  org_id             = var.organization_id
+  billing_account_id = var.billing_account_id
+  prefix             = var.prefix
+  names              = var.environments
+  grant_billing_role = true
+  grant_xpn_roles    = var.grant_xpn_roles
+  generate_keys      = var.generate_service_account_keys
+}
+
+# bootstrap Terraform state GCS bucket
+
+module "gcs-tf-bootstrap" {
+  source     = "terraform-google-modules/cloud-storage/google"
+  version    = "1.0.0"
+  project_id = module.project-tf.project_id
+  prefix     = "${var.prefix}-tf"
+  names      = ["tf-bootstrap"]
+  location   = var.gcs_location
+}
+
+# per-environment Terraform state GCS buckets
+
+module "gcs-tf-environments" {
+  source          = "terraform-google-modules/cloud-storage/google"
+  version         = "1.0.0"
+  project_id      = module.project-tf.project_id
+  prefix          = "${var.prefix}-tf"
+  names           = var.environments
+  location        = var.gcs_location
+  set_admin_roles = true
+  bucket_admins = zipmap(
+    var.environments,
+    module.service-accounts-tf-environments.iam_emails_list
+  )
+}
+
+###############################################################################
+#                              Top-level folders                              #
+###############################################################################
+
+# TODO(ludomagno): move XPN admin role here after checking it now works on folders
+
+module "folders-top-level" {
+  source            = "terraform-google-modules/folders/google"
+  version           = "2.0.0"
+  parent            = var.root_node
+  names             = var.environments
+  set_roles         = true
+  per_folder_admins = module.service-accounts-tf-environments.iam_emails_list
+  folder_admin_roles = [
+    "roles/resourcemanager.folderViewer",
+    "roles/resourcemanager.projectCreator",
+    "roles/owner",
+    "roles/compute.networkAdmin",
+  ]
+}
+
+###############################################################################
+#                              Audit log exports                              #
+###############################################################################
+
+# audit logs project
+
+module "project-audit" {
+  source          = "terraform-google-modules/project-factory/google//modules/fabric-project"
+  version         = "3.2.0"
+  parent          = var.root_node
+  billing_account = var.billing_account_id
+  prefix          = var.prefix
+  name            = "audit"
+  lien_reason     = "audit"
+  activate_apis   = var.project_services
+  viewers         = var.audit_viewers
+}
+
+# audit logs destination on BigQuery
+
+module "bq-audit-export" {
+  source                   = "terraform-google-modules/log-export/google//modules/bigquery"
+  version                  = "3.0.0"
+  project_id               = module.project-audit.project_id
+  dataset_name             = "logs_audit_${replace(var.environments[0], "-", "_")}"
+  log_sink_writer_identity = module.log-sink-audit.writer_identity
+}
+
+# audit log sink
+# set the organization as parent to export audit logs for all environments
+
+module "log-sink-audit" {
+  source                 = "terraform-google-modules/log-export/google"
+  version                = "3.0.0"
+  filter                 = "logName: \"/logs/cloudaudit.googleapis.com%2Factivity\" OR logName: \"/logs/cloudaudit.googleapis.com%2Fsystem_event\""
+  log_sink_name          = "logs-audit-${var.environments[0]}"
+  parent_resource_type   = "folder"
+  parent_resource_id     = split("/", module.folders-top-level.ids_list[0])[1]
+  include_children       = "true"
+  unique_writer_identity = "true"
+  destination_uri        = "${module.bq-audit-export.destination_uri}"
+}
+
+###############################################################################
+#                    Shared resources (GCR, GCS, KMS, etc.)                   #
+###############################################################################
+
+# shared resources project
+# see the README file for additional options on managing shared services
+
+module "project-shared-resources" {
+  source                 = "terraform-google-modules/project-factory/google//modules/fabric-project"
+  version                = "3.2.0"
+  parent                 = var.root_node
+  billing_account        = var.billing_account_id
+  prefix                 = var.prefix
+  name                   = "shared"
+  lien_reason            = "shared"
+  activate_apis          = var.project_services
+  extra_bindings_roles   = var.shared_bindings_roles
+  extra_bindings_members = var.shared_bindings_members
+}
+
+# Add further modules here for resources that are common to all environments
+# like GCS buckets (used to hold shared assets), Container Registry, KMS, etc.

organization-bootstrap/environments/outputs.tf

@@ -0,0 +1,62 @@
+# Copyright 2019 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+output "terraform_project" {
+  description = "Project that holds the base Terraform resources."
+  value       = module.project-tf.project_id
+}
+
+output "bootstrap_tf_gcs_bucket" {
+  description = "GCS bucket used for the bootstrap Terraform state."
+  value       = module.gcs-tf-bootstrap.name
+}
+
+output "environment_folders" {
+  description = "Top-level environment folders."
+  value       = module.folders-top-level.ids
+}
+
+output "environment_tf_gcs_buckets" {
+  description = "GCS buckets used for each environment Terraform state."
+  value       = module.gcs-tf-environments.names
+}
+
+output "environment_service_account_keys" {
+  description = "Service account keys used to run each environment Terraform modules."
+  sensitive   = true
+  value       = module.service-accounts-tf-environments.keys
+}
+
+output "environment_service_accounts" {
+  description = "Service accounts used to run each environment Terraform modules."
+  value       = module.service-accounts-tf-environments.emails
+}
+
+output "audit_logs_bq_dataset" {
+  description = "Bigquery dataset for the audit logs export."
+  value       = module.bq-audit-export.resource_name
+}
+
+output "audit_logs_project" {
+  description = "Project that holds the audit logs export resources."
+  value       = module.project-audit.project_id
+}
+
+output "shared_resources_project" {
+  description = "Project that holdes resources shared across environments."
+  value       = module.project-shared-resources.project_id
+}
+
+# Add further outputs here for the additional modules that manage shared
+# resources, like GCR, GCS buckets, KMS, etc.

organization-bootstrap/environments/providers.tf

@@ -0,0 +1 @@
+provider "google" {}

organization-bootstrap/environments/variables.tf

@@ -0,0 +1,85 @@
+variable "audit_viewers" {
+  description = "Audit project viewers, in IAM format."
+  default     = []
+}
+
+variable "billing_account_id" {
+  description = "Billing account id used as default for new projects."
+  type        = string
+}
+
+variable "environments" {
+  description = "Environment short names."
+  type        = list(string)
+}
+
+variable "generate_service_account_keys" {
+  description = "Generate and store service account keys in the state file."
+  default     = false
+}
+
+variable "gcs_location" {
+  description = "GCS bucket location."
+  default     = "EU"
+}
+
+variable "grant_xpn_roles" {
+  description = "Grant roles needed for Shared VPC creation to service accounts."
+  default     = true
+}
+
+
+variable "organization_id" {
+  description = "Organization id."
+  type        = string
+}
+
+variable "prefix" {
+  description = "Prefix used for resources that need unique names."
+  type        = string
+}
+
+variable "root_node" {
+  description = "Root node for the new hierarchy, either 'organizations/org_id' or 'folders/folder_id'."
+  type        = string
+}
+
+variable "shared_bindings_members" {
+  description = "List of comma-delimited IAM-format members for the additional shared project bindings."
+  # example: ["user:a@example.com,b@example.com", "user:c@example.com"]
+  default = []
+}
+variable "shared_bindings_roles" {
+  description = "List of roles for additional shared project bindings."
+  # example: ["roles/storage.objectViewer", "roles/storage.admin"]
+  default = []
+}
+
+variable "terraform_owners" {
+  description = "Terraform project owners, in IAM format."
+  default     = []
+}
+
+variable "project_services" {
+  description = "Service APIs enabled by default in new projects."
+  default = [
+    "bigquery-json.googleapis.com",
+    "bigquerystorage.googleapis.com",
+    "cloudbilling.googleapis.com",
+    "cloudresourcemanager.googleapis.com",
+    "compute.googleapis.com",
+    "container.googleapis.com",
+    "containerregistry.googleapis.com",
+    "deploymentmanager.googleapis.com",
+    "iam.googleapis.com",
+    "iamcredentials.googleapis.com",
+    "logging.googleapis.com",
+    "oslogin.googleapis.com",
+    "pubsub.googleapis.com",
+    "replicapool.googleapis.com",
+    "replicapoolupdater.googleapis.com",
+    "resourceviews.googleapis.com",
+    "serviceusage.googleapis.com",
+    "storage-api.googleapis.com",
+  ]
+}

organization-bootstrap/environments/versions.tf

@@ -0,0 +1,4 @@
+
+terraform {
+  required_version = ">= 0.12"
+}