Import Fast from dev repository.
> > Co-authored-by: Julio Castillo <jccb@google.com> Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com> Co-authored-by: Simone Ruffilli <sruffilli@google.com>
This commit is contained in:
Simone Ruffilli
parent
9226024eb9
commit
55a8aca99f
|
|
@ -0,0 +1,6 @@
|
|||
# Project factory
|
||||
|
||||
The Project Factory (or PF) builds on top of your foundations to create and setup projects (and related resources) to be used for your workloads.
|
||||
It is organized in folders representing enviroments (e.g. "dev", "prod"), each implemented by a stand-alone terraform [resource factory](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c).
|
||||
|
||||
This directory contains a single project factory ([`prod/`](./prod/)) as an example - in order to implement multiple environments (e.g. "prod" and "dev") you'll need to copy the `prod` folder into one folder per environment, then customize each one following the instructions found in [`prod/README.md`](./prod/README.md).
|
||||
|
|
@ -0,0 +1,130 @@
|
|||
# Project factory
|
||||
|
||||
The Project Factory (or PF) builds on top of your foundations to create and set up projects (and related resources) to be used for your workloads.
|
||||
It is organized in folders representing environments (e.g. "dev", "prod"), each implemented by a stand-alone terraform [resource factory](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c).
|
||||
|
||||
## Design overview and choices
|
||||
|
||||
|
||||
|
||||
A single factory creates projects in a well-defined context, according to your resource management structure. In the diagram above, each Team is structured to have specific folders projects for a given environment, such as Production and Development, per the resource management structure configured in stage `01-resman`.
|
||||
|
||||
Projects for each environment across different teams are created by dedicated service accounts, as exemplified in the diagram above. While there's no intrinsic limitation regarding where the project factory can create a given project, the IAM bindings for the service account effectively enforce boundaries (e.g. the production service account shouldn't be able to create or have any access to the development projects, and vice versa).
|
||||
|
||||
The project factory takes care of the following activities:
|
||||
|
||||
* Project creation
|
||||
* API/Services enablement
|
||||
* Service accounts creation
|
||||
* IAM roles assignment for groups and service accounts
|
||||
* KMS keys roles assignment
|
||||
* Shared VPC attachment and subnets IAM binding
|
||||
* DNS zones creation and visibility configuration
|
||||
* Project-level org policies definition
|
||||
* Billing setup (billing account attachment and budget configuration)
|
||||
* Essential contacts definition (for [budget alerts](https://cloud.google.com/billing/docs/how-to/budgets) and [important notifications](https://cloud.google.com/resource-manager/docs/managing-notification-contacts?hl=en))
|
||||
|
||||
|
||||
## How to run this stage
|
||||
|
||||
This stage is meant to be executed after "foundational stages" (i.e. stages [`00-bootstrap`](../../00-bootstrap), [`01-resman`](../../01-resman), [`02-networking`](../../02-networking) and [`02-security`](../../02-security)) have been run.
|
||||
|
||||
It's of course possible to run this stage in isolation, by making sure the architectural prerequisites are satisfied (e.g. networking), and that the Service Account running the stage is granted the roles/permissions below:
|
||||
|
||||
* One service account per environment, each with appropriate permissions
|
||||
* at the organization level a custom role for networking operations including the following permissions
|
||||
* `"compute.organizations.enableXpnResource"`,
|
||||
* `"compute.organizations.disableXpnResource"`,
|
||||
* `"compute.subnetworks.setIamPolicy"`,
|
||||
* `"dns.networks.bindPrivateDNSZone"`
|
||||
* and role `"roles/orgpolicy.policyAdmin"`
|
||||
* on each folder where projects will be created
|
||||
* `"roles/logging.admin"`
|
||||
* `"roles/owner"`
|
||||
* `"roles/resourcemanager.folderAdmin"`
|
||||
* `"roles/resourcemanager.projectCreator"`
|
||||
* on the host project for the Shared VPC
|
||||
* `"roles/browser"`
|
||||
* `"roles/compute.viewer"`
|
||||
* `"roles/dns.admin"`
|
||||
* If networking is to be used (e.g. for VMs, GKE Clusters or AppEngine flex), VPC Host projects and their subnets should exist when creating projects
|
||||
* If per-environment DNS sub-zones are required, one "root" zone per environment should exist when creating projects (e.g. prod.gcp.example.com.)
|
||||
|
||||
### Providers configuration
|
||||
|
||||
If you're running this on top of FAST, you should run the following commands to create the provider file, and populate the required variables from the previous stage.
|
||||
|
||||
```bash
|
||||
# Variable `outputs_location` is set to `../../configs/example` in stage 01-resman
|
||||
$ cd fabric-fast/stages/03-project-factory/prod
|
||||
ln -s ../../../configs/example/03-project-factory-prod/providers.tf
|
||||
```
|
||||
|
||||
### Variable configuration
|
||||
|
||||
There are two broad sets of variables you will need to fill in:
|
||||
|
||||
- variables shared by other stages (org id, billing account id, etc.), or derived from a resource managed by a different stage (folder id, automation project id, etc.)
|
||||
- variables specific to resources managed by this stage
|
||||
|
||||
To avoid the tedious job of filling in the first group of variables with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files.
|
||||
|
||||
If you configured a valid path for `outputs_location` in the bootstrap and networking stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's outputs folder (under the path you specified), where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is available:
|
||||
|
||||
```bash
|
||||
# Variable `outputs_location` is set to `../../configs/example` in stages 01-bootstrap and 02-networking
|
||||
ln -s ../../../configs/example/03-project-factory-prod/terraform-bootstrap.auto.tfvars.json
|
||||
ln -s ../../../configs/example/03-project-factory-prod/terraform-networking.auto.tfvars.json
|
||||
```
|
||||
|
||||
If you're not running on top of fast, refer to the [Variables](#variables) table at the bottom of this document for a full list of variables, their origin (e.g. a stage or specific to this one), and descriptions explaining their meaning.
|
||||
|
||||
|
||||
Besides the values above, a project factory takes 2 additional inputs:
|
||||
|
||||
|
||||
* `data/defaults.yaml`, manually configured by adapting the [`prod/data/defaults.yaml.sample`](./prod/data/defaults.yaml.sample), which defines per-environment default values e.g. for billing alerts and labels.
|
||||
|
||||
* `data/projects/*.yaml`, one file per project (optionally grouped in folders), which configures each project. A [`prod/data/projects/project.yaml.sample`](./prod/data/projects/project.yaml.sample) is provided as reference and documentation for the schema. Projects will be named after the filename, e.g. `fast-prod-lab0.yaml` will generate project `fast-prod-lab0`.
|
||||
|
||||
Once the configuration is complete, the project factory can be run with the usual
|
||||
|
||||
```bash
|
||||
terraform init
|
||||
terraform apply
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- BEGIN TFDOC -->
|
||||
|
||||
## Files
|
||||
|
||||
| name | description | modules | resources |
|
||||
| ------------------------------ | ----------------- | ---------------------------- | --------- |
|
||||
| [main.tf](./main.tf) | Project factory. | <code>project-factory</code> | |
|
||||
| [outputs.tf](./outputs.tf) | Module outputs. | | |
|
||||
| [variables.tf](./variables.tf) | Module variables. | | |
|
||||
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default | producer |
|
||||
| -------------------- | --------------------------------------------------------------------- | :-----------------: | :------: | :-------------------------------------------: | :------------------------: |
|
||||
| billing_account_id | Billing account id. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
|
||||
| shared_vpc_self_link | Self link for the shared VPC. | <code>string</code> | ✓ | | <code>02-networking</code> |
|
||||
| vpc_host_project | Host project for the shared VPC. | <code>string</code> | ✓ | | <code>02-networking</code> |
|
||||
| data_dir | Relative path for the folder storing configuration data. | <code>string</code> | | <code>"data/projects"</code> | |
|
||||
| defaults_file | Relative path for the file storing the project factory configuration. | <code>string</code> | | <code>"data/defaults.yaml"</code> | |
|
||||
| environment_dns_zone | DNS zone suffix for environment. | <code>string</code> | | <code>null</code> | <code>02-networking</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive | consumers |
|
||||
| -------- | -------------------------------------- | :-------: | --------- |
|
||||
| projects | Created projects and service accounts. | | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
billing_account_id: 012345-67890A-BCDEF0
|
||||
|
||||
# [opt] Setup for billing alerts
|
||||
billing_alert:
|
||||
amount: 1000
|
||||
thresholds:
|
||||
current: [0.5, 0.8]
|
||||
forecasted: [0.5, 0.8]
|
||||
credit_treatment: INCLUDE_ALL_CREDITS
|
||||
|
||||
# [opt] Contacts for billing alerts and important notifications
|
||||
essential_contacts: ["team-contacts@example.com"]
|
||||
|
||||
# [opt] Labels set for all projects
|
||||
labels:
|
||||
environment: prod
|
||||
department: accounting
|
||||
application: example-app
|
||||
foo: bar
|
||||
|
||||
# [opt] Additional notification channels for billing
|
||||
notification_channels: []
|
||||
|
|
@ -0,0 +1,99 @@
|
|||
# [opt] Billing account id - overrides default if set
|
||||
billing_account_id: 012345-67890A-BCDEF0
|
||||
|
||||
# [opt] Billing alerts config - overrides default if set
|
||||
billing_alert:
|
||||
amount: 10
|
||||
thresholds:
|
||||
current:
|
||||
- 0.5
|
||||
- 0.8
|
||||
forecasted: []
|
||||
|
||||
# [opt] DNS zones to be created as children of the environment_dns_zone defined in defaults
|
||||
dns_zones:
|
||||
- lorem
|
||||
- ipsum
|
||||
|
||||
# [opt] Contacts for billing alerts and important notifications
|
||||
essential_contacts:
|
||||
- team-a-contacts@example.com
|
||||
|
||||
# Folder the project will be created as children of
|
||||
folder_id: folders/012345678901
|
||||
|
||||
# [opt] Authoritative IAM bindings in group => [roles] format
|
||||
group_iam:
|
||||
test-team-foobar@fast-lab-0.gcp-pso-italy.net:
|
||||
- roles/compute.admin
|
||||
|
||||
# [opt] Authoritative IAM bindings in role => [principals] format
|
||||
# Generally used to grant roles to service accounts external to the project
|
||||
iam:
|
||||
roles/compute.admin:
|
||||
- serviceAccount:service-account
|
||||
|
||||
# [opt] Service robots and keys they will be assigned as cryptoKeyEncrypterDecrypter
|
||||
# in service => [keys] format
|
||||
kms_service_agents:
|
||||
compute: [key1, key2]
|
||||
storage: [key1, key2]
|
||||
|
||||
# [opt] Labels for the project - merged with the ones defined in defaults
|
||||
labels:
|
||||
environment: prod
|
||||
|
||||
# [opt] Org policy overrides defined at project level
|
||||
org_policies:
|
||||
policy_boolean:
|
||||
constraints/compute.disableGuestAttributesAccess: true
|
||||
policy_list:
|
||||
constraints/compute.trustedImageProjects:
|
||||
inherit_from_parent: null
|
||||
status: true
|
||||
suggested_value: null
|
||||
values:
|
||||
- projects/fast-prod-iac-core-0
|
||||
|
||||
# [opt] Service account to create for the project and their roles on the project
|
||||
# in name => [roles] format
|
||||
service_accounts:
|
||||
another-service-account:
|
||||
- roles/compute.admin
|
||||
my-service-account:
|
||||
- roles/compute.admin
|
||||
|
||||
# [opt] APIs to enable on the project.
|
||||
services:
|
||||
- storage.googleapis.com
|
||||
- stackdriver.googleapis.com
|
||||
- compute.googleapis.com
|
||||
|
||||
# [opt] Roles to assign to the robots service accounts in robot => [roles] format
|
||||
services_iam:
|
||||
compute:
|
||||
- roles/storage.objectViewer
|
||||
|
||||
# [opt] VPC setup.
|
||||
# If set enables the `compute.googleapis.com` service and configures
|
||||
# service project attachment
|
||||
vpc:
|
||||
|
||||
# [opt] If set, enables the container API
|
||||
gke_setup:
|
||||
|
||||
# Grants "roles/container.hostServiceAgentUser" to the container robot if set
|
||||
enable_host_service_agent: false
|
||||
|
||||
# Grants "roles/compute.securityAdmin" to the container robot if set
|
||||
enable_security_admin: true
|
||||
|
||||
# Host project the project will be service project of
|
||||
host_project: fast-prod-net-spoke-0
|
||||
|
||||
# [opt] Subnets in the host project where principals will be granted networkUser
|
||||
# in region/subnet-name => [principals]
|
||||
subnets_iam:
|
||||
europe-west1/prod-default-ew1:
|
||||
- user:foobar@example.com
|
||||
- serviceAccount:service-account1
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 56 KiB |
|
|
@ -0,0 +1,57 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Project factory.
|
||||
|
||||
|
||||
locals {
|
||||
_defaults = yamldecode(file(var.defaults_file))
|
||||
_defaults_net = {
|
||||
billing_account_id = var.billing_account_id
|
||||
environment_dns_zone = var.environment_dns_zone
|
||||
shared_vpc_self_link = var.shared_vpc_self_link
|
||||
vpc_host_project = var.vpc_host_project
|
||||
}
|
||||
defaults = merge(local._defaults, local._defaults_net)
|
||||
projects = {
|
||||
for f in fileset("${var.data_dir}", "**/*.yaml") :
|
||||
trimsuffix(f, ".yaml") => yamldecode(file("${var.data_dir}/${f}"))
|
||||
}
|
||||
}
|
||||
|
||||
module "projects" {
|
||||
#TODO(sruffilli): Pin to release
|
||||
source = "github.com/terraform-google-modules/cloud-foundation-fabric/examples/factories/project-factory"
|
||||
for_each = local.projects
|
||||
defaults = local.defaults
|
||||
project_id = each.key
|
||||
billing_account_id = try(each.value.billing_account_id, null)
|
||||
billing_alert = try(each.value.billing_alert, null)
|
||||
dns_zones = try(each.value.dns_zones, [])
|
||||
essential_contacts = try(each.value.essential_contacts, [])
|
||||
folder_id = each.value.folder_id
|
||||
group_iam = try(each.value.group_iam, {})
|
||||
iam = try(each.value.iam, {})
|
||||
kms_service_agents = try(each.value.kms, {})
|
||||
labels = try(each.value.labels, {})
|
||||
org_policies = try(each.value.org_policies, null)
|
||||
service_accounts = try(each.value.service_accounts, {})
|
||||
services = try(each.value.services, [])
|
||||
services_iam = try(each.value.services_iam, {})
|
||||
vpc = try(each.value.vpc, null)
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
output "projects" {
|
||||
description = "Created projects and service accounts."
|
||||
value = module.projects
|
||||
}
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
#TODO: tfdoc annotations
|
||||
|
||||
variable "billing_account_id" {
|
||||
# tfdoc:variable:source 00-bootstrap
|
||||
description = "Billing account id."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "data_dir" {
|
||||
description = "Relative path for the folder storing configuration data."
|
||||
type = string
|
||||
default = "data/projects"
|
||||
}
|
||||
|
||||
variable "environment_dns_zone" {
|
||||
# tfdoc:variable:source 02-networking
|
||||
description = "DNS zone suffix for environment."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "defaults_file" {
|
||||
description = "Relative path for the file storing the project factory configuration."
|
||||
type = string
|
||||
default = "data/defaults.yaml"
|
||||
}
|
||||
|
||||
variable "shared_vpc_self_link" {
|
||||
# tfdoc:variable:source 02-networking
|
||||
description = "Self link for the shared VPC."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "vpc_host_project" {
|
||||
# tfdoc:variable:source 02-networking
|
||||
description = "Host project for the shared VPC."
|
||||
type = string
|
||||
}
|
||||
Loading…
Reference in New Issue