diff --git a/modules/project-factory/README.md b/modules/project-factory/README.md
index e38e411d..e93fc6d0 100644
--- a/modules/project-factory/README.md
+++ b/modules/project-factory/README.md
@@ -23,10 +23,12 @@ The code is meant to be executed by a high level service accounts with powerful
- [Folder hierarchy](#folder-hierarchy)
- [Projects](#projects)
- - [Leveraging project defaults, merges, optionals](#leveraging-project-defaults-merges-optionals)
+ - [Factory-wide project defaults, merges, optionals](#factory-wide-project-defaults-merges-optionals)
- [Service accounts](#service-accounts)
+ - [Automation project and resources](#automation-project-and-resources)
- [Billing budgets](#billing-budgets)
- [Example](#example)
+- [Files](#files)
- [Variables](#variables)
- [Outputs](#outputs)
- [Tests](#tests)
@@ -51,7 +53,7 @@ Refer to the [example](#example) below for actual examples of the YAML definitio
The project factory is configured via the `factories_config.projects_data_path` variable, and project files are also read from the hierarchy describe in the previous section when enabled. The YAML format mirrors the project module, refer to the [example](#example) below for actual examples of the YAML definitions.
-### Leveraging project defaults, merges, optionals
+### Factory-wide project defaults, merges, optionals
In addition to the YAML-based project configurations, the factory accepts three additional sets of inputs via Terraform variables:
@@ -81,6 +83,54 @@ service_accounts:
Both the `display_name` and `iam_self_roles` attributes are optional.
+### Automation project and resources
+
+Project configurations also support defining service accounts and storage buckets to support automation, created in a separate controlling project so as to be outside of the sphere of control of the managed project.
+
+Automation resources are defined via the `automation` attribute in project configurations, which supports:
+
+- a mandatory `project` attribute to define the external controlling project
+- an optional `service_accounts` list where each element will define a service account in the controlling project
+- an optional `buckets` map where each key will define a bucket in the controlling project, and the map of roles/principals in the corresponding value assigned on the created bucket; principals can refer to the created service accounts by key
+
+Service accounts and buckets will be prefixed with the project name, and use the key specified in the YAML file as a suffix.
+
+```yaml
+# file name: prod-app-example-0
+# prefix via factory defaults: foo
+# project id: foo-prod-app-example-0
+billing_account: 012345-67890A-BCDEF0
+parent: folders/12345678
+services:
+ - compute.googleapis.com
+ - stackdriver.googleapis.com
+iam:
+ roles/owner:
+ - rw
+ roles/viewer:
+ - ro
+automation:
+ project: foo-prod-iac-core-0
+ service_accounts:
+ # sa name: foo-prod-app-example-0-rw
+ rw:
+ description: Read/write automation sa for app example 0.
+ # sa name: foo-prod-app-example-0-ro
+ ro:
+ description: Read-only automation sa for app example 0.
+ buckets:
+ # bucket name: foo-prod-app-example-0-state
+ state:
+ description: Terraform state bucket for app example 0.
+ iam:
+ roles/storage.objectCreator:
+ - rw
+ roles/storage.objectViewer:
+ - rw
+ - ro
+ - group:devops@example.org
+```
+
## Billing budgets
The billing budgets factory integrates the `[`billing-account`](../billing-account/) module functionality, and adds support for easy referencing budgets in project files.
@@ -102,7 +152,7 @@ billing_budgets:
- test-100
```
-The example below shows how to use the billing budgets factory.
+A simple billing budget example is show in the [example](#example) below.
## Example
@@ -155,7 +205,7 @@ module "project-factory" {
projects_data_path = "data/projects"
}
}
-# tftest modules=13 resources=48 files=prj-app-1,prj-app-2,prj-app-3,budget-test-100,h-0-0,h-1-0,h-0-1,h-1-1,h-1-1-p0 inventory=example.yaml
+# tftest modules=16 resources=55 files=prj-app-1,prj-app-2,prj-app-3,budget-test-100,h-0-0,h-1-0,h-0-1,h-1-1,h-1-1-p0 inventory=example.yaml
```
A simple hierarchy of folders:
@@ -191,7 +241,7 @@ billing_account: 012345-67890A-BCDEF0
services:
- container.googleapis.com
- storage.googleapis.com
-# tftest-file id=h-1-1-p0 path=data/hierarchy/bar/baz/bar-baz-0.yaml
+# tftest-file id=h-1-1-p0 path=data/hierarchy/bar/baz/bar-baz-iac-0.yaml
```
More traditional project definitions via the project factory data:
@@ -264,12 +314,36 @@ shared_vpc_service_config:
# tftest-file id=prj-app-2 path=data/projects/prj-app-2.yaml
```
+This project uses a reference to a hierarchy folder, and defines a controlling project via the `automation` attributes:
+
```yaml
-# project app-3
-parent: folders/12345678
+parent: bar/baz
services:
- run.googleapis.com
- storage.googleapis.com
+iam:
+ "roles/owner":
+ - rw
+ "roles/viewer":
+ - ro
+automation:
+ project: bar-baz-iac-0
+ service_accounts:
+ rw:
+ description: Read/write automation sa for app example 0.
+ ro:
+ description: Read-only automation sa for app example 0.
+ buckets:
+ state:
+ description: Terraform state bucket for app example 0.
+ iam:
+ roles/storage.objectCreator:
+ - rw
+ roles/storage.objectViewer:
+ - rw
+ - ro
+ - group:devops@example.org
+
# tftest-file id=prj-app-3 path=data/projects/prj-app-3.yaml
```
@@ -297,7 +371,21 @@ update_rules:
# tftest-file id=budget-test-100 path=data/budgets/test-100.yaml
```
+
+## Files
+
+| name | description | modules |
+|---|---|---|
+| [automation.tf](./automation.tf) | Automation projects locals and resources. | gcs
· iam-service-account
|
+| [factory-budgets.tf](./factory-budgets.tf) | Billing budget factory locals. | |
+| [factory-folders.tf](./factory-folders.tf) | Folder hierarchy factory locals. | |
+| [factory-projects.tf](./factory-projects.tf) | Projects factory locals. | |
+| [folders.tf](./folders.tf) | Folder hierarchy factory resources. | folder
|
+| [main.tf](./main.tf) | Projects and billing budgets factory resources. | billing-account
· iam-service-account
· project
|
+| [outputs.tf](./outputs.tf) | Module outputs. | |
+| [variables.tf](./variables.tf) | Module variables. | |
+
## Variables
| name | description | type | required | default |
diff --git a/modules/project-factory/automation.tf b/modules/project-factory/automation.tf
new file mode 100644
index 00000000..104739f7
--- /dev/null
+++ b/modules/project-factory/automation.tf
@@ -0,0 +1,109 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+# tfdoc:file:description Automation projects locals and resources.
+
+locals {
+ automation_buckets = flatten([
+ for k, v in local.projects : [
+ for ks, kv in try(v.automation.buckets, {}) : merge(kv, {
+ automation_project = v.automation.project
+ name = ks
+ prefix = v.prefix
+ project = k
+ })
+ ]
+ ])
+ automation_sa = flatten([
+ for k, v in local.projects : [
+ for ks, kv in try(v.automation.service_accounts, {}) : merge(kv, {
+ automation_project = v.automation.project
+ name = ks
+ prefix = v.prefix
+ project = k
+ })
+ ]
+ ])
+}
+
+module "automation-buckets" {
+ source = "../gcs"
+ for_each = {
+ for k in local.automation_buckets : "${k.project}/${k.name}" => k
+ }
+ project_id = each.value.automation_project
+ prefix = each.value.prefix
+ name = "${each.value.project}-${each.value.name}"
+ encryption_key = lookup(each.value, "encryption_key", null)
+ # try interpolating service accounts by key in principals
+ iam = {
+ for k, v in lookup(each.value, "iam", {}) : k => [
+ for vv in v : try(
+ module.automation-service-accounts["${each.value.project}/${vv}"].iam_email,
+ vv
+ )
+ ]
+ }
+ iam_bindings = {
+ for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, {
+ members = [
+ for vv in v.members : try(
+ module.automation-service-accounts["${each.value.project}/${vv}"].iam_email,
+ vv
+ )
+ ]
+ })
+ }
+ iam_bindings_additive = {
+ for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, {
+ member = try(
+ module.automation-service-accounts["${each.value.project}/${v.member}"].iam_email,
+ v.member
+ )
+ })
+ }
+ labels = lookup(each.value, "labels", {})
+ location = lookup(each.value, "location", "EU")
+ storage_class = lookup(each.value, "storage_class", "MULTI_REGIONAL")
+ uniform_bucket_level_access = lookup(each.value, "uniform_bucket_level_access", true)
+ versioning = lookup(each.value, "versioning", false)
+}
+
+module "automation-service-accounts" {
+ source = "../iam-service-account"
+ for_each = {
+ for k in local.automation_sa : "${k.project}/${k.name}" => k
+ }
+ project_id = each.value.automation_project
+ prefix = each.value.prefix
+ name = "${each.value.project}-${each.value.name}"
+ description = lookup(each.value, "description", null)
+ display_name = lookup(
+ each.value,
+ "display_name",
+ "Service account ${each.value.name} for ${each.value.project}."
+ )
+ iam = lookup(each.value, "iam", {})
+ iam_bindings = lookup(each.value, "iam_bindings", {})
+ iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {})
+ iam_billing_roles = lookup(each.value, "iam_billing_roles", {})
+ iam_folder_roles = lookup(each.value, "iam_folder_roles", {})
+ iam_organization_roles = lookup(each.value, "iam_organization_roles", {})
+ iam_project_roles = lookup(each.value, "iam_project_roles", {})
+ iam_sa_roles = lookup(each.value, "iam_sa_roles", {})
+ # we don't interpolate buckets here as we can't use a dynamic key
+ iam_storage_roles = lookup(each.value, "iam_storage_roles", {})
+}
diff --git a/modules/project-factory/factory-budgets.tf b/modules/project-factory/factory-budgets.tf
index 228eb2c8..82f4b6bd 100644
--- a/modules/project-factory/factory-budgets.tf
+++ b/modules/project-factory/factory-budgets.tf
@@ -14,6 +14,8 @@
* limitations under the License.
*/
+# tfdoc:file:description Billing budget factory locals.
+
locals {
# reimplement the billing account factory here to interpolate projects
_budget_path = try(pathexpand(var.factories_config.budgets.budgets_data_path), null)
diff --git a/modules/project-factory/factory-folders.tf b/modules/project-factory/factory-folders.tf
index 99e91a47..cd9537f4 100644
--- a/modules/project-factory/factory-folders.tf
+++ b/modules/project-factory/factory-folders.tf
@@ -14,6 +14,8 @@
* limitations under the License.
*/
+# tfdoc:file:description Folder hierarchy factory locals.
+
locals {
_folders_path = try(
pathexpand(var.factories_config.hierarchy.folders_data_path), null
@@ -51,53 +53,3 @@ check "hierarchy-data" {
error_message = "No default set for hierarchy parent ids."
}
}
-
-module "hierarchy-folder-lvl-1" {
- source = "../folder"
- for_each = { for k, v in local.folders : k => v if v.level == 1 }
- parent = try(
- # allow the YAML data to set the parent for this level
- lookup(
- var.factories_config.hierarchy.parent_ids,
- each.value.parent,
- # use the value as is if it's not in the parents map
- each.value.parent
- ),
- # use the default value in the initial parents map
- var.factories_config.hierarchy.parent_ids.default
- # fail if we don't have an explicit parent
- )
- name = each.value.name
- iam = lookup(each.value, "iam", {})
- iam_bindings = lookup(each.value, "iam_bindings", {})
- iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {})
- iam_by_principals = lookup(each.value, "iam_by_principals", {})
- org_policies = lookup(each.value, "org_policies", {})
- tag_bindings = lookup(each.value, "tag_bindings", {})
-}
-
-module "hierarchy-folder-lvl-2" {
- source = "../folder"
- for_each = { for k, v in local.folders : k => v if v.level == 2 }
- parent = module.hierarchy-folder-lvl-1[each.value.parent_key].id
- name = each.value.name
- iam = lookup(each.value, "iam", {})
- iam_bindings = lookup(each.value, "iam_bindings", {})
- iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {})
- iam_by_principals = lookup(each.value, "iam_by_principals", {})
- org_policies = lookup(each.value, "org_policies", {})
- tag_bindings = lookup(each.value, "tag_bindings", {})
-}
-
-module "hierarchy-folder-lvl-3" {
- source = "../folder"
- for_each = { for k, v in local.folders : k => v if v.level == 3 }
- parent = module.hierarchy-folder-lvl-2[each.value.parent_key].id
- name = each.value.name
- iam = lookup(each.value, "iam", {})
- iam_bindings = lookup(each.value, "iam_bindings", {})
- iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {})
- iam_by_principals = lookup(each.value, "iam_by_principals", {})
- org_policies = lookup(each.value, "org_policies", {})
- tag_bindings = lookup(each.value, "tag_bindings", {})
-}
diff --git a/modules/project-factory/factory-projects.tf b/modules/project-factory/factory-projects.tf
index 1c5c2f19..aef73ef0 100644
--- a/modules/project-factory/factory-projects.tf
+++ b/modules/project-factory/factory-projects.tf
@@ -13,6 +13,9 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+
+# tfdoc:file:description Projects factory locals.
+
locals {
_hierarchy_projects = (
{
diff --git a/modules/project-factory/folders.tf b/modules/project-factory/folders.tf
new file mode 100644
index 00000000..a846ce27
--- /dev/null
+++ b/modules/project-factory/folders.tf
@@ -0,0 +1,67 @@
+/**
+ * Copyright 2024 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+# tfdoc:file:description Folder hierarchy factory resources.
+
+module "hierarchy-folder-lvl-1" {
+ source = "../folder"
+ for_each = { for k, v in local.folders : k => v if v.level == 1 }
+ parent = try(
+ # allow the YAML data to set the parent for this level
+ lookup(
+ var.factories_config.hierarchy.parent_ids,
+ each.value.parent,
+ # use the value as is if it's not in the parents map
+ each.value.parent
+ ),
+ # use the default value in the initial parents map
+ var.factories_config.hierarchy.parent_ids.default
+ # fail if we don't have an explicit parent
+ )
+ name = each.value.name
+ iam = lookup(each.value, "iam", {})
+ iam_bindings = lookup(each.value, "iam_bindings", {})
+ iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {})
+ iam_by_principals = lookup(each.value, "iam_by_principals", {})
+ org_policies = lookup(each.value, "org_policies", {})
+ tag_bindings = lookup(each.value, "tag_bindings", {})
+}
+
+module "hierarchy-folder-lvl-2" {
+ source = "../folder"
+ for_each = { for k, v in local.folders : k => v if v.level == 2 }
+ parent = module.hierarchy-folder-lvl-1[each.value.parent_key].id
+ name = each.value.name
+ iam = lookup(each.value, "iam", {})
+ iam_bindings = lookup(each.value, "iam_bindings", {})
+ iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {})
+ iam_by_principals = lookup(each.value, "iam_by_principals", {})
+ org_policies = lookup(each.value, "org_policies", {})
+ tag_bindings = lookup(each.value, "tag_bindings", {})
+}
+
+module "hierarchy-folder-lvl-3" {
+ source = "../folder"
+ for_each = { for k, v in local.folders : k => v if v.level == 3 }
+ parent = module.hierarchy-folder-lvl-2[each.value.parent_key].id
+ name = each.value.name
+ iam = lookup(each.value, "iam", {})
+ iam_bindings = lookup(each.value, "iam_bindings", {})
+ iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {})
+ iam_by_principals = lookup(each.value, "iam_by_principals", {})
+ org_policies = lookup(each.value, "org_policies", {})
+ tag_bindings = lookup(each.value, "tag_bindings", {})
+}
diff --git a/modules/project-factory/main.tf b/modules/project-factory/main.tf
index 2f440de6..0df66a7c 100644
--- a/modules/project-factory/main.tf
+++ b/modules/project-factory/main.tf
@@ -14,6 +14,8 @@
* limitations under the License.
*/
+# tfdoc:file:description Projects and billing budgets factory resources.
+
module "projects" {
source = "../project"
for_each = local.projects
@@ -31,10 +33,35 @@ module "projects" {
)
default_service_account = try(each.value.default_service_account, "keep")
descriptive_name = try(each.value.descriptive_name, null)
- iam = try(each.value.iam, {})
- iam_bindings = try(each.value.iam_bindings, {})
- iam_bindings_additive = try(each.value.iam_bindings_additive, {})
- iam_by_principals = try(each.value.iam_by_principals, {})
+ # IAM interpolates automation service accounts
+ iam = {
+ for k, v in lookup(each.value, "iam", {}) : k => [
+ for vv in v : try(
+ module.automation-service-accounts["${each.key}/${vv}"].iam_email,
+ vv
+ )
+ ]
+ }
+ iam_bindings = {
+ for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, {
+ members = [
+ for vv in v.members : try(
+ module.automation-service-accounts["${each.key}/${vv}"].iam_email,
+ vv
+ )
+ ]
+ })
+ }
+ iam_bindings_additive = {
+ for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, {
+ member = try(
+ module.automation-service-accounts["${each.key}/${v.member}"].iam_email,
+ v.member
+ )
+ })
+ }
+ # IAM principals would trigger dynamic key errors so we don't interpolate
+ iam_by_principals = try(each.value.iam_by_principals, {})
labels = merge(
each.value.labels, var.data_merges.labels
)
diff --git a/tests/modules/project_factory/examples/example.yaml b/tests/modules/project_factory/examples/example.yaml
index 3e14fc28..592f8e53 100644
--- a/tests/modules/project_factory/examples/example.yaml
+++ b/tests/modules/project_factory/examples/example.yaml
@@ -12,6 +12,107 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+values:
+ module.project-factory.module.automation-buckets["prj-app-3/state"].google_storage_bucket.bucket:
+ autoclass:
+ - enabled: false
+ cors: []
+ custom_placement_config: []
+ default_event_based_hold: null
+ enable_object_retention: null
+ encryption: []
+ force_destroy: false
+ labels: null
+ lifecycle_rule: []
+ location: EU
+ logging: []
+ name: test-pf-prj-app-3-state
+ project: bar-baz-iac-0
+ requester_pays: null
+ retention_policy: []
+ storage_class: MULTI_REGIONAL
+ timeouts: null
+ uniform_bucket_level_access: true
+ versioning:
+ - enabled: false
+ ? module.project-factory.module.automation-buckets["prj-app-3/state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectCreator"]
+ : bucket: test-pf-prj-app-3-state
+ condition: []
+ members:
+ - serviceAccount:test-pf-prj-app-3-rw@bar-baz-iac-0.iam.gserviceaccount.com
+ role: roles/storage.objectCreator
+ ? module.project-factory.module.automation-buckets["prj-app-3/state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]
+ : bucket: test-pf-prj-app-3-state
+ condition: []
+ members:
+ - group:devops@example.org
+ - serviceAccount:test-pf-prj-app-3-ro@bar-baz-iac-0.iam.gserviceaccount.com
+ - serviceAccount:test-pf-prj-app-3-rw@bar-baz-iac-0.iam.gserviceaccount.com
+ role: roles/storage.objectViewer
+ module.project-factory.module.automation-service-accounts["prj-app-3/ro"].google_service_account.service_account[0]:
+ account_id: test-pf-prj-app-3-ro
+ create_ignore_already_exists: null
+ description: Read-only automation sa for app example 0.
+ disabled: false
+ display_name: Service account ro for prj-app-3.
+ project: bar-baz-iac-0
+ timeouts: null
+ module.project-factory.module.automation-service-accounts["prj-app-3/rw"].google_service_account.service_account[0]:
+ account_id: test-pf-prj-app-3-rw
+ create_ignore_already_exists: null
+ description: Read/write automation sa for app example 0.
+ disabled: false
+ display_name: Service account rw for prj-app-3.
+ project: bar-baz-iac-0
+ timeouts: null
+ module.project-factory.module.hierarchy-folder-lvl-1["bar"].google_folder.folder[0]:
+ display_name: Bar (level 1)
+ parent: folders/4567890
+ timeouts: null
+ module.project-factory.module.hierarchy-folder-lvl-1["foo"].google_folder.folder[0]:
+ display_name: Foo (level 1)
+ parent: folders/12345678
+ timeouts: null
+ module.project-factory.module.hierarchy-folder-lvl-1["foo"].google_folder_iam_binding.authoritative["roles/viewer"]:
+ condition: []
+ members:
+ - group:a@example.com
+ role: roles/viewer
+ module.project-factory.module.hierarchy-folder-lvl-2["bar/baz"].google_folder.folder[0]:
+ display_name: Bar Baz (level 2)
+ timeouts: null
+ module.project-factory.module.hierarchy-folder-lvl-2["foo/baz"].google_folder.folder[0]:
+ display_name: Foo Baz (level 2)
+ timeouts: null
+ module.project-factory.module.projects["bar-baz-iac-0"].data.google_storage_project_service_account.gcs_sa[0]:
+ project: test-pf-bar-baz-iac-0
+ user_project: null
+ module.project-factory.module.projects["prj-app-3"].google_project.project[0]:
+ auto_create_network: false
+ billing_account: 123456-123456-123456
+ effective_labels:
+ environment: test
+ labels:
+ environment: test
+ name: test-pf-prj-app-3
+ project_id: test-pf-prj-app-3
+ skip_delete: false
+ terraform_labels:
+ environment: test
+ timeouts: null
+ module.project-factory.module.projects["prj-app-3"].google_project_iam_binding.authoritative["roles/owner"]:
+ condition: []
+ members:
+ - serviceAccount:test-pf-prj-app-3-rw@bar-baz-iac-0.iam.gserviceaccount.com
+ project: test-pf-prj-app-3
+ role: roles/owner
+ module.project-factory.module.projects["prj-app-3"].google_project_iam_binding.authoritative["roles/viewer"]:
+ condition: []
+ members:
+ - serviceAccount:test-pf-prj-app-3-ro@bar-baz-iac-0.iam.gserviceaccount.com
+ project: test-pf-prj-app-3
+ role: roles/viewer
+
counts:
google_billing_budget: 1
google_compute_shared_vpc_service_project: 1
@@ -23,9 +124,12 @@ counts:
google_monitoring_notification_channel: 1
google_org_policy_policy: 1
google_project: 4
+ google_project_iam_binding: 2
google_project_iam_member: 6
google_project_service: 14
- google_service_account: 3
+ google_service_account: 5
+ google_storage_bucket: 1
+ google_storage_bucket_iam_binding: 2
google_storage_project_service_account: 4
- modules: 13
- resources: 48
+ modules: 16
+ resources: 55