zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix groups. Fix tests.

This commit is contained in:
lcaggio 2023-02-01 08:55:33 +01:00
parent 053d4151de
commit 840fc86b3e
8 changed files with 23 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
blueprints/data-solutions/shielded-folder/README.md
View File

@ -39,7 +39,7 @@ This separation into projects allows adhering to the least-privilege principle b
User groups provide a stable frame of reference that allows decoupling the final set of permissions from the stage where entities and resources are created, and their IAM bindings are defined. User groups provide a stable frame of reference that allows decoupling the final set of permissions from the stage where entities and resources are created, and their IAM bindings are defined.
We use three groups to control access to resources: We use three groups to control access to resources:
- `data-engineers`: They handle and run workloads on the `workload` subfolder. They have owner access to all resources in the `workload` folder in order to troubleshoot possible issues with pipelines. This team can also impersonate any service account. - `data-engineers`: They handle and run workloads on the `workload` subfolder. They have editor access to all resources in the `workload` folder in order to troubleshoot possible issues within the workload. This team can also impersonate any service account in the workload folder.
- `data-security`: They handle security configurations for the shielded folder. They have owner access to the `audit-log` and `sec-core` projects. - `data-security`: They handle security configurations for the shielded folder. They have owner access to the `audit-log` and `sec-core` projects.
## Encryption ## Encryption
@ -117,23 +117,23 @@ terraform apply
| name | description | type | required | default | | name | description | type | required | default |
|---|---|:---:|:---:|:---:| |---|---|:---:|:---:|:---:|
| [organization](variables.tf#L130) | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | | [organization](variables.tf#L129) | Organization details. | <code title="object&#40;&#123;&#10; domain &#61; string&#10; id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | |
| [prefix](variables.tf#L138) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | | [prefix](variables.tf#L137) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | |
| [access_policy](variables.tf#L17) | Access Policy name, set to null if creating one. | <code>string</code> | | <code>null</code> | | [access_policy](variables.tf#L17) | Access Policy name, set to null if creating one. | <code>string</code> | | <code>null</code> |
| [access_policy_create](variables.tf#L23) | Access Policy configuration, fill in to create. Parent is in 'organizations/123456' format. | <code title="object&#40;&#123;&#10; parent &#61; string&#10; title &#61; string&#10; scopes &#61; optional&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | | [access_policy_create](variables.tf#L23) | Access Policy configuration, fill in to create. Parent is in 'organizations/123456' format. | <code title="object&#40;&#123;&#10; parent &#61; string&#10; title &#61; string&#10; scopes &#61; optional&#40;list&#40;string&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [data_dir](variables.tf#L33) | Relative path for the folder storing configuration data. | <code>string</code> | | <code>&#34;data&#34;</code> | | [data_dir](variables.tf#L33) | Relative path for the folder storing configuration data. | <code>string</code> | | <code>&#34;data&#34;</code> |
| [enable_features](variables.tf#L39) | Flag to enable features on the solution. | <code title="object&#40;&#123;&#10; encryption &#61; bool&#10; log_sink &#61; bool&#10; vpc_sc &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; encryption &#61; false&#10; log_sink &#61; true&#10; vpc_sc &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> | | [enable_features](variables.tf#L39) | Flag to enable features on the solution. | <code title="object&#40;&#123;&#10; encryption &#61; bool&#10; log_sink &#61; bool&#10; vpc_sc &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; encryption &#61; false&#10; log_sink &#61; true&#10; vpc_sc &#61; true&#10;&#125;">&#123;&#8230;&#125;</code> |
| [folder_create](variables.tf#L52) | Provide values if folder creation is needed, uses existing folder if null. Parent is in 'folders/nnn' or 'organizations/nnn' format. | <code title="object&#40;&#123;&#10; display_name &#61; string&#10; parent &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | | [folder_create](variables.tf#L52) | Provide values if folder creation is needed, uses existing folder if null. Parent is in 'folders/nnn' or 'organizations/nnn' format. | <code title="object&#40;&#123;&#10; display_name &#61; string&#10; parent &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [folder_id](variables.tf#L61) | Folder ID in case you use folder_create=null. | <code>string</code> | | <code>null</code> | | [folder_id](variables.tf#L61) | Folder ID in case you use folder_create=null. | <code>string</code> | | <code>null</code> |
| [groups](variables.tf#L67) | User groups. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; data-engineers &#61; &#34;gcp-data-engineers&#34;&#10; data-security &#61; &#34;gcp-data-security&#34;&#10;&#125;">&#123;&#8230;&#125;</code> | | [groups](variables.tf#L67) | User groups. | <code>map&#40;string&#41;</code> | | <code title="&#123;&#10; workload-engineers &#61; &#34;gcp-data-engineers&#34;&#10; workload-security &#61; &#34;gcp-data-security&#34;&#10;&#125;">&#123;&#8230;&#125;</code> |
| [kms_keys](variables.tf#L77) | KMS keys to create, keyed by name. | <code title="map&#40;object&#40;&#123;&#10; iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; locations &#61; optional&#40;list&#40;string&#41;, &#91;&#34;global&#34;, &#34;europe&#34;, &#34;europe-west1&#34;&#93;&#41;&#10; rotation_period &#61; optional&#40;string, &#34;7776000s&#34;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | | [kms_keys](variables.tf#L76) | KMS keys to create, keyed by name. | <code title="map&#40;object&#40;&#123;&#10; iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; locations &#61; optional&#40;list&#40;string&#41;, &#91;&#34;global&#34;, &#34;europe&#34;, &#34;europe-west1&#34;&#93;&#41;&#10; rotation_period &#61; optional&#40;string, &#34;7776000s&#34;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [log_locations](variables.tf#L88) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object&#40;&#123;&#10; bq &#61; optional&#40;string, &#34;europe&#34;&#41;&#10; storage &#61; optional&#40;string, &#34;europe&#34;&#41;&#10; logging &#61; optional&#40;string, &#34;global&#34;&#41;&#10; pubsub &#61; optional&#40;string, &#34;global&#34;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; bq &#61; &#34;europe&#34;&#10; storage &#61; &#34;europe&#34;&#10; logging &#61; &#34;global&#34;&#10; pubsub &#61; null&#10;&#125;">&#123;&#8230;&#125;</code> | | [log_locations](variables.tf#L87) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object&#40;&#123;&#10; bq &#61; optional&#40;string, &#34;europe&#34;&#41;&#10; storage &#61; optional&#40;string, &#34;europe&#34;&#41;&#10; logging &#61; optional&#40;string, &#34;global&#34;&#41;&#10; pubsub &#61; optional&#40;string, &#34;global&#34;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; bq &#61; &#34;europe&#34;&#10; storage &#61; &#34;europe&#34;&#10; logging &#61; &#34;global&#34;&#10; pubsub &#61; null&#10;&#125;">&#123;&#8230;&#125;</code> |
| [log_sinks](variables.tf#L105) | Org-level log sinks, in name => {type, filter} format. | <code title="map&#40;object&#40;&#123;&#10; filter &#61; string&#10; type &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; audit-logs &#61; &#123;&#10; filter &#61; &#34;logName:&#92;&#34;&#47;logs&#47;cloudaudit.googleapis.com&#37;2Factivity&#92;&#34; OR logName:&#92;&#34;&#47;logs&#47;cloudaudit.googleapis.com&#37;2Fsystem_event&#92;&#34;&#34;&#10; type &#61; &#34;bigquery&#34;&#10; &#125;&#10; vpc-sc &#61; &#123;&#10; filter &#61; &#34;protoPayload.metadata.&#64;type&#61;&#92;&#34;type.googleapis.com&#47;google.cloud.audit.VpcServiceControlAuditMetadata&#92;&#34;&#34;&#10; type &#61; &#34;bigquery&#34;&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> | | [log_sinks](variables.tf#L104) | Org-level log sinks, in name => {type, filter} format. | <code title="map&#40;object&#40;&#123;&#10; filter &#61; string&#10; type &#61; string&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code title="&#123;&#10; audit-logs &#61; &#123;&#10; filter &#61; &#34;logName:&#92;&#34;&#47;logs&#47;cloudaudit.googleapis.com&#37;2Factivity&#92;&#34; OR logName:&#92;&#34;&#47;logs&#47;cloudaudit.googleapis.com&#37;2Fsystem_event&#92;&#34;&#34;&#10; type &#61; &#34;bigquery&#34;&#10; &#125;&#10; vpc-sc &#61; &#123;&#10; filter &#61; &#34;protoPayload.metadata.&#64;type&#61;&#92;&#34;type.googleapis.com&#47;google.cloud.audit.VpcServiceControlAuditMetadata&#92;&#34;&#34;&#10; type &#61; &#34;bigquery&#34;&#10; &#125;&#10;&#125;">&#123;&#8230;&#125;</code> |
| [projects_create](variables.tf#L148) | Provide values if projects creation is needed, uses existing project if null. Projects will be created in the shielded folder. | <code title="object&#40;&#123;&#10; billing_account_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | | [projects_create](variables.tf#L147) | Provide values if projects creation is needed, uses existing project if null. Projects will be created in the shielded folder. | <code title="object&#40;&#123;&#10; billing_account_id &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [projects_id](variables.tf#L156) | Project id, references existing projects if `projects_create` is null. Projects will be moved into the shielded folder. | <code title="object&#40;&#123;&#10; sec-core &#61; string&#10; audit-logs &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> | | [projects_id](variables.tf#L155) | Project id, references existing projects if `projects_create` is null. Projects will be moved into the shielded folder. | <code title="object&#40;&#123;&#10; sec-core &#61; string&#10; audit-logs &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [vpc_sc_access_levels](variables.tf#L165) | VPC SC access level definitions. | <code title="map&#40;object&#40;&#123;&#10; combining_function &#61; optional&#40;string&#41;&#10; conditions &#61; optional&#40;list&#40;object&#40;&#123;&#10; device_policy &#61; optional&#40;object&#40;&#123;&#10; allowed_device_management_levels &#61; optional&#40;list&#40;string&#41;&#41;&#10; allowed_encryption_statuses &#61; optional&#40;list&#40;string&#41;&#41;&#10; require_admin_approval &#61; bool&#10; require_corp_owned &#61; bool&#10; require_screen_lock &#61; optional&#40;bool&#41;&#10; os_constraints &#61; optional&#40;list&#40;object&#40;&#123;&#10; os_type &#61; string&#10; minimum_version &#61; optional&#40;string&#41;&#10; require_verified_chrome_os &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#41;&#10; &#125;&#41;&#41;&#10; ip_subnetworks &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; members &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; negate &#61; optional&#40;bool&#41;&#10; regions &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; required_access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; description &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | | [vpc_sc_access_levels](variables.tf#L164) | VPC SC access level definitions. | <code title="map&#40;object&#40;&#123;&#10; combining_function &#61; optional&#40;string&#41;&#10; conditions &#61; optional&#40;list&#40;object&#40;&#123;&#10; device_policy &#61; optional&#40;object&#40;&#123;&#10; allowed_device_management_levels &#61; optional&#40;list&#40;string&#41;&#41;&#10; allowed_encryption_statuses &#61; optional&#40;list&#40;string&#41;&#41;&#10; require_admin_approval &#61; bool&#10; require_corp_owned &#61; bool&#10; require_screen_lock &#61; optional&#40;bool&#41;&#10; os_constraints &#61; optional&#40;list&#40;object&#40;&#123;&#10; os_type &#61; string&#10; minimum_version &#61; optional&#40;string&#41;&#10; require_verified_chrome_os &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#41;&#10; &#125;&#41;&#41;&#10; ip_subnetworks &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; members &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; negate &#61; optional&#40;bool&#41;&#10; regions &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; required_access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; description &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [vpc_sc_egress_policies](variables.tf#L194) | VPC SC egress policy defnitions. | <code title="map&#40;object&#40;&#123;&#10; from &#61; object&#40;&#123;&#10; identity_type &#61; optional&#40;string, &#34;ANY_IDENTITY&#34;&#41;&#10; identities &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#10; to &#61; object&#40;&#123;&#10; operations &#61; optional&#40;list&#40;object&#40;&#123;&#10; method_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; resource_type_external &#61; optional&#40;bool, false&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | | [vpc_sc_egress_policies](variables.tf#L193) | VPC SC egress policy defnitions. | <code title="map&#40;object&#40;&#123;&#10; from &#61; object&#40;&#123;&#10; identity_type &#61; optional&#40;string, &#34;ANY_IDENTITY&#34;&#41;&#10; identities &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#10; to &#61; object&#40;&#123;&#10; operations &#61; optional&#40;list&#40;object&#40;&#123;&#10; method_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; resource_type_external &#61; optional&#40;bool, false&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [vpc_sc_ingress_policies](variables.tf#L214) | VPC SC ingress policy defnitions. | <code title="map&#40;object&#40;&#123;&#10; from &#61; object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; identity_type &#61; optional&#40;string&#41;&#10; identities &#61; optional&#40;list&#40;string&#41;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#10; to &#61; object&#40;&#123;&#10; operations &#61; optional&#40;list&#40;object&#40;&#123;&#10; method_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> | | [vpc_sc_ingress_policies](variables.tf#L213) | VPC SC ingress policy defnitions. | <code title="map&#40;object&#40;&#123;&#10; from &#61; object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; identity_type &#61; optional&#40;string&#41;&#10; identities &#61; optional&#40;list&#40;string&#41;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#10; to &#61; object&#40;&#123;&#10; operations &#61; optional&#40;list&#40;object&#40;&#123;&#10; method_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
## Outputs ## Outputs

5
blueprints/data-solutions/shielded-folder/kms.tf
View File

@ -64,9 +64,8 @@ module "sec-project" {
project_create = var.projects_create != null && var.enable_features.encryption project_create = var.projects_create != null && var.enable_features.encryption
prefix = var.projects_create == null ? null : var.prefix prefix = var.projects_create == null ? null : var.prefix
group_iam = { group_iam = {
(local.groups.data-engineers) = [ (local.groups.workload-security) = [
"roles/cloudkms.admin", "roles/editor"
"roles/viewer",
] ]
} }
services = [ services = [

5
blueprints/data-solutions/shielded-folder/log-export.tf
View File

@ -43,6 +43,11 @@ module "log-export-project" {
billing_account = try(var.projects_create.billing_account_id, null) billing_account = try(var.projects_create.billing_account_id, null)
project_create = var.projects_create != null project_create = var.projects_create != null
prefix = var.projects_create == null ? null : var.prefix prefix = var.projects_create == null ? null : var.prefix
group_iam = {
(local.groups.workload-security) = [
"roles/editor"
]
}
iam = { iam = {
# "roles/owner" = [module.automation-tf-bootstrap-sa.iam_email] # "roles/owner" = [module.automation-tf-bootstrap-sa.iam_email]
} }

3
blueprints/data-solutions/shielded-folder/main.tf
View File

@ -48,8 +48,9 @@ locals {
for k, v in local.groups : k => "group:${v}" for k, v in local.groups : k => "group:${v}"
} }
group_iam = { group_iam = {
(local.groups.data-engineers) = [ (local.groups.workload-engineers) = [
"roles/editor", "roles/editor",
"roles/iam.serviceAccountTokenCreator"
] ]
} }

5
blueprints/data-solutions/shielded-folder/variables.tf
View File

@ -68,9 +68,8 @@ variable "groups" {
description = "User groups." description = "User groups."
type = map(string) type = map(string)
default = { default = {
#TODO data-analysts = "gcp-data-analysts" workload-engineers = "gcp-data-engineers"
data-engineers = "gcp-data-engineers" workload-security = "gcp-data-security"
data-security = "gcp-data-security"
} }
} }

0
tests/blueprints/data_solutions/shielded-folder/__init__.py → tests/blueprints/data_solutions/shielded_folder/__init__.py
View File

0
tests/blueprints/data_solutions/shielded-folder/fixture/main.tf → tests/blueprints/data_solutions/shielded_folder/fixture/main.tf
View File

0
tests/blueprints/data_solutions/shielded-folder/test_plan.py → tests/blueprints/data_solutions/shielded_folder/test_plan.py
View File