1d13e3e624
- Ensure all variables and outputs are sorted - Ensure all variables and outputs have a description - Add data-solutions/data-platform-foundations to linter Fix all modules to follow these new conventions. |
||
---|---|---|
.. | ||
README.md | ||
diagram.png | ||
main.tf | ||
outputs.tf | ||
variables.tf | ||
versions.tf |
README.md
Data Platform Foundations - Environment (Step 1)
This is the first step needed to deploy Data Platform Foundations, which creates projects and service accounts. Please refer to the top-level Data Platform README for prerequisites.
The projects that will be created are:
- Common services
- Landing
- Orchestration & Transformation
- DWH
- Datamart
A main service account named projects-editor-sa
will be created under the common services project, and it will be granted editor permissions on all the projects in scope.
This is a high level diagram of the created resources:
Running the example
To create the infrastructure:
- specify your variables in a
terraform.tvars
billing_account = "1234-1234-1234"
parent = "folders/12345678"
admins = ["user:xxxxx@yyyyy.com"]
- make sure you have the right authentication setup (application default credentials, or a service account key) with the right permissions
- The output of this stage contains the values for the resources stage
- the
admins
variable contain a list of principals allowed to impersonate the service accounts. These principals will be given theiam.serviceAccountTokenCreator
role - run
terraform init
andterraform apply
Once done testing, you can clean up resources by running terraform destroy
.
CMEK configuration
You can configure GCP resources to use existing CMEK keys configuring the 'service_encryption_key_ids' variable. You need to specify a 'global' and a 'multiregional' key.
VPC-SC configuration
You can assign projects to an existing VPC-SC standard perimeter configuring the 'service_perimeter_standard' variable. You can retrieve the list of existing perimeters from the GCP console or using the following command:
''' gcloud access-context-manager perimeters list --format="json" | grep name '''
The script use 'google_access_context_manager_service_perimeter_resource' terraform resource. If this resource is used alongside the 'vpc-sc' module, remember to uncomment the lifecycle block in the 'vpc-sc' module so they don't fight over which resources should be in the perimeter.
Variables
name | description | type | required | default |
---|---|---|---|---|
billing_account_id | Billing account id. | string |
✓ | |
root_node | Parent folder or organization in 'folders/folder_id' or 'organizations/org_id' format. | string |
✓ | |
admins | List of users allowed to impersonate the service account | list(string) |
null |
|
prefix | Prefix used to generate project id and name. | string |
null |
|
project_names | Override this variable if you need non-standard names. | object({...}) |
... |
|
service_account_names | Override this variable if you need non-standard names. | object({...}) |
... |
|
service_encryption_key_ids | Cloud KMS encryption key in {LOCATION => [KEY_URL]} format. Keys belong to existing project. | object({...}) |
... |
|
service_perimeter_standard | VPC Service control standard perimeter name in the form of 'accessPolicies/ACCESS_POLICY_NAME/servicePerimeters/PERIMETER_NAME'. All projects will be added to the perimeter in enforced mode. | string |
null |
Outputs
name | description | sensitive |
---|---|---|
project_ids | Project ids for created projects. | |
service_account | Main service account. | |
service_encryption_key_ids | Cloud KMS encryption keys in {LOCATION => [KEY_URL]} format. |