cloud-foundation-fabric/fast/stages/README.md

47 lines
4.8 KiB
Markdown
Raw Normal View History

# Fast stages
Each of the folders contained here is a separate "stage", or Terraform root module.
2022-02-16 01:14:51 -08:00
Each stage can be run in isolation (for example to only bring up a hub and spoke VPC in an existing environment), but when combined together they form a modular setup that allows top-down configuration of a whole GCP organization.
When combined together, each stage is designed to leverage the previous stage's resources and to provide outputs to the following stages via predefined contracts, that regulate what is exchanged.
This has two important consequences
- any stage can be swapped out and replaced by different code as long as it respects the contract by providing a predefined set of outputs and optionally accepting a predefined set of variables
2022-02-21 05:52:25 -08:00
- data flow between stages can be partially automated (see [stage 00 documentation on output files](./00-bootstrap/README.md#output-files-and-cross-stage-variables)), reducing the effort and pain required to compile variables by hand
2022-02-16 01:14:51 -08:00
2022-02-21 05:52:25 -08:00
One important assumption is that the flow of data is always forward looking, so no stage needs to depend on outputs generated further down the chain. This greatly simplifies both the logic and the implementation, and allows stages to be effectively independent.
2022-02-16 01:14:51 -08:00
2022-02-21 05:52:25 -08:00
To achieve this, we rely on specific GCP functionality like [delegated role grants](https://medium.com/google-cloud/managing-gcp-service-usage-through-delegated-role-grants-a843610f2226) that allow controlled delegation of responsibilities, for example to allow managing IAM bindings at the organization level in different stages only for specific roles.
Refer to each stage's documentation for a detailed description of its purpose, the architectural choices made in its design, and how it can be configured and wired together to terraform a whole GCP organization. The following is a brief overview of each stage.
2022-06-10 08:16:11 -07:00
To destroy a previous FAST deployment follow the instructions detailed in [cleanup](CLEANUP.md).
## Organizational level (00-01)
2022-01-21 06:18:20 -08:00
- [Bootstrap](00-bootstrap/README.md)
2022-02-15 23:52:19 -08:00
Enables critical organization-level functionality that depends on broad permissions. It has two primary purposes. The first is to bootstrap the resources needed for automation of this and the following stages (service accounts, GCS buckets). And secondly, it applies the minimum amount of configuration needed at the organization level, to avoid the need of broad permissions later on, and to implement a minimum of security features like sinks and exports from the start.\
Initial MVP for CI/CD (#608) * preliminary support for wif in stage 0 * IAM wif role * IAM wif role TODO * add support for external SA IAM to SA module * add name output to SA module * separate cicd SA * tfdoc * GITLAB principal (untested) * make GCS name output static * outputs bucket * fix stage 1 test * tweak outputs * tfdoc * move wif_pool to automation variable * add support for top-level and repository providers * add missing boilerplate * fix branchless principal * initial workflow * symlink provider template in stages * remove service accounts from stage 0 cicd tfvars * add cicd interface variable to resman stage * fix cicd variable in resman stage * better condition on outputs_location * fix last change * change outputs_location type * revert outputs_location change * split outputs in stage 0 * update ci/cd temporary notes * rename additive IAM resource in SA module * split outputs in stage 1 * remove unused locals * fix stage 1 tests * tfdoc * Upload action files to outputs_bucket * Fix tests and README * rename template, streamline outputs * local templates and gcs output for all stage 2 * add workflows to local output files * Use lowercase WIF providers everywhere * Bring back suffix for workflow files * Remove unused files * Update READMEs * preliminary CI/CD implementation for stage 1 * fix stage 1 * stage 1 cicd * tfdoc * fix tests * readme and links for cicd and wif * refactor wif providers * refactor cicd for stage 1 * fix stage 1 * wif org policies * split identity provider configuration from cicd * add type attribute to cicd repositories * valid cicd repositories have a workflow template * refactor stage 01 * fix stage 01 tests * minimal CI/CD documentation * better check_links error reporting * fix links * Added Gitlab specific configurations Set the default issuer_uri for Gitlab. Added allowed audiences to OIDC configuration. * Fixed TF formatting in identity providers. * Changing identity provider audience to null Changing identity provider audience to default to null. * add instructions for renaming workflows * address Julio's comments Co-authored-by: Julio Castillo <jccb@google.com> Co-authored-by: alexmeissner <alexmeissner@google.com>
2022-04-11 23:17:27 -07:00
Exports: automation variables, organization-level custom roles
2022-01-21 06:18:20 -08:00
- [Resource Management](01-resman/README.md)
2022-02-15 23:52:19 -08:00
Creates the base resource hierarchy (folders) and the automation resources required later to delegate deployment of each part of the hierarchy to separate stages. This stage also configures organization-level policies and any exceptions needed by different branches of the resource hierarchy.\
Exports: folder ids, automation service account emails
## Shared resources (02)
2022-01-21 06:18:20 -08:00
- [Security](02-security/README.md)
2022-02-15 23:52:19 -08:00
Manages centralized security configurations in a separate stage, and is typically owned by the security team. This stage implements VPC Security Controls via separate perimeters for environments and central services, and creates projects to host centralized KMS keys used by the whole organization. It's meant to be easily extended to include other security-related resources which are required, like Secret Manager.\
Exports: KMS key ids
- Networking ([VPN](02-networking-vpn/README.md)/[NVA](02-networking-nva/README.md)/[Peering](02-networking-separate-envs/README.md)/[Separate environments](02-networking-separate-envs/README.md))
Manages centralized network resources in a separate stage, and is typically owned by the networking team. This stage implements a hub-and-spoke design, and includes connectivity via VPN to on-premises, and YAML-based factories for firewall rules (hierarchical and VPC-level) and subnets. It's currently available in four flavors: [spokes connected via VPN](02-networking-vpn/README.md), [and spokes connected via appliances](02-networking-nva/README.md), [spokes connected via VPC peering](02-networking-peering/README.md), and [separated network environments](02-networking-separate-envs/README.md).\
2022-02-15 23:52:19 -08:00
Exports: host project ids and numbers, vpc self links
## Environment-level resources (03)
2022-02-18 23:28:35 -08:00
- [Project Factory](03-project-factory/dev/)
YAML-based fatory to create and configure application or team-level projects. Configuration includes VPC-level settings for Shared VPC, service-level configuration for CMEK encryption via centralized keys, and service account creation for workloads and applications. This stage is meant to be used once per environment.
2022-02-18 23:28:35 -08:00
- [Data Platform](03-data-platform/dev/)
2022-09-08 22:21:24 -07:00
- [GKE Multitenant](03-gke-multitenant/dev/)
- GCE Migration (in development)