cloud-foundation-fabric/fast/stages-multitenant/1-resman-tenant
Ludovico Magnocavallo 5453c585e0
FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052)
* rename stages

* remove support for external org billing, rename output files

* resman: make groups optional, align on new billing account variable

* bootstrap: multitenant outputs

* tenant bootstrap stage, untested

* fix folder name

* fix stage 0 output names

* optional creation for tag keys in organization module

* single tenant bootstrap minus tag

* rename output files, add tenant tag key

* fix organization module tag values output

* test skipping creation for tags in organization module

* single tenant bootstrap plan working

* multitenant bootstrap

* tfdoc

* fix check links error messages

* fix links

* tfdoc

* fix links

* rename fast tests, fix bootstrap tests

* multitenant stages have their own folder, simplify stage numbering

* stage renumbering

* wip

* rename tests

* exclude fast providers in fixture

* stage 0 tests

* stage 1 tests

* network stages tests

* stage tests

* tfdoc

* fix links

* tfdoc

* multitenant tests

* remove local files

* stage links command

* fix links script, TODO

* wip

* wip single tenant bootstrap

* working tenant bootstrap

* update gitignore

* remove local files

* tfdoc

* remove local files

* allow tests for tenant bootstrap stage

* tenant bootstrap proxies stage 1 tfvars

* stage 2 and 3 service accounts and IAM in tenant bootstrap

* wip

* wip

* wip

* drop multitenant bootstrap

* tfdoc

* add missing stage 2 SAs, fix org-level IAM condition

* wip

* wip

* optional tag value creation in organization module

* stage 1 working

* linting

* linting

* READMEs

* wip

* Make stage-links script work in old macos bash

* stage links command help

* fix output file names

* diagrams

* fix svg

* stage 0 skeleton and diagram

* test svg

* test svg

* test diagram

* diagram

* readme

* fix stage links script

* stage 0 readme

* README changes

* stage readmes

* fix outputs order

* fix link

* fix tests

* stage 1 test

* skip stage example

* boilerplate

* fix tftest skip

* default bootstrap stage log sinks to log buckets

* add logging to tenant bootstrap

* move iam variables out of tenant config

* fix cicd, reintroduce missing variable

* use optional in stage 1 cicd variable

* rename extras stage

* rename and move identity providers local, use optional for cicd variable

* tfdoc

* add support for wif pool and providers, ci/cd

* tfdoc

* fix links

* better handling of modules repository

* add missing role on logging project

* fix cicd pools in locals, test cicd

* fix workflow extension

* fix module source replacement

* allow tenant bootstrap cicd sa to impersonate resman sa

* tenant workflow templates fix for no providers file

* fix output files, push github workflow template to new repository

* remove try from outpout files

* align stage 1 cicd internals to stage 0

* tfdoc

* tests

* fix tests

* tests

* improve variable descriptions

* use optional in fast features

* actually create tenant log sinks, and allow the resman sa to do it

* test

* tests

* aaaand tests again

* fast features tenant override

* fast features tenant override

* fix wording

* add missing comment

* configure pf service accounts

* add missing comment

* tfdoc

* tests

* IAM docs

* update copyright

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-02-04 15:00:45 +01:00
..
data/org-policies FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
templates FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
IAM.md FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
README.md FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
branch-data-platform.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
branch-gke.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
branch-networking.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
branch-project-factory.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
branch-sandbox.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
branch-security.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
branch-teams.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
cicd-data-platform.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
cicd-gke.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
cicd-networking.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
cicd-project-factory.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
cicd-security.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
diagram.png FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
diagram.svg FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
main.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
outputs-files.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
outputs-gcs.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
outputs.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
root_node.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00
variables.tf FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052) 2023-02-04 15:00:45 +01:00

README.md

Tenant resource management

This stage is run for a specific tenant after tenant bootstrap has successfully created initial resources for the tenant, which is then decoupled from the organization.

It is logically equivalent and almost identical in code to the corresponding organization resource management stage, with a few notable differences:

  • the hierarchy is rooted in the tenant top-level folder instead of the organization
  • there's no management of tag values and keys since they organization-level resources (it could be implemented for tenant-specific tags if the need arises)
  • automation service accounts for subsequent stages are configured but not created here (tenant-level bootstrap creates them and assigns organization-level permissions)

The stage runs with a dedicated service account for the tenant, which has no permissions at the organization level except for billing and organization policies, constrained by a condition on the tenant tag.

The following diagram is a high level reference of what this stage manages, showing one hypothetical tenant (additional tenants require additional instances of this stage being deployed):

%%{init: {'theme':'base'}}%%
classDiagram
    Tenant_root~📁~ -- tn0_automation
    Tenant_root~📁~ -- Networking~📁~
    Tenant_root~📁~ -- Security~📁~
    Tenant_root~📁~ -- Data_Platform~📁~
    Data_Platform~📁~ -- DP_Dev~📁~
    Data_Platform~📁~ -- DP_Prod~📁~
    Tenant_root~📁~ -- GKE~📁~
    GKE~📁~ -- GKE_Dev~📁~
    GKE~📁~ -- GKE_Prod~📁~
    Tenant_root~📁~ -- Teams~📁~
    Teams~📁~ -- Team_0~📁~
    Team_0~📁~ -- Team_0_Dev~📁~
    Team_0~📁~ -- Team_0_Prod~📁~
    Tenant_root~📁~ -- Sandbox~📁~
    class Tenant_root~📁~ {
        - IAM bindings()
        - org policies()
    }
    class tn0_automation {
        - GCS buckets
        - IAM bindings()
    }
    class Data_Platform~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class DP_Dev~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class DP_Prod~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class GKE~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class GKE_Dev~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class GKE_Prod~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class Networking~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class Security~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class Sandbox~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class Teams~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class Team_0~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class Team_0_Dev~📁~ {
      - IAM bindings()
      - tag bindings()
    }
    class Team_0_Prod~📁~ {
      - IAM bindings()
      - tag bindings()
    }

As most of the features of this stage follow the same design and configurations of the organization-level resource management stage, we will only focus on the tenant-specific configuration in this document.

How to run this stage

As mentioned above this stage is decoupled from organization-level stages: it uses a service account and state bucket from the tenant-specific automation project, and its tfvars and provider files are also tenant-specific.

The stage-links.sh script can be used to get the commands needed for the provider and output files, just set the variable for the tenant shortname (the same one specified in the tenant bootstrap stage) and pass a single argument with your FAST output files folder path, or GCS bucket URI:

TENANT=tn0 ../../stage-links.sh ~/fast-config

The script output can be copy/pasted to a terminal:

# copy and paste the following commands for '1-resman-tenant'

ln -s ~/fast-config/tenants/tn0/providers/1-resman-tenant-providers.tf ./
ln -s ~/fast-config/tenants/tn0/tfvars/0-bootstrap-tenant.auto.tfvars.json ./

Once that is done, stage-level configuration variables are the same as the corresponding organization-level stage.

Running the stage

Once the configuration is done just go through the usual init/apply cycle. On successful apply, a tfvars file specific for this tenant and a set of provider files will be created.

Files

name description modules resources
branch-data-platform.tf Data Platform stages resources. folder · gcs · iam-service-account
branch-gke.tf GKE multitenant stage resources. folder · gcs · iam-service-account
branch-networking.tf Networking stage resources. folder · gcs · iam-service-account
branch-project-factory.tf Project factory stage resources. gcs · iam-service-account
branch-sandbox.tf Sandbox stage resources. folder · gcs
branch-security.tf Security stage resources. folder · gcs · iam-service-account
branch-teams.tf Team stage resources. folder · gcs · iam-service-account
cicd-data-platform.tf CI/CD resources for the data platform branch. iam-service-account · source-repository
cicd-gke.tf CI/CD resources for the data platform branch. iam-service-account · source-repository
cicd-networking.tf CI/CD resources for the networking branch. iam-service-account · source-repository
cicd-project-factory.tf CI/CD resources for the teams branch. iam-service-account · source-repository
cicd-security.tf CI/CD resources for the security branch. iam-service-account · source-repository
main.tf Module-level locals and resources.
outputs-files.tf Output files persistence to local filesystem. local_file
outputs-gcs.tf Output files persistence to automation GCS bucket. google_storage_bucket_object
outputs.tf Module outputs.
root_node.tf Tenant root folder configuration. folder
variables.tf Module variables.

Variables

name description type required default producer
automation Automation resources created by the bootstrap stage. object({…}) 0-bootstrap
billing_account Billing account id. If billing account is not part of the same org set is_org_level to false. object({…}) 0-bootstrap
organization Organization details. object({…}) 0-bootstrap
prefix Prefix used for resources that need unique names. Use 9 characters or less. string 0-bootstrap
root_node Root folder node for the tenant, in folders/nnnnnn format. string
short_name Short name used to identify the tenant. string
tags Resource management tags. object({…})
cicd_repositories CI/CD repository configuration. Identity providers reference keys in the automation.federated_identity_providers variable. Set to null to disable, or set individual repositories to null if not needed. object({…}) null
custom_roles Custom roles defined at the org level, in key => id format. object({…}) null 0-bootstrap
data_dir Relative path for the folder storing configuration data. string "data"
fast_features Selective control for top-level FAST features. object({…}) {} 0-0-bootstrap
groups Group names to grant organization-level permissions. object({…}) {} 0-bootstrap
locations Optional locations for GCS, BigQuery, and logging buckets created here. object({…}) {…} 0-bootstrap
organization_policy_data_path Path for the data folder used by the organization policies factory. string null
outputs_location Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. string null
team_folders Team folders to be created. Format is described in a code comment. map(object({…})) null
test_skip_data_sources Used when testing to bypass data sources. bool false

Outputs

name description sensitive consumers
cicd_repositories WIF configuration for CI/CD repositories.
dataplatform Data for the Data Platform stage.
gke_multitenant Data for the GKE multitenant stage. 03-gke-multitenant
networking Data for the networking stage.
project_factories Data for the project factories stage.
providers Terraform provider files for this stage and dependent stages. 02-networking · 02-security · 03-dataplatform · xx-sandbox · xx-teams
sandbox Data for the sandbox stage. xx-sandbox
security Data for the networking stage. 02-security
teams Data for the teams stage.
tfvars Terraform variable files for the following stages.