FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052)
* rename stages * remove support for external org billing, rename output files * resman: make groups optional, align on new billing account variable * bootstrap: multitenant outputs * tenant bootstrap stage, untested * fix folder name * fix stage 0 output names * optional creation for tag keys in organization module * single tenant bootstrap minus tag * rename output files, add tenant tag key * fix organization module tag values output * test skipping creation for tags in organization module * single tenant bootstrap plan working * multitenant bootstrap * tfdoc * fix check links error messages * fix links * tfdoc * fix links * rename fast tests, fix bootstrap tests * multitenant stages have their own folder, simplify stage numbering * stage renumbering * wip * rename tests * exclude fast providers in fixture * stage 0 tests * stage 1 tests * network stages tests * stage tests * tfdoc * fix links * tfdoc * multitenant tests * remove local files * stage links command * fix links script, TODO * wip * wip single tenant bootstrap * working tenant bootstrap * update gitignore * remove local files * tfdoc * remove local files * allow tests for tenant bootstrap stage * tenant bootstrap proxies stage 1 tfvars * stage 2 and 3 service accounts and IAM in tenant bootstrap * wip * wip * wip * drop multitenant bootstrap * tfdoc * add missing stage 2 SAs, fix org-level IAM condition * wip * wip * optional tag value creation in organization module * stage 1 working * linting * linting * READMEs * wip * Make stage-links script work in old macos bash * stage links command help * fix output file names * diagrams * fix svg * stage 0 skeleton and diagram * test svg * test svg * test diagram * diagram * readme * fix stage links script * stage 0 readme * README changes * stage readmes * fix outputs order * fix link * fix tests * stage 1 test * skip stage example * boilerplate * fix tftest skip * default bootstrap stage log sinks to log buckets * add logging to tenant bootstrap * move iam variables out of tenant config * fix cicd, reintroduce missing variable * use optional in stage 1 cicd variable * rename extras stage * rename and move identity providers local, use optional for cicd variable * tfdoc * add support for wif pool and providers, ci/cd * tfdoc * fix links * better handling of modules repository * add missing role on logging project * fix cicd pools in locals, test cicd * fix workflow extension * fix module source replacement * allow tenant bootstrap cicd sa to impersonate resman sa * tenant workflow templates fix for no providers file * fix output files, push github workflow template to new repository * remove try from outpout files * align stage 1 cicd internals to stage 0 * tfdoc * tests * fix tests * tests * improve variable descriptions * use optional in fast features * actually create tenant log sinks, and allow the resman sa to do it * test * tests * aaaand tests again * fast features tenant override * fast features tenant override * fix wording * add missing comment * configure pf service accounts * add missing comment * tfdoc * tests * IAM docs * update copyright --------- Co-authored-by: Julio Castillo <jccb@google.com>
This commit is contained in:
Ludovico Magnocavallo
GitHub
parent
ea09a0df68
commit
5453c585e0
|
|
@ -21,13 +21,13 @@ bundle.zip
|
|||
**/*.pkrvars.hcl
|
||||
fixture_*
|
||||
fast/configs
|
||||
fast/stages/**/[0-9]*providers.tf
|
||||
fast/stages/**/terraform.tfvars
|
||||
fast/stages/**/terraform.tfvars.json
|
||||
fast/stages/**/terraform-*.auto.tfvars.json
|
||||
fast/stages/**/0*.auto.tfvars*
|
||||
fast/**/[0-9]*providers.tf
|
||||
fast/**/terraform.tfvars
|
||||
fast/**/terraform.tfvars.json
|
||||
fast/**/terraform-*.auto.tfvars.json
|
||||
fast/**/[0-9]*.auto.tfvars*
|
||||
**/node_modules
|
||||
fast/stages/**/globals.auto.tfvars.json
|
||||
fast/**/globals.auto.tfvars.json
|
||||
cloud_sql_proxy
|
||||
examples/cloud-operations/binauthz/tenant-setup.yaml
|
||||
examples/cloud-operations/binauthz/app/app.yaml
|
||||
|
|
|
|||
|
|
@ -480,7 +480,7 @@ All notable changes to this project will be documented in this file.
|
|||
- fix `tag` output on `data-catalog-policy-tag` module
|
||||
- add shared-vpc support on `gcs-to-bq-with-least-privileges`
|
||||
- new `net-ilb-l7` module
|
||||
- new [02-networking-peering](fast/stages/02-networking-peering) networking stage
|
||||
- new `02-networking-peering` networking stage
|
||||
- **incompatible change** the variable for PSA ranges in networking stages have changed
|
||||
|
||||
## [14.0.0] - 2022-02-25
|
||||
|
|
@ -499,8 +499,8 @@ All notable changes to this project will be documented in this file.
|
|||
- **incompatible change** removed `ingress_settings` configuration option in the `cloud-functions` module.
|
||||
- new [m4ce VM example](blueprints/cloud-operations/vm-migration/)
|
||||
- Support for resource management tags in the `organization`, `folder`, `project`, `compute-vm`, and `kms` modules
|
||||
- new [data platform](fast/stages/03-data-platform) stage 3
|
||||
- new [02-networking-nva](fast/stages/02-networking-nva) networking stage
|
||||
- new `data platform` stage 3
|
||||
- new `02-networking-nva` networking stage
|
||||
- allow customizing the names of custom roles
|
||||
- added `environment` and `context` resource management tags
|
||||
- use resource management tags to restrict scope of roles/orgpolicy.policyAdmin
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ The approach adapts to different high-level requirements:
|
|||
- least privilege principle
|
||||
- rely on service account impersonation
|
||||
|
||||
The code in this blueprint doesn't address Organization-level configurations (Organization policy, VPC-SC, centralized logs). We expect those elements to be managed by automation stages external to this script like those in [FAST](../../../fast) and this blueprint deployed on top of them as one of the [stages](../../../fast/stages/03-data-platform/dev/README.md).
|
||||
The code in this blueprint doesn't address Organization-level configurations (Organization policy, VPC-SC, centralized logs). We expect those elements to be managed by automation stages external to this script like those in [FAST](../../../fast) and this blueprint deployed on top of them as one of the [stages](../../../fast/stages/3-data-platform/dev/README.md).
|
||||
|
||||
### Project structure
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ This blueprint presents an opinionated architecture to handle multiple homogeneo
|
|||
|
||||
The pattern used in this design is useful, for blueprint, in cases where multiple clusters host/support the same workloads, such as in the case of a multi-regional deployment. Furthermore, combined with Anthos Config Sync and proper RBAC, this architecture can be used to host multiple tenants (e.g. teams, applications) sharing the clusters.
|
||||
|
||||
This blueprint is used as part of the [FAST GKE stage](../../../fast/stages/03-gke-multitenant/) but it can also be used independently if desired.
|
||||
This blueprint is used as part of the [FAST GKE stage](../../../fast/stages/3-gke-multitenant/) but it can also be used independently if desired.
|
||||
|
||||
<p align="center">
|
||||
<img src="diagram.png" alt="GKE multitenant">
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ A few additional features are also shown:
|
|||
- [custom BGP advertisements](https://cloud.google.com/router/docs/how-to/advertising-overview) to implement transitivity between spokes
|
||||
- [VPC Global Routing](https://cloud.google.com/network-connectivity/docs/router/how-to/configuring-routing-mode) to leverage a regional set of VPN gateways in different regions as next hops (used here for illustrative/study purpose, not usually done in real life)
|
||||
|
||||
The blueprint has been purposefully kept simple to show how to use and wire the VPC and VPN-HA modules together, and so that it can be used as a basis for experimentation. For a more complex scenario that better reflects real-life usage, including [Shared VPC](https://cloud.google.com/vpc/docs/shared-vpc) and [DNS cross-project binding](https://cloud.google.com/dns/docs/zones/cross-project-binding) please refer to the [FAST network stage](../../../fast/stages/02-networking-vpn/).
|
||||
The blueprint has been purposefully kept simple to show how to use and wire the VPC and VPN-HA modules together, and so that it can be used as a basis for experimentation. For a more complex scenario that better reflects real-life usage, including [Shared VPC](https://cloud.google.com/vpc/docs/shared-vpc) and [DNS cross-project binding](https://cloud.google.com/dns/docs/zones/cross-project-binding) please refer to the [FAST network stage](../../../fast/stages/2-networking-b-vpn/).
|
||||
|
||||
This is the high level diagram of this blueprint:
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,293 @@
|
|||
<svg fill="none" viewBox="0 0 800 400" width="800" height="400" xmlns="http://www.w3.org/2000/svg">
|
||||
<foreignObject width="100%" height="100%">
|
||||
<div xmlns="http://www.w3.org/1999/xhtml">
|
||||
<style>
|
||||
.edgePaths path {
|
||||
stroke: #bebebe !important;
|
||||
}
|
||||
.mermaidExternal > rect {
|
||||
fill: #f6f6f6 !important;
|
||||
stroke-dasharray: 5,5;
|
||||
stroke: #bebebe !important;
|
||||
}
|
||||
.mermaidOrg > rect {
|
||||
fill: #F6F6F6 !important;
|
||||
}
|
||||
.mermaidFolder > rect {
|
||||
fill: #F1F8E9 !important;
|
||||
stroke: #abd57b !important;
|
||||
}
|
||||
</style>
|
||||
<style>@import url("https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.2.1/css/all.min.css");'</style>
|
||||
<style>#graph-div{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}#graph-div .error-icon{fill:hsl(25.3846153846, 86.6666666667%, 99.1176470588%);}#graph-div .error-text{fill:rgb(0.3, 2.5500000001, 4.2000000001);stroke:rgb(0.3, 2.5500000001, 4.2000000001);}#graph-div .edge-thickness-normal{stroke-width:2px;}#graph-div .edge-thickness-thick{stroke-width:3.5px;}#graph-div .edge-pattern-solid{stroke-dasharray:0;}#graph-div .edge-pattern-dashed{stroke-dasharray:3;}#graph-div .edge-pattern-dotted{stroke-dasharray:2;}#graph-div .marker{fill:#0b0b0b;stroke:#0b0b0b;}#graph-div .marker.cross{stroke:#0b0b0b;}#graph-div svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#graph-div g.classGroup text{fill:hsl(205.3846153846, 46.6666666667%, 84.1176470588%);fill:#333;stroke:none;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:10px;}#graph-div g.classGroup text .title{font-weight:bolder;}#graph-div .nodeLabel,#graph-div .edgeLabel{color:#333;}#graph-div .edgeLabel .label rect{fill:#E3F2FD;}#graph-div .label text{fill:#333;}#graph-div .edgeLabel .label span{background:#E3F2FD;}#graph-div .classTitle{font-weight:bolder;}#graph-div .node rect,#graph-div .node circle,#graph-div .node ellipse,#graph-div .node polygon,#graph-div .node path{fill:#E3F2FD;stroke:hsl(205.3846153846, 46.6666666667%, 84.1176470588%);stroke-width:1px;}#graph-div .divider{stroke:hsl(205.3846153846, 46.6666666667%, 84.1176470588%);stroke:1;}#graph-div g.clickable{cursor:pointer;}#graph-div g.classGroup rect{fill:#E3F2FD;stroke:hsl(205.3846153846, 46.6666666667%, 84.1176470588%);}#graph-div g.classGroup line{stroke:hsl(205.3846153846, 46.6666666667%, 84.1176470588%);stroke-width:1;}#graph-div .classLabel .box{stroke:none;stroke-width:0;fill:#E3F2FD;opacity:0.5;}#graph-div .classLabel .label{fill:hsl(205.3846153846, 46.6666666667%, 84.1176470588%);font-size:10px;}#graph-div .relation{stroke:#0b0b0b;stroke-width:1;fill:none;}#graph-div .dashed-line{stroke-dasharray:3;}#graph-div .dotted-line{stroke-dasharray:1 2;}#graph-div #compositionStart,#graph-div .composition{fill:#0b0b0b!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div #compositionEnd,#graph-div .composition{fill:#0b0b0b!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div #dependencyStart,#graph-div .dependency{fill:#0b0b0b!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div #dependencyStart,#graph-div .dependency{fill:#0b0b0b!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div #extensionStart,#graph-div .extension{fill:#E3F2FD!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div #extensionEnd,#graph-div .extension{fill:#E3F2FD!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div #aggregationStart,#graph-div .aggregation{fill:#E3F2FD!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div #aggregationEnd,#graph-div .aggregation{fill:#E3F2FD!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div #lollipopStart,#graph-div .lollipop{fill:#E3F2FD!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div #lollipopEnd,#graph-div .lollipop{fill:#E3F2FD!important;stroke:#0b0b0b!important;stroke-width:1;}#graph-div .edgeTerminals{font-size:11px;}#graph-div .classTitleText{text-anchor:middle;font-size:18px;fill:#333;}#graph-div :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}</style>
|
||||
<svg aria-roledescription="classDiagram" viewBox="0 0 500.0283203125 640.9853515625" style="max-width: 100%;" xmlns="http://www.w3.org/2000/svg" width="100%" id="graph-div" height="100%" xmlns:xlink="http://www.w3.org/1999/xlink">
|
||||
<g>
|
||||
<defs>
|
||||
<marker orient="auto" markerHeight="240" markerWidth="190" refY="7" refX="0" class="marker aggregation classDiagram" id="classDiagram-aggregationStart">
|
||||
<path d="M 18,7 L9,13 L1,7 L9,1 Z"></path>
|
||||
</marker>
|
||||
</defs>
|
||||
<defs>
|
||||
<marker orient="auto" markerHeight="28" markerWidth="20" refY="7" refX="19" class="marker aggregation classDiagram" id="classDiagram-aggregationEnd">
|
||||
<path d="M 18,7 L9,13 L1,7 L9,1 Z"></path>
|
||||
</marker>
|
||||
</defs>
|
||||
<defs>
|
||||
<marker orient="auto" markerHeight="240" markerWidth="190" refY="7" refX="0" class="marker extension classDiagram" id="classDiagram-extensionStart">
|
||||
<path d="M 1,7 L18,13 V 1 Z"></path>
|
||||
</marker>
|
||||
</defs>
|
||||
<defs>
|
||||
<marker orient="auto" markerHeight="28" markerWidth="20" refY="7" refX="19" class="marker extension classDiagram" id="classDiagram-extensionEnd">
|
||||
<path d="M 1,1 V 13 L18,7 Z"></path>
|
||||
</marker>
|
||||
</defs>
|
||||
<defs>
|
||||
<marker orient="auto" markerHeight="240" markerWidth="190" refY="7" refX="0" class="marker composition classDiagram" id="classDiagram-compositionStart">
|
||||
<path d="M 18,7 L9,13 L1,7 L9,1 Z"></path>
|
||||
</marker>
|
||||
</defs>
|
||||
<defs>
|
||||
<marker orient="auto" markerHeight="28" markerWidth="20" refY="7" refX="19" class="marker composition classDiagram" id="classDiagram-compositionEnd">
|
||||
<path d="M 18,7 L9,13 L1,7 L9,1 Z"></path>
|
||||
</marker>
|
||||
</defs>
|
||||
<defs>
|
||||
<marker orient="auto" markerHeight="240" markerWidth="190" refY="7" refX="0" class="marker dependency classDiagram" id="classDiagram-dependencyStart">
|
||||
<path d="M 5,7 L9,13 L1,7 L9,1 Z"></path>
|
||||
</marker>
|
||||
</defs>
|
||||
<defs>
|
||||
<marker orient="auto" markerHeight="28" markerWidth="20" refY="7" refX="19" class="marker dependency classDiagram" id="classDiagram-dependencyEnd">
|
||||
<path d="M 18,7 L9,13 L14,7 L9,1 Z"></path>
|
||||
</marker>
|
||||
</defs>
|
||||
<defs>
|
||||
<marker orient="auto" markerHeight="240" markerWidth="190" refY="7" refX="0" class="marker lollipop classDiagram" id="classDiagram-lollipopStart">
|
||||
<circle r="6" cy="7" cx="6" fill="white" stroke="black"></circle>
|
||||
</marker>
|
||||
</defs>
|
||||
<g class="root">
|
||||
<g class="clusters"></g>
|
||||
<g class="edgePaths">
|
||||
<path style="fill:none" class="edge-pattern-solid relation" id="id1" d="M162.64356231689453,145.95778602350617L154.95414861043295,151.63083476812363C147.26473490397134,157.3038835127411,131.8859074910482,168.64998100197602,124.1964937845866,178.48969641326016C116.507080078125,188.32941182454428,116.507080078125,196.6627451578776,116.507080078125,200.82941182454428L116.507080078125,204.99607849121094"></path>
|
||||
<path style="fill:none" class="edge-pattern-solid relation" id="id2" d="M337.38475799560547,145.95778602350617L345.0741717020671,151.63083476812363C352.7635854085286,157.3038835127411,368.14241282145184,168.64998100197602,375.8318265279134,178.48969641326016C383.521240234375,188.32941182454428,383.521240234375,196.6627451578776,383.521240234375,200.82941182454428L383.521240234375,204.99607849121094"></path>
|
||||
<path style="fill:none" class="edge-pattern-solid relation" id="id3" d="M116.507080078125,379.99119567871094L116.507080078125,384.1578623453776C116.507080078125,388.32452901204425,116.507080078125,396.6578623453776,116.507080078125,404.99119567871094C116.507080078125,413.32452901204425,116.507080078125,421.6578623453776,116.507080078125,425.82452901204425L116.507080078125,429.99119567871094"></path>
|
||||
<path style="fill:none" class="edge-pattern-solid relation" id="id4" d="M383.521240234375,379.99119567871094L383.521240234375,384.1578623453776C383.521240234375,388.32452901204425,383.521240234375,396.6578623453776,383.521240234375,404.99119567871094C383.521240234375,413.32452901204425,383.521240234375,421.6578623453776,383.521240234375,425.82452901204425L383.521240234375,429.99119567871094"></path>
|
||||
</g>
|
||||
<g class="edgeLabels">
|
||||
<g class="edgeLabel">
|
||||
<g transform="translate(0, 0)" class="label">
|
||||
<foreignObject height="0" width="0">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="edgeLabel"></span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</g>
|
||||
</g>
|
||||
<g class="edgeLabel">
|
||||
<g transform="translate(0, 0)" class="label">
|
||||
<foreignObject height="0" width="0">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="edgeLabel"></span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</g>
|
||||
</g>
|
||||
<g class="edgeLabel">
|
||||
<g transform="translate(0, 0)" class="label">
|
||||
<foreignObject height="0" width="0">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="edgeLabel"></span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</g>
|
||||
</g>
|
||||
<g class="edgeLabel">
|
||||
<g transform="translate(0, 0)" class="label">
|
||||
<foreignObject height="0" width="0">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="edgeLabel"></span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
<g class="nodes">
|
||||
<g transform="translate(250.01416015625, 81.49803924560547)" id="classid-Organization-10" class="node default mermaidExternal">
|
||||
<rect height="146.99608612060547" width="174.74119567871094" y="-73.49804306030273" x="-87.37059783935547" class="outer title-state"></rect>
|
||||
<line y2="-37.49902153015137" y1="-37.49902153015137" x2="87.37059783935547" x1="-87.37059783935547" class="divider"></line>
|
||||
<line y2="6.5" y1="6.5" x2="87.37059783935547" x1="-87.37059783935547" class="divider"></line>
|
||||
<g class="label">
|
||||
<foreignObject height="0" width="0">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel"></span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -47.705074310302734, -65.99804306030273)" height="23.999021530151367" width="95.41014862060547" class="classTitle">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">Organization</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -79.87059783935547, -25.999021530151367)" height="23.999021530151367" width="129.84617614746094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">tag value [tenant]</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -79.87059783935547, 14)" height="23.999021530151367" width="100.70800018310547">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">IAM bindings()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -79.87059783935547, 41.99902153015137)" height="23.999021530151367" width="159.74119567871094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">organization policies()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(116.507080078125, 292.49363708496094)" id="classid-Tenant0-11" class="node default mermaidFolder">
|
||||
<rect height="174.99510765075684" width="174.74119567871094" y="-87.49755382537842" x="-87.37059783935547" class="outer title-state"></rect>
|
||||
<line y2="-23.499510765075684" y1="-23.499510765075684" x2="87.37059783935547" x1="-87.37059783935547" class="divider"></line>
|
||||
<line y2="-7.499510765075684" y1="-7.499510765075684" x2="87.37059783935547" x1="-87.37059783935547" class="divider"></line>
|
||||
<g class="label">
|
||||
<foreignObject transform="translate( -29.913328170776367, -79.99755382537842)" height="23.999021530151367" width="59.826656341552734">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">«folder»</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -30.157468795776367, -51.99853229522705)" height="23.999021530151367" width="60.314937591552734" class="classTitle">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">Tenant0</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -79.87059783935547, 0.0004892349243164062)" height="23.999021530151367" width="100.70800018310547">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">IAM bindings()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -79.87059783935547, 27.999510765075684)" height="23.999021530151367" width="159.74119567871094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">organization policies()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -79.87059783935547, 55.99853229522705)" height="23.999021530151367" width="98.25438690185547">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">tag bindings()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(383.521240234375, 292.49363708496094)" id="classid-Tenant1-12" class="node default mermaidFolder">
|
||||
<rect height="174.99510765075684" width="174.74119567871094" y="-87.49755382537842" x="-87.37059783935547" class="outer title-state"></rect>
|
||||
<line y2="-23.499510765075684" y1="-23.499510765075684" x2="87.37059783935547" x1="-87.37059783935547" class="divider"></line>
|
||||
<line y2="-7.499510765075684" y1="-7.499510765075684" x2="87.37059783935547" x1="-87.37059783935547" class="divider"></line>
|
||||
<g class="label">
|
||||
<foreignObject transform="translate( -29.913328170776367, -79.99755382537842)" height="23.999021530151367" width="59.826656341552734">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">«folder»</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -30.157468795776367, -51.99853229522705)" height="23.999021530151367" width="60.314937591552734" class="classTitle">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">Tenant1</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -79.87059783935547, 0.0004892349243164062)" height="23.999021530151367" width="100.70800018310547">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">IAM bindings()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -79.87059783935547, 27.999510765075684)" height="23.999021530151367" width="159.74119567871094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">organization policies()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -79.87059783935547, 55.99853229522705)" height="23.999021530151367" width="98.25438690185547">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">tag bindings()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(116.507080078125, 531.4882507324219)" id="classid-Tenant0_IaC-13" class="node default">
|
||||
<rect height="202.9941291809082" width="217.01414489746094" y="-101.4970645904541" x="-108.50707244873047" class="outer title-state"></rect>
|
||||
<line y2="-37.49902153015137" y1="-37.49902153015137" x2="108.50707244873047" x1="-108.50707244873047" class="divider"></line>
|
||||
<line y2="62.498043060302734" y1="62.498043060302734" x2="108.50707244873047" x1="-108.50707244873047" class="divider"></line>
|
||||
<g class="label">
|
||||
<foreignObject transform="translate( -34.661861419677734, -93.9970645904541)" height="23.999021530151367" width="69.32372283935547">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">«project»</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -46.221920013427734, -65.99804306030273)" height="23.999021530151367" width="92.44384002685547" class="classTitle">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">Tenant0_IaC</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -101.00707244873047, -25.999021530151367)" height="23.999021530151367" width="202.01414489746094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">service accounts [all stages]</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -101.00707244873047, 2)" height="23.999021530151367" width="197.25340270996094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">storage buckets [stage 0+1]</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -101.00707244873047, 29.999021530151367)" height="23.999021530151367" width="189.92918395996094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">optional CI/CD [stage 0+1]</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -101.00707244873047, 69.99804306030273)" height="23.999021530151367" width="100.70800018310547">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">IAM bindings()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</g>
|
||||
</g>
|
||||
<g transform="translate(383.521240234375, 531.4882507324219)" id="classid-Tenant1_IaC-14" class="node default">
|
||||
<rect height="202.9941291809082" width="217.01414489746094" y="-101.4970645904541" x="-108.50707244873047" class="outer title-state"></rect>
|
||||
<line y2="-37.49902153015137" y1="-37.49902153015137" x2="108.50707244873047" x1="-108.50707244873047" class="divider"></line>
|
||||
<line y2="62.498043060302734" y1="62.498043060302734" x2="108.50707244873047" x1="-108.50707244873047" class="divider"></line>
|
||||
<g class="label">
|
||||
<foreignObject transform="translate( -34.661861419677734, -93.9970645904541)" height="23.999021530151367" width="69.32372283935547">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">«project»</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -46.221920013427734, -65.99804306030273)" height="23.999021530151367" width="92.44384002685547" class="classTitle">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">Tenant1_IaC</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -101.00707244873047, -25.999021530151367)" height="23.999021530151367" width="202.01414489746094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">service accounts [all stages]</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -101.00707244873047, 2)" height="23.999021530151367" width="197.25340270996094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">storage buckets [stage 0+1]</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -101.00707244873047, 29.999021530151367)" height="23.999021530151367" width="189.92918395996094">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">optional CI/CD [stage 0+1]</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
<foreignObject transform="translate( -101.00707244873047, 69.99804306030273)" height="23.999021530151367" width="100.70800018310547">
|
||||
<div style="display: inline-block; white-space: nowrap;" xmlns="http://www.w3.org/1999/xhtml">
|
||||
<span class="nodeLabel">IAM bindings()</span>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
||||
</div>
|
||||
</foreignObject>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 24 KiB |
|
|
@ -12,7 +12,7 @@ Fabric FAST was initially conceived to help enterprises quickly set up a GCP org
|
|||
|
||||
### Contracts and stages
|
||||
|
||||
FAST uses the concept of stages, which individually perform precise tasks but, taken together, build a functional, ready-to-use GCP organization. More importantly, stages are modeled around the security boundaries that typically appear in mature organizations. This arrangement allows delegating ownership of each stage to the team responsible for the types of resources it manages. For example, as its name suggests, the networking stage sets up all the networking elements and is usually the responsibility of a dedicated networking team within the organization.
|
||||
FAST uses the concept of stages, which individually perform precise tasks but taken together build a functional, ready-to-use GCP organization. More importantly, stages are modeled around the security boundaries that typically appear in mature organizations. This arrangement allows delegating ownership of each stage to the team responsible for the types of resources it manages. For example, as its name suggests, the networking stage sets up all the networking elements and is usually the responsibility of a dedicated networking team within the organization.
|
||||
|
||||
From the perspective of FAST's overall design, stages also work as contacts or interfaces, defining a set of pre-requisites and inputs required to perform their designed task and generating outputs needed by other stages lower in the chain. The diagram below shows the relationships between stages.
|
||||
|
||||
|
|
@ -20,7 +20,7 @@ From the perspective of FAST's overall design, stages also work as contacts or i
|
|||
<img src="stages.svg" alt="Stages diagram">
|
||||
</p>
|
||||
|
||||
Please refer to the [stages](./stages/) section for further details on each stage.
|
||||
Please refer to the [stages](./stages/) section for further details on each stage. For details on tenant-level stages which introduce a deeper level of autonomy via nested FAST setups rooted in a top-level folder, refer to the [multitenant stages](#multitenant-organizations) section below.
|
||||
|
||||
### Security-first design
|
||||
|
||||
|
|
@ -32,11 +32,21 @@ FAST also aims to minimize the number of permissions granted to principals accor
|
|||
|
||||
A resource factory consumes a simple representation of a resource (e.g., in YAML) and deploys it (e.g., using Terraform). Used correctly, factories can help decrease the management overhead of large-scale infrastructure deployments. See "[Resource Factories: A descriptive approach to Terraform](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c)" for more details and the rationale behind factories.
|
||||
|
||||
FAST uses YAML-based factories to deploy subnets and firewall rules and, as its name suggests, in the [project factory](./stages/03-project-factory/) stage.
|
||||
FAST uses YAML-based factories to deploy subnets and firewall rules and, as its name suggests, in the [project factory](./stages/3-project-factory/) stage.
|
||||
|
||||
### CI/CD
|
||||
|
||||
One of our objectives with FAST is to provide a lightweight reference design for the IaC repositories, and a built-in implementation for running our code in automated pipelines. Our CI/CD approach leverages [Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation), and provides sample workflow configurations for several major providers. Refer to the [CI/CD section in the bootstrap stage](stages/00-bootstrap/README.md#cicd) for more details. We also provide separate [optional small stages](./extras/) to help you configure your CI/CD provider.
|
||||
One of our objectives with FAST is to provide a lightweight reference design for the IaC repositories, and a built-in implementation for running our code in automated pipelines. Our CI/CD approach leverages [Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation), and provides sample workflow configurations for several major providers. Refer to the [CI/CD section in the bootstrap stage](./stages/0-bootstrap/README.md#cicd) for more details. We also provide separate [optional small stages](./extras/) to help you configure your CI/CD provider.
|
||||
|
||||
### Multitenant organizations
|
||||
|
||||
FAST has built-in support for complex multitenant organizations, where each tenant has complete control over a separate hierarchy rooted in a top-level folder. This approach is particularly suited for large enterprises or governments, where country-level subsidiaries or government agencies have a wide degree of autonomy within a shared GCP organization managed by a central entity.
|
||||
|
||||
FAST implements multitenancy via [dedicated stages](stages-multitenant) for tenant-level bootstrap and resource management, which configure separate hierarchies within the organization rooted in top-level folders, so that subsequent FAST stages (networking, security, data, etc.) can be used directly for each tenant. The diagram below shows the relationships between organization-level and tenant-level stages.
|
||||
|
||||
<p align="center">
|
||||
<img src="stages-multitenant/stages.svg" alt="Stages diagram">
|
||||
</p>
|
||||
|
||||
## Implementation
|
||||
|
||||
|
|
@ -57,9 +67,9 @@ Those familiar with Python will note that FAST follows many of the maxims in the
|
|||
|
||||
## Roadmap
|
||||
|
||||
Besides the features already described, FAST roadmap includes:
|
||||
Besides the features already described, FAST also includes:
|
||||
|
||||
- Stage to deploy environment-specific multitenant GKE clusters following Google's best practices
|
||||
- Stage to deploy a fully featured data platform
|
||||
- Reference implementation to use FAST in CI/CD pipelines (in progress)
|
||||
- Static policy enforcement
|
||||
- Reference implementation to use FAST in CI/CD pipelines
|
||||
- Static policy enforcement (planned)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,139 @@
|
|||
# FAST GitHub repository management
|
||||
|
||||
This small extra stage allows creating and populating GitHub repositories used to host FAST stage code, including rewriting of module sources and secrets used for private modules repository access.
|
||||
|
||||
It is designed for use in a GitHub organization, and is only meant as a one-shot solution with perishable state especially when used for initial population, as you don't want Terraform to keep overwriting your changes with initial versions of files.
|
||||
|
||||
Initial population is only meant to be used with actual stage, while populating the modules repository should be done by hand to avoid hitting the GitHub hourly limit for their API.
|
||||
|
||||
Once initial population is done, you need to manually push to the repository
|
||||
|
||||
- the `.tfvars` file with custom variable values for your stages
|
||||
- the workflow configuration file generated by FAST stages
|
||||
|
||||
## GitHub provider credentials
|
||||
|
||||
A [GitHub token](https://github.com/settings/tokens) is needed to authenticate against their API. The token needs organization-level permissions, like shown in this screenshot:
|
||||
|
||||
<p align="center">
|
||||
<img src="github_token.png" alt="GitHub token scopes.">
|
||||
</p>
|
||||
|
||||
Once a token is available set it in the `GITHUB_TOKEN` environment variable before running Terraform.
|
||||
|
||||
## Variable configuration
|
||||
|
||||
The `organization` required variable sets the GitHub organization where repositories will be created, and is used to configure the Terraform provider.
|
||||
|
||||
### Modules repository and sources
|
||||
|
||||
The `modules_config` variable controls creation and management of the key and secret used to access the private modules repository, and indirectly control population of initial files: if the `modules_config` variable is not specified no module repository is know to the code, so module source paths cannot be replaced, and initial population of files cannot happen. If the variable is specified, an optional `source_ref` attribute can be set to the reference used to pin modules versions.
|
||||
|
||||
This is an example that configures the modules repository name and an optional reference, enabling initial population of repositories where the feature has been turned on:
|
||||
|
||||
```hcl
|
||||
modules_config = {
|
||||
repository_name = "GoogleCloudPlatform/cloud-foundation-fabric"
|
||||
source_ref = "v19.0.0"
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
In the above example, no key options are set so it's assumed modules will be fetched from a public repository. If modules repository authentication is needed the `key_config` attribute also needs to be set.
|
||||
|
||||
If no keypair path is specified an internally generated key will be stored as an access key in the modules repository, and as secrets in the stage repositories:
|
||||
|
||||
```hcl
|
||||
modules_config = {
|
||||
repository_name = "GoogleCloudPlatform/cloud-foundation-fabric"
|
||||
key_config = {
|
||||
create_key = true
|
||||
create_secrets = true
|
||||
}
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
To use an existing keypair pass the path to the private key, the public key name is assumed to have the same name ending with the `.pub` suffix. This is useful in cases where the access key has already been set in the modules repository, and new repositories need to be created and their corresponding secret set:
|
||||
|
||||
```hcl
|
||||
modules_config = {
|
||||
repository_name = "GoogleCloudPlatform/cloud-foundation-fabric"
|
||||
key_config = {
|
||||
create_secrets = true
|
||||
keypair_path = "~/modules-repository-key"
|
||||
}
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
### Repositories
|
||||
|
||||
The `repositories` variable is where you configure which repositories to create and whether initial population of files is desired.
|
||||
|
||||
This is an example that creates repositories for stages 00 and 01, and populates initial files for stages 00, 01, and 02:
|
||||
|
||||
```tfvars
|
||||
repositories = {
|
||||
fast_00_bootstrap = {
|
||||
create_options = {
|
||||
description = "FAST bootstrap."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
populate_from = "../../stages/0-bootstrap"
|
||||
}
|
||||
fast_01_resman = {
|
||||
create_options = {
|
||||
description = "FAST resource management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
populate_from = "../../stages/1-resman"
|
||||
}
|
||||
fast_02_networking = {
|
||||
populate_from = "../../stages/2-networking-peering"
|
||||
}
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
The `create_options` repository attribute controls creation: if the attribute is not present, the repository is assumed to be already existing.
|
||||
|
||||
Initial population depends on a modules repository being configured in the `modules_config` variable described in the preceding section and on the`populate_from` attributes in each repository where population is required, which point to the folder holding the files to be committed.
|
||||
|
||||
### Commit configuration
|
||||
|
||||
Finally, a `commit_config` variable is optional: it can be used to configure author, email and message used in commits for initial population of files, its defaults are probably fine for most use cases.
|
||||
|
||||
<!-- TFDOC OPTS files:1 -->
|
||||
<!-- BEGIN TFDOC -->
|
||||
|
||||
## Files
|
||||
|
||||
| name | description | resources |
|
||||
|---|---|---|
|
||||
| [cicd-versions.tf](./cicd-versions.tf) | Provider version. | |
|
||||
| [main.tf](./main.tf) | Module-level locals and resources. | <code>github_actions_secret</code> · <code>github_repository</code> · <code>github_repository_deploy_key</code> · <code>github_repository_file</code> · <code>tls_private_key</code> |
|
||||
| [outputs.tf](./outputs.tf) | Module outputs. | |
|
||||
| [providers.tf](./providers.tf) | Provider configuration. | |
|
||||
| [variables.tf](./variables.tf) | Module variables. | |
|
||||
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---:|:---:|:---:|
|
||||
| [organization](variables.tf#L50) | GitHub organization. | <code>string</code> | ✓ | |
|
||||
| [commmit_config](variables.tf#L17) | Configure commit metadata. | <code title="object({ author = optional(string, "FAST loader") email = optional(string, "fast-loader@fast.gcp.tf") message = optional(string, "FAST initial loading") })">object({…})</code> | | <code>{}</code> |
|
||||
| [modules_config](variables.tf#L28) | Configure access to repository module via key, and replacement for modules sources in stage repositories. | <code title="object({ repository_name = string source_ref = optional(string) key_config = optional(object({ create_key = optional(bool, false) create_secrets = optional(bool, false) keypair_path = optional(string) }), {}) })">object({…})</code> | | <code>null</code> |
|
||||
| [repositories](variables.tf#L55) | Repositories to create. | <code title="map(object({ create_options = optional(object({ allow = optional(object({ auto_merge = optional(bool) merge_commit = optional(bool) rebase_merge = optional(bool) squash_merge = optional(bool) })) auto_init = optional(bool) description = optional(string) features = optional(object({ issues = optional(bool) projects = optional(bool) wiki = optional(bool) })) templates = optional(object({ gitignore = optional(string, "Terraform") license = optional(string) repository = optional(object({ name = string owner = string })) }), {}) visibility = optional(string, "private") })) populate_from = optional(string) }))">map(object({…}))</code> | | <code>{}</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| [clone](outputs.tf#L17) | Clone repository commands. | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
|
Before Width: | Height: | Size: 55 KiB After Width: | Height: | Size: 55 KiB |
|
|
@ -1,5 +1,5 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
|
|
@ -15,9 +15,6 @@
|
|||
*/
|
||||
|
||||
locals {
|
||||
_modules_repository = [
|
||||
for k, v in var.repositories : local.repositories[k] if v.has_modules
|
||||
]
|
||||
_repository_files = flatten([
|
||||
for k, v in var.repositories : [
|
||||
for f in concat(
|
||||
|
|
@ -30,12 +27,12 @@ locals {
|
|||
}
|
||||
] if v.populate_from != null
|
||||
])
|
||||
modules_ref = var.modules_ref == null ? "" : "?ref=${var.modules_ref}"
|
||||
modules_repository = (
|
||||
length(local._modules_repository) > 0
|
||||
? local._modules_repository.0
|
||||
: null
|
||||
modules_ref = (
|
||||
try(var.modules_config.source_ref, null) == null
|
||||
? ""
|
||||
: "?ref=${var.modules_config.source_ref}"
|
||||
)
|
||||
modules_repo = try(var.modules_config.repository_name, null)
|
||||
repositories = {
|
||||
for k, v in var.repositories :
|
||||
k => v.create_options == null ? k : github_repository.default[k].name
|
||||
|
|
@ -56,6 +53,15 @@ locals {
|
|||
name = "templates/providers.tf.tpl"
|
||||
}
|
||||
if v.populate_from != null
|
||||
},
|
||||
{
|
||||
for k, v in var.repositories :
|
||||
"${k}/templates/workflow-github.yaml" => {
|
||||
repository = k
|
||||
file = "../../assets/templates/workflow-github.yaml"
|
||||
name = "templates/workflow-github.yaml"
|
||||
}
|
||||
if v.populate_from != null
|
||||
}
|
||||
)
|
||||
}
|
||||
|
|
@ -96,41 +102,49 @@ resource "github_repository" "default" {
|
|||
}
|
||||
|
||||
resource "tls_private_key" "default" {
|
||||
count = local.modules_repository != null ? 1 : 0
|
||||
algorithm = "ED25519"
|
||||
}
|
||||
|
||||
resource "github_repository_deploy_key" "default" {
|
||||
count = local.modules_repository == null ? 0 : 1
|
||||
count = (
|
||||
try(var.modules_config.key_config.create_key, null) == true ? 1 : 0
|
||||
)
|
||||
title = "Modules repository access"
|
||||
repository = local.modules_repository
|
||||
key = tls_private_key.default.0.public_key_openssh
|
||||
read_only = true
|
||||
repository = local.modules_repo
|
||||
key = (
|
||||
try(var.modules_config.key_config.keypair_path, null) == null
|
||||
? tls_private_key.default.public_key_openssh
|
||||
: file(pathexpand("${var.modules_config.key_config.keypair_path}.pub"))
|
||||
)
|
||||
read_only = true
|
||||
}
|
||||
|
||||
resource "github_actions_secret" "default" {
|
||||
for_each = local.modules_repository == null ? {} : {
|
||||
for k, v in local.repositories :
|
||||
k => v if k != local.modules_repository
|
||||
}
|
||||
repository = each.key
|
||||
secret_name = "CICD_MODULES_KEY"
|
||||
plaintext_value = tls_private_key.default.0.private_key_openssh
|
||||
for_each = (
|
||||
try(var.modules_config.key_config.create_secrets, null) == true
|
||||
? local.repositories
|
||||
: {}
|
||||
)
|
||||
repository = each.key
|
||||
secret_name = "CICD_MODULES_KEY"
|
||||
plaintext_value = (
|
||||
try(var.modules_config.key_config.keypair_path, null) == null
|
||||
? tls_private_key.default.private_key_openssh
|
||||
: file(pathexpand("${var.modules_config.key_config.keypair_path}"))
|
||||
)
|
||||
}
|
||||
|
||||
resource "github_repository_file" "default" {
|
||||
for_each = (
|
||||
local.modules_repository == null ? {} : local.repository_files
|
||||
)
|
||||
for_each = local.modules_repo == null ? {} : local.repository_files
|
||||
repository = local.repositories[each.value.repository]
|
||||
branch = "main"
|
||||
file = each.value.name
|
||||
content = (
|
||||
endswith(each.value.name, ".tf") && local.modules_repository != null
|
||||
endswith(each.value.name, ".tf") && local.modules_repo != null
|
||||
? replace(
|
||||
file(each.value.file),
|
||||
"/source\\s*=\\s*\"../../../modules/([^/\"]+)\"/",
|
||||
"source = \"git@github.com:${var.organization}/${local.modules_repository}.git//$1${local.modules_ref}\"" # "
|
||||
"source = \"git@github.com:${local.modules_repo}.git//$1${local.modules_ref}\"" # "
|
||||
)
|
||||
: file(each.value.file)
|
||||
)
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
|
|
@ -25,10 +25,26 @@ variable "commmit_config" {
|
|||
nullable = false
|
||||
}
|
||||
|
||||
variable "modules_ref" {
|
||||
description = "Optional git ref used in module sources."
|
||||
type = string
|
||||
default = null
|
||||
variable "modules_config" {
|
||||
description = "Configure access to repository module via key, and replacement for modules sources in stage repositories."
|
||||
type = object({
|
||||
repository_name = string
|
||||
source_ref = optional(string)
|
||||
key_config = optional(object({
|
||||
create_key = optional(bool, false)
|
||||
create_secrets = optional(bool, false)
|
||||
keypair_path = optional(string)
|
||||
}), {})
|
||||
})
|
||||
default = null
|
||||
validation {
|
||||
condition = (
|
||||
var.modules_config == null
|
||||
||
|
||||
try(var.modules_config.repository_name, null) != null
|
||||
)
|
||||
error_message = "Modules configuration requires a modules repository name."
|
||||
}
|
||||
}
|
||||
|
||||
variable "organization" {
|
||||
|
|
@ -63,7 +79,6 @@ variable "repositories" {
|
|||
}), {})
|
||||
visibility = optional(string, "private")
|
||||
}))
|
||||
has_modules = optional(bool, false)
|
||||
populate_from = optional(string)
|
||||
}))
|
||||
default = {}
|
||||
|
|
@ -1,105 +0,0 @@
|
|||
# FAST GitHub repository management
|
||||
|
||||
This small extra stage allows creation and management of GitHub repositories used to host FAST stage code, including initial population of files and rewriting of module sources.
|
||||
|
||||
This stage is designed for quick repository creation in a GitHub organization, and is not suited for medium or long-term repository management especially if you enable initial population of files.
|
||||
|
||||
## Initial population caveats
|
||||
|
||||
Initial file population of repositories is controlled via the `populate_from` attribute, and needs a bit of care:
|
||||
|
||||
- never run this stage with the same variables used for population once the repository starts being used, as **Terraform will manage file state and revert any changes at each apply**, which is probably not what you want.
|
||||
- initial population of the modules repository is discouraged, as the number of resulting files Terraform needs to manage is very close to the GitHub hourly limit for their API, it's much easier to populate modules via regular git commands
|
||||
|
||||
The scenario for which this stage has been designed is one-shot creation and/or population of stage repositories, running it multiple times with different variables and Terraform states if incremental creation is needed for subsequent FAST stages (e.g. GKE, data platform, etc.).
|
||||
|
||||
Once initial population is done, you need to manually push to the repository
|
||||
|
||||
- the `.tfvars` file with custom variable values for your stages
|
||||
- the workflow configuration file generated by FAST stages
|
||||
|
||||
## GitHub provider credentials
|
||||
|
||||
A [GitHub token](https://github.com/settings/tokens) is needed to authenticate against their API. The token needs organization-level permissions, like shown in this screenshot:
|
||||
|
||||
<p align="center">
|
||||
<img src="github_token.png" alt="GitHub token scopes.">
|
||||
</p>
|
||||
|
||||
## Variable configuration
|
||||
|
||||
The `organization` required variable sets the GitHub organization where repositories will be created, and is used to configure the Terraform provider.
|
||||
|
||||
The `repositories` variable is where you configure which repositories to create, whether initial population of files is desired, and which repository is used to host modules.
|
||||
|
||||
This is an example that creates repositories for stages 00 and 01, defines an existing repositories as the source for modules, and populates initial files for stages 00, 01, and 02:
|
||||
|
||||
```tfvars
|
||||
organization = "ludomagno"
|
||||
repositories = {
|
||||
fast_00_bootstrap = {
|
||||
create_options = {
|
||||
description = "FAST bootstrap."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
populate_from = "../../stages/00-bootstrap"
|
||||
}
|
||||
fast_01_resman = {
|
||||
create_options = {
|
||||
description = "FAST resource management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
populate_from = "../../stages/01-resman"
|
||||
}
|
||||
fast_02_networking = {
|
||||
populate_from = "../../stages/02-networking-peering"
|
||||
}
|
||||
fast_modules = {
|
||||
has_modules = true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `create_options` repository attribute controls creation: if the attribute is not present, the repository is assumed to be already existing.
|
||||
|
||||
Initial population depends on a modules repository being configured, identified by the `has_modules` attribute, and on `populate_from` attributes in each repository where population is required, pointing to the folder holding the files to be committed.
|
||||
|
||||
Finally, a `commit_config` variable is optional: it can be used to configure author, email and message used in commits for initial population of files, its defaults are probably fine for most use cases.
|
||||
|
||||
## Modules secret
|
||||
|
||||
When initial population is configured for a repository, this stage also adds a secret with the private key used to authenticate against the modules repository. This matches the configuration of the GitHub workflow files created for each FAST stage when CI/CD is enabled.
|
||||
|
||||
<!-- TFDOC OPTS files:1 -->
|
||||
<!-- BEGIN TFDOC -->
|
||||
|
||||
## Files
|
||||
|
||||
| name | description | resources |
|
||||
|---|---|---|
|
||||
| [cicd-versions.tf](./cicd-versions.tf) | Provider version. | |
|
||||
| [main.tf](./main.tf) | Module-level locals and resources. | <code>github_actions_secret</code> · <code>github_repository</code> · <code>github_repository_deploy_key</code> · <code>github_repository_file</code> · <code>tls_private_key</code> |
|
||||
| [outputs.tf](./outputs.tf) | Module outputs. | |
|
||||
| [providers.tf](./providers.tf) | Provider configuration. | |
|
||||
| [variables.tf](./variables.tf) | Module variables. | |
|
||||
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---:|:---:|:---:|
|
||||
| [organization](variables.tf#L34) | GitHub organization. | <code>string</code> | ✓ | |
|
||||
| [commmit_config](variables.tf#L17) | Configure commit metadata. | <code title="object({ author = optional(string, "FAST loader") email = optional(string, "fast-loader@fast.gcp.tf") message = optional(string, "FAST initial loading") })">object({…})</code> | | <code>{}</code> |
|
||||
| [modules_ref](variables.tf#L28) | Optional git ref used in module sources. | <code>string</code> | | <code>null</code> |
|
||||
| [repositories](variables.tf#L39) | Repositories to create. | <code title="map(object({ create_options = optional(object({ allow = optional(object({ auto_merge = optional(bool) merge_commit = optional(bool) rebase_merge = optional(bool) squash_merge = optional(bool) })) auto_init = optional(bool) description = optional(string) features = optional(object({ issues = optional(bool) projects = optional(bool) wiki = optional(bool) })) templates = optional(object({ gitignore = optional(string, "Terraform") license = optional(string) repository = optional(object({ name = string owner = string })) }), {}) visibility = optional(string, "private") })) has_modules = optional(bool, false) populate_from = optional(string) }))">map(object({…}))</code> | | <code>{}</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| [clone](outputs.tf#L17) | Clone repository commands. | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -2,4 +2,4 @@
|
|||
|
||||
This folder contains additional helper stages for FAST, which can be used to simplify specific operational tasks:
|
||||
|
||||
- [GitHub repository management](./00-cicd-github/)
|
||||
- [GitHub repository management](./0-cicd-github/)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,114 @@
|
|||
#!/bin/bash
|
||||
# Copyright 2023 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
if [ $# -eq 0 ]; then
|
||||
echo "Error: no folder or GCS bucket specified. Use -h or --help for usage."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$1" == "-h" || "$1" == "--help" ]]; then
|
||||
cat <<END
|
||||
Create commands to initialize stage provider and tfvars files.
|
||||
|
||||
Usage with GCS output files bucket:
|
||||
stage-links.sh GCS_BUCKET_URI
|
||||
|
||||
Usage with local output files folder:
|
||||
stage-links.sh FOLDER_PATH
|
||||
END
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ "$1" == "gs://"* ]]; then
|
||||
CMD="gcloud alpha storage cp $1"
|
||||
CP_CMD=$CMD
|
||||
elif [ ! -d "$1" ]; then
|
||||
echo "folder $1 not found"
|
||||
exit 1
|
||||
else
|
||||
CMD="ln -s $1"
|
||||
CP_CMD="cp $1"
|
||||
fi
|
||||
|
||||
GLOBALS="tfvars/globals.auto.tfvars.json"
|
||||
PROVIDER_CMD=$CMD
|
||||
STAGE_NAME=$(basename "$(pwd)")
|
||||
|
||||
case $STAGE_NAME in
|
||||
|
||||
"0-bootstrap")
|
||||
unset GLOBALS
|
||||
PROVIDER="providers/0-bootstrap-providers.tf"
|
||||
TFVARS=""
|
||||
;;
|
||||
"0-bootstrap-tenant")
|
||||
MESSAGE="remember to set the prefix in the provider file"
|
||||
PROVIDER_CMD=$CP_CMD
|
||||
PROVIDER="providers/0-bootstrap-tenant-providers.tf"
|
||||
TFVARS="tfvars/0-bootstrap.auto.tfvars.json
|
||||
tfvars/1-resman.auto.tfvars.json"
|
||||
;;
|
||||
"1-resman")
|
||||
PROVIDER="providers/${STAGE_NAME}-providers.tf"
|
||||
TFVARS="tfvars/0-bootstrap.auto.tfvars.json"
|
||||
;;
|
||||
"1-resman-tenant")
|
||||
if [[ -z "$TENANT" ]]; then
|
||||
echo "Please set a \$TENANT variable with the tenant shortname"
|
||||
exit 1
|
||||
fi
|
||||
unset GLOBALS
|
||||
PROVIDER="tenants/${TENANT}/providers/1-resman-tenant-providers.tf"
|
||||
TFVARS="tenants/${TENANT}/tfvars/0-bootstrap-tenant.auto.tfvars.json"
|
||||
;;
|
||||
"2-"*)
|
||||
PROVIDER="providers/multitenant/${STAGE_NAME}-providers.tf"
|
||||
TFVARS="tfvars/0-bootstrap.auto.tfvars.json
|
||||
tfvars/1-resman.auto.tfvars.json"
|
||||
;;
|
||||
*)
|
||||
# check for a "dev" stage 3
|
||||
echo "no stage found, trying for parent stage 3..."
|
||||
STAGE_NAME=$(basename $(dirname "$(pwd)"))
|
||||
if [[ "$STAGE_NAME" == "3-"* ]]; then
|
||||
PROVIDER="providers/${STAGE_NAME}-providers.tf"
|
||||
TFVARS="tfvars/0-bootstrap.auto.tfvars.json
|
||||
tfvars/1-resman.auto.tfvars.json
|
||||
tfvars/2-networking.auto.tfvars.json
|
||||
tfvars/2-security.auto.tfvars.json"
|
||||
else
|
||||
echo "stage '$STAGE_NAME' not found"
|
||||
fi
|
||||
;;
|
||||
|
||||
esac
|
||||
|
||||
echo -e "# copy and paste the following commands for '$STAGE_NAME'\n"
|
||||
|
||||
echo "$PROVIDER_CMD/$PROVIDER ./"
|
||||
|
||||
# if [[ -v GLOBALS ]]; then
|
||||
# OSX uses an old bash version
|
||||
if [[ ! -z ${GLOBALS+x} ]]; then
|
||||
echo "$CMD/$GLOBALS ./"
|
||||
fi
|
||||
|
||||
for f in $TFVARS; do
|
||||
echo "$CMD/$f ./"
|
||||
done
|
||||
|
||||
if [[ ! -z ${MESSAGE+x} ]]; then
|
||||
echo -e "\n# ---> $MESSAGE <---"
|
||||
fi
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
# IAM bindings reference
|
||||
|
||||
Legend: <code>+</code> additive, <code>•</code> conditional.
|
||||
|
||||
## Organization <i>[org_id #0]</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-admins</b><br><small><i>group</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code><br>[roles/resourcemanager.organizationViewer](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.organizationViewer) <code>+</code>|
|
||||
|<b>tn0-gke-dev-0</b><br><small><i>serviceAccount</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code>|
|
||||
|<b>tn0-gke-prod-0</b><br><small><i>serviceAccount</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code>|
|
||||
|<b>tn0-networking-0</b><br><small><i>serviceAccount</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code>|
|
||||
|<b>tn0-pf-dev-0</b><br><small><i>serviceAccount</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code>|
|
||||
|<b>tn0-pf-prod-0</b><br><small><i>serviceAccount</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code>|
|
||||
|<b>tn0-resman-0</b><br><small><i>serviceAccount</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code>|
|
||||
|<b>tn0-sandbox-0</b><br><small><i>serviceAccount</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code>|
|
||||
|<b>tn0-security-0</b><br><small><i>serviceAccount</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code>|
|
||||
|<b>tn0-teams-0</b><br><small><i>serviceAccount</i></small>|[roles/orgpolicy.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#orgpolicy.policyAdmin) <code>+</code><code>•</code>|
|
||||
|
||||
## Folder <i>test tenant 0 [#1]</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-admins</b><br><small><i>group</i></small>|[roles/compute.xpnAdmin](https://cloud.google.com/iam/docs/understanding-roles#compute.xpnAdmin) <br>[roles/logging.admin](https://cloud.google.com/iam/docs/understanding-roles#logging.admin) <br>[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) <br>[roles/resourcemanager.folderAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.folderAdmin) <br>[roles/resourcemanager.projectCreator](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectCreator) |
|
||||
|<b>tn0-networking-0</b><br><small><i>serviceAccount</i></small>|[roles/compute.xpnAdmin](https://cloud.google.com/iam/docs/understanding-roles#compute.xpnAdmin) |
|
||||
|<b>tn0-resman-0</b><br><small><i>serviceAccount</i></small>|[roles/compute.xpnAdmin](https://cloud.google.com/iam/docs/understanding-roles#compute.xpnAdmin) <br>[roles/logging.admin](https://cloud.google.com/iam/docs/understanding-roles#logging.admin) <br>[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) <br>[roles/resourcemanager.folderAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.folderAdmin) <br>[roles/resourcemanager.projectCreator](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectCreator) |
|
||||
|
||||
## Project <i>prod-iac-core-0</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-bootstrap-1</b><br><small><i>serviceAccount</i></small>|[roles/logging.logWriter](https://cloud.google.com/iam/docs/understanding-roles#logging.logWriter) <code>+</code>|
|
||||
|
||||
## Project <i>tn0-audit-logs-0</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>f260055713332-284719</b><br><small><i>serviceAccount</i></small>|[roles/logging.bucketWriter](https://cloud.google.com/iam/docs/understanding-roles#logging.bucketWriter) <code>+</code><code>•</code>|
|
||||
|<b>prod-resman-0</b><br><small><i>serviceAccount</i></small>|[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) |
|
||||
|<b>tn0-resman-0</b><br><small><i>serviceAccount</i></small>|[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) |
|
||||
|
||||
## Project <i>tn0-iac-core-0</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-admins</b><br><small><i>group</i></small>|[roles/iam.serviceAccountTokenCreator](https://cloud.google.com/iam/docs/understanding-roles#iam.serviceAccountTokenCreator) <br>[roles/iam.workloadIdentityPoolAdmin](https://cloud.google.com/iam/docs/understanding-roles#iam.workloadIdentityPoolAdmin) |
|
||||
|<b>SERVICE_IDENTITY_service-networking</b><br><small><i>serviceAccount</i></small>|[roles/servicenetworking.serviceAgent](https://cloud.google.com/iam/docs/understanding-roles#servicenetworking.serviceAgent) <code>+</code>|
|
||||
|<b>prod-resman-0</b><br><small><i>serviceAccount</i></small>|[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) |
|
||||
|<b>tn0-resman-0</b><br><small><i>serviceAccount</i></small>|[roles/cloudbuild.builds.editor](https://cloud.google.com/iam/docs/understanding-roles#cloudbuild.builds.editor) <br>[roles/iam.serviceAccountAdmin](https://cloud.google.com/iam/docs/understanding-roles#iam.serviceAccountAdmin) <br>[roles/iam.workloadIdentityPoolAdmin](https://cloud.google.com/iam/docs/understanding-roles#iam.workloadIdentityPoolAdmin) <br>[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) <br>[roles/source.admin](https://cloud.google.com/iam/docs/understanding-roles#source.admin) <br>[roles/storage.admin](https://cloud.google.com/iam/docs/understanding-roles#storage.admin) |
|
||||
|
|
@ -0,0 +1,210 @@
|
|||
# Tenant bootstrap
|
||||
|
||||
The primary purpose of this stage is to decouple a single tenant from centrally managed resources in the organization, so that subsequent management of the tenant's own hierarchy and resources can be implemented with a high degree of autonomy.
|
||||
|
||||
It is logically equivalent to organization-level bootstrap as it's concerned with setting up IAM bindings on a root node and creating supporting projects attached to it, but it depends on the organization-level resource management stage and uses the same service account and permissions since it operates at the hierarchy level (folders, tags, organization policies).
|
||||
|
||||
The resources and policies managed here are:
|
||||
|
||||
- the tag value in the `tenant` key used in IAM conditions
|
||||
- the billing IAM bindings for the tenant-specific automation service accounts
|
||||
- the organization-level IAM binding that allows conditional managing of org policies on the tenant folder
|
||||
- the top-level tenant folder which acts as the root of the tenant's hierarchy
|
||||
- any organization policy that needs to be set for the tenant on its root folder
|
||||
- the tenant automation and logging projects
|
||||
- service accounts for all tenant stages
|
||||
- GCS buckets for bootstrap and resource management state
|
||||
- optional CI/CD setup for this and the resource management tenant stages
|
||||
- tenant-specific Workload Identity Federation pool and providers (planned)
|
||||
|
||||
One notable difference compared to organization-level bootstrap is the creation of service accounts for all tenant stages: this is done here so that Billing and Organization Policy Admin bindings can be set, leveraging permissions of the org-level resman service account which is used to run this stage. Doing this here avoids the need to grant broad scoped permissions on the organization to tenant-level service accounts, and effectively decouples the tenant from the organization.
|
||||
|
||||
The following diagram is a high level reference of what this stage manages, showing one hypothetical tenant (additional tenants require additional instances of this stage being deployed):
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base'}}%%
|
||||
classDiagram
|
||||
Organization~🏢~ -- Tenant 0~📁~
|
||||
Tenant 0~📁~ -- tn0_automation
|
||||
Tenant 0~📁~ -- tn0_logging
|
||||
class Organization~🏢~ {
|
||||
- tag value
|
||||
- IAM bindings()
|
||||
- org policies()
|
||||
}
|
||||
class Tenant 0~📁~ {
|
||||
- log sinks
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class tn0_automation {
|
||||
- GCS buckets
|
||||
- service accounts
|
||||
- optional CI/CD
|
||||
- IAM bindings()
|
||||
}
|
||||
class tn0_logging {
|
||||
- log sink destinations
|
||||
}
|
||||
```
|
||||
|
||||
As most of the features of this stage follow the same design and configurations of the [organization-level bootstrap stage](../../stages/0-bootstrap/), we will only focus on the tenant-specific configuration in this document.
|
||||
|
||||
## Naming
|
||||
|
||||
This stage sets the prefix used to name tenant resources, and passes it downstream to the other tenant stages together with the other globals needed by the tenant. The default is to append the tenant short name (a 3 or 4 letter acronym or abbreviation) to the organization-level prefix, if that is not desired this can be changed by editing local definitions in the `main.tf` file. Just be aware that some resources have name length constraints.
|
||||
|
||||
## How to run this stage
|
||||
|
||||
The tenant bootstrap stage is the effective boundary between organization and tenant-level resources: it uses the same inputs as the organization-level resource management stage, and produces outputs which provide the needed context to all other tenant stages.
|
||||
|
||||
### Output files and cross-stage variables
|
||||
|
||||
As mentioned above, the organization-level set of output files are used here with one exception: the provider file is different since state is specific to this stage. The `stage-links.sh` script can be used to get the commands needed for the provider and output files, just pass a single argument with your FAST output files folder path, or GCS bucket URI:
|
||||
|
||||
```bash
|
||||
../../stage-links.sh ~/fast-config
|
||||
```
|
||||
|
||||
The script output can be copy/pasted to a terminal:
|
||||
|
||||
```bash
|
||||
# copy and paste the following commands for '0-bootstrap-tenant'
|
||||
|
||||
cp ~/fast-config/providers/0-bootstrap-tenant-providers.tf ./
|
||||
ln -s ~/fast-config/tfvars/globals.auto.tfvars.json ./
|
||||
ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json ./
|
||||
ln -s ~/fast-config/tfvars/1-resman.auto.tfvars.json ./
|
||||
|
||||
# ---> remember to set the prefix in the provider file <---
|
||||
```
|
||||
|
||||
As shown in the script output above, the provider file is a template used as a source for potentially multiple tenant installations, so it needs to be specifically configured for this tenant by setting the backend `prefix` to a unique string so that the Terraform state file will not overlap with other tenants. Open it in an editor and perform the change before proceeding.
|
||||
|
||||
### Global overrides
|
||||
|
||||
The globals variable file linekd above contains definition which were set for the organization, for example the locations used for log sink destinations. These might not be correct for each tenant, so this stage allows overriding them via the tenant configuration variable described in the next section.
|
||||
|
||||
### Tenant-level configuration
|
||||
|
||||
The tenant configuration resides in the `tenant_config` variable, this is an example configuration for a tenant with comments explaining the different choices that need to be made:
|
||||
|
||||
```hcl
|
||||
tenant_config = {
|
||||
# used for the top-level folder name
|
||||
descriptive_name = "My First Tenant"
|
||||
# tenant-specific groups, only the admin group is required
|
||||
# the organization domain is automatically added after the group name
|
||||
groups = {
|
||||
gcp-admins = "tn01-admins"
|
||||
# gcp-devops = "tn01-devops"
|
||||
# gcp-network-admins = "tn01-networking"
|
||||
# gcp-security-admins = "tn01-security"
|
||||
}
|
||||
# the 3 or 4 letter acronym or abbreviation used in resource names
|
||||
short_name = "tn01"
|
||||
# optional CI/CD configuration, refer to the org-level stages for information
|
||||
# cicd = {
|
||||
# branch = null
|
||||
# identity_provider = "foo-provider"
|
||||
# name = "myorg/tn01-bootstrap"
|
||||
# type = "github"
|
||||
# }
|
||||
# optional group-level IAM bindings to add to the top-level folder
|
||||
# group_iam = {
|
||||
# tn01-support = ["roles/viewer"]
|
||||
# }
|
||||
# optional IAM bindings to add to the top-level folder
|
||||
# iam = {
|
||||
# "roles/logging.admin" = [
|
||||
# "serviceAccount:foo@myprj.iam.gserviceaccount.com"
|
||||
# ]
|
||||
# }
|
||||
# optional location overrides to global locations
|
||||
# locations = {
|
||||
# bq = null
|
||||
# gcs = null
|
||||
# logging = null
|
||||
# pubsub = null
|
||||
# }
|
||||
# optional folder ids for automation and logging project folders, typically
|
||||
# added in later stages and entered here once created
|
||||
# project_parent_ids = {
|
||||
# automation = "folders/012345678"
|
||||
# logging = "folders/0123456789"
|
||||
# }
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
Configure the tenant variable in a tfvars file for this stage. A few minor points worth noting:
|
||||
|
||||
- the administrator group is the only one required here, specifying other groups only has the effect of populating the output file with group names for reuse in later stages
|
||||
- the `iam` variable is merged with the IAM bindings for service accounts in the `main.tf` file, which take precedence; if a role specified in the variable is ignored, that's probably the case
|
||||
- locations can be overridden at the attribute level, there's no need to specify those that are equal to the ones in the organization globals file
|
||||
|
||||
### Running the stage
|
||||
|
||||
Once the configuration is done just go through the usual `init/apply` cycle. On successful apply, a tfvars file specific for this tenant and a set of provider files will be created.
|
||||
|
||||
### TODO
|
||||
|
||||
- [ ] tenant-level Workload Identity Federation pool and providers configuration
|
||||
- [ ] tenant-level logging project and sinks
|
||||
|
||||
<!-- TFDOC OPTS files:1 show_extra:1 -->
|
||||
<!-- BEGIN TFDOC -->
|
||||
|
||||
## Files
|
||||
|
||||
| name | description | modules | resources |
|
||||
|---|---|---|---|
|
||||
| [automation-sas.tf](./automation-sas.tf) | Tenant automation stage 2 and 3 service accounts. | <code>iam-service-account</code> | <code>google_organization_iam_member</code> |
|
||||
| [automation.tf](./automation.tf) | Tenant automation project and resources. | <code>gcs</code> · <code>iam-service-account</code> · <code>project</code> | |
|
||||
| [billing.tf](./billing.tf) | Billing roles for standalone billing accounts. | | <code>google_billing_account_iam_member</code> |
|
||||
| [cicd.tf](./cicd.tf) | Workload Identity Federation configurations for CI/CD. | <code>iam-service-account</code> · <code>source-repository</code> | |
|
||||
| [identity-providers.tf](./identity-providers.tf) | Workload Identity Federation provider definitions. | | <code>google_iam_workload_identity_pool</code> · <code>google_iam_workload_identity_pool_provider</code> |
|
||||
| [log-export.tf](./log-export.tf) | Audit log project and sink. | <code>bigquery-dataset</code> · <code>gcs</code> · <code>logging-bucket</code> · <code>project</code> · <code>pubsub</code> | |
|
||||
| [main.tf](./main.tf) | Module-level locals and resources. | <code>folder</code> | |
|
||||
| [organization.tf](./organization.tf) | Organization tag and conditional IAM grant. | <code>organization</code> | <code>google_organization_iam_member</code> · <code>google_tags_tag_value_iam_member</code> |
|
||||
| [outputs-files.tf](./outputs-files.tf) | Output files persistence to local filesystem. | | <code>local_file</code> |
|
||||
| [outputs-gcs.tf](./outputs-gcs.tf) | Output files persistence to automation GCS bucket. | | <code>google_storage_bucket_object</code> |
|
||||
| [outputs.tf](./outputs.tf) | Module outputs. | | |
|
||||
| [variables.tf](./variables.tf) | Module variables. | | |
|
||||
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default | producer |
|
||||
|---|---|:---:|:---:|:---:|:---:|
|
||||
| [automation](variables.tf#L20) | Automation resources created by the organization-level bootstrap stage. | <code title="object({ outputs_bucket = string project_id = string project_number = string federated_identity_pool = string federated_identity_providers = map(object({ issuer = string issuer_uri = string name = string principal_tpl = string principalset_tpl = string })) })">object({…})</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [billing_account](variables.tf#L38) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | <code title="object({ id = string is_org_level = optional(bool, true) })">object({…})</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [organization](variables.tf#L193) | Organization details. | <code title="object({ domain = string id = number customer_id = string })">object({…})</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [prefix](variables.tf#L209) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [tag_keys](variables.tf#L232) | Organization tag keys. | <code title="object({ context = string environment = string tenant = string })">object({…})</code> | ✓ | | <code>1-resman</code> |
|
||||
| [tag_names](variables.tf#L243) | Customized names for resource management tags. | <code title="object({ context = string environment = string tenant = string })">object({…})</code> | ✓ | | <code>1-resman</code> |
|
||||
| [tag_values](variables.tf#L254) | Organization resource management tag values. | <code>map(string)</code> | ✓ | | <code>1-resman</code> |
|
||||
| [tenant_config](variables.tf#L261) | Tenant configuration. Short name must be 4 characters or less. | <code title="object({ descriptive_name = string groups = object({ gcp-admins = string gcp-devops = optional(string) gcp-network-admins = optional(string) gcp-security-admins = optional(string) }) short_name = string fast_features = optional(object({ data_platform = optional(bool) gke = optional(bool) project_factory = optional(bool) sandbox = optional(bool) teams = optional(bool) }), {}) locations = optional(object({ bq = optional(string) gcs = optional(string) logging = optional(string) pubsub = optional(list(string)) }), {}) })">object({…})</code> | ✓ | | |
|
||||
| [cicd_repositories](variables.tf#L51) | CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | <code title="object({ bootstrap = optional(object({ branch = optional(string) identity_provider = string name = string type = string })) resman = optional(object({ branch = optional(string) identity_provider = string name = string type = string })) })">object({…})</code> | | <code>null</code> | |
|
||||
| [custom_roles](variables.tf#L97) | Custom roles defined at the organization level, in key => id format. | <code title="object({ service_project_network_admin = string })">object({…})</code> | | <code>null</code> | <code>0-bootstrap</code> |
|
||||
| [fast_features](variables.tf#L106) | Selective control for top-level FAST features. | <code title="object({ data_platform = optional(bool, true) gke = optional(bool, true) project_factory = optional(bool, true) sandbox = optional(bool, true) teams = optional(bool, true) })">object({…})</code> | | <code>{}</code> | <code>0-bootstrap</code> |
|
||||
| [federated_identity_providers](variables.tf#L120) | Workload Identity Federation pools. The `cicd_repositories` variable references keys here. | <code title="map(object({ attribute_condition = string issuer = string custom_settings = object({ issuer_uri = string allowed_audiences = list(string) }) }))">map(object({…}))</code> | | <code>{}</code> | |
|
||||
| [group_iam](variables.tf#L134) | Tenant-level custom group IAM settings in group => [roles] format. | <code>map(list(string))</code> | | <code>{}</code> | |
|
||||
| [iam](variables.tf#L140) | Tenant-level custom IAM settings in role => [principal] format. | <code>map(list(string))</code> | | <code>{}</code> | |
|
||||
| [iam_additive](variables.tf#L146) | Tenant-level custom IAM settings in role => [principal] format for non-authoritative bindings. | <code>map(list(string))</code> | | <code>{}</code> | |
|
||||
| [locations](variables.tf#L152) | Optional locations for GCS, BigQuery, and logging buckets created here. These are the defaults set at the organization level, and can be overridden via the tenant config variable. | <code title="object({ bq = string gcs = string logging = string pubsub = list(string) })">object({…})</code> | | <code title="{ bq = "EU" gcs = "EU" logging = "global" pubsub = [] }">{…}</code> | <code>0-bootstrap</code> |
|
||||
| [log_sinks](variables.tf#L172) | Tenant-level log sinks, in name => {type, filter} format. | <code title="map(object({ filter = string type = string }))">map(object({…}))</code> | | <code title="{ audit-logs = { filter = "logName:\"/logs/cloudaudit.googleapis.com%2Factivity\" OR logName:\"/logs/cloudaudit.googleapis.com%2Fsystem_event\"" type = "logging" } }">{…}</code> | |
|
||||
| [outputs_location](variables.tf#L203) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | <code>string</code> | | <code>null</code> | |
|
||||
| [project_parent_ids](variables.tf#L219) | Optional parents for projects created here in folders/nnnnnnn format. Null values will use the tenant folder as parent. | <code title="object({ automation = string logging = string })">object({…})</code> | | <code title="{ automation = null logging = null }">{…}</code> | |
|
||||
| [test_principal](variables.tf#L301) | Used when testing to bypass the data source returning the current identity. | <code>string</code> | | <code>null</code> | |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive | consumers |
|
||||
|---|---|:---:|---|
|
||||
| [cicd_workflows](outputs.tf#L102) | CI/CD workflows for tenant bootstrap and resource management stages. | ✓ | |
|
||||
| [federated_identity](outputs.tf#L108) | Workload Identity Federation pool and providers. | | |
|
||||
| [provider](outputs.tf#L118) | Terraform provider file for tenant resource management stage. | ✓ | <code>stage-01</code> |
|
||||
| [tenant_resources](outputs.tf#L125) | Tenant-level resources. | | |
|
||||
| [tfvars](outputs.tf#L136) | Terraform variable files for the following tenant stages. | ✓ | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -0,0 +1,127 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Tenant automation stage 2 and 3 service accounts.
|
||||
|
||||
locals {
|
||||
branch_sas = {
|
||||
dp-dev = {
|
||||
condition = join(" && ", [
|
||||
"resource.matchTag('${local.tag_keys.context}', 'data')",
|
||||
"resource.matchTag('${local.tag_keys.environment}', 'development')"
|
||||
])
|
||||
description = "data platform dev"
|
||||
flag = "data_platform"
|
||||
}
|
||||
dp-prod = {
|
||||
condition = join(" && ", [
|
||||
"resource.matchTag('${local.tag_keys.context}', 'data')",
|
||||
"resource.matchTag('${local.tag_keys.environment}', 'production')"
|
||||
])
|
||||
description = "data platform prod"
|
||||
flag = "data_platform"
|
||||
}
|
||||
gke-dev = {
|
||||
condition = join(" && ", [
|
||||
"resource.matchTag('${local.tag_keys.context}', 'gke')",
|
||||
"resource.matchTag('${local.tag_keys.environment}', 'development')"
|
||||
])
|
||||
description = "GKE dev"
|
||||
flag = "gke"
|
||||
}
|
||||
gke-prod = {
|
||||
condition = join(" && ", [
|
||||
"resource.matchTag('${local.tag_keys.context}', 'gke')",
|
||||
"resource.matchTag('${local.tag_keys.environment}', 'production')"
|
||||
])
|
||||
description = "GKE prod"
|
||||
flag = "gke"
|
||||
}
|
||||
networking = {
|
||||
condition = "resource.matchTag('${local.tag_keys.context}', 'networking')"
|
||||
description = "networking"
|
||||
flag = "-"
|
||||
}
|
||||
pf-dev = {
|
||||
condition = "resource.matchTag('${local.tag_keys.environment}', 'development')"
|
||||
description = "project factory dev"
|
||||
flag = "project_factory"
|
||||
}
|
||||
pf-prod = {
|
||||
condition = "resource.matchTag('${local.tag_keys.environment}', 'production')"
|
||||
description = "project factory prod"
|
||||
flag = "project_factory"
|
||||
}
|
||||
sandbox = {
|
||||
condition = "resource.matchTag('${local.tag_keys.context}', 'sandbox')"
|
||||
description = "sandbox"
|
||||
flag = "sandbox"
|
||||
}
|
||||
security = {
|
||||
condition = "resource.matchTag('${local.tag_keys.context}', 'security')"
|
||||
description = "security"
|
||||
flag = "-"
|
||||
}
|
||||
teams = {
|
||||
condition = "resource.matchTag('${local.tag_keys.context}', 'teams')"
|
||||
description = "teams"
|
||||
flag = "teams"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module "automation-tf-resman-sa-stage2-3" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = {
|
||||
for k, v in local.branch_sas :
|
||||
k => v if lookup(local.fast_features, v.flag, true)
|
||||
}
|
||||
project_id = module.automation-project.project_id
|
||||
name = "${each.key}-0"
|
||||
display_name = "Terraform ${each.value.description} service account."
|
||||
prefix = local.prefix
|
||||
iam_billing_roles = !var.billing_account.is_org_level ? {
|
||||
(var.billing_account.id) = [
|
||||
"roles/billing.user", "roles/billing.costsManager"
|
||||
]
|
||||
} : {}
|
||||
iam_organization_roles = var.billing_account.is_org_level ? {
|
||||
(var.organization.id) = [
|
||||
"roles/billing.user", "roles/billing.costsManager"
|
||||
]
|
||||
} : {}
|
||||
}
|
||||
|
||||
# assign org policy admin with a tag-based condition to stage 2 and 3 SAs
|
||||
|
||||
resource "google_organization_iam_member" "org_policy_admin_stage2_3" {
|
||||
for_each = {
|
||||
for k, v in module.automation-tf-resman-sa-stage2-3 : k => v.iam_email
|
||||
}
|
||||
org_id = var.organization.id
|
||||
role = "roles/orgpolicy.policyAdmin"
|
||||
member = each.value
|
||||
condition {
|
||||
title = "org_policy_tag_${var.tenant_config.short_name}_${each.key}_scoped"
|
||||
description = join("", [
|
||||
"Org policy tag scoped grant for tenant ${var.tenant_config.short_name} ",
|
||||
local.branch_sas[each.key].description
|
||||
])
|
||||
expression = join(" && ", [
|
||||
local.iam_tenant_condition, local.branch_sas[each.key].condition
|
||||
])
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,141 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Tenant automation project and resources.
|
||||
|
||||
module "automation-project" {
|
||||
source = "../../../modules/project"
|
||||
billing_account = var.billing_account.id
|
||||
name = "iac-core-0"
|
||||
parent = coalesce(
|
||||
var.project_parent_ids.automation,
|
||||
module.tenant-folder.id
|
||||
)
|
||||
prefix = local.prefix
|
||||
# human (groups) IAM bindings
|
||||
group_iam = {
|
||||
(local.groups.gcp-admins) = [
|
||||
"roles/iam.serviceAccountAdmin",
|
||||
"roles/iam.serviceAccountTokenCreator",
|
||||
]
|
||||
(local.groups.gcp-admins) = [
|
||||
"roles/iam.serviceAccountTokenCreator",
|
||||
"roles/iam.workloadIdentityPoolAdmin"
|
||||
]
|
||||
}
|
||||
# machine (service accounts) IAM bindings
|
||||
iam = {
|
||||
"roles/owner" = [
|
||||
module.automation-tf-resman-sa.iam_email,
|
||||
"serviceAccount:${local.resman_sa}"
|
||||
]
|
||||
"roles/cloudbuild.builds.editor" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
"roles/iam.serviceAccountAdmin" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
"roles/iam.workloadIdentityPoolAdmin" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
"roles/source.admin" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
"roles/storage.admin" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
}
|
||||
services = [
|
||||
"accesscontextmanager.googleapis.com",
|
||||
"bigquery.googleapis.com",
|
||||
"bigqueryreservation.googleapis.com",
|
||||
"bigquerystorage.googleapis.com",
|
||||
"billingbudgets.googleapis.com",
|
||||
"cloudbilling.googleapis.com",
|
||||
"cloudbuild.googleapis.com",
|
||||
"cloudkms.googleapis.com",
|
||||
"cloudresourcemanager.googleapis.com",
|
||||
"container.googleapis.com",
|
||||
"compute.googleapis.com",
|
||||
"container.googleapis.com",
|
||||
"essentialcontacts.googleapis.com",
|
||||
"iam.googleapis.com",
|
||||
"iamcredentials.googleapis.com",
|
||||
"orgpolicy.googleapis.com",
|
||||
"pubsub.googleapis.com",
|
||||
"servicenetworking.googleapis.com",
|
||||
"serviceusage.googleapis.com",
|
||||
"sourcerepo.googleapis.com",
|
||||
"stackdriver.googleapis.com",
|
||||
"storage-component.googleapis.com",
|
||||
"storage.googleapis.com",
|
||||
"sts.googleapis.com"
|
||||
]
|
||||
}
|
||||
|
||||
# output files bucket
|
||||
|
||||
module "automation-tf-output-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
project_id = module.automation-project.project_id
|
||||
name = "iac-core-outputs-0"
|
||||
prefix = local.prefix
|
||||
location = local.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
}
|
||||
|
||||
# resource management stage bucket and service account
|
||||
|
||||
module "automation-tf-resman-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
project_id = module.automation-project.project_id
|
||||
name = "iac-core-resman-0"
|
||||
prefix = local.prefix
|
||||
location = local.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [module.automation-tf-resman-sa.iam_email]
|
||||
}
|
||||
}
|
||||
|
||||
module "automation-tf-resman-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
project_id = module.automation-project.project_id
|
||||
name = "resman-0"
|
||||
display_name = "Terraform stage 1 resman service account."
|
||||
prefix = local.prefix
|
||||
# allow SA used by CI/CD workflow to impersonate this SA
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.automation-tf-cicd-sa-resman["0"].iam_email, null)
|
||||
])
|
||||
}
|
||||
iam_billing_roles = !var.billing_account.is_org_level ? {
|
||||
(var.billing_account.id) = [
|
||||
"roles/billing.admin", "roles/billing.costsManager"
|
||||
]
|
||||
} : {}
|
||||
iam_organization_roles = var.billing_account.is_org_level ? {
|
||||
(var.organization.id) = [
|
||||
"roles/billing.admin", "roles/billing.costsManager"
|
||||
]
|
||||
} : {}
|
||||
iam_storage_roles = {
|
||||
(module.automation-tf-output-gcs.name) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Billing roles for standalone billing accounts.
|
||||
|
||||
# service account billing roles are in the SA module in automation.tf
|
||||
|
||||
resource "google_billing_account_iam_member" "billing_ext_admin" {
|
||||
for_each = toset(var.billing_account.is_org_level ? [] : [
|
||||
"group:${local.groups.gcp-admins}",
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
])
|
||||
billing_account_id = var.billing_account.id
|
||||
role = "roles/billing.admin"
|
||||
member = each.key
|
||||
}
|
||||
|
||||
resource "google_billing_account_iam_member" "billing_ext_cost_manager" {
|
||||
for_each = toset(var.billing_account.is_org_level ? [] : [
|
||||
"group:${local.groups.gcp-admins}",
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
])
|
||||
billing_account_id = var.billing_account.id
|
||||
role = "roles/billing.costsManager"
|
||||
member = each.key
|
||||
}
|
||||
|
|
@ -0,0 +1,223 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Workload Identity Federation configurations for CI/CD.
|
||||
|
||||
locals {
|
||||
_file_prefix = "tenants/${var.tenant_config.short_name}"
|
||||
# derive identity pool names from identity providers for easy reference
|
||||
cicd_identity_pools = {
|
||||
for k, v in local.cicd_identity_providers :
|
||||
k => split("/providers/", v.name)[0]
|
||||
}
|
||||
# merge org-level and tenant-level identity providers
|
||||
cicd_identity_providers = merge(
|
||||
var.automation.federated_identity_providers,
|
||||
{
|
||||
for k, v in google_iam_workload_identity_pool_provider.default :
|
||||
k => {
|
||||
issuer = local.identity_providers[k].issuer
|
||||
issuer_uri = local.identity_providers[k].issuer_uri
|
||||
name = v.name
|
||||
principal_tpl = local.identity_providers[k].principal_tpl
|
||||
principalset_tpl = local.identity_providers[k].principalset_tpl
|
||||
}
|
||||
})
|
||||
# filter CI/CD repositories to only keep valid ones
|
||||
cicd_repositories = {
|
||||
for k, v in coalesce(var.cicd_repositories, {}) : k => v
|
||||
if(
|
||||
v != null
|
||||
&&
|
||||
(
|
||||
try(v.type, null) == "sourcerepo"
|
||||
||
|
||||
contains(
|
||||
keys(local.cicd_identity_providers),
|
||||
coalesce(try(v.identity_provider, null), ":")
|
||||
)
|
||||
)
|
||||
&&
|
||||
fileexists(
|
||||
format("${path.module}/templates/workflow-%s.yaml", try(v.type, ""))
|
||||
)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
# tenant bootstrap runs in the org scope and uses top-level automation project
|
||||
|
||||
module "automation-tf-cicd-repo-bootstrap" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = {
|
||||
for k, v in local.cicd_repositories : 0 => v
|
||||
if k == "bootstrap" && try(v.type, null) == "sourcerepo"
|
||||
}
|
||||
project_id = var.automation.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = [
|
||||
local.resman_sa
|
||||
]
|
||||
"roles/source.reader" = [
|
||||
module.automation-tf-cicd-sa-bootstrap["0"].iam_email
|
||||
]
|
||||
}
|
||||
triggers = {
|
||||
"fast-${var.tenant_config.short_name}-0-bootstrap" = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = ["**/*tf", ".cloudbuild/workflow.yaml"]
|
||||
service_account = module.automation-tf-cicd-sa-bootstrap["0"].id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module "automation-tf-cicd-sa-bootstrap" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = {
|
||||
for k, v in local.cicd_repositories : 0 => v
|
||||
if k == "bootstrap" && try(v.type, null) != null
|
||||
}
|
||||
project_id = var.automation.project_id
|
||||
name = "bootstrap-1"
|
||||
display_name = "Terraform CI/CD ${var.tenant_config.short_name} bootstrap."
|
||||
prefix = local.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(var.automation.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
|
||||
module "automation-tf-org-resman-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = {
|
||||
for k, v in local.cicd_repositories : 0 => v
|
||||
if k == "bootstrap" && try(v.type, null) != null
|
||||
}
|
||||
project_id = var.automation.project_id
|
||||
name = local.resman_sa
|
||||
service_account_create = false
|
||||
iam_additive = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.automation-tf-cicd-sa-bootstrap["0"].iam_email, null)
|
||||
])
|
||||
}
|
||||
}
|
||||
|
||||
# tenant resman runs in the tenant scope and uses its own automation project
|
||||
|
||||
module "automation-tf-cicd-repo-resman" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = {
|
||||
for k, v in local.cicd_repositories : 0 => v
|
||||
if k == "resman" && try(v.type, null) == "sourcerepo"
|
||||
}
|
||||
project_id = module.automation-project.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
"roles/source.reader" = [
|
||||
module.automation-tf-cicd-sa-resman["0"].iam_email
|
||||
]
|
||||
}
|
||||
triggers = {
|
||||
fast-1-resman = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = ["**/*tf", ".cloudbuild/workflow.yaml"]
|
||||
service_account = module.automation-tf-cicd-sa-resman["0"].id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module "automation-tf-cicd-sa-resman" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = {
|
||||
for k, v in local.cicd_repositories : 0 => v
|
||||
if k == "resman" && try(v.type, null) != null
|
||||
}
|
||||
project_id = module.automation-project.project_id
|
||||
name = "resman-1"
|
||||
display_name = "Terraform CI/CD resman."
|
||||
prefix = local.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(module.automation-project.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(module.automation-tf-output-gcs.name) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 208 KiB |
|
|
@ -0,0 +1,96 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Workload Identity Federation provider definitions.
|
||||
|
||||
locals {
|
||||
identity_providers = {
|
||||
for k, v in var.federated_identity_providers : k => merge(
|
||||
v,
|
||||
lookup(local.identity_providers_defs, v.issuer, {})
|
||||
)
|
||||
}
|
||||
identity_providers_defs = {
|
||||
# https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect
|
||||
github = {
|
||||
attribute_mapping = {
|
||||
"google.subject" = "assertion.sub"
|
||||
"attribute.sub" = "assertion.sub"
|
||||
"attribute.actor" = "assertion.actor"
|
||||
"attribute.repository" = "assertion.repository"
|
||||
"attribute.repository_owner" = "assertion.repository_owner"
|
||||
"attribute.ref" = "assertion.ref"
|
||||
}
|
||||
issuer_uri = "https://token.actions.githubusercontent.com"
|
||||
principal_tpl = "principal://iam.googleapis.com/%s/subject/repo:%s:ref:refs/heads/%s"
|
||||
principalset_tpl = "principalSet://iam.googleapis.com/%s/attribute.repository/%s"
|
||||
}
|
||||
# https://docs.gitlab.com/ee/ci/cloud_services/index.html#how-it-works
|
||||
gitlab = {
|
||||
attribute_mapping = {
|
||||
"google.subject" = "assertion.sub"
|
||||
"attribute.sub" = "assertion.sub"
|
||||
"attribute.environment" = "assertion.environment"
|
||||
"attribute.environment_protected" = "assertion.environment_protected"
|
||||
"attribute.namespace_id" = "assertion.namespace_id"
|
||||
"attribute.namespace_path" = "assertion.namespace_path"
|
||||
"attribute.pipeline_id" = "assertion.pipeline_id"
|
||||
"attribute.pipeline_source" = "assertion.pipeline_source"
|
||||
"attribute.project_id" = "assertion.project_id"
|
||||
"attribute.project_path" = "assertion.project_path"
|
||||
"attribute.repository" = "assertion.project_path"
|
||||
"attribute.ref" = "assertion.ref"
|
||||
"attribute.ref_protected" = "assertion.ref_protected"
|
||||
"attribute.ref_type" = "assertion.ref_type"
|
||||
}
|
||||
allowed_audiences = ["https://gitlab.com"]
|
||||
issuer_uri = "https://gitlab.com"
|
||||
principal_tpl = "principalSet://iam.googleapis.com/%s/attribute.sub/project_path:%s:ref_type:branch:ref:%s"
|
||||
principalset_tpl = "principalSet://iam.googleapis.com/%s/attribute.repository/%s"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_iam_workload_identity_pool" "default" {
|
||||
provider = google-beta
|
||||
count = length(local.identity_providers) > 0 ? 1 : 0
|
||||
project = module.automation-project.project_id
|
||||
workload_identity_pool_id = "${var.prefix}-bootstrap"
|
||||
}
|
||||
|
||||
resource "google_iam_workload_identity_pool_provider" "default" {
|
||||
provider = google-beta
|
||||
for_each = local.identity_providers
|
||||
project = module.automation-project.project_id
|
||||
workload_identity_pool_id = (
|
||||
google_iam_workload_identity_pool.default.0.workload_identity_pool_id
|
||||
)
|
||||
workload_identity_pool_provider_id = "${var.prefix}-bootstrap-${each.key}"
|
||||
attribute_condition = each.value.attribute_condition
|
||||
attribute_mapping = each.value.attribute_mapping
|
||||
oidc {
|
||||
allowed_audiences = (
|
||||
try(each.value.custom_settings.allowed_audiences, null) != null
|
||||
? each.value.custom_settings.allowed_audiences
|
||||
: try(each.value.allowed_audiences, null)
|
||||
)
|
||||
issuer_uri = (
|
||||
try(each.value.custom_settings.issuer_uri, null) != null
|
||||
? each.value.custom_settings.issuer_uri
|
||||
: try(each.value.issuer_uri, null)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,94 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Audit log project and sink.
|
||||
|
||||
locals {
|
||||
log_sink_destinations = merge(
|
||||
# use the same dataset for all sinks with `bigquery` as destination
|
||||
{ for k, v in var.log_sinks : k => module.log-export-dataset.0 if v.type == "bigquery" },
|
||||
# use the same gcs bucket for all sinks with `storage` as destination
|
||||
{ for k, v in var.log_sinks : k => module.log-export-gcs.0 if v.type == "storage" },
|
||||
# use separate pubsub topics and logging buckets for sinks with
|
||||
# destination `pubsub` and `logging`
|
||||
module.log-export-pubsub,
|
||||
module.log-export-logbucket
|
||||
)
|
||||
log_types = toset([for k, v in var.log_sinks : v.type])
|
||||
}
|
||||
|
||||
module "log-export-project" {
|
||||
source = "../../../modules/project"
|
||||
billing_account = var.billing_account.id
|
||||
name = "audit-logs-0"
|
||||
parent = coalesce(
|
||||
var.project_parent_ids.logging,
|
||||
module.tenant-folder.id
|
||||
)
|
||||
prefix = local.prefix
|
||||
iam = {
|
||||
"roles/owner" = [
|
||||
module.automation-tf-resman-sa.iam_email,
|
||||
"serviceAccount:${local.resman_sa}"
|
||||
]
|
||||
}
|
||||
services = [
|
||||
# "cloudresourcemanager.googleapis.com",
|
||||
# "iam.googleapis.com",
|
||||
# "serviceusage.googleapis.com",
|
||||
"bigquery.googleapis.com",
|
||||
"storage.googleapis.com",
|
||||
"stackdriver.googleapis.com"
|
||||
]
|
||||
}
|
||||
|
||||
# one log export per type, with conditionals to skip those not needed
|
||||
|
||||
module "log-export-dataset" {
|
||||
source = "../../../modules/bigquery-dataset"
|
||||
count = contains(local.log_types, "bigquery") ? 1 : 0
|
||||
project_id = module.log-export-project.project_id
|
||||
id = "audit_export"
|
||||
friendly_name = "Audit logs export."
|
||||
location = var.locations.bq
|
||||
}
|
||||
|
||||
module "log-export-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
count = contains(local.log_types, "storage") ? 1 : 0
|
||||
project_id = module.log-export-project.project_id
|
||||
name = "audit-logs-0"
|
||||
prefix = local.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
}
|
||||
|
||||
module "log-export-logbucket" {
|
||||
source = "../../../modules/logging-bucket"
|
||||
for_each = toset([for k, v in var.log_sinks : k if v.type == "logging"])
|
||||
parent_type = "project"
|
||||
parent = module.log-export-project.project_id
|
||||
id = "audit-logs-${each.key}"
|
||||
location = var.locations.logging
|
||||
}
|
||||
|
||||
module "log-export-pubsub" {
|
||||
source = "../../../modules/pubsub"
|
||||
for_each = toset([for k, v in var.log_sinks : k if v.type == "pubsub"])
|
||||
project_id = module.log-export-project.project_id
|
||||
name = "audit-logs-${each.key}"
|
||||
regions = var.locations.pubsub
|
||||
}
|
||||
|
|
@ -0,0 +1,100 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
locals {
|
||||
gcs_storage_class = (
|
||||
length(split("-", local.locations.gcs)) < 2
|
||||
? "MULTI_REGIONAL"
|
||||
: "REGIONAL"
|
||||
)
|
||||
groups = {
|
||||
for k, v in var.tenant_config.groups :
|
||||
k => v == null ? null : "${v}@${var.organization.domain}"
|
||||
}
|
||||
fast_features = {
|
||||
for k, v in var.tenant_config.fast_features :
|
||||
k => v == null ? var.fast_features[k] : v
|
||||
}
|
||||
locations = {
|
||||
for k, v in var.tenant_config.locations :
|
||||
k => v == null || v == [] ? var.locations[k] : v
|
||||
}
|
||||
prefix = join("-", compact([var.prefix, var.tenant_config.short_name]))
|
||||
resman_sa = (
|
||||
var.test_principal == null
|
||||
? data.google_client_openid_userinfo.resman-sa.0.email
|
||||
: var.test_principal
|
||||
)
|
||||
}
|
||||
|
||||
data "google_client_openid_userinfo" "resman-sa" {
|
||||
count = var.test_principal == null ? 1 : 0
|
||||
}
|
||||
|
||||
module "tenant-folder" {
|
||||
source = "../../../modules/folder"
|
||||
parent = "organizations/${var.organization.id}"
|
||||
name = var.tenant_config.descriptive_name
|
||||
logging_sinks = {
|
||||
for name, attrs in var.log_sinks : name => {
|
||||
bq_partitioned_table = attrs.type == "bigquery"
|
||||
destination = local.log_sink_destinations[name].id
|
||||
filter = attrs.filter
|
||||
type = attrs.type
|
||||
}
|
||||
}
|
||||
tag_bindings = {
|
||||
tenant = try(
|
||||
module.organization.tag_values["${var.tag_names.tenant}/${var.tenant_config.short_name}"].id,
|
||||
null
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
module "tenant-folder-iam" {
|
||||
source = "../../../modules/folder"
|
||||
id = module.tenant-folder.id
|
||||
folder_create = false
|
||||
group_iam = merge(var.group_iam, {
|
||||
(local.groups.gcp-admins) = [
|
||||
"roles/logging.admin",
|
||||
"roles/owner",
|
||||
"roles/resourcemanager.folderAdmin",
|
||||
"roles/resourcemanager.projectCreator",
|
||||
"roles/compute.xpnAdmin"
|
||||
]
|
||||
})
|
||||
iam = merge(var.iam, {
|
||||
"roles/compute.xpnAdmin" = [
|
||||
module.automation-tf-resman-sa.iam_email,
|
||||
module.automation-tf-resman-sa-stage2-3["networking"].iam_email
|
||||
]
|
||||
"roles/logging.admin" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
"roles/resourcemanager.folderAdmin" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
"roles/resourcemanager.projectCreator" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
"roles/owner" = [
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
})
|
||||
iam_additive = var.iam_additive
|
||||
depends_on = [module.automation-project]
|
||||
}
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Organization tag and conditional IAM grant.
|
||||
|
||||
locals {
|
||||
iam_tenant_condition = "resource.matchTag('${local.tag_keys.tenant}', '${var.tenant_config.short_name}')"
|
||||
tag_keys = {
|
||||
for k, v in var.tag_names : k => "${var.organization.id}/${v}"
|
||||
}
|
||||
}
|
||||
|
||||
module "organization" {
|
||||
source = "../../../modules/organization"
|
||||
organization_id = "organizations/${var.organization.id}"
|
||||
iam_additive = merge(
|
||||
{
|
||||
"roles/resourcemanager.organizationViewer" = [
|
||||
"group:${local.groups.gcp-admins}"
|
||||
]
|
||||
},
|
||||
var.billing_account.is_org_level ? {
|
||||
"roles/billing.admin" = [
|
||||
"group:${local.groups.gcp-admins}",
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
"roles/billing.costsManager" = ["group:${local.groups.gcp-admins}"]
|
||||
} : {}
|
||||
)
|
||||
tags = {
|
||||
tenant = {
|
||||
id = var.tag_keys.tenant
|
||||
values = {
|
||||
(var.tenant_config.short_name) = {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_tags_tag_value_iam_member" "resman_tag_user" {
|
||||
for_each = var.tag_values
|
||||
tag_value = each.value
|
||||
role = "roles/resourcemanager.tagUser"
|
||||
member = module.automation-tf-resman-sa.iam_email
|
||||
}
|
||||
|
||||
resource "google_tags_tag_value_iam_member" "admins_tag_viewer" {
|
||||
for_each = var.tag_values
|
||||
tag_value = each.value
|
||||
role = "roles/resourcemanager.tagViewer"
|
||||
member = "group:${local.groups.gcp-admins}"
|
||||
}
|
||||
|
||||
# assign org policy admin with a tag-based condition to admin group and stage 1 SA
|
||||
|
||||
resource "google_organization_iam_member" "org_policy_admin_stage0" {
|
||||
for_each = toset([
|
||||
"group:${local.groups.gcp-admins}",
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
])
|
||||
org_id = var.organization.id
|
||||
role = "roles/orgpolicy.policyAdmin"
|
||||
member = each.key
|
||||
condition {
|
||||
title = "org_policy_tag_${var.tenant_config.short_name}_scoped"
|
||||
description = "Org policy tag scoped grant for tenant ${var.tenant_config.short_name}."
|
||||
expression = local.iam_tenant_condition
|
||||
}
|
||||
}
|
||||
|
||||
# tag-based condition for service accounts is in the automation-sa file
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Output files persistence to local filesystem.
|
||||
|
||||
locals {
|
||||
outputs_root = join("/", [
|
||||
try(pathexpand(var.outputs_location), ""),
|
||||
"tenants",
|
||||
var.tenant_config.short_name
|
||||
])
|
||||
}
|
||||
|
||||
resource "local_file" "providers" {
|
||||
count = var.outputs_location == null ? 0 : 1
|
||||
file_permission = "0644"
|
||||
filename = "${local.outputs_root}/providers/1-resman-tenant-providers.tf"
|
||||
content = try(local.provider, null)
|
||||
}
|
||||
|
||||
resource "local_file" "tfvars" {
|
||||
count = var.outputs_location == null ? 0 : 1
|
||||
file_permission = "0644"
|
||||
filename = "${local.outputs_root}/tfvars/0-bootstrap-tenant.auto.tfvars.json"
|
||||
content = jsonencode(local.tfvars)
|
||||
}
|
||||
|
||||
resource "local_file" "workflows" {
|
||||
for_each = var.outputs_location == null ? {} : local.cicd_workflows
|
||||
file_permission = "0644"
|
||||
filename = "${local.outputs_root}/workflows/${each.key}-${local.cicd_repositories[each.key].type}.yaml"
|
||||
content = each.value
|
||||
}
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Output files persistence to automation GCS bucket.
|
||||
|
||||
resource "google_storage_bucket_object" "providers" {
|
||||
bucket = module.automation-tf-output-gcs.name
|
||||
# provider suffix allows excluding via .gitignore when linked from stages
|
||||
name = "tenants/${var.tenant_config.short_name}/providers/1-resman-tenant-providers.tf"
|
||||
content = local.provider
|
||||
}
|
||||
|
||||
resource "google_storage_bucket_object" "tfvars" {
|
||||
bucket = module.automation-tf-output-gcs.name
|
||||
name = "tenants/${var.tenant_config.short_name}/tfvars/0-bootstrap-tenant.auto.tfvars.json"
|
||||
content = jsonencode(local.tfvars)
|
||||
}
|
||||
|
||||
resource "google_storage_bucket_object" "workflows" {
|
||||
for_each = local.cicd_workflows
|
||||
bucket = (
|
||||
each.key == "bootstrap"
|
||||
? var.automation.outputs_bucket
|
||||
: module.automation-tf-output-gcs.name
|
||||
)
|
||||
name = "tenants/${var.tenant_config.short_name}/workflows/${each.key}-${local.cicd_repositories[each.key].type}.yaml"
|
||||
content = each.value
|
||||
}
|
||||
|
|
@ -0,0 +1,140 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
locals {
|
||||
cicd_workflows = {
|
||||
for k, v in local.cicd_repositories : k => templatefile(
|
||||
"${path.module}/templates/workflow-${v.type}.yaml", (
|
||||
k == "bootstrap"
|
||||
? {
|
||||
identity_provider = try(
|
||||
local.cicd_identity_providers[v["identity_provider"]].name, ""
|
||||
)
|
||||
outputs_bucket = var.automation.outputs_bucket
|
||||
service_account = try(
|
||||
module.automation-tf-cicd-sa-bootstrap["0"].email, ""
|
||||
)
|
||||
stage_name = k
|
||||
tf_providers_file = ""
|
||||
tf_var_files = [
|
||||
"0-bootstrap.auto.tfvars.json",
|
||||
"1-resman.auto.tfvars.json",
|
||||
"globals.auto.tfvars.json"
|
||||
]
|
||||
}
|
||||
: {
|
||||
identity_provider = try(
|
||||
local.cicd_identity_providers[v["identity_provider"]].name, ""
|
||||
)
|
||||
outputs_bucket = module.automation-tf-output-gcs.name
|
||||
service_account = try(
|
||||
module.automation-tf-cicd-sa-resman["0"].email, ""
|
||||
)
|
||||
stage_name = k
|
||||
tf_providers_file = (
|
||||
"${local._file_prefix}/providers/1-resman-tenant-providers.tf"
|
||||
)
|
||||
tf_var_files = [
|
||||
"${local._file_prefix}/tfvars/0-bootstrap-tenant.auto.tfvars.json"
|
||||
]
|
||||
}
|
||||
)
|
||||
)
|
||||
}
|
||||
provider = templatefile(
|
||||
"${path.module}/templates/providers.tf.tpl", {
|
||||
bucket = module.automation-tf-resman-gcs.name
|
||||
name = "resman"
|
||||
sa = module.automation-tf-resman-sa.email
|
||||
}
|
||||
)
|
||||
tfvars = {
|
||||
automation = {
|
||||
outputs_bucket = module.automation-tf-output-gcs.name
|
||||
project_id = module.automation-project.project_id
|
||||
project_number = module.automation-project.number
|
||||
federated_identity_pools = compact([
|
||||
try(google_iam_workload_identity_pool.default.0.name, null),
|
||||
var.automation.federated_identity_pool,
|
||||
])
|
||||
federated_identity_providers = local.cicd_identity_providers
|
||||
service_accounts = merge(
|
||||
{ resman = module.automation-tf-resman-sa.email },
|
||||
{
|
||||
for k, v in local.branch_sas : k => try(
|
||||
module.automation-tf-resman-sa-stage2-3[k].email, null
|
||||
)
|
||||
}
|
||||
)
|
||||
}
|
||||
billing_account = var.billing_account
|
||||
custom_roles = var.custom_roles
|
||||
fast_features = local.fast_features
|
||||
groups = var.tenant_config.groups
|
||||
locations = local.locations
|
||||
organization = var.organization
|
||||
prefix = local.prefix
|
||||
root_node = module.tenant-folder.id
|
||||
short_name = var.tenant_config.short_name
|
||||
tags = {
|
||||
keys = var.tag_keys
|
||||
names = var.tag_names
|
||||
values = merge(var.tag_values, {
|
||||
for k, v in module.organization.tag_values : k => v.id
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output "cicd_workflows" {
|
||||
description = "CI/CD workflows for tenant bootstrap and resource management stages."
|
||||
sensitive = true
|
||||
value = local.cicd_workflows
|
||||
}
|
||||
|
||||
output "federated_identity" {
|
||||
description = "Workload Identity Federation pool and providers."
|
||||
value = {
|
||||
pool = try(
|
||||
google_iam_workload_identity_pool.default.0.name, null
|
||||
)
|
||||
providers = local.cicd_identity_providers
|
||||
}
|
||||
}
|
||||
|
||||
output "provider" {
|
||||
# tfdoc:output:consumers stage-01
|
||||
description = "Terraform provider file for tenant resource management stage."
|
||||
sensitive = true
|
||||
value = local.provider
|
||||
}
|
||||
|
||||
output "tenant_resources" {
|
||||
description = "Tenant-level resources."
|
||||
value = {
|
||||
bucket = module.automation-tf-resman-gcs.name
|
||||
folder = module.tenant-folder.id
|
||||
project_id = module.automation-project.project_id
|
||||
project_number = module.automation-project.number
|
||||
service_account = module.automation-tf-resman-sa.email
|
||||
}
|
||||
}
|
||||
|
||||
output "tfvars" {
|
||||
description = "Terraform variable files for the following tenant stages."
|
||||
sensitive = true
|
||||
value = local.tfvars
|
||||
}
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
terraform {
|
||||
backend "gcs" {
|
||||
bucket = "${bucket}"
|
||||
impersonate_service_account = "${sa}"
|
||||
}
|
||||
}
|
||||
provider "google" {
|
||||
impersonate_service_account = "${sa}"
|
||||
}
|
||||
provider "google-beta" {
|
||||
impersonate_service_account = "${sa}"
|
||||
}
|
||||
|
||||
# end provider.tf for ${name}
|
||||
|
|
@ -0,0 +1,190 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
name: "FAST ${stage_name} stage"
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
types:
|
||||
- closed
|
||||
- opened
|
||||
- synchronize
|
||||
|
||||
env:
|
||||
FAST_OUTPUTS_BUCKET: ${outputs_bucket}
|
||||
FAST_SERVICE_ACCOUNT: ${service_account}
|
||||
FAST_WIF_PROVIDER: ${identity_provider}
|
||||
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
|
||||
%{~ if tf_providers_file != "" ~}
|
||||
TF_PROVIDERS_FILE: ${tf_providers_file}
|
||||
%{~ endif ~}
|
||||
TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
|
||||
TF_VERSION: 1.3.2
|
||||
|
||||
jobs:
|
||||
fast-pr:
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
issues: write
|
||||
pull-requests: write
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- id: checkout
|
||||
name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
|
||||
# set up SSH key authentication to the modules repository
|
||||
- id: ssh-config
|
||||
name: Configure SSH authentication
|
||||
run: |
|
||||
ssh-agent -a "$SSH_AUTH_SOCK" > /dev/null
|
||||
ssh-add - <<< "$${{ secrets.CICD_MODULES_KEY }}"
|
||||
|
||||
# set up authentication via Workload identity Federation
|
||||
- id: gcp-auth
|
||||
name: Authenticate to Google Cloud
|
||||
uses: google-github-actions/auth@v0
|
||||
with:
|
||||
workload_identity_provider: $${{ env.FAST_WIF_PROVIDER }}
|
||||
service_account: $${{ env.FAST_SERVICE_ACCOUNT }}
|
||||
access_token_lifetime: 3600s
|
||||
|
||||
- id: gcp-sdk
|
||||
name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v0
|
||||
with:
|
||||
install_components: alpha
|
||||
|
||||
# copy provider and tfvars files
|
||||
- id: tf-config
|
||||
name: Copy Terraform output files
|
||||
run: |
|
||||
%{~ if tf_providers_file != "" ~}
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${{env.FAST_OUTPUTS_BUCKET}}/providers/$${{env.TF_PROVIDERS_FILE}}" ./
|
||||
%{~ endif ~}
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${{env.FAST_OUTPUTS_BUCKET}}/tfvars" ./
|
||||
for f in $${{env.TF_VAR_FILES}}; do
|
||||
ln -s "tfvars/$f" ./
|
||||
done
|
||||
|
||||
- id: tf-setup
|
||||
name: Set up Terraform
|
||||
uses: hashicorp/setup-terraform@v2.0.3
|
||||
with:
|
||||
terraform_version: $${{ env.TF_VERSION }}
|
||||
|
||||
# run Terraform init/validate/plan
|
||||
- id: tf-init
|
||||
name: Terraform init
|
||||
run: |
|
||||
terraform init -no-color
|
||||
|
||||
- id: tf-validate
|
||||
name: Terraform validate
|
||||
run: terraform validate -no-color
|
||||
|
||||
- id: tf-plan
|
||||
name: Terraform plan
|
||||
continue-on-error: true
|
||||
run: |
|
||||
terraform plan -input=false -out ../plan.out -no-color
|
||||
|
||||
- id: tf-apply
|
||||
if: github.event.pull_request.merged == true && success()
|
||||
name: Terraform apply
|
||||
continue-on-error: true
|
||||
run: |
|
||||
terraform apply -input=false -auto-approve -no-color ../plan.out
|
||||
|
||||
- id: pr-comment
|
||||
name: Post comment to Pull Request
|
||||
continue-on-error: true
|
||||
uses: actions/github-script@v6
|
||||
if: github.event_name == 'pull_request'
|
||||
env:
|
||||
PLAN: $${{ steps.tf-plan.outputs.stdout }}\n$${{ steps.tf-plan.outputs.stderr }}
|
||||
with:
|
||||
script: |
|
||||
const output = `### Terraform Initialization \`$${{ steps.tf-init.outcome }}\`
|
||||
|
||||
### Terraform Validation \`$${{ steps.tf-validate.outcome }}\`
|
||||
|
||||
<details><summary>Validation Output</summary>
|
||||
|
||||
\`\`\`\n
|
||||
$${{ steps.tf-validate.outputs.stdout }}
|
||||
\`\`\`
|
||||
|
||||
</details>
|
||||
|
||||
### Terraform Plan \`$${{ steps.tf-plan.outcome }}\`
|
||||
|
||||
<details><summary>Show Plan</summary>
|
||||
|
||||
\`\`\`\n
|
||||
$${process.env.PLAN.split('\n').filter(l => l.match(/^([A-Z\s].*|)$$/)).join('\n')}
|
||||
\`\`\`
|
||||
|
||||
</details>
|
||||
|
||||
### Terraform Apply \`$${{ steps.tf-apply.outcome }}\`
|
||||
|
||||
*Pusher: @$${{ github.actor }}, Action: \`$${{ github.event_name }}\`, Working Directory: \`$${{ env.tf_actions_working_dir }}\`, Workflow: \`$${{ github.workflow }}\`*`;
|
||||
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: output
|
||||
})
|
||||
|
||||
- id: pr-short-comment
|
||||
name: Post comment to Pull Request
|
||||
uses: actions/github-script@v6
|
||||
if: github.event_name == 'pull_request' && steps.pr-comment.outcome != 'success'
|
||||
with:
|
||||
script: |
|
||||
const output = `### Terraform Initialization \`$${{ steps.tf-init.outcome }}\`
|
||||
|
||||
### Terraform Validation \`$${{ steps.tf-validate.outcome }}\`
|
||||
|
||||
### Terraform Plan \`$${{ steps.tf-plan.outcome }}\`
|
||||
|
||||
Plan output is in the action log.
|
||||
|
||||
### Terraform Apply \`$${{ steps.tf-apply.outcome }}\`
|
||||
|
||||
*Pusher: @$${{ github.actor }}, Action: \`$${{ github.event_name }}\`, Working Directory: \`$${{ env.tf_actions_working_dir }}\`, Workflow: \`$${{ github.workflow }}\`*`;
|
||||
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: output
|
||||
})
|
||||
|
||||
- id: check-plan
|
||||
name: Check plan failure
|
||||
if: steps.tf-plan.outcome != 'success'
|
||||
run: exit 1
|
||||
|
||||
- id: check-apply
|
||||
name: Check apply failure
|
||||
if: github.event.pull_request.merged == true && steps.tf-apply.outcome != 'success'
|
||||
run: exit 1
|
||||
|
|
@ -0,0 +1,124 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
default:
|
||||
before_script:
|
||||
- echo "$${CI_JOB_JWT_V2}" > token.txt
|
||||
image:
|
||||
name: hashicorp/terraform
|
||||
entrypoint:
|
||||
- "/usr/bin/env"
|
||||
- "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
|
||||
variables:
|
||||
GOOGLE_CREDENTIALS: cicd-sa-credentials.json
|
||||
FAST_OUTPUTS_BUCKET: ${outputs_bucket}
|
||||
FAST_SERVICE_ACCOUNT: ${service_account}
|
||||
FAST_WIF_PROVIDER: ${identity_provider}
|
||||
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
|
||||
%{~ if tf_providers_file != "" ~}
|
||||
TF_PROVIDERS_FILE: ${tf_providers_file}
|
||||
%{~ endif ~}
|
||||
TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
|
||||
|
||||
stages:
|
||||
- gcp-auth
|
||||
- tf-files
|
||||
- tf-plan
|
||||
- tf-apply
|
||||
|
||||
cache:
|
||||
key: gcp-auth
|
||||
paths:
|
||||
- cicd-sa-credentials.json
|
||||
- .tf-setup
|
||||
|
||||
gcp-auth:
|
||||
image:
|
||||
name: google/cloud-sdk:slim
|
||||
stage: gcp-auth
|
||||
script:
|
||||
- |
|
||||
gcloud iam workload-identity-pools create-cred-config \
|
||||
$${FAST_WIF_PROVIDER} \
|
||||
--service-account=$${FAST_SERVICE_ACCOUNT} \
|
||||
--service-account-token-lifetime-seconds=3600 \
|
||||
--output-file=$${GOOGLE_CREDENTIALS} \
|
||||
--credential-source-file=token.txt
|
||||
tf-files:
|
||||
dependencies:
|
||||
- gcp-auth
|
||||
image:
|
||||
name: google/cloud-sdk:slim
|
||||
stage: tf-files
|
||||
script:
|
||||
# - gcloud components install -q alpha
|
||||
- gcloud config set auth/credential_file_override $${GOOGLE_CREDENTIALS}
|
||||
- mkdir -p .tf-setup
|
||||
%{~ if tf_providers_file != "" ~}
|
||||
- |
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${FAST_OUTPUTS_BUCKET}/providers/$${TF_PROVIDERS_FILE}" .tf-setup/
|
||||
%{~ endif ~}
|
||||
- |
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${FAST_OUTPUTS_BUCKET}/tfvars" .tf-setup/
|
||||
|
||||
tf-plan:
|
||||
# uncomment the following lines and set the SSH key secret for private modules repo
|
||||
# before_script:
|
||||
# - |
|
||||
# ssh-agent -a $SSH_AUTH_SOCK > /dev/null
|
||||
# echo "$CICD_MODULES_KEY" | base64 -d | tr -d '\r' | ssh-add - > /dev/null
|
||||
# mkdir -p ~/.ssh
|
||||
# ssh-keyscan -H 'gitlab.com' >> ~/.ssh/known_hosts
|
||||
# ssh-keyscan gitlab.com | sort -u - ~/.ssh/known_hosts -o ~/.ssh/known_hosts
|
||||
stage: tf-plan
|
||||
script:
|
||||
- cp .tf-setup/$${TF_PROVIDERS_FILE} ./
|
||||
- |
|
||||
for f in $${TF_VAR_FILES}; do
|
||||
ln -s ".tf-setup/tfvars/$f" ./
|
||||
done
|
||||
- terraform init
|
||||
- terraform validate
|
||||
- terraform plan
|
||||
dependencies:
|
||||
- tf-files
|
||||
|
||||
tf-apply:
|
||||
# uncomment the following lines and set the SSH key secret for private modules repo
|
||||
# before_script:
|
||||
# - |
|
||||
# ssh-agent -a $SSH_AUTH_SOCK > /dev/null
|
||||
# echo "$CICD_MODULES_KEY" | base64 -d | tr -d '\r' | ssh-add - > /dev/null
|
||||
# mkdir -p ~/.ssh
|
||||
# ssh-keyscan -H 'gitlab.com' >> ~/.ssh/known_hosts
|
||||
# ssh-keyscan gitlab.com | sort -u - ~/.ssh/known_hosts -o ~/.ssh/known_hosts
|
||||
stage: tf-apply
|
||||
script:
|
||||
- cp .tf-setup/$${TF_PROVIDERS_FILE} ./
|
||||
- |
|
||||
for f in $${TF_VAR_FILES}; do
|
||||
ln -s ".tf-setup/tfvars/$f" ./
|
||||
done
|
||||
- terraform init
|
||||
- terraform validate
|
||||
- terraform apply -input=false -auto-approve
|
||||
dependencies:
|
||||
- tf-files
|
||||
when: manual
|
||||
only:
|
||||
variables:
|
||||
- $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
|
|
@ -0,0 +1,100 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
steps:
|
||||
- name: alpine:3
|
||||
id: tf-download
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
mkdir -p /builder/home/.local/bin
|
||||
wget https://releases.hashicorp.com/terraform/$${_TF_VERSION}/terraform_$${_TF_VERSION}_linux_amd64.zip
|
||||
unzip terraform_$${_TF_VERSION}_linux_amd64.zip -d /builder/home/.local/bin
|
||||
rm terraform_$${_TF_VERSION}_linux_amd64.zip
|
||||
chmod 755 /builder/home/.local/bin/terraform
|
||||
- name: alpine:3
|
||||
id: tf-check-format
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform fmt -recursive -check /workspace/
|
||||
- name: gcr.io/google.com/cloudsdktool/cloud-sdk:alpine
|
||||
id: tf-files
|
||||
entrypoint: bash
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
%{~ if tf_providers_file != "" ~}
|
||||
/google-cloud-sdk/bin/gsutil cp \
|
||||
gs://$${_FAST_OUTPUTS_BUCKET}/providers/$${_TF_PROVIDERS_FILE} ./
|
||||
%{~ endif ~}
|
||||
/google-cloud-sdk/bin/gsutil cp -r \
|
||||
gs://$${_FAST_OUTPUTS_BUCKET}/tfvars ./
|
||||
for f in $${_TF_VAR_FILES}; do
|
||||
ln -s tfvars/$f ./
|
||||
done
|
||||
- name: alpine:3
|
||||
id: tf-init
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform init -no-color
|
||||
- name: alpine:3
|
||||
id: tf-check-validate
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform validate -no-color
|
||||
- name: alpine:3
|
||||
id: tf-plan
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform plan -no-color -input=false -out plan.out
|
||||
# store artifact and ask for approval here if needed
|
||||
- name: alpine:3
|
||||
id: tf-apply
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform apply -no-color -input=false -auto-approve plan.out
|
||||
options:
|
||||
env:
|
||||
- PATH=/usr/local/bin:/usr/bin:/bin:/builder/home/.local/bin
|
||||
logging: CLOUD_LOGGING_ONLY
|
||||
substitutions:
|
||||
_FAST_OUTPUTS_BUCKET: ${outputs_bucket}
|
||||
_TF_PROVIDERS_FILE: ${tf_providers_file}
|
||||
_TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
|
||||
_TF_VERSION: 1.3.2
|
||||
|
|
@ -0,0 +1,305 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# defaults for variables marked with global tfdoc annotations, can be set via
|
||||
# the tfvars file generated in stage 00 and stored in its outputs
|
||||
|
||||
variable "automation" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Automation resources created by the organization-level bootstrap stage."
|
||||
type = object({
|
||||
outputs_bucket = string
|
||||
project_id = string
|
||||
project_number = string
|
||||
federated_identity_pool = string
|
||||
federated_identity_providers = map(object({
|
||||
issuer = string
|
||||
issuer_uri = string
|
||||
name = string
|
||||
principal_tpl = string
|
||||
principalset_tpl = string
|
||||
}))
|
||||
})
|
||||
}
|
||||
|
||||
variable "billing_account" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Billing account id. If billing account is not part of the same org set `is_org_level` to false."
|
||||
type = object({
|
||||
id = string
|
||||
is_org_level = optional(bool, true)
|
||||
})
|
||||
validation {
|
||||
condition = var.billing_account.is_org_level != null
|
||||
error_message = "Invalid `null` value for `billing_account.is_org_level`."
|
||||
}
|
||||
}
|
||||
|
||||
variable "cicd_repositories" {
|
||||
description = "CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed."
|
||||
type = object({
|
||||
bootstrap = optional(object({
|
||||
branch = optional(string)
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
}))
|
||||
resman = optional(object({
|
||||
branch = optional(string)
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
}))
|
||||
})
|
||||
default = null
|
||||
validation {
|
||||
condition = alltrue([
|
||||
for k, v in coalesce(var.cicd_repositories, {}) :
|
||||
v == null || try(v.name, null) != null
|
||||
])
|
||||
error_message = "Non-null repositories need a non-null name."
|
||||
}
|
||||
validation {
|
||||
condition = alltrue([
|
||||
for k, v in coalesce(var.cicd_repositories, {}) :
|
||||
v == null || (
|
||||
try(v.identity_provider, null) != null
|
||||
||
|
||||
try(v.type, null) == "sourcerepo"
|
||||
)
|
||||
])
|
||||
error_message = "Non-null repositories need a non-null provider unless type is 'sourcerepo'."
|
||||
}
|
||||
validation {
|
||||
condition = alltrue([
|
||||
for k, v in coalesce(var.cicd_repositories, {}) :
|
||||
v == null || (
|
||||
contains(["github", "gitlab", "sourcerepo"], coalesce(try(v.type, null), "null"))
|
||||
)
|
||||
])
|
||||
error_message = "Invalid repository type, supported types: 'github' 'gitlab' or 'sourcerepo'."
|
||||
}
|
||||
}
|
||||
|
||||
variable "custom_roles" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Custom roles defined at the organization level, in key => id format."
|
||||
type = object({
|
||||
service_project_network_admin = string
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "fast_features" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Selective control for top-level FAST features."
|
||||
type = object({
|
||||
data_platform = optional(bool, true)
|
||||
gke = optional(bool, true)
|
||||
project_factory = optional(bool, true)
|
||||
sandbox = optional(bool, true)
|
||||
teams = optional(bool, true)
|
||||
})
|
||||
default = {}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "federated_identity_providers" {
|
||||
description = "Workload Identity Federation pools. The `cicd_repositories` variable references keys here."
|
||||
type = map(object({
|
||||
attribute_condition = string
|
||||
issuer = string
|
||||
custom_settings = object({
|
||||
issuer_uri = string
|
||||
allowed_audiences = list(string)
|
||||
})
|
||||
}))
|
||||
default = {}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "group_iam" {
|
||||
description = "Tenant-level custom group IAM settings in group => [roles] format."
|
||||
type = map(list(string))
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "iam" {
|
||||
description = "Tenant-level custom IAM settings in role => [principal] format."
|
||||
type = map(list(string))
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "iam_additive" {
|
||||
description = "Tenant-level custom IAM settings in role => [principal] format for non-authoritative bindings."
|
||||
type = map(list(string))
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "locations" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Optional locations for GCS, BigQuery, and logging buckets created here. These are the defaults set at the organization level, and can be overridden via the tenant config variable."
|
||||
type = object({
|
||||
bq = string
|
||||
gcs = string
|
||||
logging = string
|
||||
pubsub = list(string)
|
||||
})
|
||||
default = {
|
||||
bq = "EU"
|
||||
gcs = "EU"
|
||||
logging = "global"
|
||||
pubsub = []
|
||||
}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
# See https://cloud.google.com/architecture/exporting-stackdriver-logging-for-security-and-access-analytics
|
||||
# for additional logging filter examples
|
||||
variable "log_sinks" {
|
||||
description = "Tenant-level log sinks, in name => {type, filter} format."
|
||||
type = map(object({
|
||||
filter = string
|
||||
type = string
|
||||
}))
|
||||
default = {
|
||||
audit-logs = {
|
||||
filter = "logName:\"/logs/cloudaudit.googleapis.com%2Factivity\" OR logName:\"/logs/cloudaudit.googleapis.com%2Fsystem_event\""
|
||||
type = "logging"
|
||||
}
|
||||
}
|
||||
validation {
|
||||
condition = alltrue([
|
||||
for k, v in var.log_sinks :
|
||||
contains(["bigquery", "logging", "pubsub", "storage"], v.type)
|
||||
])
|
||||
error_message = "Type must be one of 'bigquery', 'logging', 'pubsub', 'storage'."
|
||||
}
|
||||
}
|
||||
|
||||
variable "organization" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Organization details."
|
||||
type = object({
|
||||
domain = string
|
||||
id = number
|
||||
customer_id = string
|
||||
})
|
||||
}
|
||||
|
||||
variable "outputs_location" {
|
||||
description = "Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "prefix" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Prefix used for resources that need unique names. Use 9 characters or less."
|
||||
type = string
|
||||
validation {
|
||||
condition = try(length(var.prefix), 0) < 10
|
||||
error_message = "Use a maximum of 9 characters for prefix."
|
||||
}
|
||||
}
|
||||
|
||||
variable "project_parent_ids" {
|
||||
description = "Optional parents for projects created here in folders/nnnnnnn format. Null values will use the tenant folder as parent."
|
||||
type = object({
|
||||
automation = string
|
||||
logging = string
|
||||
})
|
||||
default = {
|
||||
automation = null
|
||||
logging = null
|
||||
}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "tag_keys" {
|
||||
# tfdoc:variable:source 1-resman
|
||||
description = "Organization tag keys."
|
||||
type = object({
|
||||
context = string
|
||||
environment = string
|
||||
tenant = string
|
||||
})
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "tag_names" {
|
||||
# tfdoc:variable:source 1-resman
|
||||
description = "Customized names for resource management tags."
|
||||
type = object({
|
||||
context = string
|
||||
environment = string
|
||||
tenant = string
|
||||
})
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "tag_values" {
|
||||
# tfdoc:variable:source 1-resman
|
||||
description = "Organization resource management tag values."
|
||||
type = map(string)
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "tenant_config" {
|
||||
description = "Tenant configuration. Short name must be 4 characters or less."
|
||||
type = object({
|
||||
descriptive_name = string
|
||||
groups = object({
|
||||
gcp-admins = string
|
||||
gcp-devops = optional(string)
|
||||
gcp-network-admins = optional(string)
|
||||
gcp-security-admins = optional(string)
|
||||
})
|
||||
short_name = string
|
||||
fast_features = optional(object({
|
||||
data_platform = optional(bool)
|
||||
gke = optional(bool)
|
||||
project_factory = optional(bool)
|
||||
sandbox = optional(bool)
|
||||
teams = optional(bool)
|
||||
}), {})
|
||||
locations = optional(object({
|
||||
bq = optional(string)
|
||||
gcs = optional(string)
|
||||
logging = optional(string)
|
||||
pubsub = optional(list(string))
|
||||
}), {})
|
||||
})
|
||||
nullable = false
|
||||
validation {
|
||||
condition = alltrue([
|
||||
for a in ["descriptive_name", "groups", "short_name"] :
|
||||
var.tenant_config[a] != null
|
||||
])
|
||||
error_message = "Non-optional members must not be null."
|
||||
}
|
||||
validation {
|
||||
condition = length(var.tenant_config.short_name) < 5
|
||||
error_message = "Short name must be a string of 4 characters or less."
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
variable "test_principal" {
|
||||
description = "Used when testing to bypass the data source returning the current identity."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
# IAM bindings reference
|
||||
|
||||
Legend: <code>+</code> additive, <code>•</code> conditional.
|
||||
|
||||
## Folder <i>development [#0]</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-gke-dev-0</b><br><small><i>serviceAccount</i></small>|[roles/compute.xpnAdmin](https://cloud.google.com/iam/docs/understanding-roles#compute.xpnAdmin) <br>[roles/logging.admin](https://cloud.google.com/iam/docs/understanding-roles#logging.admin) <br>[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) <br>[roles/resourcemanager.folderAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.folderAdmin) <br>[roles/resourcemanager.projectCreator](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectCreator) |
|
||||
|
||||
## Folder <i>development [#1]</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-gke-dev-0</b><br><small><i>serviceAccount</i></small>|organizations/[org_id #0]/roles/serviceProjectNetworkAdmin |
|
||||
|<b>tn0-pf-dev-0</b><br><small><i>serviceAccount</i></small>|organizations/[org_id #0]/roles/serviceProjectNetworkAdmin |
|
||||
|
||||
## Folder <i>networking</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-networking-0</b><br><small><i>serviceAccount</i></small>|[roles/compute.xpnAdmin](https://cloud.google.com/iam/docs/understanding-roles#compute.xpnAdmin) <br>[roles/logging.admin](https://cloud.google.com/iam/docs/understanding-roles#logging.admin) <br>[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) <br>[roles/resourcemanager.folderAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.folderAdmin) <br>[roles/resourcemanager.projectCreator](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectCreator) |
|
||||
|
||||
## Folder <i>production [#0]</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-gke-prod-0</b><br><small><i>serviceAccount</i></small>|[roles/compute.xpnAdmin](https://cloud.google.com/iam/docs/understanding-roles#compute.xpnAdmin) <br>[roles/logging.admin](https://cloud.google.com/iam/docs/understanding-roles#logging.admin) <br>[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) <br>[roles/resourcemanager.folderAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.folderAdmin) <br>[roles/resourcemanager.projectCreator](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectCreator) |
|
||||
|
||||
## Folder <i>production [#1]</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-gke-prod-0</b><br><small><i>serviceAccount</i></small>|organizations/[org_id #0]/roles/serviceProjectNetworkAdmin |
|
||||
|<b>tn0-pf-prod-0</b><br><small><i>serviceAccount</i></small>|organizations/[org_id #0]/roles/serviceProjectNetworkAdmin |
|
||||
|
||||
## Folder <i>sandbox</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-sandbox-0</b><br><small><i>serviceAccount</i></small>|[roles/logging.admin](https://cloud.google.com/iam/docs/understanding-roles#logging.admin) <br>[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) <br>[roles/resourcemanager.folderAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.folderAdmin) <br>[roles/resourcemanager.projectCreator](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectCreator) |
|
||||
|
||||
## Folder <i>security</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-security-0</b><br><small><i>serviceAccount</i></small>|[roles/logging.admin](https://cloud.google.com/iam/docs/understanding-roles#logging.admin) <br>[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) <br>[roles/resourcemanager.folderAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.folderAdmin) <br>[roles/resourcemanager.projectCreator](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectCreator) |
|
||||
|
||||
## Folder <i>teams</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-teams-0</b><br><small><i>serviceAccount</i></small>|[roles/compute.xpnAdmin](https://cloud.google.com/iam/docs/understanding-roles#compute.xpnAdmin) <br>[roles/logging.admin](https://cloud.google.com/iam/docs/understanding-roles#logging.admin) <br>[roles/owner](https://cloud.google.com/iam/docs/understanding-roles#owner) <br>[roles/resourcemanager.folderAdmin](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.folderAdmin) <br>[roles/resourcemanager.projectCreator](https://cloud.google.com/iam/docs/understanding-roles#resourcemanager.projectCreator) |
|
||||
|
||||
## Folder <i>test tenant 0</i>
|
||||
|
||||
| members | roles |
|
||||
|---|---|
|
||||
|<b>tn0-networking-0</b><br><small><i>serviceAccount</i></small>|[roles/compute.orgFirewallPolicyAdmin](https://cloud.google.com/iam/docs/understanding-roles#compute.orgFirewallPolicyAdmin) <code>+</code><br>[roles/compute.xpnAdmin](https://cloud.google.com/iam/docs/understanding-roles#compute.xpnAdmin) <code>+</code>|
|
||||
|<b>tn0-security-0</b><br><small><i>serviceAccount</i></small>|[roles/accesscontextmanager.policyAdmin](https://cloud.google.com/iam/docs/understanding-roles#accesscontextmanager.policyAdmin) <code>+</code>|
|
||||
|
|
@ -0,0 +1,184 @@
|
|||
# Tenant resource management
|
||||
|
||||
This stage is run for a specific tenant after [tenant bootstrap](../0-bootstrap-tenant/) has successfully created initial resources for the tenant, which is then decoupled from the organization.
|
||||
|
||||
It is logically equivalent and almost identical in code to the corresponding [organization resource management stage](../../stages/1-resman/), with a few notable differences:
|
||||
|
||||
- the hierarchy is rooted in the tenant top-level folder instead of the organization
|
||||
- there's no management of tag values and keys since they organization-level resources (it could be implemented for tenant-specific tags if the need arises)
|
||||
- automation service accounts for subsequent stages are configured but not created here (tenant-level bootstrap creates them and assigns organization-level permissions)
|
||||
|
||||
The stage runs with a dedicated service account for the tenant, which has no permissions at the organization level except for billing and organization policies, constrained by a condition on the tenant tag.
|
||||
|
||||
The following diagram is a high level reference of what this stage manages, showing one hypothetical tenant (additional tenants require additional instances of this stage being deployed):
|
||||
|
||||
```mermaid
|
||||
%%{init: {'theme':'base'}}%%
|
||||
classDiagram
|
||||
Tenant_root~📁~ -- tn0_automation
|
||||
Tenant_root~📁~ -- Networking~📁~
|
||||
Tenant_root~📁~ -- Security~📁~
|
||||
Tenant_root~📁~ -- Data_Platform~📁~
|
||||
Data_Platform~📁~ -- DP_Dev~📁~
|
||||
Data_Platform~📁~ -- DP_Prod~📁~
|
||||
Tenant_root~📁~ -- GKE~📁~
|
||||
GKE~📁~ -- GKE_Dev~📁~
|
||||
GKE~📁~ -- GKE_Prod~📁~
|
||||
Tenant_root~📁~ -- Teams~📁~
|
||||
Teams~📁~ -- Team_0~📁~
|
||||
Team_0~📁~ -- Team_0_Dev~📁~
|
||||
Team_0~📁~ -- Team_0_Prod~📁~
|
||||
Tenant_root~📁~ -- Sandbox~📁~
|
||||
class Tenant_root~📁~ {
|
||||
- IAM bindings()
|
||||
- org policies()
|
||||
}
|
||||
class tn0_automation {
|
||||
- GCS buckets
|
||||
- IAM bindings()
|
||||
}
|
||||
class Data_Platform~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class DP_Dev~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class DP_Prod~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class GKE~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class GKE_Dev~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class GKE_Prod~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class Networking~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class Security~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class Sandbox~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class Teams~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class Team_0~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class Team_0_Dev~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
class Team_0_Prod~📁~ {
|
||||
- IAM bindings()
|
||||
- tag bindings()
|
||||
}
|
||||
```
|
||||
|
||||
As most of the features of this stage follow the same design and configurations of the [organization-level resource management stage](../../stages/1-resman/), we will only focus on the tenant-specific configuration in this document.
|
||||
|
||||
## How to run this stage
|
||||
|
||||
As mentioned above this stage is decoupled from organization-level stages: it uses a service account and state bucket from the tenant-specific automation project, and its tfvars and provider files are also tenant-specific.
|
||||
|
||||
The `stage-links.sh` script can be used to get the commands needed for the provider and output files, just set the variable for the tenant shortname (the same one specified in the tenant bootstrap stage) and pass a single argument with your FAST output files folder path, or GCS bucket URI:
|
||||
|
||||
```bash
|
||||
TENANT=tn0 ../../stage-links.sh ~/fast-config
|
||||
```
|
||||
|
||||
The script output can be copy/pasted to a terminal:
|
||||
|
||||
```bash
|
||||
# copy and paste the following commands for '1-resman-tenant'
|
||||
|
||||
ln -s ~/fast-config/tenants/tn0/providers/1-resman-tenant-providers.tf ./
|
||||
ln -s ~/fast-config/tenants/tn0/tfvars/0-bootstrap-tenant.auto.tfvars.json ./
|
||||
```
|
||||
|
||||
Once that is done, stage-level configuration variables are the same as the corresponding organization-level stage.
|
||||
|
||||
### Running the stage
|
||||
|
||||
Once the configuration is done just go through the usual `init/apply` cycle. On successful apply, a tfvars file specific for this tenant and a set of provider files will be created.
|
||||
|
||||
<!-- TFDOC OPTS files:1 show_extra:1 -->
|
||||
<!-- BEGIN TFDOC -->
|
||||
|
||||
## Files
|
||||
|
||||
| name | description | modules | resources |
|
||||
|---|---|---|---|
|
||||
| [branch-data-platform.tf](./branch-data-platform.tf) | Data Platform stages resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-gke.tf](./branch-gke.tf) | GKE multitenant stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-networking.tf](./branch-networking.tf) | Networking stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-project-factory.tf](./branch-project-factory.tf) | Project factory stage resources. | <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-sandbox.tf](./branch-sandbox.tf) | Sandbox stage resources. | <code>folder</code> · <code>gcs</code> | |
|
||||
| [branch-security.tf](./branch-security.tf) | Security stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-teams.tf](./branch-teams.tf) | Team stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [cicd-data-platform.tf](./cicd-data-platform.tf) | CI/CD resources for the data platform branch. | <code>iam-service-account</code> · <code>source-repository</code> | |
|
||||
| [cicd-gke.tf](./cicd-gke.tf) | CI/CD resources for the data platform branch. | <code>iam-service-account</code> · <code>source-repository</code> | |
|
||||
| [cicd-networking.tf](./cicd-networking.tf) | CI/CD resources for the networking branch. | <code>iam-service-account</code> · <code>source-repository</code> | |
|
||||
| [cicd-project-factory.tf](./cicd-project-factory.tf) | CI/CD resources for the teams branch. | <code>iam-service-account</code> · <code>source-repository</code> | |
|
||||
| [cicd-security.tf](./cicd-security.tf) | CI/CD resources for the security branch. | <code>iam-service-account</code> · <code>source-repository</code> | |
|
||||
| [main.tf](./main.tf) | Module-level locals and resources. | | |
|
||||
| [outputs-files.tf](./outputs-files.tf) | Output files persistence to local filesystem. | | <code>local_file</code> |
|
||||
| [outputs-gcs.tf](./outputs-gcs.tf) | Output files persistence to automation GCS bucket. | | <code>google_storage_bucket_object</code> |
|
||||
| [outputs.tf](./outputs.tf) | Module outputs. | | |
|
||||
| [root_node.tf](./root_node.tf) | Tenant root folder configuration. | <code>folder</code> | |
|
||||
| [variables.tf](./variables.tf) | Module variables. | | |
|
||||
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default | producer |
|
||||
|---|---|:---:|:---:|:---:|:---:|
|
||||
| [automation](variables.tf#L20) | Automation resources created by the bootstrap stage. | <code title="object({ outputs_bucket = string project_id = string project_number = string federated_identity_pools = list(string) federated_identity_providers = map(object({ issuer = string issuer_uri = string name = string principal_tpl = string principalset_tpl = string })) service_accounts = object({ networking = string resman = string security = string dp-dev = optional(string) dp-prod = optional(string) gke-dev = optional(string) gke-prod = optional(string) pf-dev = optional(string) pf-prod = optional(string) sandbox = optional(string) teams = optional(string) }) })">object({…})</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [billing_account](variables.tf#L51) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | <code title="object({ id = string is_org_level = optional(bool, true) })">object({…})</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [organization](variables.tf#L206) | Organization details. | <code title="object({ domain = string id = number customer_id = string })">object({…})</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [prefix](variables.tf#L228) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [root_node](variables.tf#L239) | Root folder node for the tenant, in folders/nnnnnn format. | <code>string</code> | ✓ | | |
|
||||
| [short_name](variables.tf#L244) | Short name used to identify the tenant. | <code>string</code> | ✓ | | |
|
||||
| [tags](variables.tf#L249) | Resource management tags. | <code title="object({ keys = object({ context = string environment = string tenant = string }) names = object({ context = string environment = string tenant = string }) values = map(string) })">object({…})</code> | ✓ | | |
|
||||
| [cicd_repositories](variables.tf#L64) | CI/CD repository configuration. Identity providers reference keys in the `automation.federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | <code title="object({ data_platform_dev = object({ branch = string identity_provider = string name = string type = string }) data_platform_prod = object({ branch = string identity_provider = string name = string type = string }) gke_dev = object({ branch = string identity_provider = string name = string type = string }) gke_prod = object({ branch = string identity_provider = string name = string type = string }) networking = object({ branch = string identity_provider = string name = string type = string }) project_factory_dev = object({ branch = string identity_provider = string name = string type = string }) project_factory_prod = object({ branch = string identity_provider = string name = string type = string }) security = object({ branch = string identity_provider = string name = string type = string }) })">object({…})</code> | | <code>null</code> | |
|
||||
| [custom_roles](variables.tf#L146) | Custom roles defined at the org level, in key => id format. | <code title="object({ service_project_network_admin = string })">object({…})</code> | | <code>null</code> | <code>0-bootstrap</code> |
|
||||
| [data_dir](variables.tf#L155) | Relative path for the folder storing configuration data. | <code>string</code> | | <code>"data"</code> | |
|
||||
| [fast_features](variables.tf#L161) | Selective control for top-level FAST features. | <code title="object({ data_platform = optional(bool, false) gke = optional(bool, false) project_factory = optional(bool, false) sandbox = optional(bool, false) teams = optional(bool, false) })">object({…})</code> | | <code>{}</code> | <code>0-0-bootstrap</code> |
|
||||
| [groups](variables.tf#L175) | Group names to grant organization-level permissions. | <code title="object({ gcp-devops = optional(string) gcp-network-admins = optional(string) gcp-security-admins = optional(string) })">object({…})</code> | | <code>{}</code> | <code>0-bootstrap</code> |
|
||||
| [locations](variables.tf#L188) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object({ bq = string gcs = string logging = string pubsub = list(string) })">object({…})</code> | | <code title="{ bq = "EU" gcs = "EU" logging = "global" pubsub = [] }">{…}</code> | <code>0-bootstrap</code> |
|
||||
| [organization_policy_data_path](variables.tf#L216) | Path for the data folder used by the organization policies factory. | <code>string</code> | | <code>null</code> | |
|
||||
| [outputs_location](variables.tf#L222) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | <code>string</code> | | <code>null</code> | |
|
||||
| [team_folders](variables.tf#L267) | Team folders to be created. Format is described in a code comment. | <code title="map(object({ descriptive_name = string group_iam = map(list(string)) impersonation_groups = list(string) }))">map(object({…}))</code> | | <code>null</code> | |
|
||||
| [test_skip_data_sources](variables.tf#L277) | Used when testing to bypass data sources. | <code>bool</code> | | <code>false</code> | |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive | consumers |
|
||||
|---|---|:---:|---|
|
||||
| [cicd_repositories](outputs.tf#L189) | WIF configuration for CI/CD repositories. | | |
|
||||
| [dataplatform](outputs.tf#L203) | Data for the Data Platform stage. | | |
|
||||
| [gke_multitenant](outputs.tf#L219) | Data for the GKE multitenant stage. | | <code>03-gke-multitenant</code> |
|
||||
| [networking](outputs.tf#L240) | Data for the networking stage. | | |
|
||||
| [project_factories](outputs.tf#L249) | Data for the project factories stage. | | |
|
||||
| [providers](outputs.tf#L264) | Terraform provider files for this stage and dependent stages. | ✓ | <code>02-networking</code> · <code>02-security</code> · <code>03-dataplatform</code> · <code>xx-sandbox</code> · <code>xx-teams</code> |
|
||||
| [sandbox](outputs.tf#L271) | Data for the sandbox stage. | | <code>xx-sandbox</code> |
|
||||
| [security](outputs.tf#L285) | Data for the networking stage. | | <code>02-security</code> |
|
||||
| [teams](outputs.tf#L295) | Data for the teams stage. | | |
|
||||
| [tfvars](outputs.tf#L307) | Terraform variable files for the following stages. | ✓ | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -0,0 +1,133 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Data Platform stages resources.
|
||||
|
||||
module "branch-dp-folder" {
|
||||
source = "../../../modules/folder"
|
||||
count = var.fast_features.data_platform ? 1 : 0
|
||||
parent = module.root-folder.id
|
||||
name = "Data Platform"
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.context}/data"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-dp-dev-folder" {
|
||||
source = "../../../modules/folder"
|
||||
count = var.fast_features.data_platform ? 1 : 0
|
||||
parent = module.branch-dp-folder.0.id
|
||||
name = "Development"
|
||||
group_iam = {}
|
||||
iam = {
|
||||
(local.custom_roles.service_project_network_admin) = [
|
||||
local.automation_sas_iam.dp-dev
|
||||
]
|
||||
# remove owner here and at project level if SA does not manage project resources
|
||||
"roles/owner" = [local.automation_sas_iam.dp-dev]
|
||||
"roles/logging.admin" = [local.automation_sas_iam.dp-dev]
|
||||
"roles/resourcemanager.folderAdmin" = [local.automation_sas_iam.dp-dev]
|
||||
"roles/resourcemanager.projectCreator" = [local.automation_sas_iam.dp-dev]
|
||||
}
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.environment}/development"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-dp-prod-folder" {
|
||||
source = "../../../modules/folder"
|
||||
count = var.fast_features.data_platform ? 1 : 0
|
||||
parent = module.branch-dp-folder.0.id
|
||||
name = "Production"
|
||||
group_iam = {}
|
||||
iam = {
|
||||
(local.custom_roles.service_project_network_admin) = [
|
||||
local.automation_sas_iam.dp-prod
|
||||
]
|
||||
# remove owner here and at project level if SA does not manage project resources
|
||||
"roles/owner" = [local.automation_sas_iam.dp-prod]
|
||||
"roles/logging.admin" = [local.automation_sas_iam.dp-prod]
|
||||
"roles/resourcemanager.folderAdmin" = [local.automation_sas_iam.dp-prod]
|
||||
"roles/resourcemanager.projectCreator" = [local.automation_sas_iam.dp-prod]
|
||||
}
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.environment}/production"]
|
||||
}
|
||||
}
|
||||
|
||||
# automation service accounts and buckets
|
||||
|
||||
module "branch-dp-dev-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
count = var.fast_features.data_platform ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "dp-dev-0"
|
||||
prefix = var.prefix
|
||||
service_account_create = var.test_skip_data_sources
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.branch-dp-dev-sa-cicd.0.iam_email, null)
|
||||
])
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-dp-prod-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
count = var.fast_features.data_platform ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "dp-prod-0"
|
||||
prefix = var.prefix
|
||||
service_account_create = var.test_skip_data_sources
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.branch-dp-prod-sa-cicd.0.iam_email, null)
|
||||
])
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-dp-dev-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
count = var.fast_features.data_platform ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "dev-resman-dp-0"
|
||||
prefix = var.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [local.automation_sas_iam.dp-dev]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-dp-prod-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
count = var.fast_features.data_platform ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-dp-0"
|
||||
prefix = var.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [local.automation_sas_iam.dp-prod]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,133 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description GKE multitenant stage resources.
|
||||
|
||||
module "branch-gke-folder" {
|
||||
source = "../../../modules/folder"
|
||||
count = var.fast_features.gke ? 1 : 0
|
||||
parent = module.root-folder.id
|
||||
name = "GKE"
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.context}/gke"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-gke-dev-folder" {
|
||||
source = "../../../modules/folder"
|
||||
count = var.fast_features.gke ? 1 : 0
|
||||
parent = module.branch-gke-folder.0.id
|
||||
name = "Development"
|
||||
iam = {
|
||||
"roles/owner" = [local.automation_sas_iam.gke-dev]
|
||||
"roles/logging.admin" = [local.automation_sas_iam.gke-dev]
|
||||
"roles/resourcemanager.folderAdmin" = [local.automation_sas_iam.gke-dev]
|
||||
"roles/resourcemanager.projectCreator" = [local.automation_sas_iam.gke-dev]
|
||||
"roles/compute.xpnAdmin" = [local.automation_sas_iam.gke-dev]
|
||||
}
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.environment}/development"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-gke-prod-folder" {
|
||||
source = "../../../modules/folder"
|
||||
count = var.fast_features.gke ? 1 : 0
|
||||
parent = module.branch-gke-folder.0.id
|
||||
name = "Production"
|
||||
iam = {
|
||||
"roles/owner" = [local.automation_sas_iam.gke-prod]
|
||||
"roles/logging.admin" = [local.automation_sas_iam.gke-prod]
|
||||
"roles/resourcemanager.folderAdmin" = [local.automation_sas_iam.gke-prod]
|
||||
"roles/resourcemanager.projectCreator" = [local.automation_sas_iam.gke-prod]
|
||||
"roles/compute.xpnAdmin" = [local.automation_sas_iam.gke-prod]
|
||||
}
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.environment}/production"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-gke-dev-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
count = var.fast_features.gke ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "gke-dev-0"
|
||||
prefix = var.prefix
|
||||
service_account_create = var.test_skip_data_sources
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = concat(
|
||||
(
|
||||
local.groups.gcp-devops == null
|
||||
? []
|
||||
: ["group:${local.groups.gcp-devops}"]
|
||||
),
|
||||
compact([
|
||||
try(module.branch-gke-dev-sa-cicd.0.iam_email, null)
|
||||
])
|
||||
)
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-gke-prod-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
count = var.fast_features.gke ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "gke-prod-0"
|
||||
prefix = var.prefix
|
||||
service_account_create = var.test_skip_data_sources
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = concat(
|
||||
(
|
||||
local.groups.gcp-devops == null
|
||||
? []
|
||||
: ["group:${local.groups.gcp-devops}"]
|
||||
),
|
||||
compact([
|
||||
try(module.branch-gke-prod-sa-cicd.0.iam_email, null)
|
||||
])
|
||||
)
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-gke-dev-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
count = var.fast_features.gke ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "dev-resman-gke-0"
|
||||
prefix = var.prefix
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [local.automation_sas_iam.gke-dev]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-gke-prod-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
count = var.fast_features.gke ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-gke-0"
|
||||
prefix = var.prefix
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [local.automation_sas_iam.gke-prod]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,107 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Networking stage resources.
|
||||
|
||||
module "branch-network-folder" {
|
||||
source = "../../../modules/folder"
|
||||
parent = module.root-folder.id
|
||||
name = "Networking"
|
||||
group_iam = local.groups.gcp-network-admins == null ? {} : {
|
||||
(local.groups.gcp-network-admins) = [
|
||||
# add any needed roles for resources/services not managed via Terraform,
|
||||
# or replace editor with ~viewer if no broad resource management needed
|
||||
# e.g.
|
||||
# "roles/compute.networkAdmin",
|
||||
# "roles/dns.admin",
|
||||
# "roles/compute.securityAdmin",
|
||||
"roles/editor",
|
||||
]
|
||||
}
|
||||
iam = {
|
||||
"roles/logging.admin" = [local.automation_sas_iam.networking]
|
||||
"roles/owner" = [local.automation_sas_iam.networking]
|
||||
"roles/resourcemanager.folderAdmin" = [local.automation_sas_iam.networking]
|
||||
"roles/resourcemanager.projectCreator" = [local.automation_sas_iam.networking]
|
||||
"roles/compute.xpnAdmin" = [local.automation_sas_iam.networking]
|
||||
}
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.context}/networking"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-network-prod-folder" {
|
||||
source = "../../../modules/folder"
|
||||
parent = module.branch-network-folder.id
|
||||
name = "Production"
|
||||
iam = {
|
||||
(local.custom_roles.service_project_network_admin) = concat(
|
||||
local.branch_optional_sa_lists.dp-prod,
|
||||
local.branch_optional_sa_lists.gke-prod,
|
||||
local.branch_optional_sa_lists.pf-prod,
|
||||
)
|
||||
}
|
||||
tag_bindings = {
|
||||
environment = var.tags.values["${var.tags.names.environment}/production"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-network-dev-folder" {
|
||||
source = "../../../modules/folder"
|
||||
parent = module.branch-network-folder.id
|
||||
name = "Development"
|
||||
iam = {
|
||||
(local.custom_roles.service_project_network_admin) = concat(
|
||||
local.branch_optional_sa_lists.dp-dev,
|
||||
local.branch_optional_sa_lists.gke-dev,
|
||||
local.branch_optional_sa_lists.pf-dev,
|
||||
)
|
||||
}
|
||||
tag_bindings = {
|
||||
environment = var.tags.values["${var.tags.names.environment}/development"]
|
||||
}
|
||||
}
|
||||
|
||||
# automation service account and bucket
|
||||
|
||||
module "branch-network-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
project_id = var.automation.project_id
|
||||
name = "networking-0"
|
||||
prefix = var.prefix
|
||||
service_account_create = var.test_skip_data_sources
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.branch-network-sa-cicd.0.iam_email, null)
|
||||
])
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-network-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-net-0"
|
||||
prefix = var.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [local.automation_sas_iam.networking]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Project factory stage resources.
|
||||
|
||||
module "branch-pf-dev-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
count = var.fast_features.project_factory ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "pf-dev-0"
|
||||
prefix = var.prefix
|
||||
service_account_create = var.test_skip_data_sources
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.branch-pf-dev-sa-cicd.0.iam_email, null)
|
||||
])
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-pf-prod-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
count = var.fast_features.project_factory ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "pf-prod-0"
|
||||
prefix = var.prefix
|
||||
service_account_create = var.test_skip_data_sources
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.branch-pf-prod-sa-cicd.0.iam_email, null)
|
||||
])
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-pf-dev-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
count = var.fast_features.project_factory ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "dev-resman-pf-0"
|
||||
prefix = var.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [local.automation_sas_iam.pf-dev]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-pf-prod-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
count = var.fast_features.project_factory ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-pf-0"
|
||||
prefix = var.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [local.automation_sas_iam.pf-prod]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Sandbox stage resources.
|
||||
|
||||
module "branch-sandbox-folder" {
|
||||
source = "../../../modules/folder"
|
||||
count = var.fast_features.sandbox ? 1 : 0
|
||||
parent = module.root-folder.id
|
||||
name = "Sandbox"
|
||||
iam = {
|
||||
"roles/logging.admin" = [local.automation_sas_iam.sandbox]
|
||||
"roles/owner" = [local.automation_sas_iam.sandbox]
|
||||
"roles/resourcemanager.folderAdmin" = [local.automation_sas_iam.sandbox]
|
||||
"roles/resourcemanager.projectCreator" = [local.automation_sas_iam.sandbox]
|
||||
}
|
||||
org_policies = {
|
||||
"constraints/sql.restrictPublicIp" = { enforce = false }
|
||||
"constraints/compute.vmExternalIpAccess" = { allow = { all = true } }
|
||||
}
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.context}/sandbox"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-sandbox-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
count = var.fast_features.sandbox ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "dev-resman-sbox-0"
|
||||
prefix = var.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [local.automation_sas_iam.sandbox]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Security stage resources.
|
||||
|
||||
module "branch-security-folder" {
|
||||
source = "../../../modules/folder"
|
||||
parent = module.root-folder.id
|
||||
name = "Security"
|
||||
group_iam = local.groups.gcp-security-admins == null ? {} : {
|
||||
(local.groups.gcp-security-admins) = [
|
||||
# add any needed roles for resources/services not managed via Terraform,
|
||||
# e.g.
|
||||
# "roles/bigquery.admin",
|
||||
# "roles/cloudasset.owner",
|
||||
# "roles/cloudkms.admin",
|
||||
# "roles/logging.admin",
|
||||
# "roles/secretmanager.admin",
|
||||
# "roles/storage.admin",
|
||||
"roles/viewer"
|
||||
]
|
||||
}
|
||||
iam = {
|
||||
"roles/logging.admin" = [local.automation_sas_iam.security]
|
||||
"roles/owner" = [local.automation_sas_iam.security]
|
||||
"roles/resourcemanager.folderAdmin" = [local.automation_sas_iam.security]
|
||||
"roles/resourcemanager.projectCreator" = [local.automation_sas_iam.security]
|
||||
}
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.context}/security"]
|
||||
}
|
||||
}
|
||||
|
||||
# automation service account and bucket
|
||||
|
||||
module "branch-security-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
project_id = var.automation.project_id
|
||||
name = "security-0"
|
||||
prefix = var.prefix
|
||||
service_account_create = var.test_skip_data_sources
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.branch-security-sa-cicd.0.iam_email, null)
|
||||
])
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-security-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-sec-0"
|
||||
prefix = var.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [local.automation_sas_iam.security]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,163 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Team stage resources.
|
||||
|
||||
# TODO(ludo): add support for CI/CD
|
||||
|
||||
############### top-level Teams branch and automation resources ###############
|
||||
|
||||
module "branch-teams-folder" {
|
||||
source = "../../../modules/folder"
|
||||
count = var.fast_features.teams ? 1 : 0
|
||||
parent = module.root-folder.id
|
||||
name = "Teams"
|
||||
iam = {
|
||||
"roles/logging.admin" = [local.automation_sas_iam.teams]
|
||||
"roles/owner" = [local.automation_sas_iam.teams]
|
||||
"roles/resourcemanager.folderAdmin" = [local.automation_sas_iam.teams]
|
||||
"roles/resourcemanager.projectCreator" = [local.automation_sas_iam.teams]
|
||||
"roles/compute.xpnAdmin" = [local.automation_sas_iam.teams]
|
||||
}
|
||||
tag_bindings = {
|
||||
context = var.tags.values["${var.tags.names.context}/teams"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-teams-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
count = var.fast_features.teams ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "teams-0"
|
||||
prefix = var.prefix
|
||||
service_account_create = var.test_skip_data_sources
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-teams-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
count = var.fast_features.teams ? 1 : 0
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-teams-0"
|
||||
prefix = var.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [module.branch-teams-sa.0.iam_email]
|
||||
}
|
||||
}
|
||||
|
||||
################## per-team folders and automation resources ##################
|
||||
|
||||
module "branch-teams-team-folder" {
|
||||
source = "../../../modules/folder"
|
||||
for_each = var.fast_features.teams ? coalesce(var.team_folders, {}) : {}
|
||||
parent = module.branch-teams-folder.0.id
|
||||
name = each.value.descriptive_name
|
||||
iam = {
|
||||
"roles/logging.admin" = [module.branch-teams-team-sa[each.key].iam_email]
|
||||
"roles/owner" = [module.branch-teams-team-sa[each.key].iam_email]
|
||||
"roles/resourcemanager.folderAdmin" = [module.branch-teams-team-sa[each.key].iam_email]
|
||||
"roles/resourcemanager.projectCreator" = [module.branch-teams-team-sa[each.key].iam_email]
|
||||
"roles/compute.xpnAdmin" = [module.branch-teams-team-sa[each.key].iam_email]
|
||||
}
|
||||
group_iam = each.value.group_iam == null ? {} : each.value.group_iam
|
||||
}
|
||||
|
||||
module "branch-teams-team-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = var.fast_features.teams ? coalesce(var.team_folders, {}) : {}
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-teams-${each.key}-0"
|
||||
display_name = "Terraform team ${each.key} service account."
|
||||
prefix = var.prefix
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = (
|
||||
each.value.impersonation_groups == null
|
||||
? []
|
||||
: [for g in each.value.impersonation_groups : "group:${g}"]
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-teams-team-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
for_each = var.fast_features.teams ? coalesce(var.team_folders, {}) : {}
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-teams-${each.key}-0"
|
||||
prefix = var.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [module.branch-teams-team-sa[each.key].iam_email]
|
||||
}
|
||||
}
|
||||
|
||||
# per-team environment folders where project factory SAs can create projects
|
||||
|
||||
module "branch-teams-team-dev-folder" {
|
||||
source = "../../../modules/folder"
|
||||
for_each = var.fast_features.teams ? coalesce(var.team_folders, {}) : {}
|
||||
parent = module.branch-teams-team-folder[each.key].id
|
||||
# naming: environment descriptive name
|
||||
name = "Development"
|
||||
# environment-wide human permissions on the whole teams environment
|
||||
group_iam = {}
|
||||
iam = {
|
||||
(local.custom_roles.service_project_network_admin) = (
|
||||
local.branch_optional_sa_lists.pf-dev
|
||||
)
|
||||
# remove owner here and at project level if SA does not manage project resources
|
||||
"roles/owner" = local.branch_optional_sa_lists.pf-dev
|
||||
"roles/logging.admin" = local.branch_optional_sa_lists.pf-dev
|
||||
"roles/resourcemanager.folderAdmin" = local.branch_optional_sa_lists.pf-dev
|
||||
"roles/resourcemanager.projectCreator" = local.branch_optional_sa_lists.pf-dev
|
||||
}
|
||||
tag_bindings = {
|
||||
environment = try(
|
||||
var.tags.values["${var.tags.names.environment}/development"], null
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-teams-team-prod-folder" {
|
||||
source = "../../../modules/folder"
|
||||
for_each = var.fast_features.teams ? coalesce(var.team_folders, {}) : {}
|
||||
parent = module.branch-teams-team-folder[each.key].id
|
||||
# naming: environment descriptive name
|
||||
name = "Production"
|
||||
# environment-wide human permissions on the whole teams environment
|
||||
group_iam = {}
|
||||
iam = {
|
||||
(local.custom_roles.service_project_network_admin) = (
|
||||
local.branch_optional_sa_lists.pf-prod
|
||||
)
|
||||
# remove owner here and at project level if SA does not manage project resources
|
||||
"roles/owner" = local.branch_optional_sa_lists.pf-prod
|
||||
"roles/logging.admin" = local.branch_optional_sa_lists.pf-prod
|
||||
"roles/resourcemanager.folderAdmin" = local.branch_optional_sa_lists.pf-prod
|
||||
"roles/resourcemanager.projectCreator" = local.branch_optional_sa_lists.pf-prod
|
||||
}
|
||||
tag_bindings = {
|
||||
environment = try(
|
||||
var.tags.values["${var.tags.names.environment}/production"], null
|
||||
)
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,173 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description CI/CD resources for the data platform branch.
|
||||
|
||||
# source repositories
|
||||
|
||||
module "branch-dp-dev-cicd-repo" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.data_platform_dev.type, null) == "sourcerepo"
|
||||
? { 0 = local.cicd_repositories.data_platform_dev }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = local.branch_optional_sa_lists.dp-dev
|
||||
"roles/source.reader" = compact([
|
||||
try(module.branch-dp-dev-sa-cicd.0.iam_email, "")
|
||||
])
|
||||
}
|
||||
triggers = {
|
||||
fast-03-dp-dev = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = [
|
||||
"**/*json", "**/*tf", "**/*yaml", ".cloudbuild/workflow.yaml"
|
||||
]
|
||||
service_account = module.branch-dp-dev-sa-cicd.0.id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
depends_on = [module.branch-dp-dev-sa-cicd]
|
||||
}
|
||||
|
||||
module "branch-dp-prod-cicd-repo" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.data_platform_prod.type, null) == "sourcerepo"
|
||||
? { 0 = local.cicd_repositories.data_platform_prod }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = local.branch_optional_sa_lists.dp-prod
|
||||
"roles/source.reader" = [module.branch-dp-prod-sa-cicd.0.iam_email]
|
||||
}
|
||||
triggers = {
|
||||
fast-03-dp-prod = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = [
|
||||
"**/*json", "**/*tf", "**/*yaml", ".cloudbuild/workflow.yaml"
|
||||
]
|
||||
service_account = module.branch-dp-prod-sa-cicd.0.id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
depends_on = [module.branch-dp-prod-sa-cicd]
|
||||
}
|
||||
|
||||
# SAs used by CI/CD workflows to impersonate automation SAs
|
||||
|
||||
module "branch-dp-dev-sa-cicd" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.data_platform_dev.name, null) != null
|
||||
? { 0 = local.cicd_repositories.data_platform_dev }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = "dev-resman-dp-1"
|
||||
display_name = "Terraform CI/CD data platform development service account."
|
||||
prefix = var.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {
|
||||
"roles/iam.serviceAccountUser" = local.automation_resman_sa_iam
|
||||
}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(var.automation.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-dp-prod-sa-cicd" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.data_platform_prod.name, null) != null
|
||||
? { 0 = local.cicd_repositories.data_platform_prod }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-dp-1"
|
||||
display_name = "Terraform CI/CD data platform production service account."
|
||||
prefix = var.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {
|
||||
"roles/iam.serviceAccountUser" = local.automation_resman_sa_iam
|
||||
}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(var.automation.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,175 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description CI/CD resources for the data platform branch.
|
||||
|
||||
# source repositories
|
||||
|
||||
module "branch-gke-dev-cicd-repo" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.gke_dev.type, null) == "sourcerepo"
|
||||
? { 0 = local.cicd_repositories.gke_dev }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = compact([
|
||||
try(module.branch-gke-dev-sa.0.iam_email, "")
|
||||
])
|
||||
"roles/source.reader" = compact([
|
||||
try(module.branch-gke-dev-sa-cicd.0.iam_email, "")
|
||||
])
|
||||
}
|
||||
triggers = {
|
||||
fast-03-gke-dev = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = [
|
||||
"**/*json", "**/*tf", "**/*yaml", ".cloudbuild/workflow.yaml"
|
||||
]
|
||||
service_account = module.branch-gke-dev-sa-cicd.0.id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
depends_on = [module.branch-gke-dev-sa-cicd]
|
||||
}
|
||||
|
||||
module "branch-gke-prod-cicd-repo" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.gke_prod.type, null) == "sourcerepo"
|
||||
? { 0 = local.cicd_repositories.gke_prod }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = [module.branch-gke-prod-sa.0.iam_email]
|
||||
"roles/source.reader" = [module.branch-gke-prod-sa-cicd.0.iam_email]
|
||||
}
|
||||
triggers = {
|
||||
fast-03-gke-prod = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = [
|
||||
"**/*json", "**/*tf", "**/*yaml", ".cloudbuild/workflow.yaml"
|
||||
]
|
||||
service_account = module.branch-gke-prod-sa-cicd.0.id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
depends_on = [module.branch-gke-prod-sa-cicd]
|
||||
}
|
||||
|
||||
# SAs used by CI/CD workflows to impersonate automation SAs
|
||||
|
||||
module "branch-gke-dev-sa-cicd" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.gke_dev.name, null) != null
|
||||
? { 0 = local.cicd_repositories.gke_dev }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = "dev-resman-gke-1"
|
||||
display_name = "Terraform CI/CD GKE development service account."
|
||||
prefix = var.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {
|
||||
"roles/iam.serviceAccountUser" = local.automation_resman_sa_iam
|
||||
}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(var.automation.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
|
||||
module "branch-gke-prod-sa-cicd" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.gke_prod.name, null) != null
|
||||
? { 0 = local.cicd_repositories.gke_prod }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-gke-1"
|
||||
display_name = "Terraform CI/CD GKE production service account."
|
||||
prefix = var.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {
|
||||
"roles/iam.serviceAccountUser" = local.automation_resman_sa_iam
|
||||
}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(var.automation.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,94 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description CI/CD resources for the networking branch.
|
||||
|
||||
# source repository
|
||||
|
||||
module "branch-network-cicd-repo" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.networking.type, null) == "sourcerepo"
|
||||
? { 0 = local.cicd_repositories.networking }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = [module.branch-network-sa.iam_email]
|
||||
"roles/source.reader" = [module.branch-network-sa-cicd.0.iam_email]
|
||||
}
|
||||
triggers = {
|
||||
fast-02-networking = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = ["**/*tf", ".cloudbuild/workflow.yaml"]
|
||||
service_account = module.branch-network-sa-cicd.0.id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
depends_on = [module.branch-network-sa-cicd]
|
||||
}
|
||||
|
||||
# SA used by CI/CD workflows to impersonate automation SAs
|
||||
|
||||
module "branch-network-sa-cicd" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.networking.name, null) != null
|
||||
? { 0 = local.cicd_repositories.networking }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-net-1"
|
||||
display_name = "Terraform CI/CD stage 2 networking service account."
|
||||
prefix = var.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {
|
||||
"roles/iam.serviceAccountUser" = local.automation_resman_sa_iam
|
||||
}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(var.automation.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,191 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description CI/CD resources for the teams branch.
|
||||
|
||||
# source repositories
|
||||
|
||||
moved {
|
||||
from = module.branch-teams-dev-pf-cicd-repo
|
||||
to = module.branch-pf-dev-cicd-repo
|
||||
}
|
||||
|
||||
module "branch-pf-dev-cicd-repo" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.project_factory_dev.type, null) == "sourcerepo"
|
||||
? { 0 = local.cicd_repositories.project_factory_dev }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = local.branch_optional_sa_lists.pf-dev
|
||||
"roles/source.reader" = [module.branch-pf-dev-sa-cicd.0.iam_email]
|
||||
}
|
||||
triggers = {
|
||||
fast-03-pf-dev = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = [
|
||||
"**/*json", "**/*tf", "**/*yaml", ".cloudbuild/workflow.yaml"
|
||||
]
|
||||
service_account = module.branch-pf-dev-sa-cicd.0.id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
depends_on = [module.branch-pf-dev-sa-cicd]
|
||||
}
|
||||
|
||||
moved {
|
||||
from = module.branch-teams-prod-pf-cicd-repo
|
||||
to = module.branch-pf-prod-cicd-repo
|
||||
}
|
||||
|
||||
module "branch-pf-prod-cicd-repo" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.project_factory_prod.type, null) == "sourcerepo"
|
||||
? { 0 = local.cicd_repositories.project_factory_prod }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = local.branch_optional_sa_lists.pf-prod
|
||||
"roles/source.reader" = [module.branch-pf-prod-sa-cicd.0.iam_email]
|
||||
}
|
||||
triggers = {
|
||||
fast-03-pf-prod = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = [
|
||||
"**/*json", "**/*tf", "**/*yaml", ".cloudbuild/workflow.yaml"
|
||||
]
|
||||
service_account = module.branch-pf-prod-sa-cicd.0.id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
depends_on = [module.branch-pf-prod-sa-cicd]
|
||||
}
|
||||
|
||||
# SAs used by CI/CD workflows to impersonate automation SAs
|
||||
|
||||
moved {
|
||||
from = module.branch-teams-dev-pf-sa-cicd
|
||||
to = module.branch-pf-dev-sa-cicd
|
||||
}
|
||||
|
||||
module "branch-pf-dev-sa-cicd" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.project_factory_dev.name, null) != null
|
||||
? { 0 = local.cicd_repositories.project_factory_dev }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = "dev-pf-resman-pf-1"
|
||||
display_name = "Terraform CI/CD project factory development service account."
|
||||
prefix = var.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {
|
||||
"roles/iam.serviceAccountUser" = local.automation_resman_sa_iam
|
||||
}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(var.automation.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
|
||||
moved {
|
||||
from = module.branch-teams-prod-pf-sa-cicd
|
||||
to = module.branch-pf-prod-sa-cicd
|
||||
}
|
||||
|
||||
module "branch-pf-prod-sa-cicd" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.project_factory_prod.name, null) != null
|
||||
? { 0 = local.cicd_repositories.project_factory_prod }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-pf-resman-pf-1"
|
||||
display_name = "Terraform CI/CD project factory production service account."
|
||||
prefix = var.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {
|
||||
"roles/iam.serviceAccountUser" = local.automation_resman_sa_iam
|
||||
}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
var.automation.federated_identity_pool,
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
var.automation.federated_identity_pool,
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(var.automation.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,94 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description CI/CD resources for the security branch.
|
||||
|
||||
# source repository
|
||||
|
||||
module "branch-security-cicd-repo" {
|
||||
source = "../../../modules/source-repository"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.security.type, null) == "sourcerepo"
|
||||
? { 0 = local.cicd_repositories.security }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = each.value.name
|
||||
iam = {
|
||||
"roles/source.admin" = [module.branch-security-sa.iam_email]
|
||||
"roles/source.reader" = [module.branch-security-sa-cicd.0.iam_email]
|
||||
}
|
||||
triggers = {
|
||||
fast-02-security = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = ["**/*tf", ".cloudbuild/workflow.yaml"]
|
||||
service_account = module.branch-security-sa-cicd.0.id
|
||||
substitutions = {}
|
||||
template = {
|
||||
project_id = null
|
||||
branch_name = each.value.branch
|
||||
repo_name = each.value.name
|
||||
tag_name = null
|
||||
}
|
||||
}
|
||||
}
|
||||
depends_on = [module.branch-security-sa-cicd]
|
||||
}
|
||||
|
||||
# SA used by CI/CD workflows to impersonate automation SAs
|
||||
|
||||
module "branch-security-sa-cicd" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = (
|
||||
try(local.cicd_repositories.security.name, null) != null
|
||||
? { 0 = local.cicd_repositories.security }
|
||||
: {}
|
||||
)
|
||||
project_id = var.automation.project_id
|
||||
name = "prod-resman-sec-1"
|
||||
display_name = "Terraform CI/CD stage 2 security service account."
|
||||
prefix = var.prefix
|
||||
iam = (
|
||||
each.value.type == "sourcerepo"
|
||||
# used directly from the cloud build trigger for source repos
|
||||
? {
|
||||
"roles/iam.serviceAccountUser" = local.automation_resman_sa_iam
|
||||
}
|
||||
# impersonated via workload identity federation for external repos
|
||||
: {
|
||||
"roles/iam.workloadIdentityUser" = [
|
||||
each.value.branch == null
|
||||
? format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principalset_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name
|
||||
)
|
||||
: format(
|
||||
local.cicd_identity_providers[each.value.identity_provider].principal_tpl,
|
||||
local.cicd_identity_pools[each.value.identity_provider],
|
||||
each.value.name,
|
||||
each.value.branch
|
||||
)
|
||||
]
|
||||
}
|
||||
)
|
||||
iam_project_roles = {
|
||||
(var.automation.project_id) = ["roles/logging.logWriter"]
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(var.automation.outputs_bucket) = ["roles/storage.objectViewer"]
|
||||
}
|
||||
}
|
||||
|
Before Width: | Height: | Size: 228 KiB After Width: | Height: | Size: 228 KiB |
|
Before Width: | Height: | Size: 435 KiB After Width: | Height: | Size: 435 KiB |
|
|
@ -0,0 +1,79 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
locals {
|
||||
automation_resman_sa_iam = [
|
||||
"serviceAccount:${var.automation.service_accounts.resman}"
|
||||
]
|
||||
automation_sas_iam = {
|
||||
for k, v in var.automation.service_accounts :
|
||||
k => v == null ? null : "serviceAccount:${v}"
|
||||
}
|
||||
branch_optional_sa_lists = {
|
||||
dp-dev = compact([local.automation_sas_iam.dp-dev])
|
||||
dp-prod = compact([local.automation_sas_iam.dp-prod])
|
||||
gke-dev = compact([local.automation_sas_iam.gke-dev])
|
||||
gke-prod = compact([local.automation_sas_iam.gke-prod])
|
||||
pf-dev = compact([local.automation_sas_iam.pf-dev])
|
||||
pf-prod = compact([local.automation_sas_iam.pf-prod])
|
||||
}
|
||||
# derive identity pool names from identity providers for easy reference
|
||||
cicd_identity_pools = {
|
||||
for k, v in local.cicd_identity_providers :
|
||||
k => split("/providers/", v.name)[0]
|
||||
}
|
||||
cicd_identity_providers = coalesce(
|
||||
try(var.automation.federated_identity_providers, null), {}
|
||||
)
|
||||
cicd_repositories = {
|
||||
for k, v in coalesce(var.cicd_repositories, {}) : k => v
|
||||
if(
|
||||
v != null &&
|
||||
(
|
||||
try(v.type, null) == "sourcerepo"
|
||||
||
|
||||
contains(
|
||||
keys(local.cicd_identity_providers),
|
||||
coalesce(try(v.identity_provider, null), ":")
|
||||
)
|
||||
) &&
|
||||
fileexists("${path.module}/templates/workflow-${try(v.type, "")}.yaml")
|
||||
)
|
||||
}
|
||||
cicd_workflow_var_files = {
|
||||
stage_2 = [
|
||||
"0-bootstrap-tenant.auto.tfvars.json",
|
||||
]
|
||||
stage_3 = [
|
||||
"0-bootstrap-tenant.auto.tfvars.json",
|
||||
"2-networking.auto.tfvars.json",
|
||||
"2-security.auto.tfvars.json"
|
||||
]
|
||||
}
|
||||
custom_roles = coalesce(var.custom_roles, {})
|
||||
gcs_storage_class = (
|
||||
length(split("-", var.locations.gcs)) < 2
|
||||
? "MULTI_REGIONAL"
|
||||
: "REGIONAL"
|
||||
)
|
||||
groups = {
|
||||
for k, v in var.groups :
|
||||
k => v == null ? null : "${v}@${var.organization.domain}"
|
||||
}
|
||||
groups_iam = {
|
||||
for k, v in local.groups : k => v != null ? "group:${v}" : null
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Output files persistence to local filesystem.
|
||||
|
||||
locals {
|
||||
outputs_root = join("/", [
|
||||
try(pathexpand(var.outputs_location), ""),
|
||||
"tenants",
|
||||
var.short_name
|
||||
])
|
||||
}
|
||||
|
||||
resource "local_file" "providers" {
|
||||
for_each = var.outputs_location == null ? {} : local.providers
|
||||
file_permission = "0644"
|
||||
filename = "${local.outputs_root}/providers/${each.key}-providers.tf"
|
||||
content = try(each.value, null)
|
||||
}
|
||||
|
||||
resource "local_file" "tfvars" {
|
||||
count = var.outputs_location == null ? 0 : 1
|
||||
file_permission = "0644"
|
||||
filename = "${local.outputs_root}/tfvars/1-resman.auto.tfvars.json"
|
||||
content = jsonencode(local.tfvars)
|
||||
}
|
||||
|
||||
resource "local_file" "workflows" {
|
||||
for_each = var.outputs_location == null ? {} : local.cicd_workflows
|
||||
file_permission = "0644"
|
||||
filename = "${local.outputs_root}/workflows/${replace(each.key, "_", "-")}-workflow.yaml"
|
||||
content = try(each.value, null)
|
||||
}
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Output files persistence to automation GCS bucket.
|
||||
|
||||
resource "google_storage_bucket_object" "providers" {
|
||||
for_each = local.providers
|
||||
bucket = var.automation.outputs_bucket
|
||||
name = "providers/${each.key}-providers.tf"
|
||||
content = each.value
|
||||
}
|
||||
|
||||
resource "google_storage_bucket_object" "tfvars" {
|
||||
bucket = var.automation.outputs_bucket
|
||||
name = "tfvars/1-resman.auto.tfvars.json"
|
||||
content = jsonencode(local.tfvars)
|
||||
}
|
||||
|
||||
resource "google_storage_bucket_object" "workflows" {
|
||||
for_each = local.cicd_workflows
|
||||
bucket = var.automation.outputs_bucket
|
||||
name = "workflows/${replace(each.key, "_", "-")}-workflow.yaml"
|
||||
content = each.value
|
||||
}
|
||||
|
|
@ -0,0 +1,311 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
locals {
|
||||
_tpl_providers = "${path.module}/templates/providers.tf.tpl"
|
||||
cicd_workflow_attrs = {
|
||||
data_platform_dev = {
|
||||
service_account = try(module.branch-dp-dev-sa-cicd.0.email, null)
|
||||
tf_providers_file = "3-data-platform-dev-providers.tf"
|
||||
tf_var_files = local.cicd_workflow_var_files.stage_3
|
||||
}
|
||||
data_platform_prod = {
|
||||
service_account = try(module.branch-dp-prod-sa-cicd.0.email, null)
|
||||
tf_providers_file = "3-data-platform-prod-providers.tf"
|
||||
tf_var_files = local.cicd_workflow_var_files.stage_3
|
||||
}
|
||||
gke_dev = {
|
||||
service_account = try(module.branch-gke-dev-sa-cicd.0.email, null)
|
||||
tf_providers_file = "3-gke-dev-providers.tf"
|
||||
tf_var_files = local.cicd_workflow_var_files.stage_3
|
||||
}
|
||||
gke_prod = {
|
||||
service_account = try(module.branch-gke-prod-sa-cicd.0.email, null)
|
||||
tf_providers_file = "3-gke-prod-providers.tf"
|
||||
tf_var_files = local.cicd_workflow_var_files.stage_3
|
||||
}
|
||||
networking = {
|
||||
service_account = try(module.branch-network-sa-cicd.0.email, null)
|
||||
tf_providers_file = "2-networking-providers.tf"
|
||||
tf_var_files = local.cicd_workflow_var_files.stage_2
|
||||
}
|
||||
project_factory_dev = {
|
||||
service_account = try(module.branch-pf-dev-sa-cicd.0.email, null)
|
||||
tf_providers_file = "3-project-factory-dev-providers.tf"
|
||||
tf_var_files = local.cicd_workflow_var_files.stage_3
|
||||
}
|
||||
project_factory_prod = {
|
||||
service_account = try(module.branch-pf-prod-sa-cicd.0.email, null)
|
||||
tf_providers_file = "3-project-factory-prod-providers.tf"
|
||||
tf_var_files = local.cicd_workflow_var_files.stage_3
|
||||
}
|
||||
security = {
|
||||
service_account = try(module.branch-security-sa-cicd.0.email, null)
|
||||
tf_providers_file = "2-security-providers.tf"
|
||||
tf_var_files = local.cicd_workflow_var_files.stage_2
|
||||
}
|
||||
}
|
||||
cicd_workflows = {
|
||||
for k, v in local.cicd_repositories : k => templatefile(
|
||||
"${path.module}/templates/workflow-${v.type}.yaml",
|
||||
merge(local.cicd_workflow_attrs[k], {
|
||||
identity_provider = try(
|
||||
local.cicd_identity_providers[v.identity_provider].name, null
|
||||
)
|
||||
outputs_bucket = var.automation.outputs_bucket
|
||||
stage_name = k
|
||||
})
|
||||
)
|
||||
}
|
||||
folder_ids = merge(
|
||||
{
|
||||
data-platform-dev = try(module.branch-dp-dev-folder.0.id, null)
|
||||
data-platform-prod = try(module.branch-dp-prod-folder.0.id, null)
|
||||
gke-dev = try(module.branch-gke-dev-folder.0.id, null)
|
||||
gke-prod = try(module.branch-gke-prod-folder.0.id, null)
|
||||
networking = module.branch-network-folder.id
|
||||
networking-dev = module.branch-network-dev-folder.id
|
||||
networking-prod = module.branch-network-prod-folder.id
|
||||
sandbox = try(module.branch-sandbox-folder.0.id, null)
|
||||
security = module.branch-security-folder.id
|
||||
teams = try(module.branch-teams-folder.0.id, null)
|
||||
},
|
||||
{
|
||||
for k, v in module.branch-teams-team-folder :
|
||||
"team-${k}" => v.id
|
||||
},
|
||||
{
|
||||
for k, v in module.branch-teams-team-dev-folder :
|
||||
"team-${k}-dev" => v.id
|
||||
},
|
||||
{
|
||||
for k, v in module.branch-teams-team-prod-folder :
|
||||
"team-${k}-prod" => v.id
|
||||
}
|
||||
)
|
||||
providers = merge(
|
||||
{
|
||||
"2-0-networking" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-network-gcs.name
|
||||
name = "networking"
|
||||
sa = module.branch-network-sa.email
|
||||
})
|
||||
"2-0-security" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-security-gcs.name
|
||||
name = "security"
|
||||
sa = module.branch-security-sa.email
|
||||
})
|
||||
},
|
||||
!var.fast_features.data_platform ? {} : {
|
||||
"3-0-data-platform-dev" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-dp-dev-gcs.0.name
|
||||
name = "dp-dev"
|
||||
sa = module.branch-dp-dev-sa.0.email
|
||||
})
|
||||
"3-0-data-platform-prod" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-dp-prod-gcs.0.name
|
||||
name = "dp-prod"
|
||||
sa = module.branch-dp-prod-sa.0.email
|
||||
})
|
||||
},
|
||||
!var.fast_features.gke ? {} : {
|
||||
"3-0-gke-dev" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-gke-dev-gcs.0.name
|
||||
name = "gke-dev"
|
||||
sa = module.branch-gke-dev-sa.0.email
|
||||
})
|
||||
"3-0-gke-prod" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-gke-prod-gcs.0.name
|
||||
name = "gke-prod"
|
||||
sa = module.branch-gke-prod-sa.0.email
|
||||
})
|
||||
},
|
||||
!var.fast_features.project_factory ? {} : {
|
||||
"3-0-project-factory-dev" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-pf-dev-gcs.0.name
|
||||
name = "team-dev"
|
||||
sa = var.automation.service_accounts.pf-dev
|
||||
})
|
||||
"3-0-project-factory-prod" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-pf-prod-gcs.0.name
|
||||
name = "team-prod"
|
||||
sa = var.automation.service_accounts.pf-prod
|
||||
})
|
||||
},
|
||||
!var.fast_features.sandbox ? {} : {
|
||||
"9-0-sandbox" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-sandbox-gcs.0.name
|
||||
name = "sandbox"
|
||||
sa = var.automation.service_accounts.sandbox
|
||||
})
|
||||
},
|
||||
!var.fast_features.teams ? {} : merge(
|
||||
{
|
||||
"3-teams" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-teams-gcs.0.name
|
||||
name = "teams"
|
||||
sa = module.branch-teams-sa.0.email
|
||||
})
|
||||
},
|
||||
{
|
||||
for k, v in module.branch-teams-team-sa :
|
||||
"3-teams-${k}" => templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.branch-teams-team-gcs[k].name
|
||||
name = "teams"
|
||||
sa = v.email
|
||||
})
|
||||
}
|
||||
)
|
||||
)
|
||||
tfvars = {
|
||||
folder_ids = local.folder_ids
|
||||
}
|
||||
}
|
||||
|
||||
output "cicd_repositories" {
|
||||
description = "WIF configuration for CI/CD repositories."
|
||||
value = {
|
||||
for k, v in local.cicd_repositories : k => {
|
||||
branch = v.branch
|
||||
name = v.name
|
||||
provider = try(
|
||||
local.cicd_identity_providers[v.identity_provider].name, null
|
||||
)
|
||||
service_account = local.cicd_workflow_attrs[k].service_account
|
||||
} if v != null
|
||||
}
|
||||
}
|
||||
|
||||
output "dataplatform" {
|
||||
description = "Data for the Data Platform stage."
|
||||
value = !var.fast_features.data_platform ? {} : {
|
||||
dev = {
|
||||
folder = module.branch-dp-dev-folder.0.id
|
||||
gcs_bucket = module.branch-dp-dev-gcs.0.name
|
||||
service_account = module.branch-dp-dev-sa.0.email
|
||||
}
|
||||
prod = {
|
||||
folder = module.branch-dp-prod-folder.0.id
|
||||
gcs_bucket = module.branch-dp-prod-gcs.0.name
|
||||
service_account = module.branch-dp-prod-sa.0.email
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output "gke_multitenant" {
|
||||
# tfdoc:output:consumers 03-gke-multitenant
|
||||
description = "Data for the GKE multitenant stage."
|
||||
value = (
|
||||
var.fast_features.gke
|
||||
? {
|
||||
"dev" = {
|
||||
folder = module.branch-gke-dev-folder.0.id
|
||||
gcs_bucket = module.branch-gke-dev-gcs.0.name
|
||||
service_account = module.branch-gke-dev-sa.0.email
|
||||
}
|
||||
"prod" = {
|
||||
folder = module.branch-gke-prod-folder.0.id
|
||||
gcs_bucket = module.branch-gke-prod-gcs.0.name
|
||||
service_account = module.branch-gke-prod-sa.0.email
|
||||
}
|
||||
}
|
||||
: {}
|
||||
)
|
||||
}
|
||||
|
||||
output "networking" {
|
||||
description = "Data for the networking stage."
|
||||
value = {
|
||||
folder = module.branch-network-folder.id
|
||||
gcs_bucket = module.branch-network-gcs.name
|
||||
service_account = module.branch-network-sa.iam_email
|
||||
}
|
||||
}
|
||||
|
||||
output "project_factories" {
|
||||
description = "Data for the project factories stage."
|
||||
value = !var.fast_features.project_factory ? {} : {
|
||||
dev = {
|
||||
bucket = module.branch-pf-dev-gcs.0.name
|
||||
sa = var.automation.service_accounts.pf-dev
|
||||
}
|
||||
prod = {
|
||||
bucket = module.branch-pf-prod-gcs.0.name
|
||||
sa = var.automation.service_accounts.pf-prod
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ready to use provider configurations for subsequent stages
|
||||
output "providers" {
|
||||
# tfdoc:output:consumers 02-networking 02-security 03-dataplatform xx-sandbox xx-teams
|
||||
description = "Terraform provider files for this stage and dependent stages."
|
||||
sensitive = true
|
||||
value = local.providers
|
||||
}
|
||||
|
||||
output "sandbox" {
|
||||
# tfdoc:output:consumers xx-sandbox
|
||||
description = "Data for the sandbox stage."
|
||||
value = (
|
||||
var.fast_features.sandbox
|
||||
? {
|
||||
folder = module.branch-sandbox-folder.0.id
|
||||
gcs_bucket = module.branch-sandbox-gcs.0.name
|
||||
service_account = var.automation.service_accounts.sandbox
|
||||
}
|
||||
: null
|
||||
)
|
||||
}
|
||||
|
||||
output "security" {
|
||||
# tfdoc:output:consumers 02-security
|
||||
description = "Data for the networking stage."
|
||||
value = {
|
||||
folder = module.branch-security-folder.id
|
||||
gcs_bucket = module.branch-security-gcs.name
|
||||
service_account = module.branch-security-sa.iam_email
|
||||
}
|
||||
}
|
||||
|
||||
output "teams" {
|
||||
description = "Data for the teams stage."
|
||||
value = {
|
||||
for k, v in module.branch-teams-team-folder : k => {
|
||||
folder = v.id
|
||||
gcs_bucket = module.branch-teams-team-gcs[k].name
|
||||
service_account = module.branch-teams-team-sa[k].email
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ready to use variable values for subsequent stages
|
||||
output "tfvars" {
|
||||
description = "Terraform variable files for the following stages."
|
||||
sensitive = true
|
||||
value = local.tfvars
|
||||
}
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Tenant root folder configuration.
|
||||
|
||||
module "root-folder" {
|
||||
source = "../../../modules/folder"
|
||||
id = var.root_node
|
||||
folder_create = var.test_skip_data_sources
|
||||
# start test attributes
|
||||
parent = (
|
||||
var.test_skip_data_sources ? "organizations/${var.organization.id}" : null
|
||||
)
|
||||
name = var.test_skip_data_sources ? "Test" : null
|
||||
# end test attributes
|
||||
iam_additive = {
|
||||
"roles/accesscontextmanager.policyAdmin" = [
|
||||
local.automation_sas_iam.security
|
||||
]
|
||||
"roles/compute.orgFirewallPolicyAdmin" = [
|
||||
local.automation_sas_iam.networking
|
||||
]
|
||||
"roles/compute.xpnAdmin" = [
|
||||
local.automation_sas_iam.networking
|
||||
]
|
||||
}
|
||||
org_policies_data_path = var.organization_policy_data_path
|
||||
}
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
terraform {
|
||||
backend "gcs" {
|
||||
bucket = "${bucket}"
|
||||
impersonate_service_account = "${sa}"
|
||||
%{~ if backend_extra != null ~}
|
||||
${indent(4, backend_extra)}
|
||||
%{~ endif ~}
|
||||
}
|
||||
}
|
||||
provider "google" {
|
||||
impersonate_service_account = "${sa}"
|
||||
}
|
||||
provider "google-beta" {
|
||||
impersonate_service_account = "${sa}"
|
||||
}
|
||||
|
||||
# end provider.tf for ${name}
|
||||
|
|
@ -0,0 +1,186 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
name: "FAST ${stage_name} stage"
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
types:
|
||||
- closed
|
||||
- opened
|
||||
- synchronize
|
||||
|
||||
env:
|
||||
FAST_OUTPUTS_BUCKET: ${outputs_bucket}
|
||||
FAST_SERVICE_ACCOUNT: ${service_account}
|
||||
FAST_WIF_PROVIDER: ${identity_provider}
|
||||
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
|
||||
TF_PROVIDERS_FILE: ${tf_providers_file}
|
||||
TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
|
||||
TF_VERSION: 1.3.2
|
||||
|
||||
jobs:
|
||||
fast-pr:
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
issues: write
|
||||
pull-requests: write
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- id: checkout
|
||||
name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
|
||||
# set up SSH key authentication to the modules repository
|
||||
- id: ssh-config
|
||||
name: Configure SSH authentication
|
||||
run: |
|
||||
ssh-agent -a "$SSH_AUTH_SOCK" > /dev/null
|
||||
ssh-add - <<< "$${{ secrets.CICD_MODULES_KEY }}"
|
||||
|
||||
# set up authentication via Workload identity Federation
|
||||
- id: gcp-auth
|
||||
name: Authenticate to Google Cloud
|
||||
uses: google-github-actions/auth@v0
|
||||
with:
|
||||
workload_identity_provider: $${{ env.FAST_WIF_PROVIDER }}
|
||||
service_account: $${{ env.FAST_SERVICE_ACCOUNT }}
|
||||
access_token_lifetime: 3600s
|
||||
|
||||
- id: gcp-sdk
|
||||
name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v0
|
||||
with:
|
||||
install_components: alpha
|
||||
|
||||
# copy provider and tfvars files
|
||||
- id: tf-config
|
||||
name: Copy Terraform output files
|
||||
run: |
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${{env.FAST_OUTPUTS_BUCKET}}/providers/$${{env.TF_PROVIDERS_FILE}}" ./
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${{env.FAST_OUTPUTS_BUCKET}}/tfvars" ./
|
||||
for f in $${{env.TF_VAR_FILES}}; do
|
||||
ln -s "tfvars/$f" ./
|
||||
done
|
||||
|
||||
- id: tf-setup
|
||||
name: Set up Terraform
|
||||
uses: hashicorp/setup-terraform@v2.0.3
|
||||
with:
|
||||
terraform_version: $${{ env.TF_VERSION }}
|
||||
|
||||
# run Terraform init/validate/plan
|
||||
- id: tf-init
|
||||
name: Terraform init
|
||||
run: |
|
||||
terraform init -no-color
|
||||
|
||||
- id: tf-validate
|
||||
name: Terraform validate
|
||||
run: terraform validate -no-color
|
||||
|
||||
- id: tf-plan
|
||||
name: Terraform plan
|
||||
continue-on-error: true
|
||||
run: |
|
||||
terraform plan -input=false -out ../plan.out -no-color
|
||||
|
||||
- id: tf-apply
|
||||
if: github.event.pull_request.merged == true && success()
|
||||
name: Terraform apply
|
||||
continue-on-error: true
|
||||
run: |
|
||||
terraform apply -input=false -auto-approve -no-color ../plan.out
|
||||
|
||||
- id: pr-comment
|
||||
name: Post comment to Pull Request
|
||||
continue-on-error: true
|
||||
uses: actions/github-script@v6
|
||||
if: github.event_name == 'pull_request'
|
||||
env:
|
||||
PLAN: $${{ steps.tf-plan.outputs.stdout }}\n$${{ steps.tf-plan.outputs.stderr }}
|
||||
with:
|
||||
script: |
|
||||
const output = `### Terraform Initialization \`$${{ steps.tf-init.outcome }}\`
|
||||
|
||||
### Terraform Validation \`$${{ steps.tf-validate.outcome }}\`
|
||||
|
||||
<details><summary>Validation Output</summary>
|
||||
|
||||
\`\`\`\n
|
||||
$${{ steps.tf-validate.outputs.stdout }}
|
||||
\`\`\`
|
||||
|
||||
</details>
|
||||
|
||||
### Terraform Plan \`$${{ steps.tf-plan.outcome }}\`
|
||||
|
||||
<details><summary>Show Plan</summary>
|
||||
|
||||
\`\`\`\n
|
||||
$${process.env.PLAN.split('\n').filter(l => l.match(/^([A-Z\s].*|)$$/)).join('\n')}
|
||||
\`\`\`
|
||||
|
||||
</details>
|
||||
|
||||
### Terraform Apply \`$${{ steps.tf-apply.outcome }}\`
|
||||
|
||||
*Pusher: @$${{ github.actor }}, Action: \`$${{ github.event_name }}\`, Working Directory: \`$${{ env.tf_actions_working_dir }}\`, Workflow: \`$${{ github.workflow }}\`*`;
|
||||
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: output
|
||||
})
|
||||
|
||||
- id: pr-short-comment
|
||||
name: Post comment to Pull Request
|
||||
uses: actions/github-script@v6
|
||||
if: github.event_name == 'pull_request' && steps.pr-comment.outcome != 'success'
|
||||
with:
|
||||
script: |
|
||||
const output = `### Terraform Initialization \`$${{ steps.tf-init.outcome }}\`
|
||||
|
||||
### Terraform Validation \`$${{ steps.tf-validate.outcome }}\`
|
||||
|
||||
### Terraform Plan \`$${{ steps.tf-plan.outcome }}\`
|
||||
|
||||
Plan output is in the action log.
|
||||
|
||||
### Terraform Apply \`$${{ steps.tf-apply.outcome }}\`
|
||||
|
||||
*Pusher: @$${{ github.actor }}, Action: \`$${{ github.event_name }}\`, Working Directory: \`$${{ env.tf_actions_working_dir }}\`, Workflow: \`$${{ github.workflow }}\`*`;
|
||||
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: output
|
||||
})
|
||||
|
||||
- id: check-plan
|
||||
name: Check plan failure
|
||||
if: steps.tf-plan.outcome != 'success'
|
||||
run: exit 1
|
||||
|
||||
- id: check-apply
|
||||
name: Check apply failure
|
||||
if: github.event.pull_request.merged == true && steps.tf-apply.outcome != 'success'
|
||||
run: exit 1
|
||||
|
|
@ -0,0 +1,120 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
default:
|
||||
before_script:
|
||||
- echo "$${CI_JOB_JWT_V2}" > token.txt
|
||||
image:
|
||||
name: hashicorp/terraform
|
||||
entrypoint:
|
||||
- "/usr/bin/env"
|
||||
- "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
|
||||
variables:
|
||||
GOOGLE_CREDENTIALS: cicd-sa-credentials.json
|
||||
FAST_OUTPUTS_BUCKET: ${outputs_bucket}
|
||||
FAST_SERVICE_ACCOUNT: ${service_account}
|
||||
FAST_WIF_PROVIDER: ${identity_provider}
|
||||
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
|
||||
TF_PROVIDERS_FILE: ${tf_providers_file}
|
||||
TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
|
||||
|
||||
stages:
|
||||
- gcp-auth
|
||||
- tf-files
|
||||
- tf-plan
|
||||
- tf-apply
|
||||
|
||||
cache:
|
||||
key: gcp-auth
|
||||
paths:
|
||||
- cicd-sa-credentials.json
|
||||
- .tf-setup
|
||||
|
||||
gcp-auth:
|
||||
image:
|
||||
name: google/cloud-sdk:slim
|
||||
stage: gcp-auth
|
||||
script:
|
||||
- |
|
||||
gcloud iam workload-identity-pools create-cred-config \
|
||||
$${FAST_WIF_PROVIDER} \
|
||||
--service-account=$${FAST_SERVICE_ACCOUNT} \
|
||||
--service-account-token-lifetime-seconds=3600 \
|
||||
--output-file=$${GOOGLE_CREDENTIALS} \
|
||||
--credential-source-file=token.txt
|
||||
tf-files:
|
||||
dependencies:
|
||||
- gcp-auth
|
||||
image:
|
||||
name: google/cloud-sdk:slim
|
||||
stage: tf-files
|
||||
script:
|
||||
# - gcloud components install -q alpha
|
||||
- gcloud config set auth/credential_file_override $${GOOGLE_CREDENTIALS}
|
||||
- mkdir -p .tf-setup
|
||||
- |
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${FAST_OUTPUTS_BUCKET}/providers/$${TF_PROVIDERS_FILE}" .tf-setup/
|
||||
- |
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${FAST_OUTPUTS_BUCKET}/tfvars" .tf-setup/
|
||||
|
||||
tf-plan:
|
||||
# uncomment the following lines and set the SSH key secret for private modules repo
|
||||
# before_script:
|
||||
# - |
|
||||
# ssh-agent -a $SSH_AUTH_SOCK > /dev/null
|
||||
# echo "$CICD_MODULES_KEY" | base64 -d | tr -d '\r' | ssh-add - > /dev/null
|
||||
# mkdir -p ~/.ssh
|
||||
# ssh-keyscan -H 'gitlab.com' >> ~/.ssh/known_hosts
|
||||
# ssh-keyscan gitlab.com | sort -u - ~/.ssh/known_hosts -o ~/.ssh/known_hosts
|
||||
stage: tf-plan
|
||||
script:
|
||||
- cp .tf-setup/$${TF_PROVIDERS_FILE} ./
|
||||
- |
|
||||
for f in $${TF_VAR_FILES}; do
|
||||
ln -s ".tf-setup/tfvars/$f" ./
|
||||
done
|
||||
- terraform init
|
||||
- terraform validate
|
||||
- terraform plan
|
||||
dependencies:
|
||||
- tf-files
|
||||
|
||||
tf-apply:
|
||||
# uncomment the following lines and set the SSH key secret for private modules repo
|
||||
# before_script:
|
||||
# - |
|
||||
# ssh-agent -a $SSH_AUTH_SOCK > /dev/null
|
||||
# echo "$CICD_MODULES_KEY" | base64 -d | tr -d '\r' | ssh-add - > /dev/null
|
||||
# mkdir -p ~/.ssh
|
||||
# ssh-keyscan -H 'gitlab.com' >> ~/.ssh/known_hosts
|
||||
# ssh-keyscan gitlab.com | sort -u - ~/.ssh/known_hosts -o ~/.ssh/known_hosts
|
||||
stage: tf-apply
|
||||
script:
|
||||
- cp .tf-setup/$${TF_PROVIDERS_FILE} ./
|
||||
- |
|
||||
for f in $${TF_VAR_FILES}; do
|
||||
ln -s ".tf-setup/tfvars/$f" ./
|
||||
done
|
||||
- terraform init
|
||||
- terraform validate
|
||||
- terraform apply -input=false -auto-approve
|
||||
dependencies:
|
||||
- tf-files
|
||||
when: manual
|
||||
only:
|
||||
variables:
|
||||
- $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
|
|
@ -0,0 +1,98 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
steps:
|
||||
- name: alpine:3
|
||||
id: tf-download
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
mkdir -p /builder/home/.local/bin
|
||||
wget https://releases.hashicorp.com/terraform/$${_TF_VERSION}/terraform_$${_TF_VERSION}_linux_amd64.zip
|
||||
unzip terraform_$${_TF_VERSION}_linux_amd64.zip -d /builder/home/.local/bin
|
||||
rm terraform_$${_TF_VERSION}_linux_amd64.zip
|
||||
chmod 755 /builder/home/.local/bin/terraform
|
||||
- name: alpine:3
|
||||
id: tf-check-format
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform fmt -recursive -check /workspace/
|
||||
- name: gcr.io/google.com/cloudsdktool/cloud-sdk:alpine
|
||||
id: tf-files
|
||||
entrypoint: bash
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
/google-cloud-sdk/bin/gsutil cp \
|
||||
gs://$${_FAST_OUTPUTS_BUCKET}/providers/$${_TF_PROVIDERS_FILE} ./
|
||||
/google-cloud-sdk/bin/gsutil cp -r \
|
||||
gs://$${_FAST_OUTPUTS_BUCKET}/tfvars ./
|
||||
for f in $${_TF_VAR_FILES}; do
|
||||
ln -s tfvars/$f ./
|
||||
done
|
||||
- name: alpine:3
|
||||
id: tf-init
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform init -no-color
|
||||
- name: alpine:3
|
||||
id: tf-check-validate
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform validate -no-color
|
||||
- name: alpine:3
|
||||
id: tf-plan
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform plan -no-color -input=false -out plan.out
|
||||
# store artifact and ask for approval here if needed
|
||||
- name: alpine:3
|
||||
id: tf-apply
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform apply -no-color -input=false -auto-approve plan.out
|
||||
options:
|
||||
env:
|
||||
- PATH=/usr/local/bin:/usr/bin:/bin:/builder/home/.local/bin
|
||||
logging: CLOUD_LOGGING_ONLY
|
||||
substitutions:
|
||||
_FAST_OUTPUTS_BUCKET: ${outputs_bucket}
|
||||
_TF_PROVIDERS_FILE: ${tf_providers_file}
|
||||
_TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
|
||||
_TF_VERSION: 1.3.2
|
||||
|
|
@ -0,0 +1,281 @@
|
|||
/**
|
||||
* Copyright 2023 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# defaults for variables marked with global tfdoc annotations, can be set via
|
||||
# the tfvars file generated in stage 00 and stored in its outputs
|
||||
|
||||
variable "automation" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Automation resources created by the bootstrap stage."
|
||||
type = object({
|
||||
outputs_bucket = string
|
||||
project_id = string
|
||||
project_number = string
|
||||
federated_identity_pools = list(string)
|
||||
federated_identity_providers = map(object({
|
||||
issuer = string
|
||||
issuer_uri = string
|
||||
name = string
|
||||
principal_tpl = string
|
||||
principalset_tpl = string
|
||||
}))
|
||||
service_accounts = object({
|
||||
networking = string
|
||||
resman = string
|
||||
security = string
|
||||
dp-dev = optional(string)
|
||||
dp-prod = optional(string)
|
||||
gke-dev = optional(string)
|
||||
gke-prod = optional(string)
|
||||
pf-dev = optional(string)
|
||||
pf-prod = optional(string)
|
||||
sandbox = optional(string)
|
||||
teams = optional(string)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
variable "billing_account" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Billing account id. If billing account is not part of the same org set `is_org_level` to false."
|
||||
type = object({
|
||||
id = string
|
||||
is_org_level = optional(bool, true)
|
||||
})
|
||||
validation {
|
||||
condition = var.billing_account.is_org_level != null
|
||||
error_message = "Invalid `null` value for `billing_account.is_org_level`."
|
||||
}
|
||||
}
|
||||
|
||||
variable "cicd_repositories" {
|
||||
description = "CI/CD repository configuration. Identity providers reference keys in the `automation.federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed."
|
||||
type = object({
|
||||
data_platform_dev = object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
data_platform_prod = object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
gke_dev = object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
gke_prod = object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
networking = object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
project_factory_dev = object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
project_factory_prod = object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
security = object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
})
|
||||
default = null
|
||||
validation {
|
||||
condition = alltrue([
|
||||
for k, v in coalesce(var.cicd_repositories, {}) :
|
||||
v == null || try(v.name, null) != null
|
||||
])
|
||||
error_message = "Non-null repositories need a non-null name."
|
||||
}
|
||||
validation {
|
||||
condition = alltrue([
|
||||
for k, v in coalesce(var.cicd_repositories, {}) :
|
||||
v == null || (
|
||||
try(v.identity_provider, null) != null
|
||||
||
|
||||
try(v.type, null) == "sourcerepo"
|
||||
)
|
||||
])
|
||||
error_message = "Non-null repositories need a non-null provider unless type is 'sourcerepo'."
|
||||
}
|
||||
validation {
|
||||
condition = alltrue([
|
||||
for k, v in coalesce(var.cicd_repositories, {}) :
|
||||
v == null || (
|
||||
contains(["github", "gitlab", "sourcerepo"], coalesce(try(v.type, null), "null"))
|
||||
)
|
||||
])
|
||||
error_message = "Invalid repository type, supported types: 'github' 'gitlab' or 'sourcerepo'."
|
||||
}
|
||||
}
|
||||
|
||||
variable "custom_roles" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Custom roles defined at the org level, in key => id format."
|
||||
type = object({
|
||||
service_project_network_admin = string
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "data_dir" {
|
||||
description = "Relative path for the folder storing configuration data."
|
||||
type = string
|
||||
default = "data"
|
||||
}
|
||||
|
||||
variable "fast_features" {
|
||||
# tfdoc:variable:source 0-0-bootstrap
|
||||
description = "Selective control for top-level FAST features."
|
||||
type = object({
|
||||
data_platform = optional(bool, false)
|
||||
gke = optional(bool, false)
|
||||
project_factory = optional(bool, false)
|
||||
sandbox = optional(bool, false)
|
||||
teams = optional(bool, false)
|
||||
})
|
||||
default = {}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "groups" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
# https://cloud.google.com/docs/enterprise/setup-checklist
|
||||
description = "Group names to grant organization-level permissions."
|
||||
type = object({
|
||||
gcp-devops = optional(string)
|
||||
gcp-network-admins = optional(string)
|
||||
gcp-security-admins = optional(string)
|
||||
})
|
||||
default = {}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "locations" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Optional locations for GCS, BigQuery, and logging buckets created here."
|
||||
type = object({
|
||||
bq = string
|
||||
gcs = string
|
||||
logging = string
|
||||
pubsub = list(string)
|
||||
})
|
||||
default = {
|
||||
bq = "EU"
|
||||
gcs = "EU"
|
||||
logging = "global"
|
||||
pubsub = []
|
||||
}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "organization" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Organization details."
|
||||
type = object({
|
||||
domain = string
|
||||
id = number
|
||||
customer_id = string
|
||||
})
|
||||
}
|
||||
|
||||
variable "organization_policy_data_path" {
|
||||
description = "Path for the data folder used by the organization policies factory."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "outputs_location" {
|
||||
description = "Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "prefix" {
|
||||
# tfdoc:variable:source 0-bootstrap
|
||||
description = "Prefix used for resources that need unique names. Use 9 characters or less."
|
||||
type = string
|
||||
|
||||
validation {
|
||||
condition = try(length(var.prefix), 0) < 10
|
||||
error_message = "Use a maximum of 9 characters for prefix."
|
||||
}
|
||||
}
|
||||
|
||||
variable "root_node" {
|
||||
description = "Root folder node for the tenant, in folders/nnnnnn format."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "short_name" {
|
||||
description = "Short name used to identify the tenant."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Resource management tags."
|
||||
type = object({
|
||||
keys = object({
|
||||
context = string
|
||||
environment = string
|
||||
tenant = string
|
||||
})
|
||||
names = object({
|
||||
context = string
|
||||
environment = string
|
||||
tenant = string
|
||||
})
|
||||
values = map(string)
|
||||
})
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "team_folders" {
|
||||
description = "Team folders to be created. Format is described in a code comment."
|
||||
type = map(object({
|
||||
descriptive_name = string
|
||||
group_iam = map(list(string))
|
||||
impersonation_groups = list(string)
|
||||
}))
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "test_skip_data_sources" {
|
||||
description = "Used when testing to bypass data sources."
|
||||
type = bool
|
||||
default = false
|
||||
}
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
# Things to add before sending the PR
|
||||
|
||||
- [x] multitenant resman stage
|
||||
- [ ] identity providers for tenant bootstrap stages
|
||||
- [ ] edit FAST top-level and stages READMEs for multitenant
|
||||
- [ ] multitenant stages READMEs
|
||||
- [ ] add test to ensure templates folder is the same in every stage
|
||||
- [ ] add test to ensure identity providers code is the same in bootstrap and mt bootstrap
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
# FAST multitenant stages
|
||||
|
||||
The stages in this folder set up separate resource hierarchies inside the same organization that are fully FAST-compliant, and allow each tenant to run and manage their own networking, security, or application-level stages. They are designed to be used where a high degree of autonomy is needed for each tenant, for example individual subsidiaries of a large corporation all sharing the same GCP organization.
|
||||
|
||||
The multitenant stages have the following characteristics:
|
||||
|
||||
- they support one tenant at a time, so one copy of both stages is needed for each tenant
|
||||
- they have the organization-level bootstrap and resource management stages as prerequisite
|
||||
- they are logically equivalent to the respective organization-level stages but behave slightly differently, as they actively minimize access and changes to organization or shared resources
|
||||
|
||||
Once both tenant-level stages are run, a hierarchy and a set of resources is available for the new tenant, including a separate automation project, service accounts for subsequent stages, etc.
|
||||
|
||||
The tenant-level stages require that organization-level stage 0 (bootstrap) and 1 (resource management) have been applied. Their position and role in the FAST stage flow is shown in the following diagram:
|
||||
|
||||
<p align="center">
|
||||
<img src="stages.svg" alt="Stages diagram">
|
||||
</p>
|
||||
|
||||
## Tenant bootstrap (0)
|
||||
|
||||
This stage creates the top-level root folder and tag for the tenant, and the tenant-level automation project and automation service accounts. It also sets up billing and organization-level roles for the tenant administrators group and the automation service accounts. As in the organizational-level stages, it can optionally set up CI/CD for itself and the tenant resource management stage.
|
||||
|
||||
This stage is run with the organization-level resource management service account as it leverages its permissions, and is the bridge between the organization-level stages and the tenant stages which are effectively decoupled from the rest of the organization.
|
||||
|
||||
## Tenant resource management (1)
|
||||
|
||||
This stage populates the resource hierarchy rooted in the top-level tenant folder, assigns roles to the tenant automation service accounts, and optionally sets up CI/CD for the following stages. It is functionally equivalent to the organization-level resource management stage, but runs with a tenant-specific service account and has no control over resources outside of the tenant context.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 196 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 215 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 305 KiB |
BIN
fast/stages.png
BIN
fast/stages.png
Binary file not shown.
|
Before Width: | Height: | Size: 253 KiB After Width: | Height: | Size: 142 KiB |
1948
fast/stages.svg
1948
fast/stages.svg
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 187 KiB After Width: | Height: | Size: 182 KiB |
|
|
@ -18,7 +18,7 @@ Use the following diagram as a simple high level reference for the following sec
|
|||
|
||||
As mentioned above, this stage only does the bare minimum required to bootstrap automation, and ensure that base audit and billing exports are in place from the start to provide some measure of accountability, even before the security configurations are applied in a later stage.
|
||||
|
||||
It also sets up organization-level IAM bindings so the Organization Administrator role is only used here, trading off some design freedom for ease of auditing and troubleshooting, and reducing the risk of costly security mistakes down the line. The only exception to this rule is for the [Resource Management stage](../01-resman) service account, described below.
|
||||
It also sets up organization-level IAM bindings so the Organization Administrator role is only used here, trading off some design freedom for ease of auditing and troubleshooting, and reducing the risk of costly security mistakes down the line. The only exception to this rule is for the [Resource Management stage](../1-resman) service account, described below.
|
||||
|
||||
### User groups
|
||||
|
||||
|
|
@ -28,7 +28,7 @@ We have standardized the initial set of groups on those outlined in the [GCP Ent
|
|||
|
||||
### Organization-level IAM
|
||||
|
||||
The service account used in the [Resource Management stage](../01-resman) needs to be able to grant specific permissions at the organizational level, to enable specific functionality for subsequent stages that deal with network or security resources, or billing-related activities.
|
||||
The service account used in the [Resource Management stage](../1-resman) needs to be able to grant specific permissions at the organizational level, to enable specific functionality for subsequent stages that deal with network or security resources, or billing-related activities.
|
||||
|
||||
In order to be able to assign those roles without having the full authority of the Organization Admin role, this stage defines a custom role that only allows setting IAM policies on the organization, and grants it via a [delegated role grant](https://cloud.google.com/iam/docs/setting-limits-on-granting-roles) that only allows it to be used to grant a limited subset of roles.
|
||||
|
||||
|
|
@ -89,7 +89,7 @@ This stage also implements initial support for two interrelated features
|
|||
|
||||
Workload Identity Federation support allows configuring external providers independently from CI/CD, and offers predefined attributes for a few well known ones (more can be easily added by editing the `identity-providers.tf` file). Once providers have been configured their names are passed to the following stages via interface outputs, and can be leveraged to set up access or impersonation in IAM bindings.
|
||||
|
||||
CI/CD support is fully implemented for GitHub, Gitlab, and Cloud Source Repositories / Cloud Build. For GitHub, we also offer a [separate supporting setup](../../extras/00-cicd-github/) to quickly create / configure repositories.
|
||||
CI/CD support is fully implemented for GitHub, Gitlab, and Cloud Source Repositories / Cloud Build. For GitHub, we also offer a [separate supporting setup](../../extras/0-cicd-github/) to quickly create / configure repositories.
|
||||
|
||||
<!-- TODO: add a general overview of our design -->
|
||||
|
||||
|
|
@ -419,7 +419,7 @@ The `type` attribute can be set to one of the supported repository types: `githu
|
|||
|
||||
Once the stage is applied the generated output files will contain pre-configured workflow files for each repository, that will use Workload Identity Federation via a dedicated service account for each repository to impersonate the automation service account for the stage.
|
||||
|
||||
You can use Terraform to automate creation of the repositories using the extra stage defined in [fast/extras/00-cicd-github](../../extras/00-cicd-github/) (only for Github for now).
|
||||
You can use Terraform to automate creation of the repositories using the extra stage defined in [fast/extras/0-cicd-github](../../extras/0-cicd-github/) (only for Github for now).
|
||||
|
||||
The remaining configuration is manual, as it regards the repositories themselves:
|
||||
|
||||
|
|
@ -453,7 +453,7 @@ The remaining configuration is manual, as it regards the repositories themselves
|
|||
| name | description | modules | resources |
|
||||
|---|---|---|---|
|
||||
| [automation.tf](./automation.tf) | Automation project and resources. | <code>gcs</code> · <code>iam-service-account</code> · <code>project</code> | |
|
||||
| [billing.tf](./billing.tf) | Billing export project and dataset. | <code>bigquery-dataset</code> · <code>organization</code> · <code>project</code> | <code>google_billing_account_iam_member</code> · <code>google_organization_iam_binding</code> |
|
||||
| [billing.tf](./billing.tf) | Billing export project and dataset. | <code>bigquery-dataset</code> · <code>project</code> | <code>google_billing_account_iam_member</code> |
|
||||
| [cicd.tf](./cicd.tf) | Workload Identity Federation configurations for CI/CD. | <code>iam-service-account</code> · <code>source-repository</code> | |
|
||||
| [identity-providers.tf](./identity-providers.tf) | Workload Identity Federation provider definitions. | | <code>google_iam_workload_identity_pool</code> · <code>google_iam_workload_identity_pool_provider</code> |
|
||||
| [log-export.tf](./log-export.tf) | Audit log project and sink. | <code>bigquery-dataset</code> · <code>gcs</code> · <code>logging-bucket</code> · <code>project</code> · <code>pubsub</code> | |
|
||||
|
|
@ -468,35 +468,35 @@ The remaining configuration is manual, as it regards the repositories themselves
|
|||
|
||||
| name | description | type | required | default | producer |
|
||||
|---|---|:---:|:---:|:---:|:---:|
|
||||
| [billing_account](variables.tf#L17) | Billing account id and organization id ('nnnnnnnn' or null). | <code title="object({ id = string organization_id = number })">object({…})</code> | ✓ | | |
|
||||
| [organization](variables.tf#L202) | Organization details. | <code title="object({ domain = string id = number customer_id = string })">object({…})</code> | ✓ | | |
|
||||
| [prefix](variables.tf#L217) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | |
|
||||
| [bootstrap_user](variables.tf#L25) | Email of the nominal user running this stage for the first time. | <code>string</code> | | <code>null</code> | |
|
||||
| [cicd_repositories](variables.tf#L31) | CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | <code title="object({ bootstrap = object({ branch = string identity_provider = string name = string type = string }) cicd = object({ branch = string identity_provider = string name = string type = string }) resman = object({ branch = string identity_provider = string name = string type = string }) })">object({…})</code> | | <code>null</code> | |
|
||||
| [custom_role_names](variables.tf#L83) | Names of custom roles defined at the org level. | <code title="object({ organization_iam_admin = string service_project_network_admin = string })">object({…})</code> | | <code title="{ organization_iam_admin = "organizationIamAdmin" service_project_network_admin = "serviceProjectNetworkAdmin" }">{…}</code> | |
|
||||
| [fast_features](variables.tf#L95) | Selective control for top-level FAST features. | <code title="object({ data_platform = bool gke = bool project_factory = bool sandbox = bool teams = bool })">object({…})</code> | | <code title="{ data_platform = true gke = true project_factory = true sandbox = true teams = true }">{…}</code> | |
|
||||
| [federated_identity_providers](variables.tf#L114) | Workload Identity Federation pools. The `cicd_repositories` variable references keys here. | <code title="map(object({ attribute_condition = string issuer = string custom_settings = object({ issuer_uri = string allowed_audiences = list(string) }) }))">map(object({…}))</code> | | <code>{}</code> | |
|
||||
| [groups](variables.tf#L128) | Group names to grant organization-level permissions. | <code>map(string)</code> | | <code title="{ gcp-billing-admins = "gcp-billing-admins", gcp-devops = "gcp-devops", gcp-network-admins = "gcp-network-admins" gcp-organization-admins = "gcp-organization-admins" gcp-security-admins = "gcp-security-admins" gcp-support = "gcp-devops" }">{…}</code> | |
|
||||
| [iam](variables.tf#L146) | Organization-level custom IAM settings in role => [principal] format. | <code>map(list(string))</code> | | <code>{}</code> | |
|
||||
| [iam_additive](variables.tf#L152) | Organization-level custom IAM settings in role => [principal] format for non-authoritative bindings. | <code>map(list(string))</code> | | <code>{}</code> | |
|
||||
| [locations](variables.tf#L158) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object({ bq = string gcs = string logging = string pubsub = list(string) })">object({…})</code> | | <code title="{ bq = "EU" gcs = "EU" logging = "global" pubsub = [] }">{…}</code> | |
|
||||
| [log_sinks](variables.tf#L177) | Org-level log sinks, in name => {type, filter} format. | <code title="map(object({ filter = string type = string }))">map(object({…}))</code> | | <code title="{ audit-logs = { filter = "logName:\"/logs/cloudaudit.googleapis.com%2Factivity\" OR logName:\"/logs/cloudaudit.googleapis.com%2Fsystem_event\"" type = "bigquery" } vpc-sc = { filter = "protoPayload.metadata.@type=\"type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata\"" type = "bigquery" } }">{…}</code> | |
|
||||
| [outputs_location](variables.tf#L211) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | <code>string</code> | | <code>null</code> | |
|
||||
| [project_parent_ids](variables.tf#L227) | Optional parents for projects created here in folders/nnnnnnn format. Null values will use the organization as parent. | <code title="object({ automation = string billing = string logging = string })">object({…})</code> | | <code title="{ automation = null billing = null logging = null }">{…}</code> | |
|
||||
| [billing_account](variables.tf#L17) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | <code title="object({ id = string is_org_level = optional(bool, true) })">object({…})</code> | ✓ | | |
|
||||
| [organization](variables.tf#L194) | Organization details. | <code title="object({ domain = string id = number customer_id = string })">object({…})</code> | ✓ | | |
|
||||
| [prefix](variables.tf#L209) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | |
|
||||
| [bootstrap_user](variables.tf#L29) | Email of the nominal user running this stage for the first time. | <code>string</code> | | <code>null</code> | |
|
||||
| [cicd_repositories](variables.tf#L35) | CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | <code title="object({ bootstrap = optional(object({ branch = string identity_provider = string name = string type = string })) resman = optional(object({ branch = string identity_provider = string name = string type = string })) })">object({…})</code> | | <code>null</code> | |
|
||||
| [custom_role_names](variables.tf#L81) | Names of custom roles defined at the org level. | <code title="object({ organization_iam_admin = string service_project_network_admin = string })">object({…})</code> | | <code title="{ organization_iam_admin = "organizationIamAdmin" service_project_network_admin = "serviceProjectNetworkAdmin" }">{…}</code> | |
|
||||
| [fast_features](variables.tf#L93) | Selective control for top-level FAST features. | <code title="object({ data_platform = optional(bool, false) gke = optional(bool, false) project_factory = optional(bool, false) sandbox = optional(bool, false) teams = optional(bool, false) })">object({…})</code> | | <code>{}</code> | |
|
||||
| [federated_identity_providers](variables.tf#L106) | Workload Identity Federation pools. The `cicd_repositories` variable references keys here. | <code title="map(object({ attribute_condition = string issuer = string custom_settings = object({ issuer_uri = string allowed_audiences = list(string) }) }))">map(object({…}))</code> | | <code>{}</code> | |
|
||||
| [groups](variables.tf#L120) | Group names to grant organization-level permissions. | <code>map(string)</code> | | <code title="{ gcp-billing-admins = "gcp-billing-admins", gcp-devops = "gcp-devops", gcp-network-admins = "gcp-network-admins" gcp-organization-admins = "gcp-organization-admins" gcp-security-admins = "gcp-security-admins" gcp-support = "gcp-devops" }">{…}</code> | |
|
||||
| [iam](variables.tf#L138) | Organization-level custom IAM settings in role => [principal] format. | <code>map(list(string))</code> | | <code>{}</code> | |
|
||||
| [iam_additive](variables.tf#L144) | Organization-level custom IAM settings in role => [principal] format for non-authoritative bindings. | <code>map(list(string))</code> | | <code>{}</code> | |
|
||||
| [locations](variables.tf#L150) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object({ bq = string gcs = string logging = string pubsub = list(string) })">object({…})</code> | | <code title="{ bq = "EU" gcs = "EU" logging = "global" pubsub = [] }">{…}</code> | |
|
||||
| [log_sinks](variables.tf#L169) | Org-level log sinks, in name => {type, filter} format. | <code title="map(object({ filter = string type = string }))">map(object({…}))</code> | | <code title="{ audit-logs = { filter = "logName:\"/logs/cloudaudit.googleapis.com%2Factivity\" OR logName:\"/logs/cloudaudit.googleapis.com%2Fsystem_event\"" type = "logging" } vpc-sc = { filter = "protoPayload.metadata.@type=\"type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata\"" type = "logging" } }">{…}</code> | |
|
||||
| [outputs_location](variables.tf#L203) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | <code>string</code> | | <code>null</code> | |
|
||||
| [project_parent_ids](variables.tf#L219) | Optional parents for projects created here in folders/nnnnnnn format. Null values will use the organization as parent. | <code title="object({ automation = string billing = string logging = string })">object({…})</code> | | <code title="{ automation = null billing = null logging = null }">{…}</code> | |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive | consumers |
|
||||
|---|---|:---:|---|
|
||||
| [automation](outputs.tf#L89) | Automation resources. | | |
|
||||
| [billing_dataset](outputs.tf#L94) | BigQuery dataset prepared for billing export. | | |
|
||||
| [cicd_repositories](outputs.tf#L99) | CI/CD repository configurations. | | |
|
||||
| [custom_roles](outputs.tf#L111) | Organization-level custom roles. | | |
|
||||
| [federated_identity](outputs.tf#L116) | Workload Identity Federation pool and providers. | | |
|
||||
| [outputs_bucket](outputs.tf#L126) | GCS bucket where generated output files are stored. | | |
|
||||
| [project_ids](outputs.tf#L131) | Projects created by this stage. | | |
|
||||
| [providers](outputs.tf#L141) | Terraform provider files for this stage and dependent stages. | ✓ | <code>stage-01</code> |
|
||||
| [service_accounts](outputs.tf#L148) | Automation service accounts created by this stage. | | |
|
||||
| [tfvars](outputs.tf#L158) | Terraform variable files for the following stages. | ✓ | |
|
||||
| [automation](outputs.tf#L86) | Automation resources. | | |
|
||||
| [billing_dataset](outputs.tf#L91) | BigQuery dataset prepared for billing export. | | |
|
||||
| [cicd_repositories](outputs.tf#L96) | CI/CD repository configurations. | | |
|
||||
| [custom_roles](outputs.tf#L108) | Organization-level custom roles. | | |
|
||||
| [federated_identity](outputs.tf#L113) | Workload Identity Federation pool and providers. | | |
|
||||
| [outputs_bucket](outputs.tf#L123) | GCS bucket where generated output files are stored. | | |
|
||||
| [project_ids](outputs.tf#L128) | Projects created by this stage. | | |
|
||||
| [providers](outputs.tf#L138) | Terraform provider files for this stage and dependent stages. | ✓ | <code>stage-01</code> |
|
||||
| [service_accounts](outputs.tf#L145) | Automation service accounts created by this stage. | | |
|
||||
| [tfvars](outputs.tf#L154) | Terraform variable files for the following stages. | ✓ | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -127,39 +127,6 @@ module "automation-tf-bootstrap-sa" {
|
|||
}
|
||||
}
|
||||
|
||||
# cicd stage's bucket and service account
|
||||
|
||||
module "automation-tf-cicd-gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
project_id = module.automation-project.project_id
|
||||
name = "iac-core-cicd-0"
|
||||
prefix = local.prefix
|
||||
location = var.locations.gcs
|
||||
storage_class = local.gcs_storage_class
|
||||
versioning = true
|
||||
iam = {
|
||||
"roles/storage.objectAdmin" = [module.automation-tf-cicd-provisioning-sa.iam_email]
|
||||
}
|
||||
depends_on = [module.organization]
|
||||
}
|
||||
|
||||
module "automation-tf-cicd-provisioning-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
project_id = module.automation-project.project_id
|
||||
name = "cicd-0"
|
||||
display_name = "Terraform stage 1 CICD service account."
|
||||
prefix = local.prefix
|
||||
# allow SA used by CI/CD workflow to impersonate this SA
|
||||
iam = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.automation-tf-cicd-sa["cicd"].iam_email, null)
|
||||
])
|
||||
}
|
||||
iam_storage_roles = {
|
||||
(module.automation-tf-output-gcs.name) = ["roles/storage.admin"]
|
||||
}
|
||||
}
|
||||
|
||||
# resource hierarchy stage's bucket and service account
|
||||
|
||||
module "automation-tf-resman-gcs" {
|
||||
|
|
@ -183,7 +150,8 @@ module "automation-tf-resman-sa" {
|
|||
display_name = "Terraform stage 1 resman service account."
|
||||
prefix = local.prefix
|
||||
# allow SA used by CI/CD workflow to impersonate this SA
|
||||
iam = {
|
||||
# we use additive IAM to allow tenant CI/CD SAs to impersonate it
|
||||
iam_additive = {
|
||||
"roles/iam.serviceAccountTokenCreator" = compact([
|
||||
try(module.automation-tf-cicd-sa["resman"].iam_email, null)
|
||||
])
|
||||
|
|
@ -30,7 +30,7 @@ locals {
|
|||
|
||||
module "billing-export-project" {
|
||||
source = "../../../modules/project"
|
||||
count = local.billing_org ? 1 : 0
|
||||
count = var.billing_account.is_org_level ? 1 : 0
|
||||
billing_account = var.billing_account.id
|
||||
name = "billing-exp-0"
|
||||
parent = coalesce(
|
||||
|
|
@ -52,56 +52,18 @@ module "billing-export-project" {
|
|||
|
||||
module "billing-export-dataset" {
|
||||
source = "../../../modules/bigquery-dataset"
|
||||
count = local.billing_org ? 1 : 0
|
||||
count = var.billing_account.is_org_level ? 1 : 0
|
||||
project_id = module.billing-export-project.0.project_id
|
||||
id = "billing_export"
|
||||
friendly_name = "Billing export."
|
||||
location = var.locations.bq
|
||||
}
|
||||
|
||||
# billing account in a different org
|
||||
|
||||
module "billing-organization-ext" {
|
||||
source = "../../../modules/organization"
|
||||
count = local.billing_org_ext ? 1 : 0
|
||||
organization_id = "organizations/${var.billing_account.organization_id}"
|
||||
iam_additive = {
|
||||
"roles/billing.admin" = local.billing_ext_admins
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
resource "google_organization_iam_binding" "billing_org_ext_admin_delegated" {
|
||||
# refer to organization.tf for the explanation of how this binding works
|
||||
count = local.billing_org_ext ? 1 : 0
|
||||
org_id = var.billing_account.organization_id
|
||||
# if the billing org does not have our custom role, user the predefined one
|
||||
# role = "roles/resourcemanager.organizationAdmin"
|
||||
role = join("", [
|
||||
"organizations/${var.billing_account.organization_id}/",
|
||||
"roles/${var.custom_role_names.organization_iam_admin}"
|
||||
])
|
||||
members = [module.automation-tf-resman-sa.iam_email]
|
||||
condition {
|
||||
title = "automation_sa_delegated_grants"
|
||||
description = "Automation service account delegated grants."
|
||||
expression = format(
|
||||
"api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])",
|
||||
join(",", formatlist("'%s'", [
|
||||
"roles/billing.costsManager",
|
||||
"roles/billing.user",
|
||||
]
|
||||
))
|
||||
)
|
||||
}
|
||||
depends_on = [module.billing-organization-ext]
|
||||
}
|
||||
|
||||
# standalone billing account
|
||||
|
||||
resource "google_billing_account_iam_member" "billing_ext_admin" {
|
||||
for_each = toset(
|
||||
local.billing_ext ? local.billing_ext_admins : []
|
||||
!var.billing_account.is_org_level ? local.billing_ext_admins : []
|
||||
)
|
||||
billing_account_id = var.billing_account.id
|
||||
role = "roles/billing.admin"
|
||||
|
|
@ -110,7 +72,7 @@ resource "google_billing_account_iam_member" "billing_ext_admin" {
|
|||
|
||||
resource "google_billing_account_iam_member" "billing_ext_cost_manager" {
|
||||
for_each = toset(
|
||||
local.billing_ext ? local.billing_ext_admins : []
|
||||
!var.billing_account.is_org_level ? local.billing_ext_admins : []
|
||||
)
|
||||
billing_account_id = var.billing_account.id
|
||||
role = "roles/billing.costsManager"
|
||||
|
|
@ -17,6 +17,16 @@
|
|||
# tfdoc:file:description Workload Identity Federation configurations for CI/CD.
|
||||
|
||||
locals {
|
||||
cicd_providers = {
|
||||
for k, v in google_iam_workload_identity_pool_provider.default :
|
||||
k => {
|
||||
issuer = local.identity_providers[k].issuer
|
||||
issuer_uri = local.identity_providers[k].issuer_uri
|
||||
name = v.name
|
||||
principal_tpl = local.identity_providers[k].principal_tpl
|
||||
principalset_tpl = local.identity_providers[k].principalset_tpl
|
||||
}
|
||||
}
|
||||
cicd_repositories = {
|
||||
for k, v in coalesce(var.cicd_repositories, {}) : k => v
|
||||
if(
|
||||
|
|
@ -32,18 +42,13 @@ locals {
|
|||
)
|
||||
}
|
||||
cicd_workflow_providers = {
|
||||
bootstrap = "00-bootstrap-providers.tf"
|
||||
cicd = "00-cicd-providers.tf"
|
||||
resman = "01-resman-providers.tf"
|
||||
bootstrap = "0-bootstrap-providers.tf"
|
||||
resman = "1-resman-providers.tf"
|
||||
}
|
||||
cicd_workflow_var_files = {
|
||||
bootstrap = []
|
||||
cicd = [
|
||||
"00-bootstrap.auto.tfvars.json",
|
||||
"globals.auto.tfvars.json"
|
||||
]
|
||||
resman = [
|
||||
"00-bootstrap.auto.tfvars.json",
|
||||
"0-bootstrap.auto.tfvars.json",
|
||||
"globals.auto.tfvars.json"
|
||||
]
|
||||
}
|
||||
|
|
@ -69,7 +74,7 @@ module "automation-tf-cicd-repo" {
|
|||
]
|
||||
}
|
||||
triggers = {
|
||||
"fast-00-${each.key}" = {
|
||||
"fast-0-${each.key}" = {
|
||||
filename = ".cloudbuild/workflow.yaml"
|
||||
included_files = ["**/*tf", ".cloudbuild/workflow.yaml"]
|
||||
service_account = module.automation-tf-cicd-sa[each.key].id
|
||||
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 158 KiB After Width: | Height: | Size: 158 KiB |
|
Before Width: | Height: | Size: 6.6 MiB After Width: | Height: | Size: 6.6 MiB |
|
|
@ -17,6 +17,16 @@
|
|||
# tfdoc:file:description Audit log project and sink.
|
||||
|
||||
locals {
|
||||
log_sink_destinations = merge(
|
||||
# use the same dataset for all sinks with `bigquery` as destination
|
||||
{ for k, v in var.log_sinks : k => module.log-export-dataset.0 if v.type == "bigquery" },
|
||||
# use the same gcs bucket for all sinks with `storage` as destination
|
||||
{ for k, v in var.log_sinks : k => module.log-export-gcs.0 if v.type == "storage" },
|
||||
# use separate pubsub topics and logging buckets for sinks with
|
||||
# destination `pubsub` and `logging`
|
||||
module.log-export-pubsub,
|
||||
module.log-export-logbucket
|
||||
)
|
||||
log_types = toset([for k, v in var.log_sinks : v.type])
|
||||
}
|
||||
|
||||
|
|
@ -28,10 +28,6 @@ locals {
|
|||
for k, v in local.groups :
|
||||
k => "group:${v}"
|
||||
}
|
||||
# convenience flags that express where billing account resides
|
||||
billing_ext = var.billing_account.organization_id == null
|
||||
billing_org = var.billing_account.organization_id == var.organization.id
|
||||
billing_org_ext = !local.billing_ext && !local.billing_org
|
||||
# naming: environment used in most resource names
|
||||
prefix = join("-", compact([var.prefix, "prod"]))
|
||||
}
|
||||
|
|
@ -24,7 +24,10 @@ locals {
|
|||
"domain:${var.organization.domain}"
|
||||
]
|
||||
"roles/logging.admin" = concat(
|
||||
[module.automation-tf-bootstrap-sa.iam_email],
|
||||
[
|
||||
module.automation-tf-bootstrap-sa.iam_email,
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
],
|
||||
local._iam_bootstrap_user
|
||||
)
|
||||
"roles/owner" = local._iam_bootstrap_user
|
||||
|
|
@ -35,12 +38,11 @@ locals {
|
|||
[module.automation-tf-bootstrap-sa.iam_email],
|
||||
local._iam_bootstrap_user
|
||||
)
|
||||
# the following is useful if roles/browser is not desirable
|
||||
# "roles/resourcemanager.organizationViewer" = [
|
||||
# "domain:${var.organization.domain}"
|
||||
# ]
|
||||
"roles/resourcemanager.projectCreator" = concat(
|
||||
[module.automation-tf-bootstrap-sa.iam_email],
|
||||
[
|
||||
module.automation-tf-bootstrap-sa.iam_email,
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
],
|
||||
local._iam_bootstrap_user
|
||||
)
|
||||
"roles/resourcemanager.projectMover" = [
|
||||
|
|
@ -78,8 +80,12 @@ locals {
|
|||
local.groups_iam.gcp-security-admins,
|
||||
module.automation-tf-resman-sa.iam_email
|
||||
]
|
||||
# the following is useful if roles/browser is not desirable
|
||||
# "roles/resourcemanager.organizationViewer" = [
|
||||
# "domain:${var.organization.domain}"
|
||||
# ]
|
||||
},
|
||||
local.billing_org ? {
|
||||
var.billing_account.is_org_level ? {
|
||||
"roles/billing.admin" = [
|
||||
local.groups_iam.gcp-billing-admins,
|
||||
local.groups_iam.gcp-organization-admins,
|
||||
|
|
@ -115,16 +121,6 @@ locals {
|
|||
iam_roles_additive = distinct(concat(
|
||||
keys(local._iam_additive), keys(var.iam_additive)
|
||||
))
|
||||
log_sink_destinations = merge(
|
||||
# use the same dataset for all sinks with `bigquery` as destination
|
||||
{ for k, v in var.log_sinks : k => module.log-export-dataset.0 if v.type == "bigquery" },
|
||||
# use the same gcs bucket for all sinks with `storage` as destination
|
||||
{ for k, v in var.log_sinks : k => module.log-export-gcs.0 if v.type == "storage" },
|
||||
# use separate pubsub topics and logging buckets for sinks with
|
||||
# destination `pubsub` and `logging`
|
||||
module.log-export-pubsub,
|
||||
module.log-export-logbucket
|
||||
)
|
||||
}
|
||||
|
||||
module "organization" {
|
||||
|
|
@ -220,8 +216,9 @@ resource "google_organization_iam_binding" "org_admin_delegated" {
|
|||
"roles/compute.orgFirewallPolicyAdmin",
|
||||
"roles/compute.xpnAdmin",
|
||||
"roles/orgpolicy.policyAdmin",
|
||||
"roles/resourcemanager.organizationViewer",
|
||||
],
|
||||
local.billing_org ? [
|
||||
var.billing_account.is_org_level ? [
|
||||
"roles/billing.admin",
|
||||
"roles/billing.costsManager",
|
||||
"roles/billing.user",
|
||||
|
|
@ -26,7 +26,7 @@ resource "local_file" "providers" {
|
|||
resource "local_file" "tfvars" {
|
||||
for_each = var.outputs_location == null ? {} : { 1 = 1 }
|
||||
file_permission = "0644"
|
||||
filename = "${try(pathexpand(var.outputs_location), "")}/tfvars/00-bootstrap.auto.tfvars.json"
|
||||
filename = "${try(pathexpand(var.outputs_location), "")}/tfvars/0-bootstrap.auto.tfvars.json"
|
||||
content = jsonencode(local.tfvars)
|
||||
}
|
||||
|
||||
|
|
@ -26,7 +26,7 @@ resource "google_storage_bucket_object" "providers" {
|
|||
|
||||
resource "google_storage_bucket_object" "tfvars" {
|
||||
bucket = module.automation-tf-output-gcs.name
|
||||
name = "tfvars/00-bootstrap.auto.tfvars.json"
|
||||
name = "tfvars/0-bootstrap.auto.tfvars.json"
|
||||
content = jsonencode(local.tfvars)
|
||||
}
|
||||
|
||||
|
|
@ -21,7 +21,7 @@ locals {
|
|||
for k, v in local.cicd_repositories : k => templatefile(
|
||||
"${path.module}/templates/workflow-${v.type}.yaml", {
|
||||
identity_provider = try(
|
||||
local.wif_providers[v["identity_provider"]].name, ""
|
||||
local.cicd_providers[v["identity_provider"]].name, ""
|
||||
)
|
||||
outputs_bucket = module.automation-tf-output-gcs.name
|
||||
service_account = try(
|
||||
|
|
@ -38,19 +38,26 @@ locals {
|
|||
k => try(module.organization.custom_role_id[v], null)
|
||||
}
|
||||
providers = {
|
||||
"00-bootstrap" = templatefile(local._tpl_providers, {
|
||||
bucket = module.automation-tf-bootstrap-gcs.name
|
||||
name = "bootstrap"
|
||||
sa = module.automation-tf-bootstrap-sa.email
|
||||
"0-bootstrap" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.automation-tf-bootstrap-gcs.name
|
||||
name = "bootstrap"
|
||||
sa = module.automation-tf-bootstrap-sa.email
|
||||
})
|
||||
"00-cicd" = templatefile(local._tpl_providers, {
|
||||
bucket = module.automation-tf-cicd-gcs.name
|
||||
name = "cicd"
|
||||
sa = module.automation-tf-cicd-provisioning-sa.email
|
||||
"1-resman" = templatefile(local._tpl_providers, {
|
||||
backend_extra = null
|
||||
bucket = module.automation-tf-resman-gcs.name
|
||||
name = "resman"
|
||||
sa = module.automation-tf-resman-sa.email
|
||||
})
|
||||
"01-resman" = templatefile(local._tpl_providers, {
|
||||
"0-bootstrap-tenant" = templatefile(local._tpl_providers, {
|
||||
backend_extra = join("\n", [
|
||||
"# remove the newline between quotes and set the tenant name as prefix",
|
||||
"prefix = \"",
|
||||
"\""
|
||||
])
|
||||
bucket = module.automation-tf-resman-gcs.name
|
||||
name = "resman"
|
||||
name = "bootstrap-tenant"
|
||||
sa = module.automation-tf-resman-sa.email
|
||||
})
|
||||
}
|
||||
|
|
@ -59,7 +66,7 @@ locals {
|
|||
federated_identity_pool = try(
|
||||
google_iam_workload_identity_pool.default.0.name, null
|
||||
)
|
||||
federated_identity_providers = local.wif_providers
|
||||
federated_identity_providers = local.cicd_providers
|
||||
outputs_bucket = module.automation-tf-output-gcs.name
|
||||
project_id = module.automation-project.project_id
|
||||
project_number = module.automation-project.number
|
||||
|
|
@ -74,16 +81,6 @@ locals {
|
|||
organization = var.organization
|
||||
prefix = var.prefix
|
||||
}
|
||||
wif_providers = {
|
||||
for k, v in google_iam_workload_identity_pool_provider.default :
|
||||
k => {
|
||||
issuer = local.identity_providers[k].issuer
|
||||
issuer_uri = local.identity_providers[k].issuer_uri
|
||||
name = v.name
|
||||
principal_tpl = local.identity_providers[k].principal_tpl
|
||||
principalset_tpl = local.identity_providers[k].principalset_tpl
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output "automation" {
|
||||
|
|
@ -102,7 +99,7 @@ output "cicd_repositories" {
|
|||
for k, v in local.cicd_repositories : k => {
|
||||
branch = v.branch
|
||||
name = v.name
|
||||
provider = try(local.wif_providers[v.identity_provider].name, null)
|
||||
provider = try(local.cicd_providers[v.identity_provider].name, null)
|
||||
service_account = try(module.automation-tf-cicd-sa[k].email, null)
|
||||
}
|
||||
}
|
||||
|
|
@ -119,7 +116,7 @@ output "federated_identity" {
|
|||
pool = try(
|
||||
google_iam_workload_identity_pool.default.0.name, null
|
||||
)
|
||||
providers = local.wif_providers
|
||||
providers = local.cicd_providers
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -149,7 +146,6 @@ output "service_accounts" {
|
|||
description = "Automation service accounts created by this stage."
|
||||
value = {
|
||||
bootstrap = module.automation-tf-bootstrap-sa.email
|
||||
cicd = module.automation-tf-cicd-provisioning-sa.email
|
||||
resman = module.automation-tf-resman-sa.email
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
terraform {
|
||||
backend "gcs" {
|
||||
bucket = "${bucket}"
|
||||
impersonate_service_account = "${sa}"
|
||||
%{~ if backend_extra != null ~}
|
||||
${indent(4, backend_extra)}
|
||||
%{~ endif ~}
|
||||
}
|
||||
}
|
||||
provider "google" {
|
||||
impersonate_service_account = "${sa}"
|
||||
}
|
||||
provider "google-beta" {
|
||||
impersonate_service_account = "${sa}"
|
||||
}
|
||||
|
||||
# end provider.tf for ${name}
|
||||
|
|
@ -0,0 +1,186 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
name: "FAST ${stage_name} stage"
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
types:
|
||||
- closed
|
||||
- opened
|
||||
- synchronize
|
||||
|
||||
env:
|
||||
FAST_OUTPUTS_BUCKET: ${outputs_bucket}
|
||||
FAST_SERVICE_ACCOUNT: ${service_account}
|
||||
FAST_WIF_PROVIDER: ${identity_provider}
|
||||
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
|
||||
TF_PROVIDERS_FILE: ${tf_providers_file}
|
||||
TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
|
||||
TF_VERSION: 1.3.2
|
||||
|
||||
jobs:
|
||||
fast-pr:
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
issues: write
|
||||
pull-requests: write
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- id: checkout
|
||||
name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
|
||||
# set up SSH key authentication to the modules repository
|
||||
- id: ssh-config
|
||||
name: Configure SSH authentication
|
||||
run: |
|
||||
ssh-agent -a "$SSH_AUTH_SOCK" > /dev/null
|
||||
ssh-add - <<< "$${{ secrets.CICD_MODULES_KEY }}"
|
||||
|
||||
# set up authentication via Workload identity Federation
|
||||
- id: gcp-auth
|
||||
name: Authenticate to Google Cloud
|
||||
uses: google-github-actions/auth@v0
|
||||
with:
|
||||
workload_identity_provider: $${{ env.FAST_WIF_PROVIDER }}
|
||||
service_account: $${{ env.FAST_SERVICE_ACCOUNT }}
|
||||
access_token_lifetime: 3600s
|
||||
|
||||
- id: gcp-sdk
|
||||
name: Set up Cloud SDK
|
||||
uses: google-github-actions/setup-gcloud@v0
|
||||
with:
|
||||
install_components: alpha
|
||||
|
||||
# copy provider and tfvars files
|
||||
- id: tf-config
|
||||
name: Copy Terraform output files
|
||||
run: |
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${{env.FAST_OUTPUTS_BUCKET}}/providers/$${{env.TF_PROVIDERS_FILE}}" ./
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${{env.FAST_OUTPUTS_BUCKET}}/tfvars" ./
|
||||
for f in $${{env.TF_VAR_FILES}}; do
|
||||
ln -s "tfvars/$f" ./
|
||||
done
|
||||
|
||||
- id: tf-setup
|
||||
name: Set up Terraform
|
||||
uses: hashicorp/setup-terraform@v2.0.3
|
||||
with:
|
||||
terraform_version: $${{ env.TF_VERSION }}
|
||||
|
||||
# run Terraform init/validate/plan
|
||||
- id: tf-init
|
||||
name: Terraform init
|
||||
run: |
|
||||
terraform init -no-color
|
||||
|
||||
- id: tf-validate
|
||||
name: Terraform validate
|
||||
run: terraform validate -no-color
|
||||
|
||||
- id: tf-plan
|
||||
name: Terraform plan
|
||||
continue-on-error: true
|
||||
run: |
|
||||
terraform plan -input=false -out ../plan.out -no-color
|
||||
|
||||
- id: tf-apply
|
||||
if: github.event.pull_request.merged == true && success()
|
||||
name: Terraform apply
|
||||
continue-on-error: true
|
||||
run: |
|
||||
terraform apply -input=false -auto-approve -no-color ../plan.out
|
||||
|
||||
- id: pr-comment
|
||||
name: Post comment to Pull Request
|
||||
continue-on-error: true
|
||||
uses: actions/github-script@v6
|
||||
if: github.event_name == 'pull_request'
|
||||
env:
|
||||
PLAN: $${{ steps.tf-plan.outputs.stdout }}\n$${{ steps.tf-plan.outputs.stderr }}
|
||||
with:
|
||||
script: |
|
||||
const output = `### Terraform Initialization \`$${{ steps.tf-init.outcome }}\`
|
||||
|
||||
### Terraform Validation \`$${{ steps.tf-validate.outcome }}\`
|
||||
|
||||
<details><summary>Validation Output</summary>
|
||||
|
||||
\`\`\`\n
|
||||
$${{ steps.tf-validate.outputs.stdout }}
|
||||
\`\`\`
|
||||
|
||||
</details>
|
||||
|
||||
### Terraform Plan \`$${{ steps.tf-plan.outcome }}\`
|
||||
|
||||
<details><summary>Show Plan</summary>
|
||||
|
||||
\`\`\`\n
|
||||
$${process.env.PLAN.split('\n').filter(l => l.match(/^([A-Z\s].*|)$$/)).join('\n')}
|
||||
\`\`\`
|
||||
|
||||
</details>
|
||||
|
||||
### Terraform Apply \`$${{ steps.tf-apply.outcome }}\`
|
||||
|
||||
*Pusher: @$${{ github.actor }}, Action: \`$${{ github.event_name }}\`, Working Directory: \`$${{ env.tf_actions_working_dir }}\`, Workflow: \`$${{ github.workflow }}\`*`;
|
||||
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: output
|
||||
})
|
||||
|
||||
- id: pr-short-comment
|
||||
name: Post comment to Pull Request
|
||||
uses: actions/github-script@v6
|
||||
if: github.event_name == 'pull_request' && steps.pr-comment.outcome != 'success'
|
||||
with:
|
||||
script: |
|
||||
const output = `### Terraform Initialization \`$${{ steps.tf-init.outcome }}\`
|
||||
|
||||
### Terraform Validation \`$${{ steps.tf-validate.outcome }}\`
|
||||
|
||||
### Terraform Plan \`$${{ steps.tf-plan.outcome }}\`
|
||||
|
||||
Plan output is in the action log.
|
||||
|
||||
### Terraform Apply \`$${{ steps.tf-apply.outcome }}\`
|
||||
|
||||
*Pusher: @$${{ github.actor }}, Action: \`$${{ github.event_name }}\`, Working Directory: \`$${{ env.tf_actions_working_dir }}\`, Workflow: \`$${{ github.workflow }}\`*`;
|
||||
|
||||
github.rest.issues.createComment({
|
||||
issue_number: context.issue.number,
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
body: output
|
||||
})
|
||||
|
||||
- id: check-plan
|
||||
name: Check plan failure
|
||||
if: steps.tf-plan.outcome != 'success'
|
||||
run: exit 1
|
||||
|
||||
- id: check-apply
|
||||
name: Check apply failure
|
||||
if: github.event.pull_request.merged == true && steps.tf-apply.outcome != 'success'
|
||||
run: exit 1
|
||||
|
|
@ -0,0 +1,120 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
default:
|
||||
before_script:
|
||||
- echo "$${CI_JOB_JWT_V2}" > token.txt
|
||||
image:
|
||||
name: hashicorp/terraform
|
||||
entrypoint:
|
||||
- "/usr/bin/env"
|
||||
- "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
|
||||
variables:
|
||||
GOOGLE_CREDENTIALS: cicd-sa-credentials.json
|
||||
FAST_OUTPUTS_BUCKET: ${outputs_bucket}
|
||||
FAST_SERVICE_ACCOUNT: ${service_account}
|
||||
FAST_WIF_PROVIDER: ${identity_provider}
|
||||
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
|
||||
TF_PROVIDERS_FILE: ${tf_providers_file}
|
||||
TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
|
||||
|
||||
stages:
|
||||
- gcp-auth
|
||||
- tf-files
|
||||
- tf-plan
|
||||
- tf-apply
|
||||
|
||||
cache:
|
||||
key: gcp-auth
|
||||
paths:
|
||||
- cicd-sa-credentials.json
|
||||
- .tf-setup
|
||||
|
||||
gcp-auth:
|
||||
image:
|
||||
name: google/cloud-sdk:slim
|
||||
stage: gcp-auth
|
||||
script:
|
||||
- |
|
||||
gcloud iam workload-identity-pools create-cred-config \
|
||||
$${FAST_WIF_PROVIDER} \
|
||||
--service-account=$${FAST_SERVICE_ACCOUNT} \
|
||||
--service-account-token-lifetime-seconds=3600 \
|
||||
--output-file=$${GOOGLE_CREDENTIALS} \
|
||||
--credential-source-file=token.txt
|
||||
tf-files:
|
||||
dependencies:
|
||||
- gcp-auth
|
||||
image:
|
||||
name: google/cloud-sdk:slim
|
||||
stage: tf-files
|
||||
script:
|
||||
# - gcloud components install -q alpha
|
||||
- gcloud config set auth/credential_file_override $${GOOGLE_CREDENTIALS}
|
||||
- mkdir -p .tf-setup
|
||||
- |
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${FAST_OUTPUTS_BUCKET}/providers/$${TF_PROVIDERS_FILE}" .tf-setup/
|
||||
- |
|
||||
gcloud alpha storage cp -r \
|
||||
"gs://$${FAST_OUTPUTS_BUCKET}/tfvars" .tf-setup/
|
||||
|
||||
tf-plan:
|
||||
# uncomment the following lines and set the SSH key secret for private modules repo
|
||||
# before_script:
|
||||
# - |
|
||||
# ssh-agent -a $SSH_AUTH_SOCK > /dev/null
|
||||
# echo "$CICD_MODULES_KEY" | base64 -d | tr -d '\r' | ssh-add - > /dev/null
|
||||
# mkdir -p ~/.ssh
|
||||
# ssh-keyscan -H 'gitlab.com' >> ~/.ssh/known_hosts
|
||||
# ssh-keyscan gitlab.com | sort -u - ~/.ssh/known_hosts -o ~/.ssh/known_hosts
|
||||
stage: tf-plan
|
||||
script:
|
||||
- cp .tf-setup/$${TF_PROVIDERS_FILE} ./
|
||||
- |
|
||||
for f in $${TF_VAR_FILES}; do
|
||||
ln -s ".tf-setup/tfvars/$f" ./
|
||||
done
|
||||
- terraform init
|
||||
- terraform validate
|
||||
- terraform plan
|
||||
dependencies:
|
||||
- tf-files
|
||||
|
||||
tf-apply:
|
||||
# uncomment the following lines and set the SSH key secret for private modules repo
|
||||
# before_script:
|
||||
# - |
|
||||
# ssh-agent -a $SSH_AUTH_SOCK > /dev/null
|
||||
# echo "$CICD_MODULES_KEY" | base64 -d | tr -d '\r' | ssh-add - > /dev/null
|
||||
# mkdir -p ~/.ssh
|
||||
# ssh-keyscan -H 'gitlab.com' >> ~/.ssh/known_hosts
|
||||
# ssh-keyscan gitlab.com | sort -u - ~/.ssh/known_hosts -o ~/.ssh/known_hosts
|
||||
stage: tf-apply
|
||||
script:
|
||||
- cp .tf-setup/$${TF_PROVIDERS_FILE} ./
|
||||
- |
|
||||
for f in $${TF_VAR_FILES}; do
|
||||
ln -s ".tf-setup/tfvars/$f" ./
|
||||
done
|
||||
- terraform init
|
||||
- terraform validate
|
||||
- terraform apply -input=false -auto-approve
|
||||
dependencies:
|
||||
- tf-files
|
||||
when: manual
|
||||
only:
|
||||
variables:
|
||||
- $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
|
||||
|
|
@ -0,0 +1,98 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
steps:
|
||||
- name: alpine:3
|
||||
id: tf-download
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
mkdir -p /builder/home/.local/bin
|
||||
wget https://releases.hashicorp.com/terraform/$${_TF_VERSION}/terraform_$${_TF_VERSION}_linux_amd64.zip
|
||||
unzip terraform_$${_TF_VERSION}_linux_amd64.zip -d /builder/home/.local/bin
|
||||
rm terraform_$${_TF_VERSION}_linux_amd64.zip
|
||||
chmod 755 /builder/home/.local/bin/terraform
|
||||
- name: alpine:3
|
||||
id: tf-check-format
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform fmt -recursive -check /workspace/
|
||||
- name: gcr.io/google.com/cloudsdktool/cloud-sdk:alpine
|
||||
id: tf-files
|
||||
entrypoint: bash
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
/google-cloud-sdk/bin/gsutil cp \
|
||||
gs://$${_FAST_OUTPUTS_BUCKET}/providers/$${_TF_PROVIDERS_FILE} ./
|
||||
/google-cloud-sdk/bin/gsutil cp -r \
|
||||
gs://$${_FAST_OUTPUTS_BUCKET}/tfvars ./
|
||||
for f in $${_TF_VAR_FILES}; do
|
||||
ln -s tfvars/$f ./
|
||||
done
|
||||
- name: alpine:3
|
||||
id: tf-init
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform init -no-color
|
||||
- name: alpine:3
|
||||
id: tf-check-validate
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform validate -no-color
|
||||
- name: alpine:3
|
||||
id: tf-plan
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform plan -no-color -input=false -out plan.out
|
||||
# store artifact and ask for approval here if needed
|
||||
- name: alpine:3
|
||||
id: tf-apply
|
||||
entrypoint: sh
|
||||
args:
|
||||
- -eEuo
|
||||
- pipefail
|
||||
- -c
|
||||
- |-
|
||||
terraform apply -no-color -input=false -auto-approve plan.out
|
||||
options:
|
||||
env:
|
||||
- PATH=/usr/local/bin:/usr/bin:/bin:/builder/home/.local/bin
|
||||
logging: CLOUD_LOGGING_ONLY
|
||||
substitutions:
|
||||
_FAST_OUTPUTS_BUCKET: ${outputs_bucket}
|
||||
_TF_PROVIDERS_FILE: ${tf_providers_file}
|
||||
_TF_VAR_FILES: ${tf_var_files == [] ? "''" : join("\n ", tf_var_files)}
|
||||
_TF_VERSION: 1.3.2
|
||||
|
|
@ -15,11 +15,15 @@
|
|||
*/
|
||||
|
||||
variable "billing_account" {
|
||||
description = "Billing account id and organization id ('nnnnnnnn' or null)."
|
||||
description = "Billing account id. If billing account is not part of the same org set `is_org_level` to false."
|
||||
type = object({
|
||||
id = string
|
||||
organization_id = number
|
||||
id = string
|
||||
is_org_level = optional(bool, true)
|
||||
})
|
||||
validation {
|
||||
condition = var.billing_account.is_org_level != null
|
||||
error_message = "Invalid `null` value for `billing_account.is_org_level`."
|
||||
}
|
||||
}
|
||||
|
||||
variable "bootstrap_user" {
|
||||
|
|
@ -31,24 +35,18 @@ variable "bootstrap_user" {
|
|||
variable "cicd_repositories" {
|
||||
description = "CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed."
|
||||
type = object({
|
||||
bootstrap = object({
|
||||
bootstrap = optional(object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
cicd = object({
|
||||
}))
|
||||
resman = optional(object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
resman = object({
|
||||
branch = string
|
||||
identity_provider = string
|
||||
name = string
|
||||
type = string
|
||||
})
|
||||
}))
|
||||
})
|
||||
default = null
|
||||
validation {
|
||||
|
|
@ -95,19 +93,13 @@ variable "custom_role_names" {
|
|||
variable "fast_features" {
|
||||
description = "Selective control for top-level FAST features."
|
||||
type = object({
|
||||
data_platform = bool
|
||||
gke = bool
|
||||
project_factory = bool
|
||||
sandbox = bool
|
||||
teams = bool
|
||||
data_platform = optional(bool, false)
|
||||
gke = optional(bool, false)
|
||||
project_factory = optional(bool, false)
|
||||
sandbox = optional(bool, false)
|
||||
teams = optional(bool, false)
|
||||
})
|
||||
default = {
|
||||
data_platform = true
|
||||
gke = true
|
||||
project_factory = true
|
||||
sandbox = true
|
||||
teams = true
|
||||
}
|
||||
default = {}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
|
|
@ -183,11 +175,11 @@ variable "log_sinks" {
|
|||
default = {
|
||||
audit-logs = {
|
||||
filter = "logName:\"/logs/cloudaudit.googleapis.com%2Factivity\" OR logName:\"/logs/cloudaudit.googleapis.com%2Fsystem_event\""
|
||||
type = "bigquery"
|
||||
type = "logging"
|
||||
}
|
||||
vpc-sc = {
|
||||
filter = "protoPayload.metadata.@type=\"type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata\""
|
||||
type = "bigquery"
|
||||
type = "logging"
|
||||
}
|
||||
}
|
||||
validation {
|
||||
|
|
@ -1 +0,0 @@
|
|||
../../assets/templates
|
||||
|
|
@ -1 +0,0 @@
|
|||
../../assets/templates
|
||||
|
|
@ -34,17 +34,21 @@ Additionally, a few critical benefits are directly provided by this design:
|
|||
- grouping application resources and services using teams or business logic is a flexible approach, which maps well to typical operational or budget requirements
|
||||
- automation stages (e.g. Networking) can be segregated in a simple and effective way, by creating the required service accounts and buckets for each stage here, and applying a handful of IAM roles to the relevant folder
|
||||
|
||||
For a discussion on naming, please refer to the [Bootstrap stage documentation](../00-bootstrap/README.md#naming), as the same approach is shared by all stages.
|
||||
For a discussion on naming, please refer to the [Bootstrap stage documentation](../0-bootstrap/README.md#naming), as the same approach is shared by all stages.
|
||||
|
||||
### Multitenancy
|
||||
|
||||
Fully multitenant hierarchies inside the same organization are implemented via [separate additional stages](../../stages-multitenant/) that need to be run once for each tenant, and require this stage as a prerequisite.
|
||||
|
||||
### Workload Identity Federation and CI/CD
|
||||
|
||||
This stage also implements optional support for CI/CD, much in the same way as the bootstrap stage. The only difference is on Workload Identity Federation, which is only configured in bootstrap and made available here via stage interface variables (the automatically generated `.tfvars` files).
|
||||
|
||||
For details on how to configure CI/CD please refer to the [relevant section in the bootstrap stage documentation](../00-bootstrap/README.md#cicd-repositories).
|
||||
For details on how to configure CI/CD please refer to the [relevant section in the bootstrap stage documentation](../0-bootstrap/README.md#cicd-repositories).
|
||||
|
||||
## How to run this stage
|
||||
|
||||
This stage is meant to be executed after the [bootstrap](../00-bootstrap) stage has run, as it leverages the automation service account and bucket created there. The relevant user groups must also exist, but that's one of the requirements for the previous stage too, so if you ran that successfully, you're good to go.
|
||||
This stage is meant to be executed after the [bootstrap](../0-bootstrap) stage has run, as it leverages the automation service account and bucket created there. The relevant user groups must also exist, but that's one of the requirements for the previous stage too, so if you ran that successfully, you're good to go.
|
||||
|
||||
It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the bootstrap stage for the actual roles needed.
|
||||
|
||||
|
|
@ -56,7 +60,7 @@ The default way of making sure you have the right permissions, is to use the ide
|
|||
|
||||
To simplify setup, the previous stage pre-configures a valid providers file in its output, and optionally writes it to a local file if the `outputs_location` variable is set to a valid path.
|
||||
|
||||
If you have set a valid value for `outputs_location` in the bootstrap stage (see the [bootstrap stage README](../00-bootstrap/#output-files-and-cross-stage-variables) for more details), simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
|
||||
If you have set a valid value for `outputs_location` in the bootstrap stage (see the [bootstrap stage README](../0-bootstrap/#output-files-and-cross-stage-variables) for more details), simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
|
||||
|
||||
```bash
|
||||
# `outputs_location` is set to `~/fast-config`
|
||||
|
|
@ -66,12 +70,12 @@ ln -s ~/fast-config/providers/01-resman-providers.tf .
|
|||
If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage's outputs:
|
||||
|
||||
```bash
|
||||
cd ../00-bootstrap
|
||||
cd ../0-bootstrap
|
||||
terraform output -json providers | jq -r '.["01-resman"]' \
|
||||
> ../01-resman/providers.tf
|
||||
> ../1-resman/providers.tf
|
||||
```
|
||||
|
||||
If you want to continue to rely on `outputs_location` logic, create a `terraform.tfvars` file and configure it as described [here](../00-bootstrap/#output-files-and-cross-stage-variables).
|
||||
If you want to continue to rely on `outputs_location` logic, create a `terraform.tfvars` file and configure it as described [here](../0-bootstrap/#output-files-and-cross-stage-variables).
|
||||
|
||||
### Variable configuration
|
||||
|
||||
|
|
@ -82,17 +86,17 @@ There are two broad sets of variables you will need to fill in:
|
|||
|
||||
To avoid the tedious job of filling in the first group of variable with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files.
|
||||
|
||||
If you configured a valid path for `outputs_location` in the bootstrap stage, simply link the relevant `*.auto.tfvars.json` files from the outputs folder. For this stage, you need the `globals.auto.tfvars.json` file containing global values compiled manually for the bootstrap stage, and `00-bootstrap.auto.tfvars.json` containing values derived from resources managed by the bootstrap stage:
|
||||
If you configured a valid path for `outputs_location` in the bootstrap stage, simply link the relevant `*.auto.tfvars.json` files from the outputs folder. For this stage, you need the `globals.auto.tfvars.json` file containing global values compiled manually for the bootstrap stage, and `0-bootstrap.auto.tfvars.json` containing values derived from resources managed by the bootstrap stage:
|
||||
|
||||
```bash
|
||||
# `outputs_location` is set to `~/fast-config`
|
||||
ln -s ~/fast-config/tfvars/globals.auto.tfvars.json .
|
||||
ln -s ~/fast-config/tfvars/00-bootstrap.auto.tfvars.json .
|
||||
ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json .
|
||||
```
|
||||
|
||||
A second set of variables is specific to this stage, they are all optional so if you need to customize them, create an extra `terraform.tfvars` file.
|
||||
|
||||
Refer to the [Variables](#variables) table at the bottom of this document, for a full list of variables, their origin (e.g. a stage or specific to this one), and descriptions explaining their meaning. The sections below also describe some of the possible customizations. For billing configurations, refer to the [Bootstrap documentation on billing](../00-bootstrap/README.md#billing-account) as the `billing_account` variable is identical across all stages.
|
||||
Refer to the [Variables](#variables) table at the bottom of this document, for a full list of variables, their origin (e.g. a stage or specific to this one), and descriptions explaining their meaning. The sections below also describe some of the possible customizations. For billing configurations, refer to the [Bootstrap documentation on billing](../0-bootstrap/README.md#billing-account) as the `billing_account` variable is identical across all stages.
|
||||
|
||||
Once done, you can run this stage:
|
||||
|
||||
|
|
@ -141,7 +145,7 @@ For policies where additional data is needed, a root-level `organization_policy_
|
|||
|
||||
### IAM
|
||||
|
||||
IAM roles can be easily edited in the relevant `branch-xxx.tf` file, following the best practice outlined in the [bootstrap stage](../00-bootstrap#customizations) documentation of separating user-level and service-account level IAM policies in modules' `iam_groups`, `iam`, and `iam_additive` variables.
|
||||
IAM roles can be easily edited in the relevant `branch-xxx.tf` file, following the best practice outlined in the [bootstrap stage](../0-bootstrap#customizations) documentation of separating user-level and service-account level IAM policies in modules' `iam_groups`, `iam`, and `iam_additive` variables.
|
||||
|
||||
A full reference of IAM roles managed by this stage [is available here](./IAM.md).
|
||||
|
||||
|
|
@ -156,11 +160,11 @@ Due to its simplicity, this stage lends itself easily to customizations: adding
|
|||
|
||||
| name | description | modules | resources |
|
||||
|---|---|---|---|
|
||||
| [billing.tf](./billing.tf) | Billing resources for external billing use cases. | <code>organization</code> | <code>google_billing_account_iam_member</code> |
|
||||
| [branch-data-platform.tf](./branch-data-platform.tf) | Data Platform stages resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [billing.tf](./billing.tf) | Billing resources for external billing use cases. | | <code>google_billing_account_iam_member</code> |
|
||||
| [branch-data-platform.tf](./branch-data-platform.tf) | Data Platform stages resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | <code>google_organization_iam_member</code> |
|
||||
| [branch-gke.tf](./branch-gke.tf) | GKE multitenant stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-networking.tf](./branch-networking.tf) | Networking stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-project-factory.tf](./branch-project-factory.tf) | Project factory stage resources. | <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-project-factory.tf](./branch-project-factory.tf) | Project factory stage resources. | <code>gcs</code> · <code>iam-service-account</code> | <code>google_organization_iam_member</code> |
|
||||
| [branch-sandbox.tf](./branch-sandbox.tf) | Sandbox stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-security.tf](./branch-security.tf) | Security stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
| [branch-teams.tf](./branch-teams.tf) | Team stage resources. | <code>folder</code> · <code>gcs</code> · <code>iam-service-account</code> | |
|
||||
|
|
@ -170,7 +174,7 @@ Due to its simplicity, this stage lends itself easily to customizations: adding
|
|||
| [cicd-project-factory.tf](./cicd-project-factory.tf) | CI/CD resources for the teams branch. | <code>iam-service-account</code> · <code>source-repository</code> | |
|
||||
| [cicd-security.tf](./cicd-security.tf) | CI/CD resources for the security branch. | <code>iam-service-account</code> · <code>source-repository</code> | |
|
||||
| [main.tf](./main.tf) | Module-level locals and resources. | | |
|
||||
| [organization.tf](./organization.tf) | Organization policies. | <code>organization</code> | <code>google_organization_iam_member</code> |
|
||||
| [organization.tf](./organization.tf) | Organization policies. | <code>organization</code> | |
|
||||
| [outputs-files.tf](./outputs-files.tf) | Output files persistence to local filesystem. | | <code>local_file</code> |
|
||||
| [outputs-gcs.tf](./outputs-gcs.tf) | Output files persistence to automation GCS bucket. | | <code>google_storage_bucket_object</code> |
|
||||
| [outputs.tf](./outputs.tf) | Module outputs. | | |
|
||||
|
|
@ -180,34 +184,34 @@ Due to its simplicity, this stage lends itself easily to customizations: adding
|
|||
|
||||
| name | description | type | required | default | producer |
|
||||
|---|---|:---:|:---:|:---:|:---:|
|
||||
| [automation](variables.tf#L20) | Automation resources created by the bootstrap stage. | <code title="object({ outputs_bucket = string project_id = string project_number = string federated_identity_pool = string federated_identity_providers = map(object({ issuer = string issuer_uri = string name = string principal_tpl = string principalset_tpl = string })) })">object({…})</code> | ✓ | | <code>00-bootstrap</code> |
|
||||
| [billing_account](variables.tf#L38) | Billing account id and organization id ('nnnnnnnn' or null). | <code title="object({ id = string organization_id = number })">object({…})</code> | ✓ | | <code>00-bootstrap</code> |
|
||||
| [organization](variables.tf#L197) | Organization details. | <code title="object({ domain = string id = number customer_id = string })">object({…})</code> | ✓ | | <code>00-bootstrap</code> |
|
||||
| [prefix](variables.tf#L221) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | <code>00-bootstrap</code> |
|
||||
| [cicd_repositories](variables.tf#L47) | CI/CD repository configuration. Identity providers reference keys in the `automation.federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | <code title="object({ data_platform_dev = object({ branch = string identity_provider = string name = string type = string }) data_platform_prod = object({ branch = string identity_provider = string name = string type = string }) gke_dev = object({ branch = string identity_provider = string name = string type = string }) gke_prod = object({ branch = string identity_provider = string name = string type = string }) networking = object({ branch = string identity_provider = string name = string type = string }) project_factory_dev = object({ branch = string identity_provider = string name = string type = string }) project_factory_prod = object({ branch = string identity_provider = string name = string type = string }) security = object({ branch = string identity_provider = string name = string type = string }) })">object({…})</code> | | <code>null</code> | |
|
||||
| [custom_roles](variables.tf#L129) | Custom roles defined at the org level, in key => id format. | <code title="object({ service_project_network_admin = string })">object({…})</code> | | <code>null</code> | <code>00-bootstrap</code> |
|
||||
| [data_dir](variables.tf#L138) | Relative path for the folder storing configuration data. | <code>string</code> | | <code>"data"</code> | |
|
||||
| [fast_features](variables.tf#L144) | Selective control for top-level FAST features. | <code title="object({ data_platform = bool gke = bool project_factory = bool sandbox = bool teams = bool })">object({…})</code> | | <code title="{ data_platform = true gke = true project_factory = true sandbox = true teams = true }">{…}</code> | <code>00-bootstrap</code> |
|
||||
| [groups](variables.tf#L164) | Group names to grant organization-level permissions. | <code>map(string)</code> | | <code title="{ gcp-billing-admins = "gcp-billing-admins", gcp-devops = "gcp-devops", gcp-network-admins = "gcp-network-admins" gcp-organization-admins = "gcp-organization-admins" gcp-security-admins = "gcp-security-admins" gcp-support = "gcp-support" }">{…}</code> | <code>00-bootstrap</code> |
|
||||
| [locations](variables.tf#L179) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object({ bq = string gcs = string logging = string pubsub = list(string) })">object({…})</code> | | <code title="{ bq = "EU" gcs = "EU" logging = "global" pubsub = [] }">{…}</code> | <code>00-bootstrap</code> |
|
||||
| [organization_policy_configs](variables.tf#L207) | Organization policies customization. | <code title="object({ allowed_policy_member_domains = list(string) })">object({…})</code> | | <code>null</code> | |
|
||||
| [outputs_location](variables.tf#L215) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | <code>string</code> | | <code>null</code> | |
|
||||
| [tag_names](variables.tf#L232) | Customized names for resource management tags. | <code title="object({ context = string environment = string })">object({…})</code> | | <code title="{ context = "context" environment = "environment" }">{…}</code> | |
|
||||
| [team_folders](variables.tf#L249) | Team folders to be created. Format is described in a code comment. | <code title="map(object({ descriptive_name = string group_iam = map(list(string)) impersonation_groups = list(string) }))">map(object({…}))</code> | | <code>null</code> | |
|
||||
| [automation](variables.tf#L20) | Automation resources created by the bootstrap stage. | <code title="object({ outputs_bucket = string project_id = string project_number = string federated_identity_pool = string federated_identity_providers = map(object({ issuer = string issuer_uri = string name = string principal_tpl = string principalset_tpl = string })) })">object({…})</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [billing_account](variables.tf#L38) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | <code title="object({ id = string is_org_level = optional(bool, true) })">object({…})</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [organization](variables.tf#L193) | Organization details. | <code title="object({ domain = string id = number customer_id = string })">object({…})</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [prefix](variables.tf#L217) | Prefix used for resources that need unique names. Use 9 characters or less. | <code>string</code> | ✓ | | <code>0-bootstrap</code> |
|
||||
| [cicd_repositories](variables.tf#L51) | CI/CD repository configuration. Identity providers reference keys in the `automation.federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | <code title="object({ data_platform_dev = optional(object({ branch = string identity_provider = string name = string type = string })) data_platform_prod = optional(object({ branch = string identity_provider = string name = string type = string })) gke_dev = optional(object({ branch = string identity_provider = string name = string type = string })) gke_prod = optional(object({ branch = string identity_provider = string name = string type = string })) networking = optional(object({ branch = string identity_provider = string name = string type = string })) project_factory_dev = optional(object({ branch = string identity_provider = string name = string type = string })) project_factory_prod = optional(object({ branch = string identity_provider = string name = string type = string })) security = optional(object({ branch = string identity_provider = string name = string type = string })) })">object({…})</code> | | <code>null</code> | |
|
||||
| [custom_roles](variables.tf#L133) | Custom roles defined at the org level, in key => id format. | <code title="object({ service_project_network_admin = string })">object({…})</code> | | <code>null</code> | <code>0-bootstrap</code> |
|
||||
| [data_dir](variables.tf#L142) | Relative path for the folder storing configuration data. | <code>string</code> | | <code>"data"</code> | |
|
||||
| [fast_features](variables.tf#L148) | Selective control for top-level FAST features. | <code title="object({ data_platform = optional(bool, false) gke = optional(bool, false) project_factory = optional(bool, false) sandbox = optional(bool, false) teams = optional(bool, false) })">object({…})</code> | | <code>{}</code> | <code>0-0-bootstrap</code> |
|
||||
| [groups](variables.tf#L162) | Group names to grant organization-level permissions. | <code title="object({ gcp-devops = optional(string) gcp-network-admins = optional(string) gcp-security-admins = optional(string) })">object({…})</code> | | <code>{}</code> | <code>0-bootstrap</code> |
|
||||
| [locations](variables.tf#L175) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object({ bq = string gcs = string logging = string pubsub = list(string) })">object({…})</code> | | <code title="{ bq = "EU" gcs = "EU" logging = "global" pubsub = [] }">{…}</code> | <code>0-bootstrap</code> |
|
||||
| [organization_policy_configs](variables.tf#L203) | Organization policies customization. | <code title="object({ allowed_policy_member_domains = list(string) })">object({…})</code> | | <code>null</code> | |
|
||||
| [outputs_location](variables.tf#L211) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | <code>string</code> | | <code>null</code> | |
|
||||
| [tag_names](variables.tf#L228) | Customized names for resource management tags. | <code title="object({ context = string environment = string tenant = string })">object({…})</code> | | <code title="{ context = "context" environment = "environment" tenant = "tenant" }">{…}</code> | |
|
||||
| [team_folders](variables.tf#L247) | Team folders to be created. Format is described in a code comment. | <code title="map(object({ descriptive_name = string group_iam = map(list(string)) impersonation_groups = list(string) }))">map(object({…}))</code> | | <code>null</code> | |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive | consumers |
|
||||
|---|---|:---:|---|
|
||||
| [cicd_repositories](outputs.tf#L197) | WIF configuration for CI/CD repositories. | | |
|
||||
| [dataplatform](outputs.tf#L211) | Data for the Data Platform stage. | | |
|
||||
| [gke_multitenant](outputs.tf#L227) | Data for the GKE multitenant stage. | | <code>03-gke-multitenant</code> |
|
||||
| [networking](outputs.tf#L248) | Data for the networking stage. | | |
|
||||
| [project_factories](outputs.tf#L257) | Data for the project factories stage. | | |
|
||||
| [providers](outputs.tf#L272) | Terraform provider files for this stage and dependent stages. | ✓ | <code>02-networking</code> · <code>02-security</code> · <code>03-dataplatform</code> · <code>xx-sandbox</code> · <code>xx-teams</code> |
|
||||
| [sandbox](outputs.tf#L279) | Data for the sandbox stage. | | <code>xx-sandbox</code> |
|
||||
| [security](outputs.tf#L293) | Data for the networking stage. | | <code>02-security</code> |
|
||||
| [teams](outputs.tf#L303) | Data for the teams stage. | | |
|
||||
| [tfvars](outputs.tf#L315) | Terraform variable files for the following stages. | ✓ | |
|
||||
| [cicd_repositories](outputs.tf#L210) | WIF configuration for CI/CD repositories. | | |
|
||||
| [dataplatform](outputs.tf#L224) | Data for the Data Platform stage. | | |
|
||||
| [gke_multitenant](outputs.tf#L240) | Data for the GKE multitenant stage. | | <code>03-gke-multitenant</code> |
|
||||
| [networking](outputs.tf#L261) | Data for the networking stage. | | |
|
||||
| [project_factories](outputs.tf#L270) | Data for the project factories stage. | | |
|
||||
| [providers](outputs.tf#L285) | Terraform provider files for this stage and dependent stages. | ✓ | <code>02-networking</code> · <code>02-security</code> · <code>03-dataplatform</code> · <code>xx-sandbox</code> · <code>xx-teams</code> |
|
||||
| [sandbox](outputs.tf#L292) | Data for the sandbox stage. | | <code>xx-sandbox</code> |
|
||||
| [security](outputs.tf#L306) | Data for the networking stage. | | <code>02-security</code> |
|
||||
| [teams](outputs.tf#L316) | Data for the teams stage. | | |
|
||||
| [tfvars](outputs.tf#L328) | Terraform variable files for the following stages. | ✓ | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -34,23 +34,11 @@ locals {
|
|||
|
||||
# billing account in same org (resources is in the organization.tf file)
|
||||
|
||||
# billing account in a different org
|
||||
|
||||
module "billing-organization-ext" {
|
||||
source = "../../../modules/organization"
|
||||
count = local.billing_org_ext ? 1 : 0
|
||||
organization_id = "organizations/${var.billing_account.organization_id}"
|
||||
iam_additive = {
|
||||
"roles/billing.user" = local.billing_ext_users
|
||||
"roles/billing.costsManager" = local.billing_ext_users
|
||||
}
|
||||
}
|
||||
|
||||
# standalone billing account
|
||||
|
||||
resource "google_billing_account_iam_member" "billing_ext_admin" {
|
||||
for_each = toset(
|
||||
local.billing_ext ? local.billing_ext_users : []
|
||||
!var.billing_account.is_org_level ? local.billing_ext_users : []
|
||||
)
|
||||
billing_account_id = var.billing_account.id
|
||||
role = "roles/billing.user"
|
||||
|
|
@ -59,7 +47,7 @@ resource "google_billing_account_iam_member" "billing_ext_admin" {
|
|||
|
||||
resource "google_billing_account_iam_member" "billing_ext_costsmanager" {
|
||||
for_each = toset(
|
||||
local.billing_ext ? local.billing_ext_users : []
|
||||
!var.billing_account.is_org_level ? local.billing_ext_users : []
|
||||
)
|
||||
billing_account_id = var.billing_account.id
|
||||
role = "roles/billing.costsManager"
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue