zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #558 from GoogleCloudPlatform/sruffilli-readme-fixes 

FAST - README.md - commands harmonization
This commit is contained in:
Simone Ruffilli 2022-02-24 12:24:52 +01:00 committed by GitHub
parent b5b58f3d4b 3f4b9fac9a
commit 36cca5946a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
8 changed files with 61 additions and 38 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
fast/stages/00-bootstrap/README.md
View File

@ -189,7 +189,9 @@ organization={
}
# create your own 4-letters prefix
prefix="fast"
outputs_location = "../../fast-config"
# comment out if you want to leverage automatic generation of configs
# outputs_location = "~/fast-config"
```
### Output files and cross-stage variables
@ -199,9 +201,11 @@ At any time during the life of this stage, you can configure it to automatically
Automatic generation of files is disabled by default. To enable the mechanism, set the `outputs_location` variable to a valid path on a local filesystem, e.g.
```hcl
outputs_location = "../../config"
outputs_location = "~/fast-config"
```
This is especially suited for initial bootstrapping and development. You might want to adapt it to your practices for production deployments.
Once the variable is set, `apply` will generate and manage providers and variables files, including the initial one used for this stage after the first run. You can then link these files in the relevant stages, instead of manually transfering outputs from one stage, to Terraform variables in another.
Below is the outline of the output files generated by all stages:
@ -238,8 +242,8 @@ terraform apply \
Once the initial `apply` completes successfully, configure a remote backend using the new GCS bucket, and impersonation on the automation service account for this stage. To do this you can use the generated `providers.tf` file if you have configured output files as described above, or extract its contents from Terraform's output, then migrate state with `terraform init`:
```bash
# if using output files via the outputs_location and set to `../../config`
ln -s ../../config/providers/00-bootstrap* ./
# if using output files via the outputs_location and set to `~/fast-config`
ln -s ~/fast-config/providers/00-bootstrap* ./
# or from outputs if not using output files
terraform output -json providers | jq -r '.["00-bootstrap"]' \
> providers.tf

12
fast/stages/01-resman/README.md
View File

@ -53,8 +53,8 @@ To simplify setup, the previous stage pre-configures a valid providers file in i
If you have set a valid value for `outputs_location` in the bootstrap stage (see the [bootstrap stage README](../00-bootstrap/#output-files-and-cross-stage-variables) for more details), simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
```bash
# `outputs_location` is set to `~/config`
ln -s ~/config/providers/01-resman* ./
# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/providers/01-resman-providers.tf .
```
If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage's outputs:
@ -76,13 +76,13 @@ There are two broad sets of variables you will need to fill in:
To avoid the tedious job of filling in the first group of variable with values derived from other stages' outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files.
If you configured a valid path for `outputs_location` in the bootstrap stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's outputs folder. For this stage, you need the `.tfvars` file compiled manually for the bootstrap stage, and the one generated by it:
If you configured a valid path for `outputs_location` in the bootstrap stage, simply link the relevant `terraform-*.auto.tfvars.json` files from the outputs folder. For this stage, you need the `.tfvars` file compiled manually for the bootstrap stage, and the one generated by it:
```bash
# `outputs_location` is set to `~/config`
ln -s ../../config/tfvars/00*.json ./
# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/tfvars/00-bootstrap.auto.tfvars.json .
# also copy the tfvars file used for the bootstrap stage
cp ../00-bootstrap/terraform.tfvars ./
cp ../00-bootstrap/terraform.tfvars .
```
A second set of variables is specific to this stage, they are all optional so if you need to customize them, create an extra `terraform.tfvars` file or add them to the file copied from bootstrap.

12
fast/stages/02-networking-nva/README.md
View File

@ -200,14 +200,14 @@ To simplify the setup, the previous stage pre-configures a valid providers file
If you have set a valid value for `outputs_location` in the bootstrap stage, simply link the relevant `providers.tf` file from this stage folder in the path you selected:
```bash
# `outputs_location` is set to `../../configs/example`
ln -s ../../configs/example/02-networking/providers.tf
# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/providers/02-networking-providers.tf .
```
If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage outputs:
```bash
cd ../00-bootstrap
cd ../01-resman
terraform output -json providers | jq -r '.["02-networking"]' \
> ../02-networking-nva/providers.tf
```
@ -224,12 +224,14 @@ To avoid the tedious job of filling in the first group of variables with values
If you have set a valid value for `outputs_location` in the bootstrap and in the resman stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's folder in the path you specified, where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is available:
```bash
# `outputs_location` is set to `../../configs/example`
# `outputs_location` is set to `~/fast-config`
ln -s ../../configs/example/02-networking/terraform-bootstrap.auto.tfvars.json
ln -s ../../configs/example/02-networking/terraform-resman.auto.tfvars.json
# also copy the tfvars file used for the bootstrap stage
cp ../00-bootstrap/terraform.tfvars .
```
If you want to continue to rely on `outputs_location` logic, create a `terraform.tfvars` file and configure it as described [here](../00-bootstrap/#output-files-and-cross-stage-variables).
A second set of variables is specific to this stage, they are all optional so if you need to customize them, add them to the file copied from bootstrap.
Please, refer to the [variables](#variables) table below for a map of the variable origins, and use the sections below to understand how to adapt this stage to your networking configuration.

6
fast/stages/02-networking-nva/main.tf
View File

@ -27,6 +27,12 @@ locals {
service_accounts = {
for k, v in coalesce(var.service_accounts, {}) : k => "serviceAccount:${v}"
}
stage3_sas_delegated_grants = [
"roles/composer.sharedVpcAgent",
"roles/compute.networkUser",
"roles/container.hostServiceAgentUser",
"roles/vpcaccess.user",
]
}
module "folder" {

16
fast/stages/02-networking-vpn/README.md
View File

@ -136,14 +136,14 @@ To simplify setup, the previous stage pre-configures a valid providers file in i
If you have set a valid value for `outputs_location` in the bootstrap stage, simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
```bash
# `outputs_location` is set to `../../config`
ln -s ../../config/02-networking/providers.tf
# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/providers/02-networking-providers.tf .
```
If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage's outputs:
```bash
cd ../00-bootstrap
cd ../01-resman
terraform output -json providers | jq -r '.["02-networking"]' \
> ../02-networking/providers.tf
```
@ -160,12 +160,14 @@ To avoid the tedious job of filling in the first group of variables with values
If you have set a valid value for `outputs_location` in the bootstrap and in the resman stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's folder in the path you specified, where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is available:
```bash
# `outputs_location` is set to `../../config`
ln -s ../../config/02-networking/terraform-bootstrap.auto.tfvars.json
ln -s ../../config/02-networking/terraform-resman.auto.tfvars.json
# `outputs_location` is set to `~/fast-config`
ln -s ../../configs/example/02-networking/terraform-bootstrap.auto.tfvars.json
ln -s ../../configs/example/02-networking/terraform-resman.auto.tfvars.json
# also copy the tfvars file used for the bootstrap stage
cp ../00-bootstrap/terraform.tfvars .
```
If you want to continue to rely on `outputs_location` logic, create a `terraform.tfvars` file and configure it as deacribed [here](../00-bootstrap/#output-files-and-cross-stage-variables).
A second set of variables is specific to this stage, they are all optional so if you need to customize them, add them to the file copied from bootstrap.
Please refer to the [Variables](#variables) table below for a map of the variable origins, and to the sections below on how to adapt this stage to your networking configuration.

16
fast/stages/02-security/README.md
View File

@ -57,8 +57,8 @@ To simplify setup, the previous stage pre-configures a valid providers file in i
If you have set a valid value for `outputs_location` in the resource management stage, simply link the relevant `providers.tf` file from this stage's folder in the path you specified:
```bash
# `outputs_location` is set to `../../config`
ln -s ../../config/02-security/providers.tf
# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/providers/02-security-providers.tf .
```
If you have not configured `outputs_location` in resource management, you can derive the providers file from that stage's outputs:
@ -69,8 +69,6 @@ terraform output -json providers | jq -r '.["02-security"]' \
> ../02-security/providers.tf
```
If you want to continue to rely on `outputs_location` logic, create a `terraform.tfvars` file and configure it as deacribed [here](../00-bootstrap/#output-files-and-cross-stage-variables).
### Variable configuration
There are two broad sets of variables you will need to fill in:
@ -83,12 +81,14 @@ To avoid the tedious job of filling in the first group of variables with values
If you configured a valid path for `outputs_location` in the previous stages, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's output folder (under the path you specified), where the `*` above is set to the name of the stage that produced it. For this stage, two `.tfvars` files are available:
```bash
# `outputs_location` is set to `../../config`
ln -s ../../config/02-security/terraform-bootstrap.auto.tfvars.json
ln -s ../../config/02-security/terraform-resman.auto.tfvars.json
# `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/tfvars/00-bootstrap.auto.tfvars.json .
ln -s ~/fast-config/tfvars/01-resman.auto.tfvars.json .
# also copy the tfvars file used for the bootstrap stage
cp ../00-bootstrap/terraform.tfvars .
```
A second set of optional variables is specific to this stage. If you need to customize them, create an extra `terraform.tfvars` file.
A second set of optional variables is specific to this stage. If you need to customize them add them to the file copied from bootstrap.
Refer to the [Variables](#variables) table at the bottom of this document, for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning. The sections below also describe some of the possible customizations.

14
fast/stages/03-data-platform/dev/README.md
View File

@ -51,6 +51,7 @@ Cloud KMS crypto keys can be configured wither from the [FAST security stage](..
To configure the use of Cloud KMS on resources, you have to specify the key id on the `service_encryption_keys` variable. Key locations should match resource locations.
### VPC-SC
As is often the case in real-world configurations, [VPC-SC](https://cloud.google.com/vpc-service-controls) is needed to mitigate data exfiltration. VPC-SC can be configured from the [FAST security stage](../../02-security). This step is optional, but highly recomended, and depends on customer policies and security best practices.
To configure the use of VPC-SC on the data platform, you have to specify the data platform project numbers on the `vpc_sc_perimeter_projects.dev` variable on [FAST security stage](../../02-security#perimeter-resources).
@ -81,7 +82,12 @@ The VPC host project, VPC and subnets should already exist.
### Providers configuration
If you're running this as part of a full FAST flow and using output files, the providers configuration with the right bucket and impersionation account is already available, and will be linked in via the same command use for variables and described in the next section.
If you're running this on top of Fast, you should run the following commands to create the providers file, and populate the required variables from the previous stage.
```bash
# Variable `outputs_location` is set to `~/fast-config` in stage 01-resman
ln -s ~/fast-config/providers/03-data-platform-dev-providers.tf .
```
### Variable configuration
@ -95,8 +101,10 @@ To avoid the tedious job of filling in the first group of variables with values
If you configured a valid path for `outputs_location` in the bootstrap security and networking stages, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's outputs folder under the path you specified. This will also link the providers configuration file:
```bash
# variable `outputs_location` set to `../../../config`
ln -s ../../../config/03-data-platform-prod/* ./
# Variable `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/tfvars/00-bootstrap.auto.tfvars.json .
ln -s ~/fast-config/tfvars/01-resman.auto.tfvars.json .
ln -s ~/fast-config/tfvars/02-networking.auto.tfvars.json .
```
If you're not using FAST or its output files, refer to the [Variables](#variables) table at the bottom of this document for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning.

11
fast/stages/03-project-factory/dev/README.md
View File

@ -56,9 +56,9 @@ It's of course possible to run this stage in isolation, by making sure the archi
If you're running this on top of Fast, you should run the following commands to create the providers file, and populate the required variables from the previous stage.
```bash
# Variable `outputs_location` is set to `../../../config` in stage 01-resman
# Variable `outputs_location` is set to `~/fast-config` in stage 01-resman
$ cd fabric-fast/stages/03-project-factory/dev
ln -s ../../../config/03-project-factory-dev/providers.tf
ln -s ~/fast-config/providers/03-project-factory-dev-providers.tf .
```
### Variable configuration
@ -73,9 +73,10 @@ To avoid the tedious job of filling in the first group of variables with values
If you configured a valid path for `outputs_location` in the bootstrap and networking stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's outputs folder (under the path you specified), where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is available:
```bash
# Variable `outputs_location` is set to `../../../config` in stages 01-bootstrap and the 02-networking stage in use
ln -s ../../../config/03-project-factory-dev/terraform-bootstrap.auto.tfvars.json
ln -s ../../../config/03-project-factory-dev/terraform-networking.auto.tfvars.json
# Variable `outputs_location` is set to `~/fast-config`
ln -s ~/fast-config/tfvars/00-bootstrap.auto.tfvars.json .
ln -s ~/fast-config/tfvars/01-resman.auto.tfvars.json .
ln -s ~/fast-config/tfvars/02-networking.auto.tfvars.json .
```
If you're not using Fast, refer to the [Variables](#variables) table at the bottom of this document for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning.