Commit Graph

65 Commits

Author SHA1 Message Date
Ludovico Magnocavallo 5453c585e0
FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052)
* rename stages

* remove support for external org billing, rename output files

* resman: make groups optional, align on new billing account variable

* bootstrap: multitenant outputs

* tenant bootstrap stage, untested

* fix folder name

* fix stage 0 output names

* optional creation for tag keys in organization module

* single tenant bootstrap minus tag

* rename output files, add tenant tag key

* fix organization module tag values output

* test skipping creation for tags in organization module

* single tenant bootstrap plan working

* multitenant bootstrap

* tfdoc

* fix check links error messages

* fix links

* tfdoc

* fix links

* rename fast tests, fix bootstrap tests

* multitenant stages have their own folder, simplify stage numbering

* stage renumbering

* wip

* rename tests

* exclude fast providers in fixture

* stage 0 tests

* stage 1 tests

* network stages tests

* stage tests

* tfdoc

* fix links

* tfdoc

* multitenant tests

* remove local files

* stage links command

* fix links script, TODO

* wip

* wip single tenant bootstrap

* working tenant bootstrap

* update gitignore

* remove local files

* tfdoc

* remove local files

* allow tests for tenant bootstrap stage

* tenant bootstrap proxies stage 1 tfvars

* stage 2 and 3 service accounts and IAM in tenant bootstrap

* wip

* wip

* wip

* drop multitenant bootstrap

* tfdoc

* add missing stage 2 SAs, fix org-level IAM condition

* wip

* wip

* optional tag value creation in organization module

* stage 1 working

* linting

* linting

* READMEs

* wip

* Make stage-links script work in old macos bash

* stage links command help

* fix output file names

* diagrams

* fix svg

* stage 0 skeleton and diagram

* test svg

* test svg

* test diagram

* diagram

* readme

* fix stage links script

* stage 0 readme

* README changes

* stage readmes

* fix outputs order

* fix link

* fix tests

* stage 1 test

* skip stage example

* boilerplate

* fix tftest skip

* default bootstrap stage log sinks to log buckets

* add logging to tenant bootstrap

* move iam variables out of tenant config

* fix cicd, reintroduce missing variable

* use optional in stage 1 cicd variable

* rename extras stage

* rename and move identity providers local, use optional for cicd variable

* tfdoc

* add support for wif pool and providers, ci/cd

* tfdoc

* fix links

* better handling of modules repository

* add missing role on logging project

* fix cicd pools in locals, test cicd

* fix workflow extension

* fix module source replacement

* allow tenant bootstrap cicd sa to impersonate resman sa

* tenant workflow templates fix for no providers file

* fix output files, push github workflow template to new repository

* remove try from outpout files

* align stage 1 cicd internals to stage 0

* tfdoc

* tests

* fix tests

* tests

* improve variable descriptions

* use optional in fast features

* actually create tenant log sinks, and allow the resman sa to do it

* test

* tests

* aaaand tests again

* fast features tenant override

* fast features tenant override

* fix wording

* add missing comment

* configure pf service accounts

* add missing comment

* tfdoc

* tests

* IAM docs

* update copyright

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-02-04 15:00:45 +01:00
Julio Castillo 44724f3839 Update plan_summary to support running documentation examples 2023-01-19 17:47:14 +01:00
Julio Castillo 30419ca170 Fix check_documentation output 2023-01-12 13:37:02 +01:00
Julio Castillo b840fcd74e Plan summary tool 2022-12-06 16:13:20 +01:00
Julio Castillo 27cf698ce5 Colon -> Period 2022-11-25 09:36:57 +01:00
Julio Castillo 612d7283e8 Enforce nonempty descriptions ending in a colon 2022-11-24 18:56:01 +01:00
Julio Castillo 882a3ebf78 Fix pubsub sorting 2022-11-21 13:47:24 +01:00
Julio Castillo cf39f68e9a Only look at outputs.tf in check documentation lint 2022-11-21 13:17:55 +01:00
Julio Castillo 297691565e Show different icons for documentation check failures 2022-11-21 13:17:55 +01:00
Julio Castillo e2e7bc282d Remote debug lib 2022-11-21 13:17:55 +01:00
Julio Castillo 826ff14ef9 Sort variables and outputs 2022-11-21 13:17:55 +01:00
Julio Castillo 2e700db8a7 Fix variable/output sort check 2022-11-21 13:17:55 +01:00
Julio Castillo a988775c14 Do not modify in parameters in format_* functions 2022-10-20 18:56:54 +02:00
Julio Castillo 5bd195aa8f Switch to namedtuple 2022-10-20 16:26:09 +02:00
Julio Castillo 8366933f4d Bring back sorted variables check 2022-10-20 16:15:41 +02:00
Ludovico Magnocavallo 162268a48b
Make changelog tool slower to work around inconsistencies in API results (#876)
* Make changelog tool slower to work around inconcistencies in API results

* add comment
2022-10-12 14:49:32 +02:00
Ludovico Magnocavallo 4a1465ae32
Comment redundant role in bootstrap stage, align IAM.md files, improve IAM tool (#842)
* comment redundant role

* account for duplicate folder names

* update IAM.md files
2022-09-29 08:30:01 +02:00
Ludovico Magnocavallo bbc94a9dec
fix changelog generator (#811) 2022-09-13 11:41:28 +02:00
Ludovico Magnocavallo 642f1d1366 changelog tool improvements 2022-08-04 13:27:42 +02:00
Ludovico Magnocavallo 0655f6843d fix group pulls 2022-08-02 14:17:37 +02:00
Ludovico Magnocavallo 0a58611792
Changelog generator (#763)
* first test

* test signing

* changelog load/dump

* changelog tool

* changelog generator

* lint
2022-08-02 11:45:05 +02:00
Ludovico Magnocavallo d075e67425 workflow permissions 2022-07-30 19:29:50 +02:00
Ludovico Magnocavallo 3776120ba0
first test (#762) 2022-07-30 19:04:00 +02:00
Ray Walker 35e5ffaf85
fix: Raise ValueError when check_names detects overlong names
ValueError may not be the correct type?

Closes #679
2022-06-16 10:39:05 +10:00
Taneli Leppä b28b379a7b Added a tool to update provider_meta/module_name, Terraform version and provider versions on batch basis.
Also includes a pipeline to manually release new versions and update all module names.
2022-06-09 12:40:51 +02:00
Ludovico Magnocavallo ff25809020
Fix tfdoc parsing on newllines in outputs (#649)
* fix

* use the right (ugly) Python formatter

* boilerplate on new file

* and do the actual tfdoc run on the original outputs file
2022-05-05 14:43:38 +02:00
Lorenzo Caggioni ca655265f1 Add python module 2022-04-22 00:31:28 +02:00
Ludovico Magnocavallo 725f7effce
Initial MVP for CI/CD (#608)
* preliminary support for wif in stage 0

* IAM wif role

* IAM wif role TODO

* add support for external SA IAM to SA module

* add name output to SA module

* separate cicd SA

* tfdoc

* GITLAB principal (untested)

* make GCS name output static

* outputs bucket

* fix stage 1 test

* tweak outputs

* tfdoc

* move wif_pool to automation variable

* add support for top-level and repository providers

* add missing boilerplate

* fix branchless principal

* initial workflow

* symlink provider template in stages

* remove service accounts from stage 0 cicd tfvars

* add cicd interface variable to resman stage

* fix cicd variable in resman stage

* better condition on outputs_location

* fix last change

* change outputs_location type

* revert outputs_location change

* split outputs in stage 0

* update ci/cd temporary notes

* rename additive IAM resource in SA module

* split outputs in stage 1

* remove unused locals

* fix stage 1 tests

* tfdoc

* Upload action files to outputs_bucket

* Fix tests and README

* rename template, streamline outputs

* local templates and gcs output for all stage 2

* add workflows to local output files

* Use lowercase WIF providers everywhere

* Bring back suffix for workflow files

* Remove unused files

* Update READMEs

* preliminary CI/CD implementation for stage 1

* fix stage 1

* stage 1 cicd

* tfdoc

* fix tests

* readme and links for cicd and wif

* refactor wif providers

* refactor cicd for stage 1

* fix stage 1

* wif org policies

* split identity provider configuration from cicd

* add type attribute to cicd repositories

* valid cicd repositories have a workflow template

* refactor stage 01

* fix stage 01 tests

* minimal CI/CD documentation

* better check_links error reporting

* fix links

* Added Gitlab specific configurations

Set the default issuer_uri for Gitlab. Added allowed audiences to OIDC configuration.

* Fixed TF formatting in identity providers.

* Changing identity provider audience to null

Changing identity provider audience to default to null.

* add instructions for renaming workflows

* address Julio's comments

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: alexmeissner <alexmeissner@google.com>
2022-04-12 08:17:27 +02:00
Lorenzo Caggioni 9011e370b8 Fix linting 2022-04-04 00:45:27 +02:00
Lorenzo Caggioni 6f00caaaab Handle Service Identity SA. Update IAM.md 2022-04-04 00:38:34 +02:00
Luca Prete 29b469133f Delete unused line 2022-03-28 23:14:16 +02:00
Luca Prete 4e8ef9d770 Simplify code 2022-03-28 20:36:51 +02:00
Luca Prete 3f3c7f9402 Providing default value in function arg for option external 2022-03-28 20:33:16 +02:00
Luca Prete 603732f7e8 Add option to test external links 2022-03-28 18:53:32 +02:00
Julio Castillo ba30ceb5c1
Simplify link checker and recurse into all elements with children (#599) 2022-03-28 16:54:06 +02:00
Luca Prete 0837d41bb1 Fix python syntax 2022-03-28 09:47:24 +02:00
Luca Prete 2bb9e3f64e [#597] Fix link_checker 2022-03-28 09:43:00 +02:00
John Tracy 22f03bd5d5
Update README for bigquery module (#569)
* Update README

Some variables that no longer exist. Table format is broken for view/table access.

* Remove floating role

* Fix README.md

* Fix tfdoc

* Fix tfdoc

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2022-03-23 16:34:45 +01:00
Simone Ruffilli fb90500adc
FAST: add 02-networking-peering stage. (#561)
* Networking with peering - initial commit

* README and tests

* Linter fixes

* Linter fixes

* Linter fixes

* Update README.md

* split out vpn/peering in separate files so the rest is identical

* fix dns for vpn and peering

* update tfdoc to support multiple outputs/variables files

* add peering variables

* update tfdoc for modules

* make it easier to spot errored docs

* fix doc

* yapf

* fix permadiff in firewall module source ranges

* FAST: Networking: DNS Fixes

* FAST: ability to toggle on-prem vpn creation

* FAST: fix outputs

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2022-02-28 12:27:27 +01:00
Ludovico Magnocavallo 38518ec7d5 skip links in tfdoc file list 2022-02-27 10:36:03 +01:00
Julio Castillo e7dd12fa29 Reformat all code using yapf 2022-02-14 12:27:42 +01:00
Ludovico Magnocavallo fbf4d29c88
Fix FAST names (#539)
* name checker, initial names fix

* fix name parser

* sdbx -> sbox

* explicit names

* show total name length

* check names in linting action

* validate prefix length

* fix typo
2022-02-12 13:29:22 +01:00
Ludovico Magnocavallo c8cbf9b7d7
Add utility to extract IAM bindings from state (#503)
* add utility to extract IAM bindings from state

* TODO

* TODO

* output one row per role/member pairs

* sorting

* rename, delete spurious file

* generalize output functions

* generate IAM.md

* fix org id

* add link to role in markdown

* fix org id

* replace organization ids
2022-02-05 17:34:43 +01:00
Ludovico Magnocavallo 02e8a3927d
CI link checker (#494)
* CI link checker

* fix link errors

* fix ci

* wildcard on *.md
2022-02-04 13:28:07 +01:00
Julio Castillo b11367e970 Support nullable variables in tfdoc 2022-01-28 19:09:40 +01:00
Ludovico Magnocavallo cb9340fd4f exclude in check docs 2022-01-27 17:31:08 +01:00
Julio Castillo 31f625f149 Link vars and outputs from README 2022-01-22 14:15:16 +01:00
Ludovico Magnocavallo 9a533180a0
Refactor project module, support per-file tags in tfdoc (#450)
* add support for in-doc tfdoc options overrides

* clean up project module

* add file description tags

* only output module and resource columns in tfdoc file table if they exist

* update fast READMEs

* fix check docs
2022-01-22 11:34:18 +01:00
Ludovico Magnocavallo 56877421ff
fix etra newlines in tfdoc, fixes #445 (#446) 2022-01-21 16:16:28 +01:00
Ludovico Magnocavallo e866db2c88 Merge Fabric FAST (#435)
Co-authored-by: Simone Ruffilli <sruffilli@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2022-01-19 14:19:39 +01:00