Commit Graph

499 Commits

Author SHA1 Message Date
Ludovico Magnocavallo 3d41d01efc
FAST plugin system (#1266)
* plugin folder, gitignore, serverless connector example

* add support to fast plugin variables and outputs to tfdoc

* rename folder, READMEs

* add variable description

* show diffs

* check documentation, use multiple files

* debug check doc

* try a different glob

* debug tfdoc names

* more debug

* and even more debug

* fix gitignore

* fix links

* support extra files in tests

* fix fixture, switch stage 2 peering to new tests

* tfdoc

* Allow globs in extra files

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-03-24 12:28:32 +00:00
simonebruzzechesse c4c4688adc
Small fixes to FAST Networking stage with NVAs (#1273)
* fix issue with test-resources and internet connectivity from spokes

* terraform fmt

* removed reference to startup-script in README.md
2023-03-23 09:57:01 +01:00
Ludovico Magnocavallo 5edc931bf9
add missing secret to spoke tunnels (#1265) 2023-03-17 20:52:40 +01:00
Ludovico Magnocavallo 5fb17cb3ac
Widen scope for prod project factory SA to dev (#1263)
* restrict storage role on outputs bucket for stage SAs

* grant prod project factory SA authority over prod and dev org policies

* network stages delegated grants on dev to prod pf SA

* security grants to prod pf SA on dev

* tfdoc

* tests
2023-03-17 16:24:55 +00:00
Ludo 367f4b6670
remove debug output 2023-03-17 15:35:18 +01:00
Anton KOVACH 5d8cbd3c57
Merge branch 'master' into feature/fast-cicd-github-enable-populating-of-data-directory-sample-files-and-update-dependencies 2023-03-15 11:57:21 +01:00
Ludovico Magnocavallo 2794cb6f24
Fix #1139 (#1249) 2023-03-15 11:43:43 +01:00
Anton KOVACH 1355ee4c44 Refactor to avoid explicit dependencies 2023-03-15 10:07:09 +01:00
Ludovico Magnocavallo 892b5b3446
Merge branch 'master' into feature/fast-cicd-github-enable-populating-of-data-directory-sample-files-and-update-dependencies 2023-03-14 19:25:24 +01:00
Natalia Strelkova fe7725e7d0 formatting 2023-03-14 14:48:04 +00:00
Natalia Strelkova 8bf3e11f34
location and storage class added to GKE GCS buckets 2023-03-14 15:43:55 +01:00
Anton KOVACH e344dbc4f4 Add populate_samples attribute 2023-03-13 20:29:50 +01:00
Ludovico Magnocavallo bffd5bc17b
Merge branch 'master' into feature/fast-cicd-github-enable-populating-of-data-directory-sample-files-and-update-dependencies 2023-03-13 16:01:52 +01:00
Ludovico Magnocavallo 112d9a8d9c
Allow using existing boot disk in compute-vm module (#1241)
* allow using existing boot disk in compute-vm module

* allow setting initialize params to null

* tests

* fast

* blueprints
2023-03-12 10:53:59 +01:00
Anton KOVACH 7a53511c9a Enable populating of data directory and .sample files and update dependencies
The Readme.md files reference the data directory and .sample files, but the code did not allow for their populating. This update enables the copying of the data directory and .sample files, with the data directory being populating as a data.sample directory to prevent overwriting any existing data directory.

Additionally, dependencies have been updated by adding the depends_on section to several resources to ensure that the dependencies are in the correct order. This update addresses some states that were not being handled previously.

There is a minor known issue with Pull Request creation in the current state of the code. The Pull Request is only created after the first run has occurred. A fix for this issue is currently being worked on and will be addressed in a separate Pull Request. However, this issue does not affect the main functionality of the code.
2023-03-11 15:27:41 +01:00
Ludovico Magnocavallo 6e70b4216f
add missing attribute to FAST onprem VPN examples (#1237) 2023-03-10 14:58:33 +00:00
Ludovico Magnocavallo be06554bba
Simplify VPN implementation in FAST networking stages (#1228)
* peering stage

* fix link, toc

* vpn stage

* fix link

* nva stage

* fix examples and test

* separate envs stage

* tfdoc
2023-03-09 17:57:44 +01:00
Julio Castillo 38808b37c0 Manage billing.creator role authoritatively in FAST bootstrap.
By default new orgs grant billing.creator and
resourcemanager.projectCreator to the whole domain[1]. This PR makes
FAST remove the former binding during the bootstrap (the latter is
already managed by FAST).

Fixes #1220

[1] https://cloud.google.com/resource-manager/docs/default-access-control
2023-03-07 17:52:00 +01:00
Natalia Strelkova 1f8e4cf1bf
FAQ on installing Fast on a non-empty org 2023-03-07 15:45:38 +01:00
Anton KOVACH 77db9121f9
feat: Add Pull Request support to 0-cicd-github (#1213)
* feat: Add Pull Request support to 0-cicd-github

The cloud-foundation-fabricrepository is continually evolving, and to help keep up with the changes, it would be beneficial to introduce a pull request mechanism to review and approve changes. This feature is 100% backward compatible, and by default, no pull request is created, and changes are committed directly to the main branch. However, an optional variable pull_request_config can be used to configure the title, body, head_ref, and base_ref of the pull request that will be created for the initial population or update of files. To create a pull request, in pull_request_config set the create attribute to true. base_ref defaults to main, and head_ref to the name of the head branch. If the head branch doesn't exist, it will be created from the base_ref branch.

* fix README.md

* fix pull_request_config title
2023-03-06 09:32:36 +01:00
Justin M 4eff309685
Update subnet sample yaml files to use subnet_secondary_ranges (#1203)
* Replaces 'secondary_ip_range:' with 'secondary_ip_ranges:' in samples

* Replaces 'secondary_ip_range:' with 'secondary_ip_ranges:' in tests/

* reverts previous commit- files in tests/ don't need to be changed

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-03-05 19:37:23 +01:00
Anton KOVACH e72ddb6a2a
feat: Add option to skip committing unchanged files in 0-cicd-github (#1212)
When running 0-cicd-github multiple times, files that haven't changed are also committed. This change adds an option to skip committing unchanged files to prevent unnecessary commits.

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-03-05 19:16:48 +01:00
Ludovico Magnocavallo 8fc9549c58
add support for proxy and psc subnets to module factory (#1211) 2023-03-05 17:08:43 +01:00
Ludovico Magnocavallo 96e829bdf3
Billing exclusion support for FAST mt resman (#1209)
* fix files resource parsing in tfdoc

* fix tfdoc generated output

* billing exclusion support in mt bootstrap
2023-03-03 16:23:36 +00:00
Ludovico Magnocavallo 2217abe5f0
Allow preventing creation of billing IAM roles in FAST, add instructions on delayed billing association (#1207)
* stage 0

* resman and networking stages

* tfdoc

* security stage
2023-03-03 09:24:41 +01:00
Ludovico Magnocavallo 6320c53baf
Allow multiple peer gateways in vpn ha module (#1184)
* allow multiple peer gateways in vpn ha module

* align blueprints

* fast
2023-02-27 10:18:59 +00:00
lcaggio 47855cb682
Merge branch 'master' into lcaggio/dp-projectcreate 2023-02-23 11:54:48 +01:00
Wiktor Niesiobędzki ad0840656b Add documentation about referring modules stored on CSR 2023-02-22 10:02:54 +01:00
lcaggio ac75cbe71a Fix lint. 2023-02-22 01:38:44 +01:00
lcaggio 2108b4650d Fix Tests, rely on iam additive. 2023-02-22 01:36:01 +01:00
Julio Castillo a5e905cb80 Update remaining org policies 2023-02-21 15:49:16 +01:00
Julio Castillo d3bcf625f9 Update yaml org policies 2023-02-21 15:49:16 +01:00
lcaggio 2564c9b06a Fix README 2023-02-20 01:17:08 +01:00
lcaggio 970b8ff255 Fix DP Fast variables. 2023-02-20 01:16:22 +01:00
lcaggio 63a81a9b9b Fix Fast test 2023-02-20 01:12:19 +01:00
lcaggio f4c1fa6c20 Fix tests. 2023-02-20 00:56:32 +01:00
Anton KOVACH 5905903d6e
fix module_prefix (#1164)
Add module_prefix to locals
2023-02-19 18:22:42 +00:00
Anton KOVACH 5cf60cbcf4
Fix Terraform formatting and add module_prefix attribute to modules_config (#1162)
* Fix Terraform formatting and add module/ prefix to path in 0-cicd-github repository population

fix the formatting of Terraform files and adds the module/ prefix to the module path in 0-cicd-github under repository population. Without proper formatting and module path, generated repositories may show formatting mismatches and examples in the README.md file may not run as expected.
The changes include updating the replace function with a new regular expression pattern to correctly apply the git source for modules and updating the each.value.file attribute to include the module/ prefix in the Terraform file path. This ensures that the examples in the README.md file work as intended and that the generated repositories follow best practices for Terraform code.

* revert modules/ prefix change

* Add module_prefix to modules_config

- Add module_prefix to modules_config
- Add example to Readme.md
- use module_prefix variable to specify the path

* fix tfdoc
2023-02-19 18:01:38 +01:00
Ludovico Magnocavallo 36a7347744
FAST stage docs cleanup (#1145)
* top-level and stage 0

* stage 1

* net peering

* networking

* networking

* security

* gke, dp

* checks
2023-02-15 05:42:14 +00:00
Julio Castillo 742b5bab62 Fix tfvars sample for fast bootstrap stage 2023-02-14 11:29:19 +02:00
Ludovico Magnocavallo 8708f490ce
Allow configuring regions from tfvars in FAST networking stages (#1137)
* configurable regions

* vpn, tests

* tfdoc

* separate envs

* nva

* test resources

* add new custom role for tenant network service accounts

* allow setting firewall policy name in networking stages

* fix stage links script

* set custom role to tenant networking service account

* rename tenant stage 1 provider files

* remove extra file

* fix peering and vpn

* tfdoc

* fix variable order

* tests
2023-02-08 09:59:43 +01:00
simonebruzzechesse 779c635682
Merged old bgp_peer_config parameter into bgp_peer aligning with newer version of tunnels variable available in the net-vpn-ha module (#1133)
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-02-07 22:58:28 +00:00
Ludovico Magnocavallo 2471e25c2c
post PR message on init or validate failure (#1135) 2023-02-07 10:04:03 +01:00
Ludovico Magnocavallo 9b8de3e415
fix stage links, fix stage 1 output file names (#1134) 2023-02-06 20:51:26 +01:00
Natalia Strelkova 803c9c3163 Merge branch 'master' into fast-networking-nva-no-peering-files 2023-02-04 16:12:35 +01:00
Ludovico Magnocavallo 5453c585e0
FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052)
* rename stages

* remove support for external org billing, rename output files

* resman: make groups optional, align on new billing account variable

* bootstrap: multitenant outputs

* tenant bootstrap stage, untested

* fix folder name

* fix stage 0 output names

* optional creation for tag keys in organization module

* single tenant bootstrap minus tag

* rename output files, add tenant tag key

* fix organization module tag values output

* test skipping creation for tags in organization module

* single tenant bootstrap plan working

* multitenant bootstrap

* tfdoc

* fix check links error messages

* fix links

* tfdoc

* fix links

* rename fast tests, fix bootstrap tests

* multitenant stages have their own folder, simplify stage numbering

* stage renumbering

* wip

* rename tests

* exclude fast providers in fixture

* stage 0 tests

* stage 1 tests

* network stages tests

* stage tests

* tfdoc

* fix links

* tfdoc

* multitenant tests

* remove local files

* stage links command

* fix links script, TODO

* wip

* wip single tenant bootstrap

* working tenant bootstrap

* update gitignore

* remove local files

* tfdoc

* remove local files

* allow tests for tenant bootstrap stage

* tenant bootstrap proxies stage 1 tfvars

* stage 2 and 3 service accounts and IAM in tenant bootstrap

* wip

* wip

* wip

* drop multitenant bootstrap

* tfdoc

* add missing stage 2 SAs, fix org-level IAM condition

* wip

* wip

* optional tag value creation in organization module

* stage 1 working

* linting

* linting

* READMEs

* wip

* Make stage-links script work in old macos bash

* stage links command help

* fix output file names

* diagrams

* fix svg

* stage 0 skeleton and diagram

* test svg

* test svg

* test diagram

* diagram

* readme

* fix stage links script

* stage 0 readme

* README changes

* stage readmes

* fix outputs order

* fix link

* fix tests

* stage 1 test

* skip stage example

* boilerplate

* fix tftest skip

* default bootstrap stage log sinks to log buckets

* add logging to tenant bootstrap

* move iam variables out of tenant config

* fix cicd, reintroduce missing variable

* use optional in stage 1 cicd variable

* rename extras stage

* rename and move identity providers local, use optional for cicd variable

* tfdoc

* add support for wif pool and providers, ci/cd

* tfdoc

* fix links

* better handling of modules repository

* add missing role on logging project

* fix cicd pools in locals, test cicd

* fix workflow extension

* fix module source replacement

* allow tenant bootstrap cicd sa to impersonate resman sa

* tenant workflow templates fix for no providers file

* fix output files, push github workflow template to new repository

* remove try from outpout files

* align stage 1 cicd internals to stage 0

* tfdoc

* tests

* fix tests

* tests

* improve variable descriptions

* use optional in fast features

* actually create tenant log sinks, and allow the resman sa to do it

* test

* tests

* aaaand tests again

* fast features tenant override

* fast features tenant override

* fix wording

* add missing comment

* configure pf service accounts

* add missing comment

* tfdoc

* tests

* IAM docs

* update copyright

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-02-04 15:00:45 +01:00
Natalia Strelkova 2958063625
Remove info about non-existing vpc-peering-*.tf files 2023-02-03 15:39:00 +01:00
Ludovico Magnocavallo 7b96ed429c
add missing role for initial user (#1118) 2023-01-28 08:41:22 +00:00
Ludo 83a0916bff
add missing newline 2023-01-28 09:27:31 +01:00
Ludovico Magnocavallo c1d3736b06
fix destroy in stage 1 outputs (#1099) 2023-01-19 09:35:40 +00:00