Commit Graph

115 Commits

Author SHA1 Message Date
Wiktor Niesiobędzki 1d76bfc3ff Add E2E tests for Cloud Functions and fix perma-diff 2024-07-08 16:14:21 +02:00
Wiktor Niesiobędzki 1f40f2e573 Add support for different endpoint types for Cloud NAT 2024-06-14 15:37:58 +02:00
Wiktor Niesiobędzki 0160392184 use /bin/sh for pre-commit script for portability 2024-06-07 08:15:33 +02:00
Alberto Geniola 61737c48a3
Change shebang on pre-commit checks (#2343)
Co-authored-by: Alberto Geniola <geniola@google.com>
2024-06-05 19:55:08 +02:00
Wiktor Niesiobędzki 49c1d60f67
Add outstanding checks from lint.sh to pre-commit (#2327)
* versions
* validate metadata
* check names
* check links
2024-05-31 10:22:21 +00:00
Wiktor Niesiobędzki f8f18734f1
Add pre-commit hook configuration (#2326)
* Pre-commit config

Run following linters on commit:
Terraform:
- terraform fmt
- terraform tflint

Python specific:
- yapf

Shell scripts
- shellcheck
- shfmt

YAML files:
- yamllint (disabled as of now)
- check-yaml

Other:
- end-of-file-fixer
- trailing-whitespace fixer

Fabric specific
- tools/tfdoc.py
- tools/check_boilerplate.py

* linting fixes

* Fix boilerplate check
2024-05-30 19:35:09 +02:00
Simone Ruffilli 9e1008dcca
FAST: Obsolete assets cleanup (#2315)
Remove obsolete assets
2024-05-28 09:35:13 +00:00
Wiktor Niesiobędzki cf31d83b15 Explicit YAPF style 2024-05-28 10:53:13 +02:00
Simone Ruffilli 3fe754167e FAST: Clarify max prefix len for tenant factory 2024-05-23 09:43:23 +02:00
simonebruzzechesse 1e149c18fc
New alloydb module (#2285)
* add alloydb module

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2024-05-22 09:40:25 +02:00
Ludovico Magnocavallo 7a5dd4e6db
FAST: add top-level folders and restructure teams/tenants in resman (#2254)
* remove teams and tenants from resman

* move fast features to stage 1, fix test inventories

* folders

* fix factory, add top level folder resources to outputs

* tfdoc

* stage 0 log sink defs

* tfdoc

* enable toc in resman readme

* simple tenants

* fast compatibility automation and logging

* testing fast-compatible tenants

* testing fast-compatible tenants

* tfdoc

* remove mt stages

* remove tests, fix links

* disable tflint

* fast tests

* make organization conditional in resman

* check names tool

* export real prefix to tfvars, prevent destroy errors

* prefix validation

* fix billing account export format

* tfdoc

* root node folder

* resman changes

* tenant resman roles

* first apply of tenant resman

* tenant log sinks in stage 1

* fix test vars

* tfdoc

* tenant vpc-sc access policy

* fix tests expected values

* tenant CI/CD

* identity providers

* wif

* tfdoc

* add comments to identity locals

* full-feature tenant resman apply

* tenant billing IAM

* stage test

* fix CI/CD comments

* tenant net stage verified

* tenant sec stage verified

* fix test

* README work

* tfdoc

* README

* README rewording

* README rewording

* tfdoc

* FAST excalidraw

* review comments

* diagram review changes

* add iam log sink for tenants

* remove redundant try from security stage

* Implement tflint-fast in Python driven by tftest.yaml files

* tflint

* test ci changes

* revert linting changes

* disable tflint for fast

* Create junit-style report for FAST tflint

* Remove junit-reporter

* YAPF tflint-fast.py

* Output tflint FAST to job summary

* Step summary

* Disable step_summary as output is not useful

* ignore tflint warning

* re-enable tflint on FAST

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-05-15 09:17:13 +00:00
Ludovico Magnocavallo fbc7e891db
Net LB App Internal Cross-Region recipe (#2214)
* reorder tfdoc methods

* add support for recipes to tfdoc

* fix repo url in tfdoc

* update module README

* validated untested recipe

* validated untested refactored recipe

* add optional proxy subnet creation, outputs, fixes

* tested

* tfdoc fix

* fix README

* exclude examples from test collector
2024-04-14 19:38:05 +03:00
Wiktor Niesiobędzki bc5f4c3969 Use explicit UTF-8 encoding 2024-04-02 20:34:51 +02:00
Ludovico Magnocavallo ecf5400fe3
update FAST state IAM files (#2136) 2024-03-07 00:08:09 +01:00
Mohamed Ikbal Belbahri ae6ee03de2
Add https security to cloud-functions-v1 module (#2118)
* Add https security level to cf v1 module
* Update tfdoc.py raw strings for slug var to avoid [SyntaxWarning: invalid escape sequence]
2024-02-28 21:20:56 +01:00
Wiktor Niesiobędzki bee3072568
Add support for Cloud Run v2 jobs (#1954)
Add support for Cloud Run v2 jobs

* create a separate file for service creation (service.tf) and job
  (job.tf) - for easy comparison
* add E2E tests where possibile
* remove default value for input variable `region`
* fix subnet range VPC Access Connector example
* add creation of service account for audit logs call (trigger requires
  service account)
* use provided trigger service account email in
  `local.trigger_sa_email`, so explicitly provided SA is passed to
  trigger
* set default value for vpc_connector_create.throughput.max, to match
  what is set by GCP API, as provider uses wrong default of 300 which
  results in perma-diff
* create inventory fiels for all examples

Global changes
* (tests) add input variable `project_number`, to allow assigning IAM permissions to Service Accounts in fixtures
* (tests) fix not outputting the path, when object is not found in inventory
* (tests) fix `create_e2e_sandbox.sh` - now it properly finds root of the repo

Secret Manager
* added `version_versions` output, to allow specifying versions in other modules. `versions` is sensitive and it makes it unsuitable for `for_each` values

New test fixtures
* `pubsub.tf` - creating one topic
* `secret-credential.tf` - creating Secret Manager `credential` secret
* `shared-vpc.tf` - creating two projects (host and service), and vpc in host project
* `vpc-connector.tf` - creating VPC Access Connector instance
2024-02-18 14:57:34 +01:00
simonebruzzechesse d11c380aec
Format python files in blueprints (#2079)
* format python files in blueprints
* update check on blueprints python code
* update python linter in CI workflow
2024-02-15 09:37:49 +01:00
Julio Castillo efb76b93e4
Remove unused tfeditor (#2039) 2024-02-02 10:14:14 +00:00
Wiktor Niesiobędzki 29b615e20d
Add version check to tools/lint.sh (#1955) 2023-12-30 09:09:10 +01:00
Julio Castillo fde7b76036
Allow per-module terraform fixtures (#1914)
* Allow terraform fixtures for examples

* Allow defining multiple fixtures, and named fixtures under tests/fixtures/

* Enable e2e for wiktorn

* Fix prepare_files call for e2e

* Move fixture to separate file, fix test

* Revert shallow-copying symlinks, performane penalty - 20%

* Update tfdoc.py to list used fixtures

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2023-12-29 09:43:44 +00:00
Julio Castillo b6e0557bbb
Simplify organization tags.tf locals (#1932)
* Simplify organization tags.tf locals

* Fix boilerplate

* Override github provider version for tests
2023-12-18 16:09:22 +00:00
Ludo f368165ea7
update github provider 2023-12-18 12:25:22 +01:00
Wiktor Niesiobędzki c5c127b9df Use TFTEST_E2E_ instead of TF_VAR variables
Use of TF_VAR variables modified results of `tests/examples` and
required setting different environment to run `tests/examples` and
`tests/examples_e2e` tests. No both can be run using the same
environment.
2023-11-30 07:43:46 +00:00
Simone Ruffilli 6d89b88149
versions.tf maintenance + copyright notice bump (#1782)
* Bump copyright notice to 2023

* Delete versions.tf on blueprints

* Pin provider to major version 5

* Remove comment

* Fix lint

* fix bq-ml blueprint readme

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2023-10-20 18:17:47 +02:00
Julio Castillo ef290c1c8d
Enforce mandatory types in all variables (#1737) 2023-10-06 09:44:33 +00:00
Alejandro Leal 81c6959617 Update to lint.sh and wording to some tf
fast/stages-multitenant/0-bootstrap-tenant/identity-providers.tf
fast/stages/0-bootstrap/identity-providers.tf
tools/lint.sh
2023-10-05 00:17:20 -04:00
Julio Castillo 5591dabdc5 Remove print 2023-08-31 11:20:25 +02:00
Julio Castillo 8a667ff35b Forgot to add the script 2023-08-31 11:17:39 +02:00
Julio Castillo 5bbb7bdd09 Simplify linting output in workflow 2023-08-31 10:15:12 +02:00
Ludovico Magnocavallo 8d7dba49cb
refactor tfdoc for legibility (#1544) 2023-07-29 11:11:31 +02:00
Julio Castillo 7fb656b8ae fixes 2023-07-28 17:10:49 +02:00
Julio Castillo 1800d3e617 Fix docstrings 2023-07-28 16:55:56 +02:00
Julio Castillo 4318be6cd8 Move unneeded try 2023-07-28 16:43:00 +02:00
Julio Castillo e6be368367 Simplify whitespace handling 2023-07-28 16:32:38 +02:00
Julio Castillo d61e7df562 Fix whitespace handling 2023-07-28 15:43:40 +02:00
Julio Castillo 4f123ccc74 Extend tfdoc to generate TOCs 2023-07-28 14:18:28 +02:00
Ludovico Magnocavallo 884cb8b4bf
Ensure all modules have an `id` output (#1410)
* net-vpc

* a-d

* complete modules

* fix error
2023-06-02 16:07:22 +02:00
Alejandro Leal 87cd83f5c0 Several updates
Several updates
2023-05-13 23:51:46 -04:00
Julio Castillo 87281f2017 Fix var name 2023-05-05 17:08:24 +02:00
Julio Castillo 176209ccda Fix requirements 2023-05-05 17:05:03 +02:00
Julio Castillo 72b5944fb3 Simplify return 2023-05-05 17:03:42 +02:00
Julio Castillo 873c45d6c6 Add jsonschema to tools requirements.txt 2023-05-05 17:01:08 +02:00
Julio Castillo 05ed1bf12f Add blueprint metadata validation tool 2023-05-05 16:57:48 +02:00
Julio Castillo f4b8a61981 Clean workflow files 2023-04-25 15:28:34 +02:00
Julio Castillo 4245faae73 Add archive provider 2023-04-25 15:22:27 +02:00
Julio Castillo b097d297ff Extend tests to use lockfile if available
This commints allows the test framework to use a lockfile generated
within the tools/lockfile directory. This allows reusing the provider
cache and also running tests in parallel.
2023-04-25 15:12:11 +02:00
Ludovico Magnocavallo 3d41d01efc
FAST plugin system (#1266)
* plugin folder, gitignore, serverless connector example

* add support to fast plugin variables and outputs to tfdoc

* rename folder, READMEs

* add variable description

* show diffs

* check documentation, use multiple files

* debug check doc

* try a different glob

* debug tfdoc names

* more debug

* and even more debug

* fix gitignore

* fix links

* support extra files in tests

* fix fixture, switch stage 2 peering to new tests

* tfdoc

* Allow globs in extra files

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-03-24 12:28:32 +00:00
Ludovico Magnocavallo 8fc9549c58
add support for proxy and psc subnets to module factory (#1211) 2023-03-05 17:08:43 +01:00
Ludovico Magnocavallo 96e829bdf3
Billing exclusion support for FAST mt resman (#1209)
* fix files resource parsing in tfdoc

* fix tfdoc generated output

* billing exclusion support in mt bootstrap
2023-03-03 16:23:36 +00:00
Aleksandr Averbukh 06dd38170d
Fix outdated go deps, dependabot alerts (#1208) 2023-03-03 07:15:08 +01:00