Commit Graph

283 Commits

Author SHA1 Message Date
Ludovico Magnocavallo 5453c585e0
FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052)
* rename stages

* remove support for external org billing, rename output files

* resman: make groups optional, align on new billing account variable

* bootstrap: multitenant outputs

* tenant bootstrap stage, untested

* fix folder name

* fix stage 0 output names

* optional creation for tag keys in organization module

* single tenant bootstrap minus tag

* rename output files, add tenant tag key

* fix organization module tag values output

* test skipping creation for tags in organization module

* single tenant bootstrap plan working

* multitenant bootstrap

* tfdoc

* fix check links error messages

* fix links

* tfdoc

* fix links

* rename fast tests, fix bootstrap tests

* multitenant stages have their own folder, simplify stage numbering

* stage renumbering

* wip

* rename tests

* exclude fast providers in fixture

* stage 0 tests

* stage 1 tests

* network stages tests

* stage tests

* tfdoc

* fix links

* tfdoc

* multitenant tests

* remove local files

* stage links command

* fix links script, TODO

* wip

* wip single tenant bootstrap

* working tenant bootstrap

* update gitignore

* remove local files

* tfdoc

* remove local files

* allow tests for tenant bootstrap stage

* tenant bootstrap proxies stage 1 tfvars

* stage 2 and 3 service accounts and IAM in tenant bootstrap

* wip

* wip

* wip

* drop multitenant bootstrap

* tfdoc

* add missing stage 2 SAs, fix org-level IAM condition

* wip

* wip

* optional tag value creation in organization module

* stage 1 working

* linting

* linting

* READMEs

* wip

* Make stage-links script work in old macos bash

* stage links command help

* fix output file names

* diagrams

* fix svg

* stage 0 skeleton and diagram

* test svg

* test svg

* test diagram

* diagram

* readme

* fix stage links script

* stage 0 readme

* README changes

* stage readmes

* fix outputs order

* fix link

* fix tests

* stage 1 test

* skip stage example

* boilerplate

* fix tftest skip

* default bootstrap stage log sinks to log buckets

* add logging to tenant bootstrap

* move iam variables out of tenant config

* fix cicd, reintroduce missing variable

* use optional in stage 1 cicd variable

* rename extras stage

* rename and move identity providers local, use optional for cicd variable

* tfdoc

* add support for wif pool and providers, ci/cd

* tfdoc

* fix links

* better handling of modules repository

* add missing role on logging project

* fix cicd pools in locals, test cicd

* fix workflow extension

* fix module source replacement

* allow tenant bootstrap cicd sa to impersonate resman sa

* tenant workflow templates fix for no providers file

* fix output files, push github workflow template to new repository

* remove try from outpout files

* align stage 1 cicd internals to stage 0

* tfdoc

* tests

* fix tests

* tests

* improve variable descriptions

* use optional in fast features

* actually create tenant log sinks, and allow the resman sa to do it

* test

* tests

* aaaand tests again

* fast features tenant override

* fast features tenant override

* fix wording

* add missing comment

* configure pf service accounts

* add missing comment

* tfdoc

* tests

* IAM docs

* update copyright

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-02-04 15:00:45 +01:00
javiergp ce1f86d20b
Vertex Pipelines MLOps framework blueprint (#1038)
* First release of the MLOps blueprint
2023-02-02 19:13:13 +01:00
Ludovico Magnocavallo a046f280a4
Delete package-lock.json
flagged by dependabot as containing a security vulnerability (taffydb)
2023-02-01 18:57:10 +01:00
Ludovico Magnocavallo 0da76cdbdb
Delete package-lock.json
flagged by dependabot as containing a security vulnerability (taffydb)
2023-02-01 18:56:34 +01:00
apichick 6298ee5cdb
Removed unused file package-lock.json (#1124) 2023-02-01 18:54:23 +01:00
Julio Castillo 1204b0077e
Merge branch 'master' into mci_gateway 2023-01-31 11:54:31 +00:00
Miren Esnaola 1a2745b632 In the apigee module now both the /22 and /28 peering IP ranges are passed at instance creation 2023-01-31 11:32:53 +01:00
Wiktor Niesiobędzki 1a657b31d3 Bump beta provider to 4.48
This is the first version that supports `gateway_api_config` block
2023-01-29 15:50:24 +01:00
Wiktor Niesiobędzki 2976df596e Fix tests (add JIT-ed SA for multiclusteringress.googleapis.com) 2023-01-29 15:50:24 +01:00
Ayman Farhat edd3a82453
Include cloudbuild API in project module (#1116)
* Include cloudbuild API in project module

* Increase number of resources
2023-01-27 21:38:01 +01:00
Aurélien Legrand 7f5c177cfe
Network Dashboard: PSA support for Filestore and Memorystore (#1106)
* Support for Filestore and Memorystore PSA ranges

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-01-25 15:02:30 +00:00
dependabot[bot] e780b7f98c
Bump cookiejar in /blueprints/apigee/bigquery-analytics/functions/export (#1110)
Bumps [cookiejar](https://github.com/bmeck/node-cookiejar) from 2.1.3 to 2.1.4.
- [Release notes](https://github.com/bmeck/node-cookiejar/releases)
- [Commits](https://github.com/bmeck/node-cookiejar/commits)

---
updated-dependencies:
- dependency-name: cookiejar
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-24 15:07:11 +00:00
Julio Castillo 551d45760f
Merge branch 'master' into google_gkehub_feature_membership 2023-01-23 09:11:49 +01:00
apichick e32036dbaa
Merge branch 'master' into hybrid-gke-README-fix 2023-01-22 11:22:07 +01:00
Ludovico Magnocavallo 3cca689792
Check linting for Python dashboard files (#1107)
* enable Python lint check for network dashboard

* fix linting for network dashboard
2023-01-21 17:17:51 +01:00
Miren Esnaola 4accf87bff Updated hybrid GKE readme 2023-01-20 14:41:25 +01:00
Ludovico Magnocavallo 050daa65f2
Merge branch 'master' into hybrid-gke-updates 2023-01-20 07:48:49 +01:00
Miren Esnaola 8945165bc3 Improvements in apigee hybrid-gke: now using workload identity and GLB 2023-01-19 22:11:44 +01:00
Wiktor Niesiobędzki b38ef22572 Even more test fixes 2023-01-19 09:37:48 +01:00
Wiktor Niesiobędzki 720213593e Use google_gkehub_feature_membership
Use google_gkehub_feature_membership to provision ASM on clusters.

Ensure that the cluster membership is refreshed on cluster recreation.
2023-01-19 09:37:48 +01:00
lcaggio 3c88aace1a Fix dataflow shared-vpc support 2023-01-19 01:04:11 +01:00
lcaggio 6477d0790d Remove temp file. 2023-01-19 00:39:55 +01:00
lcaggio 07a7be29e3 Add shared-vpc support on Data Playgroud blueprint 2023-01-19 00:33:31 +01:00
lcaggio 02938bdfa1 Fix Table in readme. 2023-01-16 15:58:11 +01:00
lcaggio 1672cfe9c1 Fix README. 2023-01-12 12:45:39 +01:00
lcaggio 50a52ae01a Implement PR comments. 2023-01-12 12:41:00 +01:00
lcaggio 07a975f84e Fix example test. 2023-01-11 12:37:08 +01:00
lcaggio 72abd47520 Fix documentation. 2023-01-11 08:47:42 +01:00
lcaggio c172c7bb2e Remove plg project and update to composer 2 2023-01-11 00:31:58 +01:00
lcaggio 488b6d4dc0 Remove plg project, composer 2 2023-01-11 00:31:52 +01:00
Miren Esnaola 66c37a4169 Fixes in Apigee blueprints README files 2023-01-05 11:31:07 +01:00
Ludovico Magnocavallo f7350be743
Merge branch 'master' into apigee-hybrid-gke 2023-01-05 09:11:39 +01:00
Miren Esnaola f6413747b7 Fixes in Apigee Bigquery Analytics blueprint 2023-01-04 17:30:07 +01:00
Miren Esnaola 33f5607ff1 Apigee hybrid on GKE 2023-01-04 10:53:49 +01:00
Miren Esnaola 0235690aaa Moved apigee bigquery analytics blueprint, added apigee network patterns 2022-12-23 14:38:15 +01:00
Ludovico Magnocavallo 66a402083b
Allow setting no ranges in firewall module custom rules (#1073)
* allow setting no ranges in custom firewall rules

* fix blueprint

* fix example

* fix example
2022-12-23 09:03:31 +01:00
Israel Herraiz 7df527e1ef Bump versions EVERYWHERE 2022-12-23 00:06:42 +01:00
Aurélien Legrand 5063c3b513
Network dashboard: PSA ranges support, starting with Cloud SQL (#1063)
* Adding PSA ranges support and ip utilization, starting with Cloud SQL

* yapf formatting

* refactor

* fix camelcase

* remove unused variable

* remove test file

* rename discover plugin

* fixing issue with asset name

Co-authored-by: Ludo <ludomagno@google.com>
2022-12-22 13:14:42 +01:00
Wiktor Niesiobędzki 7e18f6d7b5 Fixes for GKE
* Fix non-empty plan when spot instances are used
* Add cluster_id and recommend its use, as this prevents inconsitencies
  when only cluster is recreated (with no changes on node pool)
2022-12-21 23:14:52 +01:00
Aurélien Legrand 7acb6b3747
Update src/README.md for Network Dashboard (#1060)
* Update src/README.md for Network Dashboard

Proposing some small updates to src/README.md file for the Network Dashboard

* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2022-12-21 16:30:09 +01:00
Julio Castillo b031d7add5 Fix tests 2022-12-18 20:37:16 +01:00
Julio Castillo e700a27079 Enforce terraform fmt in examples 2022-12-18 14:00:19 +01:00
Ludovico Magnocavallo 93361d7f3d
Networking dashboard and discovery tool refactor (#1020)
* wip

* wip

* wip

* wip

* wip

* discovery

* single discovery

* page token

* batch requests

* remove plugin name

* streamline

* streamline

* dynamic routes

* dynamic routes

* forwarding rules and addresses

* batch requests

* metrics

* notes

* notes

* streamline

* fixes, dump

* streamline

* remove globals

* wip metrics

* subnet time series

* networks per project plugin

* firewall rules timeseries

* use names in metric labels

* firewall policies timeseries

* wip

* instances per network timeseries

* routes timeseries

* custom quota

* simpler quota, network peering timeseries

* peering timeseries

* timeseries names

* wip descriptors

* metric descriptors

* fixes

* wip

* Use partial for all cf init functions

* Add requirements.txt

* fix org key mismatch

* Fix folder short cli name

* Fix instance_networks when iterable is empty

* more readability and fixing some strings

* replace() -> removeprefix and remove unneeded quoting

* setdefault in init()s

* Fix next hop type

* Remove unneeded fstring

* create descriptors

* create descriptors log

* rename descriptor requests function

* non-working metrics implementation (duplicate timeseries batched)

* timeseries

* fixes

* write timseries

* fix timeseries plugins

* start documenting code

* docstrings and comments

* docstrings comments and small fixes

* rename cf to src

* discover nodes instead of just projects

* discovery node can be a folder or org

* cf entrypoint and fixes

* cf deployment

* remove old paths

* cloud function deploy readme

* diagrams

* resource ids in example

* discovery tool readme

* top-level README

* Some documentation fixes

* Add secondary ranges

* Update README.md

* add legend to scope diagram

* improve description of discovery configuration variable

* add comment in example for custom quotas file

* rename op_project to monitoring_project

* dashboard metric rename wip

* Update discover-cai-compute.py

* deploy sample dashboard

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Aurélien Legrand <aurelien.legrand01@gmail.com>
2022-12-18 10:07:24 +01:00
Ludovico Magnocavallo d2015b0bc3
Assorted module fixes (#1045)
* net-ilb-l7 use both neg types in backends

* run example tests on cloud-config-container modules

* streamline nginx-tls cos module

* add tests for cos modules

* tfdoc

* onprem needs fixing, links

* disable test

* test
2022-12-10 15:40:15 +01:00
Ludovico Magnocavallo 46f694be08
Refactor net-glb module for Terraform 1.3 (#1044)
* wip

* urlmaps

* wip

* tested

* fix managed certificate names

* simple HTTP and HTTPS examples

* gce and hybrid neg examples

* internet NEG example

* serverless NEG and backend bucket examples

* certificate example

* fix example tests

* remove stale test module

* module test

* adfs blueprint

* add support for non-classic GLB and PSC NEGs

* apigee blueprint

* glb and cloud armor blueprint

* deprecate non-working nginx reverse cluster

* api gateway blueprint

* tfdoc

* fix top-level READMEs

* switch to new tests

* only/at most
2022-12-08 17:35:44 +01:00
Brian Jung 5b71f2f54c
Adding Secondary IP Utilization calculation (#982)
* hello

* Adding secondary range IP address utilization calculation.

* using yapf to format code

* Minor fixes for Network Monitor

Co-authored-by: Brian Jung <brianhmj@google.com>
2022-12-07 11:45:20 +01:00
dependabot[bot] fee7ecfa66
Bump qs and formidable (#1037)
Bumps [qs](https://github.com/ljharb/qs) and [formidable](https://github.com/node-formidable/formidable). These dependencies needed to be updated together.

Updates `qs` from 6.9.3 to 6.11.0
- [Release notes](https://github.com/ljharb/qs/releases)
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/ljharb/qs/compare/v6.9.3...v6.11.0)

Updates `formidable` from 2.0.1 to 2.1.1
- [Release notes](https://github.com/node-formidable/formidable/releases)
- [Changelog](https://github.com/node-formidable/formidable/blob/master/CHANGELOG.md)
- [Commits](https://github.com/node-formidable/formidable/commits)

---
updated-dependencies:
- dependency-name: qs
  dependency-type: indirect
- dependency-name: formidable
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-06 16:43:34 +01:00
Thomas GRUSON d76eadef18
feat(blueprints): get audience from tfc environment
Fix typo in documentation and get audience from Terraform Cloud `TFC_WORKLOAD_IDENTITY_AUDIENCE` environment variable.
2022-12-05 15:50:35 +01:00
Julio Castillo fa63e9d5d0 Remove optional stuff 2022-11-30 16:00:53 +01:00
Julio Castillo b5593db8a2 Update h&s blueprints advertisements 2022-11-30 10:59:15 +01:00