Commit Graph

545 Commits

Author SHA1 Message Date
Christoph Grotz c9915b11a4 Fixed variable order 2023-02-24 17:27:49 +00:00
Julio Diez 5f9955b5fe Add image to show how to test different services 2023-02-24 18:21:10 +01:00
Julio Diez 1f3db47b0b Clarify how to test custom domain case 2023-02-24 18:03:47 +01:00
Christoph Grotz d97426633e Updated docs and added test 2023-02-24 17:02:50 +00:00
Christoph Grotz 3d8822d365 fixed tfdocs 2023-02-24 16:39:39 +00:00
Christoph Grotz 73764e3827 Added license boilerplate with block comment
instead of line comment
2023-02-24 16:21:02 +00:00
Christoph Grotz b229926516 formatted terraform 2023-02-24 16:18:05 +00:00
Christoph Grotz e56d466e81 Added a PSC GCLB example 2023-02-24 16:14:54 +00:00
Giorgio Conte a51c682005 Updated tf file to add the following features:
- default location of dataset to US
- changed name of vertex metastore to "default"
- add ai user and service account us to notebook SA
- add ai user to vertex sa
2023-02-24 13:27:44 +00:00
Julio Diez 1f4d17973b Add test for onprem (VPN) use case in README 2023-02-24 13:42:14 +01:00
Julio Diez 66353ac1cc Merge branch 'master' into serverless-program 2023-02-24 13:01:36 +01:00
Julio Diez 8518bffaaa Add another test for VPC-SC in README 2023-02-24 12:36:12 +01:00
Julio Diez 0f7a9a3ce3 Include TFDOC content in README 2023-02-24 12:23:04 +01:00
Julio Diez f93cdb920e Add trailing dots to descriptions 2023-02-24 12:19:31 +01:00
Julio Diez e8850ff3a1 Include tests in README 2023-02-24 12:16:22 +01:00
Julio Diez 531bfe134d Update service-running image to reflect output from new CR image in use 2023-02-24 12:05:44 +01:00
Julio Diez ad748db8a4 Add use case with custom domain description to README 2023-02-23 21:52:02 +01:00
Julio Diez 6f622e0601 DNS entries are now created for upper-level domains, not for endpoints 2023-02-23 21:18:19 +01:00
Julio Diez f41bb15162 Output all CR services URLs 2023-02-23 21:07:57 +01:00
Julio Diez d538e066a2 More explicit naming in CR services to later refer to them 2023-02-23 20:45:00 +01:00
Julio Diez 5065a52382 Fix naming in module cr2 2023-02-23 20:33:26 +01:00
Julio Diez 599d1a6ca5 Use *.run.app. domain in the DNS record 2023-02-23 20:15:34 +01:00
Julio Diez 3f45e75c35 Add DNS record for custom domain 2023-02-23 19:44:37 +01:00
Julio Diez 8d1edea487 Back to 'hello' image to distinguish between services 2023-02-23 19:21:44 +01:00
Julio Diez ba0ec82144 Create more backend services and url map for them 2023-02-23 19:16:27 +01:00
lcaggio 50856e6951 First commit 2023-02-23 18:36:03 +01:00
Julio Diez aa385241db Add two different CR services in service project 2023-02-23 18:25:16 +01:00
Julio Diez edece34342 Add L7 ILB in preparation for custom domain 2023-02-23 17:41:50 +01:00
lcaggio 47855cb682
Merge branch 'master' into lcaggio/dp-projectcreate 2023-02-23 11:54:48 +01:00
lcaggio b279c083a0 Fix README and IAM files 2023-02-23 11:54:16 +01:00
Julio Diez 91bd44f5d5 Remove 'main' from naming for CR since it can be deployed in a svc project 2023-02-22 15:54:03 +01:00
Julio Diez 29af0f3b31 Deploy CR service in service project if use case is using ILB 2023-02-22 15:29:15 +01:00
lcaggio 2108b4650d Fix Tests, rely on iam additive. 2023-02-22 01:36:01 +01:00
Julio Diez 117141cdb0 Add subcase description in README 2023-02-21 17:58:05 +01:00
Julio Diez 376ccb7824 Add a service project and corresponding test VM 2023-02-21 17:20:57 +01:00
Julio Castillo 1696f70f47 Update PF variables for org policies 2023-02-21 15:49:16 +01:00
Julio Castillo a5e905cb80 Update remaining org policies 2023-02-21 15:49:16 +01:00
Julio Castillo d3bcf625f9 Update yaml org policies 2023-02-21 15:49:16 +01:00
Julio Diez 1e794c851d Update README 2023-02-21 13:55:17 +01:00
Julio Diez eb85a4ad3b Enlarge some images more in README 2023-02-20 19:03:11 +01:00
Julio Diez 0cfe3d6a65 Enlarge some images in README 2023-02-20 19:00:20 +01:00
Julio Diez efe5e0e5f2 Fixes in README 2023-02-20 18:58:57 +01:00
Julio Diez d61ccbe5ac Add third use case description in README 2023-02-20 18:38:00 +01:00
Julio Diez 7159274533 Add terraform identity to VPC SC
If the identity that runs terraform is not included when creating a VPC SC
perimeter, it loses control afterwards.
2023-02-20 16:54:34 +01:00
lcaggio eda9597f0b Fix linting 2023-02-20 01:03:38 +01:00
lcaggio c523dce3a6 Fix 2023-02-20 01:00:57 +01:00
lcaggio f4c1fa6c20 Fix tests. 2023-02-20 00:56:32 +01:00
lcaggio db6a4f9ac7 Remove variables 2023-02-20 00:45:40 +01:00
lcaggio c8f25512eb Fix variabler to support existing projects. 2023-02-20 00:39:37 +01:00
lcaggio ad6667a8fa First commit 2023-02-19 22:37:32 +01:00
Aleksandr Averbukh befc73ec12 feat: Update TFC+WIF blueprint with TFC Dynamic Credentials feature. 2023-02-19 00:51:49 +01:00
Julio Diez bc6b89778c Update README 2023-02-17 20:01:49 +01:00
Julio Diez 559c44fce6 Access from another project, with VPC SC 2023-02-17 19:58:08 +01:00
Julio Diez de47f6077b Simplify some naming 2023-02-17 17:53:11 +01:00
Julio Diez c28039eaff Remove field, PGA is enabled by default 2023-02-17 17:41:29 +01:00
Julio Diez a77d7ea111 Add another GCP project to access CR from 2023-02-17 16:55:11 +01:00
Julio Diez 5e8b8d0290 Use case 2 image 2023-02-17 13:41:53 +01:00
Julio Diez 5b6ade4429 Keep test VM in main project even with onprem project 2023-02-17 13:35:53 +01:00
Julio Diez d31382994e Update README 2023-02-17 13:29:50 +01:00
Julio Diez 3cf536931d Update README 2023-02-17 13:26:26 +01:00
Julio Diez 25f243aa3d Update README to adapt to naming in projects 2023-02-17 13:22:20 +01:00
Julio Diez 46c041f59f Change naming from 'host' project to 'main' to avoid confusion 2023-02-17 13:15:58 +01:00
Ayman Farhat eac2065ae2 Update on docs wording 2023-02-16 17:04:05 +01:00
Ayman Farhat fcdadf521d Fix spelling mistake 2023-02-16 15:48:12 +01:00
Ayman Farhat a853dc4fe2 update Data Platform blueprint README with more example Dataflow commands 2023-02-16 15:45:20 +01:00
Julio Diez b5e51180e6 Add second use case description in README 2023-02-15 21:22:33 +01:00
Julio Diez a98248e32b Update README to reflect only private access is allowed 2023-02-15 20:53:58 +01:00
Julio Diez 9a46d0f2e6 Restrict traffic to internal sources 2023-02-15 20:45:10 +01:00
Julio Diez 30821ac58e Add first use case description in README 2023-02-15 20:44:22 +01:00
Julio Diez 15c8f92f46 Delete project resources without deleting the projects 2023-02-15 20:33:45 +01:00
Julio Diez 636a4cc01c Access CR from "onprem" environment 2023-02-15 13:37:16 +01:00
Julio Diez d0934903aa Use a more curl friendly image 2023-02-14 12:57:43 +01:00
Julio Diez 28f4c3170e Initial README 2023-02-13 21:57:54 +01:00
Julio Diez 92a57becd8 Accessing Cloud Run privately, first use case 2023-02-13 21:54:03 +01:00
lcaggio ebc4bc51a5 Workaround to mitigate issue 9164 2023-02-13 15:25:24 +01:00
Julio Diez 0cfd8dd883 Move all images to an images/ folder 2023-02-10 19:48:04 +01:00
Julio Diez 5c3fd31b02 Add missing newlines in README 2023-02-10 19:43:56 +01:00
Julio Diez 621ee6e517 Update variables table via tfdoc.py 2023-02-10 18:02:37 +01:00
Julio Diez e72602a11d Update variables table via tfdoc.py 2023-02-10 18:02:37 +01:00
Julio Diez 6c470ece11 Option to create a project 2023-02-10 18:02:37 +01:00
Julio Diez 98f9a318bc Update output and variable tables via tfdoc.py 2023-02-10 14:38:16 +01:00
Julio Diez 1335fd65da Update output and variable tables via tfdoc.py 2023-02-10 14:21:38 +01:00
Julio Diez 9fa48ea177 Add ending periods to descriptions 2023-02-10 14:13:56 +01:00
Julio Diez 4aa135f7e8 Generated output and variable tables via tfdoc.py 2023-02-10 14:01:11 +01:00
Julio Diez d19d434538 Add HTML tags to generate output and variable tables 2023-02-10 13:21:01 +01:00
Julio Diez aca3069951 Add quotes around var project_id 2023-02-10 12:59:09 +01:00
Julio Diez f31293ba04 Specify block types 2023-02-10 12:56:59 +01:00
Julio Diez d916632686 Update URL to repo 2023-02-10 12:49:39 +01:00
Julio Diez 3935867d92 Simplify conditional expressions 2023-02-10 12:48:47 +01:00
Julio Diez a8de9088c6 Alphabetical ordering for outputs and variables 2023-02-10 12:46:34 +01:00
Julio Diez a199414794 Add license boilerplate 2023-02-10 12:30:39 +01:00
Julio Diez e8303e15ba
Merge pull request #8 from juliodiez/master
Sync branch
2023-02-10 10:27:54 +01:00
Julio Diez 3e7d8c563e Infer LB creation from custom_domain var
This way we can remove one var (glb_create) and simplify it for the user.
2023-02-09 10:33:51 +01:00
Julio Diez a44759d585 Fix typos in README 2023-02-08 21:21:20 +01:00
Julio Diez 8b41008397 Add README file with the use cases 2023-02-08 21:04:51 +01:00
Julio Castillo d7b88b7d0e Fix bq factory docs 2023-02-08 18:22:10 +01:00
Julio Diez 642fff04ad Get rid of config files
Follow CFF way, instead of using config files set appropriate values in file
variables.tf and use examples in the README to show how to tweak variables.
2023-02-07 21:01:28 +01:00
Julio Castillo 08f2540872 Bump provider version 2023-02-07 15:59:29 +01:00
lcaggio d78815d3d7 Fix README 2023-02-07 13:57:26 +01:00
lcaggio 71469679e3 Add variables validation 2023-02-07 13:53:54 +01:00
lcaggio dd77c2fe79
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-07 07:58:00 +01:00
lcaggio 307cf32cf9 Fix 2023-02-06 21:34:29 +01:00
lcaggio d68bb4b320 Fixes 2023-02-06 21:33:04 +01:00
Julio Castillo e1e0c38236 Fix README and var order 2023-02-06 18:08:03 +01:00
Julio Castillo c883d5ec3a Fix README 2023-02-06 18:03:43 +01:00
Paul Woelfel bd90aa7f47
Add descriptive_name 2023-02-06 17:39:33 +01:00
Paul Woelfel f2b61b1e31
Add descriptive name as optional argument 2023-02-06 17:35:11 +01:00
lcaggio 1e4499c8ab Fixes 2023-02-06 14:58:40 +01:00
Ayman Farhat 02d8d8367a
[Feature] Update data platform blue print with Dataflow Flex template (#1105)
* Add initial dataflow template code + TF infra

* Refactor the datapipeline DAG to use flex template operator, cleanup code

* Remove unneeded bash scripts, update README with manual examples

* Refactor datapipeline_dc_tags.py and include new Flex template

* Update docs to reflect changes

* Remove sub-dependencies and keep apache beam

* Add missing license headers and update tests

* Set resouces to 291 in tests

* Update outputs via tfdoc

* Update with outputs order and tfdoc

* Correct number of resources

* Fix to add region into command from var

* Enable service account impersonation for running builds

* Update example dataflow run command to use orchestrator SA

* Remove hard coded values in example

* Keep original airflow files, add new which use Flex template as example

* Update tests and doc

* Fix number of resources in plan

* Run tfdoc remove files section in README

* Fix number of modules in tfdoc

* Update number of resources

* Add missin service account

* Update DF demo README

* Quick rename

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-02-06 07:35:40 +01:00
lcaggio 39ffdcf184
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-05 22:45:14 +01:00
lcaggio f49e5ccec7 Fix readmes and variables. 2023-02-05 22:44:31 +01:00
lcaggio 47c75f8862 Implement project_config variable 2023-02-05 21:12:46 +01:00
lcaggio 38fc9e8b9b First commit 2023-02-05 07:52:33 +01:00
lcaggio ce1cc5f75a
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-04 19:42:19 +01:00
Ludovico Magnocavallo 5453c585e0
FAST multitenant bootstrap and resource management, rename org-level FAST stages (#1052)
* rename stages

* remove support for external org billing, rename output files

* resman: make groups optional, align on new billing account variable

* bootstrap: multitenant outputs

* tenant bootstrap stage, untested

* fix folder name

* fix stage 0 output names

* optional creation for tag keys in organization module

* single tenant bootstrap minus tag

* rename output files, add tenant tag key

* fix organization module tag values output

* test skipping creation for tags in organization module

* single tenant bootstrap plan working

* multitenant bootstrap

* tfdoc

* fix check links error messages

* fix links

* tfdoc

* fix links

* rename fast tests, fix bootstrap tests

* multitenant stages have their own folder, simplify stage numbering

* stage renumbering

* wip

* rename tests

* exclude fast providers in fixture

* stage 0 tests

* stage 1 tests

* network stages tests

* stage tests

* tfdoc

* fix links

* tfdoc

* multitenant tests

* remove local files

* stage links command

* fix links script, TODO

* wip

* wip single tenant bootstrap

* working tenant bootstrap

* update gitignore

* remove local files

* tfdoc

* remove local files

* allow tests for tenant bootstrap stage

* tenant bootstrap proxies stage 1 tfvars

* stage 2 and 3 service accounts and IAM in tenant bootstrap

* wip

* wip

* wip

* drop multitenant bootstrap

* tfdoc

* add missing stage 2 SAs, fix org-level IAM condition

* wip

* wip

* optional tag value creation in organization module

* stage 1 working

* linting

* linting

* READMEs

* wip

* Make stage-links script work in old macos bash

* stage links command help

* fix output file names

* diagrams

* fix svg

* stage 0 skeleton and diagram

* test svg

* test svg

* test diagram

* diagram

* readme

* fix stage links script

* stage 0 readme

* README changes

* stage readmes

* fix outputs order

* fix link

* fix tests

* stage 1 test

* skip stage example

* boilerplate

* fix tftest skip

* default bootstrap stage log sinks to log buckets

* add logging to tenant bootstrap

* move iam variables out of tenant config

* fix cicd, reintroduce missing variable

* use optional in stage 1 cicd variable

* rename extras stage

* rename and move identity providers local, use optional for cicd variable

* tfdoc

* add support for wif pool and providers, ci/cd

* tfdoc

* fix links

* better handling of modules repository

* add missing role on logging project

* fix cicd pools in locals, test cicd

* fix workflow extension

* fix module source replacement

* allow tenant bootstrap cicd sa to impersonate resman sa

* tenant workflow templates fix for no providers file

* fix output files, push github workflow template to new repository

* remove try from outpout files

* align stage 1 cicd internals to stage 0

* tfdoc

* tests

* fix tests

* tests

* improve variable descriptions

* use optional in fast features

* actually create tenant log sinks, and allow the resman sa to do it

* test

* tests

* aaaand tests again

* fast features tenant override

* fast features tenant override

* fix wording

* add missing comment

* configure pf service accounts

* add missing comment

* tfdoc

* tests

* IAM docs

* update copyright

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2023-02-04 15:00:45 +01:00
lcaggio 45a74f37f9 Mention policies in the costomization section. 2023-02-04 06:24:35 +01:00
lcaggio f00d4d3edd
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-03 22:17:30 +01:00
Julio Diez 60c9f2cc40 Merge branch 'serverless-program' of https://github.com/juliodiez/cloud-foundation-fabric into serverless-program 2023-02-03 11:42:33 +01:00
Julio Diez 8899412d76 Separate var iap_email to override it on the CLI 2023-02-03 11:37:56 +01:00
Julio Diez 54a0ff8939 Info comment for support_email 2023-02-03 11:15:09 +01:00
Julio Diez 616125c1cd Enable service APIs in the project 2023-02-03 11:04:42 +01:00
Julio Diez 63bae85b34 Identity-Aware Proxy configuration 2023-02-02 21:47:11 +01:00
javiergp ce1f86d20b
Vertex Pipelines MLOps framework blueprint (#1038)
* First release of the MLOps blueprint
2023-02-02 19:13:13 +01:00
lcaggio fafc69a206
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-02 09:03:00 +01:00
Ludovico Magnocavallo a046f280a4
Delete package-lock.json
flagged by dependabot as containing a security vulnerability (taffydb)
2023-02-01 18:57:10 +01:00
Ludovico Magnocavallo 0da76cdbdb
Delete package-lock.json
flagged by dependabot as containing a security vulnerability (taffydb)
2023-02-01 18:56:34 +01:00
apichick 6298ee5cdb
Removed unused file package-lock.json (#1124) 2023-02-01 18:54:23 +01:00
Julio Diez 462ca562c0
Merge pull request #6 from juliodiez/master
Sync branch
2023-02-01 18:01:32 +01:00
lcaggio 30f2b49c9c README Typos 2023-02-01 17:34:31 +01:00
lcaggio e3daaf2503 Update READMEs 2023-02-01 13:02:07 +01:00
lcaggio f9e3eb7d5a Update copyright 2023-02-01 09:30:28 +01:00
lcaggio 840fc86b3e Fix groups. Fix tests. 2023-02-01 08:55:33 +01:00
lcaggio 053d4151de Chane feature name, improve README 2023-02-01 00:50:22 +01:00
lcaggio 3995ec1550
Merge branch 'master' into lcaggio/shieldedfolder 2023-02-01 00:28:01 +01:00
lcaggio f36b004664 Make VPCSC optional; ingress policy 2023-02-01 00:26:57 +01:00
lcaggio fb04e78829 Fix variables 2023-01-31 22:35:48 +01:00
Julio Castillo 1204b0077e
Merge branch 'master' into mci_gateway 2023-01-31 11:54:31 +00:00
Miren Esnaola 1a2745b632 In the apigee module now both the /22 and /28 peering IP ranges are passed at instance creation 2023-01-31 11:32:53 +01:00
lcaggio 0e5f2f71a2 Fix Linting 2023-01-30 23:57:20 +01:00
lcaggio a166c612d5 Merge branch 'master' into lcaggio/shieldedfolder 2023-01-30 23:52:46 +01:00
lcaggio 6a39d80487 Fix README 2023-01-30 23:49:47 +01:00
lcaggio 66a10d76c3 Add test 2023-01-30 23:47:33 +01:00
lcaggio a808ea3293 Draft README 2023-01-30 23:32:13 +01:00
Julio Diez 4c2e31baac Rename config file 2023-01-30 20:34:09 +01:00
Julio Diez 4c4018073f Add security configuration using Cloud Armor in the LB 2023-01-30 20:30:35 +01:00
Julio Diez 8a7aa397ef Add purpose to config files 2023-01-30 20:13:38 +01:00
Julio Diez 865a98e3ed Add ingress settings blocking requests not coming from the LB 2023-01-30 16:53:23 +01:00
Julio Diez 93276d03d8 Add custom domain output 2023-01-30 16:34:52 +01:00
Julio Diez cca1d9fe3c Output the LB IP address 2023-01-30 14:39:46 +01:00
Julio Diez e10c6ed62f Output URL as single string 2023-01-30 14:38:38 +01:00