lcaggio
3cc6c71e96
Add Minimal Data Platform blueprint ( #1362 )
...
Minimal Data Platform blueprint
2023-05-08 10:25:06 +02:00
lcaggio
c8d5b85ac9
Fix test in README
2023-04-06 15:55:34 +02:00
lcaggio
1bd5c66b42
Fix test
2023-04-06 14:21:57 +02:00
Dedeco
230cbe4903
Fix variable terraform.tfvars.sample ( #1261 )
2023-03-17 11:13:10 +01:00
Julio Castillo
be60753246
Fix failing tests
2023-02-25 19:26:11 +01:00
Julio Castillo
99e69e319f
Update data platform tests
2023-02-25 19:26:11 +01:00
lcaggio
b279c083a0
Fix README and IAM files
2023-02-23 11:54:16 +01:00
lcaggio
2108b4650d
Fix Tests, rely on iam additive.
2023-02-22 01:36:01 +01:00
lcaggio
eda9597f0b
Fix linting
2023-02-20 01:03:38 +01:00
lcaggio
c523dce3a6
Fix
2023-02-20 01:00:57 +01:00
lcaggio
f4c1fa6c20
Fix tests.
2023-02-20 00:56:32 +01:00
lcaggio
db6a4f9ac7
Remove variables
2023-02-20 00:45:40 +01:00
lcaggio
c8f25512eb
Fix variabler to support existing projects.
2023-02-20 00:39:37 +01:00
Ayman Farhat
02d8d8367a
[Feature] Update data platform blue print with Dataflow Flex template ( #1105 )
...
* Add initial dataflow template code + TF infra
* Refactor the datapipeline DAG to use flex template operator, cleanup code
* Remove unneeded bash scripts, update README with manual examples
* Refactor datapipeline_dc_tags.py and include new Flex template
* Update docs to reflect changes
* Remove sub-dependencies and keep apache beam
* Add missing license headers and update tests
* Set resouces to 291 in tests
* Update outputs via tfdoc
* Update with outputs order and tfdoc
* Correct number of resources
* Fix to add region into command from var
* Enable service account impersonation for running builds
* Update example dataflow run command to use orchestrator SA
* Remove hard coded values in example
* Keep original airflow files, add new which use Flex template as example
* Update tests and doc
* Fix number of resources in plan
* Run tfdoc remove files section in README
* Fix number of modules in tfdoc
* Update number of resources
* Add missin service account
* Update DF demo README
* Quick rename
---------
Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-02-06 07:35:40 +01:00
Ludovico Magnocavallo
5453c585e0
FAST multitenant bootstrap and resource management, rename org-level FAST stages ( #1052 )
...
* rename stages
* remove support for external org billing, rename output files
* resman: make groups optional, align on new billing account variable
* bootstrap: multitenant outputs
* tenant bootstrap stage, untested
* fix folder name
* fix stage 0 output names
* optional creation for tag keys in organization module
* single tenant bootstrap minus tag
* rename output files, add tenant tag key
* fix organization module tag values output
* test skipping creation for tags in organization module
* single tenant bootstrap plan working
* multitenant bootstrap
* tfdoc
* fix check links error messages
* fix links
* tfdoc
* fix links
* rename fast tests, fix bootstrap tests
* multitenant stages have their own folder, simplify stage numbering
* stage renumbering
* wip
* rename tests
* exclude fast providers in fixture
* stage 0 tests
* stage 1 tests
* network stages tests
* stage tests
* tfdoc
* fix links
* tfdoc
* multitenant tests
* remove local files
* stage links command
* fix links script, TODO
* wip
* wip single tenant bootstrap
* working tenant bootstrap
* update gitignore
* remove local files
* tfdoc
* remove local files
* allow tests for tenant bootstrap stage
* tenant bootstrap proxies stage 1 tfvars
* stage 2 and 3 service accounts and IAM in tenant bootstrap
* wip
* wip
* wip
* drop multitenant bootstrap
* tfdoc
* add missing stage 2 SAs, fix org-level IAM condition
* wip
* wip
* optional tag value creation in organization module
* stage 1 working
* linting
* linting
* READMEs
* wip
* Make stage-links script work in old macos bash
* stage links command help
* fix output file names
* diagrams
* fix svg
* stage 0 skeleton and diagram
* test svg
* test svg
* test diagram
* diagram
* readme
* fix stage links script
* stage 0 readme
* README changes
* stage readmes
* fix outputs order
* fix link
* fix tests
* stage 1 test
* skip stage example
* boilerplate
* fix tftest skip
* default bootstrap stage log sinks to log buckets
* add logging to tenant bootstrap
* move iam variables out of tenant config
* fix cicd, reintroduce missing variable
* use optional in stage 1 cicd variable
* rename extras stage
* rename and move identity providers local, use optional for cicd variable
* tfdoc
* add support for wif pool and providers, ci/cd
* tfdoc
* fix links
* better handling of modules repository
* add missing role on logging project
* fix cicd pools in locals, test cicd
* fix workflow extension
* fix module source replacement
* allow tenant bootstrap cicd sa to impersonate resman sa
* tenant workflow templates fix for no providers file
* fix output files, push github workflow template to new repository
* remove try from outpout files
* align stage 1 cicd internals to stage 0
* tfdoc
* tests
* fix tests
* tests
* improve variable descriptions
* use optional in fast features
* actually create tenant log sinks, and allow the resman sa to do it
* test
* tests
* aaaand tests again
* fast features tenant override
* fast features tenant override
* fix wording
* add missing comment
* configure pf service accounts
* add missing comment
* tfdoc
* tests
* IAM docs
* update copyright
---------
Co-authored-by: Julio Castillo <jccb@google.com>
2023-02-04 15:00:45 +01:00
Ayman Farhat
edd3a82453
Include cloudbuild API in project module ( #1116 )
...
* Include cloudbuild API in project module
* Increase number of resources
2023-01-27 21:38:01 +01:00
lcaggio
02938bdfa1
Fix Table in readme.
2023-01-16 15:58:11 +01:00
lcaggio
1672cfe9c1
Fix README.
2023-01-12 12:45:39 +01:00
lcaggio
50a52ae01a
Implement PR comments.
2023-01-12 12:41:00 +01:00
lcaggio
07a975f84e
Fix example test.
2023-01-11 12:37:08 +01:00
lcaggio
72abd47520
Fix documentation.
2023-01-11 08:47:42 +01:00
lcaggio
488b6d4dc0
Remove plg project, composer 2
2023-01-11 00:31:52 +01:00
Sebastian Kunze
e4fc47a063
Normalize prefix handling in blueprints ( #1003 )
2022-11-23 11:09:00 +01:00
Julio Castillo
826ff14ef9
Sort variables and outputs
2022-11-21 13:17:55 +01:00
Julio Castillo
7be3288cc7
Add tests for factory examples
2022-11-18 17:48:52 +01:00
Ludovico Magnocavallo
8c51a2a386
Enable org policy service and add README notice to modules ( #936 )
...
* enable org policy service and add README notice to modules
* fix tests
2022-11-01 14:25:07 +01:00
Julio Castillo
a988775c14
Do not modify in parameters in format_* functions
2022-10-20 18:56:54 +02:00
lcaggio
f28dc5cb04
Fix tests
2022-09-20 10:51:11 +02:00
Julio Castillo
7e9173d35d
Create fully recursive e2e test runner
2022-09-12 13:58:11 +02:00
Julio Castillo
0ee32be2fa
More updates 2
2022-09-09 16:40:37 +02:00
Julio Castillo
269894e92b
Update all internal links examples -> blueprints
2022-09-09 16:39:01 +02:00
Julio Castillo
a248602827
Rename examples folder to blueprints
2022-09-09 16:38:43 +02:00