zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,416 Commits 3 Branches 352 Tags 58 MiB
Commit Graph

12 Commits

Author SHA1 Message Date
simonebruzzechesse d11c380aec
Format python files in blueprints (#2079) 
* format python files in blueprints
* update check on blueprints python code
* update python linter in CI workflow
 2024-02-15 09:37:49 +01:00
Simone Ruffilli 6d89b88149
versions.tf maintenance + copyright notice bump (#1782) 
* Bump copyright notice to 2023

* Delete versions.tf on blueprints

* Pin provider to major version 5

* Remove comment

* Fix lint

* fix bq-ml blueprint readme

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
 2023-10-20 18:17:47 +02:00
lcaggio 6889f02954
Fix data platform roles (#1725) 
* Fix Data Platform roles

* Fix README

* Fix blueprint tests

* Update cleanup dp steps

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2023-10-04 07:31:40 +02:00
Pierre Formont 0e7cfc8b29
add AIRFLOW_VAR_ prefix to environment variables in data-platform blueprints (#1651) 
* add AIRFLOW_VAR_ prefix to env vars in minimal blueprint

* update DAGs for env vars in minimal blueprint

* add AIRFLOW_VAR_ prefix to env vars in foundation blueprint

* update DAGs for env vars in foundation blueprint

* apply tf linting

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2023-09-08 07:38:28 +00:00
Daniel De Leo 009c6d0f93
Make demo pipeline append into BQ tables (#1533) 2023-07-27 11:38:00 -04:00
Alejandro Leal 87cd83f5c0 Several updates 
Several updates
 2023-05-13 23:51:46 -04:00
lcaggio 2108b4650d Fix Tests, rely on iam additive. 2023-02-22 01:36:01 +01:00
Ayman Farhat eac2065ae2 Update on docs wording 2023-02-16 17:04:05 +01:00
Ayman Farhat fcdadf521d Fix spelling mistake 2023-02-16 15:48:12 +01:00
Ayman Farhat a853dc4fe2 update Data Platform blueprint README with more example Dataflow commands 2023-02-16 15:45:20 +01:00
Ayman Farhat 02d8d8367a
[Feature] Update data platform blue print with Dataflow Flex template (#1105) 
* Add initial dataflow template code + TF infra

* Refactor the datapipeline DAG to use flex template operator, cleanup code

* Remove unneeded bash scripts, update README with manual examples

* Refactor datapipeline_dc_tags.py and include new Flex template

* Update docs to reflect changes

* Remove sub-dependencies and keep apache beam

* Add missing license headers and update tests

* Set resouces to 291 in tests

* Update outputs via tfdoc

* Update with outputs order and tfdoc

* Correct number of resources

* Fix to add region into command from var

* Enable service account impersonation for running builds

* Update example dataflow run command to use orchestrator SA

* Remove hard coded values in example

* Keep original airflow files, add new which use Flex template as example

* Update tests and doc

* Fix number of resources in plan

* Run tfdoc remove files section in README

* Fix number of modules in tfdoc

* Update number of resources

* Add missin service account

* Update DF demo README

* Quick rename

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2023-02-06 07:35:40 +01:00
Julio Castillo a248602827 Rename examples folder to blueprints 2022-09-09 16:38:43 +02:00