zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
cloud-foundation-fabric/data-solutions
History
Yoram Ben-Yaacov fc987a9a5d
Merged all components into one main file. (#259) 
* Merged all components into one main file.

* rename datasource folder to resources

* resources output added for:
* datamart-bigquery-datasets-list
* dwh-bigquery-datasets-list
* landing-bucket-names
* landing-pubsub-list
* transformation-bucket-names

* Change the project ID with a link to the module

* add support for IAM roles to the datasets created

* Removed unused local variable module_version

* Moved from  access and access_identities to IAM.

* Update README.md

Co-authored-by: lcaggio <lcaggioni@google.com>
 		2021-06-10 09:58:40 +02:00
..
cmek-via-centralized-kms Update copyright to 2021 2021-02-15 09:38:10 +01:00
data-platform-foundations Merged all components into one main file. (#259) 2021-06-10 09:58:40 +02:00
gcs-to-bq-with-dataflow Avoid data sources in in gcs-to-bq-with-dataflow example 2021-02-15 18:15:20 +01:00
README.md Add Data Platform Foundations description to the data-solutions README file 2021-05-18 20:00:45 +03:00

README.md

GCP Data Services examples

The examples in this folder implement typical data service topologies and end-to-end scenarios, that allow testing specific features like Cloud KMS to encrypt your data, or VPC-SC to mitigate data exfiltration.

They are meant to be used as minimal but complete starting points to create actual infrastructure, and as playgrounds to experiment with specific Google Cloud features.

Examples

GCE and GCS CMEK via centralized Cloud KMS

This example implements CMEK for GCS and GCE, via keys hosted in KMS running in a centralized project. The example shows the basic resources and permissions for the typical use case of application projects implementing encryption at rest via a centrally managed KMS service.

Cloud Storage to Bigquery with Cloud Dataflow

This example implements Cloud Storage to Bigquery data import using Cloud Dataflow. All resources use CMEK hosted in Cloud KMS running in a centralized project. The example shows the basic resources and permissions for the typical use case to read, transform and import data from Cloud Storage to Bigquery.

Data Platform Foundations

This example implements a robust and flexible Data Foundation on GCP that provides opinionated defaults while allowing customers to build and scale out additional data pipelines quickly and reliably.