History

Ayman Farhat 54d805dac0 New example for a data playground Terraform setup (#655 ) * Initial commit for adding a sample data playground * Update README * Add license boilerplate to variables.tf * Apply linting rules * rename var to ptoject_id, create prefix var, remove extra zone var * Adds the option for using an existing project by default * Bundles all VPC related variables in a single vpc_config variable of type object * Add encryption_key usage example + policy_boolean * Add tests, apply linting and todos for upcoming PRs * Update variables in readme * Fix formatting via fmt * Rename test dir to fix module conflict issue * Add high level diagram and sort vars/outputs by alphabetical * Modify diagram and update main README under data examples with link / summary * Line break * Use png in diagram Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>		2022-07-10 09:27:18 +02:00
..
cloudsql-multiregion	Mention Cloud NAT in the readme	2022-04-14 10:58:12 +02:00
cmek-via-centralized-kms	Bump providers versions and pin versions for tests	2022-06-28 10:20:40 +02:00
data-platform-foundations	Update naming convention	2022-04-21 23:53:16 +02:00
data-playground	New example for a data playground Terraform setup (#655 )	2022-07-10 09:27:18 +02:00
gcs-to-bq-with-least-privileges	Bump providers versions and pin versions for tests	2022-06-28 10:20:40 +02:00
sqlserver-alwayson	Use interpolation instead of format()	2022-05-05 14:45:34 +02:00
README.md	New example for a data playground Terraform setup (#655 )	2022-07-10 09:27:18 +02:00

README.md

GCP Data Services examples

The examples in this folder implement typical data service topologies and end-to-end scenarios, that allow testing specific features like Cloud KMS to encrypt your data, or VPC-SC to mitigate data exfiltration.

They are meant to be used as minimal but complete starting points to create actual infrastructure, and as playgrounds to experiment with specific Google Cloud features.

Examples

GCE and GCS CMEK via centralized Cloud KMS

This example implements CMEK for GCS and GCE, via keys hosted in KMS running in a centralized project. The example shows the basic resources and permissions for the typical use case of application projects implementing encryption at rest via a centrally managed KMS service.

Cloud Storage to Bigquery with Cloud Dataflow with least privileges

This example implements resources required to run GCS to BigQuery Dataflow pipelines. The solution rely on a set of Services account created with the least privileges principle.

Data Platform Foundations

This example implements a robust and flexible Data Foundation on GCP that provides opinionated defaults, allowing customers to build and scale out additional data pipelines quickly and reliably.

SQL Server Always On Availability Groups

This example implements SQL Server Always On Availability Groups using Fabric modules. It builds a two node cluster with a fileshare witness instance in an existing VPC and adds the necessary firewalling. The actual setup process (apart from Active Directory operations) has been scripted, so that least amount of manual works needs to performed.

Cloud SQL instance with multi-region read replicas

This example creates a Cloud SQL instance with multi-region read replicas as described in the Cloud SQL for PostgreSQL disaster recovery article.

Data Playground starter with Cloud Vertex AI Notebook and GCS

This example creates a Vertex AI Notebook running under a VPC network and a starter GCS bucket to store inputs and outputs of data experiments.