Merge pull request #610 from GoogleCloudPlatform/sql-multi-region
Multi-region Cloud-SQL example
This commit is contained in:
Julio Castillo
GitHub
commit
8f9208fb9e
|
|
@ -28,3 +28,4 @@ fast/stages/**/terraform-*.auto.tfvars.json
|
|||
fast/stages/**/0*.auto.tfvars*
|
||||
**/node_modules
|
||||
fast/stages/**/globals.auto.tfvars.json
|
||||
cloud_sql_proxy
|
||||
|
|
@ -5,7 +5,7 @@ This section contains **[foundational examples](./foundations/)** that bootstrap
|
|||
Currently available examples:
|
||||
|
||||
- **cloud operations** - [Resource tracking and remediation via Cloud Asset feeds](./cloud-operations/asset-inventory-feed-remediation), [Granular Cloud DNS IAM via Service Directory](./cloud-operations/dns-fine-grained-iam), [Granular Cloud DNS IAM for Shared VPC](./cloud-operations/dns-shared-vpc), [Compute Engine quota monitoring](./cloud-operations/quota-monitoring), [Scheduled Cloud Asset Inventory Export to Bigquery](./cloud-operations/scheduled-asset-inventory-export-bq), [Packer image builder](./cloud-operations/packer-image-builder), [On-prem SA key management](./cloud-operations/onprem-sa-key-management), [TCP healthcheck for unmanaged GCE instances](./cloud-operations/unmanaged-instances-healthcheck)
|
||||
- **data solutions** - [GCE/GCS CMEK via centralized Cloud KMS](./data-solutions/gcs-to-bq-with-least-privileges/), [Cloud Storage to Bigquery with Cloud Dataflow with least privileges](./data-solutions/gcs-to-bq-with-least-privileges/), [Data Platform Foundations](./data-solutions/data-platform-foundations/), [SQL Server AlwaysOn availability groups example](./data-solutions/sqlserver-alwayson)
|
||||
- **data solutions** - [GCE/GCS CMEK via centralized Cloud KMS](./data-solutions/gcs-to-bq-with-least-privileges/), [Cloud Storage to Bigquery with Cloud Dataflow with least privileges](./data-solutions/gcs-to-bq-with-least-privileges/), [Data Platform Foundations](./data-solutions/data-platform-foundations/), [SQL Server AlwaysOn availability groups example](./data-solutions/sqlserver-alwayson), [Cloud SQL instance with multi-region read replicas](./data-solutions/cloudsql-multiregion/)
|
||||
- **factories** - [The why and the how of resource factories](./factories/README.md)
|
||||
- **foundations** - [single level hierarchy](./foundations/environments/) (environments), [multiple level hierarchy](./foundations/business-units/) (business units + environments)
|
||||
- **networking** - [hub and spoke via peering](./networking/hub-and-spoke-peering/), [hub and spoke via VPN](./networking/hub-and-spoke-vpn/), [DNS and Google Private Access for on-premises](./networking/onprem-google-access-dns/), [Shared VPC with GKE support](./networking/shared-vpc-gke/), [ILB as next hop](./networking/ilb-next-hop), [PSC for on-premises Cloud Function invocation](./networking/private-cloud-function-from-onprem/), [decentralized firewall](./networking/decentralized-firewall)
|
||||
|
|
|
|||
|
|
@ -24,6 +24,12 @@ This [example](./data-platform-foundations/) implements a robust and flexible Da
|
|||
|
||||
### SQL Server Always On Availability Groups
|
||||
|
||||
<a href="./sqlserver-alwayson/" title="Data Platform Foundations"><img src="https://cloud.google.com/compute/images/sqlserver-ag-architecture.svg" align="left" width="280px"></a>
|
||||
<a href="./sqlserver-alwayson/" title="SQL Server Always On Availability Groups"><img src="https://cloud.google.com/compute/images/sqlserver-ag-architecture.svg" align="left" width="280px"></a>
|
||||
This [example](./data-platform-foundations/) implements SQL Server Always On Availability Groups using Fabric modules. It builds a two node cluster with a fileshare witness instance in an existing VPC and adds the necessary firewalling. The actual setup process (apart from Active Directory operations) has been scripted, so that least amount of manual works needs to performed.
|
||||
<br clear="left">
|
||||
|
||||
### Cloud SQL instance with multi-region read replicas
|
||||
|
||||
<a href="./cloudsql-multiregion/" title="Cloud SQL instance with multi-region read replicas"><img src="./cloudsql-multiregion/diagram.png" align="left" width="280px"></a>
|
||||
This [example](./cloudsql-multiregion/) creates a [Cloud SQL instance](https://cloud.google.com/sql) with multi-region read replicas as described in the [Cloud SQL for PostgreSQL disaster recovery](https://cloud.google.com/architecture/cloud-sql-postgres-disaster-recovery-complete-failover-fallback) article.
|
||||
<br clear="left">
|
||||
|
|
@ -0,0 +1,88 @@
|
|||
# Cloud SQL instance with multi-region read replicas
|
||||
|
||||
This example creates a [Cloud SQL instance](https://cloud.google.com/sql) with multi-region read replicas as described in the [Cloud SQL for PostgreSQL disaster recovery](https://cloud.google.com/architecture/cloud-sql-postgres-disaster-recovery-complete-failover-fallback) article.
|
||||
|
||||
The solution is resilient to a regional outage. To get familiar with the procedure needed in the unfortunate case of a disaster recovery, please follow steps described in [part two](https://cloud.google.com/architecture/cloud-sql-postgres-disaster-recovery-complete-failover-fallback#phase-2) of the aforementioned article.
|
||||
|
||||
The solution will use:
|
||||
- VPC with Private Service Access to deploy the instances and VM
|
||||
- Cloud SQL - Postgre SQL instanced with Private IP
|
||||
- Goocle Cloud Storage bucket to handle database import/export
|
||||
- Google Cloud Engine instance to connect to the Posgre SQL instance
|
||||
- Google Cloud NAT to access internet resources
|
||||
|
||||
This is the high level diagram:
|
||||
|
||||
|
||||
|
||||
# Requirements
|
||||
|
||||
This example will deploy all its resources into the project defined by the `project_id` variable. Please note that we assume this project already exists. However, if you provide the appropriate values to the `project_create` variable, the project will be created as part of the deployment.
|
||||
|
||||
If `project_create` is left to `null`, the identity performing the deployment needs the `owner` role on the project defined by the `project_id` variable. Otherwise, the identity performing the deployment needs `resourcemanager.projectCreator` on the resource hierarchy node specified by `project_create.parent` and `billing.user` on the billing account specified by `project_create.billing_account_id`.
|
||||
|
||||
|
||||
## Deployment
|
||||
|
||||
Configure the Terraform variables in your `terraform.tfvars` file. You need to specify at least the `project_id` and `prefix` variables. See [`terraform.tfvars.sample`](terraform.tfvars.sample) as starting point.
|
||||
|
||||
Run Terraform init:
|
||||
|
||||
```
|
||||
$ terraform init
|
||||
$ terraform apply
|
||||
```
|
||||
|
||||
You should see the output of the Terraform script with resources created and some commands that you'll need in the following steps below.
|
||||
|
||||
## Move to real use case consideration
|
||||
|
||||
This implementation is intentionally minimal and easy to read. A real world use case should consider:
|
||||
- Using a Shared VPC
|
||||
- Using VPC-SC to mitigate data exfiltration
|
||||
|
||||
## Test your environment
|
||||
We assume all those steps are run using a user listed on `data_eng_principals`. You can authenticate as the user using the following command:
|
||||
|
||||
```
|
||||
$ gcloud init
|
||||
$ gcloud auth application-default login
|
||||
```
|
||||
|
||||
Below you can find commands to connect to the VM instance and Cloud SQL instance.
|
||||
|
||||
```
|
||||
$ gcloud compute ssh sql-test --project PROJECT_ID --zone ZONE
|
||||
sql-test:~$ cloud_sql_proxy -instances=CLOUDSQL_INSTANCE=tcp:5432
|
||||
sql-test:~$ psql 'host=127.0.0.1 port=5432 sslmode=disable dbname=DATABASE user=USER'
|
||||
```
|
||||
|
||||
You can find computed commands on the Terraform `demo_commands` output.
|
||||
<!-- BEGIN TFDOC -->
|
||||
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---:|:---:|:---:|
|
||||
| [postgres_user_password](variables.tf#L29) | `postgres` user password. | <code>string</code> | ✓ | |
|
||||
| [prefix](variables.tf#L40) | Unique prefix used for resource names. Not used for project if 'project_create' is null. | <code>string</code> | ✓ | |
|
||||
| [project_id](variables.tf#L54) | Project id, references existing project if `project_create` is null. | <code>string</code> | ✓ | |
|
||||
| [cmek_encryption](variables.tf#L17) | Flag to enable CMEK on GCP resources created. | <code>bool</code> | | <code>false</code> |
|
||||
| [data_eng_principals](variables.tf#L23) | Groups with Service Account Token creator role on service accounts in IAM format, only user supported on CloudSQL, eg 'user@domain.com'. | <code>list(string)</code> | | <code>[]</code> |
|
||||
| [postgres_database](variables.tf#L34) | `postgres` database. | <code>string</code> | | <code>"guestbook"</code> |
|
||||
| [project_create](variables.tf#L45) | Provide values if project creation is needed, uses existing project if null. Parent is in 'folders/nnn' or 'organizations/nnn' format. | <code title="object({ billing_account_id = string parent = string })">object({…})</code> | | <code>null</code> |
|
||||
| [regions](variables.tf#L59) | Map of instance_name => location where instances will be deployed. | <code>map(string)</code> | | <code title="{ primary = "europe-west1" replica = "europe-west3" }">{…}</code> |
|
||||
| [sql_configuration](variables.tf#L73) | Cloud SQL configuration | <code title="object({ availability_type = string database_version = string psa_range = string tier = string })">object({…})</code> | | <code title="{ availability_type = "REGIONAL" database_version = "POSTGRES_13" psa_range = "10.60.0.0/16" tier = "db-g1-small" }">{…}</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| [bucket](outputs.tf#L22) | Cloud storage bucket to import/export data from Cloud SQL. | |
|
||||
| [connection_names](outputs.tf#L17) | Connection name of each instance. | |
|
||||
| [demo_commands](outputs.tf#L37) | Demo commands. | |
|
||||
| [ips](outputs.tf#L27) | IP address of each instance. | |
|
||||
| [project_id](outputs.tf#L32) | ID of the project containing all the instances. | |
|
||||
| [service_accounts](outputs.tf#L46) | Service Accounts. | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# The `impersonate_service_account` option require the identity launching terraform
|
||||
# role `roles/iam.serviceAccountTokenCreator` on the Service Account specified.
|
||||
|
||||
terraform {
|
||||
backend "gcs" {
|
||||
bucket = "BUCKET_NAME"
|
||||
prefix = "PREFIX"
|
||||
impersonate_service_account = "SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"
|
||||
}
|
||||
}
|
||||
provider "google" {
|
||||
impersonate_service_account = "SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"
|
||||
}
|
||||
provider "google-beta" {
|
||||
impersonate_service_account = "SERVICE_ACCOUNT@PROJECT_ID.iam.gserviceaccount.com"
|
||||
}
|
||||
|
|
@ -0,0 +1,62 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
module "db" {
|
||||
source = "../../../modules/cloudsql-instance"
|
||||
project_id = module.project.project_id
|
||||
availability_type = var.sql_configuration.availability_type
|
||||
encryption_key_name = var.cmek_encryption ? module.kms[var.regions.primary].keys.key.id : null
|
||||
network = module.vpc.self_link
|
||||
name = "${var.prefix}-db"
|
||||
region = var.regions.primary
|
||||
database_version = var.sql_configuration.database_version
|
||||
tier = var.sql_configuration.tier
|
||||
flags = {
|
||||
"cloudsql.iam_authentication" = "on"
|
||||
}
|
||||
replicas = {
|
||||
for k, v in var.regions :
|
||||
k => {
|
||||
region = v,
|
||||
encryption_key_name = var.cmek_encryption ? module.kms[v].keys.key.id : null
|
||||
} if k != "primary"
|
||||
}
|
||||
databases = [var.postgres_database]
|
||||
users = {
|
||||
postgres = var.postgres_user_password
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_sql_user" "users" {
|
||||
for_each = toset(var.data_eng_principals)
|
||||
project = module.project.project_id
|
||||
name = each.value
|
||||
instance = module.db.name
|
||||
type = "CLOUD_IAM_USER"
|
||||
}
|
||||
|
||||
resource "google_sql_user" "service-account" {
|
||||
for_each = toset(var.data_eng_principals)
|
||||
project = module.project.project_id
|
||||
# Omit the .gserviceaccount.com suffix in the email
|
||||
name = regex("(.+)(gserviceaccount)", module.service-account-sql.email)[0]
|
||||
instance = module.db.name
|
||||
type = "CLOUD_IAM_SERVICE_ACCOUNT"
|
||||
}
|
||||
|
||||
module "service-account-sql" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
project_id = module.project.project_id
|
||||
name = "${var.prefix}-sql"
|
||||
}
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
module "gcs" {
|
||||
source = "../../../modules/gcs"
|
||||
project_id = module.project.project_id
|
||||
prefix = var.prefix
|
||||
name = "data"
|
||||
location = var.regions.primary
|
||||
storage_class = "REGIONAL"
|
||||
encryption_key = var.cmek_encryption ? module.kms[var.regions.primary].keys["key"].id : null
|
||||
force_destroy = true
|
||||
}
|
||||
|
||||
module "service-account-gcs" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
project_id = module.project.project_id
|
||||
name = "${var.prefix}-gcs"
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 33 KiB |
|
|
@ -0,0 +1,60 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
locals {
|
||||
startup-script = <<END
|
||||
#! /bin/bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y postgresql-client wget
|
||||
sudo wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 -O /usr/local/bin/cloud_sql_proxy
|
||||
sudo chmod a+x /usr/local/bin/cloud_sql_proxy
|
||||
sudo ln -s /usr/local/bin/cloud_sql_proxy /usr/lib/google-cloud-sdk/bin/cloud_sql_proxy
|
||||
END
|
||||
}
|
||||
|
||||
module "test-vm" {
|
||||
source = "../../../modules/compute-vm"
|
||||
project_id = module.project.project_id
|
||||
zone = "${var.regions.primary}-b"
|
||||
name = "sql-test"
|
||||
network_interfaces = [{
|
||||
network = module.vpc.self_link
|
||||
subnetwork = module.vpc.subnets["${var.regions.primary}/subnet"].self_link
|
||||
nat = false
|
||||
addresses = null
|
||||
}]
|
||||
attached_disks = [
|
||||
{
|
||||
name = "attached-disk"
|
||||
size = 10
|
||||
source = null
|
||||
source_type = null
|
||||
options = null
|
||||
}
|
||||
]
|
||||
service_account = module.service-account-sql.email
|
||||
service_account_scopes = ["https://www.googleapis.com/auth/cloud-platform"]
|
||||
boot_disk = {
|
||||
image = "projects/debian-cloud/global/images/family/debian-10"
|
||||
type = "pd-ssd"
|
||||
size = 10
|
||||
}
|
||||
encryption = var.cmek_encryption ? {
|
||||
encrypt_boot = true
|
||||
disk_encryption_key_raw = null
|
||||
kms_key_self_link = var.cmek_encryption ? module.kms[var.regions.primary].keys["key"].id : null
|
||||
} : null
|
||||
metadata = { startup-script = local.startup-script }
|
||||
tags = ["ssh"]
|
||||
}
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
module "kms" {
|
||||
for_each = toset(distinct(values(var.regions)))
|
||||
source = "../../../modules/kms"
|
||||
project_id = module.project.project_id
|
||||
|
||||
keyring = {
|
||||
name = "${var.prefix}-keyring-${each.value}"
|
||||
location = each.value
|
||||
}
|
||||
|
||||
keys = {
|
||||
key = null
|
||||
}
|
||||
key_iam = {
|
||||
key = {
|
||||
"roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
|
||||
"serviceAccount:${module.project.service_accounts.robots.compute}",
|
||||
"serviceAccount:${module.project.service_accounts.robots.sql}",
|
||||
"serviceAccount:${module.project.service_accounts.robots.storage}"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,117 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
locals {
|
||||
data_eng_principals_iam = [
|
||||
for k in var.data_eng_principals :
|
||||
"user:${k}"
|
||||
]
|
||||
|
||||
iam = {
|
||||
# GCS roles
|
||||
"roles/storage.objectAdmin" = [
|
||||
"serviceAccount:${module.project.service_accounts.robots.sql}",
|
||||
module.service-account-gcs.iam_email,
|
||||
]
|
||||
# CloudSQL
|
||||
"roles/cloudsql.admin" = local.data_eng_principals_iam
|
||||
"roles/cloudsql.client" = concat(
|
||||
local.data_eng_principals_iam,
|
||||
[module.service-account-sql.iam_email]
|
||||
)
|
||||
"roles/cloudsql.instanceUser" = concat(
|
||||
local.data_eng_principals_iam,
|
||||
[module.service-account-sql.iam_email]
|
||||
)
|
||||
# compute engeneering
|
||||
"roles/compute.instanceAdmin.v1" = local.data_eng_principals_iam
|
||||
"roles/compute.osLogin" = local.data_eng_principals_iam
|
||||
"roles/compute.viewer" = local.data_eng_principals_iam
|
||||
"roles/iap.tunnelResourceAccessor" = local.data_eng_principals_iam
|
||||
# common roles
|
||||
"roles/logging.admin" = local.data_eng_principals_iam
|
||||
"roles/iam.serviceAccountUser" = concat(
|
||||
local.data_eng_principals_iam
|
||||
)
|
||||
"roles/iam.serviceAccountTokenCreator" = concat(
|
||||
local.data_eng_principals_iam
|
||||
)
|
||||
# network roles
|
||||
"roles/compute.networkUser" = [
|
||||
"serviceAccount:${module.project.service_accounts.robots.sql}"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
module "project" {
|
||||
source = "../../../modules/project"
|
||||
name = var.project_id
|
||||
parent = try(var.project_create.parent, null)
|
||||
billing_account = try(var.project_create.billing_account_id, null)
|
||||
project_create = var.project_create != null
|
||||
prefix = var.project_create == null ? null : var.prefix
|
||||
iam = var.project_create != null ? local.iam : {}
|
||||
iam_additive = var.project_create == null ? local.iam : {}
|
||||
services = [
|
||||
"cloudkms.googleapis.com",
|
||||
"iap.googleapis.com",
|
||||
"logging.googleapis.com",
|
||||
"monitoring.googleapis.com",
|
||||
"networkmanagement.googleapis.com",
|
||||
"servicenetworking.googleapis.com",
|
||||
"sqladmin.googleapis.com",
|
||||
"sql-component.googleapis.com",
|
||||
"storage.googleapis.com",
|
||||
"storage-component.googleapis.com",
|
||||
]
|
||||
service_config = {
|
||||
disable_on_destroy = false, disable_dependent_services = false
|
||||
}
|
||||
}
|
||||
|
||||
module "vpc" {
|
||||
source = "../../../modules/net-vpc"
|
||||
project_id = module.project.project_id
|
||||
name = "vpc"
|
||||
subnets = [
|
||||
{
|
||||
ip_cidr_range = "10.0.0.0/20"
|
||||
name = "subnet"
|
||||
region = var.regions.primary
|
||||
secondary_ip_range = {}
|
||||
}
|
||||
]
|
||||
|
||||
psa_config = {
|
||||
ranges = { cloud-sql = var.sql_configuration.psa_range }
|
||||
routes = null
|
||||
}
|
||||
}
|
||||
|
||||
module "firewall" {
|
||||
source = "../../../modules/net-vpc-firewall"
|
||||
project_id = module.project.project_id
|
||||
network = module.vpc.name
|
||||
admin_ranges = ["10.0.0.0/20"]
|
||||
}
|
||||
|
||||
module "nat" {
|
||||
source = "../../../modules/net-cloudnat"
|
||||
project_id = module.project.project_id
|
||||
region = var.regions.primary
|
||||
name = "${var.prefix}-default"
|
||||
router_network = module.vpc.name
|
||||
}
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
output "connection_names" {
|
||||
description = "Connection name of each instance."
|
||||
value = module.db.connection_names
|
||||
}
|
||||
|
||||
output "bucket" {
|
||||
description = "Cloud storage bucket to import/export data from Cloud SQL."
|
||||
value = module.gcs.name
|
||||
}
|
||||
|
||||
output "ips" {
|
||||
description = "IP address of each instance."
|
||||
value = module.db.ips
|
||||
}
|
||||
|
||||
output "project_id" {
|
||||
description = "ID of the project containing all the instances."
|
||||
value = module.project.project_id
|
||||
}
|
||||
|
||||
output "demo_commands" {
|
||||
description = "Demo commands."
|
||||
value = {
|
||||
"01_ssh" = "gcloud compute ssh ${module.test-vm.instance.name} --project ${module.project.name} --zone ${var.regions.primary}-b"
|
||||
"02_cloud_sql_proxy" = "cloud_sql_proxy -instances=${module.db.connection_name}=tcp:5432 &"
|
||||
"03_psql" = "psql 'host=127.0.0.1 port=5432 sslmode=disable dbname=${var.postgres_database} user=postgres'"
|
||||
}
|
||||
}
|
||||
|
||||
output "service_accounts" {
|
||||
description = "Service Accounts."
|
||||
value = {
|
||||
"gcs" = module.service-account-gcs.email
|
||||
"sql" = module.service-account-sql.email
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
project_id = "datalake-001"
|
||||
prefix = "prefix"
|
||||
|
|
@ -0,0 +1,87 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
variable "cmek_encryption" {
|
||||
description = "Flag to enable CMEK on GCP resources created."
|
||||
type = bool
|
||||
default = false
|
||||
}
|
||||
|
||||
variable "data_eng_principals" {
|
||||
description = "Groups with Service Account Token creator role on service accounts in IAM format, only user supported on CloudSQL, eg 'user@domain.com'."
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "postgres_user_password" {
|
||||
description = "`postgres` user password."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "postgres_database" {
|
||||
description = "`postgres` database."
|
||||
type = string
|
||||
default = "guestbook"
|
||||
}
|
||||
|
||||
variable "prefix" {
|
||||
description = "Unique prefix used for resource names. Not used for project if 'project_create' is null."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "project_create" {
|
||||
description = "Provide values if project creation is needed, uses existing project if null. Parent is in 'folders/nnn' or 'organizations/nnn' format."
|
||||
type = object({
|
||||
billing_account_id = string
|
||||
parent = string
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "project_id" {
|
||||
description = "Project id, references existing project if `project_create` is null."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "regions" {
|
||||
description = "Map of instance_name => location where instances will be deployed."
|
||||
type = map(string)
|
||||
validation {
|
||||
condition = contains(keys(var.regions), "primary")
|
||||
error_message = "Regions map must contain `primary` as a key."
|
||||
}
|
||||
default = {
|
||||
primary = "europe-west1"
|
||||
replica = "europe-west3"
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
variable "sql_configuration" {
|
||||
description = "Cloud SQL configuration"
|
||||
type = object({
|
||||
availability_type = string
|
||||
database_version = string
|
||||
psa_range = string
|
||||
tier = string
|
||||
})
|
||||
default = {
|
||||
availability_type = "REGIONAL"
|
||||
database_version = "POSTGRES_13"
|
||||
psa_range = "10.60.0.0/16"
|
||||
tier = "db-g1-small"
|
||||
}
|
||||
}
|
||||
|
|
@ -19,6 +19,10 @@ locals {
|
|||
is_mysql = can(regex("^MYSQL", var.database_version))
|
||||
has_replicas = try(length(var.replicas) > 0, false)
|
||||
|
||||
// Enable backup if the user asks for it or if the user is deploying
|
||||
// MySQL with replicas
|
||||
enable_backup = var.backup_configuration.enabled || (local.is_mysql && local.has_replicas)
|
||||
|
||||
users = {
|
||||
for user, password in coalesce(var.users, {}) :
|
||||
(user) => (
|
||||
|
|
@ -67,24 +71,25 @@ resource "google_sql_database_instance" "primary" {
|
|||
}
|
||||
}
|
||||
|
||||
backup_configuration {
|
||||
// Enable backup if the user asks for it or if the user is
|
||||
// deploying MySQL with replicas
|
||||
enabled = var.backup_configuration.enabled || (local.is_mysql && local.has_replicas)
|
||||
dynamic "backup_configuration" {
|
||||
for_each = local.enable_backup ? { 1 = 1 } : {}
|
||||
content {
|
||||
enabled = true
|
||||
|
||||
// enable binary log if the user asks for it or we have replicas,
|
||||
// but only form MySQL
|
||||
binary_log_enabled = (
|
||||
local.is_mysql
|
||||
? var.backup_configuration.binary_log_enabled || local.has_replicas
|
||||
: null
|
||||
)
|
||||
start_time = var.backup_configuration.start_time
|
||||
location = var.backup_configuration.location
|
||||
transaction_log_retention_days = var.backup_configuration.log_retention_days
|
||||
backup_retention_settings {
|
||||
retained_backups = var.backup_configuration.retention_count
|
||||
retention_unit = "COUNT"
|
||||
// enable binary log if the user asks for it or we have replicas,
|
||||
// but only for MySQL
|
||||
binary_log_enabled = (
|
||||
local.is_mysql
|
||||
? var.backup_configuration.binary_log_enabled || local.has_replicas
|
||||
: null
|
||||
)
|
||||
start_time = var.backup_configuration.start_time
|
||||
location = var.backup_configuration.location
|
||||
transaction_log_retention_days = var.backup_configuration.log_retention_days
|
||||
backup_retention_settings {
|
||||
retained_backups = var.backup_configuration.retention_count
|
||||
retention_unit = "COUNT"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,13 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
module "test" {
|
||||
source = "../../../../../examples/data-solutions/cloudsql-multiregion/"
|
||||
data_eng_principals = ["dataeng@example.com"]
|
||||
postgres_user_password = "my-root-password"
|
||||
project_id = "project"
|
||||
project_create = {
|
||||
billing_account_id = "123456-123456-123456"
|
||||
parent = "folders/12345678"
|
||||
}
|
||||
prefix = "prefix"
|
||||
}
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
# Copyright 2022 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
def test_resources(e2e_plan_runner):
|
||||
"Test that plan works and the numbers of resources is as expected."
|
||||
modules, resources = e2e_plan_runner()
|
||||
assert len(modules) == 11
|
||||
assert len(resources) == 53
|
||||
Loading…
Reference in New Issue