Merge pull request #94 from terraform-google-modules/bigtable

Add Bigtable instance module

CHANGELOG.md

@@ -3,6 +3,7 @@
 All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
+- new `bigtable-instance` module
 
 ## [1.8.1] - 2020-06-07
 

README.md

@@ -35,7 +35,7 @@ Currently available modules:
 - **foundational** - [folders](./modules/folders), [log sinks](./modules/logging-sinks), [organization](./modules/organization), [project](./modules/project), [service accounts](./modules/iam-service-accounts)
 - **networking** - [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC peering](./modules/net-vpc-peering), [VPN static](./modules/net-vpn-static), [VPN dynamic](./modules/net-vpn-dynamic), [VPN HA](./modules/net-vpn-ha), [NAT](./modules/net-cloudnat), [address reservation](./modules/net-address), [DNS](./modules/dns), [L4 ILB](./modules/net-ilb), [Service Directory](./modules/service-directory)
 - **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [GKE cluster](./modules/gke-cluster), [GKE nodepool](./modules/gke-nodepool), [COS container](./modules/cos-container) (coredns, mysql, onprem, squid)
-- **data** - [GCS](./modules/gcs), [BigQuery dataset](./modules/bigquery-dataset), [Pub/Sub](./modules/pubsub), [Datafusion](./modules/datafusion)
+- **data** - [GCS](./modules/gcs), [BigQuery dataset](./modules/bigquery-dataset), [Pub/Sub](./modules/pubsub), [Datafusion](./modules/datafusion), [Bigtable instance](./modules/bigtable-instance)
 - **security** - [KMS](./modules/kms), [SecretManager](./modules/secret-manager)
 - **development** - [Cloud Source Repository](./modules/source-repository), [Container Registry](./modules/container-registry), [Artifact Registry](./modules/artifact-registry)
 

modules/README.md

@@ -45,6 +45,7 @@ Specific modules also offer support for non-authoritative bindings (e.g. `google
 - [Datafusion](./datafusion)
 - [GCS](./gcs)
 - [Pub/Sub](./pubsub)
+- [Bigtable instance](./bigtable-instance)
 
 ## Development
 

modules/bigtable-instance/README.md

@@ -0,0 +1,65 @@
+# Google Cloud BigTable Module
+
+This module allows managing a single BigTable instance, including access configuration and tables.
+
+## TODO
+
+- [ ] support bigtable_gc_policy
+- [ ] support bigtable_app_profile
+
+## Examples
+
+### Simple instance with access configuration
+
+```hcl
+
+module "big-table-instance" {
+  source               = "./modules/bigtable-instance"
+  project_id           = "my-project"
+  name                 = "instance"
+  cluster_id           = "instance"
+  instance_type        = "PRODUCTION"
+  tables               = {
+    test1 = { table_options = null },
+    test2 = { table_options = {
+      split_keys    = ["a", "b", "c"]
+      column_family = null
+      }
+    }
+  }
+  iam_roles         = ["viewer"]
+  iam_members       = {
+    viewer = ["user:viewer@testdomain.com"]
+  }
+}
+```
+
+<!-- BEGIN TFDOC -->
+## Variables
+
+| name | description | type | required | default |
+|---|---|:---: |:---:|:---:|
+| name | The name of the Cloud Bigtable instance. | <code title="">string</code> | ✓ |  |
+| project_id | Id of the project where datasets will be created. | <code title="">string</code> | ✓ |  |
+| zone | The zone to create the Cloud Bigtable cluster in. | <code title="">string</code> | ✓ |  |
+| *cluster_id* | The ID of the Cloud Bigtable cluster. | <code title="">string</code> |  | <code title="">europe-west1</code> |
+| *deletion_protection* | Whether or not to allow Terraform to destroy the instance. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the instance will fail. | <code title=""></code> |  | <code title="">true</code> |
+| *display_name* | The human-readable display name of the Bigtable instance. | <code title=""></code> |  | <code title="">null</code> |
+| *iam_members* | Authoritative for a given role. Updates the IAM policy to grant a role to a list of members. Other roles within the IAM policy for the instance are preserved. | <code title="map&#40;list&#40;string&#41;&#41;">map(list(string))</code> |  | <code title="">{}</code> |
+| *iam_roles* | Authoritative for a given role. Updates the IAM policy to grant a role to a list of members. | <code title="list&#40;string&#41;">list(string)</code> |  | <code title="">[]</code> |
+| *instance_type* | None | <code title="">string</code> |  | <code title="">DEVELOPMENT</code> |
+| *num_nodes* | The number of nodes in your Cloud Bigtable cluster. | <code title="">number</code> |  | <code title="">1</code> |
+| *storage_type* | The storage type to use. | <code title="">string</code> |  | <code title="">SSD</code> |
+| *table_options_defaults* | Default option of tables created in the BigTable instance. | <code title="object&#40;&#123;&#10;split_keys    &#61; list&#40;string&#41;&#10;column_family &#61; string&#10;&#125;&#41;">object({...})</code> |  | <code title="&#123;&#10;split_keys    &#61; &#91;&#93;&#10;column_family &#61; null&#10;&#125;">...</code> |
+| *tables* | Tables to be created in the BigTable instance. | <code title="map&#40;object&#40;&#123;&#10;table_options &#61; object&#40;&#123;&#10;split_keys    &#61; list&#40;string&#41;&#10;column_family &#61; string&#10;&#125;&#41;&#10;&#125;&#41;&#41;">map(object({...}))</code> |  | <code title="">{}</code> |
+
+## Outputs
+
+| name | description | sensitive |
+|---|---|:---:|
+| id | An identifier for the resource with format projects/{{project}}/instances/{{name}}. |  |
+| instance | BigTable intance. |  |
+| table_ids | Map of fully qualified table ids keyed by table name. |  |
+| tables | Table resources. |  |
+<!-- END TFDOC -->
+

modules/bigtable-instance/main.tf

@@ -0,0 +1,68 @@
+/**
+ * Copyright 2020 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+locals {
+  tables = {
+    for k, v in var.tables : k => v.table_options != null ? v.table_options : var.table_options_defaults
+  }
+
+  iam_roles_bindings = {
+    for k in var.iam_roles : k => lookup(var.iam_members, k, [])
+  }
+}
+
+resource "google_bigtable_instance" "default" {
+  project = var.project_id
+  name    = var.name
+  cluster {
+    cluster_id   = var.cluster_id
+    zone         = var.zone
+    storage_type = var.storage_type
+  }
+  instance_type = var.instance_type
+
+  display_name        = var.display_name == null ? var.display_name : var.name
+  deletion_protection = var.deletion_protection
+}
+
+resource "google_bigtable_instance_iam_binding" "default" {
+  for_each = local.iam_roles_bindings
+
+  project  = var.project_id
+  instance = google_bigtable_instance.default.name
+  role     = "roles/bigtable.${each.key}"
+  members  = each.value
+}
+
+resource "google_bigtable_table" "default" {
+  for_each      = local.tables
+  project       = var.project_id
+  instance_name = google_bigtable_instance.default.name
+  name          = each.key
+  split_keys    = each.value.split_keys
+
+  dynamic column_family {
+    for_each = each.value.column_family != null ? [""] : []
+
+    content {
+      family = each.value.column_family
+    }
+  }
+
+  # lifecycle {
+  #   prevent_destroy = true
+  # }
+}

modules/bigtable-instance/outputs.tf

@@ -0,0 +1,46 @@
+/**
+ * Copyright 2020 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+output "id" {
+  description = "An identifier for the resource with format projects/{{project}}/instances/{{name}}."
+  value       = google_bigtable_instance.default.id
+  depends_on = [
+    google_bigtable_instance_iam_binding,
+    google_bigtable_table
+  ]
+}
+
+output "instance" {
+  description = "BigTable intance."
+  value       = google_bigtable_instance.default
+  depends_on = [
+    google_bigtable_instance_iam_binding,
+    google_bigtable_table
+  ]
+}
+
+output "tables" {
+  description = "Table resources."
+  value       = google_bigtable_table.default
+}
+
+output "table_ids" {
+  description = "Map of fully qualified table ids keyed by table name."
+  value       = { for k, v in google_bigtable_table.default : v.name => v.id }
+}
+
+
+

modules/bigtable-instance/variables.tf

@@ -0,0 +1,99 @@
+/**
+ * Copyright 2019 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+variable "iam_roles" {
+  description = "Authoritative for a given role. Updates the IAM policy to grant a role to a list of members."
+  type        = list(string)
+  default     = []
+}
+
+variable "iam_members" {
+  description = "Authoritative for a given role. Updates the IAM policy to grant a role to a list of members. Other roles within the IAM policy for the instance are preserved."
+  type        = map(list(string))
+  default     = {}
+}
+
+variable "cluster_id" {
+  description = "The ID of the Cloud Bigtable cluster."
+  type        = string
+  default     = "europe-west1"
+}
+
+variable "deletion_protection" {
+  description = "Whether or not to allow Terraform to destroy the instance. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the instance will fail."
+  default     = true
+}
+
+variable "display_name" {
+  description = "The human-readable display name of the Bigtable instance."
+  default     = null
+}
+
+variable "instance_type" {
+  description = "The instance type to create. One of \"DEVELOPMENT\" or \"PRODUCTION\". Defaults to \"DEVELOPMENT\""
+  type        = string
+  default     = "DEVELOPMENT"
+}
+
+variable "name" {
+  description = "The name of the Cloud Bigtable instance."
+  type        = string
+}
+
+variable "num_nodes" {
+  description = "The number of nodes in your Cloud Bigtable cluster."
+  type        = number
+  default     = 1
+}
+
+variable "project_id" {
+  description = "Id of the project where datasets will be created."
+  type        = string
+}
+
+variable "storage_type" {
+  description = "The storage type to use."
+  type        = string
+  default     = "SSD"
+}
+
+variable "tables" {
+  description = "Tables to be created in the BigTable instance."
+  type = map(object({
+    table_options = object({
+      split_keys    = list(string)
+      column_family = string
+    })
+  }))
+  default = {}
+}
+
+variable "table_options_defaults" {
+  description = "Default option of tables created in the BigTable instance."
+  type = object({
+    split_keys    = list(string)
+    column_family = string
+  })
+  default = {
+    split_keys    = []
+    column_family = null
+  }
+}
+
+variable "zone" {
+  description = "The zone to create the Cloud Bigtable cluster in."
+  type        = string
+}

modules/bigtable-instance/versions.tf

@@ -0,0 +1,19 @@
+/**
+ * Copyright 2019 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+terraform {
+  required_version = ">= 0.12.6"
+}