4.9 KiB
4.9 KiB
Google Cloud Data Catalog Tag Module
This module allows managing Data Catalog Tag on GCP resources such as BigQuery Datasets, Tables or columns.
TODO
- Add support for entries different than Bigquery resources.
- Add support to BOOL when issue is fixed.
- Add support to RICHTEXT when issue is fixed.
Examples
Dataset Tag
module "data-catalog-tag" {
source = "./fabric/modules/data-catalog-tag"
tags = {
"landing/countries" = {
project_id = "project-data-product"
parent = "projects/project-data-product/datasets/landing"
location = "europe-west1"
template = "projects/project-datagov/locations/europe-west1/tagTemplates/demo"
fields = {
source = {
string_value = "DB-1"
}
datetime = {
timestamp_value = "2024-02-03T06:50:50.91Z"
}
num = {
double_value = 4.3
}
pii = {
enum_value = "NONE"
}
}
}
}
}
# tftest modules=1 resources=1
Table Tag
module "data-catalog-tag" {
source = "./fabric/modules/data-catalog-tag"
tags = {
"landing/countries" = {
project_id = "project-data-product"
parent = "projects/project-data-product/datasets/landing/tables/countries"
location = "europe-west1"
template = "projects/project-datagov/locations/europe-west1/tagTemplates/demo"
fields = {
source = {
string_value = "DB-1 Table-A"
}
}
}
}
}
# tftest modules=1 resources=1
Column Tag
module "data-catalog-tag" {
source = "./fabric/modules/data-catalog-tag"
tags = {
"landing/countries" = {
project_id = "project-data-product"
parent = "projects/project-data-product/datasets/landing/tables/countries"
column = "country"
location = "europe-west1"
template = "projects/project-datagov/locations/europe-west1/tagTemplates/demo"
fields = {
source = {
string_value = "DB-1 Table-A Column-B"
}
}
}
}
}
# tftest modules=1 resources=1
Factory
Similarly to other modules, a rules factory (see Resource Factories) is also included here to allow tags management via descriptive configuration files.
Factory configuration is via one optional attributes in the factory_config_path
variable specifying the path where tags files are stored.
Factory tags are merged with rules declared in code, with the latter taking precedence where both use the same key.
This is an example of a simple factory:
module "data-catalog-tag" {
source = "./fabric/modules/data-catalog-tag"
tags = {
"landing/countries" = {
project_id = "project-data-product"
parent = "projects/project-data-product/datasets/landing/tables/countries"
column = "country"
location = "europe-west1"
template = "projects/project-datagov/locations/europe-west1/tagTemplates/demo"
fields = {
source = {
string_value = "DB-1 Table-A Column-B"
}
}
}
}
factories_config = {
tags = "data"
}
}
# tftest modules=1 resources=2 files=demo_tag
# tftest-file id=demo_tag path=data/tag_1.yaml
project_id: project-data-product
parent: projects/project-data-product/datasets/exposure
location: europe-west1
template: projects/project-datagov/locations/europe-west1/tagTemplates/test
fields:
owner_email:
string_value: example@example.com
num:
double_value: 5
pii:
enum_value: NONE
Variables
name | description | type | required | default |
---|---|---|---|---|
factories_config | Paths to data files and folders that enable factory functionality. | object({…}) |
{} |
|
tags | Tags definitions in the form {TAG => TAG_DEFINITION}. | map(object({…})) |
{} |
Outputs
name | description | sensitive |
---|---|---|
data_catalog_tag_ids | Data catalog tag ids. |