6941313c7d
* factories refactor doc * Adds file schema and filesystem organization * Update 20231106-factories.md * move factories out of blueprints and create new factories README * align factory in billing-account module * align factory in dataplex-datascan module * align factory in billing-account module * align factory in net-firewall-policy module * align factory in dns-response-policy module * align factory in net-vpc-firewall module * align factory in net-vpc module * align factory variable names in FAST * remove decentralized firewall blueprint * bump terraform version * bump module versions * update top-level READMEs * move project factory to modules * fix variable names and tests * tfdoc * remove changelog link * add project factory to top-level README * fix cludrun eventarc diff * fix README * fix cludrun eventarc diff --------- Co-authored-by: Simone Ruffilli <sruffilli@google.com> |
||
---|---|---|
.. | ||
README.md | ||
main.tf | ||
outputs.tf | ||
variables.tf | ||
versions.tf |
README.md
Google Cloud Data Catalog Tag Module
This module allows managing Data Catalog Tag on GCP resources such as BigQuery Datasets, Tables or columns.
TODO
- Add support for entries different than Bigquery resources.
- Add support to BOOL when issue is fixed.
- Add support to RICHTEXT when issue is fixed.
Examples
Dataset Tag
module "data-catalog-tag" {
source = "./fabric/modules/data-catalog-tag"
tags = {
"landing/countries" = {
project_id = "project-data-product"
parent = "projects/project-data-product/datasets/landing"
location = "europe-west1"
template = "projects/project-datagov/locations/europe-west1/tagTemplates/demo"
fields = {
source = {
string_value = "DB-1"
}
datetime = {
timestamp_value = "2024-02-03T06:50:50.91Z"
}
num = {
double_value = 4.3
}
pii = {
enum_value = "NONE"
}
}
}
}
}
# tftest modules=1 resources=1
Table Tag
module "data-catalog-tag" {
source = "./fabric/modules/data-catalog-tag"
tags = {
"landing/countries" = {
project_id = "project-data-product"
parent = "projects/project-data-product/datasets/landing/tables/countries"
location = "europe-west1"
template = "projects/project-datagov/locations/europe-west1/tagTemplates/demo"
fields = {
source = {
string_value = "DB-1 Table-A"
}
}
}
}
}
# tftest modules=1 resources=1
Column Tag
module "data-catalog-tag" {
source = "./fabric/modules/data-catalog-tag"
tags = {
"landing/countries" = {
project_id = "project-data-product"
parent = "projects/project-data-product/datasets/landing/tables/countries"
column = "country"
location = "europe-west1"
template = "projects/project-datagov/locations/europe-west1/tagTemplates/demo"
fields = {
source = {
string_value = "DB-1 Table-A Column-B"
}
}
}
}
}
# tftest modules=1 resources=1
Factory
Similarly to other modules, a rules factory (see Resource Factories) is also included here to allow tags management via descriptive configuration files.
Factory configuration is via one optional attributes in the factory_config_path
variable specifying the path where tags files are stored.
Factory tags are merged with rules declared in code, with the latter taking precedence where both use the same key.
This is an example of a simple factory:
module "data-catalog-tag" {
source = "./fabric/modules/data-catalog-tag"
tags = {
"landing/countries" = {
project_id = "project-data-product"
parent = "projects/project-data-product/datasets/landing/tables/countries"
column = "country"
location = "europe-west1"
template = "projects/project-datagov/locations/europe-west1/tagTemplates/demo"
fields = {
source = {
string_value = "DB-1 Table-A Column-B"
}
}
}
}
factories_config = {
tags = "data"
}
}
# tftest modules=1 resources=2 files=demo_tag
# tftest-file id=demo_tag path=data/tag_1.yaml
project_id: project-data-product
parent: projects/project-data-product/datasets/exposure
location: europe-west1
template: projects/project-datagov/locations/europe-west1/tagTemplates/test
fields:
owner_email:
string_value: example@example.com
num:
double_value: 5
pii:
enum_value: NONE
Variables
name | description | type | required | default |
---|---|---|---|---|
factories_config | Paths to data files and folders that enable factory functionality. | object({…}) |
{} |
|
tags | Tags definitions in the form {TAG => TAG_DEFINITION}. | map(object({…})) |
{} |
Outputs
name | description | sensitive |
---|---|---|
data_catalog_tag_ids | Data catalog tag ids. |