zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into gcs2bq-least-privileges

This commit is contained in:
Ludovico Magnocavallo 2022-01-03 08:07:32 +01:00 committed by GitHub
parent eaae34b623 ee25965c89
commit 47acc03188
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
756 changed files with 2731 additions and 2402 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.ci/cloudbuild.lint.yaml
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -50,7 +50,7 @@ steps:
"factories/firewall-vpc-rules",
"foundations",
"modules",
"networking"
"networking",
]
substitutions:

2
.ci/cloudbuild.test.environments.yaml
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
.ci/cloudbuild.test.examples.yaml
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
.ci/cloudbuild.test.modules.yaml
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

6
.github/workflows/daily-tag.yml vendored
View File

@ -1,5 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -19,7 +18,7 @@ name: |
on:
workflow_dispatch:
schedule:
- cron: '0 2 * * *'
- cron: "0 2 * * *"
permissions:
contents: write
@ -44,4 +43,3 @@ jobs:
else
echo "No changes in last 24 hours"
fi

2
.github/workflows/linting.yml vendored
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

12
.github/workflows/tests.yml vendored
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -12,10 +12,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.
name: 'Tests'
name: "Tests"
on:
schedule:
- cron: '45 2 * * *'
- cron: "45 2 * * *"
pull_request:
branches:
- master
@ -35,7 +35,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.9'
python-version: "3.9"
- name: Set up Terraform
uses: hashicorp/setup-terraform@v1
@ -64,7 +64,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.9'
python-version: "3.9"
- name: Set up Terraform
uses: hashicorp/setup-terraform@v1
@ -93,7 +93,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.9'
python-version: "3.9"
- name: Set up Terraform
uses: hashicorp/setup-terraform@v1

18
CHANGELOG.md
View File

@ -5,6 +5,20 @@ All notable changes to this project will be documented in this file.
## [Unreleased]
- added new data-solutions example: Cloud Storage to Bigquery with Cloud Dataflow with least privileges
## [10.0.0] - 2021-12-31
- fix cases where bridge perimeter status resources are `null` in `vpc-sc` module
- re-release 9.0.3 as a major release as it contains breaking changes
- update hierarchical firewall resources to use the newer `google_compute_firewall_*` resources
- **incompatible change** rename `firewall_policy_attachments` to `firewall_policy_association` in the `organization` and `folder` modules
- **incompatible change** updated API for the `net-vpc-sc` module
## [9.0.3] - 2021-12-31
- update hierarchical firewall resources to use the newer `google_compute_firewall_*` resources
- **incompatible change** rename `firewall_policy_attachments` to `firewall_policy_association` in the `organization` and `folder` modules
- **incompatible change** updated API for the `net-vpc-sc` module
## [9.0.2] - 2021-12-22
- ignore description changes in firewall policy rule to avoid permadiff, add factory example to `folder` module documentation
@ -391,7 +405,9 @@ All notable changes to this project will be documented in this file.
- merge development branch with suite of new modules and end-to-end examples
[Unreleased]: https://github.com/terraform-google-modules/cloud-foundation-fabric/compare/v9.0.2...HEAD
[Unreleased]: https://github.com/terraform-google-modules/cloud-foundation-fabric/compare/v10.0.0...HEAD
[10.0.0]: https://github.com/terraform-google-modules/cloud-foundation-fabric/compare/v9.0.3...v10.0.0
[9.0.3]: https://github.com/terraform-google-modules/cloud-foundation-fabric/compare/v9.0.2...v9.0.3
[9.0.2]: https://github.com/terraform-google-modules/cloud-foundation-fabric/compare/v9.0.0...v9.0.2
[9.0.0]: https://github.com/terraform-google-modules/cloud-foundation-fabric/compare/v8.0.0...v9.0.0
[8.0.0]: https://github.com/terraform-google-modules/cloud-foundation-fabric/compare/v7.0.0...v8.0.0

3
cloud-operations/asset-inventory-feed-remediation/README.md
View File

@ -52,6 +52,7 @@ Run the `subscription_pull` command until it returns nothing, then run the follo
- the `tag_show` command to verify that the function output matches the resource state
<!-- BEGIN TFDOC -->
## Variables
@ -73,6 +74,6 @@ Run the `subscription_pull` command until it returns nothing, then run the follo
| tag_add | Instance add tag command. | |
| tag_show | Instance add tag command. | |
<!-- END TFDOC -->

2
cloud-operations/asset-inventory-feed-remediation/backend.tf.sample
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
cloud-operations/asset-inventory-feed-remediation/cf/main.py
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
cloud-operations/asset-inventory-feed-remediation/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/asset-inventory-feed-remediation/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/asset-inventory-feed-remediation/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/asset-inventory-feed-remediation/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
cloud-operations/dns-fine-grained-iam/README.md
View File

@ -99,6 +99,7 @@ dig app1.svc.example.org +short
# 127.0.0.7
```
<!-- BEGIN TFDOC -->
## Variables
@ -118,5 +119,5 @@ dig app1.svc.example.org +short
| gcloud_commands | Commands used to SSH to the VMs. | |
| vms | VM names. | |
<!-- END TFDOC -->

2
cloud-operations/dns-fine-grained-iam/backend.tf.sample
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
cloud-operations/dns-fine-grained-iam/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/dns-fine-grained-iam/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/dns-fine-grained-iam/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/dns-fine-grained-iam/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
cloud-operations/dns-shared-vpc/README.md
View File

@ -19,6 +19,7 @@ The resources created in this example are shown in the high level diagram below:
Note that Terraform 0.13 at least is required due to the use of `for_each` with modules.
<!-- BEGIN TFDOC -->
## Variables
@ -39,5 +40,5 @@ Note that Terraform 0.13 at least is required due to the use of `for_each` with
|---|---|:---:|
| teams | Team resources | |
<!-- END TFDOC -->

4
cloud-operations/dns-shared-vpc/examples/shared-vpc-example/network.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -40,4 +40,4 @@ module "cloud-dns" {
teams = var.teams
dns_domain = var.dns_domain
}
}

4
cloud-operations/dns-shared-vpc/examples/shared-vpc-example/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -22,4 +22,4 @@ output "host_project_id" {
output "shared_vpc_self_link" {
description = "Shared VPC Self link"
value = module.shared-vpc.self_link
}
}

4
cloud-operations/dns-shared-vpc/examples/shared-vpc-example/projects.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -71,4 +71,4 @@ module "project-service-2" {
attach = true
host_project = module.project-host.project_id
}
}
}

2
cloud-operations/dns-shared-vpc/examples/shared-vpc-example/test.example
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

4
cloud-operations/dns-shared-vpc/examples/shared-vpc-example/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -62,4 +62,4 @@ variable "dns_domain" {
variable "teams" {
description = "List of teams that require their own Cloud DNS instance"
default = ["appteam1", "appteam2"]
}
}

2
cloud-operations/dns-shared-vpc/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/dns-shared-vpc/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/dns-shared-vpc/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/dns-shared-vpc/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
cloud-operations/iam-delegated-role-grants/README.md
View File

@ -64,6 +64,7 @@ If you get any warnings, check the roles and remove any of them granting any of
- `resourcemanager.organizations.setIamPolicy`
<!-- BEGIN TFDOC -->
## Variables
@ -77,5 +78,5 @@ If you get any warnings, check the roles and remove any of them granting any of
| project_create | Create project instead of using an existing one. | <code>bool</code> | | <code>false</code> |
| restricted_role_grant | Role grant to which the restrictions will apply. | <code>string</code> | | <code>&#34;roles&#47;resourcemanager.projectIamAdmin&#34;</code> |
<!-- END TFDOC -->

70
cloud-operations/iam-delegated-role-grants/audit.py
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -28,50 +28,50 @@ SENSITIVE_PERMISSIONS = {
def get_role_permissions(role):
if role.startswith("roles/"):
endpoint = iam_service.roles()
elif role.startswith("projects/"):
endpoint = iam_service.projects().roles()
elif role.startswith("organizations/"):
endpoint = iam_service.organizations().roles()
else:
raise Exception(f"Invalid role {role}")
if role.startswith("roles/"):
endpoint = iam_service.roles()
elif role.startswith("projects/"):
endpoint = iam_service.projects().roles()
elif role.startswith("organizations/"):
endpoint = iam_service.organizations().roles()
else:
raise Exception(f"Invalid role {role}")
response = endpoint.get(name=role).execute()
permissions = response.get("includedPermissions")
return permissions
response = endpoint.get(name=role).execute()
permissions = response.get("includedPermissions")
return permissions
@click.command()
@click.argument("file", type=click.File("r"))
def main(file):
"""Verify that the set of GCP roles in FILE does not include the
permission setIamPolicy at project, folder or organization level
"""Verify that the set of GCP roles in FILE does not include the
permission setIamPolicy at project, folder or organization level
This program authenticates against GCP using default application
credentials to query project and organization level roles.
This program authenticates against GCP using default application
credentials to query project and organization level roles.
"""
clean_roles = [x.rstrip(" \n") for x in file]
roles = (x for x in clean_roles if x)
"""
clean_roles = [x.rstrip(" \n") for x in file]
roles = (x for x in clean_roles if x)
allok = True
for role in roles:
try:
permissions = set(get_role_permissions(role))
except Error as e:
print(f"WARNING: can't read {role}: {e}")
allok = False
else:
matched_sensitive_permissions = SENSITIVE_PERMISSIONS & permissions
if matched_sensitive_permissions:
print(f"WARNING: {role} contains {matched_sensitive_permissions}")
allok = False
else:
print(f"{role} ok")
allok = True
for role in roles:
try:
permissions = set(get_role_permissions(role))
except Error as e:
print(f"WARNING: can't read {role}: {e}")
allok = False
else:
matched_sensitive_permissions = SENSITIVE_PERMISSIONS & permissions
if matched_sensitive_permissions:
print(f"WARNING: {role} contains {matched_sensitive_permissions}")
allok = False
else:
print(f"{role} ok")
exit(0 if allok else 1)
exit(0 if allok else 1)
if __name__ == "__main__":
main()
main()

2
cloud-operations/iam-delegated-role-grants/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/iam-delegated-role-grants/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/iam-delegated-role-grants/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/iam-delegated-role-grants/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
cloud-operations/onprem-sa-key-management/README.md
View File

@ -61,6 +61,7 @@ gcloud auth activate-service-account --key-file data-uploader.json
terraform destroy -var project_id=$GOOGLE_CLOUD_PROJECT
```
<!-- BEGIN TFDOC -->
## Variables
@ -78,5 +79,5 @@ terraform destroy -var project_id=$GOOGLE_CLOUD_PROJECT
|---|---|:---:|
| sa-credentials | SA json key templates. | |
<!-- END TFDOC -->

2
cloud-operations/onprem-sa-key-management/backend.tf.sample
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
cloud-operations/onprem-sa-key-management/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/onprem-sa-key-management/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/onprem-sa-key-management/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/onprem-sa-key-management/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
cloud-operations/packer-image-builder/README.md
View File

@ -67,6 +67,7 @@ The following example assumes that provisioning of a Compute Engine VM requires
the resources over the Internet (i.e. to install OS packages). Since Compute VM has no public IP
address for security reasons, Internet connectivity is done with [Cloud NAT](https://cloud.google.com/nat/docs/overview).
<!-- BEGIN TFDOC -->
## Variables
@ -93,5 +94,5 @@ address for security reasons, Internet connectivity is done with [Cloud NAT](htt
| compute_subnetwork | Name of a subnetwork for Packer's temporary VM. | |
| compute_zone | Name of a compute engine zone for Packer's temporary VM. | |
<!-- END TFDOC -->

4
cloud-operations/packer-image-builder/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -128,4 +128,4 @@ resource "local_file" "packer-vars" {
USE_IAP = "${var.use_iap}"
})
filename = local.packer_variables_file
}
}

2
cloud-operations/packer-image-builder/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/packer-image-builder/packer/build.pkr.hcl
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/packer-image-builder/packer/install_httpd.sh
View File

@ -1,6 +1,6 @@
#!/bin/sh
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
cloud-operations/packer-image-builder/packer/variables.pkr.hcl
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/packer-image-builder/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/packer-image-builder/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
cloud-operations/quota-monitoring/README.md
View File

@ -23,6 +23,7 @@ Clone this repository or [open it in cloud shell](https://ssh.cloud.google.com/c
- `terraform init`
- `terraform apply -var project_id=my-project-id`
<!-- BEGIN TFDOC -->
## Variables
@ -37,6 +38,6 @@ Clone this repository or [open it in cloud shell](https://ssh.cloud.google.com/c
| region | Compute region used in the example. | <code>string</code> | | <code>&#34;europe-west1&#34;</code> |
| schedule_config | Schedule timer configuration in crontab format | <code>string</code> | | <code>&#34;0 &#42; &#42; &#42; &#42;&#34;</code> |
<!-- END TFDOC -->

2
cloud-operations/quota-monitoring/backend.tf.sample
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
cloud-operations/quota-monitoring/cf/main.py
View File

@ -1,5 +1,5 @@
#! /usr/bin/env python3
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
cloud-operations/quota-monitoring/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/quota-monitoring/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/quota-monitoring/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/quota-monitoring/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
cloud-operations/scheduled-asset-inventory-export-bq/README.md
View File

@ -52,6 +52,7 @@ This is an optional part, created if `cai_gcs_export` is set to `true`. The high
<!-- BEGIN TFDOC -->
## Variables
@ -79,6 +80,6 @@ This is an optional part, created if `cai_gcs_export` is set to `true`. The high
| bq-dataset | Bigquery instance details. | |
| cloud-function | Cloud Function instance details. | |
<!-- END TFDOC -->

2
cloud-operations/scheduled-asset-inventory-export-bq/backend.tf.sample
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

11
cloud-operations/scheduled-asset-inventory-export-bq/cf/main.py
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -62,7 +62,8 @@ def main_cli(project=None, bq_project=None, bq_dataset=None, bq_table=None, bq_t
the dataset specified on a dated table with the name specified.
'''
try:
_main(project, bq_project, bq_dataset, bq_table, bq_table_overwrite, target_node, read_time, verbose)
_main(project, bq_project, bq_dataset, bq_table,
bq_table_overwrite, target_node, read_time, verbose)
except RuntimeError:
logging.exception('exception raised')
@ -89,10 +90,10 @@ def _main(project=None, bq_project=None, bq_dataset=None, bq_table=None, bq_tabl
if bq_table_overwrite == False:
read_time = datetime.datetime.now()
output_config.bigquery_destination.table = '%s_%s' % (
bq_table, read_time.strftime('%Y%m%d'))
bq_table, read_time.strftime('%Y%m%d'))
else:
output_config.bigquery_destination.table = '%s_latest' % (
bq_table)
output_config.bigquery_destination.table = '%s_latest' % (
bq_table)
content_type = asset_v1.ContentType.RESOURCE
output_config.bigquery_destination.dataset = 'projects/%s/datasets/%s' % (
bq_project, bq_dataset)

12
cloud-operations/scheduled-asset-inventory-export-bq/cffile/main.py
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
@ -44,6 +44,7 @@ def _configure_logging(verbose=True):
logging.basicConfig(level=level)
warnings.filterwarnings('ignore', r'.*end user credentials.*', UserWarning)
@click.command()
@click.option('--bucket', required=True, help='GCS bucket for export')
@click.option('--filename', required=True, help='Path and filename with extension to export e.g. folder/export.json .')
@ -60,6 +61,7 @@ def main_cli(bucket=None, filename=None, format=None, bq_dataset=None, bq_table=
except RuntimeError:
logging.exception('exception raised')
def main(event, context):
'Cloud Function entry point.'
try:
@ -83,10 +85,10 @@ def _main(bucket=None, filename=None, format=None, bq_dataset=None, bq_table=Non
table_ref = dataset_ref.table(bq_table)
job_config = bigquery.job.ExtractJobConfig()
job_config.destination_format = (
getattr(bigquery.DestinationFormat, format) )
getattr(bigquery.DestinationFormat, format))
extract_job = client.extract_table(
table_ref, destination_uri, job_config=job_config
)
table_ref, destination_uri, job_config=job_config
)
try:
extract_job.result()
except (GoogleAPIError, googleapiclient.errors.HttpError) as e:
@ -96,4 +98,4 @@ def _main(bucket=None, filename=None, format=None, bq_dataset=None, bq_table=Non
if __name__ == '__main__':
main_cli()
main_cli()

2
cloud-operations/scheduled-asset-inventory-export-bq/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/scheduled-asset-inventory-export-bq/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/scheduled-asset-inventory-export-bq/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
cloud-operations/scheduled-asset-inventory-export-bq/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
data-solutions/cmek-via-centralized-kms/README.md
View File

@ -30,6 +30,7 @@ This sample creates several distinct groups of resources:
- GCS
- One bucket encrypted with a CMEK Cryptokey hosted in Cloud KMS
<!-- BEGIN TFDOC -->
## Variables
@ -56,5 +57,5 @@ This sample creates several distinct groups of resources:
| vm | GCE VM. | |
| vm_keys | GCE VM Cloud KMS crypto keys. | |
<!-- END TFDOC -->

2
data-solutions/cmek-via-centralized-kms/backend.tf.sample
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/cmek-via-centralized-kms/main.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/cmek-via-centralized-kms/outputs.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/cmek-via-centralized-kms/variables.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/cmek-via-centralized-kms/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
data-solutions/data-platform-foundations/01-environment/README.md
View File

@ -47,6 +47,7 @@ gcloud access-context-manager perimeters list --format="json" | grep name
The script use 'google_access_context_manager_service_perimeter_resource' terraform resource. If this resource is used alongside the 'vpc-sc' module, remember to uncomment the lifecycle block in the 'vpc-sc' module so they don't fight over which resources should be in the perimeter.
<!-- BEGIN TFDOC -->
## Variables
@ -70,5 +71,5 @@ The script use 'google_access_context_manager_service_perimeter_resource' terraf
| service_account | Main service account. | |
| service_encryption_key_ids | Cloud KMS encryption keys in {LOCATION => [KEY_URL]} format. | |
<!-- END TFDOC -->

2
data-solutions/data-platform-foundations/01-environment/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
data-solutions/data-platform-foundations/02-resources/README.md
View File

@ -51,6 +51,7 @@ Once done testing, you can clean up resources by running `terraform destroy`.
### CMEK configuration
You can configure GCP resources to use existing CMEK keys configuring the 'service_encryption_key_ids' variable. You need to specify a 'global' and a 'multiregional' key.
<!-- BEGIN TFDOC -->
## Variables
@ -81,5 +82,5 @@ You can configure GCP resources to use existing CMEK keys configuring the 'servi
| transformation-buckets | List of buckets created for the transformation project. | |
| transformation-vpc | Transformation VPC details | |
<!-- END TFDOC -->

2
data-solutions/data-platform-foundations/02-resources/providers.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
data-solutions/data-platform-foundations/02-resources/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
data-solutions/gcs-to-bq-with-dataflow/README.md
View File

@ -110,6 +110,7 @@ schema_bq_import.json
You can check data imported into Google BigQuery from the Google Cloud Console UI.
<!-- BEGIN TFDOC -->
## Variables
@ -136,5 +137,5 @@ You can check data imported into Google BigQuery from the Google Cloud Console U
| projects | Project ids. | |
| vm | GCE VM. | |
<!-- END TFDOC -->

2
data-solutions/gcs-to-bq-with-dataflow/backend.tf.sample
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/gcs-to-bq-with-dataflow/main.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/gcs-to-bq-with-dataflow/outputs.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/gcs-to-bq-with-dataflow/scripts/data_ingestion/data_ingestion.py
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/gcs-to-bq-with-dataflow/scripts/person_details_generator/person_details_generator.py
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/gcs-to-bq-with-dataflow/variables.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
data-solutions/gcs-to-bq-with-dataflow/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
default-versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
factories/example-environments/dev/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
factories/example-environments/dev/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

2
factories/example-environments/prod/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
factories/example-environments/prod/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
factories/firewall-hierarchical-policies/README.md
View File

@ -148,6 +148,7 @@ web_frontends:
- web-frontends@project-wf2.iam.gserviceaccount.com
```
<!-- BEGIN TFDOC -->
## Variables
@ -163,5 +164,5 @@ web_frontends:
|---|---|:---:|
| hierarchical-firewall-rules | Generated Hierarchical Firewall Rules | |
<!-- END TFDOC -->

2
factories/firewall-hierarchical-policies/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
factories/firewall-hierarchical-policies/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
factories/firewall-hierarchical-policies/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
factories/firewall-hierarchical-policies/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

3
factories/firewall-vpc-rules/flat/README.md
View File

@ -135,6 +135,7 @@ web-app-a-ingress:
- web-app-a@myproject-id.iam.gserviceaccount.com
```
<!-- BEGIN TFDOC -->
## Variables
@ -155,5 +156,5 @@ web-app-a-ingress:
| ingress_allow_rules | Ingress rules with allow blocks. | |
| ingress_deny_rules | Ingress rules with deny blocks. | |
<!-- END TFDOC -->

2
factories/firewall-vpc-rules/flat/main.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
factories/firewall-vpc-rules/flat/outputs.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
factories/firewall-vpc-rules/flat/variables.tf
View File

@ -1,5 +1,5 @@
/**
* Copyright 2021 Google LLC
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.

2
factories/firewall-vpc-rules/flat/versions.tf
View File

@ -1,4 +1,4 @@
# Copyright 2021 Google LLC
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.

Some files were not shown because too many files have changed in this diff Show More