Cloud Asset feed operations example (#110)
* first working example for asset inventory feeds * move tf files out of the tf folder * add input/outputs to README * smaller diagram * use narrow scoped service account for cf, account for gke tags in code * Update README.md * new top-level folder README * Update README.md * add TODO for DNS example in operations README * fix README conflict * Update README.md * Update README.md * update diagram * cloud shell * cloud shell * Update README.md * rename outputs, first complete README draft * Update main.py * Update README.md * Update README.md * better error handling in the cloud function * remove branch from cloud shell link
This commit is contained in:
Ludovico Magnocavallo
GitHub
parent
29b34c0a2d
commit
1b5af2d923
|
|
@ -12,3 +12,4 @@ backend-config.hcl
|
|||
credentials.json
|
||||
key.json
|
||||
terraform-ls.tf
|
||||
bundle.zip
|
||||
|
|
|
|||
|
|
@ -0,0 +1,15 @@
|
|||
# Operations examples
|
||||
|
||||
The examples in this folder show how to wire together different Google Cloud services to simplify operations, and are meant for testing, or as minimal but sufficiently complete starting points for actual use.
|
||||
|
||||
## Resource tracking and remediation via Cloud Asset feeds
|
||||
|
||||
<a href="./asset-inventory-feed-remediation" title="Resource tracking and remediation via Cloud Asset feeds"><img src="./asset-inventory-feed-remediation/diagram.png" align="left" width="280px"></a> This [example](./asset-inventory-feed-remediation) shows how to leverage [Cloud Asset Inventory feeds](https://cloud.google.com/asset-inventory/docs/monitoring-asset-changes) to stream resource changes in real time, and how to programmatically use the feed change notifications for alerting or remediation, via a Cloud Function wired to the feed PubSub queue.
|
||||
|
||||
The example's feed tracks changes to Google Compute instances, and the Cloud Function enforces policy compliance on each change so that tags match a set of simple rules. The obious use case is when instance tags are used to scope firewall rules, bu the example can easily be adapted to suit different use cases.
|
||||
|
||||
<br clear="left">
|
||||
|
||||
## Granular Cloud DNS IAM via Service Directory
|
||||
|
||||
TODO(ludoo): publish the working example
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
# Cloud Asset Inventory feeds for resource change tracking and remediation
|
||||
|
||||
This example shows how to leverage [Cloud Asset Inventory feeds](https://cloud.google.com/asset-inventory/docs/monitoring-asset-changes) to stream resource changes in real time, and how to programmatically react to changes by wiring a Cloud Function to the feed outputs.
|
||||
|
||||
The Cloud Function can then be used for different purposes:
|
||||
|
||||
- updating remote data (eg a CMDB) to reflect the changed resources
|
||||
- triggering alerts to surface critical changes
|
||||
- adapting the configuration of separate related resources
|
||||
- implementing remediation steps that enforce policy compliance by tweaking or reverting the changes.
|
||||
|
||||
This example shows a simple remediation use case: how to enforce policies on instance tags and revert non-compliant changes in near-real time, thus adding an additional measure of control when using tags for firewall rule scoping. Changing the [monitored asset](https://cloud.google.com/asset-inventory/docs/supported-asset-types) and the function logic allows simple adaptation to other common use cases:
|
||||
|
||||
- enforcing a centrally defined Cloud Armor policy in backend services
|
||||
- creating custom DNS records for instances or forwarding rules
|
||||
|
||||
The example uses a single project for ease of testing, in actual use a few changes are needed to operate at the resource hierarchy level:
|
||||
|
||||
- the feed should be set at the folder or organization level
|
||||
- the custom role used to assign tag changing permissions should be defined at the organization level
|
||||
- the role binding that grants the custom role to the Cloud Function service account should be set at the same level as the feed (folder or organization)
|
||||
|
||||
The resources created in this example are shown in the high level diagram below:
|
||||
|
||||
<img src="diagram.png" width="640px">
|
||||
|
||||
|
||||
## Running the example
|
||||
|
||||
Clone this repository or [open it in cloud shell](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2Fterraform-google-modules%2Fcloud-foundation-fabric&cloudshell_print=cloud-shell-readme.txt&cloudshell_working_dir=cloud-operations%2Fasset-inventory-feed-remediation), then go through the following steps to create resources:
|
||||
|
||||
- `terraform init`
|
||||
- `terraform apply -var project_id=my-project-id`
|
||||
- copy and paste the `feed_create` output in the console then run it to create the feed
|
||||
|
||||
Once done testing, you can clean up resources by running `terraform destroy`. To persist state, check out the `backend.tf.sample` file.
|
||||
|
||||
## Testing the example
|
||||
|
||||
The terraform outputs generate preset `gcloud` commands that you can copy and run in the console, to complete configuration and test the example:
|
||||
|
||||
- `feed_create` is run once to create the feed, as there's currently no Terraform resource available for Cloud Asset feeds
|
||||
- `subscription_pull` shows messages in the PubSub queue, to check feed message format if the Cloud Function is disabled
|
||||
- `cf_logs` shows Cloud Function logs to check that remediation works
|
||||
- `tag_add` adds a non-compliant tag to the test instance, and triggers the Cloud Function remediation process
|
||||
- `tag_show` displays the tags currently set on the test instance
|
||||
|
||||
Run the `subscription_pull` command until it returns nothing, then run the following commands in order to test remediation:
|
||||
|
||||
- the `tag_add` command
|
||||
- the `cf_logs` command until the logs show that the change has been picked up, verified, and the compliant tags have been force-set on the instance
|
||||
- the `tag_show` command to verify that the function output matches the resource state
|
||||
|
||||
|
||||
<!-- BEGIN TFDOC -->
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---: |:---:|:---:|
|
||||
| project_id | Project id that references existing project. | <code title="">string</code> | ✓ | |
|
||||
| *bundle_path* | Path used to write the intermediate Cloud Function code bundle. | <code title="">string</code> | | <code title="">./bundle.zip</code> |
|
||||
| *name* | Arbitrary string used to name created resources. | <code title="">string</code> | | <code title="">asset-feed</code> |
|
||||
| *region* | Compute region used in the example. | <code title="">string</code> | | <code title="">europe-west1</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| cf_logs | Cloud Function logs read command. | |
|
||||
| feed_create | Feed gcloud command. | |
|
||||
| subscription_pull | Subscription pull command. | |
|
||||
| tag_add | Instance add tag command. | |
|
||||
| tag_show | Instance add tag command. | |
|
||||
<!-- END TFDOC -->
|
||||
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
# Copyright 2019 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# set a valid bucket below and rename this file to backend.tf
|
||||
|
||||
terraform {
|
||||
backend "gcs" {
|
||||
bucket = ""
|
||||
prefix = "fabric/operations/feeds"
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -0,0 +1,159 @@
|
|||
# Copyright 2020 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
'''Cloud Function module to do simple instance tag enforcement.
|
||||
|
||||
This module is designed to be plugged in a Cloud Function, attached to a PubSub
|
||||
trigger that receives Cloud Inventory Asset updates on the instance type. Its
|
||||
purpose is to do live checking, validation and remediation of instance tags.
|
||||
|
||||
Tags are validated using two simple rules: global allowed tags must match the
|
||||
fixed prefixes in the `_TAG_SHARED_PREFIXES` constant, while project local
|
||||
tags must be prefixed with the project id.
|
||||
|
||||
Quickstart to create the feed and deploy the function, assuming
|
||||
all other prerequisites are in place:
|
||||
|
||||
gcloud pubsub topics create asset-feed-instance \
|
||||
--project $PROJECT
|
||||
gcloud asset feeds create instance-resource \
|
||||
--pubsub-topic projects/$PROJECT/topics/asset-feed-instance \
|
||||
--asset-types compute.googleapis.com/Instance \
|
||||
--content-type resource --project $PROJECT
|
||||
gcloud functions deploy test-feed
|
||||
--region europe-west1 --allow-unauthenticated
|
||||
--entry-point main --runtime python38
|
||||
--trigger-topic asset-feed-instance --project $PROJECT
|
||||
'''
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
|
||||
from googleapiclient import discovery
|
||||
from googleapiclient.errors import HttpError
|
||||
|
||||
|
||||
_SELF_LINK_RE = re.compile(
|
||||
r'/projects/([^/]+)/zones/([^/]+)/instances/([^/]+)')
|
||||
_TAG_SHARED_PREFIXES = ['shared-', 'gke-cluster-']
|
||||
|
||||
|
||||
class Error(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _set_tags(project, zone, name, fingerprint, tags):
|
||||
'Set specific tags on instance.'
|
||||
body = {'fingerprint': fingerprint, 'items': tags}
|
||||
compute = discovery.build('compute', 'v1', cache_discovery=False)
|
||||
try:
|
||||
result = compute.instances().setTags(project=project, zone=zone,
|
||||
instance=name, body=body).execute()
|
||||
while True:
|
||||
if 'error' in result:
|
||||
raise SystemExit(result['error'])
|
||||
if result['status'] == 'DONE':
|
||||
break
|
||||
time.sleep(1)
|
||||
result = compute.zoneOperations().get(
|
||||
project=project,
|
||||
zone=zone,
|
||||
operation=result['name']).execute()
|
||||
except HttpError as e:
|
||||
raise Error('Error setting tags: %s' % e)
|
||||
|
||||
|
||||
def _parse_asset(data):
|
||||
'Extract instance attributes from asset feed data.'
|
||||
try:
|
||||
asset_type = data['asset']['assetType']
|
||||
if asset_type != 'compute.googleapis.com/Instance':
|
||||
raise Error('ignoring sset type %s' % asset_type)
|
||||
instance = data['asset']['resource']['data']
|
||||
except KeyError:
|
||||
raise Error('missing asset data')
|
||||
# ensure we have at least status and selfLink
|
||||
for k in ('status', 'selfLink'):
|
||||
if k not in instance:
|
||||
raise Error('no %s attribute in instance data' % k)
|
||||
return instance
|
||||
|
||||
|
||||
def _parse_event(event):
|
||||
'Check PubSub event and return asset feed data.'
|
||||
if not event or 'data' not in event:
|
||||
raise Error('no event received, or no data in event')
|
||||
logging.info('parsing event data')
|
||||
try:
|
||||
data = base64.b64decode(event['data'])
|
||||
return json.loads(data)
|
||||
except binascii.Error as e:
|
||||
logging.info('received event: %s' % event)
|
||||
raise Error('cannot decode event data: %s' % e)
|
||||
except json.JSONDecodeError as e:
|
||||
logging.info('received data: %s', data)
|
||||
raise Error('event data not in JSON format: %s' % e)
|
||||
|
||||
|
||||
def _parse_self_link(self_link):
|
||||
'Parse instance self link and return project, zone, and instance name.'
|
||||
m = _SELF_LINK_RE.search(self_link)
|
||||
if not m:
|
||||
raise Error('invalid self link %s' % self_link)
|
||||
return m.groups()
|
||||
|
||||
|
||||
def _validate_tags(project, tags):
|
||||
'Validate a set of tags and return valid tags in the set.'
|
||||
_tags = []
|
||||
for tag in tags:
|
||||
shared_valid = any(tag.startswith(p) for p in _TAG_SHARED_PREFIXES)
|
||||
if shared_valid or tag.startswith(project):
|
||||
_tags.append(tag)
|
||||
return _tags
|
||||
|
||||
|
||||
def main(event=None, context=None):
|
||||
'Cloud Function entry point.'
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
try:
|
||||
data = _parse_event(event)
|
||||
instance = _parse_asset(data)
|
||||
project, zone, name = _parse_self_link(instance['selfLink'])
|
||||
except Error as e:
|
||||
logging.critical(e.args[0])
|
||||
return
|
||||
logging.info('checking %s', instance['selfLink'])
|
||||
if instance['status'] not in ('RUNNING', 'TERMINATED'):
|
||||
logging.info('ignoring status %s', instance['status'])
|
||||
return
|
||||
tags = instance.get('tags', {})
|
||||
logging.info('tags %s', tags.get('items'))
|
||||
if not tags or not tags.get('items'):
|
||||
return
|
||||
valid_tags = _validate_tags(project, tags['items'])
|
||||
if tags['items'] == valid_tags:
|
||||
logging.info('all tags are valid')
|
||||
return
|
||||
logging.info('modify tags %s %s %s %s %s', project,
|
||||
zone, name, tags['fingerprint'], valid_tags)
|
||||
try:
|
||||
_set_tags(project, zone, name, tags.get('fingerprint'), valid_tags)
|
||||
except Error as e:
|
||||
logging.critical(e.args[0])
|
||||
|
|
@ -0,0 +1 @@
|
|||
google-api-python-client
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
|
||||
|
||||
################################# Quickstart #################################
|
||||
|
||||
- terraform init
|
||||
- terraform apply -var project_id=$GOOGLE_CLOUD_PROJECT
|
||||
- run the command in the command_feed_create output
|
||||
|
||||
Refer to the README.md file for more info and testing flow.
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 102 KiB |
|
|
@ -0,0 +1,110 @@
|
|||
/**
|
||||
* Copyright 2020 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
locals {
|
||||
role_id = "projects/${module.project.project_id}/roles/${local.role_name}"
|
||||
role_name = "feeds_cf"
|
||||
}
|
||||
|
||||
module "project" {
|
||||
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/project?ref=v2.3.0"
|
||||
name = var.project_id
|
||||
project_create = false
|
||||
services = [
|
||||
"cloudasset.googleapis.com",
|
||||
"compute.googleapis.com",
|
||||
"cloudfunctions.googleapis.com"
|
||||
]
|
||||
service_config = {
|
||||
disable_on_destroy = false, disable_dependent_services = false
|
||||
}
|
||||
custom_roles = {
|
||||
(local.role_name) = [
|
||||
"compute.instances.list",
|
||||
"compute.instances.setTags",
|
||||
"compute.zones.list",
|
||||
"compute.zoneOperations.get",
|
||||
"compute.zoneOperations.list"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
module "vpc" {
|
||||
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/net-vpc?ref=v2.3.0"
|
||||
project_id = module.project.project_id
|
||||
name = var.name
|
||||
subnets = [{
|
||||
ip_cidr_range = "192.168.0.0/24"
|
||||
name = "${var.name}-default"
|
||||
region = var.region
|
||||
secondary_ip_range = {}
|
||||
}]
|
||||
}
|
||||
|
||||
module "pubsub" {
|
||||
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/pubsub?ref=v2.3.0"
|
||||
project_id = module.project.project_id
|
||||
name = var.name
|
||||
subscriptions = { "${var.name}-default" = null }
|
||||
}
|
||||
|
||||
module "service-account" {
|
||||
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/iam-service-accounts?ref=v2.3.0"
|
||||
project_id = module.project.project_id
|
||||
names = ["${var.name}-cf"]
|
||||
iam_project_roles = { (module.project.project_id) = [local.role_id] }
|
||||
}
|
||||
|
||||
module "cf" {
|
||||
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/cloud-function?ref=v2.3.0"
|
||||
project_id = module.project.project_id
|
||||
name = var.name
|
||||
bucket_name = "${var.name}-${random_pet.random.id}"
|
||||
bucket_config = {
|
||||
location = var.region
|
||||
lifecycle_delete_age = null
|
||||
}
|
||||
bundle_config = {
|
||||
source_dir = "cf"
|
||||
output_path = var.bundle_path
|
||||
}
|
||||
service_account = module.service-account.email
|
||||
trigger_config = {
|
||||
event = "google.pubsub.topic.publish"
|
||||
resource = module.pubsub.topic.id
|
||||
retry = null
|
||||
}
|
||||
}
|
||||
|
||||
module "simple-vm-example" {
|
||||
source = "github.com/terraform-google-modules/cloud-foundation-fabric//modules/compute-vm?ref=v2.3.0"
|
||||
project_id = module.project.project_id
|
||||
region = var.region
|
||||
zone = "${var.region}-b"
|
||||
name = var.name
|
||||
network_interfaces = [{
|
||||
network = module.vpc.self_link,
|
||||
subnetwork = module.vpc.subnet_self_links["${var.region}/${var.name}-default"],
|
||||
nat = false,
|
||||
addresses = null
|
||||
}]
|
||||
tags = ["${var.project_id}-test-feed", "shared-test-feed"]
|
||||
instance_count = 1
|
||||
}
|
||||
|
||||
resource "random_pet" "random" {
|
||||
length = 1
|
||||
}
|
||||
|
|
@ -0,0 +1,68 @@
|
|||
/**
|
||||
* Copyright 2020 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
output "cf_logs" {
|
||||
description = "Cloud Function logs read command."
|
||||
value = <<END
|
||||
gcloud logging read '
|
||||
logName="projects/${var.project_id}/logs/cloudfunctions.googleapis.com%2Fcloud-functions" AND
|
||||
resource.labels.function_name="${var.name}"' \
|
||||
--project ${var.project_id} \
|
||||
--format "value(severity, timestamp, textPayload)" \
|
||||
--limit 10
|
||||
END
|
||||
}
|
||||
|
||||
output "feed_create" {
|
||||
description = "Feed gcloud command."
|
||||
value = <<END
|
||||
gcloud asset feeds create ${var.name} \
|
||||
--pubsub-topic ${module.pubsub.topic.id} \
|
||||
--asset-types compute.googleapis.com/Instance \
|
||||
--content-type resource \
|
||||
--project ${module.project.project_id}
|
||||
END
|
||||
}
|
||||
|
||||
output "subscription_pull" {
|
||||
description = "Subscription pull command."
|
||||
value = <<END
|
||||
gcloud pubsub subscriptions pull ${var.name}-default \
|
||||
--auto-ack \
|
||||
--format "value(message.data)" \
|
||||
--project ${module.project.project_id}
|
||||
END
|
||||
}
|
||||
|
||||
output "tag_add" {
|
||||
description = "Instance add tag command."
|
||||
value = <<END
|
||||
gcloud compute instances add-tags ${var.name}-1 \
|
||||
--project ${module.project.project_id} \
|
||||
--zone ${var.region}-b \
|
||||
--tags foobar
|
||||
END
|
||||
}
|
||||
|
||||
output "tag_show" {
|
||||
description = "Instance add tag command."
|
||||
value = <<END
|
||||
gcloud compute instances describe ${var.name}-1 \
|
||||
--project ${module.project.project_id} \
|
||||
--zone ${var.region}-b \
|
||||
--format 'yaml(tags)'
|
||||
END
|
||||
}
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
/**
|
||||
* Copyright 2020 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
variable "bundle_path" {
|
||||
description = "Path used to write the intermediate Cloud Function code bundle."
|
||||
type = string
|
||||
default = "./bundle.zip"
|
||||
}
|
||||
|
||||
variable "name" {
|
||||
description = "Arbitrary string used to name created resources."
|
||||
type = string
|
||||
default = "asset-feed"
|
||||
}
|
||||
|
||||
variable "project_id" {
|
||||
description = "Project id that references existing project."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "Compute region used in the example."
|
||||
type = string
|
||||
default = "europe-west1"
|
||||
}
|
||||
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
The examples in this folder implement **typical network topologies** like hub and spoke, or **end-to-end scenarios** that allow testing specific features like on-premises DNS policies and Private Google Access.
|
||||
|
||||
They are meant to be used as minimal but complete strting points to create actual infrastructure, and as playgrounds to experiment with specific Google Cloud features.
|
||||
They are meant to be used as minimal but complete starting points to create actual infrastructure, and as playgrounds to experiment with specific Google Cloud features.
|
||||
|
||||
## Examples
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue