cloud-foundation-fabric/cloud-operations/asset-inventory-feed-remediation

Cloud Asset Inventory feeds for resource change tracking and remediation

This example shows how to leverage Cloud Asset Inventory feeds to stream resource changes in real time, and how to programmatically react to changes by wiring a Cloud Function to the feed outputs.

The Cloud Function can then be used for different purposes:

• updating remote data (eg a CMDB) to reflect the changed resources
• triggering alerts to surface critical changes
• adapting the configuration of separate related resources
• implementing remediation steps that enforce policy compliance by tweaking or reverting the changes.

A companion Medium article has been published for this example, refer to it for more details on the context and the specifics of running the example.

This example shows a simple remediation use case: how to enforce policies on instance tags and revert non-compliant changes in near-real time, thus adding an additional measure of control when using tags for firewall rule scoping. Changing the monitored asset and the function logic allows simple adaptation to other common use cases:

• enforcing a centrally defined Cloud Armor policy in backend services
• creating custom DNS records for instances or forwarding rules

The example uses a single project for ease of testing, in actual use a few changes are needed to operate at the resource hierarchy level:

• the feed should be set at the folder or organization level
• the custom role used to assign tag changing permissions should be defined at the organization level
• the role binding that grants the custom role to the Cloud Function service account should be set at the same level as the feed (folder or organization)

The resources created in this example are shown in the high level diagram below:

Running the example

Clone this repository or open it in cloud shell, then go through the following steps to create resources:

• terraform init
• terraform apply -var project_id=my-project-id
• copy and paste the feed_create output in the console then run it to create the feed

Once done testing, you can clean up resources by running terraform destroy. To persist state, check out the backend.tf.sample file.

Testing the example

The terraform outputs generate preset gcloud commands that you can copy and run in the console, to complete configuration and test the example:

• feed_create is run once to create the feed, as there's currently no Terraform resource available for Cloud Asset feeds
• subscription_pull shows messages in the PubSub queue, to check feed message format if the Cloud Function is disabled
• cf_logs shows Cloud Function logs to check that remediation works
• tag_add adds a non-compliant tag to the test instance, and triggers the Cloud Function remediation process
• tag_show displays the tags currently set on the test instance

Run the subscription_pull command until it returns nothing, then run the following commands in order to test remediation:

• the tag_add command
• the cf_logs command until the logs show that the change has been picked up, verified, and the compliant tags have been force-set on the instance
• the tag_show command to verify that the function output matches the resource state

Variables

name	description	type	required	default
project_id	Project id that references existing project.	string	✓
bundle_path	Path used to write the intermediate Cloud Function code bundle.	string		./bundle.zip
name	Arbitrary string used to name created resources.	string		asset-feed
region	Compute region used in the example.	string		europe-west1

Outputs

name	description	sensitive
cf_logs	Cloud Function logs read command.
feed_create	Feed gcloud command.
subscription_pull	Subscription pull command.
tag_add	Instance add tag command.
tag_show	Instance add tag command.