|
||
---|---|---|
.. | ||
cf | ||
README.md | ||
backend.tf.sample | ||
cloud-shell-readme.txt | ||
diagram.png | ||
explorer.png | ||
main.tf | ||
outputs.tf | ||
variables.tf | ||
versions.tf |
README.md
Compute Engine quota monitoring
This blueprint improves on the GCE quota exporter tool (by the same author of this blueprint), and shows a practical way of collecting and monitoring Compute Engine resource quotas via Cloud Monitoring metrics as an alternative to the recently released built-in quota metrics.
Compared to the built-in metrics, it offers a simpler representation of quotas and quota ratios which is especially useful in charts, it allows filtering or combining quotas between different projects regardless of their monitoring workspace, and it creates a default alerting policy without the need to interact directly with the monitoring API.
Regardless of its specific purpose, this blueprint is also useful in showing how to manipulate and write time series to cloud monitoring. The resources it creates are shown in the high level diagram below:
![GCP resource diagram](/zecfoundation/cloud-foundation-fabric/media/commit/303b51a0a99454e978a1dfea05f50c4aa079c2ec/blueprints/cloud-operations/quota-monitoring/diagram.png)
The solution is designed so that the Cloud Function arguments that control function execution (eg to set which project quotas to monitor) are defined in the Cloud Scheduler payload set in the PubSub message, so that a single function can be used for different configurations by creating more schedules.
Quota time series are stored using custom metrics with metric type for usage, limit and utilization; metric types are named using a common prefix and two tokens joined by a -
character:
prefix
(custom.googleapis.com/quota/)quota name
{usage,limit,utilization}
e.g:
custom.googleapis.com/quota/firewalls_usage
custom.googleapis.com/quota/firewalls_limit
custom.googleapis.com/quota/firewalls_utilization
All custom metrics are associated to the global
resource type and use gauge kind
Labels are set with project id (which may differ from the monitoring workspace projects) and region (quotas that are not region specific are labelled global
), this is how a usage/limit/utilization triplet looks in in Metrics Explorer
![GCP Metrics Explorer, usage, limit and utilization view sample](/zecfoundation/cloud-foundation-fabric/media/commit/303b51a0a99454e978a1dfea05f50c4aa079c2ec/blueprints/cloud-operations/quota-monitoring/explorer.png)
The solution can also create a basic monitoring alert policy, to demonstrate how to raise alerts when quotas utilization goes over a predefined threshold, to enable it, set variable alert_create
to true and reapply main.tf after main.py has run at least one and quota monitoring metrics have been creaed.
Running the blueprint
Clone this repository or open it in cloud shell, then go through the following steps to create resources:
terraform init
terraform apply -var project_id=my-project-id
Variables
name | description | type | required | default |
---|---|---|---|---|
project_id | Project id that references existing project. | string |
✓ | |
alert_create | Enables the creation of a sample monitoring alert, false by default. | bool |
false |
|
bundle_path | Path used to write the intermediate Cloud Function code bundle. | string |
"./bundle.zip" |
|
name | Arbitrary string used to name created resources. | string |
"quota-monitor" |
|
project_create | Create project instead of using an existing one. | bool |
false |
|
quota_config | Cloud function configuration. | object({…}) |
{…} |
|
region | Compute region used in the example. | string |
"europe-west1" |
|
schedule_config | Schedule timer configuration in crontab format. | string |
"0 * * * *" |