2020-06-18 11:12:44 -07:00
|
|
|
<!--
|
2021-01-17 16:11:26 -08:00
|
|
|
order: 10
|
2020-06-18 11:12:44 -07:00
|
|
|
-->
|
|
|
|
|
|
|
|
|
|
# Telemetry
|
|
|
|
|
|
2020-07-07 03:19:36 -07:00
|
|
|
Gather relevant insights about your application and modules with custom metrics and telemetry. {synopsis}
|
|
|
|
|
|
2020-06-18 11:12:44 -07:00
|
|
|
The Cosmos SDK enables operators and developers to gain insight into the performance and behavior of
|
|
|
|
|
their application through the use of the `telemetry` package. The Cosmos SDK currently supports
|
|
|
|
|
enabling in-memory and prometheus as telemetry sinks. This allows the ability to query for and scrape
|
|
|
|
|
metrics from a single exposed API endpoint -- `/metrics?format={text|prometheus}`, the default being
|
|
|
|
|
`text`.
|
|
|
|
|
|
|
|
|
|
If telemetry is enabled via configuration, a single global metrics collector is registered via the
|
|
|
|
|
[go-metrics](https://github.com/armon/go-metrics) library. This allows emitting and collecting
|
|
|
|
|
metrics through simple API calls.
|
|
|
|
|
|
|
|
|
|
Example:
|
|
|
|
|
|
|
|
|
|
```go
|
|
|
|
|
func EndBlocker(ctx sdk.Context, k keeper.Keeper) {
|
2020-07-17 12:33:50 -07:00
|
|
|
defer telemetry.ModuleMeasureSince(types.ModuleName, time.Now(), telemetry.MetricKeyEndBlocker)
|
2020-06-18 11:12:44 -07:00
|
|
|
|
|
|
|
|
// ...
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Developers may use the `telemetry` package directly, which provides wrappers around metric APIs
|
|
|
|
|
that include adding useful labels, or they must use the `go-metrics` library directly. It is preferable
|
|
|
|
|
to add as much context and adequate dimensionality to metrics as possible, so the `telemetry` package
|
|
|
|
|
is advised. Regardless of the package or method used, the Cosmos SDK supports the following metrics
|
|
|
|
|
types:
|
|
|
|
|
|
|
|
|
|
* gauges
|
|
|
|
|
* summaries
|
|
|
|
|
* counters
|
|
|
|
|
|
|
|
|
|
## Labels
|
|
|
|
|
|
|
|
|
|
Certain components of modules will have their name automatically added as a label (e.g. `BeginBlock`).
|
|
|
|
|
Operators may also supply the application with a global set of labels that will be applied to all
|
|
|
|
|
metrics emitted using the `telemetry` package (e.g. chain-id). Global labels are supplied as a list
|
|
|
|
|
of [name, value] tuples.
|
|
|
|
|
|
|
|
|
|
Example:
|
|
|
|
|
|
|
|
|
|
```toml
|
|
|
|
|
global-labels = [
|
|
|
|
|
["chain_id", "chain-OfXo4V"],
|
|
|
|
|
]
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Cardinality
|
|
|
|
|
|
|
|
|
|
Cardinality is key, specifically label and key cardinality. Cardinality is how many unique values of
|
|
|
|
|
something there are. So there is naturally a tradeoff between granularity and how much stress is put
|
|
|
|
|
on the telemetry sink in terms of indexing, scrape, and query performance.
|
|
|
|
|
|
|
|
|
|
Developers should take care to support metrics with enough dimensionality and granularity to be
|
|
|
|
|
useful, but not increase the cardinality beyond the sink's limits. A general rule of thumb is to not
|
|
|
|
|
exceed a cardinality of 10.
|
|
|
|
|
|
|
|
|
|
Consider the following examples with enough granularity and adequate cardinality:
|
|
|
|
|
|
|
|
|
|
* begin/end blocker time
|
|
|
|
|
* tx gas used
|
|
|
|
|
* block gas used
|
|
|
|
|
* amount of tokens minted
|
|
|
|
|
* amount of accounts created
|
|
|
|
|
|
|
|
|
|
The following examples expose too much cardinality and may not even prove to be useful:
|
|
|
|
|
|
|
|
|
|
* transfers between accounts with amount
|
|
|
|
|
* voting/deposit amount from unique addresses
|
|
|
|
|
|
|
|
|
|
## Supported Metrics
|
|
|
|
|
|
2020-10-08 05:37:20 -07:00
|
|
|
| Metric | Description | Unit | Type |
|
|
|
|
|
|:--------------------------------|:------------------------------------------------------------------------------------------|:----------------|:--------|
|
|
|
|
|
| `tx_count` | Total number of txs processed via `DeliverTx` | tx | counter |
|
|
|
|
|
| `tx_successful` | Total number of successful txs processed via `DeliverTx` | tx | counter |
|
|
|
|
|
| `tx_failed` | Total number of failed txs processed via `DeliverTx` | tx | counter |
|
|
|
|
|
| `tx_gas_used` | The total amount of gas used by a tx | gas | gauge |
|
|
|
|
|
| `tx_gas_wanted` | The total amount of gas requested by a tx | gas | gauge |
|
|
|
|
|
| `tx_msg_send` | The total amount of tokens sent in a `MsgSend` (per denom) | token | gauge |
|
|
|
|
|
| `tx_msg_withdraw_reward` | The total amount of tokens withdrawn in a `MsgWithdrawDelegatorReward` (per denom) | token | gauge |
|
|
|
|
|
| `tx_msg_withdraw_commission` | The total amount of tokens withdrawn in a `MsgWithdrawValidatorCommission` (per denom) | token | gauge |
|
|
|
|
|
| `tx_msg_delegate` | The total amount of tokens delegated in a `MsgDelegate` | token | gauge |
|
|
|
|
|
| `tx_msg_begin_unbonding` | The total amount of tokens undelegated in a `MsgUndelegate` | token | gauge |
|
|
|
|
|
| `tx_msg_begin_begin_redelegate` | The total amount of tokens redelegated in a `MsgBeginRedelegate` | token | gauge |
|
|
|
|
|
| `tx_msg_ibc_transfer` | The total amount of tokens transferred via IBC in a `MsgTransfer` (source or sink chain) | token | gauge |
|
|
|
|
|
| `ibc_transfer_packet_receive` | The total amount of tokens received in a `FungibleTokenPacketData` (source or sink chain) | token | gauge |
|
|
|
|
|
| `new_account` | Total number of new accounts created | account | counter |
|
|
|
|
|
| `gov_proposal` | Total number of governance proposals | proposal | counter |
|
|
|
|
|
| `gov_vote` | Total number of governance votes for a proposal | vote | counter |
|
|
|
|
|
| `gov_deposit` | Total number of governance deposits for a proposal | deposit | counter |
|
|
|
|
|
| `staking_delegate` | Total number of delegations | delegation | counter |
|
|
|
|
|
| `staking_undelegate` | Total number of undelegations | undelegation | counter |
|
|
|
|
|
| `staking_redelegate` | Total number of redelegations | redelegation | counter |
|
|
|
|
|
| `ibc_transfer_send` | Total number of IBC transfers sent from a chain (source or sink) | transfer | counter |
|
|
|
|
|
| `ibc_transfer_receive` | Total number of IBC transfers received to a chain (source or sink) | transfer | counter |
|
|
|
|
|
| `ibc_client_create` | Total number of clients created | create | counter |
|
|
|
|
|
| `ibc_client_update` | Total number of client updates | update | counter |
|
|
|
|
|
| `ibc_client_upgrade` | Total number of client upgrades | upgrade | counter |
|
|
|
|
|
| `ibc_client_misbehaviour` | Total number of client misbehaviours | misbehaviour | counter |
|
|
|
|
|
| `ibc_connection_open-init` | Total number of connection `OpenInit` handshakes | handshake | counter |
|
|
|
|
|
| `ibc_connection_open-try` | Total number of connection `OpenTry` handshakes | handshake | counter |
|
|
|
|
|
| `ibc_connection_open-ack` | Total number of connection `OpenAck` handshakes | handshake | counter |
|
|
|
|
|
| `ibc_connection_open-confirm` | Total number of connection `OpenConfirm` handshakes | handshake | counter |
|
|
|
|
|
| `ibc_channel_open-init` | Total number of channel `OpenInit` handshakes | handshake | counter |
|
|
|
|
|
| `ibc_channel_open-try` | Total number of channel `OpenTry` handshakes | handshake | counter |
|
|
|
|
|
| `ibc_channel_open-ack` | Total number of channel `OpenAck` handshakes | handshake | counter |
|
|
|
|
|
| `ibc_channel_open-confirm` | Total number of channel `OpenConfirm` handshakes | handshake | counter |
|
|
|
|
|
| `ibc_channel_close-init` | Total number of channel `CloseInit` handshakes | handshake | counter |
|
|
|
|
|
| `ibc_channel_close-confirm` | Total number of channel `CloseConfirm` handshakes | handshake | counter |
|
|
|
|
|
| `tx_msg_ibc_recv_packet` | Total number of IBC packets received | packet | counter |
|
|
|
|
|
| `tx_msg_ibc_acknowledge_packet` | Total number of IBC packets acknowledged | acknowledgement | counter |
|
|
|
|
|
| `ibc_timeout_packet` | Total number of IBC timeout packets | timeout | counter |
|
|
|
|
|
| `abci_check_tx` | Duration of ABCI `CheckTx` | ms | summary |
|
|
|
|
|
| `abci_deliver_tx` | Duration of ABCI `DeliverTx` | ms | summary |
|
|
|
|
|
| `abci_commit` | Duration of ABCI `Commit` | ms | summary |
|
|
|
|
|
| `abci_query` | Duration of ABCI `Query` | ms | summary |
|
|
|
|
|
| `abci_begin_block` | Duration of ABCI `BeginBlock` | ms | summary |
|
|
|
|
|
| `abci_end_block` | Duration of ABCI `EndBlock` | ms | summary |
|
|
|
|
|
| `begin_blocker` | Duration of `BeginBlock` for a given module | ms | summary |
|
|
|
|
|
| `end_blocker` | Duration of `EndBlock` for a given module | ms | summary |
|
|
|
|
|
| `store_iavl_get` | Duration of an IAVL `Store#Get` call | ms | summary |
|
|
|
|
|
| `store_iavl_set` | Duration of an IAVL `Store#Set` call | ms | summary |
|
|
|
|
|
| `store_iavl_has` | Duration of an IAVL `Store#Has` call | ms | summary |
|
|
|
|
|
| `store_iavl_delete` | Duration of an IAVL `Store#Delete` call | ms | summary |
|
|
|
|
|
| `store_iavl_commit` | Duration of an IAVL `Store#Commit` call | ms | summary |
|
|
|
|
|
| `store_iavl_query` | Duration of an IAVL `Store#Query` call | ms | summary |
|
|
|
|
|
| `store_gaskv_get` | Duration of a GasKV `Store#Get` call | ms | summary |
|
|
|
|
|
| `store_gaskv_set` | Duration of a GasKV `Store#Set` call | ms | summary |
|
|
|
|
|
| `store_gaskv_has` | Duration of a GasKV `Store#Has` call | ms | summary |
|
|
|
|
|
| `store_gaskv_delete` | Duration of a GasKV `Store#Delete` call | ms | summary |
|
|
|
|
|
| `store_cachekv_get` | Duration of a CacheKV `Store#Get` call | ms | summary |
|
|
|
|
|
| `store_cachekv_set` | Duration of a CacheKV `Store#Set` call | ms | summary |
|
|
|
|
|
| `store_cachekv_write` | Duration of a CacheKV `Store#Write` call | ms | summary |
|
|
|
|
|
| `store_cachekv_delete` | Duration of a CacheKV `Store#Delete` call | ms | summary |
|
2020-07-07 03:19:36 -07:00
|
|
|
|
|
|
|
|
## Next {hide}
|
|
|
|
|
|
|
|
|
|
Learn about the [object-capability](./ocap.md) model {hide}
|
