Merge branch 'tobz/init-workspace'
This commit is contained in:
Toby Lawrence
commit
6c9ae9c85b
|
|
@ -0,0 +1,27 @@
|
|||
# The Code of Conduct
|
||||
|
||||
This document is based on the [Rust Code of Conduct](https://www.rust-lang.org/conduct.html) and outlines the standard of conduct which is both expected and enforced as part of this project.
|
||||
|
||||
## Conduct
|
||||
|
||||
**Contact**: [toby@nuclearfurnace.com](mailto:toby@nuclearfurnace.com)
|
||||
|
||||
* We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.
|
||||
* Avoid using overtly sexual nicknames or other nicknames that might detract from a friendly, safe and welcoming environment for all.
|
||||
* Please be kind and courteous. There's no need to be mean or rude.
|
||||
* Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
|
||||
* Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
|
||||
* We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behaviour. We interpret the term "harassment" as including the definition in the <a href="http://citizencodeofconduct.org/">Citizen Code of Conduct</a>; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don't tolerate behavior that excludes people in socially marginalized groups.
|
||||
* Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the repository Owners immediately. Whether you're a regular contributor or a newcomer, we care about making this community a safe place for you and we've got your back.
|
||||
* Likewise any spamming, trolling, flaming, baiting or other attention-stealing behaviour is not welcome.
|
||||
|
||||
## Moderation
|
||||
|
||||
These are the policies for upholding our community's standards of conduct. If you feel that a thread needs moderation, please use the contact information above, or mention @tobz or @LucioFranco in the thread.
|
||||
|
||||
1. Remarks that violate this Code of Conduct, including hateful, hurtful, oppressive, or exclusionary remarks, are not allowed. (Cursing is allowed, but never targeting another user, and never in a hateful manner.)
|
||||
2. Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
|
||||
|
||||
In the Rust community we strive to go the extra step to look out for each other. Don't just aim to be technically unimpeachable, try to be your best self. In particular, avoid flirting with offensive or sensitive issues, particularly if they're off-topic; this all too often leads to unnecessary fights, hurt feelings, and damaged trust; worse, it can drive people away from the community entirely.
|
||||
|
||||
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could've communicated better — remember that it's your responsibility to make your fellow Rustaceans comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
Short version for non-lawyers:
|
||||
|
||||
metrics is MIT licensed.
|
||||
|
||||
Longer version:
|
||||
|
||||
Copyrights in the metrics project are retained by their contributors. No
|
||||
copyright assignment is required to contribute to the metrics project.
|
||||
|
||||
Some files include explicit copyright notices and/or license notices.
|
||||
For full authorship information, see the version control history.
|
||||
|
||||
Except as otherwise noted (below and/or in individual files), metrics
|
||||
is licensed under the MIT license <LICENSE> or
|
||||
<http://opensource.org/licenses/MIT>.
|
||||
|
||||
|
||||
metrics includes packages written by third parties.
|
||||
The following third party packages are included, and carry
|
||||
their own copyright notices and license terms:
|
||||
|
||||
* Portions of the API design are derived from tic
|
||||
<https://github.com/brayniac/tic>, which carries the following
|
||||
license:
|
||||
|
||||
Copyright (c) 2016 Brian Martin
|
||||
|
||||
Permission is hereby granted, free of charge, to any person
|
||||
obtaining a copy of this software and associated documentation
|
||||
files (the "Software"), to deal in the Software without restriction,
|
||||
including without limitation the rights to use, copy, modify, merge,
|
||||
publish, distribute, sublicense, and/or sell copies of the Software,
|
||||
and to permit persons to whom the Software is furnished to do so,
|
||||
subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be
|
||||
included in all copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
|
||||
BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
|
||||
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
||||
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
[workspace]
|
||||
members = [
|
||||
"metrics-core",
|
||||
"metrics",
|
||||
"metrics-util",
|
||||
"metrics-exporter-log",
|
||||
"metrics-recorder-text",
|
||||
"metrics-recorder-prometheus",
|
||||
]
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
// Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
// of this software and associated documentation files (the "Software"), to deal
|
||||
// in the Software without restriction, including without limitation the rights
|
||||
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
// copies of the Software, and to permit persons to whom the Software is
|
||||
// furnished to do so, subject to the following conditions:
|
||||
//
|
||||
// The above copyright notice and this permission notice shall be included in all
|
||||
// copies or substantial portions of the Software.
|
||||
//
|
||||
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
// SOFTWARE.
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
# metrics
|
||||
|
||||
[![conduct-badge][]][conduct] [![downloads-badge][] ![release-badge][]][crate] [![docs-badge][]][docs] [![license-badge][]](#license)
|
||||
|
||||
[conduct-badge]: https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg
|
||||
[downloads-badge]: https://img.shields.io/crates/d/metrics.svg
|
||||
[release-badge]: https://img.shields.io/crates/v/metrics.svg
|
||||
[license-badge]: https://img.shields.io/crates/l/metrics.svg
|
||||
[docs-badge]: https://docs.rs/metrics/badge.svg
|
||||
[conduct]: https://github.com/metrics-rs/metrics/blob/master/CODE_OF_CONDUCT.md
|
||||
[crate]: https://crates.io/crates/metrics
|
||||
[docs]: https://docs.rs/metrics
|
||||
|
||||
__metrics__ is a high-quality, batteries-included metrics library for Rust.
|
||||
|
||||
## code of conduct
|
||||
|
||||
**NOTE**: All conversations and contributions to this project shall adhere to the [Code of Conduct][conduct].
|
||||
|
||||
# caveat emptor
|
||||
|
||||
This crate is currently materializing! We are in the process of switching over [hotmic](https://github.com/nuclearfurnace/hotmic) to `metrics` after successfully acquiring ownership of the `metrics` crate on crates.io!
|
||||
|
||||
We apologize for the README/documentation that will reference things that don't exist yet until the switchover is complete. Thank you for your understanding!
|
||||
|
||||
## general features
|
||||
- Provides counter, gauge, and histogram support.
|
||||
- Access to ultra-high-speed timing facilities out-of-the-box with [quanta](https://github.com/nuclearfurnace/quanta).
|
||||
- Scoped metrics for effortless nesting.
|
||||
- Speed and API ergonomics allow for usage in both synchronous and asynchronous contexts.
|
||||
- Based on `metrics-core` for bring-your-own-collector/bring-your-own-exporter flexibility!
|
||||
|
||||
## performance
|
||||
|
||||
High. Tens of millions of metrics per second with metric ingest times at sub-200ns p99 on modern systems.
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
/target
|
||||
**/*.rs.bk
|
||||
Cargo.lock
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
# The Code of Conduct
|
||||
|
||||
This document is based on the [Rust Code of Conduct](https://www.rust-lang.org/conduct.html) and outlines the standard of conduct which is both expected and enforced as part of this project.
|
||||
|
||||
## Conduct
|
||||
|
||||
**Contact**: [toby@nuclearfurnace.com](mailto:toby@nuclearfurnace.com)
|
||||
|
||||
* We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.
|
||||
* Avoid using overtly sexual nicknames or other nicknames that might detract from a friendly, safe and welcoming environment for all.
|
||||
* Please be kind and courteous. There's no need to be mean or rude.
|
||||
* Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
|
||||
* Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
|
||||
* We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behaviour. We interpret the term "harassment" as including the definition in the <a href="http://citizencodeofconduct.org/">Citizen Code of Conduct</a>; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don't tolerate behavior that excludes people in socially marginalized groups.
|
||||
* Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the repository Owners immediately. Whether you're a regular contributor or a newcomer, we care about making this community a safe place for you and we've got your back.
|
||||
* Likewise any spamming, trolling, flaming, baiting or other attention-stealing behaviour is not welcome.
|
||||
|
||||
## Moderation
|
||||
|
||||
These are the policies for upholding our community's standards of conduct. If you feel that a thread needs moderation, please use the contact information above, or mention @tobz or @LucioFranco in the thread.
|
||||
|
||||
1. Remarks that violate this Code of Conduct, including hateful, hurtful, oppressive, or exclusionary remarks, are not allowed. (Cursing is allowed, but never targeting another user, and never in a hateful manner.)
|
||||
2. Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
|
||||
|
||||
In the Rust community we strive to go the extra step to look out for each other. Don't just aim to be technically unimpeachable, try to be your best self. In particular, avoid flirting with offensive or sensitive issues, particularly if they're off-topic; this all too often leads to unnecessary fights, hurt feelings, and damaged trust; worse, it can drive people away from the community entirely.
|
||||
|
||||
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could've communicated better — remember that it's your responsibility to make your fellow Rustaceans comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
[package]
|
||||
name = "metrics-core"
|
||||
version = "0.2.0"
|
||||
authors = ["Toby Lawrence <toby@nuclearfurnace.com>"]
|
||||
edition = "2018"
|
||||
|
||||
license = "MIT"
|
||||
|
||||
description = "Foundational traits for interoperable metrics libraries in Rust."
|
||||
|
||||
homepage = "https://github.com/metrics-rs/metrics"
|
||||
repository = "https://github.com/metrics-rs/metrics"
|
||||
documentation = "https://docs.rs/metrics-core"
|
||||
|
||||
readme = "README.md"
|
||||
|
||||
keywords = ["metrics", "interface", "common"]
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
// Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
// of this software and associated documentation files (the "Software"), to deal
|
||||
// in the Software without restriction, including without limitation the rights
|
||||
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
// copies of the Software, and to permit persons to whom the Software is
|
||||
// furnished to do so, subject to the following conditions:
|
||||
//
|
||||
// The above copyright notice and this permission notice shall be included in all
|
||||
// copies or substantial portions of the Software.
|
||||
//
|
||||
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
// SOFTWARE.
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
# metrics-core
|
||||
|
||||
[![conduct-badge][]][conduct] [![downloads-badge][] ![release-badge][]][crate] [![docs-badge][]][docs] [![license-badge][]](#license)
|
||||
|
||||
[conduct-badge]: https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg
|
||||
[downloads-badge]: https://img.shields.io/crates/d/metrics-core.svg
|
||||
[release-badge]: https://img.shields.io/crates/v/metrics-core.svg
|
||||
[license-badge]: https://img.shields.io/crates/l/metrics-core.svg
|
||||
[docs-badge]: https://docs.rs/metrics-core/badge.svg
|
||||
[conduct]: https://github.com/metrics-rs/metrics-core/blob/master/CODE_OF_CONDUCT.md
|
||||
[crate]: https://crates.io/crates/metrics-core
|
||||
[docs]: https://docs.rs/metrics-core
|
||||
|
||||
__metrics-core__ defines foundational traits for interoperable metrics libraries in Rust.
|
||||
|
||||
## code of conduct
|
||||
|
||||
**NOTE**: All conversations and contributions to this project shall adhere to the [Code of Conduct][conduct].
|
||||
|
||||
## mandate / goals
|
||||
|
||||
This crate acts as the minimum viable trait for metrics libraries, and consumers of that data, for interoperating with each other.
|
||||
|
||||
If your library allows users to collect metrics, it should support metrics-core to allow for flexibility in output targets. If your library provides support for a target metrics backend, it should support metrics-core so that it can be easily plugged into applications using a supported metrics library.
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
//! Foundational traits for interoperable metrics libraries in Rust.
|
||||
//!
|
||||
//! # Common Ground
|
||||
//! Most libraries, under the hood, are all based around a core set of data types: counters,
|
||||
//! gauges, and histograms. While the API surface may differ, the underlying data is the same.
|
||||
//!
|
||||
//! # Metric Types
|
||||
//!
|
||||
//! ## Counters
|
||||
//! Counters represent a single value that can only ever be incremented over time, or reset to
|
||||
//! zero.
|
||||
//!
|
||||
//! Counters are useful for tracking things like operations completed, or errors raised, where
|
||||
//! the value naturally begins at zero when a process or service is started or restarted.
|
||||
//!
|
||||
//! ## Gauges
|
||||
//! Gauges represent a single value that can go up _or_ down over time.
|
||||
//!
|
||||
//! Gauges are useful for tracking things like the current number of connected users, or a stock
|
||||
//! price, or the temperature outside.
|
||||
//!
|
||||
//! ## Histograms
|
||||
//! Histograms measure the distribution of values for a given set of measurements.
|
||||
//!
|
||||
//! Histograms are generally used to derive statistics about a particular measurement from an
|
||||
//! operation or event that happens over and over, such as the duration of a request, or number of
|
||||
//! rows returned by a particular database query.
|
||||
//!
|
||||
//! Histograms allow you to answer questions of these measurements, such as:
|
||||
//! - "What were the fastest and slowest requests in this window?"
|
||||
//! - "What is the slowest request we've seen out of 90% of the requests measured? 99%?"
|
||||
//!
|
||||
//! Histograms are a convenient way to measure behavior not only at the median, but at the edges of
|
||||
//! normal operating behavior.
|
||||
|
||||
/// A value that records metrics.
|
||||
pub trait MetricsRecorder {
|
||||
/// Records a counter.
|
||||
///
|
||||
/// From the perspective of an recorder, a counter and gauge are essentially identical, insofar
|
||||
/// as they are both a single value tied to a key. From the perspective of a collector,
|
||||
/// counters and gauges usually have slightly different modes of operation.
|
||||
///
|
||||
/// For the sake of flexibility on the exportr side, both are provided.
|
||||
fn record_counter<K: AsRef<str>>(&mut self, key: K, value: u64);
|
||||
|
||||
/// Records a gauge.
|
||||
///
|
||||
/// From the perspective of a recorder, a counter and gauge are essentially identical, insofar
|
||||
/// as they are both a single value tied to a key. From the perspective of a collector,
|
||||
/// counters and gauges usually have slightly different modes of operation.
|
||||
///
|
||||
/// For the sake of flexibility on the exportr side, both are provided.
|
||||
fn record_gauge<K: AsRef<str>>(&mut self, key: K, value: i64);
|
||||
|
||||
/// Records a histogram.
|
||||
///
|
||||
/// Recorders are expected to tally their own histogram views, so this will be called with all
|
||||
/// of the underlying observed values, and callers will need to process them accordingly.
|
||||
fn record_histogram<K: AsRef<str>>(&mut self, key: K, values: &[u64]);
|
||||
}
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
/target
|
||||
**/*.rs.bk
|
||||
Cargo.lock
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
# The Code of Conduct
|
||||
|
||||
This document is based on the [Rust Code of Conduct](https://www.rust-lang.org/conduct.html) and outlines the standard of conduct which is both expected and enforced as part of this project.
|
||||
|
||||
## Conduct
|
||||
|
||||
**Contact**: [toby@nuclearfurnace.com](mailto:toby@nuclearfurnace.com)
|
||||
|
||||
* We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.
|
||||
* Avoid using overtly sexual nicknames or other nicknames that might detract from a friendly, safe and welcoming environment for all.
|
||||
* Please be kind and courteous. There's no need to be mean or rude.
|
||||
* Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
|
||||
* Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
|
||||
* We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behaviour. We interpret the term "harassment" as including the definition in the <a href="http://citizencodeofconduct.org/">Citizen Code of Conduct</a>; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don't tolerate behavior that excludes people in socially marginalized groups.
|
||||
* Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the repository Owners immediately. Whether you're a regular contributor or a newcomer, we care about making this community a safe place for you and we've got your back.
|
||||
* Likewise any spamming, trolling, flaming, baiting or other attention-stealing behaviour is not welcome.
|
||||
|
||||
## Moderation
|
||||
|
||||
These are the policies for upholding our community's standards of conduct. If you feel that a thread needs moderation, please use the contact information above, or mention @tobz or @LucioFranco in the thread.
|
||||
|
||||
1. Remarks that violate this Code of Conduct, including hateful, hurtful, oppressive, or exclusionary remarks, are not allowed. (Cursing is allowed, but never targeting another user, and never in a hateful manner.)
|
||||
2. Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
|
||||
|
||||
In the Rust community we strive to go the extra step to look out for each other. Don't just aim to be technically unimpeachable, try to be your best self. In particular, avoid flirting with offensive or sensitive issues, particularly if they're off-topic; this all too often leads to unnecessary fights, hurt feelings, and damaged trust; worse, it can drive people away from the community entirely.
|
||||
|
||||
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could've communicated better — remember that it's your responsibility to make your fellow Rustaceans comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
[package]
|
||||
name = "metrics-exporter-log"
|
||||
version = "0.1.0"
|
||||
authors = ["Toby Lawrence <toby@nuclearfurnace.com>"]
|
||||
edition = "2018"
|
||||
|
||||
license = "MIT"
|
||||
|
||||
description = "metric exporter for outputting to logs"
|
||||
|
||||
homepage = "https://github.com/metrics-rs/metrics"
|
||||
repository = "https://github.com/metrics-rs/metrics-exporter-log"
|
||||
documentation = "https://docs.rs/metrics-exporter-log"
|
||||
|
||||
[dependencies]
|
||||
metrics-core = { path = "../metrics-core", version = "^0.2" }
|
||||
metrics = { path = "../metrics", version = "^0.9" }
|
||||
log = "^0.4"
|
||||
futures = "^0.1"
|
||||
tokio-timer = "^0.2"
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
# metrics-exporter-log
|
||||
|
||||
[![conduct-badge][]][conduct] [![downloads-badge][] ![release-badge][]][crate] [![docs-badge][]][docs] [![license-badge][]](#license)
|
||||
|
||||
[conduct-badge]: https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg
|
||||
[downloads-badge]: https://img.shields.io/crates/d/metrics-exporter-log.svg
|
||||
[release-badge]: https://img.shields.io/crates/v/metrics-exporter-log.svg
|
||||
[license-badge]: https://img.shields.io/crates/l/metrics-exporter-log.svg
|
||||
[docs-badge]: https://docs.rs/metrics-exporter-log/badge.svg
|
||||
[conduct]: https://github.com/metrics-rs/metrics-exporter-log/blob/master/CODE_OF_CONDUCT.md
|
||||
[crate]: https://crates.io/crates/metrics-exporter-log
|
||||
[docs]: https://docs.rs/metrics-exporter-log
|
||||
|
||||
__metrics-exporter-log__ is a metric exporter that outputs metrics in a textual format via the `log` crate.
|
||||
|
||||
## code of conduct
|
||||
|
||||
**NOTE**: All conversations and contributions to this project shall adhere to the [Code of Conduct][conduct].
|
||||
|
|
@ -0,0 +1,88 @@
|
|||
//! Exports metrics via the `log` crate.
|
||||
//!
|
||||
//! This exporter can utilize recorders that are able to be converted to a textual representation
|
||||
//! via [`Into`]. It will emit that output by logging via the `log` crate at the specified
|
||||
//! level.
|
||||
//!
|
||||
//! # Run Modes
|
||||
//! - `run` can be used to block the current thread, taking snapshots and exporting them on an
|
||||
//! interval
|
||||
//! - `turn` can be used to take a single snapshot and log it
|
||||
//! - `into_future` will return a [`Future`] that when driven will take a snapshot on the
|
||||
//! configured interval and log it
|
||||
#[macro_use]
|
||||
extern crate log;
|
||||
|
||||
use std::thread;
|
||||
use std::time::Duration;
|
||||
use metrics::Controller;
|
||||
use metrics_core::MetricsRecorder;
|
||||
use log::Level;
|
||||
use futures::prelude::*;
|
||||
use tokio_timer::Interval;
|
||||
|
||||
/// Exports metrics by converting them to a textual representation and logging them.
|
||||
pub struct LogExporter<R> {
|
||||
controller: Controller,
|
||||
recorder: R,
|
||||
level: Level,
|
||||
}
|
||||
|
||||
impl<R> LogExporter<R>
|
||||
where
|
||||
R: MetricsRecorder + Clone + Into<String>
|
||||
{
|
||||
/// Creates a new [`LogExporter`] that logs at the configurable level.
|
||||
///
|
||||
/// Recorders expose their output by being converted into strings.
|
||||
pub fn new(controller: Controller, recorder: R, level: Level) -> Self {
|
||||
LogExporter {
|
||||
controller,
|
||||
recorder,
|
||||
level,
|
||||
}
|
||||
}
|
||||
|
||||
/// Runs this exporter on the current thread, logging output on the given interval.
|
||||
pub fn run(&mut self, interval: Duration) {
|
||||
loop {
|
||||
thread::sleep(interval);
|
||||
|
||||
self.turn();
|
||||
}
|
||||
}
|
||||
|
||||
/// Run this exporter, logging output only once.
|
||||
pub fn turn(&self) {
|
||||
run_once(&self.controller, self.recorder.clone(), self.level);
|
||||
}
|
||||
|
||||
/// Converts this exporter into a future which logs output on the given interval.
|
||||
pub fn into_future(self, interval: Duration) -> impl Future<Item = (), Error = ()> {
|
||||
let controller = self.controller;
|
||||
let recorder = self.recorder;
|
||||
let level = self.level;
|
||||
|
||||
Interval::new_interval(interval)
|
||||
.map_err(|_| ())
|
||||
.for_each(move |_| {
|
||||
let recorder = recorder.clone();
|
||||
run_once(&controller, recorder, level);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn run_once<R>(controller: &Controller, mut recorder: R, level: Level)
|
||||
where
|
||||
R: MetricsRecorder + Into<String>
|
||||
{
|
||||
match controller.get_snapshot() {
|
||||
Ok(snapshot) => {
|
||||
snapshot.record(&mut recorder);
|
||||
let output = recorder.into();
|
||||
log!(level, "{}", output);
|
||||
},
|
||||
Err(e) => log!(Level::Error, "failed to capture snapshot: {}", e),
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
/target
|
||||
**/*.rs.bk
|
||||
Cargo.lock
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
# The Code of Conduct
|
||||
|
||||
This document is based on the [Rust Code of Conduct](https://www.rust-lang.org/conduct.html) and outlines the standard of conduct which is both expected and enforced as part of this project.
|
||||
|
||||
## Conduct
|
||||
|
||||
**Contact**: [toby@nuclearfurnace.com](mailto:toby@nuclearfurnace.com)
|
||||
|
||||
* We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.
|
||||
* Avoid using overtly sexual nicknames or other nicknames that might detract from a friendly, safe and welcoming environment for all.
|
||||
* Please be kind and courteous. There's no need to be mean or rude.
|
||||
* Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
|
||||
* Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
|
||||
* We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behaviour. We interpret the term "harassment" as including the definition in the <a href="http://citizencodeofconduct.org/">Citizen Code of Conduct</a>; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don't tolerate behavior that excludes people in socially marginalized groups.
|
||||
* Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the repository Owners immediately. Whether you're a regular contributor or a newcomer, we care about making this community a safe place for you and we've got your back.
|
||||
* Likewise any spamming, trolling, flaming, baiting or other attention-stealing behaviour is not welcome.
|
||||
|
||||
## Moderation
|
||||
|
||||
These are the policies for upholding our community's standards of conduct. If you feel that a thread needs moderation, please use the contact information above, or mention @tobz or @LucioFranco in the thread.
|
||||
|
||||
1. Remarks that violate this Code of Conduct, including hateful, hurtful, oppressive, or exclusionary remarks, are not allowed. (Cursing is allowed, but never targeting another user, and never in a hateful manner.)
|
||||
2. Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
|
||||
|
||||
In the Rust community we strive to go the extra step to look out for each other. Don't just aim to be technically unimpeachable, try to be your best self. In particular, avoid flirting with offensive or sensitive issues, particularly if they're off-topic; this all too often leads to unnecessary fights, hurt feelings, and damaged trust; worse, it can drive people away from the community entirely.
|
||||
|
||||
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could've communicated better — remember that it's your responsibility to make your fellow Rustaceans comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
[package]
|
||||
name = "metrics-recorder-prometheus"
|
||||
version = "0.1.0"
|
||||
authors = ["Toby Lawrence <toby@nuclearfurnace.com>"]
|
||||
edition = "2018"
|
||||
|
||||
license = "MIT"
|
||||
|
||||
description = "metric recorder for Prometheus exposition output"
|
||||
|
||||
homepage = "https://github.com/metrics-rs/metrics"
|
||||
repository = "https://github.com/metrics-rs/metrics-recorder-prometheus"
|
||||
documentation = "https://docs.rs/metrics-recorder-prometheus"
|
||||
|
||||
[dependencies]
|
||||
metrics-core = { path = "../metrics-core", version = "^0.2" }
|
||||
metrics-util = { path = "../metrics-util", version = "^0.1" }
|
||||
hdrhistogram = "^6.1"
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
# metrics-recorder-prometheus
|
||||
|
||||
[![conduct-badge][]][conduct] [![downloads-badge][] ![release-badge][]][crate] [![docs-badge][]][docs] [![license-badge][]](#license)
|
||||
|
||||
[conduct-badge]: https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg
|
||||
[downloads-badge]: https://img.shields.io/crates/d/metrics-recorder-prometheus.svg
|
||||
[release-badge]: https://img.shields.io/crates/v/metrics-recorder-prometheus.svg
|
||||
[license-badge]: https://img.shields.io/crates/l/metrics-recorder-prometheus.svg
|
||||
[docs-badge]: https://docs.rs/metrics-recorder-prometheus/badge.svg
|
||||
[conduct]: https://github.com/metrics-rs/metrics-recorder-prometheus/blob/master/CODE_OF_CONDUCT.md
|
||||
[crate]: https://crates.io/crates/metrics-recorder-prometheus
|
||||
[docs]: https://docs.rs/metrics-recorder-prometheus
|
||||
|
||||
__metrics-recorder-prometheus__ is a metric recorder that outputs a Prometheus exposition format.
|
||||
|
||||
## code of conduct
|
||||
|
||||
**NOTE**: All conversations and contributions to this project shall adhere to the [Code of Conduct][conduct].
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
//! Records metrics in the Prometheus exposition format.
|
||||
use std::time::SystemTime;
|
||||
use hdrhistogram::Histogram;
|
||||
use metrics_core::MetricsRecorder;
|
||||
use metrics_util::{Quantile, parse_quantiles};
|
||||
|
||||
/// Records metrics in the Prometheus exposition format.
|
||||
pub struct PrometheusRecorder {
|
||||
quantiles: Vec<Quantile>,
|
||||
output: String,
|
||||
}
|
||||
|
||||
impl PrometheusRecorder {
|
||||
/// Creates a new [`PrometheusRecorder`] with a default set of quantiles.
|
||||
///
|
||||
/// Configures the recorder with these default quantiles: 0.0, 0.5, 0.9, 0.95, 0.99, 0.999, and
|
||||
/// 1.0. If you want to customize the quantiles used, you can call
|
||||
/// [`PrometheusRecorder::with_quantiles`].
|
||||
pub fn new() -> Self {
|
||||
Self::with_quantiles(&[0.0, 0.5, 0.9, 0.95, 0.99, 0.999, 1.0])
|
||||
}
|
||||
|
||||
/// Creates a new [`PrometheusRecorder`] with the given set of quantiles.
|
||||
pub fn with_quantiles(quantiles: &[f64]) -> Self {
|
||||
let actual_quantiles = parse_quantiles(quantiles);
|
||||
Self {
|
||||
quantiles: actual_quantiles,
|
||||
output: get_prom_expo_header(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl MetricsRecorder for PrometheusRecorder {
|
||||
fn record_counter<K: AsRef<str>>(&mut self, key: K, value: u64) {
|
||||
let label = key.as_ref().replace('.', "_");
|
||||
self.output.push_str("\n# TYPE ");
|
||||
self.output.push_str(label.as_str());
|
||||
self.output.push_str(" counter\n");
|
||||
self.output.push_str(label.as_str());
|
||||
self.output.push_str(" ");
|
||||
self.output.push_str(value.to_string().as_str());
|
||||
self.output.push_str("\n");
|
||||
}
|
||||
|
||||
fn record_gauge<K: AsRef<str>>(&mut self, key: K, value: i64) {
|
||||
let label = key.as_ref().replace('.', "_");
|
||||
self.output.push_str("\n# TYPE ");
|
||||
self.output.push_str(label.as_str());
|
||||
self.output.push_str(" gauge\n");
|
||||
self.output.push_str(label.as_str());
|
||||
self.output.push_str(" ");
|
||||
self.output.push_str(value.to_string().as_str());
|
||||
self.output.push_str("\n");
|
||||
}
|
||||
|
||||
fn record_histogram<K: AsRef<str>>(&mut self, key: K, values: &[u64]) {
|
||||
let mut sum = 0;
|
||||
let mut h = Histogram::<u64>::new(3).expect("failed to create histogram");
|
||||
for value in values {
|
||||
h.record(*value).expect("failed to record histogram value");
|
||||
sum += *value;
|
||||
}
|
||||
|
||||
let label = key.as_ref().replace('.', "_");
|
||||
self.output.push_str("\n# TYPE ");
|
||||
self.output.push_str(label.as_str());
|
||||
self.output.push_str(" summary\n");
|
||||
|
||||
for quantile in &self.quantiles {
|
||||
let value = h.value_at_quantile(quantile.value());
|
||||
self.output.push_str(label.as_str());
|
||||
self.output.push_str("{quantile=\"");
|
||||
self.output.push_str(quantile.value().to_string().as_str());
|
||||
self.output.push_str("\"} ");
|
||||
self.output.push_str(value.to_string().as_str());
|
||||
self.output.push_str("\n");
|
||||
}
|
||||
self.output.push_str(label.as_str());
|
||||
self.output.push_str("_sum ");
|
||||
self.output.push_str(sum.to_string().as_str());
|
||||
self.output.push_str("\n");
|
||||
self.output.push_str(label.as_str());
|
||||
self.output.push_str("_count ");
|
||||
self.output.push_str(values.len().to_string().as_str());
|
||||
self.output.push_str("\n");
|
||||
}
|
||||
}
|
||||
|
||||
impl Clone for PrometheusRecorder {
|
||||
fn clone(&self) -> Self {
|
||||
Self {
|
||||
output: get_prom_expo_header(),
|
||||
quantiles: self.quantiles.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Into<String> for PrometheusRecorder {
|
||||
fn into(self) -> String {
|
||||
self.output
|
||||
}
|
||||
}
|
||||
|
||||
fn get_prom_expo_header() -> String {
|
||||
let ts = SystemTime::now().duration_since(SystemTime::UNIX_EPOCH)
|
||||
.map(|d| d.as_secs())
|
||||
.unwrap_or(0);
|
||||
|
||||
format!("# metrics snapshot (ts={}) (prometheus exposition format)", ts)
|
||||
}
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
/target
|
||||
**/*.rs.bk
|
||||
Cargo.lock
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
# The Code of Conduct
|
||||
|
||||
This document is based on the [Rust Code of Conduct](https://www.rust-lang.org/conduct.html) and outlines the standard of conduct which is both expected and enforced as part of this project.
|
||||
|
||||
## Conduct
|
||||
|
||||
**Contact**: [toby@nuclearfurnace.com](mailto:toby@nuclearfurnace.com)
|
||||
|
||||
* We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.
|
||||
* Avoid using overtly sexual nicknames or other nicknames that might detract from a friendly, safe and welcoming environment for all.
|
||||
* Please be kind and courteous. There's no need to be mean or rude.
|
||||
* Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
|
||||
* Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
|
||||
* We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behaviour. We interpret the term "harassment" as including the definition in the <a href="http://citizencodeofconduct.org/">Citizen Code of Conduct</a>; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don't tolerate behavior that excludes people in socially marginalized groups.
|
||||
* Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the repository Owners immediately. Whether you're a regular contributor or a newcomer, we care about making this community a safe place for you and we've got your back.
|
||||
* Likewise any spamming, trolling, flaming, baiting or other attention-stealing behaviour is not welcome.
|
||||
|
||||
## Moderation
|
||||
|
||||
These are the policies for upholding our community's standards of conduct. If you feel that a thread needs moderation, please use the contact information above, or mention @tobz or @LucioFranco in the thread.
|
||||
|
||||
1. Remarks that violate this Code of Conduct, including hateful, hurtful, oppressive, or exclusionary remarks, are not allowed. (Cursing is allowed, but never targeting another user, and never in a hateful manner.)
|
||||
2. Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
|
||||
|
||||
In the Rust community we strive to go the extra step to look out for each other. Don't just aim to be technically unimpeachable, try to be your best self. In particular, avoid flirting with offensive or sensitive issues, particularly if they're off-topic; this all too often leads to unnecessary fights, hurt feelings, and damaged trust; worse, it can drive people away from the community entirely.
|
||||
|
||||
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could've communicated better — remember that it's your responsibility to make your fellow Rustaceans comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
[package]
|
||||
name = "metrics-recorder-text"
|
||||
version = "0.1.0"
|
||||
authors = ["Toby Lawrence <toby@nuclearfurnace.com>"]
|
||||
edition = "2018"
|
||||
|
||||
license = "MIT"
|
||||
|
||||
description = "metric recorder for hierarchical, text-based output"
|
||||
|
||||
homepage = "https://github.com/metrics-rs/metrics"
|
||||
repository = "https://github.com/metrics-rs/metrics-recorder-text"
|
||||
documentation = "https://docs.rs/metrics-recorder-text"
|
||||
|
||||
[dependencies]
|
||||
metrics-core = { path = "../metrics-core", version = "^0.2" }
|
||||
metrics-util = { path = "../metrics-util", version = "^0.1" }
|
||||
hdrhistogram = "^6.1"
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
# metrics-recorder-text
|
||||
|
||||
[![conduct-badge][]][conduct] [![downloads-badge][] ![release-badge][]][crate] [![docs-badge][]][docs] [![license-badge][]](#license)
|
||||
|
||||
[conduct-badge]: https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg
|
||||
[downloads-badge]: https://img.shields.io/crates/d/metrics-recorder-text.svg
|
||||
[release-badge]: https://img.shields.io/crates/v/metrics-recorder-text.svg
|
||||
[license-badge]: https://img.shields.io/crates/l/metrics-recorder-text.svg
|
||||
[docs-badge]: https://docs.rs/metrics-recorder-text/badge.svg
|
||||
[conduct]: https://github.com/metrics-rs/metrics-recorder-text/blob/master/CODE_OF_CONDUCT.md
|
||||
[crate]: https://crates.io/crates/metrics-recorder-text
|
||||
[docs]: https://docs.rs/metrics-recorder-text
|
||||
|
||||
__metrics-recorder-text__ is a metric recorder that outputs a hierarchical, text-based format.
|
||||
|
||||
## code of conduct
|
||||
|
||||
**NOTE**: All conversations and contributions to this project shall adhere to the [Code of Conduct][conduct].
|
||||
|
|
@ -0,0 +1,261 @@
|
|||
//! Records metrics in a hierarchical, text-based format.
|
||||
//!
|
||||
//! Metric scopes are used to provide the hierarchy and indentation of metrics. As an example, for
|
||||
//! a snapshot with two metrics — `server.msgs_received` and `server.msgs_sent` — we would
|
||||
//! expect to see this output:
|
||||
//!
|
||||
//! ```c
|
||||
//! root:
|
||||
//! server:
|
||||
//! msgs_received: 42
|
||||
//! msgs_sent: 13
|
||||
//! ```
|
||||
//!
|
||||
//! If we added another metric — `configuration_reloads` — we would expect to see:
|
||||
//!
|
||||
//! ```c
|
||||
//! root:
|
||||
//! configuration_reloads: 2
|
||||
//! server:
|
||||
//! msgs_received: 42
|
||||
//! msgs_sent: 13
|
||||
//! ```
|
||||
//!
|
||||
//! Metrics are sorted alphabetically.
|
||||
//!
|
||||
//! ## Histograms
|
||||
//!
|
||||
//! Histograms are rendered with a configurable set of quantiles that are provided when creating an
|
||||
//! instance of `TextRecorder`. They are formatted using human-readable labels when displayed to
|
||||
//! the user. For example, 0.0 is rendered as "min", 1.0 as "max", and anything in between using
|
||||
//! the common "pXXX" format i.e. a quantile of 0.5 or percentile of 50 would be p50, a quantile of
|
||||
//! 0.999 or percentile of 99.9 would be p999, and so on.
|
||||
//!
|
||||
//! All histograms have the sample count of the histogram provided in the output.
|
||||
//!
|
||||
//! ```c
|
||||
//! root:
|
||||
//! connect_time count: 15
|
||||
//! connect_time min: 1334
|
||||
//! connect_time p50: 1934
|
||||
//! connect_time p99: 5330
|
||||
//! connect_time max: 139389
|
||||
//! ```
|
||||
//!
|
||||
use std::collections::{HashMap, VecDeque};
|
||||
use std::fmt::Display;
|
||||
use hdrhistogram::Histogram;
|
||||
use metrics_core::MetricsRecorder;
|
||||
use metrics_util::{Quantile, parse_quantiles};
|
||||
|
||||
/// Records metrics in a hierarchical, text-based format.
|
||||
pub struct TextRecorder {
|
||||
structure: MetricsTree,
|
||||
quantiles: Vec<Quantile>,
|
||||
}
|
||||
|
||||
impl TextRecorder {
|
||||
/// Creates a new [`TextRecorder`] with a default set of quantiles.
|
||||
///
|
||||
/// Configures the recorder with these default quantiles: 0.0, 0.5, 0.9, 0.95, 0.99, 0.999, and
|
||||
/// 1.0. If you want to customize the quantiles used, you can call
|
||||
/// [`TextRecorder::with_quantiles`].
|
||||
pub fn new() -> Self {
|
||||
Self::with_quantiles(&[0.0, 0.5, 0.9, 0.95, 0.99, 0.999, 1.0])
|
||||
}
|
||||
|
||||
/// Creates a new [`TextRecorder`] with the given set of quantiles.
|
||||
pub fn with_quantiles(quantiles: &[f64]) -> Self {
|
||||
let actual_quantiles = parse_quantiles(quantiles);
|
||||
|
||||
Self {
|
||||
structure: MetricsTree::with_level(0),
|
||||
quantiles: actual_quantiles,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl MetricsRecorder for TextRecorder {
|
||||
fn record_counter<K: AsRef<str>>(&mut self, key: K, value: u64) {
|
||||
let (name_parts, name) = name_to_parts(key.as_ref());
|
||||
let mut values = single_value_to_values(name, value);
|
||||
self.structure.insert(name_parts, &mut values);
|
||||
}
|
||||
|
||||
fn record_gauge<K: AsRef<str>>(&mut self, key: K, value: i64) {
|
||||
let (name_parts, name) = name_to_parts(key.as_ref());
|
||||
let mut values = single_value_to_values(name, value);
|
||||
self.structure.insert(name_parts, &mut values);
|
||||
}
|
||||
|
||||
fn record_histogram<K: AsRef<str>>(&mut self, key: K, values: &[u64]) {
|
||||
let mut h = Histogram::new(3).expect("failed to create histogram");
|
||||
for value in values {
|
||||
h.record(*value).expect("failed to record histogram value");
|
||||
}
|
||||
|
||||
let (name_parts, name) = name_to_parts(key.as_ref());
|
||||
let mut values = hist_to_values(name, h, &self.quantiles);
|
||||
self.structure.insert(name_parts, &mut values);
|
||||
}
|
||||
}
|
||||
|
||||
impl Clone for TextRecorder {
|
||||
fn clone(&self) -> Self {
|
||||
Self {
|
||||
structure: MetricsTree::with_level(0),
|
||||
quantiles: self.quantiles.clone(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct MetricsTree {
|
||||
level: usize,
|
||||
current: Vec<String>,
|
||||
next: HashMap<String, MetricsTree>,
|
||||
}
|
||||
|
||||
impl MetricsTree {
|
||||
pub fn with_level(level: usize) -> Self {
|
||||
MetricsTree {
|
||||
level,
|
||||
current: Vec::new(),
|
||||
next: HashMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn insert(&mut self, mut name_parts: VecDeque<String>, values: &mut Vec<String>) {
|
||||
match name_parts.len() {
|
||||
0 => {
|
||||
let indent = " ".repeat(self.level + 1);
|
||||
let mut indented = values
|
||||
.iter()
|
||||
.map(move |x| format!("{}{}", indent, x))
|
||||
.collect::<Vec<_>>();
|
||||
self.current.append(&mut indented);
|
||||
}
|
||||
_ => {
|
||||
let name = name_parts
|
||||
.pop_front()
|
||||
.expect("failed to get next name component");
|
||||
let current_level = self.level;
|
||||
let inner = self
|
||||
.next
|
||||
.entry(name)
|
||||
.or_insert_with(move || MetricsTree::with_level(current_level + 1));
|
||||
inner.insert(name_parts, values);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn into_output(self) -> String {
|
||||
let indent = " ".repeat(self.level + 1);
|
||||
let mut output = String::new();
|
||||
if self.level == 0 {
|
||||
output.push_str("\nroot:\n");
|
||||
}
|
||||
|
||||
let mut sorted = self
|
||||
.current
|
||||
.into_iter()
|
||||
.map(SortEntry::Inline)
|
||||
.chain(self.next.into_iter().map(|(k, v)| SortEntry::Nested(k, v)))
|
||||
.collect::<Vec<_>>();
|
||||
sorted.sort();
|
||||
|
||||
for entry in sorted {
|
||||
match entry {
|
||||
SortEntry::Inline(s) => {
|
||||
output.push_str(s.as_str());
|
||||
output.push_str("\n");
|
||||
}
|
||||
SortEntry::Nested(s, inner) => {
|
||||
output.push_str(indent.as_str());
|
||||
output.push_str(s.as_str());
|
||||
output.push_str(":\n");
|
||||
|
||||
let layer_output = inner.into_output();
|
||||
output.push_str(layer_output.as_str());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output
|
||||
}
|
||||
}
|
||||
|
||||
impl Into<String> for TextRecorder {
|
||||
fn into(self) -> String {
|
||||
self.structure.into_output()
|
||||
}
|
||||
}
|
||||
|
||||
enum SortEntry {
|
||||
Inline(String),
|
||||
Nested(String, MetricsTree),
|
||||
}
|
||||
|
||||
impl SortEntry {
|
||||
fn name(&self) -> &String {
|
||||
match self {
|
||||
SortEntry::Inline(s) => s,
|
||||
SortEntry::Nested(s, _) => s,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl PartialEq for SortEntry {
|
||||
fn eq(&self, other: &SortEntry) -> bool {
|
||||
self.name() == other.name()
|
||||
}
|
||||
}
|
||||
|
||||
impl Eq for SortEntry {}
|
||||
|
||||
impl std::cmp::PartialOrd for SortEntry {
|
||||
fn partial_cmp(&self, other: &SortEntry) -> Option<std::cmp::Ordering> {
|
||||
Some(self.cmp(other))
|
||||
}
|
||||
}
|
||||
|
||||
impl std::cmp::Ord for SortEntry {
|
||||
fn cmp(&self, other: &SortEntry) -> std::cmp::Ordering {
|
||||
self.name().cmp(other.name())
|
||||
}
|
||||
}
|
||||
|
||||
fn name_to_parts(name: &str) -> (VecDeque<String>, String) {
|
||||
let mut parts = name
|
||||
.split('.')
|
||||
.map(ToOwned::to_owned)
|
||||
.collect::<VecDeque<_>>();
|
||||
let name = parts.pop_back().expect("name didn't have a single part");
|
||||
|
||||
(parts, name)
|
||||
}
|
||||
|
||||
fn single_value_to_values<T>(name: String, value: T) -> Vec<String>
|
||||
where
|
||||
T: Display,
|
||||
{
|
||||
let fvalue = format!("{}: {}", name, value);
|
||||
vec![fvalue]
|
||||
}
|
||||
|
||||
fn hist_to_values(name: String, hist: Histogram<u64>, quantiles: &[Quantile]) -> Vec<String> {
|
||||
let mut values = Vec::new();
|
||||
|
||||
values.push(format!("{} count: {}", name, hist.len()));
|
||||
for quantile in quantiles {
|
||||
let value = hist.value_at_quantile(quantile.value());
|
||||
values.push(format!(
|
||||
"{} {}: {}",
|
||||
name,
|
||||
quantile.label(),
|
||||
value,
|
||||
));
|
||||
}
|
||||
|
||||
values
|
||||
}
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
/target
|
||||
**/*.rs.bk
|
||||
Cargo.lock
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# The Code of Conduct
|
||||
|
||||
This document is based on the [Rust Code of Conduct](https://www.rust-lang.org/conduct.html) and outlines the standard of conduct which is both expected and enforced as part of this project.
|
||||
|
||||
## Conduct
|
||||
|
||||
**Contact**:
|
||||
[toby@nuclearfurnace.com](mailto:toby@nuclearfurnace.com)
|
||||
[luciofranco14@gmail.com](mailto:luciofranco14@gmail.com)
|
||||
|
||||
* We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.
|
||||
* Avoid using overtly sexual nicknames or other nicknames that might detract from a friendly, safe and welcoming environment for all.
|
||||
* Please be kind and courteous. There's no need to be mean or rude.
|
||||
* Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
|
||||
* Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
|
||||
* We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behaviour. We interpret the term "harassment" as including the definition in the <a href="http://citizencodeofconduct.org/">Citizen Code of Conduct</a>; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don't tolerate behavior that excludes people in socially marginalized groups.
|
||||
* Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the repository Owners immediately. Whether you're a regular contributor or a newcomer, we care about making this community a safe place for you and we've got your back.
|
||||
* Likewise any spamming, trolling, flaming, baiting or other attention-stealing behaviour is not welcome.
|
||||
|
||||
## Moderation
|
||||
|
||||
These are the policies for upholding our community's standards of conduct. If you feel that a thread needs moderation, please use the contact information above, or mention @tobz or @LucioFranco in the thread.
|
||||
|
||||
1. Remarks that violate this Code of Conduct, including hateful, hurtful, oppressive, or exclusionary remarks, are not allowed. (Cursing is allowed, but never targeting another user, and never in a hateful manner.)
|
||||
2. Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
|
||||
|
||||
In the Rust community we strive to go the extra step to look out for each other. Don't just aim to be technically unimpeachable, try to be your best self. In particular, avoid flirting with offensive or sensitive issues, particularly if they're off-topic; this all too often leads to unnecessary fights, hurt feelings, and damaged trust; worse, it can drive people away from the community entirely.
|
||||
|
||||
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could've communicated better — remember that it's your responsibility to make your fellow Rustaceans comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
[package]
|
||||
name = "metrics-util"
|
||||
version = "0.1.0"
|
||||
authors = ["Toby Lawrence <toby@nuclearfurnace.com>"]
|
||||
edition = "2018"
|
||||
|
||||
license = "MIT"
|
||||
|
||||
description = "helper types/functions used by the metrics ecosystem"
|
||||
|
||||
homepage = "https://github.com/metrics-rs/metrics"
|
||||
repository = "https://github.com/metrics-rs/metrics-util"
|
||||
documentation = "https://docs.rs/metrics-util"
|
||||
|
||||
readme = "README.md"
|
||||
|
||||
keywords = ["metrics", "quantile", "percentile"]
|
||||
|
||||
[dependencies]
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
# metrics-util
|
||||
|
||||
[![conduct-badge][]][conduct] [![downloads-badge][] ![release-badge][]][crate] [![docs-badge][]][docs] [![license-badge][]](#license)
|
||||
|
||||
[conduct-badge]: https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg
|
||||
[downloads-badge]: https://img.shields.io/crates/d/metrics-util.svg
|
||||
[release-badge]: https://img.shields.io/crates/v/metrics-util.svg
|
||||
[license-badge]: https://img.shields.io/crates/l/metrics-util.svg
|
||||
[docs-badge]: https://docs.rs/metrics-util/badge.svg
|
||||
[conduct]: https://github.com/metrics-rs/metrics-util/blob/master/CODE_OF_CONDUCT.md
|
||||
[crate]: https://crates.io/crates/metrics-util
|
||||
[docs]: https://docs.rs/metrics-util
|
||||
|
||||
__metrics-util__ is a helper library with types/functions used within the metrics ecosystem.
|
||||
|
||||
## code of conduct
|
||||
|
||||
**NOTE**: All conversations and contributions to this project shall adhere to the [Code of Conduct][conduct].
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
//! Helper types and functions used within the metrics ecosystem.
|
||||
|
||||
/// A quantile that has both the raw value and a human-friendly display label.
|
||||
#[derive(Clone)]
|
||||
pub struct Quantile(f64, String);
|
||||
|
||||
impl Quantile {
|
||||
/// Creates a new `Quantile` from a floating-point value.
|
||||
///
|
||||
/// All values clamped between 0.0 and 1.0.
|
||||
pub fn new(quantile: f64) -> Quantile {
|
||||
let clamped = quantile.max(0.0);
|
||||
let clamped = clamped.min(1.0);
|
||||
let display = clamped * 100.0;
|
||||
|
||||
let raw_label = format!("{}", clamped);
|
||||
let label = match raw_label.as_str() {
|
||||
"0" => "min".to_string(),
|
||||
"1" => "max".to_string(),
|
||||
_ => {
|
||||
let raw = format!("p{}", display);
|
||||
raw.replace(".", "")
|
||||
},
|
||||
};
|
||||
|
||||
Quantile(clamped, label)
|
||||
}
|
||||
|
||||
/// Gets the human-friendly display label for this quantile.
|
||||
pub fn label(&self) -> &str {
|
||||
self.1.as_str()
|
||||
}
|
||||
|
||||
/// Gets the raw value for this quantile.
|
||||
pub fn value(&self) -> f64 {
|
||||
self.0
|
||||
}
|
||||
}
|
||||
|
||||
/// Parses a list of floating-point values into a list of `Quantile`s.
|
||||
pub fn parse_quantiles(quantiles: &[f64]) -> Vec<Quantile> {
|
||||
quantiles.iter()
|
||||
.map(|f| Quantile::new(*f))
|
||||
.collect()
|
||||
}
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
/target
|
||||
**/*.rs.bk
|
||||
Cargo.lock
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
# Changelog
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
## [0.8.2] - 2019-03-19
|
||||
### Added
|
||||
- Histograms now track the sum of all values they record, to support target systems like Prometheus.
|
||||
- Added the ability to get percentiles as quantiles. This is also to support target systems like Prometheus. These are derived from the existing percentile values and so can have extra decimal precision. This will be unified in a future breaking update.
|
||||
|
||||
## [0.8.1] - 2019-03-15
|
||||
### Changed
|
||||
- Fixed some issues with type visibility and documentation.
|
||||
|
||||
## [0.8.0] - 2019-03-15
|
||||
### Changed
|
||||
- Removed accessors from `Snapshot`. It is not an opaque type that can be turned into an iterator which will provide access to typed metric values so that an external consumer can get all of the values in the snapshot, including their type, for proper exporting.
|
||||
### Added
|
||||
- A new "simple" snapshot type -- `SimpleSnapshot` -- which has easy-to-use accessors for metrics, identical to what `Snapshot` used to have.
|
||||
- Allow retrieving snapshots asynchronously via `Controller::get_snapshot_async`. Utilizes a oneshot channel so the caller can poll asynchronously.
|
||||
|
||||
## [0.7.1] - 2019-01-28
|
||||
### Changed
|
||||
- Fixed a bug where new sinks with the same scope would overwrite each others metrics. [#20](https://github.com/nuclearfurnace/hotmic/pull/20)
|
||||
|
||||
## [0.7.0] - 2019-01-27
|
||||
### Changed
|
||||
- Sink scopes can now be either a `&str` or `&[&str]`.
|
||||
- Fixed a bug where the receiver loop ran its thread at 100%.
|
||||
|
||||
## [0.6.0] - 2019-01-24
|
||||
### Changed
|
||||
- Metrics auto-register themselves now. [#16](https://github.com/nuclearfurnace/hotmic/pull/16)
|
||||
|
||||
## [0.5.2] - 2019-01-19
|
||||
### Changed
|
||||
- Snapshot now implements [`Serialize`](https://docs.rs/serde/1.0.85/serde/trait.Serialize.html).
|
||||
|
||||
## [0.5.1] - 2019-01-19
|
||||
### Changed
|
||||
- Controller is now `Clone`.
|
||||
|
||||
## [0.5.0] - 2019-01-19
|
||||
### Added
|
||||
- Revamp API to provide easier usage. [#14](https://github.com/nuclearfurnace/hotmic/pull/14)
|
||||
|
||||
## [0.4.0] - 2019-01-14
|
||||
Minimum supported Rust version is now 1.31.0, courtesy of switching to the 2018 edition.
|
||||
|
||||
### Changed
|
||||
- Switch to integer-backed metric scopes. [#10](https://github.com/nuclearfurnace/hotmic/pull/10)
|
||||
### Added
|
||||
- Add clock support via `quanta`. [#12](https://github.com/nuclearfurnace/hotmic/pull/12)
|
||||
|
||||
## [0.3.0] - 2018-12-22
|
||||
### Added
|
||||
- Switch to crossbeam-channel and add scopes. [#4](https://github.com/nuclearfurnace/hotmic/pull/4)
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
# The Code of Conduct
|
||||
|
||||
This document is based on the [Rust Code of Conduct](https://www.rust-lang.org/conduct.html) and outlines the standard of conduct which is both expected and enforced as part of this project.
|
||||
|
||||
## Conduct
|
||||
|
||||
**Contact**:
|
||||
[toby@nuclearfurnace.com](mailto:toby@nuclearfurnace.com)
|
||||
[luciofranco14@gmail.com](mailto:luciofranco14@gmail.com)
|
||||
|
||||
* We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.
|
||||
* Avoid using overtly sexual nicknames or other nicknames that might detract from a friendly, safe and welcoming environment for all.
|
||||
* Please be kind and courteous. There's no need to be mean or rude.
|
||||
* Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
|
||||
* Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
|
||||
* We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behaviour. We interpret the term "harassment" as including the definition in the <a href="http://citizencodeofconduct.org/">Citizen Code of Conduct</a>; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don't tolerate behavior that excludes people in socially marginalized groups.
|
||||
* Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the repository Owners immediately. Whether you're a regular contributor or a newcomer, we care about making this community a safe place for you and we've got your back.
|
||||
* Likewise any spamming, trolling, flaming, baiting or other attention-stealing behaviour is not welcome.
|
||||
|
||||
## Moderation
|
||||
|
||||
These are the policies for upholding our community's standards of conduct. If you feel that a thread needs moderation, please use the contact information above, or mention @tobz or @LucioFranco in the thread.
|
||||
|
||||
1. Remarks that violate this Code of Conduct, including hateful, hurtful, oppressive, or exclusionary remarks, are not allowed. (Cursing is allowed, but never targeting another user, and never in a hateful manner.)
|
||||
2. Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
|
||||
|
||||
In the Rust community we strive to go the extra step to look out for each other. Don't just aim to be technically unimpeachable, try to be your best self. In particular, avoid flirting with offensive or sensitive issues, particularly if they're off-topic; this all too often leads to unnecessary fights, hurt feelings, and damaged trust; worse, it can drive people away from the community entirely.
|
||||
|
||||
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could've communicated better — remember that it's your responsibility to make your fellow Rustaceans comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
[package]
|
||||
name = "metrics"
|
||||
version = "0.9.0"
|
||||
authors = ["Toby Lawrence <toby@nuclearfurnace.com>"]
|
||||
edition = "2018"
|
||||
|
||||
license = "MIT"
|
||||
|
||||
description = "high-speed metrics collection library"
|
||||
|
||||
homepage = "https://github.com/metrics-rs/metrics"
|
||||
repository = "https://github.com/metrics-rs/metrics"
|
||||
documentation = "https://docs.rs/metrics"
|
||||
|
||||
readme = "README.md"
|
||||
|
||||
keywords = ["metrics", "telemetry", "histogram", "counter", "gauge"]
|
||||
|
||||
[profile.release]
|
||||
debug = true
|
||||
opt-level = 3
|
||||
lto = true
|
||||
|
||||
[dependencies]
|
||||
metrics-core = { path = "../metrics-core", version = "^0.2" }
|
||||
crossbeam-channel = "^0.3"
|
||||
parking_lot = "^0.7"
|
||||
fnv = "^1.0"
|
||||
hashbrown = "^0.1"
|
||||
quanta = "^0.2"
|
||||
tokio-sync = "^0.1"
|
||||
|
||||
[dev-dependencies]
|
||||
log = "^0.4"
|
||||
env_logger = "^0.6"
|
||||
getopts = "^0.2"
|
||||
hdrhistogram = "^6.1"
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
// Large portions of this library were inspired by or rewritten portions of "tic".
|
||||
// Copyright (c) 2016 Brian Martin
|
||||
//
|
||||
// Copyright (c) 2018 Nuclear Furnace
|
||||
//
|
||||
// Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
// of this software and associated documentation files (the "Software"), to deal
|
||||
// in the Software without restriction, including without limitation the rights
|
||||
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
// copies of the Software, and to permit persons to whom the Software is
|
||||
// furnished to do so, subject to the following conditions:
|
||||
//
|
||||
// The above copyright notice and this permission notice shall be included in all
|
||||
// copies or substantial portions of the Software.
|
||||
//
|
||||
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
// SOFTWARE.
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
# metrics
|
||||
|
||||
[![conduct-badge][]][conduct] [![downloads-badge][] ![release-badge][]][crate] [![docs-badge][]][docs] [![license-badge][]](#license)
|
||||
|
||||
[conduct-badge]: https://img.shields.io/badge/%E2%9D%A4-code%20of%20conduct-blue.svg
|
||||
[downloads-badge]: https://img.shields.io/crates/d/metrics.svg
|
||||
[release-badge]: https://img.shields.io/crates/v/metrics.svg
|
||||
[license-badge]: https://img.shields.io/crates/l/metrics.svg
|
||||
[docs-badge]: https://docs.rs/metrics/badge.svg
|
||||
[conduct]: https://github.com/metrics-rs/metrics/blob/master/CODE_OF_CONDUCT.md
|
||||
[crate]: https://crates.io/crates/metrics
|
||||
[docs]: https://docs.rs/metrics
|
||||
|
||||
__metrics__ is a high-quality, batteries-included metrics library for Rust.
|
||||
|
||||
## code of conduct
|
||||
|
||||
**NOTE**: All conversations and contributions to this project shall adhere to the [Code of Conduct][conduct].
|
||||
|
||||
# caveat emptor
|
||||
|
||||
This crate is currently materializing! We are in the process of switching over [hotmic](https://github.com/nuclearfurnace/hotmic) to `metrics` after successfully acquiring ownership of the `metrics` crate on crates.io!
|
||||
|
||||
We apologize for the README/documentation that will reference things that don't exist yet until the switchover is complete. Thank you for your understanding!
|
||||
|
||||
## general features
|
||||
- Provides counter, gauge, and histogram support.
|
||||
- Access to ultra-high-speed timing facilities out-of-the-box with [quanta](https://github.com/nuclearfurnace/quanta).
|
||||
- Scoped metrics for effortless nesting.
|
||||
- Speed and API ergonomics allow for usage in both synchronous and asynchronous contexts.
|
||||
- Based on `metrics-core` for bring-your-own-collector/bring-your-own-exporter flexibility!
|
||||
|
||||
## performance
|
||||
|
||||
High. Tens of millions of metrics per second with metric ingest times at sub-200ns p99 on modern systems.
|
||||
|
|
@ -0,0 +1,256 @@
|
|||
#[macro_use]
|
||||
extern crate log;
|
||||
extern crate env_logger;
|
||||
extern crate getopts;
|
||||
extern crate hdrhistogram;
|
||||
extern crate metrics;
|
||||
extern crate metrics_core;
|
||||
|
||||
use getopts::Options;
|
||||
use hdrhistogram::Histogram;
|
||||
use metrics::{snapshot::TypedMeasurement, Receiver, Sink};
|
||||
use std::{
|
||||
env,
|
||||
sync::{
|
||||
atomic::{AtomicBool, Ordering},
|
||||
Arc,
|
||||
},
|
||||
thread,
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
|
||||
struct Generator {
|
||||
stats: Sink,
|
||||
t0: Option<u64>,
|
||||
gauge: i64,
|
||||
hist: Histogram<u64>,
|
||||
done: Arc<AtomicBool>,
|
||||
}
|
||||
|
||||
impl Generator {
|
||||
fn new(stats: Sink, done: Arc<AtomicBool>) -> Generator {
|
||||
Generator {
|
||||
stats,
|
||||
t0: None,
|
||||
gauge: 0,
|
||||
hist: Histogram::<u64>::new_with_bounds(1, u64::max_value(), 3).unwrap(),
|
||||
done,
|
||||
}
|
||||
}
|
||||
|
||||
fn run(&mut self) {
|
||||
loop {
|
||||
if self.done.load(Ordering::Relaxed) {
|
||||
break;
|
||||
}
|
||||
|
||||
self.gauge += 1;
|
||||
let t1 = self.stats.clock().now();
|
||||
|
||||
if let Some(t0) = self.t0 {
|
||||
let start = self.stats.clock().now();
|
||||
let _ = self.stats.record_timing("ok", t0, t1);
|
||||
let _ = self.stats.record_gauge("total", self.gauge);
|
||||
let delta = self.stats.clock().now() - start;
|
||||
self.hist.saturating_record(delta);
|
||||
}
|
||||
self.t0 = Some(t1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Generator {
|
||||
fn drop(&mut self) {
|
||||
info!(
|
||||
" sender latency: min: {:9} p50: {:9} p95: {:9} p99: {:9} p999: {:9} max: {:9}",
|
||||
nanos_to_readable(self.hist.min()),
|
||||
nanos_to_readable(self.hist.value_at_percentile(50.0)),
|
||||
nanos_to_readable(self.hist.value_at_percentile(95.0)),
|
||||
nanos_to_readable(self.hist.value_at_percentile(99.0)),
|
||||
nanos_to_readable(self.hist.value_at_percentile(99.9)),
|
||||
nanos_to_readable(self.hist.max())
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn print_usage(program: &str, opts: &Options) {
|
||||
let brief = format!("Usage: {} [options]", program);
|
||||
print!("{}", opts.usage(&brief));
|
||||
}
|
||||
|
||||
pub fn opts() -> Options {
|
||||
let mut opts = Options::new();
|
||||
|
||||
opts.optopt(
|
||||
"d",
|
||||
"duration",
|
||||
"number of seconds to run the benchmark",
|
||||
"INTEGER",
|
||||
);
|
||||
opts.optopt("p", "producers", "number of producers", "INTEGER");
|
||||
opts.optopt(
|
||||
"c",
|
||||
"capacity",
|
||||
"maximum number of unprocessed items",
|
||||
"INTEGER",
|
||||
);
|
||||
opts.optopt(
|
||||
"b",
|
||||
"batch-size",
|
||||
"maximum number of items in a batch",
|
||||
"INTEGER",
|
||||
);
|
||||
opts.optflag("h", "help", "print this help menu");
|
||||
|
||||
opts
|
||||
}
|
||||
|
||||
fn main() {
|
||||
env_logger::init();
|
||||
|
||||
let args: Vec<String> = env::args().collect();
|
||||
let program = &args[0];
|
||||
let opts = opts();
|
||||
|
||||
let matches = match opts.parse(&args[1..]) {
|
||||
Ok(m) => m,
|
||||
Err(f) => {
|
||||
error!("Failed to parse command line args: {}", f);
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
if matches.opt_present("help") {
|
||||
print_usage(program, &opts);
|
||||
return;
|
||||
}
|
||||
|
||||
info!("metrics benchmark");
|
||||
|
||||
// Build our sink and configure the facets.
|
||||
let seconds = matches
|
||||
.opt_str("duration")
|
||||
.unwrap_or_else(|| "60".to_owned())
|
||||
.parse()
|
||||
.unwrap();
|
||||
let capacity = matches
|
||||
.opt_str("capacity")
|
||||
.unwrap_or_else(|| "1024".to_owned())
|
||||
.parse()
|
||||
.unwrap();
|
||||
let batch_size = matches
|
||||
.opt_str("batch-size")
|
||||
.unwrap_or_else(|| "256".to_owned())
|
||||
.parse()
|
||||
.unwrap();
|
||||
let producers = matches
|
||||
.opt_str("producers")
|
||||
.unwrap_or_else(|| "1".to_owned())
|
||||
.parse()
|
||||
.unwrap();
|
||||
|
||||
info!("producers: {}", producers);
|
||||
info!("capacity: {}", capacity);
|
||||
info!("batch size: {}", batch_size);
|
||||
|
||||
let mut receiver = Receiver::builder()
|
||||
.capacity(capacity)
|
||||
.batch_size(batch_size)
|
||||
.histogram(Duration::from_secs(5), Duration::from_secs(1))
|
||||
.build();
|
||||
|
||||
let sink = receiver.get_sink();
|
||||
let sink = sink.scoped(&["alpha", "pools", "primary"]);
|
||||
|
||||
info!("sink configured");
|
||||
|
||||
// Spin up our sample producers.
|
||||
let done = Arc::new(AtomicBool::new(false));
|
||||
let mut handles = Vec::new();
|
||||
|
||||
for _ in 0..producers {
|
||||
let s = sink.clone();
|
||||
let d = done.clone();
|
||||
let handle = thread::spawn(move || {
|
||||
Generator::new(s, d).run();
|
||||
});
|
||||
|
||||
handles.push(handle);
|
||||
}
|
||||
|
||||
// Spin up the sink and let 'er rip.
|
||||
let controller = receiver.get_controller();
|
||||
|
||||
thread::spawn(move || {
|
||||
receiver.run();
|
||||
});
|
||||
|
||||
// Poll the controller to figure out the sample rate.
|
||||
let mut total = 0;
|
||||
let mut t0 = Instant::now();
|
||||
|
||||
let mut snapshot_hist = Histogram::<u64>::new_with_bounds(1, u64::max_value(), 3).unwrap();
|
||||
for _ in 0..seconds {
|
||||
let t1 = Instant::now();
|
||||
|
||||
let start = Instant::now();
|
||||
let snapshot = controller.get_snapshot();
|
||||
let end = Instant::now();
|
||||
snapshot_hist.saturating_record(duration_as_nanos(end - start) as u64);
|
||||
|
||||
let turn_total = snapshot
|
||||
.unwrap()
|
||||
.into_measurements()
|
||||
.iter()
|
||||
.fold(0, |acc, m| {
|
||||
acc + match m {
|
||||
TypedMeasurement::Counter(_key, value) => *value,
|
||||
TypedMeasurement::Gauge(_key, value) => *value as u64,
|
||||
_ => 0,
|
||||
}
|
||||
});
|
||||
|
||||
let turn_delta = turn_total - total;
|
||||
total = turn_total;
|
||||
let rate = turn_delta as f64 / (duration_as_nanos(t1 - t0) / 1_000_000_000.0);
|
||||
|
||||
info!("sample ingest rate: {:.0} samples/sec", rate);
|
||||
t0 = t1;
|
||||
thread::sleep(Duration::new(1, 0));
|
||||
}
|
||||
|
||||
info!("--------------------------------------------------------------------------------");
|
||||
info!(" ingested samples total: {}", total);
|
||||
info!(
|
||||
"snapshot retrieval: min: {:9} p50: {:9} p95: {:9} p99: {:9} p999: {:9} max: {:9}",
|
||||
nanos_to_readable(snapshot_hist.min()),
|
||||
nanos_to_readable(snapshot_hist.value_at_percentile(50.0)),
|
||||
nanos_to_readable(snapshot_hist.value_at_percentile(95.0)),
|
||||
nanos_to_readable(snapshot_hist.value_at_percentile(99.0)),
|
||||
nanos_to_readable(snapshot_hist.value_at_percentile(99.9)),
|
||||
nanos_to_readable(snapshot_hist.max())
|
||||
);
|
||||
|
||||
// Wait for the producers to finish so we can get their stats too.
|
||||
done.store(true, Ordering::SeqCst);
|
||||
for handle in handles {
|
||||
let _ = handle.join();
|
||||
}
|
||||
}
|
||||
|
||||
fn duration_as_nanos(d: Duration) -> f64 {
|
||||
(d.as_secs() as f64 * 1e9) + d.subsec_nanos() as f64
|
||||
}
|
||||
|
||||
fn nanos_to_readable(t: u64) -> String {
|
||||
let f = t as f64;
|
||||
if f < 1_000.0 {
|
||||
format!("{}ns", f)
|
||||
} else if f < 1_000_000.0 {
|
||||
format!("{:.0}μs", f / 1_000.0)
|
||||
} else if f < 2_000_000_000.0 {
|
||||
format!("{:.2}ms", f / 1_000_000.0)
|
||||
} else {
|
||||
format!("{:.3}s", f / 1_000_000_000.0)
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
use crate::receiver::Receiver;
|
||||
use std::time::Duration;
|
||||
|
||||
/// A configuration builder for [`Receiver`].
|
||||
#[derive(Clone)]
|
||||
pub struct Configuration {
|
||||
pub(crate) capacity: usize,
|
||||
pub(crate) batch_size: usize,
|
||||
pub(crate) histogram_window: Duration,
|
||||
pub(crate) histogram_granularity: Duration,
|
||||
}
|
||||
|
||||
impl Default for Configuration {
|
||||
fn default() -> Configuration {
|
||||
Configuration {
|
||||
capacity: 512,
|
||||
batch_size: 64,
|
||||
histogram_window: Duration::from_secs(10),
|
||||
histogram_granularity: Duration::from_secs(1),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Configuration {
|
||||
/// Creates a new [`Configuration`] with default values.
|
||||
pub fn new() -> Configuration {
|
||||
Default::default()
|
||||
}
|
||||
|
||||
/// Sets the buffer capacity.
|
||||
///
|
||||
/// Defaults to 512.
|
||||
///
|
||||
/// This controls the size of the channel used to send metrics. This channel is shared amongst
|
||||
/// all active sinks. If this channel is full when sending a metric, that send will be blocked
|
||||
/// until the channel has free space.
|
||||
///
|
||||
/// Tweaking this value allows for a trade-off between low memory consumption and throughput
|
||||
/// burst capabilities. By default, we expect samples to occupy approximately 64 bytes. Thus,
|
||||
/// at our default value, we preallocate roughly ~32KB.
|
||||
///
|
||||
/// Generally speaking, sending and processing metrics is fast enough that the default value of
|
||||
/// 512 supports millions of samples per second.
|
||||
pub fn capacity(mut self, capacity: usize) -> Self {
|
||||
self.capacity = capacity;
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the batch size.
|
||||
///
|
||||
/// Defaults to 64.
|
||||
///
|
||||
/// This controls the size of message batches that we collect for processing. The only real
|
||||
/// reason to tweak this is to control the latency from the sender side. Larger batches lower
|
||||
/// the ingest latency in the face of high metric ingest pressure at the cost of higher ingest
|
||||
/// tail latencies.
|
||||
///
|
||||
/// Long story short, you shouldn't need to change this, but it's here if you really do.
|
||||
pub fn batch_size(mut self, batch_size: usize) -> Self {
|
||||
self.batch_size = batch_size;
|
||||
self
|
||||
}
|
||||
|
||||
/// Sets the histogram configuration.
|
||||
///
|
||||
/// Defaults to a 10 second window with 1 second granularity.
|
||||
///
|
||||
/// This controls both how long of a time window we track histogram data for, and the
|
||||
/// granularity in which we roll off old data.
|
||||
///
|
||||
/// As an example, with the default values, we would keep the last 10 seconds worth of
|
||||
/// histogram data, and would remove 1 seconds worth of data at a time as the window rolled
|
||||
/// forward.
|
||||
pub fn histogram(mut self, window: Duration, granularity: Duration) -> Self {
|
||||
self.histogram_window = window;
|
||||
self.histogram_granularity = granularity;
|
||||
self
|
||||
}
|
||||
|
||||
/// Create a [`Receiver`] based on this configuration.
|
||||
pub fn build(self) -> Receiver {
|
||||
Receiver::from_config(self)
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
use super::data::snapshot::Snapshot;
|
||||
use crossbeam_channel::{bounded, Sender};
|
||||
use std::fmt;
|
||||
use tokio_sync::oneshot;
|
||||
|
||||
/// Error conditions when retrieving a snapshot.
|
||||
#[derive(Debug)]
|
||||
pub enum SnapshotError {
|
||||
/// There was an internal error when trying to collect a snapshot.
|
||||
InternalError,
|
||||
|
||||
/// A snapshot was requested but the receiver is shutdown.
|
||||
ReceiverShutdown,
|
||||
}
|
||||
|
||||
/// Various control actions performed by a controller.
|
||||
pub(crate) enum ControlFrame {
|
||||
/// Takes a snapshot of the current metric state.
|
||||
Snapshot(Sender<Snapshot>),
|
||||
|
||||
/// Takes a snapshot of the current metric state, but uses an asynchronous channel.
|
||||
SnapshotAsync(oneshot::Sender<Snapshot>),
|
||||
}
|
||||
|
||||
/// Dedicated handle for performing operations on a running [`Receiver`](crate::receiver::Receiver).
|
||||
///
|
||||
/// The caller is able to request metric snapshots at any time without requiring mutable access to
|
||||
/// the sink. This all flows through the existing control mechanism, and so is very fast.
|
||||
#[derive(Clone)]
|
||||
pub struct Controller {
|
||||
control_tx: Sender<ControlFrame>,
|
||||
}
|
||||
|
||||
impl Controller {
|
||||
pub(crate) fn new(control_tx: Sender<ControlFrame>) -> Controller {
|
||||
Controller { control_tx }
|
||||
}
|
||||
|
||||
/// Retrieves a snapshot of the current metric state.
|
||||
pub fn get_snapshot(&self) -> Result<Snapshot, SnapshotError> {
|
||||
let (tx, rx) = bounded(0);
|
||||
let msg = ControlFrame::Snapshot(tx);
|
||||
|
||||
self.control_tx
|
||||
.send(msg)
|
||||
.map_err(|_| SnapshotError::ReceiverShutdown)
|
||||
.and_then(move |_| rx.recv().map_err(|_| SnapshotError::InternalError))
|
||||
}
|
||||
|
||||
/// Retrieves a snapshot of the current metric state asynchronously.
|
||||
pub fn get_snapshot_async(&self) -> Result<oneshot::Receiver<Snapshot>, SnapshotError> {
|
||||
let (tx, rx) = oneshot::channel();
|
||||
let msg = ControlFrame::SnapshotAsync(tx);
|
||||
|
||||
self.control_tx
|
||||
.send(msg)
|
||||
.map_err(|_| SnapshotError::ReceiverShutdown)
|
||||
.map(move |_| rx)
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Display for SnapshotError {
|
||||
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
match self {
|
||||
SnapshotError::InternalError => write!(f, "internal error during snapshot generation"),
|
||||
SnapshotError::ReceiverShutdown => write!(f, "the receiver is not currently running"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,44 @@
|
|||
use crate::data::ScopedKey;
|
||||
use fnv::FnvBuildHasher;
|
||||
use hashbrown::HashMap;
|
||||
|
||||
pub(crate) struct Counter {
|
||||
data: HashMap<ScopedKey, u64, FnvBuildHasher>,
|
||||
}
|
||||
|
||||
impl Counter {
|
||||
pub fn new() -> Counter {
|
||||
Counter {
|
||||
data: HashMap::default(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn update(&mut self, key: ScopedKey, delta: u64) {
|
||||
let value = self.data.entry(key).or_insert(0);
|
||||
*value = value.wrapping_add(delta);
|
||||
}
|
||||
|
||||
pub fn values(&self) -> Vec<(ScopedKey, u64)> {
|
||||
self.data.iter().map(|(k, v)| (k.clone(), *v)).collect()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{Counter, ScopedKey};
|
||||
|
||||
#[test]
|
||||
fn test_counter_simple_update() {
|
||||
let mut counter = Counter::new();
|
||||
|
||||
let key = ScopedKey(0, "foo".into());
|
||||
counter.update(key, 42);
|
||||
|
||||
let key2 = ScopedKey(0, "foo".to_owned().into());
|
||||
counter.update(key2, 31);
|
||||
|
||||
let values = counter.values();
|
||||
assert_eq!(values.len(), 1);
|
||||
assert_eq!(values[0].1, 73);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
use crate::data::ScopedKey;
|
||||
use fnv::FnvBuildHasher;
|
||||
use hashbrown::HashMap;
|
||||
|
||||
pub(crate) struct Gauge {
|
||||
data: HashMap<ScopedKey, i64, FnvBuildHasher>,
|
||||
}
|
||||
|
||||
impl Gauge {
|
||||
pub fn new() -> Gauge {
|
||||
Gauge {
|
||||
data: HashMap::default(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn update(&mut self, key: ScopedKey, value: i64) {
|
||||
let ivalue = self.data.entry(key).or_insert(0);
|
||||
*ivalue = value;
|
||||
}
|
||||
|
||||
pub fn values(&self) -> Vec<(ScopedKey, i64)> {
|
||||
self.data.iter().map(|(k, v)| (k.clone(), *v)).collect()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{Gauge, ScopedKey};
|
||||
|
||||
#[test]
|
||||
fn test_gauge_simple_update() {
|
||||
let mut gauge = Gauge::new();
|
||||
|
||||
let key = ScopedKey(0, "foo".into());
|
||||
gauge.update(key, 42);
|
||||
|
||||
let values = gauge.values();
|
||||
assert_eq!(values.len(), 1);
|
||||
assert_eq!(values[0].1, 42);
|
||||
|
||||
let key2 = ScopedKey(0, "foo".to_owned().into());
|
||||
gauge.update(key2, 43);
|
||||
|
||||
let values = gauge.values();
|
||||
assert_eq!(values.len(), 1);
|
||||
assert_eq!(values[0].1, 43);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,212 @@
|
|||
use crate::{data::ScopedKey, helper::duration_as_nanos};
|
||||
use fnv::FnvBuildHasher;
|
||||
use hashbrown::HashMap;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
pub(crate) struct Histogram {
|
||||
window: Duration,
|
||||
granularity: Duration,
|
||||
data: HashMap<ScopedKey, WindowedRawHistogram, FnvBuildHasher>,
|
||||
}
|
||||
|
||||
impl Histogram {
|
||||
pub fn new(window: Duration, granularity: Duration) -> Histogram {
|
||||
Histogram {
|
||||
window,
|
||||
granularity,
|
||||
data: HashMap::default(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn update(&mut self, key: ScopedKey, value: u64) {
|
||||
if let Some(wh) = self.data.get_mut(&key) {
|
||||
wh.update(value);
|
||||
} else {
|
||||
let mut wh = WindowedRawHistogram::new(self.window, self.granularity);
|
||||
wh.update(value);
|
||||
let _ = self.data.insert(key, wh);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn upkeep(&mut self, at: Instant) {
|
||||
for (_, histogram) in self.data.iter_mut() {
|
||||
histogram.upkeep(at);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn values(&self) -> Vec<(ScopedKey, HistogramSnapshot)> {
|
||||
self.data
|
||||
.iter()
|
||||
.map(|(k, v)| (k.clone(), v.snapshot()))
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) struct WindowedRawHistogram {
|
||||
buckets: Vec<Vec<u64>>,
|
||||
num_buckets: usize,
|
||||
bucket_index: usize,
|
||||
last_upkeep: Instant,
|
||||
granularity: Duration,
|
||||
}
|
||||
|
||||
impl WindowedRawHistogram {
|
||||
pub fn new(window: Duration, granularity: Duration) -> WindowedRawHistogram {
|
||||
let num_buckets =
|
||||
((duration_as_nanos(window) / duration_as_nanos(granularity)) as usize) + 1;
|
||||
let mut buckets = Vec::with_capacity(num_buckets);
|
||||
|
||||
for _ in 0..num_buckets {
|
||||
let histogram = Vec::new();
|
||||
buckets.push(histogram);
|
||||
}
|
||||
|
||||
WindowedRawHistogram {
|
||||
buckets,
|
||||
num_buckets,
|
||||
bucket_index: 0,
|
||||
last_upkeep: Instant::now(),
|
||||
granularity,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn upkeep(&mut self, at: Instant) {
|
||||
if at >= self.last_upkeep + self.granularity {
|
||||
self.bucket_index += 1;
|
||||
self.bucket_index %= self.num_buckets;
|
||||
self.buckets[self.bucket_index].clear();
|
||||
self.last_upkeep = at;
|
||||
}
|
||||
}
|
||||
|
||||
pub fn update(&mut self, value: u64) {
|
||||
self.buckets[self.bucket_index].push(value);
|
||||
}
|
||||
|
||||
pub fn snapshot(&self) -> HistogramSnapshot {
|
||||
let mut aggregate = Vec::new();
|
||||
for bucket in &self.buckets {
|
||||
aggregate.extend_from_slice(&bucket);
|
||||
}
|
||||
|
||||
HistogramSnapshot::new(aggregate)
|
||||
}
|
||||
}
|
||||
|
||||
/// A point-in-time snapshot of a single histogram.
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub struct HistogramSnapshot {
|
||||
values: Vec<u64>,
|
||||
}
|
||||
|
||||
impl HistogramSnapshot {
|
||||
pub(crate) fn new(values: Vec<u64>) -> Self {
|
||||
HistogramSnapshot { values }
|
||||
}
|
||||
|
||||
/// Gets the raw values that compromise the entire histogram.
|
||||
pub fn values(&self) -> &Vec<u64> {
|
||||
&self.values
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{Histogram, ScopedKey, WindowedRawHistogram};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
#[test]
|
||||
fn test_histogram_simple_update() {
|
||||
let mut histogram = Histogram::new(Duration::new(5, 0), Duration::new(1, 0));
|
||||
|
||||
let key = ScopedKey(0, "foo".into());
|
||||
histogram.update(key, 1245);
|
||||
|
||||
let values = histogram.values();
|
||||
assert_eq!(values.len(), 1);
|
||||
|
||||
let hdr = &values[0].1;
|
||||
assert_eq!(hdr.values().len(), 1);
|
||||
assert_eq!(hdr.values().get(0).unwrap(), &1245);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_histogram_complex_update() {
|
||||
let mut histogram = Histogram::new(Duration::new(5, 0), Duration::new(1, 0));
|
||||
|
||||
let key = ScopedKey(0, "foo".into());
|
||||
histogram.update(key.clone(), 1245);
|
||||
histogram.update(key.clone(), 213);
|
||||
histogram.update(key.clone(), 1022);
|
||||
histogram.update(key, 1248);
|
||||
|
||||
let values = histogram.values();
|
||||
assert_eq!(values.len(), 1);
|
||||
|
||||
let hdr = &values[0].1;
|
||||
assert_eq!(hdr.values().len(), 4);
|
||||
assert_eq!(hdr.values().get(0).unwrap(), &1245);
|
||||
assert_eq!(hdr.values().get(1).unwrap(), &213);
|
||||
assert_eq!(hdr.values().get(2).unwrap(), &1022);
|
||||
assert_eq!(hdr.values().get(3).unwrap(), &1248);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_windowed_histogram_rollover() {
|
||||
let mut wh = WindowedRawHistogram::new(Duration::new(5, 0), Duration::new(1, 0));
|
||||
let now = Instant::now();
|
||||
|
||||
let snapshot = wh.snapshot();
|
||||
assert_eq!(snapshot.values().len(), 0);
|
||||
|
||||
wh.update(1);
|
||||
wh.update(2);
|
||||
let snapshot = wh.snapshot();
|
||||
assert_eq!(snapshot.values().len(), 2);
|
||||
|
||||
// Roll forward 3 seconds, should still have everything.
|
||||
let now = now + Duration::new(1, 0);
|
||||
wh.upkeep(now);
|
||||
let snapshot = wh.snapshot();
|
||||
assert_eq!(snapshot.values().len(), 2);
|
||||
|
||||
let now = now + Duration::new(1, 0);
|
||||
wh.upkeep(now);
|
||||
let snapshot = wh.snapshot();
|
||||
assert_eq!(snapshot.values().len(), 2);
|
||||
|
||||
let now = now + Duration::new(1, 0);
|
||||
wh.upkeep(now);
|
||||
let snapshot = wh.snapshot();
|
||||
assert_eq!(snapshot.values().len(), 2);
|
||||
|
||||
// Pump in some new values.
|
||||
wh.update(3);
|
||||
wh.update(4);
|
||||
wh.update(5);
|
||||
|
||||
let snapshot = wh.snapshot();
|
||||
assert_eq!(snapshot.values().len(), 5);
|
||||
|
||||
// Roll forward 3 seconds, and make sure the first two values are gone.
|
||||
// You might think this should be 2 seconds, but we have one extra bucket
|
||||
// allocated so that there's always a clear bucket that we can write into.
|
||||
// This means we have more than our total window, but only having the exact
|
||||
// number of buckets would mean we were constantly missing a bucket's worth
|
||||
// of granularity.
|
||||
let now = now + Duration::new(1, 0);
|
||||
wh.upkeep(now);
|
||||
let snapshot = wh.snapshot();
|
||||
assert_eq!(snapshot.values().len(), 5);
|
||||
|
||||
let now = now + Duration::new(1, 0);
|
||||
wh.upkeep(now);
|
||||
let snapshot = wh.snapshot();
|
||||
assert_eq!(snapshot.values().len(), 5);
|
||||
|
||||
let now = now + Duration::new(1, 0);
|
||||
wh.upkeep(now);
|
||||
let snapshot = wh.snapshot();
|
||||
assert_eq!(snapshot.values().len(), 3);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,73 @@
|
|||
use std::{
|
||||
borrow::Cow,
|
||||
fmt::{self, Display},
|
||||
};
|
||||
|
||||
pub mod counter;
|
||||
pub mod gauge;
|
||||
pub mod histogram;
|
||||
pub mod snapshot;
|
||||
|
||||
pub(crate) use self::{counter::Counter, gauge::Gauge, histogram::Histogram, snapshot::Snapshot};
|
||||
|
||||
pub type MetricKey = Cow<'static, str>;
|
||||
|
||||
/// A measurement.
|
||||
///
|
||||
/// Samples are the decoupled way of submitting data into the sink.
|
||||
#[derive(Debug)]
|
||||
pub(crate) enum Sample {
|
||||
/// A counter delta.
|
||||
///
|
||||
/// The value is added directly to the existing counter, and so negative deltas will decrease
|
||||
/// the counter, and positive deltas will increase the counter.
|
||||
Count(ScopedKey, u64),
|
||||
|
||||
/// A single value, also known as a gauge.
|
||||
///
|
||||
/// Values operate in last-write-wins mode.
|
||||
///
|
||||
/// Values themselves cannot be incremented or decremented, so you must hold them externally
|
||||
/// before sending them.
|
||||
Gauge(ScopedKey, i64),
|
||||
|
||||
/// A timed sample.
|
||||
///
|
||||
/// Includes the start and end times.
|
||||
TimingHistogram(ScopedKey, u64, u64),
|
||||
|
||||
/// A single value measured over time.
|
||||
///
|
||||
/// Unlike a gauge, where the value is only ever measured at a point in time, value histogram
|
||||
/// measure values over time, and their distribution. This is nearly identical to timing
|
||||
/// histograms, since the end result is just a single number, but we don't spice it up with
|
||||
/// special unit labels or anything.
|
||||
ValueHistogram(ScopedKey, u64),
|
||||
}
|
||||
|
||||
/// An integer scoped metric key.
|
||||
#[derive(Clone, Hash, PartialEq, Eq, Debug)]
|
||||
pub(crate) struct ScopedKey(pub u64, pub MetricKey);
|
||||
|
||||
impl ScopedKey {
|
||||
pub(crate) fn id(&self) -> u64 {
|
||||
self.0
|
||||
}
|
||||
pub(crate) fn into_string_scoped(self, scope: String) -> StringScopedKey {
|
||||
StringScopedKey(scope, self.1)
|
||||
}
|
||||
}
|
||||
|
||||
/// A string scoped metric key.
|
||||
#[derive(Clone, Hash, PartialEq, Eq, Debug)]
|
||||
pub(crate) struct StringScopedKey(String, MetricKey);
|
||||
|
||||
impl Display for StringScopedKey {
|
||||
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
|
||||
if self.0.is_empty() {
|
||||
write!(f, "{}", self.1)
|
||||
} else {
|
||||
write!(f, "{}.{}", self.0, self.1.as_ref())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,161 @@
|
|||
use super::histogram::HistogramSnapshot;
|
||||
use metrics_core::MetricsRecorder;
|
||||
use std::fmt::Display;
|
||||
|
||||
/// A typed metric measurement, used in snapshots.
|
||||
///
|
||||
/// This type provides a way to wrap the value of a metric, for use in a snapshot, while also
|
||||
/// providing the overall type of the metric, so that downstream consumers who how to properly
|
||||
/// format the data.
|
||||
#[derive(Debug, PartialEq, Eq)]
|
||||
pub enum TypedMeasurement {
|
||||
Counter(String, u64),
|
||||
Gauge(String, i64),
|
||||
TimingHistogram(String, HistogramSnapshot),
|
||||
ValueHistogram(String, HistogramSnapshot),
|
||||
}
|
||||
|
||||
/// A point-in-time view of metric data.
|
||||
#[derive(Default, Debug)]
|
||||
pub struct Snapshot {
|
||||
measurements: Vec<TypedMeasurement>,
|
||||
}
|
||||
|
||||
impl Snapshot {
|
||||
/// Stores a counter value for the given metric key.
|
||||
pub(crate) fn set_count<T>(&mut self, key: T, value: u64)
|
||||
where
|
||||
T: Display,
|
||||
{
|
||||
self.measurements
|
||||
.push(TypedMeasurement::Counter(key.to_string(), value));
|
||||
}
|
||||
|
||||
/// Stores a gauge value for the given metric key.
|
||||
pub(crate) fn set_gauge<T>(&mut self, key: T, value: i64)
|
||||
where
|
||||
T: Display,
|
||||
{
|
||||
self.measurements
|
||||
.push(TypedMeasurement::Gauge(key.to_string(), value));
|
||||
}
|
||||
|
||||
/// Sets timing percentiles for the given metric key.
|
||||
///
|
||||
/// From the given `HdrHistogram`, all the specific `percentiles` will be extracted and stored.
|
||||
pub(crate) fn set_timing_histogram<T>(&mut self, key: T, h: HistogramSnapshot)
|
||||
where
|
||||
T: Display,
|
||||
{
|
||||
self.measurements
|
||||
.push(TypedMeasurement::TimingHistogram(key.to_string(), h));
|
||||
}
|
||||
|
||||
/// Sets value percentiles for the given metric key.
|
||||
///
|
||||
/// From the given `HdrHistogram`, all the specific `percentiles` will be extracted and stored.
|
||||
pub(crate) fn set_value_histogram<T>(&mut self, key: T, h: HistogramSnapshot)
|
||||
where
|
||||
T: Display,
|
||||
{
|
||||
self.measurements
|
||||
.push(TypedMeasurement::ValueHistogram(key.to_string(), h));
|
||||
}
|
||||
|
||||
/// Records this [`Snapshot`] to the provided [`MetricsRecorder`].
|
||||
pub fn record<R: MetricsRecorder>(&self, recorder: &mut R) {
|
||||
for measurement in &self.measurements {
|
||||
match measurement {
|
||||
TypedMeasurement::Counter(key, value) => recorder.record_counter(key, *value),
|
||||
TypedMeasurement::Gauge(key, value) => recorder.record_gauge(key, *value),
|
||||
TypedMeasurement::TimingHistogram(key, hs) => {
|
||||
recorder.record_histogram(key, hs.values().as_slice());
|
||||
}
|
||||
TypedMeasurement::ValueHistogram(key, hs) => {
|
||||
recorder.record_histogram(key, hs.values().as_slice());
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Converts this [`Snapshot`] to the underlying vector of measurements.
|
||||
pub fn into_measurements(self) -> Vec<TypedMeasurement> {
|
||||
self.measurements
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{HistogramSnapshot, MetricsRecorder, Snapshot, TypedMeasurement};
|
||||
use std::collections::HashMap;
|
||||
|
||||
#[derive(Default)]
|
||||
struct MockRecorder {
|
||||
counter: HashMap<String, u64>,
|
||||
gauge: HashMap<String, i64>,
|
||||
histogram: HashMap<String, u64>,
|
||||
}
|
||||
|
||||
impl MockRecorder {
|
||||
pub fn get_counter_value(&self, key: &String) -> Option<&u64> {
|
||||
self.counter.get(key)
|
||||
}
|
||||
|
||||
pub fn get_gauge_value(&self, key: &String) -> Option<&i64> {
|
||||
self.gauge.get(key)
|
||||
}
|
||||
|
||||
pub fn get_histogram_value(&self, key: &String) -> Option<&u64> {
|
||||
self.histogram.get(key)
|
||||
}
|
||||
}
|
||||
|
||||
impl MetricsRecorder for MockRecorder {
|
||||
fn record_counter<K: AsRef<str>>(&mut self, key: K, value: u64) {
|
||||
let entry = self.counter.entry(key.as_ref().to_owned()).or_insert(0);
|
||||
*entry += value;
|
||||
}
|
||||
|
||||
fn record_gauge<K: AsRef<str>>(&mut self, key: K, value: i64) {
|
||||
let entry = self.gauge.entry(key.as_ref().to_owned()).or_insert(0);
|
||||
*entry += value;
|
||||
}
|
||||
|
||||
fn record_histogram<K: AsRef<str>>(&mut self, key: K, value: u64) {
|
||||
let entry = self.histogram.entry(key.as_ref().to_owned()).or_insert(0);
|
||||
*entry += value;
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_snapshot_simple_set_and_get() {
|
||||
let key = "ok".to_owned();
|
||||
let mut snapshot = Snapshot::default();
|
||||
snapshot.set_count(key.clone(), 1);
|
||||
snapshot.set_gauge(key.clone(), 42);
|
||||
|
||||
let values = snapshot.into_measurements();
|
||||
|
||||
assert_eq!(values[0], TypedMeasurement::Counter(key.clone(), 1));
|
||||
assert_eq!(values[1], TypedMeasurement::Gauge(key.clone(), 42));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_snapshot_recorder() {
|
||||
let key = "ok".to_owned();
|
||||
let mut snapshot = Snapshot::default();
|
||||
snapshot.set_count(key.clone(), 7);
|
||||
snapshot.set_gauge(key.clone(), 42);
|
||||
|
||||
let hvalues = vec![10, 25, 42, 97];
|
||||
let histogram = HistogramSnapshot::new(hvalues);
|
||||
snapshot.set_timing_histogram(key.clone(), histogram);
|
||||
|
||||
let mut recorder = MockRecorder::default();
|
||||
snapshot.export(&mut recorder);
|
||||
|
||||
assert_eq!(recorder.get_counter_value(&key), Some(&7));
|
||||
assert_eq!(recorder.get_gauge_value(&key), Some(&42));
|
||||
assert_eq!(recorder.get_histogram_value(&key), Some(&174));
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
use std::{
|
||||
io::{Error, ErrorKind},
|
||||
time::Duration,
|
||||
};
|
||||
|
||||
/// Helpers to create an I/O error from a string.
|
||||
pub fn io_error(reason: &str) -> Error {
|
||||
Error::new(ErrorKind::Other, reason)
|
||||
}
|
||||
|
||||
/// Converts a duration to nanoseconds.
|
||||
pub fn duration_as_nanos(d: Duration) -> u64 {
|
||||
(d.as_secs() * 1_000_000_000) + u64::from(d.subsec_nanos())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::duration_as_nanos;
|
||||
use std::time::Duration;
|
||||
|
||||
#[test]
|
||||
fn test_simple_duration_as_nanos() {
|
||||
let d1 = Duration::from_secs(3);
|
||||
let d2 = Duration::from_millis(500);
|
||||
|
||||
assert_eq!(duration_as_nanos(d1), 3_000_000_000);
|
||||
assert_eq!(duration_as_nanos(d2), 500_000_000);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,133 @@
|
|||
//! High-speed metrics collection library.
|
||||
//!
|
||||
//! `metrics` provides a generalized metrics collection library targeted at users who want to log
|
||||
//! metrics at high volume and high speed.
|
||||
//!
|
||||
//! # Design
|
||||
//!
|
||||
//! The library follows a pattern of "senders" and a "receiver."
|
||||
//!
|
||||
//! Callers create a [`Receiver`], which acts as a contained unit: metric registration,
|
||||
//! aggregation, and summarization. The [`Receiver`] is intended to be spawned onto a dedicated
|
||||
//! background thread.
|
||||
//!
|
||||
//! Once a [`Receiver`] is created, callers can either create a [`Sink`] for sending metrics, or a
|
||||
//! [`Controller`] for getting metrics out.
|
||||
//!
|
||||
//! A [`Sink`] can be cheaply cloned and does not require a mutable reference to send metrics, so
|
||||
//! callers have increased flexibility in usage and control over whether or not to clone sinks,
|
||||
//! share references, etc.
|
||||
//!
|
||||
//! A [`Controller`] provides both a synchronous and asynchronous snapshotting interface, which is
|
||||
//! [`metrics-core`][metrics_core] compatible for exporting. This allows flexibility in
|
||||
//! integration amongst traditional single-threaded or hand-rolled multi-threaded applications and
|
||||
//! the emerging asynchronous Rust ecosystem.
|
||||
//!
|
||||
//! # Performance
|
||||
//!
|
||||
//! Being based on [`crossbeam-channel`][crossbeam_channel] allows us to process close to ten
|
||||
//! million metrics per second using a single core, with average ingest latencies of around 100ns.
|
||||
//!
|
||||
//! # Metrics
|
||||
//!
|
||||
//! Counters, gauges, and histograms are supported, and follow the definitions outlined in
|
||||
//! [`metrics-core`][metrics_core].
|
||||
//!
|
||||
//! Here's a simple example of creating a receiver and working with a sink:
|
||||
//!
|
||||
//! ```
|
||||
//! # extern crate metrics;
|
||||
//! use metrics::Receiver;
|
||||
//! use std::{thread, time::Duration};
|
||||
//! let receiver = Receiver::builder().build();
|
||||
//! let sink = receiver.get_sink();
|
||||
//!
|
||||
//! // We can update a counter. Counters are monotonic, unsigned integers that start at 0 and
|
||||
//! // increase over time.
|
||||
//! sink.record_count("widgets", 5);
|
||||
//!
|
||||
//! // We can update a gauge. Gauges are signed, and hold on to the last value they were updated
|
||||
//! // to, so you need to track the overall value on your own.
|
||||
//! sink.record_gauge("red_balloons", 99);
|
||||
//!
|
||||
//! // We can update a timing histogram. For timing, you also must measure the start and end
|
||||
//! // time using the built-in `Clock` exposed by the sink. The receiver internally converts the
|
||||
//! // raw values to calculate the actual wall clock time (in nanoseconds) on your behalf, so you
|
||||
//! // can't just pass in any old number.. otherwise you'll get erroneous measurements!
|
||||
//! let start = sink.clock().start();
|
||||
//! thread::sleep(Duration::from_millis(10));
|
||||
//! let end = sink.clock().end();
|
||||
//! sink.record_timing("db.gizmo_query", start, end);
|
||||
//!
|
||||
//! // Finally, we can update a value histogram. Technically speaking, value histograms aren't
|
||||
//! // fundamentally different from timing histograms. If you use a timing histogram, we do the
|
||||
//! // math for you of getting the time difference, and we make sure the metric name has the right
|
||||
//! // unit suffix so you can tell it's measuring time, but other than that, nearly identical!
|
||||
//! let buf_size = 4096;
|
||||
//! sink.record_value("buf_size", buf_size);
|
||||
//! ```
|
||||
//!
|
||||
//! # Scopes
|
||||
//!
|
||||
//! Metrics can be scoped, not unlike loggers, at the [`Sink`] level. This allows sinks to easily
|
||||
//! nest themselves without callers ever needing to care about where they're located.
|
||||
//!
|
||||
//! This feature is a simpler approach to tagging: while not as semantically rich, it provides the
|
||||
//! level of detail necessary to distinguish a single metric between multiple callsites.
|
||||
//!
|
||||
//! For example, after getting a [`Sink`] from the [`Receiver`], we can easily nest ourselves under
|
||||
//! the root scope and then send some metrics:
|
||||
//!
|
||||
//! ```
|
||||
//! # extern crate metrics;
|
||||
//! use metrics::Receiver;
|
||||
//! let receiver = Receiver::builder().build();
|
||||
//!
|
||||
//! // This sink has no scope aka the root scope. The metric will just end up as "widgets".
|
||||
//! let root_sink = receiver.get_sink();
|
||||
//! root_sink.record_count("widgets", 42);
|
||||
//!
|
||||
//! // This sink is under the "secret" scope. Since we derived ourselves from the root scope,
|
||||
//! // we're not nested under anything, but our metric name will end up being "secret.widgets".
|
||||
//! let scoped_sink = root_sink.scoped("secret");
|
||||
//! scoped_sink.record_count("widgets", 42);
|
||||
//!
|
||||
//! // This sink is under the "supersecret" scope, but we're also nested! The metric name for this
|
||||
//! // sample will end up being "secret.supersecret.widget".
|
||||
//! let scoped_sink_two = scoped_sink.scoped("supersecret");
|
||||
//! scoped_sink_two.record_count("widgets", 42);
|
||||
//!
|
||||
//! // Sinks retain their scope even when cloned, so the metric name will be the same as above.
|
||||
//! let cloned_sink = scoped_sink_two.clone();
|
||||
//! cloned_sink.record_count("widgets", 42);
|
||||
//!
|
||||
//! // This sink will be nested two levels deeper than its parent by using a slightly different
|
||||
//! // input scope: scope can be a single string, or multiple strings, which is interpreted as
|
||||
//! // nesting N levels deep.
|
||||
//! //
|
||||
//! // This metric name will end up being "super.secret.ultra.special.widgets".
|
||||
//! let scoped_sink_three = scoped_sink.scoped(&["super", "secret", "ultra", "special"]);
|
||||
//! scoped_sink_two.record_count("widgets", 42);
|
||||
//! ```
|
||||
//!
|
||||
//! [crossbeam_channel]: https://docs.rs/crossbeam-channel
|
||||
//! [metrics_core]: https://docs.rs/metrics-core
|
||||
mod configuration;
|
||||
mod control;
|
||||
mod data;
|
||||
mod helper;
|
||||
mod receiver;
|
||||
mod scopes;
|
||||
mod sink;
|
||||
|
||||
pub use self::{
|
||||
configuration::Configuration,
|
||||
control::{Controller, SnapshotError},
|
||||
data::histogram::HistogramSnapshot,
|
||||
receiver::Receiver,
|
||||
sink::{AsScoped, Sink, SinkError},
|
||||
};
|
||||
|
||||
pub mod snapshot {
|
||||
pub use super::data::snapshot::{Snapshot, TypedMeasurement};
|
||||
}
|
||||
|
|
@ -0,0 +1,220 @@
|
|||
use crate::{
|
||||
configuration::Configuration,
|
||||
control::{ControlFrame, Controller},
|
||||
data::{Counter, Gauge, Histogram, Sample, ScopedKey, Snapshot, StringScopedKey},
|
||||
scopes::Scopes,
|
||||
sink::Sink,
|
||||
};
|
||||
use crossbeam_channel::{self, bounded, tick, Select, TryRecvError};
|
||||
use quanta::Clock;
|
||||
use std::{
|
||||
sync::Arc,
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
|
||||
/// Wrapper for all messages that flow over the data channel between sink/receiver.
|
||||
pub(crate) enum MessageFrame {
|
||||
/// A normal data message holding a metric sample.
|
||||
Data(Sample),
|
||||
}
|
||||
|
||||
/// Metrics receiver which aggregates and processes samples.
|
||||
pub struct Receiver {
|
||||
config: Configuration,
|
||||
|
||||
// Sample aggregation machinery.
|
||||
msg_tx: crossbeam_channel::Sender<MessageFrame>,
|
||||
msg_rx: Option<crossbeam_channel::Receiver<MessageFrame>>,
|
||||
control_tx: crossbeam_channel::Sender<ControlFrame>,
|
||||
control_rx: Option<crossbeam_channel::Receiver<ControlFrame>>,
|
||||
|
||||
// Metric machinery.
|
||||
counter: Counter,
|
||||
gauge: Gauge,
|
||||
thistogram: Histogram,
|
||||
vhistogram: Histogram,
|
||||
|
||||
clock: Clock,
|
||||
scopes: Arc<Scopes>,
|
||||
}
|
||||
|
||||
impl Receiver {
|
||||
pub(crate) fn from_config(config: Configuration) -> Receiver {
|
||||
// Create our data, control, and buffer channels.
|
||||
let (msg_tx, msg_rx) = bounded(config.capacity);
|
||||
let (control_tx, control_rx) = bounded(16);
|
||||
|
||||
let histogram_window = config.histogram_window;
|
||||
let histogram_granularity = config.histogram_granularity;
|
||||
|
||||
Receiver {
|
||||
config,
|
||||
msg_tx,
|
||||
msg_rx: Some(msg_rx),
|
||||
control_tx,
|
||||
control_rx: Some(control_rx),
|
||||
counter: Counter::new(),
|
||||
gauge: Gauge::new(),
|
||||
thistogram: Histogram::new(histogram_window, histogram_granularity),
|
||||
vhistogram: Histogram::new(histogram_window, histogram_granularity),
|
||||
clock: Clock::new(),
|
||||
scopes: Arc::new(Scopes::new()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Gets a builder to configure a [`Receiver`] instance with.
|
||||
pub fn builder() -> Configuration {
|
||||
Configuration::default()
|
||||
}
|
||||
|
||||
/// Creates a [`Sink`] bound to this receiver.
|
||||
pub fn get_sink(&self) -> Sink {
|
||||
Sink::new_with_scope_id(
|
||||
self.msg_tx.clone(),
|
||||
self.clock.clone(),
|
||||
self.scopes.clone(),
|
||||
"".to_owned(),
|
||||
0,
|
||||
)
|
||||
}
|
||||
|
||||
/// Creates a [`Controller`] bound to this receiver.
|
||||
pub fn get_controller(&self) -> Controller {
|
||||
Controller::new(self.control_tx.clone())
|
||||
}
|
||||
|
||||
/// Run the receiver.
|
||||
///
|
||||
/// This is blocking, and should be run in a dedicated background thread.
|
||||
pub fn run(&mut self) {
|
||||
let batch_size = self.config.batch_size;
|
||||
let mut batch = Vec::with_capacity(batch_size);
|
||||
let upkeep_rx = tick(Duration::from_millis(100));
|
||||
let control_rx = self.control_rx.take().expect("failed to take control rx");
|
||||
let msg_rx = self.msg_rx.take().expect("failed to take msg rx");
|
||||
|
||||
let mut selector = Select::new();
|
||||
let _ = selector.recv(&upkeep_rx);
|
||||
let _ = selector.recv(&control_rx);
|
||||
let _ = selector.recv(&msg_rx);
|
||||
|
||||
loop {
|
||||
// Block on having something to do.
|
||||
let _ = selector.ready();
|
||||
|
||||
if upkeep_rx.try_recv().is_ok() {
|
||||
let now = Instant::now();
|
||||
self.thistogram.upkeep(now);
|
||||
self.vhistogram.upkeep(now);
|
||||
}
|
||||
|
||||
while let Ok(cframe) = control_rx.try_recv() {
|
||||
self.process_control_frame(cframe);
|
||||
}
|
||||
|
||||
loop {
|
||||
match msg_rx.try_recv() {
|
||||
Ok(mframe) => batch.push(mframe),
|
||||
Err(TryRecvError::Empty) => break,
|
||||
Err(e) => eprintln!("error receiving message frame: {}", e),
|
||||
}
|
||||
|
||||
if batch.len() == batch_size {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if !batch.is_empty() {
|
||||
for mframe in batch.drain(0..) {
|
||||
self.process_msg_frame(mframe);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Gets the string representation of an integer scope.
|
||||
///
|
||||
/// Returns `Some(scope)` if found, `None` otherwise. Scope ID `0` is reserved for the root
|
||||
/// scope.
|
||||
fn get_string_scope(&self, key: ScopedKey) -> Option<StringScopedKey> {
|
||||
let scope_id = key.id();
|
||||
if scope_id == 0 {
|
||||
return Some(key.into_string_scoped("".to_owned()));
|
||||
}
|
||||
|
||||
self.scopes
|
||||
.get(scope_id)
|
||||
.map(|scope| key.into_string_scoped(scope))
|
||||
}
|
||||
|
||||
/// Gets a snapshot of the current metrics/facets.
|
||||
fn get_snapshot(&self) -> Snapshot {
|
||||
let mut snapshot = Snapshot::default();
|
||||
let cvalues = self.counter.values();
|
||||
let gvalues = self.gauge.values();
|
||||
let tvalues = self.thistogram.values();
|
||||
let vvalues = self.vhistogram.values();
|
||||
|
||||
for (key, value) in cvalues {
|
||||
if let Some(actual_key) = self.get_string_scope(key) {
|
||||
snapshot.set_count(actual_key, value);
|
||||
}
|
||||
}
|
||||
|
||||
for (key, value) in gvalues {
|
||||
if let Some(actual_key) = self.get_string_scope(key) {
|
||||
snapshot.set_gauge(actual_key, value);
|
||||
}
|
||||
}
|
||||
|
||||
for (key, value) in tvalues {
|
||||
if let Some(actual_key) = self.get_string_scope(key) {
|
||||
snapshot.set_timing_histogram(actual_key, value);
|
||||
}
|
||||
}
|
||||
|
||||
for (key, value) in vvalues {
|
||||
if let Some(actual_key) = self.get_string_scope(key) {
|
||||
snapshot.set_value_histogram(actual_key, value);
|
||||
}
|
||||
}
|
||||
|
||||
snapshot
|
||||
}
|
||||
|
||||
/// Processes a control frame.
|
||||
fn process_control_frame(&self, msg: ControlFrame) {
|
||||
match msg {
|
||||
ControlFrame::Snapshot(tx) => {
|
||||
let snapshot = self.get_snapshot();
|
||||
let _ = tx.send(snapshot);
|
||||
}
|
||||
ControlFrame::SnapshotAsync(tx) => {
|
||||
let snapshot = self.get_snapshot();
|
||||
let _ = tx.send(snapshot);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Processes a message frame.
|
||||
fn process_msg_frame(&mut self, msg: MessageFrame) {
|
||||
match msg {
|
||||
MessageFrame::Data(sample) => match sample {
|
||||
Sample::Count(key, count) => {
|
||||
self.counter.update(key, count);
|
||||
}
|
||||
Sample::Gauge(key, value) => {
|
||||
self.gauge.update(key, value);
|
||||
}
|
||||
Sample::TimingHistogram(key, start, end) => {
|
||||
let delta = end - start;
|
||||
self.counter.update(key.clone(), 1);
|
||||
self.thistogram.update(key, delta);
|
||||
}
|
||||
Sample::ValueHistogram(key, value) => {
|
||||
self.vhistogram.update(key, value);
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
use parking_lot::RwLock;
|
||||
use std::collections::HashMap;
|
||||
|
||||
pub struct Inner {
|
||||
id: u64,
|
||||
forward: HashMap<String, u64>,
|
||||
backward: HashMap<u64, String>,
|
||||
}
|
||||
|
||||
impl Inner {
|
||||
pub fn new() -> Self {
|
||||
Inner {
|
||||
id: 1,
|
||||
forward: HashMap::new(),
|
||||
backward: HashMap::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub struct Scopes {
|
||||
inner: RwLock<Inner>,
|
||||
}
|
||||
|
||||
impl Scopes {
|
||||
pub fn new() -> Self {
|
||||
Scopes {
|
||||
inner: RwLock::new(Inner::new()),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn register(&self, scope: String) -> u64 {
|
||||
let mut wg = self.inner.write();
|
||||
|
||||
// If the key is already registered, send back the existing scope ID.
|
||||
if wg.forward.contains_key(&scope) {
|
||||
return wg.forward.get(&scope).cloned().unwrap();
|
||||
}
|
||||
|
||||
// Otherwise, take the current scope ID for this registration, store it, and increment
|
||||
// the scope ID counter for the next registration.
|
||||
let scope_id = wg.id;
|
||||
let _ = wg.forward.insert(scope.clone(), scope_id);
|
||||
let _ = wg.backward.insert(scope_id, scope);
|
||||
wg.id += 1;
|
||||
scope_id
|
||||
}
|
||||
|
||||
pub fn get(&self, scope_id: u64) -> Option<String> {
|
||||
// See if we have an entry for the scope ID, and clone the scope if so.
|
||||
let rg = self.inner.read();
|
||||
rg.backward.get(&scope_id).cloned()
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,174 @@
|
|||
use crate::{
|
||||
data::{MetricKey, Sample, ScopedKey},
|
||||
helper::io_error,
|
||||
receiver::MessageFrame,
|
||||
scopes::Scopes,
|
||||
};
|
||||
use crossbeam_channel::Sender;
|
||||
use quanta::Clock;
|
||||
use std::sync::Arc;
|
||||
|
||||
/// Erorrs during sink creation.
|
||||
#[derive(Debug)]
|
||||
pub enum SinkError {
|
||||
/// The scope value given was invalid i.e. empty or illegal characters.
|
||||
InvalidScope,
|
||||
}
|
||||
|
||||
/// A value that can be used as a metric scope.
|
||||
pub trait AsScoped<'a> {
|
||||
fn as_scoped(&'a self, base: String) -> String;
|
||||
}
|
||||
|
||||
/// Handle for sending metric samples into the receiver.
|
||||
///
|
||||
/// [`Sink`] is cloneable, and can not only send metric samples but can register and deregister
|
||||
/// metric facets at any time.
|
||||
pub struct Sink {
|
||||
msg_tx: Sender<MessageFrame>,
|
||||
clock: Clock,
|
||||
scopes: Arc<Scopes>,
|
||||
scope: String,
|
||||
scope_id: u64,
|
||||
}
|
||||
|
||||
impl Sink {
|
||||
pub(crate) fn new(
|
||||
msg_tx: Sender<MessageFrame>,
|
||||
clock: Clock,
|
||||
scopes: Arc<Scopes>,
|
||||
scope: String,
|
||||
) -> Sink {
|
||||
let scope_id = scopes.register(scope.clone());
|
||||
|
||||
Sink {
|
||||
msg_tx,
|
||||
clock,
|
||||
scopes,
|
||||
scope,
|
||||
scope_id,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn new_with_scope_id(
|
||||
msg_tx: Sender<MessageFrame>,
|
||||
clock: Clock,
|
||||
scopes: Arc<Scopes>,
|
||||
scope: String,
|
||||
scope_id: u64,
|
||||
) -> Sink {
|
||||
Sink {
|
||||
msg_tx,
|
||||
clock,
|
||||
scopes,
|
||||
scope,
|
||||
scope_id,
|
||||
}
|
||||
}
|
||||
|
||||
/// Creates a scoped clone of this [`Sink`].
|
||||
///
|
||||
/// Scoping controls the resulting metric name for any metrics sent by this [`Sink`]. For
|
||||
/// example, you might have a metric called `messages_sent`.
|
||||
///
|
||||
/// With scoping, you could have independent versions of the same metric. This is useful for
|
||||
/// having the same "base" metric name but with broken down values.
|
||||
///
|
||||
/// Going further with the above example, if you had a server, and listened on multiple
|
||||
/// addresses, maybe you would have a scoped [`Sink`] per listener, and could end up with
|
||||
/// metrics that look like this:
|
||||
/// - `listener.a.messages_sent`
|
||||
/// - `listener.b.messages_sent`
|
||||
/// - `listener.c.messages_sent`
|
||||
/// - etc
|
||||
///
|
||||
/// Scopes are also inherited. If you create a scoped [`Sink`] from another [`Sink`] which is
|
||||
/// already scoped, the scopes will be merged together using a `.` as the string separator.
|
||||
/// This makes it easy to nest scopes. Cloning a scoped [`Sink`], though, will inherit the
|
||||
/// same scope as the original.
|
||||
pub fn scoped<'a, S: AsScoped<'a> + ?Sized>(&self, scope: &'a S) -> Sink {
|
||||
let new_scope = scope.as_scoped(self.scope.clone());
|
||||
|
||||
Sink::new(
|
||||
self.msg_tx.clone(),
|
||||
self.clock.clone(),
|
||||
self.scopes.clone(),
|
||||
new_scope,
|
||||
)
|
||||
}
|
||||
|
||||
/// Gets the current time, in nanoseconds, from the internal high-speed clock.
|
||||
pub fn now(&self) -> u64 {
|
||||
self.clock.now()
|
||||
}
|
||||
|
||||
/// Records the count for a given metric.
|
||||
pub fn record_count<K: Into<MetricKey>>(&self, key: K, delta: u64) {
|
||||
let scoped_key = ScopedKey(self.scope_id, key.into());
|
||||
self.send(Sample::Count(scoped_key, delta))
|
||||
}
|
||||
|
||||
/// Records the gauge for a given metric.
|
||||
pub fn record_gauge<K: Into<MetricKey>>(&self, key: K, value: i64) {
|
||||
let scoped_key = ScopedKey(self.scope_id, key.into());
|
||||
self.send(Sample::Gauge(scoped_key, value))
|
||||
}
|
||||
|
||||
/// Records the timing histogram for a given metric.
|
||||
pub fn record_timing<K: Into<MetricKey>>(&self, key: K, start: u64, end: u64) {
|
||||
let scoped_key = ScopedKey(self.scope_id, key.into());
|
||||
self.send(Sample::TimingHistogram(scoped_key, start, end))
|
||||
}
|
||||
|
||||
/// Records the value histogram for a given metric.
|
||||
pub fn record_value<K: Into<MetricKey>>(&self, key: K, value: u64) {
|
||||
let scoped_key = ScopedKey(self.scope_id, key.into());
|
||||
self.send(Sample::ValueHistogram(scoped_key, value))
|
||||
}
|
||||
|
||||
/// Sends a raw metric sample to the receiver.
|
||||
fn send(&self, sample: Sample) {
|
||||
let _ = self
|
||||
.msg_tx
|
||||
.send(MessageFrame::Data(sample))
|
||||
.map_err(|_| io_error("failed to send sample"));
|
||||
}
|
||||
}
|
||||
|
||||
impl Clone for Sink {
|
||||
fn clone(&self) -> Sink {
|
||||
Sink {
|
||||
msg_tx: self.msg_tx.clone(),
|
||||
clock: self.clock.clone(),
|
||||
scopes: self.scopes.clone(),
|
||||
scope: self.scope.clone(),
|
||||
scope_id: self.scope_id,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> AsScoped<'a> for str {
|
||||
fn as_scoped(&'a self, mut base: String) -> String {
|
||||
if !base.is_empty() {
|
||||
base.push_str(".");
|
||||
}
|
||||
base.push_str(self);
|
||||
base
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a, 'b, T> AsScoped<'a> for T
|
||||
where
|
||||
&'a T: AsRef<[&'b str]>,
|
||||
T: 'a,
|
||||
{
|
||||
fn as_scoped(&'a self, mut base: String) -> String {
|
||||
for item in self.as_ref() {
|
||||
if !base.is_empty() {
|
||||
base.push('.');
|
||||
}
|
||||
base.push_str(item);
|
||||
}
|
||||
base
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue