`PowerOfTwoChoices` requires a Random Number Generator. In order for
this randomization source to be configurable (i.e. for tests),
`PowerOfTwoChoices` is generic over its implementation of `rand::Rng`;
however, this leads to needless boilerplate when building P2C balancers.
Because load balancers do not need a cryptographically strong RNG, we
can use `rand::SmallRng` (which is `Send + Sync`). `PowerOfTwoChoices`
exposes constructors that take a `SmallRng`.
In order to do this, the `tower-balance` crate now requires `rand = "0.5"`.
When an endpoint's state changes in some way, it may need to be rebound to a
new service, and reinserted into the load balancer. This PR changes
`tower-balance` so that, rather than ignoring duplicate `Insert`s, the new
endpoint replaces the old endpoint. The new endpoint is always placed on the
not-ready list; if the replaced endpoint was on the ready list, it is removed
prior to inserting the new endpoint into the not-ready list.
Signed-off-by: Eliza Weisman <eliza@buoyant.io>
I've implemented `std::error::Error` for the error types in the `tower-balance`, `tower-buffer`, `tower-in-flight-limit`, and `tower-reconnect` middleware crates.
This is required upstream for runconduit/conduit#442, and also just generally seems like the right thing to do as a library.
versions don't have to build both those versions and the older ones
that h2 is currently using.
Don't enable the regex support in env_logger. Applications that want
the regex support can enable it themselves; this will happen
automatically when they add their env_logger dependency.
Disable the env_logger dependency in quickcheck.
The result of this is that there are fewer dependencies. For example,
regex and its dependencies are no longer required at all, as can be
seen by observing the changes to the Cargo.lock. That said,
env_logger 0.5 does add more dependencies itself; however it seems
applications are going to use env_logger 0.5 anyway so this is still
a net gain.
Submitted on behalf of Buoyant, Inc.
Signed-off-by: Brian Smith <brian@briansmith.org>
Provides a middleware that sets a maximum number of requests that can be
in-flight for the service. A request is defined to be in-flight from the
time `call` is invoked to the time the returned response future
resolves.
This maximum is enforced across all clones of the service instance.
Test `Balance::poll_ready` by creating a random number of services, each
of which must be polled a random number of times before becoming ready.
As the balancer is polled, the test ensures that it does not become
ready prematurely and that services are promoted from not_ready to
ready.
`Balance::num_ready()` and `Balance::is_ready()` have been added to
expose the number of ready services, as well as `Balance::num_not_ready()`
to expose the number of pending services.
The new _demo_ example sends a million simulated requests through each
load balancer configuration and records the observed latency
distributions.
Furthermore, this fixes a critical bug in `Balancer`, where we did not
properly iterate through not-ready nodes.
* Use (0..n-1).rev() to iterate from right-to-left
Previously, tower-balance used a fixed round-robin strategy for load
distribution.
This change makes `Balance` generic over its load metric and selection
strategy. The following new traits have been introduced to satisfy this:
- `tower_balance::Load` provides a concrete load metric (i.e. for a service);
- `tower_balance::Choose` provides a strategy for selecting a node;
There are two load balancing configurations supported out-of-the-box:
- `tower_balance::round_robin` provides a load-ignorant round-robin balancer.
- `tower_balance::power_of_two_choices` uses the Power of Two Choices to
distribute requests to the least-loaded node. This should be used in conjunction
with `tower_balance::load::WithPendingRequests` to decorate a `Discover` instance
so that all services it produces implement `Load`.
A dual MIT / Apache 2.0 license does not make any sense. Since the
intent of the original license was to be dual under MIT or Apache 2.0,
restricting to ony MIT is OK.
Currently, `Service` does not provide a mechanism by which it can signal
to the caller that it is at capacity. This commit adds a `poll_ready`
function to the `Service` trait. Callers are able to first check
`poll_ready` before calling `Service::call`.
`poll_ready` is expected to be a hint and will be implemented in a best
effort fashion. It is permitted for a `Service` to return `Ready` from
`poll_ready` and the next invocation of `Service::call` fails.
Oliver Gould
Eliza Weisman
Sean McArthur
Carl Lerche
Brian Smith
Feynman Liang
Eliza Weisman
boats