In #293, `balance` was refactored to manage dispatching requests over a
set of equivalent inner services that may or may not be ready.
This change extracts the core logic of managing a cache of ready
services into a dedicated crate, leaving the balance crate to deal with
node selection.
* update tower-test to std::future
* refactoring tower-test tests
* everything works
* whoops, un-delete the tower dir
* cleanup & update links
* undo changes to tower-filter for this PR
* use pin_utils::unsafe_pinned
* use tokio-test
The tower-balance crate includes the `Load` and `Instrument` traits,
which are likely useful outside of balancers; and certainly have no
tight coupling with any specific balancer implementation. This change
extracts these protocol-agnostic traits into a dedicated crate.
The `Load` trait includes a latency-aware _PeakEWMA_ load strategy as
well as a simple _PendingRequests_ strategy for latency-agnostic
applications.
The `Instrument` trait is used by both of these strategies to track
in-flight requests without knowing protocol details. It is expected that
protocol-specific crates will provide, for instance, HTTP
time-to-first-byte latency strategies.
A default `NoInstrument` implementation tracks the a request until its
response future is satisfied.
This crate should only be published once tower-balance is published.
Part of https://github.com/tower-rs/tower/issues/286
Some layers cannot guarantee that they will poll inner services in a
timely fashion. For instance, the balancer polls its inner services to
check for readiness, but it does so randomly. If its inner service
must be polled several times to become ready, e.g., because it's driving
the initiation of a TLS connection, then the balancer may not drive the
handshake to completion.
The `SpawnReady` layer ensures that its inner service is driven to
readiness by spawning a background task.
Add tower-hedge, a layer that preemptively retries requests which have been
outstanding for longer than a given latency percentile. If either of the original
future or the retry future completes, that value is used. For more information
about hedge requests, see: [The Tail at Scale][1]
[1]: https://cseweb.ucsd.edu/~gmporter/classes/fa17/cse124/post/schedule/p74-dean.pdf
Signed-off-by: Alex Leong <alex@buoyant.io>
* Consolidate `limit` layers
- `InFlightLimit` and `RateLimit` are moved into `tower-limit` crate.
- `InFlightLimit` is renamed to `ConcurrencyLimit`.
Fixes#225
Provides a middleware that immediately fails any request if the
underlying service isn't ready yet. This is useful when used in
conjunction with `InFlightLimit` or others, so that requests are
rejected immediately, instead of keeping track of extra pending
requests.
This change introduces the new `tower-layer` crate and the foundational `Layer` trait to go along with it. This trait allows one to easily compose a set of `Service`s that take an inner service. These services only modify the request/response. This also provides the `Layer` implementation for many of the tower crates.
This adds the inital base tower crate, as of right now it contains
nothing and is only needed to ensure that cargo workspaces can
properly compile with rust 1.32.
See also rust-lang/rust#57524. Previously, the examples were
never even compiled.
This patch adds the `DirectService` trait, and related implementations
over it in `tower_balance` and `tower_buffer`. `DirectService` is
similar to a `Service`, but must be "driven" through calls to
`poll_service` for the futures returned by `call` to make progress.
The motivation behind adding this trait is that many current `Service`
implementations spawn long-running futures when the service is created,
which then drive the work necessary to turn requests into responses. A
simple example of this is a service that writes requests over a
`TcpStream` and reads responses over that same `TcpStream`. The
underlying stream must be read from to discover new responses, but there
is no single entity to drive that task. The returned futures would share
access to the stream (and worse yet, may get responses out of order),
and then service itself is not guaranteed to see any more calls to it as
the client is waiting for its requests to finish.
`DirectService` solves this by introducing a new method, `poll_service`,
which must be called to make progress on in-progress futures.
Furthermore, like `Future::poll`, `poll_service` must be called whenever
the associated task is notified so that the service can also respect
time-based operations like heartbeats.
The PR includes changes to both `tower_balance::Balance` and
`tower_buffer::Buffer` to add support for wrapping `DirectService`s. For
`Balance` this is straightforward: if the inner service is a `Service`,
the `Balance` also implements `Service`; if the inner service is a
`DirectService`, the `Balance` is itself also a `DirectService`. For
`Buffer`, this is more involved, as a `Buffer` turns any `DirectService`
*into* a `Service`. The `Buffer`'s `Worker` is spawned, and will
therefore drive the wrapped `DirectService`.
One complication arises in that `Buffer<T>` requires that `T: Service`,
but you can safely construct a `Buffer` over a `DirectService` per the
above. `Buffer` works around this by exposing
```rust
impl Service for HandleTo<S> where S: DirectService {}
```
And giving out `Buffer<HandleTo<S>>` when the `new_directed(s: S)`
constructor is invoked. Since `Buffer` never calls any methods on the
service it wraps, `HandleTo`'s implementation just consists of calls to
`unreachable!()`.
Note that `tower_buffer` now also includes a `DirectedService` type,
which is a wrapper around a `Service` that implements `DirectService`.
In theory, we could do away with this by adding a blanket impl:
```rust
impl<T> DirectedService for T where T: Service {}
```
but until we have specialization, this would prevent downstream users
from implementing `DirectService` themselves.
Finally, this also makes `Buffer` use a bounded mpsc channel, which
introduces a new capacity argument to `Buffer::new`.
Fixes#110.
- Inspect the request, response, and error when determining to retry
- Return a future if retry is desired
- This future allows an impl to delay a retry
- The future yields a new `Policy`, allowing state to be changed
for later retries.
`tower_watch::WatchService` provides a dynamically-bound `Service` that
updates in response to a `Watch`. A `WatchService` is constructed with a
`Watch<T>` and a `Bind<T>` -- `Bind` is a newly introduced trait that
supports instantiating new service instances with a borrowed value, i.e.
from a watch.
This can be used to reconfigure Services from a shared or otherwise
externally-controlled configuration source (for instance, a file
system).
versions don't have to build both those versions and the older ones
that h2 is currently using.
Don't enable the regex support in env_logger. Applications that want
the regex support can enable it themselves; this will happen
automatically when they add their env_logger dependency.
Disable the env_logger dependency in quickcheck.
The result of this is that there are fewer dependencies. For example,
regex and its dependencies are no longer required at all, as can be
seen by observing the changes to the Cargo.lock. That said,
env_logger 0.5 does add more dependencies itself; however it seems
applications are going to use env_logger 0.5 anyway so this is still
a net gain.
Submitted on behalf of Buoyant, Inc.
Signed-off-by: Brian Smith <brian@briansmith.org>
Lucio Franco
David Barsky
Oliver Gould
Taiki Endo
Jon Gjengset
John Doneth
Stan Bondi
Alex Leong
Carl Lerche
Sean McArthur
Brian Smith