* Add a note about v0.3.x branch to the readme
Signed-off-by: Lucio Franco <luciofranco14@gmail.com>
* Fix link
Signed-off-by: Lucio Franco <luciofranco14@gmail.com>
Prior to this change, when `Balance` dropped a failing service, `Pool`
would not be notified of this fact. This meant that it never updated
`.services`, and so it might not add a new backing `Service` (e.g., due
to `max_services`) even though no working backing exist.
With this change, dropped services notify the `Pool` so that it knows to
re-check its limits. It also gains some much-needed tests.
Of particular note is that this change lets spans trace requests through `tower::Buffer` by internally carrying the `Span` at the time of `call` along with the request to the worker.
When using a `ServiceBuilder`, it's not possible to obtain the
underlying `Layer` implementation.
Adding a `ServiceBuilder::into_inner` allows callers to retrieve this
field instead of only being able to build a `Service`.
In 03ec4aa, the balancer was changed to make a quick endpoint decision.
This, however, means that the balancer can return NotReady when it does
in fact have a ready endpoint.
This changes the balancer to separate unready endpoints, only
performing p2c over ready endpoints. Unready endpoints are tracked with
a FuturesUnordered that supports eviction via oneshots.
The main downside of this change is that the Balancer must become
generic over the Request type.
As described in #286, `Balance` had a few problems:
- it is responsible for driving all inner services to readiness, making
its `poll_ready` O(n) and not O(1);
- the `choose` abstraction was a hinderance. If a round-robin balancer
is needed it can be implemented separately without much duplicate
code; and
- endpoint errors were considered fatal to the balancer.
This changes replaces `Balance` with `p2c::Balance` and removes the
`choose` module.
Endpoint service failures now cause the service to be removed from the
balancer gracefully.
Endpoint selection is now effectively constant time, though it biases
for availability in the case when random selection does not yield an
available endpoint.
`tower-test` had to be updated so that a mocked service could fail after
advertising readiness.
The initial implementation of spawn-ready didn't properly compose over
MakeService instances. This introduces `MakeSpawnReady` as a factory
type for SpawnReady, and changes Layer to produce `MakeSpawnReady`
instances.
The tower-balance crate includes the `Load` and `Instrument` traits,
which are likely useful outside of balancers; and certainly have no
tight coupling with any specific balancer implementation. This change
extracts these protocol-agnostic traits into a dedicated crate.
The `Load` trait includes a latency-aware _PeakEWMA_ load strategy as
well as a simple _PendingRequests_ strategy for latency-agnostic
applications.
The `Instrument` trait is used by both of these strategies to track
in-flight requests without knowing protocol details. It is expected that
protocol-specific crates will provide, for instance, HTTP
time-to-first-byte latency strategies.
A default `NoInstrument` implementation tracks the a request until its
response future is satisfied.
This crate should only be published once tower-balance is published.
Part of https://github.com/tower-rs/tower/issues/286
Some layers cannot guarantee that they will poll inner services in a
timely fashion. For instance, the balancer polls its inner services to
check for readiness, but it does so randomly. If its inner service
must be polled several times to become ready, e.g., because it's driving
the initiation of a TLS connection, then the balancer may not drive the
handshake to completion.
The `SpawnReady` layer ensures that its inner service is driven to
readiness by spawning a background task.
* balance: Configure weights from keys, not services
The initial weighted balancing implementation required that the
underlying service implement `HasWeight`.
Practically, this doesn't work that well, since this may force
middlewares to implement this trait as well.
To fix this, we change the type bounds so that _keys_, not services,
must implement `HasWeight`.
This has a drawback, though, in that Weight, which contains a float,
cannot implement `Hash` or `Eq`, which is required by the balancer. This
tradeoff seems manageable, though (and is already addressed in linkerd,
for instance). We should follow-up with a change to alter the internal
representation of Weight to alleviate this.
Add tower-hedge, a layer that preemptively retries requests which have been
outstanding for longer than a given latency percentile. If either of the original
future or the retry future completes, that value is used. For more information
about hedge requests, see: [The Tail at Scale][1]
[1]: https://cseweb.ucsd.edu/~gmporter/classes/fa17/cse124/post/schedule/p74-dean.pdf
Signed-off-by: Alex Leong <alex@buoyant.io>
Creating a buffer can internally fail to spawn a worker. Before, that
error was returned immediately from `Buffer::new`. This changes `new` to
always return a `Buffer`, and the spawn error is encountered via
`poll_ready`.
Lucio Franco
Gabe Jackson
Jon Gjengset
Oliver Gould
Alex Leong
Sean McArthur
Marcus Griep
Carl Lerche
João Oliveira
Jake Shadle
David Barsky