2020-07-30 09:52:47 -07:00
|
|
|
# RiP Curl: low-latency, transaction-oriented RPC
|
|
|
|
|
|
|
|
## Problem
|
|
|
|
|
|
|
|
Solana's initial RPC implementation was created for the purpose of allowing
|
|
|
|
users to confirm transactions that had just recently been sent to the cluster.
|
|
|
|
It was designed with memory usage in mind such that any validator should be
|
|
|
|
able to support the API without concern of DoS attacks.
|
|
|
|
|
|
|
|
Later down the line, it became desirable to use that same API to support the
|
|
|
|
Solana explorer. The original design only supported minutes of history, so we
|
|
|
|
changed it to instead store transaction statuses in a local RocksDB instance
|
|
|
|
and offer days of history. We then extended that to 6 months via BigTable.
|
|
|
|
|
|
|
|
With each modification, the API became more suitable for applications serving
|
|
|
|
static content and less appealing for transaction processing. The clients poll
|
|
|
|
for transaction status instead of being notified, giving the false impression
|
|
|
|
of higher confirmation times. Furthermore, what clients can poll for is
|
|
|
|
limited, preventing them from making reasonable real-time decisions, such as
|
|
|
|
recognizing a transaction is confirmed as soon as particular, trusted
|
|
|
|
validators vote on it.
|
|
|
|
|
|
|
|
## Proposed Solution
|
|
|
|
|
|
|
|
A web-friendly, transaction-oriented, streaming API built around the
|
|
|
|
validator's ReplayStage.
|
|
|
|
|
|
|
|
Improved client experience:
|
|
|
|
|
2021-04-30 01:20:56 -07:00
|
|
|
- Support connections directly from WebAssembly apps.
|
|
|
|
- Clients can be notified of confirmation progress in real-time, including votes
|
2020-07-30 09:52:47 -07:00
|
|
|
and voter stake weight.
|
2021-04-30 01:20:56 -07:00
|
|
|
- Clients can be notified when the heaviest fork changes, if it affects the
|
2020-07-30 09:52:47 -07:00
|
|
|
transactions confirmation count.
|
|
|
|
|
|
|
|
Easier for validators to support:
|
|
|
|
|
2021-04-30 01:20:56 -07:00
|
|
|
- Each validator supports some number of concurrent connections and otherwise
|
2020-07-30 09:52:47 -07:00
|
|
|
has no significant resource constraints.
|
2021-04-30 01:20:56 -07:00
|
|
|
- Transaction status is never stored in memory and cannot be polled for.
|
|
|
|
- Signatures are only stored in memory until the desired commitment level or
|
2020-07-30 09:52:47 -07:00
|
|
|
until the blockhash expires, which ever is later.
|
|
|
|
|
|
|
|
How it works:
|
|
|
|
|
|
|
|
1. The client connects to a validator using a reliable communication channel,
|
|
|
|
such as a web socket.
|
|
|
|
2. The validator registers the signature with ReplayStage.
|
|
|
|
3. The validator sends the transaction into the Gulf Stream and retries all
|
|
|
|
known forks until the blockhash expires (not until the transaction is
|
|
|
|
accepted on only the heaviest fork). If the blockhash expires, the
|
|
|
|
signature is unregistered, the client is notified, and connection is closed.
|
|
|
|
4. As ReplayStage detects events affecting the transaction's status, it
|
|
|
|
notifies the client in real-time.
|
|
|
|
5. After confirmation that the transaction is rooted (`CommitmentLevel::Max`),
|
|
|
|
the signature is unregistered and the server closes the upstream channel.
|