tower-buffer uses tokio's mpsc channels, not the futures-rs mpsc channels.
Unlike futures-rs mpsc channels, which have capacity n+m, where n is the buffer
size and m is the number of senders, tokio channels always have buffer size n.
This means that the buffer size is shared across all peer set handles.
Thanks to @hawkw for sharing details of the Tokio internals!
Previously, we relied on the owner of the handshake future to drive it to
completion. This meant that there were cases where handshakes might never be
completed, just because nothing was actively polling them.
The CoinbaseData field can only be constructed by the transaction parser, so we
can ensure that a coinbase input is always serializable, as CoinbaseData
instances can't be constructed outside of the parser that maintains the data
size invariant.
The previous outbound peer connection logic got requests to connect to new
peers and processed them one at a time, making single connection attempts
and retrying if the connection attempt failed. This was quite slow, because
many connections fail, and we have to wait for timeouts. Instead, this logic
connects to new peers concurrently (up to 50 at a time).
Because we represent each transaction version as a different variant of the
Transaction enum, we end up in a situation where fields that are common to
different transaction versions are awkward to access, requiring a match
statement with identical match arms.
To fix this, this commit adds the following convenience methods:
* `Transaction::inputs() -> impl Iterator<Item=&TransparentInput>`;
* `Transaction::outputs() -> impl Iterator<Item=&TransparentOutput>`;
* `Transaction::lock_time() -> LockTime`;
* `Transaction::expiry_height() -> Option<ExpiryHeight>`;
The last returns an `Option` because the field is only present in V3 and V4
transactions.
There are some remaining fields that do not get common accessors, because it
probably doesn't make sense to access independently of knowing the transaction
version: `joinsplit_data`, `shielded_data`, `value_balance`.
Bitcoin does this either with `getblocks` (returns up to 500 following block
hashes) or `getheaders` (returns up to 2000 following block headers, not
just hashes). However, Bitcoin headers are much smaller than Zcash
headers, which contain a giant Equihash solution block, and many Zcash
blocks don't have many transactions in them, so the block header is
often similarly sized to the block itself. Because we're
aiming to have a highly parallel network layer, it seems better to use
`getblocks` to implement `FindBlocks` (which is necessarily sequential)
and parallelize the processing of the block downloads.
BIP34, which is included in Zcash, encodes the block height into each
block by adding it into the unused BitcoinScript field of the block's
coinbase transaction. However, this is done just by requiring that the
script pushes the block height onto the stack when it executes, and
there are multiple different ways to push data onto the stack in
BitcoinScript. Also, the genesis block does not include the block
height, by accident.
Because we want to *parse* transactions into an algebraic data type that
encodes their structural properties, rather than allow possibly-invalid
data to float through the internals of our node, we want to extract the
block height upfront and store it separately from the rest of the
coinbase data, which is inert. So the serialization code now contains
just enough logic to parse BitcoinScript-encoded block heights, and
special-case the encoding of the genesis block.
Elsewhere in the source code, the `LockTime` struct requires that we
must use block heights less than 500,000,000 (above which the number is
interpreted as a unix timestamp, not a height). To unify invariants, we
ensure that the parsing logic works with block heights up to
500,000,000, even though these are unlikely to ever be used for Zcash.
Coinbase inputs are handled differently from other inputs and have
different consensus rules, so they should be represented differently in
the source code. This lets us discard extraneous details (for instance,
it's not necessary to maintain the all-zero hash) and specialize logic.
This doesn't clean the warnings about unused items in the builder, since
those are unused for a reason (the implementation that should use them
is missing).
Henry de Valence
Deirdre Connolly
dependabot-preview[bot]