* Begin work on RFC5 implementation
* I think this is necessary
* holy shit supertrait implemented via subtrait
* implement most of the chain functions
* change to slightly better name
* implement fork
* fix outpoint handling in Chain struct
* update expect for work
* resolve review comment
* split utxo into two sets
* update the Chain definition
* just a little more
* update comment
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* apply changes from code review
* remove allow attribute in zebra-state/lib.rs
* Update zebra-state/src/memory_state.rs
Co-authored-by: teor <teor@riseup.net>
* merge ChainSet type into MemoryState
* rename state impl types
* Add error messages to asserts
* checkpoint so I can split off arbitrary changes into a PR
* export proptest impls for use in downstream crates
* add testjob for disabled feature in zebra-chain
* run rustfmt
* try to fix github actions syntax
* differentiate name
* prove that github action tests zebra-chain build without features
* revert change from last commit now that test is running
* remove accidentally introduced newline
* checkpoint
* add module doc comment
* update RFC for utxos
* add missing header
* working proptest for Chain
* apply change from chain impl PR
* setup config for proptests
* Update zebra-chain/src/block/arbitrary.rs
Co-authored-by: teor <teor@riseup.net>
* run rustfmt
Co-authored-by: teor <teor@riseup.net>
* Send the final checkpoint block to the checkpoint verifier
Also:
* route blocks with no height to the block verifier
* update an incorrect comment
* Add missing {
* rustfmt
To avoid confusion with TESTNET_BLOCKS.
Automated search and replace, made using this script:
sed -i 's/TEST_BLOCKS/BLOCKS/' \
$(grep -r TEST_BLOCKS zebra* | cut -d: -f1 | sort -u)
* Reject CommitBlock with pre-sapling blocks: they must use CommitFinalizedBlock
* Clarify adding a new Chain to an empty ChainSet
* Handle duplicate blocks correctly
* Move block test vectors into separate files
* Refactor block test vectors into their own module
* Remove some duplicate test vector data
* Add mainnet block test vectors for each network upgrade
* Add testnet block test vectors for each network upgrade
* Update the TEST_BLOCKS list
* Add basic tests using TEST_BLOCKS
* Ignore shell.nix
* Begin work on RFC5 implementation
* I think this is necessary
* holy shit supertrait implemented via subtrait
* implement most of the chain functions
* change to slightly better name
* implement fork
* fix outpoint handling in Chain struct
* update expect for work
* resolve review comment
* split utxo into two sets
* update the Chain definition
* just a little more
* update comment
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* apply changes from code review
* remove allow attribute in zebra-state/lib.rs
* Update zebra-state/src/memory_state.rs
Co-authored-by: teor <teor@riseup.net>
* merge ChainSet type into MemoryState
* rename state impl types
* Add error messages to asserts
* add module doc comment
* update RFC for utxos
* add missing header
Co-authored-by: teor <teor@riseup.net>
* export proptest impls for use in downstream crates
* add testjob for disabled feature in zebra-chain
* run rustfmt
* try to fix github actions syntax
* differentiate name
* prove that github action tests zebra-chain build without features
* revert change from last commit now that test is running
* remove accidentally introduced newline
* Update .github/workflows/ci.yml
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
This addresses at least three pain points:
- we were affected by bugs that were already fixed in git, but not in
the released crate;
- we can use service combinators to transform requests and responses;
- we can use the hedge middleware.
The version in git is still marked as 0.3.1 but these changes will be
part of tower 0.4: https://github.com/tower-rs/tower/issues/431
* wip
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* wip2: add nullifiers
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* Update book/src/dev/rfcs/0003-state-updates.md
Co-authored-by: teor <teor@riseup.net>
* Move to RFC number 5
* rfc: add PR link to state update RFC
* rfc: change state RFC to store blocks by height.
The rationale for this change is described in the document: it means
that we write blocks only to one end of the Sled tree, and hopefully
helps us with spatial access patterns.
This should help alleviate a major cause of memory use in Zebra's
current WIP Sled structure, which is that:
- blocks are stored in random, sparse order (by hash) in the B-tree;
- the `Request::GetDepth` method opens the entire block store and
queries a random part of its block data to determine whether a hash is
present;
- if present, it deserializes the complete block data of both the given
block and the current tip block, to compute the difference in block
heights.
This access pattern forces a large amount of B-tree data to remain
resident, and could probably be avoided if we didn't do that.
* rfc: add sprout and sapling anchors to sled trees.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* rfc: fill in details of state service requests.
* rfc: extract commit process from API description
* rfc: add anchor parameters to CommitBlock.
These have to be computed by a verifier, so passing them as parameters
means we don't recompute them.
* WIP for in memory state structs
* tweeks from end of session with henry
* more updates from pairing
* rewrite non-finalized state sections
* update query instructions for each request
* more updates
* updates from pairing with henry
* updates from proofreading solo
* add guide level explanation to state rfc
* add drawbacks section
* Update book/src/dev/rfcs/0005-state-updates.md
Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>
* Apply suggestions from code review
Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>
* Update book/src/dev/rfcs/0005-state-updates.md
Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>
* apply changes from code review
* clarify iteration
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* apply changes from code review
* Update book/src/dev/rfcs/0005-state-updates.md
Co-authored-by: teor <teor@riseup.net>
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Apply suggestions from code review
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* add info about default constructing chains when forking from finalized state
* Update book/src/dev/rfcs/0005-state-updates.md
Co-authored-by: teor <teor@riseup.net>
* move contextual verification out of Chain
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Jane Lusby <jane@zfnd.org>
* refactor block and tx validation errors
* rename errors module to error
* move NoTransactions to BlockError
* clarify some errors, use dbg format for hash in error
* mnake is_coinbase_first return BlockError
* add new error types for each consensus Service
Co-authored-by: Jane Lusby <jane@zfnd.org>
Using a Buffer with size 1 is a footgun because it allows only one
sender to call poll_ready at a time. This is usually undesirable
because it means that a task or service that calls poll_ready but only
makes a service call later (potentially much later) will block all other
callers.
The GetPeers requests sent while crawling the network are randomly
load-balanced over available peers. But at the very beginning, they may
be both routed to the same peer, causing network initialization to be
delayed while the second one times out (since zcashd only ever responds
to the first addr message).
Only sending one GetPeers request per candidate set update means we
crawl the network a little more slowly, but avoids hanging on start.
The Inbound service only needs the network setup for some requests, but
it can service other requests without it. Making it return
Poll::Pending until the network setup finishes means that initial
network connections may view the Inbound service as overloaded and
attempt to load-shed.
This cleans up the response processing logic a little bit along the way,
but the overall division of responsibility should be better documented
in a future commit.
This lets us distinguish between cases where the message was unsupported
(e.g., BIP11 messages), and cases where the message was uninterpretable
in context (e.g., unsolicited messages).
The original version of this commit ran into
https://github.com/rust-lang/rust/issues/64552
again. Thanks to @yaahc for suggesting a workaround (using futures combinators
to avoid writing an async block).
Remove the seed command entirely, and make the behavior it provided
(responding to `Request::Peers`) part of the ordinary functioning of the
start command.
The new `Inbound` service should be expanded to handle all request
types.
Deirdre Connolly
Dimitris Apostolou
teor
dependabot[bot]
Jane Lusby
Henry de Valence
Alfredo Garcia