* Integrate JoinSplit verifier with transaction verifier
* Add test with malformed Groth16 Output proof
* Use TryFrom instead of From in ItemWrapper to correctly propagate malformed proof errors
* Simplify by removing ItemWrapper and directly TryFrom into Item
* Fix existing tests to work with JoinSplit validation
* Apply suggestions from code review
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
ZIP-201 describes how a Zcash node should behave when it reaches a
network upgrade activation height. Zebra doesn't implement all the
details specified there, so we need to document what it does implement
and what it doesn't and why.
* Update the State RFC to match the current database format
* Formatting and name fixes
* Remove redundant generic parameter
* Remove redundant generics
* Fix history tree types
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Fix spacing
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Stop useless crawler attempts when there are no peers and no crawl responses
* Disable GitHub bug report URLs when the disk is full
* Add help text for the `zebrad start` tracing filter option
* Do prelim checking of Sprout anchors in non-finalized state
Does not check intra-transaction interstitial states yet
* Populate sprout anchors to allow other state tests to pass
* Preliminary interstitial sprout note commitment tree anchor checks implementation
* Make sure only prior anchors are checked in the same transaction
* Add tests
* Refactor a comment
* Refactor rustdoc
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Use the first `JoinSplit`s from mainnet
* Print debug messages
* Use correct blocks for the tests
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Replace usage of `discover::Change` with a tuple
Remove the assumption that a `Remove` variant would never be created
with type changes that allow the compiler to guarantee that assumption.
* Add a `version` field to the `Client` type
Keep track of the peer's reported protocol version.
* Create `LoadTrackedClient` type
A `peer::Client` type wrapper that implements `Load`. This helps with
the creation of a client service that has extra peer information to be
accessed without having to send requests.
* Use `LoadTrackedClient` in `initialize`
Ensure that `PeerSet` receives `LoadTrackedClient`s so that it will be
able to query the peer's protocol version later on.
* Require `LoadTrackedClient` in `PeerSet`
Replace the generic type with a concrete `LoadTrackedClient` so that we
can query its version.
* Create `MinimumPeerVersion` helper type
A type to track the current minimum protocol version for connected
peers based on the current block height.
* Use `MinimumPeerVersion` in handshakes
Keep the code to obtain the current minimum peer protocol version in a
central place.
* Add a `MinimumPeerVersion` instance to `PeerSet`
Prepare it to be able to disconnect from outdated peers based on the
current minimum supported peer protocol version.
* Disconnect from ready services for outdated peers
When the minimum peer protocol version is detected to have changed
(because of a network upgrade), remove all ready services of peers that
became outdated.
* Cancel added unready services of outdated peers
Only add an unready service if it's for a peer that has a supported
protocol version. Otherwise, add it but drop the cancel handle so that
the `UnreadyService` can execute and detect that it was cancelled.
* Avoid adding ready services for outdated peers
If a service becomes ready but it's for a connection to an outdated
peer, drop it.
* Improve comment inside `crawl_and_dial`
Describe an edge case that is also handled but was not explicit.
Co-authored-by: teor <teor@riseup.net>
* Test if calculated minimum peer version is correct
Given an arbitrary best chain tip height, check that the calculated
minimum peer protocol version is the expected value.
* Test if minimum version changes with chain tip
Apply an arbitrary list of chain tip height updates and check that for
each update the minimum peer version is calculated correctly.
* Test minimum peer version changed reports
Simulate a series of best chain tip height updates, and check for
minimum peer version updates at least once between them. Changes should
only be reported once.
* Create a `MockedClientHandle` helper type
Used to create and then track a mock `Client` instance.
* Add `MinimumPeerVersion::with_mock_chain_tip`
An extension method useful for tests, that contains some shared
boilerplate code.
* Bias arbitrary `Version`s to be in valid range
Give a 50% chance for an arbitrary `Version` to be in the range of
previously used values the Zcash network.
* Create a `PeerVersions` helper type
Helps with the creation of mocked client services with arbitrary
protocol versions.
* Create a `PeerSetGuard` helper type
An auxiliary type to a `PeerSet` instance created for testing. It keeps
track of any dummy endpoints of channels created and passed to the
`PeerSet` instance.
* Create a `PeerSetBuilder` helper type
Helps to reduce the code when preparing a `PeerSet` test instance.
* Test if outdated peers are rejected by `PeerSet`
Simulate a set of discovered peers being sent to the `PeerSet`. Ensure
that only up-to-date peers are kept by the `PeerSet` and that outdated
peers are dropped.
* Create `BlockHeightPairAcrossNetworkUpgrades` type
A helper type that allows the creation of arbitrary block height pairs,
where one value is before and the other is at or after the activation
height of an arbitrary network upgrade.
* Test if peers are dropped as they become outdated
Simulate a network upgrade, and check that peers that become outdated
are dropped by the `PeerSet`.
* Remove dbg! macros
Co-authored-by: teor <teor@riseup.net>
* Use a single-thread shared Tokio runtime
This allows it to pause the time and more closely resembles the
environment that's set by default for asynchronous tests.
* Add a `zebra_test::init_async` helper function
Calls `zebra_test::init` but also constructs a single-thread Tokio
runtime and returns it. This makes it simpler to initialize asynchronous
tests that can't use the `#[tokio::test]` attribute.
* Replace usages of `Runtime::new` in tests
Use the new `zebra_test::init_async()` helper function instead.
* Replace `runtime::Builder::new_current_thread()`
Use the new `zebra_test::init_async()` helper function instead.
* Replace `runtime::Builder::new_multi_thread()`
Use the new `zebra_test::init_async()` helper function instead. The test
with the change doesn't necessarily have to use a multi-thread runtime.
* Refactor the address response limit
* Limit the number of peers in the address book
* Allow changing the address book limit in tests
* Add tests for the address book length limit
* rustfmt
* fix parse_coinbase_height()
* move tests and create test for parse_coinbase_height()
* add a coinbase height round trip prop test
* fix range
Co-authored-by: teor <teor@riseup.net>
* extend examples in test
* add more round trip testing
* extend the range of test
Co-authored-by: teor <teor@riseup.net>
* add test for single byte
Co-authored-by: teor <teor@riseup.net>
* Stop checking the entire AddressBook for each connection attempt
* Stop redundant peer time checks within the address book
* Stop calling `Instant::now` 3 times for each address book update
* Only get the time once each time an address book method is called
* Update outdated comment
* Use an OrderedMap to efficiently store address book peers
* Add address book order tests
Zebra's latest beta continues implementing zero-knowledge proof and note commitment tree validation. In this release, we have finished implementing transaction header, transaction amount, and Zebra-specific NU5 validation. (NU5 mainnet validation is waiting on an `orchard` crate update, and some consensus parameter updates.)
We also fix a number of security issues that could pose a local denial of service risk, or make it easier for an attacker to make a node follow a false chain.
As of this release, Zebra will automatically download and cache the Sprout and Sapling Groth16 circuit parameters. The cache uses around 1 GB of disk space. These cached parameters are shared across all Zebra and `zcashd` instances run by the same user.
See CHANGELOG.md for the full list of changes in this release.
* Tweak a log message
* Only retry failed DNS once, then use the other DNS responses
* Limit broadcasts to half the peers
* Use a longer minimum interval for GetAddr requests
* Reduce the syncer and mempool crawler fanouts
* Stop resetting the mempool twice when it starts up
This spawns two crawlers, which send two fanouts,
so it can use up a lot of peers.
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add Transaction::sprout_joinsplits()
* Add Anchor variants to ValidateContextError
* Make Chain anchor collections pub(crate)
* tracing::instrument several methods in state
* Add contains_*_anchors methods to FinalizedState
* Add check::anchors module and function
* Verify that anchors_refer_to_earlier_treestates in when updating chains in non-finalized state
* Update zebra-state/src/service/check/anchors.rs
Co-authored-by: teor <teor@riseup.net>
* Add anchors() to sapling::ShieldedData
* Add sapling_anchors() to Transaction
* Use Transaction::sapling_anchors() in the anchors_refer_to_earlier_treestates() check
* Whoops, itertools
* Add a comment for improvement
Co-authored-by: teor <teor@riseup.net>
* Add & use a cfg(test) method on FinalizedState to prep test state with anchors to allow other tests to pass contextual checks
* Allow test nullifier checks to pass by populating anchor sets, allowing test anchor checks to pass
* Add mainnet block 419202 and its sapling note commitment tree root to test vectors
* Test sapling anchor verification using the first few Sapling blocks data
* Correct comment
* assert_eq instead of assert(bool)
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Update zebra-state/src/service/non_finalized_state.rs
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* validate non coinbase expiration height
* change var name
* move checks to transaction verifier
* Add variants and debug fields to transaction expiry errors
* Fix a failing existing test
Co-authored-by: teor <teor@riseup.net>
* Download and load Sprout parameters using zcash_proofs
Also update some librustzcash dependencies, to avoid duplicate dependencies.
* Update upstream orchard to avoid a compilation error
* Skip librustzcash batch refactor for now, to avoid compilation errors
* Change the cache ID, so we actually cache Sprout
* Move existing file checks into zcash_proofs
* Add a 1 hour timeout to parameter file downloads
* Give other tasks priority, before spawning the download task
* Update to the latest version of our modified librustzcash fork
* Change the cache key for Sprout
* Add 40 minutes to CI timeouts for occasional sprout downloads
* Update to zcash_proofs with split downloads
* Check file sizes to help debug parameter load failures in zcash_proofs
* Start the second download once the first has finished in zcash_proofs
* Document the parameter download task
* Stop hashing existing files twice
* validate consensus rule: negative fee not allowed
* fix a test TODO
* fix imports
* move import back
* fix panic text
* join consensus rule check code
* match assertion better in tests
* fix test
* fix consensus rule validation
* remove panics
* Delete a TODO
Co-authored-by: teor <teor@riseup.net>
* Rewrite PeerSet comments to split long sentences
* Replace peer set integer indexes with address-based indexes
Also improve documentation and logging.
* Security: Stop using peer addresses to choose inventory routing order
* Minor doc and code cleanups
* Stop re-using a drained HashSet
* Replace used `_cancel` with `cancel`
* Reword a comment
* Replace cloned with copied
* Start network before verifiers
This makes the Groth16 download task start as late as possible.
* Explain why the Groth16 download must happen first
* Speed up Zebra shutdown: skip waiting for the tokio runtime
* Create a `LockTime::unlocked` helper constructor
Returns a `LockTime` that is unlocked at the genesis block.
* Return `Option<LockTime>` from `lock_time` method
Prepare to return `None` for when a transaction has its lock time
disabled.
* Return `None` instead of zero `LockTime`
Because a zero lock time means that the transaction was unlocked at the
genesis block, so it was never actually locked.
* Rephrase zero lock time check comment
Clarify that the check is not redundant, and is necessary for the
genesis transaction.
Co-authored-by: teor <teor@riseup.net>
* Add a `transparent::Input::sequence` getter method
Retrieve a transparent input's sequence number.
* Check if lock time is enabled by a sequence number
Validate the consensus rule that the lock time is only enabled if at
least one transparent input has a value different from `u32::MAX` as its
sequence number.
* Add more Zcash specific details to comment
Explain the Zcash specific lock time behaviors.
Co-authored-by: teor <teor@riseup.net>
* Add `time` field to `Request::Block` variant
The block time to use to check if the transaction was unlocked and
allowed to be included in the block.
* Add `Request::block_time` getter
Returns the block time for the block that owns the transaction being
validated or the current time plus a tolerance for mempool transactions.
* Validate transaction lock times
If they are enabled by a transaction's transparent input sequence
numbers, make sure that they are in the past.
* Add comments with consensus rule parts
Make it easier to map what part of the consensus rule each match arm is
responsible for.
Co-authored-by: teor <teor@riseup.net>
* add testnet test blocks around nu5
* validate coinbase expiration height
* change const name and doc
Co-authored-by: teor <teor@riseup.net>
* change commit location
Co-authored-by: teor <teor@riseup.net>
* use pre Nu5 rules when there is no activation height
* add sapling final root to nu5 test vectors
* fix tests
Co-authored-by: teor <teor@riseup.net>
* Shut down channels and tasks on PeerSet Drop
* Document all the PeerSet fields
* Close the peer set background task handle on shutdown
* Receive background tasks during shutdown
Also, split receiving and polling background tasks into separate methods.
* Move dependency checks to the clippy job
* Split the fake activation heights into their own job
* Fix expected types
* Minimise proptest cases on Windows, macOS, and coverage
We don't expect proptests to fail on different platforms.
* Replace Zcash parameters crates with pre-downloaded local parameter files
* Download Zcash parameters using the `zcashd` script in CI and Docker
* Add a zcash_proofs dependency to zebra-consensus
* Download Sapling parameters using zcash_proofs, rather than fetch-params.sh
* Add a new `zebrad download` subcommand
This command isn't required for nomrmal usage.
But it's useful when testing, or launching multiple Zebra instances.
* Use `zebrad download` in CI to pre-download parameters
* Log a helpful hint if downloading fails
* Allow some duplicate dependencies currently hidden by orchard
* Spawn a separate task to download Groth16 parameters
* Run the parameter download with code coverage
This avoids re-compining Zebra with and without coverage.
* Update Cargo.lock after rebase
* Try to pass `download` as an argument to `zebrad` in coverage CI
* Fix copy and paste comment typos
* Add path and download examples, like zcash_proofs
* Download params in CI just like zcash_proofs does
* Delete a redundant build step
* Implement graceful shutdown for zebrad start
* Send coverage summary to /dev/null when getting the params path
* Use the correct parameters path and download commands in CI
* Explain pre-downloads
* Avoid calling params_folder twice
* Rename parameter types and methods for consistency
```sh
fastmod SaplingParams SaplingParameters zebra*
fastmod Groth16Params Groth16Parameters zebra*
fastmod PARAMS GROTH16_PARAMETERS zebra*
fastmod params_folder directory zebra*
```
And a manual variable name tweak.
* rustfmt
* Remove a redundant coverage step
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Implement graceful shutdown for the peer set
* Use the minimum lookahead limit in acceptance tests
* Enable a doctest that compiles with newly public modules
Conrado Gouvea
Janito Vaqueiro Ferreira Filho
dependabot[bot]
teor
Alfredo Garcia
Deirdre Connolly
Marek
Pili Guerra