* Redesign Sapling data model for V5 shared anchor and spends
The shared anchor is only present if there are any spends.
As part of this change, delete the manual PartialEq impl and its tests,
because we can derive PartialEq now.
* Stop creating a temporary Vec for the spend and output iterators
* Rename TransferData variants
Interactive rename using the following commands:
```sh
fastmod Spends SpendsAndMaybeOutputs
fastmod NoSpends JustOutputs
```
* Refactor out common sprout nullifier code
* Implement the AtLeastOne constrained vector type
This vector wrapper ensures that it always contains at least one element.
* Simplify Sapling TransferData using AtLeastOne
Also update the RFC to use AtLeastOne for Orchard.
* build(deps): bump vergen from 3.2.0 to 5.1.1
* fix hardcoded version for Tracing struct
* add additional metadata
* remove extra allocations for metadata
* Remove zebrad code version from release checklist
The zebrad code automatically uses the crate version now.
* Sort panic metadata into rough categories
Co-authored-by: teor <teor@riseup.net>
* serialize/deserialize spaling shielded data in v5 transaction
* fix serialize/deserialize fields order according to spec
* remove extra clone calls
* more serialize fixes
* clippy: fix empty array
* tidy comments
* Add v4 and v5 transaction tests
Also make sure that serialized bytes match if structs match.
* Test fake v5 blocks made out of pre-NU5 block test vectors
* Add outputs-only tests for v5 shared anchor serialization
* Refactor sapling::ShieldedData V5 serialization into its own impl
* Fix spec name typos
* Simplify sapling shielded data parsing
* Delete redundant V5 transaction wrappers in tests
And split out sapling ShieldedData serialization.
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Speedup proptests for Chain struct in zebra-state
* Add teor2345 requested changes
* Fix type for DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES
* More costs for PROPTEST_CASES
* Add functions for serializing and deserializing split arrays
In Transaction::V5, Zcash splits some types into multiple arrays, with a
single prefix count before the first array.
Add utility functions for serializing and deserializing the subsequent
arrays, with a paramater for the original array's length.
* Use zcash_deserialize_bytes_external_count in zebra-network
* Move some preallocate proptests to their own file
And fix the test module structure so it is consistent with the rest of
zebra-chain.
* Add a convenience alias zcash_serialize_external_count
* Explain why u64::MAX items will never be reached
* add sapling shielded data to transaction V5
* implement nullifiers
* test v5 in shielded_data_roundtrip
* Explicitly design serialization for Transaction V5
Implement serialization for V4 and V5 spends and outputs, to make sure
that the design works.
* Test serialization for v5 spends and outputs
Also add a few missing v4 tests.
* Delete a disabled proptest
* Make v5 transactions a top-level heading
And add a missing serialized type.
* Fix a comment typo
* v5 transaction RFC: split array serialization
Based on #2017
* RFC: explicitly describe serialized field order
And link to the spec
* RFC: add the shared anchor serialization rule test
Co-authored-by: teor <teor@riseup.net>
lexical-core 0.7.4 is incompatible with rustc nightly >= ~2021-03-24,
see https://github.com/rust-lang/rust/issues/81654
While we're waiting for zcash_script, rocksdb, and their dependencies to
upgrade, patch lexical-core to 0.7.5 in the zebra workspace.
Zebra avoids having a majority of addresses from a single peer by asking
3 peers for new addresses.
Also update a bunch of security comments and related documentation.
* Make proptest dependencies consistent between chain and network
* Implement Arbitrary for InventoryHash and use it in tests
* Impl Arbitrary for MetaAddr and use it in tests
Also test some extreme times in MetaAddr sanitization.
* Stop ignoring inbound message errors and handshake timeouts
To avoid hangs, Zebra needs to maintain the following invariants in the
handshake and heartbeat code:
- each handshake should run in a separate spawned task
(not yet implemented)
- every message, error, timeout, and shutdown must update the peer address state
- every await that depends on the network must have a timeout
Once the Connection is created, it should handle timeouts.
But we need to handle timeouts during handshake setup.
* Avoid hangs by adding a timeout to the candidate set update
Also increase the fanout from 1 to 2, to increase address diversity.
But only return permanent errors from `CandidateSet::update`, because
the crawler task exits if `update` returns an error.
Also log Peers response errors in the CandidateSet.
* Use the select macro in the crawler to reduce hangs
The `select` function is biased towards its first argument, risking
starvation.
As a side-benefit, this change also makes the code a lot easier to read
and maintain.
* Split CrawlerAction::Demand into separate actions
This refactor makes the code a bit easier to read, at the cost of
sometimes blocking the crawler on `candidates.next()`.
That's ok, because `next` only has a short (< 100 ms) delay. And we're
just about to spawn a separate task for each handshake.
* Spawn a separate task for each handshake
This change avoids deadlocks by letting each handshake make progress
independently.
* Move the dial task into a separate function
This refactor improves readability.
* Fix buggy future::select function usage
And document the correctness of the new code.
* Move the preallocate tests into their own files
And move the MetaAddr proptest into its own file.
Also do some minor formatting and cleanups.
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
* Rename RootHash to Commitment based on ZIP-244
Interactive replace using:
```sh
fastmod RootHash Commitment
fastmod root_hash commitment
fastmod root_bytes commitment_bytes
git mv zebra-chain/src/block/root_hash.rs zebra-chain/src/block/commitment.rs
```
All replacements were accepted.
* rustfmt
* Comment and format cleanups after interactive replace
* Distinguish Sapling tree roots from other tree roots
* Add the NU5 BlockCommitmentsHash variant to block::Commitment
This change parses the hash, but does not perform validation.
* Validate reserved values in Block::commitment
- change Block::commitment to return a Result rather than an Option
- enforce the all-zeroes reserved value consensus rules
- change `PreSaplingReserved([u8; 32])` to `PreSaplingReserved`
- change `ChainHistoryActivationReserved([u8; 32])` to `ChainHistoryActivationReserved`
- update the function comments to describe when each variant is verified
* Fix comment whitespace
* Implement SafePreallocate. Resolves#1880
* Add proptests for SafePreallocate
* Apply suggestions from code review
Comments which did not include replacement code will be addressed in a follow-up commit.
Co-authored-by: teor <teor@riseup.net>
* Rename [Safe-> Trusted]Allocate. Add doc and tests
Add tests to show that the largest allowed vec under TrustedPreallocate
is small enough to fit in a Zcash block/message (depending on type).
Add doc comments to all TrustedPreallocate test cases.
Tighten bounds on max_trusted_alloc for some types.
Note - this commit does NOT include TrustedPreallocate
impls for JoinSplitData, String, and Script.
These impls will be added in a follow up commit
* Implement SafePreallocate. Resolves#1880
* Add proptests for SafePreallocate
* Apply suggestions from code review
Comments which did not include replacement code will be addressed in a follow-up commit.
Co-authored-by: teor <teor@riseup.net>
* Rename [Safe-> Trusted]Allocate. Add doc and tests
Add tests to show that the largest allowed vec under TrustedPreallocate
is small enough to fit in a Zcash block/message (depending on type).
Add doc comments to all TrustedPreallocate test cases.
Tighten bounds on max_trusted_alloc for some types.
Note - this commit does NOT include TrustedPreallocate
impls for JoinSplitData, String, and Script.
These impls will be added in a follow up commit
* Impl TrustedPreallocate for Joinsplit
* Impl ZcashDeserialize for Vec<u8>
* Arbitrary, TrustedPreallocate, Serialize, and tests for Spend<SharedAnchor>
Co-authored-by: teor <teor@riseup.net>
* start refactoring transaction v4 for transaction v5
- move ShieldedData to sapling
- add AnchorVariant
- rename shielded_data to sapling_shielded data in V4
- move value_balance into ShieldedData
- update prop tests for new structure
* add AnchorVariant to Spend
- make anchor types available from sapling crate
- update serialize
* change shielded_balances_match() arguments
* change variable name anchor to shared_anchor in ShieldedData
* fix empty value balance serialization
* use AnchorV in shielded spends
* Rename anchor to per_spend_anchor
* Use nullifiers function directly in non-finalized state
* Use self.value_balance instead of passing it as an argument
* Add missing fields to ShieldedData PartialEq
* Derive Copy for tag types
* Add doc comments for ShieldedData refactor
* Implement a per-spend anchor compatibility iterator
Co-authored-by: teor <teor@riseup.net>
* Ed25519 async batch verification for JoinSplit signatures
We've been verifying JoinSplitSigs one-by-one pre-ZIP-215. Now as we're post-ZIP-215,
we can take advantage of the batch math to validate this signatures.
I would have pumped all the joinsplits in our MAINNET_BLOCKS test vectors but these
signatures are over the sighash, which needs the NU code to compute, and once we're
doing all that set up, we're basically doing transaction validation, so.
Resolves#1944
* Repoint to latest ed25519-zebra commit with note to point at 3.0 when released
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
teor
Kirill Fomichev
dependabot[bot]
Alfredo Garcia
Deirdre Connolly
Preston Evans