* Refactor to create `verify_sprout_shielded_data`
Move the join split verification code into a new
`verify_sprout_shielded_data` helper method that returns an
`AsyncChecks` set.
* Test if signed V4 tx. join splits are accepted
Create a fake V4 transaction with a dummy join split, and sign it
appropriately. Check if the transaction verifier accepts the
transaction.
* Test if unsigned V4 tx. joinsplit data is rejected
Create a fake V4 transaction with a dummy join split. Do NOT sign this
transaction's join split data, and check that the verifier rejects the
transaction.
* Join tests to share Tokio runtime
Otherwise one of the tests might fail incorrectly because of a
limitation in the test environment. `Batch` services spawn a task in the
Tokio runtime, but separate tests can have separate runtimes, so sharing
a `Batch` service can lead to the worker task only being available for
one of the tests.
* Describe how a ZIP-213 rule is implemented in the transaction verifier
* Move the only coinbase-specific check outside the ZIP-213 block
This change isn't required to implement the ZIP-213 rule, but it makes
it easier to identify the specific checks for coinbase transactions.
* Add a note about coinbase in the mempool
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
* README: update known issues
* Add ticket numbers
* Add network ports to README
* Make heading a bit clearer
* Update zebra listener address docs
Explain how Zebra currently uses listener addresses,
after recent changes.
* Update multiple crates to ensure bitvec 0.22.3 is being used and avoid package conflicts
* Add documentation to zebra-chain::sapling to indicate that ZIP-216 rules are enforced by jubjub
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
* Add missing documentation
Document methods to describe what they do and why.
* Create an `AsyncChecks` type alias
Make it simpler to write the `FuturesUnordered` type with boxed futures.
This will also end up being used more when refactoring to return the
checks so that the `call` method can wait on them.
* Create `verify_transparent_inputs_and_outputs`
Refactors the verification of the transparent inputs and outputs into a
separate method.
* Refactor transparent checks to use `call_all`
Instead of pushing the verifications into a stream of unordered futures,
use the `ServiceExt::call_all` method to build an equivalent stream
after building a stream of requests.
* Replace `CallAll` with `FuturesUnordered`
Make it more consistent with the rest of the code, and make sure that
the `len()` method is available to use for tracing.
Co-authored-by: teor <teor@riseup.net>
* Refactor to move wait for checks into a new method
Allow the code snipped to be reused by other transaction
version-specific check methods.
* Verify transparent inputs in V5 transactions
Use the script verifier to check the transparent inputs in a V5
transaction.
* Check `has_inputs_and_outputs` for all versions
Check if a transaction has inputs and outputs, independently of the
transaction version.
* Wait for checks in `call` method
Refactor to move the repeated code into the `call` method. Now the
validation methods return the set of asynchronous checks to wait for.
* Add helper function to mock transparent transfers
Creates a fake source UTXO, and then the input and output that represent
spending that UTXO. The initial UTXO can be configured to have a script
that either accepts or rejects any spend attempt.
* Test if transparent V4 transaction is accepted
Create a fake V4 transaction that includes a fake transparent transfer
of funds. The transfer uses a script to allow any UTXO to spend it.
* Test transaction V4 rejection based on script
Create a fake transparent transfer where the source UTXO has a script
that rejects spending. The script verifier should not accept this
transaction.
* Test if transparent V5 transaction is accepted
Create a mock V5 transaction that includes a transparent transfer of
funds. The transaction should be accepted by the verifier.
* Test transaction V5 rejection based on script
Create a fake transparent transfer where the source UTXO has a script
that rejects spending. The script verifier should not accept this
transaction.
* Update `Request::upgrade` getter documentation
Simplify it so that it won't become updated when #1683 is fixed.
Co-authored-by: teor <teor@riseup.net>
* Gossip dynamically allocated listener ports to peers
Previously, Zebra would either gossip port `0`, which is invalid, or skip
gossiping its own dynamically allocated listener port.
* Improve "no configured peers" warning
And downgrade from error to warning, because inbound-only nodes are a
valid use case.
* Move random_known_port to zebra-test
* Add tests for dynamic local listener ports and the AddressBook
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Add new CHANGELOG.md file to zebra git repo
* Update Release Checklist to add updates to CHANGELOG.md
* Add some explanation about the CHANGELOG.md file
* Fix headings to make them consistent with Keep a changelog format
* Small fix for clarity
* Add release dates to changelog
* Change order of steps to update the changelog
* Always send our local listener with the latest time
Previously, whenever there was an inbound request for peers, we would
clone the address book and update it with the local listener.
This had two impacts:
- the listener could conflict with an existing entry,
rather than unconditionally replacing it, and
- the listener was briefly included in the address book metrics.
As a side-effect, this change also makes sanitization slightly faster,
because it avoids some useless peer filtering and sorting.
* Skip listeners that are not valid for outbound connections
* Filter sanitized addresses Zebra based on address state
This fix correctly prevents Zebra gossiping client addresses to peers,
but still keeps the client in the address book to avoid reconnections.
* Add a full set of DateTime32 and Duration32 calculation methods
* Refactor sanitize to use the new DateTime32/Duration32 methods
* Security: Use canonical SocketAddrs to avoid duplicate connections
If we allow multiple variants for each peer address, we can make multiple
connections to that peer.
Also make sure sanitized MetaAddrs are valid for outbound connections.
* Test that address books contain the local listener address
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* move network_upgrade check into zebra-chain
* fix the errors
* rename function
* typo fix
* rename the check function
* make changes from last code review
* Security: Limit reconnection rate to individual peers
Reconnection Rate
Limit the reconnection rate to each individual peer by applying the
liveness cutoff to the attempt, responded, and failure time fields.
If any field is recent, the peer is skipped.
The new liveness cutoff skips any peers that have recently been attempted
or failed. (Previously, the liveness check was only applied if the peer
was in the `Responded` state, which could lead to repeated retries of
`Failed` peers, particularly in small address books.)
Reconnection Order
Zebra prefers more useful peer states, then the earliest attempted,
failed, and responded times, then the most recent gossiped last seen
times.
Before this change, Zebra took the most recent time in all the peer time
fields, and used that time for liveness and ordering. This led to
confusion between trusted and untrusted data, and success and failure
times.
Unlike the previous order, the new order:
- tries all peers in each state, before re-trying any peer in that state,
and
- only checks the the gossiped untrusted last seen time
if all other times are equal.
* Preserve the later time if changes arrive out of order
* Update CandidateSet::next documentation
* Update CandidateSet state diagram
* Fix variant names in comments
* Explain why timestamps can be left out of MetaAddrChanges
* Add a simple test for the individual peer retry limit
* Only generate valid Arbitrary PeerServices values
* Add an individual peer retry limit AddressBook and CandidateSet test
* Stop deleting recently live addresses from the address book
If we delete recently live addresses from the address book, we can get a
new entry for them, and reconnect too rapidly.
* Rename functions to match similar tokio API
* Fix docs for service sorting
* Clarify a comment
* Cleanup a variable and comments
* Remove blank lines in the CandidateSet state diagram
* Add a multi-peer proptest that checks outbound attempt fairness
* Fix a comment typo
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Simplify time maths in MetaAddr
* Create a Duration32 type to simplify calculations and comparisons
* Rename variables for clarity
* Split a string constant into multiple lines
* Make constants match rustdoc order
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Security: stop gossiping failure and attempt times as last_seen times
Previously, Zebra had a single time field for peer addresses, which was
updated every time a peer was attempted, sent a message, or failed.
This is a security issue, because the `last_seen` time should be
"the last time [a peer] connected to that node", so that
"nodes can use the time field to avoid relaying old 'addr' messages".
So Zebra was sending incorrect peer information to other nodes.
As part of this change, we split the `last_seen` time into the
following fields:
- untrusted_last_seen: gossiped from other peers
- last_response: time we got a response from a directly connected peer
- last_attempt: time we attempted to connect to a peer
- last_failure: time a connection with a peer failed
* Implement Arbitrary and strategies for MetaAddrChange
Also replace the MetaAddr Arbitrary impl with a derive.
* Write proptests for MetaAddr and MetaAddrChange
MetaAddr:
- the only times that get included in serialized MetaAddrs are
the untrusted last seen and responded times
MetaAddrChange:
- the untrusted last seen time is never updated
- the services are only updated if there has been a handshake
* stop panicking on invalid orchard nullifiers
* add context to error
* use `from_bytes_wide` for nullifiers in arbitrary
* orchard::Nullifier vec to array conversion is a bit clearer and simpler
Co-authored-by: teor <teor@riseup.net>
* Add a `Transaction::version` getter
Returns the version of the transaction as a `u32`.
* Add `Transaction::is_overwintered` helper method
Returns if the `fOverwintered` flag should be set for the transaction's
version.
* Use new helpers to serialize transaction version
Reduce the repeated code and make it less error-prone with future
changes.
* Add getter methods to `transaction::Request` type
Refactor to move the type deconstruction code into the `Request` type.
The main objective is to make it easier to split the call handler into
methods that receive the request directly.
* Refactor to create `verify_v4_transaction` helper
Split the code specific to V4 transactions into a separate helper
method.
* Create `verify_v5_transaction` helper method
Prepare a separate method to have the validation code.
* Add `UnsupportedByNetworkUpgrade` error variant
An error for when a transaction's version isn't supported by the network
upgrade of the block it's included or for the current network upgrade if
the transaction is for the mempool.
* Verify a V5 transaction's network upgrade
For now, only NU5 supports V5 transactions.
* Test that V5 transaction is rejected on Canopy
Create a fake V5 transaction and try to verify it using a block height
from Canopy's activation. The verifier should reject the transaction
with an error saying that the network upgrade does not support that
transaction version.
* Test if V5 tx. is accepted after NU5 activation
Create a fake V5 transaction and pretend it is placed in a block that
has a height after the NU5 activation. The test should succeed, but
since the NU5 activation height has not been specified yet (neither for
the testnet nor the mainnet), for now this test is marked as
`should_panic`.
* Add `TODO` comment to the code
Add more detail to what's left to do, and link to the appropriate PRs.
* Use `u32` to store transaction version
Use a type consistent with how the version is specified.
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: teor <teor@riseup.net>
* Only advance the outbound connection timer when it returns an address
Previously, we were advancing the timer even when we returned `None`.
This created large wait times when there were no eligible peers.
* Refactor to avoid overlapping sleep timers
* Add a maximum next peer delay test
Also refactor peer numbers into constants.
* Make the number of proptests overridable by the standard env var
Also cleanup the test constants.
* Test that skipping peer connections also skips their rate limits
* Allow an extra second after each sleep on loaded machines
macOS VMs seem to need this extra time to pass their tests.
* Restart test time bounds from the current time
This change avoids test failures due to cumulative errors.
Also use a single call to `Instant::now` for each test round.
And print the times when the tests fail.
* Stop generating invalid outbound peers in proptests
The candidate set proptests will fail if enough generated peers are
invalid for outbound connections.
Janito Vaqueiro Ferreira Filho
teor
Conrado Gouvea
Deirdre Connolly
Pili Guerra
Alfredo Garcia
dependabot[bot]
Conrado P. L. Gouvea
Deirdre Connolly