* checkpoint: reject older of duplicate verification requests.
If we get a duplicate block verification request, we should drop the older one
in favor of the newer one, because the older request is likely to have been
canceled. Previously, this code would accept up to four duplicate verification
requests, then fail all subsequent ones.
* sync: add a timeout layer to block requests.
Note that if this timeout is too short, we'll bring down the peer set in a
retry storm.
* sync: restart syncing on error
Restart the syncing process when an error occurs, rather than ignoring it.
Restarting means we discard all tips and start over with a new block locator,
so we can have another chance to "unstuck" ourselves.
* sync: additional debug info
* sync: handle lookahead limit correctly.
Instead of extracting all the completed task results, the previous code pulled
results out until there were fewer tasks than the lookahead limit, then
stopped. This meant that completed tasks could be left until the limit was
exceeded again. Instead, extract all completed results, and use the number of
pending tasks to decide whether to extend the tip or wait for blocks to finish.
* network: add debug instrumentation to retry policy
* sync: instrument the spawned task
* sync: streamline ObtainTips/ExtendTips logic & tracing
This change does three things:
1. It aligns the implementation of ObtainTips and ExtendTips so that they use
the same deduplication method. This means that when debugging we only have one
deduplication algorithm to focus on.
2. It streamlines the tracing output to not include information already
included in spans. Both obtain_tips and extend_tips have their own spans
attached to the events, so it's not necessary to add Scope: prefixes in
messages.
3. It changes the messages to be focused on reporting the actual
events rather than the interpretation of the events (e.g., "got genesis hash in
response" rather than "peer could not extend tip"). The motivation for this
change is that when debugging, the interpretation of events is already known to
be incorrect, in the sense that the mental model of the code (no bug) does not
match its behavior (has bug), so presenting minimally-interpreted events forces
interpretation relative to the actual code.
* sync: hack to work around zcashd behavior
* sync: localize debug statement in extend_tips
* sync: change algorithm to define tips as pairs of hashes.
This is different enough from the existing description that its comments no
longer apply, so I removed them. A further chunk of work is to change the sync
RFC to document this algorithm.
* sync: reduce block timeout
* state: add resource limits for sled
Closes#888
* sync: add a restart timeout constant
* sync: de-pub constants
Log a message with the height when we get duplicate blocks.
Downgrade some verifier errors and warnings to info and debug, because
some peers on mainnet consistently provide bad blocks.
* make zebra-checkpoints
* fix LOOKAHEAD_LIMIT scope
* add a default cli path
* change doc usage text
* add tracing
* move MAX_CHECKPOINT_HEIGHT_GAP to zebra-consensus
* do byte_reverse_hex in a map
* Add checkpoint list generation scripts
* Limit the checkpoint block data size
* Limit the checkpoint height gap
* Add Mainnet and Testnet checkpoint lists
* Parse hard-coded checkpoint lists
The lists were generated using the following limits:
- 256 MB spacing, based on block byte size, and
- 2000 blocks.
* fix: Resist CheckpointVerifier memory DoS attacks
Allow a maximum of 2 queued blocks at each height, as a tradeoff between
efficient bad block rejection, and memory usage.
Closes#628.
* fix: Make max queued blocks at height equal to fanout
* fix: Just allocate all the capacity upfront
* fix: Use with_capacity(1) and reserve_exact(1)
* Flatten consensus::verify::* to consensus::*
* Move consensus::*::tests into their own files
* Move CheckpointList into its own file
* Move Progress and Target into a types module
QueuedBlock and QueuedBlockList can stay in checkpoint.rs, because
they are tightly coupled to CheckpointVerifier.
* Return Poll::Ready(Err(_)) when verification has finished
* Turn checkpoint::init() into CheckpointVerifier::new()
* Accept IntoIterator<...> for CheckpointVerifier::new()
* Add a CheckpointList type
* Replace the state service with oneshot channels.
* Reject redundant checkpoint blocks
* impl Drop for CheckpointVerifier
* Add fields for caching blocks, and managing verify chains.
* Add current checkpoint functions
* Use a checkpoint range
* Get full backtraces with Err::Try
* Add enums for verification progress and target block heights.
* Replace install_tracing() with zebra_test::init()
* Add a test that mixes good and bad blocks
* Add timeouts to the checkpoint test futures
* Use spandoc correctly
* Refactor consensus test error handling
* Delete a checkpoint test that will soon be obsolete
* Only initialise tracing once for the block tests
* Use tracing in the checkpoint tests
* Move BlockVerifier and tests into block.rs
* Update a BlockVerifier comment
* Tweak some TODO comments
consensus: Add a checkpoint verifier stub
This stub only verifies blocks whose hashes are in the checkpoint
list.
It doesn't have tests, chain child verifies to their ancestors, or
support checkpoint maximum height queries.
Part of #429.
Henry de Valence
teor
Alfredo Garcia
Jane Lusby