Commit Graph

119 Commits

Author SHA1 Message Date
Alfredo Garcia 8539302ad8 rename checks to check 2020-08-18 11:03:40 -07:00
Alfredo Garcia 4dac4d4df7 move `is_coinbase_first` from chain to consensus 2020-08-18 11:03:40 -07:00
Henry de Valence 2712c4b72a chain: rename BlockHeader to block::Header 2020-08-17 11:46:34 -07:00
Henry de Valence 103b663c40 chain: rename BlockHeight to block::Height 2020-08-17 11:46:34 -07:00
Henry de Valence 61dea90e2f chain: rename BlockHeaderHash to block::Hash
This is the first in a sequence of changes that change the block:: items
to not include Block as a prefix in their name, in accordance with the
Rust API guidelines.
2020-08-17 11:46:34 -07:00
Henry de Valence 948b067808 chain: move Network, NetworkUpgrade to parameters
Also, avoid using star-imports of the enum variants, which pollutes the
namespace.
2020-08-17 11:46:34 -07:00
Henry de Valence dad6340cd3 chain: move BlockHeight into block 2020-08-17 11:46:34 -07:00
teor 82b0db3b41 fix: Rename ChainVerifier test variables
We removed the ready checks, but forgot to rename the variables.
2020-08-13 11:28:44 +10:00
Henry de Valence a79ce97957
Fix sync algorithm. (#887)
* checkpoint: reject older of duplicate verification requests.

If we get a duplicate block verification request, we should drop the older one
in favor of the newer one, because the older request is likely to have been
canceled.  Previously, this code would accept up to four duplicate verification
requests, then fail all subsequent ones.

* sync: add a timeout layer to block requests.

Note that if this timeout is too short, we'll bring down the peer set in a
retry storm.

* sync: restart syncing on error

Restart the syncing process when an error occurs, rather than ignoring it.
Restarting means we discard all tips and start over with a new block locator,
so we can have another chance to "unstuck" ourselves.

* sync: additional debug info

* sync: handle lookahead limit correctly.

Instead of extracting all the completed task results, the previous code pulled
results out until there were fewer tasks than the lookahead limit, then
stopped.  This meant that completed tasks could be left until the limit was
exceeded again.  Instead, extract all completed results, and use the number of
pending tasks to decide whether to extend the tip or wait for blocks to finish.

* network: add debug instrumentation to retry policy

* sync: instrument the spawned task

* sync: streamline ObtainTips/ExtendTips logic & tracing

This change does three things:

1.  It aligns the implementation of ObtainTips and ExtendTips so that they use
the same deduplication method.  This means that when debugging we only have one
deduplication algorithm to focus on.

2.  It streamlines the tracing output to not include information already
included in spans. Both obtain_tips and extend_tips have their own spans
attached to the events, so it's not necessary to add Scope: prefixes in
messages.

3.  It changes the messages to be focused on reporting the actual
events rather than the interpretation of the events (e.g., "got genesis hash in
response" rather than "peer could not extend tip").  The motivation for this
change is that when debugging, the interpretation of events is already known to
be incorrect, in the sense that the mental model of the code (no bug) does not
match its behavior (has bug), so presenting minimally-interpreted events forces
interpretation relative to the actual code.

* sync: hack to work around zcashd behavior

* sync: localize debug statement in extend_tips

* sync: change algorithm to define tips as pairs of hashes.

This is different enough from the existing description that its comments no
longer apply, so I removed them.  A further chunk of work is to change the sync
RFC to document this algorithm.

* sync: reduce block timeout

* state: add resource limits for sled

Closes #888

* sync: add a restart timeout constant

* sync: de-pub constants
2020-08-12 16:48:01 -07:00
Alfredo Garcia 0aea09c1ac
Use transcript in consensus chain tests (#858)
* change several tests to transcript in consensus chain tests

* rename transcripts

* rename state transcript

* fix spandocs

* add timeout layer to tests

* run transcripts on the wrapped timeout service, remove ready calls
2020-08-12 16:28:30 -07:00
dependabot[bot] 945b019739
build(deps): bump tracing-subscriber from 0.2.10 to 0.2.11 (#873)
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.10 to 0.2.11.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.10...tracing-subscriber-0.2.11)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-08-11 10:30:50 -07:00
dependabot[bot] 5cb61136c9
build(deps): bump tracing from 0.1.18 to 0.1.19 (#872)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.18 to 0.1.19.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.18...tracing-0.1.19)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-08-11 10:18:54 -07:00
teor ee6f0de14d refactor: Move NetworkUpgrade to zebra-chain 2020-08-10 18:54:42 +10:00
teor 65c3c503d5 fix: Improve verifier diagnostics
Log a message with the height when we get duplicate blocks.

Downgrade some verifier errors and warnings to info and debug, because
some peers on mainnet consistently provide bad blocks.
2020-08-10 15:12:08 +10:00
teor 62225653f0 fix: Reduce the previous block waiting time
Reduce the amount of time that the block verifier waits for the previous
block, before polling again.

(Waiting for 2 seconds resulted in some apparent block verifier hangs.)

This is a temporary fix, until the state layer handles context checks.
2020-08-07 13:28:02 -04:00
Henry de Valence 6c379be7f2 consensus: temporarily allow dead code in zebra_consensus::primitives. 2020-08-05 21:28:55 -07:00
Henry de Valence 2efb4eb262 consensus: add a static redjubjub::VERIFIER with fallback 2020-08-05 21:28:55 -07:00
Henry de Valence 8d4f154c9f consensus: move redjubjub to new primitives module
This will let it sit alongside a groth16 verifier.
2020-08-05 21:28:55 -07:00
Alfredo Garcia ded273413a
Use transcripts in block verify tests (#675)
* change consensus block tests to transcripts

* fix lints

Co-authored-by: Jane Lusby <jlusby42@gmail.com>

Co-authored-by: Jane Lusby <jlusby42@gmail.com>
2020-08-05 14:41:41 -07:00
teor ffb6aacf33 feature: Check difficulty in BlockVerifier 2020-08-04 12:29:07 +10:00
dependabot[bot] 8e268150a7 build(deps): bump tracing-subscriber from 0.2.9 to 0.2.10
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.9 to 0.2.10.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.9...tracing-subscriber-0.2.10)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-03 21:11:50 -04:00
dependabot[bot] f3b65f9609 build(deps): bump tracing from 0.1.17 to 0.1.18
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.17 to 0.1.18.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.17...tracing-0.1.18)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-03 15:49:30 -04:00
teor d4d1edad5a
fix: Use types to avoid ChainVerifier inconsistencies (#797) 2020-07-31 10:21:20 +10:00
teor e74ff18708 refactor: Move the checkpoint list tests to their own file 2020-07-31 07:21:43 +10:00
teor 5b9703f95a fix: Avoid creating a CheckpointVerifier if it is not needed
Also:
* make sure the checkpoints cover all pre-Sapling blocks
* do some minor refactoring to reduce duplicate code
2020-07-30 13:30:18 +10:00
teor fbeae99328 fix: Only warn on unexpected high blocks
High blocks are expected when we restart with most of the chain in our
state.

Also downgrade the message from warn to info.
2020-07-30 13:30:18 +10:00
teor cf0fd043d4 feature: Update mainnet and testnet checkpoints 2020-07-28 22:27:49 -04:00
teor 396d209638 feature: Add the MinimumDifficulty consensus parameter 2020-07-28 10:34:41 +10:00
teor 1e787aecb9 feature: Check the previous block height in BlockVerifier
This is a temporary busy-waiting fix.
2020-07-27 14:14:44 -04:00
teor 2b1e7162b1 feature: Set the testnet Canopy upgrade height
See ZIP 251 and zcash/zips#385 for details.

Closes #756.
2020-07-27 14:03:14 -04:00
teor 993532b604 feature: Add a "Genesis" network upgrade
We can use this network upgrade to implement different consensus rules
and chain context handling for genesis blocks.

Part of the chain state design in #682.
2020-07-27 14:03:14 -04:00
teor 2d3926e800 fix: Replace a return with unreachable! 2020-07-27 13:49:11 -04:00
Alfredo Garcia 5b3c6e4c6c
Port bash checkpoint scripts to zebra-checkpoints single rust binary (#740)
* make zebra-checkpoints
* fix LOOKAHEAD_LIMIT scope
* add a default cli path
* change doc usage text
* add tracing
* move MAX_CHECKPOINT_HEIGHT_GAP to zebra-consensus
* do byte_reverse_hex in a map
2020-07-25 17:53:00 +10:00
Henry de Valence 4aa00ad216 Align crate versions and user-agent with NU numbers.
We had a brief discussion on discord and it seemed like we had consensus on the
following versioning policy:

* zebrad: match major version to NU version, so we will start by releasing
  zebrad 3.0.0;

* zebra-* libraries: start by matching zebrad's version, then increment major
  versions of each library as we need to make breaking changes (potentially
  faster than the zebrad version, always respecting semver but making no
  guarantees about the longevity of major releases).

This commit sets all of the crate versions to 3.0.0-alpha.0 -- the -alpha.0
marks it as a prerelease not subject to perfect adherence to compatibility
guarantees.
2020-07-24 11:46:37 -07:00
dependabot[bot] f7c59c99b5 build(deps): bump tracing-subscriber from 0.2.8 to 0.2.9
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.8 to 0.2.9.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.8...tracing-subscriber-0.2.9)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-24 14:31:44 -04:00
teor 2acfcf3a90
Make the CheckpointVerifier handle partial restarts (#736)
Also put generic bounds on the BlockVerifier struct,
so we get better compilation errors.
2020-07-24 11:47:48 +10:00
teor 77a1fefa1e
Download genesis (#731)
* feature: Add more CheckpointVerifier tracing

* fix: Download the genesis block
2020-07-23 10:56:52 -07:00
dependabot[bot] 909b74a613
build(deps): bump tracing from 0.1.16 to 0.1.17 (#734)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.16 to 0.1.17.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.16...tracing-0.1.17)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-07-23 10:54:56 -07:00
Jane Lusby c1a1493159
use dirs crate for default location of state and config (#714)
* use dirs crate for default location of state and config
* panic if a path isn't specified for zebra-state
2020-07-23 21:12:20 +10:00
teor 132875198e fix: Simplify ChainVerifier 2020-07-23 03:46:24 -04:00
teor 0e0c36c010 fixup! feature: Add network upgrade activation heights 2020-07-23 15:52:18 +10:00
teor 4b683ea2b1 feature: Add Consensus Branch Ids 2020-07-23 15:52:18 +10:00
teor 359febb2e2 doc: Update a CheckpointList comment 2020-07-23 15:52:18 +10:00
teor c9ee85c3b5 feature: Add network upgrade activation heights 2020-07-23 15:52:18 +10:00
Henry de Valence 4a98b8fa0d Add basic metrics to the syncer. 2020-07-22 21:59:00 -07:00
Henry de Valence c2c2a28e8b Improve tracing output in chain verifier 2020-07-22 21:59:00 -07:00
Jane Lusby 7d4e717182
Add block locator request to state layer (#712)
* Add block locator request to state layer

* pass genesis in request

* Update zebrad/src/commands/start/sync.rs

* fix errors
2020-07-22 18:01:31 -07:00
teor cf9bd2c974 diagnostic: Warn on unexpected high blocks 2020-07-23 10:26:25 +10:00
teor 52002ac3c5 feature: Add tracing to chain and checkpoint verifiers 2020-07-23 10:26:25 +10:00
teor e75697300b fix: Remove some dead code lints 2020-07-23 10:26:25 +10:00