Commit Graph

968 Commits

Author SHA1 Message Date
teor 993532b604 feature: Add a "Genesis" network upgrade
We can use this network upgrade to implement different consensus rules
and chain context handling for genesis blocks.

Part of the chain state design in #682.
2020-07-27 14:03:14 -04:00
teor 5ece62f0a5 fix: Update comments for Heartwood's hashChainHistoryRoot 2020-07-27 14:00:56 -04:00
teor 12cbe410f2 fix: Hide warn-level logs in the zebra-consensus tests
Unless RUST_LOG is set.
2020-07-27 13:51:33 -04:00
teor a60299d86c fix: Change the default test log level to warn 2020-07-27 13:51:33 -04:00
teor 2d3926e800 fix: Replace a return with unreachable! 2020-07-27 13:49:11 -04:00
dependabot[bot] 842c69c60c build(deps): bump serde_json from 1.0.56 to 1.0.57
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.56 to 1.0.57.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.56...v1.0.57)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-27 13:48:15 -04:00
Alfredo Garcia 5b3c6e4c6c
Port bash checkpoint scripts to zebra-checkpoints single rust binary (#740)
* make zebra-checkpoints
* fix LOOKAHEAD_LIMIT scope
* add a default cli path
* change doc usage text
* add tracing
* move MAX_CHECKPOINT_HEIGHT_GAP to zebra-consensus
* do byte_reverse_hex in a map
2020-07-25 17:53:00 +10:00
Henry de Valence b59cfc49b7 sync: create requests sequentially to respect backpressure.
This seems like a better design on principle but also appears to give a much
nicer sawtooth pattern of queued blocks in the checkpointer and a much smoother
pattern of block requests.
2020-07-24 18:36:00 -04:00
Deirdre Connolly 0cd8b7e7bd Rename pr.yml to ci.yml 2020-07-24 18:18:46 -04:00
Deirdre Connolly e24cd5755b Add a nicer human name to coverage job 2020-07-24 18:18:46 -04:00
Deirdre Connolly 4180a2731f Turn off the CodeCov PR comment, metrics available in the checks results which link to codecov.io 2020-07-24 18:18:46 -04:00
Deirdre Connolly bbc6fb82bf Increase timeout on coverage job to 10mins, reorg config 2020-07-24 18:18:46 -04:00
Henry de Valence 4aa00ad216 Align crate versions and user-agent with NU numbers.
We had a brief discussion on discord and it seemed like we had consensus on the
following versioning policy:

* zebrad: match major version to NU version, so we will start by releasing
  zebrad 3.0.0;

* zebra-* libraries: start by matching zebrad's version, then increment major
  versions of each library as we need to make breaking changes (potentially
  faster than the zebrad version, always respecting semver but making no
  guarantees about the longevity of major releases).

This commit sets all of the crate versions to 3.0.0-alpha.0 -- the -alpha.0
marks it as a prerelease not subject to perfect adherence to compatibility
guarantees.
2020-07-24 11:46:37 -07:00
dependabot[bot] f7c59c99b5 build(deps): bump tracing-subscriber from 0.2.8 to 0.2.9
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.8 to 0.2.9.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.8...tracing-subscriber-0.2.9)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-24 14:31:44 -04:00
teor 2acfcf3a90
Make the CheckpointVerifier handle partial restarts (#736)
Also put generic bounds on the BlockVerifier struct,
so we get better compilation errors.
2020-07-24 11:47:48 +10:00
dependabot[bot] 80597087b3 build(deps): bump codecov/codecov-action from v1.0.11 to v1.0.12
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.0.11 to v1.0.12.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.0.11...07127fde53bc3ccd346d47ab2f14c390161ad108)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-23 18:48:04 -04:00
teor 77a1fefa1e
Download genesis (#731)
* feature: Add more CheckpointVerifier tracing

* fix: Download the genesis block
2020-07-23 10:56:52 -07:00
dependabot[bot] e99e9653eb
build(deps): bump sled from 0.33.0 to 0.34.0 (#735)
Bumps [sled](https://github.com/spacejam/sled) from 0.33.0 to 0.34.0.
- [Release notes](https://github.com/spacejam/sled/releases)
- [Changelog](https://github.com/spacejam/sled/blob/master/CHANGELOG.md)
- [Commits](https://github.com/spacejam/sled/compare/v0.33...v0.34)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-07-23 10:55:34 -07:00
dependabot[bot] 909b74a613
build(deps): bump tracing from 0.1.16 to 0.1.17 (#734)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.16 to 0.1.17.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.16...tracing-0.1.17)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-07-23 10:54:56 -07:00
Jane Lusby c1a1493159
use dirs crate for default location of state and config (#714)
* use dirs crate for default location of state and config
* panic if a path isn't specified for zebra-state
2020-07-23 21:12:20 +10:00
teor 132875198e fix: Simplify ChainVerifier 2020-07-23 03:46:24 -04:00
teor c95c825707 fix: Lookup the genesis hash based on the network 2020-07-23 03:46:24 -04:00
teor 41225660fb doc: The genesis coinbase is the same in every network 2020-07-23 03:46:24 -04:00
teor da09965a5f feature: Get the current minimum protocol version 2020-07-23 15:52:18 +10:00
teor 0e0c36c010 fixup! feature: Add network upgrade activation heights 2020-07-23 15:52:18 +10:00
teor 4b683ea2b1 feature: Add Consensus Branch Ids 2020-07-23 15:52:18 +10:00
teor 85f113bc18 doc: Add a TODO to the network protocol 2020-07-23 15:52:18 +10:00
teor 359febb2e2 doc: Update a CheckpointList comment 2020-07-23 15:52:18 +10:00
teor c9ee85c3b5 feature: Add network upgrade activation heights 2020-07-23 15:52:18 +10:00
Henry de Valence 4a98b8fa0d Add basic metrics to the syncer. 2020-07-22 21:59:00 -07:00
Henry de Valence c2c2a28e8b Improve tracing output in chain verifier 2020-07-22 21:59:00 -07:00
Jane Lusby 7d4e717182
Add block locator request to state layer (#712)
* Add block locator request to state layer

* pass genesis in request

* Update zebrad/src/commands/start/sync.rs

* fix errors
2020-07-22 18:01:31 -07:00
Henry de Valence 49aa41544d sync: try to ignore spurious inv messages.
Closes #697.

per  https://github.com/ZcashFoundation/zebra/issues/697#issuecomment-662742971

The response to a getblocks message is an inv message with the hashes of the
following blocks. However, inv messages are also sent unsolicited to gossip new
blocks across the network. Normally, this wouldn't be a problem, because for
every other request we filter only for the messages that are relevant to us.
But because the response to a getblocks message is an inv, the network layer
doesn't (and can't) distinguish between the response inv and the unsolicited
inv.

But there is a mitigation we can do. In our sync algorithm we have two phases:
(1) "ObtainTips" to get a set of tips to chase down, (2) repeatedly call
"ExtendTips" to extend those as far as possible. The unsolicited inv messages
have length 1, but when extending tips we expect to get more than one hash. So
we could reject responses in ExtendTips that have length 1 in order to ignore
these messages. This way we automatically ignore gossip messages during initial
block sync (while we're extending a tip) but we don't ignore length-1 responses
while trying to obtain tips (while querying the network for new tips).
2020-07-22 17:55:52 -07:00
Henry de Valence 90ec2982bf docs: fix broken links 2020-07-22 17:55:52 -07:00
Henry de Valence cc955a2bbe network: document Responses, add warning about unsolicited invs. 2020-07-22 17:55:52 -07:00
teor cf9bd2c974 diagnostic: Warn on unexpected high blocks 2020-07-23 10:26:25 +10:00
teor 52002ac3c5 feature: Add tracing to chain and checkpoint verifiers 2020-07-23 10:26:25 +10:00
teor 9b97ebbd61 feature: Choose checkpoints based on the config 2020-07-23 10:26:25 +10:00
teor 3d721a96a5 feature: Add the state config to the config file 2020-07-23 10:26:25 +10:00
teor e75697300b fix: Remove some dead code lints 2020-07-23 10:26:25 +10:00
teor 89ac2793d6 feature: Use ChainVerifier in the sync service 2020-07-23 10:26:25 +10:00
teor a998346f4c refactor: Split out a simpler chain::init function
This new chain::init function will let us hide the BlockVerifier and
CheckpointVerifier from the zebra-consensus public interface.
(If needed.)
2020-07-23 10:26:25 +10:00
Jane Lusby a722cf33f7 enable new tracing instrumentation in tokio 2020-07-22 14:39:54 -04:00
Jane Lusby e105b4f6c5
properly document guarantee provided by zebra-network (#713)
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2020-07-22 14:38:00 -04:00
Henry de Valence add093e3bf consensus: fix error parameters in redjubjub::Verifier 2020-07-22 14:29:55 -04:00
Henry de Valence 0586da7167 Revert #500 (generic errors in tower-batch).
Unfortunately, since the Batch wrapper was changed to have a generic error
type, when wrapping it in another Service, nothing constrains the error type,
so we have to specify it explicitly to avoid an inference hole.  This is pretty
unergonomic -- from the compiler error message it's very unintuitive that the
right fix is to change `Batch::new` to `Batch::<_, _, SomeError>::new`.

The options are:

1. roll back the changes that make the error type generic, so that the error
   type is a concrete type;

2. keep the error type generic but hardcode the error in the default
   constructor and add an additional code path that allows overriding the
   error.

However, there's a further issue with generic errors: the error type must be
Clone.  This problem comes from the fact that there can be multiple Batch
handles that have to share access to errors generated by the inner Batch
worker, so there's not a way to work around this.  However, almost all error
types aren't Clone, so there are fairly few error types that we would be
swapping in.

This suggests that in case (2) we would be maintaining extra code to allow
generic errors, but with restrictive enough generic bounds to make it
impractical to use generic error types.  For this reason I think that (1) is a
better option.
2020-07-22 14:29:55 -04:00
dependabot[bot] 7067ac6e0d build(deps): bump tokio from 0.2.21 to 0.2.22
Bumps [tokio](https://github.com/tokio-rs/tokio) from 0.2.21 to 0.2.22.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-0.2.21...tokio-0.2.22)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-22 12:43:38 -04:00
Henry de Valence 4a41c9254d network: avoid panic when shutting down cleanly.
When the connection sees the client_rx channel close it knows it will never get
any more requests, and it should terminate.  But instead of terminating, it
errored itself, and the method to error itself tries to pull all the
outstanding client requests from the channel in order to fail them before it
shuts down.  This results in reading from a closed channel, causing a panic.
Instead we return cleanly rather than failing (since we know there are no
outstanding requests, as the channel is closed).
2020-07-22 18:04:45 +10:00
Henry de Valence 928b0beb5d sync: unindent fetch task 2020-07-21 20:16:23 -07:00
Henry de Valence b722818e02 sync: remove redundant tracing specifier
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
2020-07-21 20:16:23 -07:00