Commit Graph

1618 Commits

Author SHA1 Message Date
teor 76a9f5b8c4 Refactor block target spacing into NetworkUpgrade methods
And add a method for the minimum difficulty time gap threshold.
2020-11-12 12:30:54 +10:00
teor 405c0644f9 Add a comment explaining the issues in ZIPs 205 and 208
And add the network to the difficulty filter error.
2020-11-12 12:30:54 +10:00
dependabot[bot] a7ad73feac build(deps): bump sled from 0.34.4 to 0.34.5
Bumps [sled](https://github.com/spacejam/sled) from 0.34.4 to 0.34.5.
- [Release notes](https://github.com/spacejam/sled/releases)
- [Changelog](https://github.com/spacejam/sled/blob/master/CHANGELOG.md)
- [Commits](https://github.com/spacejam/sled/compare/v0.34.4...v0.34.5)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-10 11:01:06 -05:00
teor 41788c3f27
Add metrics for the non-finalized state and queue (#1263) 2020-11-10 11:11:58 +10:00
Alfredo Garcia 128643d81e
Call `zebra_test::init` where needed. (#1227)
* Add missing `zebra_test::init()` to zebra-chain
* Add missing `zebra_test::init()` to zebra-consensus
* Add missing `zebra_test::init()` to zebra-network
* Add missing `zebra_test::init()` to zebra-state
* Add missing `zebra_test::init()` to zebra-test
* Add missing `zebra_test::init()` to zebrad
2020-11-10 10:29:25 +10:00
Jane Lusby 34f50d7ebb
Fix inconsistencies related to best chain order in RFC and state impl (#1267)
Prior to this PR we realized that the RFC had been drafted with the assumption that chains would be ordered from best to worst in `NonFinalizedState`. This assumption was incorrect, since `BTreeSet` only ever orders values in ascending order. This discrepancy was noticed and fixed in the code, but there were still some inconsistencies that needed to be cleaned up.

This PR updates all the incorrect or confusing comments about chain ordering in the RFC and code.
2020-11-09 15:53:16 -08:00
teor efef2a2bd7
Reduce acceptance test sled memory usage (#1236)
* Use the default memory limit in the acceptance tests

PR #1233 changed the default `memory_cache_bytes`, but left the
acceptance tests with their old value.
2020-11-10 07:42:30 +10:00
Jane Lusby 818fede30f
Reorganize memory_state to avoid giant test module (#1258)
Prior to this PR `memory_state` defined and implemented functionality for three different types, `Chain`, `NonFinalizedState`, and `QueuedBlocks`. Each of these components will need a fair number of unit tests, and I realized that as its currently organized it would be difficult to organize the tests or at a glance figure out which tests are testing which components.

This PR changes the organization of `memory_state` such that each component it exports is defined in its own module. In follow up PRs each module will get its own test module, which will focus exclusively on unit tests for the item defined there-in.

- [Tracking Issue](https://github.com/ZcashFoundation/zebra/issues/1250)
2020-11-09 10:05:18 -08:00
dependabot[bot] e40c7b57c3 build(deps): bump bs58 from 0.3.1 to 0.4.0
Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/compare/0.3.1...0.4.0)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-09 09:35:19 -05:00
Jane Lusby c9bf7f4cf6 Add tests for new sled impls needed for debug format 2020-11-09 09:12:26 -05:00
Jane Lusby c1de002f58 add key debug to assert error message 2020-11-09 09:12:26 -05:00
Jane Lusby ee7ec31aa2 Add a more relevant error message to insert panics 2020-11-09 09:12:26 -05:00
Deirdre Connolly 782f4606d0 Revert "Add message if assertion that sled map keys don't collide fails"
This reverts commit d784a23eab.
2020-11-09 09:12:26 -05:00
Deirdre Connolly 1a9490dc4f Revert "Bytes are Display-able"
This reverts commit c48e24e89b.
2020-11-09 09:12:26 -05:00
Deirdre Connolly 27d0fa8d83 Bytes are Display-able 2020-11-09 09:12:26 -05:00
Deirdre Connolly bfbdd7112b Add message if assertion that sled map keys don't collide fails
Co-authored-by: teor <teor@riseup.net>
2020-11-09 09:12:26 -05:00
Jane Lusby 76385e8620 assert that sled maps are collision free 2020-11-09 09:12:26 -05:00
dependabot[bot] a58299a0f0 build(deps): bump color-eyre from 0.5.6 to 0.5.7
Bumps [color-eyre](https://github.com/yaahc/color-eyre) from 0.5.6 to 0.5.7.
- [Release notes](https://github.com/yaahc/color-eyre/releases)
- [Changelog](https://github.com/yaahc/color-eyre/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yaahc/color-eyre/compare/v0.5.6...v0.5.7)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-09 08:40:55 -05:00
dependabot[bot] 44e6023434 build(deps): bump w9jds/firebase-action from v1.5.0 to v2.0.0
Bumps [w9jds/firebase-action](https://github.com/w9jds/firebase-action) from v1.5.0 to v2.0.0.
- [Release notes](https://github.com/w9jds/firebase-action/releases)
- [Commits](https://github.com/w9jds/firebase-action/compare/v1.5.0...0a5173f97c956332c9e71750526e0ca6f38cbe3a)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-09 08:40:03 -05:00
teor f90a749910
Clarify CheckpointVerifier errors (#1260)
And make an unreachable error into a panic.
2020-11-06 11:07:30 -08:00
Jane Lusby e8a3a28869
swap best_chain_len and related constants to u32 for consistency (#1257) 2020-11-06 14:00:10 +10:00
teor a0c7e34d63 Support systems where bash isn't in /usr/bin 2020-11-05 08:42:19 -05:00
teor fef75dadc9 Support systems where bash isn't in /usr/bin 2020-11-05 08:42:19 -05:00
teor 9f261e2213 Fix options order in metrics docker commands
And use long-form command options for readability
2020-11-05 08:39:14 -05:00
Jane Lusby 06e74d15ae
Add proptests for sled conversions (#1253)
* Add proptests for sled conversions
* document primary helper functions
2020-11-05 14:47:56 +10:00
dependabot[bot] 1e3cf6dc5c build(deps): bump tracing-subscriber from 0.2.14 to 0.2.15
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.14 to 0.2.15.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.14...tracing-subscriber-0.2.15)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-04 20:37:40 -05:00
dependabot[bot] 785fc30481 build(deps): bump hyper from 0.13.8 to 0.13.9
Bumps [hyper](https://github.com/hyperium/hyper) from 0.13.8 to 0.13.9.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.13.8...v0.13.9)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-04 20:07:18 -05:00
dependabot[bot] e87340594a build(deps): bump thiserror from 1.0.21 to 1.0.22
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.21 to 1.0.22.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.21...1.0.22)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-04 19:28:42 -05:00
dependabot[bot] b99ecbd2d7 build(deps): bump blake2b_simd from 0.5.10 to 0.5.11
Bumps [blake2b_simd](https://github.com/oconnor663/blake2_simd) from 0.5.10 to 0.5.11.
- [Release notes](https://github.com/oconnor663/blake2_simd/releases)
- [Commits](https://github.com/oconnor663/blake2_simd/compare/0.5.10...0.5.11)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-04 19:28:24 -05:00
dependabot[bot] dd7e1bb3f1 build(deps): bump blake2s_simd from 0.5.10 to 0.5.11
Bumps [blake2s_simd](https://github.com/oconnor663/blake2_simd) from 0.5.10 to 0.5.11.
- [Release notes](https://github.com/oconnor663/blake2_simd/releases)
- [Commits](https://github.com/oconnor663/blake2_simd/compare/0.5.10...0.5.11)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-04 18:54:13 -05:00
dependabot[bot] 578a3ca5ab build(deps): bump sha2 from 0.9.1 to 0.9.2
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.1 to 0.9.2.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.1...streebog-v0.9.2)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-04 18:53:49 -05:00
dependabot[bot] 2442cf162f build(deps): bump regex from 1.4.1 to 1.4.2
Bumps [regex](https://github.com/rust-lang/regex) from 1.4.1 to 1.4.2.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.4.1...1.4.2)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-02 17:01:21 -05:00
Jane Lusby 714def990e
make state service use both finalized and non-finalized state (#1239)
* make service use both finalized and non-finalized state

* Document new functions

* add documentation to sled fns

* cleanup tip fn now that errors are gone

* rename height unwrap fn
2020-11-01 10:49:34 -08:00
Henry de Valence 0ad648fb6a zebrad: make lookahead limit configurable.
Sets the default value to the previous lookahead limit.  My testing on
mainnet suggested that the newly lower value (changed when the
checkpoint frequency was decreased) is low enough to cause stalls, even
when using hedged requests.
2020-11-01 10:47:46 -08:00
teor 776e49ff0c State RFC: clarify difficulty
The difficulty validation RFC will introduce a definition of
per-block difficulty. Make it clear that the state RFC
definition is cumulative difficulty.
2020-10-30 19:12:32 -04:00
Jane Lusby 1b7c57371d
Fix format used to store transactions in sled (#1238)
## Motivation

While working on the block locator fix PR together with Henry we noticed that we'd accidentally serialized entire transactions in `tx_by_hash`, instead of serializing just the height of the block and the index of the transaction within the block, as described by the original RFC.

## Solution

We've fixed it by adding a `TransactionLocation` new type, which handles the sled format traits. We've removed the sled format impls for `Transaction` to prevent inserting the wrong data in the future. Finally we've bumped the database format to reflect the change in the format on the disk and its incompatibility with previous versions.
2020-10-30 15:24:39 -07:00
teor adbd338d6d
Add Review and Follow Up to PR Template (#1235)
* Add review guidelines to the default PR template

* Apply suggestions

Co-authored-by: teor <teor@riseup.net>

* Add a Follow Up Work section to the PR template

* Mention design RFCs in the PR template

* Put key PR review questions in bold

* Tweak PR review "skip task" process

* Update .github/pull_request_template.md

Co-authored-by: teor <teor@riseup.net>

* Shorter alternative pull request template

Add Review and Follow Up sections
Add a checklist for documentation and tests

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
2020-10-30 14:42:04 -07:00
teor 01aeaae3f8 Placate rustfmt 2020-10-30 14:41:38 -07:00
teor f37f48918b Replace into with Some in difficulty::arbitrary 2020-10-30 14:41:38 -07:00
teor d883462d83 Silence a clippy equal arguments warning
The unit tests need to pass identical arguments to an equality
comparison, to make sure equality is implemented correctly.
2020-10-30 14:40:42 -07:00
teor 1c31225aac
Implement Expanded to Compact Difficulty Conversion (#1196)
* Implement Expanded to Compact Difficulty
* Implement Arbitrary for CompactDifficulty
Remove the derive, and generate values from random block
hashes.
* Implement Arbitrary for ExpandedDifficulty and Work
* Use Arbitrary for CompactDifficulty in Arbitrary for Block
* Test difficulty on all block test vectors
And cleanup some duplicate test code
* Round-trip tests for compact test cases
* Round-trip tests for compact difficulty in block test vectors
* Make Add for Work return PartialCumulativeWork
Remove AddAssign for Work
Rewrite a proptest using Sub for PartialCumulativeWork
Use Arbitrary for Work
* Add roundtrip work sum tests
* Add roundtrip comparison difficulty tests
* Add failing proptest cases due to test bugs
* Use Some(_) rather than _.into()
* Reduce visibility of difficulty type inner values
* Split work and other difficulty proptests
This change makes sure that rejected work values don't disable property
tests on other types.
2020-10-30 11:36:59 +10:00
teor f338048012 Clarify edge case 2020-10-29 15:41:45 -04:00
teor b742657e45 State RFC: Clarify genesis sprout anchors 2020-10-29 15:41:45 -04:00
Henry de Valence e2c5b71323 state: document memory_cache_bytes, reduce default
Closes #1026

Because of the way that sled uses this parameter, the actual in-memory
size may be much larger.  Dialing this down should help avoid high
memory usage.
2020-10-29 14:31:50 -04:00
Henry de Valence 68b9a70a60 state: skip transactions in a genesis block 2020-10-29 09:58:56 -07:00
Henry de Valence 1573e0e4ae state: remove spent UTXOs in commit_finalized_direct 2020-10-29 09:58:56 -07:00
Jane Lusby c8a97f94b2 state: clean up SledState query impls 2020-10-29 09:58:56 -07:00
Jane Lusby dace92aca1 state: add SLED_FORMAT_VERSION prefix to db path
Also removes a redundant test.
2020-10-29 09:58:56 -07:00
Jane Lusby d7e8217d8a clippy: remove generated tests in empty crates 2020-10-29 09:58:56 -07:00
Jane Lusby 2095b4f0d3 state: insert into nullifier trees 2020-10-29 09:58:56 -07:00