Commit Graph

991 Commits

Author SHA1 Message Date
dependabot[bot] 64f116a861
build(deps): bump the test group with 1 update (#7284)
Bumps the test group with 1 update: [tempfile](https://github.com/Stebalien/tempfile).

- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.6.0...v3.7.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: test
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-24 17:45:55 +00:00
Arya a6bdb480bb
Release Zebra v1.1.0 (#7259)
* Updates dependencies

* Dedups bindgen, removes windows-sys from deny.toml

* Updates checkpoints

* chore: Release

* Adds changelog draft

* updates estimated_release_height

* Updates changelog

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

---------

Co-authored-by: teor <teor@riseup.net>
2023-07-22 01:18:09 +00:00
Alfredo Garcia 7ce5b8b115
fix(feature): elasticsearch for the Mainnet (#7268)
* differenciate testnet vs mainnet bulks

* reduce the number of bulk blocks to 24 in mainnet
2023-07-21 22:34:43 +00:00
teor c3e7d749ed
fix(log): Limit verbose RPC responses in logs (#7258)
* Truncate verbose RPC responses in test logs

* Downgrade verbose block template state log to debug
2023-07-21 20:35:56 +00:00
teor 512dd9bc5d
change(doc): Document how to upgrade the database format (#7261)
* Move the state format into a new doc

* Add upgrade instructions

* Link to the format upgrade docs from the upgrade code

* Fix typo

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2023-07-20 01:50:25 +00:00
Marek 9ebd56092b
change(test): Refactor the tests of non-finalized state (#7262)
* Allow generating blocks with only coinbase tx

* Fix the `forked_equals_pushed_genesis` test
2023-07-19 21:20:37 +00:00
Marek 1db4f567f7
change(state): Deduplicate note commitment trees in non-finalized state (#7239)
* Remove duplicate asserts

There are the same two asserts above the two removed ones.

* Remove workarounds for inserting trees into NFS

NFS = non finalized state

* Use correct height for constructing new chain

We were using the height of the last block instead of the initial block
to construct a new chain.

* Don't push the 0th block into a chain

* Don't commit two blocks at the same height

* Add helpers for heights

* Support the retrieval of deduped Sprout trees

* Dedup Sprout trees

* Refactor docs for adding & removing Sprout trees

* Support the retrieval of deduped Sapling trees

* Dedup Sapling trees

* Refactor docs for adding & removing Sapling trees

* Support the retrieval of deduped Orchard trees

* Dedup Orchard trees

* Refactor docs for adding & removing Orchard trees

* Make the docs for storing trees clearer
2023-07-18 23:04:11 +00:00
dependabot[bot] f46ad54ae0
build(deps): bump insta from 1.30.0 to 1.31.0 (#7226)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.30.0 to 1.31.0.
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.30.0...1.31.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-18 04:55:54 +00:00
Marek f9a5635105
fix(state): Remove workarounds for storing trees (#7218)
* Remove duplicate asserts

There are the same two asserts above the two removed ones.

* Remove workarounds for inserting trees into NFS

NFS = non finalized state

* Use correct height for constructing new chain

We were using the height of the last block instead of the initial block
to construct a new chain.

* Don't push the 0th block into a chain

* Don't commit two blocks at the same height

* Fix typo

* Generate chains with at least two blocks

---------

Co-authored-by: teor <teor@riseup.net>
2023-07-18 04:54:22 +00:00
teor 3bbe3cec4f
fix(panic): Stop panicking on async task cancellation on shutdown in network and state futures (#7219)
* Add an async-error feature and an initial module structure

* Implement checking for panics in OS threads and async tasks

* Implement waiting for panics in OS threads and async tasks

* Add a TODO to simplify some state request error handling

* Use the new panic-checking methods in zebra-state

* Use new panic-checking methods in zebra-network

* fixup! Implement waiting for panics in OS threads and async tasks

* Replace existing async code with generic panic-checking methods

* Simplify trait to a single method

* Move thread panic code into generic trait impls

* Simplify option handling

Co-authored-by: Arya <aryasolhi@gmail.com>

* Fix comment

Co-authored-by: Arya <aryasolhi@gmail.com>

* Add missing track_caller

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2023-07-18 04:53:26 +00:00
dependabot[bot] c885de4abb
build(deps): bump semver from 1.0.17 to 1.0.18 (#7227)
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.17 to 1.0.18.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.17...1.0.18)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-18 02:37:35 +00:00
Alfredo Garcia e2f010eb4f
build(deps): update ecc dependencies for `zcashd` 5.6.0, and create legacy state format compatibility layer (#7053)
* update ecc dependencies no serialization

* use zcash_primitives merke tree functions for serialization/deserialization in orchard and sapling trees

* some more work in sapling/orchard serialization/deserialization

* clippy

* fix doc links

* fix missing doc

* make orchard trees serializa/deserialize as they were before upgrade

* make sapling trees serialize/deserialize as they were before upgrade

* use legacy for sprout

* remove unused code

* readd snapshot tests

* repalce some code

* upgrade zcash_proofs

* remove legacy code for sprout

* fix the count method

* add root to serialize

* fixes in as_bytes and from_bytes

* use legacy code

* add todo about pow2 tests

* remove unused sprout code

* fix doc typos

* Add a recalculate_root() method to trees for tests

* Rename test tree types to make them easier to change

* Add TODOs for tests for old and new serialization formats

* fix doc typos

* add more test to note commitment trees

* fix comment

* fix leaf serializatiuon, reverse to old hashes in pow2 tests

* fix serialization

* put sapling SerializedTree code back

* put orchard SerializedTree code back

* clippy

* add duplicated dependencies until zebra_script updates

* fix a doc link

* minor cleanup

* remove todo comment from tests

* add one more check to tests

* update zebra_script

* update deny.toml

* replace custom function with library

* fix some tests

* update docs

* Remove duplicate dependencies from deny.toml

Based on the list at:
https://github.com/ZcashFoundation/zebra/actions/runs/5557139662/jobs/10150543673?pr=7053#step:5:14

---------

Co-authored-by: teor <teor@riseup.net>
2023-07-17 22:06:27 +00:00
Marek f1ee502ddf
Fix the position of coinbase tx (#7217) 2023-07-15 16:24:51 +00:00
teor be5cfad07f
change(state): Prepare for in-place database format upgrades, but don't make any format changes yet (#7031)
* Move format upgrades to their own module and enum

* Launch a format change thread if needed, and shut it down during shutdown

* Add some TODOs and remove a redundant timer

* Regularly check for panics in the state upgrade task

* Only run example upgrade once, change version field names

* Increment database format to 25.0.2: add format change task

* Log the running and initial disk database format versions on startup

* Add initial disk and running state versions to cached state images in CI

* Fix missing imports

* Fix typo in logs workflow command

* Add a force_save_to_disk argument to the CI workflow

* Move use_internet_connection into zebrad_config()

* fastmod can_spawn_zebrad_for_rpc can_spawn_zebrad_for_test_type zebra*

* Add a spawn_zebrad_without_rpc() function

* Remove unused copy_state() test code

* Assert that upgrades and downgrades happen with the correct versions

* Add a kill_and_return_output() method for tests

* Add a test for new_state_format() versions (no upgrades or downgrades)

* Add use_internet_connection to can_spawn_zebrad_for_test_type()

* Fix workflow parameter passing

* Check that reopening a new database doesn't upgrade (or downgrade) the format

* Allow ephemeral to be set to false even if we don't have a cached state

* Add a test type that will accept any kind of state

* When re-using a directory, configure the state test config with that path

* Actually mark newly created databases with their format versions

* Wait for the state to be opened before testing the format

* Run state format tests on mainnet and testnet configs (no network access)

* run multiple reopens in tests

* Test upgrades run correctly

* Test that version downgrades work as expected (best effort)

* Add a TODO for testing partial updates

* Fix missing test arguments

* clippy if chain

* Fix typo

* another typo

* Pass a database instance to the format upgrade task

* Fix a timing issue in the tests

* Fix version matching in CI

* Use correct env var reference

* Use correct github env file

* Wait for the database to be written before killing Zebra

* Use correct workflow syntax

* Version changes aren't always upgrades

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-07-13 21:36:15 +00:00
dependabot[bot] 7f3a4e97b7
build(deps): bump regex from 1.8.4 to 1.9.1 (#7189)
Bumps [regex](https://github.com/rust-lang/regex) from 1.8.4 to 1.9.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.8.4...1.9.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-12 21:30:36 +00:00
teor f9a48266ad
change(test): Add a recalculate_root() method to trees for tests (#7204)
* Add a recalculate_root() method to trees for tests

* Rename test tree types to make them easier to change

* Add TODOs for tests for old and new serialization formats

* fix doc typos

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-07-12 17:07:10 +00:00
dependabot[bot] 0d332646bf
build(deps): bump the serde group with 1 update (#7184)
Bumps the serde group with 1 update: [serde](https://github.com/serde-rs/serde).

- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.166...v1.0.168)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: serde
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-10 08:27:21 +00:00
dependabot[bot] 8c5bcb7487
build(deps): bump thiserror from 1.0.41 to 1.0.43 (#7174)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.41 to 1.0.43.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.41...1.0.43)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-10 03:37:10 +00:00
dependabot[bot] c1695c35e2
build(deps): bump serde_json from 1.0.99 to 1.0.100 (#7149)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.99 to 1.0.100.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.99...v1.0.100)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-06 23:23:24 +00:00
teor 9df78ffdba
change(tests): Do round-trip tests for note commitment tree data structure and RPC serialisation (#7147)
* Add an assert_frontier_eq() method to note commitment trees for tests

* Check round-trip serialization for note commitment trees

* fix typos

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-07-06 01:04:28 +00:00
dependabot[bot] 730b10a278
build(deps): bump thiserror from 1.0.40 to 1.0.41 (#7153)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.40 to 1.0.41.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.40...1.0.41)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-05 20:53:05 +00:00
teor 147b8fa3a8
cleanup(rust): Fix new nightly clippy warnings (#7135)
* Fix "comparison is always true" warning

* Add missing Send bound

* cargo clippy --fix --all-features --all-targets

* incorrect implementation of clone on a Copy type

* cargo fmt --all
2023-07-05 07:11:27 +00:00
teor f2a2a403a8
fix(log): Remove redundant startup logs, fix progress bar number, order, and wording (#7087)
* Remove duplicate "running" message, send log file opening logs to stderr

* Add missing block progress bar desc, make them consistent

* Put progress bars in a specific order in each section

* Actually make block progress bar desc consistent

* Make progress bar order even more consistent

* Fix fork blocks plural for 1 block

* Use the correct number of chain fork bars

* Disable confusing partial work display

* Add struct field category comments

* Silence a verbose inventory log
2023-07-05 07:08:59 +00:00
dependabot[bot] 8fed1823af
build(deps): bump rlimit from 0.9.1 to 0.10.0 (#7139)
Bumps [rlimit](https://github.com/Nugine/rlimit) from 0.9.1 to 0.10.0.
- [Changelog](https://github.com/Nugine/rlimit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Nugine/rlimit/compare/v0.9.1...v0.10.0)

---
updated-dependencies:
- dependency-name: rlimit
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-05 02:09:57 +00:00
Alfredo Garcia 5859fac5b1
docs(state): Use different terms for block verification and state queues (#7061)
* claridy some checkpoint verifier docs

* update documentation of `Request::CommitSemanticallyVerifiedBlock` and `Request::CommitCheckpointVerifiedBlock`

* replace `prepared` with `semantically_verified` in state service checks code

* replace `non-finalized` where needed in docs of the state service

* fix double space in doc

* replace `finalized` where needed in docs of the state service

* change some docs in state queued_blocks.rs

* Rewrite pending UTXO checkpoint block comment

* Fix trailing space in docs

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

---------

Co-authored-by: teor <teor@riseup.net>
2023-07-04 21:29:41 +00:00
teor 9b32ab7878
change(release): Update release script and check it in CI (#7128)
* Activate production features on docs.rs

* Make version requirements more flexible to work around unpublished versions

* Remove a redundant feature requirement that causes publishing issues

* fix release auto-replacement format

* cargo release replace --verbose --execute --package zebrad

* Add a missing 1.0.1 changelog entry

* Fix incorrect commands in the release checklist

* Check the crates.io release script in CI, to avoid release failures

* Fix release date in changelog

* Fix workflow typo

* Add --no-confirm to the workflow

* Fix outdated version in release check command

* After the release, add a "remove do-not-merge" step

* Set git identity before committing

* Don't overwrite tweaked versions on upgrade

* cargo release version doesn't support --dependent-version

* Update the same commands in the release checklist

* Commit to git after each change

* And in the checklist

* Add cargo release config

* In CI, allow any branch and provide a fake previous tag name

* Remove redundant `cargo release commit`

* Use a simpler release version process

* Start simplifying the release dry run in CI

* Fix a cargo dependency lint

* just skip the failing crates for now

* Fix the release checklist to match CI

* Use the latest versions of all Zebra dependencies
2023-07-04 19:01:11 +00:00
dependabot[bot] 6ec0c66aa6
build(deps): bump serde from 1.0.164 to 1.0.166 (#7134)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.164 to 1.0.166.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.164...v1.0.166)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-04 16:53:54 +00:00
dependabot[bot] 4713994e64
build(deps): bump metrics from 0.21.0 to 0.21.1 (#7131)
Bumps [metrics](https://github.com/metrics-rs/metrics) from 0.21.0 to 0.21.1.
- [Changelog](https://github.com/metrics-rs/metrics/blob/main/release.toml)
- [Commits](https://github.com/metrics-rs/metrics/compare/metrics-v0.21.0...metrics-v0.21.1)

---
updated-dependencies:
- dependency-name: metrics
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-03 23:23:33 +00:00
dependabot[bot] a8e71b3429
build(deps): bump tokio from 1.29.0 to 1.29.1 (#7114)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.29.0 to 1.29.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.29.0...tokio-1.29.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-03 05:39:10 +00:00
teor 2add0e50a9
Release Zebra v1.0.1 (#7090)
* Update license description in README for MIT-only crates

* Draft changelog with trivial issues

* Remove trivial issues

* Update changelog entries as of commit 2a31972 and PR #7103

* Update mainnet and testnet checkpoints as of 2023-06-30

* chore: Release

* Estimate release height for Zebra v1.0.1

Block height 2139118 at 2023-06-30 01:55:38 UTC
Release is likely to be 2023-07-01
2139118 + 1152 * 3 = 2142574

Then round up to the nearest 1000.
2023-07-02 23:08:40 +00:00
dependabot[bot] 5eea111922
build(deps): bump tokio from 1.28.2 to 1.29.0 (#7094)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.28.2 to 1.29.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.28.2...tokio-1.29.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-30 02:58:00 +00:00
Alfredo Garcia 455779c833
build(deps): Bump indexmap, toml and serde_json (#7100)
* bump toml, indexmap and serde_json all together

* add duplicated indexmap dep to deny.toml
2023-06-29 20:18:52 +00:00
teor 5324e5afd2
add(tests): Add snapshot tests for sprout database formats (#7057)
* Add methods for loading entire column families from the database

* Add a method that loads all the sprout trees from the database

* Add snapshot tests for sprout note commitment trees

* Add round-trip proptests for tree root database serialization

* Add a manual sprout note commitment tree database serialization snapshot test

* Add tests for 1,2,4,8 note commitments in a tree

* Remove redundant "rand" package rename in dependencies

* Randomly cache roots rather than only caching even roots

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-06-27 15:32:30 +00:00
Marek 1f1d04b547
change(state): Refactor the structure of finalizable blocks (#7035)
* Add and use `FinalizableBlock`

This commit adds `FinalizableBlock`, and uses it instead of
`ContextuallyVerifiedBlockWithTrees` in `commit_finalized_direct()`

* Use `ContextuallyVerifiedBlockWithTrees`

This commit passes `ContextuallyVerifiedBlockWithTrees` instead of
passing separate `finalized`, `history_tree` and `note_commitment_trees`
when storing blocks in the finalized state.

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* add docs to new methods

* fix existing doc

* rename `ContextuallyVerifiedBlockWithTrees` to `SemanticallyVerifiedBlockWithTrees`

* Refactor docs

* Refactor comments

* Add missing docs, fix typo

* Fix rustfmt

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-06-27 08:58:14 +00:00
teor 941be2965c
Replace a chain length assertion with a NotReadyToBeCommitted error (#7072) 2023-06-27 06:50:35 +00:00
teor 76a7ff45a9
fix(deps): Replace openssl with rustls in tests and experimental features (#7047)
* Remove openssl dependency and prevent it coming back

* Put the arguments in the right place

* Put comment in the right place

* Add a default-docker feature to zebrad and use it in workflows and Docker files

* Fix a comment typo

* Make sure that Docker production builds don't use openssl

* Rename feature to default-release-binaries
2023-06-26 05:44:19 +00:00
dependabot[bot] 7e6162a043
build(deps): bump itertools from 0.10.5 to 0.11.0 (#7050)
* build(deps): bump itertools from 0.10.5 to 0.11.0

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.5 to 0.11.0.
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.5...v0.11.0)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* add itertools as a duplicated dep

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-06-23 18:27:49 +00:00
dependabot[bot] 835e7a88f0
build(deps): bump insta from 1.29.0 to 1.30.0 (#7051)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.29.0 to 1.30.0.
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.29.0...1.30.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-23 01:00:27 +00:00
teor 3d2c5ef290
fix(concurrency): Use Arc::into_inner() to avoid potential concurrency issues, needs Rust 1.70 (#7032)
* Use Arc::into_inner() to avoid potential concurrency issues

* Remove some outdated clippy lint workarounds (fixed in Rust 1.66)

* Update the required Rust version to 1.70
2023-06-21 20:44:53 +00:00
Marek 006c2ae42b
change(state): Refactor the structure of verified blocks (#7025)
* Refactor `CheckpointVerifiedBlock`

This commit turns `CheckpointVerifiedBlock` into a wrapper of
`SemanticallyVerifiedBlock` since both structs have the same fields.

* Refactor `ContextuallyVerifiedBlockWithTrees`

This commit uses `SemanticallyVerifiedBlock` in
`ContextuallyVerifiedBlockWithTrees` instead of
`CheckpointVerifiedBlock`.
2023-06-21 16:58:11 +00:00
Marek abcabd1931
Use `OrderedUtxo` in `CheckpointVerifiedBlock` (#6971) 2023-06-19 22:48:59 +00:00
dependabot[bot] 0da2dcb503
build(deps): bump serde_json from 1.0.96 to 1.0.97 (#6987)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.96 to 1.0.97.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.96...v1.0.97)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-16 23:43:23 +00:00
Alfredo Garcia 8a7c871480
rename(state): do additional renaming for clarification purposes (#6967)
* do renames by script

```
git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/CommitFinalized/CommitCheckpointVerified/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/commit_finalized_block/commit_checkpoint_verified/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/drain_queue_and_commit_finalized/drain_finalized_queue_and_commit/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/queue_and_commit_finalized/queue_and_commit_to_finalized_state/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/queued_finalized_blocks/finalized_state_queued_blocks/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/max_queued_finalized_height/max_finalized_queue_height/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/send_finalized_block_error/send_checkpoint_verified_block_error/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/QueuedFinalized/QueuedCheckpointVerified/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/queue_and_commit_non_finalized/queue_and_commit_to_non_finalized_state/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/queued_non_finalized_blocks/non_finalized_state_queued_blocks/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/send_non_finalized_block_error/send_semantically_verified_block_error/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/QueuedNonFinalized/QueuedSemanticallyVerified/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/last_sent_finalized_block_hash/finalized_block_write_last_sent_hash/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/sent_non_finalized_block_hashes/non_finalized_block_write_sent_hashes/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/invalid_block_reset_receiver/invalid_block_write_reset_receiver/g'

cargo fmt --all
```

* add missing log renames by script

```
git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/queued finalized block/finalized state queue block/g'

git ls-tree --full-tree -r --name-only HEAD | \
grep -v 'book' | grep -v 'CHANGELOG.md' | \
xargs sed -i 's/queued non-finalized block/non-finalized state queue block/g'

cargo fmt --all
```
2023-06-15 05:12:45 +00:00
Conrado Gouvea fd78556000
fix: prevent progress bar from panicking using workaround (#6940)
* fix: prevent progress bar from panicking using workaround

* remove more progress bar lengths

* re-add set_len() calls commented out; restore one that shouldn't cause panics

* Update zebra-state/src/service/non_finalized_state.rs

Co-authored-by: teor <teor@riseup.net>

---------

Co-authored-by: teor <teor@riseup.net>
2023-06-14 19:07:02 +00:00
Alfredo Garcia 17bd7884ea
fix(build): add elasticsearch feature to block serialize (#6709)
* add elasticsearch feature to block serialize

* fix for elastic feature

* add zebra-chain elasticsearch dep to zebrad
2023-06-14 19:01:39 +00:00
Deirdre Connolly fe859bd6a1
Release 1.0.0 (#6877)
* Include zebra-* and tower-fallback versions and paths for tower-batch-control

* Remove versions for dev-dependencise

* fix(legal): Use correct licenses for tower-batch-control and zebra-network (#6927)

* Don't modify whitespace

* Resolve rustdoc and cargo-release warnings

* Bump semver

* Re-up Cargo.lock

* Update README.md

Co-authored-by: teor <teor@riseup.net>

* Update book/src/user/docker.md

Co-authored-by: teor <teor@riseup.net>

* Update 'Future Work' section post stable release

* Add release notes to CHANGELOG.md

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Remove Docker container change from the changelog

Co-authored-by: teor <teor@riseup.net>

* Point at the appropriate zebra-* crate versions

* Update ESTIMATED_RELEASE_HEIGHT to 2,121,200

* Add CHANGELOG summary of the 1.0.0 release

* Update CHANGELOG.md

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Bump tower-* beta.* versions and use them

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Pili Guerra <pili@piliguerra.com>
2023-06-14 04:35:45 +00:00
Deirdre Connolly 62d946d139
change(crates): Prepare for publishing crates to crates.io (#6908)
* change(crates): Publish tower-batch-control

* Cargo fields for zebra-test

* Cargo fields for tower-fallback

* Cargo fields for zebra-chain

* Cargo fields for zebra-consensus

* Cargo fields for zebra-network

* Cargo fields for zebra-node-services

* Cargo fields for zebra-rpc

* Cargo fields for zebra-script

* Cargo fields for zebra-state

* Cargo fields for zebrad

* Cargo fields for zebra-test

* Include zebra-test version and path for zebra-chain

* Include zebra-chain version and path for zebra-node-services

* Include zebra-chain version and path for zebra-script

* Include zebra-chain and zebra-test versions and paths for zebra-state

* Include zebra-* versions and paths for zebra-consensus

* Include zebra-* and tower-fallback versions and paths for tower-batch-control

* Include tower-* versions and paths for zebra-consensus

* Fixed tower-fallback license string

* Remove versions for dev-dependencise

* Update zebra-rpc/Cargo.toml

Co-authored-by: teor <teor@riseup.net>

* tower-batch-control license MUST remain MIT only

* Align zebra-network with upstream tower's MIT license

* Update zebra-test/Cargo.toml

Co-authored-by: teor <teor@riseup.net>

* Update zebra-utils/Cargo.toml

Co-authored-by: teor <teor@riseup.net>

* fix(legal): Use correct licenses for tower-batch-control and zebra-network (#6927)

* Don't modify whitespace

* Resolve rustdoc and cargo-release warnings

* Add missing zebra internal dependency versions

* Workaround missing zebra-utils test dependency in zebrad test-only feature

* Fix missing zebra internal dependency versions

* Add README, home page, and search terms to every crate

* Fix README publish warnings

* Change tower crates to beta versions

* Add zebra-utils to the list of allowed unused dependencies

* Update tower-batch-control/Cargo.toml

* Update tower-fallback/Cargo.toml

---------

Co-authored-by: teor <teor@riseup.net>
2023-06-13 08:46:01 +00:00
dependabot[bot] 286ebdbf8c
build(deps): bump serde from 1.0.163 to 1.0.164 (#6875)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.163 to 1.0.164.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.163...v1.0.164)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-09 01:50:38 +00:00
Deirdre Connolly 9959a6ce04
Release 1.0.0-rc.9 (#6821)
* Bump semvers

* Update zebra-utils/README.md

* Updated mainnet checkpoints against commit b7029b8

* Add testnet checkpoints from b7029b8

* Bump zebrad rust-version to 1.70

* rust-version 1.68

Co-authored-by: teor <teor@riseup.net>

* Add CHANGELOG for 1.0.0-rc.9

* Bump estimated release height to within june 7th 2023 utc-4

* Add #6801 to CHANGELOG in anticipation

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Update breaking changes in 1.0.0-rc.9 changelog

* changelog: move #6801 to Fix

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Include #6832 in the changelog

* Add missing changes to changelog

* Remove #6801 from known issues in the README

* Use the latest bug template link

---------

Co-authored-by: teor <teor@riseup.net>
2023-06-08 03:44:28 +00:00
teor f3e330995f
fix(build): Fix new nightly clippy lints and cargo feature resolution (#6814)
* Implement minor and patch database format versions

* Log and update database format versions when opening database

* Refactor the current list of column families into a constant

* Open all available column families, including from future Zebra versions

* Refactor note commitment tree lookups to go through the height methods

* Make Sapling/Orchard note commitment tree lookup forwards compatible

* Ignore errors reading column family lists from disk

* Update format version comments and TODOs

* Correctly log newly created database formats

* Fix a new cargo lint about resolver versions

* cargo clippy --fix --all-features --all-targets

* cargo fmt --all

* Add missing tokio feature in the state, revealed by the new resolver

* Add missing dev dependencies in zebra-node-services

* Add a missing `tokio` feature from PR #6813

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-06-07 06:04:04 +00:00
teor d7b90552f3
fix(state): Avoid panicking on state errors during shutdown (#6828)
* Enable cancel_all_background_work() only on macOS

* Ignore expected "during shutdown" errors, and log other errors

* Disable cancel_all_background_work() but keep the updated docs and error handling

* Add the macOS shutdown crash to the README known issues

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-06-06 23:38:48 +00:00
teor 355f1233f5
change(db): Make the first stable release forward-compatible with planned state changes (#6813)
* Implement minor and patch database format versions

* Log and update database format versions when opening database

* Refactor the current list of column families into a constant

* Open all available column families, including from future Zebra versions

* Refactor note commitment tree lookups to go through the height methods

* Make Sapling/Orchard note commitment tree lookup forwards compatible

* Ignore errors reading column family lists from disk

* Update format version comments and TODOs

* Correctly log newly created database formats

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-06-06 21:18:57 +00:00
dependabot[bot] 815c77870d
build(deps): bump once_cell from 1.17.2 to 1.18.0 (#6816)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.17.2 to 1.18.0.
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.17.2...v1.18.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-06 11:14:53 +00:00
teor 04e96c2526
feat(net): Cache a list of useful peers on disk (#6739)
* Rewrite some state cache docs to clarify

* Add a zebra_network::Config.cache_dir for peer address caches

* Add new config test files and fix config test failure message

* Create some zebra-chain and zebra-network convenience functions

* Add methods for reading and writing the peer address cache

* Add cached disk peers to the initial peers list

* Add metrics and logging for loading and storing the peer cache

* Replace log of useless redacted peer IP addresses

* Limit the peer cache minimum and maximum size, don't write empty caches

* Add a cacheable_peers() method to the address book

* Add a peer disk cache updater task to the peer set tasks

* Document that the peer cache is shared by multiple instances unless configured otherwise

* Disable peer cache read/write in disconnected tests

* Make initial peer cache updater sleep shorter for tests

* Add unit tests for reading and writing the peer cache

* Update the task list in the start command docs

* Modify the existing persistent acceptance test to check for peer caches

* Update the peer cache directory when writing test configs

* Add a CacheDir type so the default config can be enabled, but tests can disable it

* Update tests to use the CacheDir config type

* Rename some CacheDir internals

* Add config file test cases for each kind of CacheDir config

* Panic if the config contains invalid socket addresses, rather than continuing

* Add a network directory to state cache directory contents tests

* Add new network.cache_dir config to the config parsing tests
2023-06-06 08:28:14 +00:00
dependabot[bot] f197dfb75e
build(deps): bump regex from 1.8.3 to 1.8.4 (#6817)
Bumps [regex](https://github.com/rust-lang/regex) from 1.8.3 to 1.8.4.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.8.3...1.8.4)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-06 05:41:25 +00:00
dependabot[bot] 3cf3798927
build(deps): bump chrono from 0.4.25 to 0.4.26 (#6799)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.25 to 0.4.26.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.25...v0.4.26)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-06 02:04:52 +00:00
Alfredo Garcia eb07bb31d6
rename(state): Rename state verifiers and related code (#6762)
* rename verifiers

* rename `PreparedBlock` to `SemanticallyVerifiedBlock`

* rename `CommitBlock` to `SemanticallyVerifiedBlock`

* rename `FinalizedBlock` to `CheckpointVerifiedBlock`

* rename `CommitFinalizedBlock` to `CommitCheckpointVerifiedBlock`

* rename `FinalizedWithTrees` to `ContextuallyVerifiedBlockWithTrees`

* rename `ContextuallyValidBlock` to `ContextuallyVerifiedBlock`

* change some `finalized` variables or function arguments to `checkpoint_verified`

* fix docs

* document the difference between `CheckpointVerifiedBlock` and `ContextuallyVerifiedBlock`

* fix doc links

* apply suggestions to request

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to service

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to finalized_state.rs and write.rs

Co-authored-by: Marek <mail@marek.onl>

* fmt

* change some more variable names

* change a few missing generics

* fix checkpoint log issue

* rename more `prepared` vars `semantically_verified`

* fix test regex

* fix test regex 2

---------

Co-authored-by: Marek <mail@marek.onl>
2023-06-01 12:29:03 +00:00
Deirdre Connolly af4d53122f
Replace update_chain_state_with ref with update_chain_tip_with in rustdoc (#6781)
* Replace update_chain_state_with ref with update_chain_tip_with in rustdoc

Noted in the audit

* Update zebra-state/src/request.rs

Co-authored-by: teor <teor@riseup.net>

* Add fully qualified rustdoc link

* Use correct Chain path

* Use a method that's actually on the struct (not in a private trait impl)

---------

Co-authored-by: teor <teor@riseup.net>
2023-05-31 02:38:56 +00:00
dependabot[bot] 6f8c98123e
build(deps): bump chrono from 0.4.24 to 0.4.25 (#6775)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.24 to 0.4.25.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.24...v0.4.25)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-30 12:57:25 +00:00
dependabot[bot] a6f9820aa3
build(deps): bump once_cell from 1.17.1 to 1.17.2 (#6776)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.17.1 to 1.17.2.
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.17.1...v1.17.2)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-30 02:02:27 +00:00
dependabot[bot] 61d3c34a78
build(deps): bump tokio from 1.28.1 to 1.28.2 (#6777)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.28.1 to 1.28.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.28.1...tokio-1.28.2)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-29 21:42:30 +00:00
dependabot[bot] e8ae564b1c
build(deps): bump regex from 1.8.2 to 1.8.3 (#6770)
Bumps [regex](https://github.com/rust-lang/regex) from 1.8.2 to 1.8.3.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.8.2...1.8.3)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-29 11:41:17 +00:00
dependabot[bot] dceed5a893
build(deps): bump proptest from 1.1.0 to 1.2.0 (#6758)
Bumps [proptest](https://github.com/proptest-rs/proptest) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/proptest-rs/proptest/releases)
- [Changelog](https://github.com/proptest-rs/proptest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/proptest-rs/proptest/compare/v1.1.0...v1.2.0)

---
updated-dependencies:
- dependency-name: proptest
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-25 20:44:16 +00:00
dependabot[bot] 3706ff6e67
build(deps): bump regex from 1.8.1 to 1.8.2 (#6748)
Bumps [regex](https://github.com/rust-lang/regex) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.8.1...1.8.2)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-23 23:23:16 +00:00
Pili Guerra ec2e9ca276
Delete outdated `TODOs` refering to closed issues (#6732)
* ZIPs were updated to remove ambiguity, this was tracked in #1267.

* #2105 was fixed by #3039 and #2379 was closed by #3069

* #2230 was a duplicate of #2231 which was closed by #2511

* #3235 was obsoleted by #2156 which was fixed by #3505

* #1850 was fixed by #2944, #1851 was fixed by #2961 and #2902 was fixed by #2969

* We migrated to Rust 2021 edition in Jan 2022 with #3332

* #1631 was closed as not needed

* #338 was fixed by #3040 and #1162 was fixed by #3067

* #2079 was fixed by #2445

* #4794 was fixed by #6122

* #1678 stopped being an issue

* #3151 was fixed by #3934

* #3204 was closed as not needed

* #1213 was fixed by #4586

* #1774 was closed as not needed

* #4633 was closed as not needed

* Clarify behaviour of difficulty spacing

Co-authored-by: teor <teor@riseup.net>

* Update comment to reflect implemented behaviour

Co-authored-by: teor <teor@riseup.net>

* Update comment to reflect implemented behaviour when retrying block downloads

Co-authored-by: teor <teor@riseup.net>

* Update `TODO` to remove closed issue and clarify when we might want to fix

Co-authored-by: teor <teor@riseup.net>

* Update `TODO` to remove closed issue and clarify what we might want to change in future

Co-authored-by: teor <teor@riseup.net>

* Clarify benefits of how we do block verification

Co-authored-by: teor <teor@riseup.net>

* Fix rustfmt errors

---------

Co-authored-by: teor <teor@riseup.net>
2023-05-23 03:33:14 +00:00
teor 3f13072c46
cleanup(rust): Simplify code using closure capture in Rust 2021 edition (#6737)
* Simplify code using closure capture in Rust 2021 edition

* clippy: manual_next_back and unit_arg

* cargo fmt --all
2023-05-22 19:22:13 +00:00
teor b1ce0e0894
fix(security): Stop panicking on state RPC or block requests with very large heights (#6699)
* fix panic in height

* Update security comments and turn literals into constants

* Add a test-only assertion that the maximum output index can't ever be reached

* Fix a MISSING_BLOCK_ERROR_CODE that was mistakenly 0

* Add production RPC tests with excessive heights

* Add and update snapshots for production RPCs

* Add excessive height tests for most getblocktemplate RPCs

* Use correct snapshot names by running `cargo insta review`

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-05-17 15:13:12 +00:00
dependabot[bot] 81e8198ad8
build(deps): bump serde from 1.0.162 to 1.0.163 (#6678)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.162 to 1.0.163.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.162...v1.0.163)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-16 23:54:09 +00:00
Arya 3779eb19a6
Tests error conversions (#6665) 2023-05-16 21:06:46 +00:00
dependabot[bot] 185d1384fb
build(deps): bump rocksdb from 0.20.1 to 0.21.0 (#6640)
Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.20.1 to 0.21.0.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.20.1...v0.21.0)

---
updated-dependencies:
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-16 01:04:02 +00:00
dependabot[bot] 989c5979fb
build(deps): bump serde from 1.0.160 to 1.0.162 (#6621)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.160 to 1.0.162.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.160...1.0.162)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-05-14 15:05:48 +00:00
teor d58d827af9
fix(clippy): Resolve nightly clippy lints (#6623)
* Clippy: fix "unused mut" warning in tests on nightly

* Remove redundant calls to default()
2023-05-14 01:34:03 +00:00
Alfredo Garcia 01908822f1
v1.0.0-rc.8 release (#6632)
* change versions

* add mainnet and testnet checkpoints

* add changelog

* add estimated height

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* cargo update

* rername deprecated function

* fix rustfmt

* update bans

* remove digest as dup dependency

* add dup dependency for all features

---------

Co-authored-by: teor <teor@riseup.net>
2023-05-11 22:15:12 +00:00
dependabot[bot] 2f23ec8630
build(deps): bump dirs from 5.0.0 to 5.0.1 (#6592)
Bumps [dirs](https://github.com/soc/dirs-rs) from 5.0.0 to 5.0.1.
- [Release notes](https://github.com/soc/dirs-rs/releases)
- [Commits](https://github.com/soc/dirs-rs/commits)

---
updated-dependencies:
- dependency-name: dirs
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-02 17:11:36 +00:00
dependabot[bot] 5fb48c2740
build(deps): bump tokio from 1.27.0 to 1.28.0 (#6572)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.27.0 to 1.28.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.27.0...tokio-1.28.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-02 03:38:12 +00:00
dependabot[bot] 1e1e2348bf
build(deps): bump regex from 1.7.3 to 1.8.1 (#6555)
Bumps [regex](https://github.com/rust-lang/regex) from 1.7.3 to 1.8.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.7.3...1.8.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-22 12:15:07 +00:00
Arya 8a9113c4e2
Leaves at least 1 block in the partial chain (#6552) 2023-04-20 21:06:23 +00:00
dependabot[bot] ef556ea06b
build(deps): bump metrics from 0.20.1 to 0.21.0 and metrics-exporter-prometheus from 0.11.0 to 0.12.0 (#6531)
* build(deps): bump metrics from 0.20.1 to 0.21.0

Bumps [metrics](https://github.com/metrics-rs/metrics) from 0.20.1 to 0.21.0.
- [Release notes](https://github.com/metrics-rs/metrics/releases)
- [Changelog](https://github.com/metrics-rs/metrics/blob/main/release.toml)
- [Commits](https://github.com/metrics-rs/metrics/compare/metrics-v0.20.1...metrics-v0.21.0)

---
updated-dependencies:
- dependency-name: metrics
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* bumps metrics-exporter-prometheus version and adds portable-atomic 0.3.19 to deny.toml skip list

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: arya2 <aryasolhi@gmail.com>
2023-04-19 22:52:21 +00:00
Arya 9820780a35
v1.0.0-rc.7 release (#6542)
* updates changelog and bumps versions

* updates README.md

* Updates Rust version in readme

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Removes the initial label from entries

* Updates testnet checkpoints

* update mainnet checkpoints

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-04-19 03:25:51 +00:00
teor 3ba334bf20
change(deps): Upgrade shared dependencies to match `zcashd` 5.5.0 (#6536)
* Cleanup module docs and imports for sapling::arbitrary

* Upgrade dependencies shared with zcashd 5.5.0

```sh
cargo upgrade --workspace << EOF
bellman
bls12_381
group
halo2
halo2_proofs
jubjub
orchard
redjubjub
zcash_address
zcash_note_encryption
zcash_primitives
zcash_proofs
EOF
```

* Replace orchard::from_bytes_wide() with ff::from_uniform_bytes()

* Revert "disable test (#6285)" to re-enable verify_generated_halo2_proofs

This reverts commit 4d4accc0b9.
2023-04-18 10:11:38 +00:00
Arya 6fdd02220e
fix(rpc): Omit transactions with transparent coinbase spends that are immature at the next block height from block templates (#6510)
* Adds `maturity_height` to VerifiedUnminedTx

Filters out transactions that are invalid at next_block_height in getblocktemplate method

* Adds unit testing

* rustfmt

* rejects txs with immature coinbase spends from mempool

* Condenses fns for transparent coinbase spend check

* Updates calls to VerifiedUnminedTx::new()

* Update zebra-chain/src/transparent/utxo.rs

* Applies suggestions from code review
2023-04-18 03:43:39 +00:00
Arya 6ed4b6632d
checks non-finalized chain for unspent utxos before checking disk (#6513) 2023-04-14 10:03:44 +00:00
dependabot[bot] 225e5ea39a
build(deps): bump serde_json from 1.0.95 to 1.0.96 (#6502)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.95 to 1.0.96.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.95...v1.0.96)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-14 01:21:50 +00:00
teor 166526a088
feat(ui): Add a terminal-based progress bar to Zebra (#6235)
* Implement Display and to_string() for NetworkUpgrade

* Add a progress-bar feature to zebrad

* Add the progress bar writer to the tracing component

* Add a block progress bar transmitter

* Correctly shut down the progress bar, and shut it down on an interrupt

* Make it clearer that the progress task never exits

* Add a config for writing logs to a file

* Add a progress-bar feature to zebra-network

* Add a progress bar for the address book size

* Add progress bars for never attempted and failed peers

* Add an optional limit and label to connection counters

* Add open connection progress bars

* Improve CheckpointList API and CheckpointVerifier debugging

* Add checkpoint index and checkpoint queue progress bars

* Security: Limit the number of non-finalized chains tracked by Zebra

* Make some NonFinalizedState methods available with proptest-impl

* Add a non-finalized chain count progress bar

* Track the last fork height for newly forked chains

* Add a should_count_metrics to Chain

* Add a display method for PartialCumulativeWork

* Add a progress bar for each chain fork

* Add a NonFinalizedState::disable_metrics() method and switch to using it

* Move metrics out of Chain because we can't update Arc<Chain>

* Fix: consistently use best chain order when searching chains

* Track Chain progress bars in NonFinalizedState

* Display work as bits, not a multiple of the target difficulty

* Handle negative fork lengths by reporting "No fork"

* Correctly disable unused fork bars

* clippy: rewrite using `match _.cmp(_) { ... }`

* Initial mempool progress bar implementation

* Update Cargo.lock

* Add the actual transaction size as a description to the cost bar

* Only show mempool progress bars after first activation

* Add queued and rejected mempool progress bars

* Clarify cost note is actual size

* Add tracing.log_file config and progress-bar feature to zebrad docs

* Derive Clone for Chain

* Upgrade to howudoin 0.1.2 and remove some bug workarounds

* Directly call the debug formatter to Display a Network

Co-authored-by: Arya <aryasolhi@gmail.com>

* Rename the address count metric to num_addresses

Co-authored-by: Arya <aryasolhi@gmail.com>

* Simplify reverse checkpoint lookup

Co-authored-by: Arya <aryasolhi@gmail.com>

* Simplify progress bar shutdown code

Co-authored-by: Arya <aryasolhi@gmail.com>

* Remove unused MIN_TRANSPARENT_TX_MEMPOOL_SIZE

* Document that the progress task runs forever

* Fix progress log formatting

* If progress-bar is on, log to a file by default

* Create missing directories for log files

* Add file security docs for running Zebra with elevated permissions

* Document automatic log file, spell progress-bar correctly

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2023-04-13 08:42:17 +00:00
dependabot[bot] e6bd34d0bf
build(deps): bump serde from 1.0.159 to 1.0.160 (#6480)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.159 to 1.0.160.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.159...v1.0.160)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-12 03:07:44 +00:00
teor 00d46e1aa9
security(state): Limit the number of non-finalized chains tracked by Zebra (#6447)
* Security: Limit the number of non-finalized chains tracked by Zebra

* Use NonFinalizedState::chain_iter() to access private field

* Reverse the order of chain_iter()
2023-04-04 01:05:21 +00:00
Arya d7842bd467
fix(rpc): Check that mempool transactions are valid for the state's chain info in getblocktemplate (#6416)
* check last seen tip hash from mempool in fetch_mempool_transactions()

* Moves last_seen_tip_hash to ActiveState

* fixes tests and tests fixes

* continues to the next iteration of the loop to make fresh state and mempool requests if called with a long poll id

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* adds allow[unused_variable) for linter

* expects a chain tip when not(test)

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Addresses comments in code review

* - Removes second call to `last_tip_change()`

- Fixes tests using enabled mempool

* Adds note about chain tip action requirement to test method `enable()`

* updates doc comment

* Update zebrad/src/components/mempool.rs

Co-authored-by: teor <teor@riseup.net>

* fixes test

---------

Co-authored-by: teor <teor@riseup.net>
2023-04-03 23:22:07 +00:00
teor f556c4a46d
Remove an outdated assertion in a comment, replace with a test TODO (#6450) 2023-04-03 23:20:08 +00:00
dependabot[bot] 3b259b1746
build(deps): bump futures from 0.3.27 to 0.3.28 (#6442)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.27 to 0.3.28.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.27...0.3.28)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-03 04:00:53 +00:00
dependabot[bot] 5db2243c25
build(deps): bump tempfile from 3.4.0 to 3.5.0 (#6432)
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/commits)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-30 04:26:05 +00:00
dependabot[bot] 5a901a77d1
build(deps): bump serde_json from 1.0.94 to 1.0.95 (#6425)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.94 to 1.0.95.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.94...v1.0.95)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-03-30 02:13:30 +00:00
Marek 2a48d4cf25
change(chain): Refactor the handling of height differences (#6330)
* Unify the `impl`s of `Sub` and `Add` for `Height`

* Adjust tests for `Height` subtraction

* Use `Height` instead of `i32`

* Use `block:Height` in RPC tests

* Use `let .. else` statement

Co-authored-by: Arya <aryasolhi@gmail.com>

* Update zebra-consensus/src/block/subsidy/general.rs

* Refactor the handling of height differences

* Remove a redundant comment

* Update zebrad/src/components/sync/progress.rs

Co-authored-by: Arya <aryasolhi@gmail.com>

* Update progress.rs

* impl TryFrom<u32> for Height

* Make some test assertions clearer

* Refactor estimate_up_to()

* Restore a comment that was accidentally removed

* Document when estimate_distance_to_network_chain_tip() returns None

* Change HeightDiff to i64 and make Height.sub(Height) return HeightDiff (no Option)

* Update chain tip estimates for HeightDiff i64

* Update subsidy for HeightDiff i64

* Fix some height calculation test edge cases

* Fix the funding stream interval calculation

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2023-03-29 23:06:31 +00:00
dependabot[bot] a38d6c6e9a
build(deps): bump tokio from 1.26.0 to 1.27.0 (#6426)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.26.0 to 1.27.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.26.0...tokio-1.27.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-29 01:50:47 +00:00
dependabot[bot] 536e3a1785
build(deps): bump serde from 1.0.158 to 1.0.159 (#6424)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.158 to 1.0.159.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.158...v1.0.159)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-28 18:35:53 +00:00
dependabot[bot] 96e5b541a4
build(deps): bump proptest from 0.10.1 to 1.1.0 (#6415)
* build(deps): bump proptest from 0.10.1 to 1.1.0

Bumps [proptest](https://github.com/proptest-rs/proptest) from 0.10.1 to 1.1.0.
- [Release notes](https://github.com/proptest-rs/proptest/releases)
- [Changelog](https://github.com/proptest-rs/proptest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/proptest-rs/proptest/compare/v0.10.1...v1.1.0)

---
updated-dependencies:
- dependency-name: proptest
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add a quick-error duplicate dependency exception

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2023-03-28 04:13:26 +00:00
dependabot[bot] e982a437f4
build(deps): bump regex from 1.7.2 to 1.7.3 (#6412)
Bumps [regex](https://github.com/rust-lang/regex) from 1.7.2 to 1.7.3.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.7.2...1.7.3)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-28 01:42:48 +00:00
dependabot[bot] a8321f95bc
build(deps): bump indexmap from 1.9.2 to 1.9.3 (#6413)
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.9.2 to 1.9.3.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/bluss/indexmap/compare/1.9.2...1.9.3)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-28 01:41:36 +00:00
Arya 8ba3fd921a
change(rpc): Add confirmations to getrawtransaction method response (#6287)
* adds confirmation field to getrawtransaction

* Updates tests

* Adds comment about correctness

* Apply suggestion revisions from review

* fixes overflow bug and adds test for valid confirmations value

* Update test to check that confirmations isn't too high

* Update transaction request to return confirmations

* Applies suggestions from PR review

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* fixes test

* restore derives that were lost in a bad merge

---------

Co-authored-by: teor <teor@riseup.net>
2023-03-26 23:53:44 +00:00
Arya 571cbfba7a
change(state): Stop re-downloading blocks that are in non-finalized side chains (#6335)
* Adds 'Contains' request in state, and:

- adds finalized block hashes to sent_blocks

- replaces Depth call with Contains in sync, inbound, and block verifier

* removes unnecessary From impl

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* updates references to Request::Contains

* Renames zs::response::BlockLocation to KnownBlocks

* Updates AlreadyInChain error

* update docs for sent_hashes.add_finalized

* Update zebra-consensus/src/block.rs

Co-authored-by: teor <teor@riseup.net>

* Update comment for `sent_blocks` field in state service

* update KnownBlock request to check the non-finalized state before responding that a block is in the queue

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* update references to renamed method

* Clear sent_blocks when there's a reset

* Move self.finalized_block_write_sender.is_none() to can_fork_chain_at

* revert changes related to checking queue

---------

Co-authored-by: teor <teor@riseup.net>
2023-03-24 07:10:07 +00:00
Deirdre Connolly c3273e4a61
v1.0.0-rc.6 release (#6314)
* Package and version bumps

* Updated mainnet checkpoints

* Update test-checkpoints.txt

* Bump zebrad rust-version to 1.68

* update the changelog

* add suggestions

Co-authored-by: teor <teor@riseup.net>

* fix typo

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* add suggestions from review

Co-authored-by: teor <teor@riseup.net>

* add suggestion

Co-authored-by: teor <teor@riseup.net>

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-03-24 01:24:29 +00:00
dependabot[bot] fc32e68cf4
build(deps): bump insta from 1.28.0 to 1.29.0 (#6395)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.28.0 to 1.29.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.28.0...1.29.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-23 18:25:07 +00:00
dependabot[bot] 7eb08208ea
build(deps): bump dirs from 4.0.0 to 5.0.0 (#6360)
* build(deps): bump dirs from 4.0.0 to 5.0.0

Bumps [dirs](https://github.com/soc/dirs-rs) from 4.0.0 to 5.0.0.
- [Release notes](https://github.com/soc/dirs-rs/releases)
- [Commits](https://github.com/soc/dirs-rs/commits)

---
updated-dependencies:
- dependency-name: dirs
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Allow a different dirs-sys version used for a different purpose

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2023-03-22 19:37:46 +00:00
dependabot[bot] 45a96b5adc
build(deps): bump serde from 1.0.156 to 1.0.158 (#6358)
* build(deps): bump serde from 1.0.156 to 1.0.158

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.156 to 1.0.158.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.156...v1.0.158)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make MERKLE_DEPTH into a u8 to avoid a spurious serde_derive warning

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2023-03-22 17:12:34 +00:00
dependabot[bot] 06f8f8b7dd
build(deps): bump regex from 1.7.1 to 1.7.2 (#6371)
Bumps [regex](https://github.com/rust-lang/regex) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.7.1...1.7.2)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-21 23:17:58 +00:00
dependabot[bot] a97a9752d7
build(deps): bump thiserror from 1.0.39 to 1.0.40 (#6359)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.39 to 1.0.40.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.39...1.0.40)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-20 19:07:11 +00:00
Arya 6d0ce112db
change(state): Condense CodeTimer::start() and Span::current() calls in state service (#6337)
* starts the timer at the top of the call() method

* assigns 'span' at the top of the call methods
2023-03-18 15:21:11 +00:00
dependabot[bot] dbdf617124
build(deps): bump serde from 1.0.154 to 1.0.156 (#6316)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.154 to 1.0.156.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.154...v1.0.156)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-15 23:10:09 +00:00
dependabot[bot] 360c974829
build(deps): bump chrono from 0.4.23 to 0.4.24 (#6300)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.23 to 0.4.24.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.23...v0.4.24)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-03-15 17:29:10 +00:00
dependabot[bot] c75fb63385
build(deps): bump futures from 0.3.26 to 0.3.27 (#6299)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.26 to 0.3.27.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.26...0.3.27)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-14 22:21:15 +00:00
Alfredo Garcia 4dedffebbc
feat(elasticsearch): Export block data to elasticsearch database (#6274)
* add initial ES support

* hide elasticsearch behind feture, add config

* make the builds and clippy happy

* move ES code to a function

* change database name

* fix database name and panic

* increase close to tip interval

* update deny for elastic

* remove a block clone

* try to fix builds

* fix builds 2

* refactor some imports and unwraps

* typo

* change argument of elasticsearch function

* move elastic call to the end of `commit_finalized_direct`

* get height from block
2023-03-13 21:13:30 +00:00
dependabot[bot] 542f838acd
build(deps): bump serde from 1.0.152 to 1.0.154 (#6284)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.152 to 1.0.154.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.152...v1.0.154)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-10 12:35:12 +00:00
dependabot[bot] fc96bfe370
build(deps): bump rayon from 1.6.1 to 1.7.0 (#6268)
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.6.1...rayon-core-v1.7.0)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-07 14:22:51 +00:00
dependabot[bot] 5f5b5352fa
build(deps): bump thiserror from 1.0.38 to 1.0.39 (#6266)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.38 to 1.0.39.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.38...1.0.39)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-06 22:36:30 +00:00
dependabot[bot] a1105c0672
build(deps): bump tokio from 1.25.0 to 1.26.0 (#6263)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.25.0...tokio-1.26.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-02 20:51:06 +00:00
teor f858aab1d9
improve CommitBlockError message: include underlying ValidateContextError message (#6251)
Co-authored-by: dimxy <dimxy@komodoplatform.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-03-02 06:30:01 +00:00
dependabot[bot] 52c627d21d
build(deps): bump tempfile from 3.3.0 to 3.4.0 (#6240)
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/commits)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-28 10:02:51 +00:00
teor 98c634bbee
fix(deps): Remove unused dependencies, and check for them in CI (#6216)
* Remove unused dependencies

* Check for newly unused dependencies in CI

* Use the correct grep command

* Always show cargo machete output

* Ignore cargo machete exit status, use grep instead

* Use if instead of && and subshells

* Invert if logic
2023-02-26 23:42:36 +00:00
Marek 9d2b912840
Release v1.0.0-rc.5 (#6213)
* Bump updated crates

* Bump crates that depend on updated crates

* Bump the user agent

* Bump `README.md`

* Bump `docker.md`

* Remove closed known issues from "README.md"

* Update `CHANGELOG.md`

* Apply suggestions from code review

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>

* Remove the bump of `once_cell` from `CHANGELOG.md`

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-02-24 23:08:16 +00:00
teor ec43d63ed2
change(log): Log a cute message for blocks that were mined by Zebra (off by default) (#6098)
* Mark Zebra coinbase transactions with extra coinbase data

* Log when we commit a block mined by Zebra to our state

* Reduce logging instrumentation during block writes

* Remove debug types in Zebra block log

* Add network and commit to write task logs

* Apply an allow-list before we log arbitrary user messages from blocks

* Rate-limit Zebra mined block logging to once every 1000 blocks

* Add mining configs for extra coinbase data and imitating zcashd, but don't use them yet

* Check CoinbaseData size limit when building transparent transactions

* Replace LIKE_ZCASHD constants with a config

* Take extra coinbase data from the configured string

* Update the zebrad configs in the tests with new config fields
2023-02-23 00:10:11 +00:00
teor 4daedbc789
fix(consensus): Check that Zebra's state contains the social consensus chain on startup (#6163)
* Allow missing docs directly on derived error types

* Make Request::BestChainBlockHash redirect to the ReadStateService

* Re-write the checkpoint_sync documentation based on the latest consensus rules

* Expose the underlying iterator for CheckpointList

* Validate existing state block hashes at startup, but ignore the result

* Monitor state block hash checkpoint task in the start command

* Fix indentation

* Make logging consistent

* Explain the config needed for full security

* Tidy required checkpoints docs, expand other docs

* Add security and deprecation changelog entries

* Replace task handle vector with a struct

* Add a test that this consensus-critical code actually runs and finishes

* Make some state methods and types available in tests

* Add missing docs

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-02-21 05:30:29 +00:00
dependabot[bot] 7833c7a362
build(deps): bump insta from 1.26.0 to 1.28.0 (#6181)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.26.0 to 1.28.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.26.0...1.28.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-17 01:16:49 +00:00
dependabot[bot] e78c5c5d58
build(deps): bump once_cell from 1.17.0 to 1.17.1 (#6161)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.17.0 to 1.17.1.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.17.0...v1.17.1)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-16 03:04:22 +00:00
teor ae21e36018
fix(rpc): Add `getblock` RPC fields to support the latest version of `zcash/lightwalletd` (#6134)
* Stabilise the BestChainBlockHash state request

* Always include the block hash in the getblock RPC response

* Make the lightwalletd integration tests compatible with zcash/lightwalletd

* Update getblock RPC snapshots

* Return the correct missing block error code
2023-02-14 08:52:58 +00:00
teor 9452487c61
fix(state): Fix minute-long delays in block verification after a chain fork (#6122)
* Split Chain fields into sections

* Replace Chain.sprout_note_commitment_tree with a lookup method

* Add TODOs

* Show full debug info when tests fail because chains aren't equal

* Print sprout and sapling tree Nodes as hex when debugging

* Correctly revert temporary finalized tip trees and anchors

* Fix tests

* Refactor removal functions

* Replace the Chain.sapling_note_commitment_tree field with a lookup method

* Replace the Chain.orchard_note_commitment_tree field with a lookup method

* Replace the Chain.history_tree field with a lookup method and remove redundant code

* Update comments

* Ignore clippy::unwrap_in_result

* Remove redundant fork() Result

* Put conditional code in blocks

* fastmod history_tree_at_tip history_block_commitment_tree zebra-state
2023-02-13 21:44:31 +00:00
dependabot[bot] 09faf48e1f
build(deps): bump rocksdb from 0.19.0 to 0.20.1 (#6144)
* build(deps): bump rocksdb from 0.19.0 to 0.20.1

Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.19.0 to 0.20.1.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.19.0...v0.20.1)

---
updated-dependencies:
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Ignore duplicate bindgen dependency until zcash_script updates

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2023-02-13 20:01:45 +00:00
Arya e1b8c43cfa
corrects panic/timer msg for CheckBestChainTipNullifiersAndAnchors request (#6135) 2023-02-10 09:54:11 +00:00
teor daba6d7744
add(rpc): Add extra `getblock` RPC fields used by some mining pools (#6097)
* clippy: remove unnecessary return statement

* Add hash, height, and confirmations fields to getblock RPC

* Remove a test that is already checked by snapshots

* Document the performance requirements of the getblock RPC

* Update snapshots, use new naming scheme

* Fix off-by-one error in confirmations

* Fix spelling mistakes
2023-02-07 01:25:34 +00:00
dependabot[bot] 3f66b82590
build(deps): bump futures from 0.3.25 to 0.3.26 (#6058)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.25 to 0.3.26.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.25...0.3.26)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-01 08:55:37 +00:00
dependabot[bot] 9b61643df2
build(deps): bump tokio from 1.24.2 to 1.25.0 (#6050)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.2 to 1.25.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits/tokio-1.25.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-01 01:00:21 +00:00
teor 8390e4e0cd
refactor(state): Simplify difficulty and median-time-past state and mempool requests (#6031)
* Clarify function docs, rename variables, and fix log typos

* Add a ReadState best chain clone method, but don't use it yet

* Use the new calculate_median_time_past() function in existing code

* Skip a state request if the lock time is a height

* Remove dummy arguments and extra blocks from median-time-past calculation

* Update tests to remove requests that are no longer sent

* Simplify getting the best chain

Co-authored-by: Arya <aryasolhi@gmail.com>

* Clarify some function docs

Co-authored-by: Arya <aryasolhi@gmail.com>

* assigns `next_median_time_past` value from if statement

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2023-01-31 20:42:11 +00:00
dependabot[bot] bd29cf7b44
build(deps): bump rlimit from 0.9.0 to 0.9.1 (#6051)
Bumps [rlimit](https://github.com/Nugine/rlimit) from 0.9.0 to 0.9.1.
- [Release notes](https://github.com/Nugine/rlimit/releases)
- [Changelog](https://github.com/Nugine/rlimit/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Nugine/rlimit/compare/v0.9.0...v0.9.1)

---
updated-dependencies:
- dependency-name: rlimit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-31 20:38:10 +00:00
teor f3eaa040be
Increase legacy chain limit to 100,000 (#6053) 2023-01-31 20:36:59 +00:00
Alfredo Garcia 9b1d26538a
Release v1.0.0-rc.4 (#6045)
* bump v1.0.0-rc.4

* Add new mainnet and testnet checkpoints generated using zcashd

* make changes to changelog from review

* move item to new section

* change section name

---------

Co-authored-by: teor <teor@riseup.net>
2023-01-31 04:53:27 +00:00
teor e20cf957e3
fix(consensus): Verify the lock times of mempool transactions (#6027)
* Implement the BestChainNextMedianTimePast state request

* Verify the lock times of mempool transactions

* Document that the mempool already handles lock time rejections correctly

* Fix existing tests

* Add new mempool lock time success and failure tests
2023-01-27 21:46:51 +00:00
dependabot[bot] 2018d49907
build(deps): bump tokio from 1.24.1 to 1.24.2 (#5995)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.1 to 1.24.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-18 21:16:37 +00:00
teor 8c451968ee
fix(diagnostics): Hex-encode debug format of commitments, nonces, and nullifiers (#5960)
* Format commitments and nonces as hex

* Format Sprout and Sapling nullifiers as hex when debugging

* Format Sprout commitments as hex when debugging

* Format redpallas keys as hex when debugging

* Update code that we're going to delete in the next PR anyway
2023-01-17 13:57:22 +00:00
teor dcf30679bf
fix(rpc): Calculate getblocktemplate RPC testnet min and max times correctly (#5925)
* Clean up some getblocktemplate difficulty code and tests

* Fix minimum difficulty adjustment

* Use clamp rather than max/min

Co-authored-by: Arya <aryasolhi@gmail.com>

* Remove unused imports

* Document the Zebra-specific standard rule that allows testnet miners time to mine a block

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
2023-01-17 07:08:43 +00:00
dependabot[bot] 402ed3eaac
build(deps): bump mset from 0.1.0 to 0.1.1 (#5952)
Bumps [mset](https://github.com/lonnen/mset) from 0.1.0 to 0.1.1.
- [Release notes](https://github.com/lonnen/mset/releases)
- [Commits](https://github.com/lonnen/mset/commits)

---
updated-dependencies:
- dependency-name: mset
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-17 01:12:14 +00:00
Arya 3cbee9465a
change(rpc): Add proposal capability to getblocktemplate (#5870)
* adds ValidateBlock request to state

* adds `Request` enum in block verifier

skips solution check for BlockProposal requests

calls CheckBlockValidity instead of Commit block for BlockProposal requests

* uses new Request in references to chain verifier

* adds getblocktemplate proposal mode response type

* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state

* Adds PR review revisions

* adds info log in CheckBlockProposalValidity

* Reverts replacement of match statement

* adds `GetBlockTemplate::capabilities` fn

* conditions calling checkpoint verifier on !request.is_proposal

* updates references to validate_and_commit_non_finalized

* adds snapshot test, updates test vectors

* adds `should_count_metrics` to NonFinalizedState

* Returns an error from chain verifier for block proposal requests below checkpoint height

adds feature flags

* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD

* adds back block::Request to zebra-consensus lib

* updates snapshots

* Removes unnecessary network arg

* skips req in tracing intstrument for read state

* Moves out block proposal validation to its own fn

* corrects `difficulty_threshold_is_valid` docs

adds/fixes some comments, adds TODOs

general cleanup from a self-review.

* Update zebra-state/src/service.rs

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* check best chain tip

* Update zebra-state/src/service.rs

Co-authored-by: teor <teor@riseup.net>

* Applies cleanup suggestions from code review

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-11 23:39:51 +00:00
dependabot[bot] 943a72a0bb
build(deps): bump insta from 1.24.1 to 1.26.0 (#5931)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.24.1 to 1.26.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.24.1...1.26.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-10 10:11:27 +00:00
dependabot[bot] 54604b14f4
build(deps): bump regex from 1.7.0 to 1.7.1 (#5932)
Bumps [regex](https://github.com/rust-lang/regex) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.7.0...1.7.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-10 10:09:40 +00:00
teor 11baba56cb
Release Zebra 1.0.0-rc.3 (#5928)
* Changelog with trivial entries

* Delete trivial entries

* Summarise known issues in README, but don't change the list yet

* Add block timeouts to known issues

* Update the release template to add missing version files

* Bump crate versions

* Add the required Rust version to the release checklist

* Update the Rust version requirement to 1.65, Zebra now uses `let ... else ...`

* Update checkpoints

* Add checkpoints to the CHANGELOG

* Breaking Rust compiler version change

* Clarify the latest stable supported rust version
2023-01-10 03:48:03 +00:00
dependabot[bot] dbc3a070b1
build(deps): bump tokio from 1.23.1 to 1.24.1 (#5927)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.23.1 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.23.1...tokio-1.24.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-06 23:16:02 +00:00
teor cda0a0b475
refactor(state): Make implementation of block consensus rules clearer (#5915)
* Replace duplicate code with read::hash_by_height()

* Rename an unclear function to difficulty_threshold_and_time_are_valid()
2023-01-06 06:59:55 +00:00
dependabot[bot] 842365b5c8
build(deps): bump tokio from 1.23.0 to 1.23.1 (#5920)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.23.0 to 1.23.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.23.0...tokio-1.23.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-06 04:13:57 +00:00
teor 2f23de80f1
fix(clippy): Fix nightly clippy redundant code lints (#5916)
* Remove needless borrows

* Remove redundant into_iter()

* Remove unnecessary clone()
2023-01-05 15:46:04 +00:00
dependabot[bot] f3f8254e4a
build(deps): bump insta from 1.23.0 to 1.24.1 (#5913)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.23.0 to 1.24.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.23.0...1.24.1)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-04 22:17:21 +00:00
dependabot[bot] 53e836efdc
build(deps): bump once_cell from 1.16.0 to 1.17.0 (#5903)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.16.0 to 1.17.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.16.0...v1.17.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-03 04:23:09 +00:00
dependabot[bot] 0ff86153d0
build(deps): bump rlimit from 0.8.3 to 0.9.0 (#5900)
Bumps [rlimit](https://github.com/Nugine/rlimit) from 0.8.3 to 0.9.0.
- [Release notes](https://github.com/Nugine/rlimit/releases)
- [Changelog](https://github.com/Nugine/rlimit/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Nugine/rlimit/compare/v0.8.3...v0.9.0)

---
updated-dependencies:
- dependency-name: rlimit
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-28 22:30:47 +00:00
dependabot[bot] d9300a6c1b
build(deps): bump serde from 1.0.151 to 1.0.152 (#5898)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.151 to 1.0.152.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.151...v1.0.152)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-27 19:46:22 +00:00
dependabot[bot] 5b8133bd02
build(deps): bump serde from 1.0.150 to 1.0.151 (#5877)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.150 to 1.0.151.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.150...v1.0.151)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 23:22:41 +00:00
dependabot[bot] 0eb944eec3
build(deps): bump thiserror from 1.0.37 to 1.0.38 (#5880)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.37 to 1.0.38.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.37...1.0.38)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 20:42:02 +00:00
teor 80a6d3cdab
change(rpc): Return from long polling immediately when the chain tip changes (#5862)
* Add constants and fix comments for mempool timer / state watch

* Add a best_tip_changed() method to trait ChainTip

* Check for chain tip changes using a future, and return on error

* Ignore state changes before the most recent state fetch

* Add a submit old field to the getblocktemplate RPC during long polling

* Make the submit_old field optional, rather than `null` in JSON

* Update some TODOs

* Add long polling snapshot tests, use RON for deserialized coinbase fields

* Clarify some comments

* Simplify the BestTipChanged future implementations

* Fix some comment typos
2022-12-15 15:33:00 +00:00
teor f7011a903e
feature(rpc): Add basic long polling support to the `getblocktemplate` RPC (#5843)
* fastmod longpollid long_poll_id

* Fix a comment

* Implement inefficient long polling
2022-12-15 00:30:37 +00:00
teor e9d6e975b4
change(rpc): Refactor get block template RPC into stages (#5837)
* Add some TODOs

* Move and rename height_from_signed_int()

* Move get_block_template() support functions to a submodule

* Fix incorrect P2SH comments and logs

* Split initial checks into their own functions

* Split state fetch into its own function, do some cleanup

* Move get_block_template_opts to get_block_template::parameters

* Fix and simplify test imports

* Rename block_height to next_block_height

* Rename to chain_tip_and_local_time to make it clear what it contains

* Split fetching mempool transactions out, include them in long poll id

* Refactor coinbase generation

* Split default root calculation into a separate function

* Use DateTime32 for getblocktemplate times

* Use typed difficulty fields rather than strings

* Split out a generate coinbase and roots function

* Move GetBlockTemplate construction into a method

* Document what happens to unusual difficulty values

* Clean up some TODOs

* fastmod check_address check_miner_address

* cargo fmt --all

* Fix an incorrect panic message

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-12-13 21:25:04 +00:00
dependabot[bot] 68d0c55894
build(deps): bump serde from 1.0.149 to 1.0.150 (#5846)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.149 to 1.0.150.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.149...v1.0.150)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-13 02:54:21 +00:00
teor cb1045ae5f
change(mempool): Re-verify mempool transactions after a chain fork, rather than re-downloading them all (#5841)
* Move Drop from mempool::ActiveState to mempool::Downloads, to avoid bugs

* Re-verify mempool transactions after a fork

And add a marker struct for mempool download cancellation.

* Update README based on recent mitigations for some issues, tidy format

* Make mempool proptests easier to debug

* Make UnminedTx Display text much smaller

* Update tests for mempool transaction re-verification after forks

* Retry all stored and pending transactions

* Fix a test to check for mempool reset retries
2022-12-12 23:19:45 +00:00
dependabot[bot] 75f2f02ef7
build(deps): bump rayon from 1.6.0 to 1.6.1 (#5845)
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.6.0 to 1.6.1.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.6.0...rayon-core-v1.6.1)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-12 21:27:53 +00:00
dependabot[bot] baf248aa1c
build(deps): bump insta from 1.22.0 to 1.23.0 (#5847)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.22.0 to 1.23.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.22.0...1.23.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-12 19:11:53 +00:00
dependabot[bot] f1e3fda7b8
build(deps): bump insta from 1.21.2 to 1.22.0 (#5790)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.21.2...1.22.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-12-09 16:22:26 +00:00
teor 2041fda7bb
fix(test): Reduce verbose test logs (#5825)
* Remove verbose continuous_blockchain test logs

* Downgrade verbose zebra-network logs to debug

* Downgrade some state logs to debug during tests

* Mark were we would add always-on log filters, if we needed to

* Reduce the number of mempool property tests, to reduce logging
2022-12-08 23:56:01 +00:00
Arya 77b85cf767
change(rpc): Adds `getmininginfo`, `getnetworksolps` and `getnetworkhashps` methods (#5808)
* adds type and stub

* adds:
- SolutionRate state request

- getnetworksolps, getnetworkhashps, & getmininginfo RPCs

- vectors tests

* adds snapshot tests

updates ReadRequest::SolutionRate doc link

* removes random slash in doc comment

moves snapshot tests up where it can use the populated state service

* adds snapshots

* updates doc comments

* applies `num_blocks` default in RPC instead of `solution_rate`

* adds # Correctness comment

* Add testnet field to getmininginfo response

* use PartialCumulativeWork instead of u128

* document why `solution_rate` takes an extra block

* add comment explaining why the work for the last block in the iterator is not added to `total_work`

* use `as_u128` method instead of deref for PartialCumulativeWork

* Updates `chain` field of getmininginfo response

* Updates snapshots

Adds "arbitrary_precision" feature to serde_json in zebra-rpc
2022-12-08 19:56:14 +00:00
dependabot[bot] cefc98baac
build(deps): bump serde from 1.0.148 to 1.0.149 (#5788)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.148 to 1.0.149.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.148...v1.0.149)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-08 04:19:28 +00:00
teor 09836d2800
fix(clippy): Put Rust format variables inline (#5783)
* cargo clippy --fix --all-features --all-targets

With rustc 1.67.0-nightly (234151769 2022-12-03)

* cargo fmt --all
2022-12-08 01:05:57 +00:00
Alfredo Garcia 678c519032
change(rpc): Populate `blockcommitmenthash` and `defaultroot` fields in the getblocktemplate RPC (#5751)
* populate `blockcommitmenthash` and `defaultroot` missing fields

* remove assertion line manually from snaps

* fix some imports and docs

* fix some docs

* add a consistency check

* Rename a constant to FINALIZED_STATE_QUERY_RETRIES and use it everywhere

* Move tip query inside retry, split tip into tip_height and tip_hash

* Return retry failures rather than panicking

* Query relevant chain inside the retry

* Check the entire state for consistency, not just the finalized tip

Co-authored-by: teor <teor@riseup.net>
2022-12-07 22:39:11 +00:00
dependabot[bot] a5d04746a1
build(deps): bump tokio from 1.22.0 to 1.23.0 (#5802)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.22.0 to 1.23.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.22.0...tokio-1.23.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-07 22:38:28 +00:00
Deirdre Connolly cc6ff0f422
release(1.0.0-rc.2): updated versions and CHANGELOG.md (#5784)
* release(1.0.0-rc.2): updated versions and CHANGELOG.md

I haven't touched zebra-dependencies-for-audit.md.

* Update CHANGELOG.md

* Update CHANGELOG.md

Co-authored-by: Arya <aryasolhi@gmail.com>

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* update 1.0.0-rc.2 date

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

Co-authored-by: Arya <aryasolhi@gmail.com>

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-12-06 21:35:44 +00:00
teor 21c916f5fa
fix(rpc): Mine standard and minimum difficulty blocks on testnet (#5747)
* Mine both standard and min difficulty blocks on testnet

* Add a POW_ADJUSTMENT_BLOCK_SPAN and fix an incorrect assertion

* Split the testnet adjustment into its own function

* Clarify a panic message

* Fix comments
2022-12-02 01:38:05 +00:00
teor afdb3a7013
change(ci): add acceptance test for getblocktemplate RPC in CI, and fix RPC bugs (#5761)
* Re-apply: add acceptance test for getblocktemplate method in CI (#5653)

Revert "Revert "change(tests): add acceptance test for getblocktemplate method in CI (#5653)" (#5672)"

This reverts commit 6446e0ec1b.

* Fix incorrect MAX_CONTEXT_BLOCKS assertion in state

* Actually negate the miner fee for the RPC output

* Try the RPC again after waiting for transactions to verify

* Log before the test waits for the mempool to verify transactions

* Use the new ssh key secrets in CI
2022-12-01 19:39:01 +00:00
Arya c838383fd5
change(state): Check block and transaction Sprout anchors in parallel (#5742)
* parallelize anchors checks for blocks

* parallelize anchors checks for unmined_tx

* reverts par_iter in block_sapling_orchard_anchors_refer_to_final_treestates

* moves fetch_sprout_final_treestates out of rayon thread
2022-12-01 11:35:12 +00:00
teor 26ff1b4fb0
fix(rust): Make Zebra build with the latest nightly Rust (#5738)
* Remove an unused async track_caller which will soon become a warning

* Explicitly drop unused futures

* Work around a compiler panic (ICE) with flat_map()

https://github.com/rust-lang/rust/issues/105044

* Remove a redundant into_iter()

* allow(clippy::needless_collect)
2022-11-30 07:03:24 +00:00
Arya eb0a2ef581
change(mempool): Contextually validates mempool transactions in best chain (#5716)
* updates comments

* adds check nullifier no dup fns for transactions

* Adds:
- check::anchors fn for tx iter
- TODO comments for unifying nullifiers and anchors checks
- new state request

Updates unknown anchor errors to accomodate tx-only check

Calls new state fn from transaction verifier

* updates check::anchors fns to use transactions

updates TransactionContextualValidity request to check sprout anchors

adds comment mentioning TransactionContextualValidity ignores UTXOs

* conditions new state req call on is_mempool

updates tests

* fix doc link / lint error

* checks for duplicate nullifiers with closures

* Update zebra-state/src/service/check/nullifier.rs

Co-authored-by: teor <teor@riseup.net>

* documents find_duplicate_nullifier params

moves if let statement into for loop

* renames new state req/res

* asserts correct response variant in tx verifier

* adds CheckBestChainTipShieldedSpends call in tx verifier to async checks

* re-adds tracing instrumentation to check::anchors fn

renames transaction_in_state to transaction_in_chain

* adds block/tx wrapper fns for anchors checks

* uses UnminedTx instead of transaction.hash()

deletes broken test

* updates new state req/res name

* updates tests and uses par_iter for anchors checks

* Updates check::anchors pub fn docs.

* Adds:
- comments / docs
- a TransactionError variant for ValidateContextError

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* moves downcast to From impl

rustfmt

* moves the ValidateContextError into an Arc

updates comments and naming

* leaves par_iter for another PR

* puts io::Error in an Arc

* updates anchors tests to call tx_anchors check

* updates tests to call tx_no_duplicates_in_chain

slightly improves formatting

* Update zebra-consensus/src/error.rs

Co-authored-by: teor <teor@riseup.net>

* moves Arc from HistoryError to ValidateContextError

Co-authored-by: teor <teor@riseup.net>
2022-11-30 04:40:15 +00:00
dependabot[bot] 1262c88914
build(deps): bump insta from 1.21.1 to 1.21.2 (#5731)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.21.1 to 1.21.2.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.21.1...1.21.2)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-29 02:05:35 +00:00
dependabot[bot] 9e0ee2d02c
build(deps): bump serde from 1.0.147 to 1.0.148 (#5730)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.147 to 1.0.148.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.147...v1.0.148)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-28 23:11:19 +00:00
Alfredo Garcia eb66f4b1a3
feat(rpc): populate some getblocktemplate RPC block header fields using the state best chain tip (#5659)
* populate block height

* populate cur_time

* populate min_time

* populate capabilities

* populate last_block_hash

* create read state request for getblocktemplate

* refactor to get difficulty fields more properly

* populate bits and target fields

* fix tests

* add target and bits documentation

* docs

* fix docs

* docs

* remove metrixs counter calls

* apply some suggestions from code review

* hide some code behind feature

* simplify the service

* fix error handling

* remove comment

* fox doc

* panic if we dont have enough state

* bring tip data from the state

* make proposal empty

* fix time

* fix docs, consensus rules

* remove non used anymore fn

* remove another non used fn

* remove no needed change

* remove more unused changes

* remove unused anymore change

* apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* fix build and snapshots

* apply testnet consensus rule

* fix test

* rustfmt

* remove time as allowed field to be modified by the miner if mining minimum difficulty block

* move all times to before calculating difficulty

* do some cleanup

* Adjust times so the whole time range is a testnet minimum difficulty block

* Return a GetBlockTemplateChainInfo struct from the difficulty calculation

* Add a Zebra-only max_time field to the getblocktemplate RPC

Co-authored-by: teor <teor@riseup.net>
2022-11-28 09:06:32 +00:00
dependabot[bot] 92ae381207
build(deps): bump rayon from 1.5.3 to 1.6.0 (#5677)
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.3 to 1.6.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.5.3...rayon-core-v1.6.0)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-22 18:22:13 +00:00
dependabot[bot] a14ce26be4
build(deps): bump tokio from 1.21.2 to 1.22.0 (#5679)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.21.2 to 1.22.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.2...tokio-1.22.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-22 05:06:12 +00:00
dependabot[bot] ad919acf75
build(deps): bump indexmap from 1.9.1 to 1.9.2 (#5666)
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.9.1 to 1.9.2.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/bluss/indexmap/compare/1.9.1...1.9.2)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-22 02:39:18 +00:00
dependabot[bot] 44f46dd164
build(deps): bump insta from 1.21.0 to 1.21.1 (#5633)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.21.0 to 1.21.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/commits/1.21.1)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-17 15:08:35 +00:00
dependabot[bot] 2680e3c6b2
build(deps): bump chrono from 0.4.22 to 0.4.23 (#5629)
* build(deps): bump chrono from 0.4.22 to 0.4.23

Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.22 to 0.4.23.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.22...v0.4.23)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* uses Utx::timestamp_opt instead of timestamp

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: arya2 <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-16 16:47:45 +00:00
Arya 862600a41e
change(consensus) verify that mempool transaction UTXOs are in the best chain (#5616)
* Uses BestChainUtxo to find utxos for mempool

* adds missing input test

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* update other instances of the renamed InputNotFound error

* adds read::unspent_utxo fn

* adds test for success case

Co-authored-by: teor <teor@riseup.net>
2022-11-11 06:40:35 +00:00
dependabot[bot] 9b533ab059
build(deps): bump regex from 1.6.0 to 1.7.0 (#5565)
* build(deps): bump regex from 1.6.0 to 1.7.0

Bumps [regex](https://github.com/rust-lang/regex) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.6.0...1.7.0)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add a getblocktemplate-rpcs feature to zebra-chain, and fix missing feature deps

* Run OS tests using release builds

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-09 07:36:10 +00:00
teor c4fad29824
fix(sync): Pause new downloads when Zebra reaches the lookahead limit (#5561)
* Use correct release for getblocktemplate config

* Include at least 2 full checkpoints in the lookahead limit

* Increase full sync timeout to 36 hours

* Only log "synced block height too far ahead of the tip" once

* Replace AboveLookaheadHeightLimit error with pausing the syncer

* Use AboveLookaheadHeightLimit for blocks a very long way from the tip

* Also add the getblocktemplate config, and fix the test message

* Remove an outdated TODO comment

* Allow syncing again when a small number of blocks are in the queue

* Allow some dead code
2022-11-09 04:42:04 +00:00
teor 4a8349fa8a
fix(sync): Make the syncer ignore some new block verification errors (#5537)
* Fix error text for state service for syncer

* Fix error handling in syncer

* cargo fmt --all

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-04 06:57:45 +00:00
teor 142411508b
2. change(rpc): Add some transaction fields to the `getblocktemplate` RPC (#5496)
* Add documentation for the getblocktemplate RPC

* Add a new mempool::Request::Transactions

* Add conversions from Vec<UnminedTx> to merkle::Root and AuthDataRoot

* Fill in the merkle root and auth data root fields

* Delete the Coinbase type, it's the same as Transaction

* Fill in some other existing types

* Add Hex serialization support to some zebra-chain types

* Add TransactionTemplate fields and fill some in

* Fix test hangs by spawning async tasks

* Add temporary workaround for no transactions in the block

* Encode hashes and roots as hex

* Update RPC snapshots

* Add a missing Request::Transactions handler

* Fix doc warnings

* Fix fee serialization

* Update snapshots for serialization changes

* Add a missing Cargo.lock change

* Change depends type

* Remove duplicate feature entry

* Document the new RPC feature

* Fix a comment typo

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Update default roots docs

* Fix comment typo

* Fix a comment typo

Co-authored-by: Arya <aryasolhi@gmail.com>

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
2022-11-03 03:25:01 +00:00
Alfredo Garcia 79419b62ce
release(1.0.0-rc.1): bump versions and changelog for release (#5513)
* update versions for 1.0.0-rc.1

* add Cargo.lock

* Initial draft changelog

* Remove trivial changes

* Add missing getblockcount entry

* Add PRs that are about to merge

* remove bots from contributors

Co-authored-by: teor <teor@riseup.net>
2022-11-01 22:35:21 +00:00
dependabot[bot] 881e344b1b
build(deps): bump once_cell from 1.15.0 to 1.16.0 (#5510)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.15.0 to 1.16.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.15.0...v1.16.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-01 05:47:47 +00:00
teor 9cb3dbba9b
1. change(rpc): Add a mempool field to GetBlockTemplateRpcImpl, and cleanup tests (#5493)
* Add a mempool to GetBlockTemplateRpcImpl, and cleanup tests

* Update snapshot file locations

* Update snapshot instructions
2022-10-28 18:34:52 +00:00
teor c812f880cf
cleanup(clippy): Use inline format strings (#5489)
* Inline format strings using an automated clippy fix

```sh
cargo clippy --fix --all-features --all-targets -- -A clippy::all -W clippy::uninlined_format_args
cargo fmt --all
```

* Remove unused & and &mut using an automated clippy fix

```sh
cargo clippy --fix --all-features --all-targets -- -A clippy::all -W clippy::uninlined_format_args
```
2022-10-27 13:25:18 +00:00
Arya e9452d9a6f
change(mempool) reject transactions with spent outpoints or nullifiers (#5434)
* adds transactions to ChainTipBlock and rejects transactions from the mempool that have been invalidated by the latest block commit

* merges remove_same_effects in with reject_invalidated_transactions, moves nullifier retrieval to the new Storage method, rejects mined_ids, and updates tests

* updates DuplicateSpend's error message

* fixes tests

* Update zebrad/src/components/mempool/storage.rs

Co-authored-by: teor <teor@riseup.net>

* adds comment

* formatting for the proptest

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2022-10-24 19:48:45 +00:00
dependabot[bot] a2dba8cc5d
build(deps): bump serde from 1.0.145 to 1.0.147 (#5459)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.145 to 1.0.147.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.145...v1.0.147)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-22 22:31:45 +00:00
dependabot[bot] 4f5539042e
build(deps): bump futures from 0.3.24 to 0.3.25 (#5438)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.24 to 0.3.25.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.24...0.3.25)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-10-21 19:02:04 +00:00
Alfredo Garcia 8d777c892f
feat(rpc): add getblockhash rpc method (#4967)
* implement getblockhash rpc method

* make fixes

* fix some docs

* rustfmt

* add snapshot test

* rename `Hash` to `BestChainBlockHash`

* Suggestion for "add getblockhash rpc method" PR (#5428)

* Always immediately return errors in get_height_from_int()

* Explain why calculations can't overflow

* fix for rust feature

* fix some warnings

* hide state functions behind feature

* remove commented assert

* renames

* rename

* fix some warnings

* make zebra-rpc rpc features depend on zebra-state rpc features

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2022-10-21 06:01:29 +00:00
teor f31609e411
change(state): Add block channel metrics, in preparation for block fork metrics (#5327)
* Add minimum queued block height to metrics

* Add metrics to sent block cache

* Add metrics for the last sent finalized height

* Refactor sent metrics

* Add new block channel metrics to a grafana dashboard
2022-10-20 20:02:58 +00:00
dependabot[bot] c8b95ed877
build(deps): bump tracing from 0.1.36 to 0.1.37 (#5363)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.36 to 0.1.37.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.36...tracing-0.1.37)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-10-13 06:46:27 +00:00
Arya a4cb835e12
Release Zebra 1.0.0-rc.0 (#5383)
* Increment Zebra versions

* Initial draft changelog

* Add blog post to the release checklist

* Say "user testing"

Co-authored-by: teor <teor@riseup.net>
2022-10-12 01:17:55 +00:00
Arya a28350e742
change(state): Write non-finalized blocks to the state in a separate thread, to avoid network and RPC hangs (#5257)
* Add a new block commit task and channels, that don't do anything yet

* Add last_block_hash_sent to the state service, to avoid database accesses

* Update last_block_hash_sent regardless of commit errors

* Rename a field to StateService.max_queued_finalized_height

* Commit finalized blocks to the state in a separate task

* Check for panics in the block write task

* Wait for the block commit task in tests, and check for errors

* Always run a proptest that sleeps once

* Add extra debugging to state shutdowns

* Work around a RocksDB shutdown bug

* Close the finalized block channel when we're finished with it

* Only reset state queue once per error

* Update some TODOs

* Add a module doc comment

* Drop channels and check for closed channels in the block commit task

* Close state channels and tasks on drop

* Remove some duplicate fields across StateService and ReadStateService

* Try tweaking the shutdown steps

* Update and clarify some comments

* Clarify another comment

* Don't try to cancel RocksDB background work on drop

* Fix up some comments

* Remove some duplicate code

* Remove redundant workarounds for shutdown issues

* Remode a redundant channel close in the block commit task

* Remove a mistaken `!force` shutdown condition

* Remove duplicate force-shutdown code and explain it better

* Improve RPC error logging

* Wait for chain tip updates in the RPC tests

* Wait 2 seconds for chain tip updates before skipping them

* Remove an unnecessary block_in_place()

* Fix some test error messages that were changed by earlier fixes

* Expand some comments, fix typos

Co-authored-by: Marek <mail@marek.onl>

* Actually drop children of failed blocks

* Explain why we drop descendants of failed blocks

* Clarify a comment

* Wait for chain tip updates in a failing test on macOS

* Clean duplicate finalized blocks when the non-finalized state activates

* Send an error when receiving a duplicate finalized block

* Update checkpoint block behaviour, document its consensus rule

* Wait for chain tip changes in inbound_block_height_lookahead_limit test

* Wait for the genesis block to commit in the fake peer set mempool tests

* Disable unreliable mempool verification check in the send transaction test

* Appease rustfmt

* Use clear_finalized_block_queue() everywhere that blocks are dropped

* Document how Finalized and NonFinalized clones are different

* sends non-finalized blocks to the block write task

* passes ZebraDb to commit_new_chain, commit_block, and no_duplicates_in_finalized_chain instead of FinalizedState

* Update zebra-state/src/service/write.rs

Co-authored-by: teor <teor@riseup.net>

* updates comments, renames send_process_queued, other minor cleanup

* update assert_block_can_be_validated comment

* removes `mem` field from StateService

* removes `disk` field from StateService and updates block_iter to use `ZebraDb` instead of the finalized state

* updates tests that use the disk to use read_service.db instead

* moves best_tip to a read fn and returns finalized & non-finalized states from setup instead of the state service

* changes `contextual_validity` to get the network from the finalized_state instead of another param

* swaps out StateService with FinalizedState and NonFinalizedState in tests

* adds NotReadyToBeCommitted error and returns it from validate_and_commit when a blocks parent hash is not in any chain

* removes NonFinalizedWriteCmd and calls, moves update_latest_channels above rsp_tx.send

* makes parent_errors_map an indexmap

* clears non-finalized block queue when the receiver is dropped and when the StateService is being dropped

* sends non-finalized blocks to the block write task

* passes ZebraDb to commit_new_chain, commit_block, and no_duplicates_in_finalized_chain instead of FinalizedState

* updates comments, renames send_process_queued, other minor cleanup

* Update zebra-state/src/service/write.rs

Co-authored-by: teor <teor@riseup.net>

* update assert_block_can_be_validated comment

* removes `mem` field from StateService

* removes `disk` field from StateService and updates block_iter to use `ZebraDb` instead of the finalized state

* updates tests that use the disk to use read_service.db instead

* moves best_tip to a read fn and returns finalized & non-finalized states from setup instead of the state service

* changes `contextual_validity` to get the network from the finalized_state instead of another param

* swaps out StateService with FinalizedState and NonFinalizedState in tests

* adds NotReadyToBeCommitted error and returns it from validate_and_commit when a blocks parent hash is not in any chain

* removes NonFinalizedWriteCmd and calls, moves update_latest_channels above rsp_tx.send

* makes parent_errors_map an indexmap

* clears non-finalized block queue when the receiver is dropped and when the StateService is being dropped

* removes duplicate field definitions on StateService that were a result of a bad merge

* update NotReadyToBeCommitted error message

* Appear rustfmt

* Fix doc links

* Rename a function to initial_contextual_validity()

* Do error tasks on Err, and success tasks on Ok

* Simplify parent_error_map truncation

* Rewrite best_tip() to use tip()

* Rename latest_mem() to latest_non_finalized_state()

```sh
fastmod latest_mem latest_non_finalized_state zebra*
cargo fmt --all
```

* Simplify latest_non_finalized_state() using a new WatchReceiver API

* Expand some error messages

* Send the result after updating the channels, and document why

* wait for chain_tip_update before cancelling download in mempool_cancel_mined

* adds `sent_non_finalized_block_hashes` field to StateService

* adds batched sent_hash insertions and checks sent hashes in queue_and_commit_non_finalized before adding a block to the queue

* check that the `curr_buf` in SentHashes is not empty before pushing it to the `sent_bufs`

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Fix rustfmt

* Check for finalized block heights using zs_contains()

* adds known_utxos field to SentHashes

* updates comment on SentHashes.add method

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* return early when there's a duplicate hash in QueuedBlocks.queue instead of panicking

* Make finalized UTXOs near the final checkpoint available for full block verification

* Replace a checkpoint height literal with the actual config

* Update mainnet and testnet checkpoints - 7 October 2022

* Fix some state service init arguments

* Allow more lookahead in the downloader, but less lookahead in the syncer

* Add the latest config to the tests, and fix the latest config check

* Increase the number of finalized blocks checked for non-finalized block UTXO spends

* fix(log): reduce verbose logs for block commits (#5348)

* Remove some verbose block write channel logs

* Only warn about tracing endpoint if the address is actually set

* Use CloneError instead of formatting a non-cloneable error

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Increase block verify timeout

* Work around a known block timeout bug by using a shorter timeout

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-10-11 19:25:45 +00:00
teor 211dbb437b
1. fix(rpc): Fix slow getblock RPC (verbose=1) using transaction ID index (#5307)
* Add RPC timing to zcash-rpc-diff

* Use transaction hash index for verbose block requests, rather than block data
2022-10-02 23:34:44 +00:00
dependabot[bot] 9849d14f41
build(deps): bump thiserror from 1.0.35 to 1.0.37 (#5285)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.35 to 1.0.37.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.35...1.0.37)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-29 11:37:48 +00:00
dependabot[bot] e3b4708919
build(deps): bump insta from 1.20.0 to 1.21.0 (#5291)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.20.0 to 1.21.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/commits)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-29 00:22:39 +00:00
dependabot[bot] ae51274add
build(deps): bump tokio from 1.21.1 to 1.21.2 (#5289)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.21.1 to 1.21.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.1...tokio-1.21.2)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-29 00:22:13 +00:00
teor 343c5e68d4
change(state): Write finalized blocks to the state in a separate thread, to avoid network and RPC hangs (#5134)
* Add a new block commit task and channels, that don't do anything yet

* Add last_block_hash_sent to the state service, to avoid database accesses

* Update last_block_hash_sent regardless of commit errors

* Rename a field to StateService.max_queued_finalized_height

* Commit finalized blocks to the state in a separate task

* Check for panics in the block write task

* Wait for the block commit task in tests, and check for errors

* Always run a proptest that sleeps once

* Add extra debugging to state shutdowns

* Work around a RocksDB shutdown bug

* Close the finalized block channel when we're finished with it

* Only reset state queue once per error

* Update some TODOs

* Add a module doc comment

* Drop channels and check for closed channels in the block commit task

* Close state channels and tasks on drop

* Remove some duplicate fields across StateService and ReadStateService

* Try tweaking the shutdown steps

* Update and clarify some comments

* Clarify another comment

* Don't try to cancel RocksDB background work on drop

* Fix up some comments

* Remove some duplicate code

* Remove redundant workarounds for shutdown issues

* Remode a redundant channel close in the block commit task

* Remove a mistaken `!force` shutdown condition

* Remove duplicate force-shutdown code and explain it better

* Improve RPC error logging

* Wait for chain tip updates in the RPC tests

* Wait 2 seconds for chain tip updates before skipping them

* Remove an unnecessary block_in_place()

* Fix some test error messages that were changed by earlier fixes

* Expand some comments, fix typos

Co-authored-by: Marek <mail@marek.onl>

* Actually drop children of failed blocks

* Explain why we drop descendants of failed blocks

* Clarify a comment

* Wait for chain tip updates in a failing test on macOS

* Clean duplicate finalized blocks when the non-finalized state activates

* Send an error when receiving a duplicate finalized block

* Update checkpoint block behaviour, document its consensus rule

* Wait for chain tip changes in inbound_block_height_lookahead_limit test

* Wait for the genesis block to commit in the fake peer set mempool tests

* Disable unreliable mempool verification check in the send transaction test

* Appease rustfmt

* Use clear_finalized_block_queue() everywhere that blocks are dropped

* Document how Finalized and NonFinalized clones are different

* Use the same check as commit_finalized() for finalized block heights

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Marek <mail@marek.onl>
2022-09-28 16:09:56 +00:00
dependabot[bot] 23e655eaae
build(deps): bump tracing from 0.1.35 to 0.1.36 (#5273)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.35 to 0.1.36.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.35...tracing-0.1.36)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-28 03:17:49 +00:00
Marek 8ddc508034
build(deps): Bump metrics and metrics-exporter-prometheus (#5256)
* build(deps): bump metrics from 0.18.1 to 0.20.1

Bumps [metrics](https://github.com/metrics-rs/metrics) from 0.18.1 to 0.20.1.
- [Release notes](https://github.com/metrics-rs/metrics/releases)
- [Changelog](https://github.com/metrics-rs/metrics/blob/main/release.toml)
- [Commits](https://github.com/metrics-rs/metrics/compare/metrics-v0.18.1...metrics-v0.20.1)

---
updated-dependencies:
- dependency-name: metrics
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump metrics-exporter-prometheus from 0.9.0 to 0.11.0

Bumps [metrics-exporter-prometheus](https://github.com/metrics-rs/metrics) from 0.9.0 to 0.11.0.
- [Release notes](https://github.com/metrics-rs/metrics/releases)
- [Changelog](https://github.com/metrics-rs/metrics/blob/main/release.toml)
- [Commits](https://github.com/metrics-rs/metrics/compare/metrics-exporter-prometheus-v0.9.0...metrics-exporter-prometheus-v0.11.0)

---
updated-dependencies:
- dependency-name: metrics-exporter-prometheus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update `Cargo.lock`

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-27 04:19:01 +00:00
dependabot[bot] fc8bd1531a
build(deps): bump serde from 1.0.144 to 1.0.145 (#5247)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.144 to 1.0.145.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.144...v1.0.145)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-26 23:48:40 +00:00
dependabot[bot] 9c382efcc5
build(deps): bump tracing from 0.1.31 to 0.1.35 (#5225)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.31 to 0.1.35.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.31...tracing-0.1.35)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-26 22:24:11 +00:00
dependabot[bot] 44e0fa5627
build(deps): bump color-eyre from 0.6.1 to 0.6.2 (#5220)
Bumps [color-eyre](https://github.com/yaahc/color-eyre) from 0.6.1 to 0.6.2.
- [Release notes](https://github.com/yaahc/color-eyre/releases)
- [Changelog](https://github.com/yaahc/color-eyre/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yaahc/color-eyre/compare/v0.6.1...v0.6.2)

---
updated-dependencies:
- dependency-name: color-eyre
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-26 22:23:59 +00:00
Arya ec115e930f
change(zebrad): opens the database in a blocking tokio thread, which allows tokio to run other tasks (#5228)
* use spawn_blocking to run zebra_state::init from start cmd

* uses zebra_state::spawn_init in copy-state command

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-26 15:45:42 +00:00
dependabot[bot] 2465f4bf22
build(deps): bump once_cell from 1.14.0 to 1.15.0 (#5216)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-22 20:22:22 +00:00
dependabot[bot] 2b4f7ec6d5
build(deps): bump itertools from 0.10.4 to 0.10.5 (#5209)
Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.4 to 0.10.5.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/commits)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-21 02:53:22 +00:00
teor 0db014da40
fix(testnet): look back up to 10,000 blocks on testnet for a legacy chain (#5133)
* Look back 10,000 blocks on testnet for a legacy chain

* Use a smaller number of legacy chain blocks in the tests

* Fix test assertion error message

* Clarify legacy chain check comment

* cargo fmt --all

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-20 11:31:24 +00:00
dependabot[bot] b122052d3e
build(deps): bump insta from 1.19.1 to 1.20.0 (#5198)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.19.1 to 1.20.0.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.19.1...1.20.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-20 04:41:30 +00:00
teor f8ead710c3
chore: Release Zebra 1.0.0-beta.15 (#5194)
* Initial beta.15 changelog

* Increment Zebra versions and remove known issues

* Document `protoc` dependency for `lightwalletd` tests

* Explain latest sync time

* s/coverate/coverage/

Co-authored-by: Marek <mail@marek.onl>

* Style

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
2022-09-19 21:21:27 +00:00
dependabot[bot] 202dc43b11
build(deps): bump tokio from 1.21.0 to 1.21.1 (#5150)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.21.0 to 1.21.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.0...tokio-1.21.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-16 21:36:39 +00:00
Arya 2fa1879fb1
cleanup(state): condenses match arms for redirected StateService requests and moves metrics counting to Request methods (#5137)
* Move AwaitUtxos next to the other shared writeable state requests

* Rename ReadResponse::Utxos to ReadResponse::AddressUtxos

```sh
fastmod Utxos AddressUtxos zebra*
```

* Rename an out_point variable to outpoint for consistency

* Rename transparent_utxos to address_utxos

```sh
fastmod transparent_utxos address_utxos zebra*
```

* Run AwaitUtxo without accessing shared mutable chain state

* Fix some incorrect comments

* Explain why some concurrent reads are ok

* Add a TODO

* Stop using self.mem in AwaitUtxo requests

* Update state service module documentation

* Move the QueuedBlock type into the queued_blocks module

* Explain how spent UTXOs are treated by the state

* Clarify how cached Chains impact state read requests

And move repeated comments to the module header.

* fastmod ChainUtxo BestChainUtxo zebra*

* Add an AnyChainUtxo request

* Make AwaitUtxo non-blocking

* Move the finalized block queue into the StateService

* Move the queued_blocks module to the state service

* Move QueuedFinalized into queued_blocks

* Move the queued_blocks tests into their own module

* Make the FinalizedState cloneable

* cleanup of repetitive code

* fixes merge by adding back req.count_metric and removing the metrics::counter in AwaitUtxo

Co-authored-by: teor <teor@riseup.net>
2022-09-16 19:03:56 +00:00
teor bfdb29b757
3. change(state): Move the finalized queue to the StateService (#5152)
* Move the finalized block queue into the StateService

* Move the queued_blocks module to the state service

* Move QueuedFinalized into queued_blocks

* Move the queued_blocks tests into their own module

* Make the FinalizedState cloneable
2022-09-16 13:53:40 +00:00
teor 36a549ee3c
2. change(state): Run AwaitUtxo read requests without shared mutable chain state (#5107)
* Move AwaitUtxos next to the other shared writeable state requests

* Rename ReadResponse::Utxos to ReadResponse::AddressUtxos

```sh
fastmod Utxos AddressUtxos zebra*
```

* Rename an out_point variable to outpoint for consistency

* Rename transparent_utxos to address_utxos

```sh
fastmod transparent_utxos address_utxos zebra*
```

* Run AwaitUtxo without accessing shared mutable chain state

* Fix some incorrect comments

* Explain why some concurrent reads are ok

* Add a TODO

* Stop using self.mem in AwaitUtxo requests

* Update state service module documentation

* Move the QueuedBlock type into the queued_blocks module

* Explain how spent UTXOs are treated by the state

* Clarify how cached Chains impact state read requests

And move repeated comments to the module header.

* fastmod ChainUtxo BestChainUtxo zebra*

* Add an AnyChainUtxo request

* Make AwaitUtxo non-blocking
2022-09-16 04:13:26 +00:00
dependabot[bot] 3421869c56
build(deps): bump thiserror from 1.0.34 to 1.0.35 (#5160)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.34 to 1.0.35.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.34...1.0.35)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-15 09:04:53 +00:00
dependabot[bot] 5507babe87
build(deps): bump itertools from 0.10.3 to 0.10.4 (#5149)
Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.3 to 0.10.4.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.3...v0.10.4)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-14 17:45:18 +00:00
teor 00eee8652e
1. change(state): Run most StateService read requests without shared mutable chain state (#5132)
* Add TODOs for finalized blocks, non-finalized blocks, and reads

* Document how state requests use shared state

* Add a ReadStateService to the StateService

And cleanup service struct fields.

* Redirect Block and Transaction Requests to the ReadStateService

* Put AddressBalance in a consistent enum order

* Turn a repeated comment into documentation

* Tidy doc links

* Run Tip requests concurrently

* Remove some redundant timers

* Run Depth requests concurrently

* Run BlockLocator requests concurrently

* Move BlockLocator tests

* Run FindBlockHashes requests concurrently

* Run FindBlockHeaders requests concurrently

* Use a constant in documentation

Co-authored-by: Marek <mail@marek.onl>

* Link that constant correctly

* Expand block_locator() documentation

* Clarify the difference between tower::Buffers and the state's ordered queues

* Explain block locators better

Co-authored-by: Marek <mail@marek.onl>
2022-09-14 00:35:37 +00:00
teor 093d503d22
Document why we do a UTXO check that looks redundant (#5106)
Also inline the call stack for the check, so it is efficient.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-07 23:42:52 +00:00
Arya d9fae6e311
fix(test) switch zebrad to a non-blocking tracing logger (#5032)
* adds non-blocking writer for tracing subscriber

* use non_blocking writer for the fmt::Layer with the tokio-console feature as well

* adds doc comment to _guard field

* adds acceptance test

* update filter_handle type to use NonBlocking

* adds more detail on lossy non-blocking writer and sets tracing.filter to "trace" in acceptance test

* drops ZebradApp before process::exit(1) in the event of a FrameworkError

* reduces buffered lines limit to 8000

* adds tracing.buffer_limit config and some comments

* update acceptance.rs

* fix acceptance test

* fixes ambigious phrasing in comment

* updates zebrad/src/application.rs

* Find out what the join error is in the GitHub runner tests

* updates acceptance test to use recv_timeout instead of always waiting 10 seconds, removes unnecessary echo command, and reduces # of rpc requests to 500

* see if sleeping for a few seconds before exiting helps the macOS test pass

* Expand exit sleep docs

Co-authored-by: Arya <aryasolhi@gmail.com>

Co-authored-by: teor <teor@riseup.net>
2022-09-07 07:39:30 +00:00
dependabot[bot] d8cef93af6
build(deps): bump thiserror from 1.0.33 to 1.0.34 (#5074)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.33 to 1.0.34.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.33...1.0.34)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-06 09:33:08 +00:00
Marek b8712d9a1e
feat(state): Send treestate from non-finalized state to finalized state (#4721)
* Add history trees for each height in non-fin state

* Refactor formatting

* Pass the treestate to the finalized state

I created a new structure `FinalizedBlockWithTrees` that wraps the
treestate and the finalized block. I did that because the original
`FinalizedBlock` is `Eq`, but `HistoryTree` can't be `Eq`.

This makes Zebra faster because:

1. The finalized state doesn't retrieve the treestate from the disk if
the non-finalized state supplies it.

2.The finalized state doesn't recompute the treestate if the
non-finalized state supplies it.

* Check block commitment before updating hist tree

* Store Sprout commitment trees in non-fin state

* Send trees for the root block to fin-state

When committing a block and sending the treestate from the non-finalized
state to the finalized state, Zebra was sending trees that correspond to
the tip block instead of trees that correspond to the root block of the
best chain. This commit fixes that.

* Refactor doc comments

* Refactor block finalization

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-06 09:32:54 +00:00
teor fc624d009d
change(deps): bump rocksdb from 0.18.0 to 0.19.0 (#5071)
* cargo upgrade --workspace rocksdb

* Add a deny.toml exception for bindgen 0.59

* Move `valid()` and `status()` methods to raw iterators

* Update some outdated comments

* Panic on iterator failures

We might want to change this if there are any common failure modes.

* allow(clippy::unwrap_in_result) in some methods
2022-09-05 18:58:45 +00:00
dependabot[bot] 9f85bb89ae
build(deps): bump futures from 0.3.23 to 0.3.24 (#5020)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.23 to 0.3.24.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.23...0.3.24)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-04 01:15:31 +00:00
dependabot[bot] 5405bc76d9
build(deps): bump once_cell from 1.13.0 to 1.14.0 (#5057)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.13.0 to 1.14.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.13.0...v1.14.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-03 15:44:45 +00:00
dependabot[bot] c8cf8c5464
build(deps): bump tokio from 1.20.1 to 1.21.0 (#5058)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.20.1 to 1.21.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.20.1...tokio-1.21.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-03 15:44:31 +00:00
dependabot[bot] 3d6693e5d4
build(deps): bump chrono from 0.4.20 to 0.4.22 (#4984)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.20 to 0.4.22.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/v0.4.22/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.20...v0.4.22)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-02 21:11:18 +00:00
teor 3d8f4e6064
fix(ci): improve test output and test reliability (#5014)
* Rename a function to prepare_block_header_and_transaction_data_batch()

* Fix formatting of test command timeouts and child process output

* Put some #[cfg()]s in the standard Rust location

* Update some test timings

* Allow code timers to be ignored
2022-09-02 08:54:40 +00:00
Alfredo Garcia eb228faa5a
bump insta to 1.19.1 (#5018) 2022-09-01 21:51:15 +00:00
dependabot[bot] a8e0db4a42
build(deps): bump thiserror from 1.0.32 to 1.0.33 (#5019)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.32 to 1.0.33.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.32...1.0.33)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-01 21:50:48 +00:00
Marek c322533125
v1.0.0-beta.14-release (#5006)
* Bump crate versions

* Bump `USER_AGENT`

* Update `README.md`

* Update `install.md`

* Update `CHANGELOG.md`

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Apply a suggestion from code review

Co-authored-by: teor <teor@riseup.net>

* Move the "Networking" section above "Tests"

* Merge PRs that update checkpoints

* Refactor `CHANGELOG.md`

* Add missing PRs to `CHANGELOG.md`

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2022-08-31 22:00:46 +00:00
dependabot[bot] ecf2d80175
build(deps): bump futures from 0.3.21 to 0.3.23 (#4913)
* build(deps): bump futures from 0.3.21 to 0.3.23

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* updated deprecated FuturesOrdered::push to push_back

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: arya2 <aryasolhi@gmail.com>
2022-08-30 05:14:08 +00:00
teor c188678169
Revert: deserialize stored transactions in a rayon thread (#4933)
* Revert: deserialize stored transactions in a rayon thread

* Add a TODO for the reverted bug fix

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-30 02:39:34 +00:00
teor d31c3c4177
fix(consensus): Increase the number of blocks checked for legacy transactions (#4804)
* Increase the number of legacy chain check blocks, and improve logging

* Automatically adjust test message when MAX_LEGACY_CHAIN_BLOCKS changes
2022-08-29 20:25:41 +00:00
dependabot[bot] 893f4950b3
build(deps): bump serde from 1.0.142 to 1.0.144 (#4925)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.142 to 1.0.144.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.142...v1.0.144)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-29 06:59:51 +00:00
Marek a87b119a10
feat(state): Store history trees by height in the non-finalized state (#4928)
* Add history trees for each height in non-fin state

* Refactor formatting

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-29 06:59:25 +00:00
teor e243a357f9
Fix clippy::unused_parens (#4931)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-29 06:58:56 +00:00
teor e973508911
build(deps): bump chrono from 0.4.19 to 0.4.20 (#4898)
* Bump chrono to 0.4.20

* Fix clippy::assign_op_pattern

* Update deprecated constant names

* Drop old `time` crate dependency from `chrono`

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-28 17:08:27 +00:00
dependabot[bot] 905ffca3b7
build(deps): bump thiserror from 1.0.31 to 1.0.32 (#4878)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.31 to 1.0.32.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.31...1.0.32)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-05 15:02:11 +00:00
dependabot[bot] 193e8c6c04
build(deps): bump tinyvec from 1.5.1 to 1.6.0 (#4888)
Bumps [tinyvec](https://github.com/Lokathor/tinyvec) from 1.5.1 to 1.6.0.
- [Release notes](https://github.com/Lokathor/tinyvec/releases)
- [Changelog](https://github.com/Lokathor/tinyvec/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Lokathor/tinyvec/compare/v1.5.1...v1.6.0)

---
updated-dependencies:
- dependency-name: tinyvec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-04 22:03:24 +00:00
Conrado Gouvea 6fd750e168
build(deps): bump insta from 1.15.0 to 1.17.1 (#4884)
* build(deps): bump insta from 1.15.0 to 1.17.1

Bumps [insta](https://github.com/mitsuhiko/insta) from 1.15.0 to 1.17.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.15.0...1.17.1)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* make zebra_test::init() return the insta drop guard

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-04 15:44:44 +00:00
teor 42f6c27c98
Enable a transitive feature that speeds up compilation (#4796) 2022-08-03 23:42:50 +00:00
dependabot[bot] 8c47807f70
build(deps): bump serde from 1.0.141 to 1.0.142 (#4876)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.141 to 1.0.142.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.141...v1.0.142)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-03 20:48:16 +00:00
dependabot[bot] 3b29a0118e
build(deps): bump serde from 1.0.137 to 1.0.141 (#4865)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.137 to 1.0.141.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.137...v1.0.141)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-03 01:02:28 +00:00
dependabot[bot] 46bb348d11
build(deps): bump tokio from 1.20.0 to 1.20.1 (#4864)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.20.0 to 1.20.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.20.0...tokio-1.20.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-03 01:01:45 +00:00
teor 9441a1f66c
Split the zebra_state::service::read module (#4827)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-02 21:15:28 +00:00
dependabot[bot] 35ccee3431
build(deps): bump once_cell from 1.12.0 to 1.13.0 (#4749)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.12.0...v1.13.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-01 23:52:53 +00:00
dependabot[bot] bb48c9fb47
build(deps): bump regex from 1.5.6 to 1.6.0 (#4755)
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.6 to 1.6.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.6...1.6.0)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-01 09:07:08 +00:00