Commit Graph

858 Commits

Author SHA1 Message Date
Alfredo Garcia 83aa42e649
tests(config): Add tests for old configs (#4676)
* change `initial_mainnet_peers` and `initial_testnet_peers` type to `IndexSet`

* add tests for zebra config files

* add serde feature to indexmap

* remove async

* update config

* fix `stored_config_path()`

* skip tests if config is not found

* improve error

* use CARGO_MANIFEST_DIR

* remove `stored_config_is_newest` test

* move `stored_config_works` test to the end of `valid_generated_config_test`

* space
2022-06-27 00:07:37 +00:00
teor 49cda21f37
Slightly increase some syncer defaults (#4679)
This is a partial revert of PR #4670.
2022-06-24 00:58:49 +00:00
dependabot[bot] c262246266
build(deps): bump inferno from 0.11.5 to 0.11.6 (#4682)
Bumps [inferno](https://github.com/jonhoo/inferno) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/jonhoo/inferno/releases)
- [Changelog](https://github.com/jonhoo/inferno/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jonhoo/inferno/compare/v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: inferno
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-23 21:09:06 +00:00
teor c75a68e655
fix(sync): change default sync config to improve reliability (#4670)
* Decrease the default lookahead limit to 400

* Increase the block verification timeout to 10 minutes

* Halve the default concurrent downloads config

* Try to run the spawned download task before queueing the next download

* Allow verification to be cancelled if the verifier is busy
2022-06-22 18:17:21 +00:00
dependabot[bot] 82b8d7f95d
build(deps): bump tower from 0.4.12 to 0.4.13 (#4644)
Bumps [tower](https://github.com/tower-rs/tower) from 0.4.12 to 0.4.13.
- [Release notes](https://github.com/tower-rs/tower/releases)
- [Commits](https://github.com/tower-rs/tower/compare/tower-0.4.12...tower-0.4.13)

---
updated-dependencies:
- dependency-name: tower
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-21 17:12:19 +00:00
teor 3a7c2c8926
Replace the lookahead limit panic with a warning (#4662)
And decrease the minimum to 400 blocks
2022-06-21 11:07:32 +00:00
dependabot[bot] ee50f3ae45
build(deps): bump inferno from 0.11.4 to 0.11.5 (#4647)
Bumps [inferno](https://github.com/jonhoo/inferno) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/jonhoo/inferno/releases)
- [Changelog](https://github.com/jonhoo/inferno/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jonhoo/inferno/compare/v0.11.4...v0.11.5)

---
updated-dependencies:
- dependency-name: inferno
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-21 03:38:52 +00:00
Alfredo Garcia 769d069d0a
feat(state): delete old database directories (#4586)
* delete old database directories

* check if state directory exists

* skip deleting when ephemeral

* split `check_and_delete_old_databases`

* move `check_and_delete_old_databases` to state

* spawn `check_and_delete_old_databases`

* simplity a bit

* fix(state): only delete old database directories inside the cache directory (#4631)

* Add function comments, tweak log

* Simplify version parsing

* Use spawn_blocking to launch the task on a separate thread, do the cleanup last

* Abort the cleanup task when Zebra exits

* Split directory deletion into its own function, handle ownership

* Rename cache_dir to state_dir

* If an outdated state directory is outside the cache directory, don't delete it

* Minimise diffs

* add test

* fix typos

Co-authored-by: teor <teor@riseup.net>

* add `canonicalize` to test regex

* add another match to test

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-21 00:59:51 +00:00
teor 29e73b3f3e
breaking(diagnostics): make zebrad diagnostics into optional compile-time features (#4539)
* Disable the flamegraph feature by default at compile time

* Disable the journald feature by default at compile time

* Also disable inferno dependency, and rearrange features

* Disable the prometheus feature by default at compile time

* Disable the tracing filter reload feature by default at compile time

* Disable tests when corresponding features are disabled

* Add compile-time tracing features to user docs

* Add compile-time features to the metrics user docs

* Document diagnostics as part of the start command tasks and services

* breaking(diagnostics): rename "enable-sentry" feature to "sentry" (#4623)

* Also skip conflict tests when those ports are disabled

* breaking(diagnostics): rename "enable-sentry" feature to "sentry"

This is mostly:
```sh
fastmod enable-sentry sentry
```

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-16 19:56:40 +00:00
Marek b91aaf7863
Use `config.toml` instead of inner attributes for private links (#4627) 2022-06-15 23:21:27 +00:00
Alfredo Garcia 9975d5077d
doc(tests): full sync and lightwalletd tests (#4523)
* document full sync and lightwalletd tests

* clippy

* upgrade error

* move env variables

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-15 08:05:02 +00:00
teor 83b4e6f975
feat(diagnostics): Add tokio-console support to zebrad (#4519)
* Always activate tokio/tracing feature

And always build tests with all tokio features.

* Refactor tracing-subscriber init to simplify it

* Add the tokio-console feature and dependencies

* Add optional tokio-console support, and log the installed tracing layers at info level

Uses a tracing Registry for tokio-console, and a fmt::Subscriber otherwise.

* Add some TODOs based on tracing-subscriber features

* Fix up some spans

* Add a TODO for fixing a log filter bug in tokio-console mode
2022-06-15 16:43:20 +10:00
Marek cc75c3f5f9
fix(doc): Fix various doc warnings, part 3 (#4611)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fa.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

* Fix some warnings produced by `cargo doc`

* Fix some rustdoc warnings

* Fix some warnings

* Refactor some changes

* Fix some rustdoc warnings

* Fix some rustdoc warnings

* Resolve various TODOs

Co-authored-by: teor <teor@riseup.net>

* Fix some unresolved links

* Allow links to private items

* Fix some unresolved links

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-15 03:57:19 +00:00
dependabot[bot] 1fddb9a4ba
build(deps): bump reqwest from 0.11.10 to 0.11.11 (#4610)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.10 to 0.11.11.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.10...v0.11.11)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-15 01:52:28 +00:00
teor a1074954e0
Disable zcash_rpc_conflict test on macOS (#4614) 2022-06-15 00:31:18 +00:00
dependabot[bot] 0df98038ed
build(deps): bump vergen from 7.2.0 to 7.2.1 (#4582)
Bumps [vergen](https://github.com/rustyhorde/vergen) from 7.2.0 to 7.2.1.
- [Release notes](https://github.com/rustyhorde/vergen/releases)
- [Commits](https://github.com/rustyhorde/vergen/compare/7.2.0...7.2.1)

---
updated-dependencies:
- dependency-name: vergen
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 19:05:08 +00:00
dependabot[bot] 96931cc4a4
build(deps): bump tokio-stream from 0.1.8 to 0.1.9 (#4573)
Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.8...tokio-stream-0.1.9)

---
updated-dependencies:
- dependency-name: tokio-stream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 13:22:34 +00:00
dependabot[bot] 8b60317be4
build(deps): bump semver from 1.0.9 to 1.0.10 (#4589)
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.9 to 1.0.10.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.9...1.0.10)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 11:37:02 +00:00
Alfredo Garcia 6d9bb2226a
fix(config): Duration fields (#4587)
* use `humantime_serde` for config durations

* move debug config option to the bottom

* fix deserialization

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-14 06:21:24 +00:00
dependabot[bot] 8d0a1b011c
build(deps): bump tokio from 1.18.2 to 1.19.2 (#4579)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.2 to 1.19.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.2...tokio-1.19.2)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-14 06:20:48 +00:00
teor 92fd11f9ad
fix(network): when connecting to peers, ignore invalid ports, and prefer canonical ports (#4564)
* Move peer address validation into its own module

* Add a network parameter to AddressBook and some MetaAddr methods

* Reject invalid initial peers, and connect to them in preferred order

* Reject Flux/ZelCash and misconfigured Zcash peers

* Prefer canonical Zcash ports

* Make peer_preference into a struct method

* Prefer peer addresses with canonical ports for outbound connections

* Also ignore the Zcash regtest port

* Document where field and variant order is required for correctness

* Use the correct peer count

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-14 04:58:37 +00:00
Conrado Gouvea e016bbe8e3
increase lightwalletd timeout, remove testnet tests (#4584)
* increase lightwalletd timeout

* switch back to aditya's fork

* manually point to new aditya's lightwalletd image

* disable sync_one_checkpoint_testnet test

* disable restart_stop_at_height in testnet

* rever to 'latest' lightwalletd image
2022-06-13 20:13:30 +00:00
dependabot[bot] bd7e1c09e3
build(deps): bump hyper from 0.14.18 to 0.14.19 (#4533)
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.18 to 0.14.19.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.18...v0.14.19)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-04 01:00:05 +00:00
Marek 0e4c51c431
V1.0.0 beta.11 release (#4567)
* Bump crate versions

* Increment the protocol user agent

* Increment the version in `README.md`

* Increment the version in `install.md`

* Update `README.md`

* Update the changelog

* Increment the versions of crates that depend on other crates

* Update `Cargo.lock`
2022-06-03 22:55:05 +00:00
dependabot[bot] 5c4d3e1f5c
build(deps): bump inferno from 0.11.3 to 0.11.4 (#4534)
Bumps [inferno](https://github.com/jonhoo/inferno) from 0.11.3 to 0.11.4.
- [Release notes](https://github.com/jonhoo/inferno/releases)
- [Changelog](https://github.com/jonhoo/inferno/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jonhoo/inferno/compare/v0.11.3...v0.11.4)

---
updated-dependencies:
- dependency-name: inferno
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-02 15:07:55 +00:00
Marek b8b35f8da9
fix(doc): Fix various doc warnings, part 1 (#4514)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fa.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

* Fix some warnings produced by `cargo doc`

* Fix some rustdoc warnings

* Fix some warnings

* Refactor some changes

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-06-02 15:07:35 +00:00
dependabot[bot] f31c5b334e
build(deps): bump log from 0.4.14 to 0.4.17 (#4530)
Bumps [log](https://github.com/rust-lang/log) from 0.4.14 to 0.4.17.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.14...0.4.17)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-02 15:07:01 +00:00
teor 61afd02a98
fix(ci): test RPCs with zcash/lightwalletd, to fix post-NU5 failures in adityapk00/lightwalletd (#4553)
* Remove a duplicate lightwalletd error message

* Reactivate some error messages that have been fixed

* Fix confusing lightwalletd cached state path logs

* Add the gRPC tests to the lightwalletd test suite function

* Make test regexes compatible with zcash/lightwalletd

* Add logging to gRPC tests

* Switch to zcash/lightwalletd for testing
2022-06-01 07:36:59 -04:00
teor ca0520b2e8
change(deps): Upgrade tracing-subscriber and related dependencies (#4517)
* Upgrade tracing and related dependencies

```sh
cargo upgrade --workspace
tracing-error
tracing-subscrber

color-eyre

tracing-flame
tracing-journald

sentry
sentry-tracing

metrics
metrics-exporter-prometheus
reqwest
```

* Update duplicate dependency checks

* Enable the tracing/env-filter feature

* Fix type inference for metrics

Manual changes, plus:
```sh
fastmod "as _" "as f64"
```

* Tidy up some unrelated test code

* Update metrics-exporter-prometheus API

And make unused dependencies optional.

* Adjust test regexes to new tracing format

Also fix some regex bugs, and refactor to simplify.

* Disable color-eyre span traces and track caller in release builds

* Add a feature that enables extra debugging in release builds

* Clean up some redundant features

* Increase a test timeout
2022-06-01 13:53:51 +10:00
dependabot[bot] 8a00fad857
build(deps): bump indexmap from 1.8.1 to 1.8.2 (#4531)
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Changelog](https://github.com/bluss/indexmap/blob/1.8.2/RELEASES.rst)
- [Commits](https://github.com/bluss/indexmap/compare/1.8.1...1.8.2)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-30 21:59:30 +00:00
Marek 6f896ef5a5
fix(doc): Fix the syntax of links in comments (#4494)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fa.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-05-30 20:12:11 +00:00
dependabot[bot] 532674fc01
build(deps): bump vergen from 7.1.0 to 7.2.0 (#4521)
Bumps [vergen](https://github.com/rustyhorde/vergen) from 7.1.0 to 7.2.0.
- [Release notes](https://github.com/rustyhorde/vergen/releases)
- [Commits](https://github.com/rustyhorde/vergen/compare/7.1.0...7.2.0)

---
updated-dependencies:
- dependency-name: vergen
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-30 00:01:16 +00:00
teor 9cc4584bc6
change(build): Disable debug logging at compile time in release builds (#4516)
* Add zebrad features that disable debug logs at compile time

* By default, disable debug logs at compile time in release builds
2022-05-28 07:08:01 -04:00
teor 77d9d8f017
feat(log): Log the tracing level when it is set or reloaded (#4515)
* Log the tracing level when it is set or reloaded

* Add the `log` crate as a direct dependency

* Make `log` required
2022-05-27 19:49:51 +00:00
teor c2769dcc50
Activate the mempool after 2 tip syncer runs (#4501)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-26 18:51:52 +00:00
dependabot[bot] 715d204939
build(deps): bump regex from 1.5.5 to 1.5.6 (#4463)
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.5 to 1.5.6.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.5.6)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-26 08:12:55 +00:00
dependabot[bot] aa0301158e
build(deps): bump prost from 0.10.3 to 0.10.4 (#4490)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.10.3 to 0.10.4.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.3...v0.10.4)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-26 08:12:38 +00:00
teor 4add7fc53c
Permanently fix unreliable sync finished log regex (#4504)
Unlike the other recent changes, this is a permanent fix.
2022-05-26 13:40:28 +10:00
teor be91ab29ee
fix(clippy): resolve various clippy warnings (#4473)
* clippy: unused import on non-linux platforms

* Fix some instances of clippy::derive_partial_eq_without_eq

* Move a deref to fix clippy::significant_drop_in_scrutinee

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-25 20:56:32 +00:00
dependabot[bot] 2ee52f808c
build(deps): bump once_cell from 1.11.0 to 1.12.0 (#4462)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.11.0 to 1.12.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.11.0...v1.12.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-25 20:56:12 +00:00
teor 89e79474a6
Revert "Stop failing the send transaction test (#4416)" (#4487)
This reverts commit 35aaee5308.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-25 19:04:15 +00:00
teor 97b9911c7d
Stop full sync at 97%, but send transactions at 100% (#4483) 2022-05-25 13:14:38 +10:00
teor 49406f3132
fix(ci): Temporarily finish full sync at 99% (#4457)
* Finish full sync at 99.5%

* Finish full sync test harness at 99.5%

* Try 99.2% instead

* Try 99%
2022-05-24 17:59:09 +10:00
Marek 083674de4e
feat(rpc): Add `z_gettreestate` gRPC tests (#4455)
* Add gRPC tests for `z_gettreestate`

* Add a Sapling treestate test vector
2022-05-24 03:09:53 +00:00
dependabot[bot] 9425cb3f1c
build(deps): bump once_cell from 1.10.0 to 1.11.0 (#4447)
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.10.0...v1.11.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-21 23:49:36 +00:00
Conrado Gouvea 6c1c5515d5
change: fail early when using Rust 1.57 and earlier, because they don't work (#4435)
* change: set 1.58 as the minimum required Rust version for zebrad and document it

* Update README.md

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2022-05-19 23:07:46 +00:00
Alfredo Garcia 8150a720ca
release: bump versions and create changelog entry for `v1.0.0-beta.10` (#4430)
* bump versions

* add the changelog for beta 10

* Update CHANGELOG.md

Co-authored-by: Marek <mail@marek.onl>

* Distinguish CI and Rust dependency updates

* Include all the dep bumps in PR #4405

* join 2 dependencies

* Include incrementalmerkletree dep bump in changelog

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
2022-05-19 19:23:30 +00:00
Alfredo Garcia d50cf8b77a
tests(rpc): Add grpc test for `GetTaddressBalanceStream` and `GetAddressUtxosStream` (#4407)
* add test for `GetTaddressBalanceStream`

* add test for `GetAddressUtxosStream`
2022-05-19 02:30:50 +00:00
Marek ec9b569ebf
Fix some failing gRPC tests. (#4424) 2022-05-19 02:30:34 +00:00
dependabot[bot] 7f524a33a2
build(deps): bump vergen from 7.0.0 to 7.1.0 (#4420)
Bumps [vergen](https://github.com/rustyhorde/vergen) from 7.0.0 to 7.1.0.
- [Release notes](https://github.com/rustyhorde/vergen/releases)
- [Commits](https://github.com/rustyhorde/vergen/compare/7.0.0...7.1.0)

---
updated-dependencies:
- dependency-name: vergen
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-18 21:05:52 +00:00
teor 35aaee5308
Stop failing the send transaction test (#4416)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-18 19:14:39 +00:00
teor 94ba4b46a2
change(test): Require cached lightwalletd state for the send transaction tests (#4303)
* Revert "Temporarily stop requiring cached lightwalletd state for the send transaction tests"

This reverts commit f6b29b151e.

* fix(ci): add a lightwalletd cached state to the test

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-05-13 23:01:31 +00:00
Gustavo Valverde e3a65d86e0
feat(ci): add `lightwalletd_full_sync` test to CI (#4268)
* Temporarily use an earlier lightwalletd version

This checks if commit
e146dbf5c2
contains a mempool refresh deadlock bug.

* Actually rebuild the lightwalletd image

* Delete an unfinished comment

* Remove duplicate test in entrypoint.sh

* Keep a recent change to make tests consistent

* fix(ci): remove not used variable `lwd_state_dir`

* fix(ci): state wast not being added to the image name

* fix(ci): mount a docker volume with lightwalletd dir

If the volume doesn't mount this lwd cached state dir, the content won't be saved to the mounted disk in the VM

* fix(ci): lwd state condition

* docs(ci): explain disk mounting logic

* docs(ci): explain disk mounting decision better

* docs(ci): add a description for confusing input names

Co-authored-by: teor <teor@riseup.net>
2022-05-13 15:20:17 +00:00
teor f7a3a0f6bc
fix(build): put gRPC tests behind an optional feature flag to fix production build issues (#4369)
* fix(ci): sentry is not longer being activated in test builds

This removes sentry from all the test execution, as some tests might fail as sentry wasn't initially built, or it might take more time to build as it will have to build with sentry.

* fix(build): workaround the failed to fetch oauth token error

* Drop sentry dependencies when enable-sentry feature is disabled

* Make lightwalletd gRPC tests depend on a new lightwalletd-grpc-tests feature

* fix(ci): remove enable-sentry feature from tests

* Add lightwalletd-grpc-tests feature for functionality or efficiency

And document where it is just used to stop re-compilations.

* Remove redundant `cmake` and `protobuf-compiler` dependencies

* Document Zebra's optional production and test feature flags

* Minimise dependencies in zcash-params/Dockerfile

* Minimise dependencies in docker/Dockerfile

* Add a workflow TODO

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-05-11 14:06:58 +00:00
teor fee10ae014
Add height and hash info to syncer errors (#4287) 2022-05-11 06:51:06 +00:00
teor 17c5e9ad6f
Drop sentry dependencies when enable-sentry feature is disabled (#4372) 2022-05-11 05:00:14 +00:00
dependabot[bot] 41870e5b5a
build(deps): bump inferno from 0.11.2 to 0.11.3 (#4357)
Bumps [inferno](https://github.com/jonhoo/inferno) from 0.11.2 to 0.11.3.
- [Release notes](https://github.com/jonhoo/inferno/releases)
- [Changelog](https://github.com/jonhoo/inferno/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jonhoo/inferno/compare/v0.11.2...v0.11.3)

---
updated-dependencies:
- dependency-name: inferno
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-10 03:42:23 +00:00
dependabot[bot] cb0487cec9
build(deps): bump tokio from 1.18.1 to 1.18.2 (#4358)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.1 to 1.18.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.1...tokio-1.18.2)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-10 03:42:08 +00:00
teor a98e9291b5
cleanup(clippy): Remove redundant bindings, allocations, and generics (#4353)
* Remove let bindings with unit values

* Replace push(format!(_)) with write!(_)

* Remove unused lifetime in test
2022-05-10 03:41:51 +00:00
dependabot[bot] f1d6226470
build(deps): bump prost from 0.10.2 to 0.10.3 (#4348)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.10.2 to 0.10.3.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.2...v0.10.3)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-09 20:30:10 +00:00
Conrado Gouvea c190e3027c
v1.0.0-beta.9 release (#4349) 2022-05-09 13:32:53 -03:00
Gustavo Valverde 9226adea97
feat(ci): add `sending_transactions_using_lightwalletd` test to CI (#4267)
* feat(ci): add lightwalletd_*_sync tests to CI

* feat(ci): add lightwalletd RPC call test

* feat(ci): add send transactions test with lwd to CI

* fix(ci): create a variable to run transactions test

* refactor(ci): use docker in docker

This is a workaround for an issue related to disk partitioning, caused by a GCP service called Konlet, while mounting the cached disks to the VM and then to the container

* fix(build): persist docker login credentials

* fix(ci): get sync height from docker logs instead of gcp

* try: use gha cache for faster building

* fix(ci): mount disk in container to make it available in vm

* fix(build): do not invalidate cache between images

* try(docker): invalidate cache as less as possible

* fix(ci): GHA terminal is not a TTY

* fix(build): do not ignore entrypoint.sh

* fix

* fix(ci): mount using root priveleges

* fix(ci): use existing disk as cached state

* fix(ci): wait for disks to get mounted

* force rebuild

* fix failed force

* fix failed commit

* WIP

* fix(ci): some tests does not use a cached state

* wip

* refactor(ci): disk names and job segregation

* fix(ci): do not name boot and attached disk the same

* fix(ci): attach a disk to full sync, to snapshot the state

* fix(ci): use correct disk implementations

* fix(ci): use different disk name to allow test concurrency

* feat(ci): add lightwalledt send transaction test

* cleanup(ci): remove extra tests

* fix(ci): allow disk concurrency with tests

* fix(ci): add considerations for different tests

* fix(reusable): last fixes

* feat(ci): use reusable workflow for tests

* fix(rw): remove nested worflow

* fix(rw): minor fixes

* force rebuild

* fix(rw): do not use an input as job name

* fix(rw): remove variable id

* fix(ci): remove explicit conditions and id

* fix(ci): docker does not need the variable sign ($) to work

* fix(ci): mount typo

* fix(ci): if a sync fails, always delete the instance

This also reduces the amount of jobs needed.

* refactor(ci): make all test depend on the same build

* fix(ci): some tests require multiple variables

* fix(docker): variable substitution

* fix(ci): allow to run multiple commits from a PR at once

* fix(docker): lower the NETWORK env var for test names

* reduce uneeded diff

* imp(keys): use better naming for builds_disks

* imp(ci): use input defaults

* imp(ci): remove test_name in favor of test_id

* fix(ci): better key naming

* fix(ci): long disk names breaks GCP naming convention

* feat(ci): validate local state version with cached state

* fix(ci): add condition to run tests

* fix: typo

* fix: app_name should not be required

* fix: zebra_state_path shouldn't be required

* fix: reduce diff

* fix(ci): checkout to grep local state version

* Update .github/workflows/test.yml

Co-authored-by: teor <teor@riseup.net>

* revert: merge all tests into a single workflow

* Remove unused STATE_VERSION env var

* fix: minor fixes

* fix(ci): make test.patch the same as test

* fix(ci): negate the input value

* imp(ci): better cached state conditional handling

* imp(ci): exit code is captured by `docker run`

* fix(deploy): mount disks with better write performance

* fix(ci): change sync id to a broader id name

* fix(ci): use correct input validation

* fix(ci): do not make test with cached state dependant on other

* imp(ci): organiza keys better

* fix(ci): use appropiate naming

* fix(ci): create docker volume before mounting

* fix(lint): do not fail on all new changes

* imp(ci): do not report in pr review

* fix(ci): partition clean disks

* fix: typo

* fix: test called the wrong way

* fix(build): stop using gha cache

* ref(ci): validate run condition before calling reusable workflow

* fix(ci): use a better filesystem dir and fix other values

* fix: linting errors

* fix(ci): typo

* Revert "fix(build): stop using gha cache"

This reverts commit a8fbc5f416.

Cache expiration is a lesser evil than not using caching at all and then failing with a 401

* imp(ci): do not set a default for needs_zebra_state

* Update .github/workflows/test.yml

Co-authored-by: teor <teor@riseup.net>

* fix(deps): remove dependencies

* force build

* Update .github/workflows/test.yml

Co-authored-by: teor <teor@riseup.net>

* fix(docker): add RUST_LOG as an ARG and ENV

* fix(test): add `#[ignore]` to send transactions test

This test needs state then it should be marked as #[ignore]

* fix(ci): differentiate between root cache path and its dir

* Remove extra `state` directory

That was a workaround for an issue that has been fixed.

* imp(docs): use better test descriptions

Co-authored-by: teor <teor@riseup.net>

* fix: reduce unwanted diff with main

* fix(ci): make lwd conditions consistent

* Remove another extra `state` directory

Was also part of a workaround for an issue that has been fixed.

* fix(ci): use better conditionals to run test jobs

Co-authored-by: teor <teor@riseup.net>

* Tweak to support different lightwalletd versions

Some versions print `Waiting for block`, and some versions print
`Ingestor waiting for block`.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2022-05-06 05:30:38 +00:00
Janito Vaqueiro Ferreira Filho 9538ad29e5
feat(util): add a `zebra-tip-height` utility (#4289)
* Move `init_tracing` to `lib.rs`

Allow it to be reused by other binaries.

* Fix `hex` dependency inclusion in `zebra-chain`

It would previously fail because `hex` was being included without the
`serde` feature, even though we required that for `transparent::Script`.

* Implement `FromStr` for `Network`

Make it easy to receive it as a command-line parameter.

* Add `zebra-tip-height` utility

Obtains the chain tip height of a directory containing Zebra state.

* Remove Tokio dependency from `zebra-utils`

It wasn't actually used by the `zebra-tip-height` utility.

* Remove `BoxStateService` type alias

It's not needed if we don't return the unused state service. This also
allows removing the `tower` dependency.

* Remove unnecessary attribute

Leftover from copied code.

* Make `cache_dir` argument optional

Fallback to the default Zebra state cache directory.

* Remove added newline

Minor formatting fix, to avoid adding an unnecessary newline.

* Move `tip-height` command into `zebrad`

Make it available as a sub-command in `zebrad`.

* Make some zebrad sub-commands only log warnings

Co-authored-by: teor <teor@riseup.net>
2022-05-06 03:31:52 +00:00
dependabot[bot] f3f26ff85f
build(deps): bump tonic from 0.7.1 to 0.7.2 (#4325)
Bumps [tonic](https://github.com/hyperium/tonic) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.7.2)

---
updated-dependencies:
- dependency-name: tonic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-06 03:31:38 +00:00
dependabot[bot] 6d9f7befc0
build(deps): bump tonic-build from 0.7.1 to 0.7.2 (#4326)
Bumps [tonic-build](https://github.com/hyperium/tonic) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.7.2)

---
updated-dependencies:
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-06 03:31:23 +00:00
dependabot[bot] 89fe4d73fc
build(deps): bump prost from 0.10.1 to 0.10.2 (#4321)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.10.1 to 0.10.2.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.1...v0.10.2)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-05 19:56:26 +00:00
dependabot[bot] f6692d9154
build(deps): bump num-integer from 0.1.44 to 0.1.45 (#4260)
Bumps [num-integer](https://github.com/rust-num/num-integer) from 0.1.44 to 0.1.45.
- [Release notes](https://github.com/rust-num/num-integer/releases)
- [Changelog](https://github.com/rust-num/num-integer/blob/master/RELEASES.md)
- [Commits](https://github.com/rust-num/num-integer/compare/num-integer-0.1.44...num-integer-0.1.45)

---
updated-dependencies:
- dependency-name: num-integer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-05 12:44:00 +00:00
teor d720ab011a
fix(test): make the full sync tests cache state at `/zebrad-cache` (#4308)
* Rewrite the full sync tests to cache at /zebrad-cache

* Improve error reporting from cached state tests
2022-05-05 05:25:46 -04:00
teor 7cc8f7e745
Fix confusing Rust "environmental variable name" variable names (#4301)
```sh
fastmod _VAR "" zebra*
```
2022-05-05 05:59:31 +00:00
teor 5cd1133584
change(test): Stop requiring cached lightwalletd state for the send transaction test (#4302)
* Expect a Zebra cached state with at least a million blocks

* Set the zebrad timeout and failure messages based on the test type

* Temporarily stop requiring cached lightwalletd state for the send transaction tests
2022-05-05 04:44:14 +00:00
Alfredo Garcia d0e81001bc
fix(rpc): Use a structure for parameters of getaddresstxids (#4264)
* add info to lightwalletd_state_path()

* fix getaddresstxids rpc

* normalize more the env vars in lightwalletd tests

* extra logging

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2022-05-05 01:08:27 +00:00
dependabot[bot] 32056e0e5c
build(deps): bump thiserror from 1.0.30 to 1.0.31 (#4259)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.30 to 1.0.31.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.30...1.0.31)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-05 01:08:06 +00:00
Janito Vaqueiro Ferreira Filho 79d58285fb
Increase block validation timeouts (#4156)
* Increase UTXO lookup timeout

Avoid block validation failures because UTXOs aren't available on time.

* Increase the block verification timeout

Attempt to reduce the synchronization restarts and consequently improve
performance.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-05 00:01:12 +00:00
teor 56f766f9b8
fix(sync): fix testnet syncer loop on large Orchard blocks (#4286)
* Return BlockDownloadVerifyError from download_and_verify

* Check block requests and genesis for temporary errors

* Ignore DuplicateBlockQueuedForDownload as a temporary error

* Propagate error info to the syncer main loop

* Sleep after temporary genesis download and verify errors
2022-05-04 22:04:34 +00:00
dependabot[bot] e9d37c6275
build(deps): bump serde_json from 1.0.80 to 1.0.81 (#4290)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.80 to 1.0.81.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.80...v1.0.81)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-04 17:19:59 +00:00
Alfredo Garcia 25712f1935
tests(rpc): Add wallet grpc tests (#4253)
* add a test that call all the lightwalletd grpc methods

* fix some docs

* use ZF address in tests for balance and utxos

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-04 04:00:46 +00:00
dependabot[bot] 2e643c8631
build(deps): bump serde from 1.0.136 to 1.0.137 (#4261)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.136 to 1.0.137.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.136...v1.0.137)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-04 04:00:30 +00:00
dependabot[bot] 1fe66685cb
build(deps): bump serde_json from 1.0.79 to 1.0.80 (#4262)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.79 to 1.0.80.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.79...v1.0.80)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-03 23:34:58 +00:00
dependabot[bot] 47f144205c
build(deps): bump tokio from 1.18.0 to 1.18.1 (#4280)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.0...tokio-1.18.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-03 20:55:43 +00:00
dependabot[bot] 88533d9c40
build(deps): bump semver from 1.0.7 to 1.0.9 (#4258)
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.7 to 1.0.9.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.7...1.0.9)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-03 20:55:20 +00:00
dependabot[bot] 6dc8bea3f5
build(deps): bump tonic-build from 0.7.0 to 0.7.1 (#4249)
Bumps [tonic-build](https://github.com/hyperium/tonic) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.0...v0.7.1)

---
updated-dependencies:
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-02 22:58:12 +00:00
teor 59bdab17bf
test(rpc): Add Rust tests for lightwalletd sync from Zebra (#4177)
* Make the lightwalletd integration test take a test type

* Configure lightwalletd tests based on the test type

* Remove obsolete kill_on_error() in the lightwalletd test

* Refactor to simplify the test function

* Move LightwalletdTestType to the lightwalletd module

* Create a test function that runs the full lightwalletd test suite

* Actually use the cached Zebra state

* Add checks for the new integration test modes

* Populate the lightwalletd state dir in the FullSyncFromGenesis test

* Fix up state handling, fail earlier if state is invalid

* Adjust timeouts and regex escapes

* Make state requirements for each test stricter

* Move configs to the top of the test function

* Allow unexpected lightwalletd cached state in some tests

* Speed up tests slightly by removing an intermittent log check

* Move timeout selection into test type methods

* Move failure messages into test type methods

* Turn a function argument into an enum field

* Check lightwalletd state directly, rather than Zebra RPC results

* Update gRPC tests for function argument changes

* Remove duplicate env var constant and redundant code
2022-04-29 19:56:11 +00:00
dependabot[bot] 1691d9e855
build(deps): bump tokio from 1.17.0 to 1.18.0 (#4242)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.17.0...tokio-1.18.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 17:54:33 +00:00
Janito Vaqueiro Ferreira Filho 5a94a09292
add(test): Integration test to send transactions using lightwalletd (#4068)
* Export the `zebra_state::Config::db_path` method

Make it easier for tests to discover the sub-directory used to store
the chain state data.

* Generate code for interfacing with lightwalletd

Use the `tonic-build` crate to generate Rust code for communicating with
lightwalletd using gRPC.

The `*.proto` files were obtained from the Zcash lightwalletd
repository.

* Use `block::Height` instead of `Height`

Import the `block` instead to make it slightly clearer.

* Add helper function to remove a file if it exists

Try to remove it and ignore an error if it says that the file doesn't
exist. This will be used later to remove the lock file from a copied
chain state directory.

* Add helper function to copy chain state dirs

Copy an existing chain state directory into a new temporary directory.

* Add a `BoxStateService` type alias

Make it easier to write and read a boxed version of a state service.

* Add a helper function to start the state service

Make it easier to specify the state service to use an existing state
cache directory.

* Import `eyre!` macro at the module level

Allow it to be used in different places without having to repeat the
imports.

* Add `load_tip_height_from_state_directory` helper

A function to discover the current chain tip height stored in a state
cache.

* Add helper function to prepare partial sync. state

Loads a partially synchronized cached state directory into a temporary
directory that can be used by a zebrad instance, and also returns the
chain tip block height of that state.

* Add `perform_full_sync_starting_from` helper

Runs a zebrad with an existing partially synchronized state, and
finishes synchronizing it to the network chain tip.

* Add function to load transactions from a block

Use a provided state service to load all transactions from a block at a
specified height.

The state service is a generic type parameter, because
`zebra_state::service::ReadStateService` is not exported publicly. Using
a generic type parameter also allows the service to be wrapped in layers
if needed in the future.

* Add `load_transactions_from_block_after` helper

A function to load transactions from a block stored in a cached state
directory. The cached state must be synchronized to a chain tip higher
than the requested height.

* Add helper function to load some test transactions

Given a partially synchronized chain state, it will extend that chain by
performing a full synchronization, and obtain some transactions from one
of the newly added blocks.

* Update `spawn_zebrad_for_rpc_without_initial_peers`

Wait until the mempool is activated.

* Add method to start lightwalletd with RPC server

Returns the lightwalletd instance and the port that it's listening for
RPC connections.

The instance can reuse an existing cached lightwalletd state if the
`LIGHTWALLETD_DATA_DIR` environment variable is set.

* Add a `LightwalletdRpcClient` type alias

To make it easier to identify the type generated from the Protobuf
files.

* Add helper function to connect to lightwalletd

Prepare an RPC client to send requests to a lightwalletd instance.

* Add a `prepare_send_transaction_request` helper

Creates a request message for lightwalletd to send a transaction.

* Add test to send transactions using lightwalletd

Obtain some valid transactions from future blocks and try to send them
to a lightwalletd instance connected to a zebrad instance that hasn't
seen those transactions yet. The transactions should be successfully
queued in Zebra's mempool.

* Make `zebra_directory` parameter generic

Allow using a `TempDir` or a `PathBuf`.

* Move lightwalletd protobuf files

Place them closer to the module directory, so that it's clearer that
they specify the RPC protocol for lightwalletd, and not Zebra itself.

* Don't use coinbase transactions in the test

Coinbase transactions are rejected by the mempool.

* Don't remove state lock file

It is removed automatically by Zebra when it shuts down, so if it exists
it should be reported as a bug.

* Force mempool to be enabled in Zebrad instance

Speed up the initialization of the Zebrad instance used for lightwalletd
to connect to.

* Refactor to create `LIGHTWALLETD_DATA_DIR_VAR`

Document how the environment variable can be used to speed up the test.

* Check for process errors in spawned Zebra instance

Enable checking for known process failure messages.

* Add `FINISH_PARTIAL_SYNC_TIMEOUT` constant

Document why it exists and how the choice of the value affects the test.

* Add `LIGHTWALLETD_TEST_TIMEOUT` constant

And use it for the Zebrad and the Lightwalletd instances used in the
send transaction integration test.

* Check `lightwalletd` process for errors

Enable checking the lightwalletd process for known failure messages.

* Update `tonic` and `prost` dependencies

Use the latest version and fix CI failures because `rustfmt` isn't
installed in the build environment.

* Create `send_transaction_test` module

Move the send transaction using lightwalletd test and its helper
functions into a new module.

* Move `LIGHTWALLETD_TEST_TIMEOUT` constant

Place it in the parent `lightwalletd` module.

* Move gRPC helper functions and types to `rpc` mod.

Make them more accessible so that they can be used by other tests.

* Create a `cached_state` module

Move the test utility functions related to using a cached Zebra state
into the module.

* Move `perform_full_sync_starting_from` to `sync`

Keep to closer to the synchronization utility functions.

* Move Zebra cached state path variable constant

Place it in the `cached_state` module.

* Skip test if `ZEBRA_TEST_LIGHTWALLETD` is not set

Make it part of the set of tests ignored as a whole if no lightwalletd
tests should be executed.

* Move `spawn_zebrad_for_rpc_without_initial_peers`

Place it in the `launch` sub-module.

* Rename `rpc` module into `wallet_grpc`

Avoid any potential misunderstandings when the name is seen out of
context.

* Allow duplicate `heck` dependency

At least until `structopt` is updated or `zebra-utils` is updated to use
`clap` 3.

* Fix a deny.toml typo

* fix(build): CMake is required by `prost` crate

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-04-27 23:06:11 +00:00
dependabot[bot] 06dc5177f2
build(deps): bump toml from 0.5.8 to 0.5.9 (#4127)
Bumps [toml](https://github.com/alexcrichton/toml-rs) from 0.5.8 to 0.5.9.
- [Release notes](https://github.com/alexcrichton/toml-rs/releases)
- [Commits](https://github.com/alexcrichton/toml-rs/compare/0.5.8...0.5.9)

---
updated-dependencies:
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-27 04:13:19 +00:00
teor d476c18339
fix(sync): Add an extra block retry, to speed up the initial sync (#4185)
* Increase the block download retry limit to 3

* Remove an obsolete timing check from the tests

We cancel all block downloads for each sync restart,
so we don't need to wait for the maximum hedged download time.
2022-04-26 16:28:09 +00:00
Conrado Gouvea 910f4cbf40
RPC test with fully synced Zebra (#4157)
* partial implementation; starts zebra from cached state

* add fully_synced_rpc_test()

* use block from MAINNET_BLOCKS
2022-04-26 05:32:27 +00:00
dependabot[bot] a90b20f222
build(deps): bump inferno from 0.11.1 to 0.11.2 (#4191)
Bumps [inferno](https://github.com/jonhoo/inferno) from 0.11.1 to 0.11.2.
- [Release notes](https://github.com/jonhoo/inferno/releases)
- [Changelog](https://github.com/jonhoo/inferno/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jonhoo/inferno/compare/v0.11.1...v0.11.2)

---
updated-dependencies:
- dependency-name: inferno
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-25 21:00:14 +00:00
Conrado Gouvea 315eebf09c
fix(test): don't create another 'state' directory when implementing ZebradTestDirExt (#4158)
* fix(test): don't create another 'state' directory when implementing ZebradTestDirExt

* fix zebra_state_conflict() test
2022-04-22 01:10:28 +00:00
Conrado Gouvea dff25473aa
refactor: document coinbase rules, refactor to ease understanding (#4056)
* refactor: document coinbase rules, refactor to ease understanding

* Update zebra-consensus/src/block/check.rs

Co-authored-by: teor <teor@riseup.net>

* remove no longer used contains_prevout_input()

* remove unused CoinbaseHasPrevOutInput

* update coinbase description in documentation

Co-authored-by: teor <teor@riseup.net>
2022-04-20 09:31:12 +00:00
Janito Vaqueiro Ferreira Filho c47dac8d5f
change(test): Refactor how extra arguments are handled when spawing lightwalled (#4067)
* Join similar imports

Avoid the confusion that might cause one to think that they come from
different modules or crates.

* Create an `Arguments` helper type

A type to keep track of a list of arguments for a sub-process. It makes
it easier for overriding parameters with new values.

* Create an `args!` helper macro

Make it simpler to create `Arguments` instances with known values.

* Require `Arguments` for `spawn_child` method

Change the method to have an `Arguments` parameter, and merge it with
some default values before passing them forward.

* Use `Arguments` in `spawn_lightwalletd_child`

Change the method to use an `Arguments` instance, and merge it with some
default options.

* Use `Arguments` in `spawn_child_with_command`

Require an `Arguments` instance in the `spawn_child_with_command`
extension method. Makes it simpler to call from `spawn_child` and
`spawn_lightwalletd_child` extension methods.

* Test if argument order is preserved

Check that when building an `Arguments` instance, the order that the
arguments are set is preserved in the generated list of strings.

* Refactor test to improve readability

Also separates some common code to be reused by later tests.

* Test overriding arguments

Check to see if overriding arguments behaves as expected, by keeping the
argument order when overriding and not introducing duplicates.

* Refactor test to improve readability

Move out a chunk of code so that the test itself is easier to read and
to make that code reusable by a later test.

* Test that `Arguments` instances can be merged

Merge two `Arguments` instances built from two lists of arguments, and
check that the expanded strings preserve order and override rules.

* Add Eq derives on Arguments

Co-authored-by: teor <teor@riseup.net>
2022-04-19 10:28:52 +00:00
teor 75671c53a2
1.0.0-beta.8 Release (#4135)
* Update changelog for 1.0.0-beta.8

* Bump Zebra crate versions

The tower crates and their dependencies haven't changed,
so their versions don't need to be bumped.

Some Zebra crates haven't changed,
but I bumped all the versions to keep them consistent.

* Explicitly say that we support NU5 testnet 2

* Expand PR #3799 into changelog entries

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-19 13:02:17 +10:00
Alfredo Garcia d09769714f
feat(rpc): Implement an RPC transaction queue (#4015)
* Add a rpc queue

* Implement the rpc queue

* Add rpc queue tests

* Remove mutex, use broadcast channel

* Have order and limit in the queue

* fix multiple transactions channel

* Use a network argument

* Use chain tip to calculate block spacing

* Add extra time

* Finalize the state check test

* Add a retry test

* Fix description

* fix some docs

* add additional empty check to `Runner::run`

* remove non used method

* ignore some errors

* fix some docs

* add a panic checker to the queue

* add missing file changes for panic checker

* skip checks and retries if height has not changed

* change constants

* reduce the number of queue test cases

* remove suggestion

* change best tip check

* fix(rpc): Check for panics in the transaction queue (#4046)

* Check for panics in the RPC transaction queue

* Add missing pin! and abort in the start task

* Check for transaction queue panics in tests

* Fixup a new RPC test from the main branch

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-12 15:06:29 +10:00
teor 13c216cd3c
1.0.0-beta.7 Release (#4039)
* Update changelog for v1.0.0-beta.7

* Increment all crate versions

* Remove redundant release test that is now covered by CI

* Remove completed NU5 README check task from the release template

* Add Merge Freeze tool to the release checklist

* Simplify release checklist by removing unused steps
2022-04-05 18:06:46 +00:00
dependabot[bot] 50f6bea8fe
build(deps): bump semver from 1.0.6 to 1.0.7 (#3982)
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.6 to 1.0.7.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.6...1.0.7)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-31 01:33:58 +00:00
dependabot[bot] 2ecb7739d4
build(deps): bump indexmap from 1.8.0 to 1.8.1 (#4003)
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.8.0 to 1.8.1.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/bluss/indexmap/compare/1.8.0...1.8.1)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-31 01:33:47 +00:00
teor ce51ad060f
fix(rpc): use the correct RPC error code for missing blocks (#3977)
* Return error code -8 for missing blocks, like zcashd does

* Test that getblock returns error code -8 for missing blocks

* Make RegexSet failures easier to read in logs

* Update lightwalletd acceptance tests for log message changes

* Add extra failure strings for lightwalletd logs

* Improve link formatting

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-03-30 23:34:52 +00:00
teor d767f43fd9
fix(build): Stop using a test-only method in production code (#4000)
* Make proptest-impl features consistently depend on each other

* Use a non-test method for consensus checks

* Make proptest dependencies the same between crates
2022-03-30 18:23:55 +00:00
dependabot[bot] 5a3222a48f
build(deps): bump hyper from 0.14.17 to 0.14.18 (#3946)
Bumps [hyper](https://github.com/hyperium/hyper) from 0.14.17 to 0.14.18.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.17...v0.14.18)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-29 03:31:44 +00:00
Alfredo Garcia 53bf4c6ba5
1.0.0-beta.6 Release (#3966)
* 1.0.0-beta.6 release version bump

* 1.0.0-beta.6 release update the changelog

* change date of the release

* change description

Co-authored-by: teor <teor@riseup.net>

* remove phrase from description

Co-authored-by: teor <teor@riseup.net>
2022-03-28 22:36:42 +00:00