Commit Graph

1372 Commits

Author SHA1 Message Date
Marek 736f5f5415
change(scan): Remove support for starting the scanner at `zebrad` startup (#8594)
* Remove code that starts the scanner

* Update CHANGELOG

* Disable tests of the scanner

* Disable scanning tests in CI

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-06-12 18:39:00 +00:00
Marek 158d6879fc
fix: General conditional compilation attributes (#8602)
* Remove general conditional compilation flags

See https://blog.rust-lang.org/2024/05/06/check-cfg.html

The conditional compilation flags for tor were superfluous as we can
make do fine with using only features.

* Update changelog

* Add `tokio_unstable` to to `unexpected_cfgs`

* Update changelog

* Fix `tokio_unstable` in `zebrad`
2024-06-12 01:04:30 +00:00
dependabot[bot] e4e54e4b30
build(deps): bump the prod group with 2 updates (#8571)
Bumps the prod group with 2 updates: [serde](https://github.com/serde-rs/serde) and [syn](https://github.com/dtolnay/syn).


Updates `serde` from 1.0.202 to 1.0.203
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.202...v1.0.203)

Updates `syn` from 2.0.65 to 2.0.66
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.65...2.0.66)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-29 14:09:29 +00:00
Arya 3147d20f18
add(book): Adds a section on Regtest usage to the Zebra book (#8526)
* Always set mempool.debug_enable_at_height to 0 on Regtest

* Adds a Regtest with Zebra page to the book

* Deduplicates calls to is_regtest() in start fn

* upgrade min rust version in zebrad

* Display full network name in logs except for Mainnet and default Testnet

* Add example logs/code to regtest section of the book

* apply some extra information

Co-authored-by: Arya <aryasolhi@gmail.com>

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
2024-05-22 15:14:22 +00:00
dependabot[bot] 0cceb6abda
build(deps): bump the prod group with 7 updates (#8557)
Bumps the prod group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [serde](https://github.com/serde-rs/serde) | `1.0.201` | `1.0.202` |
| [toml](https://github.com/toml-rs/toml) | `0.8.12` | `0.8.13` |
| [thiserror](https://github.com/dtolnay/thiserror) | `1.0.60` | `1.0.61` |
| [insta](https://github.com/mitsuhiko/insta) | `1.38.0` | `1.39.0` |
| [prost](https://github.com/tokio-rs/prost) | `0.12.4` | `0.12.6` |
| [itertools](https://github.com/rust-itertools/itertools) | `0.12.1` | `0.13.0` |
| [syn](https://github.com/dtolnay/syn) | `2.0.63` | `2.0.65` |


Updates `serde` from 1.0.201 to 1.0.202
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.201...v1.0.202)

Updates `toml` from 0.8.12 to 0.8.13
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.8.12...toml-v0.8.13)

Updates `thiserror` from 1.0.60 to 1.0.61
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.60...1.0.61)

Updates `insta` from 1.38.0 to 1.39.0
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.38.0...1.39.0)

Updates `prost` from 0.12.4 to 0.12.6
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.12.4...v0.12.6)

Updates `itertools` from 0.12.1 to 0.13.0
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.12.1...v0.13.0)

Updates `syn` from 2.0.63 to 2.0.65
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.63...2.0.65)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-20 19:50:59 +00:00
Arya c494aa7bd1
change(network): Allow custom testnets to make peer connections and configure more parameters (#8528)
* Allow custom testnets to define more parameters and to make peer connections based on the initial_testnet_peer config field

* Updates latest stored test config

* Update zebra-network/src/config.rs

Co-authored-by: Arya <aryasolhi@gmail.com>

---------

Co-authored-by: Marek <mail@marek.onl>
2024-05-20 13:11:34 +00:00
Arya 5ca40aa58b
add(network): Adds a network magic field to `testnet::Parameters` and the config (#8524)
* Adds a `target_difficulty_limit` field on `testnet::Parameters`

* updates test to increment block nonce until finding a block that's below the difficulty threshold

* increment the nonce while the difficulty is invalid instead of while the difficulty threshold is invalid

* Adds comments

* moves network Magic type from zebra-network to zebra-chain

* Adds `network_magic` field to `testnet::Parameters` and uses the regtest network magic from zcashd

* Add a network magic config field for custom testnets

* Adds/updates tests

* Update zebra-chain/src/parameters/network/testnet.rs

* Adds a link to the Regtest network magic in zcashd

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-05-18 20:30:59 +00:00
Arya 1484d3de70
Disables several tests on Windows to avoid port conflicts (#8551) 2024-05-18 12:54:23 +00:00
Arya 99b017e2a4
add(consensus): Add a `target_difficulty_limit` field on `testnet::Parameters` (#8518)
* Adds a `target_difficulty_limit` field on `testnet::Parameters`

* updates test to increment block nonce until finding a block that's below the difficulty threshold

* increment the nonce while the difficulty is invalid instead of while the difficulty threshold is invalid

* Adds comments
2024-05-17 23:06:08 +00:00
Arya 0f5450f565
change(release): Reduce the end of support time from 20 weeks to 16 weeks (#8530)
* Reduce the end of support time from 20 weeks to 16 weeks.

* Update release-process section of the book

* Adds changelog entry

* Update CHANGELOG.md

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-05-15 21:12:52 +00:00
dependabot[bot] e8a1b17b55
build(deps): bump the prod group with 5 updates (#8519)
Bumps the prod group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [semver](https://github.com/dtolnay/semver) | `1.0.22` | `1.0.23` |
| [serde](https://github.com/serde-rs/serde) | `1.0.200` | `1.0.201` |
| [thiserror](https://github.com/dtolnay/thiserror) | `1.0.59` | `1.0.60` |
| [serde_json](https://github.com/serde-rs/json) | `1.0.116` | `1.0.117` |
| [syn](https://github.com/dtolnay/syn) | `2.0.60` | `2.0.63` |


Updates `semver` from 1.0.22 to 1.0.23
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.22...1.0.23)

Updates `serde` from 1.0.200 to 1.0.201
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.200...v1.0.201)

Updates `thiserror` from 1.0.59 to 1.0.60
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.59...1.0.60)

Updates `serde_json` from 1.0.116 to 1.0.117
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.116...v1.0.117)

Updates `syn` from 2.0.60 to 2.0.63
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.60...2.0.63)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-14 00:49:21 +00:00
Arya a6569d4881
change(mining): Restores parts of the internal-miner feature for use on Regtest (#8506)
* Allow configurable Nu5 activation height on Regtest

* Fixes test

* removes outdated TODO

* Adds `current_with_activation_height()` method and uses it in `Commitment::from_bytes()`

* Enables parts of the internal miner for use on Regtest when the solution isn't checked

* Update internal miner config field docs
2024-05-10 18:28:05 +00:00
Arya 15e1096826
change(consensus): Allow configurable NU5 activation height on Regtest (#8505)
* Allow configurable Nu5 activation height on Regtest

* Fixes test

* removes outdated TODO

* Adds `current_with_activation_height()` method and uses it in `Commitment::from_bytes()`
2024-05-08 14:46:13 +00:00
Alfredo Garcia 0ca01e6a46
chore: Release v1.7.0 (#8507)
* changelog and readme for 1.7.0 release

* chore: Release

* update zebra-scan and zebra-grpc to the release crates dry run script

* update ESTIMATED_RELEASE_HEIGHT

* update the release period to 6 weeks

* update release block

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2024-05-07 14:25:50 +00:00
Alfredo Garcia 239fcc85ba
feat(network): Configurable external address for Version network messages (#8488)
* add configurable external address for Version network messages

* apply suggestions from code review

Co-authored-by: Arya <aryasolhi@gmail.com>

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-05-03 01:07:52 +00:00
Arya 8a786fe6ce
add(consensus): Adds block construction and validation for NU5 from block height 1 for Regtest (#8475)
* Always activate Canopy at Height(1) on Regtest

* Ignores the zip 212 grace period on configured Testnets and Regtest

* - Returns early when there is no Heartwood activation height when creating or updating HistoryTree
- Skips call to `check::legacy_chain()` when no NU5 activation height is set (it would return immediately anyway)
- Replaces `.map()` with `.filter_map()` in `NetworkUpgrade::target_spacings()`
- Removes outdated TODO

* - When proof of work is disabled, skips checking if Zebra is synced to the network tip in the getblocktemplate method

* Sets full_verifier_utxo_lookahead to Height::MIN instead of panicking

* When network is regtest, skips starting sync task and commits the genesis block if it's missing in the state

* updates/fixes test config

* Adds test for committing Canopy blocks on Regtest

* - Updates median time past and difficulty checks to use fewer than 11/17/28 blocks

* uses SLOW_START_INTERVAL of 0 if PoW is disabled, adds TODOs

* Update getblocktemplate method to return reserved chain history activation root hash, uses Nu5 at height 1

Test passes.

* Updates test to expect NU5 as the default nu activation at Height(1)

* Removes invalid difficulty snapshot

* fixes tests

* removes regtest NU5 activation height config field

* Apply suggestions from code review

* Update zebra-state/src/service/check/difficulty.rs

Co-authored-by: Marek <mail@marek.onl>

* Update zebra-state/src/service/check/difficulty.rs

* Updates docs/comments, renames an argument

---------

Co-authored-by: Marek <mail@marek.onl>
2024-05-02 19:43:42 +00:00
Arya d84d7356dd
add(consensus): Adds `disable_pow` and `genesis_hash` fields, converts `equihash::Solution` to an enum (#8421)
* minor cleanup and rename

* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.

* Updates tests

* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`

* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing

* Applies some suggestions from code review

* Applies suggestions from code review

* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network

* Applies more suggestions from code review.

* moves conversions to zcash_primitives::consensus::Network to where they're used.

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* rename `network` variables and method names typed as NetworkKind to `network_kind`

* use only test block heights for the network associated with them

* Applies more suggestions from code review.

* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module

* adds activation heights field

* updates stored test config, adds a quicker test for checking that stored configs can be parsed, adds an intermediate representation of activation heights

* implement Parameters for Network

* Passes &Network directly instead of converting to zp_consensus::Network where there were conversions

* fixes a bad merge (removes a network conversion in zcash_note_encryption)

* Adds a test for the Parameters impl for zebra_chain::Network

* fixes doc links

* - Makes the `activation_heights` config field optional by adding a #[serde(default)]
- Panics if a non-zero activation height is provided for the `Genesis` network upgrade
- Always sets the `Genesis` and `BeforeOverwinter` network upgrade activation heights to 0 and 1, `BeforeOverwinter` could be overwritten by a later network upgrade
- Makes the `activation_heights` field on `Parameters` private, adds/uses an accessor method instead, and adds a builder struct and `build()` method

* small refactor of activation_heights() method

* check that activation heights are in the right order

* Updates `NetworkUpgrade::activation_height()` to return the height of the next NetworkUpgrade if it doesn't find the activation height of `&self`

* checks that the miner address is of TestnetKind on Regtest and update assertion message

* Adds a DNetworkUpgradeActivationHeights struct for better control over how activation heights can be configured

* moves all ordered network upgrades to a constant, adds a no_duplicates test, moves struct with activation heights outside deserialization impl and accepts it in `ParametersBuilder::activation_heights()` instead of a Vec

* panics if any network upgrades are configured to activate at Height(0) because it's reserved for Genesis

* Simplifies the `ParametersBuilder::activation_heights()` method and removes an unnecessary test.

* Adds Sapling HRPs as fields on testnet params. (#8398)

* Applies suggestions from code review.

* Update zebra-chain/src/parameters/network_upgrade.rs

Co-authored-by: Marek <mail@marek.onl>

* Adds `network_name` field and accessor method on `Parameters`, uses it in the `Display` impl for `Network`

* Removes unnecessary `.filter()`

* adds constraints on valid network names and a test

* adds config field for setting network name, adds "with_" prefix to ParameterBuilder setter methods

* Adds `MainnetKind`, `TestnetKind`, and `RegtestKind` to reserved network names

* updates stored test configs and fixes `network_name()` docs

* Adds a `new_regtest()` method on `Network` and `testnet::Parameters`, updates config deserialization to return an error if the initial_testnet_peers include any default initial peers AND contain configured activation heights

* Updates `activates_network_upgrades_correctly` test to check Regtest activation heights (and default Mainnet/Testnet)

* Refactors if-let & match statement into general match statement, removes constraint against including `testnet_parameters` field/section in the config when using `Mainnet` or `Regtest`

* Removes outdated TODO

* Restores `testnet_parameters` section of the latest stored config.

* Adds `with_sapling_hrps()` method and uses it to set the Regtest HRPs in `new_regtest()`.

Adds a test for Sapling HRP validation

* Checks that default Mainnet/Testnet/Regtest Sapling HRPs pass validation in `with_sapling_hrps()`

* Uses the correct constant in test

* Adds `is_regtest()` methods

* Adds genesis hash methods and fields to `testnet::Parameters` and its builder

* Copies Regtest genesis block from zcashd

* moves genesis hash check to Network.checkpoint_list()

* Checks the first line in the checkpoint list instead of the first item in the BTreeMap

* Checks that there is _some_ genesis block in the checkpoints list in `from_list()`, adds Regtest checkpoints (which only includes the genesis block)

* Adds a doc comment to `ParametersBuilder::default()`

* Adds a `disable_pow` field and skips checking proof of work if it's true

* Makes `equihash::Solution` an enum to support Regtest solutions, adds a test for validating and committing the Regtest genesis block

* use genesis_hash as first checkpoint when checkpoints are missing for configured testnets too

* Applies suggestions from code review.

* Reverts changes to `activation_height()` method

* Avoids panic in test

* Updates test docs

* drop custom panic hooks after expected panics

---------

Co-authored-by: Marek <mail@marek.onl>
2024-04-26 05:30:36 +00:00
Arya e5d4d739b7
add(consensus/network): Adds `new_regtest()` constructors to `testnet::Parameters` and `Network` (#8413)
* minor cleanup and rename

* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.

* Updates tests

* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`

* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing

* Applies some suggestions from code review

* Applies suggestions from code review

* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network

* Applies more suggestions from code review.

* moves conversions to zcash_primitives::consensus::Network to where they're used.

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* rename `network` variables and method names typed as NetworkKind to `network_kind`

* use only test block heights for the network associated with them

* Applies more suggestions from code review.

* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module

* adds activation heights field

* updates stored test config, adds a quicker test for checking that stored configs can be parsed, adds an intermediate representation of activation heights

* implement Parameters for Network

* Passes &Network directly instead of converting to zp_consensus::Network where there were conversions

* fixes a bad merge (removes a network conversion in zcash_note_encryption)

* Adds a test for the Parameters impl for zebra_chain::Network

* fixes doc links

* - Makes the `activation_heights` config field optional by adding a #[serde(default)]
- Panics if a non-zero activation height is provided for the `Genesis` network upgrade
- Always sets the `Genesis` and `BeforeOverwinter` network upgrade activation heights to 0 and 1, `BeforeOverwinter` could be overwritten by a later network upgrade
- Makes the `activation_heights` field on `Parameters` private, adds/uses an accessor method instead, and adds a builder struct and `build()` method

* small refactor of activation_heights() method

* check that activation heights are in the right order

* Updates `NetworkUpgrade::activation_height()` to return the height of the next NetworkUpgrade if it doesn't find the activation height of `&self`

* checks that the miner address is of TestnetKind on Regtest and update assertion message

* Adds a DNetworkUpgradeActivationHeights struct for better control over how activation heights can be configured

* moves all ordered network upgrades to a constant, adds a no_duplicates test, moves struct with activation heights outside deserialization impl and accepts it in `ParametersBuilder::activation_heights()` instead of a Vec

* panics if any network upgrades are configured to activate at Height(0) because it's reserved for Genesis

* Simplifies the `ParametersBuilder::activation_heights()` method and removes an unnecessary test.

* Adds Sapling HRPs as fields on testnet params. (#8398)

* Applies suggestions from code review.

* Update zebra-chain/src/parameters/network_upgrade.rs

Co-authored-by: Marek <mail@marek.onl>

* Adds `network_name` field and accessor method on `Parameters`, uses it in the `Display` impl for `Network`

* Removes unnecessary `.filter()`

* adds constraints on valid network names and a test

* adds config field for setting network name, adds "with_" prefix to ParameterBuilder setter methods

* Adds `MainnetKind`, `TestnetKind`, and `RegtestKind` to reserved network names

* updates stored test configs and fixes `network_name()` docs

* Adds a `new_regtest()` method on `Network` and `testnet::Parameters`, updates config deserialization to return an error if the initial_testnet_peers include any default initial peers AND contain configured activation heights

* Updates `activates_network_upgrades_correctly` test to check Regtest activation heights (and default Mainnet/Testnet)

* Refactors if-let & match statement into general match statement, removes constraint against including `testnet_parameters` field/section in the config when using `Mainnet` or `Regtest`

* Removes outdated TODO

* Restores `testnet_parameters` section of the latest stored config.

* Adds `with_sapling_hrps()` method and uses it to set the Regtest HRPs in `new_regtest()`.

Adds a test for Sapling HRP validation

* Checks that default Mainnet/Testnet/Regtest Sapling HRPs pass validation in `with_sapling_hrps()`

* Uses the correct constant in test

* Adds `is_regtest()` methods

* Updates test docs

* drop custom panic hooks after expected panics

---------

Co-authored-by: Marek <mail@marek.onl>
2024-04-25 04:04:05 +00:00
Arya e95a70de7a
increases timeout duration waiting on done notification in non-blocking logger test (#8462) 2024-04-24 20:28:17 +00:00
Alfredo Garcia d869ec6fe4
change(release): Adjust estimated release interval and end of support (#8429)
* change estimated release interval and end of support

* update readme
2024-04-24 19:22:07 +00:00
Alfredo Garcia 5f69870235
bump(deps): Update what we can from the prod group of deps (#8423)
* update partially the prod group of deps

* update prod deps
2024-04-24 13:54:00 +00:00
Arya 275e99ec72
add(chain): Adds a `network_name` field to `testnet::Parameters` (#8411)
* minor cleanup and rename

* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.

* Updates tests

* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`

* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing

* Applies some suggestions from code review

* Applies suggestions from code review

* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network

* Applies more suggestions from code review.

* moves conversions to zcash_primitives::consensus::Network to where they're used.

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* rename `network` variables and method names typed as NetworkKind to `network_kind`

* use only test block heights for the network associated with them

* Applies more suggestions from code review.

* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module

* adds activation heights field

* updates stored test config, adds a quicker test for checking that stored configs can be parsed, adds an intermediate representation of activation heights

* implement Parameters for Network

* Passes &Network directly instead of converting to zp_consensus::Network where there were conversions

* fixes a bad merge (removes a network conversion in zcash_note_encryption)

* Adds a test for the Parameters impl for zebra_chain::Network

* fixes doc links

* - Makes the `activation_heights` config field optional by adding a #[serde(default)]
- Panics if a non-zero activation height is provided for the `Genesis` network upgrade
- Always sets the `Genesis` and `BeforeOverwinter` network upgrade activation heights to 0 and 1, `BeforeOverwinter` could be overwritten by a later network upgrade
- Makes the `activation_heights` field on `Parameters` private, adds/uses an accessor method instead, and adds a builder struct and `build()` method

* small refactor of activation_heights() method

* check that activation heights are in the right order

* Updates `NetworkUpgrade::activation_height()` to return the height of the next NetworkUpgrade if it doesn't find the activation height of `&self`

* checks that the miner address is of TestnetKind on Regtest and update assertion message

* Adds a DNetworkUpgradeActivationHeights struct for better control over how activation heights can be configured

* moves all ordered network upgrades to a constant, adds a no_duplicates test, moves struct with activation heights outside deserialization impl and accepts it in `ParametersBuilder::activation_heights()` instead of a Vec

* panics if any network upgrades are configured to activate at Height(0) because it's reserved for Genesis

* Simplifies the `ParametersBuilder::activation_heights()` method and removes an unnecessary test.

* Adds Sapling HRPs as fields on testnet params. (#8398)

* Applies suggestions from code review.

* Update zebra-chain/src/parameters/network_upgrade.rs

Co-authored-by: Marek <mail@marek.onl>

* Adds `network_name` field and accessor method on `Parameters`, uses it in the `Display` impl for `Network`

* Removes unnecessary `.filter()`

* adds constraints on valid network names and a test

* adds config field for setting network name, adds "with_" prefix to ParameterBuilder setter methods

* Adds `MainnetKind`, `TestnetKind`, and `RegtestKind` to reserved network names

* updates stored test configs and fixes `network_name()` docs

---------

Co-authored-by: Marek <mail@marek.onl>
2024-04-24 11:03:37 +00:00
Alfredo Garcia 8cf0b7a36a
bump(zcash_script): Bump zcash script v0.1.15 and restore Windows support (#8393)
* update zcash_script and zcash_primitives

* restore windows support

* add a windows config file

* try exact output from CI

* ignore config test in windows

* disable test for windows

* remove test for windows

* change zcash_script branch to release

* bump top the last zcash_script release version

* restore `rejection_restores_internal_state_genesis` test

* fix typo in test name and enable single thread on windows

* disable single thread
2024-04-22 22:07:04 +00:00
Marek 2ba837470e
Bump `chrono` from 0.4.34 to 0.4.38 (#8400)
* Bump `chrono` from 0.4.34 to 0.4.38

* Stop using the deprecated `from_timestamp_opt`

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-04-19 23:23:57 +00:00
Arya f8c153991b
add(consensus): Add `activation_heights` field to `NetworkParameters` and implement `Parameters` for `Network` (#8383)
* minor cleanup and rename

* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.

* Updates tests

* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`

* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing

* Applies some suggestions from code review

* Applies suggestions from code review

* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network

* Applies more suggestions from code review.

* moves conversions to zcash_primitives::consensus::Network to where they're used.

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* rename `network` variables and method names typed as NetworkKind to `network_kind`

* use only test block heights for the network associated with them

* Applies more suggestions from code review.

* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module

* adds activation heights field

* updates stored test config, adds a quicker test for checking that stored configs can be parsed, adds an intermediate representation of activation heights

* implement Parameters for Network

* Passes &Network directly instead of converting to zp_consensus::Network where there were conversions

* fixes a bad merge (removes a network conversion in zcash_note_encryption)

* Adds a test for the Parameters impl for zebra_chain::Network

* fixes doc links

* - Makes the `activation_heights` config field optional by adding a #[serde(default)]
- Panics if a non-zero activation height is provided for the `Genesis` network upgrade
- Always sets the `Genesis` and `BeforeOverwinter` network upgrade activation heights to 0 and 1, `BeforeOverwinter` could be overwritten by a later network upgrade
- Makes the `activation_heights` field on `Parameters` private, adds/uses an accessor method instead, and adds a builder struct and `build()` method

* small refactor of activation_heights() method

* check that activation heights are in the right order

* Updates `NetworkUpgrade::activation_height()` to return the height of the next NetworkUpgrade if it doesn't find the activation height of `&self`

* checks that the miner address is of TestnetKind on Regtest and update assertion message

* Adds a DNetworkUpgradeActivationHeights struct for better control over how activation heights can be configured

* moves all ordered network upgrades to a constant, adds a no_duplicates test, moves struct with activation heights outside deserialization impl and accepts it in `ParametersBuilder::activation_heights()` instead of a Vec

* panics if any network upgrades are configured to activate at Height(0) because it's reserved for Genesis

* Simplifies the `ParametersBuilder::activation_heights()` method and removes an unnecessary test.

* Adds Sapling HRPs as fields on testnet params. (#8398)

* Applies suggestions from code review.

* Update zebra-chain/src/parameters/network_upgrade.rs

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2024-04-19 01:20:34 +00:00
Alfredo Garcia 887d4a2457
fix(beta-toolchain): Warnings (#8404)
* fix beta imports and other warnings

* clippy lints beta

* move attributes to after docs

Co-authored-by: Marek <mail@marek.onl>

* move some code to test module

* implement display instead of direct tostring for LongPollId

Co-authored-by: Arya <aryasolhi@gmail.com>

* fix typo

---------

Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Arya <aryasolhi@gmail.com>
2024-04-18 01:45:22 +00:00
Arya 16a39f89d0
add(consensus/network): Add an empty `Parameters` struct in `Network::Testnet` (#8368)
* minor cleanup and rename

* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.

* Updates tests

* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`

* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing

* Applies some suggestions from code review

* Applies suggestions from code review

* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network

* Applies more suggestions from code review.

* moves conversions to zcash_primitives::consensus::Network to where they're used.

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* rename `network` variables and method names typed as NetworkKind to `network_kind`

* use only test block heights for the network associated with them

* Applies more suggestions from code review.

* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module

---------

Co-authored-by: Marek <mail@marek.onl>
2024-04-17 02:20:34 +00:00
Marek 5fd15a022c
build(deps): bump the prod group with 21 updates (#8402)
* Bump `clap` from 4.5.3 to 4.5.4

* Bump `indexmap` from 2.2.5 to 2.2.6

* Bump `rayon` from 1.9.0 to 1.10.0

* Bump `tokio` from 1.36.0 to 1.37.0

* Bump `tokio-stream` from 0.1.14 to 0.1.15

* Bump `regex` from 1.10.3 to 1.10.4

* Bump `insta` from 1.36.1 to 1.38.0

* Bump `serde_json` from 1.0.113 to 1.0.115

* Bump `prost` from 0.12.2 and 0.12.3 to 0.12.4

* Bump `bitflags` from 2.4.2 to 2.5.0

* Bump `bs58` from 0.5.0 to 0.5.1

* Bump `incrementalmerkletree` from 0.5.0 to 0.5.1

* Bump `zcash_address` from 0.3.1 to 0.3.2

* Bump `bytes` from 1.5.0 to 1.6.0

* Bump `syn` from 2.0.53 to 2.0.58

* Bump `quote` from 1.0.35 to 1.0.36

* Bump `serde_yaml` from 0.9.33 to 0.9.34+deprecated

* Bump `primitive-types` + `zcash_history`

* Bump `jsonrpc` from 0.17.0 to 0.18.0
2024-04-17 02:20:28 +00:00
Arya 1498f8e24e
Release v1.6.1 (#8396)
* Updates crates versions

* Updates end of support height

* Updates README

* Updates CHANGELOG

* Updates Known Issues in README

* Updates Changelog

* updates release-crates-dry-run.sh

* updates install.md and docker.md

* Update CHANGELOG.md

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2024-04-15 23:24:40 +00:00
Elijah Hampton 49fca309cf
Adds logging of column family size and database size on startup and s… (#8336)
* Adds logging of column family size and database size on startup and shutdown

* Changes log level of column families size strings to debug. Adds TODO comment to use human_bytes crate for human-readable format of metrics. Adds print_db_metrics function to ZebraDb struct.

* Calls enumerate() on column_families to access index var

* Resolves cargo fmt checker results

* Resolves clippy lint

* Runs and fixes changes from fmt, clippy, check and test

* Removes prop.txt

* minor doc changes

---------

Co-authored-by: Elijah Hampton <elijahhampton@MBP-de-Elijah-2.lan>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-03-29 00:00:28 +00:00
Arya 949594c25a
fixes a new clippy warnings (#8373) 2024-03-25 12:57:08 +00:00
Marek f79fc6aa8e
Bump prod group (#8370) 2024-03-21 18:23:41 +00:00
idky137 4579722833
change(chain): Remove `Copy` trait impl from `Network` (#8354)
* removed derive copy from network, address and ledgerstate

* changed is_max_block_time_enforced to accept ref

* changed NetworkUpgrade::Current to accept ref

* changed NetworkUpgrade::Next to accept ref

* changed NetworkUpgrade::IsActivationHeight to accept ref

* changed NetworkUpgrade::TargetSpacingForHeight to accept ref

* changed NetworkUpgrade::TargetSpacings to accept ref

* changed NetworkUpgrade::MinimumDifficultySpacing_forHeight to accept ref

* changed NetworkUpgrade::IsTestnetMinDifficultyBlock to accept ref

* changed NetworkUpgrade::AveragingWindowTimespanForHeight to accept ref

* changed NetworkUpgrade::ActivationHeight to accept ref

* changed sapling_activation_height to accept ref

* fixed lifetime for target_spacings

* fixed sapling_activation_height

* changed transaction_to_fake_v5 and fake_v5_transactions_for_network to accept ref to network

* changed Input::vec_strategy to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_history.rs to accept ref to network

* changed functions in zebra-chain/src/history_tree.rs to accept ref to network

* changed functions in zebra-chain/src/history_tree.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/address.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/viewing_key* to accept ref to network

* changed functions in zebra-chain/src/transparent/address.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_primitives.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_note_encryption.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/history_tree* to accept ref to network

* changed functions in zebra-chain/src/block* to accept ref to network

* fixed errors in zebra-chain::parameters::network

* fixed errors in zebra-chain::parameters::network

* fixed errors in zebra-chain

* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref

* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref

* fixed errors in zebra-chain/src/block/arbitrary.rs

* finished fixing errors in zebra-chain - all crate tests pass

* changed functions in zebra-state::service::finalized_state to accept &Network

* changed functions in zebra-state::service::non_finalized_state to accept &Network

* zebra-state tests run but fail with overflow error

* zebra-state tests all pass

* converted zebra-network -- all crate tests pass

* applied all requested changes from review

* converted zebra-consensus -- all crate tests pass

* converted zebra-scan -- all crate tests pass

* converted zebra-rpc -- all crate tests pass

* converted zebra-grpc -- all crate tests pass

* converted zebrad -- all crate tests pass

* applied all requested changes from review

* fixed all clippy errors

* fixed build error in zebrad/src/components/mempool/crawler.rs
2024-03-19 20:45:27 +00:00
Arya 5e86ebb699
fix(scan): Improve gRPC method errors and add timeout to scan service to avoid hanging (#8318)
* updates ScanTask.subscribe() to return rsp_tx instead of awaiting it, returns service errors from grpc methods

* Adds a timeout to scan service and improves error message when subscribe responders are dropped

* Returns an error from `RegisterKeys` svc request when no keys are registered

* tests register keys error message

* Adds test for timeouts

* adds TODOs for fixing bugs

* Orders scan service requests in scan method

* removes CheckKeyHashes request

* fixes bad merge conflict resolution

* update test to check downcast to Elapsed error

* update errors in scan method

* update error msg in register_keys method

* fixes compiler error
2024-03-14 20:27:33 +00:00
Za Wilcox 3bef54b764
change(consensus): Refactor production code for network consensus rules to `Network` methods (#8340)
* begin refactor suggested as "step 2": https://github.com/ZcashFoundation/zebra/issues/7968#issue-2003245309
Squashed from multiple commits to enable partial rebase

* break out more little traits

* add activation implementation leveraging From<Network> for lrz::cons::

* for transfer of ownership I cannot return a type that's owned by the method

* hrp_sapling_extended_full_viewing_key

* complete implementation of interface of Parameters on Network reuse Parameters on zcash Network where possible

* move doc-comments to trait declarations (from impls)

* Simplify/complete Parameters impl for Network

* Add checkpoint_list method, move documentation, etc

* move last match network to inside network method

* add back comment

* use zcash_address for parameter types in zebra-chain

* use inherent methods instead of big parameters passthrough

* revert to implementation of From on zcash_primitives::..::Network vs &zcash_prim...

* move match

* add test to block maximum time rule

* update changelog

* finish porting target_difficutly_limit

* remove obscelete code comment

* revert non-functional change

* finish migrating target_difficulty_limit, checkpoint_list

* update changelog

---------

Co-authored-by: Hazel OHearn <gygaxis@zingolabs.org>
2024-03-12 21:41:44 +00:00
idky137 9b91d4bc0e
change(tests): Remove Matches on Network From Tests (#8295)
* added functions for fetching block vectors

* inserted new network methods for vector fetching into zebra-chain

* changed tag back to test/feature=branch

* changed tag back to test/feature=branch

* changed tag back to test/feature=branch

* changed feature tag to proptest-impl, started implementing in zebra-consensus tests

* adding methods to zebra-consensus, lifetime error in src/transaction/tests.rs, needs refactoring

* finished adding methods to zebra-consensus

* finished adding new methods to zebrad

* added new methods to zebra-rpc and zebra-scan

* finished removing statements matching on Network from tests

* updated new error message

* changed get_block_bytes() and get_block_sapling_roots_bytes to return option and removed serialization error types as per requested changes in PR review

* removed match statements from zebra_chain::transaction::arbitrary::test_transactions() and new zebra-grpc tests

* moved zebra-chain::test_utils to zebra-chain::test

* removed get_ prefix from getter methods

* renamed zebra-chain::test to zebra-chain::tests

* renamed zebra-chain::test to zebra-chain::tests

* fixed clippy warnings

* changed block_map to return clone
2024-03-05 09:12:25 -05:00
Arya e64f3dcddd
add(test): Check that scan task removes keys, unify scan task acceptance tests, and add test to CI (#8308)
* merges scan task tests and checks that the scan task removes keys

* Adds test to CI

* Skip to expected block height and assert that the result is from that height

* Updates test docs and uses zebrad state path for scan cache

* Log "Zcash network: {network}" from scan task commands test
2024-03-04 21:25:50 +00:00
Marek 58bfe974e1
chore: Release v1.6.0 (#8319)
* Prepare the CHANGELOG for v1.6.0

* chore: Release

* Set `ESTIMATED_RELEASE_HEIGHT` to 2_413_000

* Revert "chore: Release"

This reverts commit 5261e85c37.

* chore: Release

* Fix release-crates-dry-run

* Temporarily turn off bumps for `zebra-{scan,grpc}`

* Apply suggestions from code review

Co-authored-by: Arya <aryasolhi@gmail.com>

* Re-enable test bump for `zebra-scan`

* Re-enable test bumps for `zebra-{scan, grpc}`

* Turn off dry-run bumps for `zebra-{scan, grpc}`

* Re-enable bumps and disable dry-run publishing

* Re-enable dry-run publishing

* Exclude `zebra-scan` from publishing

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-02-23 23:28:44 +00:00
bishopcheckmate d1d128ac15
chore: allow crater runs (#8171)
* chore(dev-deps): always specify the version for dev-dependencies

* test(tower-batch-control): remove zebra-consensus dev dependency

Copies the Ed25519Verifier code directly to tower-batch-control tests

* test(zebra-consensus): update ed25519 verifier tests with ones from batch-control

* test(zebra-consensus): restore previous timeout values

The timeouts copied from tower-batch-control seems too low
and there must have been a reasoning why were they introduced

* chore: update dev-deps versions to beta.33

* chore(tower-batch-control): remove dev-dependency on metrics

* chore(tower-batch-control): remove zebra-chain dev-dependency

* Update zebra-scan/Cargo.toml

* bump all versions to match current release

* fix missed commas in version bumps

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
2024-02-23 12:47:10 +00:00
Arya 5a9281a7a8
fix(scan): Fix minor concurrency bug in the `scan` gRPC method (#8303)
* update scan task to notify subscribe caller once the new subscribed_keys are sent on the watch channel

* Fixes timing bug in scan gRPC method:

Joins register/subscribe scan service calls, sends SubscribeResults request first, and filters out duplicate results from channel

* Removes outdated TODO

* wraps subscribed_keys in an Arc before sending to watch channel, fixes typo

* Remove result senders for keys that have been removed
2024-02-21 23:29:13 +00:00
dependabot[bot] a8dcd98b6e
build(deps): bump the prod group with 3 updates (#8283)
Bumps the prod group with 3 updates: [clap](https://github.com/clap-rs/clap), [semver](https://github.com/dtolnay/semver) and [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb).


Updates `clap` from 4.5.0 to 4.5.1
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.0...clap_complete-v4.5.1)

Updates `semver` from 1.0.21 to 1.0.22
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.21...1.0.22)

Updates `rocksdb` from 0.21.0 to 0.22.0
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.21.0...v0.22.0)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-19 18:17:34 +00:00
Arya f20834c373
improve error message in non_blocking_logger test (#8276) 2024-02-15 22:46:04 +00:00
dependabot[bot] f342f50938
build(deps): bump the prod group with 14 updates (#8264)
* build(deps): bump the prod group with 14 updates

Bumps the prod group with 14 updates:

| Package | From | To |
| --- | --- | --- |
| [clap](https://github.com/clap-rs/clap) | `4.4.18` | `4.5.0` |
| [chrono](https://github.com/chronotope/chrono) | `0.4.33` | `0.4.34` |
| [indexmap](https://github.com/indexmap-rs/indexmap) | `2.2.2` | `2.2.3` |
| [toml](https://github.com/toml-rs/toml) | `0.8.9` | `0.8.10` |
| [thiserror](https://github.com/dtolnay/thiserror) | `1.0.56` | `1.0.57` |
| [metrics](https://github.com/metrics-rs/metrics) | `0.22.0` | `0.22.1` |
| [num-integer](https://github.com/rust-num/num-integer) | `0.1.45` | `0.1.46` |
| [metrics-exporter-prometheus](https://github.com/metrics-rs/metrics) | `0.13.0` | `0.13.1` |
| [indicatif](https://github.com/console-rs/indicatif) | `0.17.7` | `0.17.8` |
| [tempfile](https://github.com/Stebalien/tempfile) | `3.9.0` | `3.10.0` |
| [tonic](https://github.com/hyperium/tonic) | `0.10.2` | `0.11.0` |
| [tonic-build](https://github.com/hyperium/tonic) | `0.10.2` | `0.11.0` |
| [x25519-dalek](https://github.com/dalek-cryptography/curve25519-dalek) | `2.0.0` | `2.0.1` |
| [serde_with](https://github.com/jonasbb/serde_with) | `3.6.0` | `3.6.1` |


Updates `clap` from 4.4.18 to 4.5.0
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.4.18...clap_complete-v4.5.0)

Updates `chrono` from 0.4.33 to 0.4.34
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.33...v0.4.34)

Updates `indexmap` from 2.2.2 to 2.2.3
- [Changelog](https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/indexmap-rs/indexmap/compare/2.2.2...2.2.3)

Updates `toml` from 0.8.9 to 0.8.10
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.8.9...toml-v0.8.10)

Updates `thiserror` from 1.0.56 to 1.0.57
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.56...1.0.57)

Updates `metrics` from 0.22.0 to 0.22.1
- [Changelog](https://github.com/metrics-rs/metrics/blob/main/release.toml)
- [Commits](https://github.com/metrics-rs/metrics/compare/metrics-v0.22.0...metrics-v0.22.1)

Updates `num-integer` from 0.1.45 to 0.1.46
- [Changelog](https://github.com/rust-num/num-integer/blob/master/RELEASES.md)
- [Commits](https://github.com/rust-num/num-integer/compare/num-integer-0.1.45...num-integer-0.1.46)

Updates `metrics-exporter-prometheus` from 0.13.0 to 0.13.1
- [Changelog](https://github.com/metrics-rs/metrics/blob/main/release.toml)
- [Commits](https://github.com/metrics-rs/metrics/compare/metrics-exporter-prometheus-v0.13.0...metrics-v0.13.1)

Updates `indicatif` from 0.17.7 to 0.17.8
- [Release notes](https://github.com/console-rs/indicatif/releases)
- [Commits](https://github.com/console-rs/indicatif/commits)

Updates `tempfile` from 3.9.0 to 3.10.0
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.9.0...v3.10.0)

Updates `tonic` from 0.10.2 to 0.11.0
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.10.2...v0.11.0)

Updates `tonic-build` from 0.10.2 to 0.11.0
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.10.2...v0.11.0)

Updates `x25519-dalek` from 2.0.0 to 2.0.1
- [Release notes](https://github.com/dalek-cryptography/curve25519-dalek/releases)
- [Commits](https://github.com/dalek-cryptography/curve25519-dalek/compare/2.0.0...x25519-2.0.1)

Updates `serde_with` from 3.6.0 to 3.6.1
- [Release notes](https://github.com/jonasbb/serde_with/releases)
- [Commits](https://github.com/jonasbb/serde_with/compare/v3.6.0...v3.6.1)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: metrics
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: num-integer
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: metrics-exporter-prometheus
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: indicatif
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: tonic
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: x25519-dalek
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: serde_with
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
...

Signed-off-by: dependabot[bot] <support@github.com>

* update deny.toml

* update deny.toml missing dup

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-02-14 12:24:33 +00:00
Arya 3929a526e5
add(scan): Implement SubscribeResults request for scan service (#8253)
* processes SubscribeResults messages

* send tx ids of results to the subscribe channel

* replaces BoxError with Report in scan_range

* adds a watch channel for using subscribed_keys in scan_range

* updates args to process_messages in test

* adds a `subscribe` method to ScanTask for sending a SubscribeResults cmd

* updates test for process_messages to cover subscribe cmds

* impls SubscribeResult service request and updates sender type

* adds test for SubscribeResults scan service request

* adds acceptance test

* updates tests and imports

* fixes acceptance test by using spawn_blocking to avoid blocking async executor and setting an appropriate start height

* fixes test

* Applies suggestions from code review.

* use tokio mpsc channel in scan task instead of std/blocking mpsc

* use tokio mpsc channel for results sender

* adds `was_parsed_keys_empty` instead of checking that all the parsed keys are new keys

* fixes test failures related to send errors in scan task

* returns height and key for scan results from subcribe_results results receiver

* hide scan_service mod in zebra-node-service behind feature
2024-02-13 00:42:40 +00:00
Marek 6b8cbf9904
add(scan): Implement `RegisterKeys` service request to register new keys (#8251)
* Refactor obtaining of activation heights

* Impl the `RegisterKeys` service request

* Mock viewing keys for tests

* Refactor tests

* Apply suggestions from code review

Co-authored-by: Arya <aryasolhi@gmail.com>

* Remove a redundant comment

I don't think we need to assume the genesis block doesn't contain
shielded data as the comment says.

* Avoid using a single-letter variable

* Refactor mocking Sapling scanning keys

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-02-09 15:23:19 +00:00
Arya 2c0bc3ac92
add(scan): Start scanner gRPC server with `zebrad` (#8241)
* adds clear_results RPC method for zebra-scan

* adds delete_keys rpc method

* adds docs

* Update zebra-grpc/proto/scanner.proto

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Apply suggestions from code review

* start zebra-scan gRPC server from zebrad start command

* adds a test that the scanner starts with zebrad

* adds a `listen_addr` field to the shielded scan config

* updates test to use a random port and set the listen_addr config field

* fixes test

* Update zebra-scan/src/config.rs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* fixes panic when trying to open multiple mutable storage instances.

* open db in blocking task

* fixes test

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-02-07 22:36:01 +00:00
dependabot[bot] 1cfed249de
build(deps): bump the prod group with 6 updates (#8230)
Bumps the prod group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [indexmap](https://github.com/indexmap-rs/indexmap) | `2.2.1` | `2.2.2` |
| [toml](https://github.com/toml-rs/toml) | `0.8.8` | `0.8.9` |
| [tokio](https://github.com/tokio-rs/tokio) | `1.35.1` | `1.36.0` |
| [sentry](https://github.com/getsentry/sentry-rust) | `0.32.1` | `0.32.2` |
| [serde_with](https://github.com/jonasbb/serde_with) | `3.5.1` | `3.6.0` |
| [reqwest](https://github.com/seanmonstar/reqwest) | `0.11.23` | `0.11.24` |


Updates `indexmap` from 2.2.1 to 2.2.2
- [Changelog](https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/indexmap-rs/indexmap/compare/2.2.1...2.2.2)

Updates `toml` from 0.8.8 to 0.8.9
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.8.8...toml-v0.8.9)

Updates `tokio` from 1.35.1 to 1.36.0
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.35.1...tokio-1.36.0)

Updates `sentry` from 0.32.1 to 0.32.2
- [Release notes](https://github.com/getsentry/sentry-rust/releases)
- [Changelog](https://github.com/getsentry/sentry-rust/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-rust/compare/0.32.1...0.32.2)

Updates `serde_with` from 3.5.1 to 3.6.0
- [Release notes](https://github.com/jonasbb/serde_with/releases)
- [Commits](https://github.com/jonasbb/serde_with/compare/v3.5.1...v3.6.0)

Updates `reqwest` from 0.11.23 to 0.11.24
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.23...v0.11.24)

---
updated-dependencies:
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: sentry
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: serde_with
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-07 03:18:52 +00:00
Arya 2a004ffe9e
add(scan): Handle RegisterKeys messages in scan task (#8222)
* moves ScanTask to its own module in service module

* moves `process_messages()` method to scan_task.rs

* uses get_mut() and returns new keys

* updates types and adds scan_until() function

* adds and uses wait_for_height() function

* spawns scan_until_task_handler to poll scan_until_tasks FuturesUnordered

* corrects comment

* adds TODO

* updates expected test log, corrects panic msg, fixes fmt

* moves scan functions to scan_task module

* moves ScanTaskCommand and related method impls to its own module

* moves `ScanTask::mock()` test constructor to its own module

* creates `add_keys` mod and moves `scan_until()` function there

* moves scan task executor to its own module and adds ScanRangeTaskBuilder type

* renames add_keys to scan_range, moves executor to scan_task mod

* adds test for process_messages

* updates scan_height_and_store_results() to skip last_scanned_height check if there is no key height for a key in `parsed_keys`

* updates `default_test_config()`

* adds acceptance test for registering keys in ScanTask

* uses the right feature for the new acceptance test

* Applies suggestions from code review
2024-02-06 18:41:00 +00:00
Arya 80827f5294
add(scan): Implement `DeleteKeys` `ScanService` request (#8217)
* handle RemoveKeys command in ScanTask

* implements DeleteKeys request

* uses spawn_blocking and updates comments

* removes spawn_init fn

* adds test for new Storage method

* adds fake_sapling_results helper function

* adds test for scan service DeleteKeys request

* adds TODO for unwrap_or_clone

* checks the keys sent to the scan task for removal

* moves message processing to its own function

* adds a timeout for the scanner task response

* hide mock() methods behind cfg(test) or feature

* adds MAX_REQUEST_KEYS constant

* updates test to insert and delete results for a second key

* Update zebra-scan/src/init.rs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* test that the expected number of results are in the db

* fix unused import lint

* fixes unused imports

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-01-31 19:34:24 +00:00
dependabot[bot] c9401ad754
build(deps): bump the prod group with 7 updates (#8208)
* build(deps): bump the prod group with 7 updates

Bumps the prod group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [chrono](https://github.com/chronotope/chrono) | `0.4.32` | `0.4.33` |
| [indexmap](https://github.com/indexmap-rs/indexmap) | `2.1.0` | `2.2.1` |
| [serde](https://github.com/serde-rs/serde) | `1.0.195` | `1.0.196` |
| [pin-project](https://github.com/taiki-e/pin-project) | `1.1.3` | `1.1.4` |
| [serde_json](https://github.com/serde-rs/json) | `1.0.111` | `1.0.113` |
| [serde_with](https://github.com/jonasbb/serde_with) | `3.5.0` | `3.5.1` |
| [itertools](https://github.com/rust-itertools/itertools) | `0.12.0` | `0.12.1` |


Updates `chrono` from 0.4.32 to 0.4.33
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.32...v0.4.33)

Updates `indexmap` from 2.1.0 to 2.2.1
- [Changelog](https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/indexmap-rs/indexmap/compare/2.1.0...2.2.1)

Updates `serde` from 1.0.195 to 1.0.196
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.195...v1.0.196)

Updates `pin-project` from 1.1.3 to 1.1.4
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.1.3...v1.1.4)

Updates `serde_json` from 1.0.111 to 1.0.113
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.111...v1.0.113)

Updates `serde_with` from 3.5.0 to 3.5.1
- [Release notes](https://github.com/jonasbb/serde_with/releases)
- [Commits](https://github.com/jonasbb/serde_with/compare/v3.5.0...v3.5.1)

Updates `itertools` from 0.12.0 to 0.12.1
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.12.0...v0.12.1)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: pin-project
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: serde_with
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
...

Signed-off-by: dependabot[bot] <support@github.com>

* replaces call to deprecated `IndexMap::remove()` with `swap_remove`

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
2024-01-30 14:00:28 +00:00