* temporally upgrade zcash primitives to github main branch
* add allowed git repo in deny.toml
* add quotes to url
* fix denies
* change issue number for TODOs
* fix test name
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
Co-authored-by: Marek <mail@marek.onl>
* validate zebrad cache dir
* update scanner documentation
* clippy
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* add info level by default to scanner application
* remove `RUST_LOG=info` from some of the test examples on how to run
* remove `RUST_LOG=info` from the book sample
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* readd the scanner tests in the new binary context
* remove commented out tests from zebrad
* add Cargo.lock
* add local copy of `ZECPAGES_SAPLING_VIEWING_KEY` to binary scanning tests
* moves scanner-results-reader to zebra-scan, adds zebrad bin to zebra-scan, adds a short sleep before trying to connect to gRPC server in test
* always include zebrad dependency in zebra-scan
* renames zebrad bin in zebra-scan to zebrad-for-scanner
* skip test when a cached state is missing
* remove scanning config from zebrad
* move `scan_task_commands` out of zebrad acceptance tests
* remove scanner config from zebrad
* do not panic if env var is not there
* re enable start_scan_where left, remove shielded-scan feature foir scanning tests in CI
* change app names
* uncomment test
* fix the scanner tests in CI by changing the test command
* add log needed by CI?
* move test
* rename trait, docs fix
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Update `CHANGELOG.md`
* chore: Release
* Update versions in the release dry-run workflow
* Update EOS
* Estimate release height to ~ upcoming Tuesday
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* Add a "Summary" title to the changelog
* Remove the summary title
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Uses an OS-assigned port when testing rpc_server_spawn
* reads port from logs and enables 2 acceptance tests on Windows
* generalizes read listen addr from logs fn and re-enables another test
* updates zebra-grpc snapshot tests to use OS-assigned port
* Re-enable grpc vectors tests on Windows while avoiding port conflicts
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* upgrade zcash_client_backend for zebra-scan
* leave zebra-utils untouched
* remove unused
* upgrade zcash_primitives and orchard only for zebra-chain crate
* update and use TryFrom for amounts
* fix imports in serialize
* leave doc as it was
* leave doc as it was 2
* use `try_into` for amount
* use `zip_212_enforcement`
* updgrades primitives in zebra-rpc
* upgrade ecc dependencies for cargo-utils
* fix threading issue
* remove non needed Arc
* update deny.toml
* update primitives for zebra-grpc
* fix doc
* remove non needed single thread code in test
* cleanup some tests
* improve a bit the `ready_scan_block_keys` function
* clippy
* add spawn back in `scan_height_and_store_results`
* remove todo
* add note comment
* use a more explicit import of sapling stuff in serialize
* change(scan): Refactor scanning keys (#8577)
* Refactor converting dfvks to scanning keys
* Simplify handling of scanning keys
* Remove `test-dependencies` from the scanner reader
* Add comments
---------
Co-authored-by: Marek <mail@marek.onl>
* changelog and readme for 1.7.0 release
* chore: Release
* update zebra-scan and zebra-grpc to the release crates dry run script
* update ESTIMATED_RELEASE_HEIGHT
* update the release period to 6 weeks
* update release block
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* update zcash_script and zcash_primitives
* restore windows support
* add a windows config file
* try exact output from CI
* ignore config test in windows
* disable test for windows
* remove test for windows
* change zcash_script branch to release
* bump top the last zcash_script release version
* restore `rejection_restores_internal_state_genesis` test
* fix typo in test name and enable single thread on windows
* disable single thread
* Bump `chrono` from 0.4.34 to 0.4.38
* Stop using the deprecated `from_timestamp_opt`
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* minor cleanup and rename
* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.
* Updates tests
* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`
* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing
* Applies some suggestions from code review
* Applies suggestions from code review
* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network
* Applies more suggestions from code review.
* moves conversions to zcash_primitives::consensus::Network to where they're used.
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* rename `network` variables and method names typed as NetworkKind to `network_kind`
* use only test block heights for the network associated with them
* Applies more suggestions from code review.
* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module
* adds activation heights field
* updates stored test config, adds a quicker test for checking that stored configs can be parsed, adds an intermediate representation of activation heights
* implement Parameters for Network
* Passes &Network directly instead of converting to zp_consensus::Network where there were conversions
* fixes a bad merge (removes a network conversion in zcash_note_encryption)
* Adds a test for the Parameters impl for zebra_chain::Network
* fixes doc links
* - Makes the `activation_heights` config field optional by adding a #[serde(default)]
- Panics if a non-zero activation height is provided for the `Genesis` network upgrade
- Always sets the `Genesis` and `BeforeOverwinter` network upgrade activation heights to 0 and 1, `BeforeOverwinter` could be overwritten by a later network upgrade
- Makes the `activation_heights` field on `Parameters` private, adds/uses an accessor method instead, and adds a builder struct and `build()` method
* small refactor of activation_heights() method
* check that activation heights are in the right order
* Updates `NetworkUpgrade::activation_height()` to return the height of the next NetworkUpgrade if it doesn't find the activation height of `&self`
* checks that the miner address is of TestnetKind on Regtest and update assertion message
* Adds a DNetworkUpgradeActivationHeights struct for better control over how activation heights can be configured
* moves all ordered network upgrades to a constant, adds a no_duplicates test, moves struct with activation heights outside deserialization impl and accepts it in `ParametersBuilder::activation_heights()` instead of a Vec
* panics if any network upgrades are configured to activate at Height(0) because it's reserved for Genesis
* Simplifies the `ParametersBuilder::activation_heights()` method and removes an unnecessary test.
* Adds Sapling HRPs as fields on testnet params. (#8398)
* Applies suggestions from code review.
* Update zebra-chain/src/parameters/network_upgrade.rs
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* minor cleanup and rename
* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.
* Updates tests
* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`
* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing
* Applies some suggestions from code review
* Applies suggestions from code review
* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network
* Applies more suggestions from code review.
* moves conversions to zcash_primitives::consensus::Network to where they're used.
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* rename `network` variables and method names typed as NetworkKind to `network_kind`
* use only test block heights for the network associated with them
* Applies more suggestions from code review.
* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module
---------
Co-authored-by: Marek <mail@marek.onl>
* Bump `clap` from 4.5.3 to 4.5.4
* Bump `indexmap` from 2.2.5 to 2.2.6
* Bump `rayon` from 1.9.0 to 1.10.0
* Bump `tokio` from 1.36.0 to 1.37.0
* Bump `tokio-stream` from 0.1.14 to 0.1.15
* Bump `regex` from 1.10.3 to 1.10.4
* Bump `insta` from 1.36.1 to 1.38.0
* Bump `serde_json` from 1.0.113 to 1.0.115
* Bump `prost` from 0.12.2 and 0.12.3 to 0.12.4
* Bump `bitflags` from 2.4.2 to 2.5.0
* Bump `bs58` from 0.5.0 to 0.5.1
* Bump `incrementalmerkletree` from 0.5.0 to 0.5.1
* Bump `zcash_address` from 0.3.1 to 0.3.2
* Bump `bytes` from 1.5.0 to 1.6.0
* Bump `syn` from 2.0.53 to 2.0.58
* Bump `quote` from 1.0.35 to 1.0.36
* Bump `serde_yaml` from 0.9.33 to 0.9.34+deprecated
* Bump `primitive-types` + `zcash_history`
* Bump `jsonrpc` from 0.17.0 to 0.18.0
* removed derive copy from network, address and ledgerstate
* changed is_max_block_time_enforced to accept ref
* changed NetworkUpgrade::Current to accept ref
* changed NetworkUpgrade::Next to accept ref
* changed NetworkUpgrade::IsActivationHeight to accept ref
* changed NetworkUpgrade::TargetSpacingForHeight to accept ref
* changed NetworkUpgrade::TargetSpacings to accept ref
* changed NetworkUpgrade::MinimumDifficultySpacing_forHeight to accept ref
* changed NetworkUpgrade::IsTestnetMinDifficultyBlock to accept ref
* changed NetworkUpgrade::AveragingWindowTimespanForHeight to accept ref
* changed NetworkUpgrade::ActivationHeight to accept ref
* changed sapling_activation_height to accept ref
* fixed lifetime for target_spacings
* fixed sapling_activation_height
* changed transaction_to_fake_v5 and fake_v5_transactions_for_network to accept ref to network
* changed Input::vec_strategy to accept ref to network
* changed functions in zebra-chain/src/primitives/zcash_history.rs to accept ref to network
* changed functions in zebra-chain/src/history_tree.rs to accept ref to network
* changed functions in zebra-chain/src/history_tree.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/address.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/viewing_key* to accept ref to network
* changed functions in zebra-chain/src/transparent/address.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/zcash_primitives.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/zcash_note_encryption.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/history_tree* to accept ref to network
* changed functions in zebra-chain/src/block* to accept ref to network
* fixed errors in zebra-chain::parameters::network
* fixed errors in zebra-chain::parameters::network
* fixed errors in zebra-chain
* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref
* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref
* fixed errors in zebra-chain/src/block/arbitrary.rs
* finished fixing errors in zebra-chain - all crate tests pass
* changed functions in zebra-state::service::finalized_state to accept &Network
* changed functions in zebra-state::service::non_finalized_state to accept &Network
* zebra-state tests run but fail with overflow error
* zebra-state tests all pass
* converted zebra-network -- all crate tests pass
* applied all requested changes from review
* converted zebra-consensus -- all crate tests pass
* converted zebra-scan -- all crate tests pass
* converted zebra-rpc -- all crate tests pass
* converted zebra-grpc -- all crate tests pass
* converted zebrad -- all crate tests pass
* applied all requested changes from review
* fixed all clippy errors
* fixed build error in zebrad/src/components/mempool/crawler.rs
* updates ScanTask.subscribe() to return rsp_tx instead of awaiting it, returns service errors from grpc methods
* Adds a timeout to scan service and improves error message when subscribe responders are dropped
* Returns an error from `RegisterKeys` svc request when no keys are registered
* tests register keys error message
* Adds test for timeouts
* adds TODOs for fixing bugs
* Orders scan service requests in scan method
* removes CheckKeyHashes request
* fixes bad merge conflict resolution
* update test to check downcast to Elapsed error
* update errors in scan method
* update error msg in register_keys method
* fixes compiler error
* added functions for fetching block vectors
* inserted new network methods for vector fetching into zebra-chain
* changed tag back to test/feature=branch
* changed tag back to test/feature=branch
* changed tag back to test/feature=branch
* changed feature tag to proptest-impl, started implementing in zebra-consensus tests
* adding methods to zebra-consensus, lifetime error in src/transaction/tests.rs, needs refactoring
* finished adding methods to zebra-consensus
* finished adding new methods to zebrad
* added new methods to zebra-rpc and zebra-scan
* finished removing statements matching on Network from tests
* updated new error message
* changed get_block_bytes() and get_block_sapling_roots_bytes to return option and removed serialization error types as per requested changes in PR review
* removed match statements from zebra_chain::transaction::arbitrary::test_transactions() and new zebra-grpc tests
* moved zebra-chain::test_utils to zebra-chain::test
* removed get_ prefix from getter methods
* renamed zebra-chain::test to zebra-chain::tests
* renamed zebra-chain::test to zebra-chain::tests
* fixed clippy warnings
* changed block_map to return clone
* merges scan task tests and checks that the scan task removes keys
* Adds test to CI
* Skip to expected block height and assert that the result is from that height
* Updates test docs and uses zebrad state path for scan cache
* Log "Zcash network: {network}" from scan task commands test
* chore(dev-deps): always specify the version for dev-dependencies
* test(tower-batch-control): remove zebra-consensus dev dependency
Copies the Ed25519Verifier code directly to tower-batch-control tests
* test(zebra-consensus): update ed25519 verifier tests with ones from batch-control
* test(zebra-consensus): restore previous timeout values
The timeouts copied from tower-batch-control seems too low
and there must have been a reasoning why were they introduced
* chore: update dev-deps versions to beta.33
* chore(tower-batch-control): remove dev-dependency on metrics
* chore(tower-batch-control): remove zebra-chain dev-dependency
* Update zebra-scan/Cargo.toml
* bump all versions to match current release
* fix missed commas in version bumps
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
* adds tonic-reflection
* adds listen_addr to log
* Adds user guide for scan server to zebra book
* fixes typo
* Applies suggestions from code review
* update link
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
* update scan task to notify subscribe caller once the new subscribed_keys are sent on the watch channel
* Fixes timing bug in scan gRPC method:
Joins register/subscribe scan service calls, sends SubscribeResults request first, and filters out duplicate results from channel
* Removes outdated TODO
* wraps subscribed_keys in an Arc before sending to watch channel, fixes typo
* Remove result senders for keys that have been removed
* Impl generating continuous deserialized blocks
* Make `sapling_efvk_hrp` `pub`
* Don't wait for Sapling activation height in tests
* Set the sleep interval for scan service to 10 secs
* Simplify `sapling_key_to_scan_block_keys`
* Enable mocking Sapling scanning keys for Testnet
* Test the `RegisterKeys` scan service call
* Enable `shielded-scan` for `zebra-chain`
* Use an ephemeral database so results don't persist
* Don't generate blocks when mocking the state
* Improve error messages
* Simplify seeding mocked Sapling viewing keys
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* Use a manual iterator over `Network`
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* processes SubscribeResults messages
* send tx ids of results to the subscribe channel
* replaces BoxError with Report in scan_range
* adds a watch channel for using subscribed_keys in scan_range
* updates args to process_messages in test
* adds a `subscribe` method to ScanTask for sending a SubscribeResults cmd
* updates test for process_messages to cover subscribe cmds
* impls SubscribeResult service request and updates sender type
* adds test for SubscribeResults scan service request
* adds acceptance test
* updates tests and imports
* fixes acceptance test by using spawn_blocking to avoid blocking async executor and setting an appropriate start height
* fixes test
* Applies suggestions from code review.
* use tokio mpsc channel in scan task instead of std/blocking mpsc
* use tokio mpsc channel for results sender
* adds `was_parsed_keys_empty` instead of checking that all the parsed keys are new keys
* fixes test failures related to send errors in scan task
* returns height and key for scan results from subcribe_results results receiver
* hide scan_service mod in zebra-node-service behind feature
* Refactor obtaining of activation heights
* Impl the `RegisterKeys` service request
* Mock viewing keys for tests
* Refactor tests
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove a redundant comment
I don't think we need to assume the genesis block doesn't contain
shielded data as the comment says.
* Avoid using a single-letter variable
* Refactor mocking Sapling scanning keys
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* adds clear_results RPC method for zebra-scan
* adds delete_keys rpc method
* adds docs
* Update zebra-grpc/proto/scanner.proto
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Apply suggestions from code review
* start zebra-scan gRPC server from zebrad start command
* adds a test that the scanner starts with zebrad
* adds a `listen_addr` field to the shielded scan config
* updates test to use a random port and set the listen_addr config field
* fixes test
* Update zebra-scan/src/config.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* fixes panic when trying to open multiple mutable storage instances.
* open db in blocking task
* fixes test
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* moves ScanTask to its own module in service module
* moves `process_messages()` method to scan_task.rs
* uses get_mut() and returns new keys
* updates types and adds scan_until() function
* adds and uses wait_for_height() function
* spawns scan_until_task_handler to poll scan_until_tasks FuturesUnordered
* corrects comment
* adds TODO
* updates expected test log, corrects panic msg, fixes fmt
* moves scan functions to scan_task module
* moves ScanTaskCommand and related method impls to its own module
* moves `ScanTask::mock()` test constructor to its own module
* creates `add_keys` mod and moves `scan_until()` function there
* moves scan task executor to its own module and adds ScanRangeTaskBuilder type
* renames add_keys to scan_range, moves executor to scan_task mod
* adds test for process_messages
* updates scan_height_and_store_results() to skip last_scanned_height check if there is no key height for a key in `parsed_keys`
* updates `default_test_config()`
* adds acceptance test for registering keys in ScanTask
* uses the right feature for the new acceptance test
* Applies suggestions from code review
* handle RemoveKeys command in ScanTask
* implements DeleteKeys request
* uses spawn_blocking and updates comments
* removes spawn_init fn
* adds test for new Storage method
* adds fake_sapling_results helper function
* adds test for scan service DeleteKeys request
* adds TODO for unwrap_or_clone
* checks the keys sent to the scan task for removal
* moves message processing to its own function
* adds a timeout for the scanner task response
* hide mock() methods behind cfg(test) or feature
* adds MAX_REQUEST_KEYS constant
* updates test to insert and delete results for a second key
* Update zebra-scan/src/init.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* test that the expected number of results are in the db
* fix unused import lint
* fixes unused imports
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* adds/impls types in zebra-chain to be used by the zebra-scan RPC server
* Adds methods for parsing and a test
* adds test for key hash encoding
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* refactors viewing key type
* refactors/renames for viewing keys
* fixes doc links
* Apply suggestions from code review
* Apply suggestions from code review
* removes ViewingKeyHash and ViewingKeyWithHash types
* removes `to_bytes` methods
* remove outdated method and call
---------
Co-authored-by: Marek <mail@marek.onl>
Alfredo Garcia
Tomek Piotrowski
dependabot[bot]
Arya
Marek
idky137
bishopcheckmate