* processes SubscribeResults messages
* send tx ids of results to the subscribe channel
* replaces BoxError with Report in scan_range
* adds a watch channel for using subscribed_keys in scan_range
* updates args to process_messages in test
* adds a `subscribe` method to ScanTask for sending a SubscribeResults cmd
* updates test for process_messages to cover subscribe cmds
* impls SubscribeResult service request and updates sender type
* adds test for SubscribeResults scan service request
* adds acceptance test
* updates tests and imports
* fixes acceptance test by using spawn_blocking to avoid blocking async executor and setting an appropriate start height
* fixes test
* Applies suggestions from code review.
* use tokio mpsc channel in scan task instead of std/blocking mpsc
* use tokio mpsc channel for results sender
* adds `was_parsed_keys_empty` instead of checking that all the parsed keys are new keys
* fixes test failures related to send errors in scan task
* returns height and key for scan results from subcribe_results results receiver
* hide scan_service mod in zebra-node-service behind feature
* Refactor obtaining of activation heights
* Impl the `RegisterKeys` service request
* Mock viewing keys for tests
* Refactor tests
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove a redundant comment
I don't think we need to assume the genesis block doesn't contain
shielded data as the comment says.
* Avoid using a single-letter variable
* Refactor mocking Sapling scanning keys
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* adds clear_results RPC method for zebra-scan
* adds delete_keys rpc method
* adds docs
* Update zebra-grpc/proto/scanner.proto
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Apply suggestions from code review
* start zebra-scan gRPC server from zebrad start command
* adds a test that the scanner starts with zebrad
* adds a `listen_addr` field to the shielded scan config
* updates test to use a random port and set the listen_addr config field
* fixes test
* Update zebra-scan/src/config.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* fixes panic when trying to open multiple mutable storage instances.
* open db in blocking task
* fixes test
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* moves ScanTask to its own module in service module
* moves `process_messages()` method to scan_task.rs
* uses get_mut() and returns new keys
* updates types and adds scan_until() function
* adds and uses wait_for_height() function
* spawns scan_until_task_handler to poll scan_until_tasks FuturesUnordered
* corrects comment
* adds TODO
* updates expected test log, corrects panic msg, fixes fmt
* moves scan functions to scan_task module
* moves ScanTaskCommand and related method impls to its own module
* moves `ScanTask::mock()` test constructor to its own module
* creates `add_keys` mod and moves `scan_until()` function there
* moves scan task executor to its own module and adds ScanRangeTaskBuilder type
* renames add_keys to scan_range, moves executor to scan_task mod
* adds test for process_messages
* updates scan_height_and_store_results() to skip last_scanned_height check if there is no key height for a key in `parsed_keys`
* updates `default_test_config()`
* adds acceptance test for registering keys in ScanTask
* uses the right feature for the new acceptance test
* Applies suggestions from code review
* feat(docker): allow users to use Zebra + LWD with persistent states
* fix(docs): better documentation and compose execution
* chore(docs): better context box
* chore: do not commit further user changes to the `.env`
This can always be overriden by forcing this pushes if it was intended
* fix(compose): do not add extra volumes and ports
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* handle RemoveKeys command in ScanTask
* implements DeleteKeys request
* uses spawn_blocking and updates comments
* removes spawn_init fn
* adds test for new Storage method
* adds fake_sapling_results helper function
* adds test for scan service DeleteKeys request
* adds TODO for unwrap_or_clone
* checks the keys sent to the scan task for removal
* moves message processing to its own function
* adds a timeout for the scanner task response
* hide mock() methods behind cfg(test) or feature
* adds MAX_REQUEST_KEYS constant
* updates test to insert and delete results for a second key
* Update zebra-scan/src/init.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* test that the expected number of results are in the db
* fix unused import lint
* fixes unused imports
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* adds/impls types in zebra-chain to be used by the zebra-scan RPC server
* Adds methods for parsing and a test
* adds test for key hash encoding
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* refactors viewing key type
* refactors/renames for viewing keys
* fixes doc links
* Apply suggestions from code review
* Apply suggestions from code review
* removes ViewingKeyHash and ViewingKeyWithHash types
* removes `to_bytes` methods
* remove outdated method and call
---------
Co-authored-by: Marek <mail@marek.onl>
* ref(docker): Consolidate all tests in the `entrypoint.sh` script
* fix(ci): missing docker argument
* fix(ci): do not blank `ENTRYPOINT_FEATURES` vairable
* fix(docker): do not add `shielded-scan` as part of `TEST_FEATURES`
* chore: remove misleading comment
* feat(docker): add `docker-compose` file to run CI locally
* fix(docs): do not remove previous sections until we have a substitute for users
* fix(docs): revert some changes
* fix: typos
* ref(docker): Consolidate all tests in the `entrypoint.sh` script
* fix(ci): missing docker argument
* fix(ci): do not blank `ENTRYPOINT_FEATURES` vairable
* fix(docker): do not add `shielded-scan` as part of `TEST_FEATURES`
* chore: remove misleading comment
* Update docker/Dockerfile
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* ref(docker): Consolidate all tests in the `entrypoint.sh` script
* fix(ci): missing docker argument
* fix(ci): do not blank `ENTRYPOINT_FEATURES` vairable
* fix(docker): do not add `shielded-scan` as part of `TEST_FEATURES`
* chore: remove misleading comment
* Update docker/Dockerfile
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* add `zebra-grpc` crate
* add missing fields
* convert to a lib
* add zebra-scan and tonic as depenency
* add a getinfo grpc
* remove zebra-scanner dependency
* Adds scan_service field to scanner grpc server
* remove dependency
* test launching the grpc server from the zebra-scan crate (not building)
* fix async issue
* fixes build issues
* add binary for manual testing
* try fix try run
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Add the `scan_start_where_left` test to CI
* Fix passing of features
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
PR #7886 introduced this link to docs.rs for the docs of the public
APIs: https://docs.rs/releases/search?query=zebra. However, the query in
the link returns crates that are not part of the Zebra's public API,
and doesn't return the two tower-* crates that are part of Zebra.
This commit refers to a manually created list of links pointing to the
docs of the public APIs of all current Zebra crates.
We recently updated the `jsonrpc` dependency from 0.16 to 0.17. The
former version used to implicitly send an array of params, even if a
single param was passed to the request builder. The new version doesn't
do that anymore, but Zebra expects an array, so this commit explicitly
passes an array consisting of a single param to the request builder.
* Fix the rendering of expandable content
* Refactor the docs refs in the readme
* Change the search query for docs.rs
* Refactor the landing docs for the `zebrad` crate
* List Zebra crates in the landing docs for `zebrad`
* Remove `## Future Work` from the main readme
This section doesn't say anything useful.
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Patch equihash to use the solver branch
* Add an internal-miner feature and set up its dependencies
* Remove 'Experimental' from mining RPC docs
* Fix a nightly clippy::question_mark lint
* Move a byte array utility function to zebra-chain
* fixup! Add an internal-miner feature and set up its dependencies
* Add an equihash::Solution::solve() method with difficulty checks
* Check solution is valid before returning it
* Add a TODO to check for peers before mining
* Move config validation into GetBlockTemplateRpcImpl::new()
* fixup! fixup! Add an internal-miner feature and set up its dependencies
* Use the same generic constraints for GetBlockTemplateRpcImpl struct and impls
* Start adding an internal miner component
* Add the miner task to the start command
* Add basic miner code
* Split out a method to mine one block
* Spawn to a blocking thread
* Wait until a valid template is available
* Handle shutdown
* Run mining on low priority threads
* Ignore some invalid solutions
* Use a difference nonce for each solver thread
* Update TODOs
* Change the patch into a renamed dependency to simplify crate releases
* Clean up instrumentation and TODOs
* Make RPC instances cloneable and clean up generics
* Make LongPollId Copy so it's easier to use
* Add API to restart mining if there's a new block template
* Actually restart mining if there's a new block template
* Tidy instrumentation
* fixup! Move config validation into GetBlockTemplateRpcImpl::new()
* fixup! Make RPC instances cloneable and clean up generics
* Run the template generator and one miner concurrently
* Reduce logging
* Fix a bug in getblocktemplate RPC tip change detection
* Work around some watch channel change bugs
* Rate-limit template changes in the receiver
* Run one mining solver per available core
* Use updated C code with double-free protection
* Update to the latest solver branch
* Return and submit all valid solutions
* Document what INPUT_LENGTH means
* Fix watch channel change detection
* Don't return early when a mining task fails
* Spawn async miner tasks to avoid cooperative blocking, deadlocks, and improve shutdown responsiveness
* Make existing parallelism docs and configs consistent
* Add a mining parallelism config
* Use the minimum of the configured or available threads for mining
* Ignore optional feature fields in tests
* Downgrade some frequent logs to debug
* Document new zebrad features and tasks
* Describe the internal-miner feature in the CHANGELOG
* Update dependency to de-duplicate equihash solutions
* Use futures::StreamExt instead of TryStreamExt
* Fix a panic message typo
* Note default path to config file in README and docs
* Applies suggestions from review & adds new test config file
* removes default path for 'Other' OSes from ZebradConfig docs
* fixes config_tests
Arya
dependabot[bot]
Alfredo Garcia
Marek
Pili Guerra
Gustavo Valverde
teor
bishopcheckmate
Marius Kjærstad