* Update build instructions with scanning enabled
* Remove `pkg-config` from build deps in README
I tried compiling Zebra with `--all-features` on a Linux machine without
`pkg-config` or `pkfgconf` (which is an alternative to `pkg-config`),
and it worked.
* Add `protoc` to build deps in README
* chore(dev-deps): always specify the version for dev-dependencies
* test(tower-batch-control): remove zebra-consensus dev dependency
Copies the Ed25519Verifier code directly to tower-batch-control tests
* test(zebra-consensus): update ed25519 verifier tests with ones from batch-control
* test(zebra-consensus): restore previous timeout values
The timeouts copied from tower-batch-control seems too low
and there must have been a reasoning why were they introduced
* chore: update dev-deps versions to beta.33
* chore(tower-batch-control): remove dev-dependency on metrics
* chore(tower-batch-control): remove zebra-chain dev-dependency
* Update zebra-scan/Cargo.toml
* bump all versions to match current release
* fix missed commas in version bumps
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
* adds tonic-reflection
* adds listen_addr to log
* Adds user guide for scan server to zebra book
* fixes typo
* Applies suggestions from code review
* update link
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
* update scan task to notify subscribe caller once the new subscribed_keys are sent on the watch channel
* Fixes timing bug in scan gRPC method:
Joins register/subscribe scan service calls, sends SubscribeResults request first, and filters out duplicate results from channel
* Removes outdated TODO
* wraps subscribed_keys in an Arc before sending to watch channel, fixes typo
* Remove result senders for keys that have been removed
* add grpc method unit tests
* add test module description
* runs tests in sequence, adds some messages to assertions, and minor cleanups (#8296)
* fix field name change
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Adds scan method
* fix clippy warning
* Skip empty results and don't panic if the gRPC client disconnects
* moves `Results` request above `SubscribeResults` request to avoid duplicate results.
* returns early if there's a send error when sending initial results and updates response type
* move responder buffer size to const
* Adds a snapshot test for the `scan` method
* updates snapshots
Containers can generate a significant amount of logs,
consuming disk space over time. Here I added configuration
for maximum log file size (max-size) and the number of retained
log files (max-file) in the logging driver options
to control disk usage and prevent log files from
growing indefinitely.
* Impl generating continuous deserialized blocks
* Make `sapling_efvk_hrp` `pub`
* Don't wait for Sapling activation height in tests
* Set the sleep interval for scan service to 10 secs
* Simplify `sapling_key_to_scan_block_keys`
* Enable mocking Sapling scanning keys for Testnet
* Test the `RegisterKeys` scan service call
* Enable `shielded-scan` for `zebra-chain`
* Use an ephemeral database so results don't persist
* Don't generate blocks when mocking the state
* Improve error messages
* Simplify seeding mocked Sapling viewing keys
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* Use a manual iterator over `Network`
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* processes SubscribeResults messages
* send tx ids of results to the subscribe channel
* replaces BoxError with Report in scan_range
* adds a watch channel for using subscribed_keys in scan_range
* updates args to process_messages in test
* adds a `subscribe` method to ScanTask for sending a SubscribeResults cmd
* updates test for process_messages to cover subscribe cmds
* impls SubscribeResult service request and updates sender type
* adds test for SubscribeResults scan service request
* adds acceptance test
* updates tests and imports
* fixes acceptance test by using spawn_blocking to avoid blocking async executor and setting an appropriate start height
* fixes test
* Applies suggestions from code review.
* use tokio mpsc channel in scan task instead of std/blocking mpsc
* use tokio mpsc channel for results sender
* adds `was_parsed_keys_empty` instead of checking that all the parsed keys are new keys
* fixes test failures related to send errors in scan task
* returns height and key for scan results from subcribe_results results receiver
* hide scan_service mod in zebra-node-service behind feature
* Refactor obtaining of activation heights
* Impl the `RegisterKeys` service request
* Mock viewing keys for tests
* Refactor tests
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove a redundant comment
I don't think we need to assume the genesis block doesn't contain
shielded data as the comment says.
* Avoid using a single-letter variable
* Refactor mocking Sapling scanning keys
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* adds clear_results RPC method for zebra-scan
* adds delete_keys rpc method
* adds docs
* Update zebra-grpc/proto/scanner.proto
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Apply suggestions from code review
* start zebra-scan gRPC server from zebrad start command
* adds a test that the scanner starts with zebrad
* adds a `listen_addr` field to the shielded scan config
* updates test to use a random port and set the listen_addr config field
* fixes test
* Update zebra-scan/src/config.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* fixes panic when trying to open multiple mutable storage instances.
* open db in blocking task
* fixes test
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* moves ScanTask to its own module in service module
* moves `process_messages()` method to scan_task.rs
* uses get_mut() and returns new keys
* updates types and adds scan_until() function
* adds and uses wait_for_height() function
* spawns scan_until_task_handler to poll scan_until_tasks FuturesUnordered
* corrects comment
* adds TODO
* updates expected test log, corrects panic msg, fixes fmt
* moves scan functions to scan_task module
* moves ScanTaskCommand and related method impls to its own module
* moves `ScanTask::mock()` test constructor to its own module
* creates `add_keys` mod and moves `scan_until()` function there
* moves scan task executor to its own module and adds ScanRangeTaskBuilder type
* renames add_keys to scan_range, moves executor to scan_task mod
* adds test for process_messages
* updates scan_height_and_store_results() to skip last_scanned_height check if there is no key height for a key in `parsed_keys`
* updates `default_test_config()`
* adds acceptance test for registering keys in ScanTask
* uses the right feature for the new acceptance test
* Applies suggestions from code review
* feat(docker): allow users to use Zebra + LWD with persistent states
* fix(docs): better documentation and compose execution
* chore(docs): better context box
* chore: do not commit further user changes to the `.env`
This can always be overriden by forcing this pushes if it was intended
* fix(compose): do not add extra volumes and ports
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* handle RemoveKeys command in ScanTask
* implements DeleteKeys request
* uses spawn_blocking and updates comments
* removes spawn_init fn
* adds test for new Storage method
* adds fake_sapling_results helper function
* adds test for scan service DeleteKeys request
* adds TODO for unwrap_or_clone
* checks the keys sent to the scan task for removal
* moves message processing to its own function
* adds a timeout for the scanner task response
* hide mock() methods behind cfg(test) or feature
* adds MAX_REQUEST_KEYS constant
* updates test to insert and delete results for a second key
* Update zebra-scan/src/init.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* test that the expected number of results are in the db
* fix unused import lint
* fixes unused imports
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* adds/impls types in zebra-chain to be used by the zebra-scan RPC server
* Adds methods for parsing and a test
* adds test for key hash encoding
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* refactors viewing key type
* refactors/renames for viewing keys
* fixes doc links
* Apply suggestions from code review
* Apply suggestions from code review
* removes ViewingKeyHash and ViewingKeyWithHash types
* removes `to_bytes` methods
* remove outdated method and call
---------
Co-authored-by: Marek <mail@marek.onl>
* ref(docker): Consolidate all tests in the `entrypoint.sh` script
* fix(ci): missing docker argument
* fix(ci): do not blank `ENTRYPOINT_FEATURES` vairable
* fix(docker): do not add `shielded-scan` as part of `TEST_FEATURES`
* chore: remove misleading comment
* feat(docker): add `docker-compose` file to run CI locally
* fix(docs): do not remove previous sections until we have a substitute for users
* fix(docs): revert some changes
* fix: typos
* ref(docker): Consolidate all tests in the `entrypoint.sh` script
* fix(ci): missing docker argument
* fix(ci): do not blank `ENTRYPOINT_FEATURES` vairable
* fix(docker): do not add `shielded-scan` as part of `TEST_FEATURES`
* chore: remove misleading comment
* Update docker/Dockerfile
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* ref(docker): Consolidate all tests in the `entrypoint.sh` script
* fix(ci): missing docker argument
* fix(ci): do not blank `ENTRYPOINT_FEATURES` vairable
* fix(docker): do not add `shielded-scan` as part of `TEST_FEATURES`
* chore: remove misleading comment
* Update docker/Dockerfile
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* add `zebra-grpc` crate
* add missing fields
* convert to a lib
* add zebra-scan and tonic as depenency
* add a getinfo grpc
* remove zebra-scanner dependency
* Adds scan_service field to scanner grpc server
* remove dependency
* test launching the grpc server from the zebra-scan crate (not building)
* fix async issue
* fixes build issues
* add binary for manual testing
* try fix try run
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
dependabot[bot]
Gustavo Valverde
Marek
bishopcheckmate
Arya
Alfredo Garcia
Pili Guerra
kai