Commit Graph

11 Commits

Author SHA1 Message Date
Arya 80827f5294
add(scan): Implement `DeleteKeys` `ScanService` request (#8217)
* handle RemoveKeys command in ScanTask

* implements DeleteKeys request

* uses spawn_blocking and updates comments

* removes spawn_init fn

* adds test for new Storage method

* adds fake_sapling_results helper function

* adds test for scan service DeleteKeys request

* adds TODO for unwrap_or_clone

* checks the keys sent to the scan task for removal

* moves message processing to its own function

* adds a timeout for the scanner task response

* hide mock() methods behind cfg(test) or feature

* adds MAX_REQUEST_KEYS constant

* updates test to insert and delete results for a second key

* Update zebra-scan/src/init.rs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* test that the expected number of results are in the db

* fix unused import lint

* fixes unused imports

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-01-31 19:34:24 +00:00
Alfredo Garcia 78d33f3e9e
feat(grpc): Add initial `Getinfo` grpc (#8178)
* add `zebra-grpc` crate

* add missing fields

* convert to a lib

* add zebra-scan and tonic as depenency

* add a getinfo grpc

* remove zebra-scanner dependency

* Adds scan_service field to scanner grpc server

* remove dependency

* test launching the grpc server from the zebra-scan crate (not building)

* fix async issue

* fixes build issues

* add binary for manual testing

* try fix try run

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-01-26 01:29:37 +00:00
Arya 513ace2646
add(scan): Create a tower Service in zebra-scan (#8185)
* Adds ScanService and ScanTask

* renames ScannerCommand to ScanTaskCommand

* fixes doc errors

* fixes clippy lints

* panic if the scan task finishes unexpectedly

* updates TODOs

---------

Co-authored-by: Marek <mail@marek.onl>
2024-01-24 22:37:03 +00:00
teor 4b5838c500
Make sure scanner database is accessed using the correct types (#8112)
* impl TryFrom<zcash_primitives::BlockHeight> for Height

* Add type-safe read and write database methods

* Only allow typed access to the scanner DB

* Update docs

* Implement a common method as a trait

* Fix imports

* Tidy state imports

* Activate tracing logging macros in the whole scanner crate

* Fix dead code warnings
2023-12-18 16:33:49 +00:00
teor 3318eaaa22
test(scan): Add raw database format snapshots to the scanner (#8075)
* Make some scanner storage methods more flexible

* Move tests to a submodule and expose test functions and constants

* Make scanner functions clearer and easier to use

* Simplify state snapshot test code

* Add raw data snapshot tests for the scanner

* Add snapshots

* Fix import path

* Fix import conditional compilation

* fix imports

* fix imports 2

* Put read and write db exports together

* Remove confusing IntoDisk/FromDisk impl

* Fix an incorrect unused method that could panic

* Delete a test that is no longer valid

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-12-12 07:45:12 +00:00
teor db05845f98
change(scan): Use the on-disk database for keys and results (#8036)
* Expose IntoDisk and FromDisk in zebra-state

* Implement database serialization for SaplingScanningKey Strings

* Implement serialization for Vec<SaplingScannedResult> (Vec<transaction::Hash>)

* Implement seralization for SaplingScannedDatabaseIndex

* Add an is_empty() method

* Add a read method for a specific index, and document it

* Implement writing scanner results to the database

* Make read name more explicit

* Implement writing scanner keys

* Implement reading sapling keys

* Spawn blocking tasks correctly in async code

* Change storage results methods to use the database

* Update tests that use storage

* Use spawn_blocking() for database methods

* Change the check interval to slightly less than the block interval

* Expose raw database methods with shielded-scan

* fix `scan_task_starts` test

* minor doc change in test

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-11-30 21:27:46 +00:00
teor 8c717c92dd
change(scan): Create a scanner storage database, but don't use it yet (#8031)
* Create an empty storage/db module

* Use ephemeral storage in tests

* Populate storage inside new() method

* Move scanner setup into an init() method

* Pass the network to scanner init

* Create a database but don't actually use it

* Skip shutdown format checks when skipping format upgrades

* Allow the scanner to skip launching format upgrades in production

* Refactor skipping format upgrades so it is consistent

* Allow checking configs for equality

* Restore Network import
2023-11-30 12:59:15 +00:00
Alfredo Garcia 0f24c311a4
feat(zebra-scan): Connect with zebrad (#7989)
* connect zebrad with zebra-scan

* remove unwrap

* use tokio::sleep

* fix the task handler

* Don't panic on an empty state

---------

Co-authored-by: teor <teor@riseup.net>
2023-11-27 12:52:48 +00:00
Alfredo Garcia a22c8d5f42
feat(config): Allow to add keys to be scanned by the zebra-scan crate to config (#7949)
* allow user to add sapling keys to config

* apply code review suggestions

Co-authored-by: teor <teor@riseup.net>

---------

Co-authored-by: teor <teor@riseup.net>
2023-11-16 01:32:22 +00:00
Alfredo Garcia 732ee01443
feat(scanner): Add a very basic RAM database to store keys and scan results (#7942)
* add a basic RAM database for the scanner

* specify a crate version for zebra-chain dependency

* add a type for sapling keys

* rename everything to reflect sapling

* change some storage methods
2023-11-15 23:06:23 +00:00
Alfredo Garcia 86e468f3b0
poc(scanner): get started with the blockchain scanner proof of concept (#7758)
* get started with the blockchain scanner poc

* rustfmt

* fix the tests

* Reads blocks from db

* Adds conversion functions

* scans blocks and counts transactions

* fix bug

* split into 2 tests

* add duplicated dependencies to deny.toml

* upgrade zebra-scanner version

* try removing ecc duplicated dependencies

* try fix deny.toml

* remove unintended paste from deny.toml

* remove duplicated code from the other test

* remove strict version of `zcash_primitives` crate

* change description

* remove feture

* remove tokio features

* change lib doc

* add more documentation

* change expect

* do not use default in compact block creation

* more docs

* add more checks to test

* remove zebra-consensus dependency

* move all deps to dev-deps

* change crate version

* rename crate to zebra-scan

* lock file

* ifix cargo.lock

* remove internal dev dependencies versions

Co-authored-by: teor <teor@riseup.net>

* fix docs url

Co-authored-by: teor <teor@riseup.net>

* fix expect messages

Co-authored-by: teor <teor@riseup.net>

* remove duplicated in deny.toml

Co-authored-by: teor <teor@riseup.net>

* add a comment about moving code to production

---------

Co-authored-by: arya2 <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2023-11-08 22:05:51 +00:00