Commit Graph

7 Commits

Author SHA1 Message Date
teor 4306a00f3c
Scan blocks with sapling keys and write the results to the database (#8040)
* Fix availability of tokio::time in scanner

* Create a function that parses a key into a list of keys

* Pass a ChainTipChange to the scanner function

* Convert a scanned block to a sapling result

* Make it easier to pass keys and blocks

* Increase scanner wait times

* Parse keys once at the start of the scan

* Get a block from the state instead of the tip

* Don't log secret keys, only log every 100,000 blocks

* Scan each block and add the results to storage

* Move blocking tasks into spawn_blocking()

* Update the acceptance test

* Use a dummy sapling tree size

* Use a larger dummy size
2023-12-03 21:58:48 +00:00
teor db05845f98
change(scan): Use the on-disk database for keys and results (#8036)
* Expose IntoDisk and FromDisk in zebra-state

* Implement database serialization for SaplingScanningKey Strings

* Implement serialization for Vec<SaplingScannedResult> (Vec<transaction::Hash>)

* Implement seralization for SaplingScannedDatabaseIndex

* Add an is_empty() method

* Add a read method for a specific index, and document it

* Implement writing scanner results to the database

* Make read name more explicit

* Implement writing scanner keys

* Implement reading sapling keys

* Spawn blocking tasks correctly in async code

* Change storage results methods to use the database

* Update tests that use storage

* Use spawn_blocking() for database methods

* Change the check interval to slightly less than the block interval

* Expose raw database methods with shielded-scan

* fix `scan_task_starts` test

* minor doc change in test

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-11-30 21:27:46 +00:00
teor 8c717c92dd
change(scan): Create a scanner storage database, but don't use it yet (#8031)
* Create an empty storage/db module

* Use ephemeral storage in tests

* Populate storage inside new() method

* Move scanner setup into an init() method

* Pass the network to scanner init

* Create a database but don't actually use it

* Skip shutdown format checks when skipping format upgrades

* Allow the scanner to skip launching format upgrades in production

* Refactor skipping format upgrades so it is consistent

* Allow checking configs for equality

* Restore Network import
2023-11-30 12:59:15 +00:00
Alfredo Garcia 1708f9d946
change(zebra-scan): Scan only one key per backend call (#8034)
* scan one key per backend call

* fix docs
2023-11-30 00:51:32 +00:00
teor cb9452c5e3
change(ci): When building crates individually, build all targets, and run clippy (#8024)
* Run clippy and build all targets on all crates individually

* Fix prod and test features for scanner deps

* Standardise dependency order

* Remove unnecessary async in tests

* Fix an unused import in a test

* Work around a no space left on device error

* Actually just use a larger runner
2023-11-30 00:51:20 +00:00
Marek e00a762856
Create a `scan_block` function to use across scanning tasks (#7994)
* Wrap `zcash_client_backend::scanning::scan_block`

* Use the new `scan_block` fn

* Use full path for `zcash::primitives::Network`

* Add docs for `scan_block`

* Impl `From` for networks in `zcash_primitives`

* Update zebra-scan/src/tests.rs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Move code from `tests.rs` to `scan.rs`

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-11-28 21:58:18 +00:00
Alfredo Garcia 0f24c311a4
feat(zebra-scan): Connect with zebrad (#7989)
* connect zebrad with zebra-scan

* remove unwrap

* use tokio::sleep

* fix the task handler

* Don't panic on an empty state

---------

Co-authored-by: teor <teor@riseup.net>
2023-11-27 12:52:48 +00:00