str4d
b2bb1b8716
Merge pull request #881 from zcash/zcash_client_backend-cache-ivks
...
zcash_client_backend: Precompute IVKs in `scan_cached_blocks`
2023-07-25 17:00:14 +01:00
str4d
20c53fc7cd
Merge pull request #882 from zcash/nullifier-map-index
...
`zcash_client_sqlite`: Add index for foreign key of `nullifier_map` table
2023-07-25 16:59:35 +01:00
Jack Grigg
0f6970aefd
zcash_client_sqlite: Cache statement for `wallet::prune_nullifier_map`
2023-07-25 13:46:44 +00:00
Jack Grigg
e773cd3ed4
zcash_client_sqlite: Add index for foreign key of `nullifier_map` table
...
This greatly improves the performance of `wallet::prune_nullifier_map`
by speeding up `ON DELETE CASCADE`.
2023-07-25 13:45:52 +00:00
Jack Grigg
2d3d5e8fe6
zcash_client_backend: Precompute IVKs in `scan_cached_blocks`
2023-07-25 02:21:15 +00:00
Jack Grigg
b33330f9ca
zcash_client_backend: Accept owned keys in `scan_block_with_runner`
...
This internal helper function can now take owned or borrowed keys. The
public `scan_block` function is left unaltered.
2023-07-25 02:14:18 +00:00
str4d
46ca3202e4
Merge pull request #879 from zcash/877-shardtree-migration-perf
...
zcash_client_sqlite: Only import recent frontiers into `ShardTree`
2023-07-24 00:09:35 +01:00
Jack Grigg
ed06a37b09
zcash_client_sqlite: Mark existing blocks scanned in `shardtree` migration
...
The `shardtree` migration is applied to a database state that was
created via linear scanning, so we have complete wallet information for
those blocks.
2023-07-21 21:01:03 +00:00
Jack Grigg
97a3f818ec
zcash_client_sqlite: Only import recent frontiers into `ShardTree`
...
We only need to load frontiers into the ShardTree that are close enough
to the wallet's known chain tip to fill `PRUNING_DEPTH` checkpoints, so
that ShardTree's witness generation will be able to correctly handle
anchor depths. Loading frontiers further back than this doesn't add any
useful nodes to the ShardTree (as we don't support rollbacks beyond
`PRUNING_DEPTH`, and we won't be finding notes in earlier blocks), and
hurts performance (as frontier importing has a significant Merkle tree
hashing cost).
Closes zcash/librustzcash#877 .
2023-07-21 20:59:24 +00:00
str4d
2ed0747b67
Merge pull request #878 from zcash/876-nullifier-map
...
zcash_client_sqlite: Maintain a nullifier map from out-of-order scanning
2023-07-21 21:57:12 +01:00
Jack Grigg
0f2689b9c3
zcash_client_sqlite: Maintain a nullifier map from out-of-order scanning
...
Closes zcash/librustzcash#876 .
2023-07-21 18:01:52 +00:00
Jack Grigg
4d2abd5c3a
zcash_client_sqlite: Add test showing out-of-order balance problem
2023-07-21 17:53:14 +00:00
Jack Grigg
fab68d217e
zcash_client_sqlite: Fix `WalletDb::block_fully_scanned` implementation
2023-07-21 15:25:39 +00:00
str4d
8e582109ca
Merge pull request #875 from zcash/blockmeta-upsert
...
Use an upsert for `FsBlockDb::write_block_metadata`
2023-07-19 19:00:29 +01:00
Jack Grigg
963496d0ab
Use an upsert for `FsBlockDb::write_block_metadata`
2023-07-19 16:41:01 +00:00
str4d
f4221889fe
Merge pull request #872 from nuttycom/feature/pre_dag_sync-suggest_scan_ranges
...
Implement `suggest_scan_ranges` and updated chain validation.
2023-07-19 17:37:43 +01:00
str4d
c7b308b312
Rename `VALIDATION_DEPTH` constant to `VERIFY_LOOKAHEAD`
...
Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-07-19 16:03:02 +00:00
Sean Bowe
e16aa41117
Add test for join_nonoverlapping.
2023-07-19 14:54:01 +00:00
Jack Grigg
359ff55ce3
Do nothing if `update_chain_tip` is given a tip lower than we can use
...
Previously it would panic due to constructing invalid `ScanRange`s.
2023-07-19 14:54:01 +00:00
Jack Grigg
98ea08e497
Migrate to latest `shardtree` to improve performance
2023-07-19 14:54:01 +00:00
Jack Grigg
a87dca00e2
Documentation updates, fixes, and cleanups
...
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2023-07-19 14:54:01 +00:00
Jack Grigg
2a98f94f05
Add more debug and trace logging
...
The `Debug` impl for `sapling::Node` is updated to output hex-encoded
bytes for readability.
2023-07-18 17:23:57 +00:00
Jack Grigg
510944777c
Transactionally modify the wallet DB in `replace_queue_entries`
...
We don't want to delete old scan range queue entries unless we are
guaranteed to write the updated queue entries.
2023-07-18 17:23:57 +00:00
Jack Grigg
281dbd5524
Ensure that `extended_range` in `scan_complete` is a superset of `range`
...
Previously `extended_range` only covered the extent of the leaves of
all subtrees in which notes were found during a scan. When the scanned
range was large, this was not guaranteed to be contained within the
subtree leaves, causing an assertion failure when an invalid `ScanRange`
was constructed.
2023-07-18 17:23:57 +00:00
Jack Grigg
cb887efa06
Use correct levels for cap nodes in `put_shard_roots`
...
The Merkle hashes used for the note commitment trees are domain
separated by level, so when pretending that the subtree roots are leaves
of the cap tree, we need to adjust for their level not being zero.
Closes zcash/librustzcash#874 .
Co-authored-by: Sean Bowe <ewillbefull@gmail.com>
2023-07-18 17:23:57 +00:00
Jack Grigg
f7163e9dd9
Code simplifications
...
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2023-07-18 17:23:55 +00:00
Jack Grigg
bb920341a6
Fix bug in `RightFirstDisjoint` insertion logic
2023-07-18 17:22:48 +00:00
Jack Grigg
2c0acac1bd
Use explicit arguments for internal `suggest_scan_ranges` helper
...
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2023-07-18 17:22:48 +00:00
Jack Grigg
ed4b6dc9b9
Fix bugs in `ScanRange::truncate_*`
2023-07-18 17:22:48 +00:00
Jack Grigg
e3aeb63e0a
Fix off-by-one bugs with `ScanRange` end bounds
...
Maximum chain heights are end-inclusive, while `ScanRange` is
end-exclusive.
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2023-07-12 16:27:52 +00:00
Kris Nuttycombe
352e1c709a
Add a test for `scan_complete` and `update_chain_tip`
2023-07-07 20:13:45 -06:00
Kris Nuttycombe
6db4355fc4
Fix an error in dominance calculation.
2023-07-07 20:13:45 -06:00
Kris Nuttycombe
6fa0b46d8e
Implement `suggest_scan_ranges` and `update_chain_tip`
...
This implements a priority queue backed by the wallet database for scan
range ordering. The scan queue is updated on each call to `put_blocks`
or to `update_chain_tip`.
2023-07-07 20:13:45 -06:00
str4d
4d5dc28ab1
Merge pull request #871 from zcash/shardtree-bugfixes
...
Bump patched dependencies to bring in shardtree bugfixes
2023-07-07 00:06:33 +01:00
str4d
4cb36d3d9d
Merge pull request #870 from zcash/fix-get_transaction
...
zcash_client_sqlite: Fix `WalletDb::get_transaction` for unmined txs
2023-07-07 00:05:24 +01:00
Jack Grigg
8d792bb7b5
zcash_client_sqlite: Fix `WalletDb::get_transaction` for unmined txs
2023-07-06 22:24:59 +00:00
Jack Grigg
a11545459b
Bump patched dependencies to bring in shardtree bugfixes
2023-07-06 22:19:21 +00:00
str4d
c75ff850d7
Merge pull request #869 from nuttycom/feature/pre_dag_sync-put_blocks
...
Replace `put_block` with `put_blocks`
2023-07-06 22:52:15 +01:00
Kris Nuttycombe
d55fa09464
Add a check to ensure that blocks passed to `put_blocks` are sequential.
2023-07-06 12:13:39 -06:00
Kris Nuttycombe
1b5a24a655
Batch data store writes of `put_block`
...
Instead of calling `put_block` for each block scanned,
`scan_cached_blocks` will now defer the block writes until the scan of a
batch is complete and will perform the block writes and note commitment
tree updates all within a single transaction.
This should ordinarily be fine in terms of memory consumption, because
the block data being saved is pruned to only that spend an output
information that is related to transactions in the wallet, which will
normally be sparse enough that the block range size that is appropriate
for a given platform to run within a batch scanner adequately bounds the
memory consumption of this pruned representation.
2023-07-06 12:13:39 -06:00
str4d
82705a4ae4
Merge pull request #867 from nuttycom/feature/pre_dag_sync-require_scan_range
...
zcash_client_backend: Make scan range bounds required in `data_api::chain::scan_cached_blocks`
2023-07-06 18:59:17 +01:00
Kris Nuttycombe
81a32f284e
Apply suggestions from code review
...
Co-authored-by: str4d <thestr4d@gmail.com>
2023-07-06 11:24:16 -06:00
str4d
bc17102557
Merge pull request #861 from nuttycom/feature/fast_spendability
...
Add interfaces & support for subtree root insertion.
2023-07-06 12:07:29 +01:00
Kris Nuttycombe
98147e0d7d
Make `from_height` and `limit` parameters to `scan_cached_blocks` non-optional.
2023-07-05 18:56:19 -06:00
Kris Nuttycombe
faccf56f04
Split `PoolType` enum into `PoolType` and `ShieldedProtocol`
...
There are cases where we wish to return informaiton that is relevant to
a specific shielded protocol and `Transparent` is an invalid case. This
is a minor preparatory refactoring that makes this distinction
expressible.
2023-07-05 18:20:43 -06:00
Kris Nuttycombe
1e5b23aeba
zcash_client_backend: Add `put_sapling_subtree_roots` to `WalletCommitmentTrees`
...
Also add the `zcash_client_sqlite` implementation & tests for the new
method.
2023-07-05 18:18:20 -06:00
Kris Nuttycombe
d8148f90e7
Merge pull request #831 from nuttycom/feature/pre_dag_sync
...
Migrations & data storage for pre-DAG-sync
2023-07-04 13:52:10 -06:00
Kris Nuttycombe
c13c8c6678
Address comments from code review.
2023-07-03 17:08:01 -06:00
Kris Nuttycombe
c363e71fa9
Rename proto::compact::{BlockMetadata => ChainMetadata}
2023-07-03 16:19:13 -06:00
Kris Nuttycombe
42ed6ba2a1
Rename `zcash_client_backend::welding_rig` to `zcash_client_backend::scanning`
2023-07-03 10:49:03 -06:00