Commit Graph

696 Commits

Author SHA1 Message Date
Kris Nuttycombe a0935b5ecd zcash_client_backend: remove `WalletRead::is_valid_account_extfvk` 2023-10-27 15:01:26 -06:00
Kris Nuttycombe 9627c806e4 zcash_client_backend: Pass `Proposal` values by reference. 2023-10-26 16:11:10 -06:00
Kris Nuttycombe 570ea48588 Apply suggestions from code review
Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-10-26 15:24:45 -06:00
Kris Nuttycombe f1c08693a5 zcash_client_backend: Add propose_standard_transfer. 2023-10-25 08:08:49 -06:00
Kris Nuttycombe cc0cc2de84 zcash_primitives: add StandardFeeRule
`StandardFeeRule` is an enumeration of the standard fees that have
existed in the history of Zcash zips. It is provided to simplify
transition to new fee strategies; legacy elements of this enumeration
are introduced already-deprecated.
2023-10-25 08:08:49 -06:00
Kris Nuttycombe f7527e14c9 Use `NonNegativeAmount` for note and utxo value fields 2023-10-23 12:59:26 -06:00
ebfull 84eb0821de
Merge pull request #1021 from zcash/release-zcash_client_sqlite-0.8.1
Release zcash_client_sqlite 0.8.1
2023-10-19 12:24:46 -06:00
Sean Bowe 289f1fdf30 Release 0.8.1
Signed-off-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-10-19 18:26:11 +01:00
Daira Emma Hopwood 77ceb879e6 Merge remote-tracking branch 'origin/release-zcash_client_sqlite-0.8.1' 2023-10-19 01:16:24 +01:00
Sean Bowe b88adb4e82 Release 0.8.1 2023-10-18 14:58:03 -06:00
Daira Emma Hopwood 914c0ed5c9 Add regression test for incorrect note deduplication in `v_transactions`.
Signed-off-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-10-18 14:54:46 -06:00
Kris Nuttycombe 2873dd4b60 zcash_client_sqlite: Fix incorrect note deduplication in `v_transactions`
The `v_transactions` view is built upon the set of received notes,
received note values being added to the balance for the transaction and
spent notes being deducted from this balance. This fixes an error
wherein if multiple identically-valued notes were spent in a
transaction, only one of those notes' values was being counted as having
been spent.
2023-10-18 14:54:31 -06:00
ebfull f70a4fe29e
Merge pull request #1020 from nuttycom/wallet/v_transactions_note_uniqueness
zcash_client_sqlite: Fix incorrect note deduplication in `v_transactions`
2023-10-18 09:36:42 -06:00
Daira Emma Hopwood 11e909a2d1 Add regression test for incorrect note deduplication in `v_transactions`.
Signed-off-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-10-18 00:41:34 +01:00
Kris Nuttycombe 0de652c687 zcash_client_sqlite: Fix incorrect note deduplication in `v_transactions`
The `v_transactions` view is built upon the set of received notes,
received note values being added to the balance for the transaction and
spent notes being deducted from this balance. This fixes an error
wherein if multiple identically-valued notes were spent in a
transaction, only one of those notes' values was being counted as having
been spent.
2023-10-13 15:28:50 -06:00
Kris Nuttycombe 29c676ff12
Merge pull request #1009 from tw0po1nt/fix_txn_builder_panic
Gracefully handle when given an Orchard-only UA
2023-10-12 18:15:36 -06:00
Matthew Watt a788cc4c4d Fix merge conflicts 2023-10-12 06:16:22 -05:00
Kris Nuttycombe 1447d8ea01 zcash_client_backend: Move change memos into the `ChangeValue` components of `Proposal`s.
The existing API limited change outputs to having only a single memo
repeated across each change output. This change makes it so that each
proposed change output can have its own associated memo, and leaves it
up to the input selector to determine how requested change memos are
associated with change outputs.
2023-10-11 14:15:19 -06:00
Kris Nuttycombe a2b5c2c784 zcash_client_backend: Use `NonNegativeAmount` for fee and change amounts.
In order to use `uint64` for amounts that must be nonnegative in the
`proposal.proto` file, it is useful to update fee and change computation
to use `NonNegativeAmount` where possible.
2023-10-10 16:12:47 -06:00
Matthew Watt 73f263682e Remove unnecessary local 2023-10-10 07:50:07 -05:00
Matthew Watt 2a4d9e06da Run fmt 2023-10-10 07:47:08 -05:00
Matthew Watt b548e00c10
Update zcash_client_sqlite/src/wallet.rs
Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-10-10 06:36:40 -05:00
Matthew Watt 0e5a58b0b0
Update zcash_client_sqlite/src/wallet.rs
Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-10-10 06:36:20 -05:00
Kris Nuttycombe 5b40ddf072 zcash_client_backend: Use globally unique identifiers for notes.
Update zcash_client_backend error types to use (TxId, output_index)
as the identifier for notes instead of the internal database identifier.
2023-10-09 12:44:27 -06:00
sasha 74840829c8 zcash_client_backend: Make `ReceivedSaplingNote` internals private.
This also adds the txid and index to `ReceivedSaplingNote` for use as a
globally identifier for the note.
2023-10-09 12:22:12 -06:00
Matthew Watt c4175342ac Cleanup 2023-10-09 06:46:12 -05:00
Matthew Watt 9d7ac07b1a Fix test build failures 2023-10-08 19:37:02 -05:00
Matthew Watt 46ed4964c3 Display for PoolType 2023-10-08 19:19:51 -05:00
Matthew Watt 94f2240e08 Provide PoolType to UnsupportedPoolType case; stylistic changes 2023-10-08 19:05:50 -05:00
Kris Nuttycombe c16e777222 zcash_client_sqlite: Fix zcash_proofs dev-dependency 2023-09-28 17:39:59 -06:00
Kris Nuttycombe a2e772c93a zcash_client_backend: Return summary information from `scan_cached_blocks`
When scanning, a wallet only needs to update balance and transaction
information shown to users when the scan has resulted in a change to
wallet state. This modifies `scan_cached_blocks` to return the range of
block heights actually scanned, along with the counts of notes spent and
received by the wallet in that range.

Fixes #918
2023-09-28 17:39:59 -06:00
Jack Grigg f6d7714efe zcash_client_sqlite: Replace internal height tuples with `RangeInclusive`
We don't need to iterate over them, but the `*_extrema` internal methods
are semantically returning inclusive ranges, and using `RangeInclusive`
avoids bugs where the wrong half of the tuple is used (instead moving
the location of the tuple handling inside the `*_extrema` methods, which
cuts the number of occurrences from linear in the number of function
calls to constant).
2023-09-27 16:34:58 +00:00
Jack Grigg 7e89300db9 Move common package and dependency configs into workspace config
The MSRV for the main crates is 1.65, which is higher than the Rust
version that stabilised workplace dependencies (1.64). The implicit MSRV
for the component crates is still lower than this, so we don't migrate
these crates.
2023-09-26 22:01:32 +00:00
Jack Grigg 7a47f8494c zcash_client_sqlite 0.8.0 2023-09-25 18:20:44 +00:00
Jack Grigg 4bc65d66b1 zcash_client_backend 0.10.0 2023-09-25 18:19:52 +00:00
Jack Grigg 1a1abb62aa zcash_proofs 0.13.0 2023-09-25 15:11:00 +00:00
Jack Grigg d35d0961ab zcash_primitives 0.13.0 2023-09-25 15:06:57 +00:00
Jack Grigg e926151f8f zcash_client_sqlite: Use `Uuid::from_u128` for all UUIDs
Previously we used `Uuid::from_fields` to ensure that the various UUID
fields were correctly constructed, but now that we have a CI lint that
checks this automatically, we can consistently use `Uuid::from_u128`
which is easier to prepare from `uuidgen` output.
2023-09-25 12:05:42 +00:00
Jack Grigg 478b5d1858 zcash_client_sqlite: Clean up migrations graph comment 2023-09-25 11:14:24 +00:00
Kris Nuttycombe b90f285359 zcash_client_sqlite: Include spent utxos in v_transactions net value.
This fixes a bug in v_transactions whereby shielding transactions
displayed an incorrect balance.
2023-09-24 21:35:42 -06:00
Jack Grigg 30c58ebfb2 zcash_client_sqlite 0.8.0-rc.5 2023-09-22 23:26:43 +01:00
Jack Grigg 4afc24af6a zcash_client_backend 0.10.0-rc.4 2023-09-22 23:25:19 +01:00
Jack Grigg b76b028b3a zcash_client_sqlite: Set chain tip to truncation height when truncating
We don't know at truncation time what the latest chain tip is; the chain
might have reorged to a shorter heavier chain, or the reorg depth might
only be a few blocks. `WalletDb::chain_height` uses the scan queue as
its source of truth, so the `Verify` range we add during truncation
(to prioritise determining whether the rewind was sufficient) can't
extend beyond the block height we know to exist.

The next call to `WalletDb::update_chain_tip` will add additional ranges
beyond this height, which might include a `Verify` range that ends up
merging with the one added during truncation.
2023-09-22 21:09:23 +00:00
Jack Grigg 625a5ff594 zcash_client_sqlite: Remove is-mined checks from transparent balance
The `LEFT OUTER JOIN` was causing the `tx.block IS NULL` check to alias
two cases: an unspent transparent output, and a transparent output spent
in an unmined transaction. The latter only makes sense to include in the
UTXO count if the transaction is expired, and (due to limitations of the
transparent data model in the current wallet) if that expiry won't be
undone by a reorg. We now handle these two cases directly.

Partly reverts 8828276361.
Closes zcash/librustzcash#983.

Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2023-09-22 21:09:23 +00:00
Jack Grigg cd6c962719 zcash_client_sqlite: Write a test for transparent balance behaviour 2023-09-22 17:20:10 +00:00
Jack Grigg 71e38fe190 zcash_client_sqlite: Enable `TestState` to mine wallet transactions 2023-09-22 17:20:07 +00:00
Jack Grigg 9e4fa96dd7 zcash_client_sqlite: Clean up existing transparent input test 2023-09-22 17:01:00 +00:00
str4d e817a75582
Merge pull request #987 from zcash/test-harness-fixes-to-rc-branch
Merge test harness fixes to the release candidate branch
2023-09-22 18:00:22 +01:00
Kris Nuttycombe b69d80f25d zcash_client_sqlite: Remove use of boolean constants from `v_tx_outputs`
At present, [zcash/zcash-android-wallet-sdk] supports Android API 27,
which bundles SQLite 3.19. SQLite support for the `TRUE` and `FALSE`
constants were introduced in SQLite 3.23, so we cannot currently use
these constants and retain support for Android API 27.

This version support limitation applies only to the `v_transactions`
and `v_tx_outputs` views, which are considered part of the public API
of this crate; other use of more recent SQLite features is fine because
they rely upon the SQLite bundled via our use of the `rusqlite` crate
and feature compatibility is verified via the unit tests of this crate.
2023-09-20 10:09:25 -06:00
Kris Nuttycombe e7b3e9c04e zcash_client_sqlite 0.8.0-rc.4 2023-09-19 15:01:52 -06:00
Kris Nuttycombe 6d2c53111a zcash_client_backend 0.10.0-rc.3 2023-09-19 15:00:51 -06:00
Kris Nuttycombe c0e29ad103 zcash_client_sqlite: fix join to blocks table in `v_transactions`
This allows us to display additional block information when a
transparent UTXO was provided for a block that we have scanned.
Also, this ensures that memos from a wallet to itself are not
double-counted.

Co-authored-by: str4d <thestr4d@gmail.com>
2023-09-19 13:15:50 -06:00
Kris Nuttycombe 15bfb41773 Apply suggestions from code review.
Co-Authored-By: str4d <thestr4d@gmail.com>
Co-Authored-By: Daira Hopwood <daira@jacaranda.org>
2023-09-19 13:15:50 -06:00
Kris Nuttycombe 6ebd66da76 zcash_client_sqlite: Remove inner join between `transactions` and `utxos` from `v_transactions`
Under normal usage conditions, the `transactions` table is not currently
populated for transactions involving transparent UTXOs, and so this join
was always resulting in transparent UTXO information being filtered out
from the transaction history.

Fixes [zcash/ZcashLightClientKit#1271]
2023-09-19 13:15:50 -06:00
Kris Nuttycombe 1575f2db88 zcash_client_backend: make the `SpanningTree` type usable outside of `zcash_client_sqlite`
This adds the `data_api::scanning::spanning_tree` module under
a new `unstable-spanning-tree` feature flag, making it available to
other implementations who want to be able to write their own storage
backends without having to reinvent the spanning tree logic.
2023-09-18 11:22:35 -06:00
Kris Nuttycombe b32aa3d2a6 Fix missing zcash_client_backend feature dependency. 2023-09-12 09:35:18 -06:00
Kris Nuttycombe e6b34eeaa2 zcash_client_sqlite 0.8.0-rc.2 2023-09-12 08:32:34 -06:00
Kris Nuttycombe 24068cd63a zcash_client_backend 0.10.0-rc.2 2023-09-12 08:31:39 -06:00
Kris Nuttycombe 1e39daf6f8
Merge pull request #963 from nuttycom/bug/commitment_tree_depth
zcash_client_backend: Fix an error that can occur due to sparse `shardtree` checkpoints.
2023-09-12 08:05:50 -06:00
Kris Nuttycombe f2dcba3a34 Apply suggestions from code review
Co-authored-by: str4d <thestr4d@gmail.com>
2023-09-11 18:39:27 -06:00
Kris Nuttycombe 24e8c82546 zcash_client_backend: Get commitment tree depth for a given number of confirmations from the database.
This fixes the following bug:

Due to complexities related to non-linear scanning, checkpoints are only
added to the wallet's commitment tree in cases where there are notes
discovered within a scanned block. At present, the `shardtree` API only
makes it possible to add multiple checkpoints of the same tree state
when adding checkpoints at the chain tip, and this functionality is not
used by `zcash_client_backend` because we perform checkpoint insertion
in batches that may contain gaps in the case that multiple blocks
contain no Sapling notes. While it would be possible to fix this by
altering the `shardtree` API to permit explicit insertion of multiple
checkpoints of the same tree state at a given note position, this fix
takes a simpler approach.

Instead of ensuring that a checkpoint exists at every block and
computing the required checkpoint depth directly from the minimum number
of confirmations required when attempting a spend, we alter the
`WalletCommitmentTrees` API to allow internal information of the note
commitment tree to be used to determine this checkpoint depth, given the
minimum number of commitments as an argument. This allows us to select a
usable checkpoint from the sparse checkpoint set that resulted from the
sparse insertion of checkpoints described above.
2023-09-11 18:38:54 -06:00
Daira Emma Hopwood 101c9c0ea4 Add passing test for current behaviour of wallet restore from seed.
refs #936

Signed-off-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-09-12 01:33:12 +01:00
Daira Emma Hopwood 401af23484 Improve wallet tests.
Signed-off-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-09-12 01:33:12 +01:00
Kris Nuttycombe 771e4013c3 Move shardtree serialization to the `zcash_client_backend` crate 2023-09-11 18:18:08 -06:00
Kris Nuttycombe 28f1f7d296 zcash_client_sqlite: Add a test for when gaps exist in checkpoints. 2023-09-11 17:08:51 -06:00
Kris Nuttycombe 92237a5558 zcash_client_sqlite: `serialization` module should not have been public. 2023-09-11 16:48:13 -06:00
Jack Grigg 62e3299255 zcash_client_sqlite 0.8.0-rc.1 2023-09-08 19:58:46 +00:00
Jack Grigg 9db68c76ee zcash_client_backend 0.10.0-rc.1 2023-09-08 19:50:53 +00:00
Jack Grigg 87ca71b8db zcash_proofs 0.13.0-rc.1 2023-09-08 19:48:55 +00:00
Jack Grigg 67b84c25e0 zcash_primitives 0.13.0-rc.1 2023-09-08 19:45:54 +00:00
Jack Grigg 94c1f2413d Migrate to `incrementalmerkletree 0.5`, `orchard 0.6` 2023-09-08 18:57:25 +00:00
Kris Nuttycombe 61e06d2227 zcash_client_sqlite: Update CHANGELOG.md for 0.8.0 release 2023-09-08 11:02:37 -06:00
Jack Grigg 6e34e2d113 zcash_client_backend: Migrate to `prost 0.12`, `tonic 0.10` 2023-09-08 16:21:48 +00:00
Kris Nuttycombe aa039819df Fix formatting & clippy lints. 2023-09-08 08:46:20 -06:00
Jack Grigg 504efcfab7 zcash_client_sqlite: Fix scan range matching with incomplete shard
The `v_sapling_shard_scan_ranges` view pairs every scan range with every
shard range, such that each row shows an overlapping pair.

For the complete shards, this is an overlap check between two ranges,
which the previous query was performing correctly (if verbosely).

For the last incomplete shard, we have a half-open range that needs to
be handled separately. The previous query only handled the case where a
scan range was contained within the last shard, and did not handle the
case where the scan range contained the last shard.

This led to a puzzling bug, where `WalletDb::get_wallet_summary` was
sometimes treating any note received within the last shard as part of
the wallet's pending balance. If the wallet's scan queue contained a
range that encompassed the last incomplete shard, the bug in the
`v_sapling_shard_scan_ranges` view meant that it omitted any mention of
the last shard, which translated into these notes being considered
unmined when joining `sapling_received_notes` against the sub-view
`v_sapling_shards_scan_state`.

The bug was made harder to diagnose due to the previous commit's bug
that was causing scan ranges to not be correctly merged; this resulted
in smaller scan ranges that were more likely to be contained within the
last shard, making it visible in `v_sapling_shard_scan_ranges` and
enabling notes to be detected as mined.

The fixed view uses a simpler query that enables us to handle complete
and incomplete shards together.

Time spent investigating and fixing: 4.5 hours

Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2023-09-08 04:04:10 +00:00
Jack Grigg 457c9d26dd zcash_client_sqlite: Prevent empty `FoundNote` ranges in `scan_complete`
In the sandblasting block ranges, shard trees only cover a few hundred
blocks at most. When scanning block ranges much larger than this, it is
likely that when a note is discovered, its parent shard tree is entirely
contained within the scan range. In this situation, `extended_range`
would be set to `range`, and then because an extended range existed,
ranges with `FoundNote` priority would always be created - that in this
case are empty.

In an effectively-linear-scanning wallet situation, this leads to a
`SpanningTree` being constructed with adjacent `Scanned` ranges,
separated by empty `FoundNote` ranges, which it was unable to merge.

We address this by both preventing generation of the empty `FoundNote`
ranges, and by defensively fixing `SpanningTree::into_vec` to skip empty
ranges.

Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2023-09-08 03:57:21 +00:00
Jack Grigg 7a4954c242 zcash_client_sqlite: Fix off-by-one in `scan_complete`
Shard end heights are end-inclusive, while Rust `Range`s are
end-exclusive.

Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2023-09-08 03:44:03 +00:00
Jack Grigg b54969953f zcash_client_sqlite: Simplify range query in `replace_queue_entries`
Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2023-09-08 03:43:07 +00:00
Kris Nuttycombe 341379ee8d
Merge pull request #951 from nuttycom/wallet/balance_for_all_accounts
zcash_client_sqlite: Return a balance for every account from `get_wallet_summary`
2023-09-07 11:49:23 -06:00
Kris Nuttycombe 8dfab23683
Merge pull request #946 from nuttycom/test/unspendable_chain_tip
zcash_client_sqlite: Add a test demonstrating chain tip fund unavailability
2023-09-07 11:27:51 -06:00
Kris Nuttycombe 2131cd547e zcash_client_sqlite: Return a balance for every account from `get_wallet_summary`
Fixes #948
2023-09-07 11:02:59 -06:00
Kris Nuttycombe 0937c6a384 zcash_client_sqlite: Add a test demonstrating chain tip fund unavailability.
This also factors out and cleans up repeated code that checks for anchor
shard completeness.
2023-09-07 10:55:16 -06:00
Kris Nuttycombe 19582a4ca6
Apply suggestions from code review
Co-authored-by: str4d <thestr4d@gmail.com>
2023-09-07 10:29:21 -06:00
Kris Nuttycombe a6328a098f zcash_client_sqlite: Bound creation of scan ranges on wallet birthday.
This modifies `update_chain_tip` and `scan_complete` to ensure that
newly created scan ranged do not extend below the wallet birthday
height.

Fixes #947
2023-09-07 09:58:21 -06:00
Kris Nuttycombe f56c5d8f4b Measure scan progress from wallet birthday, not the fully scanned height. 2023-09-06 12:07:18 -06:00
Jack Grigg c9d58b223c zcash_client_sqlite: Fix bug in `check_shard_discontinuity`
The previous code was checking that the existing shard range was at
least partially inside the proposed insertion range, but this didn't
handle the case where the proposed insertion range was fully contained
by the existing shard range.
2023-09-06 17:13:12 +00:00
Kris Nuttycombe 65420f59df zcash_client_backend: Add `WalletRead::block_max_scanned`
Fixes #941
2023-09-06 08:49:43 -06:00
str4d b4772e948d
Merge pull request #938 from ebfull/check-subtree-discontinuity
Check for potential discontinuities when inserting into the `*_tree_shards` table
2023-09-06 01:05:18 +01:00
str4d 236e9f2cd0
Merge pull request #914 from nuttycom/sbs/nonlinear_sync_aware_balance
Add `get_wallet_summary` to `WalletRead`
2023-09-06 01:04:42 +01:00
str4d 1e12e9d0e6
Clarify zero-conf shielded note behaviour for `get_wallet_summary`
Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-09-06 00:18:11 +01:00
Kris Nuttycombe 6cbb107c71 zcash_client_sqlite: allow zero-conf transactions in unshielded balance. 2023-09-05 16:49:36 -06:00
Kris Nuttycombe 7abd1324de Apply suggestions from code review
Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>
Co-authored-by: Jack Grigg <jack@electriccoin.co>
2023-09-05 16:49:36 -06:00
Kris Nuttycombe f53ea2d778 Add `get_wallet_summary` to `WalletRead`
The intent of this API is to provide a single API which returns in a
single call:

* per-account balances, including pending values
* wallet sync progress

Fixes #865
Fixes #900
2023-09-05 16:06:55 -06:00
Sean Bowe 976b0f0931 Check for potential discontinuities when inserting into the `*_tree_shards` table.
Closes #924
2023-09-05 15:25:26 -06:00
Kris Nuttycombe 6478113855
Merge pull request #935 from nuttycom/sbs/post_907_cleanup
Address unresovled code review comments from #907
2023-09-05 10:42:43 -06:00
Kris Nuttycombe dd60f51d3c Address unresovled code review comments from #907 2023-09-05 09:30:17 -06:00
str4d f132af1f33
Documentation fixes
Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>
2023-09-05 15:54:15 +01:00
Jack Grigg afec2ee218 `zcash_client_sqlite`: Fix refactor bug: ignore pre-Sapling chain tips
The pre-refactor code did this correctly, but the comment was moved
without fully moving the logic.
2023-09-05 14:52:33 +00:00
Jack Grigg 81d7d9349c zcash_client_sqlite: Fix bugs in `update_chain_tip`
Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2023-09-02 03:01:23 +00:00
Jack Grigg 9f9a1fb92f zcash_client_sqlite: Rework `update_chain_tip` tests to pin behaviour 2023-09-02 03:01:19 +00:00