Public methods for mutation of these fields have been provided that
perform checking for overflow of the valid monetary range as part
of their operation.
Prior to this change, it's necessary to implement the entirety of the
`WalletRead` trait in order to be able to use the input selection
functionality provided by `zcash_client_backend::data_api::input_selection`.
This change factors out the minimal operations required for transaction
proposal construction to better reflect the principle of least authority
and make the input selection code reusable in more contexts.
In order to minimize the operations of the newly-created `InputSource`
and `ShieldingSource` traits, this change also removes the
`min_confirmations` field from transaction proposals, in favor of
storing explicit target and anchor heights. This has the effect of
limiting the lifetime of transaction proposals to `PRUNING_DEPTH -
min_confirmations` blocks.
The existing API limited change outputs to having only a single memo
repeated across each change output. This change makes it so that each
proposed change output can have its own associated memo, and leaves it
up to the input selector to determine how requested change memos are
associated with change outputs.
In order to use `uint64` for amounts that must be nonnegative in the
`proposal.proto` file, it is useful to update fee and change computation
to use `NonNegativeAmount` where possible.
We don't need to iterate over them, but the `*_extrema` internal methods
are semantically returning inclusive ranges, and using `RangeInclusive`
avoids bugs where the wrong half of the tuple is used (instead moving
the location of the tuple handling inside the `*_extrema` methods, which
cuts the number of occurrences from linear in the number of function
calls to constant).
We don't know at truncation time what the latest chain tip is; the chain
might have reorged to a shorter heavier chain, or the reorg depth might
only be a few blocks. `WalletDb::chain_height` uses the scan queue as
its source of truth, so the `Verify` range we add during truncation
(to prioritise determining whether the rewind was sufficient) can't
extend beyond the block height we know to exist.
The next call to `WalletDb::update_chain_tip` will add additional ranges
beyond this height, which might include a `Verify` range that ends up
merging with the one added during truncation.
The `LEFT OUTER JOIN` was causing the `tx.block IS NULL` check to alias
two cases: an unspent transparent output, and a transparent output spent
in an unmined transaction. The latter only makes sense to include in the
UTXO count if the transaction is expired, and (due to limitations of the
transparent data model in the current wallet) if that expiry won't be
undone by a reorg. We now handle these two cases directly.
Partly reverts 8828276361.
Closeszcash/librustzcash#983.
Co-authored-by: Kris Nuttycombe <kris@nutty.land>
The intent of this API is to provide a single API which returns in a
single call:
* per-account balances, including pending values
* wallet sync progress
Fixes#865Fixes#900
This also removes the zcash_client_sqlite-specific database
initialization procedures in favor of a standardized approach using the
methods available via the data access API.
When `force_rescans` is set to `true` in a call to
`replace_queue_entries`, previously scanned ranges will have their
existing priority overwritten by the scan priority for a provided range;
otherwise, the existing scan priority dominance rule continues to be
enforced. This enables us to require previously scanned ranges be
re-scanned without interfering with higher-priority scan operations.
Prior to the scan-before-sync changes, the wallet was able to assume
that the maximum scanned block height at the time of the spend was
within a few blocks of the chain tip. However, under linear scanning
after the spend-before-sync changes this invariant no longer holds,
resulting in a situation where in linear sync conditions the wallet
could attempt to create transactions with already-past expiry heights.
This change separates the notion of "chain tip" from "max scanned
height", relying upon the `scan_queue` table to maintain the wallet's
view of the consensus chain height and using information from the
`blocks` table only in situations where the latest and/or earliest
scanned height is required.
As part of this change, the `WalletRead` interface is also modified to
disambiguate these concepts.
In general, it is preferable to use globally relevant identifiers where
possible. This PR removes the `WalletRead::TxRef` associated type in
favor of using `TxId` directly for the transaction identifier, and
restricts the use of the `NoteRef` type to those scenarios where the
result of one query is intended to be used directly as the input to
another query.
Closes#834
The `add_checkpoint` method is intended to be idempotent. In the case
that we add a checkpoint at an already-checkpointed block height, we
should only raise an error in the case that the note commitment tree
position or the set of notes spent in the checkpointed block has
changed.
This implements a priority queue backed by the wallet database for scan
range ordering. The scan queue is updated on each call to `put_blocks`
or to `update_chain_tip`.
Instead of calling `put_block` for each block scanned,
`scan_cached_blocks` will now defer the block writes until the scan of a
batch is complete and will perform the block writes and note commitment
tree updates all within a single transaction.
This should ordinarily be fine in terms of memory consumption, because
the block data being saved is pruned to only that spend an output
information that is related to transactions in the wallet, which will
normally be sparse enough that the block range size that is appropriate
for a given platform to run within a batch scanner adequately bounds the
memory consumption of this pruned representation.
There are cases where we wish to return informaiton that is relevant to
a specific shielded protocol and `Transparent` is an invalid case. This
is a minor preparatory refactoring that makes this distinction
expressible.
In preparation for out-of-order range-based scanning, it is necessary
to ensure that the size of the Sapling note commitment tree is carried
along through the scan process and that stored blocks are always
persisted with the updated note commitment tree size.
`rusqlite` includes a mechanism for creating prepared statements that
automatically caches them and reuses the caches when possible. This
means that it's unnecessary for us to do our own caching, and also
offers a minor performance improvement in that we don't need to eagerly
prepare statements that we may not execute in the lifetime of a given
`WalletDb` object. It also improves code locality, because the prepared
statements are now adjacent in the code to the parameter assignment
blocks that correspond to those statements.
This also updates a number of `put_x` methods to use sqlite upsert
functionality via the `ON CONFLICT` clause, instead of having to perform
separate inserts and updates.
Memos may be absent for both sent and received notes in cases where only
compact block information has been used to populate the wallet database.
This fixes a potential crash in the case that we attempt to decode a
SQLite `NULL` as a byte array. It does, however, introduce a slight
semantic confusion that will need to be considered in the case of future
updates where a note may not have an associated memo; at present, the
only reason we might not have the memo is that we might not have
retrieved the full transaction information from the chain, but in the
future there might be other possible reasons for this absence.
Fixes#384
This is in preparation for extraction into the `incrementalmerkletree`
crate, which is not Sapling-specific and therefore cannot hard-code
the depths of these data structures.
This better reflects the semantics of wallet behavior. Also, this
adds a `zcash_client_backend::WalletRead::get_min_unspent_height`
method that replaces the deprecated & removed (and misleadingly
named) `get_rewind_height` method.
This change also settles on `account_value_delta` as the name of the
column in `v_transactions` that describes the transaction's effect on
the value of the associated account.
The remaining uses of `assert!(matches!(...))` are all in cases where
for some reason the `assert_matches` macro interferes with correct
type inference.
The change selection algorithm has the most useful information for
determining whether or not a note is dust, so this adds a new error case
to `ChangeError` that allows the change selection to report the presence
of input notes without economic value back to its caller.
This adds a set of abstractions that allow wallets to provide
independent strategies for fee estimation and note selection, and
implementations of these strategies that perform these operations in the
same fashion as the existing `spend` and `shield_transparent_funds`
functions.
This required a somewhat hefty rework of the error handling in
zcash_client_backend. It fixes an issue with the error types whereby
callees needed to have a bit too much information about the error
types produced by their callers.
Reflect the updated note selection and error handling in zcash_client_sqlite.
Previously, `shield_transparent_funds` was only shielding funds
associated with the legacy default transparent address. This meant
that transparent funds sent to unified addresses could not reliably
be shielded, as a unified address will frequently be constructed
using a diversifier index greater than zero.
This modifies the `get_transparent_receivers` method to return address
metadata containing the account ID and diversifier index used to derive
each address along with the receiver.
The legacy transparent address is never added to the `addresses` table,
but we still need to be able to receive UTXOs sent to that address. So,
we add a special case for when a UTXO matches that legacy address, and
set the account ID to 0 manually.
Due to how the wallets retrieved unspent transparent outputs from the
light wallet server, the account associated with a particular UTXO may
not be known by the light wallet. Instead of requiring the caller to
perform a separate lookup and match the address of the received UTXO
with a known account, it's simpler to perform this lookup internally at
the time of insertion or update.
In order to make this operation more efficient, the `addresses_table`
migration is modified to add a column to cache the transparent receiver
so that it may be used in the joins in the UTXO insert and update
operations.
The currently deprecated implementations of `insert_sent_utxo`,
`insert_sent_note`, `put_sent_utxo` and `put_sent_note` all store to the
same `sent_notes` table internally. Since there's no immediate plan to
change this arrangement, it's better to have a single pair of internal
`insert_sent_output` and `put_sent_output` methods instead.
This updates the data access API to provide diversified address
functionality. In order to support this change, the addresses table
is updated to store diversifier index information in big-endian order
to allow sorting by diversifier index, and account initialization
is updated to store the diversifier index accordingly. The currently
unreleased `addresses_table` migration is updated to reflect this
change.
This replaces the current wallet initialization code with a migration
that brings the database up to the state produced by release 0.3.0.
A subsequent commit will add migrations that correctly produce the
database state as of zcash/librustzcash@602270cb1f.
Fixes#369
Per ZIP 316, the Sapling FVK Encoding only includes `(ak, nk, ovk, dk)`
which is a subset of the Sapling `ExtendedFullViewingKey`. We therefore
need to use `DiversifiableFullViewingKey` inside `UnifiedFullViewingKey`
in order to make it parseable from the UFVK string encoding.
`zcash_client_sqlite::wallet::get_extended_full_viewing_keys` has been
removed as a consequence of this change: we can no longer reconstruct
the correct `ExtendedFullViewingKey` from the `UnifiedFullViewingKey`.
The account number is not stored in the ZIP 316 UFVK encoding, and in
general won't necessarily be known (e.g. if a UFVK is being imported
into a wallet).
`zcash_client_sqlite::wallet::init::init_accounts_table` reverts to its
previous behaviour of requiring the provided `&[UnifiedFullViewingKey]`
to be indexed by account number.
This is a breaking change to the database format. We don't have support
for migrations yet, so existing wallets won't work after this commit
until zcash/librustzcash#489 is done.
Kris Nuttycombe
Jack Grigg
Matthew Watt
Matthew Watt
str4d
Kris Nuttycombe
Francisco Gindre