In order to support constructing the anchor for multiple pools with a
common anchor height, we must be able to checkpoint each note commitment
tree (and consequently compute the root) at that height. Since we may
not have the information in the tree needed to do so, we require that it
be provided.
As a bonus, this change makes it possible to improve the UX around
spendability, because we will no longer require subtree ranges below
received notes to be fully scanned; the inserted frontier provides
sufficient information to make them spendable.
Neither of these should have been feature flags, as they gate breaking
changes to the Zcash consensus rules (and in some ways are incompatible
with each other), while feature flags should be additive.
This enables us to activate Sapling and NU5 at the same height, to
simplify reuse of test logic between Sapling and Orchard.
As a side-effect, `zcash_extensions` is removed from the workspace
because it enables the `zfuture` feature flag unconditionally, which
breaks workspace-level builds because this causes the `zfuture` feature
flag on `zcash_protocol` to be enabled without the corresponding feature
flag on `zcash_client_sqlite` being enabled. We will fix this by moving
from feature flags to config flags for unstable features.
This inverts the dependency relationship between `zcash_protocol` and
`zcash_address`, permitting the network constants (primarily the HRPs)
defined in `zcash_protocol` to be used directly in `zcash_address`
instead of being duplicated.
This adds a mechanism that allows a caller to verify that a given seed
generates the viewing key that is stored in the wallet for a specified
account.
Fixes#1189
In the process of making the internals of `scan_block_with_runner`
reusable across Sapling and Orchard, it became evident that key
identifier abstraction along the lines of #1175 is needed more
generally. This commit refactors the use of ZIP 32 account identifiers
and key scopes to better separate scanning concerns from ZIP 32 key
derivation. In the process, this removes a fair amount of unnecessary
polymorphism from `zcash_client_backend::wallet::WalletTx` and related
types.
This PR was extracted from https://github.com/zcash/librustzcash/pull/1175
in order to make the changes to `zcash_client_backend` usable without
the additional generalizations to `zcash_client_sqlite` made by that PR.
Co-authored-by: Andrew Arnott <andrewarnott@live.com>
In the event that the pool to which change should be sent cannot
automatically be determined based upon the inputs and outputs of a
transaction, it is up to the caller to specify where change should
be sent.
It needn't return the account id that was given as an input, and it shouldn't return an 11-byte diversifier index when a 31-bit child index is more appropriate.
This default only made sense in the context of what was supported by
`zcash_client_sqlite`, and not in any other context. Unified address
requests no longer have their parts conditioned by what feature flags
are available; instead, if a request is constructed for which the
required key parts are not supported under a particular selection of
feature flags, address generation will raise a runtime error.
This unification allows us to better isolate transparent-input dependent
functionality. Previously, `TransparentInputSource` could not be placed
under a feature flag because the interface was needed for proposal
deserialization; by instead making the transparent input source methods
flag-dependent, we can correctly produce an error when proposal
deserializaiton attempts to select transparent inputs and the feature
flag is off.
We plan to also introduce a similar flag to gate access to Sapling
functionality. Since introduction of Orchard functionality is still
nascent, it's the correct time to introduce this isolation, before
there's more functionality that needs to be isolated in this fashion.
`ReceivedNote` now allows Orchard notes to be represented as received
notes. As part of this change, received notes now track whether they
were received using internally- or externally-scoped viewing keys.
This eliminates the need to trial-regenerate notes using the wallet's
IVKs to determine scope at spend time.
Public methods for mutation of these fields have been provided that
perform checking for overflow of the valid monetary range as part
of their operation.
Prior to this change, it's necessary to implement the entirety of the
`WalletRead` trait in order to be able to use the input selection
functionality provided by `zcash_client_backend::data_api::input_selection`.
This change factors out the minimal operations required for transaction
proposal construction to better reflect the principle of least authority
and make the input selection code reusable in more contexts.
In order to minimize the operations of the newly-created `InputSource`
and `ShieldingSource` traits, this change also removes the
`min_confirmations` field from transaction proposals, in favor of
storing explicit target and anchor heights. This has the effect of
limiting the lifetime of transaction proposals to `PRUNING_DEPTH -
min_confirmations` blocks.
`fetch_params.sh` is now deprecated and the bundled proving parameters
from `wagyu-zcash-parameters` are used everywhere, so the tests should
follow suit.
Fixes#1016
This fixes an API issue whereby it was possible to execute a `Proposal`
with a different value of `min_confirmations` than that with which the
`Proposal` was constructed.
`StandardFeeRule` is an enumeration of the standard fees that have
existed in the history of Zcash zips. It is provided to simplify
transition to new fee strategies; legacy elements of this enumeration
are introduced already-deprecated.
The `v_transactions` view is built upon the set of received notes,
received note values being added to the balance for the transaction and
spent notes being deducted from this balance. This fixes an error
wherein if multiple identically-valued notes were spent in a
transaction, only one of those notes' values was being counted as having
been spent.
The existing API limited change outputs to having only a single memo
repeated across each change output. This change makes it so that each
proposed change output can have its own associated memo, and leaves it
up to the input selector to determine how requested change memos are
associated with change outputs.
In order to use `uint64` for amounts that must be nonnegative in the
`proposal.proto` file, it is useful to update fee and change computation
to use `NonNegativeAmount` where possible.
When scanning, a wallet only needs to update balance and transaction
information shown to users when the scan has resulted in a change to
wallet state. This modifies `scan_cached_blocks` to return the range of
block heights actually scanned, along with the counts of notes spent and
received by the wallet in that range.
Fixes#918
We don't need to iterate over them, but the `*_extrema` internal methods
are semantically returning inclusive ranges, and using `RangeInclusive`
avoids bugs where the wrong half of the tuple is used (instead moving
the location of the tuple handling inside the `*_extrema` methods, which
cuts the number of occurrences from linear in the number of function
calls to constant).
Previously we used `Uuid::from_fields` to ensure that the various UUID
fields were correctly constructed, but now that we have a CI lint that
checks this automatically, we can consistently use `Uuid::from_u128`
which is easier to prepare from `uuidgen` output.
We don't know at truncation time what the latest chain tip is; the chain
might have reorged to a shorter heavier chain, or the reorg depth might
only be a few blocks. `WalletDb::chain_height` uses the scan queue as
its source of truth, so the `Verify` range we add during truncation
(to prioritise determining whether the rewind was sufficient) can't
extend beyond the block height we know to exist.
The next call to `WalletDb::update_chain_tip` will add additional ranges
beyond this height, which might include a `Verify` range that ends up
merging with the one added during truncation.
The `LEFT OUTER JOIN` was causing the `tx.block IS NULL` check to alias
two cases: an unspent transparent output, and a transparent output spent
in an unmined transaction. The latter only makes sense to include in the
UTXO count if the transaction is expired, and (due to limitations of the
transparent data model in the current wallet) if that expiry won't be
undone by a reorg. We now handle these two cases directly.
Partly reverts 8828276361.
Closeszcash/librustzcash#983.
Co-authored-by: Kris Nuttycombe <kris@nutty.land>
At present, [zcash/zcash-android-wallet-sdk] supports Android API 27,
which bundles SQLite 3.19. SQLite support for the `TRUE` and `FALSE`
constants were introduced in SQLite 3.23, so we cannot currently use
these constants and retain support for Android API 27.
This version support limitation applies only to the `v_transactions`
and `v_tx_outputs` views, which are considered part of the public API
of this crate; other use of more recent SQLite features is fine because
they rely upon the SQLite bundled via our use of the `rusqlite` crate
and feature compatibility is verified via the unit tests of this crate.
This allows us to display additional block information when a
transparent UTXO was provided for a block that we have scanned.
Also, this ensures that memos from a wallet to itself are not
double-counted.
Co-authored-by: str4d <thestr4d@gmail.com>
Under normal usage conditions, the `transactions` table is not currently
populated for transactions involving transparent UTXOs, and so this join
was always resulting in transparent UTXO information being filtered out
from the transaction history.
Fixes [zcash/ZcashLightClientKit#1271]
This adds the `data_api::scanning::spanning_tree` module under
a new `unstable-spanning-tree` feature flag, making it available to
other implementations who want to be able to write their own storage
backends without having to reinvent the spanning tree logic.
This fixes the following bug:
Due to complexities related to non-linear scanning, checkpoints are only
added to the wallet's commitment tree in cases where there are notes
discovered within a scanned block. At present, the `shardtree` API only
makes it possible to add multiple checkpoints of the same tree state
when adding checkpoints at the chain tip, and this functionality is not
used by `zcash_client_backend` because we perform checkpoint insertion
in batches that may contain gaps in the case that multiple blocks
contain no Sapling notes. While it would be possible to fix this by
altering the `shardtree` API to permit explicit insertion of multiple
checkpoints of the same tree state at a given note position, this fix
takes a simpler approach.
Instead of ensuring that a checkpoint exists at every block and
computing the required checkpoint depth directly from the minimum number
of confirmations required when attempting a spend, we alter the
`WalletCommitmentTrees` API to allow internal information of the note
commitment tree to be used to determine this checkpoint depth, given the
minimum number of commitments as an argument. This allows us to select a
usable checkpoint from the sparse checkpoint set that resulted from the
sparse insertion of checkpoints described above.
The `v_sapling_shard_scan_ranges` view pairs every scan range with every
shard range, such that each row shows an overlapping pair.
For the complete shards, this is an overlap check between two ranges,
which the previous query was performing correctly (if verbosely).
For the last incomplete shard, we have a half-open range that needs to
be handled separately. The previous query only handled the case where a
scan range was contained within the last shard, and did not handle the
case where the scan range contained the last shard.
This led to a puzzling bug, where `WalletDb::get_wallet_summary` was
sometimes treating any note received within the last shard as part of
the wallet's pending balance. If the wallet's scan queue contained a
range that encompassed the last incomplete shard, the bug in the
`v_sapling_shard_scan_ranges` view meant that it omitted any mention of
the last shard, which translated into these notes being considered
unmined when joining `sapling_received_notes` against the sub-view
`v_sapling_shards_scan_state`.
The bug was made harder to diagnose due to the previous commit's bug
that was causing scan ranges to not be correctly merged; this resulted
in smaller scan ranges that were more likely to be contained within the
last shard, making it visible in `v_sapling_shard_scan_ranges` and
enabling notes to be detected as mined.
The fixed view uses a simpler query that enables us to handle complete
and incomplete shards together.
Time spent investigating and fixing: 4.5 hours
Co-authored-by: Kris Nuttycombe <kris@nutty.land>
In the sandblasting block ranges, shard trees only cover a few hundred
blocks at most. When scanning block ranges much larger than this, it is
likely that when a note is discovered, its parent shard tree is entirely
contained within the scan range. In this situation, `extended_range`
would be set to `range`, and then because an extended range existed,
ranges with `FoundNote` priority would always be created - that in this
case are empty.
In an effectively-linear-scanning wallet situation, this leads to a
`SpanningTree` being constructed with adjacent `Scanned` ranges,
separated by empty `FoundNote` ranges, which it was unable to merge.
We address this by both preventing generation of the empty `FoundNote`
ranges, and by defensively fixing `SpanningTree::into_vec` to skip empty
ranges.
Co-authored-by: Kris Nuttycombe <kris@nutty.land>
This modifies `update_chain_tip` and `scan_complete` to ensure that
newly created scan ranged do not extend below the wallet birthday
height.
Fixes#947
The previous code was checking that the existing shard range was at
least partially inside the proposed insertion range, but this didn't
handle the case where the proposed insertion range was fully contained
by the existing shard range.
The intent of this API is to provide a single API which returns in a
single call:
* per-account balances, including pending values
* wallet sync progress
Fixes#865Fixes#900
Kris Nuttycombe
Andrew Arnott
str4d
Kris Nuttycombe
Dimitris Apostolou
ebfull
Daira Emma Hopwood
Matthew Watt
Matthew Watt
sasha
str4d