Commit Graph

426 Commits

Author SHA1 Message Date
Kris Nuttycombe fd1640c242 Update CHANGELOGs 2022-10-11 13:08:15 -06:00
Kris Nuttycombe 3c837381db Disallow invalid pool/address combinations with `Recipient`. 2022-10-11 12:52:44 -06:00
Kris Nuttycombe 56b2edd498 Simplify sqlite backend storage for sent notes & utxos.
The currently deprecated implementations of `insert_sent_utxo`,
`insert_sent_note`, `put_sent_utxo` and `put_sent_note` all store to the
same `sent_notes` table internally. Since there's no immediate plan to
change this arrangement, it's better to have a single pair of internal
`insert_sent_output` and `put_sent_output` methods instead.
2022-10-11 08:57:44 -06:00
Kris Nuttycombe ee9869bbeb Use declarative style for `decrypt_transaction` 2022-10-11 08:56:26 -06:00
Kris Nuttycombe 06e43a572a Track inputs sent to wallet-internal recipients.
Ensure that we're attempting trial-decryption with the internal IVK
and correctly track internal vs. external recipients in the wallet
database.
2022-10-11 08:56:22 -06:00
str4d 1dc3cfe724
Merge pull request #657 from nuttycom/wallet/upsert_utxos
Use upsert functionality for transparent UTXOs, rather than delete/repopulate.
2022-10-11 04:17:49 +01:00
Kris Nuttycombe bd8472535c Add a test for unified address derivation. 2022-10-07 20:37:39 -06:00
Kris Nuttycombe 34a7abd653 Update unified address test vectors
This updates unified address test vectors after
https://github.com/zcash-hackworks/zcash-test-vectors/pull/89
to check addresses across multiple diversifier indices, and also
removes the superfluous binary encoding of UA strings.
2022-10-07 19:50:32 -06:00
Kris Nuttycombe b6e263d259
Apply suggestions from code review
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2022-10-05 20:33:14 -06:00
Kris Nuttycombe c5b9219f00 Look up the account by the UFVK corresponding to the USK when spending. 2022-10-05 14:35:51 -06:00
Kris Nuttycombe b7cc038050 Use unified spending keys for spends & shielding. 2022-10-05 14:35:51 -06:00
Kris Nuttycombe 6df6fec860 Add utxo received_by_account data to WalletTransparentOutput
Also, this removes wallet::delete_utxos_above.
2022-10-04 13:34:52 -06:00
Kris Nuttycombe 60785a1d7a Fix Sapling key organization.
This change moves Sapling-specific key generation into a
submodule of `zip32`. Public reexports are used to keep the
existing API consistent.
2022-10-03 20:28:04 -06:00
Jack Grigg b3f13020ac zcash_client_backend: Remove direct ff dependency 2022-10-02 23:15:51 +01:00
Jack Grigg 035e53990c zcash_client_backend: Improve estimation of `rayon` spawned task size 2022-09-24 21:01:19 +00:00
Jack Grigg 913aa0a988 zcash_client_backend: Correctly track heap usage of batch items
As of zcash/librustzcash#633, `SaplingDomain::IncomingViewingKey` now
allocates memory internally, and this memory persists as long as the
`BatchRunner` is alive. Now that we have decoupled the measurement of
heap usage for batch tasks from their internals, we can add bounds to
all of the generic parameters of `Batch` to enable correctly measuring
their actual heap usage.

We also add `DynamicUsage` impls for a bunch of `zcash_primitives` types
that will be used with `BatchRunner` (or its equivalent implementation
in `zcashd`) by callers.
2022-09-24 21:01:19 +00:00
Jack Grigg c98f04330d zcash_client_backend: Move heap tracking of batch tasks behind a trait
This enables the heap usage measurements to be conditionally enabled by
the `BatchRunner` user. Importantly, when heap usage measurements are
not enabled, the `DynamicUsage` bound on `Batch` is not required.

This refactor also fixes a bug in the prior implementation. We were
counting the heap usage of a task when it started to run, but the item
may have been in the `rayon` work-stealing queues for a non-negligible
period before then. We now count the heap usage immediately before
spawning the task into the `rayon` thread pool.
2022-09-24 17:20:55 +00:00
Jack Grigg fe258ca120 zcash_client_backend: Count `tags` in `impl DynamicUsage for Batch` 2022-09-24 13:06:04 +00:00
Jack Grigg 9a1d61cb4e zcash_client_backend: Only store successes in batch scanner
Previously we were sending an `Option<DecryptedNote>` from each `Batch`
back to its parent `BatchRunner`. However, this requires allocating
sufficient space in the channel to handle the case where every output
can be decrypted. In general this will not be the case, and we can
instead signal "nothing decrypted" by just dropping the channel sender.
This reduces the post-batch-scanning memory usage of `BatchRunner` from
being linear in the number of on-chain outputs, to being linear in the
number of outputs for the wallet.

Ported from zcash/zcash@f7f6c2070d.
2022-09-21 23:51:40 +00:00
Jack Grigg 533722b70e zcash_client_sqlite: Add dynamic memory usage tracking to `BatchRunner`
Ported from zcash/zcash@e88ea11055.
2022-09-21 23:38:44 +00:00
Daira Hopwood 20e869f501 zcash_primitives: Use prepared epk and ivk in Sapling note decryption
Co-authored-by: Jack Grigg <jack@electriccoin.co>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2022-09-15 03:40:45 +01:00
Kris Nuttycombe 3bc8627e2b
Merge pull request #637 from nuttycom/wallet/create_account
Add WalletWrite::create_account function
2022-09-14 14:30:34 -06:00
Kris Nuttycombe d086c57f2f Make init_accounts_table only permit sequential account identifiers.
Also, check to ensure that account creation does not exceed the
maximum account identifier value.
2022-09-14 13:55:31 -06:00
Kris Nuttycombe 75eb082203
Apply suggestions from code review
Co-authored-by: str4d <thestr4d@gmail.com>
2022-09-14 12:20:39 -06:00
Kris Nuttycombe d0062a87d4 Add WalletWrite::create_account function 2022-09-13 18:58:28 -06:00
Jack Grigg f7b7760051 zcash_client_backend: Add some typedefs for complex types 2022-09-13 21:58:32 +00:00
Jack Grigg 8f03208439 zcash_client_backend: Add tags to IVKs in the batch scanner
This removes the dependency on `SaplingIvk::to_repr()`, and enables us
to alter the type of `D::IncomingViewingKey` to improve the performance
of batch scanning.

For the welding rig, we already annotate the viewing keys with
`AccountId`, so we use `(AccountId, Scope)` as the tag.
2022-09-13 21:40:58 +00:00
Kris Nuttycombe b5908dc964
Merge pull request #632 from nuttycom/data_api/get_next_address
Replace `get_address` with `get_current_address` and `get_next_available_address`
2022-09-12 18:43:38 -06:00
Kris Nuttycombe 03b0ff7e0a Replace `get_address` with `get_current_address` and `get_next_available_address`
This updates the data access API to provide diversified address
functionality. In order to support this change, the addresses table
is updated to store diversifier index information in big-endian order
to allow sorting by diversifier index, and account initialization
is updated to store the diversifier index accordingly. The currently
unreleased `addresses_table` migration is updated to reflect this
change.
2022-09-12 17:58:22 -06:00
Kris Nuttycombe 95610f7b4f Add fee to the transactions table & compute correct net_value in v_transactions. 2022-09-09 16:12:29 -06:00
Jack Grigg 7112ed9da5 zcash_client_backend: Add `WalletReadTransparent::get_transparent_receivers`
Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2022-09-08 20:24:42 +00:00
Jack Grigg 7d404d2747 zcash_client_sqlite: Fix `WalletRead::get_address` return value
Previously it would return an error if the account identifier did not
correspond to a known account.
2022-09-08 18:49:41 +00:00
Jack Grigg a32714043b zcash_client_backend: Change `WalletRead::get_address` to return UA 2022-09-08 18:49:13 +00:00
Kris Nuttycombe 8439f1a4f7 Add binary encoding for unified spending keys.
This encoding is wallet-internal-only and is currently
guarded under the `unstable` feature flag.
2022-09-08 11:05:59 -06:00
Kris Nuttycombe a93f5945ab Remove redundant TryFrom/TryInto imports. 2022-09-02 12:05:15 -06:00
Kris Nuttycombe 16a0e43f2d Update to Rust 2021 and explicitly set MSRVs 2022-09-02 12:05:09 -06:00
Jack Grigg 4f5d757883 zcash_client_backend: Change to `WalletWrite::remove_unmined_tx`
This fixes a bug in the logic ported from the Android SDK: it was
possible to remove a transaction in the middle of a chain, which would
cause a long-spent note to become unspent and cause the wallet balance
to be over-counted. We now restrict transaction removal to unmined
transactions, which is sufficient for the Android SDK use cases.
2022-08-30 22:54:58 +00:00
Jack Grigg 586b7e5bb0 zcash_client_backend: Add `WalletWrite::remove_tx` method
This is to replace the database mutations in the Android SDK. It is
placed behind an `unstable` feature flag until we are satisfied that it
is suitable as a general-purpose API (or replace it).
2022-08-29 21:32:18 +00:00
Kris Nuttycombe ba1bb65a5f Apply suggestions from code review
Co-authored-by: str4d <thestr4d@gmail.com>
2022-08-26 16:09:05 -06:00
Kris Nuttycombe c4da498cf4 Shield funds to the internal Sapling key for a specified account.
This updates `shield_transparent_funds` to look up the internal
(change) address for the account specified, and use that as the
destination for shielding transparent funds.

Fixed #614
2022-08-26 16:03:25 -06:00
Jack Grigg b42344d491 zcash_client_backend: Generate Orchard component for USKs
This ensures that the resulting UFVKs and UAs also contain Orchard
components.
2022-08-24 14:56:58 +00:00
Kris Nuttycombe 397c76ca8d Add tests for batched note decryption. 2022-08-18 16:48:52 -06:00
Kris Nuttycombe 6156215d4c Add parallelized batched trial decryption to wallet scanning.
Co-authored-by: str4d <jack@electriccoin.co>
2022-08-18 16:48:22 -06:00
Kris Nuttycombe 7acd47eae8 Copy parallel batch decryption data types from zcash/zcash
This is an unmodified copy of 7d1e14ac3d/src/rust/src/wallet_scanner.rs

Co-authored-by: str4d <jack@electriccoin.co>
2022-08-18 16:40:27 -06:00
str4d 85a1522c22
Merge pull request #607 from adityapk00/oa
Return orchard address from UA
2022-08-18 23:32:19 +01:00
Kris Nuttycombe 880076b38f Improve error reporting for address & viewing key decoding. 2022-08-18 10:15:58 -06:00
Aditya Kulkarni 1fad73d5b3 Return orchard address from UA 2022-08-17 08:10:06 -05:00
Kris Nuttycombe 199c49760b Use the new name rustdoc::broken_intra_doc_links 2022-07-29 15:56:44 -06:00
Kris Nuttycombe 8a6e979cb9 Change return type of ScanningKey::to_sapling_keys to an associated type. 2022-07-28 17:28:31 -06:00
Kris Nuttycombe 73314dc682 Use batch decryption in wallet scanning.
This modifies wallet scanning to perform per-block batched
decryption. It also alters the structure of the `ScanningKey`
trait to correctly include internal (change) keys in the scan
process.
2022-07-28 17:28:31 -06:00
Kris Nuttycombe f1c2da7b1d Add sapling::NullifierDerivingKey newtype.
Nullifier computation only requires the nullifier deriving key,
not the entire Sapling viewing key. This separation of concerns
will be needed for batch decryption when wallet-internal keys
will need to be considered.
2022-07-28 15:23:21 -06:00
Kris Nuttycombe 878a428c3c Factor out witness updates from scan_output 2022-07-28 15:23:21 -06:00
Kris Nuttycombe 94c8d8e7ef Return the network type for encoding addresses as part of the network parameters. 2022-06-30 12:09:32 -06:00
Daira Hopwood 09567fc280
Merge pull request #574 from nuttycom/fix_account_key_init
Fix account key init
2022-06-29 21:36:29 +01:00
Kris Nuttycombe 0e098e1f17 Test correctness of unknown ufvk element roundtrip encoding. 2022-06-29 10:13:22 -06:00
Daira Hopwood dae2e008e9
Merge pull request #577 from daira/bosl-notice
Projects that rely on BOSL exceptions should say that they do
2022-06-29 12:59:07 +01:00
Kris Nuttycombe 0142a3db1b
Merge pull request #570 from zcash/387-migrate-from-extfvk-to-ufvk
`zcash_client_*`: Migrate from `ExtendedFullViewingKey` to `UnifiedFullViewingKey`
2022-06-28 10:49:37 -06:00
Daira Hopwood a9aa2929d2 Address feedback about the use of "permissive".
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2022-06-28 16:46:13 +01:00
Daira Hopwood 75ca854005 Projects that rely on BOSL exceptions should say that they do.
fixes #576

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2022-06-28 12:29:16 +01:00
Kris Nuttycombe 7236204b14 Don't panic if the sapling key is missing from the UFVK
Instead, just skip Sapling decryption.

Also, a trivial namespacing fix.
2022-06-27 17:44:20 -06:00
Jack Grigg 6f4de16b97 zcash_proofs 0.7.0 2022-06-24 17:00:39 +00:00
Jack Grigg e952ff637e zcash_primitives 0.7.0 2022-06-24 17:00:39 +00:00
Jack Grigg 18fb1f0b07 orchard 0.2 2022-06-24 16:25:51 +00:00
Jack Grigg b52e949bd6 zcash_client_backend: Migrate to correct ZIP 316 UFVK encoding
We also add support for parsing Orchard full viewing keys from encoded
UFVKs (rather than treating them as unknown). `UnifiedSpendingKey` still
does not have Orchard support, so `UnifiedFullViewingKey`s will be
generated without Orchard components.
2022-06-17 10:03:17 +00:00
Jack Grigg 76d015ed11 zcash_client_backend: Fix `UnifiedFullViewingKey` Sapling item type
Per ZIP 316, the Sapling FVK Encoding only includes `(ak, nk, ovk, dk)`
which is a subset of the Sapling `ExtendedFullViewingKey`. We therefore
need to use `DiversifiableFullViewingKey` inside `UnifiedFullViewingKey`
in order to make it parseable from the UFVK string encoding.

`zcash_client_sqlite::wallet::get_extended_full_viewing_keys` has been
removed as a consequence of this change: we can no longer reconstruct
the correct `ExtendedFullViewingKey` from the `UnifiedFullViewingKey`.
2022-06-17 10:03:17 +00:00
Jack Grigg d8b860207d zcash_client_backend: Remove account from `UnifiedFullViewingKey`
The account number is not stored in the ZIP 316 UFVK encoding, and in
general won't necessarily be known (e.g. if a UFVK is being imported
into a wallet).

`zcash_client_sqlite::wallet::init::init_accounts_table` reverts to its
previous behaviour of requiring the provided `&[UnifiedFullViewingKey]`
to be indexed by account number.
2022-06-17 10:03:17 +00:00
Jack Grigg 1ce289e568 zcash_client_backend: Pass UFVKs into `decrypt_transaction` 2022-06-17 10:03:17 +00:00
Jack Grigg c0e8ee0fa0 zcash_client_backend: Return UFVKs from `WalletRead` instead of ExtFVKs 2022-06-17 10:03:17 +00:00
Jack Grigg 0d0527dbf3 zcash_client_sqlite: Store UFVK/UA instead of Sapling ExtFVK/address
This is a breaking change to the database format. We don't have support
for migrations yet, so existing wallets won't work after this commit
until zcash/librustzcash#489 is done.
2022-06-14 02:18:25 +00:00
Jack Grigg e86ba927af zcash_client_backend: Add a fake UFVK encoding
We can't use the real ZIP 316 encoding until `UnifiedFullViewingKey` has
been altered to not store a Sapling `ExtendedFullViewingKey`. But making
that change first requires fully migrating `zcash_client_sqlite` in the
same commit (as its entire API is built around `ExtendedFullViewingKey`).

Instead, we define a temporary fake encoding, to enable migrating the
`zcash_client_sqlite` APIs more incrementally.
2022-06-14 02:18:25 +00:00
Jack Grigg 97aee898b2 zcash_client_backend: Box `zip321::parse::Param::Addr`
After adding the `RecipientAddress::Unified` enum case, `Param::Addr`
is now much larger than the other `Param` cases. This change reduces the
size of the `Param` enum, fixing a clippy lint.
2022-06-13 23:04:18 +00:00
Jack Grigg 555a621a10 Add `UnifiedAddress` generation to `UnifiedFullViewingKey` 2022-06-13 23:04:18 +00:00
Jack Grigg ed6016857e zcash_client_backend: Add `RecipientAddress::Unified` 2022-06-10 17:16:59 +00:00
Jack Grigg f20366cf86 zcash_client_backend: Migrate `RecipientAddress` to `zcash_address` 2022-06-10 15:49:19 +00:00
Jack Grigg 1161fc963f zcash_proofs 0.6.0 2022-05-10 23:33:05 +00:00
Jack Grigg 3ecfd26d79 zcash_primitives 0.6.0 2022-05-10 23:33:05 +00:00
Jack Grigg a565c7e113 hdwallet 0.3.1
This release bumped secp256k1 to 0.21.
2022-05-10 23:00:43 +00:00
Jack Grigg 144512b547 Migrate to `ff 0.12` 2022-05-06 20:03:49 +00:00
Kris Nuttycombe 9416ab3a3e Update to secp256k1 v0.21 2022-04-06 16:09:45 -06:00
Jack Grigg 91d9a42a08 Merge branch 'non-consensus-changes-on-branchid-37519621' into merge-non-consensus-changes-3 2022-02-28 23:22:40 +00:00
Kris Nuttycombe 600c06a408 Update MSRV to 1.56.1 2022-02-24 12:26:14 -07:00
str4d efe63ad62b
Merge pull request #515 from str4d/update-compact-proto
zcash_client_backend: Bring in latest `compact_formats.proto`
2022-02-18 00:59:42 +00:00
Jack Grigg cceae3ac34 zcash_client_backend: Address compact_formats.proto comments 2022-02-17 23:58:20 +00:00
Jack Grigg 18562c71dc zcash_client_backend: Bring in latest `compact_formats.proto`
Source: 83bb19918f/walletrpc/compact_formats.proto
2022-02-17 02:44:20 +00:00
Jack Grigg 511ad71a05 Bump `protobuf` to `~2.27.1`
`protobuf 2.26` raised its MSRV to 1.52.1, which we are now above.
`protobuf 2.27.0` raised its MSRV to 1.55.0, but this was reverted in
`protobuf 2.27.1`. In any case, it's clear this dependency bumps MSRV in
minor releases, so we should use tilde requirements for it.
2022-02-15 23:37:52 +00:00
Jack Grigg f822d7aabc Bump MSRV to 1.54.0
This is the MSRV for the orchard crate version that contains the changes
to the Orchard circuit.

Closes zcash/librustzcash#513.
2022-02-15 16:39:29 +00:00
Kris Nuttycombe e20de50b84 Make AccountId internal representation private. 2022-02-10 11:30:02 -07:00
Kris Nuttycombe a304c3953b Fix Clippy beta linting complaints. 2022-02-02 16:02:45 -07:00
Kris Nuttycombe 3699a6df97 Fix typos. 2022-02-02 14:00:05 -07:00
Kris Nuttycombe cdd899dda1 Fix documentation.
Co-authored-by: str4d <jack@electriccoin.co>
2022-02-02 12:53:59 -07:00
Kris Nuttycombe 6b189f18ff Correct the changelog relating to `store_decrypted_tx` 2022-02-02 10:33:44 -07:00
Kris Nuttycombe 9c2d485c80 Address comments from code review.
Co-authored-by: str4d <jack@electriccoin.co>
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2022-02-02 10:33:41 -07:00
Kris Nuttycombe 2dd0348792 Remove the `nullifiers` argument from `store_decrypted_tx`
This value can be obtained internally within the implementation
of `store_decrypted_tx` and does not need to be part of the
public API.
2022-02-02 10:32:35 -07:00
Kris Nuttycombe 41810e5be9 Merge remote-tracking branch 'upstream/non-consensus-changes-on-branchid-37519621' into autoshield-poc-daa 2022-02-02 10:32:35 -07:00
Kris Nuttycombe 3d51c53d68 Address comments from code review.
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2022-02-02 10:32:33 -07:00
Kris Nuttycombe d0e1f98429 Use extension traits for transparent-related data api functionality. 2022-02-02 10:30:43 -07:00
Kris Nuttycombe e8e5d94ea6 Mark functions that will be made crate-private as deprecated 2022-02-02 10:29:19 -07:00
Jack Grigg e9ecb19143 Restrict cargo to `protobuf >=2.20,<2.26`
`protobuf 2.26.0` bumped its MSRV to 1.52.1, which is incompatible with
our current MSRV.
2022-02-01 14:26:47 +00:00
Kris Nuttycombe 2f1d3da26d Update changelogs. 2022-01-27 16:25:01 -07:00
Kris Nuttycombe 6e11f2d11a Validate ZIP 321 request by roundtrip through the URI format.
Co-authored-by: str4d <jack@electriccoin.co>
2022-01-27 16:08:04 -07:00