Commit Graph

16376 Commits

Author SHA1 Message Date
Taylor Hornby ca63d98169 Untested, not working yet, use libtinfo from the debian packages
(cherry picked from commit a2c647d4bf)
2022-04-04 20:45:48 +01:00
Daira Hopwood 6f2285b035 Merge branch '2022-02-getblocktemplate-locking' (#5594) into backports-v4.6.0-2
(corresponds to merge commit 5141c2971c)
2022-04-04 20:44:19 +01:00
Larry Ruane 5869659e2e document global variables
(cherry picked from commit e170c3abd6)
2022-04-04 20:43:43 +01:00
Pieter Wuille f4e92165f0 Modernize best block mutex/cv/hash variable naming
(cherry picked from commit bitcoin/bitcoin@4a6c0e3dcf)
(cherry picked from commit c079a518c0)
2022-04-04 20:43:33 +01:00
Pieter Wuille 82b98164a5 Fix csBestBlock/cvBlockChange waiting in rpc/mining
(cherry picked from commit bitcoin/bitcoin@45dd135039)
(cherry picked from commit 4693f8165f)
2022-04-04 20:43:23 +01:00
Daira Hopwood 3898c8cccb Merge branch 'update_dockerfile_to_debian_11' (#5323) into backports-v4.6.0-2
(corresponds to merge commit 30c20c0046)
2022-04-04 20:41:16 +01:00
Charlie O'Keefe 6b0067c1e0 Update base image used by Dockerfile from debian 10 to debian 11
I built a docker image with this change and verified that it successfully started zcashd

(cherry picked from commit 21d6835efa)
2022-04-04 20:40:52 +01:00
Daira Hopwood 4bfb8a3828 Merge branch 'fix-drop-warning' (#5459) into backports-v4.6.0-2
(corresponds to merge commit ed93cf5aff)
2022-04-04 20:39:09 +01:00
Daira Hopwood 619a9a1d47 Replace call to drop with zeroization.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
(cherry picked from commit 0e073a53ee)
2022-04-04 20:35:18 +01:00
Daira Hopwood cfa4590c75 Avoid a warning by explicitly calling drop.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
(cherry picked from commit 4f7d15508f)
2022-04-04 20:35:14 +01:00
Kris Nuttycombe afb503503d Omit check of Orchard commitment root after rewind past first checkpoint.
If we no longer have any checkpoints in the Orchard wallet, we must
skip the check against the prior note commitment tree root because
we know that the Orchard wallet's state may be for a chain that is
being reorg'ed away. This is safe because we know that we will have
removed all information from the wallet that we need to perform spends
from that state, and we also know that when we start rolling forward
along the new chain that we will overwrite the initial state of the
Orchard note commitment tree.
2022-04-04 13:05:46 -06:00
Kris Nuttycombe 5b7370c55e Update test to verify rewind behavior. 2022-04-04 13:05:42 -06:00
Kris Nuttycombe 344aef435d Improve error output from OrchardWallet::get_spend_info 2022-04-04 13:05:42 -06:00
Kris Nuttycombe 6fbcba641d Fix a bug in initialization of the Orchard wallet after NU5 activation.
When initializing a new Orchard wallet after NU5 activation, it is
not valid to start from the empty note commitment tree; instead,
the note commitment tree needs to be initialized from the state of
the global Orchard Merkle frontier.

In addition, this change necessitates a change to how rewinds work,
such that in a rollback scenario with a newly initialized wallet
that does not have sufficient checkpoints to fully satisfy a requested
rewind, the rewind is allowed to proceed so long as it does not
invalidate any persisted witness data.
2022-04-04 13:05:42 -06:00
Kris Nuttycombe 4afc6a37c9 Refactor ChainTip to take a struct of Merkle trees instead of a pair.
This makes addition of the Orchard Merkle frontier easier in the future.
2022-04-04 12:04:34 -06:00
Kris Nuttycombe d9de6b64fc Adds a test demonstrating an Orchard wallet initialization bug.
If a new Orchard wallet is created after the first Orchard spend
post NU5 activation, it causes an assertion failure because the root
of the wallet's empty note commitment tree does not match the global
note commitment tree root.
2022-04-04 12:04:32 -06:00
Jack Grigg 098a70ed89 wallet: Rename `CWallet::GetBestBlock` to `GetPersistedBestBlock`
This more accurately reflects its meaning, as it corresponds to the most
recently persisted best chain (i.e. the chain tip that the wallet will
return to on restart), rather than the chain tip to which the in-memory
wallet state has been synced.
2022-04-04 17:43:17 +00:00
Jack Grigg bb072f06c9 AcceptToMemoryPool: Re-add missing code comment
The comment on `view.SetBackend(dummy)` was removed when we backported
upstream locking PRs in zcash/zcash#5017. The upstream commit in
question removed a locking scope but did not remove the reference to
that scope in the comment. Our backport removed the outdated comment,
but should have modified it instead, because otherwise the existence of
`view.SetBackend(dummy)` is very confusing (as it disconnects the cache
from `pcoinsTip`, on the assumption that everything we need from it has
been cached via calls to `CCoinsViewCache::HaveCoins` and
`CCoinsViewCache::HaveShieldedRequirements`).
2022-04-04 16:57:55 +00:00
Jack Grigg c99e7752e4 mempool: Remove duplicated anchor and nullifier assertions
Now that we call `Consensus::CheckTxShieldedInputs` on all transactions
in `CTxMemPool::check`, we don't need to separately check the nullifiers
and anchors directly. That change also fixed a bug where we weren't
previously making the same assertions about Orchard Actions.
2022-04-04 16:57:55 +00:00
Jack Grigg f1cda64602 Apply `HaveShieldedRequirements` to coinbase transactions
Both transparent and shielded inputs have contextual checks that need to
be enforced in the consensus rules. For shielded inputs, these are that
the anchors in transactions correspond to real commitment tree states
(to ensure that the spent notes existed), and that their nullifiers are
not being double-spent.

When Sprout was first added to the codebase, we added input checks in
the same places that transparent inputs were checked; namely anywhere
`CCoinsViewCache::HaveInputs` is called. These all happened to be gated
on `!tx.IsCoinBase()`, which was fine because we did not allow Sprout
JoinSplits in coinbase transactions (enforced with a non-contextual
check).

When we added Sapling we also allowed coinbase outputs to Sapling
addresses (shielded coinbase). We updated `HaveShieldedRequirements` to
check Sapling anchors and nullifiers, but didn't change the consensus
code to call it on coinbase. This was fine because Sapling Spends and
Outputs are separate, and we did not allow Sapling Spends in coinbase
transactions (meaning that there were no anchors or nullifiers to
enforce the input rules on).

Orchard falls into an interesting middle-ground:
- We allowed coinbase outputs to Orchard addresses, to enable Sapling
  shielded coinbase users to migrate to Orchard.
- Orchard uses Actions, which are a hybrid of Sprout JoinSplits and
  Sapling Spends/Outputs. That is, an Orchard Action comprises a single
  spend and a single output.

To maintain the "no shielded spends in coinbase" rule, we added an
`enableSpends` flag to the Orchard circuit. We force it to be set to
`false` for coinbase, ensuring that all Orchard spends in a coinbase use
dummy (zero-valued) notes. However, this is insufficient: the coinbase
transaction will still contain an Orchard anchor and nullifiers, and
these need to be correctly constrained.

In particular, not constraining the Orchard nullifiers in a coinbase
transaction enables a Faerie Gold attack. We explicitly require that
Orchard nullifiers are unique, so that there is a unique input to the
nullifier derivation. Without the coinbase check, the following attack
is possible:
- An adversary creates an Orchard Action sending some amount of ZEC to a
  victim address, with a dummy spent note. The entire transaction can be
  fully-shielded by placing the real spent note in a separate Action.
- The adversary uses the exact same dummy note in a coinbase
  transaction, creating the exact same output note (same victim address
  and amount).
- The victim now has two notes with the same ZEC amount, but can only
  spend one of them because they have the same nullifier.

This commit fixes the consensus bug by calling `HaveShieldedRequirements`
outside of `!tx.IsCoinBase()` gates. To simplify its usage, there is now
a `Consensus::CheckTxShieldedInputs` function that handles the logging
and validation state updates. We also move shielded input checks from
`ContextualCheckInputs` to `ContextualCheckShieldedInputs`; these now
mirror each other in that they check contextual rules on transparent and
shielded inputs respectively, followed by checking signatures.
2022-04-04 16:57:55 +00:00
ebfull a4cc6ad3d0
Merge pull request #5788 from daira/orchard-nullifier-check
Orchard nullifier and anchor consistency checks
2022-04-04 08:20:17 -06:00
Steven c8e0503c69
Merge pull request #5797 from nuttycom/feature/z_listunspent_account
Add account ID to z_listunspent results.
2022-04-04 07:17:53 -07:00
Daira Hopwood 7b72a0f6dd
Merge pull request #5807 from superbaud/fix-btest-failures
fix sporadic failures in WriteCryptedSaplingZkeyDirectToDb, StoreAndLoadSaplingZkeys, and in StoreAndRetrieveMnemonicSeedInEncryptedStore
2022-04-02 06:51:38 +01:00
Jack Grigg 3a1261efda wallet: Initialise ThreadNotifyWallets with wallet's best block
The previous code assumed that the last chain tip notified to the wallet
was equal to the node's chain tip at startup. However, this assumption
fails if the node shuts down uncleanly, or if a wallet file is moved
from one node to another.

We now try to start notifying from the wallet's best block, and if the
node doesn't have that block we fall back to the node's chain tip like
before.

Closes zcash/zcash#5805.
2022-04-02 00:38:26 +00:00
Daira Hopwood 6c23b162ba
Merge pull request #5808 from str4d/5806-treat-mnemonichdchain-corruption-as-fatal
wallet: Treat `mnemonichdchain` records as key material
2022-04-02 01:02:08 +01:00
sasha d7b13560c5 Fix the exception message for SetSeedFromMnemonic failure
It's possible for SetSeedFromMnemonic (a wrapper around zip339_phrase_to_seed) to fail for multiple reasons, including an
invalid language code or seed string -- a message about non-UTF8ness
is not correct.
2022-04-01 16:09:01 -07:00
sasha 3cf2ba8853 Fix sporadic failures in StoreAndRetrieveMnemonicSeedInEncryptedStore 2022-04-01 15:34:58 -07:00
Jack Grigg 17d576fef4 wallet: Treat `mnemonichdchain` records as key material
This is a temporary change to ensure that if this record is unreadable,
we immediately shut down the node and don't make any further changes
that could impair our ability to recover from this state later.

Part of zcash/zcash#5806.
2022-04-01 22:16:40 +00:00
Daira Hopwood 0130426dea AcceptToMemoryPool: Remove initial nullifier checks for Sprout and Sapling
(within the mempool only), that are redundant with checks in
HaveShieldedRequirements. The latter take into account nullifiers in the
chain (using CCoinsViewMemPool::GetNullifier) and include Orchard.

There is no meaningful DoS-protection motivation for the checks being
removed here, nor do they simplify reasoning about the code (if anything
the redundancy makes it more confusing).

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2022-04-01 23:16:11 +01:00
sasha d711a7c529 Fix test_wallet_zkeys failures by increasing diversifier search offset
1 chance in 2^-128 is unobservable, whereas 1 chance in 1024 happens
often.
2022-04-01 14:50:05 -07:00
Daira Hopwood 8e15446c17
Merge pull request #5802 from nuttycom/bug/z_listaddress_internal
Do not display internal addresses in z_listaddresses.
2022-04-01 22:47:09 +01:00
Kris Nuttycombe 5e4eb72a9b Do not display internal addresses in z_listaddresses.
Fixes #5800
2022-04-01 12:53:59 -06:00
Kris Nuttycombe 0a0ac9989e Add a test demonstrating that z_listaddresses reveals internal addrs. 2022-04-01 12:46:55 -06:00
Daira Hopwood f9a703ed51 This check done for Sprout and Sapling (which is separate from consensus nullifier checks)
was not being done for Orchard.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2022-04-01 19:42:30 +01:00
Kris Nuttycombe 0a5dbc10d4 Add account ID to z_listunspent results.
Fixes #5795
2022-04-01 10:02:13 -06:00
Kris Nuttycombe f7b11f6da1
Merge pull request #5775 from nuttycom/feature/orchard_list_unspent
Add Orchard support to z_listunspent.
2022-04-01 07:39:25 -06:00
str4d 445e853a60
Merge pull request #5790 from nuttycom/nu5-consensus_merge-master
Merge `master` to the nu5-consensus branch.
2022-04-01 12:08:31 +01:00
Jack Grigg e5756cc198 qa: Add test for Orchard support in `z_listunspent` 2022-04-01 02:44:56 +00:00
Kris Nuttycombe 866278f79a Merge remote-tracking branch 'upstream/master' into nu5-consensus_merge-master 2022-03-31 20:29:28 -06:00
Kris Nuttycombe 4afec83947
Merge pull request #5782 from nuttycom/cleanup/incrementalmerkletree_positions
Update to the latest incrementalmerkletree.
2022-03-31 12:57:00 -06:00
str4d bd2bc8834f
Merge pull request #5779 from str4d/persist-orchard-note-positions
wallet: Persist Orchard note positions with the witness tree
2022-03-31 19:33:08 +01:00
Kris Nuttycombe a852eb223b Update incrementalmerkletree version to fix GC bug & use updated API.
This also modifies the serialized form of the wallet's incremental
merkle tree. This will require a complete reindex for testnet wallets.
2022-03-31 10:52:48 -06:00
Kris Nuttycombe 824824d6d2 Add Orchard support to z_listunspent.
Also, ensure that Sapling internal addresses are not displayed
in z_listunspent outputs.

Fixes #5683
2022-03-31 10:41:35 -06:00
str4d fbd2912885 Apply suggestions from code review
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
Co-authored-by: Kris Nuttycombe <kris.nuttycombe@gmail.com>
2022-03-31 16:07:05 +00:00
Jack Grigg b121fd94d9 wallet: Persist Orchard tx height alongside note positions
Previously we were reconstructing the height on wallet load by looking
up the `blockHash` field of `CMerkleTx` to find the transaction's height
in the main chain. However, this field is updated whenever `AddToWallet`
is called, while the transaction's height and note positions need to be
kept in sync with `SetBestChain`, which is only called once every 10
minutes. In the case that a reorg occurs between `SetBestChain` and the
node shutting down, the resulting height on wallet load would be
inconsistent. As with note positions, any inconsistency should be
resolved by the post-load wallet rescan, which rewinds the Orchard
witness tree and unsets any position information.

Part of zcash/zcash#5784.
2022-03-31 15:45:56 +00:00
Jack Grigg 2d6cb93125 wallet: Add version information to Orchard commitment tree data
We add both C++ client version information (enabling version-specific
changes if necessary, matching the rest of the C++ wallet), and record
version information (enabling local serialization format changes).

Part of zcash/zcash#5784.
2022-03-31 15:08:09 +00:00
Jack Grigg d651f22e86 wallet: Persist Orchard note positions with the witness tree
We rewrite the entire witness tree each time we update the wallet's best
chain state, and we need the note positions to always be consistent with
the tree state (otherwise mined notes become unspendable after the node
restarts).
2022-03-30 23:23:27 +00:00
Jack Grigg ac3229201f wallet: Move Orchard note position data into a separate map 2022-03-30 23:23:27 +00:00
Jack Grigg 0255964559 qa: Add RPC test testing Orchard note position persistence
The test fails during the final `z_sendmany`, because it is selecting a
note that was detected before restarting the node. Because we force the
wallet to call `SetBestChain` on every block, the wallet doesn't need to
rescan on restart, and thus doesn't repopulate the `position` field of
the in-memory note.

This issue went unnoticed in existing tests that exercise node restarts,
because the RPC tests are fast enough that they never pass the 10-minute
timeout for writing the wallet state. This commit adds a regtest-only
config option that disables the lazy writing.
2022-03-30 23:23:27 +00:00
Kris Nuttycombe a394770ab8
Merge pull request #5772 from therealyingtong/scope-api
Update FFI to use scoped APIs for viewing keys and addresses
2022-03-30 13:04:08 -06:00