Without this fix we will get:
AttributeError: 'SpendDescriptionV5' object has no attribute 'anchor'
On certain V5 transactions rehash / calc_sha256, for example:
ca6abd8ef7d6ef158a4a35ea2c2c0cf122f2f664a88f8fa5b6fd79e48c5bed59
As of zcash/librustzcash#633, `SaplingDomain::IncomingViewingKey` now
allocates memory internally, and this memory persists as long as the
`BatchRunner` is alive. Now that we have decoupled the measurement of
heap usage for batch tasks from their internals, we can add bounds to
all of the generic parameters of `Batch` to enable correctly measuring
their actual heap usage.
Ported from zcash/librustzcash@913aa0a988.
Previously, `finalorchardroot` and `orchard.anchor` were serialized differently
in the RPC API, which made it difficult to correlate them. This changes the
serialization for `finalorchardroot` to match `orchard.anchor`.
Currently supports Zcash blocks, block headers, and transactions. Some
consensus rules are also checked, and a JSON context object can be
optionally passed to provide any necessary details for extra contextual
consensus checks.
The primary purpose of this commit is an exercise in using `cargo vet`
for tracking audits of our Rust dependency updates. `cargo update` was
run, and then a simple-to-audit subset of the dependency updates were
audited and committed.
In practice we are using 14.0.0 in most cases, as the LLVM Project have
not published Ubuntu binaries for any point release after 14.0.0 (which
we are using here).
This is in preparation for removing the ability to generate
Sprout outputs from z_shieldcoinbase. Once that is complete,
we will no longer be able to use `z_shieldcoinbase` for test
setup for uses of Sprout funds; instead, the persisted blockchain
state created in this commit will be used for tests that require
the use of Sprout funds.
This brings the test framework into line with how Sprout funds
are now used on mainnet and testnet; existing Sprout funds may
be spent, and Sprout change may be created, but no funds may
be transfered into the Sprout pool, since the activation of
ZIP 211.
This change improves clock management for zcashd by ensuring
that all clock methods (obtaining seconds, milliseconds, and
microseconds since the epoch) agree under testing conditions
using `-mocktime`, and also adds a feature that allows tests
to specify an offset to the system clock; this is useful to
allow comprehensive testing of the "timejacking attack mitigation"
consensus rules.
Since the wallet ecosystem may not be fully updated to handle
v5 transaction parsing at the point of NU5 activation, some
nodes may prefer to construct V4 transactions when not including
Orchard transaction components.
This change adds a CLI flag that allows node users to specify
that preference.
z_mergetoaddress doesn't currently have any mechanism for specifying
the minconf value. We don't want to default back to a minconf of 1,
as before, because we'll be selecting anchors with at least 3
confirmations by default, so the simplest change here is to make
the minconf value default to the number of anchor confirmations, and
add a configurable minconf at some point in the future. It's likely
that `z_mergetoaddress` will end up being deprecated in any case,
so this works for now.
Also, ensure that anywhere that we're attempting to obtain the Orchard
anchor that we don't attempt to get an anchor from a height below NU5
activation.
This changes anchor selection and Orchard authentication path generation
to default to select anchors at a the depth specified by the
`-orchardanchorconfirmations` CLI argument, with a default anchor
selection depth of 10 blocks.
If the value of `minconf` used for a particular call to `z_sendmany` is
less than the the set number of Orchard anchor confirmations, `minconf`
will be used instead for the Orchard anchor confirmations depth,
so that the selected anchor will be certain to contain any notes
selected to be spent.
Fixes#5644
We've decided to remove the option to allow all deprecated features,
because this has the effect that, if a user enables this flag, they
won't get the warning (and hence may forget to take action) at the time
that a feature is moved from the default-allowed set to the
default-denied set.
Co-authored-by: str4d <thestr4d@gmail.com>
This adds an `-allowdeprecated` CLI parameter whose value is a flag
indicating a deprecated feature that should be explicitly enabled.
Multiple instances of this argument may be provided. In the case that
this parameter is not provided, all currently deprecated RPC methods
that are not slated for removal in the next release remain available.
A user may disable all deprecated features entirely by providing the
string "none" as the argument to this parameter, or enable all
deprecated features, including those slated for removal, by providing
the string "all" as the argument to this parameter. In the case that
"all" or "none" is specified, multiple invocations of `-allowdeprecated`
are not permitted.
To explicitly enable only a specific set of deprecated features, use
`-allowdeprecated=<flag1> -allowdeprecated=<flagN> ...` when starting
zcashd. The following flags are recognized:
- "all" - enables all deprecated features.
- "none" - disables all deprecated features.
- "legacy_privacy" - enables the use of the deprecated "legacy" privacy
policy for z_sendmany. This causes the default behavior to conform to
the `FullPrivacy` directive in all cases instead of just for
transactions involving unified addresses.
- "getnewaddress" - enables the `getnewaddress` RPC method.
- "z_getnewaddress" - enables the `z_getnewaddress` RPC method.
- "zcrawreceive" - enables the `zcrawreceive` RPC method.
- "zcrawjoinsplit" - enables the `zcrawjoinsplit` RPC method.
- "zcrawkeygen" - enables the `zcrawkeygen` RPC method.
- "addrtype" - when this option is set, the deprecated `type` attribute
is returned in addition to `pool` or `address_type` (which contain the
same information) in the results of RPC methods that return address metadata.
That is, without having to launch with test_runner.py. There are
several places where the BITCOIND environment variable determines the
executable, but the default is "bitcoind"; change the default to
"src/zcashd". This does require running the test from the top-level
directory.
Also change the environment variable from BITCOIND to ZCASHD. This could
conceivable break someone, but it just makes too much sense not to do.
the warning before test 4r is generated by the check_z_mergetoaddress
at line 459, which moves all the sapling funds into taddr_2 and thus
expects (sapling_balance - DEFAULT_FEE) to show up into taddr_2.
However, the check_z_mergetoaddress_parallel at line 418 had already
funded taddr_2 (with the sum of taddr_1 and taddr_3), and so we need to
add that to the amount of money that check_z_mergetoaddress at line 459
expects to find.
For consistency, we serialize the `finalState` field in the
same (sparse) encoding as Sapling and Sprout use. In the future
we may want to update this to the dense encoding that
incrementalmerkletree::bridgetree::Frontier uses, but that's not
necessary for the moment.
This adds tests to wallet_z_sendmany that demonstrates conditions
where pool-crossing transfers were not being caught by the privacy
policy checks, which were validating only against the transfer
request rather than the actual transfer selected. This missed
cases where sending funds between unified addresses would reveal
amounts via the turnstile.
Making all privacy policy checks after note selection guards against
this error.
If a new Orchard wallet is created after the first Orchard spend
post NU5 activation, it causes an assertion failure because the root
of the wallet's empty note commitment tree does not match the global
note commitment tree root.
The test fails during the final `z_sendmany`, because it is selecting a
note that was detected before restarting the node. Because we force the
wallet to call `SetBestChain` on every block, the wallet doesn't need to
rescan on restart, and thus doesn't repopulate the `position` field of
the in-memory note.
This issue went unnoticed in existing tests that exercise node restarts,
because the RPC tests are fast enough that they never pass the 10-minute
timeout for writing the wallet state. This commit adds a regtest-only
config option that disables the lazy writing.
Make consistent use of "pool", "address type" and "receiver type",
in RPC documentaion, and deprecate bare uses of "type" in RPC APIs.
Fixes#5534
Co-authored-by: str4d <thestr4d@gmail.com>
Orchard output ordering is no longer deterministic, as we now shuffle
spends and outputs during bundle building. But we _can_ still check that
the action index of a note being spent is correct.
Also fixes a bug in the `nuparams` helper, which would have caused MSB
zeroes in consensus branch IDs to not be rendered in the `-nuparams`
option. This hadn't been encountered because we haven't yet generated a
consensus branch ID with a zero MSB.
We need to bump the `zcashd` protocol version because the new rules are
not compatible with existing rules followed by 170015 nodes, but we
_also_ need to ensure we can still bump it again once we set the testnet
reactivation height (changing node network behaviour again). This commit
also enables RPC tests to run (because previously the nodes considered
each other to be too old for NU5 to be active, and were disconnecting).
This includes:
- `orchard =0.1.0-beta.3` which includes the final circuit changes.
- The new NU5 consensus branch ID.
- Updated ZIP 244 test vectors (which use the NU5 consensus branch ID).
- The legacy_random source should only contain Sprout addresses (as
zcashd now only derives transparent addresses from the mnemonic seed).
- Sapling addresses should appear in the mnemonic_seed source, not the
legacy_seed source (as zcashd no longer ever generates keys there).
The RPC test also now has several additional checks:
- `listaddresses` does not return duplicate addresses (an address can
only come from a single source).
- Address sets are now explicitly checked (to ensure that there are no
unexpected addresses under the checked sources).
We now have a full roster of privacy policies. We can graph the
relationships between the policies; arrows point from more-private to
less-private, and each policy is permitted to reveal information covered
by all policies pointing to it (in addition to the extra information it
is permitted to reveal).
FullPrivacy
v
AllowRevealedAmounts
v v
AllowRevealedRecipients ---- AllowRevealedSenders
v / v
AllowFullyTransparent <- AllowLinkingAccountAddresses
v v
NoPrivacy
The synthetic `LegacyCompat` policy now uses the `AllowFullyTransparent`
policy for backwards compatibility, and the `FullPrivacy` policy if the
sender or recipients involve Unified Addresses.
Closeszcash/zcash#5677.
Closeszcash/zcash#5678.
This replaces the bool argument with a string argument, enabling us to
add additional privacy strategies. We also alter the default to be
backwards-compatible with existing RPC method usage, by only enabling
the strongest checks if a Unified Address is involved.
Closeszcash/zcash#5676.
Prior to this change, calling `join_network` after a network split
only worked in the case that no new non-coinbase transactions were
created during the network partition; in the case that such a
transaction was created, `join_network` would fail when waiting
for mempool synchronization, because zcashd nodes do not.
automatically broadcast their mempool contents on restart.
This change modifies `setup_network` to wait for mempool synchronization
or not on the basis of a newly added do_mempool_sync flag. In the
case of `join_network`, this flag is set to `False`; the default value
is set to `True` to preserve previous functionality elsewhere.
Tests should explicitly use the `resendwallettransactions` RPC method
to ensure that mempool transactions are rebroadcast before attempting
to synchronize the network after a `join_network` call.
We can't actually test rollback via RPC tests until
wallet persistence is implemented. This implements
a rollback scenario that will should pass after wallet
persistence is implemented.
The addition of `OrchardRawAddress` to `RecipientAddress` drives most of
the changes in this commit, which enable `z_sendmany` to send funds to
addresses in the Orchard pool once NU5 activates.
We also add Orchard-specific information to several RPCs in order for
tests to pass:
- `z_listunifiedreceivers`
- `z_getbalanceforaccount`
- `z_getbalanceforviewingkey`
This fixes a potential bug with importing the mnemonic into a third
party transparent wallet. Previously, if a user called `getnewaddress`,
made a bunch of transactions that generated at least 20 change
addresses, and then called `getnewaddress` again, the two external
addresses would be separated by a gap of more than 20. If this mnemonic
were imported into a third party transparent wallet, the wallet would
not detect any funds in the second (or subsequent) transparent addresses
because it would detect 20 unused addresses in a row (via the BIP 44
default gap limit).
Now, we track external and internal keys separately; repeated calls to
`getnewaddress` will return addresses for sequential keys. This has the
added benefit that the sequence of `getnewaddress` outputs will be the
same after restoring from a backup.
We also add a `type` field to the output objects (matching the field in
`z_viewtransaction`), now that the output type can't be distinguished
solely from the address encoding.
The ZTXOSelector type allows selection of previous Zcash
transaction outputs (both transparent outputs and shielded notes)
on the basis of either a (legacy) bare address, or for a
BIP-44 account.
We now use the empty array of pools to indicate that the default pools
should be used when providing a diversifier index parameter, instead of
needing a default sentinel value for the diversifier index parameter
(which was previously suggested as '*' which would have caused issues
for defining a consistent type for that parameter).
This adds an `allowRevealedAmounts` argument to z_sendmany. This
flag must be present to allow an amount-revealing cross-pool transfer
to be constructed.
This replaces the old implementation of asyncrpcoperation_sendmany
with one where all transaction construction is delegated to the
transaction builder. The capabilities of z_sendmany are somewhat
modified in the process:
* z_sendmany now permits sending funds from a Sprout address to
both transparent and Sapling addresses. PRIVACY NOTE: When
user sends a Sprout->Sapling transaction, the amount of the
transaction is publicly revealed.
* z_sendmany no longer supports transactions sending funds into
the Sprout pool, with the exception of change amounts when
sending from a Sprout address.
* When sending transparent coinbase funds to a set of shielded
addresses, the amount sent to recipients must fully consume
the input value and no change is permitted. This is a slightly
weaker constraint than was previously implemented; in the past,
only a single shielded recipient was allowed.
This adds a new `AddrSet` type for use in note retrieval
as a filter, in place of a heterogeneous list of `RawAddress`
values. `RawAddress` will complicate the handling of addresses
within the wallet after the addition of unified addresses,
because it does not contain transparent receiver types, and
if we retain this polymorphism it means a lot of invalid states
are represented in places we don't want them to be. It's better
to figure out what types of addresses we're working with as soon
as possible after parsing, and use monomorphic calls from there
on in.
We added support for the NU5 consensus rules in v4.5.0, which alters the
block header to contain a `hashBlockCommitments` value instead of the
chain history root. However, the output of `getblocktemplate` wasn't
returning this value; once NU5 activated, the `blockcommitmentshash`
field was being set to "null" (all-zeroes).
In v4.6.0 we added full NU5 support to `getblocktemplate`, by adding a
`defaultroots` field that gave default values for `hashBlockCommitments`
and the components required to derive it. However, in doing so we
introduced a regression in the (now-deprecated) legacy fields, where
prior to NU5 activation they contained nonsense.
This commit fixes the output of `getblocktemplate` to have the intended
semantics for all fields:
- The `blockcommitmentshash` and `authdataroot` fields in `defaultroots`
are now omitted from block templates for heights before NU5 activation.
- The legacy fields now always contain the default value to be placed
into the block header (regaining their previous semantics).
Co-authored-by: Larry Ruane <larry@z.cash>
This test currently fails with submitblock returning the error
"bad-heartwood-root-in-block".
Added authdigest to GBT coinbasetxn field because we can't obtain this
via getrawtransaction.
Co-authored-by: Jack Grigg <jack@z.cash>
This will ensure that miners can use the values returned by
getblocktemplate (in particular, the block commitment hash)
to submit a valid block using the submitblock RPC.
zcash/zcash:
The `getmininginfo` RPC now omits `currentblockize` and `currentblocktx`
when a block was never assembled via RPC on this node during its current
process instantiation. The relevant RPCs are `generate` (regtest only) and
`getblocktemplate`. Blocks are also assembled when running the internal
miner (`zcashd -gen=1`), after the node mines its first block.
(cherry picked from commit bitcoin/bitcoin@fa178a6385)
This test-only change allows python (rpc) tests to specify, for example,
that NU5 should be activated at height X, without having to specify all
the previous network upgrades. Previous upgrades can (and must) still be
specified if they activate at different block heights (than, in this
example, NU5). This makes tests easier to write (and read), especially
as the number of network upgrades increases over time.
Note that this change only affects zcashd behavior in regtest mode. For
the other network modes (testnet and mainnet), the activation heights
are hard-coded in chainparams.cpp.
Adds HD seed derivation from ZIP-339 mnemonic phrases.
This replaces random generation of transparent spending keys with BIP-44-compatible derivation from a mnemonic seed phrase, and replaces the random HD seed previously used for Sapling HD key derivation
with the same mnemonic seed. After this change, all new transparent and Sapling addresses
will be derived from a new randomly generated mnemonic seed, which is produced in such a way that
it is guaranteed to produce valid unified spending keys at account 0 for all address types.
Closes#5176.
Closes#2673.
Greg Pfeil
Daira Hopwood
Jack Grigg
DeckerSU
str4d
Kris Nuttycombe
Greg Pfeil
zancas
Jack Grigg
sasha
sasha
therealyingtong
mdr0id
Taylor Hornby
Kris Nuttycombe
Larry Ruane
sasha
Steven
Marshall Gaucher
Steven Smith
Charlie O'Keefe
Dimitris Apostolou
Larry Ruane
MarcoFalke