Fix signing raw transactions with unsynced offline nodes
This PR address the issue in two different ways:
- In `signrawtransaction` we determine the consensus branch ID (which we then later use to construct the transaction) using the chain height. We now also consider the `APPROX_RELEASE_HEIGHT` as this is a better estimation than 0 for the height of the chain if we are unsynced. (This in and of itself solves the Overwinter signing issue).
- We have added an additional parameter to `signrawtransaction` to allow manually overriding the consensus branch ID that zcashd determines we are on. This allows users to work around corner cases where the first strategy is still insufficient.
Closes#3327.
Rpc test cleanup
The main purpose of this PR was to inline the method 'wait_for_async_operation_id' from util.py in tests where we were doing this manually. To make it work in all cases, I had to change the method to return the result in the case where the call failed. I also did a bit of general clean up (removing semicolons) in the files which I changed.
Sapling keys generated from the seed are not yet persisted, so we don't
want to persist the seed or chain state either, otherwise the wallet
could end up in an inconsistent state.
Some tests are temporarily disabled because commenting out HDSeed
persistence breaks invariants inside CCryptoKeyStore.
Revert this commit during the PR for #3388.
generalize mininode.py protocol versioning
Fixes#2982 -- replace `overwintered` boolean argument to NodeConn constructor with a protocol version integer, so that adding Sapling (and beyond) will be easier.
Fix absurd fee bug reported in #3281, with tests
This was ported from the Hush `absurd_fee_bug` PR: https://github.com/MyHush/hush/pull/136
Tests executed with `./qa/pull-tester/rpc-tests.sh wallet` and pass on Ubuntu 16.04. Some tests were added for previously untested behavior as well.
Closes#3281.
Add change indicator
This adds to the json returned when calling z_listreceivedbyaddress and z_listuspent an additional field entitiled 'change' which will be either true or false depending on whether or not the note was change from another transaction.
(rpc-test) accurately account for fee without rounding error
Fix for #2807, this test compares balances after doing key exports and imports, and expects these balances to be equal. But they are not exactly equal due to transaction fees, so the test makes them "equal" by rounding a value that has had fees taken out up to the nearest hundredth of a unit (which is much more than the default fee). This obviously is somewhat sloppy. It also converts a balance to float, which really should never be used due to loss of precision (use Decimal instead).
This change makes the test accurately account for the fee using precise comparisons, and removes the use of float. This test doesn't depend on the default fee (0.0001) but instead sets the fee (to that value). This way, if the default fee changes in the future, this test will continue to run. While testing these changes, I set the fee to various values (up to the max, 0.0190), and the test still passes.
These changes decode valid SIGHASH types on signatures in assembly (asm) representations of scriptSig scripts.
This squashed commit incorporates substantial helpful feedback from jtimon, laanwj, and sipa.
When rewinding, remove insufficiently-validated blocks
If a block is insufficiently-validated against a particular branch ID, then we
cannot guarantee that even the block header will be valid under the actual
consensus rules the node will want to apply. Instead require that the blocks are
completely re-validated, by removing them from the block index (which is
equivalent to reducing their validity to BLOCK_VALID_UNKNOWN).
Closes#3100.
Return JoinSplit and JoinSplitOutput indexes in z_listreceivedbyaddress
This implements a feature request by @nathan-at-least in https://github.com/zcash/zcash/issues/2932 and I wanted to ask for feedback about the exact form of the data returned before updating tests and docs.
Add Note Locking to z_mergetoaddress
Adds note locking to `z_mergetoaddress` allowing it to be invoked multiple times before previous `z_mergetoaddress` operations have finished.
Reference issue [#3046](https://github.com/zcash/zcash/issues/3046)
Co-authored-by: Eirik Ogilvie-Wigley <eirik@z.cash>
Ignore -mempooltxinputlimit once Overwinter activates
Overwinter changes the SignatureHash function to solve the quadratic hashing problem, so this option will no longer be needed.
Mempool improvements, branch ID awareness
Whenever the local chain tip is updated, transactions in the mempool which commit to an
unmineable branch ID (for example, just before a network upgrade activates, where the
next block will have a different branch ID) will be removed.
Includes commits cherry-picked from the following upstream PRs:
- bitcoin/bitcoin#6654
- Only the mempool index change.
- bitcoin/bitcoin#6776
- bitcoin/bitcoin#7020
- bitcoin/bitcoin#6915
Part of #2074.
Strictly speaking this is not a true deduplication; the test output will be
slightly different (due to inconsistent print statements) but I think this is
close enough.
Track net value entering and exiting the Sprout circuit
Delta values will be stored for new blocks; old blocks can be filled in by
re-indexing. The net value currently in the Sprout circuit is only calculated
when delta values for all previous blocks are present.
Part of #2351.
The mempool was not synced so a block could be generated and not
clear out the mempool. This would then cause subsequent code
which expected the mempool of all nodes to be empty to hang.
The new parameter is to satisfy the principle of least astonishment
by providing a sensible default for the maximum number of transparent
inputs to shield. If users do not configure -mempooltxinputlimit
it is possible for them to create transactions with hundreds of
inputs which suffer from mining delay, due to the current state of
the network where some miners have configured -mempooltxinputlimit
as a way to deal with the problem of quadratic hashing.
Fix bug where performance-measurements.sh fails hards when given no args
Better than "$1: unbound variable", we ran into this when testing this script in the Hush repo, so we are pushing this fix upstream.
[Test] New merge test suite driver script
Running the script with no arguments will run all test stages in succession.
Passing one or more stages as arguments will run just those stages.
Closes#429.
Not moved, because upstream makes improvements to this script, and the need to
set environment variables makes it simpler to just use the given script.
Migrate MiniNode to Zcash
This enables various RPC tests that use it (most of them in the extended test suite) to properly test Zcash code.
The PR also fixes bugs in the BIP65 and BIP66 tests that were both masking and masked by the un-migrated MiniNode.
The Python module `pyblake2` is now a requirement for the RPC tests.
Part of #2530.
If a bytearray is passed in as part of an iterable, the CScript constructor
fails because b''.join() cannot be used to join a bytearray to a bytes or str in
Python 2.
Importprivkey shows address
Scratching an itch: make `importprivkey` output the corresponding address.
Without this PR, `importprivkey` shows no output. Because we're moving towards an "address-based" RPC interface, rather than "account-based", there's a gap when using `importprivkey` because there's no way to assign it to a specific account, but also no easy way to determine the address. This change fixes that wart.
--nocleanup is sufficient to leave the data directories behind. --noshutdown is
only useful if you want to inspect the nodes afterwards, and you'd need to
manually shut down both nodes before copying the data directories.
Add ability for node to reject tx from mempool by number of tx inputs
Implement short-term solution described in #2343 so that users can respond promptly to critical short-term problems caused by quadratic validation scaling, such as the getblocktemplate latency, block propagation latency, and mempool size inflation issues described in #2333.
Benchmark for calling ConnectBlock on a block with many inputs
Requires placing `block-107134.tar.gz` (containing the block, and a fake CoinsDB containing its inputs) into the base directory of the repository.
To facilitate generation of the fake CoinsDB, an additional field `valuesZat` has been added to `getrawtransaction` containing the integer number of zatoshis instead of a decimal number of ZEC.
Closes#2355.
Requires placing block-107134.tar.gz (containing the block, and a fake CoinsDB
containing its inputs) into the base directory of the repository. This can be
generated using qa/zcash/create_benchmark_archive.py (see the script for usage
details).
To facilitate generation of the fake CoinsDB, an additional field 'valueZat' has
been added to 'getrawtransaction' containing the integer number of zatoshis
instead of a decimal number of ZEC.
Closes#2355.
Fix an error reporting bug due to BrokenPipeError and ConnectionResetError not existing in Python 2
refs #2263
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Usability improvements for z_importkey
- Add height parameter to z_importkey to reduce rescan range
- Change semantics of rescan parameter, so users can explicitly force a rescan
for existing keys.
Closes#2032
Python's httplib does not graciously handle disconnections from the http server, resulting in BadStatusLine errors.
See https://bugs.python.org/issue3566 "httplib persistent connections violate MUST in RFC2616 sec 8.1.4."
This was fixed in Python 3.5.
Work around it for now.
This reverts commit bd30c3dced21fca869a14c75081f15195762afe1.
Disable windows RPC tests for now. These should be re-enabled once a
suitable Wine version is used on Travis.
- Add height parameter to z_importkey to reduce rescan range
- Change semantics of rescan parameter, so users can explicitly force a rescan
for existing keys.
Closes#2032
Bitcoin 0.12 test PRs 1
Cherry-picked from the following upstream PRs:
- bitcoin/bitcoin#6337
- bitcoin/bitcoin#6390
- bitcoin/bitcoin#5515
- bitcoin/bitcoin#6287 (partial, remainder included in bitcoin/bitcoin#6703)
- bitcoin/bitcoin#6465
Part of #2074.
JSON makes no distinction between numbers and reals, and our code
doesn't need to do so either.
This removes VREAL, as well as its specific post-processing in
`UniValue::write`. Non-monetary amounts do not need to be forcibly
formatted with 8 decimals, so the extra roundtrip was unnecessary
(and potentially loses precision).
Zcash: cherry-picked from commit 7650449a6777710cf818d41862626164da0cd412
Left over from 2aee461930
Since unspendable outputs can't be spent, there is no threshold at which it would be uneconomic to spend them.
This primarily targets transaction outputs with `OP_RETURN`.
---
Initially based on:
commit 9cf0ae26350033d43d5dd3c95054c0d1b1641eda
Author: zathras-crypto <zathrasc@gmail.com>
Date: Wed Mar 25 02:04:02 2015 -0700
Changes:
- cherry-picked on top of bitcoin:master
- added RPC test for fundrawtransaction
Test prioritisetransaction
After talking with @str4d about #1884 , I wrote a test for prioritisetransaction. It uses small blocks (11kb), and checks whether a transaction makes it into the next block after being prioritized by that node.
Should this be improved with a larger number of txs in the mempool, or by testing over multiple runs?
As for getblocktemplate(), it seems to return the prioritized transaction within the block size set by the node (about 50 txs fit in an 11kb block), but the block "sizelimit" it displays is set at 2 MB in `rpcmining.cpp` line 690:
```
result.push_back(Pair("sizelimit", (int64_t)MAX_BLOCK_SIZE));
```
This was quite confusing, I didn't think the `-blockmaxsize` parameter I was setting was working for awhile.