Mempool improvements, branch ID awareness
Whenever the local chain tip is updated, transactions in the mempool which commit to an
unmineable branch ID (for example, just before a network upgrade activates, where the
next block will have a different branch ID) will be removed.
Includes commits cherry-picked from the following upstream PRs:
- bitcoin/bitcoin#6654
- Only the mempool index change.
- bitcoin/bitcoin#6776
- bitcoin/bitcoin#7020
- bitcoin/bitcoin#6915
Part of #2074.
Add ability for node to reject tx from mempool by number of tx inputs
Implement short-term solution described in #2343 so that users can respond promptly to critical short-term problems caused by quadratic validation scaling, such as the getblocktemplate latency, block propagation latency, and mempool size inflation issues described in #2333.
Test prioritisetransaction
After talking with @str4d about #1884 , I wrote a test for prioritisetransaction. It uses small blocks (11kb), and checks whether a transaction makes it into the next block after being prioritized by that node.
Should this be improved with a larger number of txs in the mempool, or by testing over multiple runs?
As for getblocktemplate(), it seems to return the prioritized transaction within the block size set by the node (about 50 txs fit in an 11kb block), but the block "sizelimit" it displays is set at 2 MB in `rpcmining.cpp` line 690:
```
result.push_back(Pair("sizelimit", (int64_t)MAX_BLOCK_SIZE));
```
This was quite confusing, I didn't think the `-blockmaxsize` parameter I was setting was working for awhile.
Return a more informative error message when trying to spend coinbase; select non-coinbase inputs when sending to a transparent output if needed
For #1373 and #1519
Code change:
- Extra parameter added to AvailableCoins to include or exclude Coinbase coins. Default value of parameter is 'true' as current behaviour is to include Coinbase coins.
- SelectCoins, used for sending taddr->taddr, will now exclude Coinbase coins.
Unit test:
Tried to write a test to focus on the extra parameter added to AvailableCoins but could not.
Empirical testing on Testnet:
Current behaviour is that upstream RPC commands sendfrom and sendtoaddress try to spend coinbase coins returned by AvailableCoins. So the user will see:
```
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 1000.0
error: {"code":-6,"message":"Insufficient funds"}
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
error: {"code":-4,"message":"Error: The transaction was rejected! This might happen if some of the coins in your wallet were already spent, such as if you used a copy of wallet.dat and coins were spent in the copy but not marked as spent here."}
./zcash-cli sendfrom "" mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
error: {"code":-4,"message":"Error: The transaction was rejected! This might happen if some of the coins in your wallet were already spent, such as if you used a copy of wallet.dat and coins were spent in the copy but not marked as spent here."}
```
After fix is applied:
```
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 1000.0
error: {"code":-6,"message":"Insufficient funds"}
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
error: {"code":-4,"message":"Coinbase funds can only be sent to a zaddr"}
```
When non-coinbase UTXOs exist, they will now be selected and used:
```
./zcash-cli z_sendmany tnPJZHeVxegCg91utaquBRPEDBGjozfz9iLDHt7zvphFbZdspNgkTVLCGjDcadQBKNyUwKs8pNjDXuEZKrE1aNLpFwHgz4t '[{"address":"mx5fTRhLZwbYE7ZqhAPueZgQGSnwTbdvKU", "amount":0.01}]'
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 1000.0
error: {"code":-6,"message":"Insufficient funds"}
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
9818e543ac2f689d4ce8b52087607d73fecd771d45d316a1d9db092f0485aff2
./zcash-cli sendfrom "" mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
899f2894823f51f15fc73b5e0871ac943edbe0ff88e1635f86906087b72caf30
```
Extra parameter added to AvailableCoins to include or exclude Coinbase coins.
SelectCoins, used for sending taddr->taddr, will exclude Coinbase coins.
Added qa rpc test and a runtime parameter -regtestprotectcoinbase to enforce
the coinbase->zaddr consensus rule in regtest mode.
Tests error reporting of transaction signing via RPC call "signrawtransaction".
Expected results:
Test 1: create and sign a valid raw transaction with one input:
- 1) The transaction has a complete set of signatures
- 2) No script verification error occurred
Test 2: create and sign a raw transaction with one valid, one invalid and one missing input script:
- 3) The transaction has no complete set of signatures
- 4) Two script verification errors occurred
- 5) Script verification errors have certain properties ("txid", "vout", "scriptSig", "sequence", "error")
- 6) The verification errors refer to the invalid (vin 1) and missing input (vin 2)
bba2216 RPC test for "#5418 Report missing inputs in sendrawtransaction" (Jonas Schnelli)
de8e801 Report missing inputs in sendrawtransaction (Pieter Wuille)
comptool.py creates a tool for running a test suite on top of the mininode p2p
framework. It supports two types of tests: those for which we expect certain
behavior (acceptance or rejection of a block or transaction) and those for
which we are just comparing that the behavior of 2 or more nodes is the same.
blockstore.py defines BlockStore and TxStore, which provide db-backed maps
between block/tx hashes and the corresponding block or tx.
blocktools.py defines utility functions for creating and manipulating blocks
and transactions.
invalidblockrequest.py is an example test in the comptool framework, which
tests the behavior of a single node when sent two different types of invalid
blocks (a block with a duplicated transaction and a block with a bad coinbase
value).
mininode.py provides a framework for connecting to a bitcoin node over the p2p
network. NodeConn is the main object that manages connectivity to a node and
provides callbacks; the interface for those callbacks is defined by NodeConnCB.
Defined also are all data structures from bitcoin core that pass on the network
(CBlock, CTransaction, etc), along with de-/serialization functions.
maxblocksinflight.py is an example test using this framework that tests whether
a node is limiting the maximum number of in-flight block requests.
This also adds support to util.py for specifying the binary to use when
starting nodes (for tests that compare the behavior of different bitcoind
versions), and adds maxblocksinflight.py to the pull tester.
Immature coinbase spends are allowed in the memory pool if they can be mined in the next block.
They are not allowed in the memory pool if they cannot be mined in the next block.
This regression test tests those edge cases.
Ported txnmall.sh to Python, and updated to match
recent transaction malleability changes.
I also modified it so it tests both double-spending
confirmed and unconfirmed (only-in-mempool) transactions.
Renamed to txn_doublespend, since that is really what is
being tested. And told the pull-tester to run both
variations on this test.
This adds a -regetest-only undocumented (for regression testing only)
command-line option -blockversion=N to set block.nVersion.
Adds to the "has the rest of the network upgraded to a
block.nVersion we don't understand" code so it calls
-alertnotify when 51 of the last 100 blocks are up-version.
But it only alerts once, not with every subsequent new, upversion
block.
And adds a forknotify.py regression test to make sure it works.
Tested using forknotify.py:
Before adding CAlert::Notify, get:
Assertion failed: -alertnotify did not warn of up-version blocks
Before adding code to only alert once:
Assertion failed: -alertnotify excessive warning of up-version blocks
After final code in this pull:
Tests successful
The entire debug log would be huge, and could cause issues for automated tools
like travis. Printing 200 lines is an initial guess at a reasonable number,
more may be required.
This adds a -whitelist option to specify subnet ranges from which peers
that connect are whitelisted. In addition, there is a -whitebind option
which works like -bind, except peers connecting to it are also
whitelisted (allowing a separate listen port for trusted connections).
Being whitelisted has two effects (for now):
* They are immune to DoS disconnection/banning.
* Transactions they broadcast (which are valid) are always relayed,
even if they were already in the mempool. This means that a node
can function as a gateway for a local network, and that rebroadcasts
from the local network will work as expected.
Whitelisting replaces the magic exemption localhost had for DoS
disconnection (local addresses are still never banned, though), which
implied hidden service connects (from a localhost Tor node) were
incorrectly immune to DoS disconnection as well. This old
behaviour is removed for that reason, but can be restored using
-whitelist=127.0.0.1 or -whitelist=::1 can be specified. -whitebind
is safer to use in case non-trusted localhost connections are expected
(like hidden services).
- Add license headers to source files (years based on commit dates)
in `src/test` as well as `qa`
- Add `README.md` to `src/test/data` specifying MIT license
Fixes#3848
qa/rpc-tests/wallet.sh runs a three-node -regtest network,
generates a fresh blockchain, and then exercises basic wallet
sending/receiving functionality using command-line RPC.
* Use the latest version, with limited memory usage, and path to
on-disk db (try mouting qa/tmp on a tmpfs)\
* enable -debug=net
* re-enable BitcoindComparisonTool in pull-tester
Re-organize the pull-tester scripts a bit.
And disables running the blockchain tester, it is not working properly
on the pull-tester machine for reasons I cannot explain (fails to start).