Benchmark for calling ConnectBlock on a block with many inputs
Requires placing `block-107134.tar.gz` (containing the block, and a fake CoinsDB containing its inputs) into the base directory of the repository.
To facilitate generation of the fake CoinsDB, an additional field `valuesZat` has been added to `getrawtransaction` containing the integer number of zatoshis instead of a decimal number of ZEC.
Closes#2355.
Requires placing block-107134.tar.gz (containing the block, and a fake CoinsDB
containing its inputs) into the base directory of the repository. This can be
generated using qa/zcash/create_benchmark_archive.py (see the script for usage
details).
To facilitate generation of the fake CoinsDB, an additional field 'valueZat' has
been added to 'getrawtransaction' containing the integer number of zatoshis
instead of a decimal number of ZEC.
Closes#2355.
Fix an error reporting bug due to BrokenPipeError and ConnectionResetError not existing in Python 2
refs #2263
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Usability improvements for z_importkey
- Add height parameter to z_importkey to reduce rescan range
- Change semantics of rescan parameter, so users can explicitly force a rescan
for existing keys.
Closes#2032
Python's httplib does not graciously handle disconnections from the http server, resulting in BadStatusLine errors.
See https://bugs.python.org/issue3566 "httplib persistent connections violate MUST in RFC2616 sec 8.1.4."
This was fixed in Python 3.5.
Work around it for now.
This reverts commit bd30c3dced21fca869a14c75081f15195762afe1.
Disable windows RPC tests for now. These should be re-enabled once a
suitable Wine version is used on Travis.
- Add height parameter to z_importkey to reduce rescan range
- Change semantics of rescan parameter, so users can explicitly force a rescan
for existing keys.
Closes#2032
Bitcoin 0.12 test PRs 1
Cherry-picked from the following upstream PRs:
- bitcoin/bitcoin#6337
- bitcoin/bitcoin#6390
- bitcoin/bitcoin#5515
- bitcoin/bitcoin#6287 (partial, remainder included in bitcoin/bitcoin#6703)
- bitcoin/bitcoin#6465
Part of #2074.
JSON makes no distinction between numbers and reals, and our code
doesn't need to do so either.
This removes VREAL, as well as its specific post-processing in
`UniValue::write`. Non-monetary amounts do not need to be forcibly
formatted with 8 decimals, so the extra roundtrip was unnecessary
(and potentially loses precision).
Zcash: cherry-picked from commit 7650449a6777710cf818d41862626164da0cd412
Left over from 2aee461930
Since unspendable outputs can't be spent, there is no threshold at which it would be uneconomic to spend them.
This primarily targets transaction outputs with `OP_RETURN`.
---
Initially based on:
commit 9cf0ae26350033d43d5dd3c95054c0d1b1641eda
Author: zathras-crypto <zathrasc@gmail.com>
Date: Wed Mar 25 02:04:02 2015 -0700
Changes:
- cherry-picked on top of bitcoin:master
- added RPC test for fundrawtransaction
Test prioritisetransaction
After talking with @str4d about #1884 , I wrote a test for prioritisetransaction. It uses small blocks (11kb), and checks whether a transaction makes it into the next block after being prioritized by that node.
Should this be improved with a larger number of txs in the mempool, or by testing over multiple runs?
As for getblocktemplate(), it seems to return the prioritized transaction within the block size set by the node (about 50 txs fit in an 11kb block), but the block "sizelimit" it displays is set at 2 MB in `rpcmining.cpp` line 690:
```
result.push_back(Pair("sizelimit", (int64_t)MAX_BLOCK_SIZE));
```
This was quite confusing, I didn't think the `-blockmaxsize` parameter I was setting was working for awhile.
Closes#1097 so that zcash-cli displays license info like zcashd.
LicenseInfo is refactored from init.cpp to util.cpp so that the
bitcoin-cli makefile target does not need to be modified.
Closes#1857. Fixes bug where a transaction sending from a zaddr would have a priority of zero.
Transactions sent from a zaddr should now be mined sooner as they no longer have a priority of zero
Issue #1851 shows that a zaddr->taddr can be rejected from mempools
due to not meeting fee requirements given the size of the transaction.
Fee calculation for joinsplit txs has not yet been agreed upon, so
during this interim period, this patch ensures joinsplit txs using
the default fee are not rejected due to an insufficient fee.
Commit 2eeb6b randomized the order of input and output notes,
but this is now known to prevent the chaining of multiple joinsplits
in a single transaction. The root cause has yet to be determined.
This patch is a temporary fix and disables the shuffling of input
and output notes. It also adds a chained joinsplit test to the
python qa test suite.
Add z_sendmany rule that when coinbase utxos must be spent in their entirety, with no change
See #1616 - coinbase utxos must be consumed entirely, without any change, since there
is currently no way to specify a change address in z_sendmany.
Fix issue where z_sendmany amount parameter is too strict
Upstream treats an amount parameter of `1` the same as `1.0`. Third-party would like this fixed.
Return a more informative error message when trying to spend coinbase; select non-coinbase inputs when sending to a transparent output if needed
For #1373 and #1519
Code change:
- Extra parameter added to AvailableCoins to include or exclude Coinbase coins. Default value of parameter is 'true' as current behaviour is to include Coinbase coins.
- SelectCoins, used for sending taddr->taddr, will now exclude Coinbase coins.
Unit test:
Tried to write a test to focus on the extra parameter added to AvailableCoins but could not.
Empirical testing on Testnet:
Current behaviour is that upstream RPC commands sendfrom and sendtoaddress try to spend coinbase coins returned by AvailableCoins. So the user will see:
```
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 1000.0
error: {"code":-6,"message":"Insufficient funds"}
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
error: {"code":-4,"message":"Error: The transaction was rejected! This might happen if some of the coins in your wallet were already spent, such as if you used a copy of wallet.dat and coins were spent in the copy but not marked as spent here."}
./zcash-cli sendfrom "" mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
error: {"code":-4,"message":"Error: The transaction was rejected! This might happen if some of the coins in your wallet were already spent, such as if you used a copy of wallet.dat and coins were spent in the copy but not marked as spent here."}
```
After fix is applied:
```
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 1000.0
error: {"code":-6,"message":"Insufficient funds"}
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
error: {"code":-4,"message":"Coinbase funds can only be sent to a zaddr"}
```
When non-coinbase UTXOs exist, they will now be selected and used:
```
./zcash-cli z_sendmany tnPJZHeVxegCg91utaquBRPEDBGjozfz9iLDHt7zvphFbZdspNgkTVLCGjDcadQBKNyUwKs8pNjDXuEZKrE1aNLpFwHgz4t '[{"address":"mx5fTRhLZwbYE7ZqhAPueZgQGSnwTbdvKU", "amount":0.01}]'
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 1000.0
error: {"code":-6,"message":"Insufficient funds"}
./zcash-cli sendtoaddress mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
9818e543ac2f689d4ce8b52087607d73fecd771d45d316a1d9db092f0485aff2
./zcash-cli sendfrom "" mrEGRmGJhmwAa4MQjzGd86ry63vrvovu9b 0.00003000
899f2894823f51f15fc73b5e0871ac943edbe0ff88e1635f86906087b72caf30
```
Extra parameter added to AvailableCoins to include or exclude Coinbase coins.
SelectCoins, used for sending taddr->taddr, will exclude Coinbase coins.
Added qa rpc test and a runtime parameter -regtestprotectcoinbase to enforce
the coinbase->zaddr consensus rule in regtest mode.
Since the parameters changed in z8, the benchmark on speed.z.cash is showing misleading results due to variability. (The quartile and extrema bars will still show the variability with 50 runs, they just won't jump around as much between benchmark data points.)
The genesis blocks and miner tests have been regenerated, because changing the
block header serialisation format changes the block hash, and thus validity.
The Equihash solutions have been removed from the bloom test inputs for
simplicity (block validity is not checked there; only a valid serialisation is
necessary).
Final changes to the circuit
* Remove [redundant](https://github.com/scipr-lab/libsnark/issues/37) bitness constraints that I pointed out in #908.
* The depth is increased from 20 to 29. I chose 29 because the QAP degree ends up being a power of two, which is probably going to be useful for the FFT in our MPC. (Closes#16)
If we're happy with depth 29 (over 268 million possible joinsplits), this will probably be the final change to the constraint system before 1.0.
Run `zcash-gtest` in `make check` and fix performance tests.
* gtest tests weren't being run by make check
* performance tests were broken
* We need to automatically upload graphs of performance, [see this PR](https://github.com/Electric-Coin-Company/bbotzc/pull/15).
* Moves zerocash tests into `zcash`'s full test suite, we're removing them anyway later and it'd be nice to remove them in the PR instead of from buildbot
Automated performance measurement
Supersedes #843 because that one would have merged into the wrong branch. (Oh yeah and I rebased).
**REBASED AND FORCE PUSHED**
Changes to bloom tests were done by running the following commands:
sed -i 's/\(CDataStream stream(ParseHex(".\{152\}\)\(.\{8\}\)/\100000000000000000000000000000000000000000000000000000000\200/' src/test/bloom_tests.cpp
sed -i 's/\(vector<unsigned char> vch = ParseHex(".\{152\}\)\(.\{8\}\)/\100000000000000000000000000000000000000000000000000000000\200/' src/test/bloom_tests.cpp
and then reverting the single change to the transaction line.
Some specifics on consensus changes:
* Transactions must be anchored to a real anchor in the chain.
* Anchors are pushed and popped during ConnectBlock/DisconnectBlock as appropriate.
* DisconnectTip triggers evictions, under some circumstances, of transactions in the
mempool which are anchored to roots that are no longer valid.
* Commitments append to the tree at the current best root during ConnectBlock.
6af25b0 Add BIP65 to getblockchaininfo softforks list (Peter Todd)
ba1da90 Show softfork status in getblockchaininfo (Wladimir J. van der Laan)
70a427b CLTV: Add more tests to improve coverage (Esteban Ordano)
c5a27f4 Add RPC tests for the CHECKLOCKTIMEVERIFY (BIP65) soft-fork (Peter Todd)
5e82e1c Add CHECKLOCKTIMEVERIFY (BIP65) soft-fork logic (Peter Todd)
6ea5ca4 Enable CHECKLOCKTIMEVERIFY as a standard script verify flag (Peter Todd)
4fa7a04 Replace NOP2 with CHECKLOCKTIMEVERIFY (BIP65) (Peter Todd)
6ec08db Move LOCKTIME_THRESHOLD to src/script/script.h (Peter Todd)
684636b Make CScriptNum() take nMaxNumSize as an argument (Peter Todd)
bip65-cltv.py is based on the earlier BIP66 soft-fork RPC test
implemented by Pieter Wuille's 819bcf9b99
bip65-cltv-p2p.py is based on the earlier BIP66 P2P test by Suhas
Daftuar's d76412b068
Rebased-From: 308257856099e82e91881ba97f741d840184727c
45bfa13 PARTIAL: typofixes (found by misspell_fixer) (Veres Lajos)
21c406e add support for miniupnpc api version 14 (Pavel Vasin)
13bd5a7 rpc-tests: re-enable rpc-tests for Windows (Cory Fields)
ccc4ad6 net: Set SO_REUSEADDR for Windows too (Cory Fields)
1f6772e add unit test for CNetAddr::GetGroup. (Alex Morcos)
13642a5 Fix masking of irrelevant bits in address groups. (Alex Morcos)
6b51b9b Replace boost::reverse_lock with our own. (Casey Rodarmor)
626c5e6 Make sure we re-acquire lock if a task throws (Casey Rodarmor)
4877053 Add missing files to files.md (fanquake)
f171fee Handle leveldb::DestroyDB() errors on wipe failure (Adam Weiss)
c5b89fe Fix race condition on test node shutdown (Casey Rodarmor)
4a37410 Handle no chain tip available in InvalidChainFound() (Ross Nicoll)
f6d29a6 Use unique name for AlertNotify tempfile (Casey Rodarmor)
e6adac7 Delay initial pruning until after wallet init (Adam Weiss)
e0020d4 Make sure LogPrint strings are line-terminated (J Ross Nicoll)
7ff9d12 Make sure LogPrintf strings are line-terminated (Wladimir J. van der Laan)
5a39133 build: fix libressl detection (Cory Fields)
f6355e6 Avoid leaking file descriptors in RegisterLoad (Casey Rodarmor)
60457d3 locking: fix a few small issues uncovered by -Wthread-safety (Cory Fields)
a496e11 Remove bash test note from rpc-tests readme (fanquake)
49c6a64 tests: Remove old sh-based test framework (Wladimir J. van der Laan)
a37567d Add autogen.sh to source tarball. (randy-waterhouse)
1f4d7cf travis: for travis generating an extra build (Cory Fields)
This removes the `conflictedbalance.sh` test as well, but that test has
been broken for a long time and isn't part of any scripts.
What it does is, IMO, sufficiently tested by other tests.
This class groups transactions that have been confirmed in blocks into buckets, based on either their fee or their priority. Then for each bucket, the class calculates what percentage of the transactions were confirmed within various numbers of blocks. It does this by keeping an exponentially decaying moving history for each bucket and confirm block count of the percentage of transactions in that bucket that were confirmed within that number of blocks.
-Eliminate txs which didn't have all inputs available at entry from fee/pri calcs
-Add dynamic breakpoints and tracking of confirmation delays in mempool transactions
-Remove old CMinerPolicyEstimator and CBlockAverage code
-New smartfees.py
-Pass a flag to the estimation code, using IsInitialBlockDownload as a proxy for when we are still catching up and we shouldn't be counting how many blocks it takes for transactions to be included.
-Add a policyestimator unit test
Tests error reporting of transaction signing via RPC call "signrawtransaction".
Expected results:
Test 1: create and sign a valid raw transaction with one input:
- 1) The transaction has a complete set of signatures
- 2) No script verification error occurred
Test 2: create and sign a raw transaction with one valid, one invalid and one missing input script:
- 3) The transaction has no complete set of signatures
- 4) Two script verification errors occurred
- 5) Script verification errors have certain properties ("txid", "vout", "scriptSig", "sequence", "error")
- 6) The verification errors refer to the invalid (vin 1) and missing input (vin 2)
bba2216 RPC test for "#5418 Report missing inputs in sendrawtransaction" (Jonas Schnelli)
de8e801 Report missing inputs in sendrawtransaction (Pieter Wuille)
Previously, each NodeConnCB had its own lock to synchronize data structures
used by the testing thread and the networking thread, and NodeConn provided a
separate additional lock for synchronizing access to each send buffer. This
commit replaces those locks with a single global lock (mininode_lock) that we
use to synchronize access to all data structures shared by the two threads.
Updates comptool and maxblocksinflight to use the new synchronization
semantics, eliminating previous race conditions within comptool, and re-enables
invalidblockrequest.py in travis.
2703412 Fix default binary in p2p tests to use environment variable (Suhas Daftuar)
29bff0e Add some travis debugging for python scripts (Suhas Daftuar)
d76412b Add script manipulation tools for use in mininode testing framework (Suhas Daftuar)
b93974c Add comparison tool test runner, built on mininode (Suhas Daftuar)
6c1d1ba Python p2p testing framework (Suhas Daftuar)
script.py is modified from the code in python-bitcoinlib, and provides tools
for manipulating and creating CScript objects.
bignum.py is a dependency for script.py
script_test.py is an example test that uses the script tools for running a test
that compares the behavior of two nodes, in a comptool- style test, for each of
the test cases in the bitcoin unit test script files, script_valid.json and
script_invalid.json. (This test is very slow to run, but is a proof of concept
for how we can write tests to compare consensus-critical behavior between
different versions of bitcoind.)
bipdersig-p2p.py is another example test in the comptool framework, which tests
deployment of BIP DERSIG for a single node. It uses the script.py tools for
manipulating signatures to be non-DER compliant.
comptool.py creates a tool for running a test suite on top of the mininode p2p
framework. It supports two types of tests: those for which we expect certain
behavior (acceptance or rejection of a block or transaction) and those for
which we are just comparing that the behavior of 2 or more nodes is the same.
blockstore.py defines BlockStore and TxStore, which provide db-backed maps
between block/tx hashes and the corresponding block or tx.
blocktools.py defines utility functions for creating and manipulating blocks
and transactions.
invalidblockrequest.py is an example test in the comptool framework, which
tests the behavior of a single node when sent two different types of invalid
blocks (a block with a duplicated transaction and a block with a bad coinbase
value).
mininode.py provides a framework for connecting to a bitcoin node over the p2p
network. NodeConn is the main object that manages connectivity to a node and
provides callbacks; the interface for those callbacks is defined by NodeConnCB.
Defined also are all data structures from bitcoin core that pass on the network
(CBlock, CTransaction, etc), along with de-/serialization functions.
maxblocksinflight.py is an example test using this framework that tests whether
a node is limiting the maximum number of in-flight block requests.
This also adds support to util.py for specifying the binary to use when
starting nodes (for tests that compare the behavior of different bitcoind
versions), and adds maxblocksinflight.py to the pull tester.
1ec900a Remove broken+useless lock/unlock log prints (Matt Corallo)
352ed22 Add merkle blocks test (Matt Corallo)
59ed61b Add RPC call to generate and verify merkle blocks (Matt Corallo)
30da90d Add CMerkleBlock constructor for tx set + block and an empty one (Matt Corallo)
This adds a -prune=N option to bitcoind, which if set to N>0 will enable block
file pruning. When pruning is enabled, block and undo files will be deleted to
try to keep total space used by those files to below the prune target (N, in
MB) specified by the user, subject to some constraints:
- The last 288 blocks on the main chain are always kept (MIN_BLOCKS_TO_KEEP),
- N must be at least 550MB (chosen as a value for the target that could
reasonably be met, with some assumptions about block sizes, orphan rates,
etc; see comment in main.h),
- No blocks are pruned until chainActive is at least 100,000 blocks long (on
mainnet; defined separately for mainnet, testnet, and regtest in chainparams
as nPruneAfterHeight).
This unsets NODE_NETWORK if pruning is enabled.
Also included is an RPC test for pruning (pruning.py).
Thanks to @rdponticelli for earlier work on this feature; this is based in
part off that work.
has parts of @mhearn #4351
* allows querying the utxos over REST
* same binary input and outputs as mentioned in Bip64
* input format = output format
* various rpc/rest regtests
`--nocleanup` should provide a way to preserve test data, but should not have an impact on whether nodes are to be stopped after the test execution.
In particular, when currently running RPC tests with `--nocleanup`, then it may result in several active `bitcoind` processes, which are not terminated properly.
Some tests in CheckBlockIndex require chainActive.Tip(), but when reindexing, chainActive has not been set on the first call to CheckBlockIndex.
reindex.py starts a node, mines 3 blocks, stops, and reindexes with CheckBlockIndex enabled.
Adds a regression test for the wallet's ResendWalletTransactions function, which uses a new, hidden RPC command "resendwallettransactions."
I refactored main's Broadcast signal so it is passed the best-block time, which let me remove a global variable shared between main.cpp and the wallet (nTimeBestReceived).
I also manually tested the "rebroadcast unconfirmed every half hour or so" functionality by:
1. Running bitcoind -connect=0.0.0.0:8333
2. Creating a couple of send-to-self transactions
3. Connect to a peer using -addnode
4. Waited a while, monitoring debug.log, until I see:
```2015-03-23 18:48:10 ResendWalletTransactions: rebroadcast 2 unconfirmed transactions```
One last change: don't bother putting ResendWalletTransactions messages in debug.log unless unconfirmed transactions were actually rebroadcast.
1d9b378 qa/rpc-tests/wallet: Tests for sendmany (Luke Dashjr)
40a7573 rpcwallet/sendmany: Just take an array of addresses to subtract fees from, rather than an Object with all values being identical (Luke Dashjr)
292623a Subtract fee from amount (Cozz Lovan)
90a43c1 [Qt] Code-movement-only: Format confirmation message in sendcoinsdialog (Cozz Lovan)
34318d7 RPC-test based on invalidateblock for mempool coinbase spends (Gavin Andresen)
7fd6219 Make CTxMemPool::remove more effecient by avoiding recursion (Matt Corallo)
b7b4318 Make CTxMemPool::check more thourough by using CheckInputs (Matt Corallo)
723d12c Remove txn which are invalidated by coinbase maturity during reorg (Matt Corallo)
868d041 Remove coinbase-dependant transactions during reorg. (Matt Corallo)
- rest block request returns full unfolded tx details
- /rest/block/notxdetails/<HASH> returns block where transactions are only represented by its hash
Immature coinbase spends are allowed in the memory pool if they can be mined in the next block.
They are not allowed in the memory pool if they cannot be mined in the next block.
This regression test tests those edge cases.
Ported txnmall.sh to Python, and updated to match
recent transaction malleability changes.
I also modified it so it tests both double-spending
confirmed and unconfirmed (only-in-mempool) transactions.
Renamed to txn_doublespend, since that is really what is
being tested. And told the pull-tester to run both
variations on this test.
3c30f27 travis: disable rpc tests for windows until they're not so flaky (Cory Fields)
daf03e7 RPC tests: create initial chain with specific timestamps (Gavin Andresen)
a8b2ce5 regression test only setmocktime RPC call (Gavin Andresen)
Daira Hopwood
Nathan Wilcox
Jack Grigg
Homu
Jay Graber
Simon
Wladimir J. van der Laan
Pieter Wuille
Pavel Janík
Casey Rodarmor
Jonas Schnelli
zathras-crypto
mruddy
Stephen
Jonas Schnelli
Cory Fields
Sean Bowe
zkbot
Taylor Hornby
Nathan Wilcox
Taylor Hornby
Alex Morcos
Peter Todd
Veres Lajos
fanquake
Suhas Daftuar
dexX7
Corinne Dashjr
Jameson Lopp
Matt Corallo
mrbandrews
Gavin Andresen
Luke Dashjr
Cozz Lovan
bikinibabe
Matt Bogosian
Bryan Bishop
dexX7
Daniel Kraft