Commit Graph

13967 Commits

Author SHA1 Message Date
Pieter Wuille 7d991b55db Store/allow tx metadata in all undo records
Previously, transaction metadata (height, coinbase or not, and before
the previous commit also nVersion) was only stored for undo records
that correspond to the last output of a transaction being spent.

This only saves 2 bytes per undo record. Change this to storing this
information for every undo record, and stop complaining for having it
in non-last output spends. This means that undo dat written with
this patch won't be readable by older versions anymore.
2017-05-26 13:33:39 -07:00
Pieter Wuille c3aa0c1194 Report on-disk size in gettxoutsetinfo 2017-05-26 13:33:39 -07:00
Pieter Wuille d342424301 Remove/ignore tx version in utxo and undo
This makes the following changes:
* In undo data and the chainstate database, the transaction nVersion
  field is removed from the data structures, always written as 0, and
  ignored when reading.
* The definition of hash_serialized in gettxoutsetinfo is changed to no
  longer incude the nVersion field. It is renamed to hash_serialized_2
  to avoid confusion. The new definition also includes transaction
  height and coinbase information, as this information was missing
  before.

This depends on having a CHashVerifier-based undo data checksum
verifier.

Apart from changing the definition of serialized_hash, downgrading
after using this patch is supported, as no release ever used the value
of nVersion field in UTXO entries.
2017-05-26 13:27:50 -07:00
Pieter Wuille 7e00322906 Add specialization of SipHash for 256 + 32 bit data
We'll need a version of SipHash for tuples of 256 bits and 32 bits
data, when CCoinsViewCache switches from using txids to COutPoints as
keys.
2017-05-26 13:24:25 -07:00
Pieter Wuille e484652fc3 Introduce CHashVerifier to hash read data
This is necessary later, when we drop the nVersion field from the undo
data. At that point deserializing and reserializing the data won't
roundtrip anymore, and thus that approach can't be used to verify
checksums anymore.

With this CHashVerifier approach, we can deserialize while hashing the
exact serialized form that was used. This is both more efficient and
more correct in that case.
2017-05-26 13:24:25 -07:00
Pieter Wuille f54580e7e4 error() in disconnect for disk corruption, not inconsistency
The error() function unconditionally reports an error. It should only
be used for actually exception situations, and not for the type of
inconsistencies that ApplyTxInUndo/DisconnectBlock can graciously deal
with.

This also makes a subtle semantics change: in ApplyTxInUndo, when a
record with metadata is encountered (indicating it is the last spend
from a tx), don't wipe the CCoins record if it wasn't empty at that
point. This makes sure that UTXO operations never affect any other
UTXOs (including those from the same tx).
2017-05-26 13:24:25 -07:00
Pieter Wuille e66dbde6d1 Add SizeEstimate to CDBBatch
This allows estimating the in-memory size of a LevelDB batch.
2017-05-26 13:24:24 -07:00
Pieter Wuille b4b057a3e0
Merge #10445: Add test for empty chain and reorg consistency for gettxoutsetinfo.
513da90cd Add test for empty chain and reorg consistency for gettxoutsetinfo. (Gregory Maxwell)
822755a42 Fix: make CCoinsViewDbCursor::Seek work for missing keys (Pieter Wuille)

Tree-SHA512: e549921e8b8f599bf61ebe0ee7ef1d2f474043723d633e24665fe434b996a98e039612de8a1c2cd16b63f154943ff5ea1c1935e9561cfb813a00d47d926d0b22
2017-05-26 13:20:35 -07:00
Pieter Wuille b40ceed98a
Merge #10419: [trivial] Fix three recently introduced typos
efc2e3302 [trivial] Fix three recently introduced typos (practicalswift)

Tree-SHA512: 99e97f3c1350299dfce9c0c35547d480f25c0b877da311d9120f113afd3089eda31b88b2378e2370f288b0c41bb69cee0fd3abca661cd93d5a56982f90709f91
2017-05-26 11:56:21 -07:00
Gregory Maxwell 513da90cdd Add test for empty chain and reorg consistency for gettxoutsetinfo. 2017-05-25 09:09:09 +00:00
Jonas Schnelli 4314544d46
Merge #10420: Add Qt tests for wallet spends & bumpfee
5749a4882 Add Qt tests for wallet spends & bumpfee (Russell Yanofsky)

Tree-SHA512: 026785e7b5ab662f37029d0694916757e46e68bf10e1a7bf1e8538a36593ada0768c6cf3c810c66d65fad891c137fc8bb13904ed09ab3bcffd6cf43d09e48621
2017-05-24 08:46:42 +02:00
Pieter Wuille 822755a424 Fix: make CCoinsViewDbCursor::Seek work for missing keys
Thanks to Suhas Daftuar for figuring this out.
2017-05-23 22:25:52 +00:00
Pieter Wuille 4cb8757aae
Merge #10313: [Consensus] Add constant for maximum stack size
cb184b3 Add constant for maximum stack size (Gregory Sanders)

Tree-SHA512: 0ec4a28552a6eee7fba8c69278f63c61f095e4ba06cb34456bd0162cd702370d10cca621ae1797a0d5e62fb39814cbb8245d3a7adc007ca39dab7a49cbc925dc
2017-05-23 14:35:52 -07:00
Pieter Wuille f2f7e97e8c
Merge #10347: Use range-based for loops (C++11) when looping over vector elements
211adc0 Use range-based for loops (C++11) when looping over vector elements (practicalswift)

Tree-SHA512: 0e007f20dcef99d3c7a1036265e00f689d69f42e02fd82dd8389f45b52d31947e5f9388de2610d3d9bd9f554915ce0d35ebce561e5ae3a9013956d0ee4937145
2017-05-23 14:08:20 -07:00
Russell Yanofsky 5749a48821 Add Qt tests for wallet spends & bumpfee
A few code changes were needed to accompany the test:

* Adding setObjectName() calls for a few Qt controls to make them easily
  accessible from the test.

* Calling contextMenu->popup() instead of contextMenu->exec() to open
  the transaction list context menu without blocking the test thread.

* Opening the context menu at the contextualMenu event point rather than
  the cursor position (this change was not strictly needed to make the test
  work, but is more correct).

* Updating the bumped transaction row with showTransaction=true instead of
  false. This is needed to prevent the bumped tx from being hidden, so the last
  part of the test which attempts to bump the bumped tx can work. (Technically
  this change is a more general bugfix not limited to the testing environment,
  but the bug doesn't happen outside of the testing environment because in the
  full Qt client, a queued NotifyTransactionChanged notification causes the row
  to be updated twice, first with showTransaction=false, then immediately after
  with showTransaction=true.)
2017-05-23 12:39:50 -05:00
Wladimir J. van der Laan 7e96ecf075
Merge #9539: [net] Avoid initialization to a value that is never read
5844609 [net] Avoid initialization to a value that is never read (practicalswift)

Tree-SHA512: 068c3fba58034187f546688bc9b8b7317e0657e797850613fb6289a4efc28637e4d06a0fa5e57480538c6b8340ed6d6a6c6f9a96f130b698d5d60975490a03d8
2017-05-23 19:39:50 +02:00
Wladimir J. van der Laan ce8176d038
Merge #10295: [qt] Move some WalletModel functions into CWallet
108f04f Add missing LOCK2 in CWallet::GetAvailableBalance (Russell Yanofsky)
429aa9e [test] Move some tests from qt -> wallet (Russell Yanofsky)
d944bd7 [qt] Move some WalletModel functions into CWallet (Russell Yanofsky)
ef8ca17 [test] Add tests for some walletmodel functions (Russell Yanofsky)

Tree-SHA512: f6384d9f2ff3f7fb173d414588c3e7dc8c311b8ed2ce2b0979fb824a0ed83a7302890ccd3d83197f07f6fdcb6b1ca151584d90ea1961d88dfe8956c87087cde8
2017-05-23 19:32:52 +02:00
Wladimir J. van der Laan 46771514fa
Merge #9571: RPC: getblockchaininfo returns BIP signaling statistics
557c9a6 RPC: getblockchaininfo: BIP9 stats (Matthew Zipkin)

Tree-SHA512: ecf0bf47f04f92becc77acc649fdfa270e768939acce42df39d30069398d40d9a30539862f7c307e08239f78d5c58c470ca5f6e717d2ab8e24db9be0dd7bec0c
2017-05-23 19:12:29 +02:00
Wladimir J. van der Laan 433c57aa6f
Merge #10421: [qt] Remove excess logic: Prefer "return foo;" to "if (foo) { return true; } else { return false; }"
e49b868 [qt] Remove excess logic (practicalswift)

Tree-SHA512: ffcdc94843ea5ddd05ab5acec1f047def8d910da8f53c5019ec49199828d7c370efaedc801537b8e1f44e87f694d19d04b70c240416d2eddfaff9cd4abe3ca07
2017-05-23 18:46:27 +02:00
Wladimir J. van der Laan 15254e907e
Merge #10372: Add perf counter data to GetStrongRandBytes state in scheduler
888cce5 Add perf counter data to GetStrongRandBytes state in scheduler (Matt Corallo)
399fb8f Add internal method to add new random data to our internal RNG state (Matt Corallo)

Tree-SHA512: 9732a3804d015eaf48d56b60c73880014845fd17a098f1ac2eff6bd50a4beb8b8be65956ac1f0d641e96e3a53c17daddd919401edbf2e3797c3fb687730fb913
2017-05-23 18:45:28 +02:00
Wladimir J. van der Laan e76a3927c3
Merge #10410: Fix importwallet edge case rescan bug
2a8e35a Fix importwallet edge case rescan bug (Russell Yanofsky)

Tree-SHA512: 59522c962290f9ef64436349d11183dd1fd829e515d1f5ec802b63dd813d04303e28d4f3ba38df77a6c151ee4c14f3ca5d3d82204c57456ac94054de62ae4bc7
2017-05-23 16:39:19 +02:00
practicalswift efc2e3302d [trivial] Fix three recently introduced typos
```
$ git blame src/policy/fees.cpp | grep becuase
3810e976 (2017-03-07 11:33:44 -0500 789)          * checks for 2*target becuase we are taking the max over all time
$ git blame src/policy/fees.h | grep successfullly
2d2e1705 (2017-04-12 12:29:03 -0400  54)  * representing that a tx was successfullly confirmed in less than or equal to
$ git blame src/wallet/feebumper.cpp | grep "hasen't"
a3878374 (2017-05-11 09:34:39 +0200 258)     // make sure the transaction still has no descendants and hasen't been mined in the meantime
```
2017-05-23 13:22:40 +02:00
Matt Corallo 888cce57a6 Add perf counter data to GetStrongRandBytes state in scheduler 2017-05-22 16:01:52 -04:00
Matt Corallo 399fb8f2d0 Add internal method to add new random data to our internal RNG state 2017-05-22 16:01:29 -04:00
Wladimir J. van der Laan e4775167cb
Merge #10390: [wallet] remove minimum total fee option
091a9ae remove minimum total fee option (Gregory Sanders)

Tree-SHA512: 9be4df3dab0219f30917211408f47b242f7c96dd7663b06ab1cf3cc63027f14956dc680883be6c58673b6452d0f339cf893694e4f21d0d248e70760614d5a344
2017-05-22 20:01:20 +02:00
Wladimir J. van der Laan 217b416c72
Merge #10415: [tests] Speed up fuzzing by ~200x when using afl-fuzz
693247b [test] Speed up fuzzing by ~200x when using afl-fuzz (practicalswift)

Tree-SHA512: 95922fc2616b8cb00dd531ed1140a52bbda4e04292dd8c1c60a8f49dbf6ccb797a18b61180b3fb68d695456b478a1f5ae7fda47e8ecee41dd65555487aef40a3
2017-05-22 16:56:58 +02:00
Wladimir J. van der Laan 6512913830
Merge #10424: Populate services in GetLocalAddress
3070134 Populate services in GetLocalAddress (Alex Morcos)

Tree-SHA512: b822d7e898ccb5b959ccb1b1d0f159f27190c2105fbf8f5b67ae54debab6fa6a0723d65a66e7341f55cd0d80398c3fbb39a41e067b9f4e0bfa2c1cd366032404
2017-05-22 12:50:19 +02:00
MarcoFalke 8e5725666b
Merge #10433: [tests] improve tmpdir structure
b040243 [tests] improve tmpdir structure (John Newbery)

Tree-SHA512: b21ad555c3c23a43087b893ad18bd2398e1df91b82c0bf1804d07fdb582600a1c339e6f4aaca58074e52f146f459943a0e492abc045b2666d4a3a7e0e455d6dd
2017-05-22 08:59:48 +02:00
MarcoFalke a4ca0b0423
Merge #10431: Prevent shadowing the global dustRelayFee
43c5877 Prevent shadowing the global dustRelayFee. (Pavel Janík)

Tree-SHA512: 9765931a7753c484990003396afd0bb65a53f42d1cad9502017720618ce90b3c5ae68591db01e3524adecdbe6925a5eeeebf04012ba644ef3b65073af207ae5d
2017-05-22 08:14:25 +02:00
Wladimir J. van der Laan 6b99dafe42
Merge #10429: tests: fix spurious addrman test failure
6b51b0a tests: fix spurious addrman test failure (Cory Fields)

Tree-SHA512: 3d41723b1a31ff459d950331ffea7f383e4ef6187990be6a634978bead0c29d7c096f68e7edb6d4dc56069c1fe8a6f12a6daf573cb1e542b15d000eaa54ad288
2017-05-19 15:51:35 +02:00
Wladimir J. van der Laan 993a1d710e
Merge #10432: [Trivial] Add BITCOIN_FS_H endif footer in fs.h
2300a5e [Trivial] Add BITCOIN_FS_H endif footer in fs.h (Ahmad Kazi)

Tree-SHA512: fd56c5c78c12db74baeef93186b59814ac626beeecfad5db852ab8eaad8e71f7f47d64590bcdd9052cc9e97f4531ba90c1f66f0f9e6517d2f956d622803111df
2017-05-19 15:50:02 +02:00
Ahmad Kazi 2300a5e13a [Trivial] Add BITCOIN_FS_H endif footer in fs.h 2017-05-19 02:01:28 -07:00
Pavel Janík 43c587738d Prevent shadowing the global dustRelayFee. 2017-05-19 10:40:05 +02:00
practicalswift 211adc074a Use range-based for loops (C++11) when looping over vector elements 2017-05-19 09:56:16 +02:00
practicalswift 693247b82b [test] Speed up fuzzing by ~200x when using afl-fuzz
Enable the `afl-clang-fast++` features deferred forkserver (`__AFL_INIT`) and persistent mode (`__AFL_LOOP(1000)`).

Before this patch:

```
$ afl-fuzz -i input -o output -m512 -- src/test/test_bitcoin_fuzzy
[*] Validating target binary...
[!] WARNING: The target binary is pretty slow! See /usr/local/share/doc/afl/perf_tips.txt.
[+] Here are some useful stats:

    Test case count : 1 favored, 0 variable, 1 total
       Bitmap range : 1072 to 1072 bits (average: 1072.00 bits)
        Exec timing : 20.4k to 20.4k us (average: 20.4k us)
…
exec speed : 57.58/sec (slow!)
exec speed : 48.35/sec (slow!)
exec speed : 53.78/sec (slow!)
```

After this patch:

```
$ afl-fuzz -i input -o output -m512 -- src/test/test_bitcoin_fuzzy
[*] Validating target binary...
[+] Persistent mode binary detected.
[+] Deferred forkserver binary detected.
[+] Here are some useful stats:

    Test case count : 1 favored, 0 variable, 1 total
       Bitmap range : 24 to 24 bits (average: 24.00 bits)
        Exec timing : 114 to 114 us (average: 114 us)
…
exec speed : 15.9k/sec
exec speed : 13.1k/sec
exec speed : 15.1k/sec
```
2017-05-19 07:28:46 +02:00
Cory Fields 6b51b0a076 tests: fix spurious addrman test failure
When inserting two addresses of the same class, from the same source, they have
a 1/64 chance of colliding.
2017-05-19 00:00:44 -04:00
John Newbery b040243cc3 [tests] improve tmpdir structure 2017-05-18 18:11:32 -04:00
Alex Morcos 307013469f Populate services in GetLocalAddress
Previously if we didn't have any local addresses, GetLocalAddress would return
0.0.0.0 and then we'd swap in a peer's notion of our address in AdvertiseLocal,
but then nServices would never get set.
2017-05-18 16:57:53 -04:00
Wladimir J. van der Laan ea6fde3f1d
Merge #8329: Consensus: MOVEONLY: Move functions for tx verification
618d07f MOVEONLY: tx functions to consensus/tx_verify.o (Jorge Timón)

Tree-SHA512: 63fa2777c070a344dbfe61974526a770d962e049881c6f371b0034b1682c1e6e24f47454f01ee35ded20ade34488e023d4467a05369662906b99a73bb5de8497
2017-05-18 21:01:49 +02:00
practicalswift e49b868cfe [qt] Remove excess logic
Replace …

```
if (foo) { return true; } else { return false; }
```

… with the equivalent …

```
return foo;
```
2017-05-18 20:48:14 +02:00
Wladimir J. van der Laan 28c6e8d71b
Merge #10408: Net: Improvements to Tor control port parser
49a199b torcontrol: Handle escapes in Tor QuotedStrings (Jack Grigg)
0182a11 torcontrol: Log invalid parameters in Tor reply strings where meaningful (Jack Grigg)
0b6f40d torcontrol: Check for reading errors in ReadBinaryFile (Jack Grigg)
d63677b torcontrol: Fix ParseTorReplyMapping (Jack Grigg)
29f3c20 torcontrol: Add unit tests for Tor reply parsers (Jack Grigg)
d8e03c0 torcontrol: Improve comments (Jack Grigg)

Tree-SHA512: aa3ce8072d20299b38c4ba9471af7fab1f5df096c237bf40a96ee9274a357f7366f95ced0cc80f8da1f22f6455a1a8e68bad9a5ff71817eef3397b6aefcbc7ae
2017-05-18 19:58:51 +02:00
Jonas Schnelli 962cd3f058
Merge #9697: [Qt] simple fee bumper with user verification
a38783747 Make sure we re-check the conditions of a feebump during commit (Jonas Schnelli)
9b9ca538c Only update the transactionrecord if the fee bump has been commited (Jonas Schnelli)
6ed4368f1 Make sure we use nTxConfirmTarget during Qt fee bumps (Jonas Schnelli)
be08fc39d Make sure we always update the table row after a bumpfee call (Jonas Schnelli)
2678d3dc6 Show old-fee, increase a new-fee in Qt fee bumper confirmation dialog (Jonas Schnelli)
2ec911f60 Add cs_wallet lock assertion to SignTransaction() (Jonas Schnelli)
fbf385cc8 [Qt] simple fee bumper with user verification (Jonas Schnelli)

Tree-SHA512: a3ce626201abf64cee496dd1d83870de51ba633de40c48eb0219c3eba5085c038af34c284512130d2544de20c1bff9fea1b78f92e3574c21dd4e96c11b8e7d76
2017-05-18 11:18:23 +02:00
Wladimir J. van der Laan 2acface32a
Merge #9750: Bloomfilter: parameter variables made constant
64aa36e param variables made const (ロハン ダル)

Tree-SHA512: 7c19f9e7dd574c8ce8a9468555f27196735b583efe349c1309c90e1e5d2949daf6891574b4bea7122d6c6aca0c7ee4a782fe3d24918d889f7bf89227084a51cd
2017-05-18 10:09:05 +02:00
Pieter Wuille ae786098bc
Merge #10061: [net] Added SetSocketNoDelay() utility function
ad415bc [net] Added SetSocketNoDelay() utility function (Thomas Snider)

Tree-SHA512: c19e3c9910b3fc2ef86f2434f3e91d343e9cd9e2116153941de9789e2a6fc0389bffe762d21b55cda4a4b1de993afee0564c6946e65d05cef9e866b58896f9af
2017-05-17 17:36:27 -07:00
Pieter Wuille c33652576c
Merge #10395: Replace boost::function with std::function (C++11)
1b936f5 Replace boost::function with std::function (C++11) (practicalswift)

Tree-SHA512: c4faec8cf3f801842010976115681f68ffa08fbc97ba50b22e95c936840f47e1b3bd8d7fd2f5b4e094b5a46bf3d29fc90b69d975a99e77322c0d19f8a00d53d3
2017-05-17 17:22:43 -07:00
Pieter Wuille e317c0d192
Merge #8384: Add witness data output to TxInError messages
6e9e026 Expand signrawtransaction.py to cover error witness checking (Gregory Sanders)
9f7341b Add witness data output to TxInError messages (Gregory Sanders)

Tree-SHA512: 6f2a758544fa2657f3a57051bdb80fb14cb10501c8ef4ccbab7a62d4b6a823e74f40991c8796248865def24619b620b859dc2bb08dc2cc72511c1cf3897ab1a9
2017-05-17 17:14:18 -07:00
Pieter Wuille bee3529971
Merge #10196: Bugfix: PrioritiseTransaction updates the mempool tx counter
6c2e25c [qa] Test prioritise_transaction / getblocktemplate interaction (Suhas Daftuar)
acc2e4b Bugfix: PrioritiseTransaction updates the mempool tx counter (Suhas Daftuar)

Tree-SHA512: dcf834df52d84d5eb86acb847c3f28d3cffd1f78f3092b8ff8913c2c400675a071c48a19cd852fdbaac1582aa1dba23433e0e16055831ef2a5e76dde91199941
2017-05-17 13:25:22 -07:00
Pieter Wuille 318ea50a1c
Merge #10199: Better fee estimates
38bc1ec Make more json-like output from estimaterawfee (Alex Morcos)
2d2e170 Comments and improved documentation (Alex Morcos)
ef589f8 minor cleanup: remove unnecessary variable (Alex Morcos)
3ee76d6 Introduce a scale factor (Alex Morcos)
5f1f0c6 Historical block span (Alex Morcos)
aa19b8e Clean up fee estimate debug printing (Alex Morcos)
10f7cbd Track first recorded height (Alex Morcos)
3810e97 Rewrite estimateSmartFee (Alex Morcos)
c7447ec Track failures in fee estimation. (Alex Morcos)
4186d3f Expose estimaterawfee (Alex Morcos)
2681153 minor refactor: explicitly track start of new bucket range and don't update curNearBucket on final loop. (Alex Morcos)
1ba43cc Make EstimateMedianVal smarter about small failures. (Alex Morcos)
d3e30bc Refactor to update moving average on fly (Alex Morcos)
e5007ba Change parameters for fee estimation and estimates on all 3 time horizons. (Alex Morcos)
c0a273f Change file format for fee estimates. (Alex Morcos)

Tree-SHA512: 186e7508d86a1f351bb656edcd84ee9091f5f2706331eda9ee29da9c8eb5bf67b8c1f2abf6662835560e7f613b1377099054f20767f41ddcdbc89c4f9e78946d
2017-05-17 13:15:07 -07:00
Alex Morcos 38bc1ec4a4 Make more json-like output from estimaterawfee 2017-05-17 15:42:32 -04:00
Wladimir J. van der Laan e61fc2d7cb
Merge #10414: [depends] miniupnpc 2.0.20170509
af5d48c [depends] miniupnpc 2.0.20170509 (fanquake)

Tree-SHA512: e6cfa7856aa4ceb17de5e84156cdb90094eedfb08a84ba1cd9a5ce1689533e17863e3f5692a1b7b7b096a7ac88f18539e4b6b7a79f7726401effb11bf9a5ef2f
2017-05-17 20:45:48 +02:00