Commit Graph

17115 Commits

Author SHA1 Message Date
Jim Posen 6a6d764ca5 util: Move debug file management functions into Logger. 2018-04-27 16:10:00 -07:00
Jim Posen f55f4fcf05 util: Establish global logger object.
The object encapsulates logging configuration, and in a later commit,
set up routines will also be moved into the class.
2018-04-27 16:09:59 -07:00
James O'Beirne 58f9a0a002 Use --failfast when running functional tests on Travis 2018-04-27 15:01:40 -04:00
James O'Beirne bf720c1460 Add --failfast option to functional test runner
Also cleans up run_test's arguments list (no more mutable default for `args`)
and call site.
2018-04-27 15:01:31 -04:00
Matt Corallo 9cb6cdc59f Simplify semantics of ChainStateFlushed callback
Previously, ChainStateFlushed would fire either if a full flush
completed (which can happen due to memory limits, forced flush, or
on its own DATABASE_WRITE_INTERVAL timer) *or* on a
ChainStateFlushed-specific DATABASE_WRITE_INTERVAL timer. This is
both less clear for clients (as there are no guarantees about a
flush having actually happened prior to the call), and reults in
extra flushes not clearly intended by the code. We drop the second
case, providing a strong guarantee without removing the periodit
timer-based flushing.
2018-04-27 14:44:56 -04:00
Matt Corallo 50b6533aa2 scripted-diff: Rename SetBestChain callback ChainStateFlushed
This much more accurately captures the meaning of the callback.

-BEGIN VERIFY SCRIPT-
sed -i 's/SetBestChain/ChainStateFlushed/g' src/validationinterface.h src/validationinterface.cpp src/wallet/wallet.h src/wallet/wallet.cpp src/validation.cpp src/index/txindex.h src/index/txindex.cpp
-END VERIFY SCRIPT-
2018-04-27 14:44:22 -04:00
Wladimir J. van der Laan 17266a1306
Merge #13090: Remove Safe mode (achow101)
d8e9a2a Remove "rpc" category from GetWarnings (Wladimir J. van der Laan)
7da3b0a rpc: Move RPC_FORBIDDEN_BY_SAFE_MODE code to reserved section (Wladimir J. van der Laan)
2ae705d Remove Safe mode (Andrew Chow)

Pull request description:

  Rebase of #10563. Safe mode was [disabled by default and deprecated in 0.16](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.16.0.md#safe-mode-disabled-by-default), so probably should be removed for 0.17.

  > Rationale:
  >
  > Safe mode is useless. It only disables some RPC commands when large work forks are detected. Nothing else is affected by safe mode. It seems that very few people would be affected by safe mode. The people who use Core as a wallet are primarily using it through the GUI, which safe mode does not effect. In the GUI, transactions will still be made as normal; only a warning is displayed.
  >
  > I also don't think that we should be disabling RPC commands or any functionality in general. If we do, it should be done consistently, which safe mode is not. If we want to keep the idea of a safe mode around, I think that the current system needs to go first before a new system can be implemented.

Tree-SHA512: 067938f47ca6e879fb6c3c4e21f9946fd7c5da3cde67ef436f1666798c78d049225b9111dc97064f42b3bc549d3915229fa19ad5a634588f381e34fc65d64044
2018-04-27 17:03:44 +02:00
Wladimir J. van der Laan d8e9a2ac74 Remove "rpc" category from GetWarnings
No longer used after removing safe mode.

This function can likely be simplified more, but I'll leave that
for later to make this easy to review.
2018-04-27 09:27:52 +02:00
Wladimir J. van der Laan 7da3b0adb8 rpc: Move RPC_FORBIDDEN_BY_SAFE_MODE code to reserved section
Although this code is no longer ever sent back after removing safe mode,
it would be unwise to remove it from the header.

For one, it would be bad to accidentally re-use the number.

Also some API documentation / bindings are directly generated from the .h
file - this is why the "Aliases for backward compatibility" are there. We don't
want to break code that relies on this error code existing, even if it's never
generated.

So keep it around but move it to a reserved section.
2018-04-27 09:19:09 +02:00
Wladimir J. van der Laan 487dcbe80c
Merge #13002: Do not treat bare multisig outputs as IsMine unless watched
7d0f80b Use anonymous namespace instead of static functions (Pieter Wuille)
b61fb71 Mention removal of bare multisig IsMine in release notes (Pieter Wuille)
9c2a8b8 Do not treat bare multisig as IsMine (Pieter Wuille)
08f3228 Optimization: only test for witness scripts at top level (Pieter Wuille)
3619735 Track difference between scriptPubKey and P2SH execution in IsMine (Pieter Wuille)
ac6ec62 Switch to a private version of SigVersion inside IsMine (Pieter Wuille)
19fc973 Do not expose SigVersion argument to IsMine (Pieter Wuille)
fb1dfbb Remove unused IsMine overload (Pieter Wuille)
952d821 Make CScript -> CScriptID conversion explicit (Pieter Wuille)

Pull request description:

  Currently our wallet code will treat bare multisig outputs (meaning scriptPubKeys with multiple public keys + `OP_CHECKMULTISIG` operator in it) as ours without the user asking for it, as long as all private keys in it are in our wallet.

  This is a pointless feature. As it only works when all private keys are in one place, it's useless compared to single key outputs (P2PK, P2PKH, P2WPKH, P2SH-P2WPKH), and worse in terms of space, cost, UTXO size, and ability to test (due to lack of address format for them).

  Furthermore, they are problematic in that producing a list of all `scriptPubKeys` we accept is not tractable (it involves all combinations of all public keys that are ours). In further wallet changes I'd like to move to a model where all scriptPubKeys that are treated as ours are explicit, rather than defined by whatever keys we have. The current behavior of the wallet is very hard to model in such a design, so I'd like to get rid of it.

  I think there are two options:
  * Remove it entirely (do not ever accept bare multisig outputs as ours, unless watched)
  * Only accept bare multisig outputs in situations where the P2SH version of that output would also be acceptable

  This PR implements the first option. The second option was explored in #12874.

Tree-SHA512: 917ed45b3cac864cee53e27f9a3e900390c576277fbd6751b1250becea04d692b3b426fa09065a3399931013bd579c4f3dbeeb29d51d19ed0c64da75d430ad9a
2018-04-26 20:10:12 +02:00
Wladimir J. van der Laan 8d045a0f66
Merge #13003: qa: Add test for orphan handling
fa02c5b qa: Clarify documentation for send_txs_and_test (MarcoFalke)
fadfbd3 qa: Add test for orphan handling (MarcoFalke)

Pull request description:

Tree-SHA512: e0932d6bd03c73e3113f5457a3ffa3bbfc7b6075dfca8de95224d9df875e60ca6eb15cd8baa226f13de965483006559556191630a83c3bb431e79c53a85ef73f
2018-04-26 19:48:05 +02:00
Chun Kuan Lee ab3f4dd27f tests: Add test for 64-bit PE, modify 32-bit test results
9a75d29b6f change the error result from `PIE` to `DYNAMIC_BASE`. And there are no test for 64-bit, so I made one
2018-04-27 01:34:05 +08:00
Wladimir J. van der Laan 826acc9a3d
Merge #13031: Fix for utiltime to compile with msvc.
abd58a2 Fix for utiltime to compile with msvc. (Aaron Clauson)

Pull request description:

  This PR allows utiltime.cpp to compile with msvc after the changes introduced in #12973.

Tree-SHA512: 7233b1c23400bf19aef2fcb6168009ef58b9e7f8e49c46d8cf9d04394091f370e39496d24ca00b294c4164bcfc04514e329bf6bb05169406c34ce7cd8c382565
2018-04-26 17:22:47 +02:00
Andrew Chow 2ae705d841 Remove Safe mode 2018-04-26 15:33:11 +02:00
Wladimir J. van der Laan eac067ad59
Merge #12321: p2wsh and p2sh-p2wsh address in decodescript
41ff967 list the types of scripts we should consider for a witness program (fivepiece)
4f933b3 p2wpkh, p2wsh and p2sh-nested scripts in decodescript (fivepiece)

Pull request description:

  Attempts to address #12244 .  `p2wsh` addresses are returned only for scripts that are neither `p2sh` nor any witness program.

Tree-SHA512: eb47f094c1a4c2ad2bcf27a8032307e43cf787d50bf739281aeb4101d97316a2f307b05118bf138298c937fa34e15f91436443a9b313f809fad2c43e94cd1831
2018-04-26 14:50:48 +02:00
Wladimir J. van der Laan 6f8b3453f8
Merge #12240: [rpc] Introduced a new `fees` structure that aggregates all sub-field fee types denominated in BTC
7de1de7 Add new fee structure with all sub-fields denominated in BTC (mryandao)

Pull request description:

  the denomination for `fee` is current in btc while the other such as `decendentFee` and `ancestorFee` are in satoshis.

Tree-SHA512: e428f6dca1d339f89ab73e38ce5903f5465c46b159069d9bcc3f8b1140fe6657fa49a11abe0088e9f7ba9999f64af72a349a4735bf5eaa61b8e4a185b23543f3
2018-04-26 12:35:36 +02:00
Wladimir J. van der Laan 646b7f6abe
Merge #12997: [doc] build-windows: Switch to Artful, since Zesty is EOL
fad2958 [doc] build-windows: Switch to Artful, since Zesty is EOL (MarcoFalke)

Pull request description:

  According to https://wiki.ubuntu.com/Releases#End_of_Life, Zesty is EOL, so bump to Artful for now.

  Note that Artful is going to be EOL soon as well, so I am not sure how to proceed after that.

Tree-SHA512: 8efc1ade429cf053f4c69e764175aad8790ded71f04bb9963202be171f1cfa1a3de55a207c0b19271d2f536feafa754ed71a6fafc597e3023bdf0e02fa5cd5e5
2018-04-26 10:01:08 +02:00
Karl-Johan Alm 629ff8c358
-includeconf=<path> support in config handler, for including external configuration files 2018-04-26 12:46:28 +09:00
Pieter Wuille a07e8caa5d
Merge #13033: Build txindex in parallel with validation
9b2704777c [doc] Include txindex changes in the release notes. (Jim Posen)
ed77dd6b30 [test] Simple unit test for TxIndex. (Jim Posen)
6d772a3d44 [rpc] Public interfaces to GetTransaction block until synced. (Jim Posen)
a03f804f2a [index] Move disk IO logic from GetTransaction to TxIndex::FindTx. (Jim Posen)
e0a3b80033 [validation] Replace tx index code in validation code with TxIndex. (Jim Posen)
8181db88f6 [init] Initialize and start TxIndex in init code. (Jim Posen)
f90c3a62f5 [index] TxIndex method to wait until caught up. (Jim Posen)
70d510d93c [index] Allow TxIndex sync thread to be interrupted. (Jim Posen)
94b4f8bbb9 [index] TxIndex initial sync thread. (Jim Posen)
34d68bf3a3 [index] Create new TxIndex class. (Jim Posen)
c88bcec93f [db] Migration for txindex data to new, separate database. (Jim Posen)
0cb8303241 [db] Create separate database for txindex. (Jim Posen)

Pull request description:

  I'm re-opening #11857 as a new pull request because the last one stopped loading for people

  -------------------------------

  This refactors the tx index code to be in it's own class and get built concurrently with validation code. The main benefit is decoupling and moving the txindex into a separate DB. The primary motivation is to lay the groundwork for other indexers that might be desired (such as the [compact filters](https://github.com/bitcoin/bips/pull/636)). The basic idea is that the TxIndex spins up its own thread, which first syncs the txindex to the current block index, then once in sync the BlockConnected ValidationInterface hook writes new blocks.

  ### DB changes

  At the suggestion of some other developers, the txindex has been split out into a separate database. A data migration runs at startup on any nodes with a legacy txindex. Currently the migration blocks node initialization until complete.

  ### Open questions

  - Should the migration of txindex data from the old DB to the new DB block in init or should it happen in a background thread? The downside to backgrounding it is that `getrawtransaction` would return an error message saying the txindex is syncing while the migration is running.

  ### Impact

  In a sample size n=1 test where I synced nodes from scratch, the average time [Index writing](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L1903) was 3.36ms in master and 1.72ms in this branch. The average time between `UpdateTip` log lines for sequential blocks between 400,000 and IBD end on mainnet was 0.297204s in master and 0.286134s in this branch. Most likely this is just variance in IBD times, but I can try with some more trials if people want.

Tree-SHA512: 451fd7d95df89dfafceaa723cdf0f7b137615b531cf5c5035cfb54e9ccc2026cec5ac85edbcf71b7f4e2f102e36e9202b8b3a667e1504a9e1a9976ab1f0079c4
2018-04-25 18:47:47 -07:00
MarcoFalke 24106a85b3
Merge #13082: Tests: don't test against min relay fee information in mining_prioritisetransaction.py
bd9d67b752 Don't test against min relay fee information in mining_prioritisetransaction.py (Kristaps Kaupe)

Pull request description:

  Follow-up on #13032.

  As advised by @MarcoFalke.

Tree-SHA512: a690ff2e6499b0b4ee6a952c849e363738b9a5ae553266e1780fc7910a6b01e240d71b434fe87da53dc43f6e26cec23a900ec16aab64bb8f6765c26d7aa37c56
2018-04-25 21:39:30 -04:00
Kristaps Kaupe bd9d67b752 Don't test against min relay fee information in mining_prioritisetransaction.py
Same reasoning as in 55f89da1a5
2018-04-26 00:30:38 +03:00
practicalswift 66dc662c8a Add compile time checking for all cs_KeyStore runtime locking assertions 2018-04-25 22:31:40 +02:00
Jim Posen 9b2704777c [doc] Include txindex changes in the release notes. 2018-04-25 11:25:19 -07:00
Jim Posen ed77dd6b30 [test] Simple unit test for TxIndex. 2018-04-25 11:25:18 -07:00
Jim Posen 6d772a3d44 [rpc] Public interfaces to GetTransaction block until synced.
Now that the transaction index is updated asynchronously, in order to
preserve the current behavior of public interfaces, the code blocks
until the transaction index is caught up with the current state of the
blockchain.
2018-04-25 11:25:17 -07:00
Jim Posen a03f804f2a [index] Move disk IO logic from GetTransaction to TxIndex::FindTx. 2018-04-25 11:25:15 -07:00
Jim Posen e0a3b80033 [validation] Replace tx index code in validation code with TxIndex. 2018-04-25 11:25:13 -07:00
Jim Posen 8181db88f6 [init] Initialize and start TxIndex in init code. 2018-04-25 11:25:12 -07:00
Jim Posen f90c3a62f5 [index] TxIndex method to wait until caught up.
In order to preserve getrawtransaction RPC behavior, there needs to be
a way for a thread to ensure the transaction index is in sync with the
current state of the blockchain.
2018-04-25 11:25:11 -07:00
Jim Posen 70d510d93c [index] Allow TxIndex sync thread to be interrupted. 2018-04-25 11:25:09 -07:00
Jim Posen 94b4f8bbb9 [index] TxIndex initial sync thread.
TxIndex starts up a background thread to get in sync with the block
index before blocks are processed through the ValidationInterface.
2018-04-25 11:25:08 -07:00
Jim Posen 34d68bf3a3 [index] Create new TxIndex class.
The TxIndex will be responsible for building the transaction index
concurrently with the main validation thread by implementing
ValidationInterface. This does not process blocks concurrently yet.
2018-04-25 11:25:07 -07:00
Jim Posen c88bcec93f [db] Migration for txindex data to new, separate database. 2018-04-25 11:25:06 -07:00
Jim Posen 0cb8303241 [db] Create separate database for txindex.
The new TxIndexDB class will be used by a future commit in this
change set.
2018-04-25 11:25:05 -07:00
Wladimir J. van der Laan 25ad2f75f5
Merge #12830: [qt] [tests] Clarify address book error messages, add tests
5109fc4 [tests] [qt] Add tests for address book manipulation via EditAddressDialog (James O'Beirne)
9c01be1 [tests] [qt] Introduce qt/test/util with a generalized ConfirmMessage (James O'Beirne)
8cdcaee [qt] Display more helpful message when adding a send address has failed (James O'Beirne)
c5b2770 Add purpose arg to Wallet::getAddress (James O'Beirne)

Pull request description:

  Addresses https://github.com/bitcoin/bitcoin/issues/12796.

  When a user attempts to add to the address book a sending address which is already present as a receiving address, they're presented with a confusing error indicating the address is already present in the book, despite the fact that this row is currently invisible.

  ![selection_011](https://user-images.githubusercontent.com/73197/38096704-8a2948d2-3341-11e8-9632-7d563201f28c.jpg)

  This change adds a more specific error message indicating its existence as a receiving address (as discussed in the linked issue).

  ![selection_016](https://user-images.githubusercontent.com/73197/38198467-fa26164e-365a-11e8-8fc5-ddab9caf2fbd.jpg)

  This change also adds some tests exercising use of the address book via QT. Adding so much test code for such a trivial change may seem weird, but it's my hope that this will make further test-writing for address book usage (and other QT features) more approachable.

Tree-SHA512: fbdd5564f7a9a2380bbe437f3378e8d4d5fd9201efff4879b72bc23f2cc1c2eecaf2b811994c25070ee052422e48e47901787c2e62cc584774a997fe6a2a327a
2018-04-25 19:32:32 +02:00
James O'Beirne 5109fc4a9c [tests] [qt] Add tests for address book manipulation via EditAddressDialog
Also modifies corresponding QT code to allow for use within test cases.
2018-04-25 13:13:24 -04:00
James O'Beirne 9c01be1b85 [tests] [qt] Introduce qt/test/util with a generalized ConfirmMessage
ConfirmMessage is reused in future tests apart from its single usage here.
2018-04-25 13:08:53 -04:00
James O'Beirne 8cdcaee4c7 [qt] Display more helpful message when adding a send address has failed
Addresses #12796.

When we're unable to add a sending address to the address book because it
already exists as a receiving address, display a message indicating as much.
This should help avoid confusion about an address supposedly already in the
book but which isn't currently visible in the interface.
2018-04-25 13:08:53 -04:00
James O'Beirne c5b277033a Add purpose arg to Wallet::getAddress
Also make all arguments to getAddress required and document args at call sites.
2018-04-25 13:08:53 -04:00
MarcoFalke a785bc3667
Merge #13073: test: add rpcauth-test to AC_CONFIG_LINKS to fix out-of-tree make check
3e53004339 test: add rpcauth-test to AC_CONFIG_LINKS to fix out-of-tree make check (Wladimir J. van der Laan)

Pull request description:

  Add rpcauth-test (introduced #13056) to AC_CONFIG_LINKS, like the other directly called python scripts, to fix out-of-tree `make check`.
  (forgot to test this before merging, unfortunately)

Tree-SHA512: 60306ac83ee81e0f27d5b4f0420c3bd4edfa3fd5daaa067ff0c235eb55da9f6c559203c2625fed97783b2d11d9f1bed7a359b4d9204ab4af75fd1fd24b9882a8
2018-04-25 10:52:38 -04:00
Wladimir J. van der Laan 3e53004339 test: add rpcauth-test to AC_CONFIG_LINKS to fix out-of-tree make check
Add rpcauth-test (introduced #13056) to AC_CONFIG_LINKS, like the other
directly called python scripts, to fix out-of-tree `make check`.
2018-04-25 16:37:10 +02:00
Wladimir J. van der Laan 106d929780
Merge #13056: [tests] Make rpcauth.py testable and add unit tests
6674a75 [tests] Make rpcauth.py testable and add unit tests (Qasim Javed)

Pull request description:

  refs #12995

Tree-SHA512: 609a85a75dafe46723f050db1e96098f6859519afd5e1b141a332f34ab715144d32fdd8f10d245d6dc0765e77f82158fe7b8aa2e4317f0e893de491d2e0acf1f
2018-04-25 15:56:47 +02:00
Wladimir J. van der Laan e074097c1b
Merge #13051: qa: Normalize executable location
fa811b0 qa: Normalize executable location (MarcoFalke)

Pull request description:

  This removes the need to override the executable locations by just reading them from the config file. Beside making the code easier to read, running individual test on Windows is now possible by default (without providing further command line arguments).

  Note: Of course, it is still possible to manually specify the location through the `BITCOIND` environment variable, e.g. `bitcoin-qt`.

Tree-SHA512: bee6d22246796242d747120ca18aaab089f73067de213c9111182561985c5912228a0b0f7f9eec025ecfdb44db031f15652f30d67c489d481c995bb3232a7ac7
2018-04-25 15:55:13 +02:00
Qasim Javed 6674a75bfb [tests] Make rpcauth.py testable and add unit tests
refs #12995
2018-04-24 11:41:20 -07:00
MarcoFalke fa02c5b38b
qa: Clarify documentation for send_txs_and_test 2018-04-24 13:39:08 -04:00
MarcoFalke a0079d4b6d
Merge #13013: bench: Amend mempool_eviction test for witness txs
fa3bb183ad bench: Amend mempool_eviction test for witness txs (MarcoFalke)
962d223e5c bench: Move constructors out of mempool_eviction hot loop (MarcoFalke)

Pull request description:

Tree-SHA512: 997a07e067623bc2c0904a21bd490d164045cf51393af260fc79882ed010636dce82c9ebe35aae8fa5db5e73c9f3ecb6232353a0939c295034f9be574f1fcff2
2018-04-24 13:25:45 -04:00
João Barbosa e2f58f421b wallet: Make vpwallets usage thread safe 2018-04-24 17:26:04 +01:00
Wladimir J. van der Laan 476cb35551
Merge #12909: wallet: Make fee settings to be non-static members
fac0db0 wallet: Make fee settings non-static members (MarcoFalke)

Pull request description:

  The wallet header defined some globals (they were called "settings"), that should be class members instead.

  This commit is hopefully only refactoring, apart from a multiwallet bugfix: Calling the rpc `settxfee` for one wallet, would set (and change) the fee rate for all loaded wallets. (See added test case)

Tree-SHA512: 4ab6ec2f5c714742396ded5e451ec3b1ceb771e3696492de29889d866de4365b3fbe4a2784d085c8b8bd11b1ebb8a1fec99ab2c62eee716791cfc67c0cf29e1b
2018-04-24 16:37:30 +02:00
Wladimir J. van der Laan d1d54ae6a3
Merge #12953: Deprecate accounts
cead28b [docs] Add release notes for deprecated 'account' API (John Newbery)
72c9575 [wallet] [tests] Add tests for accounts/labels APIs (John Newbery)
109e05d [wallet] [rpc] Deprecate wallet 'account' API (John Newbery)
3576ab1 [wallet] [rpc] Deprecate account RPC methods (John Newbery)
3db1ba0 [tests] Set -deprecatedrpc=accounts in tests (John Newbery)
4e671f0 [tests] Rename rpc_listtransactions.py to wallet_listtransactions.py (John Newbery)
a28b907 [wallet] [rpc] Remove duplicate entries in rpcwallet.cpp's CRPCCommand table (John Newbery)

Pull request description:

  Deprecate all accounts functionality and make it only accessible by using `-deprecatedrpc=accounts`.

  Accounts specific RPCs, account arguments, and account related results all require the `-deprecatedrpc=accunts` startup option now in order to see account things.

  Several wallet functional tests use the accounts system. Those tests are unchanged, except to start the nodes with `-deprecatedrpc=accounts`. We can slowly migrate those tests to use the 'label' API instead of the 'account' API before accounts are fully removed.

Tree-SHA512: 89f4ae2fe6de4a1422f1817b0997ae22d63ab5a1a558362ce923a3871f3e42963405d6573c69c27f1764679cdee5b51bf52202cc407f1361bfd8066d652f3f37
2018-04-24 15:21:27 +02:00
Wladimir J. van der Laan 018c7e5dad
Merge #13024: test: Add rpcauth pair that generated by rpcauth.py
8b8032e test: Add rpcauth pair that generated by rpcauth (Chun Kuan Lee)

Pull request description:

  This PR adds a rpcauth pair that is randomly generated. Also checks that rpcauth.py works fine.
  Resolve #12995

Tree-SHA512: d9661f40e306bcf528dc25919c874ebcdbdd21101319985dc12ce133c80fd0021cfee5e4bfe8ee7970eccc2e24c97e596263b270fe0b79f3613ae573a825ed63
2018-04-24 14:07:23 +02:00