76ea17c79 Add mutex requirement for AddToCompactExtraTransactions(…) (practicalswift)
4616c825a Use -Wthread-safety-analysis if available (+ -Werror=thread-safety-analysis if --enable-werror) (practicalswift)
7e319d639 Fix -Wthread-safety-analysis warnings. Change the sync.h primitives to std from boost. (Matt Corallo)
Pull request description:
* Add mutex requirement for `AddToCompactExtraTransactions(…)`.
* Use `-Wthread-safety-analysis` if available.
* Rebased on top of https://github.com/TheBlueMatt/bitcoin/commits/2017-08-test-10923 - now includes: Fix -Wthread-safety-analysis warnings. Change the sync.h primitives to std from boost.
Tree-SHA512: fb7365f85daa2741c276a1c899228181a8d46af51db7fbbdffceeaff121a3eb2ab74d7c8bf5e7de879bcc5042d00d24cb4649c312d51caba45a3f6135fd8b38f
cabff7588 rpc: Make logging RPC public (Wladimir J. van der Laan)
Pull request description:
This started out as a developer hack but now it's useful enough for general use. Unhide the call by moving it to `control` category. This makes it documented in `help`.
Tree-SHA512: f45fa378558b552d4e2a110bf85100b0eaaa6180bb5f62cb54a251f66026d4625b670c69d85c281eebbf4b56b80b65618c51a5a593b8f9d0a04b31e95adc91f4
a9b6ba0b7 Add missing cs_main locks when calling blockToJSON/blockheaderToJSON (practicalswift)
Pull request description:
`blockToJSON(...)` and `blockheaderToJSON(...)` read the variable `chainActive` which requires holding the mutex `cs_main`. So does `GetDifficulty(...)`.
Tree-SHA512: bfb94f5e3238accbf6a4daddde49d53f1891c38ae9b07e25b3098c485747159258f64bb66a50e147b32beac601de89d9d04ff717b6c4f1460d329c90a53d3333
3ab545d7f addrman: Add missing lock in Clear() (CAddrMan) (practicalswift)
Pull request description:
Add missing lock in `Clear()` (`CAddrMan`).
The variable `vRandom` is guarded by the mutex `cs`.
**Note to reviewers:** Does this look correct? Should the lock cover the entire scope of the method, or should it be limited to cover only `std::vector<int>().swap(vRandom);`?
Tree-SHA512: 8833f31beaed1728fa55b13ddf9e0b8e24e395931497329be2440ce1c5113ff02871707d40830260adabd30c4ea86088f5da5cf8a821150c0d820f50a2ce386a
ff35de8 [ui] Add toggle for unblinding password fields (Thomas Snider)
Pull request description:
Proposed change for adding the ability to toggle password visibility in the password dialog. This is similar to functionality in most password managers and is specifically added with the use case of password managers in mind - the password in that case is likely pasted twice into both the new password and confirm password fields.
If this is a welcome change, I am open to suggestions on rearranging the layout.
Tree-SHA512: 1823f356f8f941cc584c44de264433e9a573cb8a358efa300a412c4458b5564d8d193969be40859195cf9c8d6768eee895ee22440d51db4f09175f9b4e28bced
This started out as a developer hack but now it's useful
enough for general use. Unhide the call by moving it to `control` category.
This makes it documented in `help`.
Commit 1.
This code was written by @TheBlueMatt in the following branch:
* https://github.com/TheBlueMatt/bitcoin/commits/2017-08-test-10923
This commit message was written by me (@practicalswift) who also squashed
@TheBlueMatt's commits into one and tried to summarize the changes made.
Commit 2.
Remove boost include. Remove boost mentions in comments.
6262915 Add unit test for stale tip checking (Suhas Daftuar)
83df257 Add CConnmanTest to mutate g_connman in tests (João Barbosa)
ac7b37c Connect to an extra outbound peer if our tip is stale (Suhas Daftuar)
db32a65 Track tip update time and last new block announcement from each peer (Suhas Daftuar)
2d4327d net: Allow connecting to extra outbound peers (Suhas Daftuar)
Pull request description:
This is an alternative approach to #11534. Rather than disconnect an outbound peer when our tip looks stale, instead try to connect to an additional outbound peer.
Periodically, check to see if we have more outbound peers than we target (ie if any extra peers are in use), and if so, disconnect the one that least recently announced a new block (breaking ties by choosing the newest peer that we connected to).
Tree-SHA512: 8f19e910e0bb36867f81783e020af225f356451899adfc7ade1895d6d3bd5afe51c83759610dfd10c62090c4fe404efa0283b2f63fde0bd7da898a1aaa7fb281
97932cd rpc: further constrain the libevent workaround (Cory Fields)
6b58360 rpc: work-around an upstream libevent bug (Cory Fields)
Pull request description:
A rare race condition may trigger while awaiting the body of a message.
This may fix some reported rpc hangs/crashes.
This work-around mimics what libevent does internally once a write has started, which is what usually happens, but not always due to the processing happening on a different thread: e7ff4ef2b4/http.c (L373)
Fixed upstream at: 5ff8eb2637
Tree-SHA512: b9fa97cae9da2a44101c5faf1e3be0b9cbdf722982d35541cf224be31430779c75e519c8ed18d06ab7487bfb1211069b28f22739f126d6c28ca62d3f73b79a52
720d9e8fa [Wallet] always show help-line of wallet encryption calls (Jonas Schnelli)
Pull request description:
We do currently show/hide the wallet encryption RPC calls from the help if the current wallet.
In case of an encrypted wallet, `encryptwallet` is hidden and `walletpassphrasechange`, `walletpassphrasechange` and `walletlock` do appear in the help.
This is no longer ideal in case of multiwallet due to the fact that one may want help infos in order to target a specific wallet.
IMO its preferable to have a static help screen (show everything always). The currently show/hidden calls do handle the possible invalid encryption-state fine.
Fixes#11588
Tree-SHA512: 513fecd15248a31361f5143685e8cdeb63dfd3fa7120828917e1db54d936dc3db60d48ce46efa5c3a563a48157fe962689879856eeeed53f904686b12aec204e
If our tip hasn't updated in a while, that may be because our peers are
not relaying blocks to us that we would consider valid. Allow connection
to an additional outbound peer in that circumstance.
Also, periodically check to see if we are exceeding our target number of
outbound peers, and disconnect the one which has least recently
announced a new block to us (choosing the newest such peer in the case
of tie).
A rare race condition may trigger while awaiting the body of a message, see
upsteam commit 5ff8eb26371c4dc56f384b2de35bea2d87814779 for details.
This may fix some reported rpc hangs/crashes.
5d465e396 Ensure backupwallet fails when attempting to backup to source file (Tomas van der Wansem)
Pull request description:
Previous behaviour was to destroy the wallet (to zero-length)
This fixes#11375
Tree-SHA512: bfd1738659b15e3f23b6bbdf55ec12269c62c820bf701daec19500b52bd5845bb5516733c6f76f36197eb155182a8a35dc239ad4de2ef1e59bbb0f124a455759
f3d4adf Make p2p-acceptablock not an extended test (Matt Corallo)
00dcda6 [qa] test that invalid blocks on an invalid chain get a disconnect (Matt Corallo)
015a525 Reject headers building on invalid chains by tracking invalidity (Matt Corallo)
932f118 Accept unrequested blocks with work equal to our tip (Matt Corallo)
3d9c70c Stop always storing blocks from whitelisted peers (Matt Corallo)
3b4ac43 Rewrite p2p-acceptblock in preparation for slight behavior changes (Matt Corallo)
Pull request description:
@sdaftuar pointed out that the version in #11487 was somewhat DoS-able as someone could feed you a valid chain that forked off the the last checkpoint block and force you to do lots of work just walking backwards across blocks for each new block they gave you. We came up with a few proposals but settled on the one implemented here as likely the simplest without obvious DoS issues. It uses our existing on-load mapBlockIndex walk to make sure everything that descends from an invalid block is marked as such, and then simply caches blocks which we attempted to connect but which were found to be invalid. To avoid DoS issues during IBD, this will need to depend on #11458.
Includes tests from #11487.
Tree-SHA512: 46aff8332908e122dae72ceb5fe8cd241902c2281a87f58a5fb486bf69d46458d84a096fdcb5f3e8e07fbcf7466232b10c429f4d67855425f11b38ac0bf612e1
b296bf1 Init: Remove redundant exit(EXIT_FAILURE) instances and replace with return false (donaloconnor)
Pull request description:
While reviewing the bitcoin code I noticed that there are a few exit(EXIT_FAILURE) at various places in the AppInit function.
This function returns to main() which will return/exit with EXIT_FAILURE so returning false instead of an explicit exit(EXIT_FAILURE) seems to be cleaner.
This PR attempts to make things a bit more consistent.
There is a subtle difference between exit() and return from main in that the exit() will not clean up any local vars but I don't think this makes a difference in this case. Using exit() might even lead to bugs in the future where the dtor of local objects are expected to be called.
Tree-SHA512: 7d104c3a752b4e7d7bc2382ef7e62543462988f1bbf13dd4077fbeff5399729b76c71a4352556f188b8d306604232477466f5bb827b58a6f3f6273f2370e1faa
659b206 Make listsinceblock refuse unknown block hash (Russell Yanofsky)
Pull request description:
Change suggested by @theuni who noticed listsinceblock would ignore invalid block hashes causing it to return a completely unfiltered list of transactions.
Tree-SHA512: 3c8fb160265780d1334e856e853ab48e2e18372b8f1fc71ae480c3f45317048cc1fee0055d5c58031981a91b9c2bdbeb8e49a889d04ecba61729ce8109f2ce3f
60b98f8 [Util] Update tinyformat.h (fanquake)
Pull request description:
Updates `tinyformat.h` to commit c42f/tinyformat@689695c upstream. Including:
8a2812d8485d9e05a34748e2e48789
@achow101 mentioned that since upgrading to Ubuntu 17.10 (GCC 7), tinyformat had been throwing lots of -Wimplicit-fallthrough warnings. However fallthrough warnings should have been silenced by #10489. cc @theuni.
The upstream commit to fix fallthrough warnings is in this PR https://github.com/c42f/tinyformat/pull/39.
The last time tinyformat.h was updated in this repo was in #8274.
Tree-SHA512: a51bd30544693550e08148daf5d244e3a3a410caff7897351eb9cd28f661dc85e193e045bb86068ee4006b2f89a7233b7573b8c50d93d2a9a15a11386fdcc605
This tracks the set of all known invalid-themselves blocks (ie
blocks which we attempted to connect but which were found to be
invalid). This is used to cheaply check if new headers build on an
invalid chain.
While we're at it we also resolve an edge-case in invalidateblock
on pruned nodes which results in them needing a reindex if they
fail to reorg.
This is a simple change that makes our accept requirements the
same as our request requirements, (ever so slightly) further
decoupling our consensus logic from our FindNextBlocksToDownload
logic in net_processing.
There is no reason to wish to store blocks on disk always just
because a peer is whitelisted. This appears to be a historical
quirk to avoid breaking things when the accept limits were added.
Reading the variable mapBlockIndex requires holding the mutex cs_main.
The new "Disconnect outbound peers relaying invalid headers" code
added in commit 37886d5e2f and merged
as part of #11568 two days ago did not lock cs_main prior to accessing
mapBlockIndex.
cc5c39d [Build] Add AM_OBJCXXFLAGS and QT_PIE_FLAGS to OBJCXXFLAGS to future-proof darwin targets (fanquake)
f8c6697 Fix automake warnings when running autogen.sh (Evan Klitzke)
Pull request description:
Adjusted @eklitzke's commit to completely remove GZIP_ENV.
Added a commit to address OBJCXXFLAGS.
Rebased on master.
Relevant info from @theuni & #11013 below.
--------
GZIP_ENV was indeed added for determinism, but gitian exports this as needed, so it's not really necessary. I'd rather just remove it.
The mm.o rule was added to support XCode 4.2's ancient version of automake. That's irrelevant now, so it makes sense to remove that too.
All darwin targets are PIE by default, so we don't technically need the flags, but I'd be more comfortable if we hooked up the OBJCXXFLAGS in case future ones are added.
--------
The second commit addresses the last point, but could probably use a better commit message.
These warnings are removed from autogen output:
```
Makefile.am:12: warning: user variable 'GZIP_ENV' defined here ...
/usr/local/Cellar/automake/1.15.1/share/automake-1.15/am/distdir.am: ... overrides Automake variable 'GZIP_ENV' defined here
src/Makefile.am: installing 'build-aux/depcomp'
src/Makefile.am:503: warning: user target '.mm.o' defined here ...
/usr/local/Cellar/automake/1.15.1/share/automake-1.15/am/depend2.am: ... overrides Automake target '.mm.o' defined here
```
Tree-SHA512: bd59df5f6d3aafe35d5e36925bfe61cc71e774583a0438d7dd946c9e7ecf6e59d42f90a58b8cfef0faa404c81050338ad4cefe721b4a949af881e73b6ab254d4
37886d5e2 Disconnect outbound peers relaying invalid headers (Suhas Daftuar)
4637f1852 moveonly: factor out headers processing into separate function (Suhas Daftuar)
Pull request description:
Alternate to #11446.
Disconnect outbound (non-manual) peers that serve us block headers that are already known to be invalid, but exempt compact block announcements from such disconnects.
We restrict disconnection to outbound peers that are using up an outbound connection slot, because we rely on those peers to give us connectivity to the honest network (our inbound peers are not chosen by us and hence could all be from an attacker/sybil). Maintaining connectivity to peers that serve us invalid headers is sometimes desirable, eg after a soft-fork, to protect unupgraded software from being partitioned off the honest network, so we prefer to only disconnect when necessary.
Compact block announcements are exempted from this logic to comply with BIP 152, which explicitly permits nodes to relay compact blocks before fully validating them.
Tree-SHA512: 3ea88e4ccc1184f292a85b17f800d401d2c3806fefc7ad5429d05d6872c53acfa5751e3df83ce6b9c0060ab289511ed70ae1323d140ccc5b12e3c8da6de49936
fd3a2f3 [tests] Add fuzz testing for BlockTransactions and BlockTransactionsRequest (practicalswift)
Pull request description:
The `BlockTransactions` deserialization code is reachable with tainted data via `ProcessMessage(…, "BLOCKTXN", vRecv [tainted], …)`.
The same thing applies to `BlockTransactionsRequest` which is reachable via `"GETBLOCKTXN"`.
Tree-SHA512: 64560ea344bc6145b940472f99866b808725745b060dedfb315be400bd94e55399f50b982149645bd7af7ed9935fd28751d7daf0d3f94a8e2ed3bc52e3325ffb
e065249 Add unit test for outbound peer eviction (Suhas Daftuar)
5a6d00c Permit disconnection of outbound peers on bad/slow chains (Suhas Daftuar)
c60fd71 Disconnecting from bad outbound peers in IBD (Suhas Daftuar)
Pull request description:
The first commit will disconnect an outbound peer that serves us a headers chain with insufficient work while we're in IBD.
The second commit introduces a way to disconnect outbound peers whose chains fall out of sync with ours:
For a given outbound peer, we check whether their best known block (which is known from the blocks they announce to us) has at least as much work as our tip. If it doesn't, we set a 20 minute timeout, and if we still haven't heard about a block with as much work as our tip had when we set the timeout, then we send a single getheaders message, and wait 2 more minutes. If after two minutes their best known block has insufficient work, we disconnect that peer.
We protect 4 of our outbound peers (who provide some "good" headers chains, ie a chain with at least as much work as our tip at some point) from being subject to this logic, to prevent excessive network topology changes as a result of this algorithm, while still ensuring that we have a reasonable number of nodes not known to be on bogus chains.
We also don't require our peers to be on the same chain as us, to prevent accidental partitioning of the network in the event of a chain split. Note that if our peers are ever on a more work chain than our tip, then we will download and validate it, and then either reorg to it, or learn of a consensus incompatibility with that peer and disconnect. This PR is designed to protect against peers that are on a less work chain which we may never try to download and validate.
Tree-SHA512: 2e0169a1dd8a7fb95980573ac4a201924bffdd724c19afcab5efcef076fdbe1f2cec7dc5f5d7e0a6327216f56d3828884f73642e00c8534b56ec2bb4c854a656
Currently we have no rotation of outbound peers. If an outbound peer
stops serving us blocks, or is on a consensus-incompatible chain with
less work than our tip (but otherwise valid headers), then we will never
disconnect that peer, even though that peer is using one of our 8
outbound connection slots. Because we rely on our outbound peers to
find an honest node in order to reach consensus, allowing an
incompatible peer to occupy one of those slots is undesirable,
particularly if it is possible for all such slots to be occupied by such
peers.
Protect against this by always checking to see if a peer's best known
block has less work than our tip, and if so, set a 20 minute timeout --
if the peer is still not known to have caught up to a chain with as much
work as ours after 20 minutes, then send a single getheaders message,
wait 2 more minutes, and if a better header hasn't been received by then,
disconnect that peer.
Note:
- we do not require that our peer sync to the same tip as ours, just an
equal or greater work tip. (Doing otherwise would risk partitioning the
network in the event of a chain split, and is also unnecessary.)
- we pick 4 of our outbound peers and do not subject them to this logic,
to be more conservative. We don't wish to permit temporary network
issues (or an attacker) to excessively disrupt network topology.
Change suggested by Cory Fields <cory-nospam-@coryfields.com> who noticed
listsinceblock would ignore invalid block hashes causing it to return a
completely unfiltered list of transactions.
6b1891e2c Add Sent and Received information to the debug menu peer list (Aaron Golliver)
8e4aa35ff move human-readable byte formatting to guiutil (Aaron Golliver)
Pull request description:
Makes the peer list display how much you've uploaded/downloaded from each peer.
Here's a screenshot ~~[outdated](https://i.imgur.com/MhPbItp.png)~~, [current](https://i.imgur.com/K1htrVv.png) of how it looks. You can now sort to see who are the peers you've uploaded the most too.
I also moved `RPCConsole::FormatBytes` to `guiutil::formatBytes` so I could use it in the peerlist
Tree-SHA512: 8845ef406e4cbe7f981879a78c063542ce90f50f45c8fa3514ba3e6e1164b4c70bb2093c4e1cac268aef0328b7b63545bc1dfa435c227f28fdb4cb0a596800f5
A peer could try to waste our resources by sending us unrequested blocks with
low work, eg to fill up our disk. Since
e2652002b6 we no longer request blocks until we
know we're on a chain with more than nMinimumChainWork (our anti-DoS
threshold), but we would still process unrequested blocks that had more work
than our tip. This commit fixes that behavior.
João Barbosa
Pieter Wuille
MarcoFalke
Wladimir J. van der Laan
practicalswift
Matt Corallo
Cory Fields
Suhas Daftuar
Jonas Schnelli
fanquake
Suhas Daftuar
Russell Yanofsky
Evan Klitzke
