Commit Graph

15423 Commits

Author SHA1 Message Date
Jack Grigg e0f210f4fc test: Fix pyflakes warnings 2021-08-17 18:50:54 +01:00
Homu 56b5f95897 Auto merge of #5274 - str4d:zip-239-prep-3, r=str4d
ZIP 239 preparations 3

Cherry-picked from the following upstream PRs:
- bitcoin/bitcoin#8080
- bitcoin/bitcoin#8082
- bitcoin/bitcoin#8126
- bitcoin/bitcoin#7910
  - This is the unsquashed version of bitcoin/bitcoin#8149
  - We take three cleanup commits to the protocol / `CInv` code.
- bitcoin/bitcoin#8822
- bitcoin/bitcoin#8880
  - Excluding the first commit (we don't have the comment it fixes yet).
- bitcoin/bitcoin#19322
2021-08-17 17:19:24 +00:00
Homu 88d8867381 Auto merge of #5278 - daira:boost-download-url, r=str4d
Update URL for Boost source download (from dl.bintray.com to boostorg.jfrog.io)

Co-authored-by: Taylor Hornby <taylor@electriccoin.co>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2021-08-17 10:22:25 +00:00
Daira Hopwood 49d4bccf51 Update URL for Boost source download (from dl.bintray.com to boostorg.jfrog.io).
Co-authored-by: Taylor Hornby <taylor@electriccoin.co>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2021-08-16 21:10:54 +01:00
Daira Hopwood 52ed61157f ZIP 339 support.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2021-08-13 21:09:35 +01:00
Jack Grigg 1e352c108c test: Print reject reason if RPC test block rejected instead of accepted 2021-08-13 16:49:46 +01:00
Suhas Daftuar d477208a28 Only use AddInventoryKnown for transactions
filterInventoryKnown is only used when relaying transactions,
so stop adding block hashes to the filter.

(cherry picked from commit bitcoin/bitcoin@383fc10ebb)

Zcash: Excludes comment change, as bitcoin/bitcoin#7129 has not been
backported  yet.
2021-08-13 16:49:46 +01:00
MarcoFalke b67c86abdb [qa] py2: Unfiddle strings into bytes explicitly
(cherry picked from commit bitcoin/bitcoin@faa41ee204)

Zcash: Excluding RPC tests or framework code we don't have.
2021-08-13 16:49:46 +01:00
Wladimir J. van der Laan d9cb6b89b0 test: Add basic test for `reject` code
Extend P2P test framework to make it possible to expect reject
codes for transactions and blocks.

(cherry picked from commit bitcoin/bitcoin@20411903d7)
2021-08-13 16:49:46 +01:00
Wladimir J. van der Laan 873c6c1295 net: Fix sent reject messages for blocks and transactions
Ever since we #5913 have been sending invalid reject messages
for transactions and blocks.

(cherry picked from commit bitcoin/bitcoin@9fc6ed6003)
2021-08-13 16:24:09 +01:00
Wladimir J. van der Laan 25e125a5d3 Move mempool rejections to new debug category
Move mempool rejections to debug category `mempoolrej`, to make it possible
to show them without enabling the entire category `mempool` which is
high volume.

(cherry picked from commit bitcoin/bitcoin@7f1f8f5edf)
2021-08-13 16:24:09 +01:00
Wladimir J. van der Laan 7c6f2c5da6 Add information to errors in ConnectBlock, CheckBlock
Add detailed state information to the errors, as it is no longer being
logged downstream.

Also add the state information to mempool rejection debug message in
ProcessMessages.

(cherry picked from commit bitcoin/bitcoin@66daed5e1b)
2021-08-13 16:24:09 +01:00
Wladimir J. van der Laan abdef34b33 Remove most logging from transaction validation
Remove unnecessary direct logging in CheckTransaction,
AcceptToMemoryPool, CheckTxInputs, CScriptCheck::operator()

All status information should be returned in the CValidationState.
Relevant debug information is also added to the CValidationState using
the recently introduced debug message.

Do keep the "BUG! PLEASE REPORT THIS! ConnectInputs failed against MANDATORY but not STANDARD flags"
error as it is meant to appear as bug in the log.

(cherry picked from commit bitcoin/bitcoin@6cab808272)
2021-08-13 16:24:09 +01:00
Wladimir J. van der Laan 3a5474e3f3 Add function to convert CValidationState to a human-readable message
It is necessary to be able to concisely log a validation state.
Convert CValidationState to a human-readable message for logging.

(cherry picked from commit 9003c7cdd88402986e48ec126258fc10d6742fc7)
2021-08-13 16:24:09 +01:00
Wladimir J. van der Laan aa97bd1c60 Introduce REJECT_INTERNAL codes for local AcceptToMempool errors
Add status codes specific to AcceptToMempool procession of transactions.
These can never happen due to block validation, and must never be sent
over the P2P network. Add assertions where appropriate.

(cherry picked from commit bitcoin/bitcoin@dc58258adf)
2021-08-13 16:24:09 +01:00
Wladimir J. van der Laan 36c70c01a1 Add debug message to CValidationState for optional extra information
Add a field `strDebugMessage` which can be passed to DoS or Invalid,
and queried using GetDebugMessage() to add extra troubleshooting
information to the validation state.

(cherry picked from commit bitcoin/bitcoin@fbf44e6f3e)
2021-08-13 16:24:09 +01:00
Wladimir J. van der Laan e65733e032 Add assertion and cast before sending reject code
This gets rid of a warning. Add an assertion to make sure that the
reject code is in the correct range for the network protocol
(if it is outside the range it must be a bug)

(cherry picked from commit bitcoin/bitcoin@5922b6774e)
2021-08-13 16:09:59 +01:00
Shaul Kfir 55d4948207 Add absurdly high fee message to validation state (for RPC propagation)
(cherry picked from commit bitcoin/bitcoin@a651403e09)
2021-08-13 16:09:59 +01:00
John Newbery 8469683eb6 [net] split PushInventory()
PushInventory() is currently called with a CInv object, which can be a
MSG_TX or MSG_BLOCK. PushInventory() only uses the type to determine
whether to add the hash to setInventoryTxToSend or
vInventoryBlockToSend.

Since the caller always knows what type of inventory they're pushing,
the CInv is wastefully constructed and thrown away, and tx/block relay
is being split out, we split the function into PushTxInventory() and
PushBlockInventory().

(cherry picked from commit bitcoin/bitcoin@f52d403b81)

Zcash: Adjusted to account for bitcoin/bitcoin#15759 not having been
backported.
2021-08-13 15:42:04 +01:00
Wladimir J. van der Laan 85a87aa679 protocol.h: Make enums in GetDataMsg concrete values
This concretizes the numbers and adds a comment to make it clear that
these numbers are fixed by the protocol, and may avoid people forgetting
to claim numbers in the future (e.g. issue #8500).

Also gets rid of a weird unused `MSG_TYPE_MAX` in the middle of the
enumeration (thanks @paveljanik for noticing).

(cherry picked from commit bitcoin/bitcoin@1df311118d)
2021-08-13 15:42:04 +01:00
Wladimir J. van der Laan 68453f5b15 protocol.h: Move MESSAGE_START_SIZE into CMessageHeader
Also move the enum to the top, and remove a deceptive TODO
comment.

(cherry picked from commit bitcoin/bitcoin@2c09a5209a)
2021-08-13 15:42:04 +01:00
Wladimir J. van der Laan 56115ccc3c net: Hardcode protocol sizes and use fixed-size types
The P2P network uses a fixed protocol, these sizes shouldn't change
based on what happens to be the architecture.

(cherry picked from commit bitcoin/bitcoin@305087bdf6)
2021-08-13 15:42:04 +01:00
Wladimir J. van der Laan e5bef4962b net: Consistent checksum handling
In principle, the checksums of P2P packets are simply 4-byte blobs which
are the first four bytes of SHA256(SHA256(payload)).

Currently they are handled as little-endian 32-bit integers half of the
time, as blobs the other half, sometimes copying the one to the other,
resulting in somewhat confused code.

This PR changes the handling to be consistent both at packet creation
and receiving, making it (I think) easier to understand.

(cherry picked from commit bitcoin/bitcoin@41e58faf04)
2021-08-13 15:42:04 +01:00
Alex Morcos 6d8962b055 tidy up CInv::GetCommand
(cherry picked from commit sipa/bitcoin@97d7402c2c)
2021-08-13 15:42:04 +01:00
Eric Lombrozo 37aa712f0e getdata enum issue fix
(cherry picked from commit sipa/bitcoin@adb1c09b87)
2021-08-13 15:42:04 +01:00
Eric Lombrozo cf7e89bb8e Removed ppszTypeName from protocol.cpp
(cherry picked from commit sipa/bitcoin@e35d0200eb)
2021-08-13 15:42:04 +01:00
Pieter Wuille d1f6072841 Get rid of CTxMempool::lookup() entirely
(cherry picked from commit bitcoin/bitcoin@288d85ddf2)
2021-08-13 15:42:04 +01:00
Pieter Wuille 53b0d2b1fa Optimization: use usec in expiration and reuse nNow
(cherry picked from commit bitcoin/bitcoin@c2a4724642)
2021-08-13 15:42:04 +01:00
Pieter Wuille 138c768542 Optimization: don't check the mempool at all if no mempool req ever
(cherry picked from commit bitcoin/bitcoin@e9b4780b29)
2021-08-13 15:42:04 +01:00
Pieter Wuille b9fdf44d43 Optimize the relay map to use shared_ptr's
* Switch mapRelay to use shared_ptr<CTransaction>
* Switch the relay code to copy mempool shared_ptr's, rather than copying
  the transaction itself.
* Change vRelayExpiration to store mapRelay iterators rather than hashes
  (smaller and faster).

(cherry picked from commit bitcoin/bitcoin@dbfb426b96)
2021-08-13 15:42:04 +01:00
Pieter Wuille e469547f3d Switch CTransaction storage in mempool to std::shared_ptr
(cherry picked from commit bitcoin/bitcoin@8d39d7a2cf)
2021-08-13 15:42:04 +01:00
Pieter Wuille e863fdaebb Add support for unique_ptr and shared_ptr to memusage
(cherry picked from commit bitcoin/bitcoin@1b9e6d3c1a)
2021-08-13 03:58:03 +01:00
Gregory Maxwell 48ead0bea1 Defer inserting into maprelay until just before relaying.
This reduces the rate of not founds by better matching the far
 end expectations, it also improves privacy by removing the
 ability to use getdata to probe for a node having a txn before
 it has been relayed.

(cherry picked from commit bitcoin/bitcoin@4d8993b346)
2021-08-13 03:58:03 +01:00
Gregory Maxwell df5875048c Do not use mempool for GETDATA for tx accepted after the last mempool req.
The ability to GETDATA a transaction which has not (yet) been relayed
 is a privacy loss vector.

The use of the mempool for this was added as part of the mempool p2p
 message and is only needed to fetch transactions returned by it.

(cherry picked from commit bitcoin/bitcoin@7e908c7b82)
2021-08-13 03:58:03 +01:00
Jack Grigg 911dcc372e ProcessGetData(): Rework IsExpiringSoon check
This brings the code more closely in line with upstream, and prepares
for subsequent performance improvements.
2021-08-13 03:58:03 +01:00
Homu fd462fd8c4 Auto merge of #5269 - str4d:zip-239-prep-2, r=str4d
ZIP 239 preparations 2

Cherry-picked from the following upstream PRs:
- bitcoin/bitcoin#6722
  - Only the ancillary commits, not the mempool limiting commits (we have our own).
- bitcoin/bitcoin#6898
  - Only the first three commits (we'll cherry-pick the main content later).
- bitcoin/bitcoin#7840
2021-08-13 01:13:18 +00:00
Jack Grigg d6dd3cea67 qa: Bump `sync_mempool` timeout for `prioritisetransaction.py`
To try and get around transient CI failures.
2021-08-13 02:12:35 +01:00
str4d fe7abcad14
Adjust code comments to remove topological-sort references
There's no point in renaming everything to reflect us not backporting the PR from upstream, but we may as well make the code comments make sense.

Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2021-08-13 00:44:23 +01:00
Homu 5b194067ea Auto merge of #5271 - nuttycom:backport/13825_kill_accounts, r=str4d
[Backport] bitcoin/bitcoin#13825 kill accounts

This is a backport of the following commits:

- from bitcoin/bitcoin#13825
- pick bitcoin/bitcoin@c9c32e6b84
  - from bitcoin/bitcoin#14023
    - pick bitcoin/bitcoin@f0dc850bf6
      - from bitcoin/bitcoin#13566
        - pick bitcoin/bitcoin@702ae1e21a
        - pick bitcoin/bitcoin@cf15761f6d
        - pick bitcoin/bitcoin@0f3d6e9ab7
        - pick bitcoin/bitcoin@7110c830f8
        - pick bitcoin/bitcoin@ef7bc8893c
        - pick bitcoin/bitcoin@4279da4785
    - pick bitcoin/bitcoin@c410f41575
    - from bitcoin/bitcoin#9614
      - pick bitcoin/bitcoin@02d9f50d5f
      - pick bitcoin/bitcoin@82b7dc373a
        - from bitcoin/bitcoin#8061
          - pick bitcoin/bitcoin@ecb9741ec3
  - from bitcoin/bitcoin#6851
    - pick bitcoin/bitcoin@3e7c89196c
  - from bitcoin/bitcoin#8828
    - pick bitcoin/bitcoin@86029e72c9
      - from bitcoin/bitcoin#8696
        - just the first two commits, as AccountingEntry is going away
        - pick bitcoin/bitcoin@d2e678d7d2
        - pick bitcoin/bitcoin@59adc86680
    - from bitcoin/bitcoin#8028
      - pick bitcoin/bitcoin@0fd599767d
2021-08-12 14:13:55 +00:00
Kris Nuttycombe df51fc8761 Remove unused account-related wallet methods. 2021-08-12 08:07:12 -06:00
John Newbery 8af7e138ac [wallet] Kill accounts
This commit does the following changes:

- [wallet] Remove 'account' argument from GetLegacyBalance()
  - GetLegacyBalance() is never called with an account argument.
    Remove the argument and helper functions.
- [wallet] Remove CWallet::ListAccountCreditDebit()
  - Function no longer used.
- [wallet] Remove AccountMove()
  - Function no longer used.
- [wallet] Remove AddAccountingEntry()
  - Function no longer used.
- [wallet] Remove GetAccountCreditDebit()
  - Function no longer used.
- [wallet] Don't rewrite accounting entries when reordering wallet transactions.
 - Accounting entries are deprecated. Don't rewrite them to the wallet
   database when re-ordering transactions.
- [wallet] Remove WriteAccountingEntry()
  - Function no longer used.
- [wallet] Don't read acentry key-values from wallet on load.
- [wallet] Remove ListAccountCreditDebit()
  - Function no longer used.
- [wallet] Remove CAccountingEntry class
  - No longer used
- [wallet] Remove GetLabelDestination
  - Function no longer used.
- [wallet] Delete unused account functions
  - ReadAccount
  - WriteAccount
  - EraseAccount
  - DeleteLabel
- [wallet] Remove fromAccount argument from CommitTransaction()
- [wallet] Remove strFromAccount.
  - No longer used.
- [wallet] Remove strSentAccount from GetAmounts().
  - No longer used.
- [wallet] Update zapwallettxes comment to remove accounts.
- [wallet] Remove CAccount
  - No longer used
- [docs] fix typo in release notes for PR 14023

(cherry picked from commit c9c32e6b844fc79467b7e24c6c916142a0d08484)
2021-08-12 08:07:12 -06:00
John Newbery 3ef50b3647 [wallet] Remove wallet account RPCs
Also remove the RPC deprecation tests for accounts, and make one small
change to another wallet test that relies on account behaviour.

(cherry picked from commit f0dc850bf698f7377797d7d68365d4fc79b0221c)
2021-08-12 08:07:12 -06:00
John Newbery 0bf8b56b29 [RPC] [wallet] allow getbalance to use min_conf and watch_only without accounts.
(cherry picked from commit 702ae1e21a09d8c31406839c4ea507c5fa276898)
2021-08-11 19:36:14 -06:00
Gregory Maxwell 6fdc4def4a Move bloom and feerate filtering to just prior to tx sending.
This will avoid sending more pointless INVs around updates, and
 prevents using filter updates to timetag transactions.

Also adds locking for fRelayTxes.

(cherry picked from commit b5599147533103efea896a1fc4ff51f2d3ad5808)
2021-08-12 00:37:51 +01:00
Pieter Wuille 2b13aff251 Return mempool queries in dependency order
(cherry picked from commit 4578215e7f787968c1d6478e6df75499bd36dd8d)
2021-08-12 00:37:51 +01:00
Pieter Wuille 86f8b51096 Handle mempool requests in send loop, subject to trickle
By eliminating queued entries from the mempool response and responding only at
trickle time, this makes the mempool no longer leak transaction arrival order
information (as the mempool itself is also sorted)-- at least no more than
relay itself leaks it.

(cherry picked from commit ed7068302c7490e8061cb3a558a0f83a465beeea)
2021-08-12 00:37:51 +01:00
Pieter Wuille 709c5163d4 Split up and optimize transaction and block inv queues
(cherry picked from commit dc13dcd2bec2613a1cd5e0395b09b449d176146f)
2021-08-12 00:37:51 +01:00
Gregory Maxwell 2a7ebb3aba Eliminate TX trickle bypass, sort TX invs for privacy and priority.
Previously Bitcoin would send 1/4 of transactions out to all peers
 instantly.  This causes high overhead because it makes >80% of
 INVs size 1.  Doing so harms privacy, because it limits the
 amount of source obscurity a transaction can receive.

These randomized broadcasts also disobeyed transaction dependencies
 and required use of the orphan pool.  Because the orphan pool is
 so small this leads to poor propagation for dependent transactions.

When the bypass wasn't in effect, transactions were sent in the
 order they were received.  This avoided creating orphans but
 undermines privacy fairly significantly.

This commit:
 Eliminates the bypass. The bypass is replaced by halving the
  average delay for outbound peers.

 Sorts candidate transactions for INV by their topological
  depth then by their feerate (then hash); removing the
  information leakage and providing priority service to
  higher fee transactions.

 Limits the amount of transactions sent in a single INV to
  7tx/sec (and twice that for outbound); this limits the
  harm of low fee transaction floods, gives faster relay
  service to higher fee transactions. The 7 sounds lower
  than it really is because received advertisements need
  not be sent, and because the aggregate rate is multipled
  by the number of peers.

(cherry picked from commit f2d3ba73860e875972738d1da1507124d0971ae5)

Zcash: Candidate transactions for INV are not sorted by their
topological depth because we haven't backported bitcoin/bitcoin#6654.
2021-08-12 00:37:51 +01:00
John Newbery 5dd21ca625 [wallet] GetBalance can take a min_depth argument.
(cherry picked from commit cf15761f6d4526d205126fbf5f088ac8edebeb57)
2021-08-11 17:07:30 -06:00
John Newbery 55f95b12d7 [wallet] factor out GetAvailableWatchOnlyBalance()
(cherry picked from commit 0f3d6e9ab75837ead3acfda342aa3ea404efb81d)
2021-08-11 17:07:30 -06:00