Commit Graph

15391 Commits

Author SHA1 Message Date
Gregory Maxwell 48ead0bea1 Defer inserting into maprelay until just before relaying.
This reduces the rate of not founds by better matching the far
 end expectations, it also improves privacy by removing the
 ability to use getdata to probe for a node having a txn before
 it has been relayed.

(cherry picked from commit bitcoin/bitcoin@4d8993b346)
2021-08-13 03:58:03 +01:00
Gregory Maxwell df5875048c Do not use mempool for GETDATA for tx accepted after the last mempool req.
The ability to GETDATA a transaction which has not (yet) been relayed
 is a privacy loss vector.

The use of the mempool for this was added as part of the mempool p2p
 message and is only needed to fetch transactions returned by it.

(cherry picked from commit bitcoin/bitcoin@7e908c7b82)
2021-08-13 03:58:03 +01:00
Jack Grigg 911dcc372e ProcessGetData(): Rework IsExpiringSoon check
This brings the code more closely in line with upstream, and prepares
for subsequent performance improvements.
2021-08-13 03:58:03 +01:00
Homu fd462fd8c4 Auto merge of #5269 - str4d:zip-239-prep-2, r=str4d
ZIP 239 preparations 2

Cherry-picked from the following upstream PRs:
- bitcoin/bitcoin#6722
  - Only the ancillary commits, not the mempool limiting commits (we have our own).
- bitcoin/bitcoin#6898
  - Only the first three commits (we'll cherry-pick the main content later).
- bitcoin/bitcoin#7840
2021-08-13 01:13:18 +00:00
Jack Grigg d6dd3cea67 qa: Bump `sync_mempool` timeout for `prioritisetransaction.py`
To try and get around transient CI failures.
2021-08-13 02:12:35 +01:00
str4d fe7abcad14
Adjust code comments to remove topological-sort references
There's no point in renaming everything to reflect us not backporting the PR from upstream, but we may as well make the code comments make sense.

Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2021-08-13 00:44:23 +01:00
Homu 5b194067ea Auto merge of #5271 - nuttycom:backport/13825_kill_accounts, r=str4d
[Backport] bitcoin/bitcoin#13825 kill accounts

This is a backport of the following commits:

- from bitcoin/bitcoin#13825
- pick bitcoin/bitcoin@c9c32e6b84
  - from bitcoin/bitcoin#14023
    - pick bitcoin/bitcoin@f0dc850bf6
      - from bitcoin/bitcoin#13566
        - pick bitcoin/bitcoin@702ae1e21a
        - pick bitcoin/bitcoin@cf15761f6d
        - pick bitcoin/bitcoin@0f3d6e9ab7
        - pick bitcoin/bitcoin@7110c830f8
        - pick bitcoin/bitcoin@ef7bc8893c
        - pick bitcoin/bitcoin@4279da4785
    - pick bitcoin/bitcoin@c410f41575
    - from bitcoin/bitcoin#9614
      - pick bitcoin/bitcoin@02d9f50d5f
      - pick bitcoin/bitcoin@82b7dc373a
        - from bitcoin/bitcoin#8061
          - pick bitcoin/bitcoin@ecb9741ec3
  - from bitcoin/bitcoin#6851
    - pick bitcoin/bitcoin@3e7c89196c
  - from bitcoin/bitcoin#8828
    - pick bitcoin/bitcoin@86029e72c9
      - from bitcoin/bitcoin#8696
        - just the first two commits, as AccountingEntry is going away
        - pick bitcoin/bitcoin@d2e678d7d2
        - pick bitcoin/bitcoin@59adc86680
    - from bitcoin/bitcoin#8028
      - pick bitcoin/bitcoin@0fd599767d
2021-08-12 14:13:55 +00:00
Kris Nuttycombe df51fc8761 Remove unused account-related wallet methods. 2021-08-12 08:07:12 -06:00
John Newbery 8af7e138ac [wallet] Kill accounts
This commit does the following changes:

- [wallet] Remove 'account' argument from GetLegacyBalance()
  - GetLegacyBalance() is never called with an account argument.
    Remove the argument and helper functions.
- [wallet] Remove CWallet::ListAccountCreditDebit()
  - Function no longer used.
- [wallet] Remove AccountMove()
  - Function no longer used.
- [wallet] Remove AddAccountingEntry()
  - Function no longer used.
- [wallet] Remove GetAccountCreditDebit()
  - Function no longer used.
- [wallet] Don't rewrite accounting entries when reordering wallet transactions.
 - Accounting entries are deprecated. Don't rewrite them to the wallet
   database when re-ordering transactions.
- [wallet] Remove WriteAccountingEntry()
  - Function no longer used.
- [wallet] Don't read acentry key-values from wallet on load.
- [wallet] Remove ListAccountCreditDebit()
  - Function no longer used.
- [wallet] Remove CAccountingEntry class
  - No longer used
- [wallet] Remove GetLabelDestination
  - Function no longer used.
- [wallet] Delete unused account functions
  - ReadAccount
  - WriteAccount
  - EraseAccount
  - DeleteLabel
- [wallet] Remove fromAccount argument from CommitTransaction()
- [wallet] Remove strFromAccount.
  - No longer used.
- [wallet] Remove strSentAccount from GetAmounts().
  - No longer used.
- [wallet] Update zapwallettxes comment to remove accounts.
- [wallet] Remove CAccount
  - No longer used
- [docs] fix typo in release notes for PR 14023

(cherry picked from commit c9c32e6b844fc79467b7e24c6c916142a0d08484)
2021-08-12 08:07:12 -06:00
John Newbery 3ef50b3647 [wallet] Remove wallet account RPCs
Also remove the RPC deprecation tests for accounts, and make one small
change to another wallet test that relies on account behaviour.

(cherry picked from commit f0dc850bf698f7377797d7d68365d4fc79b0221c)
2021-08-12 08:07:12 -06:00
John Newbery 0bf8b56b29 [RPC] [wallet] allow getbalance to use min_conf and watch_only without accounts.
(cherry picked from commit 702ae1e21a09d8c31406839c4ea507c5fa276898)
2021-08-11 19:36:14 -06:00
Gregory Maxwell 6fdc4def4a Move bloom and feerate filtering to just prior to tx sending.
This will avoid sending more pointless INVs around updates, and
 prevents using filter updates to timetag transactions.

Also adds locking for fRelayTxes.

(cherry picked from commit b5599147533103efea896a1fc4ff51f2d3ad5808)
2021-08-12 00:37:51 +01:00
Pieter Wuille 2b13aff251 Return mempool queries in dependency order
(cherry picked from commit 4578215e7f787968c1d6478e6df75499bd36dd8d)
2021-08-12 00:37:51 +01:00
Pieter Wuille 86f8b51096 Handle mempool requests in send loop, subject to trickle
By eliminating queued entries from the mempool response and responding only at
trickle time, this makes the mempool no longer leak transaction arrival order
information (as the mempool itself is also sorted)-- at least no more than
relay itself leaks it.

(cherry picked from commit ed7068302c7490e8061cb3a558a0f83a465beeea)
2021-08-12 00:37:51 +01:00
Pieter Wuille 709c5163d4 Split up and optimize transaction and block inv queues
(cherry picked from commit dc13dcd2bec2613a1cd5e0395b09b449d176146f)
2021-08-12 00:37:51 +01:00
Gregory Maxwell 2a7ebb3aba Eliminate TX trickle bypass, sort TX invs for privacy and priority.
Previously Bitcoin would send 1/4 of transactions out to all peers
 instantly.  This causes high overhead because it makes >80% of
 INVs size 1.  Doing so harms privacy, because it limits the
 amount of source obscurity a transaction can receive.

These randomized broadcasts also disobeyed transaction dependencies
 and required use of the orphan pool.  Because the orphan pool is
 so small this leads to poor propagation for dependent transactions.

When the bypass wasn't in effect, transactions were sent in the
 order they were received.  This avoided creating orphans but
 undermines privacy fairly significantly.

This commit:
 Eliminates the bypass. The bypass is replaced by halving the
  average delay for outbound peers.

 Sorts candidate transactions for INV by their topological
  depth then by their feerate (then hash); removing the
  information leakage and providing priority service to
  higher fee transactions.

 Limits the amount of transactions sent in a single INV to
  7tx/sec (and twice that for outbound); this limits the
  harm of low fee transaction floods, gives faster relay
  service to higher fee transactions. The 7 sounds lower
  than it really is because received advertisements need
  not be sent, and because the aggregate rate is multipled
  by the number of peers.

(cherry picked from commit f2d3ba73860e875972738d1da1507124d0971ae5)

Zcash: Candidate transactions for INV are not sorted by their
topological depth because we haven't backported bitcoin/bitcoin#6654.
2021-08-12 00:37:51 +01:00
John Newbery 5dd21ca625 [wallet] GetBalance can take a min_depth argument.
(cherry picked from commit cf15761f6d4526d205126fbf5f088ac8edebeb57)
2021-08-11 17:07:30 -06:00
John Newbery 55f95b12d7 [wallet] factor out GetAvailableWatchOnlyBalance()
(cherry picked from commit 0f3d6e9ab75837ead3acfda342aa3ea404efb81d)
2021-08-11 17:07:30 -06:00
John Newbery f1aecdb9b2 [wallet] deduplicate GetAvailableCredit logic
(cherry picked from commit 7110c830f8c5de3570178bf4e5d28fe3e4f109e1)
2021-08-11 17:07:30 -06:00
John Newbery 5fc98a3b34 [wallet] Factor out GetWatchOnlyBalance()
Zcash: removes GetWatchOnlyBalance (unused)

(cherry picked from commit ef7bc8893c7a953953aa457736d79c28a4f45792)
2021-08-11 17:07:30 -06:00
John Newbery f7c57ecca4 [wallet] GetBalance can take an isminefilter filter.
GetBalance() can now take an ismine filter, which is passed down to
GetAvailableCredit. This allows GetBalance to be used to get watch-only
balances.

(cherry picked from commit 4279da47855ec776f8d57c6579fe89afc9cbe8c1)
2021-08-11 17:07:30 -06:00
Russell Yanofsky 65c0c0b232 [wallet] Remove unneeded legacy getbalance code
(cherry picked from commit 02d9f50d5f3c96fe888c230d59c5afdab4c7c6a3)
2021-08-11 17:07:27 -06:00
Russell Yanofsky 35950132aa [wallet] Add GetLegacyBalance method to simplify getbalance RPC
This adds a simpler new implementation of getbalance logic along with asserts
to confirm it behaves identically to the old logic. The old logic is removed in
the next commit.

(cherry picked from commit 82b7dc373afff277d3083d09e68c6b69778f312f)
2021-08-10 14:19:15 -06:00
Patrick Strateman 6edda25d9c Move GetAccountBalance from rpcwallet.cpp into CWallet::GetAccountBalance
(cherry picked from commit ecb9741ec3067f67f595126869d0200c62064bbd)
2021-08-10 14:19:15 -06:00
John Newbery f03ce68a61 [tests] Remove wallet accounts test
The accounts API will be removed in the next commit. Remove all
functional tests for the accounts API.

(cherry picked from commit c410f415758913c933ad6c71cf50227cc85aa385)
2021-08-10 14:19:15 -06:00
Patrick Strateman 97023ea0b8 Move CWalletDB::ReorderTransactions to CWallet
(cherry picked from commit 86029e72c96436605f47153c1f09e151e1847c28)
2021-08-10 14:19:15 -06:00
Luke Dashjr 31d49b09b7 Optimisation: Store transaction list order in memory rather than compute it every need
Huge performance improvement (450%) for zapwallettxes

(cherry picked from commit 3e7c89196ceb2742b62cd183d6fab74edc40647a)
2021-08-10 14:19:15 -06:00
Patrick Strateman a113057ab9 Add CWallet::ReorderTransactions and use in accounting_tests.cpp
(cherry picked from commit d2e678d7d220e2f430c2cb4bde559acee00c8c7c)
2021-08-10 14:19:15 -06:00
Patrick Strateman 9dc4e4b0aa Add CWallet::ListAccountCreditDebit
Simple pass through for CWalletDB::ListAccountCreditDebit

(cherry picked from commit 59adc86680911a83dcccf5b23db299a370fc92f2)
2021-08-10 14:19:15 -06:00
Patrick Strateman 33568f64bd Fix insanity of CWalletDB::WriteTx and CWalletTx::WriteToDisk
(cherry picked from commit 0fd599767d2dabf25ac8c3543cb586cfc4b9d816)
2021-08-10 14:19:15 -06:00
Alex Morcos 543c909166 Add TxPriority class and comparator
(cherry picked from commit 7230187b1d8bf39da5f1502e4c9eeb8e72dccdff)
2021-08-10 05:09:51 +01:00
Alex Morcos bd2c35a93f Add a score index to the mempool.
The score index is meant to represent the order of priority for being included in a block for miners.  Initially this is set to the transactions modified (by any feeDelta) fee rate.  Index improvements and unit tests by sdaftuar.

(cherry picked from commit f3fe83673e84ef4d20b3026faa397cad17212ff8)

Zcash: Also includes some small refactors from bitcoin/bitcoin#6654 which
we have not backported.
2021-08-10 05:09:51 +01:00
Alex Morcos dde99ca891 Store the total sig op count of a tx.
Store sum of legacy and P2SH sig op counts.  This is calculated in AcceptToMemory pool and storing it saves redoing the expensive calculation in block template creation.

(cherry picked from commit c49d5bc9e6c97c47c0bd78604b2c393a7e4af097)
2021-08-10 05:09:51 +01:00
Matt Corallo af34647572 Print mempool size in KB when adding txn
(cherry picked from commit e6c7b362ab8915e2aac167fa519bd29836d482af)
2021-08-10 05:09:51 +01:00
Matt Corallo 24cfc4337d Add CFeeRate += operator
(cherry picked from commit 241d6078ba26db4d3a36227d3275be2ee34625a6)
2021-08-10 05:09:51 +01:00
Matt Corallo 983d802e4f Track (and define) ::minRelayTxFee in CTxMemPool
(cherry picked from commit e8bcdce8a245af235f8be9853c8f81c9bda56412)
2021-08-10 05:09:51 +01:00
Matt Corallo 6fcc14cd7a Fix calling mempool directly, instead of pool, in ATMP
(cherry picked from commit 9c9b66f771ad904cd665f7f5f68e3279ebb2fa7e)
2021-08-10 05:09:51 +01:00
Suhas Daftuar fc752f2c67 Reverse the sort on the mempool's feerate index
(cherry picked from commit 78b82f4a16d8aad15ef397b1a1cd075b2efc8c16)
2021-08-10 05:09:51 +01:00
Homu 1cb1ed2653 Auto merge of #5265 - str4d:zip-239-prep-1, r=str4d
ZIP 239 preparations 1

This is the first of several backports to prepare for ZIP 239. The primary
change is altering `mapRelay` to store `CTransaction`s, which we need
because ZIP 239 requires changing `Inv` messages based on transaction
versions. The other changes are mainly for conflict removal but are also
independently useful.

Backports the following upstream PRs:
- bitcoin/bitcoin#6889
- bitcoin/bitcoin#7125
- bitcoin/bitcoin#7862
- bitcoin/bitcoin#7877
2021-08-10 00:22:32 +00:00
Homu f721e3b067 Auto merge of #5268 - str4d:optimise-reindex, r=str4d
Optimise reindex

Backports https://github.com/bitcoin/bitcoin/pull/7917:

> Several changes:
>
> * Make reindex/import use `AcceptBlock` rather than `ProcessNewBlock`,
>   so activation of the resulting best chain is delayed.
>
> * That delayed activation is handled by the existing "Activating best
>   chain" step in the startup process, which is moved to `ThreadImport`.
>
> * Optimize `ActivateBestChain` to not walk the entire chain to find the
>   most work chain to connect for each block (O(n^2) -> O(n)).
>
> This has several advantages:
>
> * As the actual activation is done after all blocks have been added
>   back to the index, it gets to use the checkpoints information (better
>   progress indicator, skipping of signature checks, ...).
>
> * Having a block index with many unactivated blocks (for example, after
>   deleting the chainstate directory) becomes _much_ faster
>
> * That case also runs in the background now (as it's simply treated as
>   part of the importing process).
>
> Downsides:
>
> * All blocks are read twice from disk during reindex (once to rebuild
>   the index, once to activate).
2021-08-09 22:54:27 +00:00
Jack Grigg 01f66cd00a test: Flush wallet in WriteCryptedSaplingZkeyDirectToDb before reopening
Hopefully this should reliably prevent the race condition we see in CI.
2021-08-09 23:53:14 +01:00
Jack Grigg c173a26a2d Revert "Remove reference to -reindex-chainstate"
This reverts commit f42cde3c30.
2021-08-09 20:58:49 +01:00
Pieter Wuille d3ccecd0b9 Report reindexing progress in GUI
(cherry picked from commit b4d24e142e25a21c78ab74146c3520f2259fd7c2)

Zcash: Excludes GUI changes (as we don't have the QT GUI).
2021-08-09 20:58:49 +01:00
Pieter Wuille d6d39449e5 Add -reindex-chainstate that does not rebuild block index
(cherry picked from commit d3d75479115bc3480f163df774ee9dd2f8bd9f54)
2021-08-09 20:58:49 +01:00
Pieter Wuille dcd0ba8059 Optimize ActivateBestChain for long chains
(cherry picked from commit fb8fad1586ced69fa37c665a11916ae4c4d0df05)
2021-08-09 20:58:49 +01:00
Pieter Wuille e967b56fbe Switch reindexing to AcceptBlock in-loop and ActivateBestChain afterwards
(cherry picked from commit 316623f2c197971db9b5bcb9c84e446254b552c3)
2021-08-09 20:58:49 +01:00
Pieter Wuille eb0df5d324 Make ProcessNewBlock dbp const and update comment
(cherry picked from commit d253ec4baa21cc292cf72d453f71b4043b53e591)
2021-08-09 20:58:49 +01:00
Jack Grigg 75b9fc4f27 test: Fix race condition in p2p_txexpiringsoon
The recent changes to mempool inv logic mean that nodes are much less
likely to immediately return an `inv` message in response to a `mempool`
message. The `p2p_txexpiringsoon` RPC test was relying on the prior
behaviour.

`TestNode.sync_with_ping` now takes an optional `waiting_for` closure
that allows the caller to require that a specific message kind is
received prior to the timeout.
2021-08-09 17:43:48 +01:00
Jack Grigg 6be1a2a9da test: Wait for transaction propagation in shorter_block_times RPC test
This was previously a transient test failure, that started reliably
failing after the move to Poisson delays.
2021-08-05 20:54:47 +01:00
Daira Hopwood 465c2492e7
More precise terminology: "lock free" -> "unlocked" 2021-08-05 14:19:51 +01:00