Commit Graph

15360 Commits

Author SHA1 Message Date
Alex Morcos bd2c35a93f Add a score index to the mempool.
The score index is meant to represent the order of priority for being included in a block for miners.  Initially this is set to the transactions modified (by any feeDelta) fee rate.  Index improvements and unit tests by sdaftuar.

(cherry picked from commit f3fe83673e84ef4d20b3026faa397cad17212ff8)

Zcash: Also includes some small refactors from bitcoin/bitcoin#6654 which
we have not backported.
2021-08-10 05:09:51 +01:00
Alex Morcos dde99ca891 Store the total sig op count of a tx.
Store sum of legacy and P2SH sig op counts.  This is calculated in AcceptToMemory pool and storing it saves redoing the expensive calculation in block template creation.

(cherry picked from commit c49d5bc9e6c97c47c0bd78604b2c393a7e4af097)
2021-08-10 05:09:51 +01:00
Matt Corallo af34647572 Print mempool size in KB when adding txn
(cherry picked from commit e6c7b362ab8915e2aac167fa519bd29836d482af)
2021-08-10 05:09:51 +01:00
Matt Corallo 24cfc4337d Add CFeeRate += operator
(cherry picked from commit 241d6078ba26db4d3a36227d3275be2ee34625a6)
2021-08-10 05:09:51 +01:00
Matt Corallo 983d802e4f Track (and define) ::minRelayTxFee in CTxMemPool
(cherry picked from commit e8bcdce8a245af235f8be9853c8f81c9bda56412)
2021-08-10 05:09:51 +01:00
Matt Corallo 6fcc14cd7a Fix calling mempool directly, instead of pool, in ATMP
(cherry picked from commit 9c9b66f771ad904cd665f7f5f68e3279ebb2fa7e)
2021-08-10 05:09:51 +01:00
Suhas Daftuar fc752f2c67 Reverse the sort on the mempool's feerate index
(cherry picked from commit 78b82f4a16d8aad15ef397b1a1cd075b2efc8c16)
2021-08-10 05:09:51 +01:00
Homu 1cb1ed2653 Auto merge of #5265 - str4d:zip-239-prep-1, r=str4d
ZIP 239 preparations 1

This is the first of several backports to prepare for ZIP 239. The primary
change is altering `mapRelay` to store `CTransaction`s, which we need
because ZIP 239 requires changing `Inv` messages based on transaction
versions. The other changes are mainly for conflict removal but are also
independently useful.

Backports the following upstream PRs:
- bitcoin/bitcoin#6889
- bitcoin/bitcoin#7125
- bitcoin/bitcoin#7862
- bitcoin/bitcoin#7877
2021-08-10 00:22:32 +00:00
Homu f721e3b067 Auto merge of #5268 - str4d:optimise-reindex, r=str4d
Optimise reindex

Backports https://github.com/bitcoin/bitcoin/pull/7917:

> Several changes:
>
> * Make reindex/import use `AcceptBlock` rather than `ProcessNewBlock`,
>   so activation of the resulting best chain is delayed.
>
> * That delayed activation is handled by the existing "Activating best
>   chain" step in the startup process, which is moved to `ThreadImport`.
>
> * Optimize `ActivateBestChain` to not walk the entire chain to find the
>   most work chain to connect for each block (O(n^2) -> O(n)).
>
> This has several advantages:
>
> * As the actual activation is done after all blocks have been added
>   back to the index, it gets to use the checkpoints information (better
>   progress indicator, skipping of signature checks, ...).
>
> * Having a block index with many unactivated blocks (for example, after
>   deleting the chainstate directory) becomes _much_ faster
>
> * That case also runs in the background now (as it's simply treated as
>   part of the importing process).
>
> Downsides:
>
> * All blocks are read twice from disk during reindex (once to rebuild
>   the index, once to activate).
2021-08-09 22:54:27 +00:00
Jack Grigg 01f66cd00a test: Flush wallet in WriteCryptedSaplingZkeyDirectToDb before reopening
Hopefully this should reliably prevent the race condition we see in CI.
2021-08-09 23:53:14 +01:00
Jack Grigg c173a26a2d Revert "Remove reference to -reindex-chainstate"
This reverts commit f42cde3c30.
2021-08-09 20:58:49 +01:00
Pieter Wuille d3ccecd0b9 Report reindexing progress in GUI
(cherry picked from commit b4d24e142e25a21c78ab74146c3520f2259fd7c2)

Zcash: Excludes GUI changes (as we don't have the QT GUI).
2021-08-09 20:58:49 +01:00
Pieter Wuille d6d39449e5 Add -reindex-chainstate that does not rebuild block index
(cherry picked from commit d3d75479115bc3480f163df774ee9dd2f8bd9f54)
2021-08-09 20:58:49 +01:00
Pieter Wuille dcd0ba8059 Optimize ActivateBestChain for long chains
(cherry picked from commit fb8fad1586ced69fa37c665a11916ae4c4d0df05)
2021-08-09 20:58:49 +01:00
Pieter Wuille e967b56fbe Switch reindexing to AcceptBlock in-loop and ActivateBestChain afterwards
(cherry picked from commit 316623f2c197971db9b5bcb9c84e446254b552c3)
2021-08-09 20:58:49 +01:00
Pieter Wuille eb0df5d324 Make ProcessNewBlock dbp const and update comment
(cherry picked from commit d253ec4baa21cc292cf72d453f71b4043b53e591)
2021-08-09 20:58:49 +01:00
Jack Grigg 75b9fc4f27 test: Fix race condition in p2p_txexpiringsoon
The recent changes to mempool inv logic mean that nodes are much less
likely to immediately return an `inv` message in response to a `mempool`
message. The `p2p_txexpiringsoon` RPC test was relying on the prior
behaviour.

`TestNode.sync_with_ping` now takes an optional `waiting_for` closure
that allows the caller to require that a specific message kind is
received prior to the timeout.
2021-08-09 17:43:48 +01:00
Jack Grigg 6be1a2a9da test: Wait for transaction propagation in shorter_block_times RPC test
This was previously a transient test failure, that started reliably
failing after the move to Poisson delays.
2021-08-05 20:54:47 +01:00
Daira Hopwood 465c2492e7
More precise terminology: "lock free" -> "unlocked" 2021-08-05 14:19:51 +01:00
Pieter Wuille ad8abd68a1 Change mapRelay to store CTransactions
(cherry picked from commit 38c310299cfef419d42744362b90c1700b598953)
2021-08-04 17:12:36 +01:00
Suhas Daftuar 44787c0179 Use txid as key in mapAlreadyAskedFor
Previously we used the CInv that would be sent to the peer announcing the
transaction as the key, but using the txid instead allows us to decouple the
p2p layer from the application logic (which relies on this map to avoid
duplicate tx requests).

(cherry picked from commit 7e91f632c70ff1848a152f24ee67a06796803943)
2021-08-04 17:12:36 +01:00
Pieter Wuille 13d8f294ac Replace trickle nodes with per-node/message Poisson delays
We used to have a trickle node, a node which was chosen in each iteration of
the send loop that was privileged and allowed to send out queued up non-time
critical messages. Since the removal of the fixed sleeps in the network code,
this resulted in fast and attackable treatment of such broadcasts.

This pull request changes the 3 remaining trickle use cases by random delays:
* Local address broadcast (while also removing the the wiping of the seen filter)
* Address relay
* Inv relay (for transactions; blocks are always relayed immediately)

The code is based on older commits by Patrick Strateman.

(cherry picked from commit 5400ef6bcb9d243b2b21697775aa6491115420f3)
2021-08-04 17:12:25 +01:00
Jonas Schnelli ae9768c8b7 fix locking issue with new mempool limiting
Current master crashes on OSX with an exception: "boost: mutex lock failed in pthread_mutex_lock: Invalid argument"

(cherry picked from commit 0d699fc821048ab9316b0004e6552c8f1dc5e5f4)

Zcash: Also adds the `clear` call that this was fixing. Upstream added it
in https://github.com/bitcoin/bitcoin/pull/6722 which we never backported
(instead implementing our own mempool limiting logic).
2021-08-04 17:12:25 +01:00
Homu 2c90f7ea9d Auto merge of #5247 - nuttycom:consensus/action_flag_consistency, r=str4d
Add check for consistency between nActionsOrchard and Orchard flags.

Fixes #5245
2021-08-04 10:48:33 +00:00
Homu b67ffbb958 Auto merge of #5262 - str4d:bump-deps-4.5.0, r=str4d
Update dependencies

We now build with Rust 1.54 and Clang / libc++ 12.
2021-07-31 01:15:52 +00:00
str4d bdaf3e1ab2 depends: Greatly simplify the Clang 12 patch
We can avoid the "%s" optimization with "%s%s" :D

Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2021-07-30 19:53:18 +01:00
Jack Grigg 5bb339824d cargo fmt 2021-07-30 19:08:18 +01:00
Jack Grigg 73a33efa43 CI: Use Rust 1.54 for lints 2021-07-30 18:36:42 +01:00
Jack Grigg 5ace59b38f rust: metrics 0.17 2021-07-30 18:22:05 +01:00
Jack Grigg 6e2a5b3f92 rust: cargo update 2021-07-30 18:22:05 +01:00
Jack Grigg 18e39ff9f0 depends: Fix issue cross-compiling BDB to Windows with Clang 12 2021-07-30 18:22:05 +01:00
Jack Grigg d9bcc2d303 depends: Update utfcpp to 3.2.1 2021-07-30 18:22:05 +01:00
Jack Grigg 374f60b326 depends: Update Clang / libcxx to LLVM 12
This is the second release in a row where LLVM has cut a X.0.1 for
everything except Darwin, so I've adjusted its URLs and paths on the
assumption this will continue.
2021-07-30 18:22:05 +01:00
Jack Grigg 0a5e69ab67 depends: Update Rust to 1.54.0 2021-07-30 18:22:05 +01:00
Kris Nuttycombe 660686d412 Fix error strings to correctly reflect context. 2021-07-30 11:16:22 -06:00
Kris Nuttycombe 8a904c0a8b
Use DOS level 100 for noncontextual checks.
Co-authored-by: str4d <jack@electriccoin.co>
2021-07-30 11:14:28 -06:00
Jack Grigg 1eb848ada2 Postpone dependency updates we aren't doing in this release 2021-07-30 16:04:31 +01:00
Homu 82a50f6c53 Auto merge of #5252 - str4d:ua-encoding, r=str4d
Unified Address encoding and decoding

Closes zcash/zcash#5133.
2021-07-30 00:40:08 +00:00
str4d 037bfa32f3 Track lengths when copying receiver data from C++ to Rust
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2021-07-30 01:38:38 +01:00
Jack Grigg 635151bf75 Add constants for UA typecodes 2021-07-30 01:25:14 +01:00
Kris Nuttycombe f4fe590eea Make Sapling Spend and Ouput count, and Orchard Action count checks be noncontextual.
Sapling spend and output max element counts may be safely made
noncontextual because the existing transaction size limit checks
would be violated by transactions containing more than 2^16 such
elements.
2021-07-29 08:32:46 -06:00
Homu 81da4f92ab Auto merge of #5241 - nuttycom:consensus/check_orchard_root_bottom, r=daira
Consensus: Add check to ensure that the Orchard commitment tree is not overfull after AppendBundle.

Part of #5197.
2021-07-29 13:20:34 +00:00
Daira Hopwood ec4e9ae509
Improve error message when a block would overfill the Orchard commitment tree.
Co-authored-by: str4d <jack@electriccoin.co>
2021-07-29 14:17:35 +01:00
Jack Grigg 8e6927f901 Document why a nested call to ExtractMinerAddress is not recursive 2021-07-19 12:48:11 +01:00
str4d 5675a4eed4
Fix typo in method documentation
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2021-07-19 12:44:05 +01:00
Homu 5802be3427 Auto merge of #5254 - zcash:ci-lints, r=str4d
CI: Add workflow that runs lints
2021-07-16 21:29:23 +00:00
Jack Grigg 4026386cac CI: Add Rust lints 2021-07-14 23:11:02 +01:00
Jack Grigg b54b416d68 CI: Check scripted diffs 2021-07-14 22:58:05 +01:00
Jack Grigg 29280b9821 CI: Add workflow that runs general lints 2021-07-14 22:19:40 +01:00
Jack Grigg 612f250814 Pass network type through to UA address handling logic 2021-07-13 13:40:13 +01:00