Commit Graph

3624 Commits

Author SHA1 Message Date
Brennan 2164a50d00
Move BankIncrementalSnapshotPersistence (#31236)
* Move BankIncrementalSnapshotPersistence

* Update bank serialize ABI digest
2023-04-18 11:18:17 -07:00
Xiang Zhu e74bc4e2e7
Add a filter_by_type param for purge_old_bank_snapshots (#31191)
* Add a type_select param for purge_old_bank_snapshots

* Use flags to make the function calls more readable

* Remove the extra purge calls

* replace select_type with filter_by_type

* Add test

* Use matches

* Fix CI test on reference

* use match and call do_purge once

* Let bank_snapshots_dir be TempDir

* remove account_paths in the test

* replace bank with _bank

* Remove create_snapshot_dirs_for_tests, will take the lastest from master

* Fix merge errors
2023-04-17 16:16:41 -07:00
Xiang Zhu 5747290d51
Move reference-holding last_snapshot_storages from ABS to AHV (#31175)
* Let AHV hold and update last_snapshot_storages

* Clean up comment

* Move cloning after enqueued_time

* Minor positon change

* Remove  type last_snapshot_storages annotation
2023-04-14 14:38:44 -07:00
Andrew Fitzgerald b657004141
store slot on BlockBatchUpdate (#31190) 2023-04-14 13:15:31 -07:00
Brooks d43e19bb03
Refactors the Full/Incremental SnapshotHash types (#31186) 2023-04-13 18:01:27 -04:00
Brooks 1f67591e21
Removes `base` from `IncrementalSnapshotHash` (#31185) 2023-04-13 17:35:35 +00:00
Brooks e05957d8fa
Push starting snapshot hashes in SnapshotGossipManager::new() (#31173) 2023-04-13 11:49:17 -04:00
Andrew Fitzgerald c847236147
decision maker perf (#30618) 2023-04-12 21:40:59 -07:00
Andrew Fitzgerald 01659edd16
Forwarder: forward_packets w/o metrics (#30925) 2023-04-12 14:09:24 -07:00
Brooks 602297e29f
Add comment in SolanaGossipMananger::update_latest_full_snapshot_hash() (#31171) 2023-04-12 16:52:23 +00:00
Brooks 1761c0947b
Removes unused arg from SnapshotGossipManager::new() (#31169) 2023-04-12 16:45:31 +00:00
Brooks 944b9d574a
Only push latest snapshot hashes in SnapshotGossipManager (#31154) 2023-04-12 11:00:26 -04:00
Brooks 965dd37924
Moves SnapshotGossipManager to its own file (#31147) 2023-04-11 15:51:52 -04:00
Brooks 453f272698
Rename IncrementalSnapshotHashes to SnapshotHashes (#31136) 2023-04-11 10:30:29 -04:00
Brooks f3083ad2e0
Rename SnapshotHashes to LegacySnapshotHashes (#31086) 2023-04-10 17:52:20 -04:00
behzad nouri ce21a58b65
reworks streamer::StakedNodes (#31082)
{min,max}_stake are computed but never assigned:
https://github.com/solana-labs/solana/blob/4564bcdc1/core/src/staked_nodes_updater_service.rs#L54-L57

The updater code is also inefficient and verbose.
2023-04-10 17:07:40 +00:00
Brooks f9276d1748
Uses MAX_ACCOUNTS_HASHES instead of MAX_SNAPSHOT_HASHES in accounts_hash_verifier.rs (#31114) 2023-04-10 10:44:40 -04:00
HaoranYi fcd1fe0959
Refactor fault hash injection into lambda (#31093)
* refactor out fault hash inject output AccountsHashVerifier

* refactor faught injector out of AccountHashVerifier

* use type alias

* Apply suggestions from code review

Co-authored-by: Brooks <brooks@prumo.org>

* move type alias

* rename

---------

Co-authored-by: Brooks <brooks@prumo.org>
2023-04-07 17:50:21 -05:00
Tao Zhu 9a7b6abc91
update benches after removing packet.sender_stake (#31110) 2023-04-07 14:27:29 -05:00
Andrew Fitzgerald 926bb0c794
MultiIteratorScanner::finalize returns (payload, already_processed) (#31054)
* finalize() returns (payload, already_processed)

* Additional testing around already_handled

* Return type wrapper and comment update
2023-04-07 11:17:36 -07:00
behzad nouri 466a9a2449
removes ip_stake_map field from streamer::StakedNodes (#31078) 2023-04-07 13:27:29 +00:00
Ryo Onodera f0432ec50f
Avoid overflow in ThreadSet::any() and nits (#31098)
Avoid overflow in ThreadSet::any and etc
2023-04-07 12:45:29 +09:00
behzad nouri 4d0abebe0e
removes Packet Meta.sender_stake and find_packet_sender_stake_stage (#31077)
Packet Meta.sender_stake is unused since
https://github.com/solana-labs/solana/pull/26512
removed sender_stake from banking-stage buffer prioritization.
2023-04-06 21:33:43 +00:00
Andrew Fitzgerald 00250819b8
ThreadAwareAccountLocks (#30422) 2023-04-06 10:12:03 -07:00
Ashwin Sekar 85dbd3d94d
Add stake breakdown to metrics for HeaviestForkFailures (#31067) 2023-04-05 20:35:12 -06:00
Brennan 60c4a718a5
enhance replay partition metrics (#31010)
* enhance replay partition metrics
2023-04-04 19:57:09 -07:00
Ashwin Sekar 9cbaf0e234
Rename DeadSlotAncestorRequestStatus -> AncestorRequestStatus (#31050) 2023-04-04 20:48:45 +00:00
Tyera 3442f184f7
Remove unneeded `clippy::new_ret_no_self` allows (#31035)
Remove unneeded allows
2023-04-03 20:35:20 -06:00
behzad nouri ff9a42a354
uses Duration type instead of untyped ..._ms: u64 (#30971) 2023-03-31 15:42:49 +00:00
steviez cc8e531a5d
Enforce a minimum of 1 on full and incremental snapshot retention (#30968) 2023-03-30 10:16:36 -05:00
Illia Bobyr fe5ae7733b
ledger: confirm_slot_entries(): confirmation_elapsed metric (#30807)
Measure total time spent inside `confirm_slot_entries()`.  Useful metric
in addition to `replay_elapsed` and
`poh_verify_elapsed`/`transaction_verify_elapsed`, as it shows how PoH
and transaction verification interact with the replay process.
2023-03-29 13:11:29 -07:00
Andrew Fitzgerald 8e910b494f
Forwarder: separate get_leader_and_addr (#30922) 2023-03-28 16:34:36 -07:00
Illia Bobyr 564f8c9b17
ledger: Extract `BatchExecutionTiming` (#30806)
Extracted time metrics related to transaction execution into a separate
structure.  This allows me to call `process_entries_with_callback()`
without locking the whole instance of `ConfirmationTiming`, passing just
the `BatchExecutionTiming` part.

I want to add a new metric that starts at the beginning of the
`confirm_slot_entries()` call and ends until the very end.  In order to
use a `scopeguard::defer`, I need to be able to have an excursive
reference to it for the whole body of `confirm_slot_entries()`.

Plus a few minor renamings to clarify which verifications and results
variables actually store.  And corrected a few messages, that
incorrectly stated PoH verification, while they were actually issued
for transaction verification failures.
2023-03-28 15:37:34 -07:00
Andrew Fitzgerald b72be0f086
Forwarder: clean up packet_vec filter (#30921) 2023-03-28 14:54:04 -07:00
Andrew Fitzgerald 2dfc46c71e
Forwarder: separate update_data_budget (#30920) 2023-03-28 12:58:39 -07:00
behzad nouri 75abfc79a6
removes unused dependencies (#30917) 2023-03-28 17:25:44 +00:00
behzad nouri 29f776c676
reports cluster-nodes metrics by stake (#30912) 2023-03-28 12:45:09 +00:00
Brooks d7ae05c3fd
Unifies logging of start/stop for background services (#30916) 2023-03-28 08:32:18 -04:00
behzad nouri 49b8ea771e
simplifies ServeRepair::run_orphan (#30908) 2023-03-27 23:19:55 +00:00
Andrew Fitzgerald a575ea2ee0
LeaderBankNotifier (#30395) 2023-03-27 08:17:17 -07:00
Brooks bf7fa02214
Add units to incremental accounts hash datapoint (#30894) 2023-03-24 21:39:20 +00:00
Andrew Fitzgerald f226a34f48
Only need bank reference for update (#30879) 2023-03-24 09:48:04 -07:00
Tao Zhu 52e63e2ffa
Allow banking_stage to update prioritization_fee_cache (#30853)
* Allow banking_stage to update prioritization_fee_cache

* Update core/src/banking_stage/committer.rs

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

* move use to top

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-03-24 00:05:54 +00:00
behzad nouri a36e8f559c
removes hash-map lookups when sorting vote-account keys (#30878) 2023-03-23 22:29:35 +00:00
Ryo Onodera 6c444df9e0
Add --block-{verification,production}-method flags (noop atm) (#30746)
* Add --{replaying,banking}-backend flags (noop atm)

* Greatly simplify enums with strum

* Update programs/sbf/Cargo.lock...

* Rely on Display, removing Debug

* constify cli_names()

* Don't allow omitting bankend value

* Rename to --block-{verification,production}-method

* Use more specific name

* Actually support missing value....

* Remove strictly-unnecessary flags

* Use lazy_static! instead of abusing DefaultArgs...
2023-03-23 12:57:28 +09:00
Jeff Biseda 04f0311aa1
check data budget before accessing blockstore (#30809) 2023-03-22 15:56:06 -07:00
Jeff Biseda 94b27d8f96
add metric for repair request blockstore misses (#30810) 2023-03-22 15:47:11 -07:00
Illia Bobyr 809041b151
poh_verify => run_verification: Rename to be more accurate (#30811)
`poh_verify` actually disables transaction signature, tick count and
built in program argument verifications as well.  It is somewhat
confusing to call it `poh_verify`.
2023-03-22 11:03:30 -07:00
Brooks 35437b8dad
Makes AccountsHashVerifier aware of Incremental Accounts Hash (#30820) 2023-03-22 10:20:16 -04:00
behzad nouri 25b7811869
moves shreds deduper to shred-sigverify stage (#30786)
Shreds arriving at tvu/tvu_forward/repair sockets are each processed in
a separate thread, and since each thread has its own deduper, the
duplicates across these sockets are not filtered out.
Using a common deduper across these threads will require an RwLock
wrapper and may introduce lock contention.
The commit instead moves the shred-deduper to shred-sigverify-stage
where all these shreds arrive through the same channel.
2023-03-22 13:19:16 +00:00