Commit Graph

4129 Commits

Author SHA1 Message Date
mergify[bot] 0348991a2c
v2.0: Store epoch in MaxAge (backport of #3485) (#3501)
* Store epoch in MaxAge (#3485)

(cherry picked from commit 5a6f518c6018ab0ebd101777045c5a1470cde5db)

# Conflicts:
#	core/src/banking_stage/consumer.rs
#	core/src/banking_stage/scheduler_messages.rs
#	core/src/banking_stage/transaction_scheduler/transaction_state.rs
#	core/src/banking_stage/transaction_scheduler/transaction_state_container.rs

* resolve conflicts

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-11-07 09:01:18 -06:00
mergify[bot] 8561902444
v2.0: Scheduler: Improve TTL (backport of #3161) (#3179)
* Scheduler: Improve TTL (#3161)

Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
(cherry picked from commit 7b0a57316d)

# Conflicts:
#	accounts-db/src/accounts.rs
#	core/src/banking_stage/consumer.rs
#	core/src/banking_stage/immutable_deserialized_packet.rs
#	core/src/banking_stage/latest_unprocessed_votes.rs
#	core/src/banking_stage/transaction_scheduler/scheduler_controller.rs
#	runtime/src/bank/address_lookup_table.rs

* resolve conflicts

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-10-17 12:52:12 -05:00
mergify[bot] 573db9c5d8
v2.0: banking_stage: do not insert legacy vote ixs, refactor & unstaked (backport of #2888) (#2901)
* banking_stage: do not insert legacy vote ixs, refactor & unstaked (#2888)

* banking_stage: do not insert legacy vote ixs, refactor & unstaked

* pr feedback: use matches instead of separate fn

(cherry picked from commit 1334fb5248)

# Conflicts:
#	core/src/banking_stage/latest_unprocessed_votes.rs

* fix conflicts

* rekey feature to indicate it must not be activated

---------

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2024-09-30 19:31:55 -04:00
mergify[bot] c01560e136
v2.0: scheduler opt-in forwarding (backport of #1801) (#2285)
* scheduler opt-in forwarding (#1801)

(cherry picked from commit 61d8be0d6f)

* Scheduler: buffer packets for forwarding if forwarding is enabled (#2305)

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-09-12 10:43:21 -05:00
mergify[bot] 3853b5d536
v2.0: replay: do not refresh votes for hot spare validators (backport of #2770) (#2779)
replay: do not refresh votes for hot spare validators (#2770)

(cherry picked from commit 726a14f0f2)

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
2024-08-29 18:38:47 -04:00
mergify[bot] 2088c5870c
v2.0: replay: extend last fec set check for 32+ retransmitter signed shreds (backport of #2101) (#2192)
replay: extend last fec set check for 32+ retransmitter signed shreds (#2101)

* replay: extend last fec set check for 32+ retransmitter signed shreds

* pr feedback: use separate feature flag

* pr feedback: is_retransmitter_signed -> is_retransmitter_signed_variant, false for legacy

* pr feedback: update doc comment fail -> error

* pr feedback: hash -> bank_hash for report metrics

* refactor metrics inside blockstore fn, return block_id for future use

* pr feedback: gate metrics reporting

* pr feedback: do not distinguish impossible combos, simplify check code

* pr feedback: remove report_metrics helper fn

* pr feedback: remove metric

* pr feedback: block_id -> last_fec_set_merkle_root

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
2024-08-28 00:24:08 -04:00
mergify[bot] 11b87c1ba3
v2.0: patches bug causing false duplicate nodes error (backport of #2666) (#2681)
* customizes override logic for gossip ContactInfo (#2579)

If there are two running instances of the same node, we want the
ContactInfo with more recent start time to be propagated through
gossip regardless of wallclocks.

The commit adds custom override logic for ContactInfo to first compare
by outset timestamp.

* updates ContactInfo.outset when hot-swapping identity (#2613)

When hot-swapping identity, ContactInfo.outset should be updated so that
the new ContactInfo overrides older node with the same pubkey.

* patches bug causing false duplicate nodes error (#2666)

The bootstrap code during the validator start pushes a contact-info with
more recent timestamp to gossip. If the node is staked the contact-info
lingers in gossip causing false duplicate node instances when the fully
initialized node joins gossip later on.

The commit refreshes the timestamp on contact-info so that it overrides
the one pushed by bootstrap and avoid false duplicates error.

---------

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-08-23 20:18:47 +00:00
mergify[bot] 901de9aa2c
v2.0: replay: do not early return when marking slots duplicate confirmed (backport of #2700) (#2706)
* replay: do not early return when marking slots duplicate confirmed (#2700)

* replay: do not early return when marking slots duplicate confirmed

* pr feedback: catch panic explicitely, comments, add root test case

* pr feedback: add custom string to panic message

* pr feedback: add slot to log, use should_panic

* pr feedback: notification for {slot} -> notification for slot {slot}

(cherry picked from commit 1444baa426)

* rename mark_slots_duplicate_confirmed -> mark_slots_confirmed

---------

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2024-08-23 15:58:46 -04:00
mergify[bot] a210e0c6f5
v2.0: Check poh_recorder.start_slot() hasn't been dumped previously before checking it in ProgressMap. (backport of #2676) (#2691)
Check poh_recorder.start_slot() hasn't been dumped previously before checking it in ProgressMap. (#2676)

* Check poh_recorder.start_slot() hasn't been dumped previously before checking it in progress_map.

* Add more comments and put in checks for maybe_start_leader.

* Update core/src/replay_stage.rs

Co-authored-by: Ashwin Sekar <ashwin@solana.com>

* Use a slot which I am not leader to avoid dumping my own slot panic.

* Address reviewer comments.

* Address reviewer comments.

---------

Co-authored-by: Ashwin Sekar <ashwin@solana.com>
(cherry picked from commit 053faa6a29)

Co-authored-by: Wen <113942165+wen-coding@users.noreply.github.com>
2024-08-22 09:25:46 -07:00
mergify[bot] 1fefe5f36e
v2.0: fix: send votes to the immediate next leader (backport of #2607) (#2621) 2024-08-21 07:28:13 +08:00
mergify[bot] 6f63e4558b
v2.0: fix: ensure vote packets can be retried (backport of #2605) (#2612) 2024-08-17 10:49:19 +08:00
mergify[bot] fd3b4f324f
v2.0: replay: do not start leader for a block we already have shreds for (backport of #2416) (#2484)
* replay: do not start leader for a block we already have shreds for (#2416)

* replay: do not start leader for a block we already have shreds for

* pr feedback: comment, move existing check to blockstore fn

* move blockstore read after tick height check

* pr feedback: resuse blockstore fn in next_leader_slot

(cherry picked from commit 15dbe7fb0f)

# Conflicts:
#	poh/src/poh_recorder.rs

* fix conflicts

---------

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2024-08-15 13:40:38 -04:00
mergify[bot] 20b76501ad
v2.0: Cleanup PoH speed check error (backport of #2400) (#2458)
* Cleanup PoH speed check logs and error

The current logging and error message from the Poh speed check are
confusing. If the node fails, the error message states that the node is
too slow. But, the reported numbers are slot durations in nanoseconds
where a slower node will have a larger number. Lastly, the reported
numbers aren't labeled with a unit so it is hard to make sense of this
without looking at the actual code.

The check now computes and reports hashes per second.

(cherry picked from commit ecc05c50b8)

# Conflicts:
#	core/src/validator.rs

* merge conflicts

---------

Co-authored-by: steviez <steven@anza.xyz>
2024-08-06 14:34:55 -05:00
mergify[bot] 845faca447
v2.0: geyser: add num_partitions to block info (backport of #2158) (#2178)
geyser: add num_partitions to block info (#2158)

* geyser: add num_partitions to block info

* fix comment

(cherry picked from commit d8791a633f)

Co-authored-by: Kirill Fomichev <fanatid@ya.ru>
Co-authored-by: Tyera <tyera@anza.xyz>
2024-07-23 10:09:23 -06:00
mergify[bot] b57b883355
v2.0: Fix BankForks::new_rw_arc memory leak (backport of #1893) (#2066)
* Fix BankForks::new_rw_arc memory leak (#1893)

(cherry picked from commit d441c0f577)

* Fix flaky test test_banking_stage_entries_only_central_scheduler (#2082)

---------

Co-authored-by: Andrei Silviu Dragnea <andreisilviudragnea@gmail.com>
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-07-17 16:11:40 -07:00
mergify[bot] 5d680fd1a2
v2.0: blockstore: only consume duplicate proofs from root_slot + 1 on startup (backport of #1971) (#2114)
blockstore: only consume duplicate proofs from root_slot + 1 on startup (#1971)

* blockstore: only consume duplicate proofs from root_slot + 1 on startup

* pr feedback: update test comments

* pr feedback: add pub behind dcou for test fns

(cherry picked from commit 2a48564b59)

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
2024-07-16 21:02:23 -04:00
mergify[bot] 9a9cb755cd
v2.0: Remove deprecated symbols from solana-client (mostly re-exports) (backport of #1992) (#2041)
Remove deprecated symbols from solana-client (mostly re-exports) (#1992)

* Remove usage of deprecated re-export from SendTransactionService

* Remove usage of deprecated re-export from core

* Remove usage of deprecated re-export from solana-dos

* Remove deprecated ConnectionCache methods

* Remove deprecated udp_client re-export

* Remove deprecated tpu_connection re-export

* Remove deprecated quic_client re-export

(cherry picked from commit 3cd7621a81)

Co-authored-by: Tyera <tyera@anza.xyz>
2024-07-08 18:13:15 -06:00
mergify[bot] 50f12b0040
v2.0: Use node's latest vote for commitment calc. too (backport of #1964) (#1994)
Use node's latest vote for commitment calc. too (#1964)

* Use node's latest vote for commitment calc. too

* Make local_cluster test use finalized

* Update core/src/commitment_service.rs

Co-authored-by: Tyera <teulberg@gmail.com>

* Don't wrap with Option and update tests

---------

Co-authored-by: Tyera Eulberg <tyera@anza.xyz>
Co-authored-by: Tyera <teulberg@gmail.com>
(cherry picked from commit 556298982a)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2024-07-04 00:33:22 -06:00
mergify[bot] 0001afe8ed
v2.0: Refactor cost tracking (backport of #1954) (#1975)
* Refactor cost tracking (#1954)

* Refactor and additional metrics for cost tracking (#1888)

* Refactor and add metrics:
- Combine remove_* and update_* functions to reduce locking on cost-tracker and iteration.
- Add method to calculate executed transaction cost by directly using actual execution cost and loaded accounts size;
- Wireup histogram to report loaded accounts size;
- Report time of block limits checking;
- Move account counters from ExecuteDetailsTimings to ExecuteAccountsDetails;

* Move committed transactions adjustment into its own function

* remove histogram for loaded accounts size due to performance impact

(cherry picked from commit f8630a3522)

* rename cost_tracker.account_data_size to better describe its purpose is to tracker per-block new account allocation

---------

Co-authored-by: Tao Zhu <82401714+tao-stones@users.noreply.github.com>
Co-authored-by: Tao Zhu <tao@solana.com>
2024-07-03 14:59:02 -05:00
mergify[bot] 5b9c5b965c
v2.0: gossip: do not allow duplicate proofs for incorrect shred versions (backport of #1931) (#1941)
gossip: do not allow duplicate proofs for incorrect shred versions (#1931)

* gossip: do not allow duplicate proofs for incorrect shred versions

* pr feedback: refactor test function to take shred_version

(cherry picked from commit 69ea21e947)

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
2024-07-02 09:28:21 -04:00
Tao Zhu f8ae688668
Revert "v2.0: Refactor and additional metrics for cost tracking (backport of #1888) (#1900) (#1937)
Revert "v2.0: Refactor and additional metrics for cost tracking (backport of #1888) (#1900)"

This reverts commit 0aef62eac7.
2024-07-01 10:33:44 -06:00
mergify[bot] 0aef62eac7
v2.0: Refactor and additional metrics for cost tracking (backport of #1888) (#1900)
* Refactor and additional metrics for cost tracking (#1888)

* Refactor and add metrics:
- Combine remove_* and update_* functions to reduce locking on cost-tracker and iteration.
- Add method to calculate executed transaction cost by directly using actual execution cost and loaded accounts size;
- Wireup histogram to report loaded accounts size;
- Report time of block limits checking;
- Move account counters from ExecuteDetailsTimings to ExecuteAccountsDetails;

* Move committed transactions adjustment into its own function

(cherry picked from commit c3fadacf69)

* rename cost_tracker.account_data_size to better describe its purpose is to tracker per-block new account allocation

---------

Co-authored-by: Tao Zhu <82401714+tao-stones@users.noreply.github.com>
Co-authored-by: Tao Zhu <tao@solana.com>
2024-06-28 16:09:36 -05:00
Joe C 0f956c8ba7
SVM: Refactor program match criteria (#1784)
* SVM: hoist `program_modification_slot` up from bank

* SVM: add `check_program_modification_slot` to processing config

* SVM: hoist `program_match_criteria` up from bank

* SVM: drop `get_program_match_critera` from callbacks
2024-06-21 12:21:26 -05:00
Andrew Fitzgerald 8747c3f5e9
Clean up: remove unused _feature_set (#1803) 2024-06-20 15:35:48 -05:00
Justin Starry 66e3acce45
Rename tx counts from committed to executed (#1807) 2024-06-20 13:13:47 -07:00
Justin Starry 0d1ad0dc1d
Reuse compute budget processing (#1700)
* refactor: reuse compute budget limits

* fix tests
2024-06-20 19:22:19 +00:00
Andrew Fitzgerald 98f5d0ea63
ReadWriteAccountSet: use AHashSet (#1265) 2024-06-20 14:04:47 -05:00
Andrew Fitzgerald 246fc3c2f0
PrioGraphScheduler::complete_batch remove TransactionAccountLocks allocation (#1759) 2024-06-20 06:46:27 -05:00
Ryo Onodera 3e53b644a8
Adjust replay-related metrics for unified scheduler (#1741)
* Adjust replay-related metrics for unified schduler

* Fix grammar

* Don't compute slowest for unified scheduler

* Rename to is_unified_scheduler_enabled

* Hoist uses to top of file

* Conditionally disable replay-slot-end-to-end-stats

* Remove the misleading fairly balanced text
2024-06-20 09:12:04 +09:00
Andrew Fitzgerald d06a3aceb2
fix race condition on vote count (#1762) 2024-06-17 20:22:06 -05:00
Tyera 034cd7396a
Add num_partitions to Blockstore rewards (#1601)
* Add num_partitions field to Rewards proto definition

* Add type to hold rewards plus num_partitions

* Add Bank method to get rewards plus num_partitions for recording

* Update Blockstore::write_rewards to use num_partitions

* Update RewardsRecorderService to handle num_partitions

* Populate num_partitions in ReplayStage::record_rewards

* Write num_partitions to Bigtable

* Reword KeyedRewardsAndNumPartitions method

* Clone immediately

* Determine epoch boundary by checking parent epoch

* Rename UiConfirmedBlock field

* nit: fix comment typo

* Add test_get_rewards_and_partitions

* Add pre-activation test

* Add should_record unit test
2024-06-17 16:42:33 -06:00
Kevin Heavey 7e0688341e
remove superfluous usage of allow(dead_code) (#1669) 2024-06-13 22:52:49 +08:00
Ashwin Sekar da36ce7abe
replay: do not hold bank forks lock during mark_dead_slot (#1597) 2024-06-12 22:14:11 -04:00
Joe C ecf72523ab
SVM: Dissolve `RuntimeConfig` (#1590)
* SVM: add `compute_budget` to processing config

* bank: add `compute_budget` field

* SVM: add `transaction_account_lock_limit` to processing config

* bank: add `transaction_account_lock_limit` field

* bank: drop `runtime_config` getter

* SVM: require `compute_budget` for `prepare_program_cache_for_upcoming_feature_set`

* SVM: drop `runtime_config` from batch processor
2024-06-12 12:04:39 -05:00
Ashwin Sekar 000ca4ce01
replay: only recache tower stats for computed banks on tower adoption (#1632)
* replay: only recache tower stats for computed banks on tower adoption

* pr feedback: refactor and add test
2024-06-11 22:25:43 -04:00
Tao Zhu d049a79f21
Wire scheduler up to adjust actual loaded accounts size cost for committed transactions (#1547)
* wire up scheduler to adjust actual loaded accounts size cost for committed transactions

* update tests
2024-06-07 12:29:46 -05:00
Joe C 31334c44eb
Vote: hoist vote sender types up to runtime (#1623)
vote: hoist vote sender types up to runtime
2024-06-06 10:59:48 -05:00
Joe C 0bf34e5ca1
Compute Budget: add `frozen-abi` feature (#1606)
compute budget: add `frozen-abi` feature
2024-06-05 09:20:50 -05:00
Ashwin Sekar 06b098e448
consensus: refactor stray last vote logic in switch threshold check (#1599) 2024-06-04 22:05:12 -04:00
Andrei Silviu Dragnea f4417b1915
Simplify signature of Bank::load_and_execute_transactions (#1571) 2024-06-03 09:03:51 -05:00
Joe C 62eb4cc068
Add Compute Budget Crate (#1121) 2024-06-01 01:38:19 -05:00
Ryo Onodera 74dcd3b0ad
Make unified schedler abort on tx execution errors (#1211)
* Make unified schedler abort on tx execution errors

* Fix typo and improve wording

* Indicate panic _should_ be enforced by trait impls

* Fix typo

* Avoid closure of closure...

* Rename: is_threads_joined => are_threads_joined

* Add some comments for trashed_scheduler_inners

* Ensure 100% coverage of end_session by more tests

* Document relation of aborted thread and is_trashed

* Replace sleep()s with sleepless_testing

* Fix lints from newer rust
2024-05-31 16:13:41 +09:00
Ashwin Sekar 6859d652b2
consensus: make shallow threshold checks log only (#1506)
* consensus: make shallow threshold checks log only

* pr feedback: comment, make check more readable
2024-05-30 13:14:47 -04:00
Andrew Fitzgerald 0d34a1a160
scheduler optimization - worker precompile verification (#1531) 2024-05-30 13:27:04 +08:00
Justin Starry 2cc540a7de
refactor: reuse calculated fees (#1462)
* refactor: reuse calculated fees

* fix tests

* feedback

* fix test

* fix unused warn
2024-05-29 19:41:47 +00:00
Andrew Fitzgerald 54bd725005
ThreadAwareAccountLocks: use AHashMap (#1264) 2024-05-29 22:28:57 +08:00
Brooks 6f8d817fc6
Moves bank snapshot serialization to SnapshotPackagerService (#1508) 2024-05-29 09:59:40 -04:00
Ashwin Sekar 6bade6fd88
replay: add metrics for threshold failures (#1500)
* replay: add metrics for threshold failures

* pr feedback: separate function, match, stake_pct

* report stake values instead of pct
2024-05-28 16:32:20 -04:00
Brooks 802b74a176
Removes test_concurrent_snapshot_packaging() (#1484) 2024-05-28 11:06:09 -04:00
Justin Starry a4a009e04d
refactor: checked transaction lps is never none (#1487) 2024-05-26 11:26:53 -04:00