Commit Graph

3946 Commits

Author SHA1 Message Date
behzad nouri 79bbe4381a
adds chained_merkle_root to shredder arguments (#34952)
Working towards chaining Merkle root of erasure batches, the commit adds
chained_merkle_root to shredder arguments.
2024-01-27 15:04:31 +00:00
behzad nouri d4fdcd940a
adds feature to enable chained Merkle shreds (#34916)
During a cluster upgrade when only half of the cluster can ingest the new shred
variant, sending shreds of the new variant can cause nodes to diverge.
The commit adds a feature to enable chained Merkle shreds explicitly.
2024-01-27 15:03:16 +00:00
Brooks 02062a6b6a
Removes unused AccountsHashFaultInjector (#34977) 2024-01-26 19:21:23 -05:00
Brooks e1260a9604
Removes unused parameters from AccountsHashVerifier::new() (#34976) 2024-01-26 21:52:05 +00:00
Pankaj Garg 0d117d420c
Remove BlockhashQueue dependency from SVM related code (#34974) 2024-01-26 13:46:44 -08:00
Brooks c656ca68b8
Stops pushing accounts hashes to gossip in AccountsHashVerifier (#34971) 2024-01-26 15:25:23 -05:00
Ashwin Sekar 93271d91b0
gossip: notify state machine of duplicate proofs (#32963)
* gossip: notify state machine of duplicate proofs

* Add feature flag for ingesting duplicate proofs from Gossip.

* Use the Epoch the shred is in instead of the root bank epoch.

* Fix unittest by activating the feature.

* Add a test for feature disabled case.

* EpochSchedule is now not copyable, clone it explicitly.

* pr feedback: read epoch schedule on startup, add guard for ff recache

* pr feedback: bank_forks lock, -cached_slots_in_epoch, init ff

* pr feedback: bank.forks_try_read() -> read()

* pr feedback: fix local-cluster setup

* local-cluster: do not expose gossip internals, use retry mechanism instead

* local-cluster: split out case 4b into separate test and ignore

* pr feedback: avoid taking lock if ff is already found

* pr feedback: do not cache ff epoch

* pr feedback: bank_forks lock, revert to cached_slots_in_epoch

* pr feedback: move local variable into helper function

* pr feedback: use let else, remove epoch 0 hack

---------

Co-authored-by: Wen <crocoxu@gmail.com>
2024-01-26 07:58:37 -08:00
Andrew Fitzgerald 29737ab5e4
Use ThreadLocalMultiIterator for tests (#34947)
* Use ThreadLocalMultiIterator for tests

* some validator config was not using default_for_test
2024-01-25 11:22:27 -07:00
Pankaj Garg b161f6ce08
Create SVM folder as a placeholder for the relevant code (#34942) 2024-01-25 06:20:00 -08:00
Andrew Fitzgerald 62e7ebd0cc
BlockProductionMethod::CentralScheduler as default (#34891) 2024-01-24 15:30:32 -08:00
Brooks b150de6d10
Replaces fs-err in clean_orphaned_account_snapshot_dirs() (#34902)
* Replaces fs-err in clean_orphaned_account_snapshot_dirs()

* pr: revert info message format changes
2024-01-23 19:46:02 +00:00
Andrew Fitzgerald bb829c0bcf
remove unused functions (#34895) 2024-01-23 09:32:35 -08:00
Andrew Fitzgerald 45a2a701de
TransactionState: add TransactionCost (#34881) 2024-01-22 16:21:06 -08:00
Brooks 2b0b5ae1ba
Gracefully handles errors when archiving snapshots (#34856) 2024-01-19 11:15:53 -05:00
steviez 3dd348802f
Bubble up genesis load errors instead of exiting (#34851)
The function open_genesis_config() performs several operations that
could fail. If any of these fail, the process exits immediately.

Instead of exiting immediately, bubble up the error and let the caller
decide the appropriate action. solana-validator and solana-ledger-tool
will functionally be unchanged, but this consolidates startup failures
for both of these processes.
2024-01-19 10:25:46 -05:00
Andrew Fitzgerald 6976b750a9
Refactor: SchedulerController tests call receive then schedule (#34836) 2024-01-18 22:49:11 -08:00
Brennan 3d7ee19348
Sample peers if input peer missing for repair shred (#34818) 2024-01-18 18:33:38 -08:00
HaoranYi 6a9f729101
ForkStats: compute and wire up fork_weight (#34623)
* wire up fork stats fork_weight

* convert weight to percentage for logging

* add bank_stake

* remove old fork choice measure and stats - vote stake * lockout

* update tests

* fix bank_weight and rename it to fork_weight

* fix u64 multiple overflow in fork_weight calculation

* format fork_weight as percentage in logging

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2024-01-18 09:35:53 -06:00
Brennan 8fe4f5d0b9
Shred Repair Request (#34771)
* shred repair admin rpc
2024-01-17 14:19:32 -08:00
Andrew Fitzgerald beef8476f8
TransactionScheduler: Add funded check before graph insertion (#34699) 2024-01-12 11:54:04 -08:00
Andrew Fitzgerald 98a2873517
consumer bench same number of txs per iteration (#34752) 2024-01-12 08:24:45 -08:00
Andrew Fitzgerald be5337a839
earlier fee payer validation (#34666) 2024-01-12 08:22:39 -08:00
Ashwin Sekar fb97e93fe3
fix duplicate confirmed rollup detection for descendants (#34014)
* fix duplicate confirmed rollup detection for descendants

* pr feedback: optimistic rename -> guard new enum
2024-01-10 15:10:30 -05:00
Andrew Fitzgerald ac04a758e1
Remove chain-joining thread-selection (#34554) 2024-01-10 09:10:12 -08:00
Brooks e84974cf63
Logs the number of storages kept alive by fastboot (#34667) 2024-01-05 14:56:07 -05:00
Justin Starry 47b40757e3
Add test in banking stage for recording txs in last tick (#34635)
* Add test in banking stage for recording txs in last tick

* nits

* feedback
2024-01-06 00:52:02 +08:00
Brooks b8144979dc
clippy: core lints (#34629)
```
warning: writing `&Vec` instead of `&[_]` involves a new object where a slice will do
   --> core/src/banking_stage/consumer.rs:153:29
    |
153 |         packets_to_process: &Vec<Arc<ImmutableDeserializedPacket>>,
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: change this to: `&[Arc<ImmutableDeserializedPacket>]`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
    = note: `#[warn(clippy::ptr_arg)]` on by default

warning: `solana-core` (lib) generated 1 warning
```

```
warning: unit tests in doctest are not executed
  --> core/tests/fork-selection.rs:15:5
   |
15 |   //! #[test]
   |  _____^
16 | | //! #[ignore]
17 | | //! fn test_all_partitions() {
   | |__________________________^
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#test_attr_in_doctest
   = note: `#[warn(clippy::test_attr_in_doctest)]` on by default

warning: `solana-core` (test "fork-selection") generated 1 warning
```
2024-01-04 17:02:56 +00:00
Ashwin Sekar 19088411ff
blockstore: populate duplicate shred proofs for merkle root conflicts (#34270)
* blockstore: populate duplicate shred proofs for merkle root conflicts

* pr feedback: check test case

* pr feedback: comment

* pr feedback: match statement, shred_id, comment

* add feature flag

* pr feedback: rename ff var and perform_merkle_check

* pr feedback: move panic to callers in get_shred_from_just_inserted_or_db

* avoid unecessary write if proof is already present
2024-01-03 12:15:52 -05:00
Nick Frostbutter fc2a8794be
[docs] updated readme and fix links (#34565)
* feat: updated readme

* fix: updated links

* fix: proposal links

* fix: more links

* fix: json-rpc links

* fix: more links

* fix: zk links

* fix: managing forks

* fix: links for deprecated methods
2024-01-03 09:06:06 -05:00
Ford 057ad2ab47
Fix typo in comments in core (#34594) 2023-12-26 16:13:17 -05:00
Brooks 6b76ec1de0
Uses Bank::new_for_tests() instead of new_for_tests_with_config() (#34587) 2023-12-26 12:54:14 -05:00
GoodDaisy 03386cc7b9
Fix typos (#34459)
* Fix typos

* Fix typos

* fix typo
2023-12-21 13:06:00 -07:00
Ashwin Sekar def3bc4c4f
Add feature flag for LastIndex and Erasure duplicate proofs (#34360)
* Add feature flag for LastIndex and Erasure duplicate proofs

* pr feedback: use root bank instead of 2 params

* pr feedback: & instead of &Arc

* pr feedback: reuse fn, remove redundant clones

* rebase: fix feature set conflict
2023-12-20 10:20:30 -05:00
behzad nouri 4feadbdb7c
disables turbine and repair QUIC endpoints on mainnet-beta (#34523)
On mainnet-beta, respective QUIC endpoint are unnecessary for now until
testnet has fully migrated to QUIC. The commit disables turbine and
repair QUIC endpoints on mainnet-beta.
2023-12-19 19:12:18 +00:00
Andrew Fitzgerald d00c9a45b2
TransactionScheduler: pre_lock_filter (#34488) 2023-12-19 09:25:18 -08:00
Ryo Onodera d2b5afc410
Finish unified scheduler plumbing with min impl (#34300)
* Finalize unified scheduler plumbing with min impl

* Fix comment

* Rename leftover type name...

* Make logging text less ambiguous

* Make PhantomData simplyer without already used S

* Make TaskHandler stateless again

* Introduce HandlerContext to simplify TaskHandler

* Add comment for coexistence of Pool::{new,new_dyn}

* Fix grammar

* Remove confusing const for upcoming changes

* Demote InstalledScheduler::context() into dcou

* Delay drop of context up to return_to_pool()-ing

* Revert "Demote InstalledScheduler::context() into dcou"

This reverts commit 049a126c905df0ba8ad975c5cb1007ae90a21050.

* Revert "Delay drop of context up to return_to_pool()-ing"

This reverts commit 60b1bd2511a714690b0b2331e49bc3d0c72e3475.

* Make context handling really type-safe

* Update comment

* Fix grammar...

* Refine type aliases for boxed traits

* Swap the tuple order for readability & semantics

* Simplify PooledScheduler::result_with_timings type

* Restore .in_sequence()

* Use where for aesthetics

* Simplify if...

* Fix typo...

* Polish ::schedule_execution() a bit

* Fix rebase conflicts..

* Make test more readable

* Fix test failures after rebase...
2023-12-19 09:50:41 +09:00
Ashwin Sekar 4a8d27d921
replay: feature flag consumption of duplicate proofs from blockstore (#34372)
* replay: feature flag consumption of duplicate proofs from blockstore

* pr feedback: reorder check, add flag for restart logic
2023-12-18 19:01:24 -05:00
steviez ae6a4c0995
Only allow BankForks creation with single bank (#34449)
BankForks has two constructors; one that takes a single Bank (the root)
and one that can take an arbitrary number of Banks plus the root slot.
However, the constructor that accepts multiple banks is unnecessary; it
isn't used in production and is only used in several tests.

So, remove the multi-bank constructor and update unit tests.
2023-12-14 10:53:11 -06:00
Brooks 0b6d939e21
Removes epoch_accounts_hash featurization (#34417) 2023-12-13 15:30:39 -05:00
Brennan 07f38838ae
add additional vote lockout stake threshold (#34120)
* add additional vote lockout stake threshold
2023-12-12 06:50:58 -08:00
Lucas Steuernagel 4b762320cb
Remove WorkingSlot (#34342)
* Remove WorkingSlot
2023-12-11 16:02:53 -03:00
Justin Starry 2971e84ec8
Add metrics for locking the blockhash queue in banking stage (#34382) 2023-12-09 23:54:12 +08:00
ryleung-solana 132c910f81
Quic update identity (#33865)
Update the Quic transport layer keypair and identity when the Validator's identity keypair is updated
2023-12-08 14:53:19 +08:00
behzad nouri d5eee01950
adds feature gated code to drop legacy shreds (#34328) 2023-12-06 22:47:46 +00:00
behzad nouri b1c701e53b
patches slots_per_epoch vs last_slot ordering in shred_fetch_stage (#34338) 2023-12-06 20:47:53 +00:00
Lucas Steuernagel 1877fdb273
Use BankForks on tests - Part 4 (#34271)
* Use BankForks on tests - Part 4

* Ensure the correct slot is set
2023-12-06 13:32:04 -03:00
behzad nouri 2096626633
exits send_requests_task if the connection is closed (#33837)
receiver.recv() can unnecessarily block when the connection is already closed.
The commit exits send_requests_task if the connection is closed.
2023-12-05 16:08:06 +00:00
Andrew Fitzgerald d451675d50
TransactionScheduler: check transaction age and already processed during receive (#34253) 2023-12-04 14:41:05 -08:00
steviez c9cb727b1f
Add additional context to ReplayStage log message (#34296)
The pre-existing message was somewhat vague, so give more details about the caller
2023-12-01 13:04:39 -06:00
Lucas Steuernagel b97b3dd4ab
Use BankForks on tests - Part 3 (#34248)
* Add BankForks to core tests

* Refactor functions under DCOU
2023-12-01 13:47:22 -03:00