Commit Graph

3565 Commits

Author SHA1 Message Date
bji d77f0a22c7
Add new vote state version that replaces Lockout with LandedVote to a… (#29524)
* Add new vote state version that replaces Lockout with LandedVote to allow vote latency to be tracked in a future change. Includes a feature to be enabled which will when enabled cause the vote state to be written in the new form.

* Update feature set key to one owned by ashwin

---------

Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2023-03-20 08:31:46 -06:00
behzad nouri 46614c0e9f
removes false_positive_rate field from Deduper (#30788)
removes the false_positive_rate field from the Deduper

Deduper.false_positive_rate field is misleading because it is not
enforced until maybe_reset is called. But then maybe_reset can be
invoked with an explicit argument.
2023-03-20 13:16:52 +00:00
Brooks cd7fe76744
Removes writing BankIncrementalSnapshotPersistence in AccountsHashVerifier (#30792) 2023-03-19 21:45:13 -04:00
Xiang Zhu 8e3a30c22c
Clean orphaned account snapshot dirs (#30645)
* Clean up orphaned account snapshot hardlink dirs

* fix compilation issues

* debugged, now working.  seeing the orphaned directories deleted

* change back to eprintln + exit for account_path error

* changed eprintln to panic for now

* add test_clean_orphaned_account_snapshot_dirs for codecov check

* address a few comments and nit isseus

* directly unzip, skipped the intermediate array of tuples

* let set_up_account_run_and_snapshot_paths return Result

* 'proper' typo, and comment on return

* use map_err

* use for loop in clean_orphaned_account_snapshot_dirs, removed panic

* add test_set_up_account_run_and_snapshot_paths

* minor, replace .for_each with .all

* rename set_up_account_run_and_snapshot_paths to create_all_accounts_run_and_snapshot_dirs

* remove unnecessary closure return type

* change to for loop

* change match to unwrap_or_else

* remove create_dir_all(&account_path) in create_all

* minor comment cleanup
2023-03-17 15:22:10 -07:00
Brooks af367db6f0
Calculates accounts hash from storages in snapshot tests (#30778) 2023-03-17 15:22:02 -04:00
behzad nouri 93f696dac7
increases shred-fetch-stage deduper capacity and reset-cycle (#30690) 2023-03-17 00:05:29 +00:00
cavemanloverboy 10f49d4e26
Geyser Runtime Reload (#30352)
Support dynamic geyser plugin load, unload, and listing through admin RPC.
2023-03-16 17:03:00 -07:00
behzad nouri 7a7b020580
dedups packets using an atomic bloom filter (#30726)
Current Deduper implementation uses many bits per entry:
https://github.com/solana-labs/solana/blob/65cd55261/perf/src/deduper.rs#L70-L73
and may be saturated quickly. It also lacks api to specify desired false
positive rate.

The commit instead uses an atomic bloom filter with K hash functions.
The false positive rate is obtained by tracking popcount of bits.
2023-03-16 16:45:42 +00:00
ryleung-solana 0ed9f62602
Quic server batching (#30330) 2023-03-16 21:50:57 +08:00
Brooks 6bdbd2dfec
Removes unnecessary AccountsHashVerifier from snapshot tests (#30738) 2023-03-16 09:17:42 -04:00
Brooks 423fd6010e
Removes extraneous accounts hash calculations in snapshot tests (#30737) 2023-03-16 09:17:03 -04:00
Andrew Fitzgerald b7e76c752f
Separate stats updates from decision_maker (#30481)
* Separate stats updates from decision_maker

* BufferedPacketsDecision::bank_start

* BufferedPacketsDecision: bank_start() doc-comment

* remove unnecessary clone
2023-03-15 19:39:48 -07:00
Ryo Onodera 74970a0b5d
Remove unused ProcessOptions::entry_callback (#30600)
* Confine entry_callback under cfg(test) for clarity

* Fix ci

* Actually remove entry_callback altogether

* fix clippy
2023-03-16 09:33:18 +09:00
Brooks a5f86a8212
Verifies accounts hash in snapshot tests (#30724) 2023-03-15 12:23:44 -04:00
Tyera b389d509a8
Track max_complete_rewards_slot for use in rpc, bigtable (#30698)
* Add RewardsMessage enum

* Cache and update max_complete_rewards_slot

* Plumb max_complete_rewards_slot into JsonRpcRequestProcesseor

* Use max_complete_rewards_slot to check get_block requests

* Use max_complete_rewards_slot to limit Bigtable uploads

* Plumb max_complete_rewards_slot into RpcSubscriptions

* Use max_complete_rewards_slot to limit block subscriptions

* Nit: fix test
2023-03-14 12:08:48 -06:00
Brooks 93c43610ac
AccountsHashVerifier stores IncrementalAccountsHash in AccountsDb (#30696) 2023-03-14 12:41:44 -04:00
Brennan 9b587bf073
Create gossip vote iterator sorted by stake weight (#30697) 2023-03-14 08:43:01 -07:00
Brennan 11c942ab40
`test_verified_vote_packets_validator_gossip_votes_iterator_correct_fork` dynamic num validators (#30695)
Gossip vote test dynamic number of validators cleanup
2023-03-14 08:35:26 -07:00
Brooks 560ec08d5e
AccountsHashVerifier writes BankIncrementalSnapshotPersistence (#30587) 2023-03-13 17:44:34 -04:00
Brooks 346021a48c
Refactors common accounts hash calculation config in AccountsHashVerifier (#30677)
* Refactors common accounts hash calculation config in AccountsHashVerifier

* pr: config var
2023-03-13 19:39:28 +00:00
Brooks 6e5615e32d
Revert "AccountsHashVerifier remembers last full snapshot info (#30582)" (#30660) 2023-03-13 14:48:16 -04:00
Brooks 505e3ff5c7
AccountsHashVerifier updates AccountsDb after calculating accounts hash (#30658) 2023-03-13 16:41:24 +00:00
Trent Nelson a15139ef15
tests: share `GenesisConfig` in `validator_parallel_exit` (#30692) 2023-03-13 10:12:35 -06:00
Brooks a43f803604
AccountsHashVerifier purges old accounts hashes (#30644) 2023-03-13 11:12:11 -04:00
behzad nouri c4b2639a86
patches flaky test_retransmit_latest_unpropagated_leader_slot (#30686) 2023-03-12 22:46:05 +00:00
behzad nouri f9805b6fbb
stops nodes from broadcasting slots twice (#30681)
https://github.com/solana-labs/solana/blob/94ef881de/core/src/progress_map.rs#L178
always returns true the first time around because retry_time is None.
So every slot is broadcasted twice.
2023-03-11 02:46:08 +00:00
Andrew Fitzgerald 5f6755f58b
remove test fn (#30616)
kill banking_stage::Consumer test_fn
2023-03-10 09:30:25 -08:00
Andrew Fitzgerald b0112a5f43
BankingStage Consumer: test_buffered_packets* reworking (#30615)
* refactor test_consume_buffered_packets_interrupted without test_fn

* Fix comment

* Also check retries

* Add retryable test case
2023-03-09 09:13:04 -08:00
Ashwin Sekar 11e554b922
Fix repair_weight test bad merge failure (#30649) 2023-03-08 21:23:37 -07:00
Ashwin Sekar 31712d38de
Track pruned subtrees in repair weight (#29922) 2023-03-08 18:38:32 -07:00
Jeff Biseda 4c0ce84488
increase retransmit shreds received cache size (#30556) 2023-03-07 13:03:52 -08:00
HaoranYi 8a1c7614f1
typos (#30604) 2023-03-07 11:08:46 -06:00
Brooks 70c6c7e1f7
Uses strong types for snapshot hashes in SnapshotPackagerService (#30603) 2023-03-06 16:50:45 -05:00
Andrew Fitzgerald bba0ed702f
BankingStage Refactor: Consumer State (#30288)
* BankingStage Refactor: Consumer add state

* remove trailing comma
2023-03-06 09:13:28 -08:00
Brooks 120b0c92d1
AccountsHashVerifier remembers last full snapshot info (#30582)
AHV remembers last full snapshot info
2023-03-06 16:40:46 +00:00
steviez a8bff33387
Make backup_and_clear_blockstore() honor ValidatorConfig options (#30538)
* Add helper function to create BlockstoreOptions from ValidatorConfig

* Make backup_and_clear_blockstore() honor ValidatorConfig options

backup_and_clear_blockstore() opens a Blockstore session; however, it
is currently using Blockstore::open(). This Blockstore method uses
BlockstoreOption::default() under the hood. As a result, any validator
args that adjust Blockstore settings are not considered in
backup_and_clear_blockstore().

This is especially problematic if the non-default value of
--rocksdb-shred-compaction is being used. In this case,
backup_and_clear_blockstore() was opening the wrong directory and
incorrectly finding an empty ledger.

This change plumbs any blockstore configuration to
backup_and_clear_blockstore().
2023-03-04 21:09:41 -08:00
Brooks 6972f92c29
AHV loop uses let-else (#30583) 2023-03-04 01:59:29 +00:00
sakridge 7a8563f2c8
Panic when shred index exceeds the max per slot (#30555)
Assert when shred index exceeds the max per slot
2023-03-04 02:49:23 +01:00
Brooks 1cf0ce1215
AHV logs when stopped (#30585) 2023-03-03 23:44:30 +00:00
Brooks cd652a7e20
AHV uses metrics names like SPS's (#30584) 2023-03-03 23:38:35 +00:00
Tyera 7b1d446001
Admin RPC Service: move post-init activation to before wait-for-supermajority (#30544)
* Move AdminRpcRequestMetadataPostInit to solana-core

* Move AdminRpcRequestMetadataPostInit write to just before wait_for_supermajority

* Pass AdminRpcRequestMetadataPostInit in TestValidatorGenesis

* Fixup local-cluster
2023-03-01 19:38:11 -07:00
HaoranYi 16db984cb5
improve supermajority waiting logging (#30479)
make logging for supermajority waiting and stake percent from gossip in sync
2023-03-01 08:57:42 -06:00
Jeff Biseda 781a7cbd28
cleanup get_closeset_completion (#30516) 2023-02-27 16:56:40 -08:00
Brooks 89c07d259a
AccountsHashVerifier uses AccountsHashEnum (#30514) 2023-02-24 17:17:54 -05:00
Brennan 7847661511
Process tower after warping bank forks (#30467)
This helps ensure tower and bank forks are in sync in terms of root slot
2023-02-23 16:23:18 -08:00
Jeff Washington (jwash) 2441a06e78
drop default from PhantomData::default() (#30476) 2023-02-23 14:59:08 -08:00
Yihau Chen df3ef111f7
chore: workspace inheritance (#29893)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-02-23 22:01:54 +08:00
Michael Vines 5136ed3448
Update homepage value for all crates (#30444) 2023-02-23 02:20:18 +00:00
Jeff Biseda 55f601b25c
prevent revisiting slots in get_closest_completion (#30458) 2023-02-22 18:16:17 -08:00
Brooks 69a9520f79
Flushes accounts cache before warping (#30437) 2023-02-22 21:13:31 -05:00