Commit Graph

3533 Commits

Author SHA1 Message Date
Brooks 70c6c7e1f7
Uses strong types for snapshot hashes in SnapshotPackagerService (#30603) 2023-03-06 16:50:45 -05:00
Andrew Fitzgerald bba0ed702f
BankingStage Refactor: Consumer State (#30288)
* BankingStage Refactor: Consumer add state

* remove trailing comma
2023-03-06 09:13:28 -08:00
Brooks 120b0c92d1
AccountsHashVerifier remembers last full snapshot info (#30582)
AHV remembers last full snapshot info
2023-03-06 16:40:46 +00:00
steviez a8bff33387
Make backup_and_clear_blockstore() honor ValidatorConfig options (#30538)
* Add helper function to create BlockstoreOptions from ValidatorConfig

* Make backup_and_clear_blockstore() honor ValidatorConfig options

backup_and_clear_blockstore() opens a Blockstore session; however, it
is currently using Blockstore::open(). This Blockstore method uses
BlockstoreOption::default() under the hood. As a result, any validator
args that adjust Blockstore settings are not considered in
backup_and_clear_blockstore().

This is especially problematic if the non-default value of
--rocksdb-shred-compaction is being used. In this case,
backup_and_clear_blockstore() was opening the wrong directory and
incorrectly finding an empty ledger.

This change plumbs any blockstore configuration to
backup_and_clear_blockstore().
2023-03-04 21:09:41 -08:00
Brooks 6972f92c29
AHV loop uses let-else (#30583) 2023-03-04 01:59:29 +00:00
sakridge 7a8563f2c8
Panic when shred index exceeds the max per slot (#30555)
Assert when shred index exceeds the max per slot
2023-03-04 02:49:23 +01:00
Brooks 1cf0ce1215
AHV logs when stopped (#30585) 2023-03-03 23:44:30 +00:00
Brooks cd652a7e20
AHV uses metrics names like SPS's (#30584) 2023-03-03 23:38:35 +00:00
Tyera 7b1d446001
Admin RPC Service: move post-init activation to before wait-for-supermajority (#30544)
* Move AdminRpcRequestMetadataPostInit to solana-core

* Move AdminRpcRequestMetadataPostInit write to just before wait_for_supermajority

* Pass AdminRpcRequestMetadataPostInit in TestValidatorGenesis

* Fixup local-cluster
2023-03-01 19:38:11 -07:00
HaoranYi 16db984cb5
improve supermajority waiting logging (#30479)
make logging for supermajority waiting and stake percent from gossip in sync
2023-03-01 08:57:42 -06:00
Jeff Biseda 781a7cbd28
cleanup get_closeset_completion (#30516) 2023-02-27 16:56:40 -08:00
Brooks 89c07d259a
AccountsHashVerifier uses AccountsHashEnum (#30514) 2023-02-24 17:17:54 -05:00
Brennan 7847661511
Process tower after warping bank forks (#30467)
This helps ensure tower and bank forks are in sync in terms of root slot
2023-02-23 16:23:18 -08:00
Jeff Washington (jwash) 2441a06e78
drop default from PhantomData::default() (#30476) 2023-02-23 14:59:08 -08:00
Yihau Chen df3ef111f7
chore: workspace inheritance (#29893)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-02-23 22:01:54 +08:00
Michael Vines 5136ed3448
Update homepage value for all crates (#30444) 2023-02-23 02:20:18 +00:00
Jeff Biseda 55f601b25c
prevent revisiting slots in get_closest_completion (#30458) 2023-02-22 18:16:17 -08:00
Brooks 69a9520f79
Flushes accounts cache before warping (#30437) 2023-02-22 21:13:31 -05:00
Jeff Biseda 5221049595
stop get_unrepaired_path at root slot (#30450) 2023-02-22 15:04:09 -08:00
Brennan d2c6bd1410
Metrics for repair trees & closest completion slots (#30448) 2023-02-22 14:33:02 -08:00
Brennan e7a69dcec5
get_best_repairs minor cleanup (#30439) 2023-02-22 12:15:42 -08:00
Brennan 96dd621426
Remove ignored slots from repair (#30438) 2023-02-22 12:15:17 -08:00
Brooks 1689586213
Uses a channel for AHV -> SPS (#30406) 2023-02-22 03:36:29 +00:00
Brooks 35328ca63d
Makes AccountsHash an enum (#30416) 2023-02-21 15:20:51 -05:00
Brooks bcc4bc80c9
Removes unnecessary derives from Accounts{Delta}Hash (#30392) 2023-02-20 16:00:53 -05:00
Andrew Fitzgerald 50f553e245
Clean up: consumer saturating add assign (#30347)
Use saturating_add_assign where appropriate in Consumer
2023-02-16 15:19:43 -08:00
Brooks 4ba80ad785
Inline format args (#30364)
clippy fixes
2023-02-16 17:00:43 +00:00
Brooks febaf36e6d
Apply clippy fixes for future rust upgrade (#30363) 2023-02-16 16:12:51 +00:00
Andrew Fitzgerald 4194661bcf
Rewrite accumulate_execute_units_and_time without allocation (#30338) 2023-02-15 17:22:24 -08:00
Andrew Fitzgerald 1cefb90271
BankingStage Refactor: Simplify Consumer (#30253)
* measure! to measure_us!

* Consistent naming of transaction_recorder

* Remove outdated comment - Instant cannot be None

* use local

* Remove measure! import
2023-02-15 17:20:55 -08:00
Jeff Biseda 20614fa746
restore timestamp() in find_missing_indexes (#30345) 2023-02-15 16:12:36 -08:00
Andrew Fitzgerald b86bfbb5c5
measure_us! use Instant and duration_to_us internally (#30339) 2023-02-15 12:43:47 -08:00
Xiang Zhu 4909267c88
Add accounts hard-link files into the bank snapshot directory (#29496)
* Add accounts hard-link files into the bank snapshot directory

* Small adjustments and fixes.

* Address some of the review issues

* Fix compilation issues

* Change the latest slot snapshot storage from VecDeque to Option

* IoWithSourceAndFile and expanded comments on accounts

* last_slot_snapshot_storages in return value

* Update comments following the review input

* rename dir_accounts_hard_links to hard_link_path

* Add dir_full_state flag for add_bank_snapshot

* Let appendvec files hardlinking work with multiple accounts paths across multiple partitions

* Fixes for rebasing

* fix tests which generates account_path without adding run/

* rebasing fixes

* fix account path test failures

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* update with more review comments

* handle error from create_accounts_run_and_snapshot_dirs

* fix rebasing duplicate

* minor accounts_dir path cleanup

* minor cleanup, remove commented code

* misc review comments

* build error fix

* Fix test_incremental_snapshot_download_with_crossing_full_snapshot_interval_at_startup

* fix build error on MAX_BANK_SNAPSHOTS_TO_RETAIN

* rebase fix, update hardlink filename

* minor comment spelling fix

* rebasing fixes

* fix rebase issues; with_extension

* comments changes for review

* misc minor review issues

* bank.fill_bank_with_ticks_for_tests

* error handling on appendvec path

* fix use_jit

* minor comments refining

* Remove type AccountStorages

* get_account_path_from_appendvec_path return changed to Option

* removed appendvec_path.to_path_buf in create_accounts_run_and_snapshot_dirs

* add test_get_snapshot_accounts_hardlink_dir

* update last_snapshot_storages comment

* update last_snapshot_storages comment

* symlink map_err

* simplify test_get_snapshot_accounts_hardlink_dir with fake paths

* log last_snapshot_storages at the end of the loop
2023-02-15 09:52:07 -08:00
Tyera a020f3eb60
Add clarifying comments to SamplePerformanceService (#30296)
* Add clarifying comment

* Make jsonrpc docs more explicit
2023-02-15 10:02:53 -07:00
Andrew Fitzgerald beb3cd5ed9
BankingStage Refactor: Separate Consumer Module (#30238) 2023-02-15 08:52:13 -08:00
Illia Bobyr d2b21c09ff
SamplePerformanceService: Refactor stats snapshot logic (#30297)
Snapshot construction and interaction code was a bit more manual than necessary, even causing a bug to slip past a review.  Separated snapshot construction from the diffing of two snapshots.
This should make the logic clearer.
2023-02-14 19:01:23 -08:00
Tyera 7c35191322
Scope SamplePerformanceService Bank only for initial sample snapshot (#30316)
* Scope Bank only for initial sample snapshot

* Remove nesting
2023-02-14 23:32:26 +00:00
steviez dd9d6e308c
Fix transactions counts stored by SamplePerformanceService (#30280)
A recent change to this service to store the number of non-vote
transactions introduced a bug in the computation of the number of
transactions during the time interval. This resulted in bogus values
being stored in Blockstore and eventually getting served through RPC for
the TPS chart on explorer.
2023-02-13 19:51:34 +00:00
Proph3t 2271a3920b
chore: fix broken docs link (#30274)
docs: fix broken link
2023-02-13 13:31:16 -06:00
Jeff Biseda f4fe550004
remove sleeps from repair tests (#30252) 2023-02-13 10:28:30 -08:00
Tao Zhu 60bfc2524b
implement From trait for CostTrackerError to TransactionError (#30267)
implement From trait for CostTrackerError to TransactionError
2023-02-13 11:06:39 -06:00
Trent Nelson 8770b15bb2
remove recommendations to skip validator startup tests on failure (#30250) 2023-02-10 18:14:47 -07:00
behzad nouri ded457cd73
embeds the new gossip ContactInfo in ClusterInfo (#30022)
Working towards replacing the legacy gossip contact-info with the new
one, the commit updates the respective field in gossip cluster-info.
2023-02-10 20:07:45 +00:00
Andrew Fitzgerald 60cf8ce65b
remove unnecessary lifetime (#30108)
Remove unnecessary lifetime on function
2023-02-09 21:17:41 -08:00
Jeff Biseda 180273b97d
defer HighestShred repairs during shred propagation threshold (#30142) 2023-02-09 14:57:55 -08:00
Ashwin Sekar 67f644473b
Fix repair behavior concerning our own leader slots (#30200)
panic when trying to dump & repair a block that we produced
2023-02-09 14:30:12 -07:00
Andrew Fitzgerald 4b17acf64e
BankingStage Refactor: Add state to Committer (#30107) 2023-02-09 13:22:42 -08:00
Andrew Fitzgerald 058738424d
BankingStage Refactor: transaction recorder record transactions (#30106) 2023-02-09 08:34:02 -08:00
steviez d3dab24bbe
chore: Use `i` over `ix` variable name when naming worker threads (#30206) 2023-02-09 01:24:57 +00:00
behzad nouri 1ad69cfc38
removes dynamic cast and dynamic dispatch from connection-cache (#30128)
Dynamic dispatch forces heap allocation and adds extra overhead.
Dynamic casting as in the ones below, lacks compile-time type safety:
https://github.com/solana-labs/solana/blob/eeb622c4e/quic-client/src/lib.rs#L172-L175
https://github.com/solana-labs/solana/blob/eeb622c4e/udp-client/src/lib.rs#L52-L55

The commit removes all instances of Any, Box<dyn ...>, and Arc<dyn ...>,
and instead uses generic and associated types.

There are only two protocols QUIC and UDP; and the code which has to
work with both protocols can use a trivial thin enum wrapper.

With respect to connection-cache specifically:
* connection-cache/ConnectionCache is a single protocol cache which
  allows to use either QUIC or UDP without any build dependency on the
  other protocol.
* client/ConnectionCache is an enum wrapper around both protocols and
  can be used in the code which has to work with both QUIC and UDP.

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2023-02-09 00:50:44 +00:00