#### Problem
To allow hot-storage to use HotStorageWriter::write_account() to
implement AccountsFile::append_accounts(), it is required to
provide a Vector of StoredAccountInfo to allow AccountsDB to
properly prepare the entry for each account.
#### Summary of Changes
This PR enables HotStorageWriter::write_account() to return
Vec<StoredAccountInfo>.
#### Test Plan
Extend existing tests for HotStorageWriter to verify the correctness
of the returned Vec<StoredAccountInfo>.
* Adding metrics for prioritization fees min/max per thread
* Adding scheduled transaction prioritization fees to the metrics
* Changes after andrews comments
* fixing Taos comments
* Adding metrics to the new scheduler
* Fixing getting of min max for TransactionStateContainer
* Fix clippy CI Issue
* Changes after andrews comments about min/max for new scheduler
* Creating a new structure to store prio fee metrics
* Reporting with prio fee stats banking_stage_scheduler_counts
* merging prioritization stats into SchedulerCountMetrics
* Minor changes after andrews review
This change makes solana-ledger-tool match solana-validator CLI; the old
flag --snapshot-archive-path is retained as an argument alias and can
still be used for the sake of backwards compatibility.
#### Problem
TieredStorageReader is a wrapper enum that works for
both Hot and Cold storage readers, but its get_account()
and account_matches_owner() API are missing.
#### Summary of Changes
Add get_account() and account_matches_owner() to
TieredStorageReader.
#### Test Plan
hot.rs offers similar coverage for HotStorageReader.
* fix bank new_from_field for epoch reward status
* fix bank serde test assert for epoch reward status
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
During a cluster upgrade when only half of the cluster can ingest the new shred
variant, sending shreds of the new variant can cause nodes to diverge.
The commit adds a feature to enable chained Merkle shreds explicitly.
#### Problem
In HotStorageReader, the account_matches_owners takes
&[&Pubkey] as the address candidates. However, it should
be &[Pubkey] as defined in the accounts_file API.
#### Summary of Changes
Correct HotStorageReader::account_matches_owners() to
take &[Pubkey] instead.
#### Test Plan
Existing unit-tests
#### Problem
Using non-reference type of AccountHash in
AccountMetaOptionalFields causes an unnecessary copy
as mentioned in #34948.
#### Summary of Changes
Uses &AccountHash in AccountMetaOptionalFields to
avoid copying.
#### Test Plan
Existing unit tests.
Fixes#34948
* gossip: notify state machine of duplicate proofs
* Add feature flag for ingesting duplicate proofs from Gossip.
* Use the Epoch the shred is in instead of the root bank epoch.
* Fix unittest by activating the feature.
* Add a test for feature disabled case.
* EpochSchedule is now not copyable, clone it explicitly.
* pr feedback: read epoch schedule on startup, add guard for ff recache
* pr feedback: bank_forks lock, -cached_slots_in_epoch, init ff
* pr feedback: bank.forks_try_read() -> read()
* pr feedback: fix local-cluster setup
* local-cluster: do not expose gossip internals, use retry mechanism instead
* local-cluster: split out case 4b into separate test and ignore
* pr feedback: avoid taking lock if ff is already found
* pr feedback: do not cache ff epoch
* pr feedback: bank_forks lock, revert to cached_slots_in_epoch
* pr feedback: move local variable into helper function
* pr feedback: use let else, remove epoch 0 hack
---------
Co-authored-by: Wen <crocoxu@gmail.com>
#### Problem
So far the current HotStorageWriter::write_accounts() only writes
accounts blocks and index block.
#### Summary of Changes
The PR further writes owners block in HotStorageWriter::write_accounts().
#### Test Plan
Extended existing test for HotStorageWriter to cover the owners block.
The accounts command currently dumps every single account in the
AccountsDb. This is obviously a lot of output, so a previous change
streamed the accounts instead of collecting and dumping at the end.
The streaming approach is much more performant, but the implementation
is non-trivial. This change
- Moves the accounts output code to output.rs
- Refactor the logic to several objects that implment the functionality
- Adjust the json output to also include the summary
This change lays the groundwork for cleanly adding several more flags
that will allow for querying different subsets of accounts.
This argument is a flag and doesn't take a value; however, it had the
.value_name() modifier set with "FORMAT". This could be confusing so
remove .value_name() and add .takes_value(false)
#### Problem
In HotStorageWriter::write_accounts, it skips storing rent-epoch when
the rent-epoch equals Epoch::MAX. While the value is correct, it is
more suitable to use RENT_EXEMPT_RENT_EPOCH instead as the
goal here is to save bytes for rent-exempt accounts.
#### Summary of Changes
Replace Epoch::MAX by RENT_EXEMPT_RENT_EPOCH when checking
whether to skip storing rent-epoch in HotStorageWriter.
* Check feature_set for enable_partitioned_epoch_reward
* Keep common variable outside if case
* Keep common early return out of if case, since the first_slot_in_epoch must exist for partiion PDA to exist
* Get and parse epoch partition data PDA
* Find partition index for all addresses
* Pull relevant blocks and get rewards
* Reuse ordering and reformatting
* Remove feature deactivation from TestValidator
* Restore rewards iteration in first block in epoch for feature case to catch Voting rewards
* Add fn get_reward_map helper to dedupe code
* No need to start 2nd get_block_with_limit call with first block again
* Replace filter_map to parameterize RewardType filter expression
* Weird thing to make clippy and compiler agree (https://github.com/rust-lang/rust-clippy/issues/8098)
* Use activated_slot to ensure the right approach for past rewards epochs
* Improve help text
* Add max_accounts cli arg and halt loop accordingly
* Log totals before max_account exit
* Add test of early end on max_accounts
* Add logging when account-creation loop breaks
Brooks
Alexander Meißner
Jon C
Yueh-Hsuan Chiang
galactus
steviez
Tao Zhu
Pankaj Garg
dependabot[bot]
HaoranYi
Lijun Wang
hana
behzad nouri
Tao Zhu
Ashwin Sekar
ellttBen
Tyera
Andrew Fitzgerald