2968587d42
add compare_with_partitioned_rewards ( #32108 )
2023-06-14 07:43:09 -07:00
67a434fc5b
add calculate_rewards_for_partitioning ( #32110 )
2023-06-13 15:45:59 -07:00
65268af497
pass get_reward_interval to load_accounts ( #32084 )
2023-06-13 14:49:07 -07:00
9774826b08
add calculate_validator_rewards ( #32107 )
2023-06-13 14:45:19 -07:00
dcd66534dd
add partitioned rewards helpers ( #32085 )
...
* add partitioned rewards helpers
* remove pub(crate)
* credit -> distribution
2023-06-13 13:57:52 -07:00
0762e69863
remove build error from merge ( #32104 )
2023-06-13 12:33:40 -07:00
3fc183ca74
add calculate_stake_vote_rewards for partitioned rewards ( #32066 )
...
* add calculate_stake_vote_rewards for partitioned rewards
* check stake reward in test
* pr feedback
* add comment
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-13 11:53:30 -07:00
446b82afba
update comment ( #32099 )
2023-06-13 10:54:38 -07:00
7c5c70a522
add compare_with_partitioned_rewards_results for testing partitioned rewards ( #32086 )
2023-06-13 07:39:46 -07:00
b89ce94d33
add store_vote_accounts_partitioned ( #32061 )
...
* add store_vote_accounts_partitioned
* pr feedback
2023-06-12 14:38:30 -07:00
9b2c9b8f5a
correct feature gate use_default_units_in_fee_calculation usage ( #32077 )
2023-06-12 16:29:21 -05:00
d0ae8c38bd
Fix - `enable_alt_bn128_syscall` ( #32083 )
...
* Fixes test_runtime_feature_enable_with_program_cache.
* Adds enable_alt_bn128_syscall to FEATURES_AFFECTING_RBPF.
2023-06-12 22:37:02 +02:00
6280c63e1a
epoch rewards are stored partitioned as Vec<Vec< ( #32076 )
...
* epoch rewards are stored partitioned as Vec<Vec<
* update comment
2023-06-12 13:06:22 -07:00
7e7c286d50
factor out get_epoch_reward_calculate_param_info to share results ( #32065 )
...
* factor out get_epoch_reward_calculate_param_info to share results
* remove extra comment
2023-06-12 09:56:08 -07:00
2e5b062809
Display path in error message when file create/write fails ( #32036 )
...
Using .unwrap() will panic and display the error; however, the path
isn't displayed which makes debugging harder.
2023-06-12 11:45:26 -05:00
0645e96bc6
add calc_vote_accounts_to_store ( #32063 )
...
* add calc_vote_accounts_to_store
* update comment
2023-06-12 08:38:18 -07:00
26f4c405b2
add update_reward_history_in_partition ( #32062 )
...
* add update_reward_history_in_partition
* remove explicit type
* remove spurrious comment
2023-06-12 08:37:46 -07:00
b81a060898
add store_stake_accounts_in_partition ( #32060 )
...
* add store_stake_accounts_in_partition
* pr feedback on assert
2023-06-12 08:37:22 -07:00
a001c5eb49
add calculate_reward_points_partitioned ( #32050 )
...
* add calculate_reward_points_partitioned
* remove unnecessary clone
* add comment
* clean up test from pr feedback
2023-06-12 08:36:42 -07:00
37efc82a76
snapshot epoch reward fields tests ( #32064 )
2023-06-12 07:39:59 -07:00
2572df7452
stake tests modification: next_epoch_and_n_slots ( #32052 )
...
stake tests modification: next_epoch_and_n_slots
2023-06-11 12:26:16 -07:00
4ebec905c5
add hash_rewards_into_partitions ( #31795 )
...
* add sort_and_shuffle_partitioned_rewards
* break stake accounts into partitions by hashing
* fmt
* Update runtime/src/epoch_rewards_hasher.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update runtime/src/bank/tests.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update runtime/src/bank/tests.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update runtime/src/bank.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* fix build
* more build fixes
* use spihasher13 for reward partition
* update comments
* revert calc reward struct change to avoid abi changes for now
* cippy
* sort deps
* saturating math
* use copy traits on hasher
use parent block hash to seed rewards partition
* fmt
* use concurrent map to speed up reward partition
* clippy
* Update runtime/src/epoch_rewards_hasher.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update runtime/Cargo.toml
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update runtime/src/bank.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* rename
* address review comments
* cargo.lock
* Update runtime/src/bank.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* update test
* clippy
* comments
* move hash_address_to_partition to epoch_rewards_hasher.rs
* tests to nail down width of partitions
* rename parameter to match with comments
* hash_address takes &self
* clippy
* EpochRewardHasher -> EpochRewardsHasher
* revert "hash_address takes &self" and refactor
* stake_rewards takes by value
---------
Co-authored-by: HaoranYi <haoran.yi@gmail.com>
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-09 11:55:01 -07:00
14e90cb27d
test ancient packing with recycling ( #32021 )
...
* test ancient packing with recycling
* fix holdover from debugging and update comment
2023-06-09 10:38:34 -07:00
4a566ec938
create and canonicalize account_paths ( #32037 )
...
Co-authored-by: Brooks <brooks@prumo.org>
2023-06-09 10:04:25 -07:00
948b66ee95
Removes #[allow(dead_code)] from ancient_append_vecs.rs ( #32047 )
2023-06-09 16:47:24 +00:00
4fde0b0d4a
remove packed ancient append vec from shrink candidate list ( #32022 )
2023-06-09 09:28:49 -07:00
31f2f569a6
simplify clean_dead_slot, factor out root stats ( #32035 )
2023-06-09 07:35:38 -07:00
afbb0fa92f
Removes init-time cleanup in add_bank_snapshot() ( #31997 )
2023-06-09 08:30:35 -04:00
bfe171017c
move cost model tests ( #32034 )
2023-06-08 15:02:28 -07:00
7cea6dd57c
[TieredStorage] ByteBlockWriter::write_optional_fields ( #32025 )
...
#### Summary of Changes
This PR implements the writer function for AccountMetaOptionalFields
in ByteBlockWriter::write_optional_fields().
#### Test Plan
Unit tests are included in this PR.
Tested in mnb w/ the prototype implementation of the tiered storage (#30626 )
2023-06-08 14:09:01 -07:00
634f925d75
[TieredStorage] AccountMetaOptionalFields ( #32005 )
...
#### Summary of Changes
This PR introduces AccountMetaOptionalFields, the in-memory struct for
handling optional fields.
#### Test Plan
Unit tests are included in this PR.
Tested in mnb w/ the prototype implementation of the tiered storage (#30626 ).
2023-06-08 12:31:27 -07:00
3ba05d9c3d
Fix move_and_async_delete_path ( #32020 )
2023-06-08 10:55:52 -07:00
4b2db0d546
Add explaining comment for Err case in get_bank_snapshots() ( #31998 )
2023-06-08 12:54:24 -05:00
37a759045a
include cost of transaction's requested loaded accounts size in cost model ( #31905 )
...
* include transaction requested loaded accounts size cost in cost model
* move function to avoid circular dependency
2023-06-08 12:49:42 -05:00
8937fd9bd6
improve packed ancient test by sweeping ( #32002 )
2023-06-08 07:50:18 -07:00
3f13cd353e
Fix - LoadedProgramType::Closed ( #31922 )
...
* Makes Bank::load_program() return correct tombstones.
* Removes early TX failure caused by closed and invalid programs.
* Adjusts the feature gate of simplify_writable_program_account_check.
2023-06-08 15:40:18 +02:00
ddbc35fcc3
Add logging to move_and_async_delete_path ( #32017 )
2023-06-07 16:31:16 -04:00
36ccc71220
move StakeReward to stake_rewards.rs ( #32014 )
2023-06-07 12:35:28 -07:00
c87ff8780a
Cleanup - `Bank::reconfigure_token2_native_mint()` ( #31817 )
...
Removes Bank::reconfigure_token2_native_mint().
2023-06-07 21:14:43 +02:00
b1db4ccc9a
[TieredStorage] Make AccountMetaFlags and its fields public ( #32004 )
...
#### Problem
AccountMetaFlags and its fields are required to be public as the visibility of their get
functions generated by `bit_fields` are tied to the visibility of their fields, and the
get functions will later be used to support the TieredAccountMeta for AccountMeta.
#### Summary of Changes
Make AccountMetaFlags and its fields public
2023-06-07 11:37:56 -07:00
0f215575ca
add RewardInterval::InsideInterval ( #31799 )
...
* add RewardInterval::InsideInterval
* update abi hash
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-07 09:56:36 -07:00
ee2c2ef6c7
Cleanup - require_static_program_ids_in_transaction ( #31767 )
...
require_static_program_ids_in_transaction
2023-06-07 17:12:41 +02:00
c86e160438
Add runtime environment to FailedVerification tombstones ( #31991 )
...
* Add runtime environment to FailedVerification tombstones
* modify default variant
* prune update
* add DelayVisibility in prune_feature_set_transition
2023-06-06 22:24:39 +00:00
85a0e31c60
add ProgramExecutionTemporarilyRestricted error ( #31796 )
...
* add StakeProgramUnavailable error
* rename to ProgramExecutionTemporarilyRestricted
* Update sdk/src/transaction/error.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* fmt
* update frozen abi hash
* add account_index to ProgramExecutionTemporarilyRestricted error
* Update sdk/src/transaction/error.rs
Co-authored-by: mvines <mvines@gmail.com>
* populate transaction_details from transaction_error for ProgramExecutionTemporarilyRestricted error
* fix a test
* update abi digest
* Update sdk/src/transaction/error.rs
Co-authored-by: Brooks <brooks@prumo.org>
---------
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
Co-authored-by: HaoranYi <haoran.yi@solana.com>
Co-authored-by: HaoranYi <haoran.yi@gmail.com>
Co-authored-by: mvines <mvines@gmail.com>
Co-authored-by: Brooks <brooks@prumo.org>
2023-06-06 14:13:41 -07:00
71c78f7a21
Will failed transaction should be considered has zero execution cost by cost model ( #31976 )
2023-06-06 15:45:45 -05:00
f08fb3e158
Downgrades log to debug if reading bank snapshot from dir fails ( #31990 )
2023-06-06 15:07:45 -04:00
83ac66caa1
split TransactionCost to its own module ( #31977 )
...
* split TransactionCost to its own module
* remove unused function
2023-06-06 11:15:38 -05:00
89207a3fe5
Feature - Prune on feature set transition ( #31945 )
...
* Call create_program_runtime_environment() in Bank::apply_builtin_program_feature_transitions().
* Adds LoadedPrograms::prune_feature_set_transition().
2023-06-06 17:49:02 +02:00
40d3e8e8fa
[TieredStorage] Enable ByteBlockWriter to report the length of raw data ( #31900 )
...
#### Summary of Changes
This PR adds raw_len() to ByteBlockWriter which returns
the length of the data after decoding. This information is needed
for the cold storage to keep track of the current size of the account
block after decompression.
#### Test Plan
Improved existing tests to verify the returned raw_len().
Tested in mnb w/ the prototype implementation of the tiered storage (https://github.com/solana-labs/solana/pull/30626 )
2023-06-02 14:01:38 -07:00
37ebb709e6
Evict program cache using transaction usage counter ( #31927 )
...
* Evict program cache using tx usage counter
* address review feedback
* address review feedback
2023-06-02 11:26:56 -07:00
10995f66c0
use one definition of builtin program cost in runtime and banking stage ( #31930 )
...
* use one definition of builtin program cost in runtime and banking stage
* add loader_v4
2023-06-02 10:22:22 -05:00
858ea5a143
chore: Add name string for bank_serialize measure! ( #31926 )
...
The measurement is later logged, so include a name to more clearly
identify what the measurement was.
2023-06-02 06:54:04 -05:00
29d7a57cce
Reset program cache stats after submit ( #31929 )
2023-06-02 01:16:49 -07:00
4353ac6797
Pass Arc<AtomicBool> by value, not by reference. ( #31916 )
...
`Arc` is already a reference internally, so it does not seem to be
beneficial to pass a reference to it. Just adds an extra layer of
indirection.
Functions that need to be able to increment `Arc` reference count need
to take `Arc<AtomicBool>`, but those that just want to read the
`AtomicBool` value can accept `&AtomicBool`, making them a bit more
generic.
This change focuses specifically on `Arc<AtomicBool>`. There are other
uses of `&Arc<T>` in the code base that could be converted in a similar
manner. But it would make the change even larger.
2023-06-01 17:25:48 -07:00
2b04f28c7b
[TieredStorage] AccountMetaFlags ( #31913 )
...
#### Summary of Changes
This PR implements AccountMetaFlags, which will later be used in
the tiered accounts storage to support optional fields.
#### Test Plan
Unit tests are included in this PR.
2023-06-01 14:48:27 -07:00
1302962521
[TieredStorage] Support Lz4 in ByteBlockWriter (2/N) ( #31862 )
...
#### Summary of Changes
Support Lz4 in ByteBlockWriter.
#### Test Plan
Added test cases for Lz4.
Tested in mnb w/ the prototype implementation of the tiered storage (#30626 )
2023-06-01 10:29:08 -07:00
debe794987
Replace improper &Arc<...> with Arc<...> in Bank and Accounts ( #31892 )
...
The callstack updated in this PR passed an &Arc<...> down only to have
the bottom level clone the reference. Thus, we are giving shared
ownership so the reference is a bit redundant and arguably obscures the
intention to clone further down the callstack.
2023-05-31 12:36:44 -05:00
8e191bbc56
Downgrades log to warn if reading bank snapshot from dir fails ( #31885 )
2023-05-30 19:35:06 -04:00
33d64c6b2f
[TieredStorage] Add doc comments for byte_block.rs ( #31863 )
...
#### Summary of Changes
Add doc comments for byte_block.rs
#### Test Plan
This is a comment-only PR. No code change.
2023-05-30 16:25:25 -07:00
c569a37713
Handle errors when sending an accounts package during shutdown ( #31874 )
2023-05-30 17:42:33 -04:00
5334efd63c
Byte-blocks utils for TieredStorage (1/N) ( #31815 )
...
#### Summary of Changes
This PR includes the util structs for writing byte blocks, which will be used
by the TieredStorage writer (currently used in the prototype #30626 ).
The PR only includes the Raw encoding format. Will add more in subsequent PRs.
#### Test Plan
Tested in mnb with the prototype implementation of the tiered-storage (#30626 )
Unit tests are also included in this PR.
2023-05-30 14:14:29 -07:00
6c520396cd
bank process_new_epoch takes parent_height ( #31820 )
2023-05-30 08:58:40 -05:00
588a7a4e53
add RewardsStoreMetrics for partitioned rewards ( #31819 )
2023-05-30 08:58:00 -05:00
0c64e26d53
partitioned rewards helper functions ( #31818 )
2023-05-30 08:57:31 -05:00
8a3f446db0
Removes MAX_BANK_SNAPSHOTS_TO_RETAIN constant ( #31827 )
2023-05-25 17:13:28 -04:00
75954dc061
Banking worker ( #30970 )
2023-05-25 10:16:04 -07:00
0e93090520
Updates priority for which snapshot request to handle next ( #31798 )
2023-05-24 16:21:06 -04:00
6752019a80
Add enum AppendVecError ( #31749 )
...
#### Problem
AppendVec currently uses std::io::ErrorKind::Other for its own errors.
#### Summary of Changes
This PR introduces AppendVecError and has AppendVec use it.
2023-05-24 12:18:17 -07:00
461342cdd4
add cli args and use of partitioned rewards config ( #31800 )
...
* add cli args and use of partitioned rewards config
* update comments
2023-05-24 10:54:09 -07:00
49259dcad8
add optional ignore to `calculate_accounts_delta_hash` ( #31801 )
2023-05-24 10:51:03 -07:00
d6a54e0f4d
add partitioned_rewards.rs ( #31794 )
2023-05-24 09:59:38 -07:00
a85548cf34
Cleanup - spl_token_v3_4_0 and spl_associated_token_account_v1_1_0 ( #31764 )
...
* spl_token_v3_4_0
* spl_associated_token_account_v1_1_0
* Marks replace_program_account() as dead code.
2023-05-24 18:45:37 +02:00
e396a10312
change StartBlockHeightAndRewards from parent to current ( #31781 )
2023-05-23 20:10:51 -07:00
ae12079139
add debug to VoteReward ( #31782 )
2023-05-23 20:10:20 -07:00
8e8b2f1671
reformat comments ( #31780 )
2023-05-23 17:04:47 -07:00
f52ded35f4
async delete contents but leave directory ( #31737 )
...
* async delete contents but leave directory
* Clarified comment
2023-05-23 15:33:09 -07:00
05c932f92d
Moves transient accounts hash cache files into their own dir ( #31778 )
2023-05-23 22:14:30 +00:00
174951478f
Add new error type -- TieredStorageError ( #31633 )
...
#### Summary of Changes
This PR adds a new error type for the tiered storage.
2023-05-23 15:06:55 -07:00
a474cb24b9
Creates temporary accounts hash cache files inside accounts hash cache dir ( #31776 )
2023-05-23 17:09:16 -04:00
61afb07dbd
Purges old bank snapshots at startup ( #31656 )
...
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-05-23 13:18:05 -04:00
5448d0b1e8
Remove redundant clones ( #31685 )
...
* broadcast_stage remove redundant clone
* broadcast_fake_shreds_run remove redundant clone
* wallet remove redundant clone
* cluster_query remove redundant clone
* accounts remove redundant clones
2023-05-23 09:42:05 -07:00
1e8e024ec5
Cleanup - disable_fee_calculator ( #31766 )
...
disable_fee_calculator
2023-05-23 18:20:38 +02:00
e9d99973ca
Cleanup - bank_transaction_count_fix ( #31768 )
...
bank_transaction_count_fix
2023-05-23 18:18:15 +02:00
d866fce3d8
Cleanup - include_account_index_in_rent_error ( #31770 )
...
include_account_index_in_rent_error
2023-05-23 15:02:55 +02:00
183e7fd60f
Cleanup - instructions_sysvar_owned_by_sysvar ( #31769 )
...
instructions_sysvar_owned_by_sysvar
2023-05-23 15:02:35 +02:00
d9d2120d32
Make reconstruct_single_storage() returns SnapshotError ( #31748 )
...
#### Problem
The return error type -- io::Error used in reconstruct_single_storage()
isn't suitable to describe all possible errors inside the function such as
AccountsFileError.
#### Summary of Changes
This PR makes it return SnapshotError which can better handle
both AccountsFileError and io::Error.
2023-05-23 02:18:23 -07:00
586fd407bf
Refactor SnapshotError related code ( #31632 )
...
#### Problem
Part of the snapshot-related code uses io::Error while other parts use SnapshotError.
#### Summary of Changes
As SnapshotError can be created from io::Error, this PR makes snapshot-related
code to use SnapshotError instead.
2023-05-22 15:33:11 -07:00
c79a6e74e2
Add new error type AccountsFileError ( #31631 )
...
#### Problem
AccountsFile- and AppendVec-related code uses io::Error for their main error type. However, there're many errors under AccountsFile that don't belong to io::Error.
#### Summary of Changes
This PR introduces AccountsFileError and makes minimum changes to keep the PR small.
Subsequent changes related to this will be in separate PRs.
2023-05-22 14:09:09 -07:00
83f692ce67
Refactor - Share RBPF config and syscall registry globally ( #31727 )
...
* Adds LoadedPrograms::program_runtime_environment_v1.
* No need to explicitly remove precompiles from program_accounts_map.
2023-05-20 20:28:00 +02:00
e6d490708a
Do not overwrite builtin program in the same slot ( #31718 )
2023-05-19 05:26:31 -07:00
d391e75a60
Performance: Allowed forced inline update_index (no thread pool) ( #31455 )
2023-05-18 13:32:41 -07:00
520c647918
Refactor - Merges `BuiltinPrograms` into `LoadedPrograms` ( #31654 )
...
* Merges BuiltinPrograms into LoadedPrograms.
* Prevents built-ins from being pruned.
2023-05-18 22:18:28 +02:00
9fd6f09984
remove redundant `mut`s ( #31696 )
2023-05-18 11:08:13 -07:00
3ac5bb2a60
deserialize epoch_reward_status into BankFieldsToDeserialize ( #31698 )
2023-05-18 07:14:59 -07:00
0291454876
persist EpochRewardStatus::Active ( #31697 )
...
persist `EpochRewardStatus::Active`
2023-05-18 07:14:41 -07:00
3f625e47df
working towards persistence for EpochRewardStatus ( #31678 )
2023-05-17 13:23:20 -07:00
8ed4962229
refactor serialization tuple into an enum ( #31679 )
2023-05-17 10:53:28 -07:00
ce32cb43db
add bank.epoch_reward_status ( #31680 )
2023-05-17 10:01:14 -07:00
b03422968b
Remove Rc and RefCell to carry program cache in invoke_context ( #31684 )
2023-05-17 06:10:50 -07:00
7c6478da8f
remove unnecessary pub ( #31676 )
2023-05-16 13:53:57 -07:00
Jeff Washington (jwash)
Tao Zhu
Alexander Meißner
steviez
Andrew Fitzgerald
Brooks
Yueh-Hsuan Chiang
Pankaj Garg
Illia Bobyr