a8e07c41fe
Adds `is_incremental_accounts_hash_feature_enabled` to `AccountsPackage` ( #30368 )
2023-02-20 12:21:38 -05:00
ad6a3abd8b
Uses set_accounts_{delta,}_hash_for_tests() in serde tests ( #30390 )
2023-02-17 16:00:25 -05:00
f91e63dfe9
Upgrades Rust to 1.67.1 ( #29947 )
2023-02-17 14:08:27 -05:00
3a4ba72daf
Remove executors trait and replace with LoadedProgram ( #30348 )
...
* Remove executors trait and replace with LoadedProgram
* fill in deployment slot
* address review comments
* fix clippy warnings
* address review comments
* fix failures caught by sbf tests
2023-02-17 12:47:45 +01:00
6c58033df5
API to enumerate executable program accounts in tx batch ( #30371 )
2023-02-16 13:29:38 -08:00
febaf36e6d
Apply clippy fixes for future rust upgrade ( #30363 )
2023-02-16 16:12:51 +00:00
fa22389b22
Add version and state_complete flag into bank snapshot ( #30099 )
...
* Add version and state_complate flag into bank snapshot
* Add test_get_highest_full_snapshot_slot_and_path
* fix build checks
* fix slot_deltas panic in test_get_highest_full_snapshot_slot_and_path
* make bank rooted for test_get_highest_full_snapshot_slot_and_path
* fix bank scope problem in test_get_highest_full_snapshot_slot_and_path
* minor cleanup
* misc review issues
* remove unneeded error defs and functions
* Fix state_complete macro naming
* fix snapshot_version
2023-02-15 22:38:40 -08:00
ce4251be99
Fix the return type of AccountsFile::account_matches_owners() ( #30354 )
...
Update AccountsFile::account_matches_owners() to return the matched index
2023-02-16 03:34:36 +00:00
aeb6df3507
Abstract out AppendVec into AccountsFile enum ( #29815 )
...
Abstracts out AppendVec into AccountsFile enum.
This will allow different implementations of AccountsFile that support the AccountsDB.
2023-02-15 17:24:04 -08:00
5680c76c86
Refactors Accounts::new() fns ( #30343 )
2023-02-15 18:34:52 -05:00
615d1a8b69
Extract BankHashLamportsVerifyConfig ( #30320 )
...
* refactor BankHashLamportsVerifyConfig
* clippy
* comments
* fix bench
2023-02-15 16:03:50 -06:00
b86bfbb5c5
measure_us! use Instant and duration_to_us internally ( #30339 )
2023-02-15 12:43:47 -08:00
3bea5fc326
Return index of matching owner for a program account ( #30337 )
2023-02-15 19:41:29 +00:00
4909267c88
Add accounts hard-link files into the bank snapshot directory ( #29496 )
...
* Add accounts hard-link files into the bank snapshot directory
* Small adjustments and fixes.
* Address some of the review issues
* Fix compilation issues
* Change the latest slot snapshot storage from VecDeque to Option
* IoWithSourceAndFile and expanded comments on accounts
* last_slot_snapshot_storages in return value
* Update comments following the review input
* rename dir_accounts_hard_links to hard_link_path
* Add dir_full_state flag for add_bank_snapshot
* Let appendvec files hardlinking work with multiple accounts paths across multiple partitions
* Fixes for rebasing
* fix tests which generates account_path without adding run/
* rebasing fixes
* fix account path test failures
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* update with more review comments
* handle error from create_accounts_run_and_snapshot_dirs
* fix rebasing duplicate
* minor accounts_dir path cleanup
* minor cleanup, remove commented code
* misc review comments
* build error fix
* Fix test_incremental_snapshot_download_with_crossing_full_snapshot_interval_at_startup
* fix build error on MAX_BANK_SNAPSHOTS_TO_RETAIN
* rebase fix, update hardlink filename
* minor comment spelling fix
* rebasing fixes
* fix rebase issues; with_extension
* comments changes for review
* misc minor review issues
* bank.fill_bank_with_ticks_for_tests
* error handling on appendvec path
* fix use_jit
* minor comments refining
* Remove type AccountStorages
* get_account_path_from_appendvec_path return changed to Option
* removed appendvec_path.to_path_buf in create_accounts_run_and_snapshot_dirs
* add test_get_snapshot_accounts_hardlink_dir
* update last_snapshot_storages comment
* update last_snapshot_storages comment
* symlink map_err
* simplify test_get_snapshot_accounts_hardlink_dir with fake paths
* log last_snapshot_storages at the end of the loop
2023-02-15 09:52:07 -08:00
2e885cf99a
Removes calls to insert_default_bank_hash_stats() in tests ( #30311 )
2023-02-14 15:00:38 -05:00
b9b8cd9ccc
misc. cleanup ( #30295 )
2023-02-14 12:00:13 -08:00
27f18f96e8
typos, clippy and saturating_add_assign ( #30272 )
...
* typo
* remove clippy allow
---------
Co-authored-by: haoran <haoran@mbook>
2023-02-14 10:03:16 -06:00
253517cba3
add combine_ancient_slots_packed ( #30276 )
...
* add combine_ancient_slots_new
* pr feedback
* implement ==
2023-02-14 06:53:31 -08:00
9988978379
add ancient finish_combine_ancient_slots_packed ( #30261 )
2023-02-13 09:10:13 -08:00
60bfc2524b
implement From trait for CostTrackerError to TransactionError ( #30267 )
...
implement From trait for CostTrackerError to TransactionError
2023-02-13 11:06:39 -06:00
bcd7cf0821
add ancient write_packed_storages ( #30260 )
2023-02-13 07:25:47 -08:00
b3778d5044
cleanup ( #30251 )
2023-02-12 13:42:03 -08:00
dc605b5126
rename parameter and update comments ( #30259 )
2023-02-11 12:13:03 -08:00
c837af1af4
ancient: pass WriteAncientAccounts by &mut ( #30257 )
2023-02-11 11:41:33 -08:00
6558c8fdc9
Refactor - Delay visibility of program un-/re-/deployment ( #29654 )
...
* Use three separate HashMaps instead of the enum TxBankExecutorCacheDiff.
* Replaces all places which deploy programs by a macro.
* Adds a feature gate.
* Adjust tests.
* Makes undeployment visible immediately.
2023-02-11 11:18:25 +01:00
51d354defb
simplify AccountsToCombine.accounts_keep_slots ( #30246 )
2023-02-10 13:53:03 -08:00
a46ba32e9a
remove unused metric ( #30248 )
2023-02-10 13:36:22 -08:00
b3c2256799
Refactors a sort's comparator fn ( #30243 )
2023-02-10 15:41:55 -05:00
33cc4cc3d5
maintain sorted order on ancient target slots ( #30247 )
2023-02-10 12:08:13 -08:00
157a9d725f
add ancient pack_ancient_storages ( #30235 )
...
* add ancient pack_ancient_storages
* cleanup
2023-02-10 12:06:36 -08:00
0b5ae8abe1
Refactor - simplify geyser notify book keeping code and fix total_accounts stat ( #30207 )
...
* refactor common code
* fix total_accounts stat
* clippy
2023-02-10 12:18:50 -06:00
12411e5215
pass shrink_can_be_active through shrink fns ( #30244 )
2023-02-10 10:06:40 -08:00
dcb2d6c8ae
collect_sort_filter_ancient_slots takes tuning by ref ( #30242 )
2023-02-10 09:40:13 -08:00
0113fe340e
add ancient write_ancient_accounts_multiple_refs ( #30198 )
2023-02-10 09:39:30 -08:00
d27c860249
Replaces stable sort with unstable sort ( #30223 )
2023-02-10 09:56:11 -05:00
128ffbdf9b
restore test to all slot combos ( #30231 )
2023-02-10 06:25:37 -08:00
3e6162e69e
Add address lookup tables to minimized snapshot ( #30158 )
...
* Add address lookup tables to minimized snapshot
* Add comment for future posterity
* Add reference to the issue
* Adjust comment a bit
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
---------
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-02-10 14:46:02 +09:00
fc8c590375
improve debugging test fn compare_all_accounts ( #30218 )
2023-02-09 15:06:09 -08:00
639f3475f5
test and comment cleanup ( #30226 )
...
* test and comment cleanup
* pr feedback
2023-02-09 15:02:10 -08:00
db25ccba52
ancient add write_one_packed_storage ( #30220 )
2023-02-09 14:47:22 -08:00
2f9146e8c8
Replace txn_signature in ReplicaAccountInfo with transaction ( #30189 )
...
#### Current implementation of ReplicaAccountInfo restricts
possibilities for inflight account filtration
Current implementation includes transaction signature in
ReplicaAccountInfo. This approach does not allow to filter accounts by
matching other accounts in transaction in-flight (e. g. accept only
those accounts included in transactions for specific programs). Current
implementation forces to collect ALL accounts and transactions for some
period of time and perform such complex filtration afterwards.
#### Pass reference to transaction object instead of transaction
signature into ReplicaAccountInfo
Advanced in-flight filtration can be implemented in plugins by passing
reference to transaction for every update_account event. This change
doesn't bring any overhead comparing to current implementation (only
data type of reference is changed) and brings only minor changes in
source code.
2023-02-09 12:00:33 -08:00
c6695996b8
add test parameters to fix build ( #30227 )
...
#### Problem
build broken due to pr conflicts
#### Summary of Changes
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:56:12 -08:00
62b93df1c7
change ancient tests to correct size ( #30221 )
...
#### Problem
fix tests to be more realistic
#### Summary of Changes
allocate correctly sized append vecs
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:41:36 -08:00
e0d354f542
fix bug in test fn compare_all_accounts ( #30217 )
...
#### Problem
test code compares sets of accounts after clean/shrink operations.
The indexing was off and new tests were giving false positives.
The test code was equivalent as long as everything was in the same
order.
#### Summary of Changes
Fix test code comparison indexes.
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:28:53 -08:00
68b2b1badd
add ancient calc_accounts_to_combine ( #30195 )
...
#### Problem
Building new algorithm for packing ancient storage. Packing will occur
in 1 pass across multiple ancient slots.
This will be put in 1 dead code piece at a time with tests until all
pieces are present. Switch between current packing algorithm and this
new one is in a validator cli argument. Resulting append vecs are
correct and compatible (as a set) either way. When a new storage format
optimized for cold storage becomes available, it will only work with
this new packing algorithm, so the change will need to be complete prior
to the new storage format.
#### Summary of Changes
Add `ancient calc_accounts_to_combine` to separate accounts to prepare
for creating packed ancient append vecs. This will be used soon.
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:06:17 -08:00
a4fb1834d1
AliveAccounts includes source slot ( #30208 )
2023-02-09 05:57:58 -08:00
5569946a8f
correct a comment with right info ( #30201 )
2023-02-08 17:33:28 -06:00
375f9ae41d
LoadedPrograms cache implementation and tests ( #30139 )
2023-02-08 13:24:44 -08:00
0ca638648b
add shrink_can_be_active to remove ( #30191 )
2023-02-08 11:32:18 -08:00
b4fe1280b3
add Debug to some structs ( #30193 )
2023-02-08 10:17:07 -08:00
427bd62642
handle_dropped_roots_for_ancient takes iter ( #30190 )
2023-02-08 09:40:49 -08:00
0da4697b2f
add ancient write_ancient_accounts ( #30182 )
2023-02-08 09:40:17 -08:00
5cbd6b3a74
minor cleanup ( #30181 )
2023-02-08 08:00:14 -08:00
70d635127e
add `ancient collect_sort_filter_info` ( #30169 )
2023-02-08 07:08:20 -08:00
e14d0638e7
add ancient get_unique_accounts_from_storage_for_combining_ancient_slots ( #30176 )
2023-02-07 18:22:54 -08:00
d0623a38c8
API to match account owner against a set of owners ( #30154 )
2023-02-07 17:48:10 -08:00
bfc29a6b2f
remove storage_finder from write_accounts_to_storage ( #30172 )
...
Co-authored-by: Trent Nelson <trent@solana.com>
2023-02-07 14:44:40 -08:00
1eb304de28
NonZeroU64 for ideal_storage_size ( #30168 )
2023-02-07 13:34:27 -08:00
927da903a8
StoreTo::Storage holds storage ( #30156 )
2023-02-07 11:54:26 -08:00
d53c49c230
add ancient filter_ancient_slots ( #30152 )
2023-02-07 10:35:40 -08:00
3f913e49fc
refactor rent payment order cmp ( #30132 )
2023-02-07 09:30:48 -06:00
5dd90268b1
comments fix ( #30134 )
...
Co-authored-by: haoran <haoran@mbook>
2023-02-07 09:22:08 -06:00
db4174cbcb
ancient choose_storages_to_shrink ( #30146 )
2023-02-06 14:29:54 -08:00
a14473eb54
Prevent Rent-reward recipients from ending up RentPaying ( #30130 )
2023-02-06 12:16:36 -07:00
01f0dcdad4
add ancient ancient filter_by_smallest_capacity and truncate_to_max_storages ( #30129 )
2023-02-06 11:04:53 -08:00
8785942f56
add ancient sort_shrink_indexes_by_bytes_saved ( #30123 )
2023-02-05 12:14:14 -08:00
0bbc4774d7
minor cleanup on tests ( #30125 )
2023-02-05 12:13:39 -08:00
785a6e3a69
AncientSlotInfos, piece of packed ancient storage ( #30119 )
2023-02-03 10:53:49 -08:00
06e801a8a5
Moves BankHashInfo into serde_snapshot ( #30120 )
2023-02-03 13:17:50 -05:00
4293f11cf1
feature gate to enable compute_budget::request_heap_frame on mainnetBeta ( #30077 )
2023-02-03 09:21:57 -06:00
9b297ac89c
Removes {get,set}_bank_hash_info() test helper functions ( #30101 )
2023-02-03 10:17:38 -05:00
eeb622c4ed
test create storages can ignore TempFile ( #30094 )
2023-02-03 07:12:33 -08:00
8312141f87
Move bank tests to separate file ( #29912 )
2023-02-02 16:32:22 -08:00
3249e4a123
Removes {get,set}_bank_hash_info_from_snapshot() ( #30087 )
2023-02-02 16:50:44 -05:00
ae7803a55e
add test accounts with custom data size ( #30089 )
2023-02-02 10:47:12 -08:00
890f29be0c
fix alive_bytes test fn ( #30088 )
2023-02-02 10:46:46 -08:00
d563d37860
combine lookups in StorableAccountsWithHashesAndWriteVersions ( #30061 )
2023-02-02 10:46:17 -08:00
385a6e01f7
combine some stats into ShrinkStatsSub ( #30034 )
2023-02-02 06:21:36 -08:00
a6f43c2e3d
Slot 0 is not as special in set_bank_hash_info_from_snapshot() ( #30071 )
2023-02-02 08:57:11 -05:00
7ea0a72cc4
total_bytes -> capacity ( #30065 )
2023-02-02 05:46:13 -08:00
79e9c9f8b7
Do not populate slot 0 with default accounts delta hash/accounts hash ( #30069 )
2023-02-01 20:43:26 -05:00
22afdb21ec
rename original_bytes -> capacity ( #30064 )
2023-02-01 16:58:41 -08:00
f107b8b607
Add slot deltas into the bank snapshot directory ( #29409 )
2023-02-01 16:51:32 -08:00
232e252014
Bump serde from 1.0.144 to 1.0.152 ( #29696 )
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera <tyera@solana.com>
2023-02-01 16:27:17 -07:00
33165040bf
Only insert default BankHashStats in Accounts::new_from_parent() ( #30063 )
2023-02-01 17:47:55 -05:00
090b990e15
remove congestion multiplier from transaction fee calculation ( #29828 )
2023-02-01 16:26:50 -06:00
67d3beeb35
remove unnecessary lifetime ( #30019 )
2023-02-01 13:18:46 -08:00
e11c22a13c
clean up remove_old_stores_shrink stats ( #30018 )
2023-02-01 13:18:07 -08:00
1c4e84b8ca
impl default version of contains_multiple_slots ( #30055 )
2023-02-01 13:01:01 -08:00
5769dbf36b
is_cached_store, bool -> Storage enum ( #30057 )
2023-02-01 12:33:27 -08:00
0cf93da4b4
remove obsoleted else-if clause ( #30020 )
2023-02-01 14:28:35 -06:00
3b2c2ebf9e
add StorableAccountsBySlot to move accounts from multiple slots to 1 ( #30023 )
2023-02-01 12:10:12 -08:00
c8ed54acbd
simplify 2 StorableAccounts impls ( #30054 )
2023-02-01 12:08:28 -08:00
d048a1903f
Splits up AccountsDb::bank_hashes ( #30024 )
2023-02-01 14:32:35 -05:00
27e8af9621
add pub(crate) to shrink functions ( #30052 )
2023-02-01 11:04:51 -08:00
7bcb3cef02
report less datapoints to block_min_prioritization_fee ( #30017 )
2023-02-01 11:32:36 -06:00
5d85a34591
shrink ignores ancient slots ( #30026 )
2023-02-01 07:23:14 -08:00
28510f4260
update comments on get_ancient_append_vec_capacity ( #30030 )
2023-02-01 07:22:34 -08:00
3e0494051e
tests adding accounts can mark alive in index ( #30029 )
2023-02-01 07:22:10 -08:00
d5a7b5715c
add cli arg for choosing ancient append vec format ( #30025 )
2023-02-01 07:21:28 -08:00
a5af54669a
Limit loaded data per transaction to a fixed cap ( #29743 )
2023-01-31 22:51:35 -06:00
179b406acb
runtime: make {Stored,Account}Meta repr(C) ( #30027 )
...
Co-authored-by: Richard Patel <ripatel@jumptrading.com>
2023-01-31 20:13:56 -05:00
04a6a631bc
Bump version to v1.16 ( #30028 )
2023-01-31 17:48:33 -06:00
1e3d6349aa
Add an RPC API that can query the list of Top N secondary index keys and their sizes ( #28887 )
...
Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2023-01-31 13:47:03 -07:00
1ca78845bb
runtime: add test_type_layout ( #29987 )
...
adds unit test asserting the type layout of structures in append_vec
---------
Co-authored-by: Richard Patel <ripatel@jumptrading.com>
Co-authored-by: brooks <brooks@solana.com>
2023-01-31 11:08:55 -05:00
4dbde40e10
measure! -> measure_us! ( #29991 )
2023-01-31 07:40:50 -08:00
a12bf8c003
Update maintainers references ( #29997 )
...
* update maintainers references
* chore: update maintainers reference
2023-01-31 08:07:13 -05:00
aa72aa8779
Makes AccountsDb::bank_hashes private ( #29990 )
2023-01-30 23:25:19 -05:00
005dc96bc5
serde-snapshot uses methods to access AccountsDb::bank_hashes ( #29989 )
2023-01-30 21:32:04 -05:00
ebf3ad5604
remove unnecessary lifetimes ( #29988 )
2023-01-30 14:11:10 -08:00
8a56a856d8
refactor shrink_collect ( #29982 )
...
* refactor shrink_collect
* Update runtime/src/accounts_db.rs
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
---------
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-01-30 13:10:39 -08:00
3e5ab0437f
remove create_ancient_append_vec ( #29984 )
2023-01-30 13:07:15 -08:00
ea6dac0556
refactor update_shrink_stats ( #29983 )
2023-01-30 11:11:13 -08:00
856598969c
Account path add run parent with old path cleanup ( #29942 )
...
* Add run parent directory for accounts files
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* remove explict type def for map result
* handle create_accounts_run_and_snapshot_dirs error with expect
* update with more review comments
* minor fixes from review comments
* simplify account_filename option assignment
* handle error from create_accounts_run_and_snapshot_dirs
* use then instead of then_some for lazy evaluation
* Clean up files in the old account_path before trasitioning to the new run path
* try_exist and accounts_dir removing extra
* sync rmdir, is_dir check
* handle the account_path not deletable case
2023-01-30 10:26:43 -08:00
7ef720b76d
refactor: remove obsoleted function ( #29964 )
...
refactor: remove obsolated function
2023-01-30 09:16:49 -06:00
ac7d0ff633
Feature - Disable builtin loader ownership chains ( #29959 )
...
* Adds feature disable_builtin_loader_ownership_chains.
* Removes test_load_accounts_max_call_depth().
2023-01-29 19:27:49 +01:00
aa2e3487ba
Refactor - Simplify program accounts in transaction loading ( #29728 )
...
* Refactors the "!validated_fee_payer" case from an "else" branch to an early "return".
* Moves the early return upward.
* Removes empty entries.
* Adds account_found_and_dep_index.
* cargo fmt.
* Replaces call site of load_executable_accounts().
* Adjusts number of total loaded accounts in test_load_accounts_multiple_loaders().
* Removes test_accounts_account_not_found().
* Removes load_executable_accounts().
* Refactor back to built-in loader ownership chain loop.
2023-01-27 21:24:21 +01:00
5e35823b66
add test_stake_account_consistency_with_rent_epoch_max_feature ( #29915 )
...
* add test_stake_account_consistency_with_rent_epoch_max_feature
* create_stake_account takes id
* use test_case
* reformat panic message
2023-01-27 11:50:33 -08:00
6f4fe37bd2
add cost to additional built-in programs ( #29795 )
...
* add cost to additional built-in programs
* update a test with compute-budget cost
2023-01-27 13:47:41 -06:00
8fafbb0a06
doc: Fix documentation warnings and add some links ( #29887 )
2023-01-25 23:15:58 -08:00
867d2581a6
Moves impl of bank_hash_info_at() into hash_internal_state() ( #29913 )
2023-01-25 22:24:51 -05:00
dde4b5d1ef
Adds AccountsDb::get_bank_hash_info() ( #29908 )
2023-01-25 17:01:01 -05:00
d76c598691
Snapshot serde test uses calculate_accounts_delta_hash() ( #29905 )
2023-01-25 16:26:15 -05:00
4ebcacb4a3
Revert "Add run parent directory for accounts files ( #29794 )" ( #29899 )
...
This PR is causing OOM on master. Reverting it for now.
This reverts commit 74f89d1494
2023-01-25 10:03:01 -08:00
9193b4221d
Revert "chore: workspace inheritance ( #29509 )" ( #29892 )
...
This reverts commit a67d239dde
2023-01-25 15:50:41 +08:00
a67d239dde
chore: workspace inheritance ( #29509 )
...
* introduce workspace.package
* introduce workspace.dependencies
* read version from root cargo.toml
* pass check when version = { workspace = true }
* don't bump version when version = { workspace = true }
* including workspace Cargo.toml when bump version
* programs/sbf use workspace inheritance
* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-01-25 13:59:59 +08:00
74f89d1494
Add run parent directory for accounts files ( #29794 )
...
* Add run parent directory for accounts files
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* remove explict type def for map result
* handle create_accounts_run_and_snapshot_dirs error with expect
* update with more review comments
* minor fixes from review comments
* simplify account_filename option assignment
* handle error from create_accounts_run_and_snapshot_dirs
* use then instead of then_some for lazy evaluation
2023-01-24 16:44:35 -08:00
bda0c606a1
Renames fn to calculate_accounts_delta_hash() ( #29876 )
2023-01-24 18:55:56 -05:00
655345728b
Use saturating_add_signed() ( #29867 )
2023-01-24 17:07:50 -05:00
d67baefd70
Use Option::unzip() ( #29866 )
2023-01-24 17:07:29 -05:00
8a146361d1
includes rent_epoch in vote-accounts sanity checks ( #29861 )
...
https://github.com/solana-labs/solana/pull/26479
preserves rent epoch for rent-exempt accounts. Since the feature got
activated, vote accounts in stakes-cache have identical rent_epoch field
as the accounts in accounts-db. The commit verifies this in stakes-cache
sanity checks.
2023-01-24 19:47:38 +00:00
a06f8e68a5
Tests wait for initial accounts hash verification to complete ( #29840 )
2023-01-24 14:44:39 -05:00
07afa22671
test fn append_single_account_with_default_hash uses AccountStorageEntry ( #29839 )
2023-01-24 11:43:03 -08:00
cac52b0819
trait ShrinkCollectRefs ( #29830 )
2023-01-24 08:43:50 -08:00
8d4603edfb
update tests storages -> storage ( #29837 )
2023-01-24 07:54:28 -08:00
79843fcbe7
update tests for 1 append vec per slot ( #29836 )
2023-01-24 07:53:54 -08:00
5d24745464
if 0 alive accounts, don't create ancient append vec ( #29842 )
2023-01-24 07:52:31 -08:00
b35d173c04
Remove bank_transaction_count_fix logic ( #29695 )
...
* Remove bank_transaction_count_fix logic
* Fix test and add comment
2023-01-22 15:59:35 +08:00
272e667cb2
deprecates Pubkey::new in favor of Pubkey::{,try_}from ( #29805 )
...
The commit deprecates Pubkey::new which lacks type-safety and instead
implements TryFrom<&[u8]> and TryFrom<Vec<u8>> for Pubkey.
2023-01-21 18:06:27 +00:00
db10073c36
Promotes accounts delta hash to its own type ( #29768 )
2023-01-20 14:23:56 -05:00
6baab92ab5
Refactors calculate_capitalization() ( #29790 )
2023-01-20 13:38:41 -05:00
0a0c532c86
Refactors verify_bank_hash_and_lamports() ( #29783 )
2023-01-19 16:14:39 -05:00
d463bcc5f8
removes manual implementations of Default for enums ( #29779 )
2023-01-19 19:25:30 +00:00
c4e43f1de4
vote: encapsulate `Lockout` ( #29753 )
2023-01-18 19:28:28 -07:00
1753a0e3af
Revert "Do not exclude failed simple vote transactions from consensus" ( #29745 )
...
This reverts commit b4237f3f2c
2023-01-18 15:13:13 -05:00
9d2c71b9a3
remove type SnapshotStoragesOne ( #29750 )
2023-01-18 11:51:08 -08:00
aef8692c8f
remove SnapshotStorageOne ( #29747 )
2023-01-18 07:10:56 -08:00
f24e30e4fe
remove test method get_storages_for_slot ( #29739 )
2023-01-17 12:28:53 -08:00
8c62927d59
Adds calculate_incremental_accounts_hash() ( #29734 )
...
**Node operators,** please delete your `<ledger>/calculate_accounts_hash_cache` directory (if you have one). It has been moved/renamed to `<ledger>/accounts_hash_cache/full`.
2023-01-17 15:04:29 -05:00
e1d38c8315
remove test method slot_stores ( #29736 )
2023-01-17 10:15:25 -08:00
71a746f10e
remove some write_version checking ( #29733 )
2023-01-17 10:14:31 -08:00
4bfe4091b4
make storages not plural ( #29732 )
2023-01-17 09:50:35 -08:00
5188da0492
remove get_slot_stores from AccountStorage ( #29726 )
2023-01-17 09:26:46 -08:00
1d88654116
Revert "Cap accounts data a transaction can load by its requested limit" ( #29373 )
...
Revert "Cap accounts data a transaction can load by its requested limit (#27840 )"
This reverts commit 81dc2e56ac
2023-01-17 11:34:18 +01:00
c5380d4699
Adds symmetric tests for all cases of un-/re-/deployment inside the same transaction ( #29725 )
...
Unifies test_program_sbf_upgrade_and_invoke_in_same_tx() and test_program_sbf_upgrade_self_via_cpi() into test_program_sbf_invoke_in_same_tx_as_redeployment().
Adds test_program_sbf_invoke_in_same_tx_as_deployment() and test_program_sbf_invoke_in_same_tx_as_undeployment().
2023-01-17 09:44:27 +01:00
461dafb887
AccountStorage only holds a single append vec ( #29723 )
...
* AccountStorageReference
* tests build
* pr feedback
2023-01-16 16:22:30 -08:00
0db14ad39c
Removes full_snapshot from CalcAccountsHashConfig ( #29722 )
2023-01-16 16:22:46 -05:00
ef30083319
iterate contents of AccountStorage without exposing internals ( #29719 )
...
* iterate contents of AccountStorage without exposing internals
* Update runtime/src/accounts_db.rs
Co-authored-by: Brooks <brooks@prumo.org>
* Update runtime/src/accounts_db.rs
Co-authored-by: Brooks <brooks@prumo.org>
* Update runtime/src/accounts_db.rs
Co-authored-by: Brooks <brooks@prumo.org>
* compile errors
Co-authored-by: Brooks <brooks@prumo.org>
2023-01-16 12:01:25 -08:00
8a28877910
Use Hash::new_from_array() instead of TryFrom ( #29720 )
2023-01-16 10:53:38 -05:00
977d3453a3
storage.remove does not leak SlotStores ( #29664 )
2023-01-15 11:06:24 -08:00
9d1a0c52f8
Cleanup - `disable_deprecated_loader` ( #29653 )
...
* Removes the gating of disable_deprecated_loader.
* Adds create_program().
Renames create_deprecated_program() -> load_program().
Renames load_and_finalize_deprecated_program() -> load_and_finalize_program().
* Fixes test_sol_alloc_free_no_longer_deployable().
2023-01-14 13:02:10 +01:00
064f163b70
get_account_storage_entry handles missing shrink in progress entry correctly ( #29682 )
2023-01-13 13:11:25 -08:00
a2933f7fa6
remove some uses of write_version ( #29680 )
2023-01-13 12:05:15 -08:00
ab97d37e3d
Refactor - Simplify loader utils ( #29699 )
...
* Moves all the common code of tests and benches into loader utils.
* Removes write_sbf_program().
* Removes ignored test: test_program_sbf_test_use_latest_executor2().
2023-01-13 20:38:36 +01:00
10f8b5faa3
Bump memmap2 from 0.5.3 to 0.5.8 ( #29674 )
...
* Bump memmap2 from 0.5.3 to 0.5.8
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.5.3 to 0.5.8.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.3...v0.5.8 )
---
updated-dependencies:
- dependency-name: memmap2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-12 19:38:38 -07:00
6359d68f9f
get_slot_stores -> get_slot_storage_entry ( #29677 )
2023-01-12 09:44:53 -08:00
d0aa93de21
Refactors accounts hash cache ( #29625 )
2023-01-12 10:43:50 -05:00
3412928cad
get_slot_stores -> get_slot_storage_entry ( #29678 )
2023-01-12 07:17:18 -08:00
b64f58614d
get_slot_stores -> get_slot_storage_entry ( #29665 )
2023-01-12 07:09:56 -08:00
9d1c0c5a3c
get_slot_stores -> get_slot_storage_entry ( #29663 )
2023-01-12 04:08:08 -08:00
56f8c16497
remove remaining uses of SnapshotStorages ( #29660 )
2023-01-11 17:58:14 -08:00
14055fc3e2
remove type SnapshotStorage and uses ( #29661 )
2023-01-11 17:57:44 -08:00
69c4db27c5
Configurable hashes per tick ( #29659 )
2023-01-11 16:56:11 -08:00
3e6ba72da1
remove scan_multiple_account_storages_one_slot ( #29658 )
2023-01-11 16:26:49 -08:00
0e19b9c849
SortedStorages holds HashMap instead of Vec ( #29632 )
...
* SortedStorages holds HashMap instead of Vec
* add comment
2023-01-11 14:20:43 -08:00
4b93fa85c2
renaming/refactoring for append vecs ( #29655 )
2023-01-11 14:20:14 -08:00
544b9745c2
snapshot storage path uses 1 append vec per slot ( #29627 )
2023-01-11 12:05:15 -08:00
28071d5bf4
chore: Move new bank metric into metric submodule ( #29628 )
...
More cleanup in moving non-essential items out of bank.rs
2023-01-11 13:14:46 -06:00
0ee9993632
decouple geyser's write_version from append vec on snapshot load ( #29623 )
2023-01-11 10:59:57 -08:00
1d6999914a
factor out scan_single_account_storage ( #29633 )
2023-01-11 10:59:46 -08:00
d693167dfb
Feature: filter stake by min delegation for rewards ( #29618 )
...
filter stake by min delegation for rewards
2023-01-11 08:55:18 -06:00
864a7d8d6f
Bump bzip2 from 0.4.3 to 0.4.4 ( #29630 )
...
* Bump bzip2 from 0.4.3 to 0.4.4
Bumps [bzip2](https://github.com/alexcrichton/bzip2-rs ) from 0.4.3 to 0.4.4.
- [Release notes](https://github.com/alexcrichton/bzip2-rs/releases )
- [Commits](https://github.com/alexcrichton/bzip2-rs/commits/0.4.4 )
---
updated-dependencies:
- dependency-name: bzip2
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-10 23:08:22 -07:00
0f16120b03
remove FoundStoredAccount ( #29591 )
2023-01-10 13:57:34 -08:00
ad400419b5
assert write version is increasing per slot as we read from a slot ( #29624 )
2023-01-10 13:34:30 -08:00
c3353c27c2
verify acct size in index can easily be recalculated ( #29601 )
2023-01-10 13:33:36 -08:00
9cd3bb0c4d
never try to shrink a slot that is in the write cache ( #29615 )
2023-01-10 12:01:37 -08:00
1faf7d516b
while shrinking, old append vec is held outside of storage.map ( #29323 )
...
* while shrinking, old append vec is held outside of storage.map
* update comment
* rework ShrinkInProgress for clarity
* add tests and comment
2023-01-10 11:52:01 -08:00
1c1c8fb008
chore: Move bank new epoch metrics into separate file ( #29599 )
...
chore: Move bank new epoch metric into separate file
bank.rs is very large, so move the metrics reporting code into a
separate file. This reduces line count and also makes the actual logic
in bank.rs more readable by removing a big block of boilerplate code.
There are more metrics that we can do this for, but doing them one at a
time to avoid large diffs.
2023-01-10 13:15:50 -06:00
a5106e110a
just use slot to determine alive while shrinking ( #29613 )
2023-01-10 09:00:18 -08:00
1c862f0b66
remove unused ret val from do_shrink_slot_store ( #29616 )
2023-01-10 08:12:24 -08:00
6441ed121e
get_unique_accounts_from_storages drops plural ( #29600 )
2023-01-10 07:49:38 -08:00
d3daea65b9
use atomicu64 for bucket maxsize instead of lock ( #29569 )
2023-01-10 09:15:27 -06:00
71467808d2
convert some get_slot_stores -> get_slot_storage_entry ( #29549 )
2023-01-10 05:34:08 -08:00
c9fe21e3c3
ancient append vecs enabled at -10k by default ( #29565 )
2023-01-10 05:33:43 -08:00
1d675c6205
FoundStoredAccount removes store_id ( #29590 )
2023-01-10 05:28:41 -08:00
db277d320b
Adds TX signature to `TransactionContext` in debug mode ( #29597 )
...
Adds TX signature to TransactionContext in debug mode.
2023-01-10 00:54:26 +01:00
dda34b208c
get_unique_accounts_from_storages takes 1 append vec ( #29588 )
2023-01-09 14:53:50 -08:00
3234af41a7
shrink_collect takes a single append vec ( #29523 )
2023-01-09 13:33:21 -08:00
a703019caa
feature: don't do rewrites in rent collection ( #26491 )
...
* feature: don't do rewrites in rent collection
* modify test to specifically test this feature
2023-01-09 08:14:17 -08:00
d41da2178c
test_bad_bank_hash uses write cache ( #29559 )
2023-01-09 08:10:28 -08:00
a171402e5c
combine_one_store_into_ancient takes 1 append vec ( #29560 )
2023-01-09 08:09:38 -08:00
9692cfef98
accounts tests use store_for_tests ( #29555 )
2023-01-06 15:30:42 -08:00
2f0d849c84
set rent epoch to max on new account creation ( #29528 )
...
* set rent epoch to max on new account creation
* add comment
2023-01-06 14:21:10 -08:00
2a0a185846
accounts tests use store_for_tests ( #29554 )
2023-01-06 12:07:36 -08:00
33119f8a87
convert some get_slot_stores -> get_slot_storage_entry ( #29557 )
2023-01-06 12:06:59 -08:00
453b61676e
filler accounts only work gradually ( #29553 )
2023-01-06 12:05:32 -08:00
3fc4015a4d
do_shrink_slot_stores -> do_shrink_slot_store ( #29552 )
2023-01-06 09:23:32 -08:00
39dc034c05
ShrinkCandidates only hold 1 append vec per slot ( #29525 )
...
* ShrinkCandidates only hold 1 append vec per slot
* Fixed select_candidates_by_total_usage unit tests, use distinct slots for stores
as each slot has one store now.
* rename
Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
2023-01-06 06:54:34 -08:00
272e5a05f2
remove AccountStorage::is_empty() ( #29548 )
2023-01-06 06:51:25 -08:00
8e19909c7d
stop returning remaining # stores from mark_dirty_dead_stores ( #29545 )
2023-01-05 19:21:01 -08:00
a0f045d2c3
storage rebuilder regex cleanup ( #29408 )
...
* storage rebuilder regex cleanup
* Update runtime/src/snapshot_utils/snapshot_storage_rebuilder.rs
Co-authored-by: apfitzge <apfitzge@gmail.com>
Co-authored-by: apfitzge <apfitzge@gmail.com>
2023-01-05 16:16:24 -08:00
a6b492b2c1
Removed assert on write_version ordering ( #29530 )
...
Removed assert on write_version ordering as snapshot created by earlier version
is not honoring that.
2023-01-05 15:14:55 -08:00
a6bf68cec8
Refine appendvec sanitize error message to include path ( #29541 )
2023-01-05 14:43:10 -08:00
72a7cbd6a2
shrink fns take a single append vec ( #29522 )
2023-01-05 13:17:04 -08:00
12da2da389
fixes errors from clippy::redundant_clone ( #29536 )
...
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
2023-01-05 18:42:19 +00:00
3b7ebfe50e
migrate tests to use get_storage_for_slot ( #29518 )
2023-01-05 10:38:58 -08:00
d3f45c6901
shrink_slot_forced uses a single append vec ( #29521 )
2023-01-05 10:38:05 -08:00
d87128e02c
fixes errors from clippy::needless_borrow ( #29535 )
...
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
2023-01-05 18:21:56 +00:00
d611cc2e75
Fixes format string ( #29533 )
2023-01-05 13:06:00 -05:00
5c9beef498
fixes errors from clippy::useless_conversion ( #29534 )
...
https://rust-lang.github.io/rust-clippy/master/index.html#useless_conversion
2023-01-05 18:05:32 +00:00
1e8a8e07b6
Stream the executed transaction count in the block notification ( #29272 )
...
Problem
The plugins need to know when all transactions for a block have been all notified to serve getBlock request correctly. As block and transaction notifications are sent asynchronously to each other it will be difficult.
Summary of Changes
Include the executed transaction count in block notification which can be used to check if all transactions have been notified.
2023-01-05 09:36:19 -08:00
dbac798ef6
process_storage_slot takes a single append vec ( #29519 )
2023-01-05 07:33:47 -08:00
a140f06a67
use get_storage_for_slot() in tests ( #29517 )
2023-01-05 07:32:00 -08:00
8cdf5cdb50
get_storage_to_move_to_ancient_append_vec returns a single append vec ( #29482 )
...
get_storages_to_move_to_ancient_append_vec returns a single append vec
2023-01-05 07:31:26 -08:00
221f547ba9
remove store_ids from a few shrink data structures ( #29360 )
2023-01-05 07:30:43 -08:00
05d5aed4d8
Adds SnapshotError::IoWithSourceAndFile ( #29527 )
2023-01-05 09:08:15 -05:00
fc8cd82309
migrate tests to get_slot_storage_entry ( #29515 )
2023-01-04 13:12:48 -08:00
4e636ab153
cleanup ancient append vec tests ( #29514 )
2023-01-04 12:39:42 -08:00
150ac80cfa
filter_storages ignores reclaims ( #29512 )
2023-01-04 12:38:30 -08:00
ef72140173
remove unnecessary to_vec() ( #29516 )
2023-01-04 12:38:16 -08:00
093b1a5e10
get_storages_for_slot uses get_slot_storage_entry ( #29498 )
2023-01-04 10:15:56 -08:00
10135dbd10
should_move_to_ancient_append_vec works with a single storage ( #29484 )
2023-01-04 07:03:46 -08:00
d7bd1bf970
bank: Record non-vote transaction count ( #29383 )
...
A subsequent change to `SamplePerformanceService` introduces non-vote transaction counts, which `bank`s need to store.
Part of work on https://github.com/solana-labs/solana/issues/29159
2023-01-03 14:46:20 -08:00
89deecb979
add AccountStorage.is_empty_entry for tests ( #29489 )
2023-01-03 12:36:31 -08:00
0f2bfa2b02
cleanup get_snapshot_storages ( #29488 )
...
* cleanup get_snapshot_storages
* pr feedback
2023-01-03 12:22:28 -08:00
95d2f002bd
add AccountStorage.get_slot_storage_entry ( #29480 )
2023-01-03 12:10:34 -08:00
9ee4555f53
remove should_retain from mark_dirty_dead_stores ( #29358 )
2023-01-03 09:53:43 -08:00
102dc08378
get_snapshot_storages removes call to AccountStorage.get ( #29466 )
2023-01-03 09:15:07 -08:00
9d15ead873
remove single use AccountStorage.slot_store_count ( #29479 )
2023-01-03 09:12:27 -08:00
10eadb6dd9
add test method assert_no_storages_at_slot() ( #29483 )
2023-01-03 08:59:21 -08:00
ca9166201c
add test method get_and_assert_single_storage ( #29481 )
2023-01-03 08:38:15 -08:00
04508ccfa7
add AccountStorage.is_empty ( #29478 )
2023-01-03 08:35:10 -08:00
8c878daada
refactor RecycleStores::add_entries ( #29475 )
...
* refactor RecycleStores::add_entries
* pr feedback
2023-01-03 07:56:30 -08:00
89e733c2b3
remove unnecessary type ( #29473 )
2023-01-03 06:34:05 -08:00
3363c08ac0
Move async remove to snapshot_utils.rs ( #29406 )
2023-01-03 06:15:32 -08:00
1fd1e57f50
add test ( #29471 )
2023-01-02 20:53:43 -08:00
e86628a823
filter get_snapshot_storages for requested_slots earlier ( #29465 )
...
* filter get_snapshot_storages for requested_slots earlier
* Update runtime/src/accounts_db.rs
Co-authored-by: apfitzge <apfitzge@gmail.com>
Co-authored-by: apfitzge <apfitzge@gmail.com>
2023-01-02 20:53:30 -08:00
4cc1890f00
skip_rewrites will only be feature driven ( #29468 )
2023-01-02 20:15:42 -08:00
64466725d4
remove skip rewrite code from collect_rent_from_accounts ( #29472 )
2023-01-02 20:15:05 -08:00
6294d1eff1
cleanup in account_storage.rs ( #29467 )
2023-01-02 14:55:02 -08:00
Brooks
Pankaj Garg
Xiang Zhu
Yueh-Hsuan Chiang
HaoranYi
Andrew Fitzgerald
Jeff Washington (jwash)
Tao Zhu
Alexander Meißner
Ryo Onodera
ivandzen
Tyera
Brennan
dependabot[bot]
Richard Patel
Will Hickey
K-anon
Richard Patel
joeaba
Illia Bobyr
Yihau Chen
behzad nouri
Trent Nelson
carllin
steviez
Lijun Wang