aef8692c8f
remove SnapshotStorageOne ( #29747 )
2023-01-18 07:10:56 -08:00
f24e30e4fe
remove test method get_storages_for_slot ( #29739 )
2023-01-17 12:28:53 -08:00
8c62927d59
Adds calculate_incremental_accounts_hash() ( #29734 )
...
**Node operators,** please delete your `<ledger>/calculate_accounts_hash_cache` directory (if you have one). It has been moved/renamed to `<ledger>/accounts_hash_cache/full`.
2023-01-17 15:04:29 -05:00
e1d38c8315
remove test method slot_stores ( #29736 )
2023-01-17 10:15:25 -08:00
71a746f10e
remove some write_version checking ( #29733 )
2023-01-17 10:14:31 -08:00
4bfe4091b4
make storages not plural ( #29732 )
2023-01-17 09:50:35 -08:00
461dafb887
AccountStorage only holds a single append vec ( #29723 )
...
* AccountStorageReference
* tests build
* pr feedback
2023-01-16 16:22:30 -08:00
0db14ad39c
Removes full_snapshot from CalcAccountsHashConfig ( #29722 )
2023-01-16 16:22:46 -05:00
ef30083319
iterate contents of AccountStorage without exposing internals ( #29719 )
...
* iterate contents of AccountStorage without exposing internals
* Update runtime/src/accounts_db.rs
Co-authored-by: Brooks <brooks@prumo.org>
* Update runtime/src/accounts_db.rs
Co-authored-by: Brooks <brooks@prumo.org>
* Update runtime/src/accounts_db.rs
Co-authored-by: Brooks <brooks@prumo.org>
* compile errors
Co-authored-by: Brooks <brooks@prumo.org>
2023-01-16 12:01:25 -08:00
8a28877910
Use Hash::new_from_array() instead of TryFrom ( #29720 )
2023-01-16 10:53:38 -05:00
977d3453a3
storage.remove does not leak SlotStores ( #29664 )
2023-01-15 11:06:24 -08:00
a2933f7fa6
remove some uses of write_version ( #29680 )
2023-01-13 12:05:15 -08:00
6359d68f9f
get_slot_stores -> get_slot_storage_entry ( #29677 )
2023-01-12 09:44:53 -08:00
d0aa93de21
Refactors accounts hash cache ( #29625 )
2023-01-12 10:43:50 -05:00
3412928cad
get_slot_stores -> get_slot_storage_entry ( #29678 )
2023-01-12 07:17:18 -08:00
b64f58614d
get_slot_stores -> get_slot_storage_entry ( #29665 )
2023-01-12 07:09:56 -08:00
9d1c0c5a3c
get_slot_stores -> get_slot_storage_entry ( #29663 )
2023-01-12 04:08:08 -08:00
14055fc3e2
remove type SnapshotStorage and uses ( #29661 )
2023-01-11 17:57:44 -08:00
3e6ba72da1
remove scan_multiple_account_storages_one_slot ( #29658 )
2023-01-11 16:26:49 -08:00
4b93fa85c2
renaming/refactoring for append vecs ( #29655 )
2023-01-11 14:20:14 -08:00
544b9745c2
snapshot storage path uses 1 append vec per slot ( #29627 )
2023-01-11 12:05:15 -08:00
1d6999914a
factor out scan_single_account_storage ( #29633 )
2023-01-11 10:59:46 -08:00
0f16120b03
remove FoundStoredAccount ( #29591 )
2023-01-10 13:57:34 -08:00
ad400419b5
assert write version is increasing per slot as we read from a slot ( #29624 )
2023-01-10 13:34:30 -08:00
c3353c27c2
verify acct size in index can easily be recalculated ( #29601 )
2023-01-10 13:33:36 -08:00
9cd3bb0c4d
never try to shrink a slot that is in the write cache ( #29615 )
2023-01-10 12:01:37 -08:00
1faf7d516b
while shrinking, old append vec is held outside of storage.map ( #29323 )
...
* while shrinking, old append vec is held outside of storage.map
* update comment
* rework ShrinkInProgress for clarity
* add tests and comment
2023-01-10 11:52:01 -08:00
a5106e110a
just use slot to determine alive while shrinking ( #29613 )
2023-01-10 09:00:18 -08:00
1c862f0b66
remove unused ret val from do_shrink_slot_store ( #29616 )
2023-01-10 08:12:24 -08:00
6441ed121e
get_unique_accounts_from_storages drops plural ( #29600 )
2023-01-10 07:49:38 -08:00
71467808d2
convert some get_slot_stores -> get_slot_storage_entry ( #29549 )
2023-01-10 05:34:08 -08:00
c9fe21e3c3
ancient append vecs enabled at -10k by default ( #29565 )
2023-01-10 05:33:43 -08:00
1d675c6205
FoundStoredAccount removes store_id ( #29590 )
2023-01-10 05:28:41 -08:00
dda34b208c
get_unique_accounts_from_storages takes 1 append vec ( #29588 )
2023-01-09 14:53:50 -08:00
3234af41a7
shrink_collect takes a single append vec ( #29523 )
2023-01-09 13:33:21 -08:00
a703019caa
feature: don't do rewrites in rent collection ( #26491 )
...
* feature: don't do rewrites in rent collection
* modify test to specifically test this feature
2023-01-09 08:14:17 -08:00
a171402e5c
combine_one_store_into_ancient takes 1 append vec ( #29560 )
2023-01-09 08:09:38 -08:00
9692cfef98
accounts tests use store_for_tests ( #29555 )
2023-01-06 15:30:42 -08:00
33119f8a87
convert some get_slot_stores -> get_slot_storage_entry ( #29557 )
2023-01-06 12:06:59 -08:00
453b61676e
filler accounts only work gradually ( #29553 )
2023-01-06 12:05:32 -08:00
3fc4015a4d
do_shrink_slot_stores -> do_shrink_slot_store ( #29552 )
2023-01-06 09:23:32 -08:00
39dc034c05
ShrinkCandidates only hold 1 append vec per slot ( #29525 )
...
* ShrinkCandidates only hold 1 append vec per slot
* Fixed select_candidates_by_total_usage unit tests, use distinct slots for stores
as each slot has one store now.
* rename
Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
2023-01-06 06:54:34 -08:00
272e5a05f2
remove AccountStorage::is_empty() ( #29548 )
2023-01-06 06:51:25 -08:00
8e19909c7d
stop returning remaining # stores from mark_dirty_dead_stores ( #29545 )
2023-01-05 19:21:01 -08:00
72a7cbd6a2
shrink fns take a single append vec ( #29522 )
2023-01-05 13:17:04 -08:00
3b7ebfe50e
migrate tests to use get_storage_for_slot ( #29518 )
2023-01-05 10:38:58 -08:00
d3f45c6901
shrink_slot_forced uses a single append vec ( #29521 )
2023-01-05 10:38:05 -08:00
5c9beef498
fixes errors from clippy::useless_conversion ( #29534 )
...
https://rust-lang.github.io/rust-clippy/master/index.html#useless_conversion
2023-01-05 18:05:32 +00:00
dbac798ef6
process_storage_slot takes a single append vec ( #29519 )
2023-01-05 07:33:47 -08:00
a140f06a67
use get_storage_for_slot() in tests ( #29517 )
2023-01-05 07:32:00 -08:00
8cdf5cdb50
get_storage_to_move_to_ancient_append_vec returns a single append vec ( #29482 )
...
get_storages_to_move_to_ancient_append_vec returns a single append vec
2023-01-05 07:31:26 -08:00
221f547ba9
remove store_ids from a few shrink data structures ( #29360 )
2023-01-05 07:30:43 -08:00
fc8cd82309
migrate tests to get_slot_storage_entry ( #29515 )
2023-01-04 13:12:48 -08:00
4e636ab153
cleanup ancient append vec tests ( #29514 )
2023-01-04 12:39:42 -08:00
ef72140173
remove unnecessary to_vec() ( #29516 )
2023-01-04 12:38:16 -08:00
093b1a5e10
get_storages_for_slot uses get_slot_storage_entry ( #29498 )
2023-01-04 10:15:56 -08:00
10135dbd10
should_move_to_ancient_append_vec works with a single storage ( #29484 )
2023-01-04 07:03:46 -08:00
89deecb979
add AccountStorage.is_empty_entry for tests ( #29489 )
2023-01-03 12:36:31 -08:00
0f2bfa2b02
cleanup get_snapshot_storages ( #29488 )
...
* cleanup get_snapshot_storages
* pr feedback
2023-01-03 12:22:28 -08:00
95d2f002bd
add AccountStorage.get_slot_storage_entry ( #29480 )
2023-01-03 12:10:34 -08:00
9ee4555f53
remove should_retain from mark_dirty_dead_stores ( #29358 )
2023-01-03 09:53:43 -08:00
102dc08378
get_snapshot_storages removes call to AccountStorage.get ( #29466 )
2023-01-03 09:15:07 -08:00
9d15ead873
remove single use AccountStorage.slot_store_count ( #29479 )
2023-01-03 09:12:27 -08:00
10eadb6dd9
add test method assert_no_storages_at_slot() ( #29483 )
2023-01-03 08:59:21 -08:00
ca9166201c
add test method get_and_assert_single_storage ( #29481 )
2023-01-03 08:38:15 -08:00
04508ccfa7
add AccountStorage.is_empty ( #29478 )
2023-01-03 08:35:10 -08:00
8c878daada
refactor RecycleStores::add_entries ( #29475 )
...
* refactor RecycleStores::add_entries
* pr feedback
2023-01-03 07:56:30 -08:00
89e733c2b3
remove unnecessary type ( #29473 )
2023-01-03 06:34:05 -08:00
1fd1e57f50
add test ( #29471 )
2023-01-02 20:53:43 -08:00
e86628a823
filter get_snapshot_storages for requested_slots earlier ( #29465 )
...
* filter get_snapshot_storages for requested_slots earlier
* Update runtime/src/accounts_db.rs
Co-authored-by: apfitzge <apfitzge@gmail.com>
Co-authored-by: apfitzge <apfitzge@gmail.com>
2023-01-02 20:53:30 -08:00
4cc1890f00
skip_rewrites will only be feature driven ( #29468 )
2023-01-02 20:15:42 -08:00
9615965219
decrease frequency of random shrink of ancient append vec ( #29462 )
2023-01-02 13:29:04 -08:00
bf93a1e13f
migrate tests to not use AccountStorage::get ( #29464 )
2023-01-02 13:02:09 -08:00
e99afb4e85
handle ancient overflow case correctly ( #29458 )
2023-01-02 12:39:59 -08:00
8397f295c0
simplify get_storages_for_slot ( #29463 )
2023-01-02 12:38:48 -08:00
38d771a823
add test and comments ( #29459 )
2023-01-02 12:38:21 -08:00
6edbb61fa0
add test_shrink_ancient_overflow ( #29363 )
2023-01-02 07:09:53 -08:00
0f7ef085a0
introduce ShrinkInProgress ( #29329 )
...
* introduce ShrinkInProgress
* remove redundant check
* add comments
2022-12-21 09:03:29 -08:00
4a64f6d421
test_utils::create_test_accounts pre-allocates an append vec first ( #29336 )
...
* test_utils::create_test_accounts pre-allocates an append vec first
* remove comment
2022-12-21 09:02:42 -08:00
c3ca5b78b2
remove acctdb.min_num_stores ( #29335 )
2022-12-20 10:39:20 -08:00
098b86b883
test_minimize_accounts_db uses write cache correctly ( #29333 )
2022-12-20 09:50:56 -08:00
053775ad77
Elides unnecessary lifetimes ( #29299 )
2022-12-20 12:44:17 -05:00
c816157979
shrink avoids reshaping data by using StorableAccounts trait ( #29307 )
2022-12-19 12:29:21 -08:00
19f13a0bea
refactor get_store_for_shrink ( #29284 )
2022-12-15 22:10:39 -08:00
1e0a0e0ced
Cleans up CacheHashData ( #29267 )
2022-12-15 14:16:27 -05:00
5a687fa818
make AccountStorage::map private ( #29268 )
2022-12-14 20:03:25 -08:00
3a7b67ea55
refactor unwrap_or_default for type that will soon not have a default ( #29257 )
2022-12-14 15:11:36 -08:00
49a136a374
move AccountStorage out of accounts_db.rs ( #29261 )
2022-12-14 15:08:39 -08:00
f9db75768c
Vec<Arc<AccountStorageEntry>> becomes SnapshotStorage ( #29253 )
2022-12-14 12:18:43 -08:00
fab0ce21a2
remove shrink_candidate_slots_v1 ( #29252 )
2022-12-14 08:57:16 -08:00
b77bef4ee3
write_version -> write_version_obsolete ( #29176 )
2022-12-14 07:43:40 -08:00
62be54a75e
remove process_stale_slot_v1 and other fns ( #29244 )
2022-12-13 16:32:24 -08:00
4ae12bf5fa
fix tests broken by add_root change ( #29248 )
2022-12-13 14:43:52 -08:00
3e649d2aa0
remove add_root(caching_enabled) ( #29245 )
2022-12-13 11:26:28 -08:00
1a0a465416
handle unclean roots explicitly during index generation ( #29230 )
2022-12-13 07:51:33 -08:00
bcbf3374fd
Avoids copying hashes when computing merkle root ( #29225 )
2022-12-13 10:20:14 -05:00
a57247a78e
handle write_version correctly in geyser api ( #29224 )
2022-12-13 06:42:28 -08:00
c3001abef7
don't rely on write_version increasing - just assert that it is for now ( #29180 )
2022-12-12 19:24:45 -08:00
5e799ad563
remove AccountsDb::new_with_config.caching_enabled ( #29167 )
2022-12-12 08:50:00 -08:00
db2cc53967
remove skip rewrites from bank ( #29182 )
2022-12-10 09:27:15 -08:00
f9c4170d34
remove caching_enabled ( #29117 )
...
* warp_from_parents works with write_cache enabled
* remove caching_enabled
2022-12-09 14:31:54 -08:00
7f1b71a2b0
remove AccountsDb::new_with_config_for_tests.caching_enabled ( #29181 )
2022-12-09 13:54:37 -08:00
06a806bb9d
introduce aligned_stored_size to flush write cache ( #29147 )
2022-12-08 19:44:23 -08:00
072070ba80
test_shrink_candidate_slots uses write cache ( #29145 )
2022-12-08 15:13:05 -08:00
2d4f17e1c9
Removes AccountsDb::hash_account_with_rent_epoch() ( #29138 )
2022-12-08 13:08:36 -05:00
9eedc189fd
Uses `storages` for instance of SortedStorages ( #29133 )
2022-12-07 19:27:46 -05:00
1d95720e5e
migrate some tests to work with write cache ( #29124 )
...
* make specific tests use write cache
* fix a few more tests
2022-12-07 13:46:06 -08:00
284c41a6db
test_get_snapshot_storages_only_non_empty uses write cache ( #29111 )
2022-12-07 09:19:17 -08:00
59359c3ab4
test_shrink_collect_simple uses write cache ( #29115 )
2022-12-07 09:18:45 -08:00
00f5a176a0
test_clean_zero_lamport_and_old_roots uses write cache ( #29113 )
2022-12-07 07:30:58 -08:00
18a85edf8c
migrate tests to use write cache ( #29108 )
...
* test_store_overhead works with write cache
* test_storage_remove_account_double_remove works with write cache
* test_set_storage_count_and_alive_bytes works with write cache
* test_reuse_append_vec_id works with write cache
2022-12-07 07:03:57 -08:00
8ce6081744
test_accounts_db_purge1 uses write cache ( #29100 )
2022-12-06 14:34:40 -08:00
e5e64c8d6e
test_remove_unrooted_slot_snapshot uses write cache ( #29110 )
...
* test_remove_unrooted_slot_snapshot uses write cache
* test_purge_alive_unrooted_slots_after_clean uses with write cache
* test_get_snapshot_storages_with_base_slot uses write cache
* test_get_snapshot_storages_only_roots uses write cache
* test_get_snapshot_storages_only_older_than_or_equal_to_snapshot_slot uses write cache
* test_get_snapshot_storages_exclude_empty uses write cache
* test_clean_old_with_zero_lamport_account uses write cache
* test_clean_old_with_normal_account uses write cache
* test_clean_old_with_both_normal_and_zero_lamport_accounts uses write cache
* test_clean_max_slot_zero_lamport_account uses write cache
* test_calculate_storage_count_and_alive_bytes_2_accounts uses write cache
* test_calculate_storage_count_and_alive_bytes uses write cache
2022-12-06 14:34:18 -08:00
46e13423fd
migrate tests to work with write cache ( #29106 )
...
* test_accounts_purge_long_chained_after_snapshot_restore
* test_accountsdb_calculate_accounts_hash_from_storages works with write cache
* test_zero_lamport_new_root_not_cleaned works with write cache
* test_wrapping_append_vec_id works with write cache
* test_verify_bank_hash works with write cache
* change tests to use write cache
* use shared db to pass to tests
2022-12-06 14:33:52 -08:00
c668a58b8e
do_full_clean_refcount uses write cache ( #29112 )
2022-12-06 14:33:28 -08:00
d3fa93907b
test_clean_zero_lamport_and_dead_slot uses write cache ( #29114 )
2022-12-06 14:33:07 -08:00
9524c9dbff
patches errors from clippy::uninlined_format_args
...
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
9433c06745
patches errors from clippy::unchecked_duration_subtraction
...
https://rust-lang.github.io/rust-clippy/master/index.html#unchecked_duration_subtraction
2022-12-06 19:32:15 +00:00
ca8372efc1
with_chained_zero_lamport_accounts works with write cache ( #29104 )
2022-12-06 10:56:37 -08:00
26472a13ea
migrating tests to use write cache ( #29101 )
...
* test_account_update uses write cache
* test_accounts_db_purge_keep_live works with write cache
* enable caching for test_accounts_db_purge_keep_live
2022-12-06 10:25:37 -08:00
5d0d4a6b95
test_accounts_db_serialize_zero_and_free uses write cache ( #29102 )
2022-12-06 10:25:07 -08:00
7180345ee1
test_verify_bank_capitalization works with write cache ( #29107 )
2022-12-06 10:24:43 -08:00
5833e2f85e
test_account_grow uses write cache ( #29099 )
2022-12-06 09:28:24 -08:00
94a6665995
test_clean_multiple_zero_lamport_decrements_index_ref_count uses write cache ( #29088 )
2022-12-06 08:56:37 -08:00
fe8b15b43a
test_accountsdb_count_stores uses write cache ( #29086 )
2022-12-06 07:40:51 -08:00
93cc3b1d3d
test_shrink_stale_slots_skipped only uses non-write cache ( #29081 )
2022-12-06 07:03:30 -08:00
4de13be926
test_accounts_db_serialize1 uses write cache ( #29085 )
2022-12-06 06:36:28 -08:00
f88398cbcf
test_shrink_stale_slots_processed uses write cache ( #29082 )
2022-12-06 06:35:49 -08:00
ed0c9e285b
test_partial_clean supports write cache ( #29091 )
2022-12-06 06:35:02 -08:00
c106c7e349
test_clean_accounts_with_last_full_snapshot_slot uses write cache ( #29087 )
2022-12-05 20:41:11 -08:00
6935ff74ce
test_uncleaned_roots_with_account uses write cache ( #29084 )
2022-12-05 20:37:42 -08:00
1b49c52d98
store_uncached -> store_for_tests ( #29083 )
2022-12-05 14:42:16 -08:00
9746eb2c8d
test_lazy_gc_slot can use write cache ( #29080 )
2022-12-05 12:48:15 -08:00
8c47e2168e
test_accounts_clean_after_snapshot_restore_then_old_revives can use write cache ( #29050 )
...
* test_accounts_clean_after_snapshot_restore_then_old_revives can use write cache
* combine flush and set root
2022-12-05 12:39:26 -08:00
c65a8ce6c3
AccountsDb::get_snapshot_storages() takes a Range for slots ( #29054 )
2022-12-05 11:21:21 -05:00
e6cc5052c3
pass alive bytes to AccountsToStore::new ( #28994 )
2022-11-30 15:46:52 -08:00
2c912c9049
use files for merkle tree inputs on accounts hash ( #28965 )
...
* use files for merkle tree inputs on accounts hash
* remove multi-pass scan, simplify
* cleanup and comments
* rework writing into the hash file for clarity
* add test, cleanup
* rename
* move count into option
2022-11-30 12:27:27 -08:00
58e12cf963
avoid copies when writing to an ancient append vec ( #28981 )
...
* avoid copies when writing to an ancient append vec
* update comments
2022-11-30 10:09:23 -08:00
2427004dba
add 'bytes' to aligned/alive_total for clarity ( #28982 )
2022-11-30 07:38:57 -08:00
c8cc1270f1
log # append vecs open ( #28958 )
2022-11-29 16:48:02 -08:00
e42649a8ba
remove cli arg to specify # accounts hash scan passes ( #28976 )
2022-11-29 16:47:51 -08:00
9327658007
Promotes accounts hash to a strong type ( #28930 )
2022-11-28 10:09:47 -05:00
9a24743d2a
expect only 1 store per slot ( #28844 )
...
expect only 1 store per slot in shrink
2022-11-23 06:20:36 -08:00
40a7c6d46a
Renames fn to insert_default_bank_hash() ( #28929 )
2022-11-22 19:31:59 -05:00
5d88a9b32b
improve perf of storing accounts for shrink/ancient ( #28880 )
...
* improve perf of storing accounts for shrink/ancient
* rename
* phantom data
* update comment
* make impl Borrow<Hash> consistent
* remove unused static
2022-11-22 15:36:57 -08:00
8addf93de7
test_store_reuse uses write cache ( #28901 )
2022-11-21 11:17:33 -08:00
d698f982ee
Renames to AccountsHasher ( #28883 )
2022-11-18 18:25:44 -05:00
8cbe83c725
cleanup CalcAccountsHashDataSource ( #28882 )
2022-11-18 14:49:22 -08:00
aa1d319d48
add shrink ancient test ( #28858 )
...
* add shrink ancient test
* improvements
2022-11-18 13:11:57 -08:00
7787cb2ab2
Renames BankHashInfo field to accounts_delta_hash ( #28878 )
2022-11-18 15:35:03 -05:00
Jeff Washington (jwash)
Brooks
behzad nouri