05391b5244
packed ancient: report random stat ( #31274 )
2023-04-24 07:40:45 -07:00
8b30aa187b
ancient packing: leave same append vecs alone ( #31295 )
2023-04-24 07:40:05 -07:00
ad30a60059
update stats for packed ancient ( #31255 )
2023-04-18 13:33:02 -07:00
3cf4ffc88d
pack ancient: add test for addref_accounts_failed_to_shrink_ancient ( #31228 )
2023-04-17 20:05:59 -07:00
3d6d34ebd0
packed ancient: add revisit_accounts_with_many_refs ( #31209 )
2023-04-17 13:52:26 -07:00
a8134b8e73
packed ancient: addref when pack operation fails ( #31211 )
2023-04-16 12:24:25 -07:00
09bcf59003
packed ancient: shrink_collect accounts in parallel ( #31212 )
2023-04-16 12:23:11 -07:00
c008a557c5
packed ancient: parallel write to append vecs ( #31144 )
2023-04-14 13:57:46 -07:00
71b6370426
Abstract out StoredAccountMeta as an Enum ( #30478 )
...
#### Problem
The existing StoredAccountMeta requires the file and in-memory layout of
account metadata to match StoredMeta and AccountMeta. This limits the
flexibility to implement different accounts storage file formats.
#### Summary of Changes
This PR abstracts out StoredAccountMeta as an Enum.
2023-03-09 15:52:51 -08:00
33e179905f
getter functions for StoredAccountMeta ( #30447 )
...
This PR makes all the StoredAccountMeta fields pub(crate)
and provides getter functions to access its member fields.
This PR is a preparation step for abstracting out StoredAccountMeta.
2023-02-27 10:03:22 -08:00
ac7e7aa8f0
Move account-meta structs from append_vec.rs to account_meta.rs ( #30443 )
...
#### Problem
As we start supporting new storage formats, those account-meta
structs will be shared by different storage formats and thus need
a new home.
#### Summary of Changes
This PR creates meta.rs under account_storage and moves all the account-meta
structs out from append_vec.rs.
2023-02-22 16:10:34 -08:00
febaf36e6d
Apply clippy fixes for future rust upgrade ( #30363 )
2023-02-16 16:12:51 +00:00
aeb6df3507
Abstract out AppendVec into AccountsFile enum ( #29815 )
...
Abstracts out AppendVec into AccountsFile enum.
This will allow different implementations of AccountsFile that support the AccountsDB.
2023-02-15 17:24:04 -08:00
b86bfbb5c5
measure_us! use Instant and duration_to_us internally ( #30339 )
2023-02-15 12:43:47 -08:00
27f18f96e8
typos, clippy and saturating_add_assign ( #30272 )
...
* typo
* remove clippy allow
---------
Co-authored-by: haoran <haoran@mbook>
2023-02-14 10:03:16 -06:00
253517cba3
add combine_ancient_slots_packed ( #30276 )
...
* add combine_ancient_slots_new
* pr feedback
* implement ==
2023-02-14 06:53:31 -08:00
9988978379
add ancient finish_combine_ancient_slots_packed ( #30261 )
2023-02-13 09:10:13 -08:00
bcd7cf0821
add ancient write_packed_storages ( #30260 )
2023-02-13 07:25:47 -08:00
b3778d5044
cleanup ( #30251 )
2023-02-12 13:42:03 -08:00
dc605b5126
rename parameter and update comments ( #30259 )
2023-02-11 12:13:03 -08:00
c837af1af4
ancient: pass WriteAncientAccounts by &mut ( #30257 )
2023-02-11 11:41:33 -08:00
51d354defb
simplify AccountsToCombine.accounts_keep_slots ( #30246 )
2023-02-10 13:53:03 -08:00
33cc4cc3d5
maintain sorted order on ancient target slots ( #30247 )
2023-02-10 12:08:13 -08:00
157a9d725f
add ancient pack_ancient_storages ( #30235 )
...
* add ancient pack_ancient_storages
* cleanup
2023-02-10 12:06:36 -08:00
dcb2d6c8ae
collect_sort_filter_ancient_slots takes tuning by ref ( #30242 )
2023-02-10 09:40:13 -08:00
0113fe340e
add ancient write_ancient_accounts_multiple_refs ( #30198 )
2023-02-10 09:39:30 -08:00
128ffbdf9b
restore test to all slot combos ( #30231 )
2023-02-10 06:25:37 -08:00
639f3475f5
test and comment cleanup ( #30226 )
...
* test and comment cleanup
* pr feedback
2023-02-09 15:02:10 -08:00
db25ccba52
ancient add write_one_packed_storage ( #30220 )
2023-02-09 14:47:22 -08:00
c6695996b8
add test parameters to fix build ( #30227 )
...
#### Problem
build broken due to pr conflicts
#### Summary of Changes
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:56:12 -08:00
62b93df1c7
change ancient tests to correct size ( #30221 )
...
#### Problem
fix tests to be more realistic
#### Summary of Changes
allocate correctly sized append vecs
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:41:36 -08:00
68b2b1badd
add ancient calc_accounts_to_combine ( #30195 )
...
#### Problem
Building new algorithm for packing ancient storage. Packing will occur
in 1 pass across multiple ancient slots.
This will be put in 1 dead code piece at a time with tests until all
pieces are present. Switch between current packing algorithm and this
new one is in a validator cli argument. Resulting append vecs are
correct and compatible (as a set) either way. When a new storage format
optimized for cold storage becomes available, it will only work with
this new packing algorithm, so the change will need to be complete prior
to the new storage format.
#### Summary of Changes
Add `ancient calc_accounts_to_combine` to separate accounts to prepare
for creating packed ancient append vecs. This will be used soon.
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:06:17 -08:00
0da4697b2f
add ancient write_ancient_accounts ( #30182 )
2023-02-08 09:40:17 -08:00
5cbd6b3a74
minor cleanup ( #30181 )
2023-02-08 08:00:14 -08:00
70d635127e
add `ancient collect_sort_filter_info` ( #30169 )
2023-02-08 07:08:20 -08:00
e14d0638e7
add ancient get_unique_accounts_from_storage_for_combining_ancient_slots ( #30176 )
2023-02-07 18:22:54 -08:00
1eb304de28
NonZeroU64 for ideal_storage_size ( #30168 )
2023-02-07 13:34:27 -08:00
d53c49c230
add ancient filter_ancient_slots ( #30152 )
2023-02-07 10:35:40 -08:00
db4174cbcb
ancient choose_storages_to_shrink ( #30146 )
2023-02-06 14:29:54 -08:00
01f0dcdad4
add ancient ancient filter_by_smallest_capacity and truncate_to_max_storages ( #30129 )
2023-02-06 11:04:53 -08:00
8785942f56
add ancient sort_shrink_indexes_by_bytes_saved ( #30123 )
2023-02-05 12:14:14 -08:00
0bbc4774d7
minor cleanup on tests ( #30125 )
2023-02-05 12:13:39 -08:00
785a6e3a69
AncientSlotInfos, piece of packed ancient storage ( #30119 )
2023-02-03 10:53:49 -08:00
28510f4260
update comments on get_ancient_append_vec_capacity ( #30030 )
2023-02-01 07:22:34 -08:00
272e667cb2
deprecates Pubkey::new in favor of Pubkey::{,try_}from ( #29805 )
...
The commit deprecates Pubkey::new which lacks type-safety and instead
implements TryFrom<&[u8]> and TryFrom<Vec<u8>> for Pubkey.
2023-01-21 18:06:27 +00:00
0f16120b03
remove FoundStoredAccount ( #29591 )
2023-01-10 13:57:34 -08:00
1d675c6205
FoundStoredAccount removes store_id ( #29590 )
2023-01-10 05:28:41 -08:00
b77bef4ee3
write_version -> write_version_obsolete ( #29176 )
2022-12-14 07:43:40 -08:00
e6cc5052c3
pass alive bytes to AccountsToStore::new ( #28994 )
2022-11-30 15:46:52 -08:00
58e12cf963
avoid copies when writing to an ancient append vec ( #28981 )
...
* avoid copies when writing to an ancient append vec
* update comments
2022-11-30 10:09:23 -08:00
Jeff Washington (jwash)
Yueh-Hsuan Chiang
Brooks
Andrew Fitzgerald
HaoranYi
behzad nouri