Commit Graph

45 Commits

Author SHA1 Message Date
Jeff Washington (jwash) 51d354defb
simplify AccountsToCombine.accounts_keep_slots (#30246) 2023-02-10 13:53:03 -08:00
Jeff Washington (jwash) 33cc4cc3d5
maintain sorted order on ancient target slots (#30247) 2023-02-10 12:08:13 -08:00
Jeff Washington (jwash) 157a9d725f
add ancient pack_ancient_storages (#30235)
* add ancient pack_ancient_storages

* cleanup
2023-02-10 12:06:36 -08:00
Jeff Washington (jwash) dcb2d6c8ae
collect_sort_filter_ancient_slots takes tuning by ref (#30242) 2023-02-10 09:40:13 -08:00
Jeff Washington (jwash) 0113fe340e
add ancient write_ancient_accounts_multiple_refs (#30198) 2023-02-10 09:39:30 -08:00
Jeff Washington (jwash) 128ffbdf9b
restore test to all slot combos (#30231) 2023-02-10 06:25:37 -08:00
Jeff Washington (jwash) 639f3475f5
test and comment cleanup (#30226)
* test and comment cleanup

* pr feedback
2023-02-09 15:02:10 -08:00
Jeff Washington (jwash) db25ccba52
ancient add write_one_packed_storage (#30220) 2023-02-09 14:47:22 -08:00
Jeff Washington (jwash) c6695996b8
add test parameters to fix build (#30227)
#### Problem
build broken due to pr conflicts

#### Summary of Changes


Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:56:12 -08:00
Jeff Washington (jwash) 62b93df1c7
change ancient tests to correct size (#30221)
#### Problem
fix tests to be more realistic

#### Summary of Changes
allocate correctly sized append vecs

Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:41:36 -08:00
Jeff Washington (jwash) 68b2b1badd
add ancient calc_accounts_to_combine (#30195)
#### Problem
Building new algorithm for packing ancient storage. Packing will occur
in 1 pass across multiple ancient slots.
This will be put in 1 dead code piece at a time with tests until all
pieces are present. Switch between current packing algorithm and this
new one is in a validator cli argument. Resulting append vecs are
correct and compatible (as a set) either way. When a new storage format
optimized for cold storage becomes available, it will only work with
this new packing algorithm, so the change will need to be complete prior
to the new storage format.

#### Summary of Changes
Add `ancient calc_accounts_to_combine` to separate accounts to prepare
for creating packed ancient append vecs. This will be used soon.

Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:06:17 -08:00
Jeff Washington (jwash) 0da4697b2f
add ancient write_ancient_accounts (#30182) 2023-02-08 09:40:17 -08:00
Jeff Washington (jwash) 5cbd6b3a74
minor cleanup (#30181) 2023-02-08 08:00:14 -08:00
Jeff Washington (jwash) 70d635127e
add `ancient collect_sort_filter_info` (#30169) 2023-02-08 07:08:20 -08:00
Jeff Washington (jwash) e14d0638e7
add ancient get_unique_accounts_from_storage_for_combining_ancient_slots (#30176) 2023-02-07 18:22:54 -08:00
Jeff Washington (jwash) 1eb304de28
NonZeroU64 for ideal_storage_size (#30168) 2023-02-07 13:34:27 -08:00
Jeff Washington (jwash) d53c49c230
add ancient filter_ancient_slots (#30152) 2023-02-07 10:35:40 -08:00
Jeff Washington (jwash) db4174cbcb
ancient choose_storages_to_shrink (#30146) 2023-02-06 14:29:54 -08:00
Jeff Washington (jwash) 01f0dcdad4
add ancient ancient filter_by_smallest_capacity and truncate_to_max_storages (#30129) 2023-02-06 11:04:53 -08:00
Jeff Washington (jwash) 8785942f56
add ancient sort_shrink_indexes_by_bytes_saved (#30123) 2023-02-05 12:14:14 -08:00
Jeff Washington (jwash) 0bbc4774d7
minor cleanup on tests (#30125) 2023-02-05 12:13:39 -08:00
Jeff Washington (jwash) 785a6e3a69
AncientSlotInfos, piece of packed ancient storage (#30119) 2023-02-03 10:53:49 -08:00
Jeff Washington (jwash) 28510f4260
update comments on get_ancient_append_vec_capacity (#30030) 2023-02-01 07:22:34 -08:00
behzad nouri 272e667cb2
deprecates Pubkey::new in favor of Pubkey::{,try_}from (#29805)
The commit deprecates Pubkey::new which lacks type-safety and instead
implements TryFrom<&[u8]> and TryFrom<Vec<u8>> for Pubkey.
2023-01-21 18:06:27 +00:00
Jeff Washington (jwash) 0f16120b03
remove FoundStoredAccount (#29591) 2023-01-10 13:57:34 -08:00
Jeff Washington (jwash) 1d675c6205
FoundStoredAccount removes store_id (#29590) 2023-01-10 05:28:41 -08:00
Jeff Washington (jwash) b77bef4ee3
write_version -> write_version_obsolete (#29176) 2022-12-14 07:43:40 -08:00
Jeff Washington (jwash) e6cc5052c3
pass alive bytes to AccountsToStore::new (#28994) 2022-11-30 15:46:52 -08:00
Jeff Washington (jwash) 58e12cf963
avoid copies when writing to an ancient append vec (#28981)
* avoid copies when writing to an ancient append vec

* update comments
2022-11-30 10:09:23 -08:00
Jeff Washington (jwash) 5d88a9b32b
improve perf of storing accounts for shrink/ancient (#28880)
* improve perf of storing accounts for shrink/ancient

* rename

* phantom data

* update comment

* make impl Borrow<Hash> consistent

* remove unused static
2022-11-22 15:36:57 -08:00
Jeff Washington (jwash) 9acfa08838
remove per-account slot in ancient append vecs (#28851)
* remove per-account slot in ancient append vecs

* review fixes
2022-11-18 10:15:41 -08:00
Jeff Washington (jwash) 0b51b15af2
remove duplicate pubkey during shrink (#28689) 2022-11-01 07:45:52 -07:00
Jeff Washington (jwash) c47515d055
remove duplicate pubkey when shrinking or combining ancient (#28678) 2022-10-31 12:00:55 -07:00
Jeff Washington (jwash) ca115594ff
remove concept of 'full' ancient append vec (#28449) 2022-10-18 15:08:04 -07:00
Jeff Washington (jwash) 6efe7e095b
temporarily make ancient append vecs 10x smaller to find intermittent bugs (#26574) 2022-07-19 08:51:36 -05:00
Jeff Washington (jwash) d9eee72edb
remove FoundStoredAccount.account_size (#26460) 2022-07-07 18:37:14 -05:00
Jeff Washington (jwash) e164879f20
load_accounts_index_for_shrink uses &item instead of (&item.0, &item.1) (#26266) 2022-06-27 22:40:45 -05:00
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
Jeff Washington (jwash) 1b4e4ef548
impl squash to ancient append vecs (#24538)
* squash to ancient append vecs

* pr feedback

* clippy
2022-05-20 10:20:17 -05:00
Jeff Washington (jwash) 2cf7c73ced
cleanup types in tests (#25254) 2022-05-16 15:54:40 -05:00
Jeff Washington (jwash) 7826fef120
clippy (#25206) 2022-05-13 21:03:54 -05:00
Jeff Washington (jwash) d7dcb62fe1
ancient_append_vec refactoring (#25203) 2022-05-13 16:40:26 -05:00
Jeff Washington (jwash) 849cbe93aa
add AccountsToStore::has_overflow (#25143) 2022-05-12 10:14:27 -05:00
Jeff Washington (jwash) 11fa0db850
more ancient append vec fns (#25127) 2022-05-11 08:02:05 -05:00
Jeff Washington (jwash) c4506269a6
add ancient_append_vecs (#25119) 2022-05-10 16:41:04 -05:00