bb19ebed49
Convert tuple into dedup result struct ( #33450 )
...
convert tupe into dedup result struct
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-09-29 21:12:50 -05:00
0e9e91c65e
Drops the shrink_candidate_slots lock after done inserting ( #33459 )
2023-09-29 16:58:55 +00:00
b81ff5d654
Fixup the metrics for remove_dead_accounts_shrink_us ( #33458 )
2023-09-29 16:04:33 +00:00
6ea51280dd
Move sort timer out of loop ( #33448 )
...
move sort timer out of loop
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-09-28 20:10:41 -05:00
25c27d452c
hash dedup vec ( #33246 )
...
* hash_dedup vec algo
* reviews
* reviews
* more reviews
* simplify working_set init with add_next_item
* refactor to remove special case "new" from add_item.
The new change is that, even the new item is the new min, it will still be added to working_set.
This change will make init working_set code simpler and the loop loop check simpler.
Since the item is inserted in at the end of the vector, the cost of push into and pop from the working will be O(1), shouldn't affect performance much.
* comments
* refactor unnamed tuple in working set to SlotGroupPointer type
* use SlotGroupPointer in ItemLocation
* Add Copy traits to avoid explicty call of clone on SlotGroupPointer
* consume next in add_next_item fn (credit to jeff).
note that the old code is still correct, since before call to
add_next_item, we will have already overwritten `next` to correct value.
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-09-28 08:58:08 -05:00
3608378097
chore(solana-accounts-db): remove unused deps ( #33429 )
2023-09-27 16:27:40 +00:00
ca92e9c387
chore(solana-accounts-db): remove unused deps ( #33420 )
2023-09-27 11:02:05 +00:00
e088eb2be0
Code clean up ( #33417 )
...
clean up
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-09-26 22:35:25 +00:00
746f69772a
[TieredStorage] Streamline the handling of TieredStorageFormat ( #33396 )
...
#### Problem
The TieredStorageFormat field in the TieredStorage is only used in the write path.
#### Summary of Changes
This PR simplifies the handling of TieredStorageFormat by removing its field from
TieredStorage struct but passing via write_accounts().
2023-09-26 14:05:36 -07:00
9f6f532535
`flush_slot_cache_with_clean()` takes a single Slot ( #33413 )
2023-09-26 15:38:37 -04:00
a9b0fb492b
split hash calc ancient slot boundary to allow for ancient shrinking … ( #33216 )
...
split hash calc ancient slot boundary to allow for ancient shrinking to be behind
2023-09-26 07:27:50 -07:00
23ad476ffb
Removes unused `Versioned` trait ( #33360 )
2023-09-25 17:18:38 -04:00
40f536010f
visit_duplicate_pubkeys_during_startup uses scan ( #33397 )
2023-09-25 12:48:29 -07:00
18231e9a5a
dump final startup index stats only after startup is complete ( #33400 )
2023-09-25 12:19:52 -07:00
1b15464414
diskidx: stats for created vs reused ( #33385 )
2023-09-23 06:22:12 -07:00
7cf71011fd
disk bucket: init restart path ( #33375 )
2023-09-22 11:40:46 -07:00
7c545b0ae9
bump rust stable to 1.72.1 ( #33333 )
...
* bump rust stable to 1.72.1
* bump rust nightly to 2023-09-20
* fix nightly lint -- unused doc comment
-- rustdoc does not generate documentation for expression fields
* fix nightly lint -- unnecessarily eager cloning of iterator items
* fix nightly lint -- loop never actually loops
2023-09-20 23:53:36 -06:00
889d6c655e
Moves accounts-db benches to accounts-db crate ( #33306 )
2023-09-19 13:09:42 -04:00
c85eb73300
O(n) dedup ( #33297 )
...
O(N) dedup
Co-authored-by: jeff washington <jeff.washington@solana.com>
2023-09-18 17:55:28 -05:00
8b8a21a52f
cleanup feature: enable request heap frame instruction #30076 ( #33243 )
...
* cleanup feature: enable request heap frame instruction #30076
* update sbf tests
* removed out dated comments and test
2023-09-18 16:06:24 -05:00
17c3930bc8
in gen index, stop using big hashmap ( #33252 )
...
* in gen index, stop using big hashmap
* update accounts_data_len
* remove approx_stored_count
2023-09-18 11:13:18 -07:00
402981e3c1
cleanup test ( #33291 )
2023-09-18 10:59:42 -07:00
27caf4d1d2
add test for duplicates in generate_index and fix approx stored count ( #33290 )
...
add test
2023-09-18 09:46:52 -07:00
9e11ae6275
Make program owners a const array instead of Vec<_> ( #33275 )
...
The program owners pubkeys are constant, no need to reconstruct the
Vec<Pubkey> and Vec<&Pubkey> each time this function runs (every time we
execute transactions).
2023-09-18 10:59:03 -05:00
6283c1d568
Refactors out `unsafe` from cache_hash_data.rs ( #33271 )
2023-09-15 14:25:43 -04:00
a47f65d882
Refactors `unsafe` out of CacheHashDataFile's header ( #33270 )
2023-09-15 16:49:35 +00:00
f77b3d9389
Derives Pod for CalculateHashIntermediate ( #33267 )
2023-09-15 12:47:38 -04:00
4c42413c1f
Refactors out `unsafe` from MmapAccountHashesFile::read() ( #33266 )
2023-09-15 15:09:01 +00:00
3ad8394047
stop removing empty bins during index generation ( #33242 )
2023-09-14 07:53:33 -07:00
886eabd74d
gen idx: refactor StorageSizeAndCount population ( #33244 )
2023-09-14 07:53:05 -07:00
602f0f6ecb
generate index: fn arg: &Slot -> Slot ( #33229 )
...
fn arg: &Slot -> Slot
2023-09-13 14:14:30 -07:00
c2bf2a969d
remove unused metrics ( #33241 )
2023-09-13 14:14:10 -07:00
cb310a310a
set approx store count at generate index time ( #33240 )
2023-09-13 14:13:46 -07:00
093ae61e85
single pass through each storage to populate info ( #33227 )
2023-09-13 12:37:01 -07:00
3ffd78f2d7
add comments ( #33226 )
2023-09-12 13:51:27 -07:00
21e7f98b3a
Uses hex for the hash in accounts hash cache file names ( #33196 )
2023-09-12 11:26:08 -04:00
c61ee20b44
Transient accounts hash cache dir is unnecessary ( #33181 )
2023-09-12 11:23:07 -04:00
3cd3994939
clean up pub crate ( #33214 )
...
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-09-11 19:28:26 -05:00
09936aac0e
Removes `new()` from POD CalculateHashIntermediate and CumulativeOffset ( #33211 )
2023-09-11 22:33:25 +00:00
cf35799b2a
remove unused stats ( #33213 )
...
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-09-11 21:03:37 +00:00
659fc6837b
Removes `pub` from accounts_hash.rs items ( #33209 )
2023-09-11 20:49:51 +00:00
d724af863c
add a few accounts hash dedup tests ( #33208 )
2023-09-11 13:20:28 -07:00
6298c6c31e
Shares accounts hash cache data between full and incremental ( #33164 )
2023-09-11 15:55:48 -04:00
bbb57be0a5
Refactor - Move interfaces of address-lookup-table into the program SDK ( #33165 )
...
* Adds a module `address_lookup_table` to the SDK.
* Adds a module `address_lookup_table::instruction` to the SDK.
* Adds a module `address_lookup_table::error` to the SDK.
* Adds a module `address_lookup_table::state` to the SDK.
* Moves AddressLookupTable into SDK as well.
* Moves AddressLookupTableAccount into address_lookup_table.
* Adds deprecation messages.
* Disentangles dependencies across cargo files.
2023-09-11 21:10:40 +02:00
4dfe62a2f0
rework accounts hash calc dedup to avoid kernel file error ( #33195 )
...
* in hash calc, calculate max_inclusive_num_pubkeys
* in hash calc, dedup uses mmap files to avoid os panic
* as_mut_ptr
* remove unsafe code
* refactor count in hash files
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-09-11 10:57:21 -07:00
a145ade564
stat for time spent copying generate index contents ( #33187 )
...
* stat for time spent copying generate index contents
* rework to move stat to generate_index
* fix fmt
2023-09-08 12:55:54 -07:00
dc6b1eb653
in shrink, get stores prior to shrink starting ( #33194 )
2023-09-08 08:45:32 -07:00
0083e426b2
ShrinkCandidates only holds slot ( #33173 )
2023-09-07 12:28:05 -07:00
e331275178
remove coupling of ShrinkCandidates to HashMap ( #33176 )
2023-09-07 09:07:37 -07:00
f4816dc0cf
Moves CacheHashData test-only methods into tests module ( #33170 )
2023-09-06 15:25:58 -04:00
HaoranYi
Brooks
Yihau Chen
Yueh-Hsuan Chiang
Jeff Washington (jwash)
Trent Nelson
Tao Zhu
steviez
Alexander Meißner