4f83818fdb
get rid of WriteAccountMapEntry instance ( #19093 )
2021-09-27 16:24:19 -05:00
b3bb079d9f
AcctIdx: condense upsert to always use read lock ( #20148 )
2021-09-23 19:19:27 -05:00
334d38d200
AcctIdx: generation per thread starts at a random bucket ( #20122 )
2021-09-23 19:59:37 +00:00
4d3e32803f
AcctIdx: PreAllocatedAccountMapEntry ( #20117 )
2021-09-23 08:11:32 -05:00
0eb0d7f73b
AcctIdx: consolidate lock_and_update_slot_list ( #20090 )
2021-09-22 18:40:19 +00:00
1eeddf2e23
orderings on AcctIdx dirty ( #20087 )
2021-09-22 11:24:16 -05:00
c944bfb1d9
AcctIdx tests use disk buckets ( #20070 )
2021-09-22 08:55:58 -05:00
6c518102dd
AcctIdx: fix some missing dirty calls ( #20089 )
2021-09-22 08:45:08 -05:00
e6795e6fdd
AcctIdx: get can load from disk ( #20067 )
2021-09-21 12:40:07 -05:00
fb71e80962
AcctIdx: accessors for age ( #20023 )
2021-09-21 08:41:17 -05:00
af5b0d42a8
AcctIdx: initial index items have future age to flush ( #20010 )
2021-09-19 20:22:09 -05:00
c1d181add5
add --accounts-index-memory-limit-mb ( #19269 )
2021-09-19 18:00:15 -05:00
742155c214
call set_startup and add metrics on generate_index ( #20006 )
2021-09-18 22:08:58 -05:00
bed0049a51
AcctIdx: add age per item in in_mem acct idx ( #19981 )
2021-09-18 09:56:26 -05:00
4e038e94fd
add 'drives' to AccountsIndexConfig ( #19989 )
2021-09-18 09:54:57 -05:00
bd86f41e18
acct idx tests use 2 bins instead of 8k ( #19996 )
2021-09-18 09:53:44 -05:00
4dc2f08198
AcctIdx: hold_range_in_memory ( #19955 )
2021-09-17 22:19:29 +00:00
24b136a993
AcctIdx: configurable flush threads ( #19983 )
2021-09-17 22:02:43 +00:00
99f5684dc4
AcctIdx: handle future config defaults ( #19967 )
2021-09-17 13:12:06 -05:00
37d6f5013f
AcctIdx: initialize acct idx by config ( #19968 )
2021-09-17 16:39:41 +00:00
4d8b3aa578
AcctsIdx: introduce dirty state per pubkey ( #19960 )
2021-09-16 21:36:23 -05:00
66e0fafc21
AccountsIndexStorage holds InMemAccountsIndex[] ( #19947 )
2021-09-16 21:12:22 +00:00
eddd583cd5
AcctIdx: pass bins to BucketMapHolder ( #19906 )
2021-09-15 18:07:53 +00:00
99f2c746d1
AcctIdx: Put InMemAcctIndex behind arc ( #19904 )
2021-09-15 17:36:08 +00:00
fc647eed19
introduce trait IndexValue for accounts index ( #19859 )
2021-09-14 17:51:07 -05:00
910f241c3f
add <T> through accounts index ( #19852 )
2021-09-13 22:59:03 -05:00
dca49a614f
size and per-bin stats ( #19842 )
2021-09-13 19:26:49 -05:00
d388c5c257
move update_key_if_exists to InMemAccountsIndex ( #19803 )
2021-09-13 08:27:21 -05:00
b992c02708
in_mem_accounts_index filters by range ( #19779 )
2021-09-12 21:57:15 -05:00
0263ffb2ed
move upsert and a handful of helpers to InMemAccountsIndex ( #19799 )
2021-09-12 21:54:09 -05:00
3617d43e76
accounts index stats ( #19797 )
2021-09-12 22:14:59 +00:00
d9674f7ff0
consolidate another use of remove_if_slot_list_empty ( #19804 )
2021-09-12 19:05:44 +00:00
c9a3b8941f
move insert_new_entry_if_missing_with_lock into InMemAccountsIndex ( #19800 )
2021-09-12 13:39:29 -05:00
a295febf04
move remove_if_slot_list_empty to InMemAccountsIndex ( #19801 )
2021-09-12 10:26:04 -05:00
91c3b18e1e
InMemAccountsIndex iter returns vec<owned> ( #19778 )
2021-09-11 20:33:04 +00:00
7de2236284
accounts index get returns value ( #19777 )
2021-09-11 13:28:56 -05:00
ed7a6c6732
push use of AccountMapEntry one level deeper ( #19775 )
2021-09-11 02:09:51 +00:00
cba834808a
refactor generate_index zero lamports ( #19769 )
2021-09-10 23:46:08 +00:00
595bba95b4
accounts index traits ( #19768 )
2021-09-10 18:32:10 -05:00
9899cd359a
accounts index iterator uses copy of pubkey and account map entry ( #19767 )
2021-09-10 17:52:49 -05:00
11b10439b4
factor out InMemAccountsIndex to prepare for disk index ( #19773 )
2021-09-10 17:52:25 -05:00
d3f938f0cf
Remove Copy from AccountsIndexConfig. Not all types will support it ( #19686 )
2021-09-07 20:09:40 -05:00
e6055010eb
accounts index iterator takes &Range ( #19618 )
2021-09-03 18:00:49 -05:00
df040c05da
get rid of unneeded mut and refactor addref ( #19594 )
2021-09-03 10:45:37 -05:00
57f51352f6
refactor AccountEntry addref/unref ( #19583 )
2021-09-02 23:25:27 +00:00
682daf1117
cleanup of allocate_accounts_index ( #19585 )
2021-09-02 22:58:07 +00:00
78585a992b
add IsCached trait bounds to AccountsIndex ( #19584 )
2021-09-02 22:45:35 +00:00
2874f6b0a2
create remove_if_slot_list_empty to define accounts index behavior ( #19581 )
2021-09-02 16:54:37 -05:00
0571962776
accounts index test uses # test bins ( #19580 )
2021-09-02 16:32:42 -05:00
7eb793a55e
add debugging to test ( #19563 )
...
* add debugging to test
* update cargo
2021-09-02 02:52:16 +00:00
82a6bbe068
add Debug trait to range for accounts index ( #19531 )
2021-09-01 08:13:56 -05:00
12dc8749cf
add Copy trait to AccountInfo for fast copies to mmapped file ( #19524 )
2021-09-01 08:13:08 -05:00
2df96cd81e
remove type specifics to make funciton more flexible ( #19536 )
2021-08-31 20:03:42 -05:00
a655b01700
some basic accounts index refactoring ( #19510 )
2021-08-30 18:40:10 -05:00
02b050e0f5
replace AccountsIndex btree with hashmap of 8k bins ( #19212 )
2021-08-26 18:12:43 -05:00
7c70f2158b
accounts_index_bins to AccountsIndexConfig ( #19257 )
...
* accounts_index_bins to AccountsIndexConfig
* rename param bins -> config
* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
651343688d
Use option for account index bucket parameter ( #19150 )
2021-08-10 11:32:25 -05:00
47e0d9aa95
plumb more accounts_index bins ( #19123 )
2021-08-10 05:45:46 -05:00
db14bcc43b
AccountsIndex RefCount() returns u64 instead of atomic ( #19088 )
2021-08-09 08:59:56 -05:00
9616ae0f2c
signal to upsert whether reclaims are important ( #18900 )
...
* signal to upsert whether reclaims are important
* reclaims_must_be_empty -> previous_slot_entry_was_cached
* UPSERT_RECLAIMS_MUST_BE_EMPTY_FALSE -> UPSERT_PREVIOUS_SLOT_ENTRY_WAS_CACHED_FALSE
2021-08-09 08:58:59 -05:00
8878f526ce
rework AccountsIndex traits ( #19089 )
2021-08-06 08:39:34 -05:00
24207a09ac
remove AccountsIndex::default ( #19082 )
...
* accounts_db calls AccountsDb::new(bins)
* remove AccountsIndex::default
2021-08-05 11:38:53 -05:00
5cf28689e6
accounts_db calls AccountsDb::new(bins) ( #19068 )
2021-08-05 11:15:26 -05:00
67788ad206
move AccountsIndex upsert into static WriteAccountMapEntry ( #18899 )
...
* rework accounts index to push upsert deeper
* clean up return value of upsert_existing_key
* upsert_existing_key -> update_key_if_exists
* upsert_new_key -> upsert
* upsert_item -> lock_and_update_slot_list
* update_static -> update_slot_list
2021-08-05 08:45:08 -05:00
087db70df6
add traits required by IsCached ( #19066 )
2021-08-05 08:43:00 -05:00
dfe99efa7c
introduce AccountsIndex::default_for_tests() ( #19067 )
2021-08-04 21:58:53 -05:00
5a4979f25f
Handle 0-lamport account in index generation ( #19041 )
...
* Handle 0-lamport account in index generation
* rename duplicate to dirty keys
Co-authored-by: Carl Lin <carl@solana.com>
2021-08-04 23:33:47 +00:00
0b8d14b0fc
move towards account index being dynamically allocated ( #19034 )
2021-08-04 15:28:35 -05:00
31a620c42b
move towards accounts index being dynamic ( #19032 )
2021-08-04 09:18:05 -05:00
6dadc75a6f
during range, stop scanning earlier than currently scanning ( #18896 )
...
* during range, stop scanning earlier than currently scanning
* refactor for better testing
* clean up a different edge case
2021-07-27 13:40:45 -05:00
53d8cad206
remove unused return value from account index upsert ( #18895 )
2021-07-27 08:46:27 -05:00
181f21529d
add Debug trait to T in AccountsIndex ( #18702 )
2021-07-16 17:05:23 -05:00
958d27bb0e
accounts index scan starts looking in bin where bound would exist ( #18613 )
2021-07-15 10:26:50 -05:00
da9fdd785c
measure and speed up 2nd half of generate_index ( #18503 )
...
* add metrics and speedup storage calcs in generate_index
* add tests, refactor
* rename map variable
* rename another index variable
2021-07-15 10:26:00 -05:00
bb41cf3461
remove unnecessary generic args to make refactoring easier ( #18619 )
2021-07-13 12:05:23 -05:00
0bd8710d34
prevent excess allocation with AccountsIndexIterator ( #18605 )
2021-07-13 11:11:17 -05:00
f5ff4b2058
serial insertion of bins into accounts index ( #18469 )
2021-07-12 12:00:45 -05:00
f04340b125
move allocation out of critical section and estimate capacity better ( #18450 )
2021-07-07 15:36:40 -05:00
49c4e54b28
accounts index insert uses pubkey by value ( #18460 )
2021-07-07 15:35:35 -05:00
afd64d27c9
bin accounts index ( #18111 )
2021-06-28 15:03:57 -05:00
1f1e54b9d8
reduce pubkey copies on insert path ( #18240 )
2021-06-28 12:04:29 -05:00
e06376664b
replace account index bulk insert with iterator ( #18198 )
2021-06-25 14:31:55 -05:00
f2a2581259
metric for accounts index insertion time ( #18202 )
2021-06-25 08:36:55 -05:00
9429d0463c
chore: bump ouroboros from 0.5.1 to 0.9.3 ( #18189 )
...
* chore: bump ouroboros from 0.5.1 to 0.9.3
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.5.1 to 0.9.3.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Api changes
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-25 00:53:54 -06:00
f9fccdee85
remove unnecessary copies from accounts index code paths ( #18196 )
2021-06-24 17:56:25 -05:00
4b314be5bd
rework dirty_pubkeys from insert_new_if_missing_into_primary_index ( #18200 )
2021-06-24 14:52:11 -05:00
3b1738c000
Keep track of dirty stores on remove accounts to clean ( #17601 )
...
* Keep track of dirty stores on remove accounts to clean
and not zero_lamport key set
* Only dirty when count==0?
* Add another clean
2021-06-23 10:28:35 +02:00
ccf6b21bf8
Removed unused purged_slots in purge_older_root_entries ( #18102 )
...
Removed unused purged_slots in purge_older_root_entries
2021-06-22 15:27:37 -07:00
6514096a67
chore: cargo +nightly clippy --fix -Z unstable-options
2021-06-18 10:42:46 -07:00
0a81c37fce
introduce LockMapType for accounts_index ( #18021 )
2021-06-17 14:41:52 -05:00
ccc013e134
Handle removing slots during account scans ( #17471 )
2021-06-14 21:04:01 -07:00
c2191d885d
Fix accounts index panic in purge_exact ( #17757 )
2021-06-14 16:10:26 +02:00
d4cc975fe9
calculate_capitalization uses hash calculation ( #17443 )
...
* calculate_capitalization uses hash calculation
* feedback
* remove debugging code, clean up slot math
2021-06-14 08:53:07 -05:00
58fe1d0764
add min to roots_tracker ( #17806 )
2021-06-08 08:32:16 -06:00
14c52ab018
Skip shrink when it doesn't save anything ( #17405 )
2021-06-02 09:51:46 +02:00
8924fbf6a0
RollingBitField: bug fixes and add tests ( #17525 )
...
* RollingBitField: bug fixes and add tests
* respond to feedback
2021-05-28 16:33:07 +00:00
49402b7d82
Only add keys with multiple index entries to the clean set ( #17473 )
2021-05-27 16:31:24 +02:00
d8bc56fa51
Refactor purge_slots_from_cache_and_store() and handle_reclaims() ( #17319 )
2021-05-24 13:51:17 -07:00
3f3324231d
Hold read lock during startup shrink ( #17309 )
...
* hold read lock during account scan of shrink
* rename and improve rusty
2021-05-21 13:59:32 -05:00
8143ee5502
add AccountsIndex AccountMapsReadLock type ( #17365 )
2021-05-21 10:02:14 -05:00
Jeff Washington (jwash)
sakridge
dependabot[bot]
Lijun Wang
Alexander Meißner
carllin