Jeff Washington (jwash)
32ec8341f9
generate_index inserts ideal initial data ( #17247 )
...
* improve insert into map initially
* rework towards single code path
* rename
* update test
2021-05-19 16:21:24 -05:00
Jeff Washington (jwash)
ed9cbd50f0
move Ancestors to its own module ( #17316 )
2021-05-19 11:50:34 -05:00
Jeff Washington (jwash)
4788976517
rework slot list update ( #17232 )
2021-05-19 08:48:46 -05:00
Jeff Washington (jwash)
a3c0833a1c
tests: cleanup test ( #17305 )
...
* tests: cleanup test
* 2 more replacements
2021-05-18 18:08:37 -05:00
Jeff Washington (jwash)
d1db5448b9
hold lock to speed up insert ( #17194 )
...
* hold lock to speed up insert
* add tests
2021-05-17 11:58:33 -05:00
Jeff Washington (jwash)
3745e0babc
minor refactoring of AccountsIndex in preparation of bigger things ( #17237 )
2021-05-14 15:27:10 -05:00
carllin
239ab8799c
Remove bloat from secondary indexes ( #17048 )
2021-05-12 15:29:30 -07:00
Jeff Washington (jwash)
7d96f78821
include/exclude keys on account secondary index ( #17110 )
...
* AccountSecondaryIndexes.include/exclude
* use normal scan if key is not indexed
* add a test to ask for a scan for an excluded secondary index
* fix up cli args
2021-05-11 22:06:22 +00:00
Jeff Washington (jwash)
f39dda00e0
type AccountSecondaryIndexes = HashSet ( #17108 )
2021-05-10 14:22:48 +00:00
Jeff Washington (jwash)
a6a1355b80
fix secondary index test ( #17111 )
2021-05-10 08:54:30 -05:00
Jeff Washington (jwash)
dc0429f5e6
add metric for assumption ( #17061 )
2021-05-06 15:04:13 -05:00
Jeff Washington (jwash)
ffbe8906ed
get root lock once ( #16829 )
2021-05-05 15:17:45 -05:00
Brooks Prumo
17e6bd579f
Clean unrooted dropped banks ( #16580 ) ( #16911 )
...
In a scenario where a bank is unrooted and dropped, any keys that exist
_only_ in that bank are now cleaned up.
This work was originally based on PR #15106 .
2021-04-30 15:34:38 -05:00
Jeff Washington (jwash)
23d67e4ac7
stretchy roots tracker ( #16830 )
...
* stretchy roots tracker
* rename hash to hash_set in tests
* update comment
* try 2 widths in test
* bool iter
* add assert
* helper function for bitfield insert/remove
* introduce RollingBitFieldTester
* another bool iter replacement
* map cleanup
* map to cloned
2021-04-29 09:11:28 -05:00
Jeff Washington (jwash)
9070191b8b
stats on clean for removing roots ( #16849 )
...
* stats on clean for removing roots
* rename
* accumulate and swap metrics
2021-04-28 13:24:01 -05:00
Jeff Washington (jwash)
47ca7063f2
rootstracker.remove returns previous state ( #16831 )
2021-04-27 09:15:44 -05:00
Jeff Washington (jwash)
1cc9a1c0eb
log roots range metric ( #16636 )
...
* log roots range metric
* rename
2021-04-23 16:09:39 +00:00
Jeff Washington (jwash)
91be2903da
Reduce account index lookups during clean ( #16689 )
...
* reduce account index lookups in clean
* rename
* rename enum
* hold locks during removal from zero pubkey list
* merge with zero lamport fix
* tests
2021-04-23 09:33:14 -05:00
Jeff Washington (jwash)
9852572eb9
increase rolling slot width to 4M ( #16583 )
2021-04-20 21:18:15 -05:00
Jeff Washington (jwash)
015bc034a5
improve failing assert ( #16581 )
2021-04-19 08:55:01 -05:00
Michael Vines
a911ae00ba
clippy
2021-04-18 20:55:02 -07:00
carllin
d747614b27
Account for possibility of cache flush in load() ( #15454 )
...
* Account for possibility of cache flush in load()
* More cleaning
* More cleaning
* Remove unused method and some comment cleaning
* Fix typo
* Make the detected impossible purge race panic()!
* Finally revert to original .expect()
* Fix typos...
* Add assertion for max_root for easier reasoning
* Reframe races with LoadHint as possible opt.
* Fix test
* Make race bug tests run longer for less flaky
* Delay the clone-in-lock slow path even for RPC
* Make get_account panic-free & add its onchain ver.
* Fix rebase conflicts...
* Clean up
* Clean up comment
* Revert fn name change
* Fix flaky test...
* fmt...
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-04-17 00:23:32 +09:00
Jeff Washington (jwash)
105a6bfb46
Replace RootsTracker HashSet ( #16310 )
...
* Replace RootsTracker HashSet
* use bitvec
* cleanup, add brenchmark test
* test cleanup
* add lots of tests
* get rid of demo
* change warp test constant
* get rid of unused function
* pr feedback
* reorder use
* rework get_all to remove range checks
* add tests, fix bugs
2021-04-12 12:11:33 -05:00
Jeff Washington (jwash)
7ba3e710d3
latest_slot uses > current_max ( #16374 )
2021-04-12 10:52:24 -05:00
Jeff Washington (jwash)
6930a77a0f
prepare replace Ancestors HashMap for performance ( #16476 )
2021-04-12 10:51:57 -05:00
Jeff Washington (jwash)
95dc7b5449
reduce lock contention on latest_root ( #16306 )
2021-04-05 11:35:14 -05:00
Jeff Washington (jwash)
7a997759fa
reduce locking on get_rooted_entries ( #16304 )
2021-04-05 11:26:53 -05:00
Jeff Washington (jwash)
8a3135d17b
Account->AccountSharedData ( #15691 )
2021-03-09 15:06:07 -06:00
Greg Fitzgerald
edd159e7d7
Fix typos ( #15721 )
2021-03-05 07:16:53 +00:00
Greg Fitzgerald
2463cc1e6a
Fix typos ( #15610 )
2021-03-02 06:36:49 -08:00
carllin
e4d0d4bfae
Fix pubkey refcount for shrink + clean ( #14987 )
2021-02-04 12:44:19 -08:00
sakridge
e5225b7e68
Add ref count from storage ( #15078 )
2021-02-04 09:11:05 -08:00
carllin
2970b59853
Don't load all accounts into memory for capitalization check ( #14957 )
...
* Don't load all accounts into memory for capitalization check
2021-02-03 15:00:42 -08:00
carllin
72f10f5f29
Aggregate purge and shrink metrics ( #14763 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-27 01:39:47 -08:00
sakridge
0d32a0e0f4
Partial clean ( #14800 )
...
* Revert "Revert "Partial accounts clean (#14652 )" (#14777 )"
This reverts commit ad2e10e17b
.
* Remove squashed uncleaned keys
2021-01-24 09:50:19 -08:00
sakridge
96b27ca0cb
Remove zero_lamport_pubkeys reference ( #14808 )
2021-01-23 18:31:55 +00:00
sakridge
424bb797a6
Speed up generate_index ( #14792 )
2021-01-23 08:05:05 -08:00
carllin
c77461e428
Remove unnecesary flushes in previous roots ( #14596 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-23 04:02:44 -08:00
sakridge
ad2e10e17b
Revert "Partial accounts clean ( #14652 )" ( #14777 )
...
This reverts commit ca7914aa3d
.
2021-01-22 18:05:49 -08:00
sakridge
ca7914aa3d
Partial accounts clean ( #14652 )
...
Clean less keys by tracking the two cases:
* Touched keys per slot in uncleaned_keys derived from
accounts delta hash operation.
* Set of keys with any zero-lamport updates.
2021-01-21 11:01:53 -08:00
carllin
5f14f45282
More generic accounts purge functions ( #14595 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-17 20:31:03 -08:00
carllin
6dfad0652f
Cache account stores, flush from AccountsBackgroundService ( #13140 )
2021-01-11 17:00:23 -08:00
Ryo Onodera
c9df6134fa
Save 7G mem on mainnet fixing AccIndex overalloc. ( #14435 )
2021-01-06 09:48:59 +09:00
carllin
5affd8aa72
Add secondary indexes ( #14212 )
2020-12-31 18:06:03 -08:00
sakridge
baa9602411
Add shrink paths ( #14238 )
2020-12-21 21:33:37 -08:00
Michael Vines
7143aaa89b
Clippy
2020-12-14 08:03:29 -08:00
carllin
955b99cf69
Fix roots_and_ref_count ( #13827 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-26 13:09:20 -08:00
carllin
791fb17437
Prevent scans on unrooted slots from seeing partial clean ( #13628 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-20 13:01:04 -08:00
carllin
6276360468
Prevent scans from seeing root updates/clean ( #13464 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-16 17:23:11 -08:00
joshua-maros
a8a77614fe
Fix potential undefined behavior ( #13555 )
...
* Switch to ouroboros 0.5.1
* Update other lock files
2020-11-13 18:12:41 +09:00