Jeff Washington (jwash)
d802eb303c
reduce copy ( #17672 )
2021-06-02 14:03:02 -05:00
carllin
bbcdf073ba
Support out of band dumping of unrooted slots in AccountsDb ( #17269 )
...
* Accounts dumping logic
* Add test for interaction between cache flush and remove_unrooted_slot()
* Update comments
* Rename
* renaming
* Add more comments
* Renaming
* Fixup test and bad check
2021-06-02 09:51:10 +00:00
sakridge
14c52ab018
Skip shrink when it doesn't save anything ( #17405 )
2021-06-02 09:51:46 +02:00
Jeff Washington (jwash)
738cc9549f
parallel get_snapshot_storages ( #17589 )
2021-06-01 19:33:12 -05:00
Jeff Washington (jwash)
c8fdf0762c
add Ancestors to get_snapshot_storages to prepare for the future ( #17591 )
2021-06-01 15:14:35 -05:00
Jeff Washington (jwash)
ef5169ff24
sort storages by slot before scan ( #17411 )
...
* sort storages by slot before scan
* fix return value
2021-06-01 15:07:46 -05:00
Jeff Washington (jwash)
72bb271a94
add metric for collecting storages ( #17527 )
2021-06-01 13:17:49 -05:00
Jeff Washington (jwash)
886898eabf
rework test for flexibility ( #17592 )
...
* rework test for flexibility
* respond to pr feedback
2021-05-31 22:49:17 -05:00
Jeff Washington (jwash)
654918ab27
refactor SnapshotStorage helpers to prepare for later changes ( #17560 )
2021-05-28 13:54:53 -05:00
Jeff Washington (jwash)
55c22d3b76
add check_hash to non-index hash calculation ( #17558 )
2021-05-28 10:24:40 -05:00
sakridge
49402b7d82
Only add keys with multiple index entries to the clean set ( #17473 )
2021-05-27 16:31:24 +02:00
Jeff Washington (jwash)
3973bf08f1
move local out of call ( #17519 )
2021-05-27 08:53:11 -05:00
sakridge
3f9e3c7375
Remove copied shrink code ( #17385 )
2021-05-26 19:27:18 +02:00
Jeff Washington (jwash)
6b9d8d41a3
add --limit_load_slot_count_from_snapshot to ledger-tool ( #17417 )
2021-05-26 10:36:12 -05:00
Jeff Washington (jwash)
40bb6577fc
initialize purge stats ( #17345 )
2021-05-25 09:24:38 -05:00
Jeff Washington (jwash)
d4fc9e7ed4
ad metric for collect ( #17418 )
2021-05-24 18:14:52 -05:00
carllin
d8bc56fa51
Refactor purge_slots_from_cache_and_store() and handle_reclaims() ( #17319 )
2021-05-24 13:51:17 -07:00
Jeff Washington (jwash)
3f3324231d
Hold read lock during startup shrink ( #17309 )
...
* hold read lock during account scan of shrink
* rename and improve rusty
2021-05-21 13:59:32 -05:00
Jeff Washington (jwash)
2f55547d37
remove flatten in storage scan ( #17354 )
...
* remove flatten in storage scan
* fix tests
2021-05-21 09:26:46 -05:00
Jeff Washington (jwash)
45552d271a
take mutex once in process_dead_slots ( #17357 )
2021-05-20 19:50:58 -05:00
Jeff Washington (jwash)
0486df02ba
don't limit to thread pool when cleaning on startup ( #17317 )
2021-05-20 14:36:35 -05:00
Jeff Washington (jwash)
33ab9c4e8d
batch insert account_index items in generate_index ( #17290 )
2021-05-20 10:29:13 -05:00
Jeff Washington (jwash)
9b74988fc6
calculate_accounts_hash does not throttle threads on startup ( #17338 )
2021-05-20 10:25:54 -05:00
Jeff Washington (jwash)
a544010b03
don't log shrink metrics on first call ( #17328 )
...
* don't log shrink metrics on first call
* simplify logic
2021-05-20 00:15:35 +00:00
Jeff Washington (jwash)
b5302e7619
add calc_stored_meta_us metric ( #17318 )
2021-05-19 12:29:41 -05:00
Jeff Washington (jwash)
c20b27bc8f
shink all in parallel on startup ( #17308 )
2021-05-19 12:15:24 -05:00
Jeff Washington (jwash)
ed9cbd50f0
move Ancestors to its own module ( #17316 )
2021-05-19 11:50:34 -05:00
Tao Zhu
0781fe1b4f
Upgrade Rust to 1.52.0 ( #17096 )
...
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script
* upgrade to Rust 1.52.1
* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
2021-05-19 09:31:47 -05:00
Jeff Washington (jwash)
7fe24c455c
simplify generate_index ( #17292 )
2021-05-19 08:11:41 -05:00
Jeff Washington (jwash)
2175fc0625
calculate account infos before lock ( #17245 )
2021-05-18 08:06:19 -05:00
Jeff Washington (jwash)
3a56a56d69
StoredMetaWriteVersion ( #17293 )
2021-05-17 23:58:36 +00:00
Jeff Washington (jwash)
d461aba6a4
don't try to build secondary indexes if we don't need them ( #17246 )
2021-05-17 17:21:15 -05:00
Jeff Washington (jwash)
d1db5448b9
hold lock to speed up insert ( #17194 )
...
* hold lock to speed up insert
* add tests
2021-05-17 11:58:33 -05:00
Jeff Washington (jwash)
fd88db7339
ancestors::new -> default ( #17195 )
2021-05-14 15:20:59 -05:00
Jeff Washington (jwash)
c6f7867aa1
log mismatch ( #17209 )
2021-05-14 09:50:11 -05:00
Jeff Washington (jwash)
3dbc7744ab
metrics for generating index time ( #17192 )
...
* metrics for generating index time
* update metrics to include scan time
2021-05-13 14:32:19 -05:00
carllin
239ab8799c
Remove bloat from secondary indexes ( #17048 )
2021-05-12 15:29:30 -07:00
Jeff Washington (jwash)
7d96f78821
include/exclude keys on account secondary index ( #17110 )
...
* AccountSecondaryIndexes.include/exclude
* use normal scan if key is not indexed
* add a test to ask for a scan for an excluded secondary index
* fix up cli args
2021-05-11 22:06:22 +00:00
Jeff Washington (jwash)
f39dda00e0
type AccountSecondaryIndexes = HashSet ( #17108 )
2021-05-10 14:22:48 +00:00
Jeff Washington (jwash)
225ec00ec4
add comment ( #17059 )
2021-05-06 15:17:41 -05:00
steviez
edc2ab3e48
Test account index and store alignment ( #17038 )
...
* Use ReclaimResult::Default() instead of building subtypes
* Add test to ensure account_db store and index are aligned
2021-05-06 14:13:44 -05:00
Brooks Prumo
e6f49a3e79
Clean unrooted unfrozen banks ( #16580 ) ( #17003 )
...
Cleanup pubkeys when an unrooted, unfrozen bank is dropped. This is a
continuation of PR #16911 .
2021-05-05 15:02:02 -05:00
Jeff Washington (jwash)
ab7c96aa81
insert accounts in parallel when building initial index ( #17040 )
...
* insert accounts in parallel when building initial index
* rename nits from pr review
* rename nits from pr review
* rename nits from pr review
* rename nits from pr review
2021-05-05 17:08:45 +00:00
Jeff Washington (jwash)
541aa5ad85
tests: lamports -> lamports() ( #16982 )
2021-05-03 10:45:54 -05:00
Jeff Washington (jwash)
1fe131a5bb
Readable account trait for stored account meta ( #16690 )
...
* ReadableAccount for StoredAccountMeta
* add test
2021-04-30 16:25:02 -05:00
Jeff Washington (jwash)
763c04adf3
lamports = -> .set_lamports() ( #16980 )
2021-04-30 21:17:05 +00:00
Brooks Prumo
17e6bd579f
Clean unrooted dropped banks ( #16580 ) ( #16911 )
...
In a scenario where a bank is unrooted and dropped, any keys that exist
_only_ in that bank are now cleaned up.
This work was originally based on PR #15106 .
2021-04-30 15:34:38 -05:00
Jeff Washington (jwash)
0af84bb91e
lamports= -> .set_lamports() ( #16978 )
2021-04-30 18:20:54 +00:00
Jeff Washington (jwash)
ca7b36ad8f
lamports -> lamports() ( #16920 )
2021-04-29 10:44:46 -05:00
Jeff Washington (jwash)
f533d3be77
Write account path impl ReadableAccount ( #16779 )
2021-04-28 15:29:22 -05:00