Commit Graph

1485 Commits

Author SHA1 Message Date
Jeff Washington (jwash) 44831c18d2
reuse work in is_non_loader_key (#16521) 2021-05-21 10:07:04 -05:00
Jeff Washington (jwash) 8143ee5502
add AccountsIndex AccountMapsReadLock type (#17365) 2021-05-21 10:02:14 -05:00
Jeff Washington (jwash) 2f55547d37
remove flatten in storage scan (#17354)
* remove flatten in storage scan

* fix tests
2021-05-21 09:26:46 -05:00
Michael Vines 2c99b23ad7 Add get_sysvar() helper to sdk 2021-05-21 01:30:12 +00:00
Jeff Washington (jwash) 45552d271a
take mutex once in process_dead_slots (#17357) 2021-05-20 19:50:58 -05:00
Jeff Washington (jwash) 0486df02ba
don't limit to thread pool when cleaning on startup (#17317) 2021-05-20 14:36:35 -05:00
Jeff Washington (jwash) 33ab9c4e8d
batch insert account_index items in generate_index (#17290) 2021-05-20 10:29:13 -05:00
Jeff Washington (jwash) 9b74988fc6
calculate_accounts_hash does not throttle threads on startup (#17338) 2021-05-20 10:25:54 -05:00
Jeff Washington (jwash) a9aa533684
replace Ancestors HashMap for performance (#16477)
* replace Ancestors HashMap for performance

* add ancestors module
2021-05-20 10:11:56 -05:00
Jeff Washington (jwash) a544010b03
don't log shrink metrics on first call (#17328)
* don't log shrink metrics on first call

* simplify logic
2021-05-20 00:15:35 +00:00
Jeff Washington (jwash) 32ec8341f9
generate_index inserts ideal initial data (#17247)
* improve insert into map initially

* rework towards single code path

* rename

* update test
2021-05-19 16:21:24 -05:00
Jeff Washington (jwash) b5302e7619
add calc_stored_meta_us metric (#17318) 2021-05-19 12:29:41 -05:00
Jeff Washington (jwash) c20b27bc8f
shink all in parallel on startup (#17308) 2021-05-19 12:15:24 -05:00
Jeff Washington (jwash) ed9cbd50f0
move Ancestors to its own module (#17316) 2021-05-19 11:50:34 -05:00
Tao Zhu 0781fe1b4f
Upgrade Rust to 1.52.0 (#17096)
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script

* upgrade to Rust 1.52.1

* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
2021-05-19 09:31:47 -05:00
Jeff Washington (jwash) 4788976517
rework slot list update (#17232) 2021-05-19 08:48:46 -05:00
Jeff Washington (jwash) 7fe24c455c
simplify generate_index (#17292) 2021-05-19 08:11:41 -05:00
Tyera Eulberg 827355a6b1
Create solana-rpc crate and move subscriptions (#17320)
* Move non_circulating_supply to runtime

* Add solana-rpc crate and move max_slots

* Move subscriptions to solana-rpc

* Single use statements
2021-05-19 00:54:28 -06:00
Jeff Washington (jwash) a3c0833a1c
tests: cleanup test (#17305)
* tests: cleanup test

* 2 more replacements
2021-05-18 18:08:37 -05:00
Jeff Washington (jwash) 75335b4f58
datapoint for verify_snapshot_bank (#17306) 2021-05-18 12:14:43 -05:00
Jeff Washington (jwash) 2175fc0625
calculate account infos before lock (#17245) 2021-05-18 08:06:19 -05:00
Jeff Washington (jwash) 3a56a56d69
StoredMetaWriteVersion (#17293) 2021-05-17 23:58:36 +00:00
Ryo Onodera bcbe155575
Minor test cleanup and comments (#17283) 2021-05-18 07:22:43 +09:00
Jeff Washington (jwash) d461aba6a4
don't try to build secondary indexes if we don't need them (#17246) 2021-05-17 17:21:15 -05:00
Jeff Washington (jwash) 9d6837c904
fix comment (#17286) 2021-05-17 14:06:25 -05:00
Jeff Washington (jwash) d1db5448b9
hold lock to speed up insert (#17194)
* hold lock to speed up insert

* add tests
2021-05-17 11:58:33 -05:00
Jeff Washington (jwash) 3745e0babc
minor refactoring of AccountsIndex in preparation of bigger things (#17237) 2021-05-14 15:27:10 -05:00
Jeff Washington (jwash) fd88db7339
ancestors::new -> default (#17195) 2021-05-14 15:20:59 -05:00
Jeff Washington (jwash) c6f7867aa1
log mismatch (#17209) 2021-05-14 09:50:11 -05:00
Brooks Prumo 76dfacca61
Add documentation for BankForks::prune_non_rooted() (#17213) 2021-05-14 07:47:48 -05:00
Tyera Eulberg 27004f1b76
Return error for excluded secondary-index keys (#17193)
* Add runtime helpers to check secondary indexes for key

* Add custom rpc error

* Check secondary-index key inclusion in rpc

* Clone complete AccountSecondaryIndexes into rpc to avoid bank query
2021-05-13 21:04:21 +00:00
Jeff Washington (jwash) 3dbc7744ab
metrics for generating index time (#17192)
* metrics for generating index time

* update metrics to include scan time
2021-05-13 14:32:19 -05:00
carllin 239ab8799c
Remove bloat from secondary indexes (#17048) 2021-05-12 15:29:30 -07:00
Lijun Wang 9c42a89a43
Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
Jeff Washington (jwash) 7d96f78821
include/exclude keys on account secondary index (#17110)
* AccountSecondaryIndexes.include/exclude

* use normal scan if key is not indexed

* add a test to ask for a scan for an excluded secondary index

* fix up cli args
2021-05-11 22:06:22 +00:00
Jeff Washington (jwash) f39dda00e0
type AccountSecondaryIndexes = HashSet (#17108) 2021-05-10 14:22:48 +00:00
Jeff Washington (jwash) a6a1355b80
fix secondary index test (#17111) 2021-05-10 08:54:30 -05:00
Ryo Onodera c616c34825
Add comment for another unspoken eager collection subtlety (#17137) 2021-05-10 22:44:46 +09:00
Jeff Washington (jwash) 225ec00ec4
add comment (#17059) 2021-05-06 15:17:41 -05:00
Jeff Washington (jwash) dc0429f5e6
add metric for assumption (#17061) 2021-05-06 15:04:13 -05:00
steviez edc2ab3e48
Test account index and store alignment (#17038)
* Use ReclaimResult::Default() instead of building subtypes

* Add test to ensure account_db store and index are aligned
2021-05-06 14:13:44 -05:00
Ryo Onodera d19526e6c2
Dump rent_collector/inflation with ledger-tool cap (#17069) 2021-05-06 19:29:46 +09:00
Jeff Washington (jwash) ffbe8906ed
get root lock once (#16829) 2021-05-05 15:17:45 -05:00
Brooks Prumo e6f49a3e79
Clean unrooted unfrozen banks (#16580) (#17003)
Cleanup pubkeys when an unrooted, unfrozen bank is dropped.  This is a
continuation of PR #16911.
2021-05-05 15:02:02 -05:00
Jeff Washington (jwash) ab7c96aa81
insert accounts in parallel when building initial index (#17040)
* insert accounts in parallel when building initial index

* rename nits from pr review

* rename nits from pr review

* rename nits from pr review

* rename nits from pr review
2021-05-05 17:08:45 +00:00
Stephen Akridge 3e0fed48e7 Don't recognize temp snapshots as possible snapshots to open 2021-05-05 08:45:03 -07:00
Jeff Washington (jwash) 5786be13a4
flatten_hash_intermediate sets capacity first (#17013)
* flatten_hash_intermediate sets capacity first

* use iterator instead of for
2021-05-05 09:07:05 -05:00
sakridge d94e4ef7c9
More docs for bank.rs (#17006) 2021-05-03 17:37:18 +00:00
Jeff Washington (jwash) 541aa5ad85
tests: lamports -> lamports() (#16982) 2021-05-03 10:45:54 -05:00
Jeff Washington (jwash) 744ac1535f
tests: lamports -> ReadableAccount (#16983) 2021-05-03 10:45:15 -05:00