Commit Graph

292 Commits

Author SHA1 Message Date
Jeff Washington (jwash) c20b27bc8f
shink all in parallel on startup (#17308) 2021-05-19 12:15:24 -05:00
Jeff Washington (jwash) ed9cbd50f0
move Ancestors to its own module (#17316) 2021-05-19 11:50:34 -05:00
Tao Zhu 0781fe1b4f
Upgrade Rust to 1.52.0 (#17096)
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script

* upgrade to Rust 1.52.1

* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
2021-05-19 09:31:47 -05:00
Jeff Washington (jwash) 7fe24c455c
simplify generate_index (#17292) 2021-05-19 08:11:41 -05:00
Jeff Washington (jwash) 2175fc0625
calculate account infos before lock (#17245) 2021-05-18 08:06:19 -05:00
Jeff Washington (jwash) 3a56a56d69
StoredMetaWriteVersion (#17293) 2021-05-17 23:58:36 +00:00
Jeff Washington (jwash) d461aba6a4
don't try to build secondary indexes if we don't need them (#17246) 2021-05-17 17:21:15 -05:00
Jeff Washington (jwash) d1db5448b9
hold lock to speed up insert (#17194)
* hold lock to speed up insert

* add tests
2021-05-17 11:58:33 -05:00
Jeff Washington (jwash) fd88db7339
ancestors::new -> default (#17195) 2021-05-14 15:20:59 -05:00
Jeff Washington (jwash) c6f7867aa1
log mismatch (#17209) 2021-05-14 09:50:11 -05:00
Jeff Washington (jwash) 3dbc7744ab
metrics for generating index time (#17192)
* metrics for generating index time

* update metrics to include scan time
2021-05-13 14:32:19 -05:00
carllin 239ab8799c
Remove bloat from secondary indexes (#17048) 2021-05-12 15:29:30 -07:00
Jeff Washington (jwash) 7d96f78821
include/exclude keys on account secondary index (#17110)
* AccountSecondaryIndexes.include/exclude

* use normal scan if key is not indexed

* add a test to ask for a scan for an excluded secondary index

* fix up cli args
2021-05-11 22:06:22 +00:00
Jeff Washington (jwash) f39dda00e0
type AccountSecondaryIndexes = HashSet (#17108) 2021-05-10 14:22:48 +00:00
Jeff Washington (jwash) 225ec00ec4
add comment (#17059) 2021-05-06 15:17:41 -05:00
steviez edc2ab3e48
Test account index and store alignment (#17038)
* Use ReclaimResult::Default() instead of building subtypes

* Add test to ensure account_db store and index are aligned
2021-05-06 14:13:44 -05:00
Brooks Prumo e6f49a3e79
Clean unrooted unfrozen banks (#16580) (#17003)
Cleanup pubkeys when an unrooted, unfrozen bank is dropped.  This is a
continuation of PR #16911.
2021-05-05 15:02:02 -05:00
Jeff Washington (jwash) ab7c96aa81
insert accounts in parallel when building initial index (#17040)
* insert accounts in parallel when building initial index

* rename nits from pr review

* rename nits from pr review

* rename nits from pr review

* rename nits from pr review
2021-05-05 17:08:45 +00:00
Jeff Washington (jwash) 541aa5ad85
tests: lamports -> lamports() (#16982) 2021-05-03 10:45:54 -05:00
Jeff Washington (jwash) 1fe131a5bb
Readable account trait for stored account meta (#16690)
* ReadableAccount for StoredAccountMeta

* add test
2021-04-30 16:25:02 -05:00
Jeff Washington (jwash) 763c04adf3
lamports = -> .set_lamports() (#16980) 2021-04-30 21:17:05 +00:00
Brooks Prumo 17e6bd579f
Clean unrooted dropped banks (#16580) (#16911)
In a scenario where a bank is unrooted and dropped, any keys that exist
_only_ in that bank are now cleaned up.

This work was originally based on PR #15106.
2021-04-30 15:34:38 -05:00
Jeff Washington (jwash) 0af84bb91e
lamports= -> .set_lamports() (#16978) 2021-04-30 18:20:54 +00:00
Jeff Washington (jwash) ca7b36ad8f
lamports -> lamports() (#16920) 2021-04-29 10:44:46 -05:00
Jeff Washington (jwash) f533d3be77
Write account path impl ReadableAccount (#16779) 2021-04-28 15:29:22 -05:00
Jeff Washington (jwash) 9070191b8b
stats on clean for removing roots (#16849)
* stats on clean for removing roots

* rename

* accumulate and swap metrics
2021-04-28 13:24:01 -05:00
Brooks Prumo 77f1ffd84b
Collect uncleaned pubkeys from all slots (#16786)
While working on another issue (#16580), the list of uncleaned pubkeys
returned from `remove_uncleaned_slots_and_collect_pubkeys_up_to_slot()`
did not include unrooted slots.  This meant that during cleaning,
unrooted slots would not have their pubkeys cleaned up properly.

Now, return all uncleaned pubkeys, regardless if the slot is rooted or
not.  Additionally, update the tests to have unrooted slots to ensure
this behavior.

This is part two of PR #16879, and originally based on PR #15106.
2021-04-28 10:57:42 -05:00
Jeff Washington (jwash) a7a671b3aa
AccountSharedData.set_executable() (#16881) 2021-04-28 14:07:43 +00:00
Jeff Washington (jwash) 6381ee38eb
reclaims unref accounts from index (#16838) 2021-04-28 08:50:38 -05:00
Jeff Washington (jwash) 1864bc2080
write Option<AccountSharedData> (#16874)
* write Option<&AccountSharedData>

* add comment
2021-04-28 08:47:26 -05:00
Brooks Prumo 1eaff394da
Refactor `collect_uncleaned_pubkeys_to_slot()` (#16879)
* Refactor `collect_uncleaned_pubkeys_to_slot()`

While working on another issue (#16580), I came across
`collect_unclean_pubkeys_to_slot()` and had difficulty understanding it.
Since the function does a few logically separate things, I split the
function up.  I also added documentation, removed an unused return value,
and renamed the functions to be more specific.

This commit is to split up an existing PR (#16786), where I had both this
aesthetic change _and_ a behavioral change.
2021-04-28 08:16:12 -05:00
Jeff Washington (jwash) 1bd623cd15
private AccountSharedData.rent_epoch (#16844) 2021-04-27 13:51:13 -05:00
Jeff Washington (jwash) 3fdbaefaa6
tests: lamports -= to checked_sub (#16843) 2021-04-27 09:12:48 -05:00
Jeff Washington (jwash) 998cba74b5
AccountSharedData.executable() (#16835) 2021-04-27 09:12:17 -05:00
Jeff Washington (jwash) 81402ee7f5
pass &Pubkey through account storage, slot clean code to reduce copies (#16778)
* &Pubkey

* use trait to pass &Hash or Hash

* use impl Borrow<Hash> instead of trait

* remove old code line commented out
2021-04-27 09:10:06 -05:00
Jeff Washington (jwash) f2ab0384e4
owner -> owner() (#16783) 2021-04-26 17:06:40 +00:00
Michael Vines d9dcd28d82 Ignore racy test_load_account_and_purge_race_without_retry 2021-04-23 23:17:56 +00:00
Jeff Washington (jwash) 48c07d32f0
WritableAccount.add/subtract_lamports (#16750)
* add/sub lamports

* make add/sub return Result

* sample replacements

* cleanup

* fix up a few tests as examples

* move enum, cleanup, impl from

* fmt

* cleanup

* add lamports.rs
2021-04-23 20:20:48 +00:00
Jeff Washington (jwash) dcf2d84b24
refactor complicated 'len' (#16777) 2021-04-23 18:36:49 +00:00
Jeff Washington (jwash) 04e6aebf35
.owner = X -> .set_owner(X) (#16759) 2021-04-23 18:26:33 +00:00
Jeff Washington (jwash) fb0b76c1f3
account.owner = X -> account.set_owner(X) (#16754) 2021-04-23 12:34:22 -05:00
Jeff Washington (jwash) 1cc9a1c0eb
log roots range metric (#16636)
* log roots range metric

* rename
2021-04-23 16:09:39 +00:00
Jeff Washington (jwash) 96d21335e1
.owner -> .owner() (#16758) 2021-04-23 09:35:55 -05:00
Jeff Washington (jwash) 5591cd6ef3
.owner -> .owner() (#16757)
* .owner -> .get_owner()

* use
2021-04-23 09:35:09 -05:00
Jeff Washington (jwash) 91be2903da
Reduce account index lookups during clean (#16689)
* reduce account index lookups in clean

* rename

* rename enum

* hold locks during removal from zero pubkey list

* merge with zero lamport fix

* tests
2021-04-23 09:33:14 -05:00
Jeff Washington (jwash) 333998d008
.lamports = <number> -> .set_lamports(<number>) (#16746) 2021-04-22 18:56:47 +00:00
Jeff Washington (jwash) 8a6b80095e
Set lamports (#16747)
* lamports = -> set_lamports()

* .lamports = X -> .set_lamports(X)
2021-04-22 13:53:06 -05:00
Jeff Washington (jwash) 8d9d6b62d9
use cheaper account get (#16682) 2021-04-22 12:54:08 -05:00
Jeff Washington (jwash) 123e0bdba7
don't iterate slot_list if zero-lamport (#16678) 2021-04-21 20:38:48 -05:00
Jeff Washington (jwash) 69cbad0869
some ReadableAccount changes (#16688)
* some ReadableAccount changes

* deref
2021-04-21 12:20:37 -05:00