Commit Graph

511 Commits

Author SHA1 Message Date
Jeff Washington (jwash) dd69f3baf5
throttle index adding to allow disk flushing to keep up and reduce startup ram usage (#23773) 2022-03-20 19:56:20 -05:00
Jeff Washington (jwash) a419374fa4
factor out function (#23742) 2022-03-18 14:10:52 -05:00
Jeff Washington (jwash) b5a99b9b09
avoid data copies in writing to cache (#23674) 2022-03-15 16:42:26 -05:00
Tyera Eulberg 102dd68a03
Rename AccountsDb plugins to Geyser plugins (#23604) 2022-03-14 19:18:46 -06:00
Jeff Washington (jwash) a3242b6b86
remove unnecessary namespaces (#23646) 2022-03-14 12:28:55 -05:00
Jeff Washington (jwash) e8a8f4e9e2
fix filler acct slots in epoch (#23536) 2022-03-14 09:57:44 -05:00
HaoranYi 463cd564cf
fix typos (#23495)
* fix typos
2022-03-05 20:46:46 -06:00
Jeff Washington (jwash) ddfd4f86f3
exit early from acct bg svc if no shrink candidates (#23459) 2022-03-03 08:47:39 -06:00
Jeff Washington (jwash) a99fd09c16
allow index update to change storage slot # (#23311) 2022-03-03 08:40:48 -06:00
Jeff Washington (jwash) d909b7c80b
log hash and lamport result of calculate_accounts_hash_without_index (#23425) 2022-03-02 11:10:40 -06:00
Jeff Washington (jwash) 7b7fdb42d9
minor test refactoring (#23423) 2022-03-02 08:31:14 -06:00
Jeff Washington (jwash) 0ad4757159
plumbing for 'other_slot' in 'update_index' (#23330) 2022-02-25 12:58:08 -06:00
Jeff Washington (jwash) 7ee549e5ae
fix params for store_accounts_frozen (#23312) 2022-02-23 17:40:51 -06:00
Jeff Washington (jwash) cafc18c3f9
update_index uses ReadableAccount to reduce params (#23305) 2022-02-23 17:01:23 -06:00
Jeff Washington (jwash) a245efe83d
trait for acct data with slot per item (#23285) 2022-02-23 15:22:29 -06:00
Michael Vines a102453bae
accounts_index: Add SPL Token account indexing for token-2022 accounts (#23043) 2022-02-16 16:23:25 -08:00
Jeff Washington (jwash) a160fc30f2
make max_roots_inclusive clear (#22942) 2022-02-07 13:26:53 -06:00
Jeff Washington (jwash) 350207b05f
impl ReadableAccount for LoadedAccount (#22793) 2022-01-27 16:59:30 -06:00
Jeff Washington (jwash) 18d69c8d2b
SortedStoragesIter to remove random access requirement (#22713) 2022-01-27 10:25:13 -06:00
Jeff Washington (jwash) 8b90084ebc
fix comment typos (#22723) 2022-01-25 11:10:42 -06:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Jeff Washington (jwash) 4abf7a7f88
add ActiveStats to track when long running, expensive bg processes are running (#22608) 2022-01-21 10:51:23 -06:00
Jeff Washington (jwash) 7d7228c356
cleanup assert (#22495) 2022-01-14 11:53:24 -06:00
Justin Starry cddab635ff
Cleanup activated rent_for_sysvars feature (#22454) 2022-01-14 20:34:09 +08:00
Jeff Washington (jwash) d3b95b9226
Cleanup AccountStorage struct (#22463)
* Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)"

This reverts commit 8aa3d690b5.

* Cleanup AccountStorage struct
2022-01-13 17:51:53 -06:00
Tyera Eulberg 637e366b18
Prevent rent-paying account creation (#22292)
* Fixup typo

* Add new feature

* Add new TransactionError

* Add framework for checking account state before and after transaction processing

* Fail transactions that leave new rent-paying accounts

* Only check rent-state of writable tx accounts

* Review comments: combine process_result success behavior; log and metrics before feature activation

* Fix tests that assume rent-exempt accounts are okay

* Remove test no longer relevant

* Remove native/sysvar special case

* Move metrics submission to report legacy->legacy rent paying transitions as well
2022-01-11 11:32:25 -07:00
behzad nouri a49ef49f87
optimizes ReadOnlyAccountsCache LRU eviction implementation (#22403)
ReadOnlyAccountsCache is using a background thread, table scan and sort
to implement LRU eviction policy:
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L66-L73
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L186-L191
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L222

DashMap internally locks each shard when accessed; so a table scan in
the background thread can create a lot of lock contention.

This commit adds an index-list queue containing cached keys in the order
that they are accessed. Each hash-map entry also includes its index into
this queue.
When an item is first entered into the cache, it is added to the end of
the queue. Also each time an entry is looked up from the cache it is
moved to the end of queue. As a result, items in the queue are always
sorted in the order that they have last been accessed. When doing LRU
eviction, cache entries are evicted from the front of the queue.
Using index-list, all queue operations above are O(1) with low overhead
and so above achieves an efficient implementation of LRU cache eviction
policy.
2022-01-11 17:25:28 +00:00
Brooks Prumo d90d5ee9b6
Refactor Rent::due() with RentDue enum (#22346) 2022-01-08 09:03:46 -06:00
Jeff Washington (jwash) e0c091a9f4
factor out code in do_shrink_slot_stores (#22306) 2022-01-06 10:49:24 -06:00
Jeff Washington (jwash) 100293c4b5
refactor get_store_for_shrink (#22307) 2022-01-06 10:49:01 -06:00
Brooks Prumo ab13e39518
Only sum accounts data len from non-zero lamport accounts (#22309) 2022-01-05 23:23:37 +00:00
Jeff Washington (jwash) fb62407232
AcctIdx: appendvecid: u32 (#21842) 2022-01-03 10:35:17 -06:00
Jeff Washington (jwash) 729698e815
AcctIdx: items() uses held ranges (#21954) 2021-12-17 09:59:29 -06:00
Jeff Washington (jwash) 5d40da5688
log "next_id" to track append vec ids (#21971) 2021-12-16 22:41:34 -06:00
Jeff Washington (jwash) e11a1911ad
load_accounts_index_for_shrink ignores cached entries (#21951) 2021-12-16 16:37:08 -06:00
Jeff Washington (jwash) 347323cbb2
use bg thread pool for shrink (#21950) 2021-12-16 15:54:38 -06:00
Jeff Washington (jwash) 46e5350d8c
AcctInfo: store offset in AccountInfo as u32 (#21895) 2021-12-15 15:41:11 -06:00
Jeff Washington (jwash) e5be96d8bf
AcctIdx: consolidate next_id calls (#21929) 2021-12-15 15:39:54 -06:00
Jeff Washington (jwash) 41dd31e5f4
AcctIdx: move cached store id to bit (#21892)
* AcctIdx: move cached store id to bit

* add comments/rename
2021-12-15 11:49:24 -06:00
Jeff Washington (jwash) 02fa135815
AcctIdx: create test fn get_test() to isolate changes to AcctIdx::get() (#21909) 2021-12-15 09:09:56 -06:00
Jeff Washington (jwash) f4308bdb64
AcctIdx: StoredSize is u32 (#21894) 2021-12-14 13:50:19 -06:00
Jeff Washington (jwash) ec583bd12d
AcctIdex: use StorageLocation (#21853) 2021-12-14 10:27:17 -06:00
Jeff Washington (jwash) 6fc6673ead
AcctInfo: store_id private and add accessor (#21839) 2021-12-13 21:35:30 -06:00
Jeff Washington (jwash) 98e5ea9dce
AcctIdx: simplify AccountIndexGetResult (#21857) 2021-12-13 21:16:17 -06:00
Jeff Washington (jwash) 50f26ea9c0
AcctInfo: create AcctInfo with cache explicitly (#21847) 2021-12-13 17:35:07 -06:00
Jeff Washington (jwash) 6155ef6377
AcctInfo: make offset private, add accessor (#21838) 2021-12-13 14:43:26 -06:00
Jeff Washington (jwash) c9e0bde407
AcctIdx: AccountInfo::stored_size private (#21821) 2021-12-13 10:10:06 -06:00
Jeff Washington (jwash) 025a5a3b9c
AccountInfo: construct with new() (#21788) 2021-12-12 14:36:05 -06:00
Jeff Washington (jwash) 825f8bcea4
use AppendVecId instead of usize (#21792) 2021-12-11 21:38:13 -06:00
Jeff Washington (jwash) c5b6ea74ef
replace some .lamports checks with trait (#21783) 2021-12-11 21:37:33 -06:00