Commit Graph

1806 Commits

Author SHA1 Message Date
Jeff Washington (jwash) 8db9586599
AcctIdx: randomly evict items from in mem (#20319) 2021-09-30 02:51:54 +00:00
Jeff Washington (jwash) 5e05f12c48
AcctIdx: bucket perf improvements (#20328) 2021-09-30 01:53:26 +00:00
Jeff Washington (jwash) 59112adac1
AcctIdx: us stats were showing ms (#20324) 2021-09-29 22:18:07 +00:00
Jeff Washington (jwash) 2ec74474e9
AcctIdx: optimize should_remove_from_mem (#20327) 2021-09-29 16:41:54 -05:00
Alexander Meißner 4de5fff3ca
Simplifies return_data accessors in InvokeContext. (#20290) 2021-09-29 19:11:06 +02:00
Alexander Meißner 57c8abf499
Refactor: Merge message processor into invoke context (#20308)
* Inlines MessageProcessor::execute_instruction() in MessageProcessor::process_message().

* Moves MessageProcessor::create_pre_accounts() into ThisInvokeContext::push().

* Move instruction_recorders slice into InvokeContext.

* Makes account_indices optional in InvokeContext::push().

* Separates initial InvokeContext::push() from ThisInvokeContext::new().

* invoke_context.pop() the base invocation frame.

* Zip message.instructions.iter() and program_indices.iter().

* Moves ThisInvokeContext::new() to the beginning of the loop inside MessageProcessor::process_message().

* Hoists ThisInvokeContext::new() out of loop inside MessageProcessor::process_message().

* Removes unnecessary clone() from ThisInvokeContext::new() in MessageProcessor ::process_message().

* Stop ignoring errors from MessageProcessor::create_pre_accounts().

* Moves MessageProcessor::verify_account_references() and MessageProcessor::verify() into InvokeContext::verify().
2021-09-29 12:05:25 -05:00
Jeff Washington (jwash) 062d8e6c5a
AcctIdx: fix logic on should_remove_from_mem (#20302) 2021-09-29 13:07:40 +00:00
Jeff Washington (jwash) bd9b2f6f39
AcctIdx: throttling bg flushing to spread out work over interval (#20176)
* AcctIdx: throttling bg flushing to spread out work over interval

* prevent hang
2021-09-29 07:42:54 -05:00
Jack May 8fee9a2e1a
Dont call precompiled programs (#19930) 2021-09-29 06:25:08 +00:00
Jack May 491877de3d
Revert "Refactor: Merge MessageProcessor into InvokeContext (#20165)" (#20301)
This reverts commit df6905c3a6.
2021-09-28 16:59:01 -07:00
Jeff Washington (jwash) fa5b091b4c
prior to panicing with cap mismatch, try other calculation (#20292) 2021-09-28 16:09:36 -05:00
Jeff Washington (jwash) 320beb76f6
AcctIdx: implement wait_for_idle (#20256) 2021-09-28 11:07:19 -05:00
Tyera Eulberg 2cd9dc99b6
Restore ability for programs to upgrade themselves (#20265)
* Make helper associated fn

* Add feature definition

* Add handling to preserve program-id write lock when upgradeable loader is present; restore bpf upgrade-self test

* Use single feature
2021-09-28 09:59:08 -06:00
Jack May 4e27543415
Allow programs to realloc their accounts within limits (#19475) 2021-09-28 01:13:03 -07:00
Jeff Washington (jwash) e50a26e6cf
AcctIdx: add age_interval_ms() to remove constant use (#20264) 2021-09-28 01:22:07 +00:00
Jeff Washington (jwash) af309c126c
AcctIdx: refactor get to use more refs (#20268) 2021-09-28 00:18:36 +00:00
Jeff Washington (jwash) f6812523cd
handle race condition with flush and delete (#20255) 2021-09-27 16:55:15 -05:00
Jeff Washington (jwash) bbd2f9672d
clone -> Arc::clone to make cost clear (#20262) 2021-09-27 16:49:53 -05:00
Jeff Washington (jwash) 4f83818fdb
get rid of WriteAccountMapEntry instance (#19093) 2021-09-27 16:24:19 -05:00
Jeff Washington (jwash) e4795ae7ed
AcctIdx: keep cached or multiple slot lists in cache (#20257) 2021-09-27 16:09:26 -05:00
Jeff Washington (jwash) 9d91a6f1b7
shrink slots at startup uses more threads (#20251) 2021-09-27 14:14:55 -05:00
Jeff Washington (jwash) 8266864adc
AcctIdx: maybe advance age when a bg thread wakes up (#20250)
* fix math error with stats calc

* AcctIdx: maybe report stats when a bg thread wakes up
2021-09-27 18:39:41 +00:00
Brooks Prumo 95fbc29500
Add stats when getting snapshot storages (#20234) 2021-09-27 13:07:00 -05:00
Jeff Washington (jwash) 46263d405e
fix math error with stats calc (#20253) 2021-09-27 11:55:19 -05:00
Jeff Washington (jwash) bc50fd4ad6
allocate threads for accounts idx based on cpus (#20118) 2021-09-27 11:25:06 -05:00
Jeff Washington (jwash) 025261c05e
AcctIdx: when no disk index, always wait for stats (#20248) 2021-09-27 11:24:36 -05:00
Jeff Washington (jwash) 833f7761d2
AcctIdx: upsert uses get from mem first (#20186) 2021-09-27 11:23:53 -05:00
Jeff Washington (jwash) d8f98b3dcb
AcctIdx: avoid crazy #s on ms_per_age stat (#20184) 2021-09-27 08:25:23 -05:00
Alexander Meißner df6905c3a6
Refactor: Merge MessageProcessor into InvokeContext (#20165)
* Inlines MessageProcessor::execute_instruction() in MessageProcessor::process_message().

* Moves MessageProcessor::create_pre_accounts() into ThisInvokeContext::push().

* Hoists ThisInvokeContext::new() out of loop inside MessageProcessor::process_message().

* Moves MessageProcessor::verify_account_references() and MessageProcessor::verify() into InvokeContext::verify().
2021-09-27 08:28:45 +02:00
Jeff Washington (jwash) c6d189dac2
bg threads wait until next expected event (#20126) 2021-09-24 16:00:41 -05:00
Jeff Washington (jwash) 122206dbb1
AcctIdx: bg loop sleeps when all buckets flushed at current age (#20168) 2021-09-24 13:19:06 -05:00
Jeff Washington (jwash) 1d13594c1c
AcctIdx: store # threads in BucketHolder. Used later. (#20151) 2021-09-23 19:56:44 -05:00
Jeff Washington (jwash) b3bb079d9f
AcctIdx: condense upsert to always use read lock (#20148) 2021-09-23 19:19:27 -05:00
Brooks Prumo 1347b507da
Handle duplicate AppendVec IDs (#20096)
When reconstructing the AccountsDb, if the storages came from full and
incremental snapshots generated on different nodes, it's possible that
the AppendVec IDs could overlap/have duplicates, which would cause the
reconstruction to fail.

This commit handles this issue by unconditionally remapping the
AppendVec ID for every AppendVec.

Fixes #17088
2021-09-23 17:34:32 -05:00
Jeff Washington (jwash) 2ae1e80013
AcctIdx: items needs to load from disk (#20138) 2021-09-23 17:33:54 -05:00
Jeff Washington (jwash) 334d38d200
AcctIdx: generation per thread starts at a random bucket (#20122) 2021-09-23 19:59:37 +00:00
Jeff Washington (jwash) 254d9c8903
range takes a ref (#20140) 2021-09-23 13:57:56 -05:00
Jeff Washington (jwash) 093dd68214
AcctIdx: stats for how much time bg threads wait (#20127) 2021-09-23 18:37:14 +00:00
Jeff Washington (jwash) fe28f17718
AcctIdx: startup causes us to flush remove everything asap (#20121) 2021-09-23 12:17:52 -05:00
Jeff Washington (jwash) 5dae615208
AcctIdx: set age of cache accesses to future (#20125) 2021-09-23 11:43:48 -05:00
Jeff Washington (jwash) fb55e9a9b6
AcctIdx: when stop_flush is disabled, prepare to age (#20113) 2021-09-23 11:42:32 -05:00
Jeff Washington (jwash) 5bbb0da7b8
AcctIdx: stats for in-mem (#20114) 2021-09-23 08:14:53 -05:00
Jeff Washington (jwash) cb9d93525b
AcctIdx: metrics for loading from disk (#20124) 2021-09-23 08:14:08 -05:00
Jeff Washington (jwash) 4d3e32803f
AcctIdx: PreAllocatedAccountMapEntry (#20117) 2021-09-23 08:11:32 -05:00
Trent Nelson 7b365c564f
runtime: remove inactive delegation from stakes cache 2021-09-22 20:50:58 -06:00
Jeff Washington (jwash) 476124de50
AcctIdx: report disk bucket stats (#20120) 2021-09-22 18:17:20 -05:00
Jeff Washington (jwash) f7eebadbed
AcctIdx: remove old items from cache (#20076) 2021-09-22 21:37:15 +00:00
Jeff Washington (jwash) 0eb0d7f73b
AcctIdx: consolidate lock_and_update_slot_list (#20090) 2021-09-22 18:40:19 +00:00
Jeff Washington (jwash) 4fb77183ef
AcctIdx: move background() to bucket holder (#20085) 2021-09-22 17:40:30 +00:00
Jeff Washington (jwash) 681e8728a1
get rid of logs left in test (#20116) 2021-09-22 12:39:48 -05:00