Commit Graph

3052 Commits

Author SHA1 Message Date
Ashwin Sekar f207af765e
Split out voting and banking threads in banking stage (#27931)
* Split out voting and banking threads in banking stage

Additionally this allows us to aggressively prune the buffer for voting threads
as with the new vote state only the latest vote from each validator is
necessary.

* Update local cluster test to use new Vote ix

* Encapsulate transaction storage filtering better

* Address pr comments

* Commit cargo lock change

* clippy

* Remove unsafe impls

* pr comments

* compute_sanitized_transaction -> build_sanitized_transaction

* &Arc -> Arc

* Move test

* Refactor metrics enums

* clippy
2022-10-20 21:10:48 +00:00
Trent Nelson cb7fe87abb bank: factor accumulation out to helper on `TotalAccountsStats` 2022-10-20 10:50:51 -07:00
Jeff Washington (jwash) 09ede8b85c
reuse update_shrink_stats for combine_ancient_slots (#28466) 2022-10-19 14:17:34 -07:00
Alexander Meißner 5aca7df232
Bank::create_executor() (#28474)
* Moves the integration test test_bpf_loader_upgradeable_deploy_with_max_len() from the bpf_loader into the runtime.

* Adds Bank::create_executor().

* Adds a test for Bank::create_executor().
2022-10-19 22:28:57 +02:00
Brooks Prumo 70e83e82c6
Waits to handle snapshot requests until after startup verification is complete (#28481) 2022-10-19 13:39:47 -04:00
Jeff Washington (jwash) 2b11926c9e
simplify combine_ancient_slots (#28464) 2022-10-19 09:35:21 -07:00
Jeff Washington (jwash) 8b115ff489
factor out update_shrink_stats (#28465)
* use local for shrink_stats

* renames and remove .as_us()

* factor out update_shrink_stats

* use integer to report us in metrics
2022-10-19 08:08:04 -07:00
Jeff Washington (jwash) 040035063f
share shrink_collect code (#28454)
* share shrink_collect code

* use correct stats
2022-10-19 08:07:05 -07:00
Brooks Prumo 1cc9cf927c
Supports warping with Epoch Accounts Hash (#28459) 2022-10-19 10:37:14 -04:00
Jeff Washington (jwash) ca115594ff
remove concept of 'full' ancient append vec (#28449) 2022-10-18 15:08:04 -07:00
Jeff Washington (jwash) c89bd0d6a4
make ancient shrink consistent with shrink (#28457) 2022-10-18 15:05:32 -07:00
Jeff Washington (jwash) 825b83c95c
simple minor refactoring (#28456) 2022-10-18 15:05:06 -07:00
Jeff Washington (jwash) 3835ac166e
refactor load_accounts_index_for_shrink for easier sharing (#28455) 2022-10-18 14:55:10 -07:00
Jeff Washington (jwash) d52e28ac22
combine sort into get_unique_accounts_from_storages (#28450) 2022-10-18 11:52:48 -07:00
Jeff Washington (jwash) b583ec79e1
add tests for should_move_to_ancient_append_vec (#28438)
add tests for should_moev_to_ancient_append_vec
2022-10-18 11:41:35 -07:00
Jeff Washington (jwash) fd2e671861
remove slot# from account hash (#28405)
* remove slot# from account hash

* add feature

* fix tests

* constants to help clarify 'irrelevant' changes

* move to enum for enforcing irrelevancy

* ignore unsupported tests
2022-10-18 08:03:37 -07:00
Jeff Washington (jwash) 682999a423
accounts hash calculation uses files instead of memory (#28065) 2022-10-18 07:51:38 -07:00
Jeff Washington (jwash) b669b827fb
refactor and test get_sorted_potential_ancient_slots (#28436) 2022-10-18 07:45:43 -07:00
Justin Starry 70445b7402
Refactor: Rename `max_invoke_depth` to `max_invoke_stack_height` (#28427)
Refactor: Rename max_invoke_depth to max_invoke_stack_height
2022-10-17 23:54:56 +08:00
Jeff Washington (jwash) 28a89a1d99
remove expected rent collection and rehashing completely (#28422) 2022-10-17 07:24:42 -07:00
Jeff Washington (jwash) f896513864
remove unnecessary clones (#28406) 2022-10-16 20:33:07 -07:00
Jeff Washington (jwash) 0807105c5a
refactoring of accounts hash to prepare for using files always (#28379) 2022-10-14 12:29:43 -07:00
Brooks Prumo 31c2b29941
Sends both an EAH and a snapshot request from `set_root()` (#28363) 2022-10-14 11:00:04 -04:00
Brooks Prumo dd7fee8f32
Re-enqueues unhandled ABS requests (#28362) 2022-10-13 16:25:39 -04:00
Brooks Prumo 9cbd00fdbc
Converts PendingAccountsPackage to a channel (#28352) 2022-10-13 12:47:36 -04:00
Jeff Washington (jwash) 72d6927fa1
factor out hashing of storages for caching (#28261) 2022-10-13 08:53:54 -07:00
Jeff Washington (jwash) 53a1a14868
eliminate worthless work in store_accounts_to (#28365) 2022-10-12 13:30:20 -07:00
Alexander Meißner 664339e239
Refactor - executor_cache (#28332)
* Renames CachedExecutors => BankExecutorCache.

* Renames TransactionExecutor => TransactionExecutorCacheEntry.

* Renames TransactionExecutorStatus => TxBankExecutorCacheDiff.

* Adds TransactionExecutorCache.

* Renames the items of TxBankExecutorCacheDiff.
2022-10-12 18:09:03 +02:00
steviez db9e32d71d
Revert "log adding root every 10s (#28280)" (#28334)
This reverts commit 16853acf35.

We get similar coverage from the slot field of bank-forks_set_root;
additionally, we can see banks with bank-new_from_parent-heights.
2022-10-11 12:49:49 -05:00
Brooks Prumo 020cc090d5
Refactors ABS snapshot request handling (#28326) 2022-10-11 13:17:06 -04:00
Ian Macalinao cdf77bb824
runtime: fix typo 'transaction' (#27948) 2022-10-11 07:47:10 -07:00
Jeff Washington (jwash) 7a120b8b62
accounts_db::load returns None for zero lamport accounts (#28311) 2022-10-11 07:43:03 -07:00
Brooks Prumo d500391006
Removes redundant accounts_package_type (#28325) 2022-10-10 17:11:38 +00:00
Alexander Meißner 2fc8e533a2
Refactor - Move `executor_cache` to program-runtime crate (#28322)
* Moves CachedExecutors, related structs, consts and tests into the program-runtime crate.

* Moves TransactionExecutor, related enum and type defs into executor_cache mod.
2022-10-10 15:01:41 +02:00
Jeff Washington (jwash) 16853acf35
log adding root every 10s (#28280) 2022-10-07 09:19:34 -07:00
Jeff Washington (jwash) a1fe8dd444
add info for load race condition asserts (#28277) 2022-10-07 08:55:01 -07:00
Tao Zhu 0324573667
report additional transaction errors to metrics (#28285) 2022-10-07 10:36:22 -05:00
Jeff Washington (jwash) a400178744
hash calc chunking cleanup (#28274) 2022-10-06 23:23:26 -07:00
Jeff Washington (jwash) 76c30e7bd0
move accounts_db::load_without_fixed_root to test mod (#28271) 2022-10-06 17:11:08 -07:00
Jeff Washington (jwash) 66cd290d04
add chunk tracking in SplitAncientStorages (#28202) 2022-10-06 11:19:53 -07:00
Jeff Washington (jwash) bb9a19ac86
minor refactoring of accounts hash cache (#28260) 2022-10-06 10:50:16 -07:00
Jeff Washington (jwash) 435d4aded9
run initial hash calc in background, using background threads (#28239) 2022-10-05 19:42:06 -07:00
Brooks Prumo eec2a2a301
Move epoch accounts hash utility functions (#28219) 2022-10-05 23:29:22 +00:00
Jeff Washington (jwash) 456d4ae93a
accounts hash calc, add get_ancient_slots (#28240) 2022-10-05 15:32:39 -07:00
Jeff Washington (jwash) ad0742c7cf
exit early deep in bank freeze, avoiding some unnecessary work (#28238)
exit early on bank freeze
2022-10-05 14:49:33 -07:00
Brooks Prumo a8c6a9e5fc
Bank::freeze() waits for EAH calculation to complete (#28170) 2022-10-05 17:44:35 -04:00
Jeff Washington (jwash) 3f5eec32cd
refactor SplitAncientStorages::new for testing (#28235) 2022-10-05 12:56:29 -07:00
Jeff Washington (jwash) f464c69764
refactor into SplitAncientStorages::new (#28228) 2022-10-05 08:46:53 -07:00
Jeff Washington (jwash) 3339570a70
accounts hash scan refactoring (#28220) 2022-10-04 10:19:50 -07:00
Jeff Washington (jwash) fd11e261dd
use ref for snapshot_storages.iter_range (#28205) 2022-10-04 09:11:44 -07:00