a422387ca8
cleanup ancient append vec merge stats ( #28556 )
2022-10-24 13:11:12 -07:00
8a538f8da5
skip_rewrite skips almost all accounts with no rent ( #28552 )
2022-10-24 11:48:00 -07:00
1fbd818647
runtime: remove `Default` req on account scan interfaces ( #28533 )
2022-10-21 17:53:06 -06:00
993583d9b3
simplify get_store_for_shrink ( #28524 )
2022-10-21 14:49:34 -07:00
9eafad467c
Add convenience methods to VoteInstruction to distinguish vote types ( #28526 )
...
* Add convenience methods to VoteInstruction to distinguish vote types
* use matches! macro instead
2022-10-21 14:17:40 -06:00
fb94e50678
remove single use function for clarity ( #28523 )
2022-10-21 11:42:27 -07:00
f207af765e
Split out voting and banking threads in banking stage ( #27931 )
...
* Split out voting and banking threads in banking stage
Additionally this allows us to aggressively prune the buffer for voting threads
as with the new vote state only the latest vote from each validator is
necessary.
* Update local cluster test to use new Vote ix
* Encapsulate transaction storage filtering better
* Address pr comments
* Commit cargo lock change
* clippy
* Remove unsafe impls
* pr comments
* compute_sanitized_transaction -> build_sanitized_transaction
* &Arc -> Arc
* Move test
* Refactor metrics enums
* clippy
2022-10-20 21:10:48 +00:00
cb7fe87abb
bank: factor accumulation out to helper on `TotalAccountsStats`
2022-10-20 10:50:51 -07:00
09ede8b85c
reuse update_shrink_stats for combine_ancient_slots ( #28466 )
2022-10-19 14:17:34 -07:00
5aca7df232
Bank::create_executor() ( #28474 )
...
* Moves the integration test test_bpf_loader_upgradeable_deploy_with_max_len() from the bpf_loader into the runtime.
* Adds Bank::create_executor().
* Adds a test for Bank::create_executor().
2022-10-19 22:28:57 +02:00
70e83e82c6
Waits to handle snapshot requests until after startup verification is complete ( #28481 )
2022-10-19 13:39:47 -04:00
2b11926c9e
simplify combine_ancient_slots ( #28464 )
2022-10-19 09:35:21 -07:00
8b115ff489
factor out update_shrink_stats ( #28465 )
...
* use local for shrink_stats
* renames and remove .as_us()
* factor out update_shrink_stats
* use integer to report us in metrics
2022-10-19 08:08:04 -07:00
040035063f
share shrink_collect code ( #28454 )
...
* share shrink_collect code
* use correct stats
2022-10-19 08:07:05 -07:00
1cc9cf927c
Supports warping with Epoch Accounts Hash ( #28459 )
2022-10-19 10:37:14 -04:00
ca115594ff
remove concept of 'full' ancient append vec ( #28449 )
2022-10-18 15:08:04 -07:00
c89bd0d6a4
make ancient shrink consistent with shrink ( #28457 )
2022-10-18 15:05:32 -07:00
825b83c95c
simple minor refactoring ( #28456 )
2022-10-18 15:05:06 -07:00
3835ac166e
refactor load_accounts_index_for_shrink for easier sharing ( #28455 )
2022-10-18 14:55:10 -07:00
d52e28ac22
combine sort into get_unique_accounts_from_storages ( #28450 )
2022-10-18 11:52:48 -07:00
b583ec79e1
add tests for should_move_to_ancient_append_vec ( #28438 )
...
add tests for should_moev_to_ancient_append_vec
2022-10-18 11:41:35 -07:00
fd2e671861
remove slot# from account hash ( #28405 )
...
* remove slot# from account hash
* add feature
* fix tests
* constants to help clarify 'irrelevant' changes
* move to enum for enforcing irrelevancy
* ignore unsupported tests
2022-10-18 08:03:37 -07:00
682999a423
accounts hash calculation uses files instead of memory ( #28065 )
2022-10-18 07:51:38 -07:00
b669b827fb
refactor and test get_sorted_potential_ancient_slots ( #28436 )
2022-10-18 07:45:43 -07:00
70445b7402
Refactor: Rename `max_invoke_depth` to `max_invoke_stack_height` ( #28427 )
...
Refactor: Rename max_invoke_depth to max_invoke_stack_height
2022-10-17 23:54:56 +08:00
28a89a1d99
remove expected rent collection and rehashing completely ( #28422 )
2022-10-17 07:24:42 -07:00
f896513864
remove unnecessary clones ( #28406 )
2022-10-16 20:33:07 -07:00
0807105c5a
refactoring of accounts hash to prepare for using files always ( #28379 )
2022-10-14 12:29:43 -07:00
31c2b29941
Sends both an EAH and a snapshot request from `set_root()` ( #28363 )
2022-10-14 11:00:04 -04:00
dd7fee8f32
Re-enqueues unhandled ABS requests ( #28362 )
2022-10-13 16:25:39 -04:00
9cbd00fdbc
Converts PendingAccountsPackage to a channel ( #28352 )
2022-10-13 12:47:36 -04:00
72d6927fa1
factor out hashing of storages for caching ( #28261 )
2022-10-13 08:53:54 -07:00
53a1a14868
eliminate worthless work in store_accounts_to ( #28365 )
2022-10-12 13:30:20 -07:00
664339e239
Refactor - executor_cache ( #28332 )
...
* Renames CachedExecutors => BankExecutorCache.
* Renames TransactionExecutor => TransactionExecutorCacheEntry.
* Renames TransactionExecutorStatus => TxBankExecutorCacheDiff.
* Adds TransactionExecutorCache.
* Renames the items of TxBankExecutorCacheDiff.
2022-10-12 18:09:03 +02:00
db9e32d71d
Revert "log adding root every 10s ( #28280 )" ( #28334 )
...
This reverts commit 16853acf35
2022-10-11 12:49:49 -05:00
020cc090d5
Refactors ABS snapshot request handling ( #28326 )
2022-10-11 13:17:06 -04:00
cdf77bb824
runtime: fix typo 'transaction' ( #27948 )
2022-10-11 07:47:10 -07:00
7a120b8b62
accounts_db::load returns None for zero lamport accounts ( #28311 )
2022-10-11 07:43:03 -07:00
d500391006
Removes redundant accounts_package_type ( #28325 )
2022-10-10 17:11:38 +00:00
2fc8e533a2
Refactor - Move `executor_cache` to program-runtime crate ( #28322 )
...
* Moves CachedExecutors, related structs, consts and tests into the program-runtime crate.
* Moves TransactionExecutor, related enum and type defs into executor_cache mod.
2022-10-10 15:01:41 +02:00
16853acf35
log adding root every 10s ( #28280 )
2022-10-07 09:19:34 -07:00
a1fe8dd444
add info for load race condition asserts ( #28277 )
2022-10-07 08:55:01 -07:00
0324573667
report additional transaction errors to metrics ( #28285 )
2022-10-07 10:36:22 -05:00
a400178744
hash calc chunking cleanup ( #28274 )
2022-10-06 23:23:26 -07:00
76c30e7bd0
move accounts_db::load_without_fixed_root to test mod ( #28271 )
2022-10-06 17:11:08 -07:00
66cd290d04
add chunk tracking in SplitAncientStorages ( #28202 )
2022-10-06 11:19:53 -07:00
bb9a19ac86
minor refactoring of accounts hash cache ( #28260 )
2022-10-06 10:50:16 -07:00
435d4aded9
run initial hash calc in background, using background threads ( #28239 )
2022-10-05 19:42:06 -07:00
eec2a2a301
Move epoch accounts hash utility functions ( #28219 )
2022-10-05 23:29:22 +00:00
456d4ae93a
accounts hash calc, add get_ancient_slots ( #28240 )
2022-10-05 15:32:39 -07:00
ad0742c7cf
exit early deep in bank freeze, avoiding some unnecessary work ( #28238 )
...
exit early on bank freeze
2022-10-05 14:49:33 -07:00
a8c6a9e5fc
Bank::freeze() waits for EAH calculation to complete ( #28170 )
2022-10-05 17:44:35 -04:00
3f5eec32cd
refactor SplitAncientStorages::new for testing ( #28235 )
2022-10-05 12:56:29 -07:00
f464c69764
refactor into SplitAncientStorages::new ( #28228 )
2022-10-05 08:46:53 -07:00
3339570a70
accounts hash scan refactoring ( #28220 )
2022-10-04 10:19:50 -07:00
fd11e261dd
use ref for snapshot_storages.iter_range ( #28205 )
2022-10-04 09:11:44 -07:00
ddf95c181c
RPC: Support versioned txs in getFeeForMessage API ( #28217 )
...
* RPC: Support versioned txs in getFeeForMessage API
* Update sdk/program/src/message/sanitized.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-10-05 00:00:34 +08:00
39b37e2c6f
remove LoadZeroLamports enum ( #28204 )
...
remove feature return_none_for_zero_lamport_accounts
2022-10-04 08:06:56 -07:00
e2c0999568
rename fields in SplitAncientStorages ( #28200 )
2022-10-03 17:18:08 -07:00
df76cb5ad9
refactor into split_storages_ancient ( #28194 )
2022-10-03 14:25:33 -07:00
7b4a6ded5b
remove feature return_none_for_zero_lamport_accounts ( #28144 )
2022-10-03 12:22:56 -07:00
05ebd42ac5
remove troublesome assert ( #28195 )
2022-10-03 12:18:48 -07:00
2d4c88574c
add accounts hash get_binned_data() ( #28168 )
2022-10-03 12:17:32 -07:00
5d044ccb34
abs: Update last_full_snapshot_slot before calling clean_accounts() ( #27940 )
2022-10-03 09:34:13 -04:00
adc4e633a2
accounts hash calc refactoring ( #28164 )
2022-10-01 17:09:50 -07:00
7fd8540b49
refactor CacheHashDataFile to use mmapped data ( #28163 )
...
reafactor CacheHashDataFile to use mmapped data
2022-10-01 17:09:36 -07:00
929a311155
refactor accts hash tests ( #28165 )
2022-10-01 17:09:29 -07:00
9255506e92
refactor some accts hash tests ( #28167 )
2022-09-30 14:24:48 -07:00
2f8f6c6a31
Send Epoch Accounts Hash requests from set_root() ( #27764 )
2022-09-30 14:59:41 -04:00
cfc124c825
acct idx can no longer use write cache ( #28150 )
2022-09-30 10:55:27 -07:00
2196fcb4c8
alignment work ( #28123 )
2022-09-30 07:34:18 -07:00
80c0173452
width of hash calc bins was off by 1 ( #28124 )
2022-09-29 11:45:53 -07:00
e6b237c2e8
eah: Add log in bank ( #28110 )
2022-09-28 14:45:42 -04:00
5b4a669e5f
fix typo
2022-09-28 09:48:56 -05:00
4d02292931
Use epoch_accounts_hash::calculation_stop() ( #28106 )
2022-09-28 09:11:18 -04:00
a89010cc89
fix duplicate local variable ( #28093 )
2022-09-27 08:48:05 -07:00
c7cda6c658
refactor hash calc caching ( #27990 )
2022-09-26 15:21:47 -07:00
cdb95a362f
update upacking thread names ( #28081 )
...
* update upacking thread names
* remove - add ht
2022-09-26 14:51:31 -05:00
1aa9c57a41
hash cache get_slice ( #28018 )
2022-09-26 09:30:20 -07:00
a0f49c2e4f
Add assertions for computed block cost limit constants ( #28056 )
2022-09-26 12:50:31 +00:00
71aee4fcaf
Feature: Explicitly limit `TransactionContext::instruction_trace_capacity` ( #27938 )
...
* Renames instruction_stack_capacity => instruction_stack_capacity.
* Replaces number_of_instructions_at_transaction_level by instruction_trace_capacity.
* Adds MaxInstructionTraceLengthExceeded.
* Adjusts TransactionContext::new() parameter.
* Adds feature gate limit_max_instruction_trace_length.
* Adds test_max_instruction_trace_length().
2022-09-26 10:47:16 +02:00
de7a5f2c68
get rid of unnecessary mut ( #28017 )
2022-09-25 18:39:02 -07:00
b9f4c8e3c0
BorrowedAccount: add set_data_from_slice(), make set_data() take owned values ( #27836 )
...
* BorrowedAccount: add set_data_from_slice(), make set_data() take owned values
set_data() used to take a slice and would force alloc+copy if the caller
has owned values (eg account creation, account lookup table).
Expose set_data_from_slice() for callers that have slices, and switch
set_data() to taking an owned Vec.
* BorrowAccount: refactor common accounts_update_delta code in helper method
* BorrowedAccount: add extend_from_slice()
This allows avoiding copies appending entries to account lookup tables.
* BorrowedAccount: remove unnecessary ifs around update_accounts_resize_delta
2022-09-24 01:37:02 +01:00
f64242847d
increase instruction data cost ( #27992 )
2022-09-23 16:15:31 -05:00
9ee53e594d
patches clippy errors from new rust nightly release ( #28028 )
2022-09-23 20:57:27 +00:00
ca55fc8a05
root_bank_cache ( #27985 )
...
* BankForks: atomic root
* root bank cache
* fix and clean up
* added some tests
2022-09-23 15:01:03 -05:00
89b0abcf4d
get_one_epoch_old_slot_for_hash_calc_scan to handle many old append vecs ( #28000 )
2022-09-23 11:17:02 -07:00
315707504e
boxes StakeState in stake_account::Error::InvalidDelegation ( #28006 )
...
This avoids clippy::result_large_err error on newer rust versions:
https://rust-lang.github.io/rust-clippy/master/#result_large_err
2022-09-23 14:04:44 +00:00
fb686cc83b
Nodes with 0% commission are now included in the `getBlock` RPC method `rewards` response field.
2022-09-22 20:31:41 -07:00
9a57c64f21
patches clippy errors from new rust nightly release ( #27996 )
2022-09-22 22:23:03 +00:00
64f64dedea
Use regex in get_snapshot_file_kind ( #27728 )
...
refactor get_snapshot_file_kind to use regex
2022-09-22 13:55:48 -05:00
1ee595ca9c
remove AccountsDb::initial_blockstore_processing_complete ( #27974 )
2022-09-22 13:52:04 -04:00
a846d50b6a
Interleaved snapshot unpack versioning ( #27484 )
...
* Issue #27346 - deserialize and check snapshot version before account fields
* Update comment on SnapshotFileKind
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* SnapshotStorageRebuilderResult to RebuiltSnapshotStorage
* better error propagation from rebuild_storage
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-09-22 11:44:25 -05:00
5d7130623f
abs: Snapshot requests are handled in priority-order ( #27942 )
2022-09-21 12:39:09 -04:00
ceac6a1094
abs: Refactor determining the accounts package type ( #27944 )
2022-09-20 20:31:08 -04:00
6e24edb771
log metric in case of race condition ( #27951 )
2022-09-20 16:23:25 -07:00
ab8b3386a1
runtime: fix typo for 'exempt' ( #27947 )
2022-09-20 16:58:57 -06:00
aa52b8aec1
runtime: fix 'governed' typo ( #27946 )
2022-09-20 13:05:13 -06:00
4893dd3490
re-enable caching of hash calculation ( #27920 )
2022-09-20 10:44:57 -07:00
d74480a125
when shrinking, don't send entire old append vec to clean ( #27914 )
2022-09-20 06:32:07 -07:00
f2d6a7ecea
bank.initial_blockstore_processing_complete to avoid concurrent hash calculations ( #27776 )
...
* bank.initial_blockstore_processing_complete to avoid concurrent hash calculations
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Rename TestValidator::set_startup_verification_complete()
* Initialize with `AtomicBool::new(false)` instead of `default()`
* snapshot tests: move where `initial_blockstore_processing_completed()` is called
* fixup bank_forks.rs calling `is_initial_blockstore_processing_complete()`
* only call initial_blockstore_processing_completed() in blockstore_processor
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@solana.com>
2022-09-19 13:00:21 -07:00
9f097301df
remove dead code ( #27858 )
2022-09-19 07:36:40 -07:00
7d06f7a174
runtime: fix typo in expected_rent_collection.rs ( #27890 )
2022-09-17 21:58:33 -06:00
0e19b22e08
runtime: fix typo occured -> occurred ( #27891 )
2022-09-17 21:58:09 -06:00
c1c93adf67
runtime: fix typo recalcluate -> recalculate ( #27892 )
2022-09-17 21:57:38 -06:00
9cec2ef577
runtime: fix typo situtation -> situation ( #27893 )
2022-09-17 21:57:13 -06:00
543897a439
runtime: fix typo idnex -> index ( #27895 )
2022-09-17 21:56:25 -06:00
36fe33a1f3
runtime: ifno -> info ( #27897 )
2022-09-17 21:55:53 -06:00
c237f15295
runtime: ParitalEq -> PartialEq ( #27900 )
2022-09-17 21:55:29 -06:00
8c24840247
runtime: compleleted -> completed ( #27901 )
2022-09-17 21:53:57 -06:00
1db7da5c32
pass enable_rehashing to verify_bank_hash_and_lamports ( #27827 )
2022-09-17 06:09:06 -07:00
1d701aa4f9
ancient append vec stats: ancient_append_vecs_shrunk ( #27824 )
2022-09-16 12:58:48 -07:00
1a4b8b5b79
recycle_store_count is moved to try_recycle_store ( #27797 )
2022-09-15 21:27:09 +02:00
b9700244b5
Increase transaction account lock limit from 64 to 128 ( #27242 )
...
* Increase transaction account lock limit from 64 to 256
* fix bpf test
* Reduce lock limit from 256 to 128
2022-09-15 13:50:14 -04:00
138d5ed5b0
don't return zero lamport accounts from 'load' with feature ( #27793 )
...
* don't return zero lamport accounts from 'load'
* add feature
* rename
2022-09-15 08:46:32 -07:00
6a7fbf7aba
move create_store counting to create_store ( #27781 )
2022-09-14 22:02:29 +02:00
2a875a894b
Fix and expand EAH calculation info ( #27779 )
2022-09-14 15:23:43 -04:00
3a47dfb8e6
add bank.bank_enable_rehashing_on_accounts_hash ( #27768 )
2022-09-14 11:29:23 -07:00
4f232250e5
add `dirty_ancient_stores` to clean metrics ( #27774 )
2022-09-14 06:52:51 -07:00
416b45ac0f
calculate_capitalization gets 'enable_rehashing' ( #27767 )
2022-09-14 06:52:18 -07:00
cd44a0ab7b
acctidx: unref returns whether refcount went negative ( #27773 )
2022-09-14 06:51:50 -07:00
17007fdda8
remove ancient append vec stores from shrink_candidate_slots ( #27763 )
2022-09-14 06:50:34 -07:00
0c185d1983
Fix a constant that was missed ( #27731 )
...
Missed constant
2022-09-12 16:23:55 -05:00
0bcd9134bc
Rework 'remove_dead_slots_metadata: slots' info log message
2022-09-12 13:36:05 -07:00
d242c0fc97
pass 'enable_rehashing' through all accounts hash functions ( #27701 )
2022-09-12 12:18:09 -07:00
ac74fc16b6
cleanup test constants ( #27704 )
2022-09-12 12:00:52 -07:00
765c628546
use exit signal for acct idx bg threads ( #27483 )
2022-09-12 11:51:12 -07:00
4f22ee8f9b
uses varint encoding for vote-state lockout offsets
...
The commit removes CompactVoteStateUpdate and instead reduces serialized
size of VoteStateUpdate using varint encoding for vote-state lockout
offsets.
2022-09-12 16:31:20 +00:00
90b8a3a44d
Remove KeypairInsecureClone trait and add insecure_clone() instead ( #27396 )
...
See discussion in #26248
2022-09-12 14:59:41 +00:00
d36c5774a1
Use `.find()` instead of loop+filter+break for sending snapshot request ( #27674 )
2022-09-10 19:11:37 -04:00
bd6177d5f2
move fn only used by tests to tests ( #27699 )
2022-09-10 13:57:52 -07:00
92a5d27cb8
remove unused 'unref' fn ( #27698 )
2022-09-10 13:57:31 -07:00
1811d684b5
clean race condition with extra unref ( #27682 )
...
* clean race condition with extra unref
* clarify comment
* fix test
* cleanup test
* clippy
2022-09-10 13:57:17 -07:00
abd01553d5
tests: Keypair::new().pubkey() -> pubkey::new_rand ( #27705 )
2022-09-10 13:56:45 -07:00
1d9d5bef04
remove outdated comment ( #27703 )
2022-09-10 13:56:08 -07:00
ced8f6a512
Add feature gate to remove support for RequestUnitsDeprecated instruction ( #27503 )
...
* feature gate: remove support for RequestUnitsDeprecated instruction #27500
* review update: stop support deprecated ix for prioritization
* Apply suggestions from code review
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
2022-09-09 22:24:21 +00:00
2b554560fd
Send snapshot request for correct bank in set_root() ( #27673 )
2022-09-09 17:30:37 -04:00
6561428278
Separate add_un_ref for easier tracing ( #27652 )
2022-09-09 16:14:55 -05:00
fc396f965c
Fixup log of Epoch Accounts Hash when freezing bank ( #27688 )
2022-09-09 16:33:12 -04:00
1f00b468e5
add enable_rehashing to AccountsPackage ( #27644 )
2022-09-08 09:25:25 -07:00
82d12c4dda
suppress large debug message on test ( #27646 )
2022-09-08 07:01:01 -07:00
6a322de845
Make Accounts Background Services aware of Epoch Accounts Hash ( #27626 )
2022-09-07 20:41:40 +00:00
d3ca364e9e
reuse shared function ( #27642 )
2022-09-07 13:11:00 -07:00
22007a3c96
allow accounts hash calc to specify enable_rehashing ( #27615 )
2022-09-07 10:16:52 -07:00
a31d4a597d
serialize epoch_accounts_hash ( #27516 )
2022-09-07 10:07:00 -07:00
1cf9077d06
Include the Epoch Accounts Hash in the Bank hash ( #27541 )
2022-09-07 10:10:49 -04:00
93a4f80a2c
Handling snapshot requests is now required ( #27537 )
2022-09-07 10:08:42 -04:00
acd846b90e
when storing detailed debug info on accounts hash, we're ok with > 1 storage ( #27617 )
2022-09-07 06:38:57 -07:00
ecbd5bb807
acctidx: avoid extra addref in combine ancient slots ( #27545 )
...
* acctidx: avoid extra addref in combine ancient slots
* make code common
2022-09-06 13:57:34 -07:00
a2df1e95dc
Hold `PendingAccountsPackage` lock for both checking and submitting ( #27613 )
...
Hold package lock for both checking and submitting
2022-09-06 20:11:13 +00:00
Jeff Washington (jwash)
Trent Nelson
Ashwin Sekar
Alexander Meißner
Brooks Prumo
Justin Starry
steviez
Ian Macalinao
Tao Zhu
Haoran Yi
apfitzge
Alessandro Decina
behzad nouri
Michael Vines
kirill lykov
Christian Kamm
Jeff Washington (jwash)