5889ee30b0
add test_get_keys_to_unref_ancient ( #28664 )
2022-10-29 14:23:41 -07:00
bfa62da29e
Renames variant to CalcAccountsHashDataSource::IndexForTests ( #28623 )
2022-10-29 08:55:33 -04:00
2ebf248f47
Renames fn to Bank::update_accounts_hash() ( #28619 )
2022-10-28 17:31:16 -04:00
f98e4d9198
Renames BankHashInfo field to accounts_hash ( #28665 )
2022-10-28 15:18:25 -04:00
5a3d252899
Renames fn to Bank::update_accounts_hash_for_tests() ( #28620 )
2022-10-28 14:33:05 -04:00
8924829e7b
extract 'unref_pubkeys' for shared code ( #28653 )
2022-10-28 10:12:29 -07:00
0232944c95
Uses Storages to update accounts hash in bank_to_snapshot_archive() ( #28618 )
2022-10-28 12:22:58 -04:00
0b2bc987ca
tests for maybe_unref_accounts_already_in_ancient ( #28652 )
2022-10-28 08:24:10 -07:00
0c1df82d48
add current_ancient tests ( #28651 )
2022-10-28 07:32:23 -07:00
8ae5af2822
ancient append vec tests ( #28635 )
2022-10-27 15:10:22 -07:00
3d7cca6296
cleanup drop in ancient append vecs ( #28617 )
2022-10-27 11:16:46 -07:00
7b70aef33c
Add help text to genesis creation time assert ( #28629 )
...
If a validator starts with a snapshot from the wrong cluster, an assert
will trigger complaining about the genesis_creation_time encoded in the
snapshot differing from the genesis_creation_time embedded in
genesis.bin.
This could be a fairly easy mistake to make so add a help string to the
assertion to point operators in the right direction.
2022-10-27 18:02:36 +02:00
646514e4d5
give metric around 'remove_old_stores_shrink" better name ( #28613 )
2022-10-27 07:33:01 -07:00
4ac52c2a9d
ancient append vec comments and cleanup ( #28616 )
2022-10-27 07:32:42 -07:00
251cab4c8b
extract handle_dropped_roots_for_ancient from ancient append vec code ( #28611 )
2022-10-27 07:32:04 -07:00
a2b0d0e6f0
runtime: use correct module doc comments ( #27889 )
2022-10-27 14:39:44 +02:00
decf3de453
extract ancient maybe_unref_accounts_already_in_ancient ( #28610 )
2022-10-26 14:20:32 -07:00
d4db649e7c
extract common code remove_old_stores_shrink from shrink and ancient append vecs ( #28588 )
2022-10-26 11:49:10 -07:00
27269d833c
Uses enum for data source with calc_accounts_hash() ( #28584 )
2022-10-26 13:04:01 -04:00
252d7f68da
combine all_are_zero_lamports ( #28567 )
2022-10-26 08:55:20 -07:00
2d8665d307
Record inner instruction stack height ( #28430 )
...
* Record inner instruction stack height
* fix sbf tests
* feedback
2022-10-26 10:37:44 +08:00
1dbcb78de7
Renames fn to update_accounts_hash() ( #28574 )
2022-10-25 16:43:19 -04:00
51e1a7f2af
simplify mark_dirty_dead_stores ( #28565 )
2022-10-25 11:34:58 -07:00
470ed4f06e
update stats in shrink_collect ( #28563 )
2022-10-25 11:27:33 -07:00
e79c1cba03
Renames fn to update_accounts_hash_for_tests() ( #28571 )
2022-10-25 14:03:19 -04:00
8c57efb828
ancient append vecs: get rid of some locals in shared code ( #28559 )
2022-10-25 09:32:14 -07:00
2c837ba4e3
reorder code to prepare for sharing ( #28564 )
2022-10-25 09:15:26 -07:00
f158bab0ef
Tracks how long background requests wait before processing ( #28581 )
2022-10-25 12:10:53 -04:00
bf214d36e3
Renames fn to calculate_accounts_hash_with_verify() ( #28573 )
2022-10-25 11:55:26 -04:00
e2438cf3e2
Renames fn to calculate_accounts_hash() ( #28572 )
2022-10-25 11:53:41 -04:00
1ca7c8244f
rename merge() -> accumulate() ( #28561 )
2022-10-25 08:52:02 -07:00
bc02789c43
Renames fn to calculate_accounts_hash_from_storages() ( #28566 )
2022-10-24 21:07:00 -04:00
2354a0a343
Renames fn to calculate_accounts_hash_from_index() ( #28568 )
2022-10-24 19:20:08 -04:00
dde8f0dfad
CurrentAncientAppendVec for clarity ( #28528 )
2022-10-24 15:28:52 -07:00
8acd6bb34a
share index_read_elapsed code ( #28557 )
2022-10-24 15:19:40 -07:00
a422387ca8
cleanup ancient append vec merge stats ( #28556 )
2022-10-24 13:11:12 -07:00
8a538f8da5
skip_rewrite skips almost all accounts with no rent ( #28552 )
2022-10-24 11:48:00 -07:00
1fbd818647
runtime: remove `Default` req on account scan interfaces ( #28533 )
2022-10-21 17:53:06 -06:00
993583d9b3
simplify get_store_for_shrink ( #28524 )
2022-10-21 14:49:34 -07:00
9eafad467c
Add convenience methods to VoteInstruction to distinguish vote types ( #28526 )
...
* Add convenience methods to VoteInstruction to distinguish vote types
* use matches! macro instead
2022-10-21 14:17:40 -06:00
fb94e50678
remove single use function for clarity ( #28523 )
2022-10-21 11:42:27 -07:00
f207af765e
Split out voting and banking threads in banking stage ( #27931 )
...
* Split out voting and banking threads in banking stage
Additionally this allows us to aggressively prune the buffer for voting threads
as with the new vote state only the latest vote from each validator is
necessary.
* Update local cluster test to use new Vote ix
* Encapsulate transaction storage filtering better
* Address pr comments
* Commit cargo lock change
* clippy
* Remove unsafe impls
* pr comments
* compute_sanitized_transaction -> build_sanitized_transaction
* &Arc -> Arc
* Move test
* Refactor metrics enums
* clippy
2022-10-20 21:10:48 +00:00
cb7fe87abb
bank: factor accumulation out to helper on `TotalAccountsStats`
2022-10-20 10:50:51 -07:00
09ede8b85c
reuse update_shrink_stats for combine_ancient_slots ( #28466 )
2022-10-19 14:17:34 -07:00
5aca7df232
Bank::create_executor() ( #28474 )
...
* Moves the integration test test_bpf_loader_upgradeable_deploy_with_max_len() from the bpf_loader into the runtime.
* Adds Bank::create_executor().
* Adds a test for Bank::create_executor().
2022-10-19 22:28:57 +02:00
70e83e82c6
Waits to handle snapshot requests until after startup verification is complete ( #28481 )
2022-10-19 13:39:47 -04:00
2b11926c9e
simplify combine_ancient_slots ( #28464 )
2022-10-19 09:35:21 -07:00
8b115ff489
factor out update_shrink_stats ( #28465 )
...
* use local for shrink_stats
* renames and remove .as_us()
* factor out update_shrink_stats
* use integer to report us in metrics
2022-10-19 08:08:04 -07:00
040035063f
share shrink_collect code ( #28454 )
...
* share shrink_collect code
* use correct stats
2022-10-19 08:07:05 -07:00
1cc9cf927c
Supports warping with Epoch Accounts Hash ( #28459 )
2022-10-19 10:37:14 -04:00
ca115594ff
remove concept of 'full' ancient append vec ( #28449 )
2022-10-18 15:08:04 -07:00
c89bd0d6a4
make ancient shrink consistent with shrink ( #28457 )
2022-10-18 15:05:32 -07:00
825b83c95c
simple minor refactoring ( #28456 )
2022-10-18 15:05:06 -07:00
3835ac166e
refactor load_accounts_index_for_shrink for easier sharing ( #28455 )
2022-10-18 14:55:10 -07:00
d52e28ac22
combine sort into get_unique_accounts_from_storages ( #28450 )
2022-10-18 11:52:48 -07:00
b583ec79e1
add tests for should_move_to_ancient_append_vec ( #28438 )
...
add tests for should_moev_to_ancient_append_vec
2022-10-18 11:41:35 -07:00
fd2e671861
remove slot# from account hash ( #28405 )
...
* remove slot# from account hash
* add feature
* fix tests
* constants to help clarify 'irrelevant' changes
* move to enum for enforcing irrelevancy
* ignore unsupported tests
2022-10-18 08:03:37 -07:00
682999a423
accounts hash calculation uses files instead of memory ( #28065 )
2022-10-18 07:51:38 -07:00
b669b827fb
refactor and test get_sorted_potential_ancient_slots ( #28436 )
2022-10-18 07:45:43 -07:00
70445b7402
Refactor: Rename `max_invoke_depth` to `max_invoke_stack_height` ( #28427 )
...
Refactor: Rename max_invoke_depth to max_invoke_stack_height
2022-10-17 23:54:56 +08:00
28a89a1d99
remove expected rent collection and rehashing completely ( #28422 )
2022-10-17 07:24:42 -07:00
f896513864
remove unnecessary clones ( #28406 )
2022-10-16 20:33:07 -07:00
0807105c5a
refactoring of accounts hash to prepare for using files always ( #28379 )
2022-10-14 12:29:43 -07:00
31c2b29941
Sends both an EAH and a snapshot request from `set_root()` ( #28363 )
2022-10-14 11:00:04 -04:00
dd7fee8f32
Re-enqueues unhandled ABS requests ( #28362 )
2022-10-13 16:25:39 -04:00
9cbd00fdbc
Converts PendingAccountsPackage to a channel ( #28352 )
2022-10-13 12:47:36 -04:00
72d6927fa1
factor out hashing of storages for caching ( #28261 )
2022-10-13 08:53:54 -07:00
53a1a14868
eliminate worthless work in store_accounts_to ( #28365 )
2022-10-12 13:30:20 -07:00
664339e239
Refactor - executor_cache ( #28332 )
...
* Renames CachedExecutors => BankExecutorCache.
* Renames TransactionExecutor => TransactionExecutorCacheEntry.
* Renames TransactionExecutorStatus => TxBankExecutorCacheDiff.
* Adds TransactionExecutorCache.
* Renames the items of TxBankExecutorCacheDiff.
2022-10-12 18:09:03 +02:00
db9e32d71d
Revert "log adding root every 10s ( #28280 )" ( #28334 )
...
This reverts commit 16853acf35
2022-10-11 12:49:49 -05:00
020cc090d5
Refactors ABS snapshot request handling ( #28326 )
2022-10-11 13:17:06 -04:00
cdf77bb824
runtime: fix typo 'transaction' ( #27948 )
2022-10-11 07:47:10 -07:00
7a120b8b62
accounts_db::load returns None for zero lamport accounts ( #28311 )
2022-10-11 07:43:03 -07:00
d500391006
Removes redundant accounts_package_type ( #28325 )
2022-10-10 17:11:38 +00:00
2fc8e533a2
Refactor - Move `executor_cache` to program-runtime crate ( #28322 )
...
* Moves CachedExecutors, related structs, consts and tests into the program-runtime crate.
* Moves TransactionExecutor, related enum and type defs into executor_cache mod.
2022-10-10 15:01:41 +02:00
16853acf35
log adding root every 10s ( #28280 )
2022-10-07 09:19:34 -07:00
a1fe8dd444
add info for load race condition asserts ( #28277 )
2022-10-07 08:55:01 -07:00
0324573667
report additional transaction errors to metrics ( #28285 )
2022-10-07 10:36:22 -05:00
a400178744
hash calc chunking cleanup ( #28274 )
2022-10-06 23:23:26 -07:00
76c30e7bd0
move accounts_db::load_without_fixed_root to test mod ( #28271 )
2022-10-06 17:11:08 -07:00
66cd290d04
add chunk tracking in SplitAncientStorages ( #28202 )
2022-10-06 11:19:53 -07:00
bb9a19ac86
minor refactoring of accounts hash cache ( #28260 )
2022-10-06 10:50:16 -07:00
435d4aded9
run initial hash calc in background, using background threads ( #28239 )
2022-10-05 19:42:06 -07:00
eec2a2a301
Move epoch accounts hash utility functions ( #28219 )
2022-10-05 23:29:22 +00:00
456d4ae93a
accounts hash calc, add get_ancient_slots ( #28240 )
2022-10-05 15:32:39 -07:00
ad0742c7cf
exit early deep in bank freeze, avoiding some unnecessary work ( #28238 )
...
exit early on bank freeze
2022-10-05 14:49:33 -07:00
a8c6a9e5fc
Bank::freeze() waits for EAH calculation to complete ( #28170 )
2022-10-05 17:44:35 -04:00
3f5eec32cd
refactor SplitAncientStorages::new for testing ( #28235 )
2022-10-05 12:56:29 -07:00
f464c69764
refactor into SplitAncientStorages::new ( #28228 )
2022-10-05 08:46:53 -07:00
3339570a70
accounts hash scan refactoring ( #28220 )
2022-10-04 10:19:50 -07:00
fd11e261dd
use ref for snapshot_storages.iter_range ( #28205 )
2022-10-04 09:11:44 -07:00
ddf95c181c
RPC: Support versioned txs in getFeeForMessage API ( #28217 )
...
* RPC: Support versioned txs in getFeeForMessage API
* Update sdk/program/src/message/sanitized.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-10-05 00:00:34 +08:00
39b37e2c6f
remove LoadZeroLamports enum ( #28204 )
...
remove feature return_none_for_zero_lamport_accounts
2022-10-04 08:06:56 -07:00
e2c0999568
rename fields in SplitAncientStorages ( #28200 )
2022-10-03 17:18:08 -07:00
df76cb5ad9
refactor into split_storages_ancient ( #28194 )
2022-10-03 14:25:33 -07:00
7b4a6ded5b
remove feature return_none_for_zero_lamport_accounts ( #28144 )
2022-10-03 12:22:56 -07:00
05ebd42ac5
remove troublesome assert ( #28195 )
2022-10-03 12:18:48 -07:00
2d4c88574c
add accounts hash get_binned_data() ( #28168 )
2022-10-03 12:17:32 -07:00
5d044ccb34
abs: Update last_full_snapshot_slot before calling clean_accounts() ( #27940 )
2022-10-03 09:34:13 -04:00
adc4e633a2
accounts hash calc refactoring ( #28164 )
2022-10-01 17:09:50 -07:00
7fd8540b49
refactor CacheHashDataFile to use mmapped data ( #28163 )
...
reafactor CacheHashDataFile to use mmapped data
2022-10-01 17:09:36 -07:00
929a311155
refactor accts hash tests ( #28165 )
2022-10-01 17:09:29 -07:00
9255506e92
refactor some accts hash tests ( #28167 )
2022-09-30 14:24:48 -07:00
2f8f6c6a31
Send Epoch Accounts Hash requests from set_root() ( #27764 )
2022-09-30 14:59:41 -04:00
cfc124c825
acct idx can no longer use write cache ( #28150 )
2022-09-30 10:55:27 -07:00
2196fcb4c8
alignment work ( #28123 )
2022-09-30 07:34:18 -07:00
80c0173452
width of hash calc bins was off by 1 ( #28124 )
2022-09-29 11:45:53 -07:00
e6b237c2e8
eah: Add log in bank ( #28110 )
2022-09-28 14:45:42 -04:00
5b4a669e5f
fix typo
2022-09-28 09:48:56 -05:00
4d02292931
Use epoch_accounts_hash::calculation_stop() ( #28106 )
2022-09-28 09:11:18 -04:00
a89010cc89
fix duplicate local variable ( #28093 )
2022-09-27 08:48:05 -07:00
c7cda6c658
refactor hash calc caching ( #27990 )
2022-09-26 15:21:47 -07:00
cdb95a362f
update upacking thread names ( #28081 )
...
* update upacking thread names
* remove - add ht
2022-09-26 14:51:31 -05:00
1aa9c57a41
hash cache get_slice ( #28018 )
2022-09-26 09:30:20 -07:00
a0f49c2e4f
Add assertions for computed block cost limit constants ( #28056 )
2022-09-26 12:50:31 +00:00
71aee4fcaf
Feature: Explicitly limit `TransactionContext::instruction_trace_capacity` ( #27938 )
...
* Renames instruction_stack_capacity => instruction_stack_capacity.
* Replaces number_of_instructions_at_transaction_level by instruction_trace_capacity.
* Adds MaxInstructionTraceLengthExceeded.
* Adjusts TransactionContext::new() parameter.
* Adds feature gate limit_max_instruction_trace_length.
* Adds test_max_instruction_trace_length().
2022-09-26 10:47:16 +02:00
de7a5f2c68
get rid of unnecessary mut ( #28017 )
2022-09-25 18:39:02 -07:00
b9f4c8e3c0
BorrowedAccount: add set_data_from_slice(), make set_data() take owned values ( #27836 )
...
* BorrowedAccount: add set_data_from_slice(), make set_data() take owned values
set_data() used to take a slice and would force alloc+copy if the caller
has owned values (eg account creation, account lookup table).
Expose set_data_from_slice() for callers that have slices, and switch
set_data() to taking an owned Vec.
* BorrowAccount: refactor common accounts_update_delta code in helper method
* BorrowedAccount: add extend_from_slice()
This allows avoiding copies appending entries to account lookup tables.
* BorrowedAccount: remove unnecessary ifs around update_accounts_resize_delta
2022-09-24 01:37:02 +01:00
f64242847d
increase instruction data cost ( #27992 )
2022-09-23 16:15:31 -05:00
9ee53e594d
patches clippy errors from new rust nightly release ( #28028 )
2022-09-23 20:57:27 +00:00
ca55fc8a05
root_bank_cache ( #27985 )
...
* BankForks: atomic root
* root bank cache
* fix and clean up
* added some tests
2022-09-23 15:01:03 -05:00
89b0abcf4d
get_one_epoch_old_slot_for_hash_calc_scan to handle many old append vecs ( #28000 )
2022-09-23 11:17:02 -07:00
315707504e
boxes StakeState in stake_account::Error::InvalidDelegation ( #28006 )
...
This avoids clippy::result_large_err error on newer rust versions:
https://rust-lang.github.io/rust-clippy/master/#result_large_err
2022-09-23 14:04:44 +00:00
fb686cc83b
Nodes with 0% commission are now included in the `getBlock` RPC method `rewards` response field.
2022-09-22 20:31:41 -07:00
9a57c64f21
patches clippy errors from new rust nightly release ( #27996 )
2022-09-22 22:23:03 +00:00
64f64dedea
Use regex in get_snapshot_file_kind ( #27728 )
...
refactor get_snapshot_file_kind to use regex
2022-09-22 13:55:48 -05:00
1ee595ca9c
remove AccountsDb::initial_blockstore_processing_complete ( #27974 )
2022-09-22 13:52:04 -04:00
a846d50b6a
Interleaved snapshot unpack versioning ( #27484 )
...
* Issue #27346 - deserialize and check snapshot version before account fields
* Update comment on SnapshotFileKind
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* SnapshotStorageRebuilderResult to RebuiltSnapshotStorage
* better error propagation from rebuild_storage
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-09-22 11:44:25 -05:00
c4fa849844
chore: bump itertools from 0.10.3 to 0.10.5 ( #27962 )
...
* chore: bump itertools from 0.10.3 to 0.10.5
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.10.3 to 0.10.5.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/commits )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-09-21 15:28:49 -06:00
00e4313542
chore: bump regex from 1.5.6 to 1.6.0 ( #27678 )
...
* chore: bump regex from 1.5.6 to 1.6.0
Bumps [regex](https://github.com/rust-lang/regex ) from 1.5.6 to 1.6.0.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.5.6...1.6.0 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-09-21 18:09:33 +00:00
5d7130623f
abs: Snapshot requests are handled in priority-order ( #27942 )
2022-09-21 12:39:09 -04:00
ceac6a1094
abs: Refactor determining the accounts package type ( #27944 )
2022-09-20 20:31:08 -04:00
6e24edb771
log metric in case of race condition ( #27951 )
2022-09-20 16:23:25 -07:00
ab8b3386a1
runtime: fix typo for 'exempt' ( #27947 )
2022-09-20 16:58:57 -06:00
aa52b8aec1
runtime: fix 'governed' typo ( #27946 )
2022-09-20 13:05:13 -06:00
4893dd3490
re-enable caching of hash calculation ( #27920 )
2022-09-20 10:44:57 -07:00
d74480a125
when shrinking, don't send entire old append vec to clean ( #27914 )
2022-09-20 06:32:07 -07:00
f2d6a7ecea
bank.initial_blockstore_processing_complete to avoid concurrent hash calculations ( #27776 )
...
* bank.initial_blockstore_processing_complete to avoid concurrent hash calculations
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Rename TestValidator::set_startup_verification_complete()
* Initialize with `AtomicBool::new(false)` instead of `default()`
* snapshot tests: move where `initial_blockstore_processing_completed()` is called
* fixup bank_forks.rs calling `is_initial_blockstore_processing_complete()`
* only call initial_blockstore_processing_completed() in blockstore_processor
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@solana.com>
2022-09-19 13:00:21 -07:00
9f097301df
remove dead code ( #27858 )
2022-09-19 07:36:40 -07:00
7d06f7a174
runtime: fix typo in expected_rent_collection.rs ( #27890 )
2022-09-17 21:58:33 -06:00
0e19b22e08
runtime: fix typo occured -> occurred ( #27891 )
2022-09-17 21:58:09 -06:00
c1c93adf67
runtime: fix typo recalcluate -> recalculate ( #27892 )
2022-09-17 21:57:38 -06:00
9cec2ef577
runtime: fix typo situtation -> situation ( #27893 )
2022-09-17 21:57:13 -06:00
543897a439
runtime: fix typo idnex -> index ( #27895 )
2022-09-17 21:56:25 -06:00
36fe33a1f3
runtime: ifno -> info ( #27897 )
2022-09-17 21:55:53 -06:00
c237f15295
runtime: ParitalEq -> PartialEq ( #27900 )
2022-09-17 21:55:29 -06:00
8c24840247
runtime: compleleted -> completed ( #27901 )
2022-09-17 21:53:57 -06:00
1db7da5c32
pass enable_rehashing to verify_bank_hash_and_lamports ( #27827 )
2022-09-17 06:09:06 -07:00
1d701aa4f9
ancient append vec stats: ancient_append_vecs_shrunk ( #27824 )
2022-09-16 12:58:48 -07:00
1a4b8b5b79
recycle_store_count is moved to try_recycle_store ( #27797 )
2022-09-15 21:27:09 +02:00
b9700244b5
Increase transaction account lock limit from 64 to 128 ( #27242 )
...
* Increase transaction account lock limit from 64 to 256
* fix bpf test
* Reduce lock limit from 256 to 128
2022-09-15 13:50:14 -04:00
138d5ed5b0
don't return zero lamport accounts from 'load' with feature ( #27793 )
...
* don't return zero lamport accounts from 'load'
* add feature
* rename
2022-09-15 08:46:32 -07:00
6a7fbf7aba
move create_store counting to create_store ( #27781 )
2022-09-14 22:02:29 +02:00
2a875a894b
Fix and expand EAH calculation info ( #27779 )
2022-09-14 15:23:43 -04:00
3a47dfb8e6
add bank.bank_enable_rehashing_on_accounts_hash ( #27768 )
2022-09-14 11:29:23 -07:00
4f232250e5
add `dirty_ancient_stores` to clean metrics ( #27774 )
2022-09-14 06:52:51 -07:00
416b45ac0f
calculate_capitalization gets 'enable_rehashing' ( #27767 )
2022-09-14 06:52:18 -07:00
cd44a0ab7b
acctidx: unref returns whether refcount went negative ( #27773 )
2022-09-14 06:51:50 -07:00
17007fdda8
remove ancient append vec stores from shrink_candidate_slots ( #27763 )
2022-09-14 06:50:34 -07:00
c0e4379f43
Whickey/version v1.15 ( #27739 )
...
* Bump version to v1.13.0
* Bump version to v1.14.0
* Bump version to v1.15.0
2022-09-13 09:06:15 -05:00
0c185d1983
Fix a constant that was missed ( #27731 )
...
Missed constant
2022-09-12 16:23:55 -05:00
0bcd9134bc
Rework 'remove_dead_slots_metadata: slots' info log message
2022-09-12 13:36:05 -07:00
d242c0fc97
pass 'enable_rehashing' through all accounts hash functions ( #27701 )
2022-09-12 12:18:09 -07:00
ac74fc16b6
cleanup test constants ( #27704 )
2022-09-12 12:00:52 -07:00
765c628546
use exit signal for acct idx bg threads ( #27483 )
2022-09-12 11:51:12 -07:00
4f22ee8f9b
uses varint encoding for vote-state lockout offsets
...
The commit removes CompactVoteStateUpdate and instead reduces serialized
size of VoteStateUpdate using varint encoding for vote-state lockout
offsets.
2022-09-12 16:31:20 +00:00
90b8a3a44d
Remove KeypairInsecureClone trait and add insecure_clone() instead ( #27396 )
...
See discussion in #26248
2022-09-12 14:59:41 +00:00
d36c5774a1
Use `.find()` instead of loop+filter+break for sending snapshot request ( #27674 )
2022-09-10 19:11:37 -04:00
bd6177d5f2
move fn only used by tests to tests ( #27699 )
2022-09-10 13:57:52 -07:00
92a5d27cb8
remove unused 'unref' fn ( #27698 )
2022-09-10 13:57:31 -07:00
1811d684b5
clean race condition with extra unref ( #27682 )
...
* clean race condition with extra unref
* clarify comment
* fix test
* cleanup test
* clippy
2022-09-10 13:57:17 -07:00
abd01553d5
tests: Keypair::new().pubkey() -> pubkey::new_rand ( #27705 )
2022-09-10 13:56:45 -07:00
1d9d5bef04
remove outdated comment ( #27703 )
2022-09-10 13:56:08 -07:00
ced8f6a512
Add feature gate to remove support for RequestUnitsDeprecated instruction ( #27503 )
...
* feature gate: remove support for RequestUnitsDeprecated instruction #27500
* review update: stop support deprecated ix for prioritization
* Apply suggestions from code review
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
2022-09-09 22:24:21 +00:00
2b554560fd
Send snapshot request for correct bank in set_root() ( #27673 )
2022-09-09 17:30:37 -04:00
6561428278
Separate add_un_ref for easier tracing ( #27652 )
2022-09-09 16:14:55 -05:00
fc396f965c
Fixup log of Epoch Accounts Hash when freezing bank ( #27688 )
2022-09-09 16:33:12 -04:00
1f00b468e5
add enable_rehashing to AccountsPackage ( #27644 )
2022-09-08 09:25:25 -07:00
82d12c4dda
suppress large debug message on test ( #27646 )
2022-09-08 07:01:01 -07:00
f338aa62ba
chore: bump serde from 1.0.143 to 1.0.144 ( #27511 )
...
* chore: bump serde from 1.0.143 to 1.0.144
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.143 to 1.0.144.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.143...v1.0.144 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-09-07 16:54:44 -06:00
6a322de845
Make Accounts Background Services aware of Epoch Accounts Hash ( #27626 )
2022-09-07 20:41:40 +00:00
d3ca364e9e
reuse shared function ( #27642 )
2022-09-07 13:11:00 -07:00
22007a3c96
allow accounts hash calc to specify enable_rehashing ( #27615 )
2022-09-07 10:16:52 -07:00
a31d4a597d
serialize epoch_accounts_hash ( #27516 )
2022-09-07 10:07:00 -07:00
1cf9077d06
Include the Epoch Accounts Hash in the Bank hash ( #27541 )
2022-09-07 10:10:49 -04:00
93a4f80a2c
Handling snapshot requests is now required ( #27537 )
2022-09-07 10:08:42 -04:00
acd846b90e
when storing detailed debug info on accounts hash, we're ok with > 1 storage ( #27617 )
2022-09-07 06:38:57 -07:00
ecbd5bb807
acctidx: avoid extra addref in combine ancient slots ( #27545 )
...
* acctidx: avoid extra addref in combine ancient slots
* make code common
2022-09-06 13:57:34 -07:00
a2df1e95dc
Hold `PendingAccountsPackage` lock for both checking and submitting ( #27613 )
...
Hold package lock for both checking and submitting
2022-09-06 20:11:13 +00:00
0d01494cf8
acctidx renames for clarity ( #27606 )
2022-09-06 12:29:45 -07:00
f1297850b8
acctidx: use slot list len to determine if upsert remains in cache ( #27550 )
2022-09-06 10:59:54 -07:00
72a2c02ee7
add --accounts-db-verify-refcounts for debugging ( #27504 )
2022-09-06 09:39:39 -07:00
7bdbb18741
acctidx: factor out update_slot_list_entry ( #27546 )
2022-09-06 07:39:31 -07:00
7b579771b0
log on acct idx derefs below 0 ( #27544 )
2022-09-06 07:34:05 -07:00
ed132e75a9
rename get_account_maps_read_lock to get_bin since we don't use a loc… ( #27543 )
...
rename get_account_maps_read_lock to get_bin since we don't ues a lock anymore
2022-09-06 07:22:55 -07:00
f90e70ff5a
fix some comments ( #27547 )
...
remove stray comment
2022-09-06 07:21:19 -07:00
12d2147efa
Adds `IndexOfAccount` type ( #27599 )
...
Adds the type `IndexOfAccount`.
2022-09-06 11:31:40 +02:00
6f2e556b16
Cleanup: `TransactionContext` ( #27595 )
...
* Lets instruction_accounts_lamport_sum() have the &InstructionContext as parameter directly.
* Updates docu comments.
* Uses accessors methods instead of accessing private properties of other structs.
* Adds #![deny(clippy::indexing_slicing)].
* Has get_signers() return a Result instead of using unwrap().
* Removes InvokeContext::get_key_of_account_at_index().
2022-09-05 16:29:02 +02:00
0b94d5af18
Refactor: `InstructionContext::configure()` ( #27400 )
...
* Replaces InstructionContext::new() by InstructionContext::configure().
Adds TransactionContext::get_next_instruction_context().
* Switch back to using references as parameters.
* Hoists InstructionContext::configure() from TransactionContext::push() into InvokeContext::push().
* Moves InstructionContext::config() to the beginning of InvokeContext::push().
* Hoists InstructionContext::configure() from InvokeContext::push() into InvokeContext::process_instruction().
* Review feedback: Updates docu comments.
2022-09-03 10:34:57 +02:00
c0f41fe6d0
fix a flacky test ( #27572 )
2022-09-02 15:02:22 +00:00
6684c62280
Add SnapshotUsage to SnapshotConfig ( #27508 )
2022-09-02 08:56:23 -04:00
9b8bed86f9
Add getRecentPrioritizationFees RPC endpoint ( #27278 )
...
* Plumb priority_fee_cache into rpc
* Add PrioritizationFeeCache api
* Add getRecentPrioritizationFees rpc endpoint
* Use MAX_TX_ACCOUNT_LOCKS to limit input keys
* Remove unused cache apis
* Map fee data by slot, and make rpc account inputs optional
* Add priority_fee_cache to rpc test framework, and add test
* Add endpoint to jsonrpc docs
* Update docs/src/developing/clients/jsonrpc-api.md
* Update docs/src/developing/clients/jsonrpc-api.md
2022-09-01 23:12:12 +00:00
a6d737d773
Add EpochAccountsHash to AccountsDb ( #27538 )
2022-09-01 12:11:07 -04:00
3e40fad46b
deserialize(or eof) snapshot epoch_accounts_hash ( #27513 )
...
deserialize(or eof) snapshot prior_epoch_accounts_hash
2022-09-01 08:35:52 -07:00
0df0b94a72
randomly shrink ancient append vecs ( #27523 )
2022-09-01 08:32:43 -07:00
28c3552756
remove counter stat for clean which is already captured in clean_accounts_stats
2022-09-01 10:11:08 -05:00
a28f336eb3
add timing unit scale in field name
2022-09-01 10:11:08 -05:00
43e3545c26
add stats of dirty_slots for cleaning during hash calc
2022-09-01 10:11:08 -05:00
407717eb2a
add timing stat for account hash calc with cache
2022-09-01 10:11:08 -05:00
874085aadc
improve performance of unrefing ancient pubkeys ( #27485 )
2022-09-01 07:34:52 -07:00
af971e63ab
Create all shared buffer readers before spawning threads ( #27512 )
...
* Create all shared buffer readers before spawning threads
* Allow 'needless' collect - clippy is wrong
* move clippy allow, spacing
2022-08-31 22:49:02 -05:00
84b5a2bcf4
Refactor: `TransactionContext` ABIv2 adjustments ( #27476 )
...
* Marks methods and structs which are only to be used by the program runtime.
* Removes TransactionContext::get_keys_of_accounts().
* Adds InstructionContext::get_instruction_accounts_lamport_sum().
2022-08-31 17:47:47 +02:00
8c1e193d5d
delay trying to flush cached upserts until far future ( #26908 )
2022-08-31 08:56:26 -05:00
8bb039d08d
collect min prioritization fees when replaying sanitized transactions ( #26709 )
...
* Collect blocks' minimum prioritization fees when replaying sanitized transactions
* Limits block min-fee metrics reporting to top 10 writable accounts
* Add service thread to asynchronously update and finalize prioritization fee cache
* Add bench test for prioritization_fee_cache
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-08-31 08:00:55 -05:00
b5cca0bfa3
add tests to AccountsDb::unref_accounts ( #27474 )
2022-08-30 18:51:20 -05:00
3d03f7b47e
remove unused acct idx::upsert_on_disk ( #27479 )
2022-08-30 15:42:46 -05:00
2e4a52d010
remove storages that only contain empty ( #27452 )
2022-08-30 14:04:17 -05:00
1154bc8636
get rid of acquisition of read lock while unrefing accounts in scan ( #27388 )
2022-08-30 14:04:02 -05:00
9bde3261fd
cleanup ( #27453 )
...
* cleanup
* Update runtime/src/accounts_db.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-08-30 10:58:35 -05:00
e35cd7a34a
comment loop in function ( #27454 )
2022-08-30 10:58:02 -05:00
3d692ba046
refactor clean_dead_slots_from_accounts_index ( #27451 )
2022-08-30 10:06:52 -05:00
73ee94d674
remove old bank hashes with ancient shrink ( #27450 )
2022-08-30 09:31:20 -05:00
a4b8ab2f59
clean_dead_slots_from_accounts_index unrefs correctly ( #27461 )
2022-08-30 08:40:21 -05:00
8f3c1e3ebb
report a datapoint
2022-08-29 19:06:30 -05:00
7e722f15f2
reduce log frequence to 1m when qlen is above the threshold
2022-08-29 19:06:30 -05:00
37bba64613
remove bank drop queue stat
...
remove try_send
add warn! for excessively long bank drop queue
2022-08-29 19:06:30 -05:00
5b64107626
make pruned_bank channel unbonded.
...
In kin-sim, we found that bounded channel causes halt for account
background services. As the number of accounts grows, the time for
pruning and cleaning increases, which would leads to longer intervals
between the pruning of deaded bank slots. With 1.7B accounts, we will
exceed the 10K bounded channel threshold that causes halt of account
back ground services. Without pruning, the node will eventually run out
of memory.
2022-08-29 19:06:30 -05:00
757e46c3c7
Set cap for new allocations per transaction ( #27385 )
2022-08-29 14:30:48 -04:00
49411aaae0
Feature: Interleaved Snapshot Untar and Indexing - Stage 1 ( #26590 )
...
* Add snapshot_storage_lengths_from_fields
* Add SnapshotUnpacker
* Add SnapshotStorageRebuilder
* Rebuild snapshot storage during unpack
* move snapshot unpacker back into snapshot_utils
* use SerializedAppendVecId in storage rebuilder
* create AccountStorageMap type alias
* Wrap storage and next_append_vec_id in a type
* fixed typo
* move use StorageAndNextAppendVecId
* calculate num_rebuilder_threads more concisely
2022-08-29 13:17:27 -05:00
3c7cd62030
Move pruned_banks_receiver into PrunedBanksRequestHandler ( #27445 )
2022-08-29 13:30:06 -04:00
d0983c3cf7
Update runtime background hash verifier thread name
2022-08-26 13:44:11 -07:00
05fe061a7c
Address RUSTSEC-2022-0051
...
`lz4` to 1.24.0
`lz4-sys` to 1.9.4
2022-08-26 14:30:47 -06:00
46a48760db
Switch concurrent replay from feature to param ( #27401 )
...
* Switch concurrent replay from feature to param
2022-08-26 12:36:02 -07:00
cdb699d812
ancient append vecs and cleaning dead slots ( #27353 )
...
cleanup old code
2022-08-26 10:13:41 -07:00
5e71f339c9
Add a SanitizedMessage type that caches writable accounts indexes ( #27317 )
...
* Add a SanitizedMessage type that caches writable accounts indexes
* Add is_writable_account_cache to both SanitizedMessage variants, cache is initialized in constructors
2022-08-25 16:33:41 -05:00
9866510ef2
get rid of extra comment ( #27378 )
2022-08-24 14:10:33 -05:00
841ed68088
clean_accounts add _inclusive ( #27282 )
2022-08-24 08:56:53 -05:00
efa6201eda
Check overflow on vote tx compaction boundary ( #27185 )
...
* Check overflow on vote tx compaction boundary
Check for overflow during the conversion between VoteStateUpdate and
CompactVoteStateUpdate.
* Try removing clippy supress
2022-08-23 22:29:03 -07:00
66e2df8ddc
failed_store_id should be searched per pubkey ( #27342 )
2022-08-23 19:12:50 -05:00
2014525882
clean_rooted_entries adds _EXclusive ( #27309 )
2022-08-23 18:12:47 -05:00
1c4f7bcd37
clean optimization for store count ( #27277 )
2022-08-23 14:56:44 -05:00
56d1a88924
add _inclusive ( #27334 )
2022-08-23 14:19:13 -05:00
a5c5d70c1c
add _inclusive ( #27335 )
2022-08-23 14:19:04 -05:00
7f4dc496ef
add _inclusive ( #27336 )
...
add _inclusive, rename slot
2022-08-23 14:18:55 -05:00
326f43d644
don't mark ancient append vecs as dirty wrt clean ( #27312 )
2022-08-23 09:18:48 -05:00
b2ae7def06
Remove `total_data_size` and `data_size_changed` from `ExecuteDetailsTimings` ( #27051 )
...
* Removes total_data_size and data_size_changed from ExecuteDetailsTimings.
2022-08-23 14:58:32 +02:00
3db5e82c0a
Remove redundant and stale comment ( #27229 )
2022-08-23 13:43:25 +09:00
e4a7d01e10
Rust v1.63 ( #27303 )
...
* Upgrade to Rust v1.63.0
* Add nightly_clippy_allows
* Resolve some new clippy nightly lints
* Increase QUIC packets completion timeout
* Update quinn-udp crate
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-22 18:01:03 -07:00
a655d45f3a
remove unnecessary references ( #27219 )
2022-08-22 19:22:35 -05:00
748c6ea609
in ancient shrink, unref accounts that were already ancient shrunk ( #27294 )
2022-08-22 18:37:46 -05:00
Jeff Washington (jwash)
Brooks Prumo
steviez
Ian Macalinao
Justin Starry
Trent Nelson
Ashwin Sekar
Alexander Meißner
Tao Zhu
Haoran Yi
apfitzge
Alessandro Decina
behzad nouri
Michael Vines
dependabot[bot]
kirill lykov
Will Hickey
Christian Kamm
Jeff Washington (jwash)
Tyera Eulberg
apfitzge
Brennan Watt
Ryo Onodera