a9a94166ef
load_account_into_read_cache ( #25760 )
2022-06-03 13:34:17 -05:00
e6e2058758
Increase accounts stack size to 8mb ( #25592 )
2022-06-03 19:46:01 +02:00
779ec564e4
add data to assert ( #25749 )
2022-06-03 09:32:43 -05:00
7c95ae341d
disk idx: reduce lock calls ( #25734 )
...
* disk idx: reduce lock calls
* adjust ordering for count_buckets_flushed
2022-06-02 18:26:08 -05:00
6fc057c1d9
Remove activated evict_invalid_stakes_cache_entries feature ( #25739 )
2022-06-02 21:04:54 +00:00
ed68c0b889
epoch_stakes_reation -> epoch_stakes_creation in Bank::_new_from_parent ( #25660 )
2022-06-02 12:26:01 -05:00
eeec3fb9c9
Remove unwrap ( #25538 )
2022-06-02 12:18:46 -05:00
e8df90c34b
rename count_buckets_flushed ( #25733 )
2022-06-02 12:16:01 -05:00
ddd0ed0af1
fix ordering in disk idx ( #25735 )
2022-06-02 11:50:44 -05:00
96f71c7d4d
add metrics on rent collection pieces ( #25713 )
2022-06-02 09:08:53 -05:00
d1a6cebaa2
u64 -> duration ( #25708 )
2022-06-02 07:56:47 -05:00
369ff0858d
Helper functions for getting rent partitions between slots ( #25669 )
2022-06-01 20:20:44 -05:00
149a54b786
remove par_iter on update index below threshold ( #25699 )
2022-06-01 17:41:56 -05:00
891dcb2084
move rent collection time into datapoint ( #25714 )
2022-06-01 17:28:47 -05:00
dd0bf4ff4b
add filler accounts gradually with cache flush ( #25678 )
2022-06-01 15:47:21 -05:00
b80a82a9e5
runtime: gate nonce disable at restart slot
2022-06-01 12:00:10 -06:00
635bd40f09
runtime: disable durable nonces
2022-06-01 11:15:16 -06:00
905fef29cb
add filter to AppendVecScan ( #25664 )
2022-06-01 11:55:18 -05:00
94b0ce5d43
remove_old_historical_roots uses accounts_hash_complete_one_epoch_old ( #25673 )
2022-06-01 10:45:26 -05:00
c91fbd289f
set active state to shrink_ancient when doing intersting work ( #25677 )
2022-06-01 10:36:58 -05:00
8bb76fcd87
add ancient metrics ( #25656 )
...
* add ancient metrics
* review feedback
2022-06-01 10:36:23 -05:00
07958fd8ed
use self.init_accum ( #25663 )
2022-06-01 08:38:37 -05:00
8584a3a331
Add feature to increase timestamp bounding on the slow side to 150% of poh estimate (1s/slot) ( #25666 )
2022-05-31 16:53:06 -06:00
5994b4f53f
use object for hash scanning ( #25418 )
2022-05-31 11:51:56 -05:00
56d38e4954
Add lz4 to snapshot filename regex for matching files on load ( #25633 )
...
* Add lz4 to snapshot filename regex for matching files on load
* Add tests for lz4 archive format filenames
2022-05-31 11:06:41 -05:00
6b2f58152b
Ignore rocksdb directories when unpacking genesis archive ( #25575 )
...
Unpacking the rocksdb directory from genesis archive can have unintended
consequences, such as clobbering files in pre-existing rocksdb
directory.
2022-05-31 00:38:45 -05:00
4b1dd0f921
Fix a possible test hang in runtime::test_shared_buffer_sweep ( #25545 )
...
* Fix a possible test hang in runtime::test_shared_buffer_sweep
The test may hang if the total number of threads it wants to start
is greater than the number of threads configured in the rayon
threadpool. Check to avoid this situation from happening.
* Incorporate comment from Xiang Zhu, using std::cmp::min() to simplify the code
2022-05-30 23:39:32 +08:00
387d9d9c77
Store missing unupdated executors earlier
2022-05-28 13:42:35 +10:00
740ee0c502
Fix datapoint name ( #25614 )
2022-05-28 01:01:29 +00:00
4e6b276f6c
bump executor cache max entries ( #25600 )
...
* executor-cache: unify `remove` calls
* executor-cache: metrics - add one-hit-wonder counter
* executor-cache: bump max entries to 256
2022-05-28 01:54:10 +10:00
ce14c63bf0
fix: stop double counting execution timings ( #25605 )
2022-05-28 00:06:29 +10:00
e58de2c233
Check fee payer before loading all accounts ( #25371 )
...
* fix vote account loading
* fix clippy and rename some stuff
* fix bug
Co-authored-by: Justin Starry <justin@solana.com>
2022-05-27 13:44:29 +10:00
7384c41ea7
Speed up bank::tests::test_max_accounts_data_size_exceeded() ( #25523 )
2022-05-25 22:22:30 -04:00
26a02f6f59
Add invoke logging to builtin programs that don't have it yet ( #25230 )
...
* Refactor process_executor_chain
* Enable builtin program invoke logs by default
* feedback
2022-05-26 07:27:00 +08:00
f10c80b49f
Refactor: Rebase offset of `index_in_caller` ( #25531 )
...
* Removes the offset InstructionContext::get_number_of_program_accounts() from InstructionAccount::index_in_caller.
* Removes unreachable SyscallError::InvalidLength in orig_data_lens.get().
2022-05-25 13:43:20 +02:00
7b98ff1929
Refactor: pull out fee payer balance check ( #25519 )
2022-05-25 20:44:03 +10:00
b7149f7f54
Add helper fn to set accounts_data_size_initial for tests ( #25522 )
2022-05-25 02:43:54 +00:00
0dd3c6bf1f
Split up accounts data size field ( #25495 )
2022-05-24 20:26:32 -04:00
2fb096c486
Refactor: Adds `index_in_callee` to `InstructionAccount` ( #25490 )
...
* Adds InstructionAccount::index_in_callee
* Adjusts tests and benches.
* Adds documentation for InstructionAccount.
* Adds InstructionContext::is_duplicate()
2022-05-25 00:04:46 +02:00
e66ea7cb6a
Clean up Bank::commit_transactions parameters
2022-05-24 20:24:42 +08:00
bac05dc55a
calc slot info on max slot once per hash calc ( #25422 )
2022-05-23 10:11:00 -05:00
41f30a2383
stop logging misleading bank hash mismatch ( #25427 )
2022-05-23 08:43:25 -05:00
b05c7d91ed
Fix derive_partial_eq_without_eq clippy lint
2022-05-22 22:22:21 -07:00
f8842032c6
clippy: fix "this let-binding has unit value" warnings ( #25429 )
2022-05-22 12:17:59 -04:00
7eae98c6d9
Introduces `Result` return types to mutable accessors of `BorrowedAccount` ( #25380 )
...
* Introduces result return types to get_data_mut(), set_data() and set_data_length() of BorrowedAccount.
* Introduces result return types to set_owner(), set_lamports() and set_executable() of BorrowedAccount.
2022-05-21 17:47:09 +02:00
e9a6d1ecef
clippy: is_digit(10) -> is_ascii_digit() ( #25428 )
2022-05-21 07:03:11 -05:00
f584b249dd
special case for common case of 1 storage on append vec scan ( #25414 )
2022-05-20 17:36:30 -05:00
851958f77a
Disallow deployment of deprecated _sol_alloc_free syscall ( #25179 )
2022-05-20 13:19:41 -07:00
cfcc18f7f4
apply AccountsInAppendVec to more places ( #25408 )
2022-05-20 14:25:46 -05:00
a25212b087
Move append_vec::test_utils into its own file ( #25407 )
2022-05-20 13:37:45 -05:00
9ef0acdf5d
allow hash calc to use all threads at startup ( #25369 )
2022-05-20 11:45:01 -05:00
467431de89
add ledger-tool accounts-db-skip-initial-hash-calc ( #25354 )
2022-05-20 10:27:00 -05:00
950245c29b
simulate filler account hash calc time at delta hash ( #25387 )
2022-05-20 10:20:44 -05:00
1b4e4ef548
impl squash to ancient append vecs ( #24538 )
...
* squash to ancient append vecs
* pr feedback
* clippy
2022-05-20 10:20:17 -05:00
de03a3d3aa
filler accounts get a non-zero rent_epoch ( #25386 )
...
* fill accounts get a non-zero rent_epoch
* use snapshot_slot
* get rid of perf simulation code
2022-05-20 09:03:55 -05:00
3fe7655378
use iterator instead of vec to iterate accounts ( #25370 )
...
* user iterator instead of vec to iterate accounts
* rename
2022-05-20 08:50:24 -05:00
36ed36edf0
use previous epoch's # slots per epoch ( #25382 )
2022-05-20 08:47:55 -05:00
c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature ( #25291 )
2022-05-19 18:28:46 -07:00
021276c955
Move updating accounts data len from execute to commit ( #25256 )
2022-05-19 12:24:13 -05:00
0f703b71cc
minor test renaming ( #25297 )
2022-05-18 12:53:55 -05:00
fe91f40f8a
use previous epoch rent collector in expected rent collection ( #25289 )
2022-05-18 12:53:45 -05:00
a1522d0024
Use consistent naming for compute unit limit ( #25229 )
...
* Use consistent naming for compute unit limit
* feedback
2022-05-18 13:14:31 +08:00
0820065c98
rpc: Support token-2022 in token-specific calls ( #25150 )
...
* rpc: Support token-2022 in token-specific calls
* Address feedback
2022-05-17 21:02:43 +02:00
85a2e599bb
revert memory map for reading snapshot ( #25259 )
2022-05-17 09:32:18 -05:00
2cf7c73ced
cleanup types in tests ( #25254 )
2022-05-16 15:54:40 -05:00
21e066ef26
Merge pull request from GHSA-wxrh-ff9f-fc6w
2022-05-16 12:30:37 -07:00
3e44046a73
Support lz4 for snapshot archives ( #25089 )
...
* add lz4
* add cargo package file
* fix tests
* use fast lz4 compression
* report snapshot archive format
* add test
* code review feedback
* add cargo.lock
* fix var name
* refactor archive format parsing and add default compress cli arg
* clippy
* add from_cli_arg test
* update cargo.lock
* add lz4 support for mmap
* cargo.lock
* clippy
2022-05-16 12:44:15 -05:00
f81c5df1f0
Include account index in rent paying account tx error ( #25189 )
2022-05-16 23:35:34 +08:00
389d78e424
Don't use mmarinus package ( #25177 )
...
* Revert "Revert "Use memory map to speed up snapshot untar (#24889 )" (#25174 )"
This reverts commit fc793de2dd
2022-05-16 08:49:17 -05:00
b1b3702e6d
Prioritize transactions in banking stage by their compute unit price ( #25178 )
...
* - get prioritization fee from compute_budget instruction;
- update compute_budget::process_instruction function to take instruction iter to support sanitized versioned message;
- updated runtime.md
* update transaction fee calculation for prioritization fee rate as lamports per 10K CUs
* review changes
* fix test
* fix a bpf test
* fix bpf test
* patch feedback
* fix clippy
* fix bpf test
* feedback
* rename prioritization fee rate to compute unit price
* feedback
Co-authored-by: Justin Starry <justin@solana.com>
2022-05-16 12:06:33 +08:00
6c10515ccf
Revert "Serialize lamports per signature in snapshots ( #25181 )" ( #25228 )
...
This reverts commit 35d2a0fd69
2022-05-15 12:04:17 +00:00
35d2a0fd69
Serialize lamports per signature in snapshots ( #25181 )
...
* Serialize lamports per signature
* pr comments
2022-05-14 10:57:51 -07:00
592aa4c21b
Improve hard fork message with more detail ( #25214 )
2022-05-15 01:14:23 +09:00
3d96a1ab76
Block packets in vote-only mode ( #24906 )
2022-05-14 17:53:37 +02:00
7826fef120
clippy ( #25206 )
2022-05-13 21:03:54 -05:00
9ccdc8a050
populate ancient append vec metrics ( #25204 )
...
* populate ancient append vec metrics
* count ancient storages while enumerating storages anyway.
2022-05-13 18:20:15 -05:00
d7dcb62fe1
ancient_append_vec refactoring ( #25203 )
2022-05-13 16:40:26 -05:00
cb037e49ea
is_candidate_for_shrink respects ancient append vecs ( #25205 )
2022-05-13 16:26:46 -05:00
5eeb07fead
coalesce purge stats and reclaim result ( #25183 )
2022-05-13 12:34:11 -05:00
6f1cc5a2a7
assert we haven't squashed to ancient after hash calc ( #25168 )
2022-05-12 18:26:01 -05:00
fc793de2dd
Revert "Use memory map to speed up snapshot untar ( #24889 )" ( #25174 )
...
This reverts commit 3367e44671
2022-05-12 14:48:29 -05:00
08da486c05
additional costs in block capacity calc ( #25059 )
...
* Added additional costs to block capacity computation, and pushed alloc of CostModel all the way to the top of the call chain, instead of reallocing
* Fix two compiler errors
* Update block processing to propagate computed costs, rather than re-computing deeper in the call stack
* Clippy fix
* Reformatting fix after merge
* Add CostModel::sum_without_bpf
2022-05-12 13:52:20 -05:00
3367e44671
Use memory map to speed up snapshot untar ( #24889 )
...
* mmap
* add no_os_memory_reporting_args to ledger-tool
* use safe memory map package for unix. use regular file for windows
* fallback to untaring with filebuf when mmap fails
* update comments
2022-05-12 13:35:19 -05:00
cde15ff687
support 64-bit prioritization fee ( #25027 )
2022-05-12 11:07:36 -07:00
896729f25e
keep track of oldest slot used by last hash calculation ( #25152 )
2022-05-12 11:18:08 -05:00
abb4993a3b
SortedStorages::max_slot_inclusive() ( #25153 )
2022-05-12 10:24:03 -05:00
849cbe93aa
add AccountsToStore::has_overflow ( #25143 )
2022-05-12 10:14:27 -05:00
c56d978cb1
minor cleanup ( #25144 )
2022-05-11 14:58:43 -05:00
68800982ea
Add size_of const fns for upgradeable loader states ( #25131 )
2022-05-11 22:22:59 +08:00
a8930ee14b
add --accounts-db-ancient-append-vecs ( #25125 )
2022-05-11 08:47:07 -05:00
542a14d9b7
test-validator: Always use overridden compute budget in bank ( #25128 )
2022-05-11 15:46:51 +02:00
11fa0db850
more ancient append vec fns ( #25127 )
2022-05-11 08:02:05 -05:00
3ce877d1ab
add metric for ancient append vec count ( #25126 )
2022-05-11 08:01:04 -05:00
c4506269a6
add ancient_append_vecs ( #25119 )
2022-05-10 16:41:04 -05:00
c838e15234
Unset needs_unlock for rebatched transactions batches ( #25095 )
...
* Unset needs_unlock for rebatched transactions batches
* address review comments
2022-05-10 13:39:08 -07:00
2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations ( #24247 )
2022-05-10 16:37:41 -04:00
dc6e28099b
mark_dirty_dead_stores ( #25114 )
2022-05-10 15:08:29 -05:00
e9e267947e
simplify storage_finder -> storage ( #25109 )
2022-05-10 14:18:51 -05:00
e9a9604ca7
avoid zeroing out root stats with empty data ( #25111 )
2022-05-10 13:56:30 -05:00
b91cc87680
assert when rewrites skipped ( #25110 )
2022-05-10 13:55:48 -05:00
8e5bb0a1e2
check file size of accounts cache mmap before indexing ( #25077 )
2022-05-10 13:33:22 -05:00
2d7ce2a6c2
Revert "uses designated thread-pools for account db parallel ops ( #24954 )" ( #25053 )
...
This reverts commit e8bdc27080
2022-05-06 15:20:50 -05:00
082502d4f3
Fail tx sanitization when ix program id uses lookup table ( #25035 )
...
* Fail tx sanitization when ix program id uses lookup table
* feedback
2022-05-07 03:19:50 +08:00
df9a4a0c9d
Bank::new_for_benches uses bench config ( #25017 )
2022-05-06 13:35:18 -05:00
7401bc0fa5
remove_old_historical_roots uses correct epoch schedule ( #25020 )
...
* remove_old_historical_roots uses correct epoch schedule
* Update runtime/src/accounts_db.rs
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
2022-05-06 12:38:09 -05:00
47cd0d9061
Speed up get-data-cost function ( #25013 )
...
* Combine data_bytes_cost into the get_transaction_cost function, for speedup
* Remove a dead function, fix a value in a test
* Respond to comments by changing call signature to take a &mut struct rather than return a tuple
2022-05-06 11:39:00 -05:00
492f89a170
checks account owner when initializing a vote-account ( #25018 )
...
A VoteAccount may only wrap an account if the account owner is
solana_vote_program:id or equivalently this check returns true:
solana_vote_program::check_id(account.owner())
2022-05-06 16:22:49 +00:00
3fff34a65f
explicitly removes accounts with zero lamports from stakes cache ( #25015 )
...
Zero lamport accounts are not stored in accounts-db:
https://github.com/solana-labs/solana/blob/7b5835ddf/runtime/src/accounts_db.rs#L5007-L5014
https://github.com/solana-labs/solana/blob/7b5835ddf/runtime/src/accounts_db.rs#L4481-L4483
However, in order to purge the account from cache, stakes cache update
partially relies on accounts data to be also zeroed out :
https://github.com/solana-labs/solana/blob/7b5835ddf/runtime/src/bank.rs#L3471-L3478
This can be error-prone and introduce inconsistency between accounts-db
and vote/stake cache. This commit instead explicitly removes accounts
from cache if lamports == 0.
2022-05-06 14:04:49 +00:00
0cc97689f9
add stats for stakes_cache_check_and_store_us ( #24349 )
...
* add stats for stakes_cache_check_and_store_us
* remove redundant metrics
2022-05-05 17:22:48 -05:00
96259b4f23
when testing hash calc, also verify cap ( #25012 )
2022-05-05 15:16:15 -05:00
e0092902ea
maybe_update_rent_epoch_on_load uses correct EpochSchedule ( #24999 )
2022-05-05 13:34:03 -05:00
7b5835ddfa
removes manual implementations of serde Deserialize from vote_account ( #25006 )
...
Container attribute #[serde(from = "...")] already achieves desired
implementation.
2022-05-05 16:33:10 +00:00
f6e8e16691
chore: update function name ( #24166 )
2022-05-05 10:27:14 -05:00
e8bdc27080
uses designated thread-pools for account db parallel ops ( #24954 )
...
Instead of global rayon thread-pool, the commit uses
AccountsDb::{thread_pool,thread_pool_clean} for parallel ops in
AccountsDB::{update_index,clean_accounts_older_than_root}.
2022-05-04 22:47:12 +00:00
e8bf812a8a
allow ledger-tool halt at slot to calc hash using write cache ( #24974 )
2022-05-04 16:27:49 -05:00
40986daddf
log write cache contents from purge_slots_from_cache_and_store ( #24948 )
2022-05-04 14:28:47 -05:00
0ca54db524
use bank's epoch_schedule accessor internally ( #24918 )
2022-05-04 13:52:48 -05:00
eae9a66531
Record instructions which are precompiles ( #24743 )
...
* Record instructions which are "is_precompile".
2022-05-04 16:32:14 +02:00
e070c5ca38
default compute units per instruction ( #24899 )
2022-05-03 09:50:06 -07:00
fd44198c0a
fix typo ( #24937 )
2022-05-03 09:57:15 -05:00
755100dd7c
use correct epoch schedule ( #24904 )
2022-05-03 09:21:47 -05:00
112a0b475a
Revert "Refactor to use EpochSchedule from within RentCollector struct" ( #24893 )
...
* Revert "Ran cargo fmt"
This reverts commit 9052e41b324dfeab3b3887fb78dc56c2f7f2fff836c0bdaa78ed7813e6985472d2e605
2022-05-02 13:46:17 -05:00
546730cf42
retire deprecate_rewards_sysvar ( #24651 )
...
* retire deprecate_rewards_sysvar
* Update expected hash for bank after eliminating reqards sysvar
* Hack test_bank_update_vote_stake_rewards to pass after removing sysvar rewards account
* formatting change
* formatting cleanup
* Remove legacy validator Rewards code from unit test. Change expected inflation computation
* Update bank.rs
* Update bank.rs
* Clean up inflation abstraction function
* Formatting cleanup
* rename vars. Add struct for inflation rewards return
* format cleanup + comment
* Fix format warnings
Co-authored-by: Brennan Watt <brennan.watt@solana.com>
2022-05-02 10:10:51 -05:00
74b586ae71
Always retain the highest incremental snapshot for all full snapshots
2022-05-02 07:37:55 -07:00
a73f99823d
Do not hold lock unnecessarily when hashing ( #24815 )
2022-04-29 08:40:28 +00:00
ad7c95b8a5
Update non-circulating supply list ( #24817 )
...
Co-authored-by: publish-docs.sh <maintainers@solana.com>
2022-04-29 02:57:54 +00:00
b4503d4110
token: Add features for new program releases ( #24742 )
...
* token: Add features for new program releases
* Add PR number
2022-04-28 23:43:57 +02:00
4e58b3870c
Update all BankForks methods to return owned values ( #24801 )
2022-04-28 18:51:00 +00:00
e189603d17
Make test less brittle ( #24803 )
2022-04-28 12:09:01 -06:00
7b6880f652
Make test_bank_cloned_stake_delegations() aware of stake minimum delegation ( #24753 )
2022-04-28 09:35:07 -05:00
f22738fd7b
fix test_rent_eager_collect_rent_in_partition ( #24722 )
2022-04-26 23:06:57 -05:00
412a5a0d33
Credits auto rewind on vote recreation ( #22546 )
...
* Credits auto rewind on vote recreation
* Update comment
* Improve comments and tests
* Recommended fn rename
* Restore old feature, and replace new feature key
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-04-26 15:49:35 -06:00
3007f233f4
Log subscription for `All` no longer clobbers `Some` subscription for pubkey ( #24215 )
...
* fix: subscribing for all logs no longer clobbers pubkey-specific subscriptions
* test: write tests to cover logsSubscribe
2022-04-26 10:31:11 -07:00
534a666153
hookup 'just_rewrites' ( #24662 )
2022-04-26 12:14:25 -05:00
b6bc5d2fa1
Aggregate executor update timing metrics
2022-04-26 23:15:22 +08:00
533eadd024
Remove StakeState::get_rent_exempt_reserve()
2022-04-25 21:43:04 -07:00
9044cd7179
Check insert optimization for bank_forks ( #24641 )
...
* check insert optimization
* clippy suggested simplification
2022-04-25 19:17:21 -05:00
099258e4eb
some cleanup ( #24568 )
2022-04-25 13:17:30 -05:00
5231855261
add state for squash ancient ( #24552 )
2022-04-25 09:05:37 -05:00
be0bdd2261
explicitly ignores struct fields which do not impact PartialEq impl ( #24624 )
...
Unless struct fields are explicitly ignored in PartialEq implementation,
there are no compile-time checks that if more fields are added to the
struct, PartialEq implementation is accordingly updated.
2022-04-24 14:33:42 +00:00
8a062273de
Move error counters to be reported by leader only at end of slot ( #24581 )
...
* Add error counters to leader metrics only
* Add dependencies
2022-04-23 18:10:47 -05:00
8bfde0940f
cleanup ( #24598 )
...
* cleanup
* cargo
2022-04-23 08:28:49 -05:00
14ee36a2af
cleanup in vote/stake accounts ( #24608 )
...
* cleanup in vote/stake accounts
* reorder comparisons in accounts_equal
2022-04-23 08:28:21 -05:00
84e3342612
Process blockstore after starting the TVU
2022-04-22 21:17:49 -07:00
83e041299a
Run real snapshot packager while processing blockstore at validator startup
2022-04-22 21:17:49 -07:00
ce7d2964c9
metrics on ancient slots ( #24596 )
2022-04-22 15:01:51 -05:00
0d51596224
sim: Override slot hashes account on simulation bank ( #24543 )
...
* sim: Override slot hashes during simulation
* Add simulation test program
* Address feedback
* Add AccountOverrides explicit type
* Cargo fmt
2022-04-22 12:32:31 +02:00
a0c4be8cd7
add StorableAccountsMovingSlots ( #24555 )
2022-04-21 15:48:20 -05:00
25e9199397
cleanup FlushStats ( #24556 )
2022-04-21 14:05:01 -05:00
3f7b8d3d2c
provide default impl of 'is_empty' ( #24554 )
2022-04-21 12:49:44 -05:00
d524ae293f
collects metrics for cached vote/stake accounts consistency
...
Tracking mismatches between cached vote/stake accounts and accounts-db
in preparation of activating feature for updating rewards at
epoch-boundary using cached vote/stake accounts.
2022-04-21 15:28:41 +00:00
108aa23d90
enforces type-safety in Stakes<StakeAccount> using phantom data
...
StakeAccount<Delegation> can only wrap a stake-state which is a
Delegation; whereas StakeAccount<()> wraps any account with stake state.
As a result, StakeAccount::<Delegation>::delegation() will return
Delegation instead of Option<Delegation>.
2022-04-21 15:28:41 +00:00
f937fcbd95
updates rewards at epoch boundary using cached accounts
...
Loading vote and stake accounts from accounts-db takes a significant
portion of time updating rewards at epoch boundary.
This commit bypasses accounts-db and instead uses vote and stake
accounts cached in bank stakes:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/stakes.rs#L148-L152
These cached accounts are synchronized with accounts-db after each
transaction, and so there should not be any change in the resulting
computation:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/bank.rs#L4526
Nevertheless, to avoid any chances of introducing a consensus issue, the
switch to cached account is feature gated.
2022-04-21 15:28:41 +00:00
b4491ff4ba
adds StakesEnum type representing Stakes<StakeAccount|Delegation>
...
For backward compatibility, we can only serialize and deserialize
Stakes<Delegation>. However Bank caches Stakes<StakeAccount>. This type
mismatch incurs a conversion cost at epoch boundary when updating
EpochStakes.
This commit adds StakesEnum which allows EpochStakes to include either a
Stakes<StakeAccount> or Stakes<Delegation> and so bypass the conversion
cost between the two at the epoch boundary.
2022-04-21 15:28:41 +00:00
454ef38e43
caches StakeAccount instead of Delegation in Stakes
...
The commit makes values in stake_delegations map in Stakes struct
generic. Stakes<Delegation> is equivalent to the old code and is used
for backward compatibility in BankFieldsTo{Serialize,Deserialize}.
But banks cache Stakes<StakeAccount> which includes the entire stake
account and StakeState deserialized from account. Doing so, will remove
the need to load stake account from accounts-db when working with
stake-delegations.
2022-04-21 15:28:41 +00:00
8c4d6357fe
adds StakeAccount type embodying an Account and a StakeState
...
The added type does sanity checks on the Account and stores deserialized
StakeState. Following commits will use this type instead of Delegation
in Stakes.
2022-04-21 15:28:41 +00:00
d5127abf46
Only add hashes for completed blocks to recent blockhashes ( #24389 )
...
* Only add hashes for completed blocks to recent blockhashes
* feedback
2022-04-21 21:05:29 +08:00
a21fc3f303
Apply transaction actual execution units to cost_tracker ( #24311 )
...
* Pass the sum of consumed compute units to cost_tracker
* cost model tracks builtins and bpf programs separately, enabling adjust block cost by actual bpf programs execution costs
* Copied nightly-only experimental `checked_add_(un)signed` implementation to sdk
* Add function to update cost tracker with execution cost adjustment
* Review suggestion - using enum instead of struct for CommitTransactionDetails
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
* review - rename variable to distinguish accumulated_consumed_units from individual compute_units_consumed
* not to use signed integer operations
* Review - using saturating_add_assign!(), and checked_*().unwrap_or()
* Review - using Ordering enum to cmp
* replace checked_ with saturating_
* review - remove unnecessary Option<>
* Review - add function to report number of non-zero units account to metrics
2022-04-21 07:38:07 +00:00
79923c3b58
Refactor: Rename BlockhashQueue fields and methods for clarity ( #24426 )
2022-04-21 11:57:17 +08:00
52f43a986d
skip rewrites in bank ( #23985 )
2022-04-20 14:08:46 -05:00
b1c89cf6d4
append_vec::remaining_bytes ( #24531 )
2022-04-20 13:44:57 -05:00
813d06529d
stats for rewrites ( #24529 )
2022-04-20 10:43:50 -05:00
05438c3cd6
implement remove_old_historical_roots ( #24492 )
2022-04-20 10:43:32 -05:00
cdc3c41001
helpers to skip rewrites in bank ( #24507 )
2022-04-20 10:21:02 -05:00
0d797e2fff
contains_multiple_slots ( #24500 )
2022-04-20 09:53:36 -05:00
cfe2177e16
validator option: accounts-db-skip-rewrites ( #24504 )
2022-04-20 09:41:00 -05:00
957849f4a5
load_to_collect_rent_eagerly returns loaded_slot ( #24506 )
2022-04-20 09:14:03 -05:00
ef5e7cce6e
type PubkeyAccountSlot ( #24503 )
2022-04-19 21:34:00 -05:00
4be01ec75a
remove temp dead_code markers ( #24508 )
2022-04-19 21:30:57 -05:00
9e4999ef6a
Remove halt_at_slot from RuntimeConfig, it's not a runtime concern
2022-04-19 19:23:58 -07:00
0e2e0c8b7d
Extract most storage-related services from the Tvu abstraction
2022-04-19 19:23:58 -07:00
255a6a729d
add historical_roots_len stat ( #24497 )
2022-04-19 16:53:24 -05:00
d23c04bb68
max_slot_in_storages_exclusive -> INclusive ( #24450 )
2022-04-19 13:59:41 -05:00
94b0186a96
Cost model tracks builtins and bpf programs separately ( #24468 )
...
* Cost model tracks builtins and bpf programs separatele (enables adjusting block cost by actual bpf programs execution costs)
* Address reviews: expand test; add metrics stat
2022-04-19 13:25:47 -05:00
2ad1baa753
Add const fn StakeState::size_of and static assertion ( #24416 )
2022-04-20 01:04:12 +08:00
c4923d29b4
bank_hash_at uses rewrites ( #24439 )
2022-04-19 11:29:29 -05:00
4fdf966393
demote shink counter to debug ( #24458 )
2022-04-19 10:02:23 -05:00
569b15d863
maybe_update_rent_epoch_on_load ( #24055 )
2022-04-19 08:29:36 -05:00
dc98510d6d
accounts hash calls maybe_rehash_skipped_rewrite ( #24316 )
2022-04-19 08:29:09 -05:00
5d1adf1270
Fix signature count ( #24471 )
...
* Fix signature count
* protect the signature count futher
2022-04-19 17:59:06 +08:00
d6ff50bcc1
Remove `KeyedAccount` in builtin program "system instruction processor" ( #24452 )
...
* Makes sure that there is only one KeyedAccount at a time.
Moves keyed_account_at_index() into create_account(), transfer_verified(), transfer() and transfer_with_seed() to do so.
* Uses transaction_context.get_key_of_account_at_index() when only the key is needed.
* Uses "!instruction_context.is_signer()" instead of ".signer_key().is_none()".
* Replaces KeyedAccount by BorrowedAccount.
* Removes unused code.
2022-04-19 09:28:07 +02:00
0e0f306996
mechanism for disabling hash calc caching ( #24437 )
2022-04-18 22:44:30 -05:00
94d3ec37d2
collect_rent_eagerly can just do rewrites ( #24438 )
2022-04-18 22:44:19 -05:00
bf1e0c89e8
rename local: max_slot_in_storages -> inclusive ( #24453 )
2022-04-18 22:42:02 -05:00
8b854d4141
Remove `KeyedAccount` in builtin program "nonce keyed account" ( #24436 )
...
* Makes sure that there is only one KeyedAccount at a time.
Moves keyed_account_at_index() into withdraw_nonce_account() to do so.
* Replaces KeyedAccount by BorrowedAccount.
* Removes unused code.
2022-04-18 22:19:51 +02:00
ee384f7254
remove 'startup' from verify_snapshot_bank ( #24441 )
2022-04-18 14:35:49 -05:00
bcc5b96bda
remove redundant EpochSchedule ( #24444 )
2022-04-18 13:04:48 -05:00
9052e41b32
Ran cargo fmt
2022-04-18 11:57:40 -05:00
87fb78dc56
Fix build error introduced by my editor setup
2022-04-18 11:57:40 -05:00
c2f7f2fff8
Remove redundant epoch_schedule from AccountsPackage
2022-04-18 11:57:40 -05:00
ed7813e698
Fixes to initial code
2022-04-18 11:57:40 -05:00
5472d2e605
Removing redundant EpochSchedule param from fns
2022-04-18 11:57:40 -05:00
c6a751d658
Detect and report bank drop signal queue full and disconnect events ( #24112 )
...
* nonblocking send when when droping banks
* detect and report drop signal queue full/disconnect events
* comments
* use counter for reporting bank_drop_queue events
* reduce log
* use datapoint to report stats
* logging instead of reporting bank drop signal full
* fix a corner case for reporting
* fix build
2022-04-18 11:19:11 -05:00
578d59c802
Remove the code that handles cost update for separate pr
2022-04-17 19:26:24 -05:00
85281de981
move number utility functions to sdk; log error if cost adjust overflows i64
2022-04-17 19:26:24 -05:00
9dadfb2e2c
Add checked_add_signed() to apply cost adjustment to cost_tracker
2022-04-17 19:26:24 -05:00
094da35b91
Address review comments:
...
1. use was_executed to correctly identify transactions requires cost adjustment;
2. add function to specifically handle executino cost adjustment without have to copy accounts
2022-04-17 19:26:24 -05:00
29ca21ed78
undo transaction cost from cost_tracker if it was not executed successfully
2022-04-17 19:26:24 -05:00
f33ad34531
Add feature_set_override parameter to mock_process_instruction() ( #24386 )
2022-04-15 13:43:04 -05:00
b4fd9124bf
log secondary index contents on startup ( #24348 )
2022-04-15 13:30:03 -05:00
8c9430359e
add early exit in get_corrected_rent_epoch_on_load ( #24331 )
2022-04-15 13:28:16 -05:00
ba7a2efa66
SlotInfoInEpoch ( #24332 )
2022-04-15 13:27:41 -05:00
4ed647d8ec
Test that tick slot hashes update the recent blockhash queue ( #24242 )
2022-04-16 00:30:20 +08:00
0e7b0597db
check for rewrites skipped in closure ( #24330 )
2022-04-14 13:46:18 -05:00
d20b4c9958
rename function param to avoid conflict ( #24342 )
2022-04-14 11:56:32 -05:00
a91b0c8ea3
dashmap -> rwlock<hashmap> for rewrites ( #24327 )
2022-04-14 11:55:58 -05:00
e3ef0741be
simplify bank drop calls ( #24142 )
...
* simplify bank drop calls
* clippy: import
* Update ledger/src/blockstore_processor.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/accounts_background_service.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* cleanup
* format
* use msb of bank_id to indicates that we are dropping
* clippy
* restore bank id
* clippy
* revert is-serialized_with_abs flag
* assert
* clippy
* whitespace
* fix bank drop callback check
* more fix
* remove msb dropping implementation
* fix
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-14 08:43:54 -05:00
255ef66a27
move feature activation log to correct fn for hash ( #24326 )
2022-04-13 17:43:33 -05:00
6a474f29cd
default to disk index ( #24251 )
2022-04-13 09:24:50 -05:00
b6b8783323
add maybe_update_rent_epoch_on_load ( #24294 )
2022-04-13 08:55:24 -05:00
929753a11f
typo ( #24291 )
2022-04-12 16:46:59 -05:00
69e9ad5571
update comment ( #24288 )
2022-04-12 12:37:46 -05:00
2d4d639635
add expected_rent_collection ( #24028 )
...
* add expected_rent_collection
* update some comments for clarity and resolve a todo
* add test for 'LeaveAloneNoRent'
2022-04-12 11:32:23 -05:00
bdbca3362e
increase test timeout ( #24277 )
2022-04-12 09:54:57 -05:00
1bc49d219d
IndexLimitMb option adds 'Unspecified' state ( #24249 )
2022-04-12 09:38:09 -05:00
605036c117
move test fn into its own mod ( #24212 )
...
* move test fn into its own mod
* pub
2022-04-12 09:36:05 -05:00
f3aa80d3f8
typo ( #24257 )
2022-04-12 09:08:35 -05:00
9b8850f99e
test-validator: Add `--max-compute-units` flag ( #24130 )
...
* test-validator: Add `--max-compute-units` flag
* Add `RuntimeConfig` for tweaking runtime behavior
* Actually add the file
* Move RuntimeConfig to runtime
2022-04-12 02:28:10 +02:00
4fd184c131
Use Release/Acquire instead of SeqCst for is_bank_drop_callback_enabled ( #24134 )
2022-04-11 19:19:17 -05:00
6fbe2b936c
fix comment ( #24254 )
2022-04-11 18:53:45 -05:00
c0019edf00
document WaitableCondvar ( #24252 )
2022-04-11 14:45:23 -05:00
9ac2245970
remove clone ( #24244 )
2022-04-11 13:15:00 -05:00
60b2155bd3
Add accounts-filler-size command line option ( #23896 )
2022-04-11 13:10:09 -05:00
e14933c54d
move bank test fn to its test_utils mod ( #24171 )
2022-04-11 10:42:24 -05:00
8eef3d9713
Add tests to the blockhash queue ( #24238 )
2022-04-11 19:36:24 +08:00
a058f348a2
Address review comments
2022-04-08 14:37:55 -05:00
2ed29771f2
Unittest for cost tracker after process_and_record_transactions
2022-04-08 14:37:55 -05:00
924b8ea1eb
Adjustments to cost_tracker updates
...
- don't store pending tx signatures and costs in CostTracker
- apply tx costs to global state immediately again
- go from commit_or_cancel to update_or_remove, where the cost tracker
is either updated with the true costs for successful tx, or the costs
of a retryable tx is removed
- move the function into qos_service and hold the cost tracker lock for
the whole loop
2022-04-08 14:37:55 -05:00
9e07272af8
- Only commit successfully executed transactions' cost to cost_tracker;
...
- In-fly transactions are pended in cost_tracker until being committed
or cancelled;
2022-04-08 14:37:55 -05:00
03ed334ebb
Double the chunk size for sending the program binary data in tx
2022-04-08 09:06:40 -07:00
210f6a6fab
move hash calculation out of acct bg svc ( #23689 )
...
* move hash calculation out of acct bg svc
* pr feedback
2022-04-08 10:42:03 -05:00
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
48d1af01c8
add metrics around rewards ( #24160 )
2022-04-07 11:44:26 -05:00
f7b2951c79
move around some index metrics to reduce locks ( #24161 )
2022-04-07 09:43:19 -05:00
42c094739d
add test cfg attribute ( #24154 )
2022-04-07 09:05:20 -05:00
550ca7bf92
compare contents of serialized banks instead of exact file format ( #24141 )
...
* compare contents of serialized banks instead of exact file format
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
* get rid of clone
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
fddd162645
reserialize bank in ahv by first writing to temp file in abs ( #23947 )
2022-04-06 21:39:26 -05:00
c322842257
Replace channel with Mutex<Option> for AccountsPackage ( #24013 )
2022-04-06 05:47:19 -05:00
4ea59d8cb4
Set drop callback on first root bank ( #23999 )
2022-04-05 13:02:33 -05:00
4a11fa072f
hash_account_with_rent_epoch ( #24104 )
2022-04-05 08:10:31 -05:00
6a7f6585ce
persist historical_roots ( #24029 )
2022-04-04 13:13:11 -05:00
132f08486a
remove basically duplicate function ( #24107 )
2022-04-04 12:55:05 -05:00
997db7637c
do simple math without floats
2022-04-04 12:32:00 -05:00
f8f3edac3c
update comment ( #24108 )
2022-04-04 11:06:01 -05:00
2820b64eb3
roots_original -> historical_roots ( #24063 )
2022-04-04 09:12:12 -05:00
ef3e3dce7a
hides implementation details of vote-accounts from public interface ( #24087 )
2022-04-04 13:20:26 +00:00
fa7eb7f30c
improves Stakes::activate_epoch performance ( #24068 )
...
Tested with mainnet stakes obtained from the ledger at 5 recent epoch
boundaries, this code is ~30% faster than current master.
Current code:
epoch: 289, elapsed: 82901us
epoch: 290, elapsed: 80525us
epoch: 291, elapsed: 79122us
epoch: 292, elapsed: 79961us
epoch: 293, elapsed: 78965us
This commit:
epoch: 289, elapsed: 61710us
epoch: 290, elapsed: 55721us
epoch: 291, elapsed: 55886us
epoch: 292, elapsed: 55399us
epoch: 293, elapsed: 56803us
2022-04-02 22:48:51 +00:00
0ca5a0ec68
prior_roots -> historical_roots ( #24064 )
2022-04-02 12:01:43 -05:00
ec97d6d078
rename remove_old_roots ( #24059 )
2022-04-02 12:01:13 -05:00
3ca4fffa78
root -> alive_root ( #24062 )
2022-04-02 12:00:52 -05:00
792bbf75ab
Support sending versioned txs in AsyncClient ( #23982 )
2022-04-02 11:12:02 +08:00
1b45c509c3
Refactor: Use `InstructionContext::get_instruction_data()` ( #24014 )
...
* Adds transaction_context and instruction_context where invoke_context.get_keyed_accounts() is used.
* Use instruction_context.get_instruction_data() instead of an explicit parameter.
* Removes instruction_data parameter from Executor::execute().
* Removes instruction_data parameter from ProcessInstructionWithContext.
2022-04-01 15:48:05 +02:00
31997f8251
hash calc scanning takes config ( #24016 )
2022-03-31 14:26:37 -05:00
9c8dad33c7
add epoch_schedule and rent_collector to hash calc ( #24012 )
2022-03-31 10:51:18 -05:00
da001d54e5
calculate_accounts_hash_helper uses config ( #24003 )
2022-03-31 09:29:45 -05:00
cb5e67d327
Use Rent sysvar directly for stake split instruction ( #24008 )
...
* Use Rent sysvar directly for stake split ix
* Add feature to gate rent sysvar change
* fix tests
* cargo clippy
2022-03-31 16:46:35 +08:00
125f9634fd
add hash calc config.use_write_cache ( #24005 )
2022-03-30 17:19:34 -05:00
82c5230bc2
AccountsPackage::new less brittle ( #23968 )
2022-03-30 14:06:15 -05:00
1fb82d7924
fix typo in comments ( #24004 )
2022-03-30 09:47:51 -05:00
af9344fe22
SortedStorages refactoring ( #23998 )
2022-03-30 09:19:03 -05:00
5636570d6d
add roots_original to roots tracker ( #23849 )
2022-03-30 08:52:45 -05:00
da844d7be5
refactoring of SortedStorages tests to make other changes easier ( #23990 )
2022-03-29 22:06:48 -05:00
5a613e9b6e
use CalcAccountsHashConfig in calculate_accounts_hash ( #23987 )
2022-03-29 22:05:47 -05:00
794645d092
Adds check_number_of_instruction_accounts() to all builtin programs except for the address-lookup-table. ( #23984 )
2022-03-29 19:06:50 +02:00
31b707b625
Specify if archive size datapoint is for full or incremental snapshots ( #23941 )
2022-03-26 12:29:13 -05:00
c24de17278
remove index hash calculation as an option ( #23928 )
2022-03-25 15:32:53 -05:00
ec78702bc8
RollingBitField::get_all_less_than ( #23919 )
2022-03-25 15:20:22 -05:00
acfd22712b
RollingBitFIeld to its own file ( #23917 )
2022-03-25 10:37:00 -05:00
55d61023f7
document 'accounts' hash ( #23907 )
2022-03-24 15:58:52 -05:00
140c8dd01f
Refactor: Replaces KeyedAccount in_get_sysvar_with_account_check ( #23905 )
...
* Replaces all use sites of get_sysvar_with_account_check by get_sysvar_with_account_check2.
* Removes get_sysvar_with_account_check.
* Renames get_sysvar_with_account_check2 to get_sysvar_with_account_check.
2022-03-24 19:30:42 +01:00
37c36ce3fa
pass stats separately from CalcAccountsHashConfig ( #23892 )
2022-03-24 12:48:47 -05:00
82328fd9d8
move max_clean_root deeper in flush cache ( #23869 )
2022-03-24 12:45:49 -05:00
c31db81ac4
Use VoteAccountsHashMap type alias in all applicable spots ( #23904 )
2022-03-24 12:09:48 -05:00
a22a2384bf
fix ci test error ( #23908 )
2022-03-24 11:30:20 -05:00
5b916961b5
HashCalc uses self.accounts_cache ( #23890 )
2022-03-24 10:34:28 -05:00
f2aea3b7c7
flush_slot_cache takes [Slot] ( #23865 )
2022-03-24 10:24:36 -05:00
9d3b17c635
HashCalc uses self.accounts_index ( #23888 )
2022-03-24 10:06:32 -05:00
396b49a7c1
Start saving/loading prior_roots(_with_hash) to snapshot ( #23844 )
...
* Start saving/loading prior_roots(_with_hash) to snapshot
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* update comment
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-03-24 10:06:24 -05:00
b22165ad69
hash calc uses self.filler_account_suffix ( #23887 )
2022-03-24 09:58:06 -05:00
9022931689
calc hash uses self.num_hash_scan_passes ( #23883 )
2022-03-24 09:44:42 -05:00
e3eb002f66
Log storage size stats at hash calc ( #23843 )
2022-03-24 09:40:35 -05:00
f1a411c897
add epoch_schedule and rent_collector to hash calc ( #23857 )
2022-03-24 09:39:22 -05:00
db5d68f01f
HashCalc uses self.accounts_hash_cache_path ( #23882 )
2022-03-24 09:31:55 -05:00
90009f330b
small refactor to shorten the lock on slot_under_contention hashset ( #23891 )
...
* small refactor to shorten the lock on slot_under_contention hashset
* adding comments
* comments
2022-03-24 08:20:56 -05:00
91c2729856
Replaces keyed_account get_signers() by InstructionContext::get_signers(). ( #23863 )
2022-03-24 12:57:51 +01:00
5a892af2fe
disable 'check_hash' on accounts hash calc ( #23873 )
2022-03-23 21:03:31 -05:00
3e22d4b286
calc hash uses self.thread_pool_clean ( #23881 )
2022-03-23 20:52:38 -05:00
486f7b7673
use array access function ( #23895 )
2022-03-23 17:03:01 -07:00
dc3863ef14
flush_slot_cache_with_clean ( #23868 )
2022-03-23 14:09:56 -05:00
260f899eda
write cache: hashmap to set ( #23866 )
2022-03-23 14:05:45 -05:00
9e61fe7583
add AccountsHashConfig to manage parameters ( #23850 )
2022-03-23 13:44:23 -05:00
b1280b670a
calculate_accounts_hash_without_index takes &self ( #23846 )
...
* calculate_accounts_hash_without_index takes &self
* Update runtime/src/snapshot_package.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-23 11:57:32 -05:00
7b89222fde
don't start extra threads for shrink/clean/hash ( #23858 )
2022-03-23 11:53:37 -05:00
493a8e2348
remove random flushing of write cache ( #23845 )
2022-03-23 08:45:44 -05:00
bc35e1c5f5
snapshot code needs all storages for hash calc ( #23840 )
2022-03-22 21:27:54 -05:00
9f0ca6d88a
Refactor: Remove `trait` from nonce keyed account ( #23811 )
...
* Removes the trait `NonceKeyedAccount`.
2022-03-23 02:09:30 +01:00
7af48465fa
transaction-status: Add return data to meta ( #23688 )
...
* transaction-status: Add return data to meta
* Add return data to simulation results
* Use pretty-hex for printing return data
* Update arg name, make TransactionRecord struct
* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
1089a38aaf
AcctIdx: rework scan and write to disk ( #23794 )
2022-03-22 11:54:12 -05:00
89ba3ff139
log fail to evict ( #23815 )
2022-03-22 09:19:38 -05:00
24f6855f86
AcctIdx: only remove a fixed number of items per write lock ( #23795 )
2022-03-21 16:55:04 -05:00
cb06126388
Set accounts_data_len on feature activation ( #23730 )
2022-03-21 12:28:26 -05:00
965ab9186d
AcctIdx: fix infinite loop ( #23806 )
2022-03-21 10:58:36 -05:00
15357480ec
Refactor instruction compilation and update message account key ordering ( #23729 )
...
* Refactor: Make instruction compilation usable for other message versions
* apply trents feedback
* Fix tests
* Fix bpf compatiblity
2022-03-21 20:53:32 +08:00
258db77100
AcctIdx: factor 'scan' out of flush_internal ( #23777 )
2022-03-20 22:00:38 -05:00
f34434f96b
Drop lock ( #23765 )
2022-03-20 21:27:24 -04:00
dd69f3baf5
throttle index adding to allow disk flushing to keep up and reduce startup ram usage ( #23773 )
2022-03-20 19:56:20 -05:00
335c4b668b
Fix bug in bank/sysvar_cache tests ( #23780 )
2022-03-19 21:38:18 -05:00
df29276eb0
AcctIdx: remove -> evict ( #23775 )
2022-03-18 17:13:21 -05:00
a419374fa4
factor out function ( #23742 )
2022-03-18 14:10:52 -05:00
56428be629
Not exposing inner cost_table to encapsulating implementation details,
...
making future change easier.
2022-03-18 12:58:43 -05:00
998e7d18f9
AcctIdx: never retry a bucket flush ( #23732 )
2022-03-18 12:20:42 -05:00
0ed23899e7
directly use compute_budget MAX_UNITS and DEFAULT_UNITS
2022-03-18 08:53:11 -05:00
a4cacf3389
add deterministic default cost
2022-03-18 08:53:11 -05:00
857576d76f
AcctIdx: move write to disk outside in mem write lock ( #23731 )
2022-03-17 23:09:41 -05:00
7ff8c80e25
Add accounts_data_len to bank snapshot ( #23714 )
2022-03-17 20:14:54 -05:00
664deb2157
AcctIdx: get rid of unused is_dirty ( #23733 )
2022-03-17 16:29:36 -05:00
66b1f55351
AcctIdx: cheaper check for should evict while flushing ( #23705 )
2022-03-17 08:46:32 -05:00
3a46f45650
AcctIdx: add stats for flushing and estimated memory ( #23709 )
2022-03-17 08:46:00 -05:00
b4350a2522
Make solana-address-lookup-table-program crate bpf compatible ( #23700 )
2022-03-17 08:21:07 +08:00
be0aeea01a
AcctIdx: check for range holds outside lock ( #23706 )
2022-03-16 17:44:59 -05:00
caddb851be
acct idx flush keeps slot_list read lock open ( #23704 )
2022-03-16 17:29:04 -05:00
fa7926580a
minor VoteAccount refactoring ( #23686 )
2022-03-16 09:50:12 -05:00
3252dc7203
uses structural sharing for stake-delegations hash-map ( #23585 )
...
StakeDelegations is using Arc to implement copy-on-write semantics:
https://github.com/solana-labs/solana/blob/58c0db970/runtime/src/stake_delegations.rs#L14-L16
However a single delegation change will still clone the entire hash-map,
resulting in excessive memory use as observed in:
https://github.com/solana-labs/solana/issues/23061#issuecomment-1063444072
This commit instead uses immutable hash-map implementing structural
sharing:
> which means that if two data structures are mostly copies of each
> other, most of the memory they take up will be shared between them.
https://docs.rs/im/latest/im/
2022-03-16 12:58:05 +00:00
584ac80b1e
Revert: `KeyedAccount` refactoings in builtin programs ( #23649 )
...
* Revert "Replaces KeyedAccount by BorrowedAccount in the BPF loader. (#23056 )"
6c56eb9663#23217 )"
ee7e411d68#23214 )"
1a68f81f89#23302 )"
a14c7c37ee#23348 )"
e2fa6a0f7a#23375 )"
ee3fc39f1c
2022-03-16 11:30:01 +01:00
2d3501dff9
make upsert infallible op
2022-03-15 17:05:41 -05:00
b5a99b9b09
avoid data copies in writing to cache ( #23674 )
2022-03-15 16:42:26 -05:00
8c4f010b8d
fix logging: only walk remote dir if it exist ( #23663 )
2022-03-15 08:56:22 -05:00
8c8f9694e0
Refactor: Sanitized transaction creation ( #23558 )
...
* Refactor: SanitizedTransaction::try_create optionally computes hash
* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
f05ac7a899
move to get_slots_in_epoch ( #23657 )
2022-03-14 22:47:45 -05:00
102dd68a03
Rename AccountsDb plugins to Geyser plugins ( #23604 )
2022-03-14 19:18:46 -06:00
c68c0e881e
consolidate use-jit flag ( #23652 )
2022-03-14 15:00:00 -07:00
0c684721d8
Separate remotely downloaded snapshot archives ( #23510 )
...
* seperate remotely downloaded snapshot archives
* add str const for snapshot download dir
* only walk remote sub directory
* move directory creation outside of loop
* move is_remote to traits
* clippy simplify
* clippy
* clippy
* add unittest
* fix local cluster tests
* look for remote snapshot archive in remote foler
* create remote dir in tests
* use snapshot download dir constant
* extract build_remote_dir fn
* fix build
* code review - walking snapshot archived dirs explicitly
* fix build
* fix build
* fix comments
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* clippy
* borrow to avoid copy
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-14 14:03:59 -05:00
a3242b6b86
remove unnecessary namespaces ( #23646 )
2022-03-14 12:28:55 -05:00
11be3fb7fa
Add FlushGuard to ensure `flushing_active` is used safely ( #23571 )
2022-03-14 10:35:00 -05:00
e8a8f4e9e2
fix filler acct slots in epoch ( #23536 )
2022-03-14 09:57:44 -05:00
7758c32035
Banking Stage drops transactions that'll exceed the total account data size limit ( #23537 )
2022-03-13 15:58:57 +00:00
ccff006948
Add more logs to `purge_old_snapshot_archives` ( #23531 )
2022-03-11 15:01:48 -06:00
3c6840050c
Ensure blocks do not exceed the max accounts data size during Replay Stage ( #23422 )
2022-03-10 10:24:31 -06:00
17b00ad3a4
Add quic-client module ( #23166 )
...
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
2022-03-09 21:33:05 -05:00
1fe0d6eeeb
Set ordering flushing_active.swap() to AcqRel ( #23567 )
2022-03-10 01:22:42 +00:00
1eddb6d1e9
Move ArchiveFormat into own module ( #23562 )
2022-03-09 16:09:34 -06:00
9bbccbe27c
Use `AsRef<Path>` instead of `PathBuf` for parameters ( #23560 )
2022-03-09 16:08:33 -06:00
ba54b30101
clippy ( #23563 )
2022-03-09 15:33:50 -06:00
5a0cd05866
Revert "- estimate a program cost as 2 standard deviation above mean"
...
This reverts commit a25ac1c988
2022-03-08 17:18:44 -08:00
9acbfa5eb1
Revert "use EMA in place of Welford"
...
This reverts commit 6587dbfa47
2022-03-08 17:18:44 -08:00
c878c9e2cb
Revert "1. Persist to blockstore less frequently;"
...
This reverts commit 7aa1fb4e24
2022-03-08 17:18:44 -08:00
cc4d75a16f
fix typo ( #23535 )
2022-03-08 18:28:00 -06:00
5599bd9442
Remove BankFromArchiveTimings from ledger/
2022-03-08 08:11:50 -08:00
3114c199bd
Add RPC support for versioned transactions ( #22530 )
...
* Add RPC support for versioned transactions
* fix doc tests
* Add rpc test for versioned txs
* Switch to preflight bank
2022-03-08 15:20:34 +08:00
ddbf5c782f
Fix broken doc links in solana-runtime ( #23504 )
2022-03-07 22:21:40 +00:00
181fffb916
rename status filename to be consistent ( #23501 )
2022-03-07 17:34:35 +00:00
463cd564cf
fix typos ( #23495 )
...
* fix typos
2022-03-05 20:46:46 -06:00
38db1dead4
Refactor RPC tests ( #23483 )
2022-03-04 18:20:11 +08:00
b28acd2d4d
Reduce fn visibility
2022-03-04 09:52:46 +01:00
7f608965ef
new counter data point for unexpected rent_epoch ( #23449 )
2022-03-03 09:09:31 -06:00
ddfd4f86f3
exit early from acct bg svc if no shrink candidates ( #23459 )
2022-03-03 08:47:39 -06:00
a99fd09c16
allow index update to change storage slot # ( #23311 )
2022-03-03 08:40:48 -06:00
634f4eb37d
(LedgerStore) Use different path for different blockstore storage type. ( #23236 )
...
#### Summary of Changes
To avoid mixing the use of different shred storage types, each shred storage type
will have its blockstore in a different directory.
This PR still keeps the RocksFifo setting hidden. The default ShredStorageType and
blockstore directory are still RocksLevel and `rocksdb`.
Will follow-up with PRs on making FIFO option public in ledger-tool and validator.
#### Test Plan
* Added a new test to verify the existence of `rocksdb-fifo` directory when FIFO compaction is used.
* Updated existing test to verify the current setting still store ledger under `rocksdb` directory.
* Manually ran ledger_cleanup_test with both level and fifo compaction and verified the resulting ledger.
* Ran a validator with this PR.
2022-03-02 18:30:22 -08:00
97d40ba3da
Resized accounts must be rent exempt
2022-03-02 13:02:02 -08:00
e88da2ec0a
minor test cleanup ( #23447 )
...
* minor test cleanup
* fmt
2022-03-02 12:10:05 -06:00
e630eb73d7
rework test to avoid multi-epoch gap ( #23393 )
2022-03-02 11:12:10 -06:00
ef8b7d9c62
ledger tool halt at slot verify hash ( #23424 )
2022-03-02 11:11:18 -06:00
d909b7c80b
log hash and lamport result of calculate_accounts_hash_without_index ( #23425 )
2022-03-02 11:10:40 -06:00
7b7fdb42d9
minor test refactoring ( #23423 )
2022-03-02 08:31:14 -06:00
c8cb940b4e
tweak 2 rent tests to narrow epoch ranges ( #23420 )
2022-03-02 08:22:33 -06:00
454e82683e
refactor rent test ( #23392 )
2022-03-01 08:47:09 -06:00
6b2683f7da
refactor test ( #23384 )
2022-03-01 08:22:10 -06:00
19448ba078
Allow sub-rent-exempt-minimum transfers to `1nc1nerator` ( #23382 )
...
* Add failing test
* Allow small burns to incinerator
* Use check_id method
2022-02-28 23:56:34 -07:00
f0a235d16f
add comment ( #23378 )
2022-02-28 15:18:42 -06:00
4eeb9f4648
acct idx comments ( #23377 )
2022-02-28 15:00:53 -06:00
611d745241
refactor rent_due for normal case of exempt ( #23350 )
2022-02-28 09:42:42 -06:00
30dafc7135
list -> slot_list ( #23355 )
2022-02-27 23:08:58 -06:00
36484f4f08
Prevent new RentPaying state created by paying fees ( #23358 )
...
* Add failing test
* Check fee-payer rent-state change on load
* Add more test cases
* Review comments
2022-02-26 12:10:01 -07:00
0ad4757159
plumbing for 'other_slot' in 'update_index' ( #23330 )
2022-02-25 12:58:08 -06:00
5e0086c1ee
followup safety checks for #23295
2022-02-24 17:50:58 -07:00
2b0f16e7c3
slot per account in StorableAccounts ( #23313 )
2022-02-24 10:34:38 -06:00
017170c99d
DiskIdx: new items are not dirty by default ( #23123 )
2022-02-24 10:17:35 -06:00
c0d0724be0
clippy ( #23310 )
2022-02-23 18:07:00 -06:00
7ee549e5ae
fix params for store_accounts_frozen ( #23312 )
2022-02-23 17:40:51 -06:00
227df52213
update comment ( #23314 )
2022-02-23 17:33:06 -06:00
cafc18c3f9
update_index uses ReadableAccount to reduce params ( #23305 )
2022-02-23 17:01:23 -06:00
a245efe83d
trait for acct data with slot per item ( #23285 )
2022-02-23 15:22:29 -06:00
c81dd602c4
Hack fix for ICE as seen in CI
2022-02-23 11:17:39 -07:00
2996f1f783
add comments ( #23275 )
2022-02-23 10:53:11 -06:00
d0e85c293f
Fix rustfmt check ( #23296 )
2022-02-23 16:38:53 +08:00
20d031e2b8
Refactor ExecuteTimings w/ enum-indexed array ( #23085 )
2022-02-22 14:46:56 -08:00
88e1192c6b
Refactor new() and default() for CostTracker ( #23267 )
2022-02-22 10:44:23 -06:00
8b32e80ee2
DiskIdx: rename 'remove' to 'evict' ( #23188 )
2022-02-22 09:40:25 -06:00
7ebf398ed7
AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB" ( #23194 )
...
* AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB"
* ignore env var when starting as validator
* Update runtime/src/bucket_map_holder.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-02-22 09:40:12 -06:00
bcda74f42f
Fix builtin handling on epoch boundaries ( #23256 )
2022-02-22 20:54:08 +08:00
72c68695b5
Do not keep oldest full snapshot
2022-02-21 17:10:31 -07:00
7e08ae1d0c
Revert "Add simulation detection countermeasure ( #22880 )" ( #23261 )
...
This reverts commit c42b80f099
2022-02-21 21:15:37 +00:00
04d23a1597
fix memory ordering in append_vec ( #23215 )
2022-02-20 20:30:49 -06:00
70ebab2c82
Add rustfmt.toml and `cargo fmt` ( #23238 )
...
* fmt
* formatted
Co-authored-by: Lucas B <buffalu@jito.network>
2022-02-19 13:32:29 +08:00
1719d2349f
Skip adding builtins if they will be removed ( #23233 )
...
* Add failing test for precompile transition
* Skip adding builtins if they will be removed
* cargo clean
* nits
* fix abi check
* remove workaround
Co-authored-by: Jack May <jack@solana.com>
2022-02-18 18:36:59 -08:00
ee7e411d68
Replaces `KeyedAccount` by `BorrowedAccount` in `system_instruction_processor`. ( #23217 )
...
* Adds InstructionContext::check_number_of_instruction_accounts() and InstructionContext::get_instruction_account_key().
Bases check_sysvar_account() on instuction account indices.
* Adds instruction_account_indices enums to system_instruction_processor.
* Reorders parameters and adds InstructionContext.
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 1).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 2).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 3).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 4).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 5).
* Code cleanup
2022-02-19 02:06:33 +01:00
970f543ef6
Precompiles owned by the native loader ( #23237 )
2022-02-18 15:41:59 -08:00
1a68f81f89
Replaces `KeyedAccount` by `BorrowedAccount` in `nonce_keyed_account`. ( #23214 )
...
* Adds get_sysvar_with_account_check2 for ABIv2.
* Replaces get_signers() and get_sysvar_with_account_check() in system_instruction_processor.
* Replaces KeyedAccount by BorrowedAccount in nonce_keyed_account.
2022-02-17 17:36:55 +01:00
da00b39f4f
Cleanup: get_program_key() and get_loader_key() in TransactionContext ( #23191 )
...
* Moves TransactionContext::get_program_key() to InstructionContext::get_program_key().
* Removes TransactionContext::get_loader_key().
* Test full program and loader executable account chain in BPF loader.
2022-02-17 10:16:28 +01:00
a102453bae
accounts_index: Add SPL Token account indexing for token-2022 accounts ( #23043 )
2022-02-16 16:23:25 -08:00
Jeff Washington (jwash)
sakridge
Justin Starry
apfitzge
carllin
Trent Nelson
Tyera Eulberg
steviez
Wei Hu
Christian Kamm
buffalu
Brooks Prumo
Alexander Meißner
Michael Vines
Jack May
Jon Cinque
HaoranYi
Tao Zhu
Justin Starry
Ashwin Sekar
Ryo Onodera
Jeff Washington (jwash)
Jason
Pankaj Garg
DimAn
behzad nouri
Jae-Heon Ji
Dan Albert
Steven Luscher
Jason Davis
Tao Zhu
Giorgio Gambino
Dmitri Makarov
ryleung-solana
Brian Anderson
Yueh-Hsuan Chiang
Gavin Chan
Brooks Prumo