Brooks Prumo
d35c1a7d04
Use BankStatusCache in bank.rs ( #26114 )
2022-06-21 23:16:16 +00:00
Jeff Washington (jwash)
43e0d29b18
Revert "fix double count of account index del stats ( #25797 )" ( #26103 )
...
This reverts commit 05cb25d8da
.
2022-06-21 14:05:26 -05:00
Jeff Washington (jwash)
7a3dc3a57b
remove unused param ( #26096 )
2022-06-21 13:10:41 -05:00
Boqin Qin(秦 伯钦)
95ea506d75
runtime: fix double-readlock in in_mem_accounts_index.rs ( #26047 )
2022-06-20 15:44:19 +01:00
Tyera Eulberg
752c851070
Clean up feature preventing new rent-paying accounts ( #26000 )
...
* Clean up require_rent_exempt_accounts feature
* Fix tests in bank_client, system_instruction_processor
* Fix tests in snapshot_utils
* Fix bank tests
* Ignore rent tests
* Fix stake-accounts tests
2022-06-16 15:35:25 -06:00
Alexander Meißner
bf9ca9827e
Refactor: instruction account index ( #25825 )
...
* Adds methods based on instruction_account_index to InstructionContext.
Removes methods which are based on index_in_instruction.
* Adjusts program-runtime.
* Adjusts runtime.
* Adjusts bpf loader.
* Adjusts built-in programs.
* Adjusts program-test and bpf tests.
2022-06-16 18:46:17 +02:00
Brooks Prumo
54675b2dd6
Add bank tests for accounts_data_size and resize transactions / rent collection ( #25884 )
2022-06-15 19:40:43 -05:00
Jeff Washington (jwash)
b02c412d5b
generate_index straight to disk and batched ( #25947 )
...
* generate_index straight to disk and batched
* renames and comments
* handle in-mem case correctly
* use mutex
2022-06-15 18:14:39 -05:00
Jeff Washington (jwash)
631ea93259
parallel rent collection avoids overlapping ranges ( #25991 )
2022-06-15 16:04:38 -05:00
Tyera Eulberg
99cf1e280e
Clean up warp-timestamp features ( #25994 )
2022-06-15 14:11:41 -06:00
Jeff Washington (jwash)
80f90d6e96
disable logging per thread on rent collection load ( #25982 )
2022-06-15 13:16:00 -05:00
apfitzge
e1efc5bebb
append_vec.rs: remove new_empty_map (unused) ( #25885 )
2022-06-15 08:59:46 -05:00
Jeff Washington (jwash)
ae37359b6b
AcctIdx: only advance age on thread 0 ( #25943 )
2022-06-14 21:43:42 -05:00
Alexander Meißner
4652377e5b
Removes the feature gate of reject_empty_instruction_without_program. ( #25870 )
2022-06-14 18:00:12 +02:00
Jeff Washington (jwash)
8c6455022f
remove dead code marker ( #25945 )
2022-06-14 10:47:26 -05:00
Jeff Washington (jwash)
e24cc537a4
avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink ( #25936 )
2022-06-14 10:10:44 -05:00
Jeff Washington (jwash)
01f41b8c76
get rid of dead code marker ( #25944 )
2022-06-14 08:44:40 -05:00
Steven Luscher
45d11f3d26
Implement `VoteInstruction::AuthorizeWithSeed` & `VoteInstruction::AuthorizeWithSeedChecked` ( #25928 )
...
* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeWithSeed`
* [vote_authorize_with_seed] You can now update a vote account's authority if it's a derived key for which you control the base key
* [vote_authorize_with_seed] Add test helper to create a vote account whose authorities are derived keys
* [vote_authorize_with_seed] Write tests to assert the behavior of `VoteInstruction::AuthorizeWithSeed`
* [vote_authorize_with_seed] Feature gate the `VoteInstruction::AuthorizeWithSeed` processor
* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeWithSeed` to transaction status parser
* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeWithSeed` to docs
* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeCheckedWithSeed`
* [vote_authorize_with_seed] You can now update a vote account's authority (while checking that the new authority has signed) if it's a derived
key for which you control the base key
* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeCheckedWithSeed` to transaction status parser
* [vote_authorize_with_seed] Write tests to assert the behavior of `VoteInstruction::AuthorizeCheckedWithSeed`
2022-06-13 20:36:44 -07:00
Jeff Washington (jwash)
306faed3a8
update comment ( #25949 )
2022-06-13 17:43:29 -05:00
apfitzge
d56a706b20
Add static vector to get all static ids in runtime ( #25909 )
...
* Add function to get all static ids in runtime
* static_ids.rs: use a lazy_static ref instead of a function
2022-06-13 13:21:16 -05:00
apfitzge
3e1fd4dad0
snapshot_utils.rs: remove unnecessary sort ( #25893 )
2022-06-13 11:12:33 -05:00
apfitzge
04969a2282
serialize snapshots dir first ( #25889 )
2022-06-13 09:33:32 -05:00
Jeff Washington (jwash)
3eff7648ff
collect rent from multiple partitions in parallel ( #25774 )
2022-06-13 07:53:49 -05:00
Michael Vines
b4237f3f2c
Do not exclude failed simple vote transactions from consensus
2022-06-12 22:11:23 -07:00
Brooks Prumo
a2c180dc0b
Add EvictionsGuard to InMemAccountsIndex ( #25847 )
2022-06-12 17:31:58 -05:00
Brooks Prumo
53695ecd21
Remove SlotT type alias ( #25908 )
2022-06-10 22:04:56 -05:00
Jack May
9fb0e76dc2
cleanup feature; do_support_realloc ( #25882 )
2022-06-10 15:33:19 -07:00
apfitzge
0fffccb4a3
account_info.rs: docs offset is reduced_offset * ALIGN_BOUNDARY_OFFSET ( #25848 )
2022-06-10 16:19:53 -05:00
ivandzen
2251aa2809
Report transaction signature on update account ( #25726 )
...
There's no direct linkage between account-update events and transaction-events
Issue is linked with geyser plugin interface
Currently, there's no straightforward way to determine what notify_transaction call is related to particular update_account call. So, there's no simple way to determine what transaction caused what changes in accounts. This issue is especially valuable in case when several transactions modifies single account in the same slot.
Add txn_signature field to ReplicaTransactionInfo structure
This additional field will contain the signature of transaction which caused this account update event. This modification is not bringing significant overhead because all necessary information for such linkage already available inside Accounts::collect_accounts_to_store method
2022-06-10 10:24:35 -07:00
Brooks Prumo
bea35d5fbe
Add ApplyFeatureActivationsCaller enum ( #25858 )
2022-06-10 10:15:13 -05:00
Brooks Prumo
fedd3f26d0
Fix new cost tracker with accounts data size limit ( #25843 )
2022-06-10 07:39:43 -05:00
behzad nouri
b4190319a1
feat(nonce): adds system instruction to upgrade legacy nonce versions ( #25789 )
...
https://github.com/solana-labs/solana/pull/25788
permanently disables durable transactions with legacy nonce versions
which are within chain blockhash domain.
This commit adds a new system instruction for a one-time idempotent
upgrade of legacy nonce accounts in order to bump them out of chain
blockhash domain.
2022-06-10 00:04:29 +00:00
behzad nouri
3c1ce3cc93
permanently disables durable nonces with chain blockhash domain ( #25788 )
...
https://github.com/solana-labs/solana/pull/25744
separated durable nonce and blockhash domains, which will stop double
execution going forward. However it is possible that a durable
transaction has *already* been executed once as a normal transaction and
it is now a valid durable transaction. #25744 cannot stop such
transactions to be re-executed until the nonce accounts are advanced.
This commit adds a new nonce version indicating that the nonce is moved
out of the blockhash domain, and permanently disables durable
transactions for legacy nonces which are in the blockhash domain.
2022-06-09 15:28:37 +00:00
Brooks Prumo
e5f36aa371
Change `WaitableCondvar`'s mutex to hold `()` ( #25857 )
2022-06-09 09:03:22 -05:00
Justin Starry
6a7edc02c6
Reject durable nonce txs that don't use an advanceable nonce ( #25832 )
...
* Reject durable nonce txs that use the latest durable nonce hash
* feedback
2022-06-08 21:34:57 +00:00
Justin Starry
b2b426d4bf
Reject durable nonce transactions not signed by authority ( #25831 )
2022-06-08 14:43:09 -05:00
HaoranYi
40b1655eb5
fix intermittent bucket age test failure ( #25841 )
...
* fix double count of account index del stats
* refactor to seperate delete and insert account index stats
* fix test
* increase test timing tolerance for bucket age
* Revert "increase test timing tolerance for bucket age"
This reverts commit 2b91cf902208e4585d25dff1d4298d412c9da516.
* Revert "Revert "increase test timing tolerance for bucket age""
This reverts commit 8f07367b64ddc45a9a693729ea2725895627d14c.
2022-06-08 16:15:49 +00:00
HaoranYi
28142f8a6d
Refactor: separate delete and insert account index stats ( #25800 )
...
* fix double count of account index del stats
* refactor to seperate delete and insert account index stats
* fix test
* increase test timing tolerance for bucket age
* Revert "increase test timing tolerance for bucket age"
This reverts commit 2b91cf902208e4585d25dff1d4298d412c9da516.
2022-06-08 09:05:37 -05:00
Brooks Prumo
d4e4871c47
Add tests to Bank for accounts data size ( #25816 )
2022-06-07 18:28:53 +00:00
HaoranYi
05cb25d8da
fix double count of account index del stats ( #25797 )
2022-06-07 11:50:57 -05:00
apfitzge
e6c21a3036
Convert Measure::this to measure! and remove Measure::this ( #25776 )
...
* Remove the args param from Measure::this since we don't ever use it
* banking_stage.rs: convert to measure!
* poh_recorder.rs: convert to measure!
* cost_update_service.rs: convert to measure!
* poh_service.rs: convert to measure!
* bank.rs: convert to measure!
* measure.rs: Remove Measure::this now that all have been converted to measure!
2022-06-06 20:21:05 -05:00
Ashwin Sekar
8caced68ce
Serialize lamports per signature ( #25364 )
...
* Serialize lamports per signature
* Add full snapshot archive test, enable features in previous tests
2022-06-06 13:07:02 -07:00
Brooks Prumo
ec64d5261f
Avoid RMWs on shared data inside parallel loops: collect_rent_from_accounts() ( #25790 )
2022-06-06 13:32:22 -05:00
Justin Starry
61ad8fcc55
Fix bank executor stats and remove copy-on-write semantics ( #25621 )
...
* Fix bank executor stats and remove copy-on-write semantics
* Remove clone implementation for CachedExecutors
* feedback
2022-06-04 13:54:26 +00:00
behzad nouri
9851774133
adds feature gate enabling durable nonce
...
Previous commit separates durable nonce and blockhash domains with a
feature gate. A 2nd feature added in this commit enables durable nonce
at least one epoch after the 1st feature.
By the time 2nd feature is activated, some nonce accounts will have an
old blockhash, but no nonce account can have a recent blockhash.
As a result no transaction (durable or normal) can be executed twice.
2022-06-04 12:39:31 +00:00
behzad nouri
5ee157f43d
separates durable nonce and blockhash domains
...
AdvanceNonceAccount instruction updates nonce to blockhash. This makes it
possible that a durable transaction is executed twice both as a normal
transaction and a nonce transaction if it uses blockhash (as opposed to nonce)
for its recent_blockhash field.
The commit prevents this double execution by separating nonce and blockhash
domains; when advancing nonce account, blockhash is hashed with a fixed string.
As a result a blockhash cannot be a valid nonce value; and if transaction was
once executed as a normal transaction it cannot be re-executed as a durable
transaction again and vice-versa.
2022-06-04 12:39:31 +00:00
Jeff Washington (jwash)
8b0d5b2485
parallelize rent collection ( #25767 )
2022-06-03 18:00:02 -05:00
Brooks Prumo
edebd7e495
Grab read-lock instead of write-lock in From<PreAllocatedAccountMapEntry<T>> for (Slot, T) ( #25770 )
2022-06-03 20:48:32 +00:00
Christian Kamm
0da0e0da63
Bank: Batch account stores in collect_rent_eagerly ( #25707 )
...
* Bank: Batch account stores in collect_rent_eagerly
Previously store() calls were done one-by-one, which leads to suboptimal
performance.
* Accounts: Remove store_slow_cached()
* clippy
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2022-06-03 15:00:39 -05:00
Jeff Washington (jwash)
646bd2e488
extract prefix_from_pubkey ( #25766 )
2022-06-03 14:03:15 -05:00
Jeff Washington (jwash)
a9a94166ef
load_account_into_read_cache ( #25760 )
2022-06-03 13:34:17 -05:00
sakridge
e6e2058758
Increase accounts stack size to 8mb ( #25592 )
2022-06-03 19:46:01 +02:00
Jeff Washington (jwash)
779ec564e4
add data to assert ( #25749 )
2022-06-03 09:32:43 -05:00
Jeff Washington (jwash)
7c95ae341d
disk idx: reduce lock calls ( #25734 )
...
* disk idx: reduce lock calls
* adjust ordering for count_buckets_flushed
2022-06-02 18:26:08 -05:00
Justin Starry
6fc057c1d9
Remove activated evict_invalid_stakes_cache_entries feature ( #25739 )
2022-06-02 21:04:54 +00:00
apfitzge
ed68c0b889
epoch_stakes_reation -> epoch_stakes_creation in Bank::_new_from_parent ( #25660 )
2022-06-02 12:26:01 -05:00
carllin
eeec3fb9c9
Remove unwrap ( #25538 )
2022-06-02 12:18:46 -05:00
Jeff Washington (jwash)
e8df90c34b
rename count_buckets_flushed ( #25733 )
2022-06-02 12:16:01 -05:00
Jeff Washington (jwash)
ddd0ed0af1
fix ordering in disk idx ( #25735 )
2022-06-02 11:50:44 -05:00
Jeff Washington (jwash)
96f71c7d4d
add metrics on rent collection pieces ( #25713 )
2022-06-02 09:08:53 -05:00
Jeff Washington (jwash)
d1a6cebaa2
u64 -> duration ( #25708 )
2022-06-02 07:56:47 -05:00
apfitzge
369ff0858d
Helper functions for getting rent partitions between slots ( #25669 )
2022-06-01 20:20:44 -05:00
Jeff Washington (jwash)
149a54b786
remove par_iter on update index below threshold ( #25699 )
2022-06-01 17:41:56 -05:00
Jeff Washington (jwash)
891dcb2084
move rent collection time into datapoint ( #25714 )
2022-06-01 17:28:47 -05:00
Jeff Washington (jwash)
dd0bf4ff4b
add filler accounts gradually with cache flush ( #25678 )
2022-06-01 15:47:21 -05:00
Trent Nelson
b80a82a9e5
runtime: gate nonce disable at restart slot
2022-06-01 12:00:10 -06:00
Trent Nelson
635bd40f09
runtime: disable durable nonces
2022-06-01 11:15:16 -06:00
Jeff Washington (jwash)
905fef29cb
add filter to AppendVecScan ( #25664 )
2022-06-01 11:55:18 -05:00
Jeff Washington (jwash)
94b0ce5d43
remove_old_historical_roots uses accounts_hash_complete_one_epoch_old ( #25673 )
2022-06-01 10:45:26 -05:00
Jeff Washington (jwash)
c91fbd289f
set active state to shrink_ancient when doing intersting work ( #25677 )
2022-06-01 10:36:58 -05:00
Jeff Washington (jwash)
8bb76fcd87
add ancient metrics ( #25656 )
...
* add ancient metrics
* review feedback
2022-06-01 10:36:23 -05:00
Jeff Washington (jwash)
07958fd8ed
use self.init_accum ( #25663 )
2022-06-01 08:38:37 -05:00
Tyera Eulberg
8584a3a331
Add feature to increase timestamp bounding on the slow side to 150% of poh estimate (1s/slot) ( #25666 )
2022-05-31 16:53:06 -06:00
Jeff Washington (jwash)
5994b4f53f
use object for hash scanning ( #25418 )
2022-05-31 11:51:56 -05:00
apfitzge
56d38e4954
Add lz4 to snapshot filename regex for matching files on load ( #25633 )
...
* Add lz4 to snapshot filename regex for matching files on load
* Add tests for lz4 archive format filenames
2022-05-31 11:06:41 -05:00
steviez
6b2f58152b
Ignore rocksdb directories when unpacking genesis archive ( #25575 )
...
Unpacking the rocksdb directory from genesis archive can have unintended
consequences, such as clobbering files in pre-existing rocksdb
directory.
2022-05-31 00:38:45 -05:00
Wei Hu
4b1dd0f921
Fix a possible test hang in runtime::test_shared_buffer_sweep ( #25545 )
...
* Fix a possible test hang in runtime::test_shared_buffer_sweep
The test may hang if the total number of threads it wants to start
is greater than the number of threads configured in the rayon
threadpool. Check to avoid this situation from happening.
* Incorporate comment from Xiang Zhu, using std::cmp::min() to simplify the code
2022-05-30 23:39:32 +08:00
Christian Kamm
387d9d9c77
Store missing unupdated executors earlier
2022-05-28 13:42:35 +10:00
carllin
740ee0c502
Fix datapoint name ( #25614 )
2022-05-28 01:01:29 +00:00
Trent Nelson
4e6b276f6c
bump executor cache max entries ( #25600 )
...
* executor-cache: unify `remove` calls
* executor-cache: metrics - add one-hit-wonder counter
* executor-cache: bump max entries to 256
2022-05-28 01:54:10 +10:00
Justin Starry
ce14c63bf0
fix: stop double counting execution timings ( #25605 )
2022-05-28 00:06:29 +10:00
buffalu
e58de2c233
Check fee payer before loading all accounts ( #25371 )
...
* fix vote account loading
* fix clippy and rename some stuff
* fix bug
Co-authored-by: Justin Starry <justin@solana.com>
2022-05-27 13:44:29 +10:00
Brooks Prumo
7384c41ea7
Speed up bank::tests::test_max_accounts_data_size_exceeded() ( #25523 )
2022-05-25 22:22:30 -04:00
Justin Starry
26a02f6f59
Add invoke logging to builtin programs that don't have it yet ( #25230 )
...
* Refactor process_executor_chain
* Enable builtin program invoke logs by default
* feedback
2022-05-26 07:27:00 +08:00
Alexander Meißner
f10c80b49f
Refactor: Rebase offset of `index_in_caller` ( #25531 )
...
* Removes the offset InstructionContext::get_number_of_program_accounts() from InstructionAccount::index_in_caller.
* Removes unreachable SyscallError::InvalidLength in orig_data_lens.get().
2022-05-25 13:43:20 +02:00
buffalu
7b98ff1929
Refactor: pull out fee payer balance check ( #25519 )
2022-05-25 20:44:03 +10:00
Brooks Prumo
b7149f7f54
Add helper fn to set accounts_data_size_initial for tests ( #25522 )
2022-05-25 02:43:54 +00:00
Brooks Prumo
0dd3c6bf1f
Split up accounts data size field ( #25495 )
2022-05-24 20:26:32 -04:00
Alexander Meißner
2fb096c486
Refactor: Adds `index_in_callee` to `InstructionAccount` ( #25490 )
...
* Adds InstructionAccount::index_in_callee
* Adjusts tests and benches.
* Adds documentation for InstructionAccount.
* Adds InstructionContext::is_duplicate()
2022-05-25 00:04:46 +02:00
Justin Starry
e66ea7cb6a
Clean up Bank::commit_transactions parameters
2022-05-24 20:24:42 +08:00
Jeff Washington (jwash)
bac05dc55a
calc slot info on max slot once per hash calc ( #25422 )
2022-05-23 10:11:00 -05:00
Jeff Washington (jwash)
41f30a2383
stop logging misleading bank hash mismatch ( #25427 )
2022-05-23 08:43:25 -05:00
Michael Vines
b05c7d91ed
Fix derive_partial_eq_without_eq clippy lint
2022-05-22 22:22:21 -07:00
Brooks Prumo
f8842032c6
clippy: fix "this let-binding has unit value" warnings ( #25429 )
2022-05-22 12:17:59 -04:00
Alexander Meißner
7eae98c6d9
Introduces `Result` return types to mutable accessors of `BorrowedAccount` ( #25380 )
...
* Introduces result return types to get_data_mut(), set_data() and set_data_length() of BorrowedAccount.
* Introduces result return types to set_owner(), set_lamports() and set_executable() of BorrowedAccount.
2022-05-21 17:47:09 +02:00
Brooks Prumo
e9a6d1ecef
clippy: is_digit(10) -> is_ascii_digit() ( #25428 )
2022-05-21 07:03:11 -05:00
Jeff Washington (jwash)
f584b249dd
special case for common case of 1 storage on append vec scan ( #25414 )
2022-05-20 17:36:30 -05:00
Jack May
851958f77a
Disallow deployment of deprecated _sol_alloc_free syscall ( #25179 )
2022-05-20 13:19:41 -07:00
Jeff Washington (jwash)
cfcc18f7f4
apply AccountsInAppendVec to more places ( #25408 )
2022-05-20 14:25:46 -05:00
Brooks Prumo
a25212b087
Move append_vec::test_utils into its own file ( #25407 )
2022-05-20 13:37:45 -05:00
Jeff Washington (jwash)
9ef0acdf5d
allow hash calc to use all threads at startup ( #25369 )
2022-05-20 11:45:01 -05:00
Jeff Washington (jwash)
467431de89
add ledger-tool accounts-db-skip-initial-hash-calc ( #25354 )
2022-05-20 10:27:00 -05:00
Jeff Washington (jwash)
950245c29b
simulate filler account hash calc time at delta hash ( #25387 )
2022-05-20 10:20:44 -05:00
Jeff Washington (jwash)
1b4e4ef548
impl squash to ancient append vecs ( #24538 )
...
* squash to ancient append vecs
* pr feedback
* clippy
2022-05-20 10:20:17 -05:00
Jeff Washington (jwash)
de03a3d3aa
filler accounts get a non-zero rent_epoch ( #25386 )
...
* fill accounts get a non-zero rent_epoch
* use snapshot_slot
* get rid of perf simulation code
2022-05-20 09:03:55 -05:00
Jeff Washington (jwash)
3fe7655378
use iterator instead of vec to iterate accounts ( #25370 )
...
* user iterator instead of vec to iterate accounts
* rename
2022-05-20 08:50:24 -05:00
Jeff Washington (jwash)
36ed36edf0
use previous epoch's # slots per epoch ( #25382 )
2022-05-20 08:47:55 -05:00
Michael Vines
c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature ( #25291 )
2022-05-19 18:28:46 -07:00
Brooks Prumo
021276c955
Move updating accounts data len from execute to commit ( #25256 )
2022-05-19 12:24:13 -05:00
Jeff Washington (jwash)
0f703b71cc
minor test renaming ( #25297 )
2022-05-18 12:53:55 -05:00
Jeff Washington (jwash)
fe91f40f8a
use previous epoch rent collector in expected rent collection ( #25289 )
2022-05-18 12:53:45 -05:00
Justin Starry
a1522d0024
Use consistent naming for compute unit limit ( #25229 )
...
* Use consistent naming for compute unit limit
* feedback
2022-05-18 13:14:31 +08:00
Jon Cinque
0820065c98
rpc: Support token-2022 in token-specific calls ( #25150 )
...
* rpc: Support token-2022 in token-specific calls
* Address feedback
2022-05-17 21:02:43 +02:00
HaoranYi
85a2e599bb
revert memory map for reading snapshot ( #25259 )
2022-05-17 09:32:18 -05:00
Jeff Washington (jwash)
2cf7c73ced
cleanup types in tests ( #25254 )
2022-05-16 15:54:40 -05:00
Jack May
21e066ef26
Merge pull request from GHSA-wxrh-ff9f-fc6w
2022-05-16 12:30:37 -07:00
HaoranYi
3e44046a73
Support lz4 for snapshot archives ( #25089 )
...
* add lz4
* add cargo package file
* fix tests
* use fast lz4 compression
* report snapshot archive format
* add test
* code review feedback
* add cargo.lock
* fix var name
* refactor archive format parsing and add default compress cli arg
* clippy
* add from_cli_arg test
* update cargo.lock
* add lz4 support for mmap
* cargo.lock
* clippy
2022-05-16 12:44:15 -05:00
Justin Starry
f81c5df1f0
Include account index in rent paying account tx error ( #25189 )
2022-05-16 23:35:34 +08:00
HaoranYi
389d78e424
Don't use mmarinus package ( #25177 )
...
* Revert "Revert "Use memory map to speed up snapshot untar (#24889 )" (#25174 )"
This reverts commit fc793de2dd
.
* not use mmarinus
* enable secondary build
* Revert "enable secondary build"
This reverts commit 5aa43a94bbad59497506f02235597e7adc0dc47b.
* macbuild
* Revert "macbuild"
This reverts commit 0da92945a40dc5431eff4bd91054b407e22418fb.
2022-05-16 08:49:17 -05:00
Tao Zhu
b1b3702e6d
Prioritize transactions in banking stage by their compute unit price ( #25178 )
...
* - get prioritization fee from compute_budget instruction;
- update compute_budget::process_instruction function to take instruction iter to support sanitized versioned message;
- updated runtime.md
* update transaction fee calculation for prioritization fee rate as lamports per 10K CUs
* review changes
* fix test
* fix a bpf test
* fix bpf test
* patch feedback
* fix clippy
* fix bpf test
* feedback
* rename prioritization fee rate to compute unit price
* feedback
Co-authored-by: Justin Starry <justin@solana.com>
2022-05-16 12:06:33 +08:00
Justin Starry
6c10515ccf
Revert "Serialize lamports per signature in snapshots ( #25181 )" ( #25228 )
...
This reverts commit 35d2a0fd69
.
2022-05-15 12:04:17 +00:00
Ashwin Sekar
35d2a0fd69
Serialize lamports per signature in snapshots ( #25181 )
...
* Serialize lamports per signature
* pr comments
2022-05-14 10:57:51 -07:00
Ryo Onodera
592aa4c21b
Improve hard fork message with more detail ( #25214 )
2022-05-15 01:14:23 +09:00
sakridge
3d96a1ab76
Block packets in vote-only mode ( #24906 )
2022-05-14 17:53:37 +02:00
Jeff Washington (jwash)
7826fef120
clippy ( #25206 )
2022-05-13 21:03:54 -05:00
Jeff Washington (jwash)
9ccdc8a050
populate ancient append vec metrics ( #25204 )
...
* populate ancient append vec metrics
* count ancient storages while enumerating storages anyway.
2022-05-13 18:20:15 -05:00
Jeff Washington (jwash)
d7dcb62fe1
ancient_append_vec refactoring ( #25203 )
2022-05-13 16:40:26 -05:00
Jeff Washington (jwash)
cb037e49ea
is_candidate_for_shrink respects ancient append vecs ( #25205 )
2022-05-13 16:26:46 -05:00
apfitzge
5eeb07fead
coalesce purge stats and reclaim result ( #25183 )
2022-05-13 12:34:11 -05:00
Jeff Washington (jwash)
6f1cc5a2a7
assert we haven't squashed to ancient after hash calc ( #25168 )
2022-05-12 18:26:01 -05:00
Jeff Washington (jwash)
fc793de2dd
Revert "Use memory map to speed up snapshot untar ( #24889 )" ( #25174 )
...
This reverts commit 3367e44671
.
2022-05-12 14:48:29 -05:00
Jason
08da486c05
additional costs in block capacity calc ( #25059 )
...
* Added additional costs to block capacity computation, and pushed alloc of CostModel all the way to the top of the call chain, instead of reallocing
* Fix two compiler errors
* Update block processing to propagate computed costs, rather than re-computing deeper in the call stack
* Clippy fix
* Reformatting fix after merge
* Add CostModel::sum_without_bpf
2022-05-12 13:52:20 -05:00
HaoranYi
3367e44671
Use memory map to speed up snapshot untar ( #24889 )
...
* mmap
* add no_os_memory_reporting_args to ledger-tool
* use safe memory map package for unix. use regular file for windows
* fallback to untaring with filebuf when mmap fails
* update comments
2022-05-12 13:35:19 -05:00
Jack May
cde15ff687
support 64-bit prioritization fee ( #25027 )
2022-05-12 11:07:36 -07:00
Jeff Washington (jwash)
896729f25e
keep track of oldest slot used by last hash calculation ( #25152 )
2022-05-12 11:18:08 -05:00
Jeff Washington (jwash)
abb4993a3b
SortedStorages::max_slot_inclusive() ( #25153 )
2022-05-12 10:24:03 -05:00
Jeff Washington (jwash)
849cbe93aa
add AccountsToStore::has_overflow ( #25143 )
2022-05-12 10:14:27 -05:00
Jeff Washington (jwash)
c56d978cb1
minor cleanup ( #25144 )
2022-05-11 14:58:43 -05:00
Justin Starry
68800982ea
Add size_of const fns for upgradeable loader states ( #25131 )
2022-05-11 22:22:59 +08:00
Jeff Washington (jwash)
a8930ee14b
add --accounts-db-ancient-append-vecs ( #25125 )
2022-05-11 08:47:07 -05:00
Jon Cinque
542a14d9b7
test-validator: Always use overridden compute budget in bank ( #25128 )
2022-05-11 15:46:51 +02:00
Jeff Washington (jwash)
11fa0db850
more ancient append vec fns ( #25127 )
2022-05-11 08:02:05 -05:00
Jeff Washington (jwash)
3ce877d1ab
add metric for ancient append vec count ( #25126 )
2022-05-11 08:01:04 -05:00
Jeff Washington (jwash)
c4506269a6
add ancient_append_vecs ( #25119 )
2022-05-10 16:41:04 -05:00
Pankaj Garg
c838e15234
Unset needs_unlock for rebatched transactions batches ( #25095 )
...
* Unset needs_unlock for rebatched transactions batches
* address review comments
2022-05-10 13:39:08 -07:00
DimAn
2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations ( #24247 )
2022-05-10 16:37:41 -04:00
Jeff Washington (jwash)
dc6e28099b
mark_dirty_dead_stores ( #25114 )
2022-05-10 15:08:29 -05:00
Jeff Washington (jwash)
e9e267947e
simplify storage_finder -> storage ( #25109 )
2022-05-10 14:18:51 -05:00
Jeff Washington (jwash)
e9a9604ca7
avoid zeroing out root stats with empty data ( #25111 )
2022-05-10 13:56:30 -05:00
Jeff Washington (jwash)
b91cc87680
assert when rewrites skipped ( #25110 )
2022-05-10 13:55:48 -05:00
Jeff Washington (jwash)
8e5bb0a1e2
check file size of accounts cache mmap before indexing ( #25077 )
2022-05-10 13:33:22 -05:00
Jeff Washington (jwash)
2d7ce2a6c2
Revert "uses designated thread-pools for account db parallel ops ( #24954 )" ( #25053 )
...
This reverts commit e8bdc27080
.
2022-05-06 15:20:50 -05:00
Justin Starry
082502d4f3
Fail tx sanitization when ix program id uses lookup table ( #25035 )
...
* Fail tx sanitization when ix program id uses lookup table
* feedback
2022-05-07 03:19:50 +08:00
Jeff Washington (jwash)
df9a4a0c9d
Bank::new_for_benches uses bench config ( #25017 )
2022-05-06 13:35:18 -05:00
Jeff Washington (jwash)
7401bc0fa5
remove_old_historical_roots uses correct epoch schedule ( #25020 )
...
* remove_old_historical_roots uses correct epoch schedule
* Update runtime/src/accounts_db.rs
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
2022-05-06 12:38:09 -05:00
Jason
47cd0d9061
Speed up get-data-cost function ( #25013 )
...
* Combine data_bytes_cost into the get_transaction_cost function, for speedup
* Remove a dead function, fix a value in a test
* Respond to comments by changing call signature to take a &mut struct rather than return a tuple
2022-05-06 11:39:00 -05:00
behzad nouri
492f89a170
checks account owner when initializing a vote-account ( #25018 )
...
A VoteAccount may only wrap an account if the account owner is
solana_vote_program:id or equivalently this check returns true:
solana_vote_program::check_id(account.owner())
2022-05-06 16:22:49 +00:00
behzad nouri
3fff34a65f
explicitly removes accounts with zero lamports from stakes cache ( #25015 )
...
Zero lamport accounts are not stored in accounts-db:
https://github.com/solana-labs/solana/blob/7b5835ddf/runtime/src/accounts_db.rs#L5007-L5014
https://github.com/solana-labs/solana/blob/7b5835ddf/runtime/src/accounts_db.rs#L4481-L4483
However, in order to purge the account from cache, stakes cache update
partially relies on accounts data to be also zeroed out :
https://github.com/solana-labs/solana/blob/7b5835ddf/runtime/src/bank.rs#L3471-L3478
This can be error-prone and introduce inconsistency between accounts-db
and vote/stake cache. This commit instead explicitly removes accounts
from cache if lamports == 0.
2022-05-06 14:04:49 +00:00
Jeff Washington (jwash)
0cc97689f9
add stats for stakes_cache_check_and_store_us ( #24349 )
...
* add stats for stakes_cache_check_and_store_us
* remove redundant metrics
2022-05-05 17:22:48 -05:00
Jeff Washington (jwash)
96259b4f23
when testing hash calc, also verify cap ( #25012 )
2022-05-05 15:16:15 -05:00
Jeff Washington (jwash)
e0092902ea
maybe_update_rent_epoch_on_load uses correct EpochSchedule ( #24999 )
2022-05-05 13:34:03 -05:00
behzad nouri
7b5835ddfa
removes manual implementations of serde Deserialize from vote_account ( #25006 )
...
Container attribute #[serde(from = "...")] already achieves desired
implementation.
2022-05-05 16:33:10 +00:00
Jae-Heon Ji
f6e8e16691
chore: update function name ( #24166 )
2022-05-05 10:27:14 -05:00
behzad nouri
e8bdc27080
uses designated thread-pools for account db parallel ops ( #24954 )
...
Instead of global rayon thread-pool, the commit uses
AccountsDb::{thread_pool,thread_pool_clean} for parallel ops in
AccountsDB::{update_index,clean_accounts_older_than_root}.
2022-05-04 22:47:12 +00:00
Jeff Washington (jwash)
e8bf812a8a
allow ledger-tool halt at slot to calc hash using write cache ( #24974 )
2022-05-04 16:27:49 -05:00
Jeff Washington (jwash)
40986daddf
log write cache contents from purge_slots_from_cache_and_store ( #24948 )
2022-05-04 14:28:47 -05:00
apfitzge
0ca54db524
use bank's epoch_schedule accessor internally ( #24918 )
2022-05-04 13:52:48 -05:00
Alexander Meißner
eae9a66531
Record instructions which are precompiles ( #24743 )
...
* Record instructions which are "is_precompile".
2022-05-04 16:32:14 +02:00
Jack May
e070c5ca38
default compute units per instruction ( #24899 )
2022-05-03 09:50:06 -07:00
HaoranYi
fd44198c0a
fix typo ( #24937 )
2022-05-03 09:57:15 -05:00
Jeff Washington (jwash)
755100dd7c
use correct epoch schedule ( #24904 )
2022-05-03 09:21:47 -05:00
apfitzge
112a0b475a
Revert "Refactor to use EpochSchedule from within RentCollector struct" ( #24893 )
...
* Revert "Ran cargo fmt"
This reverts commit 9052e41b32
.
* Revert "Fix build error introduced by my editor setup, part 2"
This reverts commit 4dfeab3b38
.
* Revert "Fix build error introduced by my editor setup"
This reverts commit 87fb78dc56
.
* Revert "Remove redundant epoch_schedule from AccountsPackage"
This reverts commit c2f7f2fff8
.
* Revert "Fix a test"
This reverts commit 36c0bdaa78
.
* Revert "Fixes to initial code"
This reverts commit ed7813e698
.
* Revert "Removing redundant EpochSchedule param from fns"
This reverts commit 5472d2e605
.
2022-05-02 13:46:17 -05:00
Jeff Washington (jwash)
546730cf42
retire deprecate_rewards_sysvar ( #24651 )
...
* retire deprecate_rewards_sysvar
* Update expected hash for bank after eliminating reqards sysvar
* Hack test_bank_update_vote_stake_rewards to pass after removing sysvar rewards account
* formatting change
* formatting cleanup
* Remove legacy validator Rewards code from unit test. Change expected inflation computation
* Update bank.rs
* Update bank.rs
* Clean up inflation abstraction function
* Formatting cleanup
* rename vars. Add struct for inflation rewards return
* format cleanup + comment
* Fix format warnings
Co-authored-by: Brennan Watt <brennan.watt@solana.com>
2022-05-02 10:10:51 -05:00
Michael Vines
74b586ae71
Always retain the highest incremental snapshot for all full snapshots
2022-05-02 07:37:55 -07:00
Brooks Prumo
a73f99823d
Do not hold lock unnecessarily when hashing ( #24815 )
2022-04-29 08:40:28 +00:00
Dan Albert
ad7c95b8a5
Update non-circulating supply list ( #24817 )
...
Co-authored-by: publish-docs.sh <maintainers@solana.com>
2022-04-29 02:57:54 +00:00
Jon Cinque
b4503d4110
token: Add features for new program releases ( #24742 )
...
* token: Add features for new program releases
* Add PR number
2022-04-28 23:43:57 +02:00
Justin Starry
4e58b3870c
Update all BankForks methods to return owned values ( #24801 )
2022-04-28 18:51:00 +00:00
Tyera Eulberg
e189603d17
Make test less brittle ( #24803 )
2022-04-28 12:09:01 -06:00
Brooks Prumo
7b6880f652
Make test_bank_cloned_stake_delegations() aware of stake minimum delegation ( #24753 )
2022-04-28 09:35:07 -05:00
Jeff Washington (jwash)
f22738fd7b
fix test_rent_eager_collect_rent_in_partition ( #24722 )
2022-04-26 23:06:57 -05:00
Ryo Onodera
412a5a0d33
Credits auto rewind on vote recreation ( #22546 )
...
* Credits auto rewind on vote recreation
* Update comment
* Improve comments and tests
* Recommended fn rename
* Restore old feature, and replace new feature key
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-04-26 15:49:35 -06:00
Steven Luscher
3007f233f4
Log subscription for `All` no longer clobbers `Some` subscription for pubkey ( #24215 )
...
* fix: subscribing for all logs no longer clobbers pubkey-specific subscriptions
* test: write tests to cover logsSubscribe
2022-04-26 10:31:11 -07:00
Jeff Washington (jwash)
534a666153
hookup 'just_rewrites' ( #24662 )
2022-04-26 12:14:25 -05:00
Justin Starry
b6bc5d2fa1
Aggregate executor update timing metrics
2022-04-26 23:15:22 +08:00
Michael Vines
533eadd024
Remove StakeState::get_rent_exempt_reserve()
2022-04-25 21:43:04 -07:00
HaoranYi
9044cd7179
Check insert optimization for bank_forks ( #24641 )
...
* check insert optimization
* clippy suggested simplification
2022-04-25 19:17:21 -05:00
Jeff Washington (jwash)
099258e4eb
some cleanup ( #24568 )
2022-04-25 13:17:30 -05:00
Jeff Washington (jwash)
5231855261
add state for squash ancient ( #24552 )
2022-04-25 09:05:37 -05:00
behzad nouri
be0bdd2261
explicitly ignores struct fields which do not impact PartialEq impl ( #24624 )
...
Unless struct fields are explicitly ignored in PartialEq implementation,
there are no compile-time checks that if more fields are added to the
struct, PartialEq implementation is accordingly updated.
2022-04-24 14:33:42 +00:00
carllin
8a062273de
Move error counters to be reported by leader only at end of slot ( #24581 )
...
* Add error counters to leader metrics only
* Add dependencies
2022-04-23 18:10:47 -05:00
Jeff Washington (jwash)
8bfde0940f
cleanup ( #24598 )
...
* cleanup
* cargo
2022-04-23 08:28:49 -05:00
Jeff Washington (jwash)
14ee36a2af
cleanup in vote/stake accounts ( #24608 )
...
* cleanup in vote/stake accounts
* reorder comparisons in accounts_equal
2022-04-23 08:28:21 -05:00
Michael Vines
84e3342612
Process blockstore after starting the TVU
2022-04-22 21:17:49 -07:00
Michael Vines
83e041299a
Run real snapshot packager while processing blockstore at validator startup
2022-04-22 21:17:49 -07:00
Jeff Washington (jwash)
ce7d2964c9
metrics on ancient slots ( #24596 )
2022-04-22 15:01:51 -05:00
Jon Cinque
0d51596224
sim: Override slot hashes account on simulation bank ( #24543 )
...
* sim: Override slot hashes during simulation
* Add simulation test program
* Address feedback
* Add AccountOverrides explicit type
* Cargo fmt
2022-04-22 12:32:31 +02:00
Jeff Washington (jwash)
a0c4be8cd7
add StorableAccountsMovingSlots ( #24555 )
2022-04-21 15:48:20 -05:00
Jeff Washington (jwash)
25e9199397
cleanup FlushStats ( #24556 )
2022-04-21 14:05:01 -05:00
Jeff Washington (jwash)
3f7b8d3d2c
provide default impl of 'is_empty' ( #24554 )
2022-04-21 12:49:44 -05:00
behzad nouri
d524ae293f
collects metrics for cached vote/stake accounts consistency
...
Tracking mismatches between cached vote/stake accounts and accounts-db
in preparation of activating feature for updating rewards at
epoch-boundary using cached vote/stake accounts.
2022-04-21 15:28:41 +00:00
behzad nouri
108aa23d90
enforces type-safety in Stakes<StakeAccount> using phantom data
...
StakeAccount<Delegation> can only wrap a stake-state which is a
Delegation; whereas StakeAccount<()> wraps any account with stake state.
As a result, StakeAccount::<Delegation>::delegation() will return
Delegation instead of Option<Delegation>.
2022-04-21 15:28:41 +00:00
behzad nouri
f937fcbd95
updates rewards at epoch boundary using cached accounts
...
Loading vote and stake accounts from accounts-db takes a significant
portion of time updating rewards at epoch boundary.
This commit bypasses accounts-db and instead uses vote and stake
accounts cached in bank stakes:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/stakes.rs#L148-L152
These cached accounts are synchronized with accounts-db after each
transaction, and so there should not be any change in the resulting
computation:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/bank.rs#L4526
Nevertheless, to avoid any chances of introducing a consensus issue, the
switch to cached account is feature gated.
2022-04-21 15:28:41 +00:00
behzad nouri
b4491ff4ba
adds StakesEnum type representing Stakes<StakeAccount|Delegation>
...
For backward compatibility, we can only serialize and deserialize
Stakes<Delegation>. However Bank caches Stakes<StakeAccount>. This type
mismatch incurs a conversion cost at epoch boundary when updating
EpochStakes.
This commit adds StakesEnum which allows EpochStakes to include either a
Stakes<StakeAccount> or Stakes<Delegation> and so bypass the conversion
cost between the two at the epoch boundary.
2022-04-21 15:28:41 +00:00
behzad nouri
454ef38e43
caches StakeAccount instead of Delegation in Stakes
...
The commit makes values in stake_delegations map in Stakes struct
generic. Stakes<Delegation> is equivalent to the old code and is used
for backward compatibility in BankFieldsTo{Serialize,Deserialize}.
But banks cache Stakes<StakeAccount> which includes the entire stake
account and StakeState deserialized from account. Doing so, will remove
the need to load stake account from accounts-db when working with
stake-delegations.
2022-04-21 15:28:41 +00:00
behzad nouri
8c4d6357fe
adds StakeAccount type embodying an Account and a StakeState
...
The added type does sanity checks on the Account and stores deserialized
StakeState. Following commits will use this type instead of Delegation
in Stakes.
2022-04-21 15:28:41 +00:00
Justin Starry
d5127abf46
Only add hashes for completed blocks to recent blockhashes ( #24389 )
...
* Only add hashes for completed blocks to recent blockhashes
* feedback
2022-04-21 21:05:29 +08:00
Tao Zhu
a21fc3f303
Apply transaction actual execution units to cost_tracker ( #24311 )
...
* Pass the sum of consumed compute units to cost_tracker
* cost model tracks builtins and bpf programs separately, enabling adjust block cost by actual bpf programs execution costs
* Copied nightly-only experimental `checked_add_(un)signed` implementation to sdk
* Add function to update cost tracker with execution cost adjustment
* Review suggestion - using enum instead of struct for CommitTransactionDetails
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
* review - rename variable to distinguish accumulated_consumed_units from individual compute_units_consumed
* not to use signed integer operations
* Review - using saturating_add_assign!(), and checked_*().unwrap_or()
* Review - using Ordering enum to cmp
* replace checked_ with saturating_
* review - remove unnecessary Option<>
* Review - add function to report number of non-zero units account to metrics
2022-04-21 07:38:07 +00:00
Justin Starry
79923c3b58
Refactor: Rename BlockhashQueue fields and methods for clarity ( #24426 )
2022-04-21 11:57:17 +08:00
Jeff Washington (jwash)
52f43a986d
skip rewrites in bank ( #23985 )
2022-04-20 14:08:46 -05:00
Jeff Washington (jwash)
b1c89cf6d4
append_vec::remaining_bytes ( #24531 )
2022-04-20 13:44:57 -05:00
Jeff Washington (jwash)
813d06529d
stats for rewrites ( #24529 )
2022-04-20 10:43:50 -05:00
Jeff Washington (jwash)
05438c3cd6
implement remove_old_historical_roots ( #24492 )
2022-04-20 10:43:32 -05:00
Jeff Washington (jwash)
cdc3c41001
helpers to skip rewrites in bank ( #24507 )
2022-04-20 10:21:02 -05:00
Jeff Washington (jwash)
0d797e2fff
contains_multiple_slots ( #24500 )
2022-04-20 09:53:36 -05:00
Jeff Washington (jwash)
cfe2177e16
validator option: accounts-db-skip-rewrites ( #24504 )
2022-04-20 09:41:00 -05:00
Jeff Washington (jwash)
957849f4a5
load_to_collect_rent_eagerly returns loaded_slot ( #24506 )
2022-04-20 09:14:03 -05:00
Jeff Washington (jwash)
ef5e7cce6e
type PubkeyAccountSlot ( #24503 )
2022-04-19 21:34:00 -05:00
Jeff Washington (jwash)
4be01ec75a
remove temp dead_code markers ( #24508 )
2022-04-19 21:30:57 -05:00
Michael Vines
9e4999ef6a
Remove halt_at_slot from RuntimeConfig, it's not a runtime concern
2022-04-19 19:23:58 -07:00
Michael Vines
0e2e0c8b7d
Extract most storage-related services from the Tvu abstraction
2022-04-19 19:23:58 -07:00
Jeff Washington (jwash)
255a6a729d
add historical_roots_len stat ( #24497 )
2022-04-19 16:53:24 -05:00
Jeff Washington (jwash)
d23c04bb68
max_slot_in_storages_exclusive -> INclusive ( #24450 )
2022-04-19 13:59:41 -05:00
Tao Zhu
94b0186a96
Cost model tracks builtins and bpf programs separately ( #24468 )
...
* Cost model tracks builtins and bpf programs separatele (enables adjusting block cost by actual bpf programs execution costs)
* Address reviews: expand test; add metrics stat
2022-04-19 13:25:47 -05:00
Justin Starry
2ad1baa753
Add const fn StakeState::size_of and static assertion ( #24416 )
2022-04-20 01:04:12 +08:00
Jeff Washington (jwash)
c4923d29b4
bank_hash_at uses rewrites ( #24439 )
2022-04-19 11:29:29 -05:00
HaoranYi
4fdf966393
demote shink counter to debug ( #24458 )
2022-04-19 10:02:23 -05:00
Jeff Washington (jwash)
569b15d863
maybe_update_rent_epoch_on_load ( #24055 )
2022-04-19 08:29:36 -05:00
Jeff Washington (jwash)
dc98510d6d
accounts hash calls maybe_rehash_skipped_rewrite ( #24316 )
2022-04-19 08:29:09 -05:00
Jack May
5d1adf1270
Fix signature count ( #24471 )
...
* Fix signature count
* protect the signature count futher
2022-04-19 17:59:06 +08:00
Alexander Meißner
d6ff50bcc1
Remove `KeyedAccount` in builtin program "system instruction processor" ( #24452 )
...
* Makes sure that there is only one KeyedAccount at a time.
Moves keyed_account_at_index() into create_account(), transfer_verified(), transfer() and transfer_with_seed() to do so.
* Uses transaction_context.get_key_of_account_at_index() when only the key is needed.
* Uses "!instruction_context.is_signer()" instead of ".signer_key().is_none()".
* Replaces KeyedAccount by BorrowedAccount.
* Removes unused code.
2022-04-19 09:28:07 +02:00
Jeff Washington (jwash)
0e0f306996
mechanism for disabling hash calc caching ( #24437 )
2022-04-18 22:44:30 -05:00
Jeff Washington (jwash)
94d3ec37d2
collect_rent_eagerly can just do rewrites ( #24438 )
2022-04-18 22:44:19 -05:00
Jeff Washington (jwash)
bf1e0c89e8
rename local: max_slot_in_storages -> inclusive ( #24453 )
2022-04-18 22:42:02 -05:00
Alexander Meißner
8b854d4141
Remove `KeyedAccount` in builtin program "nonce keyed account" ( #24436 )
...
* Makes sure that there is only one KeyedAccount at a time.
Moves keyed_account_at_index() into withdraw_nonce_account() to do so.
* Replaces KeyedAccount by BorrowedAccount.
* Removes unused code.
2022-04-18 22:19:51 +02:00
Jeff Washington (jwash)
ee384f7254
remove 'startup' from verify_snapshot_bank ( #24441 )
2022-04-18 14:35:49 -05:00
Jeff Washington (jwash)
bcc5b96bda
remove redundant EpochSchedule ( #24444 )
2022-04-18 13:04:48 -05:00
Jason Davis
9052e41b32
Ran cargo fmt
2022-04-18 11:57:40 -05:00
Jason Davis
87fb78dc56
Fix build error introduced by my editor setup
2022-04-18 11:57:40 -05:00
Jason Davis
c2f7f2fff8
Remove redundant epoch_schedule from AccountsPackage
2022-04-18 11:57:40 -05:00
Jason Davis
ed7813e698
Fixes to initial code
2022-04-18 11:57:40 -05:00
Jason Davis
5472d2e605
Removing redundant EpochSchedule param from fns
2022-04-18 11:57:40 -05:00
HaoranYi
c6a751d658
Detect and report bank drop signal queue full and disconnect events ( #24112 )
...
* nonblocking send when when droping banks
* detect and report drop signal queue full/disconnect events
* comments
* use counter for reporting bank_drop_queue events
* reduce log
* use datapoint to report stats
* logging instead of reporting bank drop signal full
* fix a corner case for reporting
* fix build
2022-04-18 11:19:11 -05:00
Tao Zhu
578d59c802
Remove the code that handles cost update for separate pr
2022-04-17 19:26:24 -05:00
Tao Zhu
85281de981
move number utility functions to sdk; log error if cost adjust overflows i64
2022-04-17 19:26:24 -05:00
Tao Zhu
9dadfb2e2c
Add checked_add_signed() to apply cost adjustment to cost_tracker
2022-04-17 19:26:24 -05:00
Tao Zhu
094da35b91
Address review comments:
...
1. use was_executed to correctly identify transactions requires cost adjustment;
2. add function to specifically handle executino cost adjustment without have to copy accounts
2022-04-17 19:26:24 -05:00
Tao Zhu
29ca21ed78
undo transaction cost from cost_tracker if it was not executed successfully
2022-04-17 19:26:24 -05:00
Brooks Prumo
f33ad34531
Add feature_set_override parameter to mock_process_instruction() ( #24386 )
2022-04-15 13:43:04 -05:00
Jeff Washington (jwash)
b4fd9124bf
log secondary index contents on startup ( #24348 )
2022-04-15 13:30:03 -05:00
Jeff Washington (jwash)
8c9430359e
add early exit in get_corrected_rent_epoch_on_load ( #24331 )
2022-04-15 13:28:16 -05:00
Jeff Washington (jwash)
ba7a2efa66
SlotInfoInEpoch ( #24332 )
2022-04-15 13:27:41 -05:00
Justin Starry
4ed647d8ec
Test that tick slot hashes update the recent blockhash queue ( #24242 )
2022-04-16 00:30:20 +08:00
Jeff Washington (jwash)
0e7b0597db
check for rewrites skipped in closure ( #24330 )
2022-04-14 13:46:18 -05:00
Jeff Washington (jwash)
d20b4c9958
rename function param to avoid conflict ( #24342 )
2022-04-14 11:56:32 -05:00
Jeff Washington (jwash)
a91b0c8ea3
dashmap -> rwlock<hashmap> for rewrites ( #24327 )
2022-04-14 11:55:58 -05:00
HaoranYi
e3ef0741be
simplify bank drop calls ( #24142 )
...
* simplify bank drop calls
* clippy: import
* Update ledger/src/blockstore_processor.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/accounts_background_service.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* cleanup
* format
* use msb of bank_id to indicates that we are dropping
* clippy
* restore bank id
* clippy
* revert is-serialized_with_abs flag
* assert
* clippy
* whitespace
* fix bank drop callback check
* more fix
* remove msb dropping implementation
* fix
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-14 08:43:54 -05:00
Jeff Washington (jwash)
255ef66a27
move feature activation log to correct fn for hash ( #24326 )
2022-04-13 17:43:33 -05:00
Jeff Washington (jwash)
6a474f29cd
default to disk index ( #24251 )
2022-04-13 09:24:50 -05:00
Jeff Washington (jwash)
b6b8783323
add maybe_update_rent_epoch_on_load ( #24294 )
2022-04-13 08:55:24 -05:00
HaoranYi
929753a11f
typo ( #24291 )
2022-04-12 16:46:59 -05:00
Jeff Washington (jwash)
69e9ad5571
update comment ( #24288 )
2022-04-12 12:37:46 -05:00
Jeff Washington (jwash)
2d4d639635
add expected_rent_collection ( #24028 )
...
* add expected_rent_collection
* update some comments for clarity and resolve a todo
* add test for 'LeaveAloneNoRent'
2022-04-12 11:32:23 -05:00
Jeff Washington (jwash)
bdbca3362e
increase test timeout ( #24277 )
2022-04-12 09:54:57 -05:00
Jeff Washington (jwash)
1bc49d219d
IndexLimitMb option adds 'Unspecified' state ( #24249 )
2022-04-12 09:38:09 -05:00
HaoranYi
605036c117
move test fn into its own mod ( #24212 )
...
* move test fn into its own mod
* pub
2022-04-12 09:36:05 -05:00
HaoranYi
f3aa80d3f8
typo ( #24257 )
2022-04-12 09:08:35 -05:00
Jon Cinque
9b8850f99e
test-validator: Add `--max-compute-units` flag ( #24130 )
...
* test-validator: Add `--max-compute-units` flag
* Add `RuntimeConfig` for tweaking runtime behavior
* Actually add the file
* Move RuntimeConfig to runtime
2022-04-12 02:28:10 +02:00
Brooks Prumo
4fd184c131
Use Release/Acquire instead of SeqCst for is_bank_drop_callback_enabled ( #24134 )
2022-04-11 19:19:17 -05:00
Jeff Washington (jwash)
6fbe2b936c
fix comment ( #24254 )
2022-04-11 18:53:45 -05:00
Jeff Washington (jwash)
c0019edf00
document WaitableCondvar ( #24252 )
2022-04-11 14:45:23 -05:00
Jeff Washington (jwash)
9ac2245970
remove clone ( #24244 )
2022-04-11 13:15:00 -05:00
Giorgio Gambino
60b2155bd3
Add accounts-filler-size command line option ( #23896 )
2022-04-11 13:10:09 -05:00
HaoranYi
e14933c54d
move bank test fn to its test_utils mod ( #24171 )
2022-04-11 10:42:24 -05:00
Justin Starry
8eef3d9713
Add tests to the blockhash queue ( #24238 )
2022-04-11 19:36:24 +08:00
Christian Kamm
a058f348a2
Address review comments
2022-04-08 14:37:55 -05:00
Christian Kamm
2ed29771f2
Unittest for cost tracker after process_and_record_transactions
2022-04-08 14:37:55 -05:00
Christian Kamm
924b8ea1eb
Adjustments to cost_tracker updates
...
- don't store pending tx signatures and costs in CostTracker
- apply tx costs to global state immediately again
- go from commit_or_cancel to update_or_remove, where the cost tracker
is either updated with the true costs for successful tx, or the costs
of a retryable tx is removed
- move the function into qos_service and hold the cost tracker lock for
the whole loop
2022-04-08 14:37:55 -05:00
Tao Zhu
9e07272af8
- Only commit successfully executed transactions' cost to cost_tracker;
...
- In-fly transactions are pended in cost_tracker until being committed
or cancelled;
2022-04-08 14:37:55 -05:00
Dmitri Makarov
03ed334ebb
Double the chunk size for sending the program binary data in tx
2022-04-08 09:06:40 -07:00
Jeff Washington (jwash)
210f6a6fab
move hash calculation out of acct bg svc ( #23689 )
...
* move hash calculation out of acct bg svc
* pr feedback
2022-04-08 10:42:03 -05:00
Jeff Washington (jwash)
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
Jeff Washington (jwash)
48d1af01c8
add metrics around rewards ( #24160 )
2022-04-07 11:44:26 -05:00
Jeff Washington (jwash)
f7b2951c79
move around some index metrics to reduce locks ( #24161 )
2022-04-07 09:43:19 -05:00
HaoranYi
42c094739d
add test cfg attribute ( #24154 )
2022-04-07 09:05:20 -05:00
Jeff Washington (jwash)
550ca7bf92
compare contents of serialized banks instead of exact file format ( #24141 )
...
* compare contents of serialized banks instead of exact file format
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
* get rid of clone
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
Jeff Washington (jwash)
fddd162645
reserialize bank in ahv by first writing to temp file in abs ( #23947 )
2022-04-06 21:39:26 -05:00
Brooks Prumo
c322842257
Replace channel with Mutex<Option> for AccountsPackage ( #24013 )
2022-04-06 05:47:19 -05:00
carllin
4ea59d8cb4
Set drop callback on first root bank ( #23999 )
2022-04-05 13:02:33 -05:00
Jeff Washington (jwash)
4a11fa072f
hash_account_with_rent_epoch ( #24104 )
2022-04-05 08:10:31 -05:00
Jeff Washington (jwash)
6a7f6585ce
persist historical_roots ( #24029 )
2022-04-04 13:13:11 -05:00
Jeff Washington (jwash)
132f08486a
remove basically duplicate function ( #24107 )
2022-04-04 12:55:05 -05:00
Tao Zhu
997db7637c
do simple math without floats
2022-04-04 12:32:00 -05:00
Jeff Washington (jwash)
f8f3edac3c
update comment ( #24108 )
2022-04-04 11:06:01 -05:00
Jeff Washington (jwash)
2820b64eb3
roots_original -> historical_roots ( #24063 )
2022-04-04 09:12:12 -05:00
behzad nouri
ef3e3dce7a
hides implementation details of vote-accounts from public interface ( #24087 )
2022-04-04 13:20:26 +00:00
behzad nouri
fa7eb7f30c
improves Stakes::activate_epoch performance ( #24068 )
...
Tested with mainnet stakes obtained from the ledger at 5 recent epoch
boundaries, this code is ~30% faster than current master.
Current code:
epoch: 289, elapsed: 82901us
epoch: 290, elapsed: 80525us
epoch: 291, elapsed: 79122us
epoch: 292, elapsed: 79961us
epoch: 293, elapsed: 78965us
This commit:
epoch: 289, elapsed: 61710us
epoch: 290, elapsed: 55721us
epoch: 291, elapsed: 55886us
epoch: 292, elapsed: 55399us
epoch: 293, elapsed: 56803us
2022-04-02 22:48:51 +00:00
Jeff Washington (jwash)
0ca5a0ec68
prior_roots -> historical_roots ( #24064 )
2022-04-02 12:01:43 -05:00
Jeff Washington (jwash)
ec97d6d078
rename remove_old_roots ( #24059 )
2022-04-02 12:01:13 -05:00
Jeff Washington (jwash)
3ca4fffa78
root -> alive_root ( #24062 )
2022-04-02 12:00:52 -05:00
Justin Starry
792bbf75ab
Support sending versioned txs in AsyncClient ( #23982 )
2022-04-02 11:12:02 +08:00
Alexander Meißner
1b45c509c3
Refactor: Use `InstructionContext::get_instruction_data()` ( #24014 )
...
* Adds transaction_context and instruction_context where invoke_context.get_keyed_accounts() is used.
* Use instruction_context.get_instruction_data() instead of an explicit parameter.
* Removes instruction_data parameter from Executor::execute().
* Removes instruction_data parameter from ProcessInstructionWithContext.
2022-04-01 15:48:05 +02:00
Jeff Washington (jwash)
31997f8251
hash calc scanning takes config ( #24016 )
2022-03-31 14:26:37 -05:00
Jeff Washington (jwash)
9c8dad33c7
add epoch_schedule and rent_collector to hash calc ( #24012 )
2022-03-31 10:51:18 -05:00
Jeff Washington (jwash)
da001d54e5
calculate_accounts_hash_helper uses config ( #24003 )
2022-03-31 09:29:45 -05:00
Justin Starry
cb5e67d327
Use Rent sysvar directly for stake split instruction ( #24008 )
...
* Use Rent sysvar directly for stake split ix
* Add feature to gate rent sysvar change
* fix tests
* cargo clippy
2022-03-31 16:46:35 +08:00
Jeff Washington (jwash)
125f9634fd
add hash calc config.use_write_cache ( #24005 )
2022-03-30 17:19:34 -05:00
Jeff Washington (jwash)
82c5230bc2
AccountsPackage::new less brittle ( #23968 )
2022-03-30 14:06:15 -05:00
HaoranYi
1fb82d7924
fix typo in comments ( #24004 )
2022-03-30 09:47:51 -05:00
Jeff Washington (jwash)
af9344fe22
SortedStorages refactoring ( #23998 )
2022-03-30 09:19:03 -05:00
Jeff Washington (jwash)
5636570d6d
add roots_original to roots tracker ( #23849 )
2022-03-30 08:52:45 -05:00
Jeff Washington (jwash)
da844d7be5
refactoring of SortedStorages tests to make other changes easier ( #23990 )
2022-03-29 22:06:48 -05:00
Jeff Washington (jwash)
5a613e9b6e
use CalcAccountsHashConfig in calculate_accounts_hash ( #23987 )
2022-03-29 22:05:47 -05:00
Alexander Meißner
794645d092
Adds check_number_of_instruction_accounts() to all builtin programs except for the address-lookup-table. ( #23984 )
2022-03-29 19:06:50 +02:00
Brooks Prumo
31b707b625
Specify if archive size datapoint is for full or incremental snapshots ( #23941 )
2022-03-26 12:29:13 -05:00
Jeff Washington (jwash)
c24de17278
remove index hash calculation as an option ( #23928 )
2022-03-25 15:32:53 -05:00
Jeff Washington (jwash)
ec78702bc8
RollingBitField::get_all_less_than ( #23919 )
2022-03-25 15:20:22 -05:00
Jeff Washington (jwash)
acfd22712b
RollingBitFIeld to its own file ( #23917 )
2022-03-25 10:37:00 -05:00
Jeff Washington (jwash)
55d61023f7
document 'accounts' hash ( #23907 )
2022-03-24 15:58:52 -05:00
Alexander Meißner
140c8dd01f
Refactor: Replaces KeyedAccount in_get_sysvar_with_account_check ( #23905 )
...
* Replaces all use sites of get_sysvar_with_account_check by get_sysvar_with_account_check2.
* Removes get_sysvar_with_account_check.
* Renames get_sysvar_with_account_check2 to get_sysvar_with_account_check.
2022-03-24 19:30:42 +01:00
Jeff Washington (jwash)
37c36ce3fa
pass stats separately from CalcAccountsHashConfig ( #23892 )
2022-03-24 12:48:47 -05:00
Jeff Washington (jwash)
82328fd9d8
move max_clean_root deeper in flush cache ( #23869 )
2022-03-24 12:45:49 -05:00
steviez
c31db81ac4
Use VoteAccountsHashMap type alias in all applicable spots ( #23904 )
2022-03-24 12:09:48 -05:00
Jeff Washington (jwash)
a22a2384bf
fix ci test error ( #23908 )
2022-03-24 11:30:20 -05:00
Jeff Washington (jwash)
5b916961b5
HashCalc uses self.accounts_cache ( #23890 )
2022-03-24 10:34:28 -05:00
Jeff Washington (jwash)
f2aea3b7c7
flush_slot_cache takes [Slot] ( #23865 )
2022-03-24 10:24:36 -05:00
Jeff Washington (jwash)
9d3b17c635
HashCalc uses self.accounts_index ( #23888 )
2022-03-24 10:06:32 -05:00
Jeff Washington (jwash)
396b49a7c1
Start saving/loading prior_roots(_with_hash) to snapshot ( #23844 )
...
* Start saving/loading prior_roots(_with_hash) to snapshot
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* update comment
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-03-24 10:06:24 -05:00
Jeff Washington (jwash)
b22165ad69
hash calc uses self.filler_account_suffix ( #23887 )
2022-03-24 09:58:06 -05:00
Jeff Washington (jwash)
9022931689
calc hash uses self.num_hash_scan_passes ( #23883 )
2022-03-24 09:44:42 -05:00
Jeff Washington (jwash)
e3eb002f66
Log storage size stats at hash calc ( #23843 )
2022-03-24 09:40:35 -05:00
Jeff Washington (jwash)
f1a411c897
add epoch_schedule and rent_collector to hash calc ( #23857 )
2022-03-24 09:39:22 -05:00
Jeff Washington (jwash)
db5d68f01f
HashCalc uses self.accounts_hash_cache_path ( #23882 )
2022-03-24 09:31:55 -05:00
HaoranYi
90009f330b
small refactor to shorten the lock on slot_under_contention hashset ( #23891 )
...
* small refactor to shorten the lock on slot_under_contention hashset
* adding comments
* comments
2022-03-24 08:20:56 -05:00
Alexander Meißner
91c2729856
Replaces keyed_account get_signers() by InstructionContext::get_signers(). ( #23863 )
2022-03-24 12:57:51 +01:00
Jeff Washington (jwash)
5a892af2fe
disable 'check_hash' on accounts hash calc ( #23873 )
2022-03-23 21:03:31 -05:00
Jeff Washington (jwash)
3e22d4b286
calc hash uses self.thread_pool_clean ( #23881 )
2022-03-23 20:52:38 -05:00
Jack May
486f7b7673
use array access function ( #23895 )
2022-03-23 17:03:01 -07:00
Jeff Washington (jwash)
dc3863ef14
flush_slot_cache_with_clean ( #23868 )
2022-03-23 14:09:56 -05:00
Jeff Washington (jwash)
260f899eda
write cache: hashmap to set ( #23866 )
2022-03-23 14:05:45 -05:00
Jeff Washington (jwash)
9e61fe7583
add AccountsHashConfig to manage parameters ( #23850 )
2022-03-23 13:44:23 -05:00
Jeff Washington (jwash)
b1280b670a
calculate_accounts_hash_without_index takes &self ( #23846 )
...
* calculate_accounts_hash_without_index takes &self
* Update runtime/src/snapshot_package.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-23 11:57:32 -05:00
Jeff Washington (jwash)
7b89222fde
don't start extra threads for shrink/clean/hash ( #23858 )
2022-03-23 11:53:37 -05:00
Jeff Washington (jwash)
493a8e2348
remove random flushing of write cache ( #23845 )
2022-03-23 08:45:44 -05:00
Jeff Washington (jwash)
bc35e1c5f5
snapshot code needs all storages for hash calc ( #23840 )
2022-03-22 21:27:54 -05:00
Alexander Meißner
9f0ca6d88a
Refactor: Remove `trait` from nonce keyed account ( #23811 )
...
* Removes the trait `NonceKeyedAccount`.
2022-03-23 02:09:30 +01:00
Jon Cinque
7af48465fa
transaction-status: Add return data to meta ( #23688 )
...
* transaction-status: Add return data to meta
* Add return data to simulation results
* Use pretty-hex for printing return data
* Update arg name, make TransactionRecord struct
* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Jeff Washington (jwash)
1089a38aaf
AcctIdx: rework scan and write to disk ( #23794 )
2022-03-22 11:54:12 -05:00
Jeff Washington (jwash)
89ba3ff139
log fail to evict ( #23815 )
2022-03-22 09:19:38 -05:00
Jeff Washington (jwash)
24f6855f86
AcctIdx: only remove a fixed number of items per write lock ( #23795 )
2022-03-21 16:55:04 -05:00