Commit Graph

1308 Commits

Author SHA1 Message Date
behzad nouri 3c1ce3cc93
permanently disables durable nonces with chain blockhash domain (#25788)
https://github.com/solana-labs/solana/pull/25744
separated durable nonce and blockhash domains, which will stop double
execution going forward. However it is possible that a durable
transaction has *already* been executed once as a normal transaction and
it is now a valid durable transaction. #25744 cannot stop such
transactions to be re-executed until the nonce accounts are advanced.

This commit adds a new nonce version indicating that the nonce is moved
out of the blockhash domain, and permanently disables durable
transactions for legacy nonces which are in the blockhash domain.
2022-06-09 15:28:37 +00:00
Justin Starry 6a7edc02c6
Reject durable nonce txs that don't use an advanceable nonce (#25832)
* Reject durable nonce txs that use the latest durable nonce hash

* feedback
2022-06-08 21:34:57 +00:00
Justin Starry b2b426d4bf
Reject durable nonce transactions not signed by authority (#25831) 2022-06-08 14:43:09 -05:00
Brooks Prumo d4e4871c47
Add tests to Bank for accounts data size (#25816) 2022-06-07 18:28:53 +00:00
apfitzge e6c21a3036
Convert Measure::this to measure! and remove Measure::this (#25776)
* Remove the args param from Measure::this since we don't ever use it

* banking_stage.rs: convert to measure!

* poh_recorder.rs: convert to measure!

* cost_update_service.rs: convert to measure!

* poh_service.rs: convert to measure!

* bank.rs: convert to measure!

* measure.rs: Remove Measure::this now that all have been converted to measure!
2022-06-06 20:21:05 -05:00
Ashwin Sekar 8caced68ce
Serialize lamports per signature (#25364)
* Serialize lamports per signature

* Add full snapshot archive test, enable features in previous tests
2022-06-06 13:07:02 -07:00
Brooks Prumo ec64d5261f
Avoid RMWs on shared data inside parallel loops: collect_rent_from_accounts() (#25790) 2022-06-06 13:32:22 -05:00
Justin Starry 61ad8fcc55
Fix bank executor stats and remove copy-on-write semantics (#25621)
* Fix bank executor stats and remove copy-on-write semantics

* Remove clone implementation for CachedExecutors

* feedback
2022-06-04 13:54:26 +00:00
behzad nouri 9851774133 adds feature gate enabling durable nonce
Previous commit separates durable nonce and blockhash domains with a
feature gate. A 2nd feature added in this commit enables durable nonce
at least one epoch after the 1st feature.
By the time 2nd feature is activated, some nonce accounts will have an
old blockhash, but no nonce account can have a recent blockhash.
As a result no transaction (durable or normal) can be executed twice.
2022-06-04 12:39:31 +00:00
behzad nouri 5ee157f43d separates durable nonce and blockhash domains
AdvanceNonceAccount instruction updates nonce to blockhash. This makes it
possible that a durable transaction is executed twice both as a normal
transaction and a nonce transaction if it uses blockhash (as opposed to nonce)
for its recent_blockhash field.

The commit prevents this double execution by separating nonce and blockhash
domains; when advancing nonce account, blockhash is hashed with a fixed string.
As a result a blockhash cannot be a valid nonce value; and if transaction was
once executed as a normal transaction it cannot be re-executed as a durable
transaction again and vice-versa.
2022-06-04 12:39:31 +00:00
Jeff Washington (jwash) 8b0d5b2485
parallelize rent collection (#25767) 2022-06-03 18:00:02 -05:00
Christian Kamm 0da0e0da63
Bank: Batch account stores in collect_rent_eagerly (#25707)
* Bank: Batch account stores in collect_rent_eagerly

Previously store() calls were done one-by-one, which leads to suboptimal
performance.

* Accounts: Remove store_slow_cached()

* clippy

Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2022-06-03 15:00:39 -05:00
Jeff Washington (jwash) 646bd2e488
extract prefix_from_pubkey (#25766) 2022-06-03 14:03:15 -05:00
Jeff Washington (jwash) a9a94166ef
load_account_into_read_cache (#25760) 2022-06-03 13:34:17 -05:00
Justin Starry 6fc057c1d9
Remove activated evict_invalid_stakes_cache_entries feature (#25739) 2022-06-02 21:04:54 +00:00
apfitzge ed68c0b889
epoch_stakes_reation -> epoch_stakes_creation in Bank::_new_from_parent (#25660) 2022-06-02 12:26:01 -05:00
Jeff Washington (jwash) 96f71c7d4d
add metrics on rent collection pieces (#25713) 2022-06-02 09:08:53 -05:00
apfitzge 369ff0858d
Helper functions for getting rent partitions between slots (#25669) 2022-06-01 20:20:44 -05:00
Jeff Washington (jwash) 891dcb2084
move rent collection time into datapoint (#25714) 2022-06-01 17:28:47 -05:00
Trent Nelson b80a82a9e5 runtime: gate nonce disable at restart slot 2022-06-01 12:00:10 -06:00
Trent Nelson 635bd40f09 runtime: disable durable nonces 2022-06-01 11:15:16 -06:00
Tyera Eulberg 8584a3a331
Add feature to increase timestamp bounding on the slow side to 150% of poh estimate (1s/slot) (#25666) 2022-05-31 16:53:06 -06:00
Christian Kamm 387d9d9c77 Store missing unupdated executors earlier 2022-05-28 13:42:35 +10:00
carllin 740ee0c502
Fix datapoint name (#25614) 2022-05-28 01:01:29 +00:00
Trent Nelson 4e6b276f6c
bump executor cache max entries (#25600)
* executor-cache: unify `remove` calls

* executor-cache: metrics - add one-hit-wonder counter

* executor-cache: bump max entries to 256
2022-05-28 01:54:10 +10:00
Brooks Prumo 7384c41ea7
Speed up bank::tests::test_max_accounts_data_size_exceeded() (#25523) 2022-05-25 22:22:30 -04:00
Brooks Prumo b7149f7f54
Add helper fn to set accounts_data_size_initial for tests (#25522) 2022-05-25 02:43:54 +00:00
Brooks Prumo 0dd3c6bf1f
Split up accounts data size field (#25495) 2022-05-24 20:26:32 -04:00
Justin Starry e66ea7cb6a Clean up Bank::commit_transactions parameters 2022-05-24 20:24:42 +08:00
Jeff Washington (jwash) 41f30a2383
stop logging misleading bank hash mismatch (#25427) 2022-05-23 08:43:25 -05:00
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
Alexander Meißner 7eae98c6d9
Introduces `Result` return types to mutable accessors of `BorrowedAccount` (#25380)
* Introduces result return types to get_data_mut(), set_data() and set_data_length() of BorrowedAccount.

* Introduces result return types to set_owner(), set_lamports() and set_executable() of BorrowedAccount.
2022-05-21 17:47:09 +02:00
Jack May 851958f77a
Disallow deployment of deprecated _sol_alloc_free syscall (#25179) 2022-05-20 13:19:41 -07:00
Jeff Washington (jwash) 9ef0acdf5d
allow hash calc to use all threads at startup (#25369) 2022-05-20 11:45:01 -05:00
Jeff Washington (jwash) 467431de89
add ledger-tool accounts-db-skip-initial-hash-calc (#25354) 2022-05-20 10:27:00 -05:00
Brooks Prumo 021276c955
Move updating accounts data len from execute to commit (#25256) 2022-05-19 12:24:13 -05:00
Justin Starry a1522d0024
Use consistent naming for compute unit limit (#25229)
* Use consistent naming for compute unit limit

* feedback
2022-05-18 13:14:31 +08:00
Jack May 21e066ef26
Merge pull request from GHSA-wxrh-ff9f-fc6w 2022-05-16 12:30:37 -07:00
Justin Starry f81c5df1f0
Include account index in rent paying account tx error (#25189) 2022-05-16 23:35:34 +08:00
Tao Zhu b1b3702e6d
Prioritize transactions in banking stage by their compute unit price (#25178)
* - get prioritization fee from compute_budget instruction;
- update compute_budget::process_instruction function to take instruction iter to support sanitized versioned message;
- updated runtime.md

* update transaction fee calculation for prioritization fee rate as lamports per 10K CUs

* review changes

* fix test

* fix a bpf test

* fix bpf test

* patch feedback

* fix clippy

* fix bpf test

* feedback

* rename prioritization fee rate to compute unit price

* feedback

Co-authored-by: Justin Starry <justin@solana.com>
2022-05-16 12:06:33 +08:00
Ryo Onodera 592aa4c21b
Improve hard fork message with more detail (#25214) 2022-05-15 01:14:23 +09:00
Jack May cde15ff687
support 64-bit prioritization fee (#25027) 2022-05-12 11:07:36 -07:00
Jon Cinque 542a14d9b7
test-validator: Always use overridden compute budget in bank (#25128) 2022-05-11 15:46:51 +02:00
Pankaj Garg c838e15234
Unset needs_unlock for rebatched transactions batches (#25095)
* Unset needs_unlock for rebatched transactions batches

* address review comments
2022-05-10 13:39:08 -07:00
Jeff Washington (jwash) b91cc87680
assert when rewrites skipped (#25110) 2022-05-10 13:55:48 -05:00
Justin Starry 082502d4f3
Fail tx sanitization when ix program id uses lookup table (#25035)
* Fail tx sanitization when ix program id uses lookup table

* feedback
2022-05-07 03:19:50 +08:00
Jeff Washington (jwash) df9a4a0c9d
Bank::new_for_benches uses bench config (#25017) 2022-05-06 13:35:18 -05:00
behzad nouri 492f89a170
checks account owner when initializing a vote-account (#25018)
A VoteAccount may only wrap an account if the account owner is
solana_vote_program:id or equivalently this check returns true:
solana_vote_program::check_id(account.owner())
2022-05-06 16:22:49 +00:00
Jeff Washington (jwash) 0cc97689f9
add stats for stakes_cache_check_and_store_us (#24349)
* add stats for stakes_cache_check_and_store_us

* remove redundant metrics
2022-05-05 17:22:48 -05:00
Jeff Washington (jwash) e0092902ea
maybe_update_rent_epoch_on_load uses correct EpochSchedule (#24999) 2022-05-05 13:34:03 -05:00
Jeff Washington (jwash) e8bf812a8a
allow ledger-tool halt at slot to calc hash using write cache (#24974) 2022-05-04 16:27:49 -05:00
apfitzge 0ca54db524
use bank's epoch_schedule accessor internally (#24918) 2022-05-04 13:52:48 -05:00
Jack May e070c5ca38
default compute units per instruction (#24899) 2022-05-03 09:50:06 -07:00
apfitzge 112a0b475a
Revert "Refactor to use EpochSchedule from within RentCollector struct" (#24893)
* Revert "Ran cargo fmt"

This reverts commit 9052e41b32.

* Revert "Fix build error introduced by my editor setup, part 2"

This reverts commit 4dfeab3b38.

* Revert "Fix build error introduced by my editor setup"

This reverts commit 87fb78dc56.

* Revert "Remove redundant epoch_schedule from AccountsPackage"

This reverts commit c2f7f2fff8.

* Revert "Fix a test"

This reverts commit 36c0bdaa78.

* Revert "Fixes to initial code"

This reverts commit ed7813e698.

* Revert "Removing redundant EpochSchedule param from fns"

This reverts commit 5472d2e605.
2022-05-02 13:46:17 -05:00
Jeff Washington (jwash) 546730cf42
retire deprecate_rewards_sysvar (#24651)
* retire deprecate_rewards_sysvar

* Update expected hash for bank after eliminating reqards sysvar

* Hack test_bank_update_vote_stake_rewards to pass after removing sysvar rewards account

* formatting change

* formatting cleanup

* Remove legacy validator Rewards code from unit test. Change expected inflation computation

* Update bank.rs

* Update bank.rs

* Clean up inflation abstraction function

* Formatting cleanup

* rename vars. Add struct for inflation rewards return

* format cleanup + comment

* Fix format warnings

Co-authored-by: Brennan Watt <brennan.watt@solana.com>
2022-05-02 10:10:51 -05:00
Brooks Prumo a73f99823d
Do not hold lock unnecessarily when hashing (#24815) 2022-04-29 08:40:28 +00:00
Jon Cinque b4503d4110
token: Add features for new program releases (#24742)
* token: Add features for new program releases

* Add PR number
2022-04-28 23:43:57 +02:00
Tyera Eulberg e189603d17
Make test less brittle (#24803) 2022-04-28 12:09:01 -06:00
Brooks Prumo 7b6880f652
Make test_bank_cloned_stake_delegations() aware of stake minimum delegation (#24753) 2022-04-28 09:35:07 -05:00
Jeff Washington (jwash) f22738fd7b
fix test_rent_eager_collect_rent_in_partition (#24722) 2022-04-26 23:06:57 -05:00
Ryo Onodera 412a5a0d33
Credits auto rewind on vote recreation (#22546)
* Credits auto rewind on vote recreation

* Update comment

* Improve comments and tests

* Recommended fn rename

* Restore old feature, and replace new feature key

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-04-26 15:49:35 -06:00
Steven Luscher 3007f233f4
Log subscription for `All` no longer clobbers `Some` subscription for pubkey (#24215)
* fix: subscribing for all logs no longer clobbers pubkey-specific subscriptions
* test: write tests to cover logsSubscribe
2022-04-26 10:31:11 -07:00
Jeff Washington (jwash) 534a666153
hookup 'just_rewrites' (#24662) 2022-04-26 12:14:25 -05:00
Justin Starry b6bc5d2fa1 Aggregate executor update timing metrics 2022-04-26 23:15:22 +08:00
behzad nouri be0bdd2261
explicitly ignores struct fields which do not impact PartialEq impl (#24624)
Unless struct fields are explicitly ignored in PartialEq implementation,
there are no compile-time checks that if more fields are added to the
struct, PartialEq implementation is accordingly updated.
2022-04-24 14:33:42 +00:00
carllin 8a062273de
Move error counters to be reported by leader only at end of slot (#24581)
* Add error counters to leader metrics only

* Add dependencies
2022-04-23 18:10:47 -05:00
Michael Vines 83e041299a Run real snapshot packager while processing blockstore at validator startup 2022-04-22 21:17:49 -07:00
Jon Cinque 0d51596224
sim: Override slot hashes account on simulation bank (#24543)
* sim: Override slot hashes during simulation

* Add simulation test program

* Address feedback

* Add AccountOverrides explicit type

* Cargo fmt
2022-04-22 12:32:31 +02:00
behzad nouri d524ae293f collects metrics for cached vote/stake accounts consistency
Tracking mismatches between cached vote/stake accounts and accounts-db
in preparation of activating feature for updating rewards at
epoch-boundary using cached vote/stake accounts.
2022-04-21 15:28:41 +00:00
behzad nouri 108aa23d90 enforces type-safety in Stakes<StakeAccount> using phantom data
StakeAccount<Delegation> can only wrap a stake-state which is a
Delegation; whereas StakeAccount<()> wraps any account with stake state.

As a result, StakeAccount::<Delegation>::delegation() will return
Delegation instead of Option<Delegation>.
2022-04-21 15:28:41 +00:00
behzad nouri f937fcbd95 updates rewards at epoch boundary using cached accounts
Loading vote and stake accounts from accounts-db takes a significant
portion of time updating rewards at epoch boundary.

This commit bypasses accounts-db and instead uses vote and stake
accounts cached in bank stakes:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/stakes.rs#L148-L152

These cached accounts are synchronized with accounts-db after each
transaction, and so there should not be any change in the resulting
computation:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/bank.rs#L4526

Nevertheless, to avoid any chances of introducing a consensus issue, the
switch to cached account is feature gated.
2022-04-21 15:28:41 +00:00
behzad nouri b4491ff4ba adds StakesEnum type representing Stakes<StakeAccount|Delegation>
For backward compatibility, we can only serialize and deserialize
Stakes<Delegation>. However Bank caches Stakes<StakeAccount>. This type
mismatch incurs a conversion cost at epoch boundary when updating
EpochStakes.

This commit adds StakesEnum which allows EpochStakes to include either a
Stakes<StakeAccount> or Stakes<Delegation> and so bypass the conversion
cost between the two at the epoch boundary.
2022-04-21 15:28:41 +00:00
behzad nouri 454ef38e43 caches StakeAccount instead of Delegation in Stakes
The commit makes values in stake_delegations map in Stakes struct
generic. Stakes<Delegation> is equivalent to the old code and is used
for backward compatibility in BankFieldsTo{Serialize,Deserialize}.

But banks cache Stakes<StakeAccount> which includes the entire stake
account and StakeState deserialized from account. Doing so, will remove
the need to load stake account from accounts-db when working with
stake-delegations.
2022-04-21 15:28:41 +00:00
behzad nouri 8c4d6357fe adds StakeAccount type embodying an Account and a StakeState
The added type does sanity checks on the Account and stores deserialized
StakeState. Following commits will use this type instead of Delegation
in Stakes.
2022-04-21 15:28:41 +00:00
Justin Starry d5127abf46
Only add hashes for completed blocks to recent blockhashes (#24389)
* Only add hashes for completed blocks to recent blockhashes

* feedback
2022-04-21 21:05:29 +08:00
Tao Zhu a21fc3f303
Apply transaction actual execution units to cost_tracker (#24311)
* Pass the sum of consumed compute units to cost_tracker

* cost model tracks builtins and bpf programs separately, enabling adjust block cost by actual bpf programs execution costs

* Copied nightly-only experimental `checked_add_(un)signed` implementation to sdk

* Add function to update cost tracker with execution cost adjustment

* Review suggestion - using enum instead of struct for CommitTransactionDetails
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>

* review - rename variable to distinguish accumulated_consumed_units from individual compute_units_consumed

* not to use signed integer operations

* Review - using saturating_add_assign!(), and checked_*().unwrap_or()

* Review - using Ordering enum to cmp

* replace checked_ with saturating_

* review - remove unnecessary Option<>

* Review - add function to report number of non-zero units account to metrics
2022-04-21 07:38:07 +00:00
Justin Starry 79923c3b58
Refactor: Rename BlockhashQueue fields and methods for clarity (#24426) 2022-04-21 11:57:17 +08:00
Jeff Washington (jwash) 52f43a986d
skip rewrites in bank (#23985) 2022-04-20 14:08:46 -05:00
Jeff Washington (jwash) 813d06529d
stats for rewrites (#24529) 2022-04-20 10:43:50 -05:00
Jeff Washington (jwash) cdc3c41001
helpers to skip rewrites in bank (#24507) 2022-04-20 10:21:02 -05:00
Jeff Washington (jwash) 957849f4a5
load_to_collect_rent_eagerly returns loaded_slot (#24506) 2022-04-20 09:14:03 -05:00
Jeff Washington (jwash) ef5e7cce6e
type PubkeyAccountSlot (#24503) 2022-04-19 21:34:00 -05:00
Jeff Washington (jwash) c4923d29b4
bank_hash_at uses rewrites (#24439) 2022-04-19 11:29:29 -05:00
Jeff Washington (jwash) 569b15d863
maybe_update_rent_epoch_on_load (#24055) 2022-04-19 08:29:36 -05:00
Jack May 5d1adf1270
Fix signature count (#24471)
* Fix signature count

* protect the signature count futher
2022-04-19 17:59:06 +08:00
Jeff Washington (jwash) 94d3ec37d2
collect_rent_eagerly can just do rewrites (#24438) 2022-04-18 22:44:19 -05:00
Jeff Washington (jwash) ee384f7254
remove 'startup' from verify_snapshot_bank (#24441) 2022-04-18 14:35:49 -05:00
Jason Davis ed7813e698 Fixes to initial code 2022-04-18 11:57:40 -05:00
Jason Davis 5472d2e605 Removing redundant EpochSchedule param from fns 2022-04-18 11:57:40 -05:00
Justin Starry 4ed647d8ec
Test that tick slot hashes update the recent blockhash queue (#24242) 2022-04-16 00:30:20 +08:00
Jeff Washington (jwash) a91b0c8ea3
dashmap -> rwlock<hashmap> for rewrites (#24327) 2022-04-14 11:55:58 -05:00
HaoranYi e3ef0741be
simplify bank drop calls (#24142)
* simplify bank drop calls

* clippy: import

* Update ledger/src/blockstore_processor.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/accounts_background_service.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* cleanup

* format

* use msb of bank_id to indicates that we are dropping

* clippy

* restore bank id

* clippy

* revert is-serialized_with_abs flag

* assert

* clippy

* whitespace

* fix bank drop callback check

* more fix

* remove msb dropping implementation

* fix

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-14 08:43:54 -05:00
Jeff Washington (jwash) b6b8783323
add maybe_update_rent_epoch_on_load (#24294) 2022-04-13 08:55:24 -05:00
Jeff Washington (jwash) 2d4d639635
add expected_rent_collection (#24028)
* add expected_rent_collection

* update some comments for clarity and resolve a todo

* add test for 'LeaveAloneNoRent'
2022-04-12 11:32:23 -05:00
Jeff Washington (jwash) bdbca3362e
increase test timeout (#24277) 2022-04-12 09:54:57 -05:00
HaoranYi f3aa80d3f8
typo (#24257) 2022-04-12 09:08:35 -05:00
Jeff Washington (jwash) 9ac2245970
remove clone (#24244) 2022-04-11 13:15:00 -05:00
HaoranYi e14933c54d
move bank test fn to its test_utils mod (#24171) 2022-04-11 10:42:24 -05:00
Justin Starry 8eef3d9713
Add tests to the blockhash queue (#24238) 2022-04-11 19:36:24 +08:00
Tao Zhu 9e07272af8 - Only commit successfully executed transactions' cost to cost_tracker;
- In-fly transactions are pended in cost_tracker until being committed
  or cancelled;
2022-04-08 14:37:55 -05:00
Jeff Washington (jwash) 48d1af01c8
add metrics around rewards (#24160) 2022-04-07 11:44:26 -05:00
Jeff Washington (jwash) 550ca7bf92
compare contents of serialized banks instead of exact file format (#24141)
* compare contents of serialized banks instead of exact file format

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* pr feedback

* get rid of clone

* pr feedback

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
carllin 4ea59d8cb4
Set drop callback on first root bank (#23999) 2022-04-05 13:02:33 -05:00
Alexander Meißner 1b45c509c3
Refactor: Use `InstructionContext::get_instruction_data()` (#24014)
* Adds transaction_context and instruction_context where invoke_context.get_keyed_accounts() is used.

* Use instruction_context.get_instruction_data() instead of an explicit parameter.

* Removes instruction_data parameter from Executor::execute().

* Removes instruction_data parameter from ProcessInstructionWithContext.
2022-04-01 15:48:05 +02:00
Jeff Washington (jwash) 9c8dad33c7
add epoch_schedule and rent_collector to hash calc (#24012) 2022-03-31 10:51:18 -05:00
steviez c31db81ac4
Use VoteAccountsHashMap type alias in all applicable spots (#23904) 2022-03-24 12:09:48 -05:00
Jeff Washington (jwash) f1a411c897
add epoch_schedule and rent_collector to hash calc (#23857) 2022-03-24 09:39:22 -05:00
Jeff Washington (jwash) 7b89222fde
don't start extra threads for shrink/clean/hash (#23858) 2022-03-23 11:53:37 -05:00
Jon Cinque 7af48465fa
transaction-status: Add return data to meta (#23688)
* transaction-status: Add return data to meta

* Add return data to simulation results

* Use pretty-hex for printing return data

* Update arg name, make TransactionRecord struct

* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Brooks Prumo cb06126388
Set accounts_data_len on feature activation (#23730) 2022-03-21 12:28:26 -05:00
Tao Zhu 0ed23899e7 directly use compute_budget MAX_UNITS and DEFAULT_UNITS 2022-03-18 08:53:11 -05:00
Brooks Prumo 7ff8c80e25
Add accounts_data_len to bank snapshot (#23714) 2022-03-17 20:14:54 -05:00
behzad nouri 3252dc7203
uses structural sharing for stake-delegations hash-map (#23585)
StakeDelegations is using Arc to implement copy-on-write semantics:
https://github.com/solana-labs/solana/blob/58c0db970/runtime/src/stake_delegations.rs#L14-L16

However a single delegation change will still clone the entire hash-map,
resulting in excessive memory use as observed in:
https://github.com/solana-labs/solana/issues/23061#issuecomment-1063444072

This commit instead uses immutable hash-map implementing structural
sharing:
> which means that if two data structures are mostly copies of each
> other, most of the memory they take up will be shared between them.
https://docs.rs/im/latest/im/
2022-03-16 12:58:05 +00:00
Justin Starry 8c8f9694e0
Refactor: Sanitized transaction creation (#23558)
* Refactor: SanitizedTransaction::try_create optionally computes hash

* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
Jeff Washington (jwash) f05ac7a899
move to get_slots_in_epoch (#23657) 2022-03-14 22:47:45 -05:00
Jack May c68c0e881e
consolidate use-jit flag (#23652) 2022-03-14 15:00:00 -07:00
Brooks Prumo 7758c32035
Banking Stage drops transactions that'll exceed the total account data size limit (#23537) 2022-03-13 15:58:57 +00:00
Brooks Prumo 3c6840050c
Ensure blocks do not exceed the max accounts data size during Replay Stage (#23422) 2022-03-10 10:24:31 -06:00
Jack May 97d40ba3da Resized accounts must be rent exempt 2022-03-02 13:02:02 -08:00
Jeff Washington (jwash) e88da2ec0a
minor test cleanup (#23447)
* minor test cleanup

* fmt
2022-03-02 12:10:05 -06:00
Jeff Washington (jwash) e630eb73d7
rework test to avoid multi-epoch gap (#23393) 2022-03-02 11:12:10 -06:00
Jeff Washington (jwash) ef8b7d9c62
ledger tool halt at slot verify hash (#23424) 2022-03-02 11:11:18 -06:00
Jeff Washington (jwash) 454e82683e
refactor rent test (#23392) 2022-03-01 08:47:09 -06:00
Jeff Washington (jwash) 6b2683f7da
refactor test (#23384) 2022-03-01 08:22:10 -06:00
Tyera Eulberg 19448ba078
Allow sub-rent-exempt-minimum transfers to `1nc1nerator` (#23382)
* Add failing test

* Allow small burns to incinerator

* Use check_id method
2022-02-28 23:56:34 -07:00
Tyera Eulberg 36484f4f08
Prevent new RentPaying state created by paying fees (#23358)
* Add failing test

* Check fee-payer rent-state change on load

* Add more test cases

* Review comments
2022-02-26 12:10:01 -07:00
Jeff Washington (jwash) c0d0724be0
clippy (#23310) 2022-02-23 18:07:00 -06:00
Justin Starry d0e85c293f
Fix rustfmt check (#23296) 2022-02-23 16:38:53 +08:00
Gavin Chan 20d031e2b8
Refactor ExecuteTimings w/ enum-indexed array (#23085) 2022-02-22 14:46:56 -08:00
Justin Starry bcda74f42f
Fix builtin handling on epoch boundaries (#23256) 2022-02-22 20:54:08 +08:00
Tyera Eulberg 7e08ae1d0c
Revert "Add simulation detection countermeasure (#22880)" (#23261)
This reverts commit c42b80f099.
2022-02-21 21:15:37 +00:00
buffalu 70ebab2c82
Add rustfmt.toml and `cargo fmt` (#23238)
* fmt

* formatted

Co-authored-by: Lucas B <buffalu@jito.network>
2022-02-19 13:32:29 +08:00
Justin Starry 1719d2349f
Skip adding builtins if they will be removed (#23233)
* Add failing test for precompile transition

* Skip adding builtins if they will be removed

* cargo clean

* nits

* fix abi check

* remove workaround

Co-authored-by: Jack May <jack@solana.com>
2022-02-18 18:36:59 -08:00
Jack May 970f543ef6
Precompiles owned by the native loader (#23237) 2022-02-18 15:41:59 -08:00
Alexander Meißner da00b39f4f
Cleanup: get_program_key() and get_loader_key() in TransactionContext (#23191)
* Moves TransactionContext::get_program_key() to InstructionContext::get_program_key().

* Removes TransactionContext::get_loader_key().

* Test full program and loader executable account chain in BPF loader.
2022-02-17 10:16:28 +01:00
Justin Starry 813725dfec
Fix ed25519 builtin program handling (#23182)
* Fix ed25519 builtin program handling

* Fix tests

* Add integration tests for processing transactions with ed25519 ixs

* Fix another test

* fix formatting
2022-02-16 18:48:55 +00:00
Michael Vines c42b80f099
Add simulation detection countermeasure (#22880)
* Add simulation detection countermeasures

* Add program and test using TestValidator

* Remove incinerator deposit

* Remove incinerator

* Update Cargo.lock

* Add more features to simulation bank

* Update Cargo.lock per rebase

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2022-02-15 13:09:59 +01:00
Jack May 3d9874b95a
Add fees to tx-wide caps (#22081) 2022-02-11 16:23:16 -08:00
Brooks Prumo 0a1ab945bc
Move cap_accounts_data_len feature gate only around new error (#23048) 2022-02-10 11:57:00 -06:00
Alexander Meißner 226a71f073
Refactor: Use instruction_trace instead of instruction_context_stack (#22988)
* Moves stack_height of the instruction_trace into InstructionContext.

* Uses nesting_level instead of stack_height internally in transaction_context.rs

* Uses the instruction_trace instead of the instruction_context_stack internally.

* Adds feature gate: record_instruction_in_transaction_context_push.
2022-02-09 20:04:49 +01:00
Jeff Washington (jwash) 205cd4609b
bank::get_rent_collector_from_parent (#23016) 2022-02-09 12:31:58 -06:00
Jeff Washington (jwash) fb47c1f0a3
consolidate Rent construction in test code (#23018) 2022-02-08 21:51:52 -06:00
Brooks Prumo 869cfc9a1c
Return the accounts data len delta after processing messages (#22986) 2022-02-08 19:24:47 -06:00
Brooks Prumo f0f4042680
Put accounts data len updates behind feature gate (#22918) 2022-02-08 08:51:16 -06:00
Justin Starry ba215e94f6
Refactor: Add AccountKeys struct for static and dynamic message keys (#22960) 2022-02-05 20:00:31 +08:00
Alexander Meißner 660f6981c6
Cleanup: TransactionContext (#22910)
* Adds BorrowedAccount::check_sysvar().

* Adds BorrowedAccount::get_data_mut().

* Implements account resizing in BorrowedAccount.

* Exposes is_signer() and is_writable() in InstructionContext.

* Removes AccountMeta and get_instruction_accounts_metas().

* Makes throwing errors in BorrowedAccount optional.

* Removes result return values from BorrowedAccount.
2022-02-03 17:19:42 +01:00
Justin Starry 60af1a4cce
Refactor: Add trait for loading addresses (#22903) 2022-02-03 11:00:12 +00:00
carllin bd1850df25
Return actual committed transactions from process_transactions() (#22802) 2022-02-03 03:56:36 -05:00
Jack May ab02dba96f
Add get_processed_sibling_instruction syscall (#22859) 2022-02-02 16:45:57 -08:00
Will Hickey 75563f6c7b
Reject close of active vote accounts (#22651)
* 10461 Reject close of vote accounts unless it earned no credits in the previous epoch. This is checked by comparing current epoch (from clock sysvar) with the most recent epoch with credits in vote state.
2022-02-02 14:16:24 -06:00
Jeff Washington (jwash) fb95fa68a2
cleanup api (#22845) 2022-01-31 11:39:23 -06:00
carllin 94a5aee484
Add new_from_parent() timings (#22744) 2022-01-27 20:35:13 -05:00
Justin Starry 071e97053f
Perf: Reduce write locks on blockhash queue (#22729)
* Perf: Reduce write locks on blockhash queue

* Add comment about thread safety

* Add comment about write starvation
2022-01-26 16:24:27 +08:00
Jon Cinque fc21af4e6e
spl-associated-token-account: Add feature for new program (#22648)
* spl-associated-token-account: Add feature for new program

* Address feedback
2022-01-24 22:54:41 +01:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Brooks Prumo 9977396d8f
Remove unused fields from Bank (#22491) 2022-01-21 06:03:41 -06:00
Justin Starry 7d34a7acac
Perf: Only check executors cache for executable bpf program ids (#22624)
* Only check executors cache for executable bpf program ids

* switch to native loader check

* clean up tests

* fix tests

* clippy
2022-01-21 18:32:30 +08:00
Justin Starry 7f20c6149e
Refactor: move simple vote parsing to runtime (#22537) 2022-01-20 10:39:21 +08:00
Alexander Meißner b448472037
Refactor: Move `InstructionRecorder` into `TransactionContext` (#22578)
* Moves InstructionRecorder into TransactionContext.

* Adds assertions for number_of_instructions_at_transaction_level.
2022-01-19 22:40:09 +01:00
Justin Starry cc76a73c49
Refactor: move compute budget runtime logic into solana-program-runtime (#22543) 2022-01-17 20:48:00 +08:00
Justin Starry 2c38a9213f
Revert "Refactor: move compute budget runtime logic into solana-program-runtime (#22533)" (#22542)
This reverts commit b27976626a.
2022-01-17 17:43:17 +08:00
Justin Starry b27976626a
Refactor: move compute budget runtime logic into solana-program-runtime (#22533) 2022-01-17 15:52:33 +08:00
Brooks Prumo 70633b5c2f
Add Bank::accounts_data_len to SetRootMetrics (#22509) 2022-01-14 20:00:07 -06:00
Tao Zhu 9c9f2dd5bd port counting vote CUs to block cost (#22477) 2022-01-14 10:50:29 -06:00
Justin Starry cddab635ff
Cleanup activated rent_for_sysvars feature (#22454) 2022-01-14 20:34:09 +08:00
Michael Vines 4c577d7f8c `Bank::get_fee_for_message` is now nonce aware 2022-01-13 17:27:38 -08:00
Jeff Washington (jwash) 7711cd74c3
partition_from_pubkey (#22430)
* Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)"

This reverts commit 8aa3d690b5.

* partition_from_pubkey
2022-01-13 17:02:42 -06:00
Justin Starry 2370e61431
Perf: Store deserialized sysvars in the sysvars cache (#22455)
* Perf: Store deserialized sysvars in sysvars cache

* add bench
2022-01-13 05:36:21 +00:00
Justin Starry 7171c95bdd Refactor: move sysvar cache to new module 2022-01-12 12:35:28 -07:00
Tyera Eulberg 3ca16de851
Only examine explicit tx accounts for rent state (#22441)
* Add failing test

* Fix: only examine accounts explicitly included in a tx
2022-01-11 20:23:51 -07:00
Brooks Prumo c45dde6164
Handle accounts data size changes due to rent-collected accounts (#22412) 2022-01-11 17:20:28 -06:00
Tyera Eulberg 637e366b18
Prevent rent-paying account creation (#22292)
* Fixup typo

* Add new feature

* Add new TransactionError

* Add framework for checking account state before and after transaction processing

* Fail transactions that leave new rent-paying accounts

* Only check rent-state of writable tx accounts

* Review comments: combine process_result success behavior; log and metrics before feature activation

* Fix tests that assume rent-exempt accounts are okay

* Remove test no longer relevant

* Remove native/sysvar special case

* Move metrics submission to report legacy->legacy rent paying transitions as well
2022-01-11 11:32:25 -07:00
Brooks Prumo 9bc2592da1
Use lazy_rent_collection directly (#22410) 2022-01-10 10:47:08 -06:00
Trent Nelson 3b4aad9df1 bank: fix executor cache metrics 2022-01-09 04:22:39 +00:00
Brooks Prumo d90d5ee9b6
Refactor Rent::due() with RentDue enum (#22346) 2022-01-08 09:03:46 -06:00
Jack May 4a9f4e2505 improve multi executor cache addition 2022-01-08 03:47:23 -07:00
Trent Nelson ad3cb0bc93 --amend 2022-01-08 08:19:27 +00:00
Trent Nelson 4ce48307bb bank: prime new executor cache entry use-counts 2022-01-08 08:19:27 +00:00
Trent Nelson 6d76db1de5 bank: Add executors cache metrics 2022-01-07 16:53:27 -07:00
behzad nouri c2389fc209
removes CowCachedExecutors (#22343)
Copy-on-write semantics for cached executors can be implemented by a
simple Arc<CachedExecutors> as opposed to CowCachedExecutors:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L244-L247

This will also avoid the need for double locking as in:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3490-L3491
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3525-L3526
2022-01-07 14:06:29 +00:00
Justin Starry 52d12cc802
Add runtime support for address table lookups (#22223)
* Add support for address table lookups in runtime

* feedback

* feedback
2022-01-07 11:59:09 +08:00
Jack May f1e2598baa
Retain executor cache counts (#22322) 2022-01-06 08:56:00 -08:00
Trent Nelson 72fc6096a0 Use saturating_add_assign macro 2022-01-06 03:56:46 -07:00
Carl Lin b25e4a200b Add execute metrics 2022-01-06 03:56:46 -07:00
Trent Nelson 7d32909e17 move `ExecuteTimings` from `runtime::bank` to `program_runtime::timings` 2022-01-06 03:56:46 -07:00
Jack May 12e160269e
cache executors on failed transactions (#22308) 2022-01-05 22:09:03 -08:00
Justin Starry f2ed6f09ee Skip updating already cached executors if unmodified 2022-01-05 16:08:10 -07:00
Alexander Meißner 9f63493789
Refactor: Remove KeyedAccounts (2) (#22274)
* Adds InstructionContext::get_signers().
Improves error messages when modifying borrowed accounts.

* Removes keyed_accounts from InvokeContext tests.

* Removes keyed_accounts from message_processor.rs

* Removes keyed_accounts from bank.rs

* Removes keyed_accounts from bpf serialization.
2022-01-05 09:39:37 +01:00
Justin Starry 45458e7139
Refactor: Improve type safety and readability of transaction execution (#22215)
* Refactor Bank::load_and_execute_transactions

* Refactor: improve type safety of TransactionExecutionResult

* Add enum for extra type safety in execution results

* feedback
2022-01-05 10:15:15 +08:00
Brooks Prumo 1460f00e0f
Consume from AccountsDataMeter (#21994) 2022-01-04 10:00:21 -06:00
Justin Starry 2b5e00d36d
Limit number of accounts that a transaction can lock (#22201) 2022-01-04 14:25:23 +08:00
Alexander Meißner 73e6038986
Refactor: Remove `KeyedAccount` from program runtime (#22226)
* Makes error handling in BorrowedAccount optional.
Adds BorrowedAccount ::get_rent_epoch().
Exposes InstructionContext::get_index_in_transaction().
Turns accounts and account_keys into pinned boxed slices.

* Introduces "unsafe" to InvokeContext::push().

* Turns &TransactionContext into &mut TransactionContext in InvokeContext.

* Push and pop InstructionContext in InvokeContext.
Makes test_process_cross_program and test_native_invoke symmetric.
Removes the borrow check from test_invoke_context_verify.

* Removes keyed_accounts from prepare_instruction()

* Removes usage of invoke_stack.

* Removes keyed_accounts from program-test.

* Removes caller_write_privileges.

* Removes keyed_accounts from BPF parameter (de-)serialization.
2022-01-03 23:30:56 +01:00
carllin 33d0b5e011
Revert "Count compute units even when transaction errors (#22059)" (#22174)
This reverts commit eaa8c67bde.
2021-12-30 02:42:32 -05:00
Justin Starry b1d9a2e60e
Don't forward packets received from TPU forwards port (#22078)
* Don't forward packets received from TPU forwards port

* Add banking stage test
2021-12-29 19:34:31 +01:00
carllin eaa8c67bde
Count compute units even when transaction errors (#22059) 2021-12-28 17:05:11 -05:00
Tyera Eulberg 422a095647
Add (preflight) simulation to BanksClient (#22084)
* Add more-legitimate conversion from legacy Transaction to SanitizedTransaction

* Add Banks method with preflight checks

* Expose BanksClient method with preflight checks

* Unwrap simulation err

* Add Bank simulation method that works on unfrozen Banks

* Add simpler api

* Better name: BanksTransactionResultWithSimulation
2021-12-28 19:25:46 +00:00
Brooks Prumo e3b20c443a
Use load_accounts_data_len() instead of Atomic .load() (#22144) 2021-12-28 13:38:20 +00:00
Brooks Prumo 800472ddf5
Add AccountsDataMeter to InvokeContext (#21813) 2021-12-28 05:14:48 -06:00
Alexander Meißner a06646631c
Feature: TransactionContext, InstructionContext and BorrowedAccount (#21706)
* Adds TransactionContext, InstructionContext and BorrowedAccount.

* Redirects the usage of accounts in InvokeContext through TransactionContext.
Also use the types declared in transaction_context.rs everywhere.

* Adjusts all affected tests.
2021-12-27 18:49:32 +01:00
Alexander Meißner cc947cad03
Refactor: CPI Instruction Recording (#22111)
* Unifies all InstructionRecorders of a transaction into one.

* Stops explicitly compiling CPI instructions for recording,
uses the indices gathered from instruction_accounts instead.
2021-12-25 13:35:43 +01:00