Commit Graph

1663 Commits

Author SHA1 Message Date
Pankaj Garg c86e160438
Add runtime environment to FailedVerification tombstones (#31991)
* Add runtime environment to FailedVerification tombstones

* modify default variant

* prune update

* add DelayVisibility in prune_feature_set_transition
2023-06-06 22:24:39 +00:00
Jeff Washington (jwash) 85a0e31c60
add ProgramExecutionTemporarilyRestricted error (#31796)
* add StakeProgramUnavailable error

* rename to ProgramExecutionTemporarilyRestricted

* Update sdk/src/transaction/error.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* fmt

* update frozen abi hash

* add account_index to ProgramExecutionTemporarilyRestricted error

* Update sdk/src/transaction/error.rs

Co-authored-by: mvines <mvines@gmail.com>

* populate transaction_details from transaction_error for ProgramExecutionTemporarilyRestricted error

* fix a test

* update abi digest

* Update sdk/src/transaction/error.rs

Co-authored-by: Brooks <brooks@prumo.org>

---------

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
Co-authored-by: HaoranYi <haoran.yi@solana.com>
Co-authored-by: HaoranYi <haoran.yi@gmail.com>
Co-authored-by: mvines <mvines@gmail.com>
Co-authored-by: Brooks <brooks@prumo.org>
2023-06-06 14:13:41 -07:00
Alexander Meißner 89207a3fe5
Feature - Prune on feature set transition (#31945)
* Call create_program_runtime_environment() in Bank::apply_builtin_program_feature_transitions().

* Adds LoadedPrograms::prune_feature_set_transition().
2023-06-06 17:49:02 +02:00
Pankaj Garg 37ebb709e6
Evict program cache using transaction usage counter (#31927)
* Evict program cache using tx usage counter

* address review feedback

* address review feedback
2023-06-02 11:26:56 -07:00
Pankaj Garg 29d7a57cce
Reset program cache stats after submit (#31929) 2023-06-02 01:16:49 -07:00
steviez debe794987
Replace improper &Arc<...> with Arc<...> in Bank and Accounts (#31892)
The callstack updated in this PR passed an &Arc<...> down only to have
the bottom level clone the reference. Thus, we are giving shared
ownership so the reference is a bit redundant and arguably obscures the
intention to clone further down the callstack.
2023-05-31 12:36:44 -05:00
Jeff Washington (jwash) 6c520396cd
bank process_new_epoch takes parent_height (#31820) 2023-05-30 08:58:40 -05:00
Jeff Washington (jwash) 0c64e26d53
partitioned rewards helper functions (#31818) 2023-05-30 08:57:31 -05:00
Andrew Fitzgerald 75954dc061
Banking worker (#30970) 2023-05-25 10:16:04 -07:00
Alexander Meißner a85548cf34
Cleanup - spl_token_v3_4_0 and spl_associated_token_account_v1_1_0 (#31764)
* spl_token_v3_4_0

* spl_associated_token_account_v1_1_0

* Marks replace_program_account() as dead code.
2023-05-24 18:45:37 +02:00
Jeff Washington (jwash) e396a10312
change StartBlockHeightAndRewards from parent to current (#31781) 2023-05-23 20:10:51 -07:00
Jeff Washington (jwash) ae12079139
add debug to VoteReward (#31782) 2023-05-23 20:10:20 -07:00
Jeff Washington (jwash) 8e8b2f1671
reformat comments (#31780) 2023-05-23 17:04:47 -07:00
Alexander Meißner 1e8e024ec5
Cleanup - disable_fee_calculator (#31766)
disable_fee_calculator
2023-05-23 18:20:38 +02:00
Alexander Meißner 83f692ce67
Refactor - Share RBPF config and syscall registry globally (#31727)
* Adds LoadedPrograms::program_runtime_environment_v1.

* No need to explicitly remove precompiles from program_accounts_map.
2023-05-20 20:28:00 +02:00
Pankaj Garg e6d490708a
Do not overwrite builtin program in the same slot (#31718) 2023-05-19 05:26:31 -07:00
Alexander Meißner 520c647918
Refactor - Merges `BuiltinPrograms` into `LoadedPrograms` (#31654)
* Merges BuiltinPrograms into LoadedPrograms.

* Prevents built-ins from being pruned.
2023-05-18 22:18:28 +02:00
Jeff Washington (jwash) 0291454876
persist EpochRewardStatus::Active (#31697)
persist `EpochRewardStatus::Active`
2023-05-18 07:14:41 -07:00
Jeff Washington (jwash) ce32cb43db
add bank.epoch_reward_status (#31680) 2023-05-17 10:01:14 -07:00
Pankaj Garg b03422968b
Remove Rc and RefCell to carry program cache in invoke_context (#31684) 2023-05-17 06:10:50 -07:00
Jeff Washington (jwash) 7c6478da8f
remove unnecessary pub (#31676) 2023-05-16 13:53:57 -07:00
Jeff Washington (jwash) 5bc41a0eb8
prepare load_transaction_accounts to recognize reward interval (#31673) 2023-05-16 12:55:35 -07:00
Jeff Washington (jwash) d841ff7fe1
support snapshot deserialization with EpochRewardStatus (#31672)
* support snapshot deserialization with EpochRewardStatus

* pub(crate)
2023-05-16 12:53:52 -07:00
Maximilian Schneider 31aafc8ae6
race condition when running solana-program-test (#31637)
* hold bank freeze lock in banks-server to prevent RC on slot change

* clean up comments and loop

---------

Co-authored-by: Maximilian Schneider <max@mango.markets>
2023-05-15 12:25:06 -07:00
Andrew Fitzgerald 2c869ef778
QoS refactor: Allow pre-filtering (#31542) 2023-05-12 08:53:22 -07:00
Jeff Washington (jwash) 122b05b9f5
pass include_slot_in_hash through hash calcs to allow rehashing if hash is not stored (#31579)
* pass include_slot_in_hash through hash calcs to allow rehashing

* tests use each include_slot_in_hash value

* move include_slot_in_hash

* typo

* reorder struct init

* spelling is hard
2023-05-11 13:23:29 -07:00
Jeff Washington (jwash) e0b914e775
use highest alive root to determine ancient for packing (#31573) 2023-05-11 12:13:15 -07:00
Alexander Meißner bbd8be6cbe
Refactor - Renames loader-v3 to loader-v4. (#31570)
Renames loader-v3 to loader-v4.
2023-05-11 08:29:06 +02:00
Pankaj Garg 04425b81ce
Add stats for LoadedPrograms (#31577) 2023-05-10 12:08:55 -07:00
Jeff Washington (jwash) fed1bb7d54
use max alive root to determine ancient cutoff for clean stats (#31545)
* use max alive root to determine ancient cutoff for clean stats

* fix math and add tests
2023-05-10 08:05:31 -07:00
Alexander Meißner 9e72668a77
Refactor - Simplify built-in program feature transition (#31565)
* Removes builtins::get_builtins(), instead derive them from Bank::builtin_programs.

* Removes BuiltinAction.
Turns BuiltinFeatureTransition from an enum into a struct.

* Makes Bank::remove_builtin() place tombstones.
2023-05-10 16:53:36 +02:00
Pankaj Garg ab89e2cb15
Remove executor cache from the codebase (#31566) 2023-05-10 07:11:04 -07:00
Alexander Meißner 8e9b064ab3
Cleanup - `add_compute_budget_program` and `versioned_tx_message_enabled` (#31560)
* Moves compute_budget_program and address_lookup_table_program into genesis_builtins().

* Removes feature gate of versioned_tx_message_enabled.
2023-05-10 01:14:08 +02:00
Pankaj Garg 2210af60ee
Replace executor cache usage with LoadedPrograms cache (#31462)
* Replace executor cache usage with LoadedPrograms cache

* clippy fixes

* update cache with updated programs

* fixes

* more cleanup

* update tx batch cache with the tx results

* address review comments

* handle program closing backward compatibility

* handle unloaded programs during extraction
2023-05-09 15:44:38 -07:00
Pankaj Garg 3845423e17
Generalize matching criteria for tombstone at boot up (#31547)
Generalize matching criteria for tombstone at bootup
2023-05-09 08:26:42 -07:00
behzad nouri 8e638b785a
removes feature gate code sending votes to tpu-vote-port (#31529) 2023-05-08 18:12:35 +00:00
steviez e62e4fbabd
Replace inc_new_counter_* with datapoint_* in Bank (#31495)
Counters can only report a single field; this adds an extra step to
debugging efforts to figure out the context for where/why the counted
event occurred. Use datapoints instead to subtmit some extra context.
2023-05-05 16:33:35 -05:00
Andrew Fitzgerald 18cd4311af
remove counters in hotpath (#31398) 2023-05-04 09:36:28 -07:00
Pankaj Garg 96e170bd37
Do not evict unloaded programs (#31465)
* Do not evict unloaded programs

* cleanup
2023-05-04 07:49:54 -07:00
Jeff Washington (jwash) ddd1f2fce0
remove unnecessary casts (#31461) 2023-05-03 08:54:25 -07:00
Pankaj Garg ba6f7e80e8
Remove obsolete code for lazily loading programs (#31395) 2023-05-02 14:39:27 -07:00
Alexander Meißner ae75c7cb3f
Refactoring - Remove redundant definition of `BuiltinProgram` (#31429)
* Replaces BuiltinProgram in the program-runtime with the one from solana_rbpf.

* Adjusts the runtimes built-ins to use Arc<LoadedProgram>.

* Adjusts the tests and benchmarks.
2023-05-02 23:01:28 +02:00
Jeff Washington (jwash) 58ce19446f
replace tuple with `VoteReward` struct (#31408) 2023-05-01 10:42:05 -07:00
Pankaj Garg 8eebf64464
Add a second level cache to contain tx batch specific programs (#31413)
* Add a second level cache to contain tx batch specific programs

* fix clippy

* address review comments
2023-05-01 09:09:43 -07:00
Dmitri Makarov 74bd5f87af
Add minor extensions to Bank API (#31396)
* Make load_program method of Bank public for use in ledger-tool

* Add an accessor method to get a reference to builtin_programs of Bank

* Add a parameter to Bank::load_program to control debugging_features
2023-04-28 19:02:49 -04:00
Pankaj Garg 94dc8fed55
Check program modification slots during cold start (#31331) 2023-04-28 06:22:14 -07:00
Brennan 685ebcb3b0
Rework bank to allow RentCollectionCycleParams private (#31361) 2023-04-26 14:48:51 -07:00
Brennan a540690778
Move partition logic out of bank (#31343)
* Move partition logic out of bank
2023-04-26 12:37:02 -07:00
Alexander Meißner 7002c568fb
Cleanup - Removes the CLI option "no-bpf-jit" (#31329)
* Enables JIT everywhere by default.

* Removes the CLI argument "no-bpf-jit" and its plumbering through out the validator code base.

* Removes with_jit bpf_loader variants.

* Removes the to_builtin!() macro.
2023-04-25 19:04:11 +02:00
Alexander Meißner 4b0e16d06f
Refactor - `BuiltinPrograms` (#31324)
* Moves BuiltinProgram of the program runtime into its own file.

* Unifies the runtimes Builtin and program runtimes BuiltinProgram definitions.

* Moves BuiltinPrograms from bank.rs into the program runtime.
2023-04-24 19:01:40 +02:00
Alexander Meißner 8162358b8b
Refactor - Moves the system program into its own crate (#31244)
* Moves the system program into its own crate in "programs/system".

* Moves bank tests back into the runtime to resolve cyclic dependency.
2023-04-19 12:58:08 +02:00
Trent Nelson f34a6bcfce
runtime: transpose `VoteAccount::vote_state()` return to improve ergonomics (#31256) 2023-04-18 14:48:52 -06:00
Brennan 2164a50d00
Move BankIncrementalSnapshotPersistence (#31236)
* Move BankIncrementalSnapshotPersistence

* Update bank serialize ABI digest
2023-04-18 11:18:17 -07:00
Brennan f5fe2607b6
Make VerifyAccountsHashConfig private (#31235) 2023-04-18 07:31:23 -07:00
Brennan 93d0d25d77
Move rent debit out of bank (#31204)
* Move rent debit out of bank

* Clean up imports and visibility

* Fix imports

* rename public mod rent_debits
2023-04-14 19:41:10 +00:00
Alexander Meißner 8eb31f6cfd
Refactor - Syscalls in RBPF CLI (#31109)
* Makes members of SyscallContext public.
Removes check_aligned and check_size from SyscallContext.

* Replaces InvokeContext::set_syscall_context() in tests with mock_create_vm!().

* Passes SyscallContext directly to InvokeContext::set_syscall_context().

* Merges TraceLogStackFrame into SyscallContext.

* Removes the create_vm!() macro.

* Moves BpfAllocator from bpf_loader into program_runtime.

* Frees BpfAllocator from Rc<RefCell<>>.

* Removes unused code from BpfAllocator.

* Consume CUs for heap before doing the allocation.

* Exposes syscalls in rbpf-cli.

* Adds debugging_features parameter to load_program_from_bytes() and load_program_from_account().

* Removes test_bpf_loader_non_terminating_program() as that is already tested in RBPF.

* Moves stack and heap allocation back onto the program runtime stack.
2023-04-14 15:20:08 +02:00
HaoranYi 1db0683a50
Refactor reward payout code - part 4 (#31159)
extract redeem_reward fn
2023-04-12 10:20:21 -05:00
HaoranYi f2e3a0d821
Refactor reward payout code - part 3 (#31157)
extract calculate points fn
2023-04-12 09:36:23 -05:00
HaoranYi 9e14cdc19f
Refactor reward payout code - part 2 (#31156)
* extract load_vote_stake_accounts fn

* rename
2023-04-12 08:40:08 -05:00
HaoranYi 20d1615ccd
Refactor reward payout code - part 1 (#31155)
type alias
2023-04-11 16:23:52 -05:00
Pankaj Garg 12e3e8b467
Add provisions to unload a program from the cache (#31036) 2023-04-10 16:42:05 -07:00
HaoranYi e12c250cf5
Extract new_epoch processing out of new_bank (#31127)
* extract new epoch processing out of new_bank

* review feedbacks

* remove epoch_schedule param

* use RewardCalcTracer
2023-04-10 16:10:45 -05:00
Andrew Fitzgerald 15011eaa5a
separate check_transaction_age from check_age (#30994) 2023-04-07 09:04:03 -07:00
HaoranYi 684901879d
Refactor squash timing report (#31041)
* typo

* refactor squash timing report

* address review comments
2023-04-05 08:53:49 -05:00
Alexander Meißner a6ad37f3ad
Cleanup - `load_and_get_programs_from_cache()` (#30959)
* Replaces assign_program() by replenish() in load_and_get_programs_from_cache().

* Replaces LoadedProgramEntry by a boolean.

* Only lock the global cache once all missing programs have been loaded

* Factors out call of self.rc.accounts.filter_executable_program_accounts().
2023-03-29 16:11:14 +02:00
Pankaj Garg aebc191c38
Stop loading program accounts if program exists in cache (#30703)
* Stop loading program accounts if program exists in cache

* load accounts for upgradeable programs

* revert loader change to conditionally use program data account

* load instruction accounts

* generate TransactionExecutorCache from loaded programs

* cleanup account_found_and_dep_index variable

* address review comments

* handle tombstones in loader

* unify tombstone constructor

* handle multiple tombstones
2023-03-28 11:49:56 -07:00
Pankaj Garg 0df28f6d51
Increment usage count for loaded programs and call eviction (#30900) 2023-03-27 06:48:05 -07:00
Tao Zhu 3e500d9e92
Feature gate builtin consumes static units during processing instruction (#30702)
* add feature gate
* builtins consume statically defined units at beginning of process_instruction()
* Add new instructionError; return error if builtin did not consume units to enforce builtin to consume units;
* updated related tests
* updated ProgramTest with deactivated native_programs_consume_cu feature to continue support existing mock/test programs that do not consume units
2023-03-24 11:31:01 -05:00
Tao Zhu 21c287a64f
include loaded accounts data size limit in transaction fee calculation (#30659)
* include loaded accounts data size limit in transaction base fee calculation
* citing compute_budget for heap cost;
* update sbf tests

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2023-03-21 18:41:41 -05:00
HaoranYi d646820a6e
Do not log detailed epoch stakes (#30825)
* do not log detailed epoch stakes

* allow dead code

* log stake details at trace level

* no need dead_code

* Update runtime/src/bank.rs

Co-authored-by: Brooks <brooks@prumo.org>

---------

Co-authored-by: Brooks <brooks@prumo.org>
2023-03-21 14:43:06 -05:00
Brooks 836494fc5f
Cleans *up to* the last full snapshot slot in verify_snapshot_bank() (#30818)
Cleans up to the last full snapshot slot in verify_snapshot_bank()
2023-03-21 15:49:49 +00:00
Brooks 989c80342b
Adds plumbing and tests for verify_snapshot_bank() with Incremental Accounts Hash (#30789) 2023-03-19 21:44:38 -04:00
Brooks d3273ba118
Removes unused `ancestors` param from `get_snapshot_storages` (#30773) 2023-03-18 17:17:41 -04:00
Ryo Onodera 05ee06814e
Fix wrongly swapped parent slot and block height (#30744) 2023-03-17 20:07:52 +09:00
Tao Zhu 17ba051e1f
Revert "add default_cost as mandatory field for Builtin (#30639)" (#30749)
This reverts commit 7b95c8e698.
2023-03-16 11:21:06 -05:00
Brooks 8be99c1c7d
Verifies incremental accounts hash in verify_accounts_hash_and_lamports (#30735) 2023-03-16 09:48:06 -04:00
Brooks 65cd55261a
Makes Bank::get_snapshot_hash() aware of incremental accounts hashes (#30719) 2023-03-14 18:03:38 -04:00
Brooks 5e5b7f00a2
Stores capitalization in account hashes map (#30635) 2023-03-13 10:50:45 -04:00
Brooks 17b48edd7b
Renames types to be consistent with verify_accounts_hash (#30674) 2023-03-10 20:02:14 -05:00
Brooks 63623e4df2
Renames fns to verify_accounts_hash (#30672) 2023-03-10 18:11:47 -05:00
Brooks 64f583b37d
Improves logs in bank verification at startup (#30671)
* Improves logs in bank verification at startup

* pr: should_clean
2023-03-10 21:01:49 +00:00
Tao Zhu 7b95c8e698
add default_cost as mandatory field for Builtin (#30639)
* add default_cost as mandatory field for Builtin

* updated tests

* set zkp program default to VerifyTransfer CUs

---------

Co-authored-by: Jon Cinque <joncinque@pm.me>
2023-03-10 14:02:24 -06:00
Alessandro Decina 13107b4eb6
Enable JIT in Bank::new_with_config_for_tests() (#30661)
The most common validator configuration is JIT on, so test that. On
platforms that don't support JIT we'll still fallback on the
interpreter.
2023-03-10 22:52:33 +11:00
Ryo Onodera 0b9c040fcf
Add .codecov.yml with checks disabled (#30666)
* Add .codecov.yml with checks disabled

* Trigger test-coverage.sh...
2023-03-10 20:11:22 +09:00
Tao Zhu 3b9438fd49
Replace hardcoded loaded accounts size limit with compute budget instruction (#30506)
1. replace hardcoded loaded accounts data size limit with compute budget instruction;
2. new transaction error for invalid account data size limit
3. test requested limit with combination of features and transactions
2023-03-08 19:41:45 -06:00
Jeff Washington (jwash) 85eaf1e7ab
improve BankCreationFreezingProgress and add tests (#30513) 2023-02-27 07:54:08 -08:00
Tao Zhu 9ec5e9b866
refactor to remove additional is_simple_vote check (#30521) 2023-02-24 23:49:04 -06:00
Brooks 64c7b401cd
BankIncrementalSnapshotPersistence uses strong accounts hash types (#30502) 2023-02-24 18:15:23 -05:00
Tao Zhu 66ea750182
add compute budget instruction to set loaded accounts data size limit (#30377)
* add compute budget instruction to set accounts data size limit

* changes names to explicitly for loaded accounts data size
2023-02-24 09:27:49 -06:00
Brooks b7d31e134c
Refactors Accounts::new_from_parent() (#30475) 2023-02-23 16:50:51 -05:00
Jeff Washington (jwash) c032dc2754
add bank progress stats (#30405)
* add bank progress stats

* handle more bank creation code paths

* use wrapping sub
2023-02-23 06:29:02 -08:00
Pankaj Garg 0e13dcca00
Replenish executor cache eagerly during transaction processing (#30425) 2023-02-23 05:13:39 -08:00
Pankaj Garg b1f5b0d790
Find and load missing programs in LoadedPrograms cache (#30275)
* Find and load missing programs in LoadedPrograms cache

- filter program accounts in a transaction batch
- filter the accounts that are missing in LoadedPrograms cache
- load the programs before processing the transactions
- unit tests

* address review comments

* fix clippy

* address review comments

* fix test

* fix more tests
2023-02-21 12:53:35 -08:00
Brooks 35328ca63d
Makes AccountsHash an enum (#30416) 2023-02-21 15:20:51 -05:00
Pankaj Garg 3a4ba72daf
Remove executors trait and replace with LoadedProgram (#30348)
* Remove executors trait and replace with LoadedProgram

* fill in deployment slot

* address review comments

* fix clippy warnings

* address review comments

* fix failures caught by sbf tests
2023-02-17 12:47:45 +01:00
HaoranYi 615d1a8b69
Extract BankHashLamportsVerifyConfig (#30320)
* refactor BankHashLamportsVerifyConfig

* clippy

* comments

* fix bench
2023-02-15 16:03:50 -06:00
Alexander Meißner 6558c8fdc9
Refactor - Delay visibility of program un-/re-/deployment (#29654)
* Use three separate HashMaps instead of the enum TxBankExecutorCacheDiff.

* Replaces all places which deploy programs by a macro.

* Adds a feature gate.

* Adjust tests.

* Makes undeployment visible immediately.
2023-02-11 11:18:25 +01:00
Brooks b3c2256799
Refactors a sort's comparator fn (#30243) 2023-02-10 15:41:55 -05:00
Brooks d27c860249
Replaces stable sort with unstable sort (#30223) 2023-02-10 09:56:11 -05:00
Ryo Onodera 3e6162e69e
Add address lookup tables to minimized snapshot (#30158)
* Add address lookup tables to minimized snapshot

* Add comment for future posterity

* Add reference to the issue

* Adjust comment a bit

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-02-10 14:46:02 +09:00
Pankaj Garg 375f9ae41d
LoadedPrograms cache implementation and tests (#30139) 2023-02-08 13:24:44 -08:00
HaoranYi 3f913e49fc
refactor rent payment order cmp (#30132) 2023-02-07 09:30:48 -06:00
Tyera a14473eb54
Prevent Rent-reward recipients from ending up RentPaying (#30130) 2023-02-06 12:16:36 -07:00
Tao Zhu 4293f11cf1
feature gate to enable compute_budget::request_heap_frame on mainnetBeta (#30077) 2023-02-03 09:21:57 -06:00
Brennan 8312141f87
Move bank tests to separate file (#29912) 2023-02-02 16:32:22 -08:00
Tao Zhu 090b990e15
remove congestion multiplier from transaction fee calculation (#29828) 2023-02-01 16:26:50 -06:00
Jeff Washington (jwash) 1c4e84b8ca
impl default version of contains_multiple_slots (#30055) 2023-02-01 13:01:01 -08:00
Brooks d048a1903f
Splits up AccountsDb::bank_hashes (#30024) 2023-02-01 14:32:35 -05:00
Tao Zhu a5af54669a
Limit loaded data per transaction to a fixed cap (#29743) 2023-01-31 22:51:35 -06:00
Jeff Washington (jwash) 5e35823b66
add test_stake_account_consistency_with_rent_epoch_max_feature (#29915)
* add test_stake_account_consistency_with_rent_epoch_max_feature

* create_stake_account takes id

* use test_case

* reformat panic message
2023-01-27 11:50:33 -08:00
Brooks 867d2581a6
Moves impl of bank_hash_info_at() into hash_internal_state() (#29913) 2023-01-25 22:24:51 -05:00
Brooks 655345728b
Use saturating_add_signed() (#29867) 2023-01-24 17:07:50 -05:00
Brooks a06f8e68a5
Tests wait for initial accounts hash verification to complete (#29840) 2023-01-24 14:44:39 -05:00
Tyera b35d173c04
Remove bank_transaction_count_fix logic (#29695)
* Remove bank_transaction_count_fix logic

* Fix test and add comment
2023-01-22 15:59:35 +08:00
behzad nouri 272e667cb2
deprecates Pubkey::new in favor of Pubkey::{,try_}from (#29805)
The commit deprecates Pubkey::new which lacks type-safety and instead
implements TryFrom<&[u8]> and TryFrom<Vec<u8>> for Pubkey.
2023-01-21 18:06:27 +00:00
Brooks db10073c36
Promotes accounts delta hash to its own type (#29768) 2023-01-20 14:23:56 -05:00
Brooks 6baab92ab5
Refactors calculate_capitalization() (#29790) 2023-01-20 13:38:41 -05:00
Jeff Washington (jwash) 9d2c71b9a3
remove type SnapshotStoragesOne (#29750) 2023-01-18 11:51:08 -08:00
Tao Zhu 1d88654116
Revert "Cap accounts data a transaction can load by its requested limit" (#29373)
Revert "Cap accounts data a transaction can load by its requested limit (#27840)"

This reverts commit 81dc2e56ac.
2023-01-17 11:34:18 +01:00
Jeff Washington (jwash) 9d1c0c5a3c
get_slot_stores -> get_slot_storage_entry (#29663) 2023-01-12 04:08:08 -08:00
Brennan Watt 69c4db27c5
Configurable hashes per tick (#29659) 2023-01-11 16:56:11 -08:00
Jeff Washington (jwash) 544b9745c2
snapshot storage path uses 1 append vec per slot (#29627) 2023-01-11 12:05:15 -08:00
steviez 28071d5bf4
chore: Move new bank metric into metric submodule (#29628)
More cleanup in moving non-essential items out of bank.rs
2023-01-11 13:14:46 -06:00
HaoranYi d693167dfb
Feature: filter stake by min delegation for rewards (#29618)
filter stake by min delegation for rewards
2023-01-11 08:55:18 -06:00
steviez 1c1c8fb008
chore: Move bank new epoch metrics into separate file (#29599)
chore: Move bank new epoch metric into separate file

bank.rs is very large, so move the metrics reporting code into a
separate file. This reduces line count and also makes the actual logic
in bank.rs more readable by removing a big block of boilerplate code.

There are more metrics that we can do this for, but doing them one at a
time to avoid large diffs.
2023-01-10 13:15:50 -06:00
Alexander Meißner db277d320b
Adds TX signature to `TransactionContext` in debug mode (#29597)
Adds TX signature to TransactionContext in debug mode.
2023-01-10 00:54:26 +01:00
Jeff Washington (jwash) a703019caa
feature: don't do rewrites in rent collection (#26491)
* feature: don't do rewrites in rent collection

* modify test to specifically test this feature
2023-01-09 08:14:17 -08:00
behzad nouri 5c9beef498
fixes errors from clippy::useless_conversion (#29534)
https://rust-lang.github.io/rust-clippy/master/index.html#useless_conversion
2023-01-05 18:05:32 +00:00
Lijun Wang 1e8a8e07b6
Stream the executed transaction count in the block notification (#29272)
Problem

The plugins need to know when all transactions for a block have been all notified to serve getBlock request correctly. As block and transaction notifications are sent asynchronously to each other it will be difficult.

Summary of Changes

Include the executed transaction count in block notification which can be used to check if all transactions have been notified.
2023-01-05 09:36:19 -08:00
Illia Bobyr d7bd1bf970
bank: Record non-vote transaction count (#29383)
A subsequent change to `SamplePerformanceService` introduces non-vote transaction counts, which `bank`s need to store.

Part of work on https://github.com/solana-labs/solana/issues/29159
2023-01-03 14:46:20 -08:00
Jeff Washington (jwash) 4cc1890f00
skip_rewrites will only be feature driven (#29468) 2023-01-02 20:15:42 -08:00
Jeff Washington (jwash) 64466725d4
remove skip rewrite code from collect_rent_from_accounts (#29472) 2023-01-02 20:15:05 -08:00
Jeff Washington (jwash) d5fdb7bd0a
remove metric time_hashing_skipped_rewrites_us (#29470) 2023-01-02 14:03:04 -08:00
Jeff Washington (jwash) 67c7e343a2
test permutations of set_exempt_rent_epoch_max (#29461) 2023-01-02 11:59:50 -08:00
Jeff Washington (jwash) cf1aa4beea
feature: set rent_epoch to Epoch::MAX (#28690)
* check android builds

* feature: set rent_epoch to Epoch::MAX

* tweaks

* Update runtime/src/rent_collector.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* simplify changes to tests

* back out some test changes

* calculate_rent_result passes through Exempt

* move calc outside loop

* if rent epoch is already max, use 'NoRentCollectionNow'

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2023-01-02 07:51:08 -08:00
Tyera 1e0d3931fd
Fix "tranaction" typo in code base (#29347)
Fix typos
2022-12-21 18:02:36 -07:00
Jeff Washington (jwash) 3e649d2aa0
remove add_root(caching_enabled) (#29245) 2022-12-13 11:26:28 -08:00
Jeff Washington (jwash) b779134020
remove process_stale_slot_with_budget (#29236) 2022-12-13 07:50:14 -08:00
Jeff Washington (jwash) 5983363b03
remove test_store_scan_consistency(caching_enabled) (#29233) 2022-12-13 07:18:55 -08:00
Jeff Washington (jwash) ceeec9d625
remove enable_caching from Bank::new_with_config_for_tests (#29215) 2022-12-12 15:33:12 -08:00
Jeff Washington (jwash) 2c2324f4ea
remove caching_enabled from Bank::new_with_paths_for_tests (#29214) 2022-12-12 15:30:46 -08:00
Jeff Washington (jwash) dd4393e46d
remove accounts_caching from Bank::new_with_paths (#29213) 2022-12-12 15:18:12 -08:00
Jeff Washington (jwash) 536b879aa7
remove accounts.new_with_config_for_tests.caching_enabled (#29168) 2022-12-12 11:31:51 -08:00
Jeff Washington (jwash) 7b1d00ee80
remove dead just_rewrites (#29188) 2022-12-12 10:04:16 -08:00
Jeff Washington (jwash) db2cc53967
remove skip rewrites from bank (#29182) 2022-12-10 09:27:15 -08:00
Brooks Prumo 5f74fb9c87
Removes erroneous duplicate `#[ignore]` (#29193) 2022-12-09 20:58:22 -05:00
Jeff Washington (jwash) f9c4170d34
remove caching_enabled (#29117)
* warp_from_parents works with write_cache enabled

* remove caching_enabled
2022-12-09 14:31:54 -08:00
Jeff Washington (jwash) 631a98a3b6
warp_from_parents works with write_cache enabled (#29185) 2022-12-09 14:28:18 -08:00
Jeff Washington (jwash) 04c10a9382
panic on process_stale_slot_with_budget and remove test (#29045) 2022-12-09 13:25:34 -08:00
Jeff Washington (jwash) a3c95eaa53
remove BankTestConfig.caching_enabled (#29173) 2022-12-09 11:52:36 -08:00
Jeff Washington (jwash) ec5098a723
remove bank_test_config_caching_enabled (#29170) 2022-12-09 08:28:02 -08:00
Jeff Washington (jwash) 2f4731570a
ledger-tool create-snapshot uses write cache (#29119)
* ledger-tool create-snapshot uses write cache

* ledger tool create snapshot works with write cache
2022-12-08 06:09:52 -08:00
Jeff Washington (jwash) 019f276a91
test_shrink_candidate_slots_cached works with write cache (#29143) 2022-12-08 06:08:31 -08:00
behzad nouri 9524c9dbff patches errors from clippy::uninlined_format_args
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
Jeff Washington (jwash) b4075a0e29
bank tests use write cache (#28885) 2022-12-05 14:08:28 -08:00
Jeff Washington (jwash) 293a5860b8
test_process_stale_slot_with_budget never uses write cache (#29076) 2022-12-05 11:40:14 -08:00
Jeff Washington (jwash) 1036d4409b
test_verify_snapshot_bank works without write cache (#29075) 2022-12-05 11:00:01 -08:00
Brooks Prumo c65a8ce6c3
AccountsDb::get_snapshot_storages() takes a Range for slots (#29054) 2022-12-05 11:21:21 -05:00
Jeff Washington (jwash) babc241345
tests use write cache (#29027)
* test_add_precompiled_account uses write cache

* test_add_precompiled_account_squatted_while_not_replacing uses write cache
2022-12-02 10:26:51 -08:00
Jeff Washington (jwash) 5104f4ef0d
test_bank_update_vote_stake_rewards works with write cache (#29044) 2022-12-02 10:19:43 -08:00
Jeff Washington (jwash) 00ba78bd2d
test_bank_update_rewards_determinism works with write cache (#29043) 2022-12-02 09:56:58 -08:00
Jeff Washington (jwash) 6852d5768a
test_add_builtin_account* works with write cache (#29030) 2022-12-02 09:48:21 -08:00
Jeff Washington (jwash) 87a3130226
test_bank_hash_internal_state works with write cache (#29042) 2022-12-02 09:37:22 -08:00
Jeff Washington (jwash) d17a740a99
test_bank_hash_internal_state_verify works with write cache (#29041) 2022-12-02 09:22:42 -08:00
Jeff Washington (jwash) 63e19c0068
do_test_clean_dropped_unrooted_banks uses write cache (#29029) 2022-12-02 08:04:57 -08:00
Jeff Washington (jwash) 911a8d2a4a
test_add_instruction_processor_for_existing_unrelated_accounts works with write cache (#29031) 2022-12-02 08:04:34 -08:00
Jeff Washington (jwash) 20aeff3632
test_purge_empty_accounts uses write cache (#29018) 2022-12-01 15:22:26 -08:00
Jeff Washington (jwash) dbd6ce8f21
test_bank_update_sysvar_account uses write cache (#29022)
* test_add_precompiled_account_inherited_cap_while_replacing uses write cache

* test_bank_update_sysvar_account uses write cache
2022-12-01 15:21:05 -08:00
Jeff Washington (jwash) a9c8f26a66
test_add_precompiled_account_inherited_cap_while_replacing uses write cache (#29019) 2022-12-01 14:07:39 -08:00
Brooks Prumo 9327658007
Promotes accounts hash to a strong type (#28930) 2022-11-28 10:09:47 -05:00
Brooks Prumo 04016e3bcf
Don't wait for EAH unless feature is enabled (#28938) 2022-11-23 09:11:28 -05:00
Jeff Washington (jwash) 5d88a9b32b
improve perf of storing accounts for shrink/ancient (#28880)
* improve perf of storing accounts for shrink/ancient

* rename

* phantom data

* update comment

* make impl Borrow<Hash> consistent

* remove unused static
2022-11-22 15:36:57 -08:00
Jeff Washington (jwash) f22104d46b
use write cache by default in some tests (#28876) 2022-11-18 14:35:52 -08:00
Brooks Prumo 7787cb2ab2
Renames BankHashInfo field to accounts_delta_hash (#28878) 2022-11-18 15:35:03 -05:00
Justin Starry 7371608722
Support versioned transactions in program test framework (#28739)
* Support versioned transactions in program test framework

* use working bank

* Update to process_transaction_with_metadata

* Migrate client apis from Transaction to Into<VersionedTransaction>

* feedback
2022-11-18 23:43:52 +08:00
Trent Nelson 53a579bed1 ledger-tool: stream output of `accounts` subcommand 2022-11-17 12:45:26 -07:00
K-anon e8c8235474
Get Index Key Size RPC Support (#28383)
Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2022-11-16 20:39:11 -07:00
Tao Zhu ae48ac97dd
fix account resize test by requesting max tx data size (#28826)
* fix account resize test by requesting max tx data size
* define data size limit in incremental of 1024
2022-11-16 17:52:15 -06:00
Brooks Prumo c1e440acb6
Warping calculates and sets new EAH (#28809) 2022-11-15 23:16:38 -05:00
behzad nouri 28956d7653
removes feature gate code preventing crediting accounts that end rent paying (#28818) 2022-11-15 19:55:34 +00:00
Brooks Prumo d798e751a0
Disables EAH with short epochs (#28803) 2022-11-15 13:26:19 -05:00
Brooks Prumo ac9b11762a
Lowers test's account data size to avoid hitting MaxLoadedAccountsDataSizeExceed (#28814) 2022-11-15 16:52:35 +00:00
Jeff Washington (jwash) 84cfdf23fc
remove LoadedAccount::stored_size (#28802) 2022-11-15 05:47:12 -08:00
Brooks Prumo 503da50f2c
Verifies EAH in SnapshotHash (#28775) 2022-11-14 16:02:08 -05:00
Brooks Prumo 0bfea02056
Snapshots wait for EAH calculations to complete (#28777) 2022-11-14 11:34:44 -06:00
Tao Zhu 81dc2e56ac
Cap accounts data a transaction can load by its requested limit (#27840)
- Add new compute-budget instruction to set transaction-wide accounts data size limit
- Set default accounts data limit to 10MB, and max to 100MB, per transaction;
- Add getters to make changing default and/or max values easier in the future with feature gates;
- added error counter for transactions exceed data size limit
2022-11-14 10:29:35 -06:00
Alexander Meißner 4142f42319
Refactor - Move `Executor` in program-runtime crate (#28782)
* Moves CreateMetrics into the program-runtime crate.

* Moves the Executor trait into executor.rs

* Removes the first_instruction_account parameter from Executor::execute().
2022-11-10 21:11:00 +01:00
Brooks Prumo d1ba42180d
clippy for rust 1.65.0 (#28765) 2022-11-09 19:39:38 +00:00
Brooks Prumo 0a62f01cab
Removes EAH string from datapoint metric (#28724) 2022-11-02 14:14:08 -04:00
behzad nouri 17680fe837
removes feature gate code preserving rent_epoch for rent exempt accounts (#28507)
* removes feature gate code preserving rent_epoch for rent exempt accounts

* update test expected hashes

Co-authored-by: jeff washington <jeff.washington@solana.com>
2022-11-01 13:36:59 +00:00
Brooks Prumo 59bf1809fe
Uses SnapshotHash type in snapshot archive fields (#28681) 2022-10-31 14:28:35 -04:00
Brooks Prumo bfa62da29e
Renames variant to CalcAccountsHashDataSource::IndexForTests (#28623) 2022-10-29 08:55:33 -04:00
Brooks Prumo 2ebf248f47
Renames fn to Bank::update_accounts_hash() (#28619) 2022-10-28 17:31:16 -04:00
Brooks Prumo 5a3d252899
Renames fn to Bank::update_accounts_hash_for_tests() (#28620) 2022-10-28 14:33:05 -04:00
steviez 7b70aef33c
Add help text to genesis creation time assert (#28629)
If a validator starts with a snapshot from the wrong cluster, an assert
will trigger complaining about the genesis_creation_time encoded in the
snapshot differing from the genesis_creation_time embedded in
genesis.bin.

This could be a fairly easy mistake to make so add a help string to the
assertion to point operators in the right direction.
2022-10-27 18:02:36 +02:00
Brooks Prumo 27269d833c
Uses enum for data source with calc_accounts_hash() (#28584) 2022-10-26 13:04:01 -04:00
Justin Starry 2d8665d307
Record inner instruction stack height (#28430)
* Record inner instruction stack height

* fix sbf tests

* feedback
2022-10-26 10:37:44 +08:00
Brooks Prumo 1dbcb78de7
Renames fn to update_accounts_hash() (#28574) 2022-10-25 16:43:19 -04:00
Jeff Washington (jwash) 8a538f8da5
skip_rewrite skips almost all accounts with no rent (#28552) 2022-10-24 11:48:00 -07:00
Ashwin Sekar f207af765e
Split out voting and banking threads in banking stage (#27931)
* Split out voting and banking threads in banking stage

Additionally this allows us to aggressively prune the buffer for voting threads
as with the new vote state only the latest vote from each validator is
necessary.

* Update local cluster test to use new Vote ix

* Encapsulate transaction storage filtering better

* Address pr comments

* Commit cargo lock change

* clippy

* Remove unsafe impls

* pr comments

* compute_sanitized_transaction -> build_sanitized_transaction

* &Arc -> Arc

* Move test

* Refactor metrics enums

* clippy
2022-10-20 21:10:48 +00:00