Commit Graph

1663 Commits

Author SHA1 Message Date
Pankaj Garg 356a827087
Move NonceInfo, NonceFull and NoncePartial out of bank.rs (#32375)
* Move NonceInfo, NonceFull and NoncePartial out of bank.rs

* fix imports

* move test_nonce_info to nonce_info.rs
2023-07-05 10:44:54 -07:00
Ryo Onodera 5b6c3ba81b
Make prepare_simulation_batch() more usable (#32304)
* Make prepare_simulation_batch() more usable

* Rename

* Use `use`
2023-06-29 11:20:42 +09:00
Wen 6f72258e3e
Vote refresh fix when outside slothash (#29948)
* When there are too many pubkeys in one slot, kick the one with lowest
stake out.

* Cache last_root to reduce read locks we need.

* Use slots_in_epoch to limit number of slots in the map.

* Fix lint errors.

* Only cache stake and slots per epoch once per epoch.

* Revert "Only cache stake and slots per epoch once per epoch."

This reverts commit 8658aad0083456794b4c4403adaf9c74d1a71d09.

* Vote at the tip of current fork if last vote is outside SlotHash
of the tip and last vote expired.

* Add unittest when last vote is outside slothash, we should vote at the tip
of the current fork.

* Revert "Use slots_in_epoch to limit number of slots in the map."

This reverts commit 93574f57a48d2a70fbbc0f62fa8810d3b6bee0af.

* Revert "Cache last_root to reduce read locks we need."

This reverts commit bb114ec2b62cb9c0207328b19c415f6116be0f1c.

* Revert "When there are too many pubkeys in one slot, kick the one with lowest"

This reverts commit 711e29a6a025fd4f11fbc97dcbbe90e4832be04c.

* Move new vote generation when last vote is outside slothash into the
main path, this actually makes more sense since we don't select where
to vote in two different places, and all the vote generation logic
is seamlessly inherited.

* - Move vote refresh to be behind select vote and do not refresh vote if a new
  vote is selected.
- Check whether last vote is inside slothash inside select_vote_and_reset_forks
- rename slot_within_slothash to is_in_slothashes_history
- remove one unittest for now, more tests will be added in a separate CL

* Remove new test, it will be in another file.

* Add is_in_slot_hashes_history test in the new file.

* Add unittest for the case when last vote is outside slot hashes.

* Small improvements and more unittests.

* Fix bad merge.

* Update docs/src/terminology.md

Co-authored-by: mvines <mvines@gmail.com>

* Put SwitchForkDecision::FailedSwitchThreshold logic into separate function.

* Make linter happy.

---------

Co-authored-by: mvines <mvines@gmail.com>
2023-06-26 18:21:24 -07:00
Pankaj Garg 089fb464a9
Move message_processor to program-runtime (#32268) 2023-06-24 15:55:39 -07:00
Tyera 02eaaae513
Clean up credits_auto_rewind feature (#32211)
* Remove credits_auto_rewind feature logic from stake program

* Obey clean-up orders

* Clippy prefers this match

* Remove credits_auto_rewind feature helpers from Bank

* Comment nit for language, and shrink to fit on one line
2023-06-24 10:26:27 -06:00
Ryo Onodera f62d231886
Make `ci/_` to print to stderr instead of stdout (#32238)
* make _ print to stderr instead...

* Trigger normal build procedure
2023-06-22 15:46:22 +09:00
steviez 20a7cdd43d
Restrict access to Bank's HardForks (#32180)
* Restrict access to Bank's HardForks

Callers could previously obtain a a lock to read/write HardForks from
any Bank. This would allow any caller to modify, and creates the
opportunity for inconsistent handling of what is considered a valid hard
fork (ie too old).

This PR adds a function to Bank so consistent sanity checks can be
applied; the caller will already have a Bank as that is where they would
have obtained the HardForks from in the first place. Additionally,
change the getter to return a copy of HardForks (simple Vec).

* Allow hard fork at bank slot if bank is not yet frozen
2023-06-20 23:44:43 -05:00
HaoranYi 2e9aeff8dd
Enable partitioned epoch reward by feature id (#32174)
enable partitioned epoch reward feature by feature id
update stake rewards tests with partitioned epoch rewards feature enable

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-20 12:12:50 -05:00
galactus 2ceabd9368
Simd 47 syscall sysvar last restart slot (#31957)
* add sysvar and logic for last restart slot

* cleanup

* add test for getting last restart slot from account

* format code

* add some basic rustdoc

* copy+paste error

* feature flag for last_restart_slot

* add to sysvars.md

* updated wording in sysvars.md

* rename sol_get_last_restart_slot_sysvar > sol_get_last_restart_slot

* create sbf C header for sol_get_last_restart_slot

* cleanup imports

* reverted hardened_unpack workaround

* cleanup imports

* cleanup logs + blank lines

* Implementing ui changes for last restart slot, nit

* Some more nit change and implementing the UI for sysvar

* fixing the CI

* Minor clippy fix

* format changes

* changes suggested by mvines and lichtso

* increase timeout in local_cluster test

* fix code format

* use keypair for feature flag from mvines

* delete test.json file

* Revert "increase timeout in local_cluster test"

This reverts commit a67465ae22.

* last restart slot should be always less than or equal to current slot

* fixing bug

* changes after  steviez comments

* format issue fixed

* fixing the comment on premature application of future hardfork

* nit change in test

Co-authored-by: steviez <steven@solana.com>

* reverting sysvar_cache.rs because change was not necessary

---------

Co-authored-by: steve-gg <grooviegermanikus@gmail.com>
Co-authored-by: steviez <steven@solana.com>
2023-06-16 20:14:02 +00:00
Jeff Washington (jwash) 174ceba82b
rename to stake_rewards_by_partition for clarity (#32165) 2023-06-16 19:28:53 +00:00
HaoranYi 39d9f8974d
Revert "add test for is_partitioned_rewards_feature_enabled (#32158)" (#32176)
This reverts commit d06b099ecb.

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-16 11:56:05 -05:00
HaoranYi d06b099ecb
add test for is_partitioned_rewards_feature_enabled (#32158)
* add set_partitioned_rewards_enabled_for_test fn

* reveiws

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-16 11:07:07 -05:00
HaoranYi af7b7f1436
Refactor reward block calculation fn (#32167)
refactor reward block calculation fn

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-16 08:40:23 -05:00
Jeff Washington (jwash) b1b7ae5e09
cleanup math in distribute_partitioned_epoch_rewards (#32164) 2023-06-15 16:17:38 -07:00
Jeff Washington (jwash) b4265217e6
update begin_partitioned_rewards to include sysvar (#32151)
* update begin_partitioned_rewards to include sysvar

* cleanup
2023-06-15 14:00:24 -07:00
Jeff Washington (jwash) d79b565c72
connect `begin_partitioned_rewards` to bank (#32152) 2023-06-15 11:51:10 -07:00
Jeff Washington (jwash) e7a676dc01
add distribute_partitioned_epoch_rewards (#32124)
* add distribute_partitioned_epoch_rewards

* reviews

* reanme

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-15 11:32:44 -05:00
Jeff Washington (jwash) 988bff93c8
deactivate_epoch_rewards_status burns sysvar (#32143)
* deactivate_epoch_reward_status burns sysvar

* warn to info

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-15 10:36:50 -05:00
Jeff Washington (jwash) 00b5c40122
add partitioned rewards sysvar fns (#32128) 2023-06-14 16:08:15 -07:00
Jeff Washington (jwash) 60e36afd42
add begin_partitioned_rewards (#32116)
* add begin_partitioned_rewards

* pr feedback

* Apply suggestions from code review

Co-authored-by: Brooks <brooks@prumo.org>

---------

Co-authored-by: HaoranYi <haoran.yi@gmail.com>
Co-authored-by: Brooks <brooks@prumo.org>
2023-06-14 14:24:49 -07:00
Jeff Washington (jwash) 088ac5ec01
add is_partitioned_rewards_code_enabled (#32123)
* add partitioned_rewards_code_enabled

* renames
2023-06-14 14:36:53 -05:00
Jeff Washington (jwash) f487dfbc77
add distribute_epoch_rewards_in_partition (#32125) 2023-06-14 11:03:17 -07:00
Jeff Washington (jwash) a1ce3cbaa3
partitioned rewards fn for tests: force_reward_interval_end_for_tests (#32127)
partitioned rewards fn for tests
2023-06-14 10:59:48 -07:00
Jeff Washington (jwash) 4eb7db8737
in testing, hash_internal_state ignores epoch_rewards_sysvar (#32122) 2023-06-14 10:57:29 -07:00
Jeff Washington (jwash) 9cf10b7aae
optionally compare with partitioned rewards (#32126) 2023-06-14 10:55:09 -07:00
Jeff Washington (jwash) c01250fc58
add calculate_rewards_and_distribute_vote_rewards (#32109)
* add calculate_rewards_and_distribute_vote_rewards

* warn -> info
2023-06-14 08:38:25 -07:00
Jeff Washington (jwash) a44b080b7f
add partitioned_rewards_feature_enabled (#32115) 2023-06-14 07:47:12 -07:00
Jeff Washington (jwash) 1fa73e778f
add deactivate_epoch_reward_status (#32114) 2023-06-14 07:46:34 -07:00
Jeff Washington (jwash) 2968587d42
add compare_with_partitioned_rewards (#32108) 2023-06-14 07:43:09 -07:00
Jeff Washington (jwash) 67a434fc5b
add calculate_rewards_for_partitioning (#32110) 2023-06-13 15:45:59 -07:00
Jeff Washington (jwash) 65268af497
pass get_reward_interval to load_accounts (#32084) 2023-06-13 14:49:07 -07:00
Jeff Washington (jwash) 9774826b08
add calculate_validator_rewards (#32107) 2023-06-13 14:45:19 -07:00
Jeff Washington (jwash) dcd66534dd
add partitioned rewards helpers (#32085)
* add partitioned rewards helpers

* remove pub(crate)

* credit -> distribution
2023-06-13 13:57:52 -07:00
Jeff Washington (jwash) 0762e69863
remove build error from merge (#32104) 2023-06-13 12:33:40 -07:00
Jeff Washington (jwash) 3fc183ca74
add calculate_stake_vote_rewards for partitioned rewards (#32066)
* add calculate_stake_vote_rewards for partitioned rewards

* check stake reward in test

* pr feedback

* add comment

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-13 11:53:30 -07:00
Jeff Washington (jwash) 446b82afba
update comment (#32099) 2023-06-13 10:54:38 -07:00
Jeff Washington (jwash) 7c5c70a522
add compare_with_partitioned_rewards_results for testing partitioned rewards (#32086) 2023-06-13 07:39:46 -07:00
Jeff Washington (jwash) b89ce94d33
add store_vote_accounts_partitioned (#32061)
* add store_vote_accounts_partitioned

* pr feedback
2023-06-12 14:38:30 -07:00
Alexander Meißner d0ae8c38bd
Fix - `enable_alt_bn128_syscall` (#32083)
* Fixes test_runtime_feature_enable_with_program_cache.

* Adds enable_alt_bn128_syscall to FEATURES_AFFECTING_RBPF.
2023-06-12 22:37:02 +02:00
Jeff Washington (jwash) 6280c63e1a
epoch rewards are stored partitioned as Vec<Vec< (#32076)
* epoch rewards are stored partitioned as Vec<Vec<

* update comment
2023-06-12 13:06:22 -07:00
Jeff Washington (jwash) 7e7c286d50
factor out get_epoch_reward_calculate_param_info to share results (#32065)
* factor out get_epoch_reward_calculate_param_info to share results

* remove extra comment
2023-06-12 09:56:08 -07:00
Jeff Washington (jwash) 0645e96bc6
add calc_vote_accounts_to_store (#32063)
* add calc_vote_accounts_to_store

* update comment
2023-06-12 08:38:18 -07:00
Jeff Washington (jwash) 26f4c405b2
add update_reward_history_in_partition (#32062)
* add update_reward_history_in_partition

* remove explicit type

* remove spurrious comment
2023-06-12 08:37:46 -07:00
Jeff Washington (jwash) b81a060898
add store_stake_accounts_in_partition (#32060)
* add store_stake_accounts_in_partition

* pr feedback on assert
2023-06-12 08:37:22 -07:00
Jeff Washington (jwash) a001c5eb49
add calculate_reward_points_partitioned (#32050)
* add calculate_reward_points_partitioned

* remove unnecessary clone

* add comment

* clean up test from pr feedback
2023-06-12 08:36:42 -07:00
Tao Zhu 37a759045a
include cost of transaction's requested loaded accounts size in cost model (#31905)
* include transaction requested loaded accounts size cost in cost model

* move function to avoid circular dependency
2023-06-08 12:49:42 -05:00
Alexander Meißner 3f13cd353e
Fix - LoadedProgramType::Closed (#31922)
* Makes Bank::load_program() return correct tombstones.

* Removes early TX failure caused by closed and invalid programs.

* Adjusts the feature gate of simplify_writable_program_account_check.
2023-06-08 15:40:18 +02:00
Jeff Washington (jwash) 36ccc71220
move StakeReward to stake_rewards.rs (#32014) 2023-06-07 12:35:28 -07:00
Alexander Meißner c87ff8780a
Cleanup - `Bank::reconfigure_token2_native_mint()` (#31817)
Removes Bank::reconfigure_token2_native_mint().
2023-06-07 21:14:43 +02:00
Alexander Meißner ee2c2ef6c7
Cleanup - require_static_program_ids_in_transaction (#31767)
require_static_program_ids_in_transaction
2023-06-07 17:12:41 +02:00
Pankaj Garg c86e160438
Add runtime environment to FailedVerification tombstones (#31991)
* Add runtime environment to FailedVerification tombstones

* modify default variant

* prune update

* add DelayVisibility in prune_feature_set_transition
2023-06-06 22:24:39 +00:00
Jeff Washington (jwash) 85a0e31c60
add ProgramExecutionTemporarilyRestricted error (#31796)
* add StakeProgramUnavailable error

* rename to ProgramExecutionTemporarilyRestricted

* Update sdk/src/transaction/error.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* fmt

* update frozen abi hash

* add account_index to ProgramExecutionTemporarilyRestricted error

* Update sdk/src/transaction/error.rs

Co-authored-by: mvines <mvines@gmail.com>

* populate transaction_details from transaction_error for ProgramExecutionTemporarilyRestricted error

* fix a test

* update abi digest

* Update sdk/src/transaction/error.rs

Co-authored-by: Brooks <brooks@prumo.org>

---------

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
Co-authored-by: HaoranYi <haoran.yi@solana.com>
Co-authored-by: HaoranYi <haoran.yi@gmail.com>
Co-authored-by: mvines <mvines@gmail.com>
Co-authored-by: Brooks <brooks@prumo.org>
2023-06-06 14:13:41 -07:00
Alexander Meißner 89207a3fe5
Feature - Prune on feature set transition (#31945)
* Call create_program_runtime_environment() in Bank::apply_builtin_program_feature_transitions().

* Adds LoadedPrograms::prune_feature_set_transition().
2023-06-06 17:49:02 +02:00
Pankaj Garg 37ebb709e6
Evict program cache using transaction usage counter (#31927)
* Evict program cache using tx usage counter

* address review feedback

* address review feedback
2023-06-02 11:26:56 -07:00
Pankaj Garg 29d7a57cce
Reset program cache stats after submit (#31929) 2023-06-02 01:16:49 -07:00
steviez debe794987
Replace improper &Arc<...> with Arc<...> in Bank and Accounts (#31892)
The callstack updated in this PR passed an &Arc<...> down only to have
the bottom level clone the reference. Thus, we are giving shared
ownership so the reference is a bit redundant and arguably obscures the
intention to clone further down the callstack.
2023-05-31 12:36:44 -05:00
Jeff Washington (jwash) 6c520396cd
bank process_new_epoch takes parent_height (#31820) 2023-05-30 08:58:40 -05:00
Jeff Washington (jwash) 0c64e26d53
partitioned rewards helper functions (#31818) 2023-05-30 08:57:31 -05:00
Andrew Fitzgerald 75954dc061
Banking worker (#30970) 2023-05-25 10:16:04 -07:00
Alexander Meißner a85548cf34
Cleanup - spl_token_v3_4_0 and spl_associated_token_account_v1_1_0 (#31764)
* spl_token_v3_4_0

* spl_associated_token_account_v1_1_0

* Marks replace_program_account() as dead code.
2023-05-24 18:45:37 +02:00
Jeff Washington (jwash) e396a10312
change StartBlockHeightAndRewards from parent to current (#31781) 2023-05-23 20:10:51 -07:00
Jeff Washington (jwash) ae12079139
add debug to VoteReward (#31782) 2023-05-23 20:10:20 -07:00
Jeff Washington (jwash) 8e8b2f1671
reformat comments (#31780) 2023-05-23 17:04:47 -07:00
Alexander Meißner 1e8e024ec5
Cleanup - disable_fee_calculator (#31766)
disable_fee_calculator
2023-05-23 18:20:38 +02:00
Alexander Meißner 83f692ce67
Refactor - Share RBPF config and syscall registry globally (#31727)
* Adds LoadedPrograms::program_runtime_environment_v1.

* No need to explicitly remove precompiles from program_accounts_map.
2023-05-20 20:28:00 +02:00
Pankaj Garg e6d490708a
Do not overwrite builtin program in the same slot (#31718) 2023-05-19 05:26:31 -07:00
Alexander Meißner 520c647918
Refactor - Merges `BuiltinPrograms` into `LoadedPrograms` (#31654)
* Merges BuiltinPrograms into LoadedPrograms.

* Prevents built-ins from being pruned.
2023-05-18 22:18:28 +02:00
Jeff Washington (jwash) 0291454876
persist EpochRewardStatus::Active (#31697)
persist `EpochRewardStatus::Active`
2023-05-18 07:14:41 -07:00
Jeff Washington (jwash) ce32cb43db
add bank.epoch_reward_status (#31680) 2023-05-17 10:01:14 -07:00
Pankaj Garg b03422968b
Remove Rc and RefCell to carry program cache in invoke_context (#31684) 2023-05-17 06:10:50 -07:00
Jeff Washington (jwash) 7c6478da8f
remove unnecessary pub (#31676) 2023-05-16 13:53:57 -07:00
Jeff Washington (jwash) 5bc41a0eb8
prepare load_transaction_accounts to recognize reward interval (#31673) 2023-05-16 12:55:35 -07:00
Jeff Washington (jwash) d841ff7fe1
support snapshot deserialization with EpochRewardStatus (#31672)
* support snapshot deserialization with EpochRewardStatus

* pub(crate)
2023-05-16 12:53:52 -07:00
Maximilian Schneider 31aafc8ae6
race condition when running solana-program-test (#31637)
* hold bank freeze lock in banks-server to prevent RC on slot change

* clean up comments and loop

---------

Co-authored-by: Maximilian Schneider <max@mango.markets>
2023-05-15 12:25:06 -07:00
Andrew Fitzgerald 2c869ef778
QoS refactor: Allow pre-filtering (#31542) 2023-05-12 08:53:22 -07:00
Jeff Washington (jwash) 122b05b9f5
pass include_slot_in_hash through hash calcs to allow rehashing if hash is not stored (#31579)
* pass include_slot_in_hash through hash calcs to allow rehashing

* tests use each include_slot_in_hash value

* move include_slot_in_hash

* typo

* reorder struct init

* spelling is hard
2023-05-11 13:23:29 -07:00
Jeff Washington (jwash) e0b914e775
use highest alive root to determine ancient for packing (#31573) 2023-05-11 12:13:15 -07:00
Alexander Meißner bbd8be6cbe
Refactor - Renames loader-v3 to loader-v4. (#31570)
Renames loader-v3 to loader-v4.
2023-05-11 08:29:06 +02:00
Pankaj Garg 04425b81ce
Add stats for LoadedPrograms (#31577) 2023-05-10 12:08:55 -07:00
Jeff Washington (jwash) fed1bb7d54
use max alive root to determine ancient cutoff for clean stats (#31545)
* use max alive root to determine ancient cutoff for clean stats

* fix math and add tests
2023-05-10 08:05:31 -07:00
Alexander Meißner 9e72668a77
Refactor - Simplify built-in program feature transition (#31565)
* Removes builtins::get_builtins(), instead derive them from Bank::builtin_programs.

* Removes BuiltinAction.
Turns BuiltinFeatureTransition from an enum into a struct.

* Makes Bank::remove_builtin() place tombstones.
2023-05-10 16:53:36 +02:00
Pankaj Garg ab89e2cb15
Remove executor cache from the codebase (#31566) 2023-05-10 07:11:04 -07:00
Alexander Meißner 8e9b064ab3
Cleanup - `add_compute_budget_program` and `versioned_tx_message_enabled` (#31560)
* Moves compute_budget_program and address_lookup_table_program into genesis_builtins().

* Removes feature gate of versioned_tx_message_enabled.
2023-05-10 01:14:08 +02:00
Pankaj Garg 2210af60ee
Replace executor cache usage with LoadedPrograms cache (#31462)
* Replace executor cache usage with LoadedPrograms cache

* clippy fixes

* update cache with updated programs

* fixes

* more cleanup

* update tx batch cache with the tx results

* address review comments

* handle program closing backward compatibility

* handle unloaded programs during extraction
2023-05-09 15:44:38 -07:00
Pankaj Garg 3845423e17
Generalize matching criteria for tombstone at boot up (#31547)
Generalize matching criteria for tombstone at bootup
2023-05-09 08:26:42 -07:00
behzad nouri 8e638b785a
removes feature gate code sending votes to tpu-vote-port (#31529) 2023-05-08 18:12:35 +00:00
steviez e62e4fbabd
Replace inc_new_counter_* with datapoint_* in Bank (#31495)
Counters can only report a single field; this adds an extra step to
debugging efforts to figure out the context for where/why the counted
event occurred. Use datapoints instead to subtmit some extra context.
2023-05-05 16:33:35 -05:00
Andrew Fitzgerald 18cd4311af
remove counters in hotpath (#31398) 2023-05-04 09:36:28 -07:00
Pankaj Garg 96e170bd37
Do not evict unloaded programs (#31465)
* Do not evict unloaded programs

* cleanup
2023-05-04 07:49:54 -07:00
Jeff Washington (jwash) ddd1f2fce0
remove unnecessary casts (#31461) 2023-05-03 08:54:25 -07:00
Pankaj Garg ba6f7e80e8
Remove obsolete code for lazily loading programs (#31395) 2023-05-02 14:39:27 -07:00
Alexander Meißner ae75c7cb3f
Refactoring - Remove redundant definition of `BuiltinProgram` (#31429)
* Replaces BuiltinProgram in the program-runtime with the one from solana_rbpf.

* Adjusts the runtimes built-ins to use Arc<LoadedProgram>.

* Adjusts the tests and benchmarks.
2023-05-02 23:01:28 +02:00
Jeff Washington (jwash) 58ce19446f
replace tuple with `VoteReward` struct (#31408) 2023-05-01 10:42:05 -07:00
Pankaj Garg 8eebf64464
Add a second level cache to contain tx batch specific programs (#31413)
* Add a second level cache to contain tx batch specific programs

* fix clippy

* address review comments
2023-05-01 09:09:43 -07:00
Dmitri Makarov 74bd5f87af
Add minor extensions to Bank API (#31396)
* Make load_program method of Bank public for use in ledger-tool

* Add an accessor method to get a reference to builtin_programs of Bank

* Add a parameter to Bank::load_program to control debugging_features
2023-04-28 19:02:49 -04:00
Pankaj Garg 94dc8fed55
Check program modification slots during cold start (#31331) 2023-04-28 06:22:14 -07:00
Brennan 685ebcb3b0
Rework bank to allow RentCollectionCycleParams private (#31361) 2023-04-26 14:48:51 -07:00
Brennan a540690778
Move partition logic out of bank (#31343)
* Move partition logic out of bank
2023-04-26 12:37:02 -07:00
Alexander Meißner 7002c568fb
Cleanup - Removes the CLI option "no-bpf-jit" (#31329)
* Enables JIT everywhere by default.

* Removes the CLI argument "no-bpf-jit" and its plumbering through out the validator code base.

* Removes with_jit bpf_loader variants.

* Removes the to_builtin!() macro.
2023-04-25 19:04:11 +02:00
Alexander Meißner 4b0e16d06f
Refactor - `BuiltinPrograms` (#31324)
* Moves BuiltinProgram of the program runtime into its own file.

* Unifies the runtimes Builtin and program runtimes BuiltinProgram definitions.

* Moves BuiltinPrograms from bank.rs into the program runtime.
2023-04-24 19:01:40 +02:00
Alexander Meißner 8162358b8b
Refactor - Moves the system program into its own crate (#31244)
* Moves the system program into its own crate in "programs/system".

* Moves bank tests back into the runtime to resolve cyclic dependency.
2023-04-19 12:58:08 +02:00
Trent Nelson f34a6bcfce
runtime: transpose `VoteAccount::vote_state()` return to improve ergonomics (#31256) 2023-04-18 14:48:52 -06:00
Brennan 2164a50d00
Move BankIncrementalSnapshotPersistence (#31236)
* Move BankIncrementalSnapshotPersistence

* Update bank serialize ABI digest
2023-04-18 11:18:17 -07:00
Brennan f5fe2607b6
Make VerifyAccountsHashConfig private (#31235) 2023-04-18 07:31:23 -07:00
Brennan 93d0d25d77
Move rent debit out of bank (#31204)
* Move rent debit out of bank

* Clean up imports and visibility

* Fix imports

* rename public mod rent_debits
2023-04-14 19:41:10 +00:00
Alexander Meißner 8eb31f6cfd
Refactor - Syscalls in RBPF CLI (#31109)
* Makes members of SyscallContext public.
Removes check_aligned and check_size from SyscallContext.

* Replaces InvokeContext::set_syscall_context() in tests with mock_create_vm!().

* Passes SyscallContext directly to InvokeContext::set_syscall_context().

* Merges TraceLogStackFrame into SyscallContext.

* Removes the create_vm!() macro.

* Moves BpfAllocator from bpf_loader into program_runtime.

* Frees BpfAllocator from Rc<RefCell<>>.

* Removes unused code from BpfAllocator.

* Consume CUs for heap before doing the allocation.

* Exposes syscalls in rbpf-cli.

* Adds debugging_features parameter to load_program_from_bytes() and load_program_from_account().

* Removes test_bpf_loader_non_terminating_program() as that is already tested in RBPF.

* Moves stack and heap allocation back onto the program runtime stack.
2023-04-14 15:20:08 +02:00
HaoranYi 1db0683a50
Refactor reward payout code - part 4 (#31159)
extract redeem_reward fn
2023-04-12 10:20:21 -05:00
HaoranYi f2e3a0d821
Refactor reward payout code - part 3 (#31157)
extract calculate points fn
2023-04-12 09:36:23 -05:00
HaoranYi 9e14cdc19f
Refactor reward payout code - part 2 (#31156)
* extract load_vote_stake_accounts fn

* rename
2023-04-12 08:40:08 -05:00
HaoranYi 20d1615ccd
Refactor reward payout code - part 1 (#31155)
type alias
2023-04-11 16:23:52 -05:00
Pankaj Garg 12e3e8b467
Add provisions to unload a program from the cache (#31036) 2023-04-10 16:42:05 -07:00
HaoranYi e12c250cf5
Extract new_epoch processing out of new_bank (#31127)
* extract new epoch processing out of new_bank

* review feedbacks

* remove epoch_schedule param

* use RewardCalcTracer
2023-04-10 16:10:45 -05:00
Andrew Fitzgerald 15011eaa5a
separate check_transaction_age from check_age (#30994) 2023-04-07 09:04:03 -07:00
HaoranYi 684901879d
Refactor squash timing report (#31041)
* typo

* refactor squash timing report

* address review comments
2023-04-05 08:53:49 -05:00
Alexander Meißner a6ad37f3ad
Cleanup - `load_and_get_programs_from_cache()` (#30959)
* Replaces assign_program() by replenish() in load_and_get_programs_from_cache().

* Replaces LoadedProgramEntry by a boolean.

* Only lock the global cache once all missing programs have been loaded

* Factors out call of self.rc.accounts.filter_executable_program_accounts().
2023-03-29 16:11:14 +02:00
Pankaj Garg aebc191c38
Stop loading program accounts if program exists in cache (#30703)
* Stop loading program accounts if program exists in cache

* load accounts for upgradeable programs

* revert loader change to conditionally use program data account

* load instruction accounts

* generate TransactionExecutorCache from loaded programs

* cleanup account_found_and_dep_index variable

* address review comments

* handle tombstones in loader

* unify tombstone constructor

* handle multiple tombstones
2023-03-28 11:49:56 -07:00
Pankaj Garg 0df28f6d51
Increment usage count for loaded programs and call eviction (#30900) 2023-03-27 06:48:05 -07:00
Tao Zhu 3e500d9e92
Feature gate builtin consumes static units during processing instruction (#30702)
* add feature gate
* builtins consume statically defined units at beginning of process_instruction()
* Add new instructionError; return error if builtin did not consume units to enforce builtin to consume units;
* updated related tests
* updated ProgramTest with deactivated native_programs_consume_cu feature to continue support existing mock/test programs that do not consume units
2023-03-24 11:31:01 -05:00
Tao Zhu 21c287a64f
include loaded accounts data size limit in transaction fee calculation (#30659)
* include loaded accounts data size limit in transaction base fee calculation
* citing compute_budget for heap cost;
* update sbf tests

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2023-03-21 18:41:41 -05:00
HaoranYi d646820a6e
Do not log detailed epoch stakes (#30825)
* do not log detailed epoch stakes

* allow dead code

* log stake details at trace level

* no need dead_code

* Update runtime/src/bank.rs

Co-authored-by: Brooks <brooks@prumo.org>

---------

Co-authored-by: Brooks <brooks@prumo.org>
2023-03-21 14:43:06 -05:00
Brooks 836494fc5f
Cleans *up to* the last full snapshot slot in verify_snapshot_bank() (#30818)
Cleans up to the last full snapshot slot in verify_snapshot_bank()
2023-03-21 15:49:49 +00:00
Brooks 989c80342b
Adds plumbing and tests for verify_snapshot_bank() with Incremental Accounts Hash (#30789) 2023-03-19 21:44:38 -04:00
Brooks d3273ba118
Removes unused `ancestors` param from `get_snapshot_storages` (#30773) 2023-03-18 17:17:41 -04:00
Ryo Onodera 05ee06814e
Fix wrongly swapped parent slot and block height (#30744) 2023-03-17 20:07:52 +09:00
Tao Zhu 17ba051e1f
Revert "add default_cost as mandatory field for Builtin (#30639)" (#30749)
This reverts commit 7b95c8e698.
2023-03-16 11:21:06 -05:00
Brooks 8be99c1c7d
Verifies incremental accounts hash in verify_accounts_hash_and_lamports (#30735) 2023-03-16 09:48:06 -04:00
Brooks 65cd55261a
Makes Bank::get_snapshot_hash() aware of incremental accounts hashes (#30719) 2023-03-14 18:03:38 -04:00
Brooks 5e5b7f00a2
Stores capitalization in account hashes map (#30635) 2023-03-13 10:50:45 -04:00
Brooks 17b48edd7b
Renames types to be consistent with verify_accounts_hash (#30674) 2023-03-10 20:02:14 -05:00
Brooks 63623e4df2
Renames fns to verify_accounts_hash (#30672) 2023-03-10 18:11:47 -05:00
Brooks 64f583b37d
Improves logs in bank verification at startup (#30671)
* Improves logs in bank verification at startup

* pr: should_clean
2023-03-10 21:01:49 +00:00
Tao Zhu 7b95c8e698
add default_cost as mandatory field for Builtin (#30639)
* add default_cost as mandatory field for Builtin

* updated tests

* set zkp program default to VerifyTransfer CUs

---------

Co-authored-by: Jon Cinque <joncinque@pm.me>
2023-03-10 14:02:24 -06:00
Alessandro Decina 13107b4eb6
Enable JIT in Bank::new_with_config_for_tests() (#30661)
The most common validator configuration is JIT on, so test that. On
platforms that don't support JIT we'll still fallback on the
interpreter.
2023-03-10 22:52:33 +11:00
Ryo Onodera 0b9c040fcf
Add .codecov.yml with checks disabled (#30666)
* Add .codecov.yml with checks disabled

* Trigger test-coverage.sh...
2023-03-10 20:11:22 +09:00
Tao Zhu 3b9438fd49
Replace hardcoded loaded accounts size limit with compute budget instruction (#30506)
1. replace hardcoded loaded accounts data size limit with compute budget instruction;
2. new transaction error for invalid account data size limit
3. test requested limit with combination of features and transactions
2023-03-08 19:41:45 -06:00
Jeff Washington (jwash) 85eaf1e7ab
improve BankCreationFreezingProgress and add tests (#30513) 2023-02-27 07:54:08 -08:00
Tao Zhu 9ec5e9b866
refactor to remove additional is_simple_vote check (#30521) 2023-02-24 23:49:04 -06:00
Brooks 64c7b401cd
BankIncrementalSnapshotPersistence uses strong accounts hash types (#30502) 2023-02-24 18:15:23 -05:00
Tao Zhu 66ea750182
add compute budget instruction to set loaded accounts data size limit (#30377)
* add compute budget instruction to set accounts data size limit

* changes names to explicitly for loaded accounts data size
2023-02-24 09:27:49 -06:00
Brooks b7d31e134c
Refactors Accounts::new_from_parent() (#30475) 2023-02-23 16:50:51 -05:00
Jeff Washington (jwash) c032dc2754
add bank progress stats (#30405)
* add bank progress stats

* handle more bank creation code paths

* use wrapping sub
2023-02-23 06:29:02 -08:00
Pankaj Garg 0e13dcca00
Replenish executor cache eagerly during transaction processing (#30425) 2023-02-23 05:13:39 -08:00
Pankaj Garg b1f5b0d790
Find and load missing programs in LoadedPrograms cache (#30275)
* Find and load missing programs in LoadedPrograms cache

- filter program accounts in a transaction batch
- filter the accounts that are missing in LoadedPrograms cache
- load the programs before processing the transactions
- unit tests

* address review comments

* fix clippy

* address review comments

* fix test

* fix more tests
2023-02-21 12:53:35 -08:00
Brooks 35328ca63d
Makes AccountsHash an enum (#30416) 2023-02-21 15:20:51 -05:00
Pankaj Garg 3a4ba72daf
Remove executors trait and replace with LoadedProgram (#30348)
* Remove executors trait and replace with LoadedProgram

* fill in deployment slot

* address review comments

* fix clippy warnings

* address review comments

* fix failures caught by sbf tests
2023-02-17 12:47:45 +01:00
HaoranYi 615d1a8b69
Extract BankHashLamportsVerifyConfig (#30320)
* refactor BankHashLamportsVerifyConfig

* clippy

* comments

* fix bench
2023-02-15 16:03:50 -06:00
Alexander Meißner 6558c8fdc9
Refactor - Delay visibility of program un-/re-/deployment (#29654)
* Use three separate HashMaps instead of the enum TxBankExecutorCacheDiff.

* Replaces all places which deploy programs by a macro.

* Adds a feature gate.

* Adjust tests.

* Makes undeployment visible immediately.
2023-02-11 11:18:25 +01:00
Brooks b3c2256799
Refactors a sort's comparator fn (#30243) 2023-02-10 15:41:55 -05:00
Brooks d27c860249
Replaces stable sort with unstable sort (#30223) 2023-02-10 09:56:11 -05:00
Ryo Onodera 3e6162e69e
Add address lookup tables to minimized snapshot (#30158)
* Add address lookup tables to minimized snapshot

* Add comment for future posterity

* Add reference to the issue

* Adjust comment a bit

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-02-10 14:46:02 +09:00