Commit Graph

430 Commits

Author SHA1 Message Date
mergify[bot] 2408949818
v2.0: program-runtime: double program cache size (backport of #3481) (#3494)
program-runtime: double program cache size (#3481)

The cache is currently getting thrashed and programs are getting
reloaded pretty much at every single slot. Double the cache size, which
makes reloading happen only due to random eviction sometimes picking a
popular entry.

The JIT code size with the new cache size is about 800MB.

This change reduces jit time 15x.

(cherry picked from commit fb4adda5a8a59d7eafe942157260c023b962925a)

Co-authored-by: Alessandro Decina <alessandro.d@gmail.com>
2024-11-09 10:16:09 +07:00
mergify[bot] b57b883355
v2.0: Fix BankForks::new_rw_arc memory leak (backport of #1893) (#2066)
* Fix BankForks::new_rw_arc memory leak (#1893)

(cherry picked from commit d441c0f577)

* Fix flaky test test_banking_stage_entries_only_central_scheduler (#2082)

---------

Co-authored-by: Andrei Silviu Dragnea <andreisilviudragnea@gmail.com>
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-07-17 16:11:40 -07:00
mergify[bot] 0001afe8ed
v2.0: Refactor cost tracking (backport of #1954) (#1975)
* Refactor cost tracking (#1954)

* Refactor and additional metrics for cost tracking (#1888)

* Refactor and add metrics:
- Combine remove_* and update_* functions to reduce locking on cost-tracker and iteration.
- Add method to calculate executed transaction cost by directly using actual execution cost and loaded accounts size;
- Wireup histogram to report loaded accounts size;
- Report time of block limits checking;
- Move account counters from ExecuteDetailsTimings to ExecuteAccountsDetails;

* Move committed transactions adjustment into its own function

* remove histogram for loaded accounts size due to performance impact

(cherry picked from commit f8630a3522)

* rename cost_tracker.account_data_size to better describe its purpose is to tracker per-block new account allocation

---------

Co-authored-by: Tao Zhu <82401714+tao-stones@users.noreply.github.com>
Co-authored-by: Tao Zhu <tao@solana.com>
2024-07-03 14:59:02 -05:00
Tao Zhu f8ae688668
Revert "v2.0: Refactor and additional metrics for cost tracking (backport of #1888) (#1900) (#1937)
Revert "v2.0: Refactor and additional metrics for cost tracking (backport of #1888) (#1900)"

This reverts commit 0aef62eac7.
2024-07-01 10:33:44 -06:00
mergify[bot] 0aef62eac7
v2.0: Refactor and additional metrics for cost tracking (backport of #1888) (#1900)
* Refactor and additional metrics for cost tracking (#1888)

* Refactor and add metrics:
- Combine remove_* and update_* functions to reduce locking on cost-tracker and iteration.
- Add method to calculate executed transaction cost by directly using actual execution cost and loaded accounts size;
- Wireup histogram to report loaded accounts size;
- Report time of block limits checking;
- Move account counters from ExecuteDetailsTimings to ExecuteAccountsDetails;

* Move committed transactions adjustment into its own function

(cherry picked from commit c3fadacf69)

* rename cost_tracker.account_data_size to better describe its purpose is to tracker per-block new account allocation

---------

Co-authored-by: Tao Zhu <82401714+tao-stones@users.noreply.github.com>
Co-authored-by: Tao Zhu <tao@solana.com>
2024-06-28 16:09:36 -05:00
Justin Starry 0d1ad0dc1d
Reuse compute budget processing (#1700)
* refactor: reuse compute budget limits

* fix tests
2024-06-20 19:22:19 +00:00
Alessandro Decina 2107adcf35
bpf_loader: use an explicit thread-local pool for stack and heap memory (#1370)
* Rename ComputeBudget::max_invoke_stack_height to max_instruction_stack_depth

The new name is consistent with the existing
ComputeBudget::max_instruction_trace_length.

Also expose compute_budget:MAX_INSTRUCTION_DEPTH.

* bpf_loader: use an explicit thread-local pool for stack and heap memory

Use a fixed thread-local pool to hold stack and heap memory. This
mitigates the long standing issue of jemalloc causing TLB shootdowns to
serve such frequent large allocations.

Because we need 1 stack and 1 heap region per instruction, and the
current max instruction nesting is hardcoded to 5, the pre-allocated
size is (MAX_STACK + MAX_HEAP) * 5 * NUM_THREADS. With the current
limits that's about 2.5MB per thread. Note that this is memory that
would eventually get allocated anyway, we're just pre-allocating it now.

* programs/sbf: add test for stack/heap zeroing

Add TEST_STACK_HEAP_ZEROED which tests that stack and heap regions are
zeroed across reuse from the memory pool.
2024-06-20 08:57:11 +07:00
Ryo Onodera 3e53b644a8
Adjust replay-related metrics for unified scheduler (#1741)
* Adjust replay-related metrics for unified schduler

* Fix grammar

* Don't compute slowest for unified scheduler

* Rename to is_unified_scheduler_enabled

* Hoist uses to top of file

* Conditionally disable replay-slot-end-to-end-stats

* Remove the misleading fairly balanced text
2024-06-20 09:12:04 +09:00
Lucas Ste ebd063eb79
Add Shuttle multithreading test infrastructure (#1634) 2024-06-17 19:24:43 -03:00
Joe C 3af8f0cfdb
Program Runtime: Unify transaction batch program caches (#1399)
* local program cache: add `modified_entries` field

* use `modified_entries` for modified program cache

* invoke context: make `program_cache_for_tx_batch` mutable

* invoke context: unify local program cache instances

* remove `find_program_in_cache` alias
2024-06-17 17:08:13 -05:00
Joe C 5148dcccf7
SVM: change modified programs result to hash map (#1391)
* SVM: change modified programs result to hash map

* take entries
2024-06-12 22:54:29 +00:00
Justin Starry e4b0055e52
Validate transaction fee payer earlier (#1529) 2024-06-11 07:49:11 -07:00
Joe C b1508010c0
Syscall: `GetEpochStake` (#1152)
* SVM: add `get_vote_accounts` callback

* program-runtime: invoke context: add `vote_accounts`

* sdk: add feature gate for sol-get-epoch-stake

* syscall: get-epoch-stake

* syscall: get-epoch-stake: tests

* drop `get_compute_meter`

* refactor for total stake

* refactor API for total stake
2024-06-11 09:28:31 -05:00
Joe C 0bf34e5ca1
Compute Budget: add `frozen-abi` feature (#1606)
compute budget: add `frozen-abi` feature
2024-06-05 09:20:50 -05:00
Pankaj Garg 9830fb6761
SVM: remove usage of program cache from load_program_with_pubkey() (#1582)
* SVM: remove usage of program cache from Bank::load_program()

* return value instead of reference from get_environments_for_epoch()
2024-06-04 14:44:00 -07:00
Alexander Meißner e43047452a
Metrics - Adds `ProgramCacheStats::water_level` (#1556)
Adds ProgramCacheStats::water_level
2024-06-04 22:53:44 +02:00
Joe C 62eb4cc068
Add Compute Budget Crate (#1121) 2024-06-01 01:38:19 -05:00
Alexander Meißner 6455a2a249
Fix - `ExecuteDetailsTimings::execute_us` (#1499)
Explicitly stops and restarts the execution timer during CPI
2024-05-31 10:25:06 +02:00
Joe C 1c0bd2d6df
Invoke Context: Dedupe compute budgets (#1469)
invoke context: dedupe compute budgets
2024-05-23 14:22:07 +00:00
Kevin Heavey cadba689cb
Make solana-frozen-abi optional in all remaining crates (#1278)
* put most AbiExample derivations behind a cfg_attr

* feature gate all `extern crate solana_frozen_abi_macro;`

* use cfg_attr wherever we were deriving both AbiExample and AbiEnumVisitor

* fix cases where AbiEnumVisitor was still being derived unconditionally

* fix a case where AbiExample was derived unconditionally

* fix more cases where both AbiEnumVisitor and AbiExample were derived unconditionally

* two more cases where AbiExample and AbiEnumVisitor were unconditionally derived

* fix remaining unconditional derivations of AbiEnumVisitor

* fix cases where AbiExample is the first thing derived

* fix most remaining unconditional derivations of AbiExample

* move all `frozen_abi(digest =` behind cfg_attr

* replace incorrect cfg with cfg_attr

* fix one more unconditionally derived AbiExample

* feature gate AbiExample impls

* add frozen-abi feature to required Cargo.toml files

* make frozen-abi features activate recursively

* fmt

* add missing feature gating

* fix accidentally changed digest

* activate frozen-abi in relevant test scripts

* don't activate solana-program's frozen-abi in sdk dev-dependencies

* update to handle AbiExample derivation on new AppendVecFileBacking enum

* revert toml formatting

* remove unused frozen-abi entries from address-lookup-table Cargo.toml

* remove toml references to solana-address-lookup-table-program/frozen-abi

* update lock file

* remove no-longer-used generic param
2024-05-17 14:42:58 +02:00
Joe C 664bf100b5
program-runtime: invoke context: rename `programs_loaded_for_tx` to `program_cache_for_tx_batch` (#1390)
program-runtime: invoke context: rename `programs_loaded_for_tx`
2024-05-16 19:08:01 -05:00
hana d8e9da5b5d
implement SyscallGetSysvar (#1307)
generic mechanism to fetch arbitrary slices of account data for a specially selected list of sysvars
2024-05-16 15:08:15 -07:00
Joe C 10e95fe656
SVM: Hoist program cache recompilation up from Bank (#1077)
* SVM: hoist program cache recompilation up from bank

* program cache: rename `recompilation` to `cache preparation`
2024-05-16 20:01:04 +00:00
hana e3b55cf5ce
Rework SysvarCache implementation (#560)
program-runtime: modify sysvar cache impl

in prep for sol_sysvar_get, rework how SysvarCache stores data internally
we store account data directly, except also storing StakeHistory and SlotHashes as objects
these object representations can be removed after native programs are converted to bpf
2024-05-13 08:41:11 -07:00
Alexander Meißner 9c3c252482
Feature - ProgramCache index implementation switch (#1269)
* Removes direct usage of ProgramCache::entries in get_entries_to_load().

* Adds enum to switch between index implementations.

* Inserts match self.index where ever necessary.
2024-05-11 01:16:55 +02:00
Alexander Meißner 979f619077
Fix - Reloading of unload entries after finishing the recompilation phase (#1199)
* Adds reproducer to test_feature_activation_loaded_programs_epoch_transition.

* Differentiate entries by environment instead of adjusting the effective slot.

* Fixes tests of assign_program().

* Fixes env order in test_feature_activation_loaded_programs_recompilation_phase().

* Turns test_load_program_effective_slot() into test_load_program_environment().

* Adds comments inside ProgramCache::assign_program().
2024-05-09 22:15:02 +02:00
Yihau Chen f050705f98
clippy: unnecessary_get_then_check (#1242) 2024-05-09 00:17:18 +08:00
Alessandro Decina 10e5086343
program cache: reduce contention (#1192)
* program cache: reduce contention

Before this change we used to take the write lock to extract(). This
means that even in the ideal case (all programs are already cached),
the cache was contended by all batches and all operations were
serialized.

With this change we now take the write lock only when we store a new
entry in the cache, and take the read lock to extract(). This means
that in the common case where most/all programs are cached, there is no
contention and all batches progress in parallel.

This improves node replay perf by 20-25% on current mnb traffic.

* ProgramCache: remove SecondLevel structure
2024-05-06 19:26:55 +02:00
Alexander Meißner cf65d2db5e
Refactor - ProgramCache tests (#1047)
* Adds get_entries_to_load().

* Differentiates account not found and unloaded in match_missing().

* Orders assert_eq!(match_missing(), false) in front of cache.extract().

* Makes cargo clippy happy.

* Adds test_extract_nonexistent().
2024-05-03 15:32:59 +02:00
Alexander Meißner 57f92b75c7
Refactor - Reduces direct usage of `ProgramCache::entries` in tests. (#1147)
Reduces direct usage of cache.entries in tests.
2024-05-03 09:39:22 +02:00
Joe C 34b76ac580
Program-Runtime: Add `EnvironmentConfig` to `InvokeContext` (#1059)
* program-runtime: add `EnvironmentConfig` to `InvokeContext`

* move `blockhash` to `EnvironmentConfig`

* move `lamports_per_signature` to `EnvironmentConfig`

* move `feature_set` to `EnvironmentConfig`

* move `sysvar_cache` to `EnvironmentConfig`

* add `get_feature_set` getter
2024-05-01 15:22:43 -05:00
Kevin Heavey 075c79039a
make the LogCollector fields public (#1075) 2024-04-27 07:38:53 +00:00
Ryo Onodera 90bea332b2
Reduce ProgramCache write lock contention (#1037)
* Reduce ProgramCache write-lock contention

* Also consider programs modified by tx

* Memoize program cache write lock
2024-04-27 14:38:11 +09:00
Alexander Meißner 878869b212
Adds comment explaining the possible `ProgramCacheEntryType` transitions. (#908)
Adds comment explaining the possible ProgramCacheEntryType transitions.
2024-04-25 16:17:15 +02:00
Alexander Meißner e43338f3d6
Refactor - Rename `LoadedProgram` to `ProgramCacheEntry` (#880)
LoadedProgramOwner => ProgramCacheEntryOwner.
LoadedProgramType => ProgramCacheEntryType.
LoadedProgram => ProgramCacheEntry.
LoadedProgramStats => ProgramCacheStats.
LoadedProgramMatchCriteria => ProgramCacheMatchCriteria.
LoadedProgramsForTxBatch => ProgramCacheForTxBatch.
2024-04-18 21:38:08 +02:00
Kevin Heavey 8edc6ccb5c
Move Stats::submit to solana-runtime (#686)
* Move Stats::submit to solana-runtime

* fmt

* rename Stats to LoadedProgramStats

* move submit_loaded_programs_stats to bank::metricss and rename it to report_loaded_programs_stats

* add new log method to LoadedProgramStats and move some code there

* fix unused import

* reload stats in LoadedProgramStats::log

* remove old import
2024-04-17 17:43:43 +00:00
Alexander Meißner ac36dbbc87
Refactor - `LoadedProgramType::Loaded` and `LoadedProgramOwner` (#606)
* Adds LoadedProgram::account_owner() and LoadedProgramOwner.
Merges LoadedProgramTypes LegacyV0, LegacyV1, Typed and TestLoaded into Loaded.

* Review feedback.
2024-04-17 17:48:14 +02:00
Alexander Meißner d448c4e5f0
Fix - `ProgramCache::assign_program()` Closed => Loaded in same slot (#673)
* Refactors test_add_builtin() to use LoadedProgram::new_builtin() and ProgramCache::extract().

* Revert Closed => Loaded transition

* Adds more loaded programs deployment tests
2024-04-12 14:59:51 +02:00
Kevin Heavey e04dbaa305
move ThreadExecuteTimings to solana-ledger (#690)
* move ThreadExecuteTimings to solana-ledger
* run cargo-for-all-lock-files tree
* move ThreadExecuteTimings below BatchExecutionTiming
* remove ReplaySlotStats to solana-ledger
* remove  `#![recursion_limit = "2048"]` from program-runtime
2024-04-11 17:59:16 +02:00
Joe C 03ef611f0c
program-runtime: hoist `RuntimeConfig` up to SVM (#630)
program-runtime: hoist `RuntimeConfig` out into SVM
2024-04-07 10:45:57 -05:00
Alexander Meißner 55c05c5ea5
Fix - `FailedVerification` and `Closed` tombstones (#419)
* Only the verifier can cause FailedVerification, everything else is Closed

* Removes the environments parameter from load_program_accounts().

* cargo fmt

* Simplify invocation of deployed program

* Attempt to invoke a program before it is deployed

* Attempt to invoke a buffer before it is used in a deployment

* Escalates Option return value of load_program_accounts() to load_program_with_pubkey().

* Review feedback
2024-04-05 13:03:18 +02:00
Joe C 526979d589
program-runtime: hoist `MessageProcessor` up to SVM (#586) 2024-04-04 18:20:01 -05:00
Andrew Fitzgerald 65f54bec88
Add ProgramCacheUs to execute timings (#473) 2024-04-02 10:03:55 -05:00
Illia Bobyr 8cb7000973
loaded-programs: Use `Measure::end_as_us()` (#81)
It is a bit nicer API, compared to the `stop()/as_us()` pair.  Does not require the value to be `mut`.
2024-04-02 06:31:24 -07:00
Joe C 005c1375f8
Program Runtime: Refactor `InvokeContext` out of `MessageProcessor::process_message` (#509) 2024-04-02 05:06:13 -05:00
Pankaj Garg 91b1ee3df6
Fix: deploy program on last slot of epoch during environment change (#101)
* Fix: deploy program on last slot of epoch during environment change

* solana-runtime: deploy at last epoch slot test

* disable deployment of sol_alloc_free

* Move tx-batch-constructor to its own function

* use new_from_cache

---------

Co-authored-by: Alessandro Decina <alessandro.d@gmail.com>
2024-03-22 07:43:28 -07:00
Dmitri Makarov 09ae5872b3
Rename LoadedPrograms to ProgramCache for readability (#339) 2024-03-20 15:26:45 -04:00
HaoranYi 0f8f8cd970
Revert deprecate executable feature (#309)
* revert deprecate executable feature

* add native loader account transfer test

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2024-03-20 10:54:15 -05:00
Brennan 81075e60b3
Loaded program test robustness (#59)
* tests robust to prog entry count
2024-03-13 08:21:54 -07:00
Dmitri Makarov ba43f74dcf
[SVM] Move RuntimeConfig to program-runtime (#96)
RuntimeConfig doesn't use anything SVM specific and logically belongs
in program runtime rather than SVM.  This change moves the definition
of RuntimeConfig struct from the SVM crate to program-runtime and
adjusts `use` statements accordingly.
2024-03-07 10:16:16 -08:00