* Rename ComputeBudget::max_invoke_stack_height to max_instruction_stack_depth
The new name is consistent with the existing
ComputeBudget::max_instruction_trace_length.
Also expose compute_budget:MAX_INSTRUCTION_DEPTH.
* bpf_loader: use an explicit thread-local pool for stack and heap memory
Use a fixed thread-local pool to hold stack and heap memory. This
mitigates the long standing issue of jemalloc causing TLB shootdowns to
serve such frequent large allocations.
Because we need 1 stack and 1 heap region per instruction, and the
current max instruction nesting is hardcoded to 5, the pre-allocated
size is (MAX_STACK + MAX_HEAP) * 5 * NUM_THREADS. With the current
limits that's about 2.5MB per thread. Note that this is memory that
would eventually get allocated anyway, we're just pre-allocating it now.
* programs/sbf: add test for stack/heap zeroing
Add TEST_STACK_HEAP_ZEROED which tests that stack and heap regions are
zeroed across reuse from the memory pool.
Consider this scenario:
- Program increases length of an account
- Program start CPI and adds this account as a read-only account
- In fn update_callee_account() we resize account, which may change
the pointer
- Once CPI finishes, the program continues and may read/write from
the account. The mapping must be up-to-date else we use stale
pointers.
Note that we always call callee_account.set_data_length(), which
may change the pointer. In testing I found that resizing a vector
from 10240 down to 127 sometimes changes its pointer. So, always
update the pointer.
* extract curve25519 crate
* remove obsolete comment
* fix Cargo.toml files
* fix imports
* update lock file
* remove unused deps from zk-token-sdk
* fmt
* add solana-curve25519 patch
* add missing override to programs/sbf/Cargo.toml
* copy over an allow()
* move new crate to curves dir
* use workspace version
* add back missing dev dep
* add missing dependencies to programs/sbf
* fmt
* move dep to the correct dependency table
* remove #[cfg(not(target_os = "solana"))] above errors mod
* local program cache: add `modified_entries` field
* use `modified_entries` for modified program cache
* invoke context: make `program_cache_for_tx_batch` mutable
* invoke context: unify local program cache instances
* remove `find_program_in_cache` alias
* Reorders all translate_slice() to the beginning of SyscallInvokeSignedRust::translate_instruction().
* Uses the host slice instead the guest one.
* Reorders consume_compute_meter() to happen before heap allocation.
* extract solana-poseidon crate
* update imports and lock files
* remove now-unused light-poseidon dep from program crate
* add missing posiedon dep when target_os = "solana"
* set solana-poseidon version to 2.0.0
* sort workspace members
* use workspace version in Cargo.toml
Co-authored-by: Joe C <joecaulfield29@yahoo.com>
* move syscall definition to solana-poseidon crate
Co-authored-by: Joe C <joecaulfield29@yahoo.com>
* finish moving sol_poseidon defn to solana-poseidon crate
* update imports in examples
---------
Co-authored-by: Joe C <joecaulfield29@yahoo.com>
program-runtime: modify sysvar cache impl
in prep for sol_sysvar_get, rework how SysvarCache stores data internally
we store account data directly, except also storing StakeHistory and SlotHashes as objects
these object representations can be removed after native programs are converted to bpf
* AccountSharedData::reserve: remove extra alloc + memcpy
Calling data_mut().reserve(additional) used to result in _two_ allocs
and memcpys: the first to unshare the underlying vector, and the second
upon calling reserve since Arc::make_mut clones so it uses capacity ==
len.
With this fix we manually "unshare" allocating with capacity = len +
additional, therefore saving on extra alloc and memcpy.
* AccountSharedData::set_data_from_slice: skip extra alloc + memcpy
We used call make_data_mut() from set_data_from_slice() from the days
when direct mapping couldn't deal with accounts getting shrunk. That
changed in https://github.com/solana-labs/solana/pull/32649 (see the
if callee_account.capacity() < min_capacity check in
cpi.rs:update_caller_account()).
With this change we don't call make_data_mut() anymore before
set_data_from_slice(), saving the cost of a memcpy since
set_data_from_slice() overrides the whole account content anyway.
* alt_bn128: simplify errors in sycalls (alt_bn128, compress, poseidon)
* add TODO for feature gate. remove validate from compress
* add feature gate
* fix one more error case
* all changes under feature gate
* revert removing from()
* return unexpected errors in lib
* add comment to remove error types, once the feature gate is activated
* remove unnecessary/impossible error
* fix mispelled comments
* runtime: core_bpf_migration: add migration path
* runtime: core_bpf_migration: add tests for migration path
* comments
* function name
* rent lamports
* bank operations ordering
* make migration method a bank method
* update deployment slot to current slot
* invoke loader deploy directly
* Fix: deploy program on last slot of epoch during environment change
* solana-runtime: deploy at last epoch slot test
* disable deployment of sol_alloc_free
* Move tx-batch-constructor to its own function
* use new_from_cache
---------
Co-authored-by: Alessandro Decina <alessandro.d@gmail.com>
* restrict curve25519 multiscalar multiplication vector length to 512
* add syscall tests for msm vector length
* add new feature gate `curve25519_restrict_msm_length`
* update tests for feature new gate
* Update programs/bpf_loader/src/syscalls/mod.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* remove length guard on the multisicalar mult lib function
---------
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* use PROGRAM_OWNER + program data for account executable
mock account data with executable_meta in precompiled program and update
test_bank_hash_consistency test
pr: return const slice and add comments
pr: use ReadableAccount
use const to get rid of magic number
add featuregate disable_bpf_loader_instructions to disable bpf loader management instructions, and deprecate_executable_meta_update_in_bpf_loader to deprecate executable flag update in bpf loader
deprecate usage of executable in Account
fix a test
fix sbp bench
fix sbf program tests
add feature gate to account and borrowed account apis
fix tests
more test fixes
* restore bpf_loader v2 tests
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
* Use 2's random selection to evict program cache
* implement decaying of usage counter
* replace RwLock with AtomicU64
* address review comments
* remove -> swap_remove
iter_memory_pair_chunks was iterating regions in reverse, but not memory
_within_ regions in reverse.
This commit fixes the issue and simplifies the implementation by removing
nested loops which made control flow hard to reason about.
If the vector holding an account is reallocated during execution of a callee,
we must zero the spare capacity regardless of whether the account size changed,
because the underlying vector might contain uninitialized memory in the spare
capacity.
mergify[bot]
Sammy Harris
Alessandro Decina
Sean Young
Kevin Heavey
Lucas Ste
Joe C
Alexander Meißner
Yihau Chen
hana
Joe C
samkim-crypto
Emanuele Cesena
Tyera
Pankaj Garg
HaoranYi
Brooks
Justin Starry
Tao Zhu
Pankaj Garg
Andrew Fitzgerald