* use PROGRAM_OWNER + program data for account executable
mock account data with executable_meta in precompiled program and update
test_bank_hash_consistency test
pr: return const slice and add comments
pr: use ReadableAccount
use const to get rid of magic number
add featuregate disable_bpf_loader_instructions to disable bpf loader management instructions, and deprecate_executable_meta_update_in_bpf_loader to deprecate executable flag update in bpf loader
deprecate usage of executable in Account
fix a test
fix sbp bench
fix sbf program tests
add feature gate to account and borrowed account apis
fix tests
more test fixes
* restore bpf_loader v2 tests
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
* Disables verification-less reloading.
* Refactors LoadedPrograms::extract() to use a mutable parameter instead of returning the list of missing entries.
* Refactors LoadedPrograms::extract() to use a mutable parameter instead of returning a LoadedProgramsForTxBatch.
* Adds explicit SecondLevel structure to LoadedPrograms.
* Adds cooperative_loading_task.
* Implements cooperative loading in the bank.
* Fixes fork graph in tests.
* Adds LoadingTaskWaiter.
* Environment mismatch needs to just skip the entry.
reserve(additional) reserves additional bytes on top of the current _length_
not capacity. Before this fix we could potentially reserve less capacity than
required.
* Split compute budget instruction processing from ComputeBudget struct itself, so CB instructions can be processed elsewhere without involving ComputeBudget
* updated tests
* avoid built ComputeBudget from dated ComputeBudgetLimits in this refactoring PR
* Clean-up program-runtime/src/compute_budget_processor.rs
* Add test for a corner case that deprecated instruction is used to request units greater than max limit;
* Update code to handle the corner case.
Revert "Split compute budget instructions process from struct itself (#33513)"
This reverts commit c73bebe984. This
was found to be a consensus breaking change.
* Split compute budget instruction processing from ComputeBudget struct itself, allow compute_budget_instructions be processed elsewhere without having to instantiate ComputeBudget
* updated tests
* solana-program - altbn128: add g1 & g2 compression
still fixing tests for point of infinity
feat: proof compression syscall working
add rust test to ci
remove prints
added c test
added sycall pricing
* fixed ci checks
* refactored altbn128 and compression
* programs/sbf: add TEST_[FORBID|ALLOW]_WRITE_AFTER_OWNERSHIP_CHANGE*
* programs/sbf: add tests for the AccessViolation -> InstructionError mapping
* cpi: add more tests
* programs/sbf: add tests for immutable AccountInfo pointers
* programs/sbf: add tests for verification of SolAccountInfo pointers too
* programs/sbf: add tests for ref_to_len_in_vm handling in CPI
Add TEST_FORBID_LEN_UPDATE_AFTER_OWNERSHIP_CHANGE_MOVING_DATA_POINTER
and TEST_FORBID_LEN_UPDATE_AFTER_OWNERSHIP_CHANGE that exercise the new
logic.
* cpi: tweak tests
Remove some copy pasta and rename two tests to better describe what they're doing
* cpi: add tests that check that CPI updates all accounts at once
* direct mapping: test that writes to executable accounts trigger ExecutableDataModified
* programs/sbf: add explicit tests for when an account's data allocation changes
Computing Poseidon[0] hashes is too expensive to be done in a Solana
program in one transaction. Poseidon is a zero-knowlege proof friendly
hash function, used by the majority of ZK-based projects, including the
ones built on top of Solana.
This change introduces the `sol_poseidon` syscall which takes 2D byte
slice as an input and then calculates a Poseidon hash using a BN254
curve and the following Poseidon parameters:
* x^5 S-boxes
* width - 2 ≤ t ≤ 13
* inputs - 1 ≤ n ≤ 12
* 8 full rounds and partial rounds depending on t: [56, 57, 56, 60, 60,
63, 64, 63, 60, 66, 60, 65]
Computation of Poseidon hashes is done with the light-poseidon[1]
crate, which is audited[2] and compatible with Circom[3] (BN254 curve,
the same parameters and constants).
Proposed compute costs depend on number of inputs and are based on
light-poseidon benchmarks[4].
[0] https://www.poseidon-hash.info/
[1] https://crates.io/crates/light-poseidon
[2] https://github.com/Lightprotocol/light-poseidon/blob/main/assets/audit.pdf
[3] https://docs.circom.io/
[4] https://github.com/Lightprotocol/light-poseidon/tree/main#performance
* remove unnecessary hashes around raw string literals
* remove unncessary literal `unwrap()`s
* remove panicking `unwrap()`
* remove unnecessary `unwrap()`
* use `[]` instead of `vec![]` where applicable
* remove (more) unnecessary explicit `into_iter()` calls
* remove redundant pattern matching
* don't cast to same type and constness
* do not `cfg(any(...` a single item
* remove needless pass by `&mut`
* prefer `or_default()` to `or_insert_with(T::default())`
* `filter_map()` better written as `filter()`
* incorrect `PartialOrd` impl on `Ord` type
* replace "slow zero-filled `Vec` initializations"
* remove redundant local bindings
* add required lifetime to associated constant
In most cases, either a &Bank or an Arc<Bank> is more proper.
- &Bank is used if the function only needs a momentary reference
- Arc<Bank> is used if the function needs its' own copy
This PR leaves several instances of &Arc<Bank> around; these instances
are situations where a clone may only happen conditionally.
* Fix C headers (the name of the syscall was incorrect).
* Add C SBF tests using the alt_bn128 syscall.
* Fix the Rust SBF program:
* Do not use serde and array-bytes, provide test cases as byte arrays
directly.
* Use the `custom_heap_default` macro.
* Replace `bpf` with `sbf` in the crate name.
* Execute both previously existing Rust tests and new C tests in
`programs/sbf/tests`, so they are actually tested on CI.
* AccountSharedData: make data_mut() private
This ensures that the inner Vec is never handed out. This is in
preparation of enforcing that the capacity of the inner vec never
shrinks, which is required for direct mapping.
* Adds the feature bpf_account_data_direct_mapping.
* Remaps EbpfError::AccessViolation into InstructionError::ReadonlyDataModified.
* WIP: Memory regions for each instruction account in create_vm().
* Fix serialization benches, run both copy and !copy variants
* rbpf-cli: fix build
* BorrowedAccount: ensure that account capacity is never reduced
Accounts can be directly mapped in address space. Their capacity can't
be reduced mid transaction as that would create holes in vm address
space that point to invalid host memory.
* bpf_load: run serialization tests for both copy and !copy account data
* bpf_loader: add Serializer::write_account
* fix lints
* BorrowedAccount: make_data_mut is host only
* Fix unused import warning
* Fix lints
* cpi: add explicit direct_mapping arg to update_(callee|caller)_account
* cpi: rename account_data_or_only_realloc_padding to serialized_data
* cpi: add CallerAccount::original_data_len comment
* cpi: add update_callee_account direct_mapping test
* cpi: add test_update_caller_account_data_direct_mapping and fix bug
We used to have a bug in zeroing data when shrinking account, where we zeroed
the spare account capacity but not the realloc padding.
* cpi: add tests for mutated readonly accounts
* cpi: update_caller_account doesn't need to change .serialized_data when direct_mapping is on
* cpi: update_caller_account: ensure that account capacity is always enough
Introduce a better way to ensure that account capacity never goes below what
might be mapped in memory regions.
* cpi: zero account capacity using the newly introduced BorrowedAccount::spare_data_capacity_mut()
Before we were using BorrowedAccount::get_data_mut() to get the base pointer to
the account data, then we were slicing the spare capacity from it. Calling
get_data_mut() doesn't work if an account has been closed tho, since the
current program doesn't own the account anymore and therefore get_data_mut()
errors out.
* bpf_loader: fix same lint for the umpteenth time
* bpf_loader: map AccessViolation to ReadonlyDataModified only for account region violations
* programs/sbf: realloc: add test for large write after realloc
Add a test that after a realloc does a large write that spans the
original account length and the realloc area. This ensures that memory
mapping works correctly across the boundary.
* programs/sbf: run test_program_sbf_realloc with both direct_mapping on and off
By default test banks test with all features on. This ensures we keep
testing the existing code until the new feature is enabled.
* bpf_loader: tweak memcmp syscall
Split the actual memcmp code in a separate function. Remove check
indexing the slices since the slices are guaranteed to have the correct
length by construction.
* bpf_loader: tweak the memset syscall
Use slice::fill, which is effectively memset.
* bpf_loader: syscalls: update mem syscalls to work with non contiguous memory
With direct mapping enabled, accounts can now span multiple memory
regions.
* fix lint, rebase mem_ops
* Implement CoW for writable accounts
* Fix CI
* Move CoW to the MemoryMapping level
* Update after rbpf API change
* Fix merge screwup
* Add create_vm macro. Fix benches.
* cpi: simplify update_caller_account
Simplify the logic to update a caller's memory region when a callee
causes an account data pointer to change (eg during CoW)
* benches/bpf_loader: move serialization out of create_vm bench
* benches/bpf_loader: don't copy accounts when direct mapping is on
* Fix review nits
* bpf_loader: mem_ops: handle u64 overflow in MemoryChunkIterator::new
When starting at u64::MAX, the chunk iterator would always return the
empty sequence (None on the first next()) call, instead of returning a
memory access violation.
Use checked instead of saturating arithmetic to detect the condition and
error out.
This commit also adds more tests around boundary conditions.
* Fix loader-v3 tests: data_mut => data_as_mut_slice
* Fix CI
* bpf_loader: fix tuner bench: account must be writable
With direct mapping on, invalid writes are caught early meaning the
tuner would fail on the first store and not consume the whole budget
like the benchmark expects.
---------
Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
* Moves the registration of bpf_loader built-ins from the ledger into the runtime.
* Removes bpf_loader built-ins registration in tests and benchmarks.
* Removes declare_builtin!() of bpf_loader built-ins.
* Removes built-ins from the SDK.
* Adjusts tests.
* Uses declare_process_instruction!() in all tests.
* Adds post_adjustments to mock_process_instruction().
Removes "solana_sbf_rust_external_spend" from assert_instruction_count() as it panics.
* Moves stable_log::program_invoke(), stable_log::program_success() and stable_log::program_failure() calls from bpf_loader into InvokeContext::process_executable_chain().
* Turns result of ProcessInstructionWithContext from InstructionError into Box<dyn std::error::Error>.
* Bump to solana_rbpf v0.3.0
* Removes Result from return type of EbpfVm::new().
* Turns EbpfError into Box<dyn std::error::Error>.
* Removes BpfError.
* Removes SyscallError::InstructionError.
* Adds a type alias for Box<dyn std::error::Error> in syscalls.
* add feature gate
* builtins consume statically defined units at beginning of process_instruction()
* Add new instructionError; return error if builtin did not consume units to enforce builtin to consume units;
* updated related tests
* updated ProgramTest with deactivated native_programs_consume_cu feature to continue support existing mock/test programs that do not consume units
* include loaded accounts data size limit in transaction base fee calculation
* citing compute_budget for heap cost;
* update sbf tests
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* add default_cost as mandatory field for Builtin
* updated tests
* set zkp program default to VerifyTransfer CUs
---------
Co-authored-by: Jon Cinque <joncinque@pm.me>
* Use three separate HashMaps instead of the enum TxBankExecutorCacheDiff.
* Replaces all places which deploy programs by a macro.
* Adds a feature gate.
* Adjust tests.
* Makes undeployment visible immediately.
Tao Zhu
HaoranYi
Alexander Meißner
Joe C
Lucas Steuernagel
Brooks
Alessandro Decina
Ryo Onodera
Dmitri Makarov
steviez
ananas-block
vadorovsky
Trent Nelson
Pankaj Garg
behzad nouri