Revert "Split compute budget instructions process from struct itself (#33513)"
This reverts commit c73bebe984. This
was found to be a consensus breaking change.
* Split compute budget instruction processing from ComputeBudget struct itself, allow compute_budget_instructions be processed elsewhere without having to instantiate ComputeBudget
* updated tests
* transaction_context: update make_data_mut comment
* bpf_loader: cpi: pass SerializeAccountMetadata to CallerAccount::from*
We now have a way to provide CallerAccount with trusted values coming
from our internal serialization code and not from untrusted vm space
* bpf_loader: direct_mapping: enforce account info pointers to be immutable
When direct mapping is enabled, we might need to update account data
memory regions across CPI calls. Since the only way we have to retrieve
the regions is based on their vm addresses, we enforce vm addresses to
be stable. Accounts can still be mutated and resized of course, but it
must be done in place.
This also locks all other AccountInfo pointers, since there's no legitimate
reason to make them point to anything else.
* bpf_loader: cpi: access ref_to_len_in_vm through VmValue
Direct mapping needs to translate vm values at each access since
permissions of the underlying memory might have changed.
* direct mapping: improve memory permission tracking across CPI calls
Ensure that the data and realloc regions of an account always track the
account's permissions. In order to do this, we also need to split
realloc regions in their own self contained regions, where before we
had:
[account fields][account data][account realloc + more account fields + next account fields][next account data][...]
we now have:
[account fields][account data][account realloc][more account fields + next account fields][next account data][...]
Tested in TEST_[FORBID|ALLOW]_WRITE_AFTER_OWNERSHIP_CHANGE*
Additionally when direct mapping is on, we must update all perms at once before
doing account data updates. Otherwise, updating an account might write into
another account whose perms we haven't updated yet. Tested in
TEST_FORBID_LEN_UPDATE_AFTER_OWNERSHIP_CHANGE.
* bpf_loader: serialization: address review comment don't return vm_addr from push_account_region
* bpf_loader: rename push_account_region to push_account_data_region
* cpi: fix slow edge case zeroing extra account capacity after shrinking an account
When returning from CPI we need to zero all the account memory up to the
original length only if we know we're potentially dealing with uninitialized
memory.
When we know that the spare capacity has deterministic content, we only need to
zero new_len..prev_len.
This fixes a slow edge case that was triggerable by the following scenario:
- load a large account (say 10MB) into the vm
- shrink to 10 bytes - would memset 10..10MB
- shrink to 9 bytes - would memset 9..10MB
- shrink to 8 bytes - would memset 8..10MB
- ...
Now instead in the scenario above the following will happen:
- load a large account (say 10MB) into the vm
- shrink to 10 bytes - memsets 10..10MB
- shrink to 9 bytes - memsets 9..10
- shrink to 8 bytes - memset 8..9
- ...
* bpf_loader: add account_data_region_memory_state helper
Shared between serialization and CPI to figure out the MemoryState of an
account.
* cpi: direct_mapping: error out if ref_to_len_in_vm points to account memory
If ref_to_len_in_vm is allowed to be in account memory, calles could mutate it,
essentially letting callees directly mutate callers memory.
* bpf_loader: direct_mapping: map AccessViolation -> InstructionError
Return the proper ReadonlyDataModified / ExecutableDataModified /
ExternalAccountDataModified depending on where the violation occurs
* bpf_loader: cpi: remove unnecessary infallible slice::get call
* Integrate program loader-v4 with bank
* fix tests
* new struct for ProgramRuntimeEnvironments
* remove environment from program_runtime_environment_v
* move find_program_in_cache() to invoke_context
* cleanup
* bpf_loader: move computing original account lengths inside serialize_paramters_(aligned|unaligned)
This is in preparation of returning more than just the original length
* bpf_loader: deserialize*: take original lens as an iterator instead of a slice
This is in preparation of extracting account lenghts from a larger
context
* bpf_loader: introduce SerializedAccountMetadata
Instead of passing original_account_lengths around as Vec<usize>,
introduce an explicit type that includes the length and soon more.
* Replaces BuiltinProgram in the program-runtime with the one from solana_rbpf.
* Adjusts the runtimes built-ins to use Arc<LoadedProgram>.
* Adjusts the tests and benchmarks.
* Moves BuiltinProgram of the program runtime into its own file.
* Unifies the runtimes Builtin and program runtimes BuiltinProgram definitions.
* Moves BuiltinPrograms from bank.rs into the program runtime.
* Removes test_program_entry_debug().
* Uses declare_process_instruction!() in all tests and benchmarks.
* Replaces with ProcessInstructionWithContext with solana_rbpf::BuiltInFunction.
* Makes members of SyscallContext public.
Removes check_aligned and check_size from SyscallContext.
* Replaces InvokeContext::set_syscall_context() in tests with mock_create_vm!().
* Passes SyscallContext directly to InvokeContext::set_syscall_context().
* Merges TraceLogStackFrame into SyscallContext.
* Removes the create_vm!() macro.
* Moves BpfAllocator from bpf_loader into program_runtime.
* Frees BpfAllocator from Rc<RefCell<>>.
* Removes unused code from BpfAllocator.
* Consume CUs for heap before doing the allocation.
* Exposes syscalls in rbpf-cli.
* Adds debugging_features parameter to load_program_from_bytes() and load_program_from_account().
* Removes test_bpf_loader_non_terminating_program() as that is already tested in RBPF.
* Moves stack and heap allocation back onto the program runtime stack.
* Uses declare_process_instruction!() in all tests.
* Adds post_adjustments to mock_process_instruction().
Removes "solana_sbf_rust_external_spend" from assert_instruction_count() as it panics.
* Moves stable_log::program_invoke(), stable_log::program_success() and stable_log::program_failure() calls from bpf_loader into InvokeContext::process_executable_chain().
* Turns result of ProcessInstructionWithContext from InstructionError into Box<dyn std::error::Error>.
* Bump to solana_rbpf v0.3.0
* Removes Result from return type of EbpfVm::new().
* Turns EbpfError into Box<dyn std::error::Error>.
* Removes BpfError.
* Removes SyscallError::InstructionError.
* Adds a type alias for Box<dyn std::error::Error> in syscalls.
* Uses InvokeContext::process_instruction() in mock_process_instruction().
* Uses InvokeContext::process_instruction() in tests of loader-v3.
* Only throw InstructionError::BuiltinProgramsMustConsumeComputeUnits if result.is_ok().
* Adds CU cost to loader-v3.
* add feature gate
* builtins consume statically defined units at beginning of process_instruction()
* Add new instructionError; return error if builtin did not consume units to enforce builtin to consume units;
* updated related tests
* updated ProgramTest with deactivated native_programs_consume_cu feature to continue support existing mock/test programs that do not consume units
* add default_cost as mandatory field for Builtin
* updated tests
* set zkp program default to VerifyTransfer CUs
---------
Co-authored-by: Jon Cinque <joncinque@pm.me>
* Added support of consumed BPF units tracing
* Fixed "Inconsistent trace log stack" when logging units before first instruction trace
* Rewrote the solution in order to reuse `trace_log_stack` and check whether tracing is enabled
* Instruction tracing is disabled by default
* Fixed bug with mishmash of traces from different programs in single log
* Transformed instructions trace log into the stack of (sub)programs' trace logs
* Bumps solana_rbpf to v0.2.36
* Removes ThisInstructionMeter.
* Removes one "unsafe" expression.
* Removes redundant call to solana_rbpf:🧝:register_bpf_function().
* Adjusts SyscallFunction and SyscallRegistry.
* Inlines ProgramEnvironment into EbpfVm.
* Refactors trait SyscallConsume into fn consume_compute_meter().
* Inlines ComputeMeter into InvokeContext.
* Removes solana-metrics dependency from bpf_loader.
* Replaces RBPF tracer functionality by the debugger.
* Take compute_units_consumed from execute_program().
* Merges execute_program_interpreted() and execute_program_jit().
* Moves CachedExecutors, related structs, consts and tests into the program-runtime crate.
* Moves TransactionExecutor, related enum and type defs into executor_cache mod.
benches/bpf_loader: make account writable in bench_instruction_count_tuner
The tuner program writes to its input account. This fixes the benchmark with
the direct_mapping branch, where we do enforce permissions before execution.
* BorrowedAccount: add set_data_from_slice(), make set_data() take owned values
set_data() used to take a slice and would force alloc+copy if the caller
has owned values (eg account creation, account lookup table).
Expose set_data_from_slice() for callers that have slices, and switch
set_data() to taking an owned Vec.
* BorrowAccount: refactor common accounts_update_delta code in helper method
* BorrowedAccount: add extend_from_slice()
This allows avoiding copies appending entries to account lookup tables.
* BorrowedAccount: remove unnecessary ifs around update_accounts_resize_delta
* Lets instruction_accounts_lamport_sum() have the &InstructionContext as parameter directly.
* Updates docu comments.
* Uses accessors methods instead of accessing private properties of other structs.
* Adds #![deny(clippy::indexing_slicing)].
* Has get_signers() return a Result instead of using unwrap().
* Removes InvokeContext::get_key_of_account_at_index().
* Replaces InstructionContext::new() by InstructionContext::configure().
Adds TransactionContext::get_next_instruction_context().
* Switch back to using references as parameters.
* Hoists InstructionContext::configure() from TransactionContext::push() into InvokeContext::push().
* Moves InstructionContext::config() to the beginning of InvokeContext::push().
* Hoists InstructionContext::configure() from InvokeContext::push() into InvokeContext::process_instruction().
* Review feedback: Updates docu comments.
* Flattens TransactionContext::instruction_trace.
* Stop the search at transaction level.
* Renames get_instruction_context_at => get_instruction_context_at_nesting_level.
* Removes TransactionContext::get_instruction_trace().
Adds TransactionContext::get_instruction_trace_length() and TransactionContext::get_instruction_context_at_index().
* Have TransactionContext::instruction_accounts_lamport_sum() accept an iterator instead of a slice.
* Removes instruction_trace from ExecutionRecord.
* make InstructionContext::new() private
Alexander Meißner
steviez
Tao Zhu
Brooks
Alessandro Decina
Pankaj Garg
Dmitri Makarov
Denis K
behzad nouri
Justin Starry