blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Account data direct mapping (#28053) 

* AccountSharedData: make data_mut() private

This ensures that the inner Vec is never handed out. This is in
preparation of enforcing that the capacity of the inner vec never
shrinks, which is required for direct mapping.

* Adds the feature bpf_account_data_direct_mapping.

* Remaps EbpfError::AccessViolation into InstructionError::ReadonlyDataModified.

* WIP: Memory regions for each instruction account in create_vm().

* Fix serialization benches, run both copy and !copy variants

* rbpf-cli: fix build

* BorrowedAccount: ensure that account capacity is never reduced

Accounts can be directly mapped in address space. Their capacity can't
be reduced mid transaction as that would create holes in vm address
space that point to invalid host memory.

* bpf_load: run serialization tests for both copy and !copy account data

* bpf_loader: add Serializer::write_account

* fix lints

* BorrowedAccount: make_data_mut is host only

* Fix unused import warning

* Fix lints

* cpi: add explicit direct_mapping arg to update_(callee|caller)_account

* cpi: rename account_data_or_only_realloc_padding to serialized_data

* cpi: add CallerAccount::original_data_len comment

* cpi: add update_callee_account direct_mapping test

* cpi: add test_update_caller_account_data_direct_mapping and fix bug

We used to have a bug in zeroing data when shrinking account, where we zeroed
the spare account capacity but not the realloc padding.

* cpi: add tests for mutated readonly accounts

* cpi: update_caller_account doesn't need to change .serialized_data when direct_mapping is on

* cpi: update_caller_account: ensure that account capacity is always enough

Introduce a better way to ensure that account capacity never goes below what
might be mapped in memory regions.

* cpi: zero account capacity using the newly introduced BorrowedAccount::spare_data_capacity_mut()

Before we were using BorrowedAccount::get_data_mut() to get the base pointer to
the account data, then we were slicing the spare capacity from it. Calling
get_data_mut() doesn't work if an account has been closed tho, since the
current program doesn't own the account anymore and therefore get_data_mut()
errors out.

* bpf_loader: fix same lint for the umpteenth time

* bpf_loader: map AccessViolation to ReadonlyDataModified only for account region violations

* programs/sbf: realloc: add test for large write after realloc

Add a test that after a realloc does a large write that spans the
original account length and the realloc area. This ensures that memory
mapping works correctly across the boundary.

* programs/sbf: run test_program_sbf_realloc with both direct_mapping on and off

By default test banks test with all features on. This ensures we keep
testing the existing code until the new feature is enabled.

* bpf_loader: tweak memcmp syscall

Split the actual memcmp code in a separate function. Remove check
indexing the slices since the slices are guaranteed to have the correct
length by construction.

* bpf_loader: tweak the memset syscall

Use slice::fill, which is effectively memset.

* bpf_loader: syscalls: update mem syscalls to work with non contiguous memory

With direct mapping enabled, accounts can now span multiple memory
regions.

* fix lint, rebase mem_ops

* Implement CoW for writable accounts

* Fix CI

* Move CoW to the MemoryMapping level

* Update after rbpf API change

* Fix merge screwup

* Add create_vm macro. Fix benches.

* cpi: simplify update_caller_account

Simplify the logic to update a caller's memory region when a callee
causes an account data pointer to change (eg during CoW)

* benches/bpf_loader: move serialization out of  create_vm bench

* benches/bpf_loader: don't copy accounts when direct mapping is on

* Fix review nits

* bpf_loader: mem_ops: handle u64 overflow in MemoryChunkIterator::new

When starting at u64::MAX, the chunk iterator would always return the
empty sequence (None on the first next()) call, instead of returning a
memory access violation.

Use checked instead of saturating arithmetic to detect the condition and
error out.

This commit also adds more tests around boundary conditions.

* Fix loader-v3 tests: data_mut => data_as_mut_slice

* Fix CI

* bpf_loader: fix tuner bench: account must be writable

With direct mapping on, invalid writes are caught early meaning the
tuner would fail on the first store and not consume the whole budget
like the benchmark expects.

---------

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
This commit is contained in:
Alessandro Decina 2023-04-29 06:54:39 +10:00 committed by GitHub
parent bd5893907e
commit 117a194b73
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 2919 additions and 945 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
program-test/src/lib.rs
View File

@ -123,7 +123,8 @@ pub fn builtin_process_instruction(
invoke_context
.transaction_context
.get_current_instruction_context()?,
true,
true, // should_cap_ix_accounts
true, // copy_account_data // There is no VM so direct mapping can not be implemented here
)?;
// Deserialize data back into instruction params

37
programs/bpf_loader/benches/serialization.rs
View File

@ -126,7 +126,20 @@ fn bench_serialize_unaligned(bencher: &mut Bencher) {
.get_current_instruction_context()
.unwrap();
bencher.iter(|| {
let _ = serialize_parameters(&transaction_context, instruction_context, true).unwrap();
let _ =
serialize_parameters(&transaction_context, instruction_context, true, false).unwrap();
});
}
#[bench]
fn bench_serialize_unaligned_copy_account_data(bencher: &mut Bencher) {
let transaction_context = create_inputs(bpf_loader_deprecated::id(), 7);
let instruction_context = transaction_context
.get_current_instruction_context()
.unwrap();
bencher.iter(|| {
let _ =
serialize_parameters(&transaction_context, instruction_context, true, true).unwrap();
});
}
@ -138,7 +151,21 @@ fn bench_serialize_aligned(bencher: &mut Bencher) {
.unwrap();
bencher.iter(|| {
let _ = serialize_parameters(&transaction_context, instruction_context, true).unwrap();
let _ =
serialize_parameters(&transaction_context, instruction_context, true, false).unwrap();
});
}
#[bench]
fn bench_serialize_aligned_copy_account_data(bencher: &mut Bencher) {
let transaction_context = create_inputs(bpf_loader::id(), 7);
let instruction_context = transaction_context
.get_current_instruction_context()
.unwrap();
bencher.iter(|| {
let _ =
serialize_parameters(&transaction_context, instruction_context, true, true).unwrap();
});
}
@ -149,7 +176,8 @@ fn bench_serialize_unaligned_max_accounts(bencher: &mut Bencher) {
.get_current_instruction_context()
.unwrap();
bencher.iter(|| {
let _ = serialize_parameters(&transaction_context, instruction_context, true).unwrap();
let _ =
serialize_parameters(&transaction_context, instruction_context, true, false).unwrap();
});
}
@ -161,6 +189,7 @@ fn bench_serialize_aligned_max_accounts(bencher: &mut Bencher) {
.unwrap();
bencher.iter(|| {
let _ = serialize_parameters(&transaction_context, instruction_context, true).unwrap();
let _ =
serialize_parameters(&transaction_context, instruction_context, true, false).unwrap();
});
}

86
programs/bpf_loader/src/lib.rs
View File

@ -20,21 +20,24 @@ use {
aligned_memory::AlignedMemory,
ebpf::{self, HOST_ALIGN, MM_HEAP_START},
elf::Executable,
memory_region::{MemoryCowCallback, MemoryMapping, MemoryRegion},
error::EbpfError,
memory_region::{AccessType, MemoryCowCallback, MemoryMapping, MemoryRegion},
verifier::RequisiteVerifier,
vm::{ContextObject, EbpfVm, ProgramResult, VerifiedExecutable},
},
solana_sdk::{
account::WritableAccount,
bpf_loader, bpf_loader_deprecated,
bpf_loader_upgradeable::{self, UpgradeableLoaderState},
clock::Slot,
entrypoint::SUCCESS,
entrypoint::{MAX_PERMITTED_DATA_INCREASE, SUCCESS},
feature_set::{
cap_accounts_data_allocations_per_transaction, cap_bpf_program_instruction_accounts,
delay_visibility_of_program_deployment, disable_deploy_of_alloc_free_syscall,
enable_bpf_loader_extend_program_ix, enable_bpf_loader_set_authority_checked_ix,
enable_program_redeployment_cooldown, limit_max_instruction_trace_length,
native_programs_consume_cu, remove_bpf_loader_incorrect_program_id, FeatureSet,
bpf_account_data_direct_mapping, cap_accounts_data_allocations_per_transaction,
cap_bpf_program_instruction_accounts, delay_visibility_of_program_deployment,
disable_deploy_of_alloc_free_syscall, enable_bpf_loader_extend_program_ix,
enable_bpf_loader_set_authority_checked_ix, enable_program_redeployment_cooldown,
limit_max_instruction_trace_length, native_programs_consume_cu,
remove_bpf_loader_incorrect_program_id, FeatureSet,
},
instruction::{AccountMeta, InstructionError},
loader_instruction::LoaderInstruction,
@ -302,8 +305,31 @@ pub fn create_vm<'a, 'b>(
) -> Result<EbpfVm<'a, RequisiteVerifier, InvokeContext<'b>>, Box<dyn std::error::Error>> {
let stack_size = stack.len();
let heap_size = heap.len();
let memory_mapping =
create_memory_mapping(program.get_executable(), stack, heap, regions, None)?;
let accounts = Arc::clone(invoke_context.transaction_context.accounts());
let memory_mapping = create_memory_mapping(
program.get_executable(),
stack,
heap,
regions,
Some(Box::new(move |index_in_transaction| {
// The two calls below can't really fail. If they fail because of a bug,
// whatever is writing will trigger an EbpfError::AccessViolation like
// if the region was readonly, and the transaction will fail gracefully.
let mut account = accounts
.try_borrow_mut(index_in_transaction as IndexOfAccount)
.map_err(|_| ())?;
accounts
.touch(index_in_transaction as IndexOfAccount)
.map_err(|_| ())?;
if account.is_shared() {
// See BorrowedAccount::make_data_mut() as to why we reserve extra
// MAX_PERMITTED_DATA_INCREASE bytes here.
account.reserve(MAX_PERMITTED_DATA_INCREASE);
}
Ok(account.data_as_mut_slice().as_mut_ptr() as u64)
})),
)?;
invoke_context.set_syscall_context(SyscallContext {
allocator: BpfAllocator::new(heap_size as u64),
orig_account_lengths,
@ -1524,6 +1550,9 @@ fn execute<'a, 'b: 'a>(
let use_jit = false;
#[cfg(all(not(target_os = "windows"), target_arch = "x86_64"))]
let use_jit = executable.get_executable().get_compiled_program().is_some();
let bpf_account_data_direct_mapping = invoke_context
.feature_set
.is_active(&bpf_account_data_direct_mapping::id());
let mut serialize_time = Measure::start("serialize");
let (parameter_bytes, regions, account_lengths) = serialization::serialize_parameters(
@ -1532,9 +1561,19 @@ fn execute<'a, 'b: 'a>(
invoke_context
.feature_set
.is_active(&cap_bpf_program_instruction_accounts::ID),
!bpf_account_data_direct_mapping,
)?;
serialize_time.stop();
// save the account addresses so in case of AccessViolation below we can
// map to InstructionError::ReadonlyDataModified, which is easier to
// diagnose from developers
let account_region_addrs = regions
.iter()
.map(|r| r.vm_addr..r.vm_addr.saturating_add(r.len))
.collect::<Vec<_>>();
let addr_is_account_data = |addr: u64| account_region_addrs.iter().any(|r| r.contains(&addr));
let mut create_vm_time = Measure::start("create_vm");
let mut execute_time;
let execution_result = {
@ -1597,7 +1636,24 @@ fn execute<'a, 'b: 'a>(
};
Err(Box::new(error) as Box<dyn std::error::Error>)
}
ProgramResult::Err(error) => Err(error),
ProgramResult::Err(error) => {
let error = match error.downcast_ref() {
Some(EbpfError::AccessViolation(
_pc,
AccessType::Store,
address,
_size,
_section_name,
)) if addr_is_account_data(*address) => {
// We can get here if direct_mapping is enabled and a program tries to
// write to a readonly account. Map the error to ReadonlyDataModified so
// it's easier for devs to diagnose what happened.
Box::new(InstructionError::ReadonlyDataModified)
}
_ => error,
};
Err(error)
}
_ => Ok(()),
}
};
@ -1606,12 +1662,14 @@ fn execute<'a, 'b: 'a>(
fn deserialize_parameters(
invoke_context: &mut InvokeContext,
parameter_bytes: &[u8],
copy_account_data: bool,
) -> Result<(), InstructionError> {
serialization::deserialize_parameters(
invoke_context.transaction_context,
invoke_context
.transaction_context
.get_current_instruction_context()?,
copy_account_data,
parameter_bytes,
&invoke_context.get_syscall_context()?.orig_account_lengths,
)
@ -1619,8 +1677,12 @@ fn execute<'a, 'b: 'a>(
let mut deserialize_time = Measure::start("deserialize");
let execute_or_deserialize_result = execution_result.and_then(|_| {
deserialize_parameters(invoke_context, parameter_bytes.as_slice())
.map_err(|error| Box::new(error) as Box<dyn std::error::Error>)
deserialize_parameters(
invoke_context,
parameter_bytes.as_slice(),
!bpf_account_data_direct_mapping,
)
.map_err(|error| Box::new(error) as Box<dyn std::error::Error>)
});
deserialize_time.stop();

937
programs/bpf_loader/src/serialization.rs
View File

File diff suppressed because it is too large Load Diff

970
programs/bpf_loader/src/syscalls/cpi.rs
View File

File diff suppressed because it is too large Load Diff

915
programs/bpf_loader/src/syscalls/mem_ops.rs
View File

@ -1,4 +1,9 @@
use {super::*, crate::declare_syscall};
use {
super::*,
crate::declare_syscall,
solana_rbpf::{error::EbpfError, memory_region::MemoryRegion},
std::slice,
};
fn mem_op_consume(invoke_context: &mut InvokeContext, n: u64) -> Result<(), Error> {
let compute_budget = invoke_context.get_compute_budget();
@ -26,32 +31,8 @@ declare_syscall!(
return Err(SyscallError::CopyOverlapping.into());
}
let dst_ptr = translate_slice_mut::<u8>(
memory_mapping,
dst_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?
.as_mut_ptr();
let src_ptr = translate_slice::<u8>(
memory_mapping,
src_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?
.as_ptr();
if !is_nonoverlapping(src_ptr as usize, n as usize, dst_ptr as usize, n as usize) {
unsafe {
std::ptr::copy(src_ptr, dst_ptr, n as usize);
}
} else {
unsafe {
std::ptr::copy_nonoverlapping(src_ptr, dst_ptr, n as usize);
}
}
Ok(0)
// host addresses can overlap so we always invoke memmove
memmove(invoke_context, dst_addr, src_addr, n, memory_mapping)
}
);
@ -69,24 +50,7 @@ declare_syscall!(
) -> Result<u64, Error> {
mem_op_consume(invoke_context, n)?;
let dst = translate_slice_mut::<u8>(
memory_mapping,
dst_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?;
let src = translate_slice::<u8>(
memory_mapping,
src_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?;
unsafe {
std::ptr::copy(src.as_ptr(), dst.as_mut_ptr(), n as usize);
}
Ok(0)
memmove(invoke_context, dst_addr, src_addr, n, memory_mapping)
}
);
@ -104,36 +68,46 @@ declare_syscall!(
) -> Result<u64, Error> {
mem_op_consume(invoke_context, n)?;
let s1 = translate_slice::<u8>(
memory_mapping,
s1_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?;
let s2 = translate_slice::<u8>(
memory_mapping,
s2_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?;
let cmp_result = translate_type_mut::<i32>(
memory_mapping,
cmp_result_addr,
invoke_context.get_check_aligned(),
)?;
let mut i = 0;
while i < n as usize {
let a = *s1.get(i).ok_or(SyscallError::InvalidLength)?;
let b = *s2.get(i).ok_or(SyscallError::InvalidLength)?;
if a != b {
*cmp_result = (a as i32).saturating_sub(b as i32);
return Ok(0);
};
i = i.saturating_add(1);
if invoke_context
.feature_set
.is_active(&feature_set::bpf_account_data_direct_mapping::id())
{
let cmp_result = translate_type_mut::<i32>(
memory_mapping,
cmp_result_addr,
invoke_context.get_check_aligned(),
)?;
*cmp_result = memcmp_non_contiguous(s1_addr, s2_addr, n, memory_mapping)?;
} else {
let s1 = translate_slice::<u8>(
memory_mapping,
s1_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?;
let s2 = translate_slice::<u8>(
memory_mapping,
s2_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?;
let cmp_result = translate_type_mut::<i32>(
memory_mapping,
cmp_result_addr,
invoke_context.get_check_aligned(),
)?;
debug_assert_eq!(s1.len(), n as usize);
debug_assert_eq!(s2.len(), n as usize);
// Safety:
// memcmp is marked unsafe since it assumes that the inputs are at least
// `n` bytes long. `s1` and `s2` are guaranteed to be exactly `n` bytes
// long because `translate_slice` would have failed otherwise.
*cmp_result = unsafe { memcmp(s1, s2, n as usize) };
}
*cmp_result = 0;
Ok(0)
}
);
@ -143,7 +117,7 @@ declare_syscall!(
SyscallMemset,
fn inner_call(
invoke_context: &mut InvokeContext,
s_addr: u64,
dst_addr: u64,
c: u64,
n: u64,
_arg4: u64,
@ -152,16 +126,793 @@ declare_syscall!(
) -> Result<u64, Error> {
mem_op_consume(invoke_context, n)?;
let s = translate_slice_mut::<u8>(
if invoke_context
.feature_set
.is_active(&feature_set::bpf_account_data_direct_mapping::id())
{
memset_non_contiguous(dst_addr, c as u8, n, memory_mapping)
} else {
let s = translate_slice_mut::<u8>(
memory_mapping,
dst_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?;
s.fill(c as u8);
Ok(0)
}
}
);
fn memmove(
invoke_context: &mut InvokeContext,
dst_addr: u64,
src_addr: u64,
n: u64,
memory_mapping: &MemoryMapping,
) -> Result<u64, Error> {
if invoke_context
.feature_set
.is_active(&feature_set::bpf_account_data_direct_mapping::id())
{
memmove_non_contiguous(dst_addr, src_addr, n, memory_mapping)
} else {
let dst_ptr = translate_slice_mut::<u8>(
memory_mapping,
s_addr,
dst_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?;
for val in s.iter_mut().take(n as usize) {
*val = c as u8;
}
)?
.as_mut_ptr();
let src_ptr = translate_slice::<u8>(
memory_mapping,
src_addr,
n,
invoke_context.get_check_aligned(),
invoke_context.get_check_size(),
)?
.as_ptr();
unsafe { std::ptr::copy(src_ptr, dst_ptr, n as usize) };
Ok(0)
}
);
}
fn memmove_non_contiguous(
dst_addr: u64,
src_addr: u64,
n: u64,
memory_mapping: &MemoryMapping,
) -> Result<u64, Error> {
let reverse = dst_addr.wrapping_sub(src_addr) < n;
iter_memory_pair_chunks(
AccessType::Load,
src_addr,
AccessType::Store,
dst_addr,
n,
memory_mapping,
reverse,
|src_host_addr, dst_host_addr, chunk_len| {
unsafe {
std::ptr::copy(
src_host_addr as *const u8,
dst_host_addr as *mut u8,
chunk_len,
)
};
Ok(0)
},
)
}
// Marked unsafe since it assumes that the slices are at least `n` bytes long.
unsafe fn memcmp(s1: &[u8], s2: &[u8], n: usize) -> i32 {
for i in 0..n {
let a = *s1.get_unchecked(i);
let b = *s2.get_unchecked(i);
if a != b {
return (a as i32).saturating_sub(b as i32);
};
}
0
}
fn memcmp_non_contiguous(
src_addr: u64,
dst_addr: u64,
n: u64,
memory_mapping: &MemoryMapping,
) -> Result<i32, Error> {
match iter_memory_pair_chunks(
AccessType::Load,
src_addr,
AccessType::Load,
dst_addr,
n,
memory_mapping,
false,
|s1_addr, s2_addr, chunk_len| {
let res = unsafe {
let s1 = slice::from_raw_parts(s1_addr as *const u8, chunk_len);
let s2 = slice::from_raw_parts(s2_addr as *const u8, chunk_len);
// Safety:
// memcmp is marked unsafe since it assumes that s1 and s2 are exactly chunk_len
// long. The whole point of iter_memory_pair_chunks is to find same length chunks
// across two memory regions.
memcmp(s1, s2, chunk_len)
};
if res != 0 {
return Err(MemcmpError::Diff(res).into());
}
Ok(0)
},
) {
Ok(res) => Ok(res),
Err(error) => match error.downcast_ref() {
Some(MemcmpError::Diff(diff)) => Ok(*diff),
_ => Err(error),
},
}
}
#[derive(Debug)]
enum MemcmpError {
Diff(i32),
}
impl std::fmt::Display for MemcmpError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
MemcmpError::Diff(diff) => write!(f, "memcmp diff: {diff}"),
}
}
}
impl std::error::Error for MemcmpError {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
match self {
MemcmpError::Diff(_) => None,
}
}
}
fn memset_non_contiguous(
dst_addr: u64,
c: u8,
n: u64,
memory_mapping: &MemoryMapping,
) -> Result<u64, Error> {
let dst_chunk_iter = MemoryChunkIterator::new(memory_mapping, AccessType::Store, dst_addr, n)?;
for item in dst_chunk_iter {
let (dst_region, dst_vm_addr, dst_len) = item?;
let dst_host_addr = Result::from(dst_region.vm_to_host(dst_vm_addr, dst_len as u64))?;
unsafe { slice::from_raw_parts_mut(dst_host_addr as *mut u8, dst_len).fill(c) }
}
Ok(0)
}
fn iter_memory_pair_chunks<T, F>(
src_access: AccessType,
src_addr: u64,
dst_access: AccessType,
mut dst_addr: u64,
n: u64,
memory_mapping: &MemoryMapping,
reverse: bool,
mut fun: F,
) -> Result<T, Error>
where
T: Default,
F: FnMut(*const u8, *const u8, usize) -> Result<T, Error>,
{
let mut src_chunk_iter = MemoryChunkIterator::new(memory_mapping, src_access, src_addr, n)
.map_err(EbpfError::from)?;
loop {
// iterate source chunks
let (src_region, src_vm_addr, mut src_len) = match if reverse {
src_chunk_iter.next_back()
} else {
src_chunk_iter.next()
} {
Some(item) => item?,
None => break,
};
let mut src_host_addr = Result::from(src_region.vm_to_host(src_vm_addr, src_len as u64))?;
let mut dst_chunk_iter = MemoryChunkIterator::new(memory_mapping, dst_access, dst_addr, n)
.map_err(EbpfError::from)?;
// iterate over destination chunks until this source chunk has been completely copied
while src_len > 0 {
loop {
let (dst_region, dst_vm_addr, dst_len) = match if reverse {
dst_chunk_iter.next_back()
} else {
dst_chunk_iter.next()
} {
Some(item) => item?,
None => break,
};
let dst_host_addr =
Result::from(dst_region.vm_to_host(dst_vm_addr, dst_len as u64))?;
let chunk_len = src_len.min(dst_len);
fun(
src_host_addr as *const u8,
dst_host_addr as *const u8,
chunk_len,
)?;
src_len = src_len.saturating_sub(chunk_len);
if reverse {
dst_addr = dst_addr.saturating_sub(chunk_len as u64);
} else {
dst_addr = dst_addr.saturating_add(chunk_len as u64);
}
if src_len == 0 {
break;
}
src_host_addr = src_host_addr.saturating_add(chunk_len as u64);
}
}
}
Ok(T::default())
}
struct MemoryChunkIterator<'a> {
memory_mapping: &'a MemoryMapping<'a>,
access_type: AccessType,
initial_vm_addr: u64,
vm_addr_start: u64,
// exclusive end index (start + len, so one past the last valid address)
vm_addr_end: u64,
len: u64,
}
impl<'a> MemoryChunkIterator<'a> {
fn new(
memory_mapping: &'a MemoryMapping,
access_type: AccessType,
vm_addr: u64,
len: u64,
) -> Result<MemoryChunkIterator<'a>, EbpfError> {
let vm_addr_end = vm_addr.checked_add(len).ok_or(EbpfError::AccessViolation(
0,
access_type,
vm_addr,
len,
"unknown",
))?;
Ok(MemoryChunkIterator {
memory_mapping,
access_type,
initial_vm_addr: vm_addr,
len,
vm_addr_start: vm_addr,
vm_addr_end,
})
}
fn region(&mut self, vm_addr: u64) -> Result<&'a MemoryRegion, Error> {
match self.memory_mapping.region(self.access_type, vm_addr) {
Ok(region) => Ok(region),
Err(error) => match error.downcast_ref() {
Some(EbpfError::AccessViolation(pc, access_type, _vm_addr, _len, name)) => {
Err(Box::new(EbpfError::AccessViolation(
*pc,
*access_type,
self.initial_vm_addr,
self.len,
name,
)))
}
Some(EbpfError::StackAccessViolation(pc, access_type, _vm_addr, _len, frame)) => {
Err(Box::new(EbpfError::StackAccessViolation(
*pc,
*access_type,
self.initial_vm_addr,
self.len,
*frame,
)))
}
_ => Err(error),
},
}
}
}
impl<'a> Iterator for MemoryChunkIterator<'a> {
type Item = Result<(&'a MemoryRegion, u64, usize), Error>;
fn next(&mut self) -> Option<Self::Item> {
if self.vm_addr_start == self.vm_addr_end {
return None;
}
let region = match self.region(self.vm_addr_start) {
Ok(region) => region,
Err(e) => {
self.vm_addr_start = self.vm_addr_end;
return Some(Err(e));
}
};
let vm_addr = self.vm_addr_start;
let chunk_len = if region.vm_addr_end <= self.vm_addr_end {
// consume the whole region
let len = region.vm_addr_end.saturating_sub(self.vm_addr_start);
self.vm_addr_start = region.vm_addr_end;
len
} else {
// consume part of the region
let len = self.vm_addr_end.saturating_sub(self.vm_addr_start);
self.vm_addr_start = self.vm_addr_end;
len
};
Some(Ok((region, vm_addr, chunk_len as usize)))
}
}
impl<'a> DoubleEndedIterator for MemoryChunkIterator<'a> {
fn next_back(&mut self) -> Option<Self::Item> {
if self.vm_addr_start == self.vm_addr_end {
return None;
}
let region = match self.region(self.vm_addr_end.saturating_sub(1)) {
Ok(region) => region,
Err(e) => {
self.vm_addr_start = self.vm_addr_end;
return Some(Err(e));
}
};
let chunk_len = if region.vm_addr >= self.vm_addr_start {
// consume the whole region
let len = self.vm_addr_end.saturating_sub(region.vm_addr);
self.vm_addr_end = region.vm_addr;
len
} else {
// consume part of the region
let len = self.vm_addr_end.saturating_sub(self.vm_addr_start);
self.vm_addr_end = self.vm_addr_start;
len
};
Some(Ok((region, self.vm_addr_end, chunk_len as usize)))
}
}
#[cfg(test)]
#[allow(clippy::indexing_slicing)]
mod tests {
use {super::*, solana_rbpf::ebpf::MM_PROGRAM_START};
fn to_chunk_vec<'a>(
iter: impl Iterator<Item = Result<(&'a MemoryRegion, u64, usize), Error>>,
) -> Vec<(u64, usize)> {
iter.flat_map(|res| res.map(|(_, vm_addr, len)| (vm_addr, len)))
.collect::<Vec<_>>()
}
#[test]
#[should_panic(expected = "AccessViolation")]
fn test_memory_chunk_iterator_no_regions() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let memory_mapping = MemoryMapping::new(vec![], &config).unwrap();
let mut src_chunk_iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, 0, 1).unwrap();
src_chunk_iter.next().unwrap().unwrap();
}
#[test]
#[should_panic(expected = "AccessViolation")]
fn test_memory_chunk_iterator_new_out_of_bounds_upper() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let memory_mapping = MemoryMapping::new(vec![], &config).unwrap();
let mut src_chunk_iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, u64::MAX, 1).unwrap();
src_chunk_iter.next().unwrap().unwrap();
}
#[test]
fn test_memory_chunk_iterator_out_of_bounds() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mem1 = vec![0xFF; 42];
let memory_mapping = MemoryMapping::new(
vec![MemoryRegion::new_readonly(&mem1, MM_PROGRAM_START)],
&config,
)
.unwrap();
// check oob at the lower bound on the first next()
let mut src_chunk_iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, MM_PROGRAM_START - 1, 42)
.unwrap();
assert!(matches!(
src_chunk_iter.next().unwrap().unwrap_err().downcast_ref().unwrap(),
EbpfError::AccessViolation(0, AccessType::Load, addr, 42, "unknown") if *addr == MM_PROGRAM_START - 1
));
// check oob at the upper bound. Since the memory mapping isn't empty,
// this always happens on the second next().
let mut src_chunk_iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, MM_PROGRAM_START, 43)
.unwrap();
assert!(src_chunk_iter.next().unwrap().is_ok());
assert!(matches!(
src_chunk_iter.next().unwrap().unwrap_err().downcast_ref().unwrap(),
EbpfError::AccessViolation(0, AccessType::Load, addr, 43, "program") if *addr == MM_PROGRAM_START
));
// check oob at the upper bound on the first next_back()
let mut src_chunk_iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, MM_PROGRAM_START, 43)
.unwrap()
.rev();
assert!(matches!(
src_chunk_iter.next().unwrap().unwrap_err().downcast_ref().unwrap(),
EbpfError::AccessViolation(0, AccessType::Load, addr, 43, "program") if *addr == MM_PROGRAM_START
));
// check oob at the upper bound on the 2nd next_back()
let mut src_chunk_iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, MM_PROGRAM_START - 1, 43)
.unwrap()
.rev();
assert!(src_chunk_iter.next().unwrap().is_ok());
assert!(matches!(
src_chunk_iter.next().unwrap().unwrap_err().downcast_ref().unwrap(),
EbpfError::AccessViolation(0, AccessType::Load, addr, 43, "unknown") if *addr == MM_PROGRAM_START - 1
));
}
#[test]
fn test_memory_chunk_iterator_one() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mem1 = vec![0xFF; 42];
let memory_mapping = MemoryMapping::new(
vec![MemoryRegion::new_readonly(&mem1, MM_PROGRAM_START)],
&config,
)
.unwrap();
// check lower bound
let mut src_chunk_iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, MM_PROGRAM_START - 1, 1)
.unwrap();
assert!(src_chunk_iter.next().unwrap().is_err());
// check upper bound
let mut src_chunk_iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, MM_PROGRAM_START + 42, 1)
.unwrap();
assert!(src_chunk_iter.next().unwrap().is_err());
for (vm_addr, len) in [
(MM_PROGRAM_START, 0),
(MM_PROGRAM_START + 42, 0),
(MM_PROGRAM_START, 1),
(MM_PROGRAM_START, 42),
(MM_PROGRAM_START + 41, 1),
] {
for rev in [true, false] {
let iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, vm_addr, len)
.unwrap();
let res = if rev {
to_chunk_vec(iter.rev())
} else {
to_chunk_vec(iter)
};
if len == 0 {
assert_eq!(res, &[]);
} else {
assert_eq!(res, &[(vm_addr, len as usize)]);
}
}
}
}
#[test]
fn test_memory_chunk_iterator_two() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mem1 = vec![0x11; 8];
let mem2 = vec![0x22; 4];
let memory_mapping = MemoryMapping::new(
vec![
MemoryRegion::new_readonly(&mem1, MM_PROGRAM_START),
MemoryRegion::new_readonly(&mem2, MM_PROGRAM_START + 8),
],
&config,
)
.unwrap();
for (vm_addr, len, mut expected) in [
(MM_PROGRAM_START, 8, vec![(MM_PROGRAM_START, 8)]),
(
MM_PROGRAM_START + 7,
2,
vec![(MM_PROGRAM_START + 7, 1), (MM_PROGRAM_START + 8, 1)],
),
(MM_PROGRAM_START + 8, 4, vec![(MM_PROGRAM_START + 8, 4)]),
] {
for rev in [false, true] {
let iter =
MemoryChunkIterator::new(&memory_mapping, AccessType::Load, vm_addr, len)
.unwrap();
let res = if rev {
expected.reverse();
to_chunk_vec(iter.rev())
} else {
to_chunk_vec(iter)
};
assert_eq!(res, expected);
}
}
}
#[test]
fn test_iter_memory_pair_chunks_short() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mem1 = vec![0x11; 8];
let mem2 = vec![0x22; 4];
let memory_mapping = MemoryMapping::new(
vec![
MemoryRegion::new_readonly(&mem1, MM_PROGRAM_START),
MemoryRegion::new_readonly(&mem2, MM_PROGRAM_START + 8),
],
&config,
)
.unwrap();
// dst is shorter than src
assert!(matches!(
iter_memory_pair_chunks(
AccessType::Load,
MM_PROGRAM_START,
AccessType::Load,
MM_PROGRAM_START + 8,
8,
&memory_mapping,
false,
|_src, _dst, _len| Ok::<_, Error>(0),
).unwrap_err().downcast_ref().unwrap(),
EbpfError::AccessViolation(0, AccessType::Load, addr, 8, "program") if *addr == MM_PROGRAM_START + 8
));
// src is shorter than dst
assert!(matches!(
iter_memory_pair_chunks(
AccessType::Load,
MM_PROGRAM_START + 10,
AccessType::Load,
MM_PROGRAM_START + 2,
3,
&memory_mapping,
false,
|_src, _dst, _len| Ok::<_, Error>(0),
).unwrap_err().downcast_ref().unwrap(),
EbpfError::AccessViolation(0, AccessType::Load, addr, 3, "program") if *addr == MM_PROGRAM_START + 10
));
}
#[test]
#[should_panic(expected = "AccessViolation(0, Store, 4294967296, 4")]
fn test_memmove_non_contiguous_readonly() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mem1 = vec![0x11; 8];
let mem2 = vec![0x22; 4];
let memory_mapping = MemoryMapping::new(
vec![
MemoryRegion::new_readonly(&mem1, MM_PROGRAM_START),
MemoryRegion::new_readonly(&mem2, MM_PROGRAM_START + 8),
],
&config,
)
.unwrap();
memmove_non_contiguous(MM_PROGRAM_START, MM_PROGRAM_START + 8, 4, &memory_mapping).unwrap();
}
#[test]
fn test_overlapping_memmove_non_contiguous_right() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mem1 = vec![0x11; 1];
let mut mem2 = vec![0x22; 2];
let mut mem3 = vec![0x33; 3];
let mut mem4 = vec![0x44; 4];
let memory_mapping = MemoryMapping::new(
vec![
MemoryRegion::new_readonly(&mem1, MM_PROGRAM_START),
MemoryRegion::new_writable(&mut mem2, MM_PROGRAM_START + 1),
MemoryRegion::new_writable(&mut mem3, MM_PROGRAM_START + 3),
MemoryRegion::new_writable(&mut mem4, MM_PROGRAM_START + 6),
],
&config,
)
.unwrap();
// overlapping memmove right - the implementation will copy backwards
assert_eq!(
memmove_non_contiguous(MM_PROGRAM_START + 1, MM_PROGRAM_START, 7, &memory_mapping)
.unwrap(),
0
);
assert_eq!(&mem1, &[0x11]);
assert_eq!(&mem2, &[0x11, 0x22]);
assert_eq!(&mem3, &[0x22, 0x33, 0x33]);
assert_eq!(&mem4, &[0x33, 0x44, 0x44, 0x44]);
}
#[test]
fn test_overlapping_memmove_non_contiguous_left() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mut mem1 = vec![0x11; 1];
let mut mem2 = vec![0x22; 2];
let mut mem3 = vec![0x33; 3];
let mut mem4 = vec![0x44; 4];
let memory_mapping = MemoryMapping::new(
vec![
MemoryRegion::new_writable(&mut mem1, MM_PROGRAM_START),
MemoryRegion::new_writable(&mut mem2, MM_PROGRAM_START + 1),
MemoryRegion::new_writable(&mut mem3, MM_PROGRAM_START + 3),
MemoryRegion::new_writable(&mut mem4, MM_PROGRAM_START + 6),
],
&config,
)
.unwrap();
// overlapping memmove left - the implementation will copy forward
assert_eq!(
memmove_non_contiguous(MM_PROGRAM_START, MM_PROGRAM_START + 1, 7, &memory_mapping)
.unwrap(),
0
);
assert_eq!(&mem1, &[0x22]);
assert_eq!(&mem2, &[0x22, 0x33]);
assert_eq!(&mem3, &[0x33, 0x33, 0x44]);
assert_eq!(&mem4, &[0x44, 0x44, 0x44, 0x44]);
}
#[test]
#[should_panic(expected = "AccessViolation(0, Store, 4294967296, 9")]
fn test_memset_non_contiguous_readonly() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mut mem1 = vec![0x11; 8];
let mem2 = vec![0x22; 4];
let memory_mapping = MemoryMapping::new(
vec![
MemoryRegion::new_writable(&mut mem1, MM_PROGRAM_START),
MemoryRegion::new_readonly(&mem2, MM_PROGRAM_START + 8),
],
&config,
)
.unwrap();
assert_eq!(
memset_non_contiguous(MM_PROGRAM_START, 0x33, 9, &memory_mapping).unwrap(),
0
);
}
#[test]
fn test_memset_non_contiguous() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mem1 = vec![0x11; 1];
let mut mem2 = vec![0x22; 2];
let mut mem3 = vec![0x33; 3];
let mut mem4 = vec![0x44; 4];
let memory_mapping = MemoryMapping::new(
vec![
MemoryRegion::new_readonly(&mem1, MM_PROGRAM_START),
MemoryRegion::new_writable(&mut mem2, MM_PROGRAM_START + 1),
MemoryRegion::new_writable(&mut mem3, MM_PROGRAM_START + 3),
MemoryRegion::new_writable(&mut mem4, MM_PROGRAM_START + 6),
],
&config,
)
.unwrap();
assert_eq!(
memset_non_contiguous(MM_PROGRAM_START + 1, 0x55, 7, &memory_mapping).unwrap(),
0
);
assert_eq!(&mem1, &[0x11]);
assert_eq!(&mem2, &[0x55, 0x55]);
assert_eq!(&mem3, &[0x55, 0x55, 0x55]);
assert_eq!(&mem4, &[0x55, 0x55, 0x44, 0x44]);
}
#[test]
fn test_memcmp_non_contiguous() {
let config = Config {
aligned_memory_mapping: false,
..Config::default()
};
let mem1 = b"foo".to_vec();
let mem2 = b"barbad".to_vec();
let mem3 = b"foobarbad".to_vec();
let memory_mapping = MemoryMapping::new(
vec![
MemoryRegion::new_readonly(&mem1, MM_PROGRAM_START),
MemoryRegion::new_readonly(&mem2, MM_PROGRAM_START + 3),
MemoryRegion::new_readonly(&mem3, MM_PROGRAM_START + 9),
],
&config,
)
.unwrap();
// non contiguous src
assert_eq!(
memcmp_non_contiguous(MM_PROGRAM_START, MM_PROGRAM_START + 9, 9, &memory_mapping)
.unwrap(),
0
);
// non contiguous dst
assert_eq!(
memcmp_non_contiguous(
MM_PROGRAM_START + 10,
MM_PROGRAM_START + 1,
8,
&memory_mapping
)
.unwrap(),
0
);
// diff
assert_eq!(
memcmp_non_contiguous(
MM_PROGRAM_START + 1,
MM_PROGRAM_START + 11,
5,
&memory_mapping
)
.unwrap(),
unsafe { memcmp(b"oobar", b"obarb", 5) }
);
}
}

3
programs/bpf_loader/src/syscalls/mod.rs
View File

@ -30,6 +30,7 @@ use {
big_mod_exp::{big_mod_exp, BigModExpParams},
blake3, bpf_loader, bpf_loader_deprecated, bpf_loader_upgradeable,
entrypoint::{BPF_ALIGN_OF_U128, MAX_PERMITTED_DATA_INCREASE, SUCCESS},
feature_set::bpf_account_data_direct_mapping,
feature_set::FeatureSet,
feature_set::{
self, blake3_syscall_enabled, check_syscall_outputs_do_not_overlap,
@ -174,7 +175,7 @@ pub fn create_loader<'a>(
enable_elf_vaddr: false,
reject_rodata_stack_overlap: false,
new_elf_parser: feature_set.is_active(&switch_to_new_elf_parser::id()),
aligned_memory_mapping: true,
aligned_memory_mapping: !feature_set.is_active(&bpf_account_data_direct_mapping::id()),
// Warning, do not use `Config::default()` so that configuration here is explicit.
};

2
programs/loader-v3/src/lib.rs
View File

@ -691,7 +691,7 @@ mod tests {
authority_address,
})
.unwrap();
program_account.data_mut()[loader_v3::LoaderV3State::program_data_offset()..]
program_account.data_as_mut_slice()[loader_v3::LoaderV3State::program_data_offset()..]
.copy_from_slice(&elf_bytes);
program_account
}

20
programs/sbf/benches/bpf_loader.rs
View File

@ -3,7 +3,10 @@
#![allow(clippy::uninlined_format_args)]
#![allow(clippy::integer_arithmetic)]
use {solana_rbpf::memory_region::MemoryState, std::slice};
use {
solana_rbpf::memory_region::MemoryState,
solana_sdk::feature_set::bpf_account_data_direct_mapping, std::slice,
};
extern crate test;
@ -66,7 +69,7 @@ macro_rules! with_mock_invoke_context {
index_in_caller: 2,
index_in_callee: 0,
is_signer: false,
is_writable: false,
is_writable: true,
}];
solana_program_runtime::with_mock_invoke_context!(
$invoke_context,
@ -223,6 +226,9 @@ fn bench_create_vm(bencher: &mut Bencher) {
const BUDGET: u64 = 200_000;
invoke_context.mock_set_remaining(BUDGET);
let direct_mapping = invoke_context
.feature_set
.is_active(&bpf_account_data_direct_mapping::id());
let loader = create_loader(
&invoke_context.feature_set,
&ComputeBudget::default(),
@ -244,7 +250,8 @@ fn bench_create_vm(bencher: &mut Bencher) {
.transaction_context
.get_current_instruction_context()
.unwrap(),
true, // should_cap_ix_accounts
true, // should_cap_ix_accounts
!direct_mapping, // copy_account_data
)
.unwrap();
@ -267,6 +274,10 @@ fn bench_instruction_count_tuner(_bencher: &mut Bencher) {
const BUDGET: u64 = 200_000;
invoke_context.mock_set_remaining(BUDGET);
let direct_mapping = invoke_context
.feature_set
.is_active(&bpf_account_data_direct_mapping::id());
// Serialize account data
let (_serialized, regions, account_lengths) = serialize_parameters(
invoke_context.transaction_context,
@ -274,7 +285,8 @@ fn bench_instruction_count_tuner(_bencher: &mut Bencher) {
.transaction_context
.get_current_instruction_context()
.unwrap(),
true, // should_cap_ix_accounts
true, // should_cap_ix_accounts
!direct_mapping, // copy_account_data
)
.unwrap();

12
programs/sbf/rust/realloc/src/instructions.rs
View File

@ -15,6 +15,7 @@ pub const DEALLOC_AND_ASSIGN_TO_CALLER: u8 = 7;
pub const CHECK: u8 = 8;
pub const ZERO_INIT: u8 = 9;
pub const REALLOC_EXTEND_AND_UNDO: u8 = 10;
pub const EXTEND_AND_WRITE_U64: u8 = 11;
pub fn realloc(program_id: &Pubkey, address: &Pubkey, size: usize, bump: &mut u8) -> Instruction {
let mut instruction_data = vec![REALLOC, *bump];
@ -88,3 +89,14 @@ pub fn realloc_extend_and_undo(
vec![AccountMeta::new(*address, false)],
)
}
pub fn extend_and_write_u64(program_id: &Pubkey, address: &Pubkey, value: u64) -> Instruction {
let mut instruction_data = vec![EXTEND_AND_WRITE_U64];
instruction_data.extend_from_slice(&value.to_le_bytes());
Instruction::new_with_bytes(
*program_id,
&instruction_data,
vec![AccountMeta::new(*address, false)],
)
}

38
programs/sbf/rust/realloc/src/processor.rs
View File

@ -13,7 +13,7 @@ use {
pubkey::Pubkey,
system_instruction, system_program,
},
std::convert::TryInto,
std::{convert::TryInto, mem},
};
solana_program::entrypoint!(process_instruction);
@ -61,6 +61,42 @@ fn process_instruction(
assert_eq!(new_len, account.data_len());
account.try_borrow_mut_data()?[pre_len..].fill(fill);
}
// extend the account and do a 8-bytes write across the original account
// length and the realloc region
EXTEND_AND_WRITE_U64 => {
let pre_len = account.data_len();
let new_len = mem::size_of::<u64>();
assert!(pre_len < new_len);
account.realloc(new_len, false)?;
assert_eq!(new_len, account.data_len());
let (bytes, _) = instruction_data[1..].split_at(new_len);
let value = u64::from_le_bytes(bytes.try_into().unwrap());
msg!(
"write {} to account {:p}",
value,
account.try_borrow_data().unwrap().as_ptr()
);
// exercise memory write
unsafe {
*account
.try_borrow_mut_data()
.unwrap()
.as_mut_ptr()
.cast::<u64>() = value
};
// exercise memory read
assert_eq!(
unsafe {
*account
.try_borrow_mut_data()
.unwrap()
.as_ptr()
.cast::<u64>()
},
value
);
}
REALLOC_AND_ASSIGN => {
msg!("realloc and assign");
account.realloc(MAX_PERMITTED_DATA_INCREASE, false)?;

539
programs/sbf/tests/programs.rs
View File

@ -39,7 +39,7 @@ use {
clock::MAX_PROCESSING_AGE,
compute_budget::ComputeBudgetInstruction,
entrypoint::MAX_PERMITTED_DATA_INCREASE,
feature_set::FeatureSet,
feature_set::{self, FeatureSet},
fee::FeeStructure,
loader_instruction,
message::{v0::LoadedAddresses, SanitizedMessage},
@ -1110,7 +1110,7 @@ fn test_program_sbf_invoke_sanity() {
.map(|ix| &message.account_keys[ix.instruction.program_id_index as usize])
.cloned()
.collect();
assert_eq!(invoked_programs, vec![system_program::id()]);
assert_eq!(invoked_programs, vec![]);
assert_eq!(
result.unwrap_err(),
TransactionError::InstructionError(0, InstructionError::ProgramFailedToComplete)
@ -2761,200 +2761,57 @@ fn test_program_sbf_realloc() {
} = create_genesis_config(1_000_000_000_000);
let mint_pubkey = mint_keypair.pubkey();
let signer = &[&mint_keypair];
for direct_mapping in [false, true] {
let mut bank = Bank::new_for_tests(&genesis_config);
let feature_set = Arc::make_mut(&mut bank.feature_set);
// by default test banks have all features enabled, so we only need to
// disable when needed
if !direct_mapping {
feature_set.deactivate(&feature_set::bpf_account_data_direct_mapping::id());
}
let bank = Arc::new(bank);
let bank_client = BankClient::new_shared(&bank);
let bank = Bank::new_for_tests(&genesis_config);
let bank = Arc::new(bank);
let bank_client = BankClient::new_shared(&bank);
let program_id = load_program(
&bank_client,
&bpf_loader::id(),
&mint_keypair,
"solana_sbf_rust_realloc",
);
let program_id = load_program(
&bank_client,
&bpf_loader::id(),
&mint_keypair,
"solana_sbf_rust_realloc",
);
let mut bump = 0;
let keypair = Keypair::new();
let pubkey = keypair.pubkey();
let account = AccountSharedData::new(START_BALANCE, 5, &program_id);
bank.store_account(&pubkey, &account);
let mut bump = 0;
let keypair = Keypair::new();
let pubkey = keypair.pubkey();
let account = AccountSharedData::new(START_BALANCE, 5, &program_id);
bank.store_account(&pubkey, &account);
// Realloc RO account
let mut instruction = realloc(&program_id, &pubkey, 0, &mut bump);
instruction.accounts[0].is_writable = false;
assert_eq!(
bank_client
.send_and_confirm_message(signer, Message::new(&[instruction], Some(&mint_pubkey),),)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::ReadonlyDataModified)
);
// Realloc RO account
let mut instruction = realloc(&program_id, &pubkey, 0, &mut bump);
instruction.accounts[0].is_writable = false;
assert_eq!(
bank_client
.send_and_confirm_message(signer, Message::new(&[instruction], Some(&mint_pubkey),),)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::ReadonlyDataModified)
);
// Realloc account to overflow
assert_eq!(
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(&program_id, &pubkey, usize::MAX, &mut bump)],
Some(&mint_pubkey),
),
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::InvalidRealloc)
);
// Realloc account to 0
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(&program_id, &pubkey, 0, &mut bump)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(0, data.len());
// Realloc account to max then undo
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc_extend_and_undo(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE,
&mut bump,
)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(0, data.len());
// Realloc account to max + 1 then undo
assert_eq!(
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc_extend_and_undo(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE + 1,
&mut bump,
)],
Some(&mint_pubkey),
),
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::InvalidRealloc)
);
// Realloc to max + 1
assert_eq!(
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE + 1,
&mut bump
)],
Some(&mint_pubkey),
),
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::InvalidRealloc)
);
// Realloc to max length in max increase increments
for i in 0..MAX_PERMITTED_DATA_LENGTH as usize / MAX_PERMITTED_DATA_INCREASE {
let mut bump = i as u64;
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc_extend_and_fill(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE,
1,
&mut bump,
)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!((i + 1) * MAX_PERMITTED_DATA_INCREASE, data.len());
}
for i in 0..data.len() {
assert_eq!(data[i], 1);
}
// and one more time should fail
assert_eq!(
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc_extend(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE,
&mut bump
)],
Some(&mint_pubkey),
// Realloc account to overflow
assert_eq!(
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(&program_id, &pubkey, usize::MAX, &mut bump)],
Some(&mint_pubkey),
),
)
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::InvalidRealloc)
);
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::InvalidRealloc)
);
// Realloc to 0
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(&program_id, &pubkey, 0, &mut bump)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(0, data.len());
// Realloc and assign
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[Instruction::new_with_bytes(
program_id,
&[REALLOC_AND_ASSIGN],
vec![AccountMeta::new(pubkey, false)],
)],
Some(&mint_pubkey),
),
)
.unwrap();
let account = bank.get_account(&pubkey).unwrap();
assert_eq!(&solana_sdk::system_program::id(), account.owner());
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(MAX_PERMITTED_DATA_INCREASE, data.len());
// Realloc to 0 with wrong owner
assert_eq!(
// Realloc account to 0
bank_client
.send_and_confirm_message(
signer,
@ -2963,82 +2820,264 @@ fn test_program_sbf_realloc() {
Some(&mint_pubkey),
),
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::AccountDataSizeChanged)
);
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(0, data.len());
// realloc and assign to self via cpi
assert_eq!(
// Realloc account to max then undo
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc_extend_and_undo(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE,
&mut bump,
)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(0, data.len());
// Realloc account to max + 1 then undo
assert_eq!(
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc_extend_and_undo(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE + 1,
&mut bump,
)],
Some(&mint_pubkey),
),
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::InvalidRealloc)
);
// Realloc to max + 1
assert_eq!(
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE + 1,
&mut bump
)],
Some(&mint_pubkey),
),
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::InvalidRealloc)
);
// Realloc to max length in max increase increments
for i in 0..MAX_PERMITTED_DATA_LENGTH as usize / MAX_PERMITTED_DATA_INCREASE {
let mut bump = i as u64;
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc_extend_and_fill(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE,
1,
&mut bump,
)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!((i + 1) * MAX_PERMITTED_DATA_INCREASE, data.len());
}
for i in 0..data.len() {
assert_eq!(data[i], 1);
}
// and one more time should fail
assert_eq!(
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc_extend(
&program_id,
&pubkey,
MAX_PERMITTED_DATA_INCREASE,
&mut bump
)],
Some(&mint_pubkey),
)
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::InvalidRealloc)
);
// Realloc to 6 bytes
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(&program_id, &pubkey, 6, &mut bump)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(6, data.len());
// Extend by 2 bytes and write a u64. This ensures that we can do writes that span the original
// account length (6 bytes) and the realloc data (2 bytes).
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[extend_and_write_u64(
&program_id,
&pubkey,
0x1122334455667788,
)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(8, data.len());
assert_eq!(0x1122334455667788, unsafe { *data.as_ptr().cast::<u64>() });
// Realloc to 0
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(&program_id, &pubkey, 0, &mut bump)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(0, data.len());
// Realloc and assign
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[Instruction::new_with_bytes(
program_id,
&[REALLOC_AND_ASSIGN],
vec![AccountMeta::new(pubkey, false)],
)],
Some(&mint_pubkey),
),
)
.unwrap();
let account = bank.get_account(&pubkey).unwrap();
assert_eq!(&solana_sdk::system_program::id(), account.owner());
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(MAX_PERMITTED_DATA_INCREASE, data.len());
// Realloc to 0 with wrong owner
assert_eq!(
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(&program_id, &pubkey, 0, &mut bump)],
Some(&mint_pubkey),
),
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::AccountDataSizeChanged)
);
// realloc and assign to self via cpi
assert_eq!(
bank_client
.send_and_confirm_message(
&[&mint_keypair, &keypair],
Message::new(
&[Instruction::new_with_bytes(
program_id,
&[REALLOC_AND_ASSIGN_TO_SELF_VIA_SYSTEM_PROGRAM],
vec![
AccountMeta::new(pubkey, true),
AccountMeta::new(solana_sdk::system_program::id(), false),
],
)],
Some(&mint_pubkey),
)
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::AccountDataSizeChanged)
);
// Assign to self and realloc via cpi
bank_client
.send_and_confirm_message(
&[&mint_keypair, &keypair],
Message::new(
&[Instruction::new_with_bytes(
program_id,
&[REALLOC_AND_ASSIGN_TO_SELF_VIA_SYSTEM_PROGRAM],
&[ASSIGN_TO_SELF_VIA_SYSTEM_PROGRAM_AND_REALLOC],
vec![
AccountMeta::new(pubkey, true),
AccountMeta::new(solana_sdk::system_program::id(), false),
],
)],
Some(&mint_pubkey),
)
),
)
.unwrap_err()
.unwrap(),
TransactionError::InstructionError(0, InstructionError::AccountDataSizeChanged)
);
.unwrap();
let account = bank.get_account(&pubkey).unwrap();
assert_eq!(&program_id, account.owner());
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(2 * MAX_PERMITTED_DATA_INCREASE, data.len());
// Assign to self and realloc via cpi
bank_client
.send_and_confirm_message(
&[&mint_keypair, &keypair],
Message::new(
&[Instruction::new_with_bytes(
program_id,
&[ASSIGN_TO_SELF_VIA_SYSTEM_PROGRAM_AND_REALLOC],
vec![
AccountMeta::new(pubkey, true),
AccountMeta::new(solana_sdk::system_program::id(), false),
],
)],
Some(&mint_pubkey),
),
)
.unwrap();
let account = bank.get_account(&pubkey).unwrap();
assert_eq!(&program_id, account.owner());
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(2 * MAX_PERMITTED_DATA_INCREASE, data.len());
// Realloc to 0
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(&program_id, &pubkey, 0, &mut bump)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(0, data.len());
// Realloc to 0
bank_client
.send_and_confirm_message(
signer,
Message::new(
&[realloc(&program_id, &pubkey, 0, &mut bump)],
Some(&mint_pubkey),
),
)
.unwrap();
let data = bank_client.get_account_data(&pubkey).unwrap().unwrap();
assert_eq!(0, data.len());
// zero-init
bank_client
.send_and_confirm_message(
&[&mint_keypair, &keypair],
Message::new(
&[Instruction::new_with_bytes(
program_id,
&[ZERO_INIT],
vec![AccountMeta::new(pubkey, true)],
)],
Some(&mint_pubkey),
),
)
.unwrap();
// zero-init
bank_client
.send_and_confirm_message(
&[&mint_keypair, &keypair],
Message::new(
&[Instruction::new_with_bytes(
program_id,
&[ZERO_INIT],
vec![AccountMeta::new(pubkey, true)],
)],
Some(&mint_pubkey),
),
)
.unwrap();
}
}
#[test]

1
rbpf-cli/src/main.rs
View File

@ -235,6 +235,7 @@ before execting it in the virtual machine.",
.get_current_instruction_context()
.unwrap(),
true, // should_cap_ix_accounts
true,
)
.unwrap();

6
runtime/src/bank/tests.rs
View File

@ -7445,7 +7445,9 @@ fn test_bank_executor_cache() {
upgrade_authority_address: None,
})
.unwrap();
programdata_account.data_mut()[programdata_data_offset..].copy_from_slice(&elf);
let mut data = programdata_account.data().to_vec();
data[programdata_data_offset..].copy_from_slice(&elf);
programdata_account.set_data_from_slice(&data);
programdata_account.set_rent_epoch(1);
bank.store_account_and_update_capitalization(&key1, &program_account);
bank.store_account_and_update_capitalization(&programdata_key, &programdata_account);
@ -7494,7 +7496,7 @@ fn test_bank_load_program() {
upgrade_authority_address: None,
})
.unwrap();
programdata_account.data_mut()[programdata_data_offset..].copy_from_slice(&elf);
programdata_account.data_as_mut_slice()[programdata_data_offset..].copy_from_slice(&elf);
programdata_account.set_rent_epoch(1);
bank.store_account_and_update_capitalization(&key1, &program_account);
bank.store_account_and_update_capitalization(&programdata_key, &programdata_account);

39
sdk/src/account.rs
View File

@ -13,7 +13,9 @@ use {
solana_program::{account_info::AccountInfo, debug_account_data::*, sysvar::Sysvar},
std::{
cell::{Ref, RefCell},
fmt, ptr,
fmt,
mem::MaybeUninit,
ptr,
rc::Rc,
sync::Arc,
},
@ -174,7 +176,6 @@ pub trait WritableAccount: ReadableAccount {
fn saturating_sub_lamports(&mut self, lamports: u64) {
self.set_lamports(self.lamports().saturating_sub(lamports))
}
fn data_mut(&mut self) -> &mut Vec<u8>;
fn data_as_mut_slice(&mut self) -> &mut [u8];
fn set_owner(&mut self, owner: Pubkey);
fn copy_into_owner_from_slice(&mut self, source: &[u8]);
@ -228,9 +229,6 @@ impl WritableAccount for Account {
fn set_lamports(&mut self, lamports: u64) {
self.lamports = lamports;
}
fn data_mut(&mut self) -> &mut Vec<u8> {
&mut self.data
}
fn data_as_mut_slice(&mut self) -> &mut [u8] {
&mut self.data
}
@ -267,9 +265,6 @@ impl WritableAccount for AccountSharedData {
fn set_lamports(&mut self, lamports: u64) {
self.lamports = lamports;
}
fn data_mut(&mut self) -> &mut Vec<u8> {
Arc::make_mut(&mut self.data)
}
fn data_as_mut_slice(&mut self) -> &mut [u8] {
&mut self.data_mut()[..]
}
@ -540,6 +535,30 @@ impl Account {
}
impl AccountSharedData {
pub fn is_shared(&self) -> bool {
Arc::strong_count(&self.data) > 1
}
pub fn reserve(&mut self, additional: usize) {
self.data_mut().reserve(additional)
}
pub fn capacity(&self) -> usize {
self.data.capacity()
}
fn data_mut(&mut self) -> &mut Vec<u8> {
Arc::make_mut(&mut self.data)
}
pub fn resize(&mut self, new_len: usize, value: u8) {
self.data_mut().resize(new_len, value)
}
pub fn extend_from_slice(&mut self, data: &[u8]) {
self.data_mut().extend_from_slice(data)
}
pub fn set_data_from_slice(&mut self, new_data: &[u8]) {
let data = match Arc::get_mut(&mut self.data) {
// The buffer isn't shared, so we're going to memcpy in place.
@ -584,6 +603,10 @@ impl AccountSharedData {
self.data = Arc::new(data);
}
pub fn spare_data_capacity_mut(&mut self) -> &mut [MaybeUninit<u8>] {
self.data_mut().spare_capacity_mut()
}
pub fn new(lamports: u64, space: usize, owner: &Pubkey) -> Self {
shared_new(lamports, space, owner)
}

4
sdk/src/feature_set.rs
View File

@ -617,6 +617,9 @@ pub mod delay_visibility_of_program_deployment {
pub mod apply_cost_tracker_during_replay {
solana_sdk::declare_id!("2ry7ygxiYURULZCrypHhveanvP5tzZ4toRwVp89oCNSj");
}
pub mod bpf_account_data_direct_mapping {
solana_sdk::declare_id!("9gwzizfABsKUereT6phZZxbTzuAnovkgwpVVpdcSxv9h");
}
pub mod add_set_tx_loaded_accounts_data_size_instruction {
solana_sdk::declare_id!("G6vbf1UBok8MWb8m25ex86aoQHeKTzDKzuZADHkShqm6");
@ -822,6 +825,7 @@ lazy_static! {
(clean_up_delegation_errors::id(), "Return InsufficientDelegation instead of InsufficientFunds or InsufficientStake where applicable #31206"),
(vote_state_add_vote_latency::id(), "replace Lockout with LandedVote (including vote latency) in vote state #31264"),
(checked_arithmetic_in_fee_validation::id(), "checked arithmetic in fee validation #31273"),
(bpf_account_data_direct_mapping::id(), "use memory regions to map account data into the rbpf vm instead of copying the data"),
/*************** ADD NEW FEATURES HERE ***************/
]
.iter()

252
sdk/src/transaction_context.rs
View File

@ -4,12 +4,16 @@
#[cfg(all(not(target_os = "solana"), debug_assertions))]
use crate::signature::Signature;
#[cfg(not(target_os = "solana"))]
use crate::{
account::WritableAccount,
rent::Rent,
system_instruction::{
MAX_PERMITTED_ACCOUNTS_DATA_ALLOCATIONS_PER_TRANSACTION, MAX_PERMITTED_DATA_LENGTH,
use {
crate::{
account::WritableAccount,
rent::Rent,
system_instruction::{
MAX_PERMITTED_ACCOUNTS_DATA_ALLOCATIONS_PER_TRANSACTION, MAX_PERMITTED_DATA_LENGTH,
},
},
solana_program::entrypoint::MAX_PERMITTED_DATA_INCREASE,
std::mem::MaybeUninit,
};
use {
crate::{
@ -18,9 +22,10 @@ use {
pubkey::Pubkey,
},
std::{
cell::{RefCell, RefMut},
cell::{Ref, RefCell, RefMut},
collections::HashSet,
pin::Pin,
sync::Arc,
},
};
@ -51,15 +56,93 @@ pub struct InstructionAccount {
/// An account key and the matching account
pub type TransactionAccount = (Pubkey, AccountSharedData);
#[derive(Clone, Debug, PartialEq)]
pub struct TransactionAccounts {
accounts: Vec<RefCell<AccountSharedData>>,
touched_flags: RefCell<Box<[bool]>>,
is_early_verification_of_account_modifications_enabled: bool,
}
impl TransactionAccounts {
#[cfg(not(target_os = "solana"))]
fn new(
accounts: Vec<RefCell<AccountSharedData>>,
is_early_verification_of_account_modifications_enabled: bool,
) -> TransactionAccounts {
TransactionAccounts {
touched_flags: RefCell::new(vec![false; accounts.len()].into_boxed_slice()),
accounts,
is_early_verification_of_account_modifications_enabled,
}
}
fn len(&self) -> usize {
self.accounts.len()
}
pub fn get(&self, index: IndexOfAccount) -> Option<&RefCell<AccountSharedData>> {
self.accounts.get(index as usize)
}
#[cfg(not(target_os = "solana"))]
pub fn touch(&self, index: IndexOfAccount) -> Result<(), InstructionError> {
if self.is_early_verification_of_account_modifications_enabled {
*self
.touched_flags
.borrow_mut()
.get_mut(index as usize)
.ok_or(InstructionError::NotEnoughAccountKeys)? = true;
}
Ok(())
}
#[cfg(not(target_os = "solana"))]
pub fn touched_count(&self) -> usize {
self.touched_flags
.borrow()
.iter()
.fold(0usize, |accumulator, was_touched| {
accumulator.saturating_add(*was_touched as usize)
})
}
pub fn try_borrow(
&self,
index: IndexOfAccount,
) -> Result<Ref<'_, AccountSharedData>, InstructionError> {
self.accounts
.get(index as usize)
.ok_or(InstructionError::MissingAccount)?
.try_borrow()
.map_err(|_| InstructionError::AccountBorrowFailed)
}
pub fn try_borrow_mut(
&self,
index: IndexOfAccount,
) -> Result<RefMut<'_, AccountSharedData>, InstructionError> {
self.accounts
.get(index as usize)
.ok_or(InstructionError::MissingAccount)?
.try_borrow_mut()
.map_err(|_| InstructionError::AccountBorrowFailed)
}
pub fn into_accounts(self) -> Vec<AccountSharedData> {
self.accounts
.into_iter()
.map(|account| account.into_inner())
.collect()
}
}
/// Loaded transaction shared between runtime and programs.
///
/// This context is valid for the entire duration of a transaction being processed.
#[derive(Debug, Clone, PartialEq)]
pub struct TransactionContext {
account_keys: Pin<Box<[Pubkey]>>,
accounts: Pin<Box<[RefCell<AccountSharedData>]>>,
#[cfg(not(target_os = "solana"))]
account_touched_flags: RefCell<Pin<Box<[bool]>>>,
accounts: Arc<TransactionAccounts>,
instruction_stack_capacity: usize,
instruction_trace_capacity: usize,
instruction_stack: Vec<usize>,
@ -84,16 +167,13 @@ impl TransactionContext {
instruction_stack_capacity: usize,
instruction_trace_capacity: usize,
) -> Self {
let (account_keys, accounts): (Vec<Pubkey>, Vec<RefCell<AccountSharedData>>) =
transaction_accounts
.into_iter()
.map(|(key, account)| (key, RefCell::new(account)))
.unzip();
let account_touched_flags = vec![false; accounts.len()];
let (account_keys, accounts): (Vec<_>, Vec<_>) = transaction_accounts
.into_iter()
.map(|(key, account)| (key, RefCell::new(account)))
.unzip();
Self {
account_keys: Pin::new(account_keys.into_boxed_slice()),
accounts: Pin::new(accounts.into_boxed_slice()),
account_touched_flags: RefCell::new(Pin::new(account_touched_flags.into_boxed_slice())),
accounts: Arc::new(TransactionAccounts::new(accounts, rent.is_some())),
instruction_stack_capacity,
instruction_trace_capacity,
instruction_stack: Vec::with_capacity(instruction_stack_capacity),
@ -113,10 +193,15 @@ impl TransactionContext {
if !self.instruction_stack.is_empty() {
return Err(InstructionError::CallDepth);
}
Ok(Vec::from(Pin::into_inner(self.accounts))
.into_iter()
.map(|account| account.into_inner())
.collect())
Ok(Arc::try_unwrap(self.accounts)
.expect("transaction_context.accounts has unexpected outstanding refs")
.into_accounts())
}
#[cfg(not(target_os = "solana"))]
pub fn accounts(&self) -> &Arc<TransactionAccounts> {
&self.accounts
}
/// Returns true if `enable_early_verification_of_account_modifications` is active
@ -159,7 +244,7 @@ impl TransactionContext {
index_in_transaction: IndexOfAccount,
) -> Result<&RefCell<AccountSharedData>, InstructionError> {
self.accounts
.get(index_in_transaction as usize)
.get(index_in_transaction)
.ok_or(InstructionError::NotEnoughAccountKeys)
}
@ -553,7 +638,7 @@ impl InstructionContext {
) -> Result<BorrowedAccount<'a>, InstructionError> {
let account = transaction_context
.accounts
.get(index_in_transaction as usize)
.get(index_in_transaction)
.ok_or(InstructionError::MissingAccount)?
.try_borrow_mut()
.map_err(|_| InstructionError::AccountBorrowFailed)?;
@ -663,6 +748,11 @@ pub struct BorrowedAccount<'a> {
}
impl<'a> BorrowedAccount<'a> {
/// Returns the transaction context
pub fn transaction_context(&self) -> &TransactionContext {
self.transaction_context
}
/// Returns the index of this account (transaction wide)
#[inline]
pub fn get_index_in_transaction(&self) -> IndexOfAccount {
@ -782,23 +872,35 @@ impl<'a> BorrowedAccount<'a> {
pub fn get_data_mut(&mut self) -> Result<&mut [u8], InstructionError> {
self.can_data_be_changed()?;
self.touch()?;
self.make_data_mut();
Ok(self.account.data_as_mut_slice())
}
/// Returns the spare capacity of the vector backing the account data.
///
/// This method should only ever be used during CPI, where after a shrinking
/// realloc we want to zero the spare capacity.
#[cfg(not(target_os = "solana"))]
pub fn spare_data_capacity_mut(&mut self) -> Result<&mut [MaybeUninit<u8>], InstructionError> {
debug_assert!(!self.account.is_shared());
Ok(self.account.spare_data_capacity_mut())
}
/// Overwrites the account data and size (transaction wide).
///
/// Call this when you have an owned buffer and want to replace the account
/// data with it.
/// You should always prefer set_data_from_slice(). Calling this method is
/// currently safe but requires some special casing during CPI when direct
/// account mapping is enabled.
///
/// If you have a slice, use [`Self::set_data_from_slice()`].
/// Currently only used by tests and the program-test crate.
#[cfg(not(target_os = "solana"))]
pub fn set_data(&mut self, data: Vec<u8>) -> Result<(), InstructionError> {
self.can_data_be_resized(data.len())?;
self.can_data_be_changed()?;
self.touch()?;
self.update_accounts_resize_delta(data.len())?;
self.account.set_data(data);
Ok(())
}
@ -806,14 +908,19 @@ impl<'a> BorrowedAccount<'a> {
///
/// Call this when you have a slice of data you do not own and want to
/// replace the account data with it.
///
/// If you have an owned buffer (eg [`Vec<u8>`]), use [`Self::set_data()`].
#[cfg(not(target_os = "solana"))]
pub fn set_data_from_slice(&mut self, data: &[u8]) -> Result<(), InstructionError> {
self.can_data_be_resized(data.len())?;
self.can_data_be_changed()?;
self.touch()?;
self.update_accounts_resize_delta(data.len())?;
// Calling make_data_mut() here guarantees that set_data_from_slice()
// copies in places, extending the account capacity if necessary but
// never reducing it. This is required as the account migh be directly
// mapped into a MemoryRegion, and therefore reducing capacity would
// leave a hole in the vm address space. After CPI or upon program
// termination, the runtime will zero the extra capacity.
self.make_data_mut();
self.account.set_data_from_slice(data);
Ok(())
@ -832,7 +939,7 @@ impl<'a> BorrowedAccount<'a> {
}
self.touch()?;
self.update_accounts_resize_delta(new_length)?;
self.account.data_mut().resize(new_length, 0);
self.account.resize(new_length, 0);
Ok(())
}
@ -849,10 +956,58 @@ impl<'a> BorrowedAccount<'a> {
self.touch()?;
self.update_accounts_resize_delta(new_len)?;
self.account.data_mut().extend_from_slice(data);
// Even if extend_from_slice never reduces capacity, still realloc using
// make_data_mut() if necessary so that we grow the account of the full
// max realloc length in one go, avoiding smaller reallocations.
self.make_data_mut();
self.account.extend_from_slice(data);
Ok(())
}
/// Reserves capacity for at least additional more elements to be inserted
/// in the given account. Does nothing if capacity is already sufficient.
#[cfg(not(target_os = "solana"))]
pub fn reserve(&mut self, additional: usize) -> Result<(), InstructionError> {
self.can_data_be_changed()?;
self.make_data_mut();
self.account.reserve(additional);
Ok(())
}
/// Returns the number of bytes the account can hold without reallocating.
#[cfg(not(target_os = "solana"))]
pub fn capacity(&self) -> usize {
self.account.capacity()
}
/// Returns whether the underlying AccountSharedData is shared.
///
/// The data is shared if the account has been loaded from the accounts database and has never
/// been written to. Writing to an account unshares it.
///
/// During account serialization, if an account is shared it'll get mapped as CoW, else it'll
/// get mapped directly as writable.
#[cfg(not(target_os = "solana"))]
pub fn is_shared(&self) -> bool {
self.account.is_shared()
}
#[cfg(not(target_os = "solana"))]
fn make_data_mut(&mut self) {
// if the account is still shared, it means this is the first time we're
// about to write into it. Make the account mutable by copying it in a
// buffer with MAX_PERMITTED_DATA_INCREASE capacity so that if the
// transaction reallocs, we don't have to copy the whole account data a
// second time to fullfill the realloc.
//
// NOTE: The account memory region CoW code in Serializer::push_account_region() implements
// the same logic and must be kept in sync.
if self.account.is_shared() {
self.account.reserve(MAX_PERMITTED_DATA_INCREASE);
}
}
/// Deserializes the account data into a state
#[cfg(not(target_os = "solana"))]
pub fn get_state<T: serde::de::DeserializeOwned>(&self) -> Result<T, InstructionError> {
@ -1023,19 +1178,9 @@ impl<'a> BorrowedAccount<'a> {
#[cfg(not(target_os = "solana"))]
fn touch(&self) -> Result<(), InstructionError> {
if self
.transaction_context
.is_early_verification_of_account_modifications_enabled()
{
*self
.transaction_context
.account_touched_flags
.try_borrow_mut()
.map_err(|_| InstructionError::GenericError)?
.get_mut(self.index_in_transaction as usize)
.ok_or(InstructionError::NotEnoughAccountKeys)? = true;
}
Ok(())
self.transaction_context
.accounts()
.touch(self.index_in_transaction)
}
#[cfg(not(target_os = "solana"))]
@ -1064,23 +1209,14 @@ pub struct ExecutionRecord {
#[cfg(not(target_os = "solana"))]
impl From<TransactionContext> for ExecutionRecord {
fn from(context: TransactionContext) -> Self {
let account_touched_flags = context
.account_touched_flags
.try_borrow()
.expect("borrowing transaction_context.account_touched_flags failed");
let touched_account_count = account_touched_flags
.iter()
.fold(0u64, |accumulator, was_touched| {
accumulator.saturating_add(*was_touched as u64)
});
let accounts = Arc::try_unwrap(context.accounts)
.expect("transaction_context.accounts has unexpectd outstanding refs");
let touched_account_count = accounts.touched_count() as u64;
let accounts = accounts.into_accounts();
Self {
accounts: Vec::from(Pin::into_inner(context.account_keys))
.into_iter()
.zip(
Vec::from(Pin::into_inner(context.accounts))
.into_iter()
.map(|account| account.into_inner()),
)
.zip(accounts)
.collect(),
return_data: context.return_data,
touched_account_count,