* Let AHV hold and update last_snapshot_storages
* Clean up comment
* Move cloning after enqueued_time
* Minor positon change
* Remove type last_snapshot_storages annotation
* Makes members of SyscallContext public.
Removes check_aligned and check_size from SyscallContext.
* Replaces InvokeContext::set_syscall_context() in tests with mock_create_vm!().
* Passes SyscallContext directly to InvokeContext::set_syscall_context().
* Merges TraceLogStackFrame into SyscallContext.
* Removes the create_vm!() macro.
* Moves BpfAllocator from bpf_loader into program_runtime.
* Frees BpfAllocator from Rc<RefCell<>>.
* Removes unused code from BpfAllocator.
* Consume CUs for heap before doing the allocation.
* Exposes syscalls in rbpf-cli.
* Adds debugging_features parameter to load_program_from_bytes() and load_program_from_account().
* Removes test_bpf_loader_non_terminating_program() as that is already tested in RBPF.
* Moves stack and heap allocation back onto the program runtime stack.
* bank_from_latest_snapshot_dir
* Clear the account paths before constructing a bank from a snapshot dir
* Clear the account paths before constructing a bank from a snapshot dir (#31170)
* use the highest POST snapshot dir to construct a bank
* Use POST to fix test_bank_from_latest_snapshot_dir
* Revert "Clear the account paths before constructing a bank from a snapshot dir"
This reverts commit 272bb9fee14a09d7fde3cddb3d439f2cdadb5493.
* Remove the set_accounts_hash_for_tests call
* Revert "Revert "Clear the account paths before constructing a bank from a snapshot dir""
This reverts commit 2e345d8b98b8fb3b07f00ce050132c678acf88d4.
* Add remove_incomplete_bank_snapshot_dir
* Remove incomplete snapshot dir if incomplete
* Revert "Add remove_incomplete_bank_snapshot_dir"
This reverts commit aaafef74803ca4e2050b8cd83ef352c5a59509b4.
* Replace metadata's is_file with simple PathBuf's is_file
* In test verify the deletion of the incomplete snapshot dir
* Add comments to explain the snapshot dir cleanup
* Uses declare_process_instruction!() in all tests.
* Adds post_adjustments to mock_process_instruction().
Removes "solana_sbf_rust_external_spend" from assert_instruction_count() as it panics.
* Moves stable_log::program_invoke(), stable_log::program_success() and stable_log::program_failure() calls from bpf_loader into InvokeContext::process_executable_chain().
* Turns result of ProcessInstructionWithContext from InstructionError into Box<dyn std::error::Error>.
* Bump to solana_rbpf v0.3.0
* Removes Result from return type of EbpfVm::new().
* Turns EbpfError into Box<dyn std::error::Error>.
* Removes BpfError.
* Removes SyscallError::InstructionError.
* Adds a type alias for Box<dyn std::error::Error> in syscalls.
* relax Sized bound for Signers in Transaction and Client
* also relax Sized bounds in client, runtime, thin-client and tpu-client
* add tests for using non-sized transaction signers
* fix macro for thin_client vs ?Sized
* move tests to transactions, add Sized relaxation to mut macro
* fix clippy warning
* get rid of unnecessary imports
* Replaces assign_program() by replenish() in load_and_get_programs_from_cache().
* Replaces LoadedProgramEntry by a boolean.
* Only lock the global cache once all missing programs have been loaded
* Factors out call of self.rc.accounts.filter_executable_program_accounts().
* add feature gate
* builtins consume statically defined units at beginning of process_instruction()
* Add new instructionError; return error if builtin did not consume units to enforce builtin to consume units;
* updated related tests
* updated ProgramTest with deactivated native_programs_consume_cu feature to continue support existing mock/test programs that do not consume units
* Construct a bank from a snapshot directory
* Clean up. Remove archiving releated data structures
* Fix auto checks
* remove ArchiveFormat::None
* fix merge error
* remove incremental snapshot dir
* Minor cleanup, remove unused functiond defs
* remove pub fn bank_from_latest_snapshot_dir
* rename bank_from_snapshot_dir to bank_from_snapshot
* Clean up invalid comments
* A few minor review changes
* Removed insert_slot_storage_file
* Add comment explain hardlink symlink
* Skip the whole verify_snapshot_bank call for the from_dir case
* Add bank.set_initial_accounts_hash_verification_completed()
* address review issues: appendvec to append_vec, replace unwrap with expect, etc
* AtomicAppendVecId, remove arc on bank etc
* slice, CI error on &snapshot_version_path
* measure_build_storage
* move snapshot_from
* remove measure_name from build_storage_from_snapshot_dir
* remove from_dir specific next_append_vec_id logic
* revert insert_slot_storage_file change
* init next_append_vec_id to fix the substraction underflow
* remove measure from build_storage_from_snapshot_dir
* make measure name more specific
* refactor status_cache deserialization into a function
* remove reference to pass the ci check
* track next appendvec id
* verify that the next_append_vec_id tracking is correct
* clean up usize
* in build_storage_from_snapshot_dir remove expect
* test max appendvecc id tracking with multiple banks in the test
* cleared expect and unwrap in streaming_snapshot_dir_files
* rebase cleanup
* change to measure!
* dereference arc in the right way
* include loaded accounts data size limit in transaction base fee calculation
* citing compute_budget for heap cost;
* update sbf tests
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
#### Problem
As we start supporting new storage formats, alignment-related constants and macros
defined in append_vec.rs aren't only specific to AppendVec.
#### Summary of Changes
Move alignment-related constants/macros from append_vec.rs to accounts_file.rs
* Clean up orphaned account snapshot hardlink dirs
* fix compilation issues
* debugged, now working. seeing the orphaned directories deleted
* change back to eprintln + exit for account_path error
* changed eprintln to panic for now
* add test_clean_orphaned_account_snapshot_dirs for codecov check
* address a few comments and nit isseus
* directly unzip, skipped the intermediate array of tuples
* let set_up_account_run_and_snapshot_paths return Result
* 'proper' typo, and comment on return
* use map_err
* use for loop in clean_orphaned_account_snapshot_dirs, removed panic
* add test_set_up_account_run_and_snapshot_paths
* minor, replace .for_each with .all
* rename set_up_account_run_and_snapshot_paths to create_all_accounts_run_and_snapshot_dirs
* remove unnecessary closure return type
* change to for loop
* change match to unwrap_or_else
* remove create_dir_all(&account_path) in create_all
* minor comment cleanup
#### Problem
Accounts db currently use AppendVec::new_from_file() directly
to create a new AcountsFile instance from an existing file.
However, this method should be abstracted out to AccountsFile
so that an existing file can be opened correctly using the right format.
#### Summary of Changes
Add new_from_file() API to AccountsFile which will open an existing
file based on its accounts file format.
Currently, it only supports AppendVec.
* add default_cost as mandatory field for Builtin
* updated tests
* set zkp program default to VerifyTransfer CUs
---------
Co-authored-by: Jon Cinque <joncinque@pm.me>
#### Problem
The existing StoredAccountMeta requires the file and in-memory layout of
account metadata to match StoredMeta and AccountMeta. This limits the
flexibility to implement different accounts storage file formats.
#### Summary of Changes
This PR abstracts out StoredAccountMeta as an Enum.
1. replace hardcoded loaded accounts data size limit with compute budget instruction;
2. new transaction error for invalid account data size limit
3. test requested limit with combination of features and transactions
As `Range` is a special case of an `Iterator`, this is a generalization
that makes `get_all_accounts()` accept other kinds of ranges. Such as
`RangeInclusive`.
#### Problem
The existing append_accounts returns a vector of offsets, which accounts-db
later uses it to infer the storage size of each account. However, such inference
does not apply to the new hot/cold storage format.
#### Summary of Changes
This PR changes the return value of append_accounts() to `Vec<StoredAccountInfo>`,
which includes both offset and storage size information of an account.
* support loading snapshots with > 1 append vec per slot
* pr feedback
* drain to into_values
* unwrap() -> expect()
* loop to find non-conflicting file and add test
All the references returned from a `StoredAccountMeta` instance
actually come from the underlying account. As written, the lifetimes of
the returned references are constrained by the lifetime of the
`StoredAccountMeta` instance.
While technically correct, it is overly strict. It is totally fine to
use the returned reference as long as the underlying account is alive.
Which could be longer than the lifetime of the `StoredAccountMeta`
wrapper.
Xiang Zhu
Jeff Washington (jwash)
Brennan
Andrew Fitzgerald
Alexander Meißner
Brooks
HaoranYi
Tao Zhu
Pankaj Garg
steviez
blackghost1987
Yueh-Hsuan Chiang
Ryo Onodera
Madhav Goyal
Alessandro Decina
Illia Bobyr