#### Summary of Changes
This PR includes the implementation of the footer for the tiered account storage.
Tiered account storage proposal: #30551
The prototype implementation of the tiered account storage: #30626.
Counters can only report a single field; this adds an extra step to
debugging efforts to figure out the context for where/why the counted
event occurred. Use datapoints instead to subtmit some extra context.
* Replaces BuiltinProgram in the program-runtime with the one from solana_rbpf.
* Adjusts the runtimes built-ins to use Arc<LoadedProgram>.
* Adjusts the tests and benchmarks.
* Remove snapshot_links
* Change the function name from snapshot_dir to bank_snapshot_dir
* Format fix
* Fix test_concurrent_snapshot_packaging
* Fix clippy error
* Fix nits
* Fix nits 2nd try
* Use get_bank_snapshots_dir
* Use slot_dir
* Revert "Use get_bank_snapshots_dir" because get_bank_snapshots_dir is private to crate
This reverts commit 1ed9b3b2c8e84689a918beee7159f63c56500a96.
Within the contents of snapshot_utils::add_bank_snapshot(), metrics were
being reported in several datapoints and counters. This adds extra
overhead, and makes it harder to correlate fields that correspond to the
same snapshot.
So, combine the counters and multiple datapoints into a single
datapoint.
* Make load_program method of Bank public for use in ledger-tool
* Add an accessor method to get a reference to builtin_programs of Bank
* Add a parameter to Bank::load_program to control debugging_features
* AccountSharedData: make data_mut() private
This ensures that the inner Vec is never handed out. This is in
preparation of enforcing that the capacity of the inner vec never
shrinks, which is required for direct mapping.
* Adds the feature bpf_account_data_direct_mapping.
* Remaps EbpfError::AccessViolation into InstructionError::ReadonlyDataModified.
* WIP: Memory regions for each instruction account in create_vm().
* Fix serialization benches, run both copy and !copy variants
* rbpf-cli: fix build
* BorrowedAccount: ensure that account capacity is never reduced
Accounts can be directly mapped in address space. Their capacity can't
be reduced mid transaction as that would create holes in vm address
space that point to invalid host memory.
* bpf_load: run serialization tests for both copy and !copy account data
* bpf_loader: add Serializer::write_account
* fix lints
* BorrowedAccount: make_data_mut is host only
* Fix unused import warning
* Fix lints
* cpi: add explicit direct_mapping arg to update_(callee|caller)_account
* cpi: rename account_data_or_only_realloc_padding to serialized_data
* cpi: add CallerAccount::original_data_len comment
* cpi: add update_callee_account direct_mapping test
* cpi: add test_update_caller_account_data_direct_mapping and fix bug
We used to have a bug in zeroing data when shrinking account, where we zeroed
the spare account capacity but not the realloc padding.
* cpi: add tests for mutated readonly accounts
* cpi: update_caller_account doesn't need to change .serialized_data when direct_mapping is on
* cpi: update_caller_account: ensure that account capacity is always enough
Introduce a better way to ensure that account capacity never goes below what
might be mapped in memory regions.
* cpi: zero account capacity using the newly introduced BorrowedAccount::spare_data_capacity_mut()
Before we were using BorrowedAccount::get_data_mut() to get the base pointer to
the account data, then we were slicing the spare capacity from it. Calling
get_data_mut() doesn't work if an account has been closed tho, since the
current program doesn't own the account anymore and therefore get_data_mut()
errors out.
* bpf_loader: fix same lint for the umpteenth time
* bpf_loader: map AccessViolation to ReadonlyDataModified only for account region violations
* programs/sbf: realloc: add test for large write after realloc
Add a test that after a realloc does a large write that spans the
original account length and the realloc area. This ensures that memory
mapping works correctly across the boundary.
* programs/sbf: run test_program_sbf_realloc with both direct_mapping on and off
By default test banks test with all features on. This ensures we keep
testing the existing code until the new feature is enabled.
* bpf_loader: tweak memcmp syscall
Split the actual memcmp code in a separate function. Remove check
indexing the slices since the slices are guaranteed to have the correct
length by construction.
* bpf_loader: tweak the memset syscall
Use slice::fill, which is effectively memset.
* bpf_loader: syscalls: update mem syscalls to work with non contiguous memory
With direct mapping enabled, accounts can now span multiple memory
regions.
* fix lint, rebase mem_ops
* Implement CoW for writable accounts
* Fix CI
* Move CoW to the MemoryMapping level
* Update after rbpf API change
* Fix merge screwup
* Add create_vm macro. Fix benches.
* cpi: simplify update_caller_account
Simplify the logic to update a caller's memory region when a callee
causes an account data pointer to change (eg during CoW)
* benches/bpf_loader: move serialization out of create_vm bench
* benches/bpf_loader: don't copy accounts when direct mapping is on
* Fix review nits
* bpf_loader: mem_ops: handle u64 overflow in MemoryChunkIterator::new
When starting at u64::MAX, the chunk iterator would always return the
empty sequence (None on the first next()) call, instead of returning a
memory access violation.
Use checked instead of saturating arithmetic to detect the condition and
error out.
This commit also adds more tests around boundary conditions.
* Fix loader-v3 tests: data_mut => data_as_mut_slice
* Fix CI
* bpf_loader: fix tuner bench: account must be writable
With direct mapping on, invalid writes are caught early meaning the
tuner would fail on the first store and not consume the whole budget
like the benchmark expects.
---------
Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
* Add failing test
* Add config struct for vote-account creation
* Add create-vote-account ix builders that use CreateVoteAccountConfig
* Test now passes; add case to demonstrate failure after activation
* Deprecate existing ix builders
* Use new builders in solana-cli
* Query feature status to use correct VoteState size in solana-cli
* Fix tests and clippy warnings
* Improve ugly conditional block
* Enforce used_underscore_binding
* Fix all
* Work around for cfg()-ed code...
* ci....
* Make clipply fixes more pleasant
* Clone exit signal while intentionally shadowing
* Use more verbose code to avoid any #[allow(...)]s
* AHV processes the snapshot dirs in place
Let account pacakge use the snapshot dir, so AHV computes the accounts hash and turns the pre snapshot dir into a post snapshot dir
* fix status cache path to maintain the archive layout for the in-place snapshot dir archiving
* fix test_package_snapshots
* Fix test_concurrent_snapshot_packaging
* Remove debug change.
* Fix snapshot_links path
* change to borrow for bank_snapshots_dir
* Reverted changes in create_and_verify_snapshot
* Fix param errors
* Fix rebase errors
* Remove NOTE 1
* Remove unwrap
* Remove the variables to make it apparent taht snapshot_links is the bank_snapshots_dir
* Use soft link instead of hard link for snapshot and status cache
* After switching to soft symlinking, the src path should be absolute
* Moves the registration of bpf_loader built-ins from the ledger into the runtime.
* Removes bpf_loader built-ins registration in tests and benchmarks.
* Removes declare_builtin!() of bpf_loader built-ins.
* Removes built-ins from the SDK.
* Adjusts tests.
* Enables JIT everywhere by default.
* Removes the CLI argument "no-bpf-jit" and its plumbering through out the validator code base.
* Removes with_jit bpf_loader variants.
* Removes the to_builtin!() macro.
* Moves BuiltinProgram of the program runtime into its own file.
* Unifies the runtimes Builtin and program runtimes BuiltinProgram definitions.
* Moves BuiltinPrograms from bank.rs into the program runtime.
* Removes test_program_entry_debug().
* Uses declare_process_instruction!() in all tests and benchmarks.
* Replaces with ProcessInstructionWithContext with solana_rbpf::BuiltInFunction.
Add new vote state version that replaces Lockout with LandedVote to allow vote latency to be tracked in a future change.
Includes a feature to be enabled which will when enabled cause the vote state to be written in the new form.
* Add a type_select param for purge_old_bank_snapshots
* Use flags to make the function calls more readable
* Remove the extra purge calls
* replace select_type with filter_by_type
* Add test
* Use matches
* Fix CI test on reference
* use match and call do_purge once
* Let bank_snapshots_dir be TempDir
* remove account_paths in the test
* replace bank with _bank
* Remove create_snapshot_dirs_for_tests, will take the lastest from master
* Fix merge errors
* Create fn create_snapshot_dirs_for_tests to refactor the common test snapshot dirs creation code
* Fix trailing space
* Fix missed set_accounts_delta_hash
* Let bank_snapshots_dir be TempDir
* Same change in test_bank_from_latest_snapshot_dir
* Use borrow to avoid releasing the tmp dir
* One more to change to reference
* Let AHV hold and update last_snapshot_storages
* Clean up comment
* Move cloning after enqueued_time
* Minor positon change
* Remove type last_snapshot_storages annotation
* Makes members of SyscallContext public.
Removes check_aligned and check_size from SyscallContext.
* Replaces InvokeContext::set_syscall_context() in tests with mock_create_vm!().
* Passes SyscallContext directly to InvokeContext::set_syscall_context().
* Merges TraceLogStackFrame into SyscallContext.
* Removes the create_vm!() macro.
* Moves BpfAllocator from bpf_loader into program_runtime.
* Frees BpfAllocator from Rc<RefCell<>>.
* Removes unused code from BpfAllocator.
* Consume CUs for heap before doing the allocation.
* Exposes syscalls in rbpf-cli.
* Adds debugging_features parameter to load_program_from_bytes() and load_program_from_account().
* Removes test_bpf_loader_non_terminating_program() as that is already tested in RBPF.
* Moves stack and heap allocation back onto the program runtime stack.
* bank_from_latest_snapshot_dir
* Clear the account paths before constructing a bank from a snapshot dir
* Clear the account paths before constructing a bank from a snapshot dir (#31170)
* use the highest POST snapshot dir to construct a bank
* Use POST to fix test_bank_from_latest_snapshot_dir
* Revert "Clear the account paths before constructing a bank from a snapshot dir"
This reverts commit 272bb9fee14a09d7fde3cddb3d439f2cdadb5493.
* Remove the set_accounts_hash_for_tests call
* Revert "Revert "Clear the account paths before constructing a bank from a snapshot dir""
This reverts commit 2e345d8b98b8fb3b07f00ce050132c678acf88d4.
* Add remove_incomplete_bank_snapshot_dir
* Remove incomplete snapshot dir if incomplete
* Revert "Add remove_incomplete_bank_snapshot_dir"
This reverts commit aaafef74803ca4e2050b8cd83ef352c5a59509b4.
* Replace metadata's is_file with simple PathBuf's is_file
* In test verify the deletion of the incomplete snapshot dir
* Add comments to explain the snapshot dir cleanup
* Uses declare_process_instruction!() in all tests.
* Adds post_adjustments to mock_process_instruction().
Removes "solana_sbf_rust_external_spend" from assert_instruction_count() as it panics.
* Moves stable_log::program_invoke(), stable_log::program_success() and stable_log::program_failure() calls from bpf_loader into InvokeContext::process_executable_chain().
* Turns result of ProcessInstructionWithContext from InstructionError into Box<dyn std::error::Error>.
* Bump to solana_rbpf v0.3.0
* Removes Result from return type of EbpfVm::new().
* Turns EbpfError into Box<dyn std::error::Error>.
* Removes BpfError.
* Removes SyscallError::InstructionError.
* Adds a type alias for Box<dyn std::error::Error> in syscalls.
* relax Sized bound for Signers in Transaction and Client
* also relax Sized bounds in client, runtime, thin-client and tpu-client
* add tests for using non-sized transaction signers
* fix macro for thin_client vs ?Sized
* move tests to transactions, add Sized relaxation to mut macro
* fix clippy warning
* get rid of unnecessary imports
* Replaces assign_program() by replenish() in load_and_get_programs_from_cache().
* Replaces LoadedProgramEntry by a boolean.
* Only lock the global cache once all missing programs have been loaded
* Factors out call of self.rc.accounts.filter_executable_program_accounts().
* add feature gate
* builtins consume statically defined units at beginning of process_instruction()
* Add new instructionError; return error if builtin did not consume units to enforce builtin to consume units;
* updated related tests
* updated ProgramTest with deactivated native_programs_consume_cu feature to continue support existing mock/test programs that do not consume units
* Construct a bank from a snapshot directory
* Clean up. Remove archiving releated data structures
* Fix auto checks
* remove ArchiveFormat::None
* fix merge error
* remove incremental snapshot dir
* Minor cleanup, remove unused functiond defs
* remove pub fn bank_from_latest_snapshot_dir
* rename bank_from_snapshot_dir to bank_from_snapshot
* Clean up invalid comments
* A few minor review changes
* Removed insert_slot_storage_file
* Add comment explain hardlink symlink
* Skip the whole verify_snapshot_bank call for the from_dir case
* Add bank.set_initial_accounts_hash_verification_completed()
* address review issues: appendvec to append_vec, replace unwrap with expect, etc
* AtomicAppendVecId, remove arc on bank etc
* slice, CI error on &snapshot_version_path
* measure_build_storage
* move snapshot_from
* remove measure_name from build_storage_from_snapshot_dir
* remove from_dir specific next_append_vec_id logic
* revert insert_slot_storage_file change
* init next_append_vec_id to fix the substraction underflow
* remove measure from build_storage_from_snapshot_dir
* make measure name more specific
* refactor status_cache deserialization into a function
* remove reference to pass the ci check
* track next appendvec id
* verify that the next_append_vec_id tracking is correct
* clean up usize
* in build_storage_from_snapshot_dir remove expect
* test max appendvecc id tracking with multiple banks in the test
* cleared expect and unwrap in streaming_snapshot_dir_files
* rebase cleanup
* change to measure!
* dereference arc in the right way
* include loaded accounts data size limit in transaction base fee calculation
* citing compute_budget for heap cost;
* update sbf tests
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
#### Problem
As we start supporting new storage formats, alignment-related constants and macros
defined in append_vec.rs aren't only specific to AppendVec.
#### Summary of Changes
Move alignment-related constants/macros from append_vec.rs to accounts_file.rs
* Clean up orphaned account snapshot hardlink dirs
* fix compilation issues
* debugged, now working. seeing the orphaned directories deleted
* change back to eprintln + exit for account_path error
* changed eprintln to panic for now
* add test_clean_orphaned_account_snapshot_dirs for codecov check
* address a few comments and nit isseus
* directly unzip, skipped the intermediate array of tuples
* let set_up_account_run_and_snapshot_paths return Result
* 'proper' typo, and comment on return
* use map_err
* use for loop in clean_orphaned_account_snapshot_dirs, removed panic
* add test_set_up_account_run_and_snapshot_paths
* minor, replace .for_each with .all
* rename set_up_account_run_and_snapshot_paths to create_all_accounts_run_and_snapshot_dirs
* remove unnecessary closure return type
* change to for loop
* change match to unwrap_or_else
* remove create_dir_all(&account_path) in create_all
* minor comment cleanup
#### Problem
Accounts db currently use AppendVec::new_from_file() directly
to create a new AcountsFile instance from an existing file.
However, this method should be abstracted out to AccountsFile
so that an existing file can be opened correctly using the right format.
#### Summary of Changes
Add new_from_file() API to AccountsFile which will open an existing
file based on its accounts file format.
Currently, it only supports AppendVec.
* add default_cost as mandatory field for Builtin
* updated tests
* set zkp program default to VerifyTransfer CUs
---------
Co-authored-by: Jon Cinque <joncinque@pm.me>
#### Problem
The existing StoredAccountMeta requires the file and in-memory layout of
account metadata to match StoredMeta and AccountMeta. This limits the
flexibility to implement different accounts storage file formats.
#### Summary of Changes
This PR abstracts out StoredAccountMeta as an Enum.
1. replace hardcoded loaded accounts data size limit with compute budget instruction;
2. new transaction error for invalid account data size limit
3. test requested limit with combination of features and transactions
As `Range` is a special case of an `Iterator`, this is a generalization
that makes `get_all_accounts()` accept other kinds of ranges. Such as
`RangeInclusive`.
#### Problem
The existing append_accounts returns a vector of offsets, which accounts-db
later uses it to infer the storage size of each account. However, such inference
does not apply to the new hot/cold storage format.
#### Summary of Changes
This PR changes the return value of append_accounts() to `Vec<StoredAccountInfo>`,
which includes both offset and storage size information of an account.
* support loading snapshots with > 1 append vec per slot
* pr feedback
* drain to into_values
* unwrap() -> expect()
* loop to find non-conflicting file and add test
All the references returned from a `StoredAccountMeta` instance
actually come from the underlying account. As written, the lifetimes of
the returned references are constrained by the lifetime of the
`StoredAccountMeta` instance.
While technically correct, it is overly strict. It is totally fine to
use the returned reference as long as the underlying account is alive.
Which could be longer than the lifetime of the `StoredAccountMeta`
wrapper.
This PR makes all the StoredAccountMeta fields pub(crate)
and provides getter functions to access its member fields.
This PR is a preparation step for abstracting out StoredAccountMeta.
* introduce workspace.package
* introduce workspace.dependencies
* read version from root cargo.toml
* pass check when version = { workspace = true }
* don't bump version when version = { workspace = true }
* including workspace Cargo.toml when bump version
* programs/sbf use workspace inheritance
* fix increasing cargo version ignore program/sbf/Cargo.toml
#### Problem
As we start supporting new storage formats, those account-meta
structs will be shared by different storage formats and thus need
a new home.
#### Summary of Changes
This PR creates meta.rs under account_storage and moves all the account-meta
structs out from append_vec.rs.
* Find and load missing programs in LoadedPrograms cache
- filter program accounts in a transaction batch
- filter the accounts that are missing in LoadedPrograms cache
- load the programs before processing the transactions
- unit tests
* address review comments
* fix clippy
* address review comments
* fix test
* fix more tests
* Add version and state_complate flag into bank snapshot
* Add test_get_highest_full_snapshot_slot_and_path
* fix build checks
* fix slot_deltas panic in test_get_highest_full_snapshot_slot_and_path
* make bank rooted for test_get_highest_full_snapshot_slot_and_path
* fix bank scope problem in test_get_highest_full_snapshot_slot_and_path
* minor cleanup
* misc review issues
* remove unneeded error defs and functions
* Fix state_complete macro naming
* fix snapshot_version
* Add accounts hard-link files into the bank snapshot directory
* Small adjustments and fixes.
* Address some of the review issues
* Fix compilation issues
* Change the latest slot snapshot storage from VecDeque to Option
* IoWithSourceAndFile and expanded comments on accounts
* last_slot_snapshot_storages in return value
* Update comments following the review input
* rename dir_accounts_hard_links to hard_link_path
* Add dir_full_state flag for add_bank_snapshot
* Let appendvec files hardlinking work with multiple accounts paths across multiple partitions
* Fixes for rebasing
* fix tests which generates account_path without adding run/
* rebasing fixes
* fix account path test failures
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* update with more review comments
* handle error from create_accounts_run_and_snapshot_dirs
* fix rebasing duplicate
* minor accounts_dir path cleanup
* minor cleanup, remove commented code
* misc review comments
* build error fix
* Fix test_incremental_snapshot_download_with_crossing_full_snapshot_interval_at_startup
* fix build error on MAX_BANK_SNAPSHOTS_TO_RETAIN
* rebase fix, update hardlink filename
* minor comment spelling fix
* rebasing fixes
* fix rebase issues; with_extension
* comments changes for review
* misc minor review issues
* bank.fill_bank_with_ticks_for_tests
* error handling on appendvec path
* fix use_jit
* minor comments refining
* Remove type AccountStorages
* get_account_path_from_appendvec_path return changed to Option
* removed appendvec_path.to_path_buf in create_accounts_run_and_snapshot_dirs
* add test_get_snapshot_accounts_hardlink_dir
* update last_snapshot_storages comment
* update last_snapshot_storages comment
* symlink map_err
* simplify test_get_snapshot_accounts_hardlink_dir with fake paths
* log last_snapshot_storages at the end of the loop
* Use three separate HashMaps instead of the enum TxBankExecutorCacheDiff.
* Replaces all places which deploy programs by a macro.
* Adds a feature gate.
* Adjust tests.
* Makes undeployment visible immediately.
* Add address lookup tables to minimized snapshot
* Add comment for future posterity
* Add reference to the issue
* Adjust comment a bit
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
---------
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
#### Current implementation of ReplicaAccountInfo restricts
possibilities for inflight account filtration
Current implementation includes transaction signature in
ReplicaAccountInfo. This approach does not allow to filter accounts by
matching other accounts in transaction in-flight (e. g. accept only
those accounts included in transactions for specific programs). Current
implementation forces to collect ALL accounts and transactions for some
period of time and perform such complex filtration afterwards.
#### Pass reference to transaction object instead of transaction
signature into ReplicaAccountInfo
Advanced in-flight filtration can be implemented in plugins by passing
reference to transaction for every update_account event. This change
doesn't bring any overhead comparing to current implementation (only
data type of reference is changed) and brings only minor changes in
source code.
#### Problem
build broken due to pr conflicts
#### Summary of Changes
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
#### Problem
fix tests to be more realistic
#### Summary of Changes
allocate correctly sized append vecs
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
#### Problem
test code compares sets of accounts after clean/shrink operations.
The indexing was off and new tests were giving false positives.
The test code was equivalent as long as everything was in the same
order.
#### Summary of Changes
Fix test code comparison indexes.
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
#### Problem
Building new algorithm for packing ancient storage. Packing will occur
in 1 pass across multiple ancient slots.
This will be put in 1 dead code piece at a time with tests until all
pieces are present. Switch between current packing algorithm and this
new one is in a validator cli argument. Resulting append vecs are
correct and compatible (as a set) either way. When a new storage format
optimized for cold storage becomes available, it will only work with
this new packing algorithm, so the change will need to be complete prior
to the new storage format.
#### Summary of Changes
Add `ancient calc_accounts_to_combine` to separate accounts to prepare
for creating packed ancient append vecs. This will be used soon.
Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
adds unit test asserting the type layout of structures in append_vec
---------
Co-authored-by: Richard Patel <ripatel@jumptrading.com>
Co-authored-by: brooks <brooks@solana.com>
* Add run parent directory for accounts files
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* remove explict type def for map result
* handle create_accounts_run_and_snapshot_dirs error with expect
* update with more review comments
* minor fixes from review comments
* simplify account_filename option assignment
* handle error from create_accounts_run_and_snapshot_dirs
* use then instead of then_some for lazy evaluation
* Clean up files in the old account_path before trasitioning to the new run path
* try_exist and accounts_dir removing extra
* sync rmdir, is_dir check
* handle the account_path not deletable case
* Refactors the "!validated_fee_payer" case from an "else" branch to an early "return".
* Moves the early return upward.
* Removes empty entries.
* Adds account_found_and_dep_index.
* cargo fmt.
* Replaces call site of load_executable_accounts().
* Adjusts number of total loaded accounts in test_load_accounts_multiple_loaders().
* Removes test_accounts_account_not_found().
* Removes load_executable_accounts().
* Refactor back to built-in loader ownership chain loop.
Alexander Meißner
Pankaj Garg
Brooks
Jeff Washington (jwash)
behzad nouri
Yueh-Hsuan Chiang
steviez
HaoranYi
Andrew Fitzgerald
Xiang Zhu
Dmitri Makarov
Alessandro Decina
Tyera
Ryo Onodera
Brennan
Tao Zhu
bji
Trent Nelson
blackghost1987
Madhav Goyal
Illia Bobyr
K-anon
Yihau Chen
Michael Vines
ivandzen
dependabot[bot]
Richard Patel
Will Hickey
Richard Patel
joeaba