Commit Graph

3943 Commits

Author SHA1 Message Date
Xiang Zhu 31784b2ecc
Add remove_incomplete_bank_snapshot_dir (#31131)
* Add remove_incomplete_bank_snapshot_dir

* Remove incomplete snapshot dir if incomplete

* Revert "Add remove_incomplete_bank_snapshot_dir"

This reverts commit aaafef74803ca4e2050b8cd83ef352c5a59509b4.

* Replace metadata's is_file with simple PathBuf's is_file

* In test verify the deletion of the incomplete snapshot dir

* Add comments to explain the snapshot dir cleanup
2023-04-11 17:56:26 -07:00
HaoranYi 20d1615ccd
Refactor reward payout code - part 1 (#31155)
type alias
2023-04-11 16:23:52 -05:00
Tao Zhu 5816d33b57
Add test for transaction has too high prioritization fee (#31097)
* Add test for transaction has too high prioritization fee than payer account balance
2023-04-11 16:22:19 -05:00
Pankaj Garg 12e3e8b467
Add provisions to unload a program from the cache (#31036) 2023-04-10 16:42:05 -07:00
HaoranYi e12c250cf5
Extract new_epoch processing out of new_bank (#31127)
* extract new epoch processing out of new_bank

* review feedbacks

* remove epoch_schedule param

* use RewardCalcTracer
2023-04-10 16:10:45 -05:00
Jeff Washington (jwash) d63359a3ff
disk index: batch insert (#31094) 2023-04-10 11:08:54 -07:00
Andrew Fitzgerald 15011eaa5a
separate check_transaction_age from check_age (#30994) 2023-04-07 09:04:03 -07:00
Alexander Meißner 0cd57addcf
Cleanup - mock_process_instruction() (#31088)
* Uses declare_process_instruction!() in all tests.

* Adds post_adjustments to mock_process_instruction().
Removes "solana_sbf_rust_external_spend" from assert_instruction_count() as it panics.
2023-04-07 12:53:19 +02:00
Jeff Washington (jwash) 6dc18fe241
fix cli arg for ancient append vecs (#31066) 2023-04-06 12:03:31 -07:00
HaoranYi 684901879d
Refactor squash timing report (#31041)
* typo

* refactor squash timing report

* address review comments
2023-04-05 08:53:49 -05:00
Alexander Meißner 24a87f33a8
Refactor - Cleanup error handling in program runtime (#30693)
* Moves stable_log::program_invoke(), stable_log::program_success() and stable_log::program_failure() calls from bpf_loader into InvokeContext::process_executable_chain().

* Turns result of ProcessInstructionWithContext from InstructionError into Box<dyn std::error::Error>.

* Bump to solana_rbpf v0.3.0

* Removes Result from return type of EbpfVm::new().

* Turns EbpfError into Box<dyn std::error::Error>.

* Removes BpfError.

* Removes SyscallError::InstructionError.

* Adds a type alias for Box<dyn std::error::Error> in syscalls.
2023-04-05 15:50:34 +02:00
Xiang Zhu fac76715e9
bank_from_snapshot_dir returns BankFromDirTimings (#31051) 2023-04-04 17:10:56 -07:00
Jeff Washington (jwash) aa3e0b9c20
disk index: add stat: disk_index_failed_resizes (#31039) 2023-04-04 09:39:06 -07:00
Jeff Washington (jwash) bc343a431c
disk index: set_anticipated_count to optimally grow disk buckets at startup (#31033)
* disk index: set_anticipated_count to optimally grow disk buckets at startup

* remove atomic
2023-04-03 20:28:50 -07:00
Alexander Meißner a0c7fde90e
Cleanup - mock InvokeContext (#31007)
* Turns with_mock_invoke_context() into a macro.

* Removes prepare_mock_invoke_context().

* Replaces InvokeContext::new_mock() with with_mock_invoke_context().

* Removes InvokeContext::new_mock().

* Removes Cow from InvokeContext::sysvar_cache.

* Removes override parameters from mock_process_instruction().

* cargo fmt
2023-04-03 17:23:24 +02:00
Jeff Washington (jwash) 07f4789257
fix assert (#31012) 2023-04-03 06:40:54 -07:00
Jeff Washington (jwash) 9600643860
reduce contention on startup index generation (#31006) 2023-03-31 12:46:42 -07:00
Pankaj Garg bc44ac73db
Return error early if program is a tombstone (#30940) 2023-03-30 11:16:01 -07:00
steviez cc8e531a5d
Enforce a minimum of 1 on full and incremental snapshot retention (#30968) 2023-03-30 10:16:36 -05:00
blackghost1987 9f9d6f8a89
Relax `Sized` bound for `Signers` in `Transaction` and `Client` (#30568)
* relax Sized bound for Signers in Transaction and Client

* also relax Sized bounds in client, runtime, thin-client and tpu-client

* add tests for using non-sized transaction signers

* fix macro for thin_client vs ?Sized

* move tests to transactions, add Sized relaxation to mut macro

* fix clippy warning

* get rid of unnecessary imports
2023-03-30 13:12:48 +00:00
Jeff Washington (jwash) d542496d10
fix typo in info log (#30964)
fix typo
2023-03-29 13:23:06 -07:00
Alexander Meißner a6ad37f3ad
Cleanup - `load_and_get_programs_from_cache()` (#30959)
* Replaces assign_program() by replenish() in load_and_get_programs_from_cache().

* Replaces LoadedProgramEntry by a boolean.

* Only lock the global cache once all missing programs have been loaded

* Factors out call of self.rc.accounts.filter_executable_program_accounts().
2023-03-29 16:11:14 +02:00
steviez 4bd7de4887
Update comments around snapshot retention policy unit tests (#30955) 2023-03-29 08:55:05 -05:00
Jeff Washington (jwash) 823d55ea9f
disk index find_entry -> find_index_entry (#30941) 2023-03-29 06:54:03 -07:00
Brooks 1f420f2b03
Moves shrink after flush and clean (#30918) 2023-03-28 16:11:56 -04:00
Pankaj Garg 035c974a8c
Remove redundant check during loading upgradeable program for writing (#30561)
* Remove redundant check during loading upgradeable program

* Apply clippy suggestion
2023-03-28 11:50:49 -07:00
Pankaj Garg aebc191c38
Stop loading program accounts if program exists in cache (#30703)
* Stop loading program accounts if program exists in cache

* load accounts for upgradeable programs

* revert loader change to conditionally use program data account

* load instruction accounts

* generate TransactionExecutorCache from loaded programs

* cleanup account_found_and_dep_index variable

* address review comments

* handle tombstones in loader

* unify tombstone constructor

* handle multiple tombstones
2023-03-28 11:49:56 -07:00
Brooks e7887cfb06
RootBankCache must use Acquire-Release semantics when accessing BankForks::root (#30936) 2023-03-28 14:35:02 -04:00
Jeff Washington (jwash) aaac046b6b
flush individual buckets every n ages (#30855)
disk index flush individual buckets every n ages
2023-03-28 09:26:34 -07:00
Brooks d7ae05c3fd
Unifies logging of start/stop for background services (#30916) 2023-03-28 08:32:18 -04:00
Xiang Zhu bf986d64f8
Add more explict error messages for hardlink operations (#30913) 2023-03-27 19:55:10 -07:00
Pankaj Garg 0df28f6d51
Increment usage count for loaded programs and call eviction (#30900) 2023-03-27 06:48:05 -07:00
Andrew Fitzgerald 2a05b25732
Standardize Thread Naming - Prioritization Fee Cache (#30890)
Thread naming in accordance with #27237
2023-03-24 13:48:55 -07:00
Andrew Fitzgerald f226a34f48
Only need bank reference for update (#30879) 2023-03-24 09:48:04 -07:00
Tao Zhu 3e500d9e92
Feature gate builtin consumes static units during processing instruction (#30702)
* add feature gate
* builtins consume statically defined units at beginning of process_instruction()
* Add new instructionError; return error if builtin did not consume units to enforce builtin to consume units;
* updated related tests
* updated ProgramTest with deactivated native_programs_consume_cu feature to continue support existing mock/test programs that do not consume units
2023-03-24 11:31:01 -05:00
Jeff Washington (jwash) 3ebfda5aef
add stat for disk buckets flushed (#30850) 2023-03-22 15:04:17 -07:00
Xiang Zhu d69f60229d
Construct bank from snapshot dir (#30171)
* Construct a bank from a snapshot directory

* Clean up.  Remove archiving releated data structures

* Fix auto checks

* remove ArchiveFormat::None

* fix merge error

* remove incremental snapshot dir

* Minor cleanup, remove unused functiond defs

* remove pub fn bank_from_latest_snapshot_dir

* rename bank_from_snapshot_dir to bank_from_snapshot

* Clean up invalid comments

* A few minor review changes

* Removed insert_slot_storage_file

* Add comment explain hardlink symlink

* Skip the whole verify_snapshot_bank call for the from_dir case

* Add bank.set_initial_accounts_hash_verification_completed()

* address review issues: appendvec to append_vec, replace unwrap with expect, etc

* AtomicAppendVecId, remove arc on bank etc

* slice, CI error on &snapshot_version_path

* measure_build_storage

* move snapshot_from

* remove measure_name from build_storage_from_snapshot_dir

* remove from_dir specific next_append_vec_id logic

* revert insert_slot_storage_file change

* init next_append_vec_id to fix the substraction underflow

* remove measure from build_storage_from_snapshot_dir

* make measure name more specific

* refactor status_cache deserialization into a function

* remove reference to pass the ci check

* track next appendvec id

* verify that the next_append_vec_id tracking is correct

* clean up usize

* in build_storage_from_snapshot_dir remove expect

* test max appendvecc id tracking with multiple banks in the test

* cleared expect and unwrap in streaming_snapshot_dir_files

* rebase cleanup

* change to measure!

* dereference arc in the right way
2023-03-22 13:49:23 -07:00
Jeff Washington (jwash) 9a1d5ea95d
at startup, keep duplicates in in-memory index since they will be cleaned shortly (#30736)
at startup, keep duplicates in in-memory index since they will be cleaned soon
2023-03-22 08:33:39 -07:00
Brooks 35437b8dad
Makes AccountsHashVerifier aware of Incremental Accounts Hash (#30820) 2023-03-22 10:20:16 -04:00
Tao Zhu 21c287a64f
include loaded accounts data size limit in transaction fee calculation (#30659)
* include loaded accounts data size limit in transaction base fee calculation
* citing compute_budget for heap cost;
* update sbf tests

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2023-03-21 18:41:41 -05:00
HaoranYi d646820a6e
Do not log detailed epoch stakes (#30825)
* do not log detailed epoch stakes

* allow dead code

* log stake details at trace level

* no need dead_code

* Update runtime/src/bank.rs

Co-authored-by: Brooks <brooks@prumo.org>

---------

Co-authored-by: Brooks <brooks@prumo.org>
2023-03-21 14:43:06 -05:00
Jeff Washington (jwash) 2216647f7e
better duplicate key stats during index generation (#30829) 2023-03-21 12:02:23 -07:00
Brooks b64d0de771
Makes snapshot_utils aware of Incremental Accounts Hash (#30804) 2023-03-21 16:34:30 +00:00
HaoranYi 6a6d5ee498
typo (#30824) 2023-03-21 11:19:16 -05:00
Brooks 836494fc5f
Cleans *up to* the last full snapshot slot in verify_snapshot_bank() (#30818)
Cleans up to the last full snapshot slot in verify_snapshot_bank()
2023-03-21 15:49:49 +00:00
Brooks e5edcb9c48
Sets IncrementalAccountsHash in serde_snapshot when reconstructing AccountsDb (#30801) 2023-03-21 10:51:04 -04:00
Jeff Washington (jwash) ce0e23fbab
accounts index does not randomly flush dirty entries (#30783) 2023-03-20 12:23:30 -07:00
Yueh-Hsuan Chiang 72c6099ea0
Move alignment related consts from append_vec.rs to accounts_file.rs (#30782)
#### Problem
As we start supporting new storage formats, alignment-related constants and macros
defined in append_vec.rs aren't only specific to AppendVec.

#### Summary of Changes
Move alignment-related constants/macros from append_vec.rs to accounts_file.rs
2023-03-20 11:34:18 -07:00
HaoranYi f42f61804f
typos (#30797) 2023-03-20 10:47:15 -05:00
Jeff Washington (jwash) df2c95119c
add buckets stats for # files and total file size (#30780)
* add buckets stats for # files and total file size

* change metrics to bytes

* get index file size correct
2023-03-20 08:22:31 -07:00