Commit Graph

264 Commits

Author SHA1 Message Date
Alexander Meißner 7002c568fb
Cleanup - Removes the CLI option "no-bpf-jit" (#31329)
* Enables JIT everywhere by default.

* Removes the CLI argument "no-bpf-jit" and its plumbering through out the validator code base.

* Removes with_jit bpf_loader variants.

* Removes the to_builtin!() macro.
2023-04-25 19:04:11 +02:00
Alexander Meißner e5490b8d09
Refactor - Use builtin function interface from rbpf (#31221)
* Removes test_program_entry_debug().

* Uses declare_process_instruction!() in all tests and benchmarks.

* Replaces with ProcessInstructionWithContext with solana_rbpf::BuiltInFunction.
2023-04-21 18:08:32 +02:00
Brennan f5fe2607b6
Make VerifyAccountsHashConfig private (#31235) 2023-04-18 07:31:23 -07:00
Brennan 93d0d25d77
Move rent debit out of bank (#31204)
* Move rent debit out of bank

* Clean up imports and visibility

* Fix imports

* rename public mod rent_debits
2023-04-14 19:41:10 +00:00
steviez c834d2fc95
Prevent process_blockstore_from_root() from holding Bank for too long (#31179)
blockstore_processor::process_blockstore_from_root() starts with a
BankForks that contains exactly one Bank. The function grabs an Arc of
this initial Bank, and does some logging and initial setup before
processing more slots in load_frozen_forks().

process_blockstore_from_root() holds that Arc until it returns. This
increases the ref count and prevents the initial Bank from getting
cleaned up in a timely manner if load_frozen_forks() prunes that initial
Bank from BankForks.

This change extracts the needed information from the Arc<Bank>, and
drops the Arc so that the Bank can be dropped in a timely manner.
2023-04-13 04:10:26 +00:00
Alexander Meißner 24a87f33a8
Refactor - Cleanup error handling in program runtime (#30693)
* Moves stable_log::program_invoke(), stable_log::program_success() and stable_log::program_failure() calls from bpf_loader into InvokeContext::process_executable_chain().

* Turns result of ProcessInstructionWithContext from InstructionError into Box<dyn std::error::Error>.

* Bump to solana_rbpf v0.3.0

* Removes Result from return type of EbpfVm::new().

* Turns EbpfError into Box<dyn std::error::Error>.

* Removes BpfError.

* Removes SyscallError::InstructionError.

* Adds a type alias for Box<dyn std::error::Error> in syscalls.
2023-04-05 15:50:34 +02:00
Illia Bobyr fe5ae7733b
ledger: confirm_slot_entries(): confirmation_elapsed metric (#30807)
Measure total time spent inside `confirm_slot_entries()`.  Useful metric
in addition to `replay_elapsed` and
`poh_verify_elapsed`/`transaction_verify_elapsed`, as it shows how PoH
and transaction verification interact with the replay process.
2023-03-29 13:11:29 -07:00
Illia Bobyr 564f8c9b17
ledger: Extract `BatchExecutionTiming` (#30806)
Extracted time metrics related to transaction execution into a separate
structure.  This allows me to call `process_entries_with_callback()`
without locking the whole instance of `ConfirmationTiming`, passing just
the `BatchExecutionTiming` part.

I want to add a new metric that starts at the beginning of the
`confirm_slot_entries()` call and ends until the very end.  In order to
use a `scopeguard::defer`, I need to be able to have an excursive
reference to it for the whole body of `confirm_slot_entries()`.

Plus a few minor renamings to clarify which verifications and results
variables actually store.  And corrected a few messages, that
incorrectly stated PoH verification, while they were actually issued
for transaction verification failures.
2023-03-28 15:37:34 -07:00
Pankaj Garg bf488eb384
Update documentation of instruction error and add a unit test (#30915)
* Update documentation of instruction error and add a unit test

* update test name
2023-03-28 11:54:49 -07:00
Andrew Fitzgerald f226a34f48
Only need bank reference for update (#30879) 2023-03-24 09:48:04 -07:00
Tao Zhu 653083806f
only executed transactions are used to update prioritization_fee_cache (#30868)
* only executed transactions are used to update prioritization_fee_cache

* Update ledger/src/blockstore_processor.rs
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-03-23 16:33:02 -05:00
Andrew Fitzgerald b14fcf5a12
rename process_entries_with_callback to process_entries (#30870) 2023-03-23 13:37:57 -07:00
Illia Bobyr 809041b151
poh_verify => run_verification: Rename to be more accurate (#30811)
`poh_verify` actually disables transaction signature, tick count and
built in program argument verifications as well.  It is somewhat
confusing to call it `poh_verify`.
2023-03-22 11:03:30 -07:00
steviez bc933c63ce
Fix SlotMeta connected tracking (#28069)
Fix SlotMeta is_connected tracking

The tracking of connected status was previously based upon an assumption
that would be practically false for all validators. The connected status
of slots played into whether Blockstore would signal ReplayStage that it
had new shreds ready to be replayed. Prior to the change, we would never
signal and ReplayStage would always wait the entire duration of a 100ms
timeout before restarting its' main processing loop.

This commit introduces a change where we mark snapshot slots as
connected. A validator may not have a path all the way back to genesis
itself; however, snapshots are taken at known roots so we extend the
connected status to these slots. Once a node has been bootstrapped once
to have is connected, the logic persists in Blockstore such that all
children on the main fork also get their connected status updated
properly.
2023-03-21 20:17:58 +08:00
Illia Bobyr f4cde7a51c
doc: ledger: Document `ConfirmationTiming` (#30784)
There is some logic related to how timing values are collected that is not immediately obvious. It is better to document it, rather than requiring everyone interested to reverse engineer it from the code.
2023-03-20 13:50:58 -07:00
Brooks 989c80342b
Adds plumbing and tests for verify_snapshot_bank() with Incremental Accounts Hash (#30789) 2023-03-19 21:44:38 -04:00
Illia Bobyr ea7c72f7a8
ledger: confirm_slot_entries(): Refactor: reduce indent (#30785)
No functional changes.  Use early return pattern, to reduce indentation
in the most important part of the confirmation logic.
2023-03-18 19:25:18 -07:00
Ryo Onodera 74970a0b5d
Remove unused ProcessOptions::entry_callback (#30600)
* Confine entry_callback under cfg(test) for clarity

* Fix ci

* Actually remove entry_callback altogether

* fix clippy
2023-03-16 09:33:18 +09:00
Brooks 17b48edd7b
Renames types to be consistent with verify_accounts_hash (#30674) 2023-03-10 20:02:14 -05:00
Brooks 63623e4df2
Renames fns to verify_accounts_hash (#30672) 2023-03-10 18:11:47 -05:00
steviez c617c345d9
chore: Remove clippy exception that is no longer needed (#30289)
We use the Vec that results from .collect(), so the exception to allow a
needless collect is no longer appropriate.
2023-02-14 00:07:41 -06:00
Tao Zhu 51bace2123
add cost tracking for each bank during replay (#30035)
* add cost tracking for each bank during replay
* add feature gate
2023-02-13 13:09:48 -06:00
steviez d3dab24bbe
chore: Use `i` over `ix` variable name when naming worker threads (#30206) 2023-02-09 01:24:57 +00:00
Tao Zhu fdaba113ea
Refactor: remove unused variable (#29955)
remove unused variable
2023-01-27 20:43:13 +00:00
carllin 1753a0e3af
Revert "Do not exclude failed simple vote transactions from consensus" (#29745)
This reverts commit b4237f3f2c.
2023-01-18 15:13:13 -05:00
Tao Zhu 3d63f93bb0
remove dated cost checking feature (#29598) 2023-01-11 14:05:33 -06:00
Jeff Washington (jwash) fec8f61566
remove ProcessOptions::accounts_db_caching_enabled (#29217) 2022-12-12 20:25:00 -08:00
Jeff Washington (jwash) 2c2324f4ea
remove caching_enabled from Bank::new_with_paths_for_tests (#29214) 2022-12-12 15:30:46 -08:00
Jeff Washington (jwash) dd4393e46d
remove accounts_caching from Bank::new_with_paths (#29213) 2022-12-12 15:18:12 -08:00
behzad nouri 9524c9dbff patches errors from clippy::uninlined_format_args
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
Maximilian Schneider c8b0c3ede9
Update cost model to use requested_cu instead of estimated cu #27608 (#28281)
* Update cost model to use requested_cu instead of estimated cu #27608

* remove CostUpdate and CostModel from replay/tvu

* revive cost update service to send cost tracker stats

* CostModel is now static

* remove unused package

Co-authored-by: Tao Zhu <tao@solana.com>
2022-11-22 11:55:56 -06:00
Brooks Prumo 2bafb0cb12
Requires EAH state cannot be Invalid (#28817) 2022-11-17 11:01:01 -05:00
steviez b8acb1b350
Fix blockstore_processor::load_frozen_forks() halt_at_slot behavior II (#28367)
PR #28317 previously attempted to fix a case where blockstore processing
would create children banks for slots past the halt_at_slot.

However, the previous fix didn't handle the case where a slot could be
strictly less than the halt_at_slot, but have children that were greater
than the halt_at_slot. For example, this could happen if a child of slot
S is S+n where n > 1.

Thus, this change covers our processing logic to cover this second case
as well.
2022-10-12 17:24:27 -05:00
steviez 2929c8f7a2
Fix blockstore_processor::load_frozen_forks() halt_at_slot behavior (#28317)
load_frozen_forks() finds new slots to process by creating new Banks for
the children of the current slot in process_next_slots(). Prior to this
change, we would then immediately check if we had reached the
halt_at_slot and correctly halt processing when appropriate. As such, it
would be possible for Banks to be created for slots beyond the
halt_at_slot.

While a potential child slot that is past halt_at_slot wouldn't be
replayed, the Bank being created still alters some universal state in
AccountsDb. So, this change moves the halt_at_slot check before we
create children Banks in process_next_slots().
2022-10-10 14:37:23 -05:00
steviez 9e8f21ec26
Add metric to track number of slots processed in load_frozen_forks() (#28247) 2022-10-06 12:17:49 -05:00
Brooks Prumo a8c6a9e5fc
Bank::freeze() waits for EAH calculation to complete (#28170) 2022-10-05 17:44:35 -04:00
Jeff Washington (jwash) 7ae7a3ff0e
after replay at validator startup don't verify hash calc (#28222) 2022-10-04 14:04:04 -07:00
Jeff Washington (jwash) 990fb68c34
add sub timings in load_frozen_forks (#28223) 2022-10-04 13:46:02 -07:00
Jeff Washington (jwash) cfc124c825
acct idx can no longer use write cache (#28150) 2022-09-30 10:55:27 -07:00
steviez e4affb9fea
Add Blockstore::highest_slot() method (#27981) 2022-09-23 04:53:43 -05:00
Brooks Prumo 1ee595ca9c
remove AccountsDb::initial_blockstore_processing_complete (#27974) 2022-09-22 13:52:04 -04:00
Jeff Washington (jwash) 7ddacc4bb4
remove extra accounts cap call at startup (#27949) 2022-09-21 09:13:00 -07:00
Jeff Washington (jwash) f2d6a7ecea
bank.initial_blockstore_processing_complete to avoid concurrent hash calculations (#27776)
* bank.initial_blockstore_processing_complete to avoid concurrent hash calculations

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Rename TestValidator::set_startup_verification_complete()

* Initialize with `AtomicBool::new(false)` instead of `default()`

* snapshot tests: move where `initial_blockstore_processing_completed()` is called

* fixup bank_forks.rs calling `is_initial_blockstore_processing_complete()`

* only call initial_blockstore_processing_completed() in blockstore_processor

Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@solana.com>
2022-09-19 13:00:21 -07:00
Jeff Washington (jwash) 765c628546
use exit signal for acct idx bg threads (#27483) 2022-09-12 11:51:12 -07:00
Jeff Washington (jwash) f0770c199e
fix ledger tool final hash calc (#27725)
fix ledger tool final hash calc
2022-09-12 11:09:34 -07:00
Tao Zhu 8bb039d08d
collect min prioritization fees when replaying sanitized transactions (#26709)
* Collect blocks' minimum prioritization fees when replaying sanitized transactions

* Limits block min-fee metrics reporting to top 10 writable accounts

* Add service thread to asynchronously update and finalize prioritization fee cache

* Add bench test for prioritization_fee_cache

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-08-31 08:00:55 -05:00
Brennan Watt 5150877fb3
Attempt to skip redundant startup account verification (#26999) 2022-08-25 09:29:57 -07:00
Michael Vines 3f4731b37f Standardize thread names
Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case
2022-08-20 07:49:39 -07:00
steviez 1d66ae0f66
Bail out of execute_batches() early for empty batches slice (#26932)
The caller of execute_batches() that assembles batches may call this
function with empty batches; we know we can bail early in this scenario.
2022-08-08 12:14:49 -05:00
Justin Starry 69598ed4c0
Refactor: Add `RuntimeConfig` field to Bank (#26946)
* Refactor: Simplify arguments for bank constructor methods

* Refactor: Add RuntimeConfig to Bank fields

* Arc wrap runtime_config

* Arc wrap all runtime config usages

* Remove Copy trait derivation from RuntimeConfig

* Remove some arc wrapping
2022-08-05 20:49:00 +01:00