If a slot is marked as optimistically confirmed, it is probable but not
guaranteed that its' ancestors will also be marked as optimistically
confirmed in the Blockstore. Given the importance of examining
optimistically confirmed slots around cluster restarts, manually walk
an AncestorIterator to avoid the chance of a slot improperly being
ignored in cluster restart scenarios.
The optional args allow reuse by ledger-tool repair roots command Also,
hold cleanup lock for duration of Blockstore::scan_and_fix_roots().
This prevents a scenario where scan_and_fix_roots() could identify a
slot as needing to be marked root, that slot getting cleaned by
LedgerCleanupService, and then scan_and_fix_roots() marking the slot as
root on the now purged slot.
* Restrict access to Bank's HardForks
Callers could previously obtain a a lock to read/write HardForks from
any Bank. This would allow any caller to modify, and creates the
opportunity for inconsistent handling of what is considered a valid hard
fork (ie too old).
This PR adds a function to Bank so consistent sanity checks can be
applied; the caller will already have a Bank as that is where they would
have obtained the HardForks from in the first place. Additionally,
change the getter to return a copy of HardForks (simple Vec).
* Allow hard fork at bank slot if bank is not yet frozen
`Arc` is already a reference internally, so it does not seem to be
beneficial to pass a reference to it. Just adds an extra layer of
indirection.
Functions that need to be able to increment `Arc` reference count need
to take `Arc<AtomicBool>`, but those that just want to read the
`AtomicBool` value can accept `&AtomicBool`, making them a bit more
generic.
This change focuses specifically on `Arc<AtomicBool>`. There are other
uses of `&Arc<T>` in the code base that could be converted in a similar
manner. But it would make the change even larger.
The callstack updated in this PR passed an &Arc<...> down only to have
the bottom level clone the reference. Thus, we are giving shared
ownership so the reference is a bit redundant and arguably obscures the
intention to clone further down the callstack.
* Use spl-token ids directly in program-id checks
* Remove id redefinitions
* Deprecate pubkey_from_spl_token and remove usage
* Deprecate spl_token_pubkey and remove usage
* Deprecate native mint helpers and remove usage
* Deprecate spl_token_instruction and remove usage
* Rename variable to disabiguate tx indexes from entry indexes
* Send entry notifications from blockstore_processor
* Escalate log for send failure to WARN
Bump rocksdb from 0.19.0 to 0.21.0
* Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.19.0 to 0.21.0.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.19.0...v0.21.0)
---
updated-dependencies:
- dependency-name: rocksdb
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Update Env::default() to Env::new()
Env::default() was removed and replaced with Env::new(). The change is
in name only, between versions, ffi::rocksdb_create_default_env() is
still called under the hood by both functions.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Steven Czabaniuk <steven@solana.com>
* Notify replay of pruned duplicate confirmed slots
* Ingest replay signal and run ancestor hashes for pruned
* Forward PDC to ancestor hashes and ingest pruned dumps from ancestor hashes service
* Add local-cluster test
* Move entry_notifier_interface
* Add EntryNotifierService
* Use descriptive struct in sender/receiver
* Optionally initialize EntryNotifierService in validator
* Plumb EntryNotfierSender into Tvu, blockstore_processor
* Plumb EntryNotfierSender into Tpu
* Only return one option when constructing EntryNotifierService
The function seemed to be in need of some cleanup. Changes include:
- Remove redundant argument
- Shift related variable definitions to be adjacent
- Shift several variables into inner scope
- Rename several variables
- Insert newlines into log statement that was very long in addition to
moving some of the logged values to debug
* Remove metric counters from blockstore_processor
The removed bank-par_execute_entries-count values are captured in
replay-slot metrics. Additionally, it was at debug level so effectively
dead code.
The removed load_frozen_forks-cluster-confirmed-root counter has been
combined into process_blockstore_from_root datapoint.
* Update doc comment
* Replaces BuiltinProgram in the program-runtime with the one from solana_rbpf.
* Adjusts the runtimes built-ins to use Arc<LoadedProgram>.
* Adjusts the tests and benchmarks.
* Add failing test
* Add config struct for vote-account creation
* Add create-vote-account ix builders that use CreateVoteAccountConfig
* Test now passes; add case to demonstrate failure after activation
* Deprecate existing ix builders
* Use new builders in solana-cli
* Query feature status to use correct VoteState size in solana-cli
* Fix tests and clippy warnings
* Improve ugly conditional block
* Enforce used_underscore_binding
* Fix all
* Work around for cfg()-ed code...
* ci....
* Make clipply fixes more pleasant
* Clone exit signal while intentionally shadowing
* Use more verbose code to avoid any #[allow(...)]s
* Moves the registration of bpf_loader built-ins from the ledger into the runtime.
* Removes bpf_loader built-ins registration in tests and benchmarks.
* Removes declare_builtin!() of bpf_loader built-ins.
* Removes built-ins from the SDK.
* Adjusts tests.
* Enables JIT everywhere by default.
* Removes the CLI argument "no-bpf-jit" and its plumbering through out the validator code base.
* Removes with_jit bpf_loader variants.
* Removes the to_builtin!() macro.
* Moves BuiltinProgram of the program runtime into its own file.
* Unifies the runtimes Builtin and program runtimes BuiltinProgram definitions.
* Moves BuiltinPrograms from bank.rs into the program runtime.
* Removes test_program_entry_debug().
* Uses declare_process_instruction!() in all tests and benchmarks.
* Replaces with ProcessInstructionWithContext with solana_rbpf::BuiltInFunction.
blockstore_processor::process_blockstore_from_root() starts with a
BankForks that contains exactly one Bank. The function grabs an Arc of
this initial Bank, and does some logging and initial setup before
processing more slots in load_frozen_forks().
process_blockstore_from_root() holds that Arc until it returns. This
increases the ref count and prevents the initial Bank from getting
cleaned up in a timely manner if load_frozen_forks() prunes that initial
Bank from BankForks.
This change extracts the needed information from the Arc<Bank>, and
drops the Arc so that the Bank can be dropped in a timely manner.
Additionally, change the function definition to use Slot instead of u64.
Slot is defined as an alias of u64 so these are functionally equivalent,
but using Slot over u64 is more expressive of the intent.
Slots refer to time windows where a block could be produced whereas
height refers to how many blocks are actually in a fork. This function
is operating on a list of slots, so the use of "height" is incorrect and
misleading.
* Moves stable_log::program_invoke(), stable_log::program_success() and stable_log::program_failure() calls from bpf_loader into InvokeContext::process_executable_chain().
* Turns result of ProcessInstructionWithContext from InstructionError into Box<dyn std::error::Error>.
* Bump to solana_rbpf v0.3.0
* Removes Result from return type of EbpfVm::new().
* Turns EbpfError into Box<dyn std::error::Error>.
* Removes BpfError.
* Removes SyscallError::InstructionError.
* Adds a type alias for Box<dyn std::error::Error> in syscalls.
Measure total time spent inside `confirm_slot_entries()`. Useful metric
in addition to `replay_elapsed` and
`poh_verify_elapsed`/`transaction_verify_elapsed`, as it shows how PoH
and transaction verification interact with the replay process.
Extracted time metrics related to transaction execution into a separate
structure. This allows me to call `process_entries_with_callback()`
without locking the whole instance of `ConfirmationTiming`, passing just
the `BatchExecutionTiming` part.
I want to add a new metric that starts at the beginning of the
`confirm_slot_entries()` call and ends until the very end. In order to
use a `scopeguard::defer`, I need to be able to have an excursive
reference to it for the whole body of `confirm_slot_entries()`.
Plus a few minor renamings to clarify which verifications and results
variables actually store. And corrected a few messages, that
incorrectly stated PoH verification, while they were actually issued
for transaction verification failures.
* only executed transactions are used to update prioritization_fee_cache
* Update ledger/src/blockstore_processor.rs
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
In cluster restart scenarios, it is desirable to know the latest
optimistic slot(s), which the subcommand latest-optimistic-slots will
return. However, it is also useful to know whether slots contain only
votes or if they contain votes and user transactions.
This PR adds an extra column of output to show whether an optimistically
confirmed slot is vote only (contains zero non-vote transactions).
Additionally, a flag has been added to enable filtering output to
exclude vote only slots.
`poh_verify` actually disables transaction signature, tick count and
built in program argument verifications as well. It is somewhat
confusing to call it `poh_verify`.
Shreds arriving at tvu/tvu_forward/repair sockets are each processed in
a separate thread, and since each thread has its own deduper, the
duplicates across these sockets are not filtered out.
Using a common deduper across these threads will require an RwLock
wrapper and may introduce lock contention.
The commit instead moves the shred-deduper to shred-sigverify-stage
where all these shreds arrive through the same channel.
steviez
Brooks
Alexander Meißner
Illia Bobyr
Tyera
Ashwin Sekar
dependabot[bot]
Pankaj Garg
Ryo Onodera
Xiang Zhu
Brennan
Andrew Fitzgerald
Tao Zhu
behzad nouri