TransactionStatusService needs Primary access in order to write
transaction status into the Blockstore if enable_rpc_transaction_history
is set to True. The current logic attempts to get Primary access for the
service.
However, in the event that this function had been called with a
Blockstore that already had Primary access, this second attempt to get
Primary access would fail. So, only attempt to open with Primary access
when necessary AND when the current access level is not sufficient.
A future change will add more cases to this if-else if-...-else chain.
Using a match statement will be easier to follow then a very long
if-else if-... chain.
This change was broken out in order to have a higher signal to noise
ratio in the subsequent change.
The open_blockstore() helper currently takes multiple configurable
options. While the arguments are parsed at a high enough scope in
main.rs to avoid repeated calls, this parsing is duplicated in the
separate subcommand files (program.rs and bigtable.rs).
The repeated parsing is redundant, and also prone to having to missing
an arg (as was the case with bigtable not having wal_recovery_mode).
So, this PR consolidates the parsing to a single function and uses that
function across the previous callers.
warning: `flatten()` will run forever if the iterator repeatedly produces an `Err`
--> ledger-tool/src/main.rs:2649:39
|
2649 | for line in f.lines().flatten() {
| ^^^^^^^^^ help: replace with: `map_while(Result::ok)`
|
note: this expression returning a `std::io::Lines` may produce an infinite number of `Err` in case of a read error
--> ledger-tool/src/main.rs:2649:29
|
2649 | for line in f.lines().flatten() {
| ^^^^^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#lines_filter_map_ok
= note: `#[warn(clippy::lines_filter_map_ok)]` on by default
warning: `solana-ledger-tool` (bin "solana-ledger-tool" test) generated 1 warning
* Add cli flag to show entry data
* Add display structs
* Add writeln_entry helper fn
* Add entry conversion method
* Populate Display for CliBlockWithEntries
* Add ctor from flattened block and entries iterator
* Support show_entries
* Finalize unified scheduler plumbing with min impl
* Fix comment
* Rename leftover type name...
* Make logging text less ambiguous
* Make PhantomData simplyer without already used S
* Make TaskHandler stateless again
* Introduce HandlerContext to simplify TaskHandler
* Add comment for coexistence of Pool::{new,new_dyn}
* Fix grammar
* Remove confusing const for upcoming changes
* Demote InstalledScheduler::context() into dcou
* Delay drop of context up to return_to_pool()-ing
* Revert "Demote InstalledScheduler::context() into dcou"
This reverts commit 049a126c905df0ba8ad975c5cb1007ae90a21050.
* Revert "Delay drop of context up to return_to_pool()-ing"
This reverts commit 60b1bd2511a714690b0b2331e49bc3d0c72e3475.
* Make context handling really type-safe
* Update comment
* Fix grammar...
* Refine type aliases for boxed traits
* Swap the tuple order for readability & semantics
* Simplify PooledScheduler::result_with_timings type
* Restore .in_sequence()
* Use where for aesthetics
* Simplify if...
* Fix typo...
* Polish ::schedule_execution() a bit
* Fix rebase conflicts..
* Make test more readable
* Fix test failures after rebase...
* Return early if reference set is empty
* Remove unreachable case
* Update reference_last_block
* Limit reference set by last_block_checked
* Return superflous-block data too
* Sort missing/superfluous blocks
* Fix tests
* Move last_block_checked calculation into missing_blocks()
* Prevent missing_slots panic
* Fix test
Several commands call load_and_process_ledger() which can fail in a
number of ways. These callers currently all handle the result in the
same way by matching the return Result:
- The Ok(_) case uses the returned types as normal
- The Err(_) case prints an error message and exits
This error handling is redundant, and a helper could remove the
duplicate code. So, this PR adds a wrapper around that checks the
result and unwraps OR prints error messages and exits.
load_and_process_ledger() performs many checks and sub-operations that
can fail. The current error handling prints an error message and exits
immediately. The long error/help messages written inline add clutter
to the functions actual implementation.
This PR creates a new error enum for all of these previous error
conditions, and bubbles up the error to let the caller decide what to
do instead of exiting immediately.
Long string literals can cause rustfmt to fail, which results in rustfmt
failing to format entire functions. There are several instances of this
in ledger-tool, so format these files with wrapped strings so that
formatting will apply to functions again.
Note that this PR was created by adding format_strings = true to
rustfmt.toml; however, this change does NOT persist that rule as the
rule would format the entire repo.
There are operations in bank_fork_utils that may fail; we explicitly
call std::process::exit() on several of these. Granted we may end up
exiting the process higher up the callstack, bubbling the errors up
allow a caller that could handle the error to do so.
* Add helper to get a protobuf cell without bothering with bincode
* Add conversion back to EntrySummaries
* Add LedgerStorage::get_block_entries
* Add ledger-tool bigtable entries subcommand
* Move CliEntries into ledger_tool::output for now
* Fetch entries with read-only token
Currently, the file is generated when a node drops a block that was
produced by another node. However, it would also be beneficial to see
the account state when a node drops its' own block.
Output the file in this additional failure codepath
This helper simply called std::mem::size_of<Self::Index>(). However, all
of the underlying functions that create keys manually copy fields into a
byte array. The fields are copied in end-to-end whereas size_of() might
include alignment bytes.
For example, a (u64, u32) only has 12 bytes of "data", but it would
have size 16 due to the 4 alignment padding bytes that would be
added to get the u32 (size 4) aligned with the u64 (size 8).
The Blockstore currently maintains a RwLock<Slot> of the maximum root
it has seen inserted. The value is initialized during
Blockstore::open() and updated during calls to Blockstore::set_roots().
The max root is queried fairly often for several use cases, and caching
the value is cheaper than constructing an iterator to look it up every
time.
However, the access patterns of these RwLock match that of an atomic.
That is, there is no critical section of code that is run while the
lock is head. Rather, read/write locks are acquired in order to read/
update, respectively. So, change the RwLock<u64> to an AtomicU64.
* allow test feature to skip rewrites
* hook up cli arg for test skip rewrites, update tests
* fix sanity checker
* add account hash to abi to fix a test
* reviews
* use hashmap to collect skip_rewrites. exclude skip_rewrites from dirty
pubkey set
* accumulate skipped_rewrite in reduce
* mutex
* fmt
* skip hash verify for this test flag
* add skipped rewrites num stat
* skip bank hash verify not account hash verify
* reviews
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
* Do not unnecessarily re-verify unloaded program
* clippy fixes
* new unit test for extract
* fixes after rebase
* fixes after rebase
* address review comments
* fix clippy
* allow pedantic invalid cast lint
* allow lint with false-positive triggered by `test-case` crate
* nightly `fmt` correction
* adapt to rust layout changes
* remove dubious test
* Use transmute instead of pointer cast and de/ref when check_aligned is false.
* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.
* bump rust nightly to 2023-08-25
* Upgrades Rust to 1.72.0
---------
Co-authored-by: Trent Nelson <trent@solana.com>
In most cases, either a &Bank or an Arc<Bank> is more proper.
- &Bank is used if the function only needs a momentary reference
- Arc<Bank> is used if the function needs its' own copy
This PR leaves several instances of &Arc<Bank> around; these instances
are situations where a clone may only happen conditionally.
* Integrate program loader-v4 with bank
* fix tests
* new struct for ProgramRuntimeEnvironments
* remove environment from program_runtime_environment_v
* move find_program_in_cache() to invoke_context
* cleanup
When ledger-tool runs, it may create secondary directories for things
like accounts, accounts-index, etc as not to potentially interfere with
solana-validator's directories. These would show as multiple directories
with ".ledger-tool" appended to the typical directory name.
To more clearly group these items, make the default directories for
snapshots, accounts and accounts-index all within a common "ledger-tool"
directory.
When a consensus divergance occurs, the current workflow involves a
handful of manual steps to hone in on the offending slot and
transaction. This process isn't overly difficult to execute; however, it
is tedious and currently involves creating and parsing logs.
This change introduces functionality to output a debug file that
contains the components go into the bank hash. The file can be generated
in two ways:
- Via solana-validator when the node realizes it has diverged
- Via solana-ledger-tool verify by passing a flag
When a divergance occurs now, the steps to debug would be:
- Grab the file from the node that diverged
- Generate a file for the same slot with ledger-tool with a known good
version
- Diff the files, they are pretty-printed json
steviez
HaoranYi
Tyera
Ryo Onodera
Brooks
Alexander Meißner
Jeff Washington (jwash)
Justin Starry
Andrew Fitzgerald
Pankaj Garg
Alessandro Decina
Jon Cinque