RuntimeConfig doesn't use anything SVM specific and logically belongs
in program runtime rather than SVM. This change moves the definition
of RuntimeConfig struct from the SVM crate to program-runtime and
adjusts `use` statements accordingly.
ledger-tool: verify: add --verify-slots and --verify-slots-details
This adds:
--record-slots <FILENAME>
Write the slot hashes to this file.
--record-slots-config hash-only|accounts
Store the bank (=accounts) json file, or not.
--verify-slots <FILENAME>
Verify slot hashes against this file.
The first case can be used to dump a list of (slot, hash) to a json file
during a replay. The second case can be used to check slot hashes against
previously recorded values.
This is useful for debugging consensus failures, eg:
# on good commit/branch
ledger-tool verify --record-slots good.json --record-slots-config=accounts
# on bad commit or potentially consensus breaking branch
ledger-tool verify --verify-slots good.json
On a hash mismatch an error will be logged with the expected hash vs the
computed hash.
There are lots of operations that could fail, including lots of the
Blockstore calls. The old code matched on Ok(_) or did unwrap()'s
which clutter the code and increase indentation.
This change wraps the entire command in a function that returns a
Result. The wrapper then does a single unwrap_or_else() and prints
any error message. Everywhere else is now free to use the ? operator
This change makes solana-ledger-tool match solana-validator CLI; the old
flag --snapshot-archive-path is retained as an argument alias and can
still be used for the sake of backwards compatibility.
The accounts command currently dumps every single account in the
AccountsDb. This is obviously a lot of output, so a previous change
streamed the accounts instead of collecting and dumping at the end.
The streaming approach is much more performant, but the implementation
is non-trivial. This change
- Moves the accounts output code to output.rs
- Refactor the logic to several objects that implment the functionality
- Adjust the json output to also include the summary
This change lays the groundwork for cleanly adding several more flags
that will allow for querying different subsets of accounts.
This argument is a flag and doesn't take a value; however, it had the
.value_name() modifier set with "FORMAT". This could be confusing so
remove .value_name() and add .takes_value(false)
The function open_genesis_config() performs several operations that
could fail. If any of these fail, the process exits immediately.
Instead of exiting immediately, bubble up the error and let the caller
decide the appropriate action. solana-validator and solana-ledger-tool
will functionally be unchanged, but this consolidates startup failures
for both of these processes.
The code to parse process options was repeated in several (8) commands
that replay block processing. So, move the argument parsing into a
common helper that can be used by all of those commands.
* Update proto files with tonic-build v0.9.2
* Manually ignore invalid doc-tests
* Add new ReadRowsRequest fields
* Add LedgerStorageConfig::max_message_size and default value
* Add BigtableConnection::max_message_size and use on client creation
* Add max_message_size to RpcBigtableConfig and make const pub
* Add solana-validator cli arg
solana-ledger-tool is a bit of a kitchen sink and currently has
upwards of 30 commands at the top level of the CLI. UI aside, the
contents of ledger-tool/src/main.rs are somewhat cluttered.
To reduce clutter and introduce a more hierarchical structure, this
change introduces a new blockstore subcommand and moves all commands
that only touch the blockstore to a blockstore subcommand. For example,
solana-ledger-tool slot X
is now invoked like
solana-ledger-tool blockstore slot X
However, given how long the tool has existed, support is retained for
calling these old commands without the extra blockstore command. But,
the commands are not listed in the help in order to steer new users to
use the newer calling structure.
TransactionStatusService needs Primary access in order to write
transaction status into the Blockstore if enable_rpc_transaction_history
is set to True. The current logic attempts to get Primary access for the
service.
However, in the event that this function had been called with a
Blockstore that already had Primary access, this second attempt to get
Primary access would fail. So, only attempt to open with Primary access
when necessary AND when the current access level is not sufficient.
A future change will add more cases to this if-else if-...-else chain.
Using a match statement will be easier to follow then a very long
if-else if-... chain.
This change was broken out in order to have a higher signal to noise
ratio in the subsequent change.
The open_blockstore() helper currently takes multiple configurable
options. While the arguments are parsed at a high enough scope in
main.rs to avoid repeated calls, this parsing is duplicated in the
separate subcommand files (program.rs and bigtable.rs).
The repeated parsing is redundant, and also prone to having to missing
an arg (as was the case with bigtable not having wal_recovery_mode).
So, this PR consolidates the parsing to a single function and uses that
function across the previous callers.
warning: `flatten()` will run forever if the iterator repeatedly produces an `Err`
--> ledger-tool/src/main.rs:2649:39
|
2649 | for line in f.lines().flatten() {
| ^^^^^^^^^ help: replace with: `map_while(Result::ok)`
|
note: this expression returning a `std::io::Lines` may produce an infinite number of `Err` in case of a read error
--> ledger-tool/src/main.rs:2649:29
|
2649 | for line in f.lines().flatten() {
| ^^^^^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#lines_filter_map_ok
= note: `#[warn(clippy::lines_filter_map_ok)]` on by default
warning: `solana-ledger-tool` (bin "solana-ledger-tool" test) generated 1 warning
* Add cli flag to show entry data
* Add display structs
* Add writeln_entry helper fn
* Add entry conversion method
* Populate Display for CliBlockWithEntries
* Add ctor from flattened block and entries iterator
* Support show_entries
* Finalize unified scheduler plumbing with min impl
* Fix comment
* Rename leftover type name...
* Make logging text less ambiguous
* Make PhantomData simplyer without already used S
* Make TaskHandler stateless again
* Introduce HandlerContext to simplify TaskHandler
* Add comment for coexistence of Pool::{new,new_dyn}
* Fix grammar
* Remove confusing const for upcoming changes
* Demote InstalledScheduler::context() into dcou
* Delay drop of context up to return_to_pool()-ing
* Revert "Demote InstalledScheduler::context() into dcou"
This reverts commit 049a126c905df0ba8ad975c5cb1007ae90a21050.
* Revert "Delay drop of context up to return_to_pool()-ing"
This reverts commit 60b1bd2511a714690b0b2331e49bc3d0c72e3475.
* Make context handling really type-safe
* Update comment
* Fix grammar...
* Refine type aliases for boxed traits
* Swap the tuple order for readability & semantics
* Simplify PooledScheduler::result_with_timings type
* Restore .in_sequence()
* Use where for aesthetics
* Simplify if...
* Fix typo...
* Polish ::schedule_execution() a bit
* Fix rebase conflicts..
* Make test more readable
* Fix test failures after rebase...
* Return early if reference set is empty
* Remove unreachable case
* Update reference_last_block
* Limit reference set by last_block_checked
* Return superflous-block data too
* Sort missing/superfluous blocks
* Fix tests
* Move last_block_checked calculation into missing_blocks()
* Prevent missing_slots panic
* Fix test
Several commands call load_and_process_ledger() which can fail in a
number of ways. These callers currently all handle the result in the
same way by matching the return Result:
- The Ok(_) case uses the returned types as normal
- The Err(_) case prints an error message and exits
This error handling is redundant, and a helper could remove the
duplicate code. So, this PR adds a wrapper around that checks the
result and unwraps OR prints error messages and exits.
load_and_process_ledger() performs many checks and sub-operations that
can fail. The current error handling prints an error message and exits
immediately. The long error/help messages written inline add clutter
to the functions actual implementation.
This PR creates a new error enum for all of these previous error
conditions, and bubbles up the error to let the caller decide what to
do instead of exiting immediately.
Long string literals can cause rustfmt to fail, which results in rustfmt
failing to format entire functions. There are several instances of this
in ledger-tool, so format these files with wrapped strings so that
formatting will apply to functions again.
Note that this PR was created by adding format_strings = true to
rustfmt.toml; however, this change does NOT persist that rule as the
rule would format the entire repo.
HaoranYi
Pankaj Garg
steviez
Brooks
Dmitri Makarov
Will Hickey
Yihau Chen
Tyera
Sean Young
Alexander Meißner
Ryo Onodera
Brooks
Pankaj Garg
steviez