Commit Graph

471 Commits

Author SHA1 Message Date
Brooks 5c7ab5dc08
ledger-tool does *not* fastboot by default (#34228) 2023-11-27 13:48:28 -05:00
steviez 9a7b681f0c
Remove key_size() method from Column trait (#34021)
This helper simply called std::mem::size_of<Self::Index>(). However, all
of the underlying functions that create keys manually copy fields into a
byte array. The fields are copied in end-to-end whereas size_of() might
include alignment bytes.

For example, a (u64, u32) only has 12 bytes of "data", but it would
have size 16 due to the 4 alignment padding bytes that would be
added to get the u32 (size 4) aligned with the u64 (size 8).
2023-11-19 23:05:32 -06:00
Brooks e02f25d5a2
Removes filler accounts (#34115) 2023-11-19 20:36:57 -05:00
Brooks b4c652ef6c
cli: Hides the filler accounts args (#34113) 2023-11-16 12:58:52 -05:00
Jeff Washington (jwash) a18debc34a
allow test feature to skip rewrites (#33851)
* allow test feature to skip rewrites

* hook up cli arg for test skip rewrites, update tests

* fix sanity checker

* add account hash to abi to fix a test

* reviews

* use hashmap to collect skip_rewrites. exclude skip_rewrites from dirty
pubkey set

* accumulate skipped_rewrite in reduce

* mutex

* fmt

* skip hash verify for this test flag

* add skipped rewrites num stat

* skip bank hash verify not account hash verify

* reviews

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-10-27 09:14:05 -05:00
Ryo Onodera 1704789247
Define tick related helper test methods (#33537)
* Define tick related helper methods

* dcou VoteSimulator

* blacklist ledger-tool for dcou

* fix dcou ci...

* github
2023-10-10 09:23:18 +09:00
Andrew Fitzgerald 5a95e5676e
Manually add lookup table addresses instead of sanitizing (#33273) 2023-10-04 08:04:43 -07:00
Brooks 1d39c3167d
Adds default info to cli arg for --accounts-hash-cache-path (#33331) 2023-09-20 23:45:55 -04:00
Brooks 2839d51d5e
Unhides fastboot cli arg (#33329) 2023-09-20 16:52:52 -04:00
steviez a15564d719
ledger-tool: Cleanup argument parsing logic (#33283)
value_t!() macro and .unwrap_or() are more succinct than if/else blocks,
and the value_t!() macro handles parsing values from strings.
2023-09-18 11:07:35 -05:00
Brooks adee97fe38
Ledger-tool CLI can specify accounts hash cache path (#33118) 2023-09-02 16:17:03 -04:00
Alexander Meißner 9e703f85de
Upgrades Rust to 1.72.0 & nightly-2023-08-25 (#32961)
* allow pedantic invalid cast lint

* allow lint with false-positive triggered by `test-case` crate

* nightly `fmt` correction

* adapt to rust layout changes

* remove dubious test

* Use transmute instead of pointer cast and de/ref when check_aligned is false.

* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.

* bump rust nightly to 2023-08-25

* Upgrades Rust to 1.72.0

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-09-01 07:26:13 +00:00
steviez a4c8cc3ce0
Remove improper uses of &Arc<Bank> (#32802)
In most cases, either a &Bank or an Arc<Bank> is more proper.
- &Bank is used if the function only needs a momentary reference
- Arc<Bank> is used if the function needs its' own copy

This PR leaves several instances of &Arc<Bank> around; these instances
are situations where a clone may only happen conditionally.
2023-08-18 16:46:34 -05:00
Brooks 004577d94a
Adds docs to scan_all_accounts() (#32888) 2023-08-18 16:04:07 +00:00
steviez 6bbf514e78
Add ability to output components that go into Bank hash (#32632)
When a consensus divergance occurs, the current workflow involves a
handful of manual steps to hone in on the offending slot and
transaction. This process isn't overly difficult to execute; however, it
is tedious and currently involves creating and parsing logs.

This change introduces functionality to output a debug file that
contains the components go into the bank hash. The file can be generated
in two ways:
- Via solana-validator when the node realizes it has diverged
- Via solana-ledger-tool verify by passing a flag

When a divergance occurs now, the steps to debug would be:
- Grab the file from the node that diverged
- Generate a file for the same slot with ledger-tool with a known good
  version
- Diff the files, they are pretty-printed json
2023-08-15 00:12:05 -05:00
Jon Cinque c73a56faf8
stake: Rename `StakeStateWithFlags` -> `StakeStateV2` (#32795) 2023-08-11 01:07:21 +02:00
Pankaj Garg f4287d70bb
Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
steviez b97c451200
ledger-tool: Relax ledger requirements for program subcommand (#32770)
Several of the program command subcommands do not require a ledger as
they act solely on the program object. So, defer checking the ledger
path until we know we need to load. Additionally, remove genesis arg
from these commands that do not load a genesis.bin
2023-08-09 14:13:25 -05:00
Jon Cinque 8e4a9a94ed
sdk: Add new version of `StakeState` to avoid breaking downstream users (#32736)
* sdk: Rename `StakeState` -> `StakeStateWithFlags`

* Add back `StakeFlags` with a deprecation warning
2023-08-09 00:05:40 +02:00
steviez 028f10d3d1
ledger-tool: Add flag to ignore open file descriptor limit error (#32624)
The current desired open file descriptor limit is 1,000,000. This is
quite a large number, and not needed for every command. Namely, commands
that do not unpack a snapshot and create an AccountsDB will likely not
use this many files.

There is already an option in BlockstoreOptions to ignore errors if the
desired value cannot be set; this PR just bubbles that option up to a
CLI flag in ledger-tool.
2023-07-27 11:50:45 -05:00
steviez 9980771788
ledger-tool: Workaround clap bug for encoding arg in genesis command (#32626)
The --encoding flag only makes sense for the genesis command if the
genesis accounts will be printed. Hence, the encoding flag used the
Arg::requires() function on --accounts argument.

However, in what appears to be a clap bug, this made the --accounts flag
required even though it should not be. So, remove the .requires()
dependency between these two arguments.
2023-07-26 13:25:56 -05:00
Pankaj Garg aba637d5d9
Split snapshot_utils.rs into snapshot_bank_utils.rs (#32612) 2023-07-24 16:31:03 -07:00
HaoranYi 17af3ab10a
Add stake_flags to stake state (#32524)
add stake_flags to stake state

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-07-24 09:09:40 -05:00
behzad nouri d54b6204be
removes instances of clippy::manual_let_else (#32417) 2023-07-09 21:41:36 +00:00
steviez 74d54ccd7c
ledger-tool: Add flag to store tx metadata in verify subcommand (#32400) 2023-07-08 17:33:22 -05:00
Illia Bobyr 282e043177
`cargo fmt` using 1.6.0-nightly (#32390)
Seems like rustfmt 1.6.0 can now format `let/else` statements.
1.5.2 we use in our `solana-nightly` does not mind the reformatting.

```
$ cargo +nightly fmt --version
rustfmt 1.6.0-nightly (f20afcc 2023-07-04)

$ cargo +nightly fmt
$ git add -u
$ git commit

$ ./cargo nightly fmt --version
+ exec cargo +nightly-2023-04-19 fmt --version
rustfmt 1.5.2-nightly (c609da5 2023-04-18)

$ ./cargo nightly fmt
$ git diff
[empty output]
```
2023-07-06 20:45:29 -07:00
Pankaj Garg 4674b0099f
Move CostModel and CostTracker to its own crate (#32354)
* Move CostModel and CostTracker to its own crate

* compile new crate and update imports

* update sbf Cargo.lock

* fix AbiExample

* fix cargo sort

* Fix AbiExample
2023-07-06 10:08:18 -07:00
steviez 5feebd2dc8
ledger-tool: Manually walk optimistic slots ancestors (#32362)
If a slot is marked as optimistically confirmed, it is probable but not
guaranteed that its' ancestors will also be marked as optimistically
confirmed in the Blockstore. Given the importance of examining
optimistically confirmed slots around cluster restarts, manually walk
an AncestorIterator to avoid the chance of a slot improperly being
ignored in cluster restart scenarios.
2023-07-05 10:18:15 -04:00
steviez d5ad29d837
Make Blockstore::scan_and_fix_roots() take optional start/stop slots (#32289)
The optional args allow reuse by ledger-tool repair roots command Also,
hold cleanup lock for duration of Blockstore::scan_and_fix_roots().

This prevents a scenario where scan_and_fix_roots() could identify a
slot as needing to be marked root, that slot getting cleaned by
LedgerCleanupService, and then scan_and_fix_roots() marking the slot as
root on the now purged slot.
2023-06-28 22:32:03 -05:00
Brooks ee6684b2eb
Adds UseSnapshotArchivesAtStartup::WhenNewest (#32284) 2023-06-27 14:26:40 -04:00
Brooks 5f1b5b877a
Replace boot_from_local_state with use_snapshot_archives_at_startup (#32260) 2023-06-26 12:44:25 -04:00
Brooks 64ecfaf769
Renames --boot-from-local-state to --use-snapshot-archives-at-startup (#32217) 2023-06-23 18:34:18 -04:00
steviez 20a7cdd43d
Restrict access to Bank's HardForks (#32180)
* Restrict access to Bank's HardForks

Callers could previously obtain a a lock to read/write HardForks from
any Bank. This would allow any caller to modify, and creates the
opportunity for inconsistent handling of what is considered a valid hard
fork (ie too old).

This PR adds a function to Bank so consistent sanity checks can be
applied; the caller will already have a Bank as that is where they would
have obtained the HardForks from in the first place. Additionally,
change the getter to return a copy of HardForks (simple Vec).

* Allow hard fork at bank slot if bank is not yet frozen
2023-06-20 23:44:43 -05:00
Brooks 2aac9fca35
Adds --boot-from-local-state to ledger-tool create-snapshot (#32032) 2023-06-16 12:58:57 -04:00
Alexander Meißner ee2c2ef6c7
Cleanup - require_static_program_ids_in_transaction (#31767)
require_static_program_ids_in_transaction
2023-06-07 17:12:41 +02:00
Brooks 7b37a1672f
Enables fastboot for ledger-tool (#31921) 2023-06-06 18:32:24 -04:00
steviez 1d6b03358f
ledger-tool: Rename load_frozen_forks() and adjust error messages (#31903)
This function has morphed as it has been updated, and the existing
function name is a bit of misnomer. Update the function name to use
terminology that I think is more clear and more consistent with what we
use core/src/validator.rs. Additionally, adjust the error messages to be
more clear about what exactly is wrong for when we can't process.
2023-05-31 21:12:49 -05:00
Jeff Washington (jwash) aff9c03e7a
fix cli args (#31804) 2023-05-25 07:47:00 -07:00
Jeff Washington (jwash) 461342cdd4
add cli args and use of partitioned rewards config (#31800)
* add cli args and use of partitioned rewards config

* update comments
2023-05-24 10:54:09 -07:00
Dmitri Makarov fb79e2bbb8
Reorganize ledger-tool run sub-command (#31572)
* Rename ledger-tool run subcommand to program subcommand

* Reorganize ledger-tool program command to multiple subcommands
2023-05-10 15:25:38 -04:00
Dmitri Makarov f09e791a09
Add debugging/tracing capabilities to ledger-tool, previously rbp… (#31346)
* Add debug and trace capabilities to ledger-tool, previously rbpf-cli

* Dump nested logs from saved traces in invoke_context

* Change the ledger-tool run input format to represent keys as strings
2023-05-05 05:14:41 -04:00
Dmitri Makarov e03826b6d8
Refactor ledger-tool moving some of functions to a separate module (#31359)
* Refactor ledger-tool moving some of functions to a separate module

* Rename utils to ledger_utils, move arg parsing funcs to a new module
2023-04-27 14:11:04 -04:00
Brooks c73b6545e6
Spins up AccountsHashVerifier in ledger-tool (#31358) 2023-04-26 12:07:35 -04:00
Alexander Meißner 7002c568fb
Cleanup - Removes the CLI option "no-bpf-jit" (#31329)
* Enables JIT everywhere by default.

* Removes the CLI argument "no-bpf-jit" and its plumbering through out the validator code base.

* Removes with_jit bpf_loader variants.

* Removes the to_builtin!() macro.
2023-04-25 19:04:11 +02:00
Brooks ca1bde3591
Use Arc instead of &Arc in AccountsBackgroundService::new (#31268) 2023-04-19 11:10:41 -04:00
Trent Nelson f34a6bcfce
runtime: transpose `VoteAccount::vote_state()` return to improve ergonomics (#31256) 2023-04-18 14:48:52 -06:00
steviez f3935f679e
ledger-tool: Add deprecation warning for --no-compaction (#31178)
The argument is deprecated with a change in default behavior but this
may not be obvious to a user; the warning makes this a bit more visible.
2023-04-15 00:18:03 -05:00
Brooks d43e19bb03
Refactors the Full/Incremental SnapshotHash types (#31186) 2023-04-13 18:01:27 -04:00
Jeff Washington (jwash) 3d6248d76a
ledger-tool verify cli for final hash calc (#31153) 2023-04-12 07:30:45 -07:00
steviez cc8e531a5d
Enforce a minimum of 1 on full and incremental snapshot retention (#30968) 2023-03-30 10:16:36 -05:00