Commit Graph

393 Commits

Author SHA1 Message Date
Trent Nelson c4e43f1de4
vote: encapsulate `Lockout` (#29753) 2023-01-18 19:28:28 -07:00
steviez 9bd0ce4082
ledger-tool: Clean account paths in all cases (#29609)
The existing logic would clean account paths only when we had secondary
access to the blockstore. However, the access mode shouldn't dictate
when we clean accounts data; we can and should clean account data from
previous runs in all instances given that we always start over from a
snapshot.
2023-01-17 21:53:51 -06:00
Giovanni Napoli 5eab3fb314
Add output in JSON for `solana-ledger-tool bounds` subcommand (#28410)
Introduce a struct to store all of the relevant slot/root information, and then output all in one go at the end as either human-readable or json. There are some slight changes to the human-readable format for the case of an empty ledger
2023-01-12 23:21:04 -06:00
Brooks d0aa93de21
Refactors accounts hash cache (#29625) 2023-01-12 10:43:50 -05:00
steviez 3992cee195
ledger-tool: Update dated println!() calls (#29636)
Removed an "ok" print that occurs after several commands; we already
print a log statement that indicates ledger-tool is done and how long it
took. Additionally, switch a println to info to avoid polluting stdout
incase we want to print information in a more easily readable format
such as json.
2023-01-10 20:41:47 -06:00
Jeff Washington (jwash) c9fe21e3c3
ancient append vecs enabled at -10k by default (#29565) 2023-01-10 05:33:43 -08:00
Boqin Qin(秦 伯钦)@MetaSecure cd5c5804ab
ledger-tool: fix double-readlock in main (#27322) 2023-01-06 19:30:07 +00:00
HaoranYi 4a8b980b81
update leger tool help for db verify refcounts (#29486) 2023-01-03 13:34:06 -06:00
Xiang Zhu 3363c08ac0
Move async remove to snapshot_utils.rs (#29406) 2023-01-03 06:15:32 -08:00
Jeff Washington (jwash) 4cc1890f00
skip_rewrites will only be feature driven (#29468) 2023-01-02 20:15:42 -08:00
Brooks Prumo 4530d161b7
Fixes format strings (#29327) 2022-12-19 17:26:38 -05:00
steviez 0609dce36c
Make ledger-tool list-roots subcommand inclusive of start-root arg (#29265) 2022-12-15 22:33:57 -06:00
Jeff Washington (jwash) b95835143e
remove AccountsBackgroundService::new(caching_enabled) (#29234) 2022-12-13 07:18:02 -08:00
Jeff Washington (jwash) fec8f61566
remove ProcessOptions::accounts_db_caching_enabled (#29217) 2022-12-12 20:25:00 -08:00
Brooks Prumo e02d1670fb
Uses Storages to calculate accounts hash when warping in `ledger-tool create-snapshot` (#29194) 2022-12-12 11:22:15 -05:00
Jeff Washington (jwash) 631a98a3b6
warp_from_parents works with write_cache enabled (#29185) 2022-12-09 14:28:18 -08:00
HaoranYi ca5d8c4b4d
refactor sysmonitor config (#29132) 2022-12-09 07:43:03 -06:00
Jeff Washington (jwash) 2f4731570a
ledger-tool create-snapshot uses write cache (#29119)
* ledger-tool create-snapshot uses write cache

* ledger tool create snapshot works with write cache
2022-12-08 06:09:52 -08:00
HaoranYi 33b15240ac
Revert #28945 (#29127)
revert #28945
2022-12-06 17:08:56 -06:00
behzad nouri 9524c9dbff patches errors from clippy::uninlined_format_args
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
haoran 037f03c806 fix 2022-12-06 13:31:50 -06:00
haoran 1f7d2785ef fix build 2022-12-06 13:31:50 -06:00
Haoran Yi e1ba5a2a63 add monitoring for open file descriptors stat 2022-12-06 13:31:50 -06:00
Maximilian Schneider c8b0c3ede9
Update cost model to use requested_cu instead of estimated cu #27608 (#28281)
* Update cost model to use requested_cu instead of estimated cu #27608

* remove CostUpdate and CostModel from replay/tvu

* revive cost update service to send cost tracker stats

* CostModel is now static

* remove unused package

Co-authored-by: Tao Zhu <tao@solana.com>
2022-11-22 11:55:56 -06:00
apfitzge 08e017b5ba
Improve error message (#28898) 2022-11-22 10:02:44 -06:00
Jeff Washington (jwash) f0ef316b19
remove cli arg "--no-accounts-db-caching" (#28912) 2022-11-21 12:38:35 -08:00
steviez 7bc78196b5
ledger-tool: collapse repeated encoding argument parsing (#28859) 2022-11-21 13:15:14 -05:00
Trent Nelson 53a579bed1 ledger-tool: stream output of `accounts` subcommand 2022-11-17 12:45:26 -07:00
Jeff Washington (jwash) f69e847137
allow ancient append vec threshold to be set on cli (#28707)
* allow ancient append vec threshold to be set on cli

* pr responses
2022-11-11 11:30:05 -08:00
Brooks Prumo d1ba42180d
clippy for rust 1.65.0 (#28765) 2022-11-09 19:39:38 +00:00
steviez 54fc768fcc
Single source ledger-tool encoding arg (#28622)
The argument is nearly identical for both instances that use it, so
de-duplicate the repeated config.
2022-10-27 07:35:27 +00:00
Trent Nelson 2c0831aa12 ledger-tool: remove inefficient base58 encoding options 2022-10-19 19:16:52 -07:00
Brooks Prumo dd7fee8f32
Re-enqueues unhandled ABS requests (#28362) 2022-10-13 16:25:39 -04:00
Brooks Prumo 9cbd00fdbc
Converts PendingAccountsPackage to a channel (#28352) 2022-10-13 12:47:36 -04:00
Jeff Washington (jwash) 7ae7a3ff0e
after replay at validator startup don't verify hash calc (#28222) 2022-10-04 14:04:04 -07:00
steviez 7fef7d569a
Make RocksFifo storage size API expose an Option<u64> (#28192)
A fifo rocksdb instance must be opened with max size parameter on the
fifo columns. To support this, we previously plumbed a constant up to
callers that provided a default if unbounded growth desired.

This change attempts to be more rusty by exposing an option for this
value, and converting the option to a constant at the lowest level
possible.
2022-10-04 12:38:40 -05:00
steviez 8cc8a0656a
Cleanup solana-ledger-tool slot output (#28129) 2022-10-03 23:25:44 -05:00
Yueh-Hsuan Chiang 6b17bee5a8
Remove the const default for RocksFifo (#27965)
#### Summary of Changes
Removes the constant default for ShredStorageType::RocksFifo
as the shred storage size is either user-specified or derived
from --limit-ledger-size in #27459.
2022-10-01 15:10:54 -07:00
Yueh-Hsuan Chiang 599677f965
Derive rocksdb_fifo_shred_storage_size based on limit-ledger-size (#27459)
### Problem
When FIFO compaction is used while --rocksdb_fifo_shred_storage_size
is unspecified, the FIFO shred storage size is set to a const default based
on the default `--limit-ledger-size`.

### Summary of the Change
When --rocksdb_fifo_shred_storage_size is unspecified, it is now
derived from `--limit-ledger-size` by reserving 1500 bytes for each
shred.
2022-09-28 00:32:27 -07:00
Amit Panghal 860cd607fd
add flag to genesis cmd to dump account data in specified format (#27695)
* feat: add flag to genesis cmd to dump account data in jsonParsed format

* update Cargo.lock

* add json-parsed flag

* fix formatting

* fix formatting again

* use rustfmt

* use --encoding

* fix for failing checks

* Apply suggestions from code review

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* address review comments

* address review comments

* fix indent

* fix print fmt

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-09-18 22:52:53 -06:00
steviez 1d5314b5e6
Make solana-ledger-tool retain all snapshots by default (#27788) 2022-09-15 16:21:12 -05:00
Yueh-Hsuan Chiang 70c4f6ea96
Enable ledger-tool copy command to infer target_db shred storage type. (#27438)
#### Problem
The ledger-tool copy command currently assumes target_db uses the same
shred storage type as the source ledger.

#### Summary of Changes
This PR enables ledger-tool copy command to infer the target_db shred storage
type based on whether rocksdb or rocksdb_fifo exists under the target_db directory
(same way as how ledger-tool infers the shred storage type for the main db.)
If the ledger-tool is not able to infer the shred storage type of the target_db, then
the default level compaction will be used.
2022-09-13 10:27:58 -07:00
Jeff Washington (jwash) 765c628546
use exit signal for acct idx bg threads (#27483) 2022-09-12 11:51:12 -07:00
Jeff Washington (jwash) 518e16744d
move --accounts-db-verify-refcounts under verify (#27651) 2022-09-08 07:59:24 -07:00
Xiang Zhu 4308c300b4
In ledger-tool delete the account files in the async way (#27622)
* In ledger-tool delete the account files in the async way

* format changes by ./cargo nightly fmt --all
2022-09-07 14:35:06 -07:00
Brooks Prumo 93a4f80a2c
Handling snapshot requests is now required (#27537) 2022-09-07 10:08:42 -04:00
Yueh-Hsuan Chiang ed00365101
Add ledger tool command print-file-metadata (#26790)
Add ledger-tool command print-file-metadata

#### Summary of Changes
This PR adds a ledger tool subcommand print-file-metadata.
```
USAGE:
    solana-ledger-tool print-file-metadata [FLAGS] [OPTIONS] [SST_FILE_NAME]

    Prints the metadata of the specified ledger-store file.
    If no file name is unspecified, then it will print the metadata of all ledger files
```
2022-09-06 21:46:35 -07:00
Jeff Washington (jwash) 72a2c02ee7
add --accounts-db-verify-refcounts for debugging (#27504) 2022-09-06 09:39:39 -07:00
Brooks Prumo 6684c62280
Add SnapshotUsage to SnapshotConfig (#27508) 2022-09-02 08:56:23 -04:00
Brooks Prumo 3c7cd62030
Move pruned_banks_receiver into PrunedBanksRequestHandler (#27445) 2022-08-29 13:30:06 -04:00