Richard Patel
f7c6901191
ledger-tool: support Geyser accounts updates ( #26909 )
2022-08-09 15:44:57 +00:00
dependabot[bot]
07a0076fc2
chore: bump bytecount from 0.6.2 to 0.6.3 ( #27014 )
...
Bumps [bytecount](https://github.com/llogiq/bytecount ) from 0.6.2 to 0.6.3.
- [Release notes](https://github.com/llogiq/bytecount/releases )
- [Commits](https://github.com/llogiq/bytecount/commits )
---
updated-dependencies:
- dependency-name: bytecount
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-09 00:33:43 -06:00
Will Hickey
ed8c224374
Bump version to v1.12 ( #26967 )
2022-08-06 13:20:30 -05:00
Tyera Eulberg
66919e5adb
Unpin tokio for non-rpc crates ( #26957 )
2022-08-05 20:50:45 -06:00
steviez
300666dce7
Make `solana-ledger-tool` run AccountsBackgroundService ( #26914 )
...
Prior to this change, long running commands like `solana-ledger-tool
verify` would OOM due to AccountsDb cleanup not happening.
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-04 15:44:31 -05:00
github-actions[bot]
fbf1bf6d86
Bump Version to 1.11.6 ( #26906 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-08-03 12:48:43 -05:00
Brennan Watt
f3b760dd91
Add IO metrics ( #26804 )
...
* Add Disk IO metrics
2022-08-02 14:29:53 -07:00
Jeff Washington (jwash)
c7462b7a52
ledger tool verify can store debug info on hash calc ( #26837 )
2022-07-29 15:54:56 -05:00
github-actions[bot]
5d038b9d2a
Bump Version to 1.11.5 ( #26758 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-25 13:05:14 -06:00
Trent Nelson
0fff5b8bf8
ledger-tool: optionally `graph` vote states
2022-07-22 17:43:01 -06:00
Trent Nelson
633950e029
ledger-tool: factor out `graph` options to config struct
2022-07-22 17:43:01 -06:00
Trent Nelson
a541827fb9
ledger-tool: fix off-by-one in `graph` skipped slot label
2022-07-22 17:43:01 -06:00
Yueh-Hsuan Chiang
dcbda2c6c5
Allow ledger tool to automatically detect the shred compaction style ( #26182 )
...
#### Problem
Ledger-tool doesn't support shred-compaction-type other than the default rocksdb level compaction.
#### Summary of Changes
This PR enables ledger-tool to automatically detect the shred-compaction-type of the specified ledger.
#### Test Plan
New ledger-tool tests are added for both level and fifo compactions.
2022-07-19 01:19:17 +08:00
github-actions[bot]
fd5df1cf25
Bump Version to 1.11.4 ( #26578 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-11 23:30:38 -05:00
apfitzge
1c2f6a4c41
Bugfix: slots_connected snapshot_slot not full ( #26506 )
...
* slots_connected check used by ledger-tool should not require a full slot for snapshot slot
* Cleaner Result<Option<>> unwrap/default
* return false if no meta for starting slot
* Add clarifying comments
2022-07-11 13:45:12 -05:00
github-actions[bot]
9d937fb8a0
Bump Version to 1.11.3 ( #26481 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-07 14:39:46 -05:00
Jeff Washington (jwash)
26176af4aa
cleanup cli arg help ( #26366 )
2022-07-05 14:49:05 -05:00
Jeff Washington (jwash)
95ae82e074
add info to ledger tool for total execution time ( #26368 )
2022-07-05 09:23:06 -05:00
Nicholas Rempel
2b21c1da29
Sort Cargo.toml files
2022-06-24 12:41:38 -07:00
github-actions[bot]
5c2f819f99
Bump Version to 1.11.2 ( #26159 )
2022-06-22 21:16:18 -05:00
apfitzge
f4189c0305
ledger-tool minimized snapshots ( #25334 )
...
* working on local snapshot
* Parallelization for slot storage minimization
* Additional clean-up and fixes
* make --minimize an option of create-snapshot
* remove now unnecessary function
* Parallelize parts of minimized account set generation
* clippy fixes
* Add rent collection accounts and voting node_pubkeys
* Simplify programdata_accounts generation
* Loop over storages to get slot set
* Parallelize minimized slot set generation
* Parallelize adding owners and programdata_accounts
* Remove some now unncessary checks on the blockstore
* Add a warning for minimized snapshots across epoch boundary
* Simplify ledger-tool minimize
* Clarify names of bank's minimization helper functions
* Remove unnecesary funciton, fix line spacing
* Use DashSets instead of HashSets for minimized account and slot sets
* Filter storages uses all threads instead of thread_pool
* Add some additional comments on functions for minimization
* Moved more into bank and parallelized
* Update programs/bpf/Cargo.lock for dashmap in ledger
* Clippy fix
* ledger-tool: convert minimize_bank_for_snapshot Measure into measure!
* bank.rs: convert minimize_bank_for_snapshot Measure into measure!
* accounts_db.rs: convert minimize_accounts_db Measure into measure!
* accounts_db.rs: add comment about use of minimize_accounts_db
* ledger-tool: CLI argument clarification
* minimization functions: make infos unique
* bank.rs: Add test_get_rent_collection_accounts_between_slots
* bank.rs: Add test_minimization_add_vote_accounts
* bank.rs: Add test_minimization_add_stake_accounts
* bank.rs: Add test_minimization_add_owner_accounts
* bank.rs: Add test_minimization_add_programdata_accounts
* accounts_db.rs: Add test_minimize_accounts_db
* bank.rs: Add negative case and comments in test_get_rent_collection_accounts_between_slots
* bank.rs: Negative test in test_minimization_add_programdata_accounts
* use new static runtime and sdk ids
* bank comments to doc comments
* Only need to insert the maximum slot a key is found in
* rename remove_pubkeys to purge_pubkeys
* add comment on builtins::get_pubkeys
* prevent excessive logging of removed dead slots
* don't need to remove slot from shrink slot candidates
* blockstore.rs: get_accounts_used_in_range shouldn't return Result
* blockstore.rs: get_accounts_used_in_range: parallelize slot loop
* report filtering progress on time instead of count
* parallelize loop over snapshot storages
* WIP: move some bank minimization functionality into a new class
* WIP: move some accounts_db minimization functionality into SnapshotMinimizer
* WIP: Use new SnapshotMinimizer
* SnapshotMinimizer: fix use statements
* remove bank and accounts_db minimization code, where possible
* measure! doesn't take a closure
* fix use statement in blockstore
* log_dead_slots does not need pub(crate)
* get_unique_accounts_from_storages does not need pub(crate)
* different way to get stake accounts/nodes
* fix tests
* move rent collection account functionality to snapshot minimizer
* move accounts_db minimize behavior to snapshot minimizer
* clean up
* Use bank reference instead of Arc. Additional comments
* Add a comment to blockstore function
* Additional clarifying comments
* Moved all non-transaction account accumulation into the SnapshotMinimizer.
* transaction_account_set does not need to be mutable now
* Add comment about load_to_collect_rent_eagerly
* Update log_dead_slots comment
* remove duplicate measure/print of get_minimized_slot_set
2022-06-22 13:17:43 -04:00
Jeff Washington (jwash)
6513bb3b1b
add options to create snapshot ( #26095 )
2022-06-21 13:07:05 -05:00
Will Hickey
51f26dc96e
Bump version to 1.11.1 ( #26104 )
2022-06-21 12:07:46 -05:00
Tyera Eulberg
2866ca4b1c
Add ledger-tool bigtable upload loop ( #26030 )
...
* Add ledger-tool bigtable upload loop
* Limit range on caller side, switch to while loop, and remove now-obsolete option
2022-06-17 19:31:13 +00:00
Tyera Eulberg
a9c9acfe80
Improve repair-roots help ( #26008 )
2022-06-16 10:03:27 -06:00
Dan Coombs
32a58dd9e0
Add ability to use a non-default app profile id in bigtable requests ( #25968 )
...
* Add ability to use a non-default app profile id in bigtable requests
* Only run subcommand once when getting global configs
* Remove unneded scoping on option type
2022-06-16 00:58:16 +00:00
Brennan Watt
ba04063956
Add CPUmetrics ( #25802 )
...
Add in some CPU utilization metrics such as: number of vCPUs, clock frequency, average load across different time intervals, and number of total threads
2022-06-07 11:34:25 -07:00
apfitzge
934da5ef99
Fix pre-check of blockstore slots during load_bank_forks ( #25632 )
...
Fix pre-check of blockstore slts during load_bank_forks. Now iterates from starting_slot to halt_slot via slot_meta.next_slots to confirm they are connected.
2022-06-01 20:19:42 -05:00
Yueh-Hsuan Chiang
5b67960c76
(Refactor) Move blocktore options related stuff to blockstore_options.rs ( #25509 )
...
#### Problem
blockstore_db.rs has a mutual dependency between blockstore_metrics.rs.
#### Summary of Changes
This PR removes the mutual dependency by moving the option-related stuff
out from blockstore_db.rs to its new home --- blockstore_options.rs.
By doing this, we address the mutual dependency and also make the code cleaner.
2022-05-26 16:59:26 -07:00
apfitzge
328cca9fcd
Add a --debug-key option to ledger-tool verify ( #25391 )
2022-05-24 19:56:35 -05:00
Jeff Biseda
61c5a471e8
preserve optimistic_slot in blockstore ( #25311 )
2022-05-24 12:03:28 -07:00
kirill lykov
2cb500a388
fix problem with matches in ledger-tool ( #25482 )
2022-05-23 12:39:58 -05:00
Michael Vines
9d9773bd2a
Use write! instead of format! to pacify clippy
2022-05-22 22:22:21 -07:00
Michael Vines
b05c7d91ed
Fix derive_partial_eq_without_eq clippy lint
2022-05-22 22:22:21 -07:00
steviez
d65aab4d27
Fix ledger-tool slot to output tx even when tx status is missing ( #25434 )
2022-05-20 20:56:06 -06:00
Jeff Washington (jwash)
467431de89
add ledger-tool accounts-db-skip-initial-hash-calc ( #25354 )
2022-05-20 10:27:00 -05:00
apfitzge
1e7d877494
Create-snapshot check if slot is available on startup ( #25329 )
...
* Add a check during ledger-tool create-snapshot startup to see if the snapshot slot is available
* check all slots from the start to snapshot_slot during load_bank_forks
* unwrap_or_default incremental snapshot slot before comparison
* Improve error messages on missing or not full slots
2022-05-20 09:11:18 -05:00
sakridge
cdac141417
Sort per-program and print a summary of all slots in the range ( #25267 )
2022-05-19 23:26:29 +02:00
apfitzge
83b245642f
--snapshot-archive-format should not conflict w/ --no-snapshot ( #25368 )
2022-05-19 12:49:59 -05:00
dependabot[bot]
3b47250d4a
chore: bump signal-hook from 0.3.13 to 0.3.14 ( #25281 )
...
* chore: bump signal-hook from 0.3.13 to 0.3.14
Bumps [signal-hook](https://github.com/vorner/signal-hook ) from 0.3.13 to 0.3.14.
- [Release notes](https://github.com/vorner/signal-hook/releases )
- [Changelog](https://github.com/vorner/signal-hook/blob/master/CHANGELOG.md )
- [Commits](https://github.com/vorner/signal-hook/compare/v0.3.13...v0.3.14 )
---
updated-dependencies:
- dependency-name: signal-hook
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-18 09:28:52 -06:00
steviez
3d8e5e91ed
Fix ledger-tool from erroring with unspecified archive_format ( #25298 )
...
value_t_or_exit()! will error if the user doesn't specify a value at
runtime, use value_of() instead which will give either the default value
or whatever the user specified.
2022-05-17 15:46:31 -06:00
HaoranYi
3e44046a73
Support lz4 for snapshot archives ( #25089 )
...
* add lz4
* add cargo package file
* fix tests
* use fast lz4 compression
* report snapshot archive format
* add test
* code review feedback
* add cargo.lock
* fix var name
* refactor archive format parsing and add default compress cli arg
* clippy
* add from_cli_arg test
* update cargo.lock
* add lz4 support for mmap
* cargo.lock
* clippy
2022-05-16 12:44:15 -05:00
Tyera Eulberg
bc005e3408
Add configurable limit to number of blocks to check before Bigtable upload ( #24716 )
...
* Add ConfirmedBlockUploadConfig, no behavior changes
* Add comment
* A little DRY cleanup
* Add configurable limit to number of blocks to check in Blockstore and Bigtable before uploading
* Limit blockstore and bigtable look-ahead
* Exit iterator early when reach ending_slot
* Use rooted_slot_iterator instead of slot_meta_iterator
* Only check blocks in the ledger
2022-05-13 07:34:02 +00:00
Jeff Washington (jwash)
a8930ee14b
add --accounts-db-ancient-append-vecs ( #25125 )
2022-05-11 08:47:07 -05:00
DimAn
2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations ( #24247 )
2022-05-10 16:37:41 -04:00
apfitzge
b83fd8b958
ledger-tool: clear non-primary access accounts directory in load_bank_forks ( #24958 )
2022-05-09 13:17:13 -05:00
Justin Starry
082502d4f3
Fail tx sanitization when ix program id uses lookup table ( #25035 )
...
* Fail tx sanitization when ix program id uses lookup table
* feedback
2022-05-07 03:19:50 +08:00
Tyera Eulberg
c4aa082efc
Rollback tokio to LTS release v1.14 ( #25028 )
2022-05-06 06:34:49 +00:00
dependabot[bot]
bece7f32c8
chore: bump log from 0.4.16 to 0.4.17 ( #24987 )
...
* chore: bump log from 0.4.16 to 0.4.17
Bumps [log](https://github.com/rust-lang/log ) from 0.4.16 to 0.4.17.
- [Release notes](https://github.com/rust-lang/log/releases )
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/log/commits/0.4.17 )
---
updated-dependencies:
- dependency-name: log
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-05 01:49:18 +00:00
dependabot[bot]
61a9faae17
chore: bump serde_json from 1.0.80 to 1.0.81 ( #24960 )
...
* chore: bump serde_json from 1.0.80 to 1.0.81
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.80 to 1.0.81.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.80...v1.0.81 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2022-05-04 14:57:38 -06:00
dependabot[bot]
2c9d2a2140
chore: bump serde_json from 1.0.79 to 1.0.80 ( #24943 )
...
* chore: bump serde_json from 1.0.79 to 1.0.80
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.79 to 1.0.80.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.79...v1.0.80 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-03 13:46:45 -06:00
HaoranYi
554467ebd3
add no_os_memory_stats_reporting_arg ( #24938 )
2022-05-03 11:23:30 -05:00
DimAn
8846aa12ca
ledger-tool: save new snapshots to --snapshot-archive-path ( #24883 )
2022-05-02 11:00:15 -06:00
steviez
428cf54c91
Change BlockStore TryPrimaryThenSecondary to just Secondary ( #23391 )
2022-04-29 20:05:39 -05:00
Justin Starry
4e58b3870c
Update all BankForks methods to return owned values ( #24801 )
2022-04-28 18:51:00 +00:00
Ryo Onodera
412a5a0d33
Credits auto rewind on vote recreation ( #22546 )
...
* Credits auto rewind on vote recreation
* Update comment
* Improve comments and tests
* Recommended fn rename
* Restore old feature, and replace new feature key
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-04-26 15:49:35 -06:00
Michael Vines
533eadd024
Remove StakeState::get_rent_exempt_reserve()
2022-04-25 21:43:04 -07:00
Michael Vines
83e041299a
Run real snapshot packager while processing blockstore at validator startup
2022-04-22 21:17:49 -07:00
steviez
2a770e9726
Use unsigned_abs() instead of casting abs() ( #24521 )
2022-04-20 12:01:33 -05:00
Jeff Washington (jwash)
cfe2177e16
validator option: accounts-db-skip-rewrites ( #24504 )
2022-04-20 09:41:00 -05:00
Michael Vines
9e4999ef6a
Remove halt_at_slot from RuntimeConfig, it's not a runtime concern
2022-04-19 19:23:58 -07:00
Michael Vines
c6f3da4879
blockstore_processor now accepts an Arc<Rwlock<BankForks>>
2022-04-19 19:23:58 -07:00
Rachael Pai
aea17c35ae
Add a stringified credential option for LedgerStorage ( #24314 )
...
* add a stringified credential option for LedgerStorage
* fix clippy::useless-format warning
* change CredentialOption to enum CredentialType
* rename credential_option to credential_type
* restore LedgerStorage new fn signature
* fmt
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-04-13 14:35:06 -06:00
Jeff Washington (jwash)
1bc49d219d
IndexLimitMb option adds 'Unspecified' state ( #24249 )
2022-04-12 09:38:09 -05:00
Jon Cinque
9b8850f99e
test-validator: Add `--max-compute-units` flag ( #24130 )
...
* test-validator: Add `--max-compute-units` flag
* Add `RuntimeConfig` for tweaking runtime behavior
* Actually add the file
* Move RuntimeConfig to runtime
2022-04-12 02:28:10 +02:00
Giorgio Gambino
60b2155bd3
Add accounts-filler-size command line option ( #23896 )
2022-04-11 13:10:09 -05:00
Jeff Washington (jwash)
64abd008ca
make ledger-tool arg help consistent ( #24203 )
2022-04-08 15:45:09 -05:00
Christian Kamm
a058f348a2
Address review comments
2022-04-08 14:37:55 -05:00
Brooks Prumo
c322842257
Replace channel with Mutex<Option> for AccountsPackage ( #24013 )
2022-04-06 05:47:19 -05:00
steviez
b61c0a4a21
Add accounts arg to genesis command to dump genesis account info ( #23879 )
2022-03-24 14:26:08 -05:00
Will Hickey
c4ecfa5716
Bump version to v1.11 ( #23807 )
...
* Revert crossbeam_epoch to stable. 0.9.8 only works with nightly
* Remove unneeded unit expression
2022-03-21 17:40:50 -05:00
Trent Nelson
9b32b72990
bigtable: allow custom instance names
2022-03-21 11:26:49 -06:00
Trent Nelson
f513195468
bigtable: add a config ctor for `LedgerStorage`
2022-03-21 11:26:49 -06:00
Yueh-Hsuan Chiang
f999eef452
(LedgerStore) Rename BlockstoreAdvancedOptions to LedgerColumnOptions ( #23764 )
...
This PR renames BlockstoreAdvancedOptions to LedgerColumnOptions, as we will
pass-down this struct to LedgerColumn to allow it to perform metric reporting.
2022-03-18 11:13:35 -07:00
Will Hickey
2f58c9e501
Bump version to 1.10.4 ( #23743 )
2022-03-17 14:02:13 -05:00
Justin Starry
0eccacbd5b
Add CLI support for versioned transactions ( #23606 )
2022-03-17 11:43:04 +08:00
Michael Vines
2da4e3eb6c
Add --no-os-memory-stats-reporting
2022-03-15 17:07:40 -07:00
steviez
e7cf84a593
Remove AccessType arg from create_new_ledger ( #23591 )
...
Creating a new ledger implicitly means that no other process could have
previously held access to it. Additionally, creating a new ledger
implicitly requires writing, so it follows that Primary access is
required and we can drop access type as an argument.
2022-03-15 00:07:16 -05:00
Justin Starry
8c8f9694e0
Refactor: Sanitized transaction creation ( #23558 )
...
* Refactor: SanitizedTransaction::try_create optionally computes hash
* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
Michael Vines
115f376465
Factor out bank_forks_utils::load_bank_forks()
2022-03-14 15:29:58 -07:00
Will Hickey
63bf0f66af
Bump version to 1.10.3 ( #23648 )
2022-03-14 11:18:45 -05:00
Michael Vines
01b6f97f0b
Pin histogram crate version to fix cargo publish
2022-03-13 16:19:30 -07:00
Yueh-Hsuan Chiang
1e20bd8f9a
(LedgerStore) Include storage type as a tag in RocksDB metric reporting ( #23523 )
...
#### Summary of Changes
This PR further enables group by operation on storage type in blockstore_rocksdb_cfs metrics.
Such group-by allows us to further compare the performance metrics between rocks-level and
rocks-fifo.
To make things extensible, this PR introduces BlockstoreAdvancedOptions and move shred_storage_type.
All fields in BlockstoreAdvancedOptions will support group-by operation in blockstore_rocksdb_cfs.
Dependency: #23580
2022-03-11 15:17:34 -08:00
Brooks Prumo
d20dd21600
Sort tables in Cargo.toml files ( #23602 )
2022-03-11 11:05:57 -06:00
sakridge
9b591286d7
Revert "chore: bump dashmap from 4.0.2 to 5.1.0 ( #23372 )" ( #23592 )
...
This reverts commit 3a0271c113
.
2022-03-11 09:12:27 +01:00
Justin Starry
021135978d
Refactor: Split up cli transaction display methods ( #23547 )
2022-03-11 10:49:53 +08:00
Will Hickey
b444836a97
Bump version to 1.10.2 ( #23597 )
2022-03-10 16:41:06 -06:00
Justin Starry
3114c199bd
Add RPC support for versioned transactions ( #22530 )
...
* Add RPC support for versioned transactions
* fix doc tests
* Add rpc test for versioned txs
* Switch to preflight bank
2022-03-08 15:20:34 +08:00
dependabot[bot]
3a0271c113
chore: bump dashmap from 4.0.2 to 5.1.0 ( #23372 )
...
* chore: bump dashmap from 4.0.2 to 5.1.0
Bumps [dashmap](https://github.com/xacrimon/dashmap ) from 4.0.2 to 5.1.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases )
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.1.0 )
---
updated-dependencies:
- dependency-name: dashmap
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-03-07 00:00:26 -05:00
dependabot[bot]
61d7bdd66f
chore: bump serde_json from 1.0.78 to 1.0.79 ( #23461 )
...
* chore: bump serde_json from 1.0.78 to 1.0.79
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.78...v1.0.79 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2022-03-02 23:38:06 -07:00
Yueh-Hsuan Chiang
634f4eb37d
(LedgerStore) Use different path for different blockstore storage type. ( #23236 )
...
#### Summary of Changes
To avoid mixing the use of different shred storage types, each shred storage type
will have its blockstore in a different directory.
This PR still keeps the RocksFifo setting hidden. The default ShredStorageType and
blockstore directory are still RocksLevel and `rocksdb`.
Will follow-up with PRs on making FIFO option public in ledger-tool and validator.
#### Test Plan
* Added a new test to verify the existence of `rocksdb-fifo` directory when FIFO compaction is used.
* Updated existing test to verify the current setting still store ledger under `rocksdb` directory.
* Manually ran ledger_cleanup_test with both level and fifo compaction and verified the resulting ledger.
* Ran a validator with this PR.
2022-03-02 18:30:22 -08:00
Will Hickey
1a99251498
Bump version to 1.10.1 ( #23453 )
2022-03-02 13:47:01 -06:00
Jeff Washington (jwash)
4bc440666a
add arg --disable_accounts_disk_index ( #23308 )
2022-02-23 18:07:24 -06:00
Yueh-Hsuan Chiang
9213fcb11b
Introduce experimental FIFO-compaction option for shreds in blockstore ( #22140 )
2022-02-10 11:34:03 -08:00
Justin Starry
d5dec989b9
Enforce tx metadata upload with static types ( #23028 )
2022-02-10 13:28:18 +08:00
Justin Starry
60af1a4cce
Refactor: Add trait for loading addresses ( #22903 )
2022-02-03 11:00:12 +00:00
dependabot[bot]
db481e1799
chore: bump serde_json from 1.0.75 to 1.0.78 ( #22748 )
...
* chore: bump serde_json from 1.0.75 to 1.0.78
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.75 to 1.0.78.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.75...v1.0.78 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-26 18:01:02 -07:00
dependabot[bot]
95bbb70c91
chore: bump serde_json from 1.0.74 to 1.0.75 ( #22541 )
...
* chore: bump serde_json from 1.0.74 to 1.0.75
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.74 to 1.0.75.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.74...v1.0.75 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-21 12:26:51 -07:00
Justin Starry
f804ccdece
Store address table lookups in blockstore and bigtable ( #22402 )
2022-01-14 15:24:41 +08:00
Jeff Washington (jwash)
e291342c4a
Revert "chore: bump dashmap from 4.0.2 to 5.0.0 ( #21824 )" ( #22488 )
...
This reverts commit 8aa3d690b5
.
2022-01-13 13:06:39 -06:00
Jeff Biseda
8b66625c95
convert std::sync::mpsc to crossbeam_channel ( #22264 )
2022-01-11 02:44:46 -08:00
Yueh-Hsuan Chiang
5771c36d3f
Rename open_with_access_type() to open_with_options() ( #22123 )
2022-01-07 12:11:43 -08:00
steviez
9f1f64e384
Cleanup ledger-tool analyze-storage command ( #22310 )
...
* Make ledger-tool analyze-storage use Blockstore::open()
Opening a large ledger may require setting a larger open file descriptor
limit. Blockstore::open() does this whereas the underlying Database
object that analyze-storage was opening does not.
* Move key_size call lookup to take advantage of traits
* Fix typo where analyze worked on wrong column
* Make analyze-storage analyze all columns
2022-01-06 23:40:02 -06:00
pieceofr
d9220652ad
[ledger-tool]compare_blocks ( #22229 )
...
* 1.made load_credentials accept credential path as a parameter. 2.partial implement bigtable comparasion function
* finding missing blocks in bigtables in a specified range
* refactor compare-blocks,add unit test for missing_blocks and fmt
* compare-block fix last block bug
* refactor compare-block and improve wording
* Update ledger-tool/src/bigtable.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* update compare-block command-line description
* style:improve wording/naming/code style
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-01-06 14:36:03 +08:00
Yueh-Hsuan Chiang
e8b7f96a89
Add struct BlockstoreOptions ( #22121 )
2022-01-03 18:30:45 -10:00
dependabot[bot]
ecbfc70bfa
Bump serde_json from 1.0.73 to 1.0.74 ( #22231 )
...
* Bump serde_json from 1.0.73 to 1.0.74
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.73 to 1.0.74.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.73...v1.0.74 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-03 10:36:59 -07:00
Justin Starry
2a00382d71
Refactor: cleanup solana_transaction_status crate ( #22230 )
2022-01-03 15:45:18 +00:00
behzad nouri
f67ecd5c18
removes unused Cargo dependencies ( #22022 )
...
Also moving some to [dev-dependencies] so that they are not propagated
to other packages which depend on the package.
2021-12-22 17:01:33 +00:00
dependabot[bot]
3356c6afb2
chore: bump signal-hook from 0.3.12 to 0.3.13 ( #22037 )
...
Bumps [signal-hook](https://github.com/vorner/signal-hook ) from 0.3.12 to 0.3.13.
- [Release notes](https://github.com/vorner/signal-hook/releases )
- [Changelog](https://github.com/vorner/signal-hook/blob/master/CHANGELOG.md )
- [Commits](https://github.com/vorner/signal-hook/compare/v0.3.12...v0.3.13 )
---
updated-dependencies:
- dependency-name: signal-hook
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 21:40:16 -07:00
Justin Starry
6ff0be6a82
Clean up demote program write lock feature ( #21949 )
...
* Clean up demote program write lock feature
* fix test
2021-12-16 17:27:22 -05:00
behzad nouri
8d980f07ba
uses Option<Slot> for SlotMeta.parent_slot ( #21808 )
...
SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205
The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-14 18:57:11 +00:00
dependabot[bot]
426f2507d5
chore: bump serde_yaml from 0.8.21 to 0.8.23 ( #21867 )
...
* chore: bump serde_yaml from 0.8.21 to 0.8.23
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml ) from 0.8.21 to 0.8.23.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases )
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.21...0.8.23 )
---
updated-dependencies:
- dependency-name: serde_yaml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-14 10:23:26 -07:00
dependabot[bot]
018b54dbd7
chore: bump serde_json from 1.0.72 to 1.0.73 ( #21856 )
...
* chore: bump serde_json from 1.0.72 to 1.0.73
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.72...v1.0.73 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-14 04:08:22 +00:00
dependabot[bot]
8aa3d690b5
chore: bump dashmap from 4.0.2 to 5.0.0 ( #21824 )
...
* chore: bump dashmap from 4.0.2 to 5.0.0
Bumps [dashmap](https://github.com/xacrimon/dashmap ) from 4.0.2 to 5.0.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases )
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.0.0 )
---
updated-dependencies:
- dependency-name: dashmap
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-13 13:23:58 -07:00
dependabot[bot]
c965517a6b
chore: bump signal-hook from 0.3.10 to 0.3.12 ( #21721 )
...
Bumps [signal-hook](https://github.com/vorner/signal-hook ) from 0.3.10 to 0.3.12.
- [Release notes](https://github.com/vorner/signal-hook/releases )
- [Changelog](https://github.com/vorner/signal-hook/blob/master/CHANGELOG.md )
- [Commits](https://github.com/vorner/signal-hook/compare/v0.3.10...v0.3.12 )
---
updated-dependencies:
- dependency-name: signal-hook
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-09 21:43:31 +00:00
steviez
94b1cf47ca
Plumb verbose_level through ledger-tool slot subcommand ( #21670 )
...
* Add commas for readability and fix plurality in a printout
* Pass verbose_level into slot subcommand instead of default to max
This is useful for when a full printout isn't necessary, such as just
trying to determine the slot blockhash. The equivalent behavior before
this change would have been using "-vv" with the command.
2021-12-07 15:14:39 -06:00
dependabot[bot]
f6801a4af4
chore: bump itertools from 0.10.1 to 0.10.3 ( #21643 )
...
* chore: bump itertools from 0.10.1 to 0.10.3
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.10.1 to 0.10.3.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.1...v0.10.3 )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-06 23:23:26 -07:00
Trent Nelson
d6f22433d0
Bump version to v1.10.0
2021-12-04 20:17:54 +00:00
Michael Vines
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
Tyera Eulberg
2b269dbe0e
Ledger-tool: only require ledger dir when necessary ( #21575 )
...
* Don't canonicalize ledger_path unless ledger_path will be used
* Single use statement
2021-12-02 17:23:51 -07:00
Michael Vines
dd12d90eac
Upgrade to Rust 2021
2021-11-30 20:43:46 -08:00
dependabot[bot]
a125e7b8c2
chore: bump serde_json from 1.0.71 to 1.0.72 ( #21426 )
...
* chore: bump serde_json from 1.0.71 to 1.0.72
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-30 18:03:09 +00:00
Jeff Washington (jwash)
b1b334a17f
ledger-tool uses jemalloc like validator ( #21351 )
2021-11-19 09:48:07 -06:00
Jeff Washington (jwash)
79d21d6805
add --accounts-index-scan-results-limit-mb to allow scans to abort ( #21327 )
...
* ScanConfig -> &ScanConfig
* add --accounts-index-scan-results-limit-mb to allow scans to abort
* feedback
2021-11-19 09:00:19 -06:00
dependabot[bot]
89102540b1
chore: bump serde_json from 1.0.70 to 1.0.71 ( #21338 )
...
* chore: bump serde_json from 1.0.70 to 1.0.71
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.70...v1.0.71 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-18 02:58:30 +00:00
Jeff Washington (jwash)
0f69a14247
Add ability to abort scan ( #21314 )
2021-11-17 13:10:29 -06:00
Jeff Biseda
d5de0c8e12
add --no-os-network-stats-reporting option ( #21296 )
2021-11-16 10:26:03 -08:00
dependabot[bot]
55ea672041
chore: bump serde_json from 1.0.69 to 1.0.70 ( #21285 )
...
* chore: bump serde_json from 1.0.69 to 1.0.70
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.69...v1.0.70 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-11-15 15:27:47 -07:00
Jeff Washington (jwash)
77a59ca3e3
ledger-tool verify supports --accounts-db-skip-shrink ( #21284 )
2021-11-15 14:42:56 -06:00
Jeff Washington (jwash)
a043b19988
ledger tool verify launches system monitor to report mem ( #21265 )
2021-11-15 09:29:30 -06:00
Brooks Prumo
8b2133b078
Refactor total stats calculation into shared fn ( #21212 )
2021-11-09 19:13:37 -06:00
dependabot[bot]
3c5f505d3e
chore: bump serde_json from 1.0.68 to 1.0.69 ( #21211 )
...
* chore: bump serde_json from 1.0.68 to 1.0.69
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.68...v1.0.69 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-09 11:04:40 -07:00
Brooks Prumo
6f3c27f1da
Add total accounts stats to ledger-tool accounts ( #21187 )
2021-11-08 14:45:30 -06:00
Brooks Prumo
2031522809
Add flags to not print account contents ( #21178 )
...
It takes a long time to print the contents of all the accounts. I'll be
adding more to the "accounts" subcommand and would like to skip printing
the account contents.
2021-11-05 08:59:12 -05:00
Justin Starry
140a5f633d
Simplify replay vote tracking by using packet metadata ( #21112 )
2021-11-03 09:02:48 +00:00
dependabot[bot]
855b3ccdc0
chore: bump signal-hook from 0.2.3 to 0.3.10 ( #21081 )
...
* chore: bump signal-hook from 0.2.3 to 0.3.10
Bumps [signal-hook](https://github.com/vorner/signal-hook ) from 0.2.3 to 0.3.10.
- [Release notes](https://github.com/vorner/signal-hook/releases )
- [Changelog](https://github.com/vorner/signal-hook/blob/master/CHANGELOG.md )
- [Commits](https://github.com/vorner/signal-hook/compare/v0.2.3...v0.3.10 )
---
updated-dependencies:
- dependency-name: signal-hook
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update apis
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-10-30 00:21:54 -06:00
Jeff Washington (jwash)
43ea579f63
add cli for --accounts-hash-num-passes ( #20827 )
2021-10-25 09:45:46 -05:00
Tao Zhu
c2bfce90b3
- cost_tracker is data member of a bank, it can report metrics when bank is frozen ( #20802 )
...
- removed cost_tracker_stats and histogram
- move stats reporting outside of bank freeze
2021-10-24 22:19:23 -05:00
Michael Vines
350bb561eb
Clippy
2021-10-23 08:21:20 +00:00
Brooks Prumo
50571685d4
Take incremental snapshots with ledger-tool ( #20857 )
2021-10-21 21:02:12 -05:00
carllin
b5f21d5e34
Add ledger tool to remove dead slot ( #20810 )
2021-10-19 20:23:16 -07:00
Tao Zhu
7496b5784b
- make cost_tracker a member of bank, remove shared instance from TPU; ( #20627 )
...
- decouple cost_model from cost_tracker; allowing one cost_model
instance being shared within a validator;
- update cost_model api to calculate_cost(&self...)->transaction_cost
2021-10-19 14:37:33 -05:00
sakridge
3b40fa3f44
Add is_full to ledger-tool ( #20183 )
2021-10-13 12:50:18 +02:00
Tao Zhu
005d6863fd
- move cost tracker into bank, so each bank has its own cost tracker; ( #20527 )
...
- move related modules to runtime
2021-10-12 08:51:33 -05:00
Jeff Washington (jwash)
a8e000a2a6
add filler accounts to bloat validator and predict failure ( #20491 )
...
* add filler accounts to bloat validator and predict failure
* assert no accounts match filler
* cleanup magic numbers
* panic if can't load from snapshot with filler accounts specified
* some renames
* renames
* into_par_iter
* clean filler accts, too
2021-10-11 12:46:27 -05:00
Jeff Washington (jwash)
6cd1a03bb5
AcctIdx: fix ledger-tool arg options ( #20513 )
2021-10-07 19:43:21 +00:00
Trent Nelson
767f740305
Bump version to 1.9.0
2021-10-06 17:57:41 -07:00
Michael Vines
7027d56064
Resolve nightly-2021-10-05 clippy complaints
2021-10-06 10:37:58 -07:00
Tao Zhu
03913f6661
add tx count and thread id to stats, each stat reports and resets when slot changes ( #20451 )
2021-10-06 00:09:19 -05:00
Justin Starry
250a8503fe
Make rewards tracer async friendly ( #20452 )
2021-10-05 21:30:08 -04:00