Commit Graph

478 Commits

Author SHA1 Message Date
Justin Starry 0eccacbd5b
Add CLI support for versioned transactions (#23606) 2022-03-17 11:43:04 +08:00
Michael Vines 2da4e3eb6c Add --no-os-memory-stats-reporting 2022-03-15 17:07:40 -07:00
steviez e7cf84a593
Remove AccessType arg from create_new_ledger (#23591)
Creating a new ledger implicitly means that no other process could have
previously held access to it. Additionally, creating a new ledger
implicitly requires writing, so it follows that Primary access is
required and we can drop access type as an argument.
2022-03-15 00:07:16 -05:00
Justin Starry 8c8f9694e0
Refactor: Sanitized transaction creation (#23558)
* Refactor: SanitizedTransaction::try_create optionally computes hash

* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
Michael Vines 115f376465 Factor out bank_forks_utils::load_bank_forks() 2022-03-14 15:29:58 -07:00
Will Hickey 63bf0f66af
Bump version to 1.10.3 (#23648) 2022-03-14 11:18:45 -05:00
Michael Vines 01b6f97f0b Pin histogram crate version to fix cargo publish 2022-03-13 16:19:30 -07:00
Yueh-Hsuan Chiang 1e20bd8f9a
(LedgerStore) Include storage type as a tag in RocksDB metric reporting (#23523)
#### Summary of Changes
This PR further enables group by operation on storage type in blockstore_rocksdb_cfs metrics.
Such group-by allows us to further compare the performance metrics between rocks-level and
rocks-fifo.

To make things extensible, this PR introduces BlockstoreAdvancedOptions and move shred_storage_type. 
All fields in BlockstoreAdvancedOptions will support group-by operation in blockstore_rocksdb_cfs.

Dependency: #23580
2022-03-11 15:17:34 -08:00
Brooks Prumo d20dd21600
Sort tables in Cargo.toml files (#23602) 2022-03-11 11:05:57 -06:00
sakridge 9b591286d7
Revert "chore: bump dashmap from 4.0.2 to 5.1.0 (#23372)" (#23592)
This reverts commit 3a0271c113.
2022-03-11 09:12:27 +01:00
Justin Starry 021135978d
Refactor: Split up cli transaction display methods (#23547) 2022-03-11 10:49:53 +08:00
Will Hickey b444836a97
Bump version to 1.10.2 (#23597) 2022-03-10 16:41:06 -06:00
Justin Starry 3114c199bd
Add RPC support for versioned transactions (#22530)
* Add RPC support for versioned transactions

* fix doc tests

* Add rpc test for versioned txs

* Switch to preflight bank
2022-03-08 15:20:34 +08:00
dependabot[bot] 3a0271c113
chore: bump dashmap from 4.0.2 to 5.1.0 (#23372)
* chore: bump dashmap from 4.0.2 to 5.1.0

Bumps [dashmap](https://github.com/xacrimon/dashmap) from 4.0.2 to 5.1.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases)
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.1.0)

---
updated-dependencies:
- dependency-name: dashmap
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-03-07 00:00:26 -05:00
dependabot[bot] 61d7bdd66f
chore: bump serde_json from 1.0.78 to 1.0.79 (#23461)
* chore: bump serde_json from 1.0.78 to 1.0.79

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.78...v1.0.79)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2022-03-02 23:38:06 -07:00
Yueh-Hsuan Chiang 634f4eb37d
(LedgerStore) Use different path for different blockstore storage type. (#23236)
#### Summary of Changes
To avoid mixing the use of different shred storage types, each shred storage type
will have its blockstore in a different directory.

This PR still keeps the RocksFifo setting hidden.  The default ShredStorageType and
blockstore directory are still RocksLevel and `rocksdb`.

Will follow-up with PRs on making FIFO option public in ledger-tool and validator.

#### Test Plan
* Added a new test to verify the existence of `rocksdb-fifo` directory when FIFO compaction is used.
* Updated existing test to verify the current setting still store ledger under `rocksdb` directory.
* Manually ran ledger_cleanup_test with both level and fifo compaction and verified the resulting ledger.
* Ran a validator with this PR.
2022-03-02 18:30:22 -08:00
Will Hickey 1a99251498
Bump version to 1.10.1 (#23453) 2022-03-02 13:47:01 -06:00
Jeff Washington (jwash) 4bc440666a
add arg --disable_accounts_disk_index (#23308) 2022-02-23 18:07:24 -06:00
Yueh-Hsuan Chiang 9213fcb11b
Introduce experimental FIFO-compaction option for shreds in blockstore (#22140) 2022-02-10 11:34:03 -08:00
Justin Starry d5dec989b9
Enforce tx metadata upload with static types (#23028) 2022-02-10 13:28:18 +08:00
Justin Starry 60af1a4cce
Refactor: Add trait for loading addresses (#22903) 2022-02-03 11:00:12 +00:00
dependabot[bot] db481e1799
chore: bump serde_json from 1.0.75 to 1.0.78 (#22748)
* chore: bump serde_json from 1.0.75 to 1.0.78

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.75 to 1.0.78.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.75...v1.0.78)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-26 18:01:02 -07:00
dependabot[bot] 95bbb70c91
chore: bump serde_json from 1.0.74 to 1.0.75 (#22541)
* chore: bump serde_json from 1.0.74 to 1.0.75

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.74 to 1.0.75.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.74...v1.0.75)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-21 12:26:51 -07:00
Justin Starry f804ccdece
Store address table lookups in blockstore and bigtable (#22402) 2022-01-14 15:24:41 +08:00
Jeff Washington (jwash) e291342c4a
Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)" (#22488)
This reverts commit 8aa3d690b5.
2022-01-13 13:06:39 -06:00
Jeff Biseda 8b66625c95
convert std::sync::mpsc to crossbeam_channel (#22264) 2022-01-11 02:44:46 -08:00
Yueh-Hsuan Chiang 5771c36d3f
Rename open_with_access_type() to open_with_options() (#22123) 2022-01-07 12:11:43 -08:00
steviez 9f1f64e384
Cleanup ledger-tool analyze-storage command (#22310)
* Make ledger-tool analyze-storage use Blockstore::open()

Opening a large ledger may require setting a larger open file descriptor
limit. Blockstore::open() does this whereas the underlying Database
object that analyze-storage was opening does not.

* Move key_size call lookup to take advantage of traits

* Fix typo where analyze worked on wrong column

* Make analyze-storage analyze all columns
2022-01-06 23:40:02 -06:00
pieceofr d9220652ad
[ledger-tool]compare_blocks (#22229)
* 1.made load_credentials accept credential path as a parameter. 2.partial implement bigtable comparasion function

* finding missing blocks in bigtables in a specified range

* refactor compare-blocks,add unit test for missing_blocks and fmt

* compare-block fix last block bug

* refactor compare-block and improve wording

* Update ledger-tool/src/bigtable.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* update compare-block command-line description

* style:improve wording/naming/code style

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-01-06 14:36:03 +08:00
Yueh-Hsuan Chiang e8b7f96a89
Add struct BlockstoreOptions (#22121) 2022-01-03 18:30:45 -10:00
dependabot[bot] ecbfc70bfa
Bump serde_json from 1.0.73 to 1.0.74 (#22231)
* Bump serde_json from 1.0.73 to 1.0.74

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.73 to 1.0.74.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.73...v1.0.74)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-03 10:36:59 -07:00
Justin Starry 2a00382d71
Refactor: cleanup solana_transaction_status crate (#22230) 2022-01-03 15:45:18 +00:00
behzad nouri f67ecd5c18
removes unused Cargo dependencies (#22022)
Also moving some to [dev-dependencies] so that they are not propagated
to other packages which depend on the package.
2021-12-22 17:01:33 +00:00
dependabot[bot] 3356c6afb2
chore: bump signal-hook from 0.3.12 to 0.3.13 (#22037)
Bumps [signal-hook](https://github.com/vorner/signal-hook) from 0.3.12 to 0.3.13.
- [Release notes](https://github.com/vorner/signal-hook/releases)
- [Changelog](https://github.com/vorner/signal-hook/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/signal-hook/compare/v0.3.12...v0.3.13)

---
updated-dependencies:
- dependency-name: signal-hook
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 21:40:16 -07:00
Justin Starry 6ff0be6a82
Clean up demote program write lock feature (#21949)
* Clean up demote program write lock feature

* fix test
2021-12-16 17:27:22 -05:00
behzad nouri 8d980f07ba
uses Option<Slot> for SlotMeta.parent_slot (#21808)
SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-14 18:57:11 +00:00
dependabot[bot] 426f2507d5
chore: bump serde_yaml from 0.8.21 to 0.8.23 (#21867)
* chore: bump serde_yaml from 0.8.21 to 0.8.23

Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.21 to 0.8.23.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.21...0.8.23)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-14 10:23:26 -07:00
dependabot[bot] 018b54dbd7
chore: bump serde_json from 1.0.72 to 1.0.73 (#21856)
* chore: bump serde_json from 1.0.72 to 1.0.73

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.72...v1.0.73)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-14 04:08:22 +00:00
dependabot[bot] 8aa3d690b5
chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)
* chore: bump dashmap from 4.0.2 to 5.0.0

Bumps [dashmap](https://github.com/xacrimon/dashmap) from 4.0.2 to 5.0.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases)
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.0.0)

---
updated-dependencies:
- dependency-name: dashmap
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-13 13:23:58 -07:00
dependabot[bot] c965517a6b
chore: bump signal-hook from 0.3.10 to 0.3.12 (#21721)
Bumps [signal-hook](https://github.com/vorner/signal-hook) from 0.3.10 to 0.3.12.
- [Release notes](https://github.com/vorner/signal-hook/releases)
- [Changelog](https://github.com/vorner/signal-hook/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/signal-hook/compare/v0.3.10...v0.3.12)

---
updated-dependencies:
- dependency-name: signal-hook
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-09 21:43:31 +00:00
steviez 94b1cf47ca
Plumb verbose_level through ledger-tool slot subcommand (#21670)
* Add commas for readability and fix plurality in a printout
* Pass verbose_level into slot subcommand instead of default to max

This is useful for when a full printout isn't necessary, such as just
trying to determine the slot blockhash. The equivalent behavior before
this change would have been using "-vv" with the command.
2021-12-07 15:14:39 -06:00
dependabot[bot] f6801a4af4
chore: bump itertools from 0.10.1 to 0.10.3 (#21643)
* chore: bump itertools from 0.10.1 to 0.10.3

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.1 to 0.10.3.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.1...v0.10.3)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-06 23:23:26 -07:00
Trent Nelson d6f22433d0 Bump version to v1.10.0 2021-12-04 20:17:54 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Tyera Eulberg 2b269dbe0e
Ledger-tool: only require ledger dir when necessary (#21575)
* Don't canonicalize ledger_path unless ledger_path will be used

* Single use statement
2021-12-02 17:23:51 -07:00
Michael Vines dd12d90eac Upgrade to Rust 2021 2021-11-30 20:43:46 -08:00
dependabot[bot] a125e7b8c2
chore: bump serde_json from 1.0.71 to 1.0.72 (#21426)
* chore: bump serde_json from 1.0.71 to 1.0.72

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-30 18:03:09 +00:00
Jeff Washington (jwash) b1b334a17f
ledger-tool uses jemalloc like validator (#21351) 2021-11-19 09:48:07 -06:00
Jeff Washington (jwash) 79d21d6805
add --accounts-index-scan-results-limit-mb to allow scans to abort (#21327)
* ScanConfig -> &ScanConfig

* add --accounts-index-scan-results-limit-mb to allow scans to abort

* feedback
2021-11-19 09:00:19 -06:00
dependabot[bot] 89102540b1
chore: bump serde_json from 1.0.70 to 1.0.71 (#21338)
* chore: bump serde_json from 1.0.70 to 1.0.71

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.70...v1.0.71)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-18 02:58:30 +00:00