Commit Graph

457 Commits

Author SHA1 Message Date
dependabot[bot] b97113408b
chore: bump futures-util from 0.3.15 to 0.3.16 (#18878)
* chore: bump futures-util from 0.3.15 to 0.3.16

Bumps [futures-util](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Nudge Cargo.lock

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-26 18:19:53 +00:00
Brooks Prumo d1debcd971
Add incremental snapshot utils (#18504)
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks.  This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.

Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.

Also of note, some renaming has happened:
  1. Snapshots are now either `full_` or `incremental_` throughout the
     codebase.  If not specified, the code applies to both.
  2. Bank snapshots now are called "bank snapshots"
     (before they were called "slot snapshots", "bank snapshots", or
      just "snapshots").  The one exception is within `Bank`, where they
     are still just "snapshots", because they are already "bank
     snapshots".
  3. Snapshot archives now have `_archive` in the code.  This
     should clear up an ambiguity between bank snapshots and snapshot
     archives.
2021-07-22 14:40:37 -05:00
Justin Starry d166b9856a
Move transaction sanitization earlier in the pipeline (#18655)
* Move transaction sanitization earlier in the pipeline

* Renamed HashedTransaction to SanitizedTransaction

* Implement deref for sanitized transaction

* bring back process_transactions test method

* Use sanitized transactions for cost model calculation
2021-07-15 22:51:27 -05:00
Trent Nelson 3a85b77bb5 hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade 2021-07-15 18:43:55 +00:00
Trent Nelson 568660b402 Revert "Remove feature switch for secp256k1 program (#18467)"
This reverts commit fd574dcb3b.
2021-07-15 18:43:55 +00:00
sakridge 7f2254225e
Move entry/poh to own crate to speed up poh bench build (#18225) 2021-07-14 14:16:29 +02:00
Tao Zhu 350baece21
Explicitly sanitize program id indexes before usage
1. check transaction has valid program_id before using it to avoid possible panic;
2. change calculate_cost function signature to return Result;
3. add CostModelError enum, update return type from Result<_, str> to Result<_, CostModelError>
2021-07-13 17:29:22 -05:00
Jeff Washington (jwash) d092fa1f03
add ledger-tool verify verify-accounts-index option (#18375)
* add ledger-tool verify verify-accounts-index option

* comment, merge, respond to feedback, cleanup
2021-07-13 11:06:18 -05:00
Michael Vines 4098af3b5b Record vote account commission with voting/staking rewards and surface in RPC 2021-07-12 15:09:44 -07:00
Justin Starry fd574dcb3b
Remove feature switch for secp256k1 program (#18467)
* Remove feature switch for secp256k1 program

* fix tests
2021-07-09 10:08:03 -05:00
Tao Zhu b6dff12923
update ledger tool to restore cost table from blockstore (#18489)
* update ledger tool to restore cost model from blockstore when compute-slot-cost

* Move initialize_cost_table into cost_model, so the function can be tested and shared between validator and ledger-tool

* refactor and simplify a test
2021-07-07 23:44:51 -05:00
Tao Zhu 0e039b4094
Aggregate cost_model into cost_tracker (#18374)
* * aggregate cost_model into cost_tracker, decouple it from banking_stage to prevent accidental deadlock. * Simplified code, removed unused functions

* review fixes
2021-07-06 15:41:25 +00:00
carllin 0eca92de18
Make set roots an iterator (#18357) 2021-07-01 20:02:40 -07:00
Brooks Prumo 89a3e4f91e
Move SnapshotConfig into its own module (#18331)
Also move ArchiveFormat to snapshot_utils, and do not
reexport SnapshotVersion.
2021-07-01 08:55:26 -05:00
dependabot[bot] 78968d132f
chore: bump log from 0.4.11 to 0.4.14 (#18323)
* chore: bump log from 0.4.11 to 0.4.14

Bumps [log](https://github.com/rust-lang/log) from 0.4.11 to 0.4.14.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.11...0.4.14)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make version consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-30 18:32:01 +00:00
dependabot[bot] ff8aef9d27
chore: bump csv from 1.1.3 to 1.1.6 (#18261)
Bumps [csv](https://github.com/BurntSushi/rust-csv) from 1.1.3 to 1.1.6.
- [Release notes](https://github.com/BurntSushi/rust-csv/releases)
- [Commits](https://github.com/BurntSushi/rust-csv/compare/1.1.3...1.1.6)

---
updated-dependencies:
- dependency-name: csv
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-28 09:56:42 -06:00
dependabot[bot] d923ff9cfb
chore: bump serde_yaml from 0.8.13 to 0.8.17 (#18169)
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.13 to 0.8.17.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.13...0.8.17)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-23 09:44:34 -06:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Jon Cinque 1b1d34da59
Refactor stake program into solana_program (#17906)
* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt
2021-06-15 18:04:00 +02:00
carllin ccc013e134
Handle removing slots during account scans (#17471) 2021-06-14 21:04:01 -07:00
dependabot[bot] a0872232d3
chore: bump itertools from 0.9.0 to 0.10.1 (#17929)
* chore: bump itertools from 0.9.0 to 0.10.1

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.9.0 to 0.10.1.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.9.0...v0.10.1)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix versions

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-14 18:32:20 +00:00
dependabot[bot] 820806c6dc
chore: bump bytecount from 0.6.0 to 0.6.2 (#17927)
Bumps [bytecount](https://github.com/llogiq/bytecount) from 0.6.0 to 0.6.2.
- [Release notes](https://github.com/llogiq/bytecount/releases)
- [Commits](https://github.com/llogiq/bytecount/commits)

---
updated-dependencies:
- dependency-name: bytecount
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-14 10:05:11 -06:00
Jeff Washington (jwash) d4cc975fe9
calculate_capitalization uses hash calculation (#17443)
* calculate_capitalization uses hash calculation

* feedback

* remove debugging code, clean up slot math
2021-06-14 08:53:07 -05:00
dependabot[bot] 5c72fef85a
chore: bump signal-hook from 0.1.15 to 0.2.3 (#17827)
Bumps [signal-hook](https://github.com/vorner/signal-hook) from 0.1.15 to 0.2.3.
- [Release notes](https://github.com/vorner/signal-hook/releases)
- [Changelog](https://github.com/vorner/signal-hook/blob/v0.2.3/CHANGELOG.md)
- [Commits](https://github.com/vorner/signal-hook/compare/v0.1.15...v0.2.3)

---
updated-dependencies:
- dependency-name: signal-hook
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-08 12:12:17 -06:00
dependabot[bot] 4bba92d3da
chore: bump serde_json from 1.0.62 to 1.0.64 (#17735)
* chore: bump serde_json from 1.0.62 to 1.0.64

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.62 to 1.0.64.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.62...v1.0.64)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-04 17:45:26 +00:00
Tao Zhu 3a12f92c34
Add subcommand "compute-slot-cost" to ledger tool (#17181)
* include cost_model and cost_tracker from banking stage

* add positional subcommand, compute-slot-cost, to ledger-tool to run cost-model on slots in ledger

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-06-02 11:29:02 -05:00
dependabot[bot] ac452f9bb8
Bump tempfile from 3.1.0 to 3.2.0 (#17587)
* Bump tempfile from 3.1.0 to 3.2.0

Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.1.0...v3.2.0)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-05-31 18:05:07 +00:00
dependabot[bot] cf5ad7cb7d
Bump futures from 0.3.8 to 0.3.15 (#17574)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.8 to 0.3.15.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.8...0.3.15)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-28 20:04:07 +00:00
dependabot[bot] 86c07bf785
Bump bs58 from 0.3.1 to 0.4.0 (#17573)
* Bump bs58 from 0.3.1 to 0.4.0

Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/compare/0.3.1...0.4.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fixup

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-05-28 18:30:34 +00:00
Michael Vines 8eab0e8602
Bump version to v1.8.0 (#17541) 2021-05-27 08:51:53 -07:00
Michael Vines 9541411c15 Plumb transaction-level rewards (aka "rent debits") into the `getTransaction` RPC method 2021-05-27 03:05:05 +00:00
Jeff Washington (jwash) 6b9d8d41a3
add --limit_load_slot_count_from_snapshot to ledger-tool (#17417) 2021-05-26 10:36:12 -05:00
Jeff Washington (jwash) 99313a4ba2
add ledger-tool verify option to verify hash calculation (#17412) 2021-05-24 18:15:57 -05:00
Lijun Wang 9c42a89a43
Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
Tyera Eulberg 6e9deaf1bd
Move block-time caching earlier (#17109)
* Require that blockstore block-time only be recognized slot, instead of root

* Move cache_block_time to after Bank freeze

* Single use statement

* Pass transaction_status_sender by reference

* Remove unnecessary slot-existence check before caching block time altogether

* Move block-time existence check into Blockstore::cache_block_time, Blockstore no longer needed in blockstore_processor helper
2021-05-10 13:14:56 -06:00
Tyera Eulberg ddfbae260f
Add ledger-tool for restoring roots to the Roots CF (#17045)
* Add ledger-tool for restoring roots to the Roots CF

* Print successful repair data, and repair in chunks

* Add parameter to limit num slots checked for root repair
2021-05-06 14:12:01 -06:00
Ryo Onodera d19526e6c2
Dump rent_collector/inflation with ledger-tool cap (#17069) 2021-05-06 19:29:46 +09:00
Jeff Washington (jwash) 82e11588a5
lamports -> lamports() (#16915) 2021-04-28 14:55:55 -05:00
Jeff Washington (jwash) da3342759b
private AccountSharedData.rent_epoch (#16877) 2021-04-28 08:52:20 -05:00
Jeff Washington (jwash) 2f5102587c
owner() lifetime issue (#16875) 2021-04-27 12:57:47 -05:00
Jeff Washington (jwash) 998cba74b5
AccountSharedData.executable() (#16835) 2021-04-27 09:12:17 -05:00
Jeff Washington (jwash) ca14c18998
owner -> owner() (#16782) 2021-04-23 22:49:47 +00:00
Jeff Washington (jwash) fc12841d95
Readonlyaccounts (#16743)
* lamports -> lamports()

* format
2021-04-22 20:04:55 +00:00
Jeff Washington (jwash) 333998d008
.lamports = <number> -> .set_lamports(<number>) (#16746) 2021-04-22 18:56:47 +00:00
Michael Vines a911ae00ba clippy 2021-04-18 20:55:02 -07:00
François Garillot b08cff9e77
Simplify some pattern-matches (#16402)
When those match an exact combinator on Option / Result.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
2021-04-08 12:40:37 -06:00
Michael Vines 43feef7362 Adjust tokio version to just "1" 2021-04-05 17:13:20 +00:00
carllin 4e5ef6bce2
Add cluster state verifier logging (#16330)
* Add cluster state verifier logging

* Add duplicate-slots iterator to ledger tool
2021-04-02 21:48:44 -07:00
Michael Vines a2eb655322 =1.7.0 2021-03-16 07:51:07 +00:00
Michael Vines 0c9ca5522c Bump version to v1.7.0 2021-03-13 09:01:21 +00:00
Ryo Onodera 4bbeb9c033
Remove old feature: simple_capitalization (#15763)
* Remove old feature: simple_capitalization

* Fix another failing test in core

* Finish up test cleanup

* Further clean up a bit
2021-03-12 11:12:40 +09:00
Michael Vines aa2b2d6b75 Default --ledger arg to "ledger" for `solana-validator` and `solana-ledger-tool` 2021-03-11 13:09:30 -08:00
Trent Nelson 24d18b3cf2 docs: add docs links for crates published to crates.io 2021-03-11 08:38:18 +00:00
Michael Vines c836cd85c3 Default to the BPF JIT. Use the `solana-validator --no-bpf-jit` flag to disable 2021-03-10 08:37:35 -08:00
Jeff Washington (jwash) 52e54e1100
account.data -> data() (#15778) 2021-03-09 22:31:33 +00:00
Jeff Washington (jwash) 8a3135d17b
Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
Ryo Onodera 8b0c6db871
Remove old feature: cumulative_rent_related_fixes (#15754) 2021-03-08 09:58:50 +09:00
Michael Vines e92cff3efa create-snapshot subcommad now accepts the ROOT keyword 2021-02-26 16:40:10 -08:00
Michael Vines 501696243f --help cleanup 2021-02-26 16:40:10 -08:00
carllin a6b9327cd0
Fix root scan in ledger tool (#15532) 2021-02-25 15:52:16 -08:00
Tyera Eulberg d521dfe63c
Implement OutputFormat for confirm in Cli and ledger-tool bigtable (#15528)
* Add CliTransaction struct

* Impl DisplayFormat for decode-transaction

* Add block-time to transaction println, writeln

* Impl DisplayFormat for confirm

* Use DisplayFormat in ledger-tool bigtable confirm
2021-02-25 14:15:52 -07:00
Tyera Eulberg d5f235d997
Implement OutputFormat for block in Cli and ledger-tool bigtable (#15524)
* Impl DisplayFormat for solana block

* Use DisplayFormat in ledger-tool bigtable block
2021-02-24 16:14:34 -07:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Tyera Eulberg 98e3e570d2 Add --force arg for bigtable upload 2021-02-16 17:24:07 -08:00
Ryo Onodera 4e99aa5fa6
More failure codepath tracing (#15246) 2021-02-12 15:24:23 +09:00
sakridge 91b374da5a
Fix typo (#15190) 2021-02-08 20:37:33 +00:00
sakridge bbae23358c
ledger-tool cleanup and additions (#15179)
* Plumb allow-dead-slots to ledger-tool verify

* ledger-tool cleanup and add some useful missing args

Print root slots and how many unrooted past last root.
2021-02-06 17:26:42 -08:00
sakridge 11b84cb870
Add jit and caching args to ledger-tool (#15177) 2021-02-06 19:49:24 +00:00
Jeff Washington (jwash) fbf9dc47e9
Threadpool2 (#15151)
* rework thread pool for hash calculation

* rename
2021-02-05 18:48:16 -06:00
Jeff Washington (jwash) fabecdc86c
use thread pool for non-index hash calculations (#15149) 2021-02-05 19:48:55 +00:00
Tyera Eulberg d1563f0ccd
Bump tonic, prost, tarpc, tokio (#15013)
* Update tonic & prost, and regenerate proto

* Reignore doc code

* Revert pull #14367, but pin tokio to v0.2 for jsonrpc

* Bump backoff and goauth -> and therefore tokio

* Bump tokio in faucet, net-utils

* Bump remaining tokio, plus tarpc
2021-02-05 00:21:53 -07:00
Trent Nelson a2aea0ca33 cli-output: Add option sigverify status to `println_transaction()` output 2021-02-02 18:44:22 +00:00
Michael Vines 709aa74e11 Adapt create-snapshot to avoid triggering recent internal bank sanity checks 2021-02-01 18:29:05 -08:00
Michael Vines 164fae44df /i/o/ 2021-01-31 08:22:01 -08:00
Michael Vines cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
Michael Vines bf1943e489 Add solana-test-validator --warp-slot argument 2021-01-22 21:17:02 -08:00
Michael Vines dd5a2ef05f Add convenience function to create a snapshot archive out of any Bank 2021-01-22 21:17:02 -08:00
Tyera Eulberg ca95302038
Update bigtable confirm to use confirmation_status (#14750) 2021-01-21 20:36:11 +00:00
Ryo Onodera dcaa025822
Configure Bigtable's timeout, enabling by default (#14657)
* Configure bigtable's timeout when read-only

* Review comments

* Apply nits (thanks!)

Co-authored-by: Michael Vines <mvines@gmail.com>

* Timeout in the streamed decoding as well

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-19 13:57:16 +00:00
carllin 6dfad0652f
Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
Michael Vines 7be6770808 Rename CompressionType to ArchiveFormat 2021-01-09 09:07:49 -08:00
Michael Vines 3d077fb656 Revert "Upgrade in-tree tokio 0.2 usage to tokio 0.3"
This reverts commit 444ed768dc.
2020-12-31 06:26:53 +00:00
Michael Vines 444ed768dc Upgrade in-tree tokio 0.2 usage to tokio 0.3 2020-12-29 09:23:23 -08:00
sakridge baa9602411
Add shrink paths (#14238) 2020-12-21 21:33:37 -08:00
Michael Vines 0b92720fdb Don't require increased open file limit in solana-test-validator
Travis CI in particular does not allow the open file limit to be
increased.
2020-12-16 20:43:08 -08:00
Michael Vines 1e977ac025 Bump version to v1.6.0 2020-12-15 18:28:04 +00:00
Michael Vines 7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
Ryo Onodera de9ac43ebf
Simple cap by including sysvars and native programs (#13884)
* Simpler cap by including sysvars and native programs

* Fix tests

* Add comment

* revert some unrelated code

* Update test_bank_update_sysvar_account for cap.

* Test cap. for add_native_program using new helper

* Improve the cap adjustment with new tests

* Fix typo...

* Adjust test for improved code coverage

* Rename simpler_capitalization => simple_capitalization

* More rename and bonus commenting
2020-12-14 21:03:24 +09:00
dependabot[bot] 88d950e5a8
chore: bump log from 0.4.8 to 0.4.11 (#13691)
* chore: bump log from 0.4.8 to 0.4.11

Bumps [log](https://github.com/rust-lang/log) from 0.4.8 to 0.4.11.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.11)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 13:48:23 +09:00
behzad nouri f6a7732ff9
adds ledger-tool mode to print raw shred metadata (#13962) 2020-12-08 17:31:24 +00:00
Ryo Onodera 82c75c3786
genesis, ledger-tool: --inflation/--vote-commission-percentaage (#13989) 2020-12-07 16:21:16 +00:00
Michael Vines c59cb28bbf Add --remove-account option to create-snapshot command 2020-12-05 08:57:00 +00:00
Ryo Onodera 480dd15b1e
Ledger tool cap basic columns (#13904)
* ledger-tool cap: Add more basic columns

* simplify InflationRecord::rewarded_epoch type
2020-12-02 07:10:03 +00:00
behzad nouri e1793e5a13
caches vote-state de-serialized from vote accounts (#13795)
Gossip and other places repeatedly de-serialize vote-state stored in
vote accounts. Ideally the first de-serialization should cache the
result.

This commit adds new VoteAccount type which lazily de-serializes
VoteState from Account data and caches the result internally.

Serialize and Deserialize traits are manually implemented to match
existing code. So, despite changes to frozen_abi, this commit should be
backward compatible.
2020-11-30 17:18:33 +00:00
Ryo Onodera 6048342c57
Enhance ledger-tool for multi-epoch rewards (#13837)
* Support ledger-tool for multi-epoch rewards

* nits

* Ensure not to skip some records in csv
2020-11-27 04:20:47 +00:00
Ryo Onodera 3bc7d85986
ledger-tool cap: output credits_observed (#13746) 2020-11-21 16:13:07 +00:00
Ryo Onodera 43d5e47ea9
Rewrite stake accounts for clear migration (#13461)
* Reduce overage stake by rewritng stake accounts

* Write tests and finish implemention

* Create and use new feature gate

* Clean up logging

* Fix typo

* Simplify enable_rewrite_stake

* Fix typo...

* Even simplify gating

* Add metrics
2020-11-20 05:15:06 +09:00
Ryo Onodera bcd303a447
ledger-tool cap: delegation owner and stake v2 flag (#13602)
* Output delegation owner as well

* Add --enable-stake-program-v2

* Small cleanup and add sanity assertion

* Fix typo...
2020-11-15 19:38:46 +00:00
dependabot[bot] fc54fb88ae chore: bump futures from 0.3.5 to 0.3.8
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.5 to 0.3.8.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.5...0.3.8)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-14 08:31:19 -08:00
Ryo Onodera a81e7e7749
Output more inflation calc details in ledger-tool (#13345)
* Output more inflation calc details in ledger-tool

* Fix broken ci...

* Rename confusing variables

* Fix panic by wrapping PointValue with Opiton...

* Minor modifications

* Remove explict needless flush; Drop already does

* Yet another csv field adjustments

* Add data_size and rename epochs to earned_epochs

* Introduce null_tracer

* Unwrap Option in new_from_parent_with_tracer

* Don't shorten identifiers

* Allow irrefutable_let_patterns temporalily

* More null_tracer

* More field adjustments
2020-11-08 16:43:50 +09:00
Ryo Onodera 363c148dbe
Disable eager rent collection for less noise (#13275) 2020-10-29 14:16:38 +00:00
Ryo Onodera 4698ee5e4a
Improve final report of ledger-tool capitalization (#13232) 2020-10-28 17:10:09 +00:00
Ryo Onodera 7d2962135d
Use pico inflation for ledger-tool capitalization --enable-inflation (#13215)
* Use pico inflation for ledger-tool capitalization --enable-inflation

* rust fmt
2020-10-28 15:30:39 +09:00
Ryo Onodera 6d4c69b7c3
Use zstd for create-snapshot (#13214) 2020-10-28 05:30:25 +00:00
Ryo Onodera 0776fa05c7
Add ledger-tool dead-slots and improve purge a lot (#13065)
* Add ledger-tool dead-slots and improve purge a lot

* Reduce batch size...

* Add --dead-slots-only and fixed purge ordering
2020-10-21 17:45:21 +00:00
Michael Vines 11df2e2236 Bump version to v1.5.0 2020-10-08 04:51:36 +00:00
Ryo Onodera 026e7de819
Show commit in `--version` and ledger-tool's log (#12636)
* Show commit in `--version` and ledger-tool's log

* Another handy hidden env var

* Fix test

* Rename to semver!

* Fix syntax error...
2020-10-04 00:30:26 +09:00
carllin 06f84c65f1
Fix rooted accounts cleanup, simplify locking (#12194)
Co-authored-by: Carl Lin <carl@solana.com>
2020-09-28 16:04:46 -07:00
Justin Starry 731a943239
Remove transaction encoding from storage layer (#12404) 2020-09-24 13:10:29 +08:00
Trent Nelson 325a7e9f86 Move CLI cli_output module to its own crate 2020-09-23 15:58:44 -06:00
Ryo Onodera de4a613610
Check bank capitalization (#11927)
* Check bank capitalization

* Simplify and unify capitalization calculation

* Improve and add tests

* Avoid overflow and inhibit automatic restart

* Fix test

* Tweak checked sum for cap. and add tests

* Fix broken build after merge conflicts..

* Rename to ClusterType

* Rename confusing method

* Clarify comment

* Verify cap. in rent and inflation tests

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
2020-09-12 01:48:06 +09:00
Michael Vines 10ce839ec0 Speed up `bigtable transaction-history` command 2020-09-10 05:22:10 +00:00
Michael Vines 8143ea0dea Add `bigtable transaction-history --query-chunk-size` argument 2020-09-09 20:37:20 -07:00
Ryo Onodera 53b8ea4464
Rename to ClusterType and restore devnet compat. (#12068)
* Rename to ClusterType and restore devnet compat.

* De-duplicate parse code and add comments

* Adjust default Devnet genesis & reduce it in tests
2020-09-08 23:55:09 +09:00
Michael Vines 2665c5b3c2 Add --show-transactions flag to `bigtable transaction-history` command 2020-09-07 21:34:11 -07:00
Michael Vines d3611f74c8 Add exit flag for bigtable upload operations 2020-09-04 16:01:49 -07:00
Michael Vines 91a56caed2 Relocate BigTable uploader to ledger/ crate 2020-09-04 16:01:49 -07:00
Michael Vines a07980536a Add bank-hash subcommand 2020-08-30 17:09:20 +00:00
Michael Vines 8df263340f modify-genesis now writes elsewhere and produces a full genesis.tar.bz2 2020-08-28 19:58:55 -07:00
Michael Vines df7710a131 Add ability to fork a local cluster from the latest mainnet-beta snapshot 2020-08-28 16:35:27 -07:00
Ryo Onodera 57174cdabe
Make ledger-tool accounts print rent_epoch and slot (#11845) 2020-08-27 12:28:40 +09:00
Michael Vines d1500ae229 The end_slot argument to purge is now optional 2020-08-18 20:17:44 -07:00
Michael Vines adc984a225 Rename Binary64 to Base64. Establish Base58 encoding 2020-08-17 17:26:29 -07:00
Michael Vines b5f3ced860 Add base64 (binary64) encoding for getConfirmedTransaction/getConfirmedBlock 2020-08-17 17:26:29 -07:00
Tyera Eulberg 6c5b8f324a
Rpc: Add until parameter for getConfirmedSignaturesForAddress2 (#11644)
* Refactor bigtable apis to accept start and end keys

* Make helper fn to deserialize cell data

* Refactor get_confirmed_signatures_for_address to use get_row_data range

* Add until param to get_confirmed_signatures_for_address

* Add until param to blockstore api

* Plumb until through client/cli

* Simplify client params
2020-08-15 10:42:17 -06:00
carllin 7e25130529
Send votes from banking stage to vote listener (#11434)
*  Send votes from banking stage to vote listener

Co-authored-by: Carl <carl@solana.com>
2020-08-07 11:21:35 -07:00
Michael Vines 9abb7db5f8 Upgrade help description 2020-08-05 23:04:16 -07:00
Michael Vines 4222932e08 Plug getConfirmedSignaturesForAddress2 into bigtable storage 2020-08-05 23:04:16 -07:00
Michael Vines 227ea934ff after -> before 2020-08-05 14:27:12 -07:00
Michael Vines 22c46ebf96 Report block status more frequently 2020-08-05 14:27:12 -07:00
Michael Vines dfae9a9864 Plumb Bigtable ledger storage into the RPC subsystem 2020-08-05 14:27:12 -07:00
Michael Vines f9049d6ee4 Add ledger-tool bigtable subcommands 2020-08-05 14:27:12 -07:00
Trent Nelson e12ab9d0dd Bump version to 1.4.0 2020-08-05 12:04:15 -06:00
carllin bf18524368
Add hook for getting vote transactions on replay (#11264)
* Add hook for getting vote transactions on replay

Co-authored-by: Carl <carl@solana.com>
2020-07-29 23:17:40 -07:00
Ryo Onodera 6935e88148
Freeze warped bank to show freeze logs (#11141) 2020-07-21 15:51:56 +09:00
Ryo Onodera 3db246f596
Add --warp-epoch and --force-inflation to ledger-tool cap. (#11107)
* Add --warp-epoch and --force-inflation to ledger-tool cap.

* Add more arguments

* Address review comments

* Fix message

* Fix various typos...
2020-07-20 19:09:38 +09:00
Tyera Eulberg 9a80e31bae
Expose tss to the other blockstore_processor path (#11070) 2020-07-14 21:14:48 -06:00
Michael Vines bed74d803f Add purge --no-compaction flag 2020-07-14 15:02:01 -07:00
dependabot-preview[bot] 5055028231 Bump serde_json from 1.0.54 to 1.0.56
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.54 to 1.0.56.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.54...v1.0.56)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-07-14 11:13:22 -07:00
sakridge 6429042b6e
Add --print-account-stats/--snapshot-archive-path arguments to ledger-tool (#10945)
Allows for seeing how the accounts are being stored
and specifying a different snapshot source directory.
2020-07-08 09:32:11 -07:00
sakridge 58a475b789
Add db recovery methods (#10838) 2020-07-06 12:43:45 -07:00
sakridge 1269e348fb
Add a way to re-hash the bank (#10765) 2020-07-06 12:42:41 -07:00
carllin 823126b372
Fix parsing (#10924)
Co-authored-by: Carl <carl@solana.com>
2020-07-06 02:39:42 -07:00
Ryo Onodera b89e506cbb
Consistently create temp dirs under ledger/farf (#10848) 2020-07-01 11:38:59 +09:00
sakridge 1ffad2d051
More guard rails for restart with ledger procedure (#10853)
* Add expected_bank_hash required for supermajority
* Print snapshot hash in ledger-tool create-snapshot.
2020-06-30 12:43:48 -07:00
carllin 2dfa48daf9
Add ability to parse logs in ledger-tool (#10840)
Co-authored-by: Carl <carl@solana.com>
2020-06-30 02:20:54 -07:00
sakridge 583cec922b
Remove ledger purge batching (#10830) 2020-06-29 14:44:35 -07:00
sakridge ea30c157e0
Better slot printing (#10811) 2020-06-27 10:47:02 -07:00
sakridge aa544b2245
Add ledger copy command (#10756) 2020-06-25 11:56:36 -07:00
Trent Nelson d42247c652
ledger-tool: Ignore SIGUSR1 (#10730)
Prevents warehouse archive calls getting KO'd by logrotate
2020-06-21 18:08:17 +00:00