blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
20,785 Commits 180 Branches 498 Tags 557 MiB
Commit Graph

586 Commits

Author SHA1 Message Date
Brooks Prumo 9d9482b9d8
Plumb `maximum_incremental_snapshot_archives_to_retain` (#19640) 2021-09-06 18:01:56 -05:00
Tyera Eulberg decec3cd8b
Demote write locks on transaction program ids (#19593) 
* Add feature

* Demote write lock on program ids

* Fixup bpf tests

* Update MappedMessage::is_writable

* Comma nit

* Review comments
 2021-09-04 03:05:30 +00:00
Brooks Prumo 5e25ee5ebe
Add maximum_incremental_snapshot_archives_to_retain to SnapshotConfig (#19612) 2021-09-03 20:21:32 +00:00
Brooks Prumo 7ab0aec61f
Rename maximum_full_snapshot_archives_to_retain (#19610) 
To prepare for adding maximum_incremental_snapshot_archives_to_retain,
rename the current field in SnapshotConfig.
 2021-09-03 11:28:10 -05:00
Brooks Prumo e9374d32a3
Revert "Make startup aware of Incremental Snapshots (#19550)" (#19599) 
This reverts commit d45ced0a5d.
 2021-09-02 19:14:41 -05:00
Brooks Prumo d45ced0a5d
Make startup aware of Incremental Snapshots (#19550) 2021-09-02 19:05:15 -05:00
Brooks Prumo 1d5a8ebc6a
Revert "Add LastFullSnapshotSlot to SnapshotConfig (#19341)" (#19529) 
This reverts commit 4d361af976.
 2021-08-31 22:03:19 -05:00
Brooks Prumo fe9ee9134a
Make background services aware of incremental snapshots (#19401) 
AccountsBackgroundService now knows about incremental snapshots.  It is
now also in charge of deciding if an AccountsPackage is destined to be a
SnapshotPackage or not (or just used by AccountsHashVerifier).

!!! New behavior changes !!!

Taking snapshots (both bank and archive) **MUST** succeed.

This is required because of how the last full snapshot slot is
calculated, which is used by AccountsBackgroundService when calling
`clean_accounts()`.

File system calls are now unwrapped and will result in a crash. As Trent told me:

>Well I think if a snapshot fails due to some IO error, it's very likely that the operator is going to have to intervene before it works.  We should exit error in this case, otherwise the validator might happily spin for several more hours, never successfully writing a complete snapshot, before something else brings it down.  This would leave the validator's last local snapshot many more slots behind than it would be had we exited outright and potentially force the operator to abandon ledger continuity in favor of a quick catchup

Other errors will set the `exit` flag to `true`, and the node will gracefully shutdown.

Fixes #19167 
Fixes #19168
 2021-08-31 18:33:27 -05:00
dependabot[bot] c21bf29ce7
chore: bump serde_json from 1.0.66 to 1.0.67 (#19501) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.66 to 1.0.67.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.66...v1.0.67)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-31 06:05:25 +00:00
dependabot[bot] 8e4360cf9c
chore: bump futures from 0.3.16 to 0.3.17 (#19513) 
* chore: bump futures from 0.3.16 to 0.3.17

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-08-30 22:12:38 -06:00
dependabot[bot] fbffdf9f3d
chore: bump futures-util from 0.3.16 to 0.3.17 (#19498) 
* chore: bump futures-util from 0.3.16 to 0.3.17

Bumps [futures-util](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17)

---
updated-dependencies:
- dependency-name: futures-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 17:45:47 +00:00
behzad nouri 8ad52fa095
implements copy-on-write for vote-accounts (#19362) 
Bank::vote_accounts redundantly clones vote-accounts HashMap even though
an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/95c998a19/runtime/src/bank.rs#L5174-L5186

This commit implements copy-on-write semantics for vote-accounts by
wrapping the underlying HashMap in Arc<...>.
 2021-08-30 15:54:01 +00:00
dependabot[bot] db2be00289
chore: bump serde_yaml from 0.8.19 to 0.8.20 (#19464) 
* chore: bump serde_yaml from 0.8.19 to 0.8.20

Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.19 to 0.8.20.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.19...0.8.20)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-27 09:33:08 -06:00
dependabot[bot] c721300ab5
chore: bump serde_yaml from 0.8.18 to 0.8.19 (#19372) 
* chore: bump serde_yaml from 0.8.18 to 0.8.19

Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.18 to 0.8.19.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.18...0.8.19)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-08-23 10:23:16 -06:00
Brooks Prumo 6d939811e9
Name snapshots consistently (#19346) 
#### Problem

Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:

```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```

#### Summary of Changes

For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.


Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-08-21 15:41:03 -05:00
Brooks Prumo 4d361af976
Add LastFullSnapshotSlot to SnapshotConfig (#19341) 2021-08-20 17:06:53 +00:00
Jon Cinque 73aa004c59
stake: Remove v2 program references (#19308) 
* stake: Remove v2 program references

* Remove stake v2 feature, along with stake rewrite
 2021-08-20 01:08:44 -04:00
dependabot[bot] 39459085ef
chore: bump serde_yaml from 0.8.17 to 0.8.18 (#19280) 
* chore: bump serde_yaml from 0.8.17 to 0.8.18

Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.17 to 0.8.18.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.17...0.8.18)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-08-18 08:23:20 -06:00
Justin Starry c50b01cb60
Store versioned transactions in the ledger, disabled by default (#19139) 
* Add support for versioned transactions, but disable by default

* merge conflicts

* trent's feedback

* bump Cargo.lock

* Fix transaction error encoding

* Rename legacy_transaction method

* cargo clippy

* Clean up casts, int arithmetic, and unused methods

* Check for duplicates in sanitized message conversion

* fix clippy

* fix new test

* Fix bpf conditional compilation for message module
 2021-08-17 15:17:56 -07:00
Jeff Washington (jwash) 7c70f2158b
accounts_index_bins to AccountsIndexConfig (#19257) 
* accounts_index_bins to AccountsIndexConfig

* rename param bins -> config

* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
 2021-08-17 14:50:01 -05:00
Jeff Washington (jwash) e91988c977
cli for num account index bins (#19085) 2021-08-11 11:45:25 -05:00
Tyera Eulberg db40cb41b2
Handle new security advisories (#19126) 
* Update hyper 0.14

* Add audit ignores

* Bump assert_cmd and remove audit ignore

* Remove unnecessary unmaintained dependency

* Remove pin-project-lite warning
 2021-08-09 19:36:03 +00:00
Brooks Prumo fd937548a0
Move SnapshotArchiveInfo and friends into its own module (#19114) 2021-08-08 07:57:06 -05:00
Brooks Prumo 00890957ee
Add snapshot_utils::bank_from_latest_snapshot_archives() (#18983) 
While reviewing PR #18565, as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots.  A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify.  Additionally, new tests have been
written and existing tests have been updated to use this new function.

Fixes #18973

While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled.  Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`.  And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats.  This bundling simplifies bank
rebuilding.
 2021-08-06 20:16:06 -05:00
Brooks Prumo ca14475085
Add incremental_snapshot_archive_interval_slots to SnapshotConfig (#19026) 
This commit also renames `snapshot_interval_slots` to
`full_snapshot_archive_interval_slots`, updates the comments on the
fields, and make appropriate updates where SnapshotConfig is used.
 2021-08-04 14:40:20 -05:00
dependabot[bot] fe1ee49807
chore: bump serde_json from 1.0.65 to 1.0.66 (#18990) 
* chore: bump serde_json from 1.0.65 to 1.0.66

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.65 to 1.0.66.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.65...v1.0.66)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-07-30 10:46:47 -06:00
dependabot[bot] 0cf52e2060
chore: bump serde_json from 1.0.64 to 1.0.65 (#18968) 
* chore: bump serde_json from 1.0.64 to 1.0.65

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.64 to 1.0.65.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.64...v1.0.65)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-07-29 16:52:02 +00:00
Tyera Eulberg 467c18e0d1
Add cli-output helpers (#18933) 
* Add OutputFormat helper to reduce copy-pasta

* Add CliSignOnlyData constructor
 2021-07-28 04:21:23 +00:00
dependabot[bot] 4e41c05acd
chore: bump futures from 0.3.15 to 0.3.16 (#18914) 
* chore: bump futures from 0.3.15 to 0.3.16

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-07-27 07:59:34 -06:00
dependabot[bot] b97113408b
chore: bump futures-util from 0.3.15 to 0.3.16 (#18878) 
* chore: bump futures-util from 0.3.15 to 0.3.16

Bumps [futures-util](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Nudge Cargo.lock

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-07-26 18:19:53 +00:00
Brooks Prumo d1debcd971
Add incremental snapshot utils (#18504) 
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks.  This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.

Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.

Also of note, some renaming has happened:
  1. Snapshots are now either `full_` or `incremental_` throughout the
     codebase.  If not specified, the code applies to both.
  2. Bank snapshots now are called "bank snapshots"
     (before they were called "slot snapshots", "bank snapshots", or
      just "snapshots").  The one exception is within `Bank`, where they
     are still just "snapshots", because they are already "bank
     snapshots".
  3. Snapshot archives now have `_archive` in the code.  This
     should clear up an ambiguity between bank snapshots and snapshot
     archives.
 2021-07-22 14:40:37 -05:00
Justin Starry d166b9856a
Move transaction sanitization earlier in the pipeline (#18655) 
* Move transaction sanitization earlier in the pipeline

* Renamed HashedTransaction to SanitizedTransaction

* Implement deref for sanitized transaction

* bring back process_transactions test method

* Use sanitized transactions for cost model calculation
 2021-07-15 22:51:27 -05:00
Trent Nelson 3a85b77bb5 hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade 2021-07-15 18:43:55 +00:00
Trent Nelson 568660b402 Revert "Remove feature switch for secp256k1 program (#18467)" 
This reverts commit fd574dcb3b.
 2021-07-15 18:43:55 +00:00
sakridge 7f2254225e
Move entry/poh to own crate to speed up poh bench build (#18225) 2021-07-14 14:16:29 +02:00
Tao Zhu 350baece21
Explicitly sanitize program id indexes before usage 
1. check transaction has valid program_id before using it to avoid possible panic;
2. change calculate_cost function signature to return Result;
3. add CostModelError enum, update return type from Result<_, str> to Result<_, CostModelError>
 2021-07-13 17:29:22 -05:00
Jeff Washington (jwash) d092fa1f03
add ledger-tool verify verify-accounts-index option (#18375) 
* add ledger-tool verify verify-accounts-index option

* comment, merge, respond to feedback, cleanup
 2021-07-13 11:06:18 -05:00
Michael Vines 4098af3b5b Record vote account commission with voting/staking rewards and surface in RPC 2021-07-12 15:09:44 -07:00
Justin Starry fd574dcb3b
Remove feature switch for secp256k1 program (#18467) 
* Remove feature switch for secp256k1 program

* fix tests
 2021-07-09 10:08:03 -05:00
Tao Zhu b6dff12923
update ledger tool to restore cost table from blockstore (#18489) 
* update ledger tool to restore cost model from blockstore when compute-slot-cost

* Move initialize_cost_table into cost_model, so the function can be tested and shared between validator and ledger-tool

* refactor and simplify a test
 2021-07-07 23:44:51 -05:00
Tao Zhu 0e039b4094
Aggregate cost_model into cost_tracker (#18374) 
* * aggregate cost_model into cost_tracker, decouple it from banking_stage to prevent accidental deadlock. * Simplified code, removed unused functions

* review fixes
 2021-07-06 15:41:25 +00:00
carllin 0eca92de18
Make set roots an iterator (#18357) 2021-07-01 20:02:40 -07:00
Brooks Prumo 89a3e4f91e
Move SnapshotConfig into its own module (#18331) 
Also move ArchiveFormat to snapshot_utils, and do not
reexport SnapshotVersion.
 2021-07-01 08:55:26 -05:00
dependabot[bot] 78968d132f
chore: bump log from 0.4.11 to 0.4.14 (#18323) 
* chore: bump log from 0.4.11 to 0.4.14

Bumps [log](https://github.com/rust-lang/log) from 0.4.11 to 0.4.14.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.11...0.4.14)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make version consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-30 18:32:01 +00:00
dependabot[bot] ff8aef9d27
chore: bump csv from 1.1.3 to 1.1.6 (#18261) 
Bumps [csv](https://github.com/BurntSushi/rust-csv) from 1.1.3 to 1.1.6.
- [Release notes](https://github.com/BurntSushi/rust-csv/releases)
- [Commits](https://github.com/BurntSushi/rust-csv/compare/1.1.3...1.1.6)

---
updated-dependencies:
- dependency-name: csv
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-28 09:56:42 -06:00
dependabot[bot] d923ff9cfb
chore: bump serde_yaml from 0.8.13 to 0.8.17 (#18169) 
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.13 to 0.8.17.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.13...0.8.17)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-23 09:44:34 -06:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Jon Cinque 1b1d34da59
Refactor stake program into solana_program (#17906) 
* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt
 2021-06-15 18:04:00 +02:00
carllin ccc013e134
Handle removing slots during account scans (#17471) 2021-06-14 21:04:01 -07:00
dependabot[bot] a0872232d3
chore: bump itertools from 0.9.0 to 0.10.1 (#17929) 
* chore: bump itertools from 0.9.0 to 0.10.1

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.9.0 to 0.10.1.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.9.0...v0.10.1)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix versions

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-14 18:32:20 +00:00
dependabot[bot] 820806c6dc
chore: bump bytecount from 0.6.0 to 0.6.2 (#17927) 
Bumps [bytecount](https://github.com/llogiq/bytecount) from 0.6.0 to 0.6.2.
- [Release notes](https://github.com/llogiq/bytecount/releases)
- [Commits](https://github.com/llogiq/bytecount/commits)

---
updated-dependencies:
- dependency-name: bytecount
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-14 10:05:11 -06:00
Jeff Washington (jwash) d4cc975fe9
calculate_capitalization uses hash calculation (#17443) 
* calculate_capitalization uses hash calculation

* feedback

* remove debugging code, clean up slot math
 2021-06-14 08:53:07 -05:00
dependabot[bot] 5c72fef85a
chore: bump signal-hook from 0.1.15 to 0.2.3 (#17827) 
Bumps [signal-hook](https://github.com/vorner/signal-hook) from 0.1.15 to 0.2.3.
- [Release notes](https://github.com/vorner/signal-hook/releases)
- [Changelog](https://github.com/vorner/signal-hook/blob/v0.2.3/CHANGELOG.md)
- [Commits](https://github.com/vorner/signal-hook/compare/v0.1.15...v0.2.3)

---
updated-dependencies:
- dependency-name: signal-hook
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-08 12:12:17 -06:00
dependabot[bot] 4bba92d3da
chore: bump serde_json from 1.0.62 to 1.0.64 (#17735) 
* chore: bump serde_json from 1.0.62 to 1.0.64

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.62 to 1.0.64.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.62...v1.0.64)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-04 17:45:26 +00:00
Tao Zhu 3a12f92c34
Add subcommand "compute-slot-cost" to ledger tool (#17181) 
* include cost_model and cost_tracker from banking stage

* add positional subcommand, compute-slot-cost, to ledger-tool to run cost-model on slots in ledger

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-06-02 11:29:02 -05:00
dependabot[bot] ac452f9bb8
Bump tempfile from 3.1.0 to 3.2.0 (#17587) 
* Bump tempfile from 3.1.0 to 3.2.0

Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.1.0...v3.2.0)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-05-31 18:05:07 +00:00
dependabot[bot] cf5ad7cb7d
Bump futures from 0.3.8 to 0.3.15 (#17574) 
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.8 to 0.3.15.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.8...0.3.15)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-28 20:04:07 +00:00
dependabot[bot] 86c07bf785
Bump bs58 from 0.3.1 to 0.4.0 (#17573) 
* Bump bs58 from 0.3.1 to 0.4.0

Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/compare/0.3.1...0.4.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fixup

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-05-28 18:30:34 +00:00
Michael Vines 8eab0e8602
Bump version to v1.8.0 (#17541) 2021-05-27 08:51:53 -07:00
Michael Vines 9541411c15 Plumb transaction-level rewards (aka "rent debits") into the `getTransaction` RPC method 2021-05-27 03:05:05 +00:00
Jeff Washington (jwash) 6b9d8d41a3
add --limit_load_slot_count_from_snapshot to ledger-tool (#17417) 2021-05-26 10:36:12 -05:00
Jeff Washington (jwash) 99313a4ba2
add ledger-tool verify option to verify hash calculation (#17412) 2021-05-24 18:15:57 -05:00
Lijun Wang 9c42a89a43
Issue #17008 -- make snapshot archives to hold on to configurable. (#17158) 
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
 2021-05-12 10:32:27 -07:00
Tyera Eulberg 6e9deaf1bd
Move block-time caching earlier (#17109) 
* Require that blockstore block-time only be recognized slot, instead of root

* Move cache_block_time to after Bank freeze

* Single use statement

* Pass transaction_status_sender by reference

* Remove unnecessary slot-existence check before caching block time altogether

* Move block-time existence check into Blockstore::cache_block_time, Blockstore no longer needed in blockstore_processor helper
 2021-05-10 13:14:56 -06:00
Tyera Eulberg ddfbae260f
Add ledger-tool for restoring roots to the Roots CF (#17045) 
* Add ledger-tool for restoring roots to the Roots CF

* Print successful repair data, and repair in chunks

* Add parameter to limit num slots checked for root repair
 2021-05-06 14:12:01 -06:00
Ryo Onodera d19526e6c2
Dump rent_collector/inflation with ledger-tool cap (#17069) 2021-05-06 19:29:46 +09:00
Jeff Washington (jwash) 82e11588a5
lamports -> lamports() (#16915) 2021-04-28 14:55:55 -05:00
Jeff Washington (jwash) da3342759b
private AccountSharedData.rent_epoch (#16877) 2021-04-28 08:52:20 -05:00
Jeff Washington (jwash) 2f5102587c
owner() lifetime issue (#16875) 2021-04-27 12:57:47 -05:00
Jeff Washington (jwash) 998cba74b5
AccountSharedData.executable() (#16835) 2021-04-27 09:12:17 -05:00
Jeff Washington (jwash) ca14c18998
owner -> owner() (#16782) 2021-04-23 22:49:47 +00:00
Jeff Washington (jwash) fc12841d95
Readonlyaccounts (#16743) 
* lamports -> lamports()

* format
 2021-04-22 20:04:55 +00:00
Jeff Washington (jwash) 333998d008
.lamports = <number> -> .set_lamports(<number>) (#16746) 2021-04-22 18:56:47 +00:00
Michael Vines a911ae00ba clippy 2021-04-18 20:55:02 -07:00
François Garillot b08cff9e77
Simplify some pattern-matches (#16402) 
When those match an exact combinator on Option / Result.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
 2021-04-08 12:40:37 -06:00
Michael Vines 43feef7362 Adjust tokio version to just "1" 2021-04-05 17:13:20 +00:00
carllin 4e5ef6bce2
Add cluster state verifier logging (#16330) 
* Add cluster state verifier logging

* Add duplicate-slots iterator to ledger tool
 2021-04-02 21:48:44 -07:00
Michael Vines a2eb655322 =1.7.0 2021-03-16 07:51:07 +00:00
Michael Vines 0c9ca5522c Bump version to v1.7.0 2021-03-13 09:01:21 +00:00
Ryo Onodera 4bbeb9c033
Remove old feature: simple_capitalization (#15763) 
* Remove old feature: simple_capitalization

* Fix another failing test in core

* Finish up test cleanup

* Further clean up a bit
 2021-03-12 11:12:40 +09:00
Michael Vines aa2b2d6b75 Default --ledger arg to "ledger" for `solana-validator` and `solana-ledger-tool` 2021-03-11 13:09:30 -08:00
Trent Nelson 24d18b3cf2 docs: add docs links for crates published to crates.io 2021-03-11 08:38:18 +00:00
Michael Vines c836cd85c3 Default to the BPF JIT. Use the `solana-validator --no-bpf-jit` flag to disable 2021-03-10 08:37:35 -08:00
Jeff Washington (jwash) 52e54e1100
account.data -> data() (#15778) 2021-03-09 22:31:33 +00:00
Jeff Washington (jwash) 8a3135d17b
Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
Ryo Onodera 8b0c6db871
Remove old feature: cumulative_rent_related_fixes (#15754) 2021-03-08 09:58:50 +09:00
Michael Vines e92cff3efa create-snapshot subcommad now accepts the ROOT keyword 2021-02-26 16:40:10 -08:00
Michael Vines 501696243f --help cleanup 2021-02-26 16:40:10 -08:00
carllin a6b9327cd0
Fix root scan in ledger tool (#15532) 2021-02-25 15:52:16 -08:00
Tyera Eulberg d521dfe63c
Implement OutputFormat for confirm in Cli and ledger-tool bigtable (#15528) 
* Add CliTransaction struct

* Impl DisplayFormat for decode-transaction

* Add block-time to transaction println, writeln

* Impl DisplayFormat for confirm

* Use DisplayFormat in ledger-tool bigtable confirm
 2021-02-25 14:15:52 -07:00
Tyera Eulberg d5f235d997
Implement OutputFormat for block in Cli and ledger-tool bigtable (#15524) 
* Impl DisplayFormat for solana block

* Use DisplayFormat in ledger-tool bigtable block
 2021-02-24 16:14:34 -07:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Tyera Eulberg 98e3e570d2 Add --force arg for bigtable upload 2021-02-16 17:24:07 -08:00
Ryo Onodera 4e99aa5fa6
More failure codepath tracing (#15246) 2021-02-12 15:24:23 +09:00
sakridge 91b374da5a
Fix typo (#15190) 2021-02-08 20:37:33 +00:00
sakridge bbae23358c
ledger-tool cleanup and additions (#15179) 
* Plumb allow-dead-slots to ledger-tool verify

* ledger-tool cleanup and add some useful missing args

Print root slots and how many unrooted past last root.
 2021-02-06 17:26:42 -08:00
sakridge 11b84cb870
Add jit and caching args to ledger-tool (#15177) 2021-02-06 19:49:24 +00:00
Jeff Washington (jwash) fbf9dc47e9
Threadpool2 (#15151) 
* rework thread pool for hash calculation

* rename
 2021-02-05 18:48:16 -06:00
Jeff Washington (jwash) fabecdc86c
use thread pool for non-index hash calculations (#15149) 2021-02-05 19:48:55 +00:00
Tyera Eulberg d1563f0ccd
Bump tonic, prost, tarpc, tokio (#15013) 
* Update tonic & prost, and regenerate proto

* Reignore doc code

* Revert pull #14367, but pin tokio to v0.2 for jsonrpc

* Bump backoff and goauth -> and therefore tokio

* Bump tokio in faucet, net-utils

* Bump remaining tokio, plus tarpc
 2021-02-05 00:21:53 -07:00
Trent Nelson a2aea0ca33 cli-output: Add option sigverify status to `println_transaction()` output 2021-02-02 18:44:22 +00:00
Michael Vines 709aa74e11 Adapt create-snapshot to avoid triggering recent internal bank sanity checks 2021-02-01 18:29:05 -08:00
Michael Vines 164fae44df /i/o/ 2021-01-31 08:22:01 -08:00
Michael Vines cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
Michael Vines bf1943e489 Add solana-test-validator --warp-slot argument 2021-01-22 21:17:02 -08:00
Michael Vines dd5a2ef05f Add convenience function to create a snapshot archive out of any Bank 2021-01-22 21:17:02 -08:00
Tyera Eulberg ca95302038
Update bigtable confirm to use confirmation_status (#14750) 2021-01-21 20:36:11 +00:00
Ryo Onodera dcaa025822
Configure Bigtable's timeout, enabling by default (#14657) 
* Configure bigtable's timeout when read-only

* Review comments

* Apply nits (thanks!)

Co-authored-by: Michael Vines <mvines@gmail.com>

* Timeout in the streamed decoding as well

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-01-19 13:57:16 +00:00
carllin 6dfad0652f
Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
Michael Vines 7be6770808 Rename CompressionType to ArchiveFormat 2021-01-09 09:07:49 -08:00
Michael Vines 3d077fb656 Revert "Upgrade in-tree tokio 0.2 usage to tokio 0.3" 
This reverts commit 444ed768dc.
 2020-12-31 06:26:53 +00:00
Michael Vines 444ed768dc Upgrade in-tree tokio 0.2 usage to tokio 0.3 2020-12-29 09:23:23 -08:00
sakridge baa9602411
Add shrink paths (#14238) 2020-12-21 21:33:37 -08:00
Michael Vines 0b92720fdb Don't require increased open file limit in solana-test-validator 
Travis CI in particular does not allow the open file limit to be
increased.
 2020-12-16 20:43:08 -08:00
Michael Vines 1e977ac025 Bump version to v1.6.0 2020-12-15 18:28:04 +00:00
Michael Vines 7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
Ryo Onodera de9ac43ebf
Simple cap by including sysvars and native programs (#13884) 
* Simpler cap by including sysvars and native programs

* Fix tests

* Add comment

* revert some unrelated code

* Update test_bank_update_sysvar_account for cap.

* Test cap. for add_native_program using new helper

* Improve the cap adjustment with new tests

* Fix typo...

* Adjust test for improved code coverage

* Rename simpler_capitalization => simple_capitalization

* More rename and bonus commenting
 2020-12-14 21:03:24 +09:00
dependabot[bot] 88d950e5a8
chore: bump log from 0.4.8 to 0.4.11 (#13691) 
* chore: bump log from 0.4.8 to 0.4.11

Bumps [log](https://github.com/rust-lang/log) from 0.4.8 to 0.4.11.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.11)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2020-12-13 13:48:23 +09:00
behzad nouri f6a7732ff9
adds ledger-tool mode to print raw shred metadata (#13962) 2020-12-08 17:31:24 +00:00
Ryo Onodera 82c75c3786
genesis, ledger-tool: --inflation/--vote-commission-percentaage (#13989) 2020-12-07 16:21:16 +00:00
Michael Vines c59cb28bbf Add --remove-account option to create-snapshot command 2020-12-05 08:57:00 +00:00
Ryo Onodera 480dd15b1e
Ledger tool cap basic columns (#13904) 
* ledger-tool cap: Add more basic columns

* simplify InflationRecord::rewarded_epoch type
 2020-12-02 07:10:03 +00:00
behzad nouri e1793e5a13
caches vote-state de-serialized from vote accounts (#13795) 
Gossip and other places repeatedly de-serialize vote-state stored in
vote accounts. Ideally the first de-serialization should cache the
result.

This commit adds new VoteAccount type which lazily de-serializes
VoteState from Account data and caches the result internally.

Serialize and Deserialize traits are manually implemented to match
existing code. So, despite changes to frozen_abi, this commit should be
backward compatible.
 2020-11-30 17:18:33 +00:00
Ryo Onodera 6048342c57
Enhance ledger-tool for multi-epoch rewards (#13837) 
* Support ledger-tool for multi-epoch rewards

* nits

* Ensure not to skip some records in csv
 2020-11-27 04:20:47 +00:00
Ryo Onodera 3bc7d85986
ledger-tool cap: output credits_observed (#13746) 2020-11-21 16:13:07 +00:00
Ryo Onodera 43d5e47ea9
Rewrite stake accounts for clear migration (#13461) 
* Reduce overage stake by rewritng stake accounts

* Write tests and finish implemention

* Create and use new feature gate

* Clean up logging

* Fix typo

* Simplify enable_rewrite_stake

* Fix typo...

* Even simplify gating

* Add metrics
 2020-11-20 05:15:06 +09:00
Ryo Onodera bcd303a447
ledger-tool cap: delegation owner and stake v2 flag (#13602) 
* Output delegation owner as well

* Add --enable-stake-program-v2

* Small cleanup and add sanity assertion

* Fix typo...
 2020-11-15 19:38:46 +00:00
dependabot[bot] fc54fb88ae chore: bump futures from 0.3.5 to 0.3.8 
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.5 to 0.3.8.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.5...0.3.8)

Signed-off-by: dependabot[bot] <support@github.com>
 2020-11-14 08:31:19 -08:00
Ryo Onodera a81e7e7749
Output more inflation calc details in ledger-tool (#13345) 
* Output more inflation calc details in ledger-tool

* Fix broken ci...

* Rename confusing variables

* Fix panic by wrapping PointValue with Opiton...

* Minor modifications

* Remove explict needless flush; Drop already does

* Yet another csv field adjustments

* Add data_size and rename epochs to earned_epochs

* Introduce null_tracer

* Unwrap Option in new_from_parent_with_tracer

* Don't shorten identifiers

* Allow irrefutable_let_patterns temporalily

* More null_tracer

* More field adjustments
 2020-11-08 16:43:50 +09:00
Ryo Onodera 363c148dbe
Disable eager rent collection for less noise (#13275) 2020-10-29 14:16:38 +00:00
Ryo Onodera 4698ee5e4a
Improve final report of ledger-tool capitalization (#13232) 2020-10-28 17:10:09 +00:00
Ryo Onodera 7d2962135d
Use pico inflation for ledger-tool capitalization --enable-inflation (#13215) 
* Use pico inflation for ledger-tool capitalization --enable-inflation

* rust fmt
 2020-10-28 15:30:39 +09:00
Ryo Onodera 6d4c69b7c3
Use zstd for create-snapshot (#13214) 2020-10-28 05:30:25 +00:00
Ryo Onodera 0776fa05c7
Add ledger-tool dead-slots and improve purge a lot (#13065) 
* Add ledger-tool dead-slots and improve purge a lot

* Reduce batch size...

* Add --dead-slots-only and fixed purge ordering
 2020-10-21 17:45:21 +00:00
Michael Vines 11df2e2236 Bump version to v1.5.0 2020-10-08 04:51:36 +00:00
Ryo Onodera 026e7de819
Show commit in `--version` and ledger-tool's log (#12636) 
* Show commit in `--version` and ledger-tool's log

* Another handy hidden env var

* Fix test

* Rename to semver!

* Fix syntax error...
 2020-10-04 00:30:26 +09:00
carllin 06f84c65f1
Fix rooted accounts cleanup, simplify locking (#12194) 
Co-authored-by: Carl Lin <carl@solana.com>
 2020-09-28 16:04:46 -07:00
Justin Starry 731a943239
Remove transaction encoding from storage layer (#12404) 2020-09-24 13:10:29 +08:00
Trent Nelson 325a7e9f86 Move CLI cli_output module to its own crate 2020-09-23 15:58:44 -06:00
Ryo Onodera de4a613610
Check bank capitalization (#11927) 
* Check bank capitalization

* Simplify and unify capitalization calculation

* Improve and add tests

* Avoid overflow and inhibit automatic restart

* Fix test

* Tweak checked sum for cap. and add tests

* Fix broken build after merge conflicts..

* Rename to ClusterType

* Rename confusing method

* Clarify comment

* Verify cap. in rent and inflation tests

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
 2020-09-12 01:48:06 +09:00
Michael Vines 10ce839ec0 Speed up `bigtable transaction-history` command 2020-09-10 05:22:10 +00:00
Michael Vines 8143ea0dea Add `bigtable transaction-history --query-chunk-size` argument 2020-09-09 20:37:20 -07:00
Ryo Onodera 53b8ea4464
Rename to ClusterType and restore devnet compat. (#12068) 
* Rename to ClusterType and restore devnet compat.

* De-duplicate parse code and add comments

* Adjust default Devnet genesis & reduce it in tests
 2020-09-08 23:55:09 +09:00
Michael Vines 2665c5b3c2 Add --show-transactions flag to `bigtable transaction-history` command 2020-09-07 21:34:11 -07:00
Michael Vines d3611f74c8 Add exit flag for bigtable upload operations 2020-09-04 16:01:49 -07:00
Michael Vines 91a56caed2 Relocate BigTable uploader to ledger/ crate 2020-09-04 16:01:49 -07:00
Michael Vines a07980536a Add bank-hash subcommand 2020-08-30 17:09:20 +00:00
Michael Vines 8df263340f modify-genesis now writes elsewhere and produces a full genesis.tar.bz2 2020-08-28 19:58:55 -07:00
Michael Vines df7710a131 Add ability to fork a local cluster from the latest mainnet-beta snapshot 2020-08-28 16:35:27 -07:00
Ryo Onodera 57174cdabe
Make ledger-tool accounts print rent_epoch and slot (#11845) 2020-08-27 12:28:40 +09:00
Michael Vines d1500ae229 The end_slot argument to purge is now optional 2020-08-18 20:17:44 -07:00
Michael Vines adc984a225 Rename Binary64 to Base64. Establish Base58 encoding 2020-08-17 17:26:29 -07:00
Michael Vines b5f3ced860 Add base64 (binary64) encoding for getConfirmedTransaction/getConfirmedBlock 2020-08-17 17:26:29 -07:00
Tyera Eulberg 6c5b8f324a
Rpc: Add until parameter for getConfirmedSignaturesForAddress2 (#11644) 
* Refactor bigtable apis to accept start and end keys

* Make helper fn to deserialize cell data

* Refactor get_confirmed_signatures_for_address to use get_row_data range

* Add until param to get_confirmed_signatures_for_address

* Add until param to blockstore api

* Plumb until through client/cli

* Simplify client params
 2020-08-15 10:42:17 -06:00
carllin 7e25130529
Send votes from banking stage to vote listener (#11434) 
*  Send votes from banking stage to vote listener

Co-authored-by: Carl <carl@solana.com>
 2020-08-07 11:21:35 -07:00
Michael Vines 9abb7db5f8 Upgrade help description 2020-08-05 23:04:16 -07:00
Michael Vines 4222932e08 Plug getConfirmedSignaturesForAddress2 into bigtable storage 2020-08-05 23:04:16 -07:00
Michael Vines 227ea934ff after -> before 2020-08-05 14:27:12 -07:00
Michael Vines 22c46ebf96 Report block status more frequently 2020-08-05 14:27:12 -07:00
Michael Vines dfae9a9864 Plumb Bigtable ledger storage into the RPC subsystem 2020-08-05 14:27:12 -07:00
Michael Vines f9049d6ee4 Add ledger-tool bigtable subcommands 2020-08-05 14:27:12 -07:00
Trent Nelson e12ab9d0dd Bump version to 1.4.0 2020-08-05 12:04:15 -06:00
carllin bf18524368
Add hook for getting vote transactions on replay (#11264) 
* Add hook for getting vote transactions on replay

Co-authored-by: Carl <carl@solana.com>
 2020-07-29 23:17:40 -07:00
Ryo Onodera 6935e88148
Freeze warped bank to show freeze logs (#11141) 2020-07-21 15:51:56 +09:00
Ryo Onodera 3db246f596
Add --warp-epoch and --force-inflation to ledger-tool cap. (#11107) 
* Add --warp-epoch and --force-inflation to ledger-tool cap.

* Add more arguments

* Address review comments

* Fix message

* Fix various typos...
 2020-07-20 19:09:38 +09:00
Tyera Eulberg 9a80e31bae
Expose tss to the other blockstore_processor path (#11070) 2020-07-14 21:14:48 -06:00
Michael Vines bed74d803f Add purge --no-compaction flag 2020-07-14 15:02:01 -07:00
dependabot-preview[bot] 5055028231 Bump serde_json from 1.0.54 to 1.0.56 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.54 to 1.0.56.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.54...v1.0.56)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-07-14 11:13:22 -07:00
sakridge 6429042b6e
Add --print-account-stats/--snapshot-archive-path arguments to ledger-tool (#10945) 
Allows for seeing how the accounts are being stored
and specifying a different snapshot source directory.
 2020-07-08 09:32:11 -07:00
sakridge 58a475b789
Add db recovery methods (#10838) 2020-07-06 12:43:45 -07:00
sakridge 1269e348fb
Add a way to re-hash the bank (#10765) 2020-07-06 12:42:41 -07:00
carllin 823126b372
Fix parsing (#10924) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-06 02:39:42 -07:00
Ryo Onodera b89e506cbb
Consistently create temp dirs under ledger/farf (#10848) 2020-07-01 11:38:59 +09:00
sakridge 1ffad2d051
More guard rails for restart with ledger procedure (#10853) 
* Add expected_bank_hash required for supermajority
* Print snapshot hash in ledger-tool create-snapshot.
 2020-06-30 12:43:48 -07:00
carllin 2dfa48daf9
Add ability to parse logs in ledger-tool (#10840) 
Co-authored-by: Carl <carl@solana.com>
 2020-06-30 02:20:54 -07:00
sakridge 583cec922b
Remove ledger purge batching (#10830) 2020-06-29 14:44:35 -07:00
sakridge ea30c157e0
Better slot printing (#10811) 2020-06-27 10:47:02 -07:00
sakridge aa544b2245
Add ledger copy command (#10756) 2020-06-25 11:56:36 -07:00
Trent Nelson d42247c652
ledger-tool: Ignore SIGUSR1 (#10730) 
Prevents warehouse archive calls getting KO'd by logrotate
 2020-06-21 18:08:17 +00:00
Kristofer Peterson 6d81eede93
Add CLI options and runtime support for selection of output snapshot version. (#10536) 2020-06-19 14:38:37 +09:00
Greg Fitzgerald 6ee222363e
Move BankForks to solana_runtime (#10637) 
* Move BankForks to solana_runtime

* Update imports
 2020-06-17 15:27:03 +00:00
R. M. Shea 48febf153b
Change documentation introduction to indicate Foundation IP transfer (#10440) 2020-06-13 14:41:05 -07:00
dependabot-preview[bot] e5a2c75fab Bump serde_yaml from 0.8.12 to 0.8.13 
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.12 to 0.8.13.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.12...0.8.13)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-13 09:48:15 -07:00
dependabot-preview[bot] a264a9dd0b Bump serde_json from 1.0.53 to 1.0.54 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.53 to 1.0.54.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.53...v1.0.54)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-09 16:57:46 -07:00
Trent Nelson b329e6d1a4 Add --warp-slot argument to |solana-ledger-tool create-snapshot| 2020-06-09 09:16:21 -07:00
Ryo Onodera b515cc3ae5
Don't share same snapshot dir for secondary access (#10384) 
automerge
 2020-06-03 02:45:48 -07:00
Ryo Onodera caa7f7a0c9
Support opening an in-use rocksdb as secondary (#10209) 
automerge
 2020-06-02 21:32:44 -07:00
Michael Vines 1303c4964b Surface more helpful error message than 'blockstore error' 2020-06-02 11:11:06 -07:00
Michael Vines 9eefb2c929 Bump version to 1.3.0 2020-05-28 15:01:13 -07:00
Michael Vines 5ac2ae1178
Purge next slots to avoid a blockstore_processor panic on restart (#10281) 2020-05-27 19:55:40 -07:00
Michael Vines 156387aba4
LedgerCleanupService no longer causes an OOM and actually purges (#10199) 
* cleanup_ledger() now services new_root_receiver while purging
* purge_slots() now fully deletes before compacting
* Add ledger pruning grafana graph
 2020-05-24 21:41:54 -07:00
Ryo Onodera 8d32441b96
Fix ledger-tool create-snapshot and add sanity test (#10212) 
automerge
 2020-05-24 08:27:36 -07:00
Greg Fitzgerald d9919b99d2
Remove folds (#10128) 
automerge
 2020-05-19 18:13:41 -07:00
Kristofer Peterson 58ef02f02b
9951 clippy errors in the test suite (#10030) 
automerge
 2020-05-15 09:35:43 -07:00
Michael Vines 2521f75c18
Advertise node software version in gossip (#9981) 
* Advertise node version in gossip

* Remove solana_clap_utils::version! macro
 2020-05-11 15:02:01 -07:00
dependabot-preview[bot] 6660e93c39
Bump clap from 2.33.0 to 2.33.1 (#9978) 
* Bump clap from 2.33.0 to 2.33.1

Bumps [clap](https://github.com/clap-rs/clap) from 2.33.0 to 2.33.1.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/v2.33.1/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v2.33.0...v2.33.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2020-05-11 14:32:50 -07:00
dependabot-preview[bot] 9c0b80ea1b
Bump serde_yaml from 0.8.11 to 0.8.12 (#9966) 
automerge
 2020-05-10 18:11:23 -07:00
dependabot-preview[bot] 0541431ea8
Bump serde_json from 1.0.52 to 1.0.53 (#9957) 
* Bump serde_json from 1.0.52 to 1.0.53

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.52 to 1.0.53.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.52...v1.0.53)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2020-05-10 11:46:29 -07:00
Ryo Onodera 00e45ec935
Maintain sysvar balances for consistent market cap. (#9936) 
* Maintain sysvar balances for consistent market cap.

* Unindent
 2020-05-09 02:42:32 +09:00
Michael Vines 09ae61651a
Eliminate BankForksInfo (#9887) 2020-05-06 08:24:59 -07:00
Ryo Onodera a91236012d
Pass around --max-genesis-archive-unpacked-size (#9161) 
automerge
 2020-04-29 18:53:34 -07:00
Michael Vines c11abf88b7
Clean up `use` to keep rust 1.43.0 from complaining (#9740) 2020-04-27 16:54:11 -07:00
sakridge fa20963b93
Revert shred fs (#9712) 
* Revert "Untar is called for shred archives that do not exist. (#9565)"

This reverts commit 729cb5eec6.

* Revert "Dont insert shred payload into rocksdb (#9366)"

This reverts commit 5ed39de8c5.
 2020-04-24 15:04:23 -07:00
dependabot-preview[bot] 64e5684d45 Bump bs58 from 0.3.0 to 0.3.1 
Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-04-21 08:07:00 -07:00
Michael Vines 914b022663
cli: Add transaction-history (#9614) 
automerge
 2020-04-20 22:01:09 -07:00
Michael Vines 47ae57610a Only build x86_64-unknown-linux-gnu on docs.rs 2020-04-16 19:06:17 -07:00
anatoly yakovenko 5ed39de8c5
Dont insert shred payload into rocksdb (#9366) 
automerge
 2020-04-16 18:20:55 -07:00
Michael Vines 7e7cbec8a1 Passing -v/--verbose to `solana confirm` now displays the full transaction 2020-04-16 08:19:30 -07:00
Michael Vines 3f1399cb0d accounts subcommand now prints account balances in SOL instead of lamports 2020-04-12 10:09:59 -07:00
Michael Vines d5a9ee97f2
Add --allow-dead-slots argument to `slot`/`print`/`json` commands (#9408) 
automerge
 2020-04-09 20:10:51 -07:00
Michael Vines f3afe5c99c
Remove dead code (#9404) 
automerge
 2020-04-09 13:09:59 -07:00
Ryo Onodera cb6cf189b4
Improve ledger-tool/accounts for easier debuging (#9370) 
automerge
 2020-04-08 10:25:46 -07:00
sakridge 8ed05c27f2
Improve ledger-tool help (#9365) 2020-04-07 19:21:31 -07:00
dependabot-preview[bot] 62cb2cd13c Bump serde_json from 1.0.49 to 1.0.51 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.49 to 1.0.51.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.49...v1.0.51)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-04-06 08:40:55 -07:00
Tyera Eulberg 49e2cc6593
Rework TransactionStatus index in blockstore (#9281) 
automerge
 2020-04-04 20:24:06 -07:00
sakridge 9493de4443
Add snapshot compression option (#9276) 2020-04-03 13:13:49 -07:00
dependabot-preview[bot] 2ed3e2160d
Bump serde_json from 1.0.48 to 1.0.49 (#9139) 
* Bump serde_json from 1.0.48 to 1.0.49

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.48 to 1.0.49.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.48...v1.0.49)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Update serde_json in programs

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Justin Starry <justin@solana.com>
 2020-03-28 12:19:40 -07:00
Michael Vines d7fa40087c
Bump version to 1.2.0 2020-03-28 09:44:13 -07:00
dependabot-preview[bot] a9564d207b
Bump assert_cmd from 0.12.1 to 1.0.0 (#9104) 
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd) from 0.12.1 to 1.0.0.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases)
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v0.12.1...v1.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-27 09:57:01 -06:00
Ryo Onodera f987c18a7e
Strictly validate the contents of snapshot/genesis (#8959) 
automerge
 2020-03-25 02:46:41 -07:00
Michael Vines 7d6ea6c17e
ledger-tool can now decode stake instructions (#9045) 
automerge
 2020-03-24 05:23:29 -07:00
Michael Vines 70c167182a
ledger tool now outputs transaction status information if available (#9024) 
automerge
 2020-03-23 12:49:21 -07:00
Michael Vines ff2c183ac1
Add set-dead-slot command (#9008) 2020-03-21 21:43:33 -07:00
dependabot-preview[bot] fad08a19cc
Bump serde_json from 1.0.46 to 1.0.48 (#8260) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.46 to 1.0.48.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.46...v1.0.48)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-09 20:20:19 -06:00
sakridge de34187db0
Add purge function to ledger-tool (#8719) 2020-03-08 12:40:56 -07:00
Michael Vines acb23e8ef0
Groom ledger-tool bounds output (#8710) 2020-03-07 09:05:15 -07:00
Michael Vines a08e2cc434
nit: clean up MessageHeader output 2020-03-04 00:16:19 -07:00
Michael Vines 13551885c2 --wait-for-supermajority now requires a SLOT 2020-03-02 12:59:35 -07:00
Michael Vines 0d4cb252c4 Adapt local-cluster/ 2020-02-26 23:32:53 -07:00
Michael Vines fcabc6f799 Rename snapshot.tar.bz2 to snapshot-<slot>-<hash>.tar.bz2 2020-02-26 23:32:53 -07:00
Michael Vines fbf78b83c4 Add retry mechanism when downloading genesis and snapshots 2020-02-26 14:21:37 -07:00
Michael Vines 89baa94002 Drop print- prefix from slot/accounts command 2020-02-24 14:46:12 -07:00
Michael Vines 1ef3478709 Add genesis subcommand 2020-02-24 14:46:12 -07:00
Michael Vines 73063544bd Move shred_version module to sdk/ 2020-02-24 14:46:12 -07:00
Dan Albert 2d9d2f1e99
Update cargo versions from 1.0 to 1.1 (#8397) 2020-02-21 23:09:45 -08:00
sakridge b7386f9d84
Add --trusted-validator support for snapshot hash validation (#8390) 2020-02-21 18:42:24 -08:00
Michael Vines 48f58a88bc Bump version to 1.0.0 2020-02-20 23:52:19 -07:00
Ryo Onodera d238371b0c
Correct missing entry handling to avoid bad warns (#8339) 
* Correct missing entry handling to avoid bad warns

* Pass storage entries to AccountStorageSerialize

* Fix CI.....

* Add tests and reorder condition for cheapest first

* Remove unneeded reference
 2020-02-21 15:27:55 +09:00
Michael Vines 18fd52367e
If the node was loaded from a snapshot, advertise it in gossip (#8364) 
automerge
 2020-02-20 18:53:26 -08:00
Michael Vines 1bf2285fa2 ledger-tool: Add print-accounts command 2020-02-14 19:59:48 -07:00
Michael Vines 04ef977509
Remove unwanted println 2020-02-07 12:59:00 -07:00
Michael Vines 8b5598fabd
Surface shred version more in tools (#8163) 
automerge
 2020-02-07 08:57:54 -08:00
François Garillot f016c9a669
Maintenance : simplify a few patterns, remove unneeded dependencies (#8137) 
* Simplify a few pattern matches

* Removing unneeded dependencies, upgrading internal version #s

 net-shaper: Removing log, semver, serde_derive
 bench-tps: Removing serde, serde_derive
 banking-bench: Removing solana
 ledger-tool: Removing bincode, serde, serde_derive
 librapay: Removing solana, language_e2e_tests
 log-analyzer: Removing log, semver, serde_derive
 exchange: Removing solana
 core: Removing crc, memmap, symlink, untrusted
 perf: Removing serde_derive
 genesis: Removing hex, serde_derive
 sdk-c: Removing sha2
 sys-tuner: Removing semver
 bench-exchange: Removing bincode, bs58, env_logger, serde, serde_derive, untrusted, ws
 btc_spv_bin: Removing serde_json
 btc_spv: Removing chrono
 bpf_loader: Removing serde
 ledger: Removing dlopen, dlopen_derive, serde_derive
 move_loader: Removing byteorder, libc, language_e2e_tests
 ownable: Removing serde, serde_derive
 client: Removing rand
 archiver-utils: Removing rand_chacha
 validator: Removing serde_json, tempfile
 param_passing_dep: Removing solana
 failure: Removing log
 vest: Removing log
 vote-signer: Removing bs58, serde
 local-cluster: Removing symlink
 keygen: Removing rpassword
 install: Removing bs58, log
 upload-perf: Removing log
 runtime: Removing serde_json
 stake: Removing rand

* Add modified Cargo.lock

* fixup! Simplify a few pattern matches

* fixup! Simplify a few pattern matches
 2020-02-06 10:02:38 -07:00
Jack May 8436457e75
Rename program_utils.rs (#8127) 2020-02-05 12:48:30 -08:00
dependabot-preview[bot] 65c24db83c
Bump serde_json from 1.0.44 to 1.0.46 (#8087) 
* Bump serde_json from 1.0.44 to 1.0.46

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.44 to 1.0.46.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.44...v1.0.46)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Update Move's serde_json to v1.0.46

Co-authored-by: Jack May <jack@solana.com>
 2020-02-04 16:59:23 -08:00
Dan Albert c324e71768
Bump cargo toml versions to 0.24.0 (#7976) 2020-01-25 11:04:27 -06:00
Michael Vines 989355e885 Add ability to hard fork at any slot (#7801) 
automerge
 2020-01-24 17:27:04 -08:00
Michael Vines bda5f949bb Add create-snapshot command 2020-01-23 22:21:27 -07:00
Michael Vines 6f5e0cd161 Type grooming 2020-01-23 13:37:13 -07:00
Michael Vines 1269a79a4d Unify ledger_path arg handling with validator/ 2020-01-23 13:37:13 -07:00