Commit Graph

422 Commits

Author SHA1 Message Date
Jon Cinque 73aa004c59
stake: Remove v2 program references (#19308)
* stake: Remove v2 program references

* Remove stake v2 feature, along with stake rewrite
2021-08-20 01:08:44 -04:00
Justin Starry c50b01cb60
Store versioned transactions in the ledger, disabled by default (#19139)
* Add support for versioned transactions, but disable by default

* merge conflicts

* trent's feedback

* bump Cargo.lock

* Fix transaction error encoding

* Rename legacy_transaction method

* cargo clippy

* Clean up casts, int arithmetic, and unused methods

* Check for duplicates in sanitized message conversion

* fix clippy

* fix new test

* Fix bpf conditional compilation for message module
2021-08-17 15:17:56 -07:00
Jeff Washington (jwash) 7c70f2158b
accounts_index_bins to AccountsIndexConfig (#19257)
* accounts_index_bins to AccountsIndexConfig

* rename param bins -> config

* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
Jeff Washington (jwash) e91988c977
cli for num account index bins (#19085) 2021-08-11 11:45:25 -05:00
Brooks Prumo fd937548a0
Move SnapshotArchiveInfo and friends into its own module (#19114) 2021-08-08 07:57:06 -05:00
Brooks Prumo 00890957ee
Add snapshot_utils::bank_from_latest_snapshot_archives() (#18983)
While reviewing PR #18565, as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots.  A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify.  Additionally, new tests have been
written and existing tests have been updated to use this new function.

Fixes #18973

While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled.  Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`.  And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats.  This bundling simplifies bank
rebuilding.
2021-08-06 20:16:06 -05:00
Brooks Prumo ca14475085
Add incremental_snapshot_archive_interval_slots to SnapshotConfig (#19026)
This commit also renames `snapshot_interval_slots` to
`full_snapshot_archive_interval_slots`, updates the comments on the
fields, and make appropriate updates where SnapshotConfig is used.
2021-08-04 14:40:20 -05:00
Brooks Prumo d1debcd971
Add incremental snapshot utils (#18504)
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks.  This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.

Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.

Also of note, some renaming has happened:
  1. Snapshots are now either `full_` or `incremental_` throughout the
     codebase.  If not specified, the code applies to both.
  2. Bank snapshots now are called "bank snapshots"
     (before they were called "slot snapshots", "bank snapshots", or
      just "snapshots").  The one exception is within `Bank`, where they
     are still just "snapshots", because they are already "bank
     snapshots".
  3. Snapshot archives now have `_archive` in the code.  This
     should clear up an ambiguity between bank snapshots and snapshot
     archives.
2021-07-22 14:40:37 -05:00
Justin Starry d166b9856a
Move transaction sanitization earlier in the pipeline (#18655)
* Move transaction sanitization earlier in the pipeline

* Renamed HashedTransaction to SanitizedTransaction

* Implement deref for sanitized transaction

* bring back process_transactions test method

* Use sanitized transactions for cost model calculation
2021-07-15 22:51:27 -05:00
Trent Nelson 3a85b77bb5 hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade 2021-07-15 18:43:55 +00:00
Trent Nelson 568660b402 Revert "Remove feature switch for secp256k1 program (#18467)"
This reverts commit fd574dcb3b.
2021-07-15 18:43:55 +00:00
sakridge 7f2254225e
Move entry/poh to own crate to speed up poh bench build (#18225) 2021-07-14 14:16:29 +02:00
Tao Zhu 350baece21
Explicitly sanitize program id indexes before usage
1. check transaction has valid program_id before using it to avoid possible panic;
2. change calculate_cost function signature to return Result;
3. add CostModelError enum, update return type from Result<_, str> to Result<_, CostModelError>
2021-07-13 17:29:22 -05:00
Jeff Washington (jwash) d092fa1f03
add ledger-tool verify verify-accounts-index option (#18375)
* add ledger-tool verify verify-accounts-index option

* comment, merge, respond to feedback, cleanup
2021-07-13 11:06:18 -05:00
Michael Vines 4098af3b5b Record vote account commission with voting/staking rewards and surface in RPC 2021-07-12 15:09:44 -07:00
Justin Starry fd574dcb3b
Remove feature switch for secp256k1 program (#18467)
* Remove feature switch for secp256k1 program

* fix tests
2021-07-09 10:08:03 -05:00
Tao Zhu b6dff12923
update ledger tool to restore cost table from blockstore (#18489)
* update ledger tool to restore cost model from blockstore when compute-slot-cost

* Move initialize_cost_table into cost_model, so the function can be tested and shared between validator and ledger-tool

* refactor and simplify a test
2021-07-07 23:44:51 -05:00
Tao Zhu 0e039b4094
Aggregate cost_model into cost_tracker (#18374)
* * aggregate cost_model into cost_tracker, decouple it from banking_stage to prevent accidental deadlock. * Simplified code, removed unused functions

* review fixes
2021-07-06 15:41:25 +00:00
carllin 0eca92de18
Make set roots an iterator (#18357) 2021-07-01 20:02:40 -07:00
Brooks Prumo 89a3e4f91e
Move SnapshotConfig into its own module (#18331)
Also move ArchiveFormat to snapshot_utils, and do not
reexport SnapshotVersion.
2021-07-01 08:55:26 -05:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Jon Cinque 1b1d34da59
Refactor stake program into solana_program (#17906)
* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt
2021-06-15 18:04:00 +02:00
carllin ccc013e134
Handle removing slots during account scans (#17471) 2021-06-14 21:04:01 -07:00
Jeff Washington (jwash) d4cc975fe9
calculate_capitalization uses hash calculation (#17443)
* calculate_capitalization uses hash calculation

* feedback

* remove debugging code, clean up slot math
2021-06-14 08:53:07 -05:00
Tao Zhu 3a12f92c34
Add subcommand "compute-slot-cost" to ledger tool (#17181)
* include cost_model and cost_tracker from banking stage

* add positional subcommand, compute-slot-cost, to ledger-tool to run cost-model on slots in ledger

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-06-02 11:29:02 -05:00
Michael Vines 9541411c15 Plumb transaction-level rewards (aka "rent debits") into the `getTransaction` RPC method 2021-05-27 03:05:05 +00:00
Jeff Washington (jwash) 6b9d8d41a3
add --limit_load_slot_count_from_snapshot to ledger-tool (#17417) 2021-05-26 10:36:12 -05:00
Jeff Washington (jwash) 99313a4ba2
add ledger-tool verify option to verify hash calculation (#17412) 2021-05-24 18:15:57 -05:00
Lijun Wang 9c42a89a43
Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
Tyera Eulberg 6e9deaf1bd
Move block-time caching earlier (#17109)
* Require that blockstore block-time only be recognized slot, instead of root

* Move cache_block_time to after Bank freeze

* Single use statement

* Pass transaction_status_sender by reference

* Remove unnecessary slot-existence check before caching block time altogether

* Move block-time existence check into Blockstore::cache_block_time, Blockstore no longer needed in blockstore_processor helper
2021-05-10 13:14:56 -06:00
Tyera Eulberg ddfbae260f
Add ledger-tool for restoring roots to the Roots CF (#17045)
* Add ledger-tool for restoring roots to the Roots CF

* Print successful repair data, and repair in chunks

* Add parameter to limit num slots checked for root repair
2021-05-06 14:12:01 -06:00
Ryo Onodera d19526e6c2
Dump rent_collector/inflation with ledger-tool cap (#17069) 2021-05-06 19:29:46 +09:00
Jeff Washington (jwash) 82e11588a5
lamports -> lamports() (#16915) 2021-04-28 14:55:55 -05:00
Jeff Washington (jwash) da3342759b
private AccountSharedData.rent_epoch (#16877) 2021-04-28 08:52:20 -05:00
Jeff Washington (jwash) 2f5102587c
owner() lifetime issue (#16875) 2021-04-27 12:57:47 -05:00
Jeff Washington (jwash) 998cba74b5
AccountSharedData.executable() (#16835) 2021-04-27 09:12:17 -05:00
Jeff Washington (jwash) ca14c18998
owner -> owner() (#16782) 2021-04-23 22:49:47 +00:00
Jeff Washington (jwash) fc12841d95
Readonlyaccounts (#16743)
* lamports -> lamports()

* format
2021-04-22 20:04:55 +00:00
Jeff Washington (jwash) 333998d008
.lamports = <number> -> .set_lamports(<number>) (#16746) 2021-04-22 18:56:47 +00:00
Michael Vines a911ae00ba clippy 2021-04-18 20:55:02 -07:00
François Garillot b08cff9e77
Simplify some pattern-matches (#16402)
When those match an exact combinator on Option / Result.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
2021-04-08 12:40:37 -06:00
carllin 4e5ef6bce2
Add cluster state verifier logging (#16330)
* Add cluster state verifier logging

* Add duplicate-slots iterator to ledger tool
2021-04-02 21:48:44 -07:00
Ryo Onodera 4bbeb9c033
Remove old feature: simple_capitalization (#15763)
* Remove old feature: simple_capitalization

* Fix another failing test in core

* Finish up test cleanup

* Further clean up a bit
2021-03-12 11:12:40 +09:00
Michael Vines aa2b2d6b75 Default --ledger arg to "ledger" for `solana-validator` and `solana-ledger-tool` 2021-03-11 13:09:30 -08:00
Michael Vines c836cd85c3 Default to the BPF JIT. Use the `solana-validator --no-bpf-jit` flag to disable 2021-03-10 08:37:35 -08:00
Jeff Washington (jwash) 52e54e1100
account.data -> data() (#15778) 2021-03-09 22:31:33 +00:00
Jeff Washington (jwash) 8a3135d17b
Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
Ryo Onodera 8b0c6db871
Remove old feature: cumulative_rent_related_fixes (#15754) 2021-03-08 09:58:50 +09:00
Michael Vines e92cff3efa create-snapshot subcommad now accepts the ROOT keyword 2021-02-26 16:40:10 -08:00
Michael Vines 501696243f --help cleanup 2021-02-26 16:40:10 -08:00
carllin a6b9327cd0
Fix root scan in ledger tool (#15532) 2021-02-25 15:52:16 -08:00
Tyera Eulberg d521dfe63c
Implement OutputFormat for confirm in Cli and ledger-tool bigtable (#15528)
* Add CliTransaction struct

* Impl DisplayFormat for decode-transaction

* Add block-time to transaction println, writeln

* Impl DisplayFormat for confirm

* Use DisplayFormat in ledger-tool bigtable confirm
2021-02-25 14:15:52 -07:00
Tyera Eulberg d5f235d997
Implement OutputFormat for block in Cli and ledger-tool bigtable (#15524)
* Impl DisplayFormat for solana block

* Use DisplayFormat in ledger-tool bigtable block
2021-02-24 16:14:34 -07:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Ryo Onodera 4e99aa5fa6
More failure codepath tracing (#15246) 2021-02-12 15:24:23 +09:00
sakridge 91b374da5a
Fix typo (#15190) 2021-02-08 20:37:33 +00:00
sakridge bbae23358c
ledger-tool cleanup and additions (#15179)
* Plumb allow-dead-slots to ledger-tool verify

* ledger-tool cleanup and add some useful missing args

Print root slots and how many unrooted past last root.
2021-02-06 17:26:42 -08:00
sakridge 11b84cb870
Add jit and caching args to ledger-tool (#15177) 2021-02-06 19:49:24 +00:00
Jeff Washington (jwash) fbf9dc47e9
Threadpool2 (#15151)
* rework thread pool for hash calculation

* rename
2021-02-05 18:48:16 -06:00
Jeff Washington (jwash) fabecdc86c
use thread pool for non-index hash calculations (#15149) 2021-02-05 19:48:55 +00:00
Trent Nelson a2aea0ca33 cli-output: Add option sigverify status to `println_transaction()` output 2021-02-02 18:44:22 +00:00
Michael Vines 709aa74e11 Adapt create-snapshot to avoid triggering recent internal bank sanity checks 2021-02-01 18:29:05 -08:00
Michael Vines 164fae44df /i/o/ 2021-01-31 08:22:01 -08:00
Michael Vines cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
Michael Vines bf1943e489 Add solana-test-validator --warp-slot argument 2021-01-22 21:17:02 -08:00
Michael Vines dd5a2ef05f Add convenience function to create a snapshot archive out of any Bank 2021-01-22 21:17:02 -08:00
carllin 6dfad0652f
Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
Michael Vines 7be6770808 Rename CompressionType to ArchiveFormat 2021-01-09 09:07:49 -08:00
sakridge baa9602411
Add shrink paths (#14238) 2020-12-21 21:33:37 -08:00
Michael Vines 0b92720fdb Don't require increased open file limit in solana-test-validator
Travis CI in particular does not allow the open file limit to be
increased.
2020-12-16 20:43:08 -08:00
Michael Vines 7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
Ryo Onodera de9ac43ebf
Simple cap by including sysvars and native programs (#13884)
* Simpler cap by including sysvars and native programs

* Fix tests

* Add comment

* revert some unrelated code

* Update test_bank_update_sysvar_account for cap.

* Test cap. for add_native_program using new helper

* Improve the cap adjustment with new tests

* Fix typo...

* Adjust test for improved code coverage

* Rename simpler_capitalization => simple_capitalization

* More rename and bonus commenting
2020-12-14 21:03:24 +09:00
behzad nouri f6a7732ff9
adds ledger-tool mode to print raw shred metadata (#13962) 2020-12-08 17:31:24 +00:00
Ryo Onodera 82c75c3786
genesis, ledger-tool: --inflation/--vote-commission-percentaage (#13989) 2020-12-07 16:21:16 +00:00
Michael Vines c59cb28bbf Add --remove-account option to create-snapshot command 2020-12-05 08:57:00 +00:00
Ryo Onodera 480dd15b1e
Ledger tool cap basic columns (#13904)
* ledger-tool cap: Add more basic columns

* simplify InflationRecord::rewarded_epoch type
2020-12-02 07:10:03 +00:00
behzad nouri e1793e5a13
caches vote-state de-serialized from vote accounts (#13795)
Gossip and other places repeatedly de-serialize vote-state stored in
vote accounts. Ideally the first de-serialization should cache the
result.

This commit adds new VoteAccount type which lazily de-serializes
VoteState from Account data and caches the result internally.

Serialize and Deserialize traits are manually implemented to match
existing code. So, despite changes to frozen_abi, this commit should be
backward compatible.
2020-11-30 17:18:33 +00:00
Ryo Onodera 6048342c57
Enhance ledger-tool for multi-epoch rewards (#13837)
* Support ledger-tool for multi-epoch rewards

* nits

* Ensure not to skip some records in csv
2020-11-27 04:20:47 +00:00
Ryo Onodera 3bc7d85986
ledger-tool cap: output credits_observed (#13746) 2020-11-21 16:13:07 +00:00
Ryo Onodera 43d5e47ea9
Rewrite stake accounts for clear migration (#13461)
* Reduce overage stake by rewritng stake accounts

* Write tests and finish implemention

* Create and use new feature gate

* Clean up logging

* Fix typo

* Simplify enable_rewrite_stake

* Fix typo...

* Even simplify gating

* Add metrics
2020-11-20 05:15:06 +09:00
Ryo Onodera bcd303a447
ledger-tool cap: delegation owner and stake v2 flag (#13602)
* Output delegation owner as well

* Add --enable-stake-program-v2

* Small cleanup and add sanity assertion

* Fix typo...
2020-11-15 19:38:46 +00:00
Ryo Onodera a81e7e7749
Output more inflation calc details in ledger-tool (#13345)
* Output more inflation calc details in ledger-tool

* Fix broken ci...

* Rename confusing variables

* Fix panic by wrapping PointValue with Opiton...

* Minor modifications

* Remove explict needless flush; Drop already does

* Yet another csv field adjustments

* Add data_size and rename epochs to earned_epochs

* Introduce null_tracer

* Unwrap Option in new_from_parent_with_tracer

* Don't shorten identifiers

* Allow irrefutable_let_patterns temporalily

* More null_tracer

* More field adjustments
2020-11-08 16:43:50 +09:00
Ryo Onodera 363c148dbe
Disable eager rent collection for less noise (#13275) 2020-10-29 14:16:38 +00:00
Ryo Onodera 4698ee5e4a
Improve final report of ledger-tool capitalization (#13232) 2020-10-28 17:10:09 +00:00
Ryo Onodera 7d2962135d
Use pico inflation for ledger-tool capitalization --enable-inflation (#13215)
* Use pico inflation for ledger-tool capitalization --enable-inflation

* rust fmt
2020-10-28 15:30:39 +09:00
Ryo Onodera 6d4c69b7c3
Use zstd for create-snapshot (#13214) 2020-10-28 05:30:25 +00:00
Ryo Onodera 0776fa05c7
Add ledger-tool dead-slots and improve purge a lot (#13065)
* Add ledger-tool dead-slots and improve purge a lot

* Reduce batch size...

* Add --dead-slots-only and fixed purge ordering
2020-10-21 17:45:21 +00:00
Ryo Onodera 026e7de819
Show commit in `--version` and ledger-tool's log (#12636)
* Show commit in `--version` and ledger-tool's log

* Another handy hidden env var

* Fix test

* Rename to semver!

* Fix syntax error...
2020-10-04 00:30:26 +09:00
carllin 06f84c65f1
Fix rooted accounts cleanup, simplify locking (#12194)
Co-authored-by: Carl Lin <carl@solana.com>
2020-09-28 16:04:46 -07:00
Trent Nelson 325a7e9f86 Move CLI cli_output module to its own crate 2020-09-23 15:58:44 -06:00
Ryo Onodera de4a613610
Check bank capitalization (#11927)
* Check bank capitalization

* Simplify and unify capitalization calculation

* Improve and add tests

* Avoid overflow and inhibit automatic restart

* Fix test

* Tweak checked sum for cap. and add tests

* Fix broken build after merge conflicts..

* Rename to ClusterType

* Rename confusing method

* Clarify comment

* Verify cap. in rent and inflation tests

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
2020-09-12 01:48:06 +09:00
Ryo Onodera 53b8ea4464
Rename to ClusterType and restore devnet compat. (#12068)
* Rename to ClusterType and restore devnet compat.

* De-duplicate parse code and add comments

* Adjust default Devnet genesis & reduce it in tests
2020-09-08 23:55:09 +09:00
Michael Vines a07980536a Add bank-hash subcommand 2020-08-30 17:09:20 +00:00
Michael Vines 8df263340f modify-genesis now writes elsewhere and produces a full genesis.tar.bz2 2020-08-28 19:58:55 -07:00
Michael Vines df7710a131 Add ability to fork a local cluster from the latest mainnet-beta snapshot 2020-08-28 16:35:27 -07:00
Ryo Onodera 57174cdabe
Make ledger-tool accounts print rent_epoch and slot (#11845) 2020-08-27 12:28:40 +09:00
Michael Vines d1500ae229 The end_slot argument to purge is now optional 2020-08-18 20:17:44 -07:00
carllin 7e25130529
Send votes from banking stage to vote listener (#11434)
*  Send votes from banking stage to vote listener

Co-authored-by: Carl <carl@solana.com>
2020-08-07 11:21:35 -07:00
Michael Vines f9049d6ee4 Add ledger-tool bigtable subcommands 2020-08-05 14:27:12 -07:00
carllin bf18524368
Add hook for getting vote transactions on replay (#11264)
* Add hook for getting vote transactions on replay

Co-authored-by: Carl <carl@solana.com>
2020-07-29 23:17:40 -07:00
Ryo Onodera 6935e88148
Freeze warped bank to show freeze logs (#11141) 2020-07-21 15:51:56 +09:00
Ryo Onodera 3db246f596
Add --warp-epoch and --force-inflation to ledger-tool cap. (#11107)
* Add --warp-epoch and --force-inflation to ledger-tool cap.

* Add more arguments

* Address review comments

* Fix message

* Fix various typos...
2020-07-20 19:09:38 +09:00
Tyera Eulberg 9a80e31bae
Expose tss to the other blockstore_processor path (#11070) 2020-07-14 21:14:48 -06:00
Michael Vines bed74d803f Add purge --no-compaction flag 2020-07-14 15:02:01 -07:00
sakridge 6429042b6e
Add --print-account-stats/--snapshot-archive-path arguments to ledger-tool (#10945)
Allows for seeing how the accounts are being stored
and specifying a different snapshot source directory.
2020-07-08 09:32:11 -07:00
sakridge 58a475b789
Add db recovery methods (#10838) 2020-07-06 12:43:45 -07:00
sakridge 1269e348fb
Add a way to re-hash the bank (#10765) 2020-07-06 12:42:41 -07:00
carllin 823126b372
Fix parsing (#10924)
Co-authored-by: Carl <carl@solana.com>
2020-07-06 02:39:42 -07:00
Ryo Onodera b89e506cbb
Consistently create temp dirs under ledger/farf (#10848) 2020-07-01 11:38:59 +09:00
sakridge 1ffad2d051
More guard rails for restart with ledger procedure (#10853)
* Add expected_bank_hash required for supermajority
* Print snapshot hash in ledger-tool create-snapshot.
2020-06-30 12:43:48 -07:00
carllin 2dfa48daf9
Add ability to parse logs in ledger-tool (#10840)
Co-authored-by: Carl <carl@solana.com>
2020-06-30 02:20:54 -07:00
sakridge 583cec922b
Remove ledger purge batching (#10830) 2020-06-29 14:44:35 -07:00
sakridge ea30c157e0
Better slot printing (#10811) 2020-06-27 10:47:02 -07:00
sakridge aa544b2245
Add ledger copy command (#10756) 2020-06-25 11:56:36 -07:00
Trent Nelson d42247c652
ledger-tool: Ignore SIGUSR1 (#10730)
Prevents warehouse archive calls getting KO'd by logrotate
2020-06-21 18:08:17 +00:00
Kristofer Peterson 6d81eede93
Add CLI options and runtime support for selection of output snapshot version. (#10536) 2020-06-19 14:38:37 +09:00
Greg Fitzgerald 6ee222363e
Move BankForks to solana_runtime (#10637)
* Move BankForks to solana_runtime

* Update imports
2020-06-17 15:27:03 +00:00
Trent Nelson b329e6d1a4 Add --warp-slot argument to |solana-ledger-tool create-snapshot| 2020-06-09 09:16:21 -07:00
Ryo Onodera b515cc3ae5
Don't share same snapshot dir for secondary access (#10384)
automerge
2020-06-03 02:45:48 -07:00
Ryo Onodera caa7f7a0c9
Support opening an in-use rocksdb as secondary (#10209)
automerge
2020-06-02 21:32:44 -07:00
Michael Vines 1303c4964b Surface more helpful error message than 'blockstore error' 2020-06-02 11:11:06 -07:00
Michael Vines 5ac2ae1178
Purge next slots to avoid a blockstore_processor panic on restart (#10281) 2020-05-27 19:55:40 -07:00
Michael Vines 156387aba4
LedgerCleanupService no longer causes an OOM and actually purges (#10199)
* cleanup_ledger() now services new_root_receiver while purging
* purge_slots() now fully deletes before compacting
* Add ledger pruning grafana graph
2020-05-24 21:41:54 -07:00
Ryo Onodera 8d32441b96
Fix ledger-tool create-snapshot and add sanity test (#10212)
automerge
2020-05-24 08:27:36 -07:00
Greg Fitzgerald d9919b99d2
Remove folds (#10128)
automerge
2020-05-19 18:13:41 -07:00
Michael Vines 2521f75c18
Advertise node software version in gossip (#9981)
* Advertise node version in gossip

* Remove solana_clap_utils::version! macro
2020-05-11 15:02:01 -07:00
Ryo Onodera 00e45ec935
Maintain sysvar balances for consistent market cap. (#9936)
* Maintain sysvar balances for consistent market cap.

* Unindent
2020-05-09 02:42:32 +09:00
Michael Vines 09ae61651a
Eliminate BankForksInfo (#9887) 2020-05-06 08:24:59 -07:00
Ryo Onodera a91236012d
Pass around --max-genesis-archive-unpacked-size (#9161)
automerge
2020-04-29 18:53:34 -07:00
Michael Vines c11abf88b7
Clean up `use` to keep rust 1.43.0 from complaining (#9740) 2020-04-27 16:54:11 -07:00
sakridge fa20963b93
Revert shred fs (#9712)
* Revert "Untar is called for shred archives that do not exist. (#9565)"

This reverts commit 729cb5eec6.

* Revert "Dont insert shred payload into rocksdb (#9366)"

This reverts commit 5ed39de8c5.
2020-04-24 15:04:23 -07:00
Michael Vines 914b022663
cli: Add transaction-history (#9614)
automerge
2020-04-20 22:01:09 -07:00
anatoly yakovenko 5ed39de8c5
Dont insert shred payload into rocksdb (#9366)
automerge
2020-04-16 18:20:55 -07:00
Michael Vines 7e7cbec8a1 Passing -v/--verbose to `solana confirm` now displays the full transaction 2020-04-16 08:19:30 -07:00
Michael Vines 3f1399cb0d accounts subcommand now prints account balances in SOL instead of lamports 2020-04-12 10:09:59 -07:00
Michael Vines d5a9ee97f2
Add --allow-dead-slots argument to `slot`/`print`/`json` commands (#9408)
automerge
2020-04-09 20:10:51 -07:00
Michael Vines f3afe5c99c
Remove dead code (#9404)
automerge
2020-04-09 13:09:59 -07:00
Ryo Onodera cb6cf189b4
Improve ledger-tool/accounts for easier debuging (#9370)
automerge
2020-04-08 10:25:46 -07:00
sakridge 8ed05c27f2
Improve ledger-tool help (#9365) 2020-04-07 19:21:31 -07:00
Tyera Eulberg 49e2cc6593
Rework TransactionStatus index in blockstore (#9281)
automerge
2020-04-04 20:24:06 -07:00
sakridge 9493de4443
Add snapshot compression option (#9276) 2020-04-03 13:13:49 -07:00
Ryo Onodera f987c18a7e
Strictly validate the contents of snapshot/genesis (#8959)
automerge
2020-03-25 02:46:41 -07:00
Michael Vines 7d6ea6c17e
ledger-tool can now decode stake instructions (#9045)
automerge
2020-03-24 05:23:29 -07:00
Michael Vines 70c167182a
ledger tool now outputs transaction status information if available (#9024)
automerge
2020-03-23 12:49:21 -07:00
Michael Vines ff2c183ac1
Add set-dead-slot command (#9008) 2020-03-21 21:43:33 -07:00
sakridge de34187db0
Add purge function to ledger-tool (#8719) 2020-03-08 12:40:56 -07:00
Michael Vines acb23e8ef0
Groom ledger-tool bounds output (#8710) 2020-03-07 09:05:15 -07:00
Michael Vines a08e2cc434
nit: clean up MessageHeader output 2020-03-04 00:16:19 -07:00
Michael Vines 13551885c2 --wait-for-supermajority now requires a SLOT 2020-03-02 12:59:35 -07:00
Michael Vines 0d4cb252c4 Adapt local-cluster/ 2020-02-26 23:32:53 -07:00
Michael Vines fcabc6f799 Rename snapshot.tar.bz2 to snapshot-<slot>-<hash>.tar.bz2 2020-02-26 23:32:53 -07:00
Michael Vines fbf78b83c4 Add retry mechanism when downloading genesis and snapshots 2020-02-26 14:21:37 -07:00
Michael Vines 89baa94002 Drop print- prefix from slot/accounts command 2020-02-24 14:46:12 -07:00
Michael Vines 1ef3478709 Add genesis subcommand 2020-02-24 14:46:12 -07:00
Michael Vines 73063544bd Move shred_version module to sdk/ 2020-02-24 14:46:12 -07:00
sakridge b7386f9d84
Add --trusted-validator support for snapshot hash validation (#8390) 2020-02-21 18:42:24 -08:00
Ryo Onodera d238371b0c
Correct missing entry handling to avoid bad warns (#8339)
* Correct missing entry handling to avoid bad warns

* Pass storage entries to AccountStorageSerialize

* Fix CI.....

* Add tests and reorder condition for cheapest first

* Remove unneeded reference
2020-02-21 15:27:55 +09:00
Michael Vines 18fd52367e
If the node was loaded from a snapshot, advertise it in gossip (#8364)
automerge
2020-02-20 18:53:26 -08:00
Michael Vines 1bf2285fa2 ledger-tool: Add print-accounts command 2020-02-14 19:59:48 -07:00
Michael Vines 04ef977509
Remove unwanted println 2020-02-07 12:59:00 -07:00
Michael Vines 8b5598fabd
Surface shred version more in tools (#8163)
automerge
2020-02-07 08:57:54 -08:00
Jack May 8436457e75
Rename program_utils.rs (#8127) 2020-02-05 12:48:30 -08:00
Michael Vines 989355e885 Add ability to hard fork at any slot (#7801)
automerge
2020-01-24 17:27:04 -08:00
Michael Vines bda5f949bb Add create-snapshot command 2020-01-23 22:21:27 -07:00
Michael Vines 6f5e0cd161 Type grooming 2020-01-23 13:37:13 -07:00
Michael Vines 1269a79a4d Unify ledger_path arg handling with validator/ 2020-01-23 13:37:13 -07:00
Michael Vines 1b3424ff61 Pass bank_forks by reference 2020-01-23 13:37:13 -07:00
carllin f91ffbbfdf
Add support in BlockStore for tracking duplicate slots (#7761)
* Add test

* Add new column family to track duplicate slots

* Fix clippy errors

* Introduce new SlotColumn for common implementation of Column trait
2020-01-13 17:21:39 -08:00
Greg Fitzgerald b5dba77056 Rename blocktree to blockstore (#7757)
automerge
2020-01-13 13:13:52 -08:00
Jack May 07855e3125
Allow override of RUST_LOG (#7705) 2020-01-08 09:19:12 -08:00
Michael Vines 3f69d58498
ledger-tool: Add --all option to bounds, to display all non-empty slots (#7592) 2019-12-20 20:43:53 -07:00
Sunny Gleason 7392505bd8
Modify ledger-tool to selectively load Blocktree, GenesisHash, Database so commands are more resilient (#7547) 2019-12-18 13:00:15 -05:00
Sunny Gleason 115bf2613d
feat: add analyze-storage command to ledger-tool (#7165) 2019-12-12 18:54:50 -05:00
Justin Starry b7d4330dd4
Fail fast if account paths cannot be canonicalized (#7300)
* Canonicalize account paths to avoid symlink issues

* fixes
2019-12-05 21:41:29 -05:00
Sagar Dhawan 1af4e256c9 Print slot meta when printing a slot (#7133)
automerge
2019-12-03 12:27:43 -08:00
Jack May d184d3a732
Merge native programs parts into one unit (#7047) 2019-11-20 10:12:43 -08:00
Michael Vines e7f63cd336
Upgrade to rust 1.39.0 (#6939)
* Upgrade to rust 1.39.0

* 1.39.0 clippy
2019-11-14 12:27:01 -07:00
Ryo Onodera 4fc767b3f6
Move version! from core:: to clap_utils:: (#6944)
* Move version! from core to clap-utils

* Completely move version! from core:: to clap_utils::

* rustfmt

* Do remaining transition after rebase
2019-11-14 13:10:38 +09:00
Michael Vines 4405e8a15b
Automatically run dot to generate PDFs or PNGs (#6912) 2019-11-12 20:27:15 -07:00
Michael Vines c2fdbde68f
forks graph can now optionally display all validator votes (#6885) 2019-11-12 10:13:16 -07:00
Michael Vines 25866f3652
print command now supports multiple slots and decodes system/vote instructions (#6878) 2019-11-11 23:22:20 -07:00
Michael Vines 24a7b0ce74
Add print-genesis-hash command (#6849) 2019-11-08 23:17:48 -07:00
Justin Starry 9807f47d4e
Rename genesis block to genesis config (#6816) 2019-11-08 23:56:57 -05:00
carllin 24102a7435
Allow voting on empty banks (#6719)
* Allow votes on empty banks

* Remove making first bank is_delta true, no longer necessary for idling

* Remove votable from ledger tool
2019-11-06 01:02:26 -08:00
Michael Vines 09e648f957
ledger-tool/: Include full validator voting history in fork-graph (#6756) 2019-11-05 19:40:00 -07:00
Michael Vines cea13e964c
Add --graph-forks option (#6732) 2019-11-04 23:18:30 -07:00
Michael Vines fba1af6ea9
ledger-tool can now load a ledger snapshot (#6729) 2019-11-04 22:14:55 -07:00
Michael Vines 4d52f47f87
Move get_bank_forks() into ledger/ so its available for use by ledger-tool/ (#6720) 2019-11-04 19:10:06 -07:00
Michael Vines 50a17fc00b Use Slot and Epoch type aliases instead of raw u64 (#6693)
automerge
2019-11-02 00:38:30 -07:00
Michael Vines 3879109e4c
Display full blocktree error 2019-10-25 08:37:39 -07:00
Sagar Dhawan 28d3af6f35
Add "bounds" command to ledger-tool and fix broken funtionality (#6540) 2019-10-24 22:20:52 -07:00
Greg Fitzgerald 2636418659
Move blocktree_processor to solana_ledger (#6460)
* Drop core::result dependency in bank_forks

* Move blocktree_processor into solana_ledger
2019-10-20 09:54:38 -06:00
Greg Fitzgerald 5468be2ef9 Add solana-ledger crate (#6415)
automerge
2019-10-18 09:28:51 -07:00
Justin Starry 5e31565574
Expand blocktree processor options (#6248)
* Refactor blocktree processor args and support full leader cache

* Add entry callback option

* Rename num_threads to override_num_threads

* Add test for entry callback

* Refactor cached leader schedule changes

* Add tests for blocktree process options

* Refactor test

* @mvines feedback
2019-10-08 17:58:49 -04:00
Jack May e8d88f3237
Split SDK's timing.rs (#5823) 2019-09-06 14:30:56 -07:00
Michael Vines 3450b9a44d
Rename solana to solana-core (#5583) 2019-08-21 10:23:33 -07:00
Greg Fitzgerald 471bc73a23
Fix Rust 1.37.0 compiler warnings (#5530)
Looks like most usages of trait objects should have introduced
a type variable instead.
2019-08-15 14:00:09 -06:00
Michael Vines c02373493b Add print-slot subcommand (#5478)
automerge
2019-08-09 15:57:31 -07:00
Michael Vines 8c15214923
Add --dev-halt-at-slot option (#5453) 2019-08-08 09:14:30 -07:00
Michael Vines c78db6a94b
ledger path reform: use Path/PathBuf instead of strings (#5344) 2019-07-30 15:53:41 -07:00
Sagar Dhawan 6ad9dc18d8
Add ability to prune ledger (#5128)
* Add utility to prune the ledger

* Add tests

* Fix clippy

* Fix off by one

* Rework to force delete every column

* Minor fixup
2019-07-17 14:42:29 -07:00
Pankaj Garg 1c966aac25 Facility to generate a blocktree prune list using ledger tool (#5041)
automerge
2019-07-12 16:58:13 -07:00
Michael Vines ebcdc06dc3
Restore ledger-tool print and json commands (#5048)
* Restore ledger-tool print and  json commands

* Remove obsolete read_ledger()
2019-07-11 20:33:36 -07:00
Michael Vines df43e721e3
Exit 1 on ledger verification failure 2019-05-09 12:05:51 -07:00
carllin 512bfc93cb
Add a cache for leader schedules (#3841)
* Add a cache for leader schedules
2019-04-19 02:39:44 -07:00
Michael Vines bb420cb995 Use crate_description and crate_name Clap macros 2019-03-14 00:35:34 -05:00
Michael Vines d32f5b6cca Use process_blocktree to verify the ledger 2019-03-02 08:47:31 -08:00
Michael Vines 7c4473e0aa Rename Entry.id to Entry.hash 2019-03-01 09:31:49 -08:00
Pankaj Garg 5c9777970d moved fee collection code to runtime 2019-02-21 16:35:23 -08:00
Pankaj Garg a33921ed34 address review comments 2019-02-20 08:12:37 -07:00
Greg Fitzgerald dde886f058 Move Bank to its own crate
Also:
* counters.rs to solana_metrics
* genesis_block.rs to solana_sdk
2019-02-19 07:17:04 -07:00
Greg Fitzgerald b539389741 Move all Validator dependencies from Bank to blocktree_processor 2019-02-16 15:01:26 -07:00
carllin 6e7c5f205b
Rename db_ledger to blocktree (#2698) 2019-02-07 20:52:39 -08:00
Michael Vines 4bb6549895 Genesis block is now a json file 2019-01-25 09:05:15 -08:00
Michael Vines c0b472292b Revert "Entries that result in a ProgramError are still valid entries"
This reverts commit ab23b41998128e8d663c81dd5af86a8b82e33b6e.
2019-01-18 21:06:50 -08:00
Michael Vines 4f86563352 Entries that result in a ProgramError are still valid entries 2019-01-18 13:42:04 -08:00
Michael Vines 818afc68c1 Report number of entries and last_id on successful verification 2019-01-18 13:42:04 -08:00
Michael Vines 1f6346d880 De-dup ledgers - db_ledger is now the only ledger written to disk 2019-01-04 16:37:00 -08:00
Kevin Hart 84cc240f34 Enhance ledger-tool
Add a command-line argument (min-hashes) to restrict the entries
processed by ledger-tool.  For example, --min-hashes 1 will strip
"empty" Entries, i.e. those with num_hashes = 0.

Add basic ledger tool test
2019-01-04 08:17:43 -07:00
Michael Vines fd562cb9e2 Rust 2018 cleanup 2018-12-14 21:57:15 -08:00
Michael Vines 6ac466c0a4 Move src/logger.rs into logger/ crate to unify logging across the workspace 2018-12-14 13:10:43 -08:00
Michael Vines 0fe6d61036 Move binaries from src/bin into their own crate 2018-12-13 22:25:27 -08:00