Commit Graph

373 Commits

Author SHA1 Message Date
Lijun Wang 269d995832
Make account shrink configurable #17544 (#17778)
1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544
2021-06-09 21:21:32 -07:00
dependabot[bot] 5c72fef85a
chore: bump signal-hook from 0.1.15 to 0.2.3 (#17827)
Bumps [signal-hook](https://github.com/vorner/signal-hook) from 0.1.15 to 0.2.3.
- [Release notes](https://github.com/vorner/signal-hook/releases)
- [Changelog](https://github.com/vorner/signal-hook/blob/v0.2.3/CHANGELOG.md)
- [Commits](https://github.com/vorner/signal-hook/compare/v0.1.15...v0.2.3)

---
updated-dependencies:
- dependency-name: signal-hook
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-08 12:12:17 -06:00
Govlzkoy e0ab5ee4f8
update dependence version for gag to latest support windows (#17801)
* update dependence version for gag to leatest support windows

* fix compile on windows

* add Cargo.lock
2021-06-07 23:17:16 -07:00
Tyera Eulberg 2e998ed11d
Bump jsonrpc crates (#17797)
* Bump jsonrpc crates

* Update error text

* gitignore for rpc
2021-06-07 18:03:20 +00:00
Tyera Eulberg 544b3c0d17
Create solana-poh and move remaining rpc modules to solana-rpc (#17698)
* Create solana-poh crate

* Move BigTableUploadService to solana-ledger

* Add solana-rpc to workspace

* Move dependencies to solana-rpc

* Move remaining rpc modules to solana-rpc

* Single use statement solana-poh

* Single use statement solana-rpc
2021-06-04 09:23:06 -06:00
Tyera Eulberg 3a647c4bea
Rename ValidatorExit and move to sdk (#17728) 2021-06-04 03:06:13 +00:00
dependabot[bot] a43c29e858
Bump indicatif from 0.15.0 to 0.16.2 (#17628)
* Bump indicatif from 0.15.0 to 0.16.2

Bumps [indicatif](https://github.com/mitsuhiko/indicatif) from 0.15.0 to 0.16.2.
- [Release notes](https://github.com/mitsuhiko/indicatif/releases)
- [Commits](https://github.com/mitsuhiko/indicatif/compare/0.15.0...0.16.2)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Fix message types

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-01 17:38:02 +00:00
dependabot[bot] ab0f4ff835
Bump serde from 1.0.122 to 1.0.126 (#17618)
* Bump serde from 1.0.122 to 1.0.126

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.122 to 1.0.126.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.122...v1.0.126)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-05-31 22:41:25 +00:00
Trent Nelson 974a96738a test-validator: add an arg to control faucet genesis balance 2021-05-28 10:05:37 -06:00
Ryo Onodera 1f97b2365f
Avoid full-range compactions with periodic filtered b.g. ones (#16697)
* Update rocksdb to v0.16.0

* Promote the infrequent and important log to info!

* Force background compaction by ttl without manual compaction

* Fix test

* Support no compaction mode in test_ledger_cleanup_compaction

* Fix comment

* Make compaction_interval customizable

* Avoid major compaction with periodic filtering...

* Adress lazy_static, special cfs and range check

* Clean up a bit and add comment

* Add comment

* More comments...

* Config code cleanup

* Add comment

* Use .conflicts_with()

* Nullify unneeded delete_range ops for special CFs

* Some clean ups

* Clarify the locking intention

* Ensure special CFs' consistency with PurgeType::CompactionFilter

* Fix comment

* Fix bad copy paste

* Fix various types...

* Don't use tuples

* Add a unit test for compaction_filter

* Fix typo...

* Remove flag and just use new behavior always

* Fix wrong condition negation...

* Doc. about no set_last_purged_slot in purge_slots

* Write a test and fix off-by-one bug....

* Apply suggestions from code review

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Follow up to github review suggestions

* Fix line-wrapping

* Fix conflict

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-05-28 16:42:56 +09:00
dependabot[bot] ec1a307a7c
Bump console from 0.11.3 to 0.14.1 (#16301)
* Bump console from 0.11.3 to 0.14.1

Bumps [console](https://github.com/mitsuhiko/console) from 0.11.3 to 0.14.1.
- [Release notes](https://github.com/mitsuhiko/console/releases)
- [Changelog](https://github.com/mitsuhiko/console/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/console/compare/0.11.3...0.14.1)

Signed-off-by: dependabot[bot] <support@github.com>

* Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-05-27 18:55:27 +00:00
Michael Vines 8eab0e8602
Bump version to v1.8.0 (#17541) 2021-05-27 08:51:53 -07:00
Lijun Wang 54f0fc9f0f
Use type alias for DownloadProgress callback (#17518)
Convert to use type alias for the callback and cascade the changes to callers. Thanks @jeffwashington for the help making it possible.
Changed the closure for the progress update in the validator main to FnMut and modify the abort count in the closure which is more reliable.
2021-05-26 13:26:07 -07:00
Tyera Eulberg 9a5330b7eb
Move gossip modules into solana-gossip crate (#17352)
* Move gossip modules to solana-gossip

* Update Protocol abi digest due to move

* Move gossip benches and hook up CI

* Remove unneeded Result entries

* Single use statements
2021-05-26 09:15:46 -06:00
Lijun Wang 4c17243157
snapshot download enhancement (#17415)
1. Allow the validator bootstrap code to specify the minimal snapshot download speed. If the snapshot download speed is detected below that, a different RPC can be retried. The default is 10MB/sec.

2. To prevent spinning on a number of sub-optimal choices and not making progress, the abort/retry logic is implemented with the following safe guards:
2.1 at maximum we do this retry for 5 times -- this number is configurable with default 5.
2.2 if the download in one notification round (5 second) is more than 2%, do not do retry -- it is not as bad anyway.
2.3 if the remaining estimate time is less than 1 minutes, do not abort retry as it will be done quickly anyway.
2.4 We do this abort/retry logic only at the first notification to avoid wasting download efforts -- the reasoning is being opportunistic and too greedy may not achieve overall shorter download time.

3. The download_snapshot and download_file is modified with the option allowing caller to notified of download progress via a callback. This allows the business logic of retrying to the place it belongs.
2021-05-25 09:32:12 -07:00
Tyera Eulberg 41ec1c8d50
Add blockstore-root-scan for api nodes on boot (#17402)
* Add blockstore-root-scan for api nodes on boot

* Ensure cluster-confirmed root and parents are set as root in blockstore in load_frozen_forks()

* Plumb rpc-scan-and-fix-roots validator flag
2021-05-24 13:24:47 -06:00
sakridge a8dca3976b
Refactor genesis download/load/check functions (#17276)
* Refactor genesis ingest functions

* Consolidate genesis.bin/genesis.tar.bz2 references
2021-05-24 16:45:36 +02:00
Michael Vines 305d9dd3f4 Validator progress bars are now rendered when stdout is not a terminal 2021-05-19 07:32:39 +00:00
Tyera Eulberg 827355a6b1
Create solana-rpc crate and move subscriptions (#17320)
* Move non_circulating_supply to runtime

* Add solana-rpc crate and move max_slots

* Move subscriptions to solana-rpc

* Single use statements
2021-05-19 00:54:28 -06:00
Trent Nelson 2c8dde7224 test-validator: Hint at airdrop when wallet is unavailable 2021-05-14 11:14:27 -06:00
Trent Nelson 754c708473 test-validator: Display more cluster info in dash 2021-05-14 08:36:08 +00:00
Tyera Eulberg 27004f1b76
Return error for excluded secondary-index keys (#17193)
* Add runtime helpers to check secondary indexes for key

* Add custom rpc error

* Check secondary-index key inclusion in rpc

* Clone complete AccountSecondaryIndexes into rpc to avoid bank query
2021-05-13 21:04:21 +00:00
Lijun Wang 9c42a89a43
Issue #17008 -- make snapshot archives to hold on to configurable. (#17158)
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
Jeff Washington (jwash) 7d96f78821
include/exclude keys on account secondary index (#17110)
* AccountSecondaryIndexes.include/exclude

* use normal scan if key is not indexed

* add a test to ask for a scan for an excluded secondary index

* fix up cli args
2021-05-11 22:06:22 +00:00
Michael Vines ec2b06d81d `solana-validator exit` now uses `process::exit()` to ensure prompt termination 2021-05-07 10:33:51 -07:00
Michael Vines 9ba2c53b85 Add --tower argument to specify where tower files are persisted 2021-05-05 12:20:39 -07:00
Trent Nelson f17b80236f test-validator: Plumb --limit-ledger-size 2021-05-04 08:45:24 +00:00
Michael Vines 6004c0abf5 getLeaderSchedule now supports filtered results based on validator identity 2021-04-21 17:59:26 -07:00
Tyera Eulberg 0924c2d070
Add port and gossip options to solana-test-validator (#16696) 2021-04-21 02:40:52 +00:00
Michael Vines f804ce63c2 Add --faucet-port option 2021-04-14 08:57:49 -07:00
Tyera Eulberg 70f3f7e679
Move obsolete rpc endpoints to separate api for removal (#16500)
* Move obsolete rpc methods to separate api for removal

* Remove obsolete method from docs

* Fix test using obs method
2021-04-12 20:33:40 -06:00
Michael Vines 05ad979a2d canonicalize authorized voter filepath 2021-04-12 17:52:55 -07:00
Michael Vines 2229b70c4e Add authorized-voter add/remove-all commands 2021-04-12 15:55:28 -07:00
Michael Vines ef30943c5c Fix up App formatting 2021-04-11 22:32:56 -07:00
Michael Vines a679aebc82 wait-for-restart-window works again for unstaked nodes 2021-04-04 16:52:39 +00:00
Michael Vines c8c89dd5f7 wait-for-restart-window now indicates how far away the next restart window is 2021-04-02 22:45:32 -07:00
Michael Vines aac18d7564 Validator monitor now displays the max retransmit slot 2021-03-30 18:58:32 -07:00
sakridge b99ae8f334
Skip leader slots until a vote lands (#15607) 2021-03-25 18:54:51 -07:00
Michael Vines 4ab98fff02 Avoid a panic when --slots-per-epoch is less than MINIMUM_SLOTS_PER_EPOCH 2021-03-18 05:45:21 +00:00
Michael Vines 04c99cf7ea Add --slots-per-epoch argument 2021-03-17 22:56:41 +00:00
Michael Vines 59c19d9fbf Notice the user when the --mint, --bpf-program, or --clone arguments are ignored 2021-03-17 20:04:53 +00:00
DimAn 20b53eb4b4 Revert to snapshots 2 2021-03-17 10:31:01 -07:00
DimAn 0b42379ed7 Revert to snapshots
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-03-17 10:31:01 -07:00
DimAn a43b3674c7 add missed suggestion 2021-03-17 10:31:01 -07:00
DimAn cfb01e26dd Apply suggestions from code review
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-03-17 10:31:01 -07:00
DimAn 0209d334bd Add option for separate snapshot location
(cherry picked from commit 6126878f509c69e23480a5ec22b3271e2b16e072)
2021-03-17 10:31:01 -07:00
Michael Vines a2eb655322 =1.7.0 2021-03-16 07:51:07 +00:00
Michael Vines eab182188a Fix delinquent stake display 2021-03-13 18:58:04 +00:00
Michael Vines 0c9ca5522c Bump version to v1.7.0 2021-03-13 09:01:21 +00:00
Michael Vines aa2b2d6b75 Default --ledger arg to "ledger" for `solana-validator` and `solana-ledger-tool` 2021-03-11 13:09:30 -08:00