4c17243157
snapshot download enhancement ( #17415 )
...
1. Allow the validator bootstrap code to specify the minimal snapshot download speed. If the snapshot download speed is detected below that, a different RPC can be retried. The default is 10MB/sec.
2. To prevent spinning on a number of sub-optimal choices and not making progress, the abort/retry logic is implemented with the following safe guards:
2.1 at maximum we do this retry for 5 times -- this number is configurable with default 5.
2.2 if the download in one notification round (5 second) is more than 2%, do not do retry -- it is not as bad anyway.
2.3 if the remaining estimate time is less than 1 minutes, do not abort retry as it will be done quickly anyway.
2.4 We do this abort/retry logic only at the first notification to avoid wasting download efforts -- the reasoning is being opportunistic and too greedy may not achieve overall shorter download time.
3. The download_snapshot and download_file is modified with the option allowing caller to notified of download progress via a callback. This allows the business logic of retrying to the place it belongs.
2021-05-25 09:32:12 -07:00
a8dca3976b
Refactor genesis download/load/check functions ( #17276 )
...
* Refactor genesis ingest functions
* Consolidate genesis.bin/genesis.tar.bz2 references
2021-05-24 16:45:36 +02:00
9c42a89a43
Issue #17008 -- make snapshot archives to hold on to configurable. ( #17158 )
...
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
e61b4b7d70
dl-utils: use `wide_msg` everywhere for truncation on narrow terminals
2021-04-14 23:50:42 +00:00
02b81dd05d
Update to reqwest 0.11.2
2021-03-18 11:59:41 -07:00
0209d334bd
Add option for separate snapshot location
...
(cherry picked from commit 6126878f509c69e23480a5ec22b3271e2b16e072)
2021-03-17 10:31:01 -07:00
58b980f9cd
Download snapshot files with a `tmp-` prefix so they'll automatically be cleaned up if interrupted
2021-03-17 09:10:10 -07:00
a2eb655322
=1.7.0
2021-03-16 07:51:07 +00:00
0c9ca5522c
Bump version to v1.7.0
2021-03-13 09:01:21 +00:00
24d18b3cf2
docs: add docs links for crates published to crates.io
2021-03-11 08:38:18 +00:00
7f7370c306
Re-allow clippy::integer_arithmetic at crate-level
2021-02-17 13:55:08 -07:00
600ff0d915
calculate hash from store instead of index ( #15034 )
...
* calculate hash from store instead of index
* restore update hash in abs
2021-02-04 09:00:33 -06:00
bf1943e489
Add solana-test-validator --warp-slot argument
2021-01-22 21:17:02 -08:00
7be6770808
Rename CompressionType to ArchiveFormat
2021-01-09 09:07:49 -08:00
90deadb396
Avoid creating "..tmp" files
2020-12-28 08:58:09 -08:00
7b49c85aa7
Fix download speed ( #14291 )
2020-12-27 16:25:41 -08:00
93ae177503
Do not delete ALL other snapshots before downloading a new snapshot
2020-12-21 08:58:52 +00:00
1e977ac025
Bump version to v1.6.0
2020-12-15 18:28:04 +00:00
88d950e5a8
chore: bump log from 0.4.8 to 0.4.11 ( #13691 )
...
* chore: bump log from 0.4.8 to 0.4.11
Bumps [log](https://github.com/rust-lang/log ) from 0.4.8 to 0.4.11.
- [Release notes](https://github.com/rust-lang/log/releases )
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.11 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 13:48:23 +09:00
7ee0c9e68a
Add non-progress bar download status ( #13370 )
2020-11-04 10:32:27 -08:00
8a119c1483
Bump reqwest/rayon to get past `cargo audit`
2020-10-11 18:51:09 +00:00
11df2e2236
Bump version to v1.5.0
2020-10-08 04:51:36 +00:00
e12ab9d0dd
Bump version to 1.4.0
2020-08-05 12:04:15 -06:00
6ee222363e
Move BankForks to solana_runtime ( #10637 )
...
* Move BankForks to solana_runtime
* Update imports
2020-06-17 15:27:03 +00:00
943ae9793d
Bump indicatif from 0.14.0 to 0.15.0
...
Bumps [indicatif](https://github.com/mitsuhiko/indicatif ) from 0.14.0 to 0.15.0.
- [Release notes](https://github.com/mitsuhiko/indicatif/releases )
- [Commits](https://github.com/mitsuhiko/indicatif/compare/0.14.0...0.15.0 )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-14 23:58:57 -07:00
48febf153b
Change documentation introduction to indicate Foundation IP transfer ( #10440 )
2020-06-13 14:41:05 -07:00
6cf30c38a4
Bump reqwest from 0.10.5 to 0.10.6
...
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.10.5 to 0.10.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/commits )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-03 09:01:36 -07:00
b787254bd6
Bump console from 0.10.3 to 0.11.3 ( #10314 )
...
Bumps [console](https://github.com/mitsuhiko/console ) from 0.10.3 to 0.11.3.
- [Release notes](https://github.com/mitsuhiko/console/releases )
- [Commits](https://github.com/mitsuhiko/console/commits/v0.11.3 )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-05-30 07:47:41 -06:00
9eefb2c929
Bump version to 1.3.0
2020-05-28 15:01:13 -07:00
2d3a906d55
Bump tar from 0.4.27 to 0.4.28 ( #10185 )
...
Bumps [tar](https://github.com/alexcrichton/tar-rs ) from 0.4.27 to 0.4.28.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases )
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.27...0.4.28 )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-05-22 15:55:40 -07:00
8da2e1b2f7
Bump tar from 0.4.26 to 0.4.27 ( #10150 )
...
Bumps [tar](https://github.com/alexcrichton/tar-rs ) from 0.4.26 to 0.4.27.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases )
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.26...0.4.27 )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-05-20 20:04:22 -07:00
d67ad70443
Revert "Bump console from 0.10.1 to 0.11.2 ( #9763 )"
...
This reverts commit 2de999fb61
2020-04-28 10:43:18 -07:00
2de999fb61
Bump console from 0.10.1 to 0.11.2 ( #9763 )
...
Bumps [console](https://github.com/mitsuhiko/console ) from 0.10.1 to 0.11.2.
- [Release notes](https://github.com/mitsuhiko/console/releases )
- [Commits](https://github.com/mitsuhiko/console/compare/0.10.1...0.11.2 )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-04-28 09:36:07 -07:00
c5a7df9221
Bump console from 0.10.0 to 0.10.1 ( #9726 )
...
Bumps [console](https://github.com/mitsuhiko/console ) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/mitsuhiko/console/releases )
- [Commits](https://github.com/mitsuhiko/console/compare/0.10.0...0.10.1 )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-04-27 09:16:24 -06:00
47ae57610a
Only build x86_64-unknown-linux-gnu on docs.rs
2020-04-16 19:06:17 -07:00
9493de4443
Add snapshot compression option ( #9276 )
2020-04-03 13:13:49 -07:00
d7fa40087c
Bump version to 1.2.0
2020-03-28 09:44:13 -07:00
f987c18a7e
Strictly validate the contents of snapshot/genesis ( #8959 )
...
automerge
2020-03-25 02:46:41 -07:00
bf60345b7a
Remove all snapshots not matching the desired hash
2020-03-17 22:58:01 -07:00
3a501ad69e
Remove all snapshot not matching the desired hash
2020-03-17 22:18:25 -07:00
215650f6e7
Bump console from 0.9.2 to 0.10.0 ( #8786 )
...
Bumps [console](https://github.com/mitsuhiko/console ) from 0.9.2 to 0.10.0.
- [Release notes](https://github.com/mitsuhiko/console/releases )
- [Commits](https://github.com/mitsuhiko/console/commits )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-11 12:30:15 -06:00
75a84ecdae
Bump reqwest from 0.10.1 to 0.10.4 ( #8787 )
...
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.10.1 to 0.10.4.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.10.1...v0.10.4 )
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-11 09:02:40 -06:00
97986a5241
Move download code to download-utils crate ( #8704 )
2020-03-07 07:08:01 -08:00
Lijun Wang
sakridge
Trent Nelson
Michael Vines
DimAn
Jeff Washington (jwash)
dependabot[bot]
Greg Fitzgerald
dependabot-preview[bot]
R. M. Shea
Ryo Onodera