Commit Graph

2171 Commits

Author SHA1 Message Date
dependabot-preview[bot] 095c79e863
Bump regex from 1.3.5 to 1.3.6 (#9055)
Bumps [regex](https://github.com/rust-lang/regex) from 1.3.5 to 1.3.6.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.3.5...1.3.6)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-25 17:24:14 -07:00
Ryo Onodera f987c18a7e
Strictly validate the contents of snapshot/genesis (#8959)
automerge
2020-03-25 02:46:41 -07:00
Justin Starry 8f38bc7dc0
Refactor how pubsub subscriptions are added (#9042) 2020-03-25 00:53:32 +08:00
Justin Starry 19dfb87b1f
Fix timeout for subscriptions test (#9043)
automerge
2020-03-24 01:57:28 -07:00
Justin Starry eed8087d87
Respect confirmations param for signature subscription notifications (#9019)
automerge
2020-03-23 17:00:34 -07:00
Tyera Eulberg 1b8f9e75dd
Update getSignatureStatus: support multiple signatures, include slot in each response item (#9022)
* Rename enable-rpc-get-confirmed-block

* Rename RpcTransactionStatus -> RpcTransactionStatusMeta

* Return simplified RpcTransactionStatus; Add support for multiple transactions

* Update docs

* typo
2020-03-23 11:25:39 -06:00
sakridge 4b397d15b3
Accounts cleanup service and perf improvements (#8799)
* Use atomic for ref count instead of taking rwlock

* Accounts cleanup service

* Review comments
2020-03-23 08:50:23 -07:00
sakridge 4d2b83d01f
Add option to disable rocks compaction (#9011) 2020-03-23 08:42:32 -07:00
Justin Starry a0ffcc61ae
Add slot info to Bank::get_signature_confirmation_status (#9018) 2020-03-23 21:55:15 +08:00
Justin Starry 4b4819cd07
Add slot context to rpc pubsub notifications (#9001)
automerge
2020-03-23 05:34:42 -07:00
Michael Vines 88ba8439fc
Add frozen account support (#8989)
automerge
2020-03-22 11:10:04 -07:00
Michael Vines aa24181a53
Remove blockstream unix socket support. RPC or bust (#9004)
automerge
2020-03-21 20:17:11 -07:00
Michael Vines 18c1f0dfe9
Remove stub core/src/genesis_utils.rs (#8999) 2020-03-21 10:54:40 -07:00
sakridge 909321928c
Shred fetch comment and debug message tweak (#8980)
automerge
2020-03-20 11:00:48 -07:00
sakridge 453f5ce8f2
Shred filter (#8975)
Thread bank_forks into shred fetch
2020-03-20 07:49:48 -07:00
carllin dc1db33ec9
Add Capabilities to Signal BroadcastStage to Retransmit (#8899) 2020-03-19 23:35:01 -07:00
dependabot-preview[bot] 24d871b529
Bump serde from 1.0.104 to 1.0.105 (#8954)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.104 to 1.0.105.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.104...v1.0.105)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-18 22:35:30 -07:00
dependabot-preview[bot] 2dc50cff5b
Bump bv from 0.11.0 to 0.11.1 (#8952)
automerge
2020-03-18 21:37:21 -07:00
sakridge c51049a59b
Add counter for accounts hash verification. (#8928) 2020-03-18 08:39:09 -07:00
anatoly yakovenko 9cedeb0a8d
Pull streamer out into its own module. (#8917)
automerge
2020-03-17 23:30:23 -07:00
sakridge 7079559c2d
Fix windows build by removing sys-info (#8860)
Doesn't build for windows.
2020-03-16 12:53:13 -07:00
sakridge dc347dd3d7
Add Accounts hash consistency halting (#8772)
* Accounts hash consistency halting

* Add option to inject account hash faults for testing.

Enable option in local cluster test to see that node halts.
2020-03-16 08:37:31 -07:00
Carl ead6dc553a If let 2020-03-16 07:57:07 -07:00
Carl 009c124fac Remove generic 2020-03-16 07:57:07 -07:00
Carl 9411fc00b8 Lower error level 2020-03-16 07:57:07 -07:00
carllin 9afc5da2e1
Fix vote polling (#8829)
Co-authored-by: Carl <carl@solana.com>
2020-03-15 20:31:05 -07:00
Michael Vines 5481d1a039 Validators now run a full gossip node while looking for a snapshot 2020-03-15 09:31:55 -07:00
Michael Vines 29fb79382c Rework validator vote account defaults to half voting fees 2020-03-13 20:13:33 -07:00
carllin 53b8d0d528
Remove holding Poh lock (#8838)
automerge
2020-03-13 15:15:13 -07:00
dependabot-preview[bot] 52b254071c
Bump regex from 1.3.4 to 1.3.5 (#8830)
Bumps [regex](https://github.com/rust-lang/regex) from 1.3.4 to 1.3.5.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.3.4...1.3.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-13 07:43:07 -06:00
anatoly yakovenko 9a79be5ca0
Use cluster information about slots to prioritize repair (#8820)
automerge
2020-03-12 17:34:46 -07:00
dependabot-preview[bot] f93c8290f4
Bump sys-info from 0.5.9 to 0.5.10 (#8810)
Bumps [sys-info](https://github.com/FillZpp/sys-info-rs) from 0.5.9 to 0.5.10.
- [Release notes](https://github.com/FillZpp/sys-info-rs/releases)
- [Changelog](https://github.com/FillZpp/sys-info-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/FillZpp/sys-info-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-12 13:42:31 -06:00
dependabot-preview[bot] 48ac038f7a
Bump serial_test from 0.3.2 to 0.4.0 (#8808)
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.3.2 to 0.4.0.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.3.2...v0.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-12 11:14:37 -06:00
anatoly yakovenko f64ab49307
Cluster has no way to know which slots are available (#8732)
automerge
2020-03-11 21:31:50 -07:00
Jack May 6eb4973780
Don't use move semantics if not needed (#8793) 2020-03-11 14:37:23 -07:00
dependabot-preview[bot] 75a84ecdae
Bump reqwest from 0.10.1 to 0.10.4 (#8787)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.10.1 to 0.10.4.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.10.1...v0.10.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-11 09:02:40 -06:00
dependabot-preview[bot] e41ff2df66
Bump chrono from 0.4.10 to 0.4.11 (#8755)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.10 to 0.4.11.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/master/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.10...v0.4.11)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-10 08:43:18 -06:00
dependabot-preview[bot] f88b79d42b
Bump itertools from 0.8.2 to 0.9.0 (#8756)
Bumps [itertools](https://github.com/bluss/rust-itertools) from 0.8.2 to 0.9.0.
- [Release notes](https://github.com/bluss/rust-itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bluss/rust-itertools/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-10 08:42:11 -06:00
dependabot-preview[bot] 1a0dd53450
Bump rayon from 1.2.0 to 1.3.0 (#8757)
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.2.0 to 1.3.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.2.0...rayon-core-v1.3.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-10 08:40:02 -06:00
carllin 9872430bd2
Add VoteTracker for tracking cluster's votes in gossip (#8327)
Track votes by slot in cluster_vote_listener
2020-03-09 22:03:09 -07:00
Michael Vines 36fa3a1a0a
Wait for 80% of the active stake instead of 75% 2020-03-09 20:31:09 -07:00
dependabot-preview[bot] fad08a19cc
Bump serde_json from 1.0.46 to 1.0.48 (#8260)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.46 to 1.0.48.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.46...v1.0.48)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-09 20:20:19 -06:00
dependabot-preview[bot] ff74452ef3
Bump libc from 0.2.66 to 0.2.67 (#8680)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.66 to 0.2.67.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.66...0.2.67)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
2020-03-08 21:38:58 -07:00
Michael Vines f992ee3140
Remove unnecessary snapshot hash verification (#8711) 2020-03-07 09:04:52 -07:00
sakridge 97986a5241
Move download code to download-utils crate (#8704) 2020-03-07 07:08:01 -08:00
Michael Vines cea8067219
Disable setLogFilter RPC API by default (#8693)
automerge
2020-03-06 16:03:10 -08:00
Trent Nelson 4db074a5aa
RPC: Add `getFeeCalculatorForBlockhash` method call (#8687)
Returns the `FeeCalculator` associated with the given blockhash, or
`null` if said blockhash has expired
2020-03-06 17:01:31 -07:00
Michael Vines cb6848aa80
Publish initial snapshot hash in gossip on validator startup (#8679)
automerge
2020-03-05 22:52:31 -08:00
Greg Fitzgerald 2242b1b4a5
Bump byteorder from 1.3.2 to 1.3.4 (#8159)
Bumps [byteorder](https://github.com/BurntSushi/byteorder) from 1.3.2 to 1.3.4.
- [Release notes](https://github.com/BurntSushi/byteorder/releases)
- [Changelog](https://github.com/BurntSushi/byteorder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/BurntSushi/byteorder/compare/1.3.2...1.3.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-05 17:07:18 -07:00
carllin f47a789b15
Add find_incomplete_slots (#8654)
* Add find_incomplete_slots

* Add live slots iterator
2020-03-05 10:58:00 -08:00
Trent Nelson fd00e5cb35
Store FeeCalculator with blockhash in nonce accounts (#8650)
* Copy current state version to v0

* Add `FeeCalculator` to nonce state

* fixup compile

* Dump v0 handling...

Since we new account data is all zeros, new `Current` versioned accounts
look like v0. We could hack around this with some data size checks, but
the `account_utils::*State` traits are applied to `Account`, not the
state data, so we're kind SOL...

* Create more representative test `RecentBlockhashes`

* Improve CLI nonce account display

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Fix that last bank test...

* clippy/fmt

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-03-05 07:40:26 -07:00
Michael Vines 448b957a13
Add --bind-address and --rpc-bind-address validator arguments (#8628) 2020-03-04 22:46:43 -07:00
carllin dec3da8f9d
Add orphan iterator (#8636) 2020-03-04 18:10:30 -08:00
Grimes f89c22b5ee
`solana catchup` now detects when you try to catchup to yourself (#8635)
automerge
2020-03-04 14:44:21 -08:00
carllin f23dc11a86
compute_bank_stats needs to return newly computed ForkStats (#8608)
* Fix broken confirmation, add test
2020-03-04 11:49:56 -08:00
Michael Vines 306783c661
Don't advertise the snapshot that the node was loaded from
snapshot_packager_service will remove this snapshot hash from gossip
when it starts
2020-03-02 18:58:53 -07:00
Justin Starry 8ec8204a30
Run pubsub test poller in tokio runtime (#8494) 2020-03-03 09:44:39 +08:00
Michael Vines 42c5c59800 Only gossip packaged snapshots 2020-03-02 14:17:17 -07:00
carllin 8ef8c9094a
Add ReplayStage changes for checking switch threshold (#8504)
* Refactor for supporting switch threshold check
2020-03-02 12:43:43 -08:00
Michael Vines 13551885c2 --wait-for-supermajority now requires a SLOT 2020-03-02 12:59:35 -07:00
Michael Vines 1ca4913328 Avoid is_x86_feature_detected when not building for x86 2020-03-01 18:10:43 -07:00
Michael Vines 862a4a243f
Demote gossip responder error log messages to info! 2020-03-01 10:43:20 -07:00
Michael Vines 2a5605db24 Reduce max snapshot hashes to stay under MTU 2020-02-29 09:21:52 -07:00
carllin 6a5a6387e2
Fix skipping own leader slots (#8533)
automerge
2020-02-29 00:05:35 -08:00
Michael Vines 0f31adeafb GET for /snapshot.tar.bz2 now redirects to the latest snapshot 2020-02-28 23:23:59 -07:00
Michael Vines ae817722d8 Include validator version in log 2020-02-28 23:23:59 -07:00
Trent Nelson 90bedd7e06
Split signature throughput tracking out of `FeeCalculator` (#8447)
* SDK: Split new `FeeRateGovernor` out of `FeeCalculator`

Leaving `FeeCalculator` to *only* calculate transaction fees

* Replace `FeeCalculator` with `FeeRateGovernor` as appropriate

* Expose recent `FeeRateGovernor` to clients

* Move `burn()` back into `FeeCalculator`

Appease BPF tests

* Revert "Move `burn()` back into `FeeCalculator`"

This reverts commit f3035624307196722b62ff8b74c12cfcc13b1941.

* Adjust BPF `Fee` sysvar test to reflect removal of `burn()` from `FeeCalculator`

* Make `FeeRateGovernor`'s `lamports_per_signature` private

* rebase artifacts

* fmt

* Drop 'Recent'

* Drop _with_commitment variant

* Use a more portable integer for `target_signatures_per_slot`

* Add docs for `getReeRateCalculator` JSON RPC method

* Don't return `lamports_per_signature` in `getFeeRateGovernor` JSONRPC reply
2020-02-28 13:27:01 -07:00
Michael Vines 3acf956f6f Fix test_concurrent_snapshot_packaging 2020-02-26 23:32:53 -07:00
Michael Vines 0d4cb252c4 Adapt local-cluster/ 2020-02-26 23:32:53 -07:00
Michael Vines fcabc6f799 Rename snapshot.tar.bz2 to snapshot-<slot>-<hash>.tar.bz2 2020-02-26 23:32:53 -07:00
carllin 5f766cd20b
Remove loop (#8493) 2020-02-26 19:59:28 -08:00
Michael Vines 27c5ec0149 Use the same reqwest features across the repo 2020-02-26 20:47:43 -07:00
Justin Starry e743414908
Choose more appropriate options for pubsub websocket server (#8354)
* Choose more sensible options for pubsub websocket server

* Increase max payload size for pubsub service
2020-02-27 08:54:53 +08:00
carllin d47a47924a
Update voting simulation (#8460) 2020-02-26 14:09:07 -08:00
carllin 7a2bf7e7eb
Limit leader schedule search space (#8468)
* Limit leader schedule search space

* Fix and add test

* Rename
2020-02-26 13:35:50 -08:00
Michael Vines fbf78b83c4 Add retry mechanism when downloading genesis and snapshots 2020-02-26 14:21:37 -07:00
Justin Starry 021d0a46f8
Move docs from book/ to docs/ (#8469)
automerge
2020-02-26 07:11:38 -08:00
Justin Starry 8839dbfe5b
Use runtime executor to send pubsub notifications (#8353)
automerge
2020-02-25 20:23:54 -08:00
carllin d821fd29d6
Add versioning (#8348)
automerge
2020-02-25 17:12:01 -08:00
sakridge 004f1d5aed
Combine replay stage memory reporting (#8455)
automerge
2020-02-25 16:04:27 -08:00
sakridge 1caeea8bc2
Refactor new bank paths into common function (#8454) 2020-02-25 15:49:59 -08:00
Michael Vines 2059af822d
Remove unnecessary new_banks_from_blockstore() argument (#8433)
automerge
2020-02-24 23:27:19 -08:00
carllin 39282be486
Determine vote_state ahead of time (#8303)
automerge
2020-02-24 19:27:04 -08:00
Michael Vines 73063544bd Move shred_version module to sdk/ 2020-02-24 14:46:12 -07:00
dependabot-preview[bot] 72ae82fe47
Bump crossbeam-channel from 0.3.9 to 0.4.2 (#8400)
Bumps [crossbeam-channel](https://github.com/crossbeam-rs/crossbeam) from 0.3.9 to 0.4.2.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases)
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/v0.4.2/CHANGELOG.md)
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-channel-0.3.9...v0.4.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-22 09:53:38 -07:00
Dan Albert 2d9d2f1e99
Update cargo versions from 1.0 to 1.1 (#8397) 2020-02-21 23:09:45 -08:00
sakridge b7386f9d84
Add --trusted-validator support for snapshot hash validation (#8390) 2020-02-21 18:42:24 -08:00
Michael Vines fb98df76b7
4x DEFAULT_MAX_LEDGER_SLOTS to give nodes 3 hours of slots to repair from (#8388)
automerge
2020-02-21 15:04:02 -08:00
Pankaj Garg aa80f69171
Promote some datapoints to `info` to fix dashboard (#8381)
automerge
2020-02-21 13:41:49 -08:00
Michael Vines 48f58a88bc Bump version to 1.0.0 2020-02-20 23:52:19 -07:00
Ryo Onodera d238371b0c
Correct missing entry handling to avoid bad warns (#8339)
* Correct missing entry handling to avoid bad warns

* Pass storage entries to AccountStorageSerialize

* Fix CI.....

* Add tests and reorder condition for cheapest first

* Remove unneeded reference
2020-02-21 15:27:55 +09:00
Michael Vines 18fd52367e
If the node was loaded from a snapshot, advertise it in gossip (#8364)
automerge
2020-02-20 18:53:26 -08:00
dependabot-preview[bot] b65c9ea544
Bump serial_test_derive from 0.3.2 to 0.4.0 (#8311)
Bumps [serial_test_derive](https://github.com/palfrey/serial_test) from 0.3.2 to 0.4.0.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.3.2...v0.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-20 17:06:00 -07:00
Michael Vines cc7c6c960e Search for the validator with the highest snapshot 2020-02-20 17:04:48 -07:00
Justin Starry 01697a9f5c
Remove unnecessary arc and mutex for rpc notifications (#8351) 2020-02-21 08:03:46 +08:00
Tyera Eulberg ab361a8073
Rename KeypairUtil to Signer (#8360)
automerge
2020-02-20 13:28:55 -08:00
sakridge 1720fe6a46
Snapshot hash gossip changes (#8358) 2020-02-20 11:46:13 -08:00
Pankaj Garg e50bc0d34b
Do not compress small incomplete slot list (#8355)
automerge
2020-02-20 09:48:39 -08:00
Pankaj Garg ea8d9d1aea
Bitwise compress incomplete epoch slots (#8341) 2020-02-19 20:24:09 -08:00
Sagar Dhawan 221866f74e
Process Gossip in parallel and add an upper limit (#8328) 2020-02-19 21:31:55 -06:00
anatoly yakovenko ccad5d5aaf
change warnings to infos (#8322) 2020-02-19 14:25:49 -08:00
Michael Vines 83a8e82626 Remove dead code 2020-02-18 21:08:43 -07:00
Michael Vines 7305a1f407 Reformatting 2020-02-18 21:08:43 -07:00
Pankaj Garg ac1d075d73
Drop packet if destination is unspecified (0.0.0.0/0) (#8321) 2020-02-18 16:14:20 -08:00
carllin 73a278dc64
Factor out creating genesis with vote accounts into a utility function (#8315)
automerge
2020-02-18 02:39:47 -08:00
Pankaj Garg 0d5c1239c6
Update epoch slots to include all missing slots (#8276)
* Update epoch slots to include all missing slots

* new test for compress/decompress

* address review comments

* limit cache based on size, instead of comparing roots
2020-02-17 12:39:30 -08:00
Michael Vines 1bf2285fa2 ledger-tool: Add print-accounts command 2020-02-14 19:59:48 -07:00
anatoly yakovenko 17fb8258e5
Datapoints overwhelm the metrics queue and blow up ram usage. (#8272)
automerge
2020-02-14 11:11:55 -08:00
Justin Starry 1c97b31eaf
Retain signature subscriptions that haven't been notified (#8261) 2020-02-14 01:00:50 +08:00
Greg Fitzgerald 127553ce4b
Wrap ed25519_dalek::Keypair (#8247) 2020-02-12 14:15:12 -07:00
Michael Vines ecb055a252
Expel ContactInfo::new() (#8245)
automerge
2020-02-12 12:58:51 -08:00
Michael Vines cf11d4c7dc
Nodes with a tvu and storage port are no longer double counted (#8237)
automerge
2020-02-12 11:16:07 -08:00
Michael Vines d0a4686990 Avoid assigning the serve repair port to the storage port 2020-02-12 12:00:11 -07:00
Michael Vines 1e0f2b2446 Quash 'repair listener error: Err(RecvTimeoutError(Timeout))' log spam 2020-02-12 10:35:03 -07:00
Michael Vines c4fd81fc1c The getConfirmedBlock RPC API is now disabled by default
The --enable-rpc-get-confirmed-block flag allows validators to opt-in to
the higher disk usage and IOPS.
2020-02-11 22:24:08 -07:00
Justin Starry 268bb1b59b
Fix RPC pub sub unsubscribe (#8208)
automerge
2020-02-11 17:09:40 -08:00
Michael Vines 72b11081a4 Report validator rewards in getConfirmedBlock JSON RPC 2020-02-11 17:25:45 -07:00
dependabot-preview[bot] fcac910989
Bump jsonrpc-pubsub from 14.0.5 to 14.0.6 (#8162)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 14.0.5 to 14.0.6.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/jsonrpc-pubsub-v14.0.5...ipc-14.0.6)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-11 15:44:21 -07:00
carllin d3712dd26d Factor repair from gossip (#8044) 2020-02-11 13:11:48 -07:00
Michael Vines 60877f9ba4 Revert "Check for AVX512 at runtime to avoid invalid opcode trap (#8166)"
This reverts commit ef5fb6fa46.
2020-02-11 12:56:02 -07:00
Ryo Onodera 46b6cedff4
Fix nightly clippy warnings (#8199)
automerge
2020-02-10 22:48:50 -08:00
dependabot-preview[bot] 124f77cdb1
Bump jsonrpc-ws-server from 14.0.5 to 14.0.6 (#8160)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 14.0.5 to 14.0.6.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/jsonrpc-ws-server-v14.0.5...ipc-14.0.6)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-10 19:07:52 -07:00
Trent Nelson fc15f74c3c
CLI: Harden offline signing and tests (#8052)
* CLI: Don't sanity-check stake account when offline

* Add test helper returning vote pubkey with validator

* Delegate to the BSL. No need to force

* Be sure our offline ops are truly offline

* Specify our authorities correctly

* checks
2020-02-10 18:59:05 -07:00
Pankaj Garg 1d06aa3b31
Remove repairman as its spamming cluster with unwanted repairs (#8193)
* Remove repairman as its spamming cluster with unwanted repairs

* remove obsolete test
2020-02-10 17:00:00 -08:00
carllin 0b263f8714
Fix larger than necessary allocations in streamer (#8187) 2020-02-10 11:49:07 -08:00
dependabot-preview[bot] 669282ae69
Bump jsonrpc-http-server from 14.0.5 to 14.0.6 (#8161)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 14.0.5 to 14.0.6.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/jsonrpc-http-server-v14.0.5...ipc-14.0.6)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-10 10:30:32 -07:00
Ryo Onodera 485806c488 Just define BnakSlotDelta type alias (#8186)
automerge
2020-02-10 03:11:37 -08:00
Michael Vines ef5fb6fa46
Check for AVX512 at runtime to avoid invalid opcode trap (#8166)
automerge
2020-02-07 15:01:45 -08:00
Michael Vines 99432833d2 Remove reed-solomon-erasure from core/ 2020-02-07 15:37:57 -07:00
Sagar Dhawan fa00803fbf
Filter old CrdsValues received via Pull Responses in Gossip (#8150)
* Add CrdsValue timeout checks on Pull Responses

* Allow older values to enter Crds as long as a ContactInfo exists

* Allow staked contact infos to be inserted into crds if they haven't expired

* Try and handle oveflows

* Fix test

* Some comments

* Fix compile

* fix test deadlock

* Add a test for processing timed out values received via pull response
2020-02-07 12:38:24 -08:00
Michael Vines 8b5598fabd
Surface shred version more in tools (#8163)
automerge
2020-02-07 08:57:54 -08:00
Trent Nelson 5b070ad014
CLI: Support offline fee payers (#8009)
* CLI: Support offline fee-payer

* Add some knobs to test genesis/validator helpers

* Add tests
2020-02-07 09:14:26 -07:00
Trent Nelson 27c8ba6afc
De-replicode Tower constructors (#8153)
automerge
2020-02-06 18:24:10 -08:00
carllin fe590da3b6
Revert "Factor repair from gossip (#8044)" (#8143)
This reverts commit e61257695f.
2020-02-06 11:44:20 -08:00
Sunny Gleason a7fa92b372
feat: implementation of live-slots command (#8129) 2020-02-06 14:16:30 -05:00
François Garillot f016c9a669
Maintenance : simplify a few patterns, remove unneeded dependencies (#8137)
* Simplify a few pattern matches

* Removing unneeded dependencies, upgrading internal version #s

 net-shaper: Removing log, semver, serde_derive
 bench-tps: Removing serde, serde_derive
 banking-bench: Removing solana
 ledger-tool: Removing bincode, serde, serde_derive
 librapay: Removing solana, language_e2e_tests
 log-analyzer: Removing log, semver, serde_derive
 exchange: Removing solana
 core: Removing crc, memmap, symlink, untrusted
 perf: Removing serde_derive
 genesis: Removing hex, serde_derive
 sdk-c: Removing sha2
 sys-tuner: Removing semver
 bench-exchange: Removing bincode, bs58, env_logger, serde, serde_derive, untrusted, ws
 btc_spv_bin: Removing serde_json
 btc_spv: Removing chrono
 bpf_loader: Removing serde
 ledger: Removing dlopen, dlopen_derive, serde_derive
 move_loader: Removing byteorder, libc, language_e2e_tests
 ownable: Removing serde, serde_derive
 client: Removing rand
 archiver-utils: Removing rand_chacha
 validator: Removing serde_json, tempfile
 param_passing_dep: Removing solana
 failure: Removing log
 vest: Removing log
 vote-signer: Removing bs58, serde
 local-cluster: Removing symlink
 keygen: Removing rpassword
 install: Removing bs58, log
 upload-perf: Removing log
 runtime: Removing serde_json
 stake: Removing rand

* Add modified Cargo.lock

* fixup! Simplify a few pattern matches

* fixup! Simplify a few pattern matches
2020-02-06 10:02:38 -07:00
dependabot-preview[bot] 65c24db83c
Bump serde_json from 1.0.44 to 1.0.46 (#8087)
* Bump serde_json from 1.0.44 to 1.0.46

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.44 to 1.0.46.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.44...v1.0.46)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Update Move's serde_json to v1.0.46

Co-authored-by: Jack May <jack@solana.com>
2020-02-04 16:59:23 -08:00
dependabot-preview[bot] 1420628b28
Bump nix from 0.16.1 to 0.17.0 (#8112)
Bumps [nix](https://github.com/nix-rust/nix) from 0.16.1 to 0.17.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits/v0.17.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-04 15:59:34 -07:00
carllin 0c8cee8c4a
Refactor select_fork() to avoid clones and for clarity (#8081)
* Refactor select_fork() to avoid clones and for clarity

* Add test that fork weights are increasing
2020-02-03 16:48:24 -08:00
dependabot-preview[bot] c835749563
Bump sys-info from 0.5.8 to 0.5.9 (#8089)
Bumps [sys-info](https://github.com/FillZpp/sys-info-rs) from 0.5.8 to 0.5.9.
- [Release notes](https://github.com/FillZpp/sys-info-rs/releases)
- [Changelog](https://github.com/FillZpp/sys-info-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/FillZpp/sys-info-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-03 14:49:39 -07:00
sakridge 0172d2a065
Fix consensus threshold when new root is created (#8093)
When a new root is created, the oldest slot is popped off
but when the logic checks for identical slots, it assumes
that any difference means a slot was popped off the front.
2020-02-03 13:44:34 -08:00
Justin Starry 69a6d07371
Reduce rpc client pre-flight requests by setting max-age header (#8082)
automerge
2020-02-01 04:10:26 -08:00
carllin e61257695f
Factor repair from gossip (#8044) 2020-01-31 14:23:50 -08:00
Sagar Dhawan b9988b62e4
Filter repairman peers based on shred_version (#8069) 2020-01-31 14:00:19 -08:00
Michael Vines a03d441e6f Add rpc port sanity checks, fix tests 2020-01-30 20:57:58 -07:00
Michael Vines 3900d09f6f Employ rpc_port defaults 2020-01-30 20:57:58 -07:00
Michael Vines e218f4e56e Clean up Validator::new() 2020-01-30 20:57:58 -07:00
Michael Vines 81ba18eea6 Add --private-rpc flag 2020-01-30 20:57:58 -07:00
Michael Vines 775fa0c968
Minor --expected-shred fix, clean up shred-related gossip log messages (#8041)
automerge
2020-01-30 13:22:05 -08:00
Michael Vines 0c55b37976 Add different shred test to test_tvu_peers_and_stakes 2020-01-30 10:30:32 -07:00
Sagar Dhawan 64c42e28dc
Add shred version filters to Crds Accessors (#8027)
* Add shred version filters to Crds Accessors

* Adopt entrypoint shred_version if one isn't provided
2020-01-30 00:15:37 -08:00
Michael Vines bea9cd9684 Add --expected-shred-version option 2020-01-29 20:08:42 -07:00
Michael Vines c4faccc77f getClusterNodes now excludes validators with a different shred version 2020-01-29 20:08:42 -07:00
carllin 4197cce8c9
Tower tests (#7974)
* Add testing framework for voting
2020-01-28 16:02:28 -08:00
carllin 4ffd7693d6
Add lock to make sure slot-based locktree calls are safe (#7993) 2020-01-28 13:45:41 -08:00
Michael Vines fd7d5cbe0d Fix compute_shred_version() (#7989)
automerge
2020-01-27 17:05:31 -08:00
sakridge 7058287273
Consensus fix, don't consider threshold check if.. (#7948)
* Consensus fix, don't consider threshold check if

lockouts are not increased

* Change partition tests to wait for epoch with > lockout slots

* Use atomic bool to signal partition
2020-01-27 16:49:25 -08:00
Dan Albert c324e71768
Bump cargo toml versions to 0.24.0 (#7976) 2020-01-25 11:04:27 -06:00
Michael Vines 989355e885 Add ability to hard fork at any slot (#7801)
automerge
2020-01-24 17:27:04 -08:00
Michael Vines c0f0fa24f8 Increase --wait-for-supermajority to wait for 75% online stake 2020-01-23 22:41:46 -07:00
Michael Vines b045f9a50d codemod --extensions rs get_snapshot_tar_path get_snapshot_archive_path 2020-01-23 13:37:13 -07:00
Michael Vines ce231602dc Move snapshot archive generation out of the SnapshotPackagerService 2020-01-23 13:37:13 -07:00
Michael Vines 6f5e0cd161 Type grooming 2020-01-23 13:37:13 -07:00
Michael Vines 006cbee88a Uninteresting cleanup 2020-01-22 21:24:20 -07:00
Jack May c95e5346a4
Boot the mut (#7926) 2020-01-22 17:54:06 -08:00
Michael Vines 934c32cbc6 Add mechanism to load v0.22.3 snapshots on newer Solana versions 2020-01-22 15:40:32 -07:00
Greg Fitzgerald 3aabeb2b81
Rename bootstrap leader (#7906)
* Rename bootstrap leader to bootstrap validator

It's a normal validator as soon as other validators enter the
leader schedule.

* cargo fmt

* Fix build

Thanks @CriesofCarrots!
2020-01-22 09:22:09 -07:00
Tyera Eulberg 65f5885bce
sendTransaction rpc: expect transaction as base58 string (#7913) 2020-01-21 22:16:07 -07:00
Tyera Eulberg 7a132eabb4
Update JSON-RPC documentation (#7915)
* Streamline getBlockCommitment response

* Update json-rpc docs
2020-01-21 20:17:33 -07:00
Rob Walker 7e1b380f01
Move vote_state current credits into epoch_credits (#7909)
* Move vote_state current credits into epoch_credits

* fixups

* fixup
2020-01-21 19:08:40 -08:00
Michael Vines 93036bec01 Add minimumLedgerSlot RPC API 2020-01-21 14:05:26 -07:00
Jack May 663e98969d Use a different error to test rpc response (#7900)
automerge
2020-01-21 12:42:23 -08:00
Michael Vines ebf6e1c0e9 --limit-ledger-size now accepts an optional slot count value 2020-01-20 14:20:30 -07:00
Sunny Gleason 5cf090c896
feat: implement RPC notification queue (#7863) 2020-01-20 16:08:29 -05:00
dependabot-preview[bot] 0827d52c6f Bump indexmap from 1.1.0 to 1.3.1
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.1.0 to 1.3.1.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.1.0...1.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-20 10:07:48 -07:00
Ryo Onodera a8d33c9950
Spy just for RPC to avoid premature supermajority (#7856)
* Spy just for RPC to avoid premature supermajority

* Make gossip_content_info private

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Fix misindent...

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-01-20 10:50:31 +09:00
Justin Starry 87598c7612 Consolidate tx error counters and update metrics dashboard (#7724)
automerge
2020-01-16 23:26:50 -08:00
carllin 76e20015a4
Add separate thread to check for and store duplicate slot proofs (#7834) 2020-01-16 15:27:54 -08:00
sakridge 8572b57834
Refactor chacha cuda to be able to test cuda crate but not in OpenCL (#7685)
* Refactor chacha cuda to be able to test cuda crate but not in OpenCL

chacha not implemeted in OpenCL

* Get off core::Error
2020-01-16 08:29:36 -08:00
Ryo Onodera ed0129f881 Don't depend on unused lazy_static 2020-01-16 08:43:13 -07:00
Tyera Eulberg da165d6943
Fix Rpc inconsistencies (#7826)
* Update rpc account format: remove byte arrays

* Base58-encode pubkeys in getStoragePubkeysForSlot

* Update docs
2020-01-15 15:33:53 -07:00
Tyera Eulberg f0f185509f Remove tuple from programNotification (#7819)
automerge
2020-01-15 10:52:02 -08:00
Tyera Eulberg 6d3b8b6d7d
Remove tuples from JSON RPC responses (#7806)
* Remove RpcConfirmedBlock tuple

* Remove getRecentBlockhash tuple

* Remove getProgramAccounts tuple

* Remove tuple from get_signature_confirmation_status

* Collect Rpc response types

* Camel-case epoch schedule for rpc response

* Remove getBlockCommitment tuple

* Remove getStorageTurn tuple

* Update json-rpc docs
2020-01-15 00:25:45 -07:00
Justin Starry ff1ca1e0d3
Consolidate entry tick verification into one function (#7740)
* Consolidate entry tick verification into one function

* Mark bad slots as dead in blocktree processor

* more feedback

* Add bank.is_complete

* feedback
2020-01-15 09:15:26 +08:00
Michael Vines e6af4511a8 Include shred version in gossip 2020-01-14 14:32:40 -07:00
Pankaj Garg 156292e408
Reduce grace ticks, and ignore grace ticks for missing leaders (#7764)
* Reduce grace ticks, and ignore grace ticks for missing leaders

* address review comments

* blockstore related renames
2020-01-14 05:25:41 +05:30
Greg Fitzgerald b5dba77056 Rename blocktree to blockstore (#7757)
automerge
2020-01-13 13:13:52 -08:00
Tyera Eulberg a17d5795fb getConfirmedBlock: add encoding optional parameter (#7756)
automerge
2020-01-12 21:34:30 -08:00
Trent Nelson 9754fc789e
Manage durable nonce stored value in runtime (#7684)
* Bank: Return nonce pubkey/account from `check_tx_durable_nonce`

* Forward account with HashAgeKind::DurableNonce

* Add durable nonce helper for HashAgeKind

* Add nonce util for advancing stored nonce in runtime

* Advance nonce in runtime

* Store rolled back nonce account on TX InstructionError

* nonce: Add test for replayed InstErr fee theft
2020-01-10 16:57:31 -07:00
Ryo Onodera 865c42465a
Cap file size for snapshot data files (#7182)
* save limit deserialize

* save

* Save

* Clean up

* rustfmt

* rustfmt

* Just comment out to please CI

* Fix ci...

* Move code

* Rustfmt

* Crean up control flow

* Add another comment

* Introduce predetermined constant limit on snapshot data files (deserialize side)

* Introduce predetermined constant limit on snapshot data files (serialize side)

* rustfmt

* Tweak message

* Revert dynamic memory limit

* Limit size of snapshot data file (de)serialization

* Fix test breakage

* Clean up

* Fix uses formatting

* Rename: deserialize_{for,from}_snapshot

* Simplify comment

* Use Slot

* Provide slot for status cache

* Align variable name with snapshot_status_cache_file_path

* Define serialize_snapshot_data_file_with_metrics

* Fix build.......

* De-marco serialize_snapshot_data_file_with_metrics

* Revert u64 => Slot
2020-01-10 09:49:36 +09:00
sakridge 73c93cc345
Print bank hash and hash inputs. (#7733) 2020-01-09 16:33:10 -08:00
dependabot-preview[bot] cf32fdf672 Bump reqwest from 0.10.0 to 0.10.1 (#7731)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.10.0...v0.10.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-09 17:10:55 -07:00
dependabot-preview[bot] 57858b8015 Bump reqwest from 0.9.24 to 0.10.0 (#7642)
* Bump reqwest from 0.9.24 to 0.10.0

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.9.24 to 0.10.0.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Make reqwest::blocking specific

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-01-08 13:31:43 -07:00
Jack May 07855e3125
Allow override of RUST_LOG (#7705) 2020-01-08 09:19:12 -08:00
Michael Vines 43897de12e Account for stake held by the current node while waiting for the supermajority to join gossip 2020-01-07 22:29:31 -07:00
carllin 85c3d64f29
Fix rooted slot iterator (#7695)
* Enable jumping gaps caused by snapshots in rooted slot iterator
2020-01-07 22:51:28 -05:00
Michael Vines 47dd293904 supermajority is one word 2020-01-07 15:50:59 -07:00
Michael Vines c4220a4853 clippy 2020-01-07 15:50:59 -07:00
Michael Vines 48ab88a2af Add --wait-for-super-majority to facilitate asynchronous cluster restarts 2020-01-07 15:50:59 -07:00
Tyera Eulberg 84a37a2c0c Make validator timestamping more coincident, and increase timestamp sample range (#7673)
automerge
2020-01-03 22:38:00 -08:00
Michael Vines 4fe0b116ae Measure heap usage while processing the ledger 2020-01-03 13:25:37 -07:00
Michael Vines a0fb9de515 Move thread_mem_usage module into measure/ 2020-01-03 13:25:37 -07:00
sakridge 5d42dcc9ec
Reduce constants for ledger cleanup test (#7629) 2020-01-03 12:05:14 -08:00
sakridge 96e88c90e8
Lessen test_slots_to_snapshot constants to make test faster (#7628)
Reduces test time from 6m to 45s
2020-01-03 09:58:52 -08:00
Greg Fitzgerald a707c9410e
More thiserror (#7183)
* Less solana_core::result. Module now private.

* Drop solana_core::result dependency from a few more modules

* Fix warning

* Cleanup

* Fix typo
2020-01-02 20:50:43 -07:00
Rob Walker 6b7d9942a7
Add authorized_voter history (#7643)
* Add authorized_voter history

* fixups

* coverage

* bigger vote state
2019-12-30 19:57:53 -08:00
Pankaj Garg 87b2525e03
Limit maximum number of shreds in a slot to 32K (#7584)
* Limit maximum number of shreds in a slot to 32K

* mark dead slot replay as fatal error
2019-12-30 07:42:09 -08:00
Parth 727be309b2 fix entryverification state (#7169)
automerge
2019-12-23 23:26:27 -08:00
dependabot-preview[bot] ce2d7a2d5a Bump nix from 0.16.0 to 0.16.1 (#7623)
Bumps [nix](https://github.com/nix-rust/nix) from 0.16.0 to 0.16.1.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-23 23:06:42 -07:00
dependabot-preview[bot] 488420fdf2 Bump core_affinity from 0.5.9 to 0.5.10 (#7578)
Bumps [core_affinity](https://github.com/Elzair/core_affinity_rs) from 0.5.9 to 0.5.10.
- [Release notes](https://github.com/Elzair/core_affinity_rs/releases)
- [Commits](https://github.com/Elzair/core_affinity_rs/compare/0.5.9...0.5.10)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-22 22:41:47 -07:00
Dan Albert ca10cf081f
Update cargo.toml files from 0.22.0 to 0.23.0 (#7596) 2019-12-20 21:45:42 -05:00
Rob Walker a7040896f0
Update to rust 1.40.0 (#7572)
* Update to rust 1.40.0

* fixups
2019-12-19 23:27:54 -08:00
carllin e98132fd76
Move window insertion to separate thread (#7397)
* Move window insertion to separate thread
2019-12-19 00:15:49 -08:00
Michael Vines fcda972cec Add support for multiple params 2019-12-18 23:34:01 -07:00
Tyera Eulberg c5b076ec7e Add getConfirmedBlocks rpc method (#7550)
automerge
2019-12-18 15:51:47 -08:00
Sunny Gleason 05cf5a38af
Add ledger benchmark test (#7484) 2019-12-18 18:31:04 -05:00
Michael Vines bd22b641b3 Speed up getLeaderSchedule RPC call by reducing pubkey duplication 2019-12-18 15:59:55 -07:00
Sagar Dhawan 6a9005645a
Update "limit-ledger-size" to use DeleteRange for much faster deletes (#7515)
* Update "limit-ledger-size" to use DeleteRange for much faster deletes

* Update core/src/ledger_cleanup_service.rs

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Rewrite more idiomatically

* Move max_ledger_slots to a fn for clippy

* Remove unused import

* Detect when all columns have been purged and fix a bug in deletion

* Check that more than 1 column is actually deleted

* Add helper to test that ledger meets minimum slot bounds

* Remove manual batching of deletes

* Refactor to keep some N slots older than the highest root

* Define MAX_LEDGER_SLOTS that ledger_cleanup_service will try to keep around

* Refactor compact range
2019-12-18 11:50:09 -08:00
Tyera Eulberg 6aaf742dfe Extend getConfirmedBlock rpc to return account pre- and post-balances (#7543)
automerge
2019-12-18 09:56:29 -08:00
Sunny Gleason 323673c3c0
Add `compact_cf` calls to reclaim storage during ledger slot purge (#7264) 2019-12-18 10:29:46 -05:00
Michael Vines 434cde179f GetLeaderSchedule can now return a schedule for arbitrary epochs 2019-12-17 23:41:42 -07:00
anatoly yakovenko 97589f77f8 Pipeline broadcast socket transmit and blocktree record (#7481)
automerge
2019-12-16 17:11:18 -08:00
Tyera Eulberg 3513f4ee84
Rename drone to faucet (#7508) 2019-12-16 14:05:17 -07:00
dependabot-preview[bot] 7ad64c8d45 Bump serde from 1.0.103 to 1.0.104
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.103 to 1.0.104.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.103...v1.0.104)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-16 08:50:09 -07:00
dependabot-preview[bot] 71b93468d5 Bump serial_test from 0.3.1 to 0.3.2
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.3.1...v0.3.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-15 18:19:37 -07:00
Tyera Eulberg 6b88da2b82
Hook up getBlockTime rpc to real data (#7476)
* Add blocktree timestamp helper functions and tests

* Flesh out blocktree::get_block_time

* Move stakes up into rpc to make testing easier; expand tests

* Review comments
2019-12-14 12:23:02 -07:00
Michael Vines d01ea20273 getVoteAccounts now excludes listing inactive unstaked accounts as delinquent
This keeps abandoned vote accounts out of the `solana show-validators` output
2019-12-14 09:52:03 -07:00
sakridge 98b80288ed
Optimize bank_forks critical section (#7477) 2019-12-13 17:20:31 -08:00
sakridge 9d5a07bac4
Move create_dir_all to AccountsDB::new (#7465)
AppendVec create doesn't need to try and create
paths every time and it can stall while snapshot
create is happening.
2019-12-13 16:46:16 -08:00
Pankaj Garg d94041e98d
Allow coding shred index to be different than data shred index (#7438)
* Allow coding shred index to be different than data shred index

* move fec_set_index to shred's common header

* fix bench
2019-12-12 16:50:29 -08:00
Jack May 1d172b07a8
crc 1.9.0 was yanked, fall back to 1.8 (#7453) 2019-12-12 14:30:51 -08:00
Sunny Gleason 83218c479a
code cleanup, storage_size() was Option<u64>, now Result<u64> (#7424) 2019-12-12 14:55:30 -05:00
Jack May dbb8267b09
Pin dalek version (#7448)
* Pin Dalek version

* core too
2019-12-12 11:34:43 -08:00
sakridge dd54fff978
Use pinned memory for entry verify (#7440) 2019-12-12 10:36:27 -08:00
Justin Starry dad5c62df5 Add uptime column to show-validators (#7441)
automerge
2019-12-11 22:04:54 -08:00
dependabot-preview[bot] 46a4ea8f67 Bump reqwest from 0.9.22 to 0.9.24
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.9.22 to 0.9.24.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/v0.9.24/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.9.22...v0.9.24)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-11 13:42:38 -07:00
Sunny Gleason 06415de8ee
change blocktree*::storage_size() to return Option<u64> to handle live fs changes (#7401) 2019-12-10 19:12:49 -05:00
carllin 146bc95c16
Fix sigverify metrics (#7393)
* Add filter

* Collapse logs

* Fix dashboard
2019-12-10 11:28:07 -08:00
dependabot-preview[bot] 6f457292ff Bump bincode from 1.2.0 to 1.2.1
Bumps [bincode](https://github.com/servo/bincode) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/servo/bincode/releases)
- [Commits](https://github.com/servo/bincode/compare/v1.2.0...v1.2.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-10 10:00:56 -07:00
Parth 6d2861f358
add unit test for minority fork overcommit attack (#7292)
* add unit test for minority fork overcommit attack

* add generic function to simulate fork selection
2019-12-10 22:06:16 +05:30
dependabot-preview[bot] 891767c6b7 Bump serde_json from 1.0.42 to 1.0.44
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.42 to 1.0.44.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.42...v1.0.44)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-08 09:43:24 -07:00
Trent Nelson 1ffd6b4b4d
Add program and runtime support for Durable Transaction Nonces (#6845)
* Rework transaction processing result forwarding

Durable nonce prereq

* Add Durable Nonce program API

* Add runtime changes for Durable Nonce program

* Register Durable Nonce program

* Concise comments and bad math

* Fix c/p error

* Add rent sysvar to withdraw ix

* Remove rent exempt required balance from Meta struct

* Use the helper
2019-12-07 12:54:10 -07:00
Michael Vines 77cd292828 getVoteAccounts RPC API no longer returns "idle" vote accounts, take II 2019-12-07 01:16:36 -07:00
Michael Vines 22d6951de5 Revert "getVoteAccounts RPC API no longer returns "idle" vote accounts"
This reverts commit 103e40b9689d74ece606385a67d166f55f453823.
2019-12-07 01:16:36 -07:00
Sunny Gleason c00216e3be
feat: ledger size and cleanup metrics (#7335) 2019-12-06 22:32:45 -05:00
Michael Vines 42247e0e1a getVoteAccounts RPC API no longer returns "idle" vote accounts (#7339) 2019-12-06 20:23:29 -05:00
TristanDebrunner 9ecb844de7 Split up ReplayStageConfig to make it derive Default (#7334)
automerge
2019-12-06 14:39:35 -08:00
Tyera Eulberg 3ab8185777
Add intermittent Timestamping to Votes (#7233)
* Add intermittent timestamp to Vote

* Add timestamp to VoteState, add timestamp processing to program

* Print recent timestamp with solana show-vote-account

* Add offset of 1 to timestamp Vote interval to initialize at node boot (slot 1)

* Review comments

* Cache last_timestamp in Tower and use for interval check

* Move work into Tower method

* Clarify timestamping interval

* Replace tuple with struct
2019-12-06 14:38:49 -07:00
Justin Starry b7d4330dd4
Fail fast if account paths cannot be canonicalized (#7300)
* Canonicalize account paths to avoid symlink issues

* fixes
2019-12-05 21:41:29 -05:00
sakridge cfc21e1225
Only serialize rooted append vecs (#7281) 2019-12-05 14:27:46 -08:00
Justin Starry d6c3396182
Canonicalize paths before symlink-ing when generating snapshots (#7294)
* Canonicalize paths before symlinking when generating snapshots

* cargo fmt
2019-12-05 14:58:02 -05:00
Sagar Dhawan a95d37ea25
Fix repair slowness when most peers are unable to serve requests (#7287)
* Fix repair when most peers are incapable of serving requests

* Add a test for getting the lowest slot in blocktree

* Replace some more u64s with Slot
2019-12-05 11:25:13 -08:00
TristanDebrunner fae9c08815
Add ReplayStageConfig (#7195) 2019-12-04 11:17:17 -07:00
dependabot-preview[bot] c9245751e9 Bump nix from 0.15.0 to 0.16.0 (#7216)
Bumps [nix](https://github.com/nix-rust/nix) from 0.15.0 to 0.16.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits/v0.16.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-04 09:35:45 -07:00
carllin f0a40862d6
Support local cluster edge case testing (#7135)
* Refactor local cluster to support killing a partition

* Rework run_network_partition

* Introduce fixed leader schedule

* Plumb fixed schedule into test
2019-12-03 16:31:59 -08:00
Pankaj Garg a0eafa12e3
Update to hashes_per_tick computation, and tick duration datapoint (#7127) 2019-12-02 18:02:11 -08:00
Pankaj Garg 076e384bb5
Tool to tune system parameters like PoH service priority (#7155)
* New daemon to tune system parameters like PoH service priority

* fixes for Linux

* integrate with poh_service

* fixes

* address review comments

* remove `dead_code` directive
2019-12-02 16:46:46 -08:00
Michael Vines 41cff1b49d
Remove spammy log 2019-12-02 16:52:01 -07:00
dependabot-preview[bot] 0f872af502 Bump libc from 0.2.65 to 0.2.66 (#7180)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.65 to 0.2.66.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.65...0.2.66)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-02 09:56:03 -07:00
dependabot-preview[bot] b13696ea1a Bump jsonrpc-http-server from 14.0.3 to 14.0.5 (#7181)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.3...jsonrpc-http-server-v14.0.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-02 09:55:40 -07:00
dependabot-preview[bot] 5fbbf7c748 Bump jsonrpc-ws-server from 14.0.3 to 14.0.5 (#7172)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.3...jsonrpc-ws-server-v14.0.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-29 23:06:23 -07:00
dependabot-preview[bot] e7fe0db051 Bump jsonrpc-derive from 14.0.3 to 14.0.5 (#7173)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.3...jsonrpc-derive-v14.0.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-29 14:15:02 -07:00
dependabot-preview[bot] dcb7bd8c74 Bump jsonrpc-pubsub from 14.0.3 to 14.0.5 (#7174)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.3...jsonrpc-pubsub-v14.0.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-29 14:14:57 -07:00
dependabot-preview[bot] f97626346b Bump jsonrpc-core from 14.0.3 to 14.0.5 (#7170)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-28 10:59:31 -07:00
dependabot-preview[bot] 7f4feaee08 Bump serde from 1.0.102 to 1.0.103 (#7163)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.102 to 1.0.103.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.102...v1.0.103)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-27 09:10:25 -07:00
dependabot-preview[bot] 5a30ef180a Bump serial_test_derive from 0.2.0 to 0.3.1 (#7153)
Bumps [serial_test_derive](https://github.com/palfrey/serial_test) from 0.2.0 to 0.3.1.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.2.0...v0.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-26 23:12:13 -07:00
dependabot-preview[bot] 0a0412e47e Bump serial_test from 0.2.0 to 0.3.1 (#7154)
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.2.0 to 0.3.1.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.2.0...v0.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-26 22:07:39 -07:00
dependabot-preview[bot] c810913861 Bump serde_derive from 1.0.102 to 1.0.103 (#7141)
automerge
2019-11-26 14:44:20 -08:00
dependabot-preview[bot] 2b13158e29 Bump serde_json from 1.0.41 to 1.0.42 (#7142)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.41 to 1.0.42.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.41...v1.0.42)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-26 15:09:01 -07:00
Sunny Gleason 0a0f15baca
RPC subscriptions for new slot notifications (#7114)
* feat: slot notifications via pubsub rpc w/ tests
2019-11-26 03:42:54 -05:00
Tyera Eulberg 58c144ee55
Add getBlockTime rpc api (#7130)
* Add getBlockTime rpc api

* Add getBlockTime to docs

* Fix duration rounding for common tick/slot durations; add slot duration calculation

* Expose slots_per_year

* Use genesis values instead of clock defaults to calculate block offset

* Add get-block-time cli subcommand

* Fix test_rent: decrease magic number usage
2019-11-26 00:40:36 -07:00
Michael Vines 294662a1ce
Bump version to 0.22.0 2019-11-25 21:34:50 -07:00
Sagar Dhawan 76a68c26c9
Track a Bank's parent slot independently from parent bank (#7131) 2019-11-25 15:34:51 -08:00
Tyera Eulberg 0f66e5e49b Add getConfirmedBlock test to rpc (#7120)
automerge
2019-11-25 11:08:03 -08:00
dependabot-preview[bot] 686aa3a150 Bump chrono from 0.4.9 to 0.4.10 (#7113)
automerge
2019-11-25 10:01:46 -08:00
dependabot-preview[bot] 094c391cd7 Bump itertools from 0.8.1 to 0.8.2 (#7111)
Bumps [itertools](https://github.com/bluss/rust-itertools) from 0.8.1 to 0.8.2.
- [Release notes](https://github.com/bluss/rust-itertools/releases)
- [Commits](https://github.com/bluss/rust-itertools/commits/v0.8.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-25 10:22:47 -07:00
sakridge 6c89226ccf
Purge zero lamport accounts on snapshot ingestion (#7010)
Snapshots do not load the original index, so they must
purge zero lamport accounts again.
2019-11-22 18:22:28 -08:00
Michael Vines 0bd41f98ed Avoid jemalloc in windows build (#7089)
automerge
2019-11-21 18:39:29 -08:00
anatoly yakovenko d9e7a5fcbe
Use fork weight instead of individual bank weight for fork selection. (#7079)
* Fix weight calculation

* Fix tests

* fork weight

* wait until nodes are in the leader schedule

* enable sanity

* fewer long tests
2019-11-21 15:47:08 -08:00
Tyera Eulberg c965a110f2
Use unbounded channel (#7081) 2019-11-21 14:23:40 -07:00
Ryo Onodera 8cbc450192 Create genesis.tar.bz2 in solana-genesis (#7039)
* Use clap_utils

* Create genesis.tar.bz2 in solana-genesis

* Remove shell-based genesis.tar.bz2 generation

* Make Option=>Result conv more rusty

* stop using solana_logger

* Simplify by just using vec!

* clean up abit
2019-11-21 10:57:27 -07:00
sakridge e66b29943b
datapoint for best fork weight and slot in replay (#7066) 2019-11-20 17:26:52 -08:00
Sagar Dhawan 100b9dd12a
Fix num nodes metrics (#7068)
* Fix num nodes metric

* Fix node count metrics
2019-11-20 17:00:31 -08:00
Jack May 3415db9739
Merge api/program into single units (#7061) 2019-11-20 16:32:19 -08:00
Tyera Eulberg 97ca6858b7
Write transaction status and fee into persistent store (#7030)
* Pass blocktree into execute_batch, if persist_transaction_status

* Add validator arg to enable persistent transaction status store

* Pass blocktree into banking_stage, if persist_transaction_status

* Add validator params to bash scripts

* Expose actual transaction statuses outside Bank; add tests

* Fix benches

* Offload transaction status writes to a separate thread

* Enable persistent transaction status along with rpc service

* nudge

* Review comments
2019-11-20 16:43:10 -07:00
Parth 9f6a2e51b2
add credit-debit rent handling (#6947)
* add credit-debit rent handling

* add tests

* charge rent for validator account for fee credit

* rent is stored per tx instead of account
2019-11-21 01:57:02 +05:30
anatoly yakovenko b150da837a Use epoch as the gossip purge timeout for staked nodes. (#7005)
automerge
2019-11-20 11:25:18 -08:00
Jack May d184d3a732
Merge native programs parts into one unit (#7047) 2019-11-20 10:12:43 -08:00
Jack May d2ed921bc6
Cleanup nightly warnings (#7055) 2019-11-19 20:15:37 -08:00
Pankaj Garg d32a072190
Use ticks_per_slot to calculate maximum grace ticks (#7024)
* Use ticks_per_slot to calculate maximum grace ticks

* fix test

* fix votable candidate ordering

* fixes to pick_best_fork() and a unit test

* fixes
2019-11-19 17:55:42 -08:00
Sagar Dhawan f2badf2c5d
Fix a bug where gossip loops forever while splitting messages (#7032)
* Fix a bug where gossip loops forever while splitting messages

* Get rid of while loop

* Minor clean up and rename
2019-11-19 11:51:51 -08:00
Tyera Eulberg ea656b1a3f
Add parent slot to getConfirmedBlock (#7038)
* Add parent slot to getConfirmedBlock

* Fix bad text-replace

* Use camelCase in getConfirmedBlock
2019-11-19 09:39:55 -07:00
Dan Albert b958bf9086
Fix confirmation metrics (#7035) 2019-11-19 09:51:50 -05:00
carllin 43144cfe8b
Make banks that fail threshhold check resettable (#7027) 2019-11-19 02:36:30 -08:00
carllin 11d2d2eccd
Fix progress map losing banks and recomputing stats (#7026)
* Fix progress map missing banks

* Fix confirmations

* Fix test

* Initialiize progress with frozen banks atartup
2019-11-19 02:36:00 -08:00
Sagar Dhawan 6bfe0fca1f
Add a version field to shreds (#7023)
* Add a version field to shreds

* Clippy

* Fix Chacha Golden

* Fix shredder bench compile

* Fix blocktree bench compile
2019-11-18 18:05:02 -08:00
Tyera Eulberg e0a2bb9d86
Legitimately map transactions to statuses in blocktree (#7011)
* Refactor rocksdb TransactionStatus to store/return struct; hook up map_transactions_to_statuses

* Cleanup use statements
2019-11-18 09:12:42 -07:00
Tyera Eulberg 3bc8d78801
Add ConfirmedBlock struct, and rework Blocktree apis to include block… (#7004)
* Add RpcConfirmedBlock struct, and rework Blocktree apis to include blockhash info and dummy tx statuses

* Remove unused lifetime
2019-11-17 20:17:15 -07:00
carllin b66c03667c
Log for threshold failure (#7008) 2019-11-17 17:10:16 -08:00
Michael Vines c3926e6af0
|solana-gossip spy| no longer requires an entrypoint (#6999) 2019-11-16 14:16:28 -07:00
carllin 70322d1ff8
Add error logging to dead slots (#7000) 2019-11-16 02:54:51 -08:00
anatoly yakovenko 0ca943f49b RecyclerCache for shred_sigverify (#6986)
automerge
2019-11-15 12:16:56 -08:00
sakridge 36e3ccfc68
Remvoe pinned memory (#6976) 2019-11-15 10:58:25 -08:00
anatoly yakovenko 59413b3124 Fix rules for fork selection (#6906)
automerge
2019-11-15 08:36:33 -08:00
Michael Vines 006c39380a
Display 'none' instead of 0.0.0.0 (#6973) 2019-11-14 20:24:35 -07:00
Tyera Eulberg 852a2146ab
Add Blocktree api to get transactions by slot (#6966)
* Add blocktree method to get confirmed-block txs

* Clean up use statements

* Add test, and fmt

* Plumb new blocktree method into getConfirmedBlock
2019-11-14 16:34:39 -07:00
sakridge 99b42f210c Remove unused sha2 dep (#6964)
automerge
2019-11-14 14:01:11 -08:00
Tyera Eulberg 03f7f0d18c Rename getBlock to getConfirmedBlock; remove getBlocksSince (#6961)
automerge
2019-11-14 13:14:42 -08:00
Sagar Dhawan 79d7090867
Remove obsolete references to Blob (#6957)
* Remove the name "blob" from archivers

* Remove the name "blob" from broadcast

* Remove the name "blob" from Cluset Info

* Remove the name "blob" from Repair

* Remove the name "blob" from a bunch more places

* Remove the name "blob" from tests and book
2019-11-14 11:49:31 -08:00
Michael Vines e7f63cd336
Upgrade to rust 1.39.0 (#6939)
* Upgrade to rust 1.39.0

* 1.39.0 clippy
2019-11-14 12:27:01 -07:00
Sagar Dhawan f108f483b7
Remove Blobs and switch to Packets (#6937)
* Remove Blobs and switch to Packets

* Fix some gossip messages not respecting MTU size

* Failure to serialize is not fatal

* Add log macros

* Remove unused extern

* Apparently macro use is required

* Explicitly scope macro

* Fix test compile
2019-11-14 10:24:53 -08:00
Sunny Gleason 42af8b199f
feat: add tests for invalid/failure cases (#6951) 2019-11-14 11:41:26 -05:00
carllin 43e2301e2c
Fix roots overrunning broadcast (#6884)
* Add trusted pathway for insert_shreds to avoid checks
2019-11-14 00:32:07 -08:00
Parth 7b05b3dbb3
rent collector improvments (#6888)
* avoid account copying + pre-empt rent

* adding support for base rent
2019-11-14 10:56:49 +05:30
Ryo Onodera 4fc767b3f6
Move version! from core:: to clap_utils:: (#6944)
* Move version! from core to clap-utils

* Completely move version! from core:: to clap_utils::

* rustfmt

* Do remaining transition after rebase
2019-11-14 13:10:38 +09:00
Greg Fitzgerald a3a830e1ab
Delete Service trait (#6921) 2019-11-13 11:12:09 -07:00
Michael Vines fcc2874591
Remove/address some TODOs (#6923) 2019-11-13 09:43:15 -07:00
Greg Fitzgerald 30a08f4282 Cleanup ledger macros (#6916)
automerge
2019-11-13 07:14:09 -08:00
Michael Vines 86faa3f995
Properly type RpcClient::get_version() (#6919) 2019-11-12 22:01:04 -07:00
Ryo Onodera 148a58865e Make creating new snapshot.tar.bz2 truly-atomic (#6902) 2019-11-12 14:21:56 -07:00
Ryo Onodera 3faeb7fa79 Rename solana-netutil to solana-net-utils for consistency (#6895)
* sed -i -e 's/netutil/net_utils/g' $(git grep --files-with-matches netutil :**.rs)

* sed -i -e 's/netutil/net-utils/g' $(git grep --files-with-matches netutil)

* git mv netutil/ net-utils

* Tweak a bit

* Fix rustfmt & clippy
2019-11-12 13:37:13 -07:00
Rob Walker bb00904fc8
add rent reserve for bootstrap stakes (#6876)
* genesis investor stakes

* assert rent is sufficient for these bootstrappers
2019-11-12 12:33:40 -08:00
Sunny Gleason 5903339c17
feat: return bank/block info with block-related results (#6716) 2019-11-12 14:49:41 -05:00
Michael Vines c6b108ef4f
Don't panic in sdk/ when genesis fails to load (#6892) 2019-11-12 10:24:49 -07:00
Tyera Eulberg 7e82450d7b
Serialize transaction in proper wire format instead of json (#6889) 2019-11-12 10:45:10 -05:00
Sagar Dhawan 95d6586dd7
Remove debug datapoint that isn't being plotted (#6873) 2019-11-11 14:25:25 -08:00
Sagar Dhawan 816b2d7ff8
Tune repair to be less aggressive (#6868) 2019-11-11 13:12:22 -08:00
Michael Vines 4be646c695
discover() by gossip sockaddr instead of just by gossip ip address (#6865) 2019-11-11 12:42:58 -07:00
Tyera Eulberg cc6e1ea200
Stub out `getBlocksSince` and `getBlock` methods (#6853)
* Add getBlocksSince rpc method, and initial stub of getBlock method

* Return test transactions from getBlock method

* clippy

* Add comment on get_block method
2019-11-11 13:18:34 -05:00
Michael Vines cfab36cb1d
Include channel and commit info in the version of pre-release builds (#6819) 2019-11-10 22:39:13 -07:00
Justin Starry 5835b3b8eb
Increase timeout when confirming airdrop for max commitment (#6858)
* Increase timeout when confirming airdrop for max commitment

* Add commitment to airdrop rpc trace

* Flip commitment check
2019-11-10 12:20:52 -05:00
Michael Vines 1ddf90ed08
Compress contact_info_trace() output to improve CI log rendering (#6852) 2019-11-09 01:12:18 -07:00
Justin Starry 9807f47d4e
Rename genesis block to genesis config (#6816) 2019-11-08 23:56:57 -05:00
Justin Starry 02058ea699
Reject blocks with invalid last ticks in replay stage (#6833)
* Reject blocks with invalid last ticks in replay stage

* slot_full
2019-11-08 20:21:54 -05:00
Michael Vines d1daeb44e6
Remove custom stack_size() (#6844) 2019-11-08 17:11:07 -07:00
Michael Vines efdfc5c327
Remove TODOs (#6843) 2019-11-08 16:43:18 -07:00
Michael Vines 9c00ad9ff2
Remove some low-hanging TODOs (#6839) 2019-11-08 16:41:36 -07:00
Pankaj Garg da425cc225
Don't insert coding shreds into blocktree on leader (#6831) 2019-11-08 13:54:23 -08:00
Michael Vines b800642fa4
Add new fork log message for when the node is leader for consistency (#6808) 2019-11-08 12:30:25 -07:00
Parth 5bd05fba09
require `to` account signature (#6658)
* require to signature

* fixing invocation to create_account

* fix create_account references

* address review comment

* whacking bugs in tests

* fixing stake program tests
2019-11-08 15:57:35 +05:30
sakridge 8e81bc1b49
Fix pinning (#6604)
Remove Deref implementations and add more pass-throughs to the PinnedVec
wrapper.
Warm recyclers
set_pinnable
2019-11-07 19:48:33 -08:00
Rob Walker b64b54f48f
unfork dalek ed25519 (#6776) 2019-11-07 17:08:10 -08:00
Sagar Dhawan 67d1e2903c
Upgrade Repair be more intelligent and agressive (#6789)
* Upgrade Repair be more intelligent and agressive

* Fix u64 casts

* Fix missing bracket

* Add 1 second delay to test to allow repair to kick in
2019-11-07 11:08:09 -08:00
Sunny Gleason a684984f8b
feat: add confirm_transaction, add rpc client test (#6778) 2019-11-06 22:08:03 -05:00
Jack May 65de227520
Don't panic on packet data (#6769) 2019-11-06 14:32:37 -08:00
Pankaj Garg 0ace79939b
Add reference tick to data shreds (#6772)
* Add reference tick to data shreds

* fix tests
2019-11-06 13:27:58 -08:00
Tyera Eulberg b3a75a60a4
Use rooted bank by default in rpc bank selection (#6759)
* Name anonymous parameters for clarity

* Add CommitmentConfig to select bank for rpc

* Add commitment information to jsonrpc docs

* Update send_and_confirm retries as per commitment defaults

* Pass CommitmentConfig into client requests; also various 'use' cleanup

* Use _with_commitment methods to speed local_cluster tests

* Pass CommitmentConfig into Archiver in order to enable quick confirmations in local_cluster tests

* Restore solana ping speed

* Increase wallet-sanity timeout to account for longer confirmation time
2019-11-06 14:15:00 -07:00
anatoly yakovenko 5e8668799c Fewer recyclers. (#6770)
automerge
2019-11-06 12:35:51 -08:00
anatoly yakovenko 67f636545a Refactor sigverify to stage for signing shreds on the GPU (#6635)
automerge
2019-11-06 10:52:30 -08:00
carllin 7ff2a44a63 Make last shred for an interrupted slot signed + typed (#6760) 2019-11-06 08:25:17 -07:00
Michael Vines b5074d8577
Enable JSON RPC request/response logging by default (#6758) 2019-11-06 08:23:13 -07:00
carllin 24102a7435
Allow voting on empty banks (#6719)
* Allow votes on empty banks

* Remove making first bank is_delta true, no longer necessary for idling

* Remove votable from ledger tool
2019-11-06 01:02:26 -08:00
Jack May 9614d17024
Limit deserialization of data coming off the wire (#6751)
* Limit deserialization of data coming off the wire

* Feedback and cleanup
2019-11-06 00:07:57 -08:00
Michael Vines 09e648f957
ledger-tool/: Include full validator voting history in fork-graph (#6756) 2019-11-05 19:40:00 -07:00
carllin 8f91b5aab3
Add threshold to repairman for same slot (#6728) 2019-11-05 12:48:45 -08:00
sakridge c312d4fba0
Calculate proofs collected and don't encrypt if there are none (#6698) 2019-11-05 11:38:50 -08:00
Michael Vines 7203036e3e Adjust nofiles within Blocktree::open() for all ledger/ users (#6737)
automerge
2019-11-05 11:18:49 -08:00
Tyera Eulberg c6931dcb07
Remove credit-only account handling (#6726)
* Renaming
- credit-only/credit-debit to read-only/read-write
- debitable to writable

* Remove credit handling, making credit-only accounts read-only

* Update programs to remove deprecated credit-only account designation

* Use readonly and writable instead of underscored types
2019-11-05 09:38:35 -07:00
Michael Vines fba1af6ea9
ledger-tool can now load a ledger snapshot (#6729) 2019-11-04 22:14:55 -07:00
anatoly yakovenko b825d04597 Pull perf into a separate module. (#6718)
automerge
2019-11-04 20:13:43 -08:00
Michael Vines 4d52f47f87
Move get_bank_forks() into ledger/ so its available for use by ledger-tool/ (#6720) 2019-11-04 19:10:06 -07:00
anatoly yakovenko f54cfcdb8f
Store and persists full stack of tower votes in gossip (#6695)
* vote array

wip

wip

wip

update

gossip index should match tower index

tests build

clippy

test index after expired vote

test

bank specific last vote sync time

* verify

* we are likely to see many more warnings about old votes now
2019-11-04 16:19:54 -08:00
sakridge 57983980a7 Lower verify-batch-size to debug (#6722)
automerge
2019-11-04 16:00:59 -08:00
Tyera Eulberg 33f4aaf3fd
Rename confidence to commitment (#6714) 2019-11-04 16:44:27 -07:00
Pankaj Garg c138d692b1
Show all ports for nodes in gossip table (#6717)
* Show all ports for nodes in gossip table

* review comments
2019-11-04 15:05:08 -08:00
Michael Vines 5e3697807c
Fail gracefully if AVX support is missing (#6705) 2019-11-04 11:03:39 -07:00
Sagar Dhawan 568475e2db
Fix incorrectly signed CrdsValues (#6696) 2019-11-03 10:07:51 -08:00
anatoly yakovenko 9ea398416e
Sign shreds on the GPU (#6595)
* sign gpu shreds

* wip

* checks

* tests build

* test

* tests

* test

* nits

* sign cpu test

* write out the sigs in parallel

* clippy

* cpu test

* prepare secret for gpu

* woot!

* update

* bump perf libs
2019-11-02 06:23:14 -07:00
Michael Vines 50a17fc00b Use Slot and Epoch type aliases instead of raw u64 (#6693)
automerge
2019-11-02 00:38:30 -07:00
Sagar Dhawan a57f6b70da Fix swapped repair and forwards addrs (#6691)
automerge
2019-11-01 16:01:42 -07:00
anatoly yakovenko 385b4ce959
Get rid of verified packets and use the Meta::discard flag (#6674)
* get rid of verified packets and use the disabled meta field everywhere
2019-11-01 14:23:03 -07:00
Sagar Dhawan 2d67962c2f
Send repairman shreds to the repair socket (#6671) 2019-10-31 18:23:50 -07:00
Justin Starry e8e5ddc55d
Verify number of hashes for each block of entries (#6262)
* Verify number of hashes for each block of entries

* Fix blocktree processor tick check

* Rebase once more
2019-10-31 16:38:50 -04:00
carllin 59f3dc3b6b
Fix PohRecorder Metrics (#6644)
* Update Poh Recorder Dashboard

* Update PohRecorder logging
2019-10-30 18:55:29 -07:00
Rob Walker fa12a5f70b
kill rent calculator (#6625) 2019-10-30 16:25:12 -07:00
sakridge 38491c8c4b
Reduce verify-batch-size log (#6623) 2019-10-30 13:41:11 -07:00
Sagar Dhawan 801337a422 Refactor Weighted Shuffle (#6614)
automerge
2019-10-29 21:02:11 -07:00
TristanDebrunner b4dc1a7263 Remove move feature (#6605)
automerge
2019-10-29 17:14:07 -07:00
Sagar Dhawan ef3aa2731c Fix Weighted Best calculation (#6606)
automerge
2019-10-29 17:04:11 -07:00
dependabot-preview[bot] 029a2837e4 Bump jsonrpc-http-server from 14.0.1 to 14.0.3 (#6597)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.1...v14.0.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-29 10:30:06 -07:00
anatoly yakovenko 34a9619806
SigVerify stage for shreds. (#6563) 2019-10-28 16:07:51 -07:00
Sagar Dhawan 579a02529d
Fix unnecessarily copying shreds in broadcast stage (#6588)
* Optimize coalesce_shreds to not explictly clone

* Remove Coalesce Shreds altogether

* fn no longer needs clippy exception
2019-10-28 14:58:27 -07:00
anatoly yakovenko 243fa6cf63
Shred gpu sigverify (#6520)
Implement APIs for verifying shred signatures on the GPU.
2019-10-28 10:29:38 -07:00
dependabot-preview[bot] 30c0a7d069 Bump serde from 1.0.101 to 1.0.102 (#6581)
automerge
2019-10-28 09:19:39 -07:00
dependabot-preview[bot] 71b4e765c8 Bump itertools from 0.8.0 to 0.8.1 (#6583)
Bumps [itertools](https://github.com/bluss/rust-itertools) from 0.8.0 to 0.8.1.
- [Release notes](https://github.com/bluss/rust-itertools/releases)
- [Commits](https://github.com/bluss/rust-itertools/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-28 08:26:15 -07:00
dependabot-preview[bot] 73dd5aa2d1 Bump serde_derive from 1.0.101 to 1.0.102 (#6582)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.101 to 1.0.102.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.101...v1.0.102)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-28 08:24:13 -07:00
carllin 6efaaa9d7a
Blocktree metrics (#6527)
* Add metrics for blocktree performance
* Plumb metrics through window service
2019-10-26 16:15:59 -07:00
Dan Albert 73c31d873e
Update Cargo.toml versions from 0.20.0 to 0.21.0 (#6568) 2019-10-25 17:40:49 -06:00
dependabot-preview[bot] 8a37b1e742 Bump jsonrpc-ws-server from 14.0.1 to 14.0.3 (#6553)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 09:40:42 -07:00
dependabot-preview[bot] 0cf4eb2ee4 Bump jsonrpc-core from 14.0.1 to 14.0.3 (#6552)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 09:40:25 -07:00
dependabot-preview[bot] 5496f85dbc Bump crc from 1.8.1 to 1.9.0 (#6511)
Bumps [crc](https://github.com/mrhooray/crc-rs) from 1.8.1 to 1.9.0.
- [Release notes](https://github.com/mrhooray/crc-rs/releases)
- [Commits](https://github.com/mrhooray/crc-rs/compare/1.8.1...v1.9.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 09:40:17 -07:00
dependabot-preview[bot] 1738632822 Bump jsonrpc-pubsub from 14.0.1 to 14.0.3 (#6551)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 07:41:09 -07:00
dependabot-preview[bot] bbd5dde66d Bump jsonrpc-derive from 14.0.1 to 14.0.3 (#6554)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.1...v14.0.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 07:41:03 -07:00
sakridge 53c7be32b6
Add more retransmit and streamer stats (#6534) 2019-10-24 19:27:19 -07:00
Michael Vines 397ea05aa7
spy nodes are now gossip entrypoints (#6532) 2019-10-24 15:35:33 -07:00
Rob Walker b4119c454a
credit_only credits forwarding (#6509)
* credit_only_credits_forwarding

* whack transfer_now()

* fixup

* bench should retry the airdrop TX

* fixup

* try to make bench-exchange a bit more robust, informative
2019-10-23 22:01:22 -07:00
sakridge 03d29a8311
Async poh verify (#6353)
* Async poh verify

* Up ticks_per_s to 160

GPU poh verify needs shorter poh sequences or it takes forever to
verify. Keep slot time the same at 400ms.

* Fix stats

* Don't halt on ticks

* Increase retries for local_cluster tests and make repairman test serial
2019-10-23 12:11:04 -07:00
Michael Vines 35cc74ef25
Add GenesisBlock::OperatingMode to control how cluster features are activated (#6430) 2019-10-23 11:50:10 -07:00
Sunny Gleason 0b2d4f32fa
feat: get epoch schedule rpc, update cli (#6500) 2019-10-22 16:41:18 -04:00
sakridge e6438098e1 Increase archiver polling timeout (#6501)
automerge
2019-10-22 12:15:55 -07:00
carllin b38bf90de7
Deshred blocks in parallel (#6461)
* Deshred in parallel

* Add tests for corrupt slots and parallel deshred

* Rename load_blocktree_entries to load_blocktree_entries_with_shred_count
2019-10-21 16:15:10 -07:00
Pankaj Garg 84e911361a
Use constants instead of lazy_static for shred header sizes (#6472) 2019-10-21 12:46:16 -07:00
Sagar Dhawan 364583ea5c
Fix copying packets in Window Service (#6429)
* Fix copying packets in Window Service

* Parallelize over batches instead of within batches
2019-10-21 12:04:52 -07:00
Greg Fitzgerald 9232057e95
Rename replicator to archiver (#6464)
* Rename replicator to archiver

* cargo fmt

* Fix grammar
2019-10-21 11:29:37 -06:00
dependabot-preview[bot] 91920cc390 Bump jsonrpc-pubsub from 14.0.0 to 14.0.1 (#6465)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.0...v14.0.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-21 07:48:28 -07:00
dependabot-preview[bot] cc1cc7be94 Bump jsonrpc-derive from 14.0.0 to 14.0.1 (#6466)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.0...v14.0.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-21 07:48:14 -07:00
Greg Fitzgerald 2636418659
Move blocktree_processor to solana_ledger (#6460)
* Drop core::result dependency in bank_forks

* Move blocktree_processor into solana_ledger
2019-10-20 09:54:38 -06:00
Justin Starry 31e9074ae5
Rename leader_after_slots to leader_after_n_slots (#6459) 2019-10-19 23:28:33 -04:00
Rob Walker e2c316d2d0
system_instruction_processor updates (#6448)
* zero lamport account creation

* whack create_user_account, take 2

* target->to

* ..

* ..

* update chacha golden

* update chacha golden

* ..

* ..
2019-10-19 18:23:27 -07:00
sakridge 74ee88d9bc
Add storage stage and bank_forks tests to integration (#6458) 2019-10-19 12:09:45 -07:00
Pankaj Garg badeb4d31a
Rework shred headers to fix position of signature (#6451)
* Rework shred headers to fix position of signature

* fix clippy
2019-10-18 22:55:59 -07:00
Tyera Eulberg 785c2574cd Check that transaction fee-payer is a debitable account (#6454)
automerge
2019-10-18 20:39:05 -07:00
sakridge 1a77f7ce3b Change to 0x7f which is a valid short_vec len (#6455)
automerge
2019-10-18 19:56:48 -07:00
Greg Fitzgerald 6e7dccbbfb
Add an error enum to snapshot_utils (#6453) 2019-10-18 19:16:06 -06:00
sakridge 32bfced6a4
Add offset checks for sigverify (#6452)
* Add offset checks for sigverify

* decode_len returning error instead of unwrap
2019-10-18 17:52:59 -07:00
Greg Fitzgerald e81ba8e79f Split snapshot_package module (#6447)
automerge
2019-10-18 14:58:16 -07:00
Greg Fitzgerald a19ffb353d Don't hide serialization errors (#6443)
automerge
2019-10-18 13:35:05 -07:00
dependabot-preview[bot] 171c0d5421 Bump jsonrpc-core from 14.0.0 to 14.0.1 (#6439)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-18 11:17:25 -07:00
dependabot-preview[bot] c01bc4afbd Bump jsonrpc-http-server from 14.0.0 to 14.0.1 (#6437)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-18 11:04:30 -07:00
dependabot-preview[bot] c404008743 Bump jsonrpc-ws-server from 14.0.0 to 14.0.1 (#6438)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-18 11:03:58 -07:00
Ryo Onodera 193c9a08e0 Reject TXs when there is a mismatch (#6236)
automerge
2019-10-18 09:48:35 -07:00
Greg Fitzgerald 5468be2ef9 Add solana-ledger crate (#6415)
automerge
2019-10-18 09:28:51 -07:00
TristanDebrunner 9cf9de6044 Remove the Cursor struct (#6426) 2019-10-18 09:18:36 -06:00
dependabot-preview[bot] 51dad397ed Bump libc from 0.2.64 to 0.2.65
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.64 to 0.2.65.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.64...0.2.65)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-18 08:05:49 -07:00
carllin 27c0d30a07
Fix logging (#6417) 2019-10-18 02:06:41 -07:00
Pankaj Garg 6c33c3a5ba Update shred tests to use specific error codes (#6428)
automerge
2019-10-17 22:50:38 -07:00
Sagar Dhawan 298ba34c3c
Add flag to mark a packet as discarded (#6427) 2019-10-17 16:26:29 -07:00
Greg Fitzgerald 24846b7b61 Don't use BlocktreeError from Shredder (#6423)
automerge
2019-10-17 15:44:15 -07:00
TristanDebrunner 9e52d11ad0
Remove Backend trait (#6407) 2019-10-17 15:19:27 -06:00
Greg Fitzgerald d3068c3918
Remove circular dependencies in core (#6408)
* Remove core::result dependency from blocktree

* Remove core::result dependency from shred

* Move Packet from core::packet to sdk::packet

This way we don't need to split perf_libs yet.

* Disable packet when compiling BPF programs
2019-10-17 11:37:30 -06:00
Greg Fitzgerald a931ad40c8 Remove unused code in entry (#6414)
automerge
2019-10-17 09:59:40 -07:00
Pankaj Garg 83c1831a01
Fix replay stage test (#6406) 2019-10-16 15:41:43 -07:00
carllin ccb4e32ee0
ReplayStage metrics (#6358)
* ReplayStage metrics

* Add more metrics

* Refactor get_slot_entries_with_shred_count() to detect wasted work

* Update dashboard

* Update broadcast slots to micros

* Add broadcast dashboard
2019-10-16 14:32:18 -07:00
Justin Starry 7e6e7e8406
Remove special handling of first ledger tick (#6263)
* Remove special handling of first ledger tick

* Fix subtraction overflow

* @garious feedback

* Back to height

* More tick_height name changes

* Fix off-by-one

* Fix leader tick error

* Fix merge conflict

* Fix recently added test
2019-10-16 15:53:11 -04:00
Ryo Onodera e267dfacdd Stabilize some banking stage tests (#6251)
* Stabilize some banking stage tests

Fixes #5660

* Fix CI...

* clean up

* Fix ci

* Address review nits

* Use bank.max_tick_height due to off-by-one for no PohRecord's clearing bank

* Fix CI...

* Use bank.max_tick_height() instead for clarity
2019-10-16 12:37:27 -07:00
Ryo Onodera f4c5da3c72 Fix unaligned read of short_vec pubkey_size in sigverify (#6388)
automerge
2019-10-16 11:09:17 -07:00
Pankaj Garg a258e1e0b3 Fix flaky test_recv_mmsg_batch_size (#6399)
automerge
2019-10-16 11:01:41 -07:00
dependabot-preview[bot] f6b48b0a67 Bump libc from 0.2.62 to 0.2.64
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.62 to 0.2.64.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.62...0.2.64)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-16 10:03:06 -07:00
Pankaj Garg 33052c1dd2 Cleanup shred header structures (#6378)
automerge
2019-10-15 20:48:45 -07:00
Pankaj Garg 20e800230f
Don't deserialize coding header for data shreds (#6367)
* Don't deserialize coding hdr for data shreds

* review comments

* fix tests
2019-10-15 15:18:23 -07:00
sakridge 1afc527919
Lower cluster_info-num_nodes datapoint (#6368) 2019-10-15 14:42:19 -07:00
sakridge dfca2b510b
Lower shred/receiver stats (#6365)
too many messages
2019-10-15 11:43:52 -07:00
Tyera Eulberg f8ccd90eeb Add ForkConfidenceCache methods (#6359)
automerge
2019-10-14 22:14:20 -07:00
Michael Vines 88bb55ffd2 Add get_vote_accounts() to RPC client 2019-10-14 20:38:51 -07:00
Michael Vines 5508ac6272 Add root slot to getVoteAccounts 2019-10-14 20:38:51 -07:00
dependabot-preview[bot] 2be03ca631 Bump reqwest from 0.9.21 to 0.9.22
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.9.21 to 0.9.22.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/v0.9.22/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.9.21...v0.9.22)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-14 16:33:16 -07:00
Tyera Eulberg 5d99853502 Add getBlockConfidence rpc endpoint (#6350)
automerge
2019-10-14 15:24:10 -07:00
Pankaj Garg 3313b2ff58 Fetch stage batching of forwarded txs (#6349)
automerge
2019-10-14 13:32:29 -07:00
Greg Fitzgerald 322fcea6e5
More fullnode to validator renaming (#6337) 2019-10-11 13:30:52 -06:00
Pankaj Garg 5650231df3
Increase buffer size for erasure meta DB column (#6335) 2019-10-11 12:18:11 -07:00
Justin Starry 78b2e4df9f
Revert "Revert "Bump jsonrpc-core from 13.2.0 to 14.0.0 (#6287)" (#6328)" (#6336)
This reverts commit 578aa439be.
2019-10-11 13:19:13 -04:00
carllin bf9c815b9e
Increase Index column buffers (#6268) 2019-10-10 23:17:39 -07:00
Justin Starry 578aa439be
Revert "Bump jsonrpc-core from 13.2.0 to 14.0.0 (#6287)" (#6328)
This reverts commit c2761a1259.
2019-10-11 00:32:06 -04:00
Pankaj Garg 364781366a
Use sendmmsg for broadcasting shreds (#6325)
* Replace packet with slice of data in sendmmsg

* fixes

* fix bench
2019-10-10 19:38:48 -07:00
Greg Fitzgerald c6e4641781
Remove many uses of legacy term 'fullnode' (#6324) 2019-10-10 17:33:00 -06:00
Pankaj Garg 753bd77b41
Use multicast to send retransmit packets (#6319) 2019-10-10 15:02:36 -07:00
carllin 1960ea8ed7
Add benches for shredding and poh (#6307)
* Add benches for shredding and poh

* ignore poh bench

* Factor out Poh bench as separate function
2019-10-10 14:00:24 -07:00
sakridge 1b775044f7
Use multiple retransmit stage threads/sockets (#6279) 2019-10-10 13:24:03 -07:00
Pankaj Garg 570b98c7bc
Multicast same packet to multiple destinations via sendmmg (#6316)
* Implement multicast same packet to multiple destinations using sendmmsg()
2019-10-10 13:09:15 -07:00
dependabot-preview[bot] c2761a1259 Bump jsonrpc-core from 13.2.0 to 14.0.0 (#6287)
* Bump jsonrpc-core from 13.2.0 to 14.0.0

Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 13.2.0 to 14.0.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.2.0...v14.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump all jsonrpc versions
2019-10-10 10:32:38 -06:00
carllin 1f345ce2d9
Don't grab keypair from cluster info on every iteration of broadccast (#6303) 2019-10-09 17:36:45 -07:00
Pankaj Garg ed85aa43a4
Implement sendmmsg() API (#6304)
* Implement sendmmsg()

* fixes
2019-10-09 17:06:56 -07:00
Sagar Dhawan 4b0250192a Remove remnants of the cuda feature flag (#6298)
automerge
2019-10-09 16:09:36 -07:00
carllin dd66d16fdb
Broadcast final shred for slots that are interrupted (#6269)
* Broadcast final shred for slots that are interrupted
2019-10-09 16:07:18 -07:00
Sagar Dhawan 32312f3c16
Do not retransmit Repair responses (#6284)
* Do not retransmit Repair responses

* Add a test

* Refactor neighboring functionality
2019-10-09 13:11:19 -07:00
Pankaj Garg 2db83e1a21
Remove greedy fetch in shred_fetch stage (#6278)
* Remove greedy fetch in shred_fetch stage

* cleanup
2019-10-09 10:36:05 -07:00
dependabot-preview[bot] cfbfcb5734 Bump dir-diff from 0.3.1 to 0.3.2 (#6265)
Bumps [dir-diff](https://github.com/steveklabnik/dir-diff) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/steveklabnik/dir-diff/releases)
- [Changelog](https://github.com/assert-rs/dir-diff/blob/master/CHANGELOG.md)
- [Commits](https://github.com/steveklabnik/dir-diff/compare/v0.3.1...v0.3.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-09 11:12:15 -06:00
Rob Walker 7cf90766a3
add epoch_schedule sysvar (#6256)
* add epoch_schedule sysvar

* book sheesh!
2019-10-08 22:34:26 -07:00
Justin Starry f2ee01ace3
Fix blocktree processor entry callback test (#6285) 2019-10-08 20:38:05 -04:00
Justin Starry 5e31565574
Expand blocktree processor options (#6248)
* Refactor blocktree processor args and support full leader cache

* Add entry callback option

* Rename num_threads to override_num_threads

* Add test for entry callback

* Refactor cached leader schedule changes

* Add tests for blocktree process options

* Refactor test

* @mvines feedback
2019-10-08 17:58:49 -04:00
Sagar Dhawan 723f9a9b81
Remove unnecessary locking in retransmit stage (#6276)
* Add more detailed metrics to retransmit

* Remove unnecessary locking and add more metrics
2019-10-08 14:41:16 -07:00
sakridge c5e5342325
Rearrange broadcast stats (#6274) 2019-10-08 12:50:59 -07:00
Pankaj Garg 788296047a
Increase batch size for recvmmsg() (#6260)
* Increase batch size for recvmmsg()

* fix tests

* new test
2019-10-08 09:54:49 -07:00
carllin 9dceb8ac74 Broadcast/Shredding Metrics (#6270)
automerge
2019-10-08 01:42:42 -07:00
carllin ac2374e9a1
Shred entries in parallel (#6180)
* Make shredding more parallel

* Fix erasure tests

* Fix replicator test

* Remove UnfinishedSlotInfo
2019-10-08 00:42:51 -07:00
carllin e12c577b16
remove verify_hash_internal_state (#6261) 2019-10-07 16:38:54 -07:00
sakridge ba7efbb136
Retransmit stage optimization, don't copy packets (#6250) 2019-10-07 15:33:22 -07:00
Pankaj Garg 17f169f446
BlobFetchStage cleanup post shred work (#6254) 2019-10-07 11:08:01 -07:00
Michael Vines 4870a2cbac Panic when a snapshot fails to verify 2019-10-07 08:12:18 +09:00
sakridge da7d94d0f0
Retransmit stage benchmark (#6249) 2019-10-06 12:56:17 -07:00
carllin cdef065cca
Broadcast Metrics (#6166)
* Add timing logigng to broadcast

* Shred metrics

* Fixes
2019-10-05 22:51:47 -07:00
sakridge 5ef012b2c1
Tweak debug to remove unreadable datapoints (#6060) 2019-10-04 16:25:22 -07:00
Tyera Eulberg 0c3ff6b75c Cli refactor: vote and storage program functionalities (#6242)
automerge
2019-10-04 14:18:19 -07:00
Michael Vines 7f53737000
Periodically pull from the entrypoint if it's no longer in Crdt (#6240) 2019-10-04 14:18:07 -07:00
Sagar Dhawan 23ea8ae56b
Optimize retransmit stage (#6231)
* Optimize retransmit stage

* Remove comment

* Fix test

* Skip iteration to fixup 0 stakes
2019-10-04 11:52:02 -07:00
dependabot-preview[bot] d759a447be Bump serde_json from 1.0.40 to 1.0.41 (#6226)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.40 to 1.0.41.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.40...v1.0.41)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-03 21:03:05 -06:00
dependabot-preview[bot] 7ac4ce637f Bump reqwest from 0.9.20 to 0.9.21 (#6221)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.9.20 to 0.9.21.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/v0.9.21/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.9.20...v0.9.21)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-02 22:52:48 -07:00
Michael Vines f9f5bc2eb5
More clippy 2019-10-02 21:21:07 -07:00
Michael Vines 9fe8c98047
Switch to solana-reed-solomon-erasure temporarily to fix windows build (#6211) 2019-10-02 19:01:55 -07:00
Pankaj Garg f331f1d1e9
Don't forward transaction to self (#6218) 2019-10-02 14:07:34 -07:00
sakridge 211cae5811
Remove dead constants (#6207) 2019-10-01 18:22:57 -07:00
Pankaj Garg 774e9df2e5
Finish unfininished slot before processing new slots (#6197) 2019-10-01 11:46:14 -07:00
sakridge ae7700296d
broadcast_shreds opt (#6175)
* Don't clone/copy/sort ContactInfo vec
2019-10-01 09:38:29 -07:00
Jack May 520f7c3e18
Optimize BPF logs (#6186) 2019-09-30 14:21:29 -07:00
Parth 2c8c2029d8
cli: enforce rent-exemption balance for stake, vote and program accounts in cli (#6118)
* require minimum balance for stake, vote and program accounts
2019-10-01 01:14:49 +05:30
Rob Walker 4f4618441c
split wallet staking commands (#6168)
* split wallet staking commands

* elide real home

* unit->UNIT for usage

* unit->UNIT, don't try to run SUBCOMMANDS: ;)

* more fixup

* fixups

* actually check

* shellcheck

* preserve #6158 after rebase

* fixup

* test

* too hard

* remove test
2019-09-29 21:18:15 -07:00
Michael Vines e5a7d08966 Add --expected-genesis-blockhash validator argument (#6174)
automerge
2019-09-29 19:09:24 -07:00
Michael Vines d50aef8404 Add get-epoch-info command (#6161)
automerge
2019-09-27 22:00:30 -07:00
Pankaj Garg 5637f88aff Don't try signging coding shred if fec rate is 0 (#6171)
automerge
2019-09-27 21:58:53 -07:00
sakridge 75a92d58cb
Featureize move (#5897)
* Featureize move

* Add move featured test
2019-09-27 12:19:06 -07:00
Tyera Eulberg c9e58743e7
Prevent subtract overflow panic when slot < MAX_LOCKOUT_HISTORY (#6135) 2019-09-26 19:40:18 -06:00
Michael Vines b4da83a3ab
Remove CUDA feature (#6094) 2019-09-26 13:36:51 -07:00
Rob Walker a964570b1a
add authorities to stake init (#6104)
* add authorities to stake init

* fixups

* code review
2019-09-26 13:29:29 -07:00
Rob Walker 50bbe34b66
rename locktower to tower (#6120) 2019-09-26 13:29:05 -07:00
Parth 67d07254c2
Add rent estimation rpc (#6109)
* server side new rpc endpoint

* client side rpc

* take data_len as usize

Co-Authored-By: Tyera Eulberg <teulberg@gmail.com>

* add test and documentation
2019-09-26 23:27:13 +05:30
carllin 701d90a41d
Remove some AccountStorage Serialization (#6047)
* Remove serialization of AccountStorageEntry fields

* Add metric for evaluating BankRc serialization time

* Serialize AppendVec current len

* Add dashboard metrics

* Move flush of AppendVecs to packaging thread
2019-09-25 18:07:41 -07:00
Rob Walker 43795193c4
add authorized parameters to vote api (#6072)
* add authorized parameters to vote api

* code review
2019-09-25 13:53:49 -07:00
dependabot-preview[bot] 62429585ba Bump bincode from 1.1.4 to 1.2.0 (#6065)
Bumps [bincode](https://github.com/servo/bincode) from 1.1.4 to 1.2.0.
- [Release notes](https://github.com/servo/bincode/releases)
- [Commits](https://github.com/servo/bincode/compare/v1.1.4...v1.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-25 14:44:29 -06:00
Sagar Dhawan e987d0094f
Move status cache serialization to the Snapshot Packager service (#6081)
* Move status cache serialization to the Snapshot Packager service

* Minor comment updates

* use ok_or_else instead of ok_or

* satus cache

* Remove assert when snapshot format is wrong

* Fix compile

* Remove slots_to_snapshot from bank forks

* Address review comment

* Remove unused imports
2019-09-25 13:42:19 -07:00
Pankaj Garg de3765ab70 Add erasure recovery stats to dashboard (#6079)
automerge
2019-09-24 19:59:42 -07:00
Michael Vines 5f079137e5 Remove kvstore (#6075)
automerge
2019-09-24 19:59:32 -07:00
Sagar Dhawan 374b776a3e
Fix using temp file for archive (#6058)
* Fix using temp file for archive

* Rename the temp archive instead of hardlinking it
2019-09-24 15:24:54 -07:00
Pankaj Garg 5763d63737 Additional tests for should_retransmit_and_persist (#6062)
automerge
2019-09-24 14:54:10 -07:00
Tyera Eulberg 7babfd00c1
Revert back to reqwest, using rustls feature (#6041)
* Revert back to reqwest, using rustls feature

* Cargo.lock and crate-features

* Ignore test
2019-09-24 14:10:59 -06:00
Pankaj Garg 3ed34b571c
Window service is filtering out coding shreds (#6052)
* Window service is filtering out coding shreds

* update erasure stats to indicate recovery count

* filter out outdated coding shreds

* address review comments
2019-09-24 12:25:25 -07:00
Pankaj Garg 57e90948a8
Remove dead code from cluster_info (#6051) 2019-09-24 10:27:59 -07:00
Rob Walker 26a20a7e62
nits (#6032) 2019-09-24 10:10:49 -07:00
Michael Vines 7fa809c16d
Avoid hardlinking as that confuses tar (#6042) 2019-09-23 20:12:16 -07:00
Sagar Dhawan 84f74807d4 Skip considering banks older than the latest vote slot (#6037)
automerge
2019-09-23 19:40:03 -07:00
Pankaj Garg 3a9c03cc89
Don't recover coding shreds (#6034)
* Don't recover coding shreds

* cleanup
2019-09-23 16:24:21 -07:00
Michael Vines f055d2f0cc
' => " (#6035) 2019-09-23 16:03:38 -07:00
Sagar Dhawan 62c22c6cb1
Fix really old banks triggering log spam (#6025) 2019-09-23 13:59:16 -07:00
Pankaj Garg dbd337c616
Upgrade to ReedSolomon 4.0 release (#6026) 2019-09-23 13:53:52 -07:00
Michael Vines eeda7338cc
Dump tar stdout/err on failure for better debug (#6024) 2019-09-23 13:05:09 -07:00
Dan Albert 9c9991db1d
Update cargo toml and lock files to v0.20.0 (#6016) 2019-09-22 21:45:56 -04:00
carllin 8240d1fe0a
Confidence implementation (#5993)
* Change confidence parameters

* Add status_cache_ancestors to get all relevant ancestors of a bank including roots from status cache

* Fix and add tests

* Clippy
2019-09-20 19:38:56 -07:00
carllin fd6e7020eb
Fix bank overlapping another bank's broadcast (#6012) 2019-09-20 19:37:40 -07:00
dependabot-preview[bot] 22e807c212 Bump jsonrpc-ws-server from 13.1.0 to 13.2.0 (#5976)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 13.1.0 to 13.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.1.0...v13.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-20 15:16:59 -06:00
Michael Vines a60a3efc1a
Revert "require stake, vote and executable accounts to be rent exempt (#5928)" (#6005)
This reverts commit 11e6197a83.
2019-09-20 14:10:39 -07:00
sakridge 19ae556857
hash account state on store (#5573) 2019-09-20 13:21:12 -07:00
Michael Vines 3d44cffcda
Beautify metrics datapoint logging (#5998) 2019-09-20 12:00:43 -07:00
carllin c9be9acd14
log snapshot time (#5996) 2019-09-20 08:03:00 -07:00
dependabot-preview[bot] 8eab673b1c Bump serde from 1.0.100 to 1.0.101 (#5994)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.100 to 1.0.101.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.100...v1.0.101)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-20 07:05:16 -06:00
dependabot-preview[bot] e5806d07a6 Bump jsonrpc-pubsub from 13.1.0 to 13.2.0 (#5995)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 13.1.0 to 13.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.1.0...v13.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-20 07:04:14 -06:00
Parth 11e6197a83
require stake, vote and executable accounts to be rent exempt (#5928)
* require vote account to be exempt

* make stake account rent exempt

* add rent exempted system instruction

* use rent exemption instruction in vote and stake api

* use rent exempted account while creating executable account

* updating chacha golden hash as instruction data has changed

* rent will be initialized for genesis bank too
2019-09-20 16:52:17 +05:30
Michael Vines 1d0be265d9
Add explicit validator-cuda crate (#5985) 2019-09-19 20:50:34 -07:00
Sagar Dhawan d379786c90
Fix bind errors (#5986)
* Add ability to bind to a common tcp/udp port

* Extend port range for local-net sanity and fix validator executable
2019-09-19 17:16:22 -07:00
Pankaj Garg ca9d4e34df
Broadcast stage tuning (#5989) 2019-09-19 16:29:52 -07:00
dependabot-preview[bot] 5157bdd8ce Bump jsonrpc-http-server from 13.1.0 to 13.2.0 (#5975)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 13.1.0 to 13.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.1.0...v13.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-19 13:10:54 -06:00
Rob Walker 5cbd1190b2
transaction batch (#5962)
* transaction batch

* fixup
2019-09-19 10:06:08 -07:00
dependabot-preview[bot] 1a71804ef2 Bump bs58 from 0.2.5 to 0.3.0 (#5974)
Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.2.5 to 0.3.0.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/compare/0.2.5...0.3.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-19 10:39:37 -06:00
dependabot-preview[bot] e83f6332bf Bump serde_derive from 1.0.100 to 1.0.101 (#5945)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.100 to 1.0.101.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.100...v1.0.101)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-18 21:29:40 -07:00
Pankaj Garg 0dbf7995b5
Remove unnecessary serialize of shred data (#5967)
* Remove unnecessary serialize of shred data

* remove obsolete code

* fix golden hash
2019-09-18 20:08:27 -07:00
Sagar Dhawan e0858cfe06
Add parallel shred signing to shredder (#5964) 2019-09-18 18:00:07 -07:00
Pankaj Garg 783e8672e7
Removed Shred enum (#5963)
* Remove shred enum and it's references

* rename ShredInfo to Shred

* clippy
2019-09-18 16:24:30 -07:00
Pankaj Garg 6c4e656795
Remove obsoleted code from shred (#5954)
* Remove obsoleted code from shred

* fix broken test
2019-09-18 13:56:44 -07:00
Sagar Dhawan b757294864
Add minor performance bump to shredding (#5956) 2019-09-18 12:35:52 -07:00
Rob Walker 0d4a2c5eb0
simplify poh recorder => broadcast channel (#5940)
* simplify poh recorder broadcast channel

* fixup

* fixup
2019-09-18 12:16:22 -07:00
dependabot-preview[bot] 31a276b628 Bump jsonrpc-core from 13.1.0 to 13.2.0 (#5894)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 13.1.0 to 13.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.1.0...v13.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-18 11:44:47 -06:00
dependabot-preview[bot] 75c0a268e0 Bump jsonrpc-derive from 13.1.0 to 13.2.0 (#5893)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 13.1.0 to 13.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.1.0...v13.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-18 10:48:37 -06:00
carllin 2f50d0e145
Refactor confidence from replay stage (#5938) 2019-09-17 19:43:40 -07:00
Pankaj Garg ff608992ee
Replace Shred usage with ShredInfo (#5939)
* Replace Shred usage with ShredInfo

* Fix tests

* fix clippy
2019-09-17 18:22:46 -07:00
Rob Walker a2595b44c6
test randomize with error (#5916)
* test randomize with error

* update magic numbers

* fixup

* fixup

* fixup

* no more blobs

* fixup
2019-09-17 15:11:29 -07:00
Sagar Dhawan 14cb6353c0
Change erasure ratio to 0.25 and increase data shreds to 16 (#5931)
* Change erasure ratio to 0.25 and increase data shreds to 16

* Fix case where no coding shreds are requested
2019-09-17 11:59:14 -07:00
Michael Vines e26f68fe62
Get transactions from LockedAccountsResults when possible (#5923) 2019-09-17 08:41:56 -07:00
Michael Vines f10438d530
Respect randomized transaction order when unlocking accounts (#5918) 2019-09-16 21:45:16 -07:00
Pankaj Garg 7459eb15c3
A new data-structure in shreds for partial deserialization (#5915)
* A new datastructure in shreds for partial deserialization

* fix chacha golden hash

* fix clippy and address review comments
2019-09-16 20:28:54 -07:00
Sagar Dhawan c44e7ce184
Leaders should not broadcast to replicators (#5917) 2019-09-16 17:56:34 -07:00
Sagar Dhawan 82615c703b
Switch erasure to solana-reed-solomon-erasure (#5913)
* Switch to solana-reed-solomon-erasure

* Disable Rayon for solana-reed-solomon-erasure
2019-09-16 16:14:55 -07:00
dependabot-preview[bot] bc2141fbe0 Bump ureq from 0.11.0 to 0.11.1 (#5905)
Bumps [ureq](https://github.com/algesten/ureq) from 0.11.0 to 0.11.1.
- [Release notes](https://github.com/algesten/ureq/releases)
- [Commits](https://github.com/algesten/ureq/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-16 16:51:45 -06:00
carllin d5ba90d375
Don't verify blobs that are less than root in window service (#5901) 2019-09-16 13:13:53 -07:00
Pankaj Garg 5f54573613
More shred related cleanup (#5909)
* More shred related cleanup

* fix uncle
2019-09-16 10:28:28 -07:00
Pankaj Garg c1880e3f3e
Reduce number of shreds per FEC block (#5908) 2019-09-15 10:37:12 -07:00
Pankaj Garg ee791e2e3e
Optimizations to shred writing and signing (#5890)
* Optimizations to shred writing and signing

* fix broken tests

* fixes
2019-09-14 21:05:54 -07:00
sakridge ceacc42126
Call gpu init earlier to force compilation. (#5902) 2019-09-14 12:32:57 -07:00
Michael Vines 84c8a5bbec
Add replay-stage-mark_dead_slot datapoint (#5907) 2019-09-14 08:50:53 -07:00
Pankaj Garg 8135279335
Reduce serialize/deserialize in shred recovery (#5887) 2019-09-12 21:52:13 -07:00
Sagar Dhawan c1d788880d
Limit Rayon threadpool threads (#5871) 2019-09-12 11:39:39 -07:00
Pankaj Garg 385086359c
Reduce serializations/deserializations of shreds (#5879) 2019-09-12 10:10:25 -07:00
dependabot-preview[bot] a85604b2ba Bump sys-info from 0.5.7 to 0.5.8 (#5877)
Bumps [sys-info](https://github.com/FillZpp/sys-info-rs) from 0.5.7 to 0.5.8.
- [Release notes](https://github.com/FillZpp/sys-info-rs/releases)
- [Commits](https://github.com/FillZpp/sys-info-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-11 10:53:23 -07:00
Jack May 1853771930
Add support for SDK sysvar types (#5876) 2019-09-10 18:53:02 -07:00
carllin ee4266bc59
Remove banks in locktower not in bank_forks (#5837)
* Remove unnecessary calculations from collect_vote_lockouts

* Add test for locktower startup from snapshot
2019-09-10 13:58:27 -07:00
dependabot-preview[bot] 294d531e0b Bump serde_derive from 1.0.99 to 1.0.100 (#5864)
automerge
2019-09-10 13:31:11 -07:00
dependabot-preview[bot] 0d7efe5176 Bump serde from 1.0.99 to 1.0.100 (#5862)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.99 to 1.0.100.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.99...v1.0.100)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-10 12:27:41 -07:00
Pankaj Garg b426dfb2c0
Change tx batching in banking process and record (#5832)
* Change tx batching in banking process and record

* Change batching to reduce impact on replay stage
2019-09-10 11:04:03 -07:00
Pankaj Garg fd33b27af1
Fix coding shred generator (#5865) 2019-09-10 09:35:07 -07:00
Pankaj Garg 7682db4826
Generate coding shreds on the fly based on erasure limits (#5852)
* Generate coding shreds on the fly based on erasure limits

* fix uncle
2019-09-09 17:26:51 -07:00
carllin 7607800d47
Refactor restart function in local cluster to support separate exit and restart functions (#5845) 2019-09-08 17:53:34 -07:00
Michael Vines df205f8752
Use ureq instead of influx_db_client (#5839) 2019-09-07 12:48:45 -07:00
Jack May e8d88f3237
Split SDK's timing.rs (#5823) 2019-09-06 14:30:56 -07:00
dependabot-preview[bot] f28782cb84 Bump chrono from 0.4.8 to 0.4.9 (#5775)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.8 to 0.4.9.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/master/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/commits/v0.4.9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-06 13:55:36 -06:00
Michael Vines 1f9fde5f7b
ThinClient internal name grooming (#5800) 2019-09-06 09:07:40 -07:00
Sagar Dhawan a452249bf3
Use retain on Packets instead of creating new ones (#5804)
* Use remove on Packets instead of creating a new one

* Fix compile after rebase
2019-09-05 19:16:18 -07:00
Pankaj Garg 3d3b03a123
Verify signature of recovered shred before adding them to blocktree (#5811)
* Verify signature of recovered shred before adding them to blocktree

* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
Pankaj Garg 05d2eec45c
Remove unnecessary erasure config references (#5809) 2019-09-05 14:46:41 -07:00
Pankaj Garg 3237e897d7
Adjust packet batching post-decoupling from blobs (#5783) 2019-09-05 11:22:39 -07:00
carllin bd74e63702
Offload remaining confidence cache computation to separate thread (#5792)
* Move remaining confidence cache computation to separate thread

* Move confidence cache out of bank forks
2019-09-04 23:10:25 -07:00
Pankaj Garg f78b865cba
Cleanup shreds to remove FirstShred data structure (#5789)
* Cleanup shreds to remove FirstShred data structure

* Also reduce size used by parent slot information in shred header

* clippy

* fixes

* fix chacha test
2019-09-04 21:06:47 -07:00
carllin 7062fe4b47
Refactor Blocktree for clarity and correctness (#5700)
* Refactor shreds to prevent insertion of any metadata on bad shreds

* Refactor fetching Index in blocktree

* Refactor get_slot_meta_entry

* Re-enable local cluster test

* cleanup

* Add tests for success/fail insertion of coding/data shreds

* Remove assert

* Fix and add tests for should_insert coding and data blobs
2019-09-04 17:14:42 -07:00
Pankaj Garg 02ee2a601c
Further cleanup of blocktree after Blob deprecation (#5780) 2019-09-04 12:47:09 -07:00
Tyera Eulberg b19d9a50d3
Transition to ureq http client (#5777)
* Transition to ureq http client

* Remove unwrap
2019-09-04 12:11:44 -07:00
carllin dfa6238342
Remove unnecessary construction of descendants (#5742) 2019-09-04 01:49:42 -07:00
Pankaj Garg 3b0d48e3b8
Remove blocktree blob references (#5691)
* Remove blocktree blob references

* fixes and cleanup

* replace uninitialized() call with MaybeUninit

* fix bench
2019-09-03 21:32:51 -07:00
Sagar Dhawan 62f6a78ccd
Make data plane shred filter parallel again (#5740) 2019-09-03 21:50:57 +00:00
dependabot-preview[bot] f7e039e7ac Bump chrono from 0.4.7 to 0.4.8 (#5761)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.7 to 0.4.8.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/master/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-03 14:49:48 -07:00
dependabot-preview[bot] 61bd14c40a Bump rayon from 1.1.0 to 1.2.0 (#5758)
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.1.0...v1.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-03 13:39:58 -06:00
Tyera Eulberg 85e5fbeb35
Add absoluteSlot to getEpochInfo (#5765) 2019-09-02 12:21:06 -07:00
anatoly yakovenko 475f6fe666
votes only need slots and the last bank hash (#5499)
churn

cleanup

reverse test slot hashes

test check_slots_are_valid

updates

only send the minimum bank vote difference

fixup! only send the minimum bank vote difference

some banks may not have a voting account setup

fixup! votes only need slots and the last bank hash

fixup! fixup! votes only need slots and the last bank hash

fmt

fixed compare

fixed vote

fixup! fixed vote

poke ci

filter the local votes via the last bank vote
2019-09-02 12:01:09 -07:00
sakridge 0c2a49391a
Disable pinned gpu memory (#5753) 2019-08-31 16:44:07 -07:00
Michael Vines e3a6c9234a Entrypoint RPC service discovery now blocks until the entrypoint is actually found (#5756)
automerge
2019-08-30 16:12:58 -07:00
Michael Vines 643d0b0868 Make the world flat again; remove utils/ subdirectory (#5752)
automerge
2019-08-30 11:57:39 -07:00
dependabot-preview[bot] f840eefcbf Bump bs58 from 0.2.4 to 0.2.5 (#5747)
Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.2.4 to 0.2.5.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/compare/0.2.4...0.2.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-30 11:41:41 -06:00
dependabot-preview[bot] e1f3e33bfb Bump jsonrpc-pubsub from 13.0.0 to 13.1.0 (#5708)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-30 11:40:58 -06:00
Michael Vines 4786143524 Add a more helpful error on genesis block mismatch (#5744)
automerge
2019-08-30 09:10:22 -07:00
dependabot-preview[bot] c3f07eb85a Bump jsonrpc-ws-server from 13.0.0 to 13.1.0 (#5721)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-29 09:12:02 -06:00
Pankaj Garg 8adac30c05
Integrate shreds to the replicators (#5711)
* Integrate shreds to the replicators

* fix cuda stuff

* fix cuda tests
2019-08-28 22:34:47 -07:00
dependabot-preview[bot] 156399e8aa Bump jsonrpc-http-server from 13.0.0 to 13.1.0 (#5707)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-28 14:21:17 -06:00
dependabot-preview[bot] 5745a54d4c Bump indexmap from 1.0.2 to 1.1.0 (#5706)
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.0.2 to 1.1.0.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.0.2...v1.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-28 14:20:45 -06:00
Parth 7dfb735db9
randomize tx ordering (#4978)
Summary of Changes:
This change adds functionality to randomize tx execution for every entry. It does this by implementing OrderedIterator that iterates tx slice as per the order specified. The order is generated randomly for every entry.
2019-08-28 21:08:32 +05:30
Pankaj Garg 2510f3d352
Remove extra call to serialize in shred verify (#5698) 2019-08-27 19:28:00 -07:00
Pankaj Garg 47535b9ff1 Use serialize_into to fill in shreds instead of writing one byte at a time (#5695)
automerge
2019-08-27 17:11:24 -07:00
Sunny Gleason 34ab25a88b
feat: getInflation() endpoint (#5681) 2019-08-27 18:17:03 -04:00
carllin 8b9c3a2561
Blocktree last_root to enforce a slot floor (#5593)
* Add last_root to blocktree

* Don't repair earlier than last_root

* Add integration test to make sure blocktree floor is enforced
2019-08-27 15:09:41 -07:00
Michael Vines 0c62cf8980
Add metrics for snapshot generation (#5677) 2019-08-27 13:04:20 -07:00
Pankaj Garg f1d58f980b Ignore retransmit channel error (#5680)
automerge
2019-08-27 12:41:04 -07:00
Pankaj Garg 12ad95eb5e
Erasure statistics for shreds (#5676) 2019-08-27 11:22:06 -07:00
dependabot-preview[bot] 85f03b590d Bump jsonrpc-derive from 13.0.0 to 13.1.0 (#5668)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-27 08:48:34 -07:00
Michael Vines a29f0484dc
Add newline before cluster info log (#5671) 2019-08-27 08:33:48 -07:00
dependabot-preview[bot] 8e6e72babd Bump jsonrpc-core from 13.0.0 to 13.1.0 (#5669)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-27 07:49:16 -07:00
dependabot-preview[bot] 7257d2845d Bump hex-literal from 0.2.0 to 0.2.1 (#5638)
Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.2.0 to 0.2.1.
- [Release notes](https://github.com/RustCrypto/utils/releases)
- [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.2.0...hex-literal-v0.2.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-26 21:32:42 -07:00
Michael Vines db768b4c3a
Log contact info every 10 seconds (#5663) 2019-08-26 18:31:14 -07:00
Pankaj Garg 4ac1213c9c
Integrate coding shreds and recovery (#5625)
* Integrate coding shreds and recovery

* More tests for shreds and some fixes

* address review comments

* fixes to code shred generation

* unignore tests

* fixes to recovery
2019-08-26 18:27:45 -07:00
Michael Vines a0f3208828 Ignore flaky test_banking_stage_entryfication (#5659)
automerge
2019-08-26 16:49:34 -07:00
carllin 28f2c75137 Add bigger buffers for shred column families in rocks (#5653)
automerge
2019-08-26 15:58:26 -07:00
Rob Walker 615da845cd
remove replicode in run_purge_batch() (#5630)
* remove replicode

* bigger timeout

* backout 90 min timeout
2019-08-26 09:47:48 -07:00
Michael Vines ba59741b60
Bump to 0.19.0-pre0 2019-08-25 21:47:29 -07:00
Michael Vines 52da207f83 test_snapshots_restart_validity now passes (#5644)
automerge
2019-08-25 21:33:41 -07:00
carllin 6e82978931 Fix race with LedgerCleanupService (#5622) 2019-08-23 23:40:20 -07:00
Michael Vines 4e827af392
Remove unnecessary trailing semicolons (#5636) 2019-08-23 22:47:54 -07:00
Rob Walker 0ffe7a9c8f
plumb some rent (#5610)
* plumb some rent

* nits

* fixups

* fixups

* fixups
2019-08-23 14:04:53 -07:00
Michael Vines 9b8d59d2e9
Revert "Bump indexmap from 1.0.2 to 1.1.0 (#5565)" (#5624)
This reverts commit f1ad69c84e.
2019-08-23 13:20:31 -07:00
Michael Vines 3fc5009ef2
Snapshot pipefitting through the validator cli (#5617)
* Handle 404 errors better

* Snapshot pipefitting through the validator cli

* Add download progress bar

* Log the current entrypoint slot
2019-08-23 13:02:07 -07:00
dependabot-preview[bot] f1ad69c84e Bump indexmap from 1.0.2 to 1.1.0 (#5565)
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.0.2 to 1.1.0.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.0.2...v1.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-23 11:06:24 -07:00
Rob Walker 52f6da5cee
upgrade rust to 1.37 (#5611) 2019-08-23 08:55:51 -07:00
Pankaj Garg aeaa0feb61
Add range lookups for erasure set indexes (#5612) 2019-08-22 16:32:38 -07:00
dependabot-preview[bot] 4125d01668 Bump reqwest from 0.9.19 to 0.9.20 (#5598)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.9.19 to 0.9.20.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/v0.9.20/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.9.19...v0.9.20)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-22 07:57:33 -07:00
carllin 087c43b9ef
Add snapshotting integration test (#5519)
* Add snapshotting integration test

* Update ContactInfo on restart in local cluster nodes
2019-08-21 23:59:11 -07:00
Pankaj Garg c18ea3ccc9
Fix ignored tests in blocktree (#5591) 2019-08-21 20:07:51 -07:00
Michael Vines e2d6f01ad3
solana-validator now verifies its genesis blockhash against the cluster entrypoint (#5589) 2019-08-21 18:16:40 -07:00
Michael Vines faafee6b42
to to/the the (#5590) 2019-08-21 17:46:59 -07:00
carllin 80f618f011 Add info logging around snapshot tarball generation (#5592)
automerge
2019-08-21 16:36:21 -07:00
Pankaj Garg 0dc0594aaa
Fixes to repair and orphan logic for data shreds (#5587) 2019-08-21 15:27:42 -07:00
Michael Vines d651cb7a25
Adjust |ulimit -n| automatically, no bash required (#5586) 2019-08-21 14:55:58 -07:00
dependabot-preview[bot] ab4f370e15 Bump serde_derive from 1.0.98 to 1.0.99 (#5539)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.98 to 1.0.99.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.98...v1.0.99)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-21 12:14:58 -07:00
dependabot-preview[bot] d6f824abc0 Bump bs58 from 0.2.2 to 0.2.4 (#5560)
Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.2.2 to 0.2.4.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/compare/0.2.2...0.2.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-21 12:13:52 -07:00
Michael Vines 3450b9a44d
Rename solana to solana-core (#5583) 2019-08-21 10:23:33 -07:00
Michael Vines afaf95cf53
Refine error message when ledger can't be opened (#5582) 2019-08-21 09:44:12 -07:00
carllin bb558acdf0
Change JsonRpc exit to use wait->close (#5566)
* Add wait-close-join pattern to rpc_service

* Create ValidatorExit struct
2019-08-20 23:59:31 -07:00
Pankaj Garg 4798e7fa73
Integrate data shreds (#5541)
* Insert data shreds in blocktree and database

* Integrate data shreds with rest of the code base

* address review comments, and some clippy fixes

* Fixes to some tests

* more test fixes

* ignore some local cluster tests

* ignore replicator local cluster tests
2019-08-20 17:16:06 -07:00
Michael Vines 8e0f41a790
Cargo.lock 2019-08-20 16:59:44 -07:00
Michael Vines b1203da82c
Bump to 0.18.0-pre2 2019-08-20 16:56:00 -07:00
Rob Walker 93ae98812b
change DEFAULT_NUM_TICKS_PER_SECOND to DEFAULT_TICKS_PER_SECOND (#5559) 2019-08-19 23:22:56 -07:00
Sagar Dhawan bd193535c9
Cap CrdsFilter sizes such that PullRequest no longer exceeds MTU (#5561) 2019-08-19 18:14:10 -07:00
dependabot-preview[bot] 8b7cca986a Bump serde from 1.0.98 to 1.0.99 (#5540)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.98 to 1.0.99.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.98...v1.0.99)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-19 10:06:15 -06:00
Rob Walker 814af378a7
stake cooldown (#5553)
* stake cooldown

* fixups

* sheesh
2019-08-17 18:12:30 -07:00
Rob Walker a252acf539
move netutil (#5552) 2019-08-17 15:52:12 -07:00
Rob Walker 01eb7600d9
use stake config to defeat warmup in local_cluster (#5549)
* use stake config to defeat warmup in local_cluster

* fixups
2019-08-17 12:28:36 -07:00
Michael Vines ccee6241a6
Revert "publish fixes"
This reverts commit 4d13d3871d.
2019-08-16 17:28:07 -07:00
Michael Vines 4d13d3871d
publish fixes 2019-08-16 17:03:57 -07:00
Michael Vines bb0c9d6145
Log more info at the start of PoH (#5550) 2019-08-16 16:20:20 -07:00
Tyera Eulberg 8d105042ea
Update getEpochVoteAccounts to getVoteAccounts (#5543)
* Rework getEpochVoteAccounts into getVoteAccounts

* Update client apis

* Update docs

* Review comments
2019-08-16 17:02:19 -06:00
Michael Vines 84304cb0fc
Display vote pubkey at startup (#5548) 2019-08-16 15:56:06 -07:00
Rob Walker 99eeb63f71
move the rest of cluster to local_cluster (#5535) 2019-08-16 00:00:38 -07:00
Sagar Dhawan e73cbdda61 Reduce log level for known issue (#5536)
automerge
2019-08-15 19:42:27 -07:00
Rob Walker 94f1132fb6
fix single node testnet, remove bootstrap vote (#5534) 2019-08-15 18:58:46 -07:00
Sagar Dhawan 4ee212ae4c
Coalesce gossip pull requests and serve them in batches (#5501)
* Coalesce gossip pull requests and serve them in batches

* batch all filters and immediately respond to messages in gossip

* Fix tests

* make download_from_replicator perform a greedy recv
2019-08-15 17:04:45 -07:00
carllin d5fb493aa4
Change recv to try_recv (#5533) 2019-08-15 15:17:46 -07:00
Greg Fitzgerald 471bc73a23
Fix Rust 1.37.0 compiler warnings (#5530)
Looks like most usages of trait objects should have introduced
a type variable instead.
2019-08-15 14:00:09 -06:00
dependabot-preview[bot] a948c9b7f9 Bump libc from 0.2.61 to 0.2.62 (#5527)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.61 to 0.2.62.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.61...0.2.62)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-15 09:49:14 -06:00
carllin 4688f9821f
Snapshot optimizations (#5525)
* Change serializing snapshot tar to use shell command
2019-08-14 23:14:40 -07:00
Michael Vines 9f6c9c428b
Move genesis/snapshot archive download into Rust (#5515) 2019-08-14 19:25:22 -07:00
Michael Vines b4f0f4abcc
Disable rocksdb bzip2 compression 2019-08-14 15:39:30 -07:00
Rob Walker d22848f9b1
use live stakes for consensus (#5426)
* use live stakes for consensus

* lint

* re-enable leader_failure_4

* fixups

* re-ignore leader_failure_4
2019-08-14 13:30:21 -07:00
carllin d791c70d90
Snapshot optimizations (#5518)
* Limit slots_since_snapshot size, only package latest snapshot, refactor tests

* Add test checking status_cache.roots == bank_forks.slots_since_snapshot after bank_forks.set_root()
2019-08-13 22:39:29 -07:00
Sagar Dhawan 1d0608200c
Restore blob size fix (#5516)
* Revert "Revert "Fix gossip messages growing beyond blob size  (#5460)" (#5512)"

This reverts commit 97d57d168b.

* Fix Crds filters
2019-08-13 18:04:14 -07:00
carllin cd14a940d8
Allow process_blocktree() to start processing from any root (#5484)
* Remove unnecessary entry_height from BankInfo

* Refactor process_blocktree to support process_blocktree_from_root

* Refactor to process blocktree after loading from snapshot

* On restart make sure bank_forks contains all the banks between the root and the tip of each fork, not just the head of each fork

* Account for 1 tick_per_slot in bank 0 so that blockhash of bank0 matches the tick
2019-08-13 17:20:14 -07:00
carllin 58d4e32c97
Remove serialization of future AppendVecs and serialize AccountStorage correctly (#5510) 2019-08-13 16:05:37 -07:00
Michael Vines 08f6a2ea3e
debash: Add `solana-gossip get-rpc-url` command to avoid hard coding (#5513) 2019-08-13 10:49:48 -07:00
Sagar Dhawan 97d57d168b Revert "Fix gossip messages growing beyond blob size (#5460)" (#5512)
This reverts commit a8eb0409b7.
2019-08-13 10:29:26 -07:00
Rob Walker 07d11be6ab
add global stake warmup (#5483)
* add global stake warmup

* integrate stake history into runtime

* fixup core tests

* fixup

* remove existing cooldown tests for now
2019-08-12 20:59:57 -07:00
Greg Fitzgerald a43922ccbf
Boot hashbrown (#5505)
As of Rust 1.36.0, hashbrown now implements the HashMap in std (which
implements HashSet).

https://blog.rust-lang.org/2019/07/04/Rust-1.36.0.html#a-new-hashmapk,-v%3E-implementation
2019-08-12 16:46:49 -06:00
Pankaj Garg b7a5136136
Helper functions for shreds (#5493) 2019-08-12 15:27:58 -07:00
Justin Starry 0fde19239b
Rate limit counter metrics points to one per second (#5496)
* Rate limit counter metrics points to one per second

* Remove old env var

* Test that metrics counter is incrementing

* Fix typo
2019-08-12 18:15:34 -04:00
dependabot-preview[bot] 771d1a78fd Bump libc from 0.2.60 to 0.2.61 (#5491)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.60 to 0.2.61.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.60...0.2.61)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-12 15:39:26 -06:00
Sagar Dhawan a8eb0409b7
Fix gossip messages growing beyond blob size (#5460)
* fixed bloom filter math

* Add split each pull request into multiple pulls with different filters

* Rework CrdsFilter to generate all possible masks to cover the keyspace

* Limit the bloom sizes such that each pull request is no larger than mtu
2019-08-12 13:51:29 -07:00
carllin 1b84092b94
Fix slots_since_snapshot in BankForks.add_root() (#5489) 2019-08-12 11:56:03 -07:00
Pankaj Garg b1d43ace14
Add columns for data and code shreds (#5461) 2019-08-12 10:03:57 -07:00
Michael Vines 54f4d13350
Validator log filter may now be reconfigured at runtime (#5473)
* Log filter may now be reconfigured at runtime

* Add RPC API and bash script to reconfigure the log filter
2019-08-10 22:54:46 -07:00
dependabot-preview[bot] 799d3b1575 Bump nix from 0.14.1 to 0.15.0 (#5488)
Bumps [nix](https://github.com/nix-rust/nix) from 0.14.1 to 0.15.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-10 18:48:36 -06:00
Rob Walker 5884469d11
count commitable in banking_stage (#5477) 2019-08-09 21:14:20 -07:00
Rob Walker ed093f86f9
harmonize percentage members (#5459)
* harmonize percentage members

* update tests

* update capitalization when burning fees

* verify capitalization in fee burn

* fixup
2019-08-09 13:58:46 -07:00
Michael Vines 5b4ee36cfd
Log more socket addresses at validator startup (#5471) 2019-08-08 15:38:23 -07:00
Justin Starry 4f86c0b74a
Rate limit transaction counters (#5447)
* Rate limit transaction counters

* @sakridge feedback

* Set default high metrics rate for multinode demo

* Fix tests

* Swap defaults and fix env var tests

* Only set metrics rate if not already set
2019-08-08 17:05:06 -04:00
sakridge 238aa2133d
Move local_cluster tests into own crate (#5465) 2019-08-08 11:04:33 -07:00
Michael Vines 8c15214923
Add --dev-halt-at-slot option (#5453) 2019-08-08 09:14:30 -07:00
Tyera Eulberg 5b51bb27b6
Rpc to return software version (#5456)
* Add getSoftwareVersion rpc

* Add getSoftwareVersion to doc

* Rename to getVersion and return object

* Update jsonrpc-api.md
2019-08-07 20:06:27 -06:00
Pankaj Garg 6597c71e23
Implement shred erasure recovery and reassembly (#5444)
* Implement shred erasure recovery and reassembly

* fixes and unit test

* clippy

* review comments, additional tests, and some fixes

* address review comments

* more tests and cleanup
2019-08-07 17:02:49 -07:00
carllin 8aa7a851ca
Fix hardlinking across filesystem boundaries (#5449)
* Fix hardlinking across filesystem boundaries

* create output dir for snapshot tar
2019-08-07 13:12:53 -07:00
dependabot-preview[bot] be9fa22db7 Bump hashbrown from 0.3.1 to 0.5.0 (#5450)
Bumps [hashbrown](https://github.com/rust-lang/hashbrown) from 0.3.1 to 0.5.0.
- [Release notes](https://github.com/rust-lang/hashbrown/releases)
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.3.1...v0.5.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-07 08:35:54 -06:00
carllin b72c5689c9
Blow away snapshots directory on start (#5446) 2019-08-06 21:41:38 -07:00
Sagar Dhawan 9dcf3347f5 Refactor status cache and remove complex serialize/deserialize (#5335)
automerge
2019-08-06 18:47:30 -07:00
dependabot-preview[bot] 3ae9357a36 Bump hashbrown from 0.2.2 to 0.3.1 (#5381)
Bumps [hashbrown](https://github.com/rust-lang/hashbrown) from 0.2.2 to 0.3.1.
- [Release notes](https://github.com/rust-lang/hashbrown/releases)
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/hashbrown/compare/v0.2.2...v0.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-06 10:16:18 -06:00
carllin 1dbb5c8647
Deserialize snapshots (#5417)
* Deserialize snapshots
2019-08-05 22:53:19 -07:00
Pankaj Garg c524d62ce0
Implement coding shred generation (#5415)
* Implemenet coding shred generation

* address review comments
2019-08-05 16:32:34 -07:00
Michael Vines dd4640e1ed
Revert "Revert "Bump version to 0.18.0-pre1""
This reverts commit 42c7d57fc0.
2019-08-05 15:55:13 -07:00
Michael Vines 42c7d57fc0
Revert "Bump version to 0.18.0-pre1"
This reverts commit 14f6d5c82b.
2019-08-05 15:53:55 -07:00
Tyera Eulberg efd09ecd37 Revert fork metrics (#5427)
* Revert "Remove duplicate row (#5419)"

This reverts commit a81dd80d60.

* Revert "Log fork stake-percentage in metrics, and display (#5406)"

This reverts commit 92e419f1c7.
2019-08-05 15:53:36 -07:00
Michael Vines 14f6d5c82b
Bump version to 0.18.0-pre1 2019-08-05 15:11:44 -07:00
Tyera Eulberg 92e419f1c7
Log fork stake-percentage in metrics, and display (#5406)
* Log fork stake percentage data

* Add fork stake percentage to dashboard

* Call out parent slot
2019-08-02 19:16:23 -06:00
Pankaj Garg 80bb0158b7
Initial implementation of packet shredder (#5401)
* Initial implementation of packet shredder

* tests

* clippy

* review comments
2019-08-02 15:53:42 -07:00
Michael Vines eb3991b9ba
Replay stage log message nits (#5408) 2019-08-02 10:08:42 -07:00
Michael Vines aee63f15c2
Rename state.tgz to snapshot.tgz to match rpc service 2019-08-02 10:07:29 -07:00
dependabot-preview[bot] f03e066ec5 Bump log from 0.4.7 to 0.4.8 (#5382)
Bumps [log](https://github.com/rust-lang/log) from 0.4.7 to 0.4.8.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang-nursery/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-01 14:31:18 -07:00
Rob Walker f7d3f55566
fix epoch_stakes again (#5396) 2019-08-01 14:27:47 -07:00
carllin 6cb2040a1b
Snapshot Packaging Service (#5262)
* Snapshot serialization and packaging
2019-07-31 17:58:10 -07:00
sakridge c2fc0f2418
Plumb libra accounts to genesis (#5333)
* Plumb move_loader to genesis

* Remove core dependency on genesis-programs
2019-07-31 16:10:55 -07:00
Rob Walker 9278201198
fix epoch_stakes (#5355)
* fix epoch_stakes

* fix stake_state to use stakers_epoch

* don't allow withdrawal before deactivation
2019-07-31 15:13:26 -07:00
Jack May d09afdbefe
Synchronize and cleanup instruction processor lists (#5356) 2019-07-31 14:28:14 -07:00
dependabot-preview[bot] 01d2b4e952 Bump jsonrpc-http-server from 12.1.0 to 13.0.0 (#5361)
* Bump jsonrpc-http-server from 12.1.0 to 13.0.0

Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 12.1.0 to 13.0.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v12.1.0...v13.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Update all jsonrpc crates to v13.0.0
2019-07-31 14:30:47 -06:00
Jack May 6d7cb23c61
Add command to create genesis accounts (#5343) 2019-07-30 23:43:12 -07:00
Michael Vines bd7e269280 Kill rsync (#5336)
automerge
2019-07-30 22:43:47 -07:00
Pankaj Garg b05b42d74d
Reduce max blob size (#5345)
* Reduce max blob size

* ignore test_star_network_push_rstar_200
2019-07-30 22:15:07 -07:00
dependabot-preview[bot] af733a678a Bump serde_derive from 1.0.97 to 1.0.98 (#5314)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.97 to 1.0.98.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.97...v1.0.98)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-30 21:45:34 -07:00
Michael Vines 4a336eb5ff
ValidatorConfig path reform: use Path/PathBuf for paths (#5353) 2019-07-30 19:47:24 -07:00
dependabot-preview[bot] b7e08052ae Bump serde from 1.0.97 to 1.0.98 (#5315)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.97 to 1.0.98.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.97...v1.0.98)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-30 19:46:50 -07:00
Michael Vines c78db6a94b
ledger path reform: use Path/PathBuf instead of strings (#5344) 2019-07-30 15:53:41 -07:00
carllin 7dc5cc26a6
Make max_epoch check in next_leader_at in leader schedule (#5342) 2019-07-30 15:51:02 -07:00
Sagar Dhawan d7a2b790dc
Limit the size of gossip push and gossip pull response (#5348)
* Limit the size of gossip push and gossip pull response

* Remove Default::default

* Rename var
2019-07-30 15:43:17 -07:00
Pankaj Garg a7a10e12c7
Forward transactions as packets instead of blobs (#5334)
* Forward transactions as packets instead of blobs

* clippy
2019-07-30 14:50:02 -07:00
sakridge 8d243221f0
Ignore flaky local cluster tests (#5347)
* Add logging to local_cluster tests

* Ignore flaky test_leader_failure_4, test_repairman_catchup

And crashing banking benchmarks.
2019-07-30 13:48:46 -07:00
Justin Starry 84368697af
Fix metrics when leader does not report metrics (#5291) 2019-07-30 16:18:33 -04:00
sakridge a642168369
Add move to bench-tps (#5250) 2019-07-27 15:28:00 -07:00
Dan Albert b8835312bb
Update cargo.toml files to 0.18.0-pre0 (#5303) 2019-07-27 11:42:06 -06:00
Pankaj Garg aef7bae60d
Let grace ticks to roll over into multiple leader slots (#5268)
* Let grace ticks to roll over into multiple leader slots

* address review comments
2019-07-26 11:33:51 -07:00
Tyera Eulberg dab7de7496
Add confidence cache to BankForks (#5066)
* Add confidence cache to BankForks

* Include stake-weighted lockouts in cache

* Add cache test

* Move confidence cache updates to handle_votable_bank

* Prune confidence cache on prune_non_root()

* Spin thread to process aggregate_stake_lockouts

* Add long-running thread for stake_weighted_lockouts computation
2019-07-26 11:27:57 -06:00
dependabot-preview[bot] 349ebec629 Bump serde from 1.0.94 to 1.0.97 (#5285)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.94 to 1.0.97.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.94...v1.0.97)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-26 07:22:25 -06:00
Sagar Dhawan a233a1c822
Fix poh recorder not flushing virtual ticks immediately (#5277)
* Fix poh not flushing virtual ticks immediately

* Fix test_would_be_leader_soon
2019-07-25 11:08:44 -07:00
dependabot-preview[bot] 7ee8383e02 Bump serde_derive from 1.0.94 to 1.0.97 (#5279)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.94 to 1.0.97.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.94...v1.0.97)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-25 06:47:24 -06:00
Sagar Dhawan 535df0026d
Fixes for Blocktree space amplification and slot deletion (#5266)
* Fixes for Blocktree space amplification and slot deletion
2019-07-24 17:28:08 -07:00
Sagar Dhawan 937816e67b Post warning if window service isn't receiving any data (#5269)
automerge
2019-07-24 12:46:10 -07:00
dependabot-preview[bot] 610a02c518 Bump jsonrpc-ws-server from 12.0.0 to 12.1.0 (#5261)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 12.0.0 to 12.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v12.0.0...v12.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-24 10:52:44 -06:00
Justin Starry 547a7a345f
Add logs to indicate when the leader changes (#5253) 2019-07-23 22:19:20 -04:00
Sagar Dhawan 81f4fd56c7 Log the repairee pubkey when unable to serve them (#5222)
automerge
2019-07-22 14:13:29 -07:00
dependabot-preview[bot] a6302acfd5 Bump jsonrpc-http-server from 12.0.0 to 12.1.0 (#5211)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 12.0.0 to 12.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v12.0.0...v12.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-22 10:34:27 -06:00
Michael Vines 0d8f5379a0
Add time units 2019-07-22 08:22:21 -07:00
Sagar Dhawan a07b17b9b5
Drop older slots in the ledger (#5188)
* Add facility to delete blocktree columns in range

* Add ledger cleanup service

* Add local_cluster test
2019-07-20 13:13:55 -07:00
dependabot-preview[bot] 0958905df8 Bump reqwest from 0.9.18 to 0.9.19 (#5201)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.9.18 to 0.9.19.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.9.18...v0.9.19)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-19 16:13:34 -06:00
Tyera Eulberg 3f54c0f1a6
Update struct order arbitrarily to match rpc output (#5197) 2019-07-19 11:45:04 -06:00
dependabot-preview[bot] 4684faa5e8 Bump jsonrpc-core from 12.0.0 to 12.1.0 (#5143)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 12.0.0 to 12.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v12.0.0...v12.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-19 09:59:56 -07:00
sakridge 8b69998379
Lower recovery messages (#5181) 2019-07-19 09:20:14 -07:00
dependabot-preview[bot] 19e131d710 Bump jsonrpc-derive from 12.0.0 to 12.1.0 (#5193)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 12.0.0 to 12.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v12.0.0...v12.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-19 07:34:48 -07:00
dependabot-preview[bot] 9fd34cd985 Bump untrusted from 0.6.2 to 0.7.0 (#5194)
Bumps [untrusted](https://github.com/briansmith/untrusted) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/briansmith/untrusted/releases)
- [Commits](https://github.com/briansmith/untrusted/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-19 07:34:04 -07:00
Michael Vines adfb8ff2a1
Add getEpochInfo() and getLeaderSchedule() RPC methods (#5189)
* Add getLeaderSchedule() RPC method

* Add getEpochInfo() RPC method

* Add JSON RPC docs
2019-07-19 07:31:18 -07:00
Rob Walker afa05acb32
more replay_stage grooming (#5163) 2019-07-18 14:54:27 -07:00
Sagar Dhawan a3a91ba222 Fix misleading variable name (#5176)
automerge
2019-07-18 14:07:32 -07:00
Sagar Dhawan 751b54b60b
Skip sleeping in replay stage if a bank was recently processed (#5161)
* Skip sleeping in replay stage if a bank was recently processed

* Remove return
2019-07-18 12:04:53 -07:00
Sagar Dhawan 6ad9dc18d8
Add ability to prune ledger (#5128)
* Add utility to prune the ledger

* Add tests

* Fix clippy

* Fix off by one

* Rework to force delete every column

* Minor fixup
2019-07-17 14:42:29 -07:00
Rob Walker 027ebb6670
no more OUT_DIR (#5139)
* no more OUT_DIR

* no more OUT_DIR

* more information about failure
2019-07-17 14:27:58 -07:00
Rob Walker 0ffd91df27
groom poh_recorder (#5127)
* groom poh_recorder

* fixup

* nits

* slot() from the outside means "the slot the recorder is working on"

* remove redundant check

* review comments, put next_tick back in the "is reset" check

* remove redundant check
2019-07-17 14:10:15 -07:00
Sagar Dhawan 10d85f8366 Add weighted shuffle support for values upto u64::MAX (#5151)
automerge
2019-07-17 12:44:28 -07:00
dependabot-preview[bot] bbd0455418 Bump log from 0.4.6 to 0.4.7 (#5144)
Bumps [log](https://github.com/rust-lang/log) from 0.4.6 to 0.4.7.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang-nursery/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.6...0.4.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-17 09:25:57 -06:00
Rob Walker f88c72c41e
stress tweaks (#5140) 2019-07-16 22:04:40 -07:00
sakridge 94db9cd412
Reduce banking_stage bench copy-paste code and fix programs bench (#4926) 2019-07-16 18:28:18 -07:00
Justin Starry bf5bce50a4
Fix stake pruning test (#5124) 2019-07-16 13:20:03 -04:00
Rob Walker fe87c05423
fix transaction_count (#5110)
* fix transaction_count

* add sig count to bank hash
2019-07-15 13:42:59 -07:00
sakridge 40faaef9da
Revert "Logging (#5017)" (#5096)
This reverts commit b50a3bae72.
2019-07-14 18:48:15 -07:00
Rob Walker 0d27515d09
tmp dirs target to farf (#5079) 2019-07-12 18:28:42 -07:00
Pankaj Garg 1c966aac25 Facility to generate a blocktree prune list using ledger tool (#5041)
automerge
2019-07-12 16:58:13 -07:00
Sagar Dhawan 7aecb87bce
Add a version field to blobs (#5057) 2019-07-12 13:43:19 -07:00
Michael Vines ebcdc06dc3
Restore ledger-tool print and json commands (#5048)
* Restore ledger-tool print and  json commands

* Remove obsolete read_ledger()
2019-07-11 20:33:36 -07:00
Sagar Dhawan 0a36a78133
Fix replicator segment selection (#5046) 2019-07-11 18:31:41 -07:00
Trent Nelson 5698d48dc8 merkle-tree: Make instantiation a little less painful (#5037)
automerge
2019-07-11 15:15:08 -07:00
Pankaj Garg 4c90898f0b
Dynamic erasure set configuration (#5018)
* Use local erasure session to create/broadcast coding blobs

* Individual session for each recovery (as the config might be different)

* address review comments

* new constructors for session and coding generator

* unit test for dynamic erasure config
2019-07-11 13:58:33 -07:00
Tyera Eulberg b2c776eabc
Fix getProgramAccounts RPC (#5024)
* Use scan_accounts to load accounts by program_id

* Add bank test

* Use get_program_accounts in RPC
2019-07-11 12:58:28 -06:00
Sagar Dhawan e4926e4110
Set exit when replicator run exits (#5016) 2019-07-10 16:27:18 -07:00
carllin b50a3bae72
Logging (#5017)
* Add logging to replay_stage

* locktower logging
2019-07-10 15:52:31 -07:00
Sagar Dhawan 35ec7a5156
Decouple turns from segments in PoRep (#5004)
* Decouple Segments from Turns in Storage

* Get replicator local cluster tests running in a reasonable amount of time

* Fix unused imports

* Document new RPC APIs

* Check for exit while polling
2019-07-10 13:33:29 -07:00
Mark E. Sinclair a383ea532f Implement new Index Column (#4827)
* Implement new Index Column

* Correct slicing of blobs

* Mark coding blobs as coding when they're recovered

* Prevent broadcast stages from mixing coding and data blobs in blocktree

* Mark recovered blobs as present in the index

* Fix indexing error in recovery

* Fix broken tests, and some bug fixes

* increase min stack size for coverage runs
2019-07-10 11:08:17 -07:00
Rob Walker f537482c86
remove set_leader from cluster_info (#4998) 2019-07-09 22:06:47 -07:00
Sagar Dhawan b8e7736af2 Move SLOTS_PER_SEGMENT to genesis (#4992)
automerge
2019-07-09 16:48:40 -07:00
Rob Walker f777a1a74c
groom replay_stage and poh_recorder (#4961)
* groom replay_stage and poh_recorder

* fixup

* fixup

* don't freeze() parent, need to review bank_forks and maybe vote...
2019-07-09 15:36:30 -07:00
carllin 22ef3c7c54
Blob verify (#4951)
* Ensure signable data is not out of range

* Add a broadcast stage that puts bad sizes in blobs

* Resign blob after modifyign size

* Remove assertions that fail when size != meta.size
2019-07-08 18:21:52 -07:00
Rob Walker 417e8d5064
fix blocktree_processor test_process_entries_stress (#4967) 2019-07-08 18:11:58 -07:00
Trent Nelson 1feb9bea21
Harden Merkle Tree against second pre-image attacks (#4925)
* merkle-tree: Harden against second pre-image attacks

* core/chacha: Bump test golden hash
2019-07-08 19:00:06 -06:00
Trent Nelson c5ba2e0883 bank_forks test stability (#4959)
automerge
2019-07-08 15:55:49 -07:00
Rob Walker eb4edd75e6
make bank commit_credits non public (#4944)
* make bank commit_credits non pub

* track down create() failure

* move bank_client to process_transaction(), which commits credits
2019-07-08 15:37:54 -07:00
Sagar Dhawan bb6bcd79c0
Handle replicator errors without panicking (#4955)
* Handle replicator errors without panicking

* Unwelcome println
2019-07-08 12:43:35 -07:00
Sagar Dhawan ef7022d638
Refactor replicators to not block on startup (#4932)
* Refactor replicators to not block on startup

* Ignore setup failure
2019-07-08 10:17:25 -07:00
carllin 2aac094f63
Ensure blobs are deserializable without unwrapping (#4948)
* Return result from deserializing blobs in blocktree instead of assuming deserialization will succeed

* Mark bad deserialization as dead fork

* Add test for corrupted blobs in blocktree and replay_stage
2019-07-07 14:37:12 -07:00
carllin fc180f4cbf
Halve stake of malicious validator (#4937) 2019-07-05 15:45:39 -07:00
dependabot-preview[bot] db13b52e6a Bump serde_json from 1.0.39 to 1.0.40 (#4881)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.39 to 1.0.40.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.39...v1.0.40)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-07-03 16:15:56 -06:00
Rob Walker 2ec5d2c7f5
start local_cluster nodes from genesis blocks (#4928) 2019-07-03 14:03:52 -07:00
Sagar Dhawan 55b8ff72d0
Enable parallel tests to reduce test time (#4919)
* Add crate to serialize some tests

* Ignore unused attribute warning

* Enable parallel run in CI

* Try to fix lograte tests

* Fix interdependent counter tests
2019-07-02 17:35:03 -07:00
Rob Walker 12ef0c25b5 change vote commission to u8 (from u32) (#4887)
automerge
2019-07-02 14:18:11 -07:00
Pankaj Garg 3615445a12
Broadcast run for injecting fake blobs in turbine (#4889)
* Broadcast run for injecting fake blobs in turbine

* address review comments

* new local cluster test that uses fake blob broadcast

* added a test to make sure tvu_peers ordering is guaranteed
2019-07-01 17:54:03 -07:00
Sagar Dhawan 65adce65fa
Always send pull responses to the origin addr (#4894) 2019-07-01 16:49:05 -07:00
carllin c1953dca8f
Cleanup some of banking stage (#4878)
* Add committable transactions that cause errors like InstructionErrors back to retryable list on MaxHeightReached

* Remove unnecessary logic

* Add comments/renaming for clarity
2019-07-01 12:14:40 -07:00
Tyera Eulberg d7ed3b8024 Add RPC api to return program accounts (#4876)
automerge
2019-06-29 09:59:07 -07:00
sakridge a89589a1d5
Add Measure abstraction over measuring time intervals (#4851)
Allows one to swap in different implementations. This provides
the normal Insant::now() -> .elapsed() path.
2019-06-29 15:34:49 +02:00
carllin 75b494d4a3 Lower warn to info, fetch from validator root instead of root + 1 (#4870)
* Lower warn to info, fetch from validator root instead of root + 1

* b/c -> because
2019-06-28 16:17:20 -07:00