Commit Graph

1382 Commits

Author SHA1 Message Date
Parth 6d2861f358
add unit test for minority fork overcommit attack (#7292)
* add unit test for minority fork overcommit attack

* add generic function to simulate fork selection
2019-12-10 22:06:16 +05:30
dependabot-preview[bot] 891767c6b7 Bump serde_json from 1.0.42 to 1.0.44
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.42 to 1.0.44.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.42...v1.0.44)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-08 09:43:24 -07:00
Trent Nelson 1ffd6b4b4d
Add program and runtime support for Durable Transaction Nonces (#6845)
* Rework transaction processing result forwarding

Durable nonce prereq

* Add Durable Nonce program API

* Add runtime changes for Durable Nonce program

* Register Durable Nonce program

* Concise comments and bad math

* Fix c/p error

* Add rent sysvar to withdraw ix

* Remove rent exempt required balance from Meta struct

* Use the helper
2019-12-07 12:54:10 -07:00
Michael Vines 77cd292828 getVoteAccounts RPC API no longer returns "idle" vote accounts, take II 2019-12-07 01:16:36 -07:00
Michael Vines 22d6951de5 Revert "getVoteAccounts RPC API no longer returns "idle" vote accounts"
This reverts commit 103e40b9689d74ece606385a67d166f55f453823.
2019-12-07 01:16:36 -07:00
Sunny Gleason c00216e3be
feat: ledger size and cleanup metrics (#7335) 2019-12-06 22:32:45 -05:00
Michael Vines 42247e0e1a getVoteAccounts RPC API no longer returns "idle" vote accounts (#7339) 2019-12-06 20:23:29 -05:00
TristanDebrunner 9ecb844de7 Split up ReplayStageConfig to make it derive Default (#7334)
automerge
2019-12-06 14:39:35 -08:00
Tyera Eulberg 3ab8185777
Add intermittent Timestamping to Votes (#7233)
* Add intermittent timestamp to Vote

* Add timestamp to VoteState, add timestamp processing to program

* Print recent timestamp with solana show-vote-account

* Add offset of 1 to timestamp Vote interval to initialize at node boot (slot 1)

* Review comments

* Cache last_timestamp in Tower and use for interval check

* Move work into Tower method

* Clarify timestamping interval

* Replace tuple with struct
2019-12-06 14:38:49 -07:00
Justin Starry b7d4330dd4
Fail fast if account paths cannot be canonicalized (#7300)
* Canonicalize account paths to avoid symlink issues

* fixes
2019-12-05 21:41:29 -05:00
sakridge cfc21e1225
Only serialize rooted append vecs (#7281) 2019-12-05 14:27:46 -08:00
Justin Starry d6c3396182
Canonicalize paths before symlink-ing when generating snapshots (#7294)
* Canonicalize paths before symlinking when generating snapshots

* cargo fmt
2019-12-05 14:58:02 -05:00
Sagar Dhawan a95d37ea25
Fix repair slowness when most peers are unable to serve requests (#7287)
* Fix repair when most peers are incapable of serving requests

* Add a test for getting the lowest slot in blocktree

* Replace some more u64s with Slot
2019-12-05 11:25:13 -08:00
TristanDebrunner fae9c08815
Add ReplayStageConfig (#7195) 2019-12-04 11:17:17 -07:00
dependabot-preview[bot] c9245751e9 Bump nix from 0.15.0 to 0.16.0 (#7216)
Bumps [nix](https://github.com/nix-rust/nix) from 0.15.0 to 0.16.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits/v0.16.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-04 09:35:45 -07:00
carllin f0a40862d6
Support local cluster edge case testing (#7135)
* Refactor local cluster to support killing a partition

* Rework run_network_partition

* Introduce fixed leader schedule

* Plumb fixed schedule into test
2019-12-03 16:31:59 -08:00
Pankaj Garg a0eafa12e3
Update to hashes_per_tick computation, and tick duration datapoint (#7127) 2019-12-02 18:02:11 -08:00
Pankaj Garg 076e384bb5
Tool to tune system parameters like PoH service priority (#7155)
* New daemon to tune system parameters like PoH service priority

* fixes for Linux

* integrate with poh_service

* fixes

* address review comments

* remove `dead_code` directive
2019-12-02 16:46:46 -08:00
Michael Vines 41cff1b49d
Remove spammy log 2019-12-02 16:52:01 -07:00
dependabot-preview[bot] 0f872af502 Bump libc from 0.2.65 to 0.2.66 (#7180)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.65 to 0.2.66.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.65...0.2.66)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-02 09:56:03 -07:00
dependabot-preview[bot] b13696ea1a Bump jsonrpc-http-server from 14.0.3 to 14.0.5 (#7181)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.3...jsonrpc-http-server-v14.0.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-02 09:55:40 -07:00
dependabot-preview[bot] 5fbbf7c748 Bump jsonrpc-ws-server from 14.0.3 to 14.0.5 (#7172)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.3...jsonrpc-ws-server-v14.0.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-29 23:06:23 -07:00
dependabot-preview[bot] e7fe0db051 Bump jsonrpc-derive from 14.0.3 to 14.0.5 (#7173)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.3...jsonrpc-derive-v14.0.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-29 14:15:02 -07:00
dependabot-preview[bot] dcb7bd8c74 Bump jsonrpc-pubsub from 14.0.3 to 14.0.5 (#7174)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.3...jsonrpc-pubsub-v14.0.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-29 14:14:57 -07:00
dependabot-preview[bot] f97626346b Bump jsonrpc-core from 14.0.3 to 14.0.5 (#7170)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 14.0.3 to 14.0.5.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-28 10:59:31 -07:00
dependabot-preview[bot] 7f4feaee08 Bump serde from 1.0.102 to 1.0.103 (#7163)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.102 to 1.0.103.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.102...v1.0.103)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-27 09:10:25 -07:00
dependabot-preview[bot] 5a30ef180a Bump serial_test_derive from 0.2.0 to 0.3.1 (#7153)
Bumps [serial_test_derive](https://github.com/palfrey/serial_test) from 0.2.0 to 0.3.1.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.2.0...v0.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-26 23:12:13 -07:00
dependabot-preview[bot] 0a0412e47e Bump serial_test from 0.2.0 to 0.3.1 (#7154)
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.2.0 to 0.3.1.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.2.0...v0.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-26 22:07:39 -07:00
dependabot-preview[bot] c810913861 Bump serde_derive from 1.0.102 to 1.0.103 (#7141)
automerge
2019-11-26 14:44:20 -08:00
dependabot-preview[bot] 2b13158e29 Bump serde_json from 1.0.41 to 1.0.42 (#7142)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.41 to 1.0.42.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.41...v1.0.42)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-26 15:09:01 -07:00
Sunny Gleason 0a0f15baca
RPC subscriptions for new slot notifications (#7114)
* feat: slot notifications via pubsub rpc w/ tests
2019-11-26 03:42:54 -05:00
Tyera Eulberg 58c144ee55
Add getBlockTime rpc api (#7130)
* Add getBlockTime rpc api

* Add getBlockTime to docs

* Fix duration rounding for common tick/slot durations; add slot duration calculation

* Expose slots_per_year

* Use genesis values instead of clock defaults to calculate block offset

* Add get-block-time cli subcommand

* Fix test_rent: decrease magic number usage
2019-11-26 00:40:36 -07:00
Michael Vines 294662a1ce
Bump version to 0.22.0 2019-11-25 21:34:50 -07:00
Sagar Dhawan 76a68c26c9
Track a Bank's parent slot independently from parent bank (#7131) 2019-11-25 15:34:51 -08:00
Tyera Eulberg 0f66e5e49b Add getConfirmedBlock test to rpc (#7120)
automerge
2019-11-25 11:08:03 -08:00
dependabot-preview[bot] 686aa3a150 Bump chrono from 0.4.9 to 0.4.10 (#7113)
automerge
2019-11-25 10:01:46 -08:00
dependabot-preview[bot] 094c391cd7 Bump itertools from 0.8.1 to 0.8.2 (#7111)
Bumps [itertools](https://github.com/bluss/rust-itertools) from 0.8.1 to 0.8.2.
- [Release notes](https://github.com/bluss/rust-itertools/releases)
- [Commits](https://github.com/bluss/rust-itertools/commits/v0.8.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-11-25 10:22:47 -07:00
sakridge 6c89226ccf
Purge zero lamport accounts on snapshot ingestion (#7010)
Snapshots do not load the original index, so they must
purge zero lamport accounts again.
2019-11-22 18:22:28 -08:00
Michael Vines 0bd41f98ed Avoid jemalloc in windows build (#7089)
automerge
2019-11-21 18:39:29 -08:00
anatoly yakovenko d9e7a5fcbe
Use fork weight instead of individual bank weight for fork selection. (#7079)
* Fix weight calculation

* Fix tests

* fork weight

* wait until nodes are in the leader schedule

* enable sanity

* fewer long tests
2019-11-21 15:47:08 -08:00
Tyera Eulberg c965a110f2
Use unbounded channel (#7081) 2019-11-21 14:23:40 -07:00
Ryo Onodera 8cbc450192 Create genesis.tar.bz2 in solana-genesis (#7039)
* Use clap_utils

* Create genesis.tar.bz2 in solana-genesis

* Remove shell-based genesis.tar.bz2 generation

* Make Option=>Result conv more rusty

* stop using solana_logger

* Simplify by just using vec!

* clean up abit
2019-11-21 10:57:27 -07:00
sakridge e66b29943b
datapoint for best fork weight and slot in replay (#7066) 2019-11-20 17:26:52 -08:00
Sagar Dhawan 100b9dd12a
Fix num nodes metrics (#7068)
* Fix num nodes metric

* Fix node count metrics
2019-11-20 17:00:31 -08:00
Jack May 3415db9739
Merge api/program into single units (#7061) 2019-11-20 16:32:19 -08:00
Tyera Eulberg 97ca6858b7
Write transaction status and fee into persistent store (#7030)
* Pass blocktree into execute_batch, if persist_transaction_status

* Add validator arg to enable persistent transaction status store

* Pass blocktree into banking_stage, if persist_transaction_status

* Add validator params to bash scripts

* Expose actual transaction statuses outside Bank; add tests

* Fix benches

* Offload transaction status writes to a separate thread

* Enable persistent transaction status along with rpc service

* nudge

* Review comments
2019-11-20 16:43:10 -07:00
Parth 9f6a2e51b2
add credit-debit rent handling (#6947)
* add credit-debit rent handling

* add tests

* charge rent for validator account for fee credit

* rent is stored per tx instead of account
2019-11-21 01:57:02 +05:30
anatoly yakovenko b150da837a Use epoch as the gossip purge timeout for staked nodes. (#7005)
automerge
2019-11-20 11:25:18 -08:00
Jack May d184d3a732
Merge native programs parts into one unit (#7047) 2019-11-20 10:12:43 -08:00
Jack May d2ed921bc6
Cleanup nightly warnings (#7055) 2019-11-19 20:15:37 -08:00
Pankaj Garg d32a072190
Use ticks_per_slot to calculate maximum grace ticks (#7024)
* Use ticks_per_slot to calculate maximum grace ticks

* fix test

* fix votable candidate ordering

* fixes to pick_best_fork() and a unit test

* fixes
2019-11-19 17:55:42 -08:00
Sagar Dhawan f2badf2c5d
Fix a bug where gossip loops forever while splitting messages (#7032)
* Fix a bug where gossip loops forever while splitting messages

* Get rid of while loop

* Minor clean up and rename
2019-11-19 11:51:51 -08:00
Tyera Eulberg ea656b1a3f
Add parent slot to getConfirmedBlock (#7038)
* Add parent slot to getConfirmedBlock

* Fix bad text-replace

* Use camelCase in getConfirmedBlock
2019-11-19 09:39:55 -07:00
Dan Albert b958bf9086
Fix confirmation metrics (#7035) 2019-11-19 09:51:50 -05:00
carllin 43144cfe8b
Make banks that fail threshhold check resettable (#7027) 2019-11-19 02:36:30 -08:00
carllin 11d2d2eccd
Fix progress map losing banks and recomputing stats (#7026)
* Fix progress map missing banks

* Fix confirmations

* Fix test

* Initialiize progress with frozen banks atartup
2019-11-19 02:36:00 -08:00
Sagar Dhawan 6bfe0fca1f
Add a version field to shreds (#7023)
* Add a version field to shreds

* Clippy

* Fix Chacha Golden

* Fix shredder bench compile

* Fix blocktree bench compile
2019-11-18 18:05:02 -08:00
Tyera Eulberg e0a2bb9d86
Legitimately map transactions to statuses in blocktree (#7011)
* Refactor rocksdb TransactionStatus to store/return struct; hook up map_transactions_to_statuses

* Cleanup use statements
2019-11-18 09:12:42 -07:00
Tyera Eulberg 3bc8d78801
Add ConfirmedBlock struct, and rework Blocktree apis to include block… (#7004)
* Add RpcConfirmedBlock struct, and rework Blocktree apis to include blockhash info and dummy tx statuses

* Remove unused lifetime
2019-11-17 20:17:15 -07:00
carllin b66c03667c
Log for threshold failure (#7008) 2019-11-17 17:10:16 -08:00
Michael Vines c3926e6af0
|solana-gossip spy| no longer requires an entrypoint (#6999) 2019-11-16 14:16:28 -07:00
carllin 70322d1ff8
Add error logging to dead slots (#7000) 2019-11-16 02:54:51 -08:00
anatoly yakovenko 0ca943f49b RecyclerCache for shred_sigverify (#6986)
automerge
2019-11-15 12:16:56 -08:00
sakridge 36e3ccfc68
Remvoe pinned memory (#6976) 2019-11-15 10:58:25 -08:00
anatoly yakovenko 59413b3124 Fix rules for fork selection (#6906)
automerge
2019-11-15 08:36:33 -08:00
Michael Vines 006c39380a
Display 'none' instead of 0.0.0.0 (#6973) 2019-11-14 20:24:35 -07:00
Tyera Eulberg 852a2146ab
Add Blocktree api to get transactions by slot (#6966)
* Add blocktree method to get confirmed-block txs

* Clean up use statements

* Add test, and fmt

* Plumb new blocktree method into getConfirmedBlock
2019-11-14 16:34:39 -07:00
sakridge 99b42f210c Remove unused sha2 dep (#6964)
automerge
2019-11-14 14:01:11 -08:00
Tyera Eulberg 03f7f0d18c Rename getBlock to getConfirmedBlock; remove getBlocksSince (#6961)
automerge
2019-11-14 13:14:42 -08:00
Sagar Dhawan 79d7090867
Remove obsolete references to Blob (#6957)
* Remove the name "blob" from archivers

* Remove the name "blob" from broadcast

* Remove the name "blob" from Cluset Info

* Remove the name "blob" from Repair

* Remove the name "blob" from a bunch more places

* Remove the name "blob" from tests and book
2019-11-14 11:49:31 -08:00
Michael Vines e7f63cd336
Upgrade to rust 1.39.0 (#6939)
* Upgrade to rust 1.39.0

* 1.39.0 clippy
2019-11-14 12:27:01 -07:00
Sagar Dhawan f108f483b7
Remove Blobs and switch to Packets (#6937)
* Remove Blobs and switch to Packets

* Fix some gossip messages not respecting MTU size

* Failure to serialize is not fatal

* Add log macros

* Remove unused extern

* Apparently macro use is required

* Explicitly scope macro

* Fix test compile
2019-11-14 10:24:53 -08:00
Sunny Gleason 42af8b199f
feat: add tests for invalid/failure cases (#6951) 2019-11-14 11:41:26 -05:00
carllin 43e2301e2c
Fix roots overrunning broadcast (#6884)
* Add trusted pathway for insert_shreds to avoid checks
2019-11-14 00:32:07 -08:00
Parth 7b05b3dbb3
rent collector improvments (#6888)
* avoid account copying + pre-empt rent

* adding support for base rent
2019-11-14 10:56:49 +05:30
Ryo Onodera 4fc767b3f6
Move version! from core:: to clap_utils:: (#6944)
* Move version! from core to clap-utils

* Completely move version! from core:: to clap_utils::

* rustfmt

* Do remaining transition after rebase
2019-11-14 13:10:38 +09:00
Greg Fitzgerald a3a830e1ab
Delete Service trait (#6921) 2019-11-13 11:12:09 -07:00
Michael Vines fcc2874591
Remove/address some TODOs (#6923) 2019-11-13 09:43:15 -07:00
Greg Fitzgerald 30a08f4282 Cleanup ledger macros (#6916)
automerge
2019-11-13 07:14:09 -08:00
Michael Vines 86faa3f995
Properly type RpcClient::get_version() (#6919) 2019-11-12 22:01:04 -07:00
Ryo Onodera 148a58865e Make creating new snapshot.tar.bz2 truly-atomic (#6902) 2019-11-12 14:21:56 -07:00
Ryo Onodera 3faeb7fa79 Rename solana-netutil to solana-net-utils for consistency (#6895)
* sed -i -e 's/netutil/net_utils/g' $(git grep --files-with-matches netutil :**.rs)

* sed -i -e 's/netutil/net-utils/g' $(git grep --files-with-matches netutil)

* git mv netutil/ net-utils

* Tweak a bit

* Fix rustfmt & clippy
2019-11-12 13:37:13 -07:00
Rob Walker bb00904fc8
add rent reserve for bootstrap stakes (#6876)
* genesis investor stakes

* assert rent is sufficient for these bootstrappers
2019-11-12 12:33:40 -08:00
Sunny Gleason 5903339c17
feat: return bank/block info with block-related results (#6716) 2019-11-12 14:49:41 -05:00
Michael Vines c6b108ef4f
Don't panic in sdk/ when genesis fails to load (#6892) 2019-11-12 10:24:49 -07:00
Tyera Eulberg 7e82450d7b
Serialize transaction in proper wire format instead of json (#6889) 2019-11-12 10:45:10 -05:00
Sagar Dhawan 95d6586dd7
Remove debug datapoint that isn't being plotted (#6873) 2019-11-11 14:25:25 -08:00
Sagar Dhawan 816b2d7ff8
Tune repair to be less aggressive (#6868) 2019-11-11 13:12:22 -08:00
Michael Vines 4be646c695
discover() by gossip sockaddr instead of just by gossip ip address (#6865) 2019-11-11 12:42:58 -07:00
Tyera Eulberg cc6e1ea200
Stub out `getBlocksSince` and `getBlock` methods (#6853)
* Add getBlocksSince rpc method, and initial stub of getBlock method

* Return test transactions from getBlock method

* clippy

* Add comment on get_block method
2019-11-11 13:18:34 -05:00
Michael Vines cfab36cb1d
Include channel and commit info in the version of pre-release builds (#6819) 2019-11-10 22:39:13 -07:00
Justin Starry 5835b3b8eb
Increase timeout when confirming airdrop for max commitment (#6858)
* Increase timeout when confirming airdrop for max commitment

* Add commitment to airdrop rpc trace

* Flip commitment check
2019-11-10 12:20:52 -05:00
Michael Vines 1ddf90ed08
Compress contact_info_trace() output to improve CI log rendering (#6852) 2019-11-09 01:12:18 -07:00
Justin Starry 9807f47d4e
Rename genesis block to genesis config (#6816) 2019-11-08 23:56:57 -05:00
Justin Starry 02058ea699
Reject blocks with invalid last ticks in replay stage (#6833)
* Reject blocks with invalid last ticks in replay stage

* slot_full
2019-11-08 20:21:54 -05:00
Michael Vines d1daeb44e6
Remove custom stack_size() (#6844) 2019-11-08 17:11:07 -07:00
Michael Vines efdfc5c327
Remove TODOs (#6843) 2019-11-08 16:43:18 -07:00
Michael Vines 9c00ad9ff2
Remove some low-hanging TODOs (#6839) 2019-11-08 16:41:36 -07:00
Pankaj Garg da425cc225
Don't insert coding shreds into blocktree on leader (#6831) 2019-11-08 13:54:23 -08:00
Michael Vines b800642fa4
Add new fork log message for when the node is leader for consistency (#6808) 2019-11-08 12:30:25 -07:00
Parth 5bd05fba09
require `to` account signature (#6658)
* require to signature

* fixing invocation to create_account

* fix create_account references

* address review comment

* whacking bugs in tests

* fixing stake program tests
2019-11-08 15:57:35 +05:30
sakridge 8e81bc1b49
Fix pinning (#6604)
Remove Deref implementations and add more pass-throughs to the PinnedVec
wrapper.
Warm recyclers
set_pinnable
2019-11-07 19:48:33 -08:00
Rob Walker b64b54f48f
unfork dalek ed25519 (#6776) 2019-11-07 17:08:10 -08:00
Sagar Dhawan 67d1e2903c
Upgrade Repair be more intelligent and agressive (#6789)
* Upgrade Repair be more intelligent and agressive

* Fix u64 casts

* Fix missing bracket

* Add 1 second delay to test to allow repair to kick in
2019-11-07 11:08:09 -08:00
Sunny Gleason a684984f8b
feat: add confirm_transaction, add rpc client test (#6778) 2019-11-06 22:08:03 -05:00
Jack May 65de227520
Don't panic on packet data (#6769) 2019-11-06 14:32:37 -08:00
Pankaj Garg 0ace79939b
Add reference tick to data shreds (#6772)
* Add reference tick to data shreds

* fix tests
2019-11-06 13:27:58 -08:00
Tyera Eulberg b3a75a60a4
Use rooted bank by default in rpc bank selection (#6759)
* Name anonymous parameters for clarity

* Add CommitmentConfig to select bank for rpc

* Add commitment information to jsonrpc docs

* Update send_and_confirm retries as per commitment defaults

* Pass CommitmentConfig into client requests; also various 'use' cleanup

* Use _with_commitment methods to speed local_cluster tests

* Pass CommitmentConfig into Archiver in order to enable quick confirmations in local_cluster tests

* Restore solana ping speed

* Increase wallet-sanity timeout to account for longer confirmation time
2019-11-06 14:15:00 -07:00
anatoly yakovenko 5e8668799c Fewer recyclers. (#6770)
automerge
2019-11-06 12:35:51 -08:00
anatoly yakovenko 67f636545a Refactor sigverify to stage for signing shreds on the GPU (#6635)
automerge
2019-11-06 10:52:30 -08:00
carllin 7ff2a44a63 Make last shred for an interrupted slot signed + typed (#6760) 2019-11-06 08:25:17 -07:00
Michael Vines b5074d8577
Enable JSON RPC request/response logging by default (#6758) 2019-11-06 08:23:13 -07:00
carllin 24102a7435
Allow voting on empty banks (#6719)
* Allow votes on empty banks

* Remove making first bank is_delta true, no longer necessary for idling

* Remove votable from ledger tool
2019-11-06 01:02:26 -08:00
Jack May 9614d17024
Limit deserialization of data coming off the wire (#6751)
* Limit deserialization of data coming off the wire

* Feedback and cleanup
2019-11-06 00:07:57 -08:00
Michael Vines 09e648f957
ledger-tool/: Include full validator voting history in fork-graph (#6756) 2019-11-05 19:40:00 -07:00
carllin 8f91b5aab3
Add threshold to repairman for same slot (#6728) 2019-11-05 12:48:45 -08:00
sakridge c312d4fba0
Calculate proofs collected and don't encrypt if there are none (#6698) 2019-11-05 11:38:50 -08:00
Michael Vines 7203036e3e Adjust nofiles within Blocktree::open() for all ledger/ users (#6737)
automerge
2019-11-05 11:18:49 -08:00
Tyera Eulberg c6931dcb07
Remove credit-only account handling (#6726)
* Renaming
- credit-only/credit-debit to read-only/read-write
- debitable to writable

* Remove credit handling, making credit-only accounts read-only

* Update programs to remove deprecated credit-only account designation

* Use readonly and writable instead of underscored types
2019-11-05 09:38:35 -07:00
Michael Vines fba1af6ea9
ledger-tool can now load a ledger snapshot (#6729) 2019-11-04 22:14:55 -07:00
anatoly yakovenko b825d04597 Pull perf into a separate module. (#6718)
automerge
2019-11-04 20:13:43 -08:00
Michael Vines 4d52f47f87
Move get_bank_forks() into ledger/ so its available for use by ledger-tool/ (#6720) 2019-11-04 19:10:06 -07:00
anatoly yakovenko f54cfcdb8f
Store and persists full stack of tower votes in gossip (#6695)
* vote array

wip

wip

wip

update

gossip index should match tower index

tests build

clippy

test index after expired vote

test

bank specific last vote sync time

* verify

* we are likely to see many more warnings about old votes now
2019-11-04 16:19:54 -08:00
sakridge 57983980a7 Lower verify-batch-size to debug (#6722)
automerge
2019-11-04 16:00:59 -08:00
Tyera Eulberg 33f4aaf3fd
Rename confidence to commitment (#6714) 2019-11-04 16:44:27 -07:00
Pankaj Garg c138d692b1
Show all ports for nodes in gossip table (#6717)
* Show all ports for nodes in gossip table

* review comments
2019-11-04 15:05:08 -08:00
Michael Vines 5e3697807c
Fail gracefully if AVX support is missing (#6705) 2019-11-04 11:03:39 -07:00
Sagar Dhawan 568475e2db
Fix incorrectly signed CrdsValues (#6696) 2019-11-03 10:07:51 -08:00
anatoly yakovenko 9ea398416e
Sign shreds on the GPU (#6595)
* sign gpu shreds

* wip

* checks

* tests build

* test

* tests

* test

* nits

* sign cpu test

* write out the sigs in parallel

* clippy

* cpu test

* prepare secret for gpu

* woot!

* update

* bump perf libs
2019-11-02 06:23:14 -07:00
Michael Vines 50a17fc00b Use Slot and Epoch type aliases instead of raw u64 (#6693)
automerge
2019-11-02 00:38:30 -07:00
Sagar Dhawan a57f6b70da Fix swapped repair and forwards addrs (#6691)
automerge
2019-11-01 16:01:42 -07:00
anatoly yakovenko 385b4ce959
Get rid of verified packets and use the Meta::discard flag (#6674)
* get rid of verified packets and use the disabled meta field everywhere
2019-11-01 14:23:03 -07:00
Sagar Dhawan 2d67962c2f
Send repairman shreds to the repair socket (#6671) 2019-10-31 18:23:50 -07:00
Justin Starry e8e5ddc55d
Verify number of hashes for each block of entries (#6262)
* Verify number of hashes for each block of entries

* Fix blocktree processor tick check

* Rebase once more
2019-10-31 16:38:50 -04:00
carllin 59f3dc3b6b
Fix PohRecorder Metrics (#6644)
* Update Poh Recorder Dashboard

* Update PohRecorder logging
2019-10-30 18:55:29 -07:00
Rob Walker fa12a5f70b
kill rent calculator (#6625) 2019-10-30 16:25:12 -07:00
sakridge 38491c8c4b
Reduce verify-batch-size log (#6623) 2019-10-30 13:41:11 -07:00
Sagar Dhawan 801337a422 Refactor Weighted Shuffle (#6614)
automerge
2019-10-29 21:02:11 -07:00
TristanDebrunner b4dc1a7263 Remove move feature (#6605)
automerge
2019-10-29 17:14:07 -07:00
Sagar Dhawan ef3aa2731c Fix Weighted Best calculation (#6606)
automerge
2019-10-29 17:04:11 -07:00
dependabot-preview[bot] 029a2837e4 Bump jsonrpc-http-server from 14.0.1 to 14.0.3 (#6597)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.1...v14.0.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-29 10:30:06 -07:00
anatoly yakovenko 34a9619806
SigVerify stage for shreds. (#6563) 2019-10-28 16:07:51 -07:00
Sagar Dhawan 579a02529d
Fix unnecessarily copying shreds in broadcast stage (#6588)
* Optimize coalesce_shreds to not explictly clone

* Remove Coalesce Shreds altogether

* fn no longer needs clippy exception
2019-10-28 14:58:27 -07:00
anatoly yakovenko 243fa6cf63
Shred gpu sigverify (#6520)
Implement APIs for verifying shred signatures on the GPU.
2019-10-28 10:29:38 -07:00
dependabot-preview[bot] 30c0a7d069 Bump serde from 1.0.101 to 1.0.102 (#6581)
automerge
2019-10-28 09:19:39 -07:00
dependabot-preview[bot] 71b4e765c8 Bump itertools from 0.8.0 to 0.8.1 (#6583)
Bumps [itertools](https://github.com/bluss/rust-itertools) from 0.8.0 to 0.8.1.
- [Release notes](https://github.com/bluss/rust-itertools/releases)
- [Commits](https://github.com/bluss/rust-itertools/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-28 08:26:15 -07:00
dependabot-preview[bot] 73dd5aa2d1 Bump serde_derive from 1.0.101 to 1.0.102 (#6582)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.101 to 1.0.102.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.101...v1.0.102)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-28 08:24:13 -07:00
carllin 6efaaa9d7a
Blocktree metrics (#6527)
* Add metrics for blocktree performance
* Plumb metrics through window service
2019-10-26 16:15:59 -07:00
Dan Albert 73c31d873e
Update Cargo.toml versions from 0.20.0 to 0.21.0 (#6568) 2019-10-25 17:40:49 -06:00
dependabot-preview[bot] 8a37b1e742 Bump jsonrpc-ws-server from 14.0.1 to 14.0.3 (#6553)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 09:40:42 -07:00
dependabot-preview[bot] 0cf4eb2ee4 Bump jsonrpc-core from 14.0.1 to 14.0.3 (#6552)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 09:40:25 -07:00
dependabot-preview[bot] 5496f85dbc Bump crc from 1.8.1 to 1.9.0 (#6511)
Bumps [crc](https://github.com/mrhooray/crc-rs) from 1.8.1 to 1.9.0.
- [Release notes](https://github.com/mrhooray/crc-rs/releases)
- [Commits](https://github.com/mrhooray/crc-rs/compare/1.8.1...v1.9.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 09:40:17 -07:00
dependabot-preview[bot] 1738632822 Bump jsonrpc-pubsub from 14.0.1 to 14.0.3 (#6551)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 07:41:09 -07:00
dependabot-preview[bot] bbd5dde66d Bump jsonrpc-derive from 14.0.1 to 14.0.3 (#6554)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 14.0.1 to 14.0.3.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.1...v14.0.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-25 07:41:03 -07:00
sakridge 53c7be32b6
Add more retransmit and streamer stats (#6534) 2019-10-24 19:27:19 -07:00
Michael Vines 397ea05aa7
spy nodes are now gossip entrypoints (#6532) 2019-10-24 15:35:33 -07:00
Rob Walker b4119c454a
credit_only credits forwarding (#6509)
* credit_only_credits_forwarding

* whack transfer_now()

* fixup

* bench should retry the airdrop TX

* fixup

* try to make bench-exchange a bit more robust, informative
2019-10-23 22:01:22 -07:00
sakridge 03d29a8311
Async poh verify (#6353)
* Async poh verify

* Up ticks_per_s to 160

GPU poh verify needs shorter poh sequences or it takes forever to
verify. Keep slot time the same at 400ms.

* Fix stats

* Don't halt on ticks

* Increase retries for local_cluster tests and make repairman test serial
2019-10-23 12:11:04 -07:00
Michael Vines 35cc74ef25
Add GenesisBlock::OperatingMode to control how cluster features are activated (#6430) 2019-10-23 11:50:10 -07:00
Sunny Gleason 0b2d4f32fa
feat: get epoch schedule rpc, update cli (#6500) 2019-10-22 16:41:18 -04:00
sakridge e6438098e1 Increase archiver polling timeout (#6501)
automerge
2019-10-22 12:15:55 -07:00
carllin b38bf90de7
Deshred blocks in parallel (#6461)
* Deshred in parallel

* Add tests for corrupt slots and parallel deshred

* Rename load_blocktree_entries to load_blocktree_entries_with_shred_count
2019-10-21 16:15:10 -07:00
Pankaj Garg 84e911361a
Use constants instead of lazy_static for shred header sizes (#6472) 2019-10-21 12:46:16 -07:00
Sagar Dhawan 364583ea5c
Fix copying packets in Window Service (#6429)
* Fix copying packets in Window Service

* Parallelize over batches instead of within batches
2019-10-21 12:04:52 -07:00
Greg Fitzgerald 9232057e95
Rename replicator to archiver (#6464)
* Rename replicator to archiver

* cargo fmt

* Fix grammar
2019-10-21 11:29:37 -06:00
dependabot-preview[bot] 91920cc390 Bump jsonrpc-pubsub from 14.0.0 to 14.0.1 (#6465)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.0...v14.0.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-21 07:48:28 -07:00
dependabot-preview[bot] cc1cc7be94 Bump jsonrpc-derive from 14.0.0 to 14.0.1 (#6466)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.0.0...v14.0.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-21 07:48:14 -07:00
Greg Fitzgerald 2636418659
Move blocktree_processor to solana_ledger (#6460)
* Drop core::result dependency in bank_forks

* Move blocktree_processor into solana_ledger
2019-10-20 09:54:38 -06:00
Justin Starry 31e9074ae5
Rename leader_after_slots to leader_after_n_slots (#6459) 2019-10-19 23:28:33 -04:00
Rob Walker e2c316d2d0
system_instruction_processor updates (#6448)
* zero lamport account creation

* whack create_user_account, take 2

* target->to

* ..

* ..

* update chacha golden

* update chacha golden

* ..

* ..
2019-10-19 18:23:27 -07:00
sakridge 74ee88d9bc
Add storage stage and bank_forks tests to integration (#6458) 2019-10-19 12:09:45 -07:00
Pankaj Garg badeb4d31a
Rework shred headers to fix position of signature (#6451)
* Rework shred headers to fix position of signature

* fix clippy
2019-10-18 22:55:59 -07:00
Tyera Eulberg 785c2574cd Check that transaction fee-payer is a debitable account (#6454)
automerge
2019-10-18 20:39:05 -07:00
sakridge 1a77f7ce3b Change to 0x7f which is a valid short_vec len (#6455)
automerge
2019-10-18 19:56:48 -07:00
Greg Fitzgerald 6e7dccbbfb
Add an error enum to snapshot_utils (#6453) 2019-10-18 19:16:06 -06:00
sakridge 32bfced6a4
Add offset checks for sigverify (#6452)
* Add offset checks for sigverify

* decode_len returning error instead of unwrap
2019-10-18 17:52:59 -07:00
Greg Fitzgerald e81ba8e79f Split snapshot_package module (#6447)
automerge
2019-10-18 14:58:16 -07:00
Greg Fitzgerald a19ffb353d Don't hide serialization errors (#6443)
automerge
2019-10-18 13:35:05 -07:00
dependabot-preview[bot] 171c0d5421 Bump jsonrpc-core from 14.0.0 to 14.0.1 (#6439)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-18 11:17:25 -07:00
dependabot-preview[bot] c01bc4afbd Bump jsonrpc-http-server from 14.0.0 to 14.0.1 (#6437)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-18 11:04:30 -07:00
dependabot-preview[bot] c404008743 Bump jsonrpc-ws-server from 14.0.0 to 14.0.1 (#6438)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 14.0.0 to 14.0.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-18 11:03:58 -07:00
Ryo Onodera 193c9a08e0 Reject TXs when there is a mismatch (#6236)
automerge
2019-10-18 09:48:35 -07:00
Greg Fitzgerald 5468be2ef9 Add solana-ledger crate (#6415)
automerge
2019-10-18 09:28:51 -07:00
TristanDebrunner 9cf9de6044 Remove the Cursor struct (#6426) 2019-10-18 09:18:36 -06:00
dependabot-preview[bot] 51dad397ed Bump libc from 0.2.64 to 0.2.65
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.64 to 0.2.65.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.64...0.2.65)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-18 08:05:49 -07:00
carllin 27c0d30a07
Fix logging (#6417) 2019-10-18 02:06:41 -07:00
Pankaj Garg 6c33c3a5ba Update shred tests to use specific error codes (#6428)
automerge
2019-10-17 22:50:38 -07:00
Sagar Dhawan 298ba34c3c
Add flag to mark a packet as discarded (#6427) 2019-10-17 16:26:29 -07:00
Greg Fitzgerald 24846b7b61 Don't use BlocktreeError from Shredder (#6423)
automerge
2019-10-17 15:44:15 -07:00
TristanDebrunner 9e52d11ad0
Remove Backend trait (#6407) 2019-10-17 15:19:27 -06:00
Greg Fitzgerald d3068c3918
Remove circular dependencies in core (#6408)
* Remove core::result dependency from blocktree

* Remove core::result dependency from shred

* Move Packet from core::packet to sdk::packet

This way we don't need to split perf_libs yet.

* Disable packet when compiling BPF programs
2019-10-17 11:37:30 -06:00
Greg Fitzgerald a931ad40c8 Remove unused code in entry (#6414)
automerge
2019-10-17 09:59:40 -07:00
Pankaj Garg 83c1831a01
Fix replay stage test (#6406) 2019-10-16 15:41:43 -07:00
carllin ccb4e32ee0
ReplayStage metrics (#6358)
* ReplayStage metrics

* Add more metrics

* Refactor get_slot_entries_with_shred_count() to detect wasted work

* Update dashboard

* Update broadcast slots to micros

* Add broadcast dashboard
2019-10-16 14:32:18 -07:00
Justin Starry 7e6e7e8406
Remove special handling of first ledger tick (#6263)
* Remove special handling of first ledger tick

* Fix subtraction overflow

* @garious feedback

* Back to height

* More tick_height name changes

* Fix off-by-one

* Fix leader tick error

* Fix merge conflict

* Fix recently added test
2019-10-16 15:53:11 -04:00
Ryo Onodera e267dfacdd Stabilize some banking stage tests (#6251)
* Stabilize some banking stage tests

Fixes #5660

* Fix CI...

* clean up

* Fix ci

* Address review nits

* Use bank.max_tick_height due to off-by-one for no PohRecord's clearing bank

* Fix CI...

* Use bank.max_tick_height() instead for clarity
2019-10-16 12:37:27 -07:00
Ryo Onodera f4c5da3c72 Fix unaligned read of short_vec pubkey_size in sigverify (#6388)
automerge
2019-10-16 11:09:17 -07:00
Pankaj Garg a258e1e0b3 Fix flaky test_recv_mmsg_batch_size (#6399)
automerge
2019-10-16 11:01:41 -07:00
dependabot-preview[bot] f6b48b0a67 Bump libc from 0.2.62 to 0.2.64
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.62 to 0.2.64.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.62...0.2.64)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-16 10:03:06 -07:00
Pankaj Garg 33052c1dd2 Cleanup shred header structures (#6378)
automerge
2019-10-15 20:48:45 -07:00
Pankaj Garg 20e800230f
Don't deserialize coding header for data shreds (#6367)
* Don't deserialize coding hdr for data shreds

* review comments

* fix tests
2019-10-15 15:18:23 -07:00
sakridge 1afc527919
Lower cluster_info-num_nodes datapoint (#6368) 2019-10-15 14:42:19 -07:00
sakridge dfca2b510b
Lower shred/receiver stats (#6365)
too many messages
2019-10-15 11:43:52 -07:00
Tyera Eulberg f8ccd90eeb Add ForkConfidenceCache methods (#6359)
automerge
2019-10-14 22:14:20 -07:00
Michael Vines 88bb55ffd2 Add get_vote_accounts() to RPC client 2019-10-14 20:38:51 -07:00
Michael Vines 5508ac6272 Add root slot to getVoteAccounts 2019-10-14 20:38:51 -07:00
dependabot-preview[bot] 2be03ca631 Bump reqwest from 0.9.21 to 0.9.22
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.9.21 to 0.9.22.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/v0.9.22/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.9.21...v0.9.22)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-14 16:33:16 -07:00
Tyera Eulberg 5d99853502 Add getBlockConfidence rpc endpoint (#6350)
automerge
2019-10-14 15:24:10 -07:00
Pankaj Garg 3313b2ff58 Fetch stage batching of forwarded txs (#6349)
automerge
2019-10-14 13:32:29 -07:00
Greg Fitzgerald 322fcea6e5
More fullnode to validator renaming (#6337) 2019-10-11 13:30:52 -06:00
Pankaj Garg 5650231df3
Increase buffer size for erasure meta DB column (#6335) 2019-10-11 12:18:11 -07:00
Justin Starry 78b2e4df9f
Revert "Revert "Bump jsonrpc-core from 13.2.0 to 14.0.0 (#6287)" (#6328)" (#6336)
This reverts commit 578aa439be.
2019-10-11 13:19:13 -04:00
carllin bf9c815b9e
Increase Index column buffers (#6268) 2019-10-10 23:17:39 -07:00
Justin Starry 578aa439be
Revert "Bump jsonrpc-core from 13.2.0 to 14.0.0 (#6287)" (#6328)
This reverts commit c2761a1259.
2019-10-11 00:32:06 -04:00
Pankaj Garg 364781366a
Use sendmmsg for broadcasting shreds (#6325)
* Replace packet with slice of data in sendmmsg

* fixes

* fix bench
2019-10-10 19:38:48 -07:00
Greg Fitzgerald c6e4641781
Remove many uses of legacy term 'fullnode' (#6324) 2019-10-10 17:33:00 -06:00
Pankaj Garg 753bd77b41
Use multicast to send retransmit packets (#6319) 2019-10-10 15:02:36 -07:00
carllin 1960ea8ed7
Add benches for shredding and poh (#6307)
* Add benches for shredding and poh

* ignore poh bench

* Factor out Poh bench as separate function
2019-10-10 14:00:24 -07:00
sakridge 1b775044f7
Use multiple retransmit stage threads/sockets (#6279) 2019-10-10 13:24:03 -07:00
Pankaj Garg 570b98c7bc
Multicast same packet to multiple destinations via sendmmg (#6316)
* Implement multicast same packet to multiple destinations using sendmmsg()
2019-10-10 13:09:15 -07:00
dependabot-preview[bot] c2761a1259 Bump jsonrpc-core from 13.2.0 to 14.0.0 (#6287)
* Bump jsonrpc-core from 13.2.0 to 14.0.0

Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 13.2.0 to 14.0.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.2.0...v14.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump all jsonrpc versions
2019-10-10 10:32:38 -06:00
carllin 1f345ce2d9
Don't grab keypair from cluster info on every iteration of broadccast (#6303) 2019-10-09 17:36:45 -07:00
Pankaj Garg ed85aa43a4
Implement sendmmsg() API (#6304)
* Implement sendmmsg()

* fixes
2019-10-09 17:06:56 -07:00
Sagar Dhawan 4b0250192a Remove remnants of the cuda feature flag (#6298)
automerge
2019-10-09 16:09:36 -07:00
carllin dd66d16fdb
Broadcast final shred for slots that are interrupted (#6269)
* Broadcast final shred for slots that are interrupted
2019-10-09 16:07:18 -07:00
Sagar Dhawan 32312f3c16
Do not retransmit Repair responses (#6284)
* Do not retransmit Repair responses

* Add a test

* Refactor neighboring functionality
2019-10-09 13:11:19 -07:00
Pankaj Garg 2db83e1a21
Remove greedy fetch in shred_fetch stage (#6278)
* Remove greedy fetch in shred_fetch stage

* cleanup
2019-10-09 10:36:05 -07:00
dependabot-preview[bot] cfbfcb5734 Bump dir-diff from 0.3.1 to 0.3.2 (#6265)
Bumps [dir-diff](https://github.com/steveklabnik/dir-diff) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/steveklabnik/dir-diff/releases)
- [Changelog](https://github.com/assert-rs/dir-diff/blob/master/CHANGELOG.md)
- [Commits](https://github.com/steveklabnik/dir-diff/compare/v0.3.1...v0.3.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-09 11:12:15 -06:00
Rob Walker 7cf90766a3
add epoch_schedule sysvar (#6256)
* add epoch_schedule sysvar

* book sheesh!
2019-10-08 22:34:26 -07:00
Justin Starry f2ee01ace3
Fix blocktree processor entry callback test (#6285) 2019-10-08 20:38:05 -04:00
Justin Starry 5e31565574
Expand blocktree processor options (#6248)
* Refactor blocktree processor args and support full leader cache

* Add entry callback option

* Rename num_threads to override_num_threads

* Add test for entry callback

* Refactor cached leader schedule changes

* Add tests for blocktree process options

* Refactor test

* @mvines feedback
2019-10-08 17:58:49 -04:00
Sagar Dhawan 723f9a9b81
Remove unnecessary locking in retransmit stage (#6276)
* Add more detailed metrics to retransmit

* Remove unnecessary locking and add more metrics
2019-10-08 14:41:16 -07:00
sakridge c5e5342325
Rearrange broadcast stats (#6274) 2019-10-08 12:50:59 -07:00
Pankaj Garg 788296047a
Increase batch size for recvmmsg() (#6260)
* Increase batch size for recvmmsg()

* fix tests

* new test
2019-10-08 09:54:49 -07:00
carllin 9dceb8ac74 Broadcast/Shredding Metrics (#6270)
automerge
2019-10-08 01:42:42 -07:00
carllin ac2374e9a1
Shred entries in parallel (#6180)
* Make shredding more parallel

* Fix erasure tests

* Fix replicator test

* Remove UnfinishedSlotInfo
2019-10-08 00:42:51 -07:00
carllin e12c577b16
remove verify_hash_internal_state (#6261) 2019-10-07 16:38:54 -07:00
sakridge ba7efbb136
Retransmit stage optimization, don't copy packets (#6250) 2019-10-07 15:33:22 -07:00
Pankaj Garg 17f169f446
BlobFetchStage cleanup post shred work (#6254) 2019-10-07 11:08:01 -07:00
Michael Vines 4870a2cbac Panic when a snapshot fails to verify 2019-10-07 08:12:18 +09:00
sakridge da7d94d0f0
Retransmit stage benchmark (#6249) 2019-10-06 12:56:17 -07:00
carllin cdef065cca
Broadcast Metrics (#6166)
* Add timing logigng to broadcast

* Shred metrics

* Fixes
2019-10-05 22:51:47 -07:00
sakridge 5ef012b2c1
Tweak debug to remove unreadable datapoints (#6060) 2019-10-04 16:25:22 -07:00
Tyera Eulberg 0c3ff6b75c Cli refactor: vote and storage program functionalities (#6242)
automerge
2019-10-04 14:18:19 -07:00
Michael Vines 7f53737000
Periodically pull from the entrypoint if it's no longer in Crdt (#6240) 2019-10-04 14:18:07 -07:00
Sagar Dhawan 23ea8ae56b
Optimize retransmit stage (#6231)
* Optimize retransmit stage

* Remove comment

* Fix test

* Skip iteration to fixup 0 stakes
2019-10-04 11:52:02 -07:00
dependabot-preview[bot] d759a447be Bump serde_json from 1.0.40 to 1.0.41 (#6226)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.40 to 1.0.41.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.40...v1.0.41)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-03 21:03:05 -06:00
dependabot-preview[bot] 7ac4ce637f Bump reqwest from 0.9.20 to 0.9.21 (#6221)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.9.20 to 0.9.21.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/v0.9.21/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.9.20...v0.9.21)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-10-02 22:52:48 -07:00
Michael Vines f9f5bc2eb5
More clippy 2019-10-02 21:21:07 -07:00
Michael Vines 9fe8c98047
Switch to solana-reed-solomon-erasure temporarily to fix windows build (#6211) 2019-10-02 19:01:55 -07:00