Commit Graph

42 Commits

Author SHA1 Message Date
Sagar Dhawan ffdc1814c6 Add counters for gossip verification failures (#2094) 2018-12-17 20:12:50 -08:00
Michael Vines b7144560c9 Include port number when gossip bind_to fails 2018-12-17 09:31:31 -08:00
Michael Vines eedc8c7812 Move src/netutil.rs into its own crate 2018-12-14 20:13:34 -08:00
carllin 9ef5e51c0f
Cleanup slot remnants in db_ledger (#2153)
* Cleanup slot remnants in db_ledger
2018-12-14 17:05:41 -08:00
Sathish 8ee0e9632c
Switch to using hashbrown version of HashMap and (#2158)
HashSet for improved performance and memory usage
2018-12-14 15:10:10 -08:00
Michael Vines 6ac466c0a4 Move src/logger.rs into logger/ crate to unify logging across the workspace 2018-12-14 13:10:43 -08:00
Carl a05a378db4 cleanup 2018-12-12 19:12:51 -08:00
Carl 245362db96 Make a dummy version of serving repairs from db_ledger 2018-12-12 19:12:51 -08:00
Mark b1b190b80d Fix too many args in Tvu::new (#2114)
* Reduce args in Tvu::new under to 8

Now pass in sockets through a the crate::tvu::Sockets struct

Move ClusterInfo.keypair to pub(crate) in order to remove redundant
signing keypair parameter

* remove commented code
2018-12-12 18:57:48 -08:00
Sagar Dhawan 4788a4f775
Correctly describe repair and retransmit peers (#2110) 2018-12-11 15:51:47 -08:00
Pankaj Garg 9243bc58db
Metrics for window repair (#2106)
* Metrics for window repair

- Also increase max repair length

* fix vote counters, and add repair window graph

* update per node graphs

* revert max repair length change
2018-12-11 15:43:41 -08:00
Rob Walker aeee25e703
add tick_height to Entry to be able to repair by period, chain forks of Entries, etc. (#2096) 2018-12-10 20:03:04 -08:00
Greg Fitzgerald 5e703dc70a Free up the term 'replicate' for exclusive use in replicator
Also, align Sockets field names with ContactInfo.
2018-12-10 15:26:43 -07:00
Greg Fitzgerald ec5a8141eb cargo fix --edition 2018-12-08 23:19:55 -07:00
Greg Fitzgerald 0a83b17cdd
Upgrade to Rust 1.31.0 (#2052)
* Upgrade to Rust 1.31.0
* Upgrade nightly
* Fix all clippy warnings
* Revert relaxed version check and update
2018-12-07 20:01:28 -07:00
Greg Fitzgerald 97b1156a7a Rename Ncp to GossipService
And BroadcastStage to BroadcastService since it's not included in the
TPU pipeline.
2018-12-06 15:48:19 -07:00
Michael Vines c99f93e40a Remove signature.rs indirection 2018-12-03 13:50:00 -08:00
Sagar Dhawan b21facab7b
Add metrics for prune messages (#1981) 2018-12-01 14:05:40 -08:00
Sagar Dhawan 34c3a0cc1f
Add signature verification to gossip (#1937) 2018-12-01 12:00:30 -08:00
Sathish 9f0b06bb86 Filter out leader node while retransmitting blobs (#1894) 2018-11-24 20:33:49 -08:00
Rob Walker 3d113611cc
remove Result<> return from ClusterInfo::new() (#1869)
strip Result<> for ClusterInfo::new()
2018-11-19 11:25:14 -08:00
Sagar Dhawan 07667771ef
Fix Gossip Pushes going to invalid addresses (#1858) 2018-11-17 19:57:28 -08:00
Michael Vines d96a6b42a5 Move drone into its own crate 2018-11-16 20:42:21 -08:00
Michael Vines 6ac5700f2e Move metrics into its own crate 2018-11-16 15:10:07 -08:00
anatoly yakovenko a41254e18c
Add scalable gossip library (#1546)
* Cluster Replicated Data Store

Separate the data storage and merge strategy from the network IO boundary.
Implement an eager push overlay for transporting recent messages.

Simulation shows fast convergence with 20k nodes.
2018-11-15 13:23:26 -08:00
carllin 6335be803c
Broadcast last tick before leader rotation (#1766)
* Broadcast last tick before leader rotation to everybody on network

* Add test

* Refactor broadcast
2018-11-13 02:21:37 -08:00
Rob Walker fa72160c95
add last_id to Entry, PohEntry (#1783)
add prev_id to Entry, PohEntry
2018-11-12 17:03:23 -08:00
Rob Walker 6c10458b5b
leader slots in Blobs (#1732)
* add leader slot to Blobs
* remove get_X() methods in favor of X() methods for Blob
* add slot to get_scheduled_leader()
2018-11-07 13:18:14 -08:00
Rob Walker 9b43b00d5c
remove tick_count, leader_scheduler, from broadcast code (#1725) 2018-11-06 13:17:41 -08:00
Tyera Eulberg f683817b48 Remove RPU; replace with RPC 2018-11-05 20:30:47 -07:00
Michael Vines a264f8fa9b Fix |cargo test| 2018-11-02 19:04:59 -07:00
carllin 0636399b7a
Compute finality computation in new ComputeLeaderFinalityService (#1652)
* Move finality computation into a service run from the banking stage, ComputeLeaderFinalityService

* Change last ids nth to tick height, remove separate tick height from bank
2018-11-02 15:49:14 -07:00
carllin 298bd6479a
Add first leader to genesis (#1681)
* Add first leader to genesis entries, consume in genesis.sh

* Set bootstrap leader in the bank on startup, remove instantiation of bootstrap leader from bin/fullnode

* Remove need to initialize bootstrap leader in leader_scheduler, now can be read from genesis entries

* Add separate interface new_with_leader() in mint for creating genesis leader entries
2018-11-02 14:32:05 -07:00
carllin 47f1fa3f2e
Remove purging of leader id from cluster info (#1642) 2018-10-31 12:30:48 -07:00
carllin f6c8e1a4bf
Vote contract (#1552)
* Add Vote Contract

* Move ownership of LeaderScheduler from Fullnode to the bank

* Modified ReplicateStage to consume leader information from bank

* Restart RPC Services in Leader To Validator Transition

* Make VoteContract Context Free

* Remove voting from ClusterInfo and Tpu

* Remove dependency on ActiveValidators in LeaderScheduler

* Switch VoteContract to have two steps 1) Register 2) Vote. Change thin client to create + register a voting account on fullnode startup

* Remove check in leader_to_validator transition for unique references to bank, b/c jsonrpc service and rpcpubsub hold references through jsonhttpserver
2018-10-25 16:58:40 -07:00
Michael Vines e47fcb196b s/solana_program_interface/solana[_-]sdk/g 2018-10-25 12:31:45 -07:00
Pankaj Garg 8d9912b4e2
Move ledger write to its own stage (#1577)
* Move ledger write to its own stage

- Also, rename write_stage to leader_vote_stage, as write functionality
  is moved to a different stage

* Address review comments

* Fix leader rotation test failure

* address review comments
2018-10-23 14:42:48 -07:00
carllin 0bd1412562
Switch leader scheduler to use PoH ticks instead of Entry height (#1519)
* Add PoH height to process_ledger()

* Moved broadcast_stage Leader Scheduling logic to use Poh height instead of entry_height

* Moved LeaderScheduler logic to PoH in ReplicateStage

* Fix Leader scheduling tests to use PoH instead of entry height

* Change is_leader detection in repair() to use PoH instead of entry height

* Add tests to LeaderScheduler for new functionality

* fix Entry::new and genesis block PoH counts

* Moved LeaderScheduler to PoH ticks

* Cleanup to resolve PR comments
2018-10-18 22:57:48 -07:00
Rob Walker d0f43e9934 consolidate tmp ledgers 2018-10-18 08:45:31 -06:00
anatoly yakovenko 6aaa350145
effeciently pack gossip responsens and only respond up to max size. (#1493) 2018-10-14 06:45:02 -07:00
carllin 9931ac9780
Leader scheduler plumbing (#1440)
* Added LeaderScheduler module and tests

* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage

* Add LeaderScheduler plumbing for Tvu, window, and tests

* Fix bank and switch tests to use new LeaderScheduler

* move leader rotation check from window service to replicate stage

* Add replicate_stage leader rotation exit test

* removed leader scheduler from the window service and associated modules/tests

* Corrected is_leader calculation in repair() function in window.rs

* Integrate LeaderScheduler with write_stage for leader to validator transitions

* Integrated LeaderScheduler with BroadcastStage

* Removed gossip leader rotation from crdt

* Add multi validator, leader test

* Comments and cleanup

* Remove unneeded checks from broadcast stage

* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role

* Set new leader in validator -> validator transitions

* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail

* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops

* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
2018-10-10 16:49:41 -07:00
Greg Fitzgerald 95701114e3 Crdt -> ClusterInfo 2018-10-09 03:49:39 -06:00