Commit Graph

122 Commits

Author SHA1 Message Date
Rob Walker d0f43e9934 consolidate tmp ledgers 2018-10-18 08:45:31 -06:00
Rob Walker 912eb5e8e9
remove bank.is_leader, dead code (#1516) 2018-10-16 15:26:44 -07:00
Tyera Eulberg 1b2e9122d5 Pubsub listen on random open port when rpc does (quiet some test errors) 2018-10-16 00:11:26 -06:00
Michael Vines 537436bd5e RPC PubSub now uses a well-known socket 2018-10-16 00:11:26 -06:00
Tyera Eulberg 1515bba9c6
Use cluster_info in rpc to get current leader addresses (#1480) 2018-10-12 14:25:56 -06:00
carllin 47f69f2d24
1) Switch broken tests to generate an empty tick in their ledgers to use as last_id, 2) Fix bug where PoH generator in BankingStage did not referenced the last tick instead of the last entry on startup, causing ledger verification to fail on the new tick added by the PoH generator (#1479) 2018-10-12 00:39:10 -07:00
Stephen Akridge 9dd4dc2088 Mark failing tests as ignore 2018-10-11 15:32:36 -07:00
carllin 9931ac9780
Leader scheduler plumbing (#1440)
* Added LeaderScheduler module and tests

* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage

* Add LeaderScheduler plumbing for Tvu, window, and tests

* Fix bank and switch tests to use new LeaderScheduler

* move leader rotation check from window service to replicate stage

* Add replicate_stage leader rotation exit test

* removed leader scheduler from the window service and associated modules/tests

* Corrected is_leader calculation in repair() function in window.rs

* Integrate LeaderScheduler with write_stage for leader to validator transitions

* Integrated LeaderScheduler with BroadcastStage

* Removed gossip leader rotation from crdt

* Add multi validator, leader test

* Comments and cleanup

* Remove unneeded checks from broadcast stage

* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role

* Set new leader in validator -> validator transitions

* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail

* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops

* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
2018-10-10 16:49:41 -07:00
Greg Fitzgerald 95701114e3 Crdt -> ClusterInfo 2018-10-09 03:49:39 -06:00
Carl 5491422b12 Fix validator_to_leader_transition test to not start up tpu after shutting down tvu, as the tpu now outputs ticks that will mess up the verification check 2018-10-03 21:58:29 -07:00
Pankaj Garg e10574c64d Remove recycler and it's usage
- The memory usage due to recycler was high, and incrementing with
  time.
2018-09-27 10:42:37 -06:00
jackcmay 9c47e022dc
break dependency of programs on solana core (#1371)
* break dependency of programs on Solana core
2018-09-27 07:49:26 -07:00
Greg Fitzgerald b7ae5b712a Move Pubkey into its own module 2018-09-26 20:40:40 -06:00
Anatoly Yakovenko 93c4f6c9b8 Synchronize PoH, bank last_id queue and ledger entry channel.
PoH, bank's last_id queue and the Entry channel need to have a synchronized order of ids.
2018-09-26 16:19:03 -07:00
carllin e7383a7e66
Validator to leader (#1303)
* Add check in window_service to exit in checks for leader rotation, and propagate that service exit up to fullnode

* Added logic to shutdown Tvu once ReplicateStage finishes

* Added test for successfully shutting down validator and starting up leader

* Add test for leader validator interaction

* fix streamer to check for exit signal before checking socket again to prevent busy leaders from never returning

* PR comments - Rewrite make_consecutive_blobs() function, revert genesis function change
2018-09-25 15:41:29 -07:00
Greg Fitzgerald 95677a81c5 Pacify clippy 2018-09-24 13:36:31 -06:00
Jack May a1f01fb8f8 revert is_some to not is_none, causes test failure 2018-09-23 17:09:18 -06:00
Jack May b2be0e2e5e fix clippy warning 2018-09-23 17:09:18 -06:00
Jack May 1a45587c08 fix clippy warnings 2018-09-23 17:09:18 -06:00
Tyera Eulberg ad4fef4f09 Doc for rpc_port configuration 2018-09-21 18:06:20 -06:00
Tyera Eulberg e3b3701e13 Add RPC port option to fullnode 2018-09-21 18:06:20 -06:00
Carl 55126f5fb6 Marked Tvu functionality in Fullnode as unused for now 2018-09-19 16:05:31 -07:00
Anatoly Yakovenko 431692d9d0 Use a Drop trait to keep track of lifetimes for recycled objects.
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
2018-09-19 16:59:42 -06:00
Carl cd2d50e06c Changed transition to restart Rpu rather than modify bank to prevent lock contention 2018-09-19 10:48:05 -06:00
Carl b10de40506 Made LEADER_ROTATION_INTERVAL settable so that integration tests don't time out 2018-09-19 10:48:05 -06:00
Carl 2030dfa435 Implement PR comments, tidy up 2018-09-19 10:48:05 -06:00
Carl bfe64f5f6e Added integration test for transitioning leader to validator to see that tpu pipeline can exit and restart a tvu. Fixed Tpu and broadcast stage so that exiting later stages in the pipeline also causes earlier stages to exit. 2018-09-19 10:48:05 -06:00
Carl 6d27751365 give fullnode ownership of state needed to dynamically start up a tpu or tvu for role transition 2018-09-19 10:48:05 -06:00
Carl 1fb1c0a681 added jointypes to the stages in the tpu involved in leader rotation 2018-09-19 10:48:05 -06:00
Carl 062f654fe0 formatted code 2018-09-19 10:48:05 -06:00
Carl 98b47d2540 Added check in broadcast stage to exit after transmitting last blob before leader rotation. Also added tests 2018-09-19 10:48:05 -06:00
Carl f28ba3937b Added check in write stage to exit when scheduled entry_height for leader rotation is detected 2018-09-19 10:48:05 -06:00
Carl 91cf14e641 Rewrote service trait join() method to allow thread join handles to return values other than () 2018-09-19 10:48:05 -06:00
Pankaj Garg 5310b6e5a2
Move entry->blob creation out of write stage (#1257)
- The write stage will output vector of entries
- Broadcast stage will create blobs out of the entries
- Helps reduce MIPS requirements for write stage
2018-09-18 13:49:10 -07:00
Anatoly Yakovenko fd36954477 clippy 2018-09-15 05:12:53 -06:00
Michael Vines 4196cf43e8 cargo fmt 2018-09-14 16:37:49 -07:00
sakridge 4dc30ea104
Add recycler stats (#1187) 2018-09-13 14:49:48 -07:00
carllin 8706774ea7
Rewrote service trait join() method to allow thread join handles to return values other than () (#1213) 2018-09-13 14:00:17 -07:00
Rob Walker b52230097e groom Fullnode's new_with_bank() to match new() more 2018-09-12 09:24:42 -07:00
Rob Walker a8fdb8a5a7 use a single BlobRecycler per fullnode 2018-09-11 16:56:54 -07:00
Rob Walker b313b7f6f9 Revert "move rpc_server to drop() semantics instead of having its own thread"
This reverts commit 40aa0654fa.
2018-09-10 22:48:33 -07:00
Rob Walker 40aa0654fa move rpc_server to drop() semantics instead of having its own thread 2018-09-10 20:25:53 -07:00
Greg Fitzgerald fc64e1853c Initialize Window, not SharedWindow
Wrap with Arc<RwLock>> when/if needed, no earlier.
2018-09-10 11:40:26 -06:00
sakridge d82ec2634c
Fix is_leader boolean (#1115)
A node is the leader if the address is none
2018-09-04 13:38:24 -07:00
Greg Fitzgerald c9a1ac9b8c Don't propogate errors we'll never handle 2018-09-04 06:01:32 -10:00
Rob Walker 176e806d94 rework of netwrk rendezvous
* rename NodeInfo field of Node from "data" to "info"
      (touches a lot of files)

  * update client to use gossip to find leader, a la drone

  * rework multinode scripts
      * move more stuff into rust
      * added usage to all
      * no more rsync unless you're a validator (TODO: whack that, too)
  * fullnode doesn't bail if drone isn't up yet, just keeps trying
  * drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
Rob Walker 1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
Greg Fitzgerald 4ab58f069a Add back JsonRpcService changes 2018-08-23 13:42:54 -07:00
Greg Fitzgerald 85f96d926a Pacify clippy 2018-08-23 13:42:54 -07:00
Greg Fitzgerald 816de4f8ec Hoist shared code between leaders and validators 2018-08-23 13:42:54 -07:00