Commit Graph

46 Commits

Author SHA1 Message Date
Pankaj Garg 0f4fd8367d
Add counters for channel pressure and time spent in TPU pipeline (#1324)
* Add counters for channel pressure and time spent in TPU pipeline

* Fixed failing tests

* Fix rust format issue
2018-09-24 17:13:49 -07:00
Rob Walker be31da3dce
lastidnotfound step 2: (#1300)
lastidnotfound step 2:
  * move "record stage", aka poh_service into banking stage
  * remove Entry.has_more, is incompatible with leader rotation
  * rewrite entry_next_hash in terms of Poh
  * simplify and unify transaction hashing (no embedded nulls)
  * register_last_entry from banking stage, fixes #1171 (w00t!)
  * new PoH doesn't generate empty ledger entries, so some fixes necessary in 
         multinode tests that rely on that (e.g. giving validators airdrops)
  * make window repair less patient, if we've been waiting for an answer, 
          don't be shy about most recent blobs
   * delete recorder and record stage
   * make more verbost  thin_client error reporting
   * more tracing in window (sigh)
2018-09-21 21:01:13 -07:00
carllin c50ac96f75
Moved deserialization of blobs to entries from replicate_stage to window_service (#1287) 2018-09-21 16:01:24 -07:00
Michael Vines beeb09646a suppress warning: unused variable: recycler 2018-09-20 19:46:20 -07:00
Greg Fitzgerald 6073cd57fa Boot Recycler::recycle() 2018-09-20 17:08:51 -06:00
Anatoly Yakovenko 431692d9d0 Use a Drop trait to keep track of lifetimes for recycled objects.
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
2018-09-19 16:59:42 -06:00
Jay Kickliter fd51599fa8 Replace replace(..., None) with take()
This is strictly for simplicity, since Option::take() is imlemented with replace().
2018-09-15 05:12:09 -06:00
Michael Vines 4196cf43e8 cargo fmt 2018-09-14 16:37:49 -07:00
Greg Fitzgerald a093d5c809 Fix erasure build 2018-09-10 11:40:26 -06:00
Greg Fitzgerald fc64e1853c Initialize Window, not SharedWindow
Wrap with Arc<RwLock>> when/if needed, no earlier.
2018-09-10 11:40:26 -06:00
Greg Fitzgerald 7f669094de Split window into two modules 2018-09-10 11:40:26 -06:00
Greg Fitzgerald 5025d89c88 Inline window method implementations 2018-09-10 11:40:26 -06:00
Greg Fitzgerald 2b44c4504a Use WindowUtil for more idiomatic code 2018-09-10 11:40:26 -06:00
Greg Fitzgerald d2c9beb843 Add a trait to pretend Window is an object 2018-09-10 11:40:26 -06:00
Greg Fitzgerald 71a617b4dc Fix erasure build 2018-09-07 13:18:19 -06:00
Greg Fitzgerald e9f44b6661 window -> window_service 2018-09-07 13:18:19 -06:00
Greg Fitzgerald 9693de1867 Reposition parameters 2018-09-07 13:18:19 -06:00
Greg Fitzgerald f7ea95aed1 Hoist lock, reposition parameters 2018-09-07 13:18:19 -06:00
Greg Fitzgerald f07ce59be8 Toggle parameters 2018-09-07 13:18:19 -06:00
Greg Fitzgerald da423b6cf0 Hoist read lock 2018-09-07 13:18:19 -06:00
Greg Fitzgerald d5f60b68e4 Hoist window write lock 2018-09-07 13:18:19 -06:00
Greg Fitzgerald 78b3a8f7f9 Hoist repair_window() branches
This probably would have been done if repair_window() was unit-tested.
2018-09-07 13:18:19 -06:00
Greg Fitzgerald d77699c126 Do the easy check first
All functions above operate on immutable values, so this shouldn't
change functionality, but no repair_window() tests to be certain.hI
2018-09-07 13:18:19 -06:00
Greg Fitzgerald 09ba0dae15 Remove redundant clone() 2018-09-07 13:18:19 -06:00
Greg Fitzgerald a5c7575207 Rewrite find_next_missing, call it clear_slots 2018-09-07 13:18:19 -06:00
Greg Fitzgerald 50f040530b Remove redundant cast 2018-09-07 13:18:19 -06:00
Greg Fitzgerald 7f99c90539 Simplify using early return and Result::ok() 2018-09-07 13:18:19 -06:00
Greg Fitzgerald d8564b725c Don't reference window to get each slot 2018-09-07 13:18:19 -06:00
Greg Fitzgerald e4de25442a Hoist write lock
It needed to be passed the lock before, because it contained a
branch where one side didn't require locking. Now that that
defensive programming was hoisted, we can hoist the write lock
as well, leaving a simpler function for unit testing.
2018-09-07 13:18:19 -06:00
Greg Fitzgerald 3b2ea8fd40 Hoist untested branch in window
If there were unit tests for this function, the author would have
written it this way to make their own life easier.
2018-09-07 13:18:19 -06:00
Greg Fitzgerald 0fa27f65bb Use the default Pubkey formatter instead of debug_id() 2018-09-06 16:31:47 -06:00
Greg Fitzgerald 68c72d6f34 Fix nightly build 2018-09-04 20:56:40 -06:00
Rob Walker 226d3b9471
Trace recycle() calls (#968)
* trace recycle() calls fixes #810
2018-09-05 05:07:02 +09:00
Greg Fitzgerald 2aa72cc72e Return a Vec from to_blobs() instead of using a mut parameter 2018-09-04 07:50:23 -10:00
Greg Fitzgerald 8cc030ef84 Use Vec instead of VecDeque for SharedBlobs 2018-09-04 07:50:23 -10:00
Greg Fitzgerald 4e2deaa33b Less mut 2018-09-04 06:01:32 -10:00
Greg Fitzgerald c9a1ac9b8c Don't propogate errors we'll never handle 2018-09-04 06:01:32 -10:00
anatoly yakovenko fb4b33b81b
make the repair_backoff test more robust (#1095)
* more the repair_backoff test more robust

* fix names and magic numbers
2018-08-31 12:40:56 -10:00
Rob Walker 176e806d94 rework of netwrk rendezvous
* rename NodeInfo field of Node from "data" to "info"
      (touches a lot of files)

  * update client to use gossip to find leader, a la drone

  * rework multinode scripts
      * move more stuff into rust
      * added usage to all
      * no more rsync unless you're a validator (TODO: whack that, too)
  * fullnode doesn't bail if drone isn't up yet, just keeps trying
  * drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
Rob Walker 63e44dcc35 continue rendezvous refactor for gossip and repair
* remove trailing whitespace in ci/audit.sh

  * code review fixups
     * rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
     * remove out-of-date TODO in localnet-sanity.sh

  * remove features=test and code that was using it (localhost prohibitions in
      crdt) added TODO in crdt.rs, maybe we should boot localhost in production
      networks?

  * boot tvu_window from NodeInfo: instead, send repair requests from the repair
      socket (to gossip on peer) and answer repair requests via the sockaddr
      from the repair request

  * remove various unused pub functions

  * banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
Rob Walker 1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
anatoly yakovenko 48762834d9
Randomize repair requests (#1059)
* randomize packet repair requests

* exponential random repair requests

* use gen_range to get a uniform distribution
2018-08-27 07:05:48 -07:00
sakridge 6147e54686
Cap repair requests timeout (#958) 2018-08-23 15:30:21 -07:00
Pankaj Garg 982afa87a6
Retransmit blobs from leader from window (#975)
- Some nodes don't have leader information while leader is broadcasting
  blobs to those nodes. Such blobs are not retransmitted. This change
  rertansmits the blobs once the leader's identity is know.
2018-08-14 21:51:37 -07:00
Greg Fitzgerald dc2a840985 Move FullNode::new_window into window module 2018-08-13 20:17:16 -06:00
Greg Fitzgerald 2727067b94 Move winow into its own module 2018-08-13 20:17:16 -06:00