Commit Graph

183 Commits

Author SHA1 Message Date
Greg Fitzgerald 7292ece7ad Free up term instruction for new multi-instruction feature 2018-09-26 14:17:15 -06:00
carllin e7383a7e66
Validator to leader (#1303)
* Add check in window_service to exit in checks for leader rotation, and propagate that service exit up to fullnode

* Added logic to shutdown Tvu once ReplicateStage finishes

* Added test for successfully shutting down validator and starting up leader

* Add test for leader validator interaction

* fix streamer to check for exit signal before checking socket again to prevent busy leaders from never returning

* PR comments - Rewrite make_consecutive_blobs() function, revert genesis function change
2018-09-25 15:41:29 -07:00
Greg Fitzgerald 95677a81c5 Pacify clippy 2018-09-24 13:36:31 -06:00
sakridge a9355c33b2
Placeholder storage contract and replicator client (#1286)
* Add hooks for executing the storage contract

* Add store_ledger stage
  Similar to replicate_stage but no voting/banking stuff, just convert
  blobs to entries and write the ledger out

* Add storage_addr to tests and add new NodeInfo constructor
  to reduce duplication...
2018-09-21 15:32:15 -07:00
Anatoly Yakovenko 431692d9d0 Use a Drop trait to keep track of lifetimes for recycled objects.
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
2018-09-19 16:59:42 -06:00
Carl b10de40506 Made LEADER_ROTATION_INTERVAL settable so that integration tests don't time out 2018-09-19 10:48:05 -06:00
Carl 2030dfa435 Implement PR comments, tidy up 2018-09-19 10:48:05 -06:00
Carl f28ba3937b Added check in write stage to exit when scheduled entry_height for leader rotation is detected 2018-09-19 10:48:05 -06:00
anatoly yakovenko 6ec0e42220
budget as separate contract and system call contract (#1189)
* budget and system contracts and verification

* contract check_id methods
* system call contract
* verify contract execution rules
* move system into its own file
* allocate before transfer for budget
* store error in budget context
* budget contract and tests without bank
* moved budget of of bank
2018-09-17 13:36:31 -07:00
Pankaj Garg e142aafca9
Use multiple sockets for receiving blobs on validators (#1228)
* Use multiple sockets for receiving blobs on validators

- The blobs that are broadcasted by leader or retransmitted by peer
  validators are received on replicate_port
- Using reuse_addr/reuse_port, multiple sockets can be opened for
  the same port
- This allows the kernel to queue data to user space app on multiple
  socket queues, preventing over-running one queue
- This helps with reducing packets dropped due to queue over-runs

Fixes #1224

* Fixed failing tests
2018-09-14 16:56:06 -07:00
Michael Vines 4196cf43e8 cargo fmt 2018-09-14 16:37:49 -07:00
Pankaj Garg 1d7e87d430 Increase number of sockets for transaction processing 2018-09-13 14:22:07 -06:00
Rob Walker a91b785ba5 move fullnode trace generation into crdt 2018-09-10 13:47:57 -07:00
Greg Fitzgerald fc64e1853c Initialize Window, not SharedWindow
Wrap with Arc<RwLock>> when/if needed, no earlier.
2018-09-10 11:40:26 -06:00
Rob Walker 97c3125a78
improve localnet-sanity's robustness (#1160)
* fix poll_gossip_for_leader() loop to actually wait
         for 30 seconds
    * reduce reuseaddr use to only when necessary,
         try to avoid already bound sockets
    * move nat.rs to netutil.rs
    * add gossip tracing to thin_client and bench-tps
2018-09-09 04:50:43 +09:00
Greg Fitzgerald a722802c95 Window write lock to read lock 2018-09-07 13:18:19 -06:00
Greg Fitzgerald 0fa27f65bb Use the default Pubkey formatter instead of debug_id() 2018-09-06 16:31:47 -06:00
Pankaj Garg 05460eec0d
Open multiple sockets for transaction UDP port (#1128)
* Reuse UDP port and open multiple sockets for transaction address

* Fixed failing crdt tests

* Add tests for reusing UDP ports

* Address review comments

* Updated bench-streamer to use multiple receive sockets

* Fix minimum number of recv sockets for bench-streamer

* Address review comments

Fixes #1132

* Moved bind_to function to nat.rs
2018-09-06 14:13:40 -07:00
Rob Walker 226d3b9471
Trace recycle() calls (#968)
* trace recycle() calls fixes #810
2018-09-05 05:07:02 +09:00
Greg Fitzgerald 8cc030ef84 Use Vec instead of VecDeque for SharedBlobs 2018-09-04 07:50:23 -10:00
Greg Fitzgerald ab7a2960b1 Don't use product name in solana library 2018-09-04 06:01:32 -10:00
Rob Walker 176e806d94 rework of netwrk rendezvous
* rename NodeInfo field of Node from "data" to "info"
      (touches a lot of files)

  * update client to use gossip to find leader, a la drone

  * rework multinode scripts
      * move more stuff into rust
      * added usage to all
      * no more rsync unless you're a validator (TODO: whack that, too)
  * fullnode doesn't bail if drone isn't up yet, just keeps trying
  * drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
Rob Walker ba27596076 fixups 2018-08-31 23:21:07 +09:00
Rob Walker 63e44dcc35 continue rendezvous refactor for gossip and repair
* remove trailing whitespace in ci/audit.sh

  * code review fixups
     * rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
     * remove out-of-date TODO in localnet-sanity.sh

  * remove features=test and code that was using it (localhost prohibitions in
      crdt) added TODO in crdt.rs, maybe we should boot localhost in production
      networks?

  * boot tvu_window from NodeInfo: instead, send repair requests from the repair
      socket (to gossip on peer) and answer repair requests via the sockaddr
      from the repair request

  * remove various unused pub functions

  * banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
Rob Walker 1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
anatoly yakovenko 738247ad44
advertise valid gossip address in drone and wallet (#1066)
* advertize valid gossip address in drone and wallet

get rid of asserts

check for valid ip address

check for valid address

ip address

* tests

* cleanup

* cleanup

* print error

* bump

* disable tests

* disable nightly
2018-08-26 11:36:27 -07:00
anatoly yakovenko 5b0bb7e607
Skip invalid nodes for finality (#1068)
* skip invalid nodes for finality

* check valid last_ids only

* fixup!

* fixup!
2018-08-25 23:12:41 -07:00
anatoly yakovenko f7c0d30167
Disallow localhost in deployment (#1064)
* disallow localhost in deployment

* tests

* fmt

* integration tests do not have a flag to check

* fmt
2018-08-25 21:09:18 -07:00
Anatoly Yakovenko 8e98c7c9d6 fix purge test 2018-08-25 19:56:09 -07:00
Pankaj Garg d3fac8a06f
Dynamically bind to available UDP ports in Fullnode (#920)
* Dynamically bind to available UDP ports in Fullnode

* Added tests for dynamic port binding

- Also removed hard coding of port range from CRDT
2018-08-25 10:24:16 -07:00
Rob Walker f784500fbb fixups
fixes #907
2018-08-13 21:07:26 -07:00
Greg Fitzgerald 2727067b94 Move winow into its own module 2018-08-13 20:17:16 -06:00
Rob Walker e8da903c6c move tmp_ledger back to target dir 2018-08-13 16:52:53 -07:00
Rob Walker 96d6985895 rework read_ledger, LedgerWriter, and LedgerWindow for recover()
fixes #910
2018-08-10 18:07:23 -07:00
Greg Fitzgerald bca2294655 cargo fmt 2018-08-09 13:41:37 -06:00
Greg Fitzgerald 4a980568ac Rename sig variables to signature
We'll avoid introducing three-letter terms to free up the namespace
for three-letter acronyms.

But recognize the term "sigverify", a verb, to verify a digital
signature.
2018-08-09 13:41:37 -06:00
Greg Fitzgerald 9d436fc5f8 Rename pk variables to pubkey 2018-08-09 13:41:37 -06:00
Greg Fitzgerald ad331e6d56 Rename PublicKey type to Pubkey
Recognize pubkey as a noun meaning the public key of a keypair.
2018-08-09 13:41:37 -06:00
Greg Fitzgerald b2067d2721 Rename kp variables to keypair 2018-08-09 13:41:37 -06:00
Greg Fitzgerald c2bbe4344e Rename KeyPair to Keypair 2018-08-09 13:41:37 -06:00
Rob Walker ce8a4fa831 allow received to outpace window, we're already constraining repair
correctly identify sender in ledger_window repair responses, enabling re-transmission
2018-08-08 15:10:44 -07:00
Stephen Akridge 8331aab26a Enable Crdt debug messages to debug validators 2018-08-08 14:22:20 -07:00
Rob Walker 9783d47fd1 write a "unit" test for WindowLedger (it was working ;)
clear flags on fresh blobs, lest they sometimes impersonate coding blobs...

fix bug: advance *received whether the blob_index is in the window or not,
  failure to do so results in a stalled repair request pipeline
2018-08-08 04:28:09 -07:00
Stephen Akridge 38be61bd22 Check for log level before doing perf counter work
Perf counters, especially when running the dynamic test can cause
functions like crdt::apply_updates to be really slow (>500ms).
2018-08-08 00:16:53 -07:00
Rob Walker c64e2acf8b set destination address when for ledger window repair responses 2018-08-07 23:31:01 -07:00
Rob Walker fbc754ea25 plug in LedgerWindow
fixes #872
2018-08-07 17:27:53 -07:00
pgarg66 1b6d472cb2
Fixed counters for coalescing and broadcast index (#900) 2018-08-07 16:46:48 -07:00
pgarg66 64d6d3015a
Counters for broadcasted blob idx and coalesced packets (#897) 2018-08-07 14:54:26 -07:00
Michael Vines 8cbaa19d2e Report the address that failed to bind 2018-08-06 11:21:25 -07:00
Rob Walker f2b1a04bca cargo fmt fixups 2018-08-03 11:59:25 -07:00