Commit Graph

110 Commits

Author SHA1 Message Date
Pankaj Garg e142aafca9
Use multiple sockets for receiving blobs on validators (#1228)
* Use multiple sockets for receiving blobs on validators

- The blobs that are broadcasted by leader or retransmitted by peer
  validators are received on replicate_port
- Using reuse_addr/reuse_port, multiple sockets can be opened for
  the same port
- This allows the kernel to queue data to user space app on multiple
  socket queues, preventing over-running one queue
- This helps with reducing packets dropped due to queue over-runs

Fixes #1224

* Fixed failing tests
2018-09-14 16:56:06 -07:00
Michael Vines 4196cf43e8 cargo fmt 2018-09-14 16:37:49 -07:00
Rob Walker a8fdb8a5a7 use a single BlobRecycler per fullnode 2018-09-11 16:56:54 -07:00
Greg Fitzgerald fc64e1853c Initialize Window, not SharedWindow
Wrap with Arc<RwLock>> when/if needed, no earlier.
2018-09-10 11:40:26 -06:00
anatoly yakovenko c34d911eaf
Migrate Budget DSL to use the Account state (#979)
* Migrate Budget DSL to use the Account state instead of global bank data structures.

* Serialize Instruction into Transaction::userdata.
* Store the pending set in the Account::userdata
* Enforce the token balance rules on contract execution. This becomes the entry point for generic contracts.
* This pr will have a performance impact on the bank. The next set of changes will fix this by locking each account during multi threaded execution of all the contracts.
* With this change a contract transaction needs to store its state under an address. That address could be the destination of the tokens, or any random address. For the latter, an extra step would be needed to claim the tokens which isn't implemented by budget_dsl at the moment.
* test tracking issue 1157
2018-09-07 20:18:36 -07:00
Greg Fitzgerald 0fa27f65bb Use the default Pubkey formatter instead of debug_id() 2018-09-06 16:31:47 -06:00
Greg Fitzgerald c9a1ac9b8c Don't propogate errors we'll never handle 2018-09-04 06:01:32 -10:00
Rob Walker 176e806d94 rework of netwrk rendezvous
* rename NodeInfo field of Node from "data" to "info"
      (touches a lot of files)

  * update client to use gossip to find leader, a la drone

  * rework multinode scripts
      * move more stuff into rust
      * added usage to all
      * no more rsync unless you're a validator (TODO: whack that, too)
  * fullnode doesn't bail if drone isn't up yet, just keeps trying
  * drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
Rob Walker 63e44dcc35 continue rendezvous refactor for gossip and repair
* remove trailing whitespace in ci/audit.sh

  * code review fixups
     * rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
     * remove out-of-date TODO in localnet-sanity.sh

  * remove features=test and code that was using it (localhost prohibitions in
      crdt) added TODO in crdt.rs, maybe we should boot localhost in production
      networks?

  * boot tvu_window from NodeInfo: instead, send repair requests from the repair
      socket (to gossip on peer) and answer repair requests via the sockaddr
      from the repair request

  * remove various unused pub functions

  * banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
Rob Walker 1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
anatoly yakovenko f7c0d30167
Disallow localhost in deployment (#1064)
* disallow localhost in deployment

* tests

* fmt

* integration tests do not have a flag to check

* fmt
2018-08-25 21:09:18 -07:00
Greg Fitzgerald be58fdf1bb Less constructors 2018-08-23 13:42:54 -07:00
Greg Fitzgerald 9c5e69bf3d Don't offer two ways to specify a leader 2018-08-23 13:42:54 -07:00
Pankaj Garg 24b0031925 Reduce number of nodes in multinode test (#1003) 2018-08-20 13:40:42 -07:00
Tyera Eulberg f73f3941cd Revert ill-advised jsonrpc marker, and handle jsonrpc server close 2018-08-18 12:29:49 -06:00
Tyera Eulberg 1eb8724a89 Disable Rpc module for other tests to prevent port conflicts 2018-08-18 12:29:49 -06:00
Rob Walker 3ae867bdd6 fixups 2018-08-18 02:22:52 -07:00
Pankaj Garg f6f0900506
Large network test to not poll validator for sigs (#998)
- The finality is already reached. The test will check the signature
  in validators once, instead of polling. This will help speed up the test.
2018-08-17 14:38:19 -07:00
Anatoly Yakovenko a284030ecc Account type with state
comments

fixups!

fixups!

fixups for a real Result<> from get_balance()

on 2nd thought, be more rigorous

Merge branch 'rob-solana-accounts_with_state' into accounts_with_state

update

review comments

comments

get rid of option
2018-08-16 14:44:51 -07:00
Pankaj Garg c2983f824e
Refactored large network test to use finality to assert success (#978) 2018-08-15 20:05:43 -07:00
anatoly yakovenko 88d6fea999 Revert "Accounts with state (#954)"
This reverts commit c23fa289c3.
2018-08-15 19:44:39 -07:00
anatoly yakovenko c23fa289c3
Accounts with state (#954)
* Account type with state

* fixed test according to @rob-solana
2018-08-15 14:32:11 -07:00
Pankaj Garg db35f220f7
Run multinode test for enough iterations for a small node count test (#971) 2018-08-15 10:44:14 -07:00
Pankaj Garg 982afa87a6
Retransmit blobs from leader from window (#975)
- Some nodes don't have leader information while leader is broadcasting
  blobs to those nodes. Such blobs are not retransmitted. This change
  rertansmits the blobs once the leader's identity is know.
2018-08-14 21:51:37 -07:00
Greg Fitzgerald b801ca477d Declare fullnode a word 2018-08-13 20:17:16 -06:00
Greg Fitzgerald 2727067b94 Move winow into its own module 2018-08-13 20:17:16 -06:00
Pankaj Garg bf15cad36b
Add get_finality request and use it from multinode test (#941) 2018-08-13 08:55:13 -07:00
anatoly yakovenko f07c038266
Fix bank coalescing (#949)
* fix bank coalescing

* comments

* fix bench

* fix bench

* backout banking stage coalescing

* 120 nodes

* 100
2018-08-12 10:04:21 -07:00
Rob Walker 96d6985895 rework read_ledger, LedgerWriter, and LedgerWindow for recover()
fixes #910
2018-08-10 18:07:23 -07:00
Greg Fitzgerald ad331e6d56 Rename PublicKey type to Pubkey
Recognize pubkey as a noun meaning the public key of a keypair.
2018-08-09 13:41:37 -06:00
Greg Fitzgerald c2bbe4344e Rename KeyPair to Keypair 2018-08-09 13:41:37 -06:00
Greg Fitzgerald 8567253833 Ignore flaky test 2018-08-09 10:15:10 -06:00
Pankaj Garg a6857dbaaa Updated node count to 230. Increased wmem on CI large 2018-08-08 13:13:18 -07:00
pgarg66 75854cc234
Update dynamic network test with more nodes (#904)
- Check for correct OS params in test-large-network.sh
2018-08-08 06:52:57 -07:00
Rob Walker 9783d47fd1 write a "unit" test for WindowLedger (it was working ;)
clear flags on fresh blobs, lest they sometimes impersonate coding blobs...

fix bug: advance *received whether the blob_index is in the window or not,
  failure to do so results in a stalled repair request pipeline
2018-08-08 04:28:09 -07:00
Stephen Akridge 38be61bd22 Check for log level before doing perf counter work
Perf counters, especially when running the dynamic test can cause
functions like crdt::apply_updates to be really slow (>500ms).
2018-08-08 00:16:53 -07:00
Rob Walker fbc754ea25 plug in LedgerWindow
fixes #872
2018-08-07 17:27:53 -07:00
pgarg66 16772d3d51
Coalesce multiple blobs if received close together (#869)
- This helps reduce unnecessary growth of window if small blobs
  are received in small space of time
2018-08-07 10:29:57 -07:00
Kazuyoshi Kato 9c1b6288a4 Use ? instead of unwrap()
This change addresses #833, while there are still some unwrap() though.
2018-08-07 08:10:22 -07:00
Rob Walker 3beb38ac8a /tmp/farf no good on multi-user machine 2018-08-06 14:53:40 -07:00
Rob Walker 039ed01abf on 2nd thought: do not copy_ledger() for this test 2018-08-06 08:51:41 -07:00
Rob Walker ead0eb2754 move copy_ledger() back into ledger.rs
Don't recover() for copy(), as copy() is already tolerant of things
    recover() guards against.  Note: recover() is problematic if the ledger is
    "live", i.e. is currently being written to.
2018-08-06 08:51:41 -07:00
Rob Walker c3db2df7eb tweak random access ledger
* add recover_ledger() to deal with expected common ledger corruptions
  * add verify_ledger() for future use cases (ledger-tool)
  * increase ledger testing
  * allow replicate stage to run without a ledger
  * ledger-tool to output valid json
2018-08-06 08:51:41 -07:00
Rob Walker 5193819d8e Revert "Revert "plug in new ledger""
This reverts commit 57e928d1d0.
2018-08-06 08:51:41 -07:00
Rob Walker f2b1a04bca cargo fmt fixups 2018-08-03 11:59:25 -07:00
Rob Walker 57e928d1d0 Revert "plug in new ledger"
This reverts commit 46d9ba5ca0.
2018-08-03 10:24:51 -07:00
Rob Walker 46d9ba5ca0 plug in new ledger 2018-08-02 21:50:47 -07:00
Tyera Eulberg 7ff721e563 Replace pub field with AsRef impl 2018-08-01 16:15:14 -07:00
Tyera Eulberg fda3b9bbd4 Use new PublicKey format instead of hex 2018-08-01 16:15:14 -07:00
Tyera Eulberg cf70e5ff2f Handle wrapped PublicKey struct 2018-08-01 16:15:14 -07:00