Commit Graph

1694 Commits

Author SHA1 Message Date
Tyera Eulberg c196952afd Flesh out Wallet CLI & add placeholder WalletCommands 2018-09-25 16:38:51 -07:00
carllin e7383a7e66
Validator to leader (#1303)
* Add check in window_service to exit in checks for leader rotation, and propagate that service exit up to fullnode

* Added logic to shutdown Tvu once ReplicateStage finishes

* Added test for successfully shutting down validator and starting up leader

* Add test for leader validator interaction

* fix streamer to check for exit signal before checking socket again to prevent busy leaders from never returning

* PR comments - Rewrite make_consecutive_blobs() function, revert genesis function change
2018-09-25 15:41:29 -07:00
Rob Walker 8a7545197f
move tick generation back to banking_stage, add unit tests (#1332)
* move tick generation back to banking_stage, add unit tests

fixes #1217

* remove channel() stuff for synchronous comm; use a mutex
2018-09-25 15:01:51 -07:00
Greg Fitzgerald 680072e5e2 No need to special case vote failures 2018-09-25 13:43:35 -06:00
Greg Fitzgerald 4ca377a655 Delete dead code 2018-09-25 13:43:35 -06:00
Tyera Eulberg 751dd7eebb Move vote into ReplicateStage after process_entries 2018-09-25 13:43:35 -06:00
Michael Vines 8f0e0c4440 Add tic-tac-toe program 2018-09-25 12:07:41 -07:00
Tyera Eulberg db310a044c
Add Budget::And element, and supporting functions (#1329) 2018-09-25 12:38:13 -06:00
Rob Walker 88a609ade5 groom write_stage 2018-09-25 00:18:35 -07:00
Rob Walker 304d63623f give replication some time to happen
fixes #1307
2018-09-24 23:57:09 -07:00
Rob Walker 407b2682e8 remove dead code 2018-09-24 23:12:09 -07:00
Pankaj Garg 0f4fd8367d
Add counters for channel pressure and time spent in TPU pipeline (#1324)
* Add counters for channel pressure and time spent in TPU pipeline

* Fixed failing tests

* Fix rust format issue
2018-09-24 17:13:49 -07:00
Greg Fitzgerald 747ba6a8d3 Boot BudgetState::last_error 2018-09-24 17:14:23 -06:00
Greg Fitzgerald bb99fd40de Update transaction status in the bank
This will allow jsonrpc to query the system to find out if a
recent transaction failed.
2018-09-24 17:14:23 -06:00
Greg Fitzgerald e972d6639d Return errors from BudgetProgram::process_transaction 2018-09-24 17:14:23 -06:00
Greg Fitzgerald 22e77c9485 Add a way of getting transaction errors out of the bank 2018-09-24 17:14:23 -06:00
Greg Fitzgerald 95677a81c5 Pacify clippy 2018-09-24 13:36:31 -06:00
Greg Fitzgerald ea37d29d3a Pass Bank::process_transactions() a reference to the txs instead of moving them 2018-09-24 13:36:31 -06:00
sakridge e030673c9d
Do a recv on join to prevent channel destruction (#1320)
before window thread join
2018-09-24 11:50:37 -07:00
sakridge 3e76efe97e
Fix bench compilation (#1311) 2018-09-24 10:40:42 -07:00
Stephen Akridge f5a30615c1 Ignore replicator startup for now 2018-09-24 09:43:58 -06:00
Rob Walker 9e3d2956d8 remove last recycle? 2018-09-24 08:09:41 -06:00
jackcmay 26b1466ef6
Initial integration of dynamic contracts and native module loading (#1256)
* Integration of native dynamic programs
2018-09-23 22:13:44 -07:00
Jack May a1f01fb8f8 revert is_some to not is_none, causes test failure 2018-09-23 17:09:18 -06:00
Jack May b2be0e2e5e fix clippy warning 2018-09-23 17:09:18 -06:00
Jack May 1a45587c08 fix clippy warnings 2018-09-23 17:09:18 -06:00
Rob Walker a51c2f193e fix Rob and Carl crossing wires 2018-09-21 21:37:25 -07:00
Rob Walker be31da3dce
lastidnotfound step 2: (#1300)
lastidnotfound step 2:
  * move "record stage", aka poh_service into banking stage
  * remove Entry.has_more, is incompatible with leader rotation
  * rewrite entry_next_hash in terms of Poh
  * simplify and unify transaction hashing (no embedded nulls)
  * register_last_entry from banking stage, fixes #1171 (w00t!)
  * new PoH doesn't generate empty ledger entries, so some fixes necessary in 
         multinode tests that rely on that (e.g. giving validators airdrops)
  * make window repair less patient, if we've been waiting for an answer, 
          don't be shy about most recent blobs
   * delete recorder and record stage
   * make more verbost  thin_client error reporting
   * more tracing in window (sigh)
2018-09-21 21:01:13 -07:00
sakridge 54b407b4ca
Wait on blob fetch before window, Seems to fix instability (#1304)
also cleanup ledger.
2018-09-21 18:56:20 -07:00
Tyera Eulberg e87cac06da Request/reqwest improvements
- Use json macro to simplify request builds
- Add proxy option for reqwest to use TLS
- Add rpc port options for configured nodes
2018-09-21 18:06:20 -06:00
Tyera Eulberg ad4fef4f09 Doc for rpc_port configuration 2018-09-21 18:06:20 -06:00
Tyera Eulberg e3b3701e13 Add RPC port option to fullnode 2018-09-21 18:06:20 -06:00
Tyera Eulberg 9228fe11c9 Port Wallet to jsonrpc and fix tests 2018-09-21 18:06:20 -06:00
carllin 5ab38afa51
Changed the window_service in Replicator to send entries instead of blobs (#1302) 2018-09-21 16:50:58 -07:00
Rob Walker e49b8f0ce7
Update poh_service.rs 2018-09-21 16:03:54 -07:00
carllin c50ac96f75
Moved deserialization of blobs to entries from replicate_stage to window_service (#1287) 2018-09-21 16:01:24 -07:00
sakridge a9355c33b2
Placeholder storage contract and replicator client (#1286)
* Add hooks for executing the storage contract

* Add store_ledger stage
  Similar to replicate_stage but no voting/banking stuff, just convert
  blobs to entries and write the ledger out

* Add storage_addr to tests and add new NodeInfo constructor
  to reduce duplication...
2018-09-21 15:32:15 -07:00
Rob Walker 3dcee9f79e
Update poh_service.rs 2018-09-21 08:01:24 -07:00
Michael Vines 2614189157 cargo fmt 2018-09-20 19:46:20 -07:00
Michael Vines beeb09646a suppress warning: unused variable: recycler 2018-09-20 19:46:20 -07:00
Tyera Eulberg c0e7e43e96 fixup! s/contract/program 2018-09-20 19:33:54 -07:00
Anatoly Yakovenko 9bfead2e01 s/contract/program 2018-09-20 19:33:54 -07:00
Greg Fitzgerald 6073cd57fa Boot Recycler::recycle() 2018-09-20 17:08:51 -06:00
Michael Vines 5174be5fe7 Rename getAccount to getAccountInfo 2018-09-20 15:18:56 -07:00
Rob Walker 62a18d4c02
step one of lastidnotfound: record_stage->record_service, trim recorder to hashes (#1281)
step one of lastidnotfound

* record_stage->record_service, trim recorder to hashes
* doc updates, hash multiple without alloc()

cc #1171
2018-09-20 15:02:24 -07:00
Michael Vines a6c15684c9 Avoid panicking invalid instructions 2018-09-20 14:08:39 -07:00
Michael Vines 5691bf557c Handle bad account userdata better 2018-09-20 14:08:39 -07:00
Michael Vines 8f01f7cf21 Trace syscalls for more helpful logs 2018-09-20 14:08:39 -07:00
Michael Vines bb8c94ad2c Add getAccount JSON RPC request 2018-09-20 13:58:15 -07:00
Greg Fitzgerald d98e35e095 Delete no longer used PaymentPlan trait 2018-09-20 14:22:45 -06:00
Greg Fitzgerald 3163fbad0e Remove 'Plan' indirection since it's implied by BUDGET_CONTRACT_ID 2018-09-20 14:22:45 -06:00
Greg Fitzgerald 0172422961 Require a self-assigned account ID 2018-09-20 14:16:14 -06:00
Rob Walker 8ccfb26923 tests for my IP picker 2018-09-20 09:21:09 -07:00
Rob Walker 12a474b6ee sort local interfaces before selecting one 2018-09-20 09:21:09 -07:00
Greg Fitzgerald 270fd6d61c Fix compiler warnings 2018-09-20 09:47:36 -06:00
carllin 7b9c7d4150
Cleaned up find_leader_rotation function. Added testing for WriteStage find_leader_rotation_index() function (#1276) 2018-09-19 18:16:00 -07:00
Carl 55126f5fb6 Marked Tvu functionality in Fullnode as unused for now 2018-09-19 16:05:31 -07:00
Anatoly Yakovenko 431692d9d0 Use a Drop trait to keep track of lifetimes for recycled objects.
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
2018-09-19 16:59:42 -06:00
Michael Vines 6732a9078d Clarify AfterTimestamp wire format 2018-09-19 13:28:35 -07:00
sakridge 2981076a14
Add solana-upload-perf to parse json from bench and upload to influx (#1166) 2018-09-19 13:16:55 -07:00
Carl cd2d50e06c Changed transition to restart Rpu rather than modify bank to prevent lock contention 2018-09-19 10:48:05 -06:00
Carl 8c8a4ba705 debugging commit 2018-09-19 10:48:05 -06:00
Carl b10de40506 Made LEADER_ROTATION_INTERVAL settable so that integration tests don't time out 2018-09-19 10:48:05 -06:00
Carl 2030dfa435 Implement PR comments, tidy up 2018-09-19 10:48:05 -06:00
Carl bfe64f5f6e Added integration test for transitioning leader to validator to see that tpu pipeline can exit and restart a tvu. Fixed Tpu and broadcast stage so that exiting later stages in the pipeline also causes earlier stages to exit. 2018-09-19 10:48:05 -06:00
Carl 6d27751365 give fullnode ownership of state needed to dynamically start up a tpu or tvu for role transition 2018-09-19 10:48:05 -06:00
Carl 1fb1c0a681 added jointypes to the stages in the tpu involved in leader rotation 2018-09-19 10:48:05 -06:00
Carl 062f654fe0 formatted code 2018-09-19 10:48:05 -06:00
Carl d3cb161c36 Added broadcast stage test for leader rotation exit 2018-09-19 10:48:05 -06:00
Carl 98b47d2540 Added check in broadcast stage to exit after transmitting last blob before leader rotation. Also added tests 2018-09-19 10:48:05 -06:00
Carl f28ba3937b Added check in write stage to exit when scheduled entry_height for leader rotation is detected 2018-09-19 10:48:05 -06:00
Carl 91cf14e641 Rewrote service trait join() method to allow thread join handles to return values other than () 2018-09-19 10:48:05 -06:00
Pankaj Garg 0ee6c5bf9d
Read multiple entries in write stage (#1259)
- Also use rayon to parallelize to_blobs() to maximize CPU usage
2018-09-18 21:45:49 -07:00
Michael Vines 6dee632d67 Remove Signature from ApplySignature 2018-09-18 20:00:42 -07:00
Michael Vines 51e5de4d97 Log specific send_transaction error messages 2018-09-18 16:17:08 -07:00
Michael Vines 1f08b22c8e Tweak log messages 2018-09-18 16:17:08 -07:00
Michael Vines 83ae5bcee2 Detect binary changes in serialized contract userdata 2018-09-18 16:17:08 -07:00
Michael Vines 339a570b26 Update comment 2018-09-18 16:17:08 -07:00
Pankaj Garg 5310b6e5a2
Move entry->blob creation out of write stage (#1257)
- The write stage will output vector of entries
- Broadcast stage will create blobs out of the entries
- Helps reduce MIPS requirements for write stage
2018-09-18 13:49:10 -07:00
Pankaj Garg 7d14f44a7c
Move register_entry_id() call out of write stage (#1253)
* Move register_entry_id() call out of write stage

- Write stage is MIPS intensive and has become a bottleneck for
  TPU pipeline
- This will reduce the MIPS requirements for the stage

* Fix rust format issues
2018-09-18 11:42:25 -07:00
Michael Vines aebf7f88e5
Various spelling fixes 2018-09-17 19:37:59 -07:00
Michael Vines aac1571670 mint now uses the SystemContract instead of Budget 2018-09-17 18:02:40 -07:00
Anatoly Yakovenko 8bae75a8a6 system contract tests 2018-09-17 14:34:55 -07:00
Tyera Eulberg c2f7ca9d8f Change process_command return type and improve test 2018-09-17 13:45:47 -07:00
anatoly yakovenko 6ec0e42220
budget as separate contract and system call contract (#1189)
* budget and system contracts and verification

* contract check_id methods
* system call contract
* verify contract execution rules
* move system into its own file
* allocate before transfer for budget
* store error in budget context
* budget contract and tests without bank
* moved budget of of bank
2018-09-17 13:36:31 -07:00
sakridge 072b244575
Add perf counters for record/write stages (#1240) 2018-09-17 11:07:04 -07:00
Tyera Eulberg 7ac9d6c604 Create keygen helper function for use in Wallet CLI, print keypair statement 2018-09-17 11:53:33 -06:00
Tyera Eulberg 0125163190 Remove wallet.sh, update entrypoint syntax for wallet network argument 2018-09-17 11:53:33 -06:00
Tyera Eulberg a06f4b1d44 Update wallet to trigger keygen if no keypair provided and no keypair found in default location 2018-09-17 11:53:33 -06:00
Tyera Eulberg 10daa015c4 Simplify timeout arg 2018-09-17 11:53:33 -06:00
Tyera Eulberg 0babee39a4 Update wallet to take network arg 2018-09-17 11:53:33 -06:00
Tyera Eulberg 27986d7abb Standardize CLI help text 2018-09-16 15:17:10 -06:00
Tyera Eulberg 8b7edc6d64 Alphabetize 2018-09-16 15:17:10 -06:00
Michael Vines 7dfab867fe Mark --outfile parameter as required 2018-09-16 10:49:02 -07:00
Anatoly Yakovenko fd36954477 clippy 2018-09-15 05:12:53 -06:00
Jay Kickliter fd51599fa8 Replace replace(..., None) with take()
This is strictly for simplicity, since Option::take() is imlemented with replace().
2018-09-15 05:12:09 -06:00
Pankaj Garg e142aafca9
Use multiple sockets for receiving blobs on validators (#1228)
* Use multiple sockets for receiving blobs on validators

- The blobs that are broadcasted by leader or retransmitted by peer
  validators are received on replicate_port
- Using reuse_addr/reuse_port, multiple sockets can be opened for
  the same port
- This allows the kernel to queue data to user space app on multiple
  socket queues, preventing over-running one queue
- This helps with reducing packets dropped due to queue over-runs

Fixes #1224

* Fixed failing tests
2018-09-14 16:56:06 -07:00
Michael Vines 4196cf43e8 cargo fmt 2018-09-14 16:37:49 -07:00
Tyera Eulberg bcb3b3c21f Add integration tests to wallet module 2018-09-14 08:21:33 -06:00
Tyera Eulberg d8c9a1aae9 Add method to run local drone for tests 2018-09-14 08:21:33 -06:00
Tyera Eulberg 9ca2f5b3f7 Move all handling except network/gossip from /bin to wallet module 2018-09-14 08:21:33 -06:00
sakridge 4dc30ea104
Add recycler stats (#1187) 2018-09-13 14:49:48 -07:00
Pankaj Garg 90df6237c6
Implements recvmmsg() for UDP packets (#1161)
* Implemented recvmmsg() for UDP packets

- This change implements binding between libc API for recvmmsg()
- The function can receive multiple packets using one system call

Fixes #1141

* Added unit tests for recvmmsg()

* Added recv_mmsg() wrapper for non Linux OS

* Address review comments for recvmmsg()

* Remove unnecessary imports

* Moved target specific dependencies to the function
2018-09-13 14:41:28 -07:00
carllin 8706774ea7
Rewrote service trait join() method to allow thread join handles to return values other than () (#1213) 2018-09-13 14:00:17 -07:00
Pankaj Garg 1d7e87d430 Increase number of sockets for transaction processing 2018-09-13 14:22:07 -06:00
sakridge 1a4cd763f8
Fix missing recycle in recv_from (#1205)
In the error case that i>0 (we have blobs to send)
we break out of the loop and do not push the allocated r
to the v array. We should recycle this blob, otherwise it
will be dropped.
2018-09-13 08:29:18 -07:00
Michael Vines f06113500d bench-tps/net sanity: add ability to check for unexpected extra nodes 2018-09-12 15:38:57 -07:00
Rob Walker 9ab5692acf
fix "leak" in Blob::recv_from (#1198)
* fix "leak" in Blob::recv_from

fixes #1199
2018-09-12 14:45:43 -07:00
Rob Walker b52230097e groom Fullnode's new_with_bank() to match new() more 2018-09-12 09:24:42 -07:00
Rob Walker a8fdb8a5a7 use a single BlobRecycler per fullnode 2018-09-11 16:56:54 -07:00
sakridge 297f859631
Change '>=' back to '>' to fix recycling of blobs/packets (#1192)
Recycler will have a strong ref to the item so it will be at
least 1, >= will always prevent recycling.
2018-09-11 16:52:45 -07:00
Rob Walker b313b7f6f9 Revert "move rpc_server to drop() semantics instead of having its own thread"
This reverts commit 40aa0654fa.
2018-09-10 22:48:33 -07:00
Rob Walker c3fc98c48f use gossip to find the leader for every airdrop request 2018-09-10 21:29:45 -07:00
Rob Walker 40aa0654fa move rpc_server to drop() semantics instead of having its own thread 2018-09-10 20:25:53 -07:00
Michael Vines bace2880d0
Correct spelling 2018-09-10 19:58:21 -07:00
sakridge 9d80eefb81
Log the number of accounts each 250k txes (#1178) 2018-09-10 17:40:00 -07:00
Michael Vines 2be0dbddbb
Correct spelling 2018-09-10 13:48:43 -07:00
Rob Walker a91b785ba5 move fullnode trace generation into crdt 2018-09-10 13:47:57 -07:00
Tyera Eulberg 0ef05de889 Add sleep to prevent spinning thread 2018-09-10 12:50:28 -07:00
Greg Fitzgerald a093d5c809 Fix erasure build 2018-09-10 11:40:26 -06:00
Greg Fitzgerald fc64e1853c Initialize Window, not SharedWindow
Wrap with Arc<RwLock>> when/if needed, no earlier.
2018-09-10 11:40:26 -06:00
Greg Fitzgerald 7f669094de Split window into two modules 2018-09-10 11:40:26 -06:00
Greg Fitzgerald 5025d89c88 Inline window method implementations 2018-09-10 11:40:26 -06:00
Greg Fitzgerald 2b44c4504a Use WindowUtil for more idiomatic code 2018-09-10 11:40:26 -06:00
Greg Fitzgerald d2c9beb843 Add a trait to pretend Window is an object 2018-09-10 11:40:26 -06:00
Michael Vines 9e6d3bf532
Correct spelling 2018-09-10 09:29:01 -07:00
anatoly yakovenko a89b611e9e
comments (#1165) 2018-09-09 07:07:38 -07:00
Rob Walker 97c3125a78
improve localnet-sanity's robustness (#1160)
* fix poll_gossip_for_leader() loop to actually wait
         for 30 seconds
    * reduce reuseaddr use to only when necessary,
         try to avoid already bound sockets
    * move nat.rs to netutil.rs
    * add gossip tracing to thin_client and bench-tps
2018-09-09 04:50:43 +09:00
Michael Vines e8206cb2d4 Echo the network address before entering a quiet polling loop 2018-09-07 21:20:00 -07:00
anatoly yakovenko c34d911eaf
Migrate Budget DSL to use the Account state (#979)
* Migrate Budget DSL to use the Account state instead of global bank data structures.

* Serialize Instruction into Transaction::userdata.
* Store the pending set in the Account::userdata
* Enforce the token balance rules on contract execution. This becomes the entry point for generic contracts.
* This pr will have a performance impact on the bank. The next set of changes will fix this by locking each account during multi threaded execution of all the contracts.
* With this change a contract transaction needs to store its state under an address. That address could be the destination of the tokens, or any random address. For the latter, an extra step would be needed to claim the tokens which isn't implemented by budget_dsl at the moment.
* test tracking issue 1157
2018-09-07 20:18:36 -07:00
Rob Walker bf29590f41
WSL needs ReuseAddr in addition to ReusePort (which it doesn't honor) (#1149) 2018-09-08 07:28:22 +09:00
Rob Walker 51b27779c9
client changes for TODOs and looping (#1138)
* remove client.sh from snap
* default to ephemeral instead of ~/.config key
* rework CLI for bench-tps
* remote multinode-demo stuff from remote-client.sh
* remove multinode-demo from remote-sanity and localnet-sanity
2018-09-08 07:07:10 +09:00
Tyera Eulberg 5169c8d08f Add method to return hash of bank state 2018-09-07 15:38:53 -06:00
Michael Vines 4b47abd3bf Fix --num-nodes argument parsing 2018-09-07 12:20:42 -07:00
Greg Fitzgerald 71a617b4dc Fix erasure build 2018-09-07 13:18:19 -06:00
Greg Fitzgerald a722802c95 Window write lock to read lock 2018-09-07 13:18:19 -06:00
Greg Fitzgerald e9f44b6661 window -> window_service 2018-09-07 13:18:19 -06:00
Greg Fitzgerald 9693de1867 Reposition parameters 2018-09-07 13:18:19 -06:00
Greg Fitzgerald f7ea95aed1 Hoist lock, reposition parameters 2018-09-07 13:18:19 -06:00
Greg Fitzgerald f07ce59be8 Toggle parameters 2018-09-07 13:18:19 -06:00
Greg Fitzgerald da423b6cf0 Hoist read lock 2018-09-07 13:18:19 -06:00
Greg Fitzgerald d5f60b68e4 Hoist window write lock 2018-09-07 13:18:19 -06:00
Greg Fitzgerald 78b3a8f7f9 Hoist repair_window() branches
This probably would have been done if repair_window() was unit-tested.
2018-09-07 13:18:19 -06:00
Greg Fitzgerald d77699c126 Do the easy check first
All functions above operate on immutable values, so this shouldn't
change functionality, but no repair_window() tests to be certain.hI
2018-09-07 13:18:19 -06:00
Greg Fitzgerald 09ba0dae15 Remove redundant clone() 2018-09-07 13:18:19 -06:00
Greg Fitzgerald a5c7575207 Rewrite find_next_missing, call it clear_slots 2018-09-07 13:18:19 -06:00
Greg Fitzgerald 50f040530b Remove redundant cast 2018-09-07 13:18:19 -06:00
Greg Fitzgerald 7f99c90539 Simplify using early return and Result::ok() 2018-09-07 13:18:19 -06:00
Greg Fitzgerald d8564b725c Don't reference window to get each slot 2018-09-07 13:18:19 -06:00
Greg Fitzgerald e4de25442a Hoist write lock
It needed to be passed the lock before, because it contained a
branch where one side didn't require locking. Now that that
defensive programming was hoisted, we can hoist the write lock
as well, leaving a simpler function for unit testing.
2018-09-07 13:18:19 -06:00
Greg Fitzgerald 3b2ea8fd40 Hoist untested branch in window
If there were unit tests for this function, the author would have
written it this way to make their own life easier.
2018-09-07 13:18:19 -06:00
Greg Fitzgerald 0fa27f65bb Use the default Pubkey formatter instead of debug_id() 2018-09-06 16:31:47 -06:00
sakridge 8f94e3f7ae
Buffer tokens when switching directions to prevent errors (#1126)
Even if transactions are dropped, accounts will have buffer
of tokens. Should reduce or eliminate AccountNotFound errors seen in the
leader while bench-tps is running.
2018-09-06 14:20:01 -07:00
Pankaj Garg 05460eec0d
Open multiple sockets for transaction UDP port (#1128)
* Reuse UDP port and open multiple sockets for transaction address

* Fixed failing crdt tests

* Add tests for reusing UDP ports

* Address review comments

* Updated bench-streamer to use multiple receive sockets

* Fix minimum number of recv sockets for bench-streamer

* Address review comments

Fixes #1132

* Moved bind_to function to nat.rs
2018-09-06 14:13:40 -07:00
sakridge d9e4bce6ad
Add drop stats to bench-tps (#1127)
See how many transactions made it through
2018-09-05 11:58:41 -07:00
sakridge ffb72136c8
Remove account from balances table after error seen (#1120)
If balance goes to 0, then bank removes the account
from it's account table and returns no account error. Thin client
should also update the account to this state or it will
still have the cached balance from the last successful get_balance().
2018-09-04 21:33:19 -07:00
Greg Fitzgerald 68c72d6f34 Fix nightly build 2018-09-04 20:56:40 -06:00
sakridge d82ec2634c
Fix is_leader boolean (#1115)
A node is the leader if the address is none
2018-09-04 13:38:24 -07:00
Rob Walker c44c5f0b09
take into account size of an Entry (#1116) 2018-09-05 05:07:58 +09:00
Rob Walker 226d3b9471
Trace recycle() calls (#968)
* trace recycle() calls fixes #810
2018-09-05 05:07:02 +09:00
Stephen Akridge 2752bde683 Print to indicate what drone is doing while waiting for gossip 2018-09-04 13:45:08 -06:00
Greg Fitzgerald 2aa72cc72e Return a Vec from to_blobs() instead of using a mut parameter 2018-09-04 07:50:23 -10:00
Greg Fitzgerald 8cc030ef84 Use Vec instead of VecDeque for SharedBlobs 2018-09-04 07:50:23 -10:00
Stephen Akridge 9a9f89293a Better error handling messages for airdrops 2018-09-04 06:46:43 -10:00
anatoly yakovenko 501deeef56 accounts should never be negative (#1083) 2018-09-04 06:43:18 -10:00
Greg Fitzgerald 05f921d544 Don't call println in the test suite 2018-09-04 06:01:32 -10:00
Greg Fitzgerald ab7a2960b1 Don't use product name in solana library 2018-09-04 06:01:32 -10:00
Greg Fitzgerald 4e2deaa33b Less mut 2018-09-04 06:01:32 -10:00
Greg Fitzgerald d5ef18337c Remove redundant return value
And don't log the same error twice.
2018-09-04 06:01:32 -10:00
Greg Fitzgerald d18ea501b7 Minimize unsafe code 2018-09-04 06:01:32 -10:00
Greg Fitzgerald c9a1ac9b8c Don't propogate errors we'll never handle 2018-09-04 06:01:32 -10:00
Greg Fitzgerald c2a4cb544e Borrow, don't clone entries 2018-09-04 06:01:32 -10:00
Greg Fitzgerald 3ab12076e8 Convert voting functions to methods
More idiomatic Rust.
2018-09-04 05:53:58 -10:00
anatoly yakovenko fb4b33b81b
make the repair_backoff test more robust (#1095)
* more the repair_backoff test more robust

* fix names and magic numbers
2018-08-31 12:40:56 -10:00
Rob Walker 5c8cb96f88 rebase fixup 2018-08-31 23:21:07 +09:00
Rob Walker 176e806d94 rework of netwrk rendezvous
* rename NodeInfo field of Node from "data" to "info"
      (touches a lot of files)

  * update client to use gossip to find leader, a la drone

  * rework multinode scripts
      * move more stuff into rust
      * added usage to all
      * no more rsync unless you're a validator (TODO: whack that, too)
  * fullnode doesn't bail if drone isn't up yet, just keeps trying
  * drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
Rob Walker eb4e5a7bd0 fixups 2018-08-31 23:21:07 +09:00
Rob Walker ba27596076 fixups 2018-08-31 23:21:07 +09:00
Rob Walker 63e44dcc35 continue rendezvous refactor for gossip and repair
* remove trailing whitespace in ci/audit.sh

  * code review fixups
     * rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
     * remove out-of-date TODO in localnet-sanity.sh

  * remove features=test and code that was using it (localhost prohibitions in
      crdt) added TODO in crdt.rs, maybe we should boot localhost in production
      networks?

  * boot tvu_window from NodeInfo: instead, send repair requests from the repair
      socket (to gossip on peer) and answer repair requests via the sockaddr
      from the repair request

  * remove various unused pub functions

  * banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
Rob Walker c0ba676658 fixup 2018-08-31 23:21:07 +09:00
Rob Walker 1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
Tyera Eulberg e308a4279e Update RPC requestAirdrop endpoint to return airdrop tx signature 2018-08-28 18:27:41 -06:00
Tyera Eulberg 513a934ff6 Update request_airdrop utility function to pass along airdrop tx signature 2018-08-28 18:27:41 -06:00
Tyera Eulberg 77d820c842 Update drone module to return airdrop tx signature 2018-08-28 18:27:41 -06:00
anatoly yakovenko 48762834d9
Randomize repair requests (#1059)
* randomize packet repair requests

* exponential random repair requests

* use gen_range to get a uniform distribution
2018-08-27 07:05:48 -07:00
Anatoly Yakovenko 8d0d429acd update 2018-08-26 23:34:25 -07:00
Anatoly Yakovenko e5408368f7 fmt 2018-08-26 22:35:26 -07:00
Anatoly Yakovenko 61492fd27e exit if no leader 2018-08-26 22:35:26 -07:00
Anatoly Yakovenko bbce08a67b bench needs to discover leader as well 2018-08-26 22:35:26 -07:00
Anatoly Yakovenko a002148098 retry transfer and poll 2018-08-26 16:10:46 -07:00
Greg Fitzgerald 90ae662e4d Fix packet header offset
And update transaction offsets to use the same approach as packet.rs.
Maybe this should be serialized_size(), but thanks to this
GenericArray update, those values are the same.
2018-08-26 14:27:19 -06:00
Greg Fitzgerald 60d8f5489f Update transaction layout offsets
24 less bytes in minimal transactions. 10% TPS boost?
2018-08-26 14:27:19 -06:00
anatoly yakovenko 738247ad44
advertise valid gossip address in drone and wallet (#1066)
* advertize valid gossip address in drone and wallet

get rid of asserts

check for valid ip address

check for valid address

ip address

* tests

* cleanup

* cleanup

* print error

* bump

* disable tests

* disable nightly
2018-08-26 11:36:27 -07:00
anatoly yakovenko 5b0bb7e607
Skip invalid nodes for finality (#1068)
* skip invalid nodes for finality

* check valid last_ids only

* fixup!

* fixup!
2018-08-25 23:12:41 -07:00
anatoly yakovenko f7c0d30167
Disallow localhost in deployment (#1064)
* disallow localhost in deployment

* tests

* fmt

* integration tests do not have a flag to check

* fmt
2018-08-25 21:09:18 -07:00
Anatoly Yakovenko 8e98c7c9d6 fix purge test 2018-08-25 19:56:09 -07:00
Pankaj Garg 50661e7b8d Added poll_balance_with_timeout method (#1062)
* Added poll_balance_with_timeout method

- updated bench-tps, fullnode and wallet to use this method instead
  of repeatedly calling poll_get_balance()

* Address review comments

- Revert some changes to use wrapper poll_get_balance()

* Reverting bench-tps to use poll_get_balance

- The original code is checking if the balance has been updated,
  instead of just retrieving the balance. The logic is different
  than poll_balance_with_timeout()

* Reverting wallet to use poll_get_balance

- The break condition in the loop is different than poll_balance_with_timeout().
  It's checking if the balance has been updated.
2018-08-25 18:24:25 -07:00
Pankaj Garg ad159e0906 Fix crash in fullnode when poll_get_balance() returns error (#1058) 2018-08-25 15:25:13 -07:00
Pankaj Garg d3fac8a06f
Dynamically bind to available UDP ports in Fullnode (#920)
* Dynamically bind to available UDP ports in Fullnode

* Added tests for dynamic port binding

- Also removed hard coding of port range from CRDT
2018-08-25 10:24:16 -07:00
sakridge de379ed915
Fix sig verify counters to be unique and tweak perf counters (#1056)
print events and add current events to old value to report
2018-08-24 16:05:32 -07:00
sakridge 6fc21a4223
Don't hang in transaction_count (#1052)
Situation is there can be that there can be bad entries in
the bench-tps CRDT table until they get purged later. Threads however
are created for those bad entries and then will hang on trying
to get the transaction_count from those bad addresses and never end.
2018-08-23 20:57:13 -07:00
sakridge 71319978df
Up drone request amount (#1051)
Multiple clients will request 500k each so up this to support them.
2018-08-23 15:30:35 -07:00
sakridge 6147e54686
Cap repair requests timeout (#958) 2018-08-23 15:30:21 -07:00
Greg Fitzgerald 0c8eec2563 Cleanup Fullnode construction
leader_id was already set by Fullnode constructor. And cleanup the
rest of that code while in the neighborhood.

Thanks @CriesofCarrots!
2018-08-23 13:42:54 -07:00
Greg Fitzgerald 4ab58f069a Add back JsonRpcService changes 2018-08-23 13:42:54 -07:00
Greg Fitzgerald 85f96d926a Pacify clippy 2018-08-23 13:42:54 -07:00
Greg Fitzgerald 816de4f8ec Hoist shared code between leaders and validators 2018-08-23 13:42:54 -07:00
Greg Fitzgerald 42229a1105 Hoist thread_hdls 2018-08-23 13:42:54 -07:00
Greg Fitzgerald d8820053af Inline create_leader_threads and create_validator_threads 2018-08-23 13:42:54 -07:00
Greg Fitzgerald 731f8512c6 Hoist Arc<Bank> 2018-08-23 13:42:54 -07:00
Greg Fitzgerald a133784706 Rename mode-specific constructors and return only thread handles 2018-08-23 13:42:54 -07:00
Greg Fitzgerald be58fdf1bb Less constructors 2018-08-23 13:42:54 -07:00
Greg Fitzgerald 57daeb35d2 Drop all references to new_leader and new_validator 2018-08-23 13:42:54 -07:00
Greg Fitzgerald 9c5e69bf3d Don't offer two ways to specify a leader 2018-08-23 13:42:54 -07:00
Greg Fitzgerald cfac127e4c Extract lower-level constructor
Passing in the bank is useful for unit-tests since Fullnode doesn't
store it in a member variable.
2018-08-23 13:42:54 -07:00
Greg Fitzgerald fda4523cbf Fix broken doc 2018-08-23 13:42:54 -07:00
sakridge cabe80b129
Increment counter by number of packets received (#1049)
So that we can see the total packets/s
2018-08-23 12:32:50 -07:00
Tyera Eulberg d4c41219f9 Improve gossip use for drone and wallet
- Add utility function
  - Add thread sleep
  - Enable configurable timeout for gossip poll
2018-08-23 13:08:59 -06:00
Tyera Eulberg 4fdd9fbfca Wallet: use gossip to identify leader's port config 2018-08-23 13:08:59 -06:00
Tyera Eulberg bdf5ac9c1a Drone: use gossip to identify leader's port config 2018-08-23 13:08:59 -06:00
sakridge f1785c76a4
Rework counter increment outside apply_debits loop (#1046)
Reduces prints/atomics work inside the process_transactions loop
2018-08-23 09:42:59 -07:00
Tyera Eulberg 2de8fe9c5f Pass bank to rpc as reference 2018-08-23 09:06:17 -06:00
Tyera Eulberg d910ed68a3 Use balance to verify requestAirdrop success 2018-08-23 09:06:17 -06:00
Tyera Eulberg f7f7ecd4c6 Add json-rpc requestAirdrop endpoint 2018-08-23 09:06:17 -06:00
Tyera Eulberg a9c3a28a3b Add json-rpc sendTransaction endpoint 2018-08-23 09:06:17 -06:00
Stephen Akridge c3ed4d28de Change average TPS to max average tps seen for any node and...
add script to collect perf stats
2018-08-22 14:55:04 -07:00
Tyera Eulberg a355d9f46c Add error catch for rpc server builder 2018-08-21 14:04:52 -06:00
Tyera Eulberg be4824c955 Add custom panic hook for RPC port bind 2018-08-21 14:04:52 -06:00
Tyera Eulberg 86c1d97c13 Fix validator rpc addr to match leader 2018-08-20 22:35:06 -07:00
Greg Fitzgerald 2d35345c50 Boot unused creates 2018-08-20 16:48:37 -06:00
Michael Vines a02910be32 Remove pubkey from getBalance response 2018-08-20 15:02:48 -07:00
Pankaj Garg b9ec97a30b
Add counter for bank transaction errors (#1015) 2018-08-20 14:56:01 -07:00
Michael Vines 9eeaf2d502 Bind RPC port on all interfaces 2018-08-20 12:45:50 -07:00
Michael Vines 8de317113c clippy: remove identity conversion 2018-08-20 10:55:55 -07:00
Tyera Eulberg 1bf15ae907 Temporarily disable cargo audit CI failure 2018-08-18 12:29:49 -06:00
Tyera Eulberg f73f3941cd Revert ill-advised jsonrpc marker, and handle jsonrpc server close 2018-08-18 12:29:49 -06:00
Tyera Eulberg d69d79612b Simplify Rpc request processing 2018-08-18 12:29:49 -06:00
Tyera Eulberg 64ea5126e0 Fix early return for invalid parameter 2018-08-18 12:29:49 -06:00
Tyera Eulberg 9df3aa50d5 Remove unnecessary solana_ prefixes 2018-08-18 12:29:49 -06:00
Tyera Eulberg cab75b7829 Handle potential panics 2018-08-18 12:29:49 -06:00
Tyera Eulberg d9fac86015 Use jsonrpc git repo, allowing removal of Default bound for Metadata 2018-08-18 12:29:49 -06:00
Tyera Eulberg 1eb8724a89 Disable Rpc module for other tests to prevent port conflicts 2018-08-18 12:29:49 -06:00
Tyera Eulberg c6662a4512 Implement Rpc in Fullnode 2018-08-18 12:29:49 -06:00
Tyera Eulberg d3c09b4e96 Update jsonrpc dependency syntax 2018-08-18 12:29:49 -06:00
Tyera Eulberg 124f6e83d2 Rpc get last id endpoint 2018-08-18 12:29:49 -06:00
Tyera Eulberg 569ff73b39 Rpc tests 2018-08-18 12:29:49 -06:00
Tyera Eulberg fc1dbddd93 Implement json-rpc functionality 2018-08-18 12:29:49 -06:00
Rob Walker 3ae867bdd6 fixups 2018-08-18 02:22:52 -07:00
Rob Walker bc5f29150b fix erasure, remove Entry "pad"
* fixes #997
 * Entry pad is no longer required since erasure coding aligns data length
2018-08-18 02:22:52 -07:00
Rob Walker 46016b8c7e crashes generate_coding() 2018-08-18 02:22:52 -07:00