9ca2f5b3f7
Move all handling except network/gossip from /bin to wallet module
2018-09-14 08:21:33 -06:00
4dc30ea104
Add recycler stats ( #1187 )
2018-09-13 14:49:48 -07:00
90df6237c6
Implements recvmmsg() for UDP packets ( #1161 )
...
* Implemented recvmmsg() for UDP packets
- This change implements binding between libc API for recvmmsg()
- The function can receive multiple packets using one system call
Fixes #1141
* Added unit tests for recvmmsg()
* Added recv_mmsg() wrapper for non Linux OS
* Address review comments for recvmmsg()
* Remove unnecessary imports
* Moved target specific dependencies to the function
2018-09-13 14:41:28 -07:00
8706774ea7
Rewrote service trait join() method to allow thread join handles to return values other than () ( #1213 )
2018-09-13 14:00:17 -07:00
1d7e87d430
Increase number of sockets for transaction processing
2018-09-13 14:22:07 -06:00
1a4cd763f8
Fix missing recycle in recv_from ( #1205 )
...
In the error case that i>0 (we have blobs to send)
we break out of the loop and do not push the allocated r
to the v array. We should recycle this blob, otherwise it
will be dropped.
2018-09-13 08:29:18 -07:00
f06113500d
bench-tps/net sanity: add ability to check for unexpected extra nodes
2018-09-12 15:38:57 -07:00
9ab5692acf
fix "leak" in Blob::recv_from ( #1198 )
...
* fix "leak" in Blob::recv_from
fixes #1199
2018-09-12 14:45:43 -07:00
b52230097e
groom Fullnode's new_with_bank() to match new() more
2018-09-12 09:24:42 -07:00
a8fdb8a5a7
use a single BlobRecycler per fullnode
2018-09-11 16:56:54 -07:00
297f859631
Change '>=' back to '>' to fix recycling of blobs/packets ( #1192 )
...
Recycler will have a strong ref to the item so it will be at
least 1, >= will always prevent recycling.
2018-09-11 16:52:45 -07:00
b313b7f6f9
Revert "move rpc_server to drop() semantics instead of having its own thread"
...
This reverts commit 40aa0654fa
2018-09-10 22:48:33 -07:00
c3fc98c48f
use gossip to find the leader for every airdrop request
2018-09-10 21:29:45 -07:00
40aa0654fa
move rpc_server to drop() semantics instead of having its own thread
2018-09-10 20:25:53 -07:00
bace2880d0
Correct spelling
2018-09-10 19:58:21 -07:00
9d80eefb81
Log the number of accounts each 250k txes ( #1178 )
2018-09-10 17:40:00 -07:00
2be0dbddbb
Correct spelling
2018-09-10 13:48:43 -07:00
a91b785ba5
move fullnode trace generation into crdt
2018-09-10 13:47:57 -07:00
0ef05de889
Add sleep to prevent spinning thread
2018-09-10 12:50:28 -07:00
a093d5c809
Fix erasure build
2018-09-10 11:40:26 -06:00
fc64e1853c
Initialize Window, not SharedWindow
...
Wrap with Arc<RwLock>> when/if needed, no earlier.
2018-09-10 11:40:26 -06:00
7f669094de
Split window into two modules
2018-09-10 11:40:26 -06:00
5025d89c88
Inline window method implementations
2018-09-10 11:40:26 -06:00
2b44c4504a
Use WindowUtil for more idiomatic code
2018-09-10 11:40:26 -06:00
d2c9beb843
Add a trait to pretend Window is an object
2018-09-10 11:40:26 -06:00
9e6d3bf532
Correct spelling
2018-09-10 09:29:01 -07:00
a89b611e9e
comments ( #1165 )
2018-09-09 07:07:38 -07:00
97c3125a78
improve localnet-sanity's robustness ( #1160 )
...
* fix poll_gossip_for_leader() loop to actually wait
for 30 seconds
* reduce reuseaddr use to only when necessary,
try to avoid already bound sockets
* move nat.rs to netutil.rs
* add gossip tracing to thin_client and bench-tps
2018-09-09 04:50:43 +09:00
e8206cb2d4
Echo the network address before entering a quiet polling loop
2018-09-07 21:20:00 -07:00
c34d911eaf
Migrate Budget DSL to use the Account state ( #979 )
...
* Migrate Budget DSL to use the Account state instead of global bank data structures.
* Serialize Instruction into Transaction::userdata.
* Store the pending set in the Account::userdata
* Enforce the token balance rules on contract execution. This becomes the entry point for generic contracts.
* This pr will have a performance impact on the bank. The next set of changes will fix this by locking each account during multi threaded execution of all the contracts.
* With this change a contract transaction needs to store its state under an address. That address could be the destination of the tokens, or any random address. For the latter, an extra step would be needed to claim the tokens which isn't implemented by budget_dsl at the moment.
* test tracking issue 1157
2018-09-07 20:18:36 -07:00
bf29590f41
WSL needs ReuseAddr in addition to ReusePort (which it doesn't honor) ( #1149 )
2018-09-08 07:28:22 +09:00
51b27779c9
client changes for TODOs and looping ( #1138 )
...
* remove client.sh from snap
* default to ephemeral instead of ~/.config key
* rework CLI for bench-tps
* remote multinode-demo stuff from remote-client.sh
* remove multinode-demo from remote-sanity and localnet-sanity
2018-09-08 07:07:10 +09:00
5169c8d08f
Add method to return hash of bank state
2018-09-07 15:38:53 -06:00
4b47abd3bf
Fix --num-nodes argument parsing
2018-09-07 12:20:42 -07:00
71a617b4dc
Fix erasure build
2018-09-07 13:18:19 -06:00
a722802c95
Window write lock to read lock
2018-09-07 13:18:19 -06:00
e9f44b6661
window -> window_service
2018-09-07 13:18:19 -06:00
9693de1867
Reposition parameters
2018-09-07 13:18:19 -06:00
f7ea95aed1
Hoist lock, reposition parameters
2018-09-07 13:18:19 -06:00
f07ce59be8
Toggle parameters
2018-09-07 13:18:19 -06:00
da423b6cf0
Hoist read lock
2018-09-07 13:18:19 -06:00
d5f60b68e4
Hoist window write lock
2018-09-07 13:18:19 -06:00
78b3a8f7f9
Hoist repair_window() branches
...
This probably would have been done if repair_window() was unit-tested.
2018-09-07 13:18:19 -06:00
d77699c126
Do the easy check first
...
All functions above operate on immutable values, so this shouldn't
change functionality, but no repair_window() tests to be certain.hI
2018-09-07 13:18:19 -06:00
09ba0dae15
Remove redundant clone()
2018-09-07 13:18:19 -06:00
a5c7575207
Rewrite find_next_missing, call it clear_slots
2018-09-07 13:18:19 -06:00
50f040530b
Remove redundant cast
2018-09-07 13:18:19 -06:00
7f99c90539
Simplify using early return and Result::ok()
2018-09-07 13:18:19 -06:00
d8564b725c
Don't reference window to get each slot
2018-09-07 13:18:19 -06:00
e4de25442a
Hoist write lock
...
It needed to be passed the lock before, because it contained a
branch where one side didn't require locking. Now that that
defensive programming was hoisted, we can hoist the write lock
as well, leaving a simpler function for unit testing.
2018-09-07 13:18:19 -06:00
3b2ea8fd40
Hoist untested branch in window
...
If there were unit tests for this function, the author would have
written it this way to make their own life easier.
2018-09-07 13:18:19 -06:00
0fa27f65bb
Use the default Pubkey formatter instead of debug_id()
2018-09-06 16:31:47 -06:00
8f94e3f7ae
Buffer tokens when switching directions to prevent errors ( #1126 )
...
Even if transactions are dropped, accounts will have buffer
of tokens. Should reduce or eliminate AccountNotFound errors seen in the
leader while bench-tps is running.
2018-09-06 14:20:01 -07:00
05460eec0d
Open multiple sockets for transaction UDP port ( #1128 )
...
* Reuse UDP port and open multiple sockets for transaction address
* Fixed failing crdt tests
* Add tests for reusing UDP ports
* Address review comments
* Updated bench-streamer to use multiple receive sockets
* Fix minimum number of recv sockets for bench-streamer
* Address review comments
Fixes #1132
* Moved bind_to function to nat.rs
2018-09-06 14:13:40 -07:00
d9e4bce6ad
Add drop stats to bench-tps ( #1127 )
...
See how many transactions made it through
2018-09-05 11:58:41 -07:00
ffb72136c8
Remove account from balances table after error seen ( #1120 )
...
If balance goes to 0, then bank removes the account
from it's account table and returns no account error. Thin client
should also update the account to this state or it will
still have the cached balance from the last successful get_balance().
2018-09-04 21:33:19 -07:00
68c72d6f34
Fix nightly build
2018-09-04 20:56:40 -06:00
d82ec2634c
Fix is_leader boolean ( #1115 )
...
A node is the leader if the address is none
2018-09-04 13:38:24 -07:00
c44c5f0b09
take into account size of an Entry ( #1116 )
2018-09-05 05:07:58 +09:00
226d3b9471
Trace recycle() calls ( #968 )
...
* trace recycle() calls fixes #810
2018-09-05 05:07:02 +09:00
2752bde683
Print to indicate what drone is doing while waiting for gossip
2018-09-04 13:45:08 -06:00
2aa72cc72e
Return a Vec from to_blobs() instead of using a mut parameter
2018-09-04 07:50:23 -10:00
8cc030ef84
Use Vec instead of VecDeque for SharedBlobs
2018-09-04 07:50:23 -10:00
9a9f89293a
Better error handling messages for airdrops
2018-09-04 06:46:43 -10:00
501deeef56
accounts should never be negative ( #1083 )
2018-09-04 06:43:18 -10:00
05f921d544
Don't call println in the test suite
2018-09-04 06:01:32 -10:00
ab7a2960b1
Don't use product name in solana library
2018-09-04 06:01:32 -10:00
4e2deaa33b
Less mut
2018-09-04 06:01:32 -10:00
d5ef18337c
Remove redundant return value
...
And don't log the same error twice.
2018-09-04 06:01:32 -10:00
d18ea501b7
Minimize unsafe code
2018-09-04 06:01:32 -10:00
c9a1ac9b8c
Don't propogate errors we'll never handle
2018-09-04 06:01:32 -10:00
c2a4cb544e
Borrow, don't clone entries
2018-09-04 06:01:32 -10:00
3ab12076e8
Convert voting functions to methods
...
More idiomatic Rust.
2018-09-04 05:53:58 -10:00
fb4b33b81b
make the repair_backoff test more robust ( #1095 )
...
* more the repair_backoff test more robust
* fix names and magic numbers
2018-08-31 12:40:56 -10:00
5c8cb96f88
rebase fixup
2018-08-31 23:21:07 +09:00
176e806d94
rework of netwrk rendezvous
...
* rename NodeInfo field of Node from "data" to "info"
(touches a lot of files)
* update client to use gossip to find leader, a la drone
* rework multinode scripts
* move more stuff into rust
* added usage to all
* no more rsync unless you're a validator (TODO: whack that, too)
* fullnode doesn't bail if drone isn't up yet, just keeps trying
* drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
eb4e5a7bd0
fixups
2018-08-31 23:21:07 +09:00
ba27596076
fixups
2018-08-31 23:21:07 +09:00
63e44dcc35
continue rendezvous refactor for gossip and repair
...
* remove trailing whitespace in ci/audit.sh
* code review fixups
* rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
* remove out-of-date TODO in localnet-sanity.sh
* remove features=test and code that was using it (localhost prohibitions in
crdt) added TODO in crdt.rs, maybe we should boot localhost in production
networks?
* boot tvu_window from NodeInfo: instead, send repair requests from the repair
socket (to gossip on peer) and answer repair requests via the sockaddr
from the repair request
* remove various unused pub functions
* banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
c0ba676658
fixup
2018-08-31 23:21:07 +09:00
1af4cee63b
fix #1079
...
* move gossip/NCP off assuming anything about its address
* use a single socket to send and receive gossip
* remove --addr/-a from CLIs
* rearrange networking utility code
* use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
* rename TestNode to Node
TODO:
* re-enable 127.0.0.1 as a valid address in crdt
* change repair request/response to a similar, single socket
* pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
* update contact_info with network truthiness instead of what the node
says?
2018-08-31 23:21:07 +09:00
e308a4279e
Update RPC requestAirdrop endpoint to return airdrop tx signature
2018-08-28 18:27:41 -06:00
513a934ff6
Update request_airdrop utility function to pass along airdrop tx signature
2018-08-28 18:27:41 -06:00
77d820c842
Update drone module to return airdrop tx signature
2018-08-28 18:27:41 -06:00
48762834d9
Randomize repair requests ( #1059 )
...
* randomize packet repair requests
* exponential random repair requests
* use gen_range to get a uniform distribution
2018-08-27 07:05:48 -07:00
8d0d429acd
update
2018-08-26 23:34:25 -07:00
e5408368f7
fmt
2018-08-26 22:35:26 -07:00
61492fd27e
exit if no leader
2018-08-26 22:35:26 -07:00
bbce08a67b
bench needs to discover leader as well
2018-08-26 22:35:26 -07:00
a002148098
retry transfer and poll
2018-08-26 16:10:46 -07:00
90ae662e4d
Fix packet header offset
...
And update transaction offsets to use the same approach as packet.rs.
Maybe this should be serialized_size(), but thanks to this
GenericArray update, those values are the same.
2018-08-26 14:27:19 -06:00
60d8f5489f
Update transaction layout offsets
...
24 less bytes in minimal transactions. 10% TPS boost?
2018-08-26 14:27:19 -06:00
738247ad44
advertise valid gossip address in drone and wallet ( #1066 )
...
* advertize valid gossip address in drone and wallet
get rid of asserts
check for valid ip address
check for valid address
ip address
* tests
* cleanup
* cleanup
* print error
* bump
* disable tests
* disable nightly
2018-08-26 11:36:27 -07:00
5b0bb7e607
Skip invalid nodes for finality ( #1068 )
...
* skip invalid nodes for finality
* check valid last_ids only
* fixup!
* fixup!
2018-08-25 23:12:41 -07:00
f7c0d30167
Disallow localhost in deployment ( #1064 )
...
* disallow localhost in deployment
* tests
* fmt
* integration tests do not have a flag to check
* fmt
2018-08-25 21:09:18 -07:00
8e98c7c9d6
fix purge test
2018-08-25 19:56:09 -07:00
50661e7b8d
Added poll_balance_with_timeout method ( #1062 )
...
* Added poll_balance_with_timeout method
- updated bench-tps, fullnode and wallet to use this method instead
of repeatedly calling poll_get_balance()
* Address review comments
- Revert some changes to use wrapper poll_get_balance()
* Reverting bench-tps to use poll_get_balance
- The original code is checking if the balance has been updated,
instead of just retrieving the balance. The logic is different
than poll_balance_with_timeout()
* Reverting wallet to use poll_get_balance
- The break condition in the loop is different than poll_balance_with_timeout().
It's checking if the balance has been updated.
2018-08-25 18:24:25 -07:00
ad159e0906
Fix crash in fullnode when poll_get_balance() returns error ( #1058 )
2018-08-25 15:25:13 -07:00
d3fac8a06f
Dynamically bind to available UDP ports in Fullnode ( #920 )
...
* Dynamically bind to available UDP ports in Fullnode
* Added tests for dynamic port binding
- Also removed hard coding of port range from CRDT
2018-08-25 10:24:16 -07:00
de379ed915
Fix sig verify counters to be unique and tweak perf counters ( #1056 )
...
print events and add current events to old value to report
2018-08-24 16:05:32 -07:00
6fc21a4223
Don't hang in transaction_count ( #1052 )
...
Situation is there can be that there can be bad entries in
the bench-tps CRDT table until they get purged later. Threads however
are created for those bad entries and then will hang on trying
to get the transaction_count from those bad addresses and never end.
2018-08-23 20:57:13 -07:00
71319978df
Up drone request amount ( #1051 )
...
Multiple clients will request 500k each so up this to support them.
2018-08-23 15:30:35 -07:00
6147e54686
Cap repair requests timeout ( #958 )
2018-08-23 15:30:21 -07:00
0c8eec2563
Cleanup Fullnode construction
...
leader_id was already set by Fullnode constructor. And cleanup the
rest of that code while in the neighborhood.
Thanks @CriesofCarrots!
2018-08-23 13:42:54 -07:00
4ab58f069a
Add back JsonRpcService changes
2018-08-23 13:42:54 -07:00
85f96d926a
Pacify clippy
2018-08-23 13:42:54 -07:00
816de4f8ec
Hoist shared code between leaders and validators
2018-08-23 13:42:54 -07:00
42229a1105
Hoist thread_hdls
2018-08-23 13:42:54 -07:00
d8820053af
Inline create_leader_threads and create_validator_threads
2018-08-23 13:42:54 -07:00
731f8512c6
Hoist Arc<Bank>
2018-08-23 13:42:54 -07:00
a133784706
Rename mode-specific constructors and return only thread handles
2018-08-23 13:42:54 -07:00
be58fdf1bb
Less constructors
2018-08-23 13:42:54 -07:00
57daeb35d2
Drop all references to new_leader and new_validator
2018-08-23 13:42:54 -07:00
9c5e69bf3d
Don't offer two ways to specify a leader
2018-08-23 13:42:54 -07:00
cfac127e4c
Extract lower-level constructor
...
Passing in the bank is useful for unit-tests since Fullnode doesn't
store it in a member variable.
2018-08-23 13:42:54 -07:00
fda4523cbf
Fix broken doc
2018-08-23 13:42:54 -07:00
cabe80b129
Increment counter by number of packets received ( #1049 )
...
So that we can see the total packets/s
2018-08-23 12:32:50 -07:00
d4c41219f9
Improve gossip use for drone and wallet
...
- Add utility function
- Add thread sleep
- Enable configurable timeout for gossip poll
2018-08-23 13:08:59 -06:00
4fdd9fbfca
Wallet: use gossip to identify leader's port config
2018-08-23 13:08:59 -06:00
bdf5ac9c1a
Drone: use gossip to identify leader's port config
2018-08-23 13:08:59 -06:00
f1785c76a4
Rework counter increment outside apply_debits loop ( #1046 )
...
Reduces prints/atomics work inside the process_transactions loop
2018-08-23 09:42:59 -07:00
2de8fe9c5f
Pass bank to rpc as reference
2018-08-23 09:06:17 -06:00
d910ed68a3
Use balance to verify requestAirdrop success
2018-08-23 09:06:17 -06:00
f7f7ecd4c6
Add json-rpc requestAirdrop endpoint
2018-08-23 09:06:17 -06:00
a9c3a28a3b
Add json-rpc sendTransaction endpoint
2018-08-23 09:06:17 -06:00
c3ed4d28de
Change average TPS to max average tps seen for any node and...
...
add script to collect perf stats
2018-08-22 14:55:04 -07:00
a355d9f46c
Add error catch for rpc server builder
2018-08-21 14:04:52 -06:00
be4824c955
Add custom panic hook for RPC port bind
2018-08-21 14:04:52 -06:00
86c1d97c13
Fix validator rpc addr to match leader
2018-08-20 22:35:06 -07:00
2d35345c50
Boot unused creates
2018-08-20 16:48:37 -06:00
a02910be32
Remove pubkey from getBalance response
2018-08-20 15:02:48 -07:00
b9ec97a30b
Add counter for bank transaction errors ( #1015 )
2018-08-20 14:56:01 -07:00
9eeaf2d502
Bind RPC port on all interfaces
2018-08-20 12:45:50 -07:00
8de317113c
clippy: remove identity conversion
2018-08-20 10:55:55 -07:00
1bf15ae907
Temporarily disable cargo audit CI failure
2018-08-18 12:29:49 -06:00
f73f3941cd
Revert ill-advised jsonrpc marker, and handle jsonrpc server close
2018-08-18 12:29:49 -06:00
d69d79612b
Simplify Rpc request processing
2018-08-18 12:29:49 -06:00
64ea5126e0
Fix early return for invalid parameter
2018-08-18 12:29:49 -06:00
9df3aa50d5
Remove unnecessary solana_ prefixes
2018-08-18 12:29:49 -06:00
cab75b7829
Handle potential panics
2018-08-18 12:29:49 -06:00
d9fac86015
Use jsonrpc git repo, allowing removal of Default bound for Metadata
2018-08-18 12:29:49 -06:00
1eb8724a89
Disable Rpc module for other tests to prevent port conflicts
2018-08-18 12:29:49 -06:00
c6662a4512
Implement Rpc in Fullnode
2018-08-18 12:29:49 -06:00
d3c09b4e96
Update jsonrpc dependency syntax
2018-08-18 12:29:49 -06:00
124f6e83d2
Rpc get last id endpoint
2018-08-18 12:29:49 -06:00
569ff73b39
Rpc tests
2018-08-18 12:29:49 -06:00
fc1dbddd93
Implement json-rpc functionality
2018-08-18 12:29:49 -06:00
3ae867bdd6
fixups
2018-08-18 02:22:52 -07:00
bc5f29150b
fix erasure, remove Entry "pad"
...
* fixes #997
* Entry pad is no longer required since erasure coding aligns data length
2018-08-18 02:22:52 -07:00
46016b8c7e
crashes generate_coding()
2018-08-18 02:22:52 -07:00
5dbecd6b6b
add logging, more conservative reset
2018-08-18 02:22:52 -07:00
208a7f16cb
Fix bench-tps nokey error
2018-08-16 19:38:26 -06:00
79d24ee227
fixed test according to @rob-solana
2018-08-16 14:44:51 -07:00
a284030ecc
Account type with state
...
comments
fixups!
fixups!
fixups for a real Result<> from get_balance()
on 2nd thought, be more rigorous
Merge branch 'rob-solana-accounts_with_state' into accounts_with_state
update
review comments
comments
get rid of option
2018-08-16 14:44:51 -07:00
88d6fea999
Revert "Accounts with state ( #954 )"
...
This reverts commit c23fa289c3
2018-08-15 19:44:39 -07:00
c23fa289c3
Accounts with state ( #954 )
...
* Account type with state
* fixed test according to @rob-solana
2018-08-15 14:32:11 -07:00
982afa87a6
Retransmit blobs from leader from window ( #975 )
...
- Some nodes don't have leader information while leader is broadcasting
blobs to those nodes. Such blobs are not retransmitted. This change
rertansmits the blobs once the leader's identity is know.
2018-08-14 21:51:37 -07:00
dccae18b53
cfg=erasure fixes, use return value of align!()
2018-08-14 12:14:59 -07:00
53e86f2fa2
use align!
2018-08-14 12:14:59 -07:00
708add0e64
fixups
2018-08-14 10:16:34 -07:00
d8991ae2ca
fix UPnP backout, fixes #969
2018-08-14 10:16:34 -07:00
5f6cbe0cf8
fixups
2018-08-13 21:07:26 -07:00
f167b0c2c5
fixups
2018-08-13 21:07:26 -07:00
f784500fbb
fixups
...
fixes #907
2018-08-13 21:07:26 -07:00
83df47323a
initialize recycled data
2018-08-13 21:07:26 -07:00
c75d4abb0b
Tuck away PoH duration
2018-08-13 20:17:16 -06:00
5216a723b1
Pacify clippy
2018-08-13 20:17:16 -06:00
b801ca477d
Declare fullnode a word
2018-08-13 20:17:16 -06:00
c830c604f4
Make BroadcastStage an actual stage
...
TODO: Why isn't BroadcastStage/RetransmitStage managed by the NCP?
2018-08-13 20:17:16 -06:00
0e66606c7f
Rename broadcaster to broadcast_stage
...
And move retransmitter code into retransmit_stage.
TODO: Add a BroadcastStage service
2018-08-13 20:17:16 -06:00
8707abe091
Fix erasure build
2018-08-13 20:17:16 -06:00
dc2a840985
Move FullNode::new_window into window module
2018-08-13 20:17:16 -06:00
2727067b94
Move winow into its own module
2018-08-13 20:17:16 -06:00
6a8a494f5d
Rename WindowStage to RetransmitStage
...
The window is used for both broadcasting from leader to validator
and retransmitting between validators.
2018-08-13 20:17:16 -06:00
a09d2e252a
Move window dependencies out of streamer
...
No tests!?
2018-08-13 20:17:16 -06:00
3e9c463ff1
Offer only 1 way to create a fullnode with an empty window
2018-08-13 20:17:16 -06:00
46d50f5bde
Remove p2p crate (and uPnP support)
2018-08-13 18:22:58 -07:00
e8da903c6c
move tmp_ledger back to target dir
2018-08-13 16:52:53 -07:00
c86e9e8568
pad max_data_size to jerasure's alignment requirements
2018-08-13 16:10:51 -07:00
835f29a178
off by 2
2018-08-13 15:12:12 -07:00
bf15cad36b
Add get_finality request and use it from multinode test ( #941 )
2018-08-13 08:55:13 -07:00
288ed7a8ea
Vote should be valid ( #945 )
...
* test that fails
* fix for test
* rename
2018-08-12 18:19:54 -07:00
f07c038266
Fix bank coalescing ( #949 )
...
* fix bank coalescing
* comments
* fix bench
* fix bench
* backout banking stage coalescing
* 120 nodes
* 100
2018-08-12 10:04:21 -07:00
5dbcb43abd
more enhancements
2018-08-10 19:53:58 -07:00
dd1eefaf62
change verify-internal to precheck
...
update to new ledger API
2018-08-10 19:53:58 -07:00
35de159d00
better error messages
2018-08-10 19:53:58 -07:00
546a1e90d5
clippy fixups
2018-08-10 19:53:58 -07:00
b033e1d904
enhance ledger-tool
...
* add json, which does the thing with json, move print to Rust's {:?}
* add --head NUM, to limit how much work gets done for print, json, verify
* add verify-internal, which very carefully checks ledger format without
trying first to "recover" it
* exit with errors on mis-usage
2018-08-10 19:53:58 -07:00
96d6985895
rework read_ledger, LedgerWriter, and LedgerWindow for recover()
...
fixes #910
2018-08-10 18:07:23 -07:00
a206f2570d
Add hostname to metrics on panic
2018-08-10 17:08:40 -07:00
2318ffc704
Use a different counter for validator account not found errors. ( #931 )
...
* Use a different counter for validator account not found errors. This is a usefull signal of something going wrong with the ledger
2018-08-10 15:18:44 -07:00
bbf9ea89c5
add some flushing to ledger
2018-08-10 11:41:31 -07:00
bca2294655
cargo fmt
2018-08-09 13:41:37 -06:00
4a980568ac
Rename sig variables to signature
...
We'll avoid introducing three-letter terms to free up the namespace
for three-letter acronyms.
But recognize the term "sigverify", a verb, to verify a digital
signature.
2018-08-09 13:41:37 -06:00
9d436fc5f8
Rename pk variables to pubkey
2018-08-09 13:41:37 -06:00
ad331e6d56
Rename PublicKey type to Pubkey
...
Recognize pubkey as a noun meaning the public key of a keypair.
2018-08-09 13:41:37 -06:00
d7e4e57548
Rename public_key variables to pubkey
2018-08-09 13:41:37 -06:00
b2067d2721
Rename kp variables to keypair
2018-08-09 13:41:37 -06:00
c2bbe4344e
Rename KeyPair to Keypair
2018-08-09 13:41:37 -06:00
ca7d4c42dd
Rename cur_hashes to num_hashes
2018-08-09 10:15:10 -06:00
ce8a4fa831
allow received to outpace window, we're already constraining repair
...
correctly identify sender in ledger_window repair responses, enabling re-transmission
2018-08-08 15:10:44 -07:00
8331aab26a
Enable Crdt debug messages to debug validators
2018-08-08 14:22:20 -07:00
9783d47fd1
write a "unit" test for WindowLedger (it was working ;)
...
clear flags on fresh blobs, lest they sometimes impersonate coding blobs...
fix bug: advance *received whether the blob_index is in the window or not,
failure to do so results in a stalled repair request pipeline
2018-08-08 04:28:09 -07:00
38be61bd22
Check for log level before doing perf counter work
...
Perf counters, especially when running the dynamic test can cause
functions like crdt::apply_updates to be really slow (>500ms).
2018-08-08 00:16:53 -07:00
c64e2acf8b
set destination address when for ledger window repair responses
2018-08-07 23:31:01 -07:00
a200cedb4b
Lower UDP data size to 64k - 128 bytes
...
rust API gives errors for packets larger than ~65500 and
wikipedia says 65507 is the max size, lowering this avoids the errors.
2018-08-07 18:39:36 -07:00
999534248b
fixups
2018-08-07 17:27:53 -07:00
fbc754ea25
plug in LedgerWindow
...
fixes #872
2018-08-07 17:27:53 -07:00
1b6d472cb2
Fixed counters for coalescing and broadcast index ( #900 )
2018-08-07 16:46:48 -07:00
2a0025bb57
get buffered IO back for ledger
2018-08-07 15:34:15 -07:00
64d6d3015a
Counters for broadcasted blob idx and coalesced packets ( #897 )
2018-08-07 14:54:26 -07:00
90550c5b58
Switch to slice arguments and remove clippy exceptions
2018-08-07 14:43:44 -07:00
1ac5d300a4
Rearrange start hash for process_ledger and add a unit test
2018-08-07 14:10:36 -07:00
16772d3d51
Coalesce multiple blobs if received close together ( #869 )
...
- This helps reduce unnecessary growth of window if small blobs
are received in small space of time
2018-08-07 10:29:57 -07:00
ceb5a76609
Refactor validator windowing
...
- a unit test for windowing functions
- issue #857
2018-08-07 08:17:32 -07:00
db2392a691
Use last_id from the entries stream instead of last_id from bank
...
bank will only register ids when has_more is not set because those are
the only ids it has advertised, so it will not register all ids,
however the entry stream will contain unbroken last_id chain, so we
need to track that to get the correct start hash.
2018-08-07 08:14:06 -07:00
575179be8e
y
2018-08-06 23:55:00 -07:00
5b6ffaecc0
s/r$/f/
2018-08-06 23:36:09 -07:00
efc72b9572
Support -V/--version on all CLI apps
...
All CLI apps that use clap (in other words, except for bench-streamer)
can use crate_version! to take the version from Cargo.toml.
This change addresses #700 .
2018-08-06 22:03:58 -07:00
5dc7177540
Remove manually created help text, use clap's text instead.
2018-08-06 19:15:52 -07:00
78a4b1287d
Initialize logger
2018-08-06 19:04:04 -07:00
c5001869f1
Add verify subcommand
2018-08-06 19:04:04 -07:00
7c31f217d5
Add voting metric even when there are not enough validators
2018-08-06 15:58:10 -07:00
3beb38ac8a
/tmp/farf no good on multi-user machine
2018-08-06 14:53:40 -07:00
8cbaa19d2e
Report the address that failed to bind
2018-08-06 11:21:25 -07:00
63d2b2eb42
adjust bank notion of entry_count to aid debugging
2018-08-06 11:20:52 -07:00
e02da9a15a
Clean up tx_count usage
2018-08-06 11:00:25 -07:00
ae111a131c
Condense stdout
2018-08-06 11:00:25 -07:00
4402e1128f
Cleanup
2018-08-06 11:00:25 -07:00
f55bb6d95c
Send/confirm a loopback payment after each batch of transactions
2018-08-06 11:00:25 -07:00
0514f5e573
sync() apparently imposes a serious performance penalty
2018-08-06 08:51:41 -07:00
637d403415
move bank.process_entries() to firsties
2018-08-06 08:51:41 -07:00
9fabd34156
remove trace! calls, re-arrange replicate_requests to have the bank earlier
2018-08-06 08:51:41 -07:00
039ed01abf
on 2nd thought: do not copy_ledger() for this test
2018-08-06 08:51:41 -07:00
ead0eb2754
move copy_ledger() back into ledger.rs
...
Don't recover() for copy(), as copy() is already tolerant of things
recover() guards against. Note: recover() is problematic if the ledger is
"live", i.e. is currently being written to.
2018-08-06 08:51:41 -07:00
c3db2df7eb
tweak random access ledger
...
* add recover_ledger() to deal with expected common ledger corruptions
* add verify_ledger() for future use cases (ledger-tool)
* increase ledger testing
* allow replicate stage to run without a ledger
* ledger-tool to output valid json
2018-08-06 08:51:41 -07:00
ee6c15d2db
start on ledger recovery with a description of what that might mean
2018-08-06 08:51:41 -07:00
715a3d50fe
Revert "Revert "clippy fixup""
...
This reverts commit d173e6ef87
2018-08-06 08:51:41 -07:00
692b125391
Revert "Revert "fixups""
...
This reverts commit e2c68d8775
2018-08-06 08:51:41 -07:00
5193819d8e
Revert "Revert "plug in new ledger""
...
This reverts commit 57e928d1d0
2018-08-06 08:51:41 -07:00
210b9d346f
Add voting metrics and -h/--help to get usage for client.sh script
2018-08-05 14:21:49 -07:00
4c4b0f551e
clippy fixups
2018-08-05 13:30:45 -07:00
6800ff1882
solana-ledger-tool initial commit
...
does nothing but convert from random-access ledger to json
2018-08-05 13:30:45 -07:00
399a3852b1
Add sigverify_stage-total_verify_time datapoint
2018-08-04 21:45:58 -07:00
906d18a709
move VOTE to trace, info too verbose
2018-08-03 23:04:54 -07:00
7fd7310b96
Prevent a node from overrunning it's receive window ( #846 )
...
- The node drops blobs that will cause it to overrun window
- The node does not ask to repair a blob that overruns the window
2018-08-03 20:15:14 -07:00
1a9e6ffdd7
Try multiple times to confirm a non-zero balance
2018-08-03 19:57:38 -07:00
4aab413154
recycle the skipped, outside-window blob, fixes #843
2018-08-03 15:02:55 -07:00
f2b1a04bca
cargo fmt fixups
2018-08-03 11:59:25 -07:00
57e928d1d0
Revert "plug in new ledger"
...
This reverts commit 46d9ba5ca0
2018-08-03 10:24:51 -07:00
e2c68d8775
Revert "fixups"
...
This reverts commit b72e91f681
2018-08-03 10:24:51 -07:00
d173e6ef87
Revert "clippy fixup"
...
This reverts commit 384b486b29
2018-08-03 10:24:51 -07:00
384b486b29
clippy fixup
2018-08-02 21:50:47 -07:00
b72e91f681
fixups
2018-08-02 21:50:47 -07:00
46d9ba5ca0
plug in new ledger
2018-08-02 21:50:47 -07:00
f570ef1c66
Defer repair request for blobs that may still be in avalanche transit ( #814 )
2018-08-02 19:12:57 -07:00
ee0195d588
Try to measure finality from time seen to when 2/3 of validator..
...
..set has voted. Add a timestamp to last_ids and use that to
see how long from when 2/3s validator set has voted on them.
2018-08-02 13:21:29 -07:00
448b8b1c17
Add Hash wrapper and supporting traits
2018-08-01 17:00:51 -07:00
4d77fa900b
Add Signature wrapper and supporting traits
2018-08-01 17:00:51 -07:00
7ccd771ccc
Only send sigverify to GPU if batch size is >64
...
Seems to be a decent crossover point for Xeon E5-2620 v4 8c,16t vs. nvidia 1080ti
2018-08-01 16:38:15 -07:00
aa9bc57b4d
Implement GenKeys without SecureRandom
2018-08-01 16:24:47 -07:00
11df477b20
Make GenKey functions mut
...
We hide the mutability to implement SecureRandom, but that's going
away.
2018-08-01 16:24:47 -07:00
7141750668
new_key -> gen_keypair
2018-08-01 16:24:47 -07:00
68675bd1ab
Less pub
2018-08-01 16:24:47 -07:00
19b3cacd60
Generate a fixed-size array instead of a vector
2018-08-01 16:24:47 -07:00
bcfaf5d994
Rebase ledger change
2018-08-01 16:15:14 -07:00
e9499ac5b8
Update PublicKey AsRef to slice
2018-08-01 16:15:14 -07:00
7ff721e563
Replace pub field with AsRef impl
2018-08-01 16:15:14 -07:00
cf70e5ff2f
Handle wrapped PublicKey struct
2018-08-01 16:15:14 -07:00
a86618faf3
Add PublicKey wrapper
...
Add custom formatting for PublicKey display and debug
2018-08-01 16:15:14 -07:00
6693386bc5
Lower errors to warnings so they don't print during tests
...
Negative tests should trigger the warnings, but errors look like
something is wrong.
2018-08-01 16:56:12 -06:00
4a8a0d03a3
Correct localhost address
2018-08-01 15:49:48 -07:00
5cda0ed964
Airdrop from the leader
2018-08-01 15:21:20 -07:00
0aba74935b
fixups
2018-08-01 14:42:58 -07:00
4eb666d4f9
provide ledger::copy()
2018-08-01 14:42:58 -07:00
d5e0cf81ff
fixups
2018-08-01 14:42:58 -07:00
3ea784aff7
clippy fixups
2018-08-01 14:42:58 -07:00
fef93958c8
fixups, tests
2018-08-01 14:42:58 -07:00
cae88c90b1
add a persistent ledger of index and data files
2018-08-01 14:42:58 -07:00
1a8da769b6
...
2018-08-01 14:42:58 -07:00
ff254fbe5f
re-instate traces
2018-08-01 09:08:38 -07:00
05153e4884
de-trace this function, new blob is not a dup
2018-08-01 09:08:38 -07:00
2ece27ee3a
fix leak
2018-08-01 09:08:38 -07:00
a58df52205
Fix build
...
Last two PRs crossed in flight. A keypair is now required for all
types of FullNode, not just validators.
2018-08-01 08:53:21 -07:00
2ea6f86199
Submit leader's vote after observing 2/3 validator votes ( #780 )
...
* fixup!
* fixups!
* send the vote and count it
* actually vote
* test
* Spelling fixes
* Process the voting transaction in the leader's bank
* Send tokens to the leader
* Give leader tokens in more cases
* Test for write_stage::leader_vote
* Request airdrop inside fullnode and not the script
* Change readme to indicate that drone should be up before leader
And start drone before leader in snap scripts
* Rename _kp => _keypair for keypairs and other review fixups
* Remove empty else
* tweak test_leader_vote numbers to be closer to testing 2/3 boundary
* combine creating blob and transaction for leader/validator
2018-07-31 22:07:53 -07:00
7c5172a65e
Converted sigverify disable flag to runtime check instead of "cfg" ( #799 )
2018-07-31 16:54:24 -07:00
821e3bc3ca
Avoid race between test_lograte and test_lograte_env
2018-07-31 16:08:01 -07:00
5dd2f737a3
clear out old blobs in find_next_missing
2018-07-31 15:54:32 -07:00
308b6c3371
Follow Shared prefix convention for Window alias ( #798 )
...
Follow Shared prefix convention for Window alias.
2018-07-30 16:56:01 -07:00
ea7fa11b3e
use size_of() instead of serialized_size() and magic number 8
2018-07-30 16:48:58 -07:00
95b98b3845
Fix --addr option
2018-07-30 16:36:26 -07:00
f27613754a
Report number of nodes found on failure too
2018-07-30 16:36:26 -07:00
79ece53e3c
Don't panic the tokio worker thread when deserialize() fails
2018-07-30 14:56:53 -07:00
f341b2ec10
fixups
2018-07-30 14:26:44 -07:00
167b079e29
fixups
2018-07-30 14:26:44 -07:00
7ded5a70be
fixups
2018-07-30 14:26:44 -07:00
fc476ff979
implement iterator for parsing length + data ledger
2018-07-30 14:26:44 -07:00
c3279c8a00
chugga
2018-07-30 14:26:44 -07:00
e471ea41da
fixups
2018-07-30 14:26:44 -07:00
552d4adff5
use a binary ledger: newline-separated, newline-escaped entries instead of json
2018-07-30 14:26:44 -07:00
0c33c9e0d7
Dynamic network test changes ( #795 )
...
- No sigverify if feature sigverify_cpu_disable is used
- Purge validators in the test if lag count increases beyond
SOLANA_DYNAMIC_NODES_PURGE_LAG environment variable
- Other useful log messages in the test
2018-07-30 13:57:10 -07:00
fae9fff24c
Unify logging initialization
2018-07-29 19:08:27 -07:00
79924e407c
Include nanoseconds in log timestamp
2018-07-29 19:08:27 -07:00
af1a2e83bc
Don't panic again when waiting for a panicked validator thread
2018-07-28 16:35:35 -07:00
029e2db2cf
Improve assert message
2018-07-28 10:40:50 -07:00
0b47404ba6
Check for default leader and use cmp::max for a bit nicer code ( #779 )
2018-07-27 15:53:31 -07:00
50e1e0ae47
use rust's rotate (in place, yay!)
2018-07-27 11:44:02 -07:00
538c3b63e1
Log the last_id being voted on
2018-07-27 11:27:51 -07:00
678b2870ff
i
2018-07-27 11:11:37 -07:00
308d8c254d
poll_get_balance no longer fails intermittently for zero balance accounts
...
While polling for a non-zero balance, it's not uncommon for one of the
get_balance requests to fail with EWOULDBLOCK. Previously when a get_balance
request failure occurred on the last iteration of the polling loop,
poll_get_balance returned an error even though the N-1 iterations may have
successfully retrieved a balance of 0.
2018-07-26 21:41:07 -07:00
f11aa4a57b
Ensure non-zero exit code if 'balance' command fails
2018-07-26 21:41:07 -07:00
7672506b45
Validators now vote once a second regardless
2018-07-26 17:07:42 -07:00
dd819cec3d
fix off by one in packet.rs
2018-07-26 09:24:44 -07:00
cbb8dee360
rework broadcast to understand a separate transmit index for coding blobs
2018-07-25 20:45:16 -07:00
e0cdcb0973
employ the simple choice for broadcast table of coding blobs: round-robin
2018-07-25 20:45:16 -07:00
a6a2a745ae
fix broadcast of erasure coding blobs
...
erasure coding blobs were being counted as window slots, skewing transmit_index
erasure coding blobs were being skipped over for broadcast, because they're
only generated when the last data blob in an erasure block is added to the
window.... rewind the index to pick up and broadcast those coding blobs
2018-07-25 20:45:16 -07:00
f372840354
Collect some datapoints while bench-tps is running
2018-07-25 20:15:43 -07:00
212874e155
Use BlobError for get_size return
2018-07-25 15:54:04 -07:00
75212f40e7
fix off by one for send_to() of blob
2018-07-25 15:16:56 -07:00
6fde65577e
fixes #756
2018-07-25 11:07:03 -07:00
edf2ffaf4e
Reduce complexity of main for clippy
...
...and readability
2018-07-25 10:16:46 -07:00
6c275ea5ef
More knobs. Arg for tx count per batch and also sustained mode
...
sustained mode overlaps tx generation with transfer. This mode seems
to have lower peak performance but higher average performance
2018-07-25 10:16:46 -07:00
23ed65b339
Transfer and sign at the same time in bench-tps
2018-07-25 10:16:46 -07:00
9c7913ac9e
trying to raise an error
2018-07-25 08:12:20 -07:00
8b01e6ac0b
implement Blob::get_size(), the counterpart of Blob::set_size()
2018-07-25 08:12:20 -07:00
ff5854396a
deserialize using get_data_size(), which refers to blob.data()'s length,
...
instead of using msg.meta.size, which refers to the entire blob's length
fixes #752
2018-07-25 08:12:20 -07:00
f0725b4900
Avoid panicking if poll_get_balance() fails while in the transaction loop
2018-07-24 23:31:28 -07:00
327ba5301d
Log token balance throughout the transfer loop
2018-07-24 22:40:12 -07:00
aa2104a21b
Reclaim tokens before exiting to avoid leaking tokens
2018-07-24 21:40:02 -07:00
0206020104
Make airdrops more robust
2018-07-24 21:40:02 -07:00
33bd1229d9
make next_entries() smarter about fitting Transactions into a Blob
2018-07-24 21:38:06 -07:00
195098ca2b
Failure test case
2018-07-24 21:38:06 -07:00
6bd18e18ea
Add error messages to ledger verify
2018-07-24 17:35:41 -07:00
2b528e2225
fixups
2018-07-24 13:04:34 -07:00
75505bbd72
fixups
2018-07-24 13:04:34 -07:00
e1fc7444f9
fixups
2018-07-24 13:04:34 -07:00
fcdb0403ba
eliminate unused parameter received, this branch fixes #636
2018-07-24 13:04:34 -07:00
caeb55d066
placate clippy and reduce replicode
2018-07-24 13:04:34 -07:00
f11e60b801
fix major bug: re-used blobs need to have their flags cleared
...
plus: lots of additional debug-ability
2018-07-24 13:04:34 -07:00
54f2146429
fixups
2018-07-24 13:04:34 -07:00
f60ee87a52
zero the tails of data blobs during generate() and recover() to enable blob reuse
2018-07-24 13:04:34 -07:00
9c06fe25df
enhance unit test to fail when erasure encodes stray bytes of data blobs
2018-07-24 13:04:34 -07:00
1eec8bf57f
fixups
2018-07-24 13:04:34 -07:00
ddb24ebb61
fixups
2018-07-24 13:04:34 -07:00
a58c83d999
prevent infinite loop on window wraparound
2018-07-24 13:04:34 -07:00
6656ec816c
protect generate and recover from u64->usize casting issues
2018-07-24 13:04:34 -07:00
8d2bd43100
fixups
2018-07-24 13:04:34 -07:00
429ea98ace
mutable-coding-blocks
2018-07-24 13:04:34 -07:00
3d80926508
fixups
2018-07-24 13:04:34 -07:00
d713e3c2cf
send coding in broadcast(), fixups
2018-07-24 13:04:34 -07:00
Tyera Eulberg
sakridge
Pankaj Garg
carllin
Michael Vines
Rob Walker
Greg Fitzgerald
anatoly yakovenko
Stephen Akridge
Anatoly Yakovenko
Tyera Eulberg
Kazuyoshi Kato