Commit Graph

193 Commits

Author SHA1 Message Date
Pankaj Garg f6f0900506
Large network test to not poll validator for sigs (#998)
- The finality is already reached. The test will check the signature
  in validators once, instead of polling. This will help speed up the test.
2018-08-17 14:38:19 -07:00
Anatoly Yakovenko a284030ecc Account type with state
comments

fixups!

fixups!

fixups for a real Result<> from get_balance()

on 2nd thought, be more rigorous

Merge branch 'rob-solana-accounts_with_state' into accounts_with_state

update

review comments

comments

get rid of option
2018-08-16 14:44:51 -07:00
Pankaj Garg c2983f824e
Refactored large network test to use finality to assert success (#978) 2018-08-15 20:05:43 -07:00
anatoly yakovenko 88d6fea999 Revert "Accounts with state (#954)"
This reverts commit c23fa289c3.
2018-08-15 19:44:39 -07:00
anatoly yakovenko c23fa289c3
Accounts with state (#954)
* Account type with state

* fixed test according to @rob-solana
2018-08-15 14:32:11 -07:00
Pankaj Garg db35f220f7
Run multinode test for enough iterations for a small node count test (#971) 2018-08-15 10:44:14 -07:00
Pankaj Garg 982afa87a6
Retransmit blobs from leader from window (#975)
- Some nodes don't have leader information while leader is broadcasting
  blobs to those nodes. Such blobs are not retransmitted. This change
  rertansmits the blobs once the leader's identity is know.
2018-08-14 21:51:37 -07:00
Greg Fitzgerald b801ca477d Declare fullnode a word 2018-08-13 20:17:16 -06:00
Greg Fitzgerald 2727067b94 Move winow into its own module 2018-08-13 20:17:16 -06:00
Pankaj Garg bf15cad36b
Add get_finality request and use it from multinode test (#941) 2018-08-13 08:55:13 -07:00
anatoly yakovenko f07c038266
Fix bank coalescing (#949)
* fix bank coalescing

* comments

* fix bench

* fix bench

* backout banking stage coalescing

* 120 nodes

* 100
2018-08-12 10:04:21 -07:00
Rob Walker 96d6985895 rework read_ledger, LedgerWriter, and LedgerWindow for recover()
fixes #910
2018-08-10 18:07:23 -07:00
Greg Fitzgerald ad331e6d56 Rename PublicKey type to Pubkey
Recognize pubkey as a noun meaning the public key of a keypair.
2018-08-09 13:41:37 -06:00
Greg Fitzgerald c2bbe4344e Rename KeyPair to Keypair 2018-08-09 13:41:37 -06:00
Greg Fitzgerald 8567253833 Ignore flaky test 2018-08-09 10:15:10 -06:00
Pankaj Garg a6857dbaaa Updated node count to 230. Increased wmem on CI large 2018-08-08 13:13:18 -07:00
pgarg66 75854cc234
Update dynamic network test with more nodes (#904)
- Check for correct OS params in test-large-network.sh
2018-08-08 06:52:57 -07:00
Rob Walker 9783d47fd1 write a "unit" test for WindowLedger (it was working ;)
clear flags on fresh blobs, lest they sometimes impersonate coding blobs...

fix bug: advance *received whether the blob_index is in the window or not,
  failure to do so results in a stalled repair request pipeline
2018-08-08 04:28:09 -07:00
Stephen Akridge 38be61bd22 Check for log level before doing perf counter work
Perf counters, especially when running the dynamic test can cause
functions like crdt::apply_updates to be really slow (>500ms).
2018-08-08 00:16:53 -07:00
Rob Walker fbc754ea25 plug in LedgerWindow
fixes #872
2018-08-07 17:27:53 -07:00
pgarg66 16772d3d51
Coalesce multiple blobs if received close together (#869)
- This helps reduce unnecessary growth of window if small blobs
  are received in small space of time
2018-08-07 10:29:57 -07:00
Kazuyoshi Kato 9c1b6288a4 Use ? instead of unwrap()
This change addresses #833, while there are still some unwrap() though.
2018-08-07 08:10:22 -07:00
Rob Walker 3beb38ac8a /tmp/farf no good on multi-user machine 2018-08-06 14:53:40 -07:00
Rob Walker 039ed01abf on 2nd thought: do not copy_ledger() for this test 2018-08-06 08:51:41 -07:00
Rob Walker ead0eb2754 move copy_ledger() back into ledger.rs
Don't recover() for copy(), as copy() is already tolerant of things
    recover() guards against.  Note: recover() is problematic if the ledger is
    "live", i.e. is currently being written to.
2018-08-06 08:51:41 -07:00
Rob Walker c3db2df7eb tweak random access ledger
* add recover_ledger() to deal with expected common ledger corruptions
  * add verify_ledger() for future use cases (ledger-tool)
  * increase ledger testing
  * allow replicate stage to run without a ledger
  * ledger-tool to output valid json
2018-08-06 08:51:41 -07:00
Rob Walker 5193819d8e Revert "Revert "plug in new ledger""
This reverts commit 57e928d1d0.
2018-08-06 08:51:41 -07:00
Rob Walker f2b1a04bca cargo fmt fixups 2018-08-03 11:59:25 -07:00
Rob Walker 57e928d1d0 Revert "plug in new ledger"
This reverts commit 46d9ba5ca0.
2018-08-03 10:24:51 -07:00
Rob Walker 46d9ba5ca0 plug in new ledger 2018-08-02 21:50:47 -07:00
Tyera Eulberg 7ff721e563 Replace pub field with AsRef impl 2018-08-01 16:15:14 -07:00
Tyera Eulberg fda3b9bbd4 Use new PublicKey format instead of hex 2018-08-01 16:15:14 -07:00
Tyera Eulberg cf70e5ff2f Handle wrapped PublicKey struct 2018-08-01 16:15:14 -07:00
Pankaj Garg d622742b84 Mark test-multinode-basic as ignore 2018-08-01 10:13:05 -06:00
sakridge 2ea6f86199 Submit leader's vote after observing 2/3 validator votes (#780)
* fixup!

* fixups!

* send the vote and count it

* actually vote

* test

* Spelling fixes

* Process the voting transaction in the leader's bank

* Send tokens to the leader

* Give leader tokens in more cases

* Test for write_stage::leader_vote

* Request airdrop inside fullnode and not the script

* Change readme to indicate that drone should be up before leader

And start drone before leader in snap scripts

* Rename _kp => _keypair for keypairs and other review fixups

* Remove empty else
* tweak test_leader_vote numbers to be closer to testing 2/3 boundary
* combine creating blob and transaction for leader/validator
2018-07-31 22:07:53 -07:00
pgarg66 7c5172a65e
Converted sigverify disable flag to runtime check instead of "cfg" (#799) 2018-07-31 16:54:24 -07:00
pgarg66 0c33c9e0d7
Dynamic network test changes (#795)
- No sigverify if feature sigverify_cpu_disable is used
- Purge validators in the test if lag count increases beyond
  SOLANA_DYNAMIC_NODES_PURGE_LAG environment variable
- Other useful log messages in the test
2018-07-30 13:57:10 -07:00
sakridge 0b47404ba6 Check for default leader and use cmp::max for a bit nicer code (#779) 2018-07-27 15:53:31 -07:00
Stephen Akridge 7f4844f426 More stats in dynamic multinode test 2018-07-27 11:55:09 -07:00
Michael Vines c52d4eca0b Stop validator first to stop voting before the leader stops 2018-07-26 17:29:32 -07:00
Michael Vines 7672506b45 Validators now vote once a second regardless 2018-07-26 17:07:42 -07:00
Rob Walker 297896bc49 honor environment variable SOLANA_DYNAMIC_NODES, obviating need to edit+compile for re-test 2018-07-25 20:45:16 -07:00
pgarg66 9daa7bdbe2
Replace rayon with threads for dynamic network test (#745) 2018-07-24 17:54:29 -07:00
pgarg66 166099b9d9
Start validators in parallel in multinode test (#727) 2018-07-23 09:27:06 -07:00
Martius Lim 84225beeef replace 'daddr' checks with 'is_valid_address()' 2018-07-20 13:09:01 -04:00
Greg Fitzgerald 28af9a39b4 Don't clone before borrowing
Clippy told us to change function parameters to references, but
wasn't able to then tell us that the clone() before borrowing
was superfluous. This patch removes those by hand.

No expectation of a performance improvement here, since we were
just cloning reference counts. Just removes a bunch of noise.
2018-07-18 08:04:31 -04:00
Anatoly Yakovenko 3d45b04da8 review comments 2018-07-17 15:51:32 -04:00
Anatoly Yakovenko 9e2f26a5d2 review comments 2018-07-17 15:51:32 -04:00
Anatoly Yakovenko a016f6e82e bulds 2018-07-17 15:51:32 -04:00
Anatoly Yakovenko eb3e5fd204 server too 2018-07-17 15:51:32 -04:00
Anatoly Yakovenko 72282dc493 fast exit dynamic test 2018-07-17 15:51:32 -04:00
anatoly yakovenko d8c9655128
Dynamic test assert (#643)
* log responder error to warn

* log responder error to warn

* fixup!

* fixed assert

* fixed bad ports issue

* comments

* test for dummy address in Crdt::new instaad of NodeInfo::new

* return error if ContactInfo supplied to Crdt::new cannot be used to connect to network

* comments
2018-07-16 19:31:52 -07:00
Rob Walker 710fa822a0 fixups 2018-07-12 22:51:55 -07:00
Rob Walker a3ff40476e Banish stdin/stdout for ledger
step one: accept a "ledger file" argument instead of "outfile"
2018-07-12 22:51:55 -07:00
Greg Fitzgerald 30f0c25b65 Fix all remaining clippy warnings
Fixes #586
2018-07-12 09:40:40 -06:00
OEM Configuration (temporary user) 468ac9facd Refactor the "ReplicatedData" struct
Rename the "ReplicatedData" struct to the "NodeInfo" struct.
Also refactors and renames the members in this struct.
2018-07-11 11:38:41 -06:00
Anatoly Yakovenko 7687436bef some cleanup on messages 2018-07-10 13:32:31 -06:00
Anatoly Yakovenko 6a1b5a222a rebase builds 2018-07-10 13:32:31 -06:00
Anatoly Yakovenko be2bf69c93 initial vote stage
wip

voting

wip

move voting into the replicate stage

update

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

fixup!

update

fixup!

fixup!

fixup!

tpu processing votes in entries before write stage

fixup!

fixup!

txs

make sure validators have an account

fixup!

fixup!

fixup!

exit fullnode correctly

exit on exit not err

try 50

add delay for voting

300

300

startup logs

par start

100

no rayon

retry longer

log leader drop

fix distance

50 nodes

100

handle deserialize error

update

fix broadcast

new table every time

tweaks

table

update

try shuffle table

skip kill

skip add

purge test

fixed tests

rebase 2

fixed tests

fixed rebase

cleanup

ok for blobs to be longer then window

fix init window

60 nodes
2018-07-10 13:32:31 -06:00
Greg Fitzgerald 0672794692 Cleanup leader restarts
Try to avoid adding a runtime codepath when all paths are
statically known.
2018-07-10 11:11:36 -06:00
Greg Fitzgerald c65c0d9b23 Expose fewer exit variables 2018-07-10 11:11:36 -06:00
Anatoly Yakovenko 63985d4595 renamed to contact_info 2018-07-09 20:40:14 -06:00
Anatoly Yakovenko 97dd1834d7 fix tests, more logs 2018-07-09 20:40:14 -06:00
Anatoly Yakovenko 2ea030be48 stick all the addrs into one struct 2018-07-09 20:40:14 -06:00
Rob Walker 90a4ab7e57 fixes issue #299 2018-07-09 14:50:14 -07:00
Rob Walker 412e15fbdc add test for populated window 2018-07-09 14:50:14 -07:00
Rob Walker ed0a590549 support an initial window filled with last up-to-WINDOW_SIZE blobs 2018-07-09 14:50:14 -07:00
Greg Fitzgerald 9c456b2fb0 Fixup the integration tests 2018-07-04 16:40:34 -06:00
Anatoly Yakovenko 22d2c962b2 ignore 2018-07-03 18:10:16 -06:00
Anatoly Yakovenko e771d36278 Better logs 2018-07-03 18:10:16 -06:00
Rob Walker 800c2dd370 make the leader append to the ledger file 2018-07-03 17:17:52 -06:00
Anatoly Yakovenko ae309f80f7 boot from file test 2018-07-03 14:00:39 -06:00
Anatoly Yakovenko fa70b3bf70 split out files, fixed a bug @garious! 2018-07-03 14:00:39 -06:00
Anatoly Yakovenko 3a90f138b2 dynamit network test
* cleaned up fullnode api
* added debug_id to ReplicatedData and crdt for debugging
2018-07-03 14:00:39 -06:00
Anatoly Yakovenko b8fe5ae076 rename server to fullnode 2018-07-02 16:34:49 -06:00
Michael Vines 3e0e09555a Undo UPnP UDP port binding 2018-07-02 14:38:01 -07:00
Michael Vines 0b56d603c2 Client NAT traversal 0.1
UPnP is now used to request a port on the NAT be forwarded to the local machine.
This obviously only works for NATs that support UPnP, and thus is not a panacea
for all NAT-related connectivity issues.

Notable hacks in this patch include a transmit/receive UDP socket pair to work
around current protocol limitations whereby the full node assumes its peer can
receive on the same UDP port it transmitted from.
2018-06-29 17:36:26 -07:00
Stephen Akridge f3df5df52c add validator catchup to multi-node test 2018-06-29 10:39:41 -07:00
Rob Walker 2f42658cd4 ... 2018-06-27 14:51:18 -07:00
OEM Configuration (temporary user) 551f639259 Some pull request fixes(linting + documentation) 2018-06-26 12:31:04 -06:00
OEM Configuration (temporary user) 08bcb62016 added remote table to update respones 2018-06-26 12:31:04 -06:00
Greg Fitzgerald f2198c2e9a cargo fmt
rustc 1.27.0

```
$ cargo fmt --version
rustfmt 0.6.1-stable (49279d71 2018-05-08)
```
2018-06-22 22:23:55 -07:00
Greg Fitzgerald 7aa05618a3 data_replicator -> ncp
Fixes #327
2018-06-07 17:11:17 -06:00
Greg Fitzgerald 7e788d3a17 No longer need explicit refs in rustc 1.26 2018-06-04 21:43:46 -06:00
anatoly yakovenko 216510c573
repair socket and receiver thread (#303)
repair socket and receiver thread
2018-06-02 08:32:51 -07:00
Greg Fitzgerald c9fd5d74b5 Boot futures 0.1
We added them thinking it'd be a good stepping stone towards an
asynchronous thin client, but it's used inconsistently and where
it used, the function is still synchronous, which is just confusing.
2018-05-31 14:13:09 -06:00
Greg Fitzgerald 60609a44ba Initialize recorder from bank's last_id 2018-05-31 12:13:18 -06:00
Anatoly Yakovenko e44e81bd17 fmt 2018-05-30 14:04:48 -06:00
Anatoly Yakovenko f5eedd2d19 fmt 2018-05-30 14:04:48 -06:00
Anatoly Yakovenko f1311075d9 integration tests 2018-05-30 14:04:48 -06:00
Anatoly Yakovenko cef1c208a5 Crdt pipeline, coalesce window repair requests in the listener by examining all of them at once, and ublock those threads from doing io. 2018-05-30 14:04:48 -06:00
Greg Fitzgerald 9c9c63572b cargo fmt
rustfmt was updated with 1.26.1
2018-05-29 20:33:45 -07:00
Greg Fitzgerald fc00594ea4 Move multinode test to integration tests 2018-05-26 00:36:50 -06:00