Commit Graph

155 Commits

Author SHA1 Message Date
Greg Fitzgerald b95db62be3 Handle errors consistently
Error handling is still clumsy. We should switch to something like
`error-chain` or `Result<T, Box<Error>>`, but until then, we can
at least be consistent across modules.
2018-07-05 17:32:41 -06:00
Anatoly Yakovenko e771d36278 Better logs 2018-07-03 18:10:16 -06:00
Anatoly Yakovenko 3a90f138b2 dynamit network test
* cleaned up fullnode api
* added debug_id to ReplicatedData and crdt for debugging
2018-07-03 14:00:39 -06:00
Anatoly Yakovenko 2952027d04 wtfr 2018-07-02 16:34:49 -06:00
Stephen Akridge 8effa4e3e0 Clear old blobs before putting in the new one
Otherwise we will just warn about overrun and not insert new blob
Also, break if the index we find is less than consumed otherwise
we can infinite loop
2018-06-29 07:14:47 -06:00
Stephen Akridge 799b249f02 Don't null blob window until we have to 2018-06-29 07:14:47 -06:00
Rob Walker 2f42658cd4 ... 2018-06-27 14:51:18 -07:00
Rob Walker d95e8030fc ... 2018-06-27 14:51:18 -07:00
Greg Fitzgerald 4aedd3f1b6 Cleanup type aliases and imports 2018-06-27 15:06:18 -06:00
Stephen Akridge f4c4b9df9c Only free in replicate if we did not hold the reference in window stage
And then free when we are consuming blobs
2018-06-26 16:51:07 -06:00
Greg Fitzgerald 6e568c69a7 Preemptive strike
Should that blob have been passed to a recycler, it would have
had too high a reference count.
2018-06-26 16:51:07 -06:00
Greg Fitzgerald 14d624ee40 Fix benchmarks too
This change will make these benchmarks way slower, because its now
cloning the transaction vector each iteration instead of the ref
counts. We need to rethink these.
2018-06-26 16:51:07 -06:00
Greg Fitzgerald 1691060a22 Assert recycler is given last reference to data
This patch likely fixes the sporadic failures in the following tests:

```
test server::tests::validator_exit ... FAILED
test streamer::test::streamer_send_test ... FAILED
test thin_client::tests::test_bad_sig ... FAILED
test drone::tests::test_send_airdrop ... FAILED
test thin_client::tests::test_thin_client ... FAILED
```
2018-06-26 16:51:07 -06:00
Greg Fitzgerald 4a534d6abb Don't clone() Arc before recycling
This might fix an awful bug where the streamer reuses a Blob
before the current user is done with it. Recycler should probably
assert ref count is one?

* Also don't collect() an iterator before iterating over it.
2018-06-25 17:33:07 -06:00
Stephen Akridge b48a8c0555 Chunk blobs into window size to avoid window overrun
Fixes #447
2018-06-25 17:33:07 -06:00
Rob Walker 1919ec247b add a clock to validator windows (part 3 of #309) (#448)
* count entries processed by Bank
 * initialize windows with initial height of Entries
2018-06-25 15:07:48 -07:00
anatoly yakovenko c24b0a1a3f
TVU rework (#352)
Refactored TVU, into stages
* blob fetch stage for blobs
* window stage for maintaining the blob window
* pulled out NCP out of the TVU so they can be separate units
TVU is now just the fetch -> window -> request and bank processing
2018-06-13 21:52:23 -07:00
Stephen Akridge a710d95243 Fix non-erasure blob nulling 2018-06-05 15:32:25 -06:00
Stephen Akridge a06535d7ed cargo fmt 2018-06-05 15:32:25 -06:00
Stephen Akridge f511ac9be7 Fixes for receiving old blobs and nulling the window with coding 2018-06-05 15:32:25 -06:00
Stephen Akridge e28ad2177e Receive fixes 2018-06-05 15:32:25 -06:00
Stephen Akridge ec3569aa39 Move receive_index to correct place 2018-06-05 15:32:25 -06:00
Stephen Akridge 246edecf53 Add receive_index for broadcast blobs and fix blobs_len position 2018-06-05 15:32:25 -06:00
Stephen Akridge b845245614 Restore more of the blob window and add is_coding helper 2018-06-05 15:32:25 -06:00
Stephen Akridge 5711fb9969 Generate coding for the current blob set not just the first coding set 2018-06-05 15:32:25 -06:00
Stephen Akridge d1eaecde9a Fix deadlock and only push to contq if it's not a coding blob 2018-06-05 15:32:25 -06:00
Stephen Akridge 33f01efe69 Fixes for erasure coding 2018-06-05 15:32:25 -06:00
Stephen Akridge 377d312c81 Revert log levels 2018-06-05 15:32:25 -06:00
Stephen Akridge badf5d5412 Add window recovery 2018-06-05 15:32:25 -06:00
Stephen Akridge 5455e8e6a9 Review comments 2018-06-05 15:32:25 -06:00
Stephen Akridge d053f78b74 Erasure refinements, fix generating orders table 2018-06-05 15:32:25 -06:00
Stephen Akridge 93b6fceb2f generate coding after indexing 2018-06-05 15:32:25 -06:00
Stephen Akridge ac7860c35d indexing blobs then coding 2018-06-05 15:32:25 -06:00
Stephen Akridge ea97529185 Fix erasure compilation 2018-06-05 15:32:25 -06:00
Greg Fitzgerald 7e788d3a17 No longer need explicit refs in rustc 1.26 2018-06-04 21:43:46 -06:00
anatoly yakovenko 216510c573
repair socket and receiver thread (#303)
repair socket and receiver thread
2018-06-02 08:32:51 -07:00
Anatoly Yakovenko 9f2086c772 names 2018-05-30 14:50:53 -06:00
Anatoly Yakovenko 3eb005d492 names for threds 2018-05-30 14:50:53 -06:00
Anatoly Yakovenko e44e81bd17 fmt 2018-05-30 14:04:48 -06:00
Anatoly Yakovenko f5eedd2d19 fmt 2018-05-30 14:04:48 -06:00
Anatoly Yakovenko cef1c208a5 Crdt pipeline, coalesce window repair requests in the listener by examining all of them at once, and ublock those threads from doing io. 2018-05-30 14:04:48 -06:00
Greg Fitzgerald 9c9c63572b cargo fmt
rustfmt was updated with 1.26.1
2018-05-29 20:33:45 -07:00
Greg Fitzgerald 134c7add57 Fix bench build 2018-05-29 18:09:03 -06:00
Anatoly Yakovenko fe93bba457 logs
poll both endpoints in client

logs

logs

logs

names

verify plan not sig

log

set udp buffer to max

drop output

more verbose about window requests

log the leader

load leader identity

readme for single node demo

update

asserts

update

replay all

rsync

dynamic file read in testnode

fix

cleanup

readme

sum

fix scripts

cleanup

cleanup

readme
2018-05-26 20:13:42 -06:00
Greg Fitzgerald bb53f69016 Fix typos 2018-05-26 00:36:50 -06:00
Greg Fitzgerald 8d26be8b89 Run benchmarks in nightly
And name functions the same way as test functions
2018-05-26 00:36:50 -06:00
Greg Fitzgerald 9f5a3d6064 events -> transactions 2018-05-25 16:47:21 -06:00
Anatoly Yakovenko c2c80232e3 logs 2018-05-24 17:40:33 -06:00
Anatoly Yakovenko 669b1694b8 exponentail backoff for retransmit 2018-05-24 17:40:33 -06:00
Anatoly Yakovenko 2128c58fbe logs and tps counting 2018-05-24 10:35:23 -06:00
Anatoly Yakovenko f3c4acc723 cleanup multi node test 2018-05-23 16:59:17 -06:00
Anatoly Yakovenko 239b925fb3 woop 2018-05-23 12:07:44 -06:00
Anatoly Yakovenko 437c485e5c cleanup 2018-05-23 12:07:44 -06:00
Stephen Akridge 8454eb79d0 Send events to the right address and set recv socket timeout 2018-05-22 13:52:50 -07:00
Greg Fitzgerald 0b91dd6163 Fix the benchmark build 2018-05-16 16:35:50 -06:00
Greg Fitzgerald 7e44005a0f Don't do error-prone things in functions that spawn threads 2018-05-15 09:53:51 -06:00
Greg Fitzgerald ee3fb985ea Hoist set_timeout 2018-05-15 09:42:28 -06:00
Anatoly Yakovenko 2d635386af rebased 2018-05-14 15:20:41 -07:00
Greg Fitzgerald d2dd005a59 accountant -> bank 2018-05-14 15:33:11 -06:00
Greg Fitzgerald bcdb058492 cargo fmt 2018-05-11 13:06:05 -06:00
Jackson Sandland 250830ade9 cargo fmt run 2018-05-11 11:38:52 -07:00
Code Cobain 458c27c6e9
Merge branch 'master' into 153-panic-cleanup 2018-05-11 11:18:45 -07:00
Jackson Sandland 4eb2e84c9f streamer.rs - panic cleanup 2018-05-10 17:38:00 -07:00
Greg Fitzgerald f384a2ce85 Move streamer-specific utility into streamer module 2018-05-10 16:58:37 -06:00
Stephen Akridge a80991f2b3 Fixes for serializing entries over blobs and reorg into ledger 2018-05-10 15:30:30 -07:00
Greg Fitzgerald e4c47e8417 Use AccountingStage in Tpu 2018-05-09 10:31:23 -06:00
Greg Fitzgerald 68c7f992fa Sooth all versions of rustfmt 2018-05-03 13:56:10 -06:00
Anatoly Yakovenko c2e2960bf7 Add broadcast impl 2018-05-03 10:34:01 -07:00
Greg Fitzgerald 48d94143e7 Fix CI 2018-05-02 11:05:11 -06:00
Robert Kelly 63cf6363a2 more rustfmt 2018-05-02 12:24:25 -04:00
Stephen Akridge 7f6a4b0ce3 Deserialize the Entry structs and process them 2018-04-27 13:15:19 -07:00
Anatoly Yakovenko ebb089b3f1 wip 2018-04-27 08:21:34 -07:00
kwangin 13a2f05776 Remove out for immutable variable 2018-04-19 23:00:16 +09:00
Anatoly Yakovenko 97e772e87a docs 2018-04-17 19:46:50 -07:00
Anatoly Yakovenko 249cead13e docs 2018-04-17 11:07:43 -07:00
Anatoly Yakovenko 7c96dea359 fmt 2018-04-17 11:05:35 -07:00
Anatoly Yakovenko 374c9921fd comments 2018-04-17 11:05:15 -07:00
Anatoly Yakovenko fb55ab8c33 format 2018-04-16 21:02:37 -07:00
Anatoly Yakovenko 13485074ac test cast 2018-04-16 20:57:15 -07:00
Anatoly Yakovenko 4944c965e4 update
heap

update

update

wip

use a vec and sort

builds

update

tests

update

fmt

update

progress

fmt

passes needs retransmit test

tests

cleanup

update

update

update

update

fmt
2018-04-16 20:33:09 -07:00
Greg Fitzgerald 3b9ef5ccab Fix the nightly build 2018-04-11 20:24:14 -06:00
Stephen Akridge f4466c8c0a Change for cuda verify integration 2018-04-05 20:00:44 -07:00
Anatoly Yakovenko 5ac7df17f9 Implement window service
Batch out of order blobs until we have a contigious window.
2018-04-03 13:53:19 -07:00
Jackson Sandland ef169a6652 94: source doc review 2018-03-30 10:43:38 -07:00
Greg Fitzgerald 55179101cd Add more documentation 2018-03-29 12:20:54 -06:00
Greg Fitzgerald ddb21d151d Nightly rustfmt
Format code with the nightly version of rustfmt, which sorts imports.
2018-03-26 22:03:28 -06:00
Anatoly Yakovenko f089abb3c5 fix bench 2018-03-25 15:37:00 -07:00
Anatoly Yakovenko 8e551f5e32 debug trait tests 2018-03-25 08:22:04 -07:00
Anatoly Yakovenko 290960c3b5 wip 2018-03-25 08:06:33 -07:00
Anatoly Yakovenko 62af09adbe wip 2018-03-25 08:05:03 -07:00
Anatoly Yakovenko e39c0b34e5 update 2018-03-25 00:06:48 -07:00
Anatoly Yakovenko 533b3170a7 responder 2018-03-24 23:31:54 -07:00
Anatoly Yakovenko f52f02a434 services 2018-03-24 18:01:40 -07:00
Greg Fitzgerald 117ab0c141 Clippy review 2018-03-22 14:50:24 -06:00
Greg Fitzgerald fad7ff8bf0 Clippy review 2018-03-22 14:31:58 -06:00
Greg Fitzgerald 803dcb0800 Mutex<bool> -> AtomicBool 2018-03-22 14:05:23 -06:00
Greg Fitzgerald 8ea97141ea Update the test to replicate the ledger 2018-03-21 17:15:32 -06:00
Greg Fitzgerald c11a3e0fdc Move streamer benchmark out of unit tests 2018-03-19 17:10:01 -06:00
Anatoly Yakovenko e8faf6d59a trait test 2018-03-14 11:28:05 -07:00
Anatoly Yakovenko baa4ea3cd8 wfmt 2018-03-14 11:14:40 -07:00