Commit Graph

723 Commits

Author SHA1 Message Date
Greg Fitzgerald 4eb1bc08a7
Merge pull request #205 from rlkelly/203__test_key_generation
203  test key generation
2018-05-12 17:26:46 -06:00
Robert Kelly ff5e1c635f increased iterations 2018-05-12 18:18:18 -04:00
Robert Kelly 6149c2fcb5 added benchmarks for two GenKeys 2018-05-12 18:08:08 -04:00
Greg Fitzgerald 6264508f5e Consistent naming of senders and receivers 2018-05-12 15:24:20 -06:00
Greg Fitzgerald a3869dd4c1 Move entry_receiver to RequestStage
This can move to AccountingStage once RequestStage stops
calling process_events().
2018-05-12 15:14:37 -06:00
Greg Fitzgerald a3d2831f8c Free up the name 'accounting_stage' 2018-05-12 14:05:57 -06:00
Robert Kelly 4cd1fa8c38 refactored seed generation 2018-05-12 15:42:27 -04:00
Greg Fitzgerald 1511dc43d7 Move RequestProcessor out of Rpu/Tvu state 2018-05-12 11:39:24 -06:00
Greg Fitzgerald 3d82807965 Delete dead code 2018-05-12 11:24:40 -06:00
Greg Fitzgerald 4180571660 Don't pass events_socket to RPU 2018-05-12 11:11:30 -06:00
Greg Fitzgerald 421d9aa501 Free up the name 'tpu' 2018-05-12 10:53:25 -06:00
Greg Fitzgerald 898f4971a2 Free up name 'thin_client_service' 2018-05-12 10:50:22 -06:00
Greg Fitzgerald 7ab3331f01 Move validation processor to its own module 2018-05-12 00:31:32 -06:00
Greg Fitzgerald b4ca414492 More object-oriented 2018-05-12 00:19:12 -06:00
Greg Fitzgerald 73abea088a No need for TPU dependency 2018-05-11 23:51:35 -06:00
Greg Fitzgerald 2376dfc139 Let thin client own the receiver channel 2018-05-11 23:46:04 -06:00
Greg Fitzgerald d2f95d5319 Move thin client service thread into thin_client_service.rs 2018-05-11 23:37:44 -06:00
Greg Fitzgerald cd96843699 Free up name ThinClientService 2018-05-11 23:37:14 -06:00
Greg Fitzgerald ca80bc33c6 Move the writer stage's utilities to its own module 2018-05-11 22:36:16 -06:00
Greg Fitzgerald 19607886f7 Move sig verification stage into its own module 2018-05-11 21:51:37 -06:00
Greg Fitzgerald 3c11a91f77 Cleanup verifier error handling 2018-05-11 21:01:07 -06:00
Greg Fitzgerald b781fdbd04 Reorganize 2018-05-11 20:50:50 -06:00
Greg Fitzgerald 765d901530 Better names 2018-05-11 20:18:04 -06:00
Greg Fitzgerald 3cedbc493e Reorder to reflect the pipeline order 2018-05-11 20:11:25 -06:00
Greg Fitzgerald 0488d0a82f Extract sig verify functions 2018-05-11 19:59:40 -06:00
Greg Fitzgerald f0be595e4c Create function for thin client thread 2018-05-11 17:58:27 -06:00
Greg Fitzgerald 55100854d6 Better names 2018-05-11 16:41:35 -06:00
Greg Fitzgerald 600a1f8866 Initialize thin client with events port 2018-05-11 16:35:53 -06:00
Greg Fitzgerald 95bf68f3f5 Correct some strange naming 2018-05-11 16:24:18 -06:00
Greg Fitzgerald bcdb058492 cargo fmt 2018-05-11 13:06:05 -06:00
Code Cobain e779496dfb
Update signature.rs 2018-05-11 11:49:22 -07:00
Code Cobain 3d77fa5fbc
Merge branch 'master' into 153-panic-cleanup 2018-05-11 11:40:20 -07:00
Jackson Sandland 250830ade9 cargo fmt run 2018-05-11 11:38:52 -07:00
Code Cobain 458c27c6e9
Merge branch 'master' into 153-panic-cleanup 2018-05-11 11:18:45 -07:00
Robert Kelly a49e664e63 Merge branch '156__remove_user_keys_in_mintdemo' of github.com:rlkelly/solana into 156__remove_user_keys_in_mintdemo 2018-05-11 14:07:48 -04:00
Robert Kelly f20380d6b4 changed RwLock to RefCell 2018-05-11 14:07:41 -04:00
Rob Kelly 05a5e551d6
Merge branch 'master' into 156__remove_user_keys_in_mintdemo 2018-05-11 13:00:44 -04:00
Robert Kelly d278b71cb2 added tests and utility method for key generation 2018-05-11 12:55:05 -04:00
Greg Fitzgerald a485c141d5
Merge pull request #199 from garious/add-accounting-stage
Fix race condition in Accountant::apply_payment()
2018-05-11 10:54:32 -06:00
Tyera Eulberg 7144090528 Fix whitespace 2018-05-11 10:40:31 -06:00
Tyera Eulberg ee0015ac38 Fix whitespace 2018-05-11 10:34:46 -06:00
Tyera Eulberg 8b7f7f1088 Generalize next tick functions to carry events 2018-05-11 09:45:42 -06:00
Jackson Sandland c95c6a75f8 tpu.rs - panic cleanup 2018-05-10 20:49:58 -07:00
Jackson Sandland 44bf79e35f transaction.rs - panic cleanup 2018-05-10 18:24:33 -07:00
Jackson Sandland bb654f286c tpu.rs - panic cleanup 2018-05-10 18:21:10 -07:00
Greg Fitzgerald 1acd2aa8cf Fix race condition in Accountant::apply_payment() 2018-05-10 19:07:15 -06:00
Jackson Sandland 18d3659b91 timing.rs - panic cleanup 2018-05-10 17:47:27 -07:00
Jackson Sandland 63a4bafa72 thin_client - panic cleanup 2018-05-10 17:46:10 -07:00
Jackson Sandland 4eb2e84c9f streamer.rs - panic cleanup 2018-05-10 17:38:00 -07:00
Jackson Sandland 73c7fb87e8 signature.rs - panic cleanup 2018-05-10 17:15:53 -07:00
Jackson Sandland c1496722aa packet.rs - panic cleanup 2018-05-10 17:11:31 -07:00
Jackson Sandland d9f81b0c8c mint.rs - panic cleanup 2018-05-10 17:06:43 -07:00
Jackson Sandland d69beaabe1 historian.rs - panic cleanup 2018-05-10 17:00:37 -07:00
Jackson Sandland b7a0bd6347 event.rs - panic cleanup 2018-05-10 16:59:13 -07:00
Jackson Sandland 882ea6b672 erasure.rs - panic cleanup 2018-05-10 16:54:21 -07:00
Greg Fitzgerald af53197c04 cargo +nightly fmt 2018-05-10 16:58:37 -06:00
Greg Fitzgerald cf186c5762 Better names 2018-05-10 16:58:37 -06:00
Greg Fitzgerald f384a2ce85 Move streamer-specific utility into streamer module 2018-05-10 16:58:37 -06:00
Greg Fitzgerald 803b76e997 More idiomatic Rust 2018-05-10 16:58:37 -06:00
Greg Fitzgerald 230d7c3dd6 Move all Request processing into thin_client_service 2018-05-10 16:58:37 -06:00
Greg Fitzgerald 4f629dd982 Add events socket instead of modifying the existing socket 2018-05-10 16:54:43 -06:00
Greg Fitzgerald 4fdd891b54 More precise function names 2018-05-10 16:54:43 -06:00
Stephen Akridge a80991f2b3 Fixes for serializing entries over blobs and reorg into ledger 2018-05-10 15:30:30 -07:00
Greg Fitzgerald 521ae21632
Merge pull request #193 from sakridge/serialize_entries_over_multiple_blobs
Serialize entries over multiple blobs
2018-05-10 13:53:48 -06:00
Jackson Sandland bcd6606a16 ecdsa.rs - panic cleanup 2018-05-09 18:19:23 -07:00
Jackson Sandland 52ebb88205 accountant.rs - simplify error messages 2018-05-09 18:16:37 -07:00
Jackson Sandland 1e91d09be7 crdt.rs - panic cleanup 2018-05-09 18:10:48 -07:00
Jackson Sandland 02c573986b historian / transaction updates 2018-05-09 17:22:14 -07:00
Jackson Sandland f2de486658 accountant.rs - panic cleanup 2018-05-09 17:19:12 -07:00
Stephen Akridge 900b4f2644 Serialize entries over multiple blobs 2018-05-09 16:03:47 -07:00
Greg Fitzgerald 801468d70d Fix nightly 2018-05-09 16:51:34 -06:00
Greg Fitzgerald 7ce11b5d1c Cleanup: use full words for field names
and optionally for variable names
2018-05-09 16:19:42 -06:00
Greg Fitzgerald f2d4799491 Cleanup: field names should be nouns 2018-05-09 16:14:40 -06:00
Greg Fitzgerald ebc458cd32 Remove redundant Arcs 2018-05-09 15:45:10 -06:00
Greg Fitzgerald 43cd631579 Add thin_client_service 2018-05-09 14:56:34 -06:00
Greg Fitzgerald bc824c1a6c Reference count the accountant
So that the thin client can reference the AccountingStage's accountant
from separate threads.
2018-05-09 14:33:20 -06:00
Greg Fitzgerald 4223aff840 Remove useless ref counts 2018-05-09 14:25:52 -06:00
Greg Fitzgerald f107c6c2ca Don't wrap thread-safe objects with mutexes 2018-05-09 14:21:42 -06:00
Greg Fitzgerald 7daf14caa7 Don't depend on client from server 2018-05-09 13:33:33 -06:00
Greg Fitzgerald ded28c705f Tuck away the Historian
The Historian is now just a utility of the accounting stage.
2018-05-09 12:25:19 -06:00
Greg Fitzgerald 778bec0777 Intercept historian output from accounting stage
We were accessing the accountant from multiple stages just to
register the ID the historian adds to Events.

This change should cause a whole lot of Arcs and Mutexes to go away.
2018-05-09 12:00:37 -06:00
Greg Fitzgerald 6967cf7f86 Boot sync_channel()
This is less useful now that we send Vec<Event> instead of Event.
2018-05-09 11:43:16 -06:00
Greg Fitzgerald 0ee3ec86bd Fix nightly 2018-05-09 10:48:56 -06:00
Greg Fitzgerald e4c47e8417 Use AccountingStage in Tpu 2018-05-09 10:31:23 -06:00
Greg Fitzgerald 98ae80f4ed Hoist historian 2018-05-09 09:26:58 -06:00
Greg Fitzgerald 876c77d0bc Extract accounting stage code from tpu 2018-05-09 09:22:46 -06:00
Greg Fitzgerald d44a6f7541 Move Accounting stage functionality into its own object 2018-05-09 09:03:00 -06:00
Greg Fitzgerald 9040c04d27 Remove redundant Tick 2018-05-09 08:18:52 -06:00
Greg Fitzgerald ebbdef0538 Ignore flakey test 2018-05-09 08:16:59 -06:00
Greg Fitzgerald bfbee988d0 No longer wait for a Tick signal to record events 2018-05-09 08:15:51 -06:00
Greg Fitzgerald 1d4d0272ca Drop support for logging a single event 2018-05-09 08:12:33 -06:00
Greg Fitzgerald 77a76f0783 Record a batch of events 2018-05-09 08:11:19 -06:00
Greg Fitzgerald d9079de262 Add a way of sending a batch of events 2018-05-09 08:05:40 -06:00
Greg Fitzgerald b3d732a1a1 No longer artificially limit the size of entries
Instead, serialize the entries and split them up over multiple
blobs.
2018-05-09 07:59:55 -06:00
Greg Fitzgerald 52f1a02938 Delete historical artifact
This was just to explain Proof of History. We have better explanations
elsewhere. Delete!
2018-05-09 07:53:24 -06:00
Jackson Sandland fe51669e85 signature.rs - panic cleanup 2018-05-08 23:21:45 -07:00
Jackson Sandland 670a6c50c9 event.rs - panic cleanup 2018-05-08 22:58:48 -07:00
Jackson Sandland 86c1aaf7d8 transaction.rs - panic cleanup 2018-05-08 22:46:22 -07:00
Jackson Sandland 658e787b60 timing.rs panic cleanup 2018-05-08 22:40:07 -07:00
Robert Kelly 40c50aef50 deterministic random wallet generationg 2018-05-09 00:07:19 -04:00
Greg Fitzgerald 1dca17fdb4 cargo +nightly fmt 2018-05-08 18:59:01 -06:00
Greg Fitzgerald 785e971698 AccountantSkel -> Tpu
The terms Stub and Skel come from OMG IDL and only made sense while
the Stub was acting as an RPC client for the the Accountant object.
Nowadays, the Stub interface looks nothing like the Accountant and
meanwhile we've recognized the multithreaded implementation is more
reminiscent of a pipelined CPU. Thus, we finally bite the bullet and
rename our modules.

AccountantSkel -> Tpu
AccountantStub -> ThinClient

Up next will be moving much of the TPU code into separate modules,
each representing a stage of the pipeline. The interface of each
will follow the precedent set by the Historian object.
2018-05-08 17:40:02 -06:00
Greg Fitzgerald 2bfa20ff85
Merge pull request #182 from garious/split-request
Control port prep
2018-05-08 17:11:34 -06:00
Greg Fitzgerald 474a9af78d
Merge pull request #187 from sakridge/fix_blob_size_check
Trust the recorder not to give us more than we can serialize
2018-05-08 17:11:18 -06:00
Stephen Akridge 4870def1fb Fix default client port, server uses 8000-8002 for gossip. 2018-05-08 15:40:55 -07:00
Stephen Akridge 3e73fb9233 Trust the recorder not to give us more than we can serialize
Also run client for 10 seconds, 5 is bit too short
2018-05-08 15:23:41 -07:00
Stephen Akridge bd0671e123 Rework sig processing threads and add perf for process/verify 2018-05-08 11:49:29 -07:00
Greg Fitzgerald 9ff1a6f0cd Add a thread to support thin clients 2018-05-07 21:44:44 -06:00
Greg Fitzgerald 62bb78f58d Prepwork to hoist processing requests 2018-05-07 15:09:08 -06:00
Greg Fitzgerald 893011c3ba Process events instead of processing only transactions
Prep work to allow clients to send any type that can end up in
the ledger.
2018-05-07 14:51:13 -06:00
Anatoly Yakovenko 85f83f2c74 fmt 2018-05-06 22:29:33 -07:00
Anatoly Yakovenko 4751e459cc fixed! 2018-05-06 22:25:05 -07:00
Anatoly Yakovenko 138efa6cec fixed constant 2018-05-06 22:06:19 -07:00
Anatoly Yakovenko a68e50935e useless timeouts i think 2018-05-06 21:48:46 -07:00
Stephen Akridge e8f5fb35ac Multinode fixes and test
* Replace magic numbers for 64k event size
* Fix gossip, dont ping yourself
* Retransmit only to listening nodes
* Multinode test in stub marked unstable
2018-05-06 21:36:06 -07:00
Greg Fitzgerald e162f24119 Limit 256 events per entry
Attempt to keep blob size under 64kb
2018-05-04 11:52:05 -06:00
Stephen Akridge 2d5313639a Factor out entry processing and fix replicate test to call global setup fn 2018-05-03 22:24:30 -07:00
Stephen Akridge 888c2ffb20 Fix bind so we can talk on external interfaces and surface send error 2018-05-03 17:05:02 -07:00
Stephen Akridge 2cdd515b12 Compiles/fmt and add assert for forward progress 2018-05-03 14:58:08 -07:00
Anatoly Yakovenko 0aad71d46e fix entry serialize 2018-05-03 14:35:04 -07:00
Greg Fitzgerald 68c7f992fa Sooth all versions of rustfmt 2018-05-03 13:56:10 -06:00
Greg Fitzgerald 1feff408ff Implement get_last_id() with transaction_count()
This is more precice than the previous implementation because it'll
drain the EntryInfo queue and return the most recent last_id instead
of the first one.
2018-05-03 13:34:57 -06:00
Greg Fitzgerald f752e02487 Implement GetLastId with EntryInfo subscription 2018-05-03 13:31:43 -06:00
Greg Fitzgerald c9c7fb0a27 Update comment
The last PR added a thread that logs entries without needing to
be driven by the client.
2018-05-03 13:27:37 -06:00
Greg Fitzgerald de680c2a8e Remove duplicate state 2018-05-03 13:24:37 -06:00
Anatoly Yakovenko c2e2960bf7 Add broadcast impl 2018-05-03 10:34:01 -07:00
Greg Fitzgerald 385d2a580c Merge pull request #168 from aeyakovenko/fix_multi_host_client_demo
multi host client demo
2018-05-03 10:21:41 -06:00
Greg Fitzgerald ae29c9b4a0 Fix nightly build 2018-05-03 09:38:59 -06:00
Anatoly Yakovenko 078f917e61 useless assert 2018-05-03 08:34:57 -07:00
Anatoly Yakovenko b65f04d500 multi host client demo
Bind to the same interface as the user supplied client address.
2018-05-03 08:28:11 -07:00
Greg Fitzgerald 4b9f115586 Hoist Historian input 2018-05-02 16:35:37 -06:00
Greg Fitzgerald c5cc91443e Rename sender/receiver to input/output 2018-05-02 15:54:53 -06:00
Greg Fitzgerald 48d94143e7 Fix CI 2018-05-02 11:05:11 -06:00
Robert Kelly 63cf6363a2 more rustfmt 2018-05-02 12:24:25 -04:00
Robert Kelly cc6de605ac rustfmt 2018-05-02 12:21:20 -04:00
Robert Kelly d0151d2b79 restored original test logic 2018-05-02 12:07:42 -04:00
Robert Kelly 6b45d453b8 modified verification map 2018-05-02 10:44:41 -04:00
Robert Kelly b992a84d67 modified verification to loop until success or failure 2018-05-02 10:15:08 -04:00
Robert Kelly cb362e9052 rust format 2018-05-01 16:38:15 -04:00
Robert Kelly ccb478c1f6 improved error handling and atomic transactions 2018-05-01 16:38:15 -04:00
Greg Fitzgerald edf6272374
Merge pull request #154 from sakridge/replicator
Replicator
2018-04-27 14:30:52 -06:00
Stephen Akridge 7f6a4b0ce3 Deserialize the Entry structs and process them 2018-04-27 13:15:19 -07:00
Stephen Akridge 3be5f25f2f Work on test_replicate to test replicate service
generate some messages to send to replicator service
2018-04-27 08:21:34 -07:00
Stephen Akridge 1b6cdd5637 Fix some compilation issues 2018-04-27 08:21:34 -07:00
Anatoly Yakovenko f752e55929 update 2018-04-27 08:21:34 -07:00
Anatoly Yakovenko ebb089b3f1 wip 2018-04-27 08:21:34 -07:00
Anatoly Yakovenko ad6303f031 docs 2018-04-27 08:21:34 -07:00
Anatoly Yakovenko 828b9d6717 docs 2018-04-27 08:21:34 -07:00
Anatoly Yakovenko 444adcd1ca update 2018-04-27 08:21:34 -07:00
Anatoly Yakovenko 69ac305883 wip 2018-04-27 08:21:34 -07:00
Anatoly Yakovenko 2ff57df2a0 state replication 2018-04-27 08:21:34 -07:00
Greg Fitzgerald 7077f4cbe2
Merge pull request #128 from garious/faster-demo
Utilize parallelized accountant in demo
2018-04-27 08:47:42 -06:00
Anatoly Yakovenko d90ab90145 bind to all 2018-04-26 13:54:29 -07:00
Anatoly Yakovenko 48018b3f5b docs 2018-04-26 13:50:57 -07:00
Anatoly Yakovenko 15584e7062 recover full network from a star 2018-04-26 13:48:42 -07:00
Greg Fitzgerald d415b17146 sleepless demo to complement sleepless nights
18 ktps on macbook pro, no gpu
2018-04-26 13:17:38 -06:00
Greg Fitzgerald 9ed953e8c3 Fix rebase fails 2018-04-26 09:35:10 -06:00
Greg Fitzgerald b60a98bd6e Startup log can reference IDs without itself 2018-04-26 08:42:34 -06:00
Greg Fitzgerald a15e30d4b3 Report transactions processed 2018-04-26 08:42:34 -06:00
Greg Fitzgerald d5d133353f Port blocking stub functions to new stateful ones 2018-04-26 08:42:34 -06:00
Greg Fitzgerald 6badc98510 Add low-level response-handling functions to skel 2018-04-26 08:42:34 -06:00
Greg Fitzgerald ea8bfb46ce Add a way to subscribe for new entry metadata 2018-04-26 08:42:34 -06:00
Greg Fitzgerald 58860ed19f WIP: New demo that makes better use of the parallelized accountant 2018-04-26 08:42:34 -06:00
Greg Fitzgerald 583f652197 Generate genesis log for the demo
This log contains a bunch of transactions that generate new
accounts, so that transactions to and from them can be processed
in parallel.
2018-04-26 08:42:34 -06:00
Greg Fitzgerald 38fdd17067 Add initializing log message to server
Handy when gesesis block is large.
2018-04-26 08:42:34 -06:00
Greg Fitzgerald 807ccd15ba Add solana-mint-demo CLI
This extends solana-mint with additional data that will be used by
both solana-client-demo and creating the demo's genesis block.
2018-04-26 08:42:34 -06:00
Greg Fitzgerald 1c923d2f9e Fix entry hash when no events and num_hashes is one 2018-04-26 08:42:34 -06:00
Greg Fitzgerald 2676b21400
Merge pull request #151 from rlkelly/139__forget_signature
added forget_signature method
2018-04-26 08:28:11 -06:00
Robert Kelly fd5ef94b5a added forget signature method 2018-04-26 07:22:11 -04:00
Robert Kelly 34d1805b54 Add FutureResult to return a Future that immediately resolves 2018-04-25 19:23:24 -07:00
Anatoly Yakovenko 55b8d0db4d cleanup 2018-04-23 23:33:21 -07:00
Anatoly Yakovenko 3d7969d8a2 initial crdt implementation 2018-04-23 23:06:28 -07:00
rleungx 3da1fa4d88 improve the error messages 2018-04-21 21:52:55 +08:00
Greg Fitzgerald 10a0c47210
Merge pull request #137 from garious/linux-hang
Workaround linux hang
2018-04-19 11:46:48 -06:00
Stephen Akridge 8181bc591b Add -h/--help options for client-demo and testnode 2018-04-19 10:22:31 -07:00
Greg Fitzgerald c6048e2bab Workaround linux hang
Without this patch, Linux systems would hang when running the demo.

The root cause (why Linux is acting differently than macOS) was
not determined, but we know the problem is caused by a known
issue in the transaction pipeline - that entries are not pulled
off the historian channel until after the full transaction batch
is processed. This patch makes the sync_channel large enough that
it should never block on a gigabit network.
2018-04-19 10:04:32 -07:00
rleungx 60015aee04 report serde parse errors to stderr 2018-04-19 23:51:57 +08:00
Greg Fitzgerald 43e6741071
Merge pull request #134 from rleungx/report-parse-errors-to-stderr
report parse errors to stderr
2018-04-19 08:38:38 -06:00
rleungx b91f6bcbff report parse errors to stderr 2018-04-19 22:24:46 +08:00
kwangin 13a2f05776 Remove out for immutable variable 2018-04-19 23:00:16 +09:00
Stephen Akridge dea5ab2f79 Add erasure rust logic under feature flag 2018-04-18 19:42:09 -07:00
Anatoly Yakovenko 29f3230089 docs 2018-04-17 19:53:18 -07:00
Anatoly Yakovenko d003efb522 fix docs 2018-04-17 19:52:46 -07:00
Anatoly Yakovenko 97e772e87a docs 2018-04-17 19:46:50 -07:00
Anatoly Yakovenko 0b33615979 udpate 2018-04-17 12:48:06 -07:00
Anatoly Yakovenko 249cead13e docs 2018-04-17 11:07:43 -07:00
Anatoly Yakovenko 7c96dea359 fmt 2018-04-17 11:05:35 -07:00
Anatoly Yakovenko 374c9921fd comments 2018-04-17 11:05:15 -07:00
Anatoly Yakovenko fb55ab8c33 format 2018-04-16 21:02:37 -07:00
Anatoly Yakovenko 13485074ac test cast 2018-04-16 20:57:15 -07:00
Anatoly Yakovenko 4944c965e4 update
heap

update

update

wip

use a vec and sort

builds

update

tests

update

fmt

update

progress

fmt

passes needs retransmit test

tests

cleanup

update

update

update

update

fmt
2018-04-16 20:33:09 -07:00
Greg Fitzgerald 7fc42de758 Fix bench 2018-04-13 00:36:23 -04:00
Greg Fitzgerald 0a30bd74c1 Tell verifiers when not to parallelize accounting
Without this patch, many batches of transactions could be tossed
into a single entry, but the parallelized accountant can only
guarentee the transactions in the batch can be processed in
parallel.

This patch signals the historian to generate a new Entry after
each batch. Validators must maintain sequential consistency
across Entries.
2018-04-12 21:08:53 -06:00
Greg Fitzgerald 9b12a79c8d cargo +nightly fmt 2018-04-12 17:04:11 -06:00
Stephen Akridge 8dc15b88eb Add skel test which sends a bad transaction, verify it doesn't make it 2018-04-12 15:01:59 -07:00
Anatoly Yakovenko c2eeeb27fd bump timer 2018-04-12 11:12:10 -07:00
Anatoly Yakovenko 180d8b67e4 requests to packets function 2018-04-12 10:44:09 -07:00
Greg Fitzgerald 51633f509d Fix test
The test was meant to ensure the signature covered the 'tokens'
field, but then when the 'plan' field was rolled in, Transaction::verify()
started failing because Plan::verify() failed. When Transaction::verify()
was split into two, the unexpected failure was exposed but went unnoticed.
This patch brings it back to its original intent, to ensure signature
verification fails if the network attempts to change the client's payment.
2018-04-11 22:17:21 -06:00
Greg Fitzgerald 705228ecc2 Remove redundant signs 2018-04-11 22:17:21 -06:00
Greg Fitzgerald 3b9ef5ccab Fix the nightly build 2018-04-11 20:24:14 -06:00
Greg Fitzgerald be9a670fb7 Add process_packets() benchmark 2018-04-11 18:02:45 -06:00
Greg Fitzgerald 6e43e7a146 Enable parallelized accountant 2018-04-11 18:01:59 -06:00
Greg Fitzgerald ab2093926a
Merge pull request #120 from aeyakovenko/fix_bench_compile
fix compile error
2018-04-11 18:01:13 -06:00
Anatoly Yakovenko 2ef3db9fab fix compile error 2018-04-11 15:40:25 -07:00
Stephen Akridge 6987b6fd58 Add tests for ecdsa sig checking 2018-04-11 12:29:44 -07:00
Greg Fitzgerald 50ccecdff5 Refactor 2018-04-11 09:02:33 -06:00
Greg Fitzgerald e838a8c28a Delete unused function 2018-04-10 21:56:13 -06:00
Greg Fitzgerald e5f7eeedbf Use iterators 2018-04-10 21:48:26 -06:00
Greg Fitzgerald d1948b5a00 Zip earlier
And remove redundant into_iter() calls.
2018-04-10 21:18:39 -06:00
Anatoly Yakovenko c934a30f66 commandline options for client and testnode 2018-04-09 21:14:52 -07:00
Anatoly Yakovenko f330739bc7 Recycler test should verifyt that its recycling 2018-04-07 07:08:42 -07:00
Greg Fitzgerald 584c8c07b8 Better symmetry
deserialize -> process -> serialize
2018-04-06 16:34:59 -06:00
Greg Fitzgerald a93ec03d2c Move creating blobs into its own function 2018-04-06 16:22:02 -06:00
Greg Fitzgerald 7bd3a8e004 Reduce cyclomatic complexity 2018-04-06 16:12:13 -06:00
Greg Fitzgerald 912a5f951e Why is msgs cloned here? 2018-04-06 15:58:11 -06:00
Greg Fitzgerald 6869089111 Parallelize deserialize 2018-04-06 15:52:58 -06:00
Greg Fitzgerald 6fd32fe850 Cleanup constants 2018-04-06 15:43:05 -06:00
Greg Fitzgerald 81e2b36d38 Cleanup packet_verify 2018-04-06 15:24:15 -06:00
Greg Fitzgerald 7d811afab1 Parallelize CPU sig verify 2018-04-06 15:21:49 -06:00
Greg Fitzgerald 5fc81dd6c8 Fix the nightly build
Nightly uses a different (but backward compatible) version of rustfmt.
2018-04-05 22:39:29 -06:00
Greg Fitzgerald 491a530d90 Support parallelization of arbitrary transactions
Still assumes witnesses are processed serially afterward.
2018-04-05 22:30:25 -06:00
Greg Fitzgerald c12da50f9b Fix race condition
Without this patch, it was possible for two transactions with the same
'from' address to drive its balance below zero. With the patch, we'll
hold a write lock from just before we verify sufficient funds until
after those funds are deducted from the account.
2018-04-05 22:30:25 -06:00
Greg Fitzgerald 41e8500fc5 Break up process_verified_transaction() 2018-04-05 22:29:13 -06:00
Stephen Akridge f4466c8c0a Change for cuda verify integration 2018-04-05 20:00:44 -07:00
Greg Fitzgerald 01326936e6 Expire all transactions after some amount of time
Reject old transactions so that we can calculate an upper bound
for memory usage, and therefore ensure the server won't slow
down over time to crash due to memory exhaustion.
2018-04-05 10:26:45 -06:00
Greg Fitzgerald c960e8d351 Reject transactions with a `last_id` that isn't from this ledger
Before this patch, a client could put any value into `last_id` and
was primarily there to ensure the transaction had a globally unique
signature. With this patch, the server can use `last_id` as an
indicator of how long its been since the transaction was created.
The server may choose to reject sufficiently old transactions so
that it can forget about old signatures.
2018-04-05 09:54:03 -06:00
Greg Fitzgerald 8d425e127b Update benchmark to avoid write locks in sig duplicate detection 2018-04-04 17:29:22 -06:00
Greg Fitzgerald 3cfb07ea38 Sort signatures by last_id
This will allow for additional concurrency as well as give the server
a means of garbage-collecting old signatures.
2018-04-04 17:06:31 -06:00
Greg Fitzgerald 76679ffb92 Per-cell locking
This allows us to use read-locks for balances most of the time. We
only lock the full table if we need to add one.
2018-04-04 16:31:13 -06:00
Greg Fitzgerald dc2ec925d7 Better test 2018-04-04 16:01:43 -06:00
Greg Fitzgerald 014bdaa355 Add benchmark for parallel transaction processing 2018-04-04 12:43:27 -06:00
Greg Fitzgerald 0c60fdd2ce Make accountant thread-safe
Before this change, parallel transaction processing required locking
the full accountant. Since we only call one method,
process_verified_transaction, the global lock equates to doing no
parallelization at all.  With this change, we only lock the data that's
being written to.
2018-04-04 12:33:03 -06:00
Anatoly Yakovenko 5ac7df17f9 Implement window service
Batch out of order blobs until we have a contigious window.
2018-04-03 13:53:19 -07:00
Greg Fitzgerald c323bd3c87 Fix clippy warnings 2018-04-03 09:55:33 -06:00
Greg Fitzgerald 2f80747dc7 Move tests
After we restructured for parallel verification, the tests here
were unreferenced by the accountant, but still meaningful to
transaction verification.
2018-04-02 21:45:21 -06:00
Greg Fitzgerald 94eea3abec fmt 2018-04-02 21:15:21 -06:00
Greg Fitzgerald fe32159673 Add a test to ensure witness data continues to be hashed 2018-04-02 21:07:38 -06:00
Greg Fitzgerald 07aa2e1260 Add witness data to entry hash
Otherwise, witnesses can be dropped or reordered by a malicious
generator.
2018-04-02 20:47:51 -06:00
Greg Fitzgerald 6fec8fad57 Adding from to the signature is redundant 2018-04-02 20:34:18 -06:00
Greg Fitzgerald 49708e92d3 Use last_id instead of seed
It doesn't really matter, but was confusing since the seed points
to an entry before the mint's deposit.
2018-04-02 15:06:42 -06:00
Greg Fitzgerald daadae7987 Move replaying ledger out of accountant 2018-04-02 14:51:55 -06:00
Greg Fitzgerald 2b788d06b7 Move the historian up to accountant_skel 2018-04-02 14:41:07 -06:00
Greg Fitzgerald 90cd9bd533 Move balance check so that log_* methods are only used to add logging 2018-04-02 14:14:49 -06:00
Greg Fitzgerald d63506f98c No longer allow deposits outside the constructor 2018-04-02 14:00:42 -06:00
Greg Fitzgerald 17de6876bb Add simpler accountant constructor 2018-04-02 13:51:44 -06:00
Greg Fitzgerald da2b4962a9 Move verify_slice() into a trait 2018-04-02 11:43:38 -06:00
Greg Fitzgerald 3abe305a21 Move reserve_signatures into accountant
Reasons Transaction signatures need to be unique:

1. guard against duplicates
2. accountant uses them as IDs to link Witness signatures to transactions via the
`pending` hash map
2018-04-02 09:38:36 -06:00
Greg Fitzgerald 46e8c09bd8 Revoke API access to first_id 2018-04-02 09:30:10 -06:00
Jackson Sandland 9f256f0929 94 - snakecase mod names 2018-03-30 13:10:27 -07:00
Jackson Sandland ef169a6652 94: source doc review 2018-03-30 10:43:38 -07:00
Greg Fitzgerald b8cf5f9427 Fix transaction logging 2018-03-29 13:50:32 -06:00
Greg Fitzgerald 2f1e585446 Better benchmark
Tolerates dropped UDP packets
2018-03-29 13:41:11 -06:00
Greg Fitzgerald 22f5985f1b Do request verification in parallel, and then process the verified requests 2018-03-29 13:18:08 -06:00
Greg Fitzgerald c59c38e50e Refactor for batch verification 2018-03-29 13:09:21 -06:00
Greg Fitzgerald 232e1bb8a3 Colocate packet dependencies 2018-03-29 12:55:41 -06:00
Greg Fitzgerald 1fbb34620c Fix compiler warning 2018-03-29 12:54:10 -06:00
Greg Fitzgerald 55179101cd Add more documentation 2018-03-29 12:20:54 -06:00
Greg Fitzgerald 878ca8c5c5 Add microbenchmark for signature verification 2018-03-28 22:02:47 -06:00
Greg Fitzgerald 4bc41d81ee Fix compiler warning 2018-03-28 21:05:21 -06:00
Greg Fitzgerald 0bec360a31 Revert TCP sync of ledger
The feature was too rushed. We technically don't need it until we
implement consensus. It'll come back another day (with many more tests!)
2018-03-28 20:16:15 -06:00
Greg Fitzgerald 98c0a2af87 tx confirmed/sec ---> tx processed/sec
Before this patch, we were waiting until the full log was
sent back across the wire, parsed, and interpreted. That was giving
us a metric of "transactions confirmed per second" instead of
"transactions processed per second". Instead, we'll just send one
tiny packet back with the balance. As soon as the balance is what
we expect it to be, we end the benchmark.
2018-03-28 16:51:21 -06:00
Greg Fitzgerald 849bced602 Fix up client demo 2018-03-28 14:40:58 -06:00
Greg Fitzgerald 27f29019ef
Merge pull request #83 from garious/tcp-client
TCP subscription service
2018-03-28 13:19:38 -06:00
Greg Fitzgerald 8642a41f2b See if Travis will tolerate executing some of the test 2018-03-28 10:25:16 -06:00
Greg Fitzgerald bf902ef5bc Ignore accountant_stub test
TODO: Figure out why this test fails on TravisCI
2018-03-28 10:05:00 -06:00
Greg Fitzgerald 116166f62d Rename project: silk -> solana 2018-03-27 16:25:12 -06:00
Greg Fitzgerald 7c9681007c Drop support for random access to the ledger
No longer store the ledger locally.
2018-03-27 14:47:03 -06:00
Greg Fitzgerald 13206e4976 Let clients subscribe to the ledger over TCP
TODO: Add more tests

Fixes #27
2018-03-27 14:46:24 -06:00
Greg Fitzgerald ddb21d151d Nightly rustfmt
Format code with the nightly version of rustfmt, which sorts imports.
2018-03-26 22:03:28 -06:00
Greg Fitzgerald c64a9fb456 Give Travis a little more time to start threads 2018-03-26 22:02:05 -06:00
Greg Fitzgerald ee19b4f86e See if CI hangs because of wait_on_signature() 2018-03-26 21:53:30 -06:00
Anatoly Yakovenko 14239e584f fix writer 2018-03-26 21:36:29 -06:00
Anatoly Yakovenko c1783d77d7 fixed test 2018-03-25 16:18:27 -07:00
Anatoly Yakovenko f089abb3c5 fix bench 2018-03-25 15:37:00 -07:00
Anatoly Yakovenko 8e551f5e32 debug trait tests 2018-03-25 08:22:04 -07:00
Anatoly Yakovenko 290960c3b5 wip 2018-03-25 08:06:33 -07:00
Anatoly Yakovenko 62af09adbe wip 2018-03-25 08:05:03 -07:00
Anatoly Yakovenko e39c0b34e5 update 2018-03-25 00:06:48 -07:00
Anatoly Yakovenko 8ad90807ee responder with larger block size 2018-03-24 23:46:25 -07:00
Anatoly Yakovenko 533b3170a7 responder 2018-03-24 23:31:54 -07:00
Anatoly Yakovenko 7732f3f5fb services 2018-03-24 18:01:54 -07:00
Anatoly Yakovenko f52f02a434 services 2018-03-24 18:01:40 -07:00
Greg Fitzgerald 9a437f0d38 Revive silk-testnode 2018-03-23 21:49:28 -06:00
Greg Fitzgerald fa44be2a9d Ignore some clippy advice 2018-03-22 14:59:25 -06:00
Greg Fitzgerald 117ab0c141 Clippy review 2018-03-22 14:50:24 -06:00
Greg Fitzgerald 7488d19ae6 Clippy review 2018-03-22 14:40:28 -06:00
Greg Fitzgerald 60524ad5f2 Clippy review 2018-03-22 14:38:06 -06:00
Greg Fitzgerald fad7ff8bf0 Clippy review 2018-03-22 14:31:58 -06:00
Greg Fitzgerald 383d445ba1 Clippy review 2018-03-22 14:15:29 -06:00
Greg Fitzgerald 803dcb0800 Mutex<bool> -> AtomicBool 2018-03-22 14:05:23 -06:00
Greg Fitzgerald 8ea97141ea Update the test to replicate the ledger 2018-03-21 17:15:32 -06:00
Greg Fitzgerald 9f232bac58 Allow clients to sync the ledger
Fixes #4
2018-03-21 15:46:49 -06:00
Greg Fitzgerald 8295cc11c0 Move JSON printing up the stack 2018-03-20 23:15:44 -06:00
Greg Fitzgerald 9a7cac1e07 Use the Entry API to remove the double lookup 2018-03-20 18:07:54 -06:00
Greg Fitzgerald c584a25ec9 Move complete_transaction from method to function
So that we can hold separate mutable references to the pending queue
and the map of balances.
2018-03-20 17:47:57 -06:00
Greg Fitzgerald bff32bf7bc Cleanup 2018-03-20 17:32:02 -06:00
Greg Fitzgerald d0e7450389 Add docs 2018-03-20 16:58:14 -06:00
Greg Fitzgerald 4da89ac8a9 Cleanup naming 2018-03-20 16:53:41 -06:00
Greg Fitzgerald f7032f7d9a Cleanup: replace bool retval with is_complete() method 2018-03-20 16:52:47 -06:00
Greg Fitzgerald 7c7e3931a0 Better docs 2018-03-20 15:52:46 -06:00