Commit Graph

2361 Commits

Author SHA1 Message Date
Greg Fitzgerald 9c07a8c26a VoteProgram -> VoteState 2019-02-01 16:03:46 -08:00
Sagar Dhawan 7a6d730db3
Skip retransmit when node is leader (#2625)
* Skip retransmit when node is leader

* Fix window test
2019-02-01 14:30:26 -08:00
Pankaj Garg d9a22705ce Broadcast Service should handle SendError
- After TVU shuts down, the brroadcast service will get a SendError
  when it tries to send blobs to it
2019-02-01 12:28:00 -08:00
Greg Fitzgerald dad0bfe447 Replace transaction traits with structs
Also:
* SystemTransaction::new -> new_account
* SystemTransaction::new_create -> new_program_account
2019-02-01 11:38:25 -08:00
Michael Vines 43cce3a8fc speling 2019-02-01 07:11:17 -08:00
Greg Fitzgerald 82a2080e45 Rename VoteSignerProxy to VotingKeypair
Works just like a normal Keypair, but will only sign voting
transactions.
2019-02-01 07:11:17 -07:00
Greg Fitzgerald 9a4abe96c7 Reduce VoteSignerProxy to KeypairUtil 2019-02-01 07:11:17 -07:00
Michael Vines d87c2eb903 Fullnode::new() - provide FullnodeConfig as a ref 2019-01-31 21:12:36 -08:00
Carl b6b179af97 Fix bad merge 2019-01-31 20:15:04 -08:00
carllin 37003da854
Fix potential of checking tvu bank for truth when its behind (#2614)
* Fix race between tpu and tvu, where tvu bank is not caught up to tpu bank

* Add test

* Cleanup Fullnode tests
2019-01-31 19:21:02 -08:00
Sagar Dhawan ed478675ba
Push and query the ClusterInfo for votes. (#2622) 2019-02-01 05:21:29 +05:30
Michael Vines 9767468b7f Remove unneeded Option 2019-01-31 13:53:59 -08:00
Rob Walker 8ba1d5f426
treat genesis special (#2615)
* treat genesis special

* fix poh_recorder to understand new world order
2019-01-31 13:53:08 -08:00
carllin 84567d36cf
Leader scheduler groundwork for Blocktree (#2599)
* Groundwork for entry tree, align constants with definitions in the book

* Fix edge case in test, node can keep generating ticks between handle_role_transition and exit() call
2019-01-31 13:44:24 -08:00
Pankaj Garg 32162ef0f1
Connect TPU's broadcast service with TVU's blob fetch stage (#2587)
* Connect TPU's broadcast service with TVU's blob fetch stage

- This is needed since ledger is being written only in TVU now

* fix clippy warnings

* fix failing test

* fix broken tests

* fixed failing tests
2019-01-31 13:43:22 -08:00
Pankaj Garg 2dd20c38b2 fix the test 2019-01-31 12:55:17 -08:00
Pankaj Garg aa1bd603e6 Fix recvmmsg test for timeout 2019-01-31 12:55:17 -08:00
anatoly yakovenko 2754ceec60
StatusDeque split into separate objects with their own root checkpoint strategy (#2613)
Split up StatusDeque into different modules

* LastIdQueue tracks last_ids
* StatusCache keeps track of signature statuses
* StatusCache stores success as a bit in a bloom filter
* Overhead for 1m Ok transactions is 4mb in memory
* Less concurrency between the objects, last_id and status_cache are read and written to at different points in the pipeline
* Each object has its own strategy for merging into the root checkpoint
2019-01-31 06:53:52 -08:00
Greg Fitzgerald 609e915169 Fix clippy warning
Always pass Arcs by reference. Then you'll only need to clone()
to cross thread boundaries.
2019-01-30 21:59:05 -07:00
Greg Fitzgerald 11f1c00ca7 Only send pubkey to ReplayStage 2019-01-30 21:59:05 -07:00
Greg Fitzgerald a74b24fdf0 Only store the fullnode's pubkey
Only vote_signer is used for signing
2019-01-30 21:59:05 -07:00
Greg Fitzgerald e25992a011 Always give Fullnode a vote signer
This will allow us to use the the signer's pubkey as the node id.

Disable voting with a configuration option.
2019-01-30 21:59:05 -07:00
Rob Walker 1b50fbbc90
remove Result<> from Blob accessors, add parent (#2608)
* remove Result<> from Blob accessors, add parent
* update chacha's golden
* fixup benches
2019-01-30 20:18:28 -08:00
Pankaj Garg a746969995
Don't set socket as blocking in recvmmsg for non Linux targets (#2611)
* Don't set socket as blocking in recvmmsg for non Linux targets

- The user of the function is controling this flag

* added a test
2019-01-30 19:47:53 -08:00
Michael Vines c01290438f Move virtual genesis tick into the ledger proper as entry 0 2019-01-30 14:02:07 -08:00
Michael Vines 9e9c82869a create_tmp_sample_ledger() need not return the genesis block 2019-01-30 14:02:07 -08:00
Michael Vines 494b143453 Delete create_tmp_genesis 2019-01-30 14:02:07 -08:00
Michael Vines 8cc1cde0fe create_tmp_sample_ledger() now returns entry_height and last_id 2019-01-30 14:02:07 -08:00
Greg Fitzgerald 883fc39c80 Rename EntryTree to Blocktree 2019-01-30 13:29:34 -07:00
Anatoly Yakovenko 1c0758e3bd Accounts refactoring for forking.
* Move last_id and signature dup handling out of Accounts.
* Accounts that handle overlays.
2019-01-30 11:36:49 -08:00
Greg Fitzgerald 668d353add Inline VoteSigner::new_vote_account
So that we can stop using the validator keypair to fund the
the voting account.
2019-01-30 10:42:42 -07:00
Greg Fitzgerald 06a1681fdc Remove redundant annotations 2019-01-30 10:42:42 -07:00
Michael Vines 16e705dc75 Boil away unneeded Fullnode::new_* functions 2019-01-29 20:10:10 -08:00
Michael Vines b52228feb9 Remove assumption that the mint starts with 10_000 tokens 2019-01-29 20:10:10 -08:00
Michael Vines 25f25d0f82 new_fullnode: don't return the genesis_block, nobody uses it 2019-01-29 17:51:07 -08:00
Greg Fitzgerald 85e7046caf Use signer for signing transactions, not constructing them 2019-01-29 18:35:05 -07:00
Greg Fitzgerald c741a960b9 Generalize Transaction::new to accept anything that implements KeypairUtil 2019-01-29 18:35:05 -07:00
Greg Fitzgerald 34c8b2cc2f Remove redundant Arc 2019-01-29 18:35:05 -07:00
Greg Fitzgerald 278effad49 Implement KeypairUtil for VoteSignerProxy 2019-01-29 18:35:05 -07:00
Rob Walker a0bed5375d
remove println!, add check to keep it out (#2585)
* remove debugging prints

* remove println!, add check to keep it out
2019-01-29 16:02:03 -08:00
Michael Vines a2c3369713
storage_state field doesn't actually exist 2019-01-29 12:34:59 -08:00
Rob Walker 1f9ab7f58f copy bank for TPU 2019-01-29 12:11:48 -08:00
Anatoly Yakovenko 3e1a926aa6 wip 2019-01-29 12:11:48 -08:00
Michael Vines df136578d4 Remove unnecessary FullnodeConfig::rpc_port option 2019-01-29 10:22:39 -08:00
Michael Vines ae7f169027 Add FullnodeConfig struct to Fullnode::new* functions
This avoids having to touch *every* Fullnode::new* call site when
a new fullnode option is added
2019-01-29 09:42:48 -08:00
Tyera Eulberg 6da7a784f2
Stream entries (#2582)
* Add entry streaming option

* Fix tests

* Remove obsolete comment

* Move entry stream functionality to struct w/ trait in order to test without i/o
2019-01-29 00:21:27 -08:00
Michael Vines 12cddf725e Harmonize Fullnode::new* function arguments 2019-01-28 22:37:56 -08:00
Michael Vines bfaf5634a1 .unwrap() in tests instead of assert!()ing .is_ok() for a better failure message 2019-01-28 16:10:32 -08:00
Stephen Akridge 53afa64634 Remove storage_state from the bank
Construct in TVU and pass to RPC and StorageStage instead.
2019-01-28 15:41:41 -08:00
Rob Walker c9bf9ce094
eliminate re-use of a TX here, we're testing for empty account balance (#2576) 2019-01-28 15:21:08 -08:00
Pankaj Garg 637f58364a remove io from the tests 2019-01-28 13:52:13 -08:00
Pankaj Garg 1bd04b26e5 Remove ignore flag from rpc_pubsub tests 2019-01-28 13:52:13 -08:00
Michael Vines 29ef9370a6 Remove LeaderSchedulerConfig options 2019-01-28 13:51:01 -08:00
Stephen Akridge 2262f279d5 Reduce boilerplate code with helper function to create
fullnode/bank/genesis
2019-01-28 13:48:58 -08:00
Michael Vines e4f477cf90 Retype num_ticks as u64 to reduce casting 2019-01-28 11:24:50 -08:00
Greg Fitzgerald 33f921235d Improve message-signing ergonomics 2019-01-26 14:57:22 -07:00
Greg Fitzgerald 1bae87d4b3 Add unit-test-friendly VoteSignerProxy constructor 2019-01-26 14:56:49 -07:00
Greg Fitzgerald 1e43fb587e Rename the module that now contains only GenKeys 2019-01-26 06:57:24 -08:00
Sagar Dhawan d65e7b9fcc
Speedup rotation (#2468)
Speedup leader to validator transitions
2019-01-26 13:58:08 +05:30
Michael Vines 4bb6549895 Genesis block is now a json file 2019-01-25 09:05:15 -08:00
Rob Walker e9e01557b7 fix leaked threads from unclosed fullnode 2019-01-25 03:02:49 -08:00
Sathish e0f046b7a5
Optimize Transaction/Instruction serialization with custom routine (#2515)
* Optimize transaction serialization with custom routine to reduce the serialized size.

* Update serialized_size to accept self as parameter

* Optimize serialize / deserialize operations
2019-01-24 21:14:15 -08:00
Pankaj Garg 81c82b5af9
Add test for ignore ProgramErrors in process_entries (#2544) 2019-01-24 13:37:12 -08:00
Pankaj Garg a9b083e585
Set fetch stage socket non blocking to false while during recv (#2542)
* Set fetch stage socket non blocking to false while during recv

* remove ProgramError changes from this PR
2019-01-24 12:46:40 -08:00
Tyera Eulberg b7be5b9a7a Add no-signer argument 2019-01-24 10:47:37 -08:00
Greg Fitzgerald aba9df8457 Remove get_stake placeholder 2019-01-23 17:03:20 -08:00
Michael Vines 27c10d4468
cargo fmt 2019-01-22 21:56:04 -08:00
Stephen Akridge a17467aefd Lower level of message from storage_stage 2019-01-22 21:23:10 -08:00
Rob Walker 965dbbe835
stop enumeration if next entry is disjoint, band-aid (#2518)
* stop enumeration if next entry is disjoint, band-aid, fies #2426
* clippy
2019-01-22 15:50:36 -08:00
Michael Vines 122a5b2f69
dedup the 2019-01-22 09:47:43 -08:00
Rob Walker c84b796e17
remove dead code (#2512) 2019-01-21 16:24:11 -08:00
Pankaj Garg 7204bb40bf
Don't fail process_entries with ProgramErrors (#2509) 2019-01-21 15:26:06 -08:00
Mark f37eb533f1
Replicator timeout (#2480)
* Add timeout to Replicator::new; used when polling for leader

* Add timeout functionality to replicator ledger download

Shares the same timeout as polling for leader

Defaults to 30 seconds

* Add docs for Replicator::new
2019-01-21 15:37:41 -06:00
Michael Vines 6e8b69fc88 Cleanup leader_addr, it's really entrypoint_addr 2019-01-21 13:06:30 -08:00
Tyera Eulberg 35764225ed Remove socket from rpc test and move integration test 2019-01-21 12:29:04 -07:00
Rob Walker e6030d66eb
split load+execute from commit in bank, insert record between them in TPU code (#2487)
* split load+execute from commit in bank, insert record between them in TPU code
* clippy
* remove clear_signatures() race with commit_transactions()
* add #[test] back
2019-01-21 10:17:04 -08:00
Pankaj Garg 6611188edf
Move subscriptions to rpc_pubsub (#2490)
* Move subscriptions to rpc_pubsub

- this helps avoid recreating pubsub_service on node's role change

* fixed tests and addressed review comments

* fix clippy errors

* address review comments
2019-01-21 09:59:09 -08:00
Stephen Akridge abbb037888 Implement storage contract logic 2019-01-21 08:36:49 -08:00
Michael Vines 132d59ca6a new_bank_from_db_ledger need not be public 2019-01-21 08:11:13 -08:00
Michael Vines 5b73a8eceb Rework fullnode boot sequence 2019-01-19 21:35:46 -08:00
Michael Vines 2073188345 Fullnode no longer fails to process a ledger with ProgramErrors in it 2019-01-18 21:06:50 -08:00
Stephen Akridge 2f1107ff4f Add randomness to broadcast 2019-01-18 13:05:35 -08:00
Stephen Akridge 1fd7bd7ede Storage fixes
* replicators generate their sample values
* fixes to replicator block height logic
2019-01-18 13:05:35 -08:00
Michael Vines c0c38463c7 Remove hard coded ports 2019-01-17 23:34:21 -08:00
Michael Vines c1e142d1dc Revert "test_rpc_new fails locally, ignore for now"
This reverts commit 0c46f15f94.
2019-01-17 23:34:21 -08:00
Rob Walker 1f87d9ba4a
add bloom benchmarking, perf improvement from Fnv ~= 8X (#2477)
* add bloom benchmarking, perf improvement from Fnv ~= 8X
* have a look at bits.set()
* ignore new benches to pacify CI (solana_upload_perf?)
2019-01-17 18:22:21 -08:00
Michael Vines e74574706e Record Transactions with program errors in the ledger, they paid the fee 2019-01-17 13:55:56 -08:00
Stephen Akridge b296a9a0c7 Rename slice to segment to match book terminology 2019-01-17 10:19:45 -08:00
Michael Vines a84b6bc7e4 Overhaul wallet rpc/drone command-line arguments 2019-01-17 08:36:05 -08:00
Stephen Akridge 8e67a18551 Add network timeouts and fix tests
test_rpc_send_tx could fail if it didn't succeed on the first try
* add some retries to consistently pass
2019-01-16 15:59:40 -08:00
Michael Vines 526344c9ac Log signature status uniformly 2019-01-16 12:17:46 -08:00
Michael Vines f8bd19f5db Log the time it look to process the ledger for easier log inspection 2019-01-16 10:45:47 -08:00
Pankaj Garg 03d4d1cb36
Store and resend votes if leader's TPU port is unknown (#2438)
* Store and resend votes if leader's TPU port is unknown

* fix build errors

* fix failing tests
2019-01-16 06:14:55 -08:00
Michael Vines 4369c1a113 RPC port is no longer reset on leader-to-validator transition 2019-01-15 16:06:56 -08:00
Michael Vines b1e57e2a30 Retry rpc requests on connection failures
Applied a blanket default retry count of 5, which seems like enough but
not excessive retries.
2019-01-15 15:30:10 -08:00
Rob Walker 45c247fa5b
bloom for forking (#2431)
* bloom for forking
* clippy fixes
* remove bloom_hash_index
2019-01-15 13:56:54 -08:00
Pankaj Garg fa4608a95d
Change leader rotation time to a multiple of ticks per block (#2414)
* Change leader rotation time to a multiple of ticks per block

* fix component dependencies

* review comments
2019-01-15 12:07:58 -08:00
Rob Walker 022a97da99
revert revert of kill window (#2427)
* remove window code from most places
* window used only for testing
* remove unecessary clippy directives
2019-01-15 10:51:53 -08:00
Michael Vines a33c76a456 Remove JsonRpcRequestProcessor dependency 2019-01-14 17:39:31 -08:00
Michael Vines 11b1bd278a Remove unused exit field 2019-01-14 17:39:12 -08:00
Michael Vines e3a96ed3fc Minor new cleanup 2019-01-14 16:04:29 -08:00
Rob Walker 447243f994
Revert "remove window code from most places" (#2417)
* Revert "Fix link to book in Local Testnet section (#2416)"

This reverts commit 710c0c9980.

* Revert "Add current leader information to dashboard (#2413)"

This reverts commit f0300c1711.

* Revert "remove window code from most places (#2389)"

This reverts commit e3c0bd5a3f.
2019-01-14 15:11:18 -08:00
Pankaj Garg f0300c1711
Add current leader information to dashboard (#2413) 2019-01-14 14:20:05 -08:00
Rob Walker e3c0bd5a3f
remove window code from most places (#2389)
* remove window code from most places
* window used only for testing
* remove unecessary clippy directives
2019-01-14 12:11:55 -08:00
Tyera Eulberg 8af61f561b
Improve Wallet coverage (#2385)
* Add trait for RpcRequestHandler trait for RpcClient and add MockRpcClient for unit tests

* Add request_airdrop integration test

* Add timestamp_tx, witness_tx, and cancel_tx to wallet integration tests; add wallet integration tests to test-stable

* Add test cases

* Ignore plentiful sleeps in unit tests
2019-01-14 00:10:03 -07:00
Michael Vines 74e503da92 Hold an accounts_db read lock as briefly as possible to avoid deadlocking 2019-01-13 21:49:09 -08:00
Greg Fitzgerald 9328ee4f63 Revert "Revert "Delete unused code and its tests""
This reverts commit d6b3991d49.
2019-01-11 14:54:17 -07:00
Pankaj Garg d7594b19fc
Implemented a trait for vote signer service (#2386)
* Implemented a trait for vote signer service

* removes need for RPC in unit tests for vote signing

* fix build errors

* address some review comments
2019-01-11 12:58:31 -08:00
Michael Vines d6b3991d49 Revert "Delete unused code and its tests"
This reverts commit e713ba06f1.
2019-01-11 07:30:28 -08:00
Greg Fitzgerald e713ba06f1 Delete unused code and its tests 2019-01-10 23:19:38 -07:00
Greg Fitzgerald 7341298a11
Cleanup tpu forwarder (#2377)
* Use unwrap() on locks

An error there generally indicates a programmer error, not a
runtime error, so a detailed runtime message is not generally useful.

* Only clone Arcs when passing them across thread boundaries

* Cleanup TPU forwarder

By separating the query from the update, all the branches get easier to
test. Also, the update operation gets so simple, that we see it
belongs over in packet.rs.

* Thanks clippy

cute
2019-01-10 13:34:48 -07:00
Greg Fitzgerald 885fe38c01 Move BloomHashIndex into its own module
This trait is for bloom, not crds.

Slightly better would be to put it in the SDK so that the trait
implementations could go into hash and pubkey, but if we don't
want compatibility constraints, this is the next best thing.
2019-01-10 10:22:16 -08:00
Pankaj Garg 2dbe8fc1a9
Refactor vote signer code (#2368)
* Refactor vote signer code

* fixed test compilation errors

* address clippy errors

* fix missing macro_use

* move macro use

* review comments
2019-01-10 09:21:38 -08:00
Sagar Dhawan 7122139e12
Rewrite TPU forwarder test (#2344) 2019-01-10 13:50:28 +05:30
Michael Vines 4e6c03c9da Avoid holding a read lock during IO 2019-01-10 00:34:50 -07:00
Stephen Akridge 0c90e1eff6 Make entry_sender optional on window_service
window_service in replicator has no need to consume the the produced entries.
2019-01-09 15:15:47 -08:00
Stephen Akridge 491bca5e4b Remove ledger.rs
Split into entry.rs for entry-constructing functions and EntrySlice
trait and db_ledger.rs for ledger helper test functions.
2019-01-09 15:15:47 -08:00
Stephen Akridge ebd676faaa Rename Block to EntrySlice 2019-01-09 15:15:47 -08:00
Stephen Akridge 045c5e8556 Remove most of the old ledger code
Removes LedgerWriter, read_ledger, LedgerWindow
2019-01-09 15:15:47 -08:00
Stephen Akridge 45b4cf2887 Remove store_ledger_stage which is no longer needed 2019-01-09 15:15:47 -08:00
Stephen Akridge 73eca72f14 Switch test to send a repair request to try and download from replicator
Removes need for read_ledger in the test and also tests replicator
download path.
2019-01-09 13:24:12 -08:00
Rob Walker 94494b64d7 whack commented out, obsolete, superceded test 2019-01-09 11:30:07 -08:00
Rob Walker b648f37b97
encapsulate erasure_cf (#2349) 2019-01-09 10:21:55 -08:00
Greg Fitzgerald f53810fcd2 Remove unused exit variable
The exit variable was only used by a test.
2019-01-08 20:22:31 -08:00
Stephen Akridge 3c6afe7707 Rename get_blob_bytes to read_blobs_bytes 2019-01-08 16:00:39 -08:00
Stephen Akridge 09296e0d71 Fix two storage tests
* test_encrypt_files_many_keys_multiple_keys passing
  - buffer chunk size unified between single key and multiple key path,
    which shouldn't be necessary but can fix later.
* test_encrypt_file_many_keys_bad_key_length passing
2019-01-08 16:00:39 -08:00
Stephen Akridge 4b3d64ec9f Convert chacha_encrypt_file to work with db_ledger blobs directly 2019-01-08 16:00:39 -08:00
Rob Walker a904e15ecc
enscapsulate data_cf (#2336)
* enscapsulate data_cf
2019-01-08 15:53:44 -08:00
Greg Fitzgerald a82a5ae184 Delete unused code
The ignored test is still broken, but at least no longer creates a
window for no reason.

Also removed all remaining references to "ncp".
2019-01-08 14:09:50 -08:00
Rob Walker bafd90807d
encapsulate meta_cf (#2335) 2019-01-08 11:41:55 -08:00
jackcmay 0f8ea6872e
Add missing error counters and load_account test cases (#2327) 2019-01-08 09:20:25 -08:00
Greg Fitzgerald d2431128c7 Remove WriteStage from TPU/TVU diagrams
Fixes #2312
2019-01-08 08:42:06 -08:00
Michael Vines a8b9899dee Add retry, restore ignored tests 2019-01-07 19:30:08 -08:00
Stephen Akridge d2cb4e003c Re-enable the --lib tests 2019-01-07 15:28:20 -08:00
Greg Fitzgerald 6000df9779 Optimize has_duplicates() for short slices 2019-01-07 13:20:04 -07:00
Greg Fitzgerald 24963e547c with_subset() -> get_subset_unchecked_mut()
A simpler, safer, and better documented use of unsafe code
2019-01-07 13:20:04 -07:00
Michael Vines 46d44ca99c Add make_rpc_request retry mechanism 2019-01-07 11:02:35 -08:00
jackcmay 0c52df7569
Consolidate locks and error handling when loading accounts(#2309) 2019-01-06 22:06:55 -08:00
Pankaj Garg 91bd38504e
Use vote signer service in fullnode (#2009)
* Use vote signer service in fullnode

* Use native types for signature and pubkey, and address other review comments

* Start local vote signer if a remote service address is not provided

* Rebased to master

* Fixes after rebase
2019-01-05 12:57:52 -08:00
Michael Vines b7dc9dbc76 RPC API now assumes a drone running on the bootstrap leader 2019-01-04 18:45:55 -08:00
Michael Vines 8b357dcb32
cargo fmt 2019-01-04 16:39:04 -08:00
Michael Vines 1f6346d880 De-dup ledgers - db_ledger is now the only ledger written to disk 2019-01-04 16:37:00 -08:00
Jack May b7bd38744c Spelling and formatting 2019-01-04 16:04:31 -08:00
Jack May f8a67e282a Ignore test_tpu_forwarder (#2307) 2019-01-04 16:02:50 -08:00
Greg Fitzgerald 0505d7bd32 Don't double-clone every account 2019-01-03 17:42:37 -07:00
Pankaj Garg 00d310f86d
Remove some metrics datapoint, as it was causing excessive logging (#2287)
- 100 nodes test was bringing down the influx DB server
2019-01-03 09:25:11 -08:00
Sagar Dhawan 0bea870b22
Dynamic N layer 'avalanche' broadcast and retransmit (#2058)
* Dynamic N layer avalanche broadcast and retransmit
2019-01-02 14:16:15 +05:30
Michael Vines 58f2598d5d Revert "Validators make a transaction to advertise their storage last_id"
This reverts commit a1759aed19.
2018-12-23 14:02:09 -08:00
carllin 58a4905916
Make reconstruct_entries_from_blobs() support Blobs and borrowed SharedBlobs, make distinction between to_blobs and to_shared_blobs (#2270) 2018-12-22 19:30:30 -08:00
Michael Vines 2c9607d5da Rename getConfirmation -> getConfirmationTime 2018-12-22 12:47:02 -08:00
Stephen Akridge a1759aed19 Validators make a transaction to advertise their storage last_id
* Also implement more storage contract logic
* Add transactions for proof validation,
* Move storage state members into system storage account userdata
2018-12-21 15:45:30 -08:00
Sathish 1a3387706d
Spawn threads based on cpu count (#2232) 2018-12-21 13:55:45 -08:00
Pankaj Garg 951d6398a0
Rename finality to confirmation (#2250)
* Rename finality to confirmation

* fix cargo fmt errors
2018-12-20 15:47:48 -08:00
Pankaj Garg 7c98545b33
Use newer votes to calculate confirmation time (#2247) 2018-12-20 15:27:47 -08:00
Pankaj Garg bb1060bdad
Reduce ticks per block to increase voting frequency (#2242) 2018-12-20 14:43:03 -08:00
Michael Vines 7ad45a91ec Fix compile error 2018-12-20 13:47:36 -08:00
Michael Vines 034c5d0422 db_ledger now fully encapsulates rocksdb 2018-12-20 12:32:25 -08:00
carllin 7148c14178
Debug broadcast (#2233)
* Account for duplicate blobs in process_blobs

* Increase max bytes for level base to match write buffer
2018-12-20 12:12:04 -08:00
Michael Vines 93fb61dc8f Re-export rocksdb::DBRawIterator until it can be encapsulated 2018-12-20 10:38:03 -08:00
Michael Vines b36ceb5be4 Remove rocksdb dependency from result.rs 2018-12-20 10:38:03 -08:00
Michael Vines 37d7ad819b Purge DB::destroy() usage 2018-12-20 10:38:03 -08:00
carllin 666af1e62d
Debug broadcast (#2208)
* Add per cf rocksdb options, increase compaction and flush threads

* Change broadcast stage to bulk write blobs

* add db_ledger function specifically for broadcast

* fix broken tests

* fix benches
2018-12-19 16:11:47 -08:00
Rob Walker b4f4347d6e
add some more tests (#2217) 2018-12-18 17:27:03 -08:00
Rob Walker 5c866dd000
test drive new coverage stuff (#2216) 2018-12-18 16:44:27 -08:00
Pankaj Garg 974249f2a5
Parallelize entry processing in replay stage in validators (#2212)
* Parallelize entry processing in replay stage in validators

- single threaded entry processing is not utlizing CPU cores to the fullest

* fix tests and address review comments
2018-12-18 16:06:05 -08:00
Rob Walker a65022aed7
DbLedger doesn't need to be mut, doesn't need an RwLock (#2215)
* DbLedger doesn't need to be mut, doesn't need an RwLock

* fix erasure cases
2018-12-18 15:18:57 -08:00
Sagar Dhawan ffdc1814c6 Add counters for gossip verification failures (#2094) 2018-12-17 20:12:50 -08:00
carllin 9720ac0019
Fix try_erasure() (#2185)
* Fix try_erasure bug

* Re-enable asserts in test_replicator_startup

* Add test for out of order process_blobs
2018-12-17 15:34:19 -08:00
anatoly yakovenko 9a6e27ac36
Accounts is to big, should be its own module (#2198)
Account module is to big, should be in its own module.
2018-12-17 12:41:23 -08:00
Michael Vines 07202205c4 Revert "ignore unstable tests"
This reverts commit bd7ef5d445071329a3b49b1f8be71b602226bbec.
2018-12-17 10:47:32 -08:00
Michael Vines b7144560c9 Include port number when gossip bind_to fails 2018-12-17 09:31:31 -08:00
anatoly yakovenko 4be6d01dfb
Move last ids (#2187)
* Break out last_ids into its own module
* Boot SignatureNotFound from BankError
* No longer return BankError from LastIds methods
* No longer piggypack on BankError for a LastIds signature status
* Drop all dependencies on the bank
* SignatureStatus -> Status and LastIds -> StatusDeque
* Unstable tests, issue 2193
2018-12-17 07:55:56 -08:00
Michael Vines fd562cb9e2 Rust 2018 cleanup 2018-12-14 21:57:15 -08:00
Michael Vines 5bf9a20d42 fullnode-config no longer depends on src/ 2018-12-14 20:13:34 -08:00
Michael Vines eedc8c7812 Move src/netutil.rs into its own crate 2018-12-14 20:13:34 -08:00
carllin 9ef5e51c0f
Cleanup slot remnants in db_ledger (#2153)
* Cleanup slot remnants in db_ledger
2018-12-14 17:05:41 -08:00
Sathish 8ee0e9632c
Switch to using hashbrown version of HashMap and (#2158)
HashSet for improved performance and memory usage
2018-12-14 15:10:10 -08:00
Michael Vines 6ac466c0a4 Move src/logger.rs into logger/ crate to unify logging across the workspace 2018-12-14 13:10:43 -08:00
Michael Vines d45fcc4381 Move src/wallet.rs into wallet/ crate 2018-12-14 12:15:18 -08:00
carllin 6fcccedb70
align tick entries' tick_height with actual number of ticks in bank (#2147) 2018-12-14 02:25:50 -08:00
Michael Vines 0fe6d61036 Move binaries from src/bin into their own crate 2018-12-13 22:25:27 -08:00
Pankaj Garg 091b21fae7
Vote every number of ticks (#2141)
* Vote every number of ticks

* address review comments

* fix for failing leader rotation tests

* remove check for vote failure from replay tests
(as votes will be cached and transmitted when leader is available)
2018-12-13 18:43:10 -08:00
Sathish 7325b19aef
Do not allocate for each metrics submission (#2146) 2018-12-13 16:40:00 -08:00
Stephen Akridge 7cdbbfa88e Storage stage updates
* Remove logging init from storage program: saw a crash in a test
  indicating the logger being init'ed twice.
* Add entry_height mining proof to indicate which segment the result is
  for
* Add an interface to get storage miner pubkeys for a given entry_height
* Add an interface to get the current storage mining entry_height
* Set the tvu socket to 0.0.0.0:0 in replicator to stop getting entries
  after the desired ledger segment is downloaded.
* Use signature of PoH height to determine which block to download for
  replicator.
2018-12-13 11:30:12 -08:00
Rob Walker 9880a86f80
remove prev_id, unused (#2150) 2018-12-13 09:24:38 -08:00
Michael Vines 647e5d76b0 Move solana-fullnode into fullnode/ 2018-12-13 01:45:29 -08:00
Michael Vines 7e4af9382e Move solana-upload-perf into upload-perf/ 2018-12-13 01:06:40 -08:00
Michael Vines 282d4a3563 Move solana-keygen into keygen/ 2018-12-13 01:06:40 -08:00
Michael Vines cafeef33c3 Relocate all keypair generation into one location: sdk/src/signature.rs 2018-12-13 01:06:40 -08:00
Rob Walker 4f48f1a850
add db_ledger genesis, rework to_blob(), to_blobs() (#2135) 2018-12-12 20:42:12 -08:00
Carl a05a378db4 cleanup 2018-12-12 19:12:51 -08:00
Carl 245362db96 Make a dummy version of serving repairs from db_ledger 2018-12-12 19:12:51 -08:00
Mark b1b190b80d Fix too many args in Tvu::new (#2114)
* Reduce args in Tvu::new under to 8

Now pass in sockets through a the crate::tvu::Sockets struct

Move ClusterInfo.keypair to pub(crate) in order to remove redundant
signing keypair parameter

* remove commented code
2018-12-12 18:57:48 -08:00
Rob Walker 3408ce89a7
add check_tick_height (#2144) 2018-12-12 18:52:11 -08:00
Michael Vines 75f407e191 Provide entire elf to bpf_loader 2018-12-12 17:14:50 -08:00
Mekagoza 9b81696a09 remove obsoleted TODO 2018-12-12 16:26:59 -08:00
Mekagoza 80e19e0ad7 Encapsulate accounts of solana:🏦:Accounts
Make the field private and expose an account_values() method that
returns the values iterator from the internal hashmap
2018-12-12 16:26:59 -08:00
carllin ae903f190e
Broadcast for slots (#2081)
* Insert blobs into db_ledger in broadcast stage to support leader to validator transitions

* Add transmitting real slots to broadcast stage

* Handle real slots instead of default slots in window

* Switch to dummy repair on slots and modify erasure to support leader rotation

* Shorten length of holding locks

* Remove logger from replicator test
2018-12-12 15:58:29 -08:00
Pankaj Garg 3413ecc2bd
Change query used to find list of nodes in the network (#2124)
* Change query used to find list of nodes in the network

* include "All" option for host selection
2018-12-12 12:38:00 -08:00
Sagar Dhawan 4788a4f775
Correctly describe repair and retransmit peers (#2110) 2018-12-11 15:51:47 -08:00
Pankaj Garg 9243bc58db
Metrics for window repair (#2106)
* Metrics for window repair

- Also increase max repair length

* fix vote counters, and add repair window graph

* update per node graphs

* revert max repair length change
2018-12-11 15:43:41 -08:00
Rob Walker 4d67aca919
add genesis and read_ledger to db_ledger (#2097) 2018-12-11 09:14:23 -08:00
Rob Walker aeee25e703
add tick_height to Entry to be able to repair by period, chain forks of Entries, etc. (#2096) 2018-12-10 20:03:04 -08:00
Greg Fitzgerald 01c524ddd2 Revert changes to counter names 2018-12-10 15:26:43 -07:00
Greg Fitzgerald 5e703dc70a Free up the term 'replicate' for exclusive use in replicator
Also, align Sockets field names with ContactInfo.
2018-12-10 15:26:43 -07:00
Pankaj Garg bc96bd3410
Fix peer count in edge dashboard (#2090)
Fixes #2075
2018-12-10 14:24:32 -08:00
Michael Vines 4ea1c030bc Give bootstrap leader one more token 2018-12-10 13:24:14 -08:00
Stephen Akridge 172e511e56 Use retry_transfer to test multiple times for replicator tokens
May fix failures in CI where replicator is trying to do an airdrop.
2018-12-10 12:19:00 -08:00
Tyera Eulberg 4481efd51e
Merge pull request #2084 from CriesofCarrots/fix-wallet-accountinuse
Fix wallet accountinuse
2018-12-10 12:20:55 -07:00
Greg Fitzgerald 0ef099421c cargo fmt 2018-12-08 23:19:55 -07:00
Greg Fitzgerald f1ae5b1795 Fix warnings 2018-12-08 23:19:55 -07:00
Greg Fitzgerald a8d6c75a24 cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition-idioms 2018-12-08 23:19:55 -07:00
Greg Fitzgerald c49e2f8bbd cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition 2018-12-08 23:19:55 -07:00
Greg Fitzgerald ec5a8141eb cargo fix --edition 2018-12-08 23:19:55 -07:00
Sathish 2de45a4da5
Update airdrop tokens to 3 for fullnode (#2051)
Filter out leader while computing the super majority stake
2018-12-08 16:54:42 -08:00
Michael Vines 73b9ee9e84 Add solana_ prefix to native_loader program
This allows its logging to show up in the default RUST_LOG=solana=info
log setting
2018-12-08 11:04:45 -08:00
Greg Fitzgerald 0a83b17cdd
Upgrade to Rust 1.31.0 (#2052)
* Upgrade to Rust 1.31.0
* Upgrade nightly
* Fix all clippy warnings
* Revert relaxed version check and update
2018-12-07 20:01:28 -07:00
Tyera Eulberg 571522e738 Update jsonrpc version 2018-12-07 17:47:54 -07:00
Stephen Akridge 3441d3399b Replicator rework
* Move more of the replicator logic into the replicator class
* Add support for the RPC interface to query the storage last_id value
  that the replicator would sign and use to pick a block.
* Fix replicator connecting to gossip and change test to exercise that
  scenario.
2018-12-07 15:20:36 -08:00
Michael Vines f4b26247c0 Genesis only needs a keypair, not the entire fullnode::Config 2018-12-06 16:31:24 -08:00
Michael Vines c4b8f0cd2f bench-tps will now generate an ephemeral identity if not provided with one
Also simplify scripts as a result
2018-12-06 16:30:48 -08:00
Greg Fitzgerald 97b1156a7a Rename Ncp to GossipService
And BroadcastStage to BroadcastService since it's not included in the
TPU pipeline.
2018-12-06 15:48:19 -07:00
Michael Vines cc2f448d92 Add fullnode --no-leader-rotation flag 2018-12-06 11:30:19 -08:00
Michael Vines 632425c7d7 Move native_loader under programs/native/ 2018-12-05 14:32:42 -08:00
dependabot[bot] ad3e36a7ab Bump rand from 0.5.5 to 0.6.1 (#1891)
* Bump rand from 0.5.5 to 0.6.1

Bumps [rand](https://github.com/rust-random/rand) from 0.5.5 to 0.6.1.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/commits)

Signed-off-by: dependabot[bot] <support@dependabot.com>

* Fix conflicts and deprecated usages

* Fix benches
2018-12-05 14:12:10 -08:00
Michael Vines 1bcafca690
Find test_tx again 2018-12-05 13:29:29 -08:00
Rob Walker e21b6d9db3
ensure we'd actually have N hashes per tick (#2011) 2018-12-05 12:49:41 -08:00
carllin 9c30bddb88
Rocks db erasure decoding (#1900)
* Change erasure to consume new RocksDb window

* Change tests for erasure

* Remove erasure from window

* Integrate erasure decoding back into window

* Remove corrupted blobs from ledger

* Replace Erasure result with result module's Result
2018-12-05 12:47:19 -08:00
Michael Vines 59e6bd115e system_program must be a static lib as it allocates Account memory 2018-12-05 10:49:06 -08:00
Michael Vines 15aef079e3 Include builtin programs for ledger verification 2018-12-05 10:49:06 -08:00
Michael Vines 42689d4842 cargo fmt 2018-12-05 10:49:06 -08:00
Michael Vines 214ed3667c Move system_transaction out of src/ 2018-12-05 10:49:06 -08:00
Michael Vines 122627dda2 Move loader_transaction out of src/ 2018-12-05 10:49:06 -08:00
Michael Vines 7af95eadcc Move vote_transaction out of src/ 2018-12-05 10:49:06 -08:00
Michael Vines 9ee858a00c Move budget_program out of src/ 2018-12-05 10:49:06 -08:00
Michael Vines 27d456bf93 Move storage_program out of src/ 2018-12-05 10:49:06 -08:00
Michael Vines ea6e042a6f Move vote_program out of src/ 2018-12-05 10:49:06 -08:00
Michael Vines a594f56c02 Add token_program.rs to sdk/ 2018-12-05 10:49:06 -08:00
Michael Vines e6fa74fe69 Remove custom Error enum, just use ProgramError 2018-12-05 10:49:06 -08:00
Michael Vines 9a4f8199d6 Move system_program out of src/ 2018-12-05 10:49:06 -08:00
Michael Vines ae0be1e857 Remove bpf_loader.rs 2018-12-05 10:49:06 -08:00
Michael Vines d010cac8a5 Remove token_program.rs 2018-12-05 10:49:06 -08:00
Michael Vines 63a758508a Add sdk native_loader.rs 2018-12-05 10:49:06 -08:00
Stephen Akridge c1c2f1f0a9 Cleanup ad-hoc rpc address formation
Lots of places where we are forming rpc addresses.
2018-12-03 18:13:55 -08:00
Michael Vines 777a0a858e Move ProgramError into sdk/ 2018-12-03 13:50:00 -08:00
Michael Vines 68e99c18c0 Remove duplicate SYSTEM_PROGRAM_ID 2018-12-03 13:50:00 -08:00
Michael Vines c99f93e40a Remove signature.rs indirection 2018-12-03 13:50:00 -08:00
carllin 4ae58cc854
Change range of leader scheduler to match current broadcasts (#1920) 2018-12-03 00:10:43 -08:00
Sagar Dhawan b21facab7b
Add metrics for prune messages (#1981) 2018-12-01 14:05:40 -08:00
Sagar Dhawan 34c3a0cc1f
Add signature verification to gossip (#1937) 2018-12-01 12:00:30 -08:00
Pankaj Garg 549bfe7412
Vote signing JSON RPC service (#1965)
* Vote signing JSON RPC service

- barebone service that listens for RPC requests

* Daemon for vote signer service

* Add request APIs for JSON RPC

* Cleanup of cargo dependencies

* Fix compiler error
2018-11-30 15:07:08 -08:00
Rob Walker 3ca826a480
re-enable test_tpu_forwarder (#1964) 2018-11-30 13:52:37 -08:00
Michael Vines bad0b55ab6 Expose which keys signed the Transaction in the SDK 2018-11-30 08:16:23 -08:00
Michael Vines 0878bd53d9 Delete stub src/transaction.rs 2018-11-29 23:07:57 -08:00
Stephen Akridge de910e1169 Make test_pubkey_distribution faster
multi-thread pubkey histogram generation.
2018-11-29 17:37:37 -08:00
Stephen Akridge 9684737de7 Add wait before checking confirm again
Otherwise we can quickly check that we
have no signature 4 times in a row.
2018-11-29 15:32:58 -08:00
Stephen Akridge ecc87ab1aa Add a an optional timeout to thin_client
Such that a negative test like test_transaction_count doesn't
have to wait num_retries * default_timeout.
2018-11-29 13:53:40 -08:00
Michael Vines fa359c6fc4 Merge vote new and register transactions 2018-11-29 12:31:34 -08:00
Stephen Akridge 8cc751d1cc Improve RPC service startup error messages with actual error
Error always fixed to message about ports but that's not the only
error that can occur.
2018-11-29 11:39:42 -08:00
Stephen Akridge 978fd6858f Move replicator_startup_test to integration test set
Sometimes fails when run multithreaded with other tests.
2018-11-29 11:39:42 -08:00
Michael Vines 41689256c6 Ensure key[0] is signed 2018-11-29 10:26:46 -08:00
Pankaj Garg 194e3100a9
Additional checks in test_bank_checkpoint_zero_balance (#1943) 2018-11-28 12:40:34 -08:00
Rob Walker 8ad4464d4b add tests for other "from" indexes signing (or not) 2018-11-28 07:56:04 -08:00
Rob Walker e7b0a736f5 verify signature is on the from account 2018-11-28 07:56:04 -08:00
Rob Walker 167eb01735 optimize bench-tps and rpc_request to work on crappy WSL boxes 2018-11-27 22:45:08 -08:00
Michael Vines 8fb5d72b13 Make insufficient tokens message more helpful 2018-11-27 17:37:25 -08:00
Pankaj Garg 8947c5a4aa
Set account to default if the balance reaches 0 in a checkpoint bank (#1932)
Fixes: #1931
2018-11-27 14:17:29 -08:00
Greg Fitzgerald a7562c9be1 Extract execute_transaction() from the bank 2018-11-27 12:35:52 -07:00
Greg Fitzgerald 08dc169f94 Hoist load_loaders()
This makes execute_transactions() stateless.
2018-11-27 12:35:52 -07:00
Greg Fitzgerald f549d8ac74 Hoist loading of loaders
This might cause a TPS boost in batched BPF transactions, since
now it'll only clone its account once per transaction instead of
once per instruction.
2018-11-27 12:35:52 -07:00
Greg Fitzgerald 1ac7536286 Pass executable_accounts into with_subset() 2018-11-27 12:35:52 -07:00
Greg Fitzgerald 88e270723f Move markdown book out of src/ 2018-11-26 09:44:19 -08:00
Michael Vines a13e25f083 Ignore flaky test_tpu_forwarder 2018-11-26 09:27:21 -08:00
Greg Fitzgerald 3d3a30e200 Fix mdbook test 2018-11-26 07:51:10 -08:00
Greg Fitzgerald 76b83ac0f4 Move testnet demos into the book
Have git readme focus on fullnode development and the book focus on
users.
2018-11-26 07:51:10 -08:00
Greg Fitzgerald 903a9bfd05 s/contract/program/ 2018-11-26 08:20:42 -07:00
Greg Fitzgerald 655ee1a64b Fix typos 2018-11-26 08:20:42 -07:00
Greg Fitzgerald e0e6c3fdb2 Extract execute_instruction() to seed new runtime module
Fixes #1528
2018-11-26 08:20:42 -07:00
Greg Fitzgerald 31f00974f2 Hoist the lookup of executable accounts 2018-11-26 08:20:42 -07:00
Greg Fitzgerald c3218bb9c2 Hoist tick_height 2018-11-26 08:20:42 -07:00
Greg Fitzgerald d2972024de Uppercase acronyms
Looks like there will be very little Rust code in the markdown book
so switching back to English capitalization conventions.
2018-11-25 22:58:07 -07:00
Greg Fitzgerald dc0a2ca656
Move disclaimer down a bit
Odd to see a disclaimer before knowing anything about what you're reading
2018-11-25 20:27:35 -07:00
Greg Fitzgerald c3fb9d5549
Cleanup book (#1904)
* Cleanup book

* Distinguish upstream from downstream validators
* Add BroadcastStage to Fullnode/Tpu diagrams
* First attempt to re-describe the runtime

* Reorg book

Push back details of the fullnode implementation
2018-11-25 16:58:38 -07:00
Sathish f5b5c54d7d
Update condition for nosigverify (#1903) 2018-11-25 13:11:07 -08:00
Sathish 9f0b06bb86 Filter out leader node while retransmitting blobs (#1894) 2018-11-24 20:33:49 -08:00
carllin 57a384d6a0
Rocks db window service (#1888)
* Add db_window module for windowing functions from RocksDb

* Replace window with db_window functions in window_service

* Fix tests

* Make note of change in db_window

* Create RocksDb ledger in bin/fullnode

* Make db_ledger functions generic

* Add db_ledger to bin/replicator
2018-11-24 19:32:33 -08:00
Greg Fitzgerald 69802e141f Add the story of how this codebase came to be 2018-11-24 14:39:53 -07:00
Greg Fitzgerald 6fc02b7424 Detect legacy programs upfront 2018-11-24 11:56:51 -07:00
Greg Fitzgerald 30cdd85028 Implement the same interface in all builtin programs 2018-11-24 11:56:51 -07:00
Greg Fitzgerald 871dd47019 Extract the part of execute_instruction that should only return a ProgramError
TODO: hoist load_executable_accounts() and then change
process_instruction() to return ProgramError.
2018-11-24 11:56:51 -07:00
Greg Fitzgerald 37f8dd57e2 Extract ProgramError from BankError 2018-11-24 11:56:51 -07:00
Greg Fitzgerald f827bfd83f Remove instruction index parameter 2018-11-24 11:56:51 -07:00
Greg Fitzgerald b3af930153 Rename process_transaction to process_instruction 2018-11-24 11:56:51 -07:00
Greg Fitzgerald cd488b7d07 Hoist program static methods to top-level functions 2018-11-24 11:56:51 -07:00
Rob Walker e2373ff51a add nosigverify command line option to ease debug 2018-11-23 16:55:04 -08:00
Greg Fitzgerald b3d2c900cd Rename BudgetState to BudgetProgram 2018-11-23 13:25:17 -07:00
Michael Vines d5adec20a3 get_ip_addr: Fall back to loopback if no better option exists 2018-11-23 13:24:41 -05:00
carllin 942256a647
Add db_ledger benchmarks (#1875)
* Add db_ledger benchmarks

* ignore benches in CI, due to timeouts
2018-11-23 06:12:43 -08:00
carllin a3321a5d80
Fix endianess in db_ledger to account for the default byte-comparator used by Rocksdb (#1885) 2018-11-22 01:35:19 -08:00
carllin 521de13571
Add maximum repair length to db_window (#1886)
* Add maximum repair length to db_window
2018-11-21 23:44:49 -08:00
Greg Fitzgerald 3abf6a8a30 Reorg the markdown book to cater to app devs
First, talk about how a client interacts with Solana to do useful
things. Then describe how the fullnode you're talking to works and
why it's so very fast.  Last, why that fullnode you don't trust
does what you asked it to anyway.
2018-11-21 15:49:57 -08:00
jackcmay 59163e2dd9
Optimize some CI stuff (#1880)
* CI Optimizations
2018-11-21 12:16:16 -08:00
carllin d9dabdfc74
Rocks db window utils (#1851)
* Implement new ledger module based on RocksDb

* Add db_window module for windowing functions from RocksDb
2018-11-19 23:20:18 -08:00
Rob Walker 6b910d1bd4 add tpu_forwarding, simplify ClusterInfo::new() from Result<Self> to Self 2018-11-19 20:45:49 -08:00
Rob Walker bbd9ea8c00
Delete settings.rs.foo 2018-11-19 13:39:08 -08:00
Anatoly Yakovenko fc67a968e8 Use known keys in the unit test to avoid random false positives. 2018-11-19 13:41:24 -07:00
Rob Walker 3d113611cc
remove Result<> return from ClusterInfo::new() (#1869)
strip Result<> for ClusterInfo::new()
2018-11-19 11:25:14 -08:00
Sagar Dhawan 07667771ef
Fix Gossip Pushes going to invalid addresses (#1858) 2018-11-17 19:57:28 -08:00
Michael Vines 3822c29415 Route program_id to program entrypoint 2018-11-17 19:42:03 -08:00
Greg Fitzgerald ff386d6585 Add disclaimer to markdown book
copy-paste from readme
2018-11-17 19:56:08 -07:00
Greg Fitzgerald 88ddb31477 teminology cleanup: leader slots and voting rounds 2018-11-17 18:56:13 -07:00
Greg Fitzgerald 077d1a41f1 Add too book 2018-11-17 18:56:13 -07:00
Greg Fitzgerald 857ab8662e backticks and missing variable descriptions 2018-11-17 18:56:13 -07:00
Greg Fitzgerald a17f9bd0f4 Work towards adding leader rotation to the book 2018-11-17 18:56:13 -07:00
Greg Fitzgerald f4b9e93b11 Migrate storage RFC to book 2018-11-17 18:55:08 -07:00
Greg Fitzgerald 2c11bf2e66 Various book cleanup
* Merge Leader and Validator diagrams
* New sdk-tools diagram
* Move terminology to just after introduction
* Purge use of LAMPORT as an acronym
* Add notes about persistent storage
2018-11-17 17:50:29 -08:00
Michael Vines 719e14b30a Add an explicit state of a reserved signature
An RPC client that fetches the signature status before the bank finishes
executing the corresponding Transaction should receive SignatureNotFound
instead of Confirmed
2018-11-17 16:40:23 -08:00
Michael Vines d1f01b5209 Fix clippy lint 2018-11-17 15:54:21 -08:00
Michael Vines d96a6b42a5 Move drone into its own crate 2018-11-16 20:42:21 -08:00
carllin cf95708c18 Set drone address to always be the initial network entry point (#1847)
* Set drone address to always be the initial network entry point, so that even when leaders rotate the client can still find the drone

* Extract drone address as a separate argument to bench-tps

* Add drone port to client.sh instead of setting it in bench-tps

* Add drone entrypoint to scripts

* Fix build error
2018-11-16 19:56:26 -08:00
Michael Vines e1c7b99450
Accounts get kicked if no tokens 2018-11-16 18:53:37 -08:00
Sagar Dhawan 12ae7b9a6b
Add test for tvu POH verification (#1844) 2018-11-16 15:48:10 -08:00
Michael Vines 6ac5700f2e Move metrics into its own crate 2018-11-16 15:10:07 -08:00
Michael Vines a0dd8617be Remove airdrop from fullnode 2018-11-16 13:25:55 -08:00
Rob Walker 1576072edb
remove spurious eprintln!() 2018-11-16 10:21:58 -08:00
carllin 03d206a7ca
Check for valid tvu, not tpu in broadcast (#1836) 2018-11-15 23:30:22 -08:00
Sathish c973de1d76
Decouple log and metrics rate (#1839)
Use separate env for log and metrics rate.

Set default log level to WARN if unset.
2018-11-15 22:27:16 -08:00
Stephen Akridge 71336965a6 Limit targets to 4 in bench-tps
Transaction got bigger so can only fit 4 targets in a
Transaction now.
2018-11-15 20:25:07 -08:00
Michael Vines e791d0f74d Drone now returns signed airdrop transactions 2018-11-15 17:13:13 -08:00
Stephen Akridge 3543a9a49f Add check for missing signature with fee'ed transaction
And update fetch-perf-libs version
2018-11-15 16:23:13 -08:00
Stephen Akridge 7dd198a99e Change signed_key to index into account_keys
If index is within the signed keys range.
2018-11-15 16:23:13 -08:00
Stephen Akridge e048116ab2 Remove signed_keys
Use first signatures.len() of account_keys for signing
2018-11-15 16:23:13 -08:00
Stephen Akridge cda9ad8565 Multiple signatures for transactions
With multiple instructions in a TX may need
multiple signatures.

Fixes #1531
2018-11-15 16:23:13 -08:00
carllin 928f375683
Rocks db (#1792)
* Add rocksdb crate

* Implement new ledger module based on RocksDb
2018-11-15 15:53:31 -08:00
Rob Walker d3e521f70e
accept other socket errors, ignore unless out of tries (#1835) 2018-11-15 15:49:37 -08:00
Michael Vines a7ee428214 Fix build 2018-11-15 14:06:57 -08:00
anatoly yakovenko a41254e18c
Add scalable gossip library (#1546)
* Cluster Replicated Data Store

Separate the data storage and merge strategy from the network IO boundary.
Implement an eager push overlay for transporting recent messages.

Simulation shows fast convergence with 20k nodes.
2018-11-15 13:23:26 -08:00
Sagar Dhawan c81a3f6ced
Fix RPC address clashes on local multi-node testnet (#1821)
* Fix RPC address clashes on local multi-node testnet
2018-11-15 10:42:02 -08:00
Tyera Eulberg a5412fc0cd Fix find port functions 2018-11-15 10:45:39 -07:00
Michael Vines 5a8938209b Expose tick_height to native programs 2018-11-14 10:33:27 -08:00
Michael Vines e33f3a2562 Publish expected native program entrypoint in sdk/ 2018-11-14 10:33:27 -08:00
Sathish 9fe210c454
Add host information to db entries (#1778)
Add new field to each db entry identifying the host
that it originated from.
2018-11-13 21:54:15 -08:00
Jack May e7b6c8b7e0 Accounts get kicked if no tokens 2018-11-13 17:23:13 -08:00
Jack May 4e553ea095 test_replicate fails locally, ignore 2018-11-13 17:13:25 -08:00
Jack May 0c46f15f94 test_rpc_new fails locally, ignore for now 2018-11-13 17:12:25 -08:00
Sathish 7b92497d21
Update counters irrespective of logging level (#1799) 2018-11-13 16:55:14 -08:00
Sagar Dhawan 729d28d910 Add poh verification before processing entries
- Replicate stage now verifies entries delivered
  by the window
- Minor refactor of entries_from_blobs
2018-11-13 14:17:00 -08:00
carllin 6335be803c
Broadcast last tick before leader rotation (#1766)
* Broadcast last tick before leader rotation to everybody on network

* Add test

* Refactor broadcast
2018-11-13 02:21:37 -08:00
Sagar Dhawan a77b1ff767
Revert "Migrate from ring to ed25519-dalek" (#1798)
* Revert "Migrate from ring to ed25519-dalek"

This reverts commit 7c610b216b.

* Fix test failures with revert
2018-11-12 22:34:43 -08:00
Tyera Eulberg c7a67b5a02 Add deploy command to test 2018-11-12 18:21:16 -07:00