Commit Graph

2697 Commits

Author SHA1 Message Date
Pankaj Garg 639c93460a
Write stage optimizations (#1534)
- Testnet dashboard shows that channel pressure for write stage
  is incrementing on every iteration of write.
- This change optimizes ledger writing by removing cloning of map
  and reducing calls to flush
2018-10-17 13:02:32 -07:00
Rob Walker 7611730cdb move off /tmp 2018-10-17 12:15:30 -07:00
Rob Walker 9df9c1433a remove another use of /tmp 2018-10-17 12:15:30 -07:00
Rob Walker 4ea422bcec run integration tests serially 2018-10-17 11:37:10 -07:00
Greg Fitzgerald 6074e4f962 Attempt to stabilize the test suite
The integration tests are allowed to open sockets, so running them
in parallel may cause "Too many open files" errors. This patch
runs the unit tests in parallel and the integration test serially.
2018-10-17 11:37:10 -07:00
Nikhil Saraf d52e6d01ec typo in readme 2018-10-17 02:04:05 -06:00
Jack May 63caca33be SystemProgram test was failing due to expected panic 2018-10-16 18:02:44 -07:00
Jack May 64efa62a74 enable logging in loaders 2018-10-16 16:55:11 -07:00
Rob Walker 912eb5e8e9
remove bank.is_leader, dead code (#1516) 2018-10-16 15:26:44 -07:00
Jack May bb628e8495 Rename loaders 2018-10-16 14:27:08 -07:00
Jack May d0c19c2c97 cargo fmt 2018-10-16 14:11:04 -07:00
Jack May 926fdb7519 Rename dynamic_program.rs to native_loader.rs 2018-10-16 14:11:04 -07:00
jackcmay c886625c83
Move from solana/rbpf fork to qmonnet/rbpf (#1511) 2018-10-16 13:13:54 -07:00
Pankaj Garg f6c10d8a2e
Add channel pressure for validator TVU stages (#1509) 2018-10-16 12:54:23 -07:00
anatoly yakovenko 2bd877528f
Par process entries (#1499)
* Parallel entry processor.
2018-10-16 12:09:48 -07:00
jackcmay d09889b1dd
Program bank integration (#1462)
Native, BPF and Lua loaders integrated into the bank
2018-10-16 09:43:49 -07:00
Tyera Eulberg 1b2e9122d5 Pubsub listen on random open port when rpc does (quiet some test errors) 2018-10-16 00:11:26 -06:00
Tyera Eulberg 7424388924 Fix session drop 2018-10-16 00:11:26 -06:00
Michael Vines 537436bd5e RPC PubSub now uses a well-known socket 2018-10-16 00:11:26 -06:00
Pankaj Garg 32fc0cd7e9
Fix bug introduced during RUST_LOG escaping (#1507)
* Fix bug introduced during RUST_LOG escaping
- remote node configuration should not be quoted

* shellcheck disable SC2090
2018-10-15 16:49:22 -07:00
Tyera Eulberg fb99494858
Improve rpc code coverage (#1487) 2018-10-15 11:01:40 -06:00
Greg Fitzgerald 5b4d4b97bc Upgrade to latest stable Rust, 1.29.2 2018-10-15 09:54:24 -06:00
Michael Vines c5180c8092 Permit RUST_LOG overrides 2018-10-14 12:40:37 -07:00
Greg Fitzgerald 515c200d86 Refactor and add test for new Entry::serialized_size() 2018-10-14 10:53:47 -06:00
Greg Fitzgerald 32aab82e32 Don't allocate to see if transactions will fit in a blob 2018-10-14 10:53:47 -06:00
anatoly yakovenko 6aaa350145
effeciently pack gossip responsens and only respond up to max size. (#1493) 2018-10-14 06:45:02 -07:00
Michael Vines d3b4dfe104 Add bool return to entrypoint signature to permit programs to fail transactions 2018-10-13 20:01:43 -07:00
Pankaj Garg 9fc30f6db4
Escape RUST_LOG configuration in remote-node.sh (#1489)
* Escape RUST_LOG configuration in remote-node.sh

- If it was set to #, it was causing other parameters to be commented out

* escape other variables as well

* disabled shell check

* Fix shellcheck error
2018-10-13 13:35:54 -07:00
Michael Vines 2d0f07091d Handle dynamic program dlopen failures gracefully 2018-10-13 11:31:10 -07:00
Michael Vines 3828eda507 Demote log messages 2018-10-13 11:31:10 -07:00
Michael Vines 1e736ec16d Demote log messages 2018-10-12 20:16:57 -07:00
Anatoly Yakovenko bba6437ea9 Use a single structure for last_ids and last_ids_sigs 2018-10-12 16:39:35 -07:00
Pankaj Garg e5ab9a856c
Upload bench output as build artifacts (#1478)
* Upload bench output as build artifacts

* Fix tags types

* Pull previous stats from metrics

* Change the default branch for comparison

* Fix formatting

* Fix build errors

* Address review comments

* Dedup some common code

* Add eval for channel info to find branch name
2018-10-12 15:13:10 -07:00
Tyera Eulberg 1515bba9c6
Use cluster_info in rpc to get current leader addresses (#1480) 2018-10-12 14:25:56 -06:00
Rob Walker 14a9ef4bbe
move PoH verification off bank.last_id() (#1476) 2018-10-12 11:50:34 -07:00
Michael Vines 041040c659 pubsub.rs -> rpc_pubsub.rs 2018-10-12 08:39:06 -07:00
carllin 47f69f2d24
1) Switch broken tests to generate an empty tick in their ledgers to use as last_id, 2) Fix bug where PoH generator in BankingStage did not referenced the last tick instead of the last entry on startup, causing ledger verification to fail on the new tick added by the PoH generator (#1479) 2018-10-12 00:39:10 -07:00
Stephen Akridge 9dd4dc2088 Mark failing tests as ignore 2018-10-11 15:32:36 -07:00
Tyera Eulberg b534c32ee3 New minor version for jsonrpc crates 2018-10-11 13:35:06 -06:00
Tyera Eulberg d2712f1457 Specify patch for jsonrpc crates 2018-10-11 11:38:14 -07:00
Stephen Akridge 183f560d06 Add raw entries interface to ledger for getting slices as [u8] 2018-10-11 09:40:34 -07:00
Michael Vines ae150c0897
Remove getAddress, it doesn't exist 2018-10-11 08:28:39 -07:00
Michael Vines 606e1396cf
Fix link 2018-10-11 08:25:38 -07:00
anatoly yakovenko 5c85e037f8
Tick entry ids as only valid last_ids (#1441)
Generate tick entry ids and only register ticks as the last_id expected by the bank.  Since the bank is MT, the in-flight pipeline of transactions cannot be close to the end of the queue or there is a high possibility that a starved thread will encode an expired last_id into the ledger.  The banking_stage therefore uses a shorter age limit for encoded last_ids then the validators.

Bench client doesn't send transactions that are older then 30 seconds.
2018-10-10 17:23:06 -07:00
Michael Vines 5c523716aa Ship native programs 2018-10-10 16:49:48 -07:00
Michael Vines 5f8cbf359e Use cdylib to avoid runtime libstd dependencies 2018-10-10 16:49:48 -07:00
Michael Vines e83834e6be Build native programs in release configuration 2018-10-10 16:49:48 -07:00
Michael Vines 02225aa95c Look for native programs in same directory as the current executable 2018-10-10 16:49:48 -07:00
carllin 9931ac9780
Leader scheduler plumbing (#1440)
* Added LeaderScheduler module and tests

* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage

* Add LeaderScheduler plumbing for Tvu, window, and tests

* Fix bank and switch tests to use new LeaderScheduler

* move leader rotation check from window service to replicate stage

* Add replicate_stage leader rotation exit test

* removed leader scheduler from the window service and associated modules/tests

* Corrected is_leader calculation in repair() function in window.rs

* Integrate LeaderScheduler with write_stage for leader to validator transitions

* Integrated LeaderScheduler with BroadcastStage

* Removed gossip leader rotation from crdt

* Add multi validator, leader test

* Comments and cleanup

* Remove unneeded checks from broadcast stage

* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role

* Set new leader in validator -> validator transitions

* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail

* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops

* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
2018-10-10 16:49:41 -07:00
Greg Fitzgerald 2ba2bc72ca Cleanup multisig lua 2018-10-10 17:17:17 -06:00