Commit Graph

87 Commits

Author SHA1 Message Date
Michael Vines 4bb6549895 Genesis block is now a json file 2019-01-25 09:05:15 -08:00
Stephen Akridge 491bca5e4b Remove ledger.rs
Split into entry.rs for entry-constructing functions and EntrySlice
trait and db_ledger.rs for ledger helper test functions.
2019-01-09 15:15:47 -08:00
Stephen Akridge ebd676faaa Rename Block to EntrySlice 2019-01-09 15:15:47 -08:00
Sathish 1a3387706d
Spawn threads based on cpu count (#2232) 2018-12-21 13:55:45 -08:00
Pankaj Garg 951d6398a0
Rename finality to confirmation (#2250)
* Rename finality to confirmation

* fix cargo fmt errors
2018-12-20 15:47:48 -08:00
Greg Fitzgerald ec5a8141eb cargo fix --edition 2018-12-08 23:19:55 -07:00
Sathish 2de45a4da5
Update airdrop tokens to 3 for fullnode (#2051)
Filter out leader while computing the super majority stake
2018-12-08 16:54:42 -08:00
Greg Fitzgerald 0a83b17cdd
Upgrade to Rust 1.31.0 (#2052)
* Upgrade to Rust 1.31.0
* Upgrade nightly
* Fix all clippy warnings
* Revert relaxed version check and update
2018-12-07 20:01:28 -07:00
Michael Vines 42689d4842 cargo fmt 2018-12-05 10:49:06 -08:00
Michael Vines 214ed3667c Move system_transaction out of src/ 2018-12-05 10:49:06 -08:00
Michael Vines c99f93e40a Remove signature.rs indirection 2018-12-03 13:50:00 -08:00
Michael Vines 0878bd53d9 Delete stub src/transaction.rs 2018-11-29 23:07:57 -08:00
Michael Vines d96a6b42a5 Move drone into its own crate 2018-11-16 20:42:21 -08:00
Michael Vines 6ac5700f2e Move metrics into its own crate 2018-11-16 15:10:07 -08:00
Rob Walker d831c5dcc9
remove dead poh code (#1747) 2018-11-08 12:55:23 -08:00
carllin 0636399b7a
Compute finality computation in new ComputeLeaderFinalityService (#1652)
* Move finality computation into a service run from the banking stage, ComputeLeaderFinalityService

* Change last ids nth to tick height, remove separate tick height from bank
2018-11-02 15:49:14 -07:00
carllin 298bd6479a
Add first leader to genesis (#1681)
* Add first leader to genesis entries, consume in genesis.sh

* Set bootstrap leader in the bank on startup, remove instantiation of bootstrap leader from bin/fullnode

* Remove need to initialize bootstrap leader in leader_scheduler, now can be read from genesis entries

* Add separate interface new_with_leader() in mint for creating genesis leader entries
2018-11-02 14:32:05 -07:00
Sathish dffa2eb04f
Do not parallelize deserialize operation (#1663)
Deserialize operations are faster when done serially with the
MT banking stage and helps with performance improvement with
reduced thread context switches.
2018-10-31 22:12:15 -07:00
Rob Walker 13bfdde228
remove ledger tail code, WINDOW_SIZE begone (#1617)
* remove WINDOW_SIZE, use window.window_size()
* move ledger tail, redundant with ledger-based repair
2018-10-30 10:05:18 -07:00
Michael Vines 236113e417 cargo fmt 2018-10-25 17:13:41 -07:00
carllin 55833e20b1
Create Poh Service (#1604)
* Create new Poh Service, replace tick generation in BankingStage
2018-10-25 14:56:21 -07:00
Pankaj Garg 5c483c9928 remove unused variable 2018-10-23 16:52:56 -06:00
Pankaj Garg a68c99d782 Fix transaction count on testnet dashboard 2018-10-23 16:52:56 -06:00
carllin 0bd1412562
Switch leader scheduler to use PoH ticks instead of Entry height (#1519)
* Add PoH height to process_ledger()

* Moved broadcast_stage Leader Scheduling logic to use Poh height instead of entry_height

* Moved LeaderScheduler logic to PoH in ReplicateStage

* Fix Leader scheduling tests to use PoH instead of entry height

* Change is_leader detection in repair() to use PoH instead of entry height

* Add tests to LeaderScheduler for new functionality

* fix Entry::new and genesis block PoH counts

* Moved LeaderScheduler to PoH ticks

* Cleanup to resolve PR comments
2018-10-18 22:57:48 -07:00
carllin 47f69f2d24
1) Switch broken tests to generate an empty tick in their ledgers to use as last_id, 2) Fix bug where PoH generator in BankingStage did not referenced the last tick instead of the last entry on startup, causing ledger verification to fail on the new tick added by the PoH generator (#1479) 2018-10-12 00:39:10 -07:00
anatoly yakovenko 5c85e037f8
Tick entry ids as only valid last_ids (#1441)
Generate tick entry ids and only register ticks as the last_id expected by the bank.  Since the bank is MT, the in-flight pipeline of transactions cannot be close to the end of the queue or there is a high possibility that a starved thread will encode an expired last_id into the ledger.  The banking_stage therefore uses a shorter age limit for encoded last_ids then the validators.

Bench client doesn't send transactions that are older then 30 seconds.
2018-10-10 17:23:06 -07:00
carllin 9931ac9780
Leader scheduler plumbing (#1440)
* Added LeaderScheduler module and tests

* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage

* Add LeaderScheduler plumbing for Tvu, window, and tests

* Fix bank and switch tests to use new LeaderScheduler

* move leader rotation check from window service to replicate stage

* Add replicate_stage leader rotation exit test

* removed leader scheduler from the window service and associated modules/tests

* Corrected is_leader calculation in repair() function in window.rs

* Integrate LeaderScheduler with write_stage for leader to validator transitions

* Integrated LeaderScheduler with BroadcastStage

* Removed gossip leader rotation from crdt

* Add multi validator, leader test

* Comments and cleanup

* Remove unneeded checks from broadcast stage

* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role

* Set new leader in validator -> validator transitions

* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail

* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops

* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
2018-10-10 16:49:41 -07:00
Michael Vines 8fe7b96629 Purge BudgetTransaction from banking_stage 2018-10-08 11:34:04 -07:00
anatoly yakovenko 1a68807ad9
Enable mt-bank (#1368)
* Enable mt-bank

* cleanup and interleaving lock tests
2018-10-04 13:15:54 -07:00
anatoly yakovenko e7de7c32db
Transactions with multiple programs. (#1381)
Transactions contain a vector of instructions that are executed atomically.
Bench shows a 2.3x speed up when using 5 instructions per tx.
2018-09-28 16:16:35 -07:00
Greg Fitzgerald 423e7ebc3f Pacify clippy 2018-09-27 16:21:12 -06:00
Pankaj Garg e10574c64d Remove recycler and it's usage
- The memory usage due to recycler was high, and incrementing with
  time.
2018-09-27 10:42:37 -06:00
Rob Walker a23c230603
fix reverse loop in write_stage, simplify banking_stage, add tooling to help find this (#1366) 2018-09-26 18:37:24 -07:00
Anatoly Yakovenko 93c4f6c9b8 Synchronize PoH, bank last_id queue and ledger entry channel.
PoH, bank's last_id queue and the Entry channel need to have a synchronized order of ids.
2018-09-26 16:19:03 -07:00
Greg Fitzgerald ca962371b8 Fix build
Two PRs crossed in flight.
2018-09-26 14:40:48 -06:00
Rob Walker e6f8922e35
fix issue #1347 (#1355) 2018-09-26 13:31:39 -07:00
Greg Fitzgerald df3b78c18c Move BudgetTransaction into its own module 2018-09-26 14:17:15 -06:00
Greg Fitzgerald c83dcea87d Move SystemTransaction into its own module 2018-09-26 14:17:15 -06:00
Greg Fitzgerald 694add9919 Move budget-specific and system-specific tx constructors into traits
These functions pull in budget-specific and system-specific
dependencies that aren't needed by the runtime.
2018-09-26 14:17:15 -06:00
Rob Walker 8a7545197f
move tick generation back to banking_stage, add unit tests (#1332)
* move tick generation back to banking_stage, add unit tests

fixes #1217

* remove channel() stuff for synchronous comm; use a mutex
2018-09-25 15:01:51 -07:00
Pankaj Garg 0f4fd8367d
Add counters for channel pressure and time spent in TPU pipeline (#1324)
* Add counters for channel pressure and time spent in TPU pipeline

* Fixed failing tests

* Fix rust format issue
2018-09-24 17:13:49 -07:00
Greg Fitzgerald 95677a81c5 Pacify clippy 2018-09-24 13:36:31 -06:00
Greg Fitzgerald ea37d29d3a Pass Bank::process_transactions() a reference to the txs instead of moving them 2018-09-24 13:36:31 -06:00
Rob Walker be31da3dce
lastidnotfound step 2: (#1300)
lastidnotfound step 2:
  * move "record stage", aka poh_service into banking stage
  * remove Entry.has_more, is incompatible with leader rotation
  * rewrite entry_next_hash in terms of Poh
  * simplify and unify transaction hashing (no embedded nulls)
  * register_last_entry from banking stage, fixes #1171 (w00t!)
  * new PoH doesn't generate empty ledger entries, so some fixes necessary in 
         multinode tests that rely on that (e.g. giving validators airdrops)
  * make window repair less patient, if we've been waiting for an answer, 
          don't be shy about most recent blobs
   * delete recorder and record stage
   * make more verbost  thin_client error reporting
   * more tracing in window (sigh)
2018-09-21 21:01:13 -07:00
Anatoly Yakovenko 431692d9d0 Use a Drop trait to keep track of lifetimes for recycled objects.
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
2018-09-19 16:59:42 -06:00
Carl 2030dfa435 Implement PR comments, tidy up 2018-09-19 10:48:05 -06:00
Carl bfe64f5f6e Added integration test for transitioning leader to validator to see that tpu pipeline can exit and restart a tvu. Fixed Tpu and broadcast stage so that exiting later stages in the pipeline also causes earlier stages to exit. 2018-09-19 10:48:05 -06:00
Michael Vines 4196cf43e8 cargo fmt 2018-09-14 16:37:49 -07:00
carllin 8706774ea7
Rewrote service trait join() method to allow thread join handles to return values other than () (#1213) 2018-09-13 14:00:17 -07:00
Rob Walker 226d3b9471
Trace recycle() calls (#968)
* trace recycle() calls fixes #810
2018-09-05 05:07:02 +09:00