Commit Graph

185 Commits

Author SHA1 Message Date
Pankaj Garg 6611188edf
Move subscriptions to rpc_pubsub (#2490)
* Move subscriptions to rpc_pubsub

- this helps avoid recreating pubsub_service on node's role change

* fixed tests and addressed review comments

* fix clippy errors

* address review comments
2019-01-21 09:59:09 -08:00
Michael Vines 132d59ca6a new_bank_from_db_ledger need not be public 2019-01-21 08:11:13 -08:00
Michael Vines 5b73a8eceb Rework fullnode boot sequence 2019-01-19 21:35:46 -08:00
Stephen Akridge 1fd7bd7ede Storage fixes
* replicators generate their sample values
* fixes to replicator block height logic
2019-01-18 13:05:35 -08:00
Michael Vines f8bd19f5db Log the time it look to process the ledger for easier log inspection 2019-01-16 10:45:47 -08:00
Michael Vines 4369c1a113 RPC port is no longer reset on leader-to-validator transition 2019-01-15 16:06:56 -08:00
Rob Walker 022a97da99
revert revert of kill window (#2427)
* remove window code from most places
* window used only for testing
* remove unecessary clippy directives
2019-01-15 10:51:53 -08:00
Rob Walker 447243f994
Revert "remove window code from most places" (#2417)
* Revert "Fix link to book in Local Testnet section (#2416)"

This reverts commit 710c0c9980.

* Revert "Add current leader information to dashboard (#2413)"

This reverts commit f0300c1711.

* Revert "remove window code from most places (#2389)"

This reverts commit e3c0bd5a3f.
2019-01-14 15:11:18 -08:00
Rob Walker e3c0bd5a3f
remove window code from most places (#2389)
* remove window code from most places
* window used only for testing
* remove unecessary clippy directives
2019-01-14 12:11:55 -08:00
Pankaj Garg d7594b19fc
Implemented a trait for vote signer service (#2386)
* Implemented a trait for vote signer service

* removes need for RPC in unit tests for vote signing

* fix build errors

* address some review comments
2019-01-11 12:58:31 -08:00
Pankaj Garg 2dbe8fc1a9
Refactor vote signer code (#2368)
* Refactor vote signer code

* fixed test compilation errors

* address clippy errors

* fix missing macro_use

* move macro use

* review comments
2019-01-10 09:21:38 -08:00
Stephen Akridge 491bca5e4b Remove ledger.rs
Split into entry.rs for entry-constructing functions and EntrySlice
trait and db_ledger.rs for ledger helper test functions.
2019-01-09 15:15:47 -08:00
Greg Fitzgerald f53810fcd2 Remove unused exit variable
The exit variable was only used by a test.
2019-01-08 20:22:31 -08:00
Michael Vines a8b9899dee Add retry, restore ignored tests 2019-01-07 19:30:08 -08:00
Stephen Akridge d2cb4e003c Re-enable the --lib tests 2019-01-07 15:28:20 -08:00
Pankaj Garg 91bd38504e
Use vote signer service in fullnode (#2009)
* Use vote signer service in fullnode

* Use native types for signature and pubkey, and address other review comments

* Start local vote signer if a remote service address is not provided

* Rebased to master

* Fixes after rebase
2019-01-05 12:57:52 -08:00
Michael Vines b7dc9dbc76 RPC API now assumes a drone running on the bootstrap leader 2019-01-04 18:45:55 -08:00
Michael Vines 1f6346d880 De-dup ledgers - db_ledger is now the only ledger written to disk 2019-01-04 16:37:00 -08:00
Sagar Dhawan 0bea870b22
Dynamic N layer 'avalanche' broadcast and retransmit (#2058)
* Dynamic N layer avalanche broadcast and retransmit
2019-01-02 14:16:15 +05:30
Michael Vines 37d7ad819b Purge DB::destroy() usage 2018-12-20 10:38:03 -08:00
Pankaj Garg 974249f2a5
Parallelize entry processing in replay stage in validators (#2212)
* Parallelize entry processing in replay stage in validators

- single threaded entry processing is not utlizing CPU cores to the fullest

* fix tests and address review comments
2018-12-18 16:06:05 -08:00
Rob Walker a65022aed7
DbLedger doesn't need to be mut, doesn't need an RwLock (#2215)
* DbLedger doesn't need to be mut, doesn't need an RwLock

* fix erasure cases
2018-12-18 15:18:57 -08:00
Michael Vines 07202205c4 Revert "ignore unstable tests"
This reverts commit bd7ef5d445071329a3b49b1f8be71b602226bbec.
2018-12-17 10:47:32 -08:00
anatoly yakovenko 4be6d01dfb
Move last ids (#2187)
* Break out last_ids into its own module
* Boot SignatureNotFound from BankError
* No longer return BankError from LastIds methods
* No longer piggypack on BankError for a LastIds signature status
* Drop all dependencies on the bank
* SignatureStatus -> Status and LastIds -> StatusDeque
* Unstable tests, issue 2193
2018-12-17 07:55:56 -08:00
Michael Vines 5bf9a20d42 fullnode-config no longer depends on src/ 2018-12-14 20:13:34 -08:00
Stephen Akridge 7cdbbfa88e Storage stage updates
* Remove logging init from storage program: saw a crash in a test
  indicating the logger being init'ed twice.
* Add entry_height mining proof to indicate which segment the result is
  for
* Add an interface to get storage miner pubkeys for a given entry_height
* Add an interface to get the current storage mining entry_height
* Set the tvu socket to 0.0.0.0:0 in replicator to stop getting entries
  after the desired ledger segment is downloaded.
* Use signature of PoH height to determine which block to download for
  replicator.
2018-12-13 11:30:12 -08:00
Rob Walker 4f48f1a850
add db_ledger genesis, rework to_blob(), to_blobs() (#2135) 2018-12-12 20:42:12 -08:00
Carl 245362db96 Make a dummy version of serving repairs from db_ledger 2018-12-12 19:12:51 -08:00
Mark b1b190b80d Fix too many args in Tvu::new (#2114)
* Reduce args in Tvu::new under to 8

Now pass in sockets through a the crate::tvu::Sockets struct

Move ClusterInfo.keypair to pub(crate) in order to remove redundant
signing keypair parameter

* remove commented code
2018-12-12 18:57:48 -08:00
carllin ae903f190e
Broadcast for slots (#2081)
* Insert blobs into db_ledger in broadcast stage to support leader to validator transitions

* Add transmitting real slots to broadcast stage

* Handle real slots instead of default slots in window

* Switch to dummy repair on slots and modify erasure to support leader rotation

* Shorten length of holding locks

* Remove logger from replicator test
2018-12-12 15:58:29 -08:00
Pankaj Garg 3413ecc2bd
Change query used to find list of nodes in the network (#2124)
* Change query used to find list of nodes in the network

* include "All" option for host selection
2018-12-12 12:38:00 -08:00
Greg Fitzgerald 5e703dc70a Free up the term 'replicate' for exclusive use in replicator
Also, align Sockets field names with ContactInfo.
2018-12-10 15:26:43 -07:00
Greg Fitzgerald ec5a8141eb cargo fix --edition 2018-12-08 23:19:55 -07:00
Sathish 2de45a4da5
Update airdrop tokens to 3 for fullnode (#2051)
Filter out leader while computing the super majority stake
2018-12-08 16:54:42 -08:00
Greg Fitzgerald 0a83b17cdd
Upgrade to Rust 1.31.0 (#2052)
* Upgrade to Rust 1.31.0
* Upgrade nightly
* Fix all clippy warnings
* Revert relaxed version check and update
2018-12-07 20:01:28 -07:00
Greg Fitzgerald 97b1156a7a Rename Ncp to GossipService
And BroadcastStage to BroadcastService since it's not included in the
TPU pipeline.
2018-12-06 15:48:19 -07:00
Michael Vines c99f93e40a Remove signature.rs indirection 2018-12-03 13:50:00 -08:00
carllin 4ae58cc854
Change range of leader scheduler to match current broadcasts (#1920) 2018-12-03 00:10:43 -08:00
Sagar Dhawan 34c3a0cc1f
Add signature verification to gossip (#1937) 2018-12-01 12:00:30 -08:00
carllin 57a384d6a0
Rocks db window service (#1888)
* Add db_window module for windowing functions from RocksDb

* Replace window with db_window functions in window_service

* Fix tests

* Make note of change in db_window

* Create RocksDb ledger in bin/fullnode

* Make db_ledger functions generic

* Add db_ledger to bin/replicator
2018-11-24 19:32:33 -08:00
Rob Walker 6b910d1bd4 add tpu_forwarding, simplify ClusterInfo::new() from Result<Self> to Self 2018-11-19 20:45:49 -08:00
Rob Walker 3d113611cc
remove Result<> return from ClusterInfo::new() (#1869)
strip Result<> for ClusterInfo::new()
2018-11-19 11:25:14 -08:00
Michael Vines d96a6b42a5 Move drone into its own crate 2018-11-16 20:42:21 -08:00
Michael Vines 6ac5700f2e Move metrics into its own crate 2018-11-16 15:10:07 -08:00
Michael Vines a0dd8617be Remove airdrop from fullnode 2018-11-16 13:25:55 -08:00
anatoly yakovenko a41254e18c
Add scalable gossip library (#1546)
* Cluster Replicated Data Store

Separate the data storage and merge strategy from the network IO boundary.
Implement an eager push overlay for transporting recent messages.

Simulation shows fast convergence with 20k nodes.
2018-11-15 13:23:26 -08:00
Sagar Dhawan 729d28d910 Add poh verification before processing entries
- Replicate stage now verifies entries delivered
  by the window
- Minor refactor of entries_from_blobs
2018-11-13 14:17:00 -08:00
carllin 6335be803c
Broadcast last tick before leader rotation (#1766)
* Broadcast last tick before leader rotation to everybody on network

* Add test

* Refactor broadcast
2018-11-13 02:21:37 -08:00
Sagar Dhawan a77b1ff767
Revert "Migrate from ring to ed25519-dalek" (#1798)
* Revert "Migrate from ring to ed25519-dalek"

This reverts commit 7c610b216b.

* Fix test failures with revert
2018-11-12 22:34:43 -08:00
Greg Fitzgerald fea86b2955 No longer serialize as JSON-encoded pkcs8
That's supposed to be an ASCII format, but we're not making use
of it. We can switch back to that some day, but if we do, it shouldn't
be JSON-encoded.
2018-11-11 09:22:52 -07:00