Commit Graph

325 Commits

Author SHA1 Message Date
Michael Vines a17903a89f Tweak process_blocktree() signature to return a BankForks 2019-02-19 20:01:22 -08:00
Greg Fitzgerald dde886f058 Move Bank to its own crate
Also:
* counters.rs to solana_metrics
* genesis_block.rs to solana_sdk
2019-02-19 07:17:04 -07:00
Greg Fitzgerald ad9cd23202 Notify subscribers from ReplayStage 2019-02-18 20:04:30 -07:00
Greg Fitzgerald 5916177dc8 Drop RpcPubSubService's dependency on the Bank
Pass in RpcSubscriptions instead, which let's you choose a
bank fork when it's time to send notifications.
2019-02-18 20:04:30 -07:00
Greg Fitzgerald e0b8f4202d Use slot height for BankForks ids 2019-02-18 17:27:20 -07:00
Greg Fitzgerald e4119268ca Delete expensive integration test in unit-test suite 2019-02-18 09:27:09 -07:00
Carl f33c6eb95f delete leader rotation signal from banking stage 2019-02-17 19:30:45 -07:00
Anatoly Yakovenko 5f55a9be84 fmt 2019-02-17 15:43:13 -08:00
Anatoly Yakovenko 300e3d151d remove the signal sender since its superfelous to a recv error 2019-02-17 15:43:13 -08:00
Anatoly Yakovenko 54dfe708c1 use ref for new_from_parent; test that transactions don't leak to parent 2019-02-17 15:02:08 -07:00
Anatoly Yakovenko 8166925f04 copy a new bank 2019-02-17 15:02:08 -07:00
Anatoly Yakovenko 64f1d93cc3 Use the accounts list from parents up to finalized bank for Account::load apis.
Borrow checker

query the previous parents accounts

cleanup!

s/tree/parents

Tests!  Last_ids need to be inherited as well otherwise nothing works.

new_from_parent
2019-02-17 15:02:08 -07:00
Greg Fitzgerald c646845cd3 Move RpcService into its own module 2019-02-17 12:20:05 -07:00
Greg Fitzgerald eb483bc053 Move RpcPubSubService into its own module 2019-02-17 12:20:05 -07:00
Greg Fitzgerald 9fa8105ae8 Add a way to make a DAG of checkpointed Banks 2019-02-16 21:49:06 -07:00
Greg Fitzgerald ac35fe9ed1 Flip the dependency; Create bank before scheduler 2019-02-16 14:16:48 -07:00
Greg Fitzgerald 3d70afc578 Boot leader scheduler from the bank
Functional change: the leader scheduler is no longer implicitly
updated by PohRecorder via register_tick(). That's intended to
be a "feature" (crossing fingers).
2019-02-16 14:16:48 -07:00
Greg Fitzgerald 97c93629a5 Don't use the Bank's LeaderScheduler 2019-02-16 10:55:58 -07:00
Greg Fitzgerald 643384e1ec Add LeaderScheduler constructor to Bank
By passing a config and not a Arc'ed LeaderScheduler, callers
need to use `Bank::leader_scheduler` to access the scheduler.
By using the new constructor, there should be no incentive to
reach into the bank for that object.
2019-02-16 10:55:58 -07:00
Greg Fitzgerald 4467d5eb4c Extract process_ledger from Bank
Fullnode was the only real consumer of process_ledger and it was
only there to process a Blocktree. Blocktree is a tree, and a
ledger is a sequence, so something's clearly not right here.
Drop all other dependencies on process_ledger (only one test) so
that it can be fixed up in isolation.
2019-02-16 08:07:26 -07:00
Michael Vines 626a381ddc Collect and re-forward packets received while TpuForwarder is shutting down 2019-02-14 18:40:31 -08:00
Michael Vines 8790a92f07 Adjust create_counter to avoid imposing an AtomicUsize import on users 2019-02-13 20:24:04 -08:00
Michael Vines 0f8ff07b51 tpu now hangs on to its cluster_info 2019-02-13 16:16:18 -08:00
Michael Vines 863956d09c Add multinode test for two nodes rotating at 1 tick per slot 2019-02-12 21:17:06 -08:00
Michael Vines 7118178e2c Correctly compute max_tick_height when starting up a node 2019-02-12 21:17:06 -08:00
Michael Vines 1eabe66c85 setup_leader_validator: remove unnecessary ticks_per_slot parameter 2019-02-12 21:17:06 -08:00
Michael Vines e341b33f21 Remove ticks_per_slot from Blocktree::write_entries(), it already knows 2019-02-12 15:52:27 -08:00
Michael Vines 6abdd6401d clippy: passing BlocktreeConfig by ref is ok 2019-02-12 15:52:27 -08:00
Michael Vines 6632c7026d Pass a BlocktreeConfig into all ledger helper functions 2019-02-12 15:52:27 -08:00
Michael Vines c474cf1eef Pass BlocktreeConfig around as a reference 2019-02-12 15:52:27 -08:00
carllin be71f49d80
Change write_entries() and create_tmp_ledger() to take ticks_per_slot (#2736)
* Change write_entries() and create_tmp_ledger() to take ticks_per_slot

* PR nits
2019-02-12 13:14:33 -08:00
Sagar Dhawan 8b39eb5e4e
Replace Blob Ids with Forward property (#2734)
* Replace Blob Id with Blob forwarding

* Update simulation to properly propagate blobs
2019-02-12 10:56:48 -08:00
Pankaj Garg 0002b5dd02 Write to ledger in BroadcastService
- Also disconnect the channel between TPU and TVU
2019-02-12 08:41:02 -08:00
Michael Vines 709598541f
Remove stale TODO comment 2019-02-11 22:13:07 -08:00
Michael Vines b595bf8f44 Set blob_index correctly when tick_height is at the last tick of a slot 2019-02-11 19:50:33 -08:00
Michael Vines f6979a090e leader_scheduler: reduce the amount of special case handling for tick_height 0 2019-02-11 19:05:14 -08:00
Michael Vines aac1a58651 Try harder to keep LeaderSchedulerConfig and BlocktreeConfig in sync 2019-02-11 13:10:12 -08:00
Michael Vines 095afdfe47 Merge leader_to_validator/validator_to_leader 2019-02-11 08:57:44 -08:00
Michael Vines 4ae1783b97 Remove code duplication between leader_to_validator/validator_to_leader 2019-02-10 17:53:42 -08:00
Michael Vines 7dec40ff05 slot 0 now contains the same number of ticks as all subsequent slots 2019-02-10 16:34:10 -08:00
carllin 4b38ecd916
fix tpu tvu bank race (#2707)
* Fix tpu tvu bank race

* Test highlighting race between tvu and tpu banks during leader to leader transitions
2019-02-10 16:28:52 -08:00
Michael Vines da5b777ee7 Purge Default::default() 2019-02-09 10:12:32 -08:00
Michael Vines 56734dca3b Align Tpu::new() and Tpu::switch_to_leader() arguments 2019-02-07 21:33:49 -08:00
carllin 6e7c5f205b
Rename db_ledger to blocktree (#2698) 2019-02-07 20:52:39 -08:00
carllin fd7db7a954
Support multiple forks in the ledger (#2277)
* Modify db_ledger to support per_slot metadata, add signal for updates, and add chaining to slots in db_ledger

* Modify replay stage to ask db_ledger for updates based on slots

* Add repair send/receive metrics

* Add repair service, remove old repair code

* Fix tmp_copy_ledger and setup for tests to account for multiple slots and tick limits within slots
2019-02-07 15:10:54 -08:00
Michael Vines 1f0b3f954a leader_scheduler: replace older terminology with ticks, slots and epochs 2019-02-07 10:42:57 -08:00
Michael Vines 95ac6305bc Remove unnecessary dependencies on fullnode mod 2019-02-06 21:31:48 -08:00
Michael Vines ab4828aae7 Replace role_notifiers tuple with two explicit fields 2019-02-06 21:31:48 -08:00
Michael Vines c506423e70 Remove superfluous imports 2019-02-06 21:31:48 -08:00
Michael Vines f0843fc5f1 NodeServices: de-pub, remove dead code 2019-02-06 21:31:48 -08:00
Michael Vines c87e035302 Remove multinode test dependency on Fullnode internals 2019-02-06 20:38:22 -08:00
Michael Vines abb9a72b27 Reduce Fullnode public API surface 2019-02-06 20:04:51 -08:00
Sagar Dhawan acc6bf1564 Don't over complicate the solution 2019-02-06 19:55:12 -08:00
Sagar Dhawan db688207a5 Add abort signals to tvu/tpu receivers 2019-02-06 19:55:12 -08:00
Sagar Dhawan 9681c4d468 Fix resource hogging when waiting for role transition 2019-02-06 19:55:12 -08:00
Michael Vines d9e2b94d7a bank::new_with_leader_scheduler_config() - remove Option<> 2019-02-06 19:47:09 -08:00
Michael Vines f789038baa Consolidate fullnode ledger helpers 2019-02-06 19:47:09 -08:00
Michael Vines 2e23b03f94 Remove dead code 2019-02-06 19:47:09 -08:00
Michael Vines fa553029d5 Temporarily disable test_validator_to_leader_transition 2019-02-06 14:23:10 -08:00
Michael Vines c5a74ada05 leader_scheduler: remove bootstrap_height 2019-02-06 14:23:10 -08:00
Michael Vines f90d96367d Add Fullnode::run() to optionally manage node role transitions automatically 2019-02-06 14:23:10 -08:00
carllin 3feda8a315
ReplayStage asking ledger for updates (#2597)
* Modify replay stage to ask db_ledger for updates instead of reading from upstream channel

* Add signal for db_ledger to update listeners about updates

* fix flaky test
2019-02-04 15:33:43 -08:00
Michael Vines 43cce3a8fc speling 2019-02-01 07:11:17 -08:00
Greg Fitzgerald 82a2080e45 Rename VoteSignerProxy to VotingKeypair
Works just like a normal Keypair, but will only sign voting
transactions.
2019-02-01 07:11:17 -07:00
Michael Vines d87c2eb903 Fullnode::new() - provide FullnodeConfig as a ref 2019-01-31 21:12:36 -08:00
carllin 37003da854
Fix potential of checking tvu bank for truth when its behind (#2614)
* Fix race between tpu and tvu, where tvu bank is not caught up to tpu bank

* Add test

* Cleanup Fullnode tests
2019-01-31 19:21:02 -08:00
Rob Walker 8ba1d5f426
treat genesis special (#2615)
* treat genesis special

* fix poh_recorder to understand new world order
2019-01-31 13:53:08 -08:00
Pankaj Garg 32162ef0f1
Connect TPU's broadcast service with TVU's blob fetch stage (#2587)
* Connect TPU's broadcast service with TVU's blob fetch stage

- This is needed since ledger is being written only in TVU now

* fix clippy warnings

* fix failing test

* fix broken tests

* fixed failing tests
2019-01-31 13:43:22 -08:00
Greg Fitzgerald 609e915169 Fix clippy warning
Always pass Arcs by reference. Then you'll only need to clone()
to cross thread boundaries.
2019-01-30 21:59:05 -07:00
Greg Fitzgerald a74b24fdf0 Only store the fullnode's pubkey
Only vote_signer is used for signing
2019-01-30 21:59:05 -07:00
Greg Fitzgerald e25992a011 Always give Fullnode a vote signer
This will allow us to use the the signer's pubkey as the node id.

Disable voting with a configuration option.
2019-01-30 21:59:05 -07:00
Michael Vines c01290438f Move virtual genesis tick into the ledger proper as entry 0 2019-01-30 14:02:07 -08:00
Michael Vines 9e9c82869a create_tmp_sample_ledger() need not return the genesis block 2019-01-30 14:02:07 -08:00
Michael Vines 494b143453 Delete create_tmp_genesis 2019-01-30 14:02:07 -08:00
Michael Vines 8cc1cde0fe create_tmp_sample_ledger() now returns entry_height and last_id 2019-01-30 14:02:07 -08:00
Michael Vines 16e705dc75 Boil away unneeded Fullnode::new_* functions 2019-01-29 20:10:10 -08:00
Rob Walker 1f9ab7f58f copy bank for TPU 2019-01-29 12:11:48 -08:00
Anatoly Yakovenko 3e1a926aa6 wip 2019-01-29 12:11:48 -08:00
Michael Vines df136578d4 Remove unnecessary FullnodeConfig::rpc_port option 2019-01-29 10:22:39 -08:00
Michael Vines ae7f169027 Add FullnodeConfig struct to Fullnode::new* functions
This avoids having to touch *every* Fullnode::new* call site when
a new fullnode option is added
2019-01-29 09:42:48 -08:00
Tyera Eulberg 6da7a784f2
Stream entries (#2582)
* Add entry streaming option

* Fix tests

* Remove obsolete comment

* Move entry stream functionality to struct w/ trait in order to test without i/o
2019-01-29 00:21:27 -08:00
Michael Vines 12cddf725e Harmonize Fullnode::new* function arguments 2019-01-28 22:37:56 -08:00
Stephen Akridge 53afa64634 Remove storage_state from the bank
Construct in TVU and pass to RPC and StorageStage instead.
2019-01-28 15:41:41 -08:00
Michael Vines 29ef9370a6 Remove LeaderSchedulerConfig options 2019-01-28 13:51:01 -08:00
Greg Fitzgerald 1bae87d4b3 Add unit-test-friendly VoteSignerProxy constructor 2019-01-26 14:56:49 -07:00
Sagar Dhawan d65e7b9fcc
Speedup rotation (#2468)
Speedup leader to validator transitions
2019-01-26 13:58:08 +05:30
Michael Vines 4bb6549895 Genesis block is now a json file 2019-01-25 09:05:15 -08:00
Rob Walker e9e01557b7 fix leaked threads from unclosed fullnode 2019-01-25 03:02:49 -08:00
Tyera Eulberg b7be5b9a7a Add no-signer argument 2019-01-24 10:47:37 -08:00
Michael Vines 6e8b69fc88 Cleanup leader_addr, it's really entrypoint_addr 2019-01-21 13:06:30 -08:00
Pankaj Garg 6611188edf
Move subscriptions to rpc_pubsub (#2490)
* Move subscriptions to rpc_pubsub

- this helps avoid recreating pubsub_service on node's role change

* fixed tests and addressed review comments

* fix clippy errors

* address review comments
2019-01-21 09:59:09 -08:00
Michael Vines 132d59ca6a new_bank_from_db_ledger need not be public 2019-01-21 08:11:13 -08:00
Michael Vines 5b73a8eceb Rework fullnode boot sequence 2019-01-19 21:35:46 -08:00
Stephen Akridge 1fd7bd7ede Storage fixes
* replicators generate their sample values
* fixes to replicator block height logic
2019-01-18 13:05:35 -08:00
Michael Vines f8bd19f5db Log the time it look to process the ledger for easier log inspection 2019-01-16 10:45:47 -08:00
Michael Vines 4369c1a113 RPC port is no longer reset on leader-to-validator transition 2019-01-15 16:06:56 -08:00
Rob Walker 022a97da99
revert revert of kill window (#2427)
* remove window code from most places
* window used only for testing
* remove unecessary clippy directives
2019-01-15 10:51:53 -08:00
Rob Walker 447243f994
Revert "remove window code from most places" (#2417)
* Revert "Fix link to book in Local Testnet section (#2416)"

This reverts commit 710c0c9980.

* Revert "Add current leader information to dashboard (#2413)"

This reverts commit f0300c1711.

* Revert "remove window code from most places (#2389)"

This reverts commit e3c0bd5a3f.
2019-01-14 15:11:18 -08:00
Rob Walker e3c0bd5a3f
remove window code from most places (#2389)
* remove window code from most places
* window used only for testing
* remove unecessary clippy directives
2019-01-14 12:11:55 -08:00
Pankaj Garg d7594b19fc
Implemented a trait for vote signer service (#2386)
* Implemented a trait for vote signer service

* removes need for RPC in unit tests for vote signing

* fix build errors

* address some review comments
2019-01-11 12:58:31 -08:00