* Add db_window module for windowing functions from RocksDb
* Replace window with db_window functions in window_service
* Fix tests
* Make note of change in db_window
* Create RocksDb ledger in bin/fullnode
* Make db_ledger functions generic
* Add db_ledger to bin/replicator
* Cluster Replicated Data Store
Separate the data storage and merge strategy from the network IO boundary.
Implement an eager push overlay for transporting recent messages.
Simulation shows fast convergence with 20k nodes.
* Add PoH height to process_ledger()
* Moved broadcast_stage Leader Scheduling logic to use Poh height instead of entry_height
* Moved LeaderScheduler logic to PoH in ReplicateStage
* Fix Leader scheduling tests to use PoH instead of entry height
* Change is_leader detection in repair() to use PoH instead of entry height
* Add tests to LeaderScheduler for new functionality
* fix Entry::new and genesis block PoH counts
* Moved LeaderScheduler to PoH ticks
* Cleanup to resolve PR comments
* Added LeaderScheduler module and tests
* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage
* Add LeaderScheduler plumbing for Tvu, window, and tests
* Fix bank and switch tests to use new LeaderScheduler
* move leader rotation check from window service to replicate stage
* Add replicate_stage leader rotation exit test
* removed leader scheduler from the window service and associated modules/tests
* Corrected is_leader calculation in repair() function in window.rs
* Integrate LeaderScheduler with write_stage for leader to validator transitions
* Integrated LeaderScheduler with BroadcastStage
* Removed gossip leader rotation from crdt
* Add multi validator, leader test
* Comments and cleanup
* Remove unneeded checks from broadcast stage
* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role
* Set new leader in validator -> validator transitions
* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail
* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops
* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
* Add check in window_service to exit in checks for leader rotation, and propagate that service exit up to fullnode
* Added logic to shutdown Tvu once ReplicateStage finishes
* Added test for successfully shutting down validator and starting up leader
* Add test for leader validator interaction
* fix streamer to check for exit signal before checking socket again to prevent busy leaders from never returning
* PR comments - Rewrite make_consecutive_blobs() function, revert genesis function change
lastidnotfound step 2:
* move "record stage", aka poh_service into banking stage
* remove Entry.has_more, is incompatible with leader rotation
* rewrite entry_next_hash in terms of Poh
* simplify and unify transaction hashing (no embedded nulls)
* register_last_entry from banking stage, fixes#1171 (w00t!)
* new PoH doesn't generate empty ledger entries, so some fixes necessary in
multinode tests that rely on that (e.g. giving validators airdrops)
* make window repair less patient, if we've been waiting for an answer,
don't be shy about most recent blobs
* delete recorder and record stage
* make more verbost thin_client error reporting
* more tracing in window (sigh)
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
* Use multiple sockets for receiving blobs on validators
- The blobs that are broadcasted by leader or retransmitted by peer
validators are received on replicate_port
- Using reuse_addr/reuse_port, multiple sockets can be opened for
the same port
- This allows the kernel to queue data to user space app on multiple
socket queues, preventing over-running one queue
- This helps with reducing packets dropped due to queue over-runs
Fixes#1224
* Fixed failing tests
Greg Fitzgerald
carllin
Rob Walker
Michael Vines
anatoly yakovenko
Pankaj Garg
jackcmay
Stephen Akridge
Anatoly Yakovenko