* Modify db_ledger to support per_slot metadata, add signal for updates, and add chaining to slots in db_ledger
* Modify replay stage to ask db_ledger for updates based on slots
* Add repair send/receive metrics
* Add repair service, remove old repair code
* Fix tmp_copy_ledger and setup for tests to account for multiple slots and tick limits within slots
* Connect TPU's broadcast service with TVU's blob fetch stage
- This is needed since ledger is being written only in TVU now
* fix clippy warnings
* fix failing test
* fix broken tests
* fixed failing tests
* Reorg TVU code to look like TVU diagram
And move channel creation into LedgerWriteStage so that it can
be used in the same was as all the other stages.
* Delete commented out code
* Stub out architecture documentation
* Add book HTML generation and book tests to CI
* Add heading
* Better table of contents
* Reference existing documentation
Move ASCII art from code comments into rendered SVG
* Attempt to fix CI
* Add lamport docs
And truncate lines to 80 characters
* Fix links
And reference shorter, newer description of PoH.
* Replace ASCII art with SVG
* Streamline for Pillbox
* Update path before optional install
* Use $CARGO_HOME instead of $HOME
* Delete code
Attempt to describe all data structures without code.
* Boot RPU from docs, add JsonRpcService
Also, use Rust naming conventions in the block diagrams to
minimize the jump from docs to code.
* Latest code uses tick_height
* Rename bob/ folder to art/
A home for any ASCII art
* Import JSON RPC API
* More mdbook docs
* Add Ncp
* Cleanup links
* Move pipelining description into fullnode description
* Move high-level transaction docs into top-level doc
* Delete unused files
* Move finality computation into a service run from the banking stage, ComputeLeaderFinalityService
* Change last ids nth to tick height, remove separate tick height from bank
* Add Vote Contract
* Move ownership of LeaderScheduler from Fullnode to the bank
* Modified ReplicateStage to consume leader information from bank
* Restart RPC Services in Leader To Validator Transition
* Make VoteContract Context Free
* Remove voting from ClusterInfo and Tpu
* Remove dependency on ActiveValidators in LeaderScheduler
* Switch VoteContract to have two steps 1) Register 2) Vote. Change thin client to create + register a voting account on fullnode startup
* Remove check in leader_to_validator transition for unique references to bank, b/c jsonrpc service and rpcpubsub hold references through jsonhttpserver
* Move ledger write to its own stage
- Also, rename write_stage to leader_vote_stage, as write functionality
is moved to a different stage
* Address review comments
* Fix leader rotation test failure
* address review comments
* Add PoH height to process_ledger()
* Moved broadcast_stage Leader Scheduling logic to use Poh height instead of entry_height
* Moved LeaderScheduler logic to PoH in ReplicateStage
* Fix Leader scheduling tests to use PoH instead of entry height
* Change is_leader detection in repair() to use PoH instead of entry height
* Add tests to LeaderScheduler for new functionality
* fix Entry::new and genesis block PoH counts
* Moved LeaderScheduler to PoH ticks
* Cleanup to resolve PR comments
* Added LeaderScheduler module and tests
* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage
* Add LeaderScheduler plumbing for Tvu, window, and tests
* Fix bank and switch tests to use new LeaderScheduler
* move leader rotation check from window service to replicate stage
* Add replicate_stage leader rotation exit test
* removed leader scheduler from the window service and associated modules/tests
* Corrected is_leader calculation in repair() function in window.rs
* Integrate LeaderScheduler with write_stage for leader to validator transitions
* Integrated LeaderScheduler with BroadcastStage
* Removed gossip leader rotation from crdt
* Add multi validator, leader test
* Comments and cleanup
* Remove unneeded checks from broadcast stage
* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role
* Set new leader in validator -> validator transitions
* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail
* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops
* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
lastidnotfound step 2:
* move "record stage", aka poh_service into banking stage
* remove Entry.has_more, is incompatible with leader rotation
* rewrite entry_next_hash in terms of Poh
* simplify and unify transaction hashing (no embedded nulls)
* register_last_entry from banking stage, fixes#1171 (w00t!)
* new PoH doesn't generate empty ledger entries, so some fixes necessary in
multinode tests that rely on that (e.g. giving validators airdrops)
* make window repair less patient, if we've been waiting for an answer,
don't be shy about most recent blobs
* delete recorder and record stage
* make more verbost thin_client error reporting
* more tracing in window (sigh)
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
- The write stage will output vector of entries
- Broadcast stage will create blobs out of the entries
- Helps reduce MIPS requirements for write stage
* Move register_entry_id() call out of write stage
- Write stage is MIPS intensive and has become a bottleneck for
TPU pipeline
- This will reduce the MIPS requirements for the stage
* Fix rust format issues
* Reuse UDP port and open multiple sockets for transaction address
* Fixed failing crdt tests
* Add tests for reusing UDP ports
* Address review comments
* Updated bench-streamer to use multiple receive sockets
* Fix minimum number of recv sockets for bench-streamer
* Address review comments
Fixes#1132
* Moved bind_to function to nat.rs
* move gossip/NCP off assuming anything about its address
* use a single socket to send and receive gossip
* remove --addr/-a from CLIs
* rearrange networking utility code
* use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
* rename TestNode to Node
TODO:
* re-enable 127.0.0.1 as a valid address in crdt
* change repair request/response to a similar, single socket
* pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
* update contact_info with network truthiness instead of what the node
says?
Michael Vines
carllin
Sagar Dhawan
Pankaj Garg
Greg Fitzgerald
Sathish
Rob Walker
Anatoly Yakovenko
Carl
sakridge