`Target` now only has a `Nodes` and an `AllExcept` variant, to specify
a message's target via a whitelist or blacklist. This avoids cloning
the message content and simplifies the code in several places.
* Added extra message types
* Add send functions for new message types
* Store original value message received from proposer
* Modify handle_value for optimized broadcast
* Modify handle_echo for optimized broadcast
* Add handle_echo_hash function for optimized broadcast
* Add handle_can_decode function for optimized broadcast
* Fixes handle_ready and send_echo functions:
1) Modify handle_ready function for optimized broadcast
2) Modify send_echo function to send `Echo` messages to different subset of nodes from
handle_value and handle_ready functions
* Remove value_message and fix typos
* Add functions for filtering all_ids
* Separate send_echo to send_echo_left and send_echo_remaining
* Rename pessimism_factor to fault_estimate
* Remove redundant bools from Broadcast struct
* Fix multiple counting of nodes who sent both `Echo` and `EchoHash` by changing
`echos` map structure
* Allow conflicting `CanDecode`s from same node
* Fix left and right iterators for `Echo` and `EchoHash` messages
* Fixes bugs in left and right iterators and adds additional checks in handle
functions
* Change can_decodes to BTreeMap<Digest, BTreeSet<N>> and fix send_can_decode
* Minor fixes
* Modify send_echo_remaining to take a hash parameter
* Fix bug in left and right iterators.
* Excluding proposer in iterator led to infinite loop when our_id == proposer_id
* Fix bug in handle_echo and compute_output
* send_can_decode call in handle_echo returned early
* compute_output needed `N - f` full `Echo`s to decode
* Refactor `echos` map to take an EchoContent Enum for `Echo` and `EchoHash` messages
* Run rustfmt
* Refactor to avoid index access and multiple map lookups
* Fix comments and minor refactorings.
* Add an`AllExcept(BTreeSet<N>)` type to `Target` enum to enable sending messages
to non-validators from Broadcast.
* Use `Target::AllExcept` in Broadcast to send `Echo` messages to all non-validator nodes.
* Add `AllExcept(_)` match arms for `Target` match expressions.
* Rename `AllExcept` parameter from `known` to `exclude`.
* Modify send_can_decode to send to all nodes who haven't sent an `Echo`.
* Update docs for broadcast
* Improve formatting and add comments for broadcast docs.
* Fix formatting.
* Allow for sending multiple `CanDecode` messages with different hashes.
* Fix comments.
* Fix bug in sending `Echo`s when node has not received `CanDecode`.
* Extract test framework into hbbft_testing crate.
Also update Rust and some dependencies.
* Rename DHB tests.
They are only called `net_dynamic_hb` because we used to have two test
frameworks and two DHB test modules.
Binary agreement erroneously kept handling queued incoming messages for
an epoch `n`, even if during the handling, the epoch was incremented to
`n + 1`. This caused `DuplicateAux` faults in the tests, but it could
potentially break consensus.
* Ported threshold_sign test to the new framework
* Ported the first three broadcast tests to the new framework
* Extracted messages storting and random swapping to reusable functions
Used to compose ProposeAdversary's behavior without having to duplicate code
* Implemented ProposeAdversary for the new integration testing framework
Added "id()" function to the "NodeMutHandle", required for sending messages to all nodes
ProposeAdversary needs access to all faulty node's netinfo. We follow the example of the binary_agreement_mitm integration test of using an reference counted Mutex to make netinfo available on both Consensus Protocol construction and in the Adversary implementation.
Unlike binary_agreement_mitm every faulty node needs to use its own netinfo for the broadcast algorithm, so we store all nodeinfo structures in a Map instead of just the nodeinfo of the first node.
Ideallly the new network simulation library should provide netinfo similar to the old library to avoid these hideous workarounds.
* Migrated test_broadcast_random_delivery_adv_propose to the new network simulator
Refactored the implementation of ProposeAdversary to closely resemble the behavior in the old network simulator library.
Implemented a pick_random_node function to sort messages for a random node id. Switched from using "inject_message" to joining messages generated by adversaries' temporary Broadcast Consensus Protocols with the Step generated by regular operation.
* Ported RandomAdversary to the new network simulator library
Ported all broadcast integration tests and replaced the old tests with the new.
* Eliminated the old broadcast integration test, replaced with the new
* Ported subset test to the new framework
Adjusted message queue size as suggested by Andreas
* Ported the first three honey_badger tests to the new framework
* Re-implemented FaultyShareAdversary for the new framework
Eliminated the old honey_badger integration tests, replaced with implementations using the new net simulator framework
* Fixed issues reported by clippy
* Fixed issues reported on code review
* Fixed issues reported by clippy
* Implemented a broadcast test dropping messages similar to the tests written in the old framework
* Picking the proposer id at random, verifying all possible output cases
If the proposer is faulty the message queue may starve, but the outputs of all correct nodes need to be empty, if the broadcast protocol produces output nonetheless all correct nodes need to have the same output.
If the proposer was correct all correct nodes need to output its proposed value.
* Eliminated duplicated semicolon
* Consistently using TestRng and proptest in all newly ported tests
* Increased the drop_and_re_add test message limit to 20k per node
* Removed unnecessary closure
* Increased the tolerance for deviations from the expected value range to eliminate random test failures
`SyncKeyGen` should tolerate duplicate `Part` messages as long as they
are identical.
The `drop_and_re_add` test had an arithmetic overflow, because it tried
to remove more faulty nodes than nodes in total.
* Choose pivot node at random
* Choose random number of nodes for removing in net_dynamic_hb test
* Docs and code small fixes
* clippy fix
* Cargo fmt for stable toolchain and add rust-toolchain file as well
* Remove rust-toolchain file
* Fix grammar and improve selecting nodes for removing
* Simplify selecting nodes for remove
* Fix tests
* Remove net_dynamic_hb.proptest-regressions file
Implementing an epoch setter for the `DynamicHoneyBadgerBuilder` enables the creation of a `DynamicHoneyBadger` that will join the consensus at a given epoch.
* started waiting for a full epoch after node removal in net_dynamic_hb
* clarified the use of the stored join plan
* go back to rejoining the node in the same epoch it was removed
* cleanup of debug prints
* clippy lints and more cleanup
* cleaned up unused methods
* review comments; cleaned up net_dynamic_hb
* relaxed the condition on the readd input epoch
* updated the fault error in tests
* impl. old validator removal from sender queue peer list
* provided current validators for sender queue peer removal
* renamed validators as nodes in the sender queue
* Revert "renamed validators as nodes in the sender queue"
This reverts commit 78e1e1569d5f624c469bf752a5bf874b434a61d2.
* cleaned up the SQ builder and moved removal of old validators to triggers
* computing participant transitions from batches in the sender queue
* added a missing comment
* removing old validators as soon as all messages are delivered up to the last epoch
* review comments
* rejoined Node 0 in the old DHB test
* DHB test uses the first step of the DHB algorithm on the restarted node
* changed test batch verification to account for node 0 removal
* updated net_dynamic_hb test to cope with the removal of node 0
* relaxed verification of batches to only check inclusion of node 0 transactions
* corrected test state transitions in DHB and QHB tests
* added a builder function for a joining QHB
* rejoin the pivot node instead of node 0
* changed VirtualNet::verify_batches to take a full node as an argument
* corrected a variable name
* correction: use the pivot node ID instead of indices
* corrected the pivot node ID
* simplified a find
* simplified a conditional statement
* corrected the inference of expected output in verify_batches
* WIP on DHB and QHB tests; VirtualNet::verify_batches made more general
* readded node 0 in the DHB test when InProgress change is output
* allowed node 0 to miss a few batches while it is removed in the QHB test
* edition and rebase fixes
* refactored the use of process_step
* added VirtualNet functionality of node insertion and removal
* restarting the pivot node after all validators add it as peer
* clippy lints in net_dynamic_hb
* added observer in the QHB test; removed the DHB TestNetwork test
* fixed rng usage in the QHB test
* check output length when verifying batches; comment correction
* Use `OsRng` in place of `thread_rng`.
This changes the defaults of any builder by instantiating an `OsRng` instead of
a `thread_rng`, the former being much more secure than the latter.
Additionally, all the unit tests that still instantiate RNGs manually used `OsRng`s
as well; while there is no actual need for this level of security in tests, the performance overhead is very small and random number generation complexity has such a small impact on these tests that the convenience of being able to ban `thread_rng` from the codebase altogether, setting a good example and avoid issues when refactoring later greatly outweigh the negatives.
* Instead of storing random number generators in the various consensus algorithm instances, pass them in from the outside whenever they are needed.
This changes a large amount of interfaces (and in this commit is only partially done, since `DistAlgorithm` needs to be fundamentally altered as well.
It also obsoletes parts of the `util` module.
* Added an `R: Rng` type parameter to both methods of `DistAlgorithm`, forcing callers to pass in their own Rngs.
* Fixed documentation grammar and spelling in some of the altered interfaces due to RNG refactoring.
* Move `rng` argument to the end of the argument for most functions.
Also includes a reformatting due to Rust 1.30.
* Updated tests, accomodate `rng`-API changes.
* Fixed remaining compilation issues with new RNG code.
* Fix illegal `self` import outside curly braces.
* Cleaned up comments and fixed broken definition of `broadcast_input`.
* Updated existing test cases to properly work with static dispatch randomness.
* Do not use boxed `Rng`s for key generation in test networks.
* Use the passed-in `Rng` in `ReorderingAdversary`, instead of storing a boxed one.
* Fixed clippy lints after refactoring.
* Removed some no-longer necessary manual `fmt::Debug` implementations in test framework.
* Use `OsRng` even in tests in `binary_agreement_mitm`.
* Use a proper deterministic RNG in tests `binary_agreement_mitm`.
* Refactor `examples/simulation.rs` by not using `ThreadRng`, passing generic `Rng` parameters throughout and using a type alias instead of a newtype as the `Transaction`.
* Remove `thread_rng` use from `examples/node.rs`.
* Explicitly construct `InternalContrib` in `DynamicHoneyBadger::propose`.
* Fixed typo in description of `DistAlgorithm` trait.
* added fault checking in the net framework
* check that the node in the fault report is not faulty
* simplified a condition
* made error on fault a parameter of VirtualNet
* updated the BA test to error on fault
* explained errors and refactored an assignment
* typo fix
* test expected participants when receiving a batch from a correct node
* restrict the batch participant check to correct participants
* check only for the absence of unexpected batch participants
* check the minimum number of contributors for each batch
* corrected the minimum number of batch contributions
* removed an irrelevant FIXME
* added an equivalence check for batches output in the same epoch
* comment correction
Andreas Fackler
Weiliang Li
Andrew Gross
falcon-updater
Pawan Dhananjay
Andreas Fackler
Vladimir Komendantskiy
David Forstenlechner
Constantine Solovev
Sebastian Geisler
phahulin
Marc Brinkmann
Demi Marie Obenour