hbbft/src/honey_badger/honey_badger.rs

238 lines
8.5 KiB
Rust
Raw Permalink Normal View History

use std::collections::btree_map::Entry;
use std::collections::BTreeMap;
use std::sync::Arc;
2018-10-29 07:36:56 -07:00
use derivative::Derivative;
use rand::Rng;
2019-04-01 05:37:14 -07:00
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use super::epoch_state::EpochState;
2018-12-18 07:17:46 -08:00
use super::{Batch, Error, FaultKind, HoneyBadgerBuilder, Message, Result};
use crate::{ConsensusProtocol, Contribution, Fault, NetworkInfo, NodeIdT};
use super::Params;
/// An instance of the Honey Badger Byzantine fault tolerant consensus algorithm.
#[derive(Derivative)]
#[derivative(Debug)]
pub struct HoneyBadger<C, N> {
/// Shared network data.
pub(super) netinfo: Arc<NetworkInfo<N>>,
2018-10-24 00:42:59 -07:00
/// A session identifier. Different session IDs foil replay attacks in two instances with the
/// same epoch numbers and the same validators.
pub(super) session_id: u64,
/// The earliest epoch from which we have not yet received output.
2018-07-31 13:27:22 -07:00
pub(super) epoch: u64,
/// Whether we have already submitted a proposal for the current epoch.
2018-07-31 13:27:22 -07:00
pub(super) has_input: bool,
/// The subalgorithms for ongoing epochs.
pub(super) epochs: BTreeMap<u64, EpochState<C, N>>,
/// Parameters controlling Honey Badger's behavior and performance.
pub(super) params: Params,
Better proptest persistence through deterministic randomness. (#248) * Add support for RNG instantiation in proptests. * Use `proptest` module strategy to create the rng for `net_dynamic_honey_badger`. * Use seed generation instead of RNG instantiation in tests. * Remove fixed RNG in `generate_map`. * `VirtualNet` now supports setting the random generator through the builder. * Add missing `time_limit` field to `::std::fmt::Debug` trait implementation on `NetBuilder`. * Pass an instantiated random number generator through `NewNodeInfo` as a convenience. * Make the random number generator of `DynamicHoneyBadgerBuilder` configurable, at the cost of now requiring mutability to call `build_first_node()`. * Ensure RNGs are derive from passed in seed in `net_dynamic_hb` tests. * Correct inappropriate use of `random::Random` instead of `Rng::gen` to generate dependent values in `binary_agreement`. The original implementation used `rand::random()`, which will always use the `thread_rng`, ignoring the fact that an RNG has actually been passed in. * Do not use `OsRng` but passed in RNG instead. * Use reference/non-reference passing of rngs more in line with the `rand` crates conventions. * Document `rng` field on `DynamicHoneyBadger`. * Make `SyncKeyGen` work with the extend (`encrypt_with_rng`) API of `threshold_crypto`. * Use passed-in random number generator in `HoneyBadger`. * Create `SubRng` crate in new `util` module to replace `create_rng()`. * Use an RNG seeded from the configure RNG when reinitializing `DynamicHoneyBadger`. * Use the correct branch of `threshold_crypto` with support for passing RNGs.
2018-10-02 07:24:51 -07:00
}
/// A `HoneyBadger` step, possibly containing multiple outputs.
pub type Step<C, N> = crate::CpStep<HoneyBadger<C, N>>;
2018-07-09 04:35:26 -07:00
impl<C, N> ConsensusProtocol for HoneyBadger<C, N>
where
2018-10-24 02:38:14 -07:00
C: Contribution + Serialize + DeserializeOwned,
N: NodeIdT,
{
2018-08-29 09:08:35 -07:00
type NodeId = N;
type Input = C;
type Output = Batch<C, N>;
type Message = Message<N>;
type Error = Error;
2018-12-18 07:17:46 -08:00
type FaultKind = FaultKind;
OsRng / external RNG Refactoring (#357) * Use `OsRng` in place of `thread_rng`. This changes the defaults of any builder by instantiating an `OsRng` instead of a `thread_rng`, the former being much more secure than the latter. Additionally, all the unit tests that still instantiate RNGs manually used `OsRng`s as well; while there is no actual need for this level of security in tests, the performance overhead is very small and random number generation complexity has such a small impact on these tests that the convenience of being able to ban `thread_rng` from the codebase altogether, setting a good example and avoid issues when refactoring later greatly outweigh the negatives. * Instead of storing random number generators in the various consensus algorithm instances, pass them in from the outside whenever they are needed. This changes a large amount of interfaces (and in this commit is only partially done, since `DistAlgorithm` needs to be fundamentally altered as well. It also obsoletes parts of the `util` module. * Added an `R: Rng` type parameter to both methods of `DistAlgorithm`, forcing callers to pass in their own Rngs. * Fixed documentation grammar and spelling in some of the altered interfaces due to RNG refactoring. * Move `rng` argument to the end of the argument for most functions. Also includes a reformatting due to Rust 1.30. * Updated tests, accomodate `rng`-API changes. * Fixed remaining compilation issues with new RNG code. * Fix illegal `self` import outside curly braces. * Cleaned up comments and fixed broken definition of `broadcast_input`. * Updated existing test cases to properly work with static dispatch randomness. * Do not use boxed `Rng`s for key generation in test networks. * Use the passed-in `Rng` in `ReorderingAdversary`, instead of storing a boxed one. * Fixed clippy lints after refactoring. * Removed some no-longer necessary manual `fmt::Debug` implementations in test framework. * Use `OsRng` even in tests in `binary_agreement_mitm`. * Use a proper deterministic RNG in tests `binary_agreement_mitm`. * Refactor `examples/simulation.rs` by not using `ThreadRng`, passing generic `Rng` parameters throughout and using a type alias instead of a newtype as the `Transaction`. * Remove `thread_rng` use from `examples/node.rs`. * Explicitly construct `InternalContrib` in `DynamicHoneyBadger::propose`. * Fixed typo in description of `DistAlgorithm` trait.
2018-12-14 04:51:09 -08:00
fn handle_input<R: Rng>(&mut self, input: Self::Input, rng: &mut R) -> Result<Step<C, N>> {
self.propose(&input, rng)
}
OsRng / external RNG Refactoring (#357) * Use `OsRng` in place of `thread_rng`. This changes the defaults of any builder by instantiating an `OsRng` instead of a `thread_rng`, the former being much more secure than the latter. Additionally, all the unit tests that still instantiate RNGs manually used `OsRng`s as well; while there is no actual need for this level of security in tests, the performance overhead is very small and random number generation complexity has such a small impact on these tests that the convenience of being able to ban `thread_rng` from the codebase altogether, setting a good example and avoid issues when refactoring later greatly outweigh the negatives. * Instead of storing random number generators in the various consensus algorithm instances, pass them in from the outside whenever they are needed. This changes a large amount of interfaces (and in this commit is only partially done, since `DistAlgorithm` needs to be fundamentally altered as well. It also obsoletes parts of the `util` module. * Added an `R: Rng` type parameter to both methods of `DistAlgorithm`, forcing callers to pass in their own Rngs. * Fixed documentation grammar and spelling in some of the altered interfaces due to RNG refactoring. * Move `rng` argument to the end of the argument for most functions. Also includes a reformatting due to Rust 1.30. * Updated tests, accomodate `rng`-API changes. * Fixed remaining compilation issues with new RNG code. * Fix illegal `self` import outside curly braces. * Cleaned up comments and fixed broken definition of `broadcast_input`. * Updated existing test cases to properly work with static dispatch randomness. * Do not use boxed `Rng`s for key generation in test networks. * Use the passed-in `Rng` in `ReorderingAdversary`, instead of storing a boxed one. * Fixed clippy lints after refactoring. * Removed some no-longer necessary manual `fmt::Debug` implementations in test framework. * Use `OsRng` even in tests in `binary_agreement_mitm`. * Use a proper deterministic RNG in tests `binary_agreement_mitm`. * Refactor `examples/simulation.rs` by not using `ThreadRng`, passing generic `Rng` parameters throughout and using a type alias instead of a newtype as the `Transaction`. * Remove `thread_rng` use from `examples/node.rs`. * Explicitly construct `InternalContrib` in `DynamicHoneyBadger::propose`. * Fixed typo in description of `DistAlgorithm` trait.
2018-12-14 04:51:09 -08:00
fn handle_message<R: Rng>(
&mut self,
sender_id: &Self::NodeId,
message: Self::Message,
_rng: &mut R,
) -> Result<Step<C, N>> {
self.handle_message(sender_id, message)
}
fn terminated(&self) -> bool {
false
}
fn our_id(&self) -> &N {
2018-08-29 09:08:35 -07:00
self.netinfo.our_id()
}
}
impl<C, N> HoneyBadger<C, N>
where
2018-10-24 02:38:14 -07:00
C: Contribution + Serialize + DeserializeOwned,
N: NodeIdT,
{
/// Returns a new `HoneyBadgerBuilder` configured to use the node IDs and cryptographic keys
2018-06-28 14:07:11 -07:00
/// specified by `netinfo`.
pub fn builder(netinfo: Arc<NetworkInfo<N>>) -> HoneyBadgerBuilder<C, N> {
2018-06-28 14:07:11 -07:00
HoneyBadgerBuilder::new(netinfo)
}
/// Proposes a contribution in the current epoch.
///
/// Returns an error if we already made a proposal in this epoch.
///
/// If we are the only validator, this will immediately output a batch, containing our
/// proposal.
OsRng / external RNG Refactoring (#357) * Use `OsRng` in place of `thread_rng`. This changes the defaults of any builder by instantiating an `OsRng` instead of a `thread_rng`, the former being much more secure than the latter. Additionally, all the unit tests that still instantiate RNGs manually used `OsRng`s as well; while there is no actual need for this level of security in tests, the performance overhead is very small and random number generation complexity has such a small impact on these tests that the convenience of being able to ban `thread_rng` from the codebase altogether, setting a good example and avoid issues when refactoring later greatly outweigh the negatives. * Instead of storing random number generators in the various consensus algorithm instances, pass them in from the outside whenever they are needed. This changes a large amount of interfaces (and in this commit is only partially done, since `DistAlgorithm` needs to be fundamentally altered as well. It also obsoletes parts of the `util` module. * Added an `R: Rng` type parameter to both methods of `DistAlgorithm`, forcing callers to pass in their own Rngs. * Fixed documentation grammar and spelling in some of the altered interfaces due to RNG refactoring. * Move `rng` argument to the end of the argument for most functions. Also includes a reformatting due to Rust 1.30. * Updated tests, accomodate `rng`-API changes. * Fixed remaining compilation issues with new RNG code. * Fix illegal `self` import outside curly braces. * Cleaned up comments and fixed broken definition of `broadcast_input`. * Updated existing test cases to properly work with static dispatch randomness. * Do not use boxed `Rng`s for key generation in test networks. * Use the passed-in `Rng` in `ReorderingAdversary`, instead of storing a boxed one. * Fixed clippy lints after refactoring. * Removed some no-longer necessary manual `fmt::Debug` implementations in test framework. * Use `OsRng` even in tests in `binary_agreement_mitm`. * Use a proper deterministic RNG in tests `binary_agreement_mitm`. * Refactor `examples/simulation.rs` by not using `ThreadRng`, passing generic `Rng` parameters throughout and using a type alias instead of a newtype as the `Transaction`. * Remove `thread_rng` use from `examples/node.rs`. * Explicitly construct `InternalContrib` in `DynamicHoneyBadger::propose`. * Fixed typo in description of `DistAlgorithm` trait.
2018-12-14 04:51:09 -08:00
pub fn propose<R: Rng>(&mut self, proposal: &C, rng: &mut R) -> Result<Step<C, N>> {
if !self.netinfo.is_validator() {
return Ok(Step::default());
}
self.has_input = true;
let step = self.epoch_state_mut(self.epoch)?.propose(proposal, rng)?;
Ok(step.join(self.try_output_batches()?))
}
/// Handles a message received from `sender_id`.
///
/// This must be called with every message we receive from another node.
pub fn handle_message(&mut self, sender_id: &N, message: Message<N>) -> Result<Step<C, N>> {
if !self.netinfo.is_node_validator(sender_id) {
return Err(Error::UnknownSender);
}
let Message { epoch, content } = message;
if epoch > self.epoch + self.params.max_future_epochs {
Ok(Fault::new(sender_id.clone(), FaultKind::UnexpectedHbMessageEpoch).into())
} else if epoch < self.epoch {
// The message is late; discard it.
Ok(Step::default())
} else {
let step = self
.epoch_state_mut(epoch)?
.handle_message_content(sender_id, content)?;
2018-10-25 08:07:52 -07:00
Ok(step.join(self.try_output_batches()?))
}
}
/// Returns `true` if input for the current epoch has already been provided.
pub fn has_input(&self) -> bool {
!self.netinfo.is_validator() || self.has_input
}
/// Returns the current encryption schedule that determines in which epochs contributions are
/// encrypted.
pub fn get_encryption_schedule(&self) -> EncryptionSchedule {
self.params.encryption_schedule
}
/// Returns the epoch of the next batch that will be output.
pub fn next_epoch(&self) -> u64 {
self.epoch
}
/// Returns the information about the node IDs in the network, and the threshold key shares.
pub fn netinfo(&self) -> &Arc<NetworkInfo<N>> {
&self.netinfo
}
2019-07-23 01:48:00 -07:00
/// Skips all epochs before the specified one.
///
/// This must only be called if it is guaranteed to be called in all instances that have not
/// progressed to that epoch yet. Otherwise an instance can be left behind in a skipped epoch.
pub fn skip_to_epoch(&mut self, epoch: u64) {
while self.epoch < epoch {
self.update_epoch();
}
}
/// Returns the number of validators from which we have already received a proposal for the
/// current epoch.
///
/// This can be used to find out whether our node is stalling progress. Depending on the
/// application logic, nodes may e.g. only propose when they have any pending transactions. In
/// that case, they should repeatedly call this method: if it returns _f + 1_ or more, that
/// means at least one correct node has proposed a contribution. In that case, we might want to
/// propose one, too, even if it's empty, to avoid unnecessarily delaying the next batch.
pub fn received_proposals(&self) -> usize {
self.epochs
.get(&self.epoch)
.map_or(0, EpochState::received_proposals)
}
/// Increments the epoch number and clears any state that is local to the finished epoch.
2018-10-25 08:07:52 -07:00
fn update_epoch(&mut self) {
// Clear the state of the old epoch.
self.epochs.remove(&self.epoch);
self.epoch += 1;
self.has_input = false;
}
/// Tries to decrypt contributions from all proposers and output those in a batch.
fn try_output_batches(&mut self) -> Result<Step<C, N>> {
let mut step = Step::default();
while let Some((batch, fault_log)) = self
.epochs
.get(&self.epoch)
.and_then(EpochState::try_output_batch)
{
// Queue the output and advance the epoch.
step.output.push(batch);
step.fault_log.extend(fault_log);
2018-10-25 08:07:52 -07:00
self.update_epoch();
}
Ok(step)
}
/// Returns a mutable reference to the state of the given `epoch`. Initializes a new one, if it
/// doesn't exist yet.
fn epoch_state_mut(&mut self, epoch: u64) -> Result<&mut EpochState<C, N>> {
Ok(match self.epochs.entry(epoch) {
Entry::Occupied(entry) => entry.into_mut(),
Entry::Vacant(entry) => entry.insert(EpochState::new(
self.netinfo.clone(),
2018-10-24 00:42:59 -07:00
self.session_id,
epoch,
self.params.subset_handling_strategy.clone(),
self.params.encryption_schedule.use_on_epoch(epoch),
)?),
})
}
2018-10-25 08:07:52 -07:00
/// Returns the maximum future epochs of the Honey Badger algorithm instance.
pub fn max_future_epochs(&self) -> u64 {
self.params.max_future_epochs
2018-10-25 08:07:52 -07:00
}
/// Returns the parameters controlling Honey Badger's behavior and performance.
pub fn params(&self) -> &Params {
&self.params
}
}
/// How frequently Threshold Encryption should be used.
#[derive(Clone, Copy, Eq, PartialEq, Serialize, Deserialize, Hash, Debug)]
pub enum EncryptionSchedule {
/// Always encrypt. All contributions are encrypted in every epoch.
Always,
/// Never encrypt. All contributions are plaintext in every epoch.
Never,
/// Every _n_-th epoch uses encryption. In all other epochs, contributions are plaintext.
EveryNthEpoch(u32),
/// With `TickTock(n, m)`, `n` epochs use encryption, followed by `m` epochs that don't.
/// `m` out of `n + m` epochs will use plaintext contributions.
TickTock(u32, u32),
}
impl EncryptionSchedule {
/// Returns `true` if the contributions in the `epoch` should be encrypted.
pub fn use_on_epoch(self, epoch: u64) -> bool {
match self {
EncryptionSchedule::Always => true,
EncryptionSchedule::Never => false,
EncryptionSchedule::EveryNthEpoch(n) => (epoch % u64::from(n)) == 0,
EncryptionSchedule::TickTock(on, off) => (epoch % u64::from(on + off)) <= u64::from(on),
}
}
}