Change the QHB criterion for moving on to the next epoch.

`QueueingHoneyBadger` now waits after an output, and only makes its
proposal for the next epoch when:

* there are pending transactions in the queue,
* there are pending key generation or vote messages, or
* _f + 1_ other validators have already made their proposal.

This rule should work well for small networks: With 1 - 3 nodes, it will
produce a new batch whenever at least one of them has transactions to
contribute. In larger networks, it prevents an adversary controlling _f_
nodes from producing lots of empty epochs.

An exception is made for a currently joining validator: We will commit
up to _(N + 1)² + 1_ key generation messages for them, which is the
maximum number a correct node will send.
This commit is contained in:
Andreas Fackler 2018-08-01 10:41:09 +02:00
parent b3e1452a3a
commit d19afc2634
7 changed files with 199 additions and 70 deletions

View File

@ -164,6 +164,11 @@ impl<NodeUid: Clone + Debug + Ord + Rand> CommonSubset<NodeUid> {
self.process_broadcast(&uid, |bc| bc.input(value))
}
/// Returns the number of validators from which we have already received a proposal.
pub(crate) fn received_proposals(&self) -> usize {
self.broadcast_results.len()
}
/// Receives a broadcast message from a remote node `sender_id` concerning a
/// value proposed by the node `proposer_id`.
fn handle_broadcast(

View File

@ -61,7 +61,7 @@ where
vote_counter: VoteCounter::new(arc_netinfo, 0),
key_gen_msg_buffer: Vec::new(),
honey_badger,
key_gen: None,
key_gen_state: None,
incoming_queue: Vec::new(),
}
}
@ -104,7 +104,7 @@ where
vote_counter: VoteCounter::new(arc_netinfo, join_plan.epoch),
key_gen_msg_buffer: Vec::new(),
honey_badger,
key_gen: None,
key_gen_state: None,
incoming_queue: Vec::new(),
};
let step = match join_plan.change {

View File

@ -13,8 +13,8 @@
//! `Change` input variant, which contains either a vote `Add(node_id, public_key)`, to add an
//! existing observer to the set of validators, or `Remove(node_id)` to remove it. Each
//! validator can have at most one active vote, and casting another vote revokes the previous one.
//! Once a simple majority of validators has the same active vote, a reconfiguration process
//! begins: They create new cryptographic key shares for the new group of validators.
//! Once _f + 1_ validators have the same active vote, a reconfiguration process begins: They
//! create new cryptographic key shares for the new group of validators.
//!
//! The state of that process after each epoch is communicated via the `change` field in `Batch`.
//! When this contains an `InProgress(..)` value, key generation begins. The joining validator (in
@ -50,10 +50,10 @@
//! contributions in its own batch. The other transactions are processed: votes are counted and key
//! generation messages are passed into a `SyncKeyGen` instance.
//!
//! Whenever a change receives a majority of votes, the votes are reset and key generation for that
//! Whenever a change receives _f + 1_ votes, the votes are reset and key generation for that
//! change begins. If key generation completes successfully, the Honey Badger instance is dropped,
//! and replaced by a new one with the new set of participants. If a different change gains a
//! majority before that happens, key generation resets again, and is attempted for the new change.
//! and replaced by a new one with the new set of participants. If a different change wins a
//! vote before that happens, key generation resets again, and is attempted for the new change.
use rand::Rand;
use std::collections::BTreeMap;
@ -67,7 +67,7 @@ use crypto::{PublicKey, PublicKeySet, Signature};
use serde::{Deserialize, Serialize};
use self::votes::{SignedVote, VoteCounter};
use fault_log::{FaultKind, FaultLog};
use fault_log::{Fault, FaultKind, FaultLog};
use honey_badger::{self, HoneyBadger, Message as HbMessage};
use messaging::{self, DistAlgorithm, NetworkInfo, Target};
use sync_key_gen::{Ack, Part, PartOutcome, SyncKeyGen};
@ -108,7 +108,7 @@ pub struct DynamicHoneyBadger<C, NodeUid: Rand> {
/// The `HoneyBadger` instance with the current set of nodes.
honey_badger: HoneyBadger<InternalContrib<C, NodeUid>, NodeUid>,
/// The current key generation process, and the change it applies to.
key_gen: Option<(SyncKeyGen<NodeUid>, Change<NodeUid>)>,
key_gen_state: Option<KeyGenState<NodeUid>>,
/// A queue for messages from future epochs that cannot be handled yet.
incoming_queue: Vec<(NodeUid, Message<NodeUid>)>,
}
@ -154,10 +154,9 @@ where
Message::HoneyBadger(_, hb_msg) => {
self.handle_honey_badger_message(sender_id, hb_msg)
}
Message::KeyGen(_, kg_msg, sig) => {
self.handle_key_gen_message(sender_id, kg_msg, *sig)?;
Ok(Step::default())
}
Message::KeyGen(_, kg_msg, sig) => self
.handle_key_gen_message(sender_id, kg_msg, *sig)
.map(FaultLog::into),
Message::SignedVote(signed_vote) => self
.vote_counter
.add_pending_vote(sender_id, signed_vote)
@ -218,6 +217,34 @@ where
&self.netinfo
}
/// Returns `true` if we should make our contribution for the next epoch, even if we don't have
/// content ourselves, to avoid stalling the network.
///
/// By proposing only if this returns `true`, you can prevent an adversary from making the
/// network output empty baches indefinitely, but it also means that the network won't advance
/// if fewer than _f + 1_ nodes have pending contributions.
pub fn should_propose(&self) -> bool {
if self.has_input() {
return false; // We have already proposed.
}
if self.honey_badger.received_proposals() > self.netinfo.num_faulty() {
return true; // At least one correct node wants to move on to the next epoch.
}
let is_our_vote = |signed_vote: &SignedVote<_>| signed_vote.voter() == self.our_id();
if self.vote_counter.pending_votes().any(is_our_vote) {
return true; // We have pending input to vote for a validator change.
}
let kgs = match self.key_gen_state {
None => return false, // No ongoing key generation.
Some(ref kgs) => kgs,
};
// If either we or the candidate have a pending key gen message, we should propose.
let ours_or_candidates = |msg: &SignedKeyGenMsg<_>| {
msg.1 == *self.our_id() || Some(&msg.1) == kgs.change.candidate()
};
self.key_gen_msg_buffer.iter().any(ours_or_candidates)
}
/// Handles a message for the `HoneyBadger` instance.
fn handle_honey_badger_message(
&mut self,
@ -243,11 +270,41 @@ where
sender_id: &NodeUid,
kg_msg: KeyGenMessage,
sig: Signature,
) -> Result<()> {
self.verify_signature(sender_id, &sig, &kg_msg)?;
) -> Result<FaultLog<NodeUid>> {
if !self.verify_signature(sender_id, &sig, &kg_msg)? {
info!("Invalid signature from {:?} for: {:?}.", sender_id, kg_msg);
let fault_kind = FaultKind::InvalidKeyGenMessageSignature;
return Ok(Fault::new(sender_id.clone(), fault_kind).into());
}
let kgs = match self.key_gen_state {
Some(ref mut kgs) => kgs,
None => {
info!(
"Unexpected key gen message from {:?}: {:?}.",
sender_id, kg_msg
);
return Ok(Fault::new(sender_id.clone(), FaultKind::UnexpectedKeyGenMessage).into());
}
};
// If the joining node is correct, it will send at most (N + 1)² + 1 key generation
// messages.
if Some(sender_id) == kgs.change.candidate() {
let n = self.netinfo.num_nodes() + 1;
if kgs.candidate_msg_count > n * n {
info!(
"Too many key gen messages from candidate {:?}: {:?}.",
sender_id, kg_msg
);
let fault_kind = FaultKind::TooManyCandidateKeyGenMessages;
return Ok(Fault::new(sender_id.clone(), fault_kind).into());
}
kgs.candidate_msg_count += 1;
}
let tx = SignedKeyGenMsg(self.start_epoch, sender_id.clone(), kg_msg, sig);
self.key_gen_msg_buffer.push(tx);
Ok(())
Ok(FaultLog::default())
}
/// Processes all pending batches output by Honey Badger.
@ -272,7 +329,7 @@ where
} = int_contrib;
step.fault_log
.extend(self.vote_counter.add_committed_votes(&id, votes)?);
batch.contributions.insert(id, contrib);
batch.contributions.insert(id.clone(), contrib);
self.key_gen_msg_buffer
.retain(|skgm| !key_gen_messages.contains(skgm));
for SignedKeyGenMsg(epoch, s_id, kg_msg, sig) in key_gen_messages {
@ -281,9 +338,12 @@ where
continue;
}
if !self.verify_signature(&s_id, &sig, &kg_msg)? {
info!("Invalid signature from {:?} for: {:?}.", s_id, kg_msg);
info!(
"Invalid signature in {:?}'s batch from {:?} for: {:?}.",
id, s_id, kg_msg
);
let fault_kind = FaultKind::InvalidKeyGenMessageSignature;
step.fault_log.append(s_id.clone(), fault_kind);
step.fault_log.append(id.clone(), fault_kind);
continue;
}
step.extend(match kg_msg {
@ -293,14 +353,14 @@ where
}
}
if let Some((key_gen, change)) = self.take_ready_key_gen() {
if let Some(kgs) = self.take_ready_key_gen() {
// If DKG completed, apply the change, restart Honey Badger, and inform the user.
debug!("{:?} DKG for {:?} complete!", self.our_id(), change);
self.netinfo = key_gen.into_network_info();
debug!("{:?} DKG for {:?} complete!", self.our_id(), kgs.change);
self.netinfo = kgs.key_gen.into_network_info();
self.restart_honey_badger(batch.epoch + 1);
batch.set_change(ChangeState::Complete(change), &self.netinfo);
} else if let Some(change) = self.vote_counter.compute_majority().cloned() {
// If there is a majority, restart DKG. Inform the user about the current change.
batch.set_change(ChangeState::Complete(kgs.change), &self.netinfo);
} else if let Some(change) = self.vote_counter.compute_winner().cloned() {
// If there is a new change, restart DKG. Inform the user about the current change.
step.extend(self.update_key_gen(batch.epoch + 1, &change)?);
batch.set_change(ChangeState::InProgress(change), &self.netinfo);
}
@ -316,10 +376,10 @@ where
Ok(step)
}
/// If the majority of votes has changed, restarts Key Generation for the set of nodes implied
/// If the winner of the vote has changed, restarts Key Generation for the set of nodes implied
/// by the current change.
fn update_key_gen(&mut self, epoch: u64, change: &Change<NodeUid>) -> Result<Step<C, NodeUid>> {
if self.key_gen.as_ref().map(|&(_, ref ch)| ch) == Some(change) {
if self.key_gen_state.as_ref().map(|kgs| &kgs.change) == Some(change) {
return Ok(Step::default()); // The change is the same as before. Continue DKG as is.
}
debug!("{:?} Restarting DKG for {:?}.", self.our_id(), change);
@ -338,7 +398,7 @@ where
let sk = self.netinfo.secret_key().clone();
let our_uid = self.our_id().clone();
let (key_gen, part) = SyncKeyGen::new(our_uid, sk, pub_keys, threshold);
self.key_gen = Some((key_gen, change.clone()));
self.key_gen_state = Some(KeyGenState::new(key_gen, change.clone()));
if let Some(part) = part {
self.send_transaction(KeyGenMessage::Part(part))
} else {
@ -360,10 +420,8 @@ where
/// Handles a `Part` message that was output by Honey Badger.
fn handle_part(&mut self, sender_id: &NodeUid, part: Part) -> Result<Step<C, NodeUid>> {
let handle = |&mut (ref mut key_gen, _): &mut (SyncKeyGen<NodeUid>, _)| {
key_gen.handle_part(&sender_id, part)
};
match self.key_gen.as_mut().and_then(handle) {
let handle = |kgs: &mut KeyGenState<NodeUid>| kgs.key_gen.handle_part(&sender_id, part);
match self.key_gen_state.as_mut().and_then(handle) {
Some(PartOutcome::Valid(ack)) => self.send_transaction(KeyGenMessage::Ack(ack)),
Some(PartOutcome::Invalid(fault_log)) => Ok(fault_log.into()),
None => Ok(Step::default()),
@ -372,8 +430,8 @@ where
/// Handles an `Ack` message that was output by Honey Badger.
fn handle_ack(&mut self, sender_id: &NodeUid, ack: Ack) -> Result<FaultLog<NodeUid>> {
if let Some(&mut (ref mut key_gen, _)) = self.key_gen.as_mut() {
Ok(key_gen.handle_ack(&sender_id, ack))
if let Some(kgs) = self.key_gen_state.as_mut() {
Ok(kgs.key_gen.handle_ack(sender_id, ack))
} else {
Ok(FaultLog::new())
}
@ -394,18 +452,18 @@ where
Ok(Target::All.message(msg).into())
}
/// If the current Key Generation process is ready, returns the `SyncKeyGen`.
/// If the current Key Generation process is ready, returns the `KeyGenState`.
///
/// We require the minimum number of completed proposals (`SyncKeyGen::is_ready`) and if a new
/// node is joining, we require in addition that the new node's proposal is complete. That way
/// the new node knows that it's key is secret, without having to trust any number of nodes.
fn take_ready_key_gen(&mut self) -> Option<(SyncKeyGen<NodeUid>, Change<NodeUid>)> {
let is_ready = |&(ref key_gen, ref change): &(SyncKeyGen<_>, Change<_>)| {
let candidate_ready = |id: &NodeUid| key_gen.is_node_ready(id);
key_gen.is_ready() && change.candidate().map_or(true, candidate_ready)
};
if self.key_gen.as_ref().map_or(false, is_ready) {
self.key_gen.take()
fn take_ready_key_gen(&mut self) -> Option<KeyGenState<NodeUid>> {
if self
.key_gen_state
.as_ref()
.map_or(false, KeyGenState::is_ready)
{
self.key_gen_state.take()
} else {
None
}
@ -413,6 +471,8 @@ where
/// Returns `true` if the signature of `kg_msg` by the node with the specified ID is valid.
/// Returns an error if the payload fails to serialize.
///
/// This accepts signatures from both validators and the currently joining candidate, if any.
fn verify_signature(
&self,
node_id: &NodeUid,
@ -421,15 +481,12 @@ where
) -> Result<bool> {
let ser =
bincode::serialize(kg_msg).map_err(|err| ErrorKind::VerifySignatureBincode(*err))?;
let pk_opt = (self.netinfo.public_key(node_id)).or_else(|| {
self.key_gen
.iter()
.filter_map(|&(_, ref change): &(_, Change<_>)| match *change {
Change::Add(ref id, ref pk) if id == node_id => Some(pk),
Change::Add(_, _) | Change::Remove(_) => None,
})
.next()
});
let get_candidate_key = || {
self.key_gen_state
.as_ref()
.and_then(|kgs| kgs.candidate_key(node_id))
};
let pk_opt = self.netinfo.public_key(node_id).or_else(get_candidate_key);
Ok(pk_opt.map_or(false, |pk| pk.verify(&sig, ser)))
}
}
@ -504,3 +561,40 @@ pub struct JoinPlan<NodeUid: Ord> {
/// The public keys of the nodes taking part in key generation.
pub_keys: BTreeMap<NodeUid, PublicKey>,
}
/// The ongoing key generation, together with information about the validator change.
#[derive(Debug)]
struct KeyGenState<NodeUid> {
/// The key generation instance.
key_gen: SyncKeyGen<NodeUid>,
/// The change for which key generation is performed.
change: Change<NodeUid>,
/// The number of key generation messages received from the candidate. At most _N² + 1_ are
/// accepted.
candidate_msg_count: usize,
}
impl<NodeUid: Ord + Clone + Debug> KeyGenState<NodeUid> {
fn new(key_gen: SyncKeyGen<NodeUid>, change: Change<NodeUid>) -> Self {
KeyGenState {
key_gen,
change,
candidate_msg_count: 0,
}
}
/// Returns `true` if the candidate's, if any, as well as enough validators' key generation
/// parts have been completed.
fn is_ready(&self) -> bool {
let candidate_ready = |id: &NodeUid| self.key_gen.is_node_ready(id);
self.key_gen.is_ready() && self.change.candidate().map_or(true, candidate_ready)
}
/// If the node `node_id` is the currently joining candidate, returns its public key.
fn candidate_key(&self, node_id: &NodeUid) -> Option<&PublicKey> {
match self.change {
Change::Add(ref id, ref pk) if id == node_id => Some(pk),
Change::Add(_, _) | Change::Remove(_) => None,
}
}
}

View File

@ -13,7 +13,7 @@ use messaging::NetworkInfo;
/// A buffer and counter collecting pending and committed votes for validator set changes.
///
/// This is reset whenever the set of validators changes or a change reaches a majority. We call
/// This is reset whenever the set of validators changes or a change reaches _f + 1_ votes. We call
/// the epochs since the last reset the current _era_.
#[derive(Debug)]
pub struct VoteCounter<NodeUid> {
@ -134,14 +134,14 @@ where
Ok(FaultLog::new())
}
/// Returns the change that has a majority of votes, if any.
pub fn compute_majority(&self) -> Option<&Change<NodeUid>> {
/// Returns the change that has at least _f + 1_ votes, if any.
pub fn compute_winner(&self) -> Option<&Change<NodeUid>> {
let mut vote_counts: HashMap<&Change<NodeUid>, usize> = HashMap::new();
for vote in self.committed.values() {
let change = &vote.change;
let entry = vote_counts.entry(change).or_insert(0);
*entry += 1;
if *entry * 2 > self.netinfo.num_nodes() {
if *entry > self.netinfo.num_faulty() {
return Some(change);
}
}
@ -180,6 +180,10 @@ impl<NodeUid> SignedVote<NodeUid> {
pub fn era(&self) -> u64 {
self.vote.era
}
pub fn voter(&self) -> &NodeUid {
&self.voter
}
}
#[cfg(test)]
@ -270,14 +274,14 @@ mod tests {
#[test]
fn test_committed_votes() {
let node_num = 4;
let node_num = 4; // At most one faulty node.
let era = 5;
// Create the counter instances and the matrix of signed votes.
let (mut counters, sv) = setup(node_num, era);
// We will only use counter number 0.
let ct = &mut counters[0];
let mut vote_batch = vec![sv[1][1].clone(), sv[2][1].clone()];
let mut vote_batch = vec![sv[1][1].clone()];
// Include a vote with a wrong signature.
vote_batch.push(SignedVote {
sig: sv[2][1].sig.clone(),
@ -288,13 +292,13 @@ mod tests {
.expect("add committed");
let expected_faults = FaultLog::init(1, FaultKind::InvalidCommittedVote);
assert_eq!(faults, expected_faults);
assert_eq!(ct.compute_majority(), None);
assert_eq!(ct.compute_winner(), None);
// Adding the third vote for `Remove(1)` should return the change: It has the majority.
// Adding the second vote for `Remove(1)` should return the change: It has f + 1 votes.
let faults = ct
.add_committed_vote(&1, sv[3][1].clone())
.add_committed_vote(&1, sv[2][1].clone())
.expect("add committed");
assert!(faults.is_empty());
assert_eq!(ct.compute_majority(), Some(&Change::Remove(1)));
assert_eq!(ct.compute_winner(), Some(&Change::Remove(1)));
}
}

View File

@ -26,6 +26,11 @@ pub enum FaultKind {
/// `DynamicHoneyBadger` received a key generation message with an invalid
/// signature.
InvalidKeyGenMessageSignature,
/// `DynamicHoneyBadger` received a key generation message when there was no key generation in
/// progress.
UnexpectedKeyGenMessage,
/// `DynamicHoneyBadger` received more key generation messages from the candidate than expected.
TooManyCandidateKeyGenMessages,
/// `DynamicHoneyBadger` received a message (Accept, Propose, or Change)
/// with an invalid signature.
IncorrectPayloadSignature,

View File

@ -132,6 +132,14 @@ where
!self.netinfo.is_validator() || self.has_input
}
/// Returns the number of validators from which we have already received a proposal for the
/// current epoch.
pub(crate) fn received_proposals(&self) -> usize {
self.common_subsets
.get(&self.epoch)
.map_or(0, CommonSubset::received_proposals)
}
/// Handles a message for the given epoch.
fn handle_message_content(
&mut self,

View File

@ -4,7 +4,9 @@
//! an epoch is output, it will automatically select a list of pending transactions and propose it
//! for the next one. The user can continuously add more pending transactions to the queue.
//!
//! **Note**: `QueueingHoneyBadger` currently requires at least two validators.
//! If there are no pending transactions, no validators in the process of being added or
//! removed and not enough other nodes have proposed yet, no automatic proposal will be made: The
//! network then waits until at least _f + 1_ have any content for the next epoch.
//!
//! ## How it works
//!
@ -187,17 +189,19 @@ where
fn input(&mut self, input: Self::Input) -> Result<Step<Tx, NodeUid>> {
// User transactions are forwarded to `HoneyBadger` right away. Internal messages are
// in addition signed and broadcast.
match input {
let mut step = match input {
Input::User(tx) => {
self.queue.0.push_back(tx);
Ok(Step::default())
Step::default()
}
Input::Change(change) => Ok(self
Input::Change(change) => self
.dyn_hb
.input(Input::Change(change))
.map_err(ErrorKind::Input)?
.convert()),
}
.convert(),
};
step.extend(self.propose()?);
Ok(step)
}
fn handle_message(
@ -244,12 +248,21 @@ where
&self.dyn_hb
}
/// Returns `true` if we are ready to propose our contribution for the next epoch, i.e. if the
/// previous epoch has completed and we have either pending transactions or we are required to
/// make a proposal to avoid stalling the network.
fn can_propose(&self) -> bool {
if self.dyn_hb.has_input() {
return false; // Previous epoch is still in progress.
}
!self.queue.0.is_empty() || self.dyn_hb.should_propose()
}
/// Initiates the next epoch by proposing a batch from the queue.
fn propose(&mut self) -> Result<Step<Tx, NodeUid>> {
let amount = cmp::max(1, self.batch_size / self.dyn_hb.netinfo().num_nodes());
// TODO: This will loop indefinitely if we are the only validator.
let mut step = Step::default();
while !self.dyn_hb.has_input() {
while self.can_propose() {
let amount = cmp::max(1, self.batch_size / self.dyn_hb.netinfo().num_nodes());
let proposal = self.queue.choose(amount, self.batch_size);
step.extend(
self.dyn_hb