zebra/zebra-consensus/src/checkpoint/list.rs

242 lines
9.1 KiB
Rust
Raw Normal View History

//! Checkpoint lists for checkpoint-based block verification
//!
//! Each checkpoint consists of a coinbase height and block header hash.
//!
//! Checkpoints can be used to verify their ancestors, by chaining backwards
//! to another checkpoint, via each block's parent block hash.
#[cfg(test)]
mod tests;
use crate::BoxError;
use std::{
collections::{BTreeMap, HashSet},
ops::RangeBounds,
str::FromStr,
};
use zebra_chain::block;
use zebra_chain::parameters::Network;
/// The hard-coded checkpoints for mainnet, generated using the
/// `zebra-checkpoints` tool.
///
/// To regenerate the latest checkpoints, use the following commands:
/// ```sh
/// LAST_CHECKPOINT=$(tail -1 main-checkpoints.txt | cut -d' ' -f1)
/// echo "$LAST_CHECKPOINT"
/// zebra-checkpoints --cli /path/to/zcash-cli --last-checkpoint "$LAST_CHECKPOINT" >> main-checkpoints.txt &
/// tail -f main-checkpoints.txt
/// ```
///
/// See the checkpoints [./README.md] for more details.
const MAINNET_CHECKPOINTS: &str = include_str!("main-checkpoints.txt");
/// The hard-coded checkpoints for testnet, generated using the
/// `zebra-checkpoints` tool.
///
/// To use testnet, use the testnet checkpoints file, and run
/// `zebra-checkpoints [other args] -- -testnet`.
///
/// See [`MAINNET_CHECKPOINTS`] for detailed `zebra-checkpoints` usage
/// information.
const TESTNET_CHECKPOINTS: &str = include_str!("test-checkpoints.txt");
/// Network methods related to checkpoints
pub trait ParameterCheckpoint {
/// Returns the hash for the genesis block in `network`.
fn genesis_hash(&self) -> zebra_chain::block::Hash;
/// Returns the hard-coded checkpoint list for `network`.
fn checkpoint_list(&self) -> CheckpointList;
}
impl ParameterCheckpoint for Network {
fn genesis_hash(&self) -> zebra_chain::block::Hash {
match self {
// zcash-cli getblockhash 0
Network::Mainnet => "00040fe8ec8471911baa1db1266ea15dd06b4a8a5c453883c000b031973dce08",
// zcash-cli -testnet getblockhash 0
Network::Testnet => "05a60a92d99d85997cce3b87616c089f6124d7342af37106edc76126334a2c38",
}
.parse()
.expect("hard-coded hash parses")
}
fn checkpoint_list(&self) -> CheckpointList {
// parse calls CheckpointList::from_list
let checkpoint_list: CheckpointList = match self {
Network::Mainnet => MAINNET_CHECKPOINTS
.parse()
.expect("Hard-coded Mainnet checkpoint list parses and validates"),
Network::Testnet => TESTNET_CHECKPOINTS
.parse()
.expect("Hard-coded Testnet checkpoint list parses and validates"),
};
match checkpoint_list.hash(block::Height(0)) {
Some(hash) if hash == self.genesis_hash() => checkpoint_list,
Some(_) => {
panic!("The hard-coded genesis checkpoint does not match the network genesis hash")
}
None => unreachable!("Parser should have checked for a missing genesis checkpoint"),
}
}
}
2020-07-22 05:10:36 -07:00
/// A list of block height and hash checkpoints.
///
/// Checkpoints should be chosen to avoid forks or chain reorganizations,
/// which only happen in the last few hundred blocks in the chain.
/// (zcashd allows chain reorganizations up to 99 blocks, and prunes
/// orphaned side-chains after 288 blocks.)
2020-07-22 05:10:36 -07:00
///
/// This is actually a bijective map, but since it is read-only, we use a
/// BTreeMap, and do the value uniqueness check on initialisation.
#[derive(Clone, Debug, Eq, Hash, PartialEq)]
fix(test): make full sync test more accurate (#3555) * feat(log): log current height when logging sync progress * fix(test): log the specific error when full sync tests fail * doc(start): remove an obsolete TODO We can't decrease this log level, because the tests rely on it. * fix(test): wait until mempool activates in full sync tests Changes the log message and log test so that the test only finishes when the mempool has activated. There is still a race condition here, between the log timer and mempool activation. But it should be very rare, because the mempool is activated immediately when `is_close_to_tip()` becomes true. * fix(test): warn when Zebra stalls below the maximum checkpoint height This also improves the full sync tests, because the warning is checked before logging a successful sync. * feat(log): warn when sync stalls downloading the genesis block * fix(test): warn when the state hasn't committed a block for a long time This also improves the full sync tests, because the warning is checked before logging a successful sync. * doc(test): update some sync acceptance test comments * fix(log): use Display formatting to log chrono::Duration Debug formatting is complicated and hard to read. * fix(log): stop saying that we've activated the mempool without checking it We're not checking if the mempool is active, so we can't say that. * fix(log): minor tidying and TODOs * fix(doc): fix a typo in the tests * fix(log): explain the post-checkpoint blocks in progress warning calculations * fix(doc): explain what could happen if we don't wait for extra blocks * fix(log): add a percent symbol to a percent log Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com> Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-17 18:00:24 -08:00
pub struct CheckpointList(BTreeMap<block::Height, block::Hash>);
impl FromStr for CheckpointList {
type Err = BoxError;
/// Parse a string into a CheckpointList.
///
/// Each line has one checkpoint, consisting of a `block::Height` and
/// `block::Hash`, separated by a single space.
///
/// Assumes that the provided genesis checkpoint is correct.
fn from_str(s: &str) -> Result<Self, Self::Err> {
let mut checkpoint_list: Vec<(block::Height, block::Hash)> = Vec::new();
for checkpoint in s.lines() {
let fields = checkpoint.split(' ').collect::<Vec<_>>();
if let [height, hash] = fields[..] {
checkpoint_list.push((height.parse()?, hash.parse()?));
} else {
Err(format!("Invalid checkpoint format: expected 2 space-separated fields but found {}: '{checkpoint}'", fields.len()))?;
};
}
2021-01-21 19:57:09 -08:00
CheckpointList::from_list(checkpoint_list)
}
}
impl CheckpointList {
/// Create a new checkpoint list for `network` from `checkpoint_list`.
///
/// Assumes that the provided genesis checkpoint is correct.
///
/// Checkpoint heights and checkpoint hashes must be unique.
/// There must be a checkpoint for a genesis block at block::Height 0.
/// (All other checkpoints are optional.)
pub(crate) fn from_list(
list: impl IntoIterator<Item = (block::Height, block::Hash)>,
) -> Result<Self, BoxError> {
// BTreeMap silently ignores duplicates, so we count the checkpoints
// before adding them to the map
let original_checkpoints: Vec<(block::Height, block::Hash)> = list.into_iter().collect();
let original_len = original_checkpoints.len();
let checkpoints: BTreeMap<block::Height, block::Hash> =
original_checkpoints.into_iter().collect();
2020-07-15 21:04:27 -07:00
// Check that the list starts with the correct genesis block
match checkpoints.iter().next() {
Some((block::Height(0), hash))
if (hash == &Network::Mainnet.genesis_hash()
|| hash == &Network::Testnet.genesis_hash()) => {}
Some((block::Height(0), _)) => {
Err("the genesis checkpoint does not match the Mainnet or Testnet genesis hash")?
2020-07-15 21:04:27 -07:00
}
Some(_) => Err("checkpoints must start at the genesis block height 0")?,
None => Err("there must be at least one checkpoint, for the genesis block")?,
};
// This check rejects duplicate heights, whether they have the same or
// different hashes
if checkpoints.len() != original_len {
Err("checkpoint heights must be unique")?;
}
let block_hashes: HashSet<&block::Hash> = checkpoints.values().collect();
if block_hashes.len() != original_len {
Err("checkpoint hashes must be unique")?;
}
// Make sure all the hashes are valid. In Bitcoin, [0; 32] is the null
// hash. It is also used as the parent hash of genesis blocks.
if block_hashes.contains(&block::Hash([0; 32])) {
Err("checkpoint list contains invalid checkpoint hash: found null hash")?;
}
let checkpoints = CheckpointList(checkpoints);
if checkpoints.max_height() > block::Height::MAX {
Err("checkpoint list contains invalid checkpoint: checkpoint height is greater than the maximum block height")?;
}
Ok(checkpoints)
}
/// Return true if there is a checkpoint at `height`.
///
/// See `BTreeMap::contains_key()` for details.
pub fn contains(&self, height: block::Height) -> bool {
self.0.contains_key(&height)
}
/// Returns the hash corresponding to the checkpoint at `height`,
/// or None if there is no checkpoint at that height.
///
/// See `BTreeMap::get()` for details.
pub fn hash(&self, height: block::Height) -> Option<block::Hash> {
self.0.get(&height).cloned()
}
/// Return the block height of the highest checkpoint in the checkpoint list.
///
/// If there is only a single checkpoint, then the maximum height will be
/// zero. (The genesis block.)
pub fn max_height(&self) -> block::Height {
self.max_height_in_range(..)
.expect("checkpoint lists must have at least one checkpoint")
}
/// Return the block height of the lowest checkpoint in a sub-range.
pub fn min_height_in_range<R>(&self, range: R) -> Option<block::Height>
where
R: RangeBounds<block::Height>,
{
self.0.range(range).map(|(height, _)| *height).next()
}
/// Return the block height of the highest checkpoint in a sub-range.
pub fn max_height_in_range<R>(&self, range: R) -> Option<block::Height>
where
R: RangeBounds<block::Height>,
{
self.0.range(range).map(|(height, _)| *height).next_back()
}
/// Returns an iterator over all the checkpoints, in increasing height order.
pub fn iter(&self) -> impl Iterator<Item = (&block::Height, &block::Hash)> {
self.0.iter()
}
feat(ui): Add a terminal-based progress bar to Zebra (#6235) * Implement Display and to_string() for NetworkUpgrade * Add a progress-bar feature to zebrad * Add the progress bar writer to the tracing component * Add a block progress bar transmitter * Correctly shut down the progress bar, and shut it down on an interrupt * Make it clearer that the progress task never exits * Add a config for writing logs to a file * Add a progress-bar feature to zebra-network * Add a progress bar for the address book size * Add progress bars for never attempted and failed peers * Add an optional limit and label to connection counters * Add open connection progress bars * Improve CheckpointList API and CheckpointVerifier debugging * Add checkpoint index and checkpoint queue progress bars * Security: Limit the number of non-finalized chains tracked by Zebra * Make some NonFinalizedState methods available with proptest-impl * Add a non-finalized chain count progress bar * Track the last fork height for newly forked chains * Add a should_count_metrics to Chain * Add a display method for PartialCumulativeWork * Add a progress bar for each chain fork * Add a NonFinalizedState::disable_metrics() method and switch to using it * Move metrics out of Chain because we can't update Arc<Chain> * Fix: consistently use best chain order when searching chains * Track Chain progress bars in NonFinalizedState * Display work as bits, not a multiple of the target difficulty * Handle negative fork lengths by reporting "No fork" * Correctly disable unused fork bars * clippy: rewrite using `match _.cmp(_) { ... }` * Initial mempool progress bar implementation * Update Cargo.lock * Add the actual transaction size as a description to the cost bar * Only show mempool progress bars after first activation * Add queued and rejected mempool progress bars * Clarify cost note is actual size * Add tracing.log_file config and progress-bar feature to zebrad docs * Derive Clone for Chain * Upgrade to howudoin 0.1.2 and remove some bug workarounds * Directly call the debug formatter to Display a Network Co-authored-by: Arya <aryasolhi@gmail.com> * Rename the address count metric to num_addresses Co-authored-by: Arya <aryasolhi@gmail.com> * Simplify reverse checkpoint lookup Co-authored-by: Arya <aryasolhi@gmail.com> * Simplify progress bar shutdown code Co-authored-by: Arya <aryasolhi@gmail.com> * Remove unused MIN_TRANSPARENT_TX_MEMPOOL_SIZE * Document that the progress task runs forever * Fix progress log formatting * If progress-bar is on, log to a file by default * Create missing directories for log files * Add file security docs for running Zebra with elevated permissions * Document automatic log file, spell progress-bar correctly --------- Co-authored-by: Arya <aryasolhi@gmail.com>
2023-04-13 01:42:17 -07:00
/// Returns the checkpoint at `height`, as a zero-based index.
/// If `height` is not a checkpoint height, returns the checkpoint immediately before that height.
pub fn prev_checkpoint_index(&self, height: block::Height) -> usize {
self.0
.keys()
.rposition(|&key| key <= height)
.expect("checkpoints must start at the genesis block height 0")
}
/// Returns the number of checkpoints in the list.
//
// Checkpoint lists are never empty by construction.
#[allow(clippy::len_without_is_empty)]
pub fn len(&self) -> usize {
self.0.len()
}
}