4c4dbfe7cd
* Simplify state service initialization in test Use the test helper function to remove redundant code. * Create `BestTipHeight` helper type This type abstracts away the calculation of the best tip height based on the finalized block height and the best non-finalized chain's tip. * Add `best_tip_height` field to `StateService` The receiver endpoint is currently ignored. * Return receiver endpoint from service constructor Make it available so that the best tip height can be watched. * Update finalized height after finalizing blocks After blocks from the queue are finalized and committed to disk, update the finalized block height. * Update best non-finalized height after validation Update the value of the best non-finalized chain tip block height after a new block is committed to the non-finalized state. * Update finalized height after loading from disk When `FinalizedState` is first created, it loads the state from persistent storage, and the finalized tip height is updated. Therefore, the `best_tip_height` must be notified of the initial value. * Update the finalized height on checkpoint commit When a checkpointed block is commited, it bypasses the non-finalized state, so there's an extra place where the finalized height has to be updated. * Add `best_tip_height` to `Handshake` service It can be configured using the `Builder::with_best_tip_height`. It's currently not used, but it will be used to determine if a connection to a remote peer should be rejected or not based on that peer's protocol version. * Require best tip height to init. `zebra_network` Without it the handshake service can't properly enforce the minimum network protocol version from peers. Zebrad obtains the best tip height endpoint from `zebra_state`, and the test vectors simply use a dummy endpoint that's fixed at the genesis height. * Pass `best_tip_height` to proto. ver. negotiation The protocol version negotiation code will reject connections to peers if they are using an old protocol version. An old version is determined based on the current known best chain tip height. * Handle an optional height in `Version` Fallback to the genesis height in `None` is specified. * Reject connections to peers on old proto. versions Avoid connecting to peers that are on protocol versions that don't recognize a network update. * Document why peers on old versions are rejected Describe why it's a security issue above the check. * Test if `BestTipHeight` starts with `None` Check if initially there is no best tip height. * Test if best tip height is max. of latest values After applying a list of random updates where each one either sets the finalized height or the non-finalized height, check that the best tip height is the maximum of the most recently set finalized height and the most recently set non-finalized height. * Add `queue_and_commit_finalized` method A small refactor to make testing easier. The handling of requests for committing non-finalized and finalized blocks is now more consistent. * Add `assert_block_can_be_validated` helper Refactor to move into a separate method some assertions that are done before a block is validated. This is to allow moving these assertions more easily to simplify testing. * Remove redundant PoW block assertion It's also checked in `zebra_state::service::check::block_is_contextually_valid`, and it was getting in the way of tests that received a gossiped block before finalizing enough blocks. * Create a test strategy for test vector chain Splits a chain loaded from the test vectors in two parts, containing the blocks to finalize and the blocks to keep in the non-finalized state. * Test committing blocks update best tip height Create a mock blockchain state, with a chain of finalized blocks and a chain of non-finalized blocks. Commit all the blocks appropriately, and verify that the best tip height is updated. Co-authored-by: teor <teor@riseup.net> |
||
|---|---|---|
| .github | ||
| book | ||
| docker | ||
| grafana | ||
| tower-batch | ||
| tower-fallback | ||
| zebra-chain | ||
| zebra-client | ||
| zebra-consensus | ||
| zebra-network | ||
| zebra-rpc | ||
| zebra-script | ||
| zebra-state | ||
| zebra-test | ||
| zebra-utils | ||
| zebrad | ||
| .gitignore | ||
| CHANGELOG.md | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| Cargo.lock | ||
| Cargo.toml | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| README.md | ||
| SECURITY.md | ||
| clippy.toml | ||
| cloudbuild.yaml | ||
| codecov.yml | ||
| firebase.json | ||
| katex-header.html | ||
| prometheus.yaml | ||
README.md
About
Zebra is the Zcash Foundation's independent, consensus-compatible implementation of the Zcash protocol, currently under development. Please join us on Discord if you'd like to find out more or get involved!
Alpha Releases
Every few weeks, we release a new Zebra alpha release.
The goals of the alpha release series are to:
- participate in the Zcash network,
- replicate the Zcash chain state,
- implement the Zcash proof of work consensus rules, and
- sync on Mainnet under excellent network conditions.
Currently, Zebra does not validate all the Zcash consensus rules. It may be unreliable on Testnet, and under less-than-perfect network conditions. See our current features and roadmap for details.
Getting Started
Building zebrad requires Rust,
libclang, and a C++ compiler.
Detailed Build and Run Instructions
- Install
cargoandrustc.- Using
rustupinstalls the stable Rust toolchain, whichzebradtargets.
- Using
- Install Zebra's build dependencies:
- libclang: the
libclang,libclang-dev,llvm, orllvm-devpackages, depending on your package manager - clang or another C++ compiler:
g++,Xcode, orMSVC
- libclang: the
- Run
cargo install --locked --git https://github.com/ZcashFoundation/zebra --tag v1.0.0-alpha.14 zebrad - Run
zebrad start
If you're interested in testing out zebrad please feel free, but keep in mind
that there is a lot of key functionality still missing.
Build Troubleshooting
If you're having trouble with:
- dependencies:
- install both
libclangandclang- they are usually different packages - use
cargo installwithout--lockedto build with the latest versions of each dependency
- install both
- libclang: check out the clang-sys documentation
- g++ or MSVC++: try using clang or Xcode instead
- rustc: use rustc 1.48 or later
- Zebra does not have a minimum supported Rust version (MSRV) policy yet
System Requirements
We usually build zebrad on systems with:
- 2+ CPU cores
- 7+ GB RAM
- 14+ GB of disk space
On many-core machines (like, 32-core) the build is very fast; on 2-core machines it's less fast.
We continuously test that our builds and tests pass on:
- Windows Server 2019
- macOS Big Sur 11.0
- Ubuntu 18.04 / the latest LTS
- Debian Buster
We usually run zebrad on systems with:
- 4+ CPU cores
- 16+ GB RAM
- 50GB+ available disk space for finalized state
- 100+ Mbps network connections
zebrad might build and run fine on smaller and slower systems - we haven't
tested its exact limits yet.
Network Ports and Data Usage
By default, Zebra uses the following inbound TCP listener ports:
- 8233 on Mainnet
- 18233 on Testnet
If Zebra is configured with a specific listen_addr,
it will advertise this address to other nodes for inbound connections.
Zebra makes outbound connections to peers on any port.
But zcashd prefers peers on the default ports,
so that it can't be used for DDoS attacks on other networks.
zebrad's typical network usage is:
- initial sync: 30 GB download
- ongoing updates: 10-50 MB upload and download per day, depending on peer requests
The major constraint we've found on zebrad performance is the network weather,
especially the ability to make good connections to other Zcash network peers.
Current Features
Network:
- synchronize the chain from peers
- download gossiped blocks from peers
- answer inbound peer requests for hashes, headers, and blocks
State:
- persist block, transaction, UTXO, and nullifier indexes
- handle chain reorganizations
Proof of Work:
- validate equihash, block difficulty threshold, and difficulty adjustment
- validate transaction merkle roots
Validating proof of work increases the cost of creating a consensus split
between zebrad and zcashd.
This release also implements some other Zcash consensus rules, to check that Zebra's validation architecture supports future work on a full validating node:
- block and transaction structure
- checkpoint-based verification up to and including Canopy activation
- transaction validation (incomplete)
- transaction cryptography (incomplete)
- transaction scripts (incomplete)
- batch verification (incomplete)
Dependencies
Zebra primarily depends on pure Rust crates, and some Rust/C++ crates:
Known Issues
There are a few bugs in Zebra that we're still working on fixing:
- In rare cases, Zebra panics on shutdown #1678
- Interrupt handler does not work when a blocking task is running #1351
- Zebra should eventually exit once the task finishes. Or you can forcibly terminate the process.
- Duplicate block errors #1372
- These errors can be ignored, unless they happen frequently
Zebra's state commits changes using database transactions. If you forcibly terminate it, or it panics, any incomplete changes will be rolled back the next time it starts.
Future Work
In 2021, we intend to finish validation, add RPC support, and add wallet integration. This phased approach allows us to test Zebra's independent implementation of the consensus rules, before asking users to entrust it with their funds.
Features:
- full consensus rule validation
- transaction mempool
- wallet functionality
- RPC functionality
Performance and Reliability:
- reliable syncing on Testnet
- reliable syncing under poor network conditions
- batch verification
- performance tuning
Documentation
The Zebra website contains user documentation, such as how to run or configure Zebra, set up metrics integrations, etc., as well as developer documentation, such as design documents. We also render API documentation for the external API of our crates, as well as internal documentation for private APIs.
Architecture
Unlike zcashd, which originated as a Bitcoin Core fork and inherited its
monolithic architecture, Zebra has a modular, library-first design, with the
intent that each component can be independently reused outside of the zebrad
full node. For instance, the zebra-network crate containing the network stack
can also be used to implement anonymous transaction relay, network crawlers, or
other functionality, without requiring a full node.
At a high level, the fullnode functionality required by zebrad is factored
into several components:
-
zebra-chain, providing definitions of core data structures for Zcash, such as blocks, transactions, addresses, etc., and related functionality. It also contains the implementation of the consensus-critical serialization formats used in Zcash. The data structures inzebra-chainare defined to enforce structural validity by making invalid states unrepresentable. For instance, theTransactionenum has variants for each transaction version, and it's impossible to construct a transaction with, e.g., spend or output descriptions but no binding signature, or, e.g., a version 2 (Sprout) transaction with Sapling proofs. Currently,zebra-chainis oriented towards verifying transactions, but will be extended to support creating them in the future. -
zebra-network, providing an asynchronous, multithreaded implementation of the Zcash network protocol inherited from Bitcoin. In contrast tozcashd, each peer connection has a separate state machine, and the crate translates the external network protocol into a stateless, request/response-oriented protocol for internal use. The crate provides two interfaces:- an auto-managed connection pool that load-balances local node requests over available peers, and sends peer requests to a local inbound service, and
- a
connect_isolatedmethod that produces a peer connection completely isolated from all other node state. This can be used, for instance, to safely relay data over Tor, without revealing distinguishing information.
-
zebra-scriptprovides script validation. Currently, this is implemented by linking to the C++ script verification code fromzcashd, but in the future we may implement a pure-Rust script implementation. -
zebra-consensusperforms semantic validation of blocks and transactions: all consensus rules that can be checked independently of the chain state, such as verification of signatures, proofs, and scripts. Internally, the library usestower-batchto perform automatic, transparent batch processing of contemporaneous verification requests. -
zebra-stateis responsible for storing, updating, and querying the chain state. The state service is responsible for contextual verification: all consensus rules that check whether a new block is a valid extension of an existing chain, such as updating the nullifier set or checking that transaction inputs remain unspent. -
zebradcontains the full node, which connects these components together and implements logic to handle inbound requests from peers and the chain sync process. -
zebra-rpcandzebra-clientwill eventually contain the RPC and wallet functionality, but as mentioned above, our goal is to implement replication of chain state first before asking users to entrust Zebra with their funds.
All of these components can be reused as independent libraries, and all communication between stateful components is handled internally by internal asynchronous RPC abstraction ("microservices in one process").
Security
Zebra has a responsible disclosure policy, which we encourage security researchers to follow.
License
Zebra is distributed under the terms of both the MIT license and the Apache License (Version 2.0).
See LICENSE-APACHE and LICENSE-MIT.