2019-10-21 11:04:28 -07:00
|
|
|
Zebra Design Document
|
|
|
|
=====================
|
|
|
|
|
|
|
|
This document sketches the future design for Zebra.
|
|
|
|
|
|
|
|
Desiderata
|
|
|
|
==========
|
|
|
|
|
|
|
|
The following are general desiderata for Zebra:
|
|
|
|
|
|
|
|
* [George's list..]
|
|
|
|
|
|
|
|
* As much as reasonably possible, it and its dependencies should be
|
|
|
|
implemented in Rust. While it may not make sense to require this in
|
|
|
|
every case (for instance, it probably doesn't make sense to rewrite
|
|
|
|
libsecp256k1 in Rust, instead of using the same upstream library as
|
|
|
|
Bitcoin), we should generally aim for it.
|
|
|
|
|
|
|
|
* As much as reasonably possible, Zebra should minimize trust in
|
|
|
|
required dependencies. Note that "minimize number of dependencies"
|
|
|
|
is usually a proxy for this desideratum, but is not exactly the same:
|
|
|
|
for instance, a collection of crates like the tokio crates are all
|
|
|
|
developed together and have one trust boundary.
|
2020-06-02 16:16:17 -07:00
|
|
|
|
2019-10-21 11:04:28 -07:00
|
|
|
* Zebra should be well-factored internally into a collection of
|
|
|
|
component libraries which can be used by other applications to
|
|
|
|
perform Zcash-related tasks. Implementation details of each
|
2019-10-22 19:03:04 -07:00
|
|
|
component should not leak into all other components.
|
2020-06-02 16:16:17 -07:00
|
|
|
|
2019-10-21 11:04:28 -07:00
|
|
|
* Zebra should checkpoint on Sapling activation and drop all
|
|
|
|
Sprout-related functionality not required post-Sapling.
|
2020-06-02 16:16:17 -07:00
|
|
|
|
2020-06-03 15:24:52 -07:00
|
|
|
### Non-Goals
|
|
|
|
|
|
|
|
* Zebra keeps a copy of the chain state, so it isn't intended for
|
|
|
|
lightweight applications like light wallets. Those applications
|
|
|
|
should use a light client protocol.
|
|
|
|
|
2019-10-21 11:04:28 -07:00
|
|
|
Internal Structure
|
|
|
|
==================
|
|
|
|
|
|
|
|
The following is a list of internal component libraries (crates), and
|
|
|
|
a description of functional responsibility.
|
|
|
|
|
|
|
|
`zebra-chain`
|
|
|
|
--------------
|
|
|
|
|
|
|
|
### Internal Dependencies
|
|
|
|
|
2020-06-03 20:02:03 -07:00
|
|
|
None: these are the core data structure definitions.
|
2019-10-21 11:04:28 -07:00
|
|
|
|
|
|
|
### Responsible for
|
|
|
|
|
|
|
|
- definitions of commonly used data structures, e.g.,
|
|
|
|
- `Block`,
|
|
|
|
- `Transaction`,
|
|
|
|
- `Address`,
|
|
|
|
- `KeyPair`...
|
2020-06-07 21:19:32 -07:00
|
|
|
- parsing bytes into these data structures
|
2019-10-21 11:04:28 -07:00
|
|
|
|
|
|
|
- definitions of core traits, e.g.,
|
|
|
|
- `ZcashSerialize` and `ZcashDeserialize`, which perform
|
|
|
|
consensus-critical serialization logic.
|
|
|
|
|
|
|
|
### Exported types
|
|
|
|
|
|
|
|
- [...]
|
|
|
|
|
|
|
|
`zebra-network`
|
|
|
|
----------------
|
|
|
|
|
|
|
|
### Internal Dependencies
|
|
|
|
|
|
|
|
- `zebra-chain`
|
|
|
|
|
|
|
|
### Responsible for
|
|
|
|
|
2020-06-15 17:07:55 -07:00
|
|
|
- definition of a well structured, internal request/response protocol
|
2019-10-21 11:04:28 -07:00
|
|
|
- provides an abstraction for "this node" and "the network" using the
|
|
|
|
internal protocol
|
|
|
|
- dynamic, backpressure-driven peer set management
|
|
|
|
- per-peer state machine that translates the internal protocol to the
|
|
|
|
Bitcoin/Zcash protocol
|
|
|
|
- tokio codec for Bitcoin/Zcash message encoding.
|
|
|
|
|
|
|
|
### Exported types
|
|
|
|
|
|
|
|
- `Request`, an enum representing all possible requests in the internal protocol;
|
|
|
|
- `Response`, an enum representing all possible responses in the internal protocol;
|
|
|
|
- `AddressBook`, a data structure for storing peer addresses;
|
|
|
|
- `Config`, a configuration object for all networking-related parameters;
|
|
|
|
- `init<S: Service>(Config, S) -> (impl Service,
|
|
|
|
Arc<Mutex<AddressBook>>)`, the main entry-point.
|
|
|
|
|
|
|
|
The `init` entrypoint constructs a dynamically-sized pool of peers
|
|
|
|
sending inbound requests to the provided `S: tower::Service`
|
|
|
|
representing "this node", and returns a `Service` that can be used to
|
|
|
|
send requests to "the network", together with an `AddressBook` updated
|
|
|
|
with liveness information from the peer pool. The `AddressBook` can
|
|
|
|
be used to respond to inbound requests for peers.
|
|
|
|
|
|
|
|
All peerset management (finding new peers, creating new outbound
|
|
|
|
connections, etc) is completely encapsulated, as is responsibility for
|
|
|
|
routing outbound requests to appropriate peers.
|
|
|
|
|
2020-06-02 16:16:17 -07:00
|
|
|
`zebra-state`
|
2019-10-21 11:04:28 -07:00
|
|
|
----------------
|
|
|
|
|
|
|
|
### Internal Dependencies
|
|
|
|
|
|
|
|
- `zebra-chain` for data structure definitions.
|
|
|
|
|
|
|
|
### Responsible for
|
|
|
|
|
2020-06-07 21:21:25 -07:00
|
|
|
- block storage API
|
|
|
|
- operates on parsed block structs
|
2020-06-07 21:19:32 -07:00
|
|
|
- these structs can be converted from and into raw bytes
|
2019-10-21 11:04:28 -07:00
|
|
|
- primarily aimed at network replication, not at processing
|
|
|
|
- can be used to rebuild the database below
|
|
|
|
- maintaining a database of tx, address, etc data
|
|
|
|
- this database can be blown away and rebuilt from the blocks, which
|
|
|
|
are otherwise unused.
|
|
|
|
- threadsafe, typed lookup API that completely encapsulates the
|
|
|
|
database logic
|
|
|
|
- handles stuff like "transactions are reference counted by outputs"
|
|
|
|
etc.
|
|
|
|
- providing `tower::Service` interfaces for all of the above to
|
|
|
|
support backpressure.
|
|
|
|
|
|
|
|
### Exported types
|
|
|
|
|
2020-06-07 21:21:25 -07:00
|
|
|
- `Request`, an enum representing all possible requests in the internal protocol;
|
|
|
|
- blocks can be accessed via their chain height or hash
|
|
|
|
- confirmed transactions can be accessed via their block, or directly via their hash
|
|
|
|
- `Response`, an enum representing all possible responses in the internal protocol;
|
|
|
|
- `init() -> impl Service`, the main entry-point.
|
|
|
|
|
|
|
|
The `init` entrypoint returns a `Service` that can be used to
|
|
|
|
send requests for the chain state.
|
|
|
|
|
|
|
|
All state management (adding blocks, getting blocks by index or hash) is completely
|
|
|
|
encapsulated.
|
2019-10-21 11:04:28 -07:00
|
|
|
|
|
|
|
`zebra-script`
|
|
|
|
---------------
|
|
|
|
|
|
|
|
### Internal Dependencies
|
|
|
|
|
|
|
|
- ??? depends on how it's implemented internally
|
|
|
|
|
|
|
|
### Responsible for
|
|
|
|
|
|
|
|
- the minimal Bitcoin script implementation required for Zcash
|
2020-06-07 21:21:25 -07:00
|
|
|
- script parsing
|
|
|
|
- context-free script validation
|
2019-10-21 11:04:28 -07:00
|
|
|
|
|
|
|
### Notes
|
|
|
|
|
|
|
|
This can wrap an existing script implementation at the beginning.
|
|
|
|
|
|
|
|
If this existed in a "good" way, we could use it to implement tooling
|
|
|
|
for Zcash script inspection, debugging, etc.
|
|
|
|
|
|
|
|
### Questions
|
|
|
|
|
|
|
|
- How does this interact with NU4 script changes?
|
|
|
|
|
|
|
|
### Exported types
|
|
|
|
|
|
|
|
- [...]
|
|
|
|
|
|
|
|
`zebra-consensus`
|
|
|
|
------------------
|
|
|
|
|
|
|
|
### Internal Dependencies
|
|
|
|
|
2020-06-07 21:21:25 -07:00
|
|
|
- `zebra-chain` for data structures and parsing.
|
|
|
|
- `zebra-state` to read and update the state database.
|
|
|
|
- `zebra-script` for script parsing and validation.
|
2019-10-21 11:04:28 -07:00
|
|
|
|
|
|
|
### Responsible for
|
|
|
|
|
|
|
|
- consensus-specific parameters (network magics, genesis block, pow
|
|
|
|
parameters, etc) that determine the network consensus
|
|
|
|
- consensus logic to decide which block is the current block
|
2020-06-07 21:21:25 -07:00
|
|
|
- block and transaction verification
|
|
|
|
- context-free validation, e.g., signature, proof verification, etc.
|
|
|
|
- context-dependent validation, e.g., determining whether a
|
|
|
|
transaction is accepted in a particular chain state context.
|
|
|
|
- verifying mempool (unconfirmed) transactions
|
|
|
|
- block checkpoints
|
|
|
|
- mandatory checkpoints (genesis block, sapling activation)
|
|
|
|
- optional regular checkpoints (every Nth block)
|
|
|
|
- modifying the chain state
|
|
|
|
- adding new blocks to `ZebraState`, including chain reorganisation
|
|
|
|
- adding new transactions to `ZebraMempoolState`
|
|
|
|
- storing the transaction mempool state
|
|
|
|
- mempool transactions can be accessed via their hash
|
|
|
|
- providing `tower::Service` interfaces for all of the above to
|
|
|
|
support backpressure and batch validation.
|
2019-10-21 11:04:28 -07:00
|
|
|
|
|
|
|
### Exported types
|
|
|
|
|
2020-06-07 21:21:25 -07:00
|
|
|
- `block::init() -> impl Service`, the main entry-point for block
|
|
|
|
verification.
|
|
|
|
- `ZebraMempoolState`
|
|
|
|
- all state management (adding transactions, getting transactions
|
|
|
|
by hash) is completely encapsulated.
|
|
|
|
- `mempool::init() -> impl Service`, the main entry-point for
|
|
|
|
mempool transaction verification.
|
|
|
|
|
|
|
|
The `init` entrypoints return `Service`s that can be used to
|
|
|
|
verify blocks or transactions, and add them to the relevant state.
|
2019-10-21 11:04:28 -07:00
|
|
|
|
|
|
|
`zebra-rpc`
|
|
|
|
------------
|
|
|
|
|
|
|
|
### Internal Dependencies
|
|
|
|
|
|
|
|
- `zebra-chain` for data structure definitions
|
|
|
|
- `zebra-network` possibly? for definitions of network messages?
|
|
|
|
|
|
|
|
### Responsible for
|
|
|
|
|
|
|
|
- rpc interface
|
|
|
|
|
|
|
|
### Exported types
|
|
|
|
|
|
|
|
- [...]
|
|
|
|
|
|
|
|
`zebra-client`
|
|
|
|
-----------------
|
|
|
|
|
|
|
|
### Internal Dependencies
|
|
|
|
|
|
|
|
- `zebra-chain` for structure definitions
|
2020-06-02 16:16:17 -07:00
|
|
|
- `zebra-state` for transaction queries and client/wallet state storage
|
2019-10-21 11:04:28 -07:00
|
|
|
- `zebra-script` possibly? for constructing transactions
|
|
|
|
|
|
|
|
### Responsible for
|
|
|
|
|
|
|
|
- implementation of some event a user might trigger
|
2020-06-03 15:24:52 -07:00
|
|
|
- would be used to implement a full wallet
|
2019-10-21 11:04:28 -07:00
|
|
|
- create transactions, monitors shielded wallet state, etc.
|
|
|
|
|
2020-06-02 16:16:17 -07:00
|
|
|
### Notes
|
2019-10-21 11:04:28 -07:00
|
|
|
|
|
|
|
Communication between the client code and the rest of the node should be done
|
|
|
|
by a tower service interface. Since the `Service` trait can abstract from a
|
|
|
|
function call to RPC, this means that it will be possible for us to isolate
|
|
|
|
all client code to a subprocess.
|
|
|
|
|
|
|
|
### Exported types
|
|
|
|
|
|
|
|
- [...]
|
|
|
|
|
2019-10-22 19:03:04 -07:00
|
|
|
`zebrad`
|
2019-10-21 11:04:28 -07:00
|
|
|
---------
|
|
|
|
|
|
|
|
Abscissa-based application which loads configs, all application components,
|
|
|
|
and connects them to each other.
|
|
|
|
|
|
|
|
### Responsible for
|
|
|
|
|
|
|
|
- actually running the server
|
|
|
|
- connecting functionality in dependencies
|
|
|
|
|
|
|
|
### Internal Dependencies
|
|
|
|
|
|
|
|
- `zebra-chain`
|
|
|
|
- `zebra-network`
|
2020-06-02 16:16:17 -07:00
|
|
|
- `zebra-state`
|
2019-10-21 11:04:28 -07:00
|
|
|
- `zebra-consensus`
|
|
|
|
- `zebra-client`
|
|
|
|
- `zebra-rpc`
|
|
|
|
|
|
|
|
Unassigned functionality
|
|
|
|
------------------------
|
|
|
|
|
|
|
|
Responsibility for this functionality needs to be assigned to one of
|
|
|
|
the modules above (subject to discussion):
|
|
|
|
|
|
|
|
- [ ... add to this list ... ]
|