zips/zip-0221.rst

740 lines
29 KiB
ReStructuredText
Raw Normal View History

2019-03-31 00:57:20 -07:00
::
ZIP: 0221
2020-02-27 08:33:38 -08:00
Title: FlyClient Zcash SPV
Owners: Jack Grigg <jack@electriccoin.co>
Original-Authors: Ying Tong <yingtong@ethereum.org>
James Prestwich <james@summa.one>
Georgios Konstantopoulos <me@gakonst.com>
2019-03-31 00:57:20 -07:00
Status: Draft
Category: Consensus
Created: 2019-03-30
License: MIT
2019-03-31 00:57:20 -07:00
Terminology
===========
2020-02-27 08:33:00 -08:00
The key words "**MUST**", "**SHOULD**", and "**MAY**" in this document are to be
interpreted as described in RFC 2119. [#RFC2119]_
2020-02-27 08:33:00 -08:00
The terms "branch" and "network upgrade" in this document are to be interpreted as
described in ZIP 200. [#zip-0200]_
2019-03-31 00:57:20 -07:00
*Light client*
2020-02-27 08:33:00 -08:00
A client that is not a full participant in the network of Zcash peers. It can send and
receive payments, but does not store or validate a copy of the blockchain.
2019-03-31 00:57:20 -07:00
*High probability*
2020-02-27 08:33:00 -08:00
An event occurs with high probability if it occurs with probability 1-O(1/2^λ), where λ
is a security parameter.
2019-03-31 00:57:20 -07:00
*Negligible probability*
2020-02-27 08:33:00 -08:00
An event occurs with negligible probability if it occurs with probability O(1/2^λ),
where λ is the security parameter.
2019-03-31 00:57:20 -07:00
*Merkle mountain range (MMR)*
2020-02-27 08:55:35 -08:00
A Merkle mountain range (MMR) is a binary hash tree that allows for efficient appends of
2020-02-27 08:33:00 -08:00
new leaves without changing the value of existing nodes.
2019-03-31 00:57:20 -07:00
2019-03-31 00:57:20 -07:00
Abstract
========
2020-02-27 08:33:00 -08:00
This ZIP specifies modifications to be made to the Zcash block header format to include
Merkle Mountain Range (MMR) commitments. Sapling (NU2) introduced the
``hashFinalSaplingRoot`` field to Zcash headers. This ZIP replaces the
``hashFinalSaplingRoot`` commitment with ``hashChainHistoryRoot``, which is the root of an
MMR that commits to many features of the chain's history, including all information
present in ``hashFinalSaplingRoot``.
The MMR that produces the root provides a number of benefits to light clients, and enables
future specification of the FlyClient protocol [#FlyClient]_. This ZIP specifies only
consensus-layer changes. It does not provide any specification about the FlyClient
protocol's operation.
Background
==========
2020-02-27 08:33:00 -08:00
An MMR is a Merkle tree which allows for efficient appends, proofs, and verifications.
Informally, appending data to an MMR consists of creating a new leaf and then iteratively
merging neighboring subtrees with the same size. This takes at most `log(n)` operations
and only requires knowledge of the previous subtree roots, of which there are fewer than
`log(n)`.
(example adapted from [#mimblewimble]_)
2020-02-27 08:33:00 -08:00
To illustrate this, consider a list of 11 leaves. We first construct the biggest perfect
binary subtrees possible by joining any balanced sibling trees that are the same size. We
do this starting from the left to the right, adding a parent as soon as 2 children exist.
This leaves us with three subtrees ("mountains") of heights 3, 1, and 0:
.. code-block:: C
/\
/ \
/\ /\
/\/\/\/\ /\ /
2020-02-27 08:33:00 -08:00
Note that the first leftmost peak is always the highest. We can number this structure in
order of insertion:
.. code-block:: C
Height
3 14
/ \
/ \
/ \
/ \
2 6 13
/ \ / \
1 2 5 9 12 17
/ \ / \ / \ / \ / \ /
0 0 1 3 4 7 8 10 11 15 16 18
and represent this numbering in a flat list:
.. code-block:: python
Position 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Height 0 0 1 0 0 1 2 0 0 1 0 0 1 2 3 0 0 1 0
2020-02-27 08:33:00 -08:00
This allows us to easily jump to the right sibling of a node by adding ``2^(h+1) - 1`` to
its position, and its left sibling by subtracting ``2^h``. This allows us to efficiently
find the subtree roots ("peaks") of the mountains.
2020-02-27 08:33:00 -08:00
Once we have the positions of the mountain peaks, we "bag" them using the following
algorithm:
1. add a peak connecting the 2 left-most peaks
2. repeat 1. until we have a single peak
.. code-block:: C
Height
5 20
/ \
4 19 \
/ \ \
/ \ \
/ \ \
3 14 \ \
/ \ \ \
/ \ \ \
/ \ \ \
/ \ \ \
2 6 13 \ \
/ \ / \ \ \
1 2 5 9 12 17 \
/ \ / \ / \ / \ / \ \
0 0 1 3 4 7 8 10 11 15 16 18
2020-02-27 08:55:35 -08:00
MMR trees allow for efficient incremental set update operations (push, pop, prune). In
2020-02-27 08:33:00 -08:00
addition, MMR update operations and Merkle proofs for recent additions to the leaf set are
more efficient than other incremental Merkle tree implementations (e.g. Bitcoin's padded
2020-02-27 08:33:38 -08:00
leafset, sparse Merkle trees, and Zcash's incremental note commitment trees).
2019-03-31 00:57:20 -07:00
2019-03-31 00:57:20 -07:00
Motivation
==========
2020-02-27 08:33:00 -08:00
MMR proofs are used in the FlyClient protocol to reduce the proof size needed for light
clients to verify
- the validity of a blockchain received from a full node, and
- the inclusion of a block ``B`` in that chain, and
- certain metadata of any block or range of blocks in that chain
2020-02-27 08:33:00 -08:00
The protocol requires that an MMR that commits to the inclusion of all blocks since the
most recent network upgrade (``B_x, ..., B_(n-1))`` is formed for each block ``B_n``. The
root ``M_n`` of the MMR MUST be included in the header of ``B_n``.
(``x`` is the activation height of the most recent upgrade network upgrade)
2020-02-27 08:33:00 -08:00
FlyClient reduces the number of block headers needed for light client verification of a
valid chain, from linear (as in the current reference protocol) to logarithmic in
blockchain length. This verification is correct with high probability. It also allows
creation of subtree proofs, so light clients need only check blocks later than the most
recently verified block index. Following that, verification of a transaction inclusion
within that block follows the usual reference protocol [#ZIP-0307]_.
2020-02-27 08:33:00 -08:00
A smaller proof size could enable the verification of Zcash SPV Proofs in blockchains such
as Ethereum, enabling efficient cross-chain communication and pegs. It also reduces
bandwidth and storage requirements for resource-limited clients like mobile or IoT
devices.
Security and Privacy Considerations
===================================
2020-02-27 08:33:00 -08:00
This ZIP imposes an additional validation cost on new blocks. While this validation cost
is small, it may exacerbate any existing DoS opportunities, particularly during abnormal
events like long reorgs. Fortunately, these costs are logarithmic in the number of delete
and append operations. In the worst case scenario, a well-resourced attacker could
maintain 2 chains of approximately equal length, and alternate which chain they extend.
This would result in repeated reorgs of increasing length.
Given the performance of Blake2b, we expect this validation cost to be negligible.
However, it seems prudent to benchmark potential MMR implementations during the
implementation process. Should the validation cost be higher than expected, there are
several potential mitigations, e.g. holding recently seen nodes in memory after a reorg.
Generally, header commitments have no impact on privacy. However, FlyClient has additional
security and privacy implications. Because FlyClient is a motivating factor for this ZIP,
it seems prudent to include a brief overview. A more in-depth security analysis of
FlyClient should be performed before designing a FlyClient-based light client ecosystem
2020-02-27 08:33:38 -08:00
for Zcash.
2020-02-27 08:33:00 -08:00
FlyClient, like all light clients, requires a connection to a light client server. That
server may collect information about client requests, and may use that information to
attempt to deanonymize clients. However, because FlyClient proofs are non-interactive and
publicly verifiable, they could be shared among many light clients after the initial
server interaction.
FlyClient proofs are probabilistic. When properly constructed, there is negligible
probability that a dishonest chain commitment will be accepted by the verifier. The
security analysis assumes adversary mining power is bounded by a known fraction of
combined mining power of honest nodes, and cannot drop or tamper with messages between
client and full nodes. It also assumes the client is connected to at least one full node
and knows the genesis block. However, these security properties have not been examined
closely in chain models with rapidly-adjusting difficulty.
2019-03-31 00:57:20 -07:00
Specification
=============
.. image:: https://i.imgur.com/hhRyI99.png
:alt: zcash_MMR
*Fig 1. MMR commitment scheme*
2020-02-27 08:33:00 -08:00
The leaves of the MMR at block ``B_n`` are hash commitments to the header data and
metadata of each previous block ``B_x, ..., B_(n-1)``, where ``x`` is the block height of
the most recent network upgrade. We extend the standard MMR to allow metadata to propagate
upwards through the tree by either summing the metadata of both children, or inheriting
the metadata of a specific child as necessary. This allows us to create efficient proofs
of selected properties of a range of blocks without transmitting the entire range of
blocks or headers.
Tree Node specification
-----------------------
2020-02-27 08:33:00 -08:00
Unless otherwise noted, all hashes use BLAKE2b-256 with the personalization field set to
``'ZcashHistory' || CONSENSUS_BRANCH_ID``. ``CONSENSUS_BRANCH_ID`` is the little-endian
encoding of ``BRANCH_ID`` for the epoch of the block containing the commitment.
[#zip-0200]_ Which is to say, each node in the tree commits to the consensus branch that
produced it.
.. image:: https://i.imgur.com/9Ct2llE.png
:alt: zcash_MMR_hash
*Fig 2. Hashing MMR leaf nodes and internal nodes*
Each MMR node is defined as follows:
1. ``hashSubtreeCommitment``
2020-02-27 08:33:00 -08:00
- If the node is a leaf node, then ``hashSubtreeCommitment`` is the consensus-defined
block hash for the corresponding block.
2020-02-27 08:33:00 -08:00
* This hash is encoded in internal byte order, and does NOT use the BLAKE2b-256
personalization string described above.
* For clarity, the ``hashSubtreeCommitment`` field of leaf ``n-1`` is *precisely equal*
to the ``hashPrevBlock`` field of header ``n``
- If the node is an internal or root node
* Both child nodes are serialized
* ``hashSubtreeCommitment`` is the BLAKE2b-256 hash of ``left_child || right_child``
* For clarity, this digest uses the BLAKE2b-256 personalization string described above.
- serialized as ``char[32]``
2019-03-31 00:57:20 -07:00
2. ``nEarliestTimestamp``
2019-03-31 00:57:20 -07:00
- If the node is a leaf node
* ``nEarliestTimestamp`` is the header's timestamp
- If the node is an internal or root node
* ``nEarliestTimestamp`` is inherited from the left child
- serialized as ``nTime`` (``uint32``)
2019-03-31 00:57:20 -07:00
3. ``nLatestTimestamp``
2019-03-31 00:57:20 -07:00
- If the node is a leaf node
2019-03-31 00:57:20 -07:00
* ``nLatestTimestamp`` is the header's timestamp
- If the node is an internal or root node
2020-02-27 08:55:35 -08:00
* ``nLatestTimestamp`` is inherited from the right child
2020-02-27 08:33:00 -08:00
- Note that due to timestamp consensus rules, ``nLatestTimestamp`` may be smaller than
``nEarliestTimestamp`` in some subtrees. This may occur within subtrees smaller than
``PoWMedianBlockSpan`` blocks.
- serialized as ``nTime`` (``uint32``)
4. ``nEarliestTarget``
2019-03-31 00:57:20 -07:00
- If the node is a leaf node
2019-03-31 00:57:20 -07:00
* ``nEarliestTarget`` is the header's ``nBits`` field
- If the node is an internal or root node
2020-02-27 08:55:35 -08:00
* ``nEarliestTarget`` is inherited from the left child
- serialized as ``nBits`` (``uint32``)
5. ``nLatestTarget``
2019-03-31 00:57:20 -07:00
- If the node is a leaf node
2019-03-31 00:57:20 -07:00
* ``nLatestTarget`` is the header's ``nBits`` field
- If the node is an internal or root node
2020-02-27 08:55:35 -08:00
* ``nLatestTarget`` is inherited from the right child
- serialized as ``nBits`` (uint32)
6. ``hashEarliestSaplingRoot``
- If the node is a leaf node
2019-03-31 00:57:20 -07:00
2020-02-27 08:33:00 -08:00
* ``hashEarliestSaplingRoot`` is calculated as ``hashFinalSaplingRoot``, as implemented
in Sapling
2019-03-31 00:57:20 -07:00
- If the node is an internal or root node
* ``hashEarliestSaplingRoot`` is inherited from the left child
- serialized as ``char[32]``
7. ``hashLatestSaplingRoot``
2019-03-31 00:57:20 -07:00
- If the node is a leaf node
2020-02-27 08:33:00 -08:00
* ``hashLatestSaplingRoot`` is calculated as ``hashFinalSaplingRoot``, as implemented in
Sapling
- If the node is an internal or root node
* ``hashLatestSaplingRoot`` is inherited from the right child
- serialized as ``char[32]``
8. ``nSubTreeTotalWork``
- If the node is a leaf node
2020-02-27 08:33:00 -08:00
* ``nSubTreeTotalWork`` is the protocol-defined work of the block:
`floor(2 ** 256 / (ToTarget(nBits) + 1))`. [#block-work]_
- If the node is an internal or root node
* ``nSubTreeTotalWork`` is the sum of the ``nSubTreeTotalWork`` fields of both children
- serialized as ``uint256``
9. ``nEarliestHeight``
- If the node is a leaf node
* ``nEarliestHeight`` is the header's height
- If the node is an internal or root node
2020-02-27 08:55:35 -08:00
* ``nEarliestHeight`` is inherited from the left child
- serialized as ``CompactSize uint``
10. ``nLatestHeight``
- If the node is a leaf node
* ``nLatestHeight`` the header's height
- If the node is an internal or root node
* ``nLatestHeight`` is inherited from the right child
* serialized as ``CompactSize uint``
11. ``nShieldedTxCount``
- If the node is a leaf node
2020-02-27 08:33:00 -08:00
* ``nShieldedTxCount`` is the number of transactions in the leaf block where any of
``vJoinSplit``, ``vShieldedSpend``, or `vShieldedOutput` is non-empty
- If the node is an internal or root node
* ``nShieldedTxCount`` is the sum of the ``nShieldedTxCount`` field of both children
- serialized as ``CompactSize uint``
2020-02-27 08:33:00 -08:00
Each node, when serialized, is between 147 and 171 bytes long. The canonical serialized
representation of a node is used whenever creating child commitments for future nodes.
Other than the metadata commitments, the MMR tree's construction is standard.
2020-02-27 08:33:00 -08:00
Once the MMR has been generated, we produce ``hashChainHistoryRoot``, which we define as
the BLAKE2b-256 digest of the serialization of the root node.
Tree nodes and hashing (pseudocode)
-----------------------------------
.. code-block:: python
CONSENSUS_BRANCH_ID: bytes = b''
def H(msg: bytes) -> bytes:
return blake2b256(msg, personalization=b'ZcashHistory' + CONSENSUS_BRANCH_ID)
class ZcashMMRNode():
# leaf nodes have no children
left_child: 'Optional[ZcashMMRNode]'
right_child: 'Optional[ZcashMMRNode]'
# commitments
hashSubtreeCommitment: bytes
nEarliestTimestamp: int
nLatestTimestamp: int
nEarliestTarget: int
nLatestTarget: int
hashEarliestSaplingRoot: bytes # left child's sapling root
hashLatestSaplingRoot: bytes # right child's sapling root
nSubTreeTotalWork: int # total difficulty accumulated within each subtree
nEarliestHeight: int
nLatestHeight: int
nShieldedTxCount: int # number of shielded transactions in block
@classmethod
def from_block(Z, block: ZcashBlock) -> 'ZcashMMRNode':
'''Create a leaf node from a block'''
return Z(
left_child=None,
right_child=None,
hashSubtreeCommitment=block.header_hash,
nEarliestTimestamp=block.timestamp,
nLatestTimestamp=block.timestamp,
nEarliestTarget=block.nBits,
nLatestTarget=block.nBits,
hashEarliestSaplingRoot=block.sapling_root,
hashLatestSaplingRoot=block.sapling_root,
nSubTreeTotalWork=calculate_work(block.nBits),
nEarliestHeight=block.height,
nLatestHeight=block.height,
nShieldedTxCount=block_shielded_tx_count)
def serialize(self) -> bytes:
'''serializes a node'''
return (
self.hashSubtreeCommitment
+ serialize_uint32(self.nEarliestTimestamp)
+ serialize_uint32(self.nLatestTimestamp)
+ serialize_uint32(self.nEarliestTarget)
+ serialize_uint32(self.nLatestTarget)
+ hashEarliestSaplingRoot
+ hashLatestSaplingRoot
+ serialize_uint256(self.nSubTreeTotalWork)
+ serialize_compact_uint(self.nEarliestHeight)
+ serialize_compact_uint(self.nLatestHeight)
+ serialize_compact_uint(self.nShieldedTxCount))
def make_parent(
left_child: ZcashMMRNode,
right_child: ZcashMMRNode) -> ZcashMMRNode:
return ZcashMMRNode(
left_child=left_child,
right_child=right_child,
hashSubtreeCommitment=H(left_child.serialize() + right_child.serialize()),
nEarliestTimestamp=left_child.nEarliestTimestamp,
nLatestTimestamp=right_child.nLatestTimestamp,
nEarliestTarget=left_child.nEarliestTarget,
nLatestTarget=left_child.nLatestTarget,
hashEarliestSaplingRoot=left_child.sapling_root,
hashLatestSaplingRoot=right_child.sapling_root,
nSubTreeTotalWork=left_child.nSubTreeTotalWork + right_child.nSubTreeTotalWork,
nEarliestHeight=left_child.nEarliestHeight,
nLatestHeight=right_child.nLatestHeight,
nShieldedTxCount=left_child.count_shield + right_child.count_shield)
def make_root_commitment(root: ZcashMMRNode) -> bytes:
'''Makes the root commitment for a blockheader'''
return H(root.serialize())
Incremental push and pop (pseudocode)
-------------------------------------
2020-02-27 08:33:00 -08:00
With each new block ``B_n``, we append a new MMR leaf node corresponding to block
``B_(n-1)``. The ``append`` operation is detailed below in pseudocode (adapted from
[#FlyClient]_):
.. code-block:: python
def get_peaks(node: ZcashMMRNode) -> List[ZcashMMRNode]:
peaks: List[ZcashMMRNode] = []
left_child = node.left_child
right_child = node.right_child
# find the number of leaves in the subtree
left_leaves = left_child.latest_height - left_child.earliest_height + 1
right_leaves = right_child.latest_height - right_child.earliest_height + 1
if (left_leaves & (left_leaves - 1)) == 0:
peaks.append(left_child)
else:
peaks.extend(get_peaks(left_child))
if (right_leaves & (right_leaves - 1)) == 0:
peaks.append(right_child)
else:
peaks.extend(get_peaks(right_child))
return peaks
def append(root: ZcashMMRNode, leaf: ZcashMMRNode) -> ZcashMMRNode:
'''Append a leaf to an existing tree, return the new tree root'''
# recursively find a list of peaks in the current tree
peaks: List[ZcashMMRNode] = get_peaks(root)
merged: List[ZcashMMRNode] = []
# Merge peaks from right to left.
# This will produce a list of peaks in reverse order
current = leaf
for peak in peaks[::-1]:
current_leaves = current.latest_height - current.earliest_height + 1
peak_leaves = peak.latest_height - peak.earliest_height + 1
if current_leaves == peak_leaves:
current = make_parent(peak, current)
else:
merged.append(current)
current = peak
# finally, bag the merged peaks
return bag_peaks(merged[::-1])
2020-02-27 08:33:00 -08:00
In case of a block reorg, we have to delete the latest (i.e. rightmost) MMR leaf nodes, up
to the reorg length. This operation is ``O(log(k))`` where ``k`` is the number of leaves
in the right subtree of the MMR root.
.. code-block:: python
def delete(root: ZcashMMRNode) -> ZcashMMRNode:
'''
Delete the rightmost leaf node from an existing MMR
Return the new tree root
'''
n_leaves = root.latest_height - root.earliest_height + 1
# if there were an odd number of leaves,
# simply replace root with left_child
if n_leaves & 1:
return root.left_child
# otherwise, we need to re-bag the peaks.
else:
# first peak
peaks = [root.left_child]
# we do this traversing the right (unbalanced) side of the tree
# we keep the left side (balanced subtree or leaf) of each subtree
# until we reach a leaf
subtree_root = root.right_child
while subtree_root.left_child:
peaks.push(tmp_root.left_child)
subtree_root = tmp_root.right_child
new_root = bag_peaks(peaks)
return new_root
def bag_peaks(peaks: List[ZcashMMRNode]) -> ZcashMMRNode:
'''
"Bag" a list of peaks, and return the final root
'''
root = peaks[0]
for i in range(1, len(peaks)):
root = make_parent(root, peaks[i])
return root
Header modifications specification
----------------------------------
2020-02-27 08:33:00 -08:00
This ZIP introduces a new header version. It is identical to the current v4 header
[#zcashBlock]_, except for the following changes:
1. The version number is changed to `5`.
2020-02-27 08:55:35 -08:00
2. ``hashFinalSaplingRoot`` is replaced by ``hashChainHistoryRoot``, as described above.
The new block header format is:
.. code-block:: C
class CBlockHeader
{
public:
// header
static const size_t HEADER_SIZE=4+32+32+32+4+4+32; // excluding Equihash solution
static const int32_t CURRENT_VERSION=5;
int32_t nVersion;
uint256 hashPrevBlock;
uint256 hashMerkleRoot;
uint256 hashChainHistoryRoot;
uint32_t nTime;
uint32_t nBits;
uint256 nNonce;
std::vector<unsigned char> nSolution;
...
}
2019-03-31 00:57:20 -07:00
Rationale
=========
2019-03-31 00:57:20 -07:00
Tree nodes
----------
2019-03-31 00:57:20 -07:00
2020-02-27 08:33:00 -08:00
Nodes in the commitment tree are canonical and immutable. They are cheap to generate, as
(with the exception of ``nShieldedTxCount``) all metadata is already generated during
block construction and/or checked during block validation. Nodes are relatively compact in
memory. Approximately 140,000 blocks have elapsed since Sapling activation. Assuming a 164
byte commitment to each of these, we would have generated approximately 24 MB of
additional storage cost for the set of leaf nodes (and an additional ~24 MB for storage of
intermediate nodes).
``hashSubtreeCommitment`` forms the strucuture of the commitment tree. Other metadata
commitments were chosen to serve specific purposes. Variable-length commitments are placed
last, so that most metadata in a node can be directly indexed. We considered using
fixed-length commitments here, but opted for variable-length, in order to marginally
reduce the memory requirements for managing and updating the commitment trees.
In leaf nodes, some information is repeated. We chose to do this so that leaf nodes could
be treated identically to internal and root nodes for all algorithms and (de)serializers.
Leaf nodes are easily identifiable, as they will show proof of work in the
``hashSubtreeCommitment`` field, and their block range (calculated as
``nLatestHeight - nEarliestHeight + 1``) will be precisely 1. For the same reason, we
change the semantics of ``hashSubtreeCommitment`` in leaf nodes to commit
2020-02-27 08:33:38 -08:00
Personalized BLAKE2b-256 was selected to match existing Zcash conventions. Adding the
2020-02-27 08:33:00 -08:00
consensus branch ID to the hash personalization string ensures that valid nodes from one
branch cannot be used to make false statements about parallel consensus branches.
FlyClient Requirements and Recommendations
``````````````````````````````````````````
2020-02-27 08:33:00 -08:00
These commitments enable FlyClient in the variable-difficulty model. Specifically, they
allow light clients to reason about application of the difficulty adjustment algorithm
over a range of blocks. They were chosen via discussion with an author of the FlyClient
paper.
2019-03-31 00:57:20 -07:00
- ``nEarliestTimestamp``
- ``nLatestTimestamp``
- ``nEarliestTarget``
- ``nLatestTarget``
- ``nEarliestHeight``
- ``nLatestHeight``
- ``nSubTreeTotalWork``
2019-03-31 00:57:20 -07:00
Non-FlyClient Commitments
`````````````````````````
2020-02-27 08:33:00 -08:00
Additional metadata commitments were chosen primarily to improve light client security
guarantees. We specified commitments where we could see an obvious security benefit, but
there may be other useful metadata that we missed. We're interested in feedback and
suggestions from the implementers of the current light client.
2019-03-31 00:57:20 -07:00
2020-02-27 08:33:00 -08:00
We considered adding a commitment to the nullifier vector at each block. We would
appreciate comments from light client teams on the utility of this commitment, as well as
the proper serialization and commitment format for the nullifier vector.
- ``hashEarliestSaplingRoot``
2020-02-27 08:33:00 -08:00
* Committing to the earliest Sapling root of a range of blocks allows light clients to
check the consistency of treestate transitions over a range of blocks, without
recalculating the root from genesis.
- ``hashLatestSaplingRoot``
2020-02-27 08:33:00 -08:00
* This commitment serves the same purpose as ``hashFinalSaplingRoot`` in current Sapling
semantics.
* However, because the MMR tree commits to blocks ``B_x ... B_(n-1)``, the latest
2020-02-27 08:55:35 -08:00
commitment will describe the final treestate of the previous block, rather than the
2020-02-27 08:33:00 -08:00
current block.
* Concretely: block 500 currently commits to the final treestate of block 500 in its
header. With this ZIP, block 500 will commit to all roots up to block 499, but not the
final root of block 500.
* We feel this is an acceptable tradeoff. Using the most recent treestate as a
transaction anchor is already unsafe in reorgs. Clients should never use the most
recent treestate to generate transactions, so it is acceptable to delay commitment by
one block.
- ``nShieldedTxCount``
2020-02-27 08:33:00 -08:00
* By committing to the number of shielded transactions in blocks (and ranges of blocks),
a light client may reliably learn whether a malicious server is witholding any
shielded transactions.
* In addition, this commitment allows light clients to avoid syncing header ranges that
do not contain shielded transactions. As the primary cost of a light client is
transmission of equihash solution information in block headers, this optimization
would significantly decrease the bandwidth requirements of light clients.
Header Format Change
--------------------
2020-02-27 08:33:00 -08:00
Our primary goal was to minimize header changes. The version number is incremented to
signify the change in field semantics. This is not strictly necessary. Old light client
parsers will generally not reject the new header semantics and we expect full nodes to
follow the network upgrade. It may be the case that mining related hardware or software
has (unwisely) hardcoded the version to 4. In which case, we would recommend not changing
the header version number.
We considered adding ``hashChainHistoryRoot`` to the header as a new field. We decided
against that, as it will inherently affect more of the ecosystem. As stated earlier, we
would prefer not to introduce changes that could affect mining hardware or embedded
software.
We also considered putting ``hashChainHistoryRoot`` in the ``hashPrevBlock`` field as it
commits to the entire chain history, but quickly realized it would require massive
refactoring of the existing code base and would negatively impact performance. Reorgs in
particular are fragile, performance-critical, and rely on backwards iteration over the
chain history. If a chain were to be designed from scratch there may be some efficient
implementation that would join these commitments, but it is clearly not appropriate for
2020-02-27 08:33:38 -08:00
Zcash as it exists.
Additional Reading
==================
- `Bitcoin difficulty calculation <https://en.bitcoin.it/wiki/Difficulty>`_
- `Flyclient enabled geth fork by FlyClient authors <https://github.com/mahdiz/flyeth>`_
- `ECIP-1055: Succinct PoW Using Merkle Mountain Ranges <https://github.com/etclabscore/ECIPs/pull/11/files?short_path=44c106e#diff-44c106ea0ef54fab09596596934d3d15>`_
- `Grin project MMR implementation in Rust <https://github.com/mimblewimble/grin/tree/milestone/2.0.0/core/src/core>`_
- `Tari Project MMR implementation in Rust <https://github.com/tari-project/tari/tree/development/infrastructure/merklemountainrange>`_
- `Beam Project MMR implementation in C++ <https://github.com/BeamMW/beam/blob/master/core/merkle.cpp>`_
- `Mimblewimble MMR docs <https://github.com/mimblewimble/grin/blob/master/doc/mmr.md>`_
- `MMR Python implementation <https://github.com/proofchains/python-proofmarshal/blob/master/proofmarshal/mmr.py>`_
- `Tari MMR documentation <https://docs.rs/merklemountainrange/0.0.1/src/merklemountainrange/lib.rs.html#23-183>`_
- `Zcash Protocol Specification, Version 2018.0-beta-37 [Overwinter+Sapling] <https://github.com/zcash/zips/blob/master/protocol/protocol.pdf>`_
- `opentimestamps-server Merkle Mountain Range documentation <https://github.com/opentimestamps/opentimestamps-server/blob/master/doc/merkle-mountain-range.md>`_
References
==========
.. [#RFC2119] `Key words for use in RFCs to Indicate Requirement Levels <https://tools.ietf.org/html/rfc2119>`_
.. [#zip-0200] `ZIP 200: Network Upgrade Mechanism <https://github.com/zcash/zips/blob/master/zip-0200.rst>`_
.. [#block-work] `Section 7.6.5: Definition of Work. Zcash Protocol Specification, Version 2020.1.1 [Overwinter+Sapling+Blossom] or later <https://zips.z.cash/protocol/protocol.pdf#workdef>`_
2020-02-27 08:33:38 -08:00
.. [#zcashBlock] `Zcash block primitive <https://github.com/zcash/zcash/blob/master/src/primitives/block.h>`_
.. [#ZIP-0307] `Zcash reference light client protocol <https://github.com/zcash/zips/blob/master/zip-0307/zip-0307.rst>`_
.. [#mimblewimble] `MimbleWimble Grin MMR implementation <https://github.com/mimblewimble/grin/blob/aedac483f5a116b91a8baf6acffd70e5f980b8cc/core/src/core/pmmr/pmmr.rs>`_
.. [#FlyClient] `FlyClient protocol <https://eprint.iacr.org/2019/226.pdf>`_