2019-03-31 00:57:20 -07:00
|
|
|
|
::
|
|
|
|
|
|
2020-03-03 21:38:41 -08:00
|
|
|
|
ZIP: 221
|
|
|
|
|
Title: FlyClient - Consensus-Layer Changes
|
2020-02-26 21:55:37 -08:00
|
|
|
|
Owners: Jack Grigg <jack@electriccoin.co>
|
2020-03-10 09:00:05 -07:00
|
|
|
|
Original-Authors: Ying Tong Lai
|
2020-03-03 20:38:43 -08:00
|
|
|
|
James Prestwich
|
|
|
|
|
Georgios Konstantopoulos
|
2020-09-05 05:11:28 -07:00
|
|
|
|
Status: Final
|
2019-03-31 00:57:20 -07:00
|
|
|
|
Category: Consensus
|
|
|
|
|
Created: 2019-03-30
|
2019-06-29 14:04:20 -07:00
|
|
|
|
License: MIT
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2021-10-01 16:44:09 -07:00
|
|
|
|
|
2019-03-31 00:57:20 -07:00
|
|
|
|
Terminology
|
2020-02-27 12:52:27 -08:00
|
|
|
|
===========
|
|
|
|
|
|
2020-04-01 03:38:08 -07:00
|
|
|
|
The key words "MUST", "MUST NOT", "SHOULD", and "MAY" in this document are to be interpreted
|
|
|
|
|
as described in RFC 2119. [#RFC2119]_
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-16 14:00:35 -07:00
|
|
|
|
The terms "consensus branch", "epoch", and "network upgrade" in this document are to be
|
|
|
|
|
interpreted as described in ZIP 200. [#zip-0200]_
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
*Light client*
|
2020-02-27 08:33:00 -08:00
|
|
|
|
A client that is not a full participant in the network of Zcash peers. It can send and
|
2020-03-09 19:47:20 -07:00
|
|
|
|
receive payments, but does not store or validate a copy of the block chain.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
*High probability*
|
2021-03-31 05:40:26 -07:00
|
|
|
|
An event occurs with high probability if it occurs with probability :math:`1-O(1/2^\lambda)`,
|
|
|
|
|
where :math:`\lambda` is a security parameter.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
*Negligible probability*
|
2021-03-31 05:40:26 -07:00
|
|
|
|
An event occurs with negligible probability if it occurs with probability :math:`O(1/2^\lambda)`,
|
|
|
|
|
where :math:`\lambda` is the security parameter.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
*Merkle mountain range (MMR)*
|
2020-02-27 08:55:35 -08:00
|
|
|
|
A Merkle mountain range (MMR) is a binary hash tree that allows for efficient appends of
|
2020-02-27 08:33:00 -08:00
|
|
|
|
new leaves without changing the value of existing nodes.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2019-03-31 00:57:20 -07:00
|
|
|
|
Abstract
|
2020-02-27 12:52:27 -08:00
|
|
|
|
========
|
|
|
|
|
|
2020-03-03 21:38:41 -08:00
|
|
|
|
This ZIP specifies modifications to the Zcash block header semantics and consensus rules
|
|
|
|
|
in order to support the probabilistic verification FlyClient protocol [#FlyClient]_. The
|
|
|
|
|
``hashFinalSaplingRoot`` commitment in the block header is replaced with a commitment to
|
|
|
|
|
the root of a Merkle Mountain Range (MMR), that in turn commits to various features of the
|
|
|
|
|
chain's history, including the Sapling commitment tree.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-02-27 12:52:27 -08:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
Background
|
2020-02-27 12:52:27 -08:00
|
|
|
|
==========
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
An MMR is a Merkle tree which allows for efficient appends, proofs, and verifications.
|
|
|
|
|
Informally, appending data to an MMR consists of creating a new leaf and then iteratively
|
2020-03-10 08:58:30 -07:00
|
|
|
|
merging neighboring subtrees with the same size. This takes at most :math:`\log(n)` operations
|
2020-02-27 08:33:00 -08:00
|
|
|
|
and only requires knowledge of the previous subtree roots, of which there are fewer than
|
2020-03-10 08:58:30 -07:00
|
|
|
|
:math:`\log(n)`.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
(example adapted from [#mimblewimble]_)
|
2020-02-27 08:33:00 -08:00
|
|
|
|
To illustrate this, consider a list of 11 leaves. We first construct the biggest perfect
|
|
|
|
|
binary subtrees possible by joining any balanced sibling trees that are the same size. We
|
|
|
|
|
do this starting from the left to the right, adding a parent as soon as 2 children exist.
|
2020-03-03 20:42:47 -08:00
|
|
|
|
This leaves us with three subtrees ("mountains") of altitudes 3, 1, and 0:
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
.. code-block:: text
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
/\
|
|
|
|
|
/ \
|
2019-06-29 14:04:20 -07:00
|
|
|
|
/\ /\
|
2019-06-28 03:27:46 -07:00
|
|
|
|
/\/\/\/\ /\ /
|
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
Note that the first leftmost peak is always the highest. We can number this structure in
|
2020-03-03 21:11:26 -08:00
|
|
|
|
the order by which nodes would be created, if the leaves were inserted from left to right:
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
.. code-block:: text
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-03 20:42:47 -08:00
|
|
|
|
Altitude
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
3 14
|
|
|
|
|
/ \
|
|
|
|
|
/ \
|
|
|
|
|
/ \
|
|
|
|
|
/ \
|
|
|
|
|
2 6 13
|
|
|
|
|
/ \ / \
|
|
|
|
|
1 2 5 9 12 17
|
|
|
|
|
/ \ / \ / \ / \ / \ /
|
|
|
|
|
0 0 1 3 4 7 8 10 11 15 16 18
|
|
|
|
|
|
|
|
|
|
and represent this numbering in a flat list:
|
|
|
|
|
|
2020-03-10 08:58:30 -07:00
|
|
|
|
+----------+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
|
|
|
|
|
| Position | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 |
|
|
|
|
|
+----------+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
|
|
|
|
|
| Altitude | 0 | 0 | 1 | 0 | 0 | 1 | 2 | 0 | 0 | 1 | 0 | 0 | 1 | 2 | 3 | 0 | 0 | 1 | 0 |
|
|
|
|
|
+----------+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 20:15:33 -07:00
|
|
|
|
Let :math:`h` be the altitude of a given node. We can easily jump to the node's right
|
|
|
|
|
sibling (if it has one) by adding :math:`2^{h+1} - 1` to its position, and its left child
|
|
|
|
|
(if it has one) by subtracting :math:`2^h`. This allows us to efficiently find the subtree
|
|
|
|
|
roots ("peaks") of the mountains.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
Once we have the positions of the mountain peaks, we "bag" them using the following
|
|
|
|
|
algorithm:
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
1. Generate a node connecting the 2 left-most peaks, forming a new peak.
|
|
|
|
|
2. Repeat 1. until we have a single peak.
|
2020-03-03 21:11:26 -08:00
|
|
|
|
|
|
|
|
|
Note that the extra nodes generated during the bagging process do not follow the above
|
|
|
|
|
rules for jumping between nodes.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
.. code-block:: text
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-03 20:42:47 -08:00
|
|
|
|
Altitude
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-03 21:11:26 -08:00
|
|
|
|
5 20g
|
2019-06-28 03:27:46 -07:00
|
|
|
|
/ \
|
2020-03-03 21:11:26 -08:00
|
|
|
|
4 19g \
|
2019-06-28 03:27:46 -07:00
|
|
|
|
/ \ \
|
|
|
|
|
/ \ \
|
|
|
|
|
/ \ \
|
|
|
|
|
3 14 \ \
|
|
|
|
|
/ \ \ \
|
|
|
|
|
/ \ \ \
|
|
|
|
|
/ \ \ \
|
|
|
|
|
/ \ \ \
|
|
|
|
|
2 6 13 \ \
|
|
|
|
|
/ \ / \ \ \
|
|
|
|
|
1 2 5 9 12 17 \
|
|
|
|
|
/ \ / \ / \ / \ / \ \
|
|
|
|
|
0 0 1 3 4 7 8 10 11 15 16 18
|
|
|
|
|
|
2020-02-27 08:55:35 -08:00
|
|
|
|
MMR trees allow for efficient incremental set update operations (push, pop, prune). In
|
2020-02-27 08:33:00 -08:00
|
|
|
|
addition, MMR update operations and Merkle proofs for recent additions to the leaf set are
|
|
|
|
|
more efficient than other incremental Merkle tree implementations (e.g. Bitcoin's padded
|
2020-02-27 08:33:38 -08:00
|
|
|
|
leafset, sparse Merkle trees, and Zcash's incremental note commitment trees).
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-02-27 12:52:27 -08:00
|
|
|
|
|
2019-03-31 00:57:20 -07:00
|
|
|
|
Motivation
|
2020-02-27 12:52:27 -08:00
|
|
|
|
==========
|
|
|
|
|
|
2020-03-09 20:15:33 -07:00
|
|
|
|
MMR proofs are used in the FlyClient protocol [#FlyClient]_, to reduce the proof size
|
|
|
|
|
needed for light clients to verify:
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
- the validity of a block chain received from a full node, and
|
|
|
|
|
- the inclusion of a block :math:`B` in that chain, and
|
|
|
|
|
- certain metadata of any block or range of blocks in that chain.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
The protocol requires that an MMR that commits to the inclusion of all blocks since the
|
2020-03-16 14:30:46 -07:00
|
|
|
|
preceding network upgrade :math:`(B_x, \ldots, B_{n-1})` is formed for each block :math:`B_n`.
|
2020-03-09 19:47:20 -07:00
|
|
|
|
The root :math:`M_n` of the MMR MUST be included in the header of :math:`B_n`.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-16 14:30:46 -07:00
|
|
|
|
(:math:`x` is the activation height of the preceding network upgrade.)
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
FlyClient reduces the number of block headers needed for light client verification of a
|
|
|
|
|
valid chain, from linear (as in the current reference protocol) to logarithmic in
|
2020-03-09 19:47:20 -07:00
|
|
|
|
block chain length. This verification is correct with high probability. It also allows
|
2020-02-27 08:33:00 -08:00
|
|
|
|
creation of subtree proofs, so light clients need only check blocks later than the most
|
|
|
|
|
recently verified block index. Following that, verification of a transaction inclusion
|
2020-03-03 11:04:37 -08:00
|
|
|
|
within that block follows the usual reference protocol [#zip-0307]_.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
A smaller proof size could enable the verification of Zcash SPV Proofs in block-chain
|
|
|
|
|
protocols such as Ethereum, enabling efficient cross-chain communication and pegs. It also
|
|
|
|
|
reduces bandwidth and storage requirements for resource-limited clients like mobile or IoT
|
2020-02-27 08:33:00 -08:00
|
|
|
|
devices.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
|
2019-03-31 00:57:20 -07:00
|
|
|
|
Specification
|
2020-02-27 12:52:27 -08:00
|
|
|
|
=============
|
|
|
|
|
|
2020-03-16 14:30:46 -07:00
|
|
|
|
For a block :math:`B_n` at height :math:`n > 0` in a given block chain, define the
|
|
|
|
|
"preceding network upgrade height" :math:`x` of :math:`B_n` to be the last network
|
2020-04-01 03:32:22 -07:00
|
|
|
|
upgrade activation height in the chain that is less than :math:`n`. (For this definition,
|
2020-03-16 14:30:46 -07:00
|
|
|
|
block height :math:`0` is considered to be the height of a network upgrade activation.
|
|
|
|
|
The preceding network upgrade height of the genesis block is undefined.)
|
|
|
|
|
|
|
|
|
|
The leaves of the MMR at block :math:`B_n` are hash commitments to the header data
|
|
|
|
|
and metadata of each previous block :math:`B_x, \ldots, B_{n-1}`, where :math:`x`
|
|
|
|
|
is defined as above. We extend the standard MMR to allow metadata to propagate
|
2020-02-27 08:33:00 -08:00
|
|
|
|
upwards through the tree by either summing the metadata of both children, or inheriting
|
|
|
|
|
the metadata of a specific child as necessary. This allows us to create efficient proofs
|
|
|
|
|
of selected properties of a range of blocks without transmitting the entire range of
|
|
|
|
|
blocks or headers.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
Tree Node specification
|
2020-02-27 12:52:27 -08:00
|
|
|
|
-----------------------
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-16 14:00:35 -07:00
|
|
|
|
Unless otherwise noted, all hashes use BLAKE2b-256 with the personalization field set
|
|
|
|
|
to ``'ZcashHistory' || CONSENSUS_BRANCH_ID``. ``CONSENSUS_BRANCH_ID`` is the 4-byte
|
|
|
|
|
little-endian encoding of the consensus branch ID for the epoch of the block containing
|
|
|
|
|
the commitment. [#zip-0200]_ Which is to say, each node in the tree commits to the
|
|
|
|
|
consensus branch that produced it.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
Each MMR node is defined as follows:
|
|
|
|
|
|
|
|
|
|
1. ``hashSubtreeCommitment``
|
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
The consensus-defined block hash for the corresponding block.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
* This hash is encoded in internal byte order, and does NOT use the BLAKE2b-256
|
|
|
|
|
personalization string described above.
|
2020-03-16 14:30:46 -07:00
|
|
|
|
* For clarity, in a given consensus branch, the ``hashSubtreeCommitment`` field
|
|
|
|
|
of leaf :math:`n-1` is *precisely equal* to the ``hashPrevBlock`` field in the
|
|
|
|
|
header of the block at height :math:`x+n`, where :math:`x` is the network
|
|
|
|
|
upgrade activation height of that consensus branch.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
* Both child nodes are serialized.
|
|
|
|
|
* ``hashSubtreeCommitment`` is the BLAKE2b-256 hash of ``left_child || right_child``.
|
|
|
|
|
* For clarity, this digest uses the BLAKE2b-256 personalization string described above.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Serialized as ``char[32]``.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
2. ``nEarliestTimestamp``
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
The header's timestamp.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the left child.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-10 12:16:55 -07:00
|
|
|
|
Serialized as ``nTime`` (``uint32``).
|
|
|
|
|
|
|
|
|
|
Note that a ``uint32`` time value would overflow on 2106-02-07, but this field (and
|
|
|
|
|
``nLatestTimestamp`` below) can only hold values that occur in the ``nTime`` field of
|
|
|
|
|
a block header, which is also of type ``uint32``.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
3. ``nLatestTimestamp``
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
The header's timestamp.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the right child.
|
2020-02-27 08:55:35 -08:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Note that due to timestamp consensus rules, ``nLatestTimestamp`` may be smaller than
|
|
|
|
|
``nEarliestTimestamp`` in some subtrees. This may occur within subtrees smaller than
|
|
|
|
|
``PoWMedianBlockSpan`` blocks.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-10 12:16:55 -07:00
|
|
|
|
Serialized as ``nTime`` (``uint32``).
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 20:15:33 -07:00
|
|
|
|
4. ``nEarliestTargetBits``
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
The header's ``nBits`` field.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the left child.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Serialized as ``nBits`` (``uint32``).
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 20:15:33 -07:00
|
|
|
|
5. ``nLatestTargetBits``
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
The header's ``nBits`` field.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the right child.
|
2020-02-27 08:55:35 -08:00
|
|
|
|
|
2020-03-10 09:50:39 -07:00
|
|
|
|
Serialized as ``nBits`` (``uint32``).
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
6. ``hashEarliestSaplingRoot``
|
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
Calculated as ``hashFinalSaplingRoot``, as implemented in Sapling.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the left child.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Serialized as ``char[32]``.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
7. ``hashLatestSaplingRoot``
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
Calculated as ``hashFinalSaplingRoot``, as implemented in Sapling.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the right child.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Serialized as ``char[32]``.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2019-06-29 14:04:20 -07:00
|
|
|
|
8. ``nSubTreeTotalWork``
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
The protocol-defined work of the block:
|
2020-11-09 07:59:51 -08:00
|
|
|
|
:math:`\mathsf{floor}(2^{256} / (\mathsf{ToTarget}(\mathsf{nBits}) + 1))`. [#protocol-workdef]_
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
The sum of the ``nSubTreeTotalWork`` fields of both children.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Computations modulo :math:`2^{256}` are fine here; cumulative chain work is similarly
|
|
|
|
|
assumed elsewhere in the Zcash ecosystem to be at most :math:`2^{256}` (as inherited
|
|
|
|
|
from Bitcoin). The computed work factors are, on average, equal to the computational
|
|
|
|
|
efforts involved in the creation of the corresponding blocks, and an aggregate effort
|
|
|
|
|
of :math:`2^{256}` or more is infeasible in practice.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Serialized as ``uint256``.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
|
|
|
|
9. ``nEarliestHeight``
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
2021-05-03 15:10:20 -07:00
|
|
|
|
The height of the block.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the left child.
|
2020-02-27 08:55:35 -08:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Serialized as ``CompactSize uint``.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2019-06-29 14:04:20 -07:00
|
|
|
|
10. ``nLatestHeight``
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
2021-05-03 15:10:20 -07:00
|
|
|
|
The height of the block.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the right child.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Serialized as ``CompactSize uint``.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-03 18:30:12 -08:00
|
|
|
|
11. ``nSaplingTxCount``
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
The number of transactions in the leaf block where either of
|
2021-03-29 15:18:45 -07:00
|
|
|
|
``vSpendsSapling`` or ``vOutputsSapling`` is non-empty.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
The sum of the ``nSaplingTxCount`` field of both children.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
Serialized as ``CompactSize uint``.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2021-03-29 15:19:13 -07:00
|
|
|
|
12. [NU5 onward] ``hashEarliestOrchardRoot``
|
|
|
|
|
|
2021-03-31 05:37:49 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
Calculated as the note commitment root of the final Orchard treestate
|
|
|
|
|
(similar to ``hashEarliestSaplingRoot`` in Sapling).
|
2021-03-29 15:19:13 -07:00
|
|
|
|
|
2021-03-31 05:37:49 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the left child.
|
2021-03-29 15:19:13 -07:00
|
|
|
|
|
2021-03-31 05:37:49 -07:00
|
|
|
|
Serialized as ``char[32]``.
|
2021-03-29 15:19:13 -07:00
|
|
|
|
|
|
|
|
|
13. [NU5 onward] ``hashLatestOrchardRoot``
|
|
|
|
|
|
2021-03-31 05:37:49 -07:00
|
|
|
|
Leaf node
|
|
|
|
|
Calculated as the note commitment root of the final Orchard treestate
|
|
|
|
|
(similar to ``hashLatestSaplingRoot`` in Sapling).
|
2021-03-29 15:19:13 -07:00
|
|
|
|
|
2021-03-31 05:37:49 -07:00
|
|
|
|
Internal or root node
|
|
|
|
|
Inherited from the right child.
|
2021-03-29 15:19:13 -07:00
|
|
|
|
|
2021-03-31 05:37:49 -07:00
|
|
|
|
Serialized as ``char[32]``.
|
2021-03-29 15:19:13 -07:00
|
|
|
|
|
|
|
|
|
14. [NU5 onward] ``nOrchardTxCount``
|
|
|
|
|
|
|
|
|
|
Leaf node
|
|
|
|
|
The number of transactions in the leaf block where ``vActionsOrchard``
|
|
|
|
|
is non-empty.
|
|
|
|
|
|
|
|
|
|
Internal or root node
|
|
|
|
|
The sum of the ``nOrchardTxCount`` field of both children.
|
|
|
|
|
|
|
|
|
|
Serialized as ``CompactSize uint``.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The fields marked "[NU5 onward]" are omitted before NU5 activation [#zip-0252]_.
|
|
|
|
|
|
|
|
|
|
Each node, when serialized, is between 147 and 171 bytes long (between 212 and 244 bytes
|
|
|
|
|
after NU5 activation). The canonical serialized representation of a node is used whenever
|
|
|
|
|
creating child commitments for future nodes. Other than the metadata commitments, the
|
|
|
|
|
MMR tree's construction is standard.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
Once the MMR has been generated, we produce ``hashChainHistoryRoot``, which we define as
|
|
|
|
|
the BLAKE2b-256 digest of the serialization of the root node.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Tree nodes and hashing (pseudocode)
|
2020-02-27 12:52:27 -08:00
|
|
|
|
-----------------------------------
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2019-06-29 14:04:20 -07:00
|
|
|
|
.. code-block:: python
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-24 07:53:50 -07:00
|
|
|
|
def H(msg: bytes, consensusBranchId: bytes) -> bytes:
|
|
|
|
|
return blake2b256(msg, personalization=b'ZcashHistory' + consensusBranchId)
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
class ZcashMMRNode():
|
|
|
|
|
# leaf nodes have no children
|
2020-03-09 19:47:20 -07:00
|
|
|
|
left_child: Optional[ZcashMMRNode]
|
|
|
|
|
right_child: Optional[ZcashMMRNode]
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
# commitments
|
2020-02-27 08:56:27 -08:00
|
|
|
|
hashSubtreeCommitment: bytes
|
|
|
|
|
nEarliestTimestamp: int
|
|
|
|
|
nLatestTimestamp: int
|
2020-03-09 20:15:33 -07:00
|
|
|
|
nEarliestTargetBits: int
|
|
|
|
|
nLatestTargetBits: int
|
2021-06-20 12:48:55 -07:00
|
|
|
|
hashEarliestSaplingRoot: bytes # left child's Sapling root
|
|
|
|
|
hashLatestSaplingRoot: bytes # right child's Sapling root
|
2020-02-27 08:56:27 -08:00
|
|
|
|
nSubTreeTotalWork: int # total difficulty accumulated within each subtree
|
|
|
|
|
nEarliestHeight: int
|
|
|
|
|
nLatestHeight: int
|
2020-03-03 18:30:12 -08:00
|
|
|
|
nSaplingTxCount: int # number of Sapling transactions in block
|
2021-06-20 12:48:55 -07:00
|
|
|
|
# NU5 only.
|
|
|
|
|
hashEarliestOrchardRoot: Optional[bytes] # left child's Orchard root
|
|
|
|
|
hashLatestOrchardRoot: Optional[bytes] # right child's Orchard root
|
|
|
|
|
nSaplingTxCount: Optional[int] # number of Orchard transactions in block
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-24 07:53:50 -07:00
|
|
|
|
consensusBranchId: bytes
|
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
@classmethod
|
2020-03-09 19:47:20 -07:00
|
|
|
|
def from_block(Z, block: ZcashBlock) -> ZcashMMRNode:
|
2019-06-28 03:27:46 -07:00
|
|
|
|
'''Create a leaf node from a block'''
|
|
|
|
|
return Z(
|
|
|
|
|
left_child=None,
|
|
|
|
|
right_child=None,
|
2020-02-27 08:56:27 -08:00
|
|
|
|
hashSubtreeCommitment=block.header_hash,
|
|
|
|
|
nEarliestTimestamp=block.timestamp,
|
|
|
|
|
nLatestTimestamp=block.timestamp,
|
2020-03-09 20:15:33 -07:00
|
|
|
|
nEarliestTargetBits=block.nBits,
|
|
|
|
|
nLatestTargetBits=block.nBits,
|
2020-02-27 08:56:27 -08:00
|
|
|
|
hashEarliestSaplingRoot=block.sapling_root,
|
|
|
|
|
hashLatestSaplingRoot=block.sapling_root,
|
|
|
|
|
nSubTreeTotalWork=calculate_work(block.nBits),
|
|
|
|
|
nEarliestHeight=block.height,
|
|
|
|
|
nLatestHeight=block.height,
|
2020-03-24 07:53:50 -07:00
|
|
|
|
nSaplingTxCount=block.sapling_tx_count,
|
2021-06-20 12:48:55 -07:00
|
|
|
|
hashEarliestOrchardRoot=block.orchard_root,
|
|
|
|
|
hashLatestOrchardRoot=block.orchard_root,
|
|
|
|
|
nOrchardTxCount=block.orchard_tx_count,
|
2020-03-24 07:53:50 -07:00
|
|
|
|
consensusBranchId=block.consensusBranchId)
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
def serialize(self) -> bytes:
|
|
|
|
|
'''serializes a node'''
|
2021-06-20 12:48:55 -07:00
|
|
|
|
buf = (self.hashSubtreeCommitment
|
2020-02-27 08:56:27 -08:00
|
|
|
|
+ serialize_uint32(self.nEarliestTimestamp)
|
|
|
|
|
+ serialize_uint32(self.nLatestTimestamp)
|
2020-03-09 20:15:33 -07:00
|
|
|
|
+ serialize_uint32(self.nEarliestTargetBits)
|
|
|
|
|
+ serialize_uint32(self.nLatestTargetBits)
|
2020-02-27 08:56:27 -08:00
|
|
|
|
+ hashEarliestSaplingRoot
|
|
|
|
|
+ hashLatestSaplingRoot
|
|
|
|
|
+ serialize_uint256(self.nSubTreeTotalWork)
|
|
|
|
|
+ serialize_compact_uint(self.nEarliestHeight)
|
|
|
|
|
+ serialize_compact_uint(self.nLatestHeight)
|
2020-03-03 18:30:12 -08:00
|
|
|
|
+ serialize_compact_uint(self.nSaplingTxCount))
|
2021-06-20 12:48:55 -07:00
|
|
|
|
if hashEarliestOrchardRoot is not None:
|
|
|
|
|
buf += (hashEarliestOrchardRoot
|
|
|
|
|
+ hashLatestOrchardRoot
|
|
|
|
|
+ serialize_compact_uint(self.nOrchardTxCount))
|
|
|
|
|
return buf
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
def make_parent(
|
2019-06-29 14:04:20 -07:00
|
|
|
|
left_child: ZcashMMRNode,
|
2019-06-28 03:27:46 -07:00
|
|
|
|
right_child: ZcashMMRNode) -> ZcashMMRNode:
|
|
|
|
|
return ZcashMMRNode(
|
|
|
|
|
left_child=left_child,
|
|
|
|
|
right_child=right_child,
|
2020-03-31 05:54:41 -07:00
|
|
|
|
hashSubtreeCommitment=H(left_child.serialize() + right_child.serialize(),
|
|
|
|
|
left_child.consensusBranchId),
|
2020-02-27 08:56:27 -08:00
|
|
|
|
nEarliestTimestamp=left_child.nEarliestTimestamp,
|
|
|
|
|
nLatestTimestamp=right_child.nLatestTimestamp,
|
2020-03-09 20:15:33 -07:00
|
|
|
|
nEarliestTargetBits=left_child.nEarliestTargetBits,
|
|
|
|
|
nLatestTargetBits=right_child.nLatestTargetBits,
|
2020-02-27 08:56:27 -08:00
|
|
|
|
hashEarliestSaplingRoot=left_child.sapling_root,
|
|
|
|
|
hashLatestSaplingRoot=right_child.sapling_root,
|
|
|
|
|
nSubTreeTotalWork=left_child.nSubTreeTotalWork + right_child.nSubTreeTotalWork,
|
|
|
|
|
nEarliestHeight=left_child.nEarliestHeight,
|
|
|
|
|
nLatestHeight=right_child.nLatestHeight,
|
2020-03-24 07:53:50 -07:00
|
|
|
|
nSaplingTxCount=left_child.nSaplingTxCount + right_child.nSaplingTxCount,
|
2021-06-20 12:48:55 -07:00
|
|
|
|
hashEarliestOrchardRoot=left_child.orchard_root,
|
|
|
|
|
hashLatestOrchardRoot=right_child.orchard_root,
|
|
|
|
|
nOrchardTxCount=(left_child.nOrchardTxCount + right_child.nOrchardTxCount
|
|
|
|
|
if left_child.nOrchardTxCount is not None and right_child.nOrchardTxCount is not None
|
|
|
|
|
else None),
|
2020-03-24 07:53:50 -07:00
|
|
|
|
consensusBranchId=left_child.consensusBranchId)
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
def make_root_commitment(root: ZcashMMRNode) -> bytes:
|
|
|
|
|
'''Makes the root commitment for a blockheader'''
|
2020-03-24 07:53:50 -07:00
|
|
|
|
return H(root.serialize(), root.consensusBranchId)
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
Incremental push and pop (pseudocode)
|
2020-02-27 12:52:27 -08:00
|
|
|
|
-------------------------------------
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
With each new block :math:`B_n`, we append a new MMR leaf node corresponding to block
|
|
|
|
|
:math:`B_{n-1}`. The ``append`` operation is detailed below in pseudocode (adapted from
|
2020-02-27 08:33:00 -08:00
|
|
|
|
[#FlyClient]_):
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
.. code-block:: python
|
|
|
|
|
|
2019-09-01 17:08:24 -07:00
|
|
|
|
def get_peaks(node: ZcashMMRNode) -> List[ZcashMMRNode]:
|
|
|
|
|
peaks: List[ZcashMMRNode] = []
|
|
|
|
|
|
2020-03-03 12:12:53 -08:00
|
|
|
|
# Get number of leaves.
|
2020-03-03 12:15:15 -08:00
|
|
|
|
leaves = latest_height - (earliest_height - 1)
|
2020-03-09 20:15:33 -07:00
|
|
|
|
assert(leaves > 0)
|
2020-03-03 12:12:53 -08:00
|
|
|
|
|
|
|
|
|
# Check if the number of leaves is a power of two.
|
|
|
|
|
if (leaves & (leaves - 1)) == 0:
|
|
|
|
|
# Tree is full, hence a single peak. This also covers the
|
|
|
|
|
# case of a single isolated leaf.
|
|
|
|
|
peaks.append(node)
|
2019-09-01 17:08:24 -07:00
|
|
|
|
else:
|
2020-03-03 12:12:53 -08:00
|
|
|
|
# If the number of leaves is not a power of two, then this
|
|
|
|
|
# node must be internal, and cannot be a peak.
|
2019-09-01 17:08:24 -07:00
|
|
|
|
peaks.extend(get_peaks(left_child))
|
|
|
|
|
peaks.extend(get_peaks(right_child))
|
|
|
|
|
|
|
|
|
|
return peaks
|
|
|
|
|
|
|
|
|
|
|
2020-03-03 11:40:14 -08:00
|
|
|
|
def bag_peaks(peaks: List[ZcashMMRNode]) -> ZcashMMRNode:
|
|
|
|
|
'''
|
|
|
|
|
"Bag" a list of peaks, and return the final root
|
|
|
|
|
'''
|
|
|
|
|
root = peaks[0]
|
|
|
|
|
for i in range(1, len(peaks)):
|
|
|
|
|
root = make_parent(root, peaks[i])
|
|
|
|
|
return root
|
|
|
|
|
|
|
|
|
|
|
2019-09-01 17:08:24 -07:00
|
|
|
|
def append(root: ZcashMMRNode, leaf: ZcashMMRNode) -> ZcashMMRNode:
|
|
|
|
|
'''Append a leaf to an existing tree, return the new tree root'''
|
|
|
|
|
# recursively find a list of peaks in the current tree
|
|
|
|
|
peaks: List[ZcashMMRNode] = get_peaks(root)
|
|
|
|
|
merged: List[ZcashMMRNode] = []
|
|
|
|
|
|
|
|
|
|
# Merge peaks from right to left.
|
|
|
|
|
# This will produce a list of peaks in reverse order
|
|
|
|
|
current = leaf
|
|
|
|
|
for peak in peaks[::-1]:
|
2020-03-03 12:15:15 -08:00
|
|
|
|
current_leaves = current.latest_height - (current.earliest_height - 1)
|
|
|
|
|
peak_leaves = peak.latest_height - (peak.earliest_height - 1)
|
2019-09-01 17:08:24 -07:00
|
|
|
|
|
|
|
|
|
if current_leaves == peak_leaves:
|
|
|
|
|
current = make_parent(peak, current)
|
|
|
|
|
else:
|
|
|
|
|
merged.append(current)
|
|
|
|
|
current = peak
|
2020-03-03 12:12:16 -08:00
|
|
|
|
merged.append(current)
|
2019-09-01 17:08:24 -07:00
|
|
|
|
|
|
|
|
|
# finally, bag the merged peaks
|
|
|
|
|
return bag_peaks(merged[::-1])
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
In case of a block reorg, we have to delete the latest (i.e. rightmost) MMR leaf nodes, up
|
2020-03-09 19:47:20 -07:00
|
|
|
|
to the reorg length. This operation is :math:`O(\log(k))` where :math:`k` is the number of leaves
|
2020-02-27 08:33:00 -08:00
|
|
|
|
in the right subtree of the MMR root.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
.. code-block:: python
|
|
|
|
|
|
|
|
|
|
def delete(root: ZcashMMRNode) -> ZcashMMRNode:
|
|
|
|
|
'''
|
|
|
|
|
Delete the rightmost leaf node from an existing MMR
|
|
|
|
|
Return the new tree root
|
|
|
|
|
'''
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-03 12:15:15 -08:00
|
|
|
|
n_leaves = root.latest_height - (root.earliest_height - 1)
|
2019-06-29 14:04:20 -07:00
|
|
|
|
# if there were an odd number of leaves,
|
2019-06-28 03:27:46 -07:00
|
|
|
|
# simply replace root with left_child
|
|
|
|
|
if n_leaves & 1:
|
|
|
|
|
return root.left_child
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
# otherwise, we need to re-bag the peaks.
|
|
|
|
|
else:
|
|
|
|
|
# first peak
|
|
|
|
|
peaks = [root.left_child]
|
|
|
|
|
|
|
|
|
|
# we do this traversing the right (unbalanced) side of the tree
|
|
|
|
|
# we keep the left side (balanced subtree or leaf) of each subtree
|
|
|
|
|
# until we reach a leaf
|
|
|
|
|
subtree_root = root.right_child
|
|
|
|
|
while subtree_root.left_child:
|
2020-03-03 11:45:12 -08:00
|
|
|
|
peaks.push(subtree_root.left_child)
|
|
|
|
|
subtree_root = subtree_root.right_child
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
new_root = bag_peaks(peaks)
|
|
|
|
|
return new_root
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-03 20:14:46 -08:00
|
|
|
|
Block header semantics and consensus rules
|
|
|
|
|
------------------------------------------
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2021-03-29 15:19:13 -07:00
|
|
|
|
The following specification is accurate before NU5 activation. See [#zip-0244]_ for header
|
|
|
|
|
field changes in NU5.
|
|
|
|
|
|
2020-03-03 20:14:46 -08:00
|
|
|
|
The ``hashFinalSaplingRoot`` block header field (which was named ``hashReserved`` prior to
|
|
|
|
|
the Sapling network upgrade) is renamed to ``hashLightClientRoot``, to reflect its usage
|
2021-05-07 07:22:56 -07:00
|
|
|
|
by light clients. (In NU5, this field is renamed again to ``hashBlockCommitments`` as specified
|
|
|
|
|
in [#zip-0244]_.)
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-04-01 03:38:08 -07:00
|
|
|
|
Prior to activation of the network upgrade that deploys this ZIP, this existing consensus
|
2020-11-09 07:59:51 -08:00
|
|
|
|
rule on block headers (adjusted for the renamed field) is enforced: [#protocol-blockheader]_
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-09 19:47:20 -07:00
|
|
|
|
[Sapling onward] ``hashLightClientRoot`` MUST be :math:`\mathsf{LEBS2OSP}_{256}(\mathsf{rt})`
|
|
|
|
|
where :math:`\mathsf{rt}` is the root of the Sapling note commitment tree for the final
|
|
|
|
|
Sapling tree state of this block.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-04-01 03:38:08 -07:00
|
|
|
|
In the block that activates this ZIP, ``hashLightClientRoot`` MUST be set to all zero bytes.
|
|
|
|
|
This MUST NOT be interpreted as a root hash.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-04-01 03:38:08 -07:00
|
|
|
|
In subsequent blocks, ``hashLightClientRoot`` MUST be set to the value of ``hashChainHistoryRoot``
|
|
|
|
|
as specified above.
|
2020-03-16 14:30:46 -07:00
|
|
|
|
|
2020-03-03 20:14:46 -08:00
|
|
|
|
The block header byte format and version are not altered by this ZIP.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Rationale
|
2020-02-27 12:52:27 -08:00
|
|
|
|
=========
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
Tree nodes
|
2020-02-27 12:52:27 -08:00
|
|
|
|
----------
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
Nodes in the commitment tree are canonical and immutable. They are cheap to generate, as
|
2021-03-29 15:19:13 -07:00
|
|
|
|
(with the exception of ``nSaplingTxCount`` and ``nOrchardTxCount``) all metadata is already
|
|
|
|
|
generated during block construction and/or checked during block validation. Nodes are
|
|
|
|
|
relatively compact in memory. As of the original publication of this ZIP, approximately
|
|
|
|
|
140,000 blocks had elapsed since Sapling activation. Assuming a 164-byte commitment to
|
|
|
|
|
each of these, we would have generated approximately 24 MB of additional storage cost for
|
|
|
|
|
the set of leaf nodes (and an additional ~24 MB for storage of intermediate nodes).
|
2020-02-27 08:33:00 -08:00
|
|
|
|
|
2021-06-20 12:48:55 -07:00
|
|
|
|
``hashSubtreeCommitment`` forms the structure of the commitment tree. Other metadata
|
2021-03-29 15:19:13 -07:00
|
|
|
|
commitments were chosen to serve specific purposes. Originally variable-length commitments
|
|
|
|
|
were placed last, so that most metadata in a node could be directly indexed. We considered
|
|
|
|
|
using fixed-length commitments here, but opted for variable-length, in order to marginally
|
2020-02-27 08:33:00 -08:00
|
|
|
|
reduce the memory requirements for managing and updating the commitment trees.
|
|
|
|
|
|
2021-03-30 06:54:04 -07:00
|
|
|
|
Orchard fields are placed last, in order to avoid complicating existing uses of the other
|
|
|
|
|
fields.
|
2021-03-29 15:19:13 -07:00
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
In leaf nodes, some information is repeated. We chose to do this so that leaf nodes could
|
|
|
|
|
be treated identically to internal and root nodes for all algorithms and (de)serializers.
|
|
|
|
|
Leaf nodes are easily identifiable, as they will show proof of work in the
|
2020-03-03 21:17:27 -08:00
|
|
|
|
``hashSubtreeCommitment`` field (which commits to the block hash for leaf nodes), and
|
2020-03-10 08:58:30 -07:00
|
|
|
|
their block range (calculated as ``nLatestHeight`` - (``nEarliestHeight`` - 1))
|
|
|
|
|
will be precisely 1.
|
2020-02-27 08:33:00 -08:00
|
|
|
|
|
2020-02-27 08:33:38 -08:00
|
|
|
|
Personalized BLAKE2b-256 was selected to match existing Zcash conventions. Adding the
|
2020-02-27 08:33:00 -08:00
|
|
|
|
consensus branch ID to the hash personalization string ensures that valid nodes from one
|
2020-03-16 14:00:35 -07:00
|
|
|
|
consensus branch cannot be used to make false statements about parallel consensus branches.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
FlyClient Requirements and Recommendations
|
2020-02-27 12:52:27 -08:00
|
|
|
|
``````````````````````````````````````````
|
2020-02-27 08:33:00 -08:00
|
|
|
|
These commitments enable FlyClient in the variable-difficulty model. Specifically, they
|
|
|
|
|
allow light clients to reason about application of the difficulty adjustment algorithm
|
|
|
|
|
over a range of blocks. They were chosen via discussion with an author of the FlyClient
|
|
|
|
|
paper.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
- ``nEarliestTimestamp``
|
|
|
|
|
- ``nLatestTimestamp``
|
2020-03-09 20:15:33 -07:00
|
|
|
|
- ``nEarliestTargetBits``
|
|
|
|
|
- ``nLatestTargetBits``
|
2019-06-28 03:27:46 -07:00
|
|
|
|
- ``nEarliestHeight``
|
|
|
|
|
- ``nLatestHeight``
|
|
|
|
|
- ``nSubTreeTotalWork``
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
Non-FlyClient Commitments
|
2020-02-27 12:52:27 -08:00
|
|
|
|
`````````````````````````
|
2020-02-27 08:33:00 -08:00
|
|
|
|
Additional metadata commitments were chosen primarily to improve light client security
|
|
|
|
|
guarantees. We specified commitments where we could see an obvious security benefit, but
|
|
|
|
|
there may be other useful metadata that we missed. We're interested in feedback and
|
|
|
|
|
suggestions from the implementers of the current light client.
|
2019-03-31 00:57:20 -07:00
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
We considered adding a commitment to the nullifier vector at each block. We would
|
|
|
|
|
appreciate comments from light client teams on the utility of this commitment, as well as
|
2020-04-01 03:38:08 -07:00
|
|
|
|
the proper serialization and commitment format for the nullifier vector, for possible
|
|
|
|
|
inclusion in a future upgrade.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
- ``hashEarliestSaplingRoot``
|
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
* Committing to the earliest Sapling root of a range of blocks allows light clients to
|
2020-03-10 08:58:30 -07:00
|
|
|
|
check the consistency of treestate transitions over a range of blocks, without
|
|
|
|
|
recalculating the root from genesis.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
- ``hashLatestSaplingRoot``
|
|
|
|
|
|
2020-02-27 08:33:00 -08:00
|
|
|
|
* This commitment serves the same purpose as ``hashFinalSaplingRoot`` in current Sapling
|
|
|
|
|
semantics.
|
2020-03-09 19:47:20 -07:00
|
|
|
|
* However, because the MMR tree commits to blocks :math:`B_x \ldots B_{n-1}`, the latest
|
2020-02-27 08:55:35 -08:00
|
|
|
|
commitment will describe the final treestate of the previous block, rather than the
|
2020-02-27 08:33:00 -08:00
|
|
|
|
current block.
|
|
|
|
|
* Concretely: block 500 currently commits to the final treestate of block 500 in its
|
|
|
|
|
header. With this ZIP, block 500 will commit to all roots up to block 499, but not the
|
|
|
|
|
final root of block 500.
|
|
|
|
|
* We feel this is an acceptable tradeoff. Using the most recent treestate as a
|
|
|
|
|
transaction anchor is already unsafe in reorgs. Clients should never use the most
|
|
|
|
|
recent treestate to generate transactions, so it is acceptable to delay commitment by
|
|
|
|
|
one block.
|
2019-06-29 14:04:20 -07:00
|
|
|
|
|
2020-03-03 18:30:12 -08:00
|
|
|
|
- ``nSaplingTxCount``
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-03 18:30:12 -08:00
|
|
|
|
* By committing to the number of Sapling transactions in blocks (and ranges of blocks),
|
2020-02-27 08:33:00 -08:00
|
|
|
|
a light client may reliably learn whether a malicious server is witholding any
|
2020-03-03 18:30:12 -08:00
|
|
|
|
Sapling transactions.
|
2020-02-27 08:33:00 -08:00
|
|
|
|
* In addition, this commitment allows light clients to avoid syncing header ranges that
|
2020-03-03 18:30:12 -08:00
|
|
|
|
do not contain Sapling transactions. As the primary cost of a light client is
|
2020-03-09 19:47:20 -07:00
|
|
|
|
transmission of Equihash solution information in block headers, this optimization
|
2020-02-27 08:33:00 -08:00
|
|
|
|
would significantly decrease the bandwidth requirements of light clients.
|
2020-03-03 18:30:12 -08:00
|
|
|
|
* An earlier draft of this ZIP committed to the number of shielded transactions,
|
|
|
|
|
counting both Sprout and Sapling. This commitment would not have been useful to light
|
|
|
|
|
clients that only support Sapling addresses; they would not be able to distinguish
|
|
|
|
|
between Sapling transactions being maliciously withheld, and Sprout transactions not
|
|
|
|
|
being requested.
|
|
|
|
|
* A commitment to the number of Sprout transactions in blocks was not included, because
|
|
|
|
|
Sprout addresses are effectively deprecated at this point, and will not be supported
|
|
|
|
|
by any light clients.
|
|
|
|
|
* If a future network upgrade introduced a new shielded pool, a new commitment to that
|
|
|
|
|
pool's transactions would be added, to similarly enable future light clients that do
|
|
|
|
|
not support Sapling addresses.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2021-03-29 15:19:13 -07:00
|
|
|
|
- ``hashEarliestOrchardRoot``, ``hashLatestOrchardRoot``, and ``nOrchardTxCount``
|
|
|
|
|
|
|
|
|
|
* These are included with the same rationale as for Sapling.
|
|
|
|
|
|
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
Header Format Change
|
2020-02-27 12:52:27 -08:00
|
|
|
|
--------------------
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-03-03 20:14:46 -08:00
|
|
|
|
The primary goal of the original authors was to minimize header changes; in particular,
|
|
|
|
|
they preferred not to introduce changes that could affect mining hardware or embedded
|
|
|
|
|
software. Altering the block header format would require changes throughout the ecosystem,
|
|
|
|
|
so we decided against adding ``hashChainHistoryRoot`` to the header as a new field.
|
|
|
|
|
|
|
|
|
|
ZIP 301 states that "[Miner client software] SHOULD alert the user upon receiving jobs
|
|
|
|
|
containing block header versions they do not know about or support, and MUST ignore such
|
2020-03-09 19:47:20 -07:00
|
|
|
|
jobs." [#zip-0301]_ As the only formally defined block header version is 4, any header
|
2020-03-03 20:14:46 -08:00
|
|
|
|
version change requires changes to miner client software in order for miners to handle new
|
|
|
|
|
jobs from mining pools. We therefore do not alter the block version for this semantic
|
|
|
|
|
change. This does not make block headers ambiguous to interpret, because blocks commit to
|
|
|
|
|
their block height inside their coinbase transaction, [#bip-0034]_ and they are never
|
|
|
|
|
handled in a standalone context (unlike transactions, which exist in the mempool outside
|
|
|
|
|
of blocks).
|
|
|
|
|
|
|
|
|
|
Replacing ``hashFinalSaplingRoot`` with ``hashChainHistoryRoot`` does introduce the
|
|
|
|
|
theoretical possibility of an attack where a miner constructs a Sapling commitment tree
|
|
|
|
|
update that results in the same 32-byte value as the MMR root. We don't consider this a
|
|
|
|
|
realistic attack, both because the adversary would need to find a preimage over 32 layers
|
|
|
|
|
of Pedersen hash, and because light clients already need to update their code to include
|
|
|
|
|
the consensus branch ID for the Heartwood network upgrade, and can simultaneously make
|
|
|
|
|
changes to not rely on the value of this header field being the Sapling tree root.
|
2020-02-27 08:33:00 -08:00
|
|
|
|
|
|
|
|
|
We also considered putting ``hashChainHistoryRoot`` in the ``hashPrevBlock`` field as it
|
|
|
|
|
commits to the entire chain history, but quickly realized it would require massive
|
|
|
|
|
refactoring of the existing code base and would negatively impact performance. Reorgs in
|
|
|
|
|
particular are fragile, performance-critical, and rely on backwards iteration over the
|
|
|
|
|
chain history. If a chain were to be designed from scratch there may be some efficient
|
|
|
|
|
implementation that would join these commitments, but it is clearly not appropriate for
|
2020-02-27 08:33:38 -08:00
|
|
|
|
Zcash as it exists.
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-04-01 03:38:08 -07:00
|
|
|
|
The calculation of ``hashChainHistoryRoot`` is not well-defined for the genesis block,
|
|
|
|
|
since then :math:`n = 0` and there is no block :math:`B_{n-1}`. Also, in the case of
|
|
|
|
|
chains that activate this ZIP after genesis (including Zcash Mainnet and Testnet), the
|
|
|
|
|
``hashChainHistoryRoot`` of the activation block would commit to the whole previous epoch
|
|
|
|
|
if a special case were not made. It would be impractical to calculate this commitment all
|
|
|
|
|
at once, and so we specify that ``hashLightClientRoot`` is set to all zero bytes for that
|
|
|
|
|
block instead. The hash of the final Sapling note commitment tree root for the activation
|
|
|
|
|
block will not be encoded in that block, but will be committed to one block later in the
|
|
|
|
|
``hashLatestSaplingRoot`` field of the MMR root commitment.
|
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-02-27 13:07:00 -08:00
|
|
|
|
Security and Privacy Considerations
|
|
|
|
|
===================================
|
|
|
|
|
|
|
|
|
|
This ZIP imposes an additional validation cost on new blocks. While this validation cost
|
|
|
|
|
is small, it may exacerbate any existing DoS opportunities, particularly during abnormal
|
|
|
|
|
events like long reorgs. Fortunately, these costs are logarithmic in the number of delete
|
|
|
|
|
and append operations. In the worst case scenario, a well-resourced attacker could
|
|
|
|
|
maintain 2 chains of approximately equal length, and alternate which chain they extend.
|
|
|
|
|
This would result in repeated reorgs of increasing length.
|
|
|
|
|
|
2020-03-03 11:08:35 -08:00
|
|
|
|
Given the performance of BLAKE2b, we expect this validation cost to be negligible.
|
2020-02-27 13:07:00 -08:00
|
|
|
|
However, it seems prudent to benchmark potential MMR implementations during the
|
|
|
|
|
implementation process. Should the validation cost be higher than expected, there are
|
|
|
|
|
several potential mitigations, e.g. holding recently seen nodes in memory after a reorg.
|
|
|
|
|
|
|
|
|
|
Generally, header commitments have no impact on privacy. However, FlyClient has additional
|
|
|
|
|
security and privacy implications. Because FlyClient is a motivating factor for this ZIP,
|
|
|
|
|
it seems prudent to include a brief overview. A more in-depth security analysis of
|
|
|
|
|
FlyClient should be performed before designing a FlyClient-based light client ecosystem
|
|
|
|
|
for Zcash.
|
|
|
|
|
|
|
|
|
|
FlyClient, like all light clients, requires a connection to a light client server. That
|
|
|
|
|
server may collect information about client requests, and may use that information to
|
|
|
|
|
attempt to deanonymize clients. However, because FlyClient proofs are non-interactive and
|
|
|
|
|
publicly verifiable, they could be shared among many light clients after the initial
|
|
|
|
|
server interaction.
|
|
|
|
|
|
|
|
|
|
FlyClient proofs are probabilistic. When properly constructed, there is negligible
|
|
|
|
|
probability that a dishonest chain commitment will be accepted by the verifier. The
|
|
|
|
|
security analysis assumes adversary mining power is bounded by a known fraction of
|
|
|
|
|
combined mining power of honest nodes, and cannot drop or tamper with messages between
|
|
|
|
|
client and full nodes. It also assumes the client is connected to at least one full node
|
2020-03-23 09:31:39 -07:00
|
|
|
|
and knows the genesis block.
|
|
|
|
|
|
|
|
|
|
In addition, [#FlyClient]_ only analyses these security properties in chain models with
|
2020-03-24 07:51:29 -07:00
|
|
|
|
slowly adjusting difficulty, such as Bitcoin. That paper leaves their analysis in chains
|
|
|
|
|
with rapidly adjusting difficulty –such as Zcash or Ethereum– as an open problem, and
|
|
|
|
|
states that the FlyClient protocol provides only heuristic security guarantees in that
|
|
|
|
|
case. However, as mentioned in `FlyClient Requirements and Recommendations`_, additional
|
|
|
|
|
commitments allowing light clients to reason about application of the difficulty adjustment
|
|
|
|
|
algorithm were added in discussion with an author of the FlyClient paper. The use of these
|
|
|
|
|
fields has not been analysed in the academic security literature. It would be possible to
|
|
|
|
|
update them in a future network upgrade if further security analysis were to find any
|
|
|
|
|
deficiencies.
|
2020-02-27 13:07:00 -08:00
|
|
|
|
|
|
|
|
|
|
2020-02-29 08:13:52 -08:00
|
|
|
|
Deployment
|
|
|
|
|
==========
|
|
|
|
|
|
2020-04-01 03:38:08 -07:00
|
|
|
|
On the Zcash Mainnet and Testnet, this proposal will be deployed with the Heartwood
|
|
|
|
|
network upgrade. [#zip-0250]_
|
2020-02-29 08:13:52 -08:00
|
|
|
|
|
2021-03-29 15:19:13 -07:00
|
|
|
|
Additional fields are added on activation of the NU5 network upgrade [#zip-0252]_,
|
|
|
|
|
to support the new Orchard shielded protocol.
|
|
|
|
|
|
2020-02-29 08:13:52 -08:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
Additional Reading
|
2020-02-27 12:52:27 -08:00
|
|
|
|
==================
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
|
|
|
|
- `Flyclient enabled geth fork by FlyClient authors <https://github.com/mahdiz/flyeth>`_
|
2021-09-09 07:02:46 -07:00
|
|
|
|
- `Draft ECIP-1055: Succinct PoW Using Merkle Mountain Ranges <https://github.com/etclabscore/ECIPs/blob/f37acdbb392e18a61c8a6600ca1c4a2edee5813d/ECLIPs/ECIP-draft_succinct-proofs.md>`_
|
|
|
|
|
- `Grin project MMR implementation in Rust <https://github.com/mimblewimble/grin/tree/master/core/src/core/pmmr>`_
|
|
|
|
|
- `Tari Project MMR implementation in Rust <https://github.com/tari-project/tari/tree/development/base_layer/mmr>`_
|
2019-06-28 03:27:46 -07:00
|
|
|
|
- `Beam Project MMR implementation in C++ <https://github.com/BeamMW/beam/blob/master/core/merkle.cpp>`_
|
|
|
|
|
- `Mimblewimble MMR docs <https://github.com/mimblewimble/grin/blob/master/doc/mmr.md>`_
|
|
|
|
|
- `MMR Python implementation <https://github.com/proofchains/python-proofmarshal/blob/master/proofmarshal/mmr.py>`_
|
2021-09-09 07:02:46 -07:00
|
|
|
|
- `Tari MMR documentation <https://docs.rs/merklemountainrange/0.0.1/src/merklemountainrange/lib.rs.html>`_
|
2019-06-28 03:27:46 -07:00
|
|
|
|
- `opentimestamps-server Merkle Mountain Range documentation <https://github.com/opentimestamps/opentimestamps-server/blob/master/doc/merkle-mountain-range.md>`_
|
|
|
|
|
|
2020-02-27 12:52:27 -08:00
|
|
|
|
|
2019-06-28 03:27:46 -07:00
|
|
|
|
References
|
2020-02-27 12:52:27 -08:00
|
|
|
|
==========
|
2019-06-28 03:27:46 -07:00
|
|
|
|
|
2020-11-09 07:59:51 -08:00
|
|
|
|
.. [#RFC2119] `RFC 2119: Key words for use in RFCs to Indicate Requirement Levels <https://www.rfc-editor.org/rfc/rfc2119.html>`_
|
2021-10-01 17:06:21 -07:00
|
|
|
|
.. [#FlyClient] `FlyClient protocol <https://eprint.iacr.org/2019/226>`_
|
|
|
|
|
.. [#protocol-blockheader] `Zcash Protocol Specification, Version 2021.2.16. Section 7.6: Block Header Encoding and Consensus <protocol/protocol.pdf#blockheader>`_
|
|
|
|
|
.. [#protocol-workdef] `Zcash Protocol Specification, Version 2021.2.16. Section 7.7.5: Definition of Work <protocol/protocol.pdf#workdef>`_
|
2020-02-27 08:33:38 -08:00
|
|
|
|
.. [#zcashBlock] `Zcash block primitive <https://github.com/zcash/zcash/blob/master/src/primitives/block.h>`_
|
2019-06-28 03:27:46 -07:00
|
|
|
|
.. [#mimblewimble] `MimbleWimble Grin MMR implementation <https://github.com/mimblewimble/grin/blob/aedac483f5a116b91a8baf6acffd70e5f980b8cc/core/src/core/pmmr/pmmr.rs>`_
|
2020-03-03 20:14:46 -08:00
|
|
|
|
.. [#bip-0034] `BIP 34: Block v2, Height in Coinbase <https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki>`_
|
2020-03-17 07:21:23 -07:00
|
|
|
|
.. [#zip-0200] `ZIP 200: Network Upgrade Mechanism <zip-0200.rst>`_
|
2021-03-29 15:19:13 -07:00
|
|
|
|
.. [#zip-0244] `ZIP 244: Transaction Identifier Non-Malleability <zip-0244.rst>`_
|
2020-03-17 07:21:23 -07:00
|
|
|
|
.. [#zip-0250] `ZIP 250: Deployment of the Heartwood Network Upgrade <zip-0250.rst>`_
|
2021-03-29 15:19:13 -07:00
|
|
|
|
.. [#zip-0252] `ZIP 252: Deployment of the NU5 Network Upgrade <zip-0252.rst>`_
|
2020-02-29 08:13:52 -08:00
|
|
|
|
.. [#zip-0301] `ZIP 301: Zcash Stratum Protocol <https://github.com/zcash/zips/pull/78>`_
|
|
|
|
|
.. [#zip-0307] `ZIP 307: Light Client Protocol for Payment Detection <https://github.com/zcash/zips/pull/226>`_
|