zips/zip-0222.rst

::

  ZIP: 222
  Title: Transparent Zcash Extensions
  Owners: Jack Grigg <jack@electriccoin.co>
  Credits: Zaki Manian <zaki@manian.org>
           Daira Hopwood <daira@electriccoin.co>
           Sean Bowe <sean@electriccoin.co>
  Status: Draft
  Category: Consensus
  Created: 2019-07-01
  License: MIT


Terminology
===========

The key words "MUST" and "MAY" in this document are to be interpreted as described in
RFC 2119. [#RFC2119]_

The term "network upgrade" in this document is to be interpreted as described in ZIP 200
[#zip-0200]_.

The term "prefix-free" in this document is to be interpreted as to mean that no valid
encoding of a value may have the same binary representation as any prefix of
the binary encoding of another value of the same type.

The term "non-malleable" in this document is to be interpreted as described in ZIP 244
[#zip-0244]_.

The value MAX_MONEY is as defined in section 5.3 **Constants** of the Zcash Protocol
Specification [#protocol]_. 

Abstract
========

This proposal defines a modification to the consensus rules that enables complex forms of
transparent output preconditions to be deployed in network upgrades. This in turn enables the
creation of "transparent Zcash extensions" that augment the network's functionality in a
carefully-defined and auditable fashion.

Motivation
==========

Zcash supports a limited set preconditions that may be imposed upon how funds, both
transparent and shielded, may be spent. Spending limitation on transparent funds are
defined by what may be encoded in Bitcoin script, and spending of shielded funds is
even more restricted. As such, some use cases (for example, integration of BOLT support
[#zip-draft-bolt]_) are not yet supportable. 

Transparent zcash extensions are intended to make it possible to incrementally
add new functionality without modifying interpretation of the existing bitcoin
script, which is complex and potentially error-prone. Extensions may also serve 
as laboratories for evaluating demand for functionality which may eventually be 
candidates for inclusion in the consensus rules for shielded transactions.

Conventions
===========

We use the following notation as defined in the Zcash protocol specification
[#spec-notation]_, reproduced here for convenience:

- length(*S*) means the length of (number of elements in) *S*.

- { *a* .. *b* } means the set or type of integers from *a* through *b* inclusive.

- *a* || *b* means the concatenation of sequences *a* then *b*.

We reproduce the following conversion function from ZIP 32 [#zip-0032]_:

- :math:`\mathsf{I2LEOSP}_{\ell(k)}` is the byte sequence *S* of length :math:`\ell/8` representing in little-endian order the
  integer *k* in range :math:`\{0..2^\ell -1\}`.

Definitions
===========

encumber
    To set conditions that must be satisfied in order to spend some or all of a 
    transaction's outputs.

precondition
    A challenge that must be satisfied in order to gain spending authority over 
    an encumbered amount. Also an encoding of such a challenge.

witness
    Evidence to be evaluated against the challenge encoded by a precondition.

Specification
=============

Transparent extension
---------------------

Transparent extensions are modular software components that are distributed as
part of the zcashd consensus code. An extension defines interpretation rules
for several new pieces of data that are included as part of a transaction. 

A transparent extension is identified by a numeric ``type``. A transparent
extension may also have several modes of operation, corresponding to different
kinds of encumbrance within the extension's overall protocol. The extension is
responsible for providing mode-specific parsing and serialization of the
following data fields:

The following three values are made available to the extension (in addition to ``type``):

- A numeric ``mode``.
- A byte sequence ``precondition`` containing an encoding of the precondition that is
  prefix-free within this ``(type, mode)``.
- A non-malleable byte sequence ``witness`` containing an encoding of evidence
  purporting to satisfy the precondition that is prefix-free within this ``(type, mode)``.

The extension must implement:

- A deterministic verification algorithm ``tze_verify`` that takes as arguments ``mode``,
  ``precondition``, ``witness``, and a context object providing deterministic public
  information about the transaction as well as the block chain (up to and including the
  block that the transaction is mined in). It returns ``true`` if the precondition is
  satisfied, and ``false`` otherwise. 
- An extension MAY request that arbitrary public information about the
  transaction and block chain be included in the context object provided to it;
  these requirements MUST be satisfied by a caller integrating the extension at
  the integration point. Implmentations may restrict the information provided to
  that which may be provided in an efficient manner.

ZIPs that define a new transparent extension MUST completely specify the structure of all 
three of the specified values for each defined mode, as well as the behavior of tze_verify
and any contextual information required.

The lengths of ``precondition`` and ``witness`` are not required to be constant length, but
MUST be solely determined by the pair ``(type, mode)``.

Introduction of TZEs produces a new transparent unspent outpoint set, such that indexes
into this set of outpoints may be referred to by TZE inpoints in spending transactions.

Encoding in transactions
------------------------

We define a new transaction format that is non-malleable and contains two new fields:

- ``tze_outputs``: A list of pairs of:

  - The value being encumbered.
  - The precondition that must be satisfied to spend the value.

- ``tze_inputs``: A list of pairs of:

  - An outpoint referencing a prior precondition.
  - A witness that satisfies it.

Non-malleability of the transaction format will be specified in a separate ZIP.

A new version <TBD> transaction format and corresponding version group
identifier <TBD> will be introduced in the hard-fork network upgrade that
introduces TZE functionality. The version <TBD> format differs from the version
4 format as follows: a length-prefixed encoding of TZE inputs and outputs are
added to the serialized transaction format immediately following the fields
representing transparent inputs and outputs.

======== ====================== =========================== ===============
Version  Field                  Description                 Type
======== ====================== =========================== ===============
... as before
>= 1     ``tx_in_count``        variable-length integer     ``compactSize``
>= 1     ``tx_in``              list of inputs              ``vector``
>= 1     ``tx_out_count``       variable-length integer     ``compactSize``
>= 1     ``tx_out``             list of outputs             ``vector``
>= <TBD> ``tze_in_count``       variable-length integer     ``compactSize``
>= <TBD> ``tze_in``             list of TZE inputs          ``vector``
>= <TBD> ``tze_out_count``      variable-length integer     ``compactSize``
>= <TBD> ``tze_out``            list of TZE outputs         ``vector``
>= 1     ``lock_time``          block height or timestamp   ``uint32``
... as before
======== ====================== =========================== ===============

Both ``tze_in`` and ``tze_out`` vectors make use of the common serialized
form ``tze_data`` described below. Serialization of all integral and vector
types is as with Bitcoin.

``tze_data`` encoding:

====================== ==================================== =====================
Field                  Description                          Type           
====================== ==================================== =====================
``tze_id``             extension ``type``                   ``uint64``    
``tze_mode``           extension ``mode``                   ``uint64``
``tze_data_len``       length of precondition/witness data  ``compactSize``
``tze_data``           serialized precondition/witness data ``witness_len`` bytes
====================== ==================================== =====================

TZE Input Encoding:

====================== ==================================== ===============
Field                  Description                          Type           
====================== ==================================== ===============
``prevout_hash``       hash of previous txid                ``uint256``    
``prevout_in``         index into previous txn's outputs    ``uint32``
``witness``            witness for prevout's precondition   ``tze_data``
====================== ==================================== ===============


TZE Output Encoding:

====================== ==================================== ===============
Field                  Description                          Type           
====================== ==================================== ===============
``amount``             spendable amount, in zatoshi         ``int64``    
``precondition``       precondition encumbering ``amount``  ``tze_data``
====================== ==================================== ===============


Consensus rules
---------------

Once the TODO network upgrade activates, the following new consensus rules are enforced:

- For each ``(outpoint, witness)`` pair in ``tze_inputs``:

  - ``outpoint`` MUST reference a precondition of the same type and mode in an already-mined
    transaction.
  - ``tze_verify(mode, precondition, witness, context)`` MUST return ``true``.

- If a transaction has non-empty ``tze_inputs`` and non-empty ``tze_outputs``, then every
  element in both fields MUST have the same ``type``.

- Non-coinbase transactions MUST have at least one of the following:
  - nonempty transparent inputs
  - nonempty shielded inputs
  - nonempty ``tze_inputs``

- Transactions MUST have at least one of the following:
  - nonempty transparent outputs
  - nonempty shielded outputs
  - nonempty ``tze_outputs``

- All ``amount`` field values of ``tze_output`` records MUST be nonnegative and less than
  MAX_MONEY.

- The sum of amounts associated with all transparent, shielded, and TZE outputs within 
  a transaction MUST NOT exceed the sum of the amounts associated with transaction inputs.

Changes to signatures over transaction digests
----------------------------------------------

The newly added parts of the transaction will be included in transaction 
digests for signatures. See [#zip-0244]_ for the specification of the new
digests.

Rationale
=========

Transactions that have both TZE inputs and outputs are required to use a single extension
type, in order to prevent cross-protocol attacks. The downside is that this prevents all
TZE-encumbered value from being spent directly into a different TZE type; the value needs
to go through a regular address in between. This restriction might be relaxed in future
ZIPs for specific combinations of ``(type, mode)`` pairs that have been analyzed for
cross-protocol attacks, but we opt here for a fail-safe default behaviour.

- TODO: Maybe also disallow regular (transparent, Sprout, and Sapling) spends/outputs in
  transactions that have TZE inputs and outputs?

An earlier draft version of this ZIP stored the payloads inside transparent inputs and
outputs. Although this had the advantage of not requiring a transaction format change,
the consensus rules were significantly more complicated, and the design coupled the
extension logic too tightly to the transparent address logic. Instead, this ZIP uses
dedicated transaction fields.


Security and Privacy Considerations
===================================

This ZIP assumes that the base transaction format is non-malleable. However, the
``precondition`` and ``witness`` byte sequences are treated here as opaque. It is the
responsibility of ``tze_verify`` to enforce the following:

- ``witness`` MUST be non-malleable: any malleation MUST cause ``tze_verify`` to
  return ``false``.
- The output of ``tze_verify(mode, precondition, witness, context)`` MUST be deterministic.

ZIPs defining new extension types MUST include a section explaining how any potential
sources of malleability are handled.

This ZIP includes restrictions to prevent cross-protocol attacks, but the extension mode
is another potential attack surface. It is the responsibility of ZIPs defining new
extensions to examine the potential for cross-mode attacks within their security analysis,
and/or appropriately restrict which modes may be combined within a single transaction.


Reference Implementation
========================

TBD (link to zcashd pr)


Acknowledgements
================

The handler semantics of ``tze_verify`` were suggested by Zaki Manian, drawing on the
design of Cosmos. Daira Hopwood and Sean Bowe gave useful feedback on an early draft of
this ZIP, and helped to analyse the various sources of transaction ID malleability.

We would also like to thank the numerous other individuals who participated in discussions
at Zcon1 that led to the earlier draft version of this ZIP.


References
==========

.. [#RFC2119] `Key words for use in RFCs to Indicate Requirement Levels <https://tools.ietf.org/html/rfc2119>`_
.. [#zip-0200] `ZIP 200: Network Upgrade Activation Mechanism <https://github.com/zcash/zips/blob/master/zip-0200.rst>`_
.. [#zip-0244] `ZIP 244: Transaction Non-Malleability Support <https://github.com/zcash/zips/blob/master/zip-0244.rst>`_
.. [#zip-draft-bolt] `Draft ZIP: Add support for Blind Off-chain Lightweight Transactions (Bolt) protocol <https://github.com/zcash/zips/pull/216>`_
.. [#spec-notation] `Section 2: Notation. Zcash Protocol Specification, Version 2019.0.2 [Overwinter+Sapling] <https://github.com/zcash/zips/blob/master/protocol/protocol.pdf>`_
.. [#zip-0032] `ZIP 32: Shielded Hierarchical Deterministic Wallets <https://github.com/zcash/zips/blob/master/zip-0032.rst>`_
.. [#protocol] `Zcash Protocol Specification, Version 2020.1.11 [Overwinter+Sapling+Blossom+Heartwood] <https://github.com/zcash/zips/blob/master/protocol/protocol.pdf>`_