zips/zip-0222.rst

::

  ZIP: 222
  Title: Transparent Zcash Extensions
  Owners: Jack Grigg <jack@electriccoin.co>
  Credits: Zaki Manian <zaki@manian.org>
           Daira Hopwood <daira@electriccoin.co>
           Sean Bowe <sean@electriccoin.co>
  Status: Draft
  Category: Consensus
  Created: 2019-07-01
  License: MIT


Terminology
===========

The key words "MUST" and "MAY" in this document are to be interpreted as described in
RFC 2119. [#RFC2119]_

The term "network upgrade" in this document is to be interpreted as described in ZIP 200
[#zip-0200]_.

The term "Sapling" in this document is to be interpreted as described in ZIP 205
[#zip-0205]_.

Abstract
========

This proposal defines a modification to the consensus rules that enables whitelisted
complex forms of transparent output predicates ("transparent programs") to be deployed
in network upgrades, while ensuring that a transaction's ID is non-malleable if the
transaction uses any transparent programs.


Motivation
==========

Zcash transactions have three types of outputs: Sapling shielded outputs, Sprout shielded
outputs, and transparent outputs. All three can by design be "encumbered" by a spending
key - a user creating a transaction that spends the output must prove that they have
access to the spending key controlling the output. For Sprout shielded outputs this was
part of the zero-knowledge proof; for transparent and Sapling shielded outputs, the user
creates a signature with the spending key.

In some situations, more complex predicates are required. Zcash does not yet have any form
of shielded programmability, and thus users rely on the script system inherited from
Bitcoin to encumber transparent outputs. But even this is insufficient as-is for some
desired use cases. In particular, the Bolt protocol [#zip-draft-bolt]_ requires a custom
opcode, because the script system does not have the necessary primitives; this requires a
change to the consensus rules.

Additionally, some off-chain protocols require that transaction IDs be non-malleable in
order to be secure. Zcash transaction IDs are currently malleable in several ways:

- The ``scriptSig`` field of transparent inputs is not part of the data that is hashed for
  signing (as the signature is placed in that field), but is part of the data that is
  hashed to produce the transaction ID. An adversary can modify this field in a variety of
  ways that do not functionally alter it, but change its binary representation. This is
  the core transaction malleability problem that Bitcoin mitigated with their Segregated
  Witness soft fork [#bip-0141]_ (for transactions that opt-in to SegWit-only inputs).

- The Ed25519 signature ``joinSplitSig`` is malleable. The known sources of malleability
  are:

  - The y-coordinate of the R component of the signature is not required to be canonical.
  - There are equivalent points for R because its order may be greater than the subgroup
    order ``ell``.

  These malleability sources correspond to the verification process in ``libsodium``. Note
  that the public key ``joinSplitPubKey`` is also not required to be canonical, but this
  is not a source of malleability because the public key is hashed as given in the
  transaction when verifying the signature.

- RedJubjub signatures, as currently enforced by the consensus rules, are malleable. This
  is because the signature verification algorithm [#redjubjub]_ multiplies through by the
  cofactor before the identity check (which ensures that the single and batch verification
  algorithms agree on the set of valid signatures). An adversary can modify the signature
  by adding a nonzero point of small order to R, leaving the signature valid but altering
  its binary representation.

  - TODO: Upon further investigation, it looks like RedJubjub is not actually malleable
    for the above reason. We are analysing this and will update the proposal when we are
    certain. It is also possible that Ed25519 might also be non-malleable for a similar
    reason, but verifying this would require very careful inspection of libsodium; it is
    simpler to just exclude them.

Shielded programmability would require significant changes across the ecosystem. In the
interim, we propose in this ZIP a transparent form of Zcash extensibility that could later
be migrated to shielded programmability.


Conventions
===========

We use the following notation as defined in the Zcash protocol specification
[#spec-notation]_, reproduced here for convenience:

- length(*S*) means the length of (number of elements in) *S*.

- { *a* .. *b* } means the set or type of integers from *a* through *b* inclusive.

- *a* || *b* means the concatenation of sequences *a* then *b*.

We reproduce the following conversion function from ZIP 32 [#zip-0032]_:

- I2LEOSP\ :sub:`l`\ (*k*) is the byte sequence *S* of length *l*/8 representing in little-endian order the
  integer *k* in range {0..2\ :sup:`l`\ -1}.


Specification
=============

Transparent program
-------------------

A transparent program is defined by four properties:

- A 1-byte program type.
- A byte sequence ``predicate`` containing a prefix-free encoding describing the
  predicate.
- A non-malleable byte sequence ``witness`` containing a prefix-free encoding of evidence
  satisfying the predicate.
- A deterministic verification algorithm ``verify_program`` that takes as arguments
  ``predicate``, ``witness``, and a context object providing deterministic public
  information about the transaction as well as the block chain (up to and including the
  block that the transaction is mined in). It returns ``true`` if the predicate is
  satisfied, and ``false`` otherwise.

ZIPs that define a new transparent program must completely specify all four of these
properties.

Program types ``{0xfd..0xff}`` are reserved for future extensions to this schema.

The lengths of ``predicate`` and ``witness`` are not required to be constant length, but
MUST be solely determined by the program type.

Encoding in transactions
------------------------

We define a new transaction format that is both non-malleable and contains two new fields:

- ``tze_outputs``: A list of pairs of:

  - The value being encumbered.
  - The predicate that must be satisfied to spend the value.

- ``tze_inputs``: A list of pairs of:

  - An outpoint referencing a prior predicate.
  - A witness that satisfies it.

Non-malleability of the transaction format will be specified in a separate ZIP.

TODO: Finish encoding specification.

Consensus rules
---------------

Once the TODO network upgrade activates, the following new consensus rules are enforced:

- For each ``(outpoint, witness)`` pair in ``tze_inputs``:

  - ``outpoint`` MUST reference a predicate of the same type and mode in an already-mined
    transaction.
  - ``verify_program(predicate, witness, context)`` MUST return ``true``.

Rationale
=========

An earlier draft version of this ZIP stored the payloads inside transparent inputs and
outputs. This coupled the extension logic too tightly to the transparent address logic,
and was replaced with dedicated transaction fields.


Security and Privacy Considerations
===================================

This ZIP assumes that the base transaction format is non-malleable. However, the
``predicate`` and ``witness`` byte sequences are treated here as opaque. It is the
responsibility of ``verify_program`` to enforce the following:

- ``witness`` MUST be non-malleable: any malleation MUST cause ``verify_program`` to
  return ``false``.
- The output of ``verify_program(predicate, witness, context)`` MUST be deterministic.

ZIPs defining new program types MUST include a section explaining how any potential
sources of malleability are handled.


Reference Implementation
========================

TBD


Acknowledgements
================

The handler semantics of ``verify_program`` were suggested by Zaki Manian, drawing on the
design of Cosmos. Daira Hopwood and Sean Bowe gave useful feedback on an early draft of
this ZIP, and helped to analyse the various sources of transaction ID malleability.

We would also like to thank the numerous other individuals who participated in discussions
at Zcon1 that led to the earlier draft version of this ZIP.


References
==========

.. [#RFC2119] `Key words for use in RFCs to Indicate Requirement Levels <https://tools.ietf.org/html/rfc2119>`_
.. [#zip-0200] `ZIP 200: Network Upgrade Activation Mechanism <https://github.com/zcash/zips/blob/master/zip-0200.rst>`_
.. [#zip-0205] `ZIP 205: Deployment of the Sapling Network Upgrade <https://github.com/zcash/zips/blob/master/zip-0205.rst>`_
.. [#zip-draft-bolt] `Draft ZIP: Add support for Blind Off-chain Lightweight Transactions (Bolt) protocol <https://github.com/zcash/zips/pull/216>`_
.. [#bip-0141] `BIP 141: Segregated Witness (Consensus layer) <https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki>`_
.. [#redjubjub] `Section 5.4.6: RedDSA and RedJubjub. Zcash Protocol Specification, Version 2019.0.2 [Overwinter+Sapling] <https://github.com/zcash/zips/blob/master/protocol/protocol.pdf>`_
.. [#spec-notation] `Section 2: Notation. Zcash Protocol Specification, Version 2019.0.2 [Overwinter+Sapling] <https://github.com/zcash/zips/blob/master/protocol/protocol.pdf>`_
.. [#zip-0032] `ZIP 32: Shielded Hierarchical Deterministic Wallets <https://github.com/zcash/zips/blob/master/zip-0032.rst>`_
.. [#batch-verification] `Section B.1: RedDSA batch verification. Zcash Protocol Specification, Version 2019.0.2 [Overwinter+Sapling] <https://github.com/zcash/zips/blob/master/protocol/protocol.pdf>`_