zec
/
zips
mirror of https://github.com/zcash/zips.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
zips/zip-0317.rst

584 lines
25 KiB
ReStructuredText
Raw Permalink Blame History

::
ZIP: 317
Title: Proportional Transfer Fee Mechanism
Owners: Aditya Bharadwaj <nighthawk24@gmail.com>
Daira Hopwood <daira@electriccoin.co>
Credits: Madars Virza
Kris Nuttycombe
Jack Grigg
Francisco Gindre
Status: Active
Category: Standards / Wallet
Obsoletes: ZIP 313
Created: 2022-08-15
License: MIT
Discussions-To: <https://forum.zcashcommunity.com/t/zip-proportional-output-fee-mechanism-pofm/42808>
Pull-Request: <https://github.com/zcash/zips/pull/631>
Terminology
===========
The key words "SHOULD", "SHOULD NOT", "RECOMMENDED", and "MAY" in this document
are to be interpreted as described in BCP 14 [#BCP14]_ when, and only when, they
appear in all capitals.
The term "conventional transaction fee" in this document is in reference
to the value of a transaction fee that is conventionally used by wallets,
and that a user can reasonably expect miners on the Zcash network to accept
for including a transaction in a block.
The terms "Mainnet, "Testnet", and "zatoshi" in this document are defined
as in [#protocol-networks]_.
Abstract
========
The goal of this ZIP is to change the conventional fees for transactions
by making them dependent on the number of inputs and outputs in a transaction,
and to get buy-in for this change from wallet developers, miners and Zcash users.
Motivation
==========
In light of recent Mainnet network activity, it is time to review and update
the standard 1,000 zatoshi transaction fee set in ZIP 313 [#zip-0313]_.
The conventional transaction fee presently is 0.00001 ZEC or 1,000 zatoshis, as
specified in ZIP 313. This allowed exploration of novel use cases of the Zcash
blockchain. The Zcash network has operated for almost 2 years at a conventional
transaction fee of 1,000 zatoshis, without consideration for the total number
of inputs and outputs in each transaction. Under this conventional fee, some
usage of the chain has been characterized by high-output transactions with
1,100 outputs, paying the same conventional fee as a transaction with 2 outputs.
The objective of the new fee policy, once it is enforced, is for fees paid by
transactions to fairly reflect the processing costs that their inputs and outputs
impose on various participants in the network. This will tend to discourage
usage patterns that cause either intentional or unintentional denial of service,
while still allowing low fees for regular transaction use cases.
Requirements
============
* The conventional fee formula should not favour or discriminate against any
of the Orchard, Sapling, or transparent protocols.
* The fee for a transaction should scale linearly with the number of inputs
and/or outputs.
* Users should not be penalised for sending transactions constructed
with padding of inputs and outputs to reduce information leakage.
(The default policy employed by zcashd and the mobile SDKs pads to
two inputs and two outputs for each shielded pool used by the transaction).
* Users should be able to spend a small number of UTXOs or notes with value
below the marginal fee per input.
Specification
=============
Notation
--------
.. |br| raw:: html
<br/>
Let :math:`\mathsf{min}(a, b)` be the lesser of :math:`a` and :math:`b\!`. |br|
Let :math:`\mathsf{max}(a, b)` be the greater of :math:`a` and :math:`b\!`. |br|
Let :math:`\mathsf{floor}(x)` be the largest integer :math:`\leq x\!`. |br|
Let :math:`\mathsf{ceiling}(x)` be the smallest integer :math:`\geq x\!`.
Fee calculation
---------------
This specification defines several parameters that are used to calculate the
conventional fee:
===================================== ============= ==============================================
Parameter Value Units
===================================== ============= ==============================================
:math:`marginal\_fee` :math:`5000` zatoshis per logical action (as defined below)
:math:`grace\_actions` :math:`2` logical actions
:math:`p2pkh\_standard\_input\_size` :math:`150` bytes
:math:`p2pkh\_standard\_output\_size` :math:`34` bytes
===================================== ============= ==============================================
Wallets implementing this specification SHOULD use a conventional fee
calculated in zatoshis per the following formula:
.. math::
\begin{array}{rcl}
contribution_{\,\mathsf{Transparent}} &=&
\mathsf{max}\big(\mathsf{ceiling}\big(\frac{tx\_in\_total\_size}{p2pkh\_standard\_input\_size}\big),\,
\mathsf{ceiling}\big(\frac{tx\_out\_total\_size}{p2pkh\_standard\_output\_size}\big)\big) \\
contribution_{\,\rlap{\mathsf{Sprout}}\phantom{\mathsf{Transparent}}} &=& 2 \cdot nJoinSplit \\
contribution_{\,\rlap{\mathsf{Sapling}}\phantom{\mathsf{Transparent}}} &=& \mathsf{max}(nSpendsSapling,\, nOutputsSapling) \\
contribution_{\,\rlap{\mathsf{Orchard}}\phantom{\mathsf{Transparent}}} &=& nActionsOrchard \\
\\
logical\_actions &=& contribution_{\,\mathsf{Transparent}} +
contribution_{\,\mathsf{Sprout}} +
contribution_{\,\mathsf{Sapling}} +
contribution_{\,\mathsf{Orchard}} \\
conventional\_fee &=& marginal\_fee \cdot \mathsf{max}(grace\_actions,\, logical\_actions)
\end{array}
The inputs to this formula are taken from transaction fields defined in the Zcash protocol
specification [#protocol-txnencoding]_:
============================ ====== ===========================================
Input Units Description
============================ ====== ===========================================
:math:`tx\_in\_total\_size` bytes total size in bytes of the ``tx_in`` field
:math:`tx\_out\_total\_size` bytes total size in bytes of the ``tx_out`` field
:math:`nJoinSplit` number the number of Sprout JoinSplits
:math:`nSpendsSapling` number the number of Sapling spends
:math:`nOutputsSapling` number the number of Sapling outputs
:math:`nActionsOrchard` number the number of Orchard actions
============================ ====== ===========================================
It is not a consensus requirement that fees follow this formula; however,
wallets SHOULD create transactions that pay this fee, in order to reduce
information leakage, unless overridden by the user.
Rationale for logical actions
'''''''''''''''''''''''''''''
.. raw:: html
<details>
<summary>Click to show/hide</summary>
The intention is to make the fee paid for a transaction depend on its
impact on the network, without discriminating between different protocols
(Orchard, Sapling, or transparent). The impact on the network depends on
the numbers of inputs and outputs.
A previous proposal used :math:`inputs + outputs` instead of logical actions.
This would have disadvantaged Orchard transactions, as a result of an
Orchard Action combining an input and an output. The effect of this
combining is that Orchard requires padding of either inputs or outputs
to ensure that the number of inputs and outputs are the same. Usage of
Sapling and transparent protocols does not require this padding, and
so this could have effectively discriminated against Orchard.
.. raw:: html
</details>
Rationale for the chosen parameters
'''''''''''''''''''''''''''''''''''
.. raw:: html
<details>
<summary>Click to show/hide</summary>
Grace Actions
~~~~~~~~~~~~~
**Why not just charge per-action, without a grace window?**
* This ensures that there is no penalty to padding a 1-action
transaction to a 2-action transaction. Such padding is desirable
to reduce information leakage from input and output arity, and
is the standard approach used by `zcashd` and the mobile SDK
transaction builder.
* Without a grace window, an input with value below the marginal
fee would never be worth including in the resulting transaction.
With a grace window, an input with value below :math:`marginal\_fee`
*is* worth including, if a second input is available that covers
both the primary output amount and the conventional transaction
fee.
**Why a grace window of 2?**
A 1-in, 2-out (or 2-action) transaction is the smallest possible
transaction that permits both an output to a recipient, and a
change output. However, as stated above, `zcashd` and the mobile
SDK transaction builder will pad the number of inputs to at least 2.
Let :math:`min\_actions` be the minimum number of logical actions
that can be used to execute economically relevant transactions that
produce change. Due to the aforementioned padding, :math:`min\_actions = 2`.
Having a grace window size greater than :math:`min\_actions` would
increase the cost to create such a minimal transaction. If the
cost we believe that users will tolerate for a minimal transaction
is :math:`B`, then possible choices of :math:`marginal\_fee` are
bounded above by :math:`B / \max(min\_actions, grace\_actions)`.
Therefore, the optimal choice of :math:`grace\_actions` to maximize
the per-logical-action cost of denial-of-service attacks for a given
:math:`B`, is :math:`grace\_actions = min\_actions = 2`. This also
ensures that a denial-of-service adversary does not gain a
significant per-logical-action cost advantage by using transactions
with a smaller or larger number of logical actions.
Transparent Contribution
~~~~~~~~~~~~~~~~~~~~~~~~
The specified formula calculates the contribution of transparent inputs
and outputs based on their total size relative to a typical input or
output. Another considered approach was to calculate this contribution
simply as :math:`\mathsf{max}(transparent\_inputs, transparent\_outputs)`.
However, this would allow a denial-of-service adversary to create
transactions with transparent components containing arbitrarily large
scripts.
The chosen values for :math:`p2pkh\_standard\_input\_size` and
:math:`p2pkh\_standard\_output\_size` are based on the maximum encoded
length for P2PKH inputs and outputs, as follows:
* :math:`p2pkh\_standard\_input\_size`
* outpoint: 36 bytes
* script: 110 bytes
* 1 (overall length) + 1 (signature length) + 72 (signature) + 1 (sighash type) + 1 (pubkey length) + 33 (pubkey) + 1 (margin)
* sequence: 4 bytes
* :math:`p2pkh\_standard\_output\_size`
* value: 8 bytes
* script: 26 bytes
* 1 (script length) + 25 (P2PKH script)
P2SH outputs are smaller than P2PKH outputs, but P2SH inputs
may be larger than P2PKH inputs. For example a 2-of-3 multisig
input is around 70% larger, and is counted as such when computing
the number of logical actions.
Marginal Fee
~~~~~~~~~~~~
This returns the conventional fee for a minimal transaction (as
described above) to the original conventional fee of 10000 zatoshis
specified in [#zip-0313]_, and imposes a non-trivial cost for
potential denial-of-service attacks.
.. raw:: html
</details>
Transaction relaying
--------------------
zcashd, zebrad, and potentially other node implementations, implement
fee-based restrictions on relaying of mempool transactions. Nodes that
normally relay transactions are expected to do so for transactions that pay
at least the conventional fee as specified in this ZIP, unless there are
other reasons not to do so for robustness or denial-of-service mitigation.
If a transaction has more than :math:`block\_unpaid\_action\_limit` "unpaid actions"
as defined by the `Recommended algorithm for block template construction`_,
it will never be mined by that algorithm. Nodes MAY drop these transactions,
or transactions with more unpaid actions than a configurable limit (see the
`Deployment`_ section for actual behaviour of node implementations).
Mempool size limiting
---------------------
zcashd and zebrad limit the size of the mempool as described in [#zip-0401]_.
This specifies a :math:`low\_fee\_penalty` that is added to the "eviction weight"
if the transaction pays a fee less than the conventional transaction fee.
This threshold is modified to use the new conventional fee formula.
Block production
----------------
Miners, mining pools, and other block producers, select transactions for
inclusion in blocks using a variety of criteria. The algorithm in the
following section is planned to be implemented by `zcashd` and `zebrad`.
Recommended algorithm for block template construction
'''''''''''''''''''''''''''''''''''''''''''''''''''''
Define constants :math:`weight\_ratio\_cap = 4` and
:math:`block\_unpaid\_action\_limit = 50\!`.
Let :math:`conventional\_fee(tx)` be the conventional fee for transaction
:math:`tx` calculated according to the section `Fee calculation`_.
Let :math:`unpaid\_actions(tx) = \begin{cases}\mathsf{max}\!\left(0,\, \mathsf{max}(grace\_actions,\, tx.\!logical\_actions) - \mathsf{floor}\!\left(\frac{tx.fee}{marginal\_fee}\right)\right),&\textsf{if }tx\textsf{ is a non-coinbase transaction} \\ 0,&\textsf{if }tx\textsf{ is a coinbase transaction.}\end{cases}`
Let :math:`block\_unpaid\_actions(block) = \sum_{tx \,\in\, block}\, unpaid\_actions(tx)`.
The following algorithm is RECOMMENDED for constructing a block template
from a set of transactions in a node's mempool:
1. Set the block template :math:`T` to include a placeholder for the
coinbase transaction (see Note below).
2. For each transaction :math:`tx` in the mempool, calculate
:math:`tx.\!weight\_ratio = \mathsf{min}\!\left(\frac{\mathsf{max}(1,\, tx.fee)}{conventional\_fee(tx)},\, weight\_ratio\_cap\right)\!`
and add the transaction to the set of candidate transactions.
3. Repeat while there is any candidate transaction that pays at least the
conventional fee:
a. Pick one of those transactions at random with probability in direct
proportion to its :math:`weight\_ratio\!`, and remove it from the set of
candidate transactions. Let :math:`B` be the block template :math:`T`
with this transaction included.
b. If :math:`B` would be within the block size limit and block sigop
limit [#sigop-limit]_, set :math:`T := B\!`.
4. Repeat while there is any candidate transaction:
a. Pick one of those transactions at random with probability in direct
proportion to its :math:`weight\_ratio\!`, and remove it from the set of
candidate transactions. Let :math:`B` be the block template :math:`T`
with this transaction included.
b. If :math:`B` would be within the block size limit and block sigop
limit [#sigop-limit]_ and :math:`block\_unpaid\_actions(B) \leq block\_unpaid\_action\_limit\!`,
set :math:`T := B\!`.
5. Return :math:`T\!`.
Note: In step 1, the final coinbase transaction cannot be included at this
stage because it depends on the fees paid by other transactions. In practice,
this difficulty can be overcome by reserving sufficient space and sigops to
allow modifying the coinbase transaction as needed, when testing against the
block space and block sigop limits in steps 3b and 4b.
Rationale for block template construction algorithm
'''''''''''''''''''''''''''''''''''''''''''''''''''
It is likely that not all wallets will immediately update to pay the
(generally higher) fees specified by this ZIP. In order to be able to deploy
this block template algorithm more quickly while still giving transactions
created by such wallets a reasonable chance of being mined, we allow a
limited number of "unpaid" logical actions in each block. Roughly speaking,
if a transaction falls short of paying the conventional transaction fee by
:math:`k` times the marginal fee, we count that as :math:`k` unpaid logical
actions.
Regardless of how full the mempool is (according to the ZIP 401 [#zip-0401]_
cost limiting), and regardless of what strategy a denial-of-service adversary
may use, the number of unpaid logical actions in each block is always limited
to at most :math:`block\_unpaid\_action\_limit\!`.
The weighting in step 2 does not create a situation where the adversary gains
a significant advantage over other users by paying more than the conventional
fee, for two reasons:
1. The weight ratio cap limits the relative probability of picking a given
transaction to be at most :math:`weight\_ratio\_cap` times greater than a
transaction that pays exactly the conventional fee.
2. Compare the case where the adversary pays :math:`c` times the conventional
fee for one transaction, to that where they pay the conventional fee for
:math:`c` transactions. In the former case they are more likely to get *each*
transaction into the block relative to competing transactions from other users,
*but* those transactions take up less block space, all else (e.g. choice of
input or output types) being equal. This is not what the attacker wants;
they get a transaction into the block only at the expense of leaving more
block space for the other users' transactions.
The rationale for choosing :math:`weight\_ratio\_cap = 4` is as a compromise
between not allowing any prioritization of transactions relative to those that
pay the conventional fee, and allowing arbitrary prioritization based on ability
to pay.
Calculating :math:`tx.\!weight\_ratio` in terms of :math:`\mathsf{max}(1,\, tx.\!fee)`
rather than just :math:`tx.\!fee` avoids needing to define "with probability in direct
proportion to its :math:`weight\_ratio\!`" for the case where all remaining candidate
transactions would have :math:`weight\_ratio = 0\!`.
Incentive compatibility for miners
''''''''''''''''''''''''''''''''''
Miners have an incentive to make this change because:
* it will tend to increase the fees they are due;
* fees will act as a damping factor on the time needed to process blocks,
and therefore on orphan rate.
Security and Privacy considerations
===================================
Non-standard transaction fees may reveal specific users or wallets or wallet
versions, which would reduce privacy for those specific users and the rest
of the network. However, the advantage of faster deployment weighed against
synchronizing the change in wallet behaviour at a specific block height.
Long term, the issue of fees needs to be revisited in separate future
proposals as the blocks start getting consistently full. Wallet developers
and operators should monitor the Zcash network for rapid growth in
transaction rates, and consider further changes to fee selection and/or
other scaling solutions if necessary.
Denial of Service
-----------------
A transaction-rate-based denial of service attack occurs when an attacker
generates enough transactions over a window of time to prevent legitimate
transactions from being mined, or to hinder syncing blocks for full nodes
or miners.
There are two primary protections to this kind of attack in Zcash: the
block size limit, and transaction fees. The block size limit ensures that
full nodes and miners can keep up with the blockchain even if blocks are
completely full. However, users sending legitimate transactions may not
have their transactions confirmed in a timely manner.
This proposal does not alter how fees are paid from transactions to miners.
Deployment
==========
Wallets SHOULD deploy these changes immediately. Nodes SHOULD deploy the
change to the :math:`low\_fee\_penalty` threshold described in
`Mempool size limiting`_ immediately.
Nodes supporting block template construction SHOULD deploy the new
`Recommended algorithm for block template construction`_ immediately,
and miners SHOULD use nodes that have been upgraded to this algorithm.
Node developers SHOULD coordinate on schedules for deploying restrictions
to their policies for transaction mempool acceptance and peer-to-peer
relaying. These policy changes SHOULD NOT be deployed before the changes
to block template construction for miners described in the preceding
paragraph.
Deployment in zcashd
--------------------
`zcashd` v5.5.0 implemented use of ZIP 317 fees by default for its
internal wallet in the following PRs:
* https://github.com/zcash/zcash/pull/6527 (fee computation)
* https://github.com/zcash/zcash/pull/6524 (main implementation)
* https://github.com/zcash/zcash/pull/6559 (follow-up to #6524)
* https://github.com/zcash/zcash/pull/6568 (for ``z_shieldcoinbase``)
* https://github.com/zcash/zcash/pull/6576 (follow-up to #6568)
* https://github.com/zcash/zcash/pull/6569 (for ``z_mergetoaddress``)
`zcashd` v5.5.0 implemented the `Recommended algorithm for block template construction`_
in:
* https://github.com/zcash/zcash/pull/6460 (preparation)
* https://github.com/zcash/zcash/pull/6607 (follow-up to #6460)
* https://github.com/zcash/zcash/pull/6527 (fee computation)
* https://github.com/zcash/zcash/pull/6564 (block template construction)
The value used for :math:`block\_unpaid\_action\_limit` by `zcashd`
can be overridden using the ``-blockunpaidactionlimit`` configuration
parameter.
`zcashd` v5.5.0 also implemented the change to `Mempool size limiting`_
to use the ZIP 317 fee for the low fee penalty threshold, in:
* https://github.com/zcash/zcash/pull/6564
As described in section `Transaction relaying`_, nodes MAY drop
transactions with more unpaid actions than a given limit. From
`zcashd` v5.6.0, this is controlled by the ``-txunpaidactionlimit``
configuration option, which defaults to 50 unpaid actions (the
same default as ``-blockunpaidactionlimit``). This behaviour is
implemented in:
* https://github.com/zcash/zcash/pull/6646
Note that `zcashd` also requires transactions to pay at least a
"relay threshold" fee. As part of the ZIP 317 work, this rule was
simplified for `zcashd` v5.5.0:
* https://github.com/zcash/zcash/pull/6542/files#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2
Deployment in zebra
-------------------
`zebra` does not provide a wallet, and so does not need to calculate
ZIP 317 fees in order to construct transactions.
`zebra` v1.0.0-rc.3 implemented the current `Recommended algorithm for
block template construction`_ in:
* https://github.com/ZcashFoundation/zebra/pull/5724
* https://github.com/ZcashFoundation/zebra/pull/5776 (algorithm update)
`zebra` v1.0.0-rc.2 had implemented an earlier version of this algorithm.
The value used for :math:`block\_unpaid\_action\_limit` in `zebra` is not
configurable.
`zebra` v1.0.0-rc.2 implemented the change to `Mempool size limiting`_ in:
* https://github.com/ZcashFoundation/zebra/pull/5703
`zebra` v1.0.0-rc.8 implemented `Transaction relaying`_ changes in:
* https://github.com/ZcashFoundation/zebra/pull/6556
`zebra` uses a similar relay threshold rule to `zcashd`, but additionally
enforces a minimum fee of 100 zatoshis (this differs from `zcashd` only for
valid transactions of less than 1000 bytes, assuming that `zcashd` uses its
default value for ``-minrelaytxfee``).
Considered Alternatives
=======================
This section describes alternative proposals that have not been adopted.
In previous iterations of this specification, the marginal fee was multiplied
by the sum of inputs and outputs. This means that the alternatives given
below are roughly half of what they would be under the current formula.
Possible alternatives for the parameters:
* :math:`marginal\_fee = 250` in @nuttycom's proposal.
* :math:`marginal\_fee = 1000` adapted from @madars' proposal [#madars-1]_.
* :math:`marginal\_fee = 2500` in @daira's proposal.
* :math:`marginal\_fee = 1000` for Shielded, Shielding and De-shielding
transactions, and :math:`marginal\_fee = 10000` for Transparent transactions
adapted from @nighthawk24's proposal.
(In @madars' and @nighthawk24's original proposals, there was an additional
:math:`base\_fee` parameter that caused the relationship between fee and number
of inputs/outputs to be non-proportional above the :math:`grace\_actions`
threshold. This is no longer expressible with the formula specified above.)
Endorsements
============
The following entities and developers of the listed software expressed their
support for the updated fee mechanism:
* Zecwallet Suite (Zecwallet Lite for Desktop/iOS/Android & Zecwallet FullNode)
* Nighthawk Wallet for Android & iOS
* Electric Coin Company
* Zcash Foundation
Acknowledgements
================
Thanks to Madars Virza for initially proposing a fee mechanism similar to that
proposed in this ZIP [#madars-1]_, and for finding a potential weakness in an
earlier version of the block template construction algorithm. Thanks also to
Kris Nuttycombe, Jack Grigg, Francisco Gindre, Greg Pfeil, Teor, and
Deirdre Connolly for reviews and suggested improvements.
References
==========
.. [#BCP14] `Information on BCP 14 — "RFC 2119: Key words for use in RFCs to Indicate Requirement Levels" and "RFC 8174: Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words" <https://www.rfc-editor.org/info/bcp14>`_
.. [#protocol-networks] `Zcash Protocol Specification, Version 2022.3.8. Section 3.12: Mainnet and Testnet <protocol/protocol.pdf#networks>`_
.. [#protocol-txnencoding] `Zcash Protocol Specification, Version 2022.3.8. Section 7.1: Transaction Encoding and Consensus <protocol/protocol.pdf#txnencoding>`_
.. [#sigop-limit] `zcash/zips issue #568 - Document block transparent sigops limit consensus rule <https://github.com/zcash/zips/issues/568>`_
.. [#madars-1] `Madars concrete soft-fork proposal <https://forum.zcashcommunity.com/t/zip-reduce-default-shielded-transaction-fee-to-1000-zats/37566/89>`_
.. [#zip-0313] `ZIP 313: Reduce Conventional Transaction Fee to 1000 zatoshis <zip-0313.rst>`_
.. [#zip-0401] `ZIP 401: Addressing Mempool Denial-of-Service <zip-0401.rst>`_
Reference in New Issue View Git Blame Copy Permalink