From 3d2923933d27648c8e2e6c2c0dfbfe511aded537 Mon Sep 17 00:00:00 2001 From: str4d Date: Mon, 8 Jan 2018 13:14:15 +0000 Subject: [PATCH 01/28] Overwinter signature hashing draft 1 --- drafts/str4d-overwinter-sighash/draft1.rst | 279 +++++++++++++++++++++ 1 file changed, 279 insertions(+) create mode 100644 drafts/str4d-overwinter-sighash/draft1.rst diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst new file mode 100644 index 00000000..2d75b00c --- /dev/null +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -0,0 +1,279 @@ +:: + + ZIP: 143 + Title: Transaction Signature Verification for Overwinter + Author: Jack Grigg + Daira Hopwood + Credits: Johnson Lau + Pieter Wuille + Comments-Summary: No comments yet. + Category: Process + Created: 2017-12-27 + License: MIT + + +Terminology +=========== + +The key words "MUST" and "MUST NOT" in this document are to be interpreted as described in RFC 2119. + + +Abstract +======== + +This proposal defines a new transaction digest algorithm for signature verification from the Overwinter +network upgrade, in order to minimize redundant data hashing in verification, and to cover the input value by +the signature. + + +Motivation +========== + +There are 4 ECDSA signature verification codes in the original Zcash script system: ``CHECKSIG``, +``CHECKSIGVERIFY``, ``CHECKMULTISIG``, ``CHECKMULTISIGVERIFY`` ("sigops"). According to the sighash type +(``ALL``, ``NONE``, or ``SINGLE``, possibly modified by ``ANYONECANPAY``), a transaction digest is generated +with a double SHA256 of a serialized subset of the transaction, and the signature is verified against this +digest with a given public key. The detailed procedure is described in a Bitcoin Wiki article. [#wiki-checksig]_ + +Unfortunately, there are at least 2 weaknesses in the original SignatureHash transaction digest algorithm: + +* For the verification of each signature, the amount of data hashing is proportional to the size of the + transaction. Therefore, data hashing grows in O(n\ :sup:`2`) as the number of sigops in a transaction + increases. While a 1 MB block would normally take 2 seconds to verify with an average computer in 2015, a + 1MB transaction with 5569 sigops may take 25 seconds to verify. This could be fixed by optimizing the digest + algorithm by introducing some reusable “midstate”, so the time complexity becomes O(n). [#quadratic]_ + +* The algorithm does not involve the value being spent by the input. This is usually not a problem for online + network nodes as they could request for the specified transaction to acquire the output value. For an + offline transaction signing device ("cold wallet"), however, the lack of knowledge of input amount makes it + impossible to calculate the exact amount being spent and the transaction fee. To cope with this problem a + cold wallet must also acquire the full transaction being spent, which could be a big obstacle in the + implementation of a lightweight, air-gapped wallet. By including the input value of part of the transaction + digest, a cold wallet may safely sign a transaction by learning the value from an untrusted source. In the + case that a wrong value is provided and signed, the signature would be invalid and no funds would be lost. + [#offline-wallets]_ + +Deploying the aforementioned fixes in the original script system is not a simple task. + + +Specification +============= + +A new transaction digest algorithm is defined, but only applicable from the Overwinter upgrade block height +[#ZIP0000]_:: + [TODO: Pick one] BLAKE2[b-256|s] of the serialization of: + 1. nVersion of the transaction (4-byte little endian) + 2. hashPrevouts (32-byte hash) + 3. hashSequence (32-byte hash) + 4. hashOutputs (32-byte hash) + 5. hashJoinSplits (32-byte hash) + 6. nLocktime of the transaction (8-byte little endian) + 7. sighash type of the signature (4-byte little endian) + 8. If we are serializing an input (ie. this is not a JoinSplit signature hash): + a. outpoint (32-byte hash + 4-byte little endian) + b. scriptCode of the input (serialized as scripts inside CTxOuts) [TODO] + c. value of the output spent by this input (8-byte little endian) + d. nSequence of the input (4-byte little endian) + +The BLAKE2[b-256|s] personalization field will be set to:: + + ["ZcashSigHash"|"ZcSH"] || BRANCH_ID + +Semantics of the original sighash types remain unchanged, except the followings: + +#. The way of serialization is changed; + +#. All sighash types commit to the amount being spent by the signed input; + +#. ``SINGLE`` does not commit to the input index. When ``ANYONECANPAY`` is not set, the semantics are + unchanged since ``hashPrevouts`` and ``outpoint`` together implictly commit to the input index. When + ``SINGLE`` is used with ``ANYONECANPAY``, omission of the index commitment allows permutation of the + input-output pairs, as long as each pair is located at an equivalent index. + +Field definitions +----------------- + +The items 1, 6, 7, 8a, 8d have the same meaning as the original algorithm. [#wiki-checksig]_ + +2: ``hashPrevouts`` +``````````````````` +* If the ``ANYONECANPAY`` flag is not set, ``hashPrevouts`` is the double SHA256 of the serialization of all + input outpoints; + +* Otherwise, ``hashPrevouts`` is a ``uint256`` of ``0x0000......0000``. + +3: ``hashSequence`` +``````````````````` +* If none of the ``ANYONECANPAY``, ``SINGLE``, ``NONE`` sighash type is set, ``hashSequence`` is the double + SHA256 of the serialization of ``nSequence`` of all inputs; + +* Otherwise, ``hashSequence`` is a ``uint256`` of ``0x0000......0000``. + +4: ``hashOutputs`` +`````````````````` +* If the sighash type is neither ``SINGLE`` nor ``NONE``, ``hashOutputs`` is the double SHA256 of the + serialization of all output amount (8-byte little endian) with ``scriptPubKey`` (serialized as scripts + inside CTxOuts); + +* If sighash type is ``SINGLE`` and the input index is smaller than the number of outputs, ``hashOutputs`` is + the double SHA256 of the output amount with ``scriptPubKey`` of the same index as the input; + +* Otherwise, ``hashOutputs`` is a ``uint256`` of ``0x0000......0000``. [#01-change]_ + +5: ``hashJoinSplits`` +````````````````````` +* If ``vjoinsplits`` is non-empty, ``hashJoinSplits`` is the double SHA256 of the serialization of all + JoinSplits concatenated with the joinSplitPubKey; + +* Otherwise, ``hashJoinSplits`` is a ``uint256`` of ``0x0000......0000``. + +8b: ``scriptCode`` +`````````````````` +[TODO: TBC] + +* For ``P2PKH``, the ``scriptCode`` is ``0x1976a914{20-byte-pubkey-hash}88ac``. + +* For ``P2SH``, the ``scriptCode`` is the ``script`` serialized as scripts inside ``CTxOut``. + +8c: value +````````` +An 8-byte value of the amount of ZEC spent in this input. + +Notes +----- + +The ``hashPrevouts``, ``hashSequence``, ``hashOutputs``, and ``hashJoinSplits`` calculated in an earlier +verification may be reused in other inputs of the same transaction, so that the time complexity of the whole +hashing process reduces from O(n\ :sup:`2`) to O(n). + +Refer to the reference implementation, reproduced below, for the precise algorithm: + +.. code:: cpp + + uint256 hashPrevouts; + uint256 hashSequence; + uint256 hashOutputs; + uint256 hashJoinSplits; + + if (!(nHashType & SIGHASH_ANYONECANPAY)) { + CHashWriter ss(SER_GETHASH, 0); + for (unsigned int n = 0; n < txTo.vin.size(); n++) { + ss << txTo.vin[n].prevout; + } + hashPrevouts = ss.GetHash(); + } + + if (!(nHashType & SIGHASH_ANYONECANPAY) && (nHashType & 0x1f) != SIGHASH_SINGLE && (nHashType & 0x1f) != SIGHASH_NONE) { + CHashWriter ss(SER_GETHASH, 0); + for (unsigned int n = 0; n < txTo.vin.size(); n++) { + ss << txTo.vin[n].nSequence; + } + hashSequence = ss.GetHash(); + } + + if ((nHashType & 0x1f) != SIGHASH_SINGLE && (nHashType & 0x1f) != SIGHASH_NONE) { + CHashWriter ss(SER_GETHASH, 0); + for (unsigned int n = 0; n < txTo.vout.size(); n++) { + ss << txTo.vout[n]; + } + hashOutputs = ss.GetHash(); + } else if ((nHashType & 0x1f) == SIGHASH_SINGLE && nIn < txTo.vout.size()) { + CHashWriter ss(SER_GETHASH, 0); + ss << txTo.vout[nIn]; + hashOutputs = ss.GetHash(); + } + + if (!txTo.vjoinsplit.empty()) { + CHashWriter ss(SER_GETHASH, 0); + for (unsigned int n = 0; n < txTo.vjoinsplit.size(); n++) { + ss << txTo.vjoinsplit[n]; + } + ss << txTo.joinSplitPubKey; + hashJoinSplits = ss.GetHash(); + } + + // TODO: Update after choosing outer hash function + unsigned char personalization[16] = {}; + memcpy(personalization, "ZcashSigHash", 8); + memcpy(personalization, branchId, 4); + + CBlake2HashWriter ss(SER_GETHASH, 0, personalization); + // Version + ss << txTo.nVersion; + // Input prevouts/nSequence (none/all, depending on flags) + ss << hashPrevouts; + ss << hashSequence; + // Outputs (none/one/all, depending on flags) + ss << hashOutputs; + // JoinSplits + ss << hashJoinSplits; + // Locktime + ss << txTo.nLockTime; + // Sighash type + ss << nHashType; + + if (nIn != NOT_AN_INPUT) { + // The input being signed (replacing the scriptSig with scriptCode + amount) + // The prevout may already be contained in hashPrevout, and the nSequence + // may already be contain in hashSequence. + ss << txTo.vin[nIn].prevout; + ss << static_cast(scriptCode); + ss << amount; + ss << txTo.vin[nIn].nSequence; + } + + return ss.GetHash(); + + +Restrictions on public key type +=============================== + +[TODO: decide whether we want to implement this policy] + +As a default policy, only compressed public keys are accepted in ``P2PKH`` and ``P2SH``. Each public key +passed to a sigop must be a compressed key: the first byte MUST be either ``0x02`` or ``0x03``, and the size +MUST be 33 bytes. Transactions that break this rule will not be relayed or mined by default. + +Since this policy is preparation for a future softfork proposal, to avoid potential future funds loss, users +MUST NOT use uncompressed keys. + + +Example +======= + +TBC + + +Deployment +========== + +This proposal is deployed with the Overwinter network upgrade. + + +Backward compatibility +====================== + +This proposal is backwards-compatible with old UTXOs. It is **not** backwards-compatible with older software. +All transactions will be required to use this transaction digest algorithm for signatures, and so transactions +created by older software will be rejected by the network. + + +Reference Implementation +======================== + +TBC + + +References +========== + +.. [#wiki-checksig] https://en.bitcoin.it/wiki/OP_CHECKSIG +.. [#quadratic] + * `CVE-2013-2292 `_ + * `New Bitcoin vulnerability: A transaction that takes at least 3 minutes to verify `_ + * `The Megatransaction: Why Does It Take 25 Seconds? `_ +.. [#offline-wallets] `SIGHASH_WITHINPUTVALUE: Super-lightweight HW wallets and offline data `_ +.. [#ZIP0000] ZIP???: Overwinter Network Upgrade +.. [#01-change] In the original algorithm, a ``uint256`` of ``0x0000......0001`` is committed if the input + index for a ``SINGLE`` signature is greater than or equal to the number of outputs. In this ZIP a + ``0x0000......0000`` is commited, without changing the semantics. From 9f5dc6d729ded4ad008cd67dc9b9f327041e6200 Mon Sep 17 00:00:00 2001 From: str4d Date: Wed, 10 Jan 2018 01:08:33 +0100 Subject: [PATCH 02/28] Use BLAKE2b-256 for the outer personalized hash --- drafts/str4d-overwinter-sighash/draft1.rst | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 2d75b00c..1d2f40b1 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -61,7 +61,7 @@ Specification A new transaction digest algorithm is defined, but only applicable from the Overwinter upgrade block height [#ZIP0000]_:: - [TODO: Pick one] BLAKE2[b-256|s] of the serialization of: + BLAKE2b-256 of the serialization of: 1. nVersion of the transaction (4-byte little endian) 2. hashPrevouts (32-byte hash) 3. hashSequence (32-byte hash) @@ -75,9 +75,9 @@ A new transaction digest algorithm is defined, but only applicable from the Over c. value of the output spent by this input (8-byte little endian) d. nSequence of the input (4-byte little endian) -The BLAKE2[b-256|s] personalization field will be set to:: +The BLAKE2b-256 personalization field will be set to:: - ["ZcashSigHash"|"ZcSH"] || BRANCH_ID + "ZcashSigHash" || BRANCH_ID Semantics of the original sighash types remain unchanged, except the followings: @@ -192,10 +192,9 @@ Refer to the reference implementation, reproduced below, for the precise algorit hashJoinSplits = ss.GetHash(); } - // TODO: Update after choosing outer hash function unsigned char personalization[16] = {}; - memcpy(personalization, "ZcashSigHash", 8); - memcpy(personalization, branchId, 4); + memcpy(personalization, "ZcashSigHash", 12); + memcpy(personalization+12, branchId, 4); CBlake2HashWriter ss(SER_GETHASH, 0, personalization); // Version From 7d20a43578d60323e0af8d04606aa4c1ea5ef264 Mon Sep 17 00:00:00 2001 From: str4d Date: Wed, 10 Jan 2018 01:10:33 +0100 Subject: [PATCH 03/28] Add note about inclusion of JoinSplit proofs in the signature hash --- drafts/str4d-overwinter-sighash/draft1.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 1d2f40b1..76d8fc51 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -125,6 +125,10 @@ The items 1, 6, 7, 8a, 8d have the same meaning as the original algorithm. [#wik * If ``vjoinsplits`` is non-empty, ``hashJoinSplits`` is the double SHA256 of the serialization of all JoinSplits concatenated with the joinSplitPubKey; + * Note that the JoinSplit proofs are included in the signature hash, as with v1 and v2 transactions. In a + future transaction digest algorithm, the proofs will likely be omitted as authentication data, in the same + way that signatures are omitted here. + * Otherwise, ``hashJoinSplits`` is a ``uint256`` of ``0x0000......0000``. 8b: ``scriptCode`` From beaea49cb006f5a58e3fb49fb7df599710f9bb07 Mon Sep 17 00:00:00 2001 From: str4d Date: Wed, 10 Jan 2018 10:09:17 +0100 Subject: [PATCH 04/28] Add note about replay protection effect of hash personalization --- drafts/str4d-overwinter-sighash/draft1.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 76d8fc51..f13667b3 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -79,6 +79,10 @@ The BLAKE2b-256 personalization field will be set to:: "ZcashSigHash" || BRANCH_ID +This provides domain separation of the signature hash across parallel branches. It also adds a layer of replay +protection (transactions targeted for one branch will have invalid signatures on other branches), but this is +not relied on for security. [#overwinter-replay-protection]_ + Semantics of the original sighash types remain unchanged, except the followings: #. The way of serialization is changed; @@ -277,6 +281,10 @@ References * `The Megatransaction: Why Does It Take 25 Seconds? `_ .. [#offline-wallets] `SIGHASH_WITHINPUTVALUE: Super-lightweight HW wallets and offline data `_ .. [#ZIP0000] ZIP???: Overwinter Network Upgrade +.. [#overwinter-replay-protection] + The new transaction format introduced in the Overwinter upgrade contain the branch ID that a transaction is + committing to (for transaction format serialization unambiguity), and the new consensus rules require that + the transactions in a block match the expected branch ID for that block height. .. [#01-change] In the original algorithm, a ``uint256`` of ``0x0000......0001`` is committed if the input index for a ``SINGLE`` signature is greater than or equal to the number of outputs. In this ZIP a ``0x0000......0000`` is commited, without changing the semantics. From 35854f6a8c7fb3e3c82069931bdfb52718fdcb3f Mon Sep 17 00:00:00 2001 From: str4d Date: Wed, 10 Jan 2018 10:12:00 +0100 Subject: [PATCH 05/28] Reference use of transaction digest algorithm for JoinSplit signatures --- drafts/str4d-overwinter-sighash/draft1.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index f13667b3..da86ae88 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -34,6 +34,8 @@ There are 4 ECDSA signature verification codes in the original Zcash script syst (``ALL``, ``NONE``, or ``SINGLE``, possibly modified by ``ANYONECANPAY``), a transaction digest is generated with a double SHA256 of a serialized subset of the transaction, and the signature is verified against this digest with a given public key. The detailed procedure is described in a Bitcoin Wiki article. [#wiki-checksig]_ +The transaction digest is additionally used for the JoinSplit signature (solely with sighash type ``ALL``). +[#zcash-protocol]_ Unfortunately, there are at least 2 weaknesses in the original SignatureHash transaction digest algorithm: @@ -275,6 +277,7 @@ References ========== .. [#wiki-checksig] https://en.bitcoin.it/wiki/OP_CHECKSIG +.. [#zcash-protocol] `Zcash Protocol Specification, Section 4.6 `_ .. [#quadratic] * `CVE-2013-2292 `_ * `New Bitcoin vulnerability: A transaction that takes at least 3 minutes to verify `_ From 2a4f012692bae1bd412bdf7ca7577eff353ff77d Mon Sep 17 00:00:00 2001 From: str4d Date: Wed, 10 Jan 2018 10:44:56 +0100 Subject: [PATCH 06/28] Add transaction expiry value into transaction digest --- drafts/str4d-overwinter-sighash/draft1.rst | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index da86ae88..daf1f063 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -69,9 +69,10 @@ A new transaction digest algorithm is defined, but only applicable from the Over 3. hashSequence (32-byte hash) 4. hashOutputs (32-byte hash) 5. hashJoinSplits (32-byte hash) - 6. nLocktime of the transaction (8-byte little endian) - 7. sighash type of the signature (4-byte little endian) - 8. If we are serializing an input (ie. this is not a JoinSplit signature hash): + 6. nLockTime of the transaction (8-byte little endian) + 7. nExpiryTime of the transaction (8-byte little endian) + 8. sighash type of the signature (4-byte little endian) + 9. If we are serializing an input (ie. this is not a JoinSplit signature hash): a. outpoint (32-byte hash + 4-byte little endian) b. scriptCode of the input (serialized as scripts inside CTxOuts) [TODO] c. value of the output spent by this input (8-byte little endian) @@ -99,7 +100,7 @@ Semantics of the original sighash types remain unchanged, except the followings: Field definitions ----------------- -The items 1, 6, 7, 8a, 8d have the same meaning as the original algorithm. [#wiki-checksig]_ +The items 1, 6, 8, 9a, 9d have the same meaning as the original algorithm. [#wiki-checksig]_ 2: ``hashPrevouts`` ``````````````````` @@ -137,7 +138,12 @@ The items 1, 6, 7, 8a, 8d have the same meaning as the original algorithm. [#wik * Otherwise, ``hashJoinSplits`` is a ``uint256`` of ``0x0000......0000``. -8b: ``scriptCode`` +7: ``nExpiryTime`` +`````````````` +The block height or time at which the transaction becomes unilaterally invalid, and can never be mined. +[#ZIP-tx-expiry]_ + +9b: ``scriptCode`` `````````````````` [TODO: TBC] @@ -145,7 +151,7 @@ The items 1, 6, 7, 8a, 8d have the same meaning as the original algorithm. [#wik * For ``P2SH``, the ``scriptCode`` is the ``script`` serialized as scripts inside ``CTxOut``. -8c: value +9c: value ````````` An 8-byte value of the amount of ZEC spent in this input. @@ -218,6 +224,8 @@ Refer to the reference implementation, reproduced below, for the precise algorit ss << hashJoinSplits; // Locktime ss << txTo.nLockTime; + // Expiry time + ss << txTo.nExpiryTime; // Sighash type ss << nHashType; @@ -291,3 +299,4 @@ References .. [#01-change] In the original algorithm, a ``uint256`` of ``0x0000......0001`` is committed if the input index for a ``SINGLE`` signature is greater than or equal to the number of outputs. In this ZIP a ``0x0000......0000`` is commited, without changing the semantics. +.. [#ZIP-tx-expiry] ZIP???: Transaction expiry From 8a4101b207573bdbc14d338a45b336eafaa7c5b1 Mon Sep 17 00:00:00 2001 From: str4d Date: Thu, 11 Jan 2018 12:30:50 +0100 Subject: [PATCH 07/28] Cleanups and clarity --- drafts/str4d-overwinter-sighash/draft1.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index daf1f063..a4579ce7 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -86,9 +86,9 @@ This provides domain separation of the signature hash across parallel branches. protection (transactions targeted for one branch will have invalid signatures on other branches), but this is not relied on for security. [#overwinter-replay-protection]_ -Semantics of the original sighash types remain unchanged, except the followings: +Semantics of the original sighash types remain unchanged, except the following: -#. The way of serialization is changed; +#. The serialization format is changed; #. All sighash types commit to the amount being spent by the signed input; @@ -123,14 +123,14 @@ The items 1, 6, 8, 9a, 9d have the same meaning as the original algorithm. [#wik inside CTxOuts); * If sighash type is ``SINGLE`` and the input index is smaller than the number of outputs, ``hashOutputs`` is - the double SHA256 of the output amount with ``scriptPubKey`` of the same index as the input; + the double SHA256 of the output (serialized as above) with the same index as the input; * Otherwise, ``hashOutputs`` is a ``uint256`` of ``0x0000......0000``. [#01-change]_ 5: ``hashJoinSplits`` ````````````````````` * If ``vjoinsplits`` is non-empty, ``hashJoinSplits`` is the double SHA256 of the serialization of all - JoinSplits concatenated with the joinSplitPubKey; + JoinSplits (in their canonical transaction serialization format) concatenated with the joinSplitPubKey; * Note that the JoinSplit proofs are included in the signature hash, as with v1 and v2 transactions. In a future transaction digest algorithm, the proofs will likely be omitted as authentication data, in the same From d769180769ac691a422d74ea1e7be0b7390acbe2 Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 12 Jan 2018 12:13:56 +0100 Subject: [PATCH 08/28] Update draft based on latest transaction format specification draft --- drafts/str4d-overwinter-sighash/draft1.rst | 60 +++++++++++++--------- 1 file changed, 35 insertions(+), 25 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index a4579ce7..208a0741 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -65,26 +65,33 @@ A new transaction digest algorithm is defined, but only applicable from the Over [#ZIP0000]_:: BLAKE2b-256 of the serialization of: 1. nVersion of the transaction (4-byte little endian) - 2. hashPrevouts (32-byte hash) - 3. hashSequence (32-byte hash) - 4. hashOutputs (32-byte hash) - 5. hashJoinSplits (32-byte hash) - 6. nLockTime of the transaction (8-byte little endian) - 7. nExpiryTime of the transaction (8-byte little endian) - 8. sighash type of the signature (4-byte little endian) - 9. If we are serializing an input (ie. this is not a JoinSplit signature hash): + 2. nVersionBranch of the transaction (4-byte little endian) + 3. hashPrevouts (32-byte hash) + 4. hashSequence (32-byte hash) + 5. hashOutputs (32-byte hash) + 6. hashJoinSplits (32-byte hash) + 7. nLockTime of the transaction (8-byte little endian) + 8. nExpiryTime of the transaction (8-byte little endian) + 9. sighash type of the signature (4-byte little endian) + 10. If we are serializing an input (ie. this is not a JoinSplit signature hash): a. outpoint (32-byte hash + 4-byte little endian) b. scriptCode of the input (serialized as scripts inside CTxOuts) [TODO] c. value of the output spent by this input (8-byte little endian) d. nSequence of the input (4-byte little endian) -The BLAKE2b-256 personalization field will be set to:: +The BLAKE2b-256 personalization field is set to:: - "ZcashSigHash" || BRANCH_ID + "ZcashSigHash" || BLOCK_BRANCH_ID -This provides domain separation of the signature hash across parallel branches. It also adds a layer of replay -protection (transactions targeted for one branch will have invalid signatures on other branches), but this is -not relied on for security. [#overwinter-replay-protection]_ +``BLOCK_BRANCH_ID`` is the ``BRANCH_ID`` for the epoch of the transaction's parent block. [#ZIP-activation-mechanism]_ +Domain separation of the signature hash across parallel branches provides replay protection: transactions +targeted for one branch will have invalid signatures on other branches. This has two effects on user behavior: + +* Transaction creators MUST specify the epoch they want their transaction to be mined in. Across a network + upgrade, this means that if a transaction is not mined before the activation height, it will never be mined. + +* Transaction validators MUST NOT use ``nVersionBranch`` as the ``BLOCK_BRANCH_ID``: while these are both a + ``BRANCH_ID`` "under the hood", they are **semantically different**. Semantics of the original sighash types remain unchanged, except the following: @@ -100,23 +107,28 @@ Semantics of the original sighash types remain unchanged, except the following: Field definitions ----------------- -The items 1, 6, 8, 9a, 9d have the same meaning as the original algorithm. [#wiki-checksig]_ +The items 1, 7, 9, 10a, 10d have the same meaning as the original algorithm. [#wiki-checksig]_ -2: ``hashPrevouts`` +2: ``nVersionBranch`` +````````````````````` +The ``BRANCH_ID`` of the epoch in which the transaction format corresponding to ``nVersion`` was introduced. +[#ZIP-overwinter-tx-format]_ + +3: ``hashPrevouts`` ``````````````````` * If the ``ANYONECANPAY`` flag is not set, ``hashPrevouts`` is the double SHA256 of the serialization of all input outpoints; * Otherwise, ``hashPrevouts`` is a ``uint256`` of ``0x0000......0000``. -3: ``hashSequence`` +4: ``hashSequence`` ``````````````````` * If none of the ``ANYONECANPAY``, ``SINGLE``, ``NONE`` sighash type is set, ``hashSequence`` is the double SHA256 of the serialization of ``nSequence`` of all inputs; * Otherwise, ``hashSequence`` is a ``uint256`` of ``0x0000......0000``. -4: ``hashOutputs`` +5: ``hashOutputs`` `````````````````` * If the sighash type is neither ``SINGLE`` nor ``NONE``, ``hashOutputs`` is the double SHA256 of the serialization of all output amount (8-byte little endian) with ``scriptPubKey`` (serialized as scripts @@ -127,7 +139,7 @@ The items 1, 6, 8, 9a, 9d have the same meaning as the original algorithm. [#wik * Otherwise, ``hashOutputs`` is a ``uint256`` of ``0x0000......0000``. [#01-change]_ -5: ``hashJoinSplits`` +6: ``hashJoinSplits`` ````````````````````` * If ``vjoinsplits`` is non-empty, ``hashJoinSplits`` is the double SHA256 of the serialization of all JoinSplits (in their canonical transaction serialization format) concatenated with the joinSplitPubKey; @@ -138,12 +150,12 @@ The items 1, 6, 8, 9a, 9d have the same meaning as the original algorithm. [#wik * Otherwise, ``hashJoinSplits`` is a ``uint256`` of ``0x0000......0000``. -7: ``nExpiryTime`` +8: ``nExpiryTime`` `````````````` The block height or time at which the transaction becomes unilaterally invalid, and can never be mined. [#ZIP-tx-expiry]_ -9b: ``scriptCode`` +10b: ``scriptCode`` `````````````````` [TODO: TBC] @@ -151,7 +163,7 @@ The block height or time at which the transaction becomes unilaterally invalid, * For ``P2SH``, the ``scriptCode`` is the ``script`` serialized as scripts inside ``CTxOut``. -9c: value +10c: value ````````` An 8-byte value of the amount of ZEC spent in this input. @@ -292,10 +304,8 @@ References * `The Megatransaction: Why Does It Take 25 Seconds? `_ .. [#offline-wallets] `SIGHASH_WITHINPUTVALUE: Super-lightweight HW wallets and offline data `_ .. [#ZIP0000] ZIP???: Overwinter Network Upgrade -.. [#overwinter-replay-protection] - The new transaction format introduced in the Overwinter upgrade contain the branch ID that a transaction is - committing to (for transaction format serialization unambiguity), and the new consensus rules require that - the transactions in a block match the expected branch ID for that block height. +.. [#ZIP-activation-mechanism] ZIP???: Network Upgrade Activation Mechanism +.. [#ZIP-tx-format] ZIP???: Overwinter Transaction Format .. [#01-change] In the original algorithm, a ``uint256`` of ``0x0000......0001`` is committed if the input index for a ``SINGLE`` signature is greater than or equal to the number of outputs. In this ZIP a ``0x0000......0000`` is commited, without changing the semantics. From 1bfee9eebf476f8c0db60f9c486fb034996e5a5d Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 14:19:57 +0000 Subject: [PATCH 09/28] Correct field sizes --- drafts/str4d-overwinter-sighash/draft1.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 208a0741..f353f66b 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -70,8 +70,8 @@ A new transaction digest algorithm is defined, but only applicable from the Over 4. hashSequence (32-byte hash) 5. hashOutputs (32-byte hash) 6. hashJoinSplits (32-byte hash) - 7. nLockTime of the transaction (8-byte little endian) - 8. nExpiryTime of the transaction (8-byte little endian) + 7. nLockTime of the transaction (4-byte little endian) + 8. nExpiryTime of the transaction (4-byte little endian) 9. sighash type of the signature (4-byte little endian) 10. If we are serializing an input (ie. this is not a JoinSplit signature hash): a. outpoint (32-byte hash + 4-byte little endian) From cf28700c7cf916c62ee9c6dbdcc6323eaa04dcd9 Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 15:41:25 +0000 Subject: [PATCH 10/28] Update draft based on latest transaction format specification draft --- drafts/str4d-overwinter-sighash/draft1.rst | 66 +++++++++++++--------- 1 file changed, 38 insertions(+), 28 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index f353f66b..64c7133f 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -61,17 +61,16 @@ Deploying the aforementioned fixes in the original script system is not a simple Specification ============= -A new transaction digest algorithm is defined, but only applicable from the Overwinter upgrade block height -[#ZIP0000]_:: +A new transaction digest algorithm is defined:: BLAKE2b-256 of the serialization of: - 1. nVersion of the transaction (4-byte little endian) - 2. nVersionBranch of the transaction (4-byte little endian) + 1. header of the transaction (4-byte little endian) + 2. nVersionGroupId of the transaction (4-byte little endian) 3. hashPrevouts (32-byte hash) 4. hashSequence (32-byte hash) 5. hashOutputs (32-byte hash) 6. hashJoinSplits (32-byte hash) 7. nLockTime of the transaction (4-byte little endian) - 8. nExpiryTime of the transaction (4-byte little endian) + 8. nExpiryHeight of the transaction (4-byte little endian) 9. sighash type of the signature (4-byte little endian) 10. If we are serializing an input (ie. this is not a JoinSplit signature hash): a. outpoint (32-byte hash + 4-byte little endian) @@ -79,19 +78,22 @@ A new transaction digest algorithm is defined, but only applicable from the Over c. value of the output spent by this input (8-byte little endian) d. nSequence of the input (4-byte little endian) +The new algorithm MUST be used for signatures created over the Overwinter transaction format. +[#ZIP-overwinter-tx-format]_ Combined with the new consensus rule that v1 and v2 transaction formats will be +invalid from the Overwinter upgrade, [#ZIP-overwinter-tx-format]_ this effectively means that all transactions +signatures from the Overwinter activation height will use the new algorithm. [#ZIP0000]_ + The BLAKE2b-256 personalization field is set to:: - "ZcashSigHash" || BLOCK_BRANCH_ID + "ZcashSigHash" || CONSENSUS_BRANCH_ID -``BLOCK_BRANCH_ID`` is the ``BRANCH_ID`` for the epoch of the transaction's parent block. [#ZIP-activation-mechanism]_ -Domain separation of the signature hash across parallel branches provides replay protection: transactions -targeted for one branch will have invalid signatures on other branches. This has two effects on user behavior: +``CONSENSUS_BRANCH_ID`` is the little-endian encoding of ``BRANCH_ID`` for the epoch of the transaction's +parent block. [#ZIP-activation-mechanism]_ Domain separation of the signature hash across parallel branches +provides replay protection: transactions targeted for one branch will have invalid signatures on other +branches. -* Transaction creators MUST specify the epoch they want their transaction to be mined in. Across a network - upgrade, this means that if a transaction is not mined before the activation height, it will never be mined. - -* Transaction validators MUST NOT use ``nVersionBranch`` as the ``BLOCK_BRANCH_ID``: while these are both a - ``BRANCH_ID`` "under the hood", they are **semantically different**. +Transaction creators MUST specify the epoch they want their transaction to be mined in. Across a network +upgrade, this means that if a transaction is not mined before the activation height, it will never be mined. Semantics of the original sighash types remain unchanged, except the following: @@ -107,12 +109,17 @@ Semantics of the original sighash types remain unchanged, except the following: Field definitions ----------------- -The items 1, 7, 9, 10a, 10d have the same meaning as the original algorithm. [#wiki-checksig]_ +The items 7, 9, 10a, 10d have the same meaning as the original algorithm. [#wiki-checksig]_ -2: ``nVersionBranch`` -````````````````````` -The ``BRANCH_ID`` of the epoch in which the transaction format corresponding to ``nVersion`` was introduced. -[#ZIP-overwinter-tx-format]_ +1: ``header`` +````````````` +Deserialized into two transaction properties: ``fOverwintered`` and ``nVersion``. For transactions that use +this transaction digest algorithm, ``fOverwintered`` is always set. [#ZIP-overwinter-tx-format]_ + +2: ``nVersionGroupId`` +`````````````````````` +Provides domain separation of ``nVersion``. It is only defined if ``fOverwintered`` is set, which means that +it is always defined for transactions that use this algorithm. [#ZIP-overwinter-tx-format]_ 3: ``hashPrevouts`` ``````````````````` @@ -150,9 +157,9 @@ The ``BRANCH_ID`` of the epoch in which the transaction format corresponding to * Otherwise, ``hashJoinSplits`` is a ``uint256`` of ``0x0000......0000``. -8: ``nExpiryTime`` -`````````````` -The block height or time at which the transaction becomes unilaterally invalid, and can never be mined. +8: ``nExpiryHeight`` +```````````````````` +The block height after which the transaction becomes unilaterally invalid, and can never be mined. [#ZIP-tx-expiry]_ 10b: ``scriptCode`` @@ -220,13 +227,16 @@ Refer to the reference implementation, reproduced below, for the precise algorit hashJoinSplits = ss.GetHash(); } + uint32_t leConsensusBranchId = htole32(consensusBranchId); unsigned char personalization[16] = {}; memcpy(personalization, "ZcashSigHash", 12); - memcpy(personalization+12, branchId, 4); + memcpy(personalization+12, &leConsensusBranchId, 4); CBlake2HashWriter ss(SER_GETHASH, 0, personalization); - // Version - ss << txTo.nVersion; + // fOverwintered and nVersion + ss << txTo.GetHeader(); + // Version group ID + ss << txTo.nVersionGroupId; // Input prevouts/nSequence (none/all, depending on flags) ss << hashPrevouts; ss << hashSequence; @@ -236,8 +246,8 @@ Refer to the reference implementation, reproduced below, for the precise algorit ss << hashJoinSplits; // Locktime ss << txTo.nLockTime; - // Expiry time - ss << txTo.nExpiryTime; + // Expiry height + ss << txTo.nExpiryHeight; // Sighash type ss << nHashType; @@ -305,7 +315,7 @@ References .. [#offline-wallets] `SIGHASH_WITHINPUTVALUE: Super-lightweight HW wallets and offline data `_ .. [#ZIP0000] ZIP???: Overwinter Network Upgrade .. [#ZIP-activation-mechanism] ZIP???: Network Upgrade Activation Mechanism -.. [#ZIP-tx-format] ZIP???: Overwinter Transaction Format +.. [#ZIP-overwinter-tx-format] ZIP???: Overwinter Transaction Format .. [#01-change] In the original algorithm, a ``uint256`` of ``0x0000......0001`` is committed if the input index for a ``SINGLE`` signature is greater than or equal to the number of outputs. In this ZIP a ``0x0000......0000`` is commited, without changing the semantics. From 4998e99960a3f427226a293f9c35811aca7daee1 Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 15:41:39 +0000 Subject: [PATCH 11/28] Remove unnecessary metadata --- drafts/str4d-overwinter-sighash/draft1.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 64c7133f..fc2cba4a 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -6,7 +6,6 @@ Daira Hopwood Credits: Johnson Lau Pieter Wuille - Comments-Summary: No comments yet. Category: Process Created: 2017-12-27 License: MIT From 4e02d73e6964a55c6b548e20a6c5e1dd56c25afb Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 15:48:27 +0000 Subject: [PATCH 12/28] Reference BIP 143 and BUIP-HF v1.2 --- drafts/str4d-overwinter-sighash/draft1.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index fc2cba4a..9724245c 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -6,6 +6,7 @@ Daira Hopwood Credits: Johnson Lau Pieter Wuille + Bitcoin-ABC Category: Process Created: 2017-12-27 License: MIT @@ -77,6 +78,9 @@ A new transaction digest algorithm is defined:: c. value of the output spent by this input (8-byte little endian) d. nSequence of the input (4-byte little endian) +The new algorithm is based on the Bitcoin transaction digest algorithm defined in BIP 143, [#BIP0143]_ with +replay protection inspired by BUIP-HF v1.2. [#BUIP-HF]_ + The new algorithm MUST be used for signatures created over the Overwinter transaction format. [#ZIP-overwinter-tx-format]_ Combined with the new consensus rule that v1 and v2 transaction formats will be invalid from the Overwinter upgrade, [#ZIP-overwinter-tx-format]_ this effectively means that all transactions @@ -312,6 +316,8 @@ References * `New Bitcoin vulnerability: A transaction that takes at least 3 minutes to verify `_ * `The Megatransaction: Why Does It Take 25 Seconds? `_ .. [#offline-wallets] `SIGHASH_WITHINPUTVALUE: Super-lightweight HW wallets and offline data `_ +.. [#BIP0143] `Transaction Signature Verification for Version 0 Witness Program `_ +.. [#BUIP-HF] `BUIP-HF Digest for replay protected signature verification across hard forks, version 1.2 `_ .. [#ZIP0000] ZIP???: Overwinter Network Upgrade .. [#ZIP-activation-mechanism] ZIP???: Network Upgrade Activation Mechanism .. [#ZIP-overwinter-tx-format] ZIP???: Overwinter Transaction Format From f173b5b3d02acd48799bae338126c4d5d8c20688 Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 15:57:06 +0000 Subject: [PATCH 13/28] BLAKE2b-256 is used everywhere now --- drafts/str4d-overwinter-sighash/draft1.rst | 34 +++++++++++++--------- 1 file changed, 20 insertions(+), 14 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 9724245c..07cf62e3 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -62,7 +62,7 @@ Specification ============= A new transaction digest algorithm is defined:: - BLAKE2b-256 of the serialization of: + BLAKE2b-256 hash of the serialization of: 1. header of the transaction (4-byte little endian) 2. nVersionGroupId of the transaction (4-byte little endian) 3. hashPrevouts (32-byte hash) @@ -126,32 +126,32 @@ it is always defined for transactions that use this algorithm. [#ZIP-overwinter- 3: ``hashPrevouts`` ``````````````````` -* If the ``ANYONECANPAY`` flag is not set, ``hashPrevouts`` is the double SHA256 of the serialization of all - input outpoints; +* If the ``ANYONECANPAY`` flag is not set, ``hashPrevouts`` is the BLAKE2b-256 hash of the serialization of + all input outpoints; * Otherwise, ``hashPrevouts`` is a ``uint256`` of ``0x0000......0000``. 4: ``hashSequence`` ``````````````````` -* If none of the ``ANYONECANPAY``, ``SINGLE``, ``NONE`` sighash type is set, ``hashSequence`` is the double - SHA256 of the serialization of ``nSequence`` of all inputs; +* If none of the ``ANYONECANPAY``, ``SINGLE``, ``NONE`` sighash type is set, ``hashSequence`` is the + BLAKE2b-256 hash of the serialization of ``nSequence`` of all inputs; * Otherwise, ``hashSequence`` is a ``uint256`` of ``0x0000......0000``. 5: ``hashOutputs`` `````````````````` -* If the sighash type is neither ``SINGLE`` nor ``NONE``, ``hashOutputs`` is the double SHA256 of the +* If the sighash type is neither ``SINGLE`` nor ``NONE``, ``hashOutputs`` is the BLAKE2b-256 hash of the serialization of all output amount (8-byte little endian) with ``scriptPubKey`` (serialized as scripts inside CTxOuts); * If sighash type is ``SINGLE`` and the input index is smaller than the number of outputs, ``hashOutputs`` is - the double SHA256 of the output (serialized as above) with the same index as the input; + the BLAKE2b-256 hash of the output (serialized as above) with the same index as the input; * Otherwise, ``hashOutputs`` is a ``uint256`` of ``0x0000......0000``. [#01-change]_ 6: ``hashJoinSplits`` ````````````````````` -* If ``vjoinsplits`` is non-empty, ``hashJoinSplits`` is the double SHA256 of the serialization of all +* If ``vjoinsplits`` is non-empty, ``hashJoinSplits`` is the BLAKE2b-256 hash of the serialization of all JoinSplits (in their canonical transaction serialization format) concatenated with the joinSplitPubKey; * Note that the JoinSplit proofs are included in the signature hash, as with v1 and v2 transactions. In a @@ -180,6 +180,9 @@ An 8-byte value of the amount of ZEC spent in this input. Notes ----- +When generating ``hashPrevouts``, ``hashSequence``, ``hashOutputs``, and ``hashJoinSplits``, the BLAKE2b-256 +personalization field is set to ``Zcash_Inner_Hash``. + The ``hashPrevouts``, ``hashSequence``, ``hashOutputs``, and ``hashJoinSplits`` calculated in an earlier verification may be reused in other inputs of the same transaction, so that the time complexity of the whole hashing process reduces from O(n\ :sup:`2`) to O(n). @@ -188,13 +191,16 @@ Refer to the reference implementation, reproduced below, for the precise algorit .. code:: cpp + const unsigned char ZCASH_INNER_HASH_PERSONALIZATION[16] = + {'Z','c','a','s','h','_','I','n','n','e','r','_','H','a','s','h'}; + uint256 hashPrevouts; uint256 hashSequence; uint256 hashOutputs; uint256 hashJoinSplits; if (!(nHashType & SIGHASH_ANYONECANPAY)) { - CHashWriter ss(SER_GETHASH, 0); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); for (unsigned int n = 0; n < txTo.vin.size(); n++) { ss << txTo.vin[n].prevout; } @@ -202,7 +208,7 @@ Refer to the reference implementation, reproduced below, for the precise algorit } if (!(nHashType & SIGHASH_ANYONECANPAY) && (nHashType & 0x1f) != SIGHASH_SINGLE && (nHashType & 0x1f) != SIGHASH_NONE) { - CHashWriter ss(SER_GETHASH, 0); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); for (unsigned int n = 0; n < txTo.vin.size(); n++) { ss << txTo.vin[n].nSequence; } @@ -210,19 +216,19 @@ Refer to the reference implementation, reproduced below, for the precise algorit } if ((nHashType & 0x1f) != SIGHASH_SINGLE && (nHashType & 0x1f) != SIGHASH_NONE) { - CHashWriter ss(SER_GETHASH, 0); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); for (unsigned int n = 0; n < txTo.vout.size(); n++) { ss << txTo.vout[n]; } hashOutputs = ss.GetHash(); } else if ((nHashType & 0x1f) == SIGHASH_SINGLE && nIn < txTo.vout.size()) { - CHashWriter ss(SER_GETHASH, 0); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); ss << txTo.vout[nIn]; hashOutputs = ss.GetHash(); } if (!txTo.vjoinsplit.empty()) { - CHashWriter ss(SER_GETHASH, 0); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); for (unsigned int n = 0; n < txTo.vjoinsplit.size(); n++) { ss << txTo.vjoinsplit[n]; } @@ -235,7 +241,7 @@ Refer to the reference implementation, reproduced below, for the precise algorit memcpy(personalization, "ZcashSigHash", 12); memcpy(personalization+12, &leConsensusBranchId, 4); - CBlake2HashWriter ss(SER_GETHASH, 0, personalization); + CBLAKE2bWriter ss(SER_GETHASH, 0, personalization); // fOverwintered and nVersion ss << txTo.GetHeader(); // Version group ID From 12d4e4d67c98e16e03eefaa9ff938c2229c36b33 Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 16:05:07 +0000 Subject: [PATCH 14/28] Fix typo --- drafts/str4d-overwinter-sighash/draft1.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 07cf62e3..3783b63e 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -263,7 +263,7 @@ Refer to the reference implementation, reproduced below, for the precise algorit if (nIn != NOT_AN_INPUT) { // The input being signed (replacing the scriptSig with scriptCode + amount) // The prevout may already be contained in hashPrevout, and the nSequence - // may already be contain in hashSequence. + // may already be contained in hashSequence. ss << txTo.vin[nIn].prevout; ss << static_cast(scriptCode); ss << amount; From 252c22b6d0e5c93da8c6f39f762525dc88f18c96 Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 16:11:36 +0000 Subject: [PATCH 15/28] Add reference for BLAKE2 personalization field --- drafts/str4d-overwinter-sighash/draft1.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 3783b63e..ed5e96fa 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -86,7 +86,7 @@ The new algorithm MUST be used for signatures created over the Overwinter transa invalid from the Overwinter upgrade, [#ZIP-overwinter-tx-format]_ this effectively means that all transactions signatures from the Overwinter activation height will use the new algorithm. [#ZIP0000]_ -The BLAKE2b-256 personalization field is set to:: +The BLAKE2b-256 personalization field [#BLAKE2-personalization]_ is set to:: "ZcashSigHash" || CONSENSUS_BRANCH_ID @@ -327,6 +327,7 @@ References .. [#ZIP0000] ZIP???: Overwinter Network Upgrade .. [#ZIP-activation-mechanism] ZIP???: Network Upgrade Activation Mechanism .. [#ZIP-overwinter-tx-format] ZIP???: Overwinter Transaction Format +.. [#BLAKE2-personalization] `"BLAKE2: simpler, smaller, fast as MD5", Section 2.8 `_ .. [#01-change] In the original algorithm, a ``uint256`` of ``0x0000......0001`` is committed if the input index for a ``SINGLE`` signature is greater than or equal to the number of outputs. In this ZIP a ``0x0000......0000`` is commited, without changing the semantics. From 1921ce1cf0812337bd8684712e2d8dcc1db60210 Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 16:14:34 +0000 Subject: [PATCH 16/28] Remove comment about future sighash algorithms --- drafts/str4d-overwinter-sighash/draft1.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index ed5e96fa..33e2c02f 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -154,9 +154,8 @@ it is always defined for transactions that use this algorithm. [#ZIP-overwinter- * If ``vjoinsplits`` is non-empty, ``hashJoinSplits`` is the BLAKE2b-256 hash of the serialization of all JoinSplits (in their canonical transaction serialization format) concatenated with the joinSplitPubKey; - * Note that the JoinSplit proofs are included in the signature hash, as with v1 and v2 transactions. In a - future transaction digest algorithm, the proofs will likely be omitted as authentication data, in the same - way that signatures are omitted here. + * Note that while signatures are ommitted, the JoinSplit proofs are included in the signature hash, as with + v1 and v2 transactions. * Otherwise, ``hashJoinSplits`` is a ``uint256`` of ``0x0000......0000``. From 59cea039fbcffb3f54506e451ec4b31af81b37ee Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 16:16:48 +0000 Subject: [PATCH 17/28] Fix rST formatting --- drafts/str4d-overwinter-sighash/draft1.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 33e2c02f..81e5a21f 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -62,6 +62,7 @@ Specification ============= A new transaction digest algorithm is defined:: + BLAKE2b-256 hash of the serialization of: 1. header of the transaction (4-byte little endian) 2. nVersionGroupId of the transaction (4-byte little endian) From 639c1de74f2e3a7e5927de6ceae9566f1742fd4d Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 16:20:40 +0000 Subject: [PATCH 18/28] Clarify sentence --- drafts/str4d-overwinter-sighash/draft1.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 81e5a21f..0c023323 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -91,8 +91,8 @@ The BLAKE2b-256 personalization field [#BLAKE2-personalization]_ is set to:: "ZcashSigHash" || CONSENSUS_BRANCH_ID -``CONSENSUS_BRANCH_ID`` is the little-endian encoding of ``BRANCH_ID`` for the epoch of the transaction's -parent block. [#ZIP-activation-mechanism]_ Domain separation of the signature hash across parallel branches +``CONSENSUS_BRANCH_ID`` is the little-endian encoding of ``BRANCH_ID`` for the epoch of the block containing +the transaction. [#ZIP-activation-mechanism]_ Domain separation of the signature hash across parallel branches provides replay protection: transactions targeted for one branch will have invalid signatures on other branches. From 6297ac582f9b7b424211ae96e8b66b74f9e5626c Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 16:23:23 +0000 Subject: [PATCH 19/28] Add link to reference implementation --- drafts/str4d-overwinter-sighash/draft1.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 0c023323..7e264227 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -309,7 +309,7 @@ created by older software will be rejected by the network. Reference Implementation ======================== -TBC +https://github.com/zcash/zcash/pull/2903 References From 2d7401add59181ac233807d8cc3f9a737650de27 Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 9 Feb 2018 16:35:54 +0000 Subject: [PATCH 20/28] Specify scriptCode --- drafts/str4d-overwinter-sighash/draft1.rst | 24 ++++------------------ 1 file changed, 4 insertions(+), 20 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 7e264227..aac6f9ee 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -113,7 +113,7 @@ Semantics of the original sighash types remain unchanged, except the following: Field definitions ----------------- -The items 7, 9, 10a, 10d have the same meaning as the original algorithm. [#wiki-checksig]_ +The items 7, 9, 10a, 10d have the same meaning as the original algorithm from Bitcoin. [#wiki-checksig]_ 1: ``header`` ````````````` @@ -166,12 +166,9 @@ The block height after which the transaction becomes unilaterally invalid, and c [#ZIP-tx-expiry]_ 10b: ``scriptCode`` -`````````````````` -[TODO: TBC] - -* For ``P2PKH``, the ``scriptCode`` is ``0x1976a914{20-byte-pubkey-hash}88ac``. - -* For ``P2SH``, the ``scriptCode`` is the ``script`` serialized as scripts inside ``CTxOut``. +``````````````````` +The script being currently executed: ``redeemScript`` for P2SH, or ``scriptPubKey`` in the general case. This +is the same script as serialized in the Sprout transaction digest algorithm. 10c: value ````````` @@ -273,19 +270,6 @@ Refer to the reference implementation, reproduced below, for the precise algorit return ss.GetHash(); -Restrictions on public key type -=============================== - -[TODO: decide whether we want to implement this policy] - -As a default policy, only compressed public keys are accepted in ``P2PKH`` and ``P2SH``. Each public key -passed to a sigop must be a compressed key: the first byte MUST be either ``0x02`` or ``0x03``, and the size -MUST be 33 bytes. Transactions that break this rule will not be relayed or mined by default. - -Since this policy is preparation for a future softfork proposal, to avoid potential future funds loss, users -MUST NOT use uncompressed keys. - - Example ======= From 7fb24671316944f86daf236b4854934ad76907e0 Mon Sep 17 00:00:00 2001 From: str4d Date: Fri, 16 Feb 2018 22:02:57 +0000 Subject: [PATCH 21/28] Update inner hash personalizations --- drafts/str4d-overwinter-sighash/draft1.rst | 31 +++++++++++++++------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index aac6f9ee..218b21e3 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -130,6 +130,8 @@ it is always defined for transactions that use this algorithm. [#ZIP-overwinter- * If the ``ANYONECANPAY`` flag is not set, ``hashPrevouts`` is the BLAKE2b-256 hash of the serialization of all input outpoints; + * The BLAKE2b-256 personalization field is set to ``ZcashPrevoutHash``. + * Otherwise, ``hashPrevouts`` is a ``uint256`` of ``0x0000......0000``. 4: ``hashSequence`` @@ -137,6 +139,8 @@ it is always defined for transactions that use this algorithm. [#ZIP-overwinter- * If none of the ``ANYONECANPAY``, ``SINGLE``, ``NONE`` sighash type is set, ``hashSequence`` is the BLAKE2b-256 hash of the serialization of ``nSequence`` of all inputs; + * The BLAKE2b-256 personalization field is set to ``ZcashSequencHash``. + * Otherwise, ``hashSequence`` is a ``uint256`` of ``0x0000......0000``. 5: ``hashOutputs`` @@ -148,6 +152,8 @@ it is always defined for transactions that use this algorithm. [#ZIP-overwinter- * If sighash type is ``SINGLE`` and the input index is smaller than the number of outputs, ``hashOutputs`` is the BLAKE2b-256 hash of the output (serialized as above) with the same index as the input; + * The BLAKE2b-256 personalization field is set to ``ZcashOutputsHash`` in both cases above. + * Otherwise, ``hashOutputs`` is a ``uint256`` of ``0x0000......0000``. [#01-change]_ 6: ``hashJoinSplits`` @@ -155,6 +161,8 @@ it is always defined for transactions that use this algorithm. [#ZIP-overwinter- * If ``vjoinsplits`` is non-empty, ``hashJoinSplits`` is the BLAKE2b-256 hash of the serialization of all JoinSplits (in their canonical transaction serialization format) concatenated with the joinSplitPubKey; + * The BLAKE2b-256 personalization field is set to ``ZcashJSplitsHash``. + * Note that while signatures are ommitted, the JoinSplit proofs are included in the signature hash, as with v1 and v2 transactions. @@ -177,9 +185,6 @@ An 8-byte value of the amount of ZEC spent in this input. Notes ----- -When generating ``hashPrevouts``, ``hashSequence``, ``hashOutputs``, and ``hashJoinSplits``, the BLAKE2b-256 -personalization field is set to ``Zcash_Inner_Hash``. - The ``hashPrevouts``, ``hashSequence``, ``hashOutputs``, and ``hashJoinSplits`` calculated in an earlier verification may be reused in other inputs of the same transaction, so that the time complexity of the whole hashing process reduces from O(n\ :sup:`2`) to O(n). @@ -188,8 +193,14 @@ Refer to the reference implementation, reproduced below, for the precise algorit .. code:: cpp - const unsigned char ZCASH_INNER_HASH_PERSONALIZATION[16] = - {'Z','c','a','s','h','_','I','n','n','e','r','_','H','a','s','h'}; + const unsigned char ZCASH_PREVOUTS_HASH_PERSONALIZATION[16] = + {'Z','c','a','s','h','P','r','e','v','o','u','t','H','a','s','h'}; + const unsigned char ZCASH_SEQUENCE_HASH_PERSONALIZATION[16] = + {'Z','c','a','s','h','S','e','q','u','e','n','c','H','a','s','h'}; + const unsigned char ZCASH_OUTPUTS_HASH_PERSONALIZATION[16] = + {'Z','c','a','s','h','O','u','t','p','u','t','s','H','a','s','h'}; + const unsigned char ZCASH_JOINSPLITS_HASH_PERSONALIZATION[16] = + {'Z','c','a','s','h','J','S','p','l','i','t','s','H','a','s','h'}; uint256 hashPrevouts; uint256 hashSequence; @@ -197,7 +208,7 @@ Refer to the reference implementation, reproduced below, for the precise algorit uint256 hashJoinSplits; if (!(nHashType & SIGHASH_ANYONECANPAY)) { - CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_PREVOUTS_HASH_PERSONALIZATION); for (unsigned int n = 0; n < txTo.vin.size(); n++) { ss << txTo.vin[n].prevout; } @@ -205,7 +216,7 @@ Refer to the reference implementation, reproduced below, for the precise algorit } if (!(nHashType & SIGHASH_ANYONECANPAY) && (nHashType & 0x1f) != SIGHASH_SINGLE && (nHashType & 0x1f) != SIGHASH_NONE) { - CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_SEQUENCE_HASH_PERSONALIZATION); for (unsigned int n = 0; n < txTo.vin.size(); n++) { ss << txTo.vin[n].nSequence; } @@ -213,19 +224,19 @@ Refer to the reference implementation, reproduced below, for the precise algorit } if ((nHashType & 0x1f) != SIGHASH_SINGLE && (nHashType & 0x1f) != SIGHASH_NONE) { - CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_OUTPUTS_HASH_PERSONALIZATION); for (unsigned int n = 0; n < txTo.vout.size(); n++) { ss << txTo.vout[n]; } hashOutputs = ss.GetHash(); } else if ((nHashType & 0x1f) == SIGHASH_SINGLE && nIn < txTo.vout.size()) { - CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_OUTPUTS_HASH_PERSONALIZATION); ss << txTo.vout[nIn]; hashOutputs = ss.GetHash(); } if (!txTo.vjoinsplit.empty()) { - CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_INNER_HASH_PERSONALIZATION); + CBLAKE2bWriter ss(SER_GETHASH, 0, ZCASH_JOINSPLITS_HASH_PERSONALIZATION); for (unsigned int n = 0; n < txTo.vjoinsplit.size(); n++) { ss << txTo.vjoinsplit[n]; } From abb6e742c1f4576723946743a54e0eaea915e5b9 Mon Sep 17 00:00:00 2001 From: str4d Date: Mon, 26 Feb 2018 22:47:20 +0000 Subject: [PATCH 22/28] Update category --- drafts/str4d-overwinter-sighash/draft1.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 218b21e3..621add1e 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -7,7 +7,7 @@ Credits: Johnson Lau Pieter Wuille Bitcoin-ABC - Category: Process + Category: Consensus Created: 2017-12-27 License: MIT From 349b4cf324dc77fba1146b9bd7be576f8fda37eb Mon Sep 17 00:00:00 2001 From: str4d Date: Mon, 26 Feb 2018 22:53:58 +0000 Subject: [PATCH 23/28] Remove TODO (resolved in 2d7401add59181ac233807d8cc3f9a737650de27) --- drafts/str4d-overwinter-sighash/draft1.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 621add1e..250b80e4 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -75,7 +75,7 @@ A new transaction digest algorithm is defined:: 9. sighash type of the signature (4-byte little endian) 10. If we are serializing an input (ie. this is not a JoinSplit signature hash): a. outpoint (32-byte hash + 4-byte little endian) - b. scriptCode of the input (serialized as scripts inside CTxOuts) [TODO] + b. scriptCode of the input (serialized as scripts inside CTxOuts) c. value of the output spent by this input (8-byte little endian) d. nSequence of the input (4-byte little endian) From 3d0c5ecb4cde29354f10c09dde316543217a21d3 Mon Sep 17 00:00:00 2001 From: str4d Date: Mon, 26 Feb 2018 22:59:28 +0000 Subject: [PATCH 24/28] Cleanups --- drafts/str4d-overwinter-sighash/draft1.rst | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 250b80e4..e46d8a3e 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -43,7 +43,7 @@ Unfortunately, there are at least 2 weaknesses in the original SignatureHash tra transaction. Therefore, data hashing grows in O(n\ :sup:`2`) as the number of sigops in a transaction increases. While a 1 MB block would normally take 2 seconds to verify with an average computer in 2015, a 1MB transaction with 5569 sigops may take 25 seconds to verify. This could be fixed by optimizing the digest - algorithm by introducing some reusable “midstate”, so the time complexity becomes O(n). [#quadratic]_ + algorithm by introducing some reusable "midstate", so the time complexity becomes O(n). [#quadratic]_ * The algorithm does not involve the value being spent by the input. This is usually not a problem for online network nodes as they could request for the specified transaction to acquire the output value. For an @@ -73,7 +73,7 @@ A new transaction digest algorithm is defined:: 7. nLockTime of the transaction (4-byte little endian) 8. nExpiryHeight of the transaction (4-byte little endian) 9. sighash type of the signature (4-byte little endian) - 10. If we are serializing an input (ie. this is not a JoinSplit signature hash): + 10. If we are serializing an input (i.e. this is not a JoinSplit signature hash): a. outpoint (32-byte hash + 4-byte little endian) b. scriptCode of the input (serialized as scripts inside CTxOuts) c. value of the output spent by this input (8-byte little endian) @@ -84,7 +84,7 @@ replay protection inspired by BUIP-HF v1.2. [#BUIP-HF]_ The new algorithm MUST be used for signatures created over the Overwinter transaction format. [#ZIP-overwinter-tx-format]_ Combined with the new consensus rule that v1 and v2 transaction formats will be -invalid from the Overwinter upgrade, [#ZIP-overwinter-tx-format]_ this effectively means that all transactions +invalid from the Overwinter upgrade, [#ZIP-overwinter-tx-format]_ this effectively means that all transaction signatures from the Overwinter activation height will use the new algorithm. [#ZIP0000]_ The BLAKE2b-256 personalization field [#BLAKE2-personalization]_ is set to:: @@ -180,13 +180,13 @@ is the same script as serialized in the Sprout transaction digest algorithm. 10c: value ````````` -An 8-byte value of the amount of ZEC spent in this input. +An 8-byte little-endian value of the amount, in zatoshi, spent in this input. Notes ----- The ``hashPrevouts``, ``hashSequence``, ``hashOutputs``, and ``hashJoinSplits`` calculated in an earlier -verification may be reused in other inputs of the same transaction, so that the time complexity of the whole +verification can be reused in other inputs of the same transaction, so that the time complexity of the whole hashing process reduces from O(n\ :sup:`2`) to O(n). Refer to the reference implementation, reproduced below, for the precise algorithm: @@ -202,6 +202,7 @@ Refer to the reference implementation, reproduced below, for the precise algorit const unsigned char ZCASH_JOINSPLITS_HASH_PERSONALIZATION[16] = {'Z','c','a','s','h','J','S','p','l','i','t','s','H','a','s','h'}; + // The default values are zeroes uint256 hashPrevouts; uint256 hashSequence; uint256 hashOutputs; @@ -317,7 +318,7 @@ References * `New Bitcoin vulnerability: A transaction that takes at least 3 minutes to verify `_ * `The Megatransaction: Why Does It Take 25 Seconds? `_ .. [#offline-wallets] `SIGHASH_WITHINPUTVALUE: Super-lightweight HW wallets and offline data `_ -.. [#BIP0143] `Transaction Signature Verification for Version 0 Witness Program `_ +.. [#BIP0143] `BIP 143: Transaction Signature Verification for Version 0 Witness Program `_ .. [#BUIP-HF] `BUIP-HF Digest for replay protected signature verification across hard forks, version 1.2 `_ .. [#ZIP0000] ZIP???: Overwinter Network Upgrade .. [#ZIP-activation-mechanism] ZIP???: Network Upgrade Activation Mechanism From f39f59c51d42749e013c5d7c50afd1982406b7e0 Mon Sep 17 00:00:00 2001 From: str4d Date: Mon, 26 Feb 2018 23:05:52 +0000 Subject: [PATCH 25/28] Update ZIP references --- drafts/str4d-overwinter-sighash/draft1.rst | 26 ++++++++++------------ 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index e46d8a3e..4de53b76 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -82,19 +82,18 @@ A new transaction digest algorithm is defined:: The new algorithm is based on the Bitcoin transaction digest algorithm defined in BIP 143, [#BIP0143]_ with replay protection inspired by BUIP-HF v1.2. [#BUIP-HF]_ -The new algorithm MUST be used for signatures created over the Overwinter transaction format. -[#ZIP-overwinter-tx-format]_ Combined with the new consensus rule that v1 and v2 transaction formats will be -invalid from the Overwinter upgrade, [#ZIP-overwinter-tx-format]_ this effectively means that all transaction -signatures from the Overwinter activation height will use the new algorithm. [#ZIP0000]_ +The new algorithm MUST be used for signatures created over the Overwinter transaction format. [#ZIP0202]_ +Combined with the new consensus rule that v1 and v2 transaction formats will be invalid from the Overwinter +upgrade, [#ZIP0202]_ this effectively means that all transaction signatures from the Overwinter activation +height will use the new algorithm. [#ZIP0000]_ The BLAKE2b-256 personalization field [#BLAKE2-personalization]_ is set to:: "ZcashSigHash" || CONSENSUS_BRANCH_ID ``CONSENSUS_BRANCH_ID`` is the little-endian encoding of ``BRANCH_ID`` for the epoch of the block containing -the transaction. [#ZIP-activation-mechanism]_ Domain separation of the signature hash across parallel branches -provides replay protection: transactions targeted for one branch will have invalid signatures on other -branches. +the transaction. [#ZIP0200]_ Domain separation of the signature hash across parallel branches provides replay +protection: transactions targeted for one branch will have invalid signatures on other branches. Transaction creators MUST specify the epoch they want their transaction to be mined in. Across a network upgrade, this means that if a transaction is not mined before the activation height, it will never be mined. @@ -118,12 +117,12 @@ The items 7, 9, 10a, 10d have the same meaning as the original algorithm from Bi 1: ``header`` ````````````` Deserialized into two transaction properties: ``fOverwintered`` and ``nVersion``. For transactions that use -this transaction digest algorithm, ``fOverwintered`` is always set. [#ZIP-overwinter-tx-format]_ +this transaction digest algorithm, ``fOverwintered`` is always set. [#ZIP0202]_ 2: ``nVersionGroupId`` `````````````````````` Provides domain separation of ``nVersion``. It is only defined if ``fOverwintered`` is set, which means that -it is always defined for transactions that use this algorithm. [#ZIP-overwinter-tx-format]_ +it is always defined for transactions that use this algorithm. [#ZIP0202]_ 3: ``hashPrevouts`` ``````````````````` @@ -170,8 +169,7 @@ it is always defined for transactions that use this algorithm. [#ZIP-overwinter- 8: ``nExpiryHeight`` ```````````````````` -The block height after which the transaction becomes unilaterally invalid, and can never be mined. -[#ZIP-tx-expiry]_ +The block height after which the transaction becomes unilaterally invalid, and can never be mined. [#ZIP0203]_ 10b: ``scriptCode`` ``````````````````` @@ -320,11 +318,11 @@ References .. [#offline-wallets] `SIGHASH_WITHINPUTVALUE: Super-lightweight HW wallets and offline data `_ .. [#BIP0143] `BIP 143: Transaction Signature Verification for Version 0 Witness Program `_ .. [#BUIP-HF] `BUIP-HF Digest for replay protected signature verification across hard forks, version 1.2 `_ +.. [#ZIP0202] `ZIP 202: Version 3 Transaction Format for Overwinter `_ .. [#ZIP0000] ZIP???: Overwinter Network Upgrade -.. [#ZIP-activation-mechanism] ZIP???: Network Upgrade Activation Mechanism -.. [#ZIP-overwinter-tx-format] ZIP???: Overwinter Transaction Format +.. [#ZIP0200] `ZIP 200: Network Upgrade Mechanism `_ .. [#BLAKE2-personalization] `"BLAKE2: simpler, smaller, fast as MD5", Section 2.8 `_ .. [#01-change] In the original algorithm, a ``uint256`` of ``0x0000......0001`` is committed if the input index for a ``SINGLE`` signature is greater than or equal to the number of outputs. In this ZIP a ``0x0000......0000`` is commited, without changing the semantics. -.. [#ZIP-tx-expiry] ZIP???: Transaction expiry +.. [#ZIP0203] `ZIP 203: Transaction Expiry `_ From 519ca18c741e1e0b3fd9ced9ea67edb2b050476e Mon Sep 17 00:00:00 2001 From: str4d Date: Wed, 28 Feb 2018 22:38:51 +0000 Subject: [PATCH 26/28] Address remaining comments --- drafts/str4d-overwinter-sighash/draft1.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/drafts/str4d-overwinter-sighash/draft1.rst index 4de53b76..f11f1c45 100644 --- a/drafts/str4d-overwinter-sighash/draft1.rst +++ b/drafts/str4d-overwinter-sighash/draft1.rst @@ -85,7 +85,7 @@ replay protection inspired by BUIP-HF v1.2. [#BUIP-HF]_ The new algorithm MUST be used for signatures created over the Overwinter transaction format. [#ZIP0202]_ Combined with the new consensus rule that v1 and v2 transaction formats will be invalid from the Overwinter upgrade, [#ZIP0202]_ this effectively means that all transaction signatures from the Overwinter activation -height will use the new algorithm. [#ZIP0000]_ +height will use the new algorithm. [#ZIP0201]_ The BLAKE2b-256 personalization field [#BLAKE2-personalization]_ is set to:: @@ -289,7 +289,7 @@ TBC Deployment ========== -This proposal is deployed with the Overwinter network upgrade. +This proposal is deployed with the Overwinter network upgrade. [#ZIP0201]_ Backward compatibility @@ -319,7 +319,7 @@ References .. [#BIP0143] `BIP 143: Transaction Signature Verification for Version 0 Witness Program `_ .. [#BUIP-HF] `BUIP-HF Digest for replay protected signature verification across hard forks, version 1.2 `_ .. [#ZIP0202] `ZIP 202: Version 3 Transaction Format for Overwinter `_ -.. [#ZIP0000] ZIP???: Overwinter Network Upgrade +.. [#ZIP0201] `ZIP 201: Network Peer Management for Overwinter `_ .. [#ZIP0200] `ZIP 200: Network Upgrade Mechanism `_ .. [#BLAKE2-personalization] `"BLAKE2: simpler, smaller, fast as MD5", Section 2.8 `_ .. [#01-change] In the original algorithm, a ``uint256`` of ``0x0000......0001`` is committed if the input From c4baac76722f99650ab4f8751f6113757c44b48a Mon Sep 17 00:00:00 2001 From: str4d Date: Wed, 28 Feb 2018 22:39:28 +0000 Subject: [PATCH 27/28] Move ZIP 143 out of drafts --- drafts/str4d-overwinter-sighash/draft1.rst => zip-0143.rst | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename drafts/str4d-overwinter-sighash/draft1.rst => zip-0143.rst (100%) diff --git a/drafts/str4d-overwinter-sighash/draft1.rst b/zip-0143.rst similarity index 100% rename from drafts/str4d-overwinter-sighash/draft1.rst rename to zip-0143.rst From 60c74245091a121a86e1989d40d22cc703d84f0d Mon Sep 17 00:00:00 2001 From: str4d Date: Thu, 1 Mar 2018 01:18:52 +0000 Subject: [PATCH 28/28] Add terminology references --- zip-0143.rst | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/zip-0143.rst b/zip-0143.rst index f11f1c45..d0622e6a 100644 --- a/zip-0143.rst +++ b/zip-0143.rst @@ -17,6 +17,10 @@ Terminology The key words "MUST" and "MUST NOT" in this document are to be interpreted as described in RFC 2119. +The terms "branch" and "network upgrade" in this document are to be interpreted as described in ZIP 200. [#ZIP0200]_ + +The term "Overwinter" in this document is to be interpreted as described in ZIP 201. [#ZIP0201]_ + Abstract ======== @@ -309,6 +313,8 @@ https://github.com/zcash/zcash/pull/2903 References ========== +.. [#ZIP0200] `ZIP 200: Network Upgrade Mechanism `_ +.. [#ZIP0201] `ZIP 201: Network Peer Management for Overwinter `_ .. [#wiki-checksig] https://en.bitcoin.it/wiki/OP_CHECKSIG .. [#zcash-protocol] `Zcash Protocol Specification, Section 4.6 `_ .. [#quadratic] @@ -319,8 +325,6 @@ References .. [#BIP0143] `BIP 143: Transaction Signature Verification for Version 0 Witness Program `_ .. [#BUIP-HF] `BUIP-HF Digest for replay protected signature verification across hard forks, version 1.2 `_ .. [#ZIP0202] `ZIP 202: Version 3 Transaction Format for Overwinter `_ -.. [#ZIP0201] `ZIP 201: Network Peer Management for Overwinter `_ -.. [#ZIP0200] `ZIP 200: Network Upgrade Mechanism `_ .. [#BLAKE2-personalization] `"BLAKE2: simpler, smaller, fast as MD5", Section 2.8 `_ .. [#01-change] In the original algorithm, a ``uint256`` of ``0x0000......0001`` is committed if the input index for a ``SINGLE`` signature is greater than or equal to the number of outputs. In this ZIP a