From 347d6792a0057c3734139595d8c77601b0f7e074 Mon Sep 17 00:00:00 2001 From: Simon Date: Mon, 22 Jan 2018 21:27:45 -0800 Subject: [PATCH 1/8] Draft of Overwinter network handshake --- .../bitcartel-nu0-network-handshake/draft.rst | 213 ++++++++++++++++++ 1 file changed, 213 insertions(+) create mode 100644 drafts/bitcartel-nu0-network-handshake/draft.rst diff --git a/drafts/bitcartel-nu0-network-handshake/draft.rst b/drafts/bitcartel-nu0-network-handshake/draft.rst new file mode 100644 index 00000000..2d442b9e --- /dev/null +++ b/drafts/bitcartel-nu0-network-handshake/draft.rst @@ -0,0 +1,213 @@ +:: + + ZIP: ??? + Title: Network Upgrade Zero ("OverWinter") Network Handshake + Author: Simon Liu + Comments-Summary: No comments yet. + Category: Process + Created: 2018-01-15 + License: MIT + +Terminology +=========== + +The key words "MUST", "MUST NOT", "SHOULD", and "MAY" in this document are to be interpreted as described in RFC 2119. + +"Legacy" - pre-NU0 + +"NU0" - Network upgade zero + +"Overwinter" - Code-name for NU0 + +Abstract +======== + +This proposal defines an upgrade to network handshake format required for Network Upgrade Activation Mechanism [#zip-0???]_. + +Related to [#zip-0143]_ + +Motivation +========== + +With scheduled network upgrades, at the activation height, nodes on each branch should disconnect from nodes on other branches and only accept new incoming connections from nodes on the same branch (??? or accepted set of branches ???) + +Specification +============= + +When a new inbound connection is received or an outbound connection +created, a CNode object is instantiated with the version field set to +INIT_PROTO_VERSION (209). This value is not transmitted across the network, but for legacy reasons and technical debt beyond the scope of this ZIP, this value will not be changed. + +Once the two nodes have connected and perform a handshake to negotiate the protocol version, version field of CNode will be updated. As documented here https://en.bitcoin.it/wiki/Version_Handshake +on connection, "version" and "verack" messages are exchanged.:: + + L -> R: Send version message with the local peer's version + R -> L: Send version message back + R -> L: Send verack message + R: Sets version to the minimum of the 2 versions + L -> R: Send verack message after receiving version message from R + L: Sets version to the minimum of the 2 versions + +To send a version message, the node will invoke PushVersion():: + + void CNode::PushVersion() + PushMessage("version", PROTOCOL_VERSION, nLocalServices, nTime, addrYou, addrMe, + +Where: + +- Legacy PROTOCOL_VERSION is 170002 +- Overwinter PROTOCOL_VERSION is 170003 + + +Rejecting Legacy Connections +---------------------------- + +Currently, nodes will reject connections from nodes with a protocol version lower than the other node's minimum supported protocol version. This value is defined as:: + + //! disconnect from peers older than this proto version + static const int MIN_PEER_PROTO_VERSION = 170002; + + ... + + if (pfrom->nVersion < MIN_PEER_PROTO_VERSION) + { + // disconnect from old peers running protocol version we don't support. + +Prior to activation, Overwinter nodes will contain the following constants:: + + static const int PROTOCOL_VERSION = 170003; + static const int MIN_PEER_PROTO_VERSION = 170002; + +This allows pre-Overwinter nodes and Overwinter nodes to remain connected prior to activation. + +However, once activation occurs, Overwinter nodes must reject connections from legacy 170002 nodes. + +To achieve this, the version message will be upgraded for Overwinter to include one extra field, a 32-bit BRANCH_ID. Since legacy nodes do not send this field as part of their version message during the handshake, the legacy node's connection will be rejected. + +We can do this by implementing code such as:: + + bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, int64_t nTimeReceived) + ... + if (strCommand == "version") + ... + vRecv >> addrFrom >> nNonce; + vRecv >> LIMITED_STRING(pfrom->strSubVer, 256); + pfrom->cleanSubVer = SanitizeString(pfrom->strSubVer); + vRecv >> pfrom->nStartingHeight; + vRecv >> pfrom->fRelayTxes; // set to true after we get the first filter* message + + // Overwinter node + if (isOverwinterHandshakeActivated()) { + if (!vRecv.empty()) { + vRecv >> pfrom->branchID + } + else { + // disconnect as branch ID is missing + LogPrintf("peer=%d using obsolete version %i; disconnecting\n", pfrom->id, pfrom->nVersion); + pfrom->PushMessage("reject", strCommand, REJECT_OBSOLETE, + strprintf("Version must be %d or greater", OVERWINTER_PROTO_VERSION)); + pfrom->fDisconnect = true; + return false; + } + } + + +Network Coalescence +------------------- + +Prior to the activation of Overwinter, nodes running pre-Overwinter protocol version 170002 and the Overwinter protocol version 170003 remain connected with the same consensus rules, but it would be preferable for nodes supporting Overwinter to connect to other nodes supporting Overwinter. + +This will help the network partition smoothly, since nodes should already be connected to (a majority of) peers running the same protocol version. Otherwise am Overwinter node may find their connections to legacy nodes dropped at the activation height, leaving them isolated and potentially susceptible to eclipse attacks. [link] + +To assist network coalescence before the activation height, we update the eviction process to place a higher priority on evicting legacy nodes. + +This can be activated at n blocks before the activation block height, where n could be defined by a constant such as:: + + static const int NETWORK_COALESCE_BLOCK_PERIOD = 1000. + +The eviction code can be updated as follows:: + + static bool AttemptToEvictConnection(bool fPreferNewConnection) { + ... + // Protect connections with certain characteristics + ... + // Check version of eviction candidates + if (current_block_height >= (activationheight - NETWORK_COALESCE_BLOCK_PERIOD)) { + // if there exist any legacy nodes, keep them in the eviction set + // and at the same time remove overwinter nodes from eviction set. + // if there do not exist any legacy nodes, + // continue with existing behaviour. + + +Disconnecting Existing Connections +---------------------------------- + +It is likely that at the activation block height, an Overwinter node will still be connected to some Legacy nodes. + +Currently, when connecting, a node must perform the networking handshake, and send the version message, before any other messages are processed. + +To disconnect existing connections, we can modify ProcessMessage so that the protocol version is always checked after Overwinter activates. + +Example code:: + + bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, int64_t nTimeReceived) + ... + else if (pfrom->nVersion == 0) + { + // Must have a version message before anything else + Misbehaving(pfrom->GetId(), 1); + return false; + } + else if (strCommand == "verack") + { + ... + } + + // Disconnect existing connection if: + // 1. The version message has been received + // 2. Overwinter is active + // 3. Version is legacy + else if ( + pfrom->nVersion != 0 && + isOverwinterActivated() && + pfrom->nVersion < OVERWINTER_PROTO_VERSION ) + { + LogPrintf("peer=%d using obsolete version %i; disconnecting\n", pfrom->id, pfrom->nVersion); + pfrom->PushMessage("reject", strCommand, REJECT_OBSOLETE, + strprintf("Version must be %d or greater", OVERWINTER_PROTO_VERSION)); + pfrom->fDisconnect = true; + return false; + } + + + +Deployment +========== + +This proposal will be deployed with the Overwinter network upgrade. + +Testnet: + +Mainnet: + +Backward compatibility +====================== + +This proposal intentionally creates what is known as a "bilateral hard fork" between Legacy software and Overwinter compatible software. Use of this new handshake requires that all network participants upgrade their software to a compatible version within the upgrade window + +Legacy software will accept the numerically larger Overwinter protocol version as valid, but Overwinter compatible software will reject the legacy nodes as they will not send the BRANCH_ID as part of the version message. + +Reference Implementation +======================== + +TBC + + +References +========== + +Partition nodes with old protocol version from network in advance of hard fork https://github.com/zcash/zcash/issues/2775 + +https://en.bitcoin.it/wiki/Protocol_documentation#version + +.. [#zip-0???] Network Upgrade Activation Mechanism From e925580b0de928c5c20a96fe69f6313a726ca110 Mon Sep 17 00:00:00 2001 From: bitcartel Date: Tue, 6 Feb 2018 16:09:01 -0800 Subject: [PATCH 2/8] Update, remove branch id. --- .../bitcartel-nu0-network-handshake/draft.rst | 68 +++++++------------ 1 file changed, 23 insertions(+), 45 deletions(-) diff --git a/drafts/bitcartel-nu0-network-handshake/draft.rst b/drafts/bitcartel-nu0-network-handshake/draft.rst index 2d442b9e..f5f83e31 100644 --- a/drafts/bitcartel-nu0-network-handshake/draft.rst +++ b/drafts/bitcartel-nu0-network-handshake/draft.rst @@ -29,17 +29,14 @@ Related to [#zip-0143]_ Motivation ========== -With scheduled network upgrades, at the activation height, nodes on each branch should disconnect from nodes on other branches and only accept new incoming connections from nodes on the same branch (??? or accepted set of branches ???) +With scheduled network upgrades, at the activation height, nodes on each branch should disconnect from nodes on other branches and only accept new incoming connections from nodes on the same branch. Specification ============= -When a new inbound connection is received or an outbound connection -created, a CNode object is instantiated with the version field set to -INIT_PROTO_VERSION (209). This value is not transmitted across the network, but for legacy reasons and technical debt beyond the scope of this ZIP, this value will not be changed. +When a new inbound connection is received or an outbound connection created, a CNode object is instantiated with the version field set to INIT_PROTO_VERSION which has a value of 209. This value is not transmitted across the network, but for legacy reasons and technical debt beyond the scope of this ZIP, this value will not be changed. -Once the two nodes have connected and perform a handshake to negotiate the protocol version, version field of CNode will be updated. As documented here https://en.bitcoin.it/wiki/Version_Handshake -on connection, "version" and "verack" messages are exchanged.:: +Once the two nodes have connected and started the handshake to negotiate the protocol version, the version field of CNode will be updated. The handshake involves "version" and "verack" messages being exchanged.:: L -> R: Send version message with the local peer's version R -> L: Send version message back @@ -47,27 +44,34 @@ on connection, "version" and "verack" messages are exchanged.:: R: Sets version to the minimum of the 2 versions L -> R: Send verack message after receiving version message from R L: Sets version to the minimum of the 2 versions + + Source: https://en.bitcoin.it/wiki/Version_Handshake To send a version message, the node will invoke PushVersion():: - void CNode::PushVersion() - PushMessage("version", PROTOCOL_VERSION, nLocalServices, nTime, addrYou, addrMe, + void CNode::PushVersion() { + ... + PushMessage("version", PROTOCOL_VERSION, nLocalServices, nTime, addrYou, addrMe, + ... + } Where: -- Legacy PROTOCOL_VERSION is 170002 -- Overwinter PROTOCOL_VERSION is 170003 +- Pre-Overwinter PROTOCOL_VERSION is 170002 +- OVERWINTER_PROTO_VERSION is 170003 -Rejecting Legacy Connections ----------------------------- +Rejecting Pre-Overwinter Connections +------------------------------------ -Currently, nodes will reject connections from nodes with a protocol version lower than the other node's minimum supported protocol version. This value is defined as:: +Currently, nodes will reject connections from nodes with a protocol version lower than the other node's minimum supported protocol version. + +This value is defined as:: //! disconnect from peers older than this proto version static const int MIN_PEER_PROTO_VERSION = 170002; - ... +With rejection codified as:: if (pfrom->nVersion < MIN_PEER_PROTO_VERSION) { @@ -78,38 +82,12 @@ Prior to activation, Overwinter nodes will contain the following constants:: static const int PROTOCOL_VERSION = 170003; static const int MIN_PEER_PROTO_VERSION = 170002; -This allows pre-Overwinter nodes and Overwinter nodes to remain connected prior to activation. +This allows pre-Overwinter nodes (which only supports protocol version 170002) and Overwinter nodes (which support both 170002 and 170003) to remain connected prior to activation. -However, once activation occurs, Overwinter nodes must reject connections from legacy 170002 nodes. +However, once Overwinter activates, Overwinter nodes should: -To achieve this, the version message will be upgraded for Overwinter to include one extra field, a 32-bit BRANCH_ID. Since legacy nodes do not send this field as part of their version message during the handshake, the legacy node's connection will be rejected. - -We can do this by implementing code such as:: - - bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, int64_t nTimeReceived) - ... - if (strCommand == "version") - ... - vRecv >> addrFrom >> nNonce; - vRecv >> LIMITED_STRING(pfrom->strSubVer, 256); - pfrom->cleanSubVer = SanitizeString(pfrom->strSubVer); - vRecv >> pfrom->nStartingHeight; - vRecv >> pfrom->fRelayTxes; // set to true after we get the first filter* message - - // Overwinter node - if (isOverwinterHandshakeActivated()) { - if (!vRecv.empty()) { - vRecv >> pfrom->branchID - } - else { - // disconnect as branch ID is missing - LogPrintf("peer=%d using obsolete version %i; disconnecting\n", pfrom->id, pfrom->nVersion); - pfrom->PushMessage("reject", strCommand, REJECT_OBSOLETE, - strprintf("Version must be %d or greater", OVERWINTER_PROTO_VERSION)); - pfrom->fDisconnect = true; - return false; - } - } +- reject new connections from pre-Overwinter nodes +- disconnect any existing conncetions to pre-Overwinter nodes Network Coalescence @@ -195,7 +173,7 @@ Backward compatibility This proposal intentionally creates what is known as a "bilateral hard fork" between Legacy software and Overwinter compatible software. Use of this new handshake requires that all network participants upgrade their software to a compatible version within the upgrade window -Legacy software will accept the numerically larger Overwinter protocol version as valid, but Overwinter compatible software will reject the legacy nodes as they will not send the BRANCH_ID as part of the version message. +Legacy software will accept the numerically larger Overwinter protocol version as valid, but Overwinter compatible software will reject the legacy nodes once Overwinter activates by rejecting protocol versions lower than the Overwinter protocol version number. Reference Implementation ======================== From 28760533b758c2fae9b177027543e957b6316871 Mon Sep 17 00:00:00 2001 From: bitcartel Date: Thu, 8 Feb 2018 00:42:51 -0800 Subject: [PATCH 3/8] Update --- .../bitcartel-nu0-network-handshake/draft.rst | 67 +++++++++++++------ 1 file changed, 48 insertions(+), 19 deletions(-) diff --git a/drafts/bitcartel-nu0-network-handshake/draft.rst b/drafts/bitcartel-nu0-network-handshake/draft.rst index f5f83e31..19f89a97 100644 --- a/drafts/bitcartel-nu0-network-handshake/draft.rst +++ b/drafts/bitcartel-nu0-network-handshake/draft.rst @@ -71,7 +71,7 @@ This value is defined as:: //! disconnect from peers older than this proto version static const int MIN_PEER_PROTO_VERSION = 170002; -With rejection codified as:: +With rejection implemented as:: if (pfrom->nVersion < MIN_PEER_PROTO_VERSION) { @@ -84,7 +84,7 @@ Prior to activation, Overwinter nodes will contain the following constants:: This allows pre-Overwinter nodes (which only supports protocol version 170002) and Overwinter nodes (which support both 170002 and 170003) to remain connected prior to activation. -However, once Overwinter activates, Overwinter nodes should: +These values cannot be changed at run-time, so when Overwinter activates, Overwinter nodes should take steps to: - reject new connections from pre-Overwinter nodes - disconnect any existing conncetions to pre-Overwinter nodes @@ -95,36 +95,62 @@ Network Coalescence Prior to the activation of Overwinter, nodes running pre-Overwinter protocol version 170002 and the Overwinter protocol version 170003 remain connected with the same consensus rules, but it would be preferable for nodes supporting Overwinter to connect to other nodes supporting Overwinter. -This will help the network partition smoothly, since nodes should already be connected to (a majority of) peers running the same protocol version. Otherwise am Overwinter node may find their connections to legacy nodes dropped at the activation height, leaving them isolated and potentially susceptible to eclipse attacks. [link] +This would help the network partition smoothly, since nodes should already be connected to (a majority of) peers running the same protocol version. Otherwise an Overwinter node may find their connections to legacy nodes droppedsuddenly at the activation height, potentiallyleaving them isolated and susceptible to eclipse attacks. [link] To assist network coalescence before the activation height, we update the eviction process to place a higher priority on evicting legacy nodes. -This can be activated at n blocks before the activation block height, where n could be defined by a constant such as:: +Currently, an eviction process takes place when new inbound connections arrive, but the node has already connected to the maximum number of inbound peers:: - static const int NETWORK_COALESCE_BLOCK_PERIOD = 1000. + if (nInbound >= nMaxInbound) + { + if (!AttemptToEvictConnection(whitelisted)) { + // No connection to evict, disconnect the new connection + LogPrint("net", "failed to find an eviction candidate - connection dropped (full)\n"); + CloseSocket(hSocket); + return; + } + } -The eviction code can be updated as follows:: +We update this process by adding behaviour so that the set of eviction candidates will prefer pre-Overwinter node, when the chain tip is in a period N blocks before the activation block height, where N is defined as:: + + static const int NETWORK_UPGRADE_PEER_PREFERENCE_BLOCK_PERIOD = 1000. + +The eviction candidates can be modified as so:: static bool AttemptToEvictConnection(bool fPreferNewConnection) { ... // Protect connections with certain characteristics ... - // Check version of eviction candidates - if (current_block_height >= (activationheight - NETWORK_COALESCE_BLOCK_PERIOD)) { - // if there exist any legacy nodes, keep them in the eviction set - // and at the same time remove overwinter nodes from eviction set. - // if there do not exist any legacy nodes, - // continue with existing behaviour. + // Check version of eviction candidates... + // If we are connected to any pre-Overwinter nodes, keep them in the eviction set and remove any Overwinter nodes + // If we are only connected to Overwinter nodes, continue with existing behaviour. + if ((height < nActivationHeight) && + (height >= (nActivationHeight - NETWORK_UPGRADE_PEER_PREFERENCE_BLOCK_PERIOD))) + { + // Find any nodes which don't support Overwinter protocol version + BOOST_FOREACH(const CNodeRef &node, vEvictionCandidates) { + if (node->nVersion < OVERWINTER_PROTO_VERSION) { + vTmpEvictionCandidates.push_back(node); + } + } + + // Prioritize these nodes by replacing eviction set with them + if (vTmpEvictionCandidates.size() > 0) { + vEvictionCandidates = vTmpEvictionCandidates; + } + } + +The existing method of disconnecting a candidate remains: + + vEvictionCandidates[0]->fDisconnect = true; + +The existing eviction process also classifies and divides eviction candidates into groups. If a group only has one peer, it will not be evicted. This means at least one pre-Overwinter node should remain connected upto the activation block height, barring any network issues or a high ban score. Disconnecting Existing Connections ---------------------------------- -It is likely that at the activation block height, an Overwinter node will still be connected to some Legacy nodes. - -Currently, when connecting, a node must perform the networking handshake, and send the version message, before any other messages are processed. - -To disconnect existing connections, we can modify ProcessMessage so that the protocol version is always checked after Overwinter activates. +At the activation block height, an Overwinter node may still remain connected to pre-Overwinter nodes. Currently, when connecting, a node can only perform the networking handshake once, where it sends the version message before any other messages are processed. To disconnect existing pre-Overwinter connections, ProcessMessage is modified so that once Overwinter activates, the protocol version of a peer is always checked when processing inbound messages. Example code:: @@ -171,9 +197,12 @@ Mainnet: Backward compatibility ====================== -This proposal intentionally creates what is known as a "bilateral hard fork" between Legacy software and Overwinter compatible software. Use of this new handshake requires that all network participants upgrade their software to a compatible version within the upgrade window +This proposal intentionally creates what is known as a hard fork where Overwinter nodes disconnect from pre-Overwinter nodes. + +Prior to the network upgrade activating, Overwinter and pre-Overwinter nodes are compatible and can connect to each other. However, Overwinter nodes will have a preference for connecting to other Overwinter nodes, so pre-Overwinter nodes will gradually be disconnected in the run up to activation. + +Once the network upgrades, even though pre-Overwinter nodes can still accept the numerically larger protocol version used by Overwinter as being valid, Overwinter nodes will always disconnect peers using lower protocol versions. -Legacy software will accept the numerically larger Overwinter protocol version as valid, but Overwinter compatible software will reject the legacy nodes once Overwinter activates by rejecting protocol versions lower than the Overwinter protocol version number. Reference Implementation ======================== From 02c3e88fb626a3c5afedb0ed39f7d794d0311bca Mon Sep 17 00:00:00 2001 From: bitcartel Date: Thu, 8 Feb 2018 14:32:04 -0800 Subject: [PATCH 4/8] Update --- .../bitcartel-nu0-network-handshake/draft.rst | 24 ++++++++++--------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/drafts/bitcartel-nu0-network-handshake/draft.rst b/drafts/bitcartel-nu0-network-handshake/draft.rst index 19f89a97..b7afc8a1 100644 --- a/drafts/bitcartel-nu0-network-handshake/draft.rst +++ b/drafts/bitcartel-nu0-network-handshake/draft.rst @@ -95,7 +95,7 @@ Network Coalescence Prior to the activation of Overwinter, nodes running pre-Overwinter protocol version 170002 and the Overwinter protocol version 170003 remain connected with the same consensus rules, but it would be preferable for nodes supporting Overwinter to connect to other nodes supporting Overwinter. -This would help the network partition smoothly, since nodes should already be connected to (a majority of) peers running the same protocol version. Otherwise an Overwinter node may find their connections to legacy nodes droppedsuddenly at the activation height, potentiallyleaving them isolated and susceptible to eclipse attacks. [link] +This would help the network partition smoothly, since nodes should already be connected to (a majority of) peers running the same protocol version. Otherwise an Overwinter node may find their connections to legacy nodes dropped suddenly at the activation height, potentially leaving them isolated and susceptible to eclipse attacks. [link] To assist network coalescence before the activation height, we update the eviction process to place a higher priority on evicting legacy nodes. @@ -111,7 +111,7 @@ Currently, an eviction process takes place when new inbound connections arrive, } } -We update this process by adding behaviour so that the set of eviction candidates will prefer pre-Overwinter node, when the chain tip is in a period N blocks before the activation block height, where N is defined as:: +We update this process by adding behaviour so that the set of eviction candidates will prefer pre-Overwinter nodes, when the chain tip is in a period N blocks before the activation block height, where N is defined as:: static const int NETWORK_UPGRADE_PEER_PREFERENCE_BLOCK_PERIOD = 1000. @@ -144,13 +144,13 @@ The existing method of disconnecting a candidate remains: vEvictionCandidates[0]->fDisconnect = true; -The existing eviction process also classifies and divides eviction candidates into groups. If a group only has one peer, it will not be evicted. This means at least one pre-Overwinter node should remain connected upto the activation block height, barring any network issues or a high ban score. +The existing eviction process will classify and divide eviction candidates into buckets called netgroups. If a netgroup only has one peer, it will not be evicted. This means at least one pre-Overwinter node will remain connected upto the activation block height, barring any network issues or a high ban score. Disconnecting Existing Connections ---------------------------------- -At the activation block height, an Overwinter node may still remain connected to pre-Overwinter nodes. Currently, when connecting, a node can only perform the networking handshake once, where it sends the version message before any other messages are processed. To disconnect existing pre-Overwinter connections, ProcessMessage is modified so that once Overwinter activates, the protocol version of a peer is always checked when processing inbound messages. +At the activation block height, an Overwinter node may still remain connected to pre-Overwinter nodes. Currently, when connecting, a node can only perform the networking handshake once, where it sends the version message before any other messages are processed. To disconnect existing pre-Overwinter connections, ProcessMessage is modified so that once Overwinter activates, if necessary, the protocol version of anexisting peer is validated when inbound messages arrive. Example code:: @@ -167,18 +167,20 @@ Example code:: ... } - // Disconnect existing connection if: - // 1. The version message has been received - // 2. Overwinter is active - // 3. Version is legacy + // Disconnect existing peer connection when: + // 1. Minimum peer version is less than Overwinter version + // 2. The version message has been received from a peer + // 3. The peer's version is pre-Overwinter + // 4. Overwinter is active else if ( + MIN_PEER_PROTO_VERSION < OVERWINTER_PROTO_VERSION && pfrom->nVersion != 0 && - isOverwinterActivated() && - pfrom->nVersion < OVERWINTER_PROTO_VERSION ) + pfrom->nVersion < OVERWINTER_PROTO_VERSION && + NetworkUpgradeActive(GetHeight(), chainparams.GetConsensus(), Consensus::UPGRADE_OVERWINTER)) { LogPrintf("peer=%d using obsolete version %i; disconnecting\n", pfrom->id, pfrom->nVersion); pfrom->PushMessage("reject", strCommand, REJECT_OBSOLETE, - strprintf("Version must be %d or greater", OVERWINTER_PROTO_VERSION)); + strprintf("Version must be %d or greater", OVERWINTER_PROTO_VERSION)); pfrom->fDisconnect = true; return false; } From 1a380bcb4a70ed7e3dae25b21b9f79e9c3e0e149 Mon Sep 17 00:00:00 2001 From: Simon Date: Wed, 21 Feb 2018 15:58:57 -0800 Subject: [PATCH 5/8] Update --- .../bitcartel-nu0-network-handshake/draft.rst | 77 +++++++++++-------- 1 file changed, 46 insertions(+), 31 deletions(-) diff --git a/drafts/bitcartel-nu0-network-handshake/draft.rst b/drafts/bitcartel-nu0-network-handshake/draft.rst index b7afc8a1..050ce709 100644 --- a/drafts/bitcartel-nu0-network-handshake/draft.rst +++ b/drafts/bitcartel-nu0-network-handshake/draft.rst @@ -1,30 +1,45 @@ :: - ZIP: ??? - Title: Network Upgrade Zero ("OverWinter") Network Handshake + ZIP: 201 + Title: Network Peer Management for Overwinter Author: Simon Liu - Comments-Summary: No comments yet. - Category: Process + Category: Network Created: 2018-01-15 License: MIT Terminology =========== -The key words "MUST", "MUST NOT", "SHOULD", and "MAY" in this document are to be interpreted as described in RFC 2119. +The key words "MUST", "MUST NOT", "SHOULD", and "MAY" in this document are to be interpreted as described in RFC 2119. [#RFC2119]_ -"Legacy" - pre-NU0 +The terms below are to be interpreted as follows: -"NU0" - Network upgade zero +Branch + A chain of blocks with common consensus rules, where the first block in the chain is either the genesis + block, or the child of a parent block created under an older set of consensus rules (i.e. the parent block + is a member of a different branch). By definition, every block belongs to at most one branch. + +Hard fork + The creation of a new branch by a change in the consensus rules of the network. Nodes that do not recognize + the new rules will continue to follow the old branch. + +Network upgrade + An intentional hard fork undertaken by the community in order to improve the network. + +Overwinter + Code-name for the first ZCash network upgrade, also known as Network Upgrade Zero. -"Overwinter" - Code-name for NU0 Abstract ======== -This proposal defines an upgrade to network handshake format required for Network Upgrade Activation Mechanism [#zip-0???]_. +This proposal defines an upgrade to the network handshake and peer management required for network upgrades following the Network Upgrade Activation Mechanism [#zip-0200]_. -Related to [#zip-0143]_ +Related to: + +- Transaction Signature Verification for Overwinter [#zip-0143]_. +- Version 3 Transaction Format for Overwinter [#zip-0202]_. +- Transaction Expiry [#zip-0203]_. Motivation ========== @@ -36,7 +51,7 @@ Specification When a new inbound connection is received or an outbound connection created, a CNode object is instantiated with the version field set to INIT_PROTO_VERSION which has a value of 209. This value is not transmitted across the network, but for legacy reasons and technical debt beyond the scope of this ZIP, this value will not be changed. -Once the two nodes have connected and started the handshake to negotiate the protocol version, the version field of CNode will be updated. The handshake involves "version" and "verack" messages being exchanged.:: +Once the two nodes have connected and started the handshake to negotiate the protocol version, the version field of CNode will be updated. The handshake [#bitcoin-version-handshake]_ involves "version" and "verack" messages being exchanged.:: L -> R: Send version message with the local peer's version R -> L: Send version message back @@ -44,18 +59,16 @@ Once the two nodes have connected and started the handshake to negotiate the pro R: Sets version to the minimum of the 2 versions L -> R: Send verack message after receiving version message from R L: Sets version to the minimum of the 2 versions - - Source: https://en.bitcoin.it/wiki/Version_Handshake -To send a version message, the node will invoke PushVersion():: +To send a version message, the node will invoke ``PushVersion()``:: void CNode::PushVersion() { ... - PushMessage("version", PROTOCOL_VERSION, nLocalServices, nTime, addrYou, addrMe, + PushMessage("version", PROTOCOL_VERSION, nLocalServices, nTime, addrYou, addrMe, ...); ... } -Where: +where: - Pre-Overwinter PROTOCOL_VERSION is 170002 - OVERWINTER_PROTO_VERSION is 170003 @@ -82,22 +95,22 @@ Prior to activation, Overwinter nodes will contain the following constants:: static const int PROTOCOL_VERSION = 170003; static const int MIN_PEER_PROTO_VERSION = 170002; -This allows pre-Overwinter nodes (which only supports protocol version 170002) and Overwinter nodes (which support both 170002 and 170003) to remain connected prior to activation. +A pre-Overwinter node is defined to be a node for which the highest supported protocol version is < 170003. The above constant definitions allow pre-Overwinter nodes that support protocol version 170002, and Overwinter nodes (which support both protocol versions 170002 and 170003 before Overwinter activation) to remain connected until the activation. These values cannot be changed at run-time, so when Overwinter activates, Overwinter nodes should take steps to: -- reject new connections from pre-Overwinter nodes -- disconnect any existing conncetions to pre-Overwinter nodes +- reject new connections from pre-Overwinter nodes; +- disconnect any existing connections to pre-Overwinter nodes. Network Coalescence ------------------- -Prior to the activation of Overwinter, nodes running pre-Overwinter protocol version 170002 and the Overwinter protocol version 170003 remain connected with the same consensus rules, but it would be preferable for nodes supporting Overwinter to connect to other nodes supporting Overwinter. +Prior to the activation of Overwinter, nodes running pre-Overwinter protocol version 170002 and the Overwinter protocol version 170003 remain connected with the same consensus rules, but it is desirable for nodes supporting Overwinter to connect preferentially to other nodes supporting Overwinter. -This would help the network partition smoothly, since nodes should already be connected to (a majority of) peers running the same protocol version. Otherwise an Overwinter node may find their connections to legacy nodes dropped suddenly at the activation height, potentially leaving them isolated and susceptible to eclipse attacks. [link] +This would help the network partition smoothly, since nodes should already be connected to (a majority of) peers running the same protocol version. Otherwise an Overwinter node may find their connections to Sprout nodes dropped suddenly at the activation height, potentially leaving them isolated and susceptible to eclipse attacks. [#eclipse-attack] -To assist network coalescence before the activation height, we update the eviction process to place a higher priority on evicting legacy nodes. +To assist network coalescence before the activation height, we update the eviction process to place a higher priority on evicting Sprout nodes. Currently, an eviction process takes place when new inbound connections arrive, but the node has already connected to the maximum number of inbound peers:: @@ -150,7 +163,7 @@ The existing eviction process will classify and divide eviction candidates into Disconnecting Existing Connections ---------------------------------- -At the activation block height, an Overwinter node may still remain connected to pre-Overwinter nodes. Currently, when connecting, a node can only perform the networking handshake once, where it sends the version message before any other messages are processed. To disconnect existing pre-Overwinter connections, ProcessMessage is modified so that once Overwinter activates, if necessary, the protocol version of anexisting peer is validated when inbound messages arrive. +At the activation block height, an Overwinter node may still remain connected to pre-Overwinter nodes. Currently, when connecting, a node can only perform the networking handshake once, where it sends the version message before any other messages are processed. To disconnect existing pre-Overwinter connections, ``ProcessMessage`` is modified so that once Overwinter activates, if necessary, the protocol version of an existing peer is validated when inbound messages arrive. Example code:: @@ -192,9 +205,6 @@ Deployment This proposal will be deployed with the Overwinter network upgrade. -Testnet: - -Mainnet: Backward compatibility ====================== @@ -209,14 +219,19 @@ Once the network upgrades, even though pre-Overwinter nodes can still accept the Reference Implementation ======================== -TBC +https://github.com/zcash/zcash/pull/2919 References ========== -Partition nodes with old protocol version from network in advance of hard fork https://github.com/zcash/zcash/issues/2775 +.. [#RFC2119] https://tools.ietf.org/html/rfc2119 +.. [#zip-0143] `Transaction Signature Verification for Overwinter `_ +.. [#zip-0200] `Network Upgrade Activation Mechanism `_ +.. [#zip-0202] `Version 3 Transaction Format for Overwinter `_ +.. [#zip-0203] `Transaction Expiry `_ +.. [#bitcoin-verson] https://en.bitcoin.it/wiki/Protocol_documentation#version +.. [#bitcoin-version-handshake] https://en.bitcoin.it/wiki/Version_Handshake +.. [#eclipse-attack] `Eclipse Attacks on Bitcoin’s Peer-to-Peer Network `_ +.. [#partition-discussion] `Partition nodes with old protocol version from network in advance of hard fork `_ -https://en.bitcoin.it/wiki/Protocol_documentation#version - -.. [#zip-0???] Network Upgrade Activation Mechanism From e2b332cbb8653936e3bf590ef4e45ebaad52c1ba Mon Sep 17 00:00:00 2001 From: Simon Date: Mon, 26 Feb 2018 08:35:34 -0800 Subject: [PATCH 6/8] Update --- .../bitcartel-nu0-network-handshake/draft.rst | 36 ++++++++----------- 1 file changed, 15 insertions(+), 21 deletions(-) diff --git a/drafts/bitcartel-nu0-network-handshake/draft.rst b/drafts/bitcartel-nu0-network-handshake/draft.rst index 050ce709..9b1ed6e3 100644 --- a/drafts/bitcartel-nu0-network-handshake/draft.rst +++ b/drafts/bitcartel-nu0-network-handshake/draft.rst @@ -68,16 +68,13 @@ To send a version message, the node will invoke ``PushVersion()``:: ... } -where: - -- Pre-Overwinter PROTOCOL_VERSION is 170002 -- OVERWINTER_PROTO_VERSION is 170003 +where ``PROTOCOL_VERSION`` is the highest protocol version supported by the node. Rejecting Pre-Overwinter Connections ------------------------------------ -Currently, nodes will reject connections from nodes with a protocol version lower than the other node's minimum supported protocol version. +Currently, nodes will reject connections from peers with an advertised protocol version lower than the other node's minimum supported protocol version. This value is defined as:: @@ -126,7 +123,8 @@ Currently, an eviction process takes place when new inbound connections arrive, We update this process by adding behaviour so that the set of eviction candidates will prefer pre-Overwinter nodes, when the chain tip is in a period N blocks before the activation block height, where N is defined as:: - static const int NETWORK_UPGRADE_PEER_PREFERENCE_BLOCK_PERIOD = 1000. + /** The period before a network upgrade activates, where connections to upgrading peers are preferred (in blocks). */ + static const int NETWORK_UPGRADE_PEER_PREFERENCE_BLOCK_PERIOD = 24 * 24 * 3; The eviction candidates can be modified as so:: @@ -137,12 +135,13 @@ The eviction candidates can be modified as so:: // Check version of eviction candidates... // If we are connected to any pre-Overwinter nodes, keep them in the eviction set and remove any Overwinter nodes // If we are only connected to Overwinter nodes, continue with existing behaviour. - if ((height < nActivationHeight) && - (height >= (nActivationHeight - NETWORK_UPGRADE_PEER_PREFERENCE_BLOCK_PERIOD))) + if (nActivationHeight > 0 && + height < nActivationHeight && + height >= nActivationHeight - NETWORK_UPGRADE_PEER_PREFERENCE_BLOCK_PERIOD) { // Find any nodes which don't support Overwinter protocol version BOOST_FOREACH(const CNodeRef &node, vEvictionCandidates) { - if (node->nVersion < OVERWINTER_PROTO_VERSION) { + if (node->nVersion < params.vUpgrades[Consensus::UPGRADE_OVERWINTER].nProtocolVersion) { vTmpEvictionCandidates.push_back(node); } } @@ -181,19 +180,16 @@ Example code:: } // Disconnect existing peer connection when: - // 1. Minimum peer version is less than Overwinter version - // 2. The version message has been received from a peer - // 3. The peer's version is pre-Overwinter - // 4. Overwinter is active - else if ( - MIN_PEER_PROTO_VERSION < OVERWINTER_PROTO_VERSION && - pfrom->nVersion != 0 && - pfrom->nVersion < OVERWINTER_PROTO_VERSION && - NetworkUpgradeActive(GetHeight(), chainparams.GetConsensus(), Consensus::UPGRADE_OVERWINTER)) + // 1. The version message has been received + // 2. Overwinter is active + // 3. Peer version is pre-Overwinter + else if (NetworkUpgradeActive(GetHeight(), chainparams.GetConsensus(), Consensus::UPGRADE_OVERWINTER) + && (pfrom->nVersion < chainparams.GetConsensus().vUpgrades[Consensus::UPGRADE_OVERWINTER].nProtocolVersion)) { LogPrintf("peer=%d using obsolete version %i; disconnecting\n", pfrom->id, pfrom->nVersion); pfrom->PushMessage("reject", strCommand, REJECT_OBSOLETE, - strprintf("Version must be %d or greater", OVERWINTER_PROTO_VERSION)); + strprintf("Version must be %d or greater", + chainparams.GetConsensus().vUpgrades[Consensus::UPGRADE_OVERWINTER].nProtocolVersion)); pfrom->fDisconnect = true; return false; } @@ -209,8 +205,6 @@ This proposal will be deployed with the Overwinter network upgrade. Backward compatibility ====================== -This proposal intentionally creates what is known as a hard fork where Overwinter nodes disconnect from pre-Overwinter nodes. - Prior to the network upgrade activating, Overwinter and pre-Overwinter nodes are compatible and can connect to each other. However, Overwinter nodes will have a preference for connecting to other Overwinter nodes, so pre-Overwinter nodes will gradually be disconnected in the run up to activation. Once the network upgrades, even though pre-Overwinter nodes can still accept the numerically larger protocol version used by Overwinter as being valid, Overwinter nodes will always disconnect peers using lower protocol versions. From a4d346f9a92b0c9e571cf9aac2279442c29d96a1 Mon Sep 17 00:00:00 2001 From: str4d Date: Wed, 28 Feb 2018 22:31:29 +0000 Subject: [PATCH 7/8] Address remaining comments, add Overwinter upgrade details --- .../bitcartel-nu0-network-handshake/draft.rst | 38 +++++++++++-------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/drafts/bitcartel-nu0-network-handshake/draft.rst b/drafts/bitcartel-nu0-network-handshake/draft.rst index 9b1ed6e3..e54983dc 100644 --- a/drafts/bitcartel-nu0-network-handshake/draft.rst +++ b/drafts/bitcartel-nu0-network-handshake/draft.rst @@ -12,22 +12,12 @@ Terminology The key words "MUST", "MUST NOT", "SHOULD", and "MAY" in this document are to be interpreted as described in RFC 2119. [#RFC2119]_ +The terms "branch" and "network upgrade" in this document are to be interpreted as described in ZIP 200. [#zip-0200]_ + The terms below are to be interpreted as follows: -Branch - A chain of blocks with common consensus rules, where the first block in the chain is either the genesis - block, or the child of a parent block created under an older set of consensus rules (i.e. the parent block - is a member of a different branch). By definition, every block belongs to at most one branch. - -Hard fork - The creation of a new branch by a change in the consensus rules of the network. Nodes that do not recognize - the new rules will continue to follow the old branch. - -Network upgrade - An intentional hard fork undertaken by the community in order to improve the network. - Overwinter - Code-name for the first ZCash network upgrade, also known as Network Upgrade Zero. + Code-name for the first Zcash network upgrade, also known as Network Upgrade Zero. Abstract @@ -196,10 +186,26 @@ Example code:: -Deployment -========== +Deployment of Overwinter +======================== -This proposal will be deployed with the Overwinter network upgrade. +The Overwinter network upgrade defines the following network upgrade constants [#zip-0200]_: + +BRANCH_ID + ``0x5ba81b19`` + +ACTIVATION_HEIGHT + Testnet: 207500 + + Mainnet: Not yet defined. + +The following ZIPs are deployed by Overwinter: + +- ZIP 200 [#zip-0200]_ +- ZIP 201 (this ZIP) +- ZIP 202 [#zip-0202]_ +- ZIP 203 [#zip-0203]_ +- ZIP 143 [#zip-0143]_ Backward compatibility From 23253e750482eeba80f808f1ba77197432be9afa Mon Sep 17 00:00:00 2001 From: str4d Date: Wed, 28 Feb 2018 22:32:25 +0000 Subject: [PATCH 8/8] Move ZIP 201 out of drafts --- drafts/bitcartel-nu0-network-handshake/draft.rst => zip-0201.rst | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename drafts/bitcartel-nu0-network-handshake/draft.rst => zip-0201.rst (100%) diff --git a/drafts/bitcartel-nu0-network-handshake/draft.rst b/zip-0201.rst similarity index 100% rename from drafts/bitcartel-nu0-network-handshake/draft.rst rename to zip-0201.rst