Merge branch 'master' into raft-issue-795-fix

.travis.yml

@@ -4,7 +4,7 @@
 
 language: go
 go_import_path: github.com/ethereum/go-ethereum
-go: 1.11.12
+go: 1.11.x
 sudo: true
 branches:
   only:

cmd/geth/usage.go

@@ -272,14 +272,15 @@ var AppHelpFlagGroups = []flagGroup{
 		},
 	},
 	{
-		Name: "MISC",
-	},{
 		Name: "ISTANBUL",
 		Flags: []cli.Flag{
 			utils.IstanbulRequestTimeoutFlag,
 			utils.IstanbulBlockPeriodFlag,
 		},
 	},
+	{
+		Name: "MISC",
+	},
 }
 
 // byCategory sorts an array of flagGroup by Name in the order

docs/Consensus/raft.md

@@ -158,27 +158,4 @@ To add a node to the cluster, attach to a JS console and issue `raft.addPeer(eno
 
 ## FAQ
 
-**Could you have a single- or two-node cluster? More generally, could you have an even number of nodes ?**
+Answers to frequently asked questions can be found on the main [Quorum FAQ page](../FAQ.md).
-
-A cluster can tolerate failures that leave a quorum (majority) available. So a cluster of two nodes can't tolerate any failures, three nodes can tolerate one, and five nodes can tolerate two. Typically Raft clusters have an odd number of nodes, since an even number provides no failure tolerance benefit.
-
-**What happens if you don't assume minter and leader are the same node?**
-
-There's no hard reason they couldn't be different. We just co-locate the minter and leader as an optimization.
-
-* It saves one network call communicating the block to the leader.
-* It provides a simple way to choose a minter. If we didn't use the Raft leader we'd have to build in "minter election" at a higher level.
-
-Additionally there could even be multiple minters running at the same time, but this would produce contention for which blocks actually extend the chain, reducing the productivity of the cluster (see "races" above).
-
-**I thought there were no forks in a Raft-based blockchain. What's the deal with "speculative minting"?**
-
-"Speculative chains" are not forks in the blockchain. They represent a series ("chain") of blocks that have been sent through Raft, after which each of the blocks may or may not actually end up being included in *the blockchain*.
-
-**Can transactions be reversed? Since raft log entries can be disregarded as "no-ops", does this imply transaction reversal?**
-
-No. When a Raft log entry containing a new block is disregarded as a "no-op", its transactions will remain in the transaction pool, and so they will be included in a future block in the chain.
-
-**What's the deal with the block timestamp being stored in nanoseconds (instead of seconds, like other consensus mechanisms)?**
-
-With raft-based consensus we can produce far more than one block per second, which vanilla Ethereum implicitly disallows (as the default timestamp resolution is in seconds and every block must have a timestamp greater than its parent). For Raft, we store the timestamp in nanoseconds and ensure it is incremented by at least 1 nanosecond per block.

docs/FAQ.md

@@ -1,3 +1,5 @@
+### Quorum FAQ
+
 ??? question "I've run into an issue with Quorum, where do I get support?"
     The [Quorum Slack channels](https://clh7rniov2.execute-api.us-east-1.amazonaws.com/Express/) are the best place to query the community and get immediate help.
  
@@ -38,9 +40,6 @@
 ??? question "Can I create a network of Quorum nodes using different consensus mechanisms?"
     Unfortunately, that is not possible. Quorum nodes configured with raft will only be able to work correctly with other nodes running raft consensus. This applies to all other supported consensus algorithms.
 
-??? info "Known Raft consensus node misconfiguration"
-    Please see https://github.com/jpmorganchase/quorum/issues/410
-
 ??? info "Quorum version compatibility table"
     |                                     | Adding new node v2.0.x | Adding new node v2.1.x | Adding new node v2.2.x |
     | ----------------------------------- | ---------------------- | ---------------------- | ---------------------- |
@@ -48,4 +47,34 @@
     | Existing chain consisting of v2.1.x | <span style="color:red;">block sync</span>  | <span style="color:green;">block sync<br /> public txn<br /> private txn</span> | <span style="color:green;">block sync<br /> public txn<br /> private txn</span> |
     | Existing chain consisting of v2.2.x | <span style="color:red;">block sync</span>  | <span style="color:green;">block sync<br /> public txn<br /> private txn</span> | <span style="color:green;">block sync<br /> public txn<br /> private txn</span> |
 
-    **Note:** While every Quorum v2 client will be able to connect to any other v2 client, the usefullness will be severely degraded. <span style="color:red;">Red color</span> signifies that while connectivity is possible, <span style="color:red;">red colored</span> versions will be unable to send public or private txns to the rest of the net due to the EIP155 changes in the signer implemented in newer versions.
+    **Note:** While every Quorum v2 client will be able to connect to any other v2 client, the usefullness will be severely degraded. <span style="color:red;">Red color</span> signifies that while connectivity is possible, <span style="color:red;">red colored</span> versions will be unable to send public or private txns to the rest of the net due to the EIP155 changes in the signer implemented in newer versions.
+
+### Raft FAQ
+
+??? question "Could you have a single- or two-node cluster? More generally, could you have an even number of nodes?"
+    A cluster can tolerate failures that leave a quorum (majority) available. So a cluster of two nodes can't tolerate any failures, three nodes can tolerate one, and five nodes can tolerate two. Typically Raft clusters have an odd number of nodes, since an even number provides no failure tolerance benefit.
+
+??? question "What happens if you don't assume minter and leader are the same node?"
+    There's no hard reason they couldn't be different. We just co-locate the minter and leader as an optimization.
+    
+    * It saves one network call communicating the block to the leader.
+    * It provides a simple way to choose a minter. If we didn't use the Raft leader we'd have to build in "minter election" at a higher level.
+
+    Additionally there could even be multiple minters running at the same time, but this would produce contention for which blocks actually extend the chain, reducing the productivity of the cluster (see "races" above).
+
+??? question "I thought there were no forks in a Raft-based blockchain. What's the deal with "speculative minting"?"
+    "Speculative chains" are not forks in the blockchain. They represent a series ("chain") of blocks that have been sent through Raft, after which each of the blocks may or may not actually end up being included in *the blockchain*.
+
+??? question "Can transactions be reversed? Since raft log entries can be disregarded as "no-ops", does this imply transaction reversal?"
+    No. When a Raft log entry containing a new block is disregarded as a "no-op", its transactions will remain in the transaction pool, and so they will be included in a future block in the chain.
+
+??? question "What's the deal with the block timestamp being stored in nanoseconds (instead of seconds, like other consensus mechanisms)?"
+    With raft-based consensus we can produce far more than one block per second, which vanilla Ethereum implicitly disallows (as the default timestamp resolution is in seconds and every block must have a timestamp greater than its parent). For Raft, we store the timestamp in nanoseconds and ensure it is incremented by at least 1 nanosecond per block.
+
+??? question "Why do I see "Error: Number can only safely store up to 53 bits" when using web3js with Raft?"
+    As mentioned above, Raft stores the timestamp in nanoseconds, so it is too large to be held as a number in javascript.
+    You need to modify your code to take account of this. An example can be seen [here](https://github.com/jpmorganchase/quorum.js/blob/master/lib/index.js#L35).
+    A future quorum release will address this issue.
+
+??? info "Known Raft consensus node misconfiguration"
+    Please see https://github.com/jpmorganchase/quorum/issues/410

docs/Getting Started/7Nodes-Setup.md

@@ -50,7 +50,7 @@ issues with the version of curl bundled with Vagrant.
 
 * If the machine you are using has less than 8 GB memory you will likely encounter system issues such as slow down and unresponsiveness when starting the Vagrant instance as your machine will not have the capacity to run the VM.  There are several steps that can be taken to overcome this:
     1. Shutdown any running processes that are not required
-    1. If running the [7nodes example](../7Nodes), reduce the number of nodes started up.  See the [7nodes: Reducing the number of nodes](../7Nodes#reducing-the-number-of-nodes) for info on how to do this.
+    1. If running the [7nodes example](../7Nodes), reduce the number of nodes started up.  See the [7nodes: Reducing the number of nodes](../7Nodes-Setup#reducing-the-number-of-nodes) for info on how to do this.
     1. Set up and run the examples locally.  Running locally reduces the load on your memory compared to running in Vagrant.
 
 ### Running with Docker

docs/Getting Started/Creating-A-Network-From-Scratch.md

@@ -5,7 +5,7 @@ This section details easy to follow step by step instructions of how to setup on
 Let's go through step by step instructions to setup a Quorum node with Raft consensus.
 
 ## Quorum with Raft consensus
-1. On each machine build Quorum as described in the [getting set up](../Setup%20Overview%20%26%20Quickstart) section. Ensure that PATH contains geth and bootnode
+1. On each machine build Quorum as described in the [Installing](../Installing) section. Ensure that PATH contains geth and bootnode
     ```
     $ git clone https://github.com/jpmorganchase/quorum.git
     $ cd quorum
@@ -296,7 +296,7 @@ Let's go through step by step instructions to setup a Quorum node with Raft cons
 
 
 ## Quorum with Istanbul BFT consensus
-1. On each machine build Quorum as described in the [getting set up](../Setup%20Overview%20%26%20Quickstart) section. Ensure that PATH contains geth and boot node
+1. On each machine build Quorum as described in the [Installing](../Installing) section. Ensure that PATH contains geth and boot node
     ```
     $ git clone https://github.com/jpmorganchase/quorum.git
     $ cd quorum
@@ -902,7 +902,7 @@ Just execute **step 4** instruction from removing a validator node.
 
 ## Adding privacy transaction manager
 ### Tessera
-1. Build Quorum and install [Tessera](https://github.com/jpmorganchase/tessera/releases) as described in the [getting set up](../Setup%20Overview%20%26%20Quickstart) section. Ensure that PATH contains geth and bootnode. Be aware of the location of the `tessera.jar` release file
+1. Build Quorum and install [Tessera](https://github.com/jpmorganchase/tessera/releases) as described in the [Installing](../Installing) section. Ensure that PATH contains geth and bootnode. Be aware of the location of the `tessera.jar` release file
     ```
     $ git clone https://github.com/jpmorganchase/quorum.git
     $ cd quorum
@@ -1156,7 +1156,7 @@ Just execute **step 4** instruction from removing a validator node.
     ```    
 
 ### Constellation
-1. Build Quorum and install [Constellation](https://github.com/jpmorganchase/constellation/releases) as described in the [getting set up](../Setup%20Overview%20%26%20Quickstart) section. Ensure that PATH contains geth, bootnode, and constellation-node binaries
+1. Build Quorum and install [Constellation](https://github.com/jpmorganchase/constellation/releases) as described in the [Installing](../Installing) section. Ensure that PATH contains geth, bootnode, and constellation-node binaries
 2. Generate new keys with `constellation-node --generatekeys=new-node-1`
 3. Start your constellation node and send it into background with `constellation-node --url=https://127.0.0.1:9001/ --port=9001 --workdir=. --socket=tm.ipc --publickeys=new-node-1.pub --privatekeys=new-node-1.key --othernodes=https://127.0.0.1:9001/ >> constellation.log 2>&1 &`
 4. Start your node and send it into background with `PRIVATE_CONFIG=tm.ipc nohup geth --datadir new-node-1 --nodiscover --verbosity 5 --networkid 31337 --raft --raftport 50000 --rpc --rpcaddr 0.0.0.0 --rpcport 22000 --rpcapi admin,db,eth,debug,miner,net,shh,txpool,personal,web3,quorum,raft --emitcheckpoints --port 21000 2>>node.log &`

docs/Getting Started/api.md

@@ -62,17 +62,17 @@ web3.eth.sendRawPrivateTransaction(signedTransactionData [, privateData] [, call
 
 Sends a pre-signed transaction. For example can be signed using: https://github.com/SilentCicero/ethereumjs-accounts
 
-__Important:__ Please note that before calling this API, a `storeraw` api need to be called first to Quorum's private transaction manager. Instructions on how to do this can be found [here](https://github.com/jpmorganchase/tessera/wiki/Interface-&-API).
+__Important:__ Please note that before calling this API, a `storeraw` api need to be called first to Quorum's private transaction manager. Instructions on how to do this can be found [here](../../Privacy/Tessera/Usage/Interface%20&%20API/).
 
 ##### Parameters
  1. `String` - Signed transaction data in HEX format
  2. `Object` - Private data to send
     - `privateFor`: `List<String>`  - When sending a private transaction, an array of the recipients' base64-encoded public keys.
-3. `Function` - (optional) If you pass a callback the HTTP request is made asynchronous. See [this note](#using-callbacks) for details.
+3. `Function` - (optional) If you pass a callback the HTTP request is made asynchronous.
 
 ##### Returns
  `String` - The 32 Bytes transaction hash as HEX string.
- If the transaction was a contract creation use [web3.eth.getTransactionReceipt()](#web3ethgettransactionreceipt) to get the contract address, after the transaction was mined.
+ If the transaction was a contract creation use `web3.eth.getTransactionReceipt()` to get the contract address, after the transaction was mined.
  
  
 ##### Example

docs/Getting Started/running.md

@@ -171,7 +171,7 @@ Any additions to the `permissioned-nodes.json` file will be dynamically picked u
 Removing existing connected nodes from the `permissioned-nodes.json` file will not immediately drop those existing connected nodes. However, if the connection is dropped for any reason, and a subsequent connect request is made from the dropped node ids, it will be rejected as part of that new request.
 
 ## Quorum API
-Please see the [Quorum API](../../api) page for details.
+Please see the [Quorum API](../api) page for details.
 
 ## Network and Chain ID
 

p2p/discv5/node_test.go

@@ -152,7 +152,7 @@ func TestParseNode(t *testing.T) {
 			if err == nil {
 				t.Errorf("test %q:\n  got nil error, expected %#q", test.rawurl, test.wantError)
 				continue
-			} else if err.Error() != test.wantError {
+			} else if !strings.Contains(err.Error(), test.wantError) {
 				t.Errorf("test %q:\n  got error %#q, expected %#q", test.rawurl, err.Error(), test.wantError)
 				continue
 			}

p2p/enode/urlv4_test.go

@@ -139,7 +139,7 @@ func TestParseNode(t *testing.T) {
 			if err == nil {
 				t.Errorf("test %q:\n  got nil error, expected %#q", test.rawurl, test.wantError)
 				continue
-			} else if err.Error() != test.wantError {
+			} else if !strings.Contains(err.Error(), test.wantError) {
 				t.Errorf("test %q:\n  got error %#q, expected %#q", test.rawurl, err.Error(), test.wantError)
 				continue
 			}