certusone
/
kb
mirror of https://github.com/certusone/kb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Add peers and hsm pages

This commit is contained in:
Hendrik Hofstadt 2018-10-01 23:49:57 +02:00
parent b957011fc5
commit c7b0dd8142
49 changed files with 11832 additions and 134 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -1 +1,2 @@
build
source/_build

3
.vscode/settings.json vendored Normal file
View File

@ -0,0 +1,3 @@
{
"restructuredtext.confPath": "/home/hendrikhofstadt/Dev/certus/kb/source"
}

BIN
build/doctrees/environment.pickle
View File

Binary file not shown.

BIN
build/doctrees/index.doctree
View File

Binary file not shown.

BIN
build/doctrees/monitoring.doctree
View File

Binary file not shown.

2
build/html/.buildinfo
View File

@ -1,4 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 1a608ec6c7bf6070e1763b5bb5182e89
config: c3808b16821feb9f804249a399b0b358
tags: 645f666f9bcd5a90fca523b33c5a78b7

BIN
build/html/.doctrees/environment.pickle
View File

Binary file not shown.

BIN
build/html/.doctrees/index.doctree
View File

Binary file not shown.

BIN
build/html/.doctrees/monitoring.doctree
View File

Binary file not shown.

2
build/html/_sources/index.rst.txt
View File

@ -37,3 +37,5 @@ Contents
:caption: Contents:
monitoring
peers
hsm

24
build/html/_static/basic.css
View File

@ -82,9 +82,21 @@ div.sphinxsidebar input {
}
div.sphinxsidebar #searchbox input[type="text"] {
width: 170px;
float: left;
width: 80%;
padding: 0.25em;
box-sizing: border-box;
}
div.sphinxsidebar #searchbox input[type="submit"] {
float: left;
width: 20%;
border-left: none;
padding: 0.25em;
box-sizing: border-box;
}
img {
border: 0;
max-width: 100%;
@ -199,6 +211,11 @@ table.modindextable td {
/* -- general body styles --------------------------------------------------- */
div.body {
min-width: 450px;
max-width: 800px;
}
div.body p, div.body dd, div.body li, div.body blockquote {
-moz-hyphens: auto;
-ms-hyphens: auto;
@ -332,6 +349,11 @@ table.docutils {
border-collapse: collapse;
}
table.align-center {
margin-left: auto;
margin-right: auto;
}
table caption span.caption-number {
font-style: italic;
}

4
build/html/_static/doctools.js
View File

@ -70,7 +70,9 @@ jQuery.fn.highlightText = function(text, className) {
if (node.nodeType === 3) {
var val = node.nodeValue;
var pos = val.toLowerCase().indexOf(text);
if (pos >= 0 && !jQuery(node.parentNode).hasClass(className)) {
if (pos >= 0 &&
!jQuery(node.parentNode).hasClass(className) &&
!jQuery(node.parentNode).hasClass("nohighlight")) {
var span;
var isInSVG = jQuery(node).closest("body, svg, foreignObject").is("svg");
if (isInSVG) {

8
build/html/_static/jquery.js vendored
View File

File diff suppressed because one or more lines are too long

2
build/html/_static/websupport.js
View File

@ -301,7 +301,7 @@
li.hide();
// Determine where in the parents children list to insert this comment.
for(i=0; i < siblings.length; i++) {
for(var i=0; i < siblings.length; i++) {
if (comp(comment, siblings[i]) <= 0) {
$('#cd' + siblings[i].id)
.parent()

23
build/html/genindex.html
View File

@ -5,20 +5,12 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Index &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
@ -30,8 +22,7 @@
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
</head><body>
<div class="document">
@ -60,12 +51,14 @@
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@ -76,7 +69,7 @@
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.6</a>
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
</div>

35
build/html/index.html
View File

@ -4,20 +4,12 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Validator Operations Guide &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
@ -30,8 +22,7 @@
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
</head><body>
<div class="document">
@ -65,6 +56,18 @@ provide reference implementations which implement our guidelines.</p>
<li class="toctree-l2"><a class="reference internal" href="monitoring.html#symptoms-based-alerting">Symptoms-based alerting</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="peers.html">The Tendermint P2P Layer - Improving operations</a><ul>
<li class="toctree-l2"><a class="reference internal" href="peers.html#intro">Intro</a></li>
<li class="toctree-l2"><a class="reference internal" href="peers.html#the-peer-reactor">The peer reactor</a></li>
<li class="toctree-l2"><a class="reference internal" href="peers.html#operation-notes">Operation Notes</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="hsm.html">HSM for Signing</a><ul>
<li class="toctree-l2"><a class="reference internal" href="hsm.html#why-use-a-hsm">Why use a HSM</a></li>
<li class="toctree-l2"><a class="reference internal" href="hsm.html#hsm-implementations">HSM implementations</a></li>
<li class="toctree-l2"><a class="reference internal" href="hsm.html#hsm-hardware">HSM hardware</a></li>
</ul>
</li>
</ul>
</div>
</div>
@ -85,12 +88,14 @@ provide reference implementations which implement our guidelines.</p>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@ -101,7 +106,7 @@ provide reference implementations which implement our guidelines.</p>
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.6</a>
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
|

25
build/html/monitoring.html
View File

@ -4,25 +4,18 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Monitoring &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="The Tendermint P2P Layer - Improving operations" href="peers.html" />
<link rel="prev" title="Validator Operations Guide" href="index.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
@ -30,8 +23,7 @@
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
</head><body>
<div class="document">
@ -130,17 +122,20 @@ important, since it ensures that alerts are acted upon - it’s basically a to-d
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="index.html" title="previous chapter">Validator Operations Guide</a></li>
<li>Next: <a href="peers.html" title="next chapter">The Tendermint P2P Layer - Improving operations</a></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@ -151,7 +146,7 @@ important, since it ensures that alerts are acted upon - it’s basically a to-d
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.6</a>
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
|

BIN
build/html/objects.inv
View File

Binary file not shown.

17
build/html/search.html
View File

@ -4,20 +4,12 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Search &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
@ -37,8 +29,7 @@
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
</head><body>
<div class="document">
@ -89,7 +80,7 @@
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.6</a>
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
</div>

2
build/html/searchindex.js
View File

File diff suppressed because one or more lines are too long

2
docs/.buildinfo
View File

@ -1,4 +1,4 @@
# Sphinx build info version 1
# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
config: 1a608ec6c7bf6070e1763b5bb5182e89
config: c3808b16821feb9f804249a399b0b358
tags: 645f666f9bcd5a90fca523b33c5a78b7

BIN
docs/.doctrees/environment.pickle
View File

Binary file not shown.

BIN
docs/.doctrees/hsm.doctree Normal file
View File

Binary file not shown.

BIN
docs/.doctrees/index.doctree
View File

Binary file not shown.

BIN
docs/.doctrees/monitoring.doctree
View File

Binary file not shown.

BIN
docs/.doctrees/peers.doctree Normal file
View File

Binary file not shown.

2
docs/_images/KMS.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 7.6 KiB

2
docs/_images/kms.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 8.2 KiB

227
docs/_sources/hsm.rst.txt Normal file
View File

@ -0,0 +1,227 @@
HSM for Signing
===============
To ensure high security of your validator you will want to sign your votes and block proposals using a HSM_.
Why use a HSM_
##############
Intro into Tendermint
~~~~~~~~~~~~~~~~~~~~~
In the Tendermint consensus all validators in the active validator set participate by submitting block proposals and voting on them (using prevotes and precommits).
The vote of your validator is authenticated using a cryptographical signature so that other people know that the vote came from you and no one can mimic you which
would make the consensus system useless.
This coordinated consensus, which is split into several steps, ensures that at least 2/3+ of the validators have the same view_ of the network
since it requires 2/3+ votes for a block in order to finalize it.
With this restriction in place we would assume that it is impossible for two different chains (*forks*) to exist a time
since it is not possible to get 2 times 2/3+ votes on two conflicting_ blocks in a 3/3 validator set.
----
However this excludes the scenario of double voting/proposing.
In this scenario the byzantine proposer in the consensus round creates two conflicting_ blocks and sends them out to the network.
If we assume that we also have other byzantine actors in the validator set which want to profit from both chains these will also vote for both blocks.
That means that honest nodes in the network could see 2 different blocks at the same height with different contents and hashes.
From this point on the network has **forked**.
Outside viewers of the network will not know which block is correct and from now on there will not be a single truth.
This is the exact scenario we want to prevent with PBFT-like consensus systems.
How Cosmos prevents forks
~~~~~~~~~~~~~~~~~~~~~~~~~
Now that we know the 2 reasons that cause forks:
- Conflicting proposals by the same validator
- Conflicting votes by the same validator
To reduce the causes to a single one we can break it down to **double signing**. Because in order to create
2 conflicting proposals/votes a validator nees to sign 2 consensus messages for the same HRS_ pair with its key.
Tendermint allows other validators to record evidence of byzantine behaviour in blocks. Cosmos makes use of this features and slashes validators
that double sign. At the moment the slashing rate is set to 20% of all bonded Atoms at the moment which is very substantial.
This strict slashing condition makes it extremely important for validators to avoid double signing in order to ensure network security and prevent being slashed.
Problems of the default signing implementation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Lets assume a scenrio in which your validator host is compromised by a malicious actor who wants to financially
hurt you and your *customers* (people staking tokens with you).
If you are using the default FilePV which stores the private key associated with your validator on the file system
the attacker can simply steal the `priv_validator.json` file and from then on have the ability to sign any consensus
message with your validator's key therebye imposing you.
That way the attacker could **double sign** on your behalf triggering the slashing conditions for you forever since at
the moment the validator keys can not be exchanged which would essentially mean that your validator is *ruined*
The solution
~~~~~~~~~~~~
A HSM_ is a separate hardware component that stores keys and if not configured to perform differently will not
allow you to extract the private keys it stores. However it can take data from you and sign/encrypt it.
That way you can store you validator private key on a HSM and not keep it on the filesystem of the validator host.
Using a special software component Tendermint will ask the HSM to sign the consensus message without ever handling
the private key itself.
That way in the before-mentioned scenario the attacker would not be able to extract your private key and he could only
make you double sign as long as he controls your host.
This can be further mitigated by having an encrypted session between Tendermint and the HSM and doing proper secrets management.
With such measures in place it would be harder for a validator to get the HSM to sign data and you would have more time to detect and
mitigate the attack.
HSM implementations
###################
KMS by Tendermint
~~~~~~~~~~~~~~~~~
KMS (Key Management System) is a reference implementation of pluggable signing modules in a separated software component curated by Tendermint.
.. image:: kms.svg
It uses a separate software component to take care of the signing and implement double signing protection (like keeping track of the last signed messages).
This component communicates with the Tendermint node using a encrypted channel to prevent MITM_ attacks.
Signers are implemented as plugins which makes is very extendible and flexible. Examples of implemented signers include the YubiHSM2, Ledger Nano S and the
traditional file signer mentioned above.
The advantage of running a separate host (or an SGX enclave) for key management is that in case of a validator host compromise your KMS host will remain secure and
the built-in double signing protection in the KMS will prevent it from responding to double signing requests from the compromised validator host.
-----
At the time of writing the KMS service is actively being developed and not yet ready to be used.
You can watch the progress and contribute here: `KMS Github`_
Aiakos by Certus One
~~~~~~~~~~~~~~~~~~~~
While KMS connects to Tendermint via a socket Aiakos is directly integrated in the node at compile time.
Aiakos uses the PrivValidator interface of Tendermint to implement a direct wrapper to the YubiHSM2.
In order to use Aiakos you have to install the Yubico YubiHSM2 connector on your host and patch your Cosmos node source.
Your Cosmos node will then try to connect with the YubiHSM and optionally import a key specified by you to the HSM.
All consensus messages will then be signed using the HSM.
This implementation does not deliver the full security improvements that a separate KMS host brings.
How to setup a Cosmos validator with Aiakos YubiHSM2 support
------------------------------------------------------------
1. **Clone** cosmos sdk version and **checkout** the version you want to use.
2. Open the file `server/start.go`
3. Insert the code in the ``startInProcess`` function before "// create & start tendermint node"
::
if os.Getenv("AIAKOS_URL") == "" {
return nil, errors.New("no Aiakos hsm url specified. Please set AIAKOS_URL in the format host:port")
}
aiakosUrl := os.Getenv("AIAKOS_URL")
if os.Getenv("AIAKOS_SIGNING_KEY") == "" {
return nil, errors.New("no Aiakos signing key ID specified. Please set AIAKOS_SIGNING_KEY")
}
aiakosSigningKey, err := strconv.ParseUint(os.Getenv("AIAKOS_SIGNING_KEY"), 10, 16)
if err != nil {
return nil, errors.New("invalid Aiakos signing key ID.")
}
if os.Getenv("AIAKOS_AUTH_KEY") == "" {
return nil, errors.New("no Aiakos auth key ID specified. Please set AIAKOS_AUTH_KEY")
}
aiakosAuthKey, err := strconv.ParseUint(os.Getenv("AIAKOS_AUTH_KEY"), 10, 16)
if err != nil {
return nil, errors.New("invalid Aiakos auth key ID.")
}
if os.Getenv("AIAKOS_AUTH_KEY_PASSWORD") == "" {
return nil, errors.New("no Aiakos auth key password specified. Please set AIAKOS_AUTH_KEY_PASSWORD")
}
aiakosAuthPassword := os.Getenv("AIAKOS_AUTH_KEY_PASSWORD")
// Init Aiakos module
hsm, err := aiakos.NewAiakosPV(aiakosUrl, uint16(aiakosSigningKey), uint16(aiakosAuthKey), aiakosAuthPassword, ctx.Logger.With("module", "aiakos"))
if err != nil {
return nil, err
}
// Start Aiakos
err = hsm.Start()
if err != nil {
return nil, err
}
if os.Getenv("AIAKOS_IMPORT_KEY") == "TRUE" {
ctx.Logger.Info("importing private key to Aiakos because AIAKOS_IMPORT_KEY is set.")
filepv := privval.LoadOrGenFilePV(cfg.PrivValidatorFile())
key := filepv.PrivKey.(ed25519.PrivKeyEd25519)
err = hsm.ImportKey(uint16(aiakosSigningKey), key[:32])
if err != nil {
ctx.Logger.Error("Could not import key to HSM; skipping this step since it probably already exists", "error", err)
}
}
4. Add import for "github.com/certusone/aiakos" to the file's import section.
5. Run `dep ensure -v`
6. Build cosmos as described in the *README*
7. Install the YubiHSM connector_ on the host machine
8. Run the YubiHSM connector (as a service if you wish)
9. Update AuthKeys and generate a Eddsa signing-key on the HSM (optional)
Now you can run your Cosmos node with HSM support.
You need to set the following environment variables when running your node:
**AIAKOS_URL**
The URL of the YubiHSM connector. Usually localhost:12345
**AIAKOS_AUTH_KEY**
The ID of the Auth Key. Default 1
**AIAKOS_AUTH_KEY_PASSWORD**
The password of the Auth Key. Default "password"
**AIAKOS_SIGNING_KEY**
The ID of the signing key. The one you generated before or a free slot.
**AIAKOS_IMPORT_KEY**
Do you want to import your priv_validator.json to the HSM. "TRUE" if yes
--------------
Aiakos' source code can be found here: `Aiakos Github`_
--------------
HSM hardware
############
YubiHSM2
~~~~~~~~
The YubiHSM2 by Yubico is the most commonly used HSM_ among Cosmos validators.
It is quite affordable and offers the needed Eddsa standard which is not covered by many other HSMs.
The HSM runs from a USB port. We recommend you to use an internal USB port for physical security reasons.
.. [#HSM] Hardware Security Module
.. [#view] state of the blockchain, transactions and application
.. [#conflicting] containing different transactions, e.g. double-spending
.. [#HRS] pair of (block-) height, (consensus-) round, (consensus-) step
.. [#byzantine] malicious
.. [#MITM] man-in-the-middle
.. _`KMS Github`: https://github.com/tendermint/kms
.. _connector: https://www.yubico.com/products/services-software/download/yubihsm-2-libraries-and-tools/
.. _`Aiakos Github`: https://github.com/certusone/aiakos

2
docs/_sources/index.rst.txt
View File

@ -37,3 +37,5 @@ Contents
:caption: Contents:
monitoring
peers
hsm

203
docs/_sources/peers.rst.txt Normal file
View File

@ -0,0 +1,203 @@
The Tendermint P2P Layer - Improving operations
===============================================
Intro
-----
The Tendermint P2P implementation is based on a relatively simple
concept.
Types of peers
~~~~~~~~~~~~~~
Each node in the network is configured to dial a set of ``seed`` and
``persistentPeers`` when it is first started. Both of these parameters
can be set in the config.
**Persistent peers**
The Tendermint node will try to maintain a permanent
connection with this peer during its runtime. That also means that it
will persistently try to redial the node if the connection fails. That
is for example useful for the connection between Validator and Sentry
nodes because they will immediately try to reconnect after a connection
failure and there is no scenario where they could land in a
unjustifiably long backoff or generally be removed from the peers
addressbook which could cause unforseen issues in a Sentry-architecture.
**Seeds**
Seed nodes are only there to provide an up-to-date list of
peers of the network. If a node is configured to run as a seed node it
will actively search the network for new peers and store them in the
addressbook. However it will not maintain active connections with the
peers it queries. Connectinos from a seed node are meant to be
short-lived in order to just query the other peers addressbook, learn
about its new peers and then disconnect again. If you specifiy a
seednode in the config of your node it will try to dial it on start to
get an updated addressbook and get a list of peers on the network to
bootstrap its connections.
Addressbook
~~~~~~~~~~~
From the moment the node has acquired a list of peers on the network it
will store them in a weighted *addressbook*.
This addressbook stores all peers the client has ever learned about (and
possibly connected to). When a connection to a peer fails however, this
is marked in the addressbook and will lead to a backoff in a possible
attempt to reconnect. If a peer connection fails for more than x times
(where x is a constant hardcoded in Tendermint at the moment) the peer
is marked as bad and removed from the addressbook.
Connection Types
~~~~~~~~~~~~~~~~
**Inbound connection**
Every connection that was initiated by another peer
which contacted our node from the outside is called an inbound
connection. The number of maximum inbound connections can be specified
with ``max_num_inbound_peers``. In order for another peer to create a
connection to our node our P2P port (26656 by default) has to be
publicly exposed.
**Outbound connection**
Every connection that was initiated by our peer
(because of persistent peers, manual dialing or the PEX rector) is an
outbound connection. In order to establish an outbound connection the
P2P port does not have to be opened as long as outbound connections are
allowed by firewall rules.
The peer reactor
----------------
Depending on whether you have a normal or seed node the PEX (peer
exchange) reactor will execute the following loop regularly.
Normal peer
~~~~~~~~~~~
Startup
^^^^^^^
The node will check its addressbook for valid peers to connect to and
connect to all of the persistent peers specified. If the addresbook is
empty it will try to connect to on of the specified nodes.
Loop
^^^^
In the peer exchange routine the node will try to connect to new nodes
from its addressbook until it has reached the ``max_num_outbound_peers``
(as of tendermint #6fad8eaf5a7d82000c3f2933ec61e0f3917d07cf).
It will also query a random peer for its addressbook if the addressbook
of itself is not yet “full” (currently 1000 entries).
Seed node
~~~~~~~~~
Startup
^^^^^^^
*as with a normal node*
Loop
^^^^
The node will try to cleanup its connections by closing every connection
that has been checked to be healthy.
Then it will try to connect to all its known peers from the addressbook
and ask them for their addressbook.
This behaviour intends to get a picture of the network that contains
almost every public node available in order to allow new nodes to easily
bootstrap using an up-to-date addressbook.
Operation Notes
---------------
There are several possibilities of improving your operations that result
from what you have learned above.
Running outbound only nodes
~~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to reduce your DDoS surface you might want to run outbound only
nodes.
The advantage of Outbound only nodes is that you only allow connections
that are originating from your node on the Loadbalancing/Firewall layer
(see Network Topology). That way your node is not publicly reachable
from the internet except from the TCP sessions that you have established
with your peers. This only allows DoS attacks over these TCP sessions
which almost elimitates the whole surface as it is almost impossible to
perform a L3 attack with such limitations in place. Depending on your
way of selecting peers it can also reduce the surface for L7
(application layer) attacks.
Additional safety measures could be to announce a wrong IP using PEX to
irritate all nodes except those you are connecting to. That way only the
peers you have established connections with will know your true IP.
However this also increases the importance of having uncompromised peers
because other peers of potentially good actors on the network won’t be
able to connect to you and if your maximum number of outbound peers is
filled with compromised peers you will only see these nodes and no
others as we have learned above. Such a compromise may allow an attacker
to alter your image of the network rendring you unable to catch up for
example.
So it’s very important to (either):
- Set a high number of outgoing peers
- Add at least some trusted persistent peers
- Implement additional measures to either select peers or rotate peers on a regular basis
**Warning**
If your firewall is misconfigured or you are announcing a
wrong public IP (e.g. your internal Docker IP) your node will be
*outbound-only* unintentionally since no other nodes can connect from
the outside (assuming you are not configured as persistent peer using
your true IP). This can result in slow syncing and missed blocks due to
delays in consensus message gossip if you don’t apply the above
mentioned optimizations.
**Notice**
Outbound-only peers are meant as an additional measure to
protect your validator from DDoS and similar attacks. However running
only outbound peers can cause network partitioning, slow bootstrapping
for new network participants and general network destabilization. Plase
make sure that you run only a small portion of your sentries in an
outbound-only configuration to ensure the overall quality of the
network.
Running “full-duplex” nodes
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Full-Duplex or inbound/outbound nodes are the default configuration for
nodes. They allow both inbound connections to be established from the
outside as well as outbound connections to be made.
In order to run a full-duplex node your firewall needs to be opened for
both in- and outbound traffic on the relevant port (26656 by default).
Since the host can be reached from the public internet the risk for DDoS
is higher. However this configuration allows new peers to establish
connections with them and thereby benefits the overall network.
Please keep in mind to set you maximum inbound peer number in the config
appropriately to get a better view of the network.
Private nodes
~~~~~~~~~~~~~
Private nodes run on a VPN and allow only selected peers to establish
connections with them. Such a configuration could be used for
validator-validator private peerings.
In order to not leak any information about the node, it can be run with
PEX disabled and the peering with the other nodes hardcoded as
*persistent peer*.

24
docs/_static/basic.css vendored
View File

@ -82,9 +82,21 @@ div.sphinxsidebar input {
}
div.sphinxsidebar #searchbox input[type="text"] {
width: 170px;
float: left;
width: 80%;
padding: 0.25em;
box-sizing: border-box;
}
div.sphinxsidebar #searchbox input[type="submit"] {
float: left;
width: 20%;
border-left: none;
padding: 0.25em;
box-sizing: border-box;
}
img {
border: 0;
max-width: 100%;
@ -199,6 +211,11 @@ table.modindextable td {
/* -- general body styles --------------------------------------------------- */
div.body {
min-width: 450px;
max-width: 800px;
}
div.body p, div.body dd, div.body li, div.body blockquote {
-moz-hyphens: auto;
-ms-hyphens: auto;
@ -332,6 +349,11 @@ table.docutils {
border-collapse: collapse;
}
table.align-center {
margin-left: auto;
margin-right: auto;
}
table caption span.caption-number {
font-style: italic;
}

4
docs/_static/doctools.js vendored
View File

@ -70,7 +70,9 @@ jQuery.fn.highlightText = function(text, className) {
if (node.nodeType === 3) {
var val = node.nodeValue;
var pos = val.toLowerCase().indexOf(text);
if (pos >= 0 && !jQuery(node.parentNode).hasClass(className)) {
if (pos >= 0 &&
!jQuery(node.parentNode).hasClass(className) &&
!jQuery(node.parentNode).hasClass("nohighlight")) {
var span;
var isInSVG = jQuery(node).closest("body, svg, foreignObject").is("svg");
if (isInSVG) {

9
docs/_static/documentation_options.js vendored Normal file
View File

@ -0,0 +1,9 @@
var DOCUMENTATION_OPTIONS = {
URL_ROOT: document.getElementById("documentation_options").getAttribute('data-url_root'),
VERSION: '',
LANGUAGE: 'None',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};

10253
docs/_static/jquery-3.2.1.js vendored Normal file
View File

File diff suppressed because it is too large Load Diff

8
docs/_static/jquery.js vendored
View File

File diff suppressed because one or more lines are too long

2
docs/_static/websupport.js vendored
View File

@ -301,7 +301,7 @@
li.hide();
// Determine where in the parents children list to insert this comment.
for(i=0; i < siblings.length; i++) {
for(var i=0; i < siblings.length; i++) {
if (comp(comment, siblings[i]) <= 0) {
$('#cd' + siblings[i].id)
.parent()

23
docs/genindex.html
View File

@ -5,20 +5,12 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Index &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
@ -30,8 +22,7 @@
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
</head><body>
<div class="document">
@ -60,12 +51,14 @@
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@ -76,7 +69,7 @@
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.6</a>
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
</div>

286
docs/hsm.html Normal file
View File

@ -0,0 +1,286 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>HSM for Signing &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="prev" title="The Tendermint P2P Layer - Improving operations" href="peers.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="hsm-for-signing">
<h1>HSM for Signing<a class="headerlink" href="#hsm-for-signing" title="Permalink to this headline"></a></h1>
<p>To ensure high security of your validator you will want to sign your votes and block proposals using a <a class="reference internal" href="#hsm">HSM</a>.</p>
<div class="section" id="why-use-a-hsm">
<h2>Why use a <a class="reference internal" href="#hsm">HSM</a><a class="headerlink" href="#why-use-a-hsm" title="Permalink to this headline"></a></h2>
<div class="section" id="intro-into-tendermint">
<h3>Intro into Tendermint<a class="headerlink" href="#intro-into-tendermint" title="Permalink to this headline"></a></h3>
<p>In the Tendermint consensus all validators in the active validator set participate by submitting block proposals and voting on them (using prevotes and precommits).</p>
<p>The vote of your validator is authenticated using a cryptographical signature so that other people know that the vote came from you and no one can mimic you which
would make the consensus system useless.</p>
<p>This coordinated consensus, which is split into several steps, ensures that at least 2/3+ of the validators have the same <a class="reference internal" href="#view">view</a> of the network
since it requires 2/3+ votes for a block in order to finalize it.</p>
<p>With this restriction in place we would assume that it is impossible for two different chains (<em>forks</em>) to exist a time
since it is not possible to get 2 times 2/3+ votes on two <a class="reference internal" href="#conflicting">conflicting</a> blocks in a 3/3 validator set.</p>
<hr class="docutils" />
<p>However this excludes the scenario of double voting/proposing.</p>
<p>In this scenario the byzantine proposer in the consensus round creates two <a class="reference internal" href="#conflicting">conflicting</a> blocks and sends them out to the network.</p>
<p>If we assume that we also have other byzantine actors in the validator set which want to profit from both chains these will also vote for both blocks.</p>
<p>That means that honest nodes in the network could see 2 different blocks at the same height with different contents and hashes.
From this point on the network has <strong>forked</strong>.</p>
<p>Outside viewers of the network will not know which block is correct and from now on there will not be a single truth.
This is the exact scenario we want to prevent with PBFT-like consensus systems.</p>
</div>
<div class="section" id="how-cosmos-prevents-forks">
<h3>How Cosmos prevents forks<a class="headerlink" href="#how-cosmos-prevents-forks" title="Permalink to this headline"></a></h3>
<p>Now that we know the 2 reasons that cause forks:</p>
<ul class="simple">
<li>Conflicting proposals by the same validator</li>
<li>Conflicting votes by the same validator</li>
</ul>
<p>To reduce the causes to a single one we can break it down to <strong>double signing</strong>. Because in order to create
2 conflicting proposals/votes a validator nees to sign 2 consensus messages for the same <a class="reference internal" href="#hrs">HRS</a> pair with its key.</p>
<p>Tendermint allows other validators to record evidence of byzantine behaviour in blocks. Cosmos makes use of this features and slashes validators
that double sign. At the moment the slashing rate is set to 20% of all bonded Atoms at the moment which is very substantial.</p>
<p>This strict slashing condition makes it extremely important for validators to avoid double signing in order to ensure network security and prevent being slashed.</p>
</div>
<div class="section" id="problems-of-the-default-signing-implementation">
<h3>Problems of the default signing implementation<a class="headerlink" href="#problems-of-the-default-signing-implementation" title="Permalink to this headline"></a></h3>
<p>Lets assume a scenrio in which your validator host is compromised by a malicious actor who wants to financially
hurt you and your <em>customers</em> (people staking tokens with you).</p>
<p>If you are using the default FilePV which stores the private key associated with your validator on the file system
the attacker can simply steal the <cite>priv_validator.json</cite> file and from then on have the ability to sign any consensus
message with your validator’s key therebye imposing you.</p>
<p>That way the attacker could <strong>double sign</strong> on your behalf triggering the slashing conditions for you forever since at
the moment the validator keys can not be exchanged which would essentially mean that your validator is <em>ruined</em></p>
</div>
<div class="section" id="the-solution">
<h3>The solution<a class="headerlink" href="#the-solution" title="Permalink to this headline"></a></h3>
<p>A <a class="reference internal" href="#hsm">HSM</a> is a separate hardware component that stores keys and if not configured to perform differently will not
allow you to extract the private keys it stores. However it can take data from you and sign/encrypt it.</p>
<p>That way you can store you validator private key on a HSM and not keep it on the filesystem of the validator host.</p>
<p>Using a special software component Tendermint will ask the HSM to sign the consensus message without ever handling
the private key itself.</p>
<p>That way in the before-mentioned scenario the attacker would not be able to extract your private key and he could only
make you double sign as long as he controls your host.</p>
<p>This can be further mitigated by having an encrypted session between Tendermint and the HSM and doing proper secrets management.
With such measures in place it would be harder for a validator to get the HSM to sign data and you would have more time to detect and
mitigate the attack.</p>
</div>
</div>
<div class="section" id="hsm-implementations">
<h2>HSM implementations<a class="headerlink" href="#hsm-implementations" title="Permalink to this headline"></a></h2>
<div class="section" id="kms-by-tendermint">
<h3>KMS by Tendermint<a class="headerlink" href="#kms-by-tendermint" title="Permalink to this headline"></a></h3>
<p>KMS (Key Management System) is a reference implementation of pluggable signing modules in a separated software component curated by Tendermint.</p>
<img alt="_images/kms.svg" src="_images/kms.svg" /><p>It uses a separate software component to take care of the signing and implement double signing protection (like keeping track of the last signed messages).</p>
<p>This component communicates with the Tendermint node using a encrypted channel to prevent <a class="reference internal" href="#mitm">MITM</a> attacks.</p>
<p>Signers are implemented as plugins which makes is very extendible and flexible. Examples of implemented signers include the YubiHSM2, Ledger Nano S and the
traditional file signer mentioned above.</p>
<p>The advantage of running a separate host (or an SGX enclave) for key management is that in case of a validator host compromise your KMS host will remain secure and
the built-in double signing protection in the KMS will prevent it from responding to double signing requests from the compromised validator host.</p>
<hr class="docutils" />
<p>At the time of writing the KMS service is actively being developed and not yet ready to be used.</p>
<p>You can watch the progress and contribute here: <a class="reference external" href="https://github.com/tendermint/kms">KMS Github</a></p>
</div>
<div class="section" id="aiakos-by-certus-one">
<h3>Aiakos by Certus One<a class="headerlink" href="#aiakos-by-certus-one" title="Permalink to this headline"></a></h3>
<p>While KMS connects to Tendermint via a socket Aiakos is directly integrated in the node at compile time.</p>
<p>Aiakos uses the PrivValidator interface of Tendermint to implement a direct wrapper to the YubiHSM2.</p>
<p>In order to use Aiakos you have to install the Yubico YubiHSM2 connector on your host and patch your Cosmos node source.</p>
<p>Your Cosmos node will then try to connect with the YubiHSM and optionally import a key specified by you to the HSM.
All consensus messages will then be signed using the HSM.</p>
<p>This implementation does not deliver the full security improvements that a separate KMS host brings.</p>
<div class="section" id="how-to-setup-a-cosmos-validator-with-aiakos-yubihsm2-support">
<h4>How to setup a Cosmos validator with Aiakos YubiHSM2 support<a class="headerlink" href="#how-to-setup-a-cosmos-validator-with-aiakos-yubihsm2-support" title="Permalink to this headline"></a></h4>
<ol class="arabic simple">
<li><strong>Clone</strong> cosmos sdk version and <strong>checkout</strong> the version you want to use.</li>
<li>Open the file <cite>server/start.go</cite></li>
<li>Insert the code in the <code class="docutils literal notranslate"><span class="pre">startInProcess</span></code> function before “// create &amp; start tendermint node”</li>
</ol>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">Getenv</span><span class="p">(</span><span class="s2">&quot;AIAKOS_URL&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="s2">&quot;&quot;</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">nil</span><span class="p">,</span> <span class="n">errors</span><span class="o">.</span><span class="n">New</span><span class="p">(</span><span class="s2">&quot;no Aiakos hsm url specified. Please set AIAKOS_URL in the format host:port&quot;</span><span class="p">)</span>
<span class="p">}</span>
<span class="n">aiakosUrl</span> <span class="p">:</span><span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">Getenv</span><span class="p">(</span><span class="s2">&quot;AIAKOS_URL&quot;</span><span class="p">)</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">Getenv</span><span class="p">(</span><span class="s2">&quot;AIAKOS_SIGNING_KEY&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="s2">&quot;&quot;</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">nil</span><span class="p">,</span> <span class="n">errors</span><span class="o">.</span><span class="n">New</span><span class="p">(</span><span class="s2">&quot;no Aiakos signing key ID specified. Please set AIAKOS_SIGNING_KEY&quot;</span><span class="p">)</span>
<span class="p">}</span>
<span class="n">aiakosSigningKey</span><span class="p">,</span> <span class="n">err</span> <span class="p">:</span><span class="o">=</span> <span class="n">strconv</span><span class="o">.</span><span class="n">ParseUint</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">Getenv</span><span class="p">(</span><span class="s2">&quot;AIAKOS_SIGNING_KEY&quot;</span><span class="p">),</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">16</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="n">nil</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">nil</span><span class="p">,</span> <span class="n">errors</span><span class="o">.</span><span class="n">New</span><span class="p">(</span><span class="s2">&quot;invalid Aiakos signing key ID.&quot;</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">Getenv</span><span class="p">(</span><span class="s2">&quot;AIAKOS_AUTH_KEY&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="s2">&quot;&quot;</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">nil</span><span class="p">,</span> <span class="n">errors</span><span class="o">.</span><span class="n">New</span><span class="p">(</span><span class="s2">&quot;no Aiakos auth key ID specified. Please set AIAKOS_AUTH_KEY&quot;</span><span class="p">)</span>
<span class="p">}</span>
<span class="n">aiakosAuthKey</span><span class="p">,</span> <span class="n">err</span> <span class="p">:</span><span class="o">=</span> <span class="n">strconv</span><span class="o">.</span><span class="n">ParseUint</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">Getenv</span><span class="p">(</span><span class="s2">&quot;AIAKOS_AUTH_KEY&quot;</span><span class="p">),</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">16</span><span class="p">)</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="n">nil</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">nil</span><span class="p">,</span> <span class="n">errors</span><span class="o">.</span><span class="n">New</span><span class="p">(</span><span class="s2">&quot;invalid Aiakos auth key ID.&quot;</span><span class="p">)</span>
<span class="p">}</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">Getenv</span><span class="p">(</span><span class="s2">&quot;AIAKOS_AUTH_KEY_PASSWORD&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="s2">&quot;&quot;</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">nil</span><span class="p">,</span> <span class="n">errors</span><span class="o">.</span><span class="n">New</span><span class="p">(</span><span class="s2">&quot;no Aiakos auth key password specified. Please set AIAKOS_AUTH_KEY_PASSWORD&quot;</span><span class="p">)</span>
<span class="p">}</span>
<span class="n">aiakosAuthPassword</span> <span class="p">:</span><span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">Getenv</span><span class="p">(</span><span class="s2">&quot;AIAKOS_AUTH_KEY_PASSWORD&quot;</span><span class="p">)</span>
<span class="o">//</span> <span class="n">Init</span> <span class="n">Aiakos</span> <span class="n">module</span>
<span class="n">hsm</span><span class="p">,</span> <span class="n">err</span> <span class="p">:</span><span class="o">=</span> <span class="n">aiakos</span><span class="o">.</span><span class="n">NewAiakosPV</span><span class="p">(</span><span class="n">aiakosUrl</span><span class="p">,</span> <span class="n">uint16</span><span class="p">(</span><span class="n">aiakosSigningKey</span><span class="p">),</span> <span class="n">uint16</span><span class="p">(</span><span class="n">aiakosAuthKey</span><span class="p">),</span> <span class="n">aiakosAuthPassword</span><span class="p">,</span> <span class="n">ctx</span><span class="o">.</span><span class="n">Logger</span><span class="o">.</span><span class="n">With</span><span class="p">(</span><span class="s2">&quot;module&quot;</span><span class="p">,</span> <span class="s2">&quot;aiakos&quot;</span><span class="p">))</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="n">nil</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">nil</span><span class="p">,</span> <span class="n">err</span>
<span class="p">}</span>
<span class="o">//</span> <span class="n">Start</span> <span class="n">Aiakos</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">hsm</span><span class="o">.</span><span class="n">Start</span><span class="p">()</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="n">nil</span> <span class="p">{</span>
<span class="k">return</span> <span class="n">nil</span><span class="p">,</span> <span class="n">err</span>
<span class="p">}</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">Getenv</span><span class="p">(</span><span class="s2">&quot;AIAKOS_IMPORT_KEY&quot;</span><span class="p">)</span> <span class="o">==</span> <span class="s2">&quot;TRUE&quot;</span> <span class="p">{</span>
<span class="n">ctx</span><span class="o">.</span><span class="n">Logger</span><span class="o">.</span><span class="n">Info</span><span class="p">(</span><span class="s2">&quot;importing private key to Aiakos because AIAKOS_IMPORT_KEY is set.&quot;</span><span class="p">)</span>
<span class="n">filepv</span> <span class="p">:</span><span class="o">=</span> <span class="n">privval</span><span class="o">.</span><span class="n">LoadOrGenFilePV</span><span class="p">(</span><span class="n">cfg</span><span class="o">.</span><span class="n">PrivValidatorFile</span><span class="p">())</span>
<span class="n">key</span> <span class="p">:</span><span class="o">=</span> <span class="n">filepv</span><span class="o">.</span><span class="n">PrivKey</span><span class="o">.</span><span class="p">(</span><span class="n">ed25519</span><span class="o">.</span><span class="n">PrivKeyEd25519</span><span class="p">)</span>
<span class="n">err</span> <span class="o">=</span> <span class="n">hsm</span><span class="o">.</span><span class="n">ImportKey</span><span class="p">(</span><span class="n">uint16</span><span class="p">(</span><span class="n">aiakosSigningKey</span><span class="p">),</span> <span class="n">key</span><span class="p">[:</span><span class="mi">32</span><span class="p">])</span>
<span class="k">if</span> <span class="n">err</span> <span class="o">!=</span> <span class="n">nil</span> <span class="p">{</span>
<span class="n">ctx</span><span class="o">.</span><span class="n">Logger</span><span class="o">.</span><span class="n">Error</span><span class="p">(</span><span class="s2">&quot;Could not import key to HSM; skipping this step since it probably already exists&quot;</span><span class="p">,</span> <span class="s2">&quot;error&quot;</span><span class="p">,</span> <span class="n">err</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<ol class="arabic simple" start="4">
<li>Add import for “github.com/certusone/aiakos” to the file’s import section.</li>
<li>Run <cite>dep ensure -v</cite></li>
<li>Build cosmos as described in the <em>README</em></li>
<li>Install the YubiHSM <a class="reference external" href="https://www.yubico.com/products/services-software/download/yubihsm-2-libraries-and-tools/">connector</a> on the host machine</li>
<li>Run the YubiHSM connector (as a service if you wish)</li>
<li>Update AuthKeys and generate a Eddsa signing-key on the HSM (optional)</li>
</ol>
<p>Now you can run your Cosmos node with HSM support.</p>
<p>You need to set the following environment variables when running your node:</p>
<dl class="docutils">
<dt><strong>AIAKOS_URL</strong></dt>
<dd>The URL of the YubiHSM connector. Usually localhost:12345</dd>
<dt><strong>AIAKOS_AUTH_KEY</strong></dt>
<dd>The ID of the Auth Key. Default 1</dd>
<dt><strong>AIAKOS_AUTH_KEY_PASSWORD</strong></dt>
<dd>The password of the Auth Key. Default “password”</dd>
<dt><strong>AIAKOS_SIGNING_KEY</strong></dt>
<dd>The ID of the signing key. The one you generated before or a free slot.</dd>
<dt><strong>AIAKOS_IMPORT_KEY</strong></dt>
<dd>Do you want to import your priv_validator.json to the HSM. “TRUE” if yes</dd>
</dl>
<hr class="docutils" />
<p>Aiakos’ source code can be found here: <a class="reference external" href="https://github.com/certusone/aiakos">Aiakos Github</a></p>
</div>
</div>
</div>
<hr class="docutils" />
<div class="section" id="hsm-hardware">
<h2>HSM hardware<a class="headerlink" href="#hsm-hardware" title="Permalink to this headline"></a></h2>
<div class="section" id="yubihsm2">
<h3>YubiHSM2<a class="headerlink" href="#yubihsm2" title="Permalink to this headline"></a></h3>
<p>The YubiHSM2 by Yubico is the most commonly used <a class="reference internal" href="#hsm">HSM</a> among Cosmos validators.</p>
<p>It is quite affordable and offers the needed Eddsa standard which is not covered by many other HSMs.</p>
<p>The HSM runs from a USB port. We recommend you to use an internal USB port for physical security reasons.</p>
<table class="docutils footnote" frame="void" id="hsm" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[1]</td><td>Hardware Security Module</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="view" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[2]</td><td>state of the blockchain, transactions and application</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="conflicting" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[3]</td><td>containing different transactions, e.g. double-spending</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="hrs" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[4]</td><td>pair of (block-) height, (consensus-) round, (consensus-) step</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="byzantine" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[5]</td><td>malicious</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="mitm" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label">[6]</td><td>man-in-the-middle</td></tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper"><div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="peers.html" title="previous chapter">The Tendermint P2P Layer - Improving operations</a></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
|
<a href="_sources/hsm.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>

35
docs/index.html
View File

@ -4,20 +4,12 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Validator Operations Guide &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
@ -30,8 +22,7 @@
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
</head><body>
<div class="document">
@ -65,6 +56,18 @@ provide reference implementations which implement our guidelines.</p>
<li class="toctree-l2"><a class="reference internal" href="monitoring.html#symptoms-based-alerting">Symptoms-based alerting</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="peers.html">The Tendermint P2P Layer - Improving operations</a><ul>
<li class="toctree-l2"><a class="reference internal" href="peers.html#intro">Intro</a></li>
<li class="toctree-l2"><a class="reference internal" href="peers.html#the-peer-reactor">The peer reactor</a></li>
<li class="toctree-l2"><a class="reference internal" href="peers.html#operation-notes">Operation Notes</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="hsm.html">HSM for Signing</a><ul>
<li class="toctree-l2"><a class="reference internal" href="hsm.html#why-use-a-hsm">Why use a HSM</a></li>
<li class="toctree-l2"><a class="reference internal" href="hsm.html#hsm-implementations">HSM implementations</a></li>
<li class="toctree-l2"><a class="reference internal" href="hsm.html#hsm-hardware">HSM hardware</a></li>
</ul>
</li>
</ul>
</div>
</div>
@ -85,12 +88,14 @@ provide reference implementations which implement our guidelines.</p>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@ -101,7 +106,7 @@ provide reference implementations which implement our guidelines.</p>
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.6</a>
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
|

25
docs/monitoring.html
View File

@ -4,25 +4,18 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Monitoring &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="The Tendermint P2P Layer - Improving operations" href="peers.html" />
<link rel="prev" title="Validator Operations Guide" href="index.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
@ -30,8 +23,7 @@
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
</head><body>
<div class="document">
@ -130,17 +122,20 @@ important, since it ensures that alerts are acted upon - it’s basically a to-d
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="index.html" title="previous chapter">Validator Operations Guide</a></li>
<li>Next: <a href="peers.html" title="next chapter">The Tendermint P2P Layer - Improving operations</a></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
@ -151,7 +146,7 @@ important, since it ensures that alerts are acted upon - it’s basically a to-d
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.6</a>
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
|

BIN
docs/objects.inv
View File

Binary file not shown.

258
docs/peers.html Normal file
View File

@ -0,0 +1,258 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>The Tendermint P2P Layer - Improving operations &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="HSM for Signing" href="hsm.html" />
<link rel="prev" title="Monitoring" href="monitoring.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head><body>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="the-tendermint-p2p-layer-improving-operations">
<h1>The Tendermint P2P Layer - Improving operations<a class="headerlink" href="#the-tendermint-p2p-layer-improving-operations" title="Permalink to this headline"></a></h1>
<div class="section" id="intro">
<h2>Intro<a class="headerlink" href="#intro" title="Permalink to this headline"></a></h2>
<p>The Tendermint P2P implementation is based on a relatively simple
concept.</p>
<div class="section" id="types-of-peers">
<h3>Types of peers<a class="headerlink" href="#types-of-peers" title="Permalink to this headline"></a></h3>
<p>Each node in the network is configured to dial a set of <code class="docutils literal notranslate"><span class="pre">seed</span></code> and
<code class="docutils literal notranslate"><span class="pre">persistentPeers</span></code> when it is first started. Both of these parameters
can be set in the config.</p>
<dl class="docutils">
<dt><strong>Persistent peers</strong></dt>
<dd>The Tendermint node will try to maintain a permanent
connection with this peer during its runtime. That also means that it
will persistently try to redial the node if the connection fails. That
is for example useful for the connection between Validator and Sentry
nodes because they will immediately try to reconnect after a connection
failure and there is no scenario where they could land in a
unjustifiably long backoff or generally be removed from the peers
addressbook which could cause unforseen issues in a Sentry-architecture.</dd>
<dt><strong>Seeds</strong></dt>
<dd>Seed nodes are only there to provide an up-to-date list of
peers of the network. If a node is configured to run as a seed node it
will actively search the network for new peers and store them in the
addressbook. However it will not maintain active connections with the
peers it queries. Connectinos from a seed node are meant to be
short-lived in order to just query the other peers addressbook, learn
about its new peers and then disconnect again. If you specifiy a
seednode in the config of your node it will try to dial it on start to
get an updated addressbook and get a list of peers on the network to
bootstrap its connections.</dd>
</dl>
</div>
<div class="section" id="addressbook">
<h3>Addressbook<a class="headerlink" href="#addressbook" title="Permalink to this headline"></a></h3>
<p>From the moment the node has acquired a list of peers on the network it
will store them in a weighted <em>addressbook</em>.</p>
<p>This addressbook stores all peers the client has ever learned about (and
possibly connected to). When a connection to a peer fails however, this
is marked in the addressbook and will lead to a backoff in a possible
attempt to reconnect. If a peer connection fails for more than x times
(where x is a constant hardcoded in Tendermint at the moment) the peer
is marked as bad and removed from the addressbook.</p>
</div>
<div class="section" id="connection-types">
<h3>Connection Types<a class="headerlink" href="#connection-types" title="Permalink to this headline"></a></h3>
<dl class="docutils">
<dt><strong>Inbound connection</strong></dt>
<dd>Every connection that was initiated by another peer
which contacted our node from the outside is called an inbound
connection. The number of maximum inbound connections can be specified
with <code class="docutils literal notranslate"><span class="pre">max_num_inbound_peers</span></code>. In order for another peer to create a
connection to our node our P2P port (26656 by default) has to be
publicly exposed.</dd>
<dt><strong>Outbound connection</strong></dt>
<dd>Every connection that was initiated by our peer
(because of persistent peers, manual dialing or the PEX rector) is an
outbound connection. In order to establish an outbound connection the
P2P port does not have to be opened as long as outbound connections are
allowed by firewall rules.</dd>
</dl>
</div>
</div>
<div class="section" id="the-peer-reactor">
<h2>The peer reactor<a class="headerlink" href="#the-peer-reactor" title="Permalink to this headline"></a></h2>
<p>Depending on whether you have a normal or seed node the PEX (peer
exchange) reactor will execute the following loop regularly.</p>
<div class="section" id="normal-peer">
<h3>Normal peer<a class="headerlink" href="#normal-peer" title="Permalink to this headline"></a></h3>
<div class="section" id="startup">
<h4>Startup<a class="headerlink" href="#startup" title="Permalink to this headline"></a></h4>
<p>The node will check its addressbook for valid peers to connect to and
connect to all of the persistent peers specified. If the addresbook is
empty it will try to connect to on of the specified nodes.</p>
</div>
<div class="section" id="loop">
<h4>Loop<a class="headerlink" href="#loop" title="Permalink to this headline"></a></h4>
<p>In the peer exchange routine the node will try to connect to new nodes
from its addressbook until it has reached the <code class="docutils literal notranslate"><span class="pre">max_num_outbound_peers</span></code>
(as of tendermint #6fad8eaf5a7d82000c3f2933ec61e0f3917d07cf).</p>
<p>It will also query a random peer for its addressbook if the addressbook
of itself is not yet “full” (currently 1000 entries).</p>
</div>
</div>
<div class="section" id="seed-node">
<h3>Seed node<a class="headerlink" href="#seed-node" title="Permalink to this headline"></a></h3>
<div class="section" id="id1">
<h4>Startup<a class="headerlink" href="#id1" title="Permalink to this headline"></a></h4>
<p><em>as with a normal node</em></p>
</div>
<div class="section" id="id2">
<h4>Loop<a class="headerlink" href="#id2" title="Permalink to this headline"></a></h4>
<p>The node will try to cleanup its connections by closing every connection
that has been checked to be healthy.</p>
<p>Then it will try to connect to all its known peers from the addressbook
and ask them for their addressbook.</p>
<p>This behaviour intends to get a picture of the network that contains
almost every public node available in order to allow new nodes to easily
bootstrap using an up-to-date addressbook.</p>
</div>
</div>
</div>
<div class="section" id="operation-notes">
<h2>Operation Notes<a class="headerlink" href="#operation-notes" title="Permalink to this headline"></a></h2>
<p>There are several possibilities of improving your operations that result
from what you have learned above.</p>
<div class="section" id="running-outbound-only-nodes">
<h3>Running outbound only nodes<a class="headerlink" href="#running-outbound-only-nodes" title="Permalink to this headline"></a></h3>
<p>In order to reduce your DDoS surface you might want to run outbound only
nodes.</p>
<p>The advantage of Outbound only nodes is that you only allow connections
that are originating from your node on the Loadbalancing/Firewall layer
(see Network Topology). That way your node is not publicly reachable
from the internet except from the TCP sessions that you have established
with your peers. This only allows DoS attacks over these TCP sessions
which almost elimitates the whole surface as it is almost impossible to
perform a L3 attack with such limitations in place. Depending on your
way of selecting peers it can also reduce the surface for L7
(application layer) attacks.</p>
<p>Additional safety measures could be to announce a wrong IP using PEX to
irritate all nodes except those you are connecting to. That way only the
peers you have established connections with will know your true IP.</p>
<p>However this also increases the importance of having uncompromised peers
because other peers of potentially good actors on the network won’t be
able to connect to you and if your maximum number of outbound peers is
filled with compromised peers you will only see these nodes and no
others as we have learned above. Such a compromise may allow an attacker
to alter your image of the network rendring you unable to catch up for
example.</p>
<p>So it’s very important to (either):</p>
<ul class="simple">
<li>Set a high number of outgoing peers</li>
<li>Add at least some trusted persistent peers</li>
<li>Implement additional measures to either select peers or rotate peers on a regular basis</li>
</ul>
<p><strong>Warning</strong></p>
<p>If your firewall is misconfigured or you are announcing a
wrong public IP (e.g. your internal Docker IP) your node will be
<em>outbound-only</em> unintentionally since no other nodes can connect from
the outside (assuming you are not configured as persistent peer using
your true IP). This can result in slow syncing and missed blocks due to
delays in consensus message gossip if you don’t apply the above
mentioned optimizations.</p>
<p><strong>Notice</strong></p>
<p>Outbound-only peers are meant as an additional measure to
protect your validator from DDoS and similar attacks. However running
only outbound peers can cause network partitioning, slow bootstrapping
for new network participants and general network destabilization. Plase
make sure that you run only a small portion of your sentries in an
outbound-only configuration to ensure the overall quality of the
network.</p>
</div>
<div class="section" id="running-full-duplex-nodes">
<h3>Running “full-duplex” nodes<a class="headerlink" href="#running-full-duplex-nodes" title="Permalink to this headline"></a></h3>
<p>Full-Duplex or inbound/outbound nodes are the default configuration for
nodes. They allow both inbound connections to be established from the
outside as well as outbound connections to be made.</p>
<p>In order to run a full-duplex node your firewall needs to be opened for
both in- and outbound traffic on the relevant port (26656 by default).</p>
<p>Since the host can be reached from the public internet the risk for DDoS
is higher. However this configuration allows new peers to establish
connections with them and thereby benefits the overall network.</p>
<p>Please keep in mind to set you maximum inbound peer number in the config
appropriately to get a better view of the network.</p>
</div>
<div class="section" id="private-nodes">
<h3>Private nodes<a class="headerlink" href="#private-nodes" title="Permalink to this headline"></a></h3>
<p>Private nodes run on a VPN and allow only selected peers to establish
connections with them. Such a configuration could be used for
validator-validator private peerings.</p>
<p>In order to not leak any information about the node, it can be run with
PEX disabled and the peering with the other nodes hardcoded as
<em>persistent peer</em>.</p>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper"><div class="relations">
<h3>Related Topics</h3>
<ul>
<li><a href="index.html">Documentation overview</a><ul>
<li>Previous: <a href="monitoring.html" title="previous chapter">Monitoring</a></li>
<li>Next: <a href="hsm.html" title="next chapter">HSM for Signing</a></li>
</ul></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer">
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
|
<a href="_sources/peers.rst.txt"
rel="nofollow">Page source</a>
</div>
</body>
</html>

17
docs/search.html
View File

@ -4,20 +4,12 @@
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Search &#8212; Certus One Knowledge Base documentation</title>
<link rel="stylesheet" href="_static/alabaster.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
@ -37,8 +29,7 @@
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
</head>
<body>
</head><body>
<div class="document">
@ -89,7 +80,7 @@
&copy;2018, Certus One GmbH.
|
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.6.6</a>
Powered by <a href="http://sphinx-doc.org/">Sphinx 1.7.5</a>
&amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.9</a>
</div>

2
docs/searchindex.js
View File

File diff suppressed because one or more lines are too long

227
source/hsm.rst Normal file
View File

@ -0,0 +1,227 @@
HSM for Signing
===============
To ensure high security of your validator you will want to sign your votes and block proposals using a HSM_.
Why use a HSM_
##############
Intro into Tendermint
~~~~~~~~~~~~~~~~~~~~~
In the Tendermint consensus all validators in the active validator set participate by submitting block proposals and voting on them (using prevotes and precommits).
The vote of your validator is authenticated using a cryptographical signature so that other people know that the vote came from you and no one can mimic you which
would make the consensus system useless.
This coordinated consensus, which is split into several steps, ensures that at least 2/3+ of the validators have the same view_ of the network
since it requires 2/3+ votes for a block in order to finalize it.
With this restriction in place we would assume that it is impossible for two different chains (*forks*) to exist a time
since it is not possible to get 2 times 2/3+ votes on two conflicting_ blocks in a 3/3 validator set.
----
However this excludes the scenario of double voting/proposing.
In this scenario the byzantine proposer in the consensus round creates two conflicting_ blocks and sends them out to the network.
If we assume that we also have other byzantine actors in the validator set which want to profit from both chains these will also vote for both blocks.
That means that honest nodes in the network could see 2 different blocks at the same height with different contents and hashes.
From this point on the network has **forked**.
Outside viewers of the network will not know which block is correct and from now on there will not be a single truth.
This is the exact scenario we want to prevent with PBFT-like consensus systems.
How Cosmos prevents forks
~~~~~~~~~~~~~~~~~~~~~~~~~
Now that we know the 2 reasons that cause forks:
- Conflicting proposals by the same validator
- Conflicting votes by the same validator
To reduce the causes to a single one we can break it down to **double signing**. Because in order to create
2 conflicting proposals/votes a validator nees to sign 2 consensus messages for the same HRS_ pair with its key.
Tendermint allows other validators to record evidence of byzantine behaviour in blocks. Cosmos makes use of this features and slashes validators
that double sign. At the moment the slashing rate is set to 20% of all bonded Atoms at the moment which is very substantial.
This strict slashing condition makes it extremely important for validators to avoid double signing in order to ensure network security and prevent being slashed.
Problems of the default signing implementation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Lets assume a scenrio in which your validator host is compromised by a malicious actor who wants to financially
hurt you and your *customers* (people staking tokens with you).
If you are using the default FilePV which stores the private key associated with your validator on the file system
the attacker can simply steal the `priv_validator.json` file and from then on have the ability to sign any consensus
message with your validator's key therebye imposing you.
That way the attacker could **double sign** on your behalf triggering the slashing conditions for you forever since at
the moment the validator keys can not be exchanged which would essentially mean that your validator is *ruined*
The solution
~~~~~~~~~~~~
A HSM_ is a separate hardware component that stores keys and if not configured to perform differently will not
allow you to extract the private keys it stores. However it can take data from you and sign/encrypt it.
That way you can store you validator private key on a HSM and not keep it on the filesystem of the validator host.
Using a special software component Tendermint will ask the HSM to sign the consensus message without ever handling
the private key itself.
That way in the before-mentioned scenario the attacker would not be able to extract your private key and he could only
make you double sign as long as he controls your host.
This can be further mitigated by having an encrypted session between Tendermint and the HSM and doing proper secrets management.
With such measures in place it would be harder for a validator to get the HSM to sign data and you would have more time to detect and
mitigate the attack.
HSM implementations
###################
KMS by Tendermint
~~~~~~~~~~~~~~~~~
KMS (Key Management System) is a reference implementation of pluggable signing modules in a separated software component curated by Tendermint.
.. image:: kms.svg
It uses a separate software component to take care of the signing and implement double signing protection (like keeping track of the last signed messages).
This component communicates with the Tendermint node using a encrypted channel to prevent MITM_ attacks.
Signers are implemented as plugins which makes is very extendible and flexible. Examples of implemented signers include the YubiHSM2, Ledger Nano S and the
traditional file signer mentioned above.
The advantage of running a separate host (or an SGX enclave) for key management is that in case of a validator host compromise your KMS host will remain secure and
the built-in double signing protection in the KMS will prevent it from responding to double signing requests from the compromised validator host.
-----
At the time of writing the KMS service is actively being developed and not yet ready to be used.
You can watch the progress and contribute here: `KMS Github`_
Aiakos by Certus One
~~~~~~~~~~~~~~~~~~~~
While KMS connects to Tendermint via a socket Aiakos is directly integrated in the node at compile time.
Aiakos uses the PrivValidator interface of Tendermint to implement a direct wrapper to the YubiHSM2.
In order to use Aiakos you have to install the Yubico YubiHSM2 connector on your host and patch your Cosmos node source.
Your Cosmos node will then try to connect with the YubiHSM and optionally import a key specified by you to the HSM.
All consensus messages will then be signed using the HSM.
This implementation does not deliver the full security improvements that a separate KMS host brings.
How to setup a Cosmos validator with Aiakos YubiHSM2 support
------------------------------------------------------------
1. **Clone** cosmos sdk version and **checkout** the version you want to use.
2. Open the file `server/start.go`
3. Insert the code in the ``startInProcess`` function before "// create & start tendermint node"
::
if os.Getenv("AIAKOS_URL") == "" {
return nil, errors.New("no Aiakos hsm url specified. Please set AIAKOS_URL in the format host:port")
}
aiakosUrl := os.Getenv("AIAKOS_URL")
if os.Getenv("AIAKOS_SIGNING_KEY") == "" {
return nil, errors.New("no Aiakos signing key ID specified. Please set AIAKOS_SIGNING_KEY")
}
aiakosSigningKey, err := strconv.ParseUint(os.Getenv("AIAKOS_SIGNING_KEY"), 10, 16)
if err != nil {
return nil, errors.New("invalid Aiakos signing key ID.")
}
if os.Getenv("AIAKOS_AUTH_KEY") == "" {
return nil, errors.New("no Aiakos auth key ID specified. Please set AIAKOS_AUTH_KEY")
}
aiakosAuthKey, err := strconv.ParseUint(os.Getenv("AIAKOS_AUTH_KEY"), 10, 16)
if err != nil {
return nil, errors.New("invalid Aiakos auth key ID.")
}
if os.Getenv("AIAKOS_AUTH_KEY_PASSWORD") == "" {
return nil, errors.New("no Aiakos auth key password specified. Please set AIAKOS_AUTH_KEY_PASSWORD")
}
aiakosAuthPassword := os.Getenv("AIAKOS_AUTH_KEY_PASSWORD")
// Init Aiakos module
hsm, err := aiakos.NewAiakosPV(aiakosUrl, uint16(aiakosSigningKey), uint16(aiakosAuthKey), aiakosAuthPassword, ctx.Logger.With("module", "aiakos"))
if err != nil {
return nil, err
}
// Start Aiakos
err = hsm.Start()
if err != nil {
return nil, err
}
if os.Getenv("AIAKOS_IMPORT_KEY") == "TRUE" {
ctx.Logger.Info("importing private key to Aiakos because AIAKOS_IMPORT_KEY is set.")
filepv := privval.LoadOrGenFilePV(cfg.PrivValidatorFile())
key := filepv.PrivKey.(ed25519.PrivKeyEd25519)
err = hsm.ImportKey(uint16(aiakosSigningKey), key[:32])
if err != nil {
ctx.Logger.Error("Could not import key to HSM; skipping this step since it probably already exists", "error", err)
}
}
4. Add import for "github.com/certusone/aiakos" to the file's import section.
5. Run `dep ensure -v`
6. Build cosmos as described in the *README*
7. Install the YubiHSM connector_ on the host machine
8. Run the YubiHSM connector (as a service if you wish)
9. Update AuthKeys and generate a Eddsa signing-key on the HSM (optional)
Now you can run your Cosmos node with HSM support.
You need to set the following environment variables when running your node:
**AIAKOS_URL**
The URL of the YubiHSM connector. Usually localhost:12345
**AIAKOS_AUTH_KEY**
The ID of the Auth Key. Default 1
**AIAKOS_AUTH_KEY_PASSWORD**
The password of the Auth Key. Default "password"
**AIAKOS_SIGNING_KEY**
The ID of the signing key. The one you generated before or a free slot.
**AIAKOS_IMPORT_KEY**
Do you want to import your priv_validator.json to the HSM. "TRUE" if yes
--------------
Aiakos' source code can be found here: `Aiakos Github`_
--------------
HSM hardware
############
YubiHSM2
~~~~~~~~
The YubiHSM2 by Yubico is the most commonly used HSM_ among Cosmos validators.
It is quite affordable and offers the needed Eddsa standard which is not covered by many other HSMs.
The HSM runs from a USB port. We recommend you to use an internal USB port for physical security reasons.
.. [#HSM] Hardware Security Module
.. [#view] state of the blockchain, transactions and application
.. [#conflicting] containing different transactions, e.g. double-spending
.. [#HRS] pair of (block-) height, (consensus-) round, (consensus-) step
.. [#byzantine] malicious
.. [#MITM] man-in-the-middle
.. _`KMS Github`: https://github.com/tendermint/kms
.. _connector: https://www.yubico.com/products/services-software/download/yubihsm-2-libraries-and-tools/
.. _`Aiakos Github`: https://github.com/certusone/aiakos

2
source/index.rst
View File

@ -37,3 +37,5 @@ Contents
:caption: Contents:
monitoring
peers
hsm

2
source/kms.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 8.2 KiB

203
source/peers.rst Normal file
View File

@ -0,0 +1,203 @@
The Tendermint P2P Layer - Improving operations
===============================================
Intro
-----
The Tendermint P2P implementation is based on a relatively simple
concept.
Types of peers
~~~~~~~~~~~~~~
Each node in the network is configured to dial a set of ``seed`` and
``persistentPeers`` when it is first started. Both of these parameters
can be set in the config.
**Persistent peers**
The Tendermint node will try to maintain a permanent
connection with this peer during its runtime. That also means that it
will persistently try to redial the node if the connection fails. That
is for example useful for the connection between Validator and Sentry
nodes because they will immediately try to reconnect after a connection
failure and there is no scenario where they could land in a
unjustifiably long backoff or generally be removed from the peers
addressbook which could cause unforseen issues in a Sentry-architecture.
**Seeds**
Seed nodes are only there to provide an up-to-date list of
peers of the network. If a node is configured to run as a seed node it
will actively search the network for new peers and store them in the
addressbook. However it will not maintain active connections with the
peers it queries. Connectinos from a seed node are meant to be
short-lived in order to just query the other peers addressbook, learn
about its new peers and then disconnect again. If you specifiy a
seednode in the config of your node it will try to dial it on start to
get an updated addressbook and get a list of peers on the network to
bootstrap its connections.
Addressbook
~~~~~~~~~~~
From the moment the node has acquired a list of peers on the network it
will store them in a weighted *addressbook*.
This addressbook stores all peers the client has ever learned about (and
possibly connected to). When a connection to a peer fails however, this
is marked in the addressbook and will lead to a backoff in a possible
attempt to reconnect. If a peer connection fails for more than x times
(where x is a constant hardcoded in Tendermint at the moment) the peer
is marked as bad and removed from the addressbook.
Connection Types
~~~~~~~~~~~~~~~~
**Inbound connection**
Every connection that was initiated by another peer
which contacted our node from the outside is called an inbound
connection. The number of maximum inbound connections can be specified
with ``max_num_inbound_peers``. In order for another peer to create a
connection to our node our P2P port (26656 by default) has to be
publicly exposed.
**Outbound connection**
Every connection that was initiated by our peer
(because of persistent peers, manual dialing or the PEX rector) is an
outbound connection. In order to establish an outbound connection the
P2P port does not have to be opened as long as outbound connections are
allowed by firewall rules.
The peer reactor
----------------
Depending on whether you have a normal or seed node the PEX (peer
exchange) reactor will execute the following loop regularly.
Normal peer
~~~~~~~~~~~
Startup
^^^^^^^
The node will check its addressbook for valid peers to connect to and
connect to all of the persistent peers specified. If the addresbook is
empty it will try to connect to on of the specified nodes.
Loop
^^^^
In the peer exchange routine the node will try to connect to new nodes
from its addressbook until it has reached the ``max_num_outbound_peers``
(as of tendermint #6fad8eaf5a7d82000c3f2933ec61e0f3917d07cf).
It will also query a random peer for its addressbook if the addressbook
of itself is not yet “full” (currently 1000 entries).
Seed node
~~~~~~~~~
Startup
^^^^^^^
*as with a normal node*
Loop
^^^^
The node will try to cleanup its connections by closing every connection
that has been checked to be healthy.
Then it will try to connect to all its known peers from the addressbook
and ask them for their addressbook.
This behaviour intends to get a picture of the network that contains
almost every public node available in order to allow new nodes to easily
bootstrap using an up-to-date addressbook.
Operation Notes
---------------
There are several possibilities of improving your operations that result
from what you have learned above.
Running outbound only nodes
~~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to reduce your DDoS surface you might want to run outbound only
nodes.
The advantage of Outbound only nodes is that you only allow connections
that are originating from your node on the Loadbalancing/Firewall layer
(see Network Topology). That way your node is not publicly reachable
from the internet except from the TCP sessions that you have established
with your peers. This only allows DoS attacks over these TCP sessions
which almost elimitates the whole surface as it is almost impossible to
perform a L3 attack with such limitations in place. Depending on your
way of selecting peers it can also reduce the surface for L7
(application layer) attacks.
Additional safety measures could be to announce a wrong IP using PEX to
irritate all nodes except those you are connecting to. That way only the
peers you have established connections with will know your true IP.
However this also increases the importance of having uncompromised peers
because other peers of potentially good actors on the network won’t be
able to connect to you and if your maximum number of outbound peers is
filled with compromised peers you will only see these nodes and no
others as we have learned above. Such a compromise may allow an attacker
to alter your image of the network rendring you unable to catch up for
example.
So it’s very important to (either):
- Set a high number of outgoing peers
- Add at least some trusted persistent peers
- Implement additional measures to either select peers or rotate peers on a regular basis
**Warning**
If your firewall is misconfigured or you are announcing a
wrong public IP (e.g. your internal Docker IP) your node will be
*outbound-only* unintentionally since no other nodes can connect from
the outside (assuming you are not configured as persistent peer using
your true IP). This can result in slow syncing and missed blocks due to
delays in consensus message gossip if you don’t apply the above
mentioned optimizations.
**Notice**
Outbound-only peers are meant as an additional measure to
protect your validator from DDoS and similar attacks. However running
only outbound peers can cause network partitioning, slow bootstrapping
for new network participants and general network destabilization. Plase
make sure that you run only a small portion of your sentries in an
outbound-only configuration to ensure the overall quality of the
network.
Running “full-duplex” nodes
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Full-Duplex or inbound/outbound nodes are the default configuration for
nodes. They allow both inbound connections to be established from the
outside as well as outbound connections to be made.
In order to run a full-duplex node your firewall needs to be opened for
both in- and outbound traffic on the relevant port (26656 by default).
Since the host can be reached from the public internet the risk for DDoS
is higher. However this configuration allows new peers to establish
connections with them and thereby benefits the overall network.
Please keep in mind to set you maximum inbound peer number in the config
appropriately to get a better view of the network.
Private nodes
~~~~~~~~~~~~~
Private nodes run on a VPN and allow only selected peers to establish
connections with them. Such a configuration could be used for
validator-validator private peerings.
In order to not leak any information about the node, it can be run with
PEX disabled and the peering with the other nodes hardcoded as
*persistent peer*.