From 27ba6e8a426bf12aeb8a210d79a8edc08c5ed0e0 Mon Sep 17 00:00:00 2001 From: Ethan Buchman Date: Sun, 23 Sep 2018 02:25:33 -0400 Subject: [PATCH 1/2] Minor docs cleanup (#2472) * docs: link consensus to blockchain spec. closes #2422 * docs: deprecate research section. closes #2401 * docs: fix some links * docs: fix some markdown lists * docs: fix more links --- docs/.vuepress/config.js | 8 ------- docs/app-dev/getting-started.md | 2 +- docs/research/determinism.md | 2 +- docs/research/transactional-semantics.md | 24 ++----------------- docs/spec/abci/apps.md | 9 +++---- docs/spec/consensus/consensus.md | 2 +- docs/tendermint-core/running-in-production.md | 14 +++++++---- docs/tendermint-core/using-tendermint.md | 6 +++++ 8 files changed, 25 insertions(+), 42 deletions(-) diff --git a/docs/.vuepress/config.js b/docs/.vuepress/config.js index 5b064bf8..892ea204 100644 --- a/docs/.vuepress/config.js +++ b/docs/.vuepress/config.js @@ -106,14 +106,6 @@ module.exports = { "/spec/abci/apps", "/spec/abci/client-server" ] - }, - { - title: "Research", - collapsable: false, - children: [ - "/research/determinism", - "/research/transactional-semantics" - ] } ] } diff --git a/docs/app-dev/getting-started.md b/docs/app-dev/getting-started.md index ae74d98a..066deaac 100644 --- a/docs/app-dev/getting-started.md +++ b/docs/app-dev/getting-started.md @@ -7,7 +7,7 @@ application you want to run. So, to run a complete blockchain that does something useful, you must start two programs: one is Tendermint Core, the other is your application, which can be written in any programming language. Recall from [the intro to -ABCI](../introduction/introduction.md#ABCI-Overview) that Tendermint Core handles all +ABCI](../introduction/introduction.html#abci-overview) that Tendermint Core handles all the p2p and consensus stuff, and just forwards transactions to the application when they need to be validated, or when they're ready to be committed to a block. diff --git a/docs/research/determinism.md b/docs/research/determinism.md index 2e23476c..b1c20566 100644 --- a/docs/research/determinism.md +++ b/docs/research/determinism.md @@ -1,3 +1,3 @@ # On Determinism -Arguably, the most difficult part of blockchain programming is determinism - that is, ensuring that sources of indeterminism do not creep into the design of such systems. +See [Determinism](../spec/abci/abci.md#determinism). diff --git a/docs/research/transactional-semantics.md b/docs/research/transactional-semantics.md index 99b5a00c..c4b54332 100644 --- a/docs/research/transactional-semantics.md +++ b/docs/research/transactional-semantics.md @@ -1,25 +1,5 @@ # Transactional Semantics -In [Using Tendermint](../tendermint-core/using-tendermint.md#broadcast-api) we -discussed different API endpoints for sending transactions and -differences between them. +See details of the [broadcast API](../tendermint-core/using-tendermint.md#broadcast-api) +and the [mempool WAL](../tendermint-core/running-in-production.md#mempool-wal). -What we have not yet covered is transactional semantics. - -When you send a transaction using one of the available methods, it first -goes to the mempool. Currently, it does not provide strong guarantees -like "if the transaction were accepted, it would be eventually included -in a block (given CheckTx passes)." - -For instance a tx could enter the mempool, but before it can be sent to -peers the node crashes. - -We are planning to provide such guarantees by using a WAL and replaying -transactions (See -[this issue](https://github.com/tendermint/tendermint/issues/248)), but -it's non-trivial to do this all efficiently. - -The temporary solution is for clients to monitor the node and resubmit -transaction(s) and/or send them to more nodes at once, so the -probability of all of them crashing at the same time and losing the msg -decreases substantially. diff --git a/docs/spec/abci/apps.md b/docs/spec/abci/apps.md index cd88c685..0e8057b5 100644 --- a/docs/spec/abci/apps.md +++ b/docs/spec/abci/apps.md @@ -114,8 +114,8 @@ should halt before it can use more resources than it requested. When `MaxGas > -1`, Tendermint enforces the following rules: - - `GasWanted <= MaxGas` for all txs in the mempool - - `(sum of GasWanted in a block) <= MaxGas` when proposing a block +- `GasWanted <= MaxGas` for all txs in the mempool +- `(sum of GasWanted in a block) <= MaxGas` when proposing a block If `MaxGas == -1`, no rules about gas are enforced. @@ -124,8 +124,9 @@ This means it does not guarantee that committed blocks satisfy these rules! It is the application's responsibility to return non-zero response codes when gas limits are exceeded. The `GasUsed` field is ignored compltely by Tendermint. That said, applications should enforce: - - `GasUsed <= GasWanted` for any given transaction - - `(sum of GasUsed in a block) <= MaxGas` for every block + +- `GasUsed <= GasWanted` for any given transaction +- `(sum of GasUsed in a block) <= MaxGas` for every block In the future, we intend to add a `Priority` field to the responses that can be used to explicitly prioritize txs in the mempool for inclusion in a block diff --git a/docs/spec/consensus/consensus.md b/docs/spec/consensus/consensus.md index a0136bff..b58184c7 100644 --- a/docs/spec/consensus/consensus.md +++ b/docs/spec/consensus/consensus.md @@ -17,7 +17,7 @@ vote](https://godoc.org/github.com/tendermint/tendermint/types#FirstPrecommit) for something. - A vote _at_ `(H,R)` is a vote signed with the bytes for `H` and `R` - included in its [sign-bytes](../blockchain/blockchain.md). + included in its [sign-bytes](../blockchain/blockchain.md#vote). - _+2/3_ is short for "more than 2/3" - _1/3+_ is short for "1/3 or more" - A set of +2/3 of prevotes for a particular block or `` at diff --git a/docs/tendermint-core/running-in-production.md b/docs/tendermint-core/running-in-production.md index cd55be53..bdb824d6 100644 --- a/docs/tendermint-core/running-in-production.md +++ b/docs/tendermint-core/running-in-production.md @@ -6,7 +6,7 @@ By default, Tendermint uses the `syndtr/goleveldb` package for it's in-process key-value database. Unfortunately, this implementation of LevelDB seems to suffer under heavy load (see [#226](https://github.com/syndtr/goleveldb/issues/226)). It may be best to install the real C-implementaiton of LevelDB and compile Tendermint to use -that using `make build_c`. See the [install instructions](../introduction/install) for details. +that using `make build_c`. See the [install instructions](../introduction/install.md) for details. Tendermint keeps multiple distinct LevelDB databases in the `$TMROOT/data`: @@ -20,11 +20,11 @@ Tendermint keeps multiple distinct LevelDB databases in the `$TMROOT/data`: - `tx_index.db`: Indexes txs (and their results) by tx hash and by DeliverTx result tags. By default, Tendermint will only index txs by their hash, not by their DeliverTx -result tags. See [indexing transactions](../app-dev/indexing-transactions) for +result tags. See [indexing transactions](../app-dev/indexing-transactions.md) for details. There is no current strategy for pruning the databases. Consider reducing -block production by [controlling empty blocks](../tendermint-core/using-tendermint#No-Empty-Blocks) +block production by [controlling empty blocks](../tendermint-core/using-tendermint.md#no-empty-blocks) or by increasing the `consensus.timeout_commit` param. Note both of these are local settings and not enforced by the consensus. @@ -50,7 +50,9 @@ logging level, you can do so by running tendermint with ## Write Ahead Logs (WAL) Tendermint uses write ahead logs for the consensus (`cs.wal`) and the mempool -(`mempool.wal`). Both WALs have a max size of 1GB and are automatically rotated.. +(`mempool.wal`). Both WALs have a max size of 1GB and are automatically rotated. + +### Consensus WAL The `consensus.wal` is used to ensure we can recover from a crash at any point in the consensus state machine. @@ -60,7 +62,9 @@ validator. Since Tendermint validators are expected to never sign a conflicting WAL ensures we can always recover deterministically to the latest state of the consensus without using the network or re-signing any consensus messages. -If your `consensus.wal` is corrupted, see [below](#WAL-Corruption). +If your `consensus.wal` is corrupted, see [below](#wal-corruption). + +### Mempool WAL The `mempool.wal` logs all incoming txs before running CheckTx, but is otherwise not used in any programmatic way. It's just a kind of manual diff --git a/docs/tendermint-core/using-tendermint.md b/docs/tendermint-core/using-tendermint.md index 33d981e4..5ee18361 100644 --- a/docs/tendermint-core/using-tendermint.md +++ b/docs/tendermint-core/using-tendermint.md @@ -305,6 +305,12 @@ can take on the order of a second. For a quick result, use `broadcast_tx_sync`, but the transaction will not be committed until later, and by that point its effect on the state may change. +Note the mempool does not provide strong guarantees - just because a tx passed +CheckTx (ie. was accepted into the mempool), doesn't mean it will be committed, +as nodes with the tx in their mempool may crash before they get to propose. +For more information, see the [mempool +write-ahead-log](../tendermint-core/running-in-production.md#mempool-wal) + ## Tendermint Networks When `tendermint init` is run, both a `genesis.json` and From d16f52eab365ae4326a532465cb676e011d66483 Mon Sep 17 00:00:00 2001 From: Anton Kaliaev Date: Sun, 23 Sep 2018 16:31:19 +0400 Subject: [PATCH 2/2] copy without sudo (#2474) --- docs/introduction/install.md | 4 ++-- docs/spec/abci/apps.md | 4 ++-- docs/tendermint-core/running-in-production.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/introduction/install.md b/docs/introduction/install.md index f7d78aba..f3498514 100644 --- a/docs/introduction/install.md +++ b/docs/introduction/install.md @@ -95,9 +95,9 @@ wget https://github.com/google/leveldb/archive/v1.20.tar.gz && \ tar -zxvf v1.20.tar.gz && \ cd leveldb-1.20/ && \ make && \ - sudo scp -r out-static/lib* out-shared/lib* /usr/local/lib/ && \ + cp -r out-static/lib* out-shared/lib* /usr/local/lib/ && \ cd include/ && \ - sudo scp -r leveldb /usr/local/include/ && \ + cp -r leveldb /usr/local/include/ && \ sudo ldconfig && \ rm -f v1.20.tar.gz ``` diff --git a/docs/spec/abci/apps.md b/docs/spec/abci/apps.md index 0e8057b5..a8f37771 100644 --- a/docs/spec/abci/apps.md +++ b/docs/spec/abci/apps.md @@ -95,7 +95,7 @@ The `Data` field must be strictly deterministic, but can be arbitrary data. ### Gas -Ethereum introduced the notion of `gas` as an absract representation of the +Ethereum introduced the notion of `gas` as an abstract representation of the cost of resources used by nodes when processing transactions. Every operation in the Ethereum Virtual Machine uses some amount of gas, and gas can be accepted at a market-variable price. Users propose a maximum amount of gas for their transaction; if the tx uses less, they get @@ -123,7 +123,7 @@ Note that Tendermint does not currently enforce anything about Gas in the consen This means it does not guarantee that committed blocks satisfy these rules! It is the application's responsibility to return non-zero response codes when gas limits are exceeded. -The `GasUsed` field is ignored compltely by Tendermint. That said, applications should enforce: +The `GasUsed` field is ignored completely by Tendermint. That said, applications should enforce: - `GasUsed <= GasWanted` for any given transaction - `(sum of GasUsed in a block) <= MaxGas` for every block diff --git a/docs/tendermint-core/running-in-production.md b/docs/tendermint-core/running-in-production.md index bdb824d6..c774cd13 100644 --- a/docs/tendermint-core/running-in-production.md +++ b/docs/tendermint-core/running-in-production.md @@ -5,7 +5,7 @@ By default, Tendermint uses the `syndtr/goleveldb` package for it's in-process key-value database. Unfortunately, this implementation of LevelDB seems to suffer under heavy load (see [#226](https://github.com/syndtr/goleveldb/issues/226)). It may be best to -install the real C-implementaiton of LevelDB and compile Tendermint to use +install the real C-implementation of LevelDB and compile Tendermint to use that using `make build_c`. See the [install instructions](../introduction/install.md) for details. Tendermint keeps multiple distinct LevelDB databases in the `$TMROOT/data`: