Update abci and app docs (#2470)
* mempool: update some comments * make build_c * docs: notes about databases and WAL files * docs: determinism. closes #1279 * docs: small note about query paths. closes #2090 * docs: gas * docs: abci consensus params
This commit is contained in:
parent
111e627037
commit
f5824bc837
5
Makefile
5
Makefile
|
@ -23,11 +23,14 @@ check: check_tools get_vendor_deps
|
||||||
build:
|
build:
|
||||||
CGO_ENABLED=0 go build $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint/
|
CGO_ENABLED=0 go build $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint/
|
||||||
|
|
||||||
|
build_c:
|
||||||
|
CGO_ENABLED=1 go build $(BUILD_FLAGS) -tags "$(BUILD_TAGS) gcc" -o build/tendermint ./cmd/tendermint/
|
||||||
|
|
||||||
build_race:
|
build_race:
|
||||||
CGO_ENABLED=0 go build -race $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint
|
CGO_ENABLED=0 go build -race $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint
|
||||||
|
|
||||||
install:
|
install:
|
||||||
CGO_ENABLED=0 go install $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
|
CGO_ENABLED=0 go install $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
|
||||||
|
|
||||||
########################################
|
########################################
|
||||||
### Protobuf
|
### Protobuf
|
||||||
|
|
|
@ -77,8 +77,13 @@ make install
|
||||||
|
|
||||||
## Compile with CLevelDB support
|
## Compile with CLevelDB support
|
||||||
|
|
||||||
Install [LevelDB](https://github.com/google/leveldb) (minimum version is 1.7)
|
Install [LevelDB](https://github.com/google/leveldb) (minimum version is 1.7).
|
||||||
with snappy. Example for Ubuntu:
|
|
||||||
|
Build Tendermint with C libraries: `make build_c`.
|
||||||
|
|
||||||
|
### Ubuntu
|
||||||
|
|
||||||
|
Install LevelDB with snappy:
|
||||||
|
|
||||||
```
|
```
|
||||||
sudo apt-get update
|
sudo apt-get update
|
||||||
|
|
|
@ -48,16 +48,50 @@ Keys and values in tags must be UTF-8 encoded strings (e.g.
|
||||||
|
|
||||||
## Determinism
|
## Determinism
|
||||||
|
|
||||||
Some methods (`SetOption, Query, CheckTx, DeliverTx`) return
|
ABCI applications must implement deterministic finite-state machines to be
|
||||||
non-deterministic data in the form of `Info` and `Log`. The `Log` is
|
securely replicated by the Tendermint consensus. This means block execution
|
||||||
intended for the literal output from the application's logger, while the
|
over the Consensus Connection must be strictly deterministic: given the same
|
||||||
`Info` is any additional info that should be returned.
|
ordered set of requests, all nodes will compute identical responses, for all
|
||||||
|
BeginBlock, DeliverTx, EndBlock, and Commit. This is critical, because the
|
||||||
All other fields in the `Response*` of all methods must be strictly deterministic.
|
responses are included in the header of the next block, either via a Merkle root
|
||||||
|
or directly, so all nodes must agree on exactly what they are.
|
||||||
|
|
||||||
For this reason, it is recommended that applications not be exposed to any
|
For this reason, it is recommended that applications not be exposed to any
|
||||||
external user or process except via the ABCI connections to a consensus engine
|
external user or process except via the ABCI connections to a consensus engine
|
||||||
like Tendermint Core.
|
like Tendermint Core. The application must only change its state based on input
|
||||||
|
from block execution (BeginBlock, DeliverTx, EndBlock, Commit), and not through
|
||||||
|
any other kind of request. This is the only way to ensure all nodes see the same
|
||||||
|
transactions and compute the same results.
|
||||||
|
|
||||||
|
If there is some non-determinism in the state machine, consensus will eventually
|
||||||
|
fail as nodes disagree over the correct values for the block header. The
|
||||||
|
non-determinism must be fixed and the nodes restarted.
|
||||||
|
|
||||||
|
Sources of non-determinism in applications may include:
|
||||||
|
|
||||||
|
- Hardware failures
|
||||||
|
- Cosmic rays, overheating, etc.
|
||||||
|
- Node-dependent state
|
||||||
|
- Random numbers
|
||||||
|
- Time
|
||||||
|
- Underspecification
|
||||||
|
- Library version changes
|
||||||
|
- Race conditions
|
||||||
|
- Floating point numbers
|
||||||
|
- JSON serialization
|
||||||
|
- Iterating through hash-tables/maps/dictionaries
|
||||||
|
- External Sources
|
||||||
|
- Filesystem
|
||||||
|
- Network calls (eg. some external REST API service)
|
||||||
|
|
||||||
|
See [#56](https://github.com/tendermint/abci/issues/56) for original discussion.
|
||||||
|
|
||||||
|
Note that some methods (`SetOption, Query, CheckTx, DeliverTx`) return
|
||||||
|
explicitly non-deterministic data in the form of `Info` and `Log` fields. The `Log` is
|
||||||
|
intended for the literal output from the application's logger, while the
|
||||||
|
`Info` is any additional info that should be returned. These are the only fields
|
||||||
|
that are not included in block header computations, so we don't need agreement
|
||||||
|
on them. All other fields in the `Response*` must be strictly deterministic.
|
||||||
|
|
||||||
## Block Execution
|
## Block Execution
|
||||||
|
|
||||||
|
@ -217,7 +251,7 @@ Commit are included in the header of the next block.
|
||||||
be non-deterministic.
|
be non-deterministic.
|
||||||
- `Info (string)`: Additional information. May
|
- `Info (string)`: Additional information. May
|
||||||
be non-deterministic.
|
be non-deterministic.
|
||||||
- `GasWanted (int64)`: Amount of gas request for transaction.
|
- `GasWanted (int64)`: Amount of gas requested for transaction.
|
||||||
- `GasUsed (int64)`: Amount of gas consumed by transaction.
|
- `GasUsed (int64)`: Amount of gas consumed by transaction.
|
||||||
- `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing
|
- `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing
|
||||||
transactions (eg. by account).
|
transactions (eg. by account).
|
||||||
|
|
|
@ -86,18 +86,50 @@ Otherwise it should never be modified.
|
||||||
|
|
||||||
## Transaction Results
|
## Transaction Results
|
||||||
|
|
||||||
`ResponseCheckTx` and `ResponseDeliverTx` contain the same fields, though they
|
`ResponseCheckTx` and `ResponseDeliverTx` contain the same fields.
|
||||||
have slightly different effects.
|
|
||||||
|
|
||||||
In both cases, `Info` and `Log` are non-deterministic values for debugging/convenience purposes
|
The `Info` and `Log` fields are non-deterministic values for debugging/convenience purposes
|
||||||
that are otherwise ignored.
|
that are otherwise ignored.
|
||||||
|
|
||||||
In both cases, `GasWanted` and `GasUsed` parameters are currently ignored,
|
The `Data` field must be strictly deterministic, but can be arbitrary data.
|
||||||
though see issues
|
|
||||||
[#1861](https://github.com/tendermint/tendermint/issues/1861),
|
### Gas
|
||||||
[#2299](https://github.com/tendermint/tendermint/issues/2299) and
|
|
||||||
[#2310](https://github.com/tendermint/tendermint/issues/2310) for how this may
|
Ethereum introduced the notion of `gas` as an absract representation of the
|
||||||
soon change.
|
cost of resources used by nodes when processing transactions. Every operation in the
|
||||||
|
Ethereum Virtual Machine uses some amount of gas, and gas can be accepted at a market-variable price.
|
||||||
|
Users propose a maximum amount of gas for their transaction; if the tx uses less, they get
|
||||||
|
the difference credited back. Tendermint adopts a similar abstraction,
|
||||||
|
though uses it only optionally and weakly, allowing applications to define
|
||||||
|
their own sense of the cost of execution.
|
||||||
|
|
||||||
|
In Tendermint, the `ConsensusParams.BlockSize.MaxGas` limits the amount of `gas` that can be used in a block.
|
||||||
|
The default value is `-1`, meaning no limit, or that the concept of gas is
|
||||||
|
meaningless.
|
||||||
|
|
||||||
|
Responses contain a `GasWanted` and `GasUsed` field. The former is the maximum
|
||||||
|
amount of gas the sender of a tx is willing to use, and the later is how much it actually
|
||||||
|
used. Applications should enforce that `GasUsed <= GasWanted` - ie. tx execution
|
||||||
|
should halt before it can use more resources than it requested.
|
||||||
|
|
||||||
|
When `MaxGas > -1`, Tendermint enforces the following rules:
|
||||||
|
|
||||||
|
- `GasWanted <= MaxGas` for all txs in the mempool
|
||||||
|
- `(sum of GasWanted in a block) <= MaxGas` when proposing a block
|
||||||
|
|
||||||
|
If `MaxGas == -1`, no rules about gas are enforced.
|
||||||
|
|
||||||
|
Note that Tendermint does not currently enforce anything about Gas in the consensus, only the mempool.
|
||||||
|
This means it does not guarantee that committed blocks satisfy these rules!
|
||||||
|
It is the application's responsibility to return non-zero response codes when gas limits are exceeded.
|
||||||
|
|
||||||
|
The `GasUsed` field is ignored compltely by Tendermint. That said, applications should enforce:
|
||||||
|
- `GasUsed <= GasWanted` for any given transaction
|
||||||
|
- `(sum of GasUsed in a block) <= MaxGas` for every block
|
||||||
|
|
||||||
|
In the future, we intend to add a `Priority` field to the responses that can be
|
||||||
|
used to explicitly prioritize txs in the mempool for inclusion in a block
|
||||||
|
proposal. See [#1861](https://github.com/tendermint/tendermint/issues/1861).
|
||||||
|
|
||||||
### CheckTx
|
### CheckTx
|
||||||
|
|
||||||
|
@ -142,9 +174,6 @@ If the list is not empty, Tendermint will use it for the validator set.
|
||||||
This way the application can determine the initial validator set for the
|
This way the application can determine the initial validator set for the
|
||||||
blockchain.
|
blockchain.
|
||||||
|
|
||||||
ResponseInitChain also includes ConsensusParams, but these are presently
|
|
||||||
ignored.
|
|
||||||
|
|
||||||
### EndBlock
|
### EndBlock
|
||||||
|
|
||||||
Updates to the Tendermint validator set can be made by returning
|
Updates to the Tendermint validator set can be made by returning
|
||||||
|
@ -179,14 +208,74 @@ following rules:
|
||||||
|
|
||||||
Note the updates returned in block `H` will only take effect at block `H+2`.
|
Note the updates returned in block `H` will only take effect at block `H+2`.
|
||||||
|
|
||||||
|
## Consensus Parameters
|
||||||
|
|
||||||
|
ConsensusParams enforce certain limits in the blockchain, like the maximum size
|
||||||
|
of blocks, amount of gas used in a block, and the maximum acceptable age of
|
||||||
|
evidence. They can be set in InitChain and updated in EndBlock.
|
||||||
|
|
||||||
|
### BlockSize.MaxBytes
|
||||||
|
|
||||||
|
The maximum size of a complete Amino encoded block.
|
||||||
|
This is enforced by Tendermint consensus.
|
||||||
|
|
||||||
|
This implies a maximum tx size that is this MaxBytes, less the expected size of
|
||||||
|
the header, the validator set, and any included evidence in the block.
|
||||||
|
|
||||||
|
Must have `0 < MaxBytes < 100 MB`.
|
||||||
|
|
||||||
|
### BlockSize.MaxGas
|
||||||
|
|
||||||
|
The maximum of the sum of `GasWanted` in a proposed block.
|
||||||
|
This is *not* enforced by Tendermint consensus.
|
||||||
|
It is left to the app to enforce (ie. if txs are included past the
|
||||||
|
limit, they should return non-zero codes). It is used by Tendermint to limit the
|
||||||
|
txs included in a proposed block.
|
||||||
|
|
||||||
|
Must have `MaxGas >= -1`.
|
||||||
|
If `MaxGas == -1`, no limit is enforced.
|
||||||
|
|
||||||
|
### EvidenceParams.MaxAge
|
||||||
|
|
||||||
|
This is the maximum age of evidence.
|
||||||
|
This is enforced by Tendermint consensus.
|
||||||
|
If a block includes evidence older than this, the block will be rejected
|
||||||
|
(validators won't vote for it).
|
||||||
|
|
||||||
|
Must have `0 < MaxAge`.
|
||||||
|
|
||||||
|
### Updates
|
||||||
|
|
||||||
|
The application may set the consensus params during InitChain, and update them during
|
||||||
|
EndBlock.
|
||||||
|
|
||||||
|
#### InitChain
|
||||||
|
|
||||||
|
ResponseInitChain includes a ConsensusParams.
|
||||||
|
If its nil, Tendermint will use the params loaded in the genesis
|
||||||
|
file. If it's not nil, Tendermint will use it.
|
||||||
|
This way the application can determine the initial consensus params for the
|
||||||
|
blockchain.
|
||||||
|
|
||||||
|
#### EndBlock
|
||||||
|
|
||||||
|
ResponseEndBlock includes a ConsensusParams.
|
||||||
|
If its nil, Tendermint will do nothing.
|
||||||
|
If it's not nil, Tendermint will use it.
|
||||||
|
This way the application can update the consensus params over time.
|
||||||
|
|
||||||
|
Note the updates returned in block `H` will take effect right away for block
|
||||||
|
`H+1`.
|
||||||
|
|
||||||
## Query
|
## Query
|
||||||
|
|
||||||
Query is a generic method with lots of flexibility to enable diverse sets
|
Query is a generic method with lots of flexibility to enable diverse sets
|
||||||
of queries on application state. Tendermint makes use of Query to filter new peers
|
of queries on application state. Tendermint makes use of Query to filter new peers
|
||||||
based on ID and IP, and exposes Query to the user over RPC.
|
based on ID and IP, and exposes Query to the user over RPC.
|
||||||
|
|
||||||
Note that calls to Query are not replicated across nodes, but rather query the
|
Note that calls to Query are not replicated across nodes, but rather query the
|
||||||
local node's state - hence they may provide stale reads. For reads that require
|
local node's state - hence they may return stale reads. For reads that require
|
||||||
consensus, a transaction is required.
|
consensus, use a transaction.
|
||||||
|
|
||||||
The most important use of Query is to return Merkle proofs of the application state at some height
|
The most important use of Query is to return Merkle proofs of the application state at some height
|
||||||
that can be used for efficient application-specific lite-clients.
|
that can be used for efficient application-specific lite-clients.
|
||||||
|
@ -235,6 +324,15 @@ using the following paths, with no additional data:
|
||||||
If either of these queries return a non-zero ABCI code, Tendermint will refuse
|
If either of these queries return a non-zero ABCI code, Tendermint will refuse
|
||||||
to connect to the peer.
|
to connect to the peer.
|
||||||
|
|
||||||
|
### Paths
|
||||||
|
|
||||||
|
Queries are directed at paths, and may optionally include additional data.
|
||||||
|
|
||||||
|
The expectation is for there to be some number of high level paths
|
||||||
|
differentiating concerns, like `/p2p`, `/store`, and `/app`. Currently,
|
||||||
|
Tendermint only uses `/p2p`, for filtering peers. For more advanced use, see the
|
||||||
|
implementation of
|
||||||
|
[Query in the Cosmos-SDK](https://github.com/cosmos/cosmos-sdk/blob/v0.23.1/baseapp/baseapp.go#L333).
|
||||||
|
|
||||||
## Crash Recovery
|
## Crash Recovery
|
||||||
|
|
||||||
|
|
|
@ -1,5 +1,41 @@
|
||||||
# Running in production
|
# Running in production
|
||||||
|
|
||||||
|
## Database
|
||||||
|
|
||||||
|
By default, Tendermint uses the `syndtr/goleveldb` package for it's in-process
|
||||||
|
key-value database. Unfortunately, this implementation of LevelDB seems to suffer under heavy load (see
|
||||||
|
[#226](https://github.com/syndtr/goleveldb/issues/226)). It may be best to
|
||||||
|
install the real C-implementaiton of LevelDB and compile Tendermint to use
|
||||||
|
that using `make build_c`. See the [install instructions](../introduction/install) for details.
|
||||||
|
|
||||||
|
Tendermint keeps multiple distinct LevelDB databases in the `$TMROOT/data`:
|
||||||
|
|
||||||
|
- `blockstore.db`: Keeps the entire blockchain - stores blocks,
|
||||||
|
block commits, and block meta data, each indexed by height. Used to sync new
|
||||||
|
peers.
|
||||||
|
- `evidence.db`: Stores all verified evidence of misbehaviour.
|
||||||
|
- `state.db`: Stores the current blockchain state (ie. height, validators,
|
||||||
|
consensus params). Only grows if consensus params or validators change. Also
|
||||||
|
used to temporarily store intermediate results during block processing.
|
||||||
|
- `tx_index.db`: Indexes txs (and their results) by tx hash and by DeliverTx result tags.
|
||||||
|
|
||||||
|
By default, Tendermint will only index txs by their hash, not by their DeliverTx
|
||||||
|
result tags. See [indexing transactions](../app-dev/indexing-transactions) for
|
||||||
|
details.
|
||||||
|
|
||||||
|
There is no current strategy for pruning the databases. Consider reducing
|
||||||
|
block production by [controlling empty blocks](../tendermint-core/using-tendermint#No-Empty-Blocks)
|
||||||
|
or by increasing the `consensus.timeout_commit` param. Note both of these are
|
||||||
|
local settings and not enforced by the consensus.
|
||||||
|
|
||||||
|
We're working on [state
|
||||||
|
syncing](https://github.com/tendermint/tendermint/issues/828),
|
||||||
|
which will enable history to be thrown away
|
||||||
|
and recent application state to be directly synced. We'll need to develop solutions
|
||||||
|
for archival nodes that allow queries on historical transactions and states.
|
||||||
|
The Cosmos project has had much success just dumping the latest state of a
|
||||||
|
blockchain to disk and starting a new chain from that state.
|
||||||
|
|
||||||
## Logging
|
## Logging
|
||||||
|
|
||||||
Default logging level (`main:info,state:info,*:`) should suffice for
|
Default logging level (`main:info,state:info,*:`) should suffice for
|
||||||
|
@ -11,6 +47,29 @@ you're trying to debug Tendermint or asked to provide logs with debug
|
||||||
logging level, you can do so by running tendermint with
|
logging level, you can do so by running tendermint with
|
||||||
`--log_level="*:debug"`.
|
`--log_level="*:debug"`.
|
||||||
|
|
||||||
|
## Write Ahead Logs (WAL)
|
||||||
|
|
||||||
|
Tendermint uses write ahead logs for the consensus (`cs.wal`) and the mempool
|
||||||
|
(`mempool.wal`). Both WALs have a max size of 1GB and are automatically rotated..
|
||||||
|
|
||||||
|
The `consensus.wal` is used to ensure we can recover from a crash at any point
|
||||||
|
in the consensus state machine.
|
||||||
|
It writes all consensus messages (timeouts, proposals, block part, or vote)
|
||||||
|
to a single file, flushing to disk before processing messages from its own
|
||||||
|
validator. Since Tendermint validators are expected to never sign a conflicting vote, the
|
||||||
|
WAL ensures we can always recover deterministically to the latest state of the consensus without
|
||||||
|
using the network or re-signing any consensus messages.
|
||||||
|
|
||||||
|
If your `consensus.wal` is corrupted, see [below](#WAL-Corruption).
|
||||||
|
|
||||||
|
The `mempool.wal` logs all incoming txs before running CheckTx, but is
|
||||||
|
otherwise not used in any programmatic way. It's just a kind of manual
|
||||||
|
safe guard. Note the mempool provides no durability guarantees - a tx sent to one or many nodes
|
||||||
|
may never make it into the blockchain if those nodes crash before being able to
|
||||||
|
propose it. Clients must monitor their txs by subscribing over websockets,
|
||||||
|
polling for them, or using `/broadcast_tx_commit`. In the worst case, txs can be
|
||||||
|
resent from the mempool WAL manually.
|
||||||
|
|
||||||
## DOS Exposure and Mitigation
|
## DOS Exposure and Mitigation
|
||||||
|
|
||||||
Validators are supposed to setup [Sentry Node
|
Validators are supposed to setup [Sentry Node
|
||||||
|
|
|
@ -22,14 +22,14 @@ import (
|
||||||
"github.com/tendermint/tendermint/types"
|
"github.com/tendermint/tendermint/types"
|
||||||
)
|
)
|
||||||
|
|
||||||
// PreCheckFunc is an optional filter to determine if a transaction should be
|
// PreCheckFunc is an optional filter executed before CheckTx and rejects
|
||||||
// rejected. Invoked before CheckTx. An example would be to ensure that a
|
// transaction if false is returned. An example would be to ensure that a
|
||||||
// transaction isn't exceeded the block size.
|
// transaction doesn't exceeded the block size.
|
||||||
type PreCheckFunc func(types.Tx) bool
|
type PreCheckFunc func(types.Tx) bool
|
||||||
|
|
||||||
// PostCheckFunc is an optional filter executed after CheckTx and rejects
|
// PostCheckFunc is an optional filter executed after CheckTx and rejects
|
||||||
// transaction if false is returned. An example would be to ensure a
|
// transaction if false is returned. An example would be to ensure a
|
||||||
// transaction doesn't require more gas than available.
|
// transaction doesn't require more gas than available for the block.
|
||||||
type PostCheckFunc func(types.Tx, *abci.ResponseCheckTx) bool
|
type PostCheckFunc func(types.Tx, *abci.ResponseCheckTx) bool
|
||||||
|
|
||||||
/*
|
/*
|
||||||
|
|
Loading…
Reference in New Issue