Update abci and app docs (#2470)
* mempool: update some comments * make build_c * docs: notes about databases and WAL files * docs: determinism. closes #1279 * docs: small note about query paths. closes #2090 * docs: gas * docs: abci consensus params
This commit is contained in:
parent
111e627037
commit
f5824bc837
5
Makefile
5
Makefile
|
@ -23,11 +23,14 @@ check: check_tools get_vendor_deps
|
|||
build:
|
||||
CGO_ENABLED=0 go build $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint/
|
||||
|
||||
build_c:
|
||||
CGO_ENABLED=1 go build $(BUILD_FLAGS) -tags "$(BUILD_TAGS) gcc" -o build/tendermint ./cmd/tendermint/
|
||||
|
||||
build_race:
|
||||
CGO_ENABLED=0 go build -race $(BUILD_FLAGS) -tags $(BUILD_TAGS) -o build/tendermint ./cmd/tendermint
|
||||
|
||||
install:
|
||||
CGO_ENABLED=0 go install $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
|
||||
CGO_ENABLED=0 go install $(BUILD_FLAGS) -tags $(BUILD_TAGS) ./cmd/tendermint
|
||||
|
||||
########################################
|
||||
### Protobuf
|
||||
|
|
|
@ -77,8 +77,13 @@ make install
|
|||
|
||||
## Compile with CLevelDB support
|
||||
|
||||
Install [LevelDB](https://github.com/google/leveldb) (minimum version is 1.7)
|
||||
with snappy. Example for Ubuntu:
|
||||
Install [LevelDB](https://github.com/google/leveldb) (minimum version is 1.7).
|
||||
|
||||
Build Tendermint with C libraries: `make build_c`.
|
||||
|
||||
### Ubuntu
|
||||
|
||||
Install LevelDB with snappy:
|
||||
|
||||
```
|
||||
sudo apt-get update
|
||||
|
|
|
@ -48,16 +48,50 @@ Keys and values in tags must be UTF-8 encoded strings (e.g.
|
|||
|
||||
## Determinism
|
||||
|
||||
Some methods (`SetOption, Query, CheckTx, DeliverTx`) return
|
||||
non-deterministic data in the form of `Info` and `Log`. The `Log` is
|
||||
intended for the literal output from the application's logger, while the
|
||||
`Info` is any additional info that should be returned.
|
||||
|
||||
All other fields in the `Response*` of all methods must be strictly deterministic.
|
||||
ABCI applications must implement deterministic finite-state machines to be
|
||||
securely replicated by the Tendermint consensus. This means block execution
|
||||
over the Consensus Connection must be strictly deterministic: given the same
|
||||
ordered set of requests, all nodes will compute identical responses, for all
|
||||
BeginBlock, DeliverTx, EndBlock, and Commit. This is critical, because the
|
||||
responses are included in the header of the next block, either via a Merkle root
|
||||
or directly, so all nodes must agree on exactly what they are.
|
||||
|
||||
For this reason, it is recommended that applications not be exposed to any
|
||||
external user or process except via the ABCI connections to a consensus engine
|
||||
like Tendermint Core.
|
||||
like Tendermint Core. The application must only change its state based on input
|
||||
from block execution (BeginBlock, DeliverTx, EndBlock, Commit), and not through
|
||||
any other kind of request. This is the only way to ensure all nodes see the same
|
||||
transactions and compute the same results.
|
||||
|
||||
If there is some non-determinism in the state machine, consensus will eventually
|
||||
fail as nodes disagree over the correct values for the block header. The
|
||||
non-determinism must be fixed and the nodes restarted.
|
||||
|
||||
Sources of non-determinism in applications may include:
|
||||
|
||||
- Hardware failures
|
||||
- Cosmic rays, overheating, etc.
|
||||
- Node-dependent state
|
||||
- Random numbers
|
||||
- Time
|
||||
- Underspecification
|
||||
- Library version changes
|
||||
- Race conditions
|
||||
- Floating point numbers
|
||||
- JSON serialization
|
||||
- Iterating through hash-tables/maps/dictionaries
|
||||
- External Sources
|
||||
- Filesystem
|
||||
- Network calls (eg. some external REST API service)
|
||||
|
||||
See [#56](https://github.com/tendermint/abci/issues/56) for original discussion.
|
||||
|
||||
Note that some methods (`SetOption, Query, CheckTx, DeliverTx`) return
|
||||
explicitly non-deterministic data in the form of `Info` and `Log` fields. The `Log` is
|
||||
intended for the literal output from the application's logger, while the
|
||||
`Info` is any additional info that should be returned. These are the only fields
|
||||
that are not included in block header computations, so we don't need agreement
|
||||
on them. All other fields in the `Response*` must be strictly deterministic.
|
||||
|
||||
## Block Execution
|
||||
|
||||
|
@ -217,7 +251,7 @@ Commit are included in the header of the next block.
|
|||
be non-deterministic.
|
||||
- `Info (string)`: Additional information. May
|
||||
be non-deterministic.
|
||||
- `GasWanted (int64)`: Amount of gas request for transaction.
|
||||
- `GasWanted (int64)`: Amount of gas requested for transaction.
|
||||
- `GasUsed (int64)`: Amount of gas consumed by transaction.
|
||||
- `Tags ([]cmn.KVPair)`: Key-Value tags for filtering and indexing
|
||||
transactions (eg. by account).
|
||||
|
|
|
@ -86,18 +86,50 @@ Otherwise it should never be modified.
|
|||
|
||||
## Transaction Results
|
||||
|
||||
`ResponseCheckTx` and `ResponseDeliverTx` contain the same fields, though they
|
||||
have slightly different effects.
|
||||
`ResponseCheckTx` and `ResponseDeliverTx` contain the same fields.
|
||||
|
||||
In both cases, `Info` and `Log` are non-deterministic values for debugging/convenience purposes
|
||||
The `Info` and `Log` fields are non-deterministic values for debugging/convenience purposes
|
||||
that are otherwise ignored.
|
||||
|
||||
In both cases, `GasWanted` and `GasUsed` parameters are currently ignored,
|
||||
though see issues
|
||||
[#1861](https://github.com/tendermint/tendermint/issues/1861),
|
||||
[#2299](https://github.com/tendermint/tendermint/issues/2299) and
|
||||
[#2310](https://github.com/tendermint/tendermint/issues/2310) for how this may
|
||||
soon change.
|
||||
The `Data` field must be strictly deterministic, but can be arbitrary data.
|
||||
|
||||
### Gas
|
||||
|
||||
Ethereum introduced the notion of `gas` as an absract representation of the
|
||||
cost of resources used by nodes when processing transactions. Every operation in the
|
||||
Ethereum Virtual Machine uses some amount of gas, and gas can be accepted at a market-variable price.
|
||||
Users propose a maximum amount of gas for their transaction; if the tx uses less, they get
|
||||
the difference credited back. Tendermint adopts a similar abstraction,
|
||||
though uses it only optionally and weakly, allowing applications to define
|
||||
their own sense of the cost of execution.
|
||||
|
||||
In Tendermint, the `ConsensusParams.BlockSize.MaxGas` limits the amount of `gas` that can be used in a block.
|
||||
The default value is `-1`, meaning no limit, or that the concept of gas is
|
||||
meaningless.
|
||||
|
||||
Responses contain a `GasWanted` and `GasUsed` field. The former is the maximum
|
||||
amount of gas the sender of a tx is willing to use, and the later is how much it actually
|
||||
used. Applications should enforce that `GasUsed <= GasWanted` - ie. tx execution
|
||||
should halt before it can use more resources than it requested.
|
||||
|
||||
When `MaxGas > -1`, Tendermint enforces the following rules:
|
||||
|
||||
- `GasWanted <= MaxGas` for all txs in the mempool
|
||||
- `(sum of GasWanted in a block) <= MaxGas` when proposing a block
|
||||
|
||||
If `MaxGas == -1`, no rules about gas are enforced.
|
||||
|
||||
Note that Tendermint does not currently enforce anything about Gas in the consensus, only the mempool.
|
||||
This means it does not guarantee that committed blocks satisfy these rules!
|
||||
It is the application's responsibility to return non-zero response codes when gas limits are exceeded.
|
||||
|
||||
The `GasUsed` field is ignored compltely by Tendermint. That said, applications should enforce:
|
||||
- `GasUsed <= GasWanted` for any given transaction
|
||||
- `(sum of GasUsed in a block) <= MaxGas` for every block
|
||||
|
||||
In the future, we intend to add a `Priority` field to the responses that can be
|
||||
used to explicitly prioritize txs in the mempool for inclusion in a block
|
||||
proposal. See [#1861](https://github.com/tendermint/tendermint/issues/1861).
|
||||
|
||||
### CheckTx
|
||||
|
||||
|
@ -142,9 +174,6 @@ If the list is not empty, Tendermint will use it for the validator set.
|
|||
This way the application can determine the initial validator set for the
|
||||
blockchain.
|
||||
|
||||
ResponseInitChain also includes ConsensusParams, but these are presently
|
||||
ignored.
|
||||
|
||||
### EndBlock
|
||||
|
||||
Updates to the Tendermint validator set can be made by returning
|
||||
|
@ -179,14 +208,74 @@ following rules:
|
|||
|
||||
Note the updates returned in block `H` will only take effect at block `H+2`.
|
||||
|
||||
## Consensus Parameters
|
||||
|
||||
ConsensusParams enforce certain limits in the blockchain, like the maximum size
|
||||
of blocks, amount of gas used in a block, and the maximum acceptable age of
|
||||
evidence. They can be set in InitChain and updated in EndBlock.
|
||||
|
||||
### BlockSize.MaxBytes
|
||||
|
||||
The maximum size of a complete Amino encoded block.
|
||||
This is enforced by Tendermint consensus.
|
||||
|
||||
This implies a maximum tx size that is this MaxBytes, less the expected size of
|
||||
the header, the validator set, and any included evidence in the block.
|
||||
|
||||
Must have `0 < MaxBytes < 100 MB`.
|
||||
|
||||
### BlockSize.MaxGas
|
||||
|
||||
The maximum of the sum of `GasWanted` in a proposed block.
|
||||
This is *not* enforced by Tendermint consensus.
|
||||
It is left to the app to enforce (ie. if txs are included past the
|
||||
limit, they should return non-zero codes). It is used by Tendermint to limit the
|
||||
txs included in a proposed block.
|
||||
|
||||
Must have `MaxGas >= -1`.
|
||||
If `MaxGas == -1`, no limit is enforced.
|
||||
|
||||
### EvidenceParams.MaxAge
|
||||
|
||||
This is the maximum age of evidence.
|
||||
This is enforced by Tendermint consensus.
|
||||
If a block includes evidence older than this, the block will be rejected
|
||||
(validators won't vote for it).
|
||||
|
||||
Must have `0 < MaxAge`.
|
||||
|
||||
### Updates
|
||||
|
||||
The application may set the consensus params during InitChain, and update them during
|
||||
EndBlock.
|
||||
|
||||
#### InitChain
|
||||
|
||||
ResponseInitChain includes a ConsensusParams.
|
||||
If its nil, Tendermint will use the params loaded in the genesis
|
||||
file. If it's not nil, Tendermint will use it.
|
||||
This way the application can determine the initial consensus params for the
|
||||
blockchain.
|
||||
|
||||
#### EndBlock
|
||||
|
||||
ResponseEndBlock includes a ConsensusParams.
|
||||
If its nil, Tendermint will do nothing.
|
||||
If it's not nil, Tendermint will use it.
|
||||
This way the application can update the consensus params over time.
|
||||
|
||||
Note the updates returned in block `H` will take effect right away for block
|
||||
`H+1`.
|
||||
|
||||
## Query
|
||||
|
||||
Query is a generic method with lots of flexibility to enable diverse sets
|
||||
of queries on application state. Tendermint makes use of Query to filter new peers
|
||||
based on ID and IP, and exposes Query to the user over RPC.
|
||||
|
||||
Note that calls to Query are not replicated across nodes, but rather query the
|
||||
local node's state - hence they may provide stale reads. For reads that require
|
||||
consensus, a transaction is required.
|
||||
local node's state - hence they may return stale reads. For reads that require
|
||||
consensus, use a transaction.
|
||||
|
||||
The most important use of Query is to return Merkle proofs of the application state at some height
|
||||
that can be used for efficient application-specific lite-clients.
|
||||
|
@ -235,6 +324,15 @@ using the following paths, with no additional data:
|
|||
If either of these queries return a non-zero ABCI code, Tendermint will refuse
|
||||
to connect to the peer.
|
||||
|
||||
### Paths
|
||||
|
||||
Queries are directed at paths, and may optionally include additional data.
|
||||
|
||||
The expectation is for there to be some number of high level paths
|
||||
differentiating concerns, like `/p2p`, `/store`, and `/app`. Currently,
|
||||
Tendermint only uses `/p2p`, for filtering peers. For more advanced use, see the
|
||||
implementation of
|
||||
[Query in the Cosmos-SDK](https://github.com/cosmos/cosmos-sdk/blob/v0.23.1/baseapp/baseapp.go#L333).
|
||||
|
||||
## Crash Recovery
|
||||
|
||||
|
|
|
@ -1,5 +1,41 @@
|
|||
# Running in production
|
||||
|
||||
## Database
|
||||
|
||||
By default, Tendermint uses the `syndtr/goleveldb` package for it's in-process
|
||||
key-value database. Unfortunately, this implementation of LevelDB seems to suffer under heavy load (see
|
||||
[#226](https://github.com/syndtr/goleveldb/issues/226)). It may be best to
|
||||
install the real C-implementaiton of LevelDB and compile Tendermint to use
|
||||
that using `make build_c`. See the [install instructions](../introduction/install) for details.
|
||||
|
||||
Tendermint keeps multiple distinct LevelDB databases in the `$TMROOT/data`:
|
||||
|
||||
- `blockstore.db`: Keeps the entire blockchain - stores blocks,
|
||||
block commits, and block meta data, each indexed by height. Used to sync new
|
||||
peers.
|
||||
- `evidence.db`: Stores all verified evidence of misbehaviour.
|
||||
- `state.db`: Stores the current blockchain state (ie. height, validators,
|
||||
consensus params). Only grows if consensus params or validators change. Also
|
||||
used to temporarily store intermediate results during block processing.
|
||||
- `tx_index.db`: Indexes txs (and their results) by tx hash and by DeliverTx result tags.
|
||||
|
||||
By default, Tendermint will only index txs by their hash, not by their DeliverTx
|
||||
result tags. See [indexing transactions](../app-dev/indexing-transactions) for
|
||||
details.
|
||||
|
||||
There is no current strategy for pruning the databases. Consider reducing
|
||||
block production by [controlling empty blocks](../tendermint-core/using-tendermint#No-Empty-Blocks)
|
||||
or by increasing the `consensus.timeout_commit` param. Note both of these are
|
||||
local settings and not enforced by the consensus.
|
||||
|
||||
We're working on [state
|
||||
syncing](https://github.com/tendermint/tendermint/issues/828),
|
||||
which will enable history to be thrown away
|
||||
and recent application state to be directly synced. We'll need to develop solutions
|
||||
for archival nodes that allow queries on historical transactions and states.
|
||||
The Cosmos project has had much success just dumping the latest state of a
|
||||
blockchain to disk and starting a new chain from that state.
|
||||
|
||||
## Logging
|
||||
|
||||
Default logging level (`main:info,state:info,*:`) should suffice for
|
||||
|
@ -11,6 +47,29 @@ you're trying to debug Tendermint or asked to provide logs with debug
|
|||
logging level, you can do so by running tendermint with
|
||||
`--log_level="*:debug"`.
|
||||
|
||||
## Write Ahead Logs (WAL)
|
||||
|
||||
Tendermint uses write ahead logs for the consensus (`cs.wal`) and the mempool
|
||||
(`mempool.wal`). Both WALs have a max size of 1GB and are automatically rotated..
|
||||
|
||||
The `consensus.wal` is used to ensure we can recover from a crash at any point
|
||||
in the consensus state machine.
|
||||
It writes all consensus messages (timeouts, proposals, block part, or vote)
|
||||
to a single file, flushing to disk before processing messages from its own
|
||||
validator. Since Tendermint validators are expected to never sign a conflicting vote, the
|
||||
WAL ensures we can always recover deterministically to the latest state of the consensus without
|
||||
using the network or re-signing any consensus messages.
|
||||
|
||||
If your `consensus.wal` is corrupted, see [below](#WAL-Corruption).
|
||||
|
||||
The `mempool.wal` logs all incoming txs before running CheckTx, but is
|
||||
otherwise not used in any programmatic way. It's just a kind of manual
|
||||
safe guard. Note the mempool provides no durability guarantees - a tx sent to one or many nodes
|
||||
may never make it into the blockchain if those nodes crash before being able to
|
||||
propose it. Clients must monitor their txs by subscribing over websockets,
|
||||
polling for them, or using `/broadcast_tx_commit`. In the worst case, txs can be
|
||||
resent from the mempool WAL manually.
|
||||
|
||||
## DOS Exposure and Mitigation
|
||||
|
||||
Validators are supposed to setup [Sentry Node
|
||||
|
|
|
@ -22,14 +22,14 @@ import (
|
|||
"github.com/tendermint/tendermint/types"
|
||||
)
|
||||
|
||||
// PreCheckFunc is an optional filter to determine if a transaction should be
|
||||
// rejected. Invoked before CheckTx. An example would be to ensure that a
|
||||
// transaction isn't exceeded the block size.
|
||||
// PreCheckFunc is an optional filter executed before CheckTx and rejects
|
||||
// transaction if false is returned. An example would be to ensure that a
|
||||
// transaction doesn't exceeded the block size.
|
||||
type PreCheckFunc func(types.Tx) bool
|
||||
|
||||
// PostCheckFunc is an optional filter executed after CheckTx and rejects
|
||||
// transaction if false is returned. An example would be to ensure a
|
||||
// transaction doesn't require more gas than available.
|
||||
// transaction doesn't require more gas than available for the block.
|
||||
type PostCheckFunc func(types.Tx, *abci.ResponseCheckTx) bool
|
||||
|
||||
/*
|
||||
|
|
Loading…
Reference in New Issue