diff --git a/docs/spec/blockchain/blockchain.md b/docs/spec/blockchain/blockchain.md index ec1ab48c..73f4f35a 100644 --- a/docs/spec/blockchain/blockchain.md +++ b/docs/spec/blockchain/blockchain.md @@ -27,6 +27,13 @@ type Block struct { } ``` +The signatures returned along with block `X` are those validating block +`X-1`. This can be a little confusing, but consider that +the `Header` also contains the `LastCommitHash`. It would be impossible +for a Header to include the commits that sign it, as it would cause an +infinite loop here. But when we get block `X`, we find +`Header.LastCommitHash`, which must match the hash of `LastCommit`. + ## Header A block header contains metadata about the block and about the consensus, as well as commitments to @@ -34,32 +41,30 @@ the data in the current block, the previous block, and the results returned by t ```go type Header struct { - // block metadata - Version string // Version string - ChainID string // ID of the chain - Height int64 // Current block height - Time int64 // UNIX time, in millisconds + // basic block info + ChainID string `json:"chain_id"` + Height int64 `json:"height"` + Time time.Time `json:"time"` + NumTxs int64 `json:"num_txs"` + TotalTxs int64 `json:"total_txs"` - // current block - NumTxs int64 // Number of txs in this block - TxHash []byte // SimpleMerkle of the block.Txs - LastCommitHash []byte // SimpleMerkle of the block.LastCommit + // prev block info + LastBlockID BlockID `json:"last_block_id"` - // previous block - TotalTxs int64 // prevBlock.TotalTxs + block.NumTxs - LastBlockID BlockID // BlockID of prevBlock + // hashes of block data + LastCommitHash cmn.HexBytes `json:"last_commit_hash"` // commit from validators from the last block + DataHash cmn.HexBytes `json:"data_hash"` // transactions - // application - ResultsHash []byte // SimpleMerkle of []abci.Result from prevBlock - AppHash []byte // Arbitrary state digest - ValidatorsHash []byte // SimpleMerkle of the current ValidatorSet - NextValidatorsHash []byte // SimpleMerkle of the next ValidatorSet - ConsensusParamsHash []byte // SimpleMerkle of the ConsensusParams + // hashes from the app output from the prev block + ValidatorsHash cmn.HexBytes `json:"validators_hash"` // validators for the current block + NextValidatorsHash cmn.HexBytes `json:"next_validators_hash"` // validators for the next block + ConsensusHash cmn.HexBytes `json:"consensus_hash"` // consensus params for current block + AppHash cmn.HexBytes `json:"app_hash"` // state after txs from the previous block + LastResultsHash cmn.HexBytes `json:"last_results_hash"` // root hash of all results from the txs from the previous block - // consensus - EvidenceHash []byte // SimpleMerkle of []Evidence - ProposerAddress []byte // Address of the original proposer of the block -} + // consensus info + EvidenceHash cmn.HexBytes `json:"evidence_hash"` // evidence included in the block + ProposerAddress Address `json:"proposer_address"` // original proposer of the block ``` Further details on each of these fields is described below. @@ -224,6 +229,17 @@ These are the votes that committed the previous block. The first block has `block.Header.LastCommitHash == []byte{}` +### DataHash + +The `DataHash` can provide a nice check on the +[Data](https://godoc.org/github.com/tendermint/tendermint/types#Data) +returned in this same block. If you are subscribed to new blocks, via +tendermint RPC, in order to display or process the new transactions you +should at least validate that the `DataHash` is valid. If it is +important to verify autheniticity, you must wait for the `LastCommit` +from the next block to make sure the block header (including `DataHash`) +was properly signed. + ### TotalTxs ```go @@ -270,7 +286,7 @@ The first block has `block.Header.ResultsHash == []byte{}`. block.AppHash == state.AppHash ``` -Arbitrary byte array returned by the application after executing and commiting the previous block. +Arbitrary byte array returned by the application after executing and commiting the previous block. It serves as the basis for validating any merkle proofs that comes from the ABCI application and represents the state of the actual application rather than the state of the blockchain itself. The first block has `block.Header.AppHash == []byte{}`. diff --git a/docs/spec/consensus/consensus.md b/docs/spec/consensus/consensus.md index 842c73a6..c77c09d2 100644 --- a/docs/spec/consensus/consensus.md +++ b/docs/spec/consensus/consensus.md @@ -17,7 +17,7 @@ vote](https://godoc.org/github.com/tendermint/tendermint/types#FirstPrecommit) for something. - A vote _at_ `(H,R)` is a vote signed with the bytes for `H` and `R` - included in its [sign-bytes](block-structure.html#vote-sign-bytes). + included in its [sign-bytes](../blockchain/blockchain.md). - _+2/3_ is short for "more than 2/3" - _1/3+_ is short for "1/3 or more" - A set of +2/3 of prevotes for a particular block or `` at diff --git a/docs/tendermint-core/block-structure.md b/docs/tendermint-core/block-structure.md index f58e83aa..587db0ff 100644 --- a/docs/tendermint-core/block-structure.md +++ b/docs/tendermint-core/block-structure.md @@ -7,200 +7,6 @@ nodes. This blockchain is accessible via various rpc endpoints, mainly `/blockchain?minHeight=_&maxHeight=_` to get a list of headers. But what exactly is stored in these blocks? -## Block +The [specification](../spec/blockchain/blockchain.md) contains a detailed description of each component - that's the best place to get started. -A -[Block](https://godoc.org/github.com/tendermint/tendermint/types#Block) -contains: - -- a [Header](#header) contains merkle hashes for various chain states -- the - [Data](https://godoc.org/github.com/tendermint/tendermint/types#Data) - is all transactions which are to be processed -- the [LastCommit](#commit) > 2/3 signatures for the last block - -The signatures returned along with block `H` are those validating block -`H-1`. This can be a little confusing, but we must also consider that -the `Header` also contains the `LastCommitHash`. It would be impossible -for a Header to include the commits that sign it, as it would cause an -infinite loop here. But when we get block `H`, we find -`Header.LastCommitHash`, which must match the hash of `LastCommit`. - -## Header - -The -[Header](https://godoc.org/github.com/tendermint/tendermint/types#Header) -contains lots of information (follow link for up-to-date info). Notably, -it maintains the `Height`, the `LastBlockID` (to make it a chain), and -hashes of the data, the app state, and the validator set. This is -important as the only item that is signed by the validators is the -`Header`, and all other data must be validated against one of the merkle -hashes in the `Header`. - -The `DataHash` can provide a nice check on the -[Data](https://godoc.org/github.com/tendermint/tendermint/types#Data) -returned in this same block. If you are subscribed to new blocks, via -tendermint RPC, in order to display or process the new transactions you -should at least validate that the `DataHash` is valid. If it is -important to verify autheniticity, you must wait for the `LastCommit` -from the next block to make sure the block header (including `DataHash`) -was properly signed. - -The `ValidatorHash` contains a hash of the current -[Validators](https://godoc.org/github.com/tendermint/tendermint/types#Validator). -Tracking all changes in the validator set is complex, but a client can -quickly compare this hash with the [hash of the currently known -validators](https://godoc.org/github.com/tendermint/tendermint/types#ValidatorSet.Hash) -to see if there have been changes. - -The `AppHash` serves as the basis for validating any merkle proofs that -come from the ABCI application. It represents the state of the actual -application, rather that the state of the blockchain itself. This means -it's necessary in order to perform any business logic, such as verifying -an account balance. - -**Note** After the transactions are committed to a block, they still -need to be processed in a separate step, which happens between the -blocks. If you find a given transaction in the block at height `H`, the -effects of running that transaction will be first visible in the -`AppHash` from the block header at height `H+1`. - -Like the `LastCommit` issue, this is a requirement of the immutability -of the block chain, as the application only applies transactions _after_ -they are commited to the chain. - -## Commit - -The -[Commit](https://godoc.org/github.com/tendermint/tendermint/types#Commit) -contains a set of -[Votes](https://godoc.org/github.com/tendermint/tendermint/types#Vote) -that were made by the validator set to reach consensus on this block. -This is the key to the security in any PoS system, and actually no data -that cannot be traced back to a block header with a valid set of Votes -can be trusted. Thus, getting the Commit data and verifying the votes is -extremely important. - -As mentioned above, in order to find the `precommit votes` for block -header `H`, we need to query block `H+1`. Then we need to check the -votes, make sure they really are for that block, and properly formatted. -Much of this code is implemented in Go in the -[light-client](https://github.com/tendermint/light-client) package. If -you look at the code, you will notice that we need to provide the -`chainID` of the blockchain in order to properly calculate the votes. -This is to protect anyone from swapping votes between chains to fake (or -frame) a validator. Also note that this `chainID` is in the -`genesis.json` from _Tendermint_, not the `genesis.json` from the -basecoin app ([that is a different -chainID...](https://github.com/cosmos/cosmos-sdk/issues/32)). - -Once we have those votes, and we calculated the proper [sign -bytes](https://godoc.org/github.com/tendermint/tendermint/types#Vote.WriteSignBytes) -using the chainID and a [nice helper -function](https://godoc.org/github.com/tendermint/tendermint/types#SignBytes), -we can verify them. The light client is responsible for maintaining a -set of validators that we trust. Each vote only stores the validators -`Address`, as well as the `Signature`. Assuming we have a local copy of -the trusted validator set, we can look up the `Public Key` of the -validator given its `Address`, then verify that the `Signature` matches -the `SignBytes` and `Public Key`. Then we sum up the total voting power -of all validators, whose votes fulfilled all these stringent -requirements. If the total number of voting power for a single block is -greater than 2/3 of all voting power, then we can finally trust the -block header, the AppHash, and the proof we got from the ABCI -application. - -### Vote Sign Bytes - -The `sign-bytes` of a vote is produced by taking a -[stable-json](https://github.com/substack/json-stable-stringify)-like -deterministic JSON [wire](./wire-protocol.html) encoding of the vote -(excluding the `Signature` field), and wrapping it with -`{"chain_id":"my_chain","vote":...}`. - -For example, a precommit vote might have the following `sign-bytes`: - -``` -{"chain_id":"my_chain","vote":{"block_hash":"611801F57B4CE378DF1A3FFF1216656E89209A99","block_parts_header":{"hash":"B46697379DBE0774CC2C3B656083F07CA7E0F9CE","total":123},"height":1234,"round":1,"type":2}} -``` - -## Block Hash - -The [block -hash](https://godoc.org/github.com/tendermint/tendermint/types#Block.Hash) -is the [Simple Tree hash](./merkle.html#simple-tree-with-dictionaries) -of the fields of the block `Header` encoded as a list of `KVPair`s. - -## Transaction - -A transaction is any sequence of bytes. It is up to your ABCI -application to accept or reject transactions. - -## BlockID - -Many of these data structures refer to the -[BlockID](https://godoc.org/github.com/tendermint/tendermint/types#BlockID), -which is the `BlockHash` (hash of the block header, also referred to by -the next block) along with the `PartSetHeader`. The `PartSetHeader` is -explained below and is used internally to orchestrate the p2p -propogation. For clients, it is basically opaque bytes, but they must -match for all votes. - -## PartSetHeader - -The -[PartSetHeader](https://godoc.org/github.com/tendermint/tendermint/types#PartSetHeader) -contains the total number of pieces in a -[PartSet](https://godoc.org/github.com/tendermint/tendermint/types#PartSet), -and the Merkle root hash of those pieces. - -## PartSet - -PartSet is used to split a byteslice of data into parts (pieces) for -transmission. By splitting data into smaller parts and computing a -Merkle root hash on the list, you can verify that a part is legitimately -part of the complete data, and the part can be forwarded to other peers -before all the parts are known. In short, it's a fast way to securely -propagate a large chunk of data (like a block) over a gossip network. - -PartSet was inspired by the LibSwift project. - -Usage: - -``` -data := RandBytes(2 << 20) // Something large - -partSet := NewPartSetFromData(data) -partSet.Total() // Total number of 4KB parts -partSet.Count() // Equal to the Total, since we already have all the parts -partSet.Hash() // The Merkle root hash -partSet.BitArray() // A BitArray of partSet.Total() 1's - -header := partSet.Header() // Send this to the peer -header.Total // Total number of parts -header.Hash // The merkle root hash - -// Now we'll reconstruct the data from the parts -partSet2 := NewPartSetFromHeader(header) -partSet2.Total() // Same total as partSet.Total() -partSet2.Count() // Zero, since this PartSet doesn't have any parts yet. -partSet2.Hash() // Same hash as in partSet.Hash() -partSet2.BitArray() // A BitArray of partSet.Total() 0's - -// In a gossip network the parts would arrive in arbitrary order, perhaps -// in response to explicit requests for parts, or optimistically in response -// to the receiving peer's partSet.BitArray(). -for !partSet2.IsComplete() { - part := receivePartFromGossipNetwork() - added, err := partSet2.AddPart(part) - if err != nil { - // A wrong part, - // the merkle trail does not hash to partSet2.Hash() - } else if !added { - // A duplicate part already received - } -} - -data2, _ := ioutil.ReadAll(partSet2.GetReader()) -bytes.Equal(data, data2) // true -``` +To dig deeper, check out the [types package documentation](https://godoc.org/github.com/tendermint/tendermint/types). diff --git a/types/block.go b/types/block.go index f705c7db..d0a1a826 100644 --- a/types/block.go +++ b/types/block.go @@ -207,6 +207,7 @@ func (b *Block) StringShort() string { // Header defines the structure of a Tendermint block header // TODO: limit header size // NOTE: changes to the Header should be duplicated in the abci Header +// and in /docs/spec/blockchain/blockchain.md type Header struct { // basic block info ChainID string `json:"chain_id"`