zecfoundation
/
zebra
mirror of https://github.com/ZcashFoundation/zebra.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Update the State RFC to match the current database format (#3139) 

* Update the State RFC to match the current database format

* Formatting and name fixes

* Remove redundant generic parameter

* Remove redundant generics

* Fix history tree types

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Fix spacing

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
This commit is contained in:
teor 2021-12-10 23:35:35 +10:00 committed by GitHub
parent f750535961
commit 62c78ad939
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 46 additions and 48 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

84
book/src/dev/rfcs/0005-state-updates.md
View File

@ -601,28 +601,45 @@ order on byte strings is the numeric ordering).
We use the following rocksdb column families: We use the following rocksdb column families:
| Column Family | Keys | Values | Updates | | Column Family | Keys | Values | Updates |
|-----------------------|-----------------------|--------------------------------------|---------| |--------------------------------|------------------------|--------------------------------------|---------|
| `hash_by_height` | `BE32(height)` | `block::Hash` | Never | | `hash_by_height` | `block::Height` | `block::Hash` | Never |
| `height_by_hash` | `block::Hash` | `BE32(height)` | Never | | `height_by_hash` | `block::Hash` | `block::Height` | Never |
| `block_by_height` | `BE32(height)` | `Block` | Never | | `block_by_height` | `block::Height` | `Block` | Never |
| `tx_by_hash` | `transaction::Hash` | `(BE32(height) \|\| BE32(tx_index))` | Never | | `tx_by_hash` | `transaction::Hash` | `TransactionLocation` | Never |
| `utxo_by_outpoint` | `OutPoint` | `transparent::Output` | Delete | | `utxo_by_outpoint` | `OutPoint` | `transparent::Utxo` | Delete |
| `sprout_nullifiers` | `sprout::Nullifier` | `()` | Never | | `sprout_nullifiers` | `sprout::Nullifier` | `()` | Never |
| `sapling_nullifiers` | `sapling::Nullifier` | `()` | Never |
| `orchard_nullifiers` | `orchard::Nullifier` | `()` | Never |
| `sprout_anchors` | `sprout::tree::Root` | `()` | Never | | `sprout_anchors` | `sprout::tree::Root` | `()` | Never |
| `sprout_incremental` | `BE32(height)` *?* | `sprout::tree::NoteCommitmentTree` | Delete | | `sprout_note_commitment_tree` | `block::Height` | `sprout::tree::NoteCommitmentTree` | Delete |
| `sapling_nullifiers` | `sapling::Nullifier` | `()` | Never |
| `sapling_anchors` | `sapling::tree::Root` | `()` | Never | | `sapling_anchors` | `sapling::tree::Root` | `()` | Never |
| `sapling_incremental` | `BE32(height)` *?* | `sapling::tree::NoteCommitmentTree` | Delete | | `sapling_note_commitment_tree` | `block::Height` | `sapling::tree::NoteCommitmentTree` | Delete |
| `orchard_nullifiers` | `orchard::Nullifier` | `()` | Never |
| `orchard_anchors` | `orchard::tree::Root` | `()` | Never | | `orchard_anchors` | `orchard::tree::Root` | `()` | Never |
| `orchard_incremental` | `BE32(height)` *?* | `orchard::tree::NoteCommitmentTree` | Delete | | `orchard_note_commitment_tree` | `block::Height` | `orchard::tree::NoteCommitmentTree` | Delete |
| `history_incremental` | `BE32(height)` | `zcash_history::Entry` | Delete | | `history_tree` | `block::Height` | `NonEmptyHistoryTree` | Delete |
| `tip_chain_value_pool`| `BE32(height)` | `ValueBalance<NonNegative>` | Delete | | `tip_chain_value_pool` | `()` | `ValueBalance` | Update |
Zcash structures are encoded using `ZcashSerialize`/`ZcashDeserialize`. Zcash structures are encoded using `ZcashSerialize`/`ZcashDeserialize`.
Other structures are encoded using `IntoDisk`/`FromDisk`. Other structures are encoded using `IntoDisk`/`FromDisk`.
**Note:** We do not store the cumulative work for the finalized chain, because the finalized work is equal for all non-finalized chains. So the additional non-finalized work can be used to calculate the relative chain order, and choose the best chain. Block and Transaction Data:
- `Height`: 32 bits, big-endian, unsigned
- `TransactionIndex`: 32 bits, big-endian, unsigned
- `TransactionLocation`: `Height \|\| TransactionIndex`
- `TransparentOutputIndex`: 32 bits, big-endian, unsigned
- `OutPoint`: `transaction::Hash \|\| TransparentOutputIndex`
- `IsFromCoinbase` : 8 bits, boolean, zero or one
- `Utxo`: `Height \|\| IsFromCoinbase \|\| Output`
We use big-endian encoding for keys, to allow database index prefix searches.
Amounts:
- `Amount`: 64 bits, little-endian, signed
- `ValueBalance`: `[Amount; 4]`
Derived Formats:
- `*::NoteCommitmentTree`: `bincode` using `serde`
- `NonEmptyHistoryTree`: `bincode` using `serde`, using `zcash_history`'s `serde` implementation
### Implementing consensus rules using rocksdb ### Implementing consensus rules using rocksdb
[rocksdb-consensus-rules]: #rocksdb-consensus-rules [rocksdb-consensus-rules]: #rocksdb-consensus-rules
@ -673,7 +690,7 @@ So they should not be used for consensus-critical checks.
the fact that we commit blocks in order means we're writing only to the end the fact that we commit blocks in order means we're writing only to the end
of the rocksdb column family, which may help save space. of the rocksdb column family, which may help save space.
- Transaction references are stored as a `(height, index)` pair referencing the - `TransactionLocation`s are stored as a `(height, index)` pair referencing the
height of the transaction's parent block and the transaction's index in that height of the transaction's parent block and the transaction's index in that
block. This would more traditionally be a `(hash, index)` pair, but because block. This would more traditionally be a `(hash, index)` pair, but because
we store blocks by height, storing the height saves one level of indirection. we store blocks by height, storing the height saves one level of indirection.
@ -689,7 +706,11 @@ So they should not be used for consensus-critical checks.
But we map those peak indexes to heights, to make testing and debugging easier. But we map those peak indexes to heights, to make testing and debugging easier.
- The value pools are only stored for the finalized tip. - The value pools are only stored for the finalized tip.
We index it by height to make testing and debugging easier.
- We do not store the cumulative work for the finalized chain,
because the finalized work is equal for all non-finalized chains.
So the additional non-finalized work can be used to calculate the relative chain order,
and choose the best chain.
## Committing finalized blocks ## Committing finalized blocks
@ -714,40 +735,17 @@ zebra-state service's responsibility) to commit finalized blocks in order.
The genesis block does not have a parent block. For genesis blocks, The genesis block does not have a parent block. For genesis blocks,
check that `block`'s parent hash is `null` (all zeroes) and its height is `0`. check that `block`'s parent hash is `null` (all zeroes) and its height is `0`.
2. Insert: 2. Insert the block and transaction data into the relevant column families.
- `(hash, height)` into `height_by_hash`;
- `(height, hash)` into `hash_by_height`;
- `(height, block)` into `block_by_height`.
3. If the block is a genesis block, skip any transaction updates. 3. If the block is a genesis block, skip any transaction updates.
(Due to a [bug in zcashd](https://github.com/ZcashFoundation/zebra/issues/559), (Due to a [bug in zcashd](https://github.com/ZcashFoundation/zebra/issues/559),
genesis block anchors and transactions are ignored during validation.) genesis block anchors and transactions are ignored during validation.)
4. Update the `sprout_anchors` and `sapling_anchors` trees with the Sprout and 4. Update the block anchors, history tree, and chain value pools.
Sapling anchors.
5. Iterate over the enumerated transactions in the block. For each transaction: 5. Iterate over the enumerated transactions in the block. For each transaction,
update the relevant column families.
1. Insert `(transaction_hash, BE32(block_height) || BE32(tx_index))` to
`tx_by_hash`;
2. For each `TransparentInput::PrevOut { outpoint, .. }` in the
transaction's `inputs()`, remove `outpoint` from `utxo_by_output`.
3. For each `output` in the transaction's `outputs()`, construct the
`outpoint` that identifies it, and insert `(outpoint, output)` into
`utxo_by_output`.
4. For each [`JoinSplit`] description in the transaction,
insert `(nullifiers[0],())` and `(nullifiers[1],())` into
`sprout_nullifiers`.
5. For each [`Spend`] description in the transaction, insert
`(nullifier,())` into `sapling_nullifiers`.
6. For each [`Action`] description in the transaction, insert
`(nullifier,())` into `orchard_nullifiers`.
**Note**: The Sprout and Sapling anchors are the roots of the Sprout and **Note**: The Sprout and Sapling anchors are the roots of the Sprout and
Sapling note commitment trees that have already been calculated for the last Sapling note commitment trees that have already been calculated for the last