Commit Graph

27 Commits

Author SHA1 Message Date
teor 941be2965c
Replace a chain length assertion with a NotReadyToBeCommitted error (#7072) 2023-06-27 06:50:35 +00:00
Alfredo Garcia eb07bb31d6
rename(state): Rename state verifiers and related code (#6762)
* rename verifiers

* rename `PreparedBlock` to `SemanticallyVerifiedBlock`

* rename `CommitBlock` to `SemanticallyVerifiedBlock`

* rename `FinalizedBlock` to `CheckpointVerifiedBlock`

* rename `CommitFinalizedBlock` to `CommitCheckpointVerifiedBlock`

* rename `FinalizedWithTrees` to `ContextuallyVerifiedBlockWithTrees`

* rename `ContextuallyValidBlock` to `ContextuallyVerifiedBlock`

* change some `finalized` variables or function arguments to `checkpoint_verified`

* fix docs

* document the difference between `CheckpointVerifiedBlock` and `ContextuallyVerifiedBlock`

* fix doc links

* apply suggestions to request

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to service

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to finalized_state.rs and write.rs

Co-authored-by: Marek <mail@marek.onl>

* fmt

* change some more variable names

* change a few missing generics

* fix checkpoint log issue

* rename more `prepared` vars `semantically_verified`

* fix test regex

* fix test regex 2

---------

Co-authored-by: Marek <mail@marek.onl>
2023-06-01 12:29:03 +00:00
Arya 3779eb19a6
Tests error conversions (#6665) 2023-05-16 21:06:46 +00:00
teor f858aab1d9
improve CommitBlockError message: include underlying ValidateContextError message (#6251)
Co-authored-by: dimxy <dimxy@komodoplatform.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-03-02 06:30:01 +00:00
Arya eb0a2ef581
change(mempool): Contextually validates mempool transactions in best chain (#5716)
* updates comments

* adds check nullifier no dup fns for transactions

* Adds:
- check::anchors fn for tx iter
- TODO comments for unifying nullifiers and anchors checks
- new state request

Updates unknown anchor errors to accomodate tx-only check

Calls new state fn from transaction verifier

* updates check::anchors fns to use transactions

updates TransactionContextualValidity request to check sprout anchors

adds comment mentioning TransactionContextualValidity ignores UTXOs

* conditions new state req call on is_mempool

updates tests

* fix doc link / lint error

* checks for duplicate nullifiers with closures

* Update zebra-state/src/service/check/nullifier.rs

Co-authored-by: teor <teor@riseup.net>

* documents find_duplicate_nullifier params

moves if let statement into for loop

* renames new state req/res

* asserts correct response variant in tx verifier

* adds CheckBestChainTipShieldedSpends call in tx verifier to async checks

* re-adds tracing instrumentation to check::anchors fn

renames transaction_in_state to transaction_in_chain

* adds block/tx wrapper fns for anchors checks

* uses UnminedTx instead of transaction.hash()

deletes broken test

* updates new state req/res name

* updates tests and uses par_iter for anchors checks

* Updates check::anchors pub fn docs.

* Adds:
- comments / docs
- a TransactionError variant for ValidateContextError

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* moves downcast to From impl

rustfmt

* moves the ValidateContextError into an Arc

updates comments and naming

* leaves par_iter for another PR

* puts io::Error in an Arc

* updates anchors tests to call tx_anchors check

* updates tests to call tx_no_duplicates_in_chain

slightly improves formatting

* Update zebra-consensus/src/error.rs

Co-authored-by: teor <teor@riseup.net>

* moves Arc from HistoryError to ValidateContextError

Co-authored-by: teor <teor@riseup.net>
2022-11-30 04:40:15 +00:00
Arya a28350e742
change(state): Write non-finalized blocks to the state in a separate thread, to avoid network and RPC hangs (#5257)
* Add a new block commit task and channels, that don't do anything yet

* Add last_block_hash_sent to the state service, to avoid database accesses

* Update last_block_hash_sent regardless of commit errors

* Rename a field to StateService.max_queued_finalized_height

* Commit finalized blocks to the state in a separate task

* Check for panics in the block write task

* Wait for the block commit task in tests, and check for errors

* Always run a proptest that sleeps once

* Add extra debugging to state shutdowns

* Work around a RocksDB shutdown bug

* Close the finalized block channel when we're finished with it

* Only reset state queue once per error

* Update some TODOs

* Add a module doc comment

* Drop channels and check for closed channels in the block commit task

* Close state channels and tasks on drop

* Remove some duplicate fields across StateService and ReadStateService

* Try tweaking the shutdown steps

* Update and clarify some comments

* Clarify another comment

* Don't try to cancel RocksDB background work on drop

* Fix up some comments

* Remove some duplicate code

* Remove redundant workarounds for shutdown issues

* Remode a redundant channel close in the block commit task

* Remove a mistaken `!force` shutdown condition

* Remove duplicate force-shutdown code and explain it better

* Improve RPC error logging

* Wait for chain tip updates in the RPC tests

* Wait 2 seconds for chain tip updates before skipping them

* Remove an unnecessary block_in_place()

* Fix some test error messages that were changed by earlier fixes

* Expand some comments, fix typos

Co-authored-by: Marek <mail@marek.onl>

* Actually drop children of failed blocks

* Explain why we drop descendants of failed blocks

* Clarify a comment

* Wait for chain tip updates in a failing test on macOS

* Clean duplicate finalized blocks when the non-finalized state activates

* Send an error when receiving a duplicate finalized block

* Update checkpoint block behaviour, document its consensus rule

* Wait for chain tip changes in inbound_block_height_lookahead_limit test

* Wait for the genesis block to commit in the fake peer set mempool tests

* Disable unreliable mempool verification check in the send transaction test

* Appease rustfmt

* Use clear_finalized_block_queue() everywhere that blocks are dropped

* Document how Finalized and NonFinalized clones are different

* sends non-finalized blocks to the block write task

* passes ZebraDb to commit_new_chain, commit_block, and no_duplicates_in_finalized_chain instead of FinalizedState

* Update zebra-state/src/service/write.rs

Co-authored-by: teor <teor@riseup.net>

* updates comments, renames send_process_queued, other minor cleanup

* update assert_block_can_be_validated comment

* removes `mem` field from StateService

* removes `disk` field from StateService and updates block_iter to use `ZebraDb` instead of the finalized state

* updates tests that use the disk to use read_service.db instead

* moves best_tip to a read fn and returns finalized & non-finalized states from setup instead of the state service

* changes `contextual_validity` to get the network from the finalized_state instead of another param

* swaps out StateService with FinalizedState and NonFinalizedState in tests

* adds NotReadyToBeCommitted error and returns it from validate_and_commit when a blocks parent hash is not in any chain

* removes NonFinalizedWriteCmd and calls, moves update_latest_channels above rsp_tx.send

* makes parent_errors_map an indexmap

* clears non-finalized block queue when the receiver is dropped and when the StateService is being dropped

* sends non-finalized blocks to the block write task

* passes ZebraDb to commit_new_chain, commit_block, and no_duplicates_in_finalized_chain instead of FinalizedState

* updates comments, renames send_process_queued, other minor cleanup

* Update zebra-state/src/service/write.rs

Co-authored-by: teor <teor@riseup.net>

* update assert_block_can_be_validated comment

* removes `mem` field from StateService

* removes `disk` field from StateService and updates block_iter to use `ZebraDb` instead of the finalized state

* updates tests that use the disk to use read_service.db instead

* moves best_tip to a read fn and returns finalized & non-finalized states from setup instead of the state service

* changes `contextual_validity` to get the network from the finalized_state instead of another param

* swaps out StateService with FinalizedState and NonFinalizedState in tests

* adds NotReadyToBeCommitted error and returns it from validate_and_commit when a blocks parent hash is not in any chain

* removes NonFinalizedWriteCmd and calls, moves update_latest_channels above rsp_tx.send

* makes parent_errors_map an indexmap

* clears non-finalized block queue when the receiver is dropped and when the StateService is being dropped

* removes duplicate field definitions on StateService that were a result of a bad merge

* update NotReadyToBeCommitted error message

* Appear rustfmt

* Fix doc links

* Rename a function to initial_contextual_validity()

* Do error tasks on Err, and success tasks on Ok

* Simplify parent_error_map truncation

* Rewrite best_tip() to use tip()

* Rename latest_mem() to latest_non_finalized_state()

```sh
fastmod latest_mem latest_non_finalized_state zebra*
cargo fmt --all
```

* Simplify latest_non_finalized_state() using a new WatchReceiver API

* Expand some error messages

* Send the result after updating the channels, and document why

* wait for chain_tip_update before cancelling download in mempool_cancel_mined

* adds `sent_non_finalized_block_hashes` field to StateService

* adds batched sent_hash insertions and checks sent hashes in queue_and_commit_non_finalized before adding a block to the queue

* check that the `curr_buf` in SentHashes is not empty before pushing it to the `sent_bufs`

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Fix rustfmt

* Check for finalized block heights using zs_contains()

* adds known_utxos field to SentHashes

* updates comment on SentHashes.add method

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* return early when there's a duplicate hash in QueuedBlocks.queue instead of panicking

* Make finalized UTXOs near the final checkpoint available for full block verification

* Replace a checkpoint height literal with the actual config

* Update mainnet and testnet checkpoints - 7 October 2022

* Fix some state service init arguments

* Allow more lookahead in the downloader, but less lookahead in the syncer

* Add the latest config to the tests, and fix the latest config check

* Increase the number of finalized blocks checked for non-finalized block UTXO spends

* fix(log): reduce verbose logs for block commits (#5348)

* Remove some verbose block write channel logs

* Only warn about tracing endpoint if the address is actually set

* Use CloneError instead of formatting a non-cloneable error

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Increase block verify timeout

* Work around a known block timeout bug by using a shorter timeout

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-10-11 19:25:45 +00:00
teor 6ad445eb97
1. fix(perf): Run CPU-intensive state updates in parallel rayon threads (#4802)
* Split disk reads from CPU-heavy Sprout interstitial tree cryptography

* Improve anchor validation debugging and error messages

* Work around a test data bug, and save some CPU

* Remove redundant checks for empty shielded data

* Skip generating unused interstitial treestates

* Do disk fetches and quick checks, then CPU-heavy cryptography

* Wrap HistoryTree in an Arc in the state

* Run CPU-intensive chain validation and updates in parallel rayon threads

* Refactor to prepare for parallel tree root calculations

* Run finalized state note commitment tree root updates in parallel rayon threads

* Update finalized state note commitment trees using parallel rayon threads

* Fix a comment typo and add a TODO

* Split sprout treestate fetch into its own function

* Move parallel note commitment trees to zebra-chain

* Re-calculate the tree roots in the same parallel batches

* Do non-finalized note commitment tree updates in parallel threads

* Update comments about note commitment tree rebuilds

* Do post-fork tree updates in parallel threads

* Add a TODO for parallel tree updates in tests

* Fix broken intra-doc links

* Clarify documentation for sprout treestates

* Sort Cargo.toml dependencies
2022-07-22 12:19:11 -04:00
Deirdre Connolly e6ffe374d4
Validate sapling, orchard anchors (#3084)
* Add Transaction::sprout_joinsplits()

* Add Anchor variants to ValidateContextError

* Make Chain anchor collections pub(crate)

* tracing::instrument several methods in state

* Add contains_*_anchors methods to FinalizedState

* Add check::anchors module and function

* Verify that anchors_refer_to_earlier_treestates in when updating chains in non-finalized state

* Update zebra-state/src/service/check/anchors.rs

Co-authored-by: teor <teor@riseup.net>

* Add anchors() to sapling::ShieldedData

* Add sapling_anchors() to Transaction

* Use Transaction::sapling_anchors() in the anchors_refer_to_earlier_treestates() check

* Whoops, itertools

* Add a comment for improvement

Co-authored-by: teor <teor@riseup.net>

* Add & use a cfg(test) method on FinalizedState to prep test state with anchors to allow other tests to pass contextual checks

* Allow test nullifier checks to pass by populating anchor sets, allowing test anchor checks to pass

* Add mainnet block 419202 and its sapling note commitment tree root to test vectors

* Test sapling anchor verification using the first few Sapling blocks data

* Correct comment

* assert_eq instead of assert(bool)

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Update zebra-state/src/service/non_finalized_state.rs

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-11-30 16:05:35 +00:00
Marek 3c9ad89018
Add Sprout anchors to `zebra-state` (#3100)
* Add Sprout anchors to the state

* Update zebra-state/src/service/non_finalized_state/chain.rs

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Return new types of note commitments from Sprout transactions

* Refactor the tests

* Refactor some comments

Co-authored-by: teor <teor@riseup.net>

* Increment `DATABASE_FORMAT_VERSION`

* Update `test.yml` with the new image name

* Refactor the `version = 5` transaction description

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Update comment

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-11-30 11:05:58 +01:00
teor d7eb01d7f0
Add value pool/balances to non-finalized state (#2656)
* add value balances to non finalized state

* fix 2 tests

* fix remaining constrain issues in tests

* extend value pool test to non finalized

* WIP: fix tests after adding value pools to non-finalized state (#2647)

* Update Chain::eq_internal_state with Chain.value_balance

Also increase the number of cases in its tests,
because they didn't detect this bug.

* Calculate the chain value pool change before `Chain::push`

Code
- store the chain value pool change in `ContextuallyValidBlock`
- convert `PreparedBlock` to `ContextuallyValidBlock` using `with_block_and_spent_utxos`
  (rather than `from` or `into`)
- replace `block_utxos` with `new_outputs` in `PreparedBlock`
- replace `block_utxos` with `chain_value_pool_change` in `ContextuallyValidBlock`

Tests
- create test methods for `PreparedBlock` and `ContextuallyValidBlock`
- use `test_with_zero_chain_pool_change` or `test_with_zero_spent_utxos`
  to make tests pass

* fix conflicts

* build `set_current_value_pool()` only for tests

* remove redundant cfgs

* change cfg of set_current_value_pool()

* Clarify some chain field documentation

* Fix bugs in the non-finalized chain value pool calculations

1. Only revert the chain value pool balances when the tip is popped.
   Don't modify them when the root is finalized.

2. Only update or revert the chain value pool balances once per block.
   (Previously, the block changes were multiplied by the number of *transactions*.)

And make corresponding changes to method names and documentation.

* Add extra proptests to try to identify value balance failures

* Simplify some transaction generation code

* Add extra debugging info to value balance errors

* Actually update non-finalized chain value pools in `UpdateWith`

Previously, we were dropping the updated value pools in the `Ok` result.
So the initial (finalized) chain value pool balances were never modified.

* Rename and document value balance add methods

The names and documentation of these methods were confusing.

* Create genesis-based proptests that check chain value pools

* Increase coverage for some test vectors

* Test each chain value balance calculation for blocks 0-10

* Make continuous blockchain test errors easier to debug

* Test the exact transparent pool values for the first few blocks

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-25 10:57:07 -03:00
Conrado Gouvea 5c5abf6171
ZIP-221 and ZIP-244 commitment validation in non-finalized state (#2609)
* Add validation of ZIP-221 and ZIP-244 commitments

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2021-08-17 11:49:27 -03:00
Conrado Gouvea 94175c6955
Add ZIP-221 history tree to non-finalized state (#2583)
* Refactor HistoryTree into NonEmptyHistoryTree and HistoryTree

* HistoryTree: use Deref instead of AsRef; remove unneeded PartialEq

* ZIP-221: Validate chain history commitments in the non-finalized state (#2301)

* sketch of implementation

* refined implementation; still incomplete

* update librustzcash, change zcash_history to work with it

* simplified code per review; renamed MMR to HistoryTree

* expand HistoryTree implementation

* handle and propagate errors

* simplify check.rs tracing

* add suggested TODO

* add HistoryTree::prune

* fix bug in pruning

* fix compilation of tests; still need to make them pass

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* improvements from code review

* improve check.rs comments and variable names

* fix HistoryTree which should use BTreeMap and not HashMap; fix non_finalized_state prop tests

* fix finalized_state proptest

* fix non_finalized_state tests by setting the correct commitments

* renamed mmr.rs to history_tree.rs

* Add HistoryTree struct

* expand non_finalized_state protest

* fix typo

* Add HistoryTree struct

* Update zebra-chain/src/primitives/zcash_history.rs

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* fix formatting

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* history_tree.rs: fixes from code review

* fixes to work with updated HistoryTree

* Improvements from code review

* Add Debug implementations to allow comparing Chains with proptest_assert_eq

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Improvements from code review

* Restore blocks returned by PreparedChain since other tests broken; adjust tests with history trees

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-08-11 10:42:40 -03:00
teor f09f2a9022
Check remaining transaction value & make value balance signs match the spec (#2566)
* Make Amount arithmetic more generic

To modify generated amounts, we need some extra operations on `Amount`.

We also need to extend existing operations to both `NonNegative` and
`NegativeAllowed` amounts.

* Add a constrain method for ValueBalance

* Derive Eq for ValueBalance

* impl Neg for ValueBalance

* Make some Amount arithmetic expectations explicit

* Explain why we use i128 for multiplication

And expand the overflow error details.

* Expand Amount::sum error details

* Make amount::Error field order consistent

* Rename an amount::Error variant to Constraint, so it's clearer

* Add specific pool variants to ValueBalanceError

* Update coinbase remaining value consensus rule comment

This consensus rule was updated recently to include coinbase transactions,
but Zebra doesn't check block subsidy or miner fees yet.

* Add test methods for modifying transparent values and shielded value balances

* Temporarily set values and value balances to zero in proptests

In both generated chains and proptests that construct their own transactions.

Using zero values reduces value calculation and value check test coverage.
A future change will use non-zero values, and fix them so the check passes.

* Add extra fields to remaining transaction value errors

* Swap the transparent value balance sign to match shielded value balances

This makes the signs of all the chain value pools consistent.

* Use a NonNegative constraint for transparent values

This fix:
* makes the type signature match the consensus rules
* avoids having to write code to handle negative values

* Allocate total generated transaction input value to outputs

If there isn't enough input value for an output, set it to zero.

Temporarily reduce all generated values to avoid overflow.
(We'll remove this workaround when we calculate chain value balances.)

* Consistently use ValueBalanceError for ValueBalances

* Make the value balance signs match the spec

And rename and document methods so their signs are clearer.

* Convert amount::Errors to specific pool ValueBalanceErrors

* Move some error changes to the next PR

* Add extra info to remaining transaction value errors (#2585)

* Distinguish between overflow and negative remaining transaction value errors

And make some error types cloneable.

* Add methods for updating chain value pools (#2586)

* Move amount::test to amount::tests:vectors

* Make ValueBalance traits more consistent with Amount

- implement Add and Sub variants with Result and Assign
- derive Hash

* Clarify some comments and expects

* Create ValueBalance update methods for blocks and transactions

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-08-09 14:22:26 -03:00
Deirdre Connolly e719c46b1b
Track anchors and note commitment trees in zebra-state (#2458)
* Tidy chain Cargo.toml

* Organize imports

* Add method to get note commitments from all Actions in Orchard shielded data

* Add method to get note commitments from all JoinSplits in Sprout JoinSplitData

* Add Request and Response variants for awaiting anchors

* Add anchors and note commitment trees to finalized state db

* Add (From|Into)Disk impls for tree::Roots and stubs for NoteCommitmentTrees

* Track anchors and note commitment trees in Chain

Append note commitments to their trees when doing update_chain_state_with,
then use the resulting Sapling and Orchard roots to pass to history_tree, and add
new roots to the anchor sets.

* Handle errors when appending to note commitment trees

* Add comments explaining why note commitment are not removed from the tree in revert_chain_state_with

* Implementing note commitments in finalized state

* Finish serialization of Orchard tree; remove old tree when updating finalize state

* Add serialization and finalized state updates for Sprout and Sapling trees

* Partially handle trees in non-finalized state. Use Option for trees in Chain

* Rebuild trees when forking; change finalized state tree getters to not require height

* Pass empty trees to tests; use empty trees by default in Chain

* Also rebuild anchor sets when forking

* Use empty tree as default in finalized state tree getters (for now)

* Use HashMultiSet for anchors in order to make pop_root() work correctly

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS even more

* Apply suggestions from code review

* Add comments about order of note commitments and related methods/fields

* Don't use Option for trees

* Set DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES=1 and restore MAX_PARTIAL_CHAIN_BLOCKS

* Remove unneeded anchor set rebuilding in fork()

* Improve proptest formatting

* Add missing comparisons to eq_internal_state

* Renamed sprout::tree::NoteCommitmentTree::hash() to root()

* Improve comments

* Add asserts, add issues to TODOs

* Remove impl Default for Chain since it was only used by tests

* Improve documentation and assertions; add tree serialization tests

* Remove Sprout code, which will be moved to another branch

* Add todo! in Sprout tree append()

* Remove stub request, response *Anchor* handling for now

* Add test for validating Sapling note commitment tree using test blocks

* Increase database version (new columns added for note commitment trees and anchors)

* Update test to make sure the order of sapling_note_commitments() is being tested

* Improve comments and structure of the test

* Improve variable names again

* Rustfmt

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Conrado P. L. Gouvea <conradoplg@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-07-29 09:37:18 -04:00
teor 3d792f7195
Validate spends of transparent coinbase outputs (#2525)
* Validate transparent coinbase output maturity and shielding

- Add a CoinbaseSpendRestriction enum and Transaction method
- Validate transparent coinbase spends in non-finalized chains

* Don't use genesis created UTXOs for spends in generated block chains

* Refactor out a new_transaction_ordered_outputs function

* Add Transaction::outputs_mut for tests

* Generate valid transparent spends in arbitrary block chains

* When generating blocks, fixup the block contents, then the block hash

* Test that generated chains contain at least one transparent spend

* Make generated chains long enough for reliable tests

* Add transparent and shielded input and output methods to Transaction

* Split chain generation into 3 functions

* Test that unshielded and immature transparent coinbase spends fail

* Comment punctuation

* Clarify a comment

* Clarify probability calculation

* Test that shielded mature coinbase output spends succeed
2021-07-29 14:23:50 +10:00
Alfredo Garcia ee3c992ca6
Calculate the remaining value in the transparent transaction value pool (#2486)
* add value_balance methods to transparent and shielded

* add value_balance() to transaction

* check the remaining value consensus rule

* change error name

* fix doc and nitpick

* refactor value_balance() method for joinsplit

* changes to value_balance() of Inputs

* implement joinsplits() method(not working)

* remove created methods

* remove special case

* change return error in utilities

* move utils functions to transaction methods

* fix the docs

* simplify some code

* add constrains explicitly

* remove turbofish

* refactor some transaction methods

* fix value balance signs, add docs

* simplify some code

* avoid panic in consensus check

* add missing doc

* move remaining value balance check to the state

* make changes from the last review

Co-authored-by: teor <teor@riseup.net>
2021-07-29 13:49:36 +10:00
teor e6e03247ba
Reject UTXO double spends (#2511)
* Reject transparent output double-spends

Check that transparent spends use unspent outputs from:
* earlier transaction in the same block,
* earlier blocks in the parent non-finalized chain, or
* the finalized state.

* Fixup UTXOs in proptests

* Add a comment

* Clarify a consensus rule implementation

* Fix an incorrect comment

* Fix an incorrect error message

* Clarify a comment

* Document `unspent_utxos`

* Simplify the UTXO check

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Further simplify and fix the UTXO check

- split each error case into a separate check
- combine `contains` and `insert`
- add a missing check against the non-finalized unspent UTXOs
- rename arguments and edit error strings for clarity

* Share test methods between check test modules

* Make some chain fields available to tests

* Make error field names consistent with transparent::Input

* WIP: Add tests for UTXO double-spends

- accept output and spend in the same block
- accept output and spend in a later block
- reject output and double-spend all in the same block
- reject output then double-spend in a later block
- reject output, spend, then double-spend all in different blocks

* Use Extend rather than multiple pushes

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Use Extend for more pushes

* Limit the number of proptest cases, to speed up tests

* Test rejection of UTXOs that were never in the chain

* Test rejection of spends of later transactions in the same block

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-07-22 23:40:15 +00:00
teor 544be14c70
Reject duplicate Sapling and Orchard nullifiers (#2497)
* Add sapling and orchard duplicate nullifier errors

* Reject duplicate finalized sapling and orchard nullifiers

Reject duplicate sapling and orchard nullifiers in a new block,
when the block is added to a non-finalized chain,
and the duplicate nullifier is already in the finalized state.

* Reject duplicate non-finalized sapling and orchard nullifiers

Reject duplicate sapling and orchard nullifiers in a new block,
when the block is added to a non-finalized chain,
and the duplicate nullifier is in:
* the same shielded data,
* the same transaction,
* the same block, or
* an earlier block in the non-finalized chain.

* Refactor sprout nullifier tests to remove common code

* Add sapling nullifier tests

Test that the state rejects duplicate sapling nullifiers in a new block,
when the block is added to a non-finalized chain,
and the duplicate nullifier is in:
* the same shielded data,
* the same transaction,
* the same block,
* an earlier block in the non-finalized chain, or
* the finalized state.

* Add orchard nullifier tests

Test that the state rejects duplicate orchard nullifiers in a new block,
when the block is added to a non-finalized chain,
and the duplicate nullifier is in:
* the same shielded data,
* the same transaction,
* the same block,
* an earlier block in the non-finalized chain, or
* the finalized state.

* Check for specific nullifiers in the state in tests

* Replace slices with vectors in arguments

* Remove redundant code and variables

* Simplify sapling TransferData tests

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Remove an extra :

* Remove redundant vec!

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-07-19 21:39:05 -03:00
teor 0f5eced5c7
Reject duplicate sprout nullifiers in the state (#2477)
* Reject duplicate sprout nullifiers in the state

* Improve docs and error messages

* Clarify "must be present" assert logs

* Move nullifier checks to their own module

Also:
* make non-finalized nullifier checks and errors generic over
  sprout, sapling, and orchard
* create and update module and function documentation

* Fix a block type name in docs

* Move state assertions or skip them during tests

These changes enable state testing, while still asserting in production.

* Add sprout duplicate nullifier tests

* Improve comments

* Set value balance to 0 to pass future chain value pool checks

* Test finalized state in sprout nullifier accept test

* Replace assert with expect

* Improve assertion messages
2021-07-14 09:06:43 -03:00
teor 2aaf1d245f
Revert "ZIP-221: Validate chain history commitments in the non-finalized state (#2301)" (#2464)
This reverts commit 91b1fcb37b.
2021-07-08 09:25:49 -03:00
Conrado Gouvea 91b1fcb37b
ZIP-221: Validate chain history commitments in the non-finalized state (#2301)
* sketch of implementation

* refined implementation; still incomplete

* update librustzcash, change zcash_history to work with it

* simplified code per review; renamed MMR to HistoryTree

* expand HistoryTree implementation

* handle and propagate errors

* simplify check.rs tracing

* add suggested TODO

* add HistoryTree::prune

* fix bug in pruning

* fix compilation of tests; still need to make them pass

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* improvements from code review

* improve check.rs comments and variable names

* fix HistoryTree which should use BTreeMap and not HashMap; fix non_finalized_state prop tests

* fix finalized_state proptest

* fix non_finalized_state tests by setting the correct commitments

* renamed mmr.rs to history_tree.rs

* Add HistoryTree struct

* expand non_finalized_state protest

* fix typo

* Add HistoryTree struct

* Update zebra-chain/src/primitives/zcash_history.rs

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* fix formatting

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* history_tree.rs: fixes from code review

* fixes to work with updated HistoryTree

* Improvements from code review

* Add Debug implementations to allow comparing Chains with proptest_assert_eq

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-07-07 10:29:08 +10:00
teor 0e1f330720 Use error message derives in zebra-state
Rather than displaydoc.
2020-12-04 10:44:36 +10:00
teor 207ded6889 Add error context for contextual validation 2020-12-04 10:44:36 +10:00
teor 23e07a94cf Implement the block header time consensus rules 2020-12-04 10:44:36 +10:00
teor 678e6ad090 Implement difficulty_threshold_is_valid 2020-12-01 07:27:30 +10:00
Jane Lusby 8ba9d0114b
Add consensus critical check for sequential heights (#1291)
* Add consensus critical check for sequential heights
* document the check module
* Add unit tests for consensus checks
2020-11-13 14:26:16 +10:00
Jane Lusby b3634fa3e7
Make the state service use broadcast channels (#1137)
And refactor error handling
2020-10-09 18:37:24 +10:00