* Remove unused dependencies
* Check for newly unused dependencies in CI
* Use the correct grep command
* Always show cargo machete output
* Ignore cargo machete exit status, use grep instead
* Use if instead of && and subshells
* Invert if logic
* Allow missing docs directly on derived error types
* Make Request::BestChainBlockHash redirect to the ReadStateService
* Re-write the checkpoint_sync documentation based on the latest consensus rules
* Expose the underlying iterator for CheckpointList
* Validate existing state block hashes at startup, but ignore the result
* Monitor state block hash checkpoint task in the start command
* Fix indentation
* Make logging consistent
* Explain the config needed for full security
* Tidy required checkpoints docs, expand other docs
* Add security and deprecation changelog entries
* Replace task handle vector with a struct
* Add a test that this consensus-critical code actually runs and finishes
* Make some state methods and types available in tests
* Add missing docs
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Clarify function docs, rename variables, and fix log typos
* Add a ReadState best chain clone method, but don't use it yet
* Use the new calculate_median_time_past() function in existing code
* Skip a state request if the lock time is a height
* Remove dummy arguments and extra blocks from median-time-past calculation
* Update tests to remove requests that are no longer sent
* Simplify getting the best chain
Co-authored-by: Arya <aryasolhi@gmail.com>
* Clarify some function docs
Co-authored-by: Arya <aryasolhi@gmail.com>
* assigns `next_median_time_past` value from if statement
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Make it clearer that Zebra only supports transparent funding streams
* Initial getblocksubsidy RPC types and method, without ZEC conversion
* Add a ZEC fixed-point format wrapper to Amount
* Format getblocksubsidy fields in Zec
* Add snapshot tests for getblocksubsidy RPC
* Re-order RPC output to match zcashd
* Switch to formatting Zec with f64, because the getblocksubsidy RPC requires JSON numbers
* Sort RPC responses in zcashd funding stream order
* Add getblocksubsidy snapshots
* Fix a doc link
* Move Zec JSON formatter from zebra-chain to zebra-rpc
* Remove Ord impl for Zec, it's just for formatting
* Use fully-qualified path for serde derives
* Fix a clippy warning for the doc link fix
* Fix RPC comments and an error message
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* bump v1.0.0-rc.4
* Add new mainnet and testnet checkpoints generated using zcashd
* make changes to changelog from review
* move item to new section
* change section name
---------
Co-authored-by: teor <teor@riseup.net>
* Implement the BestChainNextMedianTimePast state request
* Verify the lock times of mempool transactions
* Document that the mempool already handles lock time rejections correctly
* Fix existing tests
* Add new mempool lock time success and failure tests
* make `zebra-checkpoint` util work with zebra as the backend
* update snapshots
* update documentation
* applies suggestions from code review
* irefactor zebra-checkpoints to work with zebra using deserialization of the raw block
* fix imports and derives
* rename mode to backend
* remove old stuff
* fix docs
Co-authored-by: arya2 <aryasolhi@gmail.com>
* Format commitments and nonces as hex
* Format Sprout and Sapling nullifiers as hex when debugging
* Format Sprout commitments as hex when debugging
* Format redpallas keys as hex when debugging
* Update code that we're going to delete in the next PR anyway
* adds ValidateBlock request to state
* adds `Request` enum in block verifier
skips solution check for BlockProposal requests
calls CheckBlockValidity instead of Commit block for BlockProposal requests
* uses new Request in references to chain verifier
* adds getblocktemplate proposal mode response type
* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state
* Adds PR review revisions
* adds info log in CheckBlockProposalValidity
* Reverts replacement of match statement
* adds `GetBlockTemplate::capabilities` fn
* conditions calling checkpoint verifier on !request.is_proposal
* updates references to validate_and_commit_non_finalized
* adds snapshot test, updates test vectors
* adds `should_count_metrics` to NonFinalizedState
* Returns an error from chain verifier for block proposal requests below checkpoint height
adds feature flags
* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD
* adds back block::Request to zebra-consensus lib
* updates snapshots
* Removes unnecessary network arg
* skips req in tracing intstrument for read state
* Moves out block proposal validation to its own fn
* corrects `difficulty_threshold_is_valid` docs
adds/fixes some comments, adds TODOs
general cleanup from a self-review.
* Update zebra-state/src/service.rs
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Update zebra-rpc/src/methods/get_block_template_rpcs.rs
Co-authored-by: teor <teor@riseup.net>
* check best chain tip
* Update zebra-state/src/service.rs
Co-authored-by: teor <teor@riseup.net>
* Applies cleanup suggestions from code review
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Changelog with trivial entries
* Delete trivial entries
* Summarise known issues in README, but don't change the list yet
* Add block timeouts to known issues
* Update the release template to add missing version files
* Bump crate versions
* Add the required Rust version to the release checklist
* Update the Rust version requirement to 1.65, Zebra now uses `let ... else ...`
* Update checkpoints
* Add checkpoints to the CHANGELOG
* Breaking Rust compiler version change
* Clarify the latest stable supported rust version
* Make Zebra's getblocktemplate like zcashd's
* Update snapshots
* Add missing docs
* Fix typo
* Sort coinbase outputs by serialized script for zcashd
* Sort excluding the length byte, make transaction order always stable
* Update snapshots
* Explain that `zcashd` doesn't seem to have a fixed transaction order
* Remove verbose continuous_blockchain test logs
* Downgrade verbose zebra-network logs to debug
* Downgrade some state logs to debug during tests
* Mark were we would add always-on log filters, if we needed to
* Reduce the number of mempool property tests, to reduce logging
* Box errors to deal with large error warnings, add accessor methods for error properties
* Remove or explain some large enum variant lints
* Turn some tickets into TODOs
* Allow missing docs on single-field error enum variants
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Split the conventional fee check into its own method
This will be used for block production and relaying
* Move getblocktemplate transaction selection into a new zip317 module
* Add a block_production_fee_weight field to VerifiedUnminedTx
* Add a custom Zebra minimum transaction weight for block production
* Implement ZIP-317 transaction selection for block production
* Split weighted index setup into its own function
* Split picking a transaction into its own function
* updates comments
* adds check nullifier no dup fns for transactions
* Adds:
- check::anchors fn for tx iter
- TODO comments for unifying nullifiers and anchors checks
- new state request
Updates unknown anchor errors to accomodate tx-only check
Calls new state fn from transaction verifier
* updates check::anchors fns to use transactions
updates TransactionContextualValidity request to check sprout anchors
adds comment mentioning TransactionContextualValidity ignores UTXOs
* conditions new state req call on is_mempool
updates tests
* fix doc link / lint error
* checks for duplicate nullifiers with closures
* Update zebra-state/src/service/check/nullifier.rs
Co-authored-by: teor <teor@riseup.net>
* documents find_duplicate_nullifier params
moves if let statement into for loop
* renames new state req/res
* asserts correct response variant in tx verifier
* adds CheckBestChainTipShieldedSpends call in tx verifier to async checks
* re-adds tracing instrumentation to check::anchors fn
renames transaction_in_state to transaction_in_chain
* adds block/tx wrapper fns for anchors checks
* uses UnminedTx instead of transaction.hash()
deletes broken test
* updates new state req/res name
* updates tests and uses par_iter for anchors checks
* Updates check::anchors pub fn docs.
* Adds:
- comments / docs
- a TransactionError variant for ValidateContextError
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* moves downcast to From impl
rustfmt
* moves the ValidateContextError into an Arc
updates comments and naming
* leaves par_iter for another PR
* puts io::Error in an Arc
* updates anchors tests to call tx_anchors check
* updates tests to call tx_no_duplicates_in_chain
slightly improves formatting
* Update zebra-consensus/src/error.rs
Co-authored-by: teor <teor@riseup.net>
* moves Arc from HistoryError to ValidateContextError
Co-authored-by: teor <teor@riseup.net>
* bump zcash_proofs from 0.8.0 to 0.9.0
bump zcash_primitives from 0.8.0 to 0.9.0
* update deny.toml to skip zcash_primitives 0.8.1
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Uses BestChainUtxo to find utxos for mempool
* adds missing input test
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* update other instances of the renamed InputNotFound error
* adds read::unspent_utxo fn
* adds test for success case
Co-authored-by: teor <teor@riseup.net>
* Add a getblocktemplate-rpcs feature to zebra-chain, and fix missing feature deps
* Add a coinbase transaction creation stub
* Add coinbase creation to zebra-chain
* Add coinbase creation and miner subsidy to zebra-consensus
* Add the miner config to the GetBlockTemplateRpcImpl
* Generate the coinbase transaction in the getblocktemplate RPC
* Provide fake valid block heights to getblocktemplate RPC tests
* Update getblocktemplate RPC snapshots
* Add a getblocktemplate.coinbase_tx deserialized transaction snapshot test
* Update snapshots
* Return funding stream outputs in the same order every time
* Update snapshots
* Fix a script bytes bug
* Update snapshots
* adds submitblock rpc method
* re-orders imports
* replaces thread::yield_now with async yield_now
* Fix doc warnings and unused variable warnings, add missing docs
* Mark work_id as optional
* Use the same ChainVerifier for downloaded and submitted blocks
* Revert unused changes & minor cleanups
* Document currently-unreachable code
* updates tests and submit_block response for AlreadyVerified error
* Update zebra-rpc/src/methods/get_block_template_rpcs.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* changes names from BlockVerifier to ChainVerifier and block_verifier to chain_verifier to keep it consistent with naming in zebra-consensus
* move how to run the submit_block test example to acceptance.rs
* updates snapshot tests
* moved acceptance test to a separate file
* removes extra tower::ServiceBuilder::new(), updates docs
* updates vectors and snapshot tests, changes hex decoding error in submit_block method from server error to parse error
* hides errors module in zebra-rpc behind a feature flag and adds docs.
* Updates snapshot test, adds mod docs, moves HexData to its own mod, and removes the unrelated make_server_error refactor for now
* update submit block acceptance test mod doc
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add a legacy_sigop_count field to VerifiedUnminedTx
* Add conversions from Vec<VerifiedUnminedTx> to block header roots
* Add fee and sigops field to block template transactions
* Fix up mempool request names
* Increase existing snapshot test coverage
* Document a new method parameter
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Bump zcash_primitives, zcash_note_encryption and oechard deps at once
* Bump dependencies, and update `deny.toml`
* Upgrade to zcash_script 0.1.8
* Update Cargo.lock
* Use 3-part version numbers consistently
* Get address by serializing the Output, then using zcash_primitives to parse it (#5507)
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: teor <teor@riseup.net>
* Put `tor` behind its own `--cfg` flag
* Bump x25519-dalek from 1.2.0 to 2.0.0-pre.1
* Bump zcash_proofs to 0.8.0
* Update `deny.toml`
* Update `Cargo.lock`
* Fix missing doc link warning
* Mention a ticket that tracks the dep. conflict
* Update `.codespellrc`
* Disable the other tor dependency
* Clarify a comment
* Add Tor x25519-dalek issue to README
* Update Cargo.lock
Co-authored-by: teor <teor@riseup.net>
* Increment Zebra versions
* Initial draft changelog
* Add blog post to the release checklist
* Say "user testing"
Co-authored-by: teor <teor@riseup.net>
* Add a new block commit task and channels, that don't do anything yet
* Add last_block_hash_sent to the state service, to avoid database accesses
* Update last_block_hash_sent regardless of commit errors
* Rename a field to StateService.max_queued_finalized_height
* Commit finalized blocks to the state in a separate task
* Check for panics in the block write task
* Wait for the block commit task in tests, and check for errors
* Always run a proptest that sleeps once
* Add extra debugging to state shutdowns
* Work around a RocksDB shutdown bug
* Close the finalized block channel when we're finished with it
* Only reset state queue once per error
* Update some TODOs
* Add a module doc comment
* Drop channels and check for closed channels in the block commit task
* Close state channels and tasks on drop
* Remove some duplicate fields across StateService and ReadStateService
* Try tweaking the shutdown steps
* Update and clarify some comments
* Clarify another comment
* Don't try to cancel RocksDB background work on drop
* Fix up some comments
* Remove some duplicate code
* Remove redundant workarounds for shutdown issues
* Remode a redundant channel close in the block commit task
* Remove a mistaken `!force` shutdown condition
* Remove duplicate force-shutdown code and explain it better
* Improve RPC error logging
* Wait for chain tip updates in the RPC tests
* Wait 2 seconds for chain tip updates before skipping them
* Remove an unnecessary block_in_place()
* Fix some test error messages that were changed by earlier fixes
* Expand some comments, fix typos
Co-authored-by: Marek <mail@marek.onl>
* Actually drop children of failed blocks
* Explain why we drop descendants of failed blocks
* Clarify a comment
* Wait for chain tip updates in a failing test on macOS
* Clean duplicate finalized blocks when the non-finalized state activates
* Send an error when receiving a duplicate finalized block
* Update checkpoint block behaviour, document its consensus rule
* Wait for chain tip changes in inbound_block_height_lookahead_limit test
* Wait for the genesis block to commit in the fake peer set mempool tests
* Disable unreliable mempool verification check in the send transaction test
* Appease rustfmt
* Use clear_finalized_block_queue() everywhere that blocks are dropped
* Document how Finalized and NonFinalized clones are different
* sends non-finalized blocks to the block write task
* passes ZebraDb to commit_new_chain, commit_block, and no_duplicates_in_finalized_chain instead of FinalizedState
* Update zebra-state/src/service/write.rs
Co-authored-by: teor <teor@riseup.net>
* updates comments, renames send_process_queued, other minor cleanup
* update assert_block_can_be_validated comment
* removes `mem` field from StateService
* removes `disk` field from StateService and updates block_iter to use `ZebraDb` instead of the finalized state
* updates tests that use the disk to use read_service.db instead
* moves best_tip to a read fn and returns finalized & non-finalized states from setup instead of the state service
* changes `contextual_validity` to get the network from the finalized_state instead of another param
* swaps out StateService with FinalizedState and NonFinalizedState in tests
* adds NotReadyToBeCommitted error and returns it from validate_and_commit when a blocks parent hash is not in any chain
* removes NonFinalizedWriteCmd and calls, moves update_latest_channels above rsp_tx.send
* makes parent_errors_map an indexmap
* clears non-finalized block queue when the receiver is dropped and when the StateService is being dropped
* sends non-finalized blocks to the block write task
* passes ZebraDb to commit_new_chain, commit_block, and no_duplicates_in_finalized_chain instead of FinalizedState
* updates comments, renames send_process_queued, other minor cleanup
* Update zebra-state/src/service/write.rs
Co-authored-by: teor <teor@riseup.net>
* update assert_block_can_be_validated comment
* removes `mem` field from StateService
* removes `disk` field from StateService and updates block_iter to use `ZebraDb` instead of the finalized state
* updates tests that use the disk to use read_service.db instead
* moves best_tip to a read fn and returns finalized & non-finalized states from setup instead of the state service
* changes `contextual_validity` to get the network from the finalized_state instead of another param
* swaps out StateService with FinalizedState and NonFinalizedState in tests
* adds NotReadyToBeCommitted error and returns it from validate_and_commit when a blocks parent hash is not in any chain
* removes NonFinalizedWriteCmd and calls, moves update_latest_channels above rsp_tx.send
* makes parent_errors_map an indexmap
* clears non-finalized block queue when the receiver is dropped and when the StateService is being dropped
* removes duplicate field definitions on StateService that were a result of a bad merge
* update NotReadyToBeCommitted error message
* Appear rustfmt
* Fix doc links
* Rename a function to initial_contextual_validity()
* Do error tasks on Err, and success tasks on Ok
* Simplify parent_error_map truncation
* Rewrite best_tip() to use tip()
* Rename latest_mem() to latest_non_finalized_state()
```sh
fastmod latest_mem latest_non_finalized_state zebra*
cargo fmt --all
```
* Simplify latest_non_finalized_state() using a new WatchReceiver API
* Expand some error messages
* Send the result after updating the channels, and document why
* wait for chain_tip_update before cancelling download in mempool_cancel_mined
* adds `sent_non_finalized_block_hashes` field to StateService
* adds batched sent_hash insertions and checks sent hashes in queue_and_commit_non_finalized before adding a block to the queue
* check that the `curr_buf` in SentHashes is not empty before pushing it to the `sent_bufs`
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Fix rustfmt
* Check for finalized block heights using zs_contains()
* adds known_utxos field to SentHashes
* updates comment on SentHashes.add method
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* return early when there's a duplicate hash in QueuedBlocks.queue instead of panicking
* Make finalized UTXOs near the final checkpoint available for full block verification
* Replace a checkpoint height literal with the actual config
* Update mainnet and testnet checkpoints - 7 October 2022
* Fix some state service init arguments
* Allow more lookahead in the downloader, but less lookahead in the syncer
* Add the latest config to the tests, and fix the latest config check
* Increase the number of finalized blocks checked for non-finalized block UTXO spends
* fix(log): reduce verbose logs for block commits (#5348)
* Remove some verbose block write channel logs
* Only warn about tracing endpoint if the address is actually set
* Use CloneError instead of formatting a non-cloneable error
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Increase block verify timeout
* Work around a known block timeout bug by using a shorter timeout
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* add grpc mempool test research
* add a config flag for mempool injection of transactions in test
* Only copy the inner state directory in the send transactions test
* Preload Zcash parameters in some transaction verification tests
* Add a block and transaction Hash method to convert from display order bytes
* Update test coverage docs
* Add debugging output for mempool transaction verification
* Test fetching sent mempool transactions using gRPC
* Add extra log checks to the send transaction test
* Wait for zebrad mempool activation before running gRPC tests
* Update send transaction test for lightwalletd not returning mempool transactions
* Check zebrad logs instead of disabled lightwalletd gRPCs
* Add a debug option that makes RPCs pretend the sync is finished
* Remove an unused debug option
* Remove unused test code and downgrade some logs
* Fix test log checks
* Fix some rustdoc warnings
* Fix a compilation error due to new function arguments
* Make zebrad sync timeouts consistent and remove outdated code
* Document how to increase temporary directory space for tests
* Stop checking for a log that doesn't always happen
* Remove some commented-out code
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Update a comment about run time
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add new config to new tests from the `main` branch
* Add transactions to the list, rather than replacing the list with each new block
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
dependabot[bot]
teor
Marek
Alfredo Garcia
Conrado Gouvea
Arya
Deirdre Connolly