* updates book/../state-db-upgrades.md
* updates zebra-consensus with the changes from PR #9432 and minor test changes from #9295
* updates zebra-chain with the changes from PR #9432
* updates imports in zebra-rpc
* Merges and applies changes to zebra-state from PRs #9295 and #9432
* adds a TODO.
* minor update to zebra-rpc to use the new ReadResponse::AddressBalance type
* renames add_block_info db upgrade
* moves migration from `add_balance_received` to `block_info_and_address_received`
* replaces `write` with `write_batch`
* update disk format snapshots
* combines partial_transparent_balance_change() and partial_transparent_received_change()
* fixes typo
* Replaces redundant code with a call to `construct_column_families()`
* refactors `AddressBalanceLocation` and `AddressBalanceLocationChange` into a newtypes around a common inner type.
* fixes lints and doc compilation issue
* updates snapshots
* bumps the db format version to the next major version (27.0.0) instead of the next minor version (26.1.0)
* fixes clippy lint, minor cleanups
* fixes lint
* applies suggestions from code review
* fixes a mistake in `is_reusable_major_upgrade` where v26.0.0 is not a restorable db version
* Fixes an issue around marking an upgraded db format in the disk version file before completing migrations when reusing a previous major db format and restarting Zebrad before the migration is complete.
* Apply suggestions from code review
---------
Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
* Adds a `Nu6_1` variant to `NetworkUpgrade`
* updates CONSENSUS_BRANCH_IDS to include NU6.1 and NU7 when compiled with the `zebra-test` feature
* Update zebra-chain/src/parameters/network/testnet.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Update changelog
* Update end of support
* Add release date
* Add breaking change description
* Add new PRs to changelog
* Add breaking change description to changelog
* Update list of PRs in changelog
* chore: Release
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Introduce Nu7
* Introduce Nu7 fo other crates of Zebra
* Fix of new_regtest call (as it additionally needs Nu7 arg now)
* Fix of new_regtest call (as it additionally needs Nu7 arg now) (2)
* Set Nu7 as a network update for testnet in zebra-chain network tests
* Fix serde names for NU7
* Update test snapshot in zebra-rpc to use NU7
* Copy Nu7-related changes from zsa-integration-state
* Uncomment zcash_unstable = nu7
* Fix Nu7 constants and add cfg(zcash_unstable ...) attribute according to PR #15 review comments
* Update network constant values according to PR #15 review comments
* Add zcash_unstable=nu6 Rust flag to .cargo/config.toml to use nu6 by default
* Add zcash_unstable = nu6 cfg in zebra-network/.../types.rs
* Fix nu7 activation heights in network_upgrade.rs (nu6 value + 1 does not work - causes a test failure)
* Add cfg for CURRENT_NETWORK_PROTOCOL_VERSION constant definition, add FIXMEs for a couple of Nu6 processing cases, add processing of Nu7
* Update get_blockchain_info@testnet_10.snap
* Update get_blockchain_info@mainnet_10.snap
* updated cfg flag
* remove `zcash_unstable` flag
* Remove testnet and mainnet NU7 activation heights, comment out consensus branch id and conversion to zcash primitives nu type
* Yep, it'll work after NU7
* Yep, the test should work fine after NU7
* Yep, it only needs to be a post-nu6 height.
* other fixes and cleanups
* Updates snapshots, removes unnecessary FIXMEs, fixes issues with bad merge conflict resolution
* fixes test hang when there are NUs defined without activation heights
* fixes test, applies suggestion from code review.
* Enable consensus branch ID for Nu7
---------
Co-authored-by: Dmitry Demin <dmitry@qed-it.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
* extend `getrawtransaction` verbose output
* replace strings where possible, fix tests
* fix orchard
* add todos for script assembly code
* fix an import
* fmt
* fmt
* suggestion for #9329: change coinbase field and add valueSat field to match zcashd (#9407)
* change coinbase field and add valueSat field to match zcashd
* remove valueSat; update vectors
---------
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
This also updates to the latest `zcash_client_backend` dependency, and
fixes a few imports to use base types instead of reexported versions
that may be deprecated in the future.
* feat(network): Add misbehavior tracking for peers in the address book
* - Add a `misbehavior_score` field to `MetaAddr`
- Add a `bans_by_ip` field to `AddressBook`
- Update the `AddressBook::update()` method to:
- increment misbehavior scores in its entries,
- add addr ips to bans_by_ip if the score is excessive,
- remove any addrs at the banned ip
- Avoid responding to `GetAddr` requests with addresses of misbehaving peers (return None from `sanitized()`),
- Avoid new inbound or outbound connections to banned ips
* Drops banned peer connections in peer set's `poll_ready()` method
* Adds rudimentary misbehavior score tracking
* fixes some proptests, moves and removes some TODOs
* fixes lint
* Removes outdated TODO
* Adds stub for acceptance test
* updates call to updated fn
* Stores likely inbound peer connection addresses in address book and return their IPs with the default port instead of the transient port when responding to GetAddr requests
* Avoids gossiping peer addrs from inbound connections
* updates test to check that sanitize won't return inbound peer addrs or addrs with non-zero misbehaviour scores.
updated misbehaviour score for potentially unavoidable errors.
* Updates `generate` RPC to support any network where PoW is disabled.
Updates acceptance test to check that zebrad instances disconnect once one of them advertises a block with an invalid PoW
* minor tangential cleanup
* Finishes acceptance test, sends misbehavior updates from syncer, and always updates address book entries if the update is to their misbehaviour score
* skip test on windows (to minimize risk of port conflict)
* Applies suggestions from code review
* repeat words
* Update zebra-chain/src/block/serialize.rs
Co-authored-by: Pili Guerra <1311133+mpguerra@users.noreply.github.com>
---------
Co-authored-by: Pili Guerra <1311133+mpguerra@users.noreply.github.com>
* Add serialized Mainnet blocks for tests
* Add Sapling anchors
* Add Mainnet Orchard anchors
* Remove wrong Testnet NU5 blocks
* Add Testnet blocks with V5 txs to test vectors
* Move the Sapling treestate
* Add Sapling & Orchard anchors
* Remove unneeded test for fake V5 txs
We don't need this test anymore since we have real V5 txs now.
* Add `has_transparent_inputs` to `Transaction`
* Fix `v5_with_sapling_spends` test
* Fix `binding_signatures` test
* Refactor block test vectors
* Use real V5 txs instead of fake ones
* Fix `v5_transaction_is_rejected_before_nu5` test
* Fix `v5_tx_is_accepted_after_nu5_activation` test
* Fix `v5_tx_with_no_outputs_fails_validation` test
* Move `v5_tx_with_no_outputs_fails_validation` test
* Fix `v5_tx_with_no_inputs_fails_verification` test
* Fix `v5_tx_with_orchard_actions_has_inputs..` test
* Fix `v5_coinbase_tx_without_spends_flag_passes`
* Simplify `orchard` imports
* Fix `v5_tx_with_orchard_actions_has_flags` test
* Fix `v5_coinbase_tx_with_enable_spends_fails`
* Fix `v5_tx_with_duplicate_orchard_action` test
* Fix `coinbase_outputs_are_decryptable_for_v5`
* Fix `shielded_outputs_are_not_decryptable_for_v5`
* Use `Network::iter` instead of Mainnet
* Rename basic V5 tx test
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* Return an `Ok` in tx is not coinbase
* formatting
* Update zebra-consensus/src/transaction/check.rs
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Updates `coinbase_spend_restriction()` method to always return `OnlyShieldedOutputs` on Regtest.
* Adds a `should_allow_unshielded_coinbase_spends` field to testnet::Parameters
* Adds a test
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* Renames CoinbaseSpendRestriction variants and updates their documentation.
Updates a comment.
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Marek <mail@marek.onl>
* clean-up: simplify the def of `MapServerError`
* Use `HexData` instead of `String` for TXIDs
* Remove a redundant test
We don't need such a test anymore because the deserialization is handled
by Serde now.
* Adjust tests for using `HexData`
* Make `height` and `confirmations` optional
* Use legacy error codes
* fmt
* Remove unneeded error codes
* Remove `zebra-rpc/src/constants.rs`
* Rename `MapServerError` to `MapError`
* Rename `OkOrServerError` to `OkOrError`
* Allow specifying error codes when mapping errors
* Allow setting error codes when mapping options
* Use the right error code for `getrawtransaction`
* fmt
* Add docs for the error conversion traits
* Refactor the error handling for `getblock`
* Refactor error handling in `sendrawtransaction`
* Refactor the error handling for `getblock`
* Update the error handling for `getrawtransaction`
* Refactor error handling for `z_gettreestate`
* Refactor the error handling for address parsing
* Refactor the error handling for getrawtransaction
* Update `z_gettreestate` snapshots
* Cosmetics
* Refactor error handling in `getblock`
* Refactor error handling in `getblockheader`
* Simplify `getrawtransaction`
* Check errors for `getrawtransaction`
* fmt
* Simplify proptests
* Fix unit tests for `getaddresstxids`
* Fix unit tests for `getaddressutxos`
* fix docs
* Update snapshots for `getrawtransaction`
* Update zebra-rpc/src/server/error.rs
Co-authored-by: Arya <aryasolhi@gmail.com>
* Use `transaction::Hash` instead of `HexData`
* Simplify error handling
* Update zebra-rpc/src/server/error.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Move a note on performance
* Fix a typo
* Use `String` instead of `transaction::Hash`
* Adjust and add proptests
* Reintroduce snapshots for invalid TXIDs
* Don't derive `Serialize` & `Deserialize` for txids
Deriving `serde::Serialize` & `serde::Deserialize` for
`transaction::Hash` was superfluous, and we didn't need it anywhere in
the code.
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Adds getblockheader RPC method
* Updates snapshots, adds hash/height/next_block_hash fields to verbose response
* updates getblock snapshot
* updates getblockheader response type to hex-encode fields, adds ToHex impl for sapling::tree::Root, adds snapshot and vector tests for new RPC method, adds snapshots.
* rustfmt
* fixes snapshots, matches zcashd more closely
* fixes vectors test
* updates lwd failure messages (probably doesn't matter, but seems better to handle it now than risk debugging it later)
* fixes getblock_rpc test, fixes/reverses finalsaplingroot field byte-order.
* fixes vector test, addresses remaining differences with zcashd (except the `chainwork` field), updates snapshots, and avoids a possible panic when there's a chain reorg between state queries.
* Adds a comment noting that the `relative_to_network()` method was copied from zcashd
* Apply suggestions from code review
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Adds a parameter to `zebra_consensus::router::init()` for accepting a mempool setup argument, adds and uses an `init_test()` fn for passing a closed channel receiver in tests where no mempool service is needed in the transaction verifier.
* Adds a `mempool` argument to the transaction::Verifier constructor (and a `new_for_tests()` constructor for convenience)
* Removes `Clone` impl on `transaction::Verifier` to add mempool oneshot receiver, updates tests.
* Adds TODOs
* updates transaction verifier's poll_ready() method to setup the mempool service handle.
* Updates VerifiedSet struct used in mempool storage
* Updates mempool service and its `Storage` to use the updated `VerifiedSet` `transactions()` return type.
* updates `created_outputs` when inserting or removing a transaction from the mempool's verified set
* Adds a TODO, updates field docs
* Updates `spent_utxos()` to query the mempool for unspent outputs
* Adds `spent_mempool_outpoints` as a field on tx verifier mempool response
* Updates mempool `Downloads` to return the spent_mempool_outpoints from the tx verifier response
* Updates `Storage.insert()` to accept a list of spent mempool transaction outputs
* Adds transaction dependencies when inserting a tx in `VerifiedSet`
* polls mempool svc from tx verifier when a mempool tx that creates transparent outputs has been verified.
adds a TODO for adding a `pending_outputs` field to the mempool Storage
* Adds `pending_outputs` field on mempool Storage and responds to pending outputs requests when inserting new transactions into the mempool's verified set
* replaces `UnminedTxId` type with `transaction::Hash` in mempool's verified set
* prune pending outputs when rejecting and removing same effects.
* Remove dependent transactions from verified set when removing a tx
* updates tests
* appeases clippy.
* removes unused `len()` method
* fixes doc links
* Adds transaction dependencies to the `FullTransactions` response, let the caller handle it (required to avoid moving zip317 tx selection code to mempool)
* updates block template construction to avoid including transactions unless their dependencies have already been added.
* updates tests
* Replaces placeholder setup channel with one that sends the mempool svc to the tx verifier, adds a timeout layer, adds a TODO about a concurrency bug
* Use a single query to check for unspent outputs in the mempool
* Updates `getblocktemplate` method to consider dependencies when sorting transactions for the final template
* fixes clippy lints, removes unnecessary Option in UnspentOutput response variant
* renames type alias and method, adds a TODO to use iteration instead of recursion
* Adds mempool_removes_dependent_transactions() test
* Updates Storage and VerifiedSet clear() methods to clear pending_outputs, created_outputs, and transaction_dependencies, adds TODO to use iteration instead of recursion.
* removes outdated TODO
* Adds a TODO for reporting queued transaction verification results from the mempool from the poll_ready() method
* Adds `mempool_responds_to_await_output` test
* updates mempool_responds_to_await_output test
* Uses iteration instead of recursion in verified set's remove() method and zip317 mod's dependencies_depth() method
* Adds a mempool_request_with_mempool_output_is_accepted test for the transaction verifier
* Moves delay duration before polling the mempool to a constant, uses a shorter timeout for mempool output lookups, adds a `poll_count` to MockService, and updates `mempool_request_with_unmined_output_spends_is_accepted` to check that the transaction verifier polls the mempool after verifying a mempool transaction with transparent outputs
* adds long_poll_input_mempool_tx_ids_are_sorted test
* Adds a `excludes_tx_with_unselected_dependencies` test
* Updates a TODO
* moves `TransactionDependencies` struct to `zebra-node-services`
* Updates `FullTransactions` response variant's `transaction_dependencies` type
* updates zip317 transaction selection for block templates to include dependent transactions
* Moves and refactors zip317 tx selection test to its own module, adds an `unmined_transactions_in_blocks()` method on network
* Removes `unmined_transactions_in_blocks()` test utility fn from mempool Storage test module and replaces calls to it with calls to the new test method on Network
* Fixes spelling mistake
* Adds `includes_tx_with_selected_dependencies` test
* fixes zip317 block construction issue
* Fixes vectors test
* Update zebra-node-services/src/mempool.rs
* restores `tip_rejected_exact` type
* updates affected tests
* Documents the new argument in `Storage::insert()`, updates outdated comment
* Update zebrad/src/components/mempool/storage/verified_set.rs
* fixes potential issue with calling buffered mempool's poll_ready() method without calling it.
* Avoids removing dependent transactions of transactions that have been mined onto the best chain.
* Updates `spent_utxos()` method documentation
* Avoids sorting getblocktemplate transactions in non-test compilations
* documents PendingOutputs struct
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* cargo fmt
* Applies suggestions from code review
Avoids unnecessarily rejecting dependent transactions of randomly evicted mempool transactions.
Updates `TransactionDependencies::remove_all()` to omit provided transaction id from the list of removed transaction ids.
* Applies suggestions from code review.
* Adds minor comments
* Update zebrad/src/components/mempool/storage/verified_set.rs
Co-authored-by: Marek <mail@marek.onl>
* Remove an outdated comment (#9013)
---------
Co-authored-by: Marek <mail@marek.onl>
* Update docs for min network protocol versions
* Add a TODO for updating init net protocol version
* Update `CURRENT_NETWORK_PROTOCOL_VERSION`
* Add NU6 Mainnet activation height
* Extend docs for funding streams
* Extend tests for Mainnet funding streams
* Update snapshots
Marek
Conrado Gouvea
Arya
Alfredo Garcia
idky137
natalie
Paul
Tomass
Mark Henderson
Kris Nuttycombe
Metalcape
dependabot[bot]
Dimitris Apostolou
root