* Updates `coinbase_spend_restriction()` method to always return `OnlyShieldedOutputs` on Regtest.
* Adds a `should_allow_unshielded_coinbase_spends` field to testnet::Parameters
* Adds a test
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* Renames CoinbaseSpendRestriction variants and updates their documentation.
Updates a comment.
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Marek <mail@marek.onl>
* skips re-verifying transactions in blocks that are present in the mempool.
* clippy fix
* adds a test
* fixes clippy lint
* Use NU6 & V5 tx in new test
* Uses correct consensus branch id in test
* getblock: return tx objects with verbosity=2
* fix test
* use FuturesOrdered
* Suggestion for "add(rpc): getblock: return transaction details with verbosity=2" (#9084)
* Replaces multiple service calls (per transaction) with a single call to the state service for all of a block's transactions.
* adjustments to reuse code from getrawtransaction
---------
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* update snapshot
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* clean-up: simplify the def of `MapServerError`
* Use `HexData` instead of `String` for TXIDs
* Remove a redundant test
We don't need such a test anymore because the deserialization is handled
by Serde now.
* Adjust tests for using `HexData`
* Make `height` and `confirmations` optional
* Use legacy error codes
* fmt
* Remove unneeded error codes
* Remove `zebra-rpc/src/constants.rs`
* Rename `MapServerError` to `MapError`
* Rename `OkOrServerError` to `OkOrError`
* Allow specifying error codes when mapping errors
* Allow setting error codes when mapping options
* Use the right error code for `getrawtransaction`
* fmt
* Add docs for the error conversion traits
* Refactor the error handling for `getblock`
* Refactor error handling in `sendrawtransaction`
* Refactor the error handling for `getblock`
* Update the error handling for `getrawtransaction`
* Refactor error handling for `z_gettreestate`
* Refactor the error handling for address parsing
* Refactor the error handling for getrawtransaction
* Update `z_gettreestate` snapshots
* Cosmetics
* Refactor error handling in `getblock`
* Refactor error handling in `getblockheader`
* Simplify `getrawtransaction`
* Check errors for `getrawtransaction`
* fmt
* Simplify proptests
* Fix unit tests for `getaddresstxids`
* Fix unit tests for `getaddressutxos`
* fix docs
* Update snapshots for `getrawtransaction`
* Update zebra-rpc/src/server/error.rs
Co-authored-by: Arya <aryasolhi@gmail.com>
* Use `transaction::Hash` instead of `HexData`
* Simplify error handling
* Update zebra-rpc/src/server/error.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Move a note on performance
* Fix a typo
* Use `String` instead of `transaction::Hash`
* Adjust and add proptests
* Reintroduce snapshots for invalid TXIDs
* Don't derive `Serialize` & `Deserialize` for txids
Deriving `serde::Serialize` & `serde::Deserialize` for
`transaction::Hash` was superfluous, and we didn't need it anywhere in
the code.
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Fixes bug in send transaction test
* fixes new bug in send_transaction_test
* Removes unused `load_transactions_from_future_blocks` and factors out code for sending transactions to its own fn
* corrects tx count updates to exclude coinbase txs
* fixes formatting
* Calls zebra's sendrawtransaction method if lwd's send_transaction() return an error for more detailed error info
* removes instrument
* avoids panic when a future block has only a coinbase transaction
* fixes check for gossip log (only happens when 10 txs have been added
* fixes a concurrency bug, adds more detailed errors.
* removes unnecessary wait_for_stdout calls and fixes condition for early return
* Fixes issue around missing stdout line
* Fixes bug around expected tx ids and removes outdated TODO
* Fixes issue with expected ZF funding stream address balance in post-NU6 chains
* fixes the rest of wallet_grpc_test
* Update zebrad/src/components/mempool/downloads.rs
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
- Removed deprecated actions and simplified default queue rules.
- Adjusted batch sizes for urgent and batched queues.
- Consolidated priority rules to enhance clarity and efficiency.
- Updated conditions for moving pull requests to different queues based on labels and reviews.
* change(mempool): Return verification result after attempting to insert transactions in the mempool (#8901)
* respond with mempool verification result after a transaction has been inserted or has failed to be inserted into the mempool
* returns mempool verification errors early, and fixes handling for cancellations or timeouts.
* Adds a comment in test warning against code reuse with buffered services.
* De-duplicates handling for timeout errors
* Adds a test to check for crates in the Cargo.lock file that are being pulled in from a git source.
* add `check_no_git_refs_in_cargo_lock` to CI
* try skip instead of exclude
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Update the reference Sapling treestate
Zebra's treestate serialization differs from `zcashd` in the following
way:
`zcashd` omits the serialization of empty trailing ommers, while Zebra
doesn't. This means that `zcashd` serializes the Sapling treestate for
height 419_201 as
019eb30778ddeea84c72e69e07a1689f3c8def3dc0a1939f0edcbe47279069d9310002000150715810d52caf35471d10feb487213fbd95ff209122225b7b65d27a7fb1a44d
Whereas Zebra serializes it as
019eb30778ddeea84c72e69e07a1689f3c8def3dc0a1939f0edcbe47279069d931001f000150715810d52caf35471d10feb487213fbd95ff209122225b7b65d27a7fb1a44d0000000000000000000000000000000000000000000000000000000000
Serialization Format
====================
The serialized treestate consists of optional, hex-encoded, 32-byte
hashes. If the hash is not present, it is serialized as byte 0, i.e.,
`0x00`. If the hash is present, it is prefixed by byte 1, i.e. `0x01`.
The first two hashes in the serialized treestate are the treestate's
left and right leaves. These are followed by the serialized length of
the vector of ommers. This length is serialized as 1, 3, 5, or 9 bytes.
If the length is less than 253, it is serialized as a single byte. The
length is then followed by the serialized ommers.
We can now parse the first string, produced by `zcashd`:
- `0119eb30778ddeea84c72e69e07a1689f3c8def3dc0a1939f0edcbe47279069d931`
is the serialized left leaf,
- `00` is the serialized right leaf,
- `02` is the serialized length of the vector of ommers,
- `00` is the serialized first ommer,
- `0150715810d52caf35471d10feb487213fbd95ff209122225b7b65d27a7fb1a44d`
is the serialized second ommer.
And the second one, produced by Zebra:
- `0119eb30778ddeea84c72e69e07a1689f3c8def3dc0a1939f0edcbe47279069d931`
is the serialized left leaf,
- `00` is the serialized right leaf,
- `1f` is the serialized length of the vector of ommers,
- `00` is the serialized first ommer,
- `0150715810d52caf35471d10feb487213fbd95ff209122225b7b65d27a7fb1a44d`
is the serialized second ommer
- `0000000000000000000000000000000000000000000000000000000000` are the
remaining 29 serialized ommers.
Note that both serializations represent the same treestate.
* Remove a new line char
A LWD test was expecting the `ZEBRA_TEST_LIGHTWALLETD` to be set, but this variable is needed for all LWD tests and not specifically for `lightwalletd_integration`.
We had to rename this variable on a buggy `elif` statement in our Docker entrypoint.
This was avoiding most LWD tests to run correctly.
* Adds getblockheader RPC method
* Updates snapshots, adds hash/height/next_block_hash fields to verbose response
* updates getblock snapshot
* updates getblockheader response type to hex-encode fields, adds ToHex impl for sapling::tree::Root, adds snapshot and vector tests for new RPC method, adds snapshots.
* rustfmt
* fixes snapshots, matches zcashd more closely
* fixes vectors test
* updates lwd failure messages (probably doesn't matter, but seems better to handle it now than risk debugging it later)
* fixes getblock_rpc test, fixes/reverses finalsaplingroot field byte-order.
* fixes vector test, addresses remaining differences with zcashd (except the `chainwork` field), updates snapshots, and avoids a possible panic when there's a chain reorg between state queries.
* Adds a comment noting that the `relative_to_network()` method was copied from zcashd
* Apply suggestions from code review
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Adds a parameter to `zebra_consensus::router::init()` for accepting a mempool setup argument, adds and uses an `init_test()` fn for passing a closed channel receiver in tests where no mempool service is needed in the transaction verifier.
* Adds a `mempool` argument to the transaction::Verifier constructor (and a `new_for_tests()` constructor for convenience)
* Removes `Clone` impl on `transaction::Verifier` to add mempool oneshot receiver, updates tests.
* Adds TODOs
* updates transaction verifier's poll_ready() method to setup the mempool service handle.
* Updates VerifiedSet struct used in mempool storage
* Updates mempool service and its `Storage` to use the updated `VerifiedSet` `transactions()` return type.
* updates `created_outputs` when inserting or removing a transaction from the mempool's verified set
* Adds a TODO, updates field docs
* Updates `spent_utxos()` to query the mempool for unspent outputs
* Adds `spent_mempool_outpoints` as a field on tx verifier mempool response
* Updates mempool `Downloads` to return the spent_mempool_outpoints from the tx verifier response
* Updates `Storage.insert()` to accept a list of spent mempool transaction outputs
* Adds transaction dependencies when inserting a tx in `VerifiedSet`
* polls mempool svc from tx verifier when a mempool tx that creates transparent outputs has been verified.
adds a TODO for adding a `pending_outputs` field to the mempool Storage
* Adds `pending_outputs` field on mempool Storage and responds to pending outputs requests when inserting new transactions into the mempool's verified set
* replaces `UnminedTxId` type with `transaction::Hash` in mempool's verified set
* prune pending outputs when rejecting and removing same effects.
* Remove dependent transactions from verified set when removing a tx
* updates tests
* appeases clippy.
* removes unused `len()` method
* fixes doc links
* Adds transaction dependencies to the `FullTransactions` response, let the caller handle it (required to avoid moving zip317 tx selection code to mempool)
* updates block template construction to avoid including transactions unless their dependencies have already been added.
* updates tests
* Replaces placeholder setup channel with one that sends the mempool svc to the tx verifier, adds a timeout layer, adds a TODO about a concurrency bug
* Use a single query to check for unspent outputs in the mempool
* Updates `getblocktemplate` method to consider dependencies when sorting transactions for the final template
* fixes clippy lints, removes unnecessary Option in UnspentOutput response variant
* renames type alias and method, adds a TODO to use iteration instead of recursion
* Adds mempool_removes_dependent_transactions() test
* Updates Storage and VerifiedSet clear() methods to clear pending_outputs, created_outputs, and transaction_dependencies, adds TODO to use iteration instead of recursion.
* removes outdated TODO
* Adds a TODO for reporting queued transaction verification results from the mempool from the poll_ready() method
* Adds `mempool_responds_to_await_output` test
* updates mempool_responds_to_await_output test
* Uses iteration instead of recursion in verified set's remove() method and zip317 mod's dependencies_depth() method
* Adds a mempool_request_with_mempool_output_is_accepted test for the transaction verifier
* Moves delay duration before polling the mempool to a constant, uses a shorter timeout for mempool output lookups, adds a `poll_count` to MockService, and updates `mempool_request_with_unmined_output_spends_is_accepted` to check that the transaction verifier polls the mempool after verifying a mempool transaction with transparent outputs
* adds long_poll_input_mempool_tx_ids_are_sorted test
* Adds a `excludes_tx_with_unselected_dependencies` test
* Updates a TODO
* moves `TransactionDependencies` struct to `zebra-node-services`
* Updates `FullTransactions` response variant's `transaction_dependencies` type
* updates zip317 transaction selection for block templates to include dependent transactions
* Moves and refactors zip317 tx selection test to its own module, adds an `unmined_transactions_in_blocks()` method on network
* Removes `unmined_transactions_in_blocks()` test utility fn from mempool Storage test module and replaces calls to it with calls to the new test method on Network
* Fixes spelling mistake
* Adds `includes_tx_with_selected_dependencies` test
* fixes zip317 block construction issue
* Fixes vectors test
* Update zebra-node-services/src/mempool.rs
* restores `tip_rejected_exact` type
* updates affected tests
* Documents the new argument in `Storage::insert()`, updates outdated comment
* Update zebrad/src/components/mempool/storage/verified_set.rs
* fixes potential issue with calling buffered mempool's poll_ready() method without calling it.
* Avoids removing dependent transactions of transactions that have been mined onto the best chain.
* Updates `spent_utxos()` method documentation
* Avoids sorting getblocktemplate transactions in non-test compilations
* documents PendingOutputs struct
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* cargo fmt
* Applies suggestions from code review
Avoids unnecessarily rejecting dependent transactions of randomly evicted mempool transactions.
Updates `TransactionDependencies::remove_all()` to omit provided transaction id from the list of removed transaction ids.
* Applies suggestions from code review.
* Adds minor comments
* Update zebrad/src/components/mempool/storage/verified_set.rs
Co-authored-by: Marek <mail@marek.onl>
* Remove an outdated comment (#9013)
---------
Co-authored-by: Marek <mail@marek.onl>
In some cases Zebra logs might not output the database version, and thus we should avoid creating a disk without a version.
Before this change, a disk was created without a db version number, just indicating a `-v-`, that caused other tests to fail as an actual version was not found in their regexes.
Previously, most of our deployed instances needed to sync the whole blockchain from genesis, but after implementing the mounting of cached states for the release instances, this is no longer required.
Main changes:
- Reduce the boot disk size for CD images to 10GB (in CI Zebra might need to rebuild based on test flags, requiring more disk space)
- Use `pd-standard` instead of `pd-ssd` for the boot disk
- Use `pd-balanced` instead of `pd-ssd` for the mounted disk (where most of the reads and writes happens)
- Change our `GCP_SMALL_MACHINE` from `c2-standard-4` (vCPUs: 4, RAM: 16 GiB) to `c2d-standard-2` (vCPUs: 2, RAM: 8 GiB)
- Keep long running tests `is_long_test` with `GCP_LARGE_MACHINE` (`c2d-standard-16`) and other with the new `GCP_SMALL_MACHINE` configuration (`c2d-standard-2`)
* book: add section about private testnet testing
* Apply suggestions from code review
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove temporary dependency patches
We need to enable the `legacy-api` feature of `incrementalmerkletree` to
be able to serialize note commitment trees using an old serialization
format for the `z_gettreestate` RPC.
* Use spaces instead of a tab
* Bump ECC deps to match `zcashd`
* update documentation related auth cookie method for rpcs in the zebra book
* Update book/src/user/lightwalletd.md
* update rpc auth statement
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
dependabot[bot]
Alfredo Garcia
Arya
Conrado Gouvea
Marek
Gustavo Valverde
Cypher Pepe
pinglanlu
idky137