Commit Graph

28 Commits

Author SHA1 Message Date
idky137 9b91d4bc0e
change(tests): Remove Matches on Network From Tests (#8295)
* added functions for fetching block vectors

* inserted new network methods for vector fetching into zebra-chain

* changed tag back to test/feature=branch

* changed tag back to test/feature=branch

* changed tag back to test/feature=branch

* changed feature tag to proptest-impl, started implementing in zebra-consensus tests

* adding methods to zebra-consensus, lifetime error in src/transaction/tests.rs, needs refactoring

* finished adding methods to zebra-consensus

* finished adding new methods to zebrad

* added new methods to zebra-rpc and zebra-scan

* finished removing statements matching on Network from tests

* updated new error message

* changed get_block_bytes() and get_block_sapling_roots_bytes to return option and removed serialization error types as per requested changes in PR review

* removed match statements from zebra_chain::transaction::arbitrary::test_transactions() and new zebra-grpc tests

* moved zebra-chain::test_utils to zebra-chain::test

* removed get_ prefix from getter methods

* renamed zebra-chain::test to zebra-chain::tests

* renamed zebra-chain::test to zebra-chain::tests

* fixed clippy warnings

* changed block_map to return clone
2024-03-05 09:12:25 -05:00
teor 2ac6921d60
feat(mine): Add an internal Zcash miner to Zebra (#8136)
* Patch equihash to use the solver branch

* Add an internal-miner feature and set up its dependencies

* Remove 'Experimental' from mining RPC docs

* Fix a nightly clippy::question_mark lint

* Move a byte array utility function to zebra-chain

* fixup! Add an internal-miner feature and set up its dependencies

* Add an equihash::Solution::solve() method with difficulty checks

* Check solution is valid before returning it

* Add a TODO to check for peers before mining

* Move config validation into GetBlockTemplateRpcImpl::new()

* fixup! fixup! Add an internal-miner feature and set up its dependencies

* Use the same generic constraints for GetBlockTemplateRpcImpl struct and impls

* Start adding an internal miner component

* Add the miner task to the start command

* Add basic miner code

* Split out a method to mine one block

* Spawn to a blocking thread

* Wait until a valid template is available

* Handle shutdown

* Run mining on low priority threads

* Ignore some invalid solutions

* Use a difference nonce for each solver thread

* Update TODOs

* Change the patch into a renamed dependency to simplify crate releases

* Clean up instrumentation and TODOs

* Make RPC instances cloneable and clean up generics

* Make LongPollId Copy so it's easier to use

* Add API to restart mining if there's a new block template

* Actually restart mining if there's a new block template

* Tidy instrumentation

* fixup! Move config validation into GetBlockTemplateRpcImpl::new()

* fixup! Make RPC instances cloneable and clean up generics

* Run the template generator and one miner concurrently

* Reduce logging

* Fix a bug in getblocktemplate RPC tip change detection

* Work around some watch channel change bugs

* Rate-limit template changes in the receiver

* Run one mining solver per available core

* Use updated C code with double-free protection

* Update to the latest solver branch

* Return and submit all valid solutions

* Document what INPUT_LENGTH means

* Fix watch channel change detection

* Don't return early when a mining task fails

* Spawn async miner tasks to avoid cooperative blocking, deadlocks, and improve shutdown responsiveness

* Make existing parallelism docs and configs consistent

* Add a mining parallelism config

* Use the minimum of the configured or available threads for mining

* Ignore optional feature fields in tests

* Downgrade some frequent logs to debug

* Document new zebrad features and tasks

* Describe the internal-miner feature in the CHANGELOG

* Update dependency to de-duplicate equihash solutions

* Use futures::StreamExt instead of TryStreamExt

* Fix a panic message typo
2024-01-11 14:41:01 +00:00
Marek 9acdf0e512
Make the `verbose` argument optional (#8076)
The `verbose` argument of the `getrawtransaction` RPC is specified as
optional in the documentation:
https://zcash.github.io/rpc/getrawtransaction.html.
2023-12-10 21:44:43 +00:00
Marek 1dfebd791f
fix(state): Limit the retrieval of note commitment subtrees (#7734)
* Fix the combining of Sapling subtrees

This commit fixes a bug due to which the function `sapling_subtrees`
used to always include all subtrees from the non-finalized state without
respecting the given limit.

* Fix the combining of Orchard subtrees

This commit fixes a bug due to which the function `orchard_subtrees`
used to always include all subtrees from the non-finalized state without
respecting the given limit.

* Add additional checks when retrieving subtrees

* Allow raw subtree insertions into `Chain` in tests

* Derive `Default` for `Chain`

* Derive `Default` for note commitment tree nodes

* Use `pub(super)` for `DiskWriteBatch`

Set the visibility of `DiskWriteBatch` to `pub(super)`.

* Add tests for retrieving subtrees

* Use `default()` for subtree roots in tests

This change is unrelated to the PR.

* Refactor docs for inserting subtrees

This change is unrelated to the PR.
2023-10-12 20:00:43 +00:00
Marek 5b0bc70098
Add snapshot tests for `z_getsubtreesbyindex` RPC (#7514) 2023-09-10 23:09:59 +00:00
teor c3f0f53256
refactor(app): De-duplicate and fix version handling code (#6996)
* De-duplicate app_version and user_agent code, rename to build_version

* Make RPC testnet flag forward-compatible with additional testnets

* Fix RPC tests with new argument

* Use "modified" rather than "dirty" for uncommitted changes in build metadata

* Split the vergen version into its own function
2023-06-20 02:42:06 +00:00
teor b1ce0e0894
fix(security): Stop panicking on state RPC or block requests with very large heights (#6699)
* fix panic in height

* Update security comments and turn literals into constants

* Add a test-only assertion that the maximum output index can't ever be reached

* Fix a MISSING_BLOCK_ERROR_CODE that was mistakenly 0

* Add production RPC tests with excessive heights

* Add and update snapshots for production RPCs

* Add excessive height tests for most getblocktemplate RPCs

* Use correct snapshot names by running `cargo insta review`

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-05-17 15:13:12 +00:00
Alfredo Garcia 7c67512cd5
feat(zebra-network): add user agent argument (#6601)
* add user agent as argument, use git to auto build zebra user agent

* try to fix test

* fix typo

* change expect text

* remove newline

* fix some docs

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2023-05-05 00:29:14 +00:00
Alfredo Garcia 58bd898f5b
feat(zebrad): Refuse to run zebrad when release is too old (#6351)
* refuse to run Zebra if it is too old

* update the release checklist to consider the constants

* bring newline back

* apply new end of support code

* attempt to add tests (not working yet)

* move eos to progress task

* move tests

* add acceptance test (not working)

* fix tests

* change to block height checks (ugly code)

* change warn days

* refactor estimated blocks per day, etc

* move end of support code to its own task

* change test

* fix some docs

* move constants

* remove uneeded conversions

* downgrade tracing

* reduce end of support time, fix ci changing debugs to info again

* update instructions

* add failure messages

* cargo lock update

* unify releaase name constant

* change info msg

* clippy fixes

* add a block explorer

* ignore testnet in end of support task

* change panic to 16 weeks

* add some documentation about end of support

* Tweak docs wording

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-04-28 14:13:21 +00:00
Arya d7842bd467
fix(rpc): Check that mempool transactions are valid for the state's chain info in getblocktemplate (#6416)
* check last seen tip hash from mempool in fetch_mempool_transactions()

* Moves last_seen_tip_hash to ActiveState

* fixes tests and tests fixes

* continues to the next iteration of the loop to make fresh state and mempool requests if called with a long poll id

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* adds allow[unused_variable) for linter

* expects a chain tip when not(test)

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Addresses comments in code review

* - Removes second call to `last_tip_change()`

- Fixes tests using enabled mempool

* Adds note about chain tip action requirement to test method `enable()`

* updates doc comment

* Update zebrad/src/components/mempool.rs

Co-authored-by: teor <teor@riseup.net>

* fixes test

---------

Co-authored-by: teor <teor@riseup.net>
2023-04-03 23:22:07 +00:00
Arya 8ba3fd921a
change(rpc): Add confirmations to getrawtransaction method response (#6287)
* adds confirmation field to getrawtransaction

* Updates tests

* Adds comment about correctness

* Apply suggestion revisions from review

* fixes overflow bug and adds test for valid confirmations value

* Update test to check that confirmations isn't too high

* Update transaction request to return confirmations

* Applies suggestions from PR review

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* fixes test

* restore derives that were lost in a bad merge

---------

Co-authored-by: teor <teor@riseup.net>
2023-03-26 23:53:44 +00:00
teor ec43d63ed2
change(log): Log a cute message for blocks that were mined by Zebra (off by default) (#6098)
* Mark Zebra coinbase transactions with extra coinbase data

* Log when we commit a block mined by Zebra to our state

* Reduce logging instrumentation during block writes

* Remove debug types in Zebra block log

* Add network and commit to write task logs

* Apply an allow-list before we log arbitrary user messages from blocks

* Rate-limit Zebra mined block logging to once every 1000 blocks

* Add mining configs for extra coinbase data and imitating zcashd, but don't use them yet

* Check CoinbaseData size limit when building transparent transactions

* Replace LIKE_ZCASHD constants with a config

* Take extra coinbase data from the configured string

* Update the zebrad configs in the tests with new config fields
2023-02-23 00:10:11 +00:00
teor daba6d7744
add(rpc): Add extra `getblock` RPC fields used by some mining pools (#6097)
* clippy: remove unnecessary return statement

* Add hash, height, and confirmations fields to getblock RPC

* Remove a test that is already checked by snapshots

* Document the performance requirements of the getblock RPC

* Update snapshots, use new naming scheme

* Fix off-by-one error in confirmations

* Fix spelling mistakes
2023-02-07 01:25:34 +00:00
teor a51bc2edd5
Make the verbosity argument optional in the getblock RPC (#6092) 2023-02-03 02:27:29 +00:00
Arya be2c2299b1
change(rpc): Sort transaction hashes like zcashd in getrawmempool RPC response (#5994)
* Sorts transactions like zcashd in getrawmempool

* Simplifies sort logic and condenses duplicate code

* adds comment clarifying the intended byte order for transaction hashes

* Multiplies fee by MAX_BLOCK_BYTES/tx-size instead of tx-size

- Removes feature flag on get_raw_mempool

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Always uses `TransactionIds` request in tests

* reverts switch from #[cfg()] to cfg!()

* Adds feature flag to constant

* Updates tests and removes !cfg(not(test))

* Moves up comment

* adds missing transaction_ids

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-23 04:48:18 +00:00
Arya 57fde15e5e
change(rpc): return an error from getblocktemplate method if Zebra is not synced to network tip (#5623)
* Returns error from getblocktemplate if not synced

* sets max estimated distance to 1

* removes unnecessary calls to best_tip_height

adds info log when estimated_distance_to_chain_tip is too high

* trigger GitHub actions

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-18 00:12:10 +00:00
Arya 2f3b05f8e1
change(rpc): add submitblock RPC method (#5526)
* adds submitblock rpc method

* re-orders imports

* replaces thread::yield_now with async yield_now

* Fix doc warnings and unused variable warnings, add missing docs

* Mark work_id as optional

* Use the same ChainVerifier for downloaded and submitted blocks

* Revert unused changes & minor cleanups

* Document currently-unreachable code

* updates tests and submit_block response for AlreadyVerified error

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* changes names from BlockVerifier to ChainVerifier and block_verifier to chain_verifier to keep it consistent with naming in zebra-consensus

* move how to run the submit_block test example to acceptance.rs

* updates snapshot tests

* moved acceptance test to a separate file

* removes extra tower::ServiceBuilder::new(), updates docs

* updates vectors and snapshot tests, changes hex decoding error in submit_block method from server error to parse error

* hides errors module in zebra-rpc behind a feature flag and adds docs.

* Updates snapshot test, adds mod docs, moves HexData to its own mod, and removes the unrelated make_server_error refactor for now

* update submit block acceptance test mod doc

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-04 03:57:08 +00:00
teor 71f5e63e64
3. change(rpc): Add fee and sigops fields to getblocktemplate transactions (#5508)
* Add a legacy_sigop_count field to VerifiedUnminedTx

* Add conversions from Vec<VerifiedUnminedTx> to block header roots

* Add fee and sigops field to block template transactions

* Fix up mempool request names

* Increase existing snapshot test coverage

* Document a new method parameter

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-11-03 17:03:41 +00:00
teor 9cb3dbba9b
1. change(rpc): Add a mempool field to GetBlockTemplateRpcImpl, and cleanup tests (#5493)
* Add a mempool to GetBlockTemplateRpcImpl, and cleanup tests

* Update snapshot file locations

* Update snapshot instructions
2022-10-28 18:34:52 +00:00
Alfredo Garcia fae9473076
feat(rpc): introduce getblocktemplate rpc call with stub fields (#5462)
* introduce getblocktemplate rpc call

* remove optional `longpollid` field

* add underscore

* create modules for types
2022-10-26 02:52:29 +00:00
Alfredo Garcia 8d777c892f
feat(rpc): add getblockhash rpc method (#4967)
* implement getblockhash rpc method

* make fixes

* fix some docs

* rustfmt

* add snapshot test

* rename `Hash` to `BestChainBlockHash`

* Suggestion for "add getblockhash rpc method" PR (#5428)

* Always immediately return errors in get_height_from_int()

* Explain why calculations can't overflow

* fix for rust feature

* fix some warnings

* hide state functions behind feature

* remove commented assert

* renames

* rename

* fix some warnings

* make zebra-rpc rpc features depend on zebra-state rpc features

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2022-10-21 06:01:29 +00:00
Alfredo Garcia d81a5fc576
feat(rpc): introduce `getblocktemplate-rpcs` feature (#5357)
* introduce `getblocktemplate-rpcs` feature

* reorder imports

* highlight the problem

* add docs

* add additional empty state test

* add snapshot test

* remove getblocktemplate trait

* use `cfg-if`

* leave server as it was

* add a missing space to the docs

* fix typo in test

Co-authored-by: Arya <aryasolhi@gmail.com>

* suggestion for introduce `getblocktemplate-rpcs` feature (#5418)

* adds minimal new object for the get block template methods

* updated TODO comment

Co-authored-by: Arya <aryasolhi@gmail.com>
2022-10-19 12:01:13 +00:00
teor ea34baa8b8
add(test): test disabled `lightwalletd` mempool gRPCs via zebrad logs (#5016)
* add grpc mempool test research

* add a config flag for mempool injection of transactions in test

* Only copy the inner state directory in the send transactions test

* Preload Zcash parameters in some transaction verification tests

* Add a block and transaction Hash method to convert from display order bytes

* Update test coverage docs

* Add debugging output for mempool transaction verification

* Test fetching sent mempool transactions using gRPC

* Add extra log checks to the send transaction test

* Wait for zebrad mempool activation before running gRPC tests

* Update send transaction test for lightwalletd not returning mempool transactions

* Check zebrad logs instead of disabled lightwalletd gRPCs

* Add a debug option that makes RPCs pretend the sync is finished

* Remove an unused debug option

* Remove unused test code and downgrade some logs

* Fix test log checks

* Fix some rustdoc warnings

* Fix a compilation error due to new function arguments

* Make zebrad sync timeouts consistent and remove outdated code

* Document how to increase temporary directory space for tests

* Stop checking for a log that doesn't always happen

* Remove some commented-out code

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Update a comment about run time

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Add new config to new tests from the `main` branch

* Add transactions to the list, rather than replacing the list with each new block

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-09-06 13:32:33 +00:00
Conrado Gouvea 6fd750e168
build(deps): bump insta from 1.15.0 to 1.17.1 (#4884)
* build(deps): bump insta from 1.15.0 to 1.17.1

Bumps [insta](https://github.com/mitsuhiko/insta) from 1.15.0 to 1.17.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.15.0...1.17.1)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* make zebra_test::init() return the insta drop guard

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-04 15:44:44 +00:00
teor 394d16a5a5
2. fix(perf): When writing blocks to disk, don't block other async tasks (#4199)
* Only fetch block headers from the database to answer headers requests

* Move writing to the database to a blocking thread

* Add blocking threads to tests that need them

* Remove mempool downloader requirement for a populated state

And stop populating states that don't need it,
so we can use tokio::time::pause() in those tests.

* Improve debugging for an intermittent test failure

* Try to avoid a race condition populating the mempool in tests
2022-07-22 09:16:41 +10:00
Conrado Gouvea 95f14ffdcd
add support for getblock with verbosity=1 (#4511) 2022-05-27 09:41:11 +00:00
Alfredo Garcia 191b1c02a2
add `z_get_treestate` snapshot test (#4401) 2022-05-18 14:06:12 +00:00
Alfredo Garcia 128c9cbfd8
tests(rpc): add snapshot tests for rpc methods responses (#4352)
* add snapshot tests for rpc methods responses

* change snapshots format to json

* change snapshot file name format

* fix(rpc): fixes for RPC snapshot tests (#4384)

* Create the snapshot settings once

* Snapshot testnet state, RPCs, and transparent addresses

* Tweak header comment

* Ignore some duplicated dependencies we don't control

* fix texts

Co-authored-by: Marek <mail@marek.onl>

* Redact 1.5 kB of block data in snapshots

* Fix docs

* Explain snapshot redaction better

* Make redaction substitution consistent

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Update redacted snapshots

* Make formatting consistent

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
2022-05-16 02:33:46 +00:00