Commit Graph

65 Commits

Author SHA1 Message Date
Alfredo Garcia f2be73eebb
feat(rpc): OpenAPI spec (#8342)
* add an openapi generator

* fix run command

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2024-03-14 15:04:19 +00:00
Za Wilcox 3bef54b764
change(consensus): Refactor production code for network consensus rules to `Network` methods (#8340)
* begin refactor suggested as "step 2": https://github.com/ZcashFoundation/zebra/issues/7968#issue-2003245309
Squashed from multiple commits to enable partial rebase

* break out more little traits

* add activation implementation leveraging From<Network> for lrz::cons::

* for transfer of ownership I cannot return a type that's owned by the method

* hrp_sapling_extended_full_viewing_key

* complete implementation of interface of Parameters on Network reuse Parameters on zcash Network where possible

* move doc-comments to trait declarations (from impls)

* Simplify/complete Parameters impl for Network

* Add checkpoint_list method, move documentation, etc

* move last match network to inside network method

* add back comment

* use zcash_address for parameter types in zebra-chain

* use inherent methods instead of big parameters passthrough

* revert to implementation of From on zcash_primitives::..::Network vs &zcash_prim...

* move match

* add test to block maximum time rule

* update changelog

* finish porting target_difficutly_limit

* remove obscelete code comment

* revert non-functional change

* finish migrating target_difficulty_limit, checkpoint_list

* update changelog

---------

Co-authored-by: Hazel OHearn <gygaxis@zingolabs.org>
2024-03-12 21:41:44 +00:00
teor 2ac6921d60
feat(mine): Add an internal Zcash miner to Zebra (#8136)
* Patch equihash to use the solver branch

* Add an internal-miner feature and set up its dependencies

* Remove 'Experimental' from mining RPC docs

* Fix a nightly clippy::question_mark lint

* Move a byte array utility function to zebra-chain

* fixup! Add an internal-miner feature and set up its dependencies

* Add an equihash::Solution::solve() method with difficulty checks

* Check solution is valid before returning it

* Add a TODO to check for peers before mining

* Move config validation into GetBlockTemplateRpcImpl::new()

* fixup! fixup! Add an internal-miner feature and set up its dependencies

* Use the same generic constraints for GetBlockTemplateRpcImpl struct and impls

* Start adding an internal miner component

* Add the miner task to the start command

* Add basic miner code

* Split out a method to mine one block

* Spawn to a blocking thread

* Wait until a valid template is available

* Handle shutdown

* Run mining on low priority threads

* Ignore some invalid solutions

* Use a difference nonce for each solver thread

* Update TODOs

* Change the patch into a renamed dependency to simplify crate releases

* Clean up instrumentation and TODOs

* Make RPC instances cloneable and clean up generics

* Make LongPollId Copy so it's easier to use

* Add API to restart mining if there's a new block template

* Actually restart mining if there's a new block template

* Tidy instrumentation

* fixup! Move config validation into GetBlockTemplateRpcImpl::new()

* fixup! Make RPC instances cloneable and clean up generics

* Run the template generator and one miner concurrently

* Reduce logging

* Fix a bug in getblocktemplate RPC tip change detection

* Work around some watch channel change bugs

* Rate-limit template changes in the receiver

* Run one mining solver per available core

* Use updated C code with double-free protection

* Update to the latest solver branch

* Return and submit all valid solutions

* Document what INPUT_LENGTH means

* Fix watch channel change detection

* Don't return early when a mining task fails

* Spawn async miner tasks to avoid cooperative blocking, deadlocks, and improve shutdown responsiveness

* Make existing parallelism docs and configs consistent

* Add a mining parallelism config

* Use the minimum of the configured or available threads for mining

* Ignore optional feature fields in tests

* Downgrade some frequent logs to debug

* Document new zebrad features and tasks

* Describe the internal-miner feature in the CHANGELOG

* Update dependency to de-duplicate equihash solutions

* Use futures::StreamExt instead of TryStreamExt

* Fix a panic message typo
2024-01-11 14:41:01 +00:00
Marek 9acdf0e512
Make the `verbose` argument optional (#8076)
The `verbose` argument of the `getrawtransaction` RPC is specified as
optional in the documentation:
https://zcash.github.io/rpc/getrawtransaction.html.
2023-12-10 21:44:43 +00:00
Marek 0d3892f61f
change(state): Refactor the naming of note commitment subtrees (#7855)
* Rename `node` & `Node` to `root` & `Root`

* Rename `end` to `end_height`

* Rename `Root` to `SubtreeRoot`
2023-10-30 20:06:54 +00:00
teor 5e8b4f32d8
Implement the z_get_subtrees_by_index RPC method and return type (#7436)
Revert "Temporarily remove the z_get_subtrees_by_index RPC method"
This reverts commit 30d049ee9f.
2023-09-04 21:05:39 +00:00
teor 735330618e
cleanup(state): Update some outdated comments in the state & rpcs (#7434)
* Add a module cleanup TODO

* Fix an unrelated comment

* Update the comments about the bincode legacy format
2023-08-31 16:25:11 +00:00
Alfredo Garcia b322748b66
add(rpc): note tree sizes to `getblock` api (#7278)
* add the basics

* add some docs, move code

* upgrade compact formats to https://github.com/zcash/lightwalletd/blob/v0.4.15/walletrpc/compact_formats.proto

* add a test for in sync chain

* test changing to ecc lightwalletd

* revert change of lightwalletd repo (already merged to main)

* add debug log to see whats going on with the test

* change log to tracing::info

* remove log line

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-08-15 18:48:50 +00:00
teor c3f0f53256
refactor(app): De-duplicate and fix version handling code (#6996)
* De-duplicate app_version and user_agent code, rename to build_version

* Make RPC testnet flag forward-compatible with additional testnets

* Fix RPC tests with new argument

* Use "modified" rather than "dirty" for uncommitted changes in build metadata

* Split the vergen version into its own function
2023-06-20 02:42:06 +00:00
teor a18f47d5f6
fix(app): Stop panicking at startup when parsing the app version - release blocker (#6888)
* Fix a startup panic in app_version()

* Fix a potential RPC panic in get_info()

* Fix typo
2023-06-09 05:07:34 +00:00
teor b1ce0e0894
fix(security): Stop panicking on state RPC or block requests with very large heights (#6699)
* fix panic in height

* Update security comments and turn literals into constants

* Add a test-only assertion that the maximum output index can't ever be reached

* Fix a MISSING_BLOCK_ERROR_CODE that was mistakenly 0

* Add production RPC tests with excessive heights

* Add and update snapshots for production RPCs

* Add excessive height tests for most getblocktemplate RPCs

* Use correct snapshot names by running `cargo insta review`

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-05-17 15:13:12 +00:00
Alfredo Garcia 7c67512cd5
feat(zebra-network): add user agent argument (#6601)
* add user agent as argument, use git to auto build zebra user agent

* try to fix test

* fix typo

* change expect text

* remove newline

* fix some docs

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2023-05-05 00:29:14 +00:00
Alfredo Garcia 58bd898f5b
feat(zebrad): Refuse to run zebrad when release is too old (#6351)
* refuse to run Zebra if it is too old

* update the release checklist to consider the constants

* bring newline back

* apply new end of support code

* attempt to add tests (not working yet)

* move eos to progress task

* move tests

* add acceptance test (not working)

* fix tests

* change to block height checks (ugly code)

* change warn days

* refactor estimated blocks per day, etc

* move end of support code to its own task

* change test

* fix some docs

* move constants

* remove uneeded conversions

* downgrade tracing

* reduce end of support time, fix ci changing debugs to info again

* update instructions

* add failure messages

* cargo lock update

* unify releaase name constant

* change info msg

* clippy fixes

* add a block explorer

* ignore testnet in end of support task

* change panic to 16 weeks

* add some documentation about end of support

* Tweak docs wording

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-04-28 14:13:21 +00:00
Arya d7842bd467
fix(rpc): Check that mempool transactions are valid for the state's chain info in getblocktemplate (#6416)
* check last seen tip hash from mempool in fetch_mempool_transactions()

* Moves last_seen_tip_hash to ActiveState

* fixes tests and tests fixes

* continues to the next iteration of the loop to make fresh state and mempool requests if called with a long poll id

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* adds allow[unused_variable) for linter

* expects a chain tip when not(test)

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Addresses comments in code review

* - Removes second call to `last_tip_change()`

- Fixes tests using enabled mempool

* Adds note about chain tip action requirement to test method `enable()`

* updates doc comment

* Update zebrad/src/components/mempool.rs

Co-authored-by: teor <teor@riseup.net>

* fixes test

---------

Co-authored-by: teor <teor@riseup.net>
2023-04-03 23:22:07 +00:00
Arya 8ba3fd921a
change(rpc): Add confirmations to getrawtransaction method response (#6287)
* adds confirmation field to getrawtransaction

* Updates tests

* Adds comment about correctness

* Apply suggestion revisions from review

* fixes overflow bug and adds test for valid confirmations value

* Update test to check that confirmations isn't too high

* Update transaction request to return confirmations

* Applies suggestions from PR review

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* fixes test

* restore derives that were lost in a bad merge

---------

Co-authored-by: teor <teor@riseup.net>
2023-03-26 23:53:44 +00:00
teor ec43d63ed2
change(log): Log a cute message for blocks that were mined by Zebra (off by default) (#6098)
* Mark Zebra coinbase transactions with extra coinbase data

* Log when we commit a block mined by Zebra to our state

* Reduce logging instrumentation during block writes

* Remove debug types in Zebra block log

* Add network and commit to write task logs

* Apply an allow-list before we log arbitrary user messages from blocks

* Rate-limit Zebra mined block logging to once every 1000 blocks

* Add mining configs for extra coinbase data and imitating zcashd, but don't use them yet

* Check CoinbaseData size limit when building transparent transactions

* Replace LIKE_ZCASHD constants with a config

* Take extra coinbase data from the configured string

* Update the zebrad configs in the tests with new config fields
2023-02-23 00:10:11 +00:00
teor ae21e36018
fix(rpc): Add `getblock` RPC fields to support the latest version of `zcash/lightwalletd` (#6134)
* Stabilise the BestChainBlockHash state request

* Always include the block hash in the getblock RPC response

* Make the lightwalletd integration tests compatible with zcash/lightwalletd

* Update getblock RPC snapshots

* Return the correct missing block error code
2023-02-14 08:52:58 +00:00
teor daba6d7744
add(rpc): Add extra `getblock` RPC fields used by some mining pools (#6097)
* clippy: remove unnecessary return statement

* Add hash, height, and confirmations fields to getblock RPC

* Remove a test that is already checked by snapshots

* Document the performance requirements of the getblock RPC

* Update snapshots, use new naming scheme

* Fix off-by-one error in confirmations

* Fix spelling mistakes
2023-02-07 01:25:34 +00:00
teor a51bc2edd5
Make the verbosity argument optional in the getblock RPC (#6092) 2023-02-03 02:27:29 +00:00
teor 9d97919afc
fix(rpc): Make RPC "incorrect parameters" error code match `zcashd` (#6066)
* Move RPC method constants into their own module

* Rename RPC compatibility modules to avoid confusion

* Rename RPC middleware to include its new functionality

* Use FutureExt::inspect() for logging, and only format on failure

* Log all RPC errors at info level

* Make "invalid parameters" RPC error code match `zcashd`
2023-02-01 18:20:48 +00:00
Arya be2c2299b1
change(rpc): Sort transaction hashes like zcashd in getrawmempool RPC response (#5994)
* Sorts transactions like zcashd in getrawmempool

* Simplifies sort logic and condenses duplicate code

* adds comment clarifying the intended byte order for transaction hashes

* Multiplies fee by MAX_BLOCK_BYTES/tx-size instead of tx-size

- Removes feature flag on get_raw_mempool

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Always uses `TransactionIds` request in tests

* reverts switch from #[cfg()] to cfg!()

* Adds feature flag to constant

* Updates tests and removes !cfg(not(test))

* Moves up comment

* adds missing transaction_ids

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-23 04:48:18 +00:00
Arya b0ba920a4f
change(test): Create test harness for calling getblocktemplate in proposal mode, but don't use it yet (#5884)
* adds ValidateBlock request to state

* adds `Request` enum in block verifier

skips solution check for BlockProposal requests

calls CheckBlockValidity instead of Commit block for BlockProposal requests

* uses new Request in references to chain verifier

* adds getblocktemplate proposal mode response type

* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state

* Adds PR review revisions

* adds info log in CheckBlockProposalValidity

* Reverts replacement of match statement

* adds `GetBlockTemplate::capabilities` fn

* conditions calling checkpoint verifier on !request.is_proposal

* updates references to validate_and_commit_non_finalized

* adds snapshot test, updates test vectors

* adds `should_count_metrics` to NonFinalizedState

* Returns an error from chain verifier for block proposal requests below checkpoint height

adds feature flags

* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD

* adds back block::Request to zebra-consensus lib

* updates snapshots

* Removes unnecessary network arg

* skips req in tracing intstrument for read state

* Moves out block proposal validation to its own fn

* corrects `difficulty_threshold_is_valid` docs

adds/fixes some comments, adds TODOs

general cleanup from a self-review.

* Update zebra-state/src/service.rs

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* check best chain tip

* Update zebra-state/src/service.rs

Co-authored-by: teor <teor@riseup.net>

* Applies cleanup suggestions from code review

* updates gbt acceptance test to make a block proposal

* fixes json parsing mistake

* adds retries

* returns reject reason if there are no retries left

* moves result deserialization to RPCRequestClient method, adds docs, moves jsonrpc_core to dev-dependencies

* moves sleep(EXPECTED_TX_TIME) out of loop

* updates/adds info logs in retry loop

* Revert "moves sleep(EXPECTED_TX_TIME) out of loop"

This reverts commit f7f0926f4050519687a79afc16656c3f345c004b.

* adds `allow(dead_code)`

* tests with curtime, mintime, & maxtime

* Fixes doc comment

* Logs error responses from chain_verifier CheckProposal requests

* Removes retry loop, adds num_txs log

* removes verbose info log

* sorts mempool_txs before generating merkle root

* Make imports conditional on a feature

* Disable new CI tests until bugs are fixed

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-17 04:03:40 +00:00
Arya f3366c53c7
fix(rpc): Accept HashOrHeight as first parameter of getblock and update README.md to note differences between lightwalletd forks (#5861)
* updates getblock RPC to accept HashOrHeight param

* update README.md to recommend lightwalletd fork

* updates vectors test

* Update README.md

* Update README.md

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2022-12-14 05:29:33 +00:00
teor e9d6e975b4
change(rpc): Refactor get block template RPC into stages (#5837)
* Add some TODOs

* Move and rename height_from_signed_int()

* Move get_block_template() support functions to a submodule

* Fix incorrect P2SH comments and logs

* Split initial checks into their own functions

* Split state fetch into its own function, do some cleanup

* Move get_block_template_opts to get_block_template::parameters

* Fix and simplify test imports

* Rename block_height to next_block_height

* Rename to chain_tip_and_local_time to make it clear what it contains

* Split fetching mempool transactions out, include them in long poll id

* Refactor coinbase generation

* Split default root calculation into a separate function

* Use DateTime32 for getblocktemplate times

* Use typed difficulty fields rather than strings

* Split out a generate coinbase and roots function

* Move GetBlockTemplate construction into a method

* Document what happens to unusual difficulty values

* Clean up some TODOs

* fastmod check_address check_miner_address

* cargo fmt --all

* Fix an incorrect panic message

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-12-13 21:25:04 +00:00
teor 4078e244d3
fix(lint): Box large error types to resolve the clippy large result err variant lint (#5759)
* Box errors to deal with large error warnings, add accessor methods for error properties

* Remove or explain some large enum variant lints

* Turn some tickets into TODOs

* Allow missing docs on single-field error enum variants

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-12-08 06:11:33 +00:00
teor 074733d183
fix(rpc): Shut down the RPC server properly when Zebra shuts down (#5591)
* Make the queue runner task shut down when the RpcImpl is dropped

* Move RPC server startup into the spawn() tokio future

* Return a shutdown handle from the RPC spawn() method

* Shut down the RPC server properly when Zebra shuts down

* Add a changelog entry for this security fix

* Call RpcServer::shutdown() when it is dropped, and wait

* Block on RPC server shutdown when Zebra's tasks have an error
2022-11-10 14:51:53 +00:00
teor 75f83fc5b0
Factor out a best_chain_tip_height() function (#5540) 2022-11-04 18:19:48 +00:00
Alfredo Garcia 19cb670614
rpc(config): Add a `mining` section with miner address to config (#5491)
* add mining section to zebra config

* fix features in zebrad `Cargo.toml`

* change field name

* add docs

Co-authored-by: teor <teor@riseup.net>

* add getblocktemplate-rpcs config

* fix commit

Co-authored-by: teor <teor@riseup.net>
2022-11-01 00:43:45 +00:00
teor 9cb3dbba9b
1. change(rpc): Add a mempool field to GetBlockTemplateRpcImpl, and cleanup tests (#5493)
* Add a mempool to GetBlockTemplateRpcImpl, and cleanup tests

* Update snapshot file locations

* Update snapshot instructions
2022-10-28 18:34:52 +00:00
teor c812f880cf
cleanup(clippy): Use inline format strings (#5489)
* Inline format strings using an automated clippy fix

```sh
cargo clippy --fix --all-features --all-targets -- -A clippy::all -W clippy::uninlined_format_args
cargo fmt --all
```

* Remove unused & and &mut using an automated clippy fix

```sh
cargo clippy --fix --all-features --all-targets -- -A clippy::all -W clippy::uninlined_format_args
```
2022-10-27 13:25:18 +00:00
Alfredo Garcia fae9473076
feat(rpc): introduce getblocktemplate rpc call with stub fields (#5462)
* introduce getblocktemplate rpc call

* remove optional `longpollid` field

* add underscore

* create modules for types
2022-10-26 02:52:29 +00:00
Alfredo Garcia 8d777c892f
feat(rpc): add getblockhash rpc method (#4967)
* implement getblockhash rpc method

* make fixes

* fix some docs

* rustfmt

* add snapshot test

* rename `Hash` to `BestChainBlockHash`

* Suggestion for "add getblockhash rpc method" PR (#5428)

* Always immediately return errors in get_height_from_int()

* Explain why calculations can't overflow

* fix for rust feature

* fix some warnings

* hide state functions behind feature

* remove commented assert

* renames

* rename

* fix some warnings

* make zebra-rpc rpc features depend on zebra-state rpc features

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2022-10-21 06:01:29 +00:00
Alfredo Garcia d81a5fc576
feat(rpc): introduce `getblocktemplate-rpcs` feature (#5357)
* introduce `getblocktemplate-rpcs` feature

* reorder imports

* highlight the problem

* add docs

* add additional empty state test

* add snapshot test

* remove getblocktemplate trait

* use `cfg-if`

* leave server as it was

* add a missing space to the docs

* fix typo in test

Co-authored-by: Arya <aryasolhi@gmail.com>

* suggestion for introduce `getblocktemplate-rpcs` feature (#5418)

* adds minimal new object for the get block template methods

* updated TODO comment

Co-authored-by: Arya <aryasolhi@gmail.com>
2022-10-19 12:01:13 +00:00
teor 211dbb437b
1. fix(rpc): Fix slow getblock RPC (verbose=1) using transaction ID index (#5307)
* Add RPC timing to zcash-rpc-diff

* Use transaction hash index for verbose block requests, rather than block data
2022-10-02 23:34:44 +00:00
teor 343c5e68d4
change(state): Write finalized blocks to the state in a separate thread, to avoid network and RPC hangs (#5134)
* Add a new block commit task and channels, that don't do anything yet

* Add last_block_hash_sent to the state service, to avoid database accesses

* Update last_block_hash_sent regardless of commit errors

* Rename a field to StateService.max_queued_finalized_height

* Commit finalized blocks to the state in a separate task

* Check for panics in the block write task

* Wait for the block commit task in tests, and check for errors

* Always run a proptest that sleeps once

* Add extra debugging to state shutdowns

* Work around a RocksDB shutdown bug

* Close the finalized block channel when we're finished with it

* Only reset state queue once per error

* Update some TODOs

* Add a module doc comment

* Drop channels and check for closed channels in the block commit task

* Close state channels and tasks on drop

* Remove some duplicate fields across StateService and ReadStateService

* Try tweaking the shutdown steps

* Update and clarify some comments

* Clarify another comment

* Don't try to cancel RocksDB background work on drop

* Fix up some comments

* Remove some duplicate code

* Remove redundant workarounds for shutdown issues

* Remode a redundant channel close in the block commit task

* Remove a mistaken `!force` shutdown condition

* Remove duplicate force-shutdown code and explain it better

* Improve RPC error logging

* Wait for chain tip updates in the RPC tests

* Wait 2 seconds for chain tip updates before skipping them

* Remove an unnecessary block_in_place()

* Fix some test error messages that were changed by earlier fixes

* Expand some comments, fix typos

Co-authored-by: Marek <mail@marek.onl>

* Actually drop children of failed blocks

* Explain why we drop descendants of failed blocks

* Clarify a comment

* Wait for chain tip updates in a failing test on macOS

* Clean duplicate finalized blocks when the non-finalized state activates

* Send an error when receiving a duplicate finalized block

* Update checkpoint block behaviour, document its consensus rule

* Wait for chain tip changes in inbound_block_height_lookahead_limit test

* Wait for the genesis block to commit in the fake peer set mempool tests

* Disable unreliable mempool verification check in the send transaction test

* Appease rustfmt

* Use clear_finalized_block_queue() everywhere that blocks are dropped

* Document how Finalized and NonFinalized clones are different

* Use the same check as commit_finalized() for finalized block heights

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Marek <mail@marek.onl>
2022-09-28 16:09:56 +00:00
teor 36a549ee3c
2. change(state): Run AwaitUtxo read requests without shared mutable chain state (#5107)
* Move AwaitUtxos next to the other shared writeable state requests

* Rename ReadResponse::Utxos to ReadResponse::AddressUtxos

```sh
fastmod Utxos AddressUtxos zebra*
```

* Rename an out_point variable to outpoint for consistency

* Rename transparent_utxos to address_utxos

```sh
fastmod transparent_utxos address_utxos zebra*
```

* Run AwaitUtxo without accessing shared mutable chain state

* Fix some incorrect comments

* Explain why some concurrent reads are ok

* Add a TODO

* Stop using self.mem in AwaitUtxo requests

* Update state service module documentation

* Move the QueuedBlock type into the queued_blocks module

* Explain how spent UTXOs are treated by the state

* Clarify how cached Chains impact state read requests

And move repeated comments to the module header.

* fastmod ChainUtxo BestChainUtxo zebra*

* Add an AnyChainUtxo request

* Make AwaitUtxo non-blocking
2022-09-16 04:13:26 +00:00
teor ea34baa8b8
add(test): test disabled `lightwalletd` mempool gRPCs via zebrad logs (#5016)
* add grpc mempool test research

* add a config flag for mempool injection of transactions in test

* Only copy the inner state directory in the send transactions test

* Preload Zcash parameters in some transaction verification tests

* Add a block and transaction Hash method to convert from display order bytes

* Update test coverage docs

* Add debugging output for mempool transaction verification

* Test fetching sent mempool transactions using gRPC

* Add extra log checks to the send transaction test

* Wait for zebrad mempool activation before running gRPC tests

* Update send transaction test for lightwalletd not returning mempool transactions

* Check zebrad logs instead of disabled lightwalletd gRPCs

* Add a debug option that makes RPCs pretend the sync is finished

* Remove an unused debug option

* Remove unused test code and downgrade some logs

* Fix test log checks

* Fix some rustdoc warnings

* Fix a compilation error due to new function arguments

* Make zebrad sync timeouts consistent and remove outdated code

* Document how to increase temporary directory space for tests

* Stop checking for a log that doesn't always happen

* Remove some commented-out code

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Update a comment about run time

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Add new config to new tests from the `main` branch

* Add transactions to the list, rather than replacing the list with each new block

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-09-06 13:32:33 +00:00
Alfredo Garcia 97fb85dca9
lint(clippy): add `unwrap_in_result` lint (#4667)
* `unwrap_in_result` in zebra-chain crate

* `unwrap_in_result` in zebra-script crate

* `unwrap_in_result` in zebra-state crate

* `unwrap_in_result` in zebra-consensus crate

* `unwrap_in_result` in zebra-test crate

* `unwrap_in_result` in zebra-network crate

* `unwrap_in_result` in zebra-rpc crate

* `unwrap_in_result` in zebrad crate

* rustfmt

* revert `?` and add exceptions

* explain some panics better

* move some lint positions

* replace a panic with error

* Fix rustfmt?

Co-authored-by: teor <teor@riseup.net>
2022-06-28 06:22:07 +00:00
Marek 6f896ef5a5
fix(doc): Fix the syntax of links in comments (#4494)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fa.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-05-30 20:12:11 +00:00
Conrado Gouvea 95f14ffdcd
add support for getblock with verbosity=1 (#4511) 2022-05-27 09:41:11 +00:00
Marek 7c726b246d
feat(rpc): Implement `z_gettreestate` RPC (#3990)
* Impl the elementary structure of the `z_gettreestate` RPC

* Fix merging bugs

* Fix a merge bug

* Fix a merge bug

* Move a derive attribute

Co-authored-by: teor <teor@riseup.net>

* Clarify the support of negative heights

* Add Orchard note commitment trees to the response

* Add the time to the response

* Finalize the `z_gettreestate` RPC

* Add a note that verified blocks have coinbase height

* Refactor `from_str` for `HashOrHeight`

* Fix a mistake in the docs

Co-authored-by: teor <teor@riseup.net>

* Clarify request types

Co-authored-by: teor <teor@riseup.net>

* Simplify `hash_or_height` conversion to height

Co-authored-by: teor <teor@riseup.net>

* Add a TODO about optimization

Co-authored-by: teor <teor@riseup.net>

* Add a doc comment

* Make sure Sapling & Orchard trees don't get mixed up

* Serialize Sapling commitment trees

* Refactor some comments

* Serialize Orchard commitment trees

* Serialize block heights

* Simplify the serialization of commitment trees

* Remove the block time from the RPC response

* Simplify the serialization of block heights

* Put Sapling & Orchard requests together

* Remove a redundant TODO

* Add block times to the RPC response

* Derive `Clone, Debug, Eq, PartialEq` for `GetTreestate`

Co-authored-by: teor <teor@riseup.net>

* Derive `Clone`, `Debug`, `Eq` and `PartialEq` for `SerializedTree`

* Document the fields of `GetTreestate`

* Skip the serialization of empty trees

This ensures compatibility with `zcashd` in the `z_gettreestate` RPC.

* Document the `impl` of `merkle_tree::Hashable` for nodes

* Make the structure of the JSON response consistent with `zcashd`

* Derive `Eq` for nodes

Co-authored-by: teor <teor@riseup.net>

* Convert Sapling commitment trees to a format compatible with zcashd

* Refactor the conversion of Sapling commitment trees

* Refactor some comments

* Refactor comments

* Add a description of the conversion

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Fix comment indenting

* Document the conversion between the dense and sparse formats

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2022-05-12 07:00:12 +00:00
teor 12e8130941
fix(state): return non-finalized UTXOs and tx IDs in address queries (#4356)
* Assert that address TxIDs and UTXOs are in chain order

* Always output colour by default in zcash-rpc-diff

* Cross-check getaddressutxos and getaddressbalance in zcash-rpc-diff

* Make balances with no UTXOs match in zcash-rpc-diff

* Add some TODOs

* Display the actual connected node software in zcash-rpc-diff

* Log address UTXOs request summaries

* Log address count for address UTXO requests

* Simplify zcash-rpc-diff node names

* Log chain address UTXOs request processing

* Stop ignoring all non-finalized UTXOs in address queries

* Make zcash-rpc-diff node names more consistent

* Downgrade logs to debug level

* Stop ignoring all non-finalized tx IDs in address queries
2022-05-11 21:43:17 +00:00
Alfredo Garcia d0e81001bc
fix(rpc): Use a structure for parameters of getaddresstxids (#4264)
* add info to lightwalletd_state_path()

* fix getaddresstxids rpc

* normalize more the env vars in lightwalletd tests

* extra logging

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2022-05-05 01:08:27 +00:00
teor be4e065afb
change(rpc): Simplify RPC types and add documentation (#4218)
* Simplify RPC types and add documentation

* Derive serde traits in production code
2022-05-03 04:10:21 +00:00
teor 7506655774
fix(rpc): Fix some RPC response formats to match `zcashd` (#4217)
* Match `zcashd`'s version format in `getinfo`

* Remove an incorrect array wrapper in getaddressbalance

* Use a stable sort order in getrawmempool

Because we can't match zcashd's arbitrary order,
and probably shouldn't try to.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-28 10:19:02 +00:00
Alfredo Garcia 56aabb1db1
feat(rpc): Implement `getaddressutxos` RPC method. (#4087)
* implement display for `Script`

* implement `getaddressutxos`

* fix space

* normalize list of addresses as argument to rpc methods

* implement `AddressStrings`

* make a doc clearer

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-25 03:00:52 +00:00
teor c2430c6f45
feat(rpc): implement the get_address_tx_ids RPC method query (#4119)
* Add a finalized state txids query

* Add an address transaction IDs query, without height filters

* Connect the address transaction ID query to the RPC

* Basic filtering of address transaction IDs by height range

* Add a network and range argument to the getaddresstxids test

* Test all block range combinations for mainnet

* Fix a file descriptor limit error

* Optimise seeking the first transaction for an address

The first transaction's location is part of the address location.

* Filter finalized address transaction IDs by height range

* Filter non-finalized address transaction IDs by the height range

* Fix up snapshot tests for the new height range API
2022-04-21 20:19:26 +00:00
Janito Vaqueiro Ferreira Filho e5f00c5902
feat(rpc): Implement `getaddressbalance` RPC (#4138)
* Add `Amount::serialize_as_string` helper method

A helper method that makes it easier to serialize an `Amount` as a
string. This is needed for the response type of the `getaccountbalance`
RPC.

* Implement state service call for address balance

Add `Read{Request,Response}::AddressBalance` variants and implement the
handler that calls the query function.

* Create an `AddressBalance` response type

Only contains the `balance` field which is needed by `lightwalletd`.
That field is serialized as a string, following the RPC specification.

* Implement `get_address_balance` RPC

Query the read-only state service for the information, and wrap it in an
`AddressBalance` response type so that it is serialized correctly.

* Run `rustfmt` inside `proptest!` block

Fix some minor formatting details.

* Test `get_address_balance` with valid addresses

Check that the RPC leads to a query to the mocked state service for a
balance amount.

* Test `get_address_balance` with invalid addresses

An error message should be returned by the RPC.

* Rename metric to `address_balance`

Keep it consistent with how it's named in other places.

Co-authored-by: teor <teor@riseup.net>

* Revert "Add `Amount::serialize_as_string` helper method"

This reverts commit 01b432e3d2.

* Serialize amount as an integer

This is different from what the documentation says, but it's what
lightwalletd expects.

* Add reference to RPC documentation

Make sure it is linked to for easy access.

* Create an `AddressStrings` type

To be used as the input for the `get_address_balance` RPC method.

* Use `AddressStrings` in `get_address_balance` RPC

Fix the input parameter so that the list of address strings is placed
inside a JSON map.

* Update property tests to use `AddressStrings`

Make sure the proper input type is created.

Co-authored-by: teor <teor@riseup.net>
2022-04-20 18:27:00 +00:00
Alfredo Garcia 7b7d22aabc
feat(rpc): Implement what we can of `getaddresstxids` RPC method. (#4062)
* implement `getaddresstxids` rpc method with dummy empty response

* use already public function

* fix some docs

* pass a list of addresses to the state request

* sync range errors with zcashd

* refactor a loop

* fix grammar

* fix tests

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2022-04-13 08:48:13 +00:00
Alfredo Garcia d09769714f
feat(rpc): Implement an RPC transaction queue (#4015)
* Add a rpc queue

* Implement the rpc queue

* Add rpc queue tests

* Remove mutex, use broadcast channel

* Have order and limit in the queue

* fix multiple transactions channel

* Use a network argument

* Use chain tip to calculate block spacing

* Add extra time

* Finalize the state check test

* Add a retry test

* Fix description

* fix some docs

* add additional empty check to `Runner::run`

* remove non used method

* ignore some errors

* fix some docs

* add a panic checker to the queue

* add missing file changes for panic checker

* skip checks and retries if height has not changed

* change constants

* reduce the number of queue test cases

* remove suggestion

* change best tip check

* fix(rpc): Check for panics in the transaction queue (#4046)

* Check for panics in the RPC transaction queue

* Add missing pin! and abort in the start task

* Check for transaction queue panics in tests

* Fixup a new RPC test from the main branch

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-12 15:06:29 +10:00