Commit Graph

13 Commits

Author SHA1 Message Date
Marek 4e29b097af
Remove the `getblocktemplate-rpcs` Cargo feature (#9401)
* Remove the `getblocktemplate-rpcs` Cargo feature

* Keep the feature in `zebrad/Cargo.toml`
2025-04-10 18:29:48 +00:00
Alfredo Garcia 1238ec0c63
tests(scanner): Move zebra scanner tests to binary (#8659)
* readd the scanner tests in the new binary context

* remove commented out tests from zebrad

* add Cargo.lock

* add local copy of `ZECPAGES_SAPLING_VIEWING_KEY` to binary scanning tests

* moves scanner-results-reader to zebra-scan, adds zebrad bin to zebra-scan, adds a short sleep before trying to connect to gRPC server in test

* always include zebrad dependency in zebra-scan

* renames zebrad bin in zebra-scan to zebrad-for-scanner

* skip test when a cached state is missing

* remove scanning config from zebrad

* move `scan_task_commands` out of zebrad acceptance tests

* remove scanner config from zebrad

* do not panic if env var is not there

* re enable start_scan_where left, remove shielded-scan feature foir scanning tests in CI

* change app names

* uncomment test

* fix the scanner tests in CI by changing the test command

* add log needed by CI?

* move test

* rename trait, docs fix

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-07-15 21:15:04 +00:00
Arya 8a786fe6ce
add(consensus): Adds block construction and validation for NU5 from block height 1 for Regtest (#8475)
* Always activate Canopy at Height(1) on Regtest

* Ignores the zip 212 grace period on configured Testnets and Regtest

* - Returns early when there is no Heartwood activation height when creating or updating HistoryTree
- Skips call to `check::legacy_chain()` when no NU5 activation height is set (it would return immediately anyway)
- Replaces `.map()` with `.filter_map()` in `NetworkUpgrade::target_spacings()`
- Removes outdated TODO

* - When proof of work is disabled, skips checking if Zebra is synced to the network tip in the getblocktemplate method

* Sets full_verifier_utxo_lookahead to Height::MIN instead of panicking

* When network is regtest, skips starting sync task and commits the genesis block if it's missing in the state

* updates/fixes test config

* Adds test for committing Canopy blocks on Regtest

* - Updates median time past and difficulty checks to use fewer than 11/17/28 blocks

* uses SLOW_START_INTERVAL of 0 if PoW is disabled, adds TODOs

* Update getblocktemplate method to return reserved chain history activation root hash, uses Nu5 at height 1

Test passes.

* Updates test to expect NU5 as the default nu activation at Height(1)

* Removes invalid difficulty snapshot

* fixes tests

* removes regtest NU5 activation height config field

* Apply suggestions from code review

* Update zebra-state/src/service/check/difficulty.rs

Co-authored-by: Marek <mail@marek.onl>

* Update zebra-state/src/service/check/difficulty.rs

* Updates docs/comments, renames an argument

---------

Co-authored-by: Marek <mail@marek.onl>
2024-05-02 19:43:42 +00:00
Arya 2a004ffe9e
add(scan): Handle RegisterKeys messages in scan task (#8222)
* moves ScanTask to its own module in service module

* moves `process_messages()` method to scan_task.rs

* uses get_mut() and returns new keys

* updates types and adds scan_until() function

* adds and uses wait_for_height() function

* spawns scan_until_task_handler to poll scan_until_tasks FuturesUnordered

* corrects comment

* adds TODO

* updates expected test log, corrects panic msg, fixes fmt

* moves scan functions to scan_task module

* moves ScanTaskCommand and related method impls to its own module

* moves `ScanTask::mock()` test constructor to its own module

* creates `add_keys` mod and moves `scan_until()` function there

* moves scan task executor to its own module and adds ScanRangeTaskBuilder type

* renames add_keys to scan_range, moves executor to scan_task mod

* adds test for process_messages

* updates scan_height_and_store_results() to skip last_scanned_height check if there is no key height for a key in `parsed_keys`

* updates `default_test_config()`

* adds acceptance test for registering keys in ScanTask

* uses the right feature for the new acceptance test

* Applies suggestions from code review
2024-02-06 18:41:00 +00:00
teor 1461c912f9
change(ci): Generate mainnet checkpoints in CI (#6550)
* Add extra test type modes to support zebra-checkpoints

* Add Mainnet and Testnet zebra-checkpoints test harnesses

* Add zebra-checkpoints to test docker images

* Add zebra-checkpoints test entrypoints

* Add Mainnet CI workflow for zebra-checkpoints

* Enable zebra-checkpoints feature in the test image

* Use the same features for (almost) all the docker tests

* Make workflow features match Docker features

* Add a feature note

* Add a zebra-checkpoints test feature to zebrad

* Remove the "no cached state" testnet code

* Log a startup message to standard error when launching zebra-checkpoints

* Rename tests to avoid partial name conflicts

* Fix log formatting

* Add sentry feature to experimental docker image build

* Explain what ENTRYPOINT_FEATURES is used for

* Use the correct zebra-checkpoints path

* Silence zebrad logs while generating checkpoints

* Fix zebra-checkpoints log handling

* Re-enable waiting for zebrad to fully sync

* Add documentation for how to run these tests individually

* Start generating checkpoints from the last compiled-in checkpoint

* Fix clippy lints

* Revert changes to TestType

* Wait for all the checkpoints before finishing

* Add more stderr debugging to zebra-checkpoints

* Fix an outdated module comment

* Add a workaround for zebra-checkpoints launch/run issues

* Use temp dir and log what it is

* Log extra metadata about the zebra-checkpoints binary

* Add note about unstable feature -Z bindeps

* Temporarily make the test run faster and with debug info

* Log the original test command name when showing stdout and stderr

* Try zebra-checkpoints in the system path first, then the cargo path

* Fix slow thread close bug in dual process test harness

* If the logs are shown, don't say they are hidden

* Run `zebra-checkpoints --help` to work out what's going on in CI

* Build `zebra-utils` binaries for `zebrad` integration tests

* Revert temporary debugging changes

* Revert changes that were moved to another PR
2023-04-27 04:39:43 +00:00
teor d3ce022ecc
change(utils): Add a direct connection mode to zebra-checkpoints (#6516)
* Rename variables so it's clearer what they do

* Fully document zebra-checkpoints arguments

* Remove some outdated references to zcashd

* Add a json-conversion feature for converting JSON to valid Heights

* Simplify zebra-checkpoints code using conversion methods

* Track the last checkpoint height rather than the height gap

* Move all the CLI-specific code into a single function

* Remove cfg(feature) from the test RPC client API

* Move the RpcRequestClient into zebra-node-services

* Fix the case of RpcRequestClient

* Add transport and addr arguments to zebra-checkpoints

* Make zebra-checkpoints support both CLI and direct JSON-RPC connections

* Fix RpcRequestClient compilation

* Add a suggestion for zcashd authentication failures

* Document required features

* Handle differences in CLI & direct parameter and response formats

* Replace a custom function with an existing dependency function

* Add a checkpoint list test for minimum height gaps
2023-04-26 23:35:53 +00:00
Arya 61af406d35
change(tests): adds RPC request client for condensing shared code (#5619)
* Adds RPCRequestClient

* uses RPCRequestClient in the rest of the rpc calls

* removes duplicate expect call

* fixed mistaken "get_info" method call with "getinfo"

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-16 21:23:29 +00:00
Arya c447b03223
fix(tests): add submitblock test to CI, and avoid copying the cached state directory in other tests (#5589)
* updates mod docs for tests that use future blocks

* updates submitblock test to use TestType methods

* prunes redundant code

* adds check_sync_logs_until

* adds assertion for needs cached state & rpc server

* updates get_raw_future_blocks fn with rpc calls

* updates to get_raw_future_blocks fn and submit_block test

* Rename LightwalletdTestType to TestType

* moves TestType and random_known_rpc_port_config to test_type.rs and config.rs

* moves get_raw_future_blocks to cached_state.rs

* updates ci workflows to include submit block test

* adds get_future_blocks fn and uses it in load_transactions_from_future_blocks

* updates CI docker

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Applies suggestions from code review

* Updates misnamed closure param

* updates mod docs for test_type.rs

Co-authored-by: teor <teor@riseup.net>
2022-11-10 03:40:21 +00:00
Arya 2f3b05f8e1
change(rpc): add submitblock RPC method (#5526)
* adds submitblock rpc method

* re-orders imports

* replaces thread::yield_now with async yield_now

* Fix doc warnings and unused variable warnings, add missing docs

* Mark work_id as optional

* Use the same ChainVerifier for downloaded and submitted blocks

* Revert unused changes & minor cleanups

* Document currently-unreachable code

* updates tests and submit_block response for AlreadyVerified error

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* changes names from BlockVerifier to ChainVerifier and block_verifier to chain_verifier to keep it consistent with naming in zebra-consensus

* move how to run the submit_block test example to acceptance.rs

* updates snapshot tests

* moved acceptance test to a separate file

* removes extra tower::ServiceBuilder::new(), updates docs

* updates vectors and snapshot tests, changes hex decoding error in submit_block method from server error to parse error

* hides errors module in zebra-rpc behind a feature flag and adds docs.

* Updates snapshot test, adds mod docs, moves HexData to its own mod, and removes the unrelated make_server_error refactor for now

* update submit block acceptance test mod doc

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-04 03:57:08 +00:00
Marek 6f896ef5a5
fix(doc): Fix the syntax of links in comments (#4494)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fa.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-05-30 20:12:11 +00:00
teor 59bdab17bf
test(rpc): Add Rust tests for lightwalletd sync from Zebra (#4177)
* Make the lightwalletd integration test take a test type

* Configure lightwalletd tests based on the test type

* Remove obsolete kill_on_error() in the lightwalletd test

* Refactor to simplify the test function

* Move LightwalletdTestType to the lightwalletd module

* Create a test function that runs the full lightwalletd test suite

* Actually use the cached Zebra state

* Add checks for the new integration test modes

* Populate the lightwalletd state dir in the FullSyncFromGenesis test

* Fix up state handling, fail earlier if state is invalid

* Adjust timeouts and regex escapes

* Make state requirements for each test stricter

* Move configs to the top of the test function

* Allow unexpected lightwalletd cached state in some tests

* Speed up tests slightly by removing an intermittent log check

* Move timeout selection into test type methods

* Move failure messages into test type methods

* Turn a function argument into an enum field

* Check lightwalletd state directly, rather than Zebra RPC results

* Update gRPC tests for function argument changes

* Remove duplicate env var constant and redundant code
2022-04-29 19:56:11 +00:00
Janito Vaqueiro Ferreira Filho 5a94a09292
add(test): Integration test to send transactions using lightwalletd (#4068)
* Export the `zebra_state::Config::db_path` method

Make it easier for tests to discover the sub-directory used to store
the chain state data.

* Generate code for interfacing with lightwalletd

Use the `tonic-build` crate to generate Rust code for communicating with
lightwalletd using gRPC.

The `*.proto` files were obtained from the Zcash lightwalletd
repository.

* Use `block::Height` instead of `Height`

Import the `block` instead to make it slightly clearer.

* Add helper function to remove a file if it exists

Try to remove it and ignore an error if it says that the file doesn't
exist. This will be used later to remove the lock file from a copied
chain state directory.

* Add helper function to copy chain state dirs

Copy an existing chain state directory into a new temporary directory.

* Add a `BoxStateService` type alias

Make it easier to write and read a boxed version of a state service.

* Add a helper function to start the state service

Make it easier to specify the state service to use an existing state
cache directory.

* Import `eyre!` macro at the module level

Allow it to be used in different places without having to repeat the
imports.

* Add `load_tip_height_from_state_directory` helper

A function to discover the current chain tip height stored in a state
cache.

* Add helper function to prepare partial sync. state

Loads a partially synchronized cached state directory into a temporary
directory that can be used by a zebrad instance, and also returns the
chain tip block height of that state.

* Add `perform_full_sync_starting_from` helper

Runs a zebrad with an existing partially synchronized state, and
finishes synchronizing it to the network chain tip.

* Add function to load transactions from a block

Use a provided state service to load all transactions from a block at a
specified height.

The state service is a generic type parameter, because
`zebra_state::service::ReadStateService` is not exported publicly. Using
a generic type parameter also allows the service to be wrapped in layers
if needed in the future.

* Add `load_transactions_from_block_after` helper

A function to load transactions from a block stored in a cached state
directory. The cached state must be synchronized to a chain tip higher
than the requested height.

* Add helper function to load some test transactions

Given a partially synchronized chain state, it will extend that chain by
performing a full synchronization, and obtain some transactions from one
of the newly added blocks.

* Update `spawn_zebrad_for_rpc_without_initial_peers`

Wait until the mempool is activated.

* Add method to start lightwalletd with RPC server

Returns the lightwalletd instance and the port that it's listening for
RPC connections.

The instance can reuse an existing cached lightwalletd state if the
`LIGHTWALLETD_DATA_DIR` environment variable is set.

* Add a `LightwalletdRpcClient` type alias

To make it easier to identify the type generated from the Protobuf
files.

* Add helper function to connect to lightwalletd

Prepare an RPC client to send requests to a lightwalletd instance.

* Add a `prepare_send_transaction_request` helper

Creates a request message for lightwalletd to send a transaction.

* Add test to send transactions using lightwalletd

Obtain some valid transactions from future blocks and try to send them
to a lightwalletd instance connected to a zebrad instance that hasn't
seen those transactions yet. The transactions should be successfully
queued in Zebra's mempool.

* Make `zebra_directory` parameter generic

Allow using a `TempDir` or a `PathBuf`.

* Move lightwalletd protobuf files

Place them closer to the module directory, so that it's clearer that
they specify the RPC protocol for lightwalletd, and not Zebra itself.

* Don't use coinbase transactions in the test

Coinbase transactions are rejected by the mempool.

* Don't remove state lock file

It is removed automatically by Zebra when it shuts down, so if it exists
it should be reported as a bug.

* Force mempool to be enabled in Zebrad instance

Speed up the initialization of the Zebrad instance used for lightwalletd
to connect to.

* Refactor to create `LIGHTWALLETD_DATA_DIR_VAR`

Document how the environment variable can be used to speed up the test.

* Check for process errors in spawned Zebra instance

Enable checking for known process failure messages.

* Add `FINISH_PARTIAL_SYNC_TIMEOUT` constant

Document why it exists and how the choice of the value affects the test.

* Add `LIGHTWALLETD_TEST_TIMEOUT` constant

And use it for the Zebrad and the Lightwalletd instances used in the
send transaction integration test.

* Check `lightwalletd` process for errors

Enable checking the lightwalletd process for known failure messages.

* Update `tonic` and `prost` dependencies

Use the latest version and fix CI failures because `rustfmt` isn't
installed in the build environment.

* Create `send_transaction_test` module

Move the send transaction using lightwalletd test and its helper
functions into a new module.

* Move `LIGHTWALLETD_TEST_TIMEOUT` constant

Place it in the parent `lightwalletd` module.

* Move gRPC helper functions and types to `rpc` mod.

Make them more accessible so that they can be used by other tests.

* Create a `cached_state` module

Move the test utility functions related to using a cached Zebra state
into the module.

* Move `perform_full_sync_starting_from` to `sync`

Keep to closer to the synchronization utility functions.

* Move Zebra cached state path variable constant

Place it in the `cached_state` module.

* Skip test if `ZEBRA_TEST_LIGHTWALLETD` is not set

Make it part of the set of tests ignored as a whole if no lightwalletd
tests should be executed.

* Move `spawn_zebrad_for_rpc_without_initial_peers`

Place it in the `launch` sub-module.

* Rename `rpc` module into `wallet_grpc`

Avoid any potential misunderstandings when the name is seen out of
context.

* Allow duplicate `heck` dependency

At least until `structopt` is updated or `zebra-utils` is updated to use
`clap` 3.

* Fix a deny.toml typo

* fix(build): CMake is required by `prost` crate

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-04-27 23:06:11 +00:00
teor 9a8ab9468d
T0. refactor(test): split zebrad acceptance tests into sub-modules (#3901)
* Improve launch delay docs

* Initial split of zebrad acceptance tests into modules

* Split shared lightwalletd test code into a module
2022-03-18 16:02:22 +00:00