Commit Graph

13 Commits

Author SHA1 Message Date
teor 2ac6921d60
feat(mine): Add an internal Zcash miner to Zebra (#8136)
* Patch equihash to use the solver branch

* Add an internal-miner feature and set up its dependencies

* Remove 'Experimental' from mining RPC docs

* Fix a nightly clippy::question_mark lint

* Move a byte array utility function to zebra-chain

* fixup! Add an internal-miner feature and set up its dependencies

* Add an equihash::Solution::solve() method with difficulty checks

* Check solution is valid before returning it

* Add a TODO to check for peers before mining

* Move config validation into GetBlockTemplateRpcImpl::new()

* fixup! fixup! Add an internal-miner feature and set up its dependencies

* Use the same generic constraints for GetBlockTemplateRpcImpl struct and impls

* Start adding an internal miner component

* Add the miner task to the start command

* Add basic miner code

* Split out a method to mine one block

* Spawn to a blocking thread

* Wait until a valid template is available

* Handle shutdown

* Run mining on low priority threads

* Ignore some invalid solutions

* Use a difference nonce for each solver thread

* Update TODOs

* Change the patch into a renamed dependency to simplify crate releases

* Clean up instrumentation and TODOs

* Make RPC instances cloneable and clean up generics

* Make LongPollId Copy so it's easier to use

* Add API to restart mining if there's a new block template

* Actually restart mining if there's a new block template

* Tidy instrumentation

* fixup! Move config validation into GetBlockTemplateRpcImpl::new()

* fixup! Make RPC instances cloneable and clean up generics

* Run the template generator and one miner concurrently

* Reduce logging

* Fix a bug in getblocktemplate RPC tip change detection

* Work around some watch channel change bugs

* Rate-limit template changes in the receiver

* Run one mining solver per available core

* Use updated C code with double-free protection

* Update to the latest solver branch

* Return and submit all valid solutions

* Document what INPUT_LENGTH means

* Fix watch channel change detection

* Don't return early when a mining task fails

* Spawn async miner tasks to avoid cooperative blocking, deadlocks, and improve shutdown responsiveness

* Make existing parallelism docs and configs consistent

* Add a mining parallelism config

* Use the minimum of the configured or available threads for mining

* Ignore optional feature fields in tests

* Downgrade some frequent logs to debug

* Document new zebrad features and tasks

* Describe the internal-miner feature in the CHANGELOG

* Update dependency to de-duplicate equihash solutions

* Use futures::StreamExt instead of TryStreamExt

* Fix a panic message typo
2024-01-11 14:41:01 +00:00
teor 68d7198e9f
Re-order Zebra startup, so slow services are launched last (#3091)
* Start network before verifiers

This makes the Groth16 download task start as late as possible.

* Explain why the Groth16 download must happen first

* Speed up Zebra shutdown: skip waiting for the tokio runtime
2021-11-23 17:42:44 +00:00
teor 375a997d2f
Stop downloading unnecessary blocks in Zebra acceptance tests (#3072)
* Implement graceful shutdown for the peer set

* Use the minimum lookahead limit in acceptance tests

* Enable a doctest that compiles with newly public modules
2021-11-19 01:55:38 +00:00
teor 04d2cfb3d0
Gossip recently verified block hashes to peers (#2729)
* Implement a task that gossips verified block hashes

* Log an info message for block broadcasts

* Simplify the gossip task

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Re-use the old tip change if there is no new tip change

Also improve the comments.

* Add an assertion message

* Rename task join handles and futures in start method

* Add a dedicated BlockGossipError type

This type helps distinguish between syncer and state errors.

* Test that committed blocks are gossiped to peers

Also do a minor type cleanup on the existing test code,
replacing `Option<Vec<_>>` with `Vec<_>`.

* Formatting

* Remove excess newlines

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Clear the initial gossiped blocks during test setup

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-10-07 07:46:37 -03:00
Janito Vaqueiro Ferreira Filho 1f1bf2ec4d
Replace `mock_peer_set` function with `MockService` (#2790)
* Use `MockService` in inbound test

Refactor the `mempool_requsets_for_transactions` test so that it uses a
`MockService` instead of the `mock_peer_set` function.

* Use `MockService` in the basic mempool test

Refactor the `mempool_service_basic` test so that it uses a
`MockService` instead of the `mock_peer_set` helper function.

* Remove the `mock_peer_set` helper function

It is not used anymore, since the usages were replaced with
`MockService`s.

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-09-22 15:44:13 +00:00
Conrado Gouvea 8825a52bb8
Move transaction download and verify stream into the mempool service (#2741)
* Move transaction dowloader and verifier into the mempool service

* add test for `Storage::contains_rejected()`

* Rename DownloadAndVerify->Queue; move should_download_or_verify() to previous impl

* GossipedTx -> Gossip

* Revamp error handling

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-09-13 16:28:07 -04:00
Janito Vaqueiro Ferreira Filho 069f7716db
Create initial transaction crawler for the mempool (#2646)
* Create initial `mempool::Crawler` type

The mempool crawler is responsible for periodically asking peers for
transactions to insert into the local mempool. This initial
implementation will periodically ask for transactions, but won't do
anything with them yet.

Also, the crawler is currently configured to be always enabled, but this
should be fixed to avoid crawling while Zebra is still syncing the
chain.

* Add a timeout to peer responses

Prevent the crawler from getting stuck if there's communication with a
peer that takes too long to respond.

* Run the mempool crawler in Zebra

Spawn a task for the crawler when Zebra starts.

* Test if the crawler is sending requests

Create a mock for the `PeerSet` service to intercept requests and verify
that the transaction requests are sent periodically.

* Use `full` Tokio features when testing

Make it simpler to select the features for test builds.

Co-authored-by: teor <teor@riseup.net>

* Link to the issue for crawler activation

Make it easy to navigate from the `TODO` comment to the current project
planning.

Co-authored-by: teor <teor@riseup.net>

* Link to the issue for downloading transactions

Make it easy to navigate from the `TODO` comment to the current project
planning.

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2021-08-24 11:23:53 -03:00
Henry de Valence 4b35fea492 zebrad: document Inbound, ChainSync responsibilities 2020-09-18 18:34:25 -07:00
Henry de Valence 1d0ebf89c6 zebrad: move seed command into inbound component
Remove the seed command entirely, and make the behavior it provided
(responding to `Request::Peers`) part of the ordinary functioning of the
start command.

The new `Inbound` service should be expanded to handle all request
types.
2020-09-18 18:34:25 -07:00
Henry de Valence 9b6e66c1b9 zebrad: rename Syncer to ChainSync
This name clarifies what is being synced and avoids an agent-noun
construction.
2020-09-10 09:45:52 -07:00
Henry de Valence 0bc79686b8 zebrad: move sync into components module.
Part of #1030.
2020-09-10 09:45:52 -07:00
Henry de Valence 75d3d44fb3 Metrics MVP: add two metrics and export them to Prometheus.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2020-02-14 20:14:05 -05:00
Deirdre Connolly 162b37fe8d Tracing endpoint (#3)
* Add a TracingConfig and some components

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Restructure, use dependency injection, initialize tracing

* Start a placeholder loop in start command

* Add hyper alpha.1, bump tokio to alpha.4

* Hello world endpoint using async/await from hyper 0.13 alpha

Also cleaned up some linter messages.

Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>

* Update to tracing_subscriber 0.1

* fmt

* add rust-toolchain

* Remove hyper::Version import

* wip: start filter_handler impl

* Add .rustfmt.toml

* rustfmt

* Tidy up .rustfmt.toml

* Add filter reloading handling.

* bump toolchain

* Remove generated hello world acceptance tests.

These test the behaviour of the autogenerated binary and work as examples of
how to test the behaviour of abscissa binaries.  Since we don't print "Hello
World" any more, they fail, but we don't yet have replacement behaviour to add
tests for, so they're removed for now.

* Clean up config file handling with Option::and_then.
2019-09-09 13:05:42 -07:00