Commit Graph

81 Commits

Author SHA1 Message Date
idky137 9b91d4bc0e
change(tests): Remove Matches on Network From Tests (#8295)
* added functions for fetching block vectors

* inserted new network methods for vector fetching into zebra-chain

* changed tag back to test/feature=branch

* changed tag back to test/feature=branch

* changed tag back to test/feature=branch

* changed feature tag to proptest-impl, started implementing in zebra-consensus tests

* adding methods to zebra-consensus, lifetime error in src/transaction/tests.rs, needs refactoring

* finished adding methods to zebra-consensus

* finished adding new methods to zebrad

* added new methods to zebra-rpc and zebra-scan

* finished removing statements matching on Network from tests

* updated new error message

* changed get_block_bytes() and get_block_sapling_roots_bytes to return option and removed serialization error types as per requested changes in PR review

* removed match statements from zebra_chain::transaction::arbitrary::test_transactions() and new zebra-grpc tests

* moved zebra-chain::test_utils to zebra-chain::test

* removed get_ prefix from getter methods

* renamed zebra-chain::test to zebra-chain::tests

* renamed zebra-chain::test to zebra-chain::tests

* fixed clippy warnings

* changed block_map to return clone
2024-03-05 09:12:25 -05:00
Alfredo Garcia d45864fa46
feat(remove): Temporally remove the internal miner functionality (#8184)
* remove functionality of internal miner

* fix configs

* fix typo in comment

* typo

Co-authored-by: Arya <aryasolhi@gmail.com>

* remove internal config for tests to pass

* typo

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-01-23 18:02:52 +00:00
teor 2ac6921d60
feat(mine): Add an internal Zcash miner to Zebra (#8136)
* Patch equihash to use the solver branch

* Add an internal-miner feature and set up its dependencies

* Remove 'Experimental' from mining RPC docs

* Fix a nightly clippy::question_mark lint

* Move a byte array utility function to zebra-chain

* fixup! Add an internal-miner feature and set up its dependencies

* Add an equihash::Solution::solve() method with difficulty checks

* Check solution is valid before returning it

* Add a TODO to check for peers before mining

* Move config validation into GetBlockTemplateRpcImpl::new()

* fixup! fixup! Add an internal-miner feature and set up its dependencies

* Use the same generic constraints for GetBlockTemplateRpcImpl struct and impls

* Start adding an internal miner component

* Add the miner task to the start command

* Add basic miner code

* Split out a method to mine one block

* Spawn to a blocking thread

* Wait until a valid template is available

* Handle shutdown

* Run mining on low priority threads

* Ignore some invalid solutions

* Use a difference nonce for each solver thread

* Update TODOs

* Change the patch into a renamed dependency to simplify crate releases

* Clean up instrumentation and TODOs

* Make RPC instances cloneable and clean up generics

* Make LongPollId Copy so it's easier to use

* Add API to restart mining if there's a new block template

* Actually restart mining if there's a new block template

* Tidy instrumentation

* fixup! Move config validation into GetBlockTemplateRpcImpl::new()

* fixup! Make RPC instances cloneable and clean up generics

* Run the template generator and one miner concurrently

* Reduce logging

* Fix a bug in getblocktemplate RPC tip change detection

* Work around some watch channel change bugs

* Rate-limit template changes in the receiver

* Run one mining solver per available core

* Use updated C code with double-free protection

* Update to the latest solver branch

* Return and submit all valid solutions

* Document what INPUT_LENGTH means

* Fix watch channel change detection

* Don't return early when a mining task fails

* Spawn async miner tasks to avoid cooperative blocking, deadlocks, and improve shutdown responsiveness

* Make existing parallelism docs and configs consistent

* Add a mining parallelism config

* Use the minimum of the configured or available threads for mining

* Ignore optional feature fields in tests

* Downgrade some frequent logs to debug

* Document new zebrad features and tasks

* Describe the internal-miner feature in the CHANGELOG

* Update dependency to de-duplicate equihash solutions

* Use futures::StreamExt instead of TryStreamExt

* Fix a panic message typo
2024-01-11 14:41:01 +00:00
Marek 9acdf0e512
Make the `verbose` argument optional (#8076)
The `verbose` argument of the `getrawtransaction` RPC is specified as
optional in the documentation:
https://zcash.github.io/rpc/getrawtransaction.html.
2023-12-10 21:44:43 +00:00
Arya 0a3790b73e
change(consensus): Remove Sprout and Sapling parameter download task and debug_skip_preload config (#7844)
* removes groth16 download task

* updates docs.

* Adds new config to test configs

* fixes compatibility with past configs

* uses inner config to deserialize removed field for compatibility

* update docs

* updates latest config

* Applies suggestions from code review

* Avoid duplicating hard-coded default values

---------

Co-authored-by: teor <teor@riseup.net>
2023-10-27 06:12:57 +00:00
Arya 01168c8571
change(state): Set upper bound when reading from deleting column family tx_loc_by_transparent_addr_loc (#7732)
* Uses range_iter in address_transaction_locations

* Uses range_iter in address_transaction_locations

* uses u16::MAX instead of usize::MAX

* Moves limit code into method

* adds allow(dead_code)

* Simplifies address_iterator_range

* Moves test state init out of loop

* Updates docs
2023-10-18 06:16:29 +00:00
teor 758eb6e0ea
fix(rpc): Fix bugs and performance of `getnetworksolps` & `getnetworkhashps` RPCs (#7647)
* Handle negative and zero getnetworksolsps arguments correctly

* Simplify the solsps loop structure

* Simplify loop by avoiding manual iteration and peeking

* Avoid division by zero

* Use min and max times rather than first and last times

* Refactor block iterators so they are more efficient

* Make finding chains easier

* Simplify block iteration code

* Remove implemented TODO comments

* Simplify internal iterator state

* Implement iteration by any chain item

* Iterate block headers rather than full blocks

* Ignore code that is (sometimes) dead

* Fix a dead code warning

* Add a no blocks in state error constant

* Check result values in the RPC test

* Fix invalid calculation handling
2023-10-11 02:02:51 +00:00
Alfredo Garcia 38caaafc02
change(docs): update docs for new lightwalletd version used in Zebra (#7427)
* upgrade docs for new lightwalletd repo zebra is using now

* Update book/src/user/lightwalletd.md

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2023-08-30 22:30:20 +00:00
Arya 2b81d845ea
fix(panic): Log a warning instead of panicking for unused mining configs (#7290)
* Logs warning for unused mining config

* add conditional serialization of the mining section

* rustfmt

* Simplify mining::Config argument passing

* Simplify mining config argument passing in start.rs

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2023-08-29 07:44:59 +00:00
Alfredo Garcia b322748b66
add(rpc): note tree sizes to `getblock` api (#7278)
* add the basics

* add some docs, move code

* upgrade compact formats to https://github.com/zcash/lightwalletd/blob/v0.4.15/walletrpc/compact_formats.proto

* add a test for in sync chain

* test changing to ecc lightwalletd

* revert change of lightwalletd repo (already merged to main)

* add debug log to see whats going on with the test

* change log to tracing::info

* remove log line

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-08-15 18:48:50 +00:00
Pili Guerra 7f64ff35a4
add(metrics): Track mempool actions and size bucketed by weight (copy of #6972, credit @str4d) (#7019)
* metrics: Track mempool actions and size bucketed by weight

* Fix tests

* draft fix tests

* fix `fix_arbitrary_generated_action_overflows`

* add some docs

* manually derive arbitrary

* remove unused import

---------

Co-authored-by: Jack Grigg <jack@electriccoin.co>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-07-18 23:04:32 +00:00
teor 147b8fa3a8
cleanup(rust): Fix new nightly clippy warnings (#7135)
* Fix "comparison is always true" warning

* Add missing Send bound

* cargo clippy --fix --all-features --all-targets

* incorrect implementation of clone on a Copy type

* cargo fmt --all
2023-07-05 07:11:27 +00:00
teor 56a76385f0
git ls-tree --full-tree -r --name-only HEAD | xargs sed -i -e 's/router_verifier/block_verifier_router/g' (#6998)
cargo fmt --all
2023-06-20 07:11:04 +00:00
teor c3f0f53256
refactor(app): De-duplicate and fix version handling code (#6996)
* De-duplicate app_version and user_agent code, rename to build_version

* Make RPC testnet flag forward-compatible with additional testnets

* Fix RPC tests with new argument

* Use "modified" rather than "dirty" for uncommitted changes in build metadata

* Split the vergen version into its own function
2023-06-20 02:42:06 +00:00
teor f3e330995f
fix(build): Fix new nightly clippy lints and cargo feature resolution (#6814)
* Implement minor and patch database format versions

* Log and update database format versions when opening database

* Refactor the current list of column families into a constant

* Open all available column families, including from future Zebra versions

* Refactor note commitment tree lookups to go through the height methods

* Make Sapling/Orchard note commitment tree lookup forwards compatible

* Ignore errors reading column family lists from disk

* Update format version comments and TODOs

* Correctly log newly created database formats

* Fix a new cargo lint about resolver versions

* cargo clippy --fix --all-features --all-targets

* cargo fmt --all

* Add missing tokio feature in the state, revealed by the new resolver

* Add missing dev dependencies in zebra-node-services

* Add a missing `tokio` feature from PR #6813

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-06-07 06:04:04 +00:00
Alfredo Garcia eb07bb31d6
rename(state): Rename state verifiers and related code (#6762)
* rename verifiers

* rename `PreparedBlock` to `SemanticallyVerifiedBlock`

* rename `CommitBlock` to `SemanticallyVerifiedBlock`

* rename `FinalizedBlock` to `CheckpointVerifiedBlock`

* rename `CommitFinalizedBlock` to `CommitCheckpointVerifiedBlock`

* rename `FinalizedWithTrees` to `ContextuallyVerifiedBlockWithTrees`

* rename `ContextuallyValidBlock` to `ContextuallyVerifiedBlock`

* change some `finalized` variables or function arguments to `checkpoint_verified`

* fix docs

* document the difference between `CheckpointVerifiedBlock` and `ContextuallyVerifiedBlock`

* fix doc links

* apply suggestions to request

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to service

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to finalized_state.rs and write.rs

Co-authored-by: Marek <mail@marek.onl>

* fmt

* change some more variable names

* change a few missing generics

* fix checkpoint log issue

* rename more `prepared` vars `semantically_verified`

* fix test regex

* fix test regex 2

---------

Co-authored-by: Marek <mail@marek.onl>
2023-06-01 12:29:03 +00:00
teor 8af4e572c9
fix(network): Ignore out of order Address Book changes, unless they are concurrent (#6717)
* Ignore out of order Address Book changes, and restructure the function

* Handle concurrent changes using the connection state machine order

* Handle out of order changes correctly

* Pass times through the call stack so they are consistent in tests

* Add time arguments to tests

* Fix tests that were broken by the address order checks

* fastmod wall_ local_ zebra*

* cargo fmt --all

* Fix a bug in the concurrent change check

* Test all the new apply and skip checks for address changes

* Document more edge cases and increase the concurrency time slightly

* Simplify enum ordering matches

* Fix comment typos

Co-authored-by: Arya <aryasolhi@gmail.com>

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2023-05-24 23:53:53 +00:00
teor b0d9471214
fix(log): Stop logging peer IP addresses, to protect user privacy (#6662)
* Add a PeerSocketAddr type which hides its IP address, but shows the port

* Manually replace SocketAddr with PeerSocketAddr where needed

```sh
fastmod SocketAddr PeerSocketAddr zebra-network
```

* Add missing imports

* Make converting into PeerSocketAddr easier

* Fix some unused imports

* Add a canonical_peer_addr() function

* Fix connection handling for PeerSocketAddr

* Fix serialization for PeerSocketAddr

* Fix tests for PeerSocketAddr

* Remove some unused imports

* Fix address book listener handling

* Remove redundant imports and conversions

* Update outdated IPv4-mapped IPv6 address code

* Make addresses canonical when deserializing

* Stop logging peer addresses in RPC code

* Update zebrad tests with new PeerSocketAddr type

* Update zebra-rpc tests with new PeerSocketAddr type

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-05-14 15:06:07 +00:00
Alfredo Garcia 7c67512cd5
feat(zebra-network): add user agent argument (#6601)
* add user agent as argument, use git to auto build zebra user agent

* try to fix test

* fix typo

* change expect text

* remove newline

* fix some docs

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2023-05-05 00:29:14 +00:00
Alfredo Garcia 58bd898f5b
feat(zebrad): Refuse to run zebrad when release is too old (#6351)
* refuse to run Zebra if it is too old

* update the release checklist to consider the constants

* bring newline back

* apply new end of support code

* attempt to add tests (not working yet)

* move eos to progress task

* move tests

* add acceptance test (not working)

* fix tests

* change to block height checks (ugly code)

* change warn days

* refactor estimated blocks per day, etc

* move end of support code to its own task

* change test

* fix some docs

* move constants

* remove uneeded conversions

* downgrade tracing

* reduce end of support time, fix ci changing debugs to info again

* update instructions

* add failure messages

* cargo lock update

* unify releaase name constant

* change info msg

* clippy fixes

* add a block explorer

* ignore testnet in end of support task

* change panic to 16 weeks

* add some documentation about end of support

* Tweak docs wording

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-04-28 14:13:21 +00:00
Arya d7842bd467
fix(rpc): Check that mempool transactions are valid for the state's chain info in getblocktemplate (#6416)
* check last seen tip hash from mempool in fetch_mempool_transactions()

* Moves last_seen_tip_hash to ActiveState

* fixes tests and tests fixes

* continues to the next iteration of the loop to make fresh state and mempool requests if called with a long poll id

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* adds allow[unused_variable) for linter

* expects a chain tip when not(test)

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Addresses comments in code review

* - Removes second call to `last_tip_change()`

- Fixes tests using enabled mempool

* Adds note about chain tip action requirement to test method `enable()`

* updates doc comment

* Update zebrad/src/components/mempool.rs

Co-authored-by: teor <teor@riseup.net>

* fixes test

---------

Co-authored-by: teor <teor@riseup.net>
2023-04-03 23:22:07 +00:00
Arya 8ba3fd921a
change(rpc): Add confirmations to getrawtransaction method response (#6287)
* adds confirmation field to getrawtransaction

* Updates tests

* Adds comment about correctness

* Apply suggestion revisions from review

* fixes overflow bug and adds test for valid confirmations value

* Update test to check that confirmations isn't too high

* Update transaction request to return confirmations

* Applies suggestions from PR review

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* fixes test

* restore derives that were lost in a bad merge

---------

Co-authored-by: teor <teor@riseup.net>
2023-03-26 23:53:44 +00:00
teor ec43d63ed2
change(log): Log a cute message for blocks that were mined by Zebra (off by default) (#6098)
* Mark Zebra coinbase transactions with extra coinbase data

* Log when we commit a block mined by Zebra to our state

* Reduce logging instrumentation during block writes

* Remove debug types in Zebra block log

* Add network and commit to write task logs

* Apply an allow-list before we log arbitrary user messages from blocks

* Rate-limit Zebra mined block logging to once every 1000 blocks

* Add mining configs for extra coinbase data and imitating zcashd, but don't use them yet

* Check CoinbaseData size limit when building transparent transactions

* Replace LIKE_ZCASHD constants with a config

* Take extra coinbase data from the configured string

* Update the zebrad configs in the tests with new config fields
2023-02-23 00:10:11 +00:00
Marek 83d038c067
feat(rpc): Implement the `z_validateaddress` RPC (#6185)
* Add the response type for `z_validateaddress`

* Add UAs to the address variants

* Remove a redundant TODO

* Impl `z_validateaddress`

* Add basic test vectors

* Add basic snapshots

* Remove a deprecated field from the RPC response

* Add basic snapshot data

* Check the semantic validity of UAs

* Refactor imports

* Refactor snapshot filenames

This PR removes the `.new` filename extensions from snapshots.

* Rename `address` to `unified_address`
2023-02-20 21:06:22 +00:00
Alfredo Garcia 31382d2a24
feat(rpc): Implement `z_listunifiedreceivers` (#6171)
* implement `z_listunifiedreceivers`

* add test vector

* add snapshots

* simplify sapling payment address

* send network argument to payment_address method

* use expect for impossible address errors

* remove network argument

* use already present network conversion

* add additional snapshot

* Derive common traits on the RPC struct

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-02-20 12:22:37 +00:00
teor ae21e36018
fix(rpc): Add `getblock` RPC fields to support the latest version of `zcash/lightwalletd` (#6134)
* Stabilise the BestChainBlockHash state request

* Always include the block hash in the getblock RPC response

* Make the lightwalletd integration tests compatible with zcash/lightwalletd

* Update getblock RPC snapshots

* Return the correct missing block error code
2023-02-14 08:52:58 +00:00
teor 4f289299fd
change(rpc): Simplify `getdifficulty` RPC implementation (#6105)
* Use existing Work type to calculate get_difficulty RPC, round in f64 tests

* Use a valid difficulty in all snapshots

* Explain compatibility TODO

* Fix typo

* Document consensus rules in the difficulty module

* Calculate the difficulty using the high 128 bits of the U256 values

* Require 6 significant figures of accuracy in the difficulty unit tests

* fixup! Calculate the difficulty using the high 128 bits of the U256 values

* Update snapshots
2023-02-08 23:41:41 +00:00
teor daba6d7744
add(rpc): Add extra `getblock` RPC fields used by some mining pools (#6097)
* clippy: remove unnecessary return statement

* Add hash, height, and confirmations fields to getblock RPC

* Remove a test that is already checked by snapshots

* Document the performance requirements of the getblock RPC

* Update snapshots, use new naming scheme

* Fix off-by-one error in confirmations

* Fix spelling mistakes
2023-02-07 01:25:34 +00:00
Alfredo Garcia 43cf7e6852
implement `getdifficulty` rpc method (#6099) 2023-02-07 00:01:45 +00:00
teor a51bc2edd5
Make the verbosity argument optional in the getblock RPC (#6092) 2023-02-03 02:27:29 +00:00
Arya 0793eaf687
change(rpc): add validateaddress method (#6086)
* adds validate_address method

* Adds snapshot/vectors tests

* Checks that the address is transparent

* Removes unused pubkey/scriptPubKey fields

* adds snapshot for invalid addresses, updates network mismatch log

* simplifies is_transparent method

* Returns isvalid: false instead of conversion error
2023-02-03 02:26:58 +00:00
Arya 1bb8a9c924
change(rpc): Add getpeerinfo RPC method (#5951)
* adds ValidateBlock request to state

* adds `Request` enum in block verifier

skips solution check for BlockProposal requests

calls CheckBlockValidity instead of Commit block for BlockProposal requests

* uses new Request in references to chain verifier

* adds getblocktemplate proposal mode response type

* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state

* Adds PR review revisions

* adds info log in CheckBlockProposalValidity

* Reverts replacement of match statement

* adds `GetBlockTemplate::capabilities` fn

* conditions calling checkpoint verifier on !request.is_proposal

* updates references to validate_and_commit_non_finalized

* adds snapshot test, updates test vectors

* adds `should_count_metrics` to NonFinalizedState

* Returns an error from chain verifier for block proposal requests below checkpoint height

adds feature flags

* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD

* adds back block::Request to zebra-consensus lib

* updates snapshots

* Removes unnecessary network arg

* skips req in tracing intstrument for read state

* Moves out block proposal validation to its own fn

* corrects `difficulty_threshold_is_valid` docs

adds/fixes some comments, adds TODOs

general cleanup from a self-review.

* Update zebra-state/src/service.rs

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* check best chain tip

* Update zebra-state/src/service.rs

Co-authored-by: teor <teor@riseup.net>

* Applies cleanup suggestions from code review

* updates gbt acceptance test to make a block proposal

* fixes json parsing mistake

* adds retries

* returns reject reason if there are no retries left

* moves result deserialization to RPCRequestClient method, adds docs, moves jsonrpc_core to dev-dependencies

* moves sleep(EXPECTED_TX_TIME) out of loop

* updates/adds info logs in retry loop

* Revert "moves sleep(EXPECTED_TX_TIME) out of loop"

This reverts commit f7f0926f4050519687a79afc16656c3f345c004b.

* adds `allow(dead_code)`

* tests with curtime, mintime, & maxtime

* Fixes doc comment

* Logs error responses from chain_verifier CheckProposal requests

* Removes retry loop, adds num_txs log

* removes verbose info log

* sorts mempool_txs before generating merkle root

* Make imports conditional on a feature

* Disable new CI tests until bugs are fixed

* adds support for getpeerinfo RPC

* adds vector test

* Always passes address_book to RpcServer

* Update zebra-network/src/address_book.rs

Co-authored-by: teor <teor@riseup.net>

* Renames PeerObserver to AddressBookPeers

* adds getpeerinfo acceptance test

* Asserts that addresses parse into SocketAddr

* adds launches_lightwalletd field to TestType::LaunchWithEmptyState and uses it from the get_peer_info test

* renames peer_observer mod

* uses SocketAddr as `addr` field type in PeerInfo

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-17 07:09:07 +00:00
teor dcf30679bf
fix(rpc): Calculate getblocktemplate RPC testnet min and max times correctly (#5925)
* Clean up some getblocktemplate difficulty code and tests

* Fix minimum difficulty adjustment

* Use clamp rather than max/min

Co-authored-by: Arya <aryasolhi@gmail.com>

* Remove unused imports

* Document the Zebra-specific standard rule that allows testnet miners time to mine a block

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
2023-01-17 07:08:43 +00:00
Arya 3cbee9465a
change(rpc): Add proposal capability to getblocktemplate (#5870)
* adds ValidateBlock request to state

* adds `Request` enum in block verifier

skips solution check for BlockProposal requests

calls CheckBlockValidity instead of Commit block for BlockProposal requests

* uses new Request in references to chain verifier

* adds getblocktemplate proposal mode response type

* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state

* Adds PR review revisions

* adds info log in CheckBlockProposalValidity

* Reverts replacement of match statement

* adds `GetBlockTemplate::capabilities` fn

* conditions calling checkpoint verifier on !request.is_proposal

* updates references to validate_and_commit_non_finalized

* adds snapshot test, updates test vectors

* adds `should_count_metrics` to NonFinalizedState

* Returns an error from chain verifier for block proposal requests below checkpoint height

adds feature flags

* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD

* adds back block::Request to zebra-consensus lib

* updates snapshots

* Removes unnecessary network arg

* skips req in tracing intstrument for read state

* Moves out block proposal validation to its own fn

* corrects `difficulty_threshold_is_valid` docs

adds/fixes some comments, adds TODOs

general cleanup from a self-review.

* Update zebra-state/src/service.rs

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* check best chain tip

* Update zebra-state/src/service.rs

Co-authored-by: teor <teor@riseup.net>

* Applies cleanup suggestions from code review

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-11 23:39:51 +00:00
Arya bc783a7745
fix(test): Avoid spurious rpc_getblocktemplate test failure (#5909)
* moves mock service request handling to main thread

* removes unnecessary block
2023-01-04 03:30:50 +00:00
teor f7011a903e
feature(rpc): Add basic long polling support to the `getblocktemplate` RPC (#5843)
* fastmod longpollid long_poll_id

* Fix a comment

* Implement inefficient long polling
2022-12-15 00:30:37 +00:00
Arya f3366c53c7
fix(rpc): Accept HashOrHeight as first parameter of getblock and update README.md to note differences between lightwalletd forks (#5861)
* updates getblock RPC to accept HashOrHeight param

* update README.md to recommend lightwalletd fork

* updates vectors test

* Update README.md

* Update README.md

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2022-12-14 05:29:33 +00:00
teor e9d6e975b4
change(rpc): Refactor get block template RPC into stages (#5837)
* Add some TODOs

* Move and rename height_from_signed_int()

* Move get_block_template() support functions to a submodule

* Fix incorrect P2SH comments and logs

* Split initial checks into their own functions

* Split state fetch into its own function, do some cleanup

* Move get_block_template_opts to get_block_template::parameters

* Fix and simplify test imports

* Rename block_height to next_block_height

* Rename to chain_tip_and_local_time to make it clear what it contains

* Split fetching mempool transactions out, include them in long poll id

* Refactor coinbase generation

* Split default root calculation into a separate function

* Use DateTime32 for getblocktemplate times

* Use typed difficulty fields rather than strings

* Split out a generate coinbase and roots function

* Move GetBlockTemplate construction into a method

* Document what happens to unusual difficulty values

* Clean up some TODOs

* fastmod check_address check_miner_address

* cargo fmt --all

* Fix an incorrect panic message

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-12-13 21:25:04 +00:00
Alfredo Garcia 12ff32f445
change(rpc): support transparent p2pkh miner addresses (#5827)
* support p2pkh miner address

* Tweak comments and log messages

* removes duplicate/unused method

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Arya <aryasolhi@gmail.com>
2022-12-09 05:17:55 +00:00
teor a6e6eb5051
change(rpc): Provide and parse a long poll ID, but don't use it yet (#5796)
* Declare support for long polling in the getblocktemplate RPC

* Add long polling input and ID structs

* Split out an update_checksum() function

* Implement LongPollId conversion to and from a string

* Use the LongPollId type in the RPC

* Add a longpollid field to the getblocktemplate parameters and responses, but don't use it yet

* Use multiple RPC threads with the getblocktemplate feature, to enable efficient long polling

* Update RPC snapshots

* Remove the "longpoll" capability, it's for miners, not nodes

* Use the long poll length constant in tests

* Update snapshots

* Remove the "long polling is not supported" error

* Fix minor compilation issues after the merge/rebase

* Expand long poll id comments

* Rename estimated height to local height, because that's what it actually is

* Add an invalid params test and fix the long poll id test

* Add modified config for config_tests

* Instrument all the config sub-tests

* Show the missing config file when the test fails

* Fix the generated config file

* Allow a clippy lint in tests

* Explain conversion from bytes to u32

* Remove a duplicate test case
2022-12-09 01:41:46 +00:00
Arya 77b85cf767
change(rpc): Adds `getmininginfo`, `getnetworksolps` and `getnetworkhashps` methods (#5808)
* adds type and stub

* adds:
- SolutionRate state request

- getnetworksolps, getnetworkhashps, & getmininginfo RPCs

- vectors tests

* adds snapshot tests

updates ReadRequest::SolutionRate doc link

* removes random slash in doc comment

moves snapshot tests up where it can use the populated state service

* adds snapshots

* updates doc comments

* applies `num_blocks` default in RPC instead of `solution_rate`

* adds # Correctness comment

* Add testnet field to getmininginfo response

* use PartialCumulativeWork instead of u128

* document why `solution_rate` takes an extra block

* add comment explaining why the work for the last block in the iterator is not added to `total_work`

* use `as_u128` method instead of deref for PartialCumulativeWork

* Updates `chain` field of getmininginfo response

* Updates snapshots

Adds "arbitrary_precision" feature to serde_json in zebra-rpc
2022-12-08 19:56:14 +00:00
teor 09836d2800
fix(clippy): Put Rust format variables inline (#5783)
* cargo clippy --fix --all-features --all-targets

With rustc 1.67.0-nightly (234151769 2022-12-03)

* cargo fmt --all
2022-12-08 01:05:57 +00:00
Alfredo Garcia 678c519032
change(rpc): Populate `blockcommitmenthash` and `defaultroot` fields in the getblocktemplate RPC (#5751)
* populate `blockcommitmenthash` and `defaultroot` missing fields

* remove assertion line manually from snaps

* fix some imports and docs

* fix some docs

* add a consistency check

* Rename a constant to FINALIZED_STATE_QUERY_RETRIES and use it everywhere

* Move tip query inside retry, split tip into tip_height and tip_hash

* Return retry failures rather than panicking

* Query relevant chain inside the retry

* Check the entire state for consistency, not just the finalized tip

Co-authored-by: teor <teor@riseup.net>
2022-12-07 22:39:11 +00:00
Arya 4664ab289c
change(rpc): Adds ignored jsonrequestobject argument to the getblocktemplate RPC (#5772)
* Adds ignored jsonrequestobject argument

* adds docs

returns an error for unsupported options

* returns an error when longpollid is provided

* comments out capabilities enum and uses Vec<String> instead

* fixes spelling mistake

* uncomments GetBlockTemplateCapability enum and uses allow(dead_code) instead

* adds UnknownCapability

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* adds derived traits.

* removes unused [allow(dead_code)]

* test invalid params error for block data

* add capablity to gbt vectors test

* reverts passing in default options to getblocktemplate request in vectors test

* adds a jsonrequestobject with an unknown capability to gbt acceptance test

Co-authored-by: teor <teor@riseup.net>
2022-12-07 22:38:53 +00:00
Arya 1e1222c888
change(rpc): Return an error from getblocktemplate RPC if Zebra is still syncing lots of blocks (#5769)
* updates MAX_ESTIMATED_DISTANCE_TO_NETWORK_TIP

* adds ChainSyncStatus trait

* adds sync_status to GetBlockTemplateRpcs

Returns an error from getblocktemplate rpc if !is_close_to_tip

adds MockSyncStatus for tests

* hide MockSyncStatus behind feature flag

* moves is_close_to_tip call into async block

adds tests

* Update zebra-rpc/src/methods/tests/vectors.rs

Co-authored-by: teor <teor@riseup.net>

* uses SeqCst ordering for atomic bool ops.

uses a constant for not synced error code.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-12-02 11:21:23 +00:00
teor 21c916f5fa
fix(rpc): Mine standard and minimum difficulty blocks on testnet (#5747)
* Mine both standard and min difficulty blocks on testnet

* Add a POW_ADJUSTMENT_BLOCK_SPAN and fix an incorrect assertion

* Split the testnet adjustment into its own function

* Clarify a panic message

* Fix comments
2022-12-02 01:38:05 +00:00
Alfredo Garcia eb66f4b1a3
feat(rpc): populate some getblocktemplate RPC block header fields using the state best chain tip (#5659)
* populate block height

* populate cur_time

* populate min_time

* populate capabilities

* populate last_block_hash

* create read state request for getblocktemplate

* refactor to get difficulty fields more properly

* populate bits and target fields

* fix tests

* add target and bits documentation

* docs

* fix docs

* docs

* remove metrixs counter calls

* apply some suggestions from code review

* hide some code behind feature

* simplify the service

* fix error handling

* remove comment

* fox doc

* panic if we dont have enough state

* bring tip data from the state

* make proposal empty

* fix time

* fix docs, consensus rules

* remove non used anymore fn

* remove another non used fn

* remove no needed change

* remove more unused changes

* remove unused anymore change

* apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* fix build and snapshots

* apply testnet consensus rule

* fix test

* rustfmt

* remove time as allowed field to be modified by the miner if mining minimum difficulty block

* move all times to before calculating difficulty

* do some cleanup

* Adjust times so the whole time range is a testnet minimum difficulty block

* Return a GetBlockTemplateChainInfo struct from the difficulty calculation

* Add a Zebra-only max_time field to the getblocktemplate RPC

Co-authored-by: teor <teor@riseup.net>
2022-11-28 09:06:32 +00:00
teor 63124ba962
change(mempool): Evict transactions from the mempool using the ZIP-317 conventional fee (#5703)
* Add a ZIP-317 conventional fee module

* Add a conventional fee calculation stub, and use it for mempool size limiting

* Just return a usize from zcash_serialized_size(), removing the unused Result

* Add ZIP-317 constants

* Calculate the ZIP-317 conventional fee

* Update tests

* Add a CHANGELOG entry

* Fix a comment typo

Co-authored-by: Daira Hopwood <daira@jacaranda.org>

* Fix some missing words in a comment

Co-authored-by: Arya <aryasolhi@gmail.com>

Co-authored-by: Daira Hopwood <daira@jacaranda.org>
Co-authored-by: Arya <aryasolhi@gmail.com>
2022-11-24 01:27:35 +00:00
Arya 57fde15e5e
change(rpc): return an error from getblocktemplate method if Zebra is not synced to network tip (#5623)
* Returns error from getblocktemplate if not synced

* sets max estimated distance to 1

* removes unnecessary calls to best_tip_height

adds info log when estimated_distance_to_chain_tip is too high

* trigger GitHub actions

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-18 00:12:10 +00:00
teor 7e13677197
change(rpc): generate coinbase transactions in the getblocktemplate RPC (#5580)
* Add a getblocktemplate-rpcs feature to zebra-chain, and fix missing feature deps

* Add a coinbase transaction creation stub

* Add coinbase creation to zebra-chain

* Add coinbase creation and miner subsidy to zebra-consensus

* Add the miner config to the GetBlockTemplateRpcImpl

* Generate the coinbase transaction in the getblocktemplate RPC

* Provide fake valid block heights to getblocktemplate RPC tests

* Update getblocktemplate RPC snapshots

* Add a getblocktemplate.coinbase_tx deserialized transaction snapshot test

* Update snapshots

* Return funding stream outputs in the same order every time

* Update snapshots

* Fix a script bytes bug

* Update snapshots
2022-11-10 00:12:27 +00:00