Commit Graph

135 Commits

Author SHA1 Message Date
teor a18f47d5f6
fix(app): Stop panicking at startup when parsing the app version - release blocker (#6888)
* Fix a startup panic in app_version()

* Fix a potential RPC panic in get_info()

* Fix typo
2023-06-09 05:07:34 +00:00
teor f3e330995f
fix(build): Fix new nightly clippy lints and cargo feature resolution (#6814)
* Implement minor and patch database format versions

* Log and update database format versions when opening database

* Refactor the current list of column families into a constant

* Open all available column families, including from future Zebra versions

* Refactor note commitment tree lookups to go through the height methods

* Make Sapling/Orchard note commitment tree lookup forwards compatible

* Ignore errors reading column family lists from disk

* Update format version comments and TODOs

* Correctly log newly created database formats

* Fix a new cargo lint about resolver versions

* cargo clippy --fix --all-features --all-targets

* cargo fmt --all

* Add missing tokio feature in the state, revealed by the new resolver

* Add missing dev dependencies in zebra-node-services

* Add a missing `tokio` feature from PR #6813

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-06-07 06:04:04 +00:00
Alfredo Garcia eb07bb31d6
rename(state): Rename state verifiers and related code (#6762)
* rename verifiers

* rename `PreparedBlock` to `SemanticallyVerifiedBlock`

* rename `CommitBlock` to `SemanticallyVerifiedBlock`

* rename `FinalizedBlock` to `CheckpointVerifiedBlock`

* rename `CommitFinalizedBlock` to `CommitCheckpointVerifiedBlock`

* rename `FinalizedWithTrees` to `ContextuallyVerifiedBlockWithTrees`

* rename `ContextuallyValidBlock` to `ContextuallyVerifiedBlock`

* change some `finalized` variables or function arguments to `checkpoint_verified`

* fix docs

* document the difference between `CheckpointVerifiedBlock` and `ContextuallyVerifiedBlock`

* fix doc links

* apply suggestions to request

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to service

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to finalized_state.rs and write.rs

Co-authored-by: Marek <mail@marek.onl>

* fmt

* change some more variable names

* change a few missing generics

* fix checkpoint log issue

* rename more `prepared` vars `semantically_verified`

* fix test regex

* fix test regex 2

---------

Co-authored-by: Marek <mail@marek.onl>
2023-06-01 12:29:03 +00:00
teor f9b5eb3a15
fix(log): Limit RPC failure log length, add details to RPC failure logs (#6754)
* Limit the RPC failure log length

* Add more specific logs for some RPC tests

* Make sure the block hash is always available for logging
2023-05-25 18:28:25 +00:00
teor 8af4e572c9
fix(network): Ignore out of order Address Book changes, unless they are concurrent (#6717)
* Ignore out of order Address Book changes, and restructure the function

* Handle concurrent changes using the connection state machine order

* Handle out of order changes correctly

* Pass times through the call stack so they are consistent in tests

* Add time arguments to tests

* Fix tests that were broken by the address order checks

* fastmod wall_ local_ zebra*

* cargo fmt --all

* Fix a bug in the concurrent change check

* Test all the new apply and skip checks for address changes

* Document more edge cases and increase the concurrency time slightly

* Simplify enum ordering matches

* Fix comment typos

Co-authored-by: Arya <aryasolhi@gmail.com>

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2023-05-24 23:53:53 +00:00
Pili Guerra ec2e9ca276
Delete outdated `TODOs` refering to closed issues (#6732)
* ZIPs were updated to remove ambiguity, this was tracked in #1267.

* #2105 was fixed by #3039 and #2379 was closed by #3069

* #2230 was a duplicate of #2231 which was closed by #2511

* #3235 was obsoleted by #2156 which was fixed by #3505

* #1850 was fixed by #2944, #1851 was fixed by #2961 and #2902 was fixed by #2969

* We migrated to Rust 2021 edition in Jan 2022 with #3332

* #1631 was closed as not needed

* #338 was fixed by #3040 and #1162 was fixed by #3067

* #2079 was fixed by #2445

* #4794 was fixed by #6122

* #1678 stopped being an issue

* #3151 was fixed by #3934

* #3204 was closed as not needed

* #1213 was fixed by #4586

* #1774 was closed as not needed

* #4633 was closed as not needed

* Clarify behaviour of difficulty spacing

Co-authored-by: teor <teor@riseup.net>

* Update comment to reflect implemented behaviour

Co-authored-by: teor <teor@riseup.net>

* Update comment to reflect implemented behaviour when retrying block downloads

Co-authored-by: teor <teor@riseup.net>

* Update `TODO` to remove closed issue and clarify when we might want to fix

Co-authored-by: teor <teor@riseup.net>

* Update `TODO` to remove closed issue and clarify what we might want to change in future

Co-authored-by: teor <teor@riseup.net>

* Clarify benefits of how we do block verification

Co-authored-by: teor <teor@riseup.net>

* Fix rustfmt errors

---------

Co-authored-by: teor <teor@riseup.net>
2023-05-23 03:33:14 +00:00
teor 3f13072c46
cleanup(rust): Simplify code using closure capture in Rust 2021 edition (#6737)
* Simplify code using closure capture in Rust 2021 edition

* clippy: manual_next_back and unit_arg

* cargo fmt --all
2023-05-22 19:22:13 +00:00
teor cb6667059a
fix(rpc): Use populated state in more RPC snapshot tests (#6700)
* fix panic in height

* Update security comments and turn literals into constants

* Add a test-only assertion that the maximum output index can't ever be reached

* Fix a MISSING_BLOCK_ERROR_CODE that was mistakenly 0

* Add production RPC tests with excessive heights

* Add and update snapshots for production RPCs

* Add excessive height tests for most getblocktemplate RPCs

* Use correct snapshot names by running `cargo insta review`

* Fix some RPC tests so they use a populated state

* Use correct snapshot file names by running `cargo insta review`

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-05-17 18:50:31 +00:00
teor b1ce0e0894
fix(security): Stop panicking on state RPC or block requests with very large heights (#6699)
* fix panic in height

* Update security comments and turn literals into constants

* Add a test-only assertion that the maximum output index can't ever be reached

* Fix a MISSING_BLOCK_ERROR_CODE that was mistakenly 0

* Add production RPC tests with excessive heights

* Add and update snapshots for production RPCs

* Add excessive height tests for most getblocktemplate RPCs

* Use correct snapshot names by running `cargo insta review`

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-05-17 15:13:12 +00:00
teor b0d9471214
fix(log): Stop logging peer IP addresses, to protect user privacy (#6662)
* Add a PeerSocketAddr type which hides its IP address, but shows the port

* Manually replace SocketAddr with PeerSocketAddr where needed

```sh
fastmod SocketAddr PeerSocketAddr zebra-network
```

* Add missing imports

* Make converting into PeerSocketAddr easier

* Fix some unused imports

* Add a canonical_peer_addr() function

* Fix connection handling for PeerSocketAddr

* Fix serialization for PeerSocketAddr

* Fix tests for PeerSocketAddr

* Remove some unused imports

* Fix address book listener handling

* Remove redundant imports and conversions

* Update outdated IPv4-mapped IPv6 address code

* Make addresses canonical when deserializing

* Stop logging peer addresses in RPC code

* Update zebrad tests with new PeerSocketAddr type

* Update zebra-rpc tests with new PeerSocketAddr type

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-05-14 15:06:07 +00:00
Alfredo Garcia 7c67512cd5
feat(zebra-network): add user agent argument (#6601)
* add user agent as argument, use git to auto build zebra user agent

* try to fix test

* fix typo

* change expect text

* remove newline

* fix some docs

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2023-05-05 00:29:14 +00:00
Alfredo Garcia 8075d616fb
feat(mempool): add ZIP-317 rules to mempool (#6556)
* add ZIP-317 rules to mempool

* fix some docs

* rustfmt

* fix import

* typo

* fix tests

* fix tests 2

* fix tests 3

* fix tests 4

* fix tests 5

* move constant

* fix constant for feature

* document/quote zip rules

* add Minimum Fee Rate rule

* change(mempool): Refactor the ZIP-317 minimum fee rate calculation to use usize (#6585)

* Refactor the minimum fee rate calculation to use usize

* Check for overflow if constants change

* remove 1 rule check, fix docs

---------

Co-authored-by: teor <teor@riseup.net>
2023-05-02 00:13:33 +00:00
Alfredo Garcia 58bd898f5b
feat(zebrad): Refuse to run zebrad when release is too old (#6351)
* refuse to run Zebra if it is too old

* update the release checklist to consider the constants

* bring newline back

* apply new end of support code

* attempt to add tests (not working yet)

* move eos to progress task

* move tests

* add acceptance test (not working)

* fix tests

* change to block height checks (ugly code)

* change warn days

* refactor estimated blocks per day, etc

* move end of support code to its own task

* change test

* fix some docs

* move constants

* remove uneeded conversions

* downgrade tracing

* reduce end of support time, fix ci changing debugs to info again

* update instructions

* add failure messages

* cargo lock update

* unify releaase name constant

* change info msg

* clippy fixes

* add a block explorer

* ignore testnet in end of support task

* change panic to 16 weeks

* add some documentation about end of support

* Tweak docs wording

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-04-28 14:13:21 +00:00
Arya 6fdd02220e
fix(rpc): Omit transactions with transparent coinbase spends that are immature at the next block height from block templates (#6510)
* Adds `maturity_height` to VerifiedUnminedTx

Filters out transactions that are invalid at next_block_height in getblocktemplate method

* Adds unit testing

* rustfmt

* rejects txs with immature coinbase spends from mempool

* Condenses fns for transparent coinbase spend check

* Updates calls to VerifiedUnminedTx::new()

* Update zebra-chain/src/transparent/utxo.rs

* Applies suggestions from code review
2023-04-18 03:43:39 +00:00
Arya d7842bd467
fix(rpc): Check that mempool transactions are valid for the state's chain info in getblocktemplate (#6416)
* check last seen tip hash from mempool in fetch_mempool_transactions()

* Moves last_seen_tip_hash to ActiveState

* fixes tests and tests fixes

* continues to the next iteration of the loop to make fresh state and mempool requests if called with a long poll id

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* adds allow[unused_variable) for linter

* expects a chain tip when not(test)

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Addresses comments in code review

* - Removes second call to `last_tip_change()`

- Fixes tests using enabled mempool

* Adds note about chain tip action requirement to test method `enable()`

* updates doc comment

* Update zebrad/src/components/mempool.rs

Co-authored-by: teor <teor@riseup.net>

* fixes test

---------

Co-authored-by: teor <teor@riseup.net>
2023-04-03 23:22:07 +00:00
Marek 2a48d4cf25
change(chain): Refactor the handling of height differences (#6330)
* Unify the `impl`s of `Sub` and `Add` for `Height`

* Adjust tests for `Height` subtraction

* Use `Height` instead of `i32`

* Use `block:Height` in RPC tests

* Use `let .. else` statement

Co-authored-by: Arya <aryasolhi@gmail.com>

* Update zebra-consensus/src/block/subsidy/general.rs

* Refactor the handling of height differences

* Remove a redundant comment

* Update zebrad/src/components/sync/progress.rs

Co-authored-by: Arya <aryasolhi@gmail.com>

* Update progress.rs

* impl TryFrom<u32> for Height

* Make some test assertions clearer

* Refactor estimate_up_to()

* Restore a comment that was accidentally removed

* Document when estimate_distance_to_network_chain_tip() returns None

* Change HeightDiff to i64 and make Height.sub(Height) return HeightDiff (no Option)

* Update chain tip estimates for HeightDiff i64

* Update subsidy for HeightDiff i64

* Fix some height calculation test edge cases

* Fix the funding stream interval calculation

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2023-03-29 23:06:31 +00:00
Arya 8ba3fd921a
change(rpc): Add confirmations to getrawtransaction method response (#6287)
* adds confirmation field to getrawtransaction

* Updates tests

* Adds comment about correctness

* Apply suggestion revisions from review

* fixes overflow bug and adds test for valid confirmations value

* Update test to check that confirmations isn't too high

* Update transaction request to return confirmations

* Applies suggestions from PR review

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* fixes test

* restore derives that were lost in a bad merge

---------

Co-authored-by: teor <teor@riseup.net>
2023-03-26 23:53:44 +00:00
teor ec43d63ed2
change(log): Log a cute message for blocks that were mined by Zebra (off by default) (#6098)
* Mark Zebra coinbase transactions with extra coinbase data

* Log when we commit a block mined by Zebra to our state

* Reduce logging instrumentation during block writes

* Remove debug types in Zebra block log

* Add network and commit to write task logs

* Apply an allow-list before we log arbitrary user messages from blocks

* Rate-limit Zebra mined block logging to once every 1000 blocks

* Add mining configs for extra coinbase data and imitating zcashd, but don't use them yet

* Check CoinbaseData size limit when building transparent transactions

* Replace LIKE_ZCASHD constants with a config

* Take extra coinbase data from the configured string

* Update the zebrad configs in the tests with new config fields
2023-02-23 00:10:11 +00:00
Marek 83d038c067
feat(rpc): Implement the `z_validateaddress` RPC (#6185)
* Add the response type for `z_validateaddress`

* Add UAs to the address variants

* Remove a redundant TODO

* Impl `z_validateaddress`

* Add basic test vectors

* Add basic snapshots

* Remove a deprecated field from the RPC response

* Add basic snapshot data

* Check the semantic validity of UAs

* Refactor imports

* Refactor snapshot filenames

This PR removes the `.new` filename extensions from snapshots.

* Rename `address` to `unified_address`
2023-02-20 21:06:22 +00:00
Alfredo Garcia 31382d2a24
feat(rpc): Implement `z_listunifiedreceivers` (#6171)
* implement `z_listunifiedreceivers`

* add test vector

* add snapshots

* simplify sapling payment address

* send network argument to payment_address method

* use expect for impossible address errors

* remove network argument

* use already present network conversion

* add additional snapshot

* Derive common traits on the RPC struct

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-02-20 12:22:37 +00:00
Arya cbc4b44573
Replaces tokio::spawn with joins (#6184) 2023-02-17 03:28:52 +00:00
teor ae21e36018
fix(rpc): Add `getblock` RPC fields to support the latest version of `zcash/lightwalletd` (#6134)
* Stabilise the BestChainBlockHash state request

* Always include the block hash in the getblock RPC response

* Make the lightwalletd integration tests compatible with zcash/lightwalletd

* Update getblock RPC snapshots

* Return the correct missing block error code
2023-02-14 08:52:58 +00:00
teor 4f289299fd
change(rpc): Simplify `getdifficulty` RPC implementation (#6105)
* Use existing Work type to calculate get_difficulty RPC, round in f64 tests

* Use a valid difficulty in all snapshots

* Explain compatibility TODO

* Fix typo

* Document consensus rules in the difficulty module

* Calculate the difficulty using the high 128 bits of the U256 values

* Require 6 significant figures of accuracy in the difficulty unit tests

* fixup! Calculate the difficulty using the high 128 bits of the U256 values

* Update snapshots
2023-02-08 23:41:41 +00:00
teor daba6d7744
add(rpc): Add extra `getblock` RPC fields used by some mining pools (#6097)
* clippy: remove unnecessary return statement

* Add hash, height, and confirmations fields to getblock RPC

* Remove a test that is already checked by snapshots

* Document the performance requirements of the getblock RPC

* Update snapshots, use new naming scheme

* Fix off-by-one error in confirmations

* Fix spelling mistakes
2023-02-07 01:25:34 +00:00
Alfredo Garcia 43cf7e6852
implement `getdifficulty` rpc method (#6099) 2023-02-07 00:01:45 +00:00
Arya 0aab3df731
fix(test): avoid failing getblocktemplate acceptance test when the chain tip changes (#6091)
* adds basic usage of long polling in gbt test

* adds !submit_old check before cancelling proposing a new template

* Removes break statement in long polling task

* Update zebrad/tests/common/rpc_client.rs

Co-authored-by: teor <teor@riseup.net>

* use blocking_send and watch channel

* fix "cannot block the current thread from within a runtime"

* Reduces interval between proposals and increases num proposals required.

* Runs rate-limiting sleeps in parallel to validation

* corrects comment.

---------

Co-authored-by: teor <teor@riseup.net>
2023-02-03 05:48:49 +00:00
teor a51bc2edd5
Make the verbosity argument optional in the getblock RPC (#6092) 2023-02-03 02:27:29 +00:00
Arya 0793eaf687
change(rpc): add validateaddress method (#6086)
* adds validate_address method

* Adds snapshot/vectors tests

* Checks that the address is transparent

* Removes unused pubkey/scriptPubKey fields

* adds snapshot for invalid addresses, updates network mismatch log

* simplifies is_transparent method

* Returns isvalid: false instead of conversion error
2023-02-03 02:26:58 +00:00
teor dc43dca06f
Log block submit successes and failures (#6069) 2023-02-02 03:27:06 +00:00
Alfredo Garcia b327d5bb7d
fix(getblocktemplate): change error format in proposals (#6044)
* change error format in proposals

* use trim_matches

* make regex feature dependency

* remove regex fully

* remove char in comment to leave the file as it was before

* remove assertion lines from snapshots
2023-02-02 01:35:13 +00:00
teor 9d97919afc
fix(rpc): Make RPC "incorrect parameters" error code match `zcashd` (#6066)
* Move RPC method constants into their own module

* Rename RPC compatibility modules to avoid confusion

* Rename RPC middleware to include its new functionality

* Use FutureExt::inspect() for logging, and only format on failure

* Log all RPC errors at info level

* Make "invalid parameters" RPC error code match `zcashd`
2023-02-01 18:20:48 +00:00
teor 14fdd3002b
change(ci): Run block proposal tests in CI (#5963)
* Revert "Temporarily fix the code for a disabled test"

This reverts commit d915624417cc946e53aac76449e0b5b719e03d2a.

* Check every valid time source in the proposal tests

* Activate block proposal tests in CI

* Repeats block proposal test a few times at an interval to sample different mempool contents

* Increase the number of templates tested to 10

Co-authored-by: Arya <aryasolhi@gmail.com>

---------

Co-authored-by: arya2 <aryasolhi@gmail.com>
2023-01-31 23:02:22 +00:00
teor 4dd9426e48
feat(rpc): Implement the `getblocksubsidy` RPC (#6032)
* Make it clearer that Zebra only supports transparent funding streams

* Initial getblocksubsidy RPC types and method, without ZEC conversion

* Add a ZEC fixed-point format wrapper to Amount

* Format getblocksubsidy fields in Zec

* Add snapshot tests for getblocksubsidy RPC

* Re-order RPC output to match zcashd

* Switch to formatting Zec with f64, because the getblocksubsidy RPC requires JSON numbers

* Sort RPC responses in zcashd funding stream order

* Add getblocksubsidy snapshots

* Fix a doc link

* Move Zec JSON formatter from zebra-chain to zebra-rpc

* Remove Ord impl for Zec, it's just for formatting

* Use fully-qualified path for serde derives

* Fix a clippy warning for the doc link fix

* Fix RPC comments and an error message

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-01-31 20:41:34 +00:00
Arya be2c2299b1
change(rpc): Sort transaction hashes like zcashd in getrawmempool RPC response (#5994)
* Sorts transactions like zcashd in getrawmempool

* Simplifies sort logic and condenses duplicate code

* adds comment clarifying the intended byte order for transaction hashes

* Multiplies fee by MAX_BLOCK_BYTES/tx-size instead of tx-size

- Removes feature flag on get_raw_mempool

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Always uses `TransactionIds` request in tests

* reverts switch from #[cfg()] to cfg!()

* Adds feature flag to constant

* Updates tests and removes !cfg(not(test))

* Moves up comment

* adds missing transaction_ids

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-23 04:48:18 +00:00
Arya 53b446668e
fix(rpc): Avoid selecting duplicate transactions in block templates (#6006)
* Removes candidate_tx from candidate_txs

- Creates a new WeightedIndex instead of updating the weights

- Logs tx hashes and unpaid_actions in getblocktemplate

* Applies suggestions from PR review.
2023-01-23 02:48:45 +00:00
teor b3a480757a
fix(rpc): Return detailed errors to the RPC client when a block proposal fails (#5993)
* Return detailed errors to the RPC client when a block proposal fails

* Change proposal reject type in acceptance test
2023-01-19 06:41:55 +00:00
teor 6a06cbf3ad
test(rpc): Create a script that submits block proposals to zcashd (#5944)
* Revert "Update code that we're going to delete in the next PR anyway"

This reverts commit 1fed70da9e.

* Initial zcash-test-block-template script without block proposal construction

* Try creating block data using jq and printf

* Move proposal_block_from_template() to zebra-rpc and remove eyre dependency

* Implement FromStr for DateTime32 and Duration32

* Basic block-template-to-proposal command without time source handling

* Move block proposal code into its own module

* Use time source in block-template-to-proposal

* Make block-template-to-proposal require the getblocktemplate-rpcs feature

* Use block-template-to-proposal in the test script zcash-rpc-block-template-to-proposal

* Apply new hex formatting to commitments and nonces in block proposal tests

* Re-add missing imports from rebase

* Re-add missing conversions from rebase

* Update to new method name after rebase

* Derive a Default impl

* Restore a comment that was accidentally deleted during the rebase

* Avoid a clippy::unwrap-in-result

* Temporarily fix the code for a disabled test

* Fix tool build with Docker caches

* Make clippy happy with the temporary fix

* Give a pass/fail status for each proposal response

* Accept zcashd block templates in block-template-to-proposal

* Fix pretty printing

* Zebra expects a longpollid, so give it a dummy value

* Add "required" fields which Zebra requires

* Use curtime as the dummy maxtime in zcashd templates

* Support large block proposals by reading proposal data from a file

* Test all valid time modes for each proposal

* Allow the user to set the time command

* Put debug logs into their own files

* Exit with an error status when any proposal is invalid

* Log the date and time to make it easier to match errors to node logs
2023-01-18 02:11:15 +00:00
Arya 1bb8a9c924
change(rpc): Add getpeerinfo RPC method (#5951)
* adds ValidateBlock request to state

* adds `Request` enum in block verifier

skips solution check for BlockProposal requests

calls CheckBlockValidity instead of Commit block for BlockProposal requests

* uses new Request in references to chain verifier

* adds getblocktemplate proposal mode response type

* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state

* Adds PR review revisions

* adds info log in CheckBlockProposalValidity

* Reverts replacement of match statement

* adds `GetBlockTemplate::capabilities` fn

* conditions calling checkpoint verifier on !request.is_proposal

* updates references to validate_and_commit_non_finalized

* adds snapshot test, updates test vectors

* adds `should_count_metrics` to NonFinalizedState

* Returns an error from chain verifier for block proposal requests below checkpoint height

adds feature flags

* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD

* adds back block::Request to zebra-consensus lib

* updates snapshots

* Removes unnecessary network arg

* skips req in tracing intstrument for read state

* Moves out block proposal validation to its own fn

* corrects `difficulty_threshold_is_valid` docs

adds/fixes some comments, adds TODOs

general cleanup from a self-review.

* Update zebra-state/src/service.rs

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* check best chain tip

* Update zebra-state/src/service.rs

Co-authored-by: teor <teor@riseup.net>

* Applies cleanup suggestions from code review

* updates gbt acceptance test to make a block proposal

* fixes json parsing mistake

* adds retries

* returns reject reason if there are no retries left

* moves result deserialization to RPCRequestClient method, adds docs, moves jsonrpc_core to dev-dependencies

* moves sleep(EXPECTED_TX_TIME) out of loop

* updates/adds info logs in retry loop

* Revert "moves sleep(EXPECTED_TX_TIME) out of loop"

This reverts commit f7f0926f4050519687a79afc16656c3f345c004b.

* adds `allow(dead_code)`

* tests with curtime, mintime, & maxtime

* Fixes doc comment

* Logs error responses from chain_verifier CheckProposal requests

* Removes retry loop, adds num_txs log

* removes verbose info log

* sorts mempool_txs before generating merkle root

* Make imports conditional on a feature

* Disable new CI tests until bugs are fixed

* adds support for getpeerinfo RPC

* adds vector test

* Always passes address_book to RpcServer

* Update zebra-network/src/address_book.rs

Co-authored-by: teor <teor@riseup.net>

* Renames PeerObserver to AddressBookPeers

* adds getpeerinfo acceptance test

* Asserts that addresses parse into SocketAddr

* adds launches_lightwalletd field to TestType::LaunchWithEmptyState and uses it from the get_peer_info test

* renames peer_observer mod

* uses SocketAddr as `addr` field type in PeerInfo

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-17 07:09:07 +00:00
teor dcf30679bf
fix(rpc): Calculate getblocktemplate RPC testnet min and max times correctly (#5925)
* Clean up some getblocktemplate difficulty code and tests

* Fix minimum difficulty adjustment

* Use clamp rather than max/min

Co-authored-by: Arya <aryasolhi@gmail.com>

* Remove unused imports

* Document the Zebra-specific standard rule that allows testnet miners time to mine a block

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
2023-01-17 07:08:43 +00:00
Arya b0ba920a4f
change(test): Create test harness for calling getblocktemplate in proposal mode, but don't use it yet (#5884)
* adds ValidateBlock request to state

* adds `Request` enum in block verifier

skips solution check for BlockProposal requests

calls CheckBlockValidity instead of Commit block for BlockProposal requests

* uses new Request in references to chain verifier

* adds getblocktemplate proposal mode response type

* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state

* Adds PR review revisions

* adds info log in CheckBlockProposalValidity

* Reverts replacement of match statement

* adds `GetBlockTemplate::capabilities` fn

* conditions calling checkpoint verifier on !request.is_proposal

* updates references to validate_and_commit_non_finalized

* adds snapshot test, updates test vectors

* adds `should_count_metrics` to NonFinalizedState

* Returns an error from chain verifier for block proposal requests below checkpoint height

adds feature flags

* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD

* adds back block::Request to zebra-consensus lib

* updates snapshots

* Removes unnecessary network arg

* skips req in tracing intstrument for read state

* Moves out block proposal validation to its own fn

* corrects `difficulty_threshold_is_valid` docs

adds/fixes some comments, adds TODOs

general cleanup from a self-review.

* Update zebra-state/src/service.rs

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* check best chain tip

* Update zebra-state/src/service.rs

Co-authored-by: teor <teor@riseup.net>

* Applies cleanup suggestions from code review

* updates gbt acceptance test to make a block proposal

* fixes json parsing mistake

* adds retries

* returns reject reason if there are no retries left

* moves result deserialization to RPCRequestClient method, adds docs, moves jsonrpc_core to dev-dependencies

* moves sleep(EXPECTED_TX_TIME) out of loop

* updates/adds info logs in retry loop

* Revert "moves sleep(EXPECTED_TX_TIME) out of loop"

This reverts commit f7f0926f4050519687a79afc16656c3f345c004b.

* adds `allow(dead_code)`

* tests with curtime, mintime, & maxtime

* Fixes doc comment

* Logs error responses from chain_verifier CheckProposal requests

* Removes retry loop, adds num_txs log

* removes verbose info log

* sorts mempool_txs before generating merkle root

* Make imports conditional on a feature

* Disable new CI tests until bugs are fixed

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-17 04:03:40 +00:00
Arya a8bfb1b8dc
sorts mempool_txs before generating merkle root (#5953) 2023-01-16 03:37:39 +00:00
Arya 3cbee9465a
change(rpc): Add proposal capability to getblocktemplate (#5870)
* adds ValidateBlock request to state

* adds `Request` enum in block verifier

skips solution check for BlockProposal requests

calls CheckBlockValidity instead of Commit block for BlockProposal requests

* uses new Request in references to chain verifier

* adds getblocktemplate proposal mode response type

* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state

* Adds PR review revisions

* adds info log in CheckBlockProposalValidity

* Reverts replacement of match statement

* adds `GetBlockTemplate::capabilities` fn

* conditions calling checkpoint verifier on !request.is_proposal

* updates references to validate_and_commit_non_finalized

* adds snapshot test, updates test vectors

* adds `should_count_metrics` to NonFinalizedState

* Returns an error from chain verifier for block proposal requests below checkpoint height

adds feature flags

* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD

* adds back block::Request to zebra-consensus lib

* updates snapshots

* Removes unnecessary network arg

* skips req in tracing intstrument for read state

* Moves out block proposal validation to its own fn

* corrects `difficulty_threshold_is_valid` docs

adds/fixes some comments, adds TODOs

general cleanup from a self-review.

* Update zebra-state/src/service.rs

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* check best chain tip

* Update zebra-state/src/service.rs

Co-authored-by: teor <teor@riseup.net>

* Applies cleanup suggestions from code review

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-01-11 23:39:51 +00:00
Arya bc783a7745
fix(test): Avoid spurious rpc_getblocktemplate test failure (#5909)
* moves mock service request handling to main thread

* removes unnecessary block
2023-01-04 03:30:50 +00:00
teor b08a0b6f08
change(rpc): Match `zcashd`'s block template exactly (#5867)
* Make Zebra's getblocktemplate like zcashd's

* Update snapshots

* Add missing docs

* Fix typo

* Sort coinbase outputs by serialized script for zcashd

* Sort excluding the length byte, make transaction order always stable

* Update snapshots

* Explain that `zcashd` doesn't seem to have a fixed transaction order
2022-12-19 18:52:43 +00:00
teor 80a6d3cdab
change(rpc): Return from long polling immediately when the chain tip changes (#5862)
* Add constants and fix comments for mempool timer / state watch

* Add a best_tip_changed() method to trait ChainTip

* Check for chain tip changes using a future, and return on error

* Ignore state changes before the most recent state fetch

* Add a submit old field to the getblocktemplate RPC during long polling

* Make the submit_old field optional, rather than `null` in JSON

* Update some TODOs

* Add long polling snapshot tests, use RON for deserialized coinbase fields

* Clarify some comments

* Simplify the BestTipChanged future implementations

* Fix some comment typos
2022-12-15 15:33:00 +00:00
teor f7011a903e
feature(rpc): Add basic long polling support to the `getblocktemplate` RPC (#5843)
* fastmod longpollid long_poll_id

* Fix a comment

* Implement inefficient long polling
2022-12-15 00:30:37 +00:00
Arya f3366c53c7
fix(rpc): Accept HashOrHeight as first parameter of getblock and update README.md to note differences between lightwalletd forks (#5861)
* updates getblock RPC to accept HashOrHeight param

* update README.md to recommend lightwalletd fork

* updates vectors test

* Update README.md

* Update README.md

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2022-12-14 05:29:33 +00:00
teor e9d6e975b4
change(rpc): Refactor get block template RPC into stages (#5837)
* Add some TODOs

* Move and rename height_from_signed_int()

* Move get_block_template() support functions to a submodule

* Fix incorrect P2SH comments and logs

* Split initial checks into their own functions

* Split state fetch into its own function, do some cleanup

* Move get_block_template_opts to get_block_template::parameters

* Fix and simplify test imports

* Rename block_height to next_block_height

* Rename to chain_tip_and_local_time to make it clear what it contains

* Split fetching mempool transactions out, include them in long poll id

* Refactor coinbase generation

* Split default root calculation into a separate function

* Use DateTime32 for getblocktemplate times

* Use typed difficulty fields rather than strings

* Split out a generate coinbase and roots function

* Move GetBlockTemplate construction into a method

* Document what happens to unusual difficulty values

* Clean up some TODOs

* fastmod check_address check_miner_address

* cargo fmt --all

* Fix an incorrect panic message

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-12-13 21:25:04 +00:00
Alfredo Garcia 12ff32f445
change(rpc): support transparent p2pkh miner addresses (#5827)
* support p2pkh miner address

* Tweak comments and log messages

* removes duplicate/unused method

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Arya <aryasolhi@gmail.com>
2022-12-09 05:17:55 +00:00
Arya 5ec6ad59dc
change(rpc): return u64 from `get_network_sol_ps` and remove `arbitrary_precision` feature from serde (#5829)
* return u64 instead of u128 from get_network_sol_ps

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: teor <teor@riseup.net>

* rustfmt

Co-authored-by: teor <teor@riseup.net>
2022-12-09 03:31:09 +00:00