* clippy: remove unnecessary return statement
* Add hash, height, and confirmations fields to getblock RPC
* Remove a test that is already checked by snapshots
* Document the performance requirements of the getblock RPC
* Update snapshots, use new naming scheme
* Fix off-by-one error in confirmations
* Fix spelling mistakes
* adds basic usage of long polling in gbt test
* adds !submit_old check before cancelling proposing a new template
* Removes break statement in long polling task
* Update zebrad/tests/common/rpc_client.rs
Co-authored-by: teor <teor@riseup.net>
* use blocking_send and watch channel
* fix "cannot block the current thread from within a runtime"
* Reduces interval between proposals and increases num proposals required.
* Runs rate-limiting sleeps in parallel to validation
* corrects comment.
---------
Co-authored-by: teor <teor@riseup.net>
* change error format in proposals
* use trim_matches
* make regex feature dependency
* remove regex fully
* remove char in comment to leave the file as it was before
* remove assertion lines from snapshots
* Move RPC method constants into their own module
* Rename RPC compatibility modules to avoid confusion
* Rename RPC middleware to include its new functionality
* Use FutureExt::inspect() for logging, and only format on failure
* Log all RPC errors at info level
* Make "invalid parameters" RPC error code match `zcashd`
* Revert "Temporarily fix the code for a disabled test"
This reverts commit d915624417cc946e53aac76449e0b5b719e03d2a.
* Check every valid time source in the proposal tests
* Activate block proposal tests in CI
* Repeats block proposal test a few times at an interval to sample different mempool contents
* Increase the number of templates tested to 10
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: arya2 <aryasolhi@gmail.com>
* Make it clearer that Zebra only supports transparent funding streams
* Initial getblocksubsidy RPC types and method, without ZEC conversion
* Add a ZEC fixed-point format wrapper to Amount
* Format getblocksubsidy fields in Zec
* Add snapshot tests for getblocksubsidy RPC
* Re-order RPC output to match zcashd
* Switch to formatting Zec with f64, because the getblocksubsidy RPC requires JSON numbers
* Sort RPC responses in zcashd funding stream order
* Add getblocksubsidy snapshots
* Fix a doc link
* Move Zec JSON formatter from zebra-chain to zebra-rpc
* Remove Ord impl for Zec, it's just for formatting
* Use fully-qualified path for serde derives
* Fix a clippy warning for the doc link fix
* Fix RPC comments and an error message
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Sorts transactions like zcashd in getrawmempool
* Simplifies sort logic and condenses duplicate code
* adds comment clarifying the intended byte order for transaction hashes
* Multiplies fee by MAX_BLOCK_BYTES/tx-size instead of tx-size
- Removes feature flag on get_raw_mempool
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Always uses `TransactionIds` request in tests
* reverts switch from #[cfg()] to cfg!()
* Adds feature flag to constant
* Updates tests and removes !cfg(not(test))
* Moves up comment
* adds missing transaction_ids
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Removes candidate_tx from candidate_txs
- Creates a new WeightedIndex instead of updating the weights
- Logs tx hashes and unpaid_actions in getblocktemplate
* Applies suggestions from PR review.
* Revert "Update code that we're going to delete in the next PR anyway"
This reverts commit 1fed70da9e.
* Initial zcash-test-block-template script without block proposal construction
* Try creating block data using jq and printf
* Move proposal_block_from_template() to zebra-rpc and remove eyre dependency
* Implement FromStr for DateTime32 and Duration32
* Basic block-template-to-proposal command without time source handling
* Move block proposal code into its own module
* Use time source in block-template-to-proposal
* Make block-template-to-proposal require the getblocktemplate-rpcs feature
* Use block-template-to-proposal in the test script zcash-rpc-block-template-to-proposal
* Apply new hex formatting to commitments and nonces in block proposal tests
* Re-add missing imports from rebase
* Re-add missing conversions from rebase
* Update to new method name after rebase
* Derive a Default impl
* Restore a comment that was accidentally deleted during the rebase
* Avoid a clippy::unwrap-in-result
* Temporarily fix the code for a disabled test
* Fix tool build with Docker caches
* Make clippy happy with the temporary fix
* Give a pass/fail status for each proposal response
* Accept zcashd block templates in block-template-to-proposal
* Fix pretty printing
* Zebra expects a longpollid, so give it a dummy value
* Add "required" fields which Zebra requires
* Use curtime as the dummy maxtime in zcashd templates
* Support large block proposals by reading proposal data from a file
* Test all valid time modes for each proposal
* Allow the user to set the time command
* Put debug logs into their own files
* Exit with an error status when any proposal is invalid
* Log the date and time to make it easier to match errors to node logs
* adds ValidateBlock request to state
* adds `Request` enum in block verifier
skips solution check for BlockProposal requests
calls CheckBlockValidity instead of Commit block for BlockProposal requests
* uses new Request in references to chain verifier
* adds getblocktemplate proposal mode response type
* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state
* Adds PR review revisions
* adds info log in CheckBlockProposalValidity
* Reverts replacement of match statement
* adds `GetBlockTemplate::capabilities` fn
* conditions calling checkpoint verifier on !request.is_proposal
* updates references to validate_and_commit_non_finalized
* adds snapshot test, updates test vectors
* adds `should_count_metrics` to NonFinalizedState
* Returns an error from chain verifier for block proposal requests below checkpoint height
adds feature flags
* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD
* adds back block::Request to zebra-consensus lib
* updates snapshots
* Removes unnecessary network arg
* skips req in tracing intstrument for read state
* Moves out block proposal validation to its own fn
* corrects `difficulty_threshold_is_valid` docs
adds/fixes some comments, adds TODOs
general cleanup from a self-review.
* Update zebra-state/src/service.rs
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Update zebra-rpc/src/methods/get_block_template_rpcs.rs
Co-authored-by: teor <teor@riseup.net>
* check best chain tip
* Update zebra-state/src/service.rs
Co-authored-by: teor <teor@riseup.net>
* Applies cleanup suggestions from code review
* updates gbt acceptance test to make a block proposal
* fixes json parsing mistake
* adds retries
* returns reject reason if there are no retries left
* moves result deserialization to RPCRequestClient method, adds docs, moves jsonrpc_core to dev-dependencies
* moves sleep(EXPECTED_TX_TIME) out of loop
* updates/adds info logs in retry loop
* Revert "moves sleep(EXPECTED_TX_TIME) out of loop"
This reverts commit f7f0926f4050519687a79afc16656c3f345c004b.
* adds `allow(dead_code)`
* tests with curtime, mintime, & maxtime
* Fixes doc comment
* Logs error responses from chain_verifier CheckProposal requests
* Removes retry loop, adds num_txs log
* removes verbose info log
* sorts mempool_txs before generating merkle root
* Make imports conditional on a feature
* Disable new CI tests until bugs are fixed
* adds support for getpeerinfo RPC
* adds vector test
* Always passes address_book to RpcServer
* Update zebra-network/src/address_book.rs
Co-authored-by: teor <teor@riseup.net>
* Renames PeerObserver to AddressBookPeers
* adds getpeerinfo acceptance test
* Asserts that addresses parse into SocketAddr
* adds launches_lightwalletd field to TestType::LaunchWithEmptyState and uses it from the get_peer_info test
* renames peer_observer mod
* uses SocketAddr as `addr` field type in PeerInfo
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Clean up some getblocktemplate difficulty code and tests
* Fix minimum difficulty adjustment
* Use clamp rather than max/min
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove unused imports
* Document the Zebra-specific standard rule that allows testnet miners time to mine a block
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
* adds ValidateBlock request to state
* adds `Request` enum in block verifier
skips solution check for BlockProposal requests
calls CheckBlockValidity instead of Commit block for BlockProposal requests
* uses new Request in references to chain verifier
* adds getblocktemplate proposal mode response type
* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state
* Adds PR review revisions
* adds info log in CheckBlockProposalValidity
* Reverts replacement of match statement
* adds `GetBlockTemplate::capabilities` fn
* conditions calling checkpoint verifier on !request.is_proposal
* updates references to validate_and_commit_non_finalized
* adds snapshot test, updates test vectors
* adds `should_count_metrics` to NonFinalizedState
* Returns an error from chain verifier for block proposal requests below checkpoint height
adds feature flags
* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD
* adds back block::Request to zebra-consensus lib
* updates snapshots
* Removes unnecessary network arg
* skips req in tracing intstrument for read state
* Moves out block proposal validation to its own fn
* corrects `difficulty_threshold_is_valid` docs
adds/fixes some comments, adds TODOs
general cleanup from a self-review.
* Update zebra-state/src/service.rs
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Update zebra-rpc/src/methods/get_block_template_rpcs.rs
Co-authored-by: teor <teor@riseup.net>
* check best chain tip
* Update zebra-state/src/service.rs
Co-authored-by: teor <teor@riseup.net>
* Applies cleanup suggestions from code review
* updates gbt acceptance test to make a block proposal
* fixes json parsing mistake
* adds retries
* returns reject reason if there are no retries left
* moves result deserialization to RPCRequestClient method, adds docs, moves jsonrpc_core to dev-dependencies
* moves sleep(EXPECTED_TX_TIME) out of loop
* updates/adds info logs in retry loop
* Revert "moves sleep(EXPECTED_TX_TIME) out of loop"
This reverts commit f7f0926f4050519687a79afc16656c3f345c004b.
* adds `allow(dead_code)`
* tests with curtime, mintime, & maxtime
* Fixes doc comment
* Logs error responses from chain_verifier CheckProposal requests
* Removes retry loop, adds num_txs log
* removes verbose info log
* sorts mempool_txs before generating merkle root
* Make imports conditional on a feature
* Disable new CI tests until bugs are fixed
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* adds ValidateBlock request to state
* adds `Request` enum in block verifier
skips solution check for BlockProposal requests
calls CheckBlockValidity instead of Commit block for BlockProposal requests
* uses new Request in references to chain verifier
* adds getblocktemplate proposal mode response type
* makes getblocktemplate-rpcs feature in zebra-consensus select getblocktemplate-rpcs in zebra-state
* Adds PR review revisions
* adds info log in CheckBlockProposalValidity
* Reverts replacement of match statement
* adds `GetBlockTemplate::capabilities` fn
* conditions calling checkpoint verifier on !request.is_proposal
* updates references to validate_and_commit_non_finalized
* adds snapshot test, updates test vectors
* adds `should_count_metrics` to NonFinalizedState
* Returns an error from chain verifier for block proposal requests below checkpoint height
adds feature flags
* adds "proposal" to GET_BLOCK_TEMPLATE_CAPABILITIES_FIELD
* adds back block::Request to zebra-consensus lib
* updates snapshots
* Removes unnecessary network arg
* skips req in tracing intstrument for read state
* Moves out block proposal validation to its own fn
* corrects `difficulty_threshold_is_valid` docs
adds/fixes some comments, adds TODOs
general cleanup from a self-review.
* Update zebra-state/src/service.rs
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Update zebra-rpc/src/methods/get_block_template_rpcs.rs
Co-authored-by: teor <teor@riseup.net>
* check best chain tip
* Update zebra-state/src/service.rs
Co-authored-by: teor <teor@riseup.net>
* Applies cleanup suggestions from code review
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Make Zebra's getblocktemplate like zcashd's
* Update snapshots
* Add missing docs
* Fix typo
* Sort coinbase outputs by serialized script for zcashd
* Sort excluding the length byte, make transaction order always stable
* Update snapshots
* Explain that `zcashd` doesn't seem to have a fixed transaction order
* Add constants and fix comments for mempool timer / state watch
* Add a best_tip_changed() method to trait ChainTip
* Check for chain tip changes using a future, and return on error
* Ignore state changes before the most recent state fetch
* Add a submit old field to the getblocktemplate RPC during long polling
* Make the submit_old field optional, rather than `null` in JSON
* Update some TODOs
* Add long polling snapshot tests, use RON for deserialized coinbase fields
* Clarify some comments
* Simplify the BestTipChanged future implementations
* Fix some comment typos
* Add some TODOs
* Move and rename height_from_signed_int()
* Move get_block_template() support functions to a submodule
* Fix incorrect P2SH comments and logs
* Split initial checks into their own functions
* Split state fetch into its own function, do some cleanup
* Move get_block_template_opts to get_block_template::parameters
* Fix and simplify test imports
* Rename block_height to next_block_height
* Rename to chain_tip_and_local_time to make it clear what it contains
* Split fetching mempool transactions out, include them in long poll id
* Refactor coinbase generation
* Split default root calculation into a separate function
* Use DateTime32 for getblocktemplate times
* Use typed difficulty fields rather than strings
* Split out a generate coinbase and roots function
* Move GetBlockTemplate construction into a method
* Document what happens to unusual difficulty values
* Clean up some TODOs
* fastmod check_address check_miner_address
* cargo fmt --all
* Fix an incorrect panic message
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Declare support for long polling in the getblocktemplate RPC
* Add long polling input and ID structs
* Split out an update_checksum() function
* Implement LongPollId conversion to and from a string
* Use the LongPollId type in the RPC
* Add a longpollid field to the getblocktemplate parameters and responses, but don't use it yet
* Use multiple RPC threads with the getblocktemplate feature, to enable efficient long polling
* Update RPC snapshots
* Remove the "longpoll" capability, it's for miners, not nodes
* Use the long poll length constant in tests
* Update snapshots
* Remove the "long polling is not supported" error
* Fix minor compilation issues after the merge/rebase
* Expand long poll id comments
* Rename estimated height to local height, because that's what it actually is
* Add an invalid params test and fix the long poll id test
* Add modified config for config_tests
* Instrument all the config sub-tests
* Show the missing config file when the test fails
* Fix the generated config file
* Allow a clippy lint in tests
* Explain conversion from bytes to u32
* Remove a duplicate test case
* Remove verbose continuous_blockchain test logs
* Downgrade verbose zebra-network logs to debug
* Downgrade some state logs to debug during tests
* Mark were we would add always-on log filters, if we needed to
* Reduce the number of mempool property tests, to reduce logging
* adds type and stub
* adds:
- SolutionRate state request
- getnetworksolps, getnetworkhashps, & getmininginfo RPCs
- vectors tests
* adds snapshot tests
updates ReadRequest::SolutionRate doc link
* removes random slash in doc comment
moves snapshot tests up where it can use the populated state service
* adds snapshots
* updates doc comments
* applies `num_blocks` default in RPC instead of `solution_rate`
* adds # Correctness comment
* Add testnet field to getmininginfo response
* use PartialCumulativeWork instead of u128
* document why `solution_rate` takes an extra block
* add comment explaining why the work for the last block in the iterator is not added to `total_work`
* use `as_u128` method instead of deref for PartialCumulativeWork
* Updates `chain` field of getmininginfo response
* Updates snapshots
Adds "arbitrary_precision" feature to serde_json in zebra-rpc
* Box errors to deal with large error warnings, add accessor methods for error properties
* Remove or explain some large enum variant lints
* Turn some tickets into TODOs
* Allow missing docs on single-field error enum variants
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Update ZIP-317 implementation for unpaid actions
* Split shared transaction choose and check into its own function
* Fix an incorrect address error message
* Simplify code, expand docs
* Require docs for getblocktemplate RPC types
* Account for the coinbase transaction in the transaction selection limits
* Fix a broken doc link, update comments, tidy imports
* Fix comment typos
* Use the actual block height rather than Height::MAX for the fake coinbase
* Use a 1 zat fee rather than 0, just in case someone gets clever and skips zero outputs
* populate `blockcommitmenthash` and `defaultroot` missing fields
* remove assertion line manually from snaps
* fix some imports and docs
* fix some docs
* add a consistency check
* Rename a constant to FINALIZED_STATE_QUERY_RETRIES and use it everywhere
* Move tip query inside retry, split tip into tip_height and tip_hash
* Return retry failures rather than panicking
* Query relevant chain inside the retry
* Check the entire state for consistency, not just the finalized tip
Co-authored-by: teor <teor@riseup.net>
* Adds ignored jsonrequestobject argument
* adds docs
returns an error for unsupported options
* returns an error when longpollid is provided
* comments out capabilities enum and uses Vec<String> instead
* fixes spelling mistake
* uncomments GetBlockTemplateCapability enum and uses allow(dead_code) instead
* adds UnknownCapability
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* adds derived traits.
* removes unused [allow(dead_code)]
* test invalid params error for block data
* add capablity to gbt vectors test
* reverts passing in default options to getblocktemplate request in vectors test
* adds a jsonrequestobject with an unknown capability to gbt acceptance test
Co-authored-by: teor <teor@riseup.net>
* Mine both standard and min difficulty blocks on testnet
* Add a POW_ADJUSTMENT_BLOCK_SPAN and fix an incorrect assertion
* Split the testnet adjustment into its own function
* Clarify a panic message
* Fix comments
* Split the conventional fee check into its own method
This will be used for block production and relaying
* Move getblocktemplate transaction selection into a new zip317 module
* Add a block_production_fee_weight field to VerifiedUnminedTx
* Add a custom Zebra minimum transaction weight for block production
* Implement ZIP-317 transaction selection for block production
* Split weighted index setup into its own function
* Split picking a transaction into its own function
* Re-apply: add acceptance test for getblocktemplate method in CI (#5653)
Revert "Revert "change(tests): add acceptance test for getblocktemplate method in CI (#5653)" (#5672)"
This reverts commit 6446e0ec1b.
* Fix incorrect MAX_CONTEXT_BLOCKS assertion in state
* Actually negate the miner fee for the RPC output
* Try the RPC again after waiting for transactions to verify
* Log before the test waits for the mempool to verify transactions
* Use the new ssh key secrets in CI
* populate block height
* populate cur_time
* populate min_time
* populate capabilities
* populate last_block_hash
* create read state request for getblocktemplate
* refactor to get difficulty fields more properly
* populate bits and target fields
* fix tests
* add target and bits documentation
* docs
* fix docs
* docs
* remove metrixs counter calls
* apply some suggestions from code review
* hide some code behind feature
* simplify the service
* fix error handling
* remove comment
* fox doc
* panic if we dont have enough state
* bring tip data from the state
* make proposal empty
* fix time
* fix docs, consensus rules
* remove non used anymore fn
* remove another non used fn
* remove no needed change
* remove more unused changes
* remove unused anymore change
* apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* fix build and snapshots
* apply testnet consensus rule
* fix test
* rustfmt
* remove time as allowed field to be modified by the miner if mining minimum difficulty block
* move all times to before calculating difficulty
* do some cleanup
* Adjust times so the whole time range is a testnet minimum difficulty block
* Return a GetBlockTemplateChainInfo struct from the difficulty calculation
* Add a Zebra-only max_time field to the getblocktemplate RPC
Co-authored-by: teor <teor@riseup.net>
* Add a ZIP-317 conventional fee module
* Add a conventional fee calculation stub, and use it for mempool size limiting
* Just return a usize from zcash_serialized_size(), removing the unused Result
* Add ZIP-317 constants
* Calculate the ZIP-317 conventional fee
* Update tests
* Add a CHANGELOG entry
* Fix a comment typo
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
* Fix some missing words in a comment
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
Co-authored-by: Arya <aryasolhi@gmail.com>
* Returns error from getblocktemplate if not synced
* sets max estimated distance to 1
* removes unnecessary calls to best_tip_height
adds info log when estimated_distance_to_chain_tip is too high
* trigger GitHub actions
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Make the queue runner task shut down when the RpcImpl is dropped
* Move RPC server startup into the spawn() tokio future
* Return a shutdown handle from the RPC spawn() method
* Shut down the RPC server properly when Zebra shuts down
* Add a changelog entry for this security fix
* Call RpcServer::shutdown() when it is dropped, and wait
* Block on RPC server shutdown when Zebra's tasks have an error
* Add a getblocktemplate-rpcs feature to zebra-chain, and fix missing feature deps
* Add a coinbase transaction creation stub
* Add coinbase creation to zebra-chain
* Add coinbase creation and miner subsidy to zebra-consensus
* Add the miner config to the GetBlockTemplateRpcImpl
* Generate the coinbase transaction in the getblocktemplate RPC
* Provide fake valid block heights to getblocktemplate RPC tests
* Update getblocktemplate RPC snapshots
* Add a getblocktemplate.coinbase_tx deserialized transaction snapshot test
* Update snapshots
* Return funding stream outputs in the same order every time
* Update snapshots
* Fix a script bytes bug
* Update snapshots
* Split out a select_mempool_transactions() function
* Add some TODOs
* Cleanup redundant dependencies
* Draft conversion from coinbase Transactions into TransactionTemplates
* Document a non-coinbase requirement for remaining_transaction_balance()
* Add a Network field to the getblocktemplate RPC handler
* Clarify an error message
* Re-raise panics in the getblocktemplate task, for better debugging
* Fix how the fake coinbase transaction is created
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* adds submitblock rpc method
* re-orders imports
* replaces thread::yield_now with async yield_now
* Fix doc warnings and unused variable warnings, add missing docs
* Mark work_id as optional
* Use the same ChainVerifier for downloaded and submitted blocks
* Revert unused changes & minor cleanups
* Document currently-unreachable code
* updates tests and submit_block response for AlreadyVerified error
* Update zebra-rpc/src/methods/get_block_template_rpcs.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* changes names from BlockVerifier to ChainVerifier and block_verifier to chain_verifier to keep it consistent with naming in zebra-consensus
* move how to run the submit_block test example to acceptance.rs
* updates snapshot tests
* moved acceptance test to a separate file
* removes extra tower::ServiceBuilder::new(), updates docs
* updates vectors and snapshot tests, changes hex decoding error in submit_block method from server error to parse error
* hides errors module in zebra-rpc behind a feature flag and adds docs.
* Updates snapshot test, adds mod docs, moves HexData to its own mod, and removes the unrelated make_server_error refactor for now
* update submit block acceptance test mod doc
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add a legacy_sigop_count field to VerifiedUnminedTx
* Add conversions from Vec<VerifiedUnminedTx> to block header roots
* Add fee and sigops field to block template transactions
* Fix up mempool request names
* Increase existing snapshot test coverage
* Document a new method parameter
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add documentation for the getblocktemplate RPC
* Add a new mempool::Request::Transactions
* Add conversions from Vec<UnminedTx> to merkle::Root and AuthDataRoot
* Fill in the merkle root and auth data root fields
* Delete the Coinbase type, it's the same as Transaction
* Fill in some other existing types
* Add Hex serialization support to some zebra-chain types
* Add TransactionTemplate fields and fill some in
* Fix test hangs by spawning async tasks
* Add temporary workaround for no transactions in the block
* Encode hashes and roots as hex
* Update RPC snapshots
* Add a missing Request::Transactions handler
* Fix doc warnings
* Fix fee serialization
* Update snapshots for serialization changes
* Add a missing Cargo.lock change
* Change depends type
* Remove duplicate feature entry
* Document the new RPC feature
* Fix a comment typo
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Update default roots docs
* Fix comment typo
* Fix a comment typo
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
* introduce `getblocktemplate-rpcs` feature
* reorder imports
* highlight the problem
* add docs
* add additional empty state test
* add snapshot test
* remove getblocktemplate trait
* use `cfg-if`
* leave server as it was
* add a missing space to the docs
* fix typo in test
Co-authored-by: Arya <aryasolhi@gmail.com>
* suggestion for introduce `getblocktemplate-rpcs` feature (#5418)
* adds minimal new object for the get block template methods
* updated TODO comment
Co-authored-by: Arya <aryasolhi@gmail.com>
* Add a new block commit task and channels, that don't do anything yet
* Add last_block_hash_sent to the state service, to avoid database accesses
* Update last_block_hash_sent regardless of commit errors
* Rename a field to StateService.max_queued_finalized_height
* Commit finalized blocks to the state in a separate task
* Check for panics in the block write task
* Wait for the block commit task in tests, and check for errors
* Always run a proptest that sleeps once
* Add extra debugging to state shutdowns
* Work around a RocksDB shutdown bug
* Close the finalized block channel when we're finished with it
* Only reset state queue once per error
* Update some TODOs
* Add a module doc comment
* Drop channels and check for closed channels in the block commit task
* Close state channels and tasks on drop
* Remove some duplicate fields across StateService and ReadStateService
* Try tweaking the shutdown steps
* Update and clarify some comments
* Clarify another comment
* Don't try to cancel RocksDB background work on drop
* Fix up some comments
* Remove some duplicate code
* Remove redundant workarounds for shutdown issues
* Remode a redundant channel close in the block commit task
* Remove a mistaken `!force` shutdown condition
* Remove duplicate force-shutdown code and explain it better
* Improve RPC error logging
* Wait for chain tip updates in the RPC tests
* Wait 2 seconds for chain tip updates before skipping them
* Remove an unnecessary block_in_place()
* Fix some test error messages that were changed by earlier fixes
* Expand some comments, fix typos
Co-authored-by: Marek <mail@marek.onl>
* Actually drop children of failed blocks
* Explain why we drop descendants of failed blocks
* Clarify a comment
* Wait for chain tip updates in a failing test on macOS
* Clean duplicate finalized blocks when the non-finalized state activates
* Send an error when receiving a duplicate finalized block
* Update checkpoint block behaviour, document its consensus rule
* Wait for chain tip changes in inbound_block_height_lookahead_limit test
* Wait for the genesis block to commit in the fake peer set mempool tests
* Disable unreliable mempool verification check in the send transaction test
* Appease rustfmt
* Use clear_finalized_block_queue() everywhere that blocks are dropped
* Document how Finalized and NonFinalized clones are different
* Use the same check as commit_finalized() for finalized block heights
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Marek <mail@marek.onl>
* Move AwaitUtxos next to the other shared writeable state requests
* Rename ReadResponse::Utxos to ReadResponse::AddressUtxos
```sh
fastmod Utxos AddressUtxos zebra*
```
* Rename an out_point variable to outpoint for consistency
* Rename transparent_utxos to address_utxos
```sh
fastmod transparent_utxos address_utxos zebra*
```
* Run AwaitUtxo without accessing shared mutable chain state
* Fix some incorrect comments
* Explain why some concurrent reads are ok
* Add a TODO
* Stop using self.mem in AwaitUtxo requests
* Update state service module documentation
* Move the QueuedBlock type into the queued_blocks module
* Explain how spent UTXOs are treated by the state
* Clarify how cached Chains impact state read requests
And move repeated comments to the module header.
* fastmod ChainUtxo BestChainUtxo zebra*
* Add an AnyChainUtxo request
* Make AwaitUtxo non-blocking
* add grpc mempool test research
* add a config flag for mempool injection of transactions in test
* Only copy the inner state directory in the send transactions test
* Preload Zcash parameters in some transaction verification tests
* Add a block and transaction Hash method to convert from display order bytes
* Update test coverage docs
* Add debugging output for mempool transaction verification
* Test fetching sent mempool transactions using gRPC
* Add extra log checks to the send transaction test
* Wait for zebrad mempool activation before running gRPC tests
* Update send transaction test for lightwalletd not returning mempool transactions
* Check zebrad logs instead of disabled lightwalletd gRPCs
* Add a debug option that makes RPCs pretend the sync is finished
* Remove an unused debug option
* Remove unused test code and downgrade some logs
* Fix test log checks
* Fix some rustdoc warnings
* Fix a compilation error due to new function arguments
* Make zebrad sync timeouts consistent and remove outdated code
* Document how to increase temporary directory space for tests
* Stop checking for a log that doesn't always happen
* Remove some commented-out code
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Update a comment about run time
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add new config to new tests from the `main` branch
* Add transactions to the list, rather than replacing the list with each new block
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add a config for increasing the number of RPC threads
* Add unit tests for parallel RPC servers
* Refactor tests to remove duplicate code
* Update the README
* Actually use parallel threads in some RPC tests
* Isolate RPC queries from the rest of Zebra, to improve performance
* Wrap RPC executor startup and shutdown in a std::thread
* Add tests for spawning RPC servers
* Adjust test timing and checks to reliably work cross-platform
* truncate the number of transactions in send transaction test
* Limit send transaction test to RPC queue length
Co-authored-by: teor <teor@riseup.net>
* Only fetch block headers from the database to answer headers requests
* Move writing to the database to a blocking thread
* Add blocking threads to tests that need them
* Remove mempool downloader requirement for a populated state
And stop populating states that don't need it,
so we can use tokio::time::pause() in those tests.
* Improve debugging for an intermittent test failure
* Try to avoid a race condition populating the mempool in tests
* Fix the syntax of links in comments
* Fix a mistake in the docs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove unnecessary angle brackets from a link
* Revert the changes for links that serve as references
* Revert "Revert the changes for links that serve as references"
This reverts commit 8b091aa9fa.
* Remove `<` `>` from links that serve as references
This reverts commit 046ef25620.
* Don't use `<` `>` in normal comments
* Don't use `<` `>` for normal comments
* Revert changes for comments starting with `//`
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* change: set NU5 mainnet activation height and minimum protocol version
* fix tests
* test if V5 coinbase expiry height is not bound by non-coinbase limit
* update get_blockchain_info snapshot for NU5
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Impl the elementary structure of the `z_gettreestate` RPC
* Fix merging bugs
* Fix a merge bug
* Fix a merge bug
* Move a derive attribute
Co-authored-by: teor <teor@riseup.net>
* Clarify the support of negative heights
* Add Orchard note commitment trees to the response
* Add the time to the response
* Finalize the `z_gettreestate` RPC
* Add a note that verified blocks have coinbase height
* Refactor `from_str` for `HashOrHeight`
* Fix a mistake in the docs
Co-authored-by: teor <teor@riseup.net>
* Clarify request types
Co-authored-by: teor <teor@riseup.net>
* Simplify `hash_or_height` conversion to height
Co-authored-by: teor <teor@riseup.net>
* Add a TODO about optimization
Co-authored-by: teor <teor@riseup.net>
* Add a doc comment
* Make sure Sapling & Orchard trees don't get mixed up
* Serialize Sapling commitment trees
* Refactor some comments
* Serialize Orchard commitment trees
* Serialize block heights
* Simplify the serialization of commitment trees
* Remove the block time from the RPC response
* Simplify the serialization of block heights
* Put Sapling & Orchard requests together
* Remove a redundant TODO
* Add block times to the RPC response
* Derive `Clone, Debug, Eq, PartialEq` for `GetTreestate`
Co-authored-by: teor <teor@riseup.net>
* Derive `Clone`, `Debug`, `Eq` and `PartialEq` for `SerializedTree`
* Document the fields of `GetTreestate`
* Skip the serialization of empty trees
This ensures compatibility with `zcashd` in the `z_gettreestate` RPC.
* Document the `impl` of `merkle_tree::Hashable` for nodes
* Make the structure of the JSON response consistent with `zcashd`
* Derive `Eq` for nodes
Co-authored-by: teor <teor@riseup.net>
* Convert Sapling commitment trees to a format compatible with zcashd
* Refactor the conversion of Sapling commitment trees
* Refactor some comments
* Refactor comments
* Add a description of the conversion
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Fix comment indenting
* Document the conversion between the dense and sparse formats
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Assert that address TxIDs and UTXOs are in chain order
* Always output colour by default in zcash-rpc-diff
* Cross-check getaddressutxos and getaddressbalance in zcash-rpc-diff
* Make balances with no UTXOs match in zcash-rpc-diff
* Add some TODOs
* Display the actual connected node software in zcash-rpc-diff
* Log address UTXOs request summaries
* Log address count for address UTXO requests
* Simplify zcash-rpc-diff node names
* Log chain address UTXOs request processing
* Stop ignoring all non-finalized UTXOs in address queries
* Make zcash-rpc-diff node names more consistent
* Downgrade logs to debug level
* Stop ignoring all non-finalized tx IDs in address queries
* add info to lightwalletd_state_path()
* fix getaddresstxids rpc
* normalize more the env vars in lightwalletd tests
* extra logging
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: teor <teor@riseup.net>
* Match `zcashd`'s version format in `getinfo`
* Remove an incorrect array wrapper in getaddressbalance
* Use a stable sort order in getrawmempool
Because we can't match zcashd's arbitrary order,
and probably shouldn't try to.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* implement display for `Script`
* implement `getaddressutxos`
* fix space
* normalize list of addresses as argument to rpc methods
* implement `AddressStrings`
* make a doc clearer
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Trace RPC calls by default
Enable RPC call tracing by default. This is useful for development
purposes, and should probably be removed in the future.
* Create a `TracingMiddleware` for the RPC server
A middleware that will print error messages if there are unrecognized
RPC requests.
* Use `TracingMiddleware` in the `Server`
Enable logging of unrecognized RPC requests.
* Warn user about unrecognized RPCs
Instead of reporting it as an error.
Co-authored-by: teor <teor@riseup.net>
* Revert "Trace RPC calls by default"
This reverts commit 6d7f10de22.
* Clone `Call` instead of creating a `String`
This might improve performance.
Co-authored-by: teor <teor@riseup.net>
* Add a finalized state txids query
* Add an address transaction IDs query, without height filters
* Connect the address transaction ID query to the RPC
* Basic filtering of address transaction IDs by height range
* Add a network and range argument to the getaddresstxids test
* Test all block range combinations for mainnet
* Fix a file descriptor limit error
* Optimise seeking the first transaction for an address
The first transaction's location is part of the address location.
* Filter finalized address transaction IDs by height range
* Filter non-finalized address transaction IDs by the height range
* Fix up snapshot tests for the new height range API
* Add `Amount::serialize_as_string` helper method
A helper method that makes it easier to serialize an `Amount` as a
string. This is needed for the response type of the `getaccountbalance`
RPC.
* Implement state service call for address balance
Add `Read{Request,Response}::AddressBalance` variants and implement the
handler that calls the query function.
* Create an `AddressBalance` response type
Only contains the `balance` field which is needed by `lightwalletd`.
That field is serialized as a string, following the RPC specification.
* Implement `get_address_balance` RPC
Query the read-only state service for the information, and wrap it in an
`AddressBalance` response type so that it is serialized correctly.
* Run `rustfmt` inside `proptest!` block
Fix some minor formatting details.
* Test `get_address_balance` with valid addresses
Check that the RPC leads to a query to the mocked state service for a
balance amount.
* Test `get_address_balance` with invalid addresses
An error message should be returned by the RPC.
* Rename metric to `address_balance`
Keep it consistent with how it's named in other places.
Co-authored-by: teor <teor@riseup.net>
* Revert "Add `Amount::serialize_as_string` helper method"
This reverts commit 01b432e3d2.
* Serialize amount as an integer
This is different from what the documentation says, but it's what
lightwalletd expects.
* Add reference to RPC documentation
Make sure it is linked to for easy access.
* Create an `AddressStrings` type
To be used as the input for the `get_address_balance` RPC method.
* Use `AddressStrings` in `get_address_balance` RPC
Fix the input parameter so that the list of address strings is placed
inside a JSON map.
* Update property tests to use `AddressStrings`
Make sure the proper input type is created.
Co-authored-by: teor <teor@riseup.net>
* implement `getaddresstxids` rpc method with dummy empty response
* use already public function
* fix some docs
* pass a list of addresses to the state request
* sync range errors with zcashd
* refactor a loop
* fix grammar
* fix tests
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
* Add a rpc queue
* Implement the rpc queue
* Add rpc queue tests
* Remove mutex, use broadcast channel
* Have order and limit in the queue
* fix multiple transactions channel
* Use a network argument
* Use chain tip to calculate block spacing
* Add extra time
* Finalize the state check test
* Add a retry test
* Fix description
* fix some docs
* add additional empty check to `Runner::run`
* remove non used method
* ignore some errors
* fix some docs
* add a panic checker to the queue
* add missing file changes for panic checker
* skip checks and retries if height has not changed
* change constants
* reduce the number of queue test cases
* remove suggestion
* change best tip check
* fix(rpc): Check for panics in the transaction queue (#4046)
* Check for panics in the RPC transaction queue
* Add missing pin! and abort in the start task
* Check for transaction queue panics in tests
* Fixup a new RPC test from the main branch
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Return error code -8 for missing blocks, like zcashd does
* Test that getblock returns error code -8 for missing blocks
* Make RegexSet failures easier to read in logs
* Update lightwalletd acceptance tests for log message changes
* Add extra failure strings for lightwalletd logs
* Improve link formatting
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
* Repoint zebra image links to our new zfnd.org site for now
* Remove images/
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Implement `getblockchaininfo` RPC method
* add a test for `get_blockchain_info`
* fix tohex/fromhex
* move comment
* Update lightwalletd acceptance test for getblockchaininfo RPC (#3914)
* change(rpc): Return getblockchaininfo network upgrades in height order (#3915)
* Update lightwalletd acceptance test for getblockchaininfo RPC
* Update some doc comments for network upgrades
* List network upgrades in order in the getblockchaininfo RPC
Also:
- Use a constant for the "missing consensus branch ID" RPC value
- Simplify fetching consensus branch IDs
- Make RPC type derives consistent
- Update RPC type documentation
* Make RPC type derives consistent
* Fix a confusing test comment
* get hashand height at the same time
* fix estimated_height
* fix lint
* add extra check
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* fix typo
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* split test
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* fix(rpc): ignore an expected error in the RPC acceptance tests (#3961)
* Add ignored regexes to test command failure regex methods
* Ignore empty chain error in getblockchaininfo
We expect this error when zebrad starts up with an empty state.
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Split out ReadRequest and ReadResponse state service enums
* Simplify RPC test vectors
* Split state requests into Request and ReadRequest
* Make zebra-rpc use the new state ReadRequest
* Add the ChainTip and Network to RpcImpl
* Add the new RpcImpl fields to the tests
* Simplify RPC version field using generics
* Temporarily allow unused struct fields
* feat(rpc): implement `getrawmempool` rpc method
* tests(rpc): start implementing a test
* Return a `Vec<String>` from `get_raw_mempool`
Ensure the returned type is serialized as an array of hexadecimal
strings.
* Make `getrawmempool` test a property test
Test it with random sets of transaction IDs.
* Fix code formatting inside `proptest!`
Run `rustfmt` inside the `proptest!` block.
* tests(rpc): fix lightwalletd test
* fix(rpc): remove non used return type
* tests(rpc): remove assert
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Add state service module docs and cleanup
* Move and add finalized state methods
* Add chain and non-finalized state methods
* Cleanup methods and imports
* Create a ReadStateService type
* Add a stub service implementation
* Add a TODO
* Update ReadStateService request stubs with RPC names and tickets
* Documentation updates
* Make RPC State generic bounds accept a buffered state and a read-only state
* Doc updates
* Add missing proptest-impl feature in RPC dev dependencies
* feature(rpc): start adding a `getblock` method
* fix(rpc): replace oneshot
* fix(rpc): replace a panic with error
* fix(rpc): fix test
* feature(rpc): add hex to response
* refactor(rpc): use generic instead of alias
* docs(rpc): improve docs for getblock method
* test(rpc): add a test for getblock method
* deps(rpc): remove non needed tower features
Co-authored-by: teor <teor@riseup.net>
* docs(rpc): add a note to getblock doc
* refactor(rpc): replace alias
* fix(rpc): use `zcash_serialize_to_vec()` instead of logging format
* tests(rpc): add network argument to `populated_state()`
* refactor(rpc): use an error for state service readiness
* fix(rpc): add parameter
* fix(rpc): clippy
* nit(rpc): remove new line from imports
* fix(rpc): remove commented code
* fix(rpc): simplify error
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Use a `SerializedBlock` type to help serializing blocks (#3725)
* Create a `SerializedBlock` helper type
Create a type that can be used as a byte slice, but is guaranteed to
represent a valid block.
* Use `into_iter` instead of `iter`
There's no need to borrow the elements, they can be moved out directly.
This will be necessary because `&Arc<T>` doesn't implement `Borrow<T>`,
so a `SerializedBlock` can't be built directly from an `&Arc<Block>`.
* Use `SerializedBlock` in `GetBlock`
Make the type stricter to avoid storing possibly invalid values. The
bytes are still serialized as a hexadecimal string, through the usage of
`hex`.
The `serde::Deserialize` can't be derived because `hex` requires the
type to also implement `FromHex`.
* feature(rpc): add suggestions from code review
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* tests(rpc): make sure mempool has no requests in get_block test
* fix(rpc): change height argument type in getblock method
* fix(rpc): rustfmt
* fix(rpc): replace panic
* fix(rpc): change getblock response
* fix(rpc): fix lightwalletd test
* tests(rpc): add a getblock error test
* fix(rpc): try another regex
* feature(rpc): add `getbestblockhash` RPC method
* feature(rpc): Add a `pub struct SerializedBlockHash` type
* tests(rpc): add a unit test for `getbestblockhash` method
* tests(rpc): make sure no requests are sent to mempool in getbestblockhash test
* tests(rpc): refactor check
Co-authored-by: teor <teor@riseup.net>
* fix(rpc): fixes after rebase
* refactor(rpc): refactor `GetBestBlockHash`
* fix(rpc): unused variables
Co-authored-by: teor <teor@riseup.net>
* docs(rpc): update
* fix(rpc): add panic
* fix(rpc): fix panic
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
teor
Alfredo Garcia
Arya
Conrado Gouvea
Marek
Janito Vaqueiro Ferreira Filho
Dimitris Apostolou
Deirdre Connolly