* Regroup imports in `.../tests/common/regtest.rs`
* Regroup imports in `.../methods/tests/prop.rs`
* Regroup imports in `.../get_block_template.rs`
* Regroup imports in `.../zip317.rs`
* Regroup imports in `.../get_blockchain_info.rs`
* Regroup imports in `.../types/get_raw_mempool.rs`
* Regroup imports in `.../types/transaction.rs`
* Regroup imports in `.../server/tests/vectors.rs`
* Regroup imports in `.../args.rs`
* Regroup imports in `.../main.rs`
* Regroup imports in `.../openapi-generator/main.rs`
* Regroup imports in `zebrad/src/commands/start.rs`
* Regroup imports in `.../tests/fake_peer_set.rs`
* Regroup imports in `.../components/miner.rs`
* Regroup imports in `.../tests/acceptance.rs`
* Regroup imports in `.../get_block_template.rs`
* Update changelog
* Update end of support
* Add release date
* Add breaking change description
* Add new PRs to changelog
* Add breaking change description to changelog
* Update list of PRs in changelog
* chore: Release
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* introduce new fields to `GetInfo`
* add address book to methods for getinfo connections field
* add the version field to getinfo
* track last node error or warning and display it in getinfo rpc method
* add the rest of the fields, minor cleanup
* fix the tests
* clippy
* feat(network): Add misbehavior tracking for peers in the address book
* - Add a `misbehavior_score` field to `MetaAddr`
- Add a `bans_by_ip` field to `AddressBook`
- Update the `AddressBook::update()` method to:
- increment misbehavior scores in its entries,
- add addr ips to bans_by_ip if the score is excessive,
- remove any addrs at the banned ip
- Avoid responding to `GetAddr` requests with addresses of misbehaving peers (return None from `sanitized()`),
- Avoid new inbound or outbound connections to banned ips
* Drops banned peer connections in peer set's `poll_ready()` method
* Adds rudimentary misbehavior score tracking
* fixes some proptests, moves and removes some TODOs
* fixes lint
* Removes outdated TODO
* Adds stub for acceptance test
* updates call to updated fn
* Stores likely inbound peer connection addresses in address book and return their IPs with the default port instead of the transient port when responding to GetAddr requests
* Avoids gossiping peer addrs from inbound connections
* updates test to check that sanitize won't return inbound peer addrs or addrs with non-zero misbehaviour scores.
updated misbehaviour score for potentially unavoidable errors.
* Updates `generate` RPC to support any network where PoW is disabled.
Updates acceptance test to check that zebrad instances disconnect once one of them advertises a block with an invalid PoW
* minor tangential cleanup
* Finishes acceptance test, sends misbehavior updates from syncer, and always updates address book entries if the update is to their misbehaviour score
* skip test on windows (to minimize risk of port conflict)
* Applies suggestions from code review
* add a channel for submit_block notifications to gossip task
* fix tests and gossip logic
* remove the network discriminant and add a test
* clippy suggestions
* fix unused variable
* attempt to fix the conditional compilation issues
* fix default
* Suggestions for "fix(mining): Advertise mined blocks" (#9183)
* refactor error conversions in GetBlockTemplateRpcImpl and rewords documentation
* Replaces polling mined block receiver with a select
* Skip checking that Zebra is likely synced to the network tip before returning block templates on Testnet.
* fixes a clippy lint and a concurrency bug
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove unnecessary clone
* Fix test in zebra-script
* Document that RPC server addr must be set
* Change the log level of missing port msg to info
Having the RPC port not specified is part of Zebra's normal operation,
and is the default config.
* Fix links in docs for tracing
* Update zebrad/src/commands/start.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Adds new column family for [spent_out_loc] -> [spending_tx_loc] with a read method and an update to `prepare_spending_transparent_tx_ids_batch()` for maintaining it when committing blocks to the finalized state.
Adds TODOs for remaining production changes needed for issue #8837.
* add spending tx ids for spent outpoints to non-finalized chains
* adds a `spending_transaction_hash()` read fn for the new column family
* Adds a `TransactionIdForSpentOutPoint` ReadRequest and a `TransactionId` ReadResponse
* Updates snapshots, removes outdated TODOs, moves a TODO.
* Clarifies `spent_utxos` field docs, fixes an assertion
* import TypedColumnFamily from `finalized_state` instead of from the crate.
* adds db format upgrade for spent outpoints -> tx hash
* adds revealing tx ids for nullifiers in finalized and non-finalized states
* updates nullifiers column families to include revaling transaction locations in db format upgrade
* Renames new read state request to `SpendingTransactionId` and updates its type to a `Spend` enum
* refactor db format upgrade and prepare_nullifiers_batch() to use ZebraDb instead of DiskDb, checks cancel_receiver before every db operation
* Adds acceptance test for checking that the finalized state has spending transaction ids
* Adds variant docs to zebra_state::request::Spend enum
* Updates Zebra book with the latest changes to the rocks db column families
* Updates acceptance test to check non-finalized state
* adds a few log messages to the acceptance test, reduces frequency of logs for progress updates
* fixes docs lint and skips test when there is no cached state
* Avoids returning genesis coinbase tx hash when indexes are missing
* Adds `indexer` compilation feature in zebra-state and build metadata in db format version file
* stops tracking new indexes in finalized state when feature is unselected
* stops tracking new indexes in non-finalized state when indexer feature is unselected
* condenses imports
* - adds build metadata when writing db version file, if any.
- adds the build metadata to the db version file before adding indexes.
- deletes indexes when running without the `indexer` feature
* Replaces dropping cf with deleting range of all items to avoid a panic when trying to open the db with that column family.
* Fixes lint, avoids reading coinbase transactions from disk
* updates db column families table
* Document need for having an indexed cached state and use a multi-threaded tokio runtime in has_spending_transaction_ids test
* fixes call to renamed `future_blocks` test fn
* improves test logs and fixes a disk format deserialization bug
* Replaces a new expr with a previously existing constant, fixes typo
* skips re-verifying transactions in blocks that are present in the mempool.
* clippy fix
* adds a test
* fixes clippy lint
* Use NU6 & V5 tx in new test
* Uses correct consensus branch id in test
* change(mempool): Return verification result after attempting to insert transactions in the mempool (#8901)
* respond with mempool verification result after a transaction has been inserted or has failed to be inserted into the mempool
* returns mempool verification errors early, and fixes handling for cancellations or timeouts.
* Adds a comment in test warning against code reuse with buffered services.
* De-duplicates handling for timeout errors
* Adds a parameter to `zebra_consensus::router::init()` for accepting a mempool setup argument, adds and uses an `init_test()` fn for passing a closed channel receiver in tests where no mempool service is needed in the transaction verifier.
* Adds a `mempool` argument to the transaction::Verifier constructor (and a `new_for_tests()` constructor for convenience)
* Removes `Clone` impl on `transaction::Verifier` to add mempool oneshot receiver, updates tests.
* Adds TODOs
* updates transaction verifier's poll_ready() method to setup the mempool service handle.
* Updates VerifiedSet struct used in mempool storage
* Updates mempool service and its `Storage` to use the updated `VerifiedSet` `transactions()` return type.
* updates `created_outputs` when inserting or removing a transaction from the mempool's verified set
* Adds a TODO, updates field docs
* Updates `spent_utxos()` to query the mempool for unspent outputs
* Adds `spent_mempool_outpoints` as a field on tx verifier mempool response
* Updates mempool `Downloads` to return the spent_mempool_outpoints from the tx verifier response
* Updates `Storage.insert()` to accept a list of spent mempool transaction outputs
* Adds transaction dependencies when inserting a tx in `VerifiedSet`
* polls mempool svc from tx verifier when a mempool tx that creates transparent outputs has been verified.
adds a TODO for adding a `pending_outputs` field to the mempool Storage
* Adds `pending_outputs` field on mempool Storage and responds to pending outputs requests when inserting new transactions into the mempool's verified set
* replaces `UnminedTxId` type with `transaction::Hash` in mempool's verified set
* prune pending outputs when rejecting and removing same effects.
* Remove dependent transactions from verified set when removing a tx
* updates tests
* appeases clippy.
* removes unused `len()` method
* fixes doc links
* Adds transaction dependencies to the `FullTransactions` response, let the caller handle it (required to avoid moving zip317 tx selection code to mempool)
* updates block template construction to avoid including transactions unless their dependencies have already been added.
* updates tests
* Replaces placeholder setup channel with one that sends the mempool svc to the tx verifier, adds a timeout layer, adds a TODO about a concurrency bug
* Use a single query to check for unspent outputs in the mempool
* Updates `getblocktemplate` method to consider dependencies when sorting transactions for the final template
* fixes clippy lints, removes unnecessary Option in UnspentOutput response variant
* renames type alias and method, adds a TODO to use iteration instead of recursion
* Adds mempool_removes_dependent_transactions() test
* Updates Storage and VerifiedSet clear() methods to clear pending_outputs, created_outputs, and transaction_dependencies, adds TODO to use iteration instead of recursion.
* removes outdated TODO
* Adds a TODO for reporting queued transaction verification results from the mempool from the poll_ready() method
* Adds `mempool_responds_to_await_output` test
* updates mempool_responds_to_await_output test
* Uses iteration instead of recursion in verified set's remove() method and zip317 mod's dependencies_depth() method
* Adds a mempool_request_with_mempool_output_is_accepted test for the transaction verifier
* Moves delay duration before polling the mempool to a constant, uses a shorter timeout for mempool output lookups, adds a `poll_count` to MockService, and updates `mempool_request_with_unmined_output_spends_is_accepted` to check that the transaction verifier polls the mempool after verifying a mempool transaction with transparent outputs
* adds long_poll_input_mempool_tx_ids_are_sorted test
* Adds a `excludes_tx_with_unselected_dependencies` test
* Updates a TODO
* moves `TransactionDependencies` struct to `zebra-node-services`
* Updates `FullTransactions` response variant's `transaction_dependencies` type
* updates zip317 transaction selection for block templates to include dependent transactions
* Moves and refactors zip317 tx selection test to its own module, adds an `unmined_transactions_in_blocks()` method on network
* Removes `unmined_transactions_in_blocks()` test utility fn from mempool Storage test module and replaces calls to it with calls to the new test method on Network
* Fixes spelling mistake
* Adds `includes_tx_with_selected_dependencies` test
* fixes zip317 block construction issue
* Fixes vectors test
* Update zebra-node-services/src/mempool.rs
* restores `tip_rejected_exact` type
* updates affected tests
* Documents the new argument in `Storage::insert()`, updates outdated comment
* Update zebrad/src/components/mempool/storage/verified_set.rs
* fixes potential issue with calling buffered mempool's poll_ready() method without calling it.
* Avoids removing dependent transactions of transactions that have been mined onto the best chain.
* Updates `spent_utxos()` method documentation
* Avoids sorting getblocktemplate transactions in non-test compilations
* documents PendingOutputs struct
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* cargo fmt
* Applies suggestions from code review
Avoids unnecessarily rejecting dependent transactions of randomly evicted mempool transactions.
Updates `TransactionDependencies::remove_all()` to omit provided transaction id from the list of removed transaction ids.
* Applies suggestions from code review.
* Adds minor comments
* Update zebrad/src/components/mempool/storage/verified_set.rs
Co-authored-by: Marek <mail@marek.onl>
* Remove an outdated comment (#9013)
---------
Co-authored-by: Marek <mail@marek.onl>
* Adds a mempool request to wait for a transaction verification result and uses it in `sendrawtransaction` RPC method
* removes unnecessary clone
* fix clippy warnings
* returns verification errors for all `mempool::Queue` requests, removes `QueueRpc` request variant
* returns oneshot channel in mempool::Response::Queue
* updates a test vector to check for download or verification error in mempool::response::Queued result receiver
* Always require tokio as a dependency in zebra-node-services
* checks for closed channel errors in sendrawtransaction and updates a prop test to check that verification errors are propagated correctly
* Updates `EOS_PANIC_AFTER` end of support constant from 16 weeks to 14 weeks, so the Mainnet panic height is just before the second halving and end of the current dev fund.
* version bumps
* Updates CHANGELOG.md
* Updates changelog
* updates end of support time to 10 weeks
* Adds a note to the changelog about recovering after finalizing blocks from a fork
* Add new changes on main to CHANGELOG.md
* Updates ESTIMATED_RELEASE_HEIGHT
* Apply suggestions from code review
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Update zebrad/src/components/sync/end_of_support.rs
---------
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* adds a tonic server
* Adds a test stub, moves method impls to their own modules, minor fixes.
* Moves indexer rpc mod behind a feature, adds a config field for its listen address, and initializes the indexer RPC when zebrad starts
* Skips tonic_build() in zebra-rpc build script unless indexer-rpcs feature is selected, simplifies indexer.proto file, makes tonic deps optional
* formats zebra-rpc Cargo.toml
* Adds tokio_stream dependency, adds chain_tip_change field to IndexerRPC, and implements a simple version of the chain_tip_change RPC method
* passes latest chain tip to indexer::server::init from start cmd and updates vectors test
* Update zebra-rpc/src/config.rs
* fixes a race condition in trusted_chain_sync_handles_forks_correctly
* readd the scanner tests in the new binary context
* remove commented out tests from zebrad
* add Cargo.lock
* add local copy of `ZECPAGES_SAPLING_VIEWING_KEY` to binary scanning tests
* moves scanner-results-reader to zebra-scan, adds zebrad bin to zebra-scan, adds a short sleep before trying to connect to gRPC server in test
* always include zebrad dependency in zebra-scan
* renames zebrad bin in zebra-scan to zebrad-for-scanner
* skip test when a cached state is missing
* remove scanning config from zebrad
* move `scan_task_commands` out of zebrad acceptance tests
* remove scanner config from zebrad
* do not panic if env var is not there
* re enable start_scan_where left, remove shielded-scan feature foir scanning tests in CI
* change app names
* uncomment test
* fix the scanner tests in CI by changing the test command
* add log needed by CI?
* move test
* rename trait, docs fix
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Adds an init_read_only() fn in zebra-state
* moves elasticsearch initialization to `FinalizedState::new_with_debug()`
* Updates callers of `FinalizedState::{new, new_with_debug}` to pass a bool to try enabling elasticsearch
* Adds a non-finalized read state syncer to zebra-rpc
* moves, removes, updates, or addresses TODOs
* reduces disk IO while waiting for the a new chain tip & updates the chain tip sender when the finalized tip has changed.
* Returns boxed errors from RpcRequestClient methods instead of color_eyre type
* Avoids resetting the non-finalized state when there's an error getting a block unless it has the missing block error code.
* Adds stub for acceptance test(s) and removes outdated TODO
* adds TODOs for testing
* Tests that `ChainTipChange` is updated when the non-finalized best chain grows
* adds a last_chain_tip_hash and uses a FuturesOrdered for getblock requests
* Fixes a pre-flush sync issue by using a secondary db instead of a read-only db
* Moves disk IO to blocking tasks
* Updates acceptance test to how forks are handled
* Checks synced read state for all of the expected blocks
* checks that there isn't a tip change until the best chain changes
* checks for chain tip changes in test
* run test without feature
* fixes lint
* Fixes compilation/test issues
* Adds docs / comments, moves HexData out from behind the getblocktemplate-rpcs feature flag, moves test behind the mining feature flag.
* Fixes lints
* removes syncer and rpc-syncer features
* Fixes test on Windows, applies suggestions from code review
* Updates `POLL_DELAY` documentation
* Updates method docs
* Fixes a test bug
* use rpc-client feature in zebrad production code
* use rpc-client feature in zebra-node-services for building zebra-rpc crate
---------
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Update `CHANGELOG.md`
* chore: Release
* Update versions in the release dry-run workflow
* Update EOS
* Estimate release height to ~ upcoming Tuesday
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* Add a "Summary" title to the changelog
* Remove the summary title
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove code that starts the scanner
* Update CHANGELOG
* Disable tests of the scanner
* Disable scanning tests in CI
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Always set mempool.debug_enable_at_height to 0 on Regtest
* Adds a Regtest with Zebra page to the book
* Deduplicates calls to is_regtest() in start fn
* upgrade min rust version in zebrad
* Display full network name in logs except for Mainnet and default Testnet
* Add example logs/code to regtest section of the book
* apply some extra information
Co-authored-by: Arya <aryasolhi@gmail.com>
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
* Reduce the end of support time from 20 weeks to 16 weeks.
* Update release-process section of the book
* Adds changelog entry
* Update CHANGELOG.md
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Allow configurable Nu5 activation height on Regtest
* Fixes test
* removes outdated TODO
* Adds `current_with_activation_height()` method and uses it in `Commitment::from_bytes()`
* Enables parts of the internal miner for use on Regtest when the solution isn't checked
* Update internal miner config field docs
* Allow configurable Nu5 activation height on Regtest
* Fixes test
* removes outdated TODO
* Adds `current_with_activation_height()` method and uses it in `Commitment::from_bytes()`
* changelog and readme for 1.7.0 release
* chore: Release
* update zebra-scan and zebra-grpc to the release crates dry run script
* update ESTIMATED_RELEASE_HEIGHT
* update the release period to 6 weeks
* update release block
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* Always activate Canopy at Height(1) on Regtest
* Ignores the zip 212 grace period on configured Testnets and Regtest
* - Returns early when there is no Heartwood activation height when creating or updating HistoryTree
- Skips call to `check::legacy_chain()` when no NU5 activation height is set (it would return immediately anyway)
- Replaces `.map()` with `.filter_map()` in `NetworkUpgrade::target_spacings()`
- Removes outdated TODO
* - When proof of work is disabled, skips checking if Zebra is synced to the network tip in the getblocktemplate method
* Sets full_verifier_utxo_lookahead to Height::MIN instead of panicking
* When network is regtest, skips starting sync task and commits the genesis block if it's missing in the state
* updates/fixes test config
* Adds test for committing Canopy blocks on Regtest
* - Updates median time past and difficulty checks to use fewer than 11/17/28 blocks
* uses SLOW_START_INTERVAL of 0 if PoW is disabled, adds TODOs
* Update getblocktemplate method to return reserved chain history activation root hash, uses Nu5 at height 1
Test passes.
* Updates test to expect NU5 as the default nu activation at Height(1)
* Removes invalid difficulty snapshot
* fixes tests
* removes regtest NU5 activation height config field
* Apply suggestions from code review
* Update zebra-state/src/service/check/difficulty.rs
Co-authored-by: Marek <mail@marek.onl>
* Update zebra-state/src/service/check/difficulty.rs
* Updates docs/comments, renames an argument
---------
Co-authored-by: Marek <mail@marek.onl>
* fix beta imports and other warnings
* clippy lints beta
* move attributes to after docs
Co-authored-by: Marek <mail@marek.onl>
* move some code to test module
* implement display instead of direct tostring for LongPollId
Co-authored-by: Arya <aryasolhi@gmail.com>
* fix typo
---------
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Arya <aryasolhi@gmail.com>
* minor cleanup and rename
* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.
* Updates tests
* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`
* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing
* Applies some suggestions from code review
* Applies suggestions from code review
* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network
* Applies more suggestions from code review.
* moves conversions to zcash_primitives::consensus::Network to where they're used.
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* rename `network` variables and method names typed as NetworkKind to `network_kind`
* use only test block heights for the network associated with them
* Applies more suggestions from code review.
* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module
---------
Co-authored-by: Marek <mail@marek.onl>
* Adds logging of column family size and database size on startup and shutdown
* Changes log level of column families size strings to debug. Adds TODO comment to use human_bytes crate for human-readable format of metrics. Adds print_db_metrics function to ZebraDb struct.
* Calls enumerate() on column_families to access index var
* Resolves cargo fmt checker results
* Resolves clippy lint
* Runs and fixes changes from fmt, clippy, check and test
* Removes prop.txt
* minor doc changes
---------
Co-authored-by: Elijah Hampton <elijahhampton@MBP-de-Elijah-2.lan>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Marek
Arya
Mark Henderson
Alfredo Garcia
Conrado Gouvea
pinglanlu
dependabot[bot]
Elijah Hampton