* Regroup imports in `.../tests/common/regtest.rs`
* Regroup imports in `.../methods/tests/prop.rs`
* Regroup imports in `.../get_block_template.rs`
* Regroup imports in `.../zip317.rs`
* Regroup imports in `.../get_blockchain_info.rs`
* Regroup imports in `.../types/get_raw_mempool.rs`
* Regroup imports in `.../types/transaction.rs`
* Regroup imports in `.../server/tests/vectors.rs`
* Regroup imports in `.../args.rs`
* Regroup imports in `.../main.rs`
* Regroup imports in `.../openapi-generator/main.rs`
* Regroup imports in `zebrad/src/commands/start.rs`
* Regroup imports in `.../tests/fake_peer_set.rs`
* Regroup imports in `.../components/miner.rs`
* Regroup imports in `.../tests/acceptance.rs`
* Regroup imports in `.../get_block_template.rs`
* Introduce Nu7
* Introduce Nu7 fo other crates of Zebra
* Fix of new_regtest call (as it additionally needs Nu7 arg now)
* Fix of new_regtest call (as it additionally needs Nu7 arg now) (2)
* Set Nu7 as a network update for testnet in zebra-chain network tests
* Fix serde names for NU7
* Update test snapshot in zebra-rpc to use NU7
* Copy Nu7-related changes from zsa-integration-state
* Uncomment zcash_unstable = nu7
* Fix Nu7 constants and add cfg(zcash_unstable ...) attribute according to PR #15 review comments
* Update network constant values according to PR #15 review comments
* Add zcash_unstable=nu6 Rust flag to .cargo/config.toml to use nu6 by default
* Add zcash_unstable = nu6 cfg in zebra-network/.../types.rs
* Fix nu7 activation heights in network_upgrade.rs (nu6 value + 1 does not work - causes a test failure)
* Add cfg for CURRENT_NETWORK_PROTOCOL_VERSION constant definition, add FIXMEs for a couple of Nu6 processing cases, add processing of Nu7
* Update get_blockchain_info@testnet_10.snap
* Update get_blockchain_info@mainnet_10.snap
* updated cfg flag
* remove `zcash_unstable` flag
* Remove testnet and mainnet NU7 activation heights, comment out consensus branch id and conversion to zcash primitives nu type
* Yep, it'll work after NU7
* Yep, the test should work fine after NU7
* Yep, it only needs to be a post-nu6 height.
* other fixes and cleanups
* Updates snapshots, removes unnecessary FIXMEs, fixes issues with bad merge conflict resolution
* fixes test hang when there are NUs defined without activation heights
* fixes test, applies suggestion from code review.
* Enable consensus branch ID for Nu7
---------
Co-authored-by: Dmitry Demin <dmitry@qed-it.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
The `HistoryTree` type is intended for internal use within the node; it
should not be exposed via the ReadStateService response. Instead, this
response should simply include the hash needed for block template
construction.
* fix(test): wait for genesis block before launching lightwalletd
The `lightwalletd_integration` test would fail when running with an empty
`zebrad` state (`LaunchWithEmptyState` test type). `lightwalletd` expects
the connected `zebrad` instance to have processed at least the genesis
block upon startup and treats a completely empty state as a fatal error.
This adds a wait step specifically for the empty state scenario.
The test now waits for `zebrad` to log that it has committed the genesis
block (Height 0) before launching the `lightwalletd` process. This
ensures `zebrad` is ready and prevents `lightwalletd` from exiting
prematurely.
* fix(test): use syncer state tip log for lightwalletd launch condition
* fix(test): lint
* introduce new fields to `GetInfo`
* add address book to methods for getinfo connections field
* add the version field to getinfo
* track last node error or warning and display it in getinfo rpc method
* add the rest of the fields, minor cleanup
* fix the tests
* clippy
* feat(network): Add misbehavior tracking for peers in the address book
* - Add a `misbehavior_score` field to `MetaAddr`
- Add a `bans_by_ip` field to `AddressBook`
- Update the `AddressBook::update()` method to:
- increment misbehavior scores in its entries,
- add addr ips to bans_by_ip if the score is excessive,
- remove any addrs at the banned ip
- Avoid responding to `GetAddr` requests with addresses of misbehaving peers (return None from `sanitized()`),
- Avoid new inbound or outbound connections to banned ips
* Drops banned peer connections in peer set's `poll_ready()` method
* Adds rudimentary misbehavior score tracking
* fixes some proptests, moves and removes some TODOs
* fixes lint
* Removes outdated TODO
* Adds stub for acceptance test
* updates call to updated fn
* Stores likely inbound peer connection addresses in address book and return their IPs with the default port instead of the transient port when responding to GetAddr requests
* Avoids gossiping peer addrs from inbound connections
* updates test to check that sanitize won't return inbound peer addrs or addrs with non-zero misbehaviour scores.
updated misbehaviour score for potentially unavoidable errors.
* Updates `generate` RPC to support any network where PoW is disabled.
Updates acceptance test to check that zebrad instances disconnect once one of them advertises a block with an invalid PoW
* minor tangential cleanup
* Finishes acceptance test, sends misbehavior updates from syncer, and always updates address book entries if the update is to their misbehaviour score
* skip test on windows (to minimize risk of port conflict)
* Applies suggestions from code review
* Refactor formatting & docs
* Refactor the `runtime` stage in Dockerfile
* Remove unused code from `entrypoint.sh`
* Simplify `entrypoint.sh` setup
* Revise docs & formatting
* Adjust default values for env vars
* Bump Rust v from 1.79 to 1.81 in Dockerfile
* Refactor `entrypoint.sh`
* Refactor `Dockerfile`
* Add TODOs for monitoring stage to Dockerfile
* Refactor `Dockerfile`
* Add TODOs for monitoring stage to Dockerfile
* Fix a typo
* Allow running `zebrad` in test mode
* Allow custom config for `zebrad` in test mode
* Remove `curl` from the `runtime` Docker image
* Remove redundant echos
* Remove a malfunctioning CD test
The test was using a custom config file set in `test_variables`.
However, the file was not included in the Docker image, and the
entrypoint script created a new, default one under the original file's
path. Zebra then loaded this new file, and the test passed because the
pattern in `grep_patterns` matched Zebra's output containing the
original path, even though the config file was different.
* Remove a redundant CI test
* Remove all packages from the `runtime` stage
* Docs cosmetics
* Clarify docs
* Bump Rust version
* Remove a security note
* Explicitly specify network cache dir
* Explicitly specify cookie dir
* Set UID, GID and home dir for the `zebra` user
* Set a working dir for the `zebra` user
* Don't remove `FEATURES`
* Try re-introducing the `testnet-conf` check
* `ZEBRA_CACHED_STATE_DIR` -> `ZEBRA_CACHE_DIR`
This dir doesn't hold only the state cache anymore, but also the cache
for network peers, and the cookie file.
* Refactor the dir structure
* Check that `ZEBRA_CONF_PATH` exists in the image
* Improve the check for `ZEBRA_CONF_PATH`
* Use different flag in the `ZEBRA_CONF_PATH` check
* Simplify the `ZEBRA_CONF_PATH` check
* Fix spelling
* Comment out the `testnet-conf` CI check
* Add commented out `test-zebra-conf-path` CI check
* Reintroduce `testnet-conf` CI check
* Update the `custom-conf` CI check
* Add `v2.1.0.toml` conf file
* Refine the `v2.1.0.toml` conf file
* Remove `ZEBRA_LISTEN_ADDR` from the entrypoint
* Remove `ZEBRA_CHECKPOINT_SYNC` from the entrypoint
* Stop supporting configuration of the RPC port
* Add default conf file
* Prepare Zebra's config in the entrypoint script
* Remove unneeded packages from the `deps` target
* Docs cosmetics
* Use only `$FEATURES` in entrypoint
* Simplify handling of Rust features
* Add a TODO
* Add CI debug statements
* Don't require test vars in conf test
* Reintroduce `protoc`
* Remove `-e NETWORK`
* Remove `ZEBRA_FORCE_USE_COLOR=1`
* Remove `ZEBRA_CACHE_DIR=/var/cache/zebrad-cache`
* Reintroduce the "custom-conf" test
* Set up test env the same way as prod
* Don't repeatedly check for conf file in entrypoint
* Simplify file ownership in Dockerfile
* Fix checkpoint tests in entrypoint
* Fix Zebra config CI tests
* `LIGHTWALLETD_DATA_DIR` -> `LWD_CACHE_DIR`
* Add config for `LWD_CACHE_DIR` to Dockerfile
* `/var/cache/zebrad-cache` -> `~/.cache/zebra`
* `var/cache/lwd-cache` -> `/home/zebra/.cache/lwd`
* Remove `LOG_COLOR=false` from GCP setup
* Don't specify `LWD_CACHE_DIR` in CI tests
* Don't switch to `zebra` user for tests in Docker
* Join "experimental" and "all" tests in CI
* Remove outdated docs
* Refactor tests with fake activation heights
* Fix tests for scanner
* add a channel for submit_block notifications to gossip task
* fix tests and gossip logic
* remove the network discriminant and add a test
* clippy suggestions
* fix unused variable
* attempt to fix the conditional compilation issues
* fix default
* Suggestions for "fix(mining): Advertise mined blocks" (#9183)
* refactor error conversions in GetBlockTemplateRpcImpl and rewords documentation
* Replaces polling mined block receiver with a select
* Skip checking that Zebra is likely synced to the network tip before returning block templates on Testnet.
* fixes a clippy lint and a concurrency bug
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Adds new column family for [spent_out_loc] -> [spending_tx_loc] with a read method and an update to `prepare_spending_transparent_tx_ids_batch()` for maintaining it when committing blocks to the finalized state.
Adds TODOs for remaining production changes needed for issue #8837.
* add spending tx ids for spent outpoints to non-finalized chains
* adds a `spending_transaction_hash()` read fn for the new column family
* Adds a `TransactionIdForSpentOutPoint` ReadRequest and a `TransactionId` ReadResponse
* Updates snapshots, removes outdated TODOs, moves a TODO.
* Clarifies `spent_utxos` field docs, fixes an assertion
* import TypedColumnFamily from `finalized_state` instead of from the crate.
* adds db format upgrade for spent outpoints -> tx hash
* adds revealing tx ids for nullifiers in finalized and non-finalized states
* updates nullifiers column families to include revaling transaction locations in db format upgrade
* Renames new read state request to `SpendingTransactionId` and updates its type to a `Spend` enum
* refactor db format upgrade and prepare_nullifiers_batch() to use ZebraDb instead of DiskDb, checks cancel_receiver before every db operation
* Adds acceptance test for checking that the finalized state has spending transaction ids
* Adds variant docs to zebra_state::request::Spend enum
* Updates Zebra book with the latest changes to the rocks db column families
* Updates acceptance test to check non-finalized state
* adds a few log messages to the acceptance test, reduces frequency of logs for progress updates
* fixes docs lint and skips test when there is no cached state
* Avoids returning genesis coinbase tx hash when indexes are missing
* Adds `indexer` compilation feature in zebra-state and build metadata in db format version file
* stops tracking new indexes in finalized state when feature is unselected
* stops tracking new indexes in non-finalized state when indexer feature is unselected
* condenses imports
* - adds build metadata when writing db version file, if any.
- adds the build metadata to the db version file before adding indexes.
- deletes indexes when running without the `indexer` feature
* Replaces dropping cf with deleting range of all items to avoid a panic when trying to open the db with that column family.
* Fixes lint, avoids reading coinbase transactions from disk
* updates db column families table
* Document need for having an indexed cached state and use a multi-threaded tokio runtime in has_spending_transaction_ids test
* fixes call to renamed `future_blocks` test fn
* improves test logs and fixes a disk format deserialization bug
* Replaces a new expr with a previously existing constant, fixes typo
* Adds a test to check for crates in the Cargo.lock file that are being pulled in from a git source.
* add `check_no_git_refs_in_cargo_lock` to CI
* try skip instead of exclude
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Adds a parameter to `zebra_consensus::router::init()` for accepting a mempool setup argument, adds and uses an `init_test()` fn for passing a closed channel receiver in tests where no mempool service is needed in the transaction verifier.
* Adds a `mempool` argument to the transaction::Verifier constructor (and a `new_for_tests()` constructor for convenience)
* Removes `Clone` impl on `transaction::Verifier` to add mempool oneshot receiver, updates tests.
* Adds TODOs
* updates transaction verifier's poll_ready() method to setup the mempool service handle.
* Updates VerifiedSet struct used in mempool storage
* Updates mempool service and its `Storage` to use the updated `VerifiedSet` `transactions()` return type.
* updates `created_outputs` when inserting or removing a transaction from the mempool's verified set
* Adds a TODO, updates field docs
* Updates `spent_utxos()` to query the mempool for unspent outputs
* Adds `spent_mempool_outpoints` as a field on tx verifier mempool response
* Updates mempool `Downloads` to return the spent_mempool_outpoints from the tx verifier response
* Updates `Storage.insert()` to accept a list of spent mempool transaction outputs
* Adds transaction dependencies when inserting a tx in `VerifiedSet`
* polls mempool svc from tx verifier when a mempool tx that creates transparent outputs has been verified.
adds a TODO for adding a `pending_outputs` field to the mempool Storage
* Adds `pending_outputs` field on mempool Storage and responds to pending outputs requests when inserting new transactions into the mempool's verified set
* replaces `UnminedTxId` type with `transaction::Hash` in mempool's verified set
* prune pending outputs when rejecting and removing same effects.
* Remove dependent transactions from verified set when removing a tx
* updates tests
* appeases clippy.
* removes unused `len()` method
* fixes doc links
* Adds transaction dependencies to the `FullTransactions` response, let the caller handle it (required to avoid moving zip317 tx selection code to mempool)
* updates block template construction to avoid including transactions unless their dependencies have already been added.
* updates tests
* Replaces placeholder setup channel with one that sends the mempool svc to the tx verifier, adds a timeout layer, adds a TODO about a concurrency bug
* Use a single query to check for unspent outputs in the mempool
* Updates `getblocktemplate` method to consider dependencies when sorting transactions for the final template
* fixes clippy lints, removes unnecessary Option in UnspentOutput response variant
* renames type alias and method, adds a TODO to use iteration instead of recursion
* Adds mempool_removes_dependent_transactions() test
* Updates Storage and VerifiedSet clear() methods to clear pending_outputs, created_outputs, and transaction_dependencies, adds TODO to use iteration instead of recursion.
* removes outdated TODO
* Adds a TODO for reporting queued transaction verification results from the mempool from the poll_ready() method
* Adds `mempool_responds_to_await_output` test
* updates mempool_responds_to_await_output test
* Uses iteration instead of recursion in verified set's remove() method and zip317 mod's dependencies_depth() method
* Adds a mempool_request_with_mempool_output_is_accepted test for the transaction verifier
* Moves delay duration before polling the mempool to a constant, uses a shorter timeout for mempool output lookups, adds a `poll_count` to MockService, and updates `mempool_request_with_unmined_output_spends_is_accepted` to check that the transaction verifier polls the mempool after verifying a mempool transaction with transparent outputs
* adds long_poll_input_mempool_tx_ids_are_sorted test
* Adds a `excludes_tx_with_unselected_dependencies` test
* Updates a TODO
* moves `TransactionDependencies` struct to `zebra-node-services`
* Updates `FullTransactions` response variant's `transaction_dependencies` type
* updates zip317 transaction selection for block templates to include dependent transactions
* Moves and refactors zip317 tx selection test to its own module, adds an `unmined_transactions_in_blocks()` method on network
* Removes `unmined_transactions_in_blocks()` test utility fn from mempool Storage test module and replaces calls to it with calls to the new test method on Network
* Fixes spelling mistake
* Adds `includes_tx_with_selected_dependencies` test
* fixes zip317 block construction issue
* Fixes vectors test
* Update zebra-node-services/src/mempool.rs
* restores `tip_rejected_exact` type
* updates affected tests
* Documents the new argument in `Storage::insert()`, updates outdated comment
* Update zebrad/src/components/mempool/storage/verified_set.rs
* fixes potential issue with calling buffered mempool's poll_ready() method without calling it.
* Avoids removing dependent transactions of transactions that have been mined onto the best chain.
* Updates `spent_utxos()` method documentation
* Avoids sorting getblocktemplate transactions in non-test compilations
* documents PendingOutputs struct
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* cargo fmt
* Applies suggestions from code review
Avoids unnecessarily rejecting dependent transactions of randomly evicted mempool transactions.
Updates `TransactionDependencies::remove_all()` to omit provided transaction id from the list of removed transaction ids.
* Applies suggestions from code review.
* Adds minor comments
* Update zebrad/src/components/mempool/storage/verified_set.rs
Co-authored-by: Marek <mail@marek.onl>
* Remove an outdated comment (#9013)
---------
Co-authored-by: Marek <mail@marek.onl>
* ref(ci): consolidate cached states workflows and scripts
We've been using multiple approaches to locate and retrieve cached states in GCP. However, this has made it difficult to reuse the same methods across new workflows or different scenarios.
To address this, we've streamlined the process to make it more reusable in other contexts. This change will support deploying instances from both the `main` branch and `release`, simplifying future implementations and speeding up the process.
Changes:
- Use a single bash script (`gcp-get-cached-disks.sh`) to get cached states names and availability
- Move script logic from `sub-find-cached-disks.yml` to `gcp-get-cached-disks.sh` and adapt `sub-find-cached-disks.yml` to allow to output available disks and disks names.
- Simplify parameters usage in `sub-deploy-integration-tests-gcp.yml` and convert the `Find ${{ inputs.test_id }} cached state disk` step into an independent job, to be able to use the `sub-find-cached-disks.yml` reusable workflow
- Remove repetition in `sub-ci-integration-tests-gcp.yml`
* ref(tests): Use the `ZEBRA_CACHED_STATE_DIR` env var across tests
We had a technical debt with some tests using a hardcoded value for the cache directory (`/zebrad-cache`), which generated inconsistency across disks and cached states directories.
Changes:
- Allow sync tests to use the `ZEBRA_CACHED_STATE_DIR` as the cache directory, if specified
- Update the `entrypoint.sh` to reflect this change
- Add the `ZEBRA_CACHED_STATE_DIR` variable to the missing tests in `sub-ci-integration-tests-gcp.yml`, and remove extra parameters to call reusable workflows.
* temporally upgrade zcash primitives to github main branch
* add allowed git repo in deny.toml
* add quotes to url
* fix denies
* change issue number for TODOs
* fix test name
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
Co-authored-by: Marek <mail@marek.onl>
* Addresses clippy lints
* checks network magic and returns early from `is_regtest()`
* Moves `subsidy.rs` to `zebra-chain`, refactors funding streams into structs, splits them into pre/post NU6 funding streams, and adds them as a field on `testnet::Parameters`
* Replaces Vec with HashMap, adds `ConfiguredFundingStreams` type and conversion logic with constraints.
Minor refactors
* Empties recipients list
* Adds a comment on num_addresses calculation being invalid for configured Testnets, but that being okay since configured testnet parameters are checked when they're being built
* Documentation fixes, minor cleanup, renames a test, adds TODOs, and fixes test logic
* Removes unnecessary `ParameterSubsidy` impl for &Network, adds docs and TODOs
* Adds a "deferred" FundingStreamReceiver, adds a post-NU6 funding streams, updates the `miner_fees_are_valid()` and `subsidy_is_valid()` functions to check that the deferred pool contribution is valid and that there are no unclaimed block subsidies after NU6 activation, and adds some TODOs
* adds `lockbox_input_value()` fn
* Adds TODOs for linking to relevant ZIPs and updating height ranges
* Adds `nu6_lockbox_funding_stream` acceptance test
* updates funding stream values test to check post-NU6 funding streams too, adds Mainnet/Testnet NU6 activation heights, fixes lints/compilation issue
* Reverts Mainnet/Testnet NU6 activation height definitions, updates `test_funding_stream_values()` to use a configured testnet with the post-NU6 Mainnet funding streams height range
* reverts unnecessary refactor
* appease clippy
* Adds a test for `lockbox_input_value()`
* Applies suggestions from code review
* Fixes potential panic
* Fixes bad merge
* Update zebra-chain/src/parameters/network_upgrade.rs
* Updates acceptance test to check that invalid blocks are rejected
* Checks that the original valid block template at height 2 is accepted as a block submission
* Reverts changes for coinbase should balance exactly ZIP
* updates test name
* Updates deferred pool funding stream name to "Lockbox", moves post-NU6 height ranges to constants, updates TODO
* Updates `get_block_subsidy()` RPC method to exclude lockbox funding stream from `fundingstreams` field
* Adds a TODO for updating `FundingStreamReceiver::name()` method docs
* Updates `FundingStreamRecipient::new()` to accept an iterator of items instead of an option of an iterator, updates a comment quoting the coinbase transaction balance consensus rule to note that the current code is inconsistent with the protocol spec, adds a TODO for updating the quote there once the protocol spec has been updated.
* Uses FPF Testnet address for post-NU6 testnet funding streams
* Updates the NU6 consensus branch id
* checks that coinbase transactions balance exactly
* updates test name
* Add a TODO
* Refactor `miner_fees_are_valid`
---------
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
Co-authored-by: Marek <mail@marek.onl>
* Addresses clippy lints
* checks network magic and returns early from `is_regtest()`
* Moves `subsidy.rs` to `zebra-chain`, refactors funding streams into structs, splits them into pre/post NU6 funding streams, and adds them as a field on `testnet::Parameters`
* Replaces Vec with HashMap, adds `ConfiguredFundingStreams` type and conversion logic with constraints.
Minor refactors
* Empties recipients list
* Adds a comment on num_addresses calculation being invalid for configured Testnets, but that being okay since configured testnet parameters are checked when they're being built
* Documentation fixes, minor cleanup, renames a test, adds TODOs, and fixes test logic
* Removes unnecessary `ParameterSubsidy` impl for &Network, adds docs and TODOs
* Adds a "deferred" FundingStreamReceiver, adds a post-NU6 funding streams, updates the `miner_fees_are_valid()` and `subsidy_is_valid()` functions to check that the deferred pool contribution is valid and that there are no unclaimed block subsidies after NU6 activation, and adds some TODOs
* adds `lockbox_input_value()` fn
* Adds TODOs for linking to relevant ZIPs and updating height ranges
* Adds `nu6_lockbox_funding_stream` acceptance test
* updates funding stream values test to check post-NU6 funding streams too, adds Mainnet/Testnet NU6 activation heights, fixes lints/compilation issue
* Reverts Mainnet/Testnet NU6 activation height definitions, updates `test_funding_stream_values()` to use a configured testnet with the post-NU6 Mainnet funding streams height range
* reverts unnecessary refactor
* appease clippy
* Adds a test for `lockbox_input_value()`
* Applies suggestions from code review
* Fixes potential panic
* Fixes bad merge
* Update zebra-chain/src/parameters/network_upgrade.rs
* Updates acceptance test to check that invalid blocks are rejected
* Checks that the original valid block template at height 2 is accepted as a block submission
* Reverts changes for coinbase should balance exactly ZIP
* updates test name
* Updates deferred pool funding stream name to "Lockbox", moves post-NU6 height ranges to constants, updates TODO
* Updates `get_block_subsidy()` RPC method to exclude lockbox funding stream from `fundingstreams` field
* Adds a TODO for updating `FundingStreamReceiver::name()` method docs
* Updates `FundingStreamRecipient::new()` to accept an iterator of items instead of an option of an iterator, updates a comment quoting the coinbase transaction balance consensus rule to note that the current code is inconsistent with the protocol spec, adds a TODO for updating the quote there once the protocol spec has been updated.
* Uses FPF Testnet address for post-NU6 testnet funding streams
* Updates the NU6 consensus branch id
---------
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
* adds a tonic server
* Adds a test stub, moves method impls to their own modules, minor fixes.
* Moves indexer rpc mod behind a feature, adds a config field for its listen address, and initializes the indexer RPC when zebrad starts
* Skips tonic_build() in zebra-rpc build script unless indexer-rpcs feature is selected, simplifies indexer.proto file, makes tonic deps optional
* formats zebra-rpc Cargo.toml
* Adds tokio_stream dependency, adds chain_tip_change field to IndexerRPC, and implements a simple version of the chain_tip_change RPC method
* passes latest chain tip to indexer::server::init from start cmd and updates vectors test
* Update zebra-rpc/src/config.rs
* fixes a race condition in trusted_chain_sync_handles_forks_correctly
* readd the scanner tests in the new binary context
* remove commented out tests from zebrad
* add Cargo.lock
* add local copy of `ZECPAGES_SAPLING_VIEWING_KEY` to binary scanning tests
* moves scanner-results-reader to zebra-scan, adds zebrad bin to zebra-scan, adds a short sleep before trying to connect to gRPC server in test
* always include zebrad dependency in zebra-scan
* renames zebrad bin in zebra-scan to zebrad-for-scanner
* skip test when a cached state is missing
* remove scanning config from zebrad
* move `scan_task_commands` out of zebrad acceptance tests
* remove scanner config from zebrad
* do not panic if env var is not there
* re enable start_scan_where left, remove shielded-scan feature foir scanning tests in CI
* change app names
* uncomment test
* fix the scanner tests in CI by changing the test command
* add log needed by CI?
* move test
* rename trait, docs fix
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Adds an init_read_only() fn in zebra-state
* moves elasticsearch initialization to `FinalizedState::new_with_debug()`
* Updates callers of `FinalizedState::{new, new_with_debug}` to pass a bool to try enabling elasticsearch
* Adds a non-finalized read state syncer to zebra-rpc
* moves, removes, updates, or addresses TODOs
* reduces disk IO while waiting for the a new chain tip & updates the chain tip sender when the finalized tip has changed.
* Returns boxed errors from RpcRequestClient methods instead of color_eyre type
* Avoids resetting the non-finalized state when there's an error getting a block unless it has the missing block error code.
* Adds stub for acceptance test(s) and removes outdated TODO
* adds TODOs for testing
* Tests that `ChainTipChange` is updated when the non-finalized best chain grows
* adds a last_chain_tip_hash and uses a FuturesOrdered for getblock requests
* Fixes a pre-flush sync issue by using a secondary db instead of a read-only db
* Moves disk IO to blocking tasks
* Updates acceptance test to how forks are handled
* Checks synced read state for all of the expected blocks
* checks that there isn't a tip change until the best chain changes
* checks for chain tip changes in test
* run test without feature
* fixes lint
* Fixes compilation/test issues
* Adds docs / comments, moves HexData out from behind the getblocktemplate-rpcs feature flag, moves test behind the mining feature flag.
* Fixes lints
* removes syncer and rpc-syncer features
* Fixes test on Windows, applies suggestions from code review
* Updates `POLL_DELAY` documentation
* Updates method docs
* Fixes a test bug
* use rpc-client feature in zebrad production code
* use rpc-client feature in zebra-node-services for building zebra-rpc crate
---------
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Uses an OS-assigned port when testing rpc_server_spawn
* reads port from logs and enables 2 acceptance tests on Windows
* generalizes read listen addr from logs fn and re-enables another test
* updates zebra-grpc snapshot tests to use OS-assigned port
* Re-enable grpc vectors tests on Windows while avoiding port conflicts
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove code that starts the scanner
* Update CHANGELOG
* Disable tests of the scanner
* Disable scanning tests in CI
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Allow configurable Nu5 activation height on Regtest
* Fixes test
* removes outdated TODO
* Adds `current_with_activation_height()` method and uses it in `Commitment::from_bytes()`
* Always activate Canopy at Height(1) on Regtest
* Ignores the zip 212 grace period on configured Testnets and Regtest
* - Returns early when there is no Heartwood activation height when creating or updating HistoryTree
- Skips call to `check::legacy_chain()` when no NU5 activation height is set (it would return immediately anyway)
- Replaces `.map()` with `.filter_map()` in `NetworkUpgrade::target_spacings()`
- Removes outdated TODO
* - When proof of work is disabled, skips checking if Zebra is synced to the network tip in the getblocktemplate method
* Sets full_verifier_utxo_lookahead to Height::MIN instead of panicking
* When network is regtest, skips starting sync task and commits the genesis block if it's missing in the state
* updates/fixes test config
* Adds test for committing Canopy blocks on Regtest
* - Updates median time past and difficulty checks to use fewer than 11/17/28 blocks
* uses SLOW_START_INTERVAL of 0 if PoW is disabled, adds TODOs
* Update getblocktemplate method to return reserved chain history activation root hash, uses Nu5 at height 1
Test passes.
* Updates test to expect NU5 as the default nu activation at Height(1)
* Removes invalid difficulty snapshot
* fixes tests
* removes regtest NU5 activation height config field
* Apply suggestions from code review
* Update zebra-state/src/service/check/difficulty.rs
Co-authored-by: Marek <mail@marek.onl>
* Update zebra-state/src/service/check/difficulty.rs
* Updates docs/comments, renames an argument
---------
Co-authored-by: Marek <mail@marek.onl>
* minor cleanup and rename
* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.
* Updates tests
* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`
* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing
* Applies some suggestions from code review
* Applies suggestions from code review
* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network
* Applies more suggestions from code review.
* moves conversions to zcash_primitives::consensus::Network to where they're used.
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* rename `network` variables and method names typed as NetworkKind to `network_kind`
* use only test block heights for the network associated with them
* Applies more suggestions from code review.
* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module
* adds activation heights field
* updates stored test config, adds a quicker test for checking that stored configs can be parsed, adds an intermediate representation of activation heights
* implement Parameters for Network
* Passes &Network directly instead of converting to zp_consensus::Network where there were conversions
* fixes a bad merge (removes a network conversion in zcash_note_encryption)
* Adds a test for the Parameters impl for zebra_chain::Network
* fixes doc links
* - Makes the `activation_heights` config field optional by adding a #[serde(default)]
- Panics if a non-zero activation height is provided for the `Genesis` network upgrade
- Always sets the `Genesis` and `BeforeOverwinter` network upgrade activation heights to 0 and 1, `BeforeOverwinter` could be overwritten by a later network upgrade
- Makes the `activation_heights` field on `Parameters` private, adds/uses an accessor method instead, and adds a builder struct and `build()` method
* small refactor of activation_heights() method
* check that activation heights are in the right order
* Updates `NetworkUpgrade::activation_height()` to return the height of the next NetworkUpgrade if it doesn't find the activation height of `&self`
* checks that the miner address is of TestnetKind on Regtest and update assertion message
* Adds a DNetworkUpgradeActivationHeights struct for better control over how activation heights can be configured
* moves all ordered network upgrades to a constant, adds a no_duplicates test, moves struct with activation heights outside deserialization impl and accepts it in `ParametersBuilder::activation_heights()` instead of a Vec
* panics if any network upgrades are configured to activate at Height(0) because it's reserved for Genesis
* Simplifies the `ParametersBuilder::activation_heights()` method and removes an unnecessary test.
* Adds Sapling HRPs as fields on testnet params. (#8398)
* Applies suggestions from code review.
* Update zebra-chain/src/parameters/network_upgrade.rs
Co-authored-by: Marek <mail@marek.onl>
* Adds `network_name` field and accessor method on `Parameters`, uses it in the `Display` impl for `Network`
* Removes unnecessary `.filter()`
* adds constraints on valid network names and a test
* adds config field for setting network name, adds "with_" prefix to ParameterBuilder setter methods
* Adds `MainnetKind`, `TestnetKind`, and `RegtestKind` to reserved network names
* updates stored test configs and fixes `network_name()` docs
* Adds a `new_regtest()` method on `Network` and `testnet::Parameters`, updates config deserialization to return an error if the initial_testnet_peers include any default initial peers AND contain configured activation heights
* Updates `activates_network_upgrades_correctly` test to check Regtest activation heights (and default Mainnet/Testnet)
* Refactors if-let & match statement into general match statement, removes constraint against including `testnet_parameters` field/section in the config when using `Mainnet` or `Regtest`
* Removes outdated TODO
* Restores `testnet_parameters` section of the latest stored config.
* Adds `with_sapling_hrps()` method and uses it to set the Regtest HRPs in `new_regtest()`.
Adds a test for Sapling HRP validation
* Checks that default Mainnet/Testnet/Regtest Sapling HRPs pass validation in `with_sapling_hrps()`
* Uses the correct constant in test
* Adds `is_regtest()` methods
* Adds genesis hash methods and fields to `testnet::Parameters` and its builder
* Copies Regtest genesis block from zcashd
* moves genesis hash check to Network.checkpoint_list()
* Checks the first line in the checkpoint list instead of the first item in the BTreeMap
* Checks that there is _some_ genesis block in the checkpoints list in `from_list()`, adds Regtest checkpoints (which only includes the genesis block)
* Adds a doc comment to `ParametersBuilder::default()`
* Adds a `disable_pow` field and skips checking proof of work if it's true
* Makes `equihash::Solution` an enum to support Regtest solutions, adds a test for validating and committing the Regtest genesis block
* use genesis_hash as first checkpoint when checkpoints are missing for configured testnets too
* Applies suggestions from code review.
* Reverts changes to `activation_height()` method
* Avoids panic in test
* Updates test docs
* drop custom panic hooks after expected panics
---------
Co-authored-by: Marek <mail@marek.onl>
* update zcash_script and zcash_primitives
* restore windows support
* add a windows config file
* try exact output from CI
* ignore config test in windows
* disable test for windows
* remove test for windows
* change zcash_script branch to release
* bump top the last zcash_script release version
* restore `rejection_restores_internal_state_genesis` test
* fix typo in test name and enable single thread on windows
* disable single thread
* minor cleanup and rename
* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.
* Updates tests
* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`
* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing
* Applies some suggestions from code review
* Applies suggestions from code review
* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network
* Applies more suggestions from code review.
* moves conversions to zcash_primitives::consensus::Network to where they're used.
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* rename `network` variables and method names typed as NetworkKind to `network_kind`
* use only test block heights for the network associated with them
* Applies more suggestions from code review.
* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module
* adds activation heights field
* updates stored test config, adds a quicker test for checking that stored configs can be parsed, adds an intermediate representation of activation heights
* implement Parameters for Network
* Passes &Network directly instead of converting to zp_consensus::Network where there were conversions
* fixes a bad merge (removes a network conversion in zcash_note_encryption)
* Adds a test for the Parameters impl for zebra_chain::Network
* fixes doc links
* - Makes the `activation_heights` config field optional by adding a #[serde(default)]
- Panics if a non-zero activation height is provided for the `Genesis` network upgrade
- Always sets the `Genesis` and `BeforeOverwinter` network upgrade activation heights to 0 and 1, `BeforeOverwinter` could be overwritten by a later network upgrade
- Makes the `activation_heights` field on `Parameters` private, adds/uses an accessor method instead, and adds a builder struct and `build()` method
* small refactor of activation_heights() method
* check that activation heights are in the right order
* Updates `NetworkUpgrade::activation_height()` to return the height of the next NetworkUpgrade if it doesn't find the activation height of `&self`
* checks that the miner address is of TestnetKind on Regtest and update assertion message
* Adds a DNetworkUpgradeActivationHeights struct for better control over how activation heights can be configured
* moves all ordered network upgrades to a constant, adds a no_duplicates test, moves struct with activation heights outside deserialization impl and accepts it in `ParametersBuilder::activation_heights()` instead of a Vec
* panics if any network upgrades are configured to activate at Height(0) because it's reserved for Genesis
* Simplifies the `ParametersBuilder::activation_heights()` method and removes an unnecessary test.
* Adds Sapling HRPs as fields on testnet params. (#8398)
* Applies suggestions from code review.
* Update zebra-chain/src/parameters/network_upgrade.rs
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* fix beta imports and other warnings
* clippy lints beta
* move attributes to after docs
Co-authored-by: Marek <mail@marek.onl>
* move some code to test module
* implement display instead of direct tostring for LongPollId
Co-authored-by: Arya <aryasolhi@gmail.com>
* fix typo
---------
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Arya <aryasolhi@gmail.com>
* minor cleanup and rename
* Adds an empty NetworkParameters struct to Network::Testnet variant, updates production code.
* Updates tests
* Adds `NetworkKind` and uses it instead of `Network` in `HistoryTreeParts` and `transparent::Address`
* Adds a [network.testnet_parameters] section to the config, uses `NetworkKind` as zebra_network::Config::network field type, and converts 'Network' to `NetworkKind` before serializing
* Applies some suggestions from code review
* Applies suggestions from code review
* returns b58 prefix constants directly to remove From<NetworkKind> impl for zcash_primitives::consensus::Network
* Applies more suggestions from code review.
* moves conversions to zcash_primitives::consensus::Network to where they're used.
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* rename `network` variables and method names typed as NetworkKind to `network_kind`
* use only test block heights for the network associated with them
* Applies more suggestions from code review.
* Rename `NetworkParameters` to `Parameters` and move it a new `testnet` module
---------
Co-authored-by: Marek <mail@marek.onl>
* removed derive copy from network, address and ledgerstate
* changed is_max_block_time_enforced to accept ref
* changed NetworkUpgrade::Current to accept ref
* changed NetworkUpgrade::Next to accept ref
* changed NetworkUpgrade::IsActivationHeight to accept ref
* changed NetworkUpgrade::TargetSpacingForHeight to accept ref
* changed NetworkUpgrade::TargetSpacings to accept ref
* changed NetworkUpgrade::MinimumDifficultySpacing_forHeight to accept ref
* changed NetworkUpgrade::IsTestnetMinDifficultyBlock to accept ref
* changed NetworkUpgrade::AveragingWindowTimespanForHeight to accept ref
* changed NetworkUpgrade::ActivationHeight to accept ref
* changed sapling_activation_height to accept ref
* fixed lifetime for target_spacings
* fixed sapling_activation_height
* changed transaction_to_fake_v5 and fake_v5_transactions_for_network to accept ref to network
* changed Input::vec_strategy to accept ref to network
* changed functions in zebra-chain/src/primitives/zcash_history.rs to accept ref to network
* changed functions in zebra-chain/src/history_tree.rs to accept ref to network
* changed functions in zebra-chain/src/history_tree.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/address.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/viewing_key* to accept ref to network
* changed functions in zebra-chain/src/transparent/address.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/zcash_primitives.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/zcash_note_encryption.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/history_tree* to accept ref to network
* changed functions in zebra-chain/src/block* to accept ref to network
* fixed errors in zebra-chain::parameters::network
* fixed errors in zebra-chain::parameters::network
* fixed errors in zebra-chain
* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref
* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref
* fixed errors in zebra-chain/src/block/arbitrary.rs
* finished fixing errors in zebra-chain - all crate tests pass
* changed functions in zebra-state::service::finalized_state to accept &Network
* changed functions in zebra-state::service::non_finalized_state to accept &Network
* zebra-state tests run but fail with overflow error
* zebra-state tests all pass
* converted zebra-network -- all crate tests pass
* applied all requested changes from review
* converted zebra-consensus -- all crate tests pass
* converted zebra-scan -- all crate tests pass
* converted zebra-rpc -- all crate tests pass
* converted zebra-grpc -- all crate tests pass
* converted zebrad -- all crate tests pass
* applied all requested changes from review
* fixed all clippy errors
* fixed build error in zebrad/src/components/mempool/crawler.rs
* merges scan task tests and checks that the scan task removes keys
* Adds test to CI
* Skip to expected block height and assert that the result is from that height
* Updates test docs and uses zebrad state path for scan cache
* Log "Zcash network: {network}" from scan task commands test
* processes SubscribeResults messages
* send tx ids of results to the subscribe channel
* replaces BoxError with Report in scan_range
* adds a watch channel for using subscribed_keys in scan_range
* updates args to process_messages in test
* adds a `subscribe` method to ScanTask for sending a SubscribeResults cmd
* updates test for process_messages to cover subscribe cmds
* impls SubscribeResult service request and updates sender type
* adds test for SubscribeResults scan service request
* adds acceptance test
* updates tests and imports
* fixes acceptance test by using spawn_blocking to avoid blocking async executor and setting an appropriate start height
* fixes test
* Applies suggestions from code review.
* use tokio mpsc channel in scan task instead of std/blocking mpsc
* use tokio mpsc channel for results sender
* adds `was_parsed_keys_empty` instead of checking that all the parsed keys are new keys
* fixes test failures related to send errors in scan task
* returns height and key for scan results from subcribe_results results receiver
* hide scan_service mod in zebra-node-service behind feature
* adds clear_results RPC method for zebra-scan
* adds delete_keys rpc method
* adds docs
* Update zebra-grpc/proto/scanner.proto
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Apply suggestions from code review
* start zebra-scan gRPC server from zebrad start command
* adds a test that the scanner starts with zebrad
* adds a `listen_addr` field to the shielded scan config
* updates test to use a random port and set the listen_addr config field
* fixes test
* Update zebra-scan/src/config.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* fixes panic when trying to open multiple mutable storage instances.
* open db in blocking task
* fixes test
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* moves ScanTask to its own module in service module
* moves `process_messages()` method to scan_task.rs
* uses get_mut() and returns new keys
* updates types and adds scan_until() function
* adds and uses wait_for_height() function
* spawns scan_until_task_handler to poll scan_until_tasks FuturesUnordered
* corrects comment
* adds TODO
* updates expected test log, corrects panic msg, fixes fmt
* moves scan functions to scan_task module
* moves ScanTaskCommand and related method impls to its own module
* moves `ScanTask::mock()` test constructor to its own module
* creates `add_keys` mod and moves `scan_until()` function there
* moves scan task executor to its own module and adds ScanRangeTaskBuilder type
* renames add_keys to scan_range, moves executor to scan_task mod
* adds test for process_messages
* updates scan_height_and_store_results() to skip last_scanned_height check if there is no key height for a key in `parsed_keys`
* updates `default_test_config()`
* adds acceptance test for registering keys in ScanTask
* uses the right feature for the new acceptance test
* Applies suggestions from code review
Marek
Paul
Kris Nuttycombe
Gustavo Valverde
Arya
Alfredo Garcia
dependabot[bot]
pinglanlu
idky137