* removed derive copy from network, address and ledgerstate
* changed is_max_block_time_enforced to accept ref
* changed NetworkUpgrade::Current to accept ref
* changed NetworkUpgrade::Next to accept ref
* changed NetworkUpgrade::IsActivationHeight to accept ref
* changed NetworkUpgrade::TargetSpacingForHeight to accept ref
* changed NetworkUpgrade::TargetSpacings to accept ref
* changed NetworkUpgrade::MinimumDifficultySpacing_forHeight to accept ref
* changed NetworkUpgrade::IsTestnetMinDifficultyBlock to accept ref
* changed NetworkUpgrade::AveragingWindowTimespanForHeight to accept ref
* changed NetworkUpgrade::ActivationHeight to accept ref
* changed sapling_activation_height to accept ref
* fixed lifetime for target_spacings
* fixed sapling_activation_height
* changed transaction_to_fake_v5 and fake_v5_transactions_for_network to accept ref to network
* changed Input::vec_strategy to accept ref to network
* changed functions in zebra-chain/src/primitives/zcash_history.rs to accept ref to network
* changed functions in zebra-chain/src/history_tree.rs to accept ref to network
* changed functions in zebra-chain/src/history_tree.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/address.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/viewing_key* to accept ref to network
* changed functions in zebra-chain/src/transparent/address.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/zcash_primitives.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/zcash_note_encryption.rs to accept ref to network
* changed functions in zebra-chain/src/primitives/history_tree* to accept ref to network
* changed functions in zebra-chain/src/block* to accept ref to network
* fixed errors in zebra-chain::parameters::network
* fixed errors in zebra-chain::parameters::network
* fixed errors in zebra-chain
* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref
* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref
* fixed errors in zebra-chain/src/block/arbitrary.rs
* finished fixing errors in zebra-chain - all crate tests pass
* changed functions in zebra-state::service::finalized_state to accept &Network
* changed functions in zebra-state::service::non_finalized_state to accept &Network
* zebra-state tests run but fail with overflow error
* zebra-state tests all pass
* converted zebra-network -- all crate tests pass
* applied all requested changes from review
* converted zebra-consensus -- all crate tests pass
* converted zebra-scan -- all crate tests pass
* converted zebra-rpc -- all crate tests pass
* converted zebra-grpc -- all crate tests pass
* converted zebrad -- all crate tests pass
* applied all requested changes from review
* fixed all clippy errors
* fixed build error in zebrad/src/components/mempool/crawler.rs
* Patch equihash to use the solver branch
* Add an internal-miner feature and set up its dependencies
* Remove 'Experimental' from mining RPC docs
* Fix a nightly clippy::question_mark lint
* Move a byte array utility function to zebra-chain
* fixup! Add an internal-miner feature and set up its dependencies
* Add an equihash::Solution::solve() method with difficulty checks
* Check solution is valid before returning it
* Add a TODO to check for peers before mining
* Move config validation into GetBlockTemplateRpcImpl::new()
* fixup! fixup! Add an internal-miner feature and set up its dependencies
* Use the same generic constraints for GetBlockTemplateRpcImpl struct and impls
* Start adding an internal miner component
* Add the miner task to the start command
* Add basic miner code
* Split out a method to mine one block
* Spawn to a blocking thread
* Wait until a valid template is available
* Handle shutdown
* Run mining on low priority threads
* Ignore some invalid solutions
* Use a difference nonce for each solver thread
* Update TODOs
* Change the patch into a renamed dependency to simplify crate releases
* Clean up instrumentation and TODOs
* Make RPC instances cloneable and clean up generics
* Make LongPollId Copy so it's easier to use
* Add API to restart mining if there's a new block template
* Actually restart mining if there's a new block template
* Tidy instrumentation
* fixup! Move config validation into GetBlockTemplateRpcImpl::new()
* fixup! Make RPC instances cloneable and clean up generics
* Run the template generator and one miner concurrently
* Reduce logging
* Fix a bug in getblocktemplate RPC tip change detection
* Work around some watch channel change bugs
* Rate-limit template changes in the receiver
* Run one mining solver per available core
* Use updated C code with double-free protection
* Update to the latest solver branch
* Return and submit all valid solutions
* Document what INPUT_LENGTH means
* Fix watch channel change detection
* Don't return early when a mining task fails
* Spawn async miner tasks to avoid cooperative blocking, deadlocks, and improve shutdown responsiveness
* Make existing parallelism docs and configs consistent
* Add a mining parallelism config
* Use the minimum of the configured or available threads for mining
* Ignore optional feature fields in tests
* Downgrade some frequent logs to debug
* Document new zebrad features and tasks
* Describe the internal-miner feature in the CHANGELOG
* Update dependency to de-duplicate equihash solutions
* Use futures::StreamExt instead of TryStreamExt
* Fix a panic message typo
* Provide access to DiskDb and DiskWriteBatch outside the state using a feature
* Actually let's export ZebraDb for the format upgrade code
* Pass column families to ZebraDb as an argument
* Allow the database kind to be changed in config.rs
* Use the state kind in finalized_state.rs
* Allow different database kinds in ZebraDb, but don't move the upgrade code yet
* Allow different database kinds in DiskDb
* Allow different database kinds in upgrade.rs, but don't split the upgrade code out yet
* Add new arguments to raw database tests
* Fix doc links
* Fix internal imports
* Fix unused code
* Update zebrad version metadata
* Create a specific state database delete function
* Fix state exports
* Fix zebrad tests
* Fix zebrad state write tests
* Make CI run again
* Fix dead code warnings for test methods
* Remove unnecessary async on some tests
* Fix logging required by tests
* Fix logging required in test itself
* Fix variable names
* Try to copy the message and add regexes
* Skip external branches in ci-unit-tests-docker.yml
* Patch external repositories in ci-unit-tests-docker.patch.yml
* Revert ci-unit-tests-docker.patch.yml
* Add a separate external PR patch in ci-unit-tests-docker.patch-external.yml
* Document ci-unit-tests-docker.patch.yml
* Document ci-unit-tests-docker.yml
* Skip dependabot config validation for external PRs
* Trivial Rust code modification to make Rust workflows run
* Use correct method for checking source branch refs
* Fix the check condition
* Simplify job structure
* Use correct skip condition
* Make the start docs visible so we can link to them
* Skip integration tests on external PRs
* Allow external PRs to pass CD Docker tests
* Allow external PRs to pass Docs build and deploy
* Add a progress bar config that is disabled unless the feature is on
* Simplify the default config
* Enable the progress bar feature by default, but require the config
* Rename progress bars config to avoid merge conflicts
* Use a log file when the progress bar is activated
* Document how to configure progress bars
* Handle log files in config_tests and check config path
* Fix doc link
* Fix path check
* Fix config log matching
* Fix clippy warning
* Add tracing to config tests
* It's zebrad not zebra
* cargo fmt --all
* Update release for config file changes
* Fix config test failures
* Allow printing to stdout in a method
* Move format upgrades to their own module and enum
* Launch a format change thread if needed, and shut it down during shutdown
* Add some TODOs and remove a redundant timer
* Regularly check for panics in the state upgrade task
* Only run example upgrade once, change version field names
* Increment database format to 25.0.2: add format change task
* Log the running and initial disk database format versions on startup
* Add initial disk and running state versions to cached state images in CI
* Fix missing imports
* Fix typo in logs workflow command
* Add a force_save_to_disk argument to the CI workflow
* Move use_internet_connection into zebrad_config()
* fastmod can_spawn_zebrad_for_rpc can_spawn_zebrad_for_test_type zebra*
* Add a spawn_zebrad_without_rpc() function
* Remove unused copy_state() test code
* Assert that upgrades and downgrades happen with the correct versions
* Add a kill_and_return_output() method for tests
* Add a test for new_state_format() versions (no upgrades or downgrades)
* Add use_internet_connection to can_spawn_zebrad_for_test_type()
* Fix workflow parameter passing
* Check that reopening a new database doesn't upgrade (or downgrade) the format
* Allow ephemeral to be set to false even if we don't have a cached state
* Add a test type that will accept any kind of state
* When re-using a directory, configure the state test config with that path
* Actually mark newly created databases with their format versions
* Wait for the state to be opened before testing the format
* Run state format tests on mainnet and testnet configs (no network access)
* run multiple reopens in tests
* Test upgrades run correctly
* Test that version downgrades work as expected (best effort)
* Add a TODO for testing partial updates
* Fix missing test arguments
* clippy if chain
* Fix typo
* another typo
* Pass a database instance to the format upgrade task
* Fix a timing issue in the tests
* Fix version matching in CI
* Use correct env var reference
* Use correct github env file
* Wait for the database to be written before killing Zebra
* Use correct workflow syntax
* Version changes aren't always upgrades
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* print a Zebra logo and some text in progress bar mode
* add network to printed line, add heart to logo
* print logo and message regardless of progress-bar; document how logo was generated
* De-duplicate app_version and user_agent code, rename to build_version
* Make RPC testnet flag forward-compatible with additional testnets
* Fix RPC tests with new argument
* Use "modified" rather than "dirty" for uncommitted changes in build metadata
* Split the vergen version into its own function
* updates Cargo.toml
* Migrate to abscissa 0.7.0
* Avoid panic from calling color_eyre::install twice
* Uses 'start' as the default subcommand
* updates default cmd logic
* Fixes minor cli issues
* removes outdated check in acceptance test
* Adds a test for process_cli_args, fixes version_args test.
Adds -V to process_cli_args match case
* Revert "fix(clippy): Silence future-incompat warnings until we upgrade Abscissa (#6024)"
This reverts commit dd90f79b48.
* Drops the worker guard to flush logs when zebra shuts down
* Adds cargo feature to clap
* restores process_cli_args
* updates deny.toml
* Updates EntryPoint help template
* Updates subcommand help msgs
* removes trailing whitespace, capitalizes sentences
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* revert parts of revert "Revert fix(clippy): Silence future-incompat warnings until we upgrade Abscissa"
* Applies suggestions from code review
* Moves EntryPoint to its own module
* fixes version_args test
* Updates changelog
* Prunes redundant test cases
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Revert "Prunes redundant test cases"
This reverts commit 3f73979184.
* Update zebrad/src/commands/entry_point.rs
Co-authored-by: teor <teor@riseup.net>
* Add missing import
* Updates `process_cli_args` to return a result
---------
Co-authored-by: teor <teor@riseup.net>
* Add extra vergen metadata to startup and panic reports
* Upgrade to vergen 8.1.3
* Update build script docs
* Upgrade env var names for vergen 8.1.3
* Switch to `git2` dependency feature
* Use renamed environmental variable for `git describe`
* Ignore vergen errors if `.git` doesn't exist
* Update Cargo.lock
* Remove unused deny.toml exceptions with --all-features
Some of these might be from this PR, or the missed dependency updates.
* add user agent as argument, use git to auto build zebra user agent
* try to fix test
* fix typo
* change expect text
* remove newline
* fix some docs
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* refuse to run Zebra if it is too old
* update the release checklist to consider the constants
* bring newline back
* apply new end of support code
* attempt to add tests (not working yet)
* move eos to progress task
* move tests
* add acceptance test (not working)
* fix tests
* change to block height checks (ugly code)
* change warn days
* refactor estimated blocks per day, etc
* move end of support code to its own task
* change test
* fix some docs
* move constants
* remove uneeded conversions
* downgrade tracing
* reduce end of support time, fix ci changing debugs to info again
* update instructions
* add failure messages
* cargo lock update
* unify releaase name constant
* change info msg
* clippy fixes
* add a block explorer
* ignore testnet in end of support task
* change panic to 16 weeks
* add some documentation about end of support
* Tweak docs wording
---------
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Log zebrad config path as well as config values
* Remove duplicate config log
* Only log config path and data for server commands
* Use config ref instead
* print usage info for --help flag
* update known issues and help command's help text
* Update README.md
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* updates README
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* adds non-blocking writer for tracing subscriber
* use non_blocking writer for the fmt::Layer with the tokio-console feature as well
* adds doc comment to _guard field
* adds acceptance test
* update filter_handle type to use NonBlocking
* adds more detail on lossy non-blocking writer and sets tracing.filter to "trace" in acceptance test
* drops ZebradApp before process::exit(1) in the event of a FrameworkError
* reduces buffered lines limit to 8000
* adds tracing.buffer_limit config and some comments
* update acceptance.rs
* fix acceptance test
* fixes ambigious phrasing in comment
* updates zebrad/src/application.rs
* Find out what the join error is in the GitHub runner tests
* updates acceptance test to use recv_timeout instead of always waiting 10 seconds, removes unnecessary echo command, and reduces # of rpc requests to 500
* see if sleeping for a few seconds before exiting helps the macOS test pass
* Expand exit sleep docs
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
* adds start as default subcommand for zebrad
* moves EntryPoint to submodule and adds a test
* moves all start tests to config_test to avoid listener conflicts
* Update zebrad/src/application/entry_point.rs docs
* Revert "moves all start tests to config_test to avoid listener conflicts"
This reverts commit 61ce46f5a1.
* Update based on test API changes from another PR
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Initialize the rayon threadpool with a new config for CPU-bound threads
* Verify proofs and signatures on the rayon thread pool
* Only spawn one concurrent batch per verifier, for now
* Allow tower-batch to queue multiple batches
* Fix up a potentially incorrect comment
* Rename some variables for concurrent batches
* Spawn multiple batches concurrently, without any limits
* Simplify batch worker loop using OptionFuture
* Clear pending batches once they finish
* Stop accepting new items when we're at the concurrent batch limit
* Fail queued requests on drop
* Move pending_items and the batch timer into the worker struct
* Add worker fields to batch trace logs
* Run docker tests on PR series
* During full verification, process 20 blocks concurrently
* Remove an outdated comment about yielding to other tasks
* Warn the user when Zebra cannot parse `zebrad.toml`
* Test that Zebra warns the user when it cannot parse `zebrad.toml`
* Fix up a mistaken merge change
* Suggest how to fix `zebrad.toml` when Zebra cannot parse it
Co-authored-by: teor <teor@riseup.net>
* Disable the flamegraph feature by default at compile time
* Disable the journald feature by default at compile time
* Also disable inferno dependency, and rearrange features
* Disable the prometheus feature by default at compile time
* Disable the tracing filter reload feature by default at compile time
* Disable tests when corresponding features are disabled
* Add compile-time tracing features to user docs
* Add compile-time features to the metrics user docs
* Document diagnostics as part of the start command tasks and services
* breaking(diagnostics): rename "enable-sentry" feature to "sentry" (#4623)
* Also skip conflict tests when those ports are disabled
* breaking(diagnostics): rename "enable-sentry" feature to "sentry"
This is mostly:
```sh
fastmod enable-sentry sentry
```
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Fix the syntax of links in comments
* Fix a mistake in the docs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove unnecessary angle brackets from a link
* Revert the changes for links that serve as references
* Revert "Revert the changes for links that serve as references"
This reverts commit 8b091aa9fa.
* Remove `<` `>` from links that serve as references
This reverts commit 046ef25620.
* Don't use `<` `>` in normal comments
* Don't use `<` `>` for normal comments
* Revert changes for comments starting with `//`
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Move `init_tracing` to `lib.rs`
Allow it to be reused by other binaries.
* Fix `hex` dependency inclusion in `zebra-chain`
It would previously fail because `hex` was being included without the
`serde` feature, even though we required that for `transparent::Script`.
* Implement `FromStr` for `Network`
Make it easy to receive it as a command-line parameter.
* Add `zebra-tip-height` utility
Obtains the chain tip height of a directory containing Zebra state.
* Remove Tokio dependency from `zebra-utils`
It wasn't actually used by the `zebra-tip-height` utility.
* Remove `BoxStateService` type alias
It's not needed if we don't return the unused state service. This also
allows removing the `tower` dependency.
* Remove unnecessary attribute
Leftover from copied code.
* Make `cache_dir` argument optional
Fallback to the default Zebra state cache directory.
* Remove added newline
Minor formatting fix, to avoid adding an unnecessary newline.
* Move `tip-height` command into `zebrad`
Make it available as a sub-command in `zebrad`.
* Make some zebrad sub-commands only log warnings
Co-authored-by: teor <teor@riseup.net>
* Stop useless crawler attempts when there are no peers and no crawl responses
* Disable GitHub bug report URLs when the disk is full
* Add help text for the `zebrad start` tracing filter option
* Update `tower` to version `0.4.9`
Update to latest version to add support for Tokio version 1.
* Replace usage of `ServiceExt::ready_and`
It was deprecated in favor of `ServiceExt::ready`.
* Update Tokio dependency to version `1.13.0`
This will break the build because the code isn't ready for the update,
but future commits will fix the issues.
* Replace import of `tokio::stream::StreamExt`
Use `futures::stream::StreamExt` instead, because newer versions of
Tokio don't have the `stream` feature.
* Use `IntervalStream` in `zebra-network`
In newer versions of Tokio `Interval` doesn't implement `Stream`, so the
wrapper types from `tokio-stream` have to be used instead.
* Use `IntervalStream` in `inventory_registry`
In newer versions of Tokio the `Interval` type doesn't implement
`Stream`, so `tokio_stream::wrappers::IntervalStream` has to be used
instead.
* Use `BroadcastStream` in `inventory_registry`
In newer versions of Tokio `broadcast::Receiver` doesn't implement
`Stream`, so `tokio_stream::wrappers::BroadcastStream` instead. This
also requires changing the error type that is used.
* Handle `Semaphore::acquire` error in `tower-batch`
Newer versions of Tokio can return an error if the semaphore is closed.
This shouldn't happen in `tower-batch` because the semaphore is never
closed.
* Handle `Semaphore::acquire` error in `zebrad` test
On newer versions of Tokio `Semaphore::acquire` can return an error if
the semaphore is closed. This shouldn't happen in the test because the
semaphore is never closed.
* Update some `zebra-network` dependencies
Use versions compatible with Tokio version 1.
* Upgrade Hyper to version 0.14
Use a version that supports Tokio version 1.
* Update `metrics` dependency to version 0.17
And also update the `metrics-exporter-prometheus` to version 0.6.1.
These updates are to make sure Tokio 1 is supported.
* Use `f64` as the histogram data type
`u64` isn't supported as the histogram data type in newer versions of
`metrics`.
* Update the initialization of the metrics component
Make it compatible with the new version of `metrics`.
* Simplify build version counter
Remove all constants and use the new `metrics::incement_counter!` macro.
* Change metrics output line to match on
The snapshot string isn't included in the newer version of
`metrics-exporter-prometheus`.
* Update `sentry` to version 0.23.0
Use a version compatible with Tokio version 1.
* Remove usage of `TracingIntegration`
This seems to not be available from `sentry-tracing` anymore, so it
needs to be replaced.
* Add sentry layer to tracing initialization
This seems like the replacement for `TracingIntegration`.
* Remove unnecessary conversion
Suggested by a Clippy lint.
* Update Cargo lock file
Apply all of the updates to dependencies.
* Ban duplicate tokio dependencies
Also ban git sources for tokio dependencies.
* Stop allowing sentry-tracing git repository in `deny.toml`
* Allow remaining duplicates after the tokio upgrade
* Use C: drive for CI build output on Windows
GitHub Actions uses a Windows image with two disk drives, and the
default D: drive is smaller than the C: drive. Zebra currently uses a
lot of space to build, so it has to use the C: drive to avoid CI build
failures because of insufficient space.
Co-authored-by: teor <teor@riseup.net>
* Allow deliberate instances of the new nightly clippy::derivable_impls lint
We want our config defaults to be explicit.
Not so sure about the application defaults, but they also contain a config.
* Also allow unknown lint names
Stable doesn't know about this lint, but nightly does.
* Use the git version + new commit count + hash for the app version
This helps diagnose bugs in versions of Zebra built from git branches,
rather than git version tags.
* Fill in assert
* Also log semver string
* Fix syntax
* Handle vergen using the cargo package version or raw git tag
* s/Semver/SemVer/
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Enable builds where:
* there is no google cloud git commit env var, and
* there is no `.git` directory.
By making all `vergen` env vars optional, and skipping any env vars that
don't exist.
* build(deps): bump vergen from 3.2.0 to 5.1.1
* fix hardcoded version for Tracing struct
* add additional metadata
* remove extra allocations for metadata
* Remove zebrad code version from release checklist
The zebrad code automatically uses the crate version now.
* Sort panic metadata into rough categories
Co-authored-by: teor <teor@riseup.net>
* add hint for port error
* add issue filter for port panic
* add lock file hint
* add metrics endpoint port conflict hint
* add hint for tracing endpoint port conflict
* add acceptance test for resource conflics
* Split out common conflict test code into a function
* Add state, metrics, and tracing conflict tests
* Add a full set of stderr acceptance test functions
This change makes the stdout and stderr acceptance test interfaces
identical.
* move Zcash listener opening
* add todo about hint for disk full
* add constant for lock file
* match path in state cache
* don't match windows cache path
* Use Display for state path logs
Avoids weird escaping on Windows when using Debug
* Add Windows conflict error messages
* Turn PORT_IN_USE_ERROR into a regex
And add another alternative Windows-specific port error
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Jane Lusby <jane@zfnd.org>
* Add the configured network to error reports
* Log the configured network at error level
* Create the global span immediately after activating tracing
And leak the span guard, so the span is always active.
* Include panic metadata in the report and URL
* Use `Main` and `Test` in the global span
`net=Mainnet` is a bit redundant
Arya
idky137
teor
Conrado Gouvea
Alfredo Garcia
Marek
Janito Vaqueiro Ferreira Filho
Kirill Fomichev