- Updated `getblock` RPC to more closely match zcashd ([#9006](https://github.com/ZcashFoundation/zebra/pull/9006))
- Updated error messages to include inner error types (notably for the transaction verifier) ([#9066](https://github.com/ZcashFoundation/zebra/pull/9066))
### Fixed
- Validate consensus branch ids of mempool transactions ([#9063](https://github.com/ZcashFoundation/zebra/pull/9063))
- Verify mempool transactions with unmined inputs if those inputs are in the mempool to support TEX transactions ([#8857](https://github.com/ZcashFoundation/zebra/pull/8857))
- Wait until transactions have been added to the mempool before returning success response from `sendrawtransaction` RPC ([#9067](https://github.com/ZcashFoundation/zebra/pull/9067))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @conradoplg, @cypherpepe, @gustavovalverde, @idky137, @oxarbitrage, @pinglanlu and @upbqdn
- A cookie-based authentication system for the JSON RPC endpoint ([#8900](https://github.com/ZcashFoundation/zebra/pull/8900), [#8965](https://github.com/ZcashFoundation/zebra/pull/8965))
### Changed
- Set the activation height of NU6 for Mainnet and bumped Zebra's current network protocol version ([#8960](https://github.com/ZcashFoundation/zebra/pull/8960))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @oxarbitrage and @upbqdn.
This version is a release candidate for the Zcash NU6 network upgrade on the Mainnet. While this version does not yet include the NU6 Mainnet activation height or current protocol version, all required functionality and tests are in place.
Please note that support for this release candidate is expected to conclude prior to the NU6 activation heights.
- Added fields to `getmininginfo` RPC method response ([#8860](https://github.com/ZcashFoundation/zebra/pull/8860))
- Copied the Python RPC test framework from zcashd into Zebra ([#8866](https://github.com/ZcashFoundation/zebra/pull/8866))
### Changed
- Regtest halving interval to match zcashd and added a configurable halving interval for custom testnets ([#8888](https://github.com/ZcashFoundation/zebra/pull/8888), [#8928](https://github.com/ZcashFoundation/zebra/pull/8928))
- Updates post-NU6 Major Grants funding stream address on Mainnet ([#8914](https://github.com/ZcashFoundation/zebra/pull/8914))
### Fixed
- Remove debugging output by default in Docker image ([#8870](https://github.com/ZcashFoundation/zebra/pull/8870))
- Fixes a typo in configuration file path of the docker-compose file ([#8893](https://github.com/ZcashFoundation/zebra/pull/8893))
- Return verification errors from `sendrawtransaction` RPC method ([#8788](https://github.com/ZcashFoundation/zebra/pull/8788))
- Respond to getheaders requests with a maximum of 160 block headers ([#8913](https://github.com/ZcashFoundation/zebra/pull/8913))
- Avoids panicking during contextual validation when a parent block is missing ([#8883](https://github.com/ZcashFoundation/zebra/pull/8883))
- Write database format version to disk atomically to avoid a rare panic ([#8795](https://github.com/ZcashFoundation/zebra/pull/8795))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @dismad, @gustavovalverde, @oxarbitrage, @skyl and @upbqdn
Zebra configuration to use the new state cache directory.
### Added
- A `zebra-scanner` binary replacing the `shielded-scan` compilation feature in `zebrad` ([#8608](https://github.com/ZcashFoundation/zebra/pull/8608))
- Adds a `TrustedChainSync` module for keeping up with Zebra's non-finalized best chain from a separate process ([#8596](https://github.com/ZcashFoundation/zebra/pull/8596))
- Add a tonic server in zebra-rpc with a `chain_tip_change()` method that notifies clients when Zebra's best chain tip changes ([#8674](https://github.com/ZcashFoundation/zebra/pull/8674))
- Post-NU6 funding streams, including a lockbox funding stream ([#8694](https://github.com/ZcashFoundation/zebra/pull/8694))
- Add value pool balances to `getblockchaininfo` RPC method response ([#8769](https://github.com/ZcashFoundation/zebra/pull/8769))
- Add NU6 lockbox funding stream information to `getblocksubsidy` RPC method response ([#8742](https://github.com/ZcashFoundation/zebra/pull/8742))
### Changed
- Reduce the end-of-support halt time from 16 weeks to 10 weeks ([#8734](https://github.com/ZcashFoundation/zebra/pull/8734))
- Bump the current protocol version from 170100 to 170110 ([#8804](https://github.com/ZcashFoundation/zebra/pull/8804))
- Track the balance of the deferred chain value pool ([#8732](https://github.com/ZcashFoundation/zebra/pull/8732), [#8729](https://github.com/ZcashFoundation/zebra/pull/8729))
- Require that coinbase transactions balance exactly after NU6 activation ([#8727](https://github.com/ZcashFoundation/zebra/pull/8727))
- Support in-place disk format upgrades for major database version bumps ([#8748](https://github.com/ZcashFoundation/zebra/pull/8748))
### Fixed
- Return full network upgrade activation list in `getblockchaininfo` method ([#8699](https://github.com/ZcashFoundation/zebra/pull/8699))
- Update documentation for the new `zebra-scanner` binary ([#8675](https://github.com/ZcashFoundation/zebra/pull/8675))
- Update documentation for using Zebra with lightwalletd ([#8714](https://github.com/ZcashFoundation/zebra/pull/8714))
- Reduce debug output for Network on Regtest and default Testnet ([#8760](https://github.com/ZcashFoundation/zebra/pull/8760))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @conradoplg, @dependabot[bot], @oxarbitrage, @therealyingtong and @upbqdn
- Add an init function for a standalone `ReadStateService` ([#8595](https://github.com/ZcashFoundation/zebra/pull/8595))
- Allow configuring more parameters on custom Testnets and an NU5 activation height on Regtest ([#8477](https://github.com/ZcashFoundation/zebra/pull/8477), [#8518](https://github.com/ZcashFoundation/zebra/pull/8518), [#8524](https://github.com/ZcashFoundation/zebra/pull/8524), [#8528](https://github.com/ZcashFoundation/zebra/pull/8528), [#8636](https://github.com/ZcashFoundation/zebra/pull/8636))
- Add default constructions for several RPC method responses ([#8616](https://github.com/ZcashFoundation/zebra/pull/8616), [#8505](https://github.com/ZcashFoundation/zebra/pull/8505))
- Support constructing Canopy proposal blocks from block templates ([#8505](https://github.com/ZcashFoundation/zebra/pull/8505))
#### Docs
- Document custom Testnets, Regtest and how they compare to Mainnet and the default Testnet in the Zebra book ([#8526](https://github.com/ZcashFoundation/zebra/pull/8526), [#8636](https://github.com/ZcashFoundation/zebra/pull/8636))
### Changed
- Use a default unpaid action limit of 0 so that transactions with unpaid actions are excluded from the mempool and block templates by default ([#8638](https://github.com/ZcashFoundation/zebra/pull/8638))
- Lower the mandatory checkpoint height from immediately above the ZIP-212 grace period to immediately below Canopy activation ([#8629](https://github.com/ZcashFoundation/zebra/pull/8629))
- Use the new `zcash_script` callback API in `zebra-script`, allowing Zebra's ECC dependencies to be upgraded independently of `zcash_script` and `zcashd` ([#8566](https://github.com/ZcashFoundation/zebra/pull/8566))
- Put Windows in Tier 2 of supported platforms ([#8637](https://github.com/ZcashFoundation/zebra/pull/8637))
- Remove experimental support for starting the `zebra-scan` in the `zebrad` process as a step towards moving the scanner to its own process ([#8594](https://github.com/ZcashFoundation/zebra/pull/8594))
- Reduce the end of support time from 20 weeks to 16 weeks ([#8530](https://github.com/ZcashFoundation/zebra/pull/8530))
- Restore parts of the experimental `internal-miner` feature for use on Regtest ([#8506](https://github.com/ZcashFoundation/zebra/pull/8506))
### Fixed
- Allow square brackets in network config listen addr and external addr without explicitly configuring a port in the listen addr ([#8504](https://github.com/ZcashFoundation/zebra/pull/8504))
- Fix general conditional compilation attributes ([#8602](https://github.com/ZcashFoundation/zebra/pull/8602))
- Update `median_timespan()` method to align with zcashd implementation ([#8491](https://github.com/ZcashFoundation/zebra/pull/8491))
- Allow custom Testnets to make peer connections ([#8528](https://github.com/ZcashFoundation/zebra/pull/8528))
- Refactor and simplify the serialization of empty treestates to fix an incorrect serialization of empty note commitment trees for RPCs ([#8533](https://github.com/ZcashFoundation/zebra/pull/8533))
- Fix port conflict issue in some tests and re-enable those tests on Windows where those issues were especially problematic ([#8624](https://github.com/ZcashFoundation/zebra/pull/8624))
- Fix a bug with trailing characters in the OpenAPI spec method descriptions ([#8597](https://github.com/ZcashFoundation/zebra/pull/8597))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @conradoplg, @gustavovalverde, @oxarbitrage and @upbqdn
In this release we introduce Regtest functionality to Zebra and restored Windows support. Also adjusted our Zebra release interval from 2 weeks to 6 weeks approximately.
### Added
- Preparing for upstream `zcash_client_backend` API changes ([#8425](https://github.com/ZcashFoundation/zebra/pull/8425))
- Regtest support ([#8383](https://github.com/ZcashFoundation/zebra/pull/8383), [#8421](https://github.com/ZcashFoundation/zebra/pull/8421), [#8368](https://github.com/ZcashFoundation/zebra/pull/8368), [#8413](https://github.com/ZcashFoundation/zebra/pull/8413), [#8474](https://github.com/ZcashFoundation/zebra/pull/8474), [#8475](https://github.com/ZcashFoundation/zebra/pull/8475))
- Allow Zebra users to contribute to the P2P network even if behind NAT or firewall ([#8488](https://github.com/ZcashFoundation/zebra/pull/8488))
### Changed
- Adjust estimated release interval to once every 6 weeks and the end of support from 16 to 20 weeks ([#8429](https://github.com/ZcashFoundation/zebra/pull/8429))
### Fixed
- Bump zcash script v0.1.15 and restore Windows support ([#8393](https://github.com/ZcashFoundation/zebra/pull/8393))
- Avoid possibly returning data from different blocks in `z_get_treestate` RPC method ([#8460](https://github.com/ZcashFoundation/zebra/pull/8460))
- Zebra panics with all features and no elasticsearch server available ([#8409](https://github.com/ZcashFoundation/zebra/pull/8409))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
- Log database information such as storage usage on startup and shutdown ([#8336](https://github.com/ZcashFoundation/zebra/pull/8336), [#8389](https://github.com/ZcashFoundation/zebra/pull/8389))
- OpenAPI specification for Zebra's RPC methods ([#8342](https://github.com/ZcashFoundation/zebra/pull/8342))
- Add block times to output of getblock RPC method when called with `verbosity = 2` ([#8384](https://github.com/ZcashFoundation/zebra/pull/8384))
This release serves as a hotfix for version 1.5.1, addressing issues encountered after its initial release. For more information about version 1.5.1, refer to [this link](https://github.com/ZcashFoundation/zebra/releases/tag/v1.5.2).
Following the release on GitHub, we identified difficulties in publishing the `zebra-chain` crate to crates.io. Detailed information is available in [issue #8180](https://github.com/ZcashFoundation/zebra/issues/8180) and its references.
Unfortunately, to resolve this challenge, which involves an unpublished dependency, we had to temporarily remove the internal miner support introduced in version 1.5.1.
In our efforts to reinstate this feature, we've opened a tracking ticket to monitor the progress of the required code that must be merged into the `equihash` dependency. You can follow the developments in [issue #8183](https://github.com/ZcashFoundation/zebra/issues/8183), which will only be closed once the feature is successfully restored.
### Breaking Changes
- Temporally remove the internal miner functionality ([#8184](https://github.com/ZcashFoundation/zebra/pull/8184))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
- Adds a utility for reading scanning results, and finalizes the MVP features of the scanner.
- Adds an experimental `internal-miner` feature, which mines blocks within `zebrad`. This feature is only supported on testnet. Use a more efficient GPU or ASIC for mainnet mining.
- Contains many documentation improvements.
### Added
- Add an internal Zcash miner to Zebra ([#8136](https://github.com/ZcashFoundation/zebra/pull/8136), [#8150](https://github.com/ZcashFoundation/zebra/pull/8150))
- Blockchain scanner new features:
- Don't scan and log if we are below sapling height ([#8121](https://github.com/ZcashFoundation/zebra/pull/8121))
- Restart scanning where left ([#8080](https://github.com/ZcashFoundation/zebra/pull/8080))
- Add scanning result reader utility ([#8104](https://github.com/ZcashFoundation/zebra/pull/8104), [#8157](https://github.com/ZcashFoundation/zebra/pull/8157))
- Note default path to config in docs ([#8143](https://github.com/ZcashFoundation/zebra/pull/8143))
- Document how to add a column family ([#8149](https://github.com/ZcashFoundation/zebra/pull/8149))
### Changed
- Make sure scanner database is accessed using the correct types ([#8112](https://github.com/ZcashFoundation/zebra/pull/8112))
- Move history tree and value balance to typed column families ([#8115](https://github.com/ZcashFoundation/zebra/pull/8115))
- Refactor the user documentation for scanning ([#8124](https://github.com/ZcashFoundation/zebra/pull/8124))
- Refactor user \& dev documentation ([#8145](https://github.com/ZcashFoundation/zebra/pull/8145))
- Improve feature flag docs ([#8114](https://github.com/ZcashFoundation/zebra/pull/8114))
- Allow opening the database in a read-only mode ([#8079](https://github.com/ZcashFoundation/zebra/pull/8079))
- Send all zebrad logs to the journal under systemd ([#7965](https://github.com/ZcashFoundation/zebra/pull/7965))
### Fixed
- Point to a manually created list of Zebra crates in docs ([#8160](https://github.com/ZcashFoundation/zebra/pull/8160))
- Add shielded-scan.md to the index ([#8095](https://github.com/ZcashFoundation/zebra/pull/8095))
- Elasticsearch feature, make bulk size the same for testnet and mainnet ([#8127](https://github.com/ZcashFoundation/zebra/pull/8127))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @bishopcheckmate, @gustavovalverde, @oxarbitrage, @sandakersmann, @teor2345 and @upbqdn
- further improves how Zebra recovers from network interruptions and prevents potential network hangs.
- limits the ability of synthetic nodes to spread throughout the network through Zebra to address some of the Ziggurat red team report.
As of this release, Zebra requires Rust 1.73 to build.
Finally, we've added an experimental "shielded-scan" feature and the zebra-scan crate as steps
towards supporting shielded scanning in Zebra. This feature has known security issues.
It is for experimental use only. Ongoing development is tracked in issue [#7728](https://github.com/ZcashFoundation/zebra/issues/7728).
### Important Security Warning
Do not use regular or sensitive viewing keys with Zebra's experimental scanning feature. Do not use this
feature on a shared machine. We suggest generating new keys for experimental use.
### Security
- security(net): Stop sending peer addresses from version messages directly to the address book ([#7977](https://github.com/ZcashFoundation/zebra/pull/7977))
- security(net): Limit how many addresses are sent directly to the address book for a single peer address message ([#7952](https://github.com/ZcashFoundation/zebra/pull/7952))
- security(net): Rate-limit GetAddr responses to avoid sharing the entire address book over a short period ([#7955](https://github.com/ZcashFoundation/zebra/pull/7955))
### Added
- feat(config): Add config field for the viewing keys used by zebra-scan ([#7949](https://github.com/ZcashFoundation/zebra/pull/7949))
- feat(scan): Add on-disk database to store keys and scan results ([#7942](https://github.com/ZcashFoundation/zebra/pull/7942), [#8036](https://github.com/ZcashFoundation/zebra/pull/8036))
- feat(scan): Spawn zebra-scan task from zebrad with configured viewing keys ([#7989](https://github.com/ZcashFoundation/zebra/pull/7989))
- feat(scan): Create a scan_block function to use across scanning tasks ([#7994](https://github.com/ZcashFoundation/zebra/pull/7994))
- feat(scan): Scan blocks with Sapling keys and write the results to the database ([#8040](https://github.com/ZcashFoundation/zebra/pull/8040))
- poc(scan): Proof of concept for shielded scanning ([#7758](https://github.com/ZcashFoundation/zebra/pull/7758))
- add(docker): Add `ldb` RocksDB query tool to the Dockerfile ([#8074](https://github.com/ZcashFoundation/zebra/pull/8074))
### Changed
- change(state): Expose ZebraDb methods that can create different kinds of databases ([#8002](https://github.com/ZcashFoundation/zebra/pull/8002))
- change(state): Make the types for finalized blocks consistent ([#7923](https://github.com/ZcashFoundation/zebra/pull/7923))
- change(scan): Create a scanner storage database ([#8031](https://github.com/ZcashFoundation/zebra/pull/8031))
- change(scan): Store scanned TXIDs in "display order" ([#8057](https://github.com/ZcashFoundation/zebra/pull/8057))
- change(scan): Create a function that scans one block by height, and stores the results in the database ([#8045](https://github.com/ZcashFoundation/zebra/pull/8045))
- change(scan): Store one transaction ID per database row, to make queries easier ([#8062](https://github.com/ZcashFoundation/zebra/pull/8062))
- fix(db): Fix a sprout/history tree read panic in Zebra v1.4.0, which only happens before the 25.3.0 state upgrade completes ([#7972](https://github.com/ZcashFoundation/zebra/pull/7972))
This release has the following deprecation warnings:
#### Warning: Deprecation of DockerHub Image Tags in a future release
Zebra currently publishes 11 [DockerHub tags](https://hub.docker.com/r/zfnd/zebra/tags) for each new release.
We want to reduce the number of DockerHub tags we publish in a future minor Zebra release.
Based on usage and user feedback, we could stop publishing:
- The `1` tag, which updates each release until NU6
- The `1.x` tag, which updates each patch release until the next minor release
- The `1.x.y` tag, which is the same as `v1.x.y`
- The `sha-xxxxxxx` tag, which is the same as `v1.x.y` (for production releases)
We also want to standardise experimental image tags to `-experimental`, rather than `.experimental`.
So for release 1.5.0, we might only publish these tags:
-`latest`
-`latest-experimental` (a new tag)
-`v1.5.0`
-`v1.5.0-experimental`
Please let us know if you need any other tags by [opening a GitHub ticket](https://github.com/ZcashFoundation/zebra/issues/new?assignees=&labels=C-enhancement%2CS-needs-triage&projects=&template=feature_request.yml&title=feature%3A+).
We recommend using the `latest` tag to always get the most recent Zebra release.
#### Warning: Documentation Website URL Change
We have replaced the API documentation on the [doc.zebra.zfnd.org](https://doc.zebra.zfnd.org)
website with [docs.rs](https://docs.rs/releases/search?query=zebra). All links have been updated.
Zebra's API documentation can be found on:
- [`docs.rs`](https://docs.rs/releases/search?query=zebra), which renders documentation for the
public API of the latest crate releases;
- [`doc-internal.zebra.zfnd.org`](https://doc-internal.zebra.zfnd.org/), which renders
documentation for the internal API on the `main` branch.
[doc.zebra.zfnd.org](https://doc.zebra.zfnd.org) stopped being updated a few days before this release,
and it will soon be shut down.
### Significant Changes
This release contains the following significant changes:
Zebra's mining RPCs are now available in release builds (#7740). Any Zebra instance can be used
by a solo miner or mining pool. This stabilises 12 RPCs, including `getblocktemplate`, `submitblock`,
`getmininginfo`, `getnetworksolps`, `[z_]validateaddress` and `getblocksubsidy`. For more information,
read our [mining blog post](https://zfnd.org/experimental-mining-support-in-zebra/).
Please [let us know](https://github.com/ZcashFoundation/zebra/issues/new?assignees=&labels=C-enhancement%2CS-needs-triage&projects=&template=feature_request.yml&title=feature%3A+)
if your mining pool needs extra RPC methods or fields.
Previously, parameters were stored by default in these locations:
*`~/.zcash-params` (on Linux); or
*`~/Library/Application Support/ZcashParams` (on Mac); or
*`C:\Users\Username\AppData\Roaming\ZcashParams` (on Windows)
If you have upgraded `zebrad` to 1.4.0 or later, and `zcashd` to 5.7.0 or later, you can delete the
parameter files in these directories to save approximately 700 MB disk space.
[`zcashd` have deprecated their `fetch-params.sh` script](https://github.com/zcash/zcash/blob/master/doc/release-notes/release-notes-5.7.0.md#deprecation-of-fetch-paramssh),
so it can't be used to retry failed downloads in `zebrad` 1.3.0 and earlier.
We recommend upgrading to the latest Zebra release to avoid download issues in new installs.
- Reconnect with peers after brief network interruption ([#7853](https://github.com/ZcashFoundation/zebra/pull/7853))
- Add outer timeouts for critical network operations to avoid hangs ([#7869](https://github.com/ZcashFoundation/zebra/pull/7869))
- Set iterator read bounds where possible in DiskDb, to avoid a known RocksDB denial of service issue ([#7731](https://github.com/ZcashFoundation/zebra/pull/7731), [#7732](https://github.com/ZcashFoundation/zebra/pull/7732))
- Fix concurrency issues in tree key formats, and CPU usage in genesis tree roots ([#7392](https://github.com/ZcashFoundation/zebra/pull/7392))
### Removed
- Remove the `zebrad download` command, because it no longer does anything ([#7819](https://github.com/ZcashFoundation/zebra/pull/7819))
### Added
- Enable mining RPCs by default in production builds ([#7740](https://github.com/ZcashFoundation/zebra/pull/7740))
- Re-enable macOS builds and tests in CI ([#7834](https://github.com/ZcashFoundation/zebra/pull/7834))
- Make macOS x86_64 a tier 2 supported platform in the docs ([#7843](https://github.com/ZcashFoundation/zebra/pull/7843))
- Add macOS M1 as a tier 3 supported platform ([#7851](https://github.com/ZcashFoundation/zebra/pull/7851))
### Changed
- Build Sprout and Sapling parameters into the zebrad binary, so a download server isn't needed ([#7800](https://github.com/ZcashFoundation/zebra/pull/7800), [#7844](https://github.com/ZcashFoundation/zebra/pull/7844))
- Bump ECC dependencies for `zcashd` 5.7.0 ([#7784](https://github.com/ZcashFoundation/zebra/pull/7784))
- Refactor the installation instructions for the `s-nomp` mining pool software ([#7835](https://github.com/ZcashFoundation/zebra/pull/7835))
### Fixed
- Make the `latest` Docker tag point to the production build, rather than the build with experimental features ([#7817](https://github.com/ZcashFoundation/zebra/pull/7817))
- Fix an incorrect consensus-critical ZIP 212 comment ([#7774](https://github.com/ZcashFoundation/zebra/pull/7774))
- Fix broken links to `zebra_network` and `zebra_state``Config` structs on doc-internal.zebra.zfnd.org ([#7838](https://github.com/ZcashFoundation/zebra/pull/7838))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @mpguerra, @oxarbitrage, @rex4539, @teor2345, @upbqdn, and @vuittont60.
Zebra has progress bars! When progress bars are enabled, you can see Zebra's blocks,
transactions, and peer connections in your terminal. We're asking Zebra users to test this
feature, and give us [feedback on the forums](https://forum.zcashcommunity.com/t/zebra-progress-bars/44485).
To show progress bars while running Zebra, add these lines to your `zebrad.toml`:
```toml
[tracing]
progress_bar = "summary"
```
For more details, including a known issue with time estimates,
read our [progress bars blog post](https://zfnd.org/experimental-zebra-progress-bars/).
### Security
- Fix database concurrency bugs that could have led to panics or incorrect history tree data (#7590, #7663)
### Added
- Zebra's progress bars can now be enabled using a `zebrad.toml` config (#7615)
- Add missing elasticsearch flag feature to lib docs (#7568)
- Add missing Docker variables and examples (#7552)
- Check database format is valid on startup and shutdown (#7566, #7606). We expect to catch almost all database validity errors in CI (#7602, #7627), so users are unlikely to see them on startup or shutdown.
#### Spend before Sync Support
- Add state requests and support code for the `z_getsubtreesbyindex` RPC (#7408, #7734)
- Implement the `z_getsubtreesbyindex` RPC (#7436)
- Test the `z_getsubtreesbyindex` RPC (#7515, #7521, #7566, #7514, #7628)
- Format subtree roots in little-endian order (#7466)
- Add note subtree indexes for new and existing blocks (#7437)
- Upgrade subtrees from the tip backwards, for compatibility with wallet syncing (#7531)
- Handle a subtree comparison edge case correctly (#7587)
### Changed
- Return errors instead of panicking in methods for Heights (#7591)
- Update tests for compatibility with the ECC's `lightwalletd` fork (#7349)
### Fixed
- Refactor docs for feature flags (#7567)
- Match zcashd's getblockchaininfo capitalisation for NU5 (#7454)
- Fix bugs and performance of `getnetworksolps`&`getnetworkhashps` RPCs (#7647)
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @oxarbitrage, @rex4539, @teor2345 and @upbqdn.
- Starts our work implementing "spend before sync" algorithm for lightwalletd.
- Contains an automatic database upgrade that reduces the size of Zebra's current cached state from approximately 276GB to 244GB. It does so by automatically pruning unneeded note commitment trees from an existing cache. New Zebra instances will also build their cache without these trees.
`zebrad` 1.2.0 cached states are incompatible with previous `zebrad` versions:
-`zebrad` 1.2.0 upgrades the cached state format. The new format is incompatible with previous `zebrad` versions. After upgrading to this Zebra version, don't downgrade to an earlier version.
- When earlier versions try to use states upgraded by `zebrad` 1.2.0:
-`zebrad` versions 1.0.0 and 1.0.1 will respond to some `z_gettreestate` RPC requests with incorrect empty `final_state` fields
- pre-release `zebrad` versions can panic when verifying shielded transactions, updating the state, or responding to RPC requests
- Deduplicate note commitment trees stored in the finalized state ([#7312](https://github.com/ZcashFoundation/zebra/pull/7312), [#7379](https://github.com/ZcashFoundation/zebra/pull/7379))
- Insert only the first tree in each series of identical trees into finalized state ([#7266](https://github.com/ZcashFoundation/zebra/pull/7266))
- Our testing framework now uses the ECC lightwalletd fork ([#7307](https://github.com/ZcashFoundation/zebra/pull/7307)). This was needed to start the work of implementing fast spendability. The ECC repo is now the supported implementation in Zebra, documentation was changed to reflect this. ([#7427](https://github.com/ZcashFoundation/zebra/pull/7427))
- Documentation for mining with Docker ([#7179](https://github.com/ZcashFoundation/zebra/pull/7179))
- Note tree sizes field to `getblock` RPC method ([#7278](https://github.com/ZcashFoundation/zebra/pull/7278))
- Note commitment subtree types to zebra-chain ([#7371](https://github.com/ZcashFoundation/zebra/pull/7371))
- Note subtree index handling to zebra-state, but we're not writing subtrees to the finalized state yet ([#7334](https://github.com/ZcashFoundation/zebra/pull/7334))
### Fixed
- Log a warning instead of panicking for unused mining configs ([#7290](https://github.com/ZcashFoundation/zebra/pull/7290))
- Avoid expensive note commitment tree root recalculations in eq() methods ([#7386](https://github.com/ZcashFoundation/zebra/pull/7386))
- Use the correct state version for databases without a state version file ([#7385](https://github.com/ZcashFoundation/zebra/pull/7385))
- Avoid temporary failures verifying the first non-finalized block or attempting to fork the chain before the final checkpoint ([#6810](https://github.com/ZcashFoundation/zebra/pull/6810))
- If a database format change is cancelled, also cancel the format check, and don't mark the database as upgraded ([#7442](https://github.com/ZcashFoundation/zebra/pull/7442))
This release adds new mempool metrics, fixes panics when cancelling tasks on shutdown, detects subcommand name typos on the command-line, and improves the usability of Zebra's Docker images (particularly for mining).
- Zebra now detects subcommand name typos on the command-line. If you want to give Zebra a list of tracing filters, use `zebrad start --filters debug,...` ([#7056](https://github.com/ZcashFoundation/zebra/pull/7056))
- Avoid initiating outbound handshakes with IPs for which Zebra already has an active peer ([#7029](https://github.com/ZcashFoundation/zebra/pull/7029))
- Rate-limit inbound connections per IP ([#7041](https://github.com/ZcashFoundation/zebra/pull/7041))
### Added
- Metrics tracking mempool actions and size bucketed by weight ([#7019](https://github.com/ZcashFoundation/zebra/pull/7019)) by @str4d
- Legacy state format compatibility layer and version bumps for ECC dependencies to match `zcashd` 5.6.0 ([#7053](https://github.com/ZcashFoundation/zebra/pull/7053))
- Framework for upcoming in-place database format upgrades ([#7031](https://github.com/ZcashFoundation/zebra/pull/7031))
### Changed
- Deduplicate note commitment trees in non-finalized state ([#7218](https://github.com/ZcashFoundation/zebra/pull/7218), [#7239](https://github.com/ZcashFoundation/zebra/pull/7239))
### Fixed
- Enable miners running Zebra with Docker to set their address for mining rewards ([#7178](https://github.com/ZcashFoundation/zebra/pull/7178))
- Use default RPC port when running Zebra with Docker ([#7177](https://github.com/ZcashFoundation/zebra/pull/7177), [#7162](https://github.com/ZcashFoundation/zebra/pull/7162))
- Stop panicking on async task cancellation on shutdown in network and state futures ([#7219](https://github.com/ZcashFoundation/zebra/pull/7219))
- Remove redundant startup logs, fix progress bar number, order, and wording ([#7087](https://github.com/ZcashFoundation/zebra/pull/7087))
- Organize Docker `ENV` and `ARG` values based on their usage ([#7200](https://github.com/ZcashFoundation/zebra/pull/7200))
- Avoid blocking threads by awaiting proof verification results from rayon in async context ([#6887](https://github.com/ZcashFoundation/zebra/pull/6887))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @mpguerra, @oxarbitrage, @str4d, @teor2345 and @upbqdn
Zebra's first patch release fixes multiple peer connection security issues and panics. It also significantly reduces Zebra's CPU usage. We recommend that all users upgrade to Zebra 1.0.1 or later.
As of this release, Zebra requires Rust 1.70 to build. macOS builds are no longer officially supported by the Zebra team.
If you're running `zebrad` in a terminal, you'll see a new Zebra welcome message.
Please report bugs to [the Zebra GitHub repository](https://github.com/ZcashFoundation/zebra/issues/new?assignees=&labels=C-bug%2C+S-needs-triage&projects=&template=bug_report.yml&title=)
### Breaking Changes
This release has the following breaking changes:
- Zebra limits each IP address to 1 peer connection, to prevent denial of service attacks. This can be changed using the `network.max_connections_per_ip` config. ([#6980](https://github.com/ZcashFoundation/zebra/pull/6980), [#6993](https://github.com/ZcashFoundation/zebra/pull/6993), [#7013](https://github.com/ZcashFoundation/zebra/pull/7013)).
Thank you to @dimxy from komodo for reporting this bug, and the Ziggurat team for demonstrating
its impact on testnet.
- Zebra uses new APIs in Rust 1.70 to prevent concurrency bugs that could cause hangs or panics
These platforms are no longer supported by the Zebra team:
- macOS has been moved from tier 2 to [tier 3 support](https://github.com/ZcashFoundation/zebra/blob/main/book/src/user/supported-platforms.md#tier-3) ([#6965](https://github.com/ZcashFoundation/zebra/pull/6965)). We disabled our regular macOS builds because Rust 1.70 [causes crashes during shutdown on macOS x86_64 (#6812)](https://github.com/ZcashFoundation/zebra/issues/6812). Zebra's state uses database transactions, so it should not be corrupted by the crash.
### Security
- Use Arc::into\_inner() to avoid potential hangs or panics ([#7032](https://github.com/ZcashFoundation/zebra/pull/7032))
- Replace openssl with rustls in tests and experimental features ([#7047](https://github.com/ZcashFoundation/zebra/pull/7047))
#### Network Security
- Fix long delays in accepting inbound handshakes, and delays in async operations throughout Zebra. ([#7103](https://github.com/ZcashFoundation/zebra/pull/7103)). Thank you to the Ziggurat Team for reporting this bug.
- Limit each IP address to 1 peer connection, to prevent denial of service attacks. ([#6980](https://github.com/ZcashFoundation/zebra/pull/6980), [#6993](https://github.com/ZcashFoundation/zebra/pull/6993))
- Close new peer connections from the same IP and port, rather than replacing the older connection ([#6980](https://github.com/ZcashFoundation/zebra/pull/6980))
- Reduce inbound service overloads and add a timeout ([#6950](https://github.com/ZcashFoundation/zebra/pull/6950))
- Stop panicking when handling inbound connection handshakes ([#6984](https://github.com/ZcashFoundation/zebra/pull/6984))
- Make the maximum number of connections per IP configurable ([#7013](https://github.com/ZcashFoundation/zebra/pull/7013))
- Make it easier to modify Zebra's config inside the Docker image ([#7045](https://github.com/ZcashFoundation/zebra/pull/7045))
- Print a Zebra logo and welcome text if stderr is terminal ([#6945](https://github.com/ZcashFoundation/zebra/pull/6945), [#7075](https://github.com/ZcashFoundation/zebra/pull/7075), [#7095](https://github.com/ZcashFoundation/zebra/pull/7095), [#7102](https://github.com/ZcashFoundation/zebra/pull/7102))
### Changed
- Move macOS to tier 3 support ([#6965](https://github.com/ZcashFoundation/zebra/pull/6965))
- Install from crates.io in the README, rather than a git release tag ([#6977](https://github.com/ZcashFoundation/zebra/pull/6977))
- Add extra timeout logging to peer TCP connections ([#6969](https://github.com/ZcashFoundation/zebra/pull/6969))
- Stop Zebra using 100% CPU even when idle ([#7103](https://github.com/ZcashFoundation/zebra/pull/7103)), thank you to james_katz for reporting this bug
- Avoid potential hangs in the `tokio` async runtime ([#7094](https://github.com/ZcashFoundation/zebra/pull/7094))
- Replace or add RPC content type header to support `zcashd` RPC examples ([#6885](https://github.com/ZcashFoundation/zebra/pull/6885))
- Make `zebra-network` licensing clearer ([#6995](https://github.com/ZcashFoundation/zebra/pull/6995))
#### Configuration
- Ignore error from loading config if running the 'generate' or 'download' commands ([#7014](https://github.com/ZcashFoundation/zebra/pull/7014))
- Apply force\_color to panic logs ([#6997](https://github.com/ZcashFoundation/zebra/pull/6997))
#### Logging & Error Handling
- Log a zebra-network task cancel on shutdown, rather than panicking ([#7078](https://github.com/ZcashFoundation/zebra/pull/7078))
- Fix incorrect function spans in some logs ([#6923](https://github.com/ZcashFoundation/zebra/pull/6923), [#6995](https://github.com/ZcashFoundation/zebra/pull/6995))
- Replace a state validation chain length assertion with a NotReadyToBeCommitted error ([#7072](https://github.com/ZcashFoundation/zebra/pull/7072))
#### Experimental Feature Fixes
- Add an elasticsearch feature to block serialize to fix experimental build failures ([#6709](https://github.com/ZcashFoundation/zebra/pull/6709))
- Prevent progress bar from panicking by disabling limits that are never reached ([#6940](https://github.com/ZcashFoundation/zebra/pull/6940))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @conradoplg, @dconnolly, @dimxy from komodo, james_katz, @oxarbitrage, @teor2345, @upbqdn, and the Ziggurat team.
This release also fixes a panic at startup when parsing the app version, [publishes `zebrad` to crates.io](https://crates.io/crates/zebrad), and [publishes to Docker Hub under the `latest` tag](https://hub.docker.com/r/zfnd/zebra/tags).
Please report bugs to [the Zebra GitHub repository](https://github.com/ZcashFoundation/zebra/issues/new?assignees=&labels=C-bug%2C+S-needs-triage&projects=&template=bug_report.yml&title=)
### Security
- Avoid potential concurrency bugs in outbound handshakes ([#6869](https://github.com/ZcashFoundation/zebra/pull/6869))
This release continues to address audit findings. It fixes multiple network protocol and RPC bugs,
and reduces sensitive information logging.
This is the last release candidate before the 1.0.0 stable release. Please report bugs to [the Zebra GitHub repository](https://github.com/ZcashFoundation/zebra/issues/new?assignees=&labels=C-bug%2C+S-needs-triage&projects=&template=bug_report.yml&title=)
- Stop logging peer IP addresses, to protect user privacy ([#6662](https://github.com/ZcashFoundation/zebra/pull/6662))
- Stop logging potentially sensitive user information from unmined transactions ([#6616](https://github.com/ZcashFoundation/zebra/pull/6616))
- Rate-limit MetaAddrChange::Responded from peers ([#6738](https://github.com/ZcashFoundation/zebra/pull/6738))
- Ignore out of order Address Book changes, unless they are concurrent ([#6717](https://github.com/ZcashFoundation/zebra/pull/6717))
- Limit blocks and transactions sent in response to a single request ([#6679](https://github.com/ZcashFoundation/zebra/pull/6679))
- Rate-limit and size-limit peer transaction ID messages ([#6625](https://github.com/ZcashFoundation/zebra/pull/6625))
- Stop panicking on state RPC or block requests with very large heights ([#6699](https://github.com/ZcashFoundation/zebra/pull/6699))
- Try harder to drop connections when they shut down, Credit: Ziggurat Team ([#6832](https://github.com/ZcashFoundation/zebra/pull/6832))
- Randomly drop connections when inbound service is overloaded ([#6790](https://github.com/ZcashFoundation/zebra/pull/6790))
### Added
- Report compiler version and Zebra features when starting Zebra ([#6606](https://github.com/ZcashFoundation/zebra/pull/6606))
- Update Zebra book summary to include supported platforms, platform tier policy, and versioning ([#6683](https://github.com/ZcashFoundation/zebra/pull/6683))
- Improve zebrad's help output, credit to @Rqnsom ([#6801](https://github.com/ZcashFoundation/zebra/pull/6801))
- Cache a list of useful peers on disk ([#6739](https://github.com/ZcashFoundation/zebra/pull/6739))
- Make the first stable release forward-compatible with planned state changes ([#6813](https://github.com/ZcashFoundation/zebra/pull/6813))
Starting in this release, Zebra has implemented an "end of support" halt. Just like `zcashd`, the `zebrad` binary will stop running 16 weeks after the last release date.
Also, this release adds the ZIP-317 rules to mempool transactions which should help with the Zcash network spam issue.
### Security
- Avoid inbound service overloads and fix failing tests ([#6537](https://github.com/ZcashFoundation/zebra/pull/6537))
- Avoid a rare panic when a connection is dropped ([#6566](https://github.com/ZcashFoundation/zebra/pull/6566))
- Avoid some self-connection nonce removal attacks ([#6410](https://github.com/ZcashFoundation/zebra/pull/6410))
- Reject nodes using ZClassic ports, and warn if configured with those ports ([#6567](https://github.com/ZcashFoundation/zebra/pull/6567))
### Added
- Add ZIP-317 rules to mempool ([#6556](https://github.com/ZcashFoundation/zebra/pull/6556))
- Add user agent argument to zebra-network crate ([#6601](https://github.com/ZcashFoundation/zebra/pull/6601))
- Refuse to run zebrad when release is too old ([#6351](https://github.com/ZcashFoundation/zebra/pull/6351))
### Fixed
- Handle randomness generation and invalid random values as errors in cryptographic code ([#6385](https://github.com/ZcashFoundation/zebra/pull/6385))
- When configured for testnet, automatically use the correct testnet listener port ([#6575](https://github.com/ZcashFoundation/zebra/pull/6575))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @oxarbitrage, @teor2345 and @upbqdn
This release features a security fix for unbounded memory use in zebra-network, introduces the "progress-bar" feature, and continues to address audit findings.
### Security
- Limit the number of leftover nonces in the self-connection nonce set ([#6534](https://github.com/ZcashFoundation/zebra/pull/6534))
- Allow each initial peer to send one inbound request before disconnecting any peers ([#6520](https://github.com/ZcashFoundation/zebra/pull/6520))
- Limit the number of non-finalized chains tracked by Zebra ([#6447](https://github.com/ZcashFoundation/zebra/pull/6447))
- Update dependencies that only appear in the lock file ([#6217](https://github.com/ZcashFoundation/zebra/pull/6217))
### Added
- Add confirmations to getrawtransaction method response ([#6287](https://github.com/ZcashFoundation/zebra/pull/6287))
- Add a config for writing logs to a file ([#6449](https://github.com/ZcashFoundation/zebra/pull/6449))
- Add an experimental terminal-based progress bar feature to Zebra, which is off by default ([#6235](https://github.com/ZcashFoundation/zebra/pull/6235))
- Create DockerHub image with mining enabled after each Zebra release ([#6228](https://github.com/ZcashFoundation/zebra/pull/6228))
### Changed
- Increase ZIP-401 mempool cost thresholds for Orchard transactions ([#6521](https://github.com/ZcashFoundation/zebra/pull/6521))
- Suggest making sure the RPC endpoint is enabled for checkpointing ([#6375](https://github.com/ZcashFoundation/zebra/pull/6375))
- Refactor the handling of height differences ([#6330](https://github.com/ZcashFoundation/zebra/pull/6330))
- Upgrade shared dependencies to match `zcashd` 5.5.0 ([#6536](https://github.com/ZcashFoundation/zebra/pull/6536))
- Lookup unspent UTXOs in non-finalized state before checking disk ([#6513](https://github.com/ZcashFoundation/zebra/pull/6513))
- Stop re-downloading blocks that are in non-finalized side chains ([#6335](https://github.com/ZcashFoundation/zebra/pull/6335))
### Fixed
- Validate header versions when serializing blocks ([#6475](https://github.com/ZcashFoundation/zebra/pull/6475))
- Stop ignoring new transactions after the mempool is newly activated ([#6448](https://github.com/ZcashFoundation/zebra/pull/6448))
- Fix off-by-one error in DNS seed peer retries, and clarify logs ([#6460](https://github.com/ZcashFoundation/zebra/pull/6460))
- Check that mempool transactions are valid for the state's chain info in getblocktemplate ([#6416](https://github.com/ZcashFoundation/zebra/pull/6416))
- Remove transactions with immature transparent coinbase spends from the mempool and block templates ([#6510](https://github.com/ZcashFoundation/zebra/pull/6510))
- Disable issue URLs for a known shutdown panic in abscissa ([#6486](https://github.com/ZcashFoundation/zebra/pull/6486))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @dconnolly, @gustavovalverde, @oxarbitrage, @teor2345 and @upbqdn
In this release, we fixed several minor security issues, most notably [hardening Zebra in response to the vulnerabilities recently disclosed by Halborn](https://zfnd.org/statement-on-recent-security-disclosures-by-halborn/).
-`orchard` 0.3.0 can't verify `halo2` proofs when compiled with Rust 1.69 or later (currently beta and nightly Rust). Compile Zebra with stable Rust to avoid this bug. ([halo2/#737](https://github.com/zcash/halo2/issues/737)).
Zebra tracking issue for this problem is [#6232](https://github.com/ZcashFoundation/zebra/issues/6232).
### Security
- Harden Zebra's network protocol implementation in response to the Halborn disclosures ([#6297](https://github.com/ZcashFoundation/zebra/pull/6297))
- Bump incrementalmerkletree from 0.3.0 to 0.3.1, resolving a consensus bug on 32-bit platforms ([#6258](https://github.com/ZcashFoundation/zebra/pull/6258))
- Remove unused dependencies, and check for them in CI ([#6216](https://github.com/ZcashFoundation/zebra/pull/6216))
- Validate address length before reading ([#6320](https://github.com/ZcashFoundation/zebra/pull/6320), [#6368](https://github.com/ZcashFoundation/zebra/pull/6368))
### Added
- Add instructions for mining with s-nomp to Zebra book ([#6220](https://github.com/ZcashFoundation/zebra/pull/6220))
- Export block data to elasticsearch database ([#6274](https://github.com/ZcashFoundation/zebra/pull/6274))
- Add elasticsearch section to Zebra book ([#6295](https://github.com/ZcashFoundation/zebra/pull/6295))
In this release we fixed bugs and inconsistencies between zcashd and zebrad in the output of the `getblocktemplate` RPC method. In addition, we added block proposal mode to the `getblocktemplate` RPC, while we continue the effort of adding and testing mining pool RPC methods.
### Security
- Verify the lock times of mempool transactions. Previously, Zebra was ignoring mempool transaction lock times, but checking them in mined blocks. Credit to DeckerSU for reporting this issue. ([#6027](https://github.com/ZcashFoundation/zebra/pull/6027))
- Bump bumpalo from 3.8.0 to 3.12.0, removing undefined behaviour on `wasm` targets. These targets are not supported Zebra platforms. ([#6015](https://github.com/ZcashFoundation/zebra/pull/6015))
- Bump libgit2-sys from 0.14.0+1.5.0 to 0.14.2+1.5.1, to ensure that SSH server keys are checked. Zebra only uses `libgit2` during builds, and we don't make SSH connections. ([#6014](https://github.com/ZcashFoundation/zebra/pull/6014))
- Bump tokio from 1.24.1 to 1.24.2, to fix unsoundness. The unsound methods are not directly used by Zebra. ([#5995](https://github.com/ZcashFoundation/zebra/pull/5995))
- Make `zebra-checkpoints` work for zebrad backend ([#5894](https://github.com/ZcashFoundation/zebra/pull/5894), [#5961](https://github.com/ZcashFoundation/zebra/pull/5961))
- Return detailed errors to the RPC client when a block proposal fails ([#5993](https://github.com/ZcashFoundation/zebra/pull/5993))
- Avoid selecting duplicate transactions in block templates ([#6006](https://github.com/ZcashFoundation/zebra/pull/6006))
- Calculate getblocktemplate RPC testnet min and max times correctly ([#5925](https://github.com/ZcashFoundation/zebra/pull/5925))
- Fix Merkle root transaction order in getblocktemplate RPC method ([#5953](https://github.com/ZcashFoundation/zebra/pull/5953))
### Changed
- Strings in zebra configuration file now use double quotes, caused by upgrading the `toml` crate. Old configs will still work [#6029](https://github.com/ZcashFoundation/zebra/pull/6029)
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @conradoplg, @gustavovalverde, @mpguerra, @oxarbitrage and @teor2345
This release continues our work on mining pool RPCs, and brings Zebra up to date with the latest [ZIP-317](https://zips.z.cash/zip-0317) changes. It also fixes a minor network protocol compatibility bug.
As part of this release, we upgraded `tokio` to fix potential hangs and performance issues. We encourage all users to upgrade to the latest Zebra version to benefit from these fixes.
### Breaking Changes
- Zebra now requires at least Rust 1.65, because we have started using new language features.
Any Zebra release can increase the required Rust version: only the latest stable Rust version is supported.
### Security
- Upgrade tokio from 1.22.0 to 1.23.0 to fix potential hangs and performance issues ([#5802](https://github.com/ZcashFoundation/zebra/pull/5802))
- Refactor block subsidy to handle Height::MAX without panicking ([#5787](https://github.com/ZcashFoundation/zebra/pull/5787))
- Update ZIP-317 transaction selection algorithm in the `getblocktemplate` RPC ([#5776](https://github.com/ZcashFoundation/zebra/pull/5776))
### Added
- Add the `getmininginfo`, `getnetworksolps` and `getnetworkhashps` RPC methods ([#5808](https://github.com/ZcashFoundation/zebra/pull/5808))
- Add long polling support to the `getblocktemplate` RPC ([#5772](https://github.com/ZcashFoundation/zebra/pull/5772), [#5796](https://github.com/ZcashFoundation/zebra/pull/5796), [#5837](https://github.com/ZcashFoundation/zebra/pull/5837), [#5843](https://github.com/ZcashFoundation/zebra/pull/5843), [#5862](https://github.com/ZcashFoundation/zebra/pull/5862))
- Populate `blockcommitmenthash` and `defaultroot` fields in the getblocktemplate RPC ([#5751](https://github.com/ZcashFoundation/zebra/pull/5751))
- Support transparent p2pkh miner addresses in the `getblocktemplate` RPC ([#5827](https://github.com/ZcashFoundation/zebra/pull/5827))
### Changed
- Automatically re-verify mempool transactions after a chain fork, rather than re-downloading them all ([#5841](https://github.com/ZcashFoundation/zebra/pull/5841))
- Try to match `zcashd`'s `getblocktemplate` exactly ([#5867](https://github.com/ZcashFoundation/zebra/pull/5867))
- Accept a hash or a height as the first parameter of the `getblock` RPC ([#5861](https://github.com/ZcashFoundation/zebra/pull/5861))
- Wait for 3 minutes to check Zebra is synced to the tip, rather than 2 ([#5840](https://github.com/ZcashFoundation/zebra/pull/5840))
- Update mainnet and testnet checkpoints ([#5928](https://github.com/ZcashFoundation/zebra/pull/5928))
### Fixed
- Allow peers to omit the `relay` flag in `version` messages ([#5835](https://github.com/ZcashFoundation/zebra/pull/5835))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @dconnolly, @dependabot[bot], @oxarbitrage and @teor2345
Zebra's latest release continues work on mining pool RPCs, fixes a rare RPC crash that could lead to memory corruption, and uses the ZIP-317 conventional fee for mempool size limits.
- Add a `mining` section with miner address to config ([#5491](https://github.com/ZcashFoundation/zebra/pull/5508))
### Changed
- Select getblocktemplate RPC transactions according to ZIP-317 ([#5724](https://github.com/ZcashFoundation/zebra/pull/5724))
- Add transaction fields to the `getblocktemplate` RPC ([#5496](https://github.com/ZcashFoundation/zebra/pull/5496) and [#5508](https://github.com/ZcashFoundation/zebra/pull/5508))
- Populate some getblocktemplate RPC block header fields using the state best chain tip ([#5659](https://github.com/ZcashFoundation/zebra/pull/5659))
- Return an error from getblocktemplate method if Zebra is not synced to network tip ([#5623](https://github.com/ZcashFoundation/zebra/pull/5623))
- Implement coinbase conversion to RPC `TransactionTemplate` type ([#5554](https://github.com/ZcashFoundation/zebra/pull/5554))
- Check block and transaction Sprout anchors in parallel ([#5742](https://github.com/ZcashFoundation/zebra/pull/5742))
- Contextually validates mempool transactions in best chain ([#5716](https://github.com/ZcashFoundation/zebra/pull/5716) and [#5616](https://github.com/ZcashFoundation/zebra/pull/5616))
- Generate coinbase transactions in the getblocktemplate RPC ([#5580](https://github.com/ZcashFoundation/zebra/pull/5580))
- Log loaded config path when Zebra starts up ([#5733](https://github.com/ZcashFoundation/zebra/pull/5733))
- Update mainnet and testnet checkpoints on 2022-12-01 ([#5754](https://github.com/ZcashFoundation/zebra/pull/5754))
- Bump zcash\_proofs from 0.8.0 to 0.9.0 and zcash\_primitives from 0.8.1 to 0.9.0 ([#5631](https://github.com/ZcashFoundation/zebra/pull/5631))
### Fixed
- Check network and P2SH addresses for mining config and funding streams([#5620](https://github.com/ZcashFoundation/zebra/pull/5620))
- Return an error instead of panicking in the batch verifier on shutdown ([#5530](https://github.com/ZcashFoundation/zebra/pull/5530))
- Use a more reliable release template branch name and docker command ([#5519](https://github.com/ZcashFoundation/zebra/pull/5519))
- Make the syncer ignore some new block verification errors ([#5537](https://github.com/ZcashFoundation/zebra/pull/5537))
- Pause new downloads when Zebra reaches the lookahead limit ([#5561](https://github.com/ZcashFoundation/zebra/pull/5561))
- Shut down the RPC server properly when Zebra shuts down ([#5591](https://github.com/ZcashFoundation/zebra/pull/5591))
- Print usage info for --help flag ([#5634](https://github.com/ZcashFoundation/zebra/pull/5634))
This is the second Zebra release candidate. Zebra's consensus rules, node sync, and `lightwalletd` RPCs are ready for user testing and experimental use. Zebra has not been audited yet.
This release starts work on mining pool RPCs, including some mempool fixes. It also restores support for Rust 1.64.
- Remove unused buggy cryptographic code from zebra-chain ([#5464](https://github.com/ZcashFoundation/zebra/pull/5464), [#5476](https://github.com/ZcashFoundation/zebra/pull/5476)). This code was never used in production, and it had known bugs. Anyone using it should migrate to `librustzcash` instead.
- Document how we tag and release Zebra ([#5392](https://github.com/ZcashFoundation/zebra/pull/5392))
- Document how to use Zebra with Docker ([#5504](https://github.com/ZcashFoundation/zebra/pull/5504))
### Changed
- Update mainnet and testnet checkpoints ([#5512](https://github.com/ZcashFoundation/zebra/pull/5512))
### Fixed
- Reject mempool transactions with spent outpoints or nullifiers ([#5434](https://github.com/ZcashFoundation/zebra/pull/5434))
- Allow extra lookahead blocks in the verifier, state, and block commit task queues. This reduces the number of downloaded blocks that are dropped due to the lookahead limit. ([#5465](https://github.com/ZcashFoundation/zebra/pull/5465))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @oxarbitrage, @teor2345 and @upbqdn
This is the first Zebra release candidate. Zebra's consensus rules, node sync, and `lightwalletd` RPCs are ready for user testing and experimental use. Zebra has not been audited yet.
This release also makes significant performance improvements to RPCs, and temporarily removes support for Rust 1.64.
### Breaking Changes
This release has the following breaking changes:
- Rust 1.64 is unsupported due to a performance regression when downloading the Zcash parameters.
Zebra currently builds with Rust 1.63 ([#5251](https://github.com/ZcashFoundation/zebra/pull/5251)).
- Use correct TOML syntax in Docker zebrad.toml ([#5320](https://github.com/ZcashFoundation/zebra/pull/5320))
### Major RPC Performance Improvements
This release improves RPC performance:
- Initial `lightwalletd` sync is about twice as fast ([#5307](https://github.com/ZcashFoundation/zebra/pull/5307))
- RPCs can run while a block is being committed to the state, previously they could be delayed by 5-15 seconds ([#5134](https://github.com/ZcashFoundation/zebra/pull/5134), [#5257](https://github.com/ZcashFoundation/zebra/pull/5257))
### Security
- Make default command work in docker images, disable optional listener ports ([#5313](https://github.com/ZcashFoundation/zebra/pull/5313))
- Update yanked versions of cpufeatures (orchard/aes), ed25519 (tor), and quick-xml (flamegraph) ([#5308](https://github.com/ZcashFoundation/zebra/pull/5308))
- Bump rand\_core from 0.6.3 to 0.6.4, fixes unsoundness bug ([#5175](https://github.com/ZcashFoundation/zebra/pull/5175))
### Changed
- Log git metadata and platform info when zebrad starts up ([#5200](https://github.com/ZcashFoundation/zebra/pull/5200))
- Update mainnet and testnet checkpoints ([#5360](https://github.com/ZcashFoundation/zebra/pull/5360))
- Update README for the release candidate ([#5314](https://github.com/ZcashFoundation/zebra/pull/5314))
- change(docs): Add links to CI/CD docs in the zebra book's sidebar ([#5355](https://github.com/ZcashFoundation/zebra/pull/5355))
### Fixed
- Use correct TOML syntax in Docker zebrad.toml ([#5320](https://github.com/ZcashFoundation/zebra/pull/5320))
- Look back up to 10,000 blocks on testnet for a legacy chain ([#5133](https://github.com/ZcashFoundation/zebra/pull/5133))
#### Performance
- Build zebrad with Rust 1.63 to avoid Zcash parameter download hangs ([#5251](https://github.com/ZcashFoundation/zebra/pull/5251))
- Write blocks to the state in a separate thread, to avoid network and RPC hangs ([#5134](https://github.com/ZcashFoundation/zebra/pull/5134), [#5257](https://github.com/ZcashFoundation/zebra/pull/5257))
- Fix slow getblock RPC (verbose=1) using transaction ID index ([#5307](https://github.com/ZcashFoundation/zebra/pull/5307))
- Open the database in a blocking tokio thread, which allows tokio to run other tasks ([#5228](https://github.com/ZcashFoundation/zebra/pull/5228))
### Contributors
Thank you to everyone who contributed to this release, we couldn't make Zebra without you:
@arya2, @gustavovalverde, @oxarbitrage, @teor2345 and @upbqdn.
This release fixes multiple bugs in proof and signature verification, which were causing big performance issues near the blockchain tip.
It also improves Zebra's sync performance and reliability under heavy load.
### Disk and Network Usage Changes
Zebra now uses around 50 - 100 GB of disk space, because many large transactions were recently added to the block chain. (In the longer term, several hundred GB are likely to be needed.)
When there are a lot of large user-generated transactions on the network, Zebra can upload or download 1 GB or more per day.
- Split the checkpoint and full verification [`sync` concurrency options](https://docs.rs/zebrad/latest/zebrad/components/sync/struct.Config.html) (#4726, #4758):
This option sets the number of threads to use for CPU-bound tasks, such as proof and signature verification.
By default, Zebra uses all available CPU cores.
### Rust Compiler Bug Fixes
- The Rust team has recently [fixed compilation bugs](https://blog.rust-lang.org/2022/07/19/Rust-1.62.1.html) in function coercions of `impl Trait` return types, and `async fn` lifetimes.
We recommend that you update your Rust compiler to release 1.62.1, and re-compile Zebra.
### Added
- Add a `rayon` thread pool for CPU-bound tasks (#4776)
- Run multiple cryptographic batches concurrently within each verifier (#4776)
- Run deserialization of transaction in a rayon thread (#4801)
- Run CPU-intensive state updates in parallel rayon threads (#4802)
- Run CPU-intensive state reads in parallel rayon threads (#4805)
- Support Tiers and supported platforms per Tier doc (#4773)
### Changed
- Update column family names to match Zebra's database design (#4639)
- Update Zebra's mainnet and testnet checkpoints (#4777, #4833)
- Process more blocks and batch items concurrently, so there's a batch ready for each available CPU (#4776)
- Wrap note commitment trees in an `Arc`, to reduce CPU and memory usage (#4757)
- Increment `tokio` dependency from 1.19.2 to 1.20.0, to improve diagnostics (#4780)
### Fixed
- Only verify halo2 proofs once per transaction (#4752)
- Use a separate channel for each cryptographic batch, to avoid dropped batch results (#4750)
- Improve batch fairness and latency under heavy load (#4750, #4776)
- Run all verifier cryptography on blocking CPU-bound threads, using `tokio` and `rayon` (#4750, #4776)
- Use `tokio`'s `PollSemaphore`, instead of an outdated `Semaphore` impl (#4750)
- Check batch worker tasks for panics and task termination (#4750, #4777)
- Limit the length of the `reject` network message's `message` and `reason` fields (#4687)
- Change the `bitvec` dependency from 1.0.0 to 1.0.1, to fix a performance regression (#4769)
- Fix an occasional panic when a `zebra-network` connection closes (#4782)
- Stop panicking when the connection error slot is not set (#4770)
- When writing blocks to disk, don't block other async tasks (#4199)
- When sending headers to peers, only deserialize the header data from disk (#4792)
- Return errors from `send_periodic_heartbeats_with_shutdown_handle` (#4756)
- Make FindHeaders and FindHashes run concurrently with state updates (#4826)
- Stop reading redundant blocks for every FindHashes and FindHeaders request (#4825)
- Generate sapling point outside the method (#4799)
#### CI
- Workaround lightwalletd hangs by waiting until we're near the tip (#4763)
- Split out Canopy logs into a separate job (#4730)
- Make full sync go all the way to the tip (#4709)
- Split Docker logs into sprout, other checkpoints, and full validation (#4704)
- Add a Zebra cached state update test, fix lightwalletd tests (#4813)
Zebra's latest beta continues adding support for `lightwalletd` RPC methods and continues with the testing of each of these features. Also, this beta sets the NU5 mainnet activation height.
Zebra's latest beta completes our work on the NU5 consensus rules. It continues our work on `lightwalletd` RPC methods, and contains some internal CI improvements.
### Added
#### RPCs
- Implement a retry queue for transactions sent via RPCs (#4015)
- Partially implement the `getaddresstxids` RPC method (#4062)
#### State
- Add a transparent address balance index (#3963)
- Add a transparent address UTXO index (#3999)
- Add a transparent address transaction index (#4038)
- Add transparent address indexes to the non-finalized state (#4022)
- Add a query function for transparent address balances (#4097)
### CI
- Create cached state disk image after a successful full sync test (#3986)
### Changed
#### NU5
- Update to new zcash_script V5 API (#3799)
- Update network protocol versions for the second NU5 activation on testnet (#3799)
- Update the consensus branch ID and the second NU5 testnet activation height (#3799)
- Bump database version to trigger testnet rollback (#3799)
#### State
- Store transactions in a separate database index, to improve query speed (#3934)
- Store UTXOs by transaction location rather than transaction hash (#3978)
- Stop storing redundant transparent output fields in the database (#3992)
- Use LZ4 compression for RocksDB (#4027)
- Use Ribbon filters for database index lookups (#4040)
- Tune state database file compaction configuration (#4045)
#### CI
- Put the state version in cached state disk names (#4073)
- Improve mergify merge throughput (#4094, #4120)
- Run cached state rebuilds in main branch (#4107)
- Use GitHub Branch Protection checks instead of Mergify (#4103, #4105)
- Lint and standardize the actions structure (#3940)
#### Rust Dependencies
- Update shared Zcash dependencies for the second NU5 activation on testnet (#3799)
- Disable unused rocksdb compression features (#4082)
- Bump rlimit from 0.7.0 to 0.8.3 (#4051)
#### CI Dependencies
- Bump docker/metadata-action from 3.6.2 to 3.7.0 (#4049)
- Bump google-github-actions/auth from 0.6.0 to 0.7.0 (#4050)
- Bump tj-actions/changed-files from 18.6 to 18.7 (#4065)
- Bump reviewdog/action-actionlint from 1.21.0 to 1.23.0 (#4099, #4125)
- Bump actions/checkout from 3.0.0 to 3.0.1 (#4126)
### Fixed
#### CI
- Validate tests exit code after reading the container logs (#3968, #4069)
- Give enough time to zebra before reading logs (#4123)
Zebra's latest beta fixes a `cargo install` build failure in the previous beta release. It also fixes a `lightwalletd` RPC bug, and improves test coverage.
### Changed
#### Database and State
- Update database design to put ordered list values in RocksDB keys (#3997)
- Make transparent address index database design more consistent (#4019)
#### CI
- Do not invalidate cache between PRs (#3996)
#### Dependency Updates
- Bump hyper from 0.14.17 to 0.14.18 (#3946)
- Bump indexmap from 1.8.0 to 1.8.1 (#4003)
- Bump semver from 1.0.6 to 1.0.7 (#3982)
- Bump serde-big-array from 0.3.2 to 0.4.1 (#4004)
##### Test Dependency Updates
- Bump insta from 1.13.0 to 1.14.0 (#3980)
- Bump tokio-util from 0.7.0 to 0.7.1 (#3981)
##### CI Dependency Updates
- Bump tj-actions/changed-files from 18.4 to 18.6 (#4002)
### Fixed
#### Build
- Fix a compilation error caused by a test-only method in production code (#4000)
- Add a job to ci.yml that does `cargo install --locked --path ./zebrad/ zebrad` (#3998)
#### RPC
- Tell `lightwalletd` to wait for missing blocks in the `getblock` RPC (#3977)
#### State
- Stop panicking when a state block commit fails (#4016)
- Validate ZIP-212 grace period blocks using checkpoints (#3889)
- Store Sapling and Orchard note commitment trees in finalized and non-finalized state (#3818)
- Get addresses from transparent outputs (#3802)
- Explain the different ways .txt files are usedin the CI (#3743)
### Changed
The Zebra team made a huge refactor to the database storage and the state to serve RPC calls efficiently. The refactor is accompanied with extensive low level tests in the form of snapshots.
Zebra's latest beta continues implementing zero-knowledge proof and note commitment tree validation. In this release, we have finished implementing transaction header, transaction amount, and Zebra-specific NU5 validation. (NU5 mainnet validation is waiting on an `orchard` crate update, and some consensus parameter updates.)
We also fix a number of security issues that could pose a local denial of service risk, or make it easier for an attacker to make a node follow a false chain.
As of this release, Zebra will automatically download and cache the Sprout and Sapling Groth16 circuit parameters. The cache uses around 1 GB of disk space. These cached parameters are shared across all Zebra and `zcashd` instances run by the same user.
Zebra's latest beta implements a number of consensus rules which will be needed for Zebra to fully validate all of the Zcash network consensus rules, including those which will activate with NU5.
With this release we are also fixing a number of security issues that could pose a DDoS risk or otherwise negatively impact other nodes on the network.
Finally, this release includes an upgrade to the latest version of tokio (1.14.0).
### Added
- Check Shielded Input and Output Limits (#3069, #3076)
- Sprout note commitment trees (#3051)
- Check per-block limits on transparent signature operations (#3049)
- Calculate Block Subsidy and Funding Streams (#3017, #3040)
- Check for duplicate crate dependencies in CI (#2986)
- Add unused seed peers to the Address Book (#2974, #3019)
#### Network Upgrade 5
- Verify Halo2 proofs as part of V5 transaction verification (#2645, #3039)
This is the first beta release of Zebra. Today the mempool work is fully finished and compatible with [ZIP-401](https://zips.z.cash/zip-0401) and several security issues in the network stack are fixed. In addition to that we improved our documentation specially in the `zebrad` crate while we increased our test coverage. Finally, we get started with the task of upgrading Tokio to version 1.
### Added
#### Mempool
- ZIP-401: weighted random mempool eviction (#2889, #2932)
- Reject a mempool transaction if it has internal spend conflicts (#2843)
- Limit transaction size in the mempool (#2917)
#### Cleanup and performance
- Remove unused mempool errors (#2941)
- Simplify calling add_initial_peers (#2945)
- Disable the new clippy::question_mark lint (#2946)
- Downgrade startup logs to debug to speed up CI (#2938)
- Speed up alternative state and zebrad tests in CI (#2929)
#### Tests
- Restore and update mempool tests (#2966)
- Test multiple chain resets (#2897)
### Security
- Track the number of active inbound and outbound peer connections so we can implement limits (#2912)
Zebra's latest alpha contains the last of the changes to zebra chain state ahead of NU5 testnet activation and before concentrating on mempool work. The remaining NU5 validation will be completed prior to NU5 mainnet activation.
### Added
- Reject connections from outdated peers after network upgrade activation (#2519)
#### Network Upgrade 5
- ZIP-209: Prohibit Negative Shielded Chain Value Pool Balances Partial Implementation (#2546, #2554, #2561, #2569, #2576, #2577, #2578, #2596)
Zebra's latest alpha continues our work on NU5, including Orchard and Transaction V5. It also includes documentation updates and security fixes. In particular, Zebra no longer gossips unreachable addresses to other nodes and users should update and restart any running Zebra nodes.
### Added
- Check if a previous version of Zebra followed a different consensus branch (#2366)
- API for a minimum protocol version during initial block download (#2395)
- Add a verification height to mempool transaction verification requests (#2400)
- Improved error messages by adding database path method to the Finalized State (#2349)
- Add property test strategies for V5 transactions (#2347)
- Re-use a shared Tokio runtime for some Zebra tests (#2397)
#### Documentation
- Add CHANGELOG.md file to the zebra git repo (#2346)
- Client RFC updates (#2367, #2341)
- Explain how Zebra validates shielded coinbase outputs like other shielded outputs (#2382)
- Document required request timeouts due to data dependencies (#2337)
- Update known issues and add inbound network ports to the README (#2373)
- Document shared to per-spend anchor conversion (#2363)
- Document note commitment trees storage (#2259)
#### Network Upgrade 5
- Orchard note commitment tree test vectors (#2384)
- Enable V5 transaction test vectors in the groth16 tests (#2383)
- Validate transparent inputs and outputs in V5 transactions (#2302)
- Batch math & variable-time multiscalar multiplication for RedPallas (#2288)
- Implement asynchronous verifier service for RedPallas (#2318)
- ZIP-211: Validate Disabling Addition of New Value to the Sprout Value Pool (#2399)
### Changed
- Various transaction verifier refactors (#2371, #2432)
- Remove unicode in Zebra's user agent (#2376)
- Update multiple crates to ensure bitvec 0.22.3 is being used (#2351)
- Move transaction consensus branch ID check function to zebra-chain (#2354)
- Replace primitives_types with uint (#2350)
- Refactor to return errors from state update methods to prepare for contextual validation (#2417)
- Refactor of the validation of Sapling shielded data (#2419)
#### Network Upgrade 5
- Update spend and output checks for new Orchard consensus rules (#2398)
### Fixed
- Failed tests in the cached state CI workflow are no longer ignored (#2403)
- Stop skipping the cached sync tests in CI (#2402)
- Fix intermittent errors in the groth16 verifier tests (#2412)
- Skip IPv6 tests when ZEBRA_SKIP_IPV6_TESTS environmental variable is set (#2405)
- Stop failing after the mandatory Canopy checkpoint due to incorrect coinbase script verification (#2404)
- Improved docs and panic messages for zebra_test::command (#2406)
- Gossip dynamic local listener ports to peers (#2277)
- Stop allowing JoinSplits for Halo (#2428)
- Fix failing legacy chain tests (#2427)
### Security
- Zebra no longer gossips unreachable addresses to other nodes (#2392)
- User Action Required: Update and restart any running Zebra nodes
- Avoid duplicate peer connections (#2276)
- Send local listener address to peers (#2276)
- Limit reconnection rate to individual peers (#2275)
- If Zebra was configured with a valid (not unspecified) listener address, it would gossip the ephemeral ports of inbound connections to its peers. This fix stops Zebra sending these useless addresses to its mainnet and testnet peers.
Zebra's latest alpha includes more validation of pre-NU5 consensus rules, continues our work on NU5/Orchard, and fixes some security and protocol correctness issues.
The Zebra project now has a [Code of Conduct](https://github.com/ZcashFoundation/zebra/blob/main/CODE_OF_CONDUCT.md).
### Added
- Design for Transaction V5 (#1886)
- Make shielded data and spends generic over Transaction V4 and V5 (#1946, #1989)
