* fix(ci): Ensure release deployments run and improve script clarity
This addresses several issues in the CD workflow:
- Modifies the 'build' job to remove its dependency on 'get-disk-name'.
This allows the 'build' job and subsequently 'deploy-nodes' to run
during release events, as 'get-disk-name' is intentionally skipped
for releases.
- Updates the 'Create instance template' script in 'deploy-nodes' to
correctly handle disk parameters for release events. It now ensures
that release deployments attempt to use the existing persistent disk
(e.g., 'zebrad-cache-mainnet') for state continuity, rather than
relying on a dynamic cached image from the (skipped) 'get-disk-name' job.
- Refactors in-script comments within the 'deploy-nodes' job and moved them to the top, to avoid clutter in the script
* feat(ci): Implement stateful disk policy for release MIGs
This commit introduces a new step "Configure stateful disk policy for
release MIG" to the `deploy-nodes` job in the
`.github/workflows/cd-deploy-nodes-gcp.yml` workflow.
For `release` events, this step applies a stateful policy to the
Managed Instance Group (MIG) using the command:
`gcloud compute instance-groups managed set-stateful-policy`.
The policy is configured with
`--stateful-disk "device-name=zebrad-cache-${NETWORK},auto-delete=never"`.
This ensures that the specified data disk, which holds the Zebra node's
state, is preserved during instance recreations or updates within the MIG,
maintaining data persistence for production release deployments.
* Regroup imports in `.../tests/common/regtest.rs`
* Regroup imports in `.../methods/tests/prop.rs`
* Regroup imports in `.../get_block_template.rs`
* Regroup imports in `.../zip317.rs`
* Regroup imports in `.../get_blockchain_info.rs`
* Regroup imports in `.../types/get_raw_mempool.rs`
* Regroup imports in `.../types/transaction.rs`
* Regroup imports in `.../server/tests/vectors.rs`
* Regroup imports in `.../args.rs`
* Regroup imports in `.../main.rs`
* Regroup imports in `.../openapi-generator/main.rs`
* Regroup imports in `zebrad/src/commands/start.rs`
* Regroup imports in `.../tests/fake_peer_set.rs`
* Regroup imports in `.../components/miner.rs`
* Regroup imports in `.../tests/acceptance.rs`
* Regroup imports in `.../get_block_template.rs`
* Update changelog
* Update end of support
* Add release date
* Add breaking change description
* Add new PRs to changelog
* Add breaking change description to changelog
* Update list of PRs in changelog
* chore: Release
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* add size and time to getrawtransaction object
* fix annoying clippy lint about large enum variant
* add todo
* add TODO, remove TODO
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
---------
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Tests in `sub-test-zebra-config.yml` were failing in some instances with exit code 137 (SIGKILL) or 139 (SIGSEGV) during the `docker stop` phase, even when the preceding log check (`grep`) succeeded. This occurred because zebrad sometimes doesn't shut down gracefully within the default docker stop timeout.
This change modifies the exit logic to return status 0 if the grep command passes, regardless of the exit code from `docker wait`. This aligns the step result with the actual test outcome determined by log analysis and mirrors the fix from PR #8107.
Ref: #7898
- Removed `latest_tag` and `tag_suffix` inputs from Docker build workflow, as those are no longer required.
- Split SHA tagging into PR and branch events for clarity.
- Removed tag event references and flavor customizations to prevent redundant tags.
- Updated comments for clarity on tag purposes.
Closes#7415
* Introduce Nu7
* Introduce Nu7 fo other crates of Zebra
* Fix of new_regtest call (as it additionally needs Nu7 arg now)
* Fix of new_regtest call (as it additionally needs Nu7 arg now) (2)
* Set Nu7 as a network update for testnet in zebra-chain network tests
* Fix serde names for NU7
* Update test snapshot in zebra-rpc to use NU7
* Copy Nu7-related changes from zsa-integration-state
* Uncomment zcash_unstable = nu7
* Fix Nu7 constants and add cfg(zcash_unstable ...) attribute according to PR #15 review comments
* Update network constant values according to PR #15 review comments
* Add zcash_unstable=nu6 Rust flag to .cargo/config.toml to use nu6 by default
* Add zcash_unstable = nu6 cfg in zebra-network/.../types.rs
* Fix nu7 activation heights in network_upgrade.rs (nu6 value + 1 does not work - causes a test failure)
* Add cfg for CURRENT_NETWORK_PROTOCOL_VERSION constant definition, add FIXMEs for a couple of Nu6 processing cases, add processing of Nu7
* Update get_blockchain_info@testnet_10.snap
* Update get_blockchain_info@mainnet_10.snap
* updated cfg flag
* remove `zcash_unstable` flag
* Remove testnet and mainnet NU7 activation heights, comment out consensus branch id and conversion to zcash primitives nu type
* Yep, it'll work after NU7
* Yep, the test should work fine after NU7
* Yep, it only needs to be a post-nu6 height.
* other fixes and cleanups
* Updates snapshots, removes unnecessary FIXMEs, fixes issues with bad merge conflict resolution
* fixes test hang when there are NUs defined without activation heights
* fixes test, applies suggestion from code review.
* Enable consensus branch ID for Nu7
---------
Co-authored-by: Dmitry Demin <dmitry@qed-it.com>
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
The `HistoryTree` type is intended for internal use within the node; it
should not be exposed via the ReadStateService response. Instead, this
response should simply include the hash needed for block template
construction.
* fix(ci): set `ZEBRA_CACHE_DIR` in full and checkpoint syncs
Updates the GitHub Actions workflows for GCP integration tests to explicitly set the ZEBRA_CACHE_DIR environment variable to `/home/zebra/.cache/zebra` for the following test jobs:
- regenerate-stateful-disks (Zebra checkpoint sync to mandatory checkpoint)
- test-full-sync (Zebra full sync to tip on Mainnet)
- test-full-sync-testnet (Zebra full sync to tip on Testnet)
The rationale for this change is to address permission issues encountered during tests that attempt to create or access a cached database directory. By setting ZEBRA_CACHE_DIR to a specific path within the user's home directory, we ensure that the test has the necessary write permissions to create and manage the cache directory, avoiding 'Permission denied (os error 13)' errors that occur when using default or system directories without adequate access rights.
* chore: fix typo
* refactor(docker): Enhance Zebra configuration options and entrypoint logic
- Introduced multiple methods for configuring Zebra, including custom config file paths and automatic generation from environment variables.
- Updated entrypoint script to handle configuration more robustly, checking for existing files and generating defaults as needed.
- Improved documentation in `docker.md` to clarify configuration options and their precedence.
- Adjusted environment variable handling in `docker/.env` for better clarity and functionality.
- Refactored `prepare_conf_file` to create a complete configuration based on environment variables, streamlining the setup process.
* fix(docker): remove non-essential variables and set correct defaults
* feat(ci): Centralize zebra configuration testing with Docker
- Replace multiple separate test jobs with a single comprehensive matrix-based test
- Create a new ADR documenting the design decision for centralizing Docker tests
- Move all test scenarios from both CI and CD pipelines into a single reusable workflow
- Define extensive test matrix covering network, RPC, directory, feature configurations
- Improve workflow readability with descriptive test names and clear organization
- Simplify workflow inputs to only require the Docker image identifier
* chore(ci): cleanup jobs that already don't exist
* refactor(docker): Enhance Zebra configuration options and entrypoint logic
- Introduced multiple methods for configuring Zebra, including custom config file paths and automatic generation from environment variables.
- Updated entrypoint script to handle configuration more robustly, checking for existing files and generating defaults as needed.
- Improved documentation in `docker.md` to clarify configuration options and their precedence.
- Adjusted environment variable handling in `docker/.env` for better clarity and functionality.
- Refactored `prepare_conf_file` to create a complete configuration based on environment variables, streamlining the setup process.
* fix(entrypoint): use correct `ZEBRA_RPC_PORT` variable
This also removes the logic to set the default port for RPC as this was introducing a buggy logic
* fix(docker): remove non-essential variables and set correct defaults
* chore(docker): use the default config for docker-compose as the default approach
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* chore: better comment for user understanding
* fix(entrypoint): set permissions correctly for parent directories
* chore: align documentation and implementation
* chore: clearer and aligned comments with docker-compose
* fix(entrypoint): chown the zebra config file when `ZEBRA_CONF_PATH` is not set
---------
Co-authored-by: Marek <mail@marek.onl>
* make start and end fields optional and apply new rules to match zcashd
* update docs to mark range fields as optionals
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* extend `getrawtransaction` verbose output
* replace strings where possible, fix tests
* fix orchard
* add todos for script assembly code
* fix an import
* fmt
* fmt
* suggestion for #9329: change coinbase field and add valueSat field to match zcashd (#9407)
* change coinbase field and add valueSat field to match zcashd
* remove valueSat; update vectors
---------
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* fix(test): wait for genesis block before launching lightwalletd
The `lightwalletd_integration` test would fail when running with an empty
`zebrad` state (`LaunchWithEmptyState` test type). `lightwalletd` expects
the connected `zebrad` instance to have processed at least the genesis
block upon startup and treats a completely empty state as a fatal error.
This adds a wait step specifically for the empty state scenario.
The test now waits for `zebrad` to log that it has committed the genesis
block (Height 0) before launching the `lightwalletd` process. This
ensures `zebrad` is ready and prevents `lightwalletd` from exiting
prematurely.
* fix(test): use syncer state tip log for lightwalletd launch condition
* fix(test): lint
* refactor!: remove experimental features from release builds
BREAKING CHANGE: Experimental features are no longer built into release images.
Users must now build custom images with specific features if needed.
- Remove experimental build job from release workflow
- Remove experimental features from CI matrix
- Update documentation to reflect new feature strategy
- Simplify Dockerfile by removing experimental feature args
- Keep RUN_ALL_EXPERIMENTAL_TESTS for testing purposes only
Migration: Users who need specific features should:
1. Build custom images using --build-arg FEATURES="feature1 feature2"
2. Use the documented feature flags in their builds
3. Refer to the new documentation for building with custom features
Closes #9210Closes#9204
Partially #7415
* revert(ci): Add support for experimental features in test runs
Enable passing experimental features to Docker test runs by:
- Updating workflow configuration to pass EXPERIMENTAL_FEATURES
- Modifying entrypoint script to include experimental features during test execution
- Documenting new experimental test category in workflow README
* refactor(docs): streamline Docker documentation and feature testing strategy
- Reorganized and clarified advanced usage instructions for building Docker images with custom features, including metrics support.
- Enhanced the presentation of alternatives and configuration details for Docker usage.
* Apply suggestions from code review
Co-authored-by: Marek <mail@marek.onl>
* chore: revert deleted sentence
* Update book/src/user/docker.md
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
Arya
Jack Grigg
Gustavo Valverde
Alfredo Garcia
idky137
Marek
ala-mode
Conrado Gouvea
natalie
Paul
Tomass
Kris Nuttycombe
Mark Henderson
dependabot[bot]