Release Drafter v6.1.0 has a regression that creates multiple duplicate draft
releases instead of updating existing ones. This change:
- Disables pre-release flag for draft releases
- Excludes pre-releases from being included in drafts
- Explicitly defines PR event types in workflow
See: release-drafter/release-drafter#1425
* Refactor formatting & docs
* Refactor the `runtime` stage in Dockerfile
* Remove unused code from `entrypoint.sh`
* Simplify `entrypoint.sh` setup
* Revise docs & formatting
* Adjust default values for env vars
* Bump Rust v from 1.79 to 1.81 in Dockerfile
* Refactor `entrypoint.sh`
* Refactor `Dockerfile`
* Add TODOs for monitoring stage to Dockerfile
* Refactor `Dockerfile`
* Add TODOs for monitoring stage to Dockerfile
* Fix a typo
* Allow running `zebrad` in test mode
* Allow custom config for `zebrad` in test mode
* Remove `curl` from the `runtime` Docker image
* Remove redundant echos
* Remove a malfunctioning CD test
The test was using a custom config file set in `test_variables`.
However, the file was not included in the Docker image, and the
entrypoint script created a new, default one under the original file's
path. Zebra then loaded this new file, and the test passed because the
pattern in `grep_patterns` matched Zebra's output containing the
original path, even though the config file was different.
* Remove a redundant CI test
* Remove all packages from the `runtime` stage
* Docs cosmetics
* Clarify docs
* Bump Rust version
* Remove a security note
* Explicitly specify network cache dir
* Explicitly specify cookie dir
* Set UID, GID and home dir for the `zebra` user
* Set a working dir for the `zebra` user
* Don't remove `FEATURES`
* Try re-introducing the `testnet-conf` check
* `ZEBRA_CACHED_STATE_DIR` -> `ZEBRA_CACHE_DIR`
This dir doesn't hold only the state cache anymore, but also the cache
for network peers, and the cookie file.
* Refactor the dir structure
* Check that `ZEBRA_CONF_PATH` exists in the image
* Improve the check for `ZEBRA_CONF_PATH`
* Use different flag in the `ZEBRA_CONF_PATH` check
* Simplify the `ZEBRA_CONF_PATH` check
* Fix spelling
* Comment out the `testnet-conf` CI check
* Add commented out `test-zebra-conf-path` CI check
* Reintroduce `testnet-conf` CI check
* Update the `custom-conf` CI check
* Add `v2.1.0.toml` conf file
* Refine the `v2.1.0.toml` conf file
* Remove `ZEBRA_LISTEN_ADDR` from the entrypoint
* Remove `ZEBRA_CHECKPOINT_SYNC` from the entrypoint
* Stop supporting configuration of the RPC port
* Add default conf file
* Prepare Zebra's config in the entrypoint script
* Remove unneeded packages from the `deps` target
* Docs cosmetics
* Use only `$FEATURES` in entrypoint
* Simplify handling of Rust features
* Add a TODO
* Add CI debug statements
* Don't require test vars in conf test
* Reintroduce `protoc`
* Remove `-e NETWORK`
* Remove `ZEBRA_FORCE_USE_COLOR=1`
* Remove `ZEBRA_CACHE_DIR=/var/cache/zebrad-cache`
* Reintroduce the "custom-conf" test
* Set up test env the same way as prod
* Don't repeatedly check for conf file in entrypoint
* Simplify file ownership in Dockerfile
* Fix checkpoint tests in entrypoint
* Fix Zebra config CI tests
* `LIGHTWALLETD_DATA_DIR` -> `LWD_CACHE_DIR`
* Add config for `LWD_CACHE_DIR` to Dockerfile
* `/var/cache/zebrad-cache` -> `~/.cache/zebra`
* `var/cache/lwd-cache` -> `/home/zebra/.cache/lwd`
* Remove `LOG_COLOR=false` from GCP setup
* Don't specify `LWD_CACHE_DIR` in CI tests
* Don't switch to `zebra` user for tests in Docker
* Join "experimental" and "all" tests in CI
* Remove outdated docs
* Refactor tests with fake activation heights
* Fix tests for scanner
* zebra-rpc: Correctly map JSON-RPC to/from 2.0
The existing code was only handling the `lightwalletd` client format
which includes a `"jsonrpc": "1.0"` entry, and it was returning
responses as JSON-RPC 2.0 regardless of what the client expected.
* Update zebra-rpc/src/server/http_request_compatibility.rs
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Adds some of the required fields on `getblockchaininfo` output.
* Adds state request/response variants for querying disk usage
* Adds `size_on_disk`, `chain_supply`, and `monitored` fields.
* Updates snapshots
* fixes prop tests
* fixes doc lints
* Adds missing `size()` method
* Fixes lwd integration test issue by updating get_blockchain_info to fallback on default values instead of returning an error if the state is empty.
Related: Runs state queries in parallel from getblockchaininfo RPC and removes the BlockHeader query by getting the tip block time from the latest chain tip channel.
* Updates failing proptests
* fixes lint
* Adds new invalidate_block method to non finalized state. Adds test case in vectors.rs. Updates non finalized state to track invalidated_blocks
* Removes InvalidatedBlockData struct. Wraps invalidated_blocks HashMap values with Arc. Optimizies invalidate_block fn to return early if root has hash of desired block.
* Update metrics whenever chain_set is modified
* Adds child_blocks method and refactors invalidate_block in chain.rs. Refactors invalidate_block in NonFinalizedState.
---------
Co-authored-by: Elijah Hampton <elijahhampton@pop-os.lan>
Release Drafter v6.1.0 introduced a regression that causes it to create
multiple duplicate draft releases instead of updating existing ones. This
was causing ~160 duplicate draft releases to be created for Zebra 2.2.0.
Pin to v6.0.0 which correctly finds and updates existing draft releases.
This is a known issue tracked in release-drafter/release-drafter#1425.
* add a channel for submit_block notifications to gossip task
* fix tests and gossip logic
* remove the network discriminant and add a test
* clippy suggestions
* fix unused variable
* attempt to fix the conditional compilation issues
* fix default
* Suggestions for "fix(mining): Advertise mined blocks" (#9183)
* refactor error conversions in GetBlockTemplateRpcImpl and rewords documentation
* Replaces polling mined block receiver with a select
* Skip checking that Zebra is likely synced to the network tip before returning block templates on Testnet.
* fixes a clippy lint and a concurrency bug
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
## Motivation
Our CI/CD infrastructure is complex and was lacking proper documentation, making it challenging for newcomers and contributors to understand the system. This PR adds comprehensive documentation to:
- Make our CI/CD architecture more accessible to new contributors
- Document workflow relationships and dependencies
- Explain infrastructure components and their roles
- Provide best practices and known limitations
Closes#9112
### Specifications & References
- [GitHub Actions Path Filtering Limitation](https://github.com/orgs/community/discussions/44490)
- [GitHub Actions Variables in Forks Limitation](https://github.com/orgs/community/discussions/44322)
- [Rust Caching Limitations](https://github.com/ZcashFoundation/zebra/issues/6169#issuecomment-1712776391)
## Solution
1. Added detailed CI/CD documentation in `.github/workflows/README.md`:
- Table of contents for easy navigation
- System overview and core infrastructure details
- Mermaid diagram showing workflow relationships and parallel execution
- Detailed sections on:
- Core infrastructure components
- Workflow organization
- Test execution strategy
- Infrastructure details
- Best practices
- Known issues
2. Added CI/CD section to main `README.md`:
- Brief overview of CI/CD capabilities
- Strategic placement between Getting Started and Documentation sections
- Direct link to detailed documentation
3. Documentation improvements:
- Clear explanation of patch workflows and their rationale
- Detailed infrastructure dependencies and requirements
- Comprehensive coverage of test execution patterns
- External contributor considerations and limitations
* Remove unnecessary clone
* Fix test in zebra-script
* Document that RPC server addr must be set
* Change the log level of missing port msg to info
Having the RPC port not specified is part of Zebra's normal operation,
and is the default config.
* Fix links in docs for tracing
* Update zebrad/src/commands/start.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Wait for spent UTXOs to arrive in the state if they're not found elsewhere instead of returning an error when a transaction in a candidate block has already been verified in the mempool
* updates test, avoids returning InternalDowncastErrors when AwaitUtxo requests time out
* return early if there are no missing deps
* Applies suggestions from code review
* Updates field documentation
* Adds new column family for [spent_out_loc] -> [spending_tx_loc] with a read method and an update to `prepare_spending_transparent_tx_ids_batch()` for maintaining it when committing blocks to the finalized state.
Adds TODOs for remaining production changes needed for issue #8837.
* add spending tx ids for spent outpoints to non-finalized chains
* adds a `spending_transaction_hash()` read fn for the new column family
* Adds a `TransactionIdForSpentOutPoint` ReadRequest and a `TransactionId` ReadResponse
* Updates snapshots, removes outdated TODOs, moves a TODO.
* Clarifies `spent_utxos` field docs, fixes an assertion
* import TypedColumnFamily from `finalized_state` instead of from the crate.
* adds db format upgrade for spent outpoints -> tx hash
* adds revealing tx ids for nullifiers in finalized and non-finalized states
* updates nullifiers column families to include revaling transaction locations in db format upgrade
* Renames new read state request to `SpendingTransactionId` and updates its type to a `Spend` enum
* refactor db format upgrade and prepare_nullifiers_batch() to use ZebraDb instead of DiskDb, checks cancel_receiver before every db operation
* Adds acceptance test for checking that the finalized state has spending transaction ids
* Adds variant docs to zebra_state::request::Spend enum
* Updates Zebra book with the latest changes to the rocks db column families
* Updates acceptance test to check non-finalized state
* adds a few log messages to the acceptance test, reduces frequency of logs for progress updates
* fixes docs lint and skips test when there is no cached state
* Avoids returning genesis coinbase tx hash when indexes are missing
* Adds `indexer` compilation feature in zebra-state and build metadata in db format version file
* stops tracking new indexes in finalized state when feature is unselected
* stops tracking new indexes in non-finalized state when indexer feature is unselected
* condenses imports
* - adds build metadata when writing db version file, if any.
- adds the build metadata to the db version file before adding indexes.
- deletes indexes when running without the `indexer` feature
* Replaces dropping cf with deleting range of all items to avoid a panic when trying to open the db with that column family.
* Fixes lint, avoids reading coinbase transactions from disk
* updates db column families table
* Document need for having an indexed cached state and use a multi-threaded tokio runtime in has_spending_transaction_ids test
* fixes call to renamed `future_blocks` test fn
* improves test logs and fixes a disk format deserialization bug
* Replaces a new expr with a previously existing constant, fixes typo
* Refactors getblockchaininfo to return gensis block if tip pool request fails. Adds test in prop.rs
* clippy, check and fmt
* Removes unused imports. Refactors code for correctness in genesis bytes reference and tokio pause() call
---------
Co-authored-by: Elijah Hampton <elijahhampton@pop-os.lan>
* replace jsonrpc-core with jsonrpsee-types in zebra-node-services
* remove non needed feature from dependency
* remove jsonrpc-core as a dev dependency in zebra-node-services
* add jsonrpsee-types as a dev dependency
- Renamed `no_cached_disk` input to `need_cached_disk` in `cd-deploy-nodes-gcp.yml` to clarify its purpose.
- Adjusted conditional logic for cached disk usage based on event types (release vs. non-release) in `cd-deploy-nodes-gcp.yml`.
- Removed the environment strategy matrix from `sub-deploy-integration-tests-gcp.yml` to simplify the workflow configuration.
- Enhanced comments for better clarity on caching and environment handling across workflows.
* repeat words
* Update zebra-chain/src/block/serialize.rs
Co-authored-by: Pili Guerra <1311133+mpguerra@users.noreply.github.com>
---------
Co-authored-by: Pili Guerra <1311133+mpguerra@users.noreply.github.com>
* feat(actions): use GitHub `environments` for infra deployments
* chore(workflows): enhance environment handling in CI/CD configurations
- Added dynamic environment assignment in `cd-deploy-nodes-gcp.yml` based on event type (release or dev).
- Updated `sub-build-docker-image.yml` to utilize the `inputs.environment` for environment configuration.
- Introduced a strategy matrix for environment selection in `sub-deploy-integration-tests-gcp.yml`, allowing for both dev and prod environments based on the branch.
- Ensured `sub-find-cached-disks.yml` uses the `inputs.environment` for consistency across workflows.
* fix(workflows): streamline environment input handling in CI configurations
- Removed the required environment input from `sub-ci-integration-tests-gcp.yml`.
- Updated comments in `sub-deploy-integration-tests-gcp.yml` to clarify the strategy for creating images in dev and prod environments based on the main branch.
* test: just set the environmet when diff to dev
* refactor(workflows): unify environment handling across CI configurations
- Removed the optional environment input from multiple workflow files, including `manual-zcashd-deploy.yml`, `sub-build-docker-image.yml`, and `sub-deploy-integration-tests-gcp.yml`.
- Updated environment assignment logic to consistently use 'prod' for release events across `cd-deploy-nodes-gcp.yml`, `sub-build-docker-image.yml`, `sub-find-cached-disks.yml`, and `sub-deploy-integration-tests-gcp.yml`.
- Enhanced clarity in comments regarding environment strategies in `sub-deploy-integration-tests-gcp.yml`.
* fix(workflows): update environment assignment logic for CI configurations
- Changed environment assignment in `cd-deploy-nodes-gcp.yml`, `sub-build-docker-image.yml`, and `sub-find-cached-disks.yml` to use 'dev' as a fallback when the event is not a release, to avoid a `false` fallback
## Motivation
Fix https://github.com/ZcashFoundation/zebra/actions/runs/12751834865/job/35539829251?pr=9114.
## Solution
- Remove the workflow and its associated bash script.
### Rationale
> It didn't catch publishing issues in the past, and it skips a bunch of things anyway.
> ..., it wasn't doing much with `--no-verify` anyway.
> It was working fine with the previous version of Rust. I think the latest stable version broke `--no-verify`, because it wasn't pulling dependencies from [crates.io](http://crates.io/) before (which is how we ended up with git sources on main when we were trying to publish Zebra), but now it does even if `--no-verify` is passed in explicitly.
* Add serialized Mainnet blocks for tests
* Add Sapling anchors
* Add Mainnet Orchard anchors
* Remove wrong Testnet NU5 blocks
* Add Testnet blocks with V5 txs to test vectors
* Move the Sapling treestate
* Add Sapling & Orchard anchors
* Remove unneeded test for fake V5 txs
We don't need this test anymore since we have real V5 txs now.
* Add `has_transparent_inputs` to `Transaction`
* Fix `v5_with_sapling_spends` test
* Fix `binding_signatures` test
* Refactor block test vectors
* Use real V5 txs instead of fake ones
* Fix `v5_transaction_is_rejected_before_nu5` test
* Fix `v5_tx_is_accepted_after_nu5_activation` test
* Fix `v5_tx_with_no_outputs_fails_validation` test
* Move `v5_tx_with_no_outputs_fails_validation` test
* Fix `v5_tx_with_no_inputs_fails_verification` test
* Fix `v5_tx_with_orchard_actions_has_inputs..` test
* Fix `v5_coinbase_tx_without_spends_flag_passes`
* Simplify `orchard` imports
* Fix `v5_tx_with_orchard_actions_has_flags` test
* Fix `v5_coinbase_tx_with_enable_spends_fails`
* Fix `v5_tx_with_duplicate_orchard_action` test
* Fix `coinbase_outputs_are_decryptable_for_v5`
* Fix `shielded_outputs_are_not_decryptable_for_v5`
* Use `Network::iter` instead of Mainnet
* Rename basic V5 tx test
* Apply suggestions from code review
Co-authored-by: Arya <aryasolhi@gmail.com>
* Return an `Ok` in tx is not coinbase
* formatting
* Update zebra-consensus/src/transaction/check.rs
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
- Modified job conditions in multiple workflow files to ensure they only run for pull requests from external repositories.
- Updated job names for clarity and consistency across `cd-deploy-nodes-gcp`, `ci-tests`, and `docs-deploy-firebase` workflows, and to sync them with our rulesets
- Enhanced comments to emphasize the importance of synchronizing job names across related workflow files.
Fixes#9108
Gustavo Valverde
Marek
Jack Grigg
Arya
Metalcape
dependabot[bot]
Elijah Hampton
cryptoraph
Alfredo Garcia
Fallengirl
futreall
Pili Guerra
Dimitris Apostolou
root
Marius Kjærstad
idky137