* fix(release): do not hardcode the name of our images
Our main image in DockerHub is called `zebra` not `zebrad`.
This hardcoded value is also causing the mining image to also be called `zebra` instead of `zebrad-mining-rpcs-testnet`
* fix(release): use same naming convention for `.experimental`
* chore: fix comment
* ci(build): unpin specific `buildkit` version
We previously had an issue with the following error: `cannot reuse body, request must be retried`
This commonly was a wrong error, caused by a containerd issue which has being tracked and solved here: https://github.com/docker/build-push-action/issues/761#issuecomment-1406261692
We're having errors when building, and this might be caused by an underliying error which containerd is not showing us correctly.
* ci(deploy): Use specific subnetworks on GCP VMs
* Add extra test type modes to support zebra-checkpoints
* Add Mainnet and Testnet zebra-checkpoints test harnesses
* Add zebra-checkpoints to test docker images
* Add zebra-checkpoints test entrypoints
* Add Mainnet CI workflow for zebra-checkpoints
* Enable zebra-checkpoints feature in the test image
* Use the same features for (almost) all the docker tests
* Make workflow features match Docker features
* Add a feature note
* Add a zebra-checkpoints test feature to zebrad
* Remove the "no cached state" testnet code
* Log a startup message to standard error when launching zebra-checkpoints
* Rename tests to avoid partial name conflicts
* Fix log formatting
* Add sentry feature to experimental docker image build
* Explain what ENTRYPOINT_FEATURES is used for
* Use the correct zebra-checkpoints path
* Silence zebrad logs while generating checkpoints
* Fix zebra-checkpoints log handling
* Re-enable waiting for zebrad to fully sync
* Add documentation for how to run these tests individually
* Start generating checkpoints from the last compiled-in checkpoint
* Fix clippy lints
* Revert changes to TestType
* Wait for all the checkpoints before finishing
* Add more stderr debugging to zebra-checkpoints
* Fix an outdated module comment
* Add a workaround for zebra-checkpoints launch/run issues
* Use temp dir and log what it is
* Log extra metadata about the zebra-checkpoints binary
* Add note about unstable feature -Z bindeps
* Temporarily make the test run faster and with debug info
* Log the original test command name when showing stdout and stderr
* Try zebra-checkpoints in the system path first, then the cargo path
* Fix slow thread close bug in dual process test harness
* If the logs are shown, don't say they are hidden
* Run `zebra-checkpoints --help` to work out what's going on in CI
* Build `zebra-utils` binaries for `zebrad` integration tests
* Revert temporary debugging changes
* Revert changes that were moved to another PR
* refactor(ci): use GitHub secrets and variables
We've been using values that are variable across multiple workflows,
and those can only be changed if modifying the workflows, but we should
be able to change the values without committing new changes in the code
for this purpose we're now using GitHub Variables, and even moving
non-sensitive information into variables instead of secrets. Allowing
more flexibility and other scenarios that should be easier to manage,
like deploying to Mainnet or Testnet.
* refactor(ci): use new GitHub variables for GCP auth
* fix(ci): typo
* fix(ci): do not use multiple variables for the same value
* fix(ci): typo in variable
* fix(vars): use different variables for machine types
* fix(vars): missing substitution
* fix: typo
* fix: make the input CI network override the default network
* Use the correct network variable for creating disks
---------
Co-authored-by: teor <teor@riseup.net>
* Duplicates Dockerfile
* updates mining-testnet Dockerfile with getblocktemplate-rpcs feature, Testnet by default, and an RPC port
* renames mining-testnet.Dockerfile and adds workflow for publishing images on release
* replaces space-seperated features with commas
* Adds .experimental tag suffix, removes new dockerfile, makes lightwalletd tests conditional
* updates build-args to pass on features directly
* adds "lightwalletd-grpc-tests" as default test_features in build-docker-image
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* adds tag suffix to cache keys
---------
Co-authored-by: teor <teor@riseup.net>
Docker does not allow to use multiple tags for cache images in their
`cache-from` and `cache-to` options. Which makes some images to lose
previous tags, if the same.
This cause automated mechanims for deletion to fail, as those cache images
were dependant on the main image, and deleting the cache alone was raising
an error if the main image was not deleted first.
A workaround for this is making a separate repository which can hold this
cache images. Which also reduces clutter from the main registries.
* Add latest and edge tags to Docker images
* Document how latest tag actually works
* Try a different syntax for is_default_branch
* Try again
* One last try
* Revert changes that don't work
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Use branch then main build caches
* Revert cache order to try branch cache first
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* feat(release): create Docker hub binaries when tagging
* fix(release): add a release workflow for binaries
* fix(release): trigger on tag creation, not pushing to it
* fix(release): use the same conditions for logging into DockerHub
* fix(release): add missing parameter to access GH secrets
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* ci(release): just publish to DockerHub when a release is published
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* ci(release): filter prerelease event correctly
* ci(release): fix tags
* ci(release): use `zebra` and not `zebrad` as the repository
* ci(release): do not try to login to Docker if not a release
* Update .github/workflows/build-docker-image.yml
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: teor <teor@riseup.net>
* feat(build): deploy long running instances on release
Previous behavior:
Each time we merged to main new nodes would be deployed, this is an
expected behavior as we need to ensure nodes get deployed and run
without issues, but this could also replace nodes very hastily.
Expected behavior:
We want instances which would run for a longer time, to allow us to
troubleshoot issues or inspect the behavior of this instances for longer
periods of time (2+ weeks)
Applied solution:
Deploy a versioned manage instance group (MiG) using the major version
of the release semver. We just use the first part of the version to
replace old instances, and change it when a major version is released
to keep a segregation between new and old versions.
* ci(build): allow v0 as a major version tag
* fix(build): use rust conventions for versioning
* fix(deploy): improve documentation and trigger on release
* Update .github/workflows/continous-delivery.yml
Co-authored-by: teor <teor@riseup.net>
* fix(versioning): typo
* fix(deploy): use `zebrad-v1` as the instance name, with no SHA
* fix(deploy): create and update MiG must use the same name
* docs(deployments): add Continuous Delivery process
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Previous behavior:
Sometimes Google Cloud authentication fails, this might happen before
IAM permissions are fully propagated
Expected behavior:
If the authentication fails, retry at least 3 times before exiting with
a non zero exit code
Applied solution:
Google GitHub Actions for auth recently added this a `retries` feature
which is now implemented to workaround this issue.
Note: 95a6bc2a27
Fixes https://github.com/ZcashFoundation/zebra/issues/4846
* Apply the same Rust logging settings to all GitHub workflows
* Enable full optimisations in dev builds for downloading large parameter files
* Disable beta Rust tests in CI
* fix(ci): sentry is not longer being activated in test builds
This removes sentry from all the test execution, as some tests might fail as sentry wasn't initially built, or it might take more time to build as it will have to build with sentry.
* fix(build): workaround the failed to fetch oauth token error
* refactor(build): use better stage naming and document it
* refactort(build): use multiple cache sources
* docs(build): add a comment for cache
* fix(build): remove gcr.io as it does not supports OCI images
teor
dependabot[bot]
Gustavo Valverde
Arya
Dimitris Apostolou