zebra/.github/workflows/build-docker-image.yml

185 lines
6.9 KiB
YAML
Raw Normal View History

name: Build docker image
on:
workflow_call:
inputs:
network:
required: false
type: string
image_name:
required: true
type: string
dockerfile_path:
required: true
type: string
dockerfile_target:
required: true
type: string
short_sha:
required: false
type: string
rust_backtrace:
required: false
type: string
rust_lib_backtrace:
required: false
type: string
colorbt_show_hidden:
required: false
type: string
zebra_skip_ipv6_tests:
required: false
type: string
rust_log:
required: false
type: string
default: info
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
# keep these in sync with:
# https://github.com/ZcashFoundation/zebra/blob/main/docker/Dockerfile#L83
features:
required: false
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
default: "sentry"
type: string
test_features:
required: false
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
default: "lightwalletd-grpc-tests zebra-checkpoints"
type: string
rpc_port:
required: false
type: string
tag_suffix:
required: false
type: string
no_cache:
description: 'Disable the Docker cache for this build'
required: false
type: boolean
default: false
outputs:
image_digest:
description: 'The image digest to be used on a caller workflow'
value: ${{ jobs.build.outputs.image_digest }}
jobs:
build:
name: Build images
timeout-minutes: 210
runs-on: ubuntu-latest
outputs:
image_digest: ${{ steps.docker_build.outputs.digest }}
image_name: ${{ fromJSON(steps.docker_build.outputs.metadata)['image.name'] }}
permissions:
contents: 'read'
id-token: 'write'
steps:
- uses: actions/checkout@v3.5.2
with:
persist-credentials: false
- uses: r7kamura/rust-problem-matchers@v1.3.0
- name: Inject slug/short variables
uses: rlespinasse/github-slug-action@v4
with:
short-length: 7
# Automatic tag management and OCI Image Format Specification for labels
- name: Docker meta
id: meta
uses: docker/metadata-action@v4.5.0
with:
# list of Docker images to use as base name for tags
images: |
us-docker.pkg.dev/${{ vars.GCP_PROJECT }}/zebra/${{ inputs.image_name }}
zfnd/${{ inputs.image_name }},enable=${{ github.event_name == 'release' && !github.event.release.prerelease }}
# appends inputs.tag_suffix to image tags/names
flavor: |
suffix=${{ inputs.tag_suffix }}
# generate Docker tags based on the following events/attributes
tags: |
type=schedule
# semver and ref,tag automatically add a "latest" tag, but only on stable releases
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=ref,event=tag
type=ref,event=branch
type=ref,event=pr
type=sha
# edge is the latest commit on the default branch.
type=edge,enable={{is_default_branch}}
# Setup Docker Buildx to allow use of docker cache layers from GH
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2
- name: Authenticate to Google Cloud
id: auth
uses: google-github-actions/auth@v1.1.1
with:
retries: '3'
workload_identity_provider: '${{ vars.GCP_WIF }}'
service_account: '${{ vars.GCP_ARTIFACTS_SA }}'
token_format: 'access_token'
# Some builds might take over an hour, and Google's default lifetime duration for
# an access token is 1 hour (3600s). We increase this to 3 hours (10800s)
# as some builds take over an hour.
access_token_lifetime: 10800s
- name: Login to Google Artifact Registry
uses: docker/login-action@v2.2.0
with:
registry: us-docker.pkg.dev
username: oauth2accesstoken
password: ${{ steps.auth.outputs.access_token }}
- name: Login to DockerHub
# We only publish images to DockerHub if a release is not a pre-release
# Ref: https://github.com/orgs/community/discussions/26281#discussioncomment-3251177
if: ${{ github.event_name == 'release' && !github.event.release.prerelease }}
uses: docker/login-action@v2.2.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# Build and push image to Google Artifact Registry, and possibly DockerHub
- name: Build & push
id: docker_build
uses: docker/build-push-action@v4.0.0
with:
target: ${{ inputs.dockerfile_target }}
context: .
file: ${{ inputs.dockerfile_path }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-args: |
NETWORK=${{ inputs.network }}
SHORT_SHA=${{ env.GITHUB_SHA_SHORT }}
RUST_BACKTRACE=${{ inputs.rust_backtrace }}
RUST_LIB_BACKTRACE=${{ inputs.rust_lib_backtrace }}
COLORBT_SHOW_HIDDEN=${{ inputs.colorbt_show_hidden }}
ZEBRA_SKIP_IPV6_TESTS=${{ inputs.zebra_skip_ipv6_tests }}
RUST_LOG=${{ inputs.rust_log }}
FEATURES=${{ inputs.features }}
TEST_FEATURES=${{ inputs.test_features }}
RPC_PORT=${{ inputs.rpc_port }}
push: true
# Don't read from the cache if the caller disabled it.
# https://docs.docker.com/engine/reference/commandline/buildx_build/#options
no-cache: ${{ inputs.no_cache }}
2022-05-15 22:33:08 -07:00
# To improve build speeds, for each branch we push an additional image to the registry,
# to be used as the caching layer, using the `max` caching mode.
#
# We use multiple cache sources to confirm a cache hit, starting from a per-branch cache.
# If there's no hit, we continue with a `main` branch cache, which helps us avoid
# rebuilding cargo-chef, most dependencies, and possibly some Zebra crates.
#
# The caches are tried in top-down order, the first available cache is used:
# https://github.com/moby/moby/pull/26839#issuecomment-277383550
cache-from: |
type=registry,ref=us-docker.pkg.dev/${{ vars.GCP_PROJECT }}/zebra-caching/${{ inputs.image_name }}${{ inputs.tag_suffix }}:${{ env.GITHUB_REF_SLUG_URL }}-cache
type=registry,ref=us-docker.pkg.dev/${{ vars.GCP_PROJECT }}/zebra-caching/${{ inputs.image_name }}${{ inputs.tag_suffix }}:main-cache
cache-to: |
type=registry,ref=us-docker.pkg.dev/${{ vars.GCP_PROJECT }}/zebra-caching/${{ inputs.image_name }}${{ inputs.tag_suffix }}:${{ env.GITHUB_REF_SLUG_URL }}-cache,mode=max