change(release): Make the latest tag point to the production build, rather than the build with experimental features (#7817)

* Remove DockerHub tags for v1.x.y and edge

* Make the latest tag point to the production image, not experimental

* Remove docker image file change trigger from the OS CI workflow

* Disable latest tag by default

* Enable the latest tag for the production build only

* Restore `edge` tag, it is used in CI and it is CI only

* Add Docker tag removal to CHANGELOG.md

* Actually enforce job order

* Revert tag publishing changes, but keep extra docs and tag order changes

* Update the changelog

.github/workflows/ci-unit-tests-os.yml

@@ -51,7 +51,6 @@ on:
       - '**/clippy.toml'
       # workflow definitions
       - '.github/workflows/ci-unit-tests-os.yml'
-      - '.github/workflows/sub-build-docker-image.yml'
 
 env:
   CARGO_INCREMENTAL: ${{ vars.CARGO_INCREMENTAL }}

.github/workflows/release-binaries.yml

@@ -18,19 +18,6 @@ jobs:
   # Each time this workflow is executed, a build will be triggered to create a new image
   # with the corresponding tags using information from git
 
-  # The image will be named `zebra:<semver>`
-  build:
-    name: Build Release Docker
-    uses: ./.github/workflows/sub-build-docker-image.yml
-    with:
-      dockerfile_path: ./docker/Dockerfile
-      dockerfile_target: runtime
-      image_name: zebra
-      features: ${{ vars.RUST_PROD_FEATURES }}
-      rust_log: ${{ vars.RUST_LOG }}
-    # This step needs access to Docker Hub secrets to run successfully
-    secrets: inherit
-
   # The image will be named `zebra:<semver>.experimental`
   build-experimental:
     name: Build Experimental Features Release Docker
@@ -45,6 +32,24 @@ jobs:
     # This step needs access to Docker Hub secrets to run successfully
     secrets: inherit
 
+  # The image will be named `zebra:<semver>`
+  # It should be built last, so tags with the same name point to the production build, not the experimental build.
+  build:
+    name: Build Release Docker
+    # Run this build last, regardless of whether experimental worked
+    needs: build-experimental
+    if: always()
+    uses: ./.github/workflows/sub-build-docker-image.yml
+    with:
+      dockerfile_path: ./docker/Dockerfile
+      dockerfile_target: runtime
+      image_name: zebra
+      latest_tag: true
+      features: ${{ vars.RUST_PROD_FEATURES }}
+      rust_log: ${{ vars.RUST_LOG }}
+    # This step needs access to Docker Hub secrets to run successfully
+    secrets: inherit
+
   failure-issue:
     name: Open or update issues for release binaries failures
     # When a new job is added to this workflow, add it to this list.

.github/workflows/sub-build-docker-image.yml

@@ -33,6 +33,10 @@ on:
       test_features:
         required: false
         type: string
+      latest_tag:
+        required: false
+        type: boolean
+        default: false
       tag_suffix:
         required: false
         type: string
@@ -88,19 +92,34 @@ jobs:
           # appends inputs.tag_suffix to image tags/names
           flavor: |
             suffix=${{ inputs.tag_suffix }}
+            latest=${{ inputs.latest_tag }}
           # generate Docker tags based on the following events/attributes
           tags: |
-            type=schedule
+            # These DockerHub release tags support the following use cases:
-            # semver and ref,tag automatically add a "latest" tag, but only on stable releases
+            # - `latest`: always use the latest Zebra release when you pull or update
+            # - `1`: use the latest Zebra release, but require manual intervention for the next network upgrade
+            # - `1.x`: update to bug fix releases, but don't add any new features or incompatibilities
+            # - `v1.x.y` or `1.x.y`: always use the exact version, don't automatically upgrade
+            # - `sha-zzzzzz`: always use the exact commit (the same as `1.x.y`, but also used in CI and production image tests)
+            #
+            # Stopping publishing some tags is a silently breaking change:
+            # - `1`: doesn't get expected new consensus-compatible releases or bug fixes
+            # - `1.x`: doesn't get expected bug fixes
+            #
+            # `semver` adds a "latest" tag if `inputs.latest_tag` is `true`.
             type=semver,pattern={{version}}
             type=semver,pattern={{major}}.{{minor}}
             type=semver,pattern={{major}}
             type=ref,event=tag
-            type=ref,event=branch
+            # DockerHub release and CI tags.
-            type=ref,event=pr
+            # This tag makes sure tests are using exactly the right image, even when multiple PRs run at the same time.
             type=sha
-            # edge is the latest commit on the default branch.
+            # These CI-only tags support CI on PRs, the main branch, and scheduled full syncs. 
+            # These tags do not appear on DockerHub, because DockerHub images are only published on the release event.
+            type=ref,event=pr
+            type=ref,event=branch
             type=edge,enable={{is_default_branch}}
+            type=schedule
 
       # Setup Docker Buildx to allow use of docker cache layers from GH
       - name: Set up Docker Buildx

CHANGELOG.md

@@ -11,9 +11,52 @@ Zebra's mining RPCs are now available in release builds. Our Docker images are s
 because the smaller Zcash verification parameters are now built into the `zebrad` binary.
 TODO: rest of intro
 
-This release contains the following changes:
+### Deprecation Warnings
 
-### Mining RPCs in Production Builds
+This release has the following deprecation warnings:
+
+#### Warning: Deprecation of DockerHub Image Tags in a future release
+
+Zebra currently publishes 11 [DockerHub tags](https://hub.docker.com/r/zfnd/zebra/tags) for each new release.
+We want to reduce the number of DockerHub tags we publish in a future minor Zebra release.
+
+Based on usage and user feedback, we could stop publishing:
+- The `1` tag, which updates each release until NU6
+- The `1.x` tag, which updates each patch release until the next minor release
+- The `1.x.y` tag, which is the same as `v1.x.y`
+- The `sha-xxxxxxx` tag, which is the same as `v1.x.y` (for production releases)
+
+We also want to standardise experimental image tags to `-experimental`, rather than `.experimental`.
+
+So for release 1.5.0, we might only publish these tags:
+- `latest`
+- `latest-experimental` (a new tag)
+- `v1.5.0`
+- `v1.5.0-experimental`
+
+Please let us know if you need any other tags by [opening a GitHub ticket](https://github.com/ZcashFoundation/zebra/issues/new?assignees=&labels=C-enhancement%2CS-needs-triage&projects=&template=feature_request.yml&title=feature%3A+).
+
+We recommend using the `latest` tag to always get the most recent Zebra release.
+
+#### Warning: Documentation Website URL Change
+
+We have replaced the API documentation on the [doc.zebra.zfnd.org](https://doc.zebra.zfnd.org)
+website with [docs.rs](https://docs.rs/releases/search?query=zebra). All links have been updated.
+
+Zebra's API documentation can be found on:
+- [`docs.rs`](https://docs.rs/releases/search?query=zebra), which renders documentation for the
+  public API of the latest crate releases;
+- [`doc-internal.zebra.zfnd.org`](https://doc-internal.zebra.zfnd.org/), which renders
+  documentation for the internal API on the `main` branch.
+
+[doc.zebra.zfnd.org](https://doc.zebra.zfnd.org) stopped being updated a few days before this release,
+and it will soon be shut down.
+
+### Significant Changes
+
+This release contains the following significant changes:
+
+#### Mining RPCs in Production Builds
 
 Zebra's mining RPCs are now available in release builds (#7740). Any Zebra instance can be used
 by a solo miner or mining pool. This stabilises 12 RPCs, including  `getblocktemplate`, `submitblock`,
@@ -23,7 +66,7 @@ read our [mining blog post](https://zfnd.org/experimental-mining-support-in-zebr
 Please [let us know](https://github.com/ZcashFoundation/zebra/issues/new?assignees=&labels=C-enhancement%2CS-needs-triage&projects=&template=feature_request.yml&title=feature%3A+)
 if your mining pool needs extra RPC methods or fields.
 
-### Zcash Parameters in `zebrad` Binary
+#### Zcash Parameters in `zebrad` Binary
 
 `zebrad` now bundles zk-SNARK parameters directly into its binary. This increases the binary size
 by a few megabytes, but reduces the size of the Docker image by around 600 MB because
@@ -44,17 +87,6 @@ so it can't be used to retry failed downloads in `zebrad` 1.3.0 and earlier.
 
 We recommend upgrading to the latest Zebra release to avoid download issues in new installs.
 
-### Documentation Website Change
-
-We have replaced the API documentation on the [doc.zebra.zfnd.org](https://doc.zebra.zfnd.org)
-website with [docs.rs](https://docs.rs/releases/search?query=zebra). All links have been updated.
-
-Zebra's API documentation can be found on:
-- [`docs.rs`](https://docs.rs/releases/search?query=zebra), which renders documentation for the
-  public API of the latest crate releases;
-- [`doc-internal.zebra.zfnd.org`](https://doc-internal.zebra.zfnd.org/), which renders
-  documentation for the internal API on the `main` branch.
-
 ### Security
 
 TODO: rest of changelog