RFC: Release process (#1063)
* rfc: initial draft of release process proposal This is a version of the notes I posted in slack for preliminary discussion, slightly reformatted to fit the Zebra RFC template. @yaahc suggested I post them as an RFC, to have a concrete proposal for discussion. * Add draft note and link to ticket and PR Co-authored-by: teor <teor@riseup.net>
This commit is contained in:
Henry de Valence
GitHub
parent
258bd881aa
commit
0e5a6d7efd
|
|
@ -0,0 +1,197 @@
|
|||
- Feature Name: `release_planning`
|
||||
- Start Date: 2020-09-14
|
||||
- Design PR: [ZcashFoundation/zebra#1063](https://github.com/ZcashFoundation/zebra/pull/1063)
|
||||
- Zebra Issue: [ZcashFoundation/zebra#1963](https://github.com/ZcashFoundation/zebra/issues/1963)
|
||||
|
||||
# Draft
|
||||
|
||||
Note: This is a draft Zebra RFC. See
|
||||
[ZcashFoundation/zebra#1963](https://github.com/ZcashFoundation/zebra/issues/1963)
|
||||
for more details.
|
||||
|
||||
# Summary
|
||||
[summary]: #summary
|
||||
|
||||
Release and distribution plans for Zebra.
|
||||
|
||||
# Motivation
|
||||
[motivation]: #motivation
|
||||
|
||||
We need to plan our release and distribution processes for Zebra. Since these
|
||||
processes determine how users get Zebra and place constraints on how we do
|
||||
Zebra development, it's important to think through the implications of our
|
||||
process choices.
|
||||
|
||||
# Guide-level explanation
|
||||
[guide-level-explanation]: #guide-level-explanation
|
||||
|
||||
Zebra is developed *library-first*, as a collection of independently useful
|
||||
Rust crates composed together to create `zebrad`, the full node implementation.
|
||||
This means that our release and distribution processes need to handle
|
||||
distribution of the `zebra-*` libraries, as well as of `zebrad`.
|
||||
|
||||
The official distribution channels are as follows:
|
||||
|
||||
- `zebra-*` libraries are distributed via Cargo, using
|
||||
[crates.io](https://crates.io);
|
||||
- `zebrad` is distributed in binary form via Docker images generated in CI
|
||||
(binaries) or in source form via `cargo install`.
|
||||
|
||||
The release process is controlled by pushing an appropriate tag to the
|
||||
`ZcashFoundation/zebra` git repository.
|
||||
|
||||
# Reference-level explanation
|
||||
[reference-level-explanation]: #reference-level-explanation
|
||||
|
||||
(This section should describe the mechanics of the release process in more
|
||||
detail, once we have agreement on distribution channels. Currently, one
|
||||
suggestion is described and motivated below).
|
||||
|
||||
# Rationale and alternatives
|
||||
[rationale-and-alternatives]: #rationale-and-alternatives
|
||||
|
||||
## Versioning
|
||||
|
||||
We previously agreed on a tentative versioning policy for `zebrad` and the
|
||||
component `zebra-` libraries. In both cases, we follow semver rules. For
|
||||
`zebrad`, we plan to align the major version number with the NU number, so that
|
||||
mainnet NU3 corresponds to `3.x`, mainnet NU4 to `4.x`, etc. For the `zebra-` libraries, we
|
||||
commit to following semver rules, but plan to increment the major versions as
|
||||
fast as we need to implement the features we want.
|
||||
|
||||
## Distribution Channels
|
||||
|
||||
To handle releases of the component `zebra-` libraries, there's a clear best
|
||||
answer: publish the libraries to crates.io, so that they can be used by other
|
||||
Rust projects by adding an appropriate line to the `Cargo.toml`.
|
||||
|
||||
For `zebrad` the situation is somewhat different, because it is an application,
|
||||
not a library. Because Zcash is a living protocol and `zebrad` is living
|
||||
software, whatever process we choose must consider how it handles updates.
|
||||
Broadly speaking, possible processes can be divided into three categories:
|
||||
|
||||
1. Do not expect our users to update their software and do not provide them a
|
||||
means to do so;
|
||||
|
||||
2. Use an existing update / deployment mechanism to distribute our software;
|
||||
|
||||
3. Write our own update / deployment mechanism to distribute our software.
|
||||
|
||||
The first category is mentioned for completeness, but we need to provide users
|
||||
with a way to update their software. Unfortunately, this means that standalone
|
||||
binaries without an update mechanism are not a workable option for us. The
|
||||
third category is also unfavorable, because it creates a large amount of work
|
||||
for a task that is not really the focus of our product. This suggests that we
|
||||
focus on solutions in the second category.
|
||||
|
||||
One solution in the second category is to publish Docker images. This has a
|
||||
number of attractive features. First, we already produce Docker images for our
|
||||
own cloud deployments, so there is little-to-no marginal effort required to
|
||||
produce these for others as a distribution mechanism. Second, providing Docker
|
||||
images will make it easier for us to provide a collection of related software
|
||||
in the future (e.g., providing an easy-to-deploy Prometheus / Grafana instance,
|
||||
or a sidecar Tor instance). Third, Docker has a solid upgrade story, and we
|
||||
can instruct users to use the `:latest` version of the Docker image or steer
|
||||
them to auto-update mechanisms like Watchtower.
|
||||
|
||||
While this solution works well for cloud deployments, Docker is not suitable
|
||||
everywhere. What should we do outside of Docker? One solution would be to try
|
||||
to create packages for each platform-specific package manager (Homebrew,
|
||||
something for Windows, various different flavors of Linux distribution), but
|
||||
this creates a large amount of additional work requiring platform-specific
|
||||
knowledge. Worse, this work cannot be outsourced to others without giving up
|
||||
control over our software distribution -- if, for instance, a third party
|
||||
creates a Homebrew package, and we recommend people install Zebra using that
|
||||
package, we're reliant on that third party to continue packaging our software
|
||||
forever, or leave our users stranded.
|
||||
|
||||
Instead, we can publish `zebrad` as a Rust crate and recommend `cargo install`.
|
||||
This approach has two major downsides. First, installation takes longer,
|
||||
because Zebra is compiled locally. Second, as long as we have a dependency on
|
||||
`zcashconsensus`, we'll have to instruct users to install some
|
||||
platform-specific equivalent of a `build-essential` package. And as long as
|
||||
we depend on `zcash_script`, we'll have to instruct users to install `libclang`.
|
||||
However, even for
|
||||
crates such as `zcashconsensus` that build native code, the `cargo`-managed
|
||||
build process is far, far more reliable than build processes for C or C++
|
||||
projects. We would not be asking users to run autotools or `./configure`, just
|
||||
a one-step `cargo install`. We also know that it's possible to reliably build
|
||||
Zebra on each platform with minimal additional steps, because we do so in our
|
||||
CI.
|
||||
|
||||
In contrast to these downsides, distributing `zebra` through Cargo has a number
|
||||
of upsides. First, because we distribute our libraries using crates.io, we
|
||||
already have to manage tooling for publishing to crates.io, so there's no
|
||||
additional work required to publish `zebrad` this way. Second, we get a
|
||||
cross-platform update mechanism with no additional work, since `cargo install`
|
||||
will upgrade to the latest published version. Third, we don't rely on any
|
||||
third parties to mediate the relationship between us and our users, so users
|
||||
can get updates as soon as we publish them. Fourth, unlike a system package
|
||||
manager, we can pin exact hashes of every transitive dependency (via the
|
||||
`Cargo.lock`, which `cargo install` can be configured to respect). Fifth,
|
||||
we're positioned to pick up (or contribute to) ecosystem-wide integrity
|
||||
improvements like a transparency log for `crates.io` or work on reproducible
|
||||
builds for Rust.
|
||||
|
||||
This proposal is summarized above in the [guide-level
|
||||
explanation](#guide-level-explanation).
|
||||
|
||||
## Release Processes
|
||||
|
||||
The next question is what kind of release processes and automation we should
|
||||
use. Here are two important priorities for these processes:
|
||||
|
||||
1. Reducing the friction of doing any individual release, allowing us to move
|
||||
closer to a continuous deployment model;
|
||||
2. Reducing the risk of error in the release process.
|
||||
|
||||
These are roughly in order of priority but they're clearly related, since the
|
||||
more friction we have in the release process, the greater the risk of error,
|
||||
and the greater the risk of error, the more friction we require to prevent it.
|
||||
|
||||
Automation helps to reduce friction and to reduce the risk of error, but
|
||||
although it helps enormously, it has declining returns after some point (in the
|
||||
sense that automating the final 5% of the work is significantly more complex
|
||||
and error-prone than automating the first 5% of the work). So the challenge is
|
||||
to find the right "entry point" for the automated part of the system that
|
||||
strikes the right balance.
|
||||
|
||||
One possibility is the following. CD automation is triggered by pushing a new
|
||||
tag to the git repository. Tags are specified as `crate-semver`, e.g.
|
||||
`zebra-network-3.2.8`, `zebrad-3.9.1`, etc. When a new tag is pushed, the CD
|
||||
automation parses the tag to determine the crate name. If it is `zebrad`, it
|
||||
builds a new Docker image and publishes the image and the crate. Otherwise, it
|
||||
just publishes the crate.
|
||||
|
||||
To publish a new version of any component crate, the process is:
|
||||
|
||||
1. Edit the `Cargo.toml` to increment the version number;
|
||||
2. Update `crate/CHANGELOG.md` with a few human-readable sentences describing
|
||||
changes since the last release (examples: [cl1], [cl2], [cl3]);
|
||||
3. Submit a PR with these changes;
|
||||
4. Tag the merge commit and push the tag to the git repo.
|
||||
|
||||
[cl1]: https://github.com/ZcashFoundation/ed25519-zebra/blob/main/CHANGELOG.md
|
||||
[cl2]: https://github.com/dalek-cryptography/x25519-dalek/blob/master/CHANGELOG.md
|
||||
[cl3]: https://github.com/dalek-cryptography/curve25519-dalek/blob/master/CHANGELOG.md
|
||||
|
||||
All subsequent steps (publishing to crates.io, building docker images, etc) are
|
||||
fully automated.
|
||||
|
||||
Why make these choices?
|
||||
|
||||
- Triggering on a tag, rather than some other trigger, ensures that each
|
||||
deployment corresponds to a particular, known, state of the source
|
||||
repository.
|
||||
|
||||
- Editing the version number in the `Cargo.toml` should be done manually,
|
||||
because it is the source of truth for the crate version.
|
||||
|
||||
- Changelog entries should be written by hand, rather than auto-generated from
|
||||
`git log`, because the information useful as part of a changelog is generally
|
||||
not the same as the information useful as part of commit messages. (If this
|
||||
were not the case, changelogs would not be useful, because `git log` already
|
||||
exists). Writing the changelog entries by hand would be a burden if we
|
||||
queued a massive set of changes between releases, but because releases are
|
||||
low-friction and we control the distribution channel, we can avoid this
|
||||
problem by releasing frequently, on a weekly or daily basis.
|
||||
Loading…
Reference in New Issue