For mingw32:
- We use the binaries provided by MSYS2, which do not go back as far as
libc++ 8. We use libc++ 9 here, matching the LLVM version we will be
switching to in a subsequent commit (to match the LLVM version used by
Rust 1.44). We manually specify the path to the libc++ headers,
because they don't match the headers used by Clang itself.
- We now require at least mingw-w64 6.0.0, which fixes two Clang
compilation bugs:
- 1bd66b53be
- 82b169c573
Darwin is ignored, as the Xcode SDK includes libc++.
For all other targets, we use the static libraries included in Clang
releases. We reuse the files we downloaded in native_clang for native
compilation, instead of fetching the archive twice.
Clang is used for compiling C/C++ dependencies, as well as the Zcash
binaries themselves. GCC is still required for compiling native
toolchain dependencies (such as ccache).
By blanket passing --disable-dependency-tracking to all depends packages
we end up with some warnings like:
configure: WARNING: unrecognized options: --disable-dependency-tracking
So instead, only pass it to packages that understand it.
Related to https://github.com/bitcoin/bitcoin/issues/16354.
We use pkg-config where we can, which generally replaces libtool at a
higher level and does not have the same downsides as libtool. These
archives sit in our depends tree with no purpose and pollute the final
bitcoin build with massive overlinking.
- The old patch is no longer necessary because of this upstream fix:
https://github.com/boostorg/build/pull/560
- Boost 1.72 removed a <deque> from an include, which exposed a missing
include in src/httpserver.cpp.
- Boost 1.73 moved function placeholders into the boost::placeholders
namespace.
- The new patch is a fix from just after Boost 1.74 was released, fixing
a warning that was missed.
By using the old HOST with -unknown, `make -C depends download` was
interpreting the download-linux step as a cross-compile, and attempting
to download a Rust stdlib for Linux that we weren't pinning (because we
don't support cross-compiling to x86_64 Linux - just build it native).
Follow-up to https://github.com/zcash/zcash/pull/4749.
During Gitian builds, SOURCES_PATH is set to a path within the Gitian
cache. Normally this path is created as part of creating a particular
dependency's source directory, but in some situations we may vendor Rust
crates before this folder exists.
When we first integrated Rust into our build system, we had two
limitations:
- We were building the `librustzcash` FFI library as a dependency, and
therefore needed access to its crate dependencies in the depends
system.
- Gitian builds happen offline, so we needed to fetch any crate
dependencies ahead of time, and then configure cargo to use these in
an offline environment.
At the time, `cargo` already had support for "Source Replacement", but
there was no easy way to package the dependencies in the necessary way.
What we implemented was effectively the `cargo-vendor` tool, built using
Makefiles. A noticeable downside was that we were pinning dependencies
twice: once in the `Cargo.lock` for the FFI library, and again in our
depends system.
Since then, `cargo-vendor` has been upstreamed into `cargo` itself, and
we have moved `librustzcash` into this repository. We can therefore use
`cargo vendor` directly from our pinned Rust compiler to fetch the
dependencies, and rely on our local `Cargo.lock` to pin the specific
crates we are relying on.
All the text from a make action is passed as arguments to a single
execve call, and it can't be longer than the maximum size allowed by the
operating system. We now have enough Rust crates vendored by the depends
system that we are hitting this limit here.
Use the Rust tracing crate for C++ logging
This PR swaps in the `tracing` crate (via FFI) for logging to either standard
output or `debug.log`. It transparently maps all existing `LogPrintf` and
`LogPrint` invocations to info-level `tracing` events, and passes through
correct file and line information. `error` invocations are mapped to error-level
`tracing` events, currently without line information (due to the way that
`error` is used in the codebase; swapping individual callsites to the new
`LogError` macro will provide that information).
The end-goal for this change is that we don't need to make any disruptive
changes to the codebase, but we can start to leverage `tracing`-specific
functionality where we want to, such as providing extra fields on certain log
lines (that can be filtered for), adding spans to record the flow of execution
through `zcashd`, and logging within C++ and Rust simultaneously. Support
for extra fields on spans and events will be added in a subsequent PR.
The `-debug` config options are converted at launch into their corresponding
directives for tracing's `EnvFilter`. The new `setlogfilter` RPC method allows
this filter to be reloaded dynamically. The syntax is documented in the
`setlogfilter` help text, as well as here:
https://docs.rs/tracing-subscriber/0.2.7/tracing_subscriber/filter/struct.EnvFilter.html#directives
When `-printtoconsole` is specified, the output now includes timestamps and
ANSI encoding :)
depends: Revert to using upstreams as primary download paths
We use the depends system for vendoring `zcashd` dependencies, pinning them
with SHA-256 hashes. It supports fetching dependencies from both their
upstream archive source, and a mirror operated by ECC.
In #816, we switched to the ECC mirror as the primary source, due to an
unreliable upstream (SourceForge). However, this only addressed the symptom
(that dependency builds would reliably fail with an unreliable upstream that
was serving incorrect files). In particular, if the ECC mirror were to become
similarly unreliable, the issue would return.
This PR fixes the core problem, by downloading dependencies and checking
their hashes as an atomic operation. This gives us greater resiliency, as
both the primary and fallback would need to fail in order to halt the build.
Having addressed this problem, we also switch back to using upstreams as
primary download paths.
Jack Grigg
str4d
Nate Wilcox
fanquake
Carl Dong
Dimitris Apostolou
Sean Bowe
Homu