Clang's download path includes its version, but LLVM doesn't always
publish every package for every version, so we need to support multiple
versions.
Fixeszcash/zcash#4954.
For mingw32:
- We use the binaries provided by MSYS2, which do not go back as far as
libc++ 8. We use libc++ 9 here, matching the LLVM version we will be
switching to in a subsequent commit (to match the LLVM version used by
Rust 1.44). We manually specify the path to the libc++ headers,
because they don't match the headers used by Clang itself.
- We now require at least mingw-w64 6.0.0, which fixes two Clang
compilation bugs:
- 1bd66b53be
- 82b169c573
Darwin is ignored, as the Xcode SDK includes libc++.
For all other targets, we use the static libraries included in Clang
releases. We reuse the files we downloaded in native_clang for native
compilation, instead of fetching the archive twice.
By blanket passing --disable-dependency-tracking to all depends packages
we end up with some warnings like:
configure: WARNING: unrecognized options: --disable-dependency-tracking
So instead, only pass it to packages that understand it.
Related to https://github.com/bitcoin/bitcoin/issues/16354.
When we first integrated Rust into our build system, we had two
limitations:
- We were building the `librustzcash` FFI library as a dependency, and
therefore needed access to its crate dependencies in the depends
system.
- Gitian builds happen offline, so we needed to fetch any crate
dependencies ahead of time, and then configure cargo to use these in
an offline environment.
At the time, `cargo` already had support for "Source Replacement", but
there was no easy way to package the dependencies in the necessary way.
What we implemented was effectively the `cargo-vendor` tool, built using
Makefiles. A noticeable downside was that we were pinning dependencies
twice: once in the `Cargo.lock` for the FFI library, and again in our
depends system.
Since then, `cargo-vendor` has been upstreamed into `cargo` itself, and
we have moved `librustzcash` into this repository. We can therefore use
`cargo vendor` directly from our pinned Rust compiler to fetch the
dependencies, and rely on our local `Cargo.lock` to pin the specific
crates we are relying on.
The previous behaviour was to use FALLBACK_DOWNLOAD_PATH to download
dependencies if the primary did not resolve. This was not resilient
against primaries that either mis-report HTTP status codes (e.g.
SourceForge returning 200 OK alongside a 404 webpage), or did not
guarantee artifacts to be bit-stable (e.g. GitHub regenerating commit
archive caches in a non-reproducible manner); in either case, the
incorrect file would be fetched and then the build would fail due to
hash mismatch.
The new behaviour is to download dependencies and check their hashes as
an atomic operation, and use FALLBACK_DOWNLOAD_PATH if any part of the
operation fails.
For normal users, --no-same-owner is default, but not so for root, where
it is assumed that root can change ownership willy-nilly. This is not
the case for privilege-limited container environments where we gaslight
the process into thinking it's root.
Zcash: Excludes QT changes
In some cases, such as for github revision tarballs, the upstream
filename is just ``$HASH.tar.gz``, but the local filename is
``${PACKAGE_NAME}-${HASH}.tar.gz``. With this patch, it's clear which
packages each URLs provides *and* it makes the process of updating
http://z.cash/depends/sources a simple scp rather than involving some
kind of name translation.
Closes#622.
In some cases (Travis), sources and build caches may be moved around in-between
builds, and we can't necessarily trust that everything is still intact.
This introduces pre-build checks that verify against stashed checksums.
Note that this will cause all sources to be re-downloaded, since cached sources
weren't trustworthy before this.
Broken hash logic caused all depends on some platforms (osx at least) to end up
with the same build-id. Without this fix, nothing will be rebuilt when recipes
or dependencies change.
Since the last commit will force rebuilds of all depends, take the opportunity
to clean up a few other things that would trigger rebuilds as well.
- Move source stamps to the sources dir so that SOURCES_PATH is respected for
"make download".
- Only print "fetching..." when actually downloading a file.
- Avoid using non-deterministic paths for the recipe hash (patch location).
This should ensure that all builders get the same resulting build-ids.
- Use a per-package source paths. This will allow for removing old source files
in the future.
- Use a host-agnostic path for downloads which gets cleaned up properly.
Some sources are renamed after download, since the filenames don't play nice
with (for example) gitian. This fixes the rename.
Needed for OSX build as it renames a file.
Jack Grigg
fanquake
Carl Dong
Carl Dong
kozyilmaz
Homu
Taylor Hornby
Nathan Wilcox
Cory Fields