doc: Documentation in Markdown for Depends Dir

Documentation more readable when viewed on Github.

Some extra changes by @laanwj:

- Make README.usage the default README. This is more convenient from a
  user perspective. Link to other documentation in this default README

- Add list of popular targets for cross compilation, change default to
  Win64 instead of Win32
This commit is contained in:
Shawn Wilkinson 2015-04-22 00:06:39 -04:00 committed by Wladimir J. van der Laan
parent b4c219b622
commit cc3db874b5
4 changed files with 137 additions and 100 deletions

56
depends/README.md Normal file
View File

@ -0,0 +1,56 @@
### Usage
To build dependencies for the current arch+OS:
make
To build for another arch/OS:
make HOST=host-platform-triplet
For example:
make HOST=x86_64-w64-mingw32 -j4
A prefix will be generated that's suitable for plugging into Bitcoin's
configure. In the above example, a dir named i686-w64-mingw32 will be
created. To use it for Bitcoin:
./configure --prefix=`pwd`/depends/x86_64-w64-mingw32
Common `host-platform-triplets` for cross compilation are:
- `i686-w64-mingw32` for Win32
- `x86_64-w64-mingw32` for Win64
- `x86_64-apple-darwin11` for MacOSX
- `arm-linux-gnueabihf` for Linux ARM
No other options are needed, the paths are automatically configured.
Dependency Options:
The following can be set when running make: make FOO=bar
SOURCES_PATH: downloaded sources will be placed here
BASE_CACHE: built packages will be placed here
SDK_PATH: Path where sdk's can be found (used by OSX)
FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up
NO_QT: Don't download/build/cache qt and its dependencies
NO_WALLET: Don't download/build/cache libs needed to enable the wallet
NO_UPNP: Don't download/build/cache packages needed for enabling upnp
DEBUG: disable some optimizations and enable more runtime checking
If some packages are not built, for example `make NO_WALLET=1`, the appropriate
options will be passed to bitcoin's configure. In this case, `--disable-wallet`.
Additional targets:
download: run 'make download' to fetch all sources without building them
download-osx: run 'make download-osx' to fetch all sources needed for osx builds
download-win: run 'make download-win' to fetch all sources needed for win builds
download-linux: run 'make download-linux' to fetch all sources needed for linux builds
### Other documentation
- [description.md](description.md): General description of the depends system
- [packages.md](packages.md): Steps for adding packages

View File

@ -1,34 +0,0 @@
To build dependencies for the current arch+OS:
make
To build for another arch/OS:
make HOST=host-platform-triplet && make HOST=host-platform-triplet
(For example: make HOST=i686-w64-mingw32 -j4)
A prefix will be generated that's suitable for plugging into Bitcoin's
configure. In the above example, a dir named i686-w64-mingw32 will be
created. To use it for Bitcoin:
./configure --prefix=`pwd`/depends/i686-w64-mingw32
No other options are needed, the paths are automatically configured.
Dependency Options:
The following can be set when running make: make FOO=bar
SOURCES_PATH: downloaded sources will be placed here
BASE_CACHE: built packages will be placed here
SDK_PATH: Path where sdk's can be found (used by OSX)
FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up
NO_QT: Don't download/build/cache qt and its dependencies
NO_WALLET: Don't download/build/cache libs needed to enable the wallet
NO_UPNP: Don't download/build/cache packages needed for enabling upnp
DEBUG: disable some optimizations and enable more runtime checking
If some packages are not built, for example 'make NO_WALLET=1', the appropriate
options will be passed to bitcoin's configure. In this case, --disable-wallet.
Additional targets:
download: run 'make download' to fetch all sources without building them
download-osx: run 'make download-osx' to fetch all sources needed for osx builds
download-win: run 'make download-win' to fetch all sources needed for win builds
download-linux: run 'make download-linux' to fetch all sources needed for linux builds

View File

@ -1,9 +1,7 @@
This is a system of building and caching dependencies necessary for building This is a system of building and caching dependencies necessary for building Bitcoin.
Bitcoin.
There are several features that make it different from most similar systems: There are several features that make it different from most similar systems:
- It is designed to be builder and host agnostic ### It is designed to be builder and host agnostic
In theory, binaries for any target OS/architecture can be created, from a In theory, binaries for any target OS/architecture can be created, from a
builder running any OS/architecture. In practice, build-side tools must be builder running any OS/architecture. In practice, build-side tools must be
@ -11,18 +9,18 @@ specified when the defaults don't fit, and packages must be amended to work
on new hosts. For now, a build architecture of x86_64 is assumed, either on on new hosts. For now, a build architecture of x86_64 is assumed, either on
Linux or OSX. Linux or OSX.
- No reliance on timestamps ### No reliance on timestamps
File presence is used to determine what needs to be built. This makes the File presence is used to determine what needs to be built. This makes the
results distributable and easily digestable by automated builders. results distributable and easily digestable by automated builders.
- Each build only has its specified dependencies available at build-time. ### Each build only has its specified dependencies available at build-time.
For each build, the sysroot is wiped and the (recursive) dependencies are For each build, the sysroot is wiped and the (recursive) dependencies are
installed. This makes each build deterministic, since there will never be any installed. This makes each build deterministic, since there will never be any
unknown files available to cause side-effects. unknown files available to cause side-effects.
- Each package is cached and only rebuilt as needed. ### Each package is cached and only rebuilt as needed.
Before building, a unique build-id is generated for each package. This id Before building, a unique build-id is generated for each package. This id
consists of a hash of all files used to build the package (Makefiles, packages, consists of a hash of all files used to build the package (Makefiles, packages,
@ -32,7 +30,7 @@ any other package that depends on it. If any of the main makefiles (Makefile,
funcs.mk, etc) are changed, all packages will be rebuilt. After building, the funcs.mk, etc) are changed, all packages will be rebuilt. After building, the
results are cached into a tarball that can be re-used and distributed. results are cached into a tarball that can be re-used and distributed.
- Package build results are (relatively) deterministic. ### Package build results are (relatively) deterministic.
Each package is configured and patched so that it will yield the same Each package is configured and patched so that it will yield the same
build-results with each consequent build, within a reasonable set of build-results with each consequent build, within a reasonable set of
@ -41,13 +39,13 @@ beyond the scope of this system. Additionally, the toolchain itself must be
capable of deterministic results. When revisions are properly bumped, a cached capable of deterministic results. When revisions are properly bumped, a cached
build should represent an exact single payload. build should represent an exact single payload.
- Sources are fetched and verified automatically ### Sources are fetched and verified automatically
Each package must define its source location and checksum. The build will fail Each package must define its source location and checksum. The build will fail
if the fetched source does not match. Sources may be pre-seeded and/or cached if the fetched source does not match. Sources may be pre-seeded and/or cached
as desired. as desired.
- Self-cleaning ### Self-cleaning
Build and staging dirs are wiped after use, and any previous version of a Build and staging dirs are wiped after use, and any previous version of a
cached result is removed following a successful build. Automated builders cached result is removed following a successful build. Automated builders

View File

@ -4,121 +4,137 @@ variables, and defining build commands.
The package "mylib" will be used here as an example The package "mylib" will be used here as an example
General tips: General tips:
mylib_foo is written as $(package)_foo in order to make recipes more similar. - mylib_foo is written as $(package)_foo in order to make recipes more similar.
Identifiers: ## Identifiers
Each package is required to define at least these variables: Each package is required to define at least these variables:
$(package)_version:
$(package)_version:
Version of the upstream library or program. If there is no version, a Version of the upstream library or program. If there is no version, a
placeholder such as 1.0 can be used. placeholder such as 1.0 can be used.
$(package)_download_path:
$(package)_download_path:
Location of the upstream source, without the file-name. Usually http or Location of the upstream source, without the file-name. Usually http or
ftp. ftp.
$(package)_file_name:
$(package)_file_name:
The upstream source filename available at the download path. The upstream source filename available at the download path.
$(package)_sha256_hash:
$(package)_sha256_hash:
The sha256 hash of the upstream file The sha256 hash of the upstream file
These variables are optional: These variables are optional:
$(package)_build_subdir:
$(package)_build_subdir:
cd to this dir before running configure/build/stage commands. cd to this dir before running configure/build/stage commands.
$(package)_download_file:
$(package)_download_file:
The file-name of the upstream source if it differs from how it should be The file-name of the upstream source if it differs from how it should be
stored locally. This can be used to avoid storing file-names with strange stored locally. This can be used to avoid storing file-names with strange
characters. characters.
$(package)_dependencies:
$(package)_dependencies:
Names of any other packages that this one depends on. Names of any other packages that this one depends on.
$(package)_patches:
$(package)_patches:
Filenames of any patches needed to build the package Filenames of any patches needed to build the package
$(package)_extra_sources:
$(package)_extra_sources:
Any extra files that will be fetched via $(package)_fetch_cmds. These are Any extra files that will be fetched via $(package)_fetch_cmds. These are
specified so that they can be fetched and verified via 'make download'. specified so that they can be fetched and verified via 'make download'.
Build Variables:
## Build Variables:
After defining the main identifiers, build variables may be added or customized After defining the main identifiers, build variables may be added or customized
before running the build commands. They should be added to a function called before running the build commands. They should be added to a function called
$(package)_set_vars. For example: $(package)_set_vars. For example:
define $(package)_set_vars define $(package)_set_vars
... ...
endef endef
Most variables can be prefixed with the host, architecture, or both, to make Most variables can be prefixed with the host, architecture, or both, to make
the modifications specific to that case. For example: the modifications specific to that case. For example:
Universal: $(package)_cc=gcc Universal: $(package)_cc=gcc
Linux only: $(package)_linux_cc=gcc Linux only: $(package)_linux_cc=gcc
x86_64 only: $(package)_x86_64_cc = gcc x86_64 only: $(package)_x86_64_cc = gcc
x86_64 linux only: $(package)_x86_64_linux_cc = gcc x86_64 linux only: $(package)_x86_64_linux_cc = gcc
These variables may be set to override or append their default values. These variables may be set to override or append their default values.
$(package)_cc
$(package)_cxx $(package)_cc
$(package)_objc $(package)_cxx
$(package)_objcxx $(package)_objc
$(package)_ar $(package)_objcxx
$(package)_ranlib $(package)_ar
$(package)_libtool $(package)_ranlib
$(package)_nm $(package)_libtool
$(package)_cflags $(package)_nm
$(package)_cxxflags $(package)_cflags
$(package)_ldflags $(package)_cxxflags
$(package)_cppflags $(package)_ldflags
$(package)_config_env $(package)_cppflags
$(package)_build_env $(package)_config_env
$(package)_stage_env $(package)_build_env
$(package)_build_opts $(package)_stage_env
$(package)_config_opts $(package)_build_opts
$(package)_config_opts
The *_env variables are used to add environment variables to the respective The *_env variables are used to add environment variables to the respective
commands. commands.
Many variables respect a debug/release suffix as well, in order to use them for Many variables respect a debug/release suffix as well, in order to use them for
only the appropriate build config. For example: only the appropriate build config. For example:
$(package)_cflags_release = -O3
$(package)_cflags_i686_debug = -g $(package)_cflags_release = -O3
$(package)_config_opts_release = --disable-debug $(package)_cflags_i686_debug = -g
$(package)_config_opts_release = --disable-debug
These will be used in addition to the options that do not specify These will be used in addition to the options that do not specify
debug/release. All builds are considered to be release unless DEBUG=1 is set by debug/release. All builds are considered to be release unless DEBUG=1 is set by
the user. the user. Other variables may be defined as needed.
Other variables may be defined as needed. ## Build commands:
Build commands:
For each build, a unique build dir and staging dir are created. For example, For each build, a unique build dir and staging dir are created. For example,
work/build/mylib/1.0-1adac830f6e and work/staging/mylib/1.0-1adac830f6e. `work/build/mylib/1.0-1adac830f6e` and `work/staging/mylib/1.0-1adac830f6e`.
The following build commands are available for each recipe: The following build commands are available for each recipe:
$(package)_fetch_cmds: $(package)_fetch_cmds:
Runs from: build dir Runs from: build dir
Fetch the source file. If undefined, it will be fetched and verified Fetch the source file. If undefined, it will be fetched and verified
against its hash. against its hash.
$(package)_extract_cmds:
$(package)_extract_cmds:
Runs from: build dir Runs from: build dir
Verify the source file against its hash and extract it. If undefined, the Verify the source file against its hash and extract it. If undefined, the
source is assumed to be a tarball. source is assumed to be a tarball.
$(package)_preprocess_cmds:
$(package)_preprocess_cmds:
Runs from: build dir/$(package)_build_subdir Runs from: build dir/$(package)_build_subdir
Preprocess the source as necessary. If undefined, does nothing. Preprocess the source as necessary. If undefined, does nothing.
$(package)_config_cmds:
$(package)_config_cmds:
Runs from: build dir/$(package)_build_subdir Runs from: build dir/$(package)_build_subdir
Configure the source. If undefined, does nothing. Configure the source. If undefined, does nothing.
$(package)_build_cmds:
$(package)_build_cmds:
Runs from: build dir/$(package)_build_subdir Runs from: build dir/$(package)_build_subdir
Build the source. If undefined, does nothing. Build the source. If undefined, does nothing.
$(package)_stage_cmds:
$(package)_stage_cmds:
Runs from: build dir/$(package)_build_subdir Runs from: build dir/$(package)_build_subdir
Stage the build results. If undefined, does nothing. Stage the build results. If undefined, does nothing.
The following variables are available for each recipe: The following variables are available for each recipe:
$(1)_staging_dir: package's destination sysroot path
$(1)_staging_prefix_dir: prefix path inside of the package's staging dir $(1)_staging_dir: package's destination sysroot path
$(1)_extract_dir: path to the package's extracted sources $(1)_staging_prefix_dir: prefix path inside of the package's staging dir
$(1)_build_dir: path where configure/build/stage commands will be run $(1)_extract_dir: path to the package's extracted sources
$(1)_patch_dir: path where the package's patches (if any) are found $(1)_build_dir: path where configure/build/stage commands will be run
$(1)_patch_dir: path where the package's patches (if any) are found
Notes on build commands: Notes on build commands:
@ -127,4 +143,5 @@ configure step to (usually) correctly configure automatically. Any
$($(package)_config_opts) will be appended. $($(package)_config_opts) will be appended.
Most autotools projects can be properly staged using: Most autotools projects can be properly staged using:
$(MAKE) DESTDIR=$($(package)_staging_dir) install
$(MAKE) DESTDIR=$($(package)_staging_dir) install