doc: Documentation in Markdown for Depends Dir

Documentation more readable when viewed on Github.

Some extra changes by @laanwj:

- Make README.usage the default README. This is more convenient from a
  user perspective. Link to other documentation in this default README

- Add list of popular targets for cross compilation, change default to
  Win64 instead of Win32
This commit is contained in:
Shawn Wilkinson 2015-04-22 00:06:39 -04:00 committed by Wladimir J. van der Laan
parent b4c219b622
commit cc3db874b5
4 changed files with 137 additions and 100 deletions

56
depends/README.md Normal file
View File

@ -0,0 +1,56 @@
### Usage
To build dependencies for the current arch+OS:
make
To build for another arch/OS:
make HOST=host-platform-triplet
For example:
make HOST=x86_64-w64-mingw32 -j4
A prefix will be generated that's suitable for plugging into Bitcoin's
configure. In the above example, a dir named i686-w64-mingw32 will be
created. To use it for Bitcoin:
./configure --prefix=`pwd`/depends/x86_64-w64-mingw32
Common `host-platform-triplets` for cross compilation are:
- `i686-w64-mingw32` for Win32
- `x86_64-w64-mingw32` for Win64
- `x86_64-apple-darwin11` for MacOSX
- `arm-linux-gnueabihf` for Linux ARM
No other options are needed, the paths are automatically configured.
Dependency Options:
The following can be set when running make: make FOO=bar
SOURCES_PATH: downloaded sources will be placed here
BASE_CACHE: built packages will be placed here
SDK_PATH: Path where sdk's can be found (used by OSX)
FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up
NO_QT: Don't download/build/cache qt and its dependencies
NO_WALLET: Don't download/build/cache libs needed to enable the wallet
NO_UPNP: Don't download/build/cache packages needed for enabling upnp
DEBUG: disable some optimizations and enable more runtime checking
If some packages are not built, for example `make NO_WALLET=1`, the appropriate
options will be passed to bitcoin's configure. In this case, `--disable-wallet`.
Additional targets:
download: run 'make download' to fetch all sources without building them
download-osx: run 'make download-osx' to fetch all sources needed for osx builds
download-win: run 'make download-win' to fetch all sources needed for win builds
download-linux: run 'make download-linux' to fetch all sources needed for linux builds
### Other documentation
- [description.md](description.md): General description of the depends system
- [packages.md](packages.md): Steps for adding packages

View File

@ -1,34 +0,0 @@
To build dependencies for the current arch+OS:
make
To build for another arch/OS:
make HOST=host-platform-triplet && make HOST=host-platform-triplet
(For example: make HOST=i686-w64-mingw32 -j4)
A prefix will be generated that's suitable for plugging into Bitcoin's
configure. In the above example, a dir named i686-w64-mingw32 will be
created. To use it for Bitcoin:
./configure --prefix=`pwd`/depends/i686-w64-mingw32
No other options are needed, the paths are automatically configured.
Dependency Options:
The following can be set when running make: make FOO=bar
SOURCES_PATH: downloaded sources will be placed here
BASE_CACHE: built packages will be placed here
SDK_PATH: Path where sdk's can be found (used by OSX)
FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up
NO_QT: Don't download/build/cache qt and its dependencies
NO_WALLET: Don't download/build/cache libs needed to enable the wallet
NO_UPNP: Don't download/build/cache packages needed for enabling upnp
DEBUG: disable some optimizations and enable more runtime checking
If some packages are not built, for example 'make NO_WALLET=1', the appropriate
options will be passed to bitcoin's configure. In this case, --disable-wallet.
Additional targets:
download: run 'make download' to fetch all sources without building them
download-osx: run 'make download-osx' to fetch all sources needed for osx builds
download-win: run 'make download-win' to fetch all sources needed for win builds
download-linux: run 'make download-linux' to fetch all sources needed for linux builds

View File

@ -1,9 +1,7 @@
This is a system of building and caching dependencies necessary for building
Bitcoin.
This is a system of building and caching dependencies necessary for building Bitcoin.
There are several features that make it different from most similar systems:
- It is designed to be builder and host agnostic
### It is designed to be builder and host agnostic
In theory, binaries for any target OS/architecture can be created, from a
builder running any OS/architecture. In practice, build-side tools must be
@ -11,18 +9,18 @@ specified when the defaults don't fit, and packages must be amended to work
on new hosts. For now, a build architecture of x86_64 is assumed, either on
Linux or OSX.
- No reliance on timestamps
### No reliance on timestamps
File presence is used to determine what needs to be built. This makes the
results distributable and easily digestable by automated builders.
- Each build only has its specified dependencies available at build-time.
### Each build only has its specified dependencies available at build-time.
For each build, the sysroot is wiped and the (recursive) dependencies are
installed. This makes each build deterministic, since there will never be any
unknown files available to cause side-effects.
- Each package is cached and only rebuilt as needed.
### Each package is cached and only rebuilt as needed.
Before building, a unique build-id is generated for each package. This id
consists of a hash of all files used to build the package (Makefiles, packages,
@ -32,7 +30,7 @@ any other package that depends on it. If any of the main makefiles (Makefile,
funcs.mk, etc) are changed, all packages will be rebuilt. After building, the
results are cached into a tarball that can be re-used and distributed.
- Package build results are (relatively) deterministic.
### Package build results are (relatively) deterministic.
Each package is configured and patched so that it will yield the same
build-results with each consequent build, within a reasonable set of
@ -41,13 +39,13 @@ beyond the scope of this system. Additionally, the toolchain itself must be
capable of deterministic results. When revisions are properly bumped, a cached
build should represent an exact single payload.
- Sources are fetched and verified automatically
### Sources are fetched and verified automatically
Each package must define its source location and checksum. The build will fail
if the fetched source does not match. Sources may be pre-seeded and/or cached
as desired.
- Self-cleaning
### Self-cleaning
Build and staging dirs are wiped after use, and any previous version of a
cached result is removed following a successful build. Automated builders

View File

@ -4,121 +4,137 @@ variables, and defining build commands.
The package "mylib" will be used here as an example
General tips:
mylib_foo is written as $(package)_foo in order to make recipes more similar.
- mylib_foo is written as $(package)_foo in order to make recipes more similar.
Identifiers:
## Identifiers
Each package is required to define at least these variables:
$(package)_version:
$(package)_version:
Version of the upstream library or program. If there is no version, a
placeholder such as 1.0 can be used.
$(package)_download_path:
$(package)_download_path:
Location of the upstream source, without the file-name. Usually http or
ftp.
$(package)_file_name:
$(package)_file_name:
The upstream source filename available at the download path.
$(package)_sha256_hash:
$(package)_sha256_hash:
The sha256 hash of the upstream file
These variables are optional:
$(package)_build_subdir:
$(package)_build_subdir:
cd to this dir before running configure/build/stage commands.
$(package)_download_file:
$(package)_download_file:
The file-name of the upstream source if it differs from how it should be
stored locally. This can be used to avoid storing file-names with strange
characters.
$(package)_dependencies:
$(package)_dependencies:
Names of any other packages that this one depends on.
$(package)_patches:
$(package)_patches:
Filenames of any patches needed to build the package
$(package)_extra_sources:
$(package)_extra_sources:
Any extra files that will be fetched via $(package)_fetch_cmds. These are
specified so that they can be fetched and verified via 'make download'.
Build Variables:
## Build Variables:
After defining the main identifiers, build variables may be added or customized
before running the build commands. They should be added to a function called
$(package)_set_vars. For example:
define $(package)_set_vars
...
endef
define $(package)_set_vars
...
endef
Most variables can be prefixed with the host, architecture, or both, to make
the modifications specific to that case. For example:
Universal: $(package)_cc=gcc
Linux only: $(package)_linux_cc=gcc
x86_64 only: $(package)_x86_64_cc = gcc
x86_64 linux only: $(package)_x86_64_linux_cc = gcc
Universal: $(package)_cc=gcc
Linux only: $(package)_linux_cc=gcc
x86_64 only: $(package)_x86_64_cc = gcc
x86_64 linux only: $(package)_x86_64_linux_cc = gcc
These variables may be set to override or append their default values.
$(package)_cc
$(package)_cxx
$(package)_objc
$(package)_objcxx
$(package)_ar
$(package)_ranlib
$(package)_libtool
$(package)_nm
$(package)_cflags
$(package)_cxxflags
$(package)_ldflags
$(package)_cppflags
$(package)_config_env
$(package)_build_env
$(package)_stage_env
$(package)_build_opts
$(package)_config_opts
$(package)_cc
$(package)_cxx
$(package)_objc
$(package)_objcxx
$(package)_ar
$(package)_ranlib
$(package)_libtool
$(package)_nm
$(package)_cflags
$(package)_cxxflags
$(package)_ldflags
$(package)_cppflags
$(package)_config_env
$(package)_build_env
$(package)_stage_env
$(package)_build_opts
$(package)_config_opts
The *_env variables are used to add environment variables to the respective
commands.
Many variables respect a debug/release suffix as well, in order to use them for
only the appropriate build config. For example:
$(package)_cflags_release = -O3
$(package)_cflags_i686_debug = -g
$(package)_config_opts_release = --disable-debug
$(package)_cflags_release = -O3
$(package)_cflags_i686_debug = -g
$(package)_config_opts_release = --disable-debug
These will be used in addition to the options that do not specify
debug/release. All builds are considered to be release unless DEBUG=1 is set by
the user.
the user. Other variables may be defined as needed.
Other variables may be defined as needed.
Build commands:
## Build commands:
For each build, a unique build dir and staging dir are created. For example,
work/build/mylib/1.0-1adac830f6e and work/staging/mylib/1.0-1adac830f6e.
`work/build/mylib/1.0-1adac830f6e` and `work/staging/mylib/1.0-1adac830f6e`.
The following build commands are available for each recipe:
$(package)_fetch_cmds:
$(package)_fetch_cmds:
Runs from: build dir
Fetch the source file. If undefined, it will be fetched and verified
against its hash.
$(package)_extract_cmds:
$(package)_extract_cmds:
Runs from: build dir
Verify the source file against its hash and extract it. If undefined, the
source is assumed to be a tarball.
$(package)_preprocess_cmds:
$(package)_preprocess_cmds:
Runs from: build dir/$(package)_build_subdir
Preprocess the source as necessary. If undefined, does nothing.
$(package)_config_cmds:
$(package)_config_cmds:
Runs from: build dir/$(package)_build_subdir
Configure the source. If undefined, does nothing.
$(package)_build_cmds:
$(package)_build_cmds:
Runs from: build dir/$(package)_build_subdir
Build the source. If undefined, does nothing.
$(package)_stage_cmds:
$(package)_stage_cmds:
Runs from: build dir/$(package)_build_subdir
Stage the build results. If undefined, does nothing.
The following variables are available for each recipe:
$(1)_staging_dir: package's destination sysroot path
$(1)_staging_prefix_dir: prefix path inside of the package's staging dir
$(1)_extract_dir: path to the package's extracted sources
$(1)_build_dir: path where configure/build/stage commands will be run
$(1)_patch_dir: path where the package's patches (if any) are found
$(1)_staging_dir: package's destination sysroot path
$(1)_staging_prefix_dir: prefix path inside of the package's staging dir
$(1)_extract_dir: path to the package's extracted sources
$(1)_build_dir: path where configure/build/stage commands will be run
$(1)_patch_dir: path where the package's patches (if any) are found
Notes on build commands:
@ -127,4 +143,5 @@ configure step to (usually) correctly configure automatically. Any
$($(package)_config_opts) will be appended.
Most autotools projects can be properly staged using:
$(MAKE) DESTDIR=$($(package)_staging_dir) install
$(MAKE) DESTDIR=$($(package)_staging_dir) install