dash/contrib/guix

Bootstrappable Bitcoin Core Builds

This directory contains the files necessary to perform bootstrappable Bitcoin Core builds.

Bootstrappability furthers our binary security guarantees by allowing us to audit and reproduce our toolchain instead of blindly trusting binary downloads.

We achieve bootstrappability by using Guix as a functional package manager.

Requirements

Conservatively, a x86_64 machine with:

• 16GB of free disk space on the partition that /gnu/store will reside in
• 8GB of free disk space per platform triple you're planning on building (see the HOSTS environment variable description)

Setup

Installing Guix

If you're just testing this out, you can use the Dockerfile for convenience. It automatically speeds up your builds by using substitutes. If you don't want this behaviour, refer to the next section.

Otherwise, follow the Guix installation guide.

Note: For those who like to keep their filesystems clean, Guix is designed to be very standalone and will not conflict with your system's package manager/existing setup. It only touches /var/guix, /gnu, and ~/.config/guix.

Choosing your security model

Guix allows us to achieve better binary security by using our CPU time to build everything from scratch. However, it doesn't sacrifice user choice in pursuit of this: users can decide whether or not to bootstrap and to use substitutes (pre-built packages).

After installation, you may want to consider adding substitute servers from which to download pre-built packages to speed up your build if that fits your security model (say, if you're just testing that this works). Substitute servers are set up by default if you're using the Dockerfile.

If you prefer not to use any substitutes, make sure to supply --no-substitutes like in the following snippet. The first build will take a while, but the resulting packages will be cached for future builds.

export ADDITIONAL_GUIX_COMMON_FLAGS='--no-substitutes'

Likewise, to perform a bootstrapped build (takes even longer):

export ADDITIONAL_GUIX_COMMON_FLAGS='--no-substitutes' ADDITIONAL_GUIX_ENVIRONMENT_FLAGS='--bootstrap'

Using a version of Guix with guix time-machine capabilities

Note: This entire section can be skipped if you are already using a version of Guix that has the guix time-machine command.

Once Guix is installed, if it doesn't have the guix time-machine command, pull the latest guix.

guix pull --max-jobs=4 # change number of jobs accordingly

Make sure that you are using your current profile. (You are prompted to do this at the end of the guix pull)

export PATH="${HOME}/.config/guix/current/bin${PATH:+:}$PATH"

Usage

As a Tool for Deterministic Builds

From the top of a clean Bitcoin Core repository:

./contrib/guix/guix-build.sh

After the build finishes successfully (check the status code please), compare hashes:

find output/ -type f -print0 | sort -z | xargs -r0 sha256sum

Recognized environment variables

• HOSTS

  Override the space-separated list of platform triples for which to perform a bootstrappable build. (defaults to "x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu x86_64-w64-mingw32 x86_64-apple-darwin18")

• SOURCES_PATH

  Set the depends tree download cache for sources. This is passed through to the depends tree. Setting this to the same directory across multiple builds of the depends tree can eliminate unnecessary redownloading of package sources.

• BASE_CACHE

  Set the depends tree cache for built packages. This is passed through to the depends tree. Setting this to the same directory across multiple builds of the depends tree can eliminate unnecessary building of packages.

• SDK_PATH

  Set the path where extracted SDKs can be found. This is passed through to the depends tree. Note that this is should be set to the parent directory of the actual SDK (e.g. SDK_PATH=$HOME/Downloads/macOS-SDKs instead of $HOME/Downloads/macOS-SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers).

• MAX_JOBS

  Override the maximum number of jobs to run simultaneously, you might want to do so on a memory-limited machine. This may be passed to make as in make --jobs="$MAX_JOBS" or xargs as in xargs -P"$MAX_JOBS". (defaults to the value of nproc outside the container)

• SOURCE_DATE_EPOCH

  Override the reference UNIX timestamp used for bit-for-bit reproducibility, the variable name conforms to standard. (defaults to the output of $(git log --format=%at -1))

• V

  If non-empty, will pass V=1 to all make invocations, making make output verbose.

  Note that any given value is ignored. The variable is only checked for emptiness. More concretely, this means that V= (setting V to the empty string) is interpreted the same way as not setting V at all, and that V=0 has the same effect as V=1.

• SUBSTITUTE_URLS

  A whitespace-delimited list of URLs from which to download pre-built packages. A URL is only used if its signing key is authorized (refer to the substitute servers section for more details).

• ADDITIONAL_GUIX_COMMON_FLAGS

  Additional flags to be passed to all guix commands. For a fully-bootstrapped build, set this to --bootstrap --no-substitutes (refer to the security model section for more details). Note that a fully-bootstrapped build will take quite a long time on the first run.

• ADDITIONAL_GUIX_TIMEMACHINE_FLAGS

  Additional flags to be passed to guix time-machine.

• ADDITIONAL_GUIX_ENVIRONMENT_FLAGS

  Additional flags to be passed to the invocation of guix environment inside guix time-machine.

Tips and Tricks

Speeding up builds with substitute servers

This whole section is automatically done in the convenience Dockerfiles

For those who are used to life in the fast (and trustful) lane, you can specify substitute servers from which to download pre-built packages.

For those who only want to use substitutes from the official Guix build farm and have authorized the build farm's signing key during Guix's installation, you don't need to do anything.

Step 1: Authorize the signing keys

For the official Guix build farm at https://ci.guix.gnu.org, run as root:

guix archive --authorize < ~root/.config/guix/current/share/guix/ci.guix.gnu.org.pub

For dongcarl's substitute server at https://guix.carldong.io, run as root:

wget -qO- 'https://guix.carldong.io/signing-key.pub' | guix archive --authorize

Step 2: Specify the substitute servers

The official Guix build farm at https://ci.guix.gnu.org is automatically used unless the --no-substitutes flag is supplied.

This can be overridden for all guix invocations by passing the --substitute-urls option to your invocation of guix-daemon. This can also be overridden on a call-by-call basis by passing the same --substitute-urls option to client tools such at guix environment.

To use dongcarl's substitute server for Bitcoin Core builds after having authorized his signing key:

export SUBSTITUTE_URLS='https://guix.carldong.io https://ci.guix.gnu.org'

FAQ

How can I trust the binary installation?

As mentioned at the bottom of this manual page:

The binary installation tarballs can be (re)produced and verified simply by running the following command in the Guix source tree:

make guix-binary.x86_64-linux.tar.xz

Is Guix packaged in my operating system?

Guix is shipped starting with Debian Bullseye and Ubuntu 21.04 "Hirsute Hippo". Other operating systems are working on packaging Guix as well.