mirror of
https://github.com/dashpay/dash.git
synced 2024-12-25 12:02:48 +01:00
Merge pull request #5194 from knst/guix
contrib: Enable building in Guix containers
This commit is contained in:
PastaPastaPasta
GitHub
commit
eaf1fa2394
@ -46,7 +46,7 @@ script: |
|
||||
FAKETIME_PROGS="date ar ranlib nm"
|
||||
HOST_CFLAGS="-O2 -g"
|
||||
HOST_CXXFLAGS="-O2 -g"
|
||||
HOST_LDFLAGS_BASE="-static-libstdc++"
|
||||
HOST_LDFLAGS_BASE="-static-libstdc++ -Wl,-O2"
|
||||
|
||||
export TZ="UTC"
|
||||
export BUILD_DIR="$PWD"
|
||||
|
||||
225
contrib/guix/README.md
Normal file
225
contrib/guix/README.md
Normal file
@ -0,0 +1,225 @@
|
||||
# Bootstrappable Bitcoin Core Builds
|
||||
|
||||
This directory contains the files necessary to perform bootstrappable Bitcoin
|
||||
Core builds.
|
||||
|
||||
[Bootstrappability][b17e] furthers our binary security guarantees by allowing us
|
||||
to _audit and reproduce_ our toolchain instead of blindly _trusting_ binary
|
||||
downloads.
|
||||
|
||||
We achieve bootstrappability by using Guix as a functional package manager.
|
||||
|
||||
## Requirements
|
||||
|
||||
Conservatively, a x86_64 machine with:
|
||||
|
||||
- 4GB of free disk space on the partition that /gnu/store will reside in
|
||||
- 24GB of free disk space on the partition that the Bitcoin Core git repository
|
||||
resides in
|
||||
|
||||
> Note: these requirements are slightly less onerous than those of Gitian builds
|
||||
|
||||
## Setup
|
||||
|
||||
### Installing Guix
|
||||
|
||||
If you're just testing this out, you can use the
|
||||
[Dockerfile][fanquake/guix-docker] for convenience. It automatically speeds up
|
||||
your builds by [using substitutes](#speeding-up-builds-with-substitute-servers).
|
||||
If you don't want this behaviour, refer to the [next
|
||||
section](#choosing-your-security-model).
|
||||
|
||||
Otherwise, follow the [Guix installation guide][guix/bin-install].
|
||||
|
||||
> Note: For those who like to keep their filesystems clean, Guix is designed to
|
||||
> be very standalone and _will not_ conflict with your system's package
|
||||
> manager/existing setup. It _only_ touches `/var/guix`, `/gnu`, and
|
||||
> `~/.config/guix`.
|
||||
|
||||
### Choosing your security model
|
||||
|
||||
Guix allows us to achieve better binary security by using our CPU time to build
|
||||
everything from scratch. However, it doesn't sacrifice user choice in pursuit of
|
||||
this: users can decide whether or not to bootstrap and to use substitutes.
|
||||
|
||||
After installation, you may want to consider [adding substitute
|
||||
servers](#speeding-up-builds-with-substitute-servers) to speed up your build if
|
||||
that fits your security model (say, if you're just testing that this works).
|
||||
This is skippable if you're using the [Dockerfile][fanquake/guix-docker].
|
||||
|
||||
If you prefer not to use any substitutes, make sure to set
|
||||
`ADDITIONAL_GUIX_ENVIRONMENT_FLAGS` like the following snippet. The first build
|
||||
will take a while, but the resulting packages will be cached for future builds.
|
||||
|
||||
```sh
|
||||
export ADDITIONAL_GUIX_ENVIRONMENT_FLAGS='--no-substitutes'
|
||||
```
|
||||
|
||||
Likewise, to perform a bootstrapped build (takes even longer):
|
||||
|
||||
```sh
|
||||
export ADDITIONAL_GUIX_ENVIRONMENT_FLAGS='--bootstrap --no-substitutes'
|
||||
```
|
||||
|
||||
### Using a version of Guix with `guix time-machine` capabilities
|
||||
|
||||
> Note: This entire section can be skipped if you are already using a version of
|
||||
> Guix that has [the `guix time-machine` command][guix/time-machine].
|
||||
|
||||
Once Guix is installed, if it doesn't have the `guix time-machine` command, pull
|
||||
the latest `guix`.
|
||||
|
||||
```sh
|
||||
guix pull --max-jobs=4 # change number of jobs accordingly
|
||||
```
|
||||
|
||||
Make sure that you are using your current profile. (You are prompted to do this
|
||||
at the end of the `guix pull`)
|
||||
|
||||
```bash
|
||||
export PATH="${HOME}/.config/guix/current/bin${PATH:+:}$PATH"
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### As a Development Environment
|
||||
|
||||
For a Bitcoin Core depends development environment, simply invoke
|
||||
|
||||
```sh
|
||||
guix environment --manifest=contrib/guix/manifest.scm
|
||||
```
|
||||
|
||||
And you'll land back in your shell with all the build dependencies required for
|
||||
a `depends` build injected into your environment.
|
||||
|
||||
### As a Tool for Deterministic Builds
|
||||
|
||||
From the top of a clean Bitcoin Core repository:
|
||||
|
||||
```sh
|
||||
./contrib/guix/guix-build.sh
|
||||
```
|
||||
|
||||
After the build finishes successfully (check the status code please), compare
|
||||
hashes:
|
||||
|
||||
```sh
|
||||
find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
|
||||
```
|
||||
|
||||
#### Recognized environment variables
|
||||
|
||||
* _**HOSTS**_
|
||||
|
||||
Override the space-separated list of platform triples for which to perform a
|
||||
bootstrappable build. _(defaults to "x86\_64-linux-gnu
|
||||
arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu")_
|
||||
|
||||
> Windows and OS X platform triplet support are WIP.
|
||||
|
||||
* _**SOURCES_PATH**_
|
||||
|
||||
Set the depends tree download cache for sources. This is passed through to the
|
||||
depends tree. Setting this to the same directory across multiple builds of the
|
||||
depends tree can eliminate unnecessary redownloading of package sources.
|
||||
|
||||
* _**MAX_JOBS**_
|
||||
|
||||
Override the maximum number of jobs to run simultaneously, you might want to
|
||||
do so on a memory-limited machine. This may be passed to `make` as in `make
|
||||
--jobs="$MAX_JOBS"` or `xargs` as in `xargs -P"$MAX_JOBS"`. _(defaults to the
|
||||
value of `nproc` outside the container)_
|
||||
|
||||
* _**SOURCE_DATE_EPOCH**_
|
||||
|
||||
Override the reference UNIX timestamp used for bit-for-bit reproducibility,
|
||||
the variable name conforms to [standard][r12e/source-date-epoch]. _(defaults
|
||||
to the output of `$(git log --format=%at -1)`)_
|
||||
|
||||
* _**V**_
|
||||
|
||||
If non-empty, will pass `V=1` to all `make` invocations, making `make` output
|
||||
verbose.
|
||||
|
||||
* _**ADDITIONAL_GUIX_ENVIRONMENT_FLAGS**_
|
||||
|
||||
Additional flags to be passed to `guix environment`. For a fully-bootstrapped
|
||||
build, set this to `--bootstrap --no-substitutes` (refer to the [security
|
||||
model section](#choosing-your-security-model) for more details). Note that a
|
||||
fully-bootstrapped build will take quite a long time on the first run.
|
||||
|
||||
## Tips and Tricks
|
||||
|
||||
### Speeding up builds with substitute servers
|
||||
|
||||
_This whole section is automatically done in the convenience
|
||||
[Dockerfiles][fanquake/guix-docker]_
|
||||
|
||||
For those who are used to life in the fast _(and trustful)_ lane, you can use
|
||||
[substitute servers][guix/substitutes] to enable binary downloads of packages.
|
||||
|
||||
> For those who only want to use substitutes from the official Guix build farm
|
||||
> and have authorized the build farm's signing key during Guix's installation,
|
||||
> you don't need to do anything.
|
||||
|
||||
#### Authorize the signing keys
|
||||
|
||||
For the official Guix build farm at https://ci.guix.gnu.org, run as root:
|
||||
|
||||
```
|
||||
guix archive --authorize < ~root/.config/guix/current/share/guix/ci.guix.gnu.org.pub
|
||||
```
|
||||
|
||||
For dongcarl's substitute server at https://guix.carldong.io, run as root:
|
||||
|
||||
```sh
|
||||
wget -qO- 'https://guix.carldong.io/signing-key.pub' | guix archive --authorize
|
||||
```
|
||||
|
||||
#### Use the substitute servers
|
||||
|
||||
The official Guix build farm at https://ci.guix.gnu.org is automatically used
|
||||
unless the `--no-substitutes` flag is supplied.
|
||||
|
||||
This can be overridden for all `guix` invocations by passing the
|
||||
`--substitute-urls` option to your invocation of `guix-daemon`. This can also be
|
||||
overridden on a call-by-call basis by passing the same `--substitute-urls`
|
||||
option to client tools such at `guix environment`.
|
||||
|
||||
To use dongcarl's substitute server for Bitcoin Core builds after having
|
||||
[authorized his signing key](#authorize-the-signing-keys):
|
||||
|
||||
```
|
||||
export ADDITIONAL_GUIX_ENVIRONMENT_FLAGS='--substitute-urls="https://guix.carldong.io https://ci.guix.gnu.org"'
|
||||
```
|
||||
|
||||
## FAQ
|
||||
|
||||
### How can I trust the binary installation?
|
||||
|
||||
As mentioned at the bottom of [this manual page][guix/bin-install]:
|
||||
|
||||
> The binary installation tarballs can be (re)produced and verified simply by
|
||||
> running the following command in the Guix source tree:
|
||||
>
|
||||
> make guix-binary.x86_64-linux.tar.xz
|
||||
|
||||
### When will Guix be packaged in debian?
|
||||
|
||||
Vagrant Cascadian has been making good progress on this
|
||||
[here][debian/guix-package]. We have all the pieces needed to put up an APT
|
||||
repository and will likely put one up soon.
|
||||
|
||||
[b17e]: http://bootstrappable.org/
|
||||
[r12e/source-date-epoch]: https://reproducible-builds.org/docs/source-date-epoch/
|
||||
|
||||
[guix/install.sh]: https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
|
||||
[guix/bin-install]: https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html
|
||||
[guix/env-setup]: https://www.gnu.org/software/guix/manual/en/html_node/Build-Environment-Setup.html
|
||||
[guix/substitutes]: https://www.gnu.org/software/guix/manual/en/html_node/Substitutes.html
|
||||
[guix/substitute-server-auth]: https://www.gnu.org/software/guix/manual/en/html_node/Substitute-Server-Authorization.html
|
||||
[guix/time-machine]: https://guix.gnu.org/manual/en/html_node/Invoking-guix-time_002dmachine.html
|
||||
|
||||
[debian/guix-package]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850644
|
||||
[fanquake/guix-docker]: https://github.com/fanquake/core-review/tree/master/guix
|
||||
119
contrib/guix/guix-build.sh
Executable file
119
contrib/guix/guix-build.sh
Executable file
@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env bash
|
||||
export LC_ALL=C
|
||||
set -e -o pipefail
|
||||
|
||||
# Determine the maximum number of jobs to run simultaneously (overridable by
|
||||
# environment)
|
||||
MAX_JOBS="${MAX_JOBS:-$(nproc)}"
|
||||
|
||||
# Download the depends sources now as we won't have internet access in the build
|
||||
# container
|
||||
make -C "${PWD}/depends" -j"$MAX_JOBS" download ${V:+V=1} ${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"}
|
||||
|
||||
# Determine the reference time used for determinism (overridable by environment)
|
||||
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:-$(git log --format=%at -1)}"
|
||||
|
||||
# Execute "$@" in a pinned, possibly older version of Guix, for reproducibility
|
||||
# across time.
|
||||
time-machine() {
|
||||
guix time-machine --url=https://github.com/dongcarl/guix.git \
|
||||
--commit=b066c25026f21fb57677aa34692a5034338e7ee3 \
|
||||
-- "$@"
|
||||
}
|
||||
|
||||
# Function to be called when building for host ${1} and the user interrupts the
|
||||
# build
|
||||
int_trap() {
|
||||
cat << EOF
|
||||
** INT received while building ${1}, you may want to clean up the relevant
|
||||
output, deploy, and distsrc-* directories before rebuilding
|
||||
|
||||
Hint: To blow everything away, you may want to use:
|
||||
|
||||
$ git clean -xdff --exclude='/depends/SDKs/*'
|
||||
|
||||
Specifically, this will remove all files without an entry in the index,
|
||||
excluding the SDK directory. Practically speaking, this means that all ignored
|
||||
and untracked files and directories will be wiped, allowing you to start anew.
|
||||
EOF
|
||||
}
|
||||
|
||||
# Deterministically build Bitcoin Core for HOSTs (overridable by environment)
|
||||
# shellcheck disable=SC2153
|
||||
for host in ${HOSTS=x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu x86_64-w64-mingw32}; do
|
||||
|
||||
# Display proper warning when the user interrupts the build
|
||||
trap 'int_trap ${host}' INT
|
||||
|
||||
(
|
||||
# Required for 'contrib/guix/manifest.scm' to output the right manifest
|
||||
# for the particular $HOST we're building for
|
||||
export HOST="$host"
|
||||
|
||||
# Run the build script 'contrib/guix/libexec/build.sh' in the build
|
||||
# container specified by 'contrib/guix/manifest.scm'.
|
||||
#
|
||||
# Explanation of `guix environment` flags:
|
||||
#
|
||||
# --container run command within an isolated container
|
||||
#
|
||||
# Running in an isolated container minimizes build-time differences
|
||||
# between machines and improves reproducibility
|
||||
#
|
||||
# --pure unset existing environment variables
|
||||
#
|
||||
# Same rationale as --container
|
||||
#
|
||||
# --no-cwd do not share current working directory with an
|
||||
# isolated container
|
||||
#
|
||||
# When --container is specified, the default behavior is to share
|
||||
# the current working directory with the isolated container at the
|
||||
# same exact path (e.g. mapping '/home/satoshi/bitcoin/' to
|
||||
# '/home/satoshi/bitcoin/'). This means that the $PWD inside the
|
||||
# container becomes a source of irreproducibility. --no-cwd disables
|
||||
# this behaviour.
|
||||
#
|
||||
# --share=SPEC for containers, share writable host file system
|
||||
# according to SPEC
|
||||
#
|
||||
# --share="$PWD"=/bitcoin
|
||||
#
|
||||
# maps our current working directory to /bitcoin
|
||||
# inside the isolated container, which we later cd
|
||||
# into.
|
||||
#
|
||||
# While we don't want to map our current working directory to the
|
||||
# same exact path (as this introduces irreproducibility), we do want
|
||||
# it to be at a _fixed_ path _somewhere_ inside the isolated
|
||||
# container so that we have something to build. '/bitcoin' was
|
||||
# chosen arbitrarily.
|
||||
#
|
||||
# ${SOURCES_PATH:+--share="$SOURCES_PATH"}
|
||||
#
|
||||
# make the downloaded depends sources path available
|
||||
# inside the isolated container
|
||||
#
|
||||
# The isolated container has no network access as it's in a
|
||||
# different network namespace from the main machine, so we have to
|
||||
# make the downloaded depends sources available to it. The sources
|
||||
# should have been downloaded prior to this invocation.
|
||||
#
|
||||
# shellcheck disable=SC2086
|
||||
time-machine environment --manifest="${PWD}/contrib/guix/manifest.scm" \
|
||||
--container \
|
||||
--pure \
|
||||
--no-cwd \
|
||||
--share="$PWD"=/bitcoin \
|
||||
--expose="$(git rev-parse --git-common-dir)" \
|
||||
${SOURCES_PATH:+--share="$SOURCES_PATH"} \
|
||||
${ADDITIONAL_GUIX_ENVIRONMENT_FLAGS} \
|
||||
-- env HOST="$host" \
|
||||
MAX_JOBS="$MAX_JOBS" \
|
||||
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:?unable to determine value}" \
|
||||
${V:+V=1} \
|
||||
${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"} \
|
||||
bash -c "cd /bitcoin && bash contrib/guix/libexec/build.sh"
|
||||
)
|
||||
|
||||
done
|
||||
314
contrib/guix/libexec/build.sh
Normal file
314
contrib/guix/libexec/build.sh
Normal file
@ -0,0 +1,314 @@
|
||||
#!/usr/bin/env bash
|
||||
export LC_ALL=C
|
||||
set -e -o pipefail
|
||||
export TZ=UTC
|
||||
|
||||
# Check that environment variables assumed to be set by the environment are set
|
||||
echo "Building for platform triple ${HOST:?not set} with reference timestamp ${SOURCE_DATE_EPOCH:?not set}..."
|
||||
echo "At most ${MAX_JOBS:?not set} jobs will run at once..."
|
||||
|
||||
#####################
|
||||
# Environment Setup #
|
||||
#####################
|
||||
|
||||
# The depends folder also serves as a base-prefix for depends packages for
|
||||
# $HOSTs after successfully building.
|
||||
BASEPREFIX="${PWD}/depends"
|
||||
|
||||
# Setup an output directory for our build
|
||||
OUTDIR="${OUTDIR:-${PWD}/output}"
|
||||
[ -e "$OUTDIR" ] || mkdir -p "$OUTDIR"
|
||||
|
||||
# Setup the directory where our Bitcoin Core build for HOST will occur
|
||||
DISTSRC="${DISTSRC:-${PWD}/distsrc-${HOST}}"
|
||||
if [ -e "$DISTSRC" ]; then
|
||||
echo "DISTSRC directory '${DISTSRC}' exists, probably because of previous builds... Aborting..."
|
||||
exit 1
|
||||
else
|
||||
mkdir -p "$DISTSRC"
|
||||
fi
|
||||
|
||||
# Given a package name and an output name, return the path of that output in our
|
||||
# current guix environment
|
||||
store_path() {
|
||||
grep --extended-regexp "/[^-]{32}-${1}-[^-]+${2:+-${2}}" "${GUIX_ENVIRONMENT}/manifest" \
|
||||
| head --lines=1 \
|
||||
| sed --expression='s|^[[:space:]]*"||' \
|
||||
--expression='s|"[[:space:]]*$||'
|
||||
}
|
||||
|
||||
# Set environment variables to point Guix's cross-toolchain to the right
|
||||
# includes/libs for $HOST
|
||||
case "$HOST" in
|
||||
*mingw*)
|
||||
# Determine output paths to use in CROSS_* environment variables
|
||||
CROSS_GLIBC="$(store_path "mingw-w64-x86_64-winpthreads")"
|
||||
CROSS_GCC="$(store_path "gcc-cross-${HOST}")"
|
||||
CROSS_GCC_LIBS=( "${CROSS_GCC}/lib/gcc/${HOST}"/* ) # This expands to an array of directories...
|
||||
CROSS_GCC_LIB="${CROSS_GCC_LIBS[0]}" # ...we just want the first one (there should only be one)
|
||||
|
||||
NATIVE_GCC="$(store_path gcc-glibc-2.27-toolchain)"
|
||||
export LIBRARY_PATH="${NATIVE_GCC}/lib:${NATIVE_GCC}/lib64"
|
||||
export CPATH="${NATIVE_GCC}/include"
|
||||
|
||||
export CROSS_C_INCLUDE_PATH="${CROSS_GCC_LIB}/include:${CROSS_GCC_LIB}/include-fixed:${CROSS_GLIBC}/include"
|
||||
export CROSS_CPLUS_INCLUDE_PATH="${CROSS_GCC}/include/c++:${CROSS_GCC}/include/c++/${HOST}:${CROSS_GCC}/include/c++/backward:${CROSS_C_INCLUDE_PATH}"
|
||||
export CROSS_LIBRARY_PATH="${CROSS_GCC}/lib:${CROSS_GCC}/${HOST}/lib:${CROSS_GCC_LIB}:${CROSS_GLIBC}/lib"
|
||||
;;
|
||||
*linux*)
|
||||
CROSS_GLIBC="$(store_path "glibc-cross-${HOST}")"
|
||||
CROSS_GLIBC_STATIC="$(store_path "glibc-cross-${HOST}" static)"
|
||||
CROSS_KERNEL="$(store_path "linux-libre-headers-cross-${HOST}")"
|
||||
CROSS_GCC="$(store_path "gcc-cross-${HOST}")"
|
||||
CROSS_GCC_LIBS=( "${CROSS_GCC}/lib/gcc/${HOST}"/* ) # This expands to an array of directories...
|
||||
CROSS_GCC_LIB="${CROSS_GCC_LIBS[0]}" # ...we just want the first one (there should only be one)
|
||||
|
||||
# NOTE: CROSS_C_INCLUDE_PATH is missing ${CROSS_GCC_LIB}/include-fixed, because
|
||||
# the limits.h in it is missing a '#include_next <limits.h>'
|
||||
export CROSS_C_INCLUDE_PATH="${CROSS_GCC_LIB}/include:${CROSS_GLIBC}/include:${CROSS_KERNEL}/include"
|
||||
export CROSS_CPLUS_INCLUDE_PATH="${CROSS_GCC}/include/c++:${CROSS_GCC}/include/c++/${HOST}:${CROSS_GCC}/include/c++/backward:${CROSS_C_INCLUDE_PATH}"
|
||||
export CROSS_LIBRARY_PATH="${CROSS_GCC}/lib:${CROSS_GCC}/${HOST}/lib:${CROSS_GCC_LIB}:${CROSS_GLIBC}/lib:${CROSS_GLIBC_STATIC}/lib"
|
||||
;;
|
||||
*)
|
||||
exit 1 ;;
|
||||
esac
|
||||
|
||||
# Sanity check CROSS_*_PATH directories
|
||||
IFS=':' read -ra PATHS <<< "${CROSS_C_INCLUDE_PATH}:${CROSS_CPLUS_INCLUDE_PATH}:${CROSS_LIBRARY_PATH}"
|
||||
for p in "${PATHS[@]}"; do
|
||||
if [ ! -d "$p" ]; then
|
||||
echo "'$p' doesn't exist or isn't a directory... Aborting..."
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
# Disable Guix ld auto-rpath behavior
|
||||
export GUIX_LD_WRAPPER_DISABLE_RPATH=yes
|
||||
|
||||
# Make /usr/bin if it doesn't exist
|
||||
[ -e /usr/bin ] || mkdir -p /usr/bin
|
||||
|
||||
# Symlink file and env to a conventional path
|
||||
[ -e /usr/bin/file ] || ln -s --no-dereference "$(command -v file)" /usr/bin/file
|
||||
[ -e /usr/bin/env ] || ln -s --no-dereference "$(command -v env)" /usr/bin/env
|
||||
|
||||
# Determine the correct value for -Wl,--dynamic-linker for the current $HOST
|
||||
case "$HOST" in
|
||||
*linux*)
|
||||
glibc_dynamic_linker=$(
|
||||
case "$HOST" in
|
||||
i686-linux-gnu) echo /lib/ld-linux.so.2 ;;
|
||||
x86_64-linux-gnu) echo /lib64/ld-linux-x86-64.so.2 ;;
|
||||
arm-linux-gnueabihf) echo /lib/ld-linux-armhf.so.3 ;;
|
||||
aarch64-linux-gnu) echo /lib/ld-linux-aarch64.so.1 ;;
|
||||
riscv64-linux-gnu) echo /lib/ld-linux-riscv64-lp64d.so.1 ;;
|
||||
*) exit 1 ;;
|
||||
esac
|
||||
)
|
||||
;;
|
||||
esac
|
||||
|
||||
# Environment variables for determinism
|
||||
export QT_RCC_TEST=1
|
||||
export QT_RCC_SOURCE_DATE_OVERRIDE=1
|
||||
export TAR_OPTIONS="--owner=0 --group=0 --numeric-owner --mtime='@${SOURCE_DATE_EPOCH}' --sort=name"
|
||||
export TZ="UTC"
|
||||
|
||||
####################
|
||||
# Depends Building #
|
||||
####################
|
||||
|
||||
# Build the depends tree, overriding variables that assume multilib gcc
|
||||
make -C depends --jobs="$MAX_JOBS" HOST="$HOST" \
|
||||
${V:+V=1} \
|
||||
${SOURCES_PATH+SOURCES_PATH="$SOURCES_PATH"} \
|
||||
i686_linux_CC=i686-linux-gnu-gcc \
|
||||
i686_linux_CXX=i686-linux-gnu-g++ \
|
||||
i686_linux_AR=i686-linux-gnu-ar \
|
||||
i686_linux_RANLIB=i686-linux-gnu-ranlib \
|
||||
i686_linux_NM=i686-linux-gnu-nm \
|
||||
i686_linux_STRIP=i686-linux-gnu-strip \
|
||||
x86_64_linux_CC=x86_64-linux-gnu-gcc \
|
||||
x86_64_linux_CXX=x86_64-linux-gnu-g++ \
|
||||
x86_64_linux_AR=x86_64-linux-gnu-ar \
|
||||
x86_64_linux_RANLIB=x86_64-linux-gnu-ranlib \
|
||||
x86_64_linux_NM=x86_64-linux-gnu-nm \
|
||||
x86_64_linux_STRIP=x86_64-linux-gnu-strip \
|
||||
qt_config_opts_i686_linux='-platform linux-g++ -xplatform bitcoin-linux-g++'
|
||||
|
||||
|
||||
###########################
|
||||
# Source Tarball Building #
|
||||
###########################
|
||||
|
||||
# Define DISTNAME variable.
|
||||
# shellcheck source=contrib/gitian-descriptors/assign_DISTNAME
|
||||
source contrib/gitian-descriptors/assign_DISTNAME
|
||||
|
||||
GIT_ARCHIVE="${OUTDIR}/src/${DISTNAME}.tar.gz"
|
||||
|
||||
# Create the source tarball if not already there
|
||||
if [ ! -e "$GIT_ARCHIVE" ]; then
|
||||
mkdir -p "$(dirname "$GIT_ARCHIVE")"
|
||||
git archive --output="$GIT_ARCHIVE" HEAD
|
||||
fi
|
||||
|
||||
###########################
|
||||
# Binary Tarball Building #
|
||||
###########################
|
||||
|
||||
# CONFIGFLAGS
|
||||
CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests"
|
||||
case "$HOST" in
|
||||
*linux*) CONFIGFLAGS+=" --enable-glibc-back-compat" ;;
|
||||
esac
|
||||
|
||||
# CFLAGS
|
||||
HOST_CFLAGS="-O2 -g"
|
||||
case "$HOST" in
|
||||
*linux*) HOST_CFLAGS+=" -ffile-prefix-map=${PWD}=." ;;
|
||||
*mingw*) HOST_CFLAGS+=" -fno-ident" ;;
|
||||
esac
|
||||
|
||||
# CXXFLAGS
|
||||
HOST_CXXFLAGS="$HOST_CFLAGS"
|
||||
|
||||
# LDFLAGS
|
||||
case "$HOST" in
|
||||
*linux*) HOST_LDFLAGS="-Wl,--as-needed -Wl,--dynamic-linker=$glibc_dynamic_linker -static-libstdc++ -Wl,-O2" ;;
|
||||
*mingw*) HOST_LDFLAGS="-Wl,--no-insert-timestamp" ;;
|
||||
esac
|
||||
|
||||
# Make $HOST-specific native binaries from depends available in $PATH
|
||||
export PATH="${BASEPREFIX}/${HOST}/native/bin:${PATH}"
|
||||
(
|
||||
cd "$DISTSRC"
|
||||
|
||||
# Extract the source tarball
|
||||
tar -xf "${GIT_ARCHIVE}"
|
||||
|
||||
./autogen.sh
|
||||
|
||||
# Configure this DISTSRC for $HOST
|
||||
# shellcheck disable=SC2086
|
||||
env CONFIG_SITE="${BASEPREFIX}/${HOST}/share/config.site" \
|
||||
./configure --prefix=/ \
|
||||
--disable-ccache \
|
||||
--disable-maintainer-mode \
|
||||
--disable-dependency-tracking \
|
||||
${CONFIGFLAGS} \
|
||||
CFLAGS="${HOST_CFLAGS}" \
|
||||
CXXFLAGS="${HOST_CXXFLAGS}" \
|
||||
${HOST_LDFLAGS:+LDFLAGS="${HOST_LDFLAGS}"}
|
||||
|
||||
sed -i.old 's/-lstdc++ //g' {./,src/dashbls/,src/secp256k1/}{config.status,libtool}
|
||||
|
||||
|
||||
# Build Bitcoin Core
|
||||
make --jobs="$MAX_JOBS" ${V:+V=1}
|
||||
|
||||
# Perform basic ELF security checks on a series of executables.
|
||||
make -C src --jobs=1 check-security ${V:+V=1}
|
||||
|
||||
case "$HOST" in
|
||||
*linux*|*mingw*)
|
||||
# Check that executables only contain allowed gcc, glibc and libstdc++
|
||||
# version symbols for Linux distro back-compatibility.
|
||||
make -C src --jobs=1 check-symbols ${V:+V=1}
|
||||
;;
|
||||
esac
|
||||
|
||||
# Make the os-specific installers
|
||||
case "$HOST" in
|
||||
*mingw*)
|
||||
make deploy ${V:+V=1} BITCOIN_WIN_INSTALLER="${OUTDIR}/${DISTNAME}-win64-setup-unsigned.exe"
|
||||
;;
|
||||
esac
|
||||
|
||||
# Setup the directory where our Bitcoin Core build for HOST will be
|
||||
# installed. This directory will also later serve as the input for our
|
||||
# binary tarballs.
|
||||
INSTALLPATH="${PWD}/installed/${DISTNAME}"
|
||||
mkdir -p "${INSTALLPATH}"
|
||||
# Install built Bitcoin Core to $INSTALLPATH
|
||||
make install DESTDIR="${INSTALLPATH}" ${V:+V=1}
|
||||
|
||||
(
|
||||
cd installed
|
||||
|
||||
case "$HOST" in
|
||||
*mingw*)
|
||||
mv --target-directory="$DISTNAME"/lib/ "$DISTNAME"/bin/*.dll
|
||||
;;
|
||||
esac
|
||||
|
||||
# Prune libtool and object archives
|
||||
find . -name "lib*.la" -delete
|
||||
find . -name "lib*.a" -delete
|
||||
|
||||
# Prune pkg-config files
|
||||
rm -r "${DISTNAME}/lib/pkgconfig"
|
||||
|
||||
# Split binaries and libraries from their debug symbols
|
||||
{
|
||||
find "${DISTNAME}/bin" -type f -executable -print0
|
||||
find "${DISTNAME}/lib" -type f -print0
|
||||
} | xargs -0 -n1 -P"$MAX_JOBS" -I{} "${DISTSRC}/contrib/devtools/split-debug.sh" {} {} {}.dbg
|
||||
|
||||
case "$HOST" in
|
||||
*mingw*)
|
||||
cp "${DISTSRC}/doc/README_windows.txt" "${DISTNAME}/readme.txt"
|
||||
;;
|
||||
*linux*)
|
||||
cp "${DISTSRC}/README.md" "${DISTNAME}/"
|
||||
;;
|
||||
esac
|
||||
|
||||
# Finally, deterministically produce {non-,}debug binary tarballs ready
|
||||
# for release
|
||||
case "$HOST" in
|
||||
*mingw*)
|
||||
find "${DISTNAME}" -not -name "*.dbg" -print0 \
|
||||
| xargs -0r touch --no-dereference --date="@${SOURCE_DATE_EPOCH}"
|
||||
find "${DISTNAME}" -not -name "*.dbg" \
|
||||
| sort \
|
||||
| zip -X@ "${OUTDIR}/${DISTNAME}-${HOST//x86_64-w64-mingw32/win64}.zip" \
|
||||
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST//x86_64-w64-mingw32/win64}.zip" && exit 1 )
|
||||
find "${DISTNAME}" -name "*.dbg" -print0 \
|
||||
| xargs -0r touch --no-dereference --date="@${SOURCE_DATE_EPOCH}"
|
||||
find "${DISTNAME}" -name "*.dbg" \
|
||||
| sort \
|
||||
| zip -X@ "${OUTDIR}/${DISTNAME}-${HOST//x86_64-w64-mingw32/win64}-debug.zip" \
|
||||
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST//x86_64-w64-mingw32/win64}-debug.zip" && exit 1 )
|
||||
;;
|
||||
*linux*)
|
||||
find "${DISTNAME}" -not -name "*.dbg" -print0 \
|
||||
| sort --zero-terminated \
|
||||
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
|
||||
| gzip -9n > "${OUTDIR}/${DISTNAME}-${HOST}.tar.gz" \
|
||||
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST}.tar.gz" && exit 1 )
|
||||
find "${DISTNAME}" -name "*.dbg" -print0 \
|
||||
| sort --zero-terminated \
|
||||
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
|
||||
| gzip -9n > "${OUTDIR}/${DISTNAME}-${HOST}-debug.tar.gz" \
|
||||
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST}-debug.tar.gz" && exit 1 )
|
||||
;;
|
||||
esac
|
||||
)
|
||||
)
|
||||
|
||||
case "$HOST" in
|
||||
*mingw*)
|
||||
cp -rf --target-directory=. contrib/windeploy
|
||||
(
|
||||
cd ./windeploy
|
||||
mkdir unsigned
|
||||
cp --target-directory=unsigned/ "${OUTDIR}/${DISTNAME}-win64-setup-unsigned.exe"
|
||||
find . -print0 \
|
||||
| sort --zero-terminated \
|
||||
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
|
||||
| gzip -9n > "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" \
|
||||
|| ( rm -f "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" && exit 1 )
|
||||
)
|
||||
;;
|
||||
esac
|
||||
200
contrib/guix/manifest.scm
Normal file
200
contrib/guix/manifest.scm
Normal file
@ -0,0 +1,200 @@
|
||||
(use-modules (gnu)
|
||||
(gnu packages)
|
||||
(gnu packages autotools)
|
||||
(gnu packages base)
|
||||
(gnu packages bash)
|
||||
(gnu packages check)
|
||||
(gnu packages commencement)
|
||||
(gnu packages compression)
|
||||
(gnu packages cross-base)
|
||||
(gnu packages file)
|
||||
(gnu packages gawk)
|
||||
(gnu packages gcc)
|
||||
(gnu packages installers)
|
||||
(gnu packages linux)
|
||||
(gnu packages mingw)
|
||||
(gnu packages perl)
|
||||
(gnu packages pkg-config)
|
||||
(gnu packages python)
|
||||
(gnu packages shells)
|
||||
(gnu packages bison)
|
||||
(gnu packages version-control)
|
||||
(guix build-system gnu)
|
||||
(guix build-system trivial)
|
||||
(guix gexp)
|
||||
(guix packages)
|
||||
(guix profiles)
|
||||
(guix utils))
|
||||
|
||||
(define (make-ssp-fixed-gcc xgcc)
|
||||
"Given a XGCC package, return a modified package that uses the SSP function
|
||||
from glibc instead of from libssp.so. Our `symbol-check' script will complain if
|
||||
we link against libssp.so, and thus will ensure that this works properly.
|
||||
|
||||
Taken from:
|
||||
http://www.linuxfromscratch.org/hlfs/view/development/chapter05/gcc-pass1.html"
|
||||
(package
|
||||
(inherit xgcc)
|
||||
(arguments
|
||||
(substitute-keyword-arguments (package-arguments xgcc)
|
||||
((#:make-flags flags)
|
||||
`(cons "gcc_cv_libc_provides_ssp=yes" ,flags))))))
|
||||
|
||||
(define (make-gcc-rpath-link xgcc)
|
||||
"Given a XGCC package, return a modified package that replace each instance of
|
||||
-rpath in the default system spec that's inserted by Guix with -rpath-link"
|
||||
(package
|
||||
(inherit xgcc)
|
||||
(arguments
|
||||
(substitute-keyword-arguments (package-arguments xgcc)
|
||||
((#:phases phases)
|
||||
`(modify-phases ,phases
|
||||
(add-after 'pre-configure 'replace-rpath-with-rpath-link
|
||||
(lambda _
|
||||
(substitute* (cons "gcc/config/rs6000/sysv4.h"
|
||||
(find-files "gcc/config"
|
||||
"^gnu-user.*\\.h$"))
|
||||
(("-rpath=") "-rpath-link="))
|
||||
#t))))))))
|
||||
|
||||
(define (make-cross-toolchain target
|
||||
base-gcc-for-libc
|
||||
base-kernel-headers
|
||||
base-libc
|
||||
base-gcc)
|
||||
"Create a cross-compilation toolchain package for TARGET"
|
||||
(let* ((xbinutils (cross-binutils target))
|
||||
;; 1. Build a cross-compiling gcc without targeting any libc, derived
|
||||
;; from BASE-GCC-FOR-LIBC
|
||||
(xgcc-sans-libc (cross-gcc target
|
||||
#:xgcc base-gcc-for-libc
|
||||
#:xbinutils xbinutils))
|
||||
;; 2. Build cross-compiled kernel headers with XGCC-SANS-LIBC, derived
|
||||
;; from BASE-KERNEL-HEADERS
|
||||
(xkernel (cross-kernel-headers target
|
||||
base-kernel-headers
|
||||
xgcc-sans-libc
|
||||
xbinutils))
|
||||
;; 3. Build a cross-compiled libc with XGCC-SANS-LIBC and XKERNEL,
|
||||
;; derived from BASE-LIBC
|
||||
(xlibc (cross-libc target
|
||||
base-libc
|
||||
xgcc-sans-libc
|
||||
xbinutils
|
||||
xkernel))
|
||||
;; 4. Build a cross-compiling gcc targeting XLIBC, derived from
|
||||
;; BASE-GCC
|
||||
(xgcc (cross-gcc target
|
||||
#:xgcc base-gcc
|
||||
#:xbinutils xbinutils
|
||||
#:libc xlibc)))
|
||||
;; Define a meta-package that propagates the resulting XBINUTILS, XLIBC, and
|
||||
;; XGCC
|
||||
(package
|
||||
(name (string-append target "-toolchain"))
|
||||
(version (package-version xgcc))
|
||||
(source #f)
|
||||
(build-system trivial-build-system)
|
||||
(arguments '(#:builder (begin (mkdir %output) #t)))
|
||||
(propagated-inputs
|
||||
`(("binutils" ,xbinutils)
|
||||
("libc" ,xlibc)
|
||||
("libc:static" ,xlibc "static")
|
||||
("gcc" ,xgcc)))
|
||||
(synopsis (string-append "Complete GCC tool chain for " target))
|
||||
(description (string-append "This package provides a complete GCC tool
|
||||
chain for " target " development."))
|
||||
(home-page (package-home-page xgcc))
|
||||
(license (package-license xgcc)))))
|
||||
|
||||
(define* (make-bitcoin-cross-toolchain target
|
||||
#:key
|
||||
(base-gcc-for-libc gcc-5)
|
||||
(base-kernel-headers linux-libre-headers-4.19)
|
||||
(base-libc glibc-2.27)
|
||||
(base-gcc (make-gcc-rpath-link gcc-9)))
|
||||
"Convenience wrapper around MAKE-CROSS-TOOLCHAIN with default values
|
||||
desirable for building Bitcoin Core release binaries."
|
||||
(make-cross-toolchain target
|
||||
base-gcc-for-libc
|
||||
base-kernel-headers
|
||||
base-libc
|
||||
base-gcc))
|
||||
|
||||
(define (make-gcc-with-pthreads gcc)
|
||||
(package-with-extra-configure-variable gcc "--enable-threads" "posix"))
|
||||
|
||||
(define (make-mingw-pthreads-cross-toolchain target)
|
||||
"Create a cross-compilation toolchain package for TARGET"
|
||||
(let* ((xbinutils (cross-binutils target))
|
||||
(pthreads-xlibc mingw-w64-x86_64-winpthreads)
|
||||
(pthreads-xgcc (make-gcc-with-pthreads
|
||||
(cross-gcc target
|
||||
#:xgcc (make-ssp-fixed-gcc gcc-9)
|
||||
#:xbinutils xbinutils
|
||||
#:libc pthreads-xlibc))))
|
||||
;; Define a meta-package that propagates the resulting XBINUTILS, XLIBC, and
|
||||
;; XGCC
|
||||
(package
|
||||
(name (string-append target "-posix-toolchain"))
|
||||
(version (package-version pthreads-xgcc))
|
||||
(source #f)
|
||||
(build-system trivial-build-system)
|
||||
(arguments '(#:builder (begin (mkdir %output) #t)))
|
||||
(propagated-inputs
|
||||
`(("binutils" ,xbinutils)
|
||||
("libc" ,pthreads-xlibc)
|
||||
("gcc" ,pthreads-xgcc)))
|
||||
(synopsis (string-append "Complete GCC tool chain for " target))
|
||||
(description (string-append "This package provides a complete GCC tool
|
||||
chain for " target " development."))
|
||||
(home-page (package-home-page pthreads-xgcc))
|
||||
(license (package-license pthreads-xgcc)))))
|
||||
|
||||
|
||||
(packages->manifest
|
||||
(append
|
||||
(list ;; The Basics
|
||||
bash-minimal
|
||||
which
|
||||
coreutils
|
||||
util-linux
|
||||
;; File(system) inspection
|
||||
file
|
||||
grep
|
||||
diffutils
|
||||
findutils
|
||||
;; File transformation
|
||||
patch
|
||||
gawk
|
||||
sed
|
||||
;; Compression and archiving
|
||||
tar
|
||||
bzip2
|
||||
gzip
|
||||
xz
|
||||
zlib
|
||||
;; Build tools
|
||||
gnu-make
|
||||
libtool
|
||||
autoconf
|
||||
automake
|
||||
pkg-config
|
||||
bison
|
||||
;; Scripting
|
||||
perl
|
||||
python-3.7
|
||||
;; Git
|
||||
git
|
||||
;; Native gcc 9 toolchain targeting glibc 2.27
|
||||
(make-gcc-toolchain gcc-9 glibc-2.27))
|
||||
(let ((target (getenv "HOST")))
|
||||
(cond ((string-suffix? "-mingw32" target)
|
||||
;; Windows
|
||||
(list zip (make-mingw-pthreads-cross-toolchain "x86_64-w64-mingw32") nsis-x86_64))
|
||||
((string-contains target "riscv64-linux-")
|
||||
(list (make-bitcoin-cross-toolchain "riscv64-linux-gnu"
|
||||
#:base-gcc-for-libc gcc-7)))
|
||||
((string-contains target "-linux-")
|
||||
(list (make-bitcoin-cross-toolchain target)))
|
||||
(else '())))))
|
||||
@ -249,7 +249,8 @@ define $(package)_preprocess_cmds
|
||||
echo "!host_build: QMAKE_LFLAGS += $($(package)_ldflags)" >> qtbase/mkspecs/common/gcc-base.conf && \
|
||||
sed -i.old "s|QMAKE_CC = \$$$$\$$$${CROSS_COMPILE}clang|QMAKE_CC = $($(package)_cc)|" qtbase/mkspecs/common/clang.conf && \
|
||||
sed -i.old "s|QMAKE_CXX = \$$$$\$$$${CROSS_COMPILE}clang++|QMAKE_CXX = $($(package)_cxx)|" qtbase/mkspecs/common/clang.conf && \
|
||||
sed -i.old "s/error(\"failed to parse default search paths from compiler output\")/\!darwin: error(\"failed to parse default search paths from compiler output\")/g" qtbase/mkspecs/features/toolchain.prf
|
||||
sed -i.old "s/error(\"failed to parse default search paths from compiler output\")/\!darwin: error(\"failed to parse default search paths from compiler output\")/g" qtbase/mkspecs/features/toolchain.prf && \
|
||||
sed -i.old "s/LIBRARY_PATH/(CROSS_)?\0/g" qtbase/mkspecs/features/toolchain.prf
|
||||
endef
|
||||
|
||||
define $(package)_config_cmds
|
||||
|
||||
@ -2,6 +2,11 @@ Name "@PACKAGE_NAME@ (64-bit)"
|
||||
|
||||
RequestExecutionLevel highest
|
||||
SetCompressor /SOLID lzma
|
||||
SetDateSave off
|
||||
|
||||
# Uncomment these lines when investigating reproducibility errors
|
||||
#SetCompress off
|
||||
#SetDatablockOptimize off
|
||||
|
||||
# General Symbol Definitions
|
||||
!define REGKEY "SOFTWARE\$(^Name)"
|
||||
|
||||
@ -558,7 +558,16 @@ static int CommandLineRPC(int argc, char *argv[])
|
||||
return nRet;
|
||||
}
|
||||
|
||||
#ifdef WIN32
|
||||
// Export main() and ensure working ASLR on Windows.
|
||||
// Exporting a symbol will prevent the linker from stripping
|
||||
// the .reloc section from the binary, which is a requirement
|
||||
// for ASLR. This is a temporary workaround until a fixed
|
||||
// version of binutils is used for releases.
|
||||
__declspec(dllexport) int main(int argc, char* argv[])
|
||||
#else
|
||||
int main(int argc, char* argv[])
|
||||
#endif
|
||||
{
|
||||
RegisterPrettyTerminateHander();
|
||||
RegisterPrettySignalHandlers();
|
||||
|
||||
Loading…
Reference in New Issue
Block a user