sikkienl/dash
1
0
Fork 0
mirror of https://github.com/dashpay/dash.git synced 2024-12-24 19:42:46 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge #21375: guix: Misc feedback-based fixes + hier restructuring

Browse Source 
7476b46f1893a4858616d2a8456a7c43238851ed guix: Build dmg as a static binary (Carl Dong)
06d6cf6784421290e6235fe8684d5e08ed6f1b62 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab5730dff34466caaecdd292625ef8294fc guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb43b8375a95d82101977829d08fb1e1b guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8fd31a57b0621eea616398017c2ead98 guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d163852df09575e03edcd3db73198ee guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13ecfa5e7e9fdc3030a8110b8bb263f8 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3775d198dbee34dc827353bfd18acd8 guix: Supply --link-profile (Carl Dong)
d55a1056ee565afed64e42d6f6efb6b0adc5599b guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f8bb3574cec48561e13ef3b47dedc6e guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b252bfe256cf72d363a24cf0183e926e guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7bae98ec775c707b9dec4cea1e945802 guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5d0f631e7f08f86ca8a2c2b8213319f guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab3877eb8fe0a8c0cb3d847ac0913c7336 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e97853ea9e1b879e8c76bfb01d4bef33172 guix: Use --cores instead of --max-jobs (Carl Dong)

Pull request description:

  This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
  - Resolves confusion between `--cores=` and `--max-jobs=`
    - `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
  - `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
  - `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
  - Add troubleshooting documentation to `README.md`
  - Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
  - Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
  - `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`

  A few robustness changes are also included:
  - We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
  - We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
  - We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.

  Finally, we restructure the guix building hierarchy such that it looks something like:
  ```
  guix-build-<short-hash-or-version-tag>
  ├── distsrc-<short-hash-or-version-tag>-${HOST}
  │   ├── contrib
  │   ├── depends
  │   ├── src
  │   └── ...
  ├── distsrc-<short-hash-or-version-tag>-...
  └── output
      ├── dist-archive
      │   └── bitcoin-<short-hash-or-version-tag>.tar.gz
      ├── *-linux-*
      │   ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
      │   └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
      ├── x86_64-apple-darwin18
      │   ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
      │   ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
      │   └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
      └── x86_64-w64-mingw32
          ├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
          ├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
          ├── bitcoin-<short-hash-or-version-tag>-win64.zip
          └── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
  ```
  Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.

ACKs for top commit:
  sipa:
    ACK 7476b46f1893a4858616d2a8456a7c43238851ed
  laanwj:
    ACK 7476b46f1893a4858616d2a8456a7c43238851ed

Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
This commit is contained in:
W. J. van der Laan 2021-04-06 00:59:11 +02:00 committed by PastaPastaPasta
parent 5fda1ea15a
commit dfd1045284
12 changed files with 401 additions and 77 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
contrib/gitian-descriptors/assign_DISTNAME
View File

@ -4,7 +4,7 @@
#
# A helper script to be sourced into the gitian descriptors
if RECENT_TAG="$(git describe --exact-match HEAD)"; then
if RECENT_TAG="$(git describe --exact-match HEAD 2> /dev/null)"; then
VERSION="${RECENT_TAG#v}"
else
VERSION="$(git rev-parse --short=12 HEAD)"

109
contrib/guix/README.md
View File

@ -80,6 +80,50 @@ at the end of the `guix pull`)
export PATH="${HOME}/.config/guix/current/bin${PATH:+:}$PATH"
```
### Controlling the number of threads used by `guix` build commands
By default, the scripts under `./contrib/guix` will invoke all `guix` build
commands with `--cores="$JOBS"`. Note that `$JOBS` defaults to `$(nproc)` if not
specified. However, astute manual readers will also notice that there is a
`--max-jobs=` flag (which defaults to 1 if unspecified).
Here is the difference between `--cores=` and `--max-jobs=`:
> Note: When I say "derivation," think "package"
`--cores=`
- controls the number of CPU cores to build each derivation. This is the value
passed to `make`'s `--jobs=` flag.
`--max-jobs=`
- controls how many derivations can be built in parallel
- defaults to 1
Therefore, the default is for `guix` build commands to build one derivation at a
time, utilizing `$JOBS` threads.
Specifying the `$JOBS` environment variable will only modify `--cores=`, but you
can also modify the value for `--max-jobs=` by specifying
`$ADDITIONAL_GUIX_COMMON_FLAGS`. For example, if you have a LOT of memory, you
may want to set:
```sh
export ADDITIONAL_GUIX_COMMON_FLAGS='--max-jobs=8'
```
Which allows for a maximum of 8 derivations to be built at the same time, each
utilizing `$JOBS` threads.
Or, if you'd like to avoid spurious build failures caused by issues with
parallelism within a single package, but would still like to build multiple
packages when the dependency graph allows for it, you may want to try:
```sh
export JOBS=1 ADDITIONAL_GUIX_COMMON_FLAGS='--max-jobs=8'
```
## Usage
### As a Tool for Deterministic Builds
@ -125,12 +169,16 @@ find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
the actual SDK (e.g. SDK_PATH=$HOME/Downloads/macOS-SDKs instead of
$HOME/Downloads/macOS-SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers).
* _**MAX_JOBS**_
* _**JOBS**_
Override the maximum number of jobs to run simultaneously, you might want to
do so on a memory-limited machine. This may be passed to `make` as in `make
--jobs="$MAX_JOBS"` or `xargs` as in `xargs -P"$MAX_JOBS"`. _(defaults to the
value of `nproc` outside the container)_
Override the number of jobs to run simultaneously, you might want to do so on
a memory-limited machine. This may be passed to:
- `guix` build commands as in `guix environment --cores="$JOBS"`
- `make` as in `make --jobs="$JOBS"`
- `xargs` as in `xargs -P"$JOBS"`
_(defaults to the value of `nproc` outside the container)_
* _**SOURCE_DATE_EPOCH**_
@ -217,6 +265,57 @@ To use dongcarl's substitute server for Bitcoin Core builds after having
export SUBSTITUTE_URLS='https://guix.carldong.io https://ci.guix.gnu.org'
```
## Troubleshooting
### Derivation failed to build
When you see a build failure like below:
```
building /gnu/store/...-foo-3.6.12.drv...
/ 'check' phasenote: keeping build directory `/tmp/guix-build-foo-3.6.12.drv-0'
builder for `/gnu/store/...-foo-3.6.12.drv' failed with exit code 1
build of /gnu/store/...-foo-3.6.12.drv failed
View build log at '/var/log/guix/drvs/../...-foo-3.6.12.drv.bz2'.
cannot build derivation `/gnu/store/...-qux-7.69.1.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/...-bar-3.16.5.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/...-baz-2.0.5.drv': 1 dependencies couldn't be built
guix time-machine: error: build of `/gnu/store/...-baz-2.0.5.drv' failed
```
It means that `guix` failed to build a package named `foo`, which was a
dependency of `qux`, `bar`, and `baz`. Importantly, note that the last "failed"
line is not necessarily the root cause, the first "failed" line is.
Most of the time, the build failure is due to a spurious test failure or the
package's build system/test suite breaking when running multi-threaded. To
rebuild _just_ this derivation in a single-threaded fashion:
```sh
$ guix build --cores=1 /gnu/store/...-foo-3.6.12.drv
```
If the single-threaded rebuild stil did not succeed, you may need to dig deeper.
You may view `foo`'s build logs in `less` like so (please replace paths with the
path you see in the build failure output):
```sh
$ bzcat /var/log/guix/drvs/../...-foo-3.6.12.drv.bz2 | less
```
`foo`'s build directory is also preserved and available at
`/tmp/guix-build-foo-3.6.12.drv-0`. However, if you fail to build `foo` multiple
times, it may be `/tmp/...drv-1` or `/tmp/...drv-2`. Always consult the build
failure output for the most accurate, up-to-date information.
#### python(-minimal): [Errno 84] Invalid or incomplete multibyte or wide character
This error occurs when your `$TMPDIR` (default: /tmp) exists on a filesystem
which rejects characters not present in the UTF-8 character code set. An example
is ZFS with the utf8only=on option set.
More information: https://bugs.python.org/issue37584
## FAQ
### How can I trust the binary installation?

109
contrib/guix/guix-build.sh → contrib/guix/guix-build
View File

@ -2,22 +2,26 @@
export LC_ALL=C
set -e -o pipefail
# Source the common prelude, which:
# 1. Checks if we're at the top directory of the Bitcoin Core repository
# 2. Defines a few common functions and variables
#
# shellcheck source=libexec/prelude.bash
source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
###################
## Sanity Checks ##
## SANITY CHECKS ##
###################
################
# Check 1: Make sure that we can invoke required tools
# Required non-builtin commands should be invokable
################
for cmd in git make guix cat mkdir curl; do
if ! command -v "$cmd" > /dev/null 2>&1; then
echo "ERR: This script requires that '$cmd' is installed and available in your \$PATH"
exit 1
fi
done
check_tools cat mkdir make git guix
################
# Check 2: Make sure GUIX_BUILD_OPTIONS is empty
# GUIX_BUILD_OPTIONS should be empty
################
#
# GUIX_BUILD_OPTIONS is an environment variable recognized by guix commands that
@ -45,8 +49,9 @@ exit 1
fi
################
# Check 3: Make sure that we're not in a dirty worktree
# The git worktree should not be dirty
################
if ! git diff-index --quiet HEAD -- && [ -z "$FORCE_DIRTY_WORKTREE" ]; then
cat << EOF
ERR: The current git worktree is dirty, which may lead to broken builds.
@ -60,12 +65,12 @@ Hint: To make your git worktree clean, You may want to:
using a dirty worktree
EOF
exit 1
else
GIT_COMMIT=$(git rev-parse --short=12 HEAD)
fi
mkdir -p "$VERSION_BASE"
################
# Check 4: Make sure that build directories do not exist
# Build directories should not exist
################
# Default to building for all supported HOSTs (overridable by environment)
@ -73,14 +78,12 @@ export HOSTS="${HOSTS:-x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu ri
x86_64-w64-mingw32
x86_64-apple-darwin18}"
DISTSRC_BASE="${DISTSRC_BASE:-${PWD}}"
# Usage: distsrc_for_host HOST
#
# HOST: The current platform triple we're building for
#
distsrc_for_host() {
echo "${DISTSRC_BASE}/distsrc-${GIT_COMMIT}-${1}"
echo "${DISTSRC_BASE}/distsrc-${VERSION}-${1}"
}
# Accumulate a list of build directories that already exist...
@ -106,12 +109,11 @@ for host in $hosts_distsrc_exists; do
done
exit 1
else
mkdir -p "$DISTSRC_BASE"
fi
################
# Check 5: When building for darwin, make sure that the macOS SDK exists
# When building for darwin, the macOS SDK should exists
################
for host in $HOSTS; do
@ -128,13 +130,40 @@ for host in $HOSTS; do
esac
done
################
# Check that we can connect to the guix-daemon
################
cat << EOF
Checking that we can connect to the guix-daemon...
Hint: If this hangs, you may want to try turning your guix-daemon off and on
again.
EOF
if ! guix gc --list-failures > /dev/null; then
cat << EOF
ERR: Failed to connect to the guix-daemon, please ensure that one is running and
reachable.
EOF
exit 1
fi
# Developer note: we could use `guix repl` for this check and run:
#
# (import (guix store)) (close-connection (open-connection))
#
# However, the internal API is likely to change more than the CLI invocation
#########
# Setup #
# SETUP #
#########
# Determine the maximum number of jobs to run simultaneously (overridable by
# environment)
MAX_JOBS="${MAX_JOBS:-$(nproc)}"
JOBS="${JOBS:-$(nproc)}"
# Usage: host_to_commonname HOST
#
@ -152,7 +181,7 @@ host_to_commonname() {
# Download the depends sources now as we won't have internet access in the build
# container
for host in $HOSTS; do
make -C "${PWD}/depends" -j"$MAX_JOBS" download-"$(host_to_commonname "$host")" ${V:+V=1} ${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"}
make -C "${PWD}/depends" -j"$JOBS" download-"$(host_to_commonname "$host")" ${V:+V=1} ${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"}
done
# Determine the reference time used for determinism (overridable by environment)
@ -164,19 +193,29 @@ time-machine() {
# shellcheck disable=SC2086
guix time-machine --url=https://github.com/dongcarl/guix.git \
--commit=490e39ff303f4f6873a04bfb8253755bdae1b29c \
--max-jobs="$MAX_JOBS" \
--cores="$JOBS" \
--keep-failed \
--fallback \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_TIMEMACHINE_FLAGS} \
-- "$@"
}
# Make sure an output directory exists for our builds
OUTDIR="${OUTDIR:-${PWD}/output}"
[ -e "$OUTDIR" ] || mkdir -p "$OUTDIR"
OUTDIR_BASE="${OUTDIR_BASE:-${VERSION_BASE}/output}"
mkdir -p "$OUTDIR_BASE"
# Usage: outdir_for_host HOST
#
# HOST: The current platform triple we're building for
#
outdir_for_host() {
echo "${OUTDIR_BASE}/${1}"
}
#########
# Build #
# BUILD #
#########
# Function to be called when building for host ${1} and the user interrupts the
@ -216,15 +255,15 @@ for host in $HOSTS; do
# shellcheck disable=SC2030
cat << EOF
INFO: Building commit ${GIT_COMMIT:?not set} for platform triple ${HOST:?not set}:
INFO: Building ${VERSION:?not set} for platform triple ${HOST:?not set}:
...using reference timestamp: ${SOURCE_DATE_EPOCH:?not set}
...running at most ${MAX_JOBS:?not set} jobs
...running at most ${JOBS:?not set} jobs
...from worktree directory: '${PWD}'
...bind-mounted in container to: '/bitcoin'
...in build directory: '$(distsrc_for_host "$HOST")'
...bind-mounted in container to: '$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")'
...outputting in: '${OUTDIR:?not set}'
...bind-mounted in container to: '/outdir'
...outdirting in: '$(outdir_for_host "$HOST")'
...bind-mounted in container to: '$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST")'
EOF
# Run the build script 'contrib/guix/libexec/build.sh' in the build
@ -299,24 +338,28 @@ EOF
--no-cwd \
--share="$PWD"=/bitcoin \
--share="$DISTSRC_BASE"=/distsrc-base \
--share="$OUTDIR"=/outdir \
--share="$OUTDIR_BASE"=/outdir-base \
--expose="$(git rev-parse --git-common-dir)" \
${SOURCES_PATH:+--share="$SOURCES_PATH"} \
${BASE_CACHE:+--share="$BASE_CACHE"} \
${SDK_PATH:+--share="$SDK_PATH"} \
--max-jobs="$MAX_JOBS" \
--cores="$JOBS" \
--keep-failed \
--fallback \
--link-profile \
${SUBSTITUTE_URLS:+--substitute-urls="$SUBSTITUTE_URLS"} \
${ADDITIONAL_GUIX_COMMON_FLAGS} ${ADDITIONAL_GUIX_ENVIRONMENT_FLAGS} \
-- env HOST="$host" \
MAX_JOBS="$MAX_JOBS" \
DISTNAME="$DISTNAME" \
JOBS="$JOBS" \
SOURCE_DATE_EPOCH="${SOURCE_DATE_EPOCH:?unable to determine value}" \
${V:+V=1} \
${SOURCES_PATH:+SOURCES_PATH="$SOURCES_PATH"} \
${BASE_CACHE:+BASE_CACHE="$BASE_CACHE"} \
${SDK_PATH:+SDK_PATH="$SDK_PATH"} \
DISTSRC="$(DISTSRC_BASE=/distsrc-base && distsrc_for_host "$HOST")" \
OUTDIR=/outdir \
OUTDIR="$(OUTDIR_BASE=/outdir-base && outdir_for_host "$HOST")" \
DIST_ARCHIVE_BASE=/outdir-base/dist-archive \
bash -c "cd /bitcoin && bash contrib/guix/libexec/build.sh"
)

82
contrib/guix/libexec/build.sh
View File

@ -24,9 +24,11 @@ fi
# Check that required environment variables are set
cat << EOF
Required environment variables as seen inside the container:
DIST_ARCHIVE_BASE: ${DIST_ARCHIVE_BASE:?not set}
DISTNAME: ${DISTNAME:?not set}
HOST: ${HOST:?not set}
SOURCE_DATE_EPOCH: ${SOURCE_DATE_EPOCH:?not set}
MAX_JOBS: ${MAX_JOBS:?not set}
JOBS: ${JOBS:?not set}
DISTSRC: ${DISTSRC:?not set}
OUTDIR: ${OUTDIR:?not set}
EOF
@ -52,16 +54,36 @@ store_path() {
# Set environment variables to point the NATIVE toolchain to the right
# includes/libs
NATIVE_GCC="$(store_path gcc-toolchain)"
export LIBRARY_PATH="${NATIVE_GCC}/lib:${NATIVE_GCC}/lib64"
export CPATH="${NATIVE_GCC}/include"
NATIVE_GCC_STATIC="$(store_path gcc-toolchain static)"
unset LIBRARY_PATH
unset CPATH
unset C_INCLUDE_PATH
unset CPLUS_INCLUDE_PATH
unset OBJC_INCLUDE_PATH
unset OBJCPLUS_INCLUDE_PATH
export LIBRARY_PATH="${NATIVE_GCC}/lib:${NATIVE_GCC}/lib64:${NATIVE_GCC_STATIC}/lib:${NATIVE_GCC_STATIC}/lib64"
export C_INCLUDE_PATH="${NATIVE_GCC}/include"
export CPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
export OBJC_INCLUDE_PATH="${NATIVE_GCC}/include"
export OBJCPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
prepend_to_search_env_var() {
export "${1}=${2}${!1:+:}${!1}"
}
case "$HOST" in
*darwin*)
# When targeting darwin, zlib is required by native_libdmg-hfsplus.
zlib_store_path=$(store_path "zlib")
export LIBRARY_PATH="${zlib_store_path}/lib:${LIBRARY_PATH}"
export CPATH="${zlib_store_path}/include:${CPATH}"
zlib_static_store_path=$(store_path "zlib" static)
prepend_to_search_env_var LIBRARY_PATH "${zlib_static_store_path}/lib:${zlib_store_path}/lib"
prepend_to_search_env_var C_INCLUDE_PATH "${zlib_store_path}/include"
prepend_to_search_env_var CPLUS_INCLUDE_PATH "${zlib_store_path}/include"
prepend_to_search_env_var OBJC_INCLUDE_PATH "${zlib_store_path}/include"
prepend_to_search_env_var OBJCPLUS_INCLUDE_PATH "${zlib_store_path}/include"
esac
# Set environment variables to point the CROSS toolchain to the right
@ -173,7 +195,7 @@ esac
####################
# Build the depends tree, overriding variables that assume multilib gcc
make -C depends --jobs="$MAX_JOBS" HOST="$HOST" \
make -C depends --jobs="$JOBS" HOST="$HOST" \
${V:+V=1} \
${SOURCES_PATH+SOURCES_PATH="$SOURCES_PATH"} \
${BASE_CACHE+BASE_CACHE="$BASE_CACHE"} \
@ -198,11 +220,7 @@ make -C depends --jobs="$MAX_JOBS" HOST="$HOST" \
# Source Tarball Building #
###########################
# Define DISTNAME variable.
# shellcheck source=contrib/gitian-descriptors/assign_DISTNAME
source contrib/gitian-descriptors/assign_DISTNAME
GIT_ARCHIVE="${OUTDIR}/src/${DISTNAME}.tar.gz"
GIT_ARCHIVE="${DIST_ARCHIVE_BASE}/${DISTNAME}.tar.gz"
# Create the source tarball if not already there
if [ ! -e "$GIT_ARCHIVE" ]; then
@ -268,7 +286,7 @@ mkdir -p "$DISTSRC"
# Build Bitcoin Core
make --jobs="$MAX_JOBS" ${V:+V=1}
make --jobs="$JOBS" ${V:+V=1}
# Perform basic ELF security checks on a series of executables.
make -C src --jobs=1 check-security ${V:+V=1}
@ -276,6 +294,7 @@ mkdir -p "$DISTSRC"
# version symbols for Linux distro back-compatibility.
make -C src --jobs=1 check-symbols ${V:+V=1}
mkdir -p ${OUTDIR}
# Make the os-specific installers
case "$HOST" in
*mingw*)
@ -307,9 +326,6 @@ mkdir -p "$DISTSRC"
osx_volname \
contrib/macdeploy/detached-sig-{apply,create}.sh \
"${BASEPREFIX}/${HOST}"/native/bin/dmg
for util in codesign_allocate pagestuff; do
cp --no-target-directory {"${BASEPREFIX}/${HOST}/native/bin/${HOST}-","unsigned-app-${HOST}/"}"$util"
done
mv --target-directory="unsigned-app-${HOST}" dist
(
cd "unsigned-app-${HOST}"
@ -345,7 +361,7 @@ mkdir -p "$DISTSRC"
{
find "${DISTNAME}/bin" -type f -executable -print0
find "${DISTNAME}/lib" -type f -print0
} | xargs -0 -n1 -P"$MAX_JOBS" -I{} "${DISTSRC}/contrib/devtools/split-debug.sh" {} {} {}.dbg
} | xargs -0 -n1 -P"$JOBS" -I{} "${DISTSRC}/contrib/devtools/split-debug.sh" {} {} {}.dbg
;;
esac
@ -395,21 +411,21 @@ mkdir -p "$DISTSRC"
|| ( rm -f "${OUTDIR}/${DISTNAME}-${HOST//x86_64-apple-darwin18/osx64}.tar.gz" && exit 1 )
;;
esac
)
)
) # $DISTSRC/installed
case "$HOST" in
*mingw*)
cp -rf --target-directory=. contrib/windeploy
(
cd ./windeploy
mkdir unsigned
cp --target-directory=unsigned/ "${OUTDIR}/${DISTNAME}-win64-setup-unsigned.exe"
find . -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" && exit 1 )
)
;;
esac
case "$HOST" in
*mingw*)
cp -rf --target-directory=. contrib/windeploy
(
cd ./windeploy
mkdir -p unsigned
cp --target-directory=unsigned/ "${OUTDIR}/${DISTNAME}-win64-setup-unsigned.exe"
find . -print0 \
| sort --zero-terminated \
| tar --create --no-recursion --mode='u+rw,go+r-w,a+X' --null --files-from=- \
| gzip -9n > "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" \
|| ( rm -f "${OUTDIR}/${DISTNAME}-win-unsigned.tar.gz" && exit 1 )
)
;;
esac
) # $DISTSRC

60
contrib/guix/libexec/prelude.bash Normal file
View File

@ -0,0 +1,60 @@
#!/usr/bin/env bash
export LC_ALL=C
set -e -o pipefail
# shellcheck source=../../shell/realpath.bash
source contrib/shell/realpath.bash
# shellcheck source=../../shell/git-utils.bash
source contrib/shell/git-utils.bash
################
# Required non-builtin commands should be invokable
################
check_tools() {
for cmd in "$@"; do
if ! command -v "$cmd" > /dev/null 2>&1; then
echo "ERR: This script requires that '$cmd' is installed and available in your \$PATH"
exit 1
fi
done
}
check_tools cat env readlink dirname basename git
################
# We should be at the top directory of the repository
################
same_dir() {
local resolved1 resolved2
resolved1="$(bash_realpath "${1}")"
resolved2="$(bash_realpath "${2}")"
[ "$resolved1" = "$resolved2" ]
}
if ! same_dir "${PWD}" "$(git_root)"; then
cat << EOF
ERR: This script must be invoked from the top level of the git repository
Hint: This may look something like:
env FOO=BAR ./contrib/guix/guix-<blah>
EOF
exit 1
fi
################
# Set common variables
################
VERSION="${VERSION:-$(git_head_version)}"
DISTNAME="${DISTNAME:-bitcoin-${VERSION}}"
version_base_prefix="${PWD}/guix-build-"
VERSION_BASE="${version_base_prefix}${VERSION}" # TOP
DISTSRC_BASE="${DISTSRC_BASE:-${VERSION_BASE}}"
OUTDIR_BASE="${OUTDIR_BASE:-${VERSION_BASE}/output}"

6
contrib/guix/manifest.scm
View File

@ -214,6 +214,7 @@ chain for " target " development."))
gzip
xz
zlib
(list zlib "static")
;; Build tools
gnu-make
libtool
@ -227,7 +228,8 @@ chain for " target " development."))
;; Git
git
;; Native gcc 7 toolchain
gcc-toolchain-7)
gcc-toolchain-7
(list gcc-toolchain-7 "static"))
(let ((target (getenv "HOST")))
(cond ((string-suffix? "-mingw32" target)
;; Windows
@ -237,5 +239,5 @@ chain for " target " development."))
((string-contains target "-linux-")
(list (make-bitcoin-cross-toolchain target)))
((string-contains target "darwin")
(list clang-8 binutils imagemagick libtiff librsvg font-tuffy cmake xorriso))
(list clang-toolchain-8 binutils imagemagick libtiff librsvg font-tuffy cmake xorriso))
(else '())))))

14
contrib/shell/git-utils.bash Normal file
View File

@ -0,0 +1,14 @@
#!/usr/bin/env bash
git_root() {
git rev-parse --show-toplevel 2> /dev/null
}
git_head_version() {
local recent_tag
if recent_tag="$(git describe --exact-match HEAD 2> /dev/null)"; then
echo "${recent_tag#v}"
else
git rev-parse --short=12 HEAD
fi
}

71
contrib/shell/realpath.bash Normal file
View File

@ -0,0 +1,71 @@
#!/usr/bin/env bash
# Based on realpath.sh written by Michael Kropat
# Found at: https://github.com/mkropat/sh-realpath/blob/65512368b8155b176b67122aa395ac580d9acc5b/realpath.sh
bash_realpath() {
canonicalize_path "$(resolve_symlinks "$1")"
}
resolve_symlinks() {
_resolve_symlinks "$1"
}
_resolve_symlinks() {
_assert_no_path_cycles "$@" || return
local dir_context path
if path=$(readlink -- "$1"); then
dir_context=$(dirname -- "$1")
_resolve_symlinks "$(_prepend_dir_context_if_necessary "$dir_context" "$path")" "$@"
else
printf '%s\n' "$1"
fi
}
_prepend_dir_context_if_necessary() {
if [ "$1" = . ]; then
printf '%s\n' "$2"
else
_prepend_path_if_relative "$1" "$2"
fi
}
_prepend_path_if_relative() {
case "$2" in
/* ) printf '%s\n' "$2" ;;
* ) printf '%s\n' "$1/$2" ;;
esac
}
_assert_no_path_cycles() {
local target path
target=$1
shift
for path in "$@"; do
if [ "$path" = "$target" ]; then
return 1
fi
done
}
canonicalize_path() {
if [ -d "$1" ]; then
_canonicalize_dir_path "$1"
else
_canonicalize_file_path "$1"
fi
}
_canonicalize_dir_path() {
(cd "$1" 2>/dev/null && pwd -P)
}
_canonicalize_file_path() {
local dir file
dir=$(dirname -- "$1")
file=$(basename -- "$1")
(cd "$dir" 2>/dev/null && printf '%s/%s\n' "$(pwd -P)" "$file")
}

4
depends/Makefile
View File

@ -139,8 +139,8 @@ build_id_string+=system_clang
$(host_arch)_$(host_os)_id_string+=system_clang
endif
build_id_string+=GUIX_ENVIRONMENT=$(GUIX_ENVIRONMENT)
$(host_arch)_$(host_os)_id_string+=GUIX_ENVIRONMENT=$(GUIX_ENVIRONMENT)
build_id_string+=GUIX_ENVIRONMENT=$(realpath $(GUIX_ENVIRONMENT))
$(host_arch)_$(host_os)_id_string+=GUIX_ENVIRONMENT=$(realpath $(GUIX_ENVIRONMENT))
qrencode_packages_$(NO_QR) = $(qrencode_$(host_os)_packages)

2
depends/packages/native_libdmg-hfsplus.mk
View File

@ -12,7 +12,7 @@ define $(package)_preprocess_cmds
endef
define $(package)_config_cmds
cmake -DCMAKE_INSTALL_PREFIX:PATH=$(build_prefix) -DCMAKE_C_FLAGS="-Wl,--build-id=none" ..
cmake -DCMAKE_INSTALL_PREFIX:PATH=$(build_prefix) -DCMAKE_C_FLAGS="-Wl,--build-id=none" -DCMAKE_SKIP_RPATH="ON" -DCMAKE_EXE_LINKER_FLAGS="-static" -DCMAKE_FIND_LIBRARY_SUFFIXES=".a" ..
endef
define $(package)_build_cmds

2
depends/packages/qt.mk
View File

@ -15,6 +15,7 @@ $(package)_patches+= fix_qpainter_non_determinism.patch fix_lib_paths.patch fix_
$(package)_patches+= fix_limits_header.patch
$(package)_patches+= fix_montery_include.patch
$(package)_patches += glibc_compatibility.patch
$(package)_patches+= qtbase-moc-ignore-gcc-macro.patch
# Update OSX_QT_TRANSLATIONS when this is updated
$(package)_qttranslations_file_name=qttranslations-$($(package)_suffix)
@ -239,6 +240,7 @@ define $(package)_preprocess_cmds
patch -p1 -i $($(package)_patch_dir)/fix_limits_header.patch && \
patch -p1 -i $($(package)_patch_dir)/fix_montery_include.patch && \
patch -p1 -i $($(package)_patch_dir)/glibc_compatibility.patch && \
patch -p1 -i $($(package)_patch_dir)/qtbase-moc-ignore-gcc-macro.patch && \
mkdir -p qtbase/mkspecs/macx-clang-linux &&\
cp -f qtbase/mkspecs/macx-clang/qplatformdefs.h qtbase/mkspecs/macx-clang-linux/ &&\
cp -f $($(package)_patch_dir)/mac-qmake.conf qtbase/mkspecs/macx-clang-linux/qmake.conf && \

17
depends/patches/qt/qtbase-moc-ignore-gcc-macro.patch Normal file
View File

@ -0,0 +1,17 @@
The moc executable loops through headers on CPLUS_INCLUDE_PATH and stumbles
on the GCC internal _GLIBCXX_VISIBILITY macro. Tell it to ignore it as it is
not supposed to be looking there to begin with.
Upstream report: https://bugreports.qt.io/browse/QTBUG-83160
diff --git a/qtbase/src/tools/moc/main.cpp b/qtbase/src/tools/moc/main.cpp
--- a/qtbase/src/tools/moc/main.cpp
+++ b/qtbase/src/tools/moc/main.cpp
@@ -188,6 +188,7 @@ int runMoc(int argc, char **argv)
dummyVariadicFunctionMacro.arguments += Symbol(0, PP_IDENTIFIER, "__VA_ARGS__");
pp.macros["__attribute__"] = dummyVariadicFunctionMacro;
pp.macros["__declspec"] = dummyVariadicFunctionMacro;
+ pp.macros["_GLIBCXX_VISIBILITY"] = dummyVariadicFunctionMacro;
QString filename;
QString output;