dash/depends
fanquake ed8da9277a Merge bitcoin/bitcoin#25817: build: fix cross-arch macOS M1 build reproducibility
08bd3382777bf5d24fa8f32f27abc7b6d1e20dd9 build: optimise arm64 darwin qt build using -O1 (fanquake)

Pull request description:

  Building the macOS M1 bitcoin-qt binary at a optimisation level higher than `-O1` causes reproducibility issues when building on different architectures.

  Proposing somewhat of a hammer.

  This would fix 1 of the 2 remaining HOSTS in #21194.

  Guix Build (x86_64):
  ```bash
  1b58b5109b32dca2509499c93347148e6bab5dca835081f8cbd3123bed72cce1  guix-build-08bd3382777b/output/arm64-apple-darwin/SHA256SUMS.part
  0e0d063d3832fad7c5116dabb2ac33c919f40bda04759aad4523c6247295bc9e  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.dmg
  1feb301245f2c664edcfd9ac528fe1543fc7b183b3b42637db77d57658bc2b5e  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.tar.gz
  641eb100d0a281203f9d6e36e45dc0ffc772c680d6aec462434f106b4c44e295  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin.tar.gz
  9d89920626e35939aa6cf506fc85861179f3c0e18d4ef1954750cf81336a851a  guix-build-08bd3382777b/output/dist-archive/bitcoin-08bd3382777b.tar.gz
  e7697d30084270d0b5843b3baf0d752e240c2f708f728bc2f6896f153276ca6b  guix-build-08bd3382777b/output/x86_64-apple-darwin/SHA256SUMS.part
  dd77acee082dbfd3cdad2c564bbd3bdace8df9bf32f92cf4a2debd5a996ace49  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.dmg
  85e63fccb7af12468a04a678034c42dcd775d243b2d194a52e1086a6ffbdbe84  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.tar.gz
  ff2629957608898d76a42025985e3ec4bf5dc8572794e32b4182ba6f8babb828  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin.tar.gz
  ```

  Guix Build (arm64):
  ```bash
  1b58b5109b32dca2509499c93347148e6bab5dca835081f8cbd3123bed72cce1  guix-build-08bd3382777b/output/arm64-apple-darwin/SHA256SUMS.part
  0e0d063d3832fad7c5116dabb2ac33c919f40bda04759aad4523c6247295bc9e  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.dmg
  1feb301245f2c664edcfd9ac528fe1543fc7b183b3b42637db77d57658bc2b5e  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.tar.gz
  641eb100d0a281203f9d6e36e45dc0ffc772c680d6aec462434f106b4c44e295  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin.tar.gz
  9d89920626e35939aa6cf506fc85861179f3c0e18d4ef1954750cf81336a851a  guix-build-08bd3382777b/output/dist-archive/bitcoin-08bd3382777b.tar.gz
  e7697d30084270d0b5843b3baf0d752e240c2f708f728bc2f6896f153276ca6b  guix-build-08bd3382777b/output/x86_64-apple-darwin/SHA256SUMS.part
  dd77acee082dbfd3cdad2c564bbd3bdace8df9bf32f92cf4a2debd5a996ace49  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.dmg
  85e63fccb7af12468a04a678034c42dcd775d243b2d194a52e1086a6ffbdbe84  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.tar.gz
  ff2629957608898d76a42025985e3ec4bf5dc8572794e32b4182ba6f8babb828  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin.tar.gz
  ```

ACKs for top commit:
  hebasto:
    ACK 08bd3382777bf5d24fa8f32f27abc7b6d1e20dd9
  jarolrod:
    ACK 08bd3382777bf5d24fa8f32f27abc7b6d1e20dd9

Tree-SHA512: 48da4acb1799c3153cdaf674f287c81c3da230a3476183616b74f318baa595af45b313136eb228ba13c63e0b8206a78064734f9fd0488e1e839c9e4e1d92ba25
2023-11-24 11:23:46 -06:00
..
builders merge bitcoin#23998: support OpenBSD in depends 2023-08-08 06:05:02 -05:00
hosts merge bitcoin#23998: support OpenBSD in depends 2023-08-08 06:05:02 -05:00
packages Merge bitcoin/bitcoin#25817: build: fix cross-arch macOS M1 build reproducibility 2023-11-24 11:23:46 -06:00
patches Merge bitcoin/bitcoin#28571: depends: fix unusable memory_resource in macos qt build 2023-10-12 16:47:57 +03:00
.gitignore Merge #17678: depends: Support for S390X and POWER targets 2023-04-25 23:14:25 +03:00
config.guess Merge bitcoin/bitcoin#22071: depends: latest config.guess and config.sub 2021-09-24 13:32:15 -04:00
config.site.in Merge #17658: depends: add ability to skip building qrencode 2022-05-30 19:09:39 +07:00
config.sub Merge bitcoin/bitcoin#22071: depends: latest config.guess and config.sub 2021-09-24 13:32:15 -04:00
description.md trivial: add some missing dashifications (#4772) 2022-04-19 09:09:42 +03:00
funcs.mk merge bitcoin#23998: support OpenBSD in depends 2023-08-08 06:05:02 -05:00
gen_id Merge bitcoin/bitcoin#21427: depends: Fix id_string invocations 2023-04-14 23:34:14 -05:00
Makefile merge bitcoin#23998: support OpenBSD in depends 2023-08-08 06:05:02 -05:00
packages.md Merge #16691: doc: improve depends prefix documentation 2023-04-04 12:45:27 -05:00
README.md Merge #20681: doc: Convert depends options list from html to markdown 2023-08-28 11:31:55 -05:00

Usage

To build dependencies for the current arch+OS:

make

To build for another arch/OS:

make HOST=host-platform-triplet

For example:

make HOST=x86_64-w64-mingw32 -j4

Dash Core's configure script by default will ignore the depends output. In order for it to pick up libraries, tools, and settings from the depends build, you must point it at the appropriate --prefix directory generated by the build. In the above example, a prefix dir named x86_64-w64-mingw32 will be created. To use it for Dash:

./configure --prefix=$PWD/depends/x86_64-w64-mingw32

Common host-platform-triplets for cross compilation are:

  • i686-pc-linux-gnu for Linux 32 bit
  • x86_64-pc-linux-gnu for x86 Linux
  • x86_64-w64-mingw32 for Win64
  • x86_64-apple-darwin for macOS
  • arm64-apple-darwin for ARM macOS
  • arm-linux-gnueabihf for Linux ARM 32 bit
  • aarch64-linux-gnu for Linux ARM 64 bit
  • powerpc64-linux-gnu for Linux POWER 64-bit (big endian)
  • powerpc64le-linux-gnu for Linux POWER 64-bit (little endian)
  • riscv32-linux-gnu for Linux RISC-V 32 bit
  • riscv64-linux-gnu for Linux RISC-V 64 bit
  • s390x-linux-gnu for Linux S390X
  • armv7a-linux-android for Android ARM 32 bit
  • aarch64-linux-android for Android ARM 64 bit
  • x86_64-linux-android for Android x86 64 bit

The paths are automatically configured and no other options are needed unless targeting Android.

Install the required dependencies: Ubuntu & Debian

For macOS cross compilation

sudo apt-get install curl bsdmainutils cmake libcap-dev libz-dev libbz2-dev python3-setuptools libtinfo5 xorriso

Note: You must obtain the macOS SDK before proceeding with a cross-compile. Under the depends directory, create a subdirectory named SDKs. Then, place the extracted SDK under this new directory. For more information, see SDK Extraction.

For Win64 cross compilation

For linux (including i386, ARM) cross compilation

Common linux dependencies:

sudo apt-get install make automake curl g++-multilib libtool binutils-gold bsdmainutils pkg-config python3 patch bison

For linux ARM cross compilation:

sudo apt-get install g++-arm-linux-gnueabihf binutils-arm-linux-gnueabihf

For linux AARCH64 cross compilation:

sudo apt-get install g++-aarch64-linux-gnu binutils-aarch64-linux-gnu

For linux POWER 64-bit cross compilation (there are no packages for 32-bit):

sudo apt-get install g++-powerpc64-linux-gnu binutils-powerpc64-linux-gnu g++-powerpc64le-linux-gnu binutils-powerpc64le-linux-gnu

For linux RISC-V 64-bit cross compilation (there are no packages for 32-bit):

sudo apt-get install g++-riscv64-linux-gnu binutils-riscv64-linux-gnu

RISC-V known issue: gcc-7.3.0 and gcc-7.3.1 result in a broken test_dash executable (see https://github.com/bitcoin/bitcoin/pull/13543), this is apparently fixed in gcc-8.1.0.

For linux S390X cross compilation:

sudo apt-get install g++-s390x-linux-gnu binutils-s390x-linux-gnu

Install the required dependencies: OpenBSD

pkg_add bash gtar

Dependency Options

The following can be set when running make: make FOO=bar

  • SOURCES_PATH: Downloaded sources will be placed here
  • BASE_CACHE: Built packages will be placed here
  • SDK_PATH: Path where SDKs can be found (used by macOS)
  • FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up
  • NO_QT: Don't download/build/cache Qt and its dependencies
  • NO_QR: Don't download/build/cache packages needed for enabling qrencode
  • NO_ZMQ: Don't download/build/cache packages needed for enabling ZeroMQ
  • NO_WALLET: Don't download/build/cache libs needed to enable the wallet
  • NO_BDB: Don't download/build/cache BerkeleyDB
  • NO_SQLITE: Don't download/build/cache SQLite
  • NO_UPNP: Don't download/build/cache packages needed for enabling UPnP
  • NO_NATPMP: Don't download/build/cache packages needed for enabling NAT-PMP
  • DEBUG: Disable some optimizations and enable more runtime checking
  • HOST_ID_SALT: Optional salt to use when generating host package ids
  • BUILD_ID_SALT: Optional salt to use when generating build package ids
  • FORCE_USE_SYSTEM_CLANG: (EXPERTS ONLY) When cross-compiling for macOS, use Clang found in the system's $PATH rather than the default prebuilt release of Clang from llvm.org. Clang 8 or later is required.

If some packages are not built, for example make NO_WALLET=1, the appropriate options will be passed to Dash Core's configure. In this case, --disable-wallet.

Additional targets

download: run 'make download' to fetch all sources without building them
download-osx: run 'make download-osx' to fetch all sources needed for macOS builds
download-win: run 'make download-win' to fetch all sources needed for win builds
download-linux: run 'make download-linux' to fetch all sources needed for linux builds

Android

Before proceeding with an Android build one needs to get the Android SDK and use the "SDK Manager" tool to download the NDK and one or more "Platform packages" (these are Android versions and have a corresponding API level). In order to build ANDROID_API_LEVEL (API level corresponding to the Android version targeted, e.g. Android 9.0 Pie is 28 and its "Platform package" needs to be available) and ANDROID_TOOLCHAIN_BIN (path to toolchain binaries depending on the platform the build is being performed on) need to be set.

API levels from 24 to 29 have been tested to work.

If the build includes Qt, environment variables ANDROID_SDK and ANDROID_NDK need to be set as well but can otherwise be omitted. This is an example command for a default build with no disabled dependencies:

ANDROID_SDK=/home/user/Android/Sdk ANDROID_NDK=/home/user/Android/Sdk/ndk-bundle make HOST=aarch64-linux-android ANDROID_API_LEVEL=28 ANDROID_TOOLCHAIN_BIN=/home/user/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin

Other documentation