dash/depends
fanquake 5aedcbfb43
Merge bitcoin/bitcoin#28778: depends: drop -O1 workaround from arm64 apple Qt build
664c87354f9ec5df95346eab72a034296b83914d depends: drop -O1 workaround from arm64 apple Qt build (fanquake)

Pull request description:

  Drop the workaround of setting optimization flags to -O1 for the `arm64-apple-darwin` builds. I no-longer see reproducibility issues when building across `x86_64` and `aarch64`:
  ```bash
  real7m21.192s
  user67m41.047s
  sys5m8.596s
  fedora-32gb-hel1-1 bitcoin]# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  62373549d2884e8ef8f46a77b9a93f64ebfc88603569e9d33b68fc67beaf2226  guix-build-664c87354f9e/output/arm64-apple-darwin/SHA256SUMS.part
  597889f1908fdb67a6419177a98935b7119c637a962f03f47270893c5ba3fd6f  guix-build-664c87354f9e/output/arm64-apple-darwin/bitcoin-664c87354f9e-arm64-apple-darwin-unsigned.tar.gz
  289340354532a54a42b7235c831d13fdb28751c643f0fa0fc417ab195e9b5d90  guix-build-664c87354f9e/output/arm64-apple-darwin/bitcoin-664c87354f9e-arm64-apple-darwin-unsigned.zip
  74f4ab3819a186d6c34ca0a4b9dda7c38fcb36bd9b22075a5d91df9bdd5df98a  guix-build-664c87354f9e/output/arm64-apple-darwin/bitcoin-664c87354f9e-arm64-apple-darwin.tar.gz
  f0d0dad63057c7dddf6d6ccee244e7916ac0ee26b3bef8dd35f8430280043b38  guix-build-664c87354f9e/output/dist-archive/bitcoin-664c87354f9e.tar.gz
  fedora-32gb-hel1-1 bitcoin]# uname -m
  aarch64
  ```

  ```bash
  real18m10.759s
  user108m21.656s
  sys6m12.930s
  ubuntu-32gb-hel1-1:~/bitcoin# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  62373549d2884e8ef8f46a77b9a93f64ebfc88603569e9d33b68fc67beaf2226  guix-build-664c87354f9e/output/arm64-apple-darwin/SHA256SUMS.part
  597889f1908fdb67a6419177a98935b7119c637a962f03f47270893c5ba3fd6f  guix-build-664c87354f9e/output/arm64-apple-darwin/bitcoin-664c87354f9e-arm64-apple-darwin-unsigned.tar.gz
  289340354532a54a42b7235c831d13fdb28751c643f0fa0fc417ab195e9b5d90  guix-build-664c87354f9e/output/arm64-apple-darwin/bitcoin-664c87354f9e-arm64-apple-darwin-unsigned.zip
  74f4ab3819a186d6c34ca0a4b9dda7c38fcb36bd9b22075a5d91df9bdd5df98a  guix-build-664c87354f9e/output/arm64-apple-darwin/bitcoin-664c87354f9e-arm64-apple-darwin.tar.gz
  f0d0dad63057c7dddf6d6ccee244e7916ac0ee26b3bef8dd35f8430280043b38  guix-build-664c87354f9e/output/dist-archive/bitcoin-664c87354f9e.tar.gz
  ubuntu-32gb-hel1-1:~/bitcoin# uname -m
  x86_64
  ```

ACKs for top commit:
  hebasto:
    ACK 664c87354f9ec5df95346eab72a034296b83914d.
  TheCharlatan:
    ACK 664c87354f9ec5df95346eab72a034296b83914d

Tree-SHA512: 79527df4181eb0a0c42fe526581479abcdeba8fb09e1faf52265d697d39a8f3a3532ee3c573579b9af00b7330a401e4b6f1686636f9bac6bf9839be8381a2033
2024-10-24 13:50:32 -05:00
..
builders revert: partial dash#3003 (Fix 2 common Travis failures which happen when Travis has network issues) 2024-07-11 16:31:07 +00:00
hosts partial Merge bitcoin/bitcoin#23619: build: Propagate user-defined flags to host packages 2024-10-04 12:54:22 -05:00
packages Merge bitcoin/bitcoin#28778: depends: drop -O1 workaround from arm64 apple Qt build 2024-10-24 13:50:32 -05:00
patches Merge bitcoin/bitcoin#28532: qt: enable -ltcg for windows under LTO 2024-10-24 11:17:28 -05:00
.gitignore Merge #17678: depends: Support for S390X and POWER targets 2023-04-25 23:14:25 +03:00
config.guess Merge bitcoin/bitcoin#22071: depends: latest config.guess and config.sub 2021-09-24 13:32:15 -04:00
config.site.in Merge bitcoin/bitcoin#23611: build: add LTO option to depends 2024-09-27 12:52:15 -05:00
config.sub Merge bitcoin/bitcoin#22071: depends: latest config.guess and config.sub 2021-09-24 13:32:15 -04:00
description.md trivial: add some missing dashifications (#4772) 2022-04-19 09:09:42 +03:00
funcs.mk Merge bitcoin/bitcoin#29706: depends: set two CMake options globally 2024-10-04 12:54:24 -05:00
gen_id Merge bitcoin/bitcoin#23611: build: add LTO option to depends 2024-09-27 12:52:15 -05:00
Makefile Merge bitcoin/bitcoin#23611: build: add LTO option to depends 2024-09-27 12:52:15 -05:00
packages.md Merge bitcoin/bitcoin#29706: depends: set two CMake options globally 2024-10-04 12:54:24 -05:00
README.md Merge bitcoin/bitcoin#23611: build: add LTO option to depends 2024-09-27 12:52:15 -05:00

Usage

To build dependencies for the current arch+OS:

make

To build for another arch/OS:

make HOST=host-platform-triplet

For example:

make HOST=x86_64-w64-mingw32 -j4

Dash Core's configure script by default will ignore the depends output. In order for it to pick up libraries, tools, and settings from the depends build, you must set the CONFIG_SITE environment variable to point to a config.site settings file. In the above example, a file named depends/x86_64-w64-mingw32/share/config.site will be created. To use it during compilation:

CONFIG_SITE=$PWD/depends/x86_64-w64-mingw32/share/config.site ./configure

The default install prefix when using config.site is --prefix=depends/<host-platform-triplet>, so depends build outputs will be installed in that location.

Common host-platform-triplets for cross compilation are:

  • i686-pc-linux-gnu for Linux 32 bit
  • x86_64-pc-linux-gnu for x86 Linux
  • x86_64-w64-mingw32 for Win64
  • x86_64-apple-darwin for macOS
  • arm64-apple-darwin for ARM macOS
  • arm-linux-gnueabihf for Linux ARM 32 bit
  • aarch64-linux-gnu for Linux ARM 64 bit
  • powerpc64-linux-gnu for Linux POWER 64-bit (big endian)
  • powerpc64le-linux-gnu for Linux POWER 64-bit (little endian)
  • riscv32-linux-gnu for Linux RISC-V 32 bit
  • riscv64-linux-gnu for Linux RISC-V 64 bit
  • s390x-linux-gnu for Linux S390X
  • armv7a-linux-android for Android ARM 32 bit
  • aarch64-linux-android for Android ARM 64 bit
  • x86_64-linux-android for Android x86 64 bit

The paths are automatically configured and no other options are needed unless targeting Android.

Install the required dependencies: Ubuntu & Debian

For macOS cross compilation

sudo apt-get install curl bsdmainutils cmake libz-dev python3-setuptools libtinfo5 xorriso

Note: You must obtain the macOS SDK before proceeding with a cross-compile. Under the depends directory, create a subdirectory named SDKs. Then, place the extracted SDK under this new directory. For more information, see SDK Extraction.

For Win64 cross compilation

For linux (including i386, ARM) cross compilation

Common linux dependencies:

sudo apt-get install make automake curl g++-multilib libtool binutils-gold bsdmainutils pkg-config python3 patch bison

For linux ARM cross compilation:

sudo apt-get install g++-arm-linux-gnueabihf binutils-arm-linux-gnueabihf

For linux AARCH64 cross compilation:

sudo apt-get install g++-aarch64-linux-gnu binutils-aarch64-linux-gnu

For linux POWER 64-bit cross compilation (there are no packages for 32-bit):

sudo apt-get install g++-powerpc64-linux-gnu binutils-powerpc64-linux-gnu g++-powerpc64le-linux-gnu binutils-powerpc64le-linux-gnu

For linux RISC-V 64-bit cross compilation (there are no packages for 32-bit):

sudo apt-get install g++-riscv64-linux-gnu binutils-riscv64-linux-gnu

RISC-V known issue: gcc-7.3.0 and gcc-7.3.1 result in a broken test_dash executable (see https://github.com/bitcoin/bitcoin/pull/13543), this is apparently fixed in gcc-8.1.0.

For linux S390X cross compilation:

sudo apt-get install g++-s390x-linux-gnu binutils-s390x-linux-gnu

Install the required dependencies: OpenBSD

pkg_add bash gtar

Dependency Options

The following can be set when running make: make FOO=bar

  • SOURCES_PATH: Downloaded sources will be placed here
  • BASE_CACHE: Built packages will be placed here
  • SDK_PATH: Path where SDKs can be found (used by macOS)
  • FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up
  • NO_QT: Don't download/build/cache Qt and its dependencies
  • NO_QR: Don't download/build/cache packages needed for enabling qrencode
  • NO_ZMQ: Don't download/build/cache packages needed for enabling ZeroMQ
  • NO_WALLET: Don't download/build/cache libs needed to enable the wallet
  • NO_BDB: Don't download/build/cache BerkeleyDB
  • NO_SQLITE: Don't download/build/cache SQLite
  • NO_UPNP: Don't download/build/cache packages needed for enabling UPnP
  • ALLOW_HOST_PACKAGES: Packages that are missed in dependencies (due to NO_* option or build script logic) are searched for among the host system packages using pkg-config. It allows building with packages of other (newer) versions
  • NO_NATPMP: Don't download/build/cache packages needed for enabling NAT-PMP
  • MULTIPROCESS: build libmultiprocess (experimental, requires cmake)
  • DEBUG: Disable some optimizations and enable more runtime checking
  • HOST_ID_SALT: Optional salt to use when generating host package ids
  • BUILD_ID_SALT: Optional salt to use when generating build package ids
  • FORCE_USE_SYSTEM_CLANG: (EXPERTS ONLY) When cross-compiling for macOS, use Clang found in the system's $PATH rather than the default prebuilt release of Clang from llvm.org. Clang 8 or later is required
  • LOG: Use file-based logging for individual packages. During a package build its log file resides in the depends directory, and the log file is printed out automatically in case of build error. After successful build log files are moved along with package archives
  • LTO: Use LTO when building packages.

If some packages are not built, for example make NO_WALLET=1, the appropriate options will be passed to Dash Core's configure. In this case, --disable-wallet.

Additional targets

download: run 'make download' to fetch all sources without building them
download-osx: run 'make download-osx' to fetch all sources needed for macOS builds
download-win: run 'make download-win' to fetch all sources needed for win builds
download-linux: run 'make download-linux' to fetch all sources needed for linux builds

Other documentation