Commit Graph

436 Commits

Author SHA1 Message Date
Kittywhiskers Van Gogh
df50931ae2 merge bitcoin#23948: add FreeBSD support to depends 2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
d121666341 merge bitcoin#24668: bump Qt5 version to 5.15.3 2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
d63ec2cc9b merge bitcoin#24722: patch around qt duplicate symbol issue 2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
52e0dc4919 merge bitcoin#23862: Hardcode last modified timestamp in Qt RCC 2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
de72a04582 partial bitcoin#23677: Use Android NDK r23 LTS 2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
3fde12f6ec merge bitcoin#23556: Fix regression in rendering on macOS Big Sur 2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
6e0b683366 merge bitcoin#23675: Post-pr23489 small cleanups 2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
a7f90c070c merge bitcoin#23489: Qt 5.15.2 2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
8cd556376a trivial: rearrange patches to match upstream before upgrading Qt 2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
71ab30057f partial bitcoin#22469: Add support for Android NDK r22+
excludes:
- acaac6e86a9e808244d9c69a59ab3c2d8e34cad6
2023-08-08 06:05:02 -05:00
Kittywhiskers Van Gogh
82ed6f4d3e revert: fix glibc compatibility issues
This reverts commit 4db1397927.
2023-08-01 12:07:31 -05:00
Kittywhiskers Van Gogh
e3260fc53e revert: Avoid fcntl64@GLIBC_2.28 in libsqlite3.a
This reverts commit 90c7f16596.
2023-08-01 12:07:31 -05:00
Kittywhiskers Van Gogh
b8ffea6cc3 revert: Avoid @GLIBC_2.25 symbols for compatibility
This reverts commit 6de96a86bfe51fe974f756c4d8070865d2efc2f2.
2023-08-01 12:07:31 -05:00
Kittywhiskers Van Gogh
66e1541808 merge bitcoin#21851: support cross-compiling for arm64-apple-darwin 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
ba2f56d6ec merge bitcoin#25633: don't restrict --enable-lto to non-guix cctools 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
6751b13f38 merge bitcoin#22526: use newer config.guess & config.sub in depends 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
f16a29a864 partial bitcoin#22318: Avoid @GLIBC_2.25 symbols for compatibility
contains:
 - a4b0b340df22dd0e11096c94ebe669dc68f4a05d
2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
96e8ad8062 merge bitcoin#21991: libevent 2.1.12-stable 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
ab8c26a533 merge bitcoin#23585: Drop Darwin version for better maintainability 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
19a81643c8 merge bitcoin#23817: Fix regression introduced in PR23603 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
13c2a9b586 merge bitcoin#23603: Fix x86_64 <-> arm64 cross-compiling for macOS 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
7d9a5d6ddc merge bitcoin#23618: Ditch no_sdk_version_check.patch 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
2ed704bc8c merge bitcoin#23583: Don't hard-code x86_64 as the arch when using qmake 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
30cd5ef2fa merge bitcoin#25643: compile FastFixedDtoa with -O1 to fix cross-arch reproducibility for arm32 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
b1dd417f26 merge bitcoin#23909: use a static .tiff for macOS .dmg over generating 2023-06-18 11:47:54 -05:00
Kittywhiskers Van Gogh
77539574a7 merge bitcoin#20422: mac deployment unification 2023-06-07 14:42:02 +05:30
Kittywhiskers Van Gogh
e39a1de731 merge bitcoin#23839: build with and test for control flow instrumentation on x86_64 2023-06-07 14:42:02 +05:30
Kittywhiskers Van Gogh
1e9278ea11 merge bitcoin#23535: check for control flow instrumentation 2023-06-07 14:42:02 +05:30
Kittywhiskers Van Gogh
8dab7027e3 merge bitcoin#22070: don't use cf-protection when targeting arm-apple-darwin 2023-06-07 14:42:02 +05:30
Kittywhiskers Van Gogh
cdedbea0b4 merge bitcoin#21889: check for control flow instrumentation 2023-06-07 14:42:02 +05:30
Kittywhiskers Van Gogh
313326e846 merge bitcoin#26633: update qt 5.12 url to archive location 2023-06-06 16:19:57 +05:30
Kittywhiskers Van Gogh
458ed44d19 merge bitcoin#23744: Drop support for i686-linux-android host 2023-05-31 11:06:40 -05:00
fanquake
89928a3e34 Merge bitcoin/bitcoin#22402: doc: Install Rosetta on M1-macOS for qt in depends
cdb41d5573b1e2ed1bc1d8d1dc9f77e82672ee1f doc: Install Rosetta on M1-macOS for qt in depends (Hennadii Stepanov)

Pull request description:

  On master (c609e10545492aba480ff17aff7eefc13a0b5cd8) `make -C depends qt` on Apple Silicon based macOS 11.4 ends with an error:
  ```
  /bin/sh: /Users/hebasto/bitcoin/depends/work/build/aarch64-apple-darwin20.5.0/qt/5.12.11-6c4d47a8f8f/qtbase/bin/moc: Bad CPU type in executable
  ```

  Installing Rosetta 2 fixes it.

  Explanation. On Apple Silicon macOS the `qt` package in depends actually is cross compiled. All native tools (including `moc`) are x86_64 binaries, that require Rosetta 2 to run.

ACKs for top commit:
  promag:
    ACK cdb41d5573b1e2ed1bc1d8d1dc9f77e82672ee1f.
  fanquake:
    ACK cdb41d5573b1e2ed1bc1d8d1dc9f77e82672ee1f - I have not tested after installing Rosetta 2, but I saw the same issue during my first cross-compile on an M1 box.
  Zero-1729:
    ACK cdb41d5573b1e2ed1bc1d8d1dc9f77e82672ee1f

Tree-SHA512: fb06a32d6fb40f405ce856b44f5d3af0c51089886f3be79e509e5c325614d7af58ce4480c064c17e0efb695a1f69f68d533c417f9631d46d8a630aba60ce4433
2023-05-31 10:52:02 -05:00
MarcoFalke
578fac1743 Merge #17678: depends: Support for S390X and POWER targets
11113247c323c5b98debcb512fb9db9fe5a8e7cf depends: Support for S390X targets (MarcoFalke)
989fd539d5bf590c5f6070ee2a4a9e2d3018df2c depends: Support for 64-bit POWER targets (Luke Dashjr)

Pull request description:

  Failure before:

  ```
  $ make -C depends HOST=powerpc64-linux-gnu
  ...
  ERROR: Feature 'system-zlib' was enabled, but the pre-condition 'libs.zlib' failed.

  ERROR: Feature 'xcb' was enabled, but the pre-condition 'libs.xcb' failed.

  ERROR: Feature 'system-freetype' was enabled, but the pre-condition 'features.freetype && libs.freetype' failed.

  ERROR: Feature 'fontconfig' was enabled, but the pre-condition '!config.win32 && !config.darwin && features.system-freetype && libs.fontconfig' failed.
  make: *** [funcs.mk:254: /bitcoin/depends/work/build/powerpc64-linux-gnu/qt/5.9.8-95548079095/qtbase/.stamp_configured] Error 3

  $ make -C depends HOST=s390x-linux-gnu
  ...
  ERROR: Feature 'system-zlib' was enabled, but the pre-condition 'libs.zlib' failed.

  ERROR: Feature 'xcb' was enabled, but the pre-condition 'libs.xcb' failed.

  ERROR: Feature 'system-freetype' was enabled, but the pre-condition 'features.freetype && libs.freetype' failed.

  ERROR: Feature 'fontconfig' was enabled, but the pre-condition '!config.win32 && !config.darwin && features.system-freetype && libs.fontconfig' failed.
  make: *** [funcs.mk:254: /bitcoin/depends/work/build/s390x-linux-gnu/qt/5.9.8-79c6d6ca6ec/qtbase/.stamp_configured] Error 3
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 11113247c323c5b98debcb512fb9db9fe5a8e7cf
  dongcarl:
    tested ACK 11113247c323c5b98debcb512fb9db9fe5a8e7cf
  practicalswift:
    ACK 11113247c323c5b98debcb512fb9db9fe5a8e7cf -- diff looks correct

Tree-SHA512: f990101ced0ed579168bb25762c1296c9b512c597bab924013af41832670a69ed786c6ec9b654c95fe064187797880a66c575395bc102a914c1bdb323ca7538a
2023-04-25 23:14:25 +03:00
W. J. van der Laan
4e100ea192 Merge bitcoin/bitcoin#22283: build: Replace $(AT) with .SILENT
8494dcae0e32716fd7cc7abeacf0a795a1303e6a Replace $(AT) with .SILENCE. (Dmitry Goncharov)

Pull request description:

  This reduces the amount of syntax noise in the makefiles.
  Setting V=1 still enables verbose logging.

  The only noticeable difference in behavior is that, unless V=1 is specified, make won't print its own messages like
  make: Nothing to be done for 'all', make: 'all' is up to date, or touch <file>, if -t is specified.

ACKs for top commit:
  laanwj:
    Tested ACK 8494dcae0e32716fd7cc7abeacf0a795a1303e6a

Tree-SHA512: 66b9111229995aa54a9e87f4571648727d89b8529caec651063cdfe5c00a64341371b648701d192b2334df0614617a00c28eaa56c7f08ee9c00127cada0293ab
2023-04-18 23:24:06 +03:00
fanquake
48b1186ce1 Merge bitcoin/bitcoin#22234: build: Mark print-% target as phony.
fb7be92b094477131140b58a4e3ae98366b93e76 Mark print-% target as phony. (Dmitry Goncharov)

Pull request description:

  .PHONY does not take patterns (such as print-%) as prerequisites.
  Have print-% depend on force and mark force as phony.

  This change ensures print-% rule works even when there is a file that matches the target.

  ```
  $ # on master
  $ make print-host
  host=x86_64-pc-linux-gnu
  $ touch print-host
  $ make print-host
  make: 'print-host' is up to date.
  $
  $ git co mark_print_as_phony
  Switched to branch 'mark_print_as_phony'
  $ make print-host
  host=x86_64-pc-linux-gnu
  $ touch force
  $ make print-host
  host=x86_64-pc-linux-gnu
  ```

ACKs for top commit:
  hebasto:
    ACK fb7be92b094477131140b58a4e3ae98366b93e76, tested on Linux Mint 20.2 (x86_64).

Tree-SHA512: b89ae66aa8c7aa6a7ab5f0956f9eb3b3ef9d56994b60dc2a97d498d4c1bba537845c190723e8a10310280b1b35df2cd935cc30aeb76735cac2dc621ad7823772
2023-04-16 23:40:59 +03:00
W. J. van der Laan
641dc6623a
Merge bitcoin/bitcoin#21427: depends: Fix id_string invocations
fa872c9af397837bb17859b5f43adec71239682a depends: Fix id_string invocations (Carl Dong)

Pull request description:

  Closes: #21242

  ```
  Reproduced from depends/Makefile comment:

  When invoking a shell, GNU Make special-cases exit code 127 (command not
  found) by not capturing the output but instead passing it through. This
  is not done for any other exit code.

  Therefore, we require a "|| true" to avoid this behaviour when in an
  environment where the build_* or host_* may not exist yet.
  ```

ACKs for top commit:
  laanwj:
    Concept and light code review ACK fa872c9af397837bb17859b5f43adec71239682a

Tree-SHA512: 9ce88381aec579d956572cf70c4f69dc5a3873f0d2af14a71cf24814192a89452b8280258bed8cca804e4bd2644db056d213ab733df46a10560a47079524d8ac
2023-04-14 23:34:14 -05:00
fanquake
6ad61a0efe Merge #20985: doc: add xorriso to macOS depends packages
5b41d84b3469484ff6a1f4eb7c890b6444a16091 doc: add xorriso to macOS depends packages (fanquake)

Pull request description:

  This was missed in #20470.

ACKs for top commit:
  hebasto:
    ACK 5b41d84b3469484ff6a1f4eb7c890b6444a16091, tested on Linux Mint 20.1 (x86_64).

Tree-SHA512: bcfd8468a099c69175f8a9d295c1466764ab25d6a61121b28675a09c3e96f45b6309e1523d341f4cb21d0ddee4945f00ba060ba02da835f2f0db7e694fd6c44b
2023-04-09 00:06:56 -05:00
fanquake
ececf49df0 Merge #16691: doc: improve depends prefix documentation
2483266c591f7b2e62df68ee2d13740a706415ec packages.md: document depends build targets (Russell Yanofsky)
be27161ee4bb7cb63346f1e79fc36ce33103a635 Clarify need to specify --prefix with depends (Russell Yanofsky)

Pull request description:

  There seems to be some confusion about exactly how to use depends, when to pass a prefix to `./configure` etc (see #16367, #16654).

  I've cherry-picked two of russ's commits out of #16367, as they are clear stand-alone improvements and we don't have to wait for #16367 to improve the depends documentation.

ACKs for top commit:
  Sjors:
    utACK 2483266
  hebasto:
    ACK 2483266c591f7b2e62df68ee2d13740a706415ec, I have reviewed the code and it looks OK, I agree it can be merged.
  jonasschnelli:
    ACK 2483266c591f7b2e62df68ee2d13740a706415ec

Tree-SHA512: a198c288248f573519a3b0ef384626b61cc803803280af9a448c28466e3d9949bed0332af6618dac19e81c5a6e9694afa83d976b176fd13c32a6c2c3fea3fc1f
2023-04-04 12:45:27 -05:00
Wladimir J. van der Laan
4459e60acc Merge #16352: build: prune dbus from depends
e8fabd9253400a7c3fe45b34bc572eb00ff5522d build: prune dbus from depends (fanquake)

Pull request description:

  Since #8210 (59d063d076), we've been passing `-dbus-runtime` when configuring Qt.

  ```
  qtbase-opensource-src-5.9.7 $ ./configure -h | grep -i dbus
    -no-dbus ............. Do not build the Qt D-Bus module
    -dbus-linked ......... Build Qt D-Bus and link to libdbus-1 [auto]
    -dbus-runtime ........ Build Qt D-Bus and dynamically load libdbus-1 [no]
  ```

  This means we don't actually seem to be using the `D-Bus` we build in depends. This was pointed out by theuni at the time, [here](https://github.com/bitcoin/bitcoin/pull/7993#issuecomment-223114395) and [here](https://github.com/bitcoin/bitcoin/pull/8210#issuecomment-226930545), but was never followed up. dongcarl also bought it up as part of #16150.

  I've tested building and running `bitcoin-qt` using depends on Debian. Needs further testing.

ACKs for top commit:
  laanwj:
    code review ACK e8fabd9253400a7c3fe45b34bc572eb00ff5522d

Tree-SHA512: 164e6e52b6f97c04aef42bd185e2a157bc1a42103840f9404c5a795749f45a8c2c35f35873395a3a56398b3cd5955496b90d9c885d929b434c9bc871695abe20
2023-03-29 21:01:56 +03:00
fanquake
7d78f06a92 partial Merge bitcoin/bitcoin#19817: build: macOS toolchain bump
a5550f877a2c46d01bb620ae051c0c8ed0fecd0b build: use -stdlib++-isystem with Clang 10 (fanquake)
51d9d1607f2e9d593693ca799393f068192e41aa guix: use Clang 10 for the macOS cross compile (fanquake)
b80a6af9e55325d444e117e85bbfc76d88d898a8 build: no longer patch threading out of ld64 (fanquake)
c29cba44b3706e0a2035e440e560f2d15d50433b build: Xcode 12.1, macOS SDK 10.15.6 (fanquake)
9ed2f19d385aa95f65807999bba2e18417b143dc build: native cctools 973.0.1, ld64 609 (fanquake)
f48f187cce7fa43646fb0d796c244e1515e763ec build: Clang 10.0.1 (Hennadii Stepanov)
9b193cd2a3ca20917611fbed56dfbcd8a39aeab8 build: libtapi 1100.0.11 (fanquake)

Pull request description:

  Bumps our macOS toolchain to be using the following:
  * Clang 10.0.1 (gitian) & Clang 10.0.0 (Guix)
  * ld64 609
  * libtapi 1100.0.11
  * cctools  973.0.1
  * Xcode 12.1
  * macOS SDK 10.15.6

  which are currently the most recent releases available as open source. See upstream [`cctools`](https://github.com/tpoechtrager/cctools-port/commits/973.0.1-ld64-609) and [`libtapi`](https://github.com/tpoechtrager/apple-libtapi/tree/1100.0.11).

  This should improve the possibility of Apple ARM cross-compilation in depends.

  This also removes our [patching out of pthreads usage](https://github.com/bitcoin/bitcoin/blob/master/depends/patches/native_cctools/ld64_disable_threading.patch) in `ld64`. There have been multiple changes since `ld64 450.3`, which have likely fixed the non-determinism we were working around. i.e from [InputFiles.cpp](https://opensource.apple.com/source/ld64/ld64-609/src/ld/InputFiles.cpp.auto.html):
  ```cpp
  // <rdar://problem/15002251> make implicit dylib order be deterministic by sorting by install_name
  std::sort(implicitDylibs.begin(), implicitDylibs.end(), DylibByInstallNameSorter());
  ```

  ```cpp
  // <rdar://problem/42675402> ld64 output is not deterministic due to dylib processing order
  std::sort(unprocessedDylibs.begin(), unprocessedDylibs.end(), [](const ld::dylib::File* lhs, const ld::dylib::File* rhs) {
  return strcmp(lhs->path(), rhs->path()) < 0;
  });
  ```

  Guix Build:
  ```bash
  find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  f6c3817b8fe5f7370299d1ae2533e4a3acd313ba9f9aa8d423a8956117e52dd5  guix-build-a5550f877a2c/output/dist-archive/bitcoin-a5550f877a2c.tar.gz
  4954dcf563c2d496b8d9fecd48f8e3f7fba2f319ffa254a5bc8ee12cfee6acf0  guix-build-a5550f877a2c/output/x86_64-apple-darwin18/bitcoin-a5550f877a2c-osx-unsigned.dmg
  8f6095b445c7f1a8e6accd86bb7f0696d5849402084927d2b726b7d557831c3a  guix-build-a5550f877a2c/output/x86_64-apple-darwin18/bitcoin-a5550f877a2c-osx-unsigned.tar.gz
  cc40f25477b4defc1617ae694313d80f307ddf6742fe6cc85c6bc0e215ef8be0  guix-build-a5550f877a2c/output/x86_64-apple-darwin18/bitcoin-a5550f877a2c-osx64.tar.gz
  ```

  Gitian Build:
  ```bash
  Generating report
  506a8abdefe559999b43dd9f14905b9b2b5a3363b1cd013d45ae47acc1f7ef6c  bitcoin-a5550f877a2c-osx-unsigned.dmg
  f606997f74026dd12d110d683c6f116b40df324836904ef507dd7ac787e6ebe2  bitcoin-a5550f877a2c-osx-unsigned.tar.gz
  5b495ef15f2c3260c2950921b61326912a9bf533cccd51e13818809fd225489e  bitcoin-a5550f877a2c-osx64.tar.gz
  f6c3817b8fe5f7370299d1ae2533e4a3acd313ba9f9aa8d423a8956117e52dd5  src/bitcoin-a5550f877a2c.tar.gz
  9eb0221e962d2839770963bd03c6c9e98e8bf3078566bee2ae42f06233a710fa  bitcoin-core-osx-22-res.yml
  Done.
  ```

ACKs for top commit:
  hebasto:
    ACK a5550f877a2c46d01bb620ae051c0c8ed0fecd0b

Tree-SHA512: 504c4b0f9cd3b939714a322298320c5bde07e9356a48a9a000060b36f8dce4d6134ed60c3a5188810476a28ec5b108733eabbc6fb8053231b9ea8a494cc91b12
2023-03-26 16:50:26 -05:00
W. J. van der Laan
0d5bc0b0f5 Merge #21304: guix: Add guix-clean script + establish gc-root for container profiles
867a5e172a23899a4a70eca4a396c64f1951745e guix: Register garbage collector root for containers (Carl Dong)
8f8b96fb542701b7717683caa3848390b24f77ab guix: Update hint messages to mention guix-clean (Carl Dong)
44f6d4f56b16e1dc5e8a23318b8e7aad0665f178 guix: Record precious directories and add guix-clean (Carl Dong)
84912d4b24382ae022da3a863bd6caa2b8948d94 build: Remove spaces from variable-printing rules (Carl Dong)

Pull request description:

  ```
  guix: Record precious directories and add guix-clean

  Many users have reported problems that stem from having an unclean
  working tree. To that end, I've written a guix-clean script which should
  help reset the working tree while respecting user-specified precious
  directories.

  Precious directories, such as:

  - SOURCES_PATH
  - BASE_CACHE
  - SDK_PATH
  - OUTDIR

  Should be preserved when cleaning the working tree, and are thus
  recorded in ./contrib/guix/var/precious_dirs.

  The ./contrib/guix/guix-clean script is able to parse that file and make
  sure to avoid them when cleaning out the working tree.
  ```

ACKs for top commit:
  laanwj:
    ACK 867a5e172a23899a4a70eca4a396c64f1951745e

Tree-SHA512: c498fad781ff5e6406639df2b91b687fc528273fdf266bcdba8f6eec3b3b37ecce544b6da0252f0b9c6717f9d88e844e4c7b72d1877bdbabfc6871ddd0172af5
2023-03-26 16:50:26 -05:00
W. J. van der Laan
dfd1045284 Merge #21375: guix: Misc feedback-based fixes + hier restructuring
7476b46f1893a4858616d2a8456a7c43238851ed guix: Build dmg as a static binary (Carl Dong)
06d6cf6784421290e6235fe8684d5e08ed6f1b62 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab5730dff34466caaecdd292625ef8294fc guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb43b8375a95d82101977829d08fb1e1b guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8fd31a57b0621eea616398017c2ead98 guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d163852df09575e03edcd3db73198ee guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13ecfa5e7e9fdc3030a8110b8bb263f8 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3775d198dbee34dc827353bfd18acd8 guix: Supply --link-profile (Carl Dong)
d55a1056ee565afed64e42d6f6efb6b0adc5599b guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f8bb3574cec48561e13ef3b47dedc6e guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b252bfe256cf72d363a24cf0183e926e guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7bae98ec775c707b9dec4cea1e945802 guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5d0f631e7f08f86ca8a2c2b8213319f guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab3877eb8fe0a8c0cb3d847ac0913c7336 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e97853ea9e1b879e8c76bfb01d4bef33172 guix: Use --cores instead of --max-jobs (Carl Dong)

Pull request description:

  This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
  - Resolves confusion between `--cores=` and `--max-jobs=`
    - `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
  - `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
  - `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
  - Add troubleshooting documentation to `README.md`
  - Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
  - Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
  - `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`

  A few robustness changes are also included:
  - We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
  - We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
  - We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.

  Finally, we restructure the guix building hierarchy such that it looks something like:
  ```
  guix-build-<short-hash-or-version-tag>
  ├── distsrc-<short-hash-or-version-tag>-${HOST}
  │   ├── contrib
  │   ├── depends
  │   ├── src
  │   └── ...
  ├── distsrc-<short-hash-or-version-tag>-...
  └── output
      ├── dist-archive
      │   └── bitcoin-<short-hash-or-version-tag>.tar.gz
      ├── *-linux-*
      │   ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
      │   └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
      ├── x86_64-apple-darwin18
      │   ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
      │   ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
      │   └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
      └── x86_64-w64-mingw32
          ├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
          ├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
          ├── bitcoin-<short-hash-or-version-tag>-win64.zip
          └── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
  ```
  Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.

ACKs for top commit:
  sipa:
    ACK 7476b46f1893a4858616d2a8456a7c43238851ed
  laanwj:
    ACK 7476b46f1893a4858616d2a8456a7c43238851ed

Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
2023-03-26 16:50:26 -05:00
Wladimir J. van der Laan
b207af123c Merge #20629: depends: Improve id string robustness
5200929bfe26c549d7da92c0adf8adf61e143416 depends: Include GUIX_ENVIRONMENT in id string (Carl Dong)
4c7d41858821e4fecf7cb0cec3fcad002365e6c9 depends: Improve id string robustness (Carl Dong)
b3bdff42b5a7b4b956da700b187a7254daac54ae build: Proper quoting for var printing targets (Carl Dong)

Pull request description:

  ```
  Environment variables and search paths can drastically effect the
  operation of build tools.

  Include these in our id string to mitigate against false cache hits.
  ```

  Note to builders: This will invalidate all depends output caches in `BASE_CACHE`

ACKs for top commit:
  laanwj:
    re-ACK 5200929bfe26c549d7da92c0adf8adf61e143416

Tree-SHA512: e70c98da89cde90dc54bc3be89b925787cf94bbf246e27cc9345816b312073d78a02215448f731f21d8cf033c455234a2377ff1d66c00e1f3db69c9c9687d027
2023-03-26 16:50:26 -05:00
Wladimir J. van der Laan
aec7441ac2 Merge #15277: contrib: Enable building in Guix containers
751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong)
cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong)
8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong)
3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong)

Pull request description:

  ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~

  Please read the `README.md`.

  -----

  ### Guix Introduction

  This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin.

  ### Guix Build Walkthrough

  Please read the `README.md`.

  [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718)

  [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011)

  [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439)

  <details>
  <summary>Old instructions no. 1</summary>
  In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment.

  We can then invoke
  ```
  guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes
  ```
  To have Guix:
  1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details).
  2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in.
  3. Drop you into a shell in that container.

  > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>`

  Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following:

  ```
  make -C depends -j"$(nproc)" download && \
      cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes
  ```

  We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing).
  </details>

  ### Rationale

  I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because:

  1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](72bd4ab867/contrib/gitian-descriptors/gitian-linux.yml (L10))), because Guix builds everything about the container, we can perform this on almost any Linux distro/system.
  2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)).
  3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler).

  ### Steps to Completion

  - [x] Successfully build bitcoin inside the Guix environment
  - [x] Make `check-symbols` pass
  - [x] Do the above but without nasty hacks
  - [x] Solve some of the more innocuous hacks
  - [ ] Make it cross-compile (HELP WANTED HERE)
    - [x] Linux
      - [x] x86_64-linux-gnu
      - [x] i686-linux-gnu
      - [x] aarch64-linux-gnu
      - [x] arm-linux-gnueabihf
      - [x] riscv64-linux-gnu
    - [ ] OS X
      - [ ] x86_64-apple-darwin14
    - [ ] Windows
      - [ ] x86_64-w64-mingw32
  - [ ] Maybe make importer for depends syntax
  - [ ] Document build process for future releases
  - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html)

  ### Help Wanted

  [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210)

  [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439)

  <details>
  <summary>Old content no. 1</summary>
  As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be.

  Here's what `ldd src/bitcoind` looks like when built in a Guix container:
  ```
  	linux-vdso.so.1 (0x00007ffcc2d90000)
  	libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000)
  	librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000)
  	libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000)
  	libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000)
  	libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000)
  	libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000)
  	libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000)
  	/gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000)
  ```

  And here's what it looks in one of our releases:
  ```
  	linux-vdso.so.1 (0x00007ffff52cd000)
  	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000)
  	librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000)
  	libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000)
  	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000)
  	libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000)
  	/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000)
  ```

  ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely.

  Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](d6e700e40f/contrib/gitian-descriptors/gitian-win-signer.yml (L14))
  </details>

  ### How to Help

  1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation)
  2. Try out my branch and the command described above!

ACKs for top commit:
  MarcoFalke:
    Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f
  laanwj:
    ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f

Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2023-02-20 09:09:23 -06:00
Kittywhiskers Van Gogh
dc7ad45131 merge bitcoin#25378: sqlite 3380500 in depends 2023-02-07 10:53:33 -06:00
UdjinM6
90c7f16596 build: Avoid fcntl64@GLIBC_2.28 in libsqlite3.a 2023-02-07 10:53:33 -06:00
Kittywhiskers Van Gogh
f42288c984 partial bitcoin#19077: Add sqlite as an alternative wallet database and use it for new descriptor wallets 2023-02-07 10:53:33 -06:00
Konstantin Akimov
e2224cbf8a
build: remove cmake from list of packages (#5179)
## Issue being fixed or feature implemented
Seems as `cmake` package is not needed anymore so far as `immer` moved
inside src/ directory.
`immer` is integrated to our Makefile.am and do not requires extra call
of cmake as it used to be.

## What was done?
Removed cmake from list of packages and removed package `cmake.mk`


## How Has This Been Tested?
Tested build with HEAD - succeed.
After that reverted to the old revision that have `immer` package
(85d6cadbfa).
Failed as expected:
```
make: *** No rule to make target 'cmake', needed by '/home/knst/projects/dash/depends/built/x86_64-pc-linux-gnu/bls-dash/bls-dash-1.2.0-6213882c069.tar.gz'.  Stop.
```


## Breaking Changes
No breaking changes


## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have assigned this pull request to a milestone

---------

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2023-02-07 09:31:20 -06:00
Kittywhiskers Van Gogh
797dd852b7 merge bitcoin#25964: fix mingw miniupnpc cflags 2023-01-18 19:02:39 -06:00