b6544791b4 Merge bitcoin/bitcoin#30705: test: Avoid intermittent block download timeout in p2p_ibd_stalling (merge-script)
745a819683 Merge bitcoin/bitcoin#30690: devtools, utxo-snapshot: Fix block height out of range in script (Ava Chow)
01b570e9e2 Merge bitcoin/bitcoin#29999: guix: fix suggested fake date for openssl-1.1.1l (Ava Chow)
432f352719 Merge bitcoin/bitcoin#30580: doc: Add note about distro's `g++-mingw-w64-x86-64-posix` version (merge-script)
1bd090e7fb Merge bitcoin/bitcoin#30597: doc: Drop no longer needed workaround for WSL (merge-script)
8a12237745 Merge bitcoin/bitcoin#30630: doc: Update ccache website link (merge-script)
f66547f84e Merge bitcoin/bitcoin#30588: depends: fix ZMQ CMake getcachesize check (merge-script)
ddaec964c0 Merge bitcoin/bitcoin#30565: depends: Fix `zeromq` build on OpenBSD (merge-script)
e4e5605ef8 Merge bitcoin/bitcoin#30552: test: fix constructor of msg_tx (merge-script)
df3c2392ca Merge bitcoin/bitcoin#26950: cleanse: switch to SecureZeroMemory for Windows cross-compile (merge-script)
57945ce337 Merge bitcoin/bitcoin#30506: depends: Cleanup postprocess commands after switching to CMake (merge-script)
e016ffada1 Merge bitcoin/bitcoin#29878: depends: build expat with CMake (merge-script)
62dcd43b58 Merge bitcoin/bitcoin#29880: depends: build FreeType with CMake (merge-script)
745addf6a7 Merge bitcoin/bitcoin#30245: net: Allow -proxy=[::1] on nodes with IPV6 lo only (Ava Chow)
4e144be0dd Merge bitcoin-core/gui#795: Keep focus on "Hide" while ModalOverlay is visible (Hennadii Stepanov)
69c04b2c48 Merge bitcoin/bitcoin#30372: util: Use SteadyClock in RandAddSeedPerfmon (merge-script)
ebed8af1d0 Merge bitcoin/bitcoin#30336: depends: update doc in Qt pwd patch (merge-script)
9793fb1a87 Merge bitcoin/bitcoin#30340: test: Added coverage to Block not found error using gettxoutsetinfo (Ava Chow)
479cb8bcb8 Merge bitcoin/bitcoin#30312: contrib: add R(UN)PATH check to ELF symbol-check (merge-script)
ca83773854 Merge bitcoin/bitcoin#30283: upnp: fix build with miniupnpc 2.2.8 (merge-script)
63e139d11b Merge bitcoin/bitcoin#30185: guix: show `*_FLAGS` variables in pre-build output (merge-script)
3be0d3e5cd Merge bitcoin/bitcoin#30097: crypto: disable asan for sha256_sse4 with clang and -O0 (merge-script)
3070c3e388 Merge bitcoin/bitcoin#30078: depends: set AR & RANLIB for CMake (merge-script)
Pull request description:
## Issue being fixed or feature implemented
Trivial backports
## What was done?
## How Has This Been Tested?
built locally
## Breaking Changes
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK b6544791b4
kwvg:
utACK b6544791b4
Tree-SHA512: 10b5af4e92c83fa9d6764b20bf066bba8e4c600402966fd5c1d6dad07b0549d8a42151a33f21e2f8263336c12a810a6f3fc2828d90bc98153e09c165d9e5b043
7c69baf227252511455bc06e315f6a3c7fc5a398 depends: pass verbose through to cmake based make (Max Edwards)
Pull request description:
While testing https://github.com/bitcoin/bitcoin/pull/29708 I was not able to enable verbose output to check which flags were being given to the compiler.
With this PR, running depends with V=1 will enable verbose output from makefiles generated by cmake.
How to test:
```shell
make -C depends libnatpmp V=1
```
ACKs for top commit:
hebasto:
ACK 7c69baf227252511455bc06e315f6a3c7fc5a398. Tested using the folowing command:
fanquake:
ACK 7c69baf227252511455bc06e315f6a3c7fc5a398
Tree-SHA512: 81cd1326e940c5f14cbde96735fd02b03c1150881ed88d1e8dfa9385dfa12284bfa2cdfe097ce5f43a726c1718afb76ae16f71552ab68c207d74fdc1f7bb46ae
019ad7327c397094d7648b55503bf5373b108a57 depends: set RANLIB for CMake (fanquake)
43cfb428cba04b8db98d4d0d56ffe28ad686e58c depends: set NM for CMake (fanquake)
1e4412b317f74dd64069309544fe73c95e2c10e7 depends: set AR for CMake (fanquake)
Pull request description:
Needed for #21778. Should be more correct in any case.
ACKs for top commit:
theuni:
utACK 019ad7327c397094d7648b55503bf5373b108a57. I didn't test, but I tried this approach on a few deps and it seemed to work as expected.
TheCharlatan:
ACK 019ad7327c397094d7648b55503bf5373b108a57
Tree-SHA512: 78cc8981456f7476cafca0e40fcc569e474b92004c8024d1c4268b6aab53175074a06ab17ebded8d706bf0a7f77401642dd38bb7ce2e4b04abdcd149d3d69969
978852aad8e295c26801e17f2ee9e58edd4a5703 build: Fix depends build system when working with subtargets (Hennadii Stepanov)
Pull request description:
On master (f3e0ace8ecd84009a23da6b0de47f01d79c45772), the depends build system does _not_ guarantee that dependencies packages are available for `$(package)_built` target because these dependencies being prepared in `$(host_prefix)` at `$(package)_configured` target can be wiped out during building other package.
Please consider:
```
$ cd depends
$ make clean
$ make fontconfig_configured
$ make
...
CC fcdir.lo
In file included from fcftint.h:26,
from fcdir.c:26:
../fontconfig/fcfreetype.h:27:10: fatal error: ft2build.h: No such file or directory
27 | #include <ft2build.h>
| ^~~~~~~~~~~~
compilation terminated.
make[4]: *** [Makefile:642: fcdir.lo] Error 1
make[4]: *** Waiting for unfinished jobs....
make[4]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94/src'
make[3]: *** [Makefile:503: all] Error 2
make[3]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94/src'
make[2]: *** [Makefile:581: all-recursive] Error 1
make[2]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94'
make[1]: *** [Makefile:465: all] Error 2
make[1]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94'
make: *** [funcs.mk:288: /home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94/./.stamp_built] Error 2
```
The following commands:
```
$ cd depends
$ make clean
$ make qt_configured
$ make
```
also fail.
The similar issue was reported earlier: #21381.
This PR guarantees that dependencies packages are available for `$(package)_built` target.
Guix builds:
```
accef9ffccfe280fec1114c0440092ed5d792e9c53d95abc7fe65435aa136656 guix-build-978852aad8e2/output/aarch64-linux-gnu/SHA256SUMS.part
a75f1250975525a21d2e213e23f1f0dab516d2b28d0c7d747de292fe5c906013 guix-build-978852aad8e2/output/aarch64-linux-gnu/bitcoin-978852aad8e2-aarch64-linux-gnu-debug.tar.gz
d20787af2e7a14a3b7b1d21e0d8784aa6ebad1e916f02aebfa25afe9229ba43c guix-build-978852aad8e2/output/aarch64-linux-gnu/bitcoin-978852aad8e2-aarch64-linux-gnu.tar.gz
11c94a39c084763858c6de31b221868e52554f5500c0dc5589def429bb6b54c8 guix-build-978852aad8e2/output/arm-linux-gnueabihf/SHA256SUMS.part
3935b0e14d78800977dc813a3824215797096b6fb29c84b5996f48338908a65f guix-build-978852aad8e2/output/arm-linux-gnueabihf/bitcoin-978852aad8e2-arm-linux-gnueabihf-debug.tar.gz
c828c3f87a453db443a385a266331661f623f8f4684d88eb006290c83bfa8150 guix-build-978852aad8e2/output/arm-linux-gnueabihf/bitcoin-978852aad8e2-arm-linux-gnueabihf.tar.gz
b6ff14e1cc36e568fc726b50300f77498560322b3582738eb70e7144784f7e63 guix-build-978852aad8e2/output/arm64-apple-darwin/SHA256SUMS.part
2a3e6ba5843eaf9562e9dcfdb4595024a71738079cea00e391558feca4d5bde1 guix-build-978852aad8e2/output/arm64-apple-darwin/bitcoin-978852aad8e2-arm64-apple-darwin-unsigned.dmg
ffd2ad39e8b9f95dd714513ba1e1c77666b0f3cc4b67be4ab763ebd431fe9276 guix-build-978852aad8e2/output/arm64-apple-darwin/bitcoin-978852aad8e2-arm64-apple-darwin-unsigned.tar.gz
5c9f8acd1777effc1e860b64143ba9d06ba5e3d0330e7341529eeae5cc6b3c5e guix-build-978852aad8e2/output/arm64-apple-darwin/bitcoin-978852aad8e2-arm64-apple-darwin.tar.gz
e34c693ecef6159c57fdedabff9dc3d69ec20387966083b828532c58e1e6e30b guix-build-978852aad8e2/output/dist-archive/bitcoin-978852aad8e2.tar.gz
8de4681114d96425bf9b0ccc47d49f696293ead514faa3fa83ddcccfdca2eeb2 guix-build-978852aad8e2/output/powerpc64-linux-gnu/SHA256SUMS.part
d780330a105931eb4c66a536c332eb09e4b6d8c288832bb5a74b931c4600c0b3 guix-build-978852aad8e2/output/powerpc64-linux-gnu/bitcoin-978852aad8e2-powerpc64-linux-gnu-debug.tar.gz
af036a6d714ef362a2facfe4e5c63deaa9dfa391d20542ead9ffb4a43b2b7ca2 guix-build-978852aad8e2/output/powerpc64-linux-gnu/bitcoin-978852aad8e2-powerpc64-linux-gnu.tar.gz
dc6b4365632e7de386ead512b1cdcdd3c985623f63cff8b62974bd9ed8c05d5d guix-build-978852aad8e2/output/powerpc64le-linux-gnu/SHA256SUMS.part
5543a6dec58515186b71139a4a5c603d85638672f205e7c9fabb281c98018461 guix-build-978852aad8e2/output/powerpc64le-linux-gnu/bitcoin-978852aad8e2-powerpc64le-linux-gnu-debug.tar.gz
dd791be2e61ce6cbd3e14c165ce2f8c2d22881992e0df72bd338423d092cc467 guix-build-978852aad8e2/output/powerpc64le-linux-gnu/bitcoin-978852aad8e2-powerpc64le-linux-gnu.tar.gz
2b740db8e5b9c435be3e7b186c3b4a40885302243326ec990e24fe4ba4f777da guix-build-978852aad8e2/output/riscv64-linux-gnu/SHA256SUMS.part
de899c89874d4f34afeca179a6c7c8f49b0a975983ab2b31afd9f2d365674578 guix-build-978852aad8e2/output/riscv64-linux-gnu/bitcoin-978852aad8e2-riscv64-linux-gnu-debug.tar.gz
9052b1db22c56692d99a61c3783b36c6f76537d9aec14f17d87a155beac82532 guix-build-978852aad8e2/output/riscv64-linux-gnu/bitcoin-978852aad8e2-riscv64-linux-gnu.tar.gz
5e79ddf57a94c5978ad819896786107f735d5742bbd042c2c64ae2d0681ce53a guix-build-978852aad8e2/output/x86_64-apple-darwin/SHA256SUMS.part
96443ad839f87c723db1c0a96d8ead0afc69e9d96ad45b5814344866da2dae73 guix-build-978852aad8e2/output/x86_64-apple-darwin/bitcoin-978852aad8e2-x86_64-apple-darwin-unsigned.dmg
14b0a3081772e81a463398a2702aca039d2f276e301dee9f5a0ccffbb09e2749 guix-build-978852aad8e2/output/x86_64-apple-darwin/bitcoin-978852aad8e2-x86_64-apple-darwin-unsigned.tar.gz
6bfb8252524202028308267f5e96bc30f284052f5feaa58ed3697dde27a3130f guix-build-978852aad8e2/output/x86_64-apple-darwin/bitcoin-978852aad8e2-x86_64-apple-darwin.tar.gz
5f8ea6297e246b08ffd806913897cc863feeec6522fcfb4456a59c5f154e0c2d guix-build-978852aad8e2/output/x86_64-linux-gnu/SHA256SUMS.part
40d1bcf491660d54fe20b2db24828ebf61be848eefdc38fba09ed43f6bdba4b1 guix-build-978852aad8e2/output/x86_64-linux-gnu/bitcoin-978852aad8e2-x86_64-linux-gnu-debug.tar.gz
3200e67a4dea115e8e341b4d71d84dc5e8bd2ae35e550cde6aef88d120c65eae guix-build-978852aad8e2/output/x86_64-linux-gnu/bitcoin-978852aad8e2-x86_64-linux-gnu.tar.gz
0b0bf7effc493ecc68398f23fc81647f64fdee115e8ccd7ae91e7881804ec328 guix-build-978852aad8e2/output/x86_64-w64-mingw32/SHA256SUMS.part
e2064c9ddeb4af18468f37ba8cf70004062c31e1387b4cc0fe4b445fae518e8d guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64-debug.zip
be347a901b896e0a1dc2f0f5a7f84614075805cccf1f2af8ec8df678d086fdbc guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64-setup-unsigned.exe
bab8700e9e266970e8c7cad494902058ad12d1f2a6462e0039daa637b1a0ce0d guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64-unsigned.tar.gz
c8e55e64b248fd7c9056fe811a1eba992bbb92e44857204e3024416d9ba6307d guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64.zip
```
ACKs for top commit:
TheCharlatan:
tACK 978852aad8e295c26801e17f2ee9e58edd4a5703
Tree-SHA512: c195484274433039e327d44b1949afa296e09e7470a2b138b7a8476c8bf9c1302bc21284cd5436f09aa97824aae9f362b7932ff2937b78f79df0b43e50f3dfaa
affbf58a1e52a8e60c830be6a9e0347e0ff4c28e build: Move environment variables into `$(package)_config_env` (Hennadii Stepanov)
d44fcd3c976572883bbf7f386bc88e2610dc1a58 build: Make $(package)_*_env available to all $(package)_*_cmds (Hennadii Stepanov)
Pull request description:
On master (1e7564eca8a688f39c75540877ec3bdfdde766b1) the depends build system, which is based on pure GNU Make, works, but it lacks robustness, and in some corner cases it fails. For example, see bitcoin/bitcoin#22552.
Another [bug](https://github.com/bitcoin/bitcoin/issues/22719) in the depends build system has already become a problem at least two times in the past (https://github.com/bitcoin/bitcoin/pull/16883#issuecomment-683817472 and https://github.com/bitcoin/bitcoin/pull/24134). Each time the problem was solved with other means.
The initial [solution](https://github.com/bitcoin/bitcoin/pull/19882) had some discussion. Also it was discussed on the IRC meeting in #bitcoin-core-builds channel. This PR, actually, is a resurrection of it, as the bug silently struck pretty [recently](https://github.com/bitcoin/bitcoin/pull/24134).
The bug is well described in bitcoin/bitcoin#22719.
Here is another, a bit simpler description, which requires only basic shell (bash, dash etc) experience.
After creating targets by this code:1e7564eca8/depends/funcs.mk (L280) a "draft" line of recipe like `$($(1)_config_env) $(call $(1)_config_cmds, $(1))` becomes a shell command sequence `VAR1=foo VAR2=bar command1 && command2` which is supposed to be executed in a [new sub-shell](https://www.gnu.org/software/make/manual/html_node/Execution.html#Execution).
Please note that `VAR1=foo VAR2=bar` part is visible for the first `command1` only (for details see shell docs). Example:
```sh
$ VAR1="foo" VAR2="bar" echo "begin" && printenv VAR1 && printenv VAR2 && echo "end"
begin
$ echo $?
1
```
Using the `export` command is a trivial solution:
```sh
$ export VAR1="foo" VAR2="bar"; echo "begin" && printenv VAR1 && printenv VAR2 && echo "end"
begin
foo
bar
end
$ echo $?
0
```
As a [new sub-shell](https://www.gnu.org/software/make/manual/html_node/Execution.html#Execution) is invoked for each line of the recipe, there are no side effects of using `export`. It means this solution should not be considered invasive.
Fixesbitcoin/bitcoin#22719.
---
Also this PR removes no longer needed crutch from `qt.mk`.
ACKs for top commit:
fanquake:
ACK affbf58a1e52a8e60c830be6a9e0347e0ff4c28e
Tree-SHA512: 0ce2cf82870a7774bdf1592fac50857126ae47da902e349f1092d50109223be9d6a8efd5e92ec08c2ca775b17516482aabaf232378950ade36484a883acc177b
76045bb9d6808931cd0f2933203b5b611e032ec8 depends: always set CMAKE_POSITION_INDEPENDENT_CODE=ON (fanquake)
d04623678c70ff58a20fb5c35d33cb8f483f1efb depends: always set CMAKE_INSTALL_LIBDIR=lib/ (fanquake)
Pull request description:
Set `CMAKE_INSTALL_LIBDIR=lib/` and `CMAKE_POSITION_INDEPENDENT_CODE=ON` globally in depends, rather than per-package. `CMAKE_INSTALL_LIBDIR=lib/` is needed to override the annoying [`GNUInstallDirs`](https://cmake.org/cmake/help/latest/module/GNUInstallDirs.html) `lib` vs `lib64` behaviour, and we always want PIC code. The PIC commit is the counterpart to the same Autotools change in #29488. I'm PRing these commits as I have a CMake branch building on top, and want to avoid adding the same workarounds to every package we are going to touch, but these can go in separately as the build should be tested for existing packages (i.e multiprocess).
ACKs for top commit:
hebasto:
re-ACK 76045bb9d6808931cd0f2933203b5b611e032ec8.
theuni:
utACK 76045bb9d6808931cd0f2933203b5b611e032ec8. Both changes make sense to me, and both can be overridden if needed, though I can't imagine we'd need to.
Tree-SHA512: 655a0b6b7ee5a5820f52e8e919ef03fc216d29f13f3904f72b64ce57436510e073c903039488d5740535c56e1f6221267229238c5231de5f8467d238fd562578
63c0c4ff10b2f6ed8510ef372a5b1f6a6494b179 depends: reuse _config_opts for CMake options (Cory Fields)
Pull request description:
Context: I'm [currently experimenting with building our depends with CMake when possible](https://github.com/theuni/bitcoin/commits/depends-cmake) as part of our future transition to CMake. Specifically zmq, libevent, libnatpmp, and miniupnpc all have existing CMake buildsystems. Building them with CMake will allow us to drop several deps that we currently have (autoconf, automake, pkg-config, etc) which would be unfortunate to carry over after the switch-over.
But that's not relevant for this PR. This is just a very simple change that makes the above work easier to experiment with as it [adds a needed feature for CMake packages](5733dc2000 (diff-e6ed342a25092e0a6d0308e0bfd826044578847132cc6726ac4afa2ca767b61aR20)). It's a no-op for the current builds.
---
From commit description:
This will allow us to use the existing machinery for filtering by arch, os, debug/release, etc.
For example, the following becomes possible for libevent when building with CMake:
`$(package)_config_opts_release=-DEVENT__DISABLE_DEBUG_MODE`
Now the define is only set when not building depends with DEBUG=1
ACKs for top commit:
hebasto:
ACK 63c0c4ff10b2f6ed8510ef372a5b1f6a6494b179
Tree-SHA512: 17d123219d2f98c017880196966ffebd5d09d3e2db8e443e227eb7e49da46e9d971fbe7fd2b99a324fc367e1bbe0ac3cd15b399bce98dd87fbb144d767e15fe7
86c2889518df0234b8b28112a9771b85c577eefd ci: Make log verbose in error case only (Hennadii Stepanov)
7f650883b71b67cd33026cf49af0b16f01e45541 depends: Add file-based logging for individual packages (Hennadii Stepanov)
Pull request description:
This PR adds file-based logging for individual packages in depends. To use this feature one should provide `LOG=1`.
A log file is printed out automatically in case of a build error. After successful build log files are being moved along with package archives:
```
$ make -C depends HOST=x86_64-w64-mingw32 LOG=1
$ find ./depends/built/x86_64-w64-mingw32 -name '*.log' | sort
./depends/built/x86_64-w64-mingw32/bdb/bdb-4.8.30-5100a099801.log
./depends/built/x86_64-w64-mingw32/boost/boost-1_71_0-313f82dc7de.log
./depends/built/x86_64-w64-mingw32/libevent/libevent-2.1.12-stable-3fa27048d5e.log
./depends/built/x86_64-w64-mingw32/libnatpmp/libnatpmp-4536032ae32268a45c073a4d5e91bbab4534773a-9db4850dd32.log
./depends/built/x86_64-w64-mingw32/miniupnpc/miniupnpc-2.2.2-75d9a1807e0.log
./depends/built/x86_64-w64-mingw32/native_b2/native_b2-1_71_0-3bf253c19bf.log
./depends/built/x86_64-w64-mingw32/qrencode/qrencode-3.4.4-dfac87af599.log
./depends/built/x86_64-w64-mingw32/qt/qt-5.15.2-9304e03d3ac.log
./depends/built/x86_64-w64-mingw32/sqlite/sqlite-3320100-455acafa7be.log
./depends/built/x86_64-w64-mingw32/zeromq/zeromq-4.3.1-5ff627ec84a.log
```
An example of CI tasks with package build errors -- https://cirrus-ci.com/task/5275741788045312Closes#16368.
ACKs for top commit:
laanwj:
Tested ACK 86c2889518df0234b8b28112a9771b85c577eefd
Tree-SHA512: 497f2146fd2e38c952124aecfd80ebb42be22bbc5dc59521491545f4465fc38f23da7787a0caea5686b7c30aa862f2b0c02092ae3fe863e80a5ddd14b3d324b9
e037c4fe0914d8fa9149ce7532c0d70f738e79e9 depends: always configure with --with-pic (fanquake)
Pull request description:
We currently do this sporadically. Not only amongst packages, but across OS's, i.e sometimes it's done for BSDs/Android, and sometimes not.
Configure with `--with-pic` globally instead. I think this generally makes more sense, and should not have any downsides.
See related discussion in https://github.com/bitcoin/bitcoin/pull/28846#discussion_r1399123100.
ACKs for top commit:
hebasto:
ACK e037c4fe0914d8fa9149ce7532c0d70f738e79e9.
Tree-SHA512: efc743ff92f9f99f3ac16514e98363ad395c6f956cd4be7e785b5c573685baf7fcd68c51d6a705ee8761fc676eb045b7e61676595be0eb0f70f34e99174cddc0
c0f5cc14ef9fae2b2de4222ee061729629ebb6b4 build: Fix `libmultiprocess` cross-compiling to Linux hosts (Hennadii Stepanov)
Pull request description:
To successfully call the [`capnp_generate_cpp()`](d576d975de/CMakeLists.txt (L45)) function, the `libmultiprocess` build system must be provided with paths to the native `capnp` and `capnpc-c++` tools.
This [comment](https://github.com/bitcoin/bitcoin/issues/24387#issuecomment-1054776195) points the same:
> I think `packages/libmultiprocess.mk` probably needs to be passing a `-DCAPNP_EXECUTABLE=.../depends/arm-linux-gnueabihf/native/bin/capnp` argument to cmake. Also the package should have dependencies on both `capnp` and `native_capnp`.
Fixesbitcoin/bitcoin#24387.
ACKs for top commit:
ryanofsky:
Code review ACK c0f5cc14ef9fae2b2de4222ee061729629ebb6b4
Tree-SHA512: 2986d8bf98d2761eceba21b1897145c5185a0922d4c2084e8812d4d07dc94237e5c2809036641c4f7c491a3414727fff328cba91ce138b89e37ec5cba61d8f61
735610940c0cac72343ca247eb3062b2e09512a0 build: set --build when configuring packages in depends (fanquake)
Pull request description:
After reading [#Specifying-Target-Triplets](https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/autoconf.html#Specifying-Target-Triplets) in the autoconf manuel, my understanding is that this change should mostly be a no-op, as `--build` defaults to the output of [`config.guess`](https://github.com/bitcoin/bitcoin/blob/master/depends/config.guess), however, this may be slightly more correct
> For historical reasons, whenever you specify --host, be sure to specify --build too; this will be fixed in the future.
and will quell some warnings in depends (#16354), i.e:
> configure: WARNING: If you wanted to set the --build type, don't use --host.
If a cross compiler is detected then cross compile mode will be used.
If anything, this also explicitly enables cross-compilation mode when `--host` differs from `--build`. As for "fixed in the future", this is the case for Autoconf 2.70+.
If we don't make this change, I think we should consider closing #16354. We've now got a good handle on our depends configure flags, and we can just leave the last few warnings as they are. It's also unlikely that we are going to add the `xorg-macros` package to depends.
Guix builds at 735610940c0cac72343ca247eb3062b2e09512a0:
```bash
bash-5.1# find output -type f -name *$(git rev-parse --short HEAD)*.* -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
afabf99650d57abd3df2c7e6253a3281f9814ada77d01c28ffb57c740b14fd8e output/bitcoin-735610940c0c-aarch64-linux-gnu-debug.tar.gz
6e9dedced61489f7734c5387ee05823db68eb4b890fa05f11541c1b966576446 output/bitcoin-735610940c0c-aarch64-linux-gnu.tar.gz
5333a3c5978a61966251e56fedab8e059d9d8a1a7231a0908095bb373204a81f output/bitcoin-735610940c0c-arm-linux-gnueabihf-debug.tar.gz
d28e1c166b96e58cf40b3f9801c3050c2785b25eeb261d44eeb31a32a75e134c output/bitcoin-735610940c0c-arm-linux-gnueabihf.tar.gz
e00b0149c52e9207be1cfb56ec3b715b55f41691401095f7a7038ac124f08d80 output/bitcoin-735610940c0c-osx-unsigned.dmg
ae5052ad1a05916bc6656b034c169097471fd9bdba16b606e2bcb592c9395544 output/bitcoin-735610940c0c-osx-unsigned.tar.gz
8586a9b8b82a8832397fcd0756aa9649adc3e61ae579fb370b99fe406298199f output/bitcoin-735610940c0c-osx64.tar.gz
d560a8781bae856b6593e9b731fa99de0a4ad7bd3e03084166dc98904ece9e7b output/bitcoin-735610940c0c-powerpc64-linux-gnu-debug.tar.gz
a24cf28b7efd5ebdc892e7e8809a05b3405bb4222f7d2eaa32de265214c78f28 output/bitcoin-735610940c0c-powerpc64-linux-gnu.tar.gz
ad29421842dbbe96526eb597a7ff2970f0acae8c3e75dc6c2100716a2ba20606 output/bitcoin-735610940c0c-powerpc64le-linux-gnu-debug.tar.gz
ee7a8ccfd68cc2297bb96eaaacf7b3eb939a5e39cecc3c710dbe828518f1bfbd output/bitcoin-735610940c0c-powerpc64le-linux-gnu.tar.gz
185bf4ea3b35072a8dd54aedbafc0892ec1e486c8ebd311f05f0fb47a19058cf output/bitcoin-735610940c0c-riscv64-linux-gnu-debug.tar.gz
803078b15a9bbaa567176827639895e762edbe80844b80400f8a9581e5311d32 output/bitcoin-735610940c0c-riscv64-linux-gnu.tar.gz
d920327a32a42a5abadd247f0d3f3ee0ab0488a28892c11838d99bede6456723 output/bitcoin-735610940c0c-win-unsigned.tar.gz
9f1741a37e294af153d57ad4a6dad3e5f1cdf75f1588c86cb467abf0c177ae27 output/bitcoin-735610940c0c-win64-debug.zip
2d1a91552c0dd7bb400ce6c71d2e98de702743a97afe5f4b0945865f251f3aea output/bitcoin-735610940c0c-win64-setup-unsigned.exe
8a70020bb0cb68516acf4b2cc89832d72373f31611bd075826a30a817a071cd2 output/bitcoin-735610940c0c-win64.zip
93b03837679474905538e96c44ba81ee5653795ad333dc00569b92c82dc6ea31 output/bitcoin-735610940c0c-x86_64-linux-gnu-debug.tar.gz
6ccd046a83ef5089667ad426875319beb83e2b2a5fb4d4459d5c5ce8d75f1a31 output/bitcoin-735610940c0c-x86_64-linux-gnu.tar.gz
12771acb17de84cda8e81c59704c00b1d4725ff51fa84c9067dc61a8dc795bdc output/src/bitcoin-735610940c0c.tar.gz
```
Gitian builds:
```bash
# macOS:
7d087749f610e5e3eca91c730d41afd4d404808e276242d05cedb89403de247f bitcoin-735610940c0c-osx-unsigned.dmg
96d68b4f0042dc40f5e8e362fa84dcf8e122dcb566889feb48ad8a982847dd8b bitcoin-735610940c0c-osx-unsigned.tar.gz
2dea72cbe5c3e228babd88b516a6f43b0e040e9510ea95b62f07fc0815bf8d22 bitcoin-735610940c0c-osx64.tar.gz
12771acb17de84cda8e81c59704c00b1d4725ff51fa84c9067dc61a8dc795bdc src/bitcoin-735610940c0c.tar.gz
478dbda9d551a8bed985c61c625df3f543afab26989a52bc44d5a4745715af81 bitcoin-core-osx-22-res.yml
# Windows:
c31a5118dfa21259b9868e2c4858c635becb786c2ee8af92302da7811a1efe9b bitcoin-735610940c0c-win-unsigned.tar.gz
5c171ec3aee830abc8ae843d674d5fe7a57420e782ee765b6992914ae5b7671a bitcoin-735610940c0c-win64-debug.zip
ca010ced982f75d111e01f3759b961deb83e7bf511b1ee9c9041f4830570f2a0 bitcoin-735610940c0c-win64-setup-unsigned.exe
62514b01281f81af6057fb12cc5e3813c1d8f6532d0d8ac8f6f2909a595aa1dd bitcoin-735610940c0c-win64.zip
12771acb17de84cda8e81c59704c00b1d4725ff51fa84c9067dc61a8dc795bdc src/bitcoin-735610940c0c.tar.gz
e26b55d8302a419f594396db6ff441aaeb5e275e02fde7b1b4eb092270f902e7 bitcoin-core-win-22-res.yml
# Linux:
e5be4deb91cb372de484468c0575a04f907f982e35a9841659bb213540f770d0 bitcoin-735610940c0c-aarch64-linux-gnu-debug.tar.gz
6169e822679ece2bb6af7027362a6f373eb939eef9d89edc58c21190b4083c7d bitcoin-735610940c0c-aarch64-linux-gnu.tar.gz
aee5308767c5c930853a80ab94325b2a4ab3dbe9a7a20c7a9d60d57af2b5383f bitcoin-735610940c0c-arm-linux-gnueabihf-debug.tar.gz
a9cf6e56d859fffd5a2b5f1e6360515bc6b06b048776b93cc08954859457a251 bitcoin-735610940c0c-arm-linux-gnueabihf.tar.gz
7617d85c358e6614fd40026a8d346f9edba5b5139e3daade2ec2aec72aed3a1f bitcoin-735610940c0c-powerpc64-linux-gnu-debug.tar.gz
f76e46a8b130812541ebcaaba173b4cbe2ec1781e348579b44f96e9244cc2475 bitcoin-735610940c0c-powerpc64-linux-gnu.tar.gz
19ada7abb801422bd7772d7ee559004cab11a46141a747aabd27085e8c3d200d bitcoin-735610940c0c-powerpc64le-linux-gnu-debug.tar.gz
f3a6bbefe9f47b3960e3b9eeb3f740ecddf8773ae76961ced6a3cdf82503e37f bitcoin-735610940c0c-powerpc64le-linux-gnu.tar.gz
42ca333e8f82b4ec0b2354efc7c4f1c27ed32d60c35b88e33d26833419f19e69 bitcoin-735610940c0c-riscv64-linux-gnu-debug.tar.gz
dde930d9134fb5be1374014db3f6f503b614ae801a1d77f6f4b82a5f11eb2518 bitcoin-735610940c0c-riscv64-linux-gnu.tar.gz
82e1b4cc8aac309a99458cc27fd0285215a8af53780094c4990303ce8948cf0c bitcoin-735610940c0c-x86_64-linux-gnu-debug.tar.gz
ee8caad2ccddd8eaf8aa2d19040d859130168183355b09a2a0f77a59a97fcaee bitcoin-735610940c0c-x86_64-linux-gnu.tar.gz
12771acb17de84cda8e81c59704c00b1d4725ff51fa84c9067dc61a8dc795bdc src/bitcoin-735610940c0c.tar.gz
d90545b13226a338c994d9e538f468a6fafd694273bf71e14614feeaf5727ad4 bitcoin-core-linux-22-res.yml
```
ACKs for top commit:
dongcarl:
Code Review ACK 735610940c0cac72343ca247eb3062b2e09512a0
hebasto:
ACK 735610940c0cac72343ca247eb3062b2e09512a0, tested on Linux Mint 20.1 (x86_64).
Tree-SHA512: 13ce05269309a12d03b81a539ba5585003785f7b1bb7894a9353385a6e9c709f920cc4f79322f28d7a256809754f201e70be6b917915c7b14e1461530075781d
7d0271b5c30f86e4af175a5ab7df5e593cd85195 depends: Set CMAKE_INSTALL_RPATH for native packages (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
After #19685 started setting `LDFLAGS`, the `INSTALL_RPATH_USE_LINK_PATH` cmake option used in the libmultiprocess build no longer works, so it is neccessary to set `CMAKE_INSTALL_RPATH` as a fallback.
It's unclear currently whether the bad interaction between `INSTALL_RPATH_USE_LINK_PATH` and `LDFLAGS` is a bug, but the issue is reported:
- https://github.com/bitcoin/bitcoin/issues/19981#issuecomment-696680877
- https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892Fixes#19981
ACKs for top commit:
fanquake:
ACK 7d0271b5c30f86e4af175a5ab7df5e593cd85195 - I haven't looked in depth, but I've re-read through #19981 and checked the failure by testing #19160 (with this reverted):
dongcarl:
ACK 7d0271b Looked into this a bit, it makes sense that for the things we build in depends, we want the library search to start in depends. It seems reasonable to expect this to happen automatically when `CMAKE_INSTALL_PREFIX` and `INSTALL_RPATH_USE_LINK_PATH` are set, but oh well...
Tree-SHA512: 97cc5801c3204c14cd33004423631456ca0701e2127ee5146810a76e2f4aac9de1f4b5437402a4329cda54e022dc99270fee7e38c2995765f36b3848215fa78e
b8936883573708059357a66f67fad9dc77a8bade depends: Specify LDFLAGS to cmake as well (Carl Dong)
b3f541f618fe1f3d44baf6a0dd4299173c81f752 depends: Prepend CPPFLAGS to C{,XX}FLAGS for CMake (Carl Dong)
8e121e550953711cd03d7b6c221afd065c325c5e depends: Cleanup CMake invocation (Carl Dong)
8c7cd0c6d9f295bcb6913e3c69c9dac4ce2b25ce depends: More robust cmake invocation (Carl Dong)
3ecf0eca634601da216b06f091f95456c047f39c depends: Use $($(package)_cmake) instead of cmake (Carl Dong)
Pull request description:
- Use `$($(package)_cmake)` instead of invoking `cmake` directly
- Use well-known env vars instead of overriding CMake variables
ACKs for top commit:
ryanofsky:
Code review ACK b8936883573708059357a66f67fad9dc77a8bade. Only changes since last review are new commits adding whitespace, cppflags and ldflags to cmake invocation
Tree-SHA512: cfcd8cc9dcd0b336cf48b82fca9fe4bbc7930ed397cb7a68a07066680eb4c1906a6a9b5bd2589b4b4999e8f16232fa30ee9b376b60f4456d0fff931fbf9cc19a
e2bab2aa162ae38b2bf8195b577c982402fbee9d multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e708c04ef6c9880f89d0a4cbaa6fc7c5 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52bfcd4edf8553aaf332bfeb92fc554cc build: multiprocess autotools changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.
In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.
The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649
ACKs for top commit:
practicalswift:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d
Sjors:
tACK e2bab2aa162ae38b2bf8195b577c982402fbee9d on macOS 10.15.4
hebasto:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d, tested on Linux Mint 19.3 (x86_64):
Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
e644591426fb4ee1bba5c4bfdde85eb378258272 build, refactor: Drop useless `call` Make function (Hennadii Stepanov)
Pull request description:
Using the [`call`](https://www.gnu.org/software/make/manual/html_node/Call-Function.html) function with `$(package)_*_cmds` is effectively noop because the latter, which could be found in `<package>.mk` files, do not use temporary `$(1)` variable at all.
This PR removes useless calls of the `call` function, and makes code more readable and easier to reason about.
No change in resulted dependency binaries could be easy verified with bitcoin/bitcoin/#21995.
ACKs for top commit:
laanwj:
Code review ACK e644591426fb4ee1bba5c4bfdde85eb378258272
shaavan:
Code review ACK e644591426fb4ee1bba5c4bfdde85eb378258272
Tree-SHA512: 8481fa0dc5bbf7dd6a180f7fae5a2ccc07f85b50c7a966bceb2d7e010e07e5f211ee3f74f8ac79bc5acfde5f0764264d599d959ff3ebb8511b1b4a33f79509bd
8494dcae0e32716fd7cc7abeacf0a795a1303e6a Replace $(AT) with .SILENCE. (Dmitry Goncharov)
Pull request description:
This reduces the amount of syntax noise in the makefiles.
Setting V=1 still enables verbose logging.
The only noticeable difference in behavior is that, unless V=1 is specified, make won't print its own messages like
make: Nothing to be done for 'all', make: 'all' is up to date, or touch <file>, if -t is specified.
ACKs for top commit:
laanwj:
Tested ACK 8494dcae0e32716fd7cc7abeacf0a795a1303e6a
Tree-SHA512: 66b9111229995aa54a9e87f4571648727d89b8529caec651063cdfe5c00a64341371b648701d192b2334df0614617a00c28eaa56c7f08ee9c00127cada0293ab
fa872c9af397837bb17859b5f43adec71239682a depends: Fix id_string invocations (Carl Dong)
Pull request description:
Closes: #21242
```
Reproduced from depends/Makefile comment:
When invoking a shell, GNU Make special-cases exit code 127 (command not
found) by not capturing the output but instead passing it through. This
is not done for any other exit code.
Therefore, we require a "|| true" to avoid this behaviour when in an
environment where the build_* or host_* may not exist yet.
```
ACKs for top commit:
laanwj:
Concept and light code review ACK fa872c9af397837bb17859b5f43adec71239682a
Tree-SHA512: 9ce88381aec579d956572cf70c4f69dc5a3873f0d2af14a71cf24814192a89452b8280258bed8cca804e4bd2644db056d213ab733df46a10560a47079524d8ac
1ba49bcdc216162f513e74e371ee9f26418a390e build: pass --enable-option-checking to applicable packages (fanquake)
bcff8e21b1bb360c3cf1e1d2e1a2a54ec7581720 build: only pass --disable-dependency-tracking to packages that understand it (fanquake)
Pull request description:
By blanket passing `--disable-dependency-tracking` to all depends packages we end up with warnings (i.e in `bdb` or `freetype`) like:
```bash
configure: WARNING: unrecognized options: --disable-dependency-tracking
```
Instead, only pass it to packages that actually understand it. Related to https://github.com/bitcoin/bitcoin/issues/16354.
More info on `--disable-dependency-tracking` available [here](https://www.gnu.org/software/automake/manual/html_node/Dependency-Tracking.html).
This PR also adds `--enable-option-checking` as a configure option to all applicable packages.
ACKs for top commit:
laanwj:
ACK 1ba49bcdc216162f513e74e371ee9f26418a390e
theuni:
ACK 1ba49bcdc216162f513e74e371ee9f26418a390e
Tree-SHA512: 6d3143ad5f5d1abed5e0a0b2ffbb4323f21c7bf24b0b8df26fb1b3cd16cf5309bbb830aa5aaec99164d5bbe8e9c62b97aa3e97ee1ddc2c7612bf8ff88a63885e
ef77549000ac46eededa3e4a89466024db111524 build: Include native_X.mk before X.mk (Hennadii Stepanov)
Pull request description:
Unfortunately, our depends build system still lacks 100% reliability.
On master (57982f419e36d0023c83af2dd0d683ca3160dc2a):
```
$ make --no-print-directory -C depends print-capnp_version MULTIPROCESS=1
capnp_version=
```
This PR fixes this issue:
```
$ make --no-print-directory -C depends print-capnp_version MULTIPROCESS=1
capnp_version=0.7.0
```
This PR split off from bitcoin/bitcoin#22552, bitcoin/bitcoin#22555 and bitcoin/bitcoin#22708.
ACKs for top commit:
Zero-1729:
tACK ef77549000ac46eededa3e4a89466024db111524
fanquake:
ACK ef77549000ac46eededa3e4a89466024db111524
Tree-SHA512: 9818277c82fcef778282a9fe854e48c6b6118cc757fcffa6fd4a2418e3de8b2a2b9d54e31804d22a53b28a28ffa5b8610c983add1969be0c4b3e13e0f2267271
7a89f2e6c539a54bcaa24bff41aae3910244ad3d build: Fix target name (Hennadii Stepanov)
Pull request description:
It seems like a typo :)
This PR:
- fixes errors when building a package in depends for `HOST=x86_64-apple-darwin16` (fix#19799)
- is a correct alternative to d25e0e308f from #19764
ACKs for top commit:
icota:
tACK 7a89f2e6c5
dongcarl:
Code Review ACK 7a89f2e6c539a54bcaa24bff41aae3910244ad3d
theuni:
ACK 7a89f2e6c539a54bcaa24bff41aae3910244ad3d.
Tree-SHA512: a0bcbc6805d3450e201476ef1e22e0eb53903db1586c5515314c19afd337bded887e56de0fbe62feaf359b2de15dbccd49a44f1a8b566b4c64f5ae3d94a2ab6d
6d44c5ebf9 depends: Add commands for each package for each stage (Carl Dong)
80f0e05b70 depends: Preprocessing doesn't care about deps (Carl Dong)
Pull request description:
Adds make targets for each package for each stage, e.g.
```sh
make zeromq_configured
```
ACKs for commit 6d44c5:
MarcoFalke:
ACK 6d44c5ebf97af4b357079fe4bc2130f98e1d0fd2 (Haven't looked at the code changes, but adding this feature makes sense)
ryanofsky:
ACK 6d44c5ebf97af4b357079fe4bc2130f98e1d0fd2
Tree-SHA512: f1ac0aecfd2372aed09ca63603e2634552cb3f6ff9d610f958e2a66952d7d9e870b4c32b7d996886879e6d3016532272e8b1a10c13ed7b31009c6c96f786db9f
f7696e6183 depends: qt: Don't hardcode pwd path (Carl Dong)
89bee1bdbf depends: tar: Always extract as yourself (Carl Dong)
340ef50772 depends: Defer to Python detected by autoconf (Carl Dong)
Pull request description:
Removes some implicit assumptions that the depends system has about its environment and, as a side-effect, makes it possible to build the depends tree under severely privilege-limited environments such as containers built by Guix.
Tree-SHA512: e8618f9310a0deae864b44f9b60baa29e6225ba16817973ff7830b55798ebd4343aa06da6c1f92682a7afb709d26f80d6ee794a139d4d44c27caf4f0c8fe95fc
* Retry downloading of depends sources 3 times
* Manually invoke apt-get update and install instead of using addon
This allows us to use travis_retry on these operations.
* Add exception for depends/Makefile in .gitignore
* Remove --retry from curl call
This avoids future confusion as we don't rely on curl anymore to properly
retry downloads. curls does only retry on transient errors, while Travis
might cause all kinds of errors which circumvent the retry logic of curl.
In some cases (Travis), sources and build caches may be moved around in-between
builds, and we can't necessarily trust that everything is still intact.
This introduces pre-build checks that verify against stashed checksums.
Note that this will cause all sources to be re-downloaded, since cached sources
weren't trustworthy before this.
Broken hash logic caused all depends on some platforms (osx at least) to end up
with the same build-id. Without this fix, nothing will be rebuilt when recipes
or dependencies change.
Since the last commit will force rebuilds of all depends, take the opportunity
to clean up a few other things that would trigger rebuilds as well.
- Move source stamps to the sources dir so that SOURCES_PATH is respected for
"make download".
- Only print "fetching..." when actually downloading a file.
- Avoid using non-deterministic paths for the recipe hash (patch location).
This should ensure that all builders get the same resulting build-ids.
- Use a per-package source paths. This will allow for removing old source files
in the future.
- Use a host-agnostic path for downloads which gets cleaned up properly.
Some sources are renamed after download, since the filenames don't play nice
with (for example) gitian. This fixes the rename.
Needed for OSX build as it renames a file.
