7d0271b5c30f86e4af175a5ab7df5e593cd85195 depends: Set CMAKE_INSTALL_RPATH for native packages (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
After #19685 started setting `LDFLAGS`, the `INSTALL_RPATH_USE_LINK_PATH` cmake option used in the libmultiprocess build no longer works, so it is neccessary to set `CMAKE_INSTALL_RPATH` as a fallback.
It's unclear currently whether the bad interaction between `INSTALL_RPATH_USE_LINK_PATH` and `LDFLAGS` is a bug, but the issue is reported:
- https://github.com/bitcoin/bitcoin/issues/19981#issuecomment-696680877
- https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892Fixes#19981
ACKs for top commit:
fanquake:
ACK 7d0271b5c30f86e4af175a5ab7df5e593cd85195 - I haven't looked in depth, but I've re-read through #19981 and checked the failure by testing #19160 (with this reverted):
dongcarl:
ACK 7d0271b Looked into this a bit, it makes sense that for the things we build in depends, we want the library search to start in depends. It seems reasonable to expect this to happen automatically when `CMAKE_INSTALL_PREFIX` and `INSTALL_RPATH_USE_LINK_PATH` are set, but oh well...
Tree-SHA512: 97cc5801c3204c14cd33004423631456ca0701e2127ee5146810a76e2f4aac9de1f4b5437402a4329cda54e022dc99270fee7e38c2995765f36b3848215fa78e
b8936883573708059357a66f67fad9dc77a8bade depends: Specify LDFLAGS to cmake as well (Carl Dong)
b3f541f618fe1f3d44baf6a0dd4299173c81f752 depends: Prepend CPPFLAGS to C{,XX}FLAGS for CMake (Carl Dong)
8e121e550953711cd03d7b6c221afd065c325c5e depends: Cleanup CMake invocation (Carl Dong)
8c7cd0c6d9f295bcb6913e3c69c9dac4ce2b25ce depends: More robust cmake invocation (Carl Dong)
3ecf0eca634601da216b06f091f95456c047f39c depends: Use $($(package)_cmake) instead of cmake (Carl Dong)
Pull request description:
- Use `$($(package)_cmake)` instead of invoking `cmake` directly
- Use well-known env vars instead of overriding CMake variables
ACKs for top commit:
ryanofsky:
Code review ACK b8936883573708059357a66f67fad9dc77a8bade. Only changes since last review are new commits adding whitespace, cppflags and ldflags to cmake invocation
Tree-SHA512: cfcd8cc9dcd0b336cf48b82fca9fe4bbc7930ed397cb7a68a07066680eb4c1906a6a9b5bd2589b4b4999e8f16232fa30ee9b376b60f4456d0fff931fbf9cc19a
e2bab2aa162ae38b2bf8195b577c982402fbee9d multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e708c04ef6c9880f89d0a4cbaa6fc7c5 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52bfcd4edf8553aaf332bfeb92fc554cc build: multiprocess autotools changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.
In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.
The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649
ACKs for top commit:
practicalswift:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d
Sjors:
tACK e2bab2aa162ae38b2bf8195b577c982402fbee9d on macOS 10.15.4
hebasto:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d, tested on Linux Mint 19.3 (x86_64):
Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
060a2a64d40d75fecb60b7d2b9946a67e46aa6fc ci: remove boost thread installation (fanquake)
06e1d7d81d5a56d136c6fc88f09a2b0654a164f9 build: don't build or use Boost Thread (fanquake)
7097add83c8596f81be9edd66971ffd2486357eb refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef834490c438436719f95cbaf888c4914 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)
Pull request description:
This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).
Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.
Two other points:
[Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
> It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.
Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
* The version of Boost.
* The platform you're building for.
* Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
* Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.
A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.
With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.
Previous similar PRs were #19183 & #20922. The authors are included in the commits here.
Also related to #21022 - pthread sanity checking.
ACKs for top commit:
laanwj:
Code review ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc
vasild:
ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc
Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
f13e642c831c5689cb2bb7f5c4f9cb4c0c03ef21 build: Disable valgrind when building zeromq package in depends (Hennadii Stepanov)
b970f03beae0f3ae6a796f0e3b97732fc579f6aa build: Disable libbsd when building zeromq package in depends (Hennadii Stepanov)
77899991b1e29a45bc377b21330148cb7cc08923 build: Update netbsd_kevent_void.patch (Hennadii Stepanov)
Pull request description:
Since v4.3.3 (068385c951) `libzmq` uses `libbsd` by default.
This PR disables `libbsd` explicitly, as it's not a part of our depends. Zeromq will fallback to its internal `strlcpy` implementation.
Otherwise, on systems with installed `libbsd-dev` package the `zeromq` package build system erroneously detects `libbsd` package from the host system:
```diff
--- a/libzmq.pc
+++ b/libzmq.pc
@@ -8,5 +8,5 @@
Version: 4.3.4
Libs: -L${libdir} -lzmq
Libs.private: -lpthread
-Requires.private:
+Requires.private: libbsd
Cflags: -I${includedir}
```
This causes the `configure` fails to detect the `zeromq` package:
```
configure: WARNING: libzmq version 4.x or greater not found, disabling
```
---
Other minor improvements:
- fixed `netbsd_kevent_void.patch` offset
- disabled valgrind as it's used in unit tests which we do not run:
```diff
--- a/zmq-configure-output
+++ b/zmq-configure-output
@@ -119,11 +119,6 @@
checking whether the g++ -m64 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
-checking for valgrind... valgrind
-checking for Valgrind tool memcheck... memcheck
-checking for Valgrind tool helgrind... helgrind
-checking for Valgrind tool drd... drd
-checking for Valgrind tool exp-sgcheck... exp-sgcheck
checking linker version script flag... --version-script
checking if version scripts can use complex wildcards... yes
checking for working posix_memalign... yes
```
ACKs for top commit:
fanquake:
ACK f13e642c831c5689cb2bb7f5c4f9cb4c0c03ef21
Tree-SHA512: d4c86d4a841eb6e7c32157e84972243072f905496c2a4c14ec6f6ab4216df6695cbf29baa2233ce27eaede35d1e250ad2f9975b16f570d01509f0c5da4597cad
6897c4bdf51a4aa74320ebfffa9467db14670109 build: patch depends zeromq to fix building on NetBSD Current (fanquake)
ce6dd2f1a2e2c56d86d00e0eeec34c9982017416 zeromq 4.3.4 (fanquake)
Pull request description:
This is a prerequisite for #23955. It updates zeromq to the latest available version, and adds a patch, [that I've sent upstream](https://github.com/zeromq/libzmq/pull/4326), to fix building on NetBSD Current (10).
ACKs for top commit:
hebasto:
ACK 6897c4bdf51a4aa74320ebfffa9467db14670109, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: d05d9753630faebe842e1ca70c8c4af660a38e7331a9d95e84df3a3b14564c5118ca41c4fc49fb71dfee563b63e1014e5a3f8874d652e26de59e8e188a12970e
## Issue being fixed or feature implemented
Fix linter error:
```
In depends/config.site.in line 66:
if test "x@host_os@" = xdarwin; then
^----------^ SC2268 (style): Avoid x-prefix in comparisons as it no longer serves a purpose.
Did you mean:
if test "@host_os@" = darwin; then
For more information:
https://www.shellcheck.net/wiki/SC2268 -- Avoid x-prefix in comparisons as ...
^---- failure generated from test/lint/lint-shell.sh
```
## How Has This Been Tested?
run linter `test/lint/lint-all.sh`
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
continued from a2687d4594 from dash#4379
includes:
- 4af59a407a207e7de327c1bb145d68ebc8ddedd8
inapplicable:
- 335bd7f8bc415d6419763e17b013f94e5f0e5689
above commit is inapplicable as boost 1.71 bump did away with the patch,
backported as 6a367fc198 in dash#4356
we're currently using boost 1.77, this replacement was dropped in its
patch form in bitcoin#19764 (backported as 6a367fc1 in dash#4356), the
comment was dropped in bitcoin#19761 but the change is in this commit
instead due to direct relevance.
e644591426fb4ee1bba5c4bfdde85eb378258272 build, refactor: Drop useless `call` Make function (Hennadii Stepanov)
Pull request description:
Using the [`call`](https://www.gnu.org/software/make/manual/html_node/Call-Function.html) function with `$(package)_*_cmds` is effectively noop because the latter, which could be found in `<package>.mk` files, do not use temporary `$(1)` variable at all.
This PR removes useless calls of the `call` function, and makes code more readable and easier to reason about.
No change in resulted dependency binaries could be easy verified with bitcoin/bitcoin/#21995.
ACKs for top commit:
laanwj:
Code review ACK e644591426fb4ee1bba5c4bfdde85eb378258272
shaavan:
Code review ACK e644591426fb4ee1bba5c4bfdde85eb378258272
Tree-SHA512: 8481fa0dc5bbf7dd6a180f7fae5a2ccc07f85b50c7a966bceb2d7e010e07e5f211ee3f74f8ac79bc5acfde5f0764264d599d959ff3ebb8511b1b4a33f79509bd
28f17c1a6de7b50482d8f55fe03698126862e8eb build: fix copypasta in OpenBSD C{XX} flags (fanquake)
Pull request description:
Introduced in #23998.
ACKs for top commit:
hebasto:
ACK 28f17c1a6de7b50482d8f55fe03698126862e8eb, I have reviewed the code and it looks OK, not tested on OpenBSD though.
Tree-SHA512: d905161534075f518c8924d3c42cca7ff8d4898e559f1daa9bd03dac95b109b2c3e76790fb8bc65b9e45e8a59566825afbf4dc3734ad74617dfdf797430e486b
339b4a51f6d3558c3489b14efe0c8c195295cf86 build: don't install deprecated libevent headers (fanquake)
Pull request description:
We don't use the deprecated headers now, and never should do in the
future, so there is no need for them to exist in depends.
The headers themselves are just full of includes for the newer headers.
ACKs for top commit:
hebasto:
ACK 339b4a51f6d3558c3489b14efe0c8c195295cf86
Tree-SHA512: 736fd9e3b22212da462cc05203dd253806dc59f973090357b705f2742ed4a3b8c3cc44b3173d706527f60ad93e95cf4143ec6b7db4233a489890a98f8e5c8f07
bcd36e14f0a2b89445f1436094de40696d6ad76c build: correct depends FreeBSD C{XX}FLAGS (fanquake)
7b06ffce9c50110b475c722918c55a14402346a5 build: add NetBSD support to depends (fanquake)
Pull request description:
Similar to #23948. Doesn't build the Qt package; I haven't looked at doing that yet, but have an assumption that it's going to fail out of the box similar to the FreeBSD build.
Guix Build:
```bash
```
ACKs for top commit:
theuni:
ACK bcd36e14f0a2b89445f1436094de40696d6ad76c
Tree-SHA512: 9a0946cefbcb9a92dd730b885463f3213e304c8d4b39fea8d831fc013a73d2ef998ca84e384bf45a01fa1449cf5a35eaffaa5b57a9062c2cdda34312d33ec3fc
edc9a6afdc6926b40de2fed897b9b866d58f28d2 build, refactor: Reuse expat package version in its download path (Hennadii Stepanov)
4bb7821ab243325467a89ff8ffc1bed290f2cb58 build, refactor: Use conventional version notation for boost package (Hennadii Stepanov)
Pull request description:
`boost` package:
- `.` is used as a separator in versions of other depends packages.
`expat` package:
- reuse package version in its download path
---
The straightforward way to verify this PR:
```
$ cd depends
$ make clean-all
$ make boost_fetched
$ make expat_fetched
```
ACKs for top commit:
prusnak:
ACK edc9a6a
shaavan:
ACK edc9a6afdc6926b40de2fed897b9b866d58f28d2
Tree-SHA512: c15d672fe34ac59850425d3d6a6eee5f720e16d227aad1332a563b218465879b7ee6fb865dd1bac06aedf356f9bb1c67112d9d88da8f877f04838b50a9dc97be
223b1ba7d90509a47ea07af46f4b9c3b8efbc9f8 doc: Use CONFIG_SITE instead of --prefix (Hennadii Stepanov)
Pull request description:
The current examples of `--prefix=...` option usage to point `configure` script to appropriate `depends` directory is not [standard](https://www.gnu.org/prep/standards/html_node/Directory-Variables.html). This causes some [confusion](https://github.com/bitcoin/bitcoin/pull/16691) and a bit of inconvenience.
Consider a CentOS 7 32 bit system. Packages `libdb4-devel`, `libdb4-cxx-devel`, `miniupnpc-devel` and `zeromq-devel` are unavailable from repos. After recommended build with depends:
```
cd depends
make
cd ..
./autogen.sh
./configure --prefix=$PWD/depends/i686-pc-linux-gnu
make
```
a user is unable to `make install` compiled binaries neither locally (to `~/.local`) nor system-wide (to `/usr/local`) as `--prefix` is set already.
Meanwhile, the standard approach with using [`config.site`](https://www.gnu.org/software/automake/manual/html_node/config_002esite.html) files allows both possibilities:
```
cd depends
make
cd ..
./autogen.sh
CONFIG_SITE=$PWD/depends/i686-pc-linux-gnu/share/config.site ./configure --prefix ~/.local
make
make install
```
or
```
CONFIG_SITE=$PWD/depends/i686-pc-linux-gnu/share/config.site ./configure
make
sudo make install # install to /usr/local
```
Moreover, this approach is used in [Gitian descriptors](https://github.com/bitcoin/bitcoin/tree/master/contrib/gitian-descriptors) already.
ACKs for top commit:
practicalswift:
ACK 223b1ba7d90509a47ea07af46f4b9c3b8efbc9f8: patch looks correct
fanquake:
ACK 223b1ba7d90509a47ea07af46f4b9c3b8efbc9f8
Tree-SHA512: 46d97924f0fc7e95ee4566737cf7c2ae805ca500e5c49af9aa99ecc3acede4b00329bc727a110aa1b62618dfbf5d1ca2234e736f16fbdf96d6ece5f821712f54
## Issue being fixed or feature implemented
fix 3 gui issues reported about beta2
## What was done?
pls see individual commits
## How Has This Been Tested?
to test ad0c0eb591 (there were no css
issues on my machine with locally compiled binaries): compile, run
to test 09800cff8f: run beta2 guix
compiled binaries with css from this PR e.g. on macos:
`/path/to/Dash-Qt.app/Contents/MacOS/Dash-Qt --regtest --debug-ui
--custom-css-dir=/path/to/dash/src/qt/res/css`
before 139d70701c:
<img width="224" alt="Screenshot 2023-09-30 at 22 02 32"
src="https://github.com/dashpay/dash/assets/1935069/b7a26ae9-f87f-4dde-b1f6-9eb796c22495">
after 139d70701c:
<img width="229" alt="Screenshot 2023-09-30 at 22 02 59"
src="https://github.com/dashpay/dash/assets/1935069/57d92e97-b25d-4035-9d1b-da373e51a574">
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
7b6887e75aec7e0d5c25682c26943073b7a728e5 doc: Convert depends options list from html to markdown (Wladimir J. van der Laan)
Pull request description:
This makes it easier to read in `less`, which is important for install instructions.
Rendered: [before](7ef6b1c51d/depends (dependency-options)) - [after](d97042406f/depends/README.md (dependency-options))
ACKs for top commit:
jonatack:
Code review re-ACK 7b6887e75aec7e0d5c25682c26943073b7a728e5 per `git diff d970424 7b6887e`
hebasto:
re-ACK 7b6887e75aec7e0d5c25682c26943073b7a728e5
Tree-SHA512: 02970b2bb97d2e8fb2d66470f6d70662653fda176bf6f4861742823b361fdc7ab6a2b44143480ac1a525b8d7808b6a068e8b3677dbba16cd783b4cab90470af5
