d318c4ef56465ccad1a1d4d27c52216e0b69ad4e depends: bump libmultiprocess for CMake fixes (Cory Fields)
Pull request description:
Broken out of #30454 . Bumped [even further](4883197abc (r1684802528)) after https://github.com/chaincodelabs/libmultiprocess/pull/98 was merged upstream.
hebasto Presumably this approach works now with the CMake branch?
ACKs for top commit:
ryanofsky:
Code review ACK d318c4ef56465ccad1a1d4d27c52216e0b69ad4e.
Tree-SHA512: 4b5491f73c0063d09e8339829cd831b1f4c441dd7b55a22037c9337c80cce19bb00a3e5cf925efa77d6d4e89ab45482f40f5799bc14948f8cabcbad3c3549430
21bfee0720ba72935d4f9fc4c2a2887ae5b54092 depends: bump libmultiprocess to fix capnproto deprecation warnings (Ryan Ofsky)
Pull request description:
This incorporates PR chaincodelabs/libmultiprocess#88 and reverts the NO_WERROR CI workaround added in #28735
Upstream diff: 61d5a0e661...414542f81e
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
maflcko:
lgtm ACK 21bfee0720ba72935d4f9fc4c2a2887ae5b54092
hebasto:
ACK 21bfee0720ba72935d4f9fc4c2a2887ae5b54092, I have reviewed the code and it looks OK. I've also skimmed through the related changes in the https://github.com/chaincodelabs/libmultiprocess repository.
Tree-SHA512: b5addb0deed694eeec62a0ae08b4715a811110201f39f3e6cadee8fc4e6231b0e66c844a98512072a1445bac122ab561dc1711e27fb4d7ac5c08ac46780a4acf
BACKPORT NOTICE: excludes ci/test/00_setup_env_i686_multiprocess.sh changes, however, these do not satisfy enough this to be partial to me
3333f14efac815ba3c885398a6795c7e8ce68d08 depends: Bump to capnproto-c++-1.0.1 (MarcoFalke)
Pull request description:
Reasons:
* Debian is starting to ship this version in Trixie (https://packages.debian.org/trixie/capnproto), which will likely become the version shipped with Ubuntu 24.04 LTS. So testing with this version will help to find any issues before real users start to use those distro packages.
* The feature is currently experimental, so bumping the version shouldn't cause any production issues.
* With multiprocess begin a priority project for 27.0, it seems better to do build system changes/bumps early, rather than later, to allow for more time testing them.
ACKs for top commit:
TheCharlatan:
Re-ACK 3333f14efac815ba3c885398a6795c7e8ce68d08
fanquake:
ACK 3333f14efac815ba3c885398a6795c7e8ce68d08 - the response from upstream is that [if we submit a PR, they can take a look](https://github.com/capnproto/capnproto/issues/1833#issuecomment-1792582206), so if anyone would like this to work for Windows, I'd suggest sending a patch.
ryanofsky:
Code review ACK 3333f14efac815ba3c885398a6795c7e8ce68d08
Tree-SHA512: 7d53ad1536f042ab43dbc7847126b826e7fc76694f173c348b835fd1067b8f3dd682c5bcb4887f09ee85bab69130721cd7f8fb96b2e82053d4e28bd5c38bdc5f
84934bf70e11fe4cda1cfda60113a54895d4fdd5 multiprocess: Add echoipc RPC method and test (Russell Yanofsky)
7d76cf667eff512043a28d4407cc89f58796c42b multiprocess: Add comments and documentation (Russell Yanofsky)
ddf7ecc8dfc64cf121099fb047e1ac871de94f4c multiprocess: Add bitcoin-node process spawning support (Russell Yanofsky)
10afdf0280fa93bfffb0a7665c60dc155cd84514 multiprocess: Add Ipc interface implementation (Russell Yanofsky)
745c9cebd50fea1664efef571dc1ee1bddc96102 multiprocess: Add Ipc and Init interface definitions (Russell Yanofsky)
5d62d7f6cd48bbc4e9f37ecc369f38d5e1e0036c Update libmultiprocess library (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
This PR adds basic process spawning and IPC method call support to `bitcoin-node` executables built with `--enable-multiprocess`[*].
These changes are used in https://github.com/bitcoin/bitcoin/pull/10102 to let node, gui, and wallet functionality run in different processes, and extended in https://github.com/bitcoin/bitcoin/pull/19460 and https://github.com/bitcoin/bitcoin/pull/19461 after that to allow gui and wallet processes to be started and stopped independently and connect to the node over a socket.
These changes can also be used to implement new functionality outside the `bitcoin-node` process like external indexes or pluggable transports (https://github.com/bitcoin/bitcoin/pull/18988). The `Ipc::spawnProcess` and `Ipc::serveProcess` methods added here are entry points for spawning a child process and serving a parent process, and being able to make bidirectional, multithreaded method calls between the processes. A simple example of this is implemented in commit "Add echoipc RPC method and test."
Changes in this PR aside from the echo test were originally part of #10102, but have been split and moved here for easier review, and so they can be used for other applications like external plugins.
Additional notes about this PR can be found at https://bitcoincore.reviews/19160
[*] Note: the `--enable-multiprocess` feature is still experimental, and not enabled by default, and not yet supported on windows. More information can be found in [doc/multiprocess.md](https://github.com/bitcoin/bitcoin/blob/master/doc/multiprocess.md)
ACKs for top commit:
fjahr:
re-ACK 84934bf70e11fe4cda1cfda60113a54895d4fdd5
ariard:
ACK 84934bf. Changes since last ACK fixes the silent merge conflict about `EnsureAnyNodeContext()`. Rebuilt and checked again debug command `echoipc`.
Tree-SHA512: 52a948b5e18a26d7d7a09b83003eaae9b1ed2981978c36c959fe9a55abf70ae6a627c4ff913a3428be17400a3dace30c58b5057fa75c319662c3be98f19810c6
7c218dacd0e9602b8f755be42e96c49706f96305 build: specify cmake build dir for multiprocess depends build (fanquake)
Pull request description:
When no build dir is specified, cmake will warn:
```bash
Preprocessing libmultiprocess...
Configuring libmultiprocess...
CMake Warning:
No source or binary directory provided. Both will be assumed to be the
same as the current working directory, but note that this warning will
become a fatal error in future CMake releases.
```
It's unclear if this will actually ever become an error, but it's also easy
enough to just supply the directory, and save this maybe breaking in
future.
ACKs for top commit:
ryanofsky:
Code review ACK 7c218dacd0e9602b8f755be42e96c49706f96305. I guess the purpose of the warning is to encourage people not to build in the source directory, but reasons for encouraging this don't really apply to the depends build system, so it is appropriate to disable the warning.
hebasto:
ACK 7c218dacd0e9602b8f755be42e96c49706f96305, tested on Ubuntu 22.04.
Tree-SHA512: 6904f2095fe62cead4abc644ec888c5d836e54a3c0b2a84c467029116e5d14eba35190570acaa23c6831aed9a4a65898134480cc46cdb141279ec0dc6f534d5f
e2bab2aa162ae38b2bf8195b577c982402fbee9d multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e708c04ef6c9880f89d0a4cbaa6fc7c5 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52bfcd4edf8553aaf332bfeb92fc554cc build: multiprocess autotools changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.
In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.
The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649
ACKs for top commit:
practicalswift:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d
Sjors:
tACK e2bab2aa162ae38b2bf8195b577c982402fbee9d on macOS 10.15.4
hebasto:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d, tested on Linux Mint 19.3 (x86_64):
Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
