5efebc0edbb479d2041b3fb2d43be3a77e817b3e depends: add the new LLVM debug macro (fanquake)
Pull request description:
`LIBCXX_HARDENING_MODE` is the new macro, the previous one was removed in LLVM 18.
See https://libcxx.llvm.org/Hardening.html.
Required before https://github.com/google/oss-fuzz/pull/11725 will do anything (with the bump to 18.x).
Seems reasonable to do now that almost all our test infra is using LLVM 18.
ACKs for top commit:
theuni:
ACK 5efebc0edbb479d2041b3fb2d43be3a77e817b3e
Tree-SHA512: 43078eeb5940c55ef4f95c72682f8a372dcd3eb97956b3114149c16d9f59b067a999b2aab7f34ffb57eab191524514408e2bba154ff4a6ea0cd6ec4d119c5d18
84fbf9b2841a9ba1ebd1421b9ff9fe444bb1abd9 depends: remove -g from sqlite debug flags (fanquake)
eef51afc6a29c693a68400930ef8011be41b7401 depends: add -g to DEBUG=1 flags (fanquake)
Pull request description:
Add `-g` to the base DEBUG=1 flags in depends.
Avoids the need to specify it per-package.
More alignment with `--enable-debug` behaviour in configure.
We also want to align the optimization flags, currently -O1 vs -O0, however that can be it's own PR.
ACKs for top commit:
theuni:
ACK 84fbf9b2841a9ba1ebd1421b9ff9fe444bb1abd9
Tree-SHA512: 1ee98ba0c13e4b80bb87632658b4f53ce49c73e0e7712990c30da60deca4a349a744232f2d78f243dee9a07f5b9b70f9c2c4ae34082c34ae51b37b054fac61fd
4a825039a509c43ba20b2cd7aab448b3be16bcc3 build: use _LIBCPP_ENABLE_DEBUG_MODE over ENABLE_ASSERTIONS (fanquake)
Pull request description:
`_LIBCPP_ENABLE_ASSERTIONS` is deprecated, and will be removed. [See (from libc++ __config in main)](b57df9fe9a/libcxx/include/__config (L205-L209)):
> TODO(hardening): remove this in LLVM 19.
> This is for backward compatibility -- make enabling `_LIBCPP_ENABLE_ASSERTIONS` (which predates hardening modes)
> equivalent to setting the safe mode.
> ifdef _LIBCPP_ENABLE_ASSERTIONS
> warning "_LIBCPP_ENABLE_ASSERTIONS is deprecated, please use _LIBCPP_ENABLE_SAFE_MODE instead."
From LLVM 17, `_LIBCPP_ENABLE_DEBUG_MODE` can be used instead, which also performs more checks than safe mode:
> Enables the debug mode which contains all the checks from the hardened mode and additionally more expensive checks that may affect the complexity of algorithms. The debug mode is intended to be used for testing, not in production. Mutually exclusive with `_LIBCPP_ENABLE_HARDENED_MODE` and `_LIBCPP_ENABLE_SAFE_MODE`.
See https://libcxx.llvm.org/Hardening.html.
Related to #28476.
ACKs for top commit:
MarcoFalke:
lgtm ACK 4a825039a509c43ba20b2cd7aab448b3be16bcc3 🙏
Tree-SHA512: ca52603f86214e8e9350bd2b2baa44fbde0f72f1b186da7aecd8690256dff5b2be75fe89383158298a6f683bbd6ae0dff528d2ba4cc5ece1f56cfbdee0e1dc5d
bc4fd49d09dec3791b0acd4ada285b2287361d14 depends: add _LIBCPP_ENABLE_ASSERTIONS to DEBUG mode (fanquake)
cf266b2270081f05a277ba683f00de90a741112f depends: Remove _LIBCPP_DEBUG from depends DEBUG mode (fanquake)
Pull request description:
It was deprecated in LLVM 15, turned into compile-time error in LLVM 16:
```bash
In file included from /usr/lib/llvm-16/bin/../include/c++/v1/cassert:19:
/usr/lib/llvm-16/bin/../include/c++/v1/__assert:22:5: error: "Defining _LIBCPP_DEBUG is not supported anymore.
Please use _LIBCPP_ENABLE_DEBUG_MODE instead."
^
1 error generated.
```
and has been removed entirely in LLVM 17 (main): ff573a42cd.
[Building libc++ in debug mode](https://releases.llvm.org/16.0.0/projects/libcxx/docs/DesignDocs/DebugMode.html), will also automatically set
`_LIBCPP_ENABLE_DEBUG_MODE` (the new define), so adding it to depends
doesn't seem useful, and would just result in redefinition errors.
I'm wondering if as a followup, we could enable a DEBUG build of libc++
in our MSAN CI job? i.e https://github.com/fanquake/bitcoin/tree/msan_with_enable_debug_mode.
Somewhat related to https://github.com/google/oss-fuzz/pull/9828, where
it looks like we'll have to sort out getting a DEBUG build of LLVM, and can drop the commentary about re-enabling DEBUG=1.
ACKs for top commit:
MarcoFalke:
lgtm Approach ACK bc4fd49d09dec3791b0acd4ada285b2287361d14
Tree-SHA512: 9c0f48fc428278fbf34fbb8f81e761e232506d7ab28e971cb9a9b9a81d549b4d8bbe51e2f7608d56e489428679231da5b7431443849b238a8a993ad241740282
e2bab2aa162ae38b2bf8195b577c982402fbee9d multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e708c04ef6c9880f89d0a4cbaa6fc7c5 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52bfcd4edf8553aaf332bfeb92fc554cc build: multiprocess autotools changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.
In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.
The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649
ACKs for top commit:
practicalswift:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d
Sjors:
tACK e2bab2aa162ae38b2bf8195b577c982402fbee9d on macOS 10.15.4
hebasto:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d, tested on Linux Mint 19.3 (x86_64):
Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
fa9249aaccc3ef7a0a91a822e1cb666c4c9716ec depends: Add missing -D_LIBCPP_DEBUG=1 to debug flags (MarcoFalke)
Pull request description:
Commands that can be used for testing:
```
$ cat 1.cpp
#include <vector>
int main() {
std::vector<int> foo;
foo.begin() + 7;
}
```
```
clang++ -stdlib=libc++ -D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC -D_LIBCPP_DEBUG=1 -Wall 1.cpp -o exe && ./exe
g++ -D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC -D_LIBCPP_DEBUG=1 -Wall 1.cpp -o exe && ./exe
ACKs for top commit:
practicalswift:
cr ACK fa9249aaccc3ef7a0a91a822e1cb666c4c9716ec: patch looks correct
fanquake:
ACK fa9249aaccc3ef7a0a91a822e1cb666c4c9716ec - was going to suggest adding this to the macOS CPP flags as well, however it seems doing that is less straight forward. Could be looked at by someone in a followup.
Tree-SHA512: 2ffbaaf0ccb36bcc9fa1a15426566406c6115c8878ff211a4794d982c5d198672d444a20f6c7ae9f341193f6d8118c7cc50896daf98af9553834379e47ddb39e
* GCC-7 and glibc-2.27 compat code
* Statically link libstdc++ for GCC based builds
Makes sure binaries which are built on a newer build host still work
on older distros.
* Use python3 when installing MacOS native tools
* Move actual build logic out of Travis and upgrade to gcc-7
Travis will now simply call a few scripts which do the actual work.
These scripts will first create a "builder image" which contains the
necessary environment for the actual build. Then scripts are called
inside this builder image to do the build.
This should make us more independant from Travis and also allows us
to do local CI testing.
The build matrix is also moved out of .travis.yml and instead moved
into ci/matrix.sh. This script is sourced with only "BUILD_TARGET" being
set so that it internally can figure out which other environment
variables need to be set.
This commit also upgrades the used GCC version to 7. This is due to the
use of ubuntu:bionic as base image for the builder image.
* Add Jenkinsfiles for regular CI and nightly gitian builds
* Automatically download OSX SDK in gitian-build.sh
* Remove bogus "export MAKEJOBS=-j5"
* Forward cache/src dirs into builder container
Fixes caching issues on Travis.
* fix
* Fail build immediately when building depends took too long
