BACKPORT NOTE:
Missing changes in src/fs.cpp are removed in bitcoin/bitcoin#20744 which is already backported
b367745cfe19f6de3f44b3adc90fa08e36e44bb6 ci: Make Cirrus CI Windows build with --enable-werror (Hennadii Stepanov)
c713bb2b243881a771ab288340ffeb623c82d7f6 Fix Windows build with --enable-werror on Ubuntu Focal (Hennadii Stepanov)
Pull request description:
This PR makes possible to cross-compile Windows build with `--enable-werror --enable-suppress-external-warnings`.
Some problems are fixed, others are silenced.
Also `--enable-werror` is enabled for Cirrus CI Windows build (the last one on Cirrus CI without `--enable-werror`).
ACKs for top commit:
practicalswift:
cr ACK b367745cfe19f6de3f44b3adc90fa08e36e44bb6: patch looks correct
laanwj:
Code review ACK b367745cfe19f6de3f44b3adc90fa08e36e44bb6
vasild:
ACK b367745cfe19f6de3f44b3adc90fa08e36e44bb6
jarolrod:
ACK b367745cfe19f6de3f44b3adc90fa08e36e44bb6
Tree-SHA512: 64f5c99b7dad4c0efce80cd45d7074f275bd8411235dc9e0841287bdab64b812c6f8f9d632c35531d0b8210148531f53aaaac77be7699b29d2d6aaae304dbee0
24c01934a2 fix: small fixup for bitcoin#14918 and bitcoin#21679 (Konstantin Akimov)
f358f2bcdd Merge bitcoin-core/gui#375: Emit dataChanged signal to dynamically re-sort Peers table (Hennadii Stepanov)
7aeb0adeb9 Merge bitcoin-core/gui#365: Draw "eye" sign at the beginning of watch-only addresses (Hennadii Stepanov)
c52a582a3f refactor: re-order conditions over flags for m_edge_trig_events - follow-up for bitcoin#21563 (Konstantin Akimov)
6ed62b323c Merge bitcoin/bitcoin#21563: net: Restrict period when cs_vNodes mutex is locked (MarcoFalke)
16052f10ae Merge #20791: p2p: remove unused legacyWhitelisted in AcceptConnection() (MarcoFalke)
42d4f9a9b9 partial Merge #20755: [rpc] Remove deprecated fields from getpeerinfo (MarcoFalke)
cba01aa8f9 Merge bitcoin/bitcoin#20191: wallet, refactor: make DescriptorScriptPubKeyMan agnostic of internal flag (fanquake)
397fe9c0a5 Merge bitcoin/bitcoin#22461: wallet: Change ScriptPubKeyMan::Upgrade default to True (Samuel Dobson)
97f0d91d3e Merge bitcoin/bitcoin#22376: ci: Do not clone `bitcoin-core/qa-assets` git repository if not necessary (MarcoFalke)
0698be3680 Merge bitcoin/bitcoin#22153: test: Fix p2p_leak.py intermittent failure (MarcoFalke)
58b95338eb Merge #21345: test: bring p2p_leak.py up to date (MarcoFalke)
Pull request description:
## Issue being fixed or feature implemented
Regular backports from bitcoin v22
## What was done?
See commits for list of backports.
bitcoin#20755 is partial because we need `banscore` for functional test `p2p_quorum_data.py`.
Also several minor fixes for:
- default args (bitcoin#14918, bitcoin#21679)
- fixes for CNode::CloseSocketDisconnect (related to bitcoin/bitcoin#21563)
- minor refactoring of m_edge_trig_events (related to bitcoin/bitcoin#21563)
- missing executable flags for functional tests
## How Has This Been Tested?
Run unit and functional tests.
See also a screenshot:
## Breaking Changes
```
Updated RPCs
- `getpeerinfo` no longer returns the following fields: `addnode`,
and `whitelisted`, which were previously deprecated in v21. Instead of
`addnode`, the `connection_type` field returns manual. Instead of
`whitelisted`, the `permissions` field indicates if the peer has special
privileges.
```
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
PastaPastaPasta:
utACK 24c01934a2
UdjinM6:
utACK 24c01934a2
Tree-SHA512: d457e7a63bef3edb7bbb82e54deb72e57b021ea74d40f05c5a2fca2253e97919531a3e35936851ac9ca88e9ee94f2f299dae979c53904596b8a489ebf9cd9aa6
30450a1bd5d278e285f50a7e4cfc755545960e92 Do not clone qa-assets git repository if not necessary (Kiminuo)
Pull request description:
This PR attempts to remove an unnecessary step when CI runs.
The main motivation for the change is that I locally use `MAKEJOBS="-j15" FILE_ENV="./ci/test/00_setup_env_android.sh" ./ci/test_run_all.sh` to find out if a patch of mine works or not. Cloning `bitcoin-core/qa-assets` is slow on my machine (which is by no means slow).
ACKs for top commit:
MarcoFalke:
cr ACK 30450a1bd5d278e285f50a7e4cfc755545960e92
Tree-SHA512: 5763b53da9554b06039c39f8fc729de1b106cce2a242de8f97528d001bfa01d4f48d2a128f458a3cdee3da36312354c6714839b947f313c089c2c5cb30233a39
faa359877270121b3cd442e1e5e865586ce7e530 ci: Add missing qttools5-dev install to Asan task (MarcoFalke)
Pull request description:
This is required, according to the docs:
```
$ git grep --line-number 'qtbase5-dev qttools5-dev qttools5-dev-tools' doc
doc/build-unix.md:84: sudo apt-get install qtbase5-dev qttools5-dev qttools5-dev-tools
```
Also, needed for cmake.
ACKs for top commit:
hebasto:
ACK faa359877270121b3cd442e1e5e865586ce7e530.
Tree-SHA512: c986908f757d70d958267c1e902b5d7d94589360db61ddf7b9b398cd635b2172e83510c0c77fd6032810166342a286c0f95225b6c6639acd869e1e51c3348ea7
86c2889518df0234b8b28112a9771b85c577eefd ci: Make log verbose in error case only (Hennadii Stepanov)
7f650883b71b67cd33026cf49af0b16f01e45541 depends: Add file-based logging for individual packages (Hennadii Stepanov)
Pull request description:
This PR adds file-based logging for individual packages in depends. To use this feature one should provide `LOG=1`.
A log file is printed out automatically in case of a build error. After successful build log files are being moved along with package archives:
```
$ make -C depends HOST=x86_64-w64-mingw32 LOG=1
$ find ./depends/built/x86_64-w64-mingw32 -name '*.log' | sort
./depends/built/x86_64-w64-mingw32/bdb/bdb-4.8.30-5100a099801.log
./depends/built/x86_64-w64-mingw32/boost/boost-1_71_0-313f82dc7de.log
./depends/built/x86_64-w64-mingw32/libevent/libevent-2.1.12-stable-3fa27048d5e.log
./depends/built/x86_64-w64-mingw32/libnatpmp/libnatpmp-4536032ae32268a45c073a4d5e91bbab4534773a-9db4850dd32.log
./depends/built/x86_64-w64-mingw32/miniupnpc/miniupnpc-2.2.2-75d9a1807e0.log
./depends/built/x86_64-w64-mingw32/native_b2/native_b2-1_71_0-3bf253c19bf.log
./depends/built/x86_64-w64-mingw32/qrencode/qrencode-3.4.4-dfac87af599.log
./depends/built/x86_64-w64-mingw32/qt/qt-5.15.2-9304e03d3ac.log
./depends/built/x86_64-w64-mingw32/sqlite/sqlite-3320100-455acafa7be.log
./depends/built/x86_64-w64-mingw32/zeromq/zeromq-4.3.1-5ff627ec84a.log
```
An example of CI tasks with package build errors -- https://cirrus-ci.com/task/5275741788045312Closes#16368.
ACKs for top commit:
laanwj:
Tested ACK 86c2889518df0234b8b28112a9771b85c577eefd
Tree-SHA512: 497f2146fd2e38c952124aecfd80ebb42be22bbc5dc59521491545f4465fc38f23da7787a0caea5686b7c30aa862f2b0c02092ae3fe863e80a5ddd14b3d324b9
f4cb0fbfe1 fix: no need to relay quorum commitment in case of block undo (Konstantin Akimov)
0431a33919 fix: follow-up changes for bitcoin#14193. (Konstantin Akimov)
86b76d19b6 Merge bitcoin/bitcoin#21812: ci: Enable D_GLIBCXX_DEBUG for multiprocess task (fanquake)
334496ea7e Merge bitcoin/bitcoin#21775: p2p: Limit m_block_inv_mutex (MarcoFalke)
23b83109ea Merge bitcoin/bitcoin#21750: net: remove unnecessary check of CNode::cs_vSend (MarcoFalke)
b34514191f Merge bitcoin/bitcoin#21738: test: Use clang-12 for ASAN, Add missing suppression (fanquake)
3411577473 Merge bitcoin/bitcoin#19160: multiprocess: Add basic spawn and IPC support (W. J. van der Laan)
970048d917 fix: missing changes from bitcoin#19267 - run multiprocess on CI (Konstantin Akimov)
f2b7ee73db fix: follow-up bitcoin#15402 - removed dead code (Konstantin Akimov)
274068cdbc fix: follow-up bitcoin/bitcoin#21732 - minor missing typo (MarcoFalke)
e9450a8b36 Merge #21669: test: Remove spurious double lock tsan suppressions by bumping to clang-12 (MarcoFalke)
ef92c3065c Merge #21663: ci: Fix macOS brew install command (W. J. van der Laan)
Pull request description:
## Issue being fixed or feature implemented
Just regular backports from v22
## What was done?
See commits for backports.
Also there're 2 bugs are fixed which became visible after backporting bitcoin#21775 - both are related to possible deadlocks in net_processing
## How Has This Been Tested?
Run unit and functional tests. Enabled multiprocess builds on CI
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
utACK f4cb0fbfe1
PastaPastaPasta:
utACK f4cb0fbfe1
Tree-SHA512: 3204c2aa243fa4834ccf4ff4672d183cf9b35f87b857df8543572cd685729e15fca39f85b27194233e57cbc1746e36b556efab95ce20d0aa0a7d4476a9f3c6c0
fa44f5119a0b412f0d46cad02f638727d140b451 ci: Clarify that previous_releases task is using DEBUG (MarcoFalke)
fad0f21c3caba129106799fe6c14aff323ef99f2 ci: Use clang in multiprocess task to avoid OOM (MarcoFalke)
faeabef4f386009847a0f91041d44e6f31eec618 ci: Enable D_GLIBCXX_DEBUG for multiprocess task (MarcoFalke)
Pull request description:
Enable `-D_GLIBCXX_DEBUG` via the depends `DEBUG` flag. Also `--enable-debug` to get debug symbols in traces.
ACKs for top commit:
hebasto:
ACK fa44f5119a0b412f0d46cad02f638727d140b451, I have reviewed the code and it looks OK, I agree it can be merged, and CI is green.
Tree-SHA512: ab2a216bb44ee462f9dd181ec9025962502bd4201a1118ff52b6a193398e7ea3ca465a45a5eb341e308758fc3ef34ea3521f8a1f85ed64478ef3c1f6c1b8b141
c5da2749e2f7375e292fb0982e8e252ae1adbce3 build: actually stop configure if Boost isn't available (fanquake)
cad8b527eaf7a93877e2249960866fd4db2d1c14 build: explicitly install libboost-dev package (fanquake)
Pull request description:
If Boost is not found via AX_BOOST_BASE, we don't actually stop
configuring, only a warning is emitted:
```bash
checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version MINIMUM_REQUIRED_BOOST or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
```
Instead we usually fail when one of the other AX_BOOST_* macros fails to find a library. These macros are slowly being
removed, and in any case, it makes more sense to fail earlier if Boost is missing.
If Boost is unavailable, the failure now looks like:
```bash
checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version 1.58.0 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
configure: error: Boost is not available!
```
Note that we now just pass the version into AX_BOOST_BASE, which fixes it's display in the output (rather than showing `MINIMUM_REQUIRED_BOOST`).
This PR also has a commit that adds `libboost-dev` to our install instructions and CI. This package is currently installed as a side-effect of installing our other libboost-*-dev packages. However as those continue to disappear, it makes sense to install boost-dev explicitly.
ACKs for top commit:
laanwj:
Code review ACK c5da2749e2f7375e292fb0982e8e252ae1adbce3
MarcoFalke:
Concept ACK c5da2749e2f7375e292fb0982e8e252ae1adbce3
Tree-SHA512: f866062f9d7d3a2316b6c887f17c664b9cfff41fdc0cb99ca79d641240fb01a5ae0d34140e515bc465219e1b43d5ca84f7c55f48b9c5b45a80ff2795dafd072b
faff3991a9be0ea7be31685fb46d94c212c5da34 ci: Fuzz with integer sanitizer (MarcoFalke)
Pull request description:
Otherwise the suppressions file will go out of sync
ACKs for top commit:
practicalswift:
cr ACK faff3991a9be0ea7be31685fb46d94c212c5da34: patch looks correct
Tree-SHA512: 349216d071a2c5ccf24565fe0c52d7a570ec148d515d085616a284f1ab9992ce10ff82eb17962dddbcda765bbd3a9b15e8b25f34bdbed99fc36922d4161d307c
3ffc190321836f7c79f0cb21ee71d357a0e32c44 ci: Install documented packages for "Win64" CI task (Hennadii Stepanov)
Pull request description:
This minor change has the following benefits:
- it follows the [documented](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md#building-for-64-bit-windows) way for modern Ubuntu distros (this CI task uses jammy)
- it makes package installation time shorter as no need to install the `g++-mingw-w64-x86-64-win32` package
- (not directly related to this repo) in https://github.com/bitcoin-core/gui-qml Qt 5.15.3 (but not 5.15.2) build system goes [wild](https://cirrus-ci.com/task/6231535933194240) otherwise
ACKs for top commit:
fanquake:
ACK 3ffc190321836f7c79f0cb21ee71d357a0e32c44
jarolrod:
ACK 3ffc190321836f7c79f0cb21ee71d357a0e32c44
Tree-SHA512: 41fd6deedb3febc90cc4f2037dfbf840d82ef5b1dd950a0ff458cae6c1b1024559b21c8e1135c2d37780e80dd3f9f9751d638120443d0d60c22ac160cf693e2a
46b025e00df40724175735eb5606ac73067cb3b8 test: add new python linter to check file names and permissions (windsok)
6f6bb3ebc7cb8e17a5dfc8ef55aa2d3f2dc6bdea test: fix file permissions on various scripts (windsok)
Pull request description:
Adds a new python linter test which tests for correct filenames and file permissions in the repository.
Replaces the existing tests in the `test/lint/lint-filenames.sh` and `test/lint/lint-shebang.sh` linter tests, as well as adding some new and increased testing. This increased coverage is intended to catch issues such as in #21728 and https://github.com/bitcoin/bitcoin/pull/16807/files#r345547050
Summary of tests:
* Checks every file in the repository against an allowed regexp to make sure only lowercase or uppercase alphanumerics (a-zA-Z0-9), underscores (_), hyphens (-), at (@) and dots (.) are used in repository filenames.
* Checks only source files (*.cpp, *.h, *.py, *.sh) against a stricter allowed regexp to make sure only lowercase alphanumerics (a-z0-9), underscores (_), hyphens (-) and dots (.) are used in source code filenames. Additionally there is an exception regexp for directories or files which are excepted from matching this regexp (This should replicate the existing `test/lint/lint-filenames.sh` test)
* Checks all files in the repository match an allowed executable or non-executable file permission octal. Additionally checks that for executable files, the file contains a shebang line.
* Checks that for executable `.py` and `.sh` files, the shebang line used matches an allowable list of shebangs (This should replicate the existing `test/lint/lint-shebang.sh` test)
* Checks every file that contains a shebang line to ensure it has an executable permission
Additionally updates the permissions on various files to comply with the new tests.
Fixes#21729
ACKs for top commit:
practicalswift:
cr re-ACK 46b025e00df40724175735eb5606ac73067cb3b8: patch still looks correct
kiminuo:
code review ACK 46b025e00df40724175735eb5606ac73067cb3b8 if `contrib/gitian-descriptors/assign_DISTNAME` permission change is deemed OK.
laanwj:
Code review ACK 46b025e00df40724175735eb5606ac73067cb3b8
Tree-SHA512: 1c8201a2cee0d9cbce15652b68cec9a6458a8b493fcd5392f98560aca0b1a12e668baab65a47100f116f626dadc3f591deb47f7368468c6a46c6c712c2533455
7fc1e14ce60d4e0533c7ccc65a9b24052d7a608f ci: use Ubuntu 20.04 as the default Docker container (fanquake)
Pull request description:
All but 2 of the Ubuntu CIs (native qt5 & nowallet) are already using 20.04 or 21.04.
ACKs for top commit:
MarcoFalke:
cr ACK 7fc1e14ce60d4e0533c7ccc65a9b24052d7a608f
Tree-SHA512: f35d79a87af6c6955695b5e627884f94aed19bafaed4657d03ef4db66cf47cae5311464bb39961570140325652941283b9d88dff862776e8becfff9130162917
f7f3829a68df173c54ab11ecfe162b840afb64e8 build, doc: Drop libbz2-dev from macOS cross-compiling dependencies (Hennadii Stepanov)
d8239362e283107657a5a6577890fdb198a1d334 build, doc: Drop libcap-dev from macOS cross-compiling dependencies (Hennadii Stepanov)
Pull request description:
The `libcap-dev` and `libbz2-dev` packages are no longer required when cross-compiling for macOS.
ACKs for top commit:
fanquake:
ACK f7f3829a68df173c54ab11ecfe162b840afb64e8
Tree-SHA512: 820cdc2724f3346c0942d4d4115fc7206f7bf02889d9fa6cbdbd1d9e3afa03a067c1c3fa64dff596aefdc74898178b7c7d64027a6501486e3b606f4760de04ae
faa8afe70b4992267176a97262f59413d1a1196b ci: Re-run wine tests once if they fail (MarcoFalke)
Pull request description:
Works around the intermittent wine issue: https://github.com/bitcoin/bitcoin/issues/21122#issuecomment-776517563
ACKs for top commit:
fanquake:
ACK faa8afe70b4992267176a97262f59413d1a1196b - thanks for following up with this.
Tree-SHA512: cb377a8e62c7fcf38fb5bdd6203af82c6c302c86f7fc219cb756eb7b6d3d4fe4ebf30a36ebba739199ff7814b26256a3701cf38dc5e681b18c2f249cdf6d852e
fa272dfdff711e324baefc9bdbb2bf4b52d99eb0 ci: Properly bump to focal for win cross build (MarcoFalke)
Pull request description:
Fixes#21122
ACKs for top commit:
fanquake:
ACK fa272dfdff711e324baefc9bdbb2bf4b52d99eb0 - Looks good to me. I can see the wine version output, `wine-5.0 (Ubuntu 5.0-3ubuntu1)`, in the [log](https://cirrus-ci.com/task/5743559502462976), and `make check` is running.
Tree-SHA512: cd37462afc5512e00cef5e9e7fd1bb5c43c600e833b30cdea2c1c443dc7b0e68f5f2cbaf9d7b655892059af0a226478211db1dc97c10fe4dcdfe666a784f3afc
Debug logs should not be printed to stdout, stderr exists for it.
Private keys should not be printed to console by activation very general
macros name "ENABLE_DASH_DEBUG".
This macros is used only for hdchain private keys, the location util/system.h
is too general for it.
Functional test "tool_wallet.py" is failed due to unexpected output for tsan.
It seems as easier to remove this logs due to too many issues with it
rather than address all of them.
a47635baad feat: drop symbol check from CI to unify with bitcoin, keep it for qt5 only (Konstantin Akimov)
0c38cc325e fix: drop -static-libstc++ from depends/hosts/{linux,mingw32}.mk which is clang-only (Konstantin Akimov)
14a67ee85e Merge #20182: ci: Build with --enable-werror by default, and document exceptions (MarcoFalke)
Pull request description:
## Issue being fixed or feature implemented
As discovered in #5957:
Jobs tsan/ubsan fails with backport bitcoin#20182
it fails, because both tsan and ubsan jobs use clang but all other jobs use gcc. Somehow, after configure there are set incorrect options:
```
clang++-16 -std=c++17 -c -pipe -static-libstdc++ -O1 -Werror -D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC -D_LIBCPP_DEBUG=1 -I/builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/ conftest.cpp
```
`clang` doesn't support `-static-llibstdc++` which is supposed to be gcc-only, it cause this failure:
```
clang: error: argument unused during compilation: '-static-libstdc++' [-Werror,-Wunused-command-line-argument]
```
This failure make `autoconf` to think that clang doesn't support `-Werror` and fails with this error. So, you can't activate this flag.
## What was done?
Backport bitcoin#20182 and related fixes to make it works
## How Has This Been Tested?
CI now succeed with bitcoin#20182. It means, that -Werror activated for clang; also there are not warnings such as:
```
clang: error: argument unused during compilation: '-static-libstdc++' [-Werror,-Wunused-command-line-argument]
```
https://gitlab.com/dashpay/dash/-/jobs/6494328698
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
Top commit has no ACKs.
Tree-SHA512: f912824eaa1ec7513cda7278a3df9e067b0ab48a2d174b18654c8070aa6544bac33a52f494c1e35b4eab10392c1f26df4663e21d12a4dfff7c0a4a6a01ff9551
2f6fe4e4e9e9e35e713c0a20cf891b023592110a ci: Build with --enable-werror by default, and document exceptions (Hennadii Stepanov)
Pull request description:
This PR prevents introducing of new compiler warnings in the master branch, e.g., #19986, #20162.
ACKs for top commit:
practicalswift:
cr ACK 2f6fe4e4e9e9e35e713c0a20cf891b023592110a: patch looks correct
MarcoFalke:
re-ACK 2f6fe4e4e9 🏏
vasild:
ACK 2f6fe4e
Tree-SHA512: 23b5feb5bc472658c992d882ef61af23496f25adaa19f9c79bfaef5d2db273d44981aa93b1631a7d37cb58755283c1dacf3f2d68e501522d3fa8c965ab646d19
ac24af453d16afd1993fa1f292aa41ae6b16f138 ci: use Ubuntu Focal for macOS cross build (fanquake)
Pull request description:
I had assumed Cirrus was spinning up Docker containers to run the CI,
however we are actaully running on the Cirrus machines themselves. See
`DANGER_RUN_CI_ON_HOST` and in the logs:
```bash
Running on host system without docker wrapper
```
So with this change we will actually be using Focal for the macOS cross build.
Follow up to #21036.
This originally contained Windows changes, and an attempt to get Cirrus running without `DANGER_RUN_CI_ON_HOST`, however that seems non-trival, so Windows changes have been dropped from here for now.
ACKs for top commit:
MarcoFalke:
cr ACK ac24af453d16afd1993fa1f292aa41ae6b16f138
Tree-SHA512: 587ba5acf741bcefecf1bc262fa1177f565ebfa9de56125eca19ed3c7db7b9aabfb96866e9c140681b88cb7015a3ded2bc6b4b1b235543d6f6e9dfc6984d569f
95487b055328b590ba83f258de9637ab0f9a2f17 doc: Drop mentions of Travis CI as it is no longer used (Hennadii Stepanov)
09d105ef0f8b4b06bf248721a1209c9e16e9db75 ci: Drop travis_fold feature as Travis CI is no longer used (Hennadii Stepanov)
Pull request description:
As Travis CI is no longer used, this PR:
- drops `travis_fold` feature
- drops mentions of Travis CI in docs
ACKs for top commit:
MarcoFalke:
ACK 95487b055328b590ba83f258de9637ab0f9a2f17
Tree-SHA512: 2e259bb8b1e37bcefc1251737bb2716f06ddb57c490010b373825c4e70f42ca38efae69a2f63f21f577d7cee3725b94097bdddbd313f8ebf499281cf97c53cef
fa7a4385d08797876de9a05ae8a6fde18e36bd67 ci: Fix doc typos in .cirrus.yml (MarcoFalke)
fa73674738b0b5a271ae387e2ebd76d747e02d91 ci: Run i686 centos ci config on cirrus (MarcoFalke)
fa1f949a4dc2568456f3ba30bbd8ecbc78a423ca ci: Run nowallet ci config on cirrus (MarcoFalke)
Pull request description:
Travis CI Org is shutting down, so move the configs to cirrus ci
ACKs for top commit:
practicalswift:
ACK fa7a4385d08797876de9a05ae8a6fde18e36bd67: patch looks correct
Tree-SHA512: 1b7125c7f0d2288931fb8c5e90345891d5f7c1f00c4af136afbeb36bd2836f72920a1877dacc7be787e2c8d84de2a733c1ca71931e5c101b222c1fea588619b3
0cdf2a77ddfa1d53c6fbd830d557a3f20d7fc365 ci: add tsan debug symbols option (Russell Yanofsky)
9a2f12680b3f00a207f1cdd4e0c50a3c7613aefc ci: Add tsan suppression for race in DatabaseBatch (Hennadii Stepanov)
Pull request description:
Since #19325 was merged, the corresponding change in TSan suppression file gets required.
This PR is:
- an analogous to #19226 and #19450, and
- a temporary workaround for CI fail like https://cirrus-ci.com/task/5741795508224000?command=ci#L4993
ACKs for top commit:
MarcoFalke:
ACK 0cdf2a77ddfa1d53c6fbd830d557a3f20d7fc365
Tree-SHA512: 7832f143887c8a0df99dea03e00694621710378fbe923e3592185fcd3658546a590693b513abffc5ab96e9ef76c9c4bff3330eeee69a0c5dbe7574f34c417220
057750c09d0a8331c33966d2cc2285ef82f08af8 ci: Upgrading pip version in macos environment (Tushar Singla)
Pull request description:
During each CI run, in macos native environment, python packages lief and zmq are rebuilt everytime which wastes a lot of resources and time and fixes#22206. The latest version of pip directly fetches pre-built binaries. Through this commit pip version is upgraded in macos environment before installation of these packages.
ACKs for top commit:
MarcoFalke:
cr ACK 057750c09d0a8331c33966d2cc2285ef82f08af8
Tree-SHA512: e61d02e46c8fe6a89119014d025a26aba090f9507d725315680893290f5bbc20a375ef408c71fa8db2f485b44ec91cfa0c140198ca44a9d3e0a57055b6bb9582
77777c8b5e3d0edbe1178d982a778b7cc9964b70 ci: Run windows ci config on cirrus (MarcoFalke)
3333d6942e8c5d823b97ea51ef88c8a0791742c6 ci: Run macos ci config on cirrus (MarcoFalke)
fa8b1114e60100f503673ccc7ef5618851c0c9e3 ci: Run arm ci config on cirrus (MarcoFalke)
fa0795f54dc5268bacc6acdfdbe23f9952a0e7d8 ci: Replace TRAVIS_OS_NAME with CI_OS_NAME (MarcoFalke)
fafce1a13a65cc3ae0cd5351648bb0290f9bfe8c ci: Move documentation to correct config file (MarcoFalke)
Pull request description:
ACKs for top commit:
hebasto:
ACK 77777c8b5e3d0edbe1178d982a778b7cc9964b70
Tree-SHA512: 581d3bdb2c3e3da20bd8492c6b23d90f3b4f7f9300ade384667e41de9fbbe42bbcef26dd99328839dc0b49be6e303c4c6d1b66ceb5d8009800c52c97685f9080
fa2c3c0d96b48bc1685d84186552f72e3609a331 ci: Set LC_ALL=C to allow running the s390x tests in qemu (MarcoFalke)
fac05178366a1c9100b1d19d66fb669d7266f306 travis: Remove s390x build (MarcoFalke)
Pull request description:
This has been discussed in the last meeting.
Refer to the commit body for more details.
Top commit has no ACKs.
Tree-SHA512: 8e0455286ce41c95ed2e5eb624ac534251bb4a321f13d26d14356497e0c39f841372e166373ffd4a0a9fa379636c2cfb535bd92534fff427cdcb827354e66b6c
BACKPORT NOTICE:
define/endif moved in original backport but some changes are not backported yet
---------
fa3967efdb07f1d22372f4ee2e602ea1fad04a57 test: Replace ARRAYLEN with C++11 ranged for loop (MarcoFalke)
fafc5290538fde76c3780976f4b2c11dc9f24d19 test: Run AssetTest even if built --with-libs=no (MarcoFalke)
faf58ab139949ca35b33217d010b350c9a59c61d ci: Add --with-libs=no to one ci config (MarcoFalke)
Pull request description:
`script_assets_test` doesn't call libbitcoinconsensus, so it seems confusing to require it
ACKs for top commit:
fanquake:
ACK fa3967efdb07f1d22372f4ee2e602ea1fad04a57 - looks ok to me.
Tree-SHA512: 8744fef64c5d7dc19a0ef4ae9b3df5b3f356253bf000f12723064794c5e50df071824d436059985f1112d94c1b530b65cbeb6b8435114a91b195480620eafc59
31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0 [util] add RunCommandParseJSON (Sjors Provoost)
c17f54ee535faaedf9033717403e1f775b5f1530 [ci] use boost::process (Sjors Provoost)
32128ba682033560d6eb2e4848a9f77a842016d2 [doc] include Doxygen comments for HAVE_BOOST_PROCESS (Sjors Provoost)
3c84d85f7d218fa27e9343c5cd1a55e519218980 [build] msvc: add boost::process (Sjors Provoost)
c47e4bbf0b44f2de1278f9538124ec98ee0815bb [build] make boost-process opt-in (Sjors Provoost)
929cda5470f98d1ef85c05b1cad4e2fb9227e3b0 configure: add ax_boost_process (Sjors Provoost)
8314c23d7b39fc36dde8b40b03b6efbe96f85698 [depends] boost: patch unused variable in boost_process (Sjors Provoost)
Pull request description:
Prerequisite for external signer support in #16546. Big picture overview in [this gist](https://gist.github.com/Sjors/29d06728c685e6182828c1ce9b74483d).
This adds a new dependency [boost process](https://github.com/boostorg/process/tree/boost-1.64.0). This is part of Boost since 1.64 which is part of `depends`. Because the minimum Boost version is 1.47, this functionality is skipped for older versions of Boost.
Use `./configure --with-boost-process` to opt in, which checks for the presence of Boost::Process.
We add `UniValue runCommandParseJSON(const std::string& strCommand)` to `system.{h,cpp}` which calls an arbitrary command and processes the JSON returned by it. This is currently only called by the test suite.
~For testing purposes this adds a new regtest-only RPC method `runcommand`, as well as `test/mocks/command.py` used by functional tests.~ (this is no longer the case)
TODO:
- [ ] review boost process in #15440
ACKs for top commit:
achow101:
ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0
hebasto:
re-ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, only rebased (verified with `git range-diff`) and removed an unintentional tab character since the [previous](https://github.com/bitcoin/bitcoin/pull/15382#pullrequestreview-458371035) review.
meshcollider:
Very light utACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, although I am not very confident with build stuff.
promag:
Code review ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, don't mind the nit.
ryanofsky:
Code review ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0. I left some comments below that could be ignored or followed up later. The current change is clean and comprehensive.
Tree-SHA512: c506e747014b263606e1f538ed4624a8ad7bcf4e025cb700c12cc5739964e254dc04a2bbb848996b170e2ccec3fbfa4fe9e2b3976b191222cfb82fc3e6ab182d
fa7166759789c1282609ff3ab2e80d4f70910a9f ci: Move travis workarounds to .travis.yml (MarcoFalke)
Pull request description:
It seems odd to have travis related workarounds in the general ci config files. Fix that oddity by moving the travis related workarounds to the travis yaml file.
For unexplained reasons, this should also work around and thus close#19171
ACKs for top commit:
hebasto:
ACK fa7166759789c1282609ff3ab2e80d4f70910a9f, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: b4419d38e2b41f6e4d6e6b7658f1d972c40c390a49fe78808f8640d28efd84cc6668ce292d45b7c539e65b9e2ecbad10e796cb8f9329a0f1e7d0132ce962d226
fa7e002d520d8390f3ff4b0383cfdfc14713355d ci: tsan with wallet (MarcoFalke)
Pull request description:
ACKs for top commit:
practicalswift:
ACK fa7e002d520d8390f3ff4b0383cfdfc14713355d -- patch looks correct and Travis is happy
hebasto:
ACK fa7e002d520d8390f3ff4b0383cfdfc14713355d, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 1138459bbef72f402f32dae1e28d96f174901d4248d959b538b973747c8b06f221ecd81a386a1f915c0a2faaefb9fb8f11e3be39e6c5e2468bf9ae43d9f97757
BACKPORT NOTICE:
this PR doesn't actually swithc to libc++ due to multiple CI failures such as
linking errors or other
------------------
faf62e6ed0ca45db44c370844c3515eb5a8cda12 ci: Remove unused workaround (MarcoFalke)
fa7c8509153bfd2d5b4dcff86ad27dfd73e8788b ci: Install llvm to get llvm symbolizer (MarcoFalke)
fa563cef61e8a217c5e8ec059e174afae61087a5 test: Add more tsan suppressions (MarcoFalke)
fa0cc02c0a029133f080680ae9186002a144738f ci: Mute depends logs completely (MarcoFalke)
fa906bf2988c799765a04c484269f890964ec3ee test: Extend tsan suppressions for clang stdlib (MarcoFalke)
fa10d850790bbe52d948659bb1ebbb88fe718065 ci: Use libc++ instead of libstdc++ for tsan (MarcoFalke)
fa0d5ee1126a8cff9f30f863eb8f5c78bf57e168 ci: Set halt_on_error=1 for tsan (MarcoFalke)
fa2ffe87f794caa74f80c1c2d6e6067ee4849632 ci: Deduplicate DOCKER_EXEC (MarcoFalke)
fac2eeeb9d718bdb892eef9adf333ea61ba8f3d0 cirrus: Remove no longer needed install step (MarcoFalke)
Pull request description:
According to the [ThreadSanitizer docs](https://clang.llvm.org/docs/ThreadSanitizer.html#current-status):
> C++11 threading is supported with **llvm libc++**.
For example, the thread sanitizer build is currently not checking for double lock of mutexes.
Fixes (partially) https://github.com/bitcoin/bitcoin/issues/19038#issuecomment-632138003
ACKs for top commit:
practicalswift:
ACK faf62e6ed0ca45db44c370844c3515eb5a8cda12
fanquake:
ACK faf62e6ed0ca45db44c370844c3515eb5a8cda12
hebasto:
ACK faf62e6ed0ca45db44c370844c3515eb5a8cda12, maybe re-organize commits to modify suppressions in a single one?
Tree-SHA512: 98ce5154b4736dfb811ffdb6e6f63a7bc25fe50d3b73134404a8f3715ad53626c31f9c8132dbacf85de47b9409f1e17a4399e35f78b1da30b1577167ea2982ad
60824b3c3a4221a31c3638008b05fecc040c3df2 ci: Fix configure options for macOS builds (Hennadii Stepanov)
687939e3d251125aea5f6f7202e00a12d60a899f ci: Drop Homebrew caching while using Homebrew addon on Travis (Hennadii Stepanov)
557d3f1cc065702ebbf929d39ac7c5561111ce4c ci: Do not activate Travis ccache caching strategy (Hennadii Stepanov)
2d747428e23c0b7e3e7d0ce0db4226372f5ec0dd ci: Disable functional tests on forked repos to avoid timeouts for macOS (Hennadii Stepanov)
Pull request description:
See: https://github.com/bitcoin-core/gui/issues/5#issuecomment-656819184
Additionally, this PR:
- updates macOS image to the recent 10.15.5 version
- drops Homebrew caching as the Travis Homebrew addon have been used since #18438
My forked repo build: https://travis-ci.org/github/hebasto/bitcoin/jobs/707200431
Top commit has no ACKs.
Tree-SHA512: 398e935f965a04babeb10e7b26d2341562f21a1ef671c2e7cc97c9ec79d5c31643f81ca18561ab7714b5c52e19df2e4bffe4223eadbab984daa9418ffbf8c2a8
fa74a54fad7abfe3b0c98c5a6e4780d63d35b13f ci: Increase test timeout for sanitizer configs (MarcoFalke)
Pull request description:
Hopefully fixes#19369
ACKs for top commit:
practicalswift:
ACK fa74a54fad7abfe3b0c98c5a6e4780d63d35b13f -- patch looks correct!
fanquake:
ACK fa74a54fad7abfe3b0c98c5a6e4780d63d35b13f - the test failure here is a different issue, and the problem referenced by this PR hasn't occurred, so I think this can be merged. It's also fixing the use of `--factor` which was replaced in #18986.
Tree-SHA512: bec44fff454f20b7c5f8a461560d2496765dea61186027cc0cdce5ac55be0488b6f7f172fec49b89fe59a75b455501e2b4ae91a98c4a17d5c1a722846d2b3b60
fa05f44893d228f672f39436d0cb6b3376f81ac2 ci: Upgrade most ci configs to focal (MarcoFalke)
fad67208914e5a74b64f4cc018368902ef3a2e9b doc: move doc to ci readme (MarcoFalke)
fa880773b425fcd292ed7669d237ee3151a15bc6 ci: Have one config run in xenial to test against python3.5 (MarcoFalke)
fa6ddb2fa167df52f59cb9eaabed48315ddcdb2e travis: Always run multiprocess build (MarcoFalke)
Pull request description:
Generally developers compile with recent compilers, so bumping the ci configs to a recent OS should be uncontroversial. Older OSes (especially with compiler sanitizers) need workarounds that can be dropped by running on a more recent OS.
This pull changes the asan sanitizer and the experimental multiprocess build to use focal.
Also, it runs the no_wallet config on xenial to test against python 3.5, according to `doc/dependencies.md`.
Finally, all configs that mimic gitian (win and mac) will stay at bionic.
ACKs for top commit:
Sjors:
ACK fa05f44893d228f672f39436d0cb6b3376f81ac2, assuming Travis passes
hebasto:
ACK fa05f44893d228f672f39436d0cb6b3376f81ac2
Tree-SHA512: 55ec56c71ba2280d27c1a8856a1e6c310b1fbf469d5a8a1dde228063e3892e1dd1e51408ecff7a3d77ac2ae018daa9e9bbbb60598cdeaab8c32a146b11b3e7c4
fa330ec2fe5f5ba68a8d43fff0b19584c0b1ff39 test: Remove confusing and broken use of wait_until global (MarcoFalke)
fa6583c30bf7d82cf7ffdae995f8f16524ad2c0d ci: Set increased --timeout-factor by default (MarcoFalke)
Pull request description:
Assuming that tests don't have a logic error or race, setting a high timeout should not cause any issues. The tests will still pass just as fast in the fastest case, but it allows for some buffer in case of slow disks or otherwise starved ci machines.
Fixes#19729
ACKs for top commit:
hebasto:
ACK fa330ec2fe5f5ba68a8d43fff0b19584c0b1ff39, I have reviewed the code, and it looks OK, I agree it can be merged.
Tree-SHA512: 3da80ee008c7b08bab5fdaf7804d57c79d6fed49a7d37b9c54fc89756659fcb9981fd10afc4d07bd90d93c1699fd410a0110a3cd34d016873759d114ce3cd82d
e2bab2aa162ae38b2bf8195b577c982402fbee9d multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e708c04ef6c9880f89d0a4cbaa6fc7c5 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52bfcd4edf8553aaf332bfeb92fc554cc build: multiprocess autotools changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.
In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.
The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649
ACKs for top commit:
practicalswift:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d
Sjors:
tACK e2bab2aa162ae38b2bf8195b577c982402fbee9d on macOS 10.15.4
hebasto:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d, tested on Linux Mint 19.3 (x86_64):
Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
060a2a64d40d75fecb60b7d2b9946a67e46aa6fc ci: remove boost thread installation (fanquake)
06e1d7d81d5a56d136c6fc88f09a2b0654a164f9 build: don't build or use Boost Thread (fanquake)
7097add83c8596f81be9edd66971ffd2486357eb refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef834490c438436719f95cbaf888c4914 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)
Pull request description:
This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).
Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.
Two other points:
[Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
> It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.
Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
* The version of Boost.
* The platform you're building for.
* Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
* Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.
A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.
With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.
Previous similar PRs were #19183 & #20922. The authors are included in the commits here.
Also related to #21022 - pthread sanity checking.
ACKs for top commit:
laanwj:
Code review ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc
vasild:
ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc
Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
fae49f6e424f31e93c5620d5ff893fb517ef4a8b ci: Add and document BASE_BUILD_DIR (MarcoFalke)
Pull request description:
Also fixes#18768
ACKs for top commit:
hebasto:
re-ACK fae49f6e424f31e93c5620d5ff893fb517ef4a8b, which is essentially the same as the previously [reviewed changes](https://github.com/bitcoin/bitcoin/pull/18735#pullrequestreview-400581536).
Tree-SHA512: 216565a05ccd513dd9f114b2333d3c283fd71914d32f9b05f145cb7c70633b083ff8ef60798d6f22f4be6a4d652b03806551fd74b5b596c92968501a4d9726d2
It does not includes changes that actually enable werror by default.
But this backport is necessary to prevent compilation error on mac such as this one:
In file included from mapport.cpp:26:
In file included from /builds/dashpay/dash/depends/x86_64-apple-darwin/include/miniupnpc/miniupnpc.h:14:
/builds/dashpay/dash/depends/x86_64-apple-darwin/include/miniupnpc/upnpdev.h:27:14: error: zero size arrays are an extension [-Werror,-Wzero-length-array]
char buffer[0];
fa23fbb42fa710e6be405596e2abfc3a289d020d ci: Run all tests on native mac again (MarcoFalke)
Pull request description:
They should pass again after f6072e601af68f3eee307478ad22ff3960680656
ACKs for top commit:
practicalswift:
ACK fa23fbb42fa710e6be405596e2abfc3a289d020d -- Travis is happy and so am I
Tree-SHA512: 49c16b6056d4e67d12a202744e1c56fee2788830213fe4a195955ad44c6b8ecce768a591463ffa0048821959a75b6fad4178629a8866c4a26799c4c8c13e933d
fad798be76dd5e330463c837fda768477d536078 test: Default --previous-releases to false if dir is empty (MarcoFalke)
faf1c3cc58d14f86ba5364e6ee5c8ef29cac2e26 test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option (MarcoFalke)
Pull request description:
The "auto-detection" feature is kept in place, but making it an option allows to properly document it. For example, on my machine I get:
```
$ ./test/functional/wallet_disable.py --help | grep previous-releases
--previous-releases Force test of previous releases (default: False)
ACKs for top commit:
Sjors:
re-tACK fad798b
Tree-SHA512: a7377d0d5378be0a50be278d76396cc403583617b5fc43467773eee706df698acf3f4e67651491183b9b43a8e1816b052e4c17b90272b7ec4b6ac134ad811400
