31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0 [util] add RunCommandParseJSON (Sjors Provoost)
c17f54ee535faaedf9033717403e1f775b5f1530 [ci] use boost::process (Sjors Provoost)
32128ba682033560d6eb2e4848a9f77a842016d2 [doc] include Doxygen comments for HAVE_BOOST_PROCESS (Sjors Provoost)
3c84d85f7d218fa27e9343c5cd1a55e519218980 [build] msvc: add boost::process (Sjors Provoost)
c47e4bbf0b44f2de1278f9538124ec98ee0815bb [build] make boost-process opt-in (Sjors Provoost)
929cda5470f98d1ef85c05b1cad4e2fb9227e3b0 configure: add ax_boost_process (Sjors Provoost)
8314c23d7b39fc36dde8b40b03b6efbe96f85698 [depends] boost: patch unused variable in boost_process (Sjors Provoost)
Pull request description:
Prerequisite for external signer support in #16546. Big picture overview in [this gist](https://gist.github.com/Sjors/29d06728c685e6182828c1ce9b74483d).
This adds a new dependency [boost process](https://github.com/boostorg/process/tree/boost-1.64.0). This is part of Boost since 1.64 which is part of `depends`. Because the minimum Boost version is 1.47, this functionality is skipped for older versions of Boost.
Use `./configure --with-boost-process` to opt in, which checks for the presence of Boost::Process.
We add `UniValue runCommandParseJSON(const std::string& strCommand)` to `system.{h,cpp}` which calls an arbitrary command and processes the JSON returned by it. This is currently only called by the test suite.
~For testing purposes this adds a new regtest-only RPC method `runcommand`, as well as `test/mocks/command.py` used by functional tests.~ (this is no longer the case)
TODO:
- [ ] review boost process in #15440
ACKs for top commit:
achow101:
ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0
hebasto:
re-ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, only rebased (verified with `git range-diff`) and removed an unintentional tab character since the [previous](https://github.com/bitcoin/bitcoin/pull/15382#pullrequestreview-458371035) review.
meshcollider:
Very light utACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, although I am not very confident with build stuff.
promag:
Code review ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, don't mind the nit.
ryanofsky:
Code review ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0. I left some comments below that could be ignored or followed up later. The current change is clean and comprehensive.
Tree-SHA512: c506e747014b263606e1f538ed4624a8ad7bcf4e025cb700c12cc5739964e254dc04a2bbb848996b170e2ccec3fbfa4fe9e2b3976b191222cfb82fc3e6ab182d
fa8e6df282af0d396d75b03721f1b59a520ced19 ci: Run tsan ci config on cirrus (MarcoFalke)
Pull request description:
Fixes bitcoin-core/gui#12
Copied description from #19321:
Currently it is not possible to use travis in forked repositories due to the 50 minute limit on builds. A fresh build (uncached) of the thread sanitizer config takes more than 50 minutes.
One approach to fix this could be to throw away tests until the run time is less than 50 minutes. However, the risk of being blind of failures in the thrown away tests is not worth the gain. Also, to detect them, one has to run the tsan configuration nightly and failures could only be detected post-merge.
Another approach would be to ask travis support to raise the limit for a forked repository. This is a tedious and manual one-by-one process, so I'd rather not.
Finally, a different ci provider can be used, since the config files are designed to be platform-agnostic. This is what I picked.
I kept all settings identical to the travis machine for now. Both providers run in the google cloud, so this should be a "move-only".
ACKs for top commit:
fanquake:
ACK fa8e6df282af0d396d75b03721f1b59a520ced19 - my understanding is that test coverage remains the same. Just swapping providers to work-around the Travis time-limit in other repos.
Tree-SHA512: 26fed248a4f743107160d3b9e5df57fa0be280fd065ae6fece83d254f59d58ccf3e11a245519d158da109c47b053f62ee8756215008541973c65dc28c4efb748
a92e48b02df545e620a7b1de74f647f46413d3fb test: move TEST_RUNNER_EXTRA into native tsan setup (fanquake)
Pull request description:
`feature_block.py` is being run in the tsan job, i.e [here](https://travis-ci.org/github/bitcoin/bitcoin/jobs/703122309), even though it should be excluded. My hasty assumption is that this will fix it. In any case, all other instances of `TEST_RUNNER_EXTRA` seem to have moved out of `.travis.yml` and into the different CI configurations.
ACKs for top commit:
MarcoFalke:
ACK a92e48b02df545e620a7b1de74f647f46413d3fb
practicalswift:
ACK a92e48b02df545e620a7b1de74f647f46413d3fb -- patch looks correct
hebasto:
ACK a92e48b02df545e620a7b1de74f647f46413d3fb, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 86057bef2cc87c6acdbbf94f8cd7a5147510448c3e67aacde8daf247e3ccf649cfc5afbbd10693e084f426042d98150616c0e49bfa5f32b949dff9cebd2fd95d
fa2eb3d5d6819e42bfcec8a9f02b99438fe718b9 ci: Run asan ci config on cirrus (MarcoFalke)
fa93527738a62ebc13305adcb0fd2b5128073bbc cirrus: Clear dummy task (MarcoFalke)
Pull request description:
Currently it is not possible to use travis in forked repositories due to the 50 minute limit on builds. A fresh build (uncached) of the address sanitizer config takes more than 50 minutes.
One approach to fix this could be to throw away tests until the run time is less than 50 minutes. However, the risk of being blind of failures in the thrown away tests is not worth the gain. Also, to detect them, one has to run the asan configuration nightly and failures could only be detected post-merge.
Another approach would be to ask travis support to raise the limit for a forked repository. This is a tedious and manual one-by-one process, so I'd rather not.
Finally, a different ci provider can be used, since the config files are designed to be platform-agnostic. This is what I picked.
I kept all settings identical to the travis machine for now. Both providers run in the google cloud, so this should be a "move-only".
ACKs for top commit:
hebasto:
ACK fa2eb3d5d6819e42bfcec8a9f02b99438fe718b9
Tree-SHA512: 159d7dc6f5b24583e941282cdd40465b15db787f0a658a3e81a7b1a22abdb4cb573709b9b5c4465523e0ba0060b17a68fbdbda7a9ecdeb649f31535d377bbe75
fa7e002d520d8390f3ff4b0383cfdfc14713355d ci: tsan with wallet (MarcoFalke)
Pull request description:
ACKs for top commit:
practicalswift:
ACK fa7e002d520d8390f3ff4b0383cfdfc14713355d -- patch looks correct and Travis is happy
hebasto:
ACK fa7e002d520d8390f3ff4b0383cfdfc14713355d, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 1138459bbef72f402f32dae1e28d96f174901d4248d959b538b973747c8b06f221ecd81a386a1f915c0a2faaefb9fb8f11e3be39e6c5e2468bf9ae43d9f97757
BACKPORT NOTICE:
this PR doesn't actually swithc to libc++ due to multiple CI failures such as
linking errors or other
------------------
faf62e6ed0ca45db44c370844c3515eb5a8cda12 ci: Remove unused workaround (MarcoFalke)
fa7c8509153bfd2d5b4dcff86ad27dfd73e8788b ci: Install llvm to get llvm symbolizer (MarcoFalke)
fa563cef61e8a217c5e8ec059e174afae61087a5 test: Add more tsan suppressions (MarcoFalke)
fa0cc02c0a029133f080680ae9186002a144738f ci: Mute depends logs completely (MarcoFalke)
fa906bf2988c799765a04c484269f890964ec3ee test: Extend tsan suppressions for clang stdlib (MarcoFalke)
fa10d850790bbe52d948659bb1ebbb88fe718065 ci: Use libc++ instead of libstdc++ for tsan (MarcoFalke)
fa0d5ee1126a8cff9f30f863eb8f5c78bf57e168 ci: Set halt_on_error=1 for tsan (MarcoFalke)
fa2ffe87f794caa74f80c1c2d6e6067ee4849632 ci: Deduplicate DOCKER_EXEC (MarcoFalke)
fac2eeeb9d718bdb892eef9adf333ea61ba8f3d0 cirrus: Remove no longer needed install step (MarcoFalke)
Pull request description:
According to the [ThreadSanitizer docs](https://clang.llvm.org/docs/ThreadSanitizer.html#current-status):
> C++11 threading is supported with **llvm libc++**.
For example, the thread sanitizer build is currently not checking for double lock of mutexes.
Fixes (partially) https://github.com/bitcoin/bitcoin/issues/19038#issuecomment-632138003
ACKs for top commit:
practicalswift:
ACK faf62e6ed0ca45db44c370844c3515eb5a8cda12
fanquake:
ACK faf62e6ed0ca45db44c370844c3515eb5a8cda12
hebasto:
ACK faf62e6ed0ca45db44c370844c3515eb5a8cda12, maybe re-organize commits to modify suppressions in a single one?
Tree-SHA512: 98ce5154b4736dfb811ffdb6e6f63a7bc25fe50d3b73134404a8f3715ad53626c31f9c8132dbacf85de47b9409f1e17a4399e35f78b1da30b1577167ea2982ad
60824b3c3a4221a31c3638008b05fecc040c3df2 ci: Fix configure options for macOS builds (Hennadii Stepanov)
687939e3d251125aea5f6f7202e00a12d60a899f ci: Drop Homebrew caching while using Homebrew addon on Travis (Hennadii Stepanov)
557d3f1cc065702ebbf929d39ac7c5561111ce4c ci: Do not activate Travis ccache caching strategy (Hennadii Stepanov)
2d747428e23c0b7e3e7d0ce0db4226372f5ec0dd ci: Disable functional tests on forked repos to avoid timeouts for macOS (Hennadii Stepanov)
Pull request description:
See: https://github.com/bitcoin-core/gui/issues/5#issuecomment-656819184
Additionally, this PR:
- updates macOS image to the recent 10.15.5 version
- drops Homebrew caching as the Travis Homebrew addon have been used since #18438
My forked repo build: https://travis-ci.org/github/hebasto/bitcoin/jobs/707200431
Top commit has no ACKs.
Tree-SHA512: 398e935f965a04babeb10e7b26d2341562f21a1ef671c2e7cc97c9ec79d5c31643f81ca18561ab7714b5c52e19df2e4bffe4223eadbab984daa9418ffbf8c2a8
fa05f44893d228f672f39436d0cb6b3376f81ac2 ci: Upgrade most ci configs to focal (MarcoFalke)
fad67208914e5a74b64f4cc018368902ef3a2e9b doc: move doc to ci readme (MarcoFalke)
fa880773b425fcd292ed7669d237ee3151a15bc6 ci: Have one config run in xenial to test against python3.5 (MarcoFalke)
fa6ddb2fa167df52f59cb9eaabed48315ddcdb2e travis: Always run multiprocess build (MarcoFalke)
Pull request description:
Generally developers compile with recent compilers, so bumping the ci configs to a recent OS should be uncontroversial. Older OSes (especially with compiler sanitizers) need workarounds that can be dropped by running on a more recent OS.
This pull changes the asan sanitizer and the experimental multiprocess build to use focal.
Also, it runs the no_wallet config on xenial to test against python 3.5, according to `doc/dependencies.md`.
Finally, all configs that mimic gitian (win and mac) will stay at bionic.
ACKs for top commit:
Sjors:
ACK fa05f44893d228f672f39436d0cb6b3376f81ac2, assuming Travis passes
hebasto:
ACK fa05f44893d228f672f39436d0cb6b3376f81ac2
Tree-SHA512: 55ec56c71ba2280d27c1a8856a1e6c310b1fbf469d5a8a1dde228063e3892e1dd1e51408ecff7a3d77ac2ae018daa9e9bbbb60598cdeaab8c32a146b11b3e7c4
fae656144e9ec25a0b610cd5278c1b7c42880097 travis: Re-enable s390x (MarcoFalke)
Pull request description:
According to travis, the issue has been solved. Quote
> I would like to confirm that we have resolved this issue and most of our users are reported that this issue has been resolved on their end as well. Could you please re-check and see if that still exists for you?
ACKs for top commit:
theStack:
ACK fae656144e
Tree-SHA512: cf42f96d25474a9dcf0817a049e30e29714731d708f73c40a3042b0c70a71ff08f07dd96a89f0dcd5a50a63a355cf30b3511172a32b8af7d5a2e13ad222a4b49
fa7166759789c1282609ff3ab2e80d4f70910a9f ci: Move travis workarounds to .travis.yml (MarcoFalke)
Pull request description:
It seems odd to have travis related workarounds in the general ci config files. Fix that oddity by moving the travis related workarounds to the travis yaml file.
For unexplained reasons, this should also work around and thus close#19171
ACKs for top commit:
hebasto:
ACK fa7166759789c1282609ff3ab2e80d4f70910a9f, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: b4419d38e2b41f6e4d6e6b7658f1d972c40c390a49fe78808f8640d28efd84cc6668ce292d45b7c539e65b9e2ecbad10e796cb8f9329a0f1e7d0132ce962d226
e2bab2aa162ae38b2bf8195b577c982402fbee9d multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e708c04ef6c9880f89d0a4cbaa6fc7c5 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52bfcd4edf8553aaf332bfeb92fc554cc build: multiprocess autotools changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.
In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.
The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649
ACKs for top commit:
practicalswift:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d
Sjors:
tACK e2bab2aa162ae38b2bf8195b577c982402fbee9d on macOS 10.15.4
hebasto:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d, tested on Linux Mint 19.3 (x86_64):
Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
faf552117efdd456fb17b1d9490896de3b7d67a4 ci: Set DEBIAN_FRONTEND=noninteractive (MarcoFalke)
fa006caa136f05a18428c792802523637d2234ca ci: tsan on clang-9 (MarcoFalke)
Pull request description:
Bump the compiler runtime library that includes the sanitizers from clang-8 to clang-9 to get a more recent version. Also, bump the system packages from xenial to bionic to test packages closer to what is commonly used in production.
The second commit is needed to install the `tzdata` package, which is missing on some operating systems. See https://travis-ci.org/github/MarcoFalke/bitcoin-core/jobs/688455828#L1727
ACKs for top commit:
hebasto:
ACK faf552117efdd456fb17b1d9490896de3b7d67a4
practicalswift:
ACK faf552117efdd456fb17b1d9490896de3b7d67a4 -- patch looks correct and Travis is happy
Tree-SHA512: aa38fdae5f716966a83a21d5f7c121675cf7d663148ab3baa065142c8b3850bcd4bf88526d7da0fa51f5e08f2c317b537f950fcc9eb1e69fdacb0eac8863e1c6
cbd661122e5852d543467090459d33cf8cb4a3c7 Set LD_LIBRARY_PATH consistently in travis tests (Russell Yanofsky)
fa35c34df781cf46bbd15522961f214f03b958bf Remove unused ci configs that have been moved elsewhere (MarcoFalke)
3333cb96994182bbdbb21174b691feb716858bc2 fuzz: Pass down MAKEJOBS to test_runner (MarcoFalke)
Pull request description:
Just how `MAKEJOBS` is passed down to the functional test `test_runner`, do the same for the fuzz `test_runner`.
Also includes a commit to remove unused config files, which have been moved elsewhere.
Top commit has no ACKs.
Tree-SHA512: 32557102c9e40599b432aeb004c8427e8fbb07cdf4048050cdc8241d1b029aaad306b1131007eeca8315a4f71c38a7efbb833310e056cd11b835676cd19b8902
c456145b2c65f580683df03bf10cd39000cf24d5 [test] add 0.19 backwards compatibility tests (Sjors Provoost)
b769cd142deda74fe46e231cc7b687a86514f2f1 [test] add v0.17.1 wallet upgrade test (Sjors Provoost)
9d9390dab716f07057c94e8e21f3c7dd06192f35 [tests] add wallet backwards compatility tests (Sjors Provoost)
c7ca6308968b29a0e0edc485cd06e68e5edb7c7d [scripts] support release candidates of earlier releases (Sjors Provoost)
8b1460dbd1b732f06d4cebe1fa6844286c7a0056 [tests] check v0.17.1 and v0.18.1 backwards compatibility (Sjors Provoost)
ae379cf7d12943fc192d58176673bcfe7d53da53 [scripts] build earlier releases (Sjors Provoost)
Pull request description:
This PR adds binaries for 0.17, 0.18 and 0.19 to Travis and runs a basic block propagation test.
Includes test for upgrading v0.17.1 wallets and opening master wallets with older versions.
Usage:
```sh
contrib/devtools/previous_release.sh -f -b v0.19.0.1 v0.18.1 v0.17.1
test/functional/backwards_compatibility.py
```
Travis caches these earlier releases, so it should be able to run these tests with little performance impact.
Additional scenarios where it might be useful to run tests against earlier releases:
* creating a wallet with #11403's segwit implementation, copying it to an older node and making sure the user didn't lose any funds (although this PR doesn't support `v0.15.1`)
* future consensus changes
* P2P changes (e.g. to make sure we don't accidentally ban old nodes)
ACKs for top commit:
MarcoFalke:
ACK c456145b2c65f580683df03bf10cd39000cf24d5 🔨
Tree-SHA512: 360bd870603f95b14dc0cd629532cc147344f632b808617c18e1b585dfb1f082b401e5d493a48196b719e0aeaee533ae0a773dfc9f217f704aae898576c19232
3f686d1a286f88af7baa17c0c9dc439559a66e81 ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors (practicalswift)
Pull request description:
Re-introduce the Travis valgrind fuzzing job which was removed by PR #18899. The removal seems to have been made by accident since the removed job does not appear to be the source of the problem the PR set out to fix.
---
Run fuzz testing [test cases (bitcoin-core/qa-assets)](https://github.com/bitcoin-core/qa-assets) under `valgrind`.
This would have caught `util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t) by checking gmtime_s/gmtime_r return value` (#18162) and similar cases.
This fuzzing job was introduced in #18166.
Top commit has no ACKs.
Tree-SHA512: 6e2681eb0ade6af465c5ea91ac163a337465d2130ec9880ba57a36d9af7c25682734586a32977dc25972d4f78483f339d680ea48c0ae13cf1dfa52b617aae401
6136a96cdf631f824fb89cf2720422787439acc9 ci: Rename RUN_CI_ON_HOST to DANGER_RUN_CI_ON_HOST (Hennadii Stepanov)
97ba77aa8e58c3c7ff767e6ca804d0e46370e5dc ci: Add native s390x (Hennadii Stepanov)
Pull request description:
Unlike the Docker wrapped solution (#17591) this PR suggests running on host system directly.
This approach makes builds quick and stable (see: #18106).
The excerpt from the Travis log:
```
...
Running on host system without docker wrapper
...
Byte Order: Big Endian
...
```
ACKs for top commit:
MarcoFalke:
ACK 6136a96cdf631f824fb89cf2720422787439acc9
Tree-SHA512: 1b591de13e38d10a35217e1de11cbd648a359d18d16eed166fac18ea5788b58cc9fc6d407086ed342b99e57e479efd951a0ea693710177e500eb116316b9a788
25c8b73656c90f7b78a90f25c5bcc308f8b7c598 ci: Use Homebrew addon on native macOS (Hennadii Stepanov)
596c627a1eb2b32e2755ae5b9bf32235cf8ce72b ci: Fix brew in Travis (Hennadii Stepanov)
Pull request description:
Recently almost every macOS image update on Travis breaks our builds:
- #17848
- #18436
This PR:
- fixes the error caused by the recent [update](https://changelog.travis-ci.com/xcode-11-3-1-xcode-11-2-1-xcode-11-1-and-xcode11-images-updated-142286) from 10.14.4 (18E226) to 10.14.6 (18G3020) on March 25
- leverages [Homebrew addon](https://config.travis-ci.com/ref/job/addons/homebrew) to install packages
Homebrew is not told to install `automake` and `pkg-config` packages, as the [docs](https://docs.travis-ci.com/user/reference/osx/#compilers-and-build-toolchain) states that they are pre-installed:
> - automake 1.16.1
> - pkg-config 0.29.2
Top commit has no ACKs.
Tree-SHA512: 1a70c06468fbe162503081b03dcf54614d67abf8ff0ce07d118b5bb50bbb92c182a76f769bea586c691aa82b9281a29cdef88091acc16895817a2e7cddafec6e
fa082d0a57afedca9122fac4aecd6a3070f06b04 travis: Remove valgrind (MarcoFalke)
Pull request description:
When the valgrind run was added, it took 2 hours. Travis kindly raised the timeout limit to the maximum possible of 3 hours.
Today, a full build of Bitcoin Core with all tests takes more than three hours. Thus, it is impossible to run all tests on travis.
Moreover, the feedback loop for developers that create a pull request takes at least 2 hours, but in some cases (when the travis queue is full) until the next day. This is unacceptable.
Fix both issues by removing the build from travis.
Please note that the `ci/test/` configurations are *not* removed. They will stay in the repo and can be executed anywhere (just not on travis).
ACKs for top commit:
jamesob:
ACK fa082d0a57
jnewbery:
utACK fa082d0a57afedca9122fac4aecd6a3070f06b04
Tree-SHA512: 9acaa0e2d3926014fadb7dd2e86c4e01df382e9399f6ae99f989fa609da66a77bdd1b75d6ff42d2686f38f730b8564e6dc722aa597a473290c9d30c2abe7ef0f
b0a01299ed8cc2026e91a8dcad880b700e462cee test: Disable s390 build on travis (Wladimir J. van der Laan)
Pull request description:
Travis is consistently failing on s390 due to out of disk space issues,
which causes false positives. Disable the s390 build for now until
this is fixed some other way.
ACKs for top commit:
MarcoFalke:
ACK b0a01299ed8cc2026e91a8dcad880b700e462cee 🍠
jonasschnelli:
ACK b0a01299ed8cc2026e91a8dcad880b700e462cee
Tree-SHA512: ae8a78fa03bc620d802e4992d46929a0a4b801b50a6066d8f093f93fc5ebbc010d221c04787cbfc75436a2c40f9292257bf85fbb251593e21afff11756b6d8d4
ef63f5fc1136ad2a2cd080d44142a2ee3945c238 ci: Combine 32-bit build with CentOS 7 build (Sebastian Falbesoner)
Pull request description:
Combines the CentOS build with the 32-bit (i686) build to avoid Travis bottlenecks, as suggested in #17757 by MarcoFalke. This keeps most of the properties of the 32-bit build (dash as config shell, building QT5 GUI) and just builds it with depends inside the CentOS docker container.
Making the depends in `05_before_script.sh` with unset config shell (`CONFIG_SHELL=`)
6196e93001/ci/test/05_before_script.sh (L28)
caused problems for building the library libevent (resulting in a Makefile with no shell set (`SHELL=`)), that's why I set it explicitely to `/bin/bash` if we have a CentOS Docker container.
A Travis output of this 32-bit CentOS build can be seen here: https://travis-ci.org/theStack/bitcoin/jobs/634472394 (has been restarted once due to too long build time and appearance of the `CACHE_ERR_MSG`).
For anyone wanting to verify the outputs, I found these instructions useful to reproduce a Travis build locally: https://github.com/erdc/proteus/wiki/Replicating-the-TravisCI-Environment-on-your-Local-Machine (steps 1-3). In this case it's a bit tricky since you run Docker inside Docker -- within the Travis Docker container, the CentOS Docker container is created. To make this possible, the Docker socket has to be exposed to the Travis container via bind-mounting (`docker run -v /var/run/docker.sock:/var/run/docker.sock ...`), as suggested in https://stackoverflow.com/a/33003273.
Top commit has no ACKs.
Tree-SHA512: af508241cec3a10a66c37673d56691717b78375340e910fcdd3fb3870741eba623a436e1e85b26b54f013375611896f5411c5a7fec2437d367d27172230129fe
faa079db92e614bddfe8bd323ddcd2adb77e0961 travis: Disable feature_block in tsan run (MarcoFalke)
Pull request description:
The `feature_block` test causes the travis machine to OOM, when run with the thread sanitizer.
The stderr says:
```
==27237==ERROR: ThreadSanitizer failed to allocate 0xf6000 (1007616) bytes of LargeMmapAllocator (error code: 12)
...
FATAL: ThreadSanitizer CHECK failed: /build/llvm-toolchain-3.8-_PD09B/llvm-toolchain-3.8-3.8/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
ERROR: Failed to mmap
```
(from https://travis-ci.org/bitcoin/bitcoin/jobs/588194563#L10505)
Fix this by disabling `feature_block` on travis. Longer term, I'd like to move away from travis, but I'll leave this for a follow-up.
ACKs for top commit:
fanquake:
ACK faa079db92e614bddfe8bd323ddcd2adb77e0961
Tree-SHA512: c0dc2272853aac53f68eb9e110c8500c4a92211ba89d856660bacdf6e959d875477e422b3280b743d85fc8a65e083bf9153911f12039d026e2501f426540dac4
BIP70 is removed in "merge #17165: Remove BIP70 support (#4023)".
So, this commit contains only some follow-ups to unify codebase
e09913f1c47e693b0c6fafef55b9ca78e5f3abc0 doc: specify protobuf as optional in build docs (fanquake)
376f4929f8f75011b72b2f9c3164980db482278a build: disable BIP70 support by default (fanquake)
Pull request description:
Disable BIP70 support in the GUI by default for `0.19.0` (for eventual removal in `0.20.0`?).
Users who want to compile with BIP70 support enabled can pass `--enable-bip70` to `./configure`.
I've inverted the current `--disable-bip70` test to instead pass `--enable-bip70`.
Tested configurations on `macOS` (`protobuf` installed with `brew`).
Protobuf available and `./configure`:
```
Options used to compile and link:
with wallet = yes
with gui / qt = yes
with bip70 = no
```
Protobuf available and `./configure --enable-bip70`:
```
Options used to compile and link:
with wallet = yes
with gui / qt = yes
with bip70 = yes
```
Protobuf not available (i.e `brew unlink protobuf`) and `./configure`:
```
Options used to compile and link:
with wallet = yes
with gui / qt = yes
with bip70 = no
```
Protobuf not available and `./configure --enable-bip70`:
```
checking whether to build test_bitcoin-qt... yes
checking whether to build BIP70 support... configure: error: protobuf missing
```
TODO:
- [x] Remove `protobuf` from other Travis builds
- [ ] Documentation updates (mention that `protobuf` is now optional)?
- [ ] Could split release notes into GUI and build
ACKs for top commit:
laanwj:
ACK e09913f1c47e693b0c6fafef55b9ca78e5f3abc0
elichai:
ACK e09913f1c47e693b0c6fafef55b9ca78e5f3abc0 Read the autotools changes. awesome that this removes the protobuf requirement.
practicalswift:
ACK e09913f1c47e693b0c6fafef55b9ca78e5f3abc0 -- diff looks correct
Tree-SHA512: 7bf87ae8555e24db2da2e89cc4d4e90d09be27499ad386ad65879d05df8f96d9a1384379891ac8963d17728c90e55961560813df97e849e631e2de8c08e210c8
fad88e6f86d1dd32cf01db2287df9c63e66c5116 ci: Remove use of cd (MarcoFalke)
fa2941bbf47a8a6b79b8db4a87e1aedcf6a29a5e ci: Remove unmaintained extended_lint (MarcoFalke)
fa041875284d01602647519cc452185ba1ad5a8f scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR (MarcoFalke)
fa0656d1ebc2a01d900fead4ce055d59eb818a1f ci: Add DEPENDS_DIR variable, Add documentation for folders (MarcoFalke)
faeeca87b65dd98e0efbc54443b3f8854cae9c00 scripted-diff: Move various folders to ci scratch dir (MarcoFalke)
Pull request description:
Some minor cleanups:
* Remove unused and unmaintained extended_lint to avoid ci bottlenecks and waste of CPU
* Move all folders that hold temporary ci files to the ci scratch dir (except for the build dirs)
* Add some documentation to folders and remove the `BASE_BUILD_DIR` alias for the root directory
* Fixes#17178 by removing `cd`
Top commit has no ACKs.
Tree-SHA512: f6eb9d47ab98c08af59a79c2a4bf62fc331f0f6f3174909a28f2c2f48b0234423da5aee876225d2e188619b71b008d882a20d29a7aca68248039ea5080be5af5
e1900008699bd45031b7faa9ef3d0a81d54091b0 ci-s390x: Add qemu and depends support in the ci script (Elichai Turkel)
Pull request description:
Related: #17599
This adds qemu support just like we have in arm and compile the depends.
other than that I also fixed some missing includes to make the depends compile.
ACKs for top commit:
MarcoFalke:
ACK e190000 (first commit only, didn't look at second commit)
Tree-SHA512: 2b8a39772b86408569f52cdc33832dbce7e5e9cdd710524295f3d259628cdfc017e740f6f94941307d7f8e413236814a95ba851153c617eb5fb75b4bd9a7e52f
fabd71076cd9493bd2d30a198467f5ea621b27aa ci: Print free disk space (MarcoFalke)
fad9fdbea5dfb19328282afda9588edc6f1d0ddf test: Properly deserialize integers in little-endian (MarcoFalke)
fa94fc10c881e502e6c9a71f3b7719aa955900f9 ci: Run functional tests on s390x (MarcoFalke)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: 98ba77eb56f283131fdaeb393fda86cc308f1bf9781e1e0e5736b8d616528dc8ff2e494d55ba107c138083025c66a59e382fcfa9962d4349a5fd6cbbc52484c3
## Issue being fixed or feature implemented
During reviewing TODO were found some TODOes that can be done now.
## What was done?
- fix: follow-up dash#3467 - replaced commented code to disabled code
- follow-up bitcoin#16394 - uncommented code related to `watchonly`
feature
- removed out-dated TODO in `rpc/masternode.cpp` (already done)
- fix: renamed name of clean up test_unittests: removed TODO and updated
name of variable TRAVIS
- rewritten todo inside `.travis.yml`
- fix: adds a missing description for result of rpc `mnsync`
Last commit (`mnsync`) is an only candidate for backport to v19, other
changes are non significant.
## How Has This Been Tested?
Run functional/unit tests
## Breaking Changes
No breaking changes
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
fa2056af1c travis: Properly cache and error on timeout (MarcoFalke)
fa36a333ee travis: Switch to ubuntu keyserver to avoid timeouts (MarcoFalke)
Pull request description:
The other keyserver is consistently timing out on travis: https://travis-ci.org/bitcoin/bitcoin/jobs/512689710#L405
Attempt to fix it by using a different server.
Also:
* fixes#15372
* fixes#15738
ACKs for commit fa2056:
ryanofsky:
utACK fa2056af1c71aded3a821a07ec4de71c4be0bca3. All good changes (changing keyserver, getting rid of keyserver while loop, clarifying travis error, moving travis documentation to code comment).
Tree-SHA512: ac8436616ecfee0ed579114e19f03c53ceb688fbcd95a60cffe8f15b4e569772a6ba673f353bbd789e79fe27fc5626c77fab4086768844dd51e0c6c108b52fb2
f2472f64604a0c583f950c56e8753d0bee246388 tests: Improve test runner output in case of target errors (practicalswift)
733bbec34fbec85574cc456832b2b2f807e5dce9 tests: Add --exclude integer,parse_iso8601 (temporarily) to make Travis pass until uninitialized read issue in FormatISO8601DateTime is fixed (practicalswift)
5ea81449f30a6fe6db3b6df5e8009f21a782ff44 tests: Add support for excluding fuzz targets using -x/--exclude (practicalswift)
555236f769c13518db70f5df36e5688d63486bd5 tests: Remove -detect_leaks=0 from test/fuzz/test_runner.py - no longer needed (practicalswift)
a3b539a924f8611abb3096f2bd9d35094b5577e3 ci: Run fuzz testing test cases under valgrind (practicalswift)
Pull request description:
Run fuzz testing [test cases (bitcoin-core/qa-assets)](https://github.com/bitcoin-core/qa-assets) under `valgrind`.
This would have caught `util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t) by checking gmtime_s/gmtime_r return value` (#18162) and similar cases.
ACKs for top commit:
MarcoFalke:
ACK f2472f64604a0c583f950c56e8753d0bee246388 👼
Tree-SHA512: bb0879d40167cf6906bc0ed31bed39db83c39c7beb46026f7b0ee53f28ff0526ad6fabc3f4cb3f5f18d3b8cafdcbf5f30105b35919f4e83697c71e838ed71493
711e0449cf4a0f15cabe0d64094e3add24ad44b0 ci: Remove trusty build (Hennadii Stepanov)
7f3ae224685efaeb6fe714de90e8871d12e55f34 ci: Add CentOS 7 build (Hennadii Stepanov)
Pull request description:
Arguably, CentOS is the most conservative distro of all the popular ones. Thus, it could be a good way to check the Bitcoin Core compatibility with aged dependencies.
Currently, CentOS 7 has:
- Berkeley DB == 4.8.30
- Boost == 1.53.0
- GCC == 4.8.5
- libevent == 2.0.21 < minimum required [2.0.22](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md), but tests passed
- MiniUPnPc == 2.0
- Python == 3.6.8
- qrencode == 3.4.1
- Qt == 5.9.7
- ZeroMQ == 4.1.4
~Please note that this PR is based on the bugfix #17634.~
Also trusty build has been removed for the following reasons:
- https://github.com/bitcoin/bitcoin/issues/17628#issuecomment-559448201:
> Maybe it'd make sense to replace Ubuntu Trusty with Centos 7 as the "check ancient backward compatibililty" Travis run. It's supported until 2024, apparently.
- https://github.com/bitcoin/bitcoin/pull/17635#discussion_r354811792:
> Our travis is currently running at its limit and this doesn't seem like it is adding a lot new coverage compared to the other builds.
Close#17628
ACKs for top commit:
MarcoFalke:
ACK 711e0449cf4a0f15cabe0d64094e3add24ad44b0 🚠
Tree-SHA512: 614ec8394943f482a5867067f7119bffd052924a51e32ffda9a08e10c392c4a955a3539e2f8907cb65bfd9347dadf0ba62f6d1530bbc49927c347360a5a7f73c
9999a790749c583845a3ff7a265d540c2905aa12 travis: Rework CACHE_ERR_MSG (MarcoFalke)
Pull request description:
This should fix the misconception that we can fix build problems in forked repos. See https://github.com/bitcoin/bitcoin/pull/17418#issuecomment-551917920
Top commit has no ACKs.
Tree-SHA512: e0c86f6613fa38f4d4984420fa1fdc999829407c71024c3b92fd1d35afa98f4b5073af72656c83f607d5f6fab15f9ea629ca8be0b4e892c2fb65a226036c470a
facc0da63a8fa4bd6fc2782cbe92eb9f920f2256 travis: Run unit and functional tests on native arm (MarcoFalke)
fafa064d2a8dbe24303545ab582ec84cde52ab5b ci: Remove ccache requirement on the host (MarcoFalke)
Pull request description:
This keeps the cross-compilation to make it easy to run the ci on non-arm hardware. To run this locally in qemu-user as it used to be, just `export QEMU_USER_CMD="qemu-arm -L /usr/arm-linux-gnueabihf/"`.
ACKs for top commit:
laanwj:
LGTM ACK facc0da63a8fa4bd6fc2782cbe92eb9f920f2256
practicalswift:
ACK facc0da63a8fa4bd6fc2782cbe92eb9f920f2256 -- diff looks correct and Travis seems happy
Tree-SHA512: 0dc1bc82eb93e2bd8b159e044f20fe3055f8cdfd73aaa238bd2e178397582144dfc0c6a87bd8270115dafea1a623e642bde5d5f30254f94140f1a2cdb12fc2da
fa677d1801fb9153a95a1fc9855fd5f21fc440c0 ci: Remove redundant check for TRAVIS_OS_NAME (MarcoFalke)
fadccb263baf6b8694f750623add42f966e423a3 doc: Document that GNU tools are required for linters (MarcoFalke)
4444704ca9f66cdc24ab2d444941354db1dfed06 ci: Cleanup macOS runs (MarcoFalke)
Pull request description:
* Remove a commented out cleanup task in `before_cache`
* Remove the linter run on macOS, and document that GNU tools are required to run the linters
ACKs for top commit:
Sjors:
Code review ACK fa677d1801fb9153a95a1fc9855fd5f21fc440c0
laanwj:
ACK fa677d1801fb9153a95a1fc9855fd5f21fc440c0
ryanofsky:
Code review ACK fa677d1801fb9153a95a1fc9855fd5f21fc440c0 for new third commit replacing TRAVIS_OS_NAME check with NO_DEPENDS setting
Tree-SHA512: 9122a63bbe7887d9e379123152ea4ba44324cb18033b9e6b45bfdb1af665c10ea598564b9fcd57330d208a08e4696e41b4d6175f05f0843a3a76530da114f8c6
da1f153e5e260f1744ee1bf4f24ca3a74ffea465 Add s390x tests to travis (Elichai Turkel)
2fa65e0de94f01d502e8ace89be3c5dc963dd764 Add ci script to install on s390x (Elichai Turkel)
Pull request description:
Discovered this as part of #17402 and a conversation with gmaxwell.
You can see here that the platform is indeed BE: https://travis-ci.org/elichai/bitcoin/jobs/616656410#L36
This closes https://github.com/bitcoin/bitcoin/issues/6466
ACKs for top commit:
MarcoFalke:
ACK da1f153e5e260f1744ee1bf4f24ca3a74ffea465
Tree-SHA512: e7e94e54e220257d91b24fddc79eab2bcaaadf0b2d1e7e6872d9757808ab2541728f00b1f3ab7e343305c0e7d91bb48a17a3f9621f6fff6c9fe6cde6682de408
