fae49f6e424f31e93c5620d5ff893fb517ef4a8b ci: Add and document BASE_BUILD_DIR (MarcoFalke)
Pull request description:
Also fixes#18768
ACKs for top commit:
hebasto:
re-ACK fae49f6e424f31e93c5620d5ff893fb517ef4a8b, which is essentially the same as the previously [reviewed changes](https://github.com/bitcoin/bitcoin/pull/18735#pullrequestreview-400581536).
Tree-SHA512: 216565a05ccd513dd9f114b2333d3c283fd71914d32f9b05f145cb7c70633b083ff8ef60798d6f22f4be6a4d652b03806551fd74b5b596c92968501a4d9726d2
It does not includes changes that actually enable werror by default.
But this backport is necessary to prevent compilation error on mac such as this one:
In file included from mapport.cpp:26:
In file included from /builds/dashpay/dash/depends/x86_64-apple-darwin/include/miniupnpc/miniupnpc.h:14:
/builds/dashpay/dash/depends/x86_64-apple-darwin/include/miniupnpc/upnpdev.h:27:14: error: zero size arrays are an extension [-Werror,-Wzero-length-array]
char buffer[0];
fa23fbb42fa710e6be405596e2abfc3a289d020d ci: Run all tests on native mac again (MarcoFalke)
Pull request description:
They should pass again after f6072e601af68f3eee307478ad22ff3960680656
ACKs for top commit:
practicalswift:
ACK fa23fbb42fa710e6be405596e2abfc3a289d020d -- Travis is happy and so am I
Tree-SHA512: 49c16b6056d4e67d12a202744e1c56fee2788830213fe4a195955ad44c6b8ecce768a591463ffa0048821959a75b6fad4178629a8866c4a26799c4c8c13e933d
fad798be76dd5e330463c837fda768477d536078 test: Default --previous-releases to false if dir is empty (MarcoFalke)
faf1c3cc58d14f86ba5364e6ee5c8ef29cac2e26 test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option (MarcoFalke)
Pull request description:
The "auto-detection" feature is kept in place, but making it an option allows to properly document it. For example, on my machine I get:
```
$ ./test/functional/wallet_disable.py --help | grep previous-releases
--previous-releases Force test of previous releases (default: False)
ACKs for top commit:
Sjors:
re-tACK fad798b
Tree-SHA512: a7377d0d5378be0a50be278d76396cc403583617b5fc43467773eee706df698acf3f4e67651491183b9b43a8e1816b052e4c17b90272b7ec4b6ac134ad811400
faf552117efdd456fb17b1d9490896de3b7d67a4 ci: Set DEBIAN_FRONTEND=noninteractive (MarcoFalke)
fa006caa136f05a18428c792802523637d2234ca ci: tsan on clang-9 (MarcoFalke)
Pull request description:
Bump the compiler runtime library that includes the sanitizers from clang-8 to clang-9 to get a more recent version. Also, bump the system packages from xenial to bionic to test packages closer to what is commonly used in production.
The second commit is needed to install the `tzdata` package, which is missing on some operating systems. See https://travis-ci.org/github/MarcoFalke/bitcoin-core/jobs/688455828#L1727
ACKs for top commit:
hebasto:
ACK faf552117efdd456fb17b1d9490896de3b7d67a4
practicalswift:
ACK faf552117efdd456fb17b1d9490896de3b7d67a4 -- patch looks correct and Travis is happy
Tree-SHA512: aa38fdae5f716966a83a21d5f7c121675cf7d663148ab3baa065142c8b3850bcd4bf88526d7da0fa51f5e08f2c317b537f950fcc9eb1e69fdacb0eac8863e1c6
cbd661122e5852d543467090459d33cf8cb4a3c7 Set LD_LIBRARY_PATH consistently in travis tests (Russell Yanofsky)
fa35c34df781cf46bbd15522961f214f03b958bf Remove unused ci configs that have been moved elsewhere (MarcoFalke)
3333cb96994182bbdbb21174b691feb716858bc2 fuzz: Pass down MAKEJOBS to test_runner (MarcoFalke)
Pull request description:
Just how `MAKEJOBS` is passed down to the functional test `test_runner`, do the same for the fuzz `test_runner`.
Also includes a commit to remove unused config files, which have been moved elsewhere.
Top commit has no ACKs.
Tree-SHA512: 32557102c9e40599b432aeb004c8427e8fbb07cdf4048050cdc8241d1b029aaad306b1131007eeca8315a4f71c38a7efbb833310e056cd11b835676cd19b8902
fa72a751026d43e1d01ae30e26bcfd9b4cc0cf45 ci: Document why tests can not be run on mac (MarcoFalke)
Pull request description:
Fixes#18794
Top commit has no ACKs.
Tree-SHA512: 297652eda412aa8cf7255e20a6f294d22773dad8637a3d7b5204f3b638e911ce5b2e40e85f81395a34c1b5a5b497665944c2d6ea17c70c30c0c9e0ab553f956e
fac24dea00c54fe1e2dcc7625bb65ada8b97ac0c ci: Run functional tests on mac again (MarcoFalke)
Pull request description:
ACKs for top commit:
hebasto:
ACK fac24dea00c54fe1e2dcc7625bb65ada8b97ac0c, verified travis build log.
Tree-SHA512: 406282a7ac03e5c193830b727366c7b1350639f1850aff951bf7ddd4b0c3e3ffb396b950ccb3a64ddc59500fa2739766f3c34806b4d144bc4535bb2bd765b959
e4c0cada791135e2d0a36638541c03feff0bd6bc ci, gitian: Drop unneeded python3-dev package for macOS builds (Hennadii Stepanov)
Pull request description:
ACKs for top commit:
fanquake:
ACK e4c0cada791135e2d0a36638541c03feff0bd6bc - gitian builds match and I checked that this doesn't end up installed as a side-effect of another package.
Tree-SHA512: 520a3909b106a0e005b195c5395691edf62b76ee2df43b6971b7aa193648d68e6dac69cb4f1dc474f594b015a2fc2074061865e571d89365174beb5c1780356f
34c80d9eee7d21755f2bb80f7c97fd30d2c7b656 test: Add option to git-subtree-check to do full check, add help (Wladimir J. van der Laan)
Pull request description:
This adds a brief help text to `git-subtree-check.sh` and adds an option to do a full remote check instead of having two different code paths with a successful exit status. Also make it explicit that the CI is not doing this.
ACKs for top commit:
fjahr:
tested ACK 34c80d9eee7d21755f2bb80f7c97fd30d2c7b656
Tree-SHA512: 20f672fd3b3c1d633eccf9998fdd738194cdd7d10cc206691f2dcc28bbbf8187b8d06b87814f875a06145b179f5ca1f4f4f9922972be72759cf5ac6e0c11abd1
aaaaad6ac95b402fe18d019d67897ced6b316ee0 scripted-diff: Bump copyright of files changed in 2019 (MarcoFalke)
Pull request description:
ACKs for top commit:
practicalswift:
ACK aaaaad6ac95b402fe18d019d67897ced6b316ee0
promag:
ACK aaaaad6ac95b402fe18d019d67897ced6b316ee0 🎉
fanquake:
ACK aaaaad6ac95b402fe18d019d67897ced6b316ee0 - going to merge this now because the year is over and conflicts are minimal.
Tree-SHA512: 58cb1f53bc4c1395b2766f36fabc7e2332e213780a802762fff0afd59468dad0c3265f553714d761c7a2c44ff90f7dc250f04458f4b2eb8eef8b94f8c9891321
fa80e10d94dbf86da84fc761b09fb631155a5b25 test: Add feature_taproot.py --previous_release (MarcoFalke)
85ccffa26686c6c9adbd18bdde37fc1747281bab test: move releases download incantation to README (Sjors Provoost)
29d6b1da2a862bfbb14e7821979c97416c5400e8 test: previous releases: add v0.20.1 (Sjors Provoost)
Pull request description:
Disabling the new consensus code at runtime is fine, but potentially fragile and incomplete. Fix that by giving the option to run with a version that has been compiled without any taproot code.
ACKs for top commit:
Sjors:
tACK fa80e10
NelsonGaldeman:
tACK fa80e10d94dbf86da84fc761b09fb631155a5b25
Tree-SHA512: 1a1feef823f08c05268759645a8974e1b2d39a024258f5e6acecbe25097aae3fa9302c27262978b40f1aa8e7b525b60c0047199010f2a5d6017dd6434b4066f0
0374e821bd9e9498ce9c03aa8e5435870019978b util: Hard code previous release tarball checksums (Hennadii Stepanov)
bd897ce79f72a44a2e609f95433e251a3fd9eb9c scripted-diff: Move previous_release.py to test/get_previous_releases.py (Hennadii Stepanov)
Pull request description:
#19205 introduced signature verifying for the downloaded `SHA256SUMS.asc`.
This approach is brittle and does not work in CI environment for many reasons:
- https://github.com/bitcoin/bitcoin/issues/19812#issuecomment-680760663
- https://github.com/bitcoin/bitcoin/pull/19013#discussion_r459590779
This PR:
- implements **Sjors**' [idea](https://github.com/bitcoin/bitcoin/pull/19205#pullrequestreview-426080048):
> Alternatively we might as well hard code the checksum for each `tar.gz` release in the source code, here.
- is an alternative to 5a2c31e528e6bd60635096f233252f3c717f366d (#19013)
- fixes#19812
- updates v0.17.1 to v0.17.2
ACKs for top commit:
MarcoFalke:
cr ACK 0374e821bd9e9498ce9c03aa8e5435870019978b
Sjors:
tACK 0374e821bd9e9498ce9c03aa8e5435870019978b
Tree-SHA512: cacdcf9f5209eae7da357abb3445585ad2f980920fd5bf75527ce89974d3f531a4cf8b5b35edfc116b23bfdfb45c0437cb14cbc416d76ed2dc5b9e6d33cdad71
d135c294764add81683ba47575f9a5dde7d7c07f [ci] make list of previous releases to download a setting (Sjors Provoost)
9c246b873c74834a121edba00fcaecf0cba6f9b4 [test] backwards compatibility: bump v0.19.0.1 to v0.19.1 (Sjors Provoost)
89a28e02fa46f3d5eb07ab02aa34aa95c6fcee11 [test] add v0.16.3 backwards compatibility test (Sjors Provoost)
Pull request description:
Thanks to #18774's `adjust_bitcoin_conf_for_pre_17` we can now test backwards compatibility for v0.16.3, both for sync and loading a recent wallet.
This PR bumps v0.19.0.1 to v0.19.1.
I also made the version list consistent for the `contrib/devtools/previous_release.sh` instruction, between both tests.
ACKs for top commit:
MarcoFalke:
ACK d135c294764add81683ba47575f9a5dde7d7c07f
Tree-SHA512: 5ff137a7a934237fa220f1c2807ce9abeeb75929266558bf3e4045bec7dfcd0a8747fa74d700065c568330b18badf58c60c308eb13d1eed444d4bbfe6decc48b
c456145b2c65f580683df03bf10cd39000cf24d5 [test] add 0.19 backwards compatibility tests (Sjors Provoost)
b769cd142deda74fe46e231cc7b687a86514f2f1 [test] add v0.17.1 wallet upgrade test (Sjors Provoost)
9d9390dab716f07057c94e8e21f3c7dd06192f35 [tests] add wallet backwards compatility tests (Sjors Provoost)
c7ca6308968b29a0e0edc485cd06e68e5edb7c7d [scripts] support release candidates of earlier releases (Sjors Provoost)
8b1460dbd1b732f06d4cebe1fa6844286c7a0056 [tests] check v0.17.1 and v0.18.1 backwards compatibility (Sjors Provoost)
ae379cf7d12943fc192d58176673bcfe7d53da53 [scripts] build earlier releases (Sjors Provoost)
Pull request description:
This PR adds binaries for 0.17, 0.18 and 0.19 to Travis and runs a basic block propagation test.
Includes test for upgrading v0.17.1 wallets and opening master wallets with older versions.
Usage:
```sh
contrib/devtools/previous_release.sh -f -b v0.19.0.1 v0.18.1 v0.17.1
test/functional/backwards_compatibility.py
```
Travis caches these earlier releases, so it should be able to run these tests with little performance impact.
Additional scenarios where it might be useful to run tests against earlier releases:
* creating a wallet with #11403's segwit implementation, copying it to an older node and making sure the user didn't lose any funds (although this PR doesn't support `v0.15.1`)
* future consensus changes
* P2P changes (e.g. to make sure we don't accidentally ban old nodes)
ACKs for top commit:
MarcoFalke:
ACK c456145b2c65f580683df03bf10cd39000cf24d5 🔨
Tree-SHA512: 360bd870603f95b14dc0cd629532cc147344f632b808617c18e1b585dfb1f082b401e5d493a48196b719e0aeaee533ae0a773dfc9f217f704aae898576c19232
0c6318788beaf1a31aeba5a21f3f8bb5c07cea6c ci: Limit cache size regardless of NO_DEPENDS (Hennadii Stepanov)
Pull request description:
Close#18666.
ACKs for top commit:
MarcoFalke:
ACK 0c6318788beaf1a31aeba5a21f3f8bb5c07cea6c . Depends has ccache disabled anyway and is cached regardless of whether ccache is there or not, see #17248
Tree-SHA512: b1bf98be0f844b4704abd177841b014f3900be8160496f0d12596310db607b4f544547e8c3cbfcf17c086a78afd251653363f3dd467b769ac0062bc19adc8144
fa4632c41714dfaa699bacc6a947d72668a4deef test: Move boost/stdlib includes last (MarcoFalke)
fa488f131fd4f5bab0d01376c5a5013306f1abcd scripted-diff: Bump copyright headers (MarcoFalke)
fac5c373006a9e4bcbb56843bb85f1aca4d87599 scripted-diff: Sort test includes (MarcoFalke)
Pull request description:
When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.
This pull preempts both issues by just sorting all includes in one commit.
Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.
Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.
ACKs for top commit:
practicalswift:
ACK fa4632c41714dfaa699bacc6a947d72668a4deef
jonatack:
ACK fa4632c41714dfaa, light review and sanity checks with gcc build and clang fuzz build
Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
## Issue being fixed or feature implemented
Make CI builds faster.
We have this
https://github.com/dashpay/dash/blob/develop/ci/dash/build_src.sh#L18
but by that time `CCACHE_SIZE` is set to 100M via
https://github.com/dashpay/dash/blob/develop/ci/test/00_setup_env.sh#L51
already.
## What was done?
set `CCACHE_SIZE` variable in `.gitlab-ci.yml`, drop confusing and
useless lines from `ci/dash/build_src.sh` (`CCACHE_SIZE` and
`CCACHE_COMPRESS` defaults are handled by `ci/test/00_setup_env.sh`)
## How Has This Been Tested?
I set `CCACHE_SIZE` to 400M yesterday via Gitlab UI to test it.
results:
before, trivial doc change, 100M:
https://gitlab.com/dashpay/dash/-/jobs/4931233566#L59 (~17 minutes)
after, trivial code change, 400M:
https://gitlab.com/dashpay/dash/-/jobs/4935764148#L80 (~12 minutes)
Removed it from Gitlab UI now but it should still say `Set cache size
limit to 400.0 MB` in logs for this PR.
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
40% smaller CI job artifacts. Should help with issues like
https://gitlab.com/dashpay/dash/-/jobs/4759700026 in #5493.
## What was done?
## How Has This Been Tested?
https://gitlab.com/UdjinM6/dash/-/jobs/4773517599
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
Used to avoid the following error:
```
In file included from test/fuzz/addition_overflow.cpp:7:
In file included from ./test/fuzz/util.h:26:
In file included from ./test/util/setup_common.h:16:
In file included from ./txmempool.h:31:
In file included from /builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/optional.hpp:15:
In file included from /builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/optional/optional.hpp:47:
In file included from /builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/type_traits/is_nothrow_move_assignable.hpp:16:
/builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/type_traits/has_nothrow_assign.hpp:65:7: error: builtin __has_nothrow_assign is deprecated; use __is_nothrow_assignable instead [-Werror,-Wdeprecated-builtins]
BOOST_HAS_NOTHROW_ASSIGN(T)
^
/builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/type_traits/intrinsics.hpp:205:43: note: expanded from macro 'BOOST_HAS_NOTHROW_ASSIGN'
^
/builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/serialization/serialization.hpp:61:1: note: in instantiation of template class 'boost::has_nothrow_assign<unsigned int>' requested here
BOOST_STRONG_TYPEDEF(unsigned int, version_type)
^
/builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/serialization/strong_typedef.hpp:42:57: note: expanded from macro 'BOOST_STRONG_TYPEDEF'
D& operator=(const D& rhs) BOOST_NOEXCEPT_IF(boost::has_nothrow_assign<T>::value) {t = rhs.t; return *this;} \
```
clang warnings are quite noisy and easily cause gitlab's logs
to spill over, preventing logging of dash-specific warnings and errors
from making it to the CI logs.
our dependencies mostly track upstream so regardless, we cannot
act upon those warnings, so it's better to just suppress them if they're
too noisy.
faceeae49ae85fb644996fedb970ad9d0e297ca5 ci: Disable valgrind functionl tests on forked repos to avoid timeouts (MarcoFalke)
Pull request description:
Allows people to fork our repo and run the tests again
Also print more cache stats
ACKs for top commit:
hebasto:
ACK faceeae49ae85fb644996fedb970ad9d0e297ca5, tested on my own repo: https://travis-ci.org/github/hebasto/bitcoin/jobs/676257500
Tree-SHA512: 50e44edf94fcb997438eeaf7308b2b58a0854141ecb1fbb0ba7bf5ed662f19b60899f966f579cca90ad5e789234d0e90122d8c2c854da70339058adc3a475fa6
6136a96cdf631f824fb89cf2720422787439acc9 ci: Rename RUN_CI_ON_HOST to DANGER_RUN_CI_ON_HOST (Hennadii Stepanov)
97ba77aa8e58c3c7ff767e6ca804d0e46370e5dc ci: Add native s390x (Hennadii Stepanov)
Pull request description:
Unlike the Docker wrapped solution (#17591) this PR suggests running on host system directly.
This approach makes builds quick and stable (see: #18106).
The excerpt from the Travis log:
```
...
Running on host system without docker wrapper
...
Byte Order: Big Endian
...
```
ACKs for top commit:
MarcoFalke:
ACK 6136a96cdf631f824fb89cf2720422787439acc9
Tree-SHA512: 1b591de13e38d10a35217e1de11cbd648a359d18d16eed166fac18ea5788b58cc9fc6d407086ed342b99e57e479efd951a0ea693710177e500eb116316b9a788
fa7af33b4cb12315de86315c163393bdb50fad75 ci: Run unit tests sequential once (MarcoFalke)
fa68a3e7640fc00c47f03fa565357679bdafb4b2 appveyor: Enable minimal unit test logging to aid debugging (MarcoFalke)
Pull request description:
Fixes#16976
Top commit has no ACKs.
Tree-SHA512: 1f1ee8776a67afa8c1c5a16ef170c9975b6a486c087c8eba12e97d23382befd1c2801622ec70ca8e4cd1fbedce1dec46be67677ceaf07f35f1d3f3bead0200f0
faa9491870bf29583a989c407694d01f49b65bec ci: Use Focal for fuzzers (MarcoFalke)
Pull request description:
This gives us access to clang-10, as well as a newer version of valgrind
ACKs for top commit:
fanquake:
ACK faa9491870bf29583a989c407694d01f49b65bec - [Clang 10](https://packages.ubuntu.com/focal/clang) and [valgrind 3.15](https://packages.ubuntu.com/focal/valgrind).
practicalswift:
ACK faa9491870bf29583a989c407694d01f49b65bec -- diff looks correct & contemporary clang is better than vintage clang
Tree-SHA512: 0e67232673434c0309db79c1054e3e981115083585945967e346f4d58792635832100f89911428aab928155e44e5f401207a023681ae008fdb5280cf02c4d427
25c8b73656c90f7b78a90f25c5bcc308f8b7c598 ci: Use Homebrew addon on native macOS (Hennadii Stepanov)
596c627a1eb2b32e2755ae5b9bf32235cf8ce72b ci: Fix brew in Travis (Hennadii Stepanov)
Pull request description:
Recently almost every macOS image update on Travis breaks our builds:
- #17848
- #18436
This PR:
- fixes the error caused by the recent [update](https://changelog.travis-ci.com/xcode-11-3-1-xcode-11-2-1-xcode-11-1-and-xcode11-images-updated-142286) from 10.14.4 (18E226) to 10.14.6 (18G3020) on March 25
- leverages [Homebrew addon](https://config.travis-ci.com/ref/job/addons/homebrew) to install packages
Homebrew is not told to install `automake` and `pkg-config` packages, as the [docs](https://docs.travis-ci.com/user/reference/osx/#compilers-and-build-toolchain) states that they are pre-installed:
> - automake 1.16.1
> - pkg-config 0.29.2
Top commit has no ACKs.
Tree-SHA512: 1a70c06468fbe162503081b03dcf54614d67abf8ff0ce07d118b5bb50bbb92c182a76f769bea586c691aa82b9281a29cdef88091acc16895817a2e7cddafec6e
fae1e992898aa6b36c402cec4085fbf6da9b33ad ci: Only clone bitcoin-core/qa-assets when fuzzing (MarcoFalke)
Pull request description:
Currently the only content of that repo are some seeds, so we can speed up some ci builds
ACKs for top commit:
laanwj:
ACK fae1e992898aa6b36c402cec4085fbf6da9b33ad (provided this passes travis)
Tree-SHA512: ed813738e7f24bb56a2f12aa3b398e414eb4f0ba98379836a33ff3e5602cbf42a28e89aad10e346468191ecddc03e60d5b236097112e27c07cb1c2293533ea58
fa92af5af39a08982f785542df5419d6d5a4706d ci: Run feature_block and feature_abortnode in valgrind (MarcoFalke)
fa01febeaf801bade77a613e64f18b556ae16d86 test: Remove ci timeout restriction in test_runner (MarcoFalke)
Pull request description:
Also revert commit 0a4912e46a, because some tests take too long for this to be useful anymore.
Top commit has no ACKs.
Tree-SHA512: 363f14766e1f4a5860ab668a516b41acebc6fbdf11d8defb3a95a772dbf82304ca1f5f14b1dbad97f2029503e03d92e8c69df0466a8872409c20665838f617ed
