Commit Graph

202 Commits

Author SHA1 Message Date
Konstantin Akimov
8085729789
chore: dashification: drop DOCKER_NAME_TAG
We have only one docker image for all functional test and this option is confusing
2024-01-22 19:47:11 -06:00
MarcoFalke
f8e425b437
Merge #19267: ci: Upgrade most ci configs to focal
fa05f44893d228f672f39436d0cb6b3376f81ac2 ci: Upgrade most ci configs to focal (MarcoFalke)
fad67208914e5a74b64f4cc018368902ef3a2e9b doc: move doc to ci readme (MarcoFalke)
fa880773b425fcd292ed7669d237ee3151a15bc6 ci: Have one config run in xenial to test against python3.5 (MarcoFalke)
fa6ddb2fa167df52f59cb9eaabed48315ddcdb2e travis: Always run multiprocess build (MarcoFalke)

Pull request description:

  Generally developers compile with recent compilers, so bumping the ci configs to a recent OS should be uncontroversial. Older OSes (especially with compiler sanitizers) need workarounds that can be dropped by running on a more recent OS.

  This pull changes the asan sanitizer and the experimental multiprocess build to use focal.
  Also, it runs the no_wallet config on xenial to test against python 3.5, according to `doc/dependencies.md`.

  Finally, all configs that mimic gitian (win and mac) will stay at bionic.

ACKs for top commit:
  Sjors:
    ACK fa05f44893d228f672f39436d0cb6b3376f81ac2, assuming Travis passes
  hebasto:
    ACK fa05f44893d228f672f39436d0cb6b3376f81ac2

Tree-SHA512: 55ec56c71ba2280d27c1a8856a1e6c310b1fbf469d5a8a1dde228063e3892e1dd1e51408ecff7a3d77ac2ae018daa9e9bbbb60598cdeaab8c32a146b11b3e7c4
2024-01-22 19:47:11 -06:00
MarcoFalke
261d2a0ded
Merge #19730: ci: Set increased --timeout-factor by default
fa330ec2fe5f5ba68a8d43fff0b19584c0b1ff39 test: Remove confusing and broken use of wait_until global (MarcoFalke)
fa6583c30bf7d82cf7ffdae995f8f16524ad2c0d ci: Set increased --timeout-factor by default (MarcoFalke)

Pull request description:

  Assuming that tests don't have a logic error or race, setting a high timeout should not cause any issues. The tests will still pass just as fast in the fastest case, but it allows for some buffer in case of slow disks or otherwise starved ci machines.

  Fixes #19729

ACKs for top commit:
  hebasto:
    ACK fa330ec2fe5f5ba68a8d43fff0b19584c0b1ff39, I have reviewed the code, and it looks OK, I agree it can be merged.

Tree-SHA512: 3da80ee008c7b08bab5fdaf7804d57c79d6fed49a7d37b9c54fc89756659fcb9981fd10afc4d07bd90d93c1699fd410a0110a3cd34d016873759d114ce3cd82d
2024-01-22 19:44:36 -06:00
MarcoFalke
8342f75fc6
Merge #19276: ci: Move travis workarounds to .travis.yml
fa7166759789c1282609ff3ab2e80d4f70910a9f ci: Move travis workarounds to .travis.yml (MarcoFalke)

Pull request description:

  It seems odd to have travis related workarounds in the general ci config files. Fix that oddity by moving the travis related workarounds to the travis yaml file.

  For unexplained reasons, this should also work around and thus close #19171

ACKs for top commit:
  hebasto:
    ACK fa7166759789c1282609ff3ab2e80d4f70910a9f, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: b4419d38e2b41f6e4d6e6b7658f1d972c40c390a49fe78808f8640d28efd84cc6668ce292d45b7c539e65b9e2ecbad10e796cb8f9329a0f1e7d0132ce962d226
2024-01-16 15:05:07 -06:00
fanquake
098d0fd430
Merge #18677: Multiprocess build support
e2bab2aa162ae38b2bf8195b577c982402fbee9d multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e708c04ef6c9880f89d0a4cbaa6fc7c5 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52bfcd4edf8553aaf332bfeb92fc554cc build: multiprocess autotools changes (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.

  In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.

  The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649

ACKs for top commit:
  practicalswift:
    ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d
  Sjors:
    tACK e2bab2aa162ae38b2bf8195b577c982402fbee9d on macOS 10.15.4
  hebasto:
    ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d, tested on Linux Mint 19.3 (x86_64):

Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
2024-01-16 09:34:26 -06:00
Wladimir J. van der Laan
0e8a4d54a4
Merge #21064: refactor: use std::shared_mutex & remove Boost Thread
060a2a64d40d75fecb60b7d2b9946a67e46aa6fc ci: remove boost thread installation (fanquake)
06e1d7d81d5a56d136c6fc88f09a2b0654a164f9 build: don't build or use Boost Thread (fanquake)
7097add83c8596f81be9edd66971ffd2486357eb refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef834490c438436719f95cbaf888c4914 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)

Pull request description:

  This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).

  Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.

  Two other points:

  [Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
  > It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.

  Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
  * The version of Boost.
  * The platform you're building for.
  * Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
  * Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.

  A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.

  With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.

  Previous similar PRs were #19183 & #20922. The authors are included in the commits here.
  Also related to #21022 - pthread sanity checking.

ACKs for top commit:
  laanwj:
    Code review ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc
  vasild:
    ACK 060a2a64d40d75fecb60b7d2b9946a67e46aa6fc

Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
2024-01-16 09:29:52 -06:00
fanquake
271acac3a7
Merge #18210: test: type hints in Python tests
bd7e530f010d43816bb05d6f1590d1cd36cdaa2c This PR adds initial support for type hints checking in python scripts. (Kiminuo)

Pull request description:

  This PR adds initial support for type hints checking in python scripts.

  Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention."

  [Mypy](https://mypy.readthedocs.io/en/latest/index.html) is used in `lint-python.sh` to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error.

  **Notes:**

  * [--ignore-missing-imports](https://mypy.readthedocs.io/en/latest/command_line.html#cmdoption-mypy-ignore-missing-imports) switch is passed on to `mypy` checker for now. The effect of this is that one does not need `# type: ignore` for `import zmq`. More information about import processing can be found [here](https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports). This can be changed in a follow-up PR, if it is deemed useful.
  * We are stuck with Python 3.5 until 04/2021 (see https://packages.ubuntu.com/xenial/python3). When Python version is bumped to 3.6+, one can change:
      ```python
      _opcode_instances = []  # type: List[CScriptOp]
      ```
      to
      ```python
      _opcode_instances:List[CScriptOp] = []
      ```
      for type hints that are **not** function parameters and function return types.

  **Useful resources:**

  * https://docs.python.org/3.5/library/typing.html
  * https://www.python.org/dev/peps/pep-0484/

ACKs for top commit:
  fanquake:
    ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c - the type checking is not the most robust (there are things it fails to detect), but I think this is worth adopting (in a limited capacity while we maintain 3.5 compat).
  MarcoFalke:
    ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c fine with me

Tree-SHA512: 21ef213915fb1dec6012f59ef17484e6c9e0abf542a316b63d5f21a7778ad5ebabf8961ef5fc8e5414726c2ee9c6ae07c7353fb4dd337f8fcef5791199c8987a
2024-01-10 12:07:54 -06:00
Wladimir J. van der Laan
13188e1d92
Merge #19331: build: Do not include server symbols in wallet
faca73000fa8975c28f6be8be01957c1ae94ea62 ci: Install fixed version of clang-format for linters (MarcoFalke)
fa4695da4c69646b58a8fa0b6b30146bb234deb8 build: Sort Makefile.am after renaming file (MarcoFalke)
cccc2784a3bb10fa8e43be7e68207cafb12bd915 scripted-diff: Move ui_interface to the node lib (MarcoFalke)
fa72ca6a9d90d66012765b0043fd819698b94ba8 qt: Remove unused includes (MarcoFalke)
fac96e6450d595fe67168cb7afa7692da6cc9973 wallet: Do not include server symbols (MarcoFalke)
fa0f6c58c1c6d10f04c4e65a424cc51ebca50a8c Revert "Fix link error with --enable-debug" (MarcoFalke)

Pull request description:

  This reverts a hacky workaround from commit b83cc0f, which only happens to work due to compiler optimizations. Then, it actually fixes the linker error.

  The underlying problem is that the wallet includes symbols from the server (ui_interface), which usually results in linker failures. Though, in this specific case the linker failures have not been observed (unless `-O0`) because our compilers were smart enough to strip unused symbols.

  Fix the underlying problem by creating a new header-only with the needed symbol and move ui_interface to node to clarify that this is part of libbitcoin_server.

ACKs for top commit:
  Sjors:
    ACK faca730
  laanwj:
    ACK faca73000fa8975c28f6be8be01957c1ae94ea62
  hebasto:
    re-ACK faca73000fa8975c28f6be8be01957c1ae94ea62, since the [previous](https://github.com/bitcoin/bitcoin/pull/19331#pullrequestreview-434420539) review:

Tree-SHA512: e9731f249425aaea50b6db5fc7622e10078cf006721bb87989cac190a2ff224412f6f8a7dd83efd018835302337611f5839e29e15bef366047ed591cef58dfb4
2024-01-10 12:06:05 -06:00
MarcoFalke
5a8df5e996
partial Merge #18735: ci: Add and document BASE_BUILD_DIR
fae49f6e424f31e93c5620d5ff893fb517ef4a8b ci: Add and document BASE_BUILD_DIR (MarcoFalke)

Pull request description:

  Also fixes #18768

ACKs for top commit:
  hebasto:
    re-ACK fae49f6e424f31e93c5620d5ff893fb517ef4a8b, which is essentially the same as the previously [reviewed changes](https://github.com/bitcoin/bitcoin/pull/18735#pullrequestreview-400581536).

Tree-SHA512: 216565a05ccd513dd9f114b2333d3c283fd71914d32f9b05f145cb7c70633b083ff8ef60798d6f22f4be6a4d652b03806551fd74b5b596c92968501a4d9726d2
2024-01-09 08:13:04 -06:00
Konstantin Akimov
4a753efa22
partial Merge #20182: ci: Build with --enable-werror by default, and document exceptions
It does not includes changes that actually enable werror by default.
But this backport is necessary to prevent compilation error on mac such as this one:

    In file included from mapport.cpp:26:
    In file included from /builds/dashpay/dash/depends/x86_64-apple-darwin/include/miniupnpc/miniupnpc.h:14:
    /builds/dashpay/dash/depends/x86_64-apple-darwin/include/miniupnpc/upnpdev.h:27:14: error: zero size arrays are an extension [-Werror,-Wzero-length-array]
            char buffer[0];
2024-01-06 19:30:15 -06:00
MarcoFalke
3d980d4a64
Merge #19427: ci: Run all tests on native mac again
fa23fbb42fa710e6be405596e2abfc3a289d020d ci: Run all tests on native mac again (MarcoFalke)

Pull request description:

  They should pass again after f6072e601af68f3eee307478ad22ff3960680656

ACKs for top commit:
  practicalswift:
    ACK fa23fbb42fa710e6be405596e2abfc3a289d020d -- Travis is happy and so am I

Tree-SHA512: 49c16b6056d4e67d12a202744e1c56fee2788830213fe4a195955ad44c6b8ecce768a591463ffa0048821959a75b6fad4178629a8866c4a26799c4c8c13e933d
2024-01-06 19:30:15 -06:00
MarcoFalke
2d95e9284c
Merge #19014: test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option
fad798be76dd5e330463c837fda768477d536078 test: Default --previous-releases to false if dir is empty (MarcoFalke)
faf1c3cc58d14f86ba5364e6ee5c8ef29cac2e26 test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option (MarcoFalke)

Pull request description:

  The "auto-detection" feature is kept in place, but making it an option allows to properly document it. For example, on my machine I get:

  ```
  $ ./test/functional/wallet_disable.py --help | grep previous-releases
    --previous-releases   Force test of previous releases (default: False)

ACKs for top commit:
  Sjors:
    re-tACK fad798b

Tree-SHA512: a7377d0d5378be0a50be278d76396cc403583617b5fc43467773eee706df698acf3f4e67651491183b9b43a8e1816b052e4c17b90272b7ec4b6ac134ad811400
2024-01-06 19:30:14 -06:00
Konstantin Akimov
eb2792a882
chore: TODO to replace ubsan to asan that does same plus extra 2024-01-06 19:30:14 -06:00
fanquake
4a51a75498
Merge #19008: ci: tsan on clang-9
faf552117efdd456fb17b1d9490896de3b7d67a4 ci: Set DEBIAN_FRONTEND=noninteractive (MarcoFalke)
fa006caa136f05a18428c792802523637d2234ca ci: tsan on clang-9 (MarcoFalke)

Pull request description:

  Bump the compiler runtime library that includes the sanitizers from clang-8 to clang-9 to get a more recent version. Also, bump the system packages from xenial to bionic to test packages closer to what is commonly used in production.

  The second commit is needed to install the `tzdata` package, which is missing on some operating systems. See https://travis-ci.org/github/MarcoFalke/bitcoin-core/jobs/688455828#L1727

ACKs for top commit:
  hebasto:
    ACK faf552117efdd456fb17b1d9490896de3b7d67a4
  practicalswift:
    ACK faf552117efdd456fb17b1d9490896de3b7d67a4 -- patch looks correct and Travis is happy

Tree-SHA512: aa38fdae5f716966a83a21d5f7c121675cf7d663148ab3baa065142c8b3850bcd4bf88526d7da0fa51f5e08f2c317b537f950fcc9eb1e69fdacb0eac8863e1c6
2024-01-06 19:30:14 -06:00
MarcoFalke
a5abf4b6b3
Merge #18929: ci: Pass down LD_LIBRARY_PATH and MAKEJOBS to fuzz test_runner
cbd661122e5852d543467090459d33cf8cb4a3c7 Set LD_LIBRARY_PATH consistently in travis tests (Russell Yanofsky)
fa35c34df781cf46bbd15522961f214f03b958bf Remove unused ci configs that have been moved elsewhere (MarcoFalke)
3333cb96994182bbdbb21174b691feb716858bc2 fuzz: Pass down MAKEJOBS to test_runner (MarcoFalke)

Pull request description:

  Just how `MAKEJOBS` is passed down to the functional test `test_runner`, do the same for the fuzz `test_runner`.

  Also includes a commit to remove unused config files, which have been moved elsewhere.

Top commit has no ACKs.

Tree-SHA512: 32557102c9e40599b432aeb004c8427e8fbb07cdf4048050cdc8241d1b029aaad306b1131007eeca8315a4f71c38a7efbb833310e056cd11b835676cd19b8902
2024-01-06 19:30:14 -06:00
MarcoFalke
295c99c4cf
Merge #18799: ci: Document why tests can not be run on mac
fa72a751026d43e1d01ae30e26bcfd9b4cc0cf45 ci: Document why tests can not be run on mac (MarcoFalke)

Pull request description:

  Fixes #18794

Top commit has no ACKs.

Tree-SHA512: 297652eda412aa8cf7255e20a6f294d22773dad8637a3d7b5204f3b638e911ce5b2e40e85f81395a34c1b5a5b497665944c2d6ea17c70c30c0c9e0ab553f956e
2024-01-06 19:30:13 -06:00
MarcoFalke
e02a0d8e54
Merge #18778: ci: Run functional tests on mac again
fac24dea00c54fe1e2dcc7625bb65ada8b97ac0c ci: Run functional tests on mac again (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK fac24dea00c54fe1e2dcc7625bb65ada8b97ac0c, verified travis build log.

Tree-SHA512: 406282a7ac03e5c193830b727366c7b1350639f1850aff951bf7ddd4b0c3e3ffb396b950ccb3a64ddc59500fa2739766f3c34806b4d144bc4535bb2bd765b959
2024-01-06 19:30:13 -06:00
MarcoFalke
0aa6f21149
Merge #18905: travis: Remove s390x
8c705ff1291ef7876ab1a939e2c7312aacc3dc37 travis: Remove s390x (MarcoFalke)

Pull request description:

  Fixes #18868
  Fixes #18016

Top commit has no ACKs.

Tree-SHA512: 1007b761c7e01dd2b75aa34e92e01a92a84a100c0a3a53863c755d93b2438a74601e3f2083919b8a9cfc92fb104d0c8415fad3f6c9504c76b02ad2e9712660c0
2024-01-06 19:30:12 -06:00
Konstantin Akimov
e2fe1a222b
fix: follow-up missing changes from Merge #20413: build: Require C++17 compiler 2024-01-06 19:30:12 -06:00
fanquake
bfe050d1ca Merge #21437: gitian, ci: Drop unneeded python3-dev package for macOS builds
e4c0cada791135e2d0a36638541c03feff0bd6bc ci, gitian: Drop unneeded python3-dev package for macOS builds (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK e4c0cada791135e2d0a36638541c03feff0bd6bc - gitian builds match and I checked that this doesn't end up installed as a side-effect of another package.

Tree-SHA512: 520a3909b106a0e005b195c5395691edf62b76ee2df43b6971b7aa193648d68e6dac69cb4f1dc474f594b015a2fc2074061865e571d89365174beb5c1780356f
2023-12-06 12:40:58 -06:00
Wladimir J. van der Laan
9fd9e4cf5a Merge #20567: test: Add option to git-subtree-check to do full check, add help
34c80d9eee7d21755f2bb80f7c97fd30d2c7b656 test: Add option to git-subtree-check to do full check, add help (Wladimir J. van der Laan)

Pull request description:

  This adds a brief help text to `git-subtree-check.sh` and adds an option to do a full remote check instead of having two different code paths with a successful exit status. Also make it explicit that the CI is not doing this.

ACKs for top commit:
  fjahr:
    tested ACK 34c80d9eee7d21755f2bb80f7c97fd30d2c7b656

Tree-SHA512: 20f672fd3b3c1d633eccf9998fdd738194cdd7d10cc206691f2dcc28bbbf8187b8d06b87814f875a06145b179f5ca1f4f4f9922972be72759cf5ac6e0c11abd1
2023-12-06 11:40:14 -06:00
fanquake
2da9982e55 Merge #17829: scripted-diff: Bump copyright of files changed in 2019
aaaaad6ac95b402fe18d019d67897ced6b316ee0 scripted-diff: Bump copyright of files changed in 2019 (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK aaaaad6ac95b402fe18d019d67897ced6b316ee0
  promag:
    ACK aaaaad6ac95b402fe18d019d67897ced6b316ee0 🎉
  fanquake:
    ACK aaaaad6ac95b402fe18d019d67897ced6b316ee0 - going to merge this now because the year is over and conflicts are minimal.

Tree-SHA512: 58cb1f53bc4c1395b2766f36fabc7e2332e213780a802762fff0afd59468dad0c3265f553714d761c7a2c44ff90f7dc250f04458f4b2eb8eef8b94f8c9891321
2023-12-06 11:40:14 -06:00
Konstantin Akimov
fd2e9857aa fix: follow-up partial bitcoin/bitcoin#25063 - actually load binaries with x86_64-apple-darwin platform 2023-11-24 11:23:46 -06:00
Kittywhiskers Van Gogh
00b3dd6690 ci: don't re-use config.cache when running configure script 2023-11-21 07:59:03 -06:00
Ben Woosley
eb6347916e fix: missing changes from Merge #18253: doc: Correct spelling errors in comments
doc: Correct spelling errors in comments

And ci script output.

Identified via test/lint/lint-spelling
2023-09-19 08:54:12 -05:00
Konstantin Akimov
8a9712e75b fix: dashification of test/get_previous_releases.py 2023-09-19 08:54:12 -05:00
MarcoFalke
a3684885e9 partial Merge bitcoin/bitcoin#20354: test: Add feature_taproot.py --previous_release
fa80e10d94dbf86da84fc761b09fb631155a5b25 test: Add feature_taproot.py --previous_release (MarcoFalke)
85ccffa26686c6c9adbd18bdde37fc1747281bab test: move releases download incantation to README (Sjors Provoost)
29d6b1da2a862bfbb14e7821979c97416c5400e8 test: previous releases: add v0.20.1 (Sjors Provoost)

Pull request description:

  Disabling the new consensus code at runtime is fine, but potentially fragile and incomplete. Fix that by giving the option to run with a version that has been compiled without any taproot code.

ACKs for top commit:
  Sjors:
    tACK fa80e10
  NelsonGaldeman:
    tACK fa80e10d94dbf86da84fc761b09fb631155a5b25

Tree-SHA512: 1a1feef823f08c05268759645a8974e1b2d39a024258f5e6acecbe25097aae3fa9302c27262978b40f1aa8e7b525b60c0047199010f2a5d6017dd6434b4066f0
2023-09-19 08:54:12 -05:00
MarcoFalke
da35562fa1 Merge #19813: util, ci: Hard code previous release tarball checksums
0374e821bd9e9498ce9c03aa8e5435870019978b util: Hard code previous release tarball checksums (Hennadii Stepanov)
bd897ce79f72a44a2e609f95433e251a3fd9eb9c scripted-diff: Move previous_release.py to test/get_previous_releases.py (Hennadii Stepanov)

Pull request description:

  #19205 introduced signature verifying for the downloaded `SHA256SUMS.asc`.
  This approach is brittle and does not work in CI environment for many reasons:
  - https://github.com/bitcoin/bitcoin/issues/19812#issuecomment-680760663
  - https://github.com/bitcoin/bitcoin/pull/19013#discussion_r459590779

  This PR:
  - implements **Sjors**' [idea](https://github.com/bitcoin/bitcoin/pull/19205#pullrequestreview-426080048):
  > Alternatively we might as well hard code the checksum for each `tar.gz` release in the source code, here.

  - is an alternative to 5a2c31e528e6bd60635096f233252f3c717f366d (#19013)

  - fixes #19812

  - updates v0.17.1 to v0.17.2

ACKs for top commit:
  MarcoFalke:
    cr ACK 0374e821bd9e9498ce9c03aa8e5435870019978b
  Sjors:
    tACK 0374e821bd9e9498ce9c03aa8e5435870019978b

Tree-SHA512: cacdcf9f5209eae7da357abb3445585ad2f980920fd5bf75527ce89974d3f531a4cf8b5b35edfc116b23bfdfb45c0437cb14cbc416d76ed2dc5b9e6d33cdad71
2023-09-19 08:54:12 -05:00
MarcoFalke
7c43fa5426 Merge #19205: script: previous_release.sh rewritten in python
9c34aff39309b8adc99d347e07b6ddb5366498e9 Remove previous_release.sh (Brian Liotti)
e1e5960e10a9329d9f55a3967d546ffbdd896030 script: Add previous_release.py (Brian Liotti)

Pull request description:

  Closes #18132

  Added functionality:
  1) checks file hash before untarring when using the binary download option

ACKs for top commit:
  fjahr:
    re-ACK 9c34aff39309b8adc99d347e07b6ddb5366498e9
  Sjors:
    tACK 9c34aff39309b8adc99d347e07b6ddb5366498e9

Tree-SHA512: 323f11828736a372a47f048592de8b027ddcd75b38f312dfc73f7b495d1e078bfeb384d9cdf434b3e70f2c6c0ce2da2df48e9a6460ac0e1967c6829a411c52d5
2023-09-19 08:54:12 -05:00
Konstantin Akimov
7ddcf4a282 Merge #18864: Add v0.16.3 backwards compatibility test, bump v0.19.0.1 to v0.19.1
d135c294764add81683ba47575f9a5dde7d7c07f [ci] make list of previous releases to download a setting (Sjors Provoost)
9c246b873c74834a121edba00fcaecf0cba6f9b4 [test] backwards compatibility: bump v0.19.0.1 to v0.19.1 (Sjors Provoost)
89a28e02fa46f3d5eb07ab02aa34aa95c6fcee11 [test] add v0.16.3 backwards compatibility test (Sjors Provoost)

Pull request description:

  Thanks to #18774's `adjust_bitcoin_conf_for_pre_17` we can now test backwards compatibility for v0.16.3, both for sync and loading a recent wallet.

  This PR bumps v0.19.0.1 to v0.19.1.

  I also made the version list consistent for the `contrib/devtools/previous_release.sh` instruction, between both tests.

ACKs for top commit:
  MarcoFalke:
    ACK d135c294764add81683ba47575f9a5dde7d7c07f

Tree-SHA512: 5ff137a7a934237fa220f1c2807ce9abeeb75929266558bf3e4045bec7dfcd0a8747fa74d700065c568330b18badf58c60c308eb13d1eed444d4bbfe6decc48b
2023-09-19 08:54:12 -05:00
MarcoFalke
207b1c5877 Merge #12134: Build previous releases and run functional tests
c456145b2c65f580683df03bf10cd39000cf24d5 [test] add 0.19 backwards compatibility tests (Sjors Provoost)
b769cd142deda74fe46e231cc7b687a86514f2f1 [test] add v0.17.1 wallet upgrade test (Sjors Provoost)
9d9390dab716f07057c94e8e21f3c7dd06192f35 [tests] add wallet backwards compatility tests (Sjors Provoost)
c7ca6308968b29a0e0edc485cd06e68e5edb7c7d [scripts] support release candidates of earlier releases (Sjors Provoost)
8b1460dbd1b732f06d4cebe1fa6844286c7a0056 [tests] check v0.17.1 and v0.18.1 backwards compatibility (Sjors Provoost)
ae379cf7d12943fc192d58176673bcfe7d53da53 [scripts] build earlier releases (Sjors Provoost)

Pull request description:

  This PR adds binaries for 0.17, 0.18 and 0.19 to Travis and runs a basic block propagation test.

  Includes test for upgrading v0.17.1 wallets and opening master wallets with older versions.

  Usage:

  ```sh
  contrib/devtools/previous_release.sh -f -b v0.19.0.1 v0.18.1 v0.17.1
  test/functional/backwards_compatibility.py
  ```

  Travis caches these earlier releases, so it should be able to run these tests with little performance impact.

  Additional scenarios where it might be useful to run tests against earlier releases:

  * creating a wallet with #11403's segwit implementation, copying it to an older node and making sure the user didn't lose any funds (although this PR doesn't support `v0.15.1`)
  * future consensus changes
  * P2P changes (e.g. to make sure we don't accidentally ban old nodes)

ACKs for top commit:
  MarcoFalke:
    ACK c456145b2c65f580683df03bf10cd39000cf24d5 🔨

Tree-SHA512: 360bd870603f95b14dc0cd629532cc147344f632b808617c18e1b585dfb1f082b401e5d493a48196b719e0aeaee533ae0a773dfc9f217f704aae898576c19232
2023-09-19 08:54:12 -05:00
MarcoFalke
30e75239e8 Merge #18667: ci: Limit cache size regardless of NO_DEPENDS
0c6318788beaf1a31aeba5a21f3f8bb5c07cea6c ci: Limit cache size regardless of NO_DEPENDS (Hennadii Stepanov)

Pull request description:

  Close #18666.

ACKs for top commit:
  MarcoFalke:
    ACK 0c6318788beaf1a31aeba5a21f3f8bb5c07cea6c . Depends has ccache disabled anyway and is cached regardless of whether ccache is there or not, see #17248

Tree-SHA512: b1bf98be0f844b4704abd177841b014f3900be8160496f0d12596310db607b4f544547e8c3cbfcf17c086a78afd251653363f3dd467b769ac0062bc19adc8144
2023-08-29 22:00:59 -05:00
MarcoFalke
2d4db48606 Merge #18581: ci: Print ccache stats, add pip cache, and cleanups
73f83036298e4932772704b2d5359c78d81bd7cc ci: Add pip cache (Hennadii Stepanov)
6d480667b24dc630e7c3fe7ff032cbe1a958ab77 ci: Print ccache statistics summary (Hennadii Stepanov)
a6eea715ff549febe6d4bd52f17acb50abd99120 ci: Drop unused WINEDEBUG variable (Hennadii Stepanov)

Pull request description:

  The Travis [pip cache](https://docs.travis-ci.com/user/caching/#pip-cache) is free and saves a dozen of seconds :)

  Here are some excerpts from the Travis logs with `ccache` statistics (I found useful):

  2) [Arm64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507749)
  ```
  ccache version 3.6
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Fri Apr 10 18:21:06 2020
  stats zeroed                        Fri Apr 10 18:03:04 2020
  cache hit (direct)                   156
  cache hit (preprocessed)             198
  cache miss                           143
  cache hit rate                     71.23 %
  called for link                        8
  cleanups performed                     9
  files in cache                      1255
  cache size                          80.6 MB
  max cache size                     100.0 MB
  ```
  3) [s390x](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507750)
  ```
  ccache version 3.4.1
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats zero time                     Fri Apr 10 17:15:35 2020
  cache hit (direct)                   115
  cache hit (preprocessed)             163
  cache miss                            94
  cache hit rate                     74.73 %
  called for link                        6
  cleanups performed                     0
  files in cache                       804
  cache size                         304.8 MB
  max cache size                       5.0 GB
  ```
  4) [Win64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507751)
  ```
  ccache version 3.4.1
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats zero time                     Fri Apr 10 17:38:07 2020
  cache hit (direct)                   147
  cache hit (preprocessed)             199
  cache miss                           138
  cache hit rate                     71.49 %
  called for link                        7
  cleanups performed                     7
  files in cache                      1242
  cache size                          87.0 MB
  max cache size                     100.0 MB
  ```
  5) [CentOS 7](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507752)
  ```
  ccache version 3.7.7
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Fri Apr 10 17:45:59 2020
  stats zeroed                        Fri Apr 10 17:34:27 2020
  cache hit (direct)                   152
  cache hit (preprocessed)             200
  cache miss                           144
  cache hit rate                     70.97 %
  called for link                        8
  cleanups performed                     4
  files in cache                      1352
  cache size                          86.1 MB
  max cache size                     100.0 MB
  ```
  6) [bionic](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507753)
  ```
  ccache version 3.4.1
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats zero time                     Fri Apr 10 17:21:10 2020
  cache hit (direct)                   136
  cache hit (preprocessed)              35
  cache miss                           330
  cache hit rate                     34.13 %
  called for link                        8
  cleanups performed                    18
  files in cache                      1302
  cache size                          83.8 MB
  max cache size                     100.0 MB
  ```
  7) [xenial](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507754)
  ```
  ccache version 3.2.4
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  cache hit (direct)                   161
  cache hit (preprocessed)             174
  cache miss                            95
  called for link                        7
  files in cache                      3411
  cache size                           1.1 GB
  max cache size                       5.0 GB
  ```
  10) [focal](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507757)
  ```
  ccache version 3.7.7
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Fri Apr 10 17:35:57 2020
  stats zeroed                        Fri Apr 10 17:21:32 2020
  cache hit (direct)                    78
  cache hit (preprocessed)             125
  cache miss                           107
  cache hit rate                     65.48 %
  called for link                      120
  cleanups performed                     0
  files in cache                      6218
  cache size                           1.8 GB
  max cache size                       5.0 GB
  ```
  14) [macOS 10.12](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507761)
  ```
  ccache version 3.4.1
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats zero time                     Fri Apr 10 17:47:19 2020
  cache hit (direct)                    28
  cache hit (preprocessed)               1
  cache miss                           469
  cache hit rate                      5.82 %
  called for link                        8
  cleanups performed                    17
  files in cache                      1946
  cache size                          82.6 MB
  max cache size                     100.0 MB
  ```
  15) [macOS 10.14 native](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507762)
  ```
  ccache version 3.7.8
  cache directory                     /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /usr/local/Cellar/ccache/3.7.8/etc/ccache.conf
  stats updated                       Fri Apr 10 17:38:44 2020
  stats zeroed                        Fri Apr 10 17:22:36 2020
  cache hit (direct)                   213
  cache hit (preprocessed)             293
  cache miss                           144
  cache hit rate                     77.85 %
  called for link                       11
  called for preprocessing              56
  compile failed                        30
  preprocessor error                    56
  bad compiler arguments                14
  autoconf compile/link                 77
  no input file                         66
  cleanups performed                     0
  files in cache                      1564
  cache size                         284.6 MB
  max cache size                       5.0 GB
  ```

Top commit has no ACKs.

Tree-SHA512: f9f2c6d72680bafc7a982efc04dc37ab9bba77ffd2a4cbbc56bb0e86c5c3501d57aa2f0ebeac0113e9d8daac1ee4eac1eddf74c1172c267b37ee2d9324366c8a
2023-08-29 22:00:59 -05:00
Konstantin Akimov
4aa197dbdb Merge #18673: scripted-diff: Sort test includes
fa4632c41714dfaa699bacc6a947d72668a4deef test: Move boost/stdlib includes last (MarcoFalke)
fa488f131fd4f5bab0d01376c5a5013306f1abcd scripted-diff: Bump copyright headers (MarcoFalke)
fac5c373006a9e4bcbb56843bb85f1aca4d87599 scripted-diff: Sort test includes (MarcoFalke)

Pull request description:

  When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.

  This pull preempts both issues by just sorting all includes in one commit.

  Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.

  Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.

ACKs for top commit:
  practicalswift:
    ACK fa4632c41714dfaa699bacc6a947d72668a4deef
  jonatack:
    ACK fa4632c41714dfaa, light review and sanity checks with gcc build and clang fuzz build

Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
2023-08-29 22:00:59 -05:00
Kittywhiskers Van Gogh
3b589cb2e4 merge bitcoin#21226: Fix fuzz binary compilation under windows 2023-08-29 21:55:45 -05:00
Kittywhiskers Van Gogh
dcd7babd80 merge bitcoin#20936: build fuzz tests by default 2023-08-29 21:55:45 -05:00
UdjinM6
8c4fb2ad3b
ci: set CCACHE_SIZE to 400M for Gitlab (#5547)
## Issue being fixed or feature implemented
Make CI builds faster.

We have this
https://github.com/dashpay/dash/blob/develop/ci/dash/build_src.sh#L18
but by that time `CCACHE_SIZE` is set to 100M via
https://github.com/dashpay/dash/blob/develop/ci/test/00_setup_env.sh#L51
already.

## What was done?
set `CCACHE_SIZE` variable in `.gitlab-ci.yml`, drop confusing and
useless lines from `ci/dash/build_src.sh` (`CCACHE_SIZE` and
`CCACHE_COMPRESS` defaults are handled by `ci/test/00_setup_env.sh`)

## How Has This Been Tested?
I set `CCACHE_SIZE` to 400M yesterday via Gitlab UI to test it.

results:
before, trivial doc change, 100M:
https://gitlab.com/dashpay/dash/-/jobs/4931233566#L59 (~17 minutes)
after, trivial code change, 400M:
https://gitlab.com/dashpay/dash/-/jobs/4935764148#L80 (~12 minutes)

Removed it from Gitlab UI now but it should still say `Set cache size
limit to 400.0 MB` in logs for this PR.

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-08-27 16:15:53 -05:00
UdjinM6
63b00440e0
ci: Disable creation of combined.html (#5518)
## Issue being fixed or feature implemented
40% smaller CI job artifacts. Should help with issues like
https://gitlab.com/dashpay/dash/-/jobs/4759700026 in #5493.

## What was done?

## How Has This Been Tested?
https://gitlab.com/UdjinM6/dash/-/jobs/4773517599

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
2023-08-01 12:18:16 -05:00
Kittywhiskers Van Gogh
5cb5a6edb0 merge bitcoin#22930: remove glibc back compat 2023-08-01 12:07:31 -05:00
Kittywhiskers Van Gogh
4168e1ea5a ci: add -Wno-deprecated-builtins to fuzz CXXFLAGS
Used to avoid the following error:

```
In file included from test/fuzz/addition_overflow.cpp:7:
In file included from ./test/fuzz/util.h:26:
In file included from ./test/util/setup_common.h:16:
In file included from ./txmempool.h:31:
In file included from /builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/optional.hpp:15:
In file included from /builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/optional/optional.hpp:47:
In file included from /builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/type_traits/is_nothrow_move_assignable.hpp:16:
/builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/type_traits/has_nothrow_assign.hpp:65:7: error: builtin __has_nothrow_assign is deprecated; use __is_nothrow_assignable instead [-Werror,-Wdeprecated-builtins]
      BOOST_HAS_NOTHROW_ASSIGN(T)
      ^
/builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/type_traits/intrinsics.hpp:205:43: note: expanded from macro 'BOOST_HAS_NOTHROW_ASSIGN'
                                          ^
/builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/serialization/serialization.hpp:61:1: note: in instantiation of template class 'boost::has_nothrow_assign<unsigned int>' requested here
BOOST_STRONG_TYPEDEF(unsigned int, version_type)
^
/builds/dashpay/dash/depends/x86_64-pc-linux-gnu/include/boost/serialization/strong_typedef.hpp:42:57: note: expanded from macro 'BOOST_STRONG_TYPEDEF'
    D& operator=(const D& rhs) BOOST_NOEXCEPT_IF(boost::has_nothrow_assign<T>::value) {t = rhs.t; return *this;} \
```
2023-08-01 12:07:31 -05:00
Kittywhiskers Van Gogh
67a560f03f ci: suppress external warnings on builds that use clang-15
clang warnings are quite noisy and easily cause gitlab's logs
to spill over, preventing logging of dash-specific warnings and errors
from making it to the CI logs.

our dependencies mostly track upstream so regardless, we cannot
act upon those warnings, so it's better to just suppress them if they're
too noisy.
2023-08-01 12:07:31 -05:00
Kittywhiskers Van Gogh
e50914a123 ci: explicitly specify the version of clang we want to test against 2023-08-01 12:07:31 -05:00
Kittywhiskers Van Gogh
66e1541808 merge bitcoin#21851: support cross-compiling for arm64-apple-darwin 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
ab8c26a533 merge bitcoin#23585: Drop Darwin version for better maintainability 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
b1dd417f26 merge bitcoin#23909: use a static .tiff for macOS .dmg over generating 2023-06-18 11:47:54 -05:00
Kittywhiskers Van Gogh
1be3afd82c merge bitcoin#21255: run test-symbol-check for RISC-V 2023-06-07 14:42:02 +05:30
Kittywhiskers Van Gogh
ef9300ad63 merge bitcoin#21664: use LIEF for macOS and Windows symbol & security checks 2023-06-07 14:42:02 +05:30
UdjinM6
101cb67433 fix: move test-security-check call 2023-06-07 14:41:27 +05:30
Kittywhiskers Van Gogh
0024743fa3 fix: disable security tests on win64 CI build 2023-06-07 14:40:37 +05:30
Kittywhiskers Van Gogh
994fb5739c merge bitcoin#18434: add a test-security target and run it in CI 2023-06-07 14:40:37 +05:30