Commit Graph

162 Commits

Author SHA1 Message Date
Kittywhiskers Van Gogh
67a560f03f ci: suppress external warnings on builds that use clang-15
clang warnings are quite noisy and easily cause gitlab's logs
to spill over, preventing logging of dash-specific warnings and errors
from making it to the CI logs.

our dependencies mostly track upstream so regardless, we cannot
act upon those warnings, so it's better to just suppress them if they're
too noisy.
2023-08-01 12:07:31 -05:00
Kittywhiskers Van Gogh
e50914a123 ci: explicitly specify the version of clang we want to test against 2023-08-01 12:07:31 -05:00
Kittywhiskers Van Gogh
66e1541808 merge bitcoin#21851: support cross-compiling for arm64-apple-darwin 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
ab8c26a533 merge bitcoin#23585: Drop Darwin version for better maintainability 2023-06-29 12:31:03 -05:00
Kittywhiskers Van Gogh
b1dd417f26 merge bitcoin#23909: use a static .tiff for macOS .dmg over generating 2023-06-18 11:47:54 -05:00
Kittywhiskers Van Gogh
1be3afd82c merge bitcoin#21255: run test-symbol-check for RISC-V 2023-06-07 14:42:02 +05:30
Kittywhiskers Van Gogh
ef9300ad63 merge bitcoin#21664: use LIEF for macOS and Windows symbol & security checks 2023-06-07 14:42:02 +05:30
UdjinM6
101cb67433 fix: move test-security-check call 2023-06-07 14:41:27 +05:30
Kittywhiskers Van Gogh
0024743fa3 fix: disable security tests on win64 CI build 2023-06-07 14:40:37 +05:30
Kittywhiskers Van Gogh
994fb5739c merge bitcoin#18434: add a test-security target and run it in CI 2023-06-07 14:40:37 +05:30
Konstantin Akimov
a3c59cc77a partial merge bitcoin#14794: Add AddressSanitizer (ASan) Travis build 2023-06-07 01:50:18 -05:00
MarcoFalke
04a707aa47 Merge #18683: ci: Disable valgrind functionl tests on forked repos to avoid timeouts
faceeae49ae85fb644996fedb970ad9d0e297ca5 ci: Disable valgrind functionl tests on forked repos to avoid timeouts (MarcoFalke)

Pull request description:

  Allows people to fork our repo and run the tests again

  Also print more cache stats

ACKs for top commit:
  hebasto:
    ACK faceeae49ae85fb644996fedb970ad9d0e297ca5, tested on my own repo: https://travis-ci.org/github/hebasto/bitcoin/jobs/676257500

Tree-SHA512: 50e44edf94fcb997438eeaf7308b2b58a0854141ecb1fbb0ba7bf5ed662f19b60899f966f579cca90ad5e789234d0e90122d8c2c854da70339058adc3a475fa6
2023-06-07 01:50:18 -05:00
MarcoFalke
01fc1af361 Merge #18569: ci: Add big endian native s390x build
6136a96cdf631f824fb89cf2720422787439acc9 ci: Rename RUN_CI_ON_HOST to DANGER_RUN_CI_ON_HOST (Hennadii Stepanov)
97ba77aa8e58c3c7ff767e6ca804d0e46370e5dc ci: Add native s390x (Hennadii Stepanov)

Pull request description:

  Unlike the Docker wrapped solution (#17591) this PR suggests running on host system directly.

  This approach makes builds quick and stable (see: #18106).

  The excerpt from the Travis log:
  ```
  ...
  Running on host system without docker wrapper
  ...
  Byte Order: Big Endian
  ...
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 6136a96cdf631f824fb89cf2720422787439acc9

Tree-SHA512: 1b591de13e38d10a35217e1de11cbd648a359d18d16eed166fac18ea5788b58cc9fc6d407086ed342b99e57e479efd951a0ea693710177e500eb116316b9a788
2023-06-07 01:50:18 -05:00
MarcoFalke
85be06af39 Merge #18562: ci: Run unit tests sequential once
fa7af33b4cb12315de86315c163393bdb50fad75 ci: Run unit tests sequential once (MarcoFalke)
fa68a3e7640fc00c47f03fa565357679bdafb4b2 appveyor: Enable minimal unit test logging to aid debugging (MarcoFalke)

Pull request description:

  Fixes #16976

Top commit has no ACKs.

Tree-SHA512: 1f1ee8776a67afa8c1c5a16ef170c9975b6a486c087c8eba12e97d23382befd1c2801622ec70ca8e4cd1fbedce1dec46be67677ceaf07f35f1d3f3bead0200f0
2023-06-07 01:50:18 -05:00
fanquake
3ddc17a3b2 Merge #18477: ci: Use Focal for fuzzers
faa9491870bf29583a989c407694d01f49b65bec ci: Use Focal for fuzzers (MarcoFalke)

Pull request description:

  This gives us access to clang-10, as well as a newer version of valgrind

ACKs for top commit:
  fanquake:
    ACK faa9491870bf29583a989c407694d01f49b65bec - [Clang 10](https://packages.ubuntu.com/focal/clang) and [valgrind 3.15](https://packages.ubuntu.com/focal/valgrind).
  practicalswift:
    ACK faa9491870bf29583a989c407694d01f49b65bec -- diff looks correct & contemporary clang is better than vintage clang

Tree-SHA512: 0e67232673434c0309db79c1054e3e981115083585945967e346f4d58792635832100f89911428aab928155e44e5f401207a023681ae008fdb5280cf02c4d427
2023-06-07 01:50:18 -05:00
MarcoFalke
548d63e9e6 Merge #18438: ci: Use Homebrew addon on native macOS
25c8b73656c90f7b78a90f25c5bcc308f8b7c598 ci: Use Homebrew addon on native macOS (Hennadii Stepanov)
596c627a1eb2b32e2755ae5b9bf32235cf8ce72b ci: Fix brew in Travis (Hennadii Stepanov)

Pull request description:

  Recently almost every macOS image update on Travis breaks our builds:
  - #17848
  - #18436

  This PR:
  - fixes the error caused by the recent [update](https://changelog.travis-ci.com/xcode-11-3-1-xcode-11-2-1-xcode-11-1-and-xcode11-images-updated-142286) from 10.14.4 (18E226) to 10.14.6 (18G3020) on March 25
  - leverages [Homebrew addon](https://config.travis-ci.com/ref/job/addons/homebrew) to install packages

  Homebrew is not told to install `automake` and `pkg-config` packages, as the [docs](https://docs.travis-ci.com/user/reference/osx/#compilers-and-build-toolchain) states that they are pre-installed:
  >    - automake 1.16.1
  >    - pkg-config 0.29.2

Top commit has no ACKs.

Tree-SHA512: 1a70c06468fbe162503081b03dcf54614d67abf8ff0ce07d118b5bb50bbb92c182a76f769bea586c691aa82b9281a29cdef88091acc16895817a2e7cddafec6e
2023-06-07 01:50:18 -05:00
MarcoFalke
cd16cf96a4 Merge #18430: ci: Only clone bitcoin-core/qa-assets when fuzzing
fae1e992898aa6b36c402cec4085fbf6da9b33ad ci: Only clone bitcoin-core/qa-assets when fuzzing (MarcoFalke)

Pull request description:

  Currently the only content of that repo are some seeds, so we can speed up some ci builds

ACKs for top commit:
  laanwj:
    ACK fae1e992898aa6b36c402cec4085fbf6da9b33ad (provided this passes travis)

Tree-SHA512: ed813738e7f24bb56a2f12aa3b398e414eb4f0ba98379836a33ff3e5602cbf42a28e89aad10e346468191ecddc03e60d5b236097112e27c07cb1c2293533ea58
2023-06-07 01:50:18 -05:00
MarcoFalke
32b4f8dd65 Merge #18392: ci: Run feature_block in valgrind
fa92af5af39a08982f785542df5419d6d5a4706d ci: Run feature_block and feature_abortnode in valgrind (MarcoFalke)
fa01febeaf801bade77a613e64f18b556ae16d86 test: Remove ci timeout restriction in test_runner (MarcoFalke)

Pull request description:

  Also revert commit 0a4912e46a, because some tests take too long for this to be useful anymore.

Top commit has no ACKs.

Tree-SHA512: 363f14766e1f4a5860ab668a516b41acebc6fbdf11d8defb3a95a772dbf82304ca1f5f14b1dbad97f2029503e03d92e8c69df0466a8872409c20665838f617ed
2023-06-07 01:50:18 -05:00
MarcoFalke
c2554ab891 Merge #18304: ci: Enable all functional tests in valgrind
4444edc2e6671d3f73de3725447130f73ecf0375 ci: Enable all functional tests in valgrind (MarcoFalke)

Pull request description:

  The travis timeout for our repo has been bumped to 2h, so we can run all tests in valgrind now

ACKs for top commit:
  practicalswift:
    ACK 4444edc2e6671d3f73de3725447130f73ecf0375 -- regarding the three disabled cases (`feature_abortnode`, `feature_block` and `rpc_bind`): not a big deal since MSan will take care of those once #18288 is merged. More is more :)

Tree-SHA512: ea2f798112911b6d1f3d88cfcdf0a7cdb698687248343703d6fe55da144542c961c15d888bffb41672c10aa76765615cb7c7ff93d468bfad3c51f962f24e7abb
2023-06-07 01:50:18 -05:00
MarcoFalke
86094b631e Merge #17725: ci: Add valgrind run
facb416ad5ba47b01ee52c273a5c5e3709f06f07 ci: Add valgrind run (MarcoFalke)

Pull request description:

  Fixes #17460

ACKs for top commit:
  practicalswift:
    ACK facb416ad5ba47b01ee52c273a5c5e3709f06f07

Tree-SHA512: 55396e548a76f976d7b7170b68bc5f93cfd44656162267172f66db7eb549699a2a22d3b1bb0d5f180fe0697931939e652c8cdb86b435e81e7ce572485798009d
2023-06-07 01:50:18 -05:00
Kittywhiskers Van Gogh
0b6ad25239 ci: add substitute linux64 ubsan runner 2023-05-31 11:06:40 -05:00
Kittywhiskers Van Gogh
ca564f40f2 ci: drop i686 from GitLab CI runs 2023-05-31 11:06:40 -05:00
Kittywhiskers Van Gogh
a2c820c9f0 ci: skip symbol checks on runtime sanitizer-enabled builds
TSan builds export all sorts of symbols that aren't part of the
allowlist and report sanitizer failures when enabled with glibc
compatibility enabled.

The result of `check-symbols` is valuable but Bitcoin does not
run them during their CI runs and for certain developer-oriented
builds, it's proving to be more of an obstruction. It will still
be enabled for all other builds and will prove to be important
in backporting Guix pull requests, a finger on the pulse, but that
doesn't apply to dev-oriented builds.
2023-05-31 11:06:40 -05:00
Kittywhiskers Van Gogh
f5fba25053 ci: enforce glibc backwards compatibility on all builds without runtime sanitizers 2023-05-31 11:06:40 -05:00
fanquake
c9f072febc Merge bitcoin/bitcoin#23803: ci: Disable s390x gui tests for now
fa6a548f5485ad5be0106e4727812559aefc5a20 ci: Disable s390x gui tests for now (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK fa6a548f5485ad5be0106e4727812559aefc5a20, tested locally with

Tree-SHA512: 077bcda0b4612dfbf538eb2999ce9f32ec3739c2ed5709f7c4c5d1a39738fbe6e549bd54b237152fd808736c15af1024e67c95dc87b4c3102738543522cb14d7
2023-05-31 10:52:02 -05:00
Kittywhiskers Van Gogh
4963660b15
ci: build TSan with clang 15 and add -Werror=thread-safety, fix-up stacktraces (#5375)
## Description

Pull request was inspired by the need to debug lock problems when
working on https://github.com/dashpay/dash/pull/5352.

As far as I'm aware, only macOS has `-Werror=thread-safety` as part of
its default `CXXFLAGS` despite the capability being present on Linux as
well. This PR introduces thread safety checks for that into our thread
sanitizer build.

Additionally, since we're using Clang, something that on first glimpse,
appears to be something that `stacktraces.cpp` isn't happy with, due to
`-Wl,-wrap` being available only on GCC, that no longer seems to be the
case, since the version of Clang with comes with `focal`, its `lld`
_does_ have support for `-wrap` (see [man page for `lld` on
`focal`](https://manpages.ubuntu.com/manpages/focal/en/man1/lld.1.html)).

The current `stable` version of Clang/LLVM is 15, at the time of this
pull request (see https://apt.llvm.org/) but `focal` ships with an older
version, requiring us to use the official LLVM APT repository. I feel we
should be testing with recent compilers alongside the ones shipped by
LTS distributions.

Certain bugs are only made apparent when testing on rolling release
distros or distros that have faster update cycles, like Fedora (see
https://github.com/dashpay/dash/pull/5295 for an illustration of that),
which ship with more recent compilers. Until we overhaul our CI systems
to test using those distros directly (our current infrastructure is
centered around using a "development image" with an LTS distro as the
base), this is the best we can do.

A similar pull request testing against the latest GCC stable will be
welcome as that is currently outside the scope of this PR as the changes
made were to make sure that builds were operating as expected on
Clang/LLVM 15.

---------

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2023-05-26 13:49:29 -05:00
Wladimir J. van der Laan
e27d9a7ecb Merge #18081: test: set a name for CI Docker containers
9e111db088e4137865ae068d206c769994ea0a29 test: set a name for CI Docker containers (fanquake)

Pull request description:

  Addresses one part of #16664, by making it easier to identify CI containers that are running locally. By default Docker will generate random names, like `peaceful_rubin`, with this change, we explicitly set names for all containers.

ACKs for top commit:
  MarcoFalke:
    ACK 9e111db088e4137865ae068d206c769994ea0a29

Tree-SHA512: 0a29ada0d8cf6b0e9ae7a35f4f6df7a3dcc448523ceaed01371124360d6e3d1bf351172104a5fb629488eeaa57994ba04134dcb83c261eb1dfd2f0d73edf5f60
2023-05-24 12:43:57 -05:00
Wladimir J. van der Laan
c31cdcd3d7 Merge #17767: ci: Fix qemu issues
fa569e1a9c5ad1bf8bdf866235b21aff56112224 ci: Set LC_ALL=C when running in qemu-s390x (MarcoFalke)
fa3d77623ecb25dde6b5f4be11626ed16966eb29 ci: Use debian to avoid apt install 404 errors (MarcoFalke)
fabb946090be2f604da3d7d4b1bbe93b79baf23e ci: Install needed gcc and qemu-user iff cross-compiling (MarcoFalke)
faba4672b64fb5ba89e5cb6299479887494b571a ci: Fix QEMU_USER_CMD parse issues (MarcoFalke)
fa5d709fb266c97d4db15006bf855e887a6c123b ci: Move wrap-qemu into separate script (MarcoFalke)

Pull request description:

  Now that the ci system no longer propagates files from the docker container back to the host, the wrap-qemu script is broken.

  To fix it, every statement in the script needs to be executed in the docker (with `DOCKER_EXEC`). Instead of juggling with triple escape sequences like `\\\"`, just move the script to a separate file and call it with `DOCKER_EXEC`.

  Also, fix a bunch of other bugs that prevent running the ci system in qemu

  See the `ci/README.md` on how to test. TLDR: Can be tested with (replace `arm` with `s390x` to run the s390x build):

  ```
  FILE_ENV="./ci/test/00_setup_env_arm.sh" MAKEJOBS="-j9" ./ci/test_run_all.sh
  ```

ACKs for top commit:
  laanwj:
    Code review ACK fa569e1a9c5ad1bf8bdf866235b21aff56112224

Tree-SHA512: 84ebc44a4f0261ee6c29605a6896a1833ff6c81d729e6d08dd111941f570ce73221422bd3303e1108a266ec5eab2148bd5ee1cf6bc01477d8cc9a6c5bf2b34c2
2023-05-24 12:43:57 -05:00
MarcoFalke
8d340ed4e9 Merge #17900: ci: Combine 32-bit build with CentOS 7 build
ef63f5fc1136ad2a2cd080d44142a2ee3945c238 ci: Combine 32-bit build with CentOS 7 build (Sebastian Falbesoner)

Pull request description:

  Combines the CentOS build with the 32-bit (i686) build to avoid Travis bottlenecks, as suggested in #17757 by MarcoFalke. This keeps most of the properties of the 32-bit build (dash as config shell, building QT5 GUI) and just builds it with depends inside the CentOS docker container.

  Making the depends in `05_before_script.sh` with unset config shell (`CONFIG_SHELL=`)

  6196e93001/ci/test/05_before_script.sh (L28)

  caused problems for building the library libevent (resulting in a Makefile with no shell set (`SHELL=`)), that's why I set it explicitely to `/bin/bash` if we have a CentOS Docker container.

  A Travis output of this 32-bit CentOS build can be seen here: https://travis-ci.org/theStack/bitcoin/jobs/634472394 (has been restarted once due to too long build time and appearance of the `CACHE_ERR_MSG`).

  For anyone wanting to verify the outputs, I found these instructions useful to reproduce a Travis build locally: https://github.com/erdc/proteus/wiki/Replicating-the-TravisCI-Environment-on-your-Local-Machine (steps 1-3). In this case it's a bit tricky since you run Docker inside Docker -- within the Travis Docker container, the CentOS Docker container is created. To make this possible, the Docker socket has to be exposed to the Travis container via bind-mounting (`docker run -v /var/run/docker.sock:/var/run/docker.sock ...`), as suggested in https://stackoverflow.com/a/33003273.

Top commit has no ACKs.

Tree-SHA512: af508241cec3a10a66c37673d56691717b78375340e910fcdd3fb3870741eba623a436e1e85b26b54f013375611896f5411c5a7fec2437d367d27172230129fe
2023-05-24 12:43:57 -05:00
Konstantin Akimov
b8b37f314b Merge #17891: scripted-diff: Replace CCriticalSection with RecursiveMutex
e09c701e0110350f78366fb837308c086b6503c0 scripted-diff: Bump copyright of files changed in 2020 (MarcoFalke)
6cbe6209646db8914b87bf6edbc18c6031a16f1e scripted-diff: Replace CCriticalSection with RecursiveMutex (MarcoFalke)

Pull request description:

  `RecursiveMutex` better clarifies that the mutex is recursive, see also the standard library naming: https://en.cppreference.com/w/cpp/thread/recursive_mutex

  For that reason, and to avoid different people asking me the same question repeatedly (e.g. https://github.com/bitcoin/bitcoin/pull/15932#pullrequestreview-339175124 ), remove the outdated alias `CCriticalSection` with a scripted-diff
2023-05-24 12:43:57 -05:00
Kittywhiskers Van Gogh
8b054d49c7 ci: harmonize parent images, use focal consistently 2023-05-11 09:18:48 -05:00
Kittywhiskers Van Gogh
49e01820bb merge bitcoin#18705: Remove xenial tsan workaround 2023-05-11 09:18:48 -05:00
Wladimir J. van der Laan
d4bc451492 Merge #15584: build: disable BIP70 support by default
BIP70 is removed in "merge #17165: Remove BIP70 support  (#4023)".
So, this commit contains only some follow-ups to unify codebase

e09913f1c47e693b0c6fafef55b9ca78e5f3abc0 doc: specify protobuf as optional in build docs (fanquake)
376f4929f8f75011b72b2f9c3164980db482278a build: disable BIP70 support by default (fanquake)

Pull request description:

  Disable BIP70 support in the GUI by default for `0.19.0` (for eventual removal in `0.20.0`?).

  Users who want to compile with BIP70 support enabled can pass `--enable-bip70` to `./configure`.

  I've inverted the current `--disable-bip70` test to instead pass `--enable-bip70`.

  Tested configurations on `macOS` (`protobuf` installed with `brew`).
  Protobuf available and `./configure`:
  ```
  Options used to compile and link:
    with wallet   = yes
    with gui / qt = yes
      with bip70  = no
  ```

  Protobuf available and `./configure --enable-bip70`:
  ```
  Options used to compile and link:
    with wallet   = yes
    with gui / qt = yes
      with bip70  = yes
  ```

  Protobuf not available (i.e `brew unlink protobuf`) and `./configure`:
  ```
  Options used to compile and link:
    with wallet   = yes
    with gui / qt = yes
      with bip70  = no
  ```

  Protobuf not available and `./configure --enable-bip70`:
  ```
  checking whether to build test_bitcoin-qt... yes
  checking whether to build BIP70 support... configure: error: protobuf missing
  ```

  TODO:
  - [x] Remove `protobuf` from other Travis builds
  - [ ] Documentation updates (mention that `protobuf` is now optional)?
  - [ ] Could split release notes into GUI and build

ACKs for top commit:
  laanwj:
    ACK e09913f1c47e693b0c6fafef55b9ca78e5f3abc0
  elichai:
    ACK e09913f1c47e693b0c6fafef55b9ca78e5f3abc0 Read the autotools changes. awesome that this removes the protobuf requirement.
  practicalswift:
    ACK e09913f1c47e693b0c6fafef55b9ca78e5f3abc0 -- diff looks correct

Tree-SHA512: 7bf87ae8555e24db2da2e89cc4d4e90d09be27499ad386ad65879d05df8f96d9a1384379891ac8963d17728c90e55961560813df97e849e631e2de8c08e210c8
2023-04-25 23:41:20 -05:00
MarcoFalke
b885060802 Merge #17549: ci: misc cleanups
fad88e6f86d1dd32cf01db2287df9c63e66c5116 ci: Remove use of cd (MarcoFalke)
fa2941bbf47a8a6b79b8db4a87e1aedcf6a29a5e ci: Remove unmaintained extended_lint (MarcoFalke)
fa041875284d01602647519cc452185ba1ad5a8f scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR (MarcoFalke)
fa0656d1ebc2a01d900fead4ce055d59eb818a1f ci: Add DEPENDS_DIR variable, Add documentation for folders (MarcoFalke)
faeeca87b65dd98e0efbc54443b3f8854cae9c00 scripted-diff: Move various folders to ci scratch dir (MarcoFalke)

Pull request description:

  Some minor cleanups:

  * Remove unused and unmaintained extended_lint to avoid ci bottlenecks and waste of CPU
  * Move all folders that hold temporary ci files to the ci scratch dir (except for the build dirs)
  * Add some documentation to folders and remove the `BASE_BUILD_DIR` alias for the root directory
  * Fixes #17178 by removing `cd`

Top commit has no ACKs.

Tree-SHA512: f6eb9d47ab98c08af59a79c2a4bf62fc331f0f6f3174909a28f2c2f48b0234423da5aee876225d2e188619b71b008d882a20d29a7aca68248039ea5080be5af5
2023-04-25 23:14:25 +03:00
MarcoFalke
4929913f0d Merge #17674: tests: Add initialization order fiasco detection in Travis
1f9d5af4f197e7cc0469a0bb25dcbc51dfa537f4 tests: Add initialization order fiasco detection in Travis (practicalswift)

Pull request description:

  Add initialization order fiasco detection in Travis :)

  Context: https://github.com/bitcoin/bitcoin/pull/17670#issuecomment-562035813

  This would have caught the `events_hasher` initialization order issue introduced in #17573  and fixed in #17670.

  Output in case of an initialization order fiasco:

  ```
  ==7934==ERROR: AddressSanitizer: initialization-order-fiasco on address 0x557098d79200 at pc 0x55709796b9a3 bp 0x7ffde524dc30 sp 0x7ffde524dc28
  READ of size 8 at 0x557098d79200 thread T0
      #0 0x55709796b9a2 in CSHA256::Finalize(unsigned char*) src/crypto/sha256.cpp:667:25
      #1 0x5570978150e9 in SeedEvents(CSHA512&) src/random.cpp:462:19
      #2 0x5570978145e1 in SeedSlow(CSHA512&) src/random.cpp:482:5
      #3 0x5570978149a3 in SeedStartup(CSHA512&, (anonymous namespace)::RNGState&) src/random.cpp:527:5
      #4 0x55709781102d in ProcRand(unsigned char*, int, RNGLevel) src/random.cpp:571:9
      #5 0x557097810d19 in GetRandBytes(unsigned char*, int) src/random.cpp:576:59
      #6 0x557096c2f9d5 in (anonymous namespace)::CSignatureCache::CSignatureCache() src/script/sigcache.cpp:34:9
      #7 0x557096511977 in __cxx_global_var_init.7 src/script/sigcache.cpp:67:24
      #8 0x5570965119f8 in _GLOBAL__sub_I_sigcache.cpp src/script/sigcache.cpp
      #9 0x557097bba4ac in __libc_csu_init (src/bitcoind+0x18554ac)
      #10 0x7f214b1c2b27 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:266
      #11 0x5570965347d9 in _start (src/bitcoind+0x1cf7d9)

  0x557098d79200 is located 96 bytes inside of global variable 'events_hasher' defined in 'random.cpp:456:16' (0x557098d791a0) of size 104
    registered at:
      #0 0x557096545dfd in __asan_register_globals compiler-rt/lib/asan/asan_globals.cpp:360:3
      #1 0x557097817f8b in asan.module_ctor (src/bitcoind+0x14b2f8b)

  SUMMARY: AddressSanitizer: initialization-order-fiasco src/crypto/sha256.cpp:667:25 in CSHA256::Finalize(unsigned char*)
  ```

ACKs for top commit:
  promag:
    Tested ACK 1f9d5af4f197e7cc0469a0bb25dcbc51dfa537f4, got
  MarcoFalke:
    ACK 1f9d5af4f197e7cc0469a0bb25dcbc51dfa537f4 👔

Tree-SHA512: f24ac0a313df7549193bd7f4fcfdf9b72bdfc6a6ee31d0b08e6d0752e5108fbd532106b6c86377ae0641258c9adb4921872e5d9a0154c0284e03315e0777102c
2023-04-25 23:14:25 +03:00
MarcoFalke
675a8a8586 Merge #17661: ci: use depends for s390x
e1900008699bd45031b7faa9ef3d0a81d54091b0 ci-s390x: Add qemu and depends support in the ci script (Elichai Turkel)

Pull request description:

  Related: #17599

  This adds qemu support just like we have in arm and compile the depends.

  other than that I also fixed some missing includes to make the depends compile.

ACKs for top commit:
  MarcoFalke:
    ACK e190000 (first commit only, didn't look at second commit)

Tree-SHA512: 2b8a39772b86408569f52cdc33832dbce7e5e9cdd710524295f3d259628cdfc017e740f6f94941307d7f8e413236814a95ba851153c617eb5fb75b4bd9a7e52f
2023-04-25 23:14:25 +03:00
MarcoFalke
41b4f003da Merge #17599: ci: Run functional tests on s390x
fabd71076cd9493bd2d30a198467f5ea621b27aa ci: Print free disk space (MarcoFalke)
fad9fdbea5dfb19328282afda9588edc6f1d0ddf test: Properly deserialize integers in little-endian (MarcoFalke)
fa94fc10c881e502e6c9a71f3b7719aa955900f9 ci: Run functional tests on s390x (MarcoFalke)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 98ba77eb56f283131fdaeb393fda86cc308f1bf9781e1e0e5736b8d616528dc8ff2e494d55ba107c138083025c66a59e382fcfa9962d4349a5fd6cbbc52484c3
2023-04-17 19:34:02 +03:00
UdjinM6
33f490e615
fix(tests): Fix retries for the non-deterministic test suite (#5307)
## Issue being fixed or feature implemented
the problem with retries implemented in #4793 is that they don't really
do anything besides fetching results of a failed job multiple times 🙈

## What was done?
partially reverted changes done in #4793, implemented actual job
restart. dropped `--sleep` and `--retries` and added `--attempts`
instead.

## How Has This Been Tested?
Pick any test, add some randomly failing expression somewhere and run it
with some high number of retries.

For example:
```diff
diff --git a/test/functional/feature_dip0020_activation.py b/test/functional/feature_dip0020_activation.py
index 471e4fdc66..b56a954b78 100755
--- a/test/functional/feature_dip0020_activation.py
+++ b/test/functional/feature_dip0020_activation.py
@@ -69,6 +69,7 @@ class DIP0020ActivationTest(BitcoinTestFramework):
         # Should be spendable now
         tx0id = self.node.sendrawtransaction(tx0_hex)
         assert tx0id in set(self.node.getrawmempool())
+        assert int(tx0id[0], 16) < 4
 
 
 if __name__ == '__main__':
```

On develop:
```
./test/functional/test_runner.py feature_dip0020_activation.py --retries=100 --sleep=0
```
if this fails on the first run, it keeps "failing" (simply fetching the
same results actually) till the end.

With this patch:
```
./test/functional/test_runner.py feature_dip0020_activation.py --attempts=100
```
if this fails on the first run, it can actually succeed after a few
attempts now, unless you are extremely unlucky ofc 😄

Also, check [ci results in my repo
](https://cdn.artifacts.gitlab-static.net/93/b4/93b4f8b17e5dcccab1afee165b4d74d90f05800caf65d6c48a83a1a78c979587/2023_04_08/4081291268/4478867166/job.log?response-content-type=text%2Fplain%3B%20charset%3Dutf-8&response-content-disposition=inline&Expires=1680945516&KeyName=gprd-artifacts-cdn&Signature=2d4mHCJBbgRaTDiSQ6kKIy1PdIM=).

Note:
```
...
feature_dip3_v19.py failed at attempt 1/3, Duration: 159s
...
4/179 - feature_dip3_v19.py passed, Duration: 244 s
...
feature_llmq_hpmn.py failed at attempt 1/3, Duration: 284s
...
feature_llmq_hpmn.py failed at attempt 2/3, Duration: 296s
...
11/179 - feature_llmq_hpmn.py failed, Duration: 233 s
...
```

An example with 2 tests failing initially and then passing:
https://gitlab.com/dashpay/dash/-/jobs/4089689970

## Breaking Changes
n/a


## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation

**For repository code-owners and collaborators only**
- [x] I have assigned this pull request to a milestone
2023-04-14 23:13:47 -05:00
Konstantin Akimov
e35aeddf4a
fix: reviewing TODOes at v19 (#5303)
## Issue being fixed or feature implemented
During reviewing TODO were found some TODOes that can be done now.

## What was done?
 - fix: follow-up dash#3467 - replaced commented code to disabled code
- follow-up bitcoin#16394 - uncommented code related to `watchonly`
feature
 - removed out-dated TODO in `rpc/masternode.cpp` (already done)
- fix: renamed name of clean up test_unittests: removed TODO and updated
name of variable TRAVIS
 - rewritten todo inside `.travis.yml`
 - fix: adds a missing description for result of rpc `mnsync`

Last commit (`mnsync`) is an only candidate for backport to v19, other
changes are non significant.

## How Has This Been Tested?
Run functional/unit tests

## Breaking Changes
No breaking changes


## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
2023-04-09 00:11:22 -05:00
MarcoFalke
27b5d68562 Merge #15693: travis: Switch to ubuntu keyserver to avoid timeouts
fa2056af1c travis: Properly cache and error on timeout (MarcoFalke)
fa36a333ee travis: Switch to ubuntu keyserver to avoid timeouts (MarcoFalke)

Pull request description:

  The other keyserver is consistently timing out on travis: https://travis-ci.org/bitcoin/bitcoin/jobs/512689710#L405

  Attempt to fix it by using a different server.

  Also:
  * fixes #15372
  * fixes #15738

ACKs for commit fa2056:
  ryanofsky:
    utACK fa2056af1c71aded3a821a07ec4de71c4be0bca3. All good changes (changing keyserver, getting rid of keyserver while loop, clarifying travis error, moving travis documentation to code comment).

Tree-SHA512: ac8436616ecfee0ed579114e19f03c53ceb688fbcd95a60cffe8f15b4e569772a6ba673f353bbd789e79fe27fc5626c77fab4086768844dd51e0c6c108b52fb2
2023-04-06 20:14:58 +03:00
Konstantin Akimov
8eb04b2f1f Merge #14092: tests: Dry run bench_bitcoin as part "make check" to allow for quick identification of assertion/sanitizer failures in benchmarking code
Follow-up this earlier merged backport - removed unused variable RUN_BENCH
2023-04-06 20:14:58 +03:00
MarcoFalke
cb3ed877c3 Merge #16633: travis: Fix test_runner.py timeouts
faba46da07cd8383d0bc841d37ea9cacba60e354 ci: Set --ansi in test_runner (MarcoFalke)

Pull request description:

  Fixup to:

  *  tests: Use colors and dots in test_runner.py output only if standard output is a terminal #16561

ACKs for top commit:
  practicalswift:
    ACK faba46da07cd8383d0bc841d37ea9cacba60e354 -- diff looks correct
  fanquake:
    ACK faba46da07cd8383d0bc841d37ea9cacba60e354 - assuming Travis is all green.

Tree-SHA512: 50bf6ec8e7a2987d77821816289fd87458e63237d419a149e5e04027387d2c4b3c23db58977585fda30fbc4b13686a7def0d9e00096b9a8469dea2916fd30565
2023-04-06 09:55:22 -05:00
fanquake
7d78f06a92 partial Merge bitcoin/bitcoin#19817: build: macOS toolchain bump
a5550f877a2c46d01bb620ae051c0c8ed0fecd0b build: use -stdlib++-isystem with Clang 10 (fanquake)
51d9d1607f2e9d593693ca799393f068192e41aa guix: use Clang 10 for the macOS cross compile (fanquake)
b80a6af9e55325d444e117e85bbfc76d88d898a8 build: no longer patch threading out of ld64 (fanquake)
c29cba44b3706e0a2035e440e560f2d15d50433b build: Xcode 12.1, macOS SDK 10.15.6 (fanquake)
9ed2f19d385aa95f65807999bba2e18417b143dc build: native cctools 973.0.1, ld64 609 (fanquake)
f48f187cce7fa43646fb0d796c244e1515e763ec build: Clang 10.0.1 (Hennadii Stepanov)
9b193cd2a3ca20917611fbed56dfbcd8a39aeab8 build: libtapi 1100.0.11 (fanquake)

Pull request description:

  Bumps our macOS toolchain to be using the following:
  * Clang 10.0.1 (gitian) & Clang 10.0.0 (Guix)
  * ld64 609
  * libtapi 1100.0.11
  * cctools  973.0.1
  * Xcode 12.1
  * macOS SDK 10.15.6

  which are currently the most recent releases available as open source. See upstream [`cctools`](https://github.com/tpoechtrager/cctools-port/commits/973.0.1-ld64-609) and [`libtapi`](https://github.com/tpoechtrager/apple-libtapi/tree/1100.0.11).

  This should improve the possibility of Apple ARM cross-compilation in depends.

  This also removes our [patching out of pthreads usage](https://github.com/bitcoin/bitcoin/blob/master/depends/patches/native_cctools/ld64_disable_threading.patch) in `ld64`. There have been multiple changes since `ld64 450.3`, which have likely fixed the non-determinism we were working around. i.e from [InputFiles.cpp](https://opensource.apple.com/source/ld64/ld64-609/src/ld/InputFiles.cpp.auto.html):
  ```cpp
  // <rdar://problem/15002251> make implicit dylib order be deterministic by sorting by install_name
  std::sort(implicitDylibs.begin(), implicitDylibs.end(), DylibByInstallNameSorter());
  ```

  ```cpp
  // <rdar://problem/42675402> ld64 output is not deterministic due to dylib processing order
  std::sort(unprocessedDylibs.begin(), unprocessedDylibs.end(), [](const ld::dylib::File* lhs, const ld::dylib::File* rhs) {
  return strcmp(lhs->path(), rhs->path()) < 0;
  });
  ```

  Guix Build:
  ```bash
  find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  f6c3817b8fe5f7370299d1ae2533e4a3acd313ba9f9aa8d423a8956117e52dd5  guix-build-a5550f877a2c/output/dist-archive/bitcoin-a5550f877a2c.tar.gz
  4954dcf563c2d496b8d9fecd48f8e3f7fba2f319ffa254a5bc8ee12cfee6acf0  guix-build-a5550f877a2c/output/x86_64-apple-darwin18/bitcoin-a5550f877a2c-osx-unsigned.dmg
  8f6095b445c7f1a8e6accd86bb7f0696d5849402084927d2b726b7d557831c3a  guix-build-a5550f877a2c/output/x86_64-apple-darwin18/bitcoin-a5550f877a2c-osx-unsigned.tar.gz
  cc40f25477b4defc1617ae694313d80f307ddf6742fe6cc85c6bc0e215ef8be0  guix-build-a5550f877a2c/output/x86_64-apple-darwin18/bitcoin-a5550f877a2c-osx64.tar.gz
  ```

  Gitian Build:
  ```bash
  Generating report
  506a8abdefe559999b43dd9f14905b9b2b5a3363b1cd013d45ae47acc1f7ef6c  bitcoin-a5550f877a2c-osx-unsigned.dmg
  f606997f74026dd12d110d683c6f116b40df324836904ef507dd7ac787e6ebe2  bitcoin-a5550f877a2c-osx-unsigned.tar.gz
  5b495ef15f2c3260c2950921b61326912a9bf533cccd51e13818809fd225489e  bitcoin-a5550f877a2c-osx64.tar.gz
  f6c3817b8fe5f7370299d1ae2533e4a3acd313ba9f9aa8d423a8956117e52dd5  src/bitcoin-a5550f877a2c.tar.gz
  9eb0221e962d2839770963bd03c6c9e98e8bf3078566bee2ae42f06233a710fa  bitcoin-core-osx-22-res.yml
  Done.
  ```

ACKs for top commit:
  hebasto:
    ACK a5550f877a2c46d01bb620ae051c0c8ed0fecd0b

Tree-SHA512: 504c4b0f9cd3b939714a322298320c5bde07e9356a48a9a000060b36f8dce4d6134ed60c3a5188810476a28ec5b108733eabbc6fb8053231b9ea8a494cc91b12
2023-03-26 16:50:26 -05:00
Kittywhiskers Van Gogh
daa40ea8c6 ci: add sqlite-only linux64 test build 2023-02-17 14:21:19 -06:00
MarcoFalke
b87625d738 Merge #18901: fuzz: use std::optional for sep_pos_opt variable
420fa0770f37619bfa29898d59dac45b6a477abb fuzz: use std::optional for sep_pos variable (Harris)

Pull request description:

  This PR changes the original `size_t sep_pos` to `std::optional<size_t> sep_post_opt` to remove the warning when compiling fuzz tests.

  ```shell
  warning: variable 'sep_pos' may be uninitialized when used here [-Wconditional-uninitialized]
  ```

  Also, it adds `--enable-c++17` flag to CI fuzz scripts.

ACKs for top commit:
  practicalswift:
    ACK 420fa0770f37619bfa29898d59dac45b6a477abb
  MarcoFalke:
    ACK 420fa07

Tree-SHA512: e967d5d8ab8ee7394b243ff5b28bac72d30bd14774e4a206f8c87474fad22769da76e4ba4e03cbef83b8f60e5293e9d9293b613e2e2e59e187d4e59ae6b874ca
2023-01-23 11:21:05 -06:00
UdjinM6
828fac4c14 chore: add missing copyrights via copyright_header.py insert 2023-01-13 00:49:04 +03:00
Kittywhiskers Van Gogh
305abe91f9
build/depends: commit dashpay/bls-signatures@66ee820f to source tree as vendored (#5077)
* Squashed 'src/dashbls/' content from commit 66ee820fbc

git-subtree-dir: src/dashbls
git-subtree-split: 66ee820fbc9e3b97370db8c164904af48327a124

* build: stop tracking build-system generated relic_conf.h.in

* build: add support for building bls-signatures from local subtree

* build: add exclusions to linting scripts and filters

* build: drop bls-signatures (bls-dash) from depends
2022-11-22 11:34:46 -06:00
Kittywhiskers Van Gogh
184bd6031b ci: make fuzzing builds stricter by enabling -Werror by default
Unfortunately, we need -Wno-unused-command-line-argument as configure will fail to run
with -Werror due to Dash's current build system utilizing -static-libstdc++ indiscriminately,
resulting in Clang warnings of "unused arguments".
2022-10-21 19:09:56 +05:30
fanquake
02aac6b0f8 Merge #18162: util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t) by checking gmtime_s/gmtime_r return value
12a2f377185a413b740460db36812de22ee2e041 util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t nTime) by checking gmtime_s/gmtime_r return value (practicalswift)

Pull request description:

  Avoid potential uninitialized read in `FormatISO8601DateTime(int64_t)` by checking `gmtime_s`/`gmtime_r` return value.

  Before this patch `FormatISO8601DateTime(67768036191676800)` resulted in:

  ```
  ==5930== Conditional jump or move depends on uninitialised value(s)
  ==5930==    at 0x4F44C0A: std::ostreambuf_iterator<char, std::char_traits<char> > std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<long>(std::ostreambuf_iterator<char, std::char_traits<char> >, std::ios_base&, char, long) const (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
  ==5930==    by 0x4F511A4: std::ostream& std::ostream::_M_insert<long>(long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
  ==5930==    by 0x4037C3: void tinyformat::formatValue<int>(std::ostream&, char const*, char const*, int, int const&) (tinyformat.h:358)
  ==5930==    by 0x403725: void tinyformat::detail::FormatArg::formatImpl<int>(std::ostream&, char const*, char const*, int, void const*) (tinyformat.h:543)
  ==5930==    by 0x402E02: tinyformat::detail::FormatArg::format(std::ostream&, char const*, char const*, int) const (tinyformat.h:528)
  ==5930==    by 0x401B16: tinyformat::detail::formatImpl(std::ostream&, char const*, tinyformat::detail::FormatArg const*, int) (tinyformat.h:907)
  ==5930==    by 0x4017AE: tinyformat::vformat(std::ostream&, char const*, tinyformat::FormatList const&) (tinyformat.h:1054)
  ==5930==    by 0x401765: void tinyformat::format<int, int, int, int, int, int>(std::ostream&, char const*, int const&, int const&, int const&, int const&, int const&, int const&) (tinyformat.h:1064)
  ==5930==    by 0x401656: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > tinyformat::format<int, int, int, int, int, int>(char const*, int const&, int const&, int const&, int const&, int const&, int const&) (tinyformat.h:1073)
  ==5930==    by 0x4014CC: FormatISO8601DateTime[abi:cxx11](long) (…)
  ```

  The same goes for other very large positive and negative arguments.

  Fix by simply checking the `gmtime_s`/`gmtime_r` return value :)

ACKs for top commit:
  MarcoFalke:
    ACK 12a2f377185a413b740460db36812de22ee2e041
  theStack:
    re-ACK 12a2f37718
  elichai:
    re ACK 12a2f377185a413b740460db36812de22ee2e041

Tree-SHA512: 066142670d9bf0944d41fa3f3c702b1a460b5471b93e76a619b1e818ff9bb9c09fe14c4c37e9536a04c99533f7f21d1b08ac141e1b829ff87ee54c80d0e61d48
2022-10-20 11:47:02 -04:00
MarcoFalke
f7d51ad13a Merge #18166: ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors
f2472f64604a0c583f950c56e8753d0bee246388 tests: Improve test runner output in case of target errors (practicalswift)
733bbec34fbec85574cc456832b2b2f807e5dce9 tests: Add --exclude integer,parse_iso8601 (temporarily) to make Travis pass until uninitialized read issue in FormatISO8601DateTime is fixed (practicalswift)
5ea81449f30a6fe6db3b6df5e8009f21a782ff44 tests: Add support for excluding fuzz targets using -x/--exclude (practicalswift)
555236f769c13518db70f5df36e5688d63486bd5 tests: Remove -detect_leaks=0 from test/fuzz/test_runner.py - no longer needed (practicalswift)
a3b539a924f8611abb3096f2bd9d35094b5577e3 ci: Run fuzz testing test cases under valgrind (practicalswift)

Pull request description:

  Run fuzz testing [test cases (bitcoin-core/qa-assets)](https://github.com/bitcoin-core/qa-assets) under `valgrind`.

  This would have caught `util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t) by checking gmtime_s/gmtime_r return value` (#18162) and similar cases.

ACKs for top commit:
  MarcoFalke:
    ACK f2472f64604a0c583f950c56e8753d0bee246388 👼

Tree-SHA512: bb0879d40167cf6906bc0ed31bed39db83c39c7beb46026f7b0ee53f28ff0526ad6fabc3f4cb3f5f18d3b8cafdcbf5f30105b35919f4e83697c71e838ed71493
2022-10-20 11:47:02 -04:00