sikkienl/dash
1
0
Fork 0
mirror of https://github.com/dashpay/dash.git synced 2024-12-24 19:42:46 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge #6491: ci: merge bitcoin#27314, #28954, fix multiprocess builds

Browse Source 
27d9763b1b fix: add `linux64_multiprocess` `BUILD_TARGET` to matrix, mend C(XX) (Kittywhiskers Van Gogh)
26cc5a1c90 ci: use underscore to separate variant name from target triple (Kittywhiskers Van Gogh)
d0131a5259 trivial: sort `BUILD_TARGET` on GitHub and in `matrix.sh` alphabetically (Kittywhiskers Van Gogh)
4f1b5c165b merge bitcoin#28954: Reduce use of bash -c (Kittywhiskers Van Gogh)
a49162ffae merge bitcoin#27314: Fix handling of `CXX=clang++` when building `qt` package (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * [bitcoin#27314](https://github.com/bitcoin/bitcoin/pull/27314) has been backported in this PR as [bitcoin#25838](https://github.com/bitcoin/bitcoin/pull/25838) (backported in [dash#6384](https://github.com/dashpay/dash/pull/6384)) broke Clang depends builds.

  * [bitcoin#28954](https://github.com/bitcoin/bitcoin/pull/28954) has been backported to fix a problem associated with multiprocess runs ([build](https://gitlab.com/dashpay/dash/-/jobs/8396677312#L2921)).

  * Support for multiprocess builds were enabled _proper_ in [dash#6143](https://github.com/dashpay/dash/pull/6143) but unfortunately, the configuration params for multiprocess builds were not processed by CI as the build variant was not added to `matrix.sh` ([source](6a51ab271d/ci/dash/matrix.sh)). This is evident by comparing two variants with Boost::Process enablement (`--with-boost-process`), `linux64_fuzz` ([source](6a51ab271d/ci/test/00_setup_env_native_fuzz.sh (L19))) and `linux64_multiprocess` ([source](6a51ab271d/ci/test/00_setup_env_native_multiprocess.sh (L13))).

    Looking at a `develop` (6a51ab271d) build, the fuzz build has it enabled ([source](https://gitlab.com/dashpay/dash/-/jobs/8394892905#L737)) while the multiprocess build doesn't ([source](https://gitlab.com/dashpay/dash/-/jobs/8394892909#L1524)) despite both scripts having the enablement argument.

  ## Breaking Changes

  None expected.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK 27d9763b1b
  UdjinM6:
    utACK 27d9763b1b

Tree-SHA512: 3e2fb72d4211875a162d3aecb994c5bd43b2f6d9fea0804d7e00a38a034672730f9351ceb9256ace38e32f7ef81527c8a034a870e5099a277dfd06f9fa54b480
This commit is contained in:
pasta 2024-12-16 20:41:26 -06:00
commit a9cfd39390
No known key found for this signature in database
GPG Key ID: 5255B86F912A614A
9 changed files with 46 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
.github/workflows/build.yml vendored
View File

@ -114,22 +114,22 @@ jobs:
- build_target: linux64 - build_target: linux64
host: x86_64-pc-linux-gnu host: x86_64-pc-linux-gnu
depends_on: linux64 depends_on: linux64
- build_target: linux64_tsan - build_target: linux64_cxx20
host: x86_64-pc-linux-gnu
depends_on: linux64
- build_target: linux64_ubsan
host: x86_64-pc-linux-gnu host: x86_64-pc-linux-gnu
depends_on: linux64 depends_on: linux64
- build_target: linux64_fuzz - build_target: linux64_fuzz
host: x86_64-pc-linux-gnu host: x86_64-pc-linux-gnu
depends_on: linux64 depends_on: linux64
- build_target: linux64_cxx20 - build_target: linux64_nowallet
host: x86_64-pc-linux-gnu host: x86_64-pc-linux-gnu
depends_on: linux64 depends_on: linux64
- build_target: linux64_sqlite - build_target: linux64_sqlite
host: x86_64-pc-linux-gnu host: x86_64-pc-linux-gnu
depends_on: linux64 depends_on: linux64
- build_target: linux64_nowallet - build_target: linux64_tsan
host: x86_64-pc-linux-gnu
depends_on: linux64
- build_target: linux64_ubsan
host: x86_64-pc-linux-gnu host: x86_64-pc-linux-gnu
depends_on: linux64 depends_on: linux64
container: container:

28
.gitlab-ci.yml
View File

@ -193,13 +193,13 @@ x86_64-w64-mingw32:
variables: variables:
HOST: x86_64-w64-mingw32 HOST: x86_64-w64-mingw32
x86_64-pc-linux-gnu-debug: x86_64-pc-linux-gnu_debug:
extends: .build-depends-template extends: .build-depends-template
variables: variables:
HOST: x86_64-pc-linux-gnu HOST: x86_64-pc-linux-gnu
DEP_OPTS: "DEBUG=1" DEP_OPTS: "DEBUG=1"
x86_64-pc-linux-gnu-nowallet: x86_64-pc-linux-gnu_nowallet:
extends: extends:
- .build-depends-template - .build-depends-template
- .skip-in-fast-mode-template - .skip-in-fast-mode-template
@ -207,13 +207,13 @@ x86_64-pc-linux-gnu-nowallet:
HOST: x86_64-pc-linux-gnu HOST: x86_64-pc-linux-gnu
DEP_OPTS: "NO_WALLET=1" DEP_OPTS: "NO_WALLET=1"
x86_64-pc-linux-gnu-multiprocess: x86_64-pc-linux-gnu_multiprocess:
extends: extends:
- .build-depends-template - .build-depends-template
- .skip-in-fast-mode-template - .skip-in-fast-mode-template
variables: variables:
HOST: x86_64-pc-linux-gnu HOST: x86_64-pc-linux-gnu
DEP_OPTS: "MULTIPROCESS=1" DEP_OPTS: "DEBUG=1 MULTIPROCESS=1"
x86_64-apple-darwin: x86_64-apple-darwin:
extends: extends:
@ -243,7 +243,7 @@ win64-build:
linux64-build: linux64-build:
extends: .build-template extends: .build-template
needs: needs:
- x86_64-pc-linux-gnu-debug - x86_64-pc-linux-gnu_debug
variables: variables:
BUILD_TARGET: linux64 BUILD_TARGET: linux64
@ -252,7 +252,7 @@ linux64_cxx20-build:
- .build-template - .build-template
- .skip-in-fast-mode-template - .skip-in-fast-mode-template
needs: needs:
- x86_64-pc-linux-gnu-debug - x86_64-pc-linux-gnu_debug
variables: variables:
BUILD_TARGET: linux64_cxx20 BUILD_TARGET: linux64_cxx20
@ -261,7 +261,7 @@ linux64_sqlite-build:
- .build-template - .build-template
- .skip-in-fast-mode-template - .skip-in-fast-mode-template
needs: needs:
- x86_64-pc-linux-gnu-debug - x86_64-pc-linux-gnu_debug
variables: variables:
BUILD_TARGET: linux64_sqlite BUILD_TARGET: linux64_sqlite
@ -270,7 +270,7 @@ linux64_fuzz-build:
- .build-template - .build-template
- .skip-in-fast-mode-template - .skip-in-fast-mode-template
needs: needs:
- x86_64-pc-linux-gnu-debug - x86_64-pc-linux-gnu_debug
variables: variables:
BUILD_TARGET: linux64_fuzz BUILD_TARGET: linux64_fuzz
@ -279,7 +279,7 @@ linux64_fuzz-build:
# - .build-template # - .build-template
# - .skip-in-fast-mode-template # - .skip-in-fast-mode-template
# needs: # needs:
# - x86_64-pc-linux-gnu-debug # - x86_64-pc-linux-gnu_debug
# variables: # variables:
# BUILD_TARGET: linux64_asan # BUILD_TARGET: linux64_asan
@ -288,7 +288,7 @@ linux64_tsan-build:
- .build-template - .build-template
- .skip-in-fast-mode-template - .skip-in-fast-mode-template
needs: needs:
- x86_64-pc-linux-gnu-debug - x86_64-pc-linux-gnu_debug
variables: variables:
BUILD_TARGET: linux64_tsan BUILD_TARGET: linux64_tsan
@ -297,7 +297,7 @@ linux64_ubsan-build:
- .build-template - .build-template
- .skip-in-fast-mode-template - .skip-in-fast-mode-template
needs: needs:
- x86_64-pc-linux-gnu-debug - x86_64-pc-linux-gnu_debug
variables: variables:
BUILD_TARGET: linux64_ubsan BUILD_TARGET: linux64_ubsan
@ -306,7 +306,7 @@ linux64_nowallet-build:
- .build-template - .build-template
- .skip-in-fast-mode-template - .skip-in-fast-mode-template
needs: needs:
- x86_64-pc-linux-gnu-nowallet - x86_64-pc-linux-gnu_nowallet
variables: variables:
BUILD_TARGET: linux64_nowallet BUILD_TARGET: linux64_nowallet
@ -315,7 +315,7 @@ linux64_multiprocess-build:
- .build-template - .build-template
- .skip-in-fast-mode-template - .skip-in-fast-mode-template
needs: needs:
- x86_64-pc-linux-gnu-multiprocess - x86_64-pc-linux-gnu_multiprocess
variables: variables:
BUILD_TARGET: linux64_multiprocess BUILD_TARGET: linux64_multiprocess
@ -324,7 +324,7 @@ linux64_multiprocess-build:
# - .build-template # - .build-template
# - .skip-in-fast-mode-template # - .skip-in-fast-mode-template
# needs: # needs:
# - x86_64-pc-linux-gnu-debug # - x86_64-pc-linux-gnu_debug
# variables: # variables:
# BUILD_TARGET: linux64_valgrind # BUILD_TARGET: linux64_valgrind

22
ci/dash/matrix.sh
View File

@ -18,28 +18,30 @@ export UBSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/
if [ "$BUILD_TARGET" = "arm-linux" ]; then if [ "$BUILD_TARGET" = "arm-linux" ]; then
source ./ci/test/00_setup_env_arm.sh source ./ci/test/00_setup_env_arm.sh
elif [ "$BUILD_TARGET" = "win64" ]; then
source ./ci/test/00_setup_env_win64.sh
elif [ "$BUILD_TARGET" = "linux64" ]; then elif [ "$BUILD_TARGET" = "linux64" ]; then
source ./ci/test/00_setup_env_native_qt5.sh source ./ci/test/00_setup_env_native_qt5.sh
elif [ "$BUILD_TARGET" = "linux64_asan" ]; then elif [ "$BUILD_TARGET" = "linux64_asan" ]; then
source ./ci/test/00_setup_env_native_asan.sh source ./ci/test/00_setup_env_native_asan.sh
elif [ "$BUILD_TARGET" = "linux64_cxx20" ]; then
source ./ci/test/00_setup_env_native_cxx20.sh
elif [ "$BUILD_TARGET" = "linux64_fuzz" ]; then
source ./ci/test/00_setup_env_native_fuzz.sh
elif [ "$BUILD_TARGET" = "linux64_multiprocess" ]; then
source ./ci/test/00_setup_env_native_multiprocess.sh
elif [ "$BUILD_TARGET" = "linux64_nowallet" ]; then
source ./ci/test/00_setup_env_native_nowallet.sh
elif [ "$BUILD_TARGET" = "linux64_sqlite" ]; then
source ./ci/test/00_setup_env_native_sqlite.sh
elif [ "$BUILD_TARGET" = "linux64_tsan" ]; then elif [ "$BUILD_TARGET" = "linux64_tsan" ]; then
source ./ci/test/00_setup_env_native_tsan.sh source ./ci/test/00_setup_env_native_tsan.sh
elif [ "$BUILD_TARGET" = "linux64_ubsan" ]; then elif [ "$BUILD_TARGET" = "linux64_ubsan" ]; then
source ./ci/test/00_setup_env_native_ubsan.sh source ./ci/test/00_setup_env_native_ubsan.sh
elif [ "$BUILD_TARGET" = "linux64_fuzz" ]; then
source ./ci/test/00_setup_env_native_fuzz.sh
elif [ "$BUILD_TARGET" = "linux64_cxx20" ]; then
source ./ci/test/00_setup_env_native_cxx20.sh
elif [ "$BUILD_TARGET" = "linux64_sqlite" ]; then
source ./ci/test/00_setup_env_native_sqlite.sh
elif [ "$BUILD_TARGET" = "linux64_nowallet" ]; then
source ./ci/test/00_setup_env_native_nowallet.sh
elif [ "$BUILD_TARGET" = "linux64_valgrind" ]; then elif [ "$BUILD_TARGET" = "linux64_valgrind" ]; then
source ./ci/test/00_setup_env_native_valgrind.sh source ./ci/test/00_setup_env_native_valgrind.sh
elif [ "$BUILD_TARGET" = "mac" ]; then elif [ "$BUILD_TARGET" = "mac" ]; then
source ./ci/test/00_setup_env_mac.sh source ./ci/test/00_setup_env_mac.sh
elif [ "$BUILD_TARGET" = "s390x" ]; then elif [ "$BUILD_TARGET" = "s390x" ]; then
source ./ci/test/00_setup_env_s390x.sh source ./ci/test/00_setup_env_s390x.sh
elif [ "$BUILD_TARGET" = "win64" ]; then
source ./ci/test/00_setup_env_win64.sh
fi fi

2
ci/dash/test_integrationtests.sh
View File

@ -42,7 +42,7 @@ echo "Using socketevents mode: $SOCKETEVENTS"
EXTRA_ARGS="--dashd-arg=-socketevents=$SOCKETEVENTS" EXTRA_ARGS="--dashd-arg=-socketevents=$SOCKETEVENTS"
set +e set +e
LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib ${TEST_RUNNER_ENV} ./test/functional/test_runner.py --ci --attempts=3 --ansi --combinedlogslen=4000 --timeout-factor=${TEST_RUNNER_TIMEOUT_FACTOR} ${TEST_RUNNER_EXTRA} --failfast --nocleanup --tmpdir=$(pwd)/testdatadirs $PASS_ARGS $EXTRA_ARGS LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib ./test/functional/test_runner.py --ci --attempts=3 --ansi --combinedlogslen=4000 --timeout-factor=${TEST_RUNNER_TIMEOUT_FACTOR} ${TEST_RUNNER_EXTRA} --failfast --nocleanup --tmpdir=$(pwd)/testdatadirs $PASS_ARGS $EXTRA_ARGS
RESULT=$? RESULT=$?
set -e set -e

4
ci/dash/test_unittests.sh
View File

@ -29,8 +29,8 @@ if [ "$DIRECT_WINE_EXEC_TESTS" = "true" ]; then
wine ./src/test/test_dash.exe wine ./src/test/test_dash.exe
else else
if [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then if [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
${TEST_RUNNER_ENV} ./src/test/test_dash --catch_system_errors=no -l test_suite ./src/test/test_dash --catch_system_errors=no -l test_suite
else else
${TEST_RUNNER_ENV} make $MAKEJOBS check VERBOSE=1 make $MAKEJOBS check VERBOSE=1
fi fi
fi fi

1
ci/test/00_setup_env.sh
View File

@ -43,7 +43,6 @@ export RUN_SECURITY_TESTS=${RUN_SECURITY_TESTS:-false}
# This is needed because some ci machines have slow CPU or disk, so sanitizers # This is needed because some ci machines have slow CPU or disk, so sanitizers
# might be slow or a reindex might be waiting on disk IO. # might be slow or a reindex might be waiting on disk IO.
export TEST_RUNNER_TIMEOUT_FACTOR=${TEST_RUNNER_TIMEOUT_FACTOR:-4} export TEST_RUNNER_TIMEOUT_FACTOR=${TEST_RUNNER_TIMEOUT_FACTOR:-4}
export TEST_RUNNER_ENV=${TEST_RUNNER_ENV:-}
export RUN_FUZZ_TESTS=${RUN_FUZZ_TESTS:-false} export RUN_FUZZ_TESTS=${RUN_FUZZ_TESTS:-false}
export EXPECTED_TESTS_DURATION_IN_SECONDS=${EXPECTED_TESTS_DURATION_IN_SECONDS:-1000} export EXPECTED_TESTS_DURATION_IN_SECONDS=${EXPECTED_TESTS_DURATION_IN_SECONDS:-1000}

4
ci/test/00_setup_env_native_multiprocess.sh
View File

@ -11,5 +11,5 @@ export PACKAGES="cmake python3 llvm clang"
export DEP_OPTS="DEBUG=1 MULTIPROCESS=1" export DEP_OPTS="DEBUG=1 MULTIPROCESS=1"
export GOAL="install" export GOAL="install"
export TEST_RUNNER_EXTRA="--v2transport" export TEST_RUNNER_EXTRA="--v2transport"
export BITCOIN_CONFIG="--with-boost-process --enable-debug CC=clang CXX=clang++" # Use clang to avoid OOM export BITCOIN_CONFIG="--with-boost-process --enable-debug CC=clang-16 CXX=clang++-16" # Use clang to avoid OOM
export TEST_RUNNER_ENV="BITCOIND=dash-node" export BITCOIND=dash-node # Used in functional tests

1
ci/test/00_setup_env_s390x.sh
View File

@ -19,7 +19,6 @@ fi
# Use debian to avoid 404 apt errors # Use debian to avoid 404 apt errors
export CONTAINER_NAME=ci_s390x export CONTAINER_NAME=ci_s390x
export RUN_UNIT_TESTS=true export RUN_UNIT_TESTS=true
export TEST_RUNNER_ENV="LC_ALL=C"
export RUN_FUNCTIONAL_TESTS=true export RUN_FUNCTIONAL_TESTS=true
export GOAL="install" export GOAL="install"
export BITCOIN_CONFIG="--enable-reduce-exports --disable-gui-tests --with-boost-process" # GUI tests disabled for now, see https://github.com/bitcoin/bitcoin/issues/23730 export BITCOIN_CONFIG="--enable-reduce-exports --disable-gui-tests --with-boost-process" # GUI tests disabled for now, see https://github.com/bitcoin/bitcoin/issues/23730

12
depends/packages/qt.mk
View File

@ -159,9 +159,15 @@ $(package)_config_opts_linux += -dbus-runtime
ifneq ($(LTO),) ifneq ($(LTO),)
$(package)_config_opts_linux += -ltcg $(package)_config_opts_linux += -ltcg
endif endif
$(package)_config_opts_linux += -platform linux-g++ -xplatform bitcoin-linux-g++
ifneq (,$(findstring -stdlib=libc++,$($(1)_cxx))) ifneq (,$(findstring clang,$($(package)_cxx)))
$(package)_config_opts_x86_64_linux = -xplatform linux-clang-libc++ ifneq (,$(findstring -stdlib=libc++,$($(package)_cxx)))
$(package)_config_opts_linux += -platform linux-clang-libc++ -xplatform linux-clang-libc++
else
$(package)_config_opts_linux += -platform linux-clang -xplatform linux-clang
endif
else
$(package)_config_opts_linux += -platform linux-g++ -xplatform bitcoin-linux-g++
endif endif
$(package)_config_opts_mingw32 = -no-opengl $(package)_config_opts_mingw32 = -no-opengl