Merge #5957: backport: bitcoin#19179, #19983, #20451, #20543, #20545, #20572, #20615, #20658, #20680, #20682, #20691, #20697, #20817, partial #20451

ae74ad09fb Merge #20817: lint: update list of spelling linter false positives, bump to codespell 2.0.0 (fanquake)
ab430bdb81 partial Merge #20451: lint: run mypy over contrib/devtools (Wladimir J. van der Laan)
3231ad2255 Merge #19983: Drop some TSan suppressions (MarcoFalke)
802cb9521f Merge #20697: ci: Fix COMMIT_RANGE variable value for PRs (MarcoFalke)
53ca879837 Merge #20682: ci: Install missing lint packages (MarcoFalke)
20219690a8 chore: drop travis mentioning in docs and comments (Konstantin Akimov)
8daef64f04 Merge #20691: ci, doc: Travis CI features and mentions cleanup (MarcoFalke)
8694479024 Merge #20680: ci: Only use credits for pull requests to the main repo (MarcoFalke)
9d824de302 Merge #20658: ci: Move linter task to cirrus (MarcoFalke)
955fc41375 Merge #20615: cirrus: Schedule one task with paid credits for faster CI feedback (Wladimir J. van der Laan)
5d66d57032 Merge #20572: ci: Adjust Cirrus CI task names (follow up) (MarcoFalke)
d11e379ed4 Merge #20545: ci: Adjust cirrus ci task names (MarcoFalke)
2fa526bb81 Merge #20543: ci: no-longer exclude feature_block in TSAN job (MarcoFalke)
fcb4c20802 Merge #19179: ci: Run ci configs on cirrus (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Backports from bitcoin v22, mostly CI related, other to improve lints
  CI related changes are not really important because we use gitlab runner instead cirrus, but keep them up to date for sake of codebase unification.

  ## What was done?
   - bitcoin/bitcoin#19179
   - bitcoin/bitcoin#20543
   - bitcoin/bitcoin#20545
   - bitcoin/bitcoin#20572
   - bitcoin/bitcoin#20615
   - bitcoin/bitcoin#20658
   - bitcoin/bitcoin#20680
   - bitcoin/bitcoin#20691
   - bitcoin/bitcoin#20682
   - bitcoin/bitcoin#20697
   - bitcoin/bitcoin#19983
   - partial bitcoin/bitcoin#20451
   - bitcoin/bitcoin#20817

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

Top commit has no ACKs.

Tree-SHA512: 5faffe15a75b78a9ea32b49f2371d6ff70c319a983c7a2b4ca6792ba3ea03f2170bacf5c255151da650948ad279c456475151d0db7dcd708eae540b30d88a05e
This commit is contained in:
pasta 2024-03-27 12:40:21 -05:00
commit 2a44f2ec64
No known key found for this signature in database
GPG Key ID: 52527BEDABE87984
23 changed files with 135 additions and 328 deletions

View File

@ -1,12 +1,5 @@
### Global defaults
timeout_in: 120m # https://cirrus-ci.org/faq/#instance-timed-out
container:
# https://cirrus-ci.org/faq/#are-there-any-limits
# Each project has 16 CPU in total, assign 2 to each container, so that 8 tasks run in parallel
cpu: 2
memory: 8G # Set to 8GB to avoid OOM. https://cirrus-ci.org/guide/linux/#linux-containers
kvm: true # Use kvm to avoid spurious CI failures in the default virtualization cluster, see https://github.com/bitcoin/bitcoin/issues/20093
env:
PACKAGE_MANAGER_INSTALL: "apt-get update && apt-get install -y"
MAKEJOBS: "-j4"
@ -16,11 +9,27 @@ env:
CCACHE_SIZE: "200M"
CCACHE_DIR: "/tmp/ccache_dir"
### Global task template
# https://cirrus-ci.org/guide/tips-and-tricks/#sharing-configuration-between-tasks
global_task_template: &GLOBAL_TASK_TEMPLATE
base_template: &BASE_TEMPLATE
skip: $CIRRUS_REPO_FULL_NAME == "bitcoin-core/gui" && $CIRRUS_PR == "" # No need to run on the read-only mirror, unless it is a PR. https://cirrus-ci.org/guide/writing-tasks/#conditional-task-execution
merge_base_script:
- if [ "$CIRRUS_PR" = "" ]; then exit 0; fi
- bash -c "$PACKAGE_MANAGER_INSTALL git"
- git fetch $CIRRUS_REPO_CLONE_URL $CIRRUS_BASE_BRANCH
- git config --global user.email "ci@ci.ci"
- git config --global user.name "ci"
- git merge FETCH_HEAD # Merge base to detect silent merge conflicts
stateful: false # https://cirrus-ci.org/guide/writing-tasks/#stateful-tasks
global_task_template: &GLOBAL_TASK_TEMPLATE
<< : *BASE_TEMPLATE
timeout_in: 120m # https://cirrus-ci.org/faq/#instance-timed-out
container:
# https://cirrus-ci.org/faq/#are-there-any-limits
# Each project has 16 CPU in total, assign 2 to each container, so that 8 tasks run in parallel
cpu: 2
memory: 8G # Set to 8GB to avoid OOM. https://cirrus-ci.org/guide/linux/#linux-containers
kvm: true # Use kvm to avoid spurious CI failures in the default virtualization cluster, see https://github.com/bitcoin/bitcoin/issues/20093
ccache_cache:
folder: "/tmp/ccache_dir"
depends_built_cache:
@ -29,19 +38,29 @@ global_task_template: &GLOBAL_TASK_TEMPLATE
folder: "/tmp/cirrus-ci-build/depends/sdk-sources"
depends_releases_cache:
folder: "/tmp/cirrus-ci-build/releases"
merge_base_script:
- if [ "$CIRRUS_PR" = "" ]; then exit 0; fi
- bash -c "$PACKAGE_MANAGER_INSTALL git"
- git fetch $CIRRUS_REPO_CLONE_URL $CIRRUS_BASE_BRANCH
- git config --global user.email "ci@ci.ci"
- git config --global user.name "ci"
- git merge FETCH_HEAD # Merge base to detect silent merge conflicts
ci_script:
- ./ci/test_run_all.sh
compute_credits_template: &CREDITS_TEMPLATE
# https://cirrus-ci.org/pricing/#compute-credits
# Only use credits for pull requests to the main repo
use_compute_credits: $CIRRUS_REPO_FULL_NAME == 'dashpay/dash' && $CIRRUS_PR != ""
task:
name: 'ARM [GOAL: install] [buster] [unit tests, no functional tests]'
name: 'lint [bionic]'
<< : *BASE_TEMPLATE
container:
image: ubuntu:bionic # For python 3.6, oldest supported version according to doc/dependencies.md
cpu: 1
memory: 1G
# For faster CI feedback, immediately schedule the linters
<< : *CREDITS_TEMPLATE
lint_script:
- ./ci/lint_run_all.sh
task:
name: 'ARM [unit tests, no functional tests] [buster]'
<< : *GLOBAL_TASK_TEMPLATE
arm_container:
image: debian:buster
@ -52,15 +71,26 @@ task:
QEMU_USER_CMD: "" # Disable qemu and run the test natively
task:
name: 'Win64 [GOAL: deploy] [unit tests, no gui, no boost::process, no functional tests]'
name: 'Win64 [unit tests, no gui tests, no boost::process, no functional tests] [bionic]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:bionic
image: ubuntu:focal
env:
FILE_ENV: "./ci/test/00_setup_env_win64.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [previous releases, uses qt5 dev package and some depends packages] [unsigned char]'
name: '32-bit + dash [gui] [CentOS 8]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: centos:8
env:
PACKAGE_MANAGER_INSTALL: "yum install -y"
FILE_ENV: "./ci/test/00_setup_env_i686_centos.sh"
task:
name: '[previous releases, uses qt5 dev package and some depends packages] [unsigned char] [focal]'
# For faster CI feedback, immediately schedule a task that compiles most modules
<< : *CREDITS_TEMPLATE
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
@ -68,18 +98,18 @@ task:
FILE_ENV: "./ci/test/00_setup_env_native_qt5.sh"
task:
name: 'x86_64 Linux [GOAL: install] [lunar] [depends, sanitizers: thread (TSan), no gui]'
name: '[depends, sanitizers: thread (TSan), no gui] [lunar]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:lunar
cpu: 4 # Double CPU and Memory to avoid timeout
memory: 16G
cpu: 4 # Double CPU and increase Memory to avoid timeout
memory: 24G
env:
MAKEJOBS: "-j8"
FILE_ENV: "./ci/test/00_setup_env_native_tsan.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [depends, sanitizers: memory (MSan)]'
name: '[depends, sanitizers: memory (MSan)] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
@ -87,7 +117,7 @@ task:
FILE_ENV: "./ci/test/00_setup_env_native_msan.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [no depends, only system libs, sanitizers: address/leak (ASan + LSan) + undefined (UBSan) + integer]'
name: '[no depends, sanitizers: address/leak (ASan + LSan) + undefined (UBSan) + integer] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
@ -95,7 +125,7 @@ task:
FILE_ENV: "./ci/test/00_setup_env_native_asan.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [no depends, only system libs, sanitizers: fuzzer,address,undefined]'
name: '[no depends, sanitizers: fuzzer,address,undefined] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
@ -103,7 +133,7 @@ task:
FILE_ENV: "./ci/test/00_setup_env_native_fuzz.sh"
task:
name: 'x86_64 Linux [GOAL: install] [focal] [multiprocess]'
name: '[multiprocess] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
@ -111,7 +141,15 @@ task:
FILE_ENV: "./ci/test/00_setup_env_native_multiprocess.sh"
task:
name: 'macOS 10.12 [GOAL: deploy] [no functional tests]'
name: '[no wallet] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
FILE_ENV: "./ci/test/00_setup_env_native_nowallet.sh"
task:
name: 'macOS 10.12 [gui, no tests] [bionic]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
@ -119,7 +157,7 @@ task:
FILE_ENV: "./ci/test/00_setup_env_mac.sh"
task:
name: 'macOS 10.14 native [GOAL: install] [GUI] [no depends]'
name: 'macOS 10.15 native [gui] [no depends]'
macos_brew_addon_script:
- brew install boost libevent berkeley-db4 qt miniupnpc ccache zeromq qrencode sqlite libtool automake pkg-config gnu-getopt
<< : *GLOBAL_TASK_TEMPLATE

View File

@ -1,191 +0,0 @@
# errata:
# - sudo/dist/group are set so as to get Blue Box VMs, necessary for [loopback]
# IPv6 support
version: ~> 1.0
dist: bionic
os: linux
language: minimal
arch: amd64
services:
- docker
cache:
directories:
- $BASE_BUILD_DIR/releases/$HOST
env:
global:
# DOCKER_HUB_USER
- secure: "J0T+zrRBBzBnxoioCD378cRSJUF8Vms32QMmrENVVfuJl03oBQv0QxSatLEiD4PtRPrN00mJ8/k/HdqFQ0SN1yg6a00duOZ6C9jk6t/fExBO1LNhp3Z7lcaMUrNEyRN6sgfVeznDVTGj9jBQU2HNlwHk0UD9BNp/k+Bjhrtw7Njp9JTLZKBgxrXk0WZmfk0d75Q+9DajA1j2tSfWVpwzv6HMGfqIH1wsVEakBY+tnaE6EO3IdzdWHhf/iV+Tx0tL0Qz/6mejgSuoGLTJLmeTrceTTmsDlN3B95y+XfcYDs6EP7gpzN8Ffg/mLV8xJ2Myj7+b1o+smMJZE5tLuPIeAu0YsfkPNa/tNLotOlDxF72per0kXyNYbRf+JrpxiOu9pLCtdtd40V2SsiNhpQqHVy+b+wkkqo2syTHT+9UNkqEi3WFm6TqFuOeQNtDm6R0hRl/Talw/ZEeKs68z1Cf6VyBrZG5LqtOurtWlKwXCe/Tr1i2g4BEAcbywphtCCcfgMK9YSGQnjKkGk6k9gVEa2bneYLZz1RSh2OpXnIsIJOX7X0hNd5xtQDLgbpSUlKwVcEriWeGe12GCszKNX9gdEo2bHeq4xAJ6BE+ot745z9dc+nC2BJ6Su+I5L6zBg4m2FWn0TWhPvz/6JR26t6ls/E7m+hkwJTCHnIUiFmmzpEw="
# DOCKER_HUB_PASSWORD
- secure: "RLzlMhfLqwSBrZqJOVOd61suXBn+HtUR3vOZfuFYF/Qmjjj5TE41+rObZmzc54hP/ZL+OH6blnibpvfDXlX+eN38ivFQfuxkJIGL68SJsEwNBRwW39Yw6Hl5RdI41MLCH7ByN15wifLp1JKBilHJ7XGMOUjI7P0yl7JjX8GBXUCtJbVLRugo80/yn+XQ1NdnlpbpYHNjMEQFWSODPa3pSK5McWvyQjDZDgS+IkdrZmIYJPMa7bmKH5I/edUPSmXQT905FgEwq9u8XR0SyBopli21EK9l6GkXIIvmDTYz5vT26Apvi2B4Aoazlklg+KNRUJuLGerpt6kbnU0gMSUChVkFfFhOk6GRSN3a/AUfD2FOudvMhet2QvlPHx+GYdEUr5XVo5HW42pHsqfD6eDtHd8VLTsHP0q4C8V85fNMv21lkkehy2ry8fx/RRy6x4O2wg2mua+79UkGKXp75gMKzWEcth34PCFCOu37l2F8R/ANnrQ52K/8vIQ88TtU2OpYX89fHjLojBxu+WKEBGZH2LRPsZBOUHeeO5C/xKDrhZU24ORnMW8wg66Qg5GIX1KI4a8yp73Mpues5hzpJ6wkMuRrQt40ymKndLCjv8KSd+5BfP6Or/KIrzDNYdZaasjk7JNi6rcZmm9d3fTAo+Ja/mjpUCIOo3SX14luzVCJIig="
- DOCKER_BUILD=false
stages:
- build depends
- build src
- run tests
- build docker
builddepends: &builddepends
stage: build depends
script:
- $DOCKER_RUN_IN_BUILDER ./ci/dash/build_depends.sh
buildsrc: &buildsrc
stage: build src
script:
- $DOCKER_RUN_IN_BUILDER ./ci/dash/build_depends.sh
- $DOCKER_RUN_IN_BUILDER ./ci/dash/build_src.sh
runtests: &runtests
stage: run tests
script:
- $DOCKER_RUN_IN_BUILDER ./ci/dash/build_depends.sh
- $DOCKER_RUN_IN_BUILDER ./ci/dash/build_src.sh
- $DOCKER_RUN_IN_BUILDER ./ci/dash/test_unittests.sh
- if [ "$TRAVIS_EVENT_TYPE" = "cron" ]; then extended="--extended --exclude feature_pruning,feature_dbcrash"; fi
- $DOCKER_RUN_IN_BUILDER ./ci/test_integrationtests.sh --quiet --jobs=3 ${extended}
builddocker: &builddocker
stage: build docker
script:
# no need to run tests again here
- if [ "$DOCKER_BUILD" = "true" ]; then $DOCKER_RUN_IN_BUILDER ./ci/dash/build_depends.sh && $DOCKER_RUN_IN_BUILDER ./ci/dash/build_src.sh && BUILD_DIR=build-ci/dashcore-$BUILD_TARGET ./ci/dash/build-docker.sh; fi
- CI_RETRY_EXE="travis_retry"
- CACHE_ERR_MSG="Error! Initial build successful, but not enough time remains to run later build stages and tests. See https://docs.travis-ci.com/user/customizing-the-build#build-timeouts . Please manually re-run this job by using the travis restart button. The next run should not time out because the build cache has been saved."
before_install:
- set -o errexit; source ./ci/test/00_setup_env.sh
- set -o errexit; source ./ci/test/03_before_install.sh
install:
- set -o errexit; source ./ci/test/04_install.sh
before_script:
# Temporary workaround for https://github.com/bitcoin/bitcoin/issues/16368
- for i in {1..4}; do echo "$(sleep 500)" ; done &
- set -o errexit; source ./ci/test/05_before_script.sh &> "/dev/null"
script:
- if [[ $SECONDS -gt 1200 ]]; then set +o errexit; echo "Travis early exit to cache current state"; false; else set -o errexit; source .travis/test_06_script.sh; fi
after_script:
- echo $TRAVIS_COMMIT_RANGE
jobs:
include:
# build depends
- <<: *builddepends
name: depends-arm-linux
env: BUILD_TARGET=arm-linux
- <<: *builddepends
name: depends-win64
env: BUILD_TARGET=win64
- <<: *builddepends
name: depends-linux64
env: BUILD_TARGET=linux64
- <<: *builddepends
name: depends-linux64_nowallet
env: BUILD_TARGET=linux64_nowallet
- <<: *builddepends
name: depends-linux64_release
env:
- BUILD_TARGET=linux64_release
- DOCKER_BUILD=true
- <<: *builddepends
name: depends-mac
env: BUILD_TARGET=mac
# build source
- <<: *buildsrc
name: src-arm-linux
env: BUILD_TARGET=arm-linux
- <<: *buildsrc
name: src-win64
env: BUILD_TARGET=win64
- <<: *buildsrc
name: src-linux64
env: BUILD_TARGET=linux64
- <<: *buildsrc
name: src-linux64_nowallet
env: BUILD_TARGET=linux64_nowallet
- <<: *buildsrc
name: src-linux64_release
env:
- BUILD_TARGET=linux64_release
- DOCKER_BUILD=true
- <<: *buildsrc
name: src-mac
env: BUILD_TARGET=mac
# run tests (no tests for arm-linux and mac)
- <<: *runtests
name: tests-win64
env: BUILD_TARGET=win64
- <<: *runtests
name: tests-linux64
env: BUILD_TARGET=linux64
- <<: *runtests
name: tests-linux64_nowallet
env: BUILD_TARGET=linux64_nowallet
- <<: *runtests
name: tests-linux64_release
env:
- BUILD_TARGET=linux64_release
- DOCKER_BUILD=true
# build docker
- <<: *builddocker
name: docker-linux64_release
env:
- BUILD_TARGET=linux64_release
- DOCKER_BUILD=true
before_cache:
# Save builder image
- docker save dash-builder-$BUILD_TARGET-$TRAVIS_JOB_NUMBER $(docker history -q dash-builder-$BUILD_TARGET-$TRAVIS_JOB_NUMBER | grep -v \<missing\>) | gzip -2 > $HOME/cache/docker/dash-builder-$BUILD_TARGET.tar.gz
install:
# Fix annoying Travis bug: a branch with a single commit has an empty TRAVIS_COMMIT_RANGE sometimes
- if [ -z "$TRAVIS_COMMIT_RANGE" ]; then export TRAVIS_COMMIT_RANGE="HEAD~..HEAD"; fi
# Our scripts try to be Travis agnostic
- export PULL_REQUEST="$TRAVIS_PULL_REQUEST"
- export COMMIT_RANGE="$TRAVIS_COMMIT_RANGE"
- export JOB_NUMBER="$TRAVIS_JOB_NUMBER"
- export HOST_SRC_DIR=$BASE_BUILD_DIR
- export HOST_CACHE_DIR=$HOME/cache
- export TRAVIS_COMMIT_LOG=`git log --format=fuller -1`
- source ./ci/matrix.sh
- mkdir -p $HOST_CACHE_DIR/docker && mkdir -p $HOST_CACHE_DIR/ccache && mkdir -p $HOST_CACHE_DIR/depends && mkdir -p $HOST_CACHE_DIR/sdk-sources
# Keep this as it makes caching related debugging easier
- ls -lah $HOST_CACHE_DIR && ls -lah $HOST_CACHE_DIR/depends && ls -lah $HOST_CACHE_DIR/ccache && ls -lah $HOST_CACHE_DIR/docker
# Load cached builder image
- if [ -f $HOST_CACHE_DIR/docker/dash-builder-$BUILD_TARGET.tar.gz ]; then zcat $HOST_CACHE_DIR/docker/dash-builder-$BUILD_TARGET.tar.gz | docker load || true; fi
- travis_retry docker pull ubuntu:focal
- travis_retry docker build -t $BUILDER_IMAGE_NAME --build-arg=USER_ID=$UID --build-arg=GROUP_ID=$UID --build-arg=BUILD_TARGET=$BUILD_TARGET -f contrib/containers/ci/Dockerfile ci
before_script:
# Make sure stdout is in blocking mode. Otherwise builds will fail due to large writes to stdout
# See https://github.com/travis-ci/travis-ci/issues/4704. If this gets fixed, this line can also be removed.
- python3 -c 'import os,sys,fcntl; flags = fcntl.fcntl(sys.stdout, fcntl.F_GETFL); fcntl.fcntl(sys.stdout, fcntl.F_SETFL, flags&~os.O_NONBLOCK);'
# Build docker image only for develop branch of the main repo
- if [ "$TRAVIS_REPO_SLUG" != "dashpay/dash" -o "$TRAVIS_BRANCH" != "develop" -o "$TRAVIS_PULL_REQUEST" != "false" ]; then export DOCKER_BUILD="false"; echo DOCKER_BUILD=$DOCKER_BUILD; fi
# TODO(ignore if don't use travis): Check keys and signed commits
#- if [ "$TRAVIS_REPO_SLUG" = "dashpay/dash" -a "$TRAVIS_PULL_REQUEST" = "false" ]; then travis_retry gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $(<contrib/verify-commits/trusted-keys); fi
#- if [ "$TRAVIS_REPO_SLUG" = "dashpay/dash" -a "$TRAVIS_EVENT_TYPE" = "cron" ]; then travis_wait 30 contrib/verify-commits/verify-commits.py; fi
after_script:
- echo $TRAVIS_COMMIT_RANGE
- echo $TRAVIS_COMMIT_LOG
after_success:
- if [ "$DOCKER_BUILD" = "true" ]; then docker login -u $DOCKER_HUB_USER -p $DOCKER_HUB_PASSWORD && ./ci/dash/push-docker.sh; fi
- stage: lint
name: 'lint'
env:
cache: pip
language: python
python: '3.8' # Oldest supported version according to doc/dependencies.md
install:
- set -o errexit; source ./ci/lint/04_install.sh
before_script:
- set -o errexit; source ./ci/lint/05_before_script.sh
script:
- set -o errexit; source ./ci/lint/06_script.sh
- stage: test
name: 'x86_64 Linux [GOAL: install] [focal] [no wallet]'
env: >-
FILE_ENV="./ci/test/00_setup_env_native_nowallet.sh"

View File

@ -62,7 +62,8 @@ There are also [regression and integration tests](/test), written
in Python.
These tests can be run (if the [test dependencies](/test) are installed) with: `test/functional/test_runner.py`
The Travis CI system makes sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.
The CI (Continuous Integration) systems make sure that every pull request is built for Windows, Linux, and macOS,
and that unit/sanity tests are run automatically.
### Manual Quality Assurance (QA) Testing

View File

@ -6,15 +6,16 @@
export LC_ALL=C
travis_retry sudo apt update && sudo apt install -y clang-format-9
sudo update-alternatives --install /usr/bin/clang-format clang-format $(which clang-format-9 ) 100
sudo update-alternatives --install /usr/bin/clang-format-diff clang-format-diff $(which clang-format-diff-9) 100
${CI_RETRY_EXE} apt-get update
${CI_RETRY_EXE} apt-get install -y clang-format-9 python3-pip curl git gawk jq
update-alternatives --install /usr/bin/clang-format clang-format $(which clang-format-9 ) 100
update-alternatives --install /usr/bin/clang-format-diff clang-format-diff $(which clang-format-diff-9) 100
travis_retry pip3 install codespell==1.17.1
travis_retry pip3 install flake8==3.8.3
travis_retry pip3 install vulture==2.3
travis_retry pip3 install yq
travis_retry pip3 install mypy==0.781
${CI_RETRY_EXE} pip3 install codespell==2.0.0
${CI_RETRY_EXE} pip3 install flake8==3.8.3
${CI_RETRY_EXE} pip3 install vulture==2.3
${CI_RETRY_EXE} pip3 install yq
${CI_RETRY_EXE} pip3 install mypy==0.781
SHELLCHECK_VERSION=v0.7.1
curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | tar --xz -xf - --directory /tmp/

View File

@ -1,9 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
git fetch --unshallow

View File

@ -6,13 +6,12 @@
export LC_ALL=C
if [ "$TRAVIS_EVENT_TYPE" = "pull_request" ]; then
# TRAVIS_BRANCH will be present in a Travis environment. For builds triggered
# by a pull request this is the name of the branch targeted by the pull request.
# https://docs.travis-ci.com/user/environment-variables/
COMMIT_RANGE="$TRAVIS_BRANCH..HEAD"
GIT_HEAD=$(git rev-parse HEAD)
if [ -n "$CIRRUS_PR" ]; then
COMMIT_RANGE="$CIRRUS_BASE_SHA..$GIT_HEAD"
test/lint/commit-script-check.sh $COMMIT_RANGE
fi
export COMMIT_RANGE
# This only checks that the trees are pure subtrees, it is not doing a full
# check with -r to not have to fetch all the remotes.
@ -24,8 +23,11 @@ test/lint/check-doc.py
test/lint/check-rpc-mappings.py .
test/lint/lint-all.sh
if [ "$TRAVIS_REPO_SLUG" = "bitcoin/bitcoin" ] && [ "$TRAVIS_EVENT_TYPE" = "cron" ]; then
if [ "$CIRRUS_REPO_FULL_NAME" = "dashpay/dash" ] && [ -n "$CIRRUS_CRON" ]; then
git log --merges --before="2 days ago" -1 --format='%H' > ./contrib/verify-commits/trusted-sha512-root-commit
travis_retry gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $(<contrib/verify-commits/trusted-keys) &&
${CI_RETRY_EXE} gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $(<contrib/verify-commits/trusted-keys) &&
./contrib/verify-commits/verify-commits.py --clean-merge=2;
fi
echo
git log --no-merges --oneline $COMMIT_RANGE

11
ci/lint_run_all.sh Executable file
View File

@ -0,0 +1,11 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
set -o errexit; source ./ci/test/00_setup_env.sh
set -o errexit; source ./ci/lint/04_install.sh
set -o errexit; source ./ci/lint/06_script.sh

View File

@ -1,22 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
BEGIN_FOLD () {
echo ""
CURRENT_FOLD_NAME=$1
echo "travis_fold:start:${CURRENT_FOLD_NAME}"
}
END_FOLD () {
RET=$?
echo "travis_fold:end:${CURRENT_FOLD_NAME}"
if [ $RET != 0 ]; then
echo "${CURRENT_FOLD_NAME} failed with status code ${RET}"
fi
}

View File

@ -6,9 +6,6 @@
export LC_ALL=C.UTF-8
if [[ $DOCKER_NAME_TAG == centos* ]]; then
export LC_ALL=en_US.utf8
fi
if [[ $QEMU_USER_CMD == qemu-s390* ]]; then
export LC_ALL=C
fi

View File

@ -29,14 +29,12 @@ if [ -z "$NO_DEPENDS" ]; then
# CentOS has problems building the depends if the config shell is not explicitly set
# (i.e. for libevent a Makefile with an empty SHELL variable is generated, leading to
# an error as the first command is executed)
SHELL_OPTS="CONFIG_SHELL=/bin/bash"
SHELL_OPTS="LC_ALL=en_US.UTF-8 CONFIG_SHELL=/bin/bash"
else
SHELL_OPTS="CONFIG_SHELL="
fi
DOCKER_EXEC $SHELL_OPTS make $MAKEJOBS -C depends HOST=$HOST $DEP_OPTS
fi
if [ -n "$PREVIOUS_RELEASES_TO_DOWNLOAD" ]; then
BEGIN_FOLD previous-versions
DOCKER_EXEC test/get_previous_releases.py -b -t "$PREVIOUS_RELEASES_DIR" "${PREVIOUS_RELEASES_TO_DOWNLOAD}"
END_FOLD
fi

View File

@ -7,6 +7,5 @@
export LC_ALL=C.UTF-8
set -o errexit; source ./ci/test/00_setup_env.sh
set -o errexit; source ./ci/test/03_before_install.sh
set -o errexit; source ./ci/test/04_install.sh
set -o errexit; source ./ci/test/05_before_script.sh

View File

@ -1,6 +1,6 @@
FROM phusion/baseimage:focal-1.0.0
LABEL maintainer="Dash Developers <dev@dash.org>"
LABEL description="Dockerised DashCore, built from Travis"
LABEL description="Dockerised DashCore, built from CI"
ARG USER_ID
ARG GROUP_ID

View File

@ -1,8 +1,8 @@
This docker image is built and pushed from travis. It does not contain any default configs or any
This docker image is built and pushed from CI. It does not contain any default configs or any
special startup scripts. This is to keep it simple and to not copy too much logic from the
official/supported Docker Images found at https://hub.docker.com/r/dashpay/dashd/
The image is mostly used for devops purposes, e.g. for testnet/devnet deployments. Regular users
should use the official images instead of this one.
**NOTE: Please update the image description at https://hub.docker.com/r/dashpay/dashd-develop/ when changing the above text**
**NOTE: Please update the image description at https://hub.docker.com/r/dashpay/dashd-develop/ when changing the above text**

View File

@ -6,6 +6,7 @@
import sys
import re
from multiprocess import Pool
from typing import Dict, List, Set
MAPPING = {
'core_read.cpp': 'core_io.cpp',
@ -34,7 +35,7 @@ def module_name(path):
if __name__=="__main__":
files = dict()
deps = dict()
deps: Dict[str, Set[str]] = dict()
RE = re.compile("^#include <(.*)>")
@ -48,7 +49,7 @@ if __name__=="__main__":
def handle_module2(module):
# Build the transitive closure of dependencies of module
closure = dict()
closure: Dict[str, List[str]] = dict()
for dep in deps[module]:
closure[dep] = []
while True:
@ -90,7 +91,7 @@ if __name__=="__main__":
# Loop to find the shortest (remaining) circular dependency
def shortest_c_dep():
have_cycle = False
have_cycle: bool = False
sorted_keys = None

View File

@ -11,7 +11,7 @@ Example usage:
find ../path/to/binaries -type f -executable | xargs python3 contrib/devtools/symbol-check.py
'''
import sys
from typing import Dict
from typing import Dict, List
import lief
@ -173,7 +173,7 @@ def check_version(max_versions, version, arch) -> bool:
return ver <= max_versions[lib][arch]
def check_imported_symbols(binary) -> bool:
ok = True
ok: bool = True
for symbol in binary.imported_symbols:
if not symbol.imported:
@ -189,7 +189,7 @@ def check_imported_symbols(binary) -> bool:
return ok
def check_exported_symbols(binary) -> bool:
ok = True
ok: bool = True
for symbol in binary.dynamic_symbols:
if not symbol.exported:
@ -202,7 +202,7 @@ def check_exported_symbols(binary) -> bool:
return ok
def check_ELF_libraries(binary) -> bool:
ok = True
ok: bool = True
for library in binary.libraries:
if library not in ELF_ALLOWED_LIBRARIES:
print(f'{filename}: {library} is not in ALLOWED_LIBRARIES!')
@ -210,7 +210,7 @@ def check_ELF_libraries(binary) -> bool:
return ok
def check_MACHO_libraries(binary) -> bool:
ok = True
ok: bool = True
for dylib in binary.libraries:
split = dylib.name.split('/')
if split[-1] not in MACHO_ALLOWED_LIBRARIES:
@ -229,7 +229,7 @@ def check_MACHO_sdk(binary) -> bool:
return False
def check_PE_libraries(binary) -> bool:
ok = True
ok: bool = True
for dylib in binary.libraries:
if dylib not in PE_ALLOWED_LIBRARIES:
print(f'{dylib} is not in ALLOWED_LIBRARIES!')
@ -267,7 +267,7 @@ lief.EXE_FORMATS.PE: [
}
if __name__ == '__main__':
retval = 0
retval: int = 0
for filename in sys.argv[1:]:
try:
binary = lief.parse(filename)
@ -277,7 +277,7 @@ if __name__ == '__main__':
retval = 1
continue
failed = []
failed: List[str] = []
for (name, func) in CHECKS[etype]:
if not func(binary):
failed.append(name)

View File

@ -102,7 +102,7 @@ def main():
# Iterate through commits
while True:
# Log a message to prevent Travis from timing out
# Log a message to prevent CI from timing out
logging.debug("verify-commits: [in-progress] processing commit {}".format(current_commit[:8]))
if current_commit == verified_root:

View File

@ -621,7 +621,7 @@ General Dash Core
- *Rationale*: RPC allows for better automatic testing. The test suite for
the GUI is very limited.
- Make sure pull requests pass Travis CI before merging.
- Make sure pull requests pass CI before merging.
- *Rationale*: Makes sure that they pass thorough testing, and that the tester will keep passing
on the master branch. Otherwise, all new pull requests will start failing the tests, resulting in
@ -1068,7 +1068,7 @@ Scripted diffs
--------------
For reformatting and refactoring commits where the changes can be easily automated using a bash script, we use
scripted-diff commits. The bash script is included in the commit message and our Travis CI job checks that
scripted-diff commits. The bash script is included in the commit message and our CI job checks that
the result of the script is identical to the commit. This aids reviewers since they can verify that the script
does exactly what it is supposed to do. It is also helpful for rebasing (since the same script can just be re-run
on the new master commit).

View File

@ -310,7 +310,7 @@ Use the `-v` option for verbose output.
| [`lint-python.sh`](lint/lint-python.sh) | [mypy](https://github.com/python/mypy) | [0.781](https://github.com/bitcoin/bitcoin/pull/19348) | `pip3 install mypy==0.781`
| [`lint-shell.sh`](lint/lint-shell.sh) | [ShellCheck](https://github.com/koalaman/shellcheck) | [0.7.1](https://github.com/bitcoin/bitcoin/pull/19348) | [details...](https://github.com/koalaman/shellcheck#installing)
| [`lint-shell.sh`](lint/lint-shell.sh) | [yq](https://github.com/kislyuk/yq) | default | `pip3 install yq`
| [`lint-spelling.sh`](lint/lint-spelling.sh) | [codespell](https://github.com/codespell-project/codespell) | [1.17.1](https://github.com/bitcoin/bitcoin/pull/19348) | `pip3 install codespell==1.17.1`
| [`lint-spelling.sh`](lint/lint-spelling.sh) | [codespell](https://github.com/codespell-project/codespell) | [2.0.0](https://github.com/bitcoin/bitcoin/pull/20817) | `pip3 install codespell==2.0.0`
Please be aware that on Linux distributions all dependencies are usually available as packages, but could be outdated.

View File

@ -23,7 +23,7 @@ don't have test cases for.
- The oldest supported Python version is specified in [doc/dependencies.md](/doc/dependencies.md).
Consider using [pyenv](https://github.com/pyenv/pyenv), which checks [.python-version](/.python-version),
to prevent accidentally introducing modern syntax from an unsupported Python version.
The Travis linter also checks this, but [possibly not in all cases](https://github.com/bitcoin/bitcoin/pull/14884#discussion_r239585126).
The CI linter job also checks this, but [possibly not in all cases](https://github.com/bitcoin/bitcoin/pull/14884#discussion_r239585126).
- See [the python lint script](/test/lint/lint-python.sh) that checks for violations that
could lead to bugs and issues in the test code.
- Use [type hints](https://docs.python.org/3/library/typing.html) in your code to improve code readability

View File

@ -8,14 +8,6 @@
export LC_ALL=C
# The shellcheck binary segfault/coredumps in Travis with LC_ALL=C
# It does not do so in Ubuntu 14.04, 16.04, 18.04 in versions 0.3.3, 0.3.7, 0.4.6
# respectively. So export LC_ALL=C is set as required by lint-shell-locale.sh
# but unset here in case of running in Travis.
if [ "$TRAVIS" = "true" ]; then
unset LC_ALL
fi
# Disabled warnings:
disabled=(
SC2046 # Quote this to prevent word splitting.

View File

@ -1,25 +1,20 @@
hights
asend
ba
mor
mut
objext
useable
setban
wit
unparseable
copyable
blockin
cachable
errorstring
keyserver
homogenous
hist
ser
unser
creat
crypted
fo
creat
stoll
fpr
hights
hist
inout
mor
nin
ser
unselect
lowercased
setban
stoll
unparseable
unser
useable
wit

View File

@ -15,6 +15,6 @@ if ! command -v codespell > /dev/null; then
fi
IGNORE_WORDS_FILE=test/lint/lint-spelling.ignore-words.txt
if ! codespell --check-filenames --disable-colors --quiet-level=7 --ignore-words=${IGNORE_WORDS_FILE} $(git ls-files -- ":(exclude)build-aux/m4/" ":(exclude)contrib/seeds/*.txt" ":(exclude)depends/" ":(exclude)doc/release-notes/" ":(exclude)src/bip39_english.h" ":(exclude)src/dashbls/" ":(exclude)src/crc32c/" ":(exclude)src/crypto/" ":(exclude)src/ctpl_stl.h" ":(exclude)src/cxxtimer.hpp" ":(exclude)src/util/expected.h" ":(exclude)src/immer/" ":(exclude)src/leveldb/" ":(exclude)src/qt/locale/" ":(exclude)src/qt/*.qrc" ":(exclude)src/secp256k1/" ":(exclude)src/univalue/" ":(exclude)contrib/guix/patches"); then
if ! codespell --check-filenames --disable-colors --quiet-level=7 --ignore-words=${IGNORE_WORDS_FILE} $(git ls-files -- ":(exclude)build-aux/m4/" ":(exclude)contrib/seeds/*.txt" ":(exclude)depends/" ":(exclude)doc/release-notes/" ":(exclude)src/bip39_english.h" ":(exclude)src/dashbls/" ":(exclude)src/crc32c/" ":(exclude)src/crypto/" ":(exclude)src/ctpl_stl.h" ":(exclude)src/cxxtimer.hpp" ":(exclude)src/util/expected.h" ":(exclude)src/immer/" ":(exclude)src/leveldb/" ":(exclude)src/qt/locale/" ":(exclude)src/qt/*.qrc" ":(exclude)src/secp256k1/" ":(exclude)src/univalue/" ":(exclude)contrib/builder-keys/" ":(exclude)contrib/guix/patches"); then
echo "^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in ${IGNORE_WORDS_FILE}"
fi

View File

@ -33,15 +33,9 @@ race:leveldb::DBImpl::DeleteObsoleteFiles
race:zmq::*
race:bitcoin-qt
# deadlock (TODO fix)
deadlock:CConnman::ForNode
deadlock:CConnman::GetNodeStats
deadlock:CChainState::ConnectTip
deadlock:UpdateTip
deadlock:wallet_tests::CreateWallet
# WalletBatch (unidentified deadlock)
deadlock:WalletBatch
# deadlock false positive (see: https://github.com/dashpay/dash/pull/4563)
deadlock:CChainState::ConnectTip