mirror of
https://github.com/dashpay/dash.git
synced 2024-12-24 19:42:46 +01:00
Merge #6428: backport: merge bitcoin#24778, #24766, #24849, #24844, #24853, #24895, #24802, #24932, partial bitcoin#23212, #23462 (lint backports)
852f55e23c
merge bitcoin#24932: Convert lint-locale-dependence.sh to Python (Kittywhiskers Van Gogh)f745b7f7ef
merge bitcoin#24802: convert format strings linter test to python (Kittywhiskers Van Gogh)a0b051b4ef
partial revert dash#4807: enable more multi-threading and caching in linters (Kittywhiskers Van Gogh)7864c21645
merge bitcoin#24895: Convert lint-includes.sh to Python (Kittywhiskers Van Gogh)f63bafe253
merge bitcoin#24853: Convert lint-git-commit-check.sh to Python (Kittywhiskers Van Gogh)d463d7d397
fix: clone full history on GitHub Actions build job, track `develop` (Kittywhiskers Van Gogh)d23b6614e8
merge bitcoin#24844: Convert lint-whitespace.sh to Python (Kittywhiskers Van Gogh)fe165168ba
merge bitcoin#24849: Convert lint-logs.sh to Python (Kittywhiskers Van Gogh)dbfd0b04e1
merge bitcoin#24766: convert spellchecking lint test to python (Kittywhiskers Van Gogh)5c23addddd
merge bitcoin#24778: Convert Python dead code linter test to Python (Kittywhiskers Van Gogh)829dcfb936
partial bitcoin#23462: Enable SC2046 and SC2086 shellcheck rules (Kittywhiskers Van Gogh)df6be0e8c0
partial bitcoin#23212: enable mypy import checking (Kittywhiskers Van Gogh)f77eca9256
fix: make sure that parallelized `lint-all` runs incl. python scripts (Kittywhiskers Van Gogh)e220175d0f
merge bitcoin#22861: Update test README and lint script (Kittywhiskers Van Gogh) Pull request description: ## Additional Information * Dependency for https://github.com/dashpay/dash/pull/6429 * Even though support for Python scripts was extended to `lint-all` in [bitcoin#24762](f226e8dc1f
) ([dash#6023](https://github.com/dashpay/dash/pull/6023)), the changes needed were not extended to parallelized linting added in [dash#4637](https://github.com/dashpay/dash/pull/4637). This meant the match expression didn't include non-shell scripts and additionally, `parallel` interpreted all scripts as Bash scripts. This has been resolved in this PR. * [bitcoin#23212](https://github.com/bitcoin/bitcoin/pull/23212) is partial as `--ignore-missing-imports` has not been removed from `mypy` arguments as the version of `mypy` used in the PR (0.910) isn't syntax aware to `imports` like [here](f9d044d5ec/test/functional/test_framework/crypto/bip324_cipher.py (L16-L17)
) and [here](f9d044d5ec/test/functional/test_framework/crypto/muhash.py (L9)
). <details> <summary>Lint errors:</summary> ``` dash@96b217d539f7:/src/dash$ ./test/lint/lint-python.sh Consider install flake8-cached for cached flake8 results. [...] test/functional/test_framework/crypto/muhash.py:9: error: Skipping analyzing ".chacha20": found module but no type hints or library stubs [import] test/functional/test_framework/crypto/ellswift.py:15: error: Cannot find implementation or library stub for module named "test_framework.crypto.secp256k1" [import] test/functional/test_framework/crypto/bip324_cipher.py:16: error: Skipping analyzing ".chacha20": found module but no type hints or library stubs [import] test/functional/test_framework/crypto/bip324_cipher.py:17: error: Skipping analyzing ".poly1305": found module but no type hints or library stubs [import] test/functional/test_framework/messages.py:29: error: Cannot find implementation or library stub for module named "test_framework.crypto.siphash" [import] test/functional/test_framework/messages.py:32: error: Cannot find implementation or library stub for module named "dash_hash" [import] test/functional/test_framework/script.py:20: error: Cannot find implementation or library stub for module named "test_framework.crypto.ripemd160" [import] test/functional/test_framework/key.py:16: error: Cannot find implementation or library stub for module named "test_framework.crypto" [import] test/functional/test_framework/v2_p2p.py:9: error: Cannot find implementation or library stub for module named "test_framework.crypto.bip324_cipher" [import] test/functional/test_framework/v2_p2p.py:10: error: Cannot find implementation or library stub for module named "test_framework.crypto.chacha20" [import] test/functional/test_framework/v2_p2p.py:11: error: Cannot find implementation or library stub for module named "test_framework.crypto.ellswift" [import] test/functional/test_framework/v2_p2p.py:12: error: Cannot find implementation or library stub for module named "test_framework.crypto.hkdf" [import] test/functional/p2p_v2_encrypted.py:22: error: Cannot find implementation or library stub for module named "test_framework.crypto.chacha20" [import] test/functional/feature_utxo_set_hash.py:12: error: Cannot find implementation or library stub for module named "test_framework.crypto.muhash" [import] Found 17 errors in 12 files (checked 275 source files) ``` </details> And upgrading to the latest version of `mypy` used upstream (1.4.1, [source](f7144b24be/ci/lint/04_install.sh (L52)
)) brings syntax awareness but new errors. <details> <summary>Lint errors:</summary> ``` dash@96b217d539f7:/src/dash$ ./test/lint/lint-python.sh Consider install flake8-cached for cached flake8 results. [...] test/functional/test_framework/coverage.py:23: error: Incompatible default for argument "coverage_logfile" (default has type "None", argument has type "str") [assignment] test/functional/test_framework/coverage.py:23: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True test/functional/test_framework/coverage.py:23: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase test/functional/test_framework/util.py:322: error: Incompatible default for argument "timeout" (default has type "None", argument has type "int") [assignment] test/functional/test_framework/util.py:322: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True test/functional/test_framework/util.py:322: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase test/functional/test_framework/util.py:322: error: Incompatible default for argument "coveragedir" (default has type "None", argument has type "str") [assignment] test/functional/test_framework/messages.py:32: error: Cannot find implementation or library stub for module named "dash_hash" [import] test/functional/test_framework/test_framework.py:122: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] test/functional/test_framework/test_framework.py:123: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] test/functional/test_framework/test_framework.py:124: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] test/functional/test_framework/test_framework.py:125: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] test/functional/p2p_message_capture.py:48: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs [annotation-unchecked] Found 6 errors in 5 files (checked 275 source files) ``` </details> * [bitcoin#23462](https://github.com/bitcoin/bitcoin/pull/23462) is partial as neither `SC2046` nor `SC2086` have been enabled. This backport was done for the sole purpose of backporting changes to shell scripts that would be replaced with their Python counterparts in later backports. The necessary changes needed to enable `SC2046` and `SC2086` will be done in a later pull request. * `actions/checkout` does not do a full clone nor tracks the default branch by default. It is already documented that `git merge-base` (used by `lint-git-commit-check.py` and `lint-whitespace.py`, [source](4aaa314a57/test/lint/lint-git-commit-check.py (L45)
)) doesn't play nice with it (see [actions/checkout#423](https://github.com/actions/checkout/discussions/423)). No such issue exists on GitLab ([build](https://gitlab.com/dashpay/dash/-/jobs/8456260939#L113)), but it remains present on GitHub Actions ([build](https://github.com/dashpay/dash/actions/runs/11996049800/job/33440106874?pr=6428#step:7:103)). This PR does a full clone, switches to the `develop` branch, then switches back to the intended commit, as a workaround (see working build [here](https://github.com/dashpay/dash/pull/6428#issuecomment-2495980410)) * Changes to `run-lint-format-strings.py` made in [dash#4807](https://github.com/dashpay/dash/pull/4807) were reverted as they were interfering with `lint-format-strings.py` <details> <summary>Lint error:</summary> ``` dash@96b217d539f7:/src/dash$ ./test/lint/lint-format-strings.py Traceback (most recent call last): File "/src/dash/test/lint/run-lint-format-strings.py", line 308, in <module> main() File "/src/dash/test/lint/run-lint-format-strings.py", line 304, in main sys.exit(max(exit_codes)) ValueError: max() arg is an empty sequence Traceback (most recent call last): File "/src/dash/test/lint/run-lint-format-strings.py", line 308, in <module> main() File "/src/dash/test/lint/run-lint-format-strings.py", line 304, in main sys.exit(max(exit_codes)) ValueError: max() arg is an empty sequence ``` </details> ## Breaking Changes None expected. ## Checklist - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)** - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: knst: utACK852f55e23c
UdjinM6: utACK852f55e23c
Tree-SHA512: e508311c00249e7e48f91ca23e6f62117c07497f9c4471d07659b233e43a9f4d09ee2bfe0dd76f0c9746209cdba5895cb002a708924daa19d7aa76869815a7d9
This commit is contained in:
commit
eb88589ea7
5
.github/workflows/build.yml
vendored
5
.github/workflows/build.yml
vendored
@ -140,6 +140,7 @@ jobs:
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Restore depends cache
|
||||
uses: actions/cache/restore@v4
|
||||
@ -167,7 +168,11 @@ jobs:
|
||||
|
||||
- name: Build source and run tests
|
||||
run: |
|
||||
git config --global --add advice.detachedHead false
|
||||
git config --global --add safe.directory "$PWD"
|
||||
GIT_HEAD="$(git rev-parse HEAD)"
|
||||
git checkout develop
|
||||
git checkout ${GIT_HEAD}
|
||||
CCACHE_SIZE="400M"
|
||||
CACHE_DIR="/cache"
|
||||
mkdir /output
|
||||
|
@ -252,7 +252,7 @@ EXTRA_DIST += \
|
||||
test/fuzz
|
||||
|
||||
EXTRA_DIST += \
|
||||
test/util/bitcoin-util-test.py \
|
||||
test/util/test_runner.py \
|
||||
test/util/data/bitcoin-util-test.json \
|
||||
test/util/data/blanktxv1.hex \
|
||||
test/util/data/blanktxv1.json \
|
||||
|
@ -8,14 +8,14 @@ export LC_ALL=C
|
||||
|
||||
${CI_RETRY_EXE} apt-get update
|
||||
${CI_RETRY_EXE} apt-get install -y clang-format-9 python3-pip curl git gawk jq
|
||||
update-alternatives --install /usr/bin/clang-format clang-format $(which clang-format-9 ) 100
|
||||
update-alternatives --install /usr/bin/clang-format-diff clang-format-diff $(which clang-format-diff-9) 100
|
||||
update-alternatives --install /usr/bin/clang-format clang-format "$(which clang-format-9 )" 100
|
||||
update-alternatives --install /usr/bin/clang-format-diff clang-format-diff "$(which clang-format-diff-9)" 100
|
||||
|
||||
${CI_RETRY_EXE} pip3 install codespell==2.0.0
|
||||
${CI_RETRY_EXE} pip3 install flake8==3.8.3
|
||||
${CI_RETRY_EXE} pip3 install mypy==0.910
|
||||
${CI_RETRY_EXE} pip3 install pyzmq==22.3.0
|
||||
${CI_RETRY_EXE} pip3 install vulture==2.3
|
||||
${CI_RETRY_EXE} pip3 install yq
|
||||
${CI_RETRY_EXE} pip3 install mypy==0.781
|
||||
|
||||
SHELLCHECK_VERSION=v0.8.0
|
||||
curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | tar --xz -xf - --directory /tmp/
|
||||
|
@ -25,7 +25,8 @@ test/lint/lint-all.sh
|
||||
|
||||
if [ "$CIRRUS_REPO_FULL_NAME" = "dashpay/dash" ] && [ -n "$CIRRUS_CRON" ]; then
|
||||
git log --merges --before="2 days ago" -1 --format='%H' > ./contrib/verify-commits/trusted-sha512-root-commit
|
||||
${CI_RETRY_EXE} gpg --keyserver hkps://keys.openpgp.org --recv-keys $(<contrib/verify-commits/trusted-keys) &&
|
||||
mapfile -t KEYS < contrib/verify-commits/trusted-keys
|
||||
${CI_RETRY_EXE} gpg --keyserver hkps://keys.openpgp.org --recv-keys "${KEYS[@]}" &&
|
||||
./contrib/verify-commits/verify-commits.py --clean-merge=2;
|
||||
fi
|
||||
|
||||
|
@ -12,6 +12,7 @@ fi
|
||||
|
||||
if [ "$CI_OS_NAME" == "macos" ]; then
|
||||
sudo -H pip3 install --upgrade pip
|
||||
# shellcheck disable=SC2086
|
||||
IN_GETOPT_BIN="/usr/local/opt/gnu-getopt/bin/getopt" ${CI_RETRY_EXE} pip3 install --user $PIP_PACKAGES
|
||||
fi
|
||||
|
||||
@ -49,7 +50,7 @@ else
|
||||
fi
|
||||
|
||||
DOCKER_EXEC () {
|
||||
$DOCKER_CI_CMD_PREFIX bash -c "export PATH=$BASE_SCRATCH_DIR/bins/:\$PATH && cd $P_CI_DIR && $*"
|
||||
$DOCKER_CI_CMD_PREFIX bash -c "export PATH=$BASE_SCRATCH_DIR/bins/:\$PATH && cd \"$P_CI_DIR\" && $*"
|
||||
}
|
||||
export -f DOCKER_EXEC
|
||||
|
||||
@ -59,11 +60,12 @@ fi
|
||||
|
||||
if [[ $DOCKER_NAME_TAG == centos* ]]; then
|
||||
${CI_RETRY_EXE} DOCKER_EXEC yum -y install epel-release
|
||||
${CI_RETRY_EXE} DOCKER_EXEC yum -y install $DOCKER_PACKAGES $PACKAGES
|
||||
${CI_RETRY_EXE} DOCKER_EXEC yum -y install "$DOCKER_PACKAGES" "$PACKAGES"
|
||||
elif [ "$CI_USE_APT_INSTALL" != "no" ]; then
|
||||
${CI_RETRY_EXE} DOCKER_EXEC apt-get update
|
||||
${CI_RETRY_EXE} DOCKER_EXEC apt-get install --no-install-recommends --no-upgrade -y $PACKAGES $DOCKER_PACKAGES
|
||||
${CI_RETRY_EXE} DOCKER_EXEC apt-get install --no-install-recommends --no-upgrade -y "$PACKAGES" "$DOCKER_PACKAGES"
|
||||
if [ -n "$PIP_PACKAGES" ]; then
|
||||
# shellcheck disable=SC2086
|
||||
${CI_RETRY_EXE} pip3 install --user $PIP_PACKAGES
|
||||
fi
|
||||
fi
|
||||
@ -74,14 +76,14 @@ if [ "$CI_OS_NAME" == "macos" ]; then
|
||||
else
|
||||
DOCKER_EXEC free -m -h
|
||||
DOCKER_EXEC echo "Number of CPUs \(nproc\):" \$\(nproc\)
|
||||
DOCKER_EXEC echo $(lscpu | grep Endian)
|
||||
DOCKER_EXEC echo "$(lscpu | grep Endian)"
|
||||
fi
|
||||
DOCKER_EXEC echo "Free disk space:"
|
||||
DOCKER_EXEC df -h
|
||||
|
||||
if [ "$RUN_FUZZ_TESTS" = "true" ] || [ "$RUN_UNIT_TESTS" = "true" ] || [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
|
||||
if [ ! -d ${DIR_QA_ASSETS} ]; then
|
||||
DOCKER_EXEC git clone --depth=1 https://github.com/bitcoin-core/qa-assets ${DIR_QA_ASSETS}
|
||||
if [ ! -d "${DIR_QA_ASSETS}" ]; then
|
||||
DOCKER_EXEC git clone --depth=1 https://github.com/bitcoin-core/qa-assets "${DIR_QA_ASSETS}"
|
||||
fi
|
||||
|
||||
export DIR_FUZZ_IN=${DIR_QA_ASSETS}/fuzz_seed_corpus/
|
||||
@ -91,17 +93,17 @@ DOCKER_EXEC mkdir -p "${BASE_SCRATCH_DIR}/sanitizer-output/"
|
||||
|
||||
if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
|
||||
echo "Create $BASE_ROOT_DIR"
|
||||
DOCKER_EXEC rsync -a /ro_base/ $BASE_ROOT_DIR
|
||||
DOCKER_EXEC rsync -a /ro_base/ "$BASE_ROOT_DIR"
|
||||
fi
|
||||
|
||||
if [ "$USE_BUSY_BOX" = "true" ]; then
|
||||
echo "Setup to use BusyBox utils"
|
||||
DOCKER_EXEC mkdir -p $BASE_SCRATCH_DIR/bins/
|
||||
DOCKER_EXEC mkdir -p "${BASE_SCRATCH_DIR}/bins/"
|
||||
# tar excluded for now because it requires passing in the exact archive type in ./depends (fixed in later BusyBox version)
|
||||
# find excluded for now because it does not recognize the -delete option in ./depends (fixed in later BusyBox version)
|
||||
# ar excluded for now because it does not recognize the -q option in ./depends (unknown if fixed)
|
||||
# shellcheck disable=SC1010
|
||||
DOCKER_EXEC for util in \$\(busybox --list \| grep -v "^ar$" \| grep -v "^tar$" \| grep -v "^find$"\)\; do ln -s \$\(command -v busybox\) $BASE_SCRATCH_DIR/bins/\$util\; done
|
||||
DOCKER_EXEC for util in \$\(busybox --list \| grep -v "^ar$" \| grep -v "^tar$" \| grep -v "^find$"\)\; do ln -s \$\(command -v busybox\) "${BASE_SCRATCH_DIR}/bins/\$util"\; done
|
||||
# Print BusyBox version
|
||||
DOCKER_EXEC patch --help
|
||||
fi
|
||||
|
@ -8,12 +8,12 @@ export LC_ALL=C.UTF-8
|
||||
|
||||
# Make sure default datadir does not exist and is never read by creating a dummy file
|
||||
if [ "$CI_OS_NAME" == "macos" ]; then
|
||||
echo > $HOME/Library/Application\ Support/DashCore
|
||||
echo > "${HOME}/Library/Application Support/DashCore"
|
||||
else
|
||||
DOCKER_EXEC echo \> \$HOME/.dashcore
|
||||
fi
|
||||
|
||||
DOCKER_EXEC mkdir -p ${DEPENDS_DIR}/SDKs ${DEPENDS_DIR}/sdk-sources
|
||||
DOCKER_EXEC mkdir -p "${DEPENDS_DIR}/SDKs" "${DEPENDS_DIR}/sdk-sources"
|
||||
|
||||
if [ -n "$XCODE_VERSION" ] && [ ! -f "$OSX_SDK_PATH" ]; then
|
||||
DOCKER_EXEC curl --location --fail "${SDK_URL}/${OSX_SDK_BASENAME}" -o "$OSX_SDK_PATH"
|
||||
@ -22,7 +22,7 @@ if [ -n "$XCODE_VERSION" ] && [ -f "$OSX_SDK_PATH" ]; then
|
||||
DOCKER_EXEC tar -C "${DEPENDS_DIR}/SDKs" -xf "$OSX_SDK_PATH"
|
||||
fi
|
||||
if [[ $HOST = *-mingw32 ]]; then
|
||||
DOCKER_EXEC update-alternatives --set $HOST-g++ \$\(which $HOST-g++-posix\)
|
||||
DOCKER_EXEC update-alternatives --set "${HOST}-g++" \$\(which "${HOST}-g++-posix"\)
|
||||
fi
|
||||
if [ -z "$NO_DEPENDS" ]; then
|
||||
if [[ $DOCKER_NAME_TAG == centos* ]]; then
|
||||
@ -33,7 +33,7 @@ if [ -z "$NO_DEPENDS" ]; then
|
||||
else
|
||||
SHELL_OPTS="CONFIG_SHELL="
|
||||
fi
|
||||
DOCKER_EXEC $SHELL_OPTS make $MAKEJOBS -C depends HOST=$HOST $DEP_OPTS LOG=1
|
||||
DOCKER_EXEC "$SHELL_OPTS" make "$MAKEJOBS" -C depends HOST="$HOST" "$DEP_OPTS" LOG=1
|
||||
fi
|
||||
if [ -n "$PREVIOUS_RELEASES_TO_DOWNLOAD" ]; then
|
||||
DOCKER_EXEC test/get_previous_releases.py -b -t "$PREVIOUS_RELEASES_DIR" "${PREVIOUS_RELEASES_TO_DOWNLOAD}"
|
||||
|
@ -8,7 +8,7 @@ export LC_ALL=C.UTF-8
|
||||
|
||||
for b_name in "${BASE_OUTDIR}/bin"/*; do
|
||||
# shellcheck disable=SC2044
|
||||
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name $(basename $b_name)); do
|
||||
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name "$(basename "$b_name")"); do
|
||||
echo "Wrap $b ..."
|
||||
mv "$b" "${b}_orig"
|
||||
echo '#!/usr/bin/env bash' > "$b"
|
||||
|
@ -8,7 +8,7 @@ export LC_ALL=C.UTF-8
|
||||
|
||||
for b_name in {"${BASE_OUTDIR}/bin"/*,src/secp256k1/*tests,src/minisketch/test{,-verify},src/univalue/{no_nul,test_json,unitester,object}}.exe; do
|
||||
# shellcheck disable=SC2044
|
||||
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name "$(basename $b_name)"); do
|
||||
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name "$(basename "$b_name")"); do
|
||||
if (file "$b" | grep "Windows"); then
|
||||
echo "Wrap $b ..."
|
||||
mv "$b" "${b}_orig"
|
||||
|
@ -1923,7 +1923,7 @@ AC_CONFIG_LINKS([contrib/filter-lcov.py:contrib/filter-lcov.py])
|
||||
AC_CONFIG_LINKS([contrib/macdeploy/background.tiff:contrib/macdeploy/background.tiff])
|
||||
AC_CONFIG_LINKS([test/functional/test_runner.py:test/functional/test_runner.py])
|
||||
AC_CONFIG_LINKS([test/fuzz/test_runner.py:test/fuzz/test_runner.py])
|
||||
AC_CONFIG_LINKS([test/util/bitcoin-util-test.py:test/util/bitcoin-util-test.py])
|
||||
AC_CONFIG_LINKS([test/util/test_runner.py:test/util/test_runner.py])
|
||||
AC_CONFIG_LINKS([test/util/rpcauth-test.py:test/util/rpcauth-test.py])
|
||||
|
||||
dnl boost's m4 checks do something really nasty: they export these vars. As a
|
||||
|
@ -77,11 +77,10 @@ RUN pip3 install \
|
||||
flake8==3.8.3 \
|
||||
jinja2 \
|
||||
lief==0.13.2 \
|
||||
pyzmq \
|
||||
vulture==2.3 \
|
||||
mypy==0.781 \
|
||||
yq \
|
||||
multiprocess
|
||||
multiprocess \
|
||||
mypy==0.910 \
|
||||
pyzmq==22.3.0 \
|
||||
vulture==2.3
|
||||
|
||||
# dash_hash
|
||||
ARG DASH_HASH_VERSION=1.4.0
|
||||
|
@ -16,7 +16,7 @@ BITCOINTX=${BITCOINTX:-$BINDIR/dash-tx}
|
||||
WALLET_TOOL=${WALLET_TOOL:-$BINDIR/dash-wallet}
|
||||
BITCOINQT=${BITCOINQT:-$BINDIR/qt/dash-qt}
|
||||
|
||||
[ ! -x $BITCOIND ] && echo "$BITCOIND not found or not executable." && exit 1
|
||||
[ ! -x "$BITCOIND" ] && echo "$BITCOIND not found or not executable." && exit 1
|
||||
|
||||
# Don't allow man pages to be generated for binaries built from a dirty tree
|
||||
DIRTY=""
|
||||
@ -29,7 +29,7 @@ done
|
||||
if [ -n "$DIRTY" ]
|
||||
then
|
||||
echo -e "WARNING: the following binaries were built from a dirty tree:\n"
|
||||
echo -e $DIRTY
|
||||
echo -e "$DIRTY"
|
||||
echo "man pages generated from dirty binaries should NOT be committed."
|
||||
echo "To properly generate man pages, please commit your changes to the above binaries, rebuild them, then run this script again."
|
||||
fi
|
||||
@ -45,8 +45,8 @@ $BITCOIND --version | sed -n '1!p' >> footer.h2m
|
||||
|
||||
for cmd in $BITCOIND $BITCOINCLI $BITCOINTX $WALLET_TOOL $BITCOINQT; do
|
||||
cmdname="${cmd##*/}"
|
||||
help2man -N --version-string=${BTCVER[0]} --include=footer.h2m -o ${MANDIR}/${cmdname}.1 ${cmd}
|
||||
sed -i "s/\\\-${BTCVER[1]}//g" ${MANDIR}/${cmdname}.1
|
||||
help2man -N --version-string="${BTCVER[0]}" --include=footer.h2m -o "${MANDIR}/${cmdname}.1" "${cmd}"
|
||||
sed -i "s/\\\-${BTCVER[1]}//g" "${MANDIR}/${cmdname}.1"
|
||||
done
|
||||
|
||||
rm -f footer.h2m
|
||||
|
@ -10,7 +10,7 @@ Otherwise the exit status will be 1 and it will log which executables failed whi
|
||||
import sys
|
||||
from typing import List
|
||||
|
||||
import lief
|
||||
import lief #type:ignore
|
||||
|
||||
def check_ELF_RELRO(binary) -> bool:
|
||||
'''
|
||||
|
@ -13,7 +13,7 @@ Example usage:
|
||||
import sys
|
||||
from typing import Dict, List
|
||||
|
||||
import lief
|
||||
import lief #type:ignore
|
||||
|
||||
# Debian 11 (Bullseye) EOL: 2026. https://wiki.debian.org/LTS
|
||||
#
|
||||
|
@ -20,7 +20,7 @@ expand_path() {
|
||||
cd "${1}" && pwd -P
|
||||
}
|
||||
|
||||
BDB_PREFIX="$(expand_path ${1})/db4"; shift;
|
||||
BDB_PREFIX="$(expand_path "${1}")/db4"; shift;
|
||||
BDB_VERSION='db-4.8.30.NC'
|
||||
BDB_HASH='12edc0df75bf9abd7f82f821795bcee50f42cb2e5f76a6a281b85732798364ef'
|
||||
BDB_URL="https://download.oracle.com/berkeley-db/${BDB_VERSION}.tar.gz"
|
||||
|
@ -9,11 +9,11 @@ if ! [[ "$2" =~ ^(git@)?(www.)?github.com(:|/)dashpay/dash(.git)?$ ]]; then
|
||||
fi
|
||||
|
||||
while read LINE; do
|
||||
set -- A $LINE
|
||||
set -- A "$LINE"
|
||||
if [ "$4" != "refs/heads/master" ]; then
|
||||
continue
|
||||
fi
|
||||
if ! ./contrib/verify-commits/verify-commits.py $3 > /dev/null 2>&1; then
|
||||
if ! ./contrib/verify-commits/verify-commits.py "$3" > /dev/null 2>&1; then
|
||||
echo "ERROR: A commit is not signed, can't push"
|
||||
./contrib/verify-commits/verify-commits.py
|
||||
exit 1
|
||||
|
@ -23,6 +23,7 @@ TIMESERVER=http://timestamp.comodoca.com
|
||||
CERTFILE="win-codesign.cert"
|
||||
|
||||
mkdir -p "${OUTSUBDIR}"
|
||||
# shellcheck disable=SC2046
|
||||
basename -a $(ls -1 "${SRCDIR}"/*-unsigned.exe) | while read UNSIGNED; do
|
||||
echo Signing "${UNSIGNED}"
|
||||
"${OSSLSIGNCODE}" sign -certs "${CERTFILE}" -t "${TIMESERVER}" -h sha256 -in "${SRCDIR}/${UNSIGNED}" -out "${WORKDIR}/${UNSIGNED}" "$@"
|
||||
|
@ -16,6 +16,9 @@ $ FUZZ=process_message src/test/fuzz/fuzz
|
||||
# abort fuzzing using ctrl-c
|
||||
```
|
||||
|
||||
There is also a runner script to execute all fuzz targets. Refer to
|
||||
`./test/fuzz/test_runner.py --help` for more details.
|
||||
|
||||
## Fuzzing harnesses and output
|
||||
|
||||
[`process_message`](https://github.com/dashpay/dash/blob/develop/src/test/fuzz/process_message.cpp) is a fuzzing harness for the [`ProcessMessage(...)` function (`net_processing`)](https://github.com/dashpay/dash/blob/develop/src/net_processing.cpp). The available fuzzing harnesses are found in [`src/test/fuzz/`](https://github.com/dashpay/dash/tree/develop/src/test/fuzz).
|
||||
|
@ -375,8 +375,8 @@ check-unit: $(BITCOIN_TESTS:.cpp=.cpp.test)
|
||||
|
||||
check-local: check-unit
|
||||
if BUILD_BITCOIN_TX
|
||||
@echo "Running test/util/bitcoin-util-test.py..."
|
||||
$(PYTHON) $(top_builddir)/test/util/bitcoin-util-test.py
|
||||
@echo "Running test/util/test_runner.py..."
|
||||
$(PYTHON) $(top_builddir)/test/util/test_runner.py
|
||||
endif
|
||||
@echo "Running test/util/rpcauth-test.py..."
|
||||
$(PYTHON) $(top_builddir)/test/util/rpcauth-test.py
|
||||
|
@ -5,10 +5,10 @@
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
export LC_ALL=C
|
||||
FRAMEDIR=$(dirname $0)
|
||||
FRAMEDIR=$(dirname "$0")
|
||||
for i in {0..89}
|
||||
do
|
||||
frame=$(printf "%03d" $i)
|
||||
frame=$(printf "%03d" "$i")
|
||||
angle=$((i * 4))
|
||||
convert $FRAMEDIR/../src/spinner.png -background "rgba(0,0,0,0.0)" -distort SRT $angle $FRAMEDIR/spinner-$frame.png
|
||||
convert "${FRAMEDIR}/../src/spinner.png" -background "rgba(0,0,0,0.0)" -distort SRT $angle "${FRAMEDIR}/spinner-${frame}.png"
|
||||
done
|
||||
|
@ -14,7 +14,7 @@
|
||||
#include <compat.h> // for Windows API
|
||||
#include <wincrypt.h>
|
||||
#endif
|
||||
#include <logging.h> // for LogPrintf()
|
||||
#include <logging.h>
|
||||
#include <randomenv.h>
|
||||
#include <support/allocators/secure.h>
|
||||
#include <span.h>
|
||||
|
@ -5,20 +5,24 @@ etc.
|
||||
|
||||
This directory contains the following sets of tests:
|
||||
|
||||
- [fuzz](/test/fuzz) A runner to execute all fuzz targets from
|
||||
[/src/test/fuzz](/src/test/fuzz).
|
||||
- [functional](/test/functional) which test the functionality of
|
||||
dashd and dash-qt by interacting with them through the RPC and P2P
|
||||
interfaces.
|
||||
- [util](/test/util) which tests the dash utilities, currently only
|
||||
dash-tx.
|
||||
- [util](/test/util) which tests the utilities (dash-tx, ...).
|
||||
- [lint](/test/lint/) which perform various static analysis checks.
|
||||
|
||||
The util tests are run as part of `make check` target. The functional
|
||||
The util tests are run as part of `make check` target. The fuzz tests, functional
|
||||
tests and lint scripts can be run as explained in the sections below.
|
||||
|
||||
# Running tests locally
|
||||
|
||||
Before tests can be run locally, Dash Core must be built. See the [building instructions](/doc#building) for help.
|
||||
|
||||
## Fuzz tests
|
||||
|
||||
See [/doc/fuzzing.md](/doc/fuzzing.md)
|
||||
|
||||
### Functional tests
|
||||
|
||||
@ -297,20 +301,23 @@ For ways to generate more granular profiles, see the README in
|
||||
|
||||
### Util tests
|
||||
|
||||
Util tests can be run locally by running `test/util/bitcoin-util-test.py`.
|
||||
Util tests can be run locally by running `test/util/test_runner.py`.
|
||||
Use the `-v` option for verbose output.
|
||||
|
||||
### Lint tests
|
||||
|
||||
#### Dependencies
|
||||
|
||||
| Lint test | Dependency | Version [used by CI](../ci/lint/04_install.sh) | Installation
|
||||
|-----------|:----------:|:-------------------------------------------:|--------------
|
||||
| [`lint-python.sh`](lint/lint-python.sh) | [flake8](https://gitlab.com/pycqa/flake8) | [3.8.3](https://github.com/bitcoin/bitcoin/pull/19348) | `pip3 install flake8==3.8.3`
|
||||
| [`lint-python.sh`](lint/lint-python.sh) | [mypy](https://github.com/python/mypy) | [0.781](https://github.com/bitcoin/bitcoin/pull/19348) | `pip3 install mypy==0.781`
|
||||
| [`lint-shell.sh`](lint/lint-shell.sh) | [ShellCheck](https://github.com/koalaman/shellcheck) | [0.7.2](https://github.com/bitcoin/bitcoin/pull/21749) | [details...](https://github.com/koalaman/shellcheck#installing)
|
||||
| [`lint-shell.sh`](lint/lint-shell.sh) | [yq](https://github.com/kislyuk/yq) | default | `pip3 install yq`
|
||||
| [`lint-spelling.sh`](lint/lint-spelling.sh) | [codespell](https://github.com/codespell-project/codespell) | [2.0.0](https://github.com/bitcoin/bitcoin/pull/20817) | `pip3 install codespell==2.0.0`
|
||||
| Lint test | Dependency |
|
||||
|-----------|:----------:|
|
||||
| [`lint-python.sh`](lint/lint-python.sh) | [flake8](https://gitlab.com/pycqa/flake8)
|
||||
| [`lint-python.sh`](lint/lint-python.sh) | [mypy](https://github.com/python/mypy)
|
||||
| [`lint-python.sh`](lint/lint-python.sh) | [pyzmq](https://github.com/zeromq/pyzmq)
|
||||
| [`lint-python-dead-code.py`](lint/lint-python-dead-code.py) | [vulture](https://github.com/jendrikseipp/vulture)
|
||||
| [`lint-shell.sh`](lint/lint-shell.sh) | [ShellCheck](https://github.com/koalaman/shellcheck)
|
||||
| [`lint-spelling.py`](lint/lint-spelling.py) | [codespell](https://github.com/codespell-project/codespell)
|
||||
|
||||
In use versions and install instructions are available in the [CI setup](../ci/lint/04_install.sh).
|
||||
|
||||
Please be aware that on Linux distributions all dependencies are usually available as packages, but could be outdated.
|
||||
|
||||
@ -319,7 +326,7 @@ Please be aware that on Linux distributions all dependencies are usually availab
|
||||
Individual tests can be run by directly calling the test script, e.g.:
|
||||
|
||||
```
|
||||
test/lint/lint-files.sh
|
||||
test/lint/lint-files.py
|
||||
```
|
||||
|
||||
You can run all the shell-based lint tests by running:
|
||||
|
@ -3,7 +3,7 @@
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
# These environment variables are set by the build process and read by
|
||||
# test/functional/test_runner.py and test/util/bitcoin-util-test.py
|
||||
# test/*/test_runner.py and test/util/rpcauth-test.py
|
||||
|
||||
[environment]
|
||||
PACKAGE_NAME=@PACKAGE_NAME@
|
||||
|
@ -188,7 +188,7 @@ def print_logs_plain(log_events, colors):
|
||||
def print_logs_html(log_events):
|
||||
"""Renders the iterator of log events into html."""
|
||||
try:
|
||||
import jinja2
|
||||
import jinja2 #type:ignore
|
||||
except ImportError:
|
||||
print("jinja2 not found. Try `pip install jinja2`")
|
||||
sys.exit(1)
|
||||
|
0
test/functional/data/__init__.py
Normal file
0
test/functional/data/__init__.py
Normal file
@ -40,7 +40,7 @@ except UnicodeDecodeError:
|
||||
CROSS = "x "
|
||||
CIRCLE = "o "
|
||||
|
||||
if os.name != 'nt' or sys.getwindowsversion() >= (10, 0, 14393):
|
||||
if os.name != 'nt' or sys.getwindowsversion() >= (10, 0, 14393): #type:ignore
|
||||
if os.name == 'nt':
|
||||
import ctypes
|
||||
kernel32 = ctypes.windll.kernel32 # type: ignore
|
||||
|
@ -12,7 +12,7 @@
|
||||
# one. Any remaining diff signals an error.
|
||||
|
||||
export LC_ALL=C
|
||||
if test -z $1; then
|
||||
if test -z "$1"; then
|
||||
echo "Usage: $0 <commit>..."
|
||||
exit 1
|
||||
fi
|
||||
@ -30,10 +30,10 @@ fi
|
||||
RET=0
|
||||
PREV_BRANCH=$(git name-rev --name-only HEAD)
|
||||
PREV_HEAD=$(git rev-parse HEAD)
|
||||
for commit in $(git rev-list --reverse $1); do
|
||||
if git rev-list -n 1 --pretty="%s" $commit | grep -q "^scripted-diff:"; then
|
||||
git checkout --quiet $commit^ || exit
|
||||
SCRIPT="$(git rev-list --format=%b -n1 $commit | sed '/^-BEGIN VERIFY SCRIPT-$/,/^-END VERIFY SCRIPT-$/{//!b};d')"
|
||||
for commit in $(git rev-list --reverse "$1"); do
|
||||
if git rev-list -n 1 --pretty="%s" "$commit" | grep -q "^scripted-diff:"; then
|
||||
git checkout --quiet "$commit"^ || exit
|
||||
SCRIPT="$(git rev-list --format=%b -n1 "$commit" | sed '/^-BEGIN VERIFY SCRIPT-$/,/^-END VERIFY SCRIPT-$/{//!b};d')"
|
||||
if test -z "$SCRIPT"; then
|
||||
echo "Error: missing script for: $commit"
|
||||
echo "Failed"
|
||||
@ -42,16 +42,16 @@ for commit in $(git rev-list --reverse $1); do
|
||||
echo "Running script for: $commit"
|
||||
echo "$SCRIPT"
|
||||
(eval "$SCRIPT")
|
||||
git --no-pager diff --exit-code $commit && echo "OK" || (echo "Failed"; false) || RET=1
|
||||
git --no-pager diff --exit-code "$commit" && echo "OK" || (echo "Failed"; false) || RET=1
|
||||
fi
|
||||
git reset --quiet --hard HEAD
|
||||
else
|
||||
if git rev-list "--format=%b" -n1 $commit | grep -q '^-\(BEGIN\|END\)[ a-zA-Z]*-$'; then
|
||||
if git rev-list "--format=%b" -n1 "$commit" | grep -q '^-\(BEGIN\|END\)[ a-zA-Z]*-$'; then
|
||||
echo "Error: script block marker but no scripted-diff in title of commit $commit"
|
||||
echo "Failed"
|
||||
RET=1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
git checkout --quiet $PREV_BRANCH 2>/dev/null || git checkout --quiet $PREV_HEAD
|
||||
git checkout --quiet "$PREV_BRANCH" 2>/dev/null || git checkout --quiet "$PREV_HEAD"
|
||||
exit $RET
|
||||
|
@ -82,6 +82,7 @@ if [ -z "$latest_squash" ]; then
|
||||
echo "ERROR: $DIR is not a subtree" >&2
|
||||
exit 2
|
||||
fi
|
||||
# shellcheck disable=SC2086
|
||||
set $latest_squash
|
||||
old=$1
|
||||
rev=$2
|
||||
@ -92,6 +93,7 @@ if [ -z "$tree_actual" ]; then
|
||||
echo "FAIL: subtree directory $DIR not found in $COMMIT" >&2
|
||||
exit 1
|
||||
fi
|
||||
# shellcheck disable=SC2086
|
||||
set $tree_actual
|
||||
tree_actual_type=$2
|
||||
tree_actual_tree=$3
|
||||
@ -102,23 +104,23 @@ if [ "d$tree_actual_type" != "dtree" ]; then
|
||||
fi
|
||||
|
||||
# get the tree at the time of the last subtree update
|
||||
tree_commit=$(git show -s --format="%T" $old)
|
||||
tree_commit=$(git show -s --format="%T" "$old")
|
||||
echo "$DIR in $COMMIT was last updated in commit $old (tree $tree_commit)"
|
||||
|
||||
# ... and compare the actual tree with it
|
||||
if [ "$tree_actual_tree" != "$tree_commit" ]; then
|
||||
git diff $tree_commit $tree_actual_tree >&2
|
||||
git diff "$tree_commit" "$tree_actual_tree" >&2
|
||||
echo "FAIL: subtree directory was touched without subtree merge" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ "$check_remote" != "0" ]; then
|
||||
# get the tree in the subtree commit referred to
|
||||
if [ "d$(git cat-file -t $rev 2>/dev/null)" != dcommit ]; then
|
||||
if [ "d$(git cat-file -t "$rev" 2>/dev/null)" != dcommit ]; then
|
||||
echo "subtree commit $rev unavailable: cannot compare. Did you add and fetch the remote?" >&2
|
||||
exit 1
|
||||
fi
|
||||
tree_subtree=$(git show -s --format="%T" $rev)
|
||||
tree_subtree=$(git show -s --format="%T" "$rev")
|
||||
echo "$DIR in $COMMIT was last updated to upstream commit $rev (tree $tree_subtree)"
|
||||
|
||||
# ... and compare the actual tree with it
|
||||
|
@ -30,13 +30,13 @@ if ! command -v parallel > /dev/null; then
|
||||
else
|
||||
SCRIPTS=()
|
||||
|
||||
for f in "${SCRIPTDIR}"/lint-*.sh; do
|
||||
for f in "${SCRIPTDIR}"/lint-*; do
|
||||
if [ "$(basename "$f")" != "$LINTALL" ]; then
|
||||
SCRIPTS+=("$f")
|
||||
fi
|
||||
done
|
||||
|
||||
if ! parallel --jobs 100% --will-cite --joblog parallel_out.log bash ::: "${SCRIPTS[@]}"; then
|
||||
if ! parallel --jobs 100% --will-cite --joblog parallel_out.log ::: "${SCRIPTS[@]}"; then
|
||||
echo "^---- failure generated"
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
|
99
test/lint/lint-format-strings.py
Executable file
99
test/lint/lint-format-strings.py
Executable file
@ -0,0 +1,99 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright (c) 2018-2022 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
|
||||
"""
|
||||
Lint format strings: This program checks that the number of arguments passed
|
||||
to a variadic format string function matches the number of format specifiers
|
||||
in the format string.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import re
|
||||
import sys
|
||||
|
||||
FUNCTION_NAMES_AND_NUMBER_OF_LEADING_ARGUMENTS = [
|
||||
'FatalError,0',
|
||||
'fprintf,1',
|
||||
'tfm::format,1', # Assuming tfm::::format(std::ostream&, ...
|
||||
'LogConnectFailure,1',
|
||||
'LogPrint,1',
|
||||
'LogPrintf,0',
|
||||
'LogPrintLevel,2',
|
||||
'printf,0',
|
||||
'snprintf,2',
|
||||
'sprintf,1',
|
||||
'strprintf,0',
|
||||
'vfprintf,1',
|
||||
'vprintf,1',
|
||||
'vsnprintf,1',
|
||||
'vsprintf,1',
|
||||
'WalletLogPrintf,0',
|
||||
]
|
||||
RUN_LINT_FILE = 'test/lint/run-lint-format-strings.py'
|
||||
|
||||
def check_doctest():
|
||||
command = [
|
||||
'python3',
|
||||
'-m',
|
||||
'doctest',
|
||||
RUN_LINT_FILE,
|
||||
]
|
||||
try:
|
||||
subprocess.run(command, check = True)
|
||||
except subprocess.CalledProcessError:
|
||||
sys.exit(1)
|
||||
|
||||
def get_matching_files(function_name):
|
||||
command = [
|
||||
'git',
|
||||
'grep',
|
||||
'--full-name',
|
||||
'-l',
|
||||
function_name,
|
||||
'--',
|
||||
'*.c',
|
||||
'*.cpp',
|
||||
'*.h',
|
||||
]
|
||||
try:
|
||||
return subprocess.check_output(command, stderr = subprocess.STDOUT).decode('utf-8').splitlines()
|
||||
except subprocess.CalledProcessError as e:
|
||||
if e.returncode > 1: # return code is 1 when match is empty
|
||||
print(e.output.decode('utf-8'), end='')
|
||||
sys.exit(1)
|
||||
return []
|
||||
|
||||
def main():
|
||||
exit_code = 0
|
||||
check_doctest()
|
||||
for s in FUNCTION_NAMES_AND_NUMBER_OF_LEADING_ARGUMENTS:
|
||||
function_name, skip_arguments = s.split(',')
|
||||
matching_files = get_matching_files(function_name)
|
||||
|
||||
matching_files_filtered = []
|
||||
for matching_file in matching_files:
|
||||
if not re.search('^src/(dashbls|leveldb|secp256k1|minisketch|tinyformat|univalue|test/fuzz/strprintf.cpp)', matching_file):
|
||||
matching_files_filtered.append(matching_file)
|
||||
matching_files_filtered.sort()
|
||||
|
||||
run_lint_args = [
|
||||
RUN_LINT_FILE,
|
||||
'--skip-arguments',
|
||||
skip_arguments,
|
||||
function_name,
|
||||
]
|
||||
run_lint_args.extend(matching_files_filtered)
|
||||
|
||||
try:
|
||||
subprocess.run(run_lint_args, check = True)
|
||||
except subprocess.CalledProcessError:
|
||||
exit_code = 1
|
||||
|
||||
sys.exit(exit_code)
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
@ -1,45 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Copyright (c) 2018-2020 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Lint format strings: This program checks that the number of arguments passed
|
||||
# to a variadic format string function matches the number of format specifiers
|
||||
# in the format string.
|
||||
|
||||
export LC_ALL=C
|
||||
|
||||
FUNCTION_NAMES_AND_NUMBER_OF_LEADING_ARGUMENTS=(
|
||||
"FatalError,0"
|
||||
"fprintf,1"
|
||||
"tfm::format,1" # Assuming tfm::::format(std::ostream&, ...
|
||||
"LogConnectFailure,1"
|
||||
"LogPrint,1"
|
||||
"LogPrintf,0"
|
||||
"LogPrintLevel,2"
|
||||
"printf,0"
|
||||
"snprintf,2"
|
||||
"sprintf,1"
|
||||
"strprintf,0"
|
||||
"vfprintf,1"
|
||||
"vprintf,1"
|
||||
"vsnprintf,1"
|
||||
"vsprintf,1"
|
||||
"WalletLogPrintf,0"
|
||||
)
|
||||
|
||||
EXIT_CODE=0
|
||||
if ! python3 -m doctest "test/lint/run-lint-format-strings.py"; then
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
for S in "${FUNCTION_NAMES_AND_NUMBER_OF_LEADING_ARGUMENTS[@]}"; do
|
||||
IFS="," read -r FUNCTION_NAME SKIP_ARGUMENTS <<< "${S}"
|
||||
for MATCHING_FILE in $(git grep --full-name -l "${FUNCTION_NAME}" -- "*.c" "*.cpp" "*.h" | sort | grep -vE "^src/(dashbls|leveldb|secp256k1|minisketch|tinyformat|univalue|test/fuzz/strprintf.cpp)"); do
|
||||
MATCHING_FILES+=("${MATCHING_FILE}")
|
||||
done
|
||||
if ! "test/lint/run-lint-format-strings.py" --skip-arguments "${SKIP_ARGUMENTS}" "${FUNCTION_NAME}" "${MATCHING_FILES[@]}"; then
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
done
|
||||
exit ${EXIT_CODE}
|
63
test/lint/lint-git-commit-check.py
Executable file
63
test/lint/lint-git-commit-check.py
Executable file
@ -0,0 +1,63 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright (c) 2020-2022 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Linter to check that commit messages have a new line before the body
|
||||
# or no body at all
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
|
||||
from subprocess import check_output
|
||||
|
||||
|
||||
def parse_args():
|
||||
"""Parse command line arguments."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="""
|
||||
Linter to check that commit messages have a new line before
|
||||
the body or no body at all.
|
||||
""",
|
||||
epilog=f"""
|
||||
You can manually set the commit-range with the COMMIT_RANGE
|
||||
environment variable (e.g. "COMMIT_RANGE='47ba2c3...ee50c9e'
|
||||
{sys.argv[0]}"). Defaults to current merge base when neither
|
||||
prev-commits nor the environment variable is set.
|
||||
""")
|
||||
|
||||
parser.add_argument("--prev-commits", "-p", required=False, help="The previous n commits to check")
|
||||
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main():
|
||||
args = parse_args()
|
||||
exit_code = 0
|
||||
|
||||
if not os.getenv("COMMIT_RANGE"):
|
||||
if args.prev_commits:
|
||||
commit_range = "HEAD~" + args.prev_commits + "...HEAD"
|
||||
else:
|
||||
# This assumes that the target branch of the pull request will be develop.
|
||||
merge_base = check_output(["git", "merge-base", "HEAD", "develop"], universal_newlines=True, encoding="utf8").rstrip("\n")
|
||||
commit_range = merge_base + "..HEAD"
|
||||
else:
|
||||
commit_range = os.getenv("COMMIT_RANGE")
|
||||
|
||||
commit_hashes = check_output(["git", "log", commit_range, "--format=%H"], universal_newlines=True, encoding="utf8").splitlines()
|
||||
|
||||
for hash in commit_hashes:
|
||||
commit_info = check_output(["git", "log", "--format=%B", "-n", "1", hash], universal_newlines=True, encoding="utf8").splitlines()
|
||||
if len(commit_info) >= 2:
|
||||
if commit_info[1]:
|
||||
print(f"The subject line of commit hash {hash} is followed by a non-empty line. Subject lines should always be followed by a blank line.")
|
||||
exit_code = 1
|
||||
|
||||
sys.exit(exit_code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@ -1,48 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Copyright (c) 2020 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Linter to check that commit messages have a new line before the body
|
||||
# or no body at all
|
||||
|
||||
export LC_ALL=C
|
||||
|
||||
EXIT_CODE=0
|
||||
|
||||
while getopts "?" opt; do
|
||||
case $opt in
|
||||
?)
|
||||
echo "Usage: $0 [N]"
|
||||
echo " COMMIT_RANGE='<commit range>' $0"
|
||||
echo " $0 -?"
|
||||
echo "Checks unmerged commits, the previous N commits, or a commit range."
|
||||
echo "COMMIT_RANGE='47ba2c3...ee50c9e' $0"
|
||||
exit ${EXIT_CODE}
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ -z "${COMMIT_RANGE}" ]; then
|
||||
if [ -n "$1" ]; then
|
||||
COMMIT_RANGE="HEAD~$1...HEAD"
|
||||
else
|
||||
# This assumes that the target branch of the pull request will be develop
|
||||
MERGE_BASE=$(git merge-base HEAD develop)
|
||||
COMMIT_RANGE="$MERGE_BASE..HEAD"
|
||||
fi
|
||||
fi
|
||||
|
||||
while IFS= read -r commit_hash || [[ -n "$commit_hash" ]]; do
|
||||
n_line=0
|
||||
while IFS= read -r line || [[ -n "$line" ]]; do
|
||||
n_line=$((n_line+1))
|
||||
length=${#line}
|
||||
if [ $n_line -eq 2 ] && [ $length -ne 0 ]; then
|
||||
echo "The subject line of commit hash ${commit_hash} is followed by a non-empty line. Subject lines should always be followed by a blank line."
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
done < <(git log --format=%B -n 1 "$commit_hash")
|
||||
done < <(git log "${COMMIT_RANGE}" --format=%H)
|
||||
|
||||
exit ${EXIT_CODE}
|
184
test/lint/lint-includes.py
Executable file
184
test/lint/lint-includes.py
Executable file
@ -0,0 +1,184 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright (c) 2018-2022 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Check for duplicate includes.
|
||||
# Guard against accidental introduction of new Boost dependencies.
|
||||
# Check includes: Check for duplicate includes. Enforce bracket syntax includes.
|
||||
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
from subprocess import check_output, CalledProcessError
|
||||
|
||||
|
||||
EXCLUDED_DIRS = ["src/leveldb/",
|
||||
"src/crc32c/",
|
||||
"src/secp256k1/",
|
||||
"src/minisketch/",
|
||||
"src/univalue/",
|
||||
"src/dashbls/",
|
||||
"src/immer/",
|
||||
"src/crypto/x11/"]
|
||||
|
||||
EXPECTED_BOOST_INCLUDES = ["boost/date_time/posix_time/posix_time.hpp",
|
||||
"boost/multi_index/hashed_index.hpp",
|
||||
"boost/multi_index/identity.hpp",
|
||||
"boost/multi_index/indexed_by.hpp",
|
||||
"boost/multi_index/ordered_index.hpp",
|
||||
"boost/multi_index/sequenced_index.hpp",
|
||||
"boost/multi_index/tag.hpp",
|
||||
"boost/multi_index_container.hpp",
|
||||
"boost/pool/pool_alloc.hpp",
|
||||
"boost/process.hpp",
|
||||
"boost/signals2/connection.hpp",
|
||||
"boost/signals2/optional_last_value.hpp",
|
||||
"boost/signals2/signal.hpp",
|
||||
"boost/test/included/unit_test.hpp",
|
||||
"boost/test/unit_test.hpp"]
|
||||
|
||||
EXCLUDED_BOOST_DIRS = ["src/immer/"]
|
||||
|
||||
def get_toplevel():
|
||||
return check_output(["git", "rev-parse", "--show-toplevel"], universal_newlines=True, encoding="utf8").rstrip("\n")
|
||||
|
||||
|
||||
def list_files_by_suffix(suffixes):
|
||||
exclude_args = [":(exclude)" + dir for dir in EXCLUDED_DIRS]
|
||||
|
||||
files_list = check_output(["git", "ls-files", "src"] + exclude_args, universal_newlines=True, encoding="utf8").splitlines()
|
||||
|
||||
return [file for file in files_list if file.endswith(suffixes)]
|
||||
|
||||
|
||||
def find_duplicate_includes(include_list):
|
||||
tempset = set()
|
||||
duplicates = set()
|
||||
|
||||
for inclusion in include_list:
|
||||
if inclusion in tempset:
|
||||
duplicates.add(inclusion)
|
||||
else:
|
||||
tempset.add(inclusion)
|
||||
|
||||
return duplicates
|
||||
|
||||
|
||||
def find_included_cpps():
|
||||
included_cpps = list()
|
||||
|
||||
try:
|
||||
included_cpps = check_output(["git", "grep", "-E", r"^#include [<\"][^>\"]+\.cpp[>\"]", "--", "*.cpp", "*.h"], universal_newlines=True, encoding="utf8").splitlines()
|
||||
except CalledProcessError as e:
|
||||
if e.returncode > 1:
|
||||
raise e
|
||||
|
||||
return included_cpps
|
||||
|
||||
|
||||
def find_extra_boosts():
|
||||
included_boosts = list()
|
||||
filtered_included_boost_set = set()
|
||||
exclusion_set = set()
|
||||
exclude_args = [":(exclude)" + dir for dir in EXCLUDED_BOOST_DIRS]
|
||||
|
||||
try:
|
||||
included_boosts = check_output(["git", "grep", "-E", r"^#include <boost/", "--", "*.cpp", "*.h"] + exclude_args, universal_newlines=True, encoding="utf8").splitlines()
|
||||
except CalledProcessError as e:
|
||||
if e.returncode > 1:
|
||||
raise e
|
||||
|
||||
for boost in included_boosts:
|
||||
filtered_included_boost_set.add(re.findall(r'(?<=\<).+?(?=\>)', boost)[0])
|
||||
|
||||
for expected_boost in EXPECTED_BOOST_INCLUDES:
|
||||
for boost in filtered_included_boost_set:
|
||||
if expected_boost in boost:
|
||||
exclusion_set.add(boost)
|
||||
|
||||
extra_boosts = set(filtered_included_boost_set.difference(exclusion_set))
|
||||
|
||||
return extra_boosts
|
||||
|
||||
|
||||
def find_quote_syntax_inclusions():
|
||||
exclude_args = [":(exclude)" + dir for dir in EXCLUDED_DIRS]
|
||||
quote_syntax_inclusions = list()
|
||||
|
||||
try:
|
||||
quote_syntax_inclusions = check_output(["git", "grep", r"^#include \"", "--", "*.cpp", "*.h"] + exclude_args, universal_newlines=True, encoding="utf8").splitlines()
|
||||
except CalledProcessError as e:
|
||||
if e.returncode > 1:
|
||||
raise e
|
||||
|
||||
return quote_syntax_inclusions
|
||||
|
||||
|
||||
def main():
|
||||
exit_code = 0
|
||||
|
||||
os.chdir(get_toplevel())
|
||||
|
||||
# Check for duplicate includes
|
||||
for filename in list_files_by_suffix((".cpp", ".h")):
|
||||
with open(filename, "r", encoding="utf8") as file:
|
||||
include_list = [line.rstrip("\n") for line in file if re.match(r"^#include", line)]
|
||||
|
||||
duplicates = find_duplicate_includes(include_list)
|
||||
|
||||
if duplicates:
|
||||
print(f"Duplicate include(s) in {filename}:")
|
||||
for duplicate in duplicates:
|
||||
print(duplicate)
|
||||
print("")
|
||||
exit_code = 1
|
||||
|
||||
# Check if code includes .cpp-files
|
||||
included_cpps = find_included_cpps()
|
||||
|
||||
if included_cpps:
|
||||
print("The following files #include .cpp files:")
|
||||
for included_cpp in included_cpps:
|
||||
print(included_cpp)
|
||||
print("")
|
||||
exit_code = 1
|
||||
|
||||
# Guard against accidental introduction of new Boost dependencies
|
||||
extra_boosts = find_extra_boosts()
|
||||
|
||||
if extra_boosts:
|
||||
for boost in extra_boosts:
|
||||
print(f"A new Boost dependency in the form of \"{boost}\" appears to have been introduced:")
|
||||
print(check_output(["git", "grep", boost, "--", "*.cpp", "*.h"], universal_newlines=True, encoding="utf8"))
|
||||
exit_code = 1
|
||||
|
||||
# Check if Boost dependencies are no longer used
|
||||
for expected_boost in EXPECTED_BOOST_INCLUDES:
|
||||
try:
|
||||
check_output(["git", "grep", "-q", r"^#include <%s>" % expected_boost, "--", "*.cpp", "*.h"], universal_newlines=True, encoding="utf8")
|
||||
except CalledProcessError as e:
|
||||
if e.returncode > 1:
|
||||
raise e
|
||||
else:
|
||||
print(f"Good job! The Boost dependency \"{expected_boost}\" is no longer used. "
|
||||
"Please remove it from EXPECTED_BOOST_INCLUDES in test/lint/lint-includes.py "
|
||||
"to make sure this dependency is not accidentally reintroduced.\n")
|
||||
exit_code = 1
|
||||
|
||||
# Enforce bracket syntax includes
|
||||
quote_syntax_inclusions = find_quote_syntax_inclusions()
|
||||
|
||||
if quote_syntax_inclusions:
|
||||
print("Please use bracket syntax includes (\"#include <foo.h>\") instead of quote syntax includes:")
|
||||
for quote_syntax_inclusion in quote_syntax_inclusions:
|
||||
print(quote_syntax_inclusion)
|
||||
exit_code = 1
|
||||
|
||||
sys.exit(exit_code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@ -1,103 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Copyright (c) 2018-2020 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Check for duplicate includes.
|
||||
# Guard against accidental introduction of new Boost dependencies.
|
||||
# Check includes: Check for duplicate includes. Enforce bracket syntax includes.
|
||||
|
||||
export LC_ALL=C
|
||||
IGNORE_REGEXP="/(dashbls|immer|leveldb|secp256k1|minisketch|univalue|crc32c|crypto/x11)/"
|
||||
|
||||
# cd to root folder of git repo for git ls-files to work properly
|
||||
cd "$(dirname $0)/../.." || exit 1
|
||||
|
||||
filter_suffix() {
|
||||
git ls-files | grep -E "^src/.*\.${1}"'$' | grep -Ev "${IGNORE_REGEXP}"
|
||||
}
|
||||
|
||||
EXIT_CODE=0
|
||||
|
||||
for HEADER_FILE in $(filter_suffix h); do
|
||||
DUPLICATE_INCLUDES_IN_HEADER_FILE=$(grep -E "^#include " < "${HEADER_FILE}" | sort | uniq -d)
|
||||
if [[ ${DUPLICATE_INCLUDES_IN_HEADER_FILE} != "" ]]; then
|
||||
echo "Duplicate include(s) in ${HEADER_FILE}:"
|
||||
echo "${DUPLICATE_INCLUDES_IN_HEADER_FILE}"
|
||||
echo
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
done
|
||||
|
||||
for CPP_FILE in $(filter_suffix cpp); do
|
||||
DUPLICATE_INCLUDES_IN_CPP_FILE=$(grep -E "^#include " < "${CPP_FILE}" | sort | uniq -d)
|
||||
if [[ ${DUPLICATE_INCLUDES_IN_CPP_FILE} != "" ]]; then
|
||||
echo "Duplicate include(s) in ${CPP_FILE}:"
|
||||
echo "${DUPLICATE_INCLUDES_IN_CPP_FILE}"
|
||||
echo
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
done
|
||||
|
||||
INCLUDED_CPP_FILES=$(git grep -E "^#include [<\"][^>\"]+\.cpp[>\"]" -- "*.cpp" "*.h")
|
||||
if [[ ${INCLUDED_CPP_FILES} != "" ]]; then
|
||||
echo "The following files #include .cpp files:"
|
||||
echo "${INCLUDED_CPP_FILES}"
|
||||
echo
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
|
||||
EXPECTED_BOOST_INCLUDES=(
|
||||
boost/date_time/posix_time/posix_time.hpp
|
||||
boost/multi_index/hashed_index.hpp
|
||||
boost/multi_index/identity.hpp
|
||||
boost/multi_index/indexed_by.hpp
|
||||
boost/multi_index/ordered_index.hpp
|
||||
boost/multi_index/sequenced_index.hpp
|
||||
boost/multi_index/tag.hpp
|
||||
boost/multi_index_container.hpp
|
||||
boost/pool/pool_alloc.hpp
|
||||
boost/process.hpp
|
||||
boost/signals2/connection.hpp
|
||||
boost/signals2/optional_last_value.hpp
|
||||
boost/signals2/signal.hpp
|
||||
boost/test/included/unit_test.hpp
|
||||
boost/test/unit_test.hpp
|
||||
)
|
||||
|
||||
for BOOST_INCLUDE in $(git grep '^#include <boost/' -- "*.cpp" "*.h" | grep -vE "src/(immer)/" | cut -f2 -d: | cut -f2 -d'<' | cut -f1 -d'>' | sort -u); do
|
||||
IS_EXPECTED_INCLUDE=0
|
||||
for EXPECTED_BOOST_INCLUDE in "${EXPECTED_BOOST_INCLUDES[@]}"; do
|
||||
if [[ "${BOOST_INCLUDE}" == "${EXPECTED_BOOST_INCLUDE}" ]]; then
|
||||
IS_EXPECTED_INCLUDE=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
if [[ ${IS_EXPECTED_INCLUDE} == 0 ]]; then
|
||||
EXIT_CODE=1
|
||||
echo "A new Boost dependency in the form of \"${BOOST_INCLUDE}\" appears to have been introduced:"
|
||||
git grep "${BOOST_INCLUDE}" -- "*.cpp" "*.h"
|
||||
echo
|
||||
fi
|
||||
done
|
||||
|
||||
for EXPECTED_BOOST_INCLUDE in "${EXPECTED_BOOST_INCLUDES[@]}"; do
|
||||
if ! git grep -q "^#include <${EXPECTED_BOOST_INCLUDE}>" -- "*.cpp" "*.h"; then
|
||||
echo "Good job! The Boost dependency \"${EXPECTED_BOOST_INCLUDE}\" is no longer used."
|
||||
echo "Please remove it from EXPECTED_BOOST_INCLUDES in $0"
|
||||
echo "to make sure this dependency is not accidentally reintroduced."
|
||||
echo
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
done
|
||||
|
||||
QUOTE_SYNTAX_INCLUDES=$(git grep '^#include "' -- "*.cpp" "*.h" | grep -Ev "${IGNORE_REGEXP}")
|
||||
if [[ ${QUOTE_SYNTAX_INCLUDES} != "" ]]; then
|
||||
echo "Please use bracket syntax includes (\"#include <foo.h>\") instead of quote syntax includes:"
|
||||
echo "${QUOTE_SYNTAX_INCLUDES}"
|
||||
echo
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
|
||||
exit ${EXIT_CODE}
|
264
test/lint/lint-locale-dependence.py
Executable file
264
test/lint/lint-locale-dependence.py
Executable file
@ -0,0 +1,264 @@
|
||||
#!/usr/bin/env python3
|
||||
# Copyright (c) 2018-2022 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Be aware that bitcoind and bitcoin-qt differ in terms of localization: Qt
|
||||
# opts in to POSIX localization by running setlocale(LC_ALL, "") on startup,
|
||||
# whereas no such call is made in bitcoind.
|
||||
#
|
||||
# Qt runs setlocale(LC_ALL, "") on initialization. This installs the locale
|
||||
# specified by the user's LC_ALL (or LC_*) environment variable as the new
|
||||
# C locale.
|
||||
#
|
||||
# In contrast, bitcoind does not opt in to localization -- no call to
|
||||
# setlocale(LC_ALL, "") is made and the environment variables LC_* are
|
||||
# thus ignored.
|
||||
#
|
||||
# This results in situations where bitcoind is guaranteed to be running
|
||||
# with the classic locale ("C") whereas the locale of bitcoin-qt will vary
|
||||
# depending on the user's environment variables.
|
||||
#
|
||||
# An example: Assuming the environment variable LC_ALL=de_DE then the
|
||||
# call std::to_string(1.23) will return "1.230000" in bitcoind but
|
||||
# "1,230000" in bitcoin-qt.
|
||||
#
|
||||
# From the Qt documentation:
|
||||
# "On Unix/Linux Qt is configured to use the system locale settings by default.
|
||||
# This can cause a conflict when using POSIX functions, for instance, when
|
||||
# converting between data types such as floats and strings, since the notation
|
||||
# may differ between locales. To get around this problem, call the POSIX function
|
||||
# setlocale(LC_NUMERIC,"C") right after initializing QApplication, QGuiApplication
|
||||
# or QCoreApplication to reset the locale that is used for number formatting to
|
||||
# "C"-locale."
|
||||
#
|
||||
# See https://doc.qt.io/qt-5/qcoreapplication.html#locale-settings and
|
||||
# https://stackoverflow.com/a/34878283 for more details.
|
||||
#
|
||||
# TODO: Reduce KNOWN_VIOLATIONS by replacing uses of locale dependent stoul/strtol with locale
|
||||
# independent ToIntegral<T>(...).
|
||||
# TODO: Reduce KNOWN_VIOLATIONS by replacing uses of locale dependent snprintf with strprintf.
|
||||
|
||||
import re
|
||||
import sys
|
||||
|
||||
from subprocess import check_output, CalledProcessError
|
||||
|
||||
|
||||
KNOWN_VIOLATIONS = [
|
||||
"src/bitcoin-tx.cpp.*stoul",
|
||||
"src/dbwrapper.cpp:.*vsnprintf",
|
||||
"src/test/dbwrapper_tests.cpp:.*snprintf",
|
||||
"src/test/fuzz/locale.cpp",
|
||||
"src/test/fuzz/string.cpp",
|
||||
"src/util/strencodings.cpp:.*strtoll",
|
||||
"src/util/system.cpp:.*fprintf"
|
||||
]
|
||||
|
||||
REGEXP_EXTERNAL_DEPENDENCIES_EXCLUSIONS = [
|
||||
"src/crypto/ctaes/",
|
||||
"src/leveldb/",
|
||||
"src/secp256k1/",
|
||||
"src/minisketch/",
|
||||
"src/tinyformat.h",
|
||||
"src/univalue/",
|
||||
"src/dashbls/",
|
||||
"src/immer/"
|
||||
]
|
||||
|
||||
LOCALE_DEPENDENT_FUNCTIONS = [
|
||||
"alphasort", # LC_COLLATE (via strcoll)
|
||||
"asctime", # LC_TIME (directly)
|
||||
"asprintf", # (via vasprintf)
|
||||
"atof", # LC_NUMERIC (via strtod)
|
||||
"atoi", # LC_NUMERIC (via strtol)
|
||||
"atol", # LC_NUMERIC (via strtol)
|
||||
"atoll", # (via strtoll)
|
||||
"atoq",
|
||||
"btowc", # LC_CTYPE (directly)
|
||||
"ctime", # (via asctime or localtime)
|
||||
"dprintf", # (via vdprintf)
|
||||
"fgetwc",
|
||||
"fgetws",
|
||||
"fold_case", # boost::locale::fold_case
|
||||
"fprintf", # (via vfprintf)
|
||||
"fputwc",
|
||||
"fputws",
|
||||
"fscanf", # (via __vfscanf)
|
||||
"fwprintf", # (via __vfwprintf)
|
||||
"getdate", # via __getdate_r => isspace // __localtime_r
|
||||
"getwc",
|
||||
"getwchar",
|
||||
"is_digit", # boost::algorithm::is_digit
|
||||
"is_space", # boost::algorithm::is_space
|
||||
"isalnum", # LC_CTYPE
|
||||
"isalpha", # LC_CTYPE
|
||||
"isblank", # LC_CTYPE
|
||||
"iscntrl", # LC_CTYPE
|
||||
"isctype", # LC_CTYPE
|
||||
"isdigit", # LC_CTYPE
|
||||
"isgraph", # LC_CTYPE
|
||||
"islower", # LC_CTYPE
|
||||
"isprint", # LC_CTYPE
|
||||
"ispunct", # LC_CTYPE
|
||||
"isspace", # LC_CTYPE
|
||||
"isupper", # LC_CTYPE
|
||||
"iswalnum", # LC_CTYPE
|
||||
"iswalpha", # LC_CTYPE
|
||||
"iswblank", # LC_CTYPE
|
||||
"iswcntrl", # LC_CTYPE
|
||||
"iswctype", # LC_CTYPE
|
||||
"iswdigit", # LC_CTYPE
|
||||
"iswgraph", # LC_CTYPE
|
||||
"iswlower", # LC_CTYPE
|
||||
"iswprint", # LC_CTYPE
|
||||
"iswpunct", # LC_CTYPE
|
||||
"iswspace", # LC_CTYPE
|
||||
"iswupper", # LC_CTYPE
|
||||
"iswxdigit", # LC_CTYPE
|
||||
"isxdigit", # LC_CTYPE
|
||||
"localeconv", # LC_NUMERIC + LC_MONETARY
|
||||
"mblen", # LC_CTYPE
|
||||
"mbrlen",
|
||||
"mbrtowc",
|
||||
"mbsinit",
|
||||
"mbsnrtowcs",
|
||||
"mbsrtowcs",
|
||||
"mbstowcs", # LC_CTYPE
|
||||
"mbtowc", # LC_CTYPE
|
||||
"mktime",
|
||||
"normalize", # boost::locale::normalize
|
||||
"printf", # LC_NUMERIC
|
||||
"putwc",
|
||||
"putwchar",
|
||||
"scanf", # LC_NUMERIC
|
||||
"setlocale",
|
||||
"snprintf",
|
||||
"sprintf",
|
||||
"sscanf",
|
||||
"std::locale::global",
|
||||
"std::to_string",
|
||||
"stod",
|
||||
"stof",
|
||||
"stoi",
|
||||
"stol",
|
||||
"stold",
|
||||
"stoll",
|
||||
"stoul",
|
||||
"stoull",
|
||||
"strcasecmp",
|
||||
"strcasestr",
|
||||
"strcoll", # LC_COLLATE
|
||||
#"strerror",
|
||||
"strfmon",
|
||||
"strftime", # LC_TIME
|
||||
"strncasecmp",
|
||||
"strptime",
|
||||
"strtod", # LC_NUMERIC
|
||||
"strtof",
|
||||
"strtoimax",
|
||||
"strtol", # LC_NUMERIC
|
||||
"strtold",
|
||||
"strtoll",
|
||||
"strtoq",
|
||||
"strtoul", # LC_NUMERIC
|
||||
"strtoull",
|
||||
"strtoumax",
|
||||
"strtouq",
|
||||
"strxfrm", # LC_COLLATE
|
||||
"swprintf",
|
||||
"to_lower", # boost::locale::to_lower
|
||||
"to_title", # boost::locale::to_title
|
||||
"to_upper", # boost::locale::to_upper
|
||||
"tolower", # LC_CTYPE
|
||||
"toupper", # LC_CTYPE
|
||||
"towctrans",
|
||||
"towlower", # LC_CTYPE
|
||||
"towupper", # LC_CTYPE
|
||||
"trim", # boost::algorithm::trim
|
||||
"trim_left", # boost::algorithm::trim_left
|
||||
"trim_right", # boost::algorithm::trim_right
|
||||
"ungetwc",
|
||||
"vasprintf",
|
||||
"vdprintf",
|
||||
"versionsort",
|
||||
"vfprintf",
|
||||
"vfscanf",
|
||||
"vfwprintf",
|
||||
"vprintf",
|
||||
"vscanf",
|
||||
"vsnprintf",
|
||||
"vsprintf",
|
||||
"vsscanf",
|
||||
"vswprintf",
|
||||
"vwprintf",
|
||||
"wcrtomb",
|
||||
"wcscasecmp",
|
||||
"wcscoll", # LC_COLLATE
|
||||
"wcsftime", # LC_TIME
|
||||
"wcsncasecmp",
|
||||
"wcsnrtombs",
|
||||
"wcsrtombs",
|
||||
"wcstod", # LC_NUMERIC
|
||||
"wcstof",
|
||||
"wcstoimax",
|
||||
"wcstol", # LC_NUMERIC
|
||||
"wcstold",
|
||||
"wcstoll",
|
||||
"wcstombs", # LC_CTYPE
|
||||
"wcstoul", # LC_NUMERIC
|
||||
"wcstoull",
|
||||
"wcstoumax",
|
||||
"wcswidth",
|
||||
"wcsxfrm", # LC_COLLATE
|
||||
"wctob",
|
||||
"wctomb", # LC_CTYPE
|
||||
"wctrans",
|
||||
"wctype",
|
||||
"wcwidth",
|
||||
"wprintf"
|
||||
]
|
||||
|
||||
|
||||
def find_locale_dependent_function_uses():
|
||||
regexp_locale_dependent_functions = "|".join(LOCALE_DEPENDENT_FUNCTIONS)
|
||||
exclude_args = [":(exclude)" + excl for excl in REGEXP_EXTERNAL_DEPENDENCIES_EXCLUSIONS]
|
||||
git_grep_command = ["git", "grep", "-E", "[^a-zA-Z0-9_\\`'\"<>](" + regexp_locale_dependent_functions + "(_r|_s)?)[^a-zA-Z0-9_\\`'\"<>]", "--", "*.cpp", "*.h"] + exclude_args
|
||||
git_grep_output = list()
|
||||
|
||||
try:
|
||||
git_grep_output = check_output(git_grep_command, universal_newlines=True, encoding="utf8").splitlines()
|
||||
except CalledProcessError as e:
|
||||
if e.returncode > 1:
|
||||
raise e
|
||||
|
||||
return git_grep_output
|
||||
|
||||
|
||||
def main():
|
||||
exit_code = 0
|
||||
|
||||
regexp_ignore_known_violations = "|".join(KNOWN_VIOLATIONS)
|
||||
git_grep_output = find_locale_dependent_function_uses()
|
||||
|
||||
for locale_dependent_function in LOCALE_DEPENDENT_FUNCTIONS:
|
||||
matches = [line for line in git_grep_output
|
||||
if re.search("[^a-zA-Z0-9_\\`'\"<>]" + locale_dependent_function + "(_r|_s)?[^a-zA-Z0-9_\\`'\"<>]", line)
|
||||
and not re.search("\\.(c|cpp|h):\\s*(//|\\*|/\\*|\").*" + locale_dependent_function, line)
|
||||
and not re.search(regexp_ignore_known_violations, line)]
|
||||
if matches:
|
||||
print(f"The locale dependent function {locale_dependent_function}(...) appears to be used:")
|
||||
for match in matches:
|
||||
print(match)
|
||||
print("")
|
||||
exit_code = 1
|
||||
|
||||
if exit_code == 1:
|
||||
print("Unnecessary locale depedence can cause bugs that are very tricky to isolate and fix. Please avoid using locale dependent functions if possible.\n")
|
||||
print(f"Advice not applicable in this specific case? Add an exception by updating the ignore list in {sys.argv[0]}")
|
||||
|
||||
sys.exit(exit_code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@ -1,246 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Copyright (c) 2018-2020 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
export LC_ALL=C
|
||||
|
||||
# TODO: Reduce KNOWN_VIOLATIONS by replacing uses of locale dependent stoul/strtol with locale
|
||||
# independent ToIntegral<T>(...).
|
||||
# TODO: Reduce KNOWN_VIOLATIONS by replacing uses of locale dependent snprintf with strprintf.
|
||||
|
||||
# Be aware that bitcoind and bitcoin-qt differ in terms of localization: Qt
|
||||
# opts in to POSIX localization by running setlocale(LC_ALL, "") on startup,
|
||||
# whereas no such call is made in bitcoind.
|
||||
#
|
||||
# Qt runs setlocale(LC_ALL, "") on initialization. This installs the locale
|
||||
# specified by the user's LC_ALL (or LC_*) environment variable as the new
|
||||
# C locale.
|
||||
#
|
||||
# In contrast, bitcoind does not opt in to localization -- no call to
|
||||
# setlocale(LC_ALL, "") is made and the environment variables LC_* are
|
||||
# thus ignored.
|
||||
#
|
||||
# This results in situations where bitcoind is guaranteed to be running
|
||||
# with the classic locale ("C") whereas the locale of bitcoin-qt will vary
|
||||
# depending on the user's environment variables.
|
||||
#
|
||||
# An example: Assuming the environment variable LC_ALL=de_DE then the
|
||||
# call std::to_string(1.23) will return "1.230000" in bitcoind but
|
||||
# "1,230000" in bitcoin-qt.
|
||||
#
|
||||
# From the Qt documentation:
|
||||
# "On Unix/Linux Qt is configured to use the system locale settings by default.
|
||||
# This can cause a conflict when using POSIX functions, for instance, when
|
||||
# converting between data types such as floats and strings, since the notation
|
||||
# may differ between locales. To get around this problem, call the POSIX function
|
||||
# setlocale(LC_NUMERIC,"C") right after initializing QApplication, QGuiApplication
|
||||
# or QCoreApplication to reset the locale that is used for number formatting to
|
||||
# "C"-locale."
|
||||
#
|
||||
# See https://doc.qt.io/qt-5/qcoreapplication.html#locale-settings and
|
||||
# https://stackoverflow.com/a/34878283 for more details.
|
||||
|
||||
KNOWN_VIOLATIONS=(
|
||||
"src/bitcoin-tx.cpp.*stoul"
|
||||
"src/dbwrapper.cpp:.*vsnprintf"
|
||||
"src/test/dbwrapper_tests.cpp:.*snprintf"
|
||||
"src/test/fuzz/locale.cpp"
|
||||
"src/test/fuzz/string.cpp"
|
||||
"src/util/strencodings.cpp:.*strtoll"
|
||||
"src/util/system.cpp:.*fprintf"
|
||||
)
|
||||
|
||||
REGEXP_IGNORE_EXTERNAL_DEPENDENCIES="^src/(dashbls/|immer/|crypto/ctaes/|leveldb/|secp256k1/|minisketch/|tinyformat.h|univalue/)"
|
||||
|
||||
LOCALE_DEPENDENT_FUNCTIONS=(
|
||||
alphasort # LC_COLLATE (via strcoll)
|
||||
asctime # LC_TIME (directly)
|
||||
asprintf # (via vasprintf)
|
||||
atof # LC_NUMERIC (via strtod)
|
||||
atoi # LC_NUMERIC (via strtol)
|
||||
atol # LC_NUMERIC (via strtol)
|
||||
atoll # (via strtoll)
|
||||
atoq
|
||||
btowc # LC_CTYPE (directly)
|
||||
ctime # (via asctime or localtime)
|
||||
dprintf # (via vdprintf)
|
||||
fgetwc
|
||||
fgetws
|
||||
fold_case # boost::locale::fold_case
|
||||
fprintf # (via vfprintf)
|
||||
fputwc
|
||||
fputws
|
||||
fscanf # (via __vfscanf)
|
||||
fwprintf # (via __vfwprintf)
|
||||
getdate # via __getdate_r => isspace // __localtime_r
|
||||
getwc
|
||||
getwchar
|
||||
is_digit # boost::algorithm::is_digit
|
||||
is_space # boost::algorithm::is_space
|
||||
isalnum # LC_CTYPE
|
||||
isalpha # LC_CTYPE
|
||||
isblank # LC_CTYPE
|
||||
iscntrl # LC_CTYPE
|
||||
isctype # LC_CTYPE
|
||||
isdigit # LC_CTYPE
|
||||
isgraph # LC_CTYPE
|
||||
islower # LC_CTYPE
|
||||
isprint # LC_CTYPE
|
||||
ispunct # LC_CTYPE
|
||||
isspace # LC_CTYPE
|
||||
isupper # LC_CTYPE
|
||||
iswalnum # LC_CTYPE
|
||||
iswalpha # LC_CTYPE
|
||||
iswblank # LC_CTYPE
|
||||
iswcntrl # LC_CTYPE
|
||||
iswctype # LC_CTYPE
|
||||
iswdigit # LC_CTYPE
|
||||
iswgraph # LC_CTYPE
|
||||
iswlower # LC_CTYPE
|
||||
iswprint # LC_CTYPE
|
||||
iswpunct # LC_CTYPE
|
||||
iswspace # LC_CTYPE
|
||||
iswupper # LC_CTYPE
|
||||
iswxdigit # LC_CTYPE
|
||||
isxdigit # LC_CTYPE
|
||||
localeconv # LC_NUMERIC + LC_MONETARY
|
||||
mblen # LC_CTYPE
|
||||
mbrlen
|
||||
mbrtowc
|
||||
mbsinit
|
||||
mbsnrtowcs
|
||||
mbsrtowcs
|
||||
mbstowcs # LC_CTYPE
|
||||
mbtowc # LC_CTYPE
|
||||
mktime
|
||||
normalize # boost::locale::normalize
|
||||
printf # LC_NUMERIC
|
||||
putwc
|
||||
putwchar
|
||||
scanf # LC_NUMERIC
|
||||
setlocale
|
||||
snprintf
|
||||
sprintf
|
||||
sscanf
|
||||
std::locale::global
|
||||
std::to_string
|
||||
stod
|
||||
stof
|
||||
stoi
|
||||
stol
|
||||
stold
|
||||
stoll
|
||||
stoul
|
||||
stoull
|
||||
strcasecmp
|
||||
strcasestr
|
||||
strcoll # LC_COLLATE
|
||||
# strerror
|
||||
strfmon
|
||||
strftime # LC_TIME
|
||||
strncasecmp
|
||||
strptime
|
||||
strtod # LC_NUMERIC
|
||||
strtof
|
||||
strtoimax
|
||||
strtol # LC_NUMERIC
|
||||
strtold
|
||||
strtoll
|
||||
strtoq
|
||||
strtoul # LC_NUMERIC
|
||||
strtoull
|
||||
strtoumax
|
||||
strtouq
|
||||
strxfrm # LC_COLLATE
|
||||
swprintf
|
||||
to_lower # boost::locale::to_lower
|
||||
to_title # boost::locale::to_title
|
||||
to_upper # boost::locale::to_upper
|
||||
tolower # LC_CTYPE
|
||||
toupper # LC_CTYPE
|
||||
towctrans
|
||||
towlower # LC_CTYPE
|
||||
towupper # LC_CTYPE
|
||||
trim # boost::algorithm::trim
|
||||
trim_left # boost::algorithm::trim_left
|
||||
trim_right # boost::algorithm::trim_right
|
||||
ungetwc
|
||||
vasprintf
|
||||
vdprintf
|
||||
versionsort
|
||||
vfprintf
|
||||
vfscanf
|
||||
vfwprintf
|
||||
vprintf
|
||||
vscanf
|
||||
vsnprintf
|
||||
vsprintf
|
||||
vsscanf
|
||||
vswprintf
|
||||
vwprintf
|
||||
wcrtomb
|
||||
wcscasecmp
|
||||
wcscoll # LC_COLLATE
|
||||
wcsftime # LC_TIME
|
||||
wcsncasecmp
|
||||
wcsnrtombs
|
||||
wcsrtombs
|
||||
wcstod # LC_NUMERIC
|
||||
wcstof
|
||||
wcstoimax
|
||||
wcstol # LC_NUMERIC
|
||||
wcstold
|
||||
wcstoll
|
||||
wcstombs # LC_CTYPE
|
||||
wcstoul # LC_NUMERIC
|
||||
wcstoull
|
||||
wcstoumax
|
||||
wcswidth
|
||||
wcsxfrm # LC_COLLATE
|
||||
wctob
|
||||
wctomb # LC_CTYPE
|
||||
wctrans
|
||||
wctype
|
||||
wcwidth
|
||||
wprintf
|
||||
)
|
||||
|
||||
function join_array {
|
||||
local IFS="$1"
|
||||
shift
|
||||
echo "$*"
|
||||
}
|
||||
|
||||
REGEXP_IGNORE_KNOWN_VIOLATIONS=$(join_array "|" "${KNOWN_VIOLATIONS[@]}")
|
||||
|
||||
# Invoke "git grep" only once in order to minimize run-time
|
||||
REGEXP_LOCALE_DEPENDENT_FUNCTIONS=$(join_array "|" "${LOCALE_DEPENDENT_FUNCTIONS[@]}")
|
||||
GIT_GREP_OUTPUT=$(git grep -E "[^a-zA-Z0-9_\`'\"<>](${REGEXP_LOCALE_DEPENDENT_FUNCTIONS}(_r|_s)?)[^a-zA-Z0-9_\`'\"<>]" -- "*.cpp" "*.h")
|
||||
|
||||
EXIT_CODE=0
|
||||
for LOCALE_DEPENDENT_FUNCTION in "${LOCALE_DEPENDENT_FUNCTIONS[@]}"; do
|
||||
MATCHES=$(grep -E "[^a-zA-Z0-9_\`'\"<>]${LOCALE_DEPENDENT_FUNCTION}(_r|_s)?[^a-zA-Z0-9_\`'\"<>]" <<< "${GIT_GREP_OUTPUT}" | \
|
||||
grep -vE "\.(c|cpp|h):\s*(//|\*|/\*|\").*${LOCALE_DEPENDENT_FUNCTION}")
|
||||
if [[ ${REGEXP_IGNORE_EXTERNAL_DEPENDENCIES} != "" ]]; then
|
||||
MATCHES=$(grep -vE "${REGEXP_IGNORE_EXTERNAL_DEPENDENCIES}" <<< "${MATCHES}")
|
||||
fi
|
||||
if [[ ${REGEXP_IGNORE_KNOWN_VIOLATIONS} != "" ]]; then
|
||||
MATCHES=$(grep -vE "${REGEXP_IGNORE_KNOWN_VIOLATIONS}" <<< "${MATCHES}")
|
||||
fi
|
||||
if [[ ${MATCHES} != "" ]]; then
|
||||
echo "The locale dependent function ${LOCALE_DEPENDENT_FUNCTION}(...) appears to be used:"
|
||||
echo "${MATCHES}"
|
||||
echo
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
done
|
||||
if [[ ${EXIT_CODE} != 0 ]]; then
|
||||
echo "Unnecessary locale dependence can cause bugs that are very"
|
||||
echo "tricky to isolate and fix. Please avoid using locale dependent"
|
||||
echo "functions if possible."
|
||||
echo
|
||||
echo "Advice not applicable in this specific case? Add an exception"
|
||||
echo "by updating the ignore list in $0"
|
||||
fi
|
||||
exit ${EXIT_CODE}
|
34
test/lint/lint-logs.py
Executable file
34
test/lint/lint-logs.py
Executable file
@ -0,0 +1,34 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright (c) 2018-2022 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Check that all logs are terminated with '\n'
|
||||
#
|
||||
# Some logs are continued over multiple lines. They should be explicitly
|
||||
# commented with /* Continued */
|
||||
|
||||
import re
|
||||
import sys
|
||||
|
||||
from subprocess import check_output
|
||||
|
||||
|
||||
def main():
|
||||
logs_list = check_output(["git", "grep", "--extended-regexp", r"(LogPrintLevel|LogPrintf?)\(", "--", "*.cpp"], universal_newlines=True, encoding="utf8").splitlines()
|
||||
|
||||
unterminated_logs = [line for line in logs_list if not re.search(r'(\\n"|/\* Continued \*/)', line)]
|
||||
|
||||
if unterminated_logs != []:
|
||||
print("All calls to LogPrintf(), LogPrint(), LogPrintLevel(), and WalletLogPrintf() should be terminated with \"\\n\".")
|
||||
print("")
|
||||
|
||||
for line in unterminated_logs:
|
||||
print(line)
|
||||
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@ -1,29 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Copyright (c) 2018-2019 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Check that all logs are terminated with '\n'
|
||||
#
|
||||
# Some logs are continued over multiple lines. They should be explicitly
|
||||
# commented with /* Continued */
|
||||
#
|
||||
# There are some instances of LogPrintf() in comments. Those can be
|
||||
# ignored
|
||||
|
||||
export LC_ALL=C
|
||||
|
||||
UNTERMINATED_LOGS=$(git grep --extended-regexp "(LogPrintLevel|LogPrintf?)\(" -- "*.cpp" | \
|
||||
grep -v '\\n"' | \
|
||||
grep -v '\.\.\.' | \
|
||||
grep -v "/\* Continued \*/" | \
|
||||
grep -v "LogPrint()" | \
|
||||
grep -v "LogPrintf()")
|
||||
if [[ ${UNTERMINATED_LOGS} != "" ]]; then
|
||||
# shellcheck disable=SC2028
|
||||
echo "All calls to LogPrintf(), LogPrint(), LogPrintLevel(), and WalletLogPrintf() should be terminated with \\n"
|
||||
echo
|
||||
echo "${UNTERMINATED_LOGS}"
|
||||
exit 1
|
||||
fi
|
41
test/lint/lint-python-dead-code.py
Executable file
41
test/lint/lint-python-dead-code.py
Executable file
@ -0,0 +1,41 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright (c) 2022 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
"""
|
||||
Find dead Python code.
|
||||
"""
|
||||
|
||||
from subprocess import check_output, STDOUT, CalledProcessError
|
||||
|
||||
FILES_ARGS = ['git', 'ls-files', '--', '*.py']
|
||||
|
||||
|
||||
def check_vulture_install():
|
||||
try:
|
||||
check_output(["vulture", "--version"])
|
||||
except FileNotFoundError:
|
||||
print("Skipping Python dead code linting since vulture is not installed. Install by running \"pip3 install vulture\"")
|
||||
exit(0)
|
||||
|
||||
|
||||
def main():
|
||||
check_vulture_install()
|
||||
|
||||
files = check_output(FILES_ARGS).decode("utf-8").splitlines()
|
||||
# --min-confidence 100 will only report code that is guaranteed to be unused within the analyzed files.
|
||||
# Any value below 100 introduces the risk of false positives, which would create an unacceptable maintenance burden.
|
||||
vulture_args = ['vulture', '--min-confidence=100'] + files
|
||||
|
||||
try:
|
||||
check_output(vulture_args, stderr=STDOUT)
|
||||
except CalledProcessError as e:
|
||||
print(e.output.decode("utf-8"), end="")
|
||||
print("Python dead code detection found some issues")
|
||||
exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@ -1,23 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Copyright (c) 2021 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Find dead Python code.
|
||||
|
||||
export LC_ALL=C
|
||||
|
||||
if ! command -v vulture > /dev/null; then
|
||||
echo "Skipping Python dead code linting since vulture is not installed. Install by running \"pip3 install vulture\""
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# --min-confidence 100 will only report code that is guaranteed to be unused within the analyzed files.
|
||||
# Any value below 100 introduces the risk of false positives, which would create an unacceptable maintenance burden.
|
||||
if ! vulture \
|
||||
--min-confidence 100 \
|
||||
$(git ls-files -- "*.py"); then
|
||||
echo "Python dead code detection found some issues"
|
||||
exit 1
|
||||
fi
|
@ -101,6 +101,7 @@ fi
|
||||
|
||||
EXIT_CODE=0
|
||||
|
||||
# shellcheck disable=SC2046
|
||||
if ! PYTHONWARNINGS="ignore" $FLAKECMD --ignore=B,C,E,F,I,N,W --select=$(IFS=","; echo "${enabled[*]}") $(
|
||||
if [[ $# == 0 ]]; then
|
||||
git ls-files "*.py" | grep -vE "src/(immer)/"
|
||||
@ -111,7 +112,8 @@ if ! PYTHONWARNINGS="ignore" $FLAKECMD --ignore=B,C,E,F,I,N,W --select=$(IFS=","
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
|
||||
if ! mypy --ignore-missing-imports --show-error-codes $(git ls-files "test/functional/*.py" "contrib/devtools/*.py" | grep -v contrib/devtools/github-merge.py) ; then
|
||||
mapfile -t FILES < <(git ls-files "test/functional/*.py" "contrib/devtools/*.py" | grep -v contrib/devtools/github-merge.py)
|
||||
if ! mypy --ignore-missing-imports --show-error-codes "${FILES[@]}"; then
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
|
||||
|
@ -29,8 +29,10 @@ fi
|
||||
|
||||
SHELLCHECK_CMD=(shellcheck --external-sources --check-sourced)
|
||||
EXCLUDE="--exclude=$(IFS=','; echo "${disabled[*]}")"
|
||||
SOURCED_FILES=$(git ls-files | xargs gawk '/^# shellcheck shell=/ {print FILENAME} {nextfile}') # Check shellcheck directive used for sourced files
|
||||
if ! "${SHELLCHECK_CMD[@]}" "$EXCLUDE" $SOURCED_FILES $(git ls-files -- '*.sh' | grep -vE 'src/(dashbls|immer|leveldb|secp256k1|minisketch|univalue)/'); then
|
||||
# Check shellcheck directive used for sourced files
|
||||
mapfile -t SOURCED_FILES < <(git ls-files | xargs gawk '/^# shellcheck shell=/ {print FILENAME} {nextfile}')
|
||||
mapfile -t FILES < <(git ls-files -- '*.sh' | grep -vE 'src/(dashbls|immer|leveldb|secp256k1|minisketch|univalue)/')
|
||||
if ! "${SHELLCHECK_CMD[@]}" "$EXCLUDE" "${SOURCED_FILES[@]}" "${FILES[@]}"; then
|
||||
EXIT_CODE=1
|
||||
fi
|
||||
|
||||
|
40
test/lint/lint-spelling.py
Executable file
40
test/lint/lint-spelling.py
Executable file
@ -0,0 +1,40 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright (c) 2022 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
|
||||
"""
|
||||
Warn in case of spelling errors.
|
||||
Note: Will exit successfully regardless of spelling errors.
|
||||
"""
|
||||
|
||||
from subprocess import check_output, STDOUT, CalledProcessError
|
||||
|
||||
IGNORE_WORDS_FILE = 'test/lint/spelling.ignore-words.txt'
|
||||
FILES_ARGS = ['git', 'ls-files', '--', ":(exclude)build-aux/m4/", ":(exclude)contrib/seeds/*.txt", ":(exclude)depends/", ":(exclude)doc/release-notes/", ":(exclude)src/bip39_english.h", ":(exclude)src/dashbls/", ":(exclude)src/crc32c/", ":(exclude)src/crypto/", ":(exclude)src/ctpl_stl.h", ":(exclude)src/cxxtimer.hpp", ":(exclude)src/util/expected.h", ":(exclude)src/immer/", ":(exclude)src/leveldb/", ":(exclude)src/qt/locale/", ":(exclude)src/qt/*.qrc", ":(exclude)src/secp256k1/", ":(exclude)src/minisketch/", ":(exclude)src/univalue/", ":(exclude)contrib/builder-keys/", ":(exclude)contrib/guix/patches"]
|
||||
|
||||
|
||||
def check_codespell_install():
|
||||
try:
|
||||
check_output(["codespell", "--version"])
|
||||
except FileNotFoundError:
|
||||
print("Skipping spell check linting since codespell is not installed.")
|
||||
exit(0)
|
||||
|
||||
|
||||
def main():
|
||||
check_codespell_install()
|
||||
|
||||
files = check_output(FILES_ARGS).decode("utf-8").splitlines()
|
||||
codespell_args = ['codespell', '--check-filenames', '--disable-colors', '--quiet-level=7', '--ignore-words={}'.format(IGNORE_WORDS_FILE)] + files
|
||||
|
||||
try:
|
||||
check_output(codespell_args, stderr=STDOUT)
|
||||
except CalledProcessError as e:
|
||||
print(e.output.decode("utf-8"), end="")
|
||||
print('^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in {}'.format(IGNORE_WORDS_FILE))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@ -1,20 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Copyright (c) 2018-2019 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Warn in case of spelling errors.
|
||||
# Note: Will exit successfully regardless of spelling errors.
|
||||
|
||||
export LC_ALL=C
|
||||
|
||||
if ! command -v codespell > /dev/null; then
|
||||
echo "Skipping spell check linting since codespell is not installed."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
IGNORE_WORDS_FILE="test/lint/spelling.ignore-words.txt"
|
||||
if ! codespell --check-filenames --disable-colors --quiet-level=7 --ignore-words=${IGNORE_WORDS_FILE} $(git ls-files -- ":(exclude)build-aux/m4/" ":(exclude)contrib/seeds/*.txt" ":(exclude)depends/" ":(exclude)doc/release-notes/" ":(exclude)src/bip39_english.h" ":(exclude)src/dashbls/" ":(exclude)src/crc32c/" ":(exclude)src/crypto/" ":(exclude)src/ctpl_stl.h" ":(exclude)src/cxxtimer.hpp" ":(exclude)src/util/expected.h" ":(exclude)src/immer/" ":(exclude)src/leveldb/" ":(exclude)src/qt/locale/" ":(exclude)src/qt/*.qrc" ":(exclude)src/secp256k1/" ":(exclude)src/minisketch/" ":(exclude)src/univalue/" ":(exclude)contrib/builder-keys/" ":(exclude)contrib/guix/patches"); then
|
||||
echo "^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in ${IGNORE_WORDS_FILE}"
|
||||
fi
|
138
test/lint/lint-whitespace.py
Executable file
138
test/lint/lint-whitespace.py
Executable file
@ -0,0 +1,138 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright (c) 2017-2022 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Check for new lines in diff that introduce trailing whitespace or
|
||||
# tab characters instead of spaces.
|
||||
|
||||
# We can't run this check unless we know the commit range for the PR.
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
from subprocess import check_output
|
||||
|
||||
EXCLUDED_DIRS = ["depends/patches/",
|
||||
"contrib/guix/patches/",
|
||||
"src/leveldb/",
|
||||
"src/crc32c/",
|
||||
"src/secp256k1/",
|
||||
"src/minisketch/",
|
||||
"src/univalue/",
|
||||
"src/dashbls/",
|
||||
"src/immer/",
|
||||
"src/util/expected.h",
|
||||
"doc/release-notes/",
|
||||
"src/qt/locale"]
|
||||
|
||||
def parse_args():
|
||||
"""Parse command line arguments."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="""
|
||||
Check for new lines in diff that introduce trailing whitespace
|
||||
or tab characters instead of spaces in unstaged changes, the
|
||||
previous n commits, or a commit-range.
|
||||
""",
|
||||
epilog=f"""
|
||||
You can manually set the commit-range with the COMMIT_RANGE
|
||||
environment variable (e.g. "COMMIT_RANGE='47ba2c3...ee50c9e'
|
||||
{sys.argv[0]}"). Defaults to current merge base when neither
|
||||
prev-commits nor the environment variable is set.
|
||||
""")
|
||||
|
||||
parser.add_argument("--prev-commits", "-p", required=False, help="The previous n commits to check")
|
||||
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def report_diff(selection):
|
||||
filename = ""
|
||||
seen = False
|
||||
seenln = False
|
||||
|
||||
print("The following changes were suspected:")
|
||||
|
||||
for line in selection:
|
||||
if re.match(r"^diff", line):
|
||||
filename = line
|
||||
seen = False
|
||||
elif re.match(r"^@@", line):
|
||||
linenumber = line
|
||||
seenln = False
|
||||
else:
|
||||
if not seen:
|
||||
# The first time a file is seen with trailing whitespace or a tab character, we print the
|
||||
# filename (preceded by a newline).
|
||||
print("")
|
||||
print(filename)
|
||||
seen = True
|
||||
if not seenln:
|
||||
print(linenumber)
|
||||
seenln = True
|
||||
print(line)
|
||||
|
||||
|
||||
def get_diff(commit_range, check_only_code):
|
||||
exclude_args = [":(exclude)" + dir for dir in EXCLUDED_DIRS]
|
||||
|
||||
if check_only_code:
|
||||
what_files = ["*.cpp", "*.h", "*.md", "*.py", "*.sh"]
|
||||
else:
|
||||
what_files = ["."]
|
||||
|
||||
diff = check_output(["git", "diff", "-U0", commit_range, "--"] + what_files + exclude_args, universal_newlines=True, encoding="utf8")
|
||||
|
||||
return diff
|
||||
|
||||
|
||||
def main():
|
||||
args = parse_args()
|
||||
|
||||
if not os.getenv("COMMIT_RANGE"):
|
||||
if args.prev_commits:
|
||||
commit_range = "HEAD~" + args.prev_commits + "...HEAD"
|
||||
else:
|
||||
# This assumes that the target branch of the pull request will be develop.
|
||||
merge_base = check_output(["git", "merge-base", "HEAD", "develop"], universal_newlines=True, encoding="utf8").rstrip("\n")
|
||||
commit_range = merge_base + "..HEAD"
|
||||
else:
|
||||
commit_range = os.getenv("COMMIT_RANGE")
|
||||
|
||||
whitespace_selection = []
|
||||
tab_selection = []
|
||||
|
||||
# Check if trailing whitespace was found in the diff.
|
||||
for line in get_diff(commit_range, check_only_code=False).splitlines():
|
||||
if re.match(r"^(diff --git|\@@|^\+.*\s+$)", line):
|
||||
whitespace_selection.append(line)
|
||||
|
||||
whitespace_additions = [i for i in whitespace_selection if i.startswith("+")]
|
||||
|
||||
# Check if tab characters were found in the diff.
|
||||
for line in get_diff(commit_range, check_only_code=True).splitlines():
|
||||
if re.match(r"^(diff --git|\@@|^\+.*\t)", line):
|
||||
tab_selection.append(line)
|
||||
|
||||
tab_additions = [i for i in tab_selection if i.startswith("+")]
|
||||
|
||||
ret = 0
|
||||
|
||||
if len(whitespace_additions) > 0:
|
||||
print("This diff appears to have added new lines with trailing whitespace.")
|
||||
report_diff(whitespace_selection)
|
||||
ret = 1
|
||||
|
||||
if len(tab_additions) > 0:
|
||||
print("This diff appears to have added new lines with tab characters instead of spaces.")
|
||||
report_diff(tab_selection)
|
||||
ret = 1
|
||||
|
||||
sys.exit(ret)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
@ -1,115 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Copyright (c) 2017-2020 The Bitcoin Core developers
|
||||
# Distributed under the MIT software license, see the accompanying
|
||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||
#
|
||||
# Check for new lines in diff that introduce trailing whitespace.
|
||||
|
||||
# We can't run this check unless we know the commit range for the PR.
|
||||
|
||||
export LC_ALL=C
|
||||
while getopts "?" opt; do
|
||||
case $opt in
|
||||
?)
|
||||
echo "Usage: $0 [N]"
|
||||
echo " COMMIT_RANGE='<commit range>' $0"
|
||||
echo " $0 -?"
|
||||
echo "Checks unstaged changes, the previous N commits, or a commit range."
|
||||
echo "COMMIT_RANGE='47ba2c3...ee50c9e' $0"
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ -z "${COMMIT_RANGE}" ]; then
|
||||
if [ -n "$1" ]; then
|
||||
COMMIT_RANGE="HEAD~$1...HEAD"
|
||||
else
|
||||
# This assumes that the target branch of the pull request will be develop.
|
||||
MERGE_BASE=$(git merge-base HEAD develop)
|
||||
COMMIT_RANGE="$MERGE_BASE..HEAD"
|
||||
fi
|
||||
fi
|
||||
|
||||
showdiff() {
|
||||
if ! git diff -U0 "${COMMIT_RANGE}" -- "." ":(exclude)depends/patches/" ":(exclude)contrib/guix/patches/" ":(exclude)src/dashbls/" ":(exclude)src/util/expected.h" ":(exclude)src/immer/" ":(exclude)src/leveldb/" ":(exclude)src/crc32c/" ":(exclude)src/secp256k1/" ":(exclude)src/minisketch/" ":(exclude)src/univalue/" ":(exclude)doc/release-notes/"; then
|
||||
echo "Failed to get a diff"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
showcodediff() {
|
||||
if ! git diff -U0 "${COMMIT_RANGE}" -- *.cpp *.h *.md *.py *.sh ":(exclude)src/dashbls/" ":(exclude)src/util/expected.h" ":(exclude)src/immer/" ":(exclude)src/leveldb/" ":(exclude)src/crc32c/" ":(exclude)src/secp256k1/" ":(exclude)src/minisketch/" ":(exclude)src/univalue/" ":(exclude)doc/release-notes/"; then
|
||||
echo "Failed to get a diff"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
RET=0
|
||||
|
||||
# Check if trailing whitespace was found in the diff.
|
||||
if showdiff | grep -E -q '^\+.*\s+$'; then
|
||||
echo "This diff appears to have added new lines with trailing whitespace."
|
||||
echo "The following changes were suspected:"
|
||||
FILENAME=""
|
||||
SEEN=0
|
||||
SEENLN=0
|
||||
while read -r line; do
|
||||
if [[ "$line" =~ ^diff ]]; then
|
||||
FILENAME="$line"
|
||||
SEEN=0
|
||||
elif [[ "$line" =~ ^@@ ]]; then
|
||||
LINENUMBER="$line"
|
||||
SEENLN=0
|
||||
else
|
||||
if [ "$SEEN" -eq 0 ]; then
|
||||
# The first time a file is seen with trailing whitespace, we print the
|
||||
# filename (preceded by a newline).
|
||||
echo
|
||||
echo "$FILENAME"
|
||||
SEEN=1
|
||||
fi
|
||||
if [ "$SEENLN" -eq 0 ]; then
|
||||
echo "$LINENUMBER"
|
||||
SEENLN=1
|
||||
fi
|
||||
echo "$line"
|
||||
fi
|
||||
done < <(showdiff | grep -E '^(diff --git |@@|\+.*\s+$)')
|
||||
RET=1
|
||||
fi
|
||||
|
||||
# Check if tab characters were found in the diff.
|
||||
if showcodediff | perl -nle '$MATCH++ if m{^\+.*\t}; END{exit 1 unless $MATCH>0}' > /dev/null; then
|
||||
echo "This diff appears to have added new lines with tab characters instead of spaces."
|
||||
echo "The following changes were suspected:"
|
||||
FILENAME=""
|
||||
SEEN=0
|
||||
SEENLN=0
|
||||
while read -r line; do
|
||||
if [[ "$line" =~ ^diff ]]; then
|
||||
FILENAME="$line"
|
||||
SEEN=0
|
||||
elif [[ "$line" =~ ^@@ ]]; then
|
||||
LINENUMBER="$line"
|
||||
SEENLN=0
|
||||
else
|
||||
if [ "$SEEN" -eq 0 ]; then
|
||||
# The first time a file is seen with a tab character, we print the
|
||||
# filename (preceded by a newline).
|
||||
echo
|
||||
echo "$FILENAME"
|
||||
SEEN=1
|
||||
fi
|
||||
if [ "$SEENLN" -eq 0 ]; then
|
||||
echo "$LINENUMBER"
|
||||
SEENLN=1
|
||||
fi
|
||||
echo "$line"
|
||||
fi
|
||||
done < <(showcodediff | perl -nle 'print if m{^(diff --git |@@|\+.*\t)}')
|
||||
RET=1
|
||||
fi
|
||||
|
||||
exit $RET
|
@ -11,8 +11,6 @@
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
from functools import partial
|
||||
from multiprocessing import Pool
|
||||
|
||||
FALSE_POSITIVES = [
|
||||
("src/batchedlogger.h", "strprintf(fmt, args...)"),
|
||||
@ -265,8 +263,17 @@ def count_format_specifiers(format_string):
|
||||
return n
|
||||
|
||||
|
||||
def handle_filename(filename, args):
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="This program checks that the number of arguments passed "
|
||||
"to a variadic format string function matches the number of format "
|
||||
"specifiers in the format string.")
|
||||
parser.add_argument("--skip-arguments", type=int, help="number of arguments before the format string "
|
||||
"argument (e.g. 1 in the case of fprintf)", default=0)
|
||||
parser.add_argument("function_name", help="function name (e.g. fprintf)", default=None)
|
||||
parser.add_argument("file", nargs="*", help="C++ source code file (e.g. foo.cpp)")
|
||||
args = parser.parse_args()
|
||||
exit_code = 0
|
||||
for filename in args.file:
|
||||
with open(filename, "r", encoding="utf-8") as f:
|
||||
for function_call_str in parse_function_calls(args.function_name, f.read()):
|
||||
parts = parse_function_call_and_arguments(args.function_name, function_call_str)
|
||||
@ -284,24 +291,7 @@ def handle_filename(filename, args):
|
||||
exit_code = 1
|
||||
print("{}: Expected {} argument(s) after format string but found {} argument(s): {}".format(f.name, format_specifier_count, argument_count, relevant_function_call_str))
|
||||
continue
|
||||
return exit_code
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="This program checks that the number of arguments passed "
|
||||
"to a variadic format string function matches the number of format "
|
||||
"specifiers in the format string.")
|
||||
parser.add_argument("--skip-arguments", type=int, help="number of arguments before the format string "
|
||||
"argument (e.g. 1 in the case of fprintf)", default=0)
|
||||
parser.add_argument("function_name", help="function name (e.g. fprintf)", default=None)
|
||||
parser.add_argument("file", nargs="*", help="C++ source code file (e.g. foo.cpp)")
|
||||
args = parser.parse_args()
|
||||
exit_codes = []
|
||||
|
||||
with Pool(8) as pool:
|
||||
exit_codes = pool.map(partial(handle_filename, args=args), args.file)
|
||||
|
||||
sys.exit(max(exit_codes))
|
||||
sys.exit(exit_code)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
Loading…
Reference in New Issue
Block a user