dash/test/lint/lint-locale-dependence.sh

224 lines
5.9 KiB
Bash
Raw Normal View History

#!/usr/bin/env bash
export LC_ALL=C
2020-07-29 15:29:56 +02:00
KNOWN_VIOLATIONS=(
2020-07-29 15:29:56 +02:00
"src/bench/string_cast.cpp.*atoi"
"src/dash-tx.cpp.*stoul"
"src/dash-tx.cpp.*trim_right"
"src/dbwrapper.cpp.*stoul"
"src/dbwrapper.cpp:.*vsnprintf"
"src/httprpc.cpp.*trim"
"src/init.cpp:.*atoi"
Merge #15278: Improve PID file error handling 3782075a5fd4ad0c15a6119e8cdaf136898f679e Move all PID file stuff to init.cpp (Hennadii Stepanov) 561e375c73a37934fe77a519762d81edf7a3325c Make PID file creating errors fatal (Hennadii Stepanov) 745a2ace18ce857bc712d7e66c8bad7c082c07e2 Improve PID file removing errors logging (Hennadii Stepanov) Pull request description: Digging into #15240 the lack of the proper logging has been discovered. Fixed by this PR. UPDATE (inspired by @laanwj's [comment](https://github.com/bitcoin/bitcoin/pull/15278#discussion_r252641810)): Not being able to create the PID file is fatal now. Output of `bitcoind`: ``` $ src/bitcoind -pid=/run/bitcoind/bitcoind.pid 2019-02-01T23:20:10Z Bitcoin Core version v0.17.99.0-561e375c7 (release build) 2019-02-01T23:20:10Z Assuming ancestors of block 0000000000000037a8cd3e06cd5edbfe9dd1dbcc5dacab279376ef7cfc2b4c75 have valid signatures. 2019-02-01T23:20:10Z Setting nMinimumChainWork=00000000000000000000000000000000000000000000007dbe94253893cbd463 2019-02-01T23:20:10Z Using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation 2019-02-01T23:20:10Z Using RdRand as an additional entropy source 2019-02-01T23:20:11Z Error: Unable to create the PID file '/run/bitcoind/bitcoind.pid': No such file or directory Error: Unable to create the PID file '/run/bitcoind/bitcoind.pid': No such file or directory 2019-02-01T23:20:11Z Shutdown: In progress... 2019-02-01T23:20:11Z Shutdown: Unable to remove PID file: File does not exist 2019-02-01T23:20:11Z Shutdown: done ``` Output of `bitcoin-qt`: ![screenshot from 2019-02-02 01-19-05](https://user-images.githubusercontent.com/32963518/52154886-9349b600-2688-11e9-8128-470f16790305.png) **Notes for reviewers** 1. `CreatePidFile()` has been moved from `util/system.cpp` to `init.cpp` for the following reasons: - to get the ability to use `InitError()` - now `init.cpp` contains code of both creating PID file and removing it 2. Regarding 0.18 release process: this PR modifies 1 string and introduces 2 new ones. Tree-SHA512: ac07d0f800e61ec759e427d0afc0ca43d67f232e977662253963afdd0a220d34b871050f58149fc9fabd427bfc8e0d3f6a6032f2a38f30ad366fc0d074b0f2b3 Merge #15278: Improve PID file error handling
2021-09-06 14:23:00 +02:00
"src/init.cpp:.*fprintf"
"src/qt/rpcconsole.cpp:.*atoi"
"src/rest.cpp:.*strtol"
2020-07-29 15:29:56 +02:00
"src/rpc/blockchain.cpp.*atoi"
"src/rpc/governance.cpp.*atoi"
"src/rpc/masternode.cpp.*atoi"
Backporting Statoshi and bitcoin#16728 (#2515) * Backport Statoshi This backports some of https://github.com/jlopp/statoshi. Missing stuff: README.md and client name changes, segwit and fee estimation stats. Fix RejectCodeToString Fix copy-paste mistake s/InvalidBlockFound/InvalidChainFound/ * Merge #16728: move-only: move coins statistics utils out of RPC 8a3b2eb17572ca2131778d52cc25ec359470a90f move-only: move coins statistics utils out of RPC (James O'Beirne) Pull request description: This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11): Parent PR: #15606 Issue: #15605 Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal --- In the short-term, this move-only commit will help with fuzzing (https://github.com/bitcoin/bitcoin/pull/15606#issuecomment-524482297). Later, these procedures will be used to compute statistics (particularly a content hash) for UTXO sets coming in from snapshots. Most easily reviewed with `git ... --color-moved=dimmed_zebra`. A nice follow-up would be adding unittests, which I'll do if nobody else gets around to it. ACKs for top commit: MarcoFalke: ACK 8a3b2eb17572ca2131778d52cc25ec359470a90f, checked --color-moved=dimmed-zebra Tree-SHA512: a187d2f7590ad2450b8e8fa3d038c80a04fc3d903618c24222d7e3172250ce51badea35860c86101f2ba266eb4354e6efb8d7d508b353f29276e4665a1efdf74 * Fix 16728 * Modernize StatsdClient - Reuse some functionality from netbase - Switch from GetRand to FastRandomContext - Drop `using namespace std` and add `// namespace statsd` * Introduce PeriodicStats and make StatsdClient configurable via -stats<smth> (enabled/host/port/ns/period) * Move/rename tip stats from CheckBlock to ConnectBlock * Add new false positives to lint-format-strings.py * Add snprintf in statsd_client to the list of known violations in lint-locale-dependence.sh * Fix incorrect include guard * Use bracket syntax includes * Replace magic numbers with defaults * Move connection stats calculation into its own function And bail out early if stats are disabled * assert in PeriodicStats Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com> Co-authored-by: MarcoFalke <falke.marco@gmail.com> Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
2020-12-15 17:22:23 +01:00
"src/statsd_client.cpp:.*snprintf"
"src/test/dbwrapper_tests.cpp:.*snprintf"
"src/torcontrol.cpp:.*atoi"
"src/torcontrol.cpp:.*strtol"
merge bitcoin#14555: Move util files to directory (script modified to account for Dash backports, doesn't account for rebasing) ------------- BEGIN SCRIPT --------------- mkdir -p src/util git mv src/util.h src/util/system.h git mv src/util.cpp src/util/system.cpp git mv src/utilmemory.h src/util/memory.h git mv src/utilmoneystr.h src/util/moneystr.h git mv src/utilmoneystr.cpp src/util/moneystr.cpp git mv src/utilstrencodings.h src/util/strencodings.h git mv src/utilstrencodings.cpp src/util/strencodings.cpp git mv src/utiltime.h src/util/time.h git mv src/utiltime.cpp src/util/time.cpp git mv src/utilasmap.h src/util/asmap.h git mv src/utilasmap.cpp src/util/asmap.cpp git mv src/utilstring.h src/util/string.h git mv src/utilstring.cpp src/util/string.cpp gsed -i 's/<util\.h>/<util\/system\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp') gsed -i 's/<utilmemory\.h>/<util\/memory\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp') gsed -i 's/<utilmoneystr\.h>/<util\/moneystr\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp') gsed -i 's/<utilstrencodings\.h>/<util\/strencodings\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp') gsed -i 's/<utiltime\.h>/<util\/time\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp') gsed -i 's/<utilasmap\.h>/<util\/asmap\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp') gsed -i 's/<utilstring\.h>/<util\/string\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp') gsed -i 's/BITCOIN_UTIL_H/BITCOIN_UTIL_SYSTEM_H/g' src/util/system.h gsed -i 's/BITCOIN_UTILMEMORY_H/BITCOIN_UTIL_MEMORY_H/g' src/util/memory.h gsed -i 's/BITCOIN_UTILMONEYSTR_H/BITCOIN_UTIL_MONEYSTR_H/g' src/util/moneystr.h gsed -i 's/BITCOIN_UTILSTRENCODINGS_H/BITCOIN_UTIL_STRENCODINGS_H/g' src/util/strencodings.h gsed -i 's/BITCOIN_UTILTIME_H/BITCOIN_UTIL_TIME_H/g' src/util/time.h gsed -i 's/BITCOIN_UTILASMAP_H/BITCOIN_UTIL_ASMAP_H/g' src/util/asmap.h gsed -i 's/BITCOIN_UTILSTRING_H/BITCOIN_UTIL_STRING_H/g' src/util/string.h gsed -i 's/ util\.\(h\|cpp\)/ util\/system\.\1/g' src/Makefile.am gsed -i 's/utilmemory\.\(h\|cpp\)/util\/memory\.\1/g' src/Makefile.am gsed -i 's/utilmoneystr\.\(h\|cpp\)/util\/moneystr\.\1/g' src/Makefile.am gsed -i 's/utilstrencodings\.\(h\|cpp\)/util\/strencodings\.\1/g' src/Makefile.am gsed -i 's/utiltime\.\(h\|cpp\)/util\/time\.\1/g' src/Makefile.am gsed -i 's/utilasmap\.\(h\|cpp\)/util\/asmap\.\1/g' src/Makefile.am gsed -i 's/utilstring\.\(h\|cpp\)/util\/string\.\1/g' src/Makefile.am gsed -i 's/-> util ->/-> util\/system ->/' test/lint/lint-circular-dependencies.sh gsed -i 's/src\/util\.cpp/src\/util\/system\.cpp/g' test/lint/lint-format-strings.py test/lint/lint-locale-dependence.sh gsed -i 's/src\/utilmoneystr\.cpp/src\/util\/moneystr\.cpp/g' test/lint/lint-locale-dependence.sh gsed -i 's/src\/utilstrencodings\.\(h\|cpp\)/src\/util\/strencodings\.\1/g' test/lint/lint-locale-dependence.sh ------------- END SCRIPT ---------------
2021-06-27 08:33:13 +02:00
"src/util/strencodings.cpp:.*atoi"
"src/util/strencodings.cpp:.*strtol"
"src/util/strencodings.cpp:.*strtoul"
"src/util/strencodings.h:.*atoi"
2020-07-29 15:29:56 +02:00
"src/wallet/wallet.cpp:.*atoi"
"src/util/system.cpp:.*atoi"
"src/util/system.cpp:.*fprintf"
)
REGEXP_IGNORE_EXTERNAL_DEPENDENCIES="^src/(crypto/ctaes/|leveldb/|secp256k1/|tinyformat.h|univalue/)"
LOCALE_DEPENDENT_FUNCTIONS=(
alphasort # LC_COLLATE (via strcoll)
asctime # LC_TIME (directly)
asprintf # (via vasprintf)
atof # LC_NUMERIC (via strtod)
atoi # LC_NUMERIC (via strtol)
atol # LC_NUMERIC (via strtol)
atoll # (via strtoll)
atoq
btowc # LC_CTYPE (directly)
ctime # (via asctime or localtime)
dprintf # (via vdprintf)
fgetwc
fgetws
fold_case # boost::locale::fold_case
fprintf # (via vfprintf)
fputwc
fputws
fscanf # (via __vfscanf)
fwprintf # (via __vfwprintf)
getdate # via __getdate_r => isspace // __localtime_r
getwc
getwchar
is_digit # boost::algorithm::is_digit
is_space # boost::algorithm::is_space
isalnum # LC_CTYPE
isalpha # LC_CTYPE
isblank # LC_CTYPE
iscntrl # LC_CTYPE
isctype # LC_CTYPE
isdigit # LC_CTYPE
isgraph # LC_CTYPE
islower # LC_CTYPE
isprint # LC_CTYPE
ispunct # LC_CTYPE
isspace # LC_CTYPE
isupper # LC_CTYPE
iswalnum # LC_CTYPE
iswalpha # LC_CTYPE
iswblank # LC_CTYPE
iswcntrl # LC_CTYPE
iswctype # LC_CTYPE
iswdigit # LC_CTYPE
iswgraph # LC_CTYPE
iswlower # LC_CTYPE
iswprint # LC_CTYPE
iswpunct # LC_CTYPE
iswspace # LC_CTYPE
iswupper # LC_CTYPE
iswxdigit # LC_CTYPE
isxdigit # LC_CTYPE
localeconv # LC_NUMERIC + LC_MONETARY
mblen # LC_CTYPE
mbrlen
mbrtowc
mbsinit
mbsnrtowcs
mbsrtowcs
mbstowcs # LC_CTYPE
mbtowc # LC_CTYPE
mktime
normalize # boost::locale::normalize
# printf # LC_NUMERIC
putwc
putwchar
scanf # LC_NUMERIC
setlocale
snprintf
sprintf
sscanf
stod
stof
stoi
stol
stold
stoll
stoul
stoull
strcasecmp
strcasestr
strcoll # LC_COLLATE
# strerror
strfmon
strftime # LC_TIME
strncasecmp
strptime
strtod # LC_NUMERIC
strtof
strtoimax
strtol # LC_NUMERIC
strtold
strtoll
strtoq
strtoul # LC_NUMERIC
strtoull
strtoumax
strtouq
strxfrm # LC_COLLATE
swprintf
to_lower # boost::locale::to_lower
to_title # boost::locale::to_title
to_upper # boost::locale::to_upper
tolower # LC_CTYPE
toupper # LC_CTYPE
towctrans
towlower # LC_CTYPE
towupper # LC_CTYPE
trim # boost::algorithm::trim
trim_left # boost::algorithm::trim_left
trim_right # boost::algorithm::trim_right
ungetwc
vasprintf
vdprintf
versionsort
vfprintf
vfscanf
vfwprintf
vprintf
vscanf
vsnprintf
vsprintf
vsscanf
vswprintf
vwprintf
wcrtomb
wcscasecmp
wcscoll # LC_COLLATE
wcsftime # LC_TIME
wcsncasecmp
wcsnrtombs
wcsrtombs
wcstod # LC_NUMERIC
wcstof
wcstoimax
wcstol # LC_NUMERIC
wcstold
wcstoll
wcstombs # LC_CTYPE
wcstoul # LC_NUMERIC
wcstoull
wcstoumax
wcswidth
wcsxfrm # LC_COLLATE
wctob
wctomb # LC_CTYPE
wctrans
wctype
wcwidth
wprintf
)
function join_array {
local IFS="$1"
shift
echo "$*"
}
REGEXP_IGNORE_KNOWN_VIOLATIONS=$(join_array "|" "${KNOWN_VIOLATIONS[@]}")
# Invoke "git grep" only once in order to minimize run-time
REGEXP_LOCALE_DEPENDENT_FUNCTIONS=$(join_array "|" "${LOCALE_DEPENDENT_FUNCTIONS[@]}")
GIT_GREP_OUTPUT=$(git grep -E "[^a-zA-Z0-9_\`'\"<>](${REGEXP_LOCALE_DEPENDENT_FUNCTIONS}(_r|_s)?)[^a-zA-Z0-9_\`'\"<>]" -- "*.cpp" "*.h")
EXIT_CODE=0
for LOCALE_DEPENDENT_FUNCTION in "${LOCALE_DEPENDENT_FUNCTIONS[@]}"; do
MATCHES=$(grep -E "[^a-zA-Z0-9_\`'\"<>]${LOCALE_DEPENDENT_FUNCTION}(_r|_s)?[^a-zA-Z0-9_\`'\"<>]" <<< "${GIT_GREP_OUTPUT}" | \
grep -vE "\.(c|cpp|h):\s*(//|\*|/\*|\").*${LOCALE_DEPENDENT_FUNCTION}" | \
grep -vE 'fprintf\(.*(stdout|stderr)')
if [[ ${REGEXP_IGNORE_EXTERNAL_DEPENDENCIES} != "" ]]; then
MATCHES=$(grep -vE "${REGEXP_IGNORE_EXTERNAL_DEPENDENCIES}" <<< "${MATCHES}")
fi
if [[ ${REGEXP_IGNORE_KNOWN_VIOLATIONS} != "" ]]; then
MATCHES=$(grep -vE "${REGEXP_IGNORE_KNOWN_VIOLATIONS}" <<< "${MATCHES}")
fi
if [[ ${MATCHES} != "" ]]; then
echo "The locale dependent function ${LOCALE_DEPENDENT_FUNCTION}(...) appears to be used:"
echo "${MATCHES}"
echo
EXIT_CODE=1
fi
done
if [[ ${EXIT_CODE} != 0 ]]; then
echo "Unnecessary locale dependence can cause bugs that are very"
echo "tricky to isolate and fix. Please avoid using locale dependent"
echo "functions if possible."
echo
echo "Advice not applicable in this specific case? Add an exception"
echo "by updating the ignore list in $0"
fi
exit ${EXIT_CODE}