21be609b49 In lint-format-strings, open files sequentially (Glenn Willen)
Pull request description:
In lint-format-strings, we use python argparse to read our file arguments. In
this mode, argparse opens all the files simultaneously. On OS X, where the
default filehandle limit is 128, this causes the lint to fail. Instead, ask
argparse for our filename arguments as strings, and open them one at a time
using 'with open'.
Tree-SHA512: 4c7dabf98818a7c5d83ab10c61b89a26957fe399e39e933e30c561cb45c5e8ba6f6aedcde8343da0c32ee340289a8897db6a33708e35ee381334ee27e3f4d356
341f7c7b0e macOS fix: Check for correct version of flake8 to avoid spurious warnings. The brew installed flake8 version is Python 2 based and does not work. (practicalswift)
908a559f33 macOS fix: Add excludes for checks added in the newer shellcheck version installed by brew (practicalswift)
ec4d57bbb3 macOS fix: Work around empty (sub)expression error when using BSD grep (practicalswift)
b57d7d92fe macOS fix: Avoid mapfile due to ancient version of bash shipped with macOS (practicalswift)
Pull request description:
The linters are thoroughly tested under Ubuntu which is what we use in Travis. When reading #14041 I understood that some developers were experiencing problems when running the linters on their local machines.
Assuming these local machines were running macOS I installed a fresh macOS VM, followed the instructions in `build-osx.md` and ran the linters.
This PR contains the changes needed to make `lint-all.sh` run as expected.
Ideally the linters would continuously run also under a Travis macOS environment to make sure we catch these kind of issues before merge.
Tree-SHA512: b39c9a970d14d27db1fb592539923c0bc676b5217f415d02fda3f17bf54d46faa172376e8a3ecab07ca68a3acba9aebe00b2b1b2161b2a36b85fbb672e7efb5c
fa3c910bfeab00703c947c5200a64c21225b50ef test: Move linters to test/lint, add readme (MarcoFalke)
Pull request description:
This moves the checks and linters from `devtools` to a subfolder in `test`. (Motivated by my opinion that the dev tools are mostly for generating code and updating the repo whereas the linters are read-only checks.)
Also, adds a readme to clarify that checks and linters are only meant to prevent bugs and user facing issues, not merely stylistic preference or inconsistencies. (This is motivated by the diversity in developers and work flows as well as existing code styles. It would be too disruptive to change all existing code to a single style or too burdensome to force all developers to adhere to a single style. Also note that our style guide is changing, so locking in at the wrong style "too early" would only waste resources.)
Tree-SHA512: 9b10e89f2aeaf0c8a9ae248aa891d74e0abf0569f8e5dfd266446efa8bfaf19f0ea0980abf0b0b22f0d8416ee90d7435d21a9f9285b66df43f370b7979173406
bcd4b0f5cdde2a1b562a612c78ec1ef1fe47d3dd Add linting of WalletLogPrintf(...) format strings (practicalswift)
a3e455694901a887e0feef69bd63e3aa122ea44b build: Add format string linter (practicalswift)
Pull request description:
Add format string linter.
This linter checks that the number of arguments passed to each variadic format string function matches the number of format specifiers in the format string.
Example output:
```
$ test/lint/lint-format-strings.sh
src/init.cpp: Expected 2 argument(s) after format string but found 1 argument(s):
LogPrintf("We have a mismatch here: foo=%s bar=%d\n", foo)
src/init.cpp: Expected 1 argument(s) after format string but found 2 argument(s):
LogPrint(BCLog::RPC, "RPC stopped. This is a mismatch: %s\n", s1, s2)
$ echo $?
1
```
Tree-SHA512: 19ab844a63f04bf193d66682ca42745a1c7d6c454b30222491b9fe8dc047054c4a6d3ee7921ec0676fb9ca2e7f6f93bd6c97996fb09667269bd491cb875349f3
47776a958b08382d76d69b5df7beed807af168b3 Add linter: Make sure all shell scripts opt out of locale dependence using "export LC_ALL=C" (practicalswift)
3352da8da1243c03fc83ba678d2f5d193bd5a0c2 Add "export LC_ALL=C" to all shell scripts (practicalswift)
Pull request description:
~~Make sure `LC_ALL=C` is set when using `grep` range expressions.~~
Make sure `LC_ALL=C` is set in all shell scripts.
From the `grep(1)` documentation:
> Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and character set. For example, in the default C locale, `[a-d]` is equivalent to `[abcd]`. Many locales sort characters in dictionary order, and in these locales `[a-d]` is typically not equivalent to `[abcd]`; it might be equivalent to `[aBbCcDd]`, for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the `LC_ALL` environment variable to the value C.
Context: [Locale issue found when reviewing #13450](https://github.com/bitcoin/bitcoin/pull/13450/files#r194877736)
Tree-SHA512: fd74d2612998f9b49ef9be24410e505d8c842716f84d085157fc7f9799d40e8a7b4969de783afcf99b7fae4f91bbb4559651f7dd6578a6a081a50bdea29f0909
927e1150bc207181f7d5b0948e87d04b737a1c27 Test: Harden lint-filenames.sh (wodry)
Pull request description:
- This fixes that only files with lower case file name suffix where found before, which contradicted the Regex to find uppercase characters in file names (including suffixes I guess).
- `--full-name` switch was added to git ls-files, to define that the found file always includes it's full path in the git project.
- since we know now that the file name includes the full path, we can harden the Regex to exclude the secp256k1 and univalue sub folders.
- use backslash line break to make code easier to read and avoid too long line.
Tree-SHA512: 9b55fe4965ae2084112b9f8a81bf9c657756c2cb5004986e7b6102a76adaf62c7d7a53257d9f13c5d8a1c75870b52c744d13830e3edd454a099e810357c914e5
c8176b3cc7556d7bcec39a55ae4d6ba16453baaa Add linter: Make sure we explicitly open all text files using UTF-8 or ASCII encoding in Python (practicalswift)
634bd970013eca90f4b4c1f9044eec8c97ba62c2 Explicitly specify encoding when opening text files in Python code (practicalswift)
Pull request description:
Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python.
As requested by @laanwj in #13440.
Tree-SHA512: 1651c00fe220ceb273324abd6703aee504029b96c7ef0e3029145901762c733c9b9d24927da281394fd4681a5bff774336c04eed01fafea997bb32192c334c06
Signed-off-by: pasta <pasta@dashboost.org>
# Conflicts:
# contrib/devtools/circular-dependencies.py
# contrib/linearize/linearize-data.py
# contrib/linearize/linearize-hashes.py
# contrib/seeds/generate-seeds.py
# contrib/verify-commits/verify-commits.py
# test/functional/multiwallet.py
# test/functional/notifications.py
# test/functional/test_runner.py
# test/util/rpcauth-test.py
ad691f666b Add linter: Enforce the source code file naming convention described in the developer notes (practicalswift)
Pull request description:
Add linter: Enforce the source code file naming convention described in the developer notes.
Tree-SHA512: 6458acf5383de7e81554bdd954c3a74c2bbf26286687ea69d934f11174d2f6bd573e8d2c16a7e77bbd12065e65be7700ecd7791d215f286e18f346bf964cd17d
698cfd081144845f6246171b8a2a0cfa8daaecdb docs: Mention lint-locale-dependence.sh in developer-notes.md (practicalswift)
0a4ea2f4589961868ab4d25e0277485c31938e20 build: Add linter for checking accidental locale dependence (practicalswift)
Pull request description:
This linter will check for code accidentally introducing locale dependencies.
Unnecessary locale dependence can cause bugs that are very tricky to isolate and fix. We should avoid using locale dependent functions if possible.
Context: https://github.com/bitcoin/bitcoin/pull/12881#issuecomment-378564722
Example output:
```
$ contrib/devtools/lint-locale-dependence.sh
The locale dependent function tolower(...) appears to be used:
src/init.cpp: if (s[0] == '0' && std::tolower(s[1]) == 'x') {
Unnecessary locale dependence can cause bugs that are very
tricky to isolate and fix. Please avoid using locale dependent
functions if possible.
Advice not applicable in this specific case? Add an exception
by updating the ignore list in contrib/devtools/lint-locale-dependence.sh
```
**Note to reviewers:** What is the most appropriate `LOCALE_DEPENDENT_FUNCTIONS` function list? What should be added or removed?
Tree-SHA512: 14e448828804bb02bf59070647e38b52fce120c700c903a4a8472769a2cee5dd529bd3fc182386993cb8720482cf4250b63a0a477db61b941ae4babe5c65025f
