dash/test/lint/lint-python.sh

119 lines
4.9 KiB
Bash
Raw Permalink Normal View History

#!/usr/bin/env bash
#
# Copyright (c) 2017-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
#
# Check for specified flake8 warnings in python files.
export LC_ALL=C
Merge #18210: test: type hints in Python tests bd7e530f010d43816bb05d6f1590d1cd36cdaa2c This PR adds initial support for type hints checking in python scripts. (Kiminuo) Pull request description: This PR adds initial support for type hints checking in python scripts. Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention." [Mypy](https://mypy.readthedocs.io/en/latest/index.html) is used in `lint-python.sh` to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error. **Notes:** * [--ignore-missing-imports](https://mypy.readthedocs.io/en/latest/command_line.html#cmdoption-mypy-ignore-missing-imports) switch is passed on to `mypy` checker for now. The effect of this is that one does not need `# type: ignore` for `import zmq`. More information about import processing can be found [here](https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports). This can be changed in a follow-up PR, if it is deemed useful. * We are stuck with Python 3.5 until 04/2021 (see https://packages.ubuntu.com/xenial/python3). When Python version is bumped to 3.6+, one can change: ```python _opcode_instances = [] # type: List[CScriptOp] ``` to ```python _opcode_instances:List[CScriptOp] = [] ``` for type hints that are **not** function parameters and function return types. **Useful resources:** * https://docs.python.org/3.5/library/typing.html * https://www.python.org/dev/peps/pep-0484/ ACKs for top commit: fanquake: ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c - the type checking is not the most robust (there are things it fails to detect), but I think this is worth adopting (in a limited capacity while we maintain 3.5 compat). MarcoFalke: ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c fine with me Tree-SHA512: 21ef213915fb1dec6012f59ef17484e6c9e0abf542a316b63d5f21a7778ad5ebabf8961ef5fc8e5414726c2ee9c6ae07c7353fb4dd337f8fcef5791199c8987a
2020-06-03 16:01:29 +02:00
export MYPY_CACHE_DIR="${BASE_ROOT_DIR}/test/.mypy_cache"
enabled=(
E101 # indentation contains mixed spaces and tabs
E112 # expected an indented block
E113 # unexpected indentation
E115 # expected an indented block (comment)
E116 # unexpected indentation (comment)
E125 # continuation line with same indent as next logical line
E129 # visually indented line with same indent as next logical line
E131 # continuation line unaligned for hanging indent
E133 # closing bracket is missing indentation
E223 # tab before operator
E224 # tab after operator
E242 # tab after ','
E266 # too many leading '#' for block comment
E271 # multiple spaces after keyword
E272 # multiple spaces before keyword
E273 # tab after keyword
E274 # tab before keyword
test: multiple linter warnings to suppress or fix (#5880) ## Issue being fixed or feature implemented On my local kubuntu linters have way too much spam ## What was done? See each commit ## How Has This Been Tested? Run locally. Amount of warnings decreased from thousands to fewer amount. Excluding typos, they are: ``` src/coinjoin/client.cpp:1420:5: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm] src/coinjoin/client.cpp:1426:5: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm] src/coinjoin/client.cpp:655:26: warning: Consider using std::copy_if algorithm instead of a raw loop. [useStlAlgorithm] src/coinjoin/server.cpp:593:33: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm] src/coinjoin/server.cpp:630:106: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm] src/governance/governance.cpp:1057:9: warning: C-style pointer casting [cstyleCast] src/governance/governance.cpp:1068:9: warning: C-style pointer casting [cstyleCast] src/governance/governance.cpp:1079:13: warning: C-style pointer casting [cstyleCast] src/governance/governance.cpp:1086:9: warning: C-style pointer casting [cstyleCast] src/governance/governance.cpp:1094:9: warning: C-style pointer casting [cstyleCast] src/governance/governance.cpp:1099:5: warning: C-style pointer casting [cstyleCast] src/governance/governance.cpp:1486:34: warning: Consider using std::copy_if algorithm instead of a raw loop. [useStlAlgorithm] src/llmq/commitment.cpp:102:5: warning: Consider using std::all_of or std::none_of algorithm instead of a raw loop. [useStlAlgorithm] src/llmq/instantsend.cpp:820:38: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm] src/llmq/quorums.cpp:831:102: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm] src/llmq/quorums.h:300:17: warning: C-style pointer casting [cstyleCast] src/llmq/quorums.h:301:17: warning: C-style pointer casting [cstyleCast] src/llmq/quorums.h:302:17: warning: C-style pointer casting [cstyleCast] src/llmq/quorums.h:303:17: warning: C-style pointer casting [cstyleCast] src/spork.cpp:119:58: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm] src/statsd_client.cpp:234:63: warning: C-style pointer casting [cstyleCast] Advice not applicable in this specific case? Add an exception by updating IGNORED_WARNINGS in test/lint/lint-cppcheck-dash.sh ^---- failure generated from test/lint/lint-cppcheck-dash.sh Consider install flake8-cached for cached flake8 results. test/functional/data/invalid_txs.py: error: Source file found twice under different module names: "invalid_txs" and "data.invalid_txs" test/functional/data/invalid_txs.py: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules for more info test/functional/data/invalid_txs.py: note: Common resolutions include: a) adding `__init__.py` somewhere, b) using `--explicit-package-bases` or adjusting MYPYPATH Found 1 error in 1 file (errors prevented further checking) ^---- failure generated from test/lint/lint-python.s ``` ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone
2024-02-20 15:22:37 +01:00
# TODO: enable it after bitcoin/bitcoin#26257 - too many warnings with newer flake
#E275 # missing whitespace after keyword
E304 # blank lines found after function decorator
E306 # expected 1 blank line before a nested definition
E401 # multiple imports on one line
E402 # module level import not at top of file
E502 # the backslash is redundant between brackets
E701 # multiple statements on one line (colon)
E702 # multiple statements on one line (semicolon)
E703 # statement ends with a semicolon
E711 # comparison to None should be 'if cond is None:'
E714 # test for object identity should be "is not"
E721 # do not compare types, use "isinstance()"
E742 # do not define classes named "l", "O", or "I"
E743 # do not define functions named "l", "O", or "I"
E901 # SyntaxError: invalid syntax
E902 # TokenError: EOF in multi-line string
F401 # module imported but unused
F402 # import module from line N shadowed by loop variable
F403 # 'from foo_module import *' used; unable to detect undefined names
F404 # future import(s) name after other statements
F405 # foo_function may be undefined, or defined from star imports: bar_module
F406 # "from module import *" only allowed at module level
F407 # an undefined __future__ feature name was imported
F601 # dictionary key name repeated with different values
F602 # dictionary key variable name repeated with different values
F621 # too many expressions in an assignment with star-unpacking
F622 # two or more starred expressions in an assignment (a, *b, *c = d)
F631 # assertion test is a tuple, which are always True
F632 # use ==/!= to compare str, bytes, and int literals
F701 # a break statement outside of a while or for loop
F702 # a continue statement outside of a while or for loop
F703 # a continue statement in a finally block in a loop
F704 # a yield or yield from statement outside of a function
F705 # a return statement with arguments inside a generator
F706 # a return statement outside of a function/method
F707 # an except: block as not the last exception handler
F811 # redefinition of unused name from line N
F812 # list comprehension redefines 'foo' from line N
F821 # undefined name 'Foo'
F822 # undefined name name in __all__
F823 # local variable name … referenced before assignment
F831 # duplicate argument name in function definition
F841 # local variable 'foo' is assigned to but never used
W191 # indentation contains tabs
W291 # trailing whitespace
W292 # no newline at end of file
W293 # blank line contains whitespace
W601 # .has_key() is deprecated, use "in"
W602 # deprecated form of raising exception
W603 # "<>" is deprecated, use "!="
W604 # backticks are deprecated, use "repr()"
# W605 # invalid escape sequence "x"
W606 # 'async' and 'await' are reserved keywords starting with Python 3.7
)
if ! command -v flake8 > /dev/null; then
echo "Skipping Python linting since flake8 is not installed."
exit 0
elif PYTHONWARNINGS="ignore" flake8 --version | grep -q "Python 2"; then
echo "Skipping Python linting since flake8 is running under Python 2. Install the Python 3 version of flake8."
exit 0
fi
FLAKECMD=flake8
if command -v flake8-cached > /dev/null; then
FLAKECMD=flake8-cached
else
echo "Consider install flake8-cached for cached flake8 results."
fi
EXIT_CODE=0
if ! PYTHONWARNINGS="ignore" $FLAKECMD --ignore=B,C,E,F,I,N,W --select=$(IFS=","; echo "${enabled[*]}") $(
if [[ $# == 0 ]]; then
git ls-files "*.py" | grep -vE "src/(immer)/"
else
echo "$@"
fi
); then
EXIT_CODE=1
fi
if ! mypy --ignore-missing-imports $(git ls-files "test/functional/*.py" "contrib/devtools/*.py" | grep -v contrib/devtools/github-merge.py) ; then
EXIT_CODE=1
fi
Merge #18210: test: type hints in Python tests bd7e530f010d43816bb05d6f1590d1cd36cdaa2c This PR adds initial support for type hints checking in python scripts. (Kiminuo) Pull request description: This PR adds initial support for type hints checking in python scripts. Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention." [Mypy](https://mypy.readthedocs.io/en/latest/index.html) is used in `lint-python.sh` to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error. **Notes:** * [--ignore-missing-imports](https://mypy.readthedocs.io/en/latest/command_line.html#cmdoption-mypy-ignore-missing-imports) switch is passed on to `mypy` checker for now. The effect of this is that one does not need `# type: ignore` for `import zmq`. More information about import processing can be found [here](https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports). This can be changed in a follow-up PR, if it is deemed useful. * We are stuck with Python 3.5 until 04/2021 (see https://packages.ubuntu.com/xenial/python3). When Python version is bumped to 3.6+, one can change: ```python _opcode_instances = [] # type: List[CScriptOp] ``` to ```python _opcode_instances:List[CScriptOp] = [] ``` for type hints that are **not** function parameters and function return types. **Useful resources:** * https://docs.python.org/3.5/library/typing.html * https://www.python.org/dev/peps/pep-0484/ ACKs for top commit: fanquake: ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c - the type checking is not the most robust (there are things it fails to detect), but I think this is worth adopting (in a limited capacity while we maintain 3.5 compat). MarcoFalke: ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c fine with me Tree-SHA512: 21ef213915fb1dec6012f59ef17484e6c9e0abf542a316b63d5f21a7778ad5ebabf8961ef5fc8e5414726c2ee9c6ae07c7353fb4dd337f8fcef5791199c8987a
2020-06-03 16:01:29 +02:00
exit $EXIT_CODE