dash/contrib/devtools
MarcoFalke 17acd6b472
Merge #12987: tests/tools: Enable additional Python flake8 rules for automatic linting via Travis
643aad17fa Enable additional flake8 rules (practicalswift)
f020aca297 Minor Python cleanups to make flake8 pass with the new rules enabled (practicalswift)

Pull request description:

  Enabled rules:

  ```
  * E242: tab after ','
  * E266: too many leading '#' for block comment
  * E401: multiple imports on one line
  * E402: module level import not at top of file
  * E701: multiple statements on one line (colon)
  * E901: SyntaxError: invalid syntax
  * E902: TokenError: EOF in multi-line string
  * F821: undefined name 'Foo'
  * W293: blank line contains whitespace
  * W606: 'async' and 'await' are reserved keywords starting with Python 3.7
  ```

  Note to reviewers:
  * In general we don't allow whitespace cleanups to existing code, but in order to allow for enabling Travis checking for these rules a few smaller whitespace cleanups had to made as part of this PR.
  * Use [this `?w=1` link](https://github.com/bitcoin/bitcoin/pull/12987/files?w=1) to show a diff without whitespace changes.

  Before this commit:

  ```
  $ flake8 -qq --statistics --ignore=B,C,E,F,I,N,W --select=E112,E113,E115,E116,E125,E131,E133,E223,E224,E242,E266,E271,E272,E273,E274,E275,E304,E306,E401,E402,E502,E701,E702,E703,E714,E721,E741,E742,E743,F401,E901,E902,F402,F404,F406,F407,F601,F602,F621,F622,F631,F701,F702,F703,F704,F705,F706,F707,F811,F812,F821,F822,F823,F831,F841,W292,W293,W504,W601,W602,W603,W604,W605,W606 .
  5     E266 too many leading '#' for block comment
  4     E401 multiple imports on one line
  6     E402 module level import not at top of file
  5     E701 multiple statements on one line (colon)
  1     F812 list comprehension redefines 'n' from line 159
  4     F821 undefined name 'ConnectionRefusedError'
  28    W293 blank line contains whitespace
  ```

  After this commit:

  ```
  $ flake8 -qq --statistics --ignore=B,C,E,F,I,N,W --select=E112,E113,E115,E116,E125,E131,E133,E223,E224,E242,E266,E271,E272,E273,E274,E275,E304,E306,E401,E402,E502,E701,E702,E703,E714,E721,E741,E742,E743,F401,E901,E902,F402,F404,F406,F407,F601,F602,F621,F622,F631,F701,F702,F703,F704,F705,F706,F707,F811,F812,F821,F822,F823,F831,F841,W292,W293,W504,W601,W602,W603,W604,W605,W606 .
  $
  ```

Tree-SHA512: fc7d5e752298a50d4248afc620ee2c173135b4ca008e48e02913ac968e5a24a5fd5396926047ec62f1d580d537434ccae01f249bb2f3338fa59dc630bf97ca7a
Signed-off-by: pasta <pasta@dashboost.org>
2020-07-22 12:12:46 -05:00
..
check-doc.py Merge #12820: contrib: Fix check-doc script regexes 2020-07-17 15:42:51 -05:00
check-rpc-mappings.py Merge #13448: Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python 2020-07-07 11:50:56 -05:00
circular-dependencies.py Merge #13228: Add script to detect circular dependencies between source modules 2020-07-18 03:08:13 -05:00
clang-format-diff.py Merge #13448: Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python 2020-07-07 11:50:56 -05:00
commit-script-check.sh Merge #10480: Improve commit-check-script.sh 2019-07-09 13:08:20 -05:00
copyright_header.py Merge #12987: tests/tools: Enable additional Python flake8 rules for automatic linting via Travis 2020-07-22 12:12:46 -05:00
gen-manpages.sh Merge #12308: contrib: Add support for out-of-tree builds in gen-manpages.sh 2020-06-17 14:29:55 -05:00
git-subtree-check.sh Merge #8700: [copyright] add MIT license headers to .sh scripts where missing 2018-01-11 13:20:57 +01:00
github-merge.py Merge #12987: tests/tools: Enable additional Python flake8 rules for automatic linting via Travis 2020-07-22 12:12:46 -05:00
lint-all.sh Merge #11300: Tests: Add a lint check for trailing whitespace *dash* no travis 2020-01-10 10:33:56 -06:00
lint-include-guards.sh fix compilation in headers that contain 'dashes' and adjust linter 2020-07-19 00:42:54 -05:00
lint-includes.sh Merge #13656: Remove the boost/algorithm/string/predicate.hpp dependency 2020-07-17 15:42:20 -05:00
lint-logs.sh Merge #12891: [logging] add lint-logs.sh to check for newline termination. 2020-06-11 23:20:48 -05:00
lint-python-shebang.sh Merge #12972: Add python3 script shebang lint 2020-06-27 10:43:32 -05:00
lint-python.sh Merge #12987: tests/tools: Enable additional Python flake8 rules for automatic linting via Travis 2020-07-22 12:12:46 -05:00
lint-tests.sh Merge #12895: tests: Add note about test suite name uniqueness requirement to developer notes 2020-06-11 23:20:48 -05:00
lint-whitespace.sh Merge #12572: [script] lint-whitespace: find errors more easily 2020-06-18 11:41:53 -05:00
optimize-pngs.py Merge #12987: tests/tools: Enable additional Python flake8 rules for automatic linting via Travis 2020-07-22 12:12:46 -05:00
README.md Merge #13228: Add script to detect circular dependencies between source modules 2020-07-18 03:08:13 -05:00
security-check.py Merge #10781: Python cleanups 2020-01-02 22:01:29 -06:00
split-debug.sh.in Merge #8188: Add armhf/aarch64 gitian builds 2017-12-22 17:20:42 +01:00
symbol-check.py Some Dashification (#3513) 2020-06-11 11:39:04 +03:00
test-security-check.py Merge #13448: Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python 2020-07-07 11:50:56 -05:00
update-css-files.py qt: Generalized css files, simple design changes, added scripts to keep track of color usage (#3508) 2020-06-26 20:48:20 +03:00
update-translations.py Merge #10781: Python cleanups 2020-01-02 22:01:29 -06:00

Contents

This directory contains tools for developers working on this repository.

check-doc.py

Check if all command line args are documented. The return value indicates the number of undocumented args.

clang-format-diff.py

A script to format unified git diffs according to .clang-format.

For instance, to format the last commit with 0 lines of context, the script should be called from the git root folder as follows.

git diff -U0 HEAD~1.. | ./contrib/devtools/clang-format-diff.py -p1 -i -v

copyright_header.py

Provides utilities for managing copyright headers of The Dash Core developers in repository source files. It has three subcommands:

$ ./copyright_header.py report <base_directory> [verbose]
$ ./copyright_header.py update <base_directory>
$ ./copyright_header.py insert <file>

Running these subcommands without arguments displays a usage string.

Produces a report of all copyright header notices found inside the source files of a repository. Useful to quickly visualize the state of the headers. Specifying verbose will list the full filenames of files of each category.

Updates all the copyright headers of The Dash Core developers which were changed in a year more recent than is listed. For example:

// Copyright (c) <firstYear>-<lastYear> The Dash Core developers

will be updated to:

// Copyright (c) <firstYear>-<lastModifiedYear> The Dash Core developers

where <lastModifiedYear> is obtained from the git log history.

This subcommand also handles copyright headers that have only a single year. In those cases:

// Copyright (c) <year> The Dash Core developers

will be updated to:

// Copyright (c) <year>-<lastModifiedYear> The Dash Core developers

where the update is appropriate.

Inserts a copyright header for The Dash Core developers at the top of the file in either Python or C++ style as determined by the file extension. If the file is a Python file and it has #! starting the first line, the header is inserted in the line below it.

The copyright dates will be set to be <year_introduced>-<current_year> where <year_introduced> is according to the git log history. If <year_introduced> is equal to <current_year>, it will be set as a single year rather than two hyphenated years.

If the file already has a copyright for The Dash Core developers, the script will exit.

gen-manpages.sh

A small script to automatically create manpages in ../../doc/man by running the release binaries with the -help option. This requires help2man which can be found at: https://www.gnu.org/software/help2man/

With in-tree builds this tool can be run from any directory within the repostitory. To use this tool with out-of-tree builds set BUILDDIR. For example:

BUILDDIR=$PWD/build contrib/devtools/gen-manpages.sh

git-subtree-check.sh

Run this script from the root of the repository to verify that a subtree matches the contents of the commit it claims to have been updated to.

To use, make sure that you have fetched the upstream repository branch in which the subtree is maintained:

Usage: git-subtree-check.sh DIR (COMMIT)

COMMIT may be omitted, in which case HEAD is used.

github-merge.py

A small script to automate merging pull-requests securely and sign them with GPG.

For example:

./github-merge.py 3077

(in any git repository) will help you merge pull request #3077 for the dashpay/dash repository.

What it does:

  • Fetch master and the pull request.
  • Locally construct a merge commit.
  • Show the diff that merge results in.
  • Ask you to verify the resulting source tree (so you can do a make check or whatever).
  • Ask you whether to GPG sign the merge commit.
  • Ask you whether to push the result upstream.

This means that there are no potential race conditions (where a pullreq gets updated while you're reviewing it, but before you click merge), and when using GPG signatures, that even a compromised GitHub couldn't mess with the sources.

Setup

Configuring the github-merge tool for the bitcoin repository is done in the following way:

git config githubmerge.repository dashpay/dash
git config githubmerge.testcmd "make -j4 check" (adapt to whatever you want to use for testing)
git config --global user.signingkey mykeyid (if you want to GPG sign)

optimize-pngs.py

A script to optimize png files in the dash repository (requires pngcrush).

security-check.py and test-security-check.py

Perform basic ELF security checks on a series of executables.

symbol-check.py

A script to check that the (Linux) executables produced by Gitian only contain allowed gcc, glibc and libstdc++ version symbols. This makes sure they are still compatible with the minimum supported Linux distribution versions.

Example usage after a Gitian build:

find ../gitian-builder/build -type f -executable | xargs python contrib/devtools/symbol-check.py 

If only supported symbols are used the return value will be 0 and the output will be empty.

If there are 'unsupported' symbols, the return value will be 1 a list like this will be printed:

.../64/test_dash: symbol memcpy from unsupported version GLIBC_2.14
.../64/test_dash: symbol __fdelt_chk from unsupported version GLIBC_2.15
.../64/test_dash: symbol std::out_of_range::~out_of_range() from unsupported version GLIBCXX_3.4.15
.../64/test_dash: symbol _ZNSt8__detail15_List_nod from unsupported version GLIBCXX_3.4.15

update-translations.py

Run this script from the root of the repository to update all translations from transifex. It will do the following automatically:

  • fetch all translations
  • post-process them into valid and committable format
  • add missing translations to the build system (TODO)

See doc/translation-process.md for more information.

circular-dependencies.py

Run this script from the root of the source tree (src/) to find circular dependencies in the source code. This looks only at which files include other files, treating the .cpp and .h file as one unit.

Example usage:

cd .../src
../contrib/devtools/circular-dependencies.py {*,*/*,*/*/*}.{h,cpp}