mirror of
https://github.com/dashpay/dash.git
synced 2024-12-24 11:32:46 +01:00
Merge #6429: backport: merge bitcoin#24794, #23524, #24902, #24915, #24916, #24929, #23506, #24840, #24982, partial bitcoin#25288 (lint backports: part 2)
2fa480a878partial bitcoin#25288: Reliably don't start itself (lint-all.py runs all tests twice) (Kittywhiskers Van Gogh)bda1e03b24merge bitcoin#24982: Port `lint-all.sh` to `lint-all.py` (Kittywhiskers Van Gogh)b054a0d894merge bitcoin#24840: port `lint-shell.sh` to python (Kittywhiskers Van Gogh)973ca7b46fmerge bitcoin#23506: Make more shell scripts verifiable by the `shellcheck` tool (Kittywhiskers Van Gogh)694c1a4582merge bitcoin#24929: convert shell locale linter test to Python (Kittywhiskers Van Gogh)2a7d32a5e6merge bitcoin#24916: Convert lint-python-utf8-encoding.sh to Python (Kittywhiskers Van Gogh)0321fa053amerge bitcoin#24915: Convert lint-circular-dependencies.sh to Python (Kittywhiskers Van Gogh)e3dc4b1e27merge bitcoin#24902: Convert lint-include-guards.sh to Python (Kittywhiskers Van Gogh)fc48a134b5merge bitcoin#23524: Fix typos in endif header comments (Kittywhiskers Van Gogh)1f8c3b5e95merge bitcoin#24794: Convert Python linter to Python (Kittywhiskers Van Gogh)110b6ac3dcpartial revert dash#4807: enable more multi-threading and caching in linters (Kittywhiskers Van Gogh) Pull request description: ## Additional Information * Depends on https://github.com/dashpay/dash/pull/6428 * The introduction in `flake8-cached` for `lint-python.sh` in [dash#4807](https://github.com/dashpay/dash/pull/4807) was reverted as this logic wasn't going to be ported over to the Python replacement as the `flake8-cached` repo has been archived since April 2023 ([source](https://github.com/jnoortheen/flake8-cached)) and we don't use it in CI through GitLab ([build](https://gitlab.com/dashpay/dash/-/jobs/8456994796#L144)) or GitHub Actions ([build](https://github.com/dashpay/dash/actions/runs/11981121905/job/33406844883#step:7:75)). * [bitcoin#25288](https://github.com/bitcoin/bitcoin/pull/25288) has been marked as partial as the change of the glob pattern from `{mod_path}/lint-*` to `{mod_path}/lint-*.py` as we still have `lint-cppcheck-dash.sh` around ([source](b88d9910a8/test/lint/lint-cppcheck-dash.sh)) (and the original `cppcheck` linter upstream was removed in [bitcoin#25091](https://github.com/bitcoin/bitcoin/pull/25091)). A Python port of that linter would allow for completing [bitcoin#25288](https://github.com/bitcoin/bitcoin/pull/25288). ## Breaking Changes None expected. ## Checklist - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)** - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: UdjinM6: utACK2fa480a878PastaPastaPasta: utACK2fa480a878Tree-SHA512: 48ddf11be11232df26051b39dfadac9f363d2f201b9f303cad6ddd54550e2f1881947061155da9d4eaf3f5a87cdd371368dc36b4d70eb81ff4c48a7a93af63ae
This commit is contained in:
pasta
commit
1e55310232
@ -25,7 +25,7 @@ if [ "$CHECK_DOC" = 1 ]; then
|
|||||||
# TODO: Check docs (re-enable after all Bitcoin PRs have been merged and docs fully fixed)
|
# TODO: Check docs (re-enable after all Bitcoin PRs have been merged and docs fully fixed)
|
||||||
#test/lint/check-doc.py
|
#test/lint/check-doc.py
|
||||||
# Run all linters
|
# Run all linters
|
||||||
test/lint/lint-all.sh
|
test/lint/all-lint.py
|
||||||
fi
|
fi
|
||||||
|
|
||||||
ccache --zero-stats --max-size=$CCACHE_SIZE
|
ccache --zero-stats --max-size=$CCACHE_SIZE
|
||||||
|
|||||||
@ -21,7 +21,7 @@ test/lint/git-subtree-check.sh src/minisketch
|
|||||||
test/lint/git-subtree-check.sh src/univalue
|
test/lint/git-subtree-check.sh src/univalue
|
||||||
test/lint/git-subtree-check.sh src/leveldb
|
test/lint/git-subtree-check.sh src/leveldb
|
||||||
test/lint/check-doc.py
|
test/lint/check-doc.py
|
||||||
test/lint/lint-all.sh
|
test/lint/all-lint.py
|
||||||
|
|
||||||
if [ "$CIRRUS_REPO_FULL_NAME" = "dashpay/dash" ] && [ -n "$CIRRUS_CRON" ]; then
|
if [ "$CIRRUS_REPO_FULL_NAME" = "dashpay/dash" ] && [ -n "$CIRRUS_CRON" ]; then
|
||||||
git log --merges --before="2 days ago" -1 --format='%H' > ./contrib/verify-commits/trusted-sha512-root-commit
|
git log --merges --before="2 days ago" -1 --format='%H' > ./contrib/verify-commits/trusted-sha512-root-commit
|
||||||
|
|||||||
@ -2,10 +2,10 @@
|
|||||||
export LC_ALL=C
|
export LC_ALL=C
|
||||||
set -e -o pipefail
|
set -e -o pipefail
|
||||||
|
|
||||||
# shellcheck source=../../shell/realpath.bash
|
# shellcheck source=contrib/shell/realpath.bash
|
||||||
source contrib/shell/realpath.bash
|
source contrib/shell/realpath.bash
|
||||||
|
|
||||||
# shellcheck source=../../shell/git-utils.bash
|
# shellcheck source=contrib/shell/git-utils.bash
|
||||||
source contrib/shell/git-utils.bash
|
source contrib/shell/git-utils.bash
|
||||||
|
|
||||||
################
|
################
|
||||||
|
|||||||
@ -97,4 +97,4 @@ private:
|
|||||||
CRollingBloomFilter m_discouraged GUARDED_BY(m_cs_banned) {50000, 0.000001};
|
CRollingBloomFilter m_discouraged GUARDED_BY(m_cs_banned) {50000, 0.000001};
|
||||||
};
|
};
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_BANMAN_H
|
||||||
|
|||||||
@ -35,4 +35,4 @@ public:
|
|||||||
void Flush();
|
void Flush();
|
||||||
};
|
};
|
||||||
|
|
||||||
#endif//BITCOIN_BATCHEDLOGGER_H
|
#endif // BITCOIN_BATCHEDLOGGER_H
|
||||||
|
|||||||
@ -132,4 +132,4 @@ constexpr int CalculateAmountPriority(CAmount nInputAmount)
|
|||||||
|
|
||||||
} // namespace CoinJoin
|
} // namespace CoinJoin
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_COINJOIN_COMMON_H
|
||||||
|
|||||||
@ -26,4 +26,4 @@ static constexpr CAmount COIN = 100000000;
|
|||||||
static constexpr CAmount MAX_MONEY = 21000000 * COIN;
|
static constexpr CAmount MAX_MONEY = 21000000 * COIN;
|
||||||
inline bool MoneyRange(const CAmount& nValue) { return (nValue >= 0 && nValue <= MAX_MONEY); }
|
inline bool MoneyRange(const CAmount& nValue) { return (nValue >= 0 && nValue <= MAX_MONEY); }
|
||||||
|
|
||||||
#endif // BITCOIN_CONSENSUS_AMOUNT_H
|
#endif // BITCOIN_CONSENSUS_AMOUNT_H
|
||||||
|
|||||||
@ -33,4 +33,4 @@ T* GetContext(const CoreContext& context) noexcept
|
|||||||
: nullptr;
|
: nullptr;
|
||||||
}
|
}
|
||||||
|
|
||||||
#endif // BITCOIN_CONTEXT_VARIANT_H
|
#endif // BITCOIN_CONTEXT_H
|
||||||
|
|||||||
@ -141,4 +141,4 @@ std::optional<CCreditPoolDiff> GetCreditPoolDiffForBlock(CCreditPoolManager& cpo
|
|||||||
const CBlock& block, const CBlockIndex* pindexPrev, const Consensus::Params& consensusParams,
|
const CBlock& block, const CBlockIndex* pindexPrev, const Consensus::Params& consensusParams,
|
||||||
const CAmount blockSubsidy, BlockValidationState& state);
|
const CAmount blockSubsidy, BlockValidationState& state);
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_EVO_CREDITPOOL_H
|
||||||
|
|||||||
@ -377,4 +377,4 @@ public:
|
|||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
#endif //BITCOIN_EVO_DMNSTATE_H
|
#endif // BITCOIN_EVO_DMNSTATE_H
|
||||||
|
|||||||
@ -81,6 +81,6 @@ public:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
} // namespace Governance
|
} // namespace Governance
|
||||||
#endif
|
|
||||||
|
#endif // BITCOIN_GOVERNANCE_COMMON_H
|
||||||
|
|||||||
@ -31,4 +31,4 @@ void InterruptREST();
|
|||||||
*/
|
*/
|
||||||
void StopREST();
|
void StopREST();
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_HTTPRPC_H
|
||||||
|
|||||||
@ -286,4 +286,4 @@ private:
|
|||||||
} // namespace sam
|
} // namespace sam
|
||||||
} // namespace i2p
|
} // namespace i2p
|
||||||
|
|
||||||
#endif /* BITCOIN_I2P_H */
|
#endif // BITCOIN_I2P_H
|
||||||
|
|||||||
@ -233,4 +233,4 @@ extern std::unique_ptr<CQuorumSnapshotManager> quorumSnapshotManager;
|
|||||||
|
|
||||||
} // namespace llmq
|
} // namespace llmq
|
||||||
|
|
||||||
#endif //BITCOIN_LLMQ_SNAPSHOT_H
|
#endif // BITCOIN_LLMQ_SNAPSHOT_H
|
||||||
|
|||||||
@ -50,4 +50,4 @@ struct PSBTAnalysis {
|
|||||||
*/
|
*/
|
||||||
PSBTAnalysis AnalyzePSBT(PartiallySignedTransaction psbtx);
|
PSBTAnalysis AnalyzePSBT(PartiallySignedTransaction psbtx);
|
||||||
|
|
||||||
#endif // BITCOIN_PSBT_H
|
#endif // BITCOIN_NODE_PSBT_H
|
||||||
|
|||||||
@ -66,4 +66,4 @@ public:
|
|||||||
SERIALIZE_METHODS(CFeeRate, obj) { READWRITE(obj.nSatoshisPerK); }
|
SERIALIZE_METHODS(CFeeRate, obj) { READWRITE(obj.nSatoshisPerK); }
|
||||||
};
|
};
|
||||||
|
|
||||||
#endif // BITCOIN_POLICY_FEERATE_H
|
#endif // BITCOIN_POLICY_FEERATE_H
|
||||||
|
|||||||
@ -14,4 +14,4 @@ void RandAddDynamicEnv(CSHA512& hasher);
|
|||||||
/** Gather non-cryptographic environment data that does not change over time. */
|
/** Gather non-cryptographic environment data that does not change over time. */
|
||||||
void RandAddStaticEnv(CSHA512& hasher);
|
void RandAddStaticEnv(CSHA512& hasher);
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_RANDOMENV_H
|
||||||
|
|||||||
@ -64,4 +64,4 @@ void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry,
|
|||||||
*/
|
*/
|
||||||
UniValue CreateUTXOSnapshot(NodeContext& node, CChainState& chainstate, CAutoFile& afile);
|
UniValue CreateUTXOSnapshot(NodeContext& node, CChainState& chainstate, CAutoFile& afile);
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_RPC_BLOCKCHAIN_H
|
||||||
|
|||||||
@ -42,4 +42,4 @@ bool GetTimestampIndex(CBlockTreeDB& block_tree_db, const uint32_t high, const u
|
|||||||
std::vector<uint256>& hashes)
|
std::vector<uint256>& hashes)
|
||||||
EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
|
EXCLUSIVE_LOCKS_REQUIRED(::cs_main);
|
||||||
|
|
||||||
#endif // BITCOIN_RPC_CLIENT_H
|
#endif // BITCOIN_RPC_INDEX_UTIL_H
|
||||||
|
|||||||
@ -152,4 +152,4 @@ public:
|
|||||||
size_t CallbacksPending() EXCLUSIVE_LOCKS_REQUIRED(!m_callbacks_mutex);
|
size_t CallbacksPending() EXCLUSIVE_LOCKS_REQUIRED(!m_callbacks_mutex);
|
||||||
};
|
};
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_SCHEDULER_H
|
||||||
|
|||||||
@ -36,4 +36,4 @@ bool RestartRequested();
|
|||||||
*/
|
*/
|
||||||
void WaitForShutdown();
|
void WaitForShutdown();
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_SHUTDOWN_H
|
||||||
|
|||||||
@ -295,4 +295,4 @@ template<typename C>
|
|||||||
return span.end();
|
return span.end();
|
||||||
}
|
}
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_SPAN_H
|
||||||
|
|||||||
@ -37,4 +37,4 @@ inline std::string GetExceptionWhat(const T& e)
|
|||||||
void RegisterPrettyTerminateHander();
|
void RegisterPrettyTerminateHander();
|
||||||
void RegisterPrettySignalHandlers();
|
void RegisterPrettySignalHandlers();
|
||||||
|
|
||||||
#endif//BITCOIN_STACKTRACES_H
|
#endif // BITCOIN_STACKTRACES_H
|
||||||
|
|||||||
@ -179,4 +179,4 @@ private:
|
|||||||
};
|
};
|
||||||
|
|
||||||
|
|
||||||
#endif // BITCOIN_TEST_BIGNUM_H
|
#endif // BITCOIN_TEST_SCRIPTNUM10_H
|
||||||
|
|||||||
@ -250,4 +250,4 @@ private:
|
|||||||
// define an implicit conversion here so that uint256 may be used directly in BOOST_CHECK_*
|
// define an implicit conversion here so that uint256 may be used directly in BOOST_CHECK_*
|
||||||
std::ostream& operator<<(std::ostream& os, const uint256& num);
|
std::ostream& operator<<(std::ostream& os, const uint256& num);
|
||||||
|
|
||||||
#endif
|
#endif // BITCOIN_TEST_UTIL_SETUP_COMMON_H
|
||||||
|
|||||||
@ -32,4 +32,4 @@ private:
|
|||||||
std::atomic<bool> flag;
|
std::atomic<bool> flag;
|
||||||
};
|
};
|
||||||
|
|
||||||
#endif //BITCOIN_THREADINTERRUPT_H
|
#endif // BITCOIN_THREADINTERRUPT_H
|
||||||
|
|||||||
@ -159,4 +159,4 @@ public:
|
|||||||
static void reconnect_cb(evutil_socket_t fd, short what, void *arg);
|
static void reconnect_cb(evutil_socket_t fd, short what, void *arg);
|
||||||
};
|
};
|
||||||
|
|
||||||
#endif /* BITCOIN_TORCONTROL_H */
|
#endif // BITCOIN_TORCONTROL_H
|
||||||
|
|||||||
@ -59,4 +59,4 @@ private:
|
|||||||
int m_fd{-1};
|
int m_fd{-1};
|
||||||
};
|
};
|
||||||
|
|
||||||
#endif /* BITCOIN_UTIL_EDGE_H */
|
#endif // BITCOIN_UTIL_EDGE_H
|
||||||
|
|||||||
@ -25,4 +25,4 @@ std::pair<bool,std::string> ReadBinaryFile(const fs::path &filename, size_t maxs
|
|||||||
*/
|
*/
|
||||||
bool WriteBinaryFile(const fs::path &filename, const std::string &data);
|
bool WriteBinaryFile(const fs::path &filename, const std::string &data);
|
||||||
|
|
||||||
#endif /* BITCOIN_UTIL_READWRITEFILE_H */
|
#endif // BITCOIN_UTIL_READWRITEFILE_H
|
||||||
|
|||||||
@ -55,4 +55,4 @@
|
|||||||
#endif
|
#endif
|
||||||
|
|
||||||
|
|
||||||
#endif /* BITCOIN_UTIL_TRACE_H */
|
#endif // BITCOIN_UTIL_TRACE_H
|
||||||
|
|||||||
@ -11,4 +11,4 @@ template <typename E>
|
|||||||
return static_cast<typename std::underlying_type<E>::type>(e);
|
return static_cast<typename std::underlying_type<E>::type>(e);
|
||||||
}
|
}
|
||||||
|
|
||||||
#endif //BITCOIN_UTIL_UNDERLYING_H
|
#endif // BITCOIN_UTIL_UNDERLYING_H
|
||||||
|
|||||||
@ -56,4 +56,4 @@ private:
|
|||||||
EdgeTriggeredEvents* m_edge_trig_events{nullptr};
|
EdgeTriggeredEvents* m_edge_trig_events{nullptr};
|
||||||
};
|
};
|
||||||
|
|
||||||
#endif /* BITCOIN_UTIL_WPIPE_H */
|
#endif // BITCOIN_UTIL_WPIPE_H
|
||||||
|
|||||||
@ -41,4 +41,4 @@ RPCHelpMan getaddressinfo();
|
|||||||
RPCHelpMan getrawchangeaddress();
|
RPCHelpMan getrawchangeaddress();
|
||||||
RPCHelpMan addmultisigaddress();
|
RPCHelpMan addmultisigaddress();
|
||||||
RPCHelpMan signrawtransactionwithwallet();
|
RPCHelpMan signrawtransactionwithwallet();
|
||||||
#endif //BITCOIN_WALLET_RPCWALLET_H
|
#endif // BITCOIN_WALLET_RPCWALLET_H
|
||||||
|
|||||||
@ -20,4 +20,4 @@ void SetfLargeWorkInvalidChainFound(bool flag);
|
|||||||
*/
|
*/
|
||||||
bilingual_str GetWarnings(bool verbose);
|
bilingual_str GetWarnings(bool verbose);
|
||||||
|
|
||||||
#endif // BITCOIN_WARNINGS_H
|
#endif // BITCOIN_WARNINGS_H
|
||||||
|
|||||||
@ -9,4 +9,4 @@ class CRPCTable;
|
|||||||
|
|
||||||
void RegisterZMQRPCCommands(CRPCTable& t);
|
void RegisterZMQRPCCommands(CRPCTable& t);
|
||||||
|
|
||||||
#endif // BITCOIN_ZMQ_ZMRRPC_H
|
#endif // BITCOIN_ZMQ_ZMQRPC_H
|
||||||
|
|||||||
@ -310,11 +310,11 @@ Use the `-v` option for verbose output.
|
|||||||
|
|
||||||
| Lint test | Dependency |
|
| Lint test | Dependency |
|
||||||
|-----------|:----------:|
|
|-----------|:----------:|
|
||||||
| [`lint-python.sh`](lint/lint-python.sh) | [flake8](https://gitlab.com/pycqa/flake8)
|
| [`lint-python.py`](lint/lint-python.py) | [flake8](https://gitlab.com/pycqa/flake8)
|
||||||
| [`lint-python.sh`](lint/lint-python.sh) | [mypy](https://github.com/python/mypy)
|
| [`lint-python.py`](lint/lint-python.py) | [mypy](https://github.com/python/mypy)
|
||||||
| [`lint-python.sh`](lint/lint-python.sh) | [pyzmq](https://github.com/zeromq/pyzmq)
|
| [`lint-python.py`](lint/lint-python.py) | [pyzmq](https://github.com/zeromq/pyzmq)
|
||||||
| [`lint-python-dead-code.py`](lint/lint-python-dead-code.py) | [vulture](https://github.com/jendrikseipp/vulture)
|
| [`lint-python-dead-code.py`](lint/lint-python-dead-code.py) | [vulture](https://github.com/jendrikseipp/vulture)
|
||||||
| [`lint-shell.sh`](lint/lint-shell.sh) | [ShellCheck](https://github.com/koalaman/shellcheck)
|
| [`lint-shell.py`](lint/lint-shell.py) | [ShellCheck](https://github.com/koalaman/shellcheck)
|
||||||
| [`lint-spelling.py`](lint/lint-spelling.py) | [codespell](https://github.com/codespell-project/codespell)
|
| [`lint-spelling.py`](lint/lint-spelling.py) | [codespell](https://github.com/codespell-project/codespell)
|
||||||
|
|
||||||
In use versions and install instructions are available in the [CI setup](../ci/lint/04_install.sh).
|
In use versions and install instructions are available in the [CI setup](../ci/lint/04_install.sh).
|
||||||
@ -332,7 +332,7 @@ test/lint/lint-files.py
|
|||||||
You can run all the shell-based lint tests by running:
|
You can run all the shell-based lint tests by running:
|
||||||
|
|
||||||
```
|
```
|
||||||
test/lint/lint-all.sh
|
test/lint/all-lint.py
|
||||||
```
|
```
|
||||||
|
|
||||||
# Writing functional tests
|
# Writing functional tests
|
||||||
|
|||||||
@ -39,6 +39,6 @@ To do so, add the upstream repository as remote:
|
|||||||
git remote add --fetch secp256k1 https://github.com/bitcoin-core/secp256k1.git
|
git remote add --fetch secp256k1 https://github.com/bitcoin-core/secp256k1.git
|
||||||
```
|
```
|
||||||
|
|
||||||
lint-all.sh
|
all-lint.py
|
||||||
===========
|
===========
|
||||||
Calls other scripts with the `lint-` prefix.
|
Calls other scripts with the `lint-` prefix.
|
||||||
|
|||||||
36
test/lint/all-lint.py
Executable file
36
test/lint/all-lint.py
Executable file
@ -0,0 +1,36 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
#
|
||||||
|
# Copyright (c) 2017-2022 The Bitcoin Core developers
|
||||||
|
# Distributed under the MIT software license, see the accompanying
|
||||||
|
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||||
|
#
|
||||||
|
# This script runs all test/lint/lint-* files, and fails if any exit
|
||||||
|
# with a non-zero status code.
|
||||||
|
|
||||||
|
from glob import glob
|
||||||
|
from os import path as os_path, remove
|
||||||
|
from pathlib import Path
|
||||||
|
from shutil import which
|
||||||
|
from subprocess import run
|
||||||
|
|
||||||
|
exit_code = 0
|
||||||
|
mod_path = Path(__file__).parent
|
||||||
|
lints = glob(f"{mod_path}/lint-*")
|
||||||
|
if which("parallel") and which("column"):
|
||||||
|
logfile = "parallel_out.log"
|
||||||
|
command = ["parallel", "--jobs", "100%", "--will-cite", "--joblog", logfile, ":::"] + lints
|
||||||
|
result = run(command)
|
||||||
|
if result.returncode != 0:
|
||||||
|
print(f"^---- failure generated")
|
||||||
|
exit_code = result.returncode
|
||||||
|
result = run(["column", "-t", logfile])
|
||||||
|
if os_path.isfile(logfile):
|
||||||
|
remove(logfile)
|
||||||
|
else:
|
||||||
|
for lint in lints:
|
||||||
|
result = run([lint])
|
||||||
|
if result.returncode != 0:
|
||||||
|
print(f"^---- failure generated from {lint.split('/')[-1]}")
|
||||||
|
exit_code |= result.returncode
|
||||||
|
|
||||||
|
exit(exit_code)
|
||||||
@ -1,46 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2017-2019 The Bitcoin Core developers
|
|
||||||
# Distributed under the MIT software license, see the accompanying
|
|
||||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
|
||||||
#
|
|
||||||
# This script runs all contrib/devtools/lint-* files, and fails if any exit
|
|
||||||
# with a non-zero status code.
|
|
||||||
|
|
||||||
# This script is intentionally locale dependent by not setting "export LC_ALL=C"
|
|
||||||
# in order to allow for the executed lint scripts to opt in or opt out of locale
|
|
||||||
# dependence themselves.
|
|
||||||
|
|
||||||
set -u
|
|
||||||
|
|
||||||
SCRIPTDIR=$(dirname "${BASH_SOURCE[0]}")
|
|
||||||
LINTALL=$(basename "${BASH_SOURCE[0]}")
|
|
||||||
|
|
||||||
EXIT_CODE=0
|
|
||||||
|
|
||||||
if ! command -v parallel > /dev/null; then
|
|
||||||
for f in "${SCRIPTDIR}"/lint-*; do
|
|
||||||
if [ "$(basename "$f")" != "$LINTALL" ]; then
|
|
||||||
if ! "$f"; then
|
|
||||||
echo "^---- failure generated from $f"
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
else
|
|
||||||
SCRIPTS=()
|
|
||||||
|
|
||||||
for f in "${SCRIPTDIR}"/lint-*; do
|
|
||||||
if [ "$(basename "$f")" != "$LINTALL" ]; then
|
|
||||||
SCRIPTS+=("$f")
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
if ! parallel --jobs 100% --will-cite --joblog parallel_out.log ::: "${SCRIPTS[@]}"; then
|
|
||||||
echo "^---- failure generated"
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
column -t parallel_out.log && rm parallel_out.log
|
|
||||||
fi
|
|
||||||
|
|
||||||
exit ${EXIT_CODE}
|
|
||||||
150
test/lint/lint-circular-dependencies.py
Executable file
150
test/lint/lint-circular-dependencies.py
Executable file
@ -0,0 +1,150 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
#
|
||||||
|
# Copyright (c) 2020-2022 The Bitcoin Core developers
|
||||||
|
# Distributed under the MIT software license, see the accompanying
|
||||||
|
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||||
|
#
|
||||||
|
# Check for circular dependencies
|
||||||
|
|
||||||
|
import glob
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
EXPECTED_CIRCULAR_DEPENDENCIES = (
|
||||||
|
"chainparamsbase -> util/system -> chainparamsbase",
|
||||||
|
"node/blockstorage -> validation -> node/blockstorage",
|
||||||
|
"index/coinstatsindex -> node/coinstats -> index/coinstatsindex",
|
||||||
|
"policy/fees -> txmempool -> policy/fees",
|
||||||
|
"qt/addresstablemodel -> qt/walletmodel -> qt/addresstablemodel",
|
||||||
|
"qt/recentrequeststablemodel -> qt/walletmodel -> qt/recentrequeststablemodel",
|
||||||
|
"qt/transactiontablemodel -> qt/walletmodel -> qt/transactiontablemodel",
|
||||||
|
"wallet/fees -> wallet/wallet -> wallet/fees",
|
||||||
|
"wallet/wallet -> wallet/walletdb -> wallet/wallet",
|
||||||
|
"node/coinstats -> validation -> node/coinstats",
|
||||||
|
# Dash
|
||||||
|
"banman -> common/bloom -> evo/assetlocktx -> llmq/quorums -> net -> banman",
|
||||||
|
"banman -> common/bloom -> evo/assetlocktx -> llmq/signing -> net_processing -> banman",
|
||||||
|
"coinjoin/client -> net_processing -> coinjoin/client",
|
||||||
|
"coinjoin/client -> net_processing -> coinjoin/context -> coinjoin/client",
|
||||||
|
"coinjoin/coinjoin -> llmq/chainlocks -> net -> coinjoin/coinjoin",
|
||||||
|
"coinjoin/context -> coinjoin/server -> net_processing -> coinjoin/context",
|
||||||
|
"coinjoin/server -> net_processing -> coinjoin/server",
|
||||||
|
"common/bloom -> evo/assetlocktx -> llmq/quorums -> net -> common/bloom",
|
||||||
|
"common/bloom -> evo/assetlocktx -> llmq/signing -> net_processing -> merkleblock -> common/bloom",
|
||||||
|
"consensus/tx_verify -> evo/assetlocktx -> validation -> consensus/tx_verify",
|
||||||
|
"consensus/tx_verify -> evo/assetlocktx -> validation -> txmempool -> consensus/tx_verify",
|
||||||
|
"core_io -> evo/cbtx -> evo/simplifiedmns -> core_io",
|
||||||
|
"dsnotificationinterface -> llmq/chainlocks -> node/blockstorage -> dsnotificationinterface",
|
||||||
|
"evo/assetlocktx -> validation -> txmempool -> evo/assetlocktx",
|
||||||
|
"evo/cbtx -> evo/simplifiedmns -> evo/cbtx",
|
||||||
|
"evo/chainhelper -> evo/specialtxman -> validation -> evo/chainhelper",
|
||||||
|
"evo/deterministicmns -> llmq/commitment -> evo/deterministicmns",
|
||||||
|
"evo/deterministicmns -> llmq/utils -> evo/deterministicmns",
|
||||||
|
"evo/deterministicmns -> llmq/utils -> llmq/snapshot -> evo/simplifiedmns -> evo/deterministicmns",
|
||||||
|
"evo/deterministicmns -> llmq/utils -> net -> evo/deterministicmns",
|
||||||
|
"evo/deterministicmns -> validation -> evo/deterministicmns",
|
||||||
|
"evo/deterministicmns -> validation -> txmempool -> evo/deterministicmns",
|
||||||
|
"evo/deterministicmns -> validationinterface -> evo/deterministicmns",
|
||||||
|
"evo/deterministicmns -> validationinterface -> governance/vote -> evo/deterministicmns",
|
||||||
|
"evo/mnhftx -> validation -> evo/mnhftx",
|
||||||
|
"evo/simplifiedmns -> llmq/blockprocessor -> llmq/utils -> llmq/snapshot -> evo/simplifiedmns",
|
||||||
|
"evo/specialtxman -> validation -> evo/specialtxman",
|
||||||
|
"governance/governance -> governance/object -> governance/governance",
|
||||||
|
"governance/governance -> masternode/sync -> governance/governance",
|
||||||
|
"governance/governance -> net_processing -> governance/governance",
|
||||||
|
"governance/governance -> validation -> governance/governance",
|
||||||
|
"governance/vote -> masternode/node -> validationinterface -> governance/vote",
|
||||||
|
"llmq/blockprocessor -> llmq/utils -> llmq/snapshot -> llmq/blockprocessor",
|
||||||
|
"llmq/chainlocks -> llmq/instantsend -> llmq/chainlocks",
|
||||||
|
"llmq/chainlocks -> llmq/instantsend -> net_processing -> llmq/chainlocks",
|
||||||
|
"llmq/chainlocks -> validation -> llmq/chainlocks",
|
||||||
|
"llmq/commitment -> llmq/utils -> llmq/snapshot -> llmq/commitment",
|
||||||
|
"llmq/context -> llmq/instantsend -> net_processing -> llmq/context",
|
||||||
|
"llmq/dkgsession -> llmq/dkgsessionmgr -> llmq/dkgsessionhandler -> llmq/dkgsession",
|
||||||
|
"llmq/dkgsessionhandler -> net_processing -> llmq/dkgsessionmgr -> llmq/dkgsessionhandler",
|
||||||
|
"llmq/instantsend -> net_processing -> llmq/instantsend",
|
||||||
|
"llmq/instantsend -> txmempool -> llmq/instantsend",
|
||||||
|
"llmq/instantsend -> validation -> llmq/instantsend",
|
||||||
|
"llmq/signing -> llmq/signing_shares -> llmq/signing",
|
||||||
|
"llmq/signing -> masternode/node -> validationinterface -> llmq/signing",
|
||||||
|
"llmq/signing -> net_processing -> llmq/signing",
|
||||||
|
"llmq/signing_shares -> net_processing -> llmq/signing_shares",
|
||||||
|
"logging -> util/system -> logging",
|
||||||
|
"logging -> util/system -> stacktraces -> logging",
|
||||||
|
"logging -> util/system -> sync -> logging",
|
||||||
|
"logging -> util/system -> sync -> logging/timer -> logging",
|
||||||
|
"logging -> util/system -> util/getuniquepath -> random -> logging",
|
||||||
|
"masternode/payments -> validation -> masternode/payments",
|
||||||
|
"masternode/sync -> validation -> masternode/sync",
|
||||||
|
"net -> netmessagemaker -> net",
|
||||||
|
"net_processing -> spork -> net_processing",
|
||||||
|
"netaddress -> netbase -> netaddress",
|
||||||
|
"policy/policy -> policy/settings -> policy/policy",
|
||||||
|
"qt/appearancewidget -> qt/guiutil -> qt/appearancewidget",
|
||||||
|
"qt/appearancewidget -> qt/guiutil -> qt/optionsdialog -> qt/appearancewidget",
|
||||||
|
"qt/bitcoinaddressvalidator -> qt/guiutil -> qt/bitcoinaddressvalidator",
|
||||||
|
"qt/bitcoingui -> qt/guiutil -> qt/bitcoingui",
|
||||||
|
"qt/guiutil -> qt/optionsdialog -> qt/guiutil",
|
||||||
|
"qt/guiutil -> qt/optionsdialog -> qt/optionsmodel -> qt/guiutil",
|
||||||
|
"qt/guiutil -> qt/qvalidatedlineedit -> qt/guiutil",
|
||||||
|
"rpc/blockchain -> rpc/server -> rpc/blockchain"
|
||||||
|
)
|
||||||
|
|
||||||
|
CODE_DIR = "src"
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
circular_dependencies = []
|
||||||
|
exit_code = 0
|
||||||
|
os.chdir(
|
||||||
|
CODE_DIR
|
||||||
|
) # We change dir before globbing since glob.glob's root_dir option is only available in Python 3.10
|
||||||
|
|
||||||
|
# Using glob.glob since subprocess.run's globbing won't work without shell=True
|
||||||
|
files = []
|
||||||
|
for path in ["*", "*/*", "*/*/*"]:
|
||||||
|
for extension in ["h", "cpp"]:
|
||||||
|
files.extend(glob.glob(f"{path}.{extension}"))
|
||||||
|
|
||||||
|
command = ["python3", "../contrib/devtools/circular-dependencies.py", *files]
|
||||||
|
dependencies_output = subprocess.run(
|
||||||
|
command,
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
universal_newlines=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
for dependency_str in dependencies_output.stdout.rstrip().split("\n"):
|
||||||
|
circular_dependencies.append(
|
||||||
|
re.sub("^Circular dependency: ", "", dependency_str)
|
||||||
|
)
|
||||||
|
|
||||||
|
# Check for an unexpected dependencies
|
||||||
|
for dependency in circular_dependencies:
|
||||||
|
if dependency not in EXPECTED_CIRCULAR_DEPENDENCIES:
|
||||||
|
exit_code = 1
|
||||||
|
print(
|
||||||
|
f'A new circular dependency in the form of "{dependency}" appears to have been introduced.\n',
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Check for missing expected dependencies
|
||||||
|
for expected_dependency in EXPECTED_CIRCULAR_DEPENDENCIES:
|
||||||
|
if expected_dependency not in circular_dependencies:
|
||||||
|
exit_code = 1
|
||||||
|
print(
|
||||||
|
f'Good job! The circular dependency "{expected_dependency}" is no longer present.',
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
f"Please remove it from EXPECTED_CIRCULAR_DEPENDENCIES in {__file__}",
|
||||||
|
)
|
||||||
|
print(
|
||||||
|
"to make sure this circular dependency is not accidentally reintroduced.\n",
|
||||||
|
)
|
||||||
|
|
||||||
|
sys.exit(exit_code)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@ -1,133 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2018-2020 The Bitcoin Core developers
|
|
||||||
# Distributed under the MIT software license, see the accompanying
|
|
||||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
|
||||||
#
|
|
||||||
# Check for circular dependencies
|
|
||||||
|
|
||||||
export LC_ALL=C
|
|
||||||
|
|
||||||
EXPECTED_CIRCULAR_DEPENDENCIES=(
|
|
||||||
"chainparamsbase -> util/system -> chainparamsbase"
|
|
||||||
"node/blockstorage -> validation -> node/blockstorage"
|
|
||||||
"index/coinstatsindex -> node/coinstats -> index/coinstatsindex"
|
|
||||||
"policy/fees -> txmempool -> policy/fees"
|
|
||||||
"qt/addresstablemodel -> qt/walletmodel -> qt/addresstablemodel"
|
|
||||||
"qt/recentrequeststablemodel -> qt/walletmodel -> qt/recentrequeststablemodel"
|
|
||||||
"qt/transactiontablemodel -> qt/walletmodel -> qt/transactiontablemodel"
|
|
||||||
"wallet/fees -> wallet/wallet -> wallet/fees"
|
|
||||||
"wallet/wallet -> wallet/walletdb -> wallet/wallet"
|
|
||||||
"node/coinstats -> validation -> node/coinstats"
|
|
||||||
# Dash
|
|
||||||
"dsnotificationinterface -> llmq/chainlocks -> node/blockstorage -> dsnotificationinterface"
|
|
||||||
"evo/cbtx -> evo/simplifiedmns -> evo/cbtx"
|
|
||||||
"evo/deterministicmns -> llmq/commitment -> evo/deterministicmns"
|
|
||||||
"evo/deterministicmns -> llmq/utils -> evo/deterministicmns"
|
|
||||||
"governance/governance -> governance/object -> governance/governance"
|
|
||||||
"governance/governance -> masternode/sync -> governance/governance"
|
|
||||||
"llmq/chainlocks -> llmq/instantsend -> llmq/chainlocks"
|
|
||||||
"llmq/dkgsessionhandler -> net_processing -> llmq/dkgsessionmgr -> llmq/dkgsessionhandler"
|
|
||||||
"llmq/instantsend -> net_processing -> llmq/instantsend"
|
|
||||||
"llmq/instantsend -> txmempool -> llmq/instantsend"
|
|
||||||
"llmq/instantsend -> validation -> llmq/instantsend"
|
|
||||||
"llmq/signing -> llmq/signing_shares -> llmq/signing"
|
|
||||||
"llmq/signing -> net_processing -> llmq/signing"
|
|
||||||
"llmq/signing_shares -> net_processing -> llmq/signing_shares"
|
|
||||||
"logging -> util/system -> logging"
|
|
||||||
"masternode/payments -> validation -> masternode/payments"
|
|
||||||
"masternode/sync -> validation -> masternode/sync"
|
|
||||||
"net -> netmessagemaker -> net"
|
|
||||||
"netaddress -> netbase -> netaddress"
|
|
||||||
"qt/appearancewidget -> qt/guiutil -> qt/appearancewidget"
|
|
||||||
"qt/bitcoinaddressvalidator -> qt/guiutil -> qt/bitcoinaddressvalidator"
|
|
||||||
"qt/bitcoingui -> qt/guiutil -> qt/bitcoingui"
|
|
||||||
"qt/guiutil -> qt/optionsdialog -> qt/guiutil"
|
|
||||||
"qt/guiutil -> qt/qvalidatedlineedit -> qt/guiutil"
|
|
||||||
"core_io -> evo/cbtx -> evo/simplifiedmns -> core_io"
|
|
||||||
"llmq/dkgsession -> llmq/dkgsessionmgr -> llmq/dkgsessionhandler -> llmq/dkgsession"
|
|
||||||
"logging -> util/system -> sync -> logging"
|
|
||||||
"logging -> util/system -> stacktraces -> logging"
|
|
||||||
"logging -> util/system -> util/getuniquepath -> random -> logging"
|
|
||||||
"qt/appearancewidget -> qt/guiutil -> qt/optionsdialog -> qt/appearancewidget"
|
|
||||||
"qt/guiutil -> qt/optionsdialog -> qt/optionsmodel -> qt/guiutil"
|
|
||||||
|
|
||||||
"common/bloom -> evo/assetlocktx -> llmq/quorums -> net -> common/bloom"
|
|
||||||
"common/bloom -> evo/assetlocktx -> llmq/signing -> net_processing -> merkleblock -> common/bloom"
|
|
||||||
"banman -> common/bloom -> evo/assetlocktx -> llmq/quorums -> net -> banman"
|
|
||||||
"banman -> common/bloom -> evo/assetlocktx -> llmq/signing -> net_processing -> banman"
|
|
||||||
|
|
||||||
"llmq/chainlocks -> validation -> llmq/chainlocks"
|
|
||||||
"coinjoin/coinjoin -> llmq/chainlocks -> net -> coinjoin/coinjoin"
|
|
||||||
"evo/assetlocktx -> validation -> txmempool -> evo/assetlocktx"
|
|
||||||
"evo/deterministicmns -> llmq/utils -> llmq/snapshot -> evo/simplifiedmns -> evo/deterministicmns"
|
|
||||||
"evo/deterministicmns -> llmq/utils -> net -> evo/deterministicmns"
|
|
||||||
"evo/deterministicmns -> validation -> txmempool -> evo/deterministicmns"
|
|
||||||
"policy/policy -> policy/settings -> policy/policy"
|
|
||||||
"consensus/tx_verify -> evo/assetlocktx -> validation -> consensus/tx_verify"
|
|
||||||
"consensus/tx_verify -> evo/assetlocktx -> validation -> txmempool -> consensus/tx_verify"
|
|
||||||
|
|
||||||
"evo/simplifiedmns -> llmq/blockprocessor -> llmq/utils -> llmq/snapshot -> evo/simplifiedmns"
|
|
||||||
"llmq/blockprocessor -> llmq/utils -> llmq/snapshot -> llmq/blockprocessor"
|
|
||||||
"llmq/commitment -> llmq/utils -> llmq/snapshot -> llmq/commitment"
|
|
||||||
"governance/governance -> validation -> governance/governance"
|
|
||||||
"evo/deterministicmns -> validationinterface -> governance/vote -> evo/deterministicmns"
|
|
||||||
"governance/vote -> masternode/node -> validationinterface -> governance/vote"
|
|
||||||
"llmq/signing -> masternode/node -> validationinterface -> llmq/signing"
|
|
||||||
"evo/mnhftx -> validation -> evo/mnhftx"
|
|
||||||
"evo/deterministicmns -> validation -> evo/deterministicmns"
|
|
||||||
"evo/specialtxman -> validation -> evo/specialtxman"
|
|
||||||
"evo/chainhelper -> evo/specialtxman -> validation -> evo/chainhelper"
|
|
||||||
"evo/deterministicmns -> validationinterface -> evo/deterministicmns"
|
|
||||||
"logging -> util/system -> sync -> logging/timer -> logging"
|
|
||||||
|
|
||||||
"coinjoin/client -> net_processing -> coinjoin/client"
|
|
||||||
"coinjoin/client -> net_processing -> coinjoin/context -> coinjoin/client"
|
|
||||||
"coinjoin/context -> coinjoin/server -> net_processing -> coinjoin/context"
|
|
||||||
"coinjoin/server -> net_processing -> coinjoin/server"
|
|
||||||
"llmq/context -> llmq/instantsend -> net_processing -> llmq/context"
|
|
||||||
"llmq/chainlocks -> llmq/instantsend -> net_processing -> llmq/chainlocks"
|
|
||||||
"net_processing -> spork -> net_processing"
|
|
||||||
"governance/governance -> net_processing -> governance/governance"
|
|
||||||
"rpc/blockchain -> rpc/server -> rpc/blockchain"
|
|
||||||
)
|
|
||||||
|
|
||||||
EXIT_CODE=0
|
|
||||||
|
|
||||||
CIRCULAR_DEPENDENCIES=()
|
|
||||||
|
|
||||||
IFS=$'\n'
|
|
||||||
for CIRC in $(cd src && ../contrib/devtools/circular-dependencies.py {*,*/*,*/*/*}.{h,cpp} | sed -e 's/^Circular dependency: //'); do
|
|
||||||
CIRCULAR_DEPENDENCIES+=( "$CIRC" )
|
|
||||||
IS_EXPECTED_CIRC=0
|
|
||||||
for EXPECTED_CIRC in "${EXPECTED_CIRCULAR_DEPENDENCIES[@]}"; do
|
|
||||||
if [[ "${CIRC}" == "${EXPECTED_CIRC}" ]]; then
|
|
||||||
IS_EXPECTED_CIRC=1
|
|
||||||
break
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
if [[ ${IS_EXPECTED_CIRC} == 0 ]]; then
|
|
||||||
echo "A new circular dependency in the form of \"${CIRC}\" appears to have been introduced."
|
|
||||||
echo
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
for EXPECTED_CIRC in "${EXPECTED_CIRCULAR_DEPENDENCIES[@]}"; do
|
|
||||||
IS_PRESENT_EXPECTED_CIRC=0
|
|
||||||
for CIRC in "${CIRCULAR_DEPENDENCIES[@]}"; do
|
|
||||||
if [[ "${CIRC}" == "${EXPECTED_CIRC}" ]]; then
|
|
||||||
IS_PRESENT_EXPECTED_CIRC=1
|
|
||||||
break
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
if [[ ${IS_PRESENT_EXPECTED_CIRC} == 0 ]]; then
|
|
||||||
echo "Good job! The circular dependency \"${EXPECTED_CIRC}\" is no longer present."
|
|
||||||
echo "Please remove it from EXPECTED_CIRCULAR_DEPENDENCIES in $0"
|
|
||||||
echo "to make sure this circular dependency is not accidentally reintroduced."
|
|
||||||
echo
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
exit ${EXIT_CODE}
|
|
||||||
107
test/lint/lint-include-guards.py
Executable file
107
test/lint/lint-include-guards.py
Executable file
@ -0,0 +1,107 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
#
|
||||||
|
# Copyright (c) 2018-2022 The Bitcoin Core developers
|
||||||
|
# Distributed under the MIT software license, see the accompanying
|
||||||
|
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||||
|
|
||||||
|
"""
|
||||||
|
Check include guards.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from subprocess import check_output
|
||||||
|
from typing import List
|
||||||
|
|
||||||
|
|
||||||
|
HEADER_ID_PREFIX = 'BITCOIN_'
|
||||||
|
HEADER_ID_SUFFIX = '_H'
|
||||||
|
|
||||||
|
EXCLUDE_FILES_WITH_PREFIX = ['src/crypto/ctaes',
|
||||||
|
'src/leveldb',
|
||||||
|
'src/crc32c',
|
||||||
|
'src/secp256k1',
|
||||||
|
'src/minisketch',
|
||||||
|
'src/univalue',
|
||||||
|
'src/tinyformat.h',
|
||||||
|
'src/bench/nanobench.h',
|
||||||
|
'src/test/fuzz/FuzzedDataProvider.h',
|
||||||
|
'src/bls',
|
||||||
|
'src/crypto/x11/sph',
|
||||||
|
'src/ctpl_stl.h',
|
||||||
|
'src/dashbls',
|
||||||
|
'src/gsl',
|
||||||
|
'src/immer',
|
||||||
|
'src/util/expected.h']
|
||||||
|
|
||||||
|
|
||||||
|
def _get_header_file_lst() -> List[str]:
|
||||||
|
""" Helper function to get a list of header filepaths to be
|
||||||
|
checked for include guards.
|
||||||
|
"""
|
||||||
|
git_cmd_lst = ['git', 'ls-files', '--', '*.h']
|
||||||
|
header_file_lst = check_output(
|
||||||
|
git_cmd_lst).decode('utf-8').splitlines()
|
||||||
|
|
||||||
|
header_file_lst = [hf for hf in header_file_lst
|
||||||
|
if not any(ef in hf for ef
|
||||||
|
in EXCLUDE_FILES_WITH_PREFIX)]
|
||||||
|
|
||||||
|
return header_file_lst
|
||||||
|
|
||||||
|
|
||||||
|
def _get_header_id(header_file: str) -> str:
|
||||||
|
""" Helper function to get the header id from a header file
|
||||||
|
string.
|
||||||
|
|
||||||
|
eg: 'src/wallet/walletdb.h' -> 'BITCOIN_WALLET_WALLETDB_H'
|
||||||
|
|
||||||
|
Args:
|
||||||
|
header_file: Filepath to header file.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
The header id.
|
||||||
|
"""
|
||||||
|
header_id_base = header_file.split('/')[1:]
|
||||||
|
header_id_base = '_'.join(header_id_base)
|
||||||
|
header_id_base = header_id_base.replace('.h', '').replace('-', '_')
|
||||||
|
header_id_base = header_id_base.upper()
|
||||||
|
|
||||||
|
header_id = f'{HEADER_ID_PREFIX}{header_id_base}{HEADER_ID_SUFFIX}'
|
||||||
|
|
||||||
|
return header_id
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
exit_code = 0
|
||||||
|
|
||||||
|
header_file_lst = _get_header_file_lst()
|
||||||
|
for header_file in header_file_lst:
|
||||||
|
header_id = _get_header_id(header_file)
|
||||||
|
|
||||||
|
regex_pattern = f'^#(ifndef|define|endif //) {header_id}'
|
||||||
|
|
||||||
|
with open(header_file, 'r', encoding='utf-8') as f:
|
||||||
|
header_file_contents = f.readlines()
|
||||||
|
|
||||||
|
count = 0
|
||||||
|
for header_file_contents_string in header_file_contents:
|
||||||
|
include_guard_lst = re.findall(
|
||||||
|
regex_pattern, header_file_contents_string)
|
||||||
|
|
||||||
|
count += len(include_guard_lst)
|
||||||
|
|
||||||
|
if count != 3:
|
||||||
|
print(f'{header_file} seems to be missing the expected '
|
||||||
|
'include guard:')
|
||||||
|
print(f' #ifndef {header_id}')
|
||||||
|
print(f' #define {header_id}')
|
||||||
|
print(' ...')
|
||||||
|
print(f' #endif // {header_id}\n')
|
||||||
|
exit_code = 1
|
||||||
|
|
||||||
|
sys.exit(exit_code)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
@ -1,30 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2018-2020 The Bitcoin Core developers
|
|
||||||
# Distributed under the MIT software license, see the accompanying
|
|
||||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
|
||||||
#
|
|
||||||
# Check include guards.
|
|
||||||
|
|
||||||
export LC_ALL=C
|
|
||||||
HEADER_ID_PREFIX="BITCOIN_"
|
|
||||||
HEADER_ID_SUFFIX="_H"
|
|
||||||
|
|
||||||
REGEXP_EXCLUDE_FILES_WITH_PREFIX="src/(crypto/ctaes/|dashbls/|immer/|leveldb/|crc32c/|secp256k1/|minisketch/|test/fuzz/FuzzedDataProvider.h|tinyformat.h|bench/nanobench.h|univalue/|ctpl_stl.h|bls/|crypto/x11/sph|gsl|util/expected.h)"
|
|
||||||
|
|
||||||
EXIT_CODE=0
|
|
||||||
for HEADER_FILE in $(git ls-files -- "*.h" | grep -vE "^${REGEXP_EXCLUDE_FILES_WITH_PREFIX}")
|
|
||||||
do
|
|
||||||
HEADER_ID_BASE=$(cut -f2- -d/ <<< "${HEADER_FILE}" | sed "s/\.h$//g" | tr / _ | tr - _ | tr "[:lower:]" "[:upper:]")
|
|
||||||
HEADER_ID="${HEADER_ID_PREFIX}${HEADER_ID_BASE}${HEADER_ID_SUFFIX}"
|
|
||||||
if [[ $(grep -cE "^#(ifndef|define) ${HEADER_ID}" "${HEADER_FILE}") != 2 ]]; then
|
|
||||||
echo "${HEADER_FILE} seems to be missing the expected include guard:"
|
|
||||||
echo " #ifndef ${HEADER_ID}"
|
|
||||||
echo " #define ${HEADER_ID}"
|
|
||||||
echo " ..."
|
|
||||||
echo " #endif // ${HEADER_ID}"
|
|
||||||
echo
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
exit ${EXIT_CODE}
|
|
||||||
74
test/lint/lint-python-utf8-encoding.py
Executable file
74
test/lint/lint-python-utf8-encoding.py
Executable file
@ -0,0 +1,74 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
#
|
||||||
|
# Copyright (c) 2018-2022 The Bitcoin Core developers
|
||||||
|
# Distributed under the MIT software license, see the accompanying
|
||||||
|
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||||
|
#
|
||||||
|
# Make sure we explicitly open all text files using UTF-8 (or ASCII) encoding to
|
||||||
|
# avoid potential issues on the BSDs where the locale is not always set.
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import re
|
||||||
|
|
||||||
|
from subprocess import check_output, CalledProcessError
|
||||||
|
|
||||||
|
EXCLUDED_DIRS = ["src/crc32c/",
|
||||||
|
"src/secp256k1/"]
|
||||||
|
|
||||||
|
|
||||||
|
def get_exclude_args():
|
||||||
|
return [":(exclude)" + dir for dir in EXCLUDED_DIRS]
|
||||||
|
|
||||||
|
|
||||||
|
def check_fileopens():
|
||||||
|
fileopens = list()
|
||||||
|
|
||||||
|
try:
|
||||||
|
fileopens = check_output(["git", "grep", r" open(", "--", "*.py"] + get_exclude_args(), universal_newlines=True, encoding="utf8").splitlines()
|
||||||
|
except CalledProcessError as e:
|
||||||
|
if e.returncode > 1:
|
||||||
|
raise e
|
||||||
|
|
||||||
|
filtered_fileopens = [fileopen for fileopen in fileopens if not re.search(r"encoding=.(ascii|utf8|utf-8).|open\([^,]*, ['\"][^'\"]*b[^'\"]*['\"]", fileopen)]
|
||||||
|
|
||||||
|
return filtered_fileopens
|
||||||
|
|
||||||
|
|
||||||
|
def check_checked_outputs():
|
||||||
|
checked_outputs = list()
|
||||||
|
|
||||||
|
try:
|
||||||
|
checked_outputs = check_output(["git", "grep", "check_output(", "--", "*.py"] + get_exclude_args(), universal_newlines=True, encoding="utf8").splitlines()
|
||||||
|
except CalledProcessError as e:
|
||||||
|
if e.returncode > 1:
|
||||||
|
raise e
|
||||||
|
|
||||||
|
filtered_checked_outputs = [checked_output for checked_output in checked_outputs if re.search(r"universal_newlines=True", checked_output) and not re.search(r"encoding=.(ascii|utf8|utf-8).", checked_output)]
|
||||||
|
|
||||||
|
return filtered_checked_outputs
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
exit_code = 0
|
||||||
|
|
||||||
|
nonexplicit_utf8_fileopens = check_fileopens()
|
||||||
|
if nonexplicit_utf8_fileopens:
|
||||||
|
print("Python's open(...) seems to be used to open text files without explicitly specifying encoding='utf8':\n")
|
||||||
|
for fileopen in nonexplicit_utf8_fileopens:
|
||||||
|
print(fileopen)
|
||||||
|
exit_code = 1
|
||||||
|
|
||||||
|
nonexplicit_utf8_checked_outputs = check_checked_outputs()
|
||||||
|
if nonexplicit_utf8_checked_outputs:
|
||||||
|
if nonexplicit_utf8_fileopens:
|
||||||
|
print("\n")
|
||||||
|
print("Python's check_output(...) seems to be used to get program outputs without explicitly specifying encoding='utf8':\n")
|
||||||
|
for checked_output in nonexplicit_utf8_checked_outputs:
|
||||||
|
print(checked_output)
|
||||||
|
exit_code = 1
|
||||||
|
|
||||||
|
sys.exit(exit_code)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@ -1,28 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2018-2020 The Bitcoin Core developers
|
|
||||||
# Distributed under the MIT software license, see the accompanying
|
|
||||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
|
||||||
#
|
|
||||||
# Make sure we explicitly open all text files using UTF-8 (or ASCII) encoding to
|
|
||||||
# avoid potential issues on the BSDs where the locale is not always set.
|
|
||||||
|
|
||||||
export LC_ALL=C
|
|
||||||
EXIT_CODE=0
|
|
||||||
OUTPUT=$(git grep " open(" -- "*.py" ":(exclude)src/crc32c/" ":(exclude)src/secp256k1/" | grep -vE "encoding=.(ascii|utf8|utf-8)." | grep -vE "open\([^,]*, ['\"][^'\"]*b[^'\"]*['\"]")
|
|
||||||
if [[ ${OUTPUT} != "" ]]; then
|
|
||||||
echo "Python's open(...) seems to be used to open text files without explicitly"
|
|
||||||
echo "specifying encoding=\"utf8\":"
|
|
||||||
echo
|
|
||||||
echo "${OUTPUT}"
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
OUTPUT=$(git grep "check_output(" -- "*.py" ":(exclude)src/crc32c/" ":(exclude)src/secp256k1/" | grep "universal_newlines=True" | grep -vE "encoding=.(ascii|utf8|utf-8).")
|
|
||||||
if [[ ${OUTPUT} != "" ]]; then
|
|
||||||
echo "Python's check_output(...) seems to be used to get program outputs without explicitly"
|
|
||||||
echo "specifying encoding=\"utf8\":"
|
|
||||||
echo
|
|
||||||
echo "${OUTPUT}"
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
exit ${EXIT_CODE}
|
|
||||||
136
test/lint/lint-python.py
Executable file
136
test/lint/lint-python.py
Executable file
@ -0,0 +1,136 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
#
|
||||||
|
# Copyright (c) 2022 The Bitcoin Core developers
|
||||||
|
# Distributed under the MIT software license, see the accompanying
|
||||||
|
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||||
|
|
||||||
|
"""
|
||||||
|
Check for specified flake8 and mypy warnings in python files.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import pkg_resources
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
DEPS = ['flake8', 'mypy', 'pyzmq']
|
||||||
|
MYPY_CACHE_DIR = f"{os.getenv('BASE_ROOT_DIR', '')}/test/.mypy_cache"
|
||||||
|
FILES_ARGS = ['git', 'ls-files', '--','test/functional/*.py', 'contrib/devtools/*.py', ':(exclude)contrib/devtools/github-merge.py']
|
||||||
|
EXCLUDE_DIRS = ['src/dashbls/',
|
||||||
|
'src/immer/']
|
||||||
|
|
||||||
|
ENABLED = (
|
||||||
|
'E101,' # indentation contains mixed spaces and tabs
|
||||||
|
'E112,' # expected an indented block
|
||||||
|
'E113,' # unexpected indentation
|
||||||
|
'E115,' # expected an indented block (comment)
|
||||||
|
'E116,' # unexpected indentation (comment)
|
||||||
|
'E125,' # continuation line with same indent as next logical line
|
||||||
|
'E129,' # visually indented line with same indent as next logical line
|
||||||
|
'E131,' # continuation line unaligned for hanging indent
|
||||||
|
'E133,' # closing bracket is missing indentation
|
||||||
|
'E223,' # tab before operator
|
||||||
|
'E224,' # tab after operator
|
||||||
|
'E242,' # tab after ','
|
||||||
|
'E266,' # too many leading '#' for block comment
|
||||||
|
'E271,' # multiple spaces after keyword
|
||||||
|
'E272,' # multiple spaces before keyword
|
||||||
|
'E273,' # tab after keyword
|
||||||
|
'E274,' # tab before keyword
|
||||||
|
# TODO: enable it after bitcoin/bitcoin#26257 - too many warnings with newer flake
|
||||||
|
# 'E275,' # missing whitespace after keyword
|
||||||
|
'E304,' # blank lines found after function decorator
|
||||||
|
'E306,' # expected 1 blank line before a nested definition
|
||||||
|
'E401,' # multiple imports on one line
|
||||||
|
'E402,' # module level import not at top of file
|
||||||
|
'E502,' # the backslash is redundant between brackets
|
||||||
|
'E701,' # multiple statements on one line (colon)
|
||||||
|
'E702,' # multiple statements on one line (semicolon)
|
||||||
|
'E703,' # statement ends with a semicolon
|
||||||
|
'E711,' # comparison to None should be 'if cond is None:'
|
||||||
|
'E714,' # test for object identity should be "is not"
|
||||||
|
'E721,' # do not compare types, use "isinstance()"
|
||||||
|
'E742,' # do not define classes named "l", "O", or "I"
|
||||||
|
'E743,' # do not define functions named "l", "O", or "I"
|
||||||
|
'E901,' # SyntaxError: invalid syntax
|
||||||
|
'E902,' # TokenError: EOF in multi-line string
|
||||||
|
'F401,' # module imported but unused
|
||||||
|
'F402,' # import module from line N shadowed by loop variable
|
||||||
|
'F403,' # 'from foo_module import *' used; unable to detect undefined names
|
||||||
|
'F404,' # future import(s) name after other statements
|
||||||
|
'F405,' # foo_function may be undefined, or defined from star imports: bar_module
|
||||||
|
'F406,' # "from module import *" only allowed at module level
|
||||||
|
'F407,' # an undefined __future__ feature name was imported
|
||||||
|
'F601,' # dictionary key name repeated with different values
|
||||||
|
'F602,' # dictionary key variable name repeated with different values
|
||||||
|
'F621,' # too many expressions in an assignment with star-unpacking
|
||||||
|
'F622,' # two or more starred expressions in an assignment (a, *b, *c = d)
|
||||||
|
'F631,' # assertion test is a tuple, which are always True
|
||||||
|
'F632,' # use ==/!= to compare str, bytes, and int literals
|
||||||
|
'F701,' # a break statement outside of a while or for loop
|
||||||
|
'F702,' # a continue statement outside of a while or for loop
|
||||||
|
'F703,' # a continue statement in a finally block in a loop
|
||||||
|
'F704,' # a yield or yield from statement outside of a function
|
||||||
|
'F705,' # a return statement with arguments inside a generator
|
||||||
|
'F706,' # a return statement outside of a function/method
|
||||||
|
'F707,' # an except: block as not the last exception handler
|
||||||
|
'F811,' # redefinition of unused name from line N
|
||||||
|
'F812,' # list comprehension redefines 'foo' from line N
|
||||||
|
'F821,' # undefined name 'Foo'
|
||||||
|
'F822,' # undefined name name in __all__
|
||||||
|
'F823,' # local variable name … referenced before assignment
|
||||||
|
'F831,' # duplicate argument name in function definition
|
||||||
|
'F841,' # local variable 'foo' is assigned to but never used
|
||||||
|
'W191,' # indentation contains tabs
|
||||||
|
'W291,' # trailing whitespace
|
||||||
|
'W292,' # no newline at end of file
|
||||||
|
'W293,' # blank line contains whitespace
|
||||||
|
'W601,' # .has_key() is deprecated, use "in"
|
||||||
|
'W602,' # deprecated form of raising exception
|
||||||
|
'W603,' # "<>" is deprecated, use "!="
|
||||||
|
'W604,' # backticks are deprecated, use "repr()"
|
||||||
|
# 'W605,' # invalid escape sequence "x"
|
||||||
|
'W606,' # 'async' and 'await' are reserved keywords starting with Python 3.7
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def check_dependencies():
|
||||||
|
working_set = {pkg.key for pkg in pkg_resources.working_set}
|
||||||
|
|
||||||
|
for dep in DEPS:
|
||||||
|
if dep not in working_set:
|
||||||
|
print(f"Skipping Python linting since {dep} is not installed.")
|
||||||
|
exit(0)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
check_dependencies()
|
||||||
|
|
||||||
|
if len(sys.argv) > 1:
|
||||||
|
flake8_files = sys.argv[1:]
|
||||||
|
else:
|
||||||
|
files_args = ['git', 'ls-files', '--', '*.py']
|
||||||
|
for dir in EXCLUDE_DIRS:
|
||||||
|
files_args += [f':(exclude){dir}']
|
||||||
|
flake8_files = subprocess.check_output(files_args).decode("utf-8").splitlines()
|
||||||
|
|
||||||
|
flake8_args = ['flake8', '--ignore=B,C,E,F,I,N,W', f'--select={ENABLED}'] + flake8_files
|
||||||
|
flake8_env = os.environ.copy()
|
||||||
|
flake8_env["PYTHONWARNINGS"] = "ignore"
|
||||||
|
|
||||||
|
try:
|
||||||
|
subprocess.check_call(flake8_args, env=flake8_env)
|
||||||
|
except subprocess.CalledProcessError:
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
mypy_files = subprocess.check_output(FILES_ARGS).decode("utf-8").splitlines()
|
||||||
|
mypy_args = ['mypy', '--ignore-missing-imports', '--show-error-codes'] + mypy_files
|
||||||
|
|
||||||
|
try:
|
||||||
|
subprocess.check_call(mypy_args)
|
||||||
|
except subprocess.CalledProcessError:
|
||||||
|
exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@ -1,120 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2017-2020 The Bitcoin Core developers
|
|
||||||
# Distributed under the MIT software license, see the accompanying
|
|
||||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
|
||||||
#
|
|
||||||
# Check for specified flake8 warnings in python files.
|
|
||||||
|
|
||||||
export LC_ALL=C
|
|
||||||
export MYPY_CACHE_DIR="${BASE_ROOT_DIR}/test/.mypy_cache"
|
|
||||||
|
|
||||||
enabled=(
|
|
||||||
E101 # indentation contains mixed spaces and tabs
|
|
||||||
E112 # expected an indented block
|
|
||||||
E113 # unexpected indentation
|
|
||||||
E115 # expected an indented block (comment)
|
|
||||||
E116 # unexpected indentation (comment)
|
|
||||||
E125 # continuation line with same indent as next logical line
|
|
||||||
E129 # visually indented line with same indent as next logical line
|
|
||||||
E131 # continuation line unaligned for hanging indent
|
|
||||||
E133 # closing bracket is missing indentation
|
|
||||||
E223 # tab before operator
|
|
||||||
E224 # tab after operator
|
|
||||||
E242 # tab after ','
|
|
||||||
E266 # too many leading '#' for block comment
|
|
||||||
E271 # multiple spaces after keyword
|
|
||||||
E272 # multiple spaces before keyword
|
|
||||||
E273 # tab after keyword
|
|
||||||
E274 # tab before keyword
|
|
||||||
# TODO: enable it after bitcoin/bitcoin#26257 - too many warnings with newer flake
|
|
||||||
#E275 # missing whitespace after keyword
|
|
||||||
E304 # blank lines found after function decorator
|
|
||||||
E306 # expected 1 blank line before a nested definition
|
|
||||||
E401 # multiple imports on one line
|
|
||||||
E402 # module level import not at top of file
|
|
||||||
E502 # the backslash is redundant between brackets
|
|
||||||
E701 # multiple statements on one line (colon)
|
|
||||||
E702 # multiple statements on one line (semicolon)
|
|
||||||
E703 # statement ends with a semicolon
|
|
||||||
E711 # comparison to None should be 'if cond is None:'
|
|
||||||
E714 # test for object identity should be "is not"
|
|
||||||
E721 # do not compare types, use "isinstance()"
|
|
||||||
E742 # do not define classes named "l", "O", or "I"
|
|
||||||
E743 # do not define functions named "l", "O", or "I"
|
|
||||||
E901 # SyntaxError: invalid syntax
|
|
||||||
E902 # TokenError: EOF in multi-line string
|
|
||||||
F401 # module imported but unused
|
|
||||||
F402 # import module from line N shadowed by loop variable
|
|
||||||
F403 # 'from foo_module import *' used; unable to detect undefined names
|
|
||||||
F404 # future import(s) name after other statements
|
|
||||||
F405 # foo_function may be undefined, or defined from star imports: bar_module
|
|
||||||
F406 # "from module import *" only allowed at module level
|
|
||||||
F407 # an undefined __future__ feature name was imported
|
|
||||||
F601 # dictionary key name repeated with different values
|
|
||||||
F602 # dictionary key variable name repeated with different values
|
|
||||||
F621 # too many expressions in an assignment with star-unpacking
|
|
||||||
F622 # two or more starred expressions in an assignment (a, *b, *c = d)
|
|
||||||
F631 # assertion test is a tuple, which are always True
|
|
||||||
F632 # use ==/!= to compare str, bytes, and int literals
|
|
||||||
F701 # a break statement outside of a while or for loop
|
|
||||||
F702 # a continue statement outside of a while or for loop
|
|
||||||
F703 # a continue statement in a finally block in a loop
|
|
||||||
F704 # a yield or yield from statement outside of a function
|
|
||||||
F705 # a return statement with arguments inside a generator
|
|
||||||
F706 # a return statement outside of a function/method
|
|
||||||
F707 # an except: block as not the last exception handler
|
|
||||||
F811 # redefinition of unused name from line N
|
|
||||||
F812 # list comprehension redefines 'foo' from line N
|
|
||||||
F821 # undefined name 'Foo'
|
|
||||||
F822 # undefined name name in __all__
|
|
||||||
F823 # local variable name … referenced before assignment
|
|
||||||
F831 # duplicate argument name in function definition
|
|
||||||
F841 # local variable 'foo' is assigned to but never used
|
|
||||||
W191 # indentation contains tabs
|
|
||||||
W291 # trailing whitespace
|
|
||||||
W292 # no newline at end of file
|
|
||||||
W293 # blank line contains whitespace
|
|
||||||
W601 # .has_key() is deprecated, use "in"
|
|
||||||
W602 # deprecated form of raising exception
|
|
||||||
W603 # "<>" is deprecated, use "!="
|
|
||||||
W604 # backticks are deprecated, use "repr()"
|
|
||||||
# W605 # invalid escape sequence "x"
|
|
||||||
W606 # 'async' and 'await' are reserved keywords starting with Python 3.7
|
|
||||||
)
|
|
||||||
|
|
||||||
if ! command -v flake8 > /dev/null; then
|
|
||||||
echo "Skipping Python linting since flake8 is not installed."
|
|
||||||
exit 0
|
|
||||||
elif PYTHONWARNINGS="ignore" flake8 --version | grep -q "Python 2"; then
|
|
||||||
echo "Skipping Python linting since flake8 is running under Python 2. Install the Python 3 version of flake8."
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
FLAKECMD=flake8
|
|
||||||
|
|
||||||
if command -v flake8-cached > /dev/null; then
|
|
||||||
FLAKECMD=flake8-cached
|
|
||||||
else
|
|
||||||
echo "Consider install flake8-cached for cached flake8 results."
|
|
||||||
fi
|
|
||||||
|
|
||||||
EXIT_CODE=0
|
|
||||||
|
|
||||||
# shellcheck disable=SC2046
|
|
||||||
if ! PYTHONWARNINGS="ignore" $FLAKECMD --ignore=B,C,E,F,I,N,W --select=$(IFS=","; echo "${enabled[*]}") $(
|
|
||||||
if [[ $# == 0 ]]; then
|
|
||||||
git ls-files "*.py" | grep -vE "src/(immer)/"
|
|
||||||
else
|
|
||||||
echo "$@"
|
|
||||||
fi
|
|
||||||
); then
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
|
|
||||||
mapfile -t FILES < <(git ls-files "test/functional/*.py" "contrib/devtools/*.py" | grep -v contrib/devtools/github-merge.py)
|
|
||||||
if ! mypy --ignore-missing-imports --show-error-codes "${FILES[@]}"; then
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
|
|
||||||
exit $EXIT_CODE
|
|
||||||
67
test/lint/lint-shell-locale.py
Executable file
67
test/lint/lint-shell-locale.py
Executable file
@ -0,0 +1,67 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
#
|
||||||
|
# Copyright (c) 2018-2022 The Bitcoin Core developers
|
||||||
|
# Distributed under the MIT software license, see the accompanying
|
||||||
|
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||||
|
|
||||||
|
"""
|
||||||
|
Make sure all shell scripts are:
|
||||||
|
a.) explicitly opt out of locale dependence using
|
||||||
|
"export LC_ALL=C" or "export LC_ALL=C.UTF-8", or
|
||||||
|
b.) explicitly opt in to locale dependence using the annotation below.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import re
|
||||||
|
|
||||||
|
OPT_IN_LINE = '# This script is intentionally locale dependent by not setting \"export LC_ALL=C\"'
|
||||||
|
|
||||||
|
OPT_OUT_LINES = [
|
||||||
|
'export LC_ALL=C',
|
||||||
|
'export LC_ALL=C.UTF-8',
|
||||||
|
]
|
||||||
|
|
||||||
|
def get_shell_files_list():
|
||||||
|
command = [
|
||||||
|
'git',
|
||||||
|
'ls-files',
|
||||||
|
'--',
|
||||||
|
'*.sh',
|
||||||
|
]
|
||||||
|
try:
|
||||||
|
return subprocess.check_output(command, stderr = subprocess.STDOUT).decode('utf-8').splitlines()
|
||||||
|
except subprocess.CalledProcessError as e:
|
||||||
|
if e.returncode > 1: # return code is 1 when match is empty
|
||||||
|
print(e.output.decode('utf-8'), end='')
|
||||||
|
sys.exit(1)
|
||||||
|
return []
|
||||||
|
|
||||||
|
def main():
|
||||||
|
exit_code = 0
|
||||||
|
shell_files = get_shell_files_list()
|
||||||
|
for file_path in shell_files:
|
||||||
|
if re.search('src/(dashbls|secp256k1|minisketch|univalue)/', file_path):
|
||||||
|
continue
|
||||||
|
|
||||||
|
with open(file_path, 'r', encoding='utf-8') as file_obj:
|
||||||
|
contents = file_obj.read()
|
||||||
|
|
||||||
|
if OPT_IN_LINE in contents:
|
||||||
|
continue
|
||||||
|
|
||||||
|
non_comment_pattern = re.compile(r'^\s*((?!#).+)$', re.MULTILINE)
|
||||||
|
non_comment_lines = re.findall(non_comment_pattern, contents)
|
||||||
|
if not non_comment_lines:
|
||||||
|
continue
|
||||||
|
|
||||||
|
first_non_comment_line = non_comment_lines[0]
|
||||||
|
if first_non_comment_line not in OPT_OUT_LINES:
|
||||||
|
print(f'Missing "export LC_ALL=C" (to avoid locale dependence) as first non-comment non-empty line in {file_path}')
|
||||||
|
exit_code = 1
|
||||||
|
|
||||||
|
return sys.exit(exit_code)
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
|
|
||||||
@ -1,25 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2018 The Bitcoin Core developers
|
|
||||||
# Distributed under the MIT software license, see the accompanying
|
|
||||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
|
||||||
#
|
|
||||||
# Make sure all shell scripts:
|
|
||||||
# a.) explicitly opt out of locale dependence using
|
|
||||||
# "export LC_ALL=C" or "export LC_ALL=C.UTF-8", or
|
|
||||||
# b.) explicitly opt in to locale dependence using the annotation below.
|
|
||||||
|
|
||||||
export LC_ALL=C
|
|
||||||
|
|
||||||
EXIT_CODE=0
|
|
||||||
for SHELL_SCRIPT in $(git ls-files -- "*.sh" | grep -vE "src/(dashbls|secp256k1|minisketch|univalue)/"); do
|
|
||||||
if grep -q "# This script is intentionally locale dependent by not setting \"export LC_ALL=C\"" "${SHELL_SCRIPT}"; then
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
FIRST_NON_COMMENT_LINE=$(grep -vE '^(#.*)?$' "${SHELL_SCRIPT}" | head -1)
|
|
||||||
if [[ ${FIRST_NON_COMMENT_LINE} != "export LC_ALL=C" && ${FIRST_NON_COMMENT_LINE} != "export LC_ALL=C.UTF-8" ]]; then
|
|
||||||
echo "Missing \"export LC_ALL=C\" (to avoid locale dependence) as first non-comment non-empty line in ${SHELL_SCRIPT}"
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
exit ${EXIT_CODE}
|
|
||||||
95
test/lint/lint-shell.py
Executable file
95
test/lint/lint-shell.py
Executable file
@ -0,0 +1,95 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
#
|
||||||
|
# Copyright (c) 2018-2022 The Bitcoin Core developers
|
||||||
|
# Distributed under the MIT software license, see the accompanying
|
||||||
|
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
||||||
|
|
||||||
|
"""
|
||||||
|
Check for shellcheck warnings in shell scripts.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
|
||||||
|
# Disabled warnings:
|
||||||
|
DISABLED = [
|
||||||
|
'SC2046', # Quote this to prevent word splitting.
|
||||||
|
'SC2086', # Double quote to prevent globbing and word splitting.
|
||||||
|
'SC2162', # read without -r will mangle backslashes.
|
||||||
|
]
|
||||||
|
|
||||||
|
def check_shellcheck_install():
|
||||||
|
try:
|
||||||
|
subprocess.run(['shellcheck', '--version'], stdout=subprocess.DEVNULL, check=True)
|
||||||
|
except FileNotFoundError:
|
||||||
|
print('Skipping shell linting since shellcheck is not installed.')
|
||||||
|
sys.exit(0)
|
||||||
|
|
||||||
|
def get_files(command):
|
||||||
|
output = subprocess.run(command, stdout=subprocess.PIPE, universal_newlines=True)
|
||||||
|
files = output.stdout.split('\n')
|
||||||
|
|
||||||
|
# remove whitespace element
|
||||||
|
files = list(filter(None, files))
|
||||||
|
return files
|
||||||
|
|
||||||
|
def main():
|
||||||
|
check_shellcheck_install()
|
||||||
|
|
||||||
|
# build the `exclude` flag
|
||||||
|
exclude = '--exclude=' + ','.join(DISABLED)
|
||||||
|
|
||||||
|
# build the `sourced files` list
|
||||||
|
sourced_files_cmd = [
|
||||||
|
'git',
|
||||||
|
'grep',
|
||||||
|
'-El',
|
||||||
|
r'^# shellcheck shell=',
|
||||||
|
]
|
||||||
|
sourced_files = get_files(sourced_files_cmd)
|
||||||
|
|
||||||
|
# build the `guix files` list
|
||||||
|
guix_files_cmd = [
|
||||||
|
'git',
|
||||||
|
'grep',
|
||||||
|
'-El',
|
||||||
|
r'^#!\/usr\/bin\/env bash',
|
||||||
|
'--',
|
||||||
|
'contrib/guix',
|
||||||
|
'contrib/shell',
|
||||||
|
]
|
||||||
|
guix_files = get_files(guix_files_cmd)
|
||||||
|
|
||||||
|
# build the other script files list
|
||||||
|
files_cmd = [
|
||||||
|
'git',
|
||||||
|
'ls-files',
|
||||||
|
'--',
|
||||||
|
'*.sh',
|
||||||
|
]
|
||||||
|
files = get_files(files_cmd)
|
||||||
|
# remove everything that doesn't match this regex
|
||||||
|
reg = re.compile(r'src/[dashbls,immer,leveldb,secp256k1,minisketch,univalue]')
|
||||||
|
files[:] = [file for file in files if not reg.match(file)]
|
||||||
|
|
||||||
|
# build the `shellcheck` command
|
||||||
|
shellcheck_cmd = [
|
||||||
|
'shellcheck',
|
||||||
|
'--external-sources',
|
||||||
|
'--check-sourced',
|
||||||
|
'--source-path=SCRIPTDIR',
|
||||||
|
]
|
||||||
|
shellcheck_cmd.append(exclude)
|
||||||
|
shellcheck_cmd.extend(sourced_files)
|
||||||
|
shellcheck_cmd.extend(guix_files)
|
||||||
|
shellcheck_cmd.extend(files)
|
||||||
|
|
||||||
|
# run the `shellcheck` command
|
||||||
|
try:
|
||||||
|
subprocess.check_call(shellcheck_cmd)
|
||||||
|
except subprocess.CalledProcessError:
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
@ -1,39 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
#
|
|
||||||
# Copyright (c) 2018-2020 The Bitcoin Core developers
|
|
||||||
# Distributed under the MIT software license, see the accompanying
|
|
||||||
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
|
|
||||||
#
|
|
||||||
# Check for shellcheck warnings in shell scripts.
|
|
||||||
|
|
||||||
export LC_ALL=C
|
|
||||||
|
|
||||||
# Disabled warnings:
|
|
||||||
disabled=(
|
|
||||||
SC2046 # Quote this to prevent word splitting.
|
|
||||||
SC2086 # Double quote to prevent globbing and word splitting.
|
|
||||||
SC2162 # read without -r will mangle backslashes.
|
|
||||||
)
|
|
||||||
|
|
||||||
EXIT_CODE=0
|
|
||||||
|
|
||||||
if ! command -v shellcheck > /dev/null; then
|
|
||||||
echo "Skipping shell linting since shellcheck is not installed."
|
|
||||||
exit $EXIT_CODE
|
|
||||||
fi
|
|
||||||
|
|
||||||
if ! command -v gawk > /dev/null; then
|
|
||||||
echo "Skipping shell linting since gawk is not installed."
|
|
||||||
exit $EXIT_CODE
|
|
||||||
fi
|
|
||||||
|
|
||||||
SHELLCHECK_CMD=(shellcheck --external-sources --check-sourced)
|
|
||||||
EXCLUDE="--exclude=$(IFS=','; echo "${disabled[*]}")"
|
|
||||||
# Check shellcheck directive used for sourced files
|
|
||||||
mapfile -t SOURCED_FILES < <(git ls-files | xargs gawk '/^# shellcheck shell=/ {print FILENAME} {nextfile}')
|
|
||||||
mapfile -t FILES < <(git ls-files -- '*.sh' | grep -vE 'src/(dashbls|immer|leveldb|secp256k1|minisketch|univalue)/')
|
|
||||||
if ! "${SHELLCHECK_CMD[@]}" "$EXCLUDE" "${SOURCED_FILES[@]}" "${FILES[@]}"; then
|
|
||||||
EXIT_CODE=1
|
|
||||||
fi
|
|
||||||
|
|
||||||
exit $EXIT_CODE
|
|
||||||
Loading…
Reference in New Issue
Block a user