Commit Graph

249 Commits

Author SHA1 Message Date
Wladimir J. van der Laan
a9fb8a57ab
Merge #13482: Remove boost::program_options dependency
f447a0a7079619f0d650084df192781cca9fd826 Remove program options from build system (Chun Kuan Lee)
11588c639e8912f1b28e981c1a2a0e4306dbd093 Replace boost program_options (Chun Kuan Lee)

Pull request description:

  Concept from #12744, but without parsing negated options.

Tree-SHA512: 7f418744bb8934e313d77a5f162633746ef5d043de802b9c9cd9f7c1842e7e566eb5f171cd9e2cc13317281b2449c6fbd553fa4f09b837e6af2f5d2b2aabdca2
2021-07-02 12:59:27 +03:00
Wladimir J. van der Laan
4cd32e43e4
Merge #13235: Break circular dependency: init -> * -> init by extracting shutdown.h
1fabd59e7 Break circular dependency: init -> * -> init by extracting shutdown.h (Ben Woosley)
e62fdfeea Drop unused init.h includes (Ben Woosley)

Pull request description:

  Most includers just wanted to react to pending shutdown.

  This isolates access to `fRequestShutdown` and limits access to the shutdown api functions, including the new `CancelShutdown` for setting it to `false`.

Tree-SHA512: df42f75dfbba163576710e9a67cf1228531fd99d70a2f187bfba0bcc476d6749cf88180a97e66a81bb5b6c3c7f0917de7402d26039ba7b644cb7509b02f7e267
2021-07-02 00:42:18 +03:00
Wladimir J. van der Laan
cd5a39c8af Merge #14088: tests: Don't assert(...) with side effects
ca1a093127c11bb2aea10bf96c38dbfb40f8d170 Add regression test: Don't assert(...) with side effects (practicalswift)
4c3c9c38699360f93d3c52a01a90ff15ee5e1a62 Don't assert(...) with side effects (practicalswift)

Pull request description:

  Don't `assert(...)` with side effects.

  From the developer notes:

  > **Assertions should not have side-effects**
  >
  > Rationale: Even though the source code is set to refuse to compile with assertions disabled, having side-effects in assertions is unexpected and makes the code harder to understand

  These assertions were introduced quite recently (in #14069 which was merged two days ago) and since this is a recurring thing (see #13534 – "Don't assert(foo()) where foo() has side effects" from May) I added a simple regression test for the most obvious common side effect.

Tree-SHA512: be65db9d8d5d0f5752152ba73fe3fbb0531880f156d3cd7dfdf1752709979b63214e46ae64b1adbe1e09fa121278f4087f4ae49bff16cf8f5aec16ea6bde3650
2021-07-01 16:33:23 -05:00
MarcoFalke
69c601c2c3 Merge #13939: lint: Make format string linter understand basic template parameter syntax
4441ad677a Make format string linter understand basic template parameter syntax (practicalswift)

Pull request description:

  Make format string linter understand basic template parameter syntax.

  Fixes issue described in https://github.com/bitcoin/bitcoin/pull/13705#issuecomment-412046126.

  Thanks to @ken2812221 for reporting!

Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822
2021-06-28 21:31:44 -05:00
Wladimir J. van der Laan
a54f338783
Merge #15170: refactor/lint: Add ignored shellcheck suggestions to an array
cbd9091ed5a76bb2e1e57cd0d8db035c15529341 refactor/lint: Add ignored suggestions to an array (Vidar Holen)

Pull request description:

  By adding excluded shellcheck suggestions to an array, you can avoid the current duplication
  between command and comments. This ensures that they never go out of sync, makes it easier to
  add new ones, and improves the readability of related diffs.

Tree-SHA512: 04afced1d27fda940cc5e61d7f9ed04507c8f7f7dfd0031c09898a599c6de93695923a80cb3d515a0f0bf728847592d8680c15ac2e376b48726c03ca744f13a5
2021-06-28 13:40:55 -05:00
Wladimir J. van der Laan
de60193469
Merge #15164: qa: Ignore shellcheck warning SC2236
f652f85d0c88629b4284f69b179362254307fb55 qa: Ignore shellcheck warning SC2236 (João Barbosa)

Pull request description:

  With shellcheck 0.6.0 the warning `SC2236 -  Use -n instead of ! -z` is raised. This change adds that warning to the ignored list.

Tree-SHA512: 7b0dfbce55e5da4efb927e251257993cdf67cd1c90f8490d99fa75bf6e233bbee79ea1c59d28774994c59862c5eac061313aa154833284950fc844bda60a54e9
2021-06-28 13:40:54 -05:00
Kittywhiskers Van Gogh
ef4dfa8524 merge bitcoin#14555: Move util files to directory
(script modified to account for Dash backports, doesn't account for rebasing)

------------- BEGIN SCRIPT ---------------
mkdir -p src/util
git mv src/util.h src/util/system.h
git mv src/util.cpp src/util/system.cpp
git mv src/utilmemory.h src/util/memory.h
git mv src/utilmoneystr.h src/util/moneystr.h
git mv src/utilmoneystr.cpp src/util/moneystr.cpp
git mv src/utilstrencodings.h src/util/strencodings.h
git mv src/utilstrencodings.cpp src/util/strencodings.cpp
git mv src/utiltime.h src/util/time.h
git mv src/utiltime.cpp src/util/time.cpp
git mv src/utilasmap.h src/util/asmap.h
git mv src/utilasmap.cpp src/util/asmap.cpp
git mv src/utilstring.h src/util/string.h
git mv src/utilstring.cpp src/util/string.cpp

gsed -i 's/<util\.h>/<util\/system\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmemory\.h>/<util\/memory\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmoneystr\.h>/<util\/moneystr\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstrencodings\.h>/<util\/strencodings\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utiltime\.h>/<util\/time\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilasmap\.h>/<util\/asmap\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstring\.h>/<util\/string\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')

gsed -i 's/BITCOIN_UTIL_H/BITCOIN_UTIL_SYSTEM_H/g' src/util/system.h
gsed -i 's/BITCOIN_UTILMEMORY_H/BITCOIN_UTIL_MEMORY_H/g' src/util/memory.h
gsed -i 's/BITCOIN_UTILMONEYSTR_H/BITCOIN_UTIL_MONEYSTR_H/g' src/util/moneystr.h
gsed -i 's/BITCOIN_UTILSTRENCODINGS_H/BITCOIN_UTIL_STRENCODINGS_H/g' src/util/strencodings.h
gsed -i 's/BITCOIN_UTILTIME_H/BITCOIN_UTIL_TIME_H/g' src/util/time.h
gsed -i 's/BITCOIN_UTILASMAP_H/BITCOIN_UTIL_ASMAP_H/g' src/util/asmap.h
gsed -i 's/BITCOIN_UTILSTRING_H/BITCOIN_UTIL_STRING_H/g' src/util/string.h

gsed -i 's/ util\.\(h\|cpp\)/ util\/system\.\1/g' src/Makefile.am
gsed -i 's/utilmemory\.\(h\|cpp\)/util\/memory\.\1/g' src/Makefile.am
gsed -i 's/utilmoneystr\.\(h\|cpp\)/util\/moneystr\.\1/g' src/Makefile.am
gsed -i 's/utilstrencodings\.\(h\|cpp\)/util\/strencodings\.\1/g' src/Makefile.am
gsed -i 's/utiltime\.\(h\|cpp\)/util\/time\.\1/g' src/Makefile.am
gsed -i 's/utilasmap\.\(h\|cpp\)/util\/asmap\.\1/g' src/Makefile.am
gsed -i 's/utilstring\.\(h\|cpp\)/util\/string\.\1/g' src/Makefile.am

gsed -i 's/-> util ->/-> util\/system ->/' test/lint/lint-circular-dependencies.sh
gsed -i 's/src\/util\.cpp/src\/util\/system\.cpp/g' test/lint/lint-format-strings.py test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilmoneystr\.cpp/src\/util\/moneystr\.cpp/g' test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilstrencodings\.\(h\|cpp\)/src\/util\/strencodings\.\1/g' test/lint/lint-locale-dependence.sh
------------- END   SCRIPT ---------------
2021-06-27 12:03:13 +05:30
UdjinM6
11796986df
Merge pull request #4200 from Munkybooty/backports-0.18-pr1
Backports 0.18 pr1
2021-06-26 16:10:09 +03:00
UdjinM6
c5cc285d0e
Merge pull request #4191 from kittywhiskers/checkqueue
partial #15842, merge #15849, #17342, #18710: Add local thread pool to CCheckQueue
2021-06-26 16:08:02 +03:00
Kittywhiskers Van Gogh
c9d0d92b2d merge #18710: Add local thread pool to CCheckQueue 2021-06-25 12:25:14 +05:30
Kittywhiskers Van Gogh
94e3a5de64 bitcoin#14209: Replace LogPrint macros with regular functions 2021-06-24 23:43:02 +05:30
Wladimir J. van der Laan
82659a4eb8 Merge #12257: [wallet] Use destination groups instead of coins in coin select
232f96f5c8a3920c09db92f4dbac2ad7d10ce8cf doc: Add release notes for -avoidpartialspends (Karl-Johan Alm)
e00b4699cc6d2ee5697d38dd6607eb2631c9b77a clean-up: Remove no longer used ivars from CInputCoin (Karl-Johan Alm)
43e04d13b1ffc02b1082176e87f420198b40c7b1 wallet: Remove deprecated OutputEligibleForSpending (Karl-Johan Alm)
0128121101fb3ee82f3abd3973a967a4226ffe0e test: Add basic testing for wallet groups (Karl-Johan Alm)
59d6f7b4e2f847ec1f2ff46c84e6157655984f85 wallet: Switch to using output groups instead of coins in coin selection (Karl-Johan Alm)
87ebce25d66952f5ce565bb5130dcf5e24049872 wallet: Add output grouping (Karl-Johan Alm)
bb629cb9dc567cc819724d9f4852652926e60cbf Add -avoidpartialspends and m_avoid_partial_spends (Karl-Johan Alm)
65b3eda458221644616d0fdd6ba0fe01bdbce893 wallet: Add input bytes to CInputCoin (Karl-Johan Alm)
a443d7a0ca333b0bae63e04b5d476f9ad9c7aeac moveonly: CoinElegibilityFilter into coinselection.h (Karl-Johan Alm)
173e18a289088c6087ba6fac708e322aa63b7a94 utils: Add insert() convenience templates (Karl-Johan Alm)

Pull request description:

  This PR adds an optional (off by default) `-avoidpartialspends` flag, which changes coin select to use output groups rather than outputs, where each output group corresponds to all outputs with the same destination.

  It is a privacy improvement, as each time you spend some output, any other output that is publicly associated with the destination (address) will also be spent at the same time, at the cost of fee increase for cases where coin select without group restriction would find a more optimal set of coins (see example below).

  For regular use without address reuse, this PR should have no effect on the user experience whatsoever; it only affects users who, for some reason, have multiple outputs with the same destination (i.e. address reuse).

  Nodes with this turned off will still try to avoid partial spending, if the fee of the resulting transaction is not greater than the fee of the original transaction.

  Example: a node has four outputs linked to two addresses `A` and `B`:

  * 1.0 btc to `A`
  * 0.5 btc to `A`
  * 1.0 btc to `B`
  * 0.5 btc to `B`

  The node sends 0.2 btc to `C`. Without `-avoidpartialspends`, the following coin selection will occur:
  * 0.5 btc to `A` or `B` is picked
  * 0.2 btc is output to `C`
  * 0.3 - fee is output to (unique change address)

  With `-avoidpartialspends`, the following will instead happen:
  * Both of (0.5, 1.0) btc to `A` or `B` is picked (one or the other pair)
  * 0.2 btc is output to `C`
  * 1.3 - fee is output to (unique change address)

  As noted, the pro here is that, assuming nobody sends to the address after you spend from it, you will only ever use one address once. The con is that the transaction becomes slightly larger in this case, because it is overpicking outputs to adhere to the no partial spending rule.

  This complements #10386, in particular it addresses @luke-jr and @gmaxwell's concerns in https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-300667926 and https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-302361381.

  Together with `-avoidreuse`, this fully addresses the concerns in #10065 I believe.

Tree-SHA512: 24687a4490ba59cf4198ed90052944ff4996653a4257833bb52ed24d058b3e924800c9b3790aeb6be6385b653b49e304453e5d7ff960e64c682fc23bfc447621

# Conflicts:
#	src/Makefile.am
#	src/bench/coin_selection.cpp
#	src/wallet/coincontrol.h
#	src/wallet/coinselection.cpp
#	src/wallet/coinselection.h
#	src/wallet/init.cpp
#	src/wallet/test/coinselector_tests.cpp
#	src/wallet/wallet.cpp
#	src/wallet/wallet.h
#	test/functional/test_runner.py
2021-06-24 13:21:20 -04:00
Wladimir J. van der Laan
edac135e2d Merge #13707: tests: Add usage note to check-rpc-mappings.py
03a2d680101a9fee47c6fd818df7585cfc93ac06 Tests: add usage note to check-rpc-mappings.py (Mason Simon)

Pull request description:

  This test would previously fail without a user-friendly warning message, if invoked with no arguments.

  Test plan:
  ```
  bitcoin @_@$ python3 test/lint/check-rpc-mappings.py
  Usage: test/lint/check-rpc-mappings.py ROOT-DIR
  bitcoin @_@$ echo $?
  1
  bitcoin @_@$ python3 test/lint/check-rpc-mappings.py .
  * Checking consistency between dispatch tables and vRPCConvertParams
  bitcoin @_@$ echo $?
  0
  ```

Tree-SHA512: 0b7a94125d18ba5dbf3c3281a4af60718d2e495bf7645d7c2a3e1a2a1d80cffcea1383c90247186728037f1f2b38de75d0f59ccf7f792d69edd33884f3698c07
2021-06-15 13:15:26 -04:00
Pieter Wuille
7ff6515c88
Merge #13033: Build txindex in parallel with validation
9b2704777c [doc] Include txindex changes in the release notes. (Jim Posen)
ed77dd6b30 [test] Simple unit test for TxIndex. (Jim Posen)
6d772a3d44 [rpc] Public interfaces to GetTransaction block until synced. (Jim Posen)
a03f804f2a [index] Move disk IO logic from GetTransaction to TxIndex::FindTx. (Jim Posen)
e0a3b80033 [validation] Replace tx index code in validation code with TxIndex. (Jim Posen)
8181db88f6 [init] Initialize and start TxIndex in init code. (Jim Posen)
f90c3a62f5 [index] TxIndex method to wait until caught up. (Jim Posen)
70d510d93c [index] Allow TxIndex sync thread to be interrupted. (Jim Posen)
94b4f8bbb9 [index] TxIndex initial sync thread. (Jim Posen)
34d68bf3a3 [index] Create new TxIndex class. (Jim Posen)
c88bcec93f [db] Migration for txindex data to new, separate database. (Jim Posen)
0cb8303241 [db] Create separate database for txindex. (Jim Posen)

Pull request description:

  I'm re-opening #11857 as a new pull request because the last one stopped loading for people

  -------------------------------

  This refactors the tx index code to be in it's own class and get built concurrently with validation code. The main benefit is decoupling and moving the txindex into a separate DB. The primary motivation is to lay the groundwork for other indexers that might be desired (such as the [compact filters](https://github.com/bitcoin/bips/pull/636)). The basic idea is that the TxIndex spins up its own thread, which first syncs the txindex to the current block index, then once in sync the BlockConnected ValidationInterface hook writes new blocks.

  ### DB changes

  At the suggestion of some other developers, the txindex has been split out into a separate database. A data migration runs at startup on any nodes with a legacy txindex. Currently the migration blocks node initialization until complete.

  ### Open questions

  - Should the migration of txindex data from the old DB to the new DB block in init or should it happen in a background thread? The downside to backgrounding it is that `getrawtransaction` would return an error message saying the txindex is syncing while the migration is running.

  ### Impact

  In a sample size n=1 test where I synced nodes from scratch, the average time [Index writing](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L1903) was 3.36ms in master and 1.72ms in this branch. The average time between `UpdateTip` log lines for sequential blocks between 400,000 and IBD end on mainnet was 0.297204s in master and 0.286134s in this branch. Most likely this is just variance in IBD times, but I can try with some more trials if people want.

Tree-SHA512: 451fd7d95df89dfafceaa723cdf0f7b137615b531cf5c5035cfb54e9ccc2026cec5ac85edbcf71b7f4e2f102e36e9202b8b3a667e1504a9e1a9976ab1f0079c4
2021-05-25 13:48:04 +03:00
Kittywhiskers Van Gogh
96eaef33ee Merge #13862: drop boost::interprocess::file_lock 2021-05-20 10:15:17 +05:30
UdjinM6
f488d938f9
Make shellcheck 0.7.2 happy
Fixes `SC2268: Avoid x-prefix in comparisons as it no longer serves a purpose.`
2021-05-05 14:12:04 +03:00
PastaPastaPasta
da480fcd70
refactoring: governance tidying and include refactoring (#4111)
* remove unneeded semi-colons

* remove an unneeded CGovernanceException

* adjust and optimize listVotes to vector conversion

* governance-vote.h forward declare and include in cpp

* governance: use more forward decl. less includes in headers

* remove a number of unneeded includes in dash rpc code

* resolve circular depends linter

* Reorder includes

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-04-23 00:32:03 +02:00
PastaPastaPasta
d70d987be2
refactor: remove some validationinterface includes, resolve circular depends (#4116) 2021-04-20 22:32:27 +03:00
PastaPastaPasta
5074e6df9e
Extensive include refactoring (#4095)
* Simple changes, mostly just includes

* Continued include changes combined with using pointers to avoid including more than necessary in headers

* remove dup include

* masternode-utils.cpp include net.h in all builds

* resolve linter -7 +11

* drop quorums.h from dkgsessionhandler.cpp

* Add `<utilstrencodings.h>`

* Initialize lastMNListForVotingKeys

* Refactor GetMinedCommitment to return sharedptr

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-04-15 23:41:16 -04:00
UdjinM6
6ba3b2f3b8
Merge pull request #4081 from xdustinface/pr-fix-issue-3521
backport: Some wallet related PRs
2021-04-07 00:38:41 +03:00
Wladimir J. van der Laan
d4ac6d62a9 Merge #14350: Add WalletLocation class
65f3672f3b82a6fa30e5171f85bc8d8a29e0797e wallet: Refactor to use WalletLocation (João Barbosa)
01a4c095c87500650663341533f000c6b613e9da wallet: Add WalletLocation utility class (João Barbosa)

Pull request description:

  Advantages of this change:
   - avoid resolving wallet absolute path and name repetitively and in multiple places;
   - avoid calling `GetWalletDir` in multiple places;
   - extract these details from the actual wallet implementation.

  The `WalletLocation` class can be a way to represent a wallet not yet loaded that exists in the wallet directory.

Tree-SHA512: 71ec09786e038499710e7acafe92d66ab9883fc894964e267443ae9c10a6872a10995c3987a169c436a4e793dae96b28fb97bd7f78483c4b72ac930fa23f8686
2021-04-05 03:31:50 +02:00
PastaPastaPasta
a72acc113d
Move net.h include from quorums_*.h into quorums.cpp (#4075)
* Remove net include from quorums-util.h

* circular depends -1

* include timedata.h in quorums_debug.cpp

* fix circular depends linter
2021-04-02 03:19:34 +02:00
PastaPastaPasta
7e71a54e54
trivial: various include adjustments (#4074)
* remove unused includes in validation.cpp

circular depends -2

* use more specific include in quorums.h

circular depends -1

* Remove unneeded include from quorums.cpp

circular depends -1
2021-03-31 16:02:22 -04:00
PastaPastaPasta
4a84c99bad
trivial: Remove unneeded include from quorums_instantsend.cpp (#4073)
* Remove unneeded include from quorums_instantsend.cpp

Signed-off-by: pasta <pasta@dashboost.org>

* circular depends -6
2021-03-31 16:01:59 -04:00
PastaPastaPasta
9d1c35438c
rely on stopRequested which is set when ShutdownRequested (#4069)
circular depends -1

Signed-off-by: pasta <pasta@dashboost.org>
2021-03-29 14:03:17 -04:00
PastaPastaPasta
f492291aca
refactor: llmq pretty trivial refactor circular dependancy (#4065)
* move DEFAULT_WATCH_QUORUMS to quorums.h

Signed-off-by: pasta <pasta@dashboost.org>

* resolve circular lint, -10, +4
2021-03-26 13:21:41 +01:00
UdjinM6
6bcc1bf0a3
backport bitcoin#13190: Have gArgs handle printing help 2021-03-19 18:00:24 +03:00
dustinface
ae506bae66
refactor: PrivateSend -> CoinJoin + Move the tab (#4038)
* qt|wallet|privatesend: Rename PrivateSend to CoinJoin in GUI strings

* qt: Move CoinJoin next to Transactions

* qt: Adjust status tip of privateSendCoinsMenuAction

Co-authored-by: thephez <thephez@users.noreply.github.com>

* rename: privateSend -> coinJoin

* rename: privatesend -> coinjoin

* rename: PrivateSend -> CoinJoin

* rename: use_ps -> use_cj

* rename: PRIVATESEND -> COINJOIN

* rename: privatesend -> coinjoin for files and folders

* refactor: Re-order coinjoin files in cmake/make files

* refactor: Re-order coinjoin includes where it makes sense

* test: Update lint-circular-dependencies.sh

* Few cleanups

* test: test/coinjoin_tests.cpp -> wallet/test/coinjoin_test.cpp

* s/AdvancedPSUI/AdvancedCJUI/g

* s/privateSentAmountChanged/coinJoinAmountChanged/g

* wallet: Rename "ps_salt" backwards compatible

* Minimal PrivateSend -> CoinJoin migration for settings and cmd-line

* wallet: Fix privatesendrounds -> coinjoinrounds migration

* qt: Migrate nPrivateSendAmount -> nCoinJoinAmount

* `-coinjoindenoms` never existed

* Migrate all PS options/settings

* rpc: Formatting only

* qt: Make Send/CoinJoin tabs a bit more distinguishable

Co-authored-by: thephez <thephez@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-03-17 18:36:11 -04:00
UdjinM6
8da57523b7
Make recently added help strings dynamic, add one missing string (#3996)
* docs: Add help for -recsigsmaxage option

* Make `-llmq-data-recovery` help string dynamic

* Make devnet/regtest specific help strings dynamic

Only show devnet related help strings on devnet.

* Make linter happy

* Skip FindDevNetGenesisBlock when creating devnet params for help purposes only
2021-02-23 01:46:40 +03:00
UdjinM6
e029f24c1c Fix some circular dependencies, update EXPECTED_CIRCULAR_DEPENDENCIES to make linter happy 2021-02-04 14:58:20 -06:00
Wladimir J. van der Laan
cd59867e7b Merge #13695: lint: Add linter for circular dependencies
5c613aadd64453c75cb2373c6fcc1326c3cf0b7a lint: Add linter for circular dependencies (Ben Woosley)

Pull request description:

  Protects against added circular depencies, makes it explicit in the
  code when circular dependencies have been removed.

  Modeled after EXPECTED_BOOST_INCLUDES in lint-includes.sh

  Example output:
  ```
  $ test/lint/lint-circular-dependencies.sh
  A new circular dependency in the form of "qt/paymentserver -> qt/walletmodel -> qt/paymentserver" appears to have been introduced.

  $ echo $?
  1
  $ test/lint/lint-circular-dependencies.sh
  Good job! The circular dependency "Fake" is no longer present.
  Please remove it from EXPECTED_CIRCULAR_DEPENDENCIES in test/lint/lint-circular-dependencies.sh
  to make sure this circular dependency is not accidentally reintroduced.

  $ echo $?
  1
  $ test/lint/lint-circular-dependencies.sh
  $ echo $?
  0

  ```

Tree-SHA512: 4519434de29f6d50859daed1480e531c01c1cdbc3f0a5f093251daf62ae2b5b9073fb274b86f541a985e06837aa1165b76558c5f35fb51a759d72e83f1b61e44
2021-02-04 14:58:13 -06:00
Wladimir J. van der Laan
be4345ce12
Merge #13510: Scripts and tools: Obsolete #!/bin/bash shebang
000000035b20402dea3e8168165cd4eefdc97539 Obsolete #!/bin/bash shebang (DesWurstes)

Pull request description:

  > `#!/bin/bash` assumes it is always installed to `/bin/` which can cause issues
  > `#!/usr/bin/env bash` searches the user's `PATH` to find the `bash` binary

  Details: https://github.com/dylanaraps/pure-bash-bible#obsolete-syntax

  I'm open to comments: Should I also fix `#!/bin/sh`?

Tree-SHA512: b47bb4828116aa119f1899c68fee081270d51a898535490b9c616bf0f3660ad953f29c361eafc759bc64cdd54ee6eeecb2d79e9fdb5291a996a515c719805476
2020-12-18 12:55:45 -06:00
UdjinM6
b559a8f904
Backporting Statoshi and bitcoin#16728 (#2515)
* Backport Statoshi

This backports some of https://github.com/jlopp/statoshi.

Missing stuff: README.md and client name changes, segwit and fee estimation stats.

Fix RejectCodeToString

Fix copy-paste mistake s/InvalidBlockFound/InvalidChainFound/

* Merge #16728: move-only: move coins statistics utils out of RPC

8a3b2eb17572ca2131778d52cc25ec359470a90f move-only: move coins statistics utils out of RPC (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  In the short-term, this move-only commit will help with fuzzing (https://github.com/bitcoin/bitcoin/pull/15606#issuecomment-524482297). Later, these procedures will be used to compute statistics (particularly a content hash) for UTXO sets coming in from snapshots.

  Most easily reviewed with `git ... --color-moved=dimmed_zebra`. A nice follow-up would be adding unittests, which I'll do if nobody else gets around to it.

ACKs for top commit:
  MarcoFalke:
    ACK 8a3b2eb17572ca2131778d52cc25ec359470a90f, checked --color-moved=dimmed-zebra

Tree-SHA512: a187d2f7590ad2450b8e8fa3d038c80a04fc3d903618c24222d7e3172250ce51badea35860c86101f2ba266eb4354e6efb8d7d508b353f29276e4665a1efdf74

* Fix 16728

* Modernize StatsdClient

- Reuse some functionality from netbase
- Switch from GetRand to FastRandomContext
- Drop `using namespace std` and add `// namespace statsd`

* Introduce PeriodicStats and make StatsdClient configurable via -stats<smth> (enabled/host/port/ns/period)

* Move/rename tip stats from CheckBlock to ConnectBlock

* Add new false positives to lint-format-strings.py

* Add snprintf in statsd_client to the list of known violations in lint-locale-dependence.sh

* Fix incorrect include guard

* Use bracket syntax includes

* Replace magic numbers with defaults

* Move connection stats calculation into its own function

And bail out early if stats are disabled

* assert in PeriodicStats

Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
2020-12-15 10:22:23 -06:00
MarcoFalke
7de10e4737
Merge #12284: Remove assigned but never used local variables. Enable Travis checking for unused local variables.
ea04bf7862 Enable flake8 warning F841 ("local variable 'foo' is assigned to but never used") (practicalswift)
169f3e8637 Remove assigned but never used local variables (practicalswift)

Pull request description:

  Remove assigned but never used local variables. Enable Travis checking for unused local variables.

Tree-SHA512: d6052ec9044c5d1f03d874ea3c8addd5a156779213ef9200f89d3ae53230f2fd1691aff405c3dae14178e5ef09912c4432e92f606ef4a5220ed9daa140cdee81
2020-10-26 20:35:28 -04:00
xdustinface
6088650438 test: Add boost/signals2/connection.hpp to lint-inlcudes.sh 2020-10-14 12:10:11 +01:00
UdjinM6
f7218d1899 Make shellcheck happy 2020-07-29 11:20:12 -05:00
UdjinM6
bdb479db4b Update lint-locale-dependence.sh 2020-07-29 11:06:10 -05:00
UdjinM6
2910bf0d49 Update lint-format-strings.py 2020-07-29 11:05:06 -05:00
practicalswift
2e7f07ebeb Fix warnings introduced in shellcheck v0.6.0 (Partial Merge: #15166) 2020-07-29 11:02:55 -05:00
MarcoFalke
27aef40091 Merge #15534: [test] lint-format-strings: open files sequentially (fix for OS X)
21be609b49 In lint-format-strings, open files sequentially (Glenn Willen)

Pull request description:

  In lint-format-strings, we use python argparse to read our file arguments. In
  this mode, argparse opens all the files simultaneously. On OS X, where the
  default filehandle limit is 128, this causes the lint to fail. Instead, ask
  argparse for our filename arguments as strings, and open them one at a time
  using 'with open'.

Tree-SHA512: 4c7dabf98818a7c5d83ab10c61b89a26957fe399e39e933e30c561cb45c5e8ba6f6aedcde8343da0c32ee340289a8897db6a33708e35ee381334ee27e3f4d356
2020-07-29 11:01:38 -05:00
MarcoFalke
481254c86d Merge #14115: lint: Make all linters work under the default macOS dev environment (build-osx.md)
341f7c7b0e macOS fix: Check for correct version of flake8 to avoid spurious warnings. The brew installed flake8 version is Python 2 based and does not work. (practicalswift)
908a559f33 macOS fix: Add excludes for checks added in the newer shellcheck version installed by brew (practicalswift)
ec4d57bbb3 macOS fix: Work around empty (sub)expression error when using BSD grep (practicalswift)
b57d7d92fe macOS fix: Avoid mapfile due to ancient version of bash shipped with macOS (practicalswift)

Pull request description:

  The linters are thoroughly tested under Ubuntu which is what we use in Travis. When reading #14041 I understood that some developers were experiencing problems when running the linters on their local machines.

  Assuming these local machines were running macOS I installed a fresh macOS VM, followed the instructions in `build-osx.md` and ran the linters.

  This PR contains the changes needed to make `lint-all.sh` run as expected.

  Ideally the linters would continuously run also under a Travis macOS environment to make sure we catch these kind of issues before merge.

Tree-SHA512: b39c9a970d14d27db1fb592539923c0bc676b5217f415d02fda3f17bf54d46faa172376e8a3ecab07ca68a3acba9aebe00b2b1b2161b2a36b85fbb672e7efb5c
2020-07-29 10:59:39 -05:00
Wladimir J. van der Laan
7be2b2456a Merge #13281: test: Move linters to test/lint, add readme
fa3c910bfeab00703c947c5200a64c21225b50ef test: Move linters to test/lint, add readme (MarcoFalke)

Pull request description:

  This moves the checks and linters from `devtools` to a subfolder in `test`. (Motivated by my opinion that the dev tools are mostly for generating code and updating the repo whereas the linters are read-only checks.)

  Also, adds a readme to clarify that checks and linters are only meant to prevent bugs and user facing issues, not merely stylistic preference or inconsistencies. (This is motivated by the diversity in developers and work flows as well as existing code styles. It would be too disruptive to change all existing code to a single style or too burdensome to force all developers to adhere to a single style. Also note that our style guide is changing, so locking in at the wrong style "too early" would only waste resources.)

Tree-SHA512: 9b10e89f2aeaf0c8a9ae248aa891d74e0abf0569f8e5dfd266446efa8bfaf19f0ea0980abf0b0b22f0d8416ee90d7435d21a9f9285b66df43f370b7979173406
2020-07-29 10:59:04 -05:00
Wladimir J. van der Laan
036125c9a5 Merge #13705: build: Add format string linter
bcd4b0f5cdde2a1b562a612c78ec1ef1fe47d3dd Add linting of WalletLogPrintf(...) format strings (practicalswift)
a3e455694901a887e0feef69bd63e3aa122ea44b build: Add format string linter (practicalswift)

Pull request description:

  Add format string linter.

  This linter checks that the number of arguments passed to each variadic format string function matches the number of format specifiers in the format string.

  Example output:

  ```
  $ test/lint/lint-format-strings.sh
  src/init.cpp: Expected 2 argument(s) after format string but found 1 argument(s):
    LogPrintf("We have a mismatch here: foo=%s bar=%d\n", foo)
  src/init.cpp: Expected 1 argument(s) after format string but found 2 argument(s):
    LogPrint(BCLog::RPC, "RPC stopped. This is a mismatch: %s\n", s1, s2)
  $ echo $?
  1
  ```

Tree-SHA512: 19ab844a63f04bf193d66682ca42745a1c7d6c454b30222491b9fe8dc047054c4a6d3ee7921ec0676fb9ca2e7f6f93bd6c97996fb09667269bd491cb875349f3
2020-07-28 21:35:31 -05:00
MarcoFalke
9a6c8277b6 Merge #13494: Follow-up to #13454: Fix broken build by exporting LC_ALL=C
7b23e6e13f Follow-up to #13454: Fix broken build by exporting LC_ALL=C (practicalswift)

Pull request description:

  Follow-up to #13454: Fix broken build by exporting `LC_ALL=C`.

Tree-SHA512: 5cca3182ba034dce28a0df5f4a4b343de6c2526048f17fee30e2f8d946e976b39d9cc54faae6c31bfe89022f9f4c360e9ec8e163a1690bc0656410a48bb81dbf
2020-07-28 21:35:31 -05:00
Wladimir J. van der Laan
cb5d0d8b99 Merge #13454: Make sure LC_ALL=C is set in all shell scripts
47776a958b08382d76d69b5df7beed807af168b3 Add linter: Make sure all shell scripts opt out of locale dependence using "export LC_ALL=C" (practicalswift)
3352da8da1243c03fc83ba678d2f5d193bd5a0c2 Add "export LC_ALL=C" to all shell scripts (practicalswift)

Pull request description:

  ~~Make sure `LC_ALL=C` is set when using `grep` range expressions.~~

  Make sure `LC_ALL=C` is set in all shell scripts.

  From the `grep(1)` documentation:

  > Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and character set. For example, in the default C locale, `[a-d]` is equivalent to `[abcd]`. Many  locales sort characters in dictionary order, and in these locales `[a-d]` is typically not equivalent to `[abcd]`; it might be equivalent to `[aBbCcDd]`, for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the `LC_ALL` environment variable to the value C.

  Context: [Locale issue found when reviewing #13450](https://github.com/bitcoin/bitcoin/pull/13450/files#r194877736)

Tree-SHA512: fd74d2612998f9b49ef9be24410e505d8c842716f84d085157fc7f9799d40e8a7b4969de783afcf99b7fae4f91bbb4559651f7dd6578a6a081a50bdea29f0909
2020-07-28 21:35:31 -05:00
Wladimir J. van der Laan
361d254524 Merge #13496: Test: Harden lint-filenames.sh
927e1150bc207181f7d5b0948e87d04b737a1c27 Test: Harden lint-filenames.sh (wodry)

Pull request description:

  - This fixes that only files with lower case file name suffix where found before, which contradicted the Regex to find uppercase characters in file names (including suffixes I guess).
  - `--full-name` switch was added to git ls-files, to define that the found file always includes it's full path in the git project.
  - since we know now that the file name includes the full path, we can harden the Regex to exclude the  secp256k1 and univalue sub folders.
  - use backslash line break to make code easier to read and avoid too long line.

Tree-SHA512: 9b55fe4965ae2084112b9f8a81bf9c657756c2cb5004986e7b6102a76adaf62c7d7a53257d9f13c5d8a1c75870b52c744d13830e3edd454a099e810357c914e5
2020-07-08 18:32:04 -05:00
Wladimir J. van der Laan
d1200755f1 Merge #13448: Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python
c8176b3cc7556d7bcec39a55ae4d6ba16453baaa Add linter: Make sure we explicitly open all text files using UTF-8 or ASCII encoding in Python (practicalswift)
634bd970013eca90f4b4c1f9044eec8c97ba62c2 Explicitly specify encoding when opening text files in Python code (practicalswift)

Pull request description:

  Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python.

  As requested by @laanwj in #13440.

Tree-SHA512: 1651c00fe220ceb273324abd6703aee504029b96c7ef0e3029145901762c733c9b9d24927da281394fd4681a5bff774336c04eed01fafea997bb32192c334c06
Signed-off-by: pasta <pasta@dashboost.org>

# Conflicts:
#	contrib/devtools/circular-dependencies.py
#	contrib/linearize/linearize-data.py
#	contrib/linearize/linearize-hashes.py
#	contrib/seeds/generate-seeds.py
#	contrib/verify-commits/verify-commits.py
#	test/functional/multiwallet.py
#	test/functional/notifications.py
#	test/functional/test_runner.py
#	test/util/rpcauth-test.py
2020-07-07 11:50:56 -05:00
MarcoFalke
fa3eed7af9 Merge #13450: Add linter: Enforce the source code file naming convention described in the developer notes
ad691f666b Add linter: Enforce the source code file naming convention described in the developer notes (practicalswift)

Pull request description:

  Add linter: Enforce the source code file naming convention described in the developer notes.

Tree-SHA512: 6458acf5383de7e81554bdd954c3a74c2bbf26286687ea69d934f11174d2f6bd573e8d2c16a7e77bbd12065e65be7700ecd7791d215f286e18f346bf964cd17d
2020-07-07 11:39:49 -05:00
Wladimir J. van der Laan
28b02f42e1 Merge #13041: build: Add linter checking for accidental introduction of locale dependence
698cfd081144845f6246171b8a2a0cfa8daaecdb docs: Mention lint-locale-dependence.sh in developer-notes.md (practicalswift)
0a4ea2f4589961868ab4d25e0277485c31938e20 build: Add linter for checking accidental locale dependence (practicalswift)

Pull request description:

  This linter will check for code accidentally introducing locale dependencies.

  Unnecessary locale dependence can cause bugs that are very tricky to isolate and fix. We should avoid using locale dependent functions if possible.

  Context: https://github.com/bitcoin/bitcoin/pull/12881#issuecomment-378564722

  Example output:

  ```
  $ contrib/devtools/lint-locale-dependence.sh
  The locale dependent function tolower(...) appears to be used:
  src/init.cpp:    if (s[0] == '0' && std::tolower(s[1]) == 'x') {

  Unnecessary locale dependence can cause bugs that are very
  tricky to isolate and fix. Please avoid using locale dependent
  functions if possible.

  Advice not applicable in this specific case? Add an exception
  by updating the ignore list in contrib/devtools/lint-locale-dependence.sh
  ```

  **Note to reviewers:** What is the most appropriate `LOCALE_DEPENDENT_FUNCTIONS` function list? What should be added or removed?

Tree-SHA512: 14e448828804bb02bf59070647e38b52fce120c700c903a4a8472769a2cee5dd529bd3fc182386993cb8720482cf4250b63a0a477db61b941ae4babe5c65025f
2020-07-01 20:12:03 -05:00