Change ctpl implementation to use STL queue & mutex.
Use ctpl synchronized queue instead of boost lockfree queue in bls worker aggregator.
Use smart pointers for memory management of Aggregator and VectorAggregator. With 'delete this;' the objects are prone to data race on the delete operator.
Use smart pointers for memory management of ContributionVerifier.
Pass shared_ptr by value to other threads via worker pool.
f3b90f2e05 Run all lint scripts (Julian Fleischer)
Pull request description:
The description reads:
```
# This script runs all contrib/devtools/lint-*.sh files, and fails if any exit
# with a non-zero status code.
```
This runs all scripts and returns with a non-zero exit code if any failed.
ACKs for commit f3b90f:
Tree-SHA512: 4f1f6435855dd5074a38c5887be6f096ec66f4dbe8712bdfd5fed0c510f1b2c083b7318cf3bfbdcc85982429fb7b4309e57ce48cc11736c86376658ec7ffea8f
414326952c use export LC_ALL=C.UTF-8 (Julian Fleischer)
728c82d029 make script exit if a command fails (Julian Fleischer)
506890b24d move remaining travis build steps into individual files (Julian Fleischer)
272306ea57 number .travis/ script according to build lifecycle and add README to explain (Julian Fleischer)
519e2739cf move lint stage up to resemble travis build ui (Julian Fleischer)
86d34f0e65 abort script in END_FOLD on non-zero exit code (Julian Fleischer)
4f2f88c7b0 move script sections info individual files and comply with shellcheck (Julian Fleischer)
Pull request description:
This PR is extracted from https://github.com/bitcoin/bitcoin/pull/13816 to make that one easier to review. It follows on https://github.com/bitcoin/bitcoin/pull/13849 and https://github.com/bitcoin/bitcoin/pull/13851
In here the shell script parts from `travis.yml` are extracted into `.travis/before_install.sh`, `.travis/install.sh`, `.travis/before_script.sh`, `.travis/script.sh`, and `.travis/lint.sh`.
This has the benefit that `test/lint/lint-shell.sh` will also shellcheck these parts. Also it makes the individual script parts more readable.
Tree-SHA512: c497e1687ceb1c1d795de177d3fc35af908bc8e3f781a871afabdecf031e581d4db229290627249e35ef7c09952bc34884e4734ea91d40f57b4a9efb85bba2e3
fa3e874d69 test: pruning: Check that verifychain can be called when pruned (MarcoFalke)
Pull request description:
Tree-SHA512: b921e550fdad27b791665883ed9fc50210575b0f968840a45284ad0e060ddd443ea7b9b12c17916f68ad28877c58ac51a6acc76ae4ea71bf7bfb2f744b286884
57281199b8 lint/format-strings: Correctly exclude escaped percent symbols (Luke Dashjr)
Pull request description:
The current code fails to exclude correctly for patterns like `"%%%X"`
Tree-SHA512: cac6f6fb3f06a9190310cd9764ec31cd7d636f9c831057622f418ae5fd2e1d80927a88f585d18f57b279ac21e81518f714dc1a25155377b9297741a73600461e
5c40e7b91a test: allows test_runner command line to receive parameters for each test (marcoagner)
Pull request description:
Fixes#14791.
Seems to address the asked behaviour in a simple way, but could address more if a more complex behaviour is found to be needed (e.g. call `rpc_bind --ipv4` without the ".py" and have it added).
Tree-SHA512: d7e21dd0ada36dd97dac71196bb97702a92986b181beb3753e37e3294b899fb65129aff5f9a45fe92b06fbf7c74e605ccb87a422f874f7a85d30401f4c2228c8
5d62dcf9cfb5c0b2511c10667ed47ec3b3610d72 lint: Make sure we read the command line inputs using utf-8 decoding in python (Chun Kuan Lee)
Pull request description:
Make sure we read the command line inputs using utf-8 decoding in python
occurred from travis cron job:
contrib/verify-commits/verify-commits.py should run with utf-8, otherwise it would raise UnicodeDecodeError
`UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 744: ordinal not in range(128)`
Tree-SHA512: 90e4ad57fdbbbecb0a21fc2d2b03a04f5ef125e54124719ef36e5a85326930b732b47534757a7c3a8730096f3947b009ec898191928b5c2d38f9f4b3e37db48d
# Conflicts:
# contrib/devtools/optimize-pngs.py
68400d8b96 tests: Use explicit imports (practicalswift)
Pull request description:
Enable automatic detection of undefined names in Python tests scripts. Remove wildcard imports.
Wildcard imports make it unclear which names are present in the namespace, confusing both readers and many automated tools.
An additional benefit of not using wildcard imports in tests scripts is that readers of a test script then can infer the rough testing scope just by looking at the imports.
Before this commit:
```
$ contrib/devtools/lint-python.sh | head -10
./test/functional/feature_rbf.py:8:1: F403 'from test_framework.util import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:9:1: F403 'from test_framework.script import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:10:1: F403 'from test_framework.mininode import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:15:12: F405 bytes_to_hex_str may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:17:58: F405 CScript may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:25:13: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:26:31: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:26:60: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:30:41: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:30:68: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
$
```
After this commit:
```
$ contrib/devtools/lint-python.sh | head -10
$
```
Tree-SHA512: 3f826d39cffb6438388e5efcb20a9622ff8238247e882d68f7b38609877421b2a8e10e9229575f8eb6a8fa42dec4256986692e92922c86171f750a0e887438d9
4b7091a842 Replace median fee rate with feerate percentiles (Marcin Jachymiak)
Pull request description:
Currently, the `medianfeerate` statistic is calculated from the feerate of the middle transaction of a list of transactions sorted by feerate.
This PR instead uses the value of the 50th percentile weight unit in the block, and also calculates the feerate at the 10th, 25th, 75th, and 90th percentiles. This more accurately corresponds with what is generally meant by median feerate.
Tree-SHA512: 59255e243df90d7afbe69839408c58c9723884b8ab82c66dc24a769e89c6d539db1905374a3f025ff28272fb25a0b90e92d8101103e39a6d9c0d60423a596714
e306be742932d4ea5aca0ea4768e54b2fc3dc6a0 Use 72 byte dummy signatures when watching only inputs may be used (Andrew Chow)
48b1473c898129a99212e2db36c61cf93625ea17 Use 71 byte signature for DUMMY_SIGNATURE_CREATOR (Andrew Chow)
18dfea0dd082af18dfb02981b7ee1cd44d514388 Always create 70 byte signatures with low R values (Andrew Chow)
Pull request description:
When creating signatures for transactions, always make one which has a 32 byte or smaller R and 32 byte or smaller S value. This results in signatures that are always less than 71 bytes (32 byte R + 32 byte S + 6 bytes DER + 1 byte sighash) with low R values. In most cases, the signature will be 71 bytes.
Because R is not mutable in the same way that S is, a low R value can only be found by trying different nonces. RFC 6979 for deterministic nonce generation has the option to specify additional entropy, so we simply use that and add a uin32_t counter which we increment in order to try different nonces. Nonces are sill deterministically generated as the nonce used will the be the first one where the counter results in a nonce that results in a low R value. Because different nonces need to be tried, time to produce a signature does increase. On average, it takes twice as long to make a signature as two signatures need to be created, on average, to find one with a low R.
Having a fixed size signature makes size calculations easier and also saves half a byte of transaction size, on average.
DUMMY_SIGNATURE_CREATOR has been modified to produce 71 byte dummy signatures instead of 72 byte signatures.
Tree-SHA512: 3cd791505126ce92da7c631856a97ba0b59e87d9c132feff6e0eef1dc47768e81fbb38bfbe970371bedf9714b7f61a13a5fe9f30f962c81734092a4d19a4ef33
5654efb187d24eb29a343c720e3937b01457c8b7 Ported usage of deprecated optparse module to argparse module (Kvaciral)
Pull request description:
The optparse module is deprecated since Python 2,7/3.2 . Recommend usage of the argparse module which improves upon optparse.
Tree-SHA512: ffd0e3e6f3babef1675226b107eeb7a6bab6e5199de572703da9d94e1f69c70d1c9abc353e9664b40670bb4976c06964bb2606deee52f5dfcc619f336ceb8cf8
fafe73a626 qa: Raise feature_help timeout to 5s (MarcoFalke)
faabd7bc47 qa: Use files for stdout/stderr to support Windows (MarcoFalke)
facb56ffaf qa: Run gen_rpcauth with sys.executable (MarcoFalke)
fada8966c5 qa: Close stdout and stderr file when node stops (MarcoFalke)
Pull request description:
### qa: Close stdout and stderr file when node stops
Since these files are potentially deleted by the test framework for cleanup, they should be closed first. Otherwise this will lead to errors on Windows when the tests finish successfully.
Side note: After the patch, it is no longer possible to reopen the file on Windows (see https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile)
### qa: Run gen_rpcauth with sys.executable
Similar to `test_runner.py`, the `sys.executable` needs to be passed down into subprocesses to pass on native Windows. (Should have no effect on Linux)
### qa: Use files for stdout/stderr to support Windows
It seems that using PIPE is not supported on Windows. Also, it is easier to just use the files that capture the stdout and stderr within the test node class.
Tree-SHA512: ec675012b10705978606b7fcbdb287c39a8e6e3732aae2fa4041d963a3c6993c6eac6a9a3cbd5479514e7d8017fe74c12235d1ed6fed2e8af8f3c71981e91864
fa85c985ed qa: Add p2p_invalid_locator test (MarcoFalke)
Pull request description:
Should not be merged *before* #13907
Tree-SHA512: a67ca407854c421ed20a184d0b0dc90085aed3e3431d9652a107fa3022244767e67f67e50449b7e95721f56906836b134615875f28a21e8a012eb22cfe6a66a5
fa5587fe71 qa: wait_for_verack by default (MarcoFalke)
Pull request description:
This removes the need to do so manually every time a connection is added.
Tree-SHA512: a46c92cb4df41e30778b42b9fd3dcbd8d2d82aa7503d1213cb1c1165034f648d8caee01c292e2d87d05b0f71696996eef5be8a753f35ab49e5f66b0e3bf29f21
5a05aa2db2 Add metavar to match var name in help text + Change wording for better readability (Martin Erlandsson)
Pull request description:
The help text given by `test/functional/test_runner.py -h` refers to the value `n`, which is defined as `COMBINEDLOGSLEN` in the list of commands.
To make the help text consistent, this PR changes the display name `COMBINEDLOGSLEN` to `n` by setting the argparse [`metavar`](https://docs.python.org/3/library/argparse.html#metavar) attribute. (`metavar` only changes the _displayed_ name)
Alternatively: Do the opposite and change the help text to use `COMBINEDLOGSLEN`.
---
Before PR:
```
➜ bitcoin > test/functional/test_runner.py -h | grep -A 1 combinedlogslen
--combinedlogslen COMBINEDLOGSLEN, -c COMBINEDLOGSLEN
print a combined log (of length n lines) from all test nodes and test framework to the console on failure.
```
After PR:
```
➜ bitcoin > test/functional/test_runner.py -h | grep -A 1 combinedlogslen
--combinedlogslen n, -c n
print a combined log (of length n lines) from all test nodes and test frameworks to the console on failure.
```
---
Also, fixed pluralization typo.
Tree-SHA512: a1124a4976d29fae1e8ecd7fa2ac523b7f05d541c611166532f44692995691a96faf797fa71582d78634f328b500cbee49c6ef296c8f1a898a57c050cc4e721d
086fc83571 Tests: Fix a comment (fridokus)
Pull request description:
Fix a comment that was false
Tree-SHA512: 945aa38229545e026e18c3abf53a4fbe6ec36413ce690fff7a1dd89b6e102d2b574524092e0ddf06cace82f3c040c59221b9b942be1203525814d2fbd50aaa0b
8907df9e02ec47ef249a7422faa766f06aa01e94 qa: Ensure wallet unload during walletpassphrase timeout (João Barbosa)
321decffa1fbf213462d97e5372bd0c4eeb99635 rpc: Fix wallet unload during walletpassphrase timeout (João Barbosa)
Pull request description:
Replaces the raw wallet pointer in the `RPCRunLater` callback with a `std::weak_ptr` to check if the wallet is not expired.
To test:
```
bitcoind -regtest
bitcoin-cli -regtest encryptwallet foobar
bitcoin-cli -regtest walletpassphrase foobar 5 && bitcoin-cli -regtest unloadwallet ""
```
Fixes#14452.
Tree-SHA512: 311e839234f5fb7955ab5412a2cfc1903ee7132ea56a8ab992ede3614586834886bd65192b76531ae0aa3a526b38e70ca2e1cdbabe52995906ff97b49d93c268
fa782a308dbe7bc579c122f63c1c65666fc85e91 qa: Use named args in some tests (MarcoFalke)
b4d33096734d787b0e1d754064039cbb64ce8d61 scripted-diff: Use named arguments in feature_block (MarcoFalke)
749ba35e7c9fbc21dbea27fd1be102b91313d132 scripted-diff: Pass node into p2p_segwit acceptance tests (MarcoFalke)
Pull request description:
It is confusing to use a list of arguments such as `False, False, 16, ...` where it is unclear what each of them means.
Run some scripted diffs to put meaning to them.
Tree-SHA512: d768df2375ea3c77145ebb1bf4c2d690581a379031449ded7ae160022d975eb13890aa8c6a44a5eebda8791cb2910a599326e431af76ed9e60afe1d182ada65c
# Conflicts:
# test/functional/feature_block.py
# test/functional/mining_basic.py
# test/functional/p2p_segwit.py
241f8b5de4 Fix typo in feature_blocksdir.py log message (Alexander Leishman)
Pull request description:
Typo I came across while writing some new tests.
Tree-SHA512: cc494553125a1e84f9238a14761e3fb76623e98d951811dd3bfb13595a03a1888d73859487a2cbb76c7ae85897bc64016a220a92c2636b35ea6356a5b5340d66
# Conflicts:
# test/functional/feature_blocksdir.py
a1a998cf24c0cf1232e44ec8eaf2ad6875ab5153 wallet: Fix backupwallet for multiwallets (Daniel Kraft)
Pull request description:
`backupwallet` was broken for multiwallets in their own directories (i.e. something like `DATADIR/wallets/mywallet/wallet.dat`). In this case, the backup would use `DATADIR/wallets/wallet.dat` as source file and not take the specific wallet's directory into account.
This led to either an error during the backup (if the wrong source file was not present) or would silently back up the wrong wallet; especially the latter behaviour can be quite bad for users.
Tree-SHA512: 7efe2450ca047e40719fcc7cc211ed94699056020ac737cada7b59e8240298675960570c45079add424d0aab520437d5050d956acd695a9c2452dd4317b4d2c4
5e1777777790e855a9f3c8604208bc9bd6c8c99f qa: Create unicode tempdir in test_runner (MarcoFalke)
Pull request description:
Now that wallet filenames are properly quoted when used for rpc (#13823), we can add some unicode symbols to the test_runner path. Thus, the "extern" wallet that uses a full path has a unicode symbol in its name.
Should add unicode coverage to
* `listwallets`
* `wallet.getwalletinfo`
* `(un)loadwallet`
Tree-SHA512: 1633fde56f8748df0cfef9c31a878c105dfaac85d1041b292261f44c4d40e96942aacbf7d6e839e8bbf979dc131d81c24ceb521e927fc8a5a71ba093f36b891b
fa5b440971a0dfdd64c1b86748a573fcd7dc65d3 qa: Extract rpc_timewait as test param (MarcoFalke)
Pull request description:
Also increase it for wallet_dump and wallet_groups
Tree-SHA512: 7367bc584228bda3010c453713a1505c54a8ef3d116be47dab9934d30594089dfeb27ffa862f7517fd0ec8b5dc07f4904d67ef2a53dd284cbe2a58982e410e2b
a13647b8bd [qa] Add test for too-large wallet output groups (Suhas Daftuar)
57ec1c97b2 [wallet] correctly limit output group size (Suhas Daftuar)
Pull request description:
Also add a test to ensure that output groups are being limited, even if a wallet has many outputs corresponding to the same scriptPubKey (the test fails without the first commit).
Tree-SHA512: 2aaa82005b0910488f5cbf40690d4c5e2f46949e299ef70b4cb6e440713811443d411dcbc6d71b1701fd82423073125e21747787d70830cd021c841afb732d51
247d5740d2 Ignore unknown config file options for now (Pieter Wuille)
04ce0d88ca Report when unknown config file options are ignored (Pieter Wuille)
Pull request description:
As reported by @satwo on IRC a few days ago, the current mechanism of treating unknown config file options as errors is problematic for options like `-rpcclienttimeout` which aren't defined for `bitcoind`.
A full solution would be to either make all binaries be aware of each other's options, or to permit config file options that only apply to specific binaries (`bitcoind`, `bitcoin-qt`, `bitcoin-cli`). Both of these seem too invasive to introduce for 0.17.
As a compromise, this PR makes it ignores those options, but still warn about it in the log file.
Tree-SHA512: dfddc771b91df3031a9c98d9f3292f8f4fcd1b97ebb7317b2f457e12d9f205dc63f42721302e7258dbb53f273d7cc041a65a0a9120972769555784e1f1cc9aef
64b9f27e0e Skip is_closing() check when not available. (Daniel Kraft)
Pull request description:
#13715 introduced a new check for `_transport.is_closing()` in mininode's `P2PConnection`'s. This function is [only available from Python 3.4.4](https://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseTransport.is_closing), though, while Bitcoin Core is supposed to support all Python 3.4 versions.
In this change, we make the check conditional on `is_closing` being available. If it is not, then we revert to the behaviour before the check was introduced; this means that #13579 is not fixed for old systems, but at least the tests work as they used to do before.
This includes a small refactoring from a one-line lambda to an inline function, because this makes the code easier to read with more and more conditions being added.
Fixes#13745.
Tree-SHA512: 15be03b8b49c40a946c5b354c5974858d14dc46283ad48ee25d9e269377077ce741c6b379b3f6581ab981cb65be799809afbb99da278caaa2d8d870fa4fb748f
5b82aa7352 Fix bitcoin-cli --version (Ben Woosley)
Pull request description:
By declaring the relevant option, as it is in init.cpp
2dc5ab6378/src/init.cpp (L356)
Note contrib/devtools/gen-manpages.sh relies on this version information.
Tree-SHA512: 64c1b9635e86fea42b797471f66f784ed0c7ad4d6a454a49eb433018d8936265624fa4edf2f138efe3ff938af61763bd473b7b86d558f5288687b25f59fa87ea
f447a0a7079619f0d650084df192781cca9fd826 Remove program options from build system (Chun Kuan Lee)
11588c639e8912f1b28e981c1a2a0e4306dbd093 Replace boost program_options (Chun Kuan Lee)
Pull request description:
Concept from #12744, but without parsing negated options.
Tree-SHA512: 7f418744bb8934e313d77a5f162633746ef5d043de802b9c9cd9f7c1842e7e566eb5f171cd9e2cc13317281b2449c6fbd553fa4f09b837e6af2f5d2b2aabdca2
a3fa4d6a6acf19d640a1d5879a00aa1f059e2380 QA: Fix bug in -usecli logic that converts booleans to non-lowercase strings (Jonas Schnelli)
4704e5f074e57782d058404a594a7313cf170cf0 [QA] add createwallet disableprivatekey test (Jonas Schnelli)
c7b8f343e99d9d53ea353ddce9a977f1886caf30 [Qt] Disable creating receive addresses when private keys are disabled (Jonas Schnelli)
2f15c2bc20d583b4c1788da78c9c635c36e03ed0 Add disable privatekeys option to createwallet (Jonas Schnelli)
cebefba0855cee7fbcb9474b34e6779369e8e9ce Add option to disable private keys during internal wallet creation (Jonas Schnelli)
9995a602a639b64a749545b7c3bafbf67f97324f Add facility to store wallet flags (64 bits) (Jonas Schnelli)
Pull request description:
This mode ('createwallet {"disableprivatekeys": true}') is intended for a sane pure watch-only mode, ideal for a use-case where one likes to use Bitcoin-Core in conjunction with a hardware-wallet or another solutions for cold-storage.
Since we have support for custom change addresses in `fundrawtransaction`, pure watch-only wallets including coin-selection are possible and do make sense for some use cases.
This new mode disables all forms of private key generation and ensure that no mix between hot and cold keys are possible.
Tree-SHA512: 3ebe7e8d54c4d4e5f790c348d4c292d456f573960a5b04d69ca5ef43a9217c7e7671761c6968cdc56f9a8bc235f3badd358576651af9f10855a0eb731f3fc508
ea5340c9d2 tests: fixes mininode's P2PConnection sending messages on closing transport (marcoagner)
Pull request description:
Fixes#13579.
I think one possible solution is to check for [`_transport.is_closing()`](https://docs.python.org/3.4/library/asyncio-protocol.html#asyncio.BaseTransport.is_closing) in the lambda before sending a message (compatible with Python 3.4 too). Let me know if I missed any side effects this introduces.
Tree-SHA512: cab46f81dccfec7b4460fda478a617845564520694449a9e85bf8a5f1e75f35f52cafd7c64966712c3d6c29956344d5a9dbad8851424f061eb3748bc621b900b
fa87da2f172ae2e6dc15e9ed156a3564a8ecfbdd qa: Avoid start/stop of the network thread mid-test (MarcoFalke)
Pull request description:
This simplifies test writing by removing the need to handle the network thread in tests. E.g. start thread, join thread, restart thread mid-test, adding p2p connections at the "right" time, ...
Tree-SHA512: 533642f12fef5496f1933855edcdab1a7ed901d088d34911749cd0f9e044c8a6cb1f89985ac3a7f41a512943663e4e270a61978f6f072143ae050cd102d4eab8
98ea64cf232c34d4b1aebe738b3956191667cd76 Let wallet importmulti RPC accept labels for standard scriptPubKeys (Russell Yanofsky)
Pull request description:
Allow importmulti RPC to apply address labels when importing standard scriptPubKeys. This makes the importmulti RPC less finnicky about import formats and also simpler internally.
Tree-SHA512: 102426b21239f1fa5f38162dc3f4145572caef76e63906afd786b7aff1670d6cd93456f8d85f737588eedc49c11bef2e1e8019b8b2cbf6097c77b3501b0cab1f
1fabd59e7 Break circular dependency: init -> * -> init by extracting shutdown.h (Ben Woosley)
e62fdfeea Drop unused init.h includes (Ben Woosley)
Pull request description:
Most includers just wanted to react to pending shutdown.
This isolates access to `fRequestShutdown` and limits access to the shutdown api functions, including the new `CancelShutdown` for setting it to `false`.
Tree-SHA512: df42f75dfbba163576710e9a67cf1228531fd99d70a2f187bfba0bcc476d6749cf88180a97e66a81bb5b6c3c7f0917de7402d26039ba7b644cb7509b02f7e267
