3e730bf90aaf53c41ff3a778f6aac15d163d1c0c zmq: Fix due to invalid argument and multiple notifiers (João Barbosa)
Pull request description:
ZMQ initialization is interrupted if any notifier fails, and in that case all notifiers are destroyed. The notifier shutdown assumes that the initialization had occurred. This is not valid when there are multiple notifiers and any except the last fails to initialize.
Can be tested by running test/functional/interface_zmq.py from this branch with bitcoind from master.
Closes#17185.
ACKs for top commit:
laanwj:
Code review ACK 3e730bf90aaf53c41ff3a778f6aac15d163d1c0c, thanks for adding a test
Tree-SHA512: 5da710e97dcbaa94896d019e75162d470f6d381ee07c60e5b3e9db93d11e8f7ca9bf2c509efa4486199e88c96c3e720cc96b4e35b62725d4c7db8e8e9bf6e09d
72ae20fc142457a200278cb2fedc5e32a3766b58 tests: add sync_all to fix race condition in wallet groups test (Karl-Johan Alm)
Pull request description:
This most likely fixes#19749, the intermittent CI issues with wallet_groups.
This fix is also included in #19743.
Top commit has no ACKs.
Tree-SHA512: dd6ef7f89829483e2278191c21fe0912b51fd2187c10a0fa158339c5ab9f22d93b733ae10f17ef25d8b64f44e596e66dba8d7db5c009343472f422ce4cd67d8f
7f13dfb587dd6a7a5b7dfbfe689ae0ce818fe5c9 test: test the implicit avoid partial spends functionality (Karl-Johan Alm)
b82067bf696c53f22536f9ca2e51949c164f6b06 wallet: try -avoidpartialspends mode and use its result if fees are below threshold (Karl-Johan Alm)
Pull request description:
The `-avoidpartialspends` feature is normally disabled, as it may affect the optimal fee for payments. This PR introduces a new parameter `-maxapsfee` (max avoid partial spends fee) which acts on the following values:
* -1: disable partial spend avoidance completely (do not even try it)
* 0: only do partial spend avoidance if fees are the same or better as the regular coin selection
* 1..∞: use APS variant if the absolute fee difference is less than or equal to the max APS fee
For values other than -1, the code will now try partial spend avoidance once, and if that gives a value within the accepted range, it will use that.
Example: -maxapsfee=0.00001000 means the wallet will do regular coin select, APS coin select, and then pick AKS iff the absolute fee difference is <= 1000 satoshi.
Edit: updated this to reflect the fact we are now using a max fee.
ACKs for top commit:
fjahr:
tested ACK 7f13dfb587dd6a7a5b7dfbfe689ae0ce818fe5c9
achow101:
ACK 7f13dfb587dd6a7a5b7dfbfe689ae0ce818fe5c9
jonatack:
ACK 7f13dfb58, code review, debug build, verified the test fails with `AssertionError: not(2 == 1)` for the number of vouts when `-maxapsfee=0.0001` is changed to 0, and verified the new logging with an added assertion.
meshcollider:
Code review ACK 7f13dfb587dd6a7a5b7dfbfe689ae0ce818fe5c9
Tree-SHA512: 475929df57f6191bb4e36bfbcad5a280a64bb0ecd8767b76cb2e44e2301235d0eb294a3f2fac5bbf15d35d7ecfba47acb2285feadb883c9ce31c08377e3afb3c
0cdf2a77ddfa1d53c6fbd830d557a3f20d7fc365 ci: add tsan debug symbols option (Russell Yanofsky)
9a2f12680b3f00a207f1cdd4e0c50a3c7613aefc ci: Add tsan suppression for race in DatabaseBatch (Hennadii Stepanov)
Pull request description:
Since #19325 was merged, the corresponding change in TSan suppression file gets required.
This PR is:
- an analogous to #19226 and #19450, and
- a temporary workaround for CI fail like https://cirrus-ci.com/task/5741795508224000?command=ci#L4993
ACKs for top commit:
MarcoFalke:
ACK 0cdf2a77ddfa1d53c6fbd830d557a3f20d7fc365
Tree-SHA512: 7832f143887c8a0df99dea03e00694621710378fbe923e3592185fcd3658546a590693b513abffc5ab96e9ef76c9c4bff3330eeee69a0c5dbe7574f34c417220
fa3d9ce3254882c545d700990fe8e9a678f31eed rpc: Assert that RPCArg names are equal to CRPCCommand ones (rpcdump) (MarcoFalke)
fa32c1d5ec25bc53bf989a8ae68e688593d2859d rpc: Assert that RPCArg names are equal to CRPCCommand ones (zmq) (MarcoFalke)
faaa46dc204d6d714f71dbc6f0bf02215dba0f0f rpc: Assert that RPCArg names are equal to CRPCCommand ones (mining) (MarcoFalke)
fa93bc14c7411a108dd024d391344fabf0f76369 rpc: Remove unused return type from appendCommand (MarcoFalke)
Pull request description:
This is split out from #18531 to just touch the RPC methods in misc. Description from the main pr:
### Motivation
RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.
### Changes
The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.
### Future work
> Here or follow up, makes sense to also assert type of returned UniValue?
Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:
* Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
* Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
* Auto-formatting and sanity checking the RPCExamples with RPCMan
* Checking passed-in json in self-check. Removing redundant checks
* Checking returned json against documentation to avoid regressions or false documentation
* Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static
### Bugs found
* The assert identified issue #18607
* The changes itself fixed bug #19250
ACKs for top commit:
fjahr:
tested ACK fa3d9ce3254882c545d700990fe8e9a678f31eed
promag:
Code review ACK fa3d9ce3254882c545d700990fe8e9a678f31eed.
Tree-SHA512: 068ade4b55cc195868d53b7f9a27151d45b440857bb069e261a49d102a49a38fdba5d68868516a1d66a54a73ba34681362f934ded7349e894042bde873b75719
f0aa8aeea5a183ea44a877255d12db7732f2e0a8 test: add rpc_generate functional test (Jon Atack)
92d94ffb8d07cc0d2665c901de5903a3a90d5fd0 rpc: print useful help and error message for generate (Jon Atack)
8d32d2011d3f4e1d9e587d6f80dfa4a3e9f9393d test: consider generate covered in _get_uncovered_rpc_commands() (Jon Atack)
Pull request description:
This was a requested follow-up to #19133 and #17700 to alleviate confusion and head-scratching by people following tutorials that use `generate`. See https://github.com/bitcoin/bitcoin/pull/19455#issuecomment-668172916 below, https://github.com/bitcoin/bitcoin/pull/19133#issuecomment-636860943 and https://github.com/bitcoin/bitcoin/pull/17700#issuecomment-566159096.
before
```
$ bitcoin-cli help generate
help: unknown command: generate
$ bitcoin-cli generate
error code: -32601
error message:
Method not found
```
after
```
$ bitcoin-cli help generate
generate ( nblocks maxtries ) has been replaced by the -generate cli option. Refer to -help for more information.
$ bitcoin-cli generate
error code: -32601
error message:
generate ( nblocks maxtries ) has been replaced by the -generate cli option. Refer to -help for more information.
```
In the general help it remains hidden, as requested by laanwj.
```
$ bitcoin-cli help
== Generating ==
generateblock "output" ["rawtx/txid",...]
generatetoaddress nblocks "address" ( maxtries )
generatetodescriptor num_blocks "descriptor" ( maxtries )
```
ACKs for top commit:
adamjonas:
utACK f0aa8aeea5a183ea44a877255d12db7732f2e0a8
pinheadmz:
ACK f0aa8aeea5a183ea44a877255d12db7732f2e0a8
Tree-SHA512: d083652589ad3e8228c733455245001db22397559c3946e7e573cf9bd01c46e9e88b72d934728ec7f4361436ae4c74adb8f579670b09f479011924357e729af5
fa77de2baa40ee828c850ef4068c76cc3619e87b rpc: Assert that RPCArg names are equal to CRPCCommand ones (misc) (MarcoFalke)
fa50bdc755489b2e291ea5ba0e39e44a20c6c6de rpc: Limit echo to 10 args (MarcoFalke)
fa89ca9b5bd334813fd7e7edb202c56b35076e8d refactor: Use C++11 range based for loops to simplify rpc code (MarcoFalke)
fa459bdc87bbb050ca1c8d469023a96ed798540e rpc: Treat all args after a hidden arg as hidden as well (MarcoFalke)
Pull request description:
This is split out from #18531 to just touch the RPC methods in misc. Description from the main pr:
### Motivation
RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.
### Changes
The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.
### Future work
> Here or follow up, makes sense to also assert type of returned UniValue?
Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:
* Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
* Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
* Auto-formatting and sanity checking the RPCExamples with RPCMan
* Checking passed-in json in self-check. Removing redundant checks
* Checking returned json against documentation to avoid regressions or false documentation
* Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static
### Bugs found
* The assert identified issue #18607
* The changes itself fixed bug #19250
ACKs for top commit:
laanwj:
Code review ACK fa77de2baa40ee828c850ef4068c76cc3619e87b
fjahr:
tested ACK fa77de2baa40ee828c850ef4068c76cc3619e87b
theStack:
ACK fa77de2baa
ryanofsky:
Code review ACK fa77de2baa40ee828c850ef4068c76cc3619e87b. Pretty straightfoward changes
Tree-SHA512: badae1606518c0b55ce2c0bb9025d14f05556532375eb20fd6f3bfadae1e5e6568860bff8599d037e655bf1d23f1f464ca17f4db10a6ab3d502b6e9e61c7b3d3
9c54cb16de Merge #19405: rpc, cli: add network in/out connections to `getnetworkinfo` and `-getinfo` (Wladimir J. van der Laan)
19aba38cab Merge #19200: rpc: remove deprecated getaddressinfo fields (Samuel Dobson)
f5642281cc Merge #20282: wallet: change upgradewallet return type to be an object (Samuel Dobson)
Pull request description:
## Issue being fixed or feature implemented
Bitcoin backports with breaking changes
## What was done?
- bitcoin/bitcoin#20282
- bitcoin/bitcoin#19200
- bitcoin/bitcoin#19405
## How Has This Been Tested?
Run unit/functional tests
## Breaking Changes
### RPC:
- `upgradewallet` now returns object for future extensibility (#20282)
- `getnetworkinfo` now returns fields `connections_in`, `connections_out`,
`connections_mn_in`, `connections_mn_out`, `connections_mn`
that provide the number of inbound and outbound peer
connections. These new fields are in addition to the existing `connections`
field, which returns the total number of peer connections. Old fields
`inboundconnections`, `outboundconnections`, `inboundmnconnections`,
`outboundmnconnections` and `mnconnections` are removed (#19405)
- Backwards compatibility has been dropped for two `getaddressinfo` RPC
deprecations, as notified in the 19.1.0 and 19.2.0 release notes.
The deprecated `label` field has been removed as well as the deprecated `labels` behavior of
returning a JSON object containing `name` and `purpose` key-value pairs. Since
20.1, the `labels` field returns a JSON array of label names. (#19200)
### CLI
- The `connections` field of `bitcoin-cli -getinfo` is expanded to return a JSON
object with `in`, `out` and `total` numbers of peer connections and `mn_in`,
`mn_out` and `mn_total` numbers of verified mn connections. It previously
returned a single integer value for the total number of peer connections. (#19405)
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
PastaPastaPasta:
utACK 9c54cb16de
Tree-SHA512: 9710cdd062d02d64e2eebcecca1b5c2e6ccda5ca6d9bd6d1833700f4273fcfb206ae99134f71bbc8b0843cb8ebba208c72139f5a624d79ec7362bd73b117bfb2
581b343d5bf517510ab0236583ca96628751177d Add in/out connections to cli -getinfo (Jon Atack)
d9cc13e88d096c1a171159c01cbb96444f7f8d7f UNIX_EPOCH_TIME fixup in rpc getnettotals (Jon Atack)
1ab49b81cf32b6ef9e312a0a8ac45c68a3262f0d Add in/out connections to rpc getnetworkinfo (Jon Atack)
Pull request description:
This is basic info that is present in the GUI that I've been wishing to have exposed via the RPC and CLI without needing a bash workaround or script. For human users it would also be useful to have it in `-getinfo`.
`bitcoin-cli getnetworkinfo`
```
"connections": 15,
"connections_in": 6,
"connections_out": 9,
```
`bitcoin-cli -getinfo`
```
"connections": {
"in": 6,
"out": 9,
"total": 15
},
```
Update the tests, RPC help, and release notes for the changes. Also fixup the `getnettotals` timemillis help while touching `rpc/net.cpp`.
-----
Reviewers can manually test this PR by [building from source](https://jonatack.github.io/articles/how-to-compile-bitcoin-core-and-run-the-tests), launching bitcoind, and then running `bitcoin-cli -getinfo`, `bitcoin-cli getnetworkinfo`, `bitcoin-cli help getnetworkinfo`, and `bitcoin-cli help getnettotals` (for the UNIX epoch time change).
ACKs for top commit:
eriknylund:
> tACK [581b343](581b343d5b) on master at [a0a422c](a0a422c34c), ran unit & functional tests and and confirmed changes on an existing datadir ✌️
benthecarman:
tACK `581b343`
willcl-ark:
tACK for 581b343d5bf517510ab0236583ca96628751177d, this time rebased onto master at 862fde88be706adb20a211178253636442c3ae00.
shesek:
tACK `581b343`. This provides what I needed, thanks!
n-thumann:
tACK 581b343 on master at a0a422c, ran unit & functional tests and and confirmed changes on an existing datadir ✌️
Tree-SHA512: 08dd3ac8fefae401bd8253ff3ac027603c528eeccba53cedcb127771316173a7052fce44af8fa33ac98ebc4cf2a2b11cdefd949995d55e9b9a5942b876d00dc5
BACKPORT NOTICE:
These backports #17578 and #17585 are included to 19.1 and 19.2. That's long enough!
------------------------------------------
bc01f7ae0538d3c647ce8dfbc29f7914d5df3fbb doc: release note for rpc getaddressinfo removals (Jon Atack)
90e989390ee50633fff0e4f210a1ea23ff00e012 rpc: getaddressinfo RPCResult fixup (Jon Atack)
a8507c99da10791aa69ca277128e06753942e976 rpc: remove deprecated getaddressinfo `labels: purpose` (Jon Atack)
645a8653c895e4fc7717e9e5ac045612b5deaa60 rpc: remove deprecated getaddressinfo `label` field (Jon Atack)
Pull request description:
These were deprecated in #17578 and #17585, with expected 0.21 removal notified in the 0.20 release notes.
```
- The `getaddressinfo` RPC has had its `label` field deprecated
(re-enable for this release using the configuration parameter
`-deprecatedrpc=label`). The `labels` field is altered from returning
JSON objects to returning a JSON array of label names (re-enable
previous behavior for this release using the configuration parameter
`-deprecatedrpc=labelspurpose`). Backwards compatibility using the
deprecated configuration parameters is expected to be dropped in the
0.21 release. (#17585, #17578)
```
ACKs for top commit:
Sjors:
utACK bc01f7a
adamjonas:
utACK bc01f7a
meshcollider:
utACK bc01f7ae0538d3c647ce8dfbc29f7914d5df3fbb
Tree-SHA512: ae1af381e32c4c3bde8b061a56382838513a9a82c88767843cdeae3a2ab8aa7d8c2e66e106d2b31ea07d74bb80c191a2f842c9aaecc7c5438ad9a9bc66d1b251
a93de8690b refactor: s/governanceManager/govman/g (Kittywhiskers Van Gogh)
0aa08ba80d refactor: remove CGovernanceManager global, move to NodeContext (Kittywhiskers Van Gogh)
405b8c669a refactor: s/sporkManager/sporkman/g (Kittywhiskers Van Gogh)
60fd1aa774 refactor: remove CSporkManager global, move to NodeContext (Kittywhiskers Van Gogh)
24ba2f027c refactor: remove redundant condition check in `IsOldBudgetBlockValueValid` (Kittywhiskers Van Gogh)
e2405e67fb refactor: move MasternodePayments::* functions into helper class (Kittywhiskers Van Gogh)
Pull request description:
## Additional Information
* `PeerManager`'s initialization has been moved downwards (it used to be initialized _after_ `CGovernanceManager`) to now _after_ `CMasternodeSync`. This is to avoid having to pass `CSporkManager`'s `unique_ptr` container and instead pass the its dereferenced pointer.
* `CChainstateHelper` is just proxy for helper classes meant to hold references to managers that would be needed by functions that are called by `CChainState`. It's the alternative to passing every single manager into `CChainState` through `ChainstateManager`.
Instead, they're all bunched up via `CChainstateHelper` and is accessible to `CChainState` through passing it as an argument. For this reason, it should ideally initialized _after_ all relevant managers are setup but _before_ the chain is validated. We would want to avoid deferred dereferencing if we can help it.
* Internal/private functions have been marked as `[[nodiscard]]`.
## Breaking Changes
None. Changes are limited to refactoring, no logical changes have been made.
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation **(note: N/A)**
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
PastaPastaPasta:
utACK a93de8690b
Tree-SHA512: b576ca61a94b86b8a6fa8909379d156ff902198b5824fcfb4665e5eb9d1d5e250db737f84a877de634a490146b82759c2350370903f430997423fd71142106d1
0c845e3f8995eb8dc543a63899e5633a46091b4e test: Fix wallet_listdescriptors.py if bdb is not compiled (Hennadii Stepanov)
Pull request description:
If build system is configured `--without-bdb`, the `wallet_listdescriptors.py` fails:
```
$ test/functional/wallet_listdescriptors.py --descriptors
2021-07-14T13:20:52.931000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_02p7o1c9
2021-07-14T13:21:23.377000Z TestFramework (INFO): Test that the command is not available for legacy wallets.
2021-07-14T13:21:23.381000Z TestFramework (ERROR): JSONRPC error
Traceback (most recent call last):
File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/test_framework.py", line 128, in main
self.run_test()
File "test/functional/wallet_listdescriptors.py", line 34, in run_test
node.createwallet(wallet_name='w1', descriptors=False)
File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/test_node.py", line 685, in createwallet
return self.__getattr__('createwallet')(wallet_name, disable_private_keys, blank, passphrase, avoid_reuse, descriptors, load_on_startup, external_signer)
File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__
raise JSONRPCException(response['error'], status)
test_framework.authproxy.JSONRPCException: Compiled without bdb support (required for legacy wallets) (-4)
2021-07-14T13:21:23.436000Z TestFramework (INFO): Stopping nodes
2021-07-14T13:21:24.092000Z TestFramework (WARNING): Not cleaning up dir /tmp/bitcoin_func_test_02p7o1c9
2021-07-14T13:21:24.092000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/bitcoin_func_test_02p7o1c9/test_framework.log
2021-07-14T13:21:24.092000Z TestFramework (ERROR):
2021-07-14T13:21:24.092000Z TestFramework (ERROR): Hint: Call /home/hebasto/GitHub/bitcoin/test/functional/combine_logs.py '/tmp/bitcoin_func_test_02p7o1c9' to consolidate all logs
2021-07-14T13:21:24.092000Z TestFramework (ERROR):
2021-07-14T13:21:24.092000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2021-07-14T13:21:24.092000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
2021-07-14T13:21:24.092000Z TestFramework (ERROR):
```
This PR fixes this issue.
Also see #20267.
ACKs for top commit:
achow101:
ACK 0c845e3f8995eb8dc543a63899e5633a46091b4e
Tree-SHA512: d7116a9ae30c7b7e3f55f55d2eea66f9e293c38d6757ed66d0477e4256ff5fedca5ddedafa0ef0c09f4dc1f7f973163e5a46090da26b067fdddbd9ea2ee76633
c7b7e0a69265946aecc885be911c7650911ba2e3 tests: Make only desc wallets for wallet_multwallet.py --descriptors (Andrew Chow)
d4b67ad214ada7645c4ce2d5ec336fe5c3f7f7ca Avoid creating legacy wallets in wallet_importdescriptors.py (Andrew Chow)
6c9c12bf87f95066acc28ea2270a00196eb77703 Update feature_backwards_compatibility for descriptor wallets (Andrew Chow)
9a4c631e1c00eb1661c000978b133d7aa0226290 Update wallet_labels.py to not require descriptors=False (Andrew Chow)
242aed7cc1d003e8fed574bbebd19c7e54e23402 tests: Add a --legacy-wallet that is mutually exclusive with --descriptors (Andrew Chow)
388053e1722632c2e485c56a444bc75cf0152188 Disable some tests for tool_wallet when descriptors (Andrew Chow)
47d3243160fdec7e464cfb8f869be7f5d4ee25fe Make raw multisig tests legacy wallet only in rpc_rawtransaction.py (Andrew Chow)
59d3da5bce4ebd9c2291d8f201a53ee087938b21 Do addmultisigaddress tests in legacy wallet mode in wallet_address_types.py (Andrew Chow)
25bc5dccbfd52691adca6edd418dd54290300c28 Use importdescriptors when in descriptor wallet mode in wallet_createwallet.py (Andrew Chow)
0bd1860300b13b12a25d330ba3a93ff2d13aa379 Avoid dumpprivkey and watchonly behavior in rpc_signrawtransaction.py (Andrew Chow)
08067aebfd7e838e6ce6b030c31a69422260fc6f Add script equivalent of functions in address.py (Andrew Chow)
86968882a8a26312a7af29c572313c4aff488c11 Add descriptor wallet output to tool_wallet.py (Andrew Chow)
3457679870e8eff2a7d14fe59a479692738c48b6 Use separate watchonly wallet for multisig in feature_nulldummy.py (Andrew Chow)
a42652ec10c733a5bf37e418e45d4841f54331b4 Move import and watchonly tests to be legacy wallet only in wallet_balance.py (Andrew Chow)
4b871909d6e4a51888e062d322bf53263deda15e Use importdescriptors for descriptor wallets in wallet_bumpfee.py (Andrew Chow)
c2711e4230d9a423ead24f6609691fb338b1d26b Avoid dumpprivkey in wallet_listsinceblock.py (Andrew Chow)
553dbf9af4dea96e6a3e79bba9607003342029bd Make import tests in wallet_listtransactions.py legacy wallet only (Andrew Chow)
dc81418fd01021070f3f66bab5fee1484456691a Use a separate watchonly wallet in rpc_fundrawtransaction.py (Andrew Chow)
a357111047411f18c156cd34a002a38430f2901c Update wallet_importprunedfunds to avoid dumpprivkey (Andrew Chow)
Pull request description:
I went through all the tests and checked whether they passed with descriptor wallets. This partially informed some changes in #16528. Some tests needed changes to work with descriptor wallets. These were primarily due to import and watchonly behavior. There are some tests and test cases that only test legacy wallet behavior so those tests won't be run with descriptor wallets.
This PR updates more tests to have to the `--descriptors` switch in `test_runner.py`. Additionally a mutually exclusive `--legacy-wallet` option has been added to force legacy wallets. This does nothing currently but will be useful in the future when descriptor wallets are the default. For the tests that rely on legacy wallet behavior, this option is being set so that we don't forget in the future. Those tests are `feature_segwit.py`, `wallet_watchonly.py`, `wallet_implicitsegwit.py`, `wallet_import_with_label.py`, and `wallet_import_with_label.py`.
If you invert the `--descriptors`/`--legacy-wallet` default so that descriptor wallets are the default, all tests (besides the legacy wallet specific ones) will pass.
ACKs for top commit:
MarcoFalke:
review ACK c7b7e0a69265946aecc885be911c7650911ba2e3 🎿
laanwj:
ACK c7b7e0a69265946aecc885be911c7650911ba2e3
Tree-SHA512: 2f4e87815005d1d0a2543ea7947f7cd7593d8cf5312228ef85f8e096f19739b225769961943049cb44f6f07a35b8de988e2246ab9aca5bb5a0b2e62694d5637d
647b81b70938dc4dbcf32399c56f78be395c721a wallet, rpc: add listdescriptors command (Ivan Metlushko)
Pull request description:
Looking for concept ACKs
**Rationale**: allow users to inspect the contents of their newly created descriptor wallets.
Currently the command only returns xpubs which is not very useful in itself, but there are multiples ways to extend it:
* add an option to export xprv
* with #19136 it'll be possible to return normalised descriptors suitable for a watch-only purposes
The output is compatible with `importdescriptors` command so it could be easily used for backup/recover purposes.
**Output example:**
```json
[
{
"desc": "wpkh(tpubD6NzVbkrYhZ4WW6E2ZETFyNfq2hfF23SKxqSGFvUpPAY58jmmuBybwqwFihAyQPk9KnwTt5516NDZRJ7k5QPeKjy7wuVd5WvXNxwwAs5tUD/*)#nhavpr5h",
"timestamp": 1296688602,
"active": false,
"range": [
0,
999
],
"next": 0
}
]
```
ACKs for top commit:
jonatack:
re-ACK 647b81b70938dc4dbcf32399c56f78be395c721a rebased to master, debug builds cleanly, reviewed diff since last review, tested with a descriptor wallet (and with a legacy wallet)
achow101:
re-ACK 647b81b
Tree-SHA512: 51a3620bb17c836c52cecb066d4fa9d5ff418af56809046eaee0528c4dc240a4e90fff5711ba96e399c6664e00b9ee8194e33852b1b9e75af18061296e19a8a7
a66a7a1a7060bb422eba3b8c214852416c4280d1 walletdb: don't reinitialize desc cache with multiple cache entries (Andrew Chow)
Pull request description:
When loading descriptor caches, we would accidentally reinitialize the descriptor cache when seeing that one already exists. This should have only been initializing the cache when one does not exist. However this code itself is unnecessary as the act of looking up the cache to add to it will initialize it if it didn't already exist.
This issue could be hit by trying to load a wallet that had imported a multisig descriptor. The wallet would fail to load.
A test has been added to wallet_importdescriptors.py to catch this case. Another test case has also been added to check that loading a wallet with only single key descriptors works.
ACKs for top commit:
hugohn:
tACK [a66a7a1](a66a7a1a70)
jonatack:
ACK a66a7a1a706
meshcollider:
Code review ACK a66a7a1a7060bb422eba3b8c214852416c4280d1
Tree-SHA512: 3df746421a008708eaa3bbbdd12b9ddd3e2ec111d54625a212dca7414b971cc1f6e2b1757b3232c31a2f637d1b1ef43bf3ffa4ac4216646cf1e92db5f79954f1
3a83a01694160f2e722e1bc90a328bd569b8e109 [tests] move generate_wif_key to wallet_util.py (John Newbery)
b216b0b71f7853d747af8b712fc250c699f3c320 [tests] sort imports in rpc_createmultisig.py (John Newbery)
e38081846d889fcbbbc6ca4a4d3bca26807fde2f Revert "[TESTS] Move base58 to own module to break circular dependency" (John Newbery)
Pull request description:
generate_wif_key is a wallet utility function. Move it from the EC key module to the wallet util module.
This fixes the circular dependency issue in #17977
ACKs for top commit:
MarcoFalke:
ACK 3a83a01694160f2e722e1bc90a328bd569b8e109 🍪
Tree-SHA512: 24985dffb75202721ccc0c6c5b52f1fa5d1ce7963bccde24389feb913cab4dad0c265274ca67892c46c8b64e6a065a0f23263a89be4fb9134dfefbdbe5c7238a
c75de5da5fdd3a304f9da3d8a2e0370d1723ddd0 [TESTS] Move base58 to own module to break circular dependency (Pieter Wuille)
Pull request description:
I encountered difficulties with the test framework in #17977. This fixes them, and I think the change is generally useful.
ACKs for top commit:
laanwj:
Code review ACK c75de5da5fdd3a304f9da3d8a2e0370d1723ddd0
MarcoFalke:
ACK c75de5da5fdd3a304f9da3d8a2e0370d1723ddd0 according to --color-moved=dimmed-zebra this is a move-only apart from the imports 👒
Tree-SHA512: 9e0493de3e279074f0c70e92c959b73ae30479ad6f2083a3c6bbf4b0191d65ef94854559a5b7c904f5dadc5e93129ed00f6dc0a8ccce6ba7921cd45f7119f74b
facede18a42ccbf45cb5156bd4e5c9cabb359821 test: Check that invalid witness destinations can not be imported (MarcoFalke)
Pull request description:
ACKs for top commit:
practicalswift:
ACK facede18a42ccbf45cb5156bd4e5c9cabb359821 -- thanks for adding!
Tree-SHA512: 87000606fac2e6f2780ca75cdeeb2dc1f0528d9b8f13e4156e8304ce7a6b1eb014781b6f0c59d11544bf360ba3dc5f99549470b0876132e189b9107f2c6bb38d
6b71f274ae Merge bitcoin/bitcoin#29510: wallet: `getrawchangeaddress` and `getnewaddress` failures should not affect keypools for descriptor wallets (Ava Chow)
85fa37068f refactor: use Params().ExtCoinType() for descriptor wallets (Konstantin Akimov)
da8e5639ee fix: skip functional tests which requires BDB if no bdb (see 20267) (Konstantin Akimov)
4ba44fa3c9 fix: skip interface_zmq.py which is not ready to work without bdb (Konstantin Akimov)
45fc8a4863 fix: autobackup influences an exclusive locks made by SQLite (Konstantin Akimov)
e542cd2d34 fix: missing changes from bitcoin#21634 (Konstantin Akimov)
2de7aecf6f Merge #19502: Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks (Samuel Dobson)
c172605cd7 Merge #19077: wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets (Samuel Dobson)
2439247e93 Merge bitcoin/bitcoin#23608: test: fix `feature_rbf.py --descriptors` and add to test runner (fanquake)
f6b3614754 fix: descriptor wallets follow-up to merge bitcoin#20202: Make BDB support optional (Konstantin Akimov)
a340ad641e Merge #20262: tests: Skip --descriptor tests if sqlite is not compiled (Samuel Dobson)
7d55046dfb Merge #20125: rpc, wallet: Expose database format in getwalletinfo (Samuel Dobson)
343d4b07d3 fix: descriptor wallets follow-up for bitcoin#20156: Make sqlite support optional (compile-time) (Konstantin Akimov)
fa30777494 Merge #20198: Show name, format and if uses descriptors in bitcoin-wallet tool (MarcoFalke)
14121ec5f3 Merge #18888: test: Remove RPCOverloadWrapper boilerplate (MarcoFalke)
b18351e415 Merge #20153: wallet: do not import a descriptor with hardened derivations into a watch-only wallet (Wladimir J. van der Laan)
c995e5d957 Merge #20266: wallet: fix change detection of imported internal descriptors (Wladimir J. van der Laan)
c86458250c Merge #18787: wallet: descriptor wallet release notes and cleanups (Samuel Dobson)
0949c08996 Merge #18782: wallet: Make sure no DescriptorScriptPubKeyMan or WalletDescriptor members are left uninitialized after construction (Samuel Dobson)
baa6959068 Merge #18805: tests: Add missing sync_all to wallet_importdescriptors.py (MarcoFalke)
76e08f9b3d Merge #18027: "PSBT Operations" dialog (Samuel Dobson)
c1b94b6f52 fix: wallet should be unlocked before generating keys for Descriptor wallet (Konstantin Akimov)
f293c046f4 Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan (Andrew Chow)
4064334732 fix: get receiving address for Descriptor Wallets (Konstantin Akimov)
bdbd0b14a7 chore: dashification of descriptor implementation in dash (UdjinM6)
b02fc0b2ce fix: counting calculation of internal keys for Descriptor Wallets (Konstantin Akimov)
Pull request description:
## Issue being fixed or feature implemented
This PR is a batch of backports and related fixes to add a support of native descriptor wallets to Dash Core.
There're more related backports, but this PR is a minimal package of backports to get descriptor wallets working and unit/functional tests to succeed. To do: bitcoin#20226, bitcoin#21049, bitcoin#18788, bitcoin#20267, bitcoin#19230, bitcoin#19239, bitcoin#19441, bitcoin#19568, bitcoin#19979, bitcoin-core/gui#96, bitcoin#19136, bitcoin#21277, bitcoin#21063, bitcoin#21302, bitcoin#19651, bitcoin#20191, bitcoin#22446 and other.
Prior work:
- https://github.com/dashpay/dash/pull/5580
- https://github.com/dashpay/dash/pull/5807
## What was done?
backports:
- bitcoin/bitcoin#16528
- bitcoin/bitcoin#18027
- bitcoin/bitcoin#18805
- bitcoin/bitcoin#18782
- bitcoin/bitcoin#18787
- bitcoin/bitcoin#20266
- bitcoin/bitcoin#20153
- bitcoin/bitcoin#18888
- bitcoin/bitcoin#20198
- bitcoin/bitcoin#20125
- bitcoin/bitcoin#20262
- bitcoin/bitcoin#23608
- bitcoin/bitcoin#19077
- bitcoin/bitcoin#19502
- bitcoin/bitcoin#29510
and extra fixes and missing changes for bitcoin#20156, bitcoin#20202, bitcoin#20267, bitcoin#21634 + fix of auto-backup for sqlite wallets.
## How Has This Been Tested?
There're 2 new functional tests: `wallet_importdescriptors.py` and `wallet_descriptor.py`
Beside that many functional tests run twice now: using legacy wallet and descriptor wallets: `wallet_hd.py`, `wallet_basic.py`, `wallet_labels.py`, `wallet_keypool_topup.py`, `wallet_avoidreuse.py`, `rpc_psbt.py`, `wallet_keypool_hd.py`, `rpc_createmultisig.py`, `wallet_encryption.py`.
With bitcoin#18788 expected to more tests run.
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
PastaPastaPasta:
Rebase looks good; utACK 6b71f274ae
PastaPastaPasta:
utACK 6b71f274ae
UdjinM6:
utACK 6b71f27
kwvg:
utACK 6b71f274ae
Tree-SHA512: 776c5dfe1eec2b5bebc8d606476cd981c810ac81965b348e78c13e96fff23be500c495ae68c93f669403941c96eccdd3775f2b96572163c34175900e15549b5d
2ead31fb1b17c9b183a4b81f0ae4f48e5cf67d64 [wallet] Return object from upgradewallet RPC (Sishir Giri)
Pull request description:
Change the return type of upgradewallet to be an object for future extensibility.
Also return any error string returned from the `UpgradeWallet()` function.
ACKs for top commit:
MarcoFalke:
ACK 2ead31fb1b17c9b183a4b81f0ae4f48e5cf67d64
meshcollider:
Tested ACK 2ead31fb1b17c9b183a4b81f0ae4f48e5cf67d64
Tree-SHA512: bcc7432d2f35093ec2463ea19e894fa885b698c0e8d8e4bd2f979bd4d722cbfed53ec589d6280968917893c64649dc9e40800b8d854273b0f9a1380f51afbdb1
e073f1dfda7a2a2cb2be9fe2a1d576f122596021 test: make sure keypool sizes do not change on `getrawchangeaddress`/`getnewaddress` failures (UdjinM6)
367bb7a80cc71130995672c853d4a6e0134721d6 wallet: Avoid updating `ReserveDestination::nIndex` when `GetReservedDestination` fails (UdjinM6)
Pull request description:
I think the expected behaviour of `getrawchangeaddress` and `getnewaddress` RPCs is that their failure should not affect keypool in any way. At least that's how legacy wallets work, you can confirm this behaviour by running `wallet_keypool.py --legacy-wallet` on master with e073f1dfda7a2a2cb2be9fe2a1d576f122596021 applied on top. However running `wallet_keypool.py --descriptors` on the same commit results in the following failure:
```
File "/path/to/bitcoin/test/functional/test_framework/test_framework.py", line 131, in main
self.run_test()
File "/path/to/bitcoin/test/functional/wallet_keypool.py", line 114, in run_test
assert_equal(kp_size_before, kp_size_after)
File "/path/to/bitcoin/test/functional/test_framework/util.py", line 57, in assert_equal
raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not([18, 24] == [19, 24])
```
This happens because we pass `nIndex` (which is a class member) into `GetReservedDestination` and since it's passed by reference we get an updated value back, so `nIndex` won't be equal `-1` anymore, no matter if the function failed or succeeded. This means that `ReturnDestination` (called by dtor of `ReserveDestination`) will try to return something we did not actually reserve.
The fix is to simply use a temporary variable instead of a class member and only update `nIndex` when `op_address` actually has value, basically do it the same way we do for other class members (`address` and `fInternal`) already.
ACKs for top commit:
achow101:
ACK e073f1dfda7a2a2cb2be9fe2a1d576f122596021
josibake:
ACK e073f1dfda
Tree-SHA512: 1128288a60dd4d8f306ef6f7ac66cdfeae3c9cc35c66ecada2d78fa61ac759f2a757b70fc3976ba8b5081200942b58dfabc184c01ccf911af40ba8c145344651
Skipped tests:
- feature_filelock
- wallet_descriptors
- interface_bitcoin_cli
And all functional tests which explitely specify they are using non-descriptor wallets
24d2d3341d07509ad3f37bb6f130446ad20ac807 QA: wallet_multiwallet: Check that recursive symlink directory and wallet.dat loops are ignored (Luke Dashjr)
69f59af54d15ee9800d5df86bcdb0e962c71e7c3 Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks (Luke Dashjr)
Pull request description:
Previously, an exception would be thrown, which could kill the node in some circumstances.
Includes test changes to cause failure.
Review with `?w=1`
ACKs for top commit:
hebasto:
re-ACK 24d2d3341d07509ad3f37bb6f130446ad20ac807, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/19502#pullrequestreview-520552944) review.
promag:
Tested ACK 24d2d3341d07509ad3f37bb6f130446ad20ac807, test change fails on master.
meshcollider:
utACK 24d2d3341d07509ad3f37bb6f130446ad20ac807
Tree-SHA512: f701f81b3aa3d3e15cee52ac9e7c31a73c0d8166e56bf077235294507cbcee099829fedc432a1c4b6d8780885f4e37897b44b980b08125771de3c849c000499e
c4a29d0a90b821c443c10891d9326c534d15cf97 Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky)
310b0fde04639b7446efd5c1d2701caa4b991b86 Run dumpwallet for legacy wallets only in wallet_backup.py (Andrew Chow)
6c6639ac9f6e1677da066cf809f9e3fa4d2e7c32 Include sqlite3 in documentation (Andrew Chow)
f023b7cac0eb16d3c1bf40f1f7898b290de4cc73 wallet: Enforce sqlite serialized threading mode (Andrew Chow)
6173269866306058fcb1cc825b9eb681838678ca Set and check the sqlite user version (Andrew Chow)
9d3d2d263c331e3c77b8f0d01ecc9fea0407dd17 Use network magic as sqlite wallet application ID (Andrew Chow)
9af5de3798c49f86f27bb79396e075fb8c1b2381 Use SQLite for descriptor wallets (Andrew Chow)
9b78f3ce8ed1867c37f6b9fff98f74582d44b789 walletutil: Wallets can also be sqlite (Andrew Chow)
ac38a87225be0f1103ff9629d63980550d2f372b Determine wallet file type based on file magic (Andrew Chow)
6045f77003f167bee9a85e2d53f8fc6ff2e297d8 Implement SQLiteDatabase::MakeBatch (Andrew Chow)
727e6b2a4ee5abb7f2dcbc9f7778291908dc28ad Implement SQLiteDatabase::Verify (Andrew Chow)
b4df8fdb19fcded7e6d491ecf0b705cac0ec76a1 Implement SQLiteDatabase::Rewrite (Andrew Chow)
010e3659069e6f97dd7b24483f50ed71042b84b0 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow)
ac5c1617e7f4273daf24c24da1f6bc5ef5ab2d2b Implement SQLiteDatabase::Backup (Andrew Chow)
f6f9cd6a64842ef23777312f2465e826ca04b886 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow)
bf90e033f4fe86cfb90492c7e0962278ea3a146d Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow)
7aa45620e2f2178145a2eca58ccbab3cecff08fb Add SetupSQLStatements (Andrew Chow)
6636a2608a4e5906ee8092d5731595542261e0ad Implement SQLiteBatch::Close (Andrew Chow)
93825352a36456283bf87e39b5888363ee242f21 Implement SQLiteDatabase::Close (Andrew Chow)
a0de83372be83f59015cd3d61af2303b74fb64b5 Implement SQLiteDatabase::Open (Andrew Chow)
3bfa0fe1259280f8c32b41a798c9453b73f89b02 Initialize and Shutdown sqlite3 globals (Andrew Chow)
5a488b3d77326a0d957c1233493061da1b6ec207 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow)
ca8b7e04ab89f99075b093fa248919fd10acbdf7 Implement SQLiteDatabaseVersion (Andrew Chow)
7577b6e1c88a1a7b45ecf5c7f1735bae6f5a82bf Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow)
e87df8258090138d5c22ac46b8602b618620e8a1 Add sqlite to travis and depends (Andrew Chow)
54729f3f4e6765dfded590af5fb28c88331685f8 Add libsqlite3 (Andrew Chow)
Pull request description:
This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`.
For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite.
We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use.
I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier.
ACKs for top commit:
Sjors:
re-utACK c4a29d0a90b821c443c10891d9326c534d15cf97
promag:
Tested ACK c4a29d0a90b821c443c10891d9326c534d15cf97.
fjahr:
reACK c4a29d0a90b821c443c10891d9326c534d15cf97
S3RK:
Re-review ACK c4a29d0a90b821c443c10891d9326c534d15cf97
meshcollider:
re-utACK c4a29d0a90b821c443c10891d9326c534d15cf97
hebasto:
re-ACK c4a29d0a90b821c443c10891d9326c534d15cf97, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19077#pullrequestreview-507743699) review, verified with `git range-diff master d18892dcc c4a29d0a9`.
ryanofsky:
Code review ACK c4a29d0a90b821c443c10891d9326c534d15cf97. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction.
jonatack:
ACK c4a29d0a90b821c443c10891d9326c534d15cf97, debug builds and test runs after rebase to latest master @ c2c4dbaebd9, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns.
Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c
b79dbe86a9886b3539512f6c35205f4c5454a952 test: add feature_rbf.py --descriptors to test_runner.py (Sebastian Falbesoner)
166f8ec28e48aa4c0cd94544909c488df553d8ef test: always rescan after importing private keys in `init_wallet` helper (Sebastian Falbesoner)
Pull request description:
The functional test feature_rbf.py currently fails on master branch, if descriptor wallets are used (argument `--descriptors`). This is due to the fact that in this case, a call to the helper `init_wallet`
111c3e06b3/test/functional/test_framework/test_framework.py (L428-L434)
creates a wallet without rescanning the blockchain; the test framework maps the importprivkey RPC calls to the importdescriptors RPC without rescanning by default (timestamp='now'). Fix this by always calling with `rescan=True`, which calls importdescriptors with timestamp=0. Also add `feature_rbf.py --descriptors` to the list of the test runner's calls.
Fixes#23563.
ACKs for top commit:
mjdietzx:
ACK b79dbe86a9886b3539512f6c35205f4c5454a952
Tree-SHA512: a3f3f7a4077066e3c910919d3b5e04bc6b580c1e0a06e9a2fc258950eaea5e59c0f805c8f00432aea722609f2f7e41eebfab653471b76729c5a316825a3d8c86
7411876c75471a45ad40c38c668db3a4b9413fb9 Ensure a legacy wallet for BDB format check (Andrew Chow)
586640381a2c379ce3d6366b1b4534ccc4e8ccf2 Skip --descriptor tests if sqlite is not compiled (Andrew Chow)
Pull request description:
#20156 allows sqlite to not be compiled by configuring `--without-sqlite`. However doing so and then running the test runner will result in all of the `--descriptor` tests to fail. We should be skipping those tests if sqlite was not compiled.
ACKs for top commit:
practicalswift:
ACK 7411876c75471a45ad40c38c668db3a4b9413fb9: patch looks correct
Sjors:
tACK 7411876c75471a45ad40c38c668db3a4b9413fb9
ryanofsky:
Code review ACK 7411876c75471a45ad40c38c668db3a4b9413fb9
hebasto:
ACK 7411876c75471a45ad40c38c668db3a4b9413fb9, tested on Linux Mint 20 (x86_64), tests pass for binaries compiled with:
Tree-SHA512: 1d635a66d2b7bb865300144dfefcfdaf86133aaaa020c8f440a471476ac1205d32f2df704906ce6c2ea48ddf791c3c95055f6291340b4f7b353c1b02cab5cabe
624bab00dd2cc8e2ebd77dc0a669bc8d507c3721 test: add coverage for getwalletinfo format field (Jon Atack)
5e737a009234cbd7cf53748d3d28a2da5221192f rpc, wallet: Expose database format in getwalletinfo (João Barbosa)
Pull request description:
Support for sqlite based wallets was added in #19077. This PR adds the `format` key in `getwalletinfo` response, that can be `bdb` or `sqlite`.
ACKs for top commit:
jonatack:
Tested ACK 624bab00dd2cc8e2ebd77dc0a669bc8d507c3721
laanwj:
Code review ACK 624bab00dd2cc8e2ebd77dc0a669bc8d507c3721.
MarcoFalke:
doesn't hurt ACK 624bab00dd2cc8e2ebd77dc0a669bc8d507c3721
hebasto:
ACK 624bab00dd2cc8e2ebd77dc0a669bc8d507c3721, tested on Linux Mint 20 (x86_64).
meshcollider:
utACK 624bab00dd2cc8e2ebd77dc0a669bc8d507c3721
Tree-SHA512: a81f8530f040f6381d33e073a65f281993eccfa717424ab6e651c1203cbaf27794dcb7175570459e7fdaa211565bc060d0a3ecbe70d2b6f9c49b8d5071e4441c
fa4074b395a47c54069bd9f598244701505ff11d Show name, format and if uses descriptors in bitcoin-wallet tool (Jonas Schnelli)
Pull request description:
ACKs for top commit:
MarcoFalke:
ACK fa4074b395a47c54069bd9f598244701505ff11d
jonatack:
re-ACK fa4074b395a47c54069bd9f598244701505ff11d
Tree-SHA512: cf6ee96ff21532fc4b0ba7a0fdfdc1fa485c9b1495447350fe65cd0bd919e0e0280613933265cdee069b8c29ccf015ac374535a70cac3d4fb89f4d08b3a03519
faa26d374425f52e03efff3a575c391b7862abe5 test: Remove RPCOverloadWrapper boilerplate (MarcoFalke)
Pull request description:
There are too many wrappers in test_node already, so at least the code that implements the wrappers should be as minimal as possible.
ACKs for top commit:
laanwj:
code review ACK faa26d374425f52e03efff3a575c391b7862abe5
Tree-SHA512: 94e593907de22187524e2445afb3101e40b3b599d4b4015aa8c6ca902d7586ff9daf520828759029d199a3af79e61b96b490a822a5a193ac7bf946beacb11a24
538be4219ae7e65862e4aff540af88c9421e6061 wallet: fix importdescriptor silent fail (Ivan Metlushko)
Pull request description:
Currently `importdescriptor` command will successfully import a descriptor with hardened derivations into a watch-only wallet while silently failing to expand the descriptor to fill the cache. This leads to a broken wallet state and failure to load such wallet due to missing cache on subsequent restart.
ACKs for top commit:
laanwj:
Code review ACK 538be4219ae7e65862e4aff540af88c9421e6061
achow101:
ACK 538be4219ae7e65862e4aff540af88c9421e6061
meshcollider:
utACK 538be4219ae7e65862e4aff540af88c9421e6061
Tree-SHA512: 4bdd0ab4437d55b3f1a79c3a300a0b186089155c020fe220a73d0cce274de47d90371d88918d39fd795f9fccf8db328f1e322d29a6062f9ce94a1c254398f004
bd93fc9945bfd4be117990c5d861f61ddd451f96 Fix change detection of imported internal descriptors (Andrew Chow)
Pull request description:
Import internal descriptors were having address book entries added which meant they would be detected as non-change. Fix this and add a test for it.
ACKs for top commit:
laanwj:
Code review ACK bd93fc9945bfd4be117990c5d861f61ddd451f96
meshcollider:
utACK bd93fc9945bfd4be117990c5d861f61ddd451f96
promag:
Code review ACK bd93fc9945bfd4be117990c5d861f61ddd451f96.
Tree-SHA512: 8fa9e364be317627ec171eedffdb505976c0e7f1e55bc7e8cfdffa3aeea5db24d231f55166602cd0e97a5ba621acc871de0a765c75d0c65678f83e93c3b657c5
7b2b06dfe3061b5ab4a283245930e2f7773eb3ef tests: Add missing sync_all to wallet_importdescriptors.py (Andrew Chow)
Pull request description:
node1 will sometimes do sendtoaddress before it has received a funding transaction which will cause the test to fail. sync_all to ensure it gets the transaction first.
Fixes#18800
ACKs for top commit:
instagibbs:
ACK 7b2b06d The wallet endpoint right after is indeed node 1.
Tree-SHA512: b610a771d062b5f955cd70b34337577a1ab8dacbf4be20aa74e1e8234495b0be9faff138eb1713f29decb5574446e0583e221bc2c9a6eea13611b422ea3a296a
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow)
869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow)
886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow)
f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow)
82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow)
953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)
Pull request description:
Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.
Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.
Descriptors can also be imported with a new `importdescriptors` RPC.
Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.
A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).
ACKs for top commit:
Sjors:
utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed)
jonatack:
Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82.
fjahr:
re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
instagibbs:
light re-ACK 223588b
meshcollider:
Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it
Introduce DescriptorScriptPubKeyMan as a dummy class
Add WALLET_FLAG_DESCRIPTORS
Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet
Create LegacyScriptPubKeyMan when not a descriptor wallet
Introduce WalletDescriptor class
WalletDescriptor is a Descriptor with other wallet metadata
Add a lock cs_desc_man for DescriptorScriptPubKeyMan
Store WalletDescriptor in DescriptorScriptPubKeyMan
Implement SetType in DescriptorScriptPubKeyMan
Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet
Implement IsMine for DescriptorScriptPubKeyMan
Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE
Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan
Implement IsHDEnabled in DescriptorScriptPubKeyMan
Implement GetID for DescriptorScriptPubKeyMan
Load the descriptor cache from the wallet file
Implement loading of keys for DescriptorScriptPubKeyMan
Add IsSingleType to Descriptors
IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys
Implement several simple functions in DescriptorScriptPubKeyMan
Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB
Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file
Implement SetupGeneration for DescriptorScriptPubKeyMan
Implement TopUp in DescriptorScriptPubKeyMan
Implement GetNewDestination for DescriptorScriptPubKeyMan
Implement Unlock and Encrypt in DescriptorScriptPubKeyMan
Implement GetReservedDestination in DescriptorScriptPubKeyMan
Implement ReturnDestination in DescriptorScriptPubKeyMan
Implement GetKeypoolOldestTime and only display it if greater than 0
Implement GetSolvingProvider for DescriptorScriptPubKeyMan
Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.
Implement SignTransaction in DescriptorScriptPubKeyMan
Implement SignMessage for descriptor wallets
Implement FillPSBT in DescriptorScriptPubKeyMan
FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.
Change GetMetadata to use unique_ptr<CKeyMetadata>
Implement GetMetadata in DescriptorScriptPubKeyMan
Be able to create new wallets with DescriptorScriptPubKeyMans as backing
Generate new descriptors when encrypting
Add IsLegacy to CWallet so that the GUI knows whether to show watchonly
add importdescriptors RPC and tests for native descriptor wallets
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Functional tests for descriptor wallets
Change wallet_encryption.py to use signmessage instead of dumpprivkey
Return error when no ScriptPubKeyMan is available for specified type
When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.
Implement CWallet::IsSpentKey for non-LegacySPKMans
tests: Add RPCOverloadWrapper which overloads some disabled RPCs
RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.
Add a --descriptors option to various tests
Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.
Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py <---- wallet_keypool_hd.py actually
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
df554f396b test: add multiple suppression for cppcheck to make it finally quiet (Konstantin Akimov)
e2ac86b8f7 refactor: drop dependency of CJ to fee_estimator (Konstantin Akimov)
e7e355ba8b refactor: make SetNull in CJ classes virtual to prevent warning from compiler (Konstantin Akimov)
6bc14a35e5 chore: bump cpp check 2.10 to 2.13 (Konstantin Akimov)
b4ed65e15e refactor: add multiple missing `const` (Konstantin Akimov)
add99ea862 refactor: add more consts everywhere as required by cppcheck 2.13.0 (Konstantin Akimov)
61a6c407fe fix: re-order asserts in creditpool (Konstantin Akimov)
28f7ecc11f test: tidy up and reorder warnings in lint-cppcheck-dash (Konstantin Akimov)
3abeab16d3 refactor: replace multiple C-style casts to reinterpret_cast (Konstantin Akimov)
f36bb1f085 style: add semicolumn that the end to ENTER/LEAVE CRITICAL SECTION (Konstantin Akimov)
c23c25d00b test: supress any_of suggestions in cppcheck-dash (Konstantin Akimov)
7daa9bea9a refactor: use std::any_of (Konstantin Akimov)
3f1cb8d13d test: add suppression for state.Invalid() which is widely used by cppcheck is unhappy (Konstantin Akimov)
Pull request description:
## Issue being fixed or feature implemented
https://github.com/dashpay/dash-issues/issues/67
See also https://github.com/dashpay/dash/pull/5880 for prior work.
There's too many false alarm warnings with new kubuntu 23.10 (cppcheck 2.11)
## What was done?
Bump cppcheck to version 2.13 and fix related warnings or suppressing them if they are false-alarm.
## How Has This Been Tested?
Run `test/lint/lint-cppcheck-dash.sh` and see no output.
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
Top commit has no ACKs.
Tree-SHA512: 352a36958102ee687a2f065691a5113f90a014816683335b11291a556ad09c7f8797c6ced98f0851f39f597c26493f2d5a2bf54c4e833a37d35487384b7102db
even it maybe useful lint message for some particular case, sometimes it asks
to make an refactoring that will be over-complex.
For example, it asks to refactor external loop to std::any_of:
```
for (const auto& inner_entry : vecEntries) {
if (ranges::any_of(inner_entry.vecTxDSIn,
[&txin](const auto& txdsin){
return txdsin.prevout == txin.prevout;
})) {
LogPrint(BCLog::COINJOIN, "CCoinJoinServer::%s -- ERROR: already have this txin in entries\n", __func__);
nMessageIDRet = ERR_ALREADY_HAVE;
// Two peers sent the same input? Can't really say who is the malicious one here,
// could be that someone is picking someone else's inputs randomly trying to force
// collateral consumption. Do not punish.
return false;
}
}
```
That's possible to refactor, but that's unreasonable complexity to have an
lambda inside an lambda... That's unreasonable.
Some other suggestion are also non-trivial.
One more suppression for any_of in llmq/commitment which is false-alarm:
There's used index but linter doesn't see it:
```
for (const auto i : irange::range(members.size(), size_t(llmq_params.size))) {
if (validMembers[i]) {
LogPrintfFinalCommitment("q[%s] invalid validMembers bitset. bit %d should not be set\n", quorumHash.ToString(), i);
return false;
}
if (signers[i]) {
LogPrintfFinalCommitment("q[%s] invalid signers bitset. bit %d should not be set\n", quorumHash.ToString(), i);
return false;
}
}
```
b5f4411d11 fix: extra logs to distinct WriteHDChain for encrypted/raw batches (Konstantin Akimov)
e8f84afd3e refactor: move BIP39 related code to wallet (Konstantin Akimov)
fa6519f320 refactor: move hdchain to wallet/ because it belongs there (Konstantin Akimov)
392b51b197 refactor: obsolete hdCryptedChain from ScriptPubKeyMan (Konstantin Akimov)
a0389e181f refactor: rename EncryptHDChain to EncryptAndSetHDChain (Konstantin Akimov)
b2143f9346 refactor: re-order public and private methods in scriptpubkeyman (Konstantin Akimov)
a219748f5e refactor: make SetHDChain private in ScriptPubKeyManager (Konstantin Akimov)
f418b9ac62 refactor: move Encrypt chain call inside ScriptPubKeyMan::Encrypt (Konstantin Akimov)
c3754d5183 refactor: move DecryptHDChain from public to private (Konstantin Akimov)
e08b64a1bb refactor: unify SetHDChainSingle and SetCryptedHDChainSingle (Konstantin Akimov)
59a998843b refactor: unify ScriptPubKeyMan's SetHDChain nad SetHDCryptedChain (Konstantin Akimov)
a180d73e5c refactor: unify WalletBatch's WriteHDChain and WriteCryptedHDChain (Konstantin Akimov)
9c8e4584ae refactor: unify SetHDChain and SetCryptedHDChain helpers (Konstantin Akimov)
7b72726b3e refactor: SetCryptedHDChain moved to scriptpubkeyman (Konstantin Akimov)
6993d112f3 feat: extra check failure case CWallet::GenerateNewHDChainEncrypted (Konstantin Akimov)
Pull request description:
## Issue being fixed or feature implemented
During backport bitcoin#17261 significant part of HD chain has been forgotten in CWallet due to our own implementation.
This PR do not change behaviour of HD wallets, it's just refactoring.
## What was done?
This PR refactor HD wallets implementation:
- key related stuff is moved from CWallet to LegacyScriptPubKeyMan (to follow-up bitcoin#17261)
- refactored duplicated code between hdChain and hdCryptedChain
- modules hdchain and bip39 related moved to wallet/ module
## How Has This Been Tested?
Run unit/functional tests
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
Top commit has no ACKs.
Tree-SHA512: 763ea6dd9af902dfc4dd99896c1cb16a75e3c0322e806e37ff05bc9229923057c544732ebf9fca705da870568643370867d3f81a46cab1e9d229b2949016c484
2aac093a3d60e446b85eebdf170ea6bed77bec92 test: Add test coverage for -networkactive option (Hennadii Stepanov)
3c58129b1293742a49aa196cb210ff345a7339e6 net: Log network activity status change unconditionally (Hennadii Stepanov)
62fe6aa87e4cdd8b06207abc1387c68d7bfc04c1 net: Add -networkactive option (Hennadii Stepanov)
Pull request description:
Some Bitcoin Core activity is completely local (offline), e.g., reindexing.
The `setnetworkactive` RPC command is already present. This PR adds the corresponding command-line argument / config option, and allows to start the client with disabled p2p network by providing `-networkactive=0` or `-nonetworkactive`.
This was done while reviewing #16981.
ACKs for top commit:
MarcoFalke:
re-ACK 2aac093a3d60e446b85eebdf170ea6bed77bec92 🏠
LarryRuane:
ACK 2aac093a3d60e446b85eebdf170ea6bed77bec92
Tree-SHA512: 446d791b46d7b556d7694df7b1f88cd4fbc09301fe4eaf036b45cb8166ed806156353cc03788a07b633d5887d5eee30a7c02a2d4307141c8ccc75e0a88145636
fa1cd9e1ddc6918c3d600d36eadea71eebb242b6 test: Remove unused lock arg from BitcoinTestFramework.wait_until (MarcoFalke)
fad2794e93b4f5976e81793a4a63aa03a2c8c686 test: Rename wait until helper to wait_until_helper (MarcoFalke)
facb41bf1d1b7ee552c627f9829b4494b817ce28 test: Remove unused p2p_lock in VersionBitsWarningTest (MarcoFalke)
Pull request description:
This avoids confusion with the `wait_until` member functions, which should be preferred because they take the appropriate locks and scale the timeout appropriately on their own.
ACKs for top commit:
laanwj:
Code review ACK fa1cd9e1ddc6918c3d600d36eadea71eebb242b6
hebasto:
ACK fa1cd9e1ddc6918c3d600d36eadea71eebb242b6, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 319d400085606a4c738e314824037f72998e6657d8622b363726842aba968744f23c56d27275dfe506b8cbbb6e97fc39ca1d325db05d4d67df0e8b35f2244d5c
ed5cd12869e0691a785199d2d977ce5879095180 test: Distinguish between nodes(bitcoind) and peers(mininodes) in p2p_leak.py (Dhruv Mehta)
f6f082b9343522bc8005f23937ac6ecf56548c98 test: remove `CNodeNoVersionIdle` from p2p_leak.py (Dhruv Mehta)
45cf55ccac94689e48dd0648ed2401918a778024 test: remove `CNodeNoVersionMisbehavior` from p2p_leak.py (Dhruv Mehta)
Pull request description:
- Removes `CNodeNoVersionMisbehavior` per recommendation at https://github.com/bitcoin/bitcoin/pull/19657#issuecomment-669926458
- Removes `CNodeNoVersionIdle` because it is similarly unnecessary
- As someone new to the codebase, I found it easier to understand it if `no_version_disconnect_node` tries to overwhelm the peer with any message that is not version/verack.
- Per recommendation at https://github.com/bitcoin/bitcoin/pull/19727#pullrequestreview-468093555, made a clear distinction between nodes(bitcoind) and peers(mininode interface implementations)
ACKs for top commit:
jnewbery:
tested ACK ed5cd12869e0691a785199d2d977ce5879095180
amitiuttarwar:
utACK ed5cd12869
Tree-SHA512: 310a24c91fd837e7f65177edb55fe6142fb3559fae7867c5cdd9c9a23b1a02202b935ca9a82633fa7649f3de2fa221f6da906a7b5e499fc20f7254085033757d
fa4dfd215f62e88d668311701735c332c264fa2a test: Wait until is_connected in add_p2p_connection (MarcoFalke)
Pull request description:
Moving the wait_until from the individual test scripts to the test framework simplifies two tests
ACKs for top commit:
jnewbery:
Code review ACK fa4dfd215f62e88d668311701735c332c264fa2a
theStack:
ACK fa4dfd215f☕
Tree-SHA512: 36eda7eb323614a4c4f9215f1d7b40b9f9c4036d1c08eb701ea705f3e2986fdabd2fc558965a6aadabeed861034aeaeef3c00f968ca17ed7a27e42e506cda87d
faa9a74c9e99eb43ba0d27fa906767ee88011aeb test: Fail wait_until early if connection is lost (MarcoFalke)
Pull request description:
Calling `minonode.wait_until` needs a connection to make progress (e.g. waiting for an inv), unless the mininode waits for the initial connection or for a disconnection. So for test development and failure debugging, fail early in all `wait_until`, unless opted out.
ACKs for top commit:
jnewbery:
Code review ACK faa9a74c9e99eb43ba0d27fa906767ee88011aeb.
Tree-SHA512: 4be850b96e23b87bc2ff42c028a5045d6f5cdbc9482ce6a6ba01cc5eb26710dab9e2ed547c363aac4bd5825151ee9996fb797261420b631bceeddbfa698d1dec
fa3365430c5fb57d7c0b5f2bce9fbbe290be93c3 net: Use mockable time for ping/pong, add tests (MarcoFalke)
faab4aaf2fa1153c6d76efc8113fa01b06943ece util: Add count_microseconds helper (MarcoFalke)
Pull request description:
Switch `CNode::m_ping_start` and `CNetMessage::m_time` to mockable time, so that tests can be added.
Mockable time is also type-safe, since it uses `std::chrono`
ACKs for top commit:
jonatack:
Code review re-ACK fa33654 re-read code, verified rebase per `git range-diff 4b5c919 fa94d6f fa33654`, previous tested ACKs still valid
troygiorshev:
ACK fa3365430c5fb57d7c0b5f2bce9fbbe290be93c3
Tree-SHA512: 7d632bd6019ce7c882029e71b667a61517e783af82755a85dd979ef09380934e172dec8b8f91d57b200a30a6e096aeaf01f19fee7f3aed0e0e871c72eb44d70e
4aff7a48a4e0f1075306f181a276b8a74c857022 test: check importing wallets when blocks are pruned throw an error (brunoerg)
Pull request description:
This PR adds test coverage for the following error:
437b608df2/src/wallet/rpc/backup.cpp (L513-L518)
ACKs for top commit:
andrewtoth:
ACK 4aff7a48a4e0f1075306f181a276b8a74c857022
Tree-SHA512: fbbf6056cb3759f726b8a5ff25fca51bf47e973e5d655ec164e2bec88e2dbd3b243677869d2cf33af268ea635ca0f2e9f737c4734077fc5a936ac3a24ad4b88b
fadddd13eef4428f5fa7237583d4be41a9335cd9 test: Add missing syncwithvalidationinterfacequeue (MarcoFalke)
faa211fc6e3d4984b8edff1d762dd4cba205d982 test: Misc cleanup (MarcoFalke)
fa1668bf5084a190b26b022b9e625a7be3defa6e test: Run pep-8 (MarcoFalke)
facd97ae0f0d816107aa3bc9de321244200636a0 scripted-diff: Renames (MarcoFalke)
Pull request description:
The index on the block filters is running in the background on the validation interface. To avoid intermittent test failures, it needs to be synced.
Also other cleanups.
ACKs for top commit:
lsilva01:
Tested ACK fadddd13ee on Ubuntu 20.04
Tree-SHA512: d858405db426a2f9d5620059dd88bcead4e3fba3ccc6bd8023f624b768fbcfa2203246fb0b2db620490321730d990f0e78063b21a26988c969cb126d4bd697bd
39a9ec579f023ab262a1abd1f0c869be5b1f3f4d Unconditionally check for fRelay field in test framework (Troy Giorshev)
Pull request description:
picking up #20411 (rebased onto master)
There is a discrepancy in the implementation of our p2p protocol between
bitcoind and the testing framework. The fRelay field is an optional
field at the end of a version message as of protocol version 70001.
However, when deserializing a message in bitcoind, we don't check the
version to see if it should have an fRelay field or not. Instead, we
unconditionally attempt to deserialize into the field.
This commit brings the testing framework in line with the implementation
in core.
This matters for a version message with the following fields:
Version = 60000
fRelay = 1
Bitcoind would deserialize this into a version message with
Version=60000 and fRelay=1, whereas (before this commit) our testing
framework would deserialize this into a version message with
Version=60000 and fRelay=0.
ACKs for top commit:
jnewbery:
utACK 39a9ec579f023ab262a1abd1f0c869be5b1f3f4d
Tree-SHA512: 13a23f1180b7121ba41cb85baa38094b41f4607a7c88b3384775177cb116e76faf5514760624f98a4e8a830767407c46753a7e0285158c33e0c6ce395de8f15c
fada8b019af104a0df7659ede2618f594bb3e78a test: Add missing assignment in mempool_resurrect.py (MarcoFalke)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: f438d85cd14a91eabfc380d9ee120cc7a7f9103cf0cd1cf565f675f386f82d966901c0ad3f60b8c462642fbf0a3791dbbd774f9b07668d22b58eb575c8d702c1
11a32722f09f1d81f34bd09b26248ba99f2e7f07 test: run mempool_resurrect.py even with wallet disabled (Michael Dietz)
Pull request description:
Another functional test rewritten as proposed in #20078
**Request for help:**
`node.gettransaction(txid)` fails for transactions sent with `wallet.send_self_transfer`. Even though the `txid`s look correct, are added to the mempool correctly, and removed from the mempool when a block is mined - all as expected.
However, `node.gettransaction(txid)` throws the error:
```sh
Traceback (most recent call last):
File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main
self.run_test()
File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in run_test
assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids))
File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in <lambda>
assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids))
File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__
raise JSONRPCException(response['error'], status)
test_framework.authproxy.JSONRPCException: Invalid or non-wallet transaction id (-5)
```
Anyone know what's going wrong / can point me in the right direction if I'm making a mistake, or `MiniWallet` needs to be improved for this to work correctly?
ACKs for top commit:
MarcoFalke:
ACK 11a32722f09f1d81f34bd09b26248ba99f2e7f07
Tree-SHA512: 13d83a13ec23920db716e99b68670e61329d1cc73b12063d85bc1679ee6425a9951da4d2e392ca1f27760be7be049ccdc6f504e192ed5cd24ed0ba003b66fab3
6b56c1f4d0d5857d9d61a81dc96db1b603c368b5 test: remove last_{block,header}_equals() in p2p_fingerprint.py (Sebastian Falbesoner)
136d96b71f94bde2c7471ed852d447ec008e3a30 test: use wait_for_{block,header} helpers in p2p_fingerprint.py (Sebastian Falbesoner)
Pull request description:
This small PR takes use of the message receiving helper functions `wait_for_block()` and `wait_for_header()` (from module `test_framework.p2p`) in the test `p2p_fingerprint.py`. It also simplifies the checks for very old stale blocks/headers requests by getting rid of the functions `last_block_equals()` and `last_header_equals()` and rather only testing that not any blocks/headers message is received at all. Unneeded sending of requests are also removed and calls to time.sleep(...) substituted by ping syncs.
ACKs for top commit:
guggero:
ACK 6b56c1f4
Tree-SHA512: 9114db70f3804adad4ab658236762d4fa73fef91158c5756dd1af2d24196ea740451b0028667e0c4047f1f89fe1355031921d3dfb973acc1370052a4bc12c2ab
4f67336f1105b7c34a9e8cdafa603edc1d899fb9 test: verify best blockhash after invalidating an unknown block (brunoerg)
Pull request description:
Fixes#26051
Verify the best blockhash is the same after invalidating an unknown block, not the whole `getchaintip` response.
ACKs for top commit:
instagibbs:
ACK 4f67336f1105b7c34a9e8cdafa603edc1d899fb9
Tree-SHA512: 2d71743c1d3a317ef7b750f88437df71d1aed2728d9edac8b763a343406e168b97865ab25ec4c89caf09d002e076458376618cbd0845496375f7179633c88af9
4b1d5a10537ab48e3457606ba1cf2ae26a1cb2b2 test: invalidating an unknown block throws an error (brunoerg)
Pull request description:
While playing with `invalidateblock`, I unintentionally tried to invalidate an unknown block and it threw an error. Looking at the tests I just realized there is no test coverage for this case. This PR adds it.
Top commit has no ACKs.
Tree-SHA512: 25286ead809b3ad022e759127ef3134b271fbe76cb7b50ec2b0c7e2409da8d1b01dc5e80afe73e4564cc9c9c03487a1fe772aea3456988552d2f9c8fb34c730b
4edc6893825fd8c45c53c81c73a6a7801e1b458c doc: test: suggest multi-line imports in functional test style guide (Sebastian Falbesoner)
Pull request description:
As long as I remember contributing to functional tests (~2-3 years), it was always kind of an unwritten rule that multi-line imports are preferred over single-line imports in order to reduce the possibility of potential merge conflicts -- at least if more than one symbol from a module is imported. This PR adds this rule to the style guide and adapts the example test accordingly. (Inspired by https://github.com/bitcoin/bitcoin/pull/25792#discussion_r941180819).
ACKs for top commit:
kouloumos:
ACK 4edc6893825fd8c45c53c81c73a6a7801e1b458c
1440000bytes:
ACK 4edc689382
w0xlt:
ACK 4edc689382
fanquake:
ACK 4edc6893825fd8c45c53c81c73a6a7801e1b458c
Tree-SHA512: c7b6ff62f601f4e57cc8334d291662987d6737ebca8d81c063280409f4412302172f1404ec16afc9a13007bcdba55bdab66b9b80363287e287888929cb386584
1821d92b66 test: add activate_ehf_by_name (Alessandro Rezzi)
de38dca242 feat(consensus): Generalize ehf activation (Alessandro Rezzi)
Pull request description:
## Issue being fixed or feature implemented
Try to sign/mine any ehf deployment and not only mn_rr. As asked in the review this decouples (and improves) commit [e24cb23](e24cb239f8) from PR #5799
## What was done?
See commit description
## How Has This Been Tested?
## Breaking Changes
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
Top commit has no ACKs.
Tree-SHA512: 7112a5214b473dea29a892563e6598eca089d2e17fdff8bda08c7777738f1054d2cb07e0a7dccf66214911fec64a6441e84100273ac2ed52abe1d33fb960e8cc
fa6065661a86656a29e89ed1a3529cb7103f5394 refactor: Avoid unsigned integer overflow in core_write (MarcoFalke)
Pull request description:
Also, I find the new code a bit easier to understand.
ACKs for top commit:
shaavan:
Code Review ACK fa6065661a86656a29e89ed1a3529cb7103f5394
Tree-SHA512: cd751e3b4dc97ef525eb8be8d0a49e9629389cb114df18d59a06e05388822af2939078e937f01494e6b317d601743b1a433ba47aa40c4dc602372d1f0fd0dc11
1111d33532516c16fb2e22660ac2745ce56ad6cd refactor: Make MessageBoxFlags enum underlying type unsigned (MarcoFalke)
Pull request description:
All values in the enum are unsigned. Also, flags shouldn't be treated as signed types. So clarify the underlying type and remove a sanitizer suppression.
ACKs for top commit:
hebasto:
ACK 1111d33532516c16fb2e22660ac2745ce56ad6cd, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 48b16c4a0ace1a1e4d351d6eadadbb1bc42aef7fd82e24e3ea50c62f2c04a552ed21027158d09aa97e630c8c7d732cb150c38065333d7c2accbae46593b7ed9f
fa99e108e778b5169b3de2ce557af68f1fe0ac0b Fix implicit-integer-sign-change in arith_uint256 (MarcoFalke)
Pull request description:
This refactor doesn't change behaviour, but clarifies that the numbers being dealt with aren't supposed to be negative. This helps when reading the code and allows to remove a sanitizer suppression for the whole file.
ACKs for top commit:
PastaPastaPasta:
utACK fa99e108e778b5169b3de2ce557af68f1fe0ac0b
shaavan:
ACK fa99e108e778b5169b3de2ce557af68f1fe0ac0b
Tree-SHA512: f227e2fd22021e39f0445ec041f4a299d13477c23cef0fc06c53fb3313cbe550cec329336224a7e8775d9045b8009423052b394e83d42a1e40772085dfcdd471
fa7238300c18938cdf627cacfc58d4b81602417f fuzz: Limit fuzzed time to years 2000-2100 (MarcoFalke)
Pull request description:
It doesn't make sense to fuzz times in the past, as Bitcoin Core will refuse to start in the past.
Fix that and also remove a sanitizer suppression, which would be hit in net_processing in `ProcessMessage`:
```cpp
if (addr.nTime <= 100000000 || addr.nTime > nNow + 10 * 60)
addr.nTime = nNow - 5 * 24 * 60 * 60; // <-- Here
```
This changes the format of fuzz inputs. Previously a time value was (de)serialized as 40 bytes, now it is 32 bytes.
ACKs for top commit:
mzumsande:
Code Review ACK fa7238300c18938cdf627cacfc58d4b81602417f
Tree-SHA512: ca6e7233beec2d9ef9fd481d8f1331942a4d2c8fe518b857629bebcc53a4f42ae123b994cf5d359384a0a8022098ff5a9c146600bc2593c6d88734e25bc240ad
fa7da227daf8558be14f226c4366583fdc59ba10 refactor: Fix implicit-signed-integer-truncation in cuckoocache.h (MarcoFalke)
Pull request description:
Using a file-wide suppression for this implicit truncation has several issues:
* It is file-wide, thus suppressing any other (newly introduced) issues
* The file doesn't compile with `-Wimplicit-int-conversion`
Fix both issues by making the truncation explicit.
ACKs for top commit:
fanquake:
ACK fa7da227daf8558be14f226c4366583fdc59ba10
Tree-SHA512: bf2076ed94c4e80d0d29ff883080edc7a73144c73d6d3e872ec87966177ee3160f4760fc4c774aaa6fb591f4acee450a24b0f7c82291e0bef96582a6d134046e
fae5fec0fec851568a72724000193b2747c30414 test: Remove sanitizer suppression implicit-signed-integer-truncation:netaddress.cpp (MarcoFalke)
Pull request description:
This reverts commit fa865287e5f35e0a376785834e966dd202d2959e.
This was fixed in commit efd6f904c78769ad2e93c1f1de43014d284e7561.
ACKs for top commit:
vasild:
ACK fae5fec0fec851568a72724000193b2747c30414
Tree-SHA512: 3bebf1babd5c68cbb2506bcab9b8e9ffed8697213cf66190484748741f05c59b847a103be171360f7fd6ddb57dfd86ed34a123f72860b76e533ed46bb53a4852
fa865287e5f35e0a376785834e966dd202d2959e test: Add temporary sanitizer suppression implicit-signed-integer-truncation:netaddress.cpp (MarcoFalke)
Pull request description:
This is required to unbreak the fuzzers while a fix is being worked on.
https://cirrus-ci.com/task/4787303177519104?logs=ci#L3020
```
netaddress.cpp:1190:18: runtime error: implicit conversion from type 'int' of value -1 (32-bit, signed) to type 'uint8_t' (aka 'unsigned char') changed the value to 255 (8-bit, unsigned)
ACKs for top commit:
practicalswift:
cr ACK fa865287e5f35e0a376785834e966dd202d2959e
tryphe:
untested ACK fa865287e5f35e0a376785834e966dd202d2959e
lsilva01:
ACK fa865287e5
Tree-SHA512: 4a54ec68c014c7a4c9ab268c3a04321db5eb9b2857646b41406d8d4908a3d349848b4549e80aea6afd9a0c3639522a48fe578527139519b12439eae9f0c4c46c
fa9ebedec3f982bb5bb459ea33d74c94d9b5cec4 Reject invalid coin height and output index when loading assumeutxo (MarcoFalke)
Pull request description:
It should be impossible to have a coin at a height higher than the height of the snapshot block, so reject those early to avoid integer wraparounds and hash collisions later on.
Same for the outpoint index.
Both issues were found by fuzzing:
* The height issue by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34793
* The outpoint issue by my fuzz server: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34793#c2
ACKs for top commit:
practicalswift:
cr ACK fa9ebedec3f982bb5bb459ea33d74c94d9b5cec4: patch looks correct
jamesob:
crACK fa9ebedec3
theStack:
Code review ACK fa9ebedec3f982bb5bb459ea33d74c94d9b5cec4
benthecarman:
crACK fa9ebedec3f982bb5bb459ea33d74c94d9b5cec4
Tree-SHA512: dae7caee4b3862b23ebdf2acb7edec4baf75b0dbf1409b370b1a73aa6b632b317ebfac596dcbaf4edfb1301b513f45465ea75328962460f35e2af0d7e547c9ac
faca40ec68a25180f90a5b9ef017f931354d5bc6 test: Add temporary coinstats suppressions (MarcoFalke)
Pull request description:
Needed for my fuzzer to continue to run
ACKs for top commit:
practicalswift:
cr ACK faca40ec68a25180f90a5b9ef017f931354d5bc6: suppression looks necessary (temporarily)
Tree-SHA512: 5bdff9a24a60546cfe31e775fa2aa5e238aefda2ed2604bef18c82b1b80c51ca3cbe058d6c7988fa75305258b70076036a3e430b9b7de13a111309fa7a66745b
575792e6ffe23c8236a1f8431f6be445e448809b fuzz: Add -fsanitize=integer suppression needed for RPC fuzzer (practicalswift)
Pull request description:
Add `-fsanitize=integer` suppression needed for RPC fuzzer (`generateblock`).
Context: https://github.com/bitcoin-core/qa-assets/pull/59/checks?check_run_id=2494624259
```
miner.cpp:130:21: runtime error: implicit conversion from type 'int64_t' (aka 'long') of value 244763573890 (64-bit, signed) to type 'uint32_t' (aka 'unsigned int') changed the value to 4245405314 (32-bit, unsigned)
#0 0x56143974eaf3 in BlockAssembler::CreateNewBlock(CScript const&) miner.cpp:130:21
#1 0x56143993690d in generateblock()::$_4::operator()(RPCHelpMan const&, JSONRPCRequest const&) const rpc/mining.cpp:370:127
```
ACKs for top commit:
practicalswift:
> review ACK [575792e](575792e6ff), but this function shouldn't be called by the rpc fuzzer, at least not without sanitizing num_blocks
MarcoFalke:
review ACK 575792e6ffe23c8236a1f8431f6be445e448809b
Tree-SHA512: c2133d1064bf17df0e7749ef4a0f7664b5c8082040491a1035d39f0c6e5d96997b347ef2354411e285c7f1f973e34515f1c3c88eb3de60fab64ca4d2adf6dd74
76a84c0a6cac3df711b1ed907a46c905cb85c485 test: speedup wallet_coinbase_category.py (furszy)
Pull request description:
No need to create a chain (200 extra blocks), nor use the cache, for it.
Top commit has no ACKs.
Tree-SHA512: beec64ba6c580d475e19700371ae155f4f3d74325879802da83d02f4153a752d5829b8a4ed77e0c893e79cbc26f66389eed90ac2e8b03a59f6c630ee9333355c
ceec6808d331fa082407a734cd5f3c2f1c7d11b3 test: `-whitebind` and `-bind` with `-listen=0` should throw an error (brunoerg)
Pull request description:
This PR adds test coverage for the following init error:
b9122e95f0/src/init.cpp (L872-L875)
ACKs for top commit:
laanwj:
Code review ACK ceec6808d331fa082407a734cd5f3c2f1c7d11b3
Tree-SHA512: 03068abe7199b1235f029871ab87a3dd4943738c592ad62d82cdcd3e0201e627624960bd3ea1fc6fc1e7da4b8e215ba3393d1cb8130e1108049f764e51dc75c0
5a8c321444c10c7c89c4222c0b47c2d83a1a9ea4 test: check for `getblocktxn` request with out-of-bounds tx index (Sebastian Falbesoner)
Pull request description:
This PR adds missing test coverage for the `getblocktxn` message handler, in the case that any of the contained indices is out-of-bounds:
a05876619a/src/net_processing.cpp (L2180-L2183)
ACKs for top commit:
dunxen:
ACK 5a8c321
Tree-SHA512: 2743c2c6d8aed57b22f825aefd60ba3e670321b60625a42ea7248e7b0fc41c73e9a5945153567c02824ba3b5f0fce7f4125bffc974973fc608b6ffbe49e14b65
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
that's a result of:
contrib/devtools/copyright_header.py update ./
it is not scripted diff, because it works differentlly on my localhost and in CI:
CI doesn't want to use git commit date which is mocked to 30th Dec of 2023
ea54ba2f42f6d0b23570c665c2369f977bf55cf6 [test] Fix port collisions caused by p2p_getaddr_caching.py (dergoegge)
f9682e75ac184a62c7e29287882df34c25303033 [test_framework] Set PortSeed.n directly after initialising params (dergoegge)
Pull request description:
This PR fixes the issue mentioned [here](https://github.com/bitcoin/bitcoin/pull/25096#discussion_r892558783), to avoid port collisions between nodes spun up by the test framework.
Top commit has no ACKs.
Tree-SHA512: ec9159f0af90db636f7889d664c24e1430cf2bcb3c02a9ab2dcfe531b2a4d18f6e3a0f8ba73071bdf2f7db518df9d5d86a9cd06695e67644d20fe4515fac32b7
fa76d8d4d71d844e217686881d4f630eac3a8e10 test: Actually print TSan tracebacks (MarcoFalke)
Pull request description:
Commit 5e5138a721738f47053d915e4c65f925838ad5b4 made the TSan logs to be printed before returning an error from the ci script.
However, it seems that on Cirrus CI, the `--failfast` option will kill not only all python process and bitcoind child process, but also the parent CI bash script, rendering the `trap` inefficient. I believe this bug was introduced in commit 451b96f7d2796d00eabaec56d831f9e9b1a569cc.
ACKs for top commit:
fanquake:
utACK fa76d8d4d71d844e217686881d4f630eac3a8e10
Tree-SHA512: 686f889d38a343882cb62ad6e0c2080196330e7cc7086891a7ff66d9443b455c82ba8d7e4a5cc42daa0513b0ad2743055bfe90e2f6ac88a910ee3b663fabddcd
## Issue being fixed or feature implemented
On my local kubuntu linters have way too much spam
## What was done?
See each commit
## How Has This Been Tested?
Run locally. Amount of warnings decreased from thousands to fewer
amount. Excluding typos, they are:
```
src/coinjoin/client.cpp:1420:5: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm]
src/coinjoin/client.cpp:1426:5: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm]
src/coinjoin/client.cpp:655:26: warning: Consider using std::copy_if algorithm instead of a raw loop. [useStlAlgorithm]
src/coinjoin/server.cpp:593:33: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm]
src/coinjoin/server.cpp:630:106: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm]
src/governance/governance.cpp:1057:9: warning: C-style pointer casting [cstyleCast]
src/governance/governance.cpp:1068:9: warning: C-style pointer casting [cstyleCast]
src/governance/governance.cpp:1079:13: warning: C-style pointer casting [cstyleCast]
src/governance/governance.cpp:1086:9: warning: C-style pointer casting [cstyleCast]
src/governance/governance.cpp:1094:9: warning: C-style pointer casting [cstyleCast]
src/governance/governance.cpp:1099:5: warning: C-style pointer casting [cstyleCast]
src/governance/governance.cpp:1486:34: warning: Consider using std::copy_if algorithm instead of a raw loop. [useStlAlgorithm]
src/llmq/commitment.cpp:102:5: warning: Consider using std::all_of or std::none_of algorithm instead of a raw loop. [useStlAlgorithm]
src/llmq/instantsend.cpp:820:38: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm]
src/llmq/quorums.cpp:831:102: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm]
src/llmq/quorums.h:300:17: warning: C-style pointer casting [cstyleCast]
src/llmq/quorums.h:301:17: warning: C-style pointer casting [cstyleCast]
src/llmq/quorums.h:302:17: warning: C-style pointer casting [cstyleCast]
src/llmq/quorums.h:303:17: warning: C-style pointer casting [cstyleCast]
src/spork.cpp:119:58: warning: Consider using std::any_of algorithm instead of a raw loop. [useStlAlgorithm]
src/statsd_client.cpp:234:63: warning: C-style pointer casting [cstyleCast]
Advice not applicable in this specific case? Add an exception by updating
IGNORED_WARNINGS in test/lint/lint-cppcheck-dash.sh
^---- failure generated from test/lint/lint-cppcheck-dash.sh
Consider install flake8-cached for cached flake8 results.
test/functional/data/invalid_txs.py: error: Source file found twice under different module names: "invalid_txs" and "data.invalid_txs"
test/functional/data/invalid_txs.py: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules for more info
test/functional/data/invalid_txs.py: note: Common resolutions include: a) adding `__init__.py` somewhere, b) using `--explicit-package-bases` or adjusting MYPYPATH
Found 1 error in 1 file (errors prevented further checking)
^---- failure generated from test/lint/lint-python.s
```
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
## Issue being fixed or feature implemented
fix failures like https://gitlab.com/dashpay/dash/-/jobs/6175160403
## What was done?
use `minimumAmount` option in `listunspent` rpc call to avoid picking
coins that are too small for asset lock txes
## How Has This Been Tested?
run `feature_asset_locks.py`
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
faa94cb1675d8bd511eb593176cd07aa59465225 test: Check that invalid peer traffic is accounted for (MarcoFalke)
fae243f0cb92b5648d07d0a5033e2f4de862ae99 test: Remove confusing cast to same type (int to int) (MarcoFalke)
Pull request description:
Couldn't find a test for this and it seems something we should test, so I wrote one.
ACKs for top commit:
vasild:
ACK faa94cb16
practicalswift:
ACK faa94cb1675d8bd511eb593176cd07aa59465225: patch looks correct
Tree-SHA512: efcdd35960045cdfbd14480e16e0d1d09e81eb01670ac541ac2b105e1a63818a157c95853270242234a224880873e79957832bf4231374d7fe81de30f35e3abf
## Issue being fixed or feature implemented
The architecture of bitcoin assumes that there's no any external class
that processes network messages and knows anything about PeerManager
from net_processing; no any external call for PeerManager::Misbehaving
in bitcoin. All logic related to processing messages are located in
net_processing.
Dash has many many extra types of network messages and many of them
processed by external components such as llmq/signing or
coinjoin/client. Current architecture creates multiple circular
dependency.
## What was done?
That's part II of refactorings.
This PR removes PeerManager from several constructor and let LLMQContext
to forget about PeerManager.
Prior work in this PR: https://github.com/dashpay/dash/pull/5782
## What else to do?
Some network messages are processed asynchronously in external
components such as llmq/signing, llmq/instantsend,
llmq/dkgsessionhandler. It doesn't let to refactor them easily, because
they can't just simple return status of processing; status of processing
would be available sometime later and there's need callback or other way
to pass result code without spreading PeerManager over codebase.
## How Has This Been Tested?
- Run unit/functional tests
- run a linter test/lint/lint-circular-dependencies.sh
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
## Issue being fixed or feature implemented
To prevent spending withdrawal before that's finalized, mined and
chainlocked, next things should be true:
- txes with spending of unlock and unlock themselves do not receive
islocks. That's true, because IS excluded for txes with no inputs.
- When the unlock is removed from mempool, so are the children.
These functionality has no tests, but that's crucial for consensus be
fine.
## What was done?
Implemented checks to be sure that IS is not send for Withdrawal txes.
Functional test for asset_locks.py are refactored a bit to make it
easier to read.
## How Has This Been Tested?
This PR is tests
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
9913419cc9db5f8ce7afa0c3774468c330136064 test: remove type: comments in favour of actual annotations (fanquake)
Pull request description:
Now that we require Python 3.6+, we should be using variable type
annotations directly rather than `# type:` comments.
Also takes care of the discarded value issue in p2p_message_capture.py.
See: https://github.com/bitcoin/bitcoin/pull/19509/files#r571674446.
ACKs for top commit:
MarcoFalke:
review ACK 9913419cc9db5f8ce7afa0c3774468c330136064
jnewbery:
Code review ACK 9913419cc9db5f8ce7afa0c3774468c330136064
Tree-SHA512: 63aba5eef6c1320578f66cf8a6d85ac9dbab9d30b0d21e6e966be8216e63606de12321320c2958c67933bf68d10f2e76e9c43928e5989614cea34dde4187aad8
bff7c66e67aa2f18ef70139338643656a54444fe Add documentation to contrib folder (Troy Giorshev)
381f77be858d7417209b6de0b7cd23cb7eb99261 Add Message Capture Test (Troy Giorshev)
e4f378a505922c0f544b4cfbfdb169e884e02be9 Add capture parser (Troy Giorshev)
4d1a582549bc982d55e24585b0ba06f92f21e9da Call CaptureMessage at appropriate locations (Troy Giorshev)
f2a77ff97bec09dd5fcc043d8659d8ec5dfb87c2 Add CaptureMessage (Troy Giorshev)
dbf779d5deb04f55c6e8493ce4e12ed4628638f3 Clean PushMessage and ProcessMessages (Troy Giorshev)
Pull request description:
This PR introduces per-peer message capture into Bitcoin Core. 📓
## Purpose
The purpose and scope of this feature is intentionally limited. It answers a question anyone new to Bitcoin's P2P protocol has had: "Can I see what messages my node is sending and receiving?".
## Functionality
When a new debug-only command line argument `capturemessages` is set, any message that the node receives or sends is captured. The capture occurs in the MessageHandler thread. When receiving a message, it is captured as soon as the MessageHandler thread takes the message off of the vProcessMsg queue. When sending, the message is captured just before the message is pushed onto the vSendMsg queue.
The message capture is as minimal as possible to reduce the performance impact on the node. Messages are captured to a new `message_capture` folder in the datadir. Each node has their own subfolder named with their IP address and port. Inside, received and sent messages are captured into two binary files, msgs_recv.dat and msgs_sent.dat, like so:
```
message_capture/203.0.113.7:56072/msgs_recv.dat
message_capture/203.0.113.7:56072/msgs_sent.dat
```
Because the messages are raw binary dumps, included in this PR is a Python parsing tool to convert the binary files into human-readable JSON. This script has been placed on its own and out of the way in the new `contrib/message-capture` folder. Its usage is simple and easily discovered by the autogenerated `-h` option.
## Future Maintenance
I sympathize greatly with anyone who says "the best code is no code".
The future maintenance of this feature will be minimal. The logic to deserialize the payload of the p2p messages exists in our testing framework. As long as our testing framework works, so will this tool.
Additionally, I hope that the simplicity of this tool will mean that it gets used frequently, so that problems will be discovered and solved when they are small.
## FAQ
"Why not just use Wireshark"
Yes, Wireshark has the ability to filter and decode Bitcoin messages. However, the purpose of the message capture added in this PR is to assist with debugging, primarily for new developers looking to improve their knowledge of the Bitcoin Protocol. This drives the design in a different direction than Wireshark, in two different ways. First, this tool must be convenient and simple to use. Using an external tool, like Wireshark, requires setup and interpretation of the results. To a new user who doesn't necessarily know what to expect, this is unnecessary difficulty. This tool, on the other hand, "just works". Turn on the command line flag, run your node, run the script, read the JSON. Second, because this tool is being used for debugging, we want it to be as close to the true behavior of the node as possible. A lot can happen in the SocketHandler thread that would be missed by Wireshark.
Additionally, if we are to use Wireshark, we are at the mercy of whoever it maintaining the protocol in Wireshark, both as to it being accurate and recent. As can be seen by the **many** previous attempts to include Bitcoin in Wireshark (google "bitcoin dissector") this is easier said than done.
Lastly, I truly believe that this tool will be used significantly more by being included in the codebase. It's just that much more discoverable.
ACKs for top commit:
MarcoFalke:
re-ACK bff7c66e67aa2f18ef70139338643656a54444fe only some minor changes: 👚
jnewbery:
utACK bff7c66e67aa2f18ef70139338643656a54444fe
theStack:
re-ACK bff7c66e67aa2f18ef70139338643656a54444fe
Tree-SHA512: e59e3160422269221f70f98720b47842775781c247c064071d546c24fa7a35a0e5534e8baa4b4591a750d7eb16de6b4ecf54cbee6d193b261f4f104e28c15f47
## Issue being fixed or feature implemented
Should fix failures like
https://gitlab.com/dashpay/dash/-/jobs/6107697452
## What was done?
See inline comments
## How Has This Been Tested?
Run lots of `feature_governance.py` in parallel multiple times
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
---------
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
## Issue being fixed or feature implemented
This pull request is a follow-up to
[some](https://github.com/dashpay/dash/pull/5834#discussion_r1470105685)
[feedback](https://github.com/dashpay/dash/pull/5834#discussion_r1467009815)
received on [dash#5834](https://github.com/dashpay/dash/pull/5834) as
the patterns highlighted were present in different parts of the codebase
and hence not corrected within the PR itself but addressed separately.
This is that separate PR 🙂 (with some additional cleanup of my own)
## What was done?
* This pull request will remain a draft until
[dash#5834](https://github.com/dashpay/dash/pull/5834) as it will
introduce more changes that will need to be corrected in this PR.
* Code introduced that is unique to Dash Core (CoinJoin, InstantSend,
etc.) has been excluded from un-Dashification as the purpose of it is to
reduce backport conflicts, which don't apply in those cases.
* `CWallet::CreateTransaction` and the `CreateTransactionTest` fixture
have been excluded as the former originates from
[dash#3668](https://github.com/dashpay/dash/pull/3668) and the latter
from [dash#3667](https://github.com/dashpay/dash/pull/3667) and are
distinct enough to be unique to Dash Core.
* There are certain Dashifications and SegWit-removals that prove
frustrating as it would break compatibility with programs that rely on
the naming of certain keys
* `getrawmempool`, `getmempoolancestors`, `getmempooldescendants` and
`getmempoolentry` return `vsize` which is currently an alias of `size`.
I have been advised to retain `vsize` in lieu of potential future
developments. (this was originally remedied in
219a1d08973e7ccda6e778218b9a8218b4aae034 but has since been dropped)
* `getaddressmempool`, `getaddressutxos` and `getaddressdeltas` all
return a value with the key `satoshis`. This is frustrating to rename to
`duffs` for compatibility reasons.
* `decodepsbt` returns (if applicable) `non_witness_utxo` which is
frustrating to rename simply to `utxo` for the same reason.
* `analyzepsbt` returns (if applicable) `estimated_vsize` which
frustrating to rename to `estimated_size` for the same reason.
## How Has This Been Tested?
## Breaking Changes
None
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
HD wallets are old-existsing feature, appeared in Dash years ago, but
enabling HD wallets is not trivial task that requires multiple steps and
command line/rpc calls.
Let's have them enabled by default.
## What was done?
- HD wallets are enabled by default. Currently behavior `dashd`,
`dash-qt` are similar to run with option `-usehd=1`
- the rpc `upgradewallet` do not let to upgrade from non-HD wallet to HD
wallet to don't encourage user use non-crypted wallets (postponed till
v21)
- the initialization of ScriptPubKey is updated to be sure that encypted
HD seed is never written on disk (if passphrase is provided)
- enabled and dashified a script `wallet_upgradewallet.py` which test
compatibility between different versions of wallet
## What is not done?
- wallet tool still does not support passhprase, HD seed can appear on
disk
- there's no dialog that show user a mnemonic phrase and encourage him
to make a paper backup
Before removing a command line 'usehd' (backport bitcoin#11250) need to
make at least one major release for fail-over option (if someone wish to
use non-HD wallets only).
## How Has This Been Tested?
Run unit and functional tests.
Enabled new functional test `wallet_upgradewallet.py` that has been
backported long time ago but waited this PR to be enabled.
## Breaking Changes
HD wallets are created by default.
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
---------
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
## Issue being fixed or feature implemented
Fix failures like https://gitlab.com/dashpay/dash/-/jobs/6120923632
## What was done?
Handle disconnects and reconnection of the revoked MN in the right
place.
## How Has This Been Tested?
Run multiple `feature_dip3_v19.py` in parallel a few times
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
56010f92564a94b0ca6c008c0e6f74a19fad4a2a test: hoist p2p values to test framework constants (Jon Atack)
75447f0893f9ad9bf83d182b301d139430d8de1c test: improve msg sends and p2p disconnections in p2p_invalid_messages (Jon Atack)
57960192a5362ff1a7b996995332535f4c2a25c3 test: refactor test_large_inv() into 3 tests with common method (Jon Atack)
e2b21d8a597c536a8617408d43958bfe9f98a442 test: add p2p_invalid_messages logging (Jon Atack)
9fa494dc0969c61d5ef33708a08923cca19ce091 net: update misbehavior logging for oversized messages (Jon Atack)
Pull request description:
...seen while reviewing #19264, #19252, #19304 and #19107:
in `net_processing.cpp`
- make the debug logging for oversized message size misbehavior the same for `addr`, `getdata`, `headers` and `inv` messages
in `p2p_invalid_messages`
- add missing logging
- improve assertions/message sends, move cleanup disconnections outside the assertion scopes
- split a slowish 3-part test into 3 order-independent tests
- add a few p2p constants to the test framework
ACKs for top commit:
troygiorshev:
reACK 56010f92564a94b0ca6c008c0e6f74a19fad4a2a
MarcoFalke:
ACK 56010f9256 🎛
Tree-SHA512: db67b70278f8d4c318907e105af54b54eb3afd15500f9aa0c98034f6fd4bd1cf9ad1663037bd9b237ff4890f3059b37291a6498d8d6ae2cc38efb9f045f73310
## Issue being fixed or feature implemented
kudos to kwvg to report issue: he pointed out that functional tests
randomly fail lately.
Bisect pointed out an exact commit: bitcoin#20027 - mockable time
everywhere
## What was done?
Added call `mn.node.mockscheduler` as it is done in 20027 for other
functional tests
## How Has This Been Tested?
Run 20 times. Without this patch 20% failures; with this patch - zero
failures.
```
test/functional/test_runner.py -j20 feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py feature_llmq_simplepose.py
```
## Breaking Changes
N/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
---------
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
fa2ecadd0d3283a89d27772dc0275e76277ae17e test: Fix intermittent rpc_net issue (MarcoFalke)
Pull request description:
The test fails because getpeerinfo and getnettotals are not synchronised, so a `wait_until` is needed for each RPC (separately).
Fixes https://cirrus-ci.com/task/4663366629195776?command=ci#L5034
ACKs for top commit:
jnewbery:
utACK fa2ecadd0d3283a89d27772dc0275e76277ae17e
Tree-SHA512: 5ea7128801aab8dbe3d9e6737545ff4ee770e4a9c5a2096ba2339a688424f1879ccba6bf8bcb219983acf86eb28af06fc629586613e7fe28aeffadd2c98633e8
778cd0d88d8d6dd22d7f0fb740f3ca3dbb2280a1 [tests] Remove getnettotals/getpeerinfo consistency test (John Newbery)
Pull request description:
We make no guarantees about consistency between RPC calls.
Alternative to 18784
ACKs for top commit:
MarcoFalke:
review ACK 778cd0d88d8d6dd22d7f0fb740f3ca3dbb2280a1
troygiorshev:
ACK 778cd0d88d8d6dd22d7f0fb740f3ca3dbb2280a1 after reading discussion on 18784, code review, ran test
Tree-SHA512: 438333a111cc93a09680cec47f13fbe03557d4803e5d826aec6f72e5afea62a088622645f0756e8fd2c9182c2a69ccca867d4d6fed2250364bee2b6c834adb1a
fa299ac27364bd7a59e6fb7e0c4ce476f2deec40 test: Speed up wallet_resendwallettransactions test with mockscheduler RPC (MarcoFalke)
Pull request description:
Also fixes#20143
ACKs for top commit:
guggero:
ACK fa299ac2
Tree-SHA512: 024ced4aa5f5c266e24fd0583d47b45b19c2a6ae25a06fabeacaa0ac996eec0c45f11cc34b2df17d01759b78ed31a991aa86978aafcc76cb0017382f601bf85a
c82336c493b112160d781974d4066fcb956b85f6 Remove references to CreateWalletFromFile (fanquake)
Pull request description:
`CWallet::CreateWalletFromFile()` was removed in 8b5e7297c02f3100a9cb27bfe206e3fc617ec173 but these references remain.
ACKs for top commit:
hebasto:
ACK c82336c493b112160d781974d4066fcb956b85f6
Tree-SHA512: 3dd50fe0cd5a60bbc96d265107d4739f3e08f943435f3772038963ac4be9e4a87a863412ac0d571226ea66d71550b17b52f01b9d46a6282d49feae1508fd682e
8f0b64fb513e8c6cdd1f115856100a4ef5afe23e Better error messages for invalid addresses (Bezdrighin)
Pull request description:
This PR addresses #20809.
We add more detailed error messages in case an invalid address is provided inside the 'validateaddress' and 'getaddressinfo' RPC calls. This also covers the case when a user provides an address from a wrong network.
We also add a functional test to test the new error messages.
ACKs for top commit:
kristapsk:
ACK 8f0b64fb513e8c6cdd1f115856100a4ef5afe23e
meshcollider:
Code review ACK 8f0b64fb513e8c6cdd1f115856100a4ef5afe23e
Tree-SHA512: ca0f806ab573e96b79e98d9f8c810b81fa99c638d9b5e4d99dc18c8bd2568e6a802ec305fdfb2983574a97a19a46fd53b77645f8078fb77e9deb24ad2a22cf93
… best height
## Issue being fixed or feature implemented
Platform wants to know the height of the bestchainlock when they call
submitchainlock; sooo we change the API of submitchainlock to also
return the height
## What was done?
Adjust API and tests
## How Has This Been Tested?
New tests added for this behavior
## Breaking Changes
Not really any; I **guess** that return value could be considered
breaking change; but going from nothing -> something feels unlikely to
break anything although it in theory could.
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that
apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
---------
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0 [util] add RunCommandParseJSON (Sjors Provoost)
c17f54ee535faaedf9033717403e1f775b5f1530 [ci] use boost::process (Sjors Provoost)
32128ba682033560d6eb2e4848a9f77a842016d2 [doc] include Doxygen comments for HAVE_BOOST_PROCESS (Sjors Provoost)
3c84d85f7d218fa27e9343c5cd1a55e519218980 [build] msvc: add boost::process (Sjors Provoost)
c47e4bbf0b44f2de1278f9538124ec98ee0815bb [build] make boost-process opt-in (Sjors Provoost)
929cda5470f98d1ef85c05b1cad4e2fb9227e3b0 configure: add ax_boost_process (Sjors Provoost)
8314c23d7b39fc36dde8b40b03b6efbe96f85698 [depends] boost: patch unused variable in boost_process (Sjors Provoost)
Pull request description:
Prerequisite for external signer support in #16546. Big picture overview in [this gist](https://gist.github.com/Sjors/29d06728c685e6182828c1ce9b74483d).
This adds a new dependency [boost process](https://github.com/boostorg/process/tree/boost-1.64.0). This is part of Boost since 1.64 which is part of `depends`. Because the minimum Boost version is 1.47, this functionality is skipped for older versions of Boost.
Use `./configure --with-boost-process` to opt in, which checks for the presence of Boost::Process.
We add `UniValue runCommandParseJSON(const std::string& strCommand)` to `system.{h,cpp}` which calls an arbitrary command and processes the JSON returned by it. This is currently only called by the test suite.
~For testing purposes this adds a new regtest-only RPC method `runcommand`, as well as `test/mocks/command.py` used by functional tests.~ (this is no longer the case)
TODO:
- [ ] review boost process in #15440
ACKs for top commit:
achow101:
ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0
hebasto:
re-ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, only rebased (verified with `git range-diff`) and removed an unintentional tab character since the [previous](https://github.com/bitcoin/bitcoin/pull/15382#pullrequestreview-458371035) review.
meshcollider:
Very light utACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, although I am not very confident with build stuff.
promag:
Code review ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0, don't mind the nit.
ryanofsky:
Code review ACK 31cf68a3ad1f0a5537c8419e2912b55fbfb88fa0. I left some comments below that could be ignored or followed up later. The current change is clean and comprehensive.
Tree-SHA512: c506e747014b263606e1f538ed4624a8ad7bcf4e025cb700c12cc5739964e254dc04a2bbb848996b170e2ccec3fbfa4fe9e2b3976b191222cfb82fc3e6ab182d
a76dafa51dd16e3f1ed665f6f7b6b2b6a708b9b4 ci: Add tsan suppression for race in BerkeleyBatch (Hennadii Stepanov)
Pull request description:
A temporary workaround for #19448.
Top commit has no ACKs.
Tree-SHA512: 47b83ff373e710bc9ba8c3661f9850a14417436028c42eb7765d21337ef25faaac4cf8cf93be844ae592d40264934d7d2f6b7ba0ab6c7209fc0da8fc13067769
## Issue being fixed or feature implemented
Dashmate wanted a way to know if it is safe to restart the masternode.
This new RPC indicates the number of active DKG sessions, and the number
of blocks until next potential DKG.
## What was done?
Examples of responses:
`{'active_dkgs': 0, 'next_dkg': 22}`
## How Has This Been Tested?
`feature_llmq_rotation.py` was updated
## Breaking Changes
no
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
---------
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
## Issue being fixed or feature implemented
RPC `getassetunlockstatuses` is now accepting an extra optional
parameter `height`.
When a valid `height` is passed, then the RPC returns the status of
AssetUnlock indexes up to this specific block. (Requested by Platform
team)
## What was done?
Note that in order to avoid cases that can lead to deterministic result,
when `height` is passed, then the only `chainlocked` and `unknown`
outcomes are possible.
## How Has This Been Tested?
`feature_asset_locks.py` was updated.
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
---------
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
82dee87933ed0714976ff4eb9657acfc13c6de84 test: test decodepsbt fee calculation (count input value only once per UTXO) (Sebastian Falbesoner)
Pull request description:
Fixes#19523, adding a simple test to `rpc_psbt.py` that checks that the decodepsbt fee matches the one given by the wallet (`walletcreatefundedpsbt`). This is in particular important for PSBTs with segwit inputs that have both a witness- and a non-witness-UTXO type set.
Example test run after reverting commit 75122780e2c46505d977e24c5612dfa9442ab754 ("Increment input value sum only once per UTXO in decodepsbt"):
```
$ test/functional/rpc_psbt.py
2020-07-26T11:31:44.862000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test__sutcd4y
20.00007580
2020-07-26T11:31:47.073000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/honeybadger/buidl/bitcoin_thestack/test/functional/test_framework/test_framework.py", line 118, in main
self.run_test()
File "test/functional/rpc_psbt.py", line 166, in run_test
assert_equal(decoded['fee'], created_psbt['fee'])
File "/home/honeybadger/buidl/bitcoin_thestack/test/functional/test_framework/util.py", line 49, in assert_equal
raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(20.00007580 == 0.00007580)
2020-07-26T11:31:47.125000Z TestFramework (INFO): Stopping nodes
......
```
ACKs for top commit:
achow101:
ACK 82dee87933ed0714976ff4eb9657acfc13c6de84
Tree-SHA512: 296b8a701f851d482ef6200c6cbf0cf0257a79a828ac6dbc39b05d8c2d839c6fdb9d3f5a084015295cfa3eac7c11faa2f2d52e619c11627b04c75150eead8330
BACKPORT NOTICE
fixup psbt. all missing changes belongs to src/wallet/scriptpubkeyman.h/cpp ----- they are related to descriptor wallet!
-------------------
931dd4760855e036c176a23ec2de367c460e4243 Make lint-spelling.py happy (Glenn Willen)
11a0ffb29d1b4dcc55c8826873f340ab4196af21 [gui] Load PSBT from clipboard (Glenn Willen)
a6cb0b0c29d327d01aebb98b0504f317eb19c3dc [gui] PSBT Operations Dialog (sign & broadcast) (Glenn Willen)
5dd0c03ffa3aeaa69d8a3a716f902f450d5eaaec FillPSBT: report number of inputs signed (or would sign) (Glenn Willen)
9e7b23b73387600d175aff8bd5e6624dd51f86e7 Improve TransactionErrorString messages. (Glenn Willen)
Pull request description:
Add a "PSBT Operations" dialog, reached from the "Load PSBT..." menu item, giving options to sign or broadcast the loaded PSBT as appropriate, as well as copying the result to the clipboard or saving it to a file.
This is based on Sjors' #17509, and depends on that PR going in first. (It effectively replaces the small "load PSBT" dialog from that PR with a more feature-rich one.)
Some notes:
* The way I display status information is maybe unusual (a status bar, rather than messageboxes.) I think it's helpful to have the information in it be persistent rather than transitory. But if people dislike it, I would probably move the "current state of the transaction" info to the top line of the main label, and the "what action just happened, and did it succeed" info into a messagebox.
* I don't really know much about the translation/localization stuff. I put tr() in all the places it seemed like it ought to go. I did not attempt to translate the result of TransactionErrorString (which is shared by GUI and non-GUI code); I don't know if that's correct, but it matches the "error messages in logs should be googleable in English" heuristic. I don't know whether there are things I should be doing to reduce translator effort (like minimizing the total number of distinct message strings I use, or something.)
* I don't really know how (if?) automated testing is applied to GUI code. I can make a list of PSBTs exercising all the codepaths for manual testing, if that's the right approach. Input appreciated.
ACKs for top commit:
instagibbs:
tested ACK 931dd47608
Sjors:
re-tACK 931dd4760855e036c176a23ec2de367c460e4243
jb55:
ACK 931dd4760855e036c176a23ec2de367c460e4243
achow101:
ACK 931dd4760855e036c176a23ec2de367c460e4243
Tree-SHA512: ade52471a2242f839a8bd6a1fd231443cc4b43bb9c1de3fb5ace7c5eb59eca99b1f2e9f17dfdb4b08d84d91f5fd65677db1433dd03eef51c7774963ef4e2e74f
5da96210fc2fda9fbd79531f42f91262fd7a9257 doc: release note for getpeerinfo last_block/last_transaction (Jon Atack)
cfef5a2c98b9563392a4a258fedb8bdc869c9749 test: rpc_net.py logging and test naming improvements (Jon Atack)
21c57bacda766a4f56ee75a2872f5d0f94e3901e test: getpeerinfo last_block and last_transaction tests (Jon Atack)
8a560a7d57cbd9f473d6a3782893a0e2243c55bd rpc: expose nLastBlockTime/TXTime as getpeerinfo last_block/transaction (Jon Atack)
02fbe3ae0bd91cbab2828cb7aa46f6493c82f026 net: add nLastBlockTime/TXTime to CNodeStats, CNode::copyStats (Jon Atack)
Pull request description:
This PR adds inbound peer eviction criteria `nLastBlockTime` and `nLastTXTime` to `CNodeStats` and `CNode::copyStats`, which then allows exposing them in the next commit as `last_transaction` and `last_block` Unix Epoch Time fields in RPC `getpeerinfo`.
This may be useful for writing missing eviction tests. I'd also like to add `lasttx` and `lastblk` columns to the `-netinfo` dashboard as described in https://github.com/bitcoin/bitcoin/pull/19643#issuecomment-671093420.
Relevant discussion at the p2p irc meeting http://www.erisian.com.au/bitcoin-core-dev/log-2020-08-11.html#l-549:
```text
<jonatack> i was specifically trying to observe and figure out how to test https://github.com/bitcoin/bitcoin/issues/19500
<jonatack> which made me realise that i didn't know what was going on with my peer conns in enough detail
<jonatack> i'm running bitcoin locally with nLastBlockTime and nLastTXTime added to getpeerinfo for my peer connections dashboard
<jonatack> sipa: is there a good reason why that (eviction criteria) data is not exposed through getpeerinfo currently?
<sipa> jonatack: nope; i suspect just nobody ever added it
<jonatack> sipa: thanks. will propose.
```
The last commit is optional, but I think it would be good to have logging in `rpc_net.py`.
ACKs for top commit:
jnewbery:
Code review ACK 5da96210fc2fda9fbd79531f42f91262fd7a9257
theStack:
Code Review ACK 5da96210fc2fda9fbd79531f42f91262fd7a9257
darosior:
ACK 5da96210fc2fda9fbd79531f42f91262fd7a9257
Tree-SHA512: 2db164bc979c014837a676e890869a128beb7cf40114853239e7280f57e768bcb43bff6c1ea76a61556212135281863b5290b50ff9d24fce16c5b89b55d4cd70
b6834e312a6a7bb395ec7266bc9469384639df96 Avoid 'timing mishap' warnings when mocking (Pieter Wuille)
ec3916f40a3fc644ecbbaaddef6258937c7fcfbc Use mockable time everywhere in net_processing (Pieter Wuille)
Pull request description:
The fact that net_processing uses a mix of mockable tand non-mockable time functions made it hard to write functional tests for #19988.
I'm opening this as a separate PR as I believe it's independently useful. In some ways this doesn't go quite as far as it could, as there are now several data structures that could be converted to `std::chrono` types as well now. I haven't done that here, but I'm happy to reconsider that.
ACKs for top commit:
MarcoFalke:
ACK b6834e312a 🌶
jnewbery:
utACK b6834e312a6a7bb395ec7266bc9469384639df96
naumenkogs:
utACK b6834e3
Tree-SHA512: 6528a167c57926ca12894e0c476826411baf5de2f7b01c2125b97e5f710e620f427bbb13f72bdfc3de59072e56a9c1447bce832f41c725e00e81fea019518f0e
0fcaf731997c4989b869e42d8990f742637799c2 test: use explicit p2p objects where available (Oliver Gugger)
Pull request description:
This is a follow-up patch to #19804 as suggested by MarcoFalke (https://github.com/bitcoin/bitcoin/pull/19804#discussion_r494950062).
To make the intent of the tests easier to understand, we reference the
p2p connection objects by their explicit names instead of the p2ps array.
ACKs for top commit:
theStack:
ACK 0fcaf731997c4989b869e42d8990f742637799c2
Tree-SHA512: 37db22185077beeadfa7245bb768b87d6b7a2cfb906c3c859ab92ec3d657122db7301777f0838e13dfc748f37303850144fb7553e6cb6c66903e304d6e10e659
10d61505fe77880d6989115defa5e08417f3de2d [test] remove confusing p2p property (gzhao408)
549d30faf04612d9589c81edf9770c99e3221885 scripted-diff: replace p2p with p2ps[0] in p2p_invalid_tx (gzhao408)
7a0de46aeafb351cffa3410e1aae9809fd4698ad [doc] sample code for test framework p2p objects (gzhao408)
784f757994c1306bb6584b14c0c78617d6248432 [refactor] clarify tests by referencing p2p objects directly (gzhao408)
Pull request description:
The `TestNode` has a `p2p` property which is an alias for `p2ps[0]`.
I think this should be removed because it can be confusing and misleading (to both the test writer and reviewer), especially if a TestNode has multiple p2ps connected (which is the case for many tests).
Another example is when a test has multiple subtests that connect 1 p2p and use the `p2p` property to reference it. If the subtests don't completely clean up after themselves, the subtests may affect one another.
The best way to refer to a connected p2p is use the object returned by `add_p2p_connection` like this:
```py
p2p_conn = node.add_p2p_connection(P2PInterface())
```
A good example is [p2p_invalid_locator.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/p2p_invalid_locator.py), which cleans up after itself (waits in both `wait_for_disconnect` and in `disconnect_p2ps`) but wouldn't need so much complexity if it just referenced the connections directly.
If there is only one connected, it's not really that tedious to just use `node.p2ps[0]` instead of `node.p2p` (and it can always be aliased inside the test itself).
ACKs for top commit:
robot-dreams:
utACK 10d61505fe77880d6989115defa5e08417f3de2d
jnewbery:
utACK 10d61505fe77880d6989115defa5e08417f3de2d
guggero:
Concept ACK 10d61505.
Tree-SHA512: 5965548929794ec660dae03467640cb2156d7d826cefd26d3a126472cbc2494b855c1d26bbb7b412281fbdc92b9798b9765a85c27bc1a97f7798f27f64db6f13
BACKPORT NOTICE
There's some difference with original PR but that's not important because we do not actually use travis.
The variable TRAVIS_BRANCH would be removed anyway in bitcoin#20697 - let's just skip it for simplicity
--------------
a91ab86fae91d416d664d19d2f482a8d19c115a6 lint: Use TRAVIS_BRANCH in lint-git-commit-check.sh (Fabian Jahr)
c11dc995c98e908dfd9cea64d4b34329b1dbb5c6 lint: Don't use TRAVIS_COMMIT_RANGE in whitespace linter (Fabian Jahr)
1b41ce8f5f3debae03ca60e4acada14680df9185 lint: Don't use TRAVIS_COMMIT_RANGE for commit-script-check (Fabian Jahr)
Pull request description:
This is causing problems again, very similar to #19654.
UPDATE: This now removes all remaining usages of TRAVIS_COMMIT_RANGE and instead uses TRAVIS_BRANCH for the range, including `lint-git-commit-check` where TRAVIS_COMMIT_RANGE had already been removed. For builds triggered by a pull request, TRAVIS_BRANCH is the name of the branch targeted by the pull request. In the linters there is still a fallback that assumes master as the target branch.
ACKs for top commit:
sipa:
ACK a91ab86fae91d416d664d19d2f482a8d19c115a6. See test I tried in #20075.
Tree-SHA512: 1378bdebd5d8787a83fbda5d9999cc9447209423e7f0218fe5eb240e6a32dc1b51d1cd53b4f8cd1f71574d935ac5e22e203dfe09cce17e9976a48416038e1263
fa12d8d3edfbed8d5ce746e75af94eb92372f6b7 ci: Add tsan suppression for race in wallet (MarcoFalke)
Pull request description:
Workaround to fix#19417 (Intermittent CI failure)
Top commit has no ACKs.
Tree-SHA512: 2d68783d6db1bf425ce830cb23eab2f7fa3b9ee18cfb08665e4187196af571547206646dc6dfac0b4444e3dc6c4c13ae45efb09607d2d50df20a3d0a4eec98bd
fa7e002d520d8390f3ff4b0383cfdfc14713355d ci: tsan with wallet (MarcoFalke)
Pull request description:
ACKs for top commit:
practicalswift:
ACK fa7e002d520d8390f3ff4b0383cfdfc14713355d -- patch looks correct and Travis is happy
hebasto:
ACK fa7e002d520d8390f3ff4b0383cfdfc14713355d, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 1138459bbef72f402f32dae1e28d96f174901d4248d959b538b973747c8b06f221ecd81a386a1f915c0a2faaefb9fb8f11e3be39e6c5e2468bf9ae43d9f97757
BACKPORT NOTICE:
this PR doesn't actually swithc to libc++ due to multiple CI failures such as
linking errors or other
------------------
faf62e6ed0ca45db44c370844c3515eb5a8cda12 ci: Remove unused workaround (MarcoFalke)
fa7c8509153bfd2d5b4dcff86ad27dfd73e8788b ci: Install llvm to get llvm symbolizer (MarcoFalke)
fa563cef61e8a217c5e8ec059e174afae61087a5 test: Add more tsan suppressions (MarcoFalke)
fa0cc02c0a029133f080680ae9186002a144738f ci: Mute depends logs completely (MarcoFalke)
fa906bf2988c799765a04c484269f890964ec3ee test: Extend tsan suppressions for clang stdlib (MarcoFalke)
fa10d850790bbe52d948659bb1ebbb88fe718065 ci: Use libc++ instead of libstdc++ for tsan (MarcoFalke)
fa0d5ee1126a8cff9f30f863eb8f5c78bf57e168 ci: Set halt_on_error=1 for tsan (MarcoFalke)
fa2ffe87f794caa74f80c1c2d6e6067ee4849632 ci: Deduplicate DOCKER_EXEC (MarcoFalke)
fac2eeeb9d718bdb892eef9adf333ea61ba8f3d0 cirrus: Remove no longer needed install step (MarcoFalke)
Pull request description:
According to the [ThreadSanitizer docs](https://clang.llvm.org/docs/ThreadSanitizer.html#current-status):
> C++11 threading is supported with **llvm libc++**.
For example, the thread sanitizer build is currently not checking for double lock of mutexes.
Fixes (partially) https://github.com/bitcoin/bitcoin/issues/19038#issuecomment-632138003
ACKs for top commit:
practicalswift:
ACK faf62e6ed0ca45db44c370844c3515eb5a8cda12
fanquake:
ACK faf62e6ed0ca45db44c370844c3515eb5a8cda12
hebasto:
ACK faf62e6ed0ca45db44c370844c3515eb5a8cda12, maybe re-organize commits to modify suppressions in a single one?
Tree-SHA512: 98ce5154b4736dfb811ffdb6e6f63a7bc25fe50d3b73134404a8f3715ad53626c31f9c8132dbacf85de47b9409f1e17a4399e35f78b1da30b1577167ea2982ad
39d526bde48d98af4fa27906e85db0399b6aa8b1 test: Bump linter versions (Duncan Dean)
Pull request description:
As per #19346, `mypy==0.700` was incompatible with Python 3.8.
I've bumped the versions of all the linters to their latest stable versions.
Checked with both Python 3.7 and 3.8 and everything still seems to work fine.
ACKs for top commit:
hebasto:
ACK 39d526bde48d98af4fa27906e85db0399b6aa8b1, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: f3ee7fda8095aa25aa68685e863076d52a6b82649770d24b0064d652763c0ceb8ebcbf9024fc74fca45c754e67b2a831dd070b3af23bc099140e6d27e89a5319
21f24336019d438e225c7bd6653871119883a4ee test: run mempool_spend_coinbase.py even with wallet disabled (Michael Dietz)
Pull request description:
Run the mempool spend coinbase test even when the wallet was not compiled, as proposed in #20078.
ACKs for top commit:
laanwj:
ACK 21f24336019d438e225c7bd6653871119883a4ee
Tree-SHA512: 301582c04376371cfa8f1ebb2418a4341b42ddcd9ad4f48b58bcf888d867a97bdc409972856b67a8339ac5e60124aefee82a049b4f7fc6bca7a18d7e92e090be
3b064fcb9dd3df6c438a440f0fea86e9cf7b5f57 test: run mempool_expiry.py even with wallet disabled (Michael Dietz)
Pull request description:
Run the mempool expiry test even when the wallet was not compiled, as proposed in https://github.com/bitcoin/bitcoin/issues/20078.
ACKs for top commit:
MarcoFalke:
ACK 3b064fcb9dd3df6c438a440f0fea86e9cf7b5f57
Tree-SHA512: 5860dc021d02bc3752268ec1e859505bec87174953223b34b1af8a8e4ab66d645458fbf9571c0b816a9de891c3ff41314996e580869671fccd6972c093e78154
5b77f8098de537898151ab116d0e547fd6ff9466 test: add p2p_lock acquires in p2p_leak_tx.py (Sebastian Falbesoner)
cc8c6823b4a8b74922f78ce6ce527ced9325bd49 test: use MiniWallet for p2p_leak_tx.py (Sebastian Falbesoner)
Pull request description:
This PR enables one more of the non-wallet functional tests (p2p_leak_tx.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078. It also adds missing p2p_lock acquires that need to be held while modifying internal p2p Interface state (in this case the `last_message` dictionary) to avoid data races.
ACKs for top commit:
laanwj:
Code review ACK 5b77f8098de537898151ab116d0e547fd6ff9466
Tree-SHA512: 6661bc6e3491a9af4bf040f379e5955c525136397e99d3eadde92e247580d0d87efff750e6d3b1f6d9a4e578144a433a982f574ef056b44dd6bca33873a1bae6
e15344889aac50aadee9211ac34e466867d5862b Clarify blocksonly whitelistforcerelay test (t-bast)
Pull request description:
As discussed in https://github.com/bitcoin/bitcoin/issues/19943, this test may be a bit misleading to newcomers.
We underscore the fact that our peer needs to run a modified version of Bitcoin Core to actually relay transactions to a `blocksonly` node and benefit from the `whitelistforcerelay` parameter.
ACKs for top commit:
naumenkogs:
ACK e15344889aac50aadee9211ac34e466867d5862b
Tree-SHA512: cc3526ac26c40a2d878b0ad863008663040683fd21092fcdc93866c2b0a79db8c2d29767d1f70bf56195092fca2aa2961cdbee52b5f0b1ae757cece9cd206301
b128b566725a5037fdaea99940d1b9de5553d198 test: add logging for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner)
8ee3536b2b77aeb3a48df5b34effbc7345ef34d8 test: remove unused helpers random_transaction(), make_change() and gather_inputs() (Sebastian Falbesoner)
fddce7e199308d96e366d700dca982ef088ba98b test: use MiniWallet for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner)
Pull request description:
This PR enables one more of the non-wallet functional tests (mining_getblocktemplate_longpoll.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078. Also adds missing log messages for the subtests.
This was the only functional test that used the `random_transaction` helper in `test_framework/util.py`, hence it is removed, together with other helpers (`make_change` and `gather_inputs`) that were again only used by `random_transaction`.
ACKs for top commit:
MarcoFalke:
ACK b128b566725a5037fdaea99940d1b9de5553d198
Tree-SHA512: 09a5fa7b0f5976a47040f7027236d7ec0426d5a4829a082221c4b5fae294470230e89ae3df0bca0eea26833162c03980517f5cc88761ad251c3df4c4a49bca46
faabc26a61873b2cd0390a21df571fe53c893c11 test: Replace getmempoolentry with testmempoolaccept in MiniWallet (MarcoFalke)
Pull request description:
This is a refactor to not use the return value of `sendrawtransaction` and `getmempoolentry` with the goal that submitting the tx to the mempool will become optional.
ACKs for top commit:
mjdietzx:
ACK faabc26a61873b2cd0390a21df571fe53c893c11
Tree-SHA512: 4260a59e65fed1c807530dad23f1996ba6e881bcce91995f5b498a0be6001f67b3e0251507c8801750fe105410147c68bb2f393ebe678c6a3a4d045e5d72fc19
a7599c80ebb9579df45e2d6ccf3168302cf42f03 test: run mempool_compatibility.py even with wallet disabled (Michael Dietz)
Pull request description:
Another functional test rewritten as proposed in https://github.com/bitcoin/bitcoin/issues/20078
ACKs for top commit:
MarcoFalke:
review ACK a7599c80ebb9579df45e2d6ccf3168302cf42f03 didn't test
Tree-SHA512: cc7a617e5489ed27bbdbdee85a82fa08525375061f7f4524577a6b8ecb340396adac88419b51f513be22ca53edd0a3bd5d572d9f43ffc2c18550b0ef9069d238
faf251d854e3a670533ea3e9087e82c92f3ae533 test: gettxoutproof duplicate txid (João Barbosa)
faf5eb45c4a08fbfd9a8c12534bca8adfe756ef2 test: Test empty array in gettxoutproof (MarcoFalke)
fa56e866e8ac08b35e775a4e37a4e5849e093c7d test: Run rpc_txoutproof.py even with wallet disabled (MarcoFalke)
faba790bd40b5a9e8849997785020790ff60571b test: MiniWallet: Default fee_rate in send_self_transfer, Pass in utxo_to_spend (MarcoFalke)
fa65a11d0c9a34ff7f4cc4efd53367794e751749 test: bugfix: Actually pick largest utxo (MarcoFalke)
Pull request description:
Run the consensus test even when the wallet was not compiled. Also:
* Minor bugfix in MiniWallet
* Two new test cases (one cherry-picked from #19847)
ACKs for top commit:
jnewbery:
utACK faf251d854e3a670533ea3e9087e82c92f3ae533. Thanks Marco!
kristapsk:
ACK faf251d854e3a670533ea3e9087e82c92f3ae533
Tree-SHA512: a5ab33695c88cfb3c369021d4506069c08ce298e24e891db55159130693ed3817444c72f6aad3f472235aa4597b2c601010af714411c2ec8ad9c2d2e0b00ecbc
fa188c9c59b8c3e43c31be01797f073e27a7bc10 test: Use MiniWalet in p2p_feefilter (MarcoFalke)
fa39c62eb7f39e7d249b8d46c075c4e7a9aec684 test: inline hashToHex (MarcoFalke)
Pull request description:
This introduces a minimalistic test wallet, which can be used as a drop in replacement for the Bitcoin Core wallet to create dummy transactions with a given fee rate.
ACKs for top commit:
jnewbery:
utACK fa188c9c59b8c3e43c31be01797f073e27a7bc10
Tree-SHA512: 0aad9cb14eea4f0055bd6a47cc8c8f82a16941b152598c3bf1e083aae84cca4ffa23f0b854a362a68be1b917deba1b5ec7c0207b63b0805d747ba9a7d1d82efe
84d295e51341a126a6c3cbeea7a8caa04c7b5bc3 tests: Check that segwit inputs in psbt have both UTXO types (Andrew Chow)
46004790588c24174a0bec49b540d158ce163ffd psbt: always put a non_witness_utxo and don't remove it (Andrew Chow)
5279d8bc07d601fe6a67ad665fbc7591fe73c7de psbt: Allow both non_witness_utxo and witness_utxo (Andrew Chow)
72f6bec1da198764d4648a10a61c485e7ab65e9e rpc: show both UTXOs in decodepsbt (Andrew Chow)
Pull request description:
Due to recent changes to hardware wallets, the full previous transaction will need to be provided for segwit inputs. Since some software may be checking for the existence of a `witness_utxo` to determine whether to produce a segwit signature, we keep that field to ease the transition.
Because all of the sanity checks implemented by the `IsSane` functions were related to having mixed segwit and non-segwit data in a PSBT, those functions are removed as those checks are no longer proper.
Some tests are updated/removed to accommodate this and a simple test added to check that both UTXOs are being added to segwit inputs.
As discussed in the wallet IRC meeting, our own signer will not require `non_witness_utxo` for segwit inputs.
ACKs for top commit:
Sjors:
utACK 84d295e51341a126a6c3cbeea7a8caa04c7b5bc3 (didn't retest compared to 836d6fc, but fortunately HWI's CI tracks our master branch, with a bunch of hardware wallet simulators)
ryanofsky:
Code review re-ACK 84d295e51341a126a6c3cbeea7a8caa04c7b5bc3. No changes since last review, but now I understand the context better. I think it would good to improve the comments as suggested https://github.com/bitcoin/bitcoin/pull/19215#discussion_r447889473 and maybe refer to
meshcollider:
utACK 84d295e51341a126a6c3cbeea7a8caa04c7b5bc3
Tree-SHA512: ccc1fd3c16ac3859f5aca4fa489bd40f68be0b81bbdc4dd51188bbf28827a8642dc8b605a37318e5f16cf40f1c4910052dace2f27eca21bb58435f02a443e940
7984c39be11ca04460883365e1ae2a496aaa6c0e test framework: serialize/deserialize inv type as unsigned int (Jon Atack)
407175e0c2bc797599ebd9c0a1f2ec89ad7af136 p2p: change CInv::type from int to uint32_t (Jon Atack)
Pull request description:
Fixes UBSan implicit-integer-sign-change issue per https://github.com/bitcoin/bitcoin/pull/19610#issuecomment-680686460.
Credit to Crypt-iQ for finding and reporting the issue and to vasild for the original review suggestion in https://github.com/bitcoin/bitcoin/pull/19590#pullrequestreview-455788826.
Closes#19678.
ACKs for top commit:
laanwj:
ACK 7984c39be11ca04460883365e1ae2a496aaa6c0e
MarcoFalke:
ACK 7984c39be11ca04460883365e1ae2a496aaa6c0e 🌻
vasild:
ACK 7984c39be
Tree-SHA512: 59f3a75f40ce066ca6f0bb1927197254238302b4073af1574bdbfe6ed580876437be804be4e47d51467d604f0d9e3a5875159f7f2edbb2351fdb2bb9465100b5
6d1f51343cf11b07cd401fbd0c5bc3603e185a0e [rpc] fundrawtransaction, walletcreatefundedpsbt lock manually selected coins (Sjors Provoost)
Pull request description:
When using `fundrawtransaction` and `walletcreatefundedpsbt` with `lockUnspents`, it would only lock automatically selected coins, not manually selected coins. That doesn't make much sense to me if the goal is to prevent accidentally double-spending yourself before you broadcast a transaction.
Note that when creating a transaction, manually selected coins are automatic "unlocked" (or more accurately: the lock is ignored). Earlier versions of this PR introduced an error when a locked coin is manually selected, but this idea was abandoned after some discussion. An application that uses this RPC should either rely on automatic coin selection (with `lockUnspents`) or handle lock concurrency itself with manual coin selection. In particular it needs to make sure to avoid/pause calls with automatic coin selection between calling `lockunspent` and the subsequent spending RPC.
See #7518 for historical background.
ACKs for top commit:
meshcollider:
Code review ACK 6d1f51343cf11b07cd401fbd0c5bc3603e185a0e
fjahr:
Code review ACK 6d1f51343cf11b07cd401fbd0c5bc3603e185a0e
Tree-SHA512: 8773c788d92f2656952e1beac147ba9956b8c5132d474e0880e4c89ff53642928b4cbfcd1cb3d17798b9284f02618a8830c93a9f7a4733e5bded96adff1d5d4d
fa330ec2fe5f5ba68a8d43fff0b19584c0b1ff39 test: Remove confusing and broken use of wait_until global (MarcoFalke)
fa6583c30bf7d82cf7ffdae995f8f16524ad2c0d ci: Set increased --timeout-factor by default (MarcoFalke)
Pull request description:
Assuming that tests don't have a logic error or race, setting a high timeout should not cause any issues. The tests will still pass just as fast in the fastest case, but it allows for some buffer in case of slow disks or otherwise starved ci machines.
Fixes#19729
ACKs for top commit:
hebasto:
ACK fa330ec2fe5f5ba68a8d43fff0b19584c0b1ff39, I have reviewed the code, and it looks OK, I agree it can be merged.
Tree-SHA512: 3da80ee008c7b08bab5fdaf7804d57c79d6fed49a7d37b9c54fc89756659fcb9981fd10afc4d07bd90d93c1699fd410a0110a3cd34d016873759d114ce3cd82d
This backport is marked as full, not partial, but it has only refactorings
and non-witness related changes.
Included commits are:
- test: Update test framework p2p protocol version to 70016
- Rename AddInventoryKnown() to AddKnownTx()
- Add support for tx-relay via wtxid
This adds a field to CNodeState that tracks whether to relay transactions with
that peer via wtxid, instead of txid. As of this commit the field will always
be false, but in a later commit we will add a way to negotiate turning this on
via p2p messages exchanged with the peer.
- Just pass a hash to AddInventoryKnown
Since it's only used for transactions, there's no need to pass in an inv type.
- Add wtxid to mempool unbroadcast tracking
## Issue being fixed or feature implemented
For platform needs `getrawtransactionmulti` will help to reduce amount
of rpc calls for sake of performance improvement.
## What was done?
Implemented new RPC, basic functional test, release note.
## How Has This Been Tested?
On testnet:
```
> getrawtransactionmulti '{"000000abbe61a4d9b9356cb1d7deb1132d0b444a62869e71c2f3aa8ce2361359":["6e3ef19a3f955ac75a1f84dae60d42bbe11548ef54e37033ff2d91b3c4a09e9c", "415d5fafd5ee24ada8b99c36df339785a3066170c0dca6bb1aa6a5b96cf51e35"], "0":["ec7090f01c0e9b6e29d3be8810b12c780d2fb34372a53b231ce18bb7d2f1e8b0"]}'
> getrawtransactionmulti '{"000000abbe61a4d9b9356cb1d7deb1132d0b444a62869e71c2f3aa8ce2361359":["6e3ef19a3f955ac75a1f84dae60d42bbe11548ef54e37033ff2d91b3c4a09e9c", "415d5fafd5ee24ada8b99c36df339785a3066170c0dca6bb1aa6a5b96cf51e35"], "0":["ec7090f01c0e9b6e29d3be8810b12c780d2fb34372a53b231ce18bb7d2f1e8b0"]}' true
```
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
---------
Co-authored-by: pasta <pasta@dashboost.org>
d841301010914203fb5ef02627c76fad99cb11f1 test: Add docstring to wait_until() in util.py to warn about its usage (Seleme Topuz)
1343c86c7cc1fc896696b3ed87c12039e4ef3a0c test: Update wait_until usage in tests not to use the one from utils (Seleme Topuz)
Pull request description:
Replace global (from [test_framework/util.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/test_framework/util.py#L228)) `wait_until()` usages with the ones provided by `BitcoinTestFramework` and `P2PInterface` classes.
The motivation behind this change is that the `util.wait_until()` expects a timeout, timeout_factor and lock and it is not aware of the context of the test framework. `BitcoinTestFramework` offers a `wait_until()` which has an understandable amount of default `timeout` and a shared `timeout_factor`. Moreover, on top of these, `mininode.wait_until()` also has a shared lock.
closes#19080
ACKs for top commit:
MarcoFalke:
ACK d841301010914203fb5ef02627c76fad99cb11f1 🦆
kallewoof:
utACK d841301010914203fb5ef02627c76fad99cb11f1
Tree-SHA512: 81604f4cfa87fed98071a80e4afe940b3897fe65cf680a69619a93e97d45f25b313c12227de7040e19517fa9c003291b232f1b40b2567aba0148f22c23c47a88
12410b1feb80189061eb4a2b43421e53cbb758a8 test: fix intermittent p2p_ibd_txrelay race, add test_framework.py#wait_until (Jon Atack)
Pull request description:
To fix these intermittent failures in Travis CI.
```
162/163 - p2p_ibd_txrelay.py failed, Duration: 2 s
stdout:
2020-07-19T05:44:17.213000Z TestFramework (INFO):
Check that nodes set minfilter to MAX_MONEY while still in IBD
2020-07-19T05:44:17.216000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/Users/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-apple-darwin16/test/functional/test_framework/test_framework.py", line 117, in main
self.run_test()
File "/Users/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-apple-darwin16/test/functional/p2p_ibd_txrelay.py", line 30, in run_test
assert_equal(conn_info['minfeefilter'], MAX_FEE_FILTER)
File "/Users/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-apple-darwin16/test/functional/test_framework/util.py", line 49, in assert_equal
raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(0E-8 == 0.09170997)
2020-07-19T05:44:17.293000Z TestFramework (INFO): Stopping nodes
```
At Marco's suggestion, cherry-picked part of #19134 to nicely simplify using `wait_until`.
ACKs for top commit:
vasild:
ACK 12410b1fe
Tree-SHA512: 615f509883682fd693e578b259cba35a9fa0bc519f1394e88c857e8b0650bfec5397bfa856cfa9e6d5ef81d0ee6ad02e4ad2b0eb0bd530b4c281cbe3e663790b
d5800da5199527a366024bc80cad7fcca17d5c4a [test] Remove final references to mininode (John Newbery)
5e8df3312e47a73e747ee892face55ed9ababeea test: resort imports (John Newbery)
85165d4332b0f72d30e0c584b476249b542338e6 scripted-diff: Rename mininode to p2p (John Newbery)
9e2897d020b114a10c860f90c5405be029afddba scripted-diff: Rename mininode_lock to p2p_lock (John Newbery)
Pull request description:
New contributors are often confused by the terminology in the test framework, and what the difference between a _node_ and a _peer_ is. To summarize:
- a 'node' is a bitcoind instance. This is the thing whose behavior is being tested. Each bitcoind node is managed by a python `TestNode` object which is used to start/stop the node, manage the node's data directory, read state about the node (eg process status, log file), and interact with the node over different interfaces.
- one of the interfaces that we can use to interact with the node is the p2p interface. Each connection to a node using this interface is managed by a python `P2PInterface` or derived object (which is owned by the `TestNode` object). We can open zero, one or many p2p connections to each bitcoind node. The node sees these connections as 'peers'.
For historic reasons, the word 'mininode' has been used to refer to those p2p interface objects that we use to connect to the bitcoind node (the code was originally taken from the 'mini-node' branch of https://github.com/jgarzik/pynode/tree/mini-node). However that name has proved to be confusing for new contributors, so rename the remaining references.
ACKs for top commit:
amitiuttarwar:
ACK d5800da519
MarcoFalke:
ACK d5800da5199527a366024bc80cad7fcca17d5c4a 🚞
Tree-SHA512: 2c46c2ac3c4278b6e3c647cfd8108428a41e80788fc4f0e386e5b0c47675bc687d94779496c09a3e5ea1319617295be10c422adeeff2d2bd68378e00e0eeb5de
dac7a111bdd3b0233d94cf68dae7a8bfc6ac9c64 refactor: test: use _ variable for unused loop counters (Sebastian Falbesoner)
Pull request description:
This tiny PR substitutes Python loops in the form of `for x in range(N): ...` by `for _ in range(N): ...` where applicable. The idea is indicating to the reader that a block (or statement, in list comprehensions) is just repeated N times, and that the loop counter is not used in the body, hence using the throwaway variable. This is already done quite often in the current tests (see e.g. `$ git grep "for _ in range("`). Another alternative would be using `itertools.repeat` (according to Python core developer Raymond Hettinger it's [even faster](https://twitter.com/raymondh/status/1144527183341375488)), but that doesn't seem to be widespread in use and I'm not sure about a readability increase.
The only drawback I see is that whenever one wants to debug loop iterations, one would need to introduce a loop variable again. Reviewing this is basically a no-brainer, since tests would fail immediately if a a substitution has taken place on a loop where the variable is used.
Instances to replace were found by `$ git grep "for.*in range("` and manually checked.
ACKs for top commit:
darosior:
ACK dac7a111bdd3b0233d94cf68dae7a8bfc6ac9c64
instagibbs:
manual inspection ACK dac7a111bd
practicalswift:
ACK dac7a111bdd3b0233d94cf68dae7a8bfc6ac9c64 -- the updated code is easier to reason about since the throwaway nature of a variable is expressed explicitly (using the Pythonic `_` idiom) instead of implicitly. Explicit is better than implicit was we all know by now :)
Tree-SHA512: 5f43ded9ce14e5e00b3876ec445b90acda1842f813149ae7bafa93f3ac3d510bb778e2c701187fd2c73585e6b87797bb2d2987139bd1a9ba7d58775a59392406
292828cd7744ec7eadede4ad54aa2117087c5435 [test] Test addr cache for multiple onion binds (dergoegge)
3382905befd23364989d941038bf7b1530fea0dc [net] Seed addr cache randomizer with port from binding address (dergoegge)
f10e80b6e4fbc151abbf1c20fbdcc3581d3688f0 [net] Use ConnectedThroughNetwork() instead of GetNetwork() to seed addr cache randomizer (dergoegge)
Pull request description:
The addr cache id randomizer is currently supposed to be seeded with the network of the inbound connection and the local socket (only the address is used not the port): a8098f2cef/src/net.cpp (L2800-L2804)
For inbound onion connections `CNode::addr.GetNetwork()` returns `NET_UNROUTABLE` and `CNode::addrBind` is set to `127.0.0.1:<onion bind port>`. This results in the same addr cache for all inbound connections on 127.0.0.1 binds.
To avoid the same addr cache across all onion and other 127.0.0.1 binds, we should seed the addr cache randomizer with the correct network for inbound onion connections (using `CNode::ConnectedThroughNetwork()`) as well as the port of `CNode::addrBind`.
ACKs for top commit:
sipa:
utACK 292828cd7744ec7eadede4ad54aa2117087c5435
mzumsande:
Code Review ACK 292828cd7744ec7eadede4ad54aa2117087c5435
naumenkogs:
utACK 292828cd7744ec7eadede4ad54aa2117087c5435
Tree-SHA512: d0be13bab6bc121c2926d4b168687f6c2ed4ce0c9dd19be71eb4886adeba8afc3daacdc4e232a0ba3b03a89d69b618abc5595b69abd1ad0c476d825bc6ea1f9f
ebfc308ea4b8851118e8194d837556bf443c329c test: add coverage for non-hex value to -minimumchainwork (brunoerg)
Pull request description:
This PR adds test coverage for the following init error:
b9ef5a10e2/src/init.cpp (L917-L919)
Passing a non-hex value to -minimumchainwork should throw an initial error.
ACKs for top commit:
laanwj:
Code review ACK ebfc308ea4b8851118e8194d837556bf443c329c
kristapsk:
re-ACK ebfc308ea4b8851118e8194d837556bf443c329c
Tree-SHA512: c665903757ae3b8b2480df97bb888e60ba4387b009fcb8031041822e87a155a0e4950ebe79873c1034f0826504521d82b1fdfdb5e8378b227d14ca545b8d4e11
6d7e46ce23217da53ff52f535879c393c02fa2b2 Remove `Warning:` (Prayank)
Pull request description:
Reason: I noticed that `Warning` is printed 2 times in `-getinfo` while reviewing https://github.com/bitcoin/bitcoin/pull/21832#issuecomment-851004943
Same string is used for GUI, log and stderr. If we need to add `Warning:` in GUI or other place we can always prepend to this string.
CLI:
```
Warnings: Unknown new rules activated (versionbit 28)
```
GUI:
ACKs for top commit:
MarcoFalke:
review ACK 6d7e46ce23217da53ff52f535879c393c02fa2b2
Tree-SHA512: 25760cf6d850f6c2216d651fa46574d2d21a9d58f4577ecb58f9566ea8cd07bfcd6679bb8a1f53575e441b02d295306f492c0c99a48c909e60e5d977ec1861f6
c133cdcdc3397a734d57e05494682bf9bf6f4c15 Cap listsinceblock target_confirmations param (Adam Stein)
Pull request description:
This addresses an issue brought up in #19587.
Currently, the `target_confirmations` parameter to `listsinceblock` is not checked for being too large. When `target_confirmations` is greater than one more than the current number of blocks, `listsinceblock` fails with error code -1. In comparison, when `target_confirmations` is less than 1, a -8 "Invalid parameter" error code is thrown.
This PR fixes the issue by returning a -8 "Invalid parameter" error if the `target_confirmations` value corresponds to a block with more confirmations than the genesis block. This happens if `target_confirmations` exceeds one more than the number of blocks.
ACKs for top commit:
laanwj:
Code review ACK c133cdcdc3397a734d57e05494682bf9bf6f4c15
ryanofsky:
Code review ACK c133cdcdc3397a734d57e05494682bf9bf6f4c15. Just suggested changes since last review. Thanks!
Tree-SHA512: 02680f4cb937d2c24d5019abd0ebfa188b8a50679a1e64e9c26bfe5c17eef6aea906832e6e2d492ba8a2ea160041bf185d66795ee691e340f6793db03c21b89a
## Issue being fixed or feature implemented
Running 3 nodes on RegTest as platform does uses do not let to create
`llmq_test_instantsend` quorum:
```
1. switch to `llmq_test_instantsend`:
+ self.extra_args = [["-llmqtestinstantsenddip0024=llmq_test_instantsend"]] * 5
2. removed cycle-quorum related code:
- self.move_to_next_cycle()
- self.log.info("Cycle H height:" + str(self.nodes[0].getblockcount()))
- self.move_to_next_cycle()
- self.log.info("Cycle H+C height:" + str(self.nodes[0].getblockcount()))
- self.move_to_next_cycle()
- self.log.info("Cycle H+2C height:" + str(self.nodes[0].getblockcount()))
-
- self.mine_cycle_quorum(llmq_type_name='llmq_test_dip0024', llmq_type=103)
3. added new quorum:
+ self.mine_quorum(llmq_type_name='llmq_test_instantsend', llmq_type=104)
and eventually it stucked, no quorum happens
2024-01-13T19:18:49.317000Z TestFramework (INFO): Expected quorum_0 at:984
2024-01-13T19:18:49.317000Z TestFramework (INFO): Expected quorum_0 hash:6788e18f0235a5c85f3d3c6233fe132a80e74a2912256db3ad876a8ebf026048
2024-01-13T19:18:49.317000Z TestFramework (INFO): quorumIndex 0: Waiting for phase 1 (init)
<frozen>
```
## What was done?
Updated condition to enable "llmq_test_instantsend":
- it is RegTest and DIP0024 is not active
- it is RegTest, DIP0024 is active, and specified as
`llmqTypeDIP0024InstantSend`
## How Has This Been Tested?
Run unit and functional tests.
Beside that functional test feature_asset_locks.py now uses this quorum
for instant send and that's an arrow that hit 2 birds: we have test for
command line option `-llmqtestinstantsenddip0024` and code of
feature_asset_locks.py is simplified.
## Breaking Changes
yes, that's a bugfix that fix quorum `llmq_test_instantsend` absentance
on regtest after dip-0024 activation.
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
`llmq/utils` has simple util code that used all over code base and also
have too heavy code for calculation quorums such as:
`GetAllQuorumMembers`, `EnsureQuorumConnections` and other.
These helpers for calculation quorums are used only by
evo/deterministicmns, evo/simplifiedmns and llmq/* modules, but
llmq/utils is included in many other modules for various trivial
helpers.
## What was done?
Prior work:
- https://github.com/dashpay/dash/pull/5753
- #5486
See also #4798
This PR remove all non-quorum calculation code from llmq/utils.
Eventually it happens that easier to take everything out rather than
move Quorum Calculation to new place atm:
- new module llmq/options have a code related to various params, command
line options, spork-related etc
- llmq/utils is not included in various files which do not use any
llmq/utils code
- helper `BuildCommitmentHash` goes to llmq/commitment
- helper `BuildSignHash` goes to llmq/signing
- helper `GetLLMQParam` inlined since it's trivial (it has not been
trivial when introduced ages ago)
- removed dependency of `IsQuorumEnabled` on CQuorumManager which means
`quorumManager` deglobalization is done for 90%
## How Has This Been Tested?
- Run unit functional tests
- updated circular dependencies
`test/lint/lint-circular-dependencies.sh`
- check that llmq/utils is not included without needs to calculate
Quorums Members
```
$ grep -r include src/ 2> /dev/null | grep -v .Po: | grep -vE 'llmq/utils.(h|cpp)': | grep llmq/utils
src/evo/mnauth.cpp:#include <llmq/utils.h>
src/evo/deterministicmns.cpp:#include <llmq/utils.h>
src/llmq/quorums.cpp:#include <llmq/utils.h>
src/llmq/blockprocessor.cpp:#include <llmq/utils.h>
src/llmq/commitment.cpp:#include <llmq/utils.h>
src/llmq/debug.cpp:#include <llmq/utils.h>
src/llmq/dkgsessionhandler.cpp:#include <llmq/utils.h>
src/llmq/dkgsession.cpp:#include <llmq/utils.h>
src/llmq/dkgsessionmgr.cpp:#include <llmq/utils.h>
src/rpc/quorums.cpp:#include <llmq/utils.h>
```
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
9a40cfc558b3f7fa4fff1270f969582af17479a5 [refactor] use waiting inside disconnect_p2ps (gzhao408)
aeb9fb414e2d000830287d9dd3fed7fc2eb570d2 [test] wait for disconnect_p2ps to be reflected in getpeerinfo (gzhao408)
e81942d2e1288367e8da94adb2b2a88be99e4751 [test] logging and style followups for bloomfilter tests (gzhao408)
Pull request description:
Followup to #19083 which adds bloomfilter-related tests.
1. Make test_node `disconnect_p2ps` wait until disconnection is complete to avoid race conditions (and not place the burden on tests) from MarcoFalke's [comment](https://github.com/bitcoin/bitcoin/pull/19083#discussion_r437383989). And clean up any redundant `wait_until`s in the functional tests.
2. Clean up style + logging in p2p_filter.py and p2p_nobloomfilter_messages.py and jonatack's other [comments](https://github.com/bitcoin/bitcoin/pull/19083#pullrequestreview-428955784)
ACKs for top commit:
jonatack:
Code review ACK 9a40cfc from re-reviewing the diff and `git range-diff 5cafb46 8386ad5 9a40cfc`
MarcoFalke:
ACK 9a40cfc558b3f7fa4fff1270f969582af17479a5 🐂
Tree-SHA512: 2e14b1c12fc08a355bd5ccad7a2a734a4ccda4bc7dc7bac171cb57359819fc1599d764290729af74832fac3e2be258c5d406c701e78ab6d7262835859b9a7d87
3a10d935ac8ebabdfd336569d943f042ff84b13e [p2p/refactor] move disconnect logic and remove misbehaving (gzhao408)
ff8c430c6589ea72b9e169455cf6437c8623cc52 [test] test disconnect for filterclear (gzhao408)
1c6b787e0319c44f0e0bede3f4a77ac7c2089db2 [netprocessing] disconnect node that sends filterclear (gzhao408)
Pull request description:
Nodes that don't have bloomfilters turned on (i.e. no `NODE_BLOOM` service) should disconnect peers that send them `filterclear` P2P messages.
Non-bloomfilter nodes already disconnect peers for [`filteradd` and `filterload`](19e919217e/src/net_processing.cpp (L2218)), but #8709 removed `filterclear` so it could be used to reset tx relay. This isn't needed now because using `feefilter` message is much better for this purpose (See #19204).
Also refactors existing disconnect logic for `filteradd` and `filterload` into respective message handlers and removes banning for them.
ACKs for top commit:
jnewbery:
Code review ACK 3a10d935ac8ebabdfd336569d943f042ff84b13e
naumenkogs:
utACK 3a10d93
gillichu:
tested ACK: quick test_runner on macOS [`3a10d93`](3a10d935ac)
MarcoFalke:
re-ACK 3a10d935ac only change is replacing false with true 🚝
Tree-SHA512: 7aad8b3c0b0e776a47ad52544f0c1250feb242320f9a2962542f5905042f77e297a1486f8cdc3bf0fb93cd00c1ab66a67b2ec426eb6da3fe4cda56b5e623620f
(BACKPORT NOTICE: p2p_filter.py has also fixes for d5fbd4a92a3e7c1f8266d4cb4b639a0fe4c4c61f:test/functional/p2p_filter.py)
+ Merge #18726: test: check misbehavior more independently in p2p_filter.py
---------------------
dca73941eb0f0a4c9b68efed3870b536f7dd6cfe scripted-diff: rename node to peer for mininodes (gzhao408)
0474ea25afc65546cbfe5f822c0212bf3e211023 [test] fix race conditions and test in p2p_filter (gzhao408)
4ef80f0827392a1310ca5a29cc1f8f5ca5d16f95 [test] sending invalid msgs to node with bloomfilters=0 causes disconnect (gzhao408)
497a619386008dfaec0db15ecaebcdfaf75f5011 [test] add BIP 37 test for node with fRelay=false (gzhao408)
e8acc6015695c8439fc971a12709468995b96dcf [test] add mempool msg test for node with bloomfilter enabled (gzhao408)
Pull request description:
This PR adds a few tests that are bloomfilter-related, including behavior for when bloomfilters are turned _off_:
1. Tests p2p message `msg_mempool`: a node that has `peerbloomfilters` enabled should send its mempool (disabled behavior already tested [here](https://github.com/bitcoin/bitcoin/blob/master/test/functional/p2p_mempool.py)).
2. Tests that bloomfilter peers with [`fRelay=False`](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki#extensions-to-existing-messages) in the `version` message should not receive any invs until they set the filter. The rest is the same as what’s already tested in `p2p_filter.py`.
3. Tests that peers get disconnected if they send `filterload` or `filteradd` p2p messages to a node with bloom filters disabled.
4. Refactor: renames p2p_mempool.py to p2p_nobloomfilter_messages.py.
5. Fixes race conditions in p2p_filter.py
ACKs for top commit:
MarcoFalke:
ACK dca73941eb only changes is restoring accidentally deleted test 🍮
jonatack:
ACK dca73941eb0f0a4c9b68efed3870b536f7dd6cfe modulo a few nits if you retouch, happy to re-ACK if you take any of them but don't feel obliged to.
Tree-SHA512: 442aeab0755cb8b830251ea170d1d5e6da8ac9029b3276d407a20ee3d588cc61b77b8842368de18c244056316b8c63b911776d6e106bc7c023439ab915b27ad3
af2a145e575f23c64909e6cf1fb323c603bda7ca Refactor resource exhaustion test (Troy Giorshev)
5c4648d17ba18e4194959963994cc6b37053f127 Fix "invalid message size" test (Troy Giorshev)
ff1e7b884447a5ba10553b2d964625f94e255bdc Move size limits to module-global (Troy Giorshev)
57890abf2c7919eddfec36178b1136cd44ffe883 Remove two unneeded tests (Troy Giorshev)
Pull request description:
This PR touches only the p2p_invalid_messages.py functional test module. There are two main goals accomplished here. First, it fixes the "invalid message size" test, which previously made a message that was invalid for multiple reasons. Second, it refactors the file into a single consistent style. This file appears to have originally had two authors, with different styles and some test duplication.
It should now be easier and quicker to understand this module, anticipating the upcoming [BIP324](https://github.com/bitcoin/bitcoin/pull/18242) and [AltNet](https://github.com/bitcoin/bitcoin/issues/18989) changes.
This should probably go in ahead of #19107, but the two are not strictly related.
ACKs for top commit:
jnewbery:
ACK af2a145e575f23c64909e6cf1fb323c603bda7ca
MarcoFalke:
re-ACK af2a145e57 🍦
Tree-SHA512: 9b57561e142c5eaefac5665f7355c8651670400b4db1a89525d2dfdd20e872d6873c4f6175c4222b6f5a8e5210cf5d6a52da69b925b673a2e2ac30a15d670d1c
fa0a4d6c605b8ed47796f68068d6273bef7fcaef test: remove assert_blockchain_height (MarcoFalke)
Pull request description:
This simplifies the code and solves intermittent timeouts caused by commit 0d39b5848a7a341cd2b958336861cdd4098e2616.
E.g. https://cirrus-ci.com/task/5196092369272832?command=ci#L3126
```
test 2021-02-08T12:27:56.275000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 127, in main
self.run_test()
File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/feature_assumevalid.py", line 180, in run_test
self.assert_blockchain_height(self.nodes[0], 101)
File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/feature_assumevalid.py", line 92, in assert_blockchain_height
assert False, "blockchain too short after timeout: %d" % current_height
AssertionError: blockchain too short after timeout: 101
ACKs for top commit:
glozow:
ACK fa0a4d6c60
Tree-SHA512: 3859b0c1581c21f03c775f119204cc3a219e5d86346883634886f6da46feaf254b9c6c49c1ec4581ffe409cdf05f6e91ec38c3976c3daf4a9e67f96ddc1e0dce
fa362064e383163a2585ffbc71ac1ea3bcc92663 rpc: Return total fee in mempool (MarcoFalke)
Pull request description:
This avoids having to loop over the whole mempool to query each entry's fee
ACKs for top commit:
achow101:
ACK fa362064e383163a2585ffbc71ac1ea3bcc92663
glozow:
ACK fa362064e3🧸
jnewbery:
ACK fa362064e383163a2585ffbc71ac1ea3bcc92663
Tree-SHA512: e2fa1664df39c9e187f9229fc35764ccf436f6f75889c5a206d34fff473fc21efbf2bb143f4ca7895c27659218c22884d0ec4195e7a536a5a96973fc9dd82d08
b6121edf70a8d50fd16ddbba0c3168e5e49bfc2e swapped "is" for "==" in literal comparison (Tyler Chambers)
Pull request description:
In Python 3.8+ literal comparisons using "is" instead of "==" produce a SyntaxWarning [source](https://docs.python.org/3.8/whatsnew/3.8.html#changes-in-python-behavior).
I checked the entire devtools directory, this seems to be the only occurrence.
This is a small fix, but removes the SyntaxWarning.
Fixes: #20338
ACKs for top commit:
hebasto:
re-ACK b6121edf70a8d50fd16ddbba0c3168e5e49bfc2e, only squashed since my [previous](https://github.com/bitcoin/bitcoin/pull/20346#pullrequestreview-525934568) review.
practicalswift:
re-ACK b6121edf70a8d50fd16ddbba0c3168e5e49bfc2e: patch still looks correct
theStack:
utACK b6121edf70a8d50fd16ddbba0c3168e5e49bfc2e
Tree-SHA512: 82a43495d6552fbaa3b02b58f0930b049d27aa937fe44b47714e3c059f844cc494de20674557371cbccf24fb8873ecb7376fb965ae326847eed2b855ed2d59c6
5dc6d9207778c51c10c16fac4b3663bc7905bafc test: make BIP157 messages default-constructible (MESSAGEMAP compatibility) (Sebastian Falbesoner)
71e4cfefe765c58937b3fd3125782ca8407315d2 test: p2p: add missing BIP157 message types to MESSAGEMAP (Sebastian Falbesoner)
Pull request description:
The script [message-capture-parser.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/message-capture/message-capture-parser.py) currently doesn't support parsing the BIP157 messages `getcfilters`, `getcfheaders` and `getcfcheckpt`, e.g.
```
$ ./contrib/message-capture/message-capture-parser.py msgs_recv.dat
...
WARNING - Unrecognized message type b'getcfcheckpt' in /home/thestack/bitcoin/msgs_recv.dat
...
```
This PR fixes this by adding the missing message type mappings to the [`MESSAGEMAP`](225e5b57b2/test/functional/test_framework/p2p.py (L95-L127)) in the test framework and add default-constructors for the corresponding `msg_`... classes.
Without the second commit, the following error message would occur:
```
File "/home/thestack/bitcoin/./contrib/message-capture/message-capture-parser.py", line 141, in process_file
msg = MESSAGEMAP[msgtype]()
TypeError: __init__() missing 2 required positional arguments: 'filter_type' and 'stop_hash'
```
ACKs for top commit:
dunxen:
tACK [5dc6d92](5dc6d92077)
Tree-SHA512: d656c4d38a856373f01d7c293ae7d2b27378a9fc248048ebf2a64725ef8b498b3ddf4f420704abdb20d0c68ca548f1777602c5e73b66821a20c97ae618f1d63f
faac67cab02a755b0ce67716c5e5889432b13b83 test: Fix intermittent race in p2p_unrequested_blocks.py (MacroFake)
Pull request description:
Disconnect may also result in an `OSError`, not only an `AssertionError`. Instead of maintaining a dead code path and enumerating disconnect reasons, just assume disconnection happens every time.
ACKs for top commit:
jamesob:
Code review ACK faac67cab0
Tree-SHA512: d2cec003168e421a5faed275cb2e1ef9fc63f9e8514f41d21da17e8964c79e5b453ccd72cd7ec62805f45293cf877be5bc8124ae98a515c0aa42d6e053409653
1df42bc262d994c30d390ea73b232b8d2548899e test: compare `/mempool/info` response with `getmempoolinfo` RPC (brunoerg)
Pull request description:
This PRs compares `/mempool/info` REST response with `getmempoolinfo` RPC in `interface_rest.py`.
Similar to #24936 and #24797.
ACKs for top commit:
theStack:
ACK 1df42bc262d994c30d390ea73b232b8d2548899e
Tree-SHA512: 2de36d70fa61612e7976f875e55f98e78b1cdb909b48cff18e6a70c55eda34b799e210bcd55361ea947388b7778d867290a73be4f799bb36afd65423ad49c487
3258bad996262792ba77573d6080dafa3952929c changes color of skipped functional tests (Jacob P. Fickes)
Pull request description:
changes the color of skipped functional tests (currently grey and can be hard to read/invisible on dark backgrounds) to yellow.
resolves#24791
ACKs for top commit:
theStack:
Tested ACK 3258bad996262792ba77573d6080dafa3952929c
jarolrod:
Tested ACK 3258bad996
Tree-SHA512: 3fe5ae0d3b4902b2b6bda6e89ab780feb8bf4b7cb1ce7e8467057b94a1e0a26ddeaf3cac0bc19b06ef10d8bccaac9c495029d42740fbedab8fb0d5fdd7d02eaf
bef61496ab5e12e38ac5794cd0836723af070ab5 test: compare `/mempool/contents` response with `getrawmempool` RPC (brunoerg)
5bc5cbaf310f60e89c72e8ecf3f6187c85499027 doc: add reference to `getrawmempool` RPC in `/mempool/contents` REST doc (brunoerg)
Pull request description:
This PR is similar to #24797, it compares `/mempool/contents` REST response with `getrawmempool` RPC (verbose=True) since they use the same `MempoolToJSON` function.
Also, adds a reference to `getrawmempool` RPC help to get details about the fields from `/mempool/contents`.
ACKs for top commit:
0xB10C:
ACK bef6149
Tree-SHA512: b7e9e9c765ee837986ba167b9234a9b95c9ef0a9ebcc2a03d50f6be6d3aba1480bd77c78111d95df1e4023cde6dfc64bf1e7908d9e5b6f96ca46b76611a4a9b4
9b4fa0af40cd88ed25dd77962235fbf268bdcaa7 net: Print error message if -proxy is specified without arguments (instead of continuing without proxy server) (practicalswift)
Pull request description:
Exit with error message if `-proxy` is specified without arguments (instead of continuing without proxy server).
Continuing without a proxy server when the end-user has specified `-proxy` may result in accidental loss of privacy. (The end-user might think he/she is using a proxy when he/she is not.)
Before this patch:
```
$ src/bitcoind -proxy
…
2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -listen=0
2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -upnp=0
2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -discover=0
2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -listen=0 -> setting -listenonion=0
…
2020-09-23T00:24:33Z init message: Starting network threads...
```
`bitcoind` is now running *without* a proxy server (`GetProxy(…, …) == false`, `HaveNameProxy() == false`, etc.).
Note that the "-proxy set" log messages above which the end-user might interpret as "good, my traffic is now routed via the proxy".
After this patch:
```
$ src/bitcoind -proxy
Error: No proxy server specified. Use -proxy=<ip> or -proxy=<ip:port>.
$ echo $?
1
```
ACKs for top commit:
laanwj:
re-ACK 9b4fa0af40cd88ed25dd77962235fbf268bdcaa7
kristapsk:
ACK 9b4fa0af40cd88ed25dd77962235fbf268bdcaa7, I have tested the code.
hebasto:
re-ACK 9b4fa0af40cd88ed25dd77962235fbf268bdcaa7
Tree-SHA512: 4ba7a011991699a54b5bb87ec68367c681231bf5dcd36f8c89ff9ddc2e8d29df453817b7e362597e652ad6b341a22b7274be0fd78d435e5f0fd8058e5221c4ce
It is partial de-circularisation of dependencies between that includes net_processing
Classes that still depends on net_processing but should not:
- llmq/dkgsessionmgr
- llmq/signing
- llmq/instantsend
They have asynchronous processing and with current impl that's impossible to do
5d77549d8b287eb773db695b88c165ebe3be1005 doc: Add mypy to test dependencies (Hennadii Stepanov)
7dda912e1c28b02723c9f24fa6c4e9003d928978 test: Do not swallow flake8 exit code (Hennadii Stepanov)
Pull request description:
After #18210 the `flake8` exit code in `test/lint/lint-python.sh` just not used that makes the linter broken.
This PR:
- combines exit codes of `flake8` and `mypy` into the `test/lint/lint-python.sh` exit code
- documents `mypy` as the test dependency
ACKs for top commit:
MarcoFalke:
Approach ACK 5d77549d8b287eb773db695b88c165ebe3be1005, fine with me
practicalswift:
ACK 5d77549d8b287eb773db695b88c165ebe3be1005
Tree-SHA512: e948ba04dc4d73393967ebf3c6a26c40d428d33766382a0310fc64746cb7972e027bd62e7ea76898b742a656cf7d0fcda2fdd61560a21bfd7be249cea27f3d41
bd7e530f010d43816bb05d6f1590d1cd36cdaa2c This PR adds initial support for type hints checking in python scripts. (Kiminuo)
Pull request description:
This PR adds initial support for type hints checking in python scripts.
Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention."
[Mypy](https://mypy.readthedocs.io/en/latest/index.html) is used in `lint-python.sh` to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error.
**Notes:**
* [--ignore-missing-imports](https://mypy.readthedocs.io/en/latest/command_line.html#cmdoption-mypy-ignore-missing-imports) switch is passed on to `mypy` checker for now. The effect of this is that one does not need `# type: ignore` for `import zmq`. More information about import processing can be found [here](https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports). This can be changed in a follow-up PR, if it is deemed useful.
* We are stuck with Python 3.5 until 04/2021 (see https://packages.ubuntu.com/xenial/python3). When Python version is bumped to 3.6+, one can change:
```python
_opcode_instances = [] # type: List[CScriptOp]
```
to
```python
_opcode_instances:List[CScriptOp] = []
```
for type hints that are **not** function parameters and function return types.
**Useful resources:**
* https://docs.python.org/3.5/library/typing.html
* https://www.python.org/dev/peps/pep-0484/
ACKs for top commit:
fanquake:
ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c - the type checking is not the most robust (there are things it fails to detect), but I think this is worth adopting (in a limited capacity while we maintain 3.5 compat).
MarcoFalke:
ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c fine with me
Tree-SHA512: 21ef213915fb1dec6012f59ef17484e6c9e0abf542a316b63d5f21a7778ad5ebabf8961ef5fc8e5414726c2ee9c6ae07c7353fb4dd337f8fcef5791199c8987a
fa41b0a6dac7afd77e2b94eca6520ab3d2adc231 pep-8 test/functional/test_framework/util.py (MarcoFalke)
faa841bc979ca306f5ba4d5f7b78fcc427b8e413 test: refactor: Inline adjust_bitcoin_conf_for_pre_17 (MarcoFalke)
Pull request description:
This removes mental and code complexity as well as attack surface for bikeshedding
ACKs for top commit:
Sjors:
utACK fa41b0a6dac7afd77e2b94eca6520ab3d2adc231
Tree-SHA512: 6e3c872e66d98ffaa7aecdfd64aa7dd8fbb51815a8fdaba170ce0772b4c3360084d0ebab4a5feac768ab5df50d04528d7daafc51ba07c15445c1ef94fa3efd34
62068381a3b9c065d81300be79abba7aecfdb41b [tests] Make mininode_lock non-reentrant (John Newbery)
c67c1f2c032a8efa141d776a7e5be58f052159ea [tests] Don't call super twice in P2PTxInvStore.on_inv() (John Newbery)
9d80762fa0931fe553fad241e95bcc1515ef0e95 [tests] Don't acquire mininode_lock twice in wait_for_broadcast() (John Newbery)
edae6075aa3b1169c84b65e76fd48d68242a294e [tests] Only acquire lock once in p2p_compactblocks.py (John Newbery)
Pull request description:
There's no need for mininode_lock to be reentrant.
Use a simpler non-recursive lock.
ACKs for top commit:
MarcoFalke:
ACK 62068381a3b9c065d81300be79abba7aecfdb41b 😃
jonatack:
ACK 62068381a3b9c0
Tree-SHA512: dcbc19e6c986970051705789be0ff7bec70c69cf76d5b468c2ba4cb732883ad512b1de5c3206c2eca41fa3f1c4806999df4cabbf67fc3c463bb817458e59a19c
