Commit Graph

1624 Commits

Author SHA1 Message Date
Konstantin Akimov
852adb56ae
refactor: split llmq/utils to Quorum Calculation and llmq/options (#5790)
## Issue being fixed or feature implemented
`llmq/utils` has simple util code that used all over code base and also
have too heavy code for calculation quorums such as:
`GetAllQuorumMembers`, `EnsureQuorumConnections` and other.

These helpers for calculation quorums are used only by
evo/deterministicmns, evo/simplifiedmns and llmq/* modules, but
llmq/utils is included in many other modules for various trivial
helpers.



## What was done?
Prior work:
 - https://github.com/dashpay/dash/pull/5753
 - #5486
 See also #4798

This PR remove all non-quorum calculation code from llmq/utils.
Eventually it happens that easier to take everything out rather than
move Quorum Calculation to new place atm:
- new module llmq/options have a code related to various params, command
line options, spork-related etc
- llmq/utils is not included in various files which do not use any
llmq/utils code
 - helper `BuildCommitmentHash` goes to llmq/commitment
 - helper `BuildSignHash` goes to llmq/signing
- helper `GetLLMQParam` inlined since it's trivial (it has not been
trivial when introduced ages ago)
- removed dependency of `IsQuorumEnabled` on CQuorumManager which means
`quorumManager` deglobalization is done for 90%


## How Has This Been Tested?
 - Run unit functional tests
- updated circular dependencies
`test/lint/lint-circular-dependencies.sh`
- check that llmq/utils is not included without needs to calculate
Quorums Members
```
$ grep -r include src/ 2> /dev/null | grep -v .Po: | grep -vE 'llmq/utils.(h|cpp)': | grep llmq/utils  
src/evo/mnauth.cpp:#include <llmq/utils.h>
src/evo/deterministicmns.cpp:#include <llmq/utils.h>
src/llmq/quorums.cpp:#include <llmq/utils.h>
src/llmq/blockprocessor.cpp:#include <llmq/utils.h>
src/llmq/commitment.cpp:#include <llmq/utils.h>
src/llmq/debug.cpp:#include <llmq/utils.h>
src/llmq/dkgsessionhandler.cpp:#include <llmq/utils.h>
src/llmq/dkgsession.cpp:#include <llmq/utils.h>
src/llmq/dkgsessionmgr.cpp:#include <llmq/utils.h>
src/rpc/quorums.cpp:#include <llmq/utils.h>
```


## Breaking Changes
N/A

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
2024-01-17 19:56:41 -06:00
MarcoFalke
6f02baba76
Merge #19252: test: wait for disconnect in disconnect_p2ps + bloomfilter test followups
9a40cfc558b3f7fa4fff1270f969582af17479a5 [refactor] use waiting inside disconnect_p2ps (gzhao408)
aeb9fb414e2d000830287d9dd3fed7fc2eb570d2 [test] wait for disconnect_p2ps to be reflected in getpeerinfo (gzhao408)
e81942d2e1288367e8da94adb2b2a88be99e4751 [test] logging and style followups for bloomfilter tests (gzhao408)

Pull request description:

  Followup to #19083 which adds bloomfilter-related tests.

  1. Make test_node `disconnect_p2ps` wait until disconnection is complete to avoid race conditions (and not place the burden on tests) from MarcoFalke's [comment](https://github.com/bitcoin/bitcoin/pull/19083#discussion_r437383989). And clean up any redundant `wait_until`s in the functional tests.
  2. Clean up style + logging in p2p_filter.py and p2p_nobloomfilter_messages.py and jonatack's other [comments](https://github.com/bitcoin/bitcoin/pull/19083#pullrequestreview-428955784)

ACKs for top commit:
  jonatack:
    Code review ACK 9a40cfc from re-reviewing the diff and `git range-diff 5cafb46 8386ad5 9a40cfc`
  MarcoFalke:
    ACK 9a40cfc558b3f7fa4fff1270f969582af17479a5 🐂

Tree-SHA512: 2e14b1c12fc08a355bd5ccad7a2a734a4ccda4bc7dc7bac171cb57359819fc1599d764290729af74832fac3e2be258c5d406c701e78ab6d7262835859b9a7d87
2024-01-16 15:05:10 -06:00
fanquake
65217a74a3
Merge #19260: p2p: disconnect peers that send filterclear + update existing filter msg disconnect logic
3a10d935ac8ebabdfd336569d943f042ff84b13e [p2p/refactor] move disconnect logic and remove misbehaving (gzhao408)
ff8c430c6589ea72b9e169455cf6437c8623cc52 [test] test disconnect for filterclear (gzhao408)
1c6b787e0319c44f0e0bede3f4a77ac7c2089db2 [netprocessing] disconnect node that sends filterclear (gzhao408)

Pull request description:

  Nodes that don't have bloomfilters turned on (i.e. no `NODE_BLOOM` service) should disconnect peers that send them `filterclear` P2P messages.

  Non-bloomfilter nodes already disconnect peers for [`filteradd` and `filterload`](19e919217e/src/net_processing.cpp (L2218)), but #8709 removed `filterclear` so it could be used to reset tx relay. This isn't needed now because using `feefilter` message is much better for this purpose (See #19204).

  Also refactors existing disconnect logic for `filteradd` and `filterload` into respective message handlers and removes banning for them.

ACKs for top commit:
  jnewbery:
    Code review ACK 3a10d935ac8ebabdfd336569d943f042ff84b13e
  naumenkogs:
    utACK 3a10d93
  gillichu:
    tested ACK: quick test_runner on macOS [`3a10d93`](3a10d935ac)
  MarcoFalke:
    re-ACK 3a10d935ac only change is replacing false with true 🚝

Tree-SHA512: 7aad8b3c0b0e776a47ad52544f0c1250feb242320f9a2962542f5905042f77e297a1486f8cdc3bf0fb93cd00c1ab66a67b2ec426eb6da3fe4cda56b5e623620f
2024-01-16 15:05:10 -06:00
MarcoFalke
cc6ebdb8da
Merge #19083: test: msg_mempool, fRelay, and other bloomfilter tests
(BACKPORT NOTICE: p2p_filter.py has also fixes for d5fbd4a92a3e7c1f8266d4cb4b639a0fe4c4c61f:test/functional/p2p_filter.py)
+    Merge #18726: test: check misbehavior more independently in p2p_filter.py
---------------------

dca73941eb0f0a4c9b68efed3870b536f7dd6cfe scripted-diff: rename node to peer for mininodes (gzhao408)
0474ea25afc65546cbfe5f822c0212bf3e211023 [test] fix race conditions and test in p2p_filter (gzhao408)
4ef80f0827392a1310ca5a29cc1f8f5ca5d16f95 [test] sending invalid msgs to node with bloomfilters=0 causes disconnect (gzhao408)
497a619386008dfaec0db15ecaebcdfaf75f5011 [test] add BIP 37 test for node with fRelay=false (gzhao408)
e8acc6015695c8439fc971a12709468995b96dcf [test] add mempool msg test for node with bloomfilter enabled (gzhao408)

Pull request description:

  This PR adds a few tests that are bloomfilter-related, including behavior for when bloomfilters are turned _off_:
  1. Tests p2p message `msg_mempool`: a node that has `peerbloomfilters` enabled should send its mempool (disabled behavior already tested [here](https://github.com/bitcoin/bitcoin/blob/master/test/functional/p2p_mempool.py)).
  2. Tests that bloomfilter peers with [`fRelay=False`](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki#extensions-to-existing-messages) in the `version` message should not receive any invs until they set the filter. The rest is the same as what’s already tested in `p2p_filter.py`.
  3. Tests that peers get disconnected if they send `filterload` or `filteradd` p2p messages to a node with bloom filters disabled.
  4. Refactor: renames p2p_mempool.py to p2p_nobloomfilter_messages.py.
  5. Fixes race conditions in p2p_filter.py

ACKs for top commit:
  MarcoFalke:
    ACK dca73941eb only changes is restoring accidentally deleted test 🍮
  jonatack:
    ACK dca73941eb0f0a4c9b68efed3870b536f7dd6cfe modulo a few nits if you retouch, happy to re-ACK if you take any of them but don't feel obliged to.

Tree-SHA512: 442aeab0755cb8b830251ea170d1d5e6da8ac9029b3276d407a20ee3d588cc61b77b8842368de18c244056316b8c63b911776d6e106bc7c023439ab915b27ad3
2024-01-16 15:05:09 -06:00
MarcoFalke
94680f4ee3
Merge #19177: test: Fix and clean p2p_invalid_messages functional tests
af2a145e575f23c64909e6cf1fb323c603bda7ca Refactor resource exhaustion test (Troy Giorshev)
5c4648d17ba18e4194959963994cc6b37053f127 Fix "invalid message size" test (Troy Giorshev)
ff1e7b884447a5ba10553b2d964625f94e255bdc Move size limits to module-global (Troy Giorshev)
57890abf2c7919eddfec36178b1136cd44ffe883 Remove two unneeded tests (Troy Giorshev)

Pull request description:

  This PR touches only the p2p_invalid_messages.py functional test module.  There are two main goals accomplished here.  First, it fixes the "invalid message size" test, which previously made a message that was invalid for multiple reasons.  Second, it refactors the file into a single consistent style.  This file appears to have originally had two authors, with different styles and some test duplication.

  It should now be easier and quicker to understand this module, anticipating the upcoming [BIP324](https://github.com/bitcoin/bitcoin/pull/18242) and [AltNet](https://github.com/bitcoin/bitcoin/issues/18989) changes.

  This should probably go in ahead of #19107, but the two are not strictly related.

ACKs for top commit:
  jnewbery:
    ACK af2a145e575f23c64909e6cf1fb323c603bda7ca
  MarcoFalke:
    re-ACK af2a145e57 🍦

Tree-SHA512: 9b57561e142c5eaefac5665f7355c8651670400b4db1a89525d2dfdd20e872d6873c4f6175c4222b6f5a8e5210cf5d6a52da69b925b673a2e2ac30a15d670d1c
2024-01-16 15:05:09 -06:00
MarcoFalke
5d42d044df
Merge #21117: test: remove assert_blockchain_height
fa0a4d6c605b8ed47796f68068d6273bef7fcaef test: remove assert_blockchain_height (MarcoFalke)

Pull request description:

  This simplifies the code and solves intermittent timeouts caused by commit 0d39b5848a7a341cd2b958336861cdd4098e2616.

  E.g. https://cirrus-ci.com/task/5196092369272832?command=ci#L3126

  ```
   test  2021-02-08T12:27:56.275000Z TestFramework (ERROR): Assertion failed
                                     Traceback (most recent call last):
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 127, in main
                                         self.run_test()
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/feature_assumevalid.py", line 180, in run_test
                                         self.assert_blockchain_height(self.nodes[0], 101)
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/feature_assumevalid.py", line 92, in assert_blockchain_height
                                         assert False, "blockchain too short after timeout: %d" % current_height
                                     AssertionError: blockchain too short after timeout: 101

ACKs for top commit:
  glozow:
    ACK fa0a4d6c60

Tree-SHA512: 3859b0c1581c21f03c775f119204cc3a219e5d86346883634886f6da46feaf254b9c6c49c1ec4581ffe409cdf05f6e91ec38c3976c3daf4a9e67f96ddc1e0dce
2024-01-16 09:29:49 -06:00
MarcoFalke
63885189ae
Merge #20944: rpc: Return total fee in getmempoolinfo
fa362064e383163a2585ffbc71ac1ea3bcc92663 rpc: Return total fee in mempool (MarcoFalke)

Pull request description:

  This avoids having to loop over the whole mempool to query each entry's fee

ACKs for top commit:
  achow101:
    ACK fa362064e383163a2585ffbc71ac1ea3bcc92663
  glozow:
    ACK fa362064e3 🧸
  jnewbery:
    ACK fa362064e383163a2585ffbc71ac1ea3bcc92663

Tree-SHA512: e2fa1664df39c9e187f9229fc35764ccf436f6f75889c5a206d34fff473fc21efbf2bb143f4ca7895c27659218c22884d0ec4195e7a536a5a96973fc9dd82d08
2024-01-16 09:29:49 -06:00
MacroFake
736d2bc23c
Merge bitcoin/bitcoin#25126: test: add BIP157 message parsing support (via MESSAGEMAP)
5dc6d9207778c51c10c16fac4b3663bc7905bafc test: make BIP157 messages default-constructible (MESSAGEMAP compatibility) (Sebastian Falbesoner)
71e4cfefe765c58937b3fd3125782ca8407315d2 test: p2p: add missing BIP157 message types to MESSAGEMAP (Sebastian Falbesoner)

Pull request description:

  The script [message-capture-parser.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/message-capture/message-capture-parser.py) currently doesn't support parsing the BIP157 messages `getcfilters`, `getcfheaders` and `getcfcheckpt`, e.g.
  ```
  $ ./contrib/message-capture/message-capture-parser.py msgs_recv.dat
  ...
      WARNING - Unrecognized message type b'getcfcheckpt' in /home/thestack/bitcoin/msgs_recv.dat
  ...
  ```

  This PR fixes this by adding the missing message type mappings to the [`MESSAGEMAP`](225e5b57b2/test/functional/test_framework/p2p.py (L95-L127)) in the test framework and add default-constructors for the corresponding `msg_`... classes.

  Without the second commit, the following error message would occur:
  ```
    File "/home/thestack/bitcoin/./contrib/message-capture/message-capture-parser.py", line 141, in process_file
      msg = MESSAGEMAP[msgtype]()
  TypeError: __init__() missing 2 required positional arguments: 'filter_type' and 'stop_hash'
  ```

ACKs for top commit:
  dunxen:
    tACK [5dc6d92](5dc6d92077)

Tree-SHA512: d656c4d38a856373f01d7c293ae7d2b27378a9fc248048ebf2a64725ef8b498b3ddf4f420704abdb20d0c68ca548f1777602c5e73b66821a20c97ae618f1d63f
2024-01-14 11:05:37 -06:00
MacroFake
b3343d73ab
Merge bitcoin/bitcoin#25124: test: Fix intermittent race in p2p_unrequested_blocks.py
faac67cab02a755b0ce67716c5e5889432b13b83 test: Fix intermittent race in p2p_unrequested_blocks.py (MacroFake)

Pull request description:

  Disconnect may also result in an `OSError`, not only an `AssertionError`. Instead of maintaining a dead code path and enumerating disconnect reasons, just assume disconnection happens every time.

ACKs for top commit:
  jamesob:
    Code review ACK faac67cab0

Tree-SHA512: d2cec003168e421a5faed275cb2e1ef9fc63f9e8514f41d21da17e8964c79e5b453ccd72cd7ec62805f45293cf877be5bc8124ae98a515c0aa42d6e053409653
2024-01-14 11:05:37 -06:00
MacroFake
431c1e3458
Merge bitcoin/bitcoin#25121: test: compare /mempool/info response with getmempoolinfo RPC
1df42bc262d994c30d390ea73b232b8d2548899e test: compare `/mempool/info` response with `getmempoolinfo` RPC (brunoerg)

Pull request description:

  This PRs compares `/mempool/info` REST response with `getmempoolinfo` RPC in `interface_rest.py`.
  Similar to #24936 and #24797.

ACKs for top commit:
  theStack:
    ACK 1df42bc262d994c30d390ea73b232b8d2548899e

Tree-SHA512: 2de36d70fa61612e7976f875e55f98e78b1cdb909b48cff18e6a70c55eda34b799e210bcd55361ea947388b7778d867290a73be4f799bb36afd65423ad49c487
2024-01-14 11:05:36 -06:00
laanwj
a245e52f55
Merge bitcoin/bitcoin#24793: test: Change color of skipped functional tests
3258bad996262792ba77573d6080dafa3952929c changes color of skipped functional tests (Jacob P. Fickes)

Pull request description:

  changes the color of skipped functional tests (currently grey and can be hard to read/invisible on dark backgrounds) to yellow.

  resolves #24791

ACKs for top commit:
  theStack:
    Tested ACK 3258bad996262792ba77573d6080dafa3952929c
  jarolrod:
    Tested ACK 3258bad996

Tree-SHA512: 3fe5ae0d3b4902b2b6bda6e89ab780feb8bf4b7cb1ce7e8467057b94a1e0a26ddeaf3cac0bc19b06ef10d8bccaac9c495029d42740fbedab8fb0d5fdd7d02eaf
2024-01-14 11:05:35 -06:00
MarcoFalke
710ea6e114
Merge bitcoin/bitcoin#24936: test: compare /mempool/contents response with getrawmempool RPC
bef61496ab5e12e38ac5794cd0836723af070ab5 test: compare `/mempool/contents` response with `getrawmempool` RPC (brunoerg)
5bc5cbaf310f60e89c72e8ecf3f6187c85499027 doc: add reference to `getrawmempool` RPC in `/mempool/contents` REST doc (brunoerg)

Pull request description:

  This PR is similar to #24797, it compares `/mempool/contents` REST response with `getrawmempool` RPC (verbose=True) since they use the same `MempoolToJSON` function.

  Also, adds a reference to `getrawmempool` RPC help to get details about the fields from `/mempool/contents`.

ACKs for top commit:
  0xB10C:
    ACK bef6149

Tree-SHA512: b7e9e9c765ee837986ba167b9234a9b95c9ef0a9ebcc2a03d50f6be6d3aba1480bd77c78111d95df1e4023cde6dfc64bf1e7908d9e5b6f96ca46b76611a4a9b4
2024-01-13 19:32:31 -06:00
Wladimir J. van der Laan
4293cbdb53
Merge #20003: net: Exit with error message if -proxy is specified without arguments (instead of continuing without proxy server)
9b4fa0af40cd88ed25dd77962235fbf268bdcaa7 net: Print error message if -proxy is specified without arguments (instead of continuing without proxy server) (practicalswift)

Pull request description:

  Exit with error message if `-proxy` is specified without arguments (instead of continuing without proxy server).

  Continuing without a proxy server when the end-user has specified `-proxy` may result in accidental loss of privacy. (The end-user might think he/she is using a proxy when he/she is not.)

  Before this patch:

  ```
  $ src/bitcoind -proxy
  …
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -listen=0
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -upnp=0
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -discover=0
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -listen=0 -> setting -listenonion=0
  …
  2020-09-23T00:24:33Z init message: Starting network threads...
  ```

  `bitcoind` is now running *without* a proxy server (`GetProxy(…, …) == false`, `HaveNameProxy() == false`, etc.).

  Note that the "-proxy set" log messages above which the end-user might interpret as "good, my traffic is now routed via the proxy".

  After this patch:

  ```
  $ src/bitcoind -proxy
  Error: No proxy server specified. Use -proxy=<ip> or -proxy=<ip:port>.
  $ echo $?
  1
  ```

ACKs for top commit:
  laanwj:
    re-ACK 9b4fa0af40cd88ed25dd77962235fbf268bdcaa7
  kristapsk:
    ACK 9b4fa0af40cd88ed25dd77962235fbf268bdcaa7, I have tested the code.
  hebasto:
    re-ACK 9b4fa0af40cd88ed25dd77962235fbf268bdcaa7

Tree-SHA512: 4ba7a011991699a54b5bb87ec68367c681231bf5dcd36f8c89ff9ddc2e8d29df453817b7e362597e652ad6b341a22b7274be0fd78d435e5f0fd8058e5221c4ce
2024-01-10 19:22:59 -06:00
fanquake
271acac3a7
Merge #18210: test: type hints in Python tests
bd7e530f010d43816bb05d6f1590d1cd36cdaa2c This PR adds initial support for type hints checking in python scripts. (Kiminuo)

Pull request description:

  This PR adds initial support for type hints checking in python scripts.

  Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention."

  [Mypy](https://mypy.readthedocs.io/en/latest/index.html) is used in `lint-python.sh` to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error.

  **Notes:**

  * [--ignore-missing-imports](https://mypy.readthedocs.io/en/latest/command_line.html#cmdoption-mypy-ignore-missing-imports) switch is passed on to `mypy` checker for now. The effect of this is that one does not need `# type: ignore` for `import zmq`. More information about import processing can be found [here](https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports). This can be changed in a follow-up PR, if it is deemed useful.
  * We are stuck with Python 3.5 until 04/2021 (see https://packages.ubuntu.com/xenial/python3). When Python version is bumped to 3.6+, one can change:
      ```python
      _opcode_instances = []  # type: List[CScriptOp]
      ```
      to
      ```python
      _opcode_instances:List[CScriptOp] = []
      ```
      for type hints that are **not** function parameters and function return types.

  **Useful resources:**

  * https://docs.python.org/3.5/library/typing.html
  * https://www.python.org/dev/peps/pep-0484/

ACKs for top commit:
  fanquake:
    ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c - the type checking is not the most robust (there are things it fails to detect), but I think this is worth adopting (in a limited capacity while we maintain 3.5 compat).
  MarcoFalke:
    ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c fine with me

Tree-SHA512: 21ef213915fb1dec6012f59ef17484e6c9e0abf542a316b63d5f21a7778ad5ebabf8961ef5fc8e5414726c2ee9c6ae07c7353fb4dd337f8fcef5791199c8987a
2024-01-10 12:07:54 -06:00
MarcoFalke
49b1a468c7
Merge #19294: test: refactor: Inline adjust_bitcoin_conf_for_pre_17
fa41b0a6dac7afd77e2b94eca6520ab3d2adc231 pep-8 test/functional/test_framework/util.py (MarcoFalke)
faa841bc979ca306f5ba4d5f7b78fcc427b8e413 test: refactor: Inline adjust_bitcoin_conf_for_pre_17 (MarcoFalke)

Pull request description:

  This removes mental and code complexity as well as attack surface for bikeshedding

ACKs for top commit:
  Sjors:
    utACK fa41b0a6dac7afd77e2b94eca6520ab3d2adc231

Tree-SHA512: 6e3c872e66d98ffaa7aecdfd64aa7dd8fbb51815a8fdaba170ce0772b4c3360084d0ebab4a5feac768ab5df50d04528d7daafc51ba07c15445c1ef94fa3efd34
2024-01-10 12:07:52 -06:00
MarcoFalke
152246f21a
Merge #19178: Make mininode_lock non-reentrant
62068381a3b9c065d81300be79abba7aecfdb41b [tests] Make mininode_lock non-reentrant (John Newbery)
c67c1f2c032a8efa141d776a7e5be58f052159ea [tests] Don't call super twice in P2PTxInvStore.on_inv() (John Newbery)
9d80762fa0931fe553fad241e95bcc1515ef0e95 [tests] Don't acquire mininode_lock twice in wait_for_broadcast() (John Newbery)
edae6075aa3b1169c84b65e76fd48d68242a294e [tests] Only acquire lock once in p2p_compactblocks.py (John Newbery)

Pull request description:

  There's no need for mininode_lock to be reentrant.
  Use a simpler non-recursive lock.

ACKs for top commit:
  MarcoFalke:
    ACK 62068381a3b9c065d81300be79abba7aecfdb41b 😃
  jonatack:
    ACK 62068381a3b9c0

Tree-SHA512: dcbc19e6c986970051705789be0ff7bec70c69cf76d5b468c2ba4cb732883ad512b1de5c3206c2eca41fa3f1c4806999df4cabbf67fc3c463bb817458e59a19c
2024-01-10 12:07:52 -06:00
UdjinM6
c25d9aed80
fix: A set of qdata/qwatch related fixes (#5745)
## Issue being fixed or feature implemented
Fix/tidy up a few `qdata`/`qwatch` related parts, improve performance
for regular non-watching nodes

~based on #5744 atm~

## What was done?
pls see individual commits

## How Has This Been Tested?
run tests

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2024-01-09 21:31:41 -06:00
MarcoFalke
4143e359f7
(Partial) Merge #19725: [RPC] Add connection type to getpeerinfo, improve logs
a512925e19a70d7f6b80ac530a169f45ffaafa1c [doc] Release notes (Amiti Uttarwar)
50f94b34a33c954f6e207f509c93d33267a5c3e2 [rpc] Deprecate getpeerinfo addnode field (Amiti Uttarwar)
df091b9b509f0b10e4315c0bfa2da0cc0c31c22f [refactor] Rename test file to allow any getpeerinfo deprecations. (Amiti Uttarwar)
395acfa83a5436790c1a722a5609ac9d48df235f [rpc] Add connection type to getpeerinfo RPC, update tests (Amiti Uttarwar)
49c10a9ca40967d28ae16dfea9cccc6f3a6624a1 [log] Add connection type to log statement (Amiti Uttarwar)

Pull request description:

  After #19316, we can more directly expose information about the connection type on the `getpeerinfo` RPC. Doing so also makes the existing addnode field redundant, so this PR begins the process of deprecating this field.

  This PR also includes one commit that improves a log message, as both use a shared function to return the connection type as a string.

  Suggested by sdaftuar- https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468001604 & https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468018093

ACKs for top commit:
  jnewbery:
    Code review ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c.
  sipa:
    utACK a512925e19a70d7f6b80ac530a169f45ffaafa1c
  guggero:
    Tested and code review ACK a512925e.
  MarcoFalke:
    cr ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c 🌇
  promag:
    Code review ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c.

Tree-SHA512: 601a7a38aee235ee59aca690784f886dc2ae4e418b2e6422c4b58cd597376c00f74910f66920b08a08a0bec28bf8022e71a1435785ff6ba8a188954261aba78e
2024-01-09 08:15:36 -06:00
Wladimir J. van der Laan
ae97ed8a6f
Merge #19368: test: improve functional tests compatibility with BSD/macOS
3a7e79478ab41af7c53ce14d9fca9815bffe1f73 test: retry when write to a socket fails on macOS (Ivan Metlushko)
8cf9d15b823d91d2a74fc83832fccca2219342c9 test: use pgrep for better compatibility (Ivan Metlushko)

Pull request description:

  Rationale: a few minor changes to make experience of running tests on macOS a bit better
  1.`pidof` is not available on BSD/macOS, while `pgrep` is present on BSD, Linux and macOS
  2. Add retry as a workaround for a weird behavior when writing to a socket (https://bugs.python.org/issue33450). Stacktrace attached

  Man pages:
  https://www.freebsd.org/cgi/man.cgi?query=pgrep&apropos=0&sektion=1&manpath=FreeBSD+6.0-RELEASE&arch=default&format=html
  https://man7.org/linux/man-pages/man1/pgrep.1.html

  Related to #19281

  Stacktrace example:
  ```
  ...
  33/161 - feature_abortnode.py failed, Duration: 63 s

  stdout:
  2020-06-11T10:46:43.947000Z TestFramework (INFO): Initializing test directory /var/folders/2q/d5w9zh614r7g5c8r74ln3g400000gq/T/test_runner_₿_🏃_20200611_174102/feature_abortnode_128
  2020-06-11T10:46:45.199000Z TestFramework (INFO): Waiting for crash
  2020-06-11T10:47:15.921000Z TestFramework (INFO): Node crashed - now verifying restart fails
  2020-06-11T10:47:47.068000Z TestFramework (INFO): Stopping nodes
  [node 1] Cleaning up leftover process

  stderr:
  Traceback (most recent call last):
    File "/Users/xxx/Projects/bitcoin/test/functional/feature_abortnode.py", line 50, in <module>
      AbortNodeTest().main()
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_framework.py", line 142, in main
      exit_code = self.shutdown()
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_framework.py", line 266, in shutdown
      self.stop_nodes()
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_framework.py", line 515, in stop_nodes
      node.stop_node(wait=wait)
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_node.py", line 318, in stop_node
      self.stop(wait=wait)
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/authproxy.py", line 142, in __call__
      response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/authproxy.py", line 107, in _request
      self.__conn.request(method, path, postdata, headers)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 1107, in request
      self._send_request(method, url, body, headers)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 1152, in _send_request
      self.endheaders(body)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 1103, in endheaders
      self._send_output(message_body)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 936, in _send_output
      self.send(message_body)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 908, in send
      self.sock.sendall(data)
  OSError: [Errno 41] Protocol wrong type for socket
  ```

ACKs for top commit:
  laanwj:
    ACK 3a7e79478ab41af7c53ce14d9fca9815bffe1f73

Tree-SHA512: fefbe40ce94ab29f18bbbed2a434194b1384ffa5279b1d04db7a3708e3dd422bd9e450f1db3f95a1a851fac5a626ab533c6ebcfd7ede96f8ccae9e6f3e9fff92
2024-01-06 19:30:15 -06:00
MarcoFalke
2d95e9284c
Merge #19014: test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option
fad798be76dd5e330463c837fda768477d536078 test: Default --previous-releases to false if dir is empty (MarcoFalke)
faf1c3cc58d14f86ba5364e6ee5c8ef29cac2e26 test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option (MarcoFalke)

Pull request description:

  The "auto-detection" feature is kept in place, but making it an option allows to properly document it. For example, on my machine I get:

  ```
  $ ./test/functional/wallet_disable.py --help | grep previous-releases
    --previous-releases   Force test of previous releases (default: False)

ACKs for top commit:
  Sjors:
    re-tACK fad798b

Tree-SHA512: a7377d0d5378be0a50be278d76396cc403583617b5fc43467773eee706df698acf3f4e67651491183b9b43a8e1816b052e4c17b90272b7ec4b6ac134ad811400
2024-01-06 19:30:14 -06:00
Konstantin Akimov
cf12572c67
fix: withdrawal (asset unlock) txes to use Platform Quorum on RegTest (#5800)
## Issue being fixed or feature implemented
Asset Unlock tx uses platform's quorum on devnets, testnet, mainnet, but
still quorum type "Test (100)" on Reg Tests
That's part II PR, prior work is here:
https://github.com/dashpay/dash/pull/5618

## What was done?
- Removed `consensus.llmqTypeAssetLocks` which has been kept only for
RegTest - use `consensus.llmqTypePlatform` instead.
- Functional test `feature_asset_locks.py` uses `llmq_type_test = 106`
instead `llmq_type_test = 100` for asset unlock tx
- there's 4 MNs + 3 evo nodes instead 3 MNs as before: evo nodes
requires to have IS to be active


## How Has This Been Tested?
Run unit/functional tests


## Breaking Changes
Asset Unlock tx uses correct quorum "106 llmq_test_platform" on reg test
instead "100 llmq_test"

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
2024-01-06 19:28:47 -06:00
UdjinM6
25111262cd
fix: ignore triggers from the past when voting (#5798)
## Issue being fixed or feature implemented
we should not vote on triggers from the past

## What was done?

## How Has This Been Tested?
n/a

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2024-01-06 19:27:26 -06:00
MarcoFalke
7d601cfa85
Merge bitcoin/bitcoin#23681: test: Remove false coinstatsindex test
c055f6b216659b844c8dcd4ff2a977f181099678 test: Remove false coinstatsindex test (Fabian Jahr)

Pull request description:

  This test never actually tested the behavior that it describes in the comments. This was discovered in #21590 which seems to speed up muhash which lead to the test failing.

  I can vaguely remember that the described behavior was desired by some reviewers of `coinstatsindex`: That `coinstatsindex` should be aware of stale blocks and able to return statistics on them as well. The index actually does this for blocks that it sees while the index is active, i.e. while running `coinstatsindex` all blocks will be indexed and even when they become stale the index (via `gettxoutsetinfo`) will still return a result for them when given the right hash. But this currently does not work for blocks that the node saw and that became stale _before_ the node activated `coinstatsindex`. While the index syncs initially everything but the active chain is ignored and I don't see any indication that this ever worked differently in the past.

  Introducing this behavior seems non-trivial at first glance so, while I will give this a shot, I think the test should be removed so it does not confuse users and does not block #21590.

Top commit has no ACKs.

Tree-SHA512: b05f5dfeea3e453c8bb7c761501d0d896d4412a3f0c08037955951fae9fe388c63402da401792591e18da8fb67734f47f1a297d573cdb66e0ced451698718067
2023-12-26 22:26:20 -06:00
MarcoFalke
d2b9631a90
Merge bitcoin/bitcoin#22492: wallet: Reorder locks in dumpwallet to avoid lock order assertion
9b85a5e2f7e003ca8621feaac9bdd304d19081b4 tests: Test for dumpwallet lock order issue (Andrew Chow)
25d99e6511d8c43b2025a89bcd8295de755346a7 Reorder dumpwallet so that cs_main functions go first (Andrew Chow)

Pull request description:

  When a wallet is loaded which has an unconfirmed transaction in the mempool, it will end up establishing the lock order of cs_wallet -> cs_main -> cs_KeyStore. If `dumpwallet` is used on this wallet, then a lock order of cs_wallet -> cs_KeyStore -> cs_main will be used, which causes a lock order assertion. This PR fixes this by reordering `dumpwallet` and `GetKeyBirthTimes` (only used by `dumpwallet`). Specifically, in both functions, the function calls which lock cs_main are done prior to locking cs_KeyStore. This avoids the lock order issue.

  Additionally, I have added a test case to `wallet_dump.py`. Of course testing this requires `--enable-debug`.

  Fixes #22489

ACKs for top commit:
  MarcoFalke:
    review ACK 9b85a5e2f7e003ca8621feaac9bdd304d19081b4 🎰
  ryanofsky:
    Code review ACK 9b85a5e2f7e003ca8621feaac9bdd304d19081b4. Nice to reduce lock scope, and good test!
  prayank23:
    tACK 9b85a5e2f7
  lsilva01:
    Tested ACK 9b85a5e2f7 under the same conditions reported in issue #22489 and the `dumpwallet` command completed successfully.

Tree-SHA512: d370a8f415ad64ee6a538ff419155837bcdbb167e3831b06572562289239028c6b46d80b23d227286afe875d9351f3377574ed831549ea426fb926af0e19c755
2023-12-24 11:59:46 -06:00
MarcoFalke
9e5ee6ac52
Merge #20316: test: Fix wallet_multiwallet test issue on Windows
fa00ff0399fe54f42d80daa04140d19bcfe0e2d8 test: Fix wallet_multiwallet test issue on Windows (MarcoFalke)

Pull request description:

  Fixes:

  ```
  Traceback (most recent call last):
    File "test\functional\test_framework\test_framework.py", line 126, in main
      self.run_test()
    File "test/functional/wallet_multiwallet.py", line 120, in run_test
      assert_equal(sorted(map(lambda w: w['name'], self.nodes[0].listwalletdir()['wallets'])), sorted(in_wallet_dir))
    File "test\functional\test_framework\util.py", line 49, in assert_equal
      raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
  AssertionError: not(['', 'sub\\w5', 'w', 'w1', 'w2', 'w3', 'w7', 'w7_symlink', 'w8'] == ['', 'sub/w5', 'w', 'w1', 'w2', 'w3', 'w7', 'w7_symlink', 'w8'])

ACKs for top commit:
  promag:
    ACK fa00ff0399fe54f42d80daa04140d19bcfe0e2d8.

Tree-SHA512: 7a809a352677a216465cef59e866e4881272e302e897cebf7d9645bf87aebeaf54435bb0692bb5c1381c2dd680e8a34e640ea18ca6e2a4087e3233cd9c24ed04
2023-12-24 11:59:43 -06:00
Odysseas Gabrielides
25cef45858
feat(rpc): gettxchainlocks should return mempool=false when tx not in mempool (#5742)
## Issue being fixed or feature implemented
Platform (in the scope of Withdrawals) need to be aware if a tx isn't in
mempool when requesting status of a tx using RPC `gettxchainlocks`.
cc @markin-io

## What was done?

- mempool is passed to `GetTransaction` and saving the result for
checking latter.
- If the returned tx_ref is nullptr, then the RPC returns null for the
corresponding tx in the array.

Example: 
`tx1` is mined and chainlocked, `tx2` is in mempool and `tx3` doesn't
exist.
The result now is:
`[
  {
    "height": 830,
    "chainlock": false,
    "mempool": true
  },
  {
    "height": -1,
    "chainlock": false,
    "mempool": true
  },
  {
    "height": -1,
    "chainlock": false,
    "mempool": false
  }
]`

## How Has This Been Tested?


## Breaking Changes


## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_

---------

Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2023-12-24 11:58:14 -06:00
Odysseas Gabrielides
563cc34b4e
feat(rpc): Asset Unlock status by index (#5776)
## Issue being fixed or feature implemented
Platform in the scope of credit withdrawals, need a way to get the
status of an Asset Unlock by index.

## What was done?
A new RPC was created `getassetunlockchainlocks` that accepts Asset
Unlock indexes array as parameter and return corresponding status for
each index.

The possible outcomes per each index are:
- `chainlocked`: If the Asset Unlock index is mined on a Chainlocked
block.
- `mined`: If no Chainlock information is available, and the Asset
Unlock index is mined.
- `mempooled`: If the Asset Unlock index is in the mempool.
- `unknown`: If none of the above are valid.

Note: This RPC is whitelisted for the Platform RPC user.

## How Has This Been Tested?
Inserted on `feature_asset_locks.py` covering cases where Asset Unlock
txs are in mempool, mined and not present.

## Breaking Changes
no

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_

---------

Co-authored-by: thephez <thephez@users.noreply.github.com>
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: pasta <pasta@dashboost.org>
2023-12-22 14:27:00 -06:00
UdjinM6
647f4831c1
chore: update file permissions in tests, add missing executable flag (#5778)
## Issue being fixed or feature implemented
old mode 100644
new mode 100755

## What was done?
`chmod +x test/functional/*.py`

## How Has This Been Tested?
can now run these test directly e.g. `./test/functional/rpc_quorum.py`

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-12-19 08:03:11 -06:00
UdjinM6
9a99a4abdc
fix(rpc): pass blockhash into TxToJSON so that getspecialtxes could show correct instantlock/chainlock values (#5774)
## Issue being fixed or feature implemented
`instantlock` and `chainlock` are broken in `getspecialtxes`

kudos to @thephez for finding the issue

## What was done?
pass the hash and also rename the variable to self-describing

## How Has This Been Tested?
run `getspecialtxes` on a node with and without the patch

## Breaking Changes
`instantlock` and `chainlock` will show actual values and not just
`false` all the time now (not sure if that qualifies for "breaking"
though)

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-12-19 07:43:36 -06:00
Odysseas Gabrielides
3bc77a6e1d
feat(rpc): submit chainlock signature if needed RPC (#5765)
## Issue being fixed or feature implemented
Once Platform is live, there could be an edge case where the CL could
arrive to an EvoNode faster through Platform quorum than regular P2P
propagation.

## What was done?
This PR introduces a new RPC `submitchainlock` with the following 3
mandatory parameters:
- `blockHash`, `signature` and `height`.

Besides some basic tests:
- If the block is unknown then the RPC returns an error (could happen if
the node is stucked)
- If the signature is not verified then the RPC return an error.
- If the node already has this CL, the RPC returns true.
- If the node doesn't have this CL, it inserts it, broadcast it through
the inv system and return true.

## How Has This Been Tested?
`feature_llmq_chainlocks.py` was modified with the following scenario:

1. node0 is isolated from the rest of the network
2. node1 mines a new block and waits for CL
3. Make sure node0 doesn't know the new block/CL (by checking
`getbestchainlock()`)
4. CL is submitted via the new RPC on node0
5. checking `getbestchainlock()` and make sure the CL was processed +
'known_block' is false
6. reconnect node0

## Breaking Changes
no

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_

---------

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: thephez <thephez@users.noreply.github.com>
2023-12-18 22:27:19 -06:00
MarcoFalke
8b98e1ab7b Merge bitcoin/bitcoin#18795: Test: wallet issue with orphaned rewards
e4356f6a6c18e5027a064a4d3a5deda27985f584 Testcase for wallet issue with orphaned rewards. (Daniel Kraft)

Pull request description:

  This adds a new test case demonstrating the wallet issue when block rewards are orphaned (#14148).

ACKs for top commit:
  LarryRuane:
    ACK e4356f6a6c18e5027a064a4d3a5deda27985f584
  leonardojobim:
    reACK e4356f6a6c .

Tree-SHA512: e9a2310ee1b3d52cfa302f431ed3d272bbc1b9195439ff318d9eb1006c0b28968dbe840e1600b6ff185e5d7ea57e4dcc837cef16051b5537445e10bc363b8c22
2023-12-08 21:16:00 +03:00
MarcoFalke
b8267bb5a2 Merge #17556: test: Change feature_config_args.py not to rely on strange regtest=0 behavior
ff44cae279bef7997f76db18deb1e41b39f05cb6 test: Change feature_config_args.py not to rely on strange regtest=0 behavior (Russell Yanofsky)

Pull request description:

  Update test to simply generate a normal mainnet configuration file instead of using a crazy setup where a regtest=1 config file using an includeconf in the [regtest] section includes another config file that specifies regtest=0, retroactively switching the network to mainnet.

  This setup was fragile and only worked because the triggered InitError happened early enough that none of the ignored [regtest] options mattered (only affecting log output).

  This change was originally made as part of #17493

Top commit has no ACKs.

Tree-SHA512: 3f77305454f04438493dfc2abd78a00434b30869454d1c3f54587b9c1f63239c49c90fb3b4d3a777ad130f2184e0f2dac87cee4cd23c50f1b3496a375943da01
2023-12-08 21:16:00 +03:00
MarcoFalke
cff1c7b2c3 Merge #21043: net: Avoid UBSan warning in ProcessMessage(...)
3ddbf22ed179a2db733af4b521bec5d2b13ebf4b util: Disallow negative mocktime (MarcoFalke)
f5f2f9716885e7548809e77f46b493c896a019bf net: Avoid UBSan warning in ProcessMessage(...) (practicalswift)

Pull request description:

  Avoid UBSan warning in `ProcessMessage(...)`.

  Context: https://github.com/bitcoin/bitcoin/pull/20380#issuecomment-770427182 (thanks Crypt-iQ!)

ACKs for top commit:
  MarcoFalke:
    re-ACK 3ddbf22ed179a2db733af4b521bec5d2b13ebf4b only change is adding patch written by me
  ajtowns:
    ACK 3ddbf22ed179a2db733af4b521bec5d2b13ebf4b -- code review only

Tree-SHA512: e8d7af0457ca86872b75a4e406c0a93aafd841c2962e244e147e748cc7ca118c56be0fdafe53765f4b291410030b2c3cc8f76f733b37a955d34fc885ab6037b9
2023-12-08 21:16:00 +03:00
Wladimir J. van der Laan
3f77d2312f Merge #16525: Dump transaction version as an unsigned integer in RPC/TxToUniv
e80259f1976545e4f1ab6a420644be0c32261773 Additionally treat Tx.nVersion as unsigned in joinpsbts (Matt Corallo)
970de70bdd3542e75b73c79b06f143168c361494 Dump transaction version as an unsigned integer in RPC/TxToUniv (Matt Corallo)

Pull request description:

  Consensus-wise we already treat it as an unsigned integer (the
  only rules around it are in CSV/locktime handling), but changing
  the underlying data type means touching consensus code for a
  simple cleanup change, which isn't really worth it.

  See-also, https://github.com/rust-bitcoin/rust-bitcoin/pull/299

ACKs for top commit:
  sipa:
    ACK e80259f1976545e4f1ab6a420644be0c32261773
  practicalswift:
    ACK e80259f1976545e4f1ab6a420644be0c32261773
  ajtowns:
    ACK e80259f1976545e4f1ab6a420644be0c32261773 code review -- checked all other uses of tx.nVersion treat it as unsigned (except for policy.cpp:IsStandard anyway), so looks good.
  naumenkogs:
    ACK e80259f

Tree-SHA512: 6760a2c77e24e9e1f79a336ca925f9bbca3a827ce02003c71d7f214b82ed3dea13fa7d9f87df9b9445cd58dff8b44a15571d821c876f22f8e5a372a014c9976b
2023-12-06 12:33:15 -06:00
MarcoFalke
a800821e9f Merge #18965: tests: implement base58_decode
60ed33904cf974e8f3c1b95392a23db1fe2d4a98 tests: implement base58_decode (10xcryptodev)

Pull request description:

  implements TODO: def base58_decode

ACKs for top commit:
  ryanofsky:
    Code review ACK 60ed33904cf974e8f3c1b95392a23db1fe2d4a98. Just suggested changes since last review. Thank you for taking suggestions!

Tree-SHA512: b3c06b4df041a6d88033cd077a093813a688e42d0b9aa777c715e5fd69cfba7b1bf984428bd98417d3c15232d3d48bc9c163317564f9e1d562db6611c21e2c10
2023-12-06 12:33:15 -06:00
Wladimir J. van der Laan
e29a35a997 Merge #18309: zmq: Add support to listen on multiple interfaces
e66870c5a4c2adbd30dca67d409fd5cd98697587 zmq: Append address to notify log output (nthumann)
241803da211265444e65f254f24dd184f2457fa9 test: Add zmq test to support multiple interfaces (nthumann)
a0b2e5cb6aa8db0563fac7d67a949b9baefe3a25 doc: Add release notes to support multiple interfaces (nthumann)
b1c3f180ecb63f3960506d202feebaa4271058ae doc: Adjust ZMQ usage to support multiple interfaces (nthumann)
347c94f551c3f144c44e00373e4dd61ff6d908b7 zmq: Add support to listen on multiple interfaces (Nicolas Thumann)

Pull request description:

  This PR adds support for ZeroMQ to listen on multiple interfaces, just like the RPC server.
  Currently, if you specify more than one e.g. `zmqpubhashblock` paramter, only the first one will be used. Therefore a user may be forced to listen on all interfaces (e.g. `zmqpubhashblock=0.0.0.0:28332`), which can result in an increased attack surface.
  With this PR a user can specify multiple interfaces to listen on, e.g.
  `-zmqpubhashblock=tcp://127.0.0.1:28332 -zmqpubhashblock=tcp://192.168.1.123:28332`.

ACKs for top commit:
  laanwj:
    Code review ACK e66870c5a4c2adbd30dca67d409fd5cd98697587
  instagibbs:
    reACK e66870c5a4

Tree-SHA512: f38ab4a6ff00dc821e5f4842508cefadb701e70bb3893992c1b32049be20247c8aa9476a1f886050c5f17fe7f2ce99ee30193ce2c81a7482a5a51f8fc22300c7
2023-12-06 12:33:15 -06:00
UdjinM6
8a888fb6ee
fix: improve qgetdata/qdata tests (#5744)
## Issue being fixed or feature implemented
The test is a bit broken and incomplete, some testing scenarios aren't
realistic

## What was done?
pls see individual commits

## How Has This Been Tested?
run it

## Breaking Changes
n/a, tests only

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-12-06 11:48:21 -06:00
MarcoFalke
f7f29d72fa follow-up Merge #14559: appveyor: Enable multiwallet tests - adds missing changes for wallet_multiwallet.py test
4dca7d0a98 appveyor: Enable multiwallet test (Chun Kuan Lee)

Pull request description:

  Based on #14320

  This PR enable multiwallet test on appveyor. Also re-enable symlink tests on Windows which is available after Windows Vista.

  I disable these tests in #13964 because I suppose that Windows does not support symlink, but I was wrong.

Tree-SHA512: 852cd4dedf36ec9c34aff8926cb34e6a560aea0bb9170c7a2264fc292dbb605622d561568d8df39aeb90d3d2bb700901d218ea7e7c5e21d84827c40d6370b369
2023-12-06 11:46:53 -06:00
Samuel Dobson
f6f9b9851f Merge #17219: wallet: allow transaction without change if keypool is empty
92bcd70808b9cac56b184903aa6d37baf9641b37 [wallet] allow transaction without change if keypool is empty (Sjors Provoost)
709f8685ac37510aa145ac259753583c82280038 [wallet] CreateTransaction: simplify change address check (Sjors Provoost)
5efc25f9638866941028454cfa9bae27f1519cb4 [wallet] translate "Keypool ran out" message (Sjors Provoost)

Pull request description:

  Extracted from #16944

  First this PR simplifies the check when generating a change address, by dropping `CanGetAddresses` and just letting `reservedest.GetReservedDestination` do this check.

  Second, when the keypool is empty, instead of immediately giving up, we create a dummy change address and pass that to coin selection. If we didn't need the change address (e.g. when spending the entire balance), then it's all good. If we did need a change address, we throw the original error.

ACKs for top commit:
  fjahr:
    Code review ACK 92bcd70808b9cac56b184903aa6d37baf9641b37
  jonasschnelli:
    utACK 92bcd70808b9cac56b184903aa6d37baf9641b37
  achow101:
    ACK 92bcd70808b9cac56b184903aa6d37baf9641b37
  meshcollider:
    Code review ACK 92bcd70808b9cac56b184903aa6d37baf9641b37

Tree-SHA512: 07b8c8251f57061c58a85ebf0359be63583c23bac7a2c4cefdc14820c0cdebcc90a2bb218e5ede0db11d1e204cda149e056dfd18614642070b3d56efe2735006
2023-12-06 11:46:53 -06:00
fanquake
0eba155aa7 Merge #19072: doc: Expand section on Getting Started
facef3d4131f9980a4516282f11731361559509c doc: Explain that anyone can work on good first issues, move text to CONTRIBUTING.md (MarcoFalke)
fae2fb2a196ee864e9a13fffc24a0279cd5d17e6 doc: Expand section on Getting Started (MarcoFalke)
100000d1b2c2e38d7a14a31b0af79e0e4316b04c doc: Add headings to CONTRIBUTING.md (MarcoFalke)
fab893e0caf510d4836a20194892ef9c71426c51 doc: Fix unrelated typos reported by codespell (MarcoFalke)

Pull request description:

  Some random doc changes:

  * Add sections to docs, so that they can be linked to
  * Explain that anyone (even maintainers) are allowed to work on good first issues
  * Expand section on Getting Started slightly

ACKs for top commit:
  hebasto:
    ACK facef3d4131f9980a4516282f11731361559509c
  fanquake:
    ACK facef3d4131f9980a4516282f11731361559509c

Tree-SHA512: 8998e273a76dbf4ca77e79374c14efe4dfcc5c6df6b7d801e1e1e436711dbe6f76b436f9cbc6cacb45a56827babdd6396f3bd376a9426ee7be3bb9b8a3b8e383
2023-12-06 11:40:14 -06:00
fanquake
2da9982e55 Merge #17829: scripted-diff: Bump copyright of files changed in 2019
aaaaad6ac95b402fe18d019d67897ced6b316ee0 scripted-diff: Bump copyright of files changed in 2019 (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK aaaaad6ac95b402fe18d019d67897ced6b316ee0
  promag:
    ACK aaaaad6ac95b402fe18d019d67897ced6b316ee0 🎉
  fanquake:
    ACK aaaaad6ac95b402fe18d019d67897ced6b316ee0 - going to merge this now because the year is over and conflicts are minimal.

Tree-SHA512: 58cb1f53bc4c1395b2766f36fabc7e2332e213780a802762fff0afd59468dad0c3265f553714d761c7a2c44ff90f7dc250f04458f4b2eb8eef8b94f8c9891321
2023-12-06 11:40:14 -06:00
MarcoFalke
921c93db66 Merge #16973: test: Fix combine_logs.py for AppVeyor build
d478a472eb0d666e8a762ed8d24fafbabc5f94f3 test: Fix combine_logs.py for AppVeyor build (Martin Zumsande)

Pull request description:

  Fixes #16894

  This fixes the problem of AppVeyor builds not showing `debug.log` if a functional test fails, because the windows separator `\` doesn't work together with the regex in `combine_logs.py`.

  A fix was already attempted in  #16896, however, that PR became inactive and was marked "up for grabs", plus it's a really small change.

  As suggested by jamesob, this PR uses `pathlib`: For the glob and to convert the path to a posix-style string, it leaves the regex as is (in contrast to #16896 which adjusted the regex).

  I tested this locally on Windows and Ubuntu.

Top commit has no ACKs.

Tree-SHA512: 603b4359b6009b6da874c30f69759acda03730ee5747898a0fe957a5fc37ee9ba07858c6aa2169bf4c40521f37e47138e8314d698652ea2760fa0a3f76b890bd
2023-12-06 11:40:14 -06:00
MarcoFalke
9d4282cb72 partial Merge #15893: Add test for superfluous witness record in deserialization
cc556e4a30 Add test for superfluous witness record in deserialization (Gregory Sanders)
25b0786581 Fix missing input template by making minimal tx (Gregory Sanders)

Pull request description:

  Adds coverage for changed behavior in https://github.com/bitcoin/bitcoin/pull/14039

ACKs for commit cc556e:
  MarcoFalke:
    utACK cc556e4a30b4a32eab6722f590489d89b2875de3

Tree-SHA512: 3404c8f75e87503983fac5ae27d877309eb3b902f2ec993762911c71610ca449bef0ed98bd17e029414828025b2713e1bd012e63b2a06497e34f1056acaa6321
2023-12-06 11:40:14 -06:00
MarcoFalke
549a358fe2 Merge #15102: test: Run invalid_txs.InputMissing test in feature_block
fac4e731a8 test: Run invalid_txs.InputMissing test in feature_block (MarcoFalke)

Pull request description:

Tree-SHA512: 24c3f519ba0cf417b66e0df6f5ddc0430e3f419af4705a9c85096da47ff4d8f51487d65b68f3f993800003b3f936d95d8a0bade846e1b45f95b2bdbecc9ebab7
2023-12-06 11:40:14 -06:00
MarcoFalke
56af7d6727
Merge bitcoin/bitcoin#22313: test: Add missing sync_all to feature_coinstatsindex
fafd9165e911bf33d6212ca8a613b71878c82449 test: Add missing sync_all to feature_coinstatsindex (MarcoFalke)

Pull request description:

  Sync the blocks before invalidating them to ensure all nodes are on the right tip. Otherwise nodes[0] might stay on the "stale" block and the test fails (intermittently)

ACKs for top commit:
  jamesob:
    crACK fafd9165e9

Tree-SHA512: ca567b97b839b56c91d52831eaac18d8c843d376be90c9fd8b49d2eb4a46b801a1d2402996d5dfe2bef3e2c9bd75d19ed443e3f42cc4679c5f20043ba556efc8
2023-12-03 20:45:01 -06:00
MarcoFalke
ec4cbc28d6
Merge #20954: test: Declare nodes type in test_framework.py.
5353b0c64d32e44fc411464e080d4b00fae7124e Change type definitions for "chain" and "setup_clean_chain" from type comments to Python 3.6+ types. Additionally, set type for "nodes". (Kiminuo)

Pull request description:

  ### Motivation

  When I wanted to understand better https://github.com/bitcoin/bitcoin/pull/19145/files#diff-4bebbd3b112dc222ea7e75ef051838ceffcee63b9e9234a98a4cc7251d34451b test, I noticed that navigation in PyCharm/VS Code did not work for `nodes` variable. I think this is frustrating, especially for newcomers.

  ### Summary

  * This PR modifies Python 3.5 [type comments](https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html#variables) to Python 3.6+ types and adds a proper type for `nodes` [instance attribute](https://mypy.readthedocs.io/en/stable/class_basics.html#instance-and-class-attributes).
  * This PR does not change behavior.
  * This PR is intentionally very small, if the concept is accepted, a follow-up PRs can be more ambitious.

  ### End result

  1. Open `test/functional/feature_abortnode.py`
  2. Move your caret to: `self.nodes[0].generate[caret here](3)`
  3. Use "Go to definition" [F12] should work now.

  I have tested this on PyCharm (Windows, Ubuntu) and VS Code (Windows, Ubuntu).

  Note: Some `TestNode` methods (e.g. `self.nodes[0].getblock(...)` ) use `__call__` mechanism and navigation does not work for them even with this PR.

ACKs for top commit:
  laanwj:
    ACK 5353b0c64d32e44fc411464e080d4b00fae7124e
  theStack:
    ACK 5353b0c64d32e44fc411464e080d4b00fae7124e

Tree-SHA512: 821773f052ab9b2889dc357d38c59407a4af09e3b86d7134fcca7d78e5edf3a5ede9bfb37595ea97caf9ebfcbda372bcf73763b7f89b0677670f21b3e396a12b
2023-12-03 20:44:55 -06:00
MarcoFalke
7fbc0f359c Merge bitcoin/bitcoin#14604: tests: Add test and refactor feature_block.py
55311197c483477b79883da5da09f2bc71acc7cf Added new test for future blocks reacceptance (sanket1729)
511a5af4622915c236cfb11df5234232c2983e45 Fixed inconsistencies between code and comments (sanket1729)

Pull request description:

  This Commit does 3 things:
  1) Adds a test case for checking reacceptance a previously rejected block which
  was too far in the future.
  ~~2) clean up uses of rehash or calc_sha256 where it was not needed~~
  3) While constructing block 44, this commit makes the code consistent with the expected figure in
  the comment just above it by adding a transaction to the block.
  4) Fix comment describing `sign_tx()` function

ACKs for top commit:
  duncandean:
    reACK 5531119
  brunoerg:
    reACK 55311197c483477b79883da5da09f2bc71acc7cf

Tree-SHA512: d40c72fcdbb0b2a0715adc58441eeea08147ee2ec5e371a4ccc824ebfdc6450698bd40aaeecb7ea7bfdb3cd1b264dd821b890276fff8b8d89b7225cdd9d6b546
2023-12-03 20:32:22 -06:00
MarcoFalke
d667e57dc6 Merge #20466: test: Fix intermittent p2p_fingerprint issue
fad7be584ffaf8099cc099d9378ba831c9483260 test: Fix intermittent p2p_finerprint issue (MarcoFalke)

Pull request description:

  A single sync_with_ping can't be used to drop a block announcement, as the block might be sent *after* the ping has been responded to.

  Fix that by waiting for the block.

ACKs for top commit:
  theStack:
    ACK fad7be584ffaf8099cc099d9378ba831c9483260

Tree-SHA512: d43ba9d07273486858f65a26326cc6637ef743bf7b400e5048ba7eac266fb1893283e6503dd49f179caa1abab2977315fb70ba9fba34be9a817a74259d8e4034
2023-12-03 20:32:22 -06:00
MarcoFalke
f4ea109e65 (partial) Merge bitcoin/bitcoin#21562: [net processing] Various tidying up of PeerManagerImpl ctor
fde1bf4f6136638e84cdf9806eedaae08e841bbf [net processing] Default initialize m_recent_confirmed_transactions (John Newbery)
37dcd12d539e4a875581fa049aa0f7fafeb932a4 scripted-diff: Rename recentRejects (John Newbery)
cd9902ac5054c01228d52616bf85f7196364d4ff [net processing] Default initialize recentRejects (John Newbery)
a28bfd1d4cfa523a6abf3832dbfd6183cd546944 [net processing] Default initialize m_stale_tip_check_time (John Newbery)
9190b01d8dcf03b74e9b9e1653688a97ac171b37 [net processing] Add Orphanage empty consistency check (John Newbery)

Pull request description:

  - Use default initialization of PeerManagerImpl members where possible
  - Remove unique_ptr indirection where it's not needed

ACKs for top commit:
  MarcoFalke:
    ACK fde1bf4f6136638e84cdf9806eedaae08e841bbf 👞
  theStack:
    re-ACK fde1bf4f6136638e84cdf9806eedaae08e841bbf

Tree-SHA512: 7ddedcc972df8e933e1fbe5c88b8ea17df89e1e58fc769518512c5540e49dc8eddb3f47e78d1329a6fc5644d2c1d11c981f681fd633f5218bfa4b3e6a86f3d7b
2023-12-03 20:25:16 -06:00
MarcoFalke
42aea04810 Merge bitcoin/bitcoin#22530: log: sort logging categories alphabetically
d596dba9877e7ead3fb5426cbe7e608fbcbfe3eb test: assert logging categories are sorted in rpc and help (Jon Atack)
17bbff3b88132c0c95b29b59100456b85e26df75 log, refactor: use guard clause in LogCategoriesList() (Jon Atack)
7c57297319bc386afaf06528778384fe58576ef9 log: sort LogCategoriesList and LogCategoriesString alphabetically (Jon Atack)
f720cfa824f1be863349e7016080f8fb1c3c76c2 test: verify number of categories returned by logging RPC (Jon Atack)

Pull request description:

  Sorting the logging categories seems more user-friendly with the number of categories we now have, allowing CLI users to more quickly find a particular category.

  before
  ```
  $ bitcoin-cli help logging
  ...
  The valid logging categories are: net, tor, mempool, http, bench, zmq, walletdb, rpc, estimatefee, addrman, selectcoins, reindex, cmpctblock, rand, prune, proxy, mempoolrej, libevent, coindb, qt, leveldb, validation, i2p, ipc

  $ bitcoind -h | grep -A8 "debug=<category>"
    -debug=<category>
         ...
         output all debugging information. <category> can be: net, tor,
         mempool, http, bench, zmq, walletdb, rpc, estimatefee, addrman,
         selectcoins, reindex, cmpctblock, rand, prune, proxy, mempoolrej,
         libevent, coindb, qt, leveldb, validation, i2p, ipc.

  $ bitcoin-cli logging [] '["addrman"]'
  {
    "net": false,
    "tor": true,
    "mempool": false,
    "http": false,
    "bench": false,
    "zmq": false,
    "walletdb": false,
    "rpc": false,
    "estimatefee": false,
    "addrman": false,
    "selectcoins": false,
    "reindex": false,
    "cmpctblock": false,
    "rand": false,
    "prune": false,
    "proxy": true,
    "mempoolrej": false,
    "libevent": false,
    "coindb": false,
    "qt": false,
    "leveldb": false,
    "validation": false,
    "i2p": true,
    "ipc": false
  }
  ```

  after

  ```
  $ bitcoin-cli help logging
  ...
  The valid logging categories are: addrman, bench, cmpctblock, coindb, estimatefee, http, i2p, ipc, leveldb, libevent, mempool, mempoolrej, net, proxy, prune, qt, rand, reindex, rpc, selectcoins, tor, validation, walletdb, zmq

  $ bitcoind -h | grep -A8 "debug=<category>"
    -debug=<category>
         ...
         output all debugging information. <category> can be: addrman,
         bench, cmpctblock, coindb, estimatefee, http, i2p, ipc, leveldb,
         libevent, mempool, mempoolrej, net, proxy, prune, qt, rand,
         reindex, rpc, selectcoins, tor, validation, walletdb, zmq.

  $ bitcoin-cli logging [] '["addrman"]'
  {
    "addrman": false,
    "bench": false,
    "cmpctblock": false,
    "coindb": false,
    "estimatefee": false,
    "http": false,
    "i2p": false,
    "ipc": false,
    "leveldb": false,
    "libevent": false,
    "mempool": false,
    "mempoolrej": false,
    "net": false,
    "proxy": false,
    "prune": false,
    "qt": false,
    "rand": false,
    "reindex": false,
    "rpc": false,
    "selectcoins": false,
    "tor": false,
    "validation": false,
    "walletdb": false,
    "zmq": false
  }
  ```

ACKs for top commit:
  theStack:
    re-ACK d596dba9877e7ead3fb5426cbe7e608fbcbfe3eb

Tree-SHA512: d546257f562b0a288d1b19a028f1a510aaf21bd21da058e7c84653d305ea8662ecb4647ebefd2b97411f845fe5b0b841d40d3fe6814eefcb8ce82df341dfce22
2023-12-03 20:25:16 -06:00