sikkienl/dash

mirror of https://github.com/dashpay/dash.git synced 2024-12-27 04:52:59 +01:00

Author SHA1 Message Date

Author	SHA1	Message	Date
MacroFake	6731b10288	Merge bitcoin/bitcoin#26007 : [contrib] message-capture-parser: fix AssertionError on parsing `headers` message 644772b9efffda4dac01aff54042b3162079514d message-capture-parser: fix AssertionError on parsing `headers` message (Sebastian Falbesoner) Pull request description: If a test framework message's field name is in the list of `HASH_INT_VECTORS`, we currently assume that it _always_ has to contain a vector of integers and throw otherwise: `0ebd4db32b/contrib/message-capture/message-capture-parser.py (L82-L83)` (introduced in PR #25367, commit 42bbbba7c83d1e2baad18b4c6f05bad1358eb117). However, that assumption is too strict. The (de)serialization field name "headers" is used in two different message types, one for `cfcheckpt` (where it is serialized as an integer vector), and another time for `headers` (where it is serialized as a vector of `CBlockHeader`s). Parsing the latter fails as it is not an integer vector and thus triggers the assert. Fix this by adding the integer type check as additional condition to the `HASH_INT_VECTORS` check rather than asserting. Fixes #25954. ACKs for top commit: glozow: ACK 644772b9efffda4dac01aff54042b3162079514d Tree-SHA512: c98a107f6703c6c1a81771907c25bcc171c631b57fd605fbebaedd93d651e2ef02fb5601853a9bc7d659ab531c5f47770181173a36ea2b37f584aa7a37b66505	2024-02-29 12:33:46 -06:00
MacroFake	da4fd2eaf2	Merge bitcoin/bitcoin#25367 : [contrib] message-capture-parser: fix out of bounds error for empty vectors 42bbbba7c83d1e2baad18b4c6f05bad1358eb117 message-capture-parser: fix out of bounds error for empty vectors (Sebastian Falbesoner) Pull request description: The script [message-capture-parser.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/message-capture/message-capture-parser.py) currently throws an "out of bounds" error if a message containing an empty integer vector element is tried to converted to JSON (e.g. by the BIP157 message `cfcheckpt` with empty `FilterHeaders` vector): ``` Traceback (most recent call last): File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 217, in <module> main() File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 202, in main process_file(str(capture), messages, "recv" in capture.stem, progress_bar) File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 162, in process_file msg_dict["body"] = to_jsonable(msg) File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 85, in to_jsonable elif slot in HASH_INT_VECTORS and isinstance(val[0], int): IndexError: list index out of range ``` Fix this by using the `all(...)` predicate rather to access the first element `val[0]` (which in the error case doesn't exist). ACKs for top commit: laanwj: Code review ACK 42bbbba7c83d1e2baad18b4c6f05bad1358eb117 Tree-SHA512: 139ec6b90304a69f26ec731e6f12b216fa10e554f777505b61adfa1e569f6861a4a849159dd1eae7a1aa0427e8598af226b6f0c4015020dcac8ab109fbc35dba	2024-02-27 10:02:42 -06:00
MarcoFalke	9a92452a5c	Merge #19509 : Per-Peer Message Capture bff7c66e67aa2f18ef70139338643656a54444fe Add documentation to contrib folder (Troy Giorshev) 381f77be858d7417209b6de0b7cd23cb7eb99261 Add Message Capture Test (Troy Giorshev) e4f378a505922c0f544b4cfbfdb169e884e02be9 Add capture parser (Troy Giorshev) 4d1a582549bc982d55e24585b0ba06f92f21e9da Call CaptureMessage at appropriate locations (Troy Giorshev) f2a77ff97bec09dd5fcc043d8659d8ec5dfb87c2 Add CaptureMessage (Troy Giorshev) dbf779d5deb04f55c6e8493ce4e12ed4628638f3 Clean PushMessage and ProcessMessages (Troy Giorshev) Pull request description: This PR introduces per-peer message capture into Bitcoin Core. 📓 ## Purpose The purpose and scope of this feature is intentionally limited. It answers a question anyone new to Bitcoin's P2P protocol has had: "Can I see what messages my node is sending and receiving?". ## Functionality When a new debug-only command line argument `capturemessages` is set, any message that the node receives or sends is captured. The capture occurs in the MessageHandler thread. When receiving a message, it is captured as soon as the MessageHandler thread takes the message off of the vProcessMsg queue. When sending, the message is captured just before the message is pushed onto the vSendMsg queue. The message capture is as minimal as possible to reduce the performance impact on the node. Messages are captured to a new `message_capture` folder in the datadir. Each node has their own subfolder named with their IP address and port. Inside, received and sent messages are captured into two binary files, msgs_recv.dat and msgs_sent.dat, like so: ``` message_capture/203.0.113.7:56072/msgs_recv.dat message_capture/203.0.113.7:56072/msgs_sent.dat ``` Because the messages are raw binary dumps, included in this PR is a Python parsing tool to convert the binary files into human-readable JSON. This script has been placed on its own and out of the way in the new `contrib/message-capture` folder. Its usage is simple and easily discovered by the autogenerated `-h` option. ## Future Maintenance I sympathize greatly with anyone who says "the best code is no code". The future maintenance of this feature will be minimal. The logic to deserialize the payload of the p2p messages exists in our testing framework. As long as our testing framework works, so will this tool. Additionally, I hope that the simplicity of this tool will mean that it gets used frequently, so that problems will be discovered and solved when they are small. ## FAQ "Why not just use Wireshark" Yes, Wireshark has the ability to filter and decode Bitcoin messages. However, the purpose of the message capture added in this PR is to assist with debugging, primarily for new developers looking to improve their knowledge of the Bitcoin Protocol. This drives the design in a different direction than Wireshark, in two different ways. First, this tool must be convenient and simple to use. Using an external tool, like Wireshark, requires setup and interpretation of the results. To a new user who doesn't necessarily know what to expect, this is unnecessary difficulty. This tool, on the other hand, "just works". Turn on the command line flag, run your node, run the script, read the JSON. Second, because this tool is being used for debugging, we want it to be as close to the true behavior of the node as possible. A lot can happen in the SocketHandler thread that would be missed by Wireshark. Additionally, if we are to use Wireshark, we are at the mercy of whoever it maintaining the protocol in Wireshark, both as to it being accurate and recent. As can be seen by the many previous attempts to include Bitcoin in Wireshark (google "bitcoin dissector") this is easier said than done. Lastly, I truly believe that this tool will be used significantly more by being included in the codebase. It's just that much more discoverable. ACKs for top commit: MarcoFalke: re-ACK bff7c66e67aa2f18ef70139338643656a54444fe only some minor changes: 👚 jnewbery: utACK bff7c66e67aa2f18ef70139338643656a54444fe theStack: re-ACK bff7c66e67aa2f18ef70139338643656a54444fe Tree-SHA512: e59e3160422269221f70f98720b47842775781c247c064071d546c24fa7a35a0e5534e8baa4b4591a750d7eb16de6b4ecf54cbee6d193b261f4f104e28c15f47	2024-02-14 10:34:10 -06:00

MacroFake

6731b10288

Merge bitcoin/bitcoin#26007 : [contrib] message-capture-parser: fix AssertionError on parsing headers message

644772b9efffda4dac01aff54042b3162079514d message-capture-parser: fix AssertionError on parsing `headers` message (Sebastian Falbesoner)

Pull request description:

  If a test framework message's field name is in the list of `HASH_INT_VECTORS`, we currently assume that it _always_ has to contain a vector of integers and throw otherwise:
  0ebd4db32b/contrib/message-capture/message-capture-parser.py (L82-L83)
  (introduced in PR #25367, commit 42bbbba7c83d1e2baad18b4c6f05bad1358eb117).

  However, that assumption is too strict. The (de)serialization field name "headers" is used in two different message types, one for `cfcheckpt` (where it is serialized as an integer vector), and another time for `headers` (where it is serialized as a vector of `CBlockHeader`s). Parsing the latter fails as it is not an integer vector and thus triggers the assert.

  Fix this by adding the integer type check as additional condition to the `HASH_INT_VECTORS` check rather than asserting.
  Fixes #25954.

ACKs for top commit:
  glozow:
    ACK 644772b9efffda4dac01aff54042b3162079514d

Tree-SHA512: c98a107f6703c6c1a81771907c25bcc171c631b57fd605fbebaedd93d651e2ef02fb5601853a9bc7d659ab531c5f47770181173a36ea2b37f584aa7a37b66505

2024-02-29 12:33:46 -06:00

MacroFake

da4fd2eaf2

Merge bitcoin/bitcoin#25367 : [contrib] message-capture-parser: fix out of bounds error for empty vectors

42bbbba7c83d1e2baad18b4c6f05bad1358eb117 message-capture-parser: fix out of bounds error for empty vectors (Sebastian Falbesoner)

Pull request description:

  The script [message-capture-parser.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/message-capture/message-capture-parser.py) currently throws an "out of bounds" error if a message containing an empty integer vector element is tried to converted to JSON (e.g. by the BIP157 message `cfcheckpt` with empty `FilterHeaders` vector):
  ```
  Traceback (most recent call last):
    File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 217, in <module>
      main()
    File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 202, in main
      process_file(str(capture), messages, "recv" in capture.stem, progress_bar)
    File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 162, in process_file
      msg_dict["body"] = to_jsonable(msg)
    File "/home/honey/bitcoin/./contrib/message-capture/message-capture-parser.py", line 85, in to_jsonable
      elif slot in HASH_INT_VECTORS and isinstance(val[0], int):
  IndexError: list index out of range
  ```

  Fix this by using the `all(...)` predicate rather to access the first element `val[0]` (which in the error case doesn't exist).

ACKs for top commit:
  laanwj:
    Code review ACK 42bbbba7c83d1e2baad18b4c6f05bad1358eb117

Tree-SHA512: 139ec6b90304a69f26ec731e6f12b216fa10e554f777505b61adfa1e569f6861a4a849159dd1eae7a1aa0427e8598af226b6f0c4015020dcac8ab109fbc35dba

2024-02-27 10:02:42 -06:00

MarcoFalke

9a92452a5c

Merge #19509 : Per-Peer Message Capture

bff7c66e67aa2f18ef70139338643656a54444fe Add documentation to contrib folder (Troy Giorshev)
381f77be858d7417209b6de0b7cd23cb7eb99261 Add Message Capture Test (Troy Giorshev)
e4f378a505922c0f544b4cfbfdb169e884e02be9 Add capture parser (Troy Giorshev)
4d1a582549bc982d55e24585b0ba06f92f21e9da Call CaptureMessage at appropriate locations (Troy Giorshev)
f2a77ff97bec09dd5fcc043d8659d8ec5dfb87c2 Add CaptureMessage (Troy Giorshev)
dbf779d5deb04f55c6e8493ce4e12ed4628638f3 Clean PushMessage and ProcessMessages (Troy Giorshev)

Pull request description:

  This PR introduces per-peer message capture into Bitcoin Core.  📓

  ## Purpose

  The purpose and scope of this feature is intentionally limited.  It answers a question anyone new to Bitcoin's P2P protocol has had: "Can I see what messages my node is sending and receiving?".

  ## Functionality

  When a new debug-only command line argument `capturemessages` is set, any message that the node receives or sends is captured.  The capture occurs in the MessageHandler thread.  When receiving a message, it is captured as soon as the MessageHandler thread takes the message off of the vProcessMsg queue.  When sending, the message is captured just before the message is pushed onto the vSendMsg queue.

  The message capture is as minimal as possible to reduce the performance impact on the node.  Messages are captured to a new `message_capture` folder in the datadir.  Each node has their own subfolder named with their IP address and port.  Inside, received and sent messages are captured into two binary files, msgs_recv.dat and msgs_sent.dat, like so:

  ```
  message_capture/203.0.113.7:56072/msgs_recv.dat
  message_capture/203.0.113.7:56072/msgs_sent.dat
  ```

  Because the messages are raw binary dumps, included in this PR is a Python parsing tool to convert the binary files into human-readable JSON.  This script has been placed on its own and out of the way in the new `contrib/message-capture` folder.  Its usage is simple and easily discovered by the autogenerated `-h` option.

  ## Future Maintenance

  I sympathize greatly with anyone who says "the best code is no code".

  The future maintenance of this feature will be minimal.  The logic to deserialize the payload of the p2p messages exists in our testing framework.  As long as our testing framework works, so will this tool.

  Additionally, I hope that the simplicity of this tool will mean that it gets used frequently, so that problems will be discovered and solved when they are small.

  ## FAQ

  "Why not just use Wireshark"

  Yes, Wireshark has the ability to filter and decode Bitcoin messages.  However, the purpose of the message capture added in this PR is to assist with debugging, primarily for new developers looking to improve their knowledge of the Bitcoin Protocol.  This drives the design in a different direction than Wireshark, in two different ways.  First, this tool must be convenient and simple to use.  Using an external tool, like Wireshark, requires setup and interpretation of the results.  To a new user who doesn't necessarily know what to expect, this is unnecessary difficulty.  This tool, on the other hand, "just works".  Turn on the command line flag, run your node, run the script, read the JSON.  Second, because this tool is being used for debugging, we want it to be as close to the true behavior of the node as possible.  A lot can happen in the SocketHandler thread that would be missed by Wireshark.

  Additionally, if we are to use Wireshark, we are at the mercy of whoever it maintaining the protocol in Wireshark, both as to it being accurate and recent.  As can be seen by the **many** previous attempts to include Bitcoin in Wireshark (google "bitcoin dissector") this is easier said than done.

  Lastly, I truly believe that this tool will be used significantly more by being included in the codebase.  It's just that much more discoverable.

ACKs for top commit:
  MarcoFalke:
    re-ACK bff7c66e67aa2f18ef70139338643656a54444fe only some minor changes: 👚
  jnewbery:
    utACK bff7c66e67aa2f18ef70139338643656a54444fe
  theStack:
    re-ACK bff7c66e67aa2f18ef70139338643656a54444fe

Tree-SHA512: e59e3160422269221f70f98720b47842775781c247c064071d546c24fa7a35a0e5534e8baa4b4591a750d7eb16de6b4ecf54cbee6d193b261f4f104e28c15f47

2024-02-14 10:34:10 -06:00

3 Commits