dash/test/functional/test_framework
MarcoFalke 33384816b5 Merge #14519: tests: add utility to easily profile node performance with perf
13782b8ba8 docs: add perf section to developer docs (James O'Beirne)
58180b5fd4 tests: add utility to easily profile node performance with perf (James O'Beirne)

Pull request description:

  Adds a context manager to easily (and selectively) profile node performance during functional test execution using `perf`.

  While writing some tests, I encountered some odd bitcoind slowness. I wrote up a utility (`TestNode.profile_with_perf`) that generates performance diagnostics for a node by running `perf` during the execution of a particular region of test code.

  `perf` usage is detailed in the excellent (and sadly unmerged) https://github.com/bitcoin/bitcoin/pull/12649; all due props to @eklitzke.

  ### Example

  ```python
  with node.profile_with_perf("large-msgs"):
      for i in range(200):
          node.p2p.send_message(some_large_msg)
      node.p2p.sync_with_ping()
  ```

  This generates a perf data file in the test node's datadir (`/tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data`).

  Running `perf report` generates nice output about where the node spent most of its time while running that part of the test:

  ```bash
  $ perf report -i /tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data --stdio \
    | c++filt \
    | less

  # To display the perf.data header info, please use --header/--header-only options.
  #
  #
  # Total Lost Samples: 0
  #
  # Samples: 135  of event 'cycles:pp'
  # Event count (approx.): 1458205679493582
  #
  # Children      Self  Command          Shared Object        Symbol
  # ........  ........  ...............  ...................  ........................................................................................................................................................................................................................................................................
  #
      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
                  |
                  ---CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNetMessage::readData(char const*, unsigned int)
                  |
                  ---CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      35.52%     0.00%  bitcoin-net      bitcoind             [.] std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                  |
                  ---std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                     CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

  ...
  ```

Tree-SHA512: 9ac4ceaa88818d5eca00994e8e3c8ad42ae019550d6583972a0a4f7b0c4f61032e3d0c476b4ae58756bc5eb8f8015a19a7fc26c095bd588f31d49a37ed0c6b3e
2021-08-18 22:24:36 -03:00
..
__init__.py Merge bitcoin#9956: Reorganise qa directory (#2912) 2019-05-19 23:20:34 +03:00
address.py Partial merge #8499: Add several policy limits and disable uncompressed keys for segwit scripts 2021-06-05 22:12:55 +03:00
authproxy.py Merge #15772: test: Properly log named args in authproxy 2021-07-10 12:10:51 -05:00
bignum.py Merge #10781: Python cleanups 2020-01-02 22:01:29 -06:00
blocktools.py Merge #13954: Warn (don't fail!) on spelling errors. Fix typos reported by codespell. 2021-07-19 12:51:24 -05:00
coverage.py Some Dashification (#3513) 2020-06-11 11:39:04 +03:00
key.py Merge #14365: tests: Add Python dead code linter (vulture) to Travis 2021-08-12 14:42:32 -03:00
messages.py Merge #14365: tests: Add Python dead code linter (vulture) to Travis 2021-08-12 14:42:32 -03:00
mininode.py Merge #14365: tests: Add Python dead code linter (vulture) to Travis 2021-08-12 14:42:32 -03:00
netutil.py Merge bitcoin/bitcoin#22103: test: Fix IPv6 check on BSD systems 2021-07-15 16:01:43 -05:00
script.py Merge #14816: Add CScriptNum decode python implementation in functional suite 2021-08-15 11:08:08 -04:00
siphash.py Merge bitcoin#9956: Reorganise qa directory (#2912) 2019-05-19 23:20:34 +03:00
socks5.py Merge #14365: tests: Add Python dead code linter (vulture) to Travis 2021-08-12 14:42:32 -03:00
test_framework.py Merge #14519: tests: add utility to easily profile node performance with perf 2021-08-18 22:24:36 -03:00
test_node.py Merge #14519: tests: add utility to easily profile node performance with perf 2021-08-18 22:24:36 -03:00
util.py Merge pull request #4333 from Munkybooty/backports-0.18-pr13 2021-08-16 14:24:51 -05:00