dash/test/functional/test_framework
MarcoFalke 77f106ebf5
Merge #17633: tests: Add option --valgrind to run the functional tests under Valgrind
5db506ba5943868cc2c845f717508739b7f05714 tests: Add option --valgrind to run nodes under valgrind in the functional tests (practicalswift)

Pull request description:

  What is better than fixing bugs? Fixing entire bug classes of course! :)

  Add option `--valgrind` to run the functional tests under Valgrind.

  Regular functional testing under Valgrind would have caught many of the uninitialized reads we've seen historically.

  Let's kill this bug class once and for all: let's never use an uninitialized value ever again. Or at least not one that would be triggered by running the functional tests! :)

  My hope is that this addition will make it super-easy to run the functional tests under Valgrind and thus increase the probability of people making use of it :)

  Hopefully `test/functional/test_runner.py --valgrind` will become a natural part of the pre-release QA process.

  **Usage:**

  ```
  $ test/functional/test_runner.py --help
  …
    --valgrind            run nodes under the valgrind memory error detector:
                          expect at least a ~10x slowdown, valgrind 3.14 or
                          later required
  ```

  **Live demo:**

  First, let's re-introduce a memory bug by reverting the recent P2P uninitialized read bug fix from PR #17624 ("net: Fix an uninitialized read in ProcessMessage(…, "tx", …) when receiving a transaction we already have").

  ```
  $ git diff
  diff --git a/src/consensus/validation.h b/src/consensus/validation.h
  index 3401eb64c..940adea33 100644
  --- a/src/consensus/validation.h
  +++ b/src/consensus/validation.h
  @@ -114,7 +114,7 @@ inline ValidationState::~ValidationState() {};

   class TxValidationState : public ValidationState {
   private:
  -    TxValidationResult m_result = TxValidationResult::TX_RESULT_UNSET;
  +    TxValidationResult m_result;
   public:
       bool Invalid(TxValidationResult result,
                    const std::string &reject_reason="",
  ```

  Second, let's test as normal without Valgrind:

  ```
  $ test/functional/p2p_segwit.py -l INFO
  2019-11-28T09:30:42.810000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test__fc8q3qo
  …
  2019-11-28T09:31:57.187000Z TestFramework (INFO): Subtest: test_non_standard_witness_blinding (Segwit active = True)
  …
  2019-11-28T09:32:08.265000Z TestFramework (INFO): Tests successful
  ```

  Third, let's test with `--valgrind` and see if the test fail (as we expect) when the unitialized value is used:

  ```
  $ test/functional/p2p_segwit.py -l INFO --valgrind
  2019-11-28T09:32:33.018000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_gtjecx2l
  …
  2019-11-28T09:40:36.702000Z TestFramework (INFO): Subtest: test_non_standard_witness_blinding (Segwit active = True)
  2019-11-28T09:40:37.813000Z TestFramework (ERROR): Assertion failed
  ConnectionRefusedError: [Errno 111] Connection refused
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 5db506ba5943868cc2c845f717508739b7f05714
  jonatack:
    ACK 5db506ba5943868cc2c845f717508739b7f05714

Tree-SHA512: 2eaecacf4da166febad88b2a8ee6d7ac2bcd38d4c1892ca39516b6343e8f8c8814edf5eaf14c90f11a069a0389d24f0713076112ac284de987e72fc5f6cc3795
2022-04-03 08:35:46 +05:30
..
__init__.py
address.py Merge #14324: qa: Run more tests with wallet disabled 2021-09-24 18:48:44 -04:00
authproxy.py Merge #18596: test: Try once more when RPC connection fails on Windows 2021-09-28 14:46:44 -04:00
bignum.py
blocktools.py Merge #15059: test: Add basic test for BIP34 2022-03-08 22:53:13 -05:00
coverage.py
descriptors.py merge bitcoin#15368: Descriptor checksums 2021-10-28 14:01:02 +05:30
key.py Merge #20326: tests: Fix ecdsa_verify in test framework 2021-12-28 00:27:11 -05:00
messages.py Compressed headers implementation. (#4497) 2022-03-11 22:39:12 +03:00
mininode.py Compressed headers implementation. (#4497) 2022-03-11 22:39:12 +03:00
netutil.py merge #14954: Require python 3.5 2021-08-31 11:16:12 +05:30
script.py merge #14954: Require python 3.5 2021-08-31 11:16:12 +05:30
siphash.py
socks5.py merge #14954: Require python 3.5 2021-08-31 11:16:12 +05:30
test_framework.py Merge #17633: tests: Add option --valgrind to run the functional tests under Valgrind 2022-04-03 08:35:46 +05:30
test_node.py Merge #17633: tests: Add option --valgrind to run the functional tests under Valgrind 2022-04-03 08:35:46 +05:30
util.py merge bitcoin#18077: Add NAT-PMP port forwarding support 2022-02-26 17:49:13 +05:30
wallet_util.py Merge #15108: [tests] tidy up wallet_importmulti.py 2022-03-08 22:53:13 -05:00