dash/test/functional
fanquake 807720666c Merge bitcoin/bitcoin#22002: Fix crash when parsing command line with -noincludeconf=0
fad0867d6ab9430070aa7d60bf7617a6508e0586 Cleanup -includeconf error message (MarcoFalke)
fa9f711c3746ca3962f15224285a453744cd45b3 Fix crash when parsing command line with -noincludeconf=0 (MarcoFalke)

Pull request description:

  The error message has several issues:

  * It may crash instead of cleanly shutting down, when `-noincludeconf=0` is passed
  * It doesn't quote the value
  * It includes an erroneous trailing `\n`
  * It is redundantly mentioning `"-includeconf cannot be used from commandline;"` several times, when once should be more than sufficient

  Fix all issues by:
  * Replacing `get_str()` with `write()` to fix the crash and quoting issue
  * Remove the `\n` and only print the first value to fix the other issues

  Before:

  ```
  $ ./src/bitcoind -noincludeconf=0
  terminate called after throwing an instance of 'std::runtime_error'
    what():  JSON value is not a string as expected
  Aborted (core dumped)

  $ ./src/bitcoind -includeconf='a b' -includeconf=c
  Error: Error parsing command line arguments: -includeconf cannot be used from commandline; -includeconf=a b
  -includeconf cannot be used from commandline; -includeconf=c
  ```

  After:

  ```
  $ ./src/bitcoind -noincludeconf=0
  Error: Error parsing command line arguments: -includeconf cannot be used from commandline; -includeconf=true

  $ ./src/bitcoind -includeconf='a b' -includeconf=c
  Error: Error parsing command line arguments: -includeconf cannot be used from commandline; -includeconf="a b"
  ```

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34493

  Testcase: https://github.com/bitcoin/bitcoin/files/6515429/clusterfuzz-testcase-minimized-system-6328535926046720.log

  ```
  FUZZ=system ./src/test/fuzz/fuzz ./clusterfuzz-testcase-minimized-system-6328535926046720.log
  ```

  See https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md

ACKs for top commit:
  sipa:
    utACK fad0867d6ab9430070aa7d60bf7617a6508e0586

Tree-SHA512: b44af93be6bf71b43669058c1449c4c6999f03b5b01b429851b149b12d77733408cb207e9a3edc6f0bffd6030c4c52165e8e23a1c2718ff5082a6ba254cc94a4
2023-05-31 10:52:02 -05:00
..
data fix: dashification for rpc_createmultisig introduced in bitcoin#13072 2023-04-06 20:15:47 +03:00
test_framework Merge #17891: scripted-diff: Replace CCriticalSection with RecursiveMutex 2023-05-24 12:43:57 -05:00
.gitignore
combine_logs.py
combined_log_template.html
create_cache.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
example_test.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
feature_abortnode.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
feature_addressindex.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
feature_asmap.py
feature_assumevalid.py Merge #18318: test: Bump rpc timeout in feature_assumevalid to avoid valgrind timeouts 2023-02-28 00:06:46 +03:00
feature_bip68_sequence.py Merge #16060: Bury bip9 deployments 2023-04-25 23:41:20 -05:00
feature_block.py merge bitcoin#19504: Bump minimum python version to 3.6 2023-05-11 09:18:48 -05:00
feature_blocksdir.py
feature_cltv.py Merge #16060: Bury bip9 deployments 2023-04-25 23:41:20 -05:00
feature_config_args.py Merge #17891: scripted-diff: Replace CCriticalSection with RecursiveMutex 2023-05-24 12:43:57 -05:00
feature_csv_activation.py Merge #20857: test: update documentation in feature_csv_activation.py 2023-05-31 10:52:02 -05:00
feature_dbcrash.py fix: reviewing TODOes at v19 (#5303) 2023-04-09 00:11:22 -05:00
feature_dersig.py Merge #16060: Bury bip9 deployments 2023-04-25 23:41:20 -05:00
feature_dip3_deterministicmns.py Merge #16060: Bury bip9 deployments 2023-04-25 23:41:20 -05:00
feature_dip3_v19.py fix(tests): a couple of fixes for dynamically_add/update functions (#5288) 2023-04-09 00:09:18 -05:00
feature_dip4_coinbasemerkleroots.py feat: bury dash deployments: dip0003, dip0020, dip0024, brr, bip147 (#5356) 2023-05-18 12:15:08 -05:00
feature_dip0020_activation.py feat: bury dash deployments: dip0003, dip0020, dip0024, brr, bip147 (#5356) 2023-05-18 12:15:08 -05:00
feature_fee_estimation.py Merge #16535: test: Explain why -whitelist is used in feature_fee_estimation 2023-04-04 12:45:27 -05:00
feature_filelock.py test: have feature_filelock.py test both bdb and sqlite, depending on compile flags 2023-02-17 14:21:19 -06:00
feature_governance_objects.py
feature_help.py Merge #17770: test: bump test timeouts so that functional tests run in valgrind 2022-10-03 16:08:31 -04:00
feature_includeconf.py Merge bitcoin/bitcoin#22002: Fix crash when parsing command line with -noincludeconf=0 2023-05-31 10:52:02 -05:00
feature_llmq_chainlocks.py test: Fix activate_by_name (#5367) 2023-05-11 12:25:41 +03:00
feature_llmq_connections.py test: Various test improvements (#5382) 2023-05-24 12:38:33 -05:00
feature_llmq_data_recovery.py chore: harden dip 20 and 24 activation (#5344) 2023-05-17 14:11:33 +03:00
feature_llmq_dkgerrors.py chore: run copyright_header.py update 2023-01-13 00:49:04 +03:00
feature_llmq_hpmn.py fix(tests): a couple of fixes for dynamically_add/update functions (#5288) 2023-04-09 00:09:18 -05:00
feature_llmq_is_cl_conflicts.py fix: improved reliability of functional tests that activates dip24 (#5313) 2023-05-06 21:36:45 -05:00
feature_llmq_is_migration.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
feature_llmq_is_retroactive.py chore: run copyright_header.py update 2023-01-13 00:49:04 +03:00
feature_llmq_rotation.py feat!: calculate quorum members using v20 cbtx clsig (#5366) 2023-05-17 20:27:15 +03:00
feature_llmq_signing.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
feature_llmq_simplepose.py chore: run copyright_header.py update 2023-01-13 00:49:04 +03:00
feature_loadblock.py Merge #19350: test: Refactor tests using restart_node 2023-02-27 23:12:41 -06:00
feature_logging.py Merge #19350: test: Refactor tests using restart_node 2023-02-27 23:12:41 -06:00
feature_maxuploadtarget.py Merge #18209: test: Reduce unneeded whitelist permissions in tests 2022-10-20 11:47:02 -04:00
feature_minchainwork.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
feature_multikeysporks.py chore: run copyright_header.py update 2023-01-13 00:49:04 +03:00
feature_new_quorum_type_activation.py chore: harden dip 20 and 24 activation (#5344) 2023-05-17 14:11:33 +03:00
feature_notifications.py Merge #19897: Change FILE_CHAR_BLOCKLIST to FILE_CHARS_DISALLOWED 2023-04-09 00:06:56 -05:00
feature_nulldummy.py Merge #17004: validation: Remove REJECT code from CValidationState 2023-04-17 10:42:25 -05:00
feature_proxy.py Merge #20120: net, rpc, test, bugfix: update GetNetworkName, GetNetworksInfo, regression tests 2023-05-31 10:52:02 -05:00
feature_pruning.py test: Fix feature_pruning.py (#5353) 2023-05-09 12:13:14 -05:00
feature_reindex.py Merge #18873: test: Fix intermittent sync_blocks failures 2022-09-07 22:25:40 +03:00
feature_settings.py revert: revert bitcoin#20034 (Get rid of default wallet hacks) 2022-12-04 18:02:40 -06:00
feature_shutdown.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
feature_spentindex.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
feature_sporks.py chore: run copyright_header.py update 2023-01-13 00:49:04 +03:00
feature_timestampindex.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
feature_txindex.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
feature_uacomment.py
feature_utxo_set_hash.py
feature_versionbits_warning.py
interface_bitcoin_cli.py Merge bitcoin/bitcoin#21822: test: resolve bug in interface_bitcoin_cli.py 2023-04-18 23:24:06 +03:00
interface_http.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
interface_rest.py Merge #17599: ci: Run functional tests on s390x 2023-04-17 19:34:02 +03:00
interface_rpc.py Merge #18335: bitcoin-cli: print useful error if bitcoind rpc work queue exceeded 2023-04-14 23:34:12 -05:00
interface_zmq_dash.py fix: improved reliability of functional tests that activates dip24 (#5313) 2023-05-06 21:36:45 -05:00
interface_zmq.py Merge #16740: qa: Relax so that the subscriber is ready before publishing zmq messages 2023-04-04 12:45:27 -05:00
mempool_accept.py Merge #17004: validation: Remove REJECT code from CValidationState 2023-04-17 10:42:25 -05:00
mempool_expiry.py
mempool_limit.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
mempool_package_onemore.py Merge #17705: test: re-enable CLI test support by using EncodeDecimal in json.dumps() 2022-06-08 12:35:12 +07:00
mempool_packages.py Merge #18895: p2p: unbroadcast followups: rpcs, nLastResend, mempool sanity check 2023-02-27 23:12:41 -06:00
mempool_persist.py Merge #18807: [doc / test / mempool] unbroadcast follow-ups 2023-02-27 23:12:41 -06:00
mempool_reorg.py Merge #18770: test: Remove raw-tx byte juggling in mempool_reorg 2023-01-01 20:16:57 -06:00
mempool_resurrect.py
mempool_spend_coinbase.py Enabled extra functional tests thanks to previous commit with new Decimal json output 2022-06-08 12:35:12 +07:00
mempool_unbroadcast.py Merge #18807: [doc / test / mempool] unbroadcast follow-ups 2023-02-27 23:12:41 -06:00
mempool_updatefromblock.py Merge #18485: test: Add mempool_updatefromblock.py 2023-01-11 21:42:32 -06:00
mining_basic.py Merge #19206: test: Remove leftover comment in mining_basic 2023-02-27 23:12:41 -06:00
mining_getblocktemplate_longpoll.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
mining_prioritisetransaction.py Merge #13424: Consistently validate txid / blockhash length and encoding in rpc calls 2023-04-06 20:14:58 +03:00
p2p_addrv2_relay.py fix(tests): various fixes (#4840) 2022-10-12 20:36:17 +03:00
p2p_blockfilters.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
p2p_blocksonly.py Merge #19474: doc: Use precise permission flags where possible 2023-01-19 23:37:39 -06:00
p2p_compactblocks.py Merge #19781: test: add parameterized constructor for msg_sendcmpct() 2023-04-09 00:06:56 -05:00
p2p_connect_to_devnet.py chore: run copyright_header.py update 2023-01-13 00:49:04 +03:00
p2p_disconnect_ban.py Merge #19350: test: Refactor tests using restart_node 2023-02-27 23:12:41 -06:00
p2p_eviction.py Merge #16756: test: Connection eviction logic tests 2023-02-27 23:12:41 -06:00
p2p_filter.py Merge #18593: test: complete impl. of msg_merkleblock and wait_for_merkleblock 2023-04-09 00:06:56 -05:00
p2p_fingerprint.py Merge #18690: test: Check object hashes in wait_for_getdata 2023-01-11 21:42:32 -06:00
p2p_getdata.py Merge #19060: test: Remove global wait_until from p2p_getdata 2023-01-22 00:27:52 -06:00
p2p_instantsend.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
p2p_invalid_block.py Merge #18209: test: Reduce unneeded whitelist permissions in tests 2022-10-20 11:47:02 -04:00
p2p_invalid_locator.py Merge #18593: test: complete impl. of msg_merkleblock and wait_for_merkleblock 2023-04-09 00:06:56 -05:00
p2p_invalid_messages.py (Partial) Merge #19304: test: Check that message sends successfully when header is split across two buffers 2023-02-27 23:12:41 -06:00
p2p_invalid_tx.py
p2p_leak_tx.py merge bitcoin#18764: replace inv type magic numbers by constants 2022-09-24 08:51:05 +05:30
p2p_leak.py Merge #19252: test: wait for disconnect in disconnect_p2ps + bloomfilter test followups 2023-02-27 23:12:41 -06:00
p2p_mempool.py
p2p_node_network_limited.py Merge #19252: test: wait for disconnect in disconnect_p2ps + bloomfilter test followups 2023-02-27 23:12:41 -06:00
p2p_permissions.py Merge #19474: doc: Use precise permission flags where possible 2023-01-19 23:37:39 -06:00
p2p_quorum_data.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
p2p_sendheaders_compressed.py merge bitcoin#18764: replace inv type magic numbers by constants 2022-09-24 08:51:05 +05:30
p2p_sendheaders.py Merge #13424: Consistently validate txid / blockhash length and encoding in rpc calls 2023-04-06 20:14:58 +03:00
p2p_timeouts.py Merge #18247: test: Wait for both veracks in add_p2p_connection 2022-09-08 00:02:35 +03:00
p2p_tx_download.py Merge #16197: net: Use mockable time for tx download 2023-04-06 20:14:58 +03:00
p2p_unrequested_blocks.py Merge #19474: doc: Use precise permission flags where possible 2023-01-19 23:37:39 -06:00
README.md merge bitcoin#19504: Bump minimum python version to 3.6 2023-05-11 09:18:48 -05:00
rpc_bind.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
rpc_blockchain.py feat: bury dash deployments: dip0003, dip0020, dip0024, brr, bip147 (#5356) 2023-05-18 12:15:08 -05:00
rpc_coinjoin.py
rpc_createmultisig.py fix: dashification for rpc_createmultisig introduced in bitcoin#13072 2023-04-06 20:15:47 +03:00
rpc_decodescript.py
rpc_deprecated.py
rpc_deriveaddresses.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
rpc_dumptxoutset.py feat: bury dash deployments: dip0003, dip0020, dip0024, brr, bip147 (#5356) 2023-05-18 12:15:08 -05:00
rpc_estimatefee.py Merge #18406: test: add executable flag for rpc_estimatefee.py 2023-02-28 00:06:46 +03:00
rpc_fundrawtransaction.py test: Various test improvements (#5382) 2023-05-24 12:38:33 -05:00
rpc_generateblock.py
rpc_getaddressinfo_labels_purpose_deprecation.py Merge #17578: rpc: simplify getaddressinfo labels, deprecate previous behavior 2023-04-25 23:14:25 +03:00
rpc_getblockfilter.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
rpc_getblockstats.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
rpc_getchaintips.py
rpc_getdescriptorinfo.py Merge #15443: qa: Add getdescriptorinfo functional test 2023-04-04 12:53:49 -05:00
rpc_help.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
rpc_invalidateblock.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
rpc_masternode.py fix/tests: fix masternode payments rpc, add tests (#5191) 2023-02-10 13:21:05 +02:00
rpc_misc.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
rpc_mnauth.py chore: run copyright_header.py update 2023-01-13 00:49:04 +03:00
rpc_named_arguments.py Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli 2022-06-08 12:35:12 +07:00
rpc_net.py Merge #16991: qa: Fix service flag comparison check in rpc_net test (luke-jr) 2023-04-25 23:41:20 -05:00
rpc_platform_filter.py feat!: 4k collateral high performance masternode implementation (#5039) 2023-02-14 12:48:33 -06:00
rpc_preciousblock.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
rpc_psbt.py Merge #14380: fix assert crash when specified change output spend size is unknown 2023-04-06 20:14:58 +03:00
rpc_quorum.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
rpc_rawtransaction.py Merge #15921: validation: Tidy up ValidationState interface 2023-04-17 10:42:25 -05:00
rpc_scantxoutset.py revert: revert bitcoin#20186 (Make -wallet setting not create wallets) 2022-12-04 18:02:40 -06:00
rpc_setban.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
rpc_signmessage.py Merge #18466: rpc: fix invalid parameter error codes for {sign,verify}message RPCs 2023-04-09 00:06:56 -05:00
rpc_signrawtransaction.py Merge #18545: test: refactor rpc_signrawtransaction and add logging 2023-01-11 21:42:32 -06:00
rpc_txoutproof.py Merge #13424: Consistently validate txid / blockhash length and encoding in rpc calls 2023-04-06 20:14:58 +03:00
rpc_uptime.py
rpc_users.py Merge #16334: test: rpc_users: Also test rpcauth.py with password. 2023-03-29 21:01:56 +03:00
rpc_verifychainlock.py chore: run copyright_header.py update 2023-01-13 00:49:04 +03:00
rpc_verifyislock.py rpc: split spork manipulation logic to distinct "sporkupdate" call (#4885) 2022-06-18 19:52:45 +03:00
rpc_whitelist.py Merge #17806: test: Change filemode of rpc_whitelist.py 2022-06-08 12:36:52 +07:00
test_runner.py test: Various test improvements (#5382) 2023-05-24 12:38:33 -05:00
test-shell.md Merge #17378: TestShell: Fix typos & implement cleanups 2023-01-23 12:22:29 -06:00
tool_wallet.py partial bitcoin#18788: Update more tests to work with descriptor wallets 2023-02-17 14:21:19 -06:00
wallet_abandonconflict.py Merge #19350: test: Refactor tests using restart_node 2023-02-27 23:12:41 -06:00
wallet_avoidreuse.py Merge #17891: scripted-diff: Replace CCriticalSection with RecursiveMutex 2023-05-24 12:43:57 -05:00
wallet_backup.py revert: revert bitcoin#20186 (Make -wallet setting not create wallets) 2022-12-04 18:02:40 -06:00
wallet_balance.py Merge #16766: wallet: Make IsTrusted scan parents recursively 2023-04-17 19:34:02 +03:00
wallet_basic.py Merge #17578: rpc: simplify getaddressinfo labels, deprecate previous behavior 2023-04-25 23:14:25 +03:00
wallet_coinbase_category.py
wallet_create_tx.py
wallet_createwallet.py fix: reviewing TODOes at v19 (#5303) 2023-04-09 00:11:22 -05:00
wallet_disable.py revert: revert bitcoin#20034 (Get rid of default wallet hacks) 2022-12-04 18:02:40 -06:00
wallet_dump.py test: skip bdb-specific tests if support is not compiled in 2023-02-17 14:21:19 -06:00
wallet_encryption.py
wallet_fallbackfee.py test: don't make any wallets unless wallets are required 2023-02-17 14:21:19 -06:00
wallet_groups.py Merge #18345: test: Bump timeouts to avoid valgrind failures 2022-10-03 16:08:58 -04:00
wallet_hd.py Merge #19350: test: Refactor tests using restart_node 2023-02-27 23:12:41 -06:00
wallet_import_rescan.py Merge #16465: test: Test p2sh-witness and bech32 in wallet_import_rescan 2023-04-25 23:41:20 -05:00
wallet_import_with_label.py Merge #17891: scripted-diff: Replace CCriticalSection with RecursiveMutex 2023-05-24 12:43:57 -05:00
wallet_importmulti.py Merge #17578: rpc: simplify getaddressinfo labels, deprecate previous behavior 2023-04-25 23:14:25 +03:00
wallet_importprunedfunds.py
wallet_keypool_hd.py
wallet_keypool_topup.py revert: revert bitcoin#20034 (Get rid of default wallet hacks) 2022-12-04 18:02:40 -06:00
wallet_keypool.py
wallet_labels.py Merge #17891: scripted-diff: Replace CCriticalSection with RecursiveMutex 2023-05-24 12:43:57 -05:00
wallet_listreceivedby.py Merge #17578: rpc: simplify getaddressinfo labels, deprecate previous behavior 2023-04-25 23:14:25 +03:00
wallet_listsinceblock.py Merge #17437: rpc: Expose block height of wallet transactions 2023-04-17 19:34:02 +03:00
wallet_listtransactions.py Merge #17437: rpc: Expose block height of wallet transactions 2023-04-17 19:34:02 +03:00
wallet_multiwallet.py Merge #21040: wallet: Fix already-loading message grammar 2023-04-14 23:34:12 -05:00
wallet_reorgsrestore.py Merge #19350: test: Refactor tests using restart_node 2023-02-27 23:12:41 -06:00
wallet_resendwallettransactions.py Merge #19893: test: Remove or explain syncwithvalidationinterfacequeue 2023-04-09 00:06:56 -05:00
wallet_startup.py revert: revert bitcoin#15454 (Remove the automatic creation and loading of the default wallet) 2022-12-04 18:02:40 -06:00
wallet_txn_clone.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
wallet_txn_doublespend.py merge bitcoin#19967: Replace (dis)?connect_nodes globals with TestFramework methods 2022-10-17 08:03:12 +05:30
wallet_upgradetohd.py Merge #19350: test: Refactor tests using restart_node 2023-02-27 23:12:41 -06:00
wallet_upgradewallet.py (partial) Merge #18828: test: Strip down previous releases boilerplate 2023-02-27 23:12:41 -06:00
wallet_watchonly.py Merge #16742: test: add executable flag for wallet_watchonly.py 2023-01-23 12:22:29 -06:00

Functional tests

Writing Functional Tests

Example test

The file test/functional/example_test.py is a heavily commented example of a test case that uses both the RPC and P2P interfaces. If you are writing your first test, copy that file and modify to fit your needs.

Coverage

Running test/functional/test_runner.py with the --coverage argument tracks which RPCs are called by the tests and prints a report of uncovered RPCs in the summary. This can be used (along with the --extended argument) to find out which RPCs we don't have test cases for.

Style guidelines

  • Where possible, try to adhere to PEP-8 guidelines
  • Use a python linter like flake8 before submitting PRs to catch common style nits (eg trailing whitespace, unused imports, etc)
  • The oldest supported Python version is specified in doc/dependencies.md. Consider using pyenv, which checks .python-version, to prevent accidentally introducing modern syntax from an unsupported Python version. The Travis linter also checks this, but possibly not in all cases.
  • See the python lint script that checks for violations that could lead to bugs and issues in the test code.
  • Avoid wildcard imports
  • Use a module-level docstring to describe what the test is testing, and how it is testing it.
  • When subclassing the BitcoinTestFramwork, place overrides for the set_test_params(), add_options() and setup_xxxx() methods at the top of the subclass, then locally-defined helper methods, then the run_test() method.
  • Use f'{x}' for string formatting in preference to '{}'.format(x) or '%s' % x.

Naming guidelines

  • Name the test <area>_test.py, where area can be one of the following:
    • feature for tests for full features that aren't wallet/mining/mempool, eg feature_rbf.py
    • interface for tests for other interfaces (REST, ZMQ, etc), eg interface_rest.py
    • mempool for tests for mempool behaviour, eg mempool_reorg.py
    • mining for tests for mining features, eg mining_prioritisetransaction.py
    • p2p for tests that explicitly test the p2p interface, eg p2p_disconnect_ban.py
    • rpc for tests for individual RPC methods or features, eg rpc_listtransactions.py
    • tool for tests for tools, eg tool_wallet.py
    • wallet for tests for wallet features, eg wallet_keypool.py
  • use an underscore to separate words
    • exception: for tests for specific RPCs or command line options which don't include underscores, name the test after the exact RPC or argument name, eg rpc_decodescript.py, not rpc_decode_script.py
  • Don't use the redundant word test in the name, eg interface_zmq.py, not interface_zmq_test.py

General test-writing advice

  • Set self.num_nodes to the minimum number of nodes necessary for the test. Having additional unrequired nodes adds to the execution time of the test as well as memory/CPU/disk requirements (which is important when running tests in parallel or on Travis).
  • Avoid stop-starting the nodes multiple times during the test if possible. A stop-start takes several seconds, so doing it several times blows up the runtime of the test.
  • Set the self.setup_clean_chain variable in set_test_params() to control whether or not to use the cached data directories. The cached data directories contain a 200-block pre-mined blockchain and wallets for four nodes. Each node has 25 mature blocks (25x500=12500 DASH) in its wallet.
  • When calling RPCs with lots of arguments, consider using named keyword arguments instead of positional arguments to make the intent of the call clear to readers.
  • Many of the core test framework classes such as CBlock and CTransaction don't allow new attributes to be added to their objects at runtime like typical Python objects allow. This helps prevent unpredictable side effects from typographical errors or usage of the objects outside of their intended purpose.

RPC and P2P definitions

Test writers may find it helpful to refer to the definitions for the RPC and P2P messages. These can be found in the following source files:

  • /src/rpc/* for RPCs
  • /src/wallet/rpc* for wallet RPCs
  • ProcessMessage() in /src/net_processing.cpp for parsing P2P messages

Using the P2P interface

  • messages.py contains all the definitions for objects that pass over the network (CBlock, CTransaction, etc, along with the network-level wrappers for them, msg_block, msg_tx, etc).

  • P2P tests have two threads. One thread handles all network communication with the dashd(s) being tested in a callback-based event loop; the other implements the test logic.

  • P2PConnection is the class used to connect to a dashd. P2PInterface contains the higher level logic for processing P2P payloads and connecting to the Bitcoin Core node application logic. For custom behaviour, subclass the P2PInterface object and override the callback methods.

  • Can be used to write tests where specific P2P protocol behavior is tested. Examples tests are p2p_unrequested_blocks.py, p2p_compactblocks.py.

Prototyping tests

The TestShell class exposes the BitcoinTestFramework functionality to interactive Python3 environments and can be used to prototype tests. This may be especially useful in a REPL environment with session logging utilities, such as IPython. The logs of such interactive sessions can later be adapted into permanent test cases.

Test framework modules

The following are useful modules for test developers. They are located in test/functional/test_framework/.

authproxy.py

Taken from the python-bitcoinrpc repository.

test_framework.py

Base class for functional tests.

util.py

Generally useful functions.

mininode.py

Basic code to support P2P connectivity to a dashd.

script.py

Utilities for manipulating transaction scripts (originally from python-bitcoinlib)

key.py

Test-only secp256k1 elliptic curve implementation

blocktools.py

Helper functions for creating blocks and transactions.

Benchmarking with perf

An easy way to profile node performance during functional tests is provided for Linux platforms using perf.

Perf will sample the running node and will generate profile data in the node's datadir. The profile data can then be presented using perf report or a graphical tool like hotspot.

There are two ways of invoking perf: one is to use the --perf flag when running tests, which will profile each node during the entire test run: perf begins to profile when the node starts and ends when it shuts down. The other way is the use the profile_with_perf context manager, e.g.

with node.profile_with_perf("send-big-msgs"):
    # Perform activity on the node you're interested in profiling, e.g.:
    for _ in range(10000):
        node.p2p.send_message(some_large_message)

To see useful textual output, run

perf report -i /path/to/datadir/send-big-msgs.perf.data.xxxx --stdio | c++filt | less

See also: