dash/test/functional/feature_notifications.py

169 lines
8.4 KiB
Python
Raw Normal View History

#!/usr/bin/env python3
# Copyright (c) 2014-2020 The Bitcoin Core developers
# Copyright (c) 2023-2024 The Dash Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test the -alertnotify, -blocknotify, -chainlocknotify, -instantsendnotify and -walletnotify options."""
import os
from test_framework.address import ADDRESS_BCRT1_UNSPENDABLE
from test_framework.test_framework import DashTestFramework
Merge #16898: test: Remove connect_nodes_bi fadfd844de8c53034a97dfa6f771ffe9f523fba2 test: Remove unused connect_nodes_bi (MarcoFalke) fa3b9ee8b2280af4bcbcfffff275aaf8dd125929 scripted-diff: test: Replace connect_nodes_bi with connect_nodes (MarcoFalke) faaee1e39a91b3f603881655d3980c29af09852b test: Use connect_nodes when connecting nodes in the test_framework (MarcoFalke) 1111bb91f517838e5b9f778bf6b5a9c8d561e857 test: Reformat python imports to aid scripted diff (MarcoFalke) Pull request description: By default all test nodes are connected in a chain. However, instead of just a single connection between each pair of nodes, we end up with up to four connections for a "middle" node (two outbound, two inbound, from each side). This is generally redundant (tx and block relay should succeed with just a single connection) and confusing. For example, test timeouts after a call to `sync_` may be racy and hard to reproduce. On top of that, the test `debug.log`s are hard to read because txs and block invs may be relayed on the same connection multiple times. Fix this by inlining `connect_nodes_bi` in the two tests that need it, and then replace it with a single `connect_nodes` in all other tests. Historic background: `connect_nodes_bi` has been introduced as a (temporary?) workaround for bug #5113 and #5138, which has long been fixed in #5157 and #5662. ACKs for top commit: laanwj: ACK fadfd844de8c53034a97dfa6f771ffe9f523fba2 jonasschnelli: utACK fadfd844de8c53034a97dfa6f771ffe9f523fba2 - more of less a cleanup PR. promag: Tested ACK fadfd844de8c53034a97dfa6f771ffe9f523fba2, ran extended tests. Tree-SHA512: 2d027a8fd150749c071b64438a0a78ec922178628a7dbb89fd1212b0fa34febd451798c940101155d3617c0426c2c4865174147709894f1f1bb6cfa336aa7e24
2019-09-18 20:41:14 +02:00
from test_framework.util import (
assert_equal,
force_finish_mnsync,
Merge #16898: test: Remove connect_nodes_bi fadfd844de8c53034a97dfa6f771ffe9f523fba2 test: Remove unused connect_nodes_bi (MarcoFalke) fa3b9ee8b2280af4bcbcfffff275aaf8dd125929 scripted-diff: test: Replace connect_nodes_bi with connect_nodes (MarcoFalke) faaee1e39a91b3f603881655d3980c29af09852b test: Use connect_nodes when connecting nodes in the test_framework (MarcoFalke) 1111bb91f517838e5b9f778bf6b5a9c8d561e857 test: Reformat python imports to aid scripted diff (MarcoFalke) Pull request description: By default all test nodes are connected in a chain. However, instead of just a single connection between each pair of nodes, we end up with up to four connections for a "middle" node (two outbound, two inbound, from each side). This is generally redundant (tx and block relay should succeed with just a single connection) and confusing. For example, test timeouts after a call to `sync_` may be racy and hard to reproduce. On top of that, the test `debug.log`s are hard to read because txs and block invs may be relayed on the same connection multiple times. Fix this by inlining `connect_nodes_bi` in the two tests that need it, and then replace it with a single `connect_nodes` in all other tests. Historic background: `connect_nodes_bi` has been introduced as a (temporary?) workaround for bug #5113 and #5138, which has long been fixed in #5157 and #5662. ACKs for top commit: laanwj: ACK fadfd844de8c53034a97dfa6f771ffe9f523fba2 jonasschnelli: utACK fadfd844de8c53034a97dfa6f771ffe9f523fba2 - more of less a cleanup PR. promag: Tested ACK fadfd844de8c53034a97dfa6f771ffe9f523fba2, ran extended tests. Tree-SHA512: 2d027a8fd150749c071b64438a0a78ec922178628a7dbb89fd1212b0fa34febd451798c940101155d3617c0426c2c4865174147709894f1f1bb6cfa336aa7e24
2019-09-18 20:41:14 +02:00
)
# Linux allow all characters other than \x00
# Windows disallow control characters (0-31) and /\?%:|"<>
FILE_CHAR_START = 32 if os.name == 'nt' else 1
FILE_CHAR_END = 128
FILE_CHARS_DISALLOWED = '/\\?%*:|"<>' if os.name == 'nt' else '/'
Merge #21141: wallet: Add new format string placeholders for walletnotify 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Add new format string placeholders for walletnotify to include relevant block information for transactions (Maayan Keshet) Pull request description: This patch includes two new format placeholders for walletnotify: %b - the hash of the block containting the transaction (zeroed if a mempool transaction) %h - the height of the block containing the transaction (zero if a mempool transaction) I've included test suite changes to check and validate the above functional requirements as well as doc/help description changes. **Motivation** The walletnotify option is used to be notified of new transactions relevant to the wallet of the node. A common usage pattern is to perform afterwards additional RPC calls to determine: 1. If this is a mempool transaction or not (i.e. are there any confirmations?) 2. What block was it included in? 3. Did this transaction was seen before and is now seen again because of a fork? All of these questions can be answered with the current features, but the resulting RPC calls may be expensive in a heavily used node. As this information is readily available when calling the walletnotify callback, it makes sense to save expensive round trips by optionally sending this information at that point in time. I can definitely say we would like to use it in Fireblocks, my employer. Please let me know of any questions and suggestions. ACKs for top commit: laanwj: ACK 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Tree-SHA512: d2744e2a7a883f9c3a9fd32237110e048c4b6b11fea8221c33d10b74157f65bbc4351211f441e8c1a4af5d5d38e2ba6b1943a7673dc18860c0553d7b41e00775
2021-03-15 19:46:59 +01:00
UNCONFIRMED_HASH_STRING = 'unconfirmed'
def notify_outputname(walletname, txid):
return txid if os.name == 'nt' else '{}_{}'.format(walletname, txid)
class NotificationsTest(DashTestFramework):
def set_test_params(self):
self.set_dash_test_params(6, 4, fast_dip3_enforcement=True)
def setup_network(self):
self.wallet = ''.join(chr(i) for i in range(FILE_CHAR_START, FILE_CHAR_END) if chr(i) not in FILE_CHARS_DISALLOWED)
self.alertnotify_dir = os.path.join(self.options.tmpdir, "alertnotify")
self.blocknotify_dir = os.path.join(self.options.tmpdir, "blocknotify")
self.walletnotify_dir = os.path.join(self.options.tmpdir, "walletnotify")
self.chainlocknotify_dir = os.path.join(self.options.tmpdir, "chainlocknotify")
self.instantsendnotify_dir = os.path.join(self.options.tmpdir, "instantsendnotify")
os.mkdir(self.alertnotify_dir)
os.mkdir(self.blocknotify_dir)
os.mkdir(self.walletnotify_dir)
os.mkdir(self.chainlocknotify_dir)
os.mkdir(self.instantsendnotify_dir)
# -alertnotify and -blocknotify on node0, walletnotify on node1
self.extra_args[0].append("-alertnotify=echo > {}".format(os.path.join(self.alertnotify_dir, '%s')))
self.extra_args[0].append("-blocknotify=echo > {}".format(os.path.join(self.blocknotify_dir, '%s')))
self.extra_args[1].append("-rescan")
Merge #21141: wallet: Add new format string placeholders for walletnotify 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Add new format string placeholders for walletnotify to include relevant block information for transactions (Maayan Keshet) Pull request description: This patch includes two new format placeholders for walletnotify: %b - the hash of the block containting the transaction (zeroed if a mempool transaction) %h - the height of the block containing the transaction (zero if a mempool transaction) I've included test suite changes to check and validate the above functional requirements as well as doc/help description changes. **Motivation** The walletnotify option is used to be notified of new transactions relevant to the wallet of the node. A common usage pattern is to perform afterwards additional RPC calls to determine: 1. If this is a mempool transaction or not (i.e. are there any confirmations?) 2. What block was it included in? 3. Did this transaction was seen before and is now seen again because of a fork? All of these questions can be answered with the current features, but the resulting RPC calls may be expensive in a heavily used node. As this information is readily available when calling the walletnotify callback, it makes sense to save expensive round trips by optionally sending this information at that point in time. I can definitely say we would like to use it in Fireblocks, my employer. Please let me know of any questions and suggestions. ACKs for top commit: laanwj: ACK 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Tree-SHA512: d2744e2a7a883f9c3a9fd32237110e048c4b6b11fea8221c33d10b74157f65bbc4351211f441e8c1a4af5d5d38e2ba6b1943a7673dc18860c0553d7b41e00775
2021-03-15 19:46:59 +01:00
self.extra_args[1].append("-walletnotify=echo %h_%b > {}".format(os.path.join(self.walletnotify_dir, notify_outputname('%w', '%s'))))
# -chainlocknotify on node0, -instantsendnotify on node1
self.extra_args[0].append("-chainlocknotify=echo > {}".format(os.path.join(self.chainlocknotify_dir, '%s')))
self.extra_args[1].append("-instantsendnotify=echo > {}".format(os.path.join(self.instantsendnotify_dir, notify_outputname('%w', '%s'))))
self.wallet_names = [self.default_wallet_name, self.wallet]
super().setup_network()
def run_test(self):
# remove files created during network setup
for block_file in os.listdir(self.blocknotify_dir):
os.remove(os.path.join(self.blocknotify_dir, block_file))
for tx_file in os.listdir(self.walletnotify_dir):
os.remove(os.path.join(self.walletnotify_dir, tx_file))
self.log.info("test -blocknotify")
block_count = 10
blocks = self.nodes[1].generatetoaddress(block_count, self.nodes[1].getnewaddress() if self.is_wallet_compiled() else ADDRESS_BCRT1_UNSPENDABLE)
# wait at most 10 seconds for expected number of files before reading the content
self.wait_until(lambda: len(os.listdir(self.blocknotify_dir)) == block_count, timeout=10)
# directory content should equal the generated blocks hashes
assert_equal(sorted(blocks), sorted(os.listdir(self.blocknotify_dir)))
if self.is_wallet_compiled():
self.log.info("test -walletnotify")
# wait at most 10 seconds for expected number of files before reading the content
self.wait_until(lambda: len(os.listdir(self.walletnotify_dir)) == block_count, timeout=10)
# directory content should equal the generated transaction hashes
Merge #21141: wallet: Add new format string placeholders for walletnotify 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Add new format string placeholders for walletnotify to include relevant block information for transactions (Maayan Keshet) Pull request description: This patch includes two new format placeholders for walletnotify: %b - the hash of the block containting the transaction (zeroed if a mempool transaction) %h - the height of the block containing the transaction (zero if a mempool transaction) I've included test suite changes to check and validate the above functional requirements as well as doc/help description changes. **Motivation** The walletnotify option is used to be notified of new transactions relevant to the wallet of the node. A common usage pattern is to perform afterwards additional RPC calls to determine: 1. If this is a mempool transaction or not (i.e. are there any confirmations?) 2. What block was it included in? 3. Did this transaction was seen before and is now seen again because of a fork? All of these questions can be answered with the current features, but the resulting RPC calls may be expensive in a heavily used node. As this information is readily available when calling the walletnotify callback, it makes sense to save expensive round trips by optionally sending this information at that point in time. I can definitely say we would like to use it in Fireblocks, my employer. Please let me know of any questions and suggestions. ACKs for top commit: laanwj: ACK 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Tree-SHA512: d2744e2a7a883f9c3a9fd32237110e048c4b6b11fea8221c33d10b74157f65bbc4351211f441e8c1a4af5d5d38e2ba6b1943a7673dc18860c0553d7b41e00775
2021-03-15 19:46:59 +01:00
tx_details = list(map(lambda t: (t['txid'], t['blockheight'], t['blockhash']), self.nodes[1].listtransactions("*", block_count)))
self.stop_node(1)
Merge #21141: wallet: Add new format string placeholders for walletnotify 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Add new format string placeholders for walletnotify to include relevant block information for transactions (Maayan Keshet) Pull request description: This patch includes two new format placeholders for walletnotify: %b - the hash of the block containting the transaction (zeroed if a mempool transaction) %h - the height of the block containing the transaction (zero if a mempool transaction) I've included test suite changes to check and validate the above functional requirements as well as doc/help description changes. **Motivation** The walletnotify option is used to be notified of new transactions relevant to the wallet of the node. A common usage pattern is to perform afterwards additional RPC calls to determine: 1. If this is a mempool transaction or not (i.e. are there any confirmations?) 2. What block was it included in? 3. Did this transaction was seen before and is now seen again because of a fork? All of these questions can be answered with the current features, but the resulting RPC calls may be expensive in a heavily used node. As this information is readily available when calling the walletnotify callback, it makes sense to save expensive round trips by optionally sending this information at that point in time. I can definitely say we would like to use it in Fireblocks, my employer. Please let me know of any questions and suggestions. ACKs for top commit: laanwj: ACK 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Tree-SHA512: d2744e2a7a883f9c3a9fd32237110e048c4b6b11fea8221c33d10b74157f65bbc4351211f441e8c1a4af5d5d38e2ba6b1943a7673dc18860c0553d7b41e00775
2021-03-15 19:46:59 +01:00
self.expect_wallet_notify(tx_details)
self.log.info("test -walletnotify after rescan")
# restart node to rescan to force wallet notifications
self.start_node(1)
force_finish_mnsync(self.nodes[1])
self.connect_nodes(0, 1)
self.wait_until(lambda: len(os.listdir(self.walletnotify_dir)) == block_count, timeout=10)
# directory content should equal the generated transaction hashes
Merge #21141: wallet: Add new format string placeholders for walletnotify 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Add new format string placeholders for walletnotify to include relevant block information for transactions (Maayan Keshet) Pull request description: This patch includes two new format placeholders for walletnotify: %b - the hash of the block containting the transaction (zeroed if a mempool transaction) %h - the height of the block containing the transaction (zero if a mempool transaction) I've included test suite changes to check and validate the above functional requirements as well as doc/help description changes. **Motivation** The walletnotify option is used to be notified of new transactions relevant to the wallet of the node. A common usage pattern is to perform afterwards additional RPC calls to determine: 1. If this is a mempool transaction or not (i.e. are there any confirmations?) 2. What block was it included in? 3. Did this transaction was seen before and is now seen again because of a fork? All of these questions can be answered with the current features, but the resulting RPC calls may be expensive in a heavily used node. As this information is readily available when calling the walletnotify callback, it makes sense to save expensive round trips by optionally sending this information at that point in time. I can definitely say we would like to use it in Fireblocks, my employer. Please let me know of any questions and suggestions. ACKs for top commit: laanwj: ACK 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Tree-SHA512: d2744e2a7a883f9c3a9fd32237110e048c4b6b11fea8221c33d10b74157f65bbc4351211f441e8c1a4af5d5d38e2ba6b1943a7673dc18860c0553d7b41e00775
2021-03-15 19:46:59 +01:00
tx_details = list(map(lambda t: (t['txid'], t['blockheight'], t['blockhash']), self.nodes[1].listtransactions("*", block_count)))
self.expect_wallet_notify(tx_details)
self.log.info("test -chainlocknotify")
self.activate_v19(expected_activation_height=900)
self.log.info("Activated v19 at height:" + str(self.nodes[0].getblockcount()))
self.activate_dip8()
self.nodes[0].sporkupdate("SPORK_17_QUORUM_DKG_ENABLED", 0)
self.nodes[0].sporkupdate("SPORK_19_CHAINLOCKS_ENABLED", 4070908800)
self.wait_for_sporks_same()
self.move_to_next_cycle()
self.log.info("Cycle H height:" + str(self.nodes[0].getblockcount()))
self.move_to_next_cycle()
self.log.info("Cycle H+C height:" + str(self.nodes[0].getblockcount()))
self.move_to_next_cycle()
self.log.info("Cycle H+2C height:" + str(self.nodes[0].getblockcount()))
(quorum_info_i_0, quorum_info_i_1) = self.mine_cycle_quorum(llmq_type_name='llmq_test_dip0024', llmq_type=103)
self.nodes[0].sporkupdate("SPORK_19_CHAINLOCKS_ENABLED", 0)
self.wait_for_sporks_same()
self.log.info("Mine single block, wait for chainlock")
self.bump_mocktime(1)
tip = self.nodes[0].generate(1)[-1]
self.wait_for_chainlocked_block_all_nodes(tip)
# directory content should equal the chainlocked block hash
assert_equal([tip], sorted(os.listdir(self.chainlocknotify_dir)))
if self.is_wallet_compiled():
self.log.info("test -instantsendnotify")
assert_equal(len(os.listdir(self.instantsendnotify_dir)), 0)
tx_count = 10
for _ in range(tx_count):
txid = self.nodes[0].sendtoaddress(self.nodes[1].getnewaddress(), 1)
self.wait_for_instantlock(txid, self.nodes[1])
# wait at most 10 seconds for expected number of files before reading the content
self.wait_until(lambda: len(os.listdir(self.instantsendnotify_dir)) == tx_count, timeout=10)
# directory content should equal the generated transaction hashes
txids_rpc = list(map(lambda t: notify_outputname(self.wallet, t['txid']), self.nodes[1].listtransactions("*", tx_count)))
assert_equal(sorted(txids_rpc), sorted(os.listdir(self.instantsendnotify_dir)))
# TODO: add test for `-alertnotify` large fork notifications
Merge #21141: wallet: Add new format string placeholders for walletnotify 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Add new format string placeholders for walletnotify to include relevant block information for transactions (Maayan Keshet) Pull request description: This patch includes two new format placeholders for walletnotify: %b - the hash of the block containting the transaction (zeroed if a mempool transaction) %h - the height of the block containing the transaction (zero if a mempool transaction) I've included test suite changes to check and validate the above functional requirements as well as doc/help description changes. **Motivation** The walletnotify option is used to be notified of new transactions relevant to the wallet of the node. A common usage pattern is to perform afterwards additional RPC calls to determine: 1. If this is a mempool transaction or not (i.e. are there any confirmations?) 2. What block was it included in? 3. Did this transaction was seen before and is now seen again because of a fork? All of these questions can be answered with the current features, but the resulting RPC calls may be expensive in a heavily used node. As this information is readily available when calling the walletnotify callback, it makes sense to save expensive round trips by optionally sending this information at that point in time. I can definitely say we would like to use it in Fireblocks, my employer. Please let me know of any questions and suggestions. ACKs for top commit: laanwj: ACK 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Tree-SHA512: d2744e2a7a883f9c3a9fd32237110e048c4b6b11fea8221c33d10b74157f65bbc4351211f441e8c1a4af5d5d38e2ba6b1943a7673dc18860c0553d7b41e00775
2021-03-15 19:46:59 +01:00
def expect_wallet_notify(self, tx_details):
self.wait_until(lambda: len(os.listdir(self.walletnotify_dir)) >= len(tx_details), timeout=10)
# Should have no more and no less files than expected
assert_equal(sorted(notify_outputname(self.wallet, tx_id) for tx_id, _, _ in tx_details), sorted(os.listdir(self.walletnotify_dir)))
# Should now verify contents of each file
for tx_id, blockheight, blockhash in tx_details:
fname = os.path.join(self.walletnotify_dir, notify_outputname(self.wallet, tx_id))
# Wait for the cached writes to hit storage
self.wait_until(lambda: os.path.getsize(fname) > 0, timeout=10)
Merge #21141: wallet: Add new format string placeholders for walletnotify 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Add new format string placeholders for walletnotify to include relevant block information for transactions (Maayan Keshet) Pull request description: This patch includes two new format placeholders for walletnotify: %b - the hash of the block containting the transaction (zeroed if a mempool transaction) %h - the height of the block containing the transaction (zero if a mempool transaction) I've included test suite changes to check and validate the above functional requirements as well as doc/help description changes. **Motivation** The walletnotify option is used to be notified of new transactions relevant to the wallet of the node. A common usage pattern is to perform afterwards additional RPC calls to determine: 1. If this is a mempool transaction or not (i.e. are there any confirmations?) 2. What block was it included in? 3. Did this transaction was seen before and is now seen again because of a fork? All of these questions can be answered with the current features, but the resulting RPC calls may be expensive in a heavily used node. As this information is readily available when calling the walletnotify callback, it makes sense to save expensive round trips by optionally sending this information at that point in time. I can definitely say we would like to use it in Fireblocks, my employer. Please let me know of any questions and suggestions. ACKs for top commit: laanwj: ACK 06e1fb0b170a69996a7ce1ef5203785a7bc6b278 Tree-SHA512: d2744e2a7a883f9c3a9fd32237110e048c4b6b11fea8221c33d10b74157f65bbc4351211f441e8c1a4af5d5d38e2ba6b1943a7673dc18860c0553d7b41e00775
2021-03-15 19:46:59 +01:00
with open(fname, 'rt', encoding='utf-8') as f:
text = f.read()
# Universal newline ensures '\n' on 'nt'
assert_equal(text[-1], '\n')
text = text[:-1]
if os.name == 'nt':
# On Windows, echo as above will append a whitespace
assert_equal(text[-1], ' ')
text = text[:-1]
expected = str(blockheight) + '_' + blockhash
assert_equal(text, expected)
for tx_file in os.listdir(self.walletnotify_dir):
os.remove(os.path.join(self.walletnotify_dir, tx_file))
if __name__ == '__main__':
NotificationsTest().main()