Commit Graph

1465 Commits

Author SHA1 Message Date
Konstantin Akimov
7228d918f6
fix(tests): fix for default initialization of structures in messages.py (#5008)
Excepted zeroes in non-initialized data, in-fact there are non-zero characters.

>>> print(b'\\x0'.hex())
5c7830
>>> print(b'\x00'.hex())
00
2022-09-19 13:04:20 +04:00
MarcoFalke
566889f5e8
Merge #19208: test: move sync_blocks and sync_mempool functions to test_framework.py
cc84460c164bcb2a874d4f08b3a2624e5ee9ff0a test: move sync_blocks and sync_mempool functions to test_framework.py (Roy Shao)

Pull request description:

  This PR moves `sync_blocks` and `sync_mempool` out from `test_framework/util.py` to `test_framework/test_framework.py` so they can take contextual information of test framework into account.

  * Change all reference callers to call functions from `test_framework.py`
  * Remove `**kwargs` which is not used
  * Take into account of `timeout_factor` when respecting timeout in function implementations.
  * Pass all tests by running `./test/functional/test_runner.py`

  fixes #18930

ACKs for top commit:
  MarcoFalke:
    ACK cc84460c164bcb2a874d4f08b3a2624e5ee9ff0a , reviewed with --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space 💫

Tree-SHA512: a79b2a3fa842fc26a7aacb834bb2aea88b3049916c0b754e60002a77ce94bb5954e0ea3b436bf268e9295efb62d721dfef263a09339a55c684ac3fda388c275e
2022-09-08 03:49:30 +03:00
MarcoFalke
4b70ee4f24
Merge #18474: test: check that peer is connected when calling sync_*
fac3716b09bb9ee121db629873d9694a95cae942 test: check that peer is connected when calling sync_* (MarcoFalke)

Pull request description:

  Without a connection there is no way to sync, so we can fail early and don't have to wait for the timeout

ACKs for top commit:
  jonatack:
    ACK fac3716b09bb9

Tree-SHA512: 12f771473c23e152dae4bfb201fadb2c3530cf439de64fea07d048734614543080a5d05c9c36e6e398c6a69c8279f609d34706599571814172a11bcfbea4a3b9
2022-09-08 01:40:49 +03:00
MarcoFalke
c0e3829631
Merge #18986: tests: Add capability to disable RPC timeout in functional tests
38c3dd9c706e7e84b2a4dbaf1424a3f1c3b694fc docs: Add notes on how to diasble rpc timeout in functional tests while attatching gdb. (codeShark149)
784ae096259955ea7a294114f5c021c9489d2717 test: Add capability to disable RPC timeout in functional tests. (codeShark149)

Pull request description:

  Many times, especially while debugging RPC callbacks to core using gdb, the test timeout kicks in before the response can get back. This can be annoying and requires restarting the functional test as well as gdb attachment.

  This PR adds a `--notimeout` flag into `test_framework` and sets the `rpc_timeout` accordingly if the flag is set.

  The same effect can be achieved with newly added `--factor` flag but keeping a separate flag that explicitly disables the timeout can be easier for new testers to find it out and separates its purpose from the `--factor` flag.

  Requesting review ryanofsky jnewbery as per the IRC discussion.

  Update: After initial round of review, the approach is modified to accommodate the functionality in already existing `--factor` flag. `--factor` is changed to `--timeout-factor` to express its intent better.

ACKs for top commit:
  MarcoFalke:
    ACK 38c3dd9c706e7e84b2a4dbaf1424a3f1c3b694fc and thanks for fixing up all my typos 😅
  jnewbery:
    ACK 38c3dd9c706e7e84b2a4dbaf1424a3f1c3b694fc.

Tree-SHA512: 9458dd1010288c62f8bb83f7a4893284fbbf938882dd65fc9e08810a910db07ef676e3100266028e5d4c8ce407b2267b3860595015da070c84a9d4a9816797db
2022-09-08 00:40:11 +03:00
MarcoFalke
60cfe097be
Merge #18247: test: Wait for both veracks in add_p2p_connection
faf1d047313e71658fb31f6b94fdd5d37705ab85 test: Remove redundant sync_with_ping after add_p2p_connection (MarcoFalke)
fa9064704524a0fd1fa9ea73eea45b07316ac3d1 test: Wait for both veracks in add_p2p_connection (MarcoFalke)

Pull request description:

  This fixes the race in p2p_blocksonly

  E.g. https://travis-ci.org/MarcoFalke/bitcoin-core/jobs/657038844#L4500

  ```
   ...
   test  2020-03-01T20:58:28.825000Z TestFramework.mininode (DEBUG): Closed connection to: 127.0.0.1:11828
   node0 2020-03-01T20:58:28.825642Z [net] disconnecting peer=0
   node0 2020-03-01T20:58:28.825826Z [net] Cleared nodestate for peer=0
   node0 2020-03-01T20:58:28.875835Z [http] Received a POST request for / from 127.0.0.1:53448
   node0 2020-03-01T20:58:28.876067Z [httpworker.0] ThreadRPCServer method=getmempoolinfo user=__cookie__
   test  2020-03-01T20:58:28.877000Z TestFramework.mininode (DEBUG): Connecting to Bitcoin Node: 127.0.0.1:11828
   test  2020-03-01T20:58:28.878000Z TestFramework.mininode (DEBUG): Connected & Listening: 127.0.0.1:11828
   test  2020-03-01T20:58:28.878000Z TestFramework.mininode (DEBUG): Send message to 127.0.0.1:11828: msg_version(nVersion=70014 nServices=9 nTime=Sun Mar  1 20:58:28 2020 addrTo=CAddress(nServices=1 ip=127.0.0.1 port=11828) addrFrom=CAddress(nServices=1 ip=0.0.0.0 port=0) nNonce=0x164D5DEB952A4A0B strSubVer=b'/python-mininode-tester:0.0.3/' nStartingHeight=-1 nRelay=1)
   node0 2020-03-01T20:58:28.883808Z [net] Added connection peer=1
   node0 2020-03-01T20:58:28.883950Z [net] connection from 127.0.0.1:33798 accepted
   node0 2020-03-01T20:58:28.884300Z [msghand] received: version (116 bytes) peer=1
   node0 2020-03-01T20:58:28.884483Z [msghand] sending version (114 bytes) peer=1
   node0 2020-03-01T20:58:28.884700Z [msghand] send version message: version 70015, blocks=200, us=[::]:0, peer=1
   node0 2020-03-01T20:58:28.884765Z [msghand] sending verack (0 bytes) peer=1
   test  2020-03-01T20:58:28.885000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_version(nVersion=70015 nServices=1033 nTime=Sun Mar  1 20:58:28 2020 addrTo=CAddress(nServices=0 ip=0.0.0.0 port=0) addrFrom=CAddress(nServices=1033 ip=0.0.0.0 port=0) nNonce=0x4A0F2F4C549B3399 strSubVer=b'/Satoshi:0.19.99(testnode0)/' nStartingHeight=200 nRelay=0)
   test  2020-03-01T20:58:28.885000Z TestFramework.mininode (DEBUG): Send message to 127.0.0.1:11828: msg_verack()
   test  2020-03-01T20:58:28.885000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_verack()
   node0 2020-03-01T20:58:28.885004Z [msghand] receive version message: /python-mininode-tester:0.0.3/: version 70014, blocks=-1, us=127.0.0.1:11828, peer=1
   test  2020-03-01T20:58:28.886000Z TestFramework (INFO): Check that txs from rpc are not rejected and relayed to other peers
   node0 2020-03-01T20:58:28.886556Z [http] Received a POST request for / from 127.0.0.1:53448
   node0 2020-03-01T20:58:28.886783Z [httpworker.1] ThreadRPCServer method=getpeerinfo user=__cookie__
   node0 2020-03-01T20:58:28.889032Z [http] Received a POST request for / from 127.0.0.1:53448
   node0 2020-03-01T20:58:28.889294Z [httpworker.2] ThreadRPCServer method=testmempoolaccept user=__cookie__
   node0 2020-03-01T20:58:28.891655Z [http] Received a POST request for / from 127.0.0.1:53448
   node0 2020-03-01T20:58:28.891963Z [httpworker.3] ThreadRPCServer method=sendrawtransaction user=__cookie__
   node0 2020-03-01T20:58:28.893115Z [httpworker.3] Enqueuing TransactionAddedToMempool: txid=af34fc5ff9ea8babbd4083fbb79ffd2ad5aff1d6def803c07ca5aeed880bd60f wtxid=af34fc5ff9ea8babbd4083fbb79ffd2ad5aff1d6def803c07ca5aeed880bd60f
   node0 2020-03-01T20:58:28.893443Z [scheduler] TransactionAddedToMempool: txid=af34fc5ff9ea8babbd4083fbb79ffd2ad5aff1d6def803c07ca5aeed880bd60f wtxid=af34fc5ff9ea8babbd4083fbb79ffd2ad5aff1d6def803c07ca5aeed880bd60f
   node0 2020-03-01T20:58:28.894814Z [msghand] received: verack (0 bytes) peer=1
   node0 2020-03-01T20:58:28.894937Z [msghand] sending sendheaders (0 bytes) peer=1
   node0 2020-03-01T20:58:28.895087Z [msghand] sending sendcmpct (9 bytes) peer=1
   node0 2020-03-01T20:58:28.895235Z [msghand] sending sendcmpct (9 bytes) peer=1
   node0 2020-03-01T20:58:28.895430Z [msghand] sending ping (8 bytes) peer=1
   node0 2020-03-01T20:58:28.895896Z [msghand] initial getheaders (199) to peer=1 (startheight:-1)
   test  2020-03-01T20:58:28.896000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_sendheaders()
   node0 2020-03-01T20:58:28.896016Z [msghand] sending getheaders (645 bytes) peer=1
   node0 2020-03-01T20:58:28.896607Z [msghand] sending feefilter (8 bytes) peer=1
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_sendcmpct(announce=False, version=2)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_sendcmpct(announce=False, version=1)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_ping(nonce=f735096062d217b5)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Send message to 127.0.0.1:11828: msg_pong(nonce=f735096062d217b5)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_getheaders(locator=CBlockLocator(nVersion=70014 vHave=[48924041037103782797700918670732352379567180837453042168545380831411841797392, 28010422273815860773972769588722664110955084223364219183119416607410792753789, 5954376895683677137597080246740451260829355661937599865380797589540815086241, 14500403275336359851183244421245184901482464358719551678581030092830439955257, 17853919108052771837249729512111680264864054213441538187113939176285784834878, 28843166929059356839755035875664073555480989477... (msg truncated)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_feefilter(feerate=000003e8)
   node0 2020-03-01T20:58:28.898144Z [msghand] received: pong (8 bytes) peer=1
   node0 2020-03-01T20:59:28.338539Z [scheduler] Feeding 13446 bytes of dynamic environment data into RNG
   test  2020-03-01T20:59:28.908000Z TestFramework.utils (ERROR): wait_until() failed. Predicate: ''''
                                             def test_function():
                                                 assert self.is_connected
                                                 if not self.last_message.get('tx'):
                                                     return False
                                                 return self.last_message['tx'].tx.rehash() == txid
                                     '''
   test  2020-03-01T20:59:28.908000Z TestFramework (ERROR): Assertion failed
                                     Traceback (most recent call last):
                                       File "/home/travis/build/MarcoFalke/bitcoin-core/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 112, in main
                                         self.run_test()
                                       File "/home/travis/build/MarcoFalke/bitcoin-core/build/bitcoin-x86_64-pc-linux-gnu/test/functional/p2p_blocksonly.py", line 57, in run_test
                                         self.nodes[0].p2p.wait_for_tx(txid)
                                       File "/home/travis/build/MarcoFalke/bitcoin-core/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/mininode.py", line 369, in wait_for_tx
                                         wait_until(test_function, timeout=timeout, lock=mininode_lock)
                                       File "/home/travis/build/MarcoFalke/bitcoin-core/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/util.py", line 234, in wait_until
                                         raise AssertionError("Predicate {} not true after {} seconds".format(predicate_source, timeout))
                                     AssertionError: Predicate ''''
                                             def test_function():
                                                 assert self.is_connected
                                                 if not self.last_message.get('tx'):
                                                     return False
                                                 return self.last_message['tx'].tx.rehash() == txid
                                     ''' not true after 60 seconds

ACKs for top commit:
  jonatack:
    ACK faf1d04

Tree-SHA512: 3b1a38a5c87d11c610eee0988f0c4af9bfcd978df9ac718ef611f663df2fd4a0eb04e077df5e940d15971bb2f22328fb6021cacccb6902f1e527f288ad2c4a2c
2022-09-08 00:02:35 +03:00
MarcoFalke
6bb3e54578
Merge #18617: test: add factor option to adjust test timeouts
2742c3428633b6ceaab6714635dc3adb74bf121b test: add factor option to adjust test timeouts (Harris)

Pull request description:

  This PR adds a new option **factor** that can be used to adjust timeouts in various functional tests.
  Several timeouts and functions from `authproxy`, `mininode`, `test_node` and `util` have been adapted to use this option. The factor-option definition is located in `test_framework.py`.

  Fixes https://github.com/bitcoin/bitcoin/issues/18266
  Also Fixes https://github.com/bitcoin/bitcoin/issues/18834

ACKs for top commit:
  MarcoFalke:
    Thanks! ACK 2742c3428633b6ceaab6714635dc3adb74bf121b

Tree-SHA512: 6d8421933ba2ac1b7db00b70cf2bc242d9842c48121c11aadc30b0985c4a174c86a127d6402d0cd73b993559d60d4f747872d21f9510cf4806e008349780d3ef
2022-09-08 00:02:35 +03:00
Wladimir J. van der Laan
86a11a83a2
Merge #18873: test: Fix intermittent sync_blocks failures
fa3f9a05660687bf4146e089050e944a1d6cbe3c test: Fix intermittent sync_blocks failures (MarcoFalke)

Pull request description:

  Fixes #18872
  Fixes #18737
  Fixes #18801

  See docstring for motivation and description

ACKs for top commit:
  laanwj:
    Code review ACK fa3f9a05660687bf4146e089050e944a1d6cbe3c

Tree-SHA512: acd52d386a6849f7ff1cb1a51a439dc3c76e0e3a4dd8d22030df0ebb09b44497c61c56331ff65724b695d82d86b0ebb24608f9e637008e5dacb7676b0c448889
2022-09-07 22:25:40 +03:00
MarcoFalke
31a886aa7c
Merge #16681: Tests: Use self.chain instead of 'regtest' in all current tests
1abcecc40c518a98b7d17880657ec0247abdf125 Tests: Use self.chain instead of 'regtest' in almost all current tests (Jorge Timón)

Pull request description:

  Simply avoiding the hardcoded string in more places for consistency.
  It can also allow for more easily reusing tests for other chains other than regtest.

  Separated from #8994 .
  Continues #16509 .

  It is still not complete (ie to be complete, we need the -chain parameter in #16680 and make whether acceptnonstdtxs is allowed for that chain or not customizable for regtest [or for custom chains like in #8994 ] ). But while being incomplete like #16509 , it's quite simple to review and another step forward IMO.

ACKs for top commit:
  Sjors:
    re-ACK 1abcecc. I think it's an improvement even if incomplete and if some PR's might accidentally bring "regtest" back. Subsequent improvements hopefully don't have to touch 16 files.
  elichai:
    Code review ACK 1abcecc40c518a98b7d17880657ec0247abdf125
  ryanofsky:
    Code review ACK 1abcecc40c518a98b7d17880657ec0247abdf125.
  ryanofsky:
    Code review ACK 1abcecc40c518a98b7d17880657ec0247abdf125

Tree-SHA512: 5620de6dab235ca8bd8670d6366c7b9f04f0e3ca9c5e7f87765b38e16ed80c17d7d1630c0d5fd7c5526f070830d94dc74cc2096d8ede87dc7180ed20569509ee
2022-09-07 22:25:40 +03:00
MarcoFalke
ba73a3360a
Merge #17288: Added TestShell class for interactive Python environments.
19139ee034d20ebab1b91d3ac13a8eee70b59374 Add documentation for test_shell submodule (JamesC)
f5112369cf91451d2d0bf574a9bfdaea04696939 Add TestShell class (James Chiang)
5155602a636c323424f75272ccec38588b3d71cd Move argparse() to init() (JamesC)
2ab01462f48b2d4e0d03ba842c3af8851c67c6f1 Move assert num_nodes is set into main() (JamesC)
614c645643e86c4255b98c663c10f2c227158d4b Clear TestNode objects after shutdown (JamesC)
6f40820757d25ff1ccfdfcbdf2b45b8b65308010 Add closing and flushing of logging handlers (JamesC)
6b71241291a184c9ee197bf5f0c7e1414417a0a0 Refactor TestFramework main() into setup/shutdown (JamesC)
ede8b7608e115364b5bb12e7f39d662145733de6 Remove network_event_loop instance in close() (JamesC)

Pull request description:

  This PR refactors BitcoinTestFramework to encapsulate setup and shutdown logic into dedicated methods, and adds a  ~~TestWrapper~~ TestShell child class. This wrapper allows the underlying BitcoinTestFramework to run _between user inputs_ in a REPL environment, such as a Jupyter notebook or any interactive Python3 interpreter.

  The ~~TestWrapper~~ TestShell is motivated by the opportunity to expose the test-framework as a prototyping and educational toolkit. Examples of code prototypes enabled by  ~~TestWrapper~~ TestShell can be found in the Optech [Taproot/Schnorr](https://github.com/bitcoinops/taproot-workshop) workshop repository.

  Usage example:
  ```
  >>> import sys
  >>> sys.path.insert(0, "/path/to/bitcoin/test/functional")
  ```
  ```
  >>> from test_framework.test_wrapper import TestShell
  >>> test = TestShell()
  >>> test.setup(num_nodes=2)
  20XX-XX-XXTXX:XX:XX.XXXXXXX TestFramework (INFO): Initializing test directory /path/to/bitcoin_func_test_XXXXXXX
  ```
  ```
  >>> test.nodes[0].generate(101)
  >>> test.nodes[0].getblockchaininfo()["blocks"]
  101
  ```
  ```
  >>> test.shutdown()
  20XX-XX-XXTXX:XX:XX.XXXXXXX TestFramework (INFO): Stopping nodes
  20XX-XX-XXTXX:XX:XX.XXXXXXX TestFramework (INFO): Cleaning up /path/to/bitcoin_func_test_XXXXXXX on exit
  20XX-XX-XXTXX:XX:XX.XXXXXXX TestFramework (INFO): Tests successful
  ```

  **Overview of changes to BitcoinTestFramework:**

  - Code moved to `setup()/shutdown()` methods.
  - Argument parsing logic encapsulated by `parse_args` method.
  - Success state moved to `BitcoinTestFramework.success`.

  _During Shutdown_

  - `BitcoinTestFramework` logging handlers are flushed and removed.
  - `BitcoinTestFrameowork.nodes` list is cleared.
  - `NetworkThread.network_event_loop` is reset. (NetworkThread class).

  **Behavioural changes:**
  - Test parameters can now also be set when overriding BitcoinTestFramework.setup() in addition to overriding `set_test_params` method.
  - Potential exceptions raised in BitcoinTestFramework.setup() will be handled in main().

  **Added files:**
  - ~~test_wrapper.py~~ `test_shell.py`
  - ~~test-wrapper.md~~ `test-shell.md`

ACKs for top commit:
  jamesob:
    ACK 19139ee034
  jonatack:
    ACK 19139ee034d20ebab1b91d3ac13a8eee70b59374
  jnewbery:
    Rather than invalidate the three ACKs for a minor nit, can you force push back to 19139ee034d20ebab1b91d3ac13a8eee70b59374 please? I think this PR was ready to merge before your last force push.
  jachiang:
    > Rather than invalidate the three ACKs for a minor nit, can you force push back to [19139ee](19139ee034) please? I think this PR was ready to merge before your last force push.
  jnewbery:
    ACK 19139ee034d20ebab1b91d3ac13a8eee70b59374

Tree-SHA512: 0c24f405f295a8580a9c8f1b9e0182b5d753eb08cc331424616dd50a062fb773d3719db4d08943365b1f42ccb965cc363b4bcc5beae27ac90b3460b349ed46b2
2022-09-07 21:45:31 +03:00
MarcoFalke
9e41907692
Merge #16042: test: Bump MAX_NODES to 12
fa47330397 test: Speed up cache creation (MarcoFalke)
fa6ad7a5ec test: Bump MAX_NODES to 12 (MarcoFalke)

Pull request description:

  When testing a combination of settings that affect the datadir (e.g. prune, blockfilter, ...) we may need a lot of datadirs.
  Bump the maximum number of nodes proactively from 8 to 12, so that caches get populated with 12 node dirs, as opposed to 8.

  Also, add an assert that the list of deterministic keys is exactly the number of max nodes (and not more than that.

  Also, create the cache faster.

ACKs for commit fa4733:
  laanwj:
    utACK fa473303972b7dad600d949dc9b303d8136cb7e7

Tree-SHA512: 9803c765ed52d344102f5a3bce57b05d88a7429dcb05ed66ed6c881fda8d87c2834d02d21b95fe9f39c0efe3b8527e13cf94f006588cde22e8c2cd50b2d517a6
2022-09-07 20:15:52 +03:00
MarcoFalke
5524c42b8c
Merge #15419: qa: Always refresh cache to be out of ibd
fa2cdc9ac2 test: Simplify create_cache (MarcoFalke)
fa25210d62 qa: Fix wallet_txn_doublespend issue (MarcoFalke)
1111aecbb5 qa: Always refresh stale cache to be out of ibd (MarcoFalke)
fab0d85802 qa: Remove mocktime unless required (MarcoFalke)

Pull request description:

  When starting a test, we are always in IBD because the timestamps on cached blocks are in the past. Usually, we solve that by generating a block at the beginning of the test.

  That is clumsy and might even lead to other problems such as #15360 and https://github.com/bitcoin/bitcoin/issues/14446#issuecomment-461926598

  So fix that by getting rid of mocktime and always refreshing the last block of the cache when starting the test framework.

  Should fix #14446

Tree-SHA512: 6af09800f9c86131349a103af617a54551f5f3f3260d38e14e3f30fdd3d91a0feb0100c56cbb12eae4aeac5571ae4b530b16345cbb831d2670237b53351a22c1
2022-09-07 20:15:43 +03:00
UdjinM6
44fda52d68
Merge pull request #4925 from PastaPastaPasta/develop-trivial-2022-07-17
backport: trivial backports
2022-09-06 20:35:53 +03:00
MarcoFalke
e0ba1fc72d
Merge #19624: Warn on unknown rw_settings
fa48405ef84985e5a9d38ec38e90d16596ea45b5 Warn on unknown rw_settings (MarcoFalke)

Pull request description:

  Log a warning to debug log if unknown settings are encountered. This should probably only ever happen when the software is upgraded.

  Something similar is already done for the command line and config file. See:

  * test: Add test for unknown args #16234 (commit fa7dd88b71a1c6641bd450fae29a4a31849b1afd)

ACKs for top commit:
  ryanofsky:
    Code review ACK fa48405ef84985e5a9d38ec38e90d16596ea45b5. Looks good and I could see this being helpful for debugging. Thanks for taking suggestions

Tree-SHA512: cec7d88adf84fa0a842f56b26245157736eb50df433db951e622ea07fd145b899822b24cdab1d8b36c066415ce4f0ef09b493fa8a8d691532822a59c573aafa7
2022-09-03 10:43:15 -05:00
UdjinM6
b30005d246
fix(rpc): masternode outputs should work correctly with multiple outputs created via a single tx (#4967)
* fix(rpc): `masternode outputs` should produce an array

or it would ignore all but 1 outputs produced by the same tx

* rpc: improve `masternode outputs` help

* tests: check that `masternode outputs` show all outputs produced in the same tx
2022-08-31 13:32:35 +03:00
MarcoFalke
be35f1bc67
Merge #18069: test: replace 'regtest' leftovers by self.chain
eca56f89293b74f11ca631ff2a0793e970e65841 test: replace 'regtest' leftovers by self.chain (Sebastian Falbesoner)

Pull request description:

  This is a follow-up PR to #16681 (fixes #18068), replacing all remaining hardcoded `"regtest"` strings in functional tests by `self.chain`.

Top commit has no ACKs.

Tree-SHA512: 96524649b33164938e5a95215991103ed7855ebab55ef788d4816b3fa5cbc03d8f3b0d39f2247a87522f289fd7f4daf25e059900b8462b5127eb154bbee89054
2022-08-30 09:08:17 -05:00
UdjinM6
ea957a73a0
Merge pull request #4845 from Munkybooty/backports-0.20-pr3
Backports 0.20 pr3
2022-08-29 12:41:20 +03:00
PastaPastaPasta
585eb4f14b
Merge pull request #4971 from knst/bc-bp-cwallet-3
Bitcoin backports with CWallet refactoring #17300, #16237, #16301, #16383, #16798, #16900
2022-08-28 18:26:35 -04:00
Kittywhiskers Van Gogh
71e8caf4b9
refactor: migrate globals to managed pointers in preparation for deglobalization (#4930)
* coinjoin: make CCoinJoinServer managed pointer, assign CConnman during init

* coinjoin: make CCoinJoinClientQueueManager managed pointer, assign CConnman during init

* sporks: move spork validation logic downwards after CConnman initialization

* sporks: make CSporkManager a pointer, reduce global invocations

* governance: make CGovernanceManager a pointer, reduce global invocations

* llmq: migrate LLMQ subsystem raw pointers to managed pointers

* masternode: make activeMasternodeManager a managed pointer

* masternode: make masternodeSync a managed pointer, assign CConnman during init

* refactor: make instantsend helper functions class members

* fix: send empty CDeterministicMNList if pointer isn't initialized yet

* fix: refactor governance object retrieval logic across node and ui

Update src/interfaces/node.cpp

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-08-26 16:52:53 -05:00
Wladimir J. van der Laan
bd642f66d8 Merge #17488: test: fix "bitcoind already running" warnings on macOS
1c23ea5fe67b88fd72a1ff640dd1bbb21a34fbf4 test: fix bitcoind already running warnings on macOS (fanquake)

Pull request description:

  On macOS, `pidof` installed via brew returns b'' rather than None.
  Account for this, to remove spurious warnings from the test_runner.

ACKs for top commit:
  laanwj:
    ACK 1c23ea5fe67b88fd72a1ff640dd1bbb21a34fbf4

Tree-SHA512: 640f4323d4105eac5c7abb52daf80486d5d3b4a074720490ceeb97c3dd8d73a3de9a988d2550f1e2076c620bb10d452b2959d8b723d2ee64f499878909824e31
2022-08-24 14:29:45 -04:00
MarcoFalke
640616ff10 Merge #17469: test: Remove fragile assert_memory_usage_stable
fac942ca57dce6cfa5655a3ac8664d6a051bc01f test: Remove fragile assert_memory_usage_stable (MarcoFalke)

Pull request description:

  This test fails on arm64 and a fuzz tests seems inappropriate for the functional test suite anyway, so remove it.

  Example failures:

  * https://travis-ci.org/bitcoin/bitcoin/jobs/611497963#L14517
  * https://travis-ci.org/MarcoFalke/bitcoin-core/jobs/611029104#L3876

ACKs for top commit:
  jamesob:
    ACK fac942ca57

Tree-SHA512: 3577e7ce5891d221cb798454589ba796ed0c06621a26351bb919c23bc6bb46aafcd0b11cb02bbfde64b74d67cb2950da44959a7ecdc436491a34e8b045c1ccf4
2022-08-24 14:29:45 -04:00
MarcoFalke
0a5587c2f7 Merge #17435: test: check custom ancestor limit in mempool_packages.py
49997813a4db388b2810e5e27ef771e8aa6a1f03 test: check custom ancestor limit in mempool_packages.py (Sebastian Falbesoner)

Pull request description:

  The functional test `mempool_packages.py` starts one node with default ancestor/descendant limit settings and one with a custom, reduced ancestor limit (currently `-limitancestorcount=5`). The effect of the latter had not been tested yet though. This is approached in this PR by checking on the expected mempool contents of node1 after the node0 ancestor tests are done, via the following three conditions:
  - the # of txs in the node1 mempool is equal to the the limit
  - all txs in node1 mempool are a subset of txs in node0 mempool
  - the node1 mempool txs match the start of the constructed tx-chain

  Note that this still doesn't *fully* check the expected mempool of node1 (e.g. that it isn't influenced by `prioritisetransaction` RPC on node0), hence I add another TODO. In the future it would make sense to also set a custom descendant limit when the second TODO about checking node1's mempool is approached: 89e93135ae/test/functional/mempool_packages.py (L228)

ACKs for top commit:
  MarcoFalke:
    ACK 49997813a4db388b2810e5e27ef771e8aa6a1f03 👲

Tree-SHA512: d3a1d19fb49731238ad08ee7c02e2fa81a227e3b4ef3340d68598de42ddb62be9161134f6b8e08fa76b8c9faa02fecfa01111159642e20e9f358292a757b7608
2022-08-24 14:29:45 -04:00
MarcoFalke
cb1a2f3d5e Merge #17340: Tests: speed up fundrawtransaction test
af7bae734089f6af0029b0887932ccd9a469e12e [tests] Don't stop-start unnecessarily in rpc_fundrawtransaction.py (John Newbery)
9a8505299ba392acbab4647963113b0c29495f1d [tests] Use -whitelist in rpc_fundrawtransaction.py (John Newbery)
646b593bbd0db113c6e45ab92177b8f5251e8710 [tests] Speed up rpc_fundrawtransaction.py (John Newbery)

Pull request description:

  Speed up rpc_fundrawtransaction.py

  Most of the time in rpc_fundrawtransaction.py is spent waiting for
  unconfirmed transactions to propagate. Net processing adds a poisson
  random delay to the time it will INV transactions with a mean interval
  of 5 seconds. Calls like the following:

  ```
  self.nodes[2].sendrawtransaction(signedTx['hex'])
  self.sync_all()
  self.nodes[1].generate(1)
  ````

  will therefore introduce a delay waiting for the mempools to sync.
  Instead just generate the block on the node that sent the transaction:

  ```
  self.nodes[2].sendrawtransaction(signedTx['hex'])
  self.nodes[2].generate(1)
  ```

  rpc_fundrawtransaction.py is not intended to be a test for transaction
  relay, so it's ok to do this.

ACKs for top commit:
  MarcoFalke:
    ACK af7bae734089f6af0029b0887932ccd9a469e12e 🛴

Tree-SHA512: db3407d871bfdc99a02e7304b07239dd3585ac47f27f020f1a70608b7f6386b134343c01f3e4d1c246ce734676755897671999695068d6388602fb042d178780
2022-08-24 14:29:45 -04:00
Wladimir J. van der Laan
0882c487e3 Merge #17382: rpc: Remove unused boost::this_thread::interruption_point
fa5facd3e72b6d61374b0b93b722b55e2b090020 rpc: Remove unused boost::this_thread::interruption_point (MarcoFalke)

Pull request description:

  There are predefined interruption points for `boost::thread`: https://www.boost.org/doc/libs/1_71_0/doc/html/thread/thread_management.html#interruption_points

  However, the rpc threads are `std::thread`, which does not have an `std:🧵:interrupt` member function to request interruption: https://dev.visucore.com/bitcoin/doxygen/httpserver_8cpp.html#ae1a63374e18b9abd348eb74e4243ea34

  Thus, the interruption points can be removed.

ACKs for top commit:
  laanwj:
    ACK fa5facd3e72b6d61374b0b93b722b55e2b090020, this does nothing.
  practicalswift:
    ACK fa5facd3e72b6d61374b0b93b722b55e2b090020
  jamesob:
    ACK fa5facd3e7

Tree-SHA512: 4e29a44df1f2702cbd1ffdffa559440a8bb800baab64b4116e2c3d27cd64d8d1e8aafe1dc21b1a4e3988470d03be19cae294bd5669f7abf6d487685dc8fd8d7e
2022-08-24 14:29:45 -04:00
Samuel Dobson
d839dd4c1e Merge #17258: Fix issue with conflicted mempool tx in listsinceblock
436ad436434b94982bcb7dc1d13a21949263ef73 Fix issue with conflicted mempool tx in listsinceblock (Adam Jonas)

Pull request description:

  Closes #8752 by bringing back abandoned #10470.

  This now checks that returned transactions are not conflicting with any transactions that are filtered out by the given blockhash and add a functional test to prevent this in the future.

  For more context, #8757 was closed in favor of #10470.

ACKs for top commit:
  instagibbs:
    utACK 436ad43643
  kallewoof:
    utACK 436ad436434b94982bcb7dc1d13a21949263ef73
  jonatack:
    I'm not qualifed to give an ACK here but 436ad436434b94982bcb7dc1d13a21949263ef73 appears reasonable. Built/ran tests/verified that this test fails without the change in rpcwallet.cpp:

Tree-SHA512: 63d75cd3d3f19fc84dc38899b200c96179b82b24db263cd0116ee5b715265be647157855c2e35912d2fbc49c7b37db9375d6aab0ac672f0f09bece8431de5ea9
2022-08-24 14:29:45 -04:00
UdjinM6
c078e666a4
feat(rpc): Add service field for each member in quorum info results (#4977)
* feat(rpc): Add `service` field for each member in `quorum info` results

* add test
2022-08-22 22:43:07 +03:00
fanquake
0fe885f5de Merge #16383: rpcwallet: default include_watchonly to true for watchonly wallets
72eaab073bc747425fe551777154b13a6c4c37c9 tests: functional watch-only wallet tests (William Casarin)
72ffbdc5799c1707ecad674d701b43fb80b031d0 doc: add release note for include_watchonly default changes (William Casarin)
003a3c73c0450aa18ac2ab2ca47def2b8c53a7df rpcwallet: document include_watchonly default for watchonly wallets (William Casarin)
a50d9e6c0b8e8144d3deec58ec2e3449ba081151 rpcwallet: default include_watchonly to true for watchonly wallets (William Casarin)

Pull request description:

  Right now it's a bit annoying to deal with watchonly wallets, many rpc commands have an `include_watchonly` argument that needs to be explicitly set.

  Wallets created with `createwallet` can have a `disable_private_keys` parameter, for those wallets we already know that they are watchonly, so there's no reason to have to explicitly ask for it for every command. Instead we check this wallet flag when the `include_watchonly` parameter isn't set.

ACKs for top commit:
  achow101:
    Code review ACK 72eaab073bc747425fe551777154b13a6c4c37c9
  Sjors:
    ACK 72eaab073bc747425fe551777154b13a6c4c37c9
  promag:
    ACK 72eaab073bc747425fe551777154b13a6c4c37c9, code review only, didn't look closely to the test.
  kallewoof:
    ACK 72eaab073bc747425fe551777154b13a6c4c37c9
  fanquake:
    ACK 72eaab073bc747425fe551777154b13a6c4c37c9 - I've looked over the changes, they make sense to me. Compiled and ran the tests etc.

Tree-SHA512: d3646b55e97f386594d7efc994f0712f3888475c6a5dc7f131ac9f8c49bf5d4677182b88f42b34152abe1ad101ecadd152b4c20e9d3c1267190db36f77ab8bd7
2022-08-14 20:18:21 +07:00
PastaPastaPasta
f83d857e36
Merge pull request #4478 from kittywhiskers/undashify
revert #3036, #2911, #1879, #1432: revert dashification of filenames
2022-08-11 12:34:55 -04:00
PastaPastaPasta
afbc817220
refactor/chore: update cppcheck to 2.8 with needed refactoring (#4926)
* refactor/chore: update cppcheck to 2.8 with needed refactoring

* use probably invalid index for default

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* trivial: rename skContributions -> m_sk_contributions and skContributions2 -> skContributions

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-08-11 02:05:44 +03:00
Kittywhiskers Van Gogh
887b4324d3 trivial: revert dashification of source and header filenames 2022-08-09 14:16:29 +05:30
Konstantin Akimov
ae051bb6e0
Merge #17260: Split some CWallet functions into new LegacyScriptPubKeyMan (#4938)
* Move wallet enums to walletutil.h

* MOVEONLY: Move key handling code out of wallet to keyman file

Start moving wallet and ismine code to scriptpubkeyman.h, scriptpubkeyman.cpp

The easiest way to review this commit is to run:

   git log -p -n1 --color-moved=dimmed_zebra

And check that everything is a move (other than includes and copyrights comments).

This commit is move-only and doesn't change code or affect behavior.

* Refactor: Split up CWallet and LegacyScriptPubKeyMan and classes

This moves CWallet members and methods dealing with keys to a new
LegacyScriptPubKeyMan class, and updates calling code to reference the new
class instead of CWallet.

Most of the changes are simple text replacements and variable substitutions
easily verified with:

    git log -p -n1 -U0 --word-diff-regex=.

The only nontrivial chunk of code added is the new LegacyScriptPubKeyMan class
declaration, but this code isn't new and is just selectively copied and moved
from the previous CWallet class declaration. This can be verified with:

    git log -p -n1 --color-moved=dimmed_zebra src/wallet/scriptpubkeyman.h src/wallet/wallet.h

or

    git diff HEAD~1:src/wallet/wallet.h HEAD:src/wallet/scriptpubkeyman.h

This commit does not change behavior.

* Renamed classes in scriptpubkeyman

* Fixes for conflicts, compilation and linkage errors due to previous commits

* Reordered methods in scriptpubkeyman to make further backports easier

* Reordered methods in scriptpubkeyman to make further backports easier (part II)

* Remove HDChain copy from SigningProvider class

* fixes/suggestions

Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-08-08 11:05:21 -05:00
Kittywhiskers Van Gogh
bb7d6aed99
refactor(llmq): substitute memberless class llmq::CLLMQUtils with namespace llmq::utils (#4931)
* refactor(llmq): substitute memberless class llmq::CLLMQUtils with namespace llmq::utils

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* chore: mark functions internal to `llmq::utils` as `static`

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-08-02 12:14:25 -05:00
UdjinM6
5a5278d398
fix/test: Count MN connections properly, add more tests for getnetworkinfo (#4928)
* fix: Count MN connections properly

* tests: check extended connections info returned via getnetworkinfo
2022-07-26 00:38:07 +03:00
UdjinM6
666859b47f
feat(llmq): Ensure connections between IS quorums (#4917)
* fix(llmq): Ensure connections between quorums

Every masternode will now "watch" a single node from _every other_ quorum in addition to intra-quorum connections. This should make propagation of recsigs produced by one quorum to other quorums much more reliable.

* fix: Do this only for masternodes which participate in IS quorums

* refactor: rename `CQuorumManager::EnsureQuorumConnections` to better match the actual behaviour

(and avoid confusion with `CLLMQUtils::EnsureQuorumConnections`)

* refactor: move IS quorums watch logic into `CQuorumManager::CheckQuorumConnections`

avoid calling slow `ScanQuorums` (no caching atm) inside the loop

* tests: check that inter-quorum connections are added

* use `ranges::any_of`
2022-07-18 22:26:51 +03:00
UdjinM6
0d67aa1915
fix: Pass KeyOriginInfo instead of CKeyMetadata to DeriveChildExtKey (#4918)
Resolves "hdchain -> wallet/walletdb -> ... -> hdchain" circular dependencies
2022-07-18 16:56:33 +03:00
Kittywhiskers Van Gogh
0d5a7929ac partial bitcoin#18047: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h) 2022-07-15 21:09:53 +05:30
UdjinM6
32051a35c0
fix(llmq): mark mns "bad" based on the failed connect attempts count (#4910)
* fix(llmq): mark mns "bad" based on the failed connect attempts count

Avoid using "last success time" as a proxy

* fix(tests): tweak feature_llmq_simplepose.py
2022-07-14 21:38:02 +03:00
Konstantin Akimov
0a951622f1
Bitcoin backport #16227 refactoring CWallet (#4903)
* Add HaveKey and HaveCScript to SigningProvider

* Remove CKeyStore and squash into CBasicKeyStore

* Move HaveKey static function from keystore to rpcwallet where it is used

* scripted-diff: rename CBasicKeyStore to FillableSigningProvider

-BEGIN VERIFY SCRIPT-
git grep -l "CBasicKeyStore" | xargs sed -i -e 's/CBasicKeyStore/FillableSigningProvider/g'
-END VERIFY SCRIPT-

* Move KeyOriginInfo to its own header file

* Move various SigningProviders to signingprovider.{cpp,h}

Moves all of the various SigningProviders out of sign.{cpp,h} and
keystore.{cpp,h}. As such, keystore.{cpp,h} is also removed.

Includes and the Makefile are updated to reflect this. Includes were largely
changed using:
git grep -l "keystore.h" | xargs sed -i -e 's;keystore.h;script/signingprovider.h;g'

* Remove CCryptoKeyStore and move all of it's functionality into CWallet

Instead of having a separate CCryptoKeyStore that handles the encryption
stuff, just roll it all into CWallet.

* Fixed cases of mess CWallet functions with CCryptoKeyStore and conflicts

* Move WatchOnly stuff from SigningProvider to CWallet

* Fixes for lint cirtular dependencies to calm linter

Co-authored-by: Andrew Chow <achow101-github@achow101.com>
2022-07-12 22:46:31 -05:00
PastaPastaPasta
eefdae1a53
Merge pull request #4829 from kittywhiskers/fuzz2
backport: bitcoin#18417, #18521, #18529, #18176, #18423, #17926, #18353, #18407, #18455, #18565, #18867 (fuzzing harness backports: part 2)
2022-07-12 12:32:28 -04:00
UdjinM6
a483122f5f
fix(net): Extend blocks-relay-only to also ignore some Dash-specific messages/invs (#4888)
* fix(net): Extend blocks-relay-only to also ignore some Dash-specific messages/invs

* Clarify few things
2022-07-07 18:11:38 +03:00
Kittywhiskers Van Gogh
d807cc7a8a merge bitcoin#17926: Add key_io fuzzing harness. Fuzz additional functions in existing fuzzing harnesses 2022-07-06 21:48:10 +05:30
Kittywhiskers Van Gogh
af2984b2ae merge bitcoin#18923: Never schedule MaybeCompactWalletDB when -flushwallet is off
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-07-03 00:14:47 +05:30
Kittywhiskers Van Gogh
e235d834da merge bitcoin#20671: Replace boost::optional with std::optional 2022-07-03 00:14:47 +05:30
UdjinM6
44d095b59c
Merge pull request #4880 from PastaPastaPasta/backports-19754
backport: 19754 and necessary pre-reqs
2022-07-02 21:01:26 +03:00
UdjinM6
def2e98d61
Merge pull request #4898 from PastaPastaPasta/backport-18530
Merge #18530: Add test for -blocksonly and -whitelistforcerelay param interaction
2022-06-30 22:06:15 +03:00
PastaPastaPasta
d89de3a2fa
Merge pull request #4866 from Munkybooty/backports-0.20-pr4
Backports 0.20 pr4
2022-06-30 10:08:59 -05:00
MarcoFalke
4d6f0cd2f2
Merge #18530: Add test for -blocksonly and -whitelistforcerelay param interaction
0ea5d70b4756f376342417e0019490233cb4a918 Updated comment for the condition where a transaction relay is denied (glowang)
be01449cc8eb7bb97531a967f5d1dcc7b8865d1e Add test for param interaction b/w -blocksonly and -whitelistforcerelay (glowang)

Pull request description:

  Related to: #18428

  When -blocksonly is turned on, a node would still relay transactions from whitelisted peers. This funcitonality has not been tested.

ACKs for top commit:
  MarcoFalke:
    ACK 0ea5d70b4756f376342417e0019490233cb4a918

Tree-SHA512: 4e99c88281cb518cc67f5f3be7171a7b413933047b5d24a04bb3ff2210a82e914d69079f64cd5bac9206ec435e21a622c8e69cedbc2ccb39d2328ac5c01668e5
2022-06-29 10:14:58 -05:00
Odysseas Gabrielides
088d7f2ec5
fix: Faster feature_llmq_rotation.py + introduction of llmq_devnet_dip0024 (#4878)
* Added LLMQ_DEVNET_V2

* Faster feature_llmq_rotation func test

* Updated llmq_devnet_dip0024

* Adjusted parameters

* Better comments

* Re-adjusted rotated llmq params
2022-06-28 19:14:09 -05:00
UdjinM6
ac7ed67d30
Merge pull request #4889 from PastaPastaPasta/develop-trivial-2022-06-18
Develop trivial 2022 06 18
2022-06-27 22:57:55 +03:00
PastaPastaPasta
c3f5f81a44
Merge pull request #4891 from kittywhiskers/antispam_bps
backport: bitcoin#15141, #19219 (rewrite DoS interface, use discouragement filter)
2022-06-22 17:11:56 -07:00
Vijay
6df949628f
Merge #15427: Add support for descriptors to utxoupdatepsbt (#4656)
26fe9b990995f9cb5eee21d40b4daaad19f7181f Add support for descriptors to utxoupdatepsbt (Pieter Wuille)
3135c1a2d2e2fb31bc362c848bd2456d576e408b Abstract out UpdatePSBTOutput from FillPSBT (Pieter Wuille)
fb90ec3c33e824f5abb6a68452c683d6ce8b3e4a Abstract out EvalDescriptorStringOrObject from scantxoutset (Pieter Wuille)
eaf4f887348a08c620732125ad4430e1a133d434 Abstract out IsSegWitOutput from utxoupdatepsbt (Pieter Wuille)

Pull request description:

  This adds a descriptors argument to the `utxoupdatepsbt` RPC. This means:
  * Input and output scripts and keys will be filled in when known.
  * P2SH-witness inputs will be filled in from the UTXO set when a descriptor is provided that shows they're spending segwit outputs.

  This also moves some (newly) shared code to separate functions: `UpdatePSBTOutput` (an analogue to `SignPSBTInput`), `IsSegWitOutput`, and `EvalDescriptorStringOrObject` (implementing the string or object notation parsing used in `scantxoutset`).

ACKs for top commit:
  jnewbery:
    utACK 26fe9b990995f9cb5eee21d40b4daaad19f7181f
  laanwj:
    utACK 26fe9b990995f9cb5eee21d40b4daaad19f7181f (will hold merging until response to promag's comments)
  promag:
    ACK 26fe9b9, checked refactors and tests look comprehensive. Still missing a release note but can be added later.

Tree-SHA512: 1d833b7351b59d6c5ded6da399ff371a8a2a6ad04c0a8f90e6e46105dc737fa6f2740b1e5340280d59e01f42896c40b720c042f44417e38dfbee6477b894b245

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
2022-06-22 16:42:19 -07:00