Commit Graph

129 Commits

Author SHA1 Message Date
Alexander Block
beaf3f40b2 Implement DashTestFramework.get_mninfo 2020-03-31 19:58:07 +02:00
UdjinM6
f43cdbc586
Gradually bump mocktime in wait_for_quorum_connections (#3388)
* Gradually bump mocktime in wait_for_quorum_connections

* Lower nLLMQConnectionRetryTimeout to 1 sec for regtest
2020-03-31 07:06:41 +02:00
PastaPastaPasta
975f73be7a
Merge #11648: [tests] Add messages.py (#3372)
* [tests] Tidy up mininode.py module

Mostly move only. Adds a few extra comments.

* #11648  [tests] Move test_framework Bitcoin primitives into separate module

I manually recreated this commit, since we have A LOT of conflicts in mininode. However since it is primarily just a move, it was pretty easy to recreate

Signed-off-by: Pasta <pasta@dashboost.org>

* add import to messages.py

Signed-off-by: Pasta <pasta@dashboost.org>

* move import from mininode.py to messages.py

Signed-off-by: Pasta <pasta@dashboost.org>

* fix test failure

Signed-off-by: Pasta <pasta@dashboost.org>

* remove empty line at top of messages.py

Signed-off-by: pasta <pasta@dashboost.org>

* alphabetize MESSAGEMAP seperated by if it is dash specific or not

Signed-off-by: pasta <pasta@dashboost.org>

* remove accidentally added feefilter message

Signed-off-by: pasta <pasta@dashboost.org>

* Add missing getmnlistd/mnlistdiff messages to MESSAGEMAP

Co-authored-by: John Newbery <john@johnnewbery.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-30 15:21:47 +03:00
Alexander Block
27dfb5a34d
Move wait_proc into wait_for_quorum_connections (#3384)
Adding this to wait_for_quorum_phase was useless as this was not the point
where failure was happening.
2020-03-28 00:59:02 +03:00
UdjinM6
3c90da86b3
Merge pull request #3371 from PastaPastaPasta/backports-0.16-pr12
Backports 0.16 pr12
2020-03-28 00:58:08 +03:00
Alexander Block
7df624d380 Implement tests for LLMQ connection handling 2020-03-27 15:12:20 +01:00
Alexander Block
8ab1a3734a
Bump mocktime each time waiting for phase1 fails (#3383)
This causes re-tries of LLMQ connections, which is required in cases
where 2 MNs tried to connect to each other and due to bad timing then
disconnected each other.
2020-03-27 15:12:03 +01:00
PastaPastaPasta
6e54327ef2 Apply suggestions from code review
Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-26 17:09:23 -05:00
Alexander Block
deba865b17
Also verify quorumHash when waiting for DKG phases (#3382)
This is especially important when waiting for phase 1 (initialization),
as we might have skipped a whole DKG session before while the async DKG
session handler is still in the init phase (but for the old/skipped LLMQ).
2020-03-26 13:25:24 +01:00
MarcoFalke
9b1ebdb161
Merge #12902: [qa] Handle potential cookie race when starting node
75d0e4c544 [qa] Delete cookie file before starting node (Suhas Daftuar)

Pull request description:

  When a node is restarted during a test after an unclean shutdown (such
  as with -dbcrashratio), it's possible an old cookie file was left
  behind. This can cause a race condition when restarting the node, where
  the test framework might try to connect using credentials from the
  old cookie file, just as the node will generate new credentials and
  overwrite the old file.

  Delete any such cookie file if present prior to startup.

Tree-SHA512: ae1e8bf8fd20e07c32b0715025693bb28b0e3dd34f328cae4346abf579b0c97b5db1c02782e1c46b7a3b6058d268b6d46b668e847658a6eed0be857ffb0d65dc
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
7afc672d80
Merge #12904: [qa] Ensure bitcoind processes are cleaned up when tests end
e36a0c0 [qa] Ensure bitcoind processes are cleaned up when tests end (Suhas Daftuar)

Pull request description:

  When tests fail (such as due to a bug in the test, race condition, etc), it's possible that we could follow code paths that bypass our normal node shutdown that occurs in `TestNode.stop_node`.  Add a destructor to `TestNode` that cleans this up.

Tree-SHA512: 72e04bc21462ebd0cb346fd1fe0540da454acfbad41923a0b06ea2317e9045b68e58f9adb02d8200891aca89a9d03a022eb72282aeb31a3b3afe7c6843a4b450
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
57d82cf722
Merge #12638: qa: Cache only chain and wallet for regtest datadir
fa23105 qa: Cache only chain and wallet for regtest datadir (MarcoFalke)

Pull request description:

  mempool.dat should be empty and I don't see a need to copy it around when restoring from the cache.

Tree-SHA512: f11ab69732db4dee0e9a0900570464e49085532b0cebc963877057112a7b985c477da3d32eb2093daabac9ada9e73b7c49881681ec5efa6101919b0af76001cf
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
8bf0a5c3c7
Merge #12553: Prefer wait_until over polling with time.sleep
9d7f839a2 test: Use os.path.join consistently in feature_pruning tests (Ben Woosley)
81b082277 test: Use wait_until in tests where time was used for polling (Ben Woosley)

Pull request description:

  This is prompted by and builds on #12545, a nice cleanup / consolidation of patterns.

  In cases where the exception message was meaningful, I tried to represent it as well in a comment.

  I expect #12545 will go in first, but I'm happy to squash them if that's preferred.

Tree-SHA512: 7a861244001c87fd6b59b6bc248ee741ac8178f7255d6f1fda39bc693c5ff3b7de5f53d13afe9829aef6ea69153481edb0a9d5bc07c36c4f66b4315edd180bb4
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
62bbabb6ac
Merge #12475: qa: Fix python TypeError in script.py
fa41d68a2 qa: Fix python TypeError in script.py (MarcoFalke)

Pull request description:

  `__repr__` returns string, so don't mix it with byte strings.

  This fixes

  ```
  TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'str'

Tree-SHA512: fac06e083f245209bc8a36102217580b0f6186842f4e52a686225111b0b96ff93c301640ff5e7ddef6a5b4f1689071b16a9a8dc80f28e2b060ddee29edd24ec7
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
fa59296e74
Merge #12486: [test] Round target fee to 8 decimals in assert_fee_amount
42e1b5d [test] Round target fee to 8 decimals in assert_fee_amount (Karl-Johan Alm)

Pull request description:

  The output would produce arbitrary number of decimal points, sometimes resulting in 9 decimals:
  ```
  AssertionError: Fee of 0.00000415 BTC too low! (Should be 0.000006175 BTC)
  ```
  The above looks like the expected fee is 6175 sats when in reality it's 618.

Tree-SHA512: ddbff2926a88890d6e34a58db36f0b15a917a80064be6e40e9bcbec3f05ae6202d02adcd7873733945b043fa121d4a56dd930446ec368078fe1935cbfff898ce
2020-03-24 11:43:10 -05:00
Alexander Block
cf1f8c3825
Support devnets in mininode (#3364) 2020-03-23 10:58:17 +01:00
UdjinM6
73258b363b
Merge pull request #3354 from PastaPastaPasta/backports-0.16-pr10
Backports 0.16 pr10
2020-03-10 14:57:35 +03:00
Wladimir J. van der Laan
64e33f715f
Merge #12220: Error if relative -walletdir is specified
ec527c6 Don't allow relative -walletdir paths (Russell Yanofsky)

Pull request description:

  This makes it an error to explicitly specify a non-absolute -walletdir path, and also adds a debug.log warning if a relative rather than absolute -datadir path is configured.

  Specifying paths relative to the current working directory in a daemon process can be dangerous, because files can fail to be located even if the configuration doesn't change, but the daemon is started up differently.

  Specifying a relative -datadir now adds a warning to the debug log. It would not be backwards-compatible to forbid relative -datadir paths entirely, and it could also be inconvenient for command line testing.

  Specifying a relative -walletdir now results in a startup error. But since the -walletdir option is new in 0.16.0, there should be no compatibility issues. Another reason not to use working directory paths for -walletdir specifically is that the default -walletdir is a "wallets" subdirectory inside the datadir, so it could be surprising that setting -walletdir manually would choose a directory rooted in a completely different location.

Tree-SHA512: 67cbdae677f82487a9031c5ec96b0205a488ab08718a0f4f49365e025119f3d7f6cfc88ba1eba04c1ecd8b9561a5b2c42e2e1a267af7c08c76b83e5e361f5a31
2020-03-04 10:13:31 -06:00
MarcoFalke
cc2cd7291b
Merge #11970: Add test coverage for bitcoin-cli multiwallet calls
a14dbff39e Allow multiwallet.py to be used with --usecli (Russell Yanofsky)
f6ade9ce1a [tests] allow tests to be run with --usecli (John Newbery)
ff9a363ff7 TestNodeCLI batch emulation (Russell Yanofsky)
ca9085afc5 Prevent TestNodeCLI.args mixups (Russell Yanofsky)
fcfb952bca Improve TestNodeCLI output parsing (Russell Yanofsky)

Pull request description:

  Lack of test coverage was pointed out by @jnewbery in https://github.com/bitcoin/bitcoin/pull/11687#discussion_r158133900

Tree-SHA512: 5f10e31abad11a5edab0da4e2515e39547adb6ab9e55e50427ab2eb7ec9a43d6b896b579b15863e5edc9beee7d8bf1c84d9dabd247be0760a1b9ae39e1e8ee02
2020-03-04 10:13:24 -06:00
MarcoFalke
4ec4e349d2
Merge #11677: qa: Remove unused NodeConn members
fafdad0d4 qa: Remove unused NodeConn members (MarcoFalke)

Pull request description:

  * `ver_send` and `ver_recv` were completely unused
  * `rpc` was only used once, in p2p-segwit. Imo better only pass it to the constructor in that single test

Tree-SHA512: 7f85554d6d0fd2096516ca3c608811d5370da66cde35d8031bdc921607a7a4efdb26355896012f75f713f8df09e28d46ba46be69fd96a5898fabb1a25cbcb8ad
2020-02-29 11:51:03 -06:00
MarcoFalke
a42d62d89e
Merge #11638: [tests] Dead mininode code
fb00c45c3 [tests] Explicitly disallow support for p2p versions below 60001 (John Newbery)
3858aabbd [tests] Remove support for p2p alert messages (John Newbery)
c0b127470 [tests] Remove support for bre-BIP31 ping messages (John Newbery)
2904e301c [tests] Remove dead code from mininode.py (John Newbery)

Pull request description:

  This is the first part of #11518. It removes a ~150 lines of unused code from the mininode module:

  - remove unused `deliver_sleep_time` and `EarlyDisconnectError` code
  - remove support for pre-BIP31 ping messages
  - remove support for alert message
  - explicitly don't support p2p versions lower than 60001

  Should be an easy ACK for reviewers. If all extended tests pass, then this code really was dead :)

Tree-SHA512: 508e612ceb0b094250d18e75522d51e6b14cd069443050ba4af34d6f890c58721cb5653e8bc000b60635b9474d035b0dcd9c509c0dcdb3a7501df17b787f83b0

readd is None check

Signed-off-by: Pasta <pasta@dashboost.org>
2020-02-29 11:51:03 -06:00
MarcoFalke
08171fb386
Merge #11513: [trivial] [tests] A few Python3 tidy ups
f89308532 [tests] Don't subclass from object for Python 3 (John Newbery)
8f9e3627e [tests] authproxy.py: tidy up __init__() (John Newbery)
323d8f61e [tests] fix flake8 warnings in authproxy.py (John Newbery)
fc0176d01 [tests] use python3 for authproxy.py (John Newbery)

Pull request description:

  A few trivial tidyups in the test_framework:

  - the test_framework can only be run in Python3, so remove the py2/3 compatibility workarounds in authproxy.py
  - while there, do some general tidying up of the module - fix flake8 warnings, make initialization code more compact
  - All classes in Python3 are new-style. No need to explicitly inherit from `object`.

Tree-SHA512: d15c93aa4b47c1ad7d05baa7a564053cf0294932e178c95ef335380113f42e1af314978d07d3b107292a8e3496fd840535b5571a9164182feaa062a1e9ff8b73

fix up mininode.py slightly

Signed-off-by: Pasta <pasta@dashboost.org>
2020-02-29 11:51:03 -06:00
Wladimir J. van der Laan
2c7b29bac5
Merge #11466: Specify custom wallet directory with -walletdir param
c1e5d40 Make debugging test crash easier (MeshCollider)
8263f6a Create walletdir if datadir doesn't exist and fix tests (MeshCollider)
9587a9c Default walletdir is wallets/ if it exists (MeshCollider)
d987889 Add release notes for -walletdir and wallets/ dir (MeshCollider)
80c5cbc Add test for -walletdir (MeshCollider)
0530ba0 Add -walletdir parameter to specify custom wallet dir (MeshCollider)

Pull request description:

  Closes #11348

  Adds a `-walletdir` parameter which specifies a directory to use for wallets, allowing them to be stored separately from the 'main' data directory. Creates a new `wallets/` directory in datadir if this is the first time running, and defaults to using it if it exists.

  Includes tests and release notes. Things which might need to be considered more:
  - there is no 'lock' on the wallets directory, which might be needed?
  - because this uses a new wallets/ directory by default, downgrading to an earlier version won't see the wallets in that directory (not a big deal though, users can just copy them up to the main dir)
  - jnewbery suggested putting each wallet in its own directory, which is a good idea, but out of scope for this PR IMO. EDIT: this is being done in https://github.com/bitcoin/bitcoin/pull/11687
  - doc/files.md needs updating (will do soon)

  I also considered including  a cleanup by removing caching of data directory paths and instead just initialise them once on startup (c.f. #3073), but decided it wasn't super relevant here will just complicate review.

Tree-SHA512: c8ac04bfe9a810c32055f2c8b8fa0d535e56125ceb8d96f12447dd3538bf3e5ee992b60b1cd2173bf5f3fa023a9feab12c9963593bf27ed419df929bb413398d
2020-02-08 23:33:25 -06:00
UdjinM6
7d39637b02 Bump copyright year to 2020 (#3290)
* Bump _COPYRIGHT_YEAR

* Run copyright update script

./contrib/devtools/copyright_header.py update .

* Update COPYING

* Bump copyright year in dash-cli/qt/tx and dashd map pages
2020-01-17 15:42:55 +01:00
UdjinM6
7e85f09007
Merge pull request #3284 from PastaPastaPasta/backports-0.16-pr3
Backports 0.16 pr3
2020-01-16 14:24:11 +03:00
Alexander Block
c88a7b9e76
Merge pull request #3279 from codablock/pr_backport_11818
Backport bitcoin#11818: I accidentally [deliberately] killed it [the ComparisonTestFramework]
2020-01-14 10:17:00 +01:00
Wladimir J. van der Laan
4b6d804d64 Merge #11370: [test] Add getblockchaininfo functional test
f6ffb14 [test] Add getblockchaininfo functional test (João Barbosa)
fd8f45f [test] Add restart_node to BitcoinTestFramework (João Barbosa)

Pull request description:

  Adds functional test for `getblockchaininfo`. Also deals with the fact that `pruneheight` is only in the response when pruning is enabled (related to #11366).

Tree-SHA512: 56cdec0921f572874f2fdded0990d1722d1435c3ff9979e6bff1afdccdca6f8b214dbe8d7490cdac07b5758909db085132d14340de2cce943241f7ebde7e5b6c
2020-01-11 18:22:26 -06:00
MarcoFalke
dac2112dd0 Merge #11323: mininode: add an optimistic write and disable nagle
1817398b3 mininode: add an optimistic write and disable nagle (Cory Fields)

Pull request description:

  Disclaimer: I'm not familiar with asyncore, so I'm unclear how safe this is. It works for me (tm).

  Because the poll/select loop may pause for 100msec before actually doing a send, and we have no way to force the loop awake, try sending from the calling thread if the queue is empty.

  Also, disable nagle as all sends should be either full messages or unfinished sends.

  This shaves an average of ~1 minute or so off of my accumulated runtime, and 10-15 seconds off of actual runtime.

Tree-SHA512: 6b61b8058e621dacf0b4dd353c10e3666fbda0691440eb6ebc432491ebada80a781dcd09291bf03e70112a41d3c2a0c91775ed08824b79bf8d0ebed11595c28b
2020-01-11 18:22:25 -06:00
Alexander Block
cb33702b74
Merge pull request #3276 from PastaPastaPasta/backports-0.16-pr2
Backports 0.16 pr2
2020-01-12 01:17:33 +01:00
John Newbery
9aac579ade [tests] Remove Comparison Test Framework 2020-01-11 13:56:03 +01:00
Alexander Block
91b4a38398 Backport bitcoin#11773: [tests] Change feature_block.py to use BitcoinTestFramework (#3277)
* [tests] Change feature_block.py to use BitcoinTestFramework

* [tests] Fix flake8 warnings in feature_block.py

* [tests] Tidy up feature_block.py

- move all helper methods to the end
- remove block, create_tx and create_and_sign_tx shortcuts
- remove --runbarelyexpensive option, since it defaults to True and it's
unlikely that anyone ever runs the test with this option set to false.

* [tests] Add logging to feature_block.py

* [tests] Improve assert message when wait_until() fails

* Merge #13048: [tests] Fix feature_block flakiness

c1d742025c [tests] Fix feature_block flakiness (John Newbery)

Pull request description:

  feature_block.py occasionally fails on Travis. I believe this is due to
  a a race condition when reconnecting to bitcoind after a subtest that
  expects disconnection. If the test runs ahead and sends the INV for the
  subsequent test before we've received the initial sync getheaders, then
  we may end up sending two headers messages - one as a response to the
  initial sync getheaders and one in response to the INV getheaders. If
  both of those headers fail validation with a DoS score of 50 or higher,
  then we'll unexpectedly be disconnected.

  There is only one validation failure that has a DoS score bewteen 50 and
  100, which is high-hash. That's why the test is failing immediately
  after the "Reject a block with invalid work" subtest.

  Fix is to wait for the initial getheaders from the peer before we
  start populating our blockstore. That way we won't have any invalid
  headers to respond to it with.

Tree-SHA512: dc17d795fcfaf0f8c0bf1e9732b5e11fbc8febbfafba4c231b7c13a5404a2c297dcd703a7a75bc7f353c893e12efc87f424f2201abd47ba5268af32d4d2e841f

* Temporarely rename MAX_BLOCK_SIZE -> MAX_BLOCK_BASE_SIZE

We'll undo this after the next commit. This avoids merge many conflicts and
makes reviewing easier.

* Rename MAX_BLOCK_BASE_SIZE back to MAX_BLOCK_SIZE

* Use DoS score of 100 for bad-blk-sigops

This was accidently changed to 10 while backporting bitcoin#7287 and causes
test failures in p2p-fullblocktest.py

* Use allowOptimisticSend=true when sending reject messages

This fixes test failures in p2p-fullblocktest.py which expects reject
messages to be sent/received before connections get closed.

* Fix p2p-fullblocktest.py

- CBlock and friends are still in test_framework.mininode
- "-whitelist" causes connections to not be dropped, which in turn causes
  sync_blocks with reconnect=True to fail
- "bad-cb-amount" does not cause a ban in Dash, so reconnect must be False
- Dash already bans when a header is received which is a child of an invalid
  header, causing block requests to never happen

* Backport missing changes from bitcoin#13003

bitcoin#13003 was backported out of order which causes missed changes.

* Bump p2p-fullblocktest timeouts

* Increase RPC timeout in p2p-fullblocktest.py

Co-authored-by: John Newbery <jonnynewbs@gmail.com>
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-01-11 04:31:25 +03:00
MarcoFalke
00167dea54
Merge #11078: [tests] Make p2p-leaktests.py more robust
0063d2c3d [tests] Make p2p-leaktests.py more robust (John Newbery)

Pull request description:

  There has been an example of p2p-leaktests.py failing on travis in the new service bits test (introduced in #11001 . It appeared to me that the previous p2p connections had not been fully disconnected before attempting to add new p2p connections.

  I've added a sleep and restarted the NetworkThread, but I don't know whether this will fix the problem, since I'm unable to reproduce the failure locally.

  @MarcoFalke - not sure what you want to do here? I don't think this change could make things any worse.

Tree-SHA512: f5427c26267185a903c9b75bb3925bf153b8afce70c8e493bf8f585f57d809d20643b4ee69081300b211d22e960242aecc3d719f4ddd230aa08fdc5484b55055
2020-01-10 10:33:55 -06:00
Alexander Block
9c9cac6d67
Show quorum connections in "quorum dkgstatus" and use it in mine_quorum (#3271)
Instead of statically sleeping 2 seconds
2020-01-08 08:01:26 +01:00
Alexander Block
aca6af0a0e
Use smaller LLMQs in regtest (#3269)
* Rename LLMQ_5_60 to LLMQ_TEST

* Introduce -llmqtestparams which allows to modify LLMQ_TEST on regtest

Also add support in DashTestFramework

* Use parameters size=3, threshold=2 as default for LLMQ_TEST

And fall back to the old parameters where necessary

* Wait for all contributions, even when one member is lying

Otherwise we might end up continuing too fast, which would fail the DKG.

* Update src/chainparams.cpp

Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-01-07 13:49:51 +01:00
Alexander Block
7e3ed76e54 Make a deep copy of extra_args before modifying it in set_dash_test_params (#3270)
Otherwise we end up having the same list instance for every entry in
extra_args, which means modifying index 0 also affects all other indexes.

Noticed this while looking at "ps aux | grep dashd" output. It turned out
that parameters like "-dip3params" were passed multiple times to each node.
2020-01-04 14:22:41 +03:00
Alexander Block
75bb7ec022 A few optimizations/speedups for Dash related tests (#3268)
* Avoid unnecessary connections in dip3-deterministicmns.py

This saves ~15 seconds

* Generate more blocks per generate() call

* Use -vbparams to activate dip8 faster in tests

Avoids generating/syncing many unnecessary blocks
2020-01-04 14:21:16 +03:00
MarcoFalke
561ec27683 Merge #10781: Python cleanups
78214588d Use for-loop instead of list comprehension (practicalswift)
823979436 Use the variable name _ for unused return values (practicalswift)
2e6080bbf Remove unused variables and/or function calls (practicalswift)
9b94054b7 Avoid reference to undefined name: stderr does not exist, sys.stderr does (practicalswift)
51cb6b822 Use print(...) instead of undefined printf(...) (practicalswift)
25cd520fc Use sys.exit(...) instead of exit(...): exit(...) should not be used in programs (practicalswift)

Pull request description:

  Python cleanups:
  * Avoid reference to undefined name: `stderr` does not exist, `sys.stderr` does
  * Use `print(...)` instead of undefined `printf(...)`
  * Avoid redefinition of variable (`tx`) in list comprehension
  * Remove unused variables and/or function calls
  * Use `sys.exit(...)` instead of `exit(...)`: [`exit(...)` should not be used in programs](https://github.com/bitcoin/bitcoin/pull/10753#discussion_r125935027)

Tree-SHA512: 1238dfbc1d20f7edadea5e5406a589f293065638f6234809f0d5b6ba746dffe3d276bc5884c7af388a6c798c61a8759faaccf57f381225644754c0f61914eb4b
2020-01-02 22:01:29 -06:00
Alexander Block
4c00d98ea6 Allow re-signing of IS locks when performing retroactive signing (#3219)
* Implement re-signing of InstantSend inputs when TXs come in via blocks

* Use GetAdjustedTime instead of GetTimeMillis in CSigSharesManager

This allows use of mocktime in tests.

* Expose verifiedProRegTxHash in getpeerinfo and implement wait_for_mnauth

* Allow to wait for IS and CL to NOT happen

* Bump timeout for wait_for_instantlock

* Implement tests for retroactive signing of IS and CLs

* Add wait_for_tx function to DashTestFramework

* Add -whitelist=127.0.0.1 to node0

* Use node3 for isolated block generation

* Don't test for non-receival of TXs on node4/node5
2019-12-06 12:05:58 +03:00
UdjinM6
b4b9d34675 Tests: Fix the way nodes are connected to each other in setup_network/start_masternodes (#3221)
* Tests: Connect to the control node only in start_masternodes

Masternodes should take care of intra-quorum connections themselves

* Reconnect non-masternodes back to the control node
2019-12-05 19:31:55 +01:00
UdjinM6
08f447af92 Tests: Allow specifying different cmd-line params for each masternode (#3222) 2019-12-05 19:28:33 +01:00
Nathan Marley
70b320bab0 Detect masternode mode from masternodeblsprivkey arg (#3188)
* Detect masternode mode from privkey arg

The `masternode` argument seems redundant. This change enables masternode mode
based on the presence (and validity) of the `masternodeblsprivkey` argument.

* Deprecate -masternode option

* Remove -masternode switch from functional tests

* Move -masternode deprecate warning to better place
2019-11-11 12:21:45 +03:00
Alexander Block
21d33dd56d
Merge #14413: tests: Allow closed rpc handler in assert_start_raises_init_error (#3157)
62c304ea48 tests: Allow closed http server in assert_start_raises_init_error (Chun Kuan Lee)

Pull request description:

  The rpc handler may be unregistered when http server haven't been closed yet. So it may be allowable to get -342 `non-JSON HTTP response with \'%i %s\' from server` (503 Service Unavailable)

  See https://ci.appveyor.com/project/DrahtBot/bitcoin/build/master.2001. It shows "Rejecting request while shutting down" between "RPC stopped" and "Stopped HTTP server"

Tree-SHA512: e1f50ab9096cf23494ccc2850c01516c4a75f112c99108759157b22fce2011682a4b88216f206702f6a56e960182c00098053ad75f13aa0eafe27046adae63da
2019-10-17 11:59:21 +02:00
Alexander Block
a13a9182d9 Add missing "notfound" and "getsporks" to messagemap (#3146) 2019-10-16 17:10:18 +03:00
Wladimir J. van der Laan
fefe07003b Merge #14670: http: Fix HTTP server shutdown
28479f926f21f2a91bec5a06671c60e5b0c55532 qa: Test bitcond shutdown (João Barbosa)
8d3f46ec3938e2ba17654fecacd1d2629f9915fd http: Remove timeout to exit event loop (João Barbosa)
e98a9eede2fb48ff33a020acc888cbcd83e24bbf http: Remove unnecessary event_base_loopexit call (João Barbosa)
6b13580f4e3842c11abd9b8bee7255fb2472b6fe http: Unlisten sockets after all workers quit (João Barbosa)
18e968581697078c36a3c3818f8906cf134ccadd http: Send "Connection: close" header if shutdown is requested (João Barbosa)
02e1e4eff6cda0bfc24b455a7c1583394cbff6eb rpc: Add wait argument to stop (João Barbosa)

Pull request description:

  Fixes #11777. Reverts #11006. Replaces #13501.

  With this change the HTTP server will exit gracefully, meaning that all requests will finish processing and sending the response, even if this means to wait more than 2 seconds (current time allowed to exit the event loop).

  Another small change is that connections are accepted even when the server is stopping, but HTTP requests are rejected. This can be improved later, especially if chunked replies are implemented.

  Briefly, before this PR, this is the order or events when a request arrives (RPC `stop`):
   1. `bufferevent_disable(..., EV_READ)`
   2. `StartShutdown()`
   3. `evhttp_del_accept_socket(...)`
   4. `ThreadHTTP` terminates (event loop exits) because there are no active or pending events thanks to 1. and 3.
   5. client doesn't get the response thanks to 4.

  This can be verified by applying
  ```diff
       // Event loop will exit after current HTTP requests have been handled, so
       // this reply will get back to the client.
       StartShutdown();
  +    MilliSleep(2000);
       return "Bitcoin server stopping";
   }
  ```
  and checking the log output:
  ```
      Received a POST request for / from 127.0.0.1:62443
      ThreadRPCServer method=stop user=__cookie__
      Interrupting HTTP server
  **  Exited http event loop
      Interrupting HTTP RPC server
      Interrupting RPC
      tor: Thread interrupt
      Shutdown: In progress...
      torcontrol thread exit
      Stopping HTTP RPC server
      addcon thread exit
      opencon thread exit
      Unregistering HTTP handler for / (exactmatch 1)
      Unregistering HTTP handler for /wallet/ (exactmatch 0)
      Stopping RPC
      RPC stopped.
      Stopping HTTP server
      Waiting for HTTP worker threads to exit
      msghand thread exit
      net thread exit

      ... sleep 2 seconds ...

      Waiting for HTTP event thread to exit
      Stopped HTTP server
  ```

  For this reason point 3. is moved right after all HTTP workers quit. In that moment HTTP replies are queued in the event loop which keeps spinning util all connections are closed. In order to trigger the server side close with keep alive connections (implicit in HTTP/1.1) the header `Connection: close` is sent if shutdown was requested. This can be tested by
  ```
  bitcoind -regtest
  nc localhost 18443
  POST / HTTP/1.1
  Authorization: Basic ...
  Content-Type: application/json
  Content-Length: 44

  {"jsonrpc": "2.0","method":"stop","id":123}
  ```

  Summing up, this PR:
   - removes explicit event loop exit — event loop exits once there are no active or pending events
   - changes the moment the listening sockets are removed — explained above
   - sends header `Connection: close` on active requests when shutdown was requested which is relevant when it's a persistent connection (default in HTTP 1.1) — libevent is aware of this header and closes the connection gracefully
   - removes event loop explicit break after 2 seconds timeout

Tree-SHA512: 4dac1e86abe388697c1e2dedbf31fb36a394cfafe5e64eadbf6ed01d829542785a8c3b91d1ab680d3f03f912d14fc87176428041141441d25dcb6c98a1e069d8
2019-10-14 13:45:09 +02:00
UdjinM6
e0c56246f5
Fixes and refactorings related to using mnsync in tests (#3136)
* Drop `get_mnsync_status`, `wait_to_sync` and `sync_masternodes` and introduce `force_finish_mnsync` for MNs only

* Use `force_finish_mnsync` from util.py in dip3-deterministicmns.py and drop local unused functions

Also move the call, `force_finish_mnsync` should be called before `connect_nodes_bi`
2019-10-09 19:48:12 +03:00
Wladimir J. van der Laan
ed65d51262 Merge #13003: qa: Add test for orphan handling
fa02c5b qa: Clarify documentation for send_txs_and_test (MarcoFalke)
fadfbd3 qa: Add test for orphan handling (MarcoFalke)

Pull request description:

Tree-SHA512: e0932d6bd03c73e3113f5457a3ffa3bbfc7b6075dfca8de95224d9df875e60ca6eb15cd8baa226f13de965483006559556191630a83c3bb431e79c53a85ef73f
2019-10-03 17:04:59 +02:00
Alexander Block
573d1da16f Use NodeConnCB as base for P2PDataStore
Until the renaming to P2PInterface is backported.
2019-10-03 14:04:54 +02:00
Wladimir J. van der Laan
01c138e86b Merge #11771: [tests] Change invalidtxrequest to use BitcoinTestFramework
95e2e9a [tests] Change invalidtxrequest to use BitcoinTestFramework (John Newbery)
359d067 [tests] Fix flake8 warnings in invalidtxrequest (John Newbery)
c32cf9f [tests] Add P2PDataStore class (John Newbery)
cc046f6 [tests] Reduce NodeConn connection logging from info to debug (John Newbery)

Pull request description:

  Next step in #10603

  - first commit changes log level for an internal log from INFO to DEBUG. (Not really related, but I started finding the INFO level logging annoying when debuging test failures)
  - second commit introduces a `P2PStub` class - a subclass of `NodeConnCB` which has its own block and tx store and responds appropriately to getdata requests. Not all the functionality is used in `invalidtxrequest.py`, but will be used in `invalidblockrequest.py` and `p2p-fullblocktest` when those are changed to use `BitcoinTestFramework`
  - third commit tidies up `invalidtxrequest.py`
  - fourth commit removes usage of `ComparisonTestFramework`

Tree-SHA512: f3085c73c15d6ce894e401490bce8a7fa7cf52b0c9d135ff7e351f1f6f517c99accab8588fcdc443f39ea8315329aaabd66b2baa32499df5a774737882030373
2019-10-03 14:04:54 +02:00
Wladimir J. van der Laan
f4ea836744 Merge #11849: [tests] Assert that only one NetworkThread exists
5c8ff26 [tests] Add NetworkThread assertions (John Newbery)
34e08b3 [tests] Fix network threading in functional tests (John Newbery)
74e64f2 [tests] Use network_thread_start() in tests. (John Newbery)
5fc6e71 [tests] Add network_thread_ utility functions. (John Newbery)

Pull request description:

  Add assert that only one NetworkThread exists at any time in functional tests, and fix cases where that wasn't true.

  fixes #11776

Tree-SHA512: fe5d1c59005f94bf66e11bb23ccf274b1cd9913741b56ea11dbcd21db4cc0b53b4413c0c4c16dbcd6ac611adad5e5cc2baaa39720598ce7b6393889945d06298
2019-10-03 14:04:54 +02:00
Wladimir J. van der Laan
6e6e950fca Merge #11641: qa: Only allow disconnecting all NodeConns
faaa7db qa: Only allow disconnecting all NodeConns (MarcoFalke)

Pull request description:

  Disconnecting the connection with `index=0` makes no sense when there are more than one connections, as the list "rotates around" and populates index 0 after `del`.

  Just disconnect all NodeConns in any case.

Tree-SHA512: e5cf540823fccb31634b5a11501f54222be89862e80ccafc28bc06726480f8d2153b8c1b6f859fa6a6d087876251d48a6c6035bccdaaf16831e300bc17ff613d
2019-10-03 14:04:54 +02:00