Commit Graph

17528 Commits

Author SHA1 Message Date
Alexander Block
beaf3f40b2 Implement DashTestFramework.get_mninfo 2020-03-31 19:58:07 +02:00
Alexander Block
80533f6c0d Implement "quorum selectquorum" RPC 2020-03-31 19:58:07 +02:00
Alexander Block
5edbeafcea Also run llmq-signing.py with spork21 enabled 2020-03-31 19:58:07 +02:00
Alexander Block
b212f21c15 Implement new way of concentrated signature recovery
Instead of propagating all sig shares to all LLMQ members, this will now
make all members send their individual sig share to a single member, which
is then responsible for the recovery and propagation of the recovered
signature. This process is repeated by all members every second for another
target/recovering member, until a recovered signature appears.
2020-03-31 19:58:07 +02:00
Alexander Block
45064d8dc9 Rename sigSharesToSend to sigShareBatchesToSend 2020-03-31 19:58:07 +02:00
UdjinM6
f43cdbc586
Gradually bump mocktime in wait_for_quorum_connections (#3388)
* Gradually bump mocktime in wait_for_quorum_connections

* Lower nLLMQConnectionRetryTimeout to 1 sec for regtest
2020-03-31 07:06:41 +02:00
PastaPastaPasta
3b904a0fa1
Add a note about dash_hash under dependencies in test/README.md (#3386) 2020-03-30 15:22:06 +03:00
PastaPastaPasta
975f73be7a
Merge #11648: [tests] Add messages.py (#3372)
* [tests] Tidy up mininode.py module

Mostly move only. Adds a few extra comments.

* #11648  [tests] Move test_framework Bitcoin primitives into separate module

I manually recreated this commit, since we have A LOT of conflicts in mininode. However since it is primarily just a move, it was pretty easy to recreate

Signed-off-by: Pasta <pasta@dashboost.org>

* add import to messages.py

Signed-off-by: Pasta <pasta@dashboost.org>

* move import from mininode.py to messages.py

Signed-off-by: Pasta <pasta@dashboost.org>

* fix test failure

Signed-off-by: Pasta <pasta@dashboost.org>

* remove empty line at top of messages.py

Signed-off-by: pasta <pasta@dashboost.org>

* alphabetize MESSAGEMAP seperated by if it is dash specific or not

Signed-off-by: pasta <pasta@dashboost.org>

* remove accidentally added feefilter message

Signed-off-by: pasta <pasta@dashboost.org>

* Add missing getmnlistd/mnlistdiff messages to MESSAGEMAP

Co-authored-by: John Newbery <john@johnnewbery.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-30 15:21:47 +03:00
Alexander Block
b0668028b6
Implement more randomized behavior in GetQuorumConnections (#3385)
When taking the proTxHash naively, we might end up with a few unlucky MNs
which always have to perform most of the outbound connections while other
unlucky MNs would always have to wait for inbound connections. Hashing
the proTxHash with the quorum hash makes this more random.
2020-03-28 00:59:32 +03:00
Alexander Block
27dfb5a34d
Move wait_proc into wait_for_quorum_connections (#3384)
Adding this to wait_for_quorum_phase was useless as this was not the point
where failure was happening.
2020-03-28 00:59:02 +03:00
Alexander Block
ff6f391aea
Refactor Gitlab builds to use multiple stages (#3377)
* Remove unused jenkins stuff

* Install all dependencies in builder image

Instead of only target specific dependencies.

* Use docker builder image for builds

* Optimize apt installations

* Move building of dependencies into separate stage

The build-depends-xxx jobs will create artifacts (depends/$HOST) which are
then pulled in by the build jobs with the help of "needs"

* Remove use of caches from develop branch

* Use gitlab specific extends instead of YAML anchors

* Move before_script of build_template into base_template

* Add hack for parallel installation of i686 and arm cross compilation

* Install python3-setuptools in builder image

* Remove unnecessary change-dir

* Use variables to pass BUILD_TARGET instead of relying on the job name

* Move integration tests into separate stage

* Don't use --quiet for integration tests on Gitlab
2020-03-28 00:58:51 +03:00
UdjinM6
3c90da86b3
Merge pull request #3371 from PastaPastaPasta/backports-0.16-pr12
Backports 0.16 pr12
2020-03-28 00:58:08 +03:00
Alexander Block
a5a3e51554
Merge pull request #3380 from codablock/pr_all_mns_connected
Let all LLMQ members connect to each other instead of only a few ones
2020-03-27 17:07:44 +01:00
Alexander Block
a09e36106e Fix onlyOutbound handling 2020-03-27 15:12:20 +01:00
Alexander Block
f82204db95 Move intra-quorum connection calculation into local func 2020-03-27 15:12:20 +01:00
Alexander Block
fdec67a55b Wait for ping/pong after re-connecting all nodes 2020-03-27 15:12:20 +01:00
Alexander Block
882b58c990 Use <> instead of "" for #include 2020-03-27 15:12:20 +01:00
Alexander Block
dbaf13848d Include inbound connections in output of "quorum dkgstatus" 2020-03-27 15:12:20 +01:00
Alexander Block
7df624d380 Implement tests for LLMQ connection handling 2020-03-27 15:12:20 +01:00
Alexander Block
e8bbbec259 Don't try to open masternode connections when network is disabled 2020-03-27 15:12:20 +01:00
Alexander Block
c9608bf930 Only add wallet info to protx list/info when wallet is enabled 2020-03-27 15:12:20 +01:00
Alexander Block
9ef2b05884 Add masternode meta info to protx list/info 2020-03-27 15:12:20 +01:00
Alexander Block
8f644d18d7 Implement probing of public ip/port of LLMQ members 2020-03-27 15:12:20 +01:00
Alexander Block
14bb62ac8e Connect all LLMQ members to all other members 2020-03-27 15:12:20 +01:00
Alexander Block
486463d622 Add SPORK_21_QUORUM_ALL_CONNECTED 2020-03-27 15:12:20 +01:00
Alexander Block
6c95518807 Bump proto version 2020-03-27 15:12:20 +01:00
Alexander Block
8ab1a3734a
Bump mocktime each time waiting for phase1 fails (#3383)
This causes re-tries of LLMQ connections, which is required in cases
where 2 MNs tried to connect to each other and due to bad timing then
disconnected each other.
2020-03-27 15:12:03 +01:00
Alexander Block
c68b5f68aa
Hold CEvoDB lock while iterating mined commitments (#3379) 2020-03-27 15:11:42 +01:00
PastaPastaPasta
6e54327ef2 Apply suggestions from code review
Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-26 17:09:23 -05:00
Alexander Block
deba865b17
Also verify quorumHash when waiting for DKG phases (#3382)
This is especially important when waiting for phase 1 (initialization),
as we might have skipped a whole DKG session before while the async DKG
session handler is still in the init phase (but for the old/skipped LLMQ).
2020-03-26 13:25:24 +01:00
Alexander Block
17ece14f40
Better/more logging for DKGs (#3381)
* Better/more logging for DKGs

* Use type name instead of type number in CDKGLogger
2020-03-26 13:25:01 +01:00
Alexander Block
80be2520a2
Call FlushBackgroundCallbacks before resetting CConnman (#3378) 2020-03-26 13:24:31 +01:00
Alexander Block
b6bdb8be9e
Faster opening of masternode connections (#3375)
Only sleep 100ms when we previously tried to connect a MN. The back-off
logic in ThreadOpenMasternodeConnections will prevent too many unsuccessful
connects to offline/bad nodes.
2020-03-26 13:24:06 +01:00
Alexander Block
fa9b91b50f
Merge pull request #3376 from UdjinM6/merge_11824
Merge remaining bits of #11824: Block ActivateBestChain to empty validationinterface queue
2020-03-26 13:23:44 +01:00
UdjinM6
b1dff227cb Apply some review suggestions 2020-03-25 17:51:42 +03:00
Wladimir J. van der Laan
d94dd15de7
Merge #11718: tests: move pwalletMain to wallet test fixture
49bd659 tests: move pwalletMain to wallet test fixture (Wladimir J. van der Laan)

Pull request description:

  Scope the variable instead of using an external global; this is how test fixtures are intended to be used.

  Followup to #11713.

Tree-SHA512: 7d5bda93cdfe1329c8fe39bd72965906e36dad72fbb5d344ebedf26e66b1857510d01a3c2872d7f718fdeb23365e6ba71991aafe68e82781c6767a086b6d1590
2020-03-24 11:43:10 -05:00
MarcoFalke
9b1ebdb161
Merge #12902: [qa] Handle potential cookie race when starting node
75d0e4c544 [qa] Delete cookie file before starting node (Suhas Daftuar)

Pull request description:

  When a node is restarted during a test after an unclean shutdown (such
  as with -dbcrashratio), it's possible an old cookie file was left
  behind. This can cause a race condition when restarting the node, where
  the test framework might try to connect using credentials from the
  old cookie file, just as the node will generate new credentials and
  overwrite the old file.

  Delete any such cookie file if present prior to startup.

Tree-SHA512: ae1e8bf8fd20e07c32b0715025693bb28b0e3dd34f328cae4346abf579b0c97b5db1c02782e1c46b7a3b6058d268b6d46b668e847658a6eed0be857ffb0d65dc
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
7afc672d80
Merge #12904: [qa] Ensure bitcoind processes are cleaned up when tests end
e36a0c0 [qa] Ensure bitcoind processes are cleaned up when tests end (Suhas Daftuar)

Pull request description:

  When tests fail (such as due to a bug in the test, race condition, etc), it's possible that we could follow code paths that bypass our normal node shutdown that occurs in `TestNode.stop_node`.  Add a destructor to `TestNode` that cleans this up.

Tree-SHA512: 72e04bc21462ebd0cb346fd1fe0540da454acfbad41923a0b06ea2317e9045b68e58f9adb02d8200891aca89a9d03a022eb72282aeb31a3b3afe7c6843a4b450
2020-03-24 11:43:10 -05:00
MarcoFalke
5fc1686919
Merge #12843: [tests] Test starting bitcoind with -h and -version
63048ec73d [tests] Test starting bitcoind with -h and -version (John Newbery)

Pull request description:

  Test that starting bitcoind/bitcoin-qt with `-h` and `-version` works as expected.

  Prompted by https://github.com/bitcoin/bitcoin/pull/10762#commitcomment-28345993, which is a nullpointer dereference triggered by starting bitcoin-qt with `-h`.

  On master, this test passes when run over bitcoind, but fails when running over bitcoin-qt. I used xvfb as a virtual frame buffer to test:

  ```
  BITCOIND=/home/ubuntu/bitcoin/src/qt/bitcoin-qt xvfb-run ./feature_help.py --nocleanup
  2018-03-30T17:09:37.767000Z TestFramework (INFO): Initializing test directory /tmp/user/1000/testdi4dre13
  2018-03-30T17:09:37.767000Z TestFramework (INFO): Start bitcoin with -h for help text
  2018-03-30T17:09:37.841000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main
      self.run_test()
    File "./feature_help.py", line 25, in run_test
      assert_equal(ret_code, 0)
    File "/home/ubuntu/bitcoin/test/functional/test_framework/util.py", line 39, in assert_equal
      raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
  AssertionError: not(-11 == 0)
  2018-03-30T17:09:37.842000Z TestFramework (INFO): Stopping nodes
  Traceback (most recent call last):
    File "./feature_help.py", line 42, in <module>
      HelpTest().main()
    File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 149, in main
      self.stop_nodes()
    File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 273, in stop_nodes
      node.stop_node()
    File "/home/ubuntu/bitcoin/test/functional/test_framework/test_node.py", line 141, in stop_node
      self.stop()
    File "/home/ubuntu/bitcoin/test/functional/test_framework/test_node.py", line 87, in __getattr__
      assert self.rpc_connected and self.rpc is not None, "Error: no RPC connection"
  AssertionError: Error: no RPC connection
  ```

  Passes for bitcoind and bitcoin-qt when run on #12836.

  Longer term, we should consider running functional tests over bitcoin-qt in one of the Travis jobs.

Tree-SHA512: 0c2f40f3d5f0e78c3a1b07dbee8fd383eebab27ed0bf2a98a5b9cc66613dbd7b70e363c56163a37e02f68ae7ff7b3ae1769705d0e110ca68a00f8693315730a4
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
57d82cf722
Merge #12638: qa: Cache only chain and wallet for regtest datadir
fa23105 qa: Cache only chain and wallet for regtest datadir (MarcoFalke)

Pull request description:

  mempool.dat should be empty and I don't see a need to copy it around when restoring from the cache.

Tree-SHA512: f11ab69732db4dee0e9a0900570464e49085532b0cebc963877057112a7b985c477da3d32eb2093daabac9ada9e73b7c49881681ec5efa6101919b0af76001cf
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
8bf0a5c3c7
Merge #12553: Prefer wait_until over polling with time.sleep
9d7f839a2 test: Use os.path.join consistently in feature_pruning tests (Ben Woosley)
81b082277 test: Use wait_until in tests where time was used for polling (Ben Woosley)

Pull request description:

  This is prompted by and builds on #12545, a nice cleanup / consolidation of patterns.

  In cases where the exception message was meaningful, I tried to represent it as well in a comment.

  I expect #12545 will go in first, but I'm happy to squash them if that's preferred.

Tree-SHA512: 7a861244001c87fd6b59b6bc248ee741ac8178f7255d6f1fda39bc693c5ff3b7de5f53d13afe9829aef6ea69153481edb0a9d5bc07c36c4f66b4315edd180bb4
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
62bbabb6ac
Merge #12475: qa: Fix python TypeError in script.py
fa41d68a2 qa: Fix python TypeError in script.py (MarcoFalke)

Pull request description:

  `__repr__` returns string, so don't mix it with byte strings.

  This fixes

  ```
  TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'str'

Tree-SHA512: fac06e083f245209bc8a36102217580b0f6186842f4e52a686225111b0b96ff93c301640ff5e7ddef6a5b4f1689071b16a9a8dc80f28e2b060ddee29edd24ec7
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
fa59296e74
Merge #12486: [test] Round target fee to 8 decimals in assert_fee_amount
42e1b5d [test] Round target fee to 8 decimals in assert_fee_amount (Karl-Johan Alm)

Pull request description:

  The output would produce arbitrary number of decimal points, sometimes resulting in 9 decimals:
  ```
  AssertionError: Fee of 0.00000415 BTC too low! (Should be 0.000006175 BTC)
  ```
  The above looks like the expected fee is 6175 sats when in reality it's 618.

Tree-SHA512: ddbff2926a88890d6e34a58db36f0b15a917a80064be6e40e9bcbec3f05ae6202d02adcd7873733945b043fa121d4a56dd930446ec368078fe1935cbfff898ce
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
d3e5f8ead7
Merge #12518: [0.16] Bump leveldb subtree
835a21b Squashed 'src/leveldb/' changes from c521b3ac65..64052c76c5 (MarcoFalke)

Pull request description:

  The leveldb bump is the same branch/commit as in #12451

Tree-SHA512: 585a55747c75e990c9fd73399e98e548c01bef8b960c0a99844326de43663d004d2211b4689518f2d3e51cc48a732e9c92082939c356793143db411224fa75fa
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
1e41cf9f3d
Merge #11722: Switched sync.{cpp,h} to std threading primitives.
f7f7e2c threads: add a thread_local autoconf check (Cory Fields)
bba9bd0 Switched sync.{cpp,h} to std threading primitives. (Thomas Snider)

Pull request description:

  Replaced boost threading primitives with the std equivalents.

Tree-SHA512: 72d10f9e48bfcf1db87e4a88bc698ef98eba0b29fe904570391b34a6ea1ffad474b7f192e70e3588a30e448f70f244eb4ddc5f24412a0bde2b564e76274160a5
2020-03-24 11:43:10 -05:00
Wladimir J. van der Laan
411c57995b
Merge #11884: Remove unused include in hash.cpp
3f09e03 Remove unused include in hash.cpp (Karl-Johan Alm)

Pull request description:

Tree-SHA512: 543a72656460fba1c5498a0b85c49601d9b0399a4ecc49f4acf4715c258918da729df388e3be724c3161438e903ee16ad3c50626a71483aa6d85ffdbb827742d
2020-03-24 11:43:10 -05:00
Alexander Block
8211ff0a8a
Merge pull request #3368 from codablock/pr_fmasternode
Don't relay anything to inbound/outbound fMasternode connections
2020-03-24 17:40:51 +01:00
Pieter Wuille
871b8585ca Merge #11824: Block ActivateBestChain to empty validationinterface queue
97d2b09c12 Add helper to wait for validation interface queue to catch up (Matt Corallo)
36137497f1 Block ActivateBestChain to empty validationinterface queue (Matt Corallo)
5a933cefcc Add an interface to get the queue depth out of CValidationInterface (Matt Corallo)
a99b76f269 Require no cs_main lock for ProcessNewBlock/ActivateBestChain (Matt Corallo)
a734896038 Avoid cs_main in net_processing ActivateBestChain calls (Matt Corallo)
66aa1d58a1 Refactor ProcessGetData in anticipation of avoiding cs_main for ABC (Matt Corallo)
818075adac Create new mutex for orphans, no cs_main in PLV::BlockConnected (Matt Corallo)

Pull request description:

  This should fix #11822.

  It ended up bigger than I hoped for, but its not too gnarly. Note that "
  Require no cs_main lock for ProcessNewBlock/ActivateBestChain" is mostly pure code-movement.

Tree-SHA512: 1127688545926f6099449dca6a4e6609eefc3abbd72f1c66e03d32bd8c7b31e82097d8307822cfd1dec0321703579cfdd82069cab6e17b1024e75eac694122cb
2020-03-24 17:21:59 +03:00
Alexander Block
94bcf85347
Merge pull request #3367 from codablock/pr_refactor_llmq_conns
Refactor and unify quorum connection handling
2020-03-24 11:20:45 +01:00
Alexander Block
4e8f4ea202
Merge pull request #3374 from codablock/pr_fix_scheduler_shutdown
Backport bitcoin#12266 and bitcoin#13894 to fix shutdown issues
2020-03-24 11:20:12 +01:00