* llmq|init|test: Add "mode" to -llmq-qvvec-sync parameter
This changes the paramter from `-llmq-qvvec-sync=<quorum_name>` to `-llmq-qvvec-sync=<quorum_name:mode>`
With the following definitions:
- `quorum_name`: Internal name of the quorum type
- `mode=0` - Sync always from all quorums of the type defined by `quorum_name`
- `mode=1` - Sync only if member of any from all other quorum of the type defined by `quorum_name`
`-llmq-qvvec-sync=llmq_100_67:0` To always request qvvec's from all `LLMQ_100_67`.
`-llmq-qvvec-sync=llmq_100_67:1` Only request if type member.
This means, if platform enables this on all MNs with `mode=0` we will
have all nodes asking new quorum for their verification vector instead
of only `24*100` at max.
* llmq: Adjust GetQuorumRecoveryStartOffset to use all MNs
* Turn `QvvecSyncMode` into `enum class`
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* llmq: Avoid writing commitments to evodb and altering caches when all we want is to check block candidate validity
* tests: call `getblocktemplate` to trigger `CreateNewBlock` before quorum commitment is mined
* llmq: Implement automated DKG recovery threads
* llmq: Implement quorum verification vector sync
* init: Validiate quorum data recovery related command line parameter
* test: Add quorum_data_request_timeout_seconds in DashTestFramework
* test: Test quorum data recovery in feature_llmq_data_recovery.py
* test: Add feature_llmq_data_recovery.py to BASE_SCRIPTS
* test: Fix quorum_data_request_expiration_timeout in wait_for_quorum_data
* test: Always test the existence of secretKeyShare in test_mn_quorum_data
With this change it also validates that "secretKeyShare" is not in `quorum_info` if its not expected to be in there. Before this was basically just not tested.
* llmq|test: Use bool as argument type for -llmq-data-recovery
* llmq: Always set nTimeLastSuccess to 0
* test: Set -llmq-data-recovery=0 in p2p_quorum_data.py
* test: Simplify test_mns
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* refactor: pass CQuorumCPtr to StartQuorumDataRecoveryThread
* test: Fix thread name in comment
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* version: Bump PROTOCOL_VERSION and MIN_MASTERNODE_PROTO_VERSION
* version: Introduce LLMQ_DATA_MESSAGES_VERSION for QGETDATA/QDATA support
* test: Bump MY_VERSION to 70219 (LLMQ_DATA_MESSAGES_VERSION)
* llmq: Introduce CQuorumDataRequest as wrapper for QGETDATA requests
* llmq: Implement CQuorum::{SetVerificationVector, SetSecretKeyShare}
* llmq|net|protocol: Implement QGETDATA/QDATA P2P messages
* llmq: Restrict processing QGETDATA/QDATA to masternodes only
* llmq: Implement request limiting for QGETDATA/QDATA
* llmq: Implement CQuorumManger::RequestQuorumData
* rpc: Implement "quorum getdata" as wrapper around QGETDATA
Allows to trigger sending QGETDATA messages to connected peers by RPC.
* test: Handle QGETDATA/QDATA messages in mininode
* test: Add data structures to support QGETDATA/QDATA
* test: Add some helper in test_framework.py
* test: Implement tests for QGETDATA/QDATA in p2p_quorum_data.py
* test: Add p2p_quorum_data.py to BASE_SCRIPTS
* llmq|test: Add QWATCH support for QGETDATA/QDATA
* llmq: Store CQuorumPtr in cache, not CQuorumCPtr
* llmq: Fix cache usage after recent changes
* Use uacomment to create/find specific p2ps
* No need to use network adjusted time here, GetTime should be enough
* rpc: check proTxHash
* minor tweaks
* test: Adjustments after 4e27d6513e
* llmq: Rename and improve error lambda in CQuorumManager::ProcessMessage
* llmq: Process QDATA if -watchquorums is enabled
* test: Handle qwatch messages in mininode
* test: Add test for -watchquorums support
* test: Just some empty lines
* test: Properly stop the p2p network thread at the end of the test
* rpc: Adjust "quorum getdata" parameter descriptions
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* rpc: Fix optionality of proTxHash in "quorum getdata" command
* test: Test optionality of proTxHash for "quorum getdata" command
* test: Be more specific about imports in p2p_quorum_data.py
* llmq|rpc: Add some comments about the request.GetDataMask checks
* test: Some more empty lines
* rpc: One more parameter description
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* test: Unify assert statements / drop parentheses for all of them
* fix typo
Signed-off-by: pasta <pasta@dashboost.org>
* adjust some line wrapping to 80 chars
Signed-off-by: pasta <pasta@dashboost.org>
* tests: Seperate out into dif atomic methods, add logging
Signed-off-by: pasta <pasta@dashboost.org>
* test: Avoid restarting masternodes, just let available requests expire
Just takes a lot time and isn't required imo.
* test: Drop redundant code/tests after separation
This was introduced in 9e224ec2f2
* test: Merge three tests
"test_mnauth_restriction", "test_invalid_messages" and "test_invalid_unexpected_qdata" with the resulting name "test_basics" because i don't feel like DKG recovery thing should be part of a test called "test_invalid_messages" and giving it an own test probably wouldn't make a lot sense because it would still depend on "test_invalid_messages". I also think there is no need for a separated "test_invalid_unexpected_qdata".
* test: Rename test_ratelimiting_banscore -> test_request_limit
* test: Apply python style
* test: Wrap all at 120 characters
Thats the default "draw annoying warnings" setting for PyCharm (and IMO a reasonable line length).
* test: Move some variables
* test: Optimize for speed
* tests: use wait_until in get_mininode_id
* test: Don't use `!=` to check for `None`
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: pasta <pasta@dashboost.org>
* Merge #16509: test: Adapt test framework for chains other than "regtest"
faf36838bdba7393960fce6ad0c56dc1f93f5870 test: Avoid hardcoding the chain name in combine_logs (MarcoFalke)
fa8a1d7ba30040f8c74f93fc41a61276c255a6a6 test: Adapt test framework for chains other than "regtest" (MarcoFalke)
68f546635d5de2ccfedadeabc7bc79e12e5eca6a test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley)
Pull request description:
This is required for various work in progress:
* testchains #8994
* signet #16411
* some of my locally written tests
While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes.
ACKs for top commit:
jonatack:
ACK faf36838bdba7393960fce6ad0c56dc1f93f5870, ran tests and reviewed the code.
Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6
* Add devnet support for tests
* test: make sure devnet can connect to each other and start
* Partial merge bitcoin/bitcoin#16681: Tests: Use self.chain instead of 'regtest' in almost all current tests, revert one TODO while at it
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: Jorge Timón <jtimon@jtimon.cc>
a0b604c166 [tests] skip rpc_zmq functional test when python3 zmq lib is not present (James O'Beirne)
Pull request description:
As noted in https://github.com/bitcoin/bitcoin/pull/13570/files#r201715904, the `rpc_zmq` functional test should be skipped when the `zmq` python3 package is not installed. This is breaking https://bitcoinperf.com benchmarks at the moment.
Tree-SHA512: ab519ae717f4b7a282640cf0389651723fdc108990aeb9852e8b9e96d61fa1ded2461717ae31558b37ff8401a5b1ccc41f4e858e402b8c3d98563d962599767a
c8330d4 qa: Use node.datadir instead of tmpdir in test framework (MarcoFalke)
Pull request description:
Commit c53c9831ee introduced the utility function `get_datadir_path`, however not all places in the code use this util function. Using the util function everywhere makes it easier to review pull requests related to the datadir.
This commit replaces datadir path creation with the `datadir` member of `TestNode`, which itself uses `get_datadir_path`.
Tree-SHA512: c75707ab7149d732a6d56152a5813138a33459d3d07577b60b89f2a207c83b7663fac5f203593677c9892d1c23a5eba4bd45c5c4ababf040d720b437240fcddf
a7324bd79 Format timestamps using ISO 8601 formatting (e.g. "2018-02-28T12:34:56Z") (practicalswift)
Pull request description:
Print timestamp strings in logs using [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) formatting (e.g. `2018-02-28T12:34:56Z`):
* `Z` is the zone designator for the zero [UTC](https://en.wikipedia.org/wiki/Coordinated_Universal_Time) offset.
* `T` is the delimiter used to separate date and time.
This makes it clear for the end-user that the date/time logged is specified in UTC and not in the local time zone.
Before this patch:
```
2018-02-28 12:34:56 New outbound peer connected: version: 70015, blocks=1286123, peer=0
```
After this patch:
```
2018-02-28T12:34:56Z New outbound peer connected: version: 70015, blocks=1286123, peer=0
```
Tree-SHA512: 52b53c3d3d11ddf4af521a3b5f90a79f6e6539ee5955ec56a5aa2c6a5cf29cecf166d8cb43277c62553c3325a31bcea83691acbb4e86429c523f8aff8d7b210a
Signed-off-by: pasta <pasta@dashboost.org>
face722 qa: Move common args to bitcoin.conf (MarcoFalke)
Pull request description:
Beside removing duplicates of the same args in the code, this actually helps with debugging after a test failure.
For example, `bitcoin-qt` has `server` turned off, so you'd have to turn it on every time, if you wanted to debug a temporary test datadir created by the test framework.
Also, `keypool` would fill up if you forget to specify `-keypool=1`.
Tree-SHA512: 996bec8738dc0fce7297ab1fc5b4fbe3aa31b9b6241f8c4e685db728925363ccaca6145ad1edc6bee2f360e02ac4b9a53fcdff74eb79de90793393742d52b559
* Add LLMQ_100_67 quorums
* Re-use DEPLOYMENT_V17 bit to activate LLMQ_100_67 quorums
* Add LLMQ_TEST_NEW quorum and test its activation
* Tweak mine_quorum to work correctly with multiple quorum types
And to avoid a potentialy endless "while" loop
* llmq: Rename IsQuorumTypeEnabledAtBlock -> IsQuorumTypeEnabled
* chainparams|test: Rename llmq_test_new -> llmq_test_v17
* chainparams|consensus|llmq: Rename LLMQ_TEST_NEW -> LLMQ_TEST_V17
* Tweak few strings and the name of the test
* llmq: Make GetEnabledQuorumTypes return a vector of LLMQTypes, introduce GetLLMQParams
Signed-off-by: pasta <pasta@dashboost.org>
* Tweak minSize
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* Exclude LLMQ_100_67 from Concentrated Recovery
* Update test/functional/feature_new_quorum_type_activation.py
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: pasta <pasta@dashboost.org>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* Send islock notifications for txes received after their islocks were received
Also drop UpdateWalletTransaction - its name makes no sense and it's only used once.
* tests: early islocks should trigger notifications once a corresponding tx is received
* Tweak tests
- fail if an unexpected islock is received
- drop unused variable
* llmq: Drop `c_str()` in two log statements
* test: Move create_islock to DashTestFramework in test_framework.py
Just because it's used the same way in two files
* test: Simplify send tx in zmq test
* format
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* tests: Bump mocktime in smaller intervals while (not) signaling
This is needed to avoid triggering `CMasternodeSync::Reset()`.
* tests: force faucet node to finish mnsync in prepare_datadirs
* test: Drop redundant force_finish_mnsync in interface_zmq_dash.py
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* tests: Use lower mocktime bump value to reset masternode probes in feature_llmq_simplepose.py
Bumping `50 * 60 + 1` should be enough, see https://github.com/dashpay/dash/blob/master/src/llmq/quorums_utils.cpp#L218-L222. Bumping `60 * 60 + 1` interferes with mnsync reset, see https://github.com/dashpay/dash/blob/master/src/masternode/masternode-sync.cpp#L112-L119.
* Fix expected connection count in llmq-signing.py and llmq-simplepose.py
* Sleep a couple of seconds to let mn sync tick to happen
* Move helper functions out of run_test
* Let helper functions return expect_contribution_to_fail
* No need to check for "punished" state in test_banning
* Split mninfos in test_banning and mine_quorum into "online" and "valid" sets
Needed for wait_for_masternode_probes in mine_quorum. Also, refactor and fix test_banning while at it.
fae1374 qa: Allow for partial_match when checking init error (MarcoFalke)
5812273 [Tests] Require exact match in assert_start_raises_init_eror() (John Newbery)
0ec08a6 [Tests] Move assert_start_raises_init_error method to TestNode (John Newbery)
Pull request description:
Extracted from #12379, because the changes are important on their own.
This allows for exact testing, since the match can be specified with a strict regex. Internal details (such as exact formatting of the error message) can still be fuzzed away by regex wildcards.
Tree-SHA512: 605d2c9c42362a32d42321b066637577a026d0bb8cfc1c9f5737a4ca6503ffe85457a5122cea6e1101053ccc6c8aa1bbae3602e1fa7d2988bf7d5c275f412f66
Signed-off-by: pasta <pasta@dashboost.org>
fa811b0 qa: Normalize executable location (MarcoFalke)
Pull request description:
This removes the need to override the executable locations by just reading them from the config file. Beside making the code easier to read, running individual test on Windows is now possible by default (without providing further command line arguments).
Note: Of course, it is still possible to manually specify the location through the `BITCOIND` environment variable, e.g. `bitcoin-qt`.
Tree-SHA512: bee6d22246796242d747120ca18aaab089f73067de213c9111182561985c5912228a0b0f7f9eec025ecfdb44db031f15652f30d67c489d481c995bb3232a7ac7
* test: Add --timeoutscale to the test framework
Allows to scale the test timeouts by multiplying them with the value provided
with --timeoutscale. This is mostly meant to be used by CI where time
seems to be rare from time to time.
Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>
* test: Use wait_until in llmq-signing.py
This is to let it using the same wait code as the other tests.
Also this change makes sure --timeoutscale gets applied to the wait
conditions in this test too.
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
b156ff7c3 [tests] bind functional test nodes to 127.0.0.1 (Sjors Provoost)
Pull request description:
Replaces #12200 which broke `rpc_bind.py`.
Prevents OSX firewall allow-this-application-to-accept-inbound-connections permission popups and is generally safer.
To prevent binding to `127.0.0.1`, set `self.bind_to_localhost_only = False`.
cc @jnewbery
Tree-SHA512: 5e700124c91bd0cbdee83ca44910071d71d61d8842334755b685d14fbff6454d75de1ea7de67340370386f58b41361e80e90bb4dca5c4d5992f9d2b27985f999
fa8071a0985700a4641ce77dac2cb2fa285d3afe qa: Log as utf-8 (MarcoFalke)
Pull request description:
Explicitly read and write the log files with utf-8 as encoding
Tree-SHA512: ca28f37f34a09845c736ff6c4c21733c3c39584f52c81e48ff25e5e35979c317d0989862b2b93acc7e359fbcc20b99533365455830b2ddb41eb4d8c17314534e
faefd29 qa: Prepare functional tests for Windows (MarcoFalke)
Pull request description:
* Pass `sys.executable` when calling a python script via the subprocess
module
* Don't remove the log file while it is still open and written to
* Properly use os.pathsep and os.path.sep when modifying the PATH
environment variable
* util-tests: Use os.path.join for Windows compatibility
Ref: #8227
Tree-SHA512: c507a536af104b3bde4366b6634099db826532bd3e7c35d694b5883c550920643b3eab79c76703ca67e1044ed09979e855088f7324321c8d52112514e334d614
* Trivial Dashification
* Tweak getnetworkinfo and dumpwallet help text
We don't have RBF and Segwit
* CopyrightHolders should also check for missing "Dash Core" copyright
Instead of using the initial value. This removes the need for manually
passing of -mocktime when restarting nodes.
It also fixes a few flaky test cases where nodes are getting restarted.
And really only check inbound connections for recent probes. Also bump
mocktime by an hour in llmq-simplepose.py.
This fixes flakiness of llmq-simplepose.py.
