* docs: Add help for -recsigsmaxage option
* Make `-llmq-data-recovery` help string dynamic
* Make devnet/regtest specific help strings dynamic
Only show devnet related help strings on devnet.
* Make linter happy
* Skip FindDevNetGenesisBlock when creating devnet params for help purposes only
* llmq: Use thread pool for quorum cache and data recovery threads
Move quorum data and cache thread handling into CQuorumManager.
* llmq: Fix explicit capture list
* llmq: Directly push the lamdas to workerPool
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* instantsend: Bail out early on disabled IS in more places
* instantsend: Disable InstantSend while reindexing and importing blocks
* Drop extra checks in GetInstantSendLockHashByTxid, GetInstantSendLockByInput and GetInstantSendLockByHash
* instantsend: Introduce (and use) KnownInstantSendLock
Check islock hash against both current and archived islocks
* instantsend: Optimize ProcessInstantSendLock, check for known islocks first
Two reasons:
1. GetTransaction is a potentially much heavier one
2. Doesn't make much sense to look for a tx if we have a known islock already
* Drop extra pendingInstantSendLocks.count(hash)
* Check pindexMined != nullptr
* Update src/llmq/quorums_instantsend.cpp
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Update src/llmq/quorums_instantsend.h
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* llmq: Drop InternalHasConflictingChainLock in ProcessNewChainLock
* llmq: Directly use clsig.blockHash instead of copying it into msgHash
* llmq: Reuse CInv(MSG_CLSIG, hash)
* llmq: Add const in two places
* instantsend: Mark a block with IS-locks which conflict with txes in a CL-ed block as conflicting and not as invalid
* tests: Tweak feature_llmq_is_cl_conflicts.py to test CL overriding a block with conflicting IS-locks
* instantsend|sigs: Sleep when there is no more work
Instead of sleeping only when no work has been done.
Avoids useless cycles, improves batching.
* llmq: Add and use nMaxBatchSize
* llmq: Compare to what we got in return, not what we verified at the end
It might happen that we get 32 pending but do only verify less than 32 and in this case we would assume there is no more work but it could still be more in the pipeline from my understanding.
* llmq: Rename more_work -> fMoreWork
* llmq: Be consistent with the other fMoreWork initialization
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* llmq: Avoid writing commitments to evodb and altering caches when all we want is to check block candidate validity
* tests: call `getblocktemplate` to trigger `CreateNewBlock` before quorum commitment is mined
* llmq: Implement automated DKG recovery threads
* llmq: Implement quorum verification vector sync
* init: Validiate quorum data recovery related command line parameter
* test: Add quorum_data_request_timeout_seconds in DashTestFramework
* test: Test quorum data recovery in feature_llmq_data_recovery.py
* test: Add feature_llmq_data_recovery.py to BASE_SCRIPTS
* test: Fix quorum_data_request_expiration_timeout in wait_for_quorum_data
* test: Always test the existence of secretKeyShare in test_mn_quorum_data
With this change it also validates that "secretKeyShare" is not in `quorum_info` if its not expected to be in there. Before this was basically just not tested.
* llmq|test: Use bool as argument type for -llmq-data-recovery
* llmq: Always set nTimeLastSuccess to 0
* test: Set -llmq-data-recovery=0 in p2p_quorum_data.py
* test: Simplify test_mns
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* refactor: pass CQuorumCPtr to StartQuorumDataRecoveryThread
* test: Fix thread name in comment
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* version: Bump PROTOCOL_VERSION and MIN_MASTERNODE_PROTO_VERSION
* version: Introduce LLMQ_DATA_MESSAGES_VERSION for QGETDATA/QDATA support
* test: Bump MY_VERSION to 70219 (LLMQ_DATA_MESSAGES_VERSION)
* llmq: Introduce CQuorumDataRequest as wrapper for QGETDATA requests
* llmq: Implement CQuorum::{SetVerificationVector, SetSecretKeyShare}
* llmq|net|protocol: Implement QGETDATA/QDATA P2P messages
* llmq: Restrict processing QGETDATA/QDATA to masternodes only
* llmq: Implement request limiting for QGETDATA/QDATA
* llmq: Implement CQuorumManger::RequestQuorumData
* rpc: Implement "quorum getdata" as wrapper around QGETDATA
Allows to trigger sending QGETDATA messages to connected peers by RPC.
* test: Handle QGETDATA/QDATA messages in mininode
* test: Add data structures to support QGETDATA/QDATA
* test: Add some helper in test_framework.py
* test: Implement tests for QGETDATA/QDATA in p2p_quorum_data.py
* test: Add p2p_quorum_data.py to BASE_SCRIPTS
* llmq|test: Add QWATCH support for QGETDATA/QDATA
* llmq: Store CQuorumPtr in cache, not CQuorumCPtr
* llmq: Fix cache usage after recent changes
* Use uacomment to create/find specific p2ps
* No need to use network adjusted time here, GetTime should be enough
* rpc: check proTxHash
* minor tweaks
* test: Adjustments after 4e27d6513e
* llmq: Rename and improve error lambda in CQuorumManager::ProcessMessage
* llmq: Process QDATA if -watchquorums is enabled
* test: Handle qwatch messages in mininode
* test: Add test for -watchquorums support
* test: Just some empty lines
* test: Properly stop the p2p network thread at the end of the test
* rpc: Adjust "quorum getdata" parameter descriptions
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* rpc: Fix optionality of proTxHash in "quorum getdata" command
* test: Test optionality of proTxHash for "quorum getdata" command
* test: Be more specific about imports in p2p_quorum_data.py
* llmq|rpc: Add some comments about the request.GetDataMask checks
* test: Some more empty lines
* rpc: One more parameter description
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* test: Unify assert statements / drop parentheses for all of them
* fix typo
Signed-off-by: pasta <pasta@dashboost.org>
* adjust some line wrapping to 80 chars
Signed-off-by: pasta <pasta@dashboost.org>
* tests: Seperate out into dif atomic methods, add logging
Signed-off-by: pasta <pasta@dashboost.org>
* test: Avoid restarting masternodes, just let available requests expire
Just takes a lot time and isn't required imo.
* test: Drop redundant code/tests after separation
This was introduced in 9e224ec2f2
* test: Merge three tests
"test_mnauth_restriction", "test_invalid_messages" and "test_invalid_unexpected_qdata" with the resulting name "test_basics" because i don't feel like DKG recovery thing should be part of a test called "test_invalid_messages" and giving it an own test probably wouldn't make a lot sense because it would still depend on "test_invalid_messages". I also think there is no need for a separated "test_invalid_unexpected_qdata".
* test: Rename test_ratelimiting_banscore -> test_request_limit
* test: Apply python style
* test: Wrap all at 120 characters
Thats the default "draw annoying warnings" setting for PyCharm (and IMO a reasonable line length).
* test: Move some variables
* test: Optimize for speed
* tests: use wait_until in get_mininode_id
* test: Don't use `!=` to check for `None`
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: pasta <pasta@dashboost.org>
* llmq: Detach dash-q-cachepop from caller
There should be no reason to keep this tread attached
to its parent, if so, let me know.
* llmq: Avoid nullptr access for pindexStart in ScanQuorums
* llmq: Add cacheKey in ProcessCommitment
* llmq: Erase minable commitments if they have been processed
* llmq: Add CLLMQUtils::InitQuorumsCache
* llmq: Use unordered_lru_cache for quorumsCache and rename it
* llmq: Use unordered_lru_cache for hasMinedCommitmentCache and rename it
* llmq: Drop redundant check
* llmq: Rename nMaxCount2 -> nScanCommitments
* llmq: Refactor storeCache -> fCacheExists
* llmq: Rename maxCount -> nCountRequested
* llmq: Rename result -> vecResultQuorums
* llmq: Return an empty vector if the are zero elements requested
* unordered_lru_cache: Add max_size()
* llmq: Partially reuse existing cache if more than max is requested
* llmq: std::map<LLMQType, unordered_lru_cache<...>> for scanQuoumsCache
* llmq: Drop params
* llmq: Only emplace to cache if there is something available
* llmq: Add CDKGSessionManager::WriteEncryptedContributions
Allows to store each member's encrypted contributions of the DKG.
* llmq: Store each member's contributions in the llmq database
* llmq: Add CDKGSessionManager::GetEncryptedContributions
I decided to don't cache here since its probably very unlikely this is called twice in a short period with what we have planed for it so far. We can add caching if the requirement for it changes at some point?
* Use single-threaded scheduler for IS, CL and Governance notifications
* Pass shared_ptr-s instead of objects themselves for CL, IS and Governance notifiers in CMainSignals/CValidatibnInterface
* llmq: Create shared_ptr for clsig at the root of its lifetime
* llmq: Create shared_ptr for islock clsig at the root of its lifetime
* llmq: Create shared_ptr for recSig at the root of its lifetime
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* llmq: Refactor CQuorumManager::{BuildQuorumFromCommitment, GetQuorum}
Construct and cache new quorums inside BuildQuorumFromCommitment
* llmq: Make all methods of CQuorumManager const
* More accurate handling of the BLOCK_CONFLICT_CHAINLOCK flag
* Update test/functional/feature_llmq_chainlocks.py
Co-authored-by: thephez <thephez@users.noreply.github.com>
* tests: make sure that previous tip on the reorged node is marked conflicting after chainlock
* Apply suggestions from code review
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: thephez <thephez@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* llmq: Split CSigShare creation/processing in CSigSharesManager
* rpc: Add "submit" parameter to "quorum sign"
* test: Add CSigShare and msg_qsigshare to messages.py
* test: Test the optional "submit" parameter of "quorum sign"
* random: Introduce std::shuffle alternative for FastRandomContext
3db746beb4
* random: change std::random_shuffle calls to std::shuffle
https://en.cppreference.com/w/cpp/algorithm/random_shuffle (deprecated in c++14)
* random: change FastRandomContext std::random_shuffle calls to shuffle
* random: change last std::shuffle calls to Shuffle
std::shuffle doesn't accept only two arguments so we use FastRandomContext()
* llmq: use inherited FastRandomContext
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* llmq: use inherited FastRandomContext
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Make the linter happy :)
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Do not store hashes in ProcessPendingMessageBatch and do not pass them to ReceiveMessage-s
* follow up: refactor/unify code in ProcessPendingMessageBatch
* llmq: Add an optional quorum hash to CSigningManager::AsyncSignIfMember
Allows to select the quorum to sign by its hash.
* rpc: Fix quorum selection of "quorum sign"
* test: Test the optional "quorumHash" parameter of "quorum sign"
* llmq: Move quorum checks up to avoid calling WriteVoteForId if they fail
* llmq: Drop hash parameter in PreVerifyMessage methods
* llmq: Drop some unused variables
* rpc: Drop unused variable
* llmq|net: Drop some unused CConnman parameter
* llmq: Drop some unused quorum parameter
* llmq: Drop some unused nodeId parameter
* Drop unused variables
* llmq: Drop more
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* llmq: Fix GetVerifiedContribution to return false in case of failure
* llmq: Move GetVerifiedContribution into GetVerifiedContributions
* llmq: Drop GetVerifiedContribution
* llmq: Keep cache locked while building GetVerifiedContributions result
* llmq: Read from DB into vvecPtr directly
Rename `sigSharesToAnnounce` class member in `CSigSharesManager` to `sigSharesQueuedToAnnounce` to avoid confusion with local variables/function args with the same name
* llmq: Restrict `ShouldSimulateError` to trigger for LLMQ_TEST only
Current `develop` tests fail. This was basically introduced by dashpay#3844 but it didn't come up before dashpay#3853 because the `v17` fork wasn't activated in `feature_llmq_dkgerrors.py`.
After dashpay#3853 `dip0008` activation takes [200 blocks](b95cf017c3 (diff-4a04bc0b355c780033960e8c261ee9b6d3c452897e1dcd88a15d272512266c76R539)) which was normally activated after [10 blocks](b95cf017c3 (diff-b92fa0fafafa27172736ebc88f9f9b658b1160caca512a318eefb7d93d22bf3cL18)) in `feature_llmq_dkgerrors.py`. Now with the 200 blocks `v17` gets activated during test which then leads to MN1, MN2 banning MN0 because it lies in DKG of `LLMQ_TEST` and `LLMQ_TEST_V17`.
There are other ways to solve it, like enabling `dip0008` earlier or enable `v17` later but IMO its anyway better to restrict `ShouldSimulateError` to only trigger for `LLMQ_TEST`.
* Revert "llmq: Restrict `ShouldSimulateError` to trigger for LLMQ_TEST only"
This reverts commit ec42d86126.
* llmq: Restrict `ShouldSimulateError` to trigger for LLMQ_TEST only (alternative)
Move ShouldSimulateError into CDKGSession
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Add LLMQ_100_67 quorums
* Re-use DEPLOYMENT_V17 bit to activate LLMQ_100_67 quorums
* Add LLMQ_TEST_NEW quorum and test its activation
* Tweak mine_quorum to work correctly with multiple quorum types
And to avoid a potentialy endless "while" loop
* llmq: Rename IsQuorumTypeEnabledAtBlock -> IsQuorumTypeEnabled
* chainparams|test: Rename llmq_test_new -> llmq_test_v17
* chainparams|consensus|llmq: Rename LLMQ_TEST_NEW -> LLMQ_TEST_V17
* Tweak few strings and the name of the test
* llmq: Make GetEnabledQuorumTypes return a vector of LLMQTypes, introduce GetLLMQParams
Signed-off-by: pasta <pasta@dashboost.org>
* Tweak minSize
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* Exclude LLMQ_100_67 from Concentrated Recovery
* Update test/functional/feature_new_quorum_type_activation.py
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: pasta <pasta@dashboost.org>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* Send islock notifications for txes received after their islocks were received
Also drop UpdateWalletTransaction - its name makes no sense and it's only used once.
* tests: early islocks should trigger notifications once a corresponding tx is received
* Tweak tests
- fail if an unexpected islock is received
- drop unused variable
* llmq: Drop `c_str()` in two log statements
* test: Move create_islock to DashTestFramework in test_framework.py
Just because it's used the same way in two files
* test: Simplify send tx in zmq test
* format
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* Fix potential deadlock in `CSporkManager::UpdateSpork()`
* Protect `inputRequestIds` with cs lock
* Protect `curDBTransaction` in `CEvoDB::CommitRootTransaction()`
* Check for `AssertLockNotHeld` in `EnforceBestChainLock()` instead of just having a comment in code
* Protect spork maps on (de)serialization
* Handle situations when we have evodb records but no blocks anymore
* Use IsEmpty
* Apply suggestions from code review
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
