* llmq: Use thread pool for quorum cache and data recovery threads
Move quorum data and cache thread handling into CQuorumManager.
* llmq: Fix explicit capture list
* llmq: Directly push the lamdas to workerPool
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* instantsend: Bail out early on disabled IS in more places
* instantsend: Disable InstantSend while reindexing and importing blocks
* Drop extra checks in GetInstantSendLockHashByTxid, GetInstantSendLockByInput and GetInstantSendLockByHash
* instantsend: Introduce (and use) KnownInstantSendLock
Check islock hash against both current and archived islocks
* instantsend: Optimize ProcessInstantSendLock, check for known islocks first
Two reasons:
1. GetTransaction is a potentially much heavier one
2. Doesn't make much sense to look for a tx if we have a known islock already
* Drop extra pendingInstantSendLocks.count(hash)
* Check pindexMined != nullptr
* Update src/llmq/quorums_instantsend.cpp
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Update src/llmq/quorums_instantsend.h
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* llmq: Drop InternalHasConflictingChainLock in ProcessNewChainLock
* llmq: Directly use clsig.blockHash instead of copying it into msgHash
* llmq: Reuse CInv(MSG_CLSIG, hash)
* llmq: Add const in two places
* instantsend: Mark a block with IS-locks which conflict with txes in a CL-ed block as conflicting and not as invalid
* tests: Tweak feature_llmq_is_cl_conflicts.py to test CL overriding a block with conflicting IS-locks
* qt: Fix labels in transaction list
The issue was introduced in #3682
* qt: Always use labels from TransactionStatus for transaction list
Missed this in #3155
* Update src/qt/transactiontablemodel.cpp
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* instantsend|sigs: Sleep when there is no more work
Instead of sleeping only when no work has been done.
Avoids useless cycles, improves batching.
* llmq: Add and use nMaxBatchSize
* llmq: Compare to what we got in return, not what we verified at the end
It might happen that we get 32 pending but do only verify less than 32 and in this case we would assume there is no more work but it could still be more in the pipeline from my understanding.
* llmq: Rename more_work -> fMoreWork
* llmq: Be consistent with the other fMoreWork initialization
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* docs: add help messages for missing devnet options
* docs: add help for llmq recovery
* docs: add help for regtest llmq
* docs: add help for sporkkey
* Apply suggestions from code review
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
fa5ed4f8d2 refactor: Avoid locking tx pool cs thrice (MarcoFalke)
Pull request description:
`addUnchecked` is (outside the tests) only called by ATMP, which already takes the tx pool read lock. So locking it twice more in both `addUnchecked` methods seems redundant.
Similarly `CalculateMemPoolAncestors` is (beside once in the wallet) only called in contexts, where the tx pool lock is already taken. So remove the lock there as well.
Tree-SHA512: fcf603b570da0fc529fe6db8add218663eae52845510732bee0d4611263d2429d3d3c9c8ae68493d67287d13504500ed51905ccbe711eb15a0af3b019edad543
f77e1d34fd5f17304ce319b5f962b8005592501a test: Add MempoolAncestryTests (Karl-Johan Alm)
a08d76bcfee6f563a268933357931abe32060668 mempool: Calculate descendant maximum thoroughly (Karl-Johan Alm)
6d3568371eb2559d65a3e2334252d36a262319e8 wallet: Switch to using ancestor/descendant limits (Karl-Johan Alm)
6888195b062c8c58dd776fd10b44b25554eb1f15 wallet: Strictly greater than for ancestor caps (Karl-Johan Alm)
322b12ac4e0a8c892e81a760ff7225619248b74f Remove deprecated TransactionWithinChainLimit (Karl-Johan Alm)
47847515473b054929af0c8de3d54b6672500cab Switch to GetTransactionAncestry() in OutputEligibleForSpending (Karl-Johan Alm)
475a385a80198a46a6d99846f99b968f04e9b470 Add GetTransactionAncestry to CTxMemPool for general purpose chain limit checking (Karl-Johan Alm)
46847d69d2c1cc908fd779daac7884e365955dbd mempool: Fix max descendants check (Karl-Johan Alm)
b9ef21dd727dde33f5bd3c33226b05d07eb12aac mempool: Add explicit max_descendants (Karl-Johan Alm)
Pull request description:
Currently, `TransactionWithinChainLimit` is restricted to single-output use, and needs to be called every time for different limits. If it is replaced with a chain limit value calculator, that can be called once and reused, and is generally more flexible (see e.g. #12257).
Update: this PR now corrects usage of max ancestors / max descendants, including calculating the correct max descendant value, as advertised for the two limits.
~~This change also makes `nMaxAncestors` signed, as the replacement method will return `-1` for "not in the mempool", which is different from "0", which means "no ancestors/descendants in mempool".~~
~~This is a subset of #12257.~~
Tree-SHA512: aa59c849360542362b3126c0e29d44d3d58f11898e277d38c034dc4b86a5b4500f77ac61767599ce878c876b5c446fec9c02699797eb2fa41e530ec863a00cf9
* llmq: Avoid writing commitments to evodb and altering caches when all we want is to check block candidate validity
* tests: call `getblocktemplate` to trigger `CreateNewBlock` before quorum commitment is mined
* Merge #13199: Bugfix: ensure consistency of m_failed_blocks after reconsiderblock
11fa6bb66e Bugfix: ensure consistency of m_failed_blocks after reconsiderblock (Suhas Daftuar)
Pull request description:
This was introduced in 015a5258ad and could cause a node to crash (due to assertion failure) when using the `reconsiderblock` rpc.
Tree-SHA512: 820dcd761bf983e36f5d0f16777ed75c833daaf62a6b3a4dbd17f6caaf9287223e3a202d06540ac62f8ba72926b73b0873bb76c6273ddcb19d9408f4c1cd325e
* bugfix: Mark all nearest BLOCK_FAILED_CHILD descendants (if any) as BLOCK_FAILED_VALID while removing the invalidity flag from all ancestors in ResetBlockFailureFlags
Fixes `Assertion failed: ((pindex->nStatus & BLOCK_FAILED_MASK) == 0), function CheckBlockIndex`
* tests: Make sure ResetBlockFailureFlags does the job correctly
* Wait for the expected block height, check the final chain tip hash
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
* llmq: Implement automated DKG recovery threads
* llmq: Implement quorum verification vector sync
* init: Validiate quorum data recovery related command line parameter
* test: Add quorum_data_request_timeout_seconds in DashTestFramework
* test: Test quorum data recovery in feature_llmq_data_recovery.py
* test: Add feature_llmq_data_recovery.py to BASE_SCRIPTS
* test: Fix quorum_data_request_expiration_timeout in wait_for_quorum_data
* test: Always test the existence of secretKeyShare in test_mn_quorum_data
With this change it also validates that "secretKeyShare" is not in `quorum_info` if its not expected to be in there. Before this was basically just not tested.
* llmq|test: Use bool as argument type for -llmq-data-recovery
* llmq: Always set nTimeLastSuccess to 0
* test: Set -llmq-data-recovery=0 in p2p_quorum_data.py
* test: Simplify test_mns
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* refactor: pass CQuorumCPtr to StartQuorumDataRecoveryThread
* test: Fix thread name in comment
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* version: Bump PROTOCOL_VERSION and MIN_MASTERNODE_PROTO_VERSION
* version: Introduce LLMQ_DATA_MESSAGES_VERSION for QGETDATA/QDATA support
* test: Bump MY_VERSION to 70219 (LLMQ_DATA_MESSAGES_VERSION)
* llmq: Introduce CQuorumDataRequest as wrapper for QGETDATA requests
* llmq: Implement CQuorum::{SetVerificationVector, SetSecretKeyShare}
* llmq|net|protocol: Implement QGETDATA/QDATA P2P messages
* llmq: Restrict processing QGETDATA/QDATA to masternodes only
* llmq: Implement request limiting for QGETDATA/QDATA
* llmq: Implement CQuorumManger::RequestQuorumData
* rpc: Implement "quorum getdata" as wrapper around QGETDATA
Allows to trigger sending QGETDATA messages to connected peers by RPC.
* test: Handle QGETDATA/QDATA messages in mininode
* test: Add data structures to support QGETDATA/QDATA
* test: Add some helper in test_framework.py
* test: Implement tests for QGETDATA/QDATA in p2p_quorum_data.py
* test: Add p2p_quorum_data.py to BASE_SCRIPTS
* llmq|test: Add QWATCH support for QGETDATA/QDATA
* llmq: Store CQuorumPtr in cache, not CQuorumCPtr
* llmq: Fix cache usage after recent changes
* Use uacomment to create/find specific p2ps
* No need to use network adjusted time here, GetTime should be enough
* rpc: check proTxHash
* minor tweaks
* test: Adjustments after 4e27d6513e
* llmq: Rename and improve error lambda in CQuorumManager::ProcessMessage
* llmq: Process QDATA if -watchquorums is enabled
* test: Handle qwatch messages in mininode
* test: Add test for -watchquorums support
* test: Just some empty lines
* test: Properly stop the p2p network thread at the end of the test
* rpc: Adjust "quorum getdata" parameter descriptions
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* rpc: Fix optionality of proTxHash in "quorum getdata" command
* test: Test optionality of proTxHash for "quorum getdata" command
* test: Be more specific about imports in p2p_quorum_data.py
* llmq|rpc: Add some comments about the request.GetDataMask checks
* test: Some more empty lines
* rpc: One more parameter description
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* test: Unify assert statements / drop parentheses for all of them
* fix typo
Signed-off-by: pasta <pasta@dashboost.org>
* adjust some line wrapping to 80 chars
Signed-off-by: pasta <pasta@dashboost.org>
* tests: Seperate out into dif atomic methods, add logging
Signed-off-by: pasta <pasta@dashboost.org>
* test: Avoid restarting masternodes, just let available requests expire
Just takes a lot time and isn't required imo.
* test: Drop redundant code/tests after separation
This was introduced in 9e224ec2f2
* test: Merge three tests
"test_mnauth_restriction", "test_invalid_messages" and "test_invalid_unexpected_qdata" with the resulting name "test_basics" because i don't feel like DKG recovery thing should be part of a test called "test_invalid_messages" and giving it an own test probably wouldn't make a lot sense because it would still depend on "test_invalid_messages". I also think there is no need for a separated "test_invalid_unexpected_qdata".
* test: Rename test_ratelimiting_banscore -> test_request_limit
* test: Apply python style
* test: Wrap all at 120 characters
Thats the default "draw annoying warnings" setting for PyCharm (and IMO a reasonable line length).
* test: Move some variables
* test: Optimize for speed
* tests: use wait_until in get_mininode_id
* test: Don't use `!=` to check for `None`
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: pasta <pasta@dashboost.org>
* fix typos
Signed-off-by: pasta <pasta@dashboost.org>
* remove unneeded initialization
Signed-off-by: pasta <pasta@dashboost.org>
* make method const
Signed-off-by: pasta <pasta@dashboost.org>
* use default for trivial destructor
Signed-off-by: pasta <pasta@dashboost.org>
* governance: remove all typedefs for iterators, use auto instead
Signed-off-by: pasta <pasta@dashboost.org>
* remove redundant size_type
Signed-off-by: pasta <pasta@dashboost.org>
* Remove unused and singly used typedefs
Signed-off-by: pasta <pasta@dashboost.org>
* remove unused code
Signed-off-by: pasta <pasta@dashboost.org>
* mark constructor as explicit
Signed-off-by: pasta <pasta@dashboost.org>
* remove unused typedef
Signed-off-by: pasta <pasta@dashboost.org>
* remove unneeded initialization
Signed-off-by: pasta <pasta@dashboost.org>
* remove singly used typedef
Signed-off-by: pasta <pasta@dashboost.org>
* pass const reference, and don't copy for no reason
Signed-off-by: pasta <pasta@dashboost.org>
* make method const
Signed-off-by: pasta <pasta@dashboost.org>
* make method const
Signed-off-by: pasta <pasta@dashboost.org>
* make method const
Signed-off-by: pasta <pasta@dashboost.org>
* typo
Signed-off-by: pasta <pasta@dashboost.org>
* make constructor explicit
Signed-off-by: pasta <pasta@dashboost.org>
* Clang-Tidy: 'virtual' is redundant since the function is already declared 'override'
Signed-off-by: pasta <pasta@dashboost.org>
* Clang-Tidy: Prefer using 'override' or (rarely) 'final' instead of 'virtual'
Signed-off-by: pasta <pasta@dashboost.org>
* use default for trivial destructor
Signed-off-by: pasta <pasta@dashboost.org>
* remove unused include
Signed-off-by: pasta <pasta@dashboost.org>
* remove unneeded semicolon
Signed-off-by: pasta <pasta@dashboost.org>
* fix typos
Signed-off-by: pasta <pasta@dashboost.org>
* fix typo
Signed-off-by: pasta <pasta@dashboost.org>
* mark constructor explicit
Signed-off-by: pasta <pasta@dashboost.org>
* remove unused typedef
Signed-off-by: pasta <pasta@dashboost.org>
* remove commented out code
Signed-off-by: pasta <pasta@dashboost.org>
* mark constructor explicit
Signed-off-by: pasta <pasta@dashboost.org>
* remove unused spork
Signed-off-by: pasta <pasta@dashboost.org>
* remove boolean check where always true
Signed-off-by: pasta <pasta@dashboost.org>
* make method const
Signed-off-by: pasta <pasta@dashboost.org>
* Remove nCount completely
Signed-off-by: pasta <pasta@dashboost.org>
* Use default path
Signed-off-by: pasta <pasta@dashboost.org>
* llmq: Detach dash-q-cachepop from caller
There should be no reason to keep this tread attached
to its parent, if so, let me know.
* llmq: Avoid nullptr access for pindexStart in ScanQuorums
* llmq: Add cacheKey in ProcessCommitment
* llmq: Erase minable commitments if they have been processed
* llmq: Add CLLMQUtils::InitQuorumsCache
* llmq: Use unordered_lru_cache for quorumsCache and rename it
* llmq: Use unordered_lru_cache for hasMinedCommitmentCache and rename it
* llmq: Drop redundant check
* llmq: Rename nMaxCount2 -> nScanCommitments
* llmq: Refactor storeCache -> fCacheExists
* llmq: Rename maxCount -> nCountRequested
* llmq: Rename result -> vecResultQuorums
* llmq: Return an empty vector if the are zero elements requested
* unordered_lru_cache: Add max_size()
* llmq: Partially reuse existing cache if more than max is requested
* llmq: std::map<LLMQType, unordered_lru_cache<...>> for scanQuoumsCache
* llmq: Drop params
* llmq: Only emplace to cache if there is something available
* Check mnemonic passphrase size in SetMnemonic instead of CreateWalletFromFile
* Move processing of cmd-line options and recovery via hdseed out of GenerateNewHDChain
* Implement GenerateNewHDChainEncrypted and tweak EncryptHDChain to be able to generate new encrypted HD chains in an already encrypted wallet
* rpc: Implement upgradetohd rpc
* Address review comments
* tweak rpc response
* tests: Test various non-HD to HD wallet upgrade paths
* Apply suggestions from code review
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Fix suggestions
* tests: Check upgradetohd return value
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* llmq: Add CDKGSessionManager::WriteEncryptedContributions
Allows to store each member's encrypted contributions of the DKG.
* llmq: Store each member's contributions in the llmq database
* llmq: Add CDKGSessionManager::GetEncryptedContributions
I decided to don't cache here since its probably very unlikely this is called twice in a short period with what we have planed for it so far. We can add caching if the requirement for it changes at some point?
* Use single-threaded scheduler for IS, CL and Governance notifications
* Pass shared_ptr-s instead of objects themselves for CL, IS and Governance notifiers in CMainSignals/CValidatibnInterface
* llmq: Create shared_ptr for clsig at the root of its lifetime
* llmq: Create shared_ptr for islock clsig at the root of its lifetime
* llmq: Create shared_ptr for recSig at the root of its lifetime
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* bls: Adjust CBLSIESEncryptedBlob to match CBLSIESMultiRecipientBlobs
With this change a single encrypted object of CBLSIESMultiRecipientBlobs is has the same structure like CBLSIESEncryptedBlob
* bls: Add CBLSIESEncryptedObject constructor
* bls: Add CBLSIESMultiRecipientObjects::Get
Allows to query a individual encrypted object as CBLSIESEncryptedObject
* bls: Drop unneeded constructor call
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Implement auto-recovery from hardforks
This should help users who fail to update their nodes/wallets in time when there is a hardfork.
* tests: tweak feature_llmq_chainlocks.py to check new behaviour
* tests: tidy up feature_llmq_chainlocks.py a bit
* Fix a couple of issues with multikey sporks cleanup
1. Should remove sporks with signatures from unknown signers from mapSporksActive
2. Should advance itSignerPair while doing (1)...
* tests: make sure sporks cleanup works as expected for multikey sporks
* tests: make sure multiple multikey sporks (and their cleanups) work together as expected
* Prettify node extra args
