Instead of calling `CLLMQUtils::GetAllQuorumMembers` before each `CFinalCommitment::Verify` call to get the quorum members, just get them inside `CFinalCommitment::Verify`.
* llmq: Avoid writing commitments to evodb and altering caches when all we want is to check block candidate validity
* tests: call `getblocktemplate` to trigger `CreateNewBlock` before quorum commitment is mined
* llmq: Detach dash-q-cachepop from caller
There should be no reason to keep this tread attached
to its parent, if so, let me know.
* llmq: Avoid nullptr access for pindexStart in ScanQuorums
* llmq: Add cacheKey in ProcessCommitment
* llmq: Erase minable commitments if they have been processed
* llmq: Add CLLMQUtils::InitQuorumsCache
* llmq: Use unordered_lru_cache for quorumsCache and rename it
* llmq: Use unordered_lru_cache for hasMinedCommitmentCache and rename it
* llmq: Drop redundant check
* llmq: Rename nMaxCount2 -> nScanCommitments
* llmq: Refactor storeCache -> fCacheExists
* llmq: Rename maxCount -> nCountRequested
* llmq: Rename result -> vecResultQuorums
* llmq: Return an empty vector if the are zero elements requested
* unordered_lru_cache: Add max_size()
* llmq: Partially reuse existing cache if more than max is requested
* llmq: std::map<LLMQType, unordered_lru_cache<...>> for scanQuoumsCache
* llmq: Drop params
* llmq: Only emplace to cache if there is something available
* llmq: Drop hash parameter in PreVerifyMessage methods
* llmq: Drop some unused variables
* rpc: Drop unused variable
* llmq|net: Drop some unused CConnman parameter
* llmq: Drop some unused quorum parameter
* llmq: Drop some unused nodeId parameter
* Drop unused variables
* llmq: Drop more
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Add LLMQ_100_67 quorums
* Re-use DEPLOYMENT_V17 bit to activate LLMQ_100_67 quorums
* Add LLMQ_TEST_NEW quorum and test its activation
* Tweak mine_quorum to work correctly with multiple quorum types
And to avoid a potentialy endless "while" loop
* llmq: Rename IsQuorumTypeEnabledAtBlock -> IsQuorumTypeEnabled
* chainparams|test: Rename llmq_test_new -> llmq_test_v17
* chainparams|consensus|llmq: Rename LLMQ_TEST_NEW -> LLMQ_TEST_V17
* Tweak few strings and the name of the test
* llmq: Make GetEnabledQuorumTypes return a vector of LLMQTypes, introduce GetLLMQParams
Signed-off-by: pasta <pasta@dashboost.org>
* Tweak minSize
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* Exclude LLMQ_100_67 from Concentrated Recovery
* Update test/functional/feature_new_quorum_type_activation.py
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: pasta <pasta@dashboost.org>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* Handle situations when we have evodb records but no blocks anymore
* Use IsEmpty
* Apply suggestions from code review
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Return invalid state for GetTxPayload failures instead of asserting
* Throw an error when internal `find` fails instead of asserting
* Unify mnlist checks for lists created from blocks and lists created via updates of older lists with diffs
* Wrap evo block processing in try..catch to make sure all earlier introduced throws are caught
* evo/llmq: Adjust reject reasons for CFinalCommitmentTxPayload
In CheckLLMQCommitment this reject string was used so i adjusted the
other one in CQuorumBlockProcessor::GetCommitmentsFromBlock and the new
introduced in PR #3506
* return state.DoS instead of asserting in cbtx.cpp
Signed-off-by: pasta <pasta@dashboost.org>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: pasta <pasta@dashboost.org>
* 🪲 improve evodb consistency recovering from dbcrash
* Adjust the fix
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Fix it
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Disable recovery from a crash during a fork and a corresponding part of dbcrash.py
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Skip some checks in CQuorumBlockProcessor when replaying blocks after the crash
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Process special txes in RollforwardBlock
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Update src/init.cpp
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* scripted-diff: Replace #include "" with #include <> (ryanofsky)
-BEGIN VERIFY SCRIPT-
for f in \
src/*.cpp \
src/*.h \
src/bench/*.cpp \
src/bench/*.h \
src/compat/*.cpp \
src/compat/*.h \
src/consensus/*.cpp \
src/consensus/*.h \
src/crypto/*.cpp \
src/crypto/*.h \
src/crypto/ctaes/*.h \
src/policy/*.cpp \
src/policy/*.h \
src/primitives/*.cpp \
src/primitives/*.h \
src/qt/*.cpp \
src/qt/*.h \
src/qt/test/*.cpp \
src/qt/test/*.h \
src/rpc/*.cpp \
src/rpc/*.h \
src/script/*.cpp \
src/script/*.h \
src/support/*.cpp \
src/support/*.h \
src/support/allocators/*.h \
src/test/*.cpp \
src/test/*.h \
src/wallet/*.cpp \
src/wallet/*.h \
src/wallet/test/*.cpp \
src/wallet/test/*.h \
src/zmq/*.cpp \
src/zmq/*.h
do
base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
Signed-off-by: Pasta <pasta@dashboost.org>
* scripted-diff: Replace #include "" with #include <> (Dash Specific)
-BEGIN VERIFY SCRIPT-
for f in \
src/bls/*.cpp \
src/bls/*.h \
src/evo/*.cpp \
src/evo/*.h \
src/governance/*.cpp \
src/governance/*.h \
src/llmq/*.cpp \
src/llmq/*.h \
src/masternode/*.cpp \
src/masternode/*.h \
src/privatesend/*.cpp \
src/privatesend/*.h
do
base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
Signed-off-by: Pasta <pasta@dashboost.org>
* build: Remove -I for everything but project root
Remove -I from build system for everything but the project root,
and built-in dependencies.
Signed-off-by: Pasta <pasta@dashboost.org>
# Conflicts:
# src/Makefile.test.include
* qt: refactor: Use absolute include paths in .ui files
* qt: refactor: Changes to make include paths absolute
This makes all include paths in the GUI absolute.
Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.
Signed-off-by: Pasta <pasta@dashboost.org>
# Conflicts:
# src/qt/dash.cpp
# src/qt/optionsmodel.cpp
# src/qt/test/rpcnestedtests.cpp
* test: refactor: Use absolute include paths for test data files
* Recommend #include<> syntax in developer notes
* refactor: Include obj/build.h instead of build.h
* END BACKPORT #11651 Remove trailing whitespace causing travis failure
* fix backport 11651
Signed-off-by: Pasta <pasta@dashboost.org>
* More of 11651
* fix blockchain.cpp
Signed-off-by: pasta <pasta@dashboost.org>
* Add missing "qt/" in includes
* Add missing "test/" in includes
* Fix trailing whitespaces
Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Implement CompactFull() in CDBWrapper
This allows to compact the whole DB in one go.
* Implement more compact version of CDeterministicMNListDiff
This introduces CDeterministicMNStateDiff which requires to only store
fields on-disk which actually changed.
* Avoid writing mnUniquePropertyMap to disk when storing snapshots
This map can be rebuilt by simply using AddMN for each deserialized MN.
* Implement Serialize/Unserialize in CScript
This allows us to directly use READWRITE() on scripts and removes the need
for the ugly cast to CScriptBase. This commit also changes all Dash specific
uses of CScript to not use the cast.
* Keep track of registeration counts and introduce internalID for masternodes
The "internalId" is simply the number of MNs registered so far when the
new MN is added. It is deterministic and stays the same forever.
* Use internalId as keys in MN list diffs
This reduces the used size on-disk.
* Two simple speedups in MN list diff handling
1. Avoid full compare if dmn or state pointers match in BuildDiff
2. Use std::move when adding diff to listDiff in GetListForBlock
* Implement upgrade code for old CDeterministicMNListDiff format to new format
* Track tipIndex instead of tipHeight/tipBlockHash
* Store and pass around CBlockIndex* instead of block hash and height
This allows us to switch CDeterministicMNManager::GetListForBlock to work
with CBlockIndex.
* Refactor CDeterministicMNManager::GetListForBlock to require CBlockIndex*
Instead of requiring a block hash. This allows us to remove blockHash and
prevBlockHash from CDeterministicMNListDiff without the use of cs_main
locks in GetListForBlock.
* Remove prevBlockHash, blockHash and nHeight from CDeterministicMNListDiff
* Remove access to determinisitcMNManager in CMasternodeMetaMan::ToString()
The deterministic MN manager is not fully initialized yet at the time this
is called, which results in an empty list being returned everytime.
* Better logic to determine if an upgrade is needed
Reuse the "best block" logic to figure out if an upgrade is needed. Also
use it to ensure that older nodes are unable to start after the upgrade
was performed.
* Return null block hash if it was requested with getmnlistdiff
* bump CGovernanceManager::SERIALIZATION_VERSION_STRING
* Check SERIALIZATION_VERSION_STRING before deserializing anything else
* Invoke Clear() before deserializing just to be sure
0f3471f net: make CNode's id private (Cory Fields)
9ff0a51 scripted-diff: net: Use accessor rather than node's id directly (Cory Fields)
e50c33e devtools: add script to verify scriptable changes (Cory Fields)
skipped travis changes
Tree-SHA512: a0ff50f4e1d38a2b63109b4996546c91b3e02e00d92c0bf04f48792948f78b1f6d9227a15d25c823fd4723a0277fc6a32c2c1287c7abbb7e50fd82ffb0f8d994
pnode->id to pnode->GetId()
Signed-off-by: Pasta <pasta@dashboost.org>
* Contains dashification. disables `-debug dash`
Merge #9424: Change LogAcceptCategory to use uint32_t rather than sets of strings.
6b3bb3d Change LogAcceptCategory to use uint32_t rather than sets of strings. (Gregory Maxwell)
Tree-SHA512: ebb5bcf9a7d00a32dd1390b727ff4d29330a038423611da01268d8e1d2c0229e52a1098e751d4e6db73ef4ae862e1e96d38249883fcaf12b68f55ebb01035b34
Signed-off-by: Pasta <Pasta@dash.org>
31 -> 32
Signed-off-by: Pasta <Pasta@dash.org>
* Merge #10123: Allow debug logs to be excluded from specified component
3bde556 Add -debugexclude option to switch off logging for specified components (John Newbery)
Tree-SHA512: 30202e3f2085fc2fc5dd4bedb92988f4cb162c612a42cf8f6395a7da326f34975ddc347f82bc4ddca6c84c438dc0cc6e87869f90c7ff88105dbeaa52a947fa43
* bump to uint64_t due to added Dash codes
Signed-off-by: Pasta <Pasta@dash.org>
* bump to uint64_t due to added Dash codes cont.
Signed-off-by: Pasta <Pasta@dash.org>
* string -> BCLog format
Signed-off-by: Pasta <Pasta@dash.org>
* uint32_t -> uint64_t
Signed-off-by: Pasta <Pasta@dash.org>
* Fix CBatchedLogger
* Fix most fDebug-s
* Fix `debug` rpc
* Fix BENCH and RAND conflicts
* Add ALERT and use it
* Update LogPrint-s in dash-specific code
* Tweak few log categories
Specifically:
- use PRIVATESEND in `CPrivateSendClientManager::GetRandomNotUsedMasternode()`
- use ZMQ in `CZMQPublishRawGovernanceVoteNotifier::NotifyGovernanceVote()` and `CZMQPublishRawGovernanceObjectNotifier::NotifyGovernanceObject()`
* Drop no longer used MASTERNODE category
* Merge #10153: logging: Fix off-by-one for shrinkdebugfile default
faab624 logging: Fix off-by-one for shrinkdebugfile (MarcoFalke)
Tree-SHA512: d6153e06067906172ff0611af9e585a3ecf0a7d56925b6ad7c12e75aa802441047059b9b6f6c78e79916c3f2abc8f1998bfd2d5b84201ec6421f727c08da3c21
* Shift dash-specific log categories to start from `1ul << 32` to avoid potential future conflicts with bitcoin ones
* Fix `dash` category
* remove debugCategories
Signed-off-by: Pasta <Pasta@dash.org>
* Prepend "std::" to find call
* Check for BCLog::PRIVATESEND instead of logCategories != BCLog::NONE
* Use BCLog::MNPAYMENTS category instead of checking for logCategories != BCLog::NONE
* Move "End Dash" comment below "ALERT"
When adding new entries here, we'll otherwise get confused with ordering
and might end up forgetting that adding something Dash specific must
continue with the bit after 43.
* Trivial: vout->txout
* Re-use SetHexStr in few more places
* Tweak log output
* fix v13 release notes links
* Drop no longer used stuff
* Few more trivial fixes
* Adjust few rpc help strings
* Apply review suggestions
* Harden DIP3 activation height
Also drop all related but no longer used parts.
* Pass current block index to GetCommitmentsFromBlock
* Allow to change dip3 activation height for tests
And fix them.
This fixes a bug in ScanQuorums which made it return quorums which were not
mined at the time of pindexStart. This was due to quorumHashes being based
on older blocks (the phase=0 block) which are ancestors of pindexStart even
if the commitment was actually mined in a later block.
GetMinedAndActiveCommitmentsUntilBlock is also going to be used for quorum
commitment merkle roots in CCbTx.
This also removes GetFirstMinedQuorumHash as it's not needed anymore.
* Add support for log category to CBatchedLogger
* Use "llmq" logging category in LLMQ code
* Use "chainlocks" logging category in ChainLocks code
* Log errors without logging category
* Indicate success when signing was unnecessary
* Fix typo in name of LLMQ_400_60
* Move RemoveAskFor call for CLSIGs into ProcessNewChainLock
In case we got INV items for the same CLSIG that we recreated through
HandleNewRecoveredSig, (re-)requesting of the CLSIG from other peers
becomes unnecessary.
* Move Cleanup() call in CChainLocksHandler::UpdatedBlockTip up
We bail out early in a few situations from this method, so that Cleanup()
might not be called while its at the bottom.
* Bail out from CChainLocksHandler::UpdatedBlockTip if we already got the CLSIG
* Call RemoveAskFor when QFCOMMITMENT was received
Otherwise we might end up re-requesting it for a very long time when the
commitment INV was received shortly before it got mined.
* Call RemoveSigSharesForSession when a recovered sig is received
Otherwise we end up with session data in node states lingering around until
a fake "timeout" occurs (can be seen in the logs).
* Better handling of false-positive conflicts in CSigningManager
The old code was emitting a lot of messages in logs as it treated sigs
for exactly the same session as a conflict. This commit fixes this by
looking at the signHash before logging.
Also handle a corner-case where a recovered sig might be deleted between
the HasRecoveredSigForId and GetRecoveredSigById call.
* Don't run into session timeout when sig shares come in slow
Instead of just tracking when the first share was received, we now also
track when the last (non-duplicate) share was received. Sessios will now
timeout 5 minutes after the first share arrives, or 1 minute after the last
one arrived.
* Store quorumHash of first mined commitment in evoDb
This allows to skip scanning for quorums below this block.
* Speed up CQuorumManager::ScanQuorums
This does 2 things:
1. Only call HasQuorum for blocks that are potentially a quorumBlockHash
These are only blocks which are at index 0 of each DKG interval
2. Stop scanning for quorums when we get below the first block that
contained a commitment. If no commitment was ever mined, we bail out
immediately.
* Return result instead of {}
* Remove HasQuorum() call as GetQuorum already does this
* Remove unnecessary "if (!qc.IsNull()))"
It's already checked at the top of the loop
* When necessary, remove DB_FIRST_MINED_COMMITMENT from evoDb in UndoBlock
* Allow sub-batch verification in CBLSInsecureBatchVerifier
* Implement batch verification of CDKGDebugStatus messages
* Use uint8_t for statusBitset in CDKGDebugMemberStatus and CDKGDebugSessionStatus
No need to waste one byte per member and per LLMQ type.
* Reserve 4k of buffer for CSerializedNetMsg buffer
Profiling has shown that a lot of time is spent in resizing the data
vector when large messages are involved.
* Remove nHeight from CDKGDebugStatus
This field changes every block and causes all masternodes to propagate
its status for every block, even if nothing DKG related has changed.
* Leave out session statuses when we're not a member of that session
Otherwise MNs which are not members of DKG sessions will spam the network
* Remove receivedFinalCommitment from CDKGDebugSessionStatus
This is not bound to a session and thus is prone to spam the network when
final commitments are propagated in the finalization phase.
* Add "minableCommitments" to "quorum dkgstatus"
* Hold cs_main while calling GetMinableCommitment
* Abort processing of pending debug messages when spork18 gets disabled
* Don't ask for debug messages when we've already seen them
"statuses" only contains the current messages but none of the old messages,
so nodes kept re-requesting old messages.
* Switch GetQuorumBlockHash from CBlockIndex* to nHeight
* `pindexPrev -> pindex` for ProcessCommitment
* Switch IsCommitmentRequired from CBlockIndex* to block height
* Switch GetMinableCommitment/Tx from CBlockIndex* to block height
* Add `AssertLockHeld(cs_main);`
Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>
* Allow to skip sig verification for CFinalCommitment::Verify
* Add CFinalCommitmentTxPayload and CheckLLMQCommitment and use it
As described in https://github.com/dashpay/dips/pull/31 (see discussion).
* Properly ban nodes for invalid commitments
* Add LLMQ parameters to consensus params
* Add DIP6 quorum commitment special TX
* Implement CQuorumBlockProcessor which validates and handles commitments
* Add quorum commitments to new blocks
* Propagate QFCOMMITMENT messages to all nodes
* Allow special transactions in blocks which have no inputs/outputs
But only for TRANSACTION_QUORUM_COMMITMENT for now.
* Add quorum commitments to self-crafted blocks in DIP3 tests
* Add simple fork logic for current testnet
This should avoid a fork on the current testnet. It only applies to the
current chain which activated DIP3 at height 264000 and block
00000048e6e71d4bd90e7c456dcb94683ae832fcad13e1760d8283f7e89f332f.
When we revert the chain to retest the DIP3 deployment, this fork logic
can be removed again.
* Use quorumVvecHash instead of quorumHash to make null commitments unique
Implementation of https://github.com/dashpay/dips/pull/31
* Re-add quorum commitments after pruning mempool selected blocks
* Refactor CQuorumBlockProcessor::ProcessBlock to have less nested if/else statements
Also add BEGIN/END markers for temporary code.
* Add comments/documentation to LLMQParams
* Move code which determines if a commitment is required into IsCommitmentRequired
This should make the code easier to read and also removes some duplication.
The also changes the error types that are possible from 3 to 2 now. Instead
of having "bad-qc-already-mined" and "bad-qc-not-mining-phase", there is
only "bad-qc-not-allowed" now.
* Use new parameter from consensus parames for the temporary fork
