* More/better logging for InstantSend
* Implement CRecoveredSigsDb::TruncateRecoveredSig
* Truncate recovered sigs for ISLOCKs instead of completely removing them
This makes AlreadyHave() return true even when the recovered sig is deleted
locally. This avoids re-requesting and re-processing of old recovered sigs.
* Also truncate recovered sigs for freshly received ISLOCKs
* Fix comment
* Remove LogPrints which have been commented out.
We have version control systems for a reason, if we want code to not run it should be removed. I personally see no value in keeping these around. I presume at one point they were spamming debug.log so we commented them out, but we really should have just removed them.
I believe all of this is dash specific code but any conflicts this does create are so minor they are not of concern imo.
Signed-off-by: Pasta <pasta@dashboost.org>
* remove a couple of extra comments
Signed-off-by: Pasta <pasta@dashboost.org>
* remove commented out code
Signed-off-by: Pasta <pasta@dashboost.org>
* check if we can lock before checking if it is conflicting, it is very rare a tx will actually be conflicting, whereas it is very common that a MN will not be able to sign for a specific Tx
Signed-off-by: Pasta <pasta@dashboost.org>
* remove unused variable
Signed-off-by: Pasta <pasta@dashboost.org>
* move sync check higher up
Signed-off-by: Pasta <pasta@dashboost.org>
* remove unused/unnecessary variable
Signed-off-by: Pasta <pasta@dashboost.org>
* remove unused variable
Signed-off-by: Pasta <pasta@dashboost.org>
* Revert "move sync check higher up"
This reverts commit 77fbe054df78b8bb12f686a627ef618ecff7e7a1.
* Introduce getbestchainlock rpc and fix llmq-is-cl-conflicts.py
* Add `known_block` field and move `getbestchainlock` to `blockchain` rpc category
* Add CChainLockSig::IsNull() and throw an exception in getbestchainlock if there is no known chainlock yet
* drop blockHash initializer
* Refactor `HandleFullyConfirmedBlock()`
* Pass `pindexMined` into `AddNonLockedTx()`
* Unify time import in Dash specific tests
* Refactor UpdateSpork and SetPrivKey
* Split ProcessPendingInstantSendLocks into two methods
* Split SelectQuorumForSigning into SelectQuorumForSigning and GetActiveQuorumSet
* Implement retrying of IS lock verification when the LLMQ active set rotates
* Remove unused overload of RemoveInstantSendLock
* Move deletion of recovered sigs into own method
* Remove recovered sigs for fully confirmed IS locks
* Also remove rs_t entries when removing recovered sigs from the outside
CleanupOldRecoveredSigs already does this as the last step, but when
RemoveRecoveredSig is called from the outside (e.g. from InstantSend),
these keys are not removed. This PR fixes this by storing the write time
into rs_r and later uses it to remove the rs_t entry.
Old entries will be incompatible with this (1 byte written in the past,
4 bytes written now). This checked by comparing the data size with
sizeof(uint32_t).
* Add TODO
* Remove ppszTypeName from protocol.cpp and reimplement GetCommand
This removes the need to carefully maintain ppszTypeName, which required
correct order and also did not allow to permanently remove old message
types.
To get the command name for an INV type, GetCommandInternal uses a switch
which needs to be maintained from now on.
The way this is implemented also resembles the way it is implemented in
Bitcoin today, but it's not identical. The original PR that introduced the
switch case in Bitcoin was part of the Segwit changes and thus never got
backported. I decided to implement it in a slightly different way that
avoids throwing exceptions when an unknown INV type is encountered.
IsKnownType will now also leverage GetCommandInternal() to figure out if
the INV type is known locally. This has the side effect of old/legacy
message types to return false from now on. We will depend on this side
effect in later commits when we remove legacy InstantSend code.
* Stop handling/relaying legacy IX messages
When we receive an IX message, we simply treat it as a regular TX and relay
it as such.
We'll however still request IX messages when they are announced to us. We
can't simply revert to requesting TX messages in this case as it might
result in the other peer not answering due to the TX not being in mapRelay
yet. We should at some point in the future completely drop handling of IX
messages instead.
* Remove IsNewInstantSendEnabled() and only use IsInstantSendEnabled()
* Remove legacy InstantSend from GUI
* Remove InstantSend from Bitcoin/Dash URIs
* Remove legacy InstantSend from RPC commands
* Remove legacy InstantSend from wallet
* Remove legacy instantsend.h include
* Remove legacy InstantSend from validation code
* Completely remove remaining legacy InstantSend code
* Remove now unused spork
* Fix InstantSend related test failures
* Remove now obsolete auto IS tests
* Make spork2 and spork3 disabled by default
This should have no influence on mainnet as these sporks are actually set
there. This will however affect regtest, which shouldn't have LLMQ based
InstantSend enabled by default.
* Remove instantsend tests from dip3-deterministicmns.py
These were only testing legacy InstantSend
* Fix .QCheckBox#checkUsePrivateSend styling a bit
* s/TXLEGACYLOCKREQUEST/LEGACYTXLOCKREQUEST/
* Revert "verified via InstantSend" back to "verified via LLMQ based InstantSend"
* Use cmd == nullptr instead of !cmd
* Remove last parameter from AvailableCoins call
This was for fUseInstantSend which is not present anymore since rebase
* Implement CompactFull() in CDBWrapper
This allows to compact the whole DB in one go.
* Implement more compact version of CDeterministicMNListDiff
This introduces CDeterministicMNStateDiff which requires to only store
fields on-disk which actually changed.
* Avoid writing mnUniquePropertyMap to disk when storing snapshots
This map can be rebuilt by simply using AddMN for each deserialized MN.
* Implement Serialize/Unserialize in CScript
This allows us to directly use READWRITE() on scripts and removes the need
for the ugly cast to CScriptBase. This commit also changes all Dash specific
uses of CScript to not use the cast.
* Keep track of registeration counts and introduce internalID for masternodes
The "internalId" is simply the number of MNs registered so far when the
new MN is added. It is deterministic and stays the same forever.
* Use internalId as keys in MN list diffs
This reduces the used size on-disk.
* Two simple speedups in MN list diff handling
1. Avoid full compare if dmn or state pointers match in BuildDiff
2. Use std::move when adding diff to listDiff in GetListForBlock
* Implement upgrade code for old CDeterministicMNListDiff format to new format
* Track tipIndex instead of tipHeight/tipBlockHash
* Store and pass around CBlockIndex* instead of block hash and height
This allows us to switch CDeterministicMNManager::GetListForBlock to work
with CBlockIndex.
* Refactor CDeterministicMNManager::GetListForBlock to require CBlockIndex*
Instead of requiring a block hash. This allows us to remove blockHash and
prevBlockHash from CDeterministicMNListDiff without the use of cs_main
locks in GetListForBlock.
* Remove prevBlockHash, blockHash and nHeight from CDeterministicMNListDiff
* Remove access to determinisitcMNManager in CMasternodeMetaMan::ToString()
The deterministic MN manager is not fully initialized yet at the time this
is called, which results in an empty list being returned everytime.
* Better logic to determine if an upgrade is needed
Reuse the "best block" logic to figure out if an upgrade is needed. Also
use it to ensure that older nodes are unable to start after the upgrade
was performed.
* Return null block hash if it was requested with getmnlistdiff
* bump CGovernanceManager::SERIALIZATION_VERSION_STRING
* Check SERIALIZATION_VERSION_STRING before deserializing anything else
* Invoke Clear() before deserializing just to be sure
No need to load 25 quorums when we're scanning for the larger quorums which
only have 4 active quorums. This avoids loading thousands of masternode
lists unnecessarily.
0f3471f net: make CNode's id private (Cory Fields)
9ff0a51 scripted-diff: net: Use accessor rather than node's id directly (Cory Fields)
e50c33e devtools: add script to verify scriptable changes (Cory Fields)
skipped travis changes
Tree-SHA512: a0ff50f4e1d38a2b63109b4996546c91b3e02e00d92c0bf04f48792948f78b1f6d9227a15d25c823fd4723a0277fc6a32c2c1287c7abbb7e50fd82ffb0f8d994
pnode->id to pnode->GetId()
Signed-off-by: Pasta <pasta@dashboost.org>
* Generalize CBLSLazyWrapper so that it can be used of pubkeys and secret keys
* Implement == and != operators for CBLSLazyWrapper
* Implement cached hash for CBLSLazyWrapper
* Use CBLSLazyPublicKey for CDeterministicMNState::pubKeyOperator
* Speed up GetProjectedMNPayees by sorting the MN list by last paid
Instead of updating a temporary list for each projected height and calling
GetMNPayee() on it.
* Cache intermediate lists in GetListForBlock
This avoids re-loading and applying diffs again and again.
* Only update masternode list UI max once every 3 seconds
This avoids updating the UI on every block, which turned out to be very
expensive.
* Fix compilation
* Drop time restrictions for mn list update in ClientModel
They are fully handled by MasternodeList now.
* Merge #8824: Refactor TxToJSON() and ScriptPubKeyToJSON()
0ff9320 refactor TxToJSON() and ScriptPubKeyToJSON() (jonnynewbs)
Tree-SHA512: caf7d590829e221522edd5b1ab8ce67b53a2c6986d3bbe8477eab420b1007bf60f885ed0a25ba9587e468c00768360ddc31db37847e862858573eaed5ed8b0d6
* remove witness and vsize
Signed-off-by: Pasta <Pasta@dash.org>
* Add valueSat
To preserve rpc output format
* Move extrapayload and special tx json output to `TxToUniv`
* Add spent index info
ProcessNewTransaction is now only called with pindex != nullptr when a
block is connected but never when it is disconnected, so there is no need
to check for posInBlock anymore (we don't even have it now).
This commit moves all logic of SyncTransaction into ProcessNewTransaction
and then calls it from TransactionAddedToMempool and BlockConnected.
This won't compile/work at first, but the next commits will fix this.
* Contains dashification. disables `-debug dash`
Merge #9424: Change LogAcceptCategory to use uint32_t rather than sets of strings.
6b3bb3d Change LogAcceptCategory to use uint32_t rather than sets of strings. (Gregory Maxwell)
Tree-SHA512: ebb5bcf9a7d00a32dd1390b727ff4d29330a038423611da01268d8e1d2c0229e52a1098e751d4e6db73ef4ae862e1e96d38249883fcaf12b68f55ebb01035b34
Signed-off-by: Pasta <Pasta@dash.org>
31 -> 32
Signed-off-by: Pasta <Pasta@dash.org>
* Merge #10123: Allow debug logs to be excluded from specified component
3bde556 Add -debugexclude option to switch off logging for specified components (John Newbery)
Tree-SHA512: 30202e3f2085fc2fc5dd4bedb92988f4cb162c612a42cf8f6395a7da326f34975ddc347f82bc4ddca6c84c438dc0cc6e87869f90c7ff88105dbeaa52a947fa43
* bump to uint64_t due to added Dash codes
Signed-off-by: Pasta <Pasta@dash.org>
* bump to uint64_t due to added Dash codes cont.
Signed-off-by: Pasta <Pasta@dash.org>
* string -> BCLog format
Signed-off-by: Pasta <Pasta@dash.org>
* uint32_t -> uint64_t
Signed-off-by: Pasta <Pasta@dash.org>
* Fix CBatchedLogger
* Fix most fDebug-s
* Fix `debug` rpc
* Fix BENCH and RAND conflicts
* Add ALERT and use it
* Update LogPrint-s in dash-specific code
* Tweak few log categories
Specifically:
- use PRIVATESEND in `CPrivateSendClientManager::GetRandomNotUsedMasternode()`
- use ZMQ in `CZMQPublishRawGovernanceVoteNotifier::NotifyGovernanceVote()` and `CZMQPublishRawGovernanceObjectNotifier::NotifyGovernanceObject()`
* Drop no longer used MASTERNODE category
* Merge #10153: logging: Fix off-by-one for shrinkdebugfile default
faab624 logging: Fix off-by-one for shrinkdebugfile (MarcoFalke)
Tree-SHA512: d6153e06067906172ff0611af9e585a3ecf0a7d56925b6ad7c12e75aa802441047059b9b6f6c78e79916c3f2abc8f1998bfd2d5b84201ec6421f727c08da3c21
* Shift dash-specific log categories to start from `1ul << 32` to avoid potential future conflicts with bitcoin ones
* Fix `dash` category
* remove debugCategories
Signed-off-by: Pasta <Pasta@dash.org>
* Prepend "std::" to find call
* Check for BCLog::PRIVATESEND instead of logCategories != BCLog::NONE
* Use BCLog::MNPAYMENTS category instead of checking for logCategories != BCLog::NONE
* Move "End Dash" comment below "ALERT"
When adding new entries here, we'll otherwise get confused with ordering
and might end up forgetting that adding something Dash specific must
continue with the bit after 43.
* Implement GetInstantSendLockCount in CInstantSendManager
* Add islockCountChanged signal to client model
* Show number of InstantSend locks in debug console
* Merge #9963: util: Properly handle errors during log message formatting
b651270 util: Throw tinyformat::format_error on formatting error (Wladimir J. van der Laan)
3b092bd util: Properly handle errors during log message formatting (Wladimir J. van der Laan)
Tree-SHA512: 85e3b7afec2255fc88034187f1abd6060e9421de17ed4e3d918416f393429a99cc2c974b362099aaaff6970549df47664bea4c857c4e46acc0789663201dc541
* "cast" debugMsg to a c string
Signed-off-by: Pasta <Pasta@dash.org>
"cast" debugMsg to a c string pt 2
Signed-off-by: Pasta <Pasta@dash.org>
* rename instantx.* file to instantsend.*
* Reorganize Dash Specific code into folders
add to privatesend folder the header files
Signed-off-by: Pasta <Pasta@dash.org>
add "masternode" to imports
Signed-off-by: Pasta <Pasta@dash.org>
add "masternode" to imports
Signed-off-by: Pasta <Pasta@dash.org>
add "masternode" to imports pt 2
Signed-off-by: Pasta <Pasta@dash.org>
Applies to CInstantSendManager and CChainLocksHandler. This fixes extremely
high RAM usage on reindex and resync, which happens to many/all transactions
being kept track of non-locked TXs (nonLockedTxs) and TXs per
block (blockTxs).
* Store rs_t key time in big endian
Also implement ConvertInvalidTimeKeys to convert old entries. We can remove
this later when we know that most MNs have run this code on testnet.
The way we stored the time field in the past lead to CleanupOldRecoveredSigs
iterating the keys in a strange order, causing no deletion at all and the
LLMQ DB filling up.
* Write batch in CleanupOldRecoveredSigs when it gets too large
This avoids RAM filling up and OOM getting triggered.
* Keep track of when a vote was written to the DB and clean up after week
Instead of only deleting when the corresponding recovered sig is deleted.
It sometimes happens that a masternode votes on something but a recovered
sig is never created, which leaves us with a vote that will never be
deleted.
* Apply suggestions from code review
Co-Authored-By: PastaPastaPasta <pasta@dashboost.org>
* Fall back to ReadBlockFromDisk when blockTxs is not filled yet
This handles the case where a MN is freshly started and SyncTransaction
was not been called for older transactions/blocks.
* Also update txFirstSeenTime
* Properly handle txids == nullptr
* Print inputs on which we voted
Also print the corresponding requestId. This makes debugging InstantSend
issues easier.
* Print quorum hash when signing a share
* Remove unused nodesByAddress map in CSigSharesManager::SendMessages
Not related to this PR, but a simple cleanup that should have no
side effects.
* Allow modifying simulate DKG error rates via RPC
* Don't lie to yourself :)
* Add some missing new-lines in LogPrintf calls
* More fine grained control over which messages to expect in mine_quorum
* Implement llmq-dkgerrors.py integration tests
These test DKG errors and malicious behavior.
* Move code to write archived ISLOCKs into its own method
We'll need this from another method as well later.
* Return ISLOCK instead of conflicting txid in GetConflictingTx/GetConflictingLock
* Implement GetInstantSendLocksByParent and RemoveChainedInstantSendLocks
These allow to easily delete multiple chains (actually trees) of ISLOCKs
in one go.
* Implement RemoveConflictedTx and call it from RemoveMempoolConflictsForLock
Also add "retryChildren" parameter to RemoveNonLockedTx so that we can
skip retrying of non-locked children TXs.
* Properly handle/remove conflicted TXs (between mempool and new blocks)
* Track non-locked TXs by inputs
* Implement and call ResolveBlockConflicts
* Also call ResolveBlockConflicts from ConnectBlock
But only when a block is known to have a conflict and at the same time is
ChainLocked, which causes the ISLOCK to be pruned.
* Split out RemoveChainLockConflictingLock from ResolveBlockConflicts
* Implement "quorum getrecsig" RPC
* Include decoded TX data in result of create_raw_tx
* Implement support for CLSIG in mininode.py
* Fix condition for update of nonLockedTxs.pindexMined
* Only add entries to nonLockedTxsByInputs when AddNonLockedTx is called for the first time
* Implement support for ISLOCK in mininode.py
* Implement tests for ChainLock vs InstantSend lock conflict resolution
* Handle review comment
Bail out (continue) early
* Also test conflicts in mempool instead of only in blocks
* Ask for locked TXs after removing conflicting TXs
When we removed a conflicting TX from the mempool, the correct/locked TX
is not available locally as the first-seen rule would have filtered before.
We need to re-request this TX if any other node announced it before.
* Apply suggestions from code review
Co-Authored-By: codablock <ablock84@gmail.com>
* Remove code for QDEBUGSTATUS propagation
This turned out to be too expensive and could easily take the network
down by bringing all nodes to 100% CPU usage. Better to fully remove this
functionality.
* Apply suggestions from code review
Co-Authored-By: codablock <ablock84@gmail.com>
* Update src/rpc/rpcquorums.cpp
Co-Authored-By: codablock <ablock84@gmail.com>
* Trivial: vout->txout
* Re-use SetHexStr in few more places
* Tweak log output
* fix v13 release notes links
* Drop no longer used stuff
* Few more trivial fixes
* Adjust few rpc help strings
* Apply review suggestions
* Harden DIP3 activation height
Also drop all related but no longer used parts.
* Pass current block index to GetCommitmentsFromBlock
* Allow to change dip3 activation height for tests
And fix them.
This allows AlreadyHave to check if an announced (via INV) islock was
already known in the past. This avoids requesting islocks which got
obsolete due to ChainLocks.
* Track which TXs are not locked yet and use this info in ProcessPendingRetryLockTxs
Instead of relying on ReadBlockFromDisk. This should be less disk+CPU
intensive but require more RAM.
It also fixes a bug in ProcessPendingRetryLockTxs which caused ChainLocked
parents to not be considered for retrying of its children.
* Handle review commments
* Make CBLSLazySignature thread safe
* Perform malleability check in CBLSLazySignature
* Use CBLSLazySignature in CRecoveredSig and CInstantSendLock
* Only sporadically verify self-recovered signatures
* test
* Automatically wake up select() when optimistic send was not used
But only when we know that we are actually inside select() and that it
currenlty is unlikely for it to have selected the node's socket for
sending. We accept race conditions here as the select() timeout
will ensure that we always send the data.
* Don't manually call WakeSelect() in CSigSharesManager::SendMessages
Not needed anymore
* Disable optimistic send in PushMessage by default
* Let ProcessPendingInstantSendLocks return true when it did some work
* Introduce own worker thread for CInstantSendManager
Instead of using the scheduler.
* Remove scheduler from CInstantSendManager
* Add missing reset() call for workInterrupt
This fixes a bug in ScanQuorums which made it return quorums which were not
mined at the time of pindexStart. This was due to quorumHashes being based
on older blocks (the phase=0 block) which are ancestors of pindexStart even
if the commitment was actually mined in a later block.
GetMinedAndActiveCommitmentsUntilBlock is also going to be used for quorum
commitment merkle roots in CCbTx.
This also removes GetFirstMinedQuorumHash as it's not needed anymore.
* Pass CNode* to IsMasternodeQuorumNode and let it also check verifiedProRegTxHash
This makes IsMasternodeQuorumNode return true on incoming peer connections
as well.
* Let GetMasternodeQuorumNodes also take verifiedProRegTxHash into account
This makes it return NodeIds for incoming peer connections as well.
* Remove AddParticipatingNode and the need for it
This was needed in the past when we were unable to identify incoming
connections from other quorum members. Now that we have MNAUTH, we can
easily identify all connected members.
* Don't track interestedIn quorums in CSigSharesNodeState anymore
Same as with the previous commit, we're now able to easily identify which
nodes to announce sig shares to.
* Remove unused CConnman::GetMasternodeQuorumAddresses
* Sort evo/* source files in Makefile.am
* Keep track of proRegTxHash in CConnman::masternodeQuorumNodes map
We will later need the proRegTxHash
* Fix serialization of std::tuple with const rvalue elements
Having serialization and deserialization in the same specialized template
results in compilation failures due to the "if(for_read)" branch.
* Implement MNAUTH message
This allows masternodes to authenticate themself.
* Protect fresh incoming connections for a second from eviction
Give fresh connections some time to do the VERSION/VERACK handshake and
an optional MNAUTH when it's a masternode. When an MNAUTH happened, the
incoming connection is then forever protected against eviction.
If a timeout of 1 second occurs or the first message after VERACK is not
MNAUTH, the node is not protected anymore and becomes eligable for
eviction.
* Avoid connecting to masternodes if an incoming connection is from the same one
Now that incoming connections from MNs authenticate them self, we can avoid
connecting to the same MNs through intra-quorum connections.
* Apply review suggestions
* Fix warning about size_t to int conversion
* Fix loop in CLLMQUtils::GetQuorumConnections to add at least 2 connections
When reaching very small quorum sizes, the current algorithm results in
only a single connection to be added. This would be fine usually, but is an
issue when this connection fails. We should always have at least one backup
connection.
This fixes simple PoSe test failures where the quorum size gets down to 4
with one of the 4 members being down. If other nodes are unlucky to connect
to this node, they fail as well even though 3 members in a quorum should
work fine.
* Update src/llmq/quorums_utils.cpp
Co-Authored-By: codablock <ablock84@gmail.com>
* Introduce "qsendrecsigs" to indicate that plain recovered sigs should be sent
Full nodes, including masternodes, will send this message automatically.
Other node implementations (e.g. SPV) are usually not interested and would
not send this message.
* Use std::atomic<bool> instead of std::atomic_bool
Not related to this PR, but a small enough change to include it here as
well.
* Add support for log category to CBatchedLogger
* Use "llmq" logging category in LLMQ code
* Use "chainlocks" logging category in ChainLocks code
* Log errors without logging category
* Don't rely on UTXO set in CheckCanLock
The UTXO set only works for TXs in the mempool and won't work when we try
to retroactively lock unlocked TXs from blocks.
This is safe as ProcessTx is only called when a TX was accepted into the
mempool or connected in a block, which means that all input checks were
good.
* Rename RetryLockMempoolTxs to RetryLockTxs and let it retry connected TXs
* Instead of manually calling ProcessTx, let SyncTransaction handle all cases
SyncTransaction is called from AcceptToMemoryPool and when transactions got
connected in a block. So this is the time we want to run TXs through
ProcessTx. This also enables retroactive signing of TXs that were unknown
before a new block appeared.
* Test retroactive signing and safe TXs in LLMQ ChainLocks tests
* Also test for retroactive signing of chained TXs
* Honor lockedParentTx when looking for TXs to retry signing
* Stop scanning for TXs to retry after a depth of 6
* Generate 6 block to avoid retroactive signing overloading Travis
* Avoid retroactive signing
* Don't rely on NewPoWValidBlock and use SyncTransaction to build blockTxs
NewPoWValidBlock is not guaranteed to be called when blocks come in fast.
When a block is accepted in AcceptBlock, NewPoWValidBlock is only called
when the new block is a successor of the currently active tip. This is not
the case when after the first block a second block is accepted immediately
as the first block is not connected yet.
This might be a bug actually in the handling of NewPoWValidBlock, so we
might need to check/fix this later, but currently I prefer to not touch
that part.
Instead, we now use SyncTransaction to gather TXs for blockTxs. This works
because SyncTransaction is called for all transactions in a freshly
connected block in one go. The call also happens before UpdatedBlockTip is
called, so it's fine with the existing logic.
* Use tx.IsCoinBase() instead of checking index 0
Also check for empty vin.
* Remove unused parameters from CInstantSendManager::ProcessTx
* Pass txHash in CheckCanLock by reference instead of pointer
* Dont' allow locking of TXs without inputs
* Remove unused local variable nInstantSendConfirmationsRequired
* Don't subtract 1 from nInstantSendConfirmationsRequired
This was necessary in the old system but is not necessary in the new system.
It also prevented proper retroactive signing of chained TXs in regtest as
it resulted in child TXs to return true immediately for CheckCanLock when
it should actually have waited for the parent TX to become locked first.
* Access chainActive.Height() while cs_main is locked
* Properly read and write lastChainLockBlock
"pindex" is NOT the chainlocked block after the while loop finishes. We
must use the pindex (renamed to pindexChainLock now) given on method entry.
Also, the GetLastChainLockBlock() result was not assigned to,
lastChainLockBlock which resulted in the while loop to run unnecessarily
long.
* Generalize filtering in NewPoWValidBlock and SyncTransaction
We're actually interested in all TXs that have inputs, so no need to
explicitly check for tx types.
* Use tx.IsCoinBase() instead of checking for index 0
* Handle cases where a TX is not received yet in wait_for_instantlock
* Wait on all nodes for the locks
Otherwise we end up with the sender having it locked but other nodes
not yet, failing the test.
* Fix LogPrintf call in CChainLocksHandler::DoInvalidateBlock
* Require only 3 out of 5 signatures for old InstantSend in regtest mode
* Use LLMQs of size 5 with threshold of 3 for regtest
* Fix wrong check for out-of-range bits in CFixedBitSet
* Reduce number of masternodes in masternode/LLMQ tests
* Add missing \n to LogPrintf call
* Use correct indexes for isolated/receiver/sender nodes
The way it was before resulted in nodes 1-3 being unused and 6-8 being used
for these 3 special nodes even though these are masternodes.
* Avoid stopping/starting isolated node in p2p-instantsend.py
It's enough to disable networking for this node.
* Print which DKG type aborted
* Don't directly call EnforceBestChainLock and instead schedule the call
Calling EnforceBestChainLock might result in switching chains, which in
turn might end up calling signals, so we get into a recursive call chain.
Better to call EnforceBestChainLock from the scheduler.
* Regularly call EnforceBestChainLock and reset error flags on locked chain
* Don't invalidate blocks from CChainLocksHandler::TrySignChainTip
As the name of this method implies, it's trying to sign something and not
enforce/invalidate chains. Invalidating blocks is the job of
EnforceBestChainLock.
* Only call ActivateBestChain when tip != best CL tip
* Fix unprotected access of bestChainLockBlockIndex and bail out if its null
* Fix ChainLocks tests after changes in enforcement handling
* Only invoke NotifyChainLock signal from EnforceBestChainLock
This ensures that NotifyChainLock is not prematurely called before the
block is fully connected.
* Use a mutex to ensure that only one thread executes ActivateBestChain
It might happen that 2 threads enter ActivateBestChain at the same time
start processing block by block, while randomly switching between threads
so that sometimes one thread processed the block and then another one
processes it. A mutex protects ActivateBestChain now against this race.
* Rename local copy of bestChainLockBlockIndex to currentBestChainLockBlockIndex
* Don't call ActivateBestChain when best CL is part of the main chain
It's actually not true that these should always be the same. In case a
quorum is built and the total number of masternodes in the network is below
the quorum size, we might still end up having a valid quorum as long as
the total number of masternodes is >= minSize.
* Fix remaining `print`s in tests
* use AssertLockHeld(cs) instead of relying on comments
* actually use `clsig` in `EnforceBestChainLock()`
* fix log output in `EnforceBestChainLock()`
* drop comments
* Fix deadlock in CSigSharesManager::SendMessages
Locking "cs" at this location caused a (potential) deadlock due to changed
order of cs and cs_vNodes locking. This changes the method to not require
the session object anymore which removes the need for locking.
* Pass size of LLMQ instead of llmqType into CSigSharesInv::Init
This allows use of sizes which are not supported in chainparams.
Later commits will introduce checks for "safe TXs" which might abort the
signing on first try, but succeed a few seconds later, so we periodically
retry to sign the tip.
The local node might be the bad one actually as it might not have catched
up with the chain. In that case, LLMQs might be different for the sending
and receiving node.
When ProcessMessageBatchedSigShares returns false, it's interpreted as
if an invalid/malicious message was received, causing a ban. So, we should
return "!ban" instead of just "ban".
* Ignore sig share inv messages when we don't have the quorum vvec
* Update src/llmq/quorums_signing_shares.cpp
Co-Authored-By: codablock <ablock84@gmail.com>
* On timeout, print members proTxHashes from members which did not send a share
* Move inactive quorums check above timeout checks
This allows to reuse things in the next commit
* Avoid locking cs_main through GetQuorum by using a pre-filled map
* Use find() instead of [] to access quorums map
* Return bool in ProcessMessageXXX methods to indicate misbehaviour
* Send/Receive multiple messages as part of one P2P message in CSigSharesManager
Many messages, especially QSIGSHARESINV and QGETSIGSHARES, are very small
by nature (5-14 bytes for a 50 members LLMQ). The message headers are
24 bytes, meaning that we produce a lot of overhead for these small messages.
This sums up quite a bit when thousands of signing sessions are happening
in parallel.
This commit changes all related P2P messages to send a vector of messages
instead of a single message.
* Remove bogus lines
Included these by accident
* Unify handling of BanNode in ProcessMessageXXX methods
* Remove bogus check for fMasternodeMode
* Properly use == instead of misleading >= in SendMessages
* Put "didSend = true" near PushMessage
Stop relying on the information previously found in the CSigSharesInv
and CBatchedSigShares messages and instead use the information found in
the session refereced by the session id.
This also updates a few LogPrintf calls. Previously, CSigSharesInv::ToString
also included the signHash in the returned string, which is not the case
anymore, so we have to add it manually.
We must watch out to not blindly use externally provided keys in unordered
sets/maps, as attackers might find ways to cause unbalanced hash buckets
causing performance degradation.
* Indicate success when signing was unnecessary
* Fix typo in name of LLMQ_400_60
* Move RemoveAskFor call for CLSIGs into ProcessNewChainLock
In case we got INV items for the same CLSIG that we recreated through
HandleNewRecoveredSig, (re-)requesting of the CLSIG from other peers
becomes unnecessary.
* Move Cleanup() call in CChainLocksHandler::UpdatedBlockTip up
We bail out early in a few situations from this method, so that Cleanup()
might not be called while its at the bottom.
* Bail out from CChainLocksHandler::UpdatedBlockTip if we already got the CLSIG
* Call RemoveAskFor when QFCOMMITMENT was received
Otherwise we might end up re-requesting it for a very long time when the
commitment INV was received shortly before it got mined.
* Call RemoveSigSharesForSession when a recovered sig is received
Otherwise we end up with session data in node states lingering around until
a fake "timeout" occurs (can be seen in the logs).
* Better handling of false-positive conflicts in CSigningManager
The old code was emitting a lot of messages in logs as it treated sigs
for exactly the same session as a conflict. This commit fixes this by
looking at the signHash before logging.
Also handle a corner-case where a recovered sig might be deleted between
the HasRecoveredSigForId and GetRecoveredSigById call.
* Don't run into session timeout when sig shares come in slow
Instead of just tracking when the first share was received, we now also
track when the last (non-duplicate) share was received. Sessios will now
timeout 5 minutes after the first share arrives, or 1 minute after the last
one arrived.
Instead of trying to manually figure out params for different quorum/ring sizes, connect to nodes at indexes (i+2^k)%n where k: 0..floor(log2(n-1))-1, n: size of the quorum/ring
* Implement and use SigShareMap instead of ordered map with helper methods
The old implementation was relying on the maps being ordered, which allowed
us to grab all sig shares for the same signHash by doing range queries on
the map. This has the disadvantage of being unnecessarily slow when the
maps get larger. Using an unordered map would be the naive solution, but
then it's not possible to query by range anymore.
The solution now is to have a specialized map "SigShareMap" which is
indexed by "SigShareKey". It's internally just an unordered map, indexed by
the sign hash and another unordered map for the value, indexed by the
quorum member index.
* Only use unordered maps/sets in CSigSharesManager
These are faster when maps/sets get larger.
* Use unorderes sets/maps in CSigningManager
* Don't sleep in WorkThreadMain when CPU intensive work was done
When the current iteration resulted in CPU intensive work, it's likely that
the next iteration will result in work as well. Do not sleep in that case,
as we're otherwise wasting (unused) CPU resources.
* No matter how fast we process sig shares, always force 100ms between sending
* Apply review suggestions
This removes the burden on the message handler thread when many sig batches
arrive. The expensive part of deserialization is now performed in the sig
shares worker thread.
This also removes the need for the specialized deserialization of the sig
shares which tried to avoid the malleability check, as CBLSLazySignature does
not perform malleability checks at all.
* Implement secure verification in bls_batchverifier
* Rename CBLSInsecureBatchVerifier to CBLSBatchVerifier
* Add unit tests for simple BLS verifcation and CBLSBatchVerifier
* Store quorumHash of first mined commitment in evoDb
This allows to skip scanning for quorums below this block.
* Speed up CQuorumManager::ScanQuorums
This does 2 things:
1. Only call HasQuorum for blocks that are potentially a quorumBlockHash
These are only blocks which are at index 0 of each DKG interval
2. Stop scanning for quorums when we get below the first block that
contained a commitment. If no commitment was ever mined, we bail out
immediately.
* Return result instead of {}
* Remove HasQuorum() call as GetQuorum already does this
* Remove unnecessary "if (!qc.IsNull()))"
It's already checked at the top of the loop
* When necessary, remove DB_FIRST_MINED_COMMITMENT from evoDb in UndoBlock
* Check aggPubKey for IsValid() instead of aggSig
aggSig is not reliable here as it might already be initialized by the
previous message.
* Significantly reduce sleep time for each DKG phase
Turns out the DKG is much faster then expected, and waiting multiple
minutes for each phase in a devnet is not much fun.
* Correctly use SIGN_HEIGHT_OFFSET when checking for out of bound height
* Introduce startBlockHeight to make things more explicit
* Allow sub-batch verification in CBLSInsecureBatchVerifier
* Implement batch verification of CDKGDebugStatus messages
* Use uint8_t for statusBitset in CDKGDebugMemberStatus and CDKGDebugSessionStatus
No need to waste one byte per member and per LLMQ type.
* Reserve 4k of buffer for CSerializedNetMsg buffer
Profiling has shown that a lot of time is spent in resizing the data
vector when large messages are involved.
* Remove nHeight from CDKGDebugStatus
This field changes every block and causes all masternodes to propagate
its status for every block, even if nothing DKG related has changed.
* Leave out session statuses when we're not a member of that session
Otherwise MNs which are not members of DKG sessions will spam the network
* Remove receivedFinalCommitment from CDKGDebugSessionStatus
This is not bound to a session and thus is prone to spam the network when
final commitments are propagated in the finalization phase.
* Add "minableCommitments" to "quorum dkgstatus"
* Hold cs_main while calling GetMinableCommitment
* Abort processing of pending debug messages when spork18 gets disabled
* Don't ask for debug messages when we've already seen them
"statuses" only contains the current messages but none of the old messages,
so nodes kept re-requesting old messages.
* Use fast_dip3_enforcement instead of fast_dip3_activation
DashTestFramework was refactored before ChainLocks got merged, causing tests
to fail now.
* Move updating of DKG debug status into WaitForNextPhase
Otherwise callers of the RPCs might believe that the next phase has already
started and start producing more blocks, which would then cancel the
current session if it happens faster than the phase handler thread can
progress to the next phase.
* Fix off-by-1 in phase calculations
* Fix wait_for_quorum_phase, should look for check_received_messages
* Fix wait_for_quorum_phase for complain phase
* Bump default timeout in wait_for_quorum_phase/wait_for_quorum_commitment to 15
* Fix cleanup of old recovered sigs
When iterating the db, we should also include entries that match exactly
the end time.
* Fix key not found error
* Raise AssertionError in case wait_for_quorum_phase/wait_for_quorum_commitment time out
* Fix confusion: `quorumHash` is both a class member and an argument of a function
Rename `height` too while at it
* Make sure height and hash we pass to InitNewQuorum are related
* Don't update expectedQuorumHash, make it const
This also streamlines logic a bit
* Compact phase calculation
* Decouple invCs and cs_vPendingMasternodes
Not an issue atm but we'd better avoid any potential interlocking if possible
* wrap `%` in `()`
Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>
* Switch GetQuorumBlockHash from CBlockIndex* to nHeight
* `pindexPrev -> pindex` for ProcessCommitment
* Switch IsCommitmentRequired from CBlockIndex* to block height
* Switch GetMinableCommitment/Tx from CBlockIndex* to block height
* Add `AssertLockHeld(cs_main);`
Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>
* Implement creation and propagation of dummy contributions
These act as a ping which is broadcast a few blocks before the dummy
commitments are created. They are meant to determine online/offline members.
* Use information about received dummy contributions to determine validMembers
* Fix PoSe tests
* Fix dummy DKG phase progress in PoSe tests and give tests more time
Mine one block at a time until we reach the mining phase.
* Deserialize CFinalCommitmentTxPayload instead of CFinalCommitment in TxToJSON
* Implement ToJson for CFinalCommitmentTxPayload and use it in TxToJSON
Otherwise the nVersion and nHeight members of it are not shown.
* Allow to skip sig verification for CFinalCommitment::Verify
* Add CFinalCommitmentTxPayload and CheckLLMQCommitment and use it
As described in https://github.com/dashpay/dips/pull/31 (see discussion).
* Properly ban nodes for invalid commitments
* Add SPORK_17_QUORUM_DKG_ENABLED spork
* Implement CDummyDKG and CDummyCommitment until we have the real DKG merged
This is only used on testnet/devnet/regtest and will NEVER be used on
mainnet. It is NOT SECURE AT ALL!
See comment in quorums_dummydkg.h for more details.
* Test simple PoSe in DIP3 tests
* Generate 2 instead of 4 blocks per iteration in PoSe tests
4 was based on old chainparams where I used larger phases.
* Only sleep when necessary in PoSe tests
* Fix typo in comment
* Give PoSe tests more time and sync after fast-forward
* Add LLMQ parameters to consensus params
* Add DIP6 quorum commitment special TX
* Implement CQuorumBlockProcessor which validates and handles commitments
* Add quorum commitments to new blocks
* Propagate QFCOMMITMENT messages to all nodes
* Allow special transactions in blocks which have no inputs/outputs
But only for TRANSACTION_QUORUM_COMMITMENT for now.
* Add quorum commitments to self-crafted blocks in DIP3 tests
* Add simple fork logic for current testnet
This should avoid a fork on the current testnet. It only applies to the
current chain which activated DIP3 at height 264000 and block
00000048e6e71d4bd90e7c456dcb94683ae832fcad13e1760d8283f7e89f332f.
When we revert the chain to retest the DIP3 deployment, this fork logic
can be removed again.
* Use quorumVvecHash instead of quorumHash to make null commitments unique
Implementation of https://github.com/dashpay/dips/pull/31
* Re-add quorum commitments after pruning mempool selected blocks
* Refactor CQuorumBlockProcessor::ProcessBlock to have less nested if/else statements
Also add BEGIN/END markers for temporary code.
* Add comments/documentation to LLMQParams
* Move code which determines if a commitment is required into IsCommitmentRequired
This should make the code easier to read and also removes some duplication.
The also changes the error types that are possible from 3 to 2 now. Instead
of having "bad-qc-already-mined" and "bad-qc-not-mining-phase", there is
only "bad-qc-not-allowed" now.
* Use new parameter from consensus parames for the temporary fork
