Commit Graph

177 Commits

Author SHA1 Message Date
PastaPastaPasta
29194b1f5a Backport Bitcoin#9424, Bitcoin#10123 and Bitcoin#10153 (#2918)
* Contains dashification. disables `-debug dash`
Merge #9424: Change LogAcceptCategory to use uint32_t rather than sets of strings.

6b3bb3d Change LogAcceptCategory to use uint32_t rather than sets of strings. (Gregory Maxwell)

Tree-SHA512: ebb5bcf9a7d00a32dd1390b727ff4d29330a038423611da01268d8e1d2c0229e52a1098e751d4e6db73ef4ae862e1e96d38249883fcaf12b68f55ebb01035b34
Signed-off-by: Pasta <Pasta@dash.org>

31 -> 32

Signed-off-by: Pasta <Pasta@dash.org>

* Merge #10123: Allow debug logs to be excluded from specified component

3bde556 Add -debugexclude option to switch off logging for specified components (John Newbery)

Tree-SHA512: 30202e3f2085fc2fc5dd4bedb92988f4cb162c612a42cf8f6395a7da326f34975ddc347f82bc4ddca6c84c438dc0cc6e87869f90c7ff88105dbeaa52a947fa43

* bump to uint64_t due to added Dash codes

Signed-off-by: Pasta <Pasta@dash.org>

* bump to uint64_t due to added Dash codes cont.

Signed-off-by: Pasta <Pasta@dash.org>

* string -> BCLog format

Signed-off-by: Pasta <Pasta@dash.org>

* uint32_t -> uint64_t

Signed-off-by: Pasta <Pasta@dash.org>

* Fix CBatchedLogger

* Fix most fDebug-s

* Fix `debug` rpc

* Fix BENCH and RAND conflicts

* Add ALERT and use it

* Update LogPrint-s in dash-specific code

* Tweak few log categories

Specifically:
- use PRIVATESEND in `CPrivateSendClientManager::GetRandomNotUsedMasternode()`
- use ZMQ in `CZMQPublishRawGovernanceVoteNotifier::NotifyGovernanceVote()` and `CZMQPublishRawGovernanceObjectNotifier::NotifyGovernanceObject()`

* Drop no longer used MASTERNODE category

* Merge #10153: logging: Fix off-by-one for shrinkdebugfile default

faab624 logging: Fix off-by-one for shrinkdebugfile (MarcoFalke)

Tree-SHA512: d6153e06067906172ff0611af9e585a3ecf0a7d56925b6ad7c12e75aa802441047059b9b6f6c78e79916c3f2abc8f1998bfd2d5b84201ec6421f727c08da3c21

* Shift dash-specific log categories to start from `1ul << 32` to avoid potential future conflicts with bitcoin ones

* Fix `dash` category

* remove debugCategories

Signed-off-by: Pasta <Pasta@dash.org>

* Prepend "std::" to find call

* Check for BCLog::PRIVATESEND instead of logCategories != BCLog::NONE

* Use BCLog::MNPAYMENTS category instead of checking for logCategories != BCLog::NONE

* Move "End Dash" comment below "ALERT"

When adding new entries here, we'll otherwise get confused with ordering
and might end up forgetting that adding something Dash specific must
continue with the bit after 43.
2019-05-23 00:51:39 +03:00
Alexander Block
5652ea0237 Show number of InstantSend locks in Debug Console (#2919)
* Implement GetInstantSendLockCount in CInstantSendManager

* Add islockCountChanged signal to client model

* Show number of InstantSend locks in debug console
2019-05-22 09:35:51 +03:00
PastaPastaPasta
a6eee07f29 Merge bitcoin#9963: util: Properly handle errors during log message formatting (#2917)
* Merge #9963: util: Properly handle errors during log message formatting

b651270 util: Throw tinyformat::format_error on formatting error (Wladimir J. van der Laan)
3b092bd util: Properly handle errors during log message formatting (Wladimir J. van der Laan)

Tree-SHA512: 85e3b7afec2255fc88034187f1abd6060e9421de17ed4e3d918416f393429a99cc2c974b362099aaaff6970549df47664bea4c857c4e46acc0789663201dc541

* "cast" debugMsg to a c string

Signed-off-by: Pasta <Pasta@dash.org>

"cast" debugMsg to a c string pt 2

Signed-off-by: Pasta <Pasta@dash.org>
2019-05-21 15:32:33 +02:00
PastaPastaPasta
32aa229c7f Reorganize Dash Specific code into folders (#2753)
* rename instantx.* file to instantsend.*

* Reorganize Dash Specific code into folders

add to privatesend folder the header files

Signed-off-by: Pasta <Pasta@dash.org>

add "masternode" to imports

Signed-off-by: Pasta <Pasta@dash.org>

add "masternode" to imports

Signed-off-by: Pasta <Pasta@dash.org>

add "masternode" to imports pt 2

Signed-off-by: Pasta <Pasta@dash.org>
2019-05-21 15:26:15 +02:00
UdjinM6
84162021a3 Fix duplicate -instantsendnotify invocation (#2925) 2019-05-16 10:13:02 +02:00
Alexander Block
394bec4838 Skip processing in SyncTransaction when chain is not synced yet (#2920)
Applies to CInstantSendManager and CChainLocksHandler. This fixes extremely
high RAM usage on reindex and resync, which happens to many/all transactions
being kept track of non-locked TXs (nonLockedTxs) and TXs per
block (blockTxs).
2019-05-13 13:35:46 +03:00
Alexander Block
240238b190
Fix db leaks in LLMQ db (#2914)
* Store rs_t key time in big endian

Also implement ConvertInvalidTimeKeys to convert old entries. We can remove
this later when we know that most MNs have run this code on testnet.

The way we stored the time field in the past lead to CleanupOldRecoveredSigs
iterating the keys in a strange order, causing no deletion at all and the
LLMQ DB filling up.

* Write batch in CleanupOldRecoveredSigs when it gets too large

This avoids RAM filling up and OOM getting triggered.

* Keep track of when a vote was written to the DB and clean up after week

Instead of only deleting when the corresponding recovered sig is deleted.
It sometimes happens that a masternode votes on something but a recovered
sig is never created, which leaves us with a vote that will never be
deleted.

* Apply suggestions from code review

Co-Authored-By: PastaPastaPasta <pasta@dashboost.org>
2019-05-13 10:41:48 +02:00
Alexander Block
56f31195d2
Fall back to ReadBlockFromDisk when blockTxs is not filled yet (#2908)
* Fall back to ReadBlockFromDisk when blockTxs is not filled yet

This handles the case where a MN is freshly started and SyncTransaction
was not been called for older transactions/blocks.

* Also update txFirstSeenTime

* Properly handle txids == nullptr
2019-05-10 15:54:40 +02:00
Alexander Block
7696a6fc89
Print inputs on which we voted and quorums used for signing (#2907)
* Print inputs on which we voted

Also print the corresponding requestId. This makes debugging InstantSend
issues easier.

* Print quorum hash when signing a share

* Remove unused nodesByAddress map in CSigSharesManager::SendMessages

Not related to this PR, but a simple cleanup that should have no
side effects.
2019-05-10 11:24:04 +02:00
Alexander Block
a173e6836c Implement integration tests for DKG error handling (#2905)
* Allow modifying simulate DKG error rates via RPC

* Don't lie to yourself :)

* Add some missing new-lines in LogPrintf calls

* More fine grained control over which messages to expect in mine_quorum

* Implement llmq-dkgerrors.py integration tests

These test DKG errors and malicious behavior.
2019-05-08 12:13:27 +03:00
Alexander Block
66a2cdeafc Properly handle conflicts between ChainLocks and InstantSend (#2904)
* Move code to write archived ISLOCKs into its own method

We'll need this from another method as well later.

* Return ISLOCK instead of conflicting txid in GetConflictingTx/GetConflictingLock

* Implement GetInstantSendLocksByParent and RemoveChainedInstantSendLocks

These allow to easily delete multiple chains (actually trees) of ISLOCKs
in one go.

* Implement RemoveConflictedTx and call it from RemoveMempoolConflictsForLock

Also add "retryChildren" parameter to RemoveNonLockedTx so that we can
skip retrying of non-locked children TXs.

* Properly handle/remove conflicted TXs (between mempool and new blocks)

* Track non-locked TXs by inputs

* Implement and call ResolveBlockConflicts

* Also call ResolveBlockConflicts from ConnectBlock

But only when a block is known to have a conflict and at the same time is
ChainLocked, which causes the ISLOCK to be pruned.

* Split out RemoveChainLockConflictingLock from ResolveBlockConflicts

* Implement "quorum getrecsig" RPC

* Include decoded TX data in result of create_raw_tx

* Implement support for CLSIG in mininode.py

* Fix condition for update of nonLockedTxs.pindexMined

* Only add entries to nonLockedTxsByInputs when AddNonLockedTx is called for the first time

* Implement support for ISLOCK in mininode.py

* Implement tests for ChainLock vs InstantSend lock conflict resolution

* Handle review comment

Bail out (continue) early
2019-05-07 15:14:33 +03:00
Alexander Block
7fdc66dd86
Ask for locked TXs after removing conflicting TXs (#2898)
* Also test conflicts in mempool instead of only in blocks

* Ask for locked TXs after removing conflicting TXs

When we removed a conflicting TX from the mempool, the correct/locked TX
is not available locally as the first-seen rule would have filtered before.
We need to re-request this TX if any other node announced it before.

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>
2019-05-06 15:26:27 +02:00
Alexander Block
53827a3764 Remove code for QDEBUGSTATUS propagation (#2891)
* Remove code for QDEBUGSTATUS propagation

This turned out to be too expensive and could easily take the network
down by bringing all nodes to 100% CPU usage. Better to fully remove this
functionality.

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>

* Update src/rpc/rpcquorums.cpp

Co-Authored-By: codablock <ablock84@gmail.com>
2019-05-02 02:20:06 +03:00
UdjinM6
fbd244dded
Bail out in few more places when blockchain is not synced yet (#2888)
* Bail out in few more places when blockchain is not synced yet

* Apply review suggestion
2019-04-30 15:55:11 +03:00
UdjinM6
8ffdcbf999
A bunch of mostly trivial tweaks/fixes (#2889)
* Trivial: vout->txout

* Re-use SetHexStr in few more places

* Tweak log output

* fix v13 release notes links

* Drop no longer used stuff

* Few more trivial fixes

* Adjust few rpc help strings

* Apply review suggestions
2019-04-30 15:48:21 +03:00
UdjinM6
03021fa53c
Harden DIP3 activation (#2881)
* Harden DIP3 activation height

Also drop all related but no longer used parts.

* Pass current block index to GetCommitmentsFromBlock

* Allow to change dip3 activation height for tests

And fix them.
2019-04-25 18:39:04 +03:00
UdjinM6
5cfceab860
Refactor IS-lock GUI notification and implement a similar one for ChainLocks (#2875)
* Refactor IS-lock GUI notification and implement a similar one for ChainLocks

* Initialize cachedNumISLocks in TransactionStatus ctor
2019-04-25 18:37:39 +03:00
Alexander Block
22ae0bc212 Archive islock hashes when removing confirmed islocks (#2872)
This allows AlreadyHave to check if an announced (via INV) islock was
already known in the past. This avoids requesting islocks which got
obsolete due to ChainLocks.
2019-04-16 16:40:26 +03:00
Alexander Block
cd94cbe6f0 Track which TXs are not locked yet and use this info in ProcessPendingRetryLockTxs (#2869)
* Track which TXs are not locked yet and use this info in ProcessPendingRetryLockTxs

Instead of relying on ReadBlockFromDisk. This should be less disk+CPU
intensive but require more RAM.

It also fixes a bug in ProcessPendingRetryLockTxs which caused ChainLocked
parents to not be considered for retrying of its children.

* Handle review commments
2019-04-16 16:39:34 +03:00
UdjinM6
cff9f97179
Prefix all bls/quorum threads with dash- (#2865)
Makes it easier to find them. Note: had to s/quorum/q/ to fit into 15 characters.
2019-04-15 17:42:43 +03:00
UdjinM6
5e865f9c8a
Bump mempool counter on each successful IS lock (#2864) 2019-04-15 17:41:51 +03:00
Alexander Block
0c54e41f22 Retry locking of child TXs in batches instead of per locked parent (#2858)
This especially avoids many calls to ReadBlockFromDisk
2019-04-12 14:36:52 +03:00
Alexander Block
7fe1a4a78b Also invoke WriteInstantSendLockMined when IS lock comes after the mined block (#2861) 2019-04-11 23:11:56 +03:00
Alexander Block
f32f9523b9 Use lazy BLS signatures more often and don't always verify self-recovered sigs (#2860)
* Make CBLSLazySignature thread safe

* Perform malleability check in CBLSLazySignature

* Use CBLSLazySignature in CRecoveredSig and CInstantSendLock

* Only sporadically verify self-recovered signatures

* test
2019-04-11 23:11:39 +03:00
Alexander Block
5e8ae2ceb6 Disable optimistic send in PushMessage by default (#2859)
* Automatically wake up select() when optimistic send was not used

But only when we know that we are actually inside select() and that it
currenlty is unlikely for it to have selected the node's socket for
sending. We accept race conditions here as the select() timeout
will ensure that we always send the data.

* Don't manually call WakeSelect() in CSigSharesManager::SendMessages

Not needed anymore

* Disable optimistic send in PushMessage by default
2019-04-11 15:43:22 +03:00
Alexander Block
90b1b71967 Move processing of InstantSend locks into its own worker thread (#2857)
* Let ProcessPendingInstantSendLocks return true when it did some work

* Introduce own worker thread for CInstantSendManager

Instead of using the scheduler.

* Remove scheduler from CInstantSendManager

* Add missing reset() call for workInterrupt
2019-04-11 15:43:00 +03:00
Alexander Block
ae78360e5d Add cache to CQuorumManager::ScanQuorums (#2856) 2019-04-11 15:42:35 +03:00
Alexander Block
82a47f5432 Allow testing ChainLocks enforcement with spork19 == 1 (#2854)
This is only for testnet.
2019-04-11 15:41:51 +03:00
Alexander Block
225c3898d8
Don't skip "safe TX" check when ChainLocks are not enforced yet (#2851) 2019-04-10 18:16:33 +02:00
Alexander Block
b0260e970f Do not maintain CService in masternodeQuorumNodes
Not needed anymore as we retrieve the service/address of a masternode
on-the-fly in ThreadOpenMasternodeConnections.
2019-04-09 12:32:52 +02:00
Alexander Block
b5bc7c9dac Call HandleFullyConfirmedBlock when ChainLocks are enabled but not enforced (#2844)
Otherwise IS locks never get removed before DIP8 activates via BIP9.
2019-04-08 08:07:47 +03:00
Alexander Block
565754e019
Merge pull request #2839 from codablock/pr_llmq_scanquorumsfix
Fix invalid results from ScanQuorum
2019-04-05 14:39:17 +02:00
Alexander Block
f8bedba7c4
Don't retry locks when new IS system is disabled (#2837)
Retrying still failed, but this spammed logs quite a bit.
2019-04-05 14:38:58 +02:00
Alexander Block
206e5a1b47 Use big endian inversed height in CInstantSendDb 2019-04-05 13:40:18 +02:00
Alexander Block
4b9f6cd3a9 Use big endian inversed height in BuildInversedHeightKey
Otherwise keys are not properly sorted.
2019-04-05 13:39:29 +02:00
Alexander Block
d34ec78664 Update wallet transactions when confirmed IS locks are removed 2019-04-05 08:18:56 +02:00
Alexander Block
b897505f85 Remove the need for maintaining the last ChainLocked block in the DB 2019-04-05 08:05:12 +02:00
Alexander Block
8e7083cb89 Use db.RemoveConfirmedISLocks() in NotifyChainLock to remove confirmed locks
Also move the actual logic into HandleFullyConfirmedBlock and call it
from NotifyChainLock and UpdatedBlockTip.
2019-04-05 08:05:12 +02:00
Alexander Block
4577438e87 Implement RemoveConfirmedInstantSendLocks to prune confirmed IS locks from DB 2019-04-05 08:05:12 +02:00
Alexander Block
d6e7758519 Keep track of when IS locks were mined 2019-04-05 06:36:09 +02:00
Alexander Block
20ec1de4c6
Merge pull request #2833 from codablock/pr_dip4_quorums
Implement quorum merkle roots for DIP4 coinbases and add quorums to MNLISTDIFF
2019-04-05 05:58:05 +02:00
UdjinM6
0a6416e064 Wipe llmq db on reindex (#2835) 2019-04-05 05:57:46 +02:00
UdjinM6
208406df71 Re-introduce nInstantSendKeepLock check for LLMQ-based IS when spork19 is OFF (#2829) 2019-04-05 05:55:30 +02:00
Alexander Block
7d765a0fce Track best block to later know if a DB upgrade is needed 2019-04-04 17:58:51 +02:00
Alexander Block
44a3b9c90f Don't use pindex->GetBlockHash() in ProcessCommitment
The block hash is not necessarily set at this point, for example when
coming from TestBlockValidity().
2019-04-04 12:27:35 +02:00
Alexander Block
f9dbe3ed50 Track in which block a quorum commitment was mined
And return it in GetMinedCommitment and the "quorum info" RPC
2019-04-04 12:27:35 +02:00
Alexander Block
07620746a5 Implement GetMined(AndActive)CommitmentsUntilBlock and use it in ScanQuorums
This fixes a bug in ScanQuorums which made it return quorums which were not
mined at the time of pindexStart. This was due to quorumHashes being based
on older blocks (the phase=0 block) which are ancestors of pindexStart even
if the commitment was actually mined in a later block.

GetMinedAndActiveCommitmentsUntilBlock is also going to be used for quorum
commitment merkle roots in CCbTx.

This also removes GetFirstMinedQuorumHash as it's not needed anymore.
2019-04-04 12:27:35 +02:00
Alexander Block
d5250a333a Track at which height a quorum commitment was mined
Also add upgrade code to allow smooth upgrades from v13 to v14.
2019-04-04 12:27:35 +02:00
Alexander Block
806948f90e Store the full commitment in CQuorum
Instead of just individual fields of it.
2019-04-04 12:27:35 +02:00
UdjinM6
9f04855ae0
Fix blsWorker (#2820)
Use a pointer instead of a static variable, start/stop together with other llmq modules.
2019-04-01 14:10:39 +03:00