Commit Graph

165 Commits

Author SHA1 Message Date
Alexander Block
53827a3764 Remove code for QDEBUGSTATUS propagation (#2891)
* Remove code for QDEBUGSTATUS propagation

This turned out to be too expensive and could easily take the network
down by bringing all nodes to 100% CPU usage. Better to fully remove this
functionality.

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>

* Update src/rpc/rpcquorums.cpp

Co-Authored-By: codablock <ablock84@gmail.com>
2019-05-02 02:20:06 +03:00
UdjinM6
fbd244dded
Bail out in few more places when blockchain is not synced yet (#2888)
* Bail out in few more places when blockchain is not synced yet

* Apply review suggestion
2019-04-30 15:55:11 +03:00
UdjinM6
8ffdcbf999
A bunch of mostly trivial tweaks/fixes (#2889)
* Trivial: vout->txout

* Re-use SetHexStr in few more places

* Tweak log output

* fix v13 release notes links

* Drop no longer used stuff

* Few more trivial fixes

* Adjust few rpc help strings

* Apply review suggestions
2019-04-30 15:48:21 +03:00
UdjinM6
03021fa53c
Harden DIP3 activation (#2881)
* Harden DIP3 activation height

Also drop all related but no longer used parts.

* Pass current block index to GetCommitmentsFromBlock

* Allow to change dip3 activation height for tests

And fix them.
2019-04-25 18:39:04 +03:00
UdjinM6
5cfceab860
Refactor IS-lock GUI notification and implement a similar one for ChainLocks (#2875)
* Refactor IS-lock GUI notification and implement a similar one for ChainLocks

* Initialize cachedNumISLocks in TransactionStatus ctor
2019-04-25 18:37:39 +03:00
Alexander Block
22ae0bc212 Archive islock hashes when removing confirmed islocks (#2872)
This allows AlreadyHave to check if an announced (via INV) islock was
already known in the past. This avoids requesting islocks which got
obsolete due to ChainLocks.
2019-04-16 16:40:26 +03:00
Alexander Block
cd94cbe6f0 Track which TXs are not locked yet and use this info in ProcessPendingRetryLockTxs (#2869)
* Track which TXs are not locked yet and use this info in ProcessPendingRetryLockTxs

Instead of relying on ReadBlockFromDisk. This should be less disk+CPU
intensive but require more RAM.

It also fixes a bug in ProcessPendingRetryLockTxs which caused ChainLocked
parents to not be considered for retrying of its children.

* Handle review commments
2019-04-16 16:39:34 +03:00
UdjinM6
cff9f97179
Prefix all bls/quorum threads with dash- (#2865)
Makes it easier to find them. Note: had to s/quorum/q/ to fit into 15 characters.
2019-04-15 17:42:43 +03:00
UdjinM6
5e865f9c8a
Bump mempool counter on each successful IS lock (#2864) 2019-04-15 17:41:51 +03:00
Alexander Block
0c54e41f22 Retry locking of child TXs in batches instead of per locked parent (#2858)
This especially avoids many calls to ReadBlockFromDisk
2019-04-12 14:36:52 +03:00
Alexander Block
7fe1a4a78b Also invoke WriteInstantSendLockMined when IS lock comes after the mined block (#2861) 2019-04-11 23:11:56 +03:00
Alexander Block
f32f9523b9 Use lazy BLS signatures more often and don't always verify self-recovered sigs (#2860)
* Make CBLSLazySignature thread safe

* Perform malleability check in CBLSLazySignature

* Use CBLSLazySignature in CRecoveredSig and CInstantSendLock

* Only sporadically verify self-recovered signatures

* test
2019-04-11 23:11:39 +03:00
Alexander Block
5e8ae2ceb6 Disable optimistic send in PushMessage by default (#2859)
* Automatically wake up select() when optimistic send was not used

But only when we know that we are actually inside select() and that it
currenlty is unlikely for it to have selected the node's socket for
sending. We accept race conditions here as the select() timeout
will ensure that we always send the data.

* Don't manually call WakeSelect() in CSigSharesManager::SendMessages

Not needed anymore

* Disable optimistic send in PushMessage by default
2019-04-11 15:43:22 +03:00
Alexander Block
90b1b71967 Move processing of InstantSend locks into its own worker thread (#2857)
* Let ProcessPendingInstantSendLocks return true when it did some work

* Introduce own worker thread for CInstantSendManager

Instead of using the scheduler.

* Remove scheduler from CInstantSendManager

* Add missing reset() call for workInterrupt
2019-04-11 15:43:00 +03:00
Alexander Block
ae78360e5d Add cache to CQuorumManager::ScanQuorums (#2856) 2019-04-11 15:42:35 +03:00
Alexander Block
82a47f5432 Allow testing ChainLocks enforcement with spork19 == 1 (#2854)
This is only for testnet.
2019-04-11 15:41:51 +03:00
Alexander Block
225c3898d8
Don't skip "safe TX" check when ChainLocks are not enforced yet (#2851) 2019-04-10 18:16:33 +02:00
Alexander Block
b0260e970f Do not maintain CService in masternodeQuorumNodes
Not needed anymore as we retrieve the service/address of a masternode
on-the-fly in ThreadOpenMasternodeConnections.
2019-04-09 12:32:52 +02:00
Alexander Block
b5bc7c9dac Call HandleFullyConfirmedBlock when ChainLocks are enabled but not enforced (#2844)
Otherwise IS locks never get removed before DIP8 activates via BIP9.
2019-04-08 08:07:47 +03:00
Alexander Block
565754e019
Merge pull request #2839 from codablock/pr_llmq_scanquorumsfix
Fix invalid results from ScanQuorum
2019-04-05 14:39:17 +02:00
Alexander Block
f8bedba7c4
Don't retry locks when new IS system is disabled (#2837)
Retrying still failed, but this spammed logs quite a bit.
2019-04-05 14:38:58 +02:00
Alexander Block
206e5a1b47 Use big endian inversed height in CInstantSendDb 2019-04-05 13:40:18 +02:00
Alexander Block
4b9f6cd3a9 Use big endian inversed height in BuildInversedHeightKey
Otherwise keys are not properly sorted.
2019-04-05 13:39:29 +02:00
Alexander Block
d34ec78664 Update wallet transactions when confirmed IS locks are removed 2019-04-05 08:18:56 +02:00
Alexander Block
b897505f85 Remove the need for maintaining the last ChainLocked block in the DB 2019-04-05 08:05:12 +02:00
Alexander Block
8e7083cb89 Use db.RemoveConfirmedISLocks() in NotifyChainLock to remove confirmed locks
Also move the actual logic into HandleFullyConfirmedBlock and call it
from NotifyChainLock and UpdatedBlockTip.
2019-04-05 08:05:12 +02:00
Alexander Block
4577438e87 Implement RemoveConfirmedInstantSendLocks to prune confirmed IS locks from DB 2019-04-05 08:05:12 +02:00
Alexander Block
d6e7758519 Keep track of when IS locks were mined 2019-04-05 06:36:09 +02:00
Alexander Block
20ec1de4c6
Merge pull request #2833 from codablock/pr_dip4_quorums
Implement quorum merkle roots for DIP4 coinbases and add quorums to MNLISTDIFF
2019-04-05 05:58:05 +02:00
UdjinM6
0a6416e064 Wipe llmq db on reindex (#2835) 2019-04-05 05:57:46 +02:00
UdjinM6
208406df71 Re-introduce nInstantSendKeepLock check for LLMQ-based IS when spork19 is OFF (#2829) 2019-04-05 05:55:30 +02:00
Alexander Block
7d765a0fce Track best block to later know if a DB upgrade is needed 2019-04-04 17:58:51 +02:00
Alexander Block
44a3b9c90f Don't use pindex->GetBlockHash() in ProcessCommitment
The block hash is not necessarily set at this point, for example when
coming from TestBlockValidity().
2019-04-04 12:27:35 +02:00
Alexander Block
f9dbe3ed50 Track in which block a quorum commitment was mined
And return it in GetMinedCommitment and the "quorum info" RPC
2019-04-04 12:27:35 +02:00
Alexander Block
07620746a5 Implement GetMined(AndActive)CommitmentsUntilBlock and use it in ScanQuorums
This fixes a bug in ScanQuorums which made it return quorums which were not
mined at the time of pindexStart. This was due to quorumHashes being based
on older blocks (the phase=0 block) which are ancestors of pindexStart even
if the commitment was actually mined in a later block.

GetMinedAndActiveCommitmentsUntilBlock is also going to be used for quorum
commitment merkle roots in CCbTx.

This also removes GetFirstMinedQuorumHash as it's not needed anymore.
2019-04-04 12:27:35 +02:00
Alexander Block
d5250a333a Track at which height a quorum commitment was mined
Also add upgrade code to allow smooth upgrades from v13 to v14.
2019-04-04 12:27:35 +02:00
Alexander Block
806948f90e Store the full commitment in CQuorum
Instead of just individual fields of it.
2019-04-04 12:27:35 +02:00
UdjinM6
9f04855ae0
Fix blsWorker (#2820)
Use a pointer instead of a static variable, start/stop together with other llmq modules.
2019-04-01 14:10:39 +03:00
Alexander Block
8f280f3466
Split "llmq" debug category into "llmq", "llmq-dkg" and "llmq-sigs" (#2799)
* Use llmq-sig log category for CSigSharesManager

* Use llmq-dkg log category for DKG related logging
2019-03-25 05:48:18 +01:00
Alexander Block
15c720dd43 Stop tracking interested/participating nodes and send/announce to MNAUTH peers (#2798)
* Pass CNode* to IsMasternodeQuorumNode and let it also check verifiedProRegTxHash

This makes IsMasternodeQuorumNode return true on incoming peer connections
as well.

* Let GetMasternodeQuorumNodes also take verifiedProRegTxHash into account

This makes it return NodeIds for incoming peer connections as well.

* Remove AddParticipatingNode and the need for it

This was needed in the past when we were unable to identify incoming
connections from other quorum members. Now that we have MNAUTH, we can
easily identify all connected members.

* Don't track interestedIn quorums in CSigSharesNodeState anymore

Same as with the previous commit, we're now able to easily identify which
nodes to announce sig shares to.

* Remove unused CConnman::GetMasternodeQuorumAddresses
2019-03-22 17:21:34 +03:00
Alexander Block
b18f8cb77c
Implement MNAUTH and allow unlimited inbound MN connections (#2790)
* Sort evo/* source files in Makefile.am

* Keep track of proRegTxHash in CConnman::masternodeQuorumNodes map

We will later need the proRegTxHash

* Fix serialization of std::tuple with const rvalue elements

Having serialization and deserialization in the same specialized template
results in compilation failures due to the "if(for_read)" branch.

* Implement MNAUTH message

This allows masternodes to authenticate themself.

* Protect fresh incoming connections for a second from eviction

Give fresh connections some time to do the VERSION/VERACK handshake and
an optional MNAUTH when it's a masternode. When an MNAUTH happened, the
incoming connection is then forever protected against eviction.

If a timeout of 1 second occurs or the first message after VERACK is not
MNAUTH, the node is not protected anymore and becomes eligable for
eviction.

* Avoid connecting to masternodes if an incoming connection is from the same one

Now that incoming connections from MNs authenticate them self, we can avoid
connecting to the same MNs through intra-quorum connections.

* Apply review suggestions
2019-03-22 11:52:37 +01:00
Alexander Block
7b76e7abb2
Implement BIP9 style deployment for DIP8/ChainLocks and fix a bug with late headers (#2793)
* Also update bestChainLockWithKnownBlock in AcceptedBlockHeader

* Implement BIP9 style DIP8 deployment

* Fix ChainLocks tests

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>
2019-03-22 11:51:50 +01:00
Alexander Block
02db06658e
Fix loop in CLLMQUtils::GetQuorumConnections to add at least 2 connections (#2796)
* Fix warning about size_t to int conversion

* Fix loop in CLLMQUtils::GetQuorumConnections to add at least 2 connections

When reaching very small quorum sizes, the current algorithm results in
only a single connection to be added. This would be fine usually, but is an
issue when this connection fails. We should always have at least one backup
connection.

This fixes simple PoSe test failures where the quorum size gets down to 4
with one of the 4 members being down. If other nodes are unlucky to connect
to this node, they fail as well even though 3 members in a quorum should
work fine.

* Update src/llmq/quorums_utils.cpp

Co-Authored-By: codablock <ablock84@gmail.com>
2019-03-22 11:51:12 +01:00
Alexander Block
9e70209e49
Honor bloom filters when announcing LLMQ based IS locks (#2784)
* Split out GetInstantSendLockHashByTxid from GetInstantSendLockByTxid

* Filter ISLOCK messages based on provided filter
2019-03-21 07:47:39 +01:00
Alexander Block
12274e578a
Introduce "qsendrecsigs" to indicate that plain recovered sigs should be sent (#2783)
* Introduce "qsendrecsigs" to indicate that plain recovered sigs should be sent

Full nodes, including masternodes, will send this message automatically.
Other node implementations (e.g. SPV) are usually not interested and would
not send this message.

* Use std::atomic<bool> instead of std::atomic_bool

Not related to this PR, but a small enough change to include it here as
well.
2019-03-21 07:47:02 +01:00
Alexander Block
b87821047c
Make LLMQ/InstantSend/ChainLocks code less spammy (#2781)
* Add support for log category to CBatchedLogger

* Use "llmq" logging category in LLMQ code

* Use "chainlocks" logging category in ChainLocks code

* Log errors without logging category
2019-03-21 07:46:27 +01:00
Alexander Block
591b0185c5
Bump proto version and only send LLMQ related messages to v14 nodes (#2780)
* Bump PROTOCOL_VERSION to 70214

* Introduce LLMQS_PROTO_VERSION to avoid relaying to old nodes
2019-03-20 10:58:14 +01:00
Alexander Block
c3602372cc Implement retroactive IS locking of transactions first seen in blocks instead of mempool (#2770)
* Don't rely on UTXO set in CheckCanLock

The UTXO set only works for TXs in the mempool and won't work when we try
to retroactively lock unlocked TXs from blocks.

This is safe as ProcessTx is only called when a TX was accepted into the
mempool or connected in a block, which means that all input checks were
good.

* Rename RetryLockMempoolTxs to RetryLockTxs and let it retry connected TXs

* Instead of manually calling ProcessTx, let SyncTransaction handle all cases

SyncTransaction is called from AcceptToMemoryPool and when transactions got
connected in a block. So this is the time we want to run TXs through
ProcessTx. This also enables retroactive signing of TXs that were unknown
before a new block appeared.

* Test retroactive signing and safe TXs in LLMQ ChainLocks tests

* Also test for retroactive signing of chained TXs

* Honor lockedParentTx when looking for TXs to retry signing

* Stop scanning for TXs to retry after a depth of 6

* Generate 6 block to avoid retroactive signing overloading Travis

* Avoid retroactive signing

* Don't rely on NewPoWValidBlock and use SyncTransaction to build blockTxs

NewPoWValidBlock is not guaranteed to be called when blocks come in fast.
When a block is accepted in AcceptBlock, NewPoWValidBlock is only called
when the new block is a successor of the currently active tip. This is not
the case when after the first block a second block is accepted immediately
as the first block is not connected yet.

This might be a bug actually in the handling of NewPoWValidBlock, so we
might need to check/fix this later, but currently I prefer to not touch
that part.

Instead, we now use SyncTransaction to gather TXs for blockTxs. This works
because SyncTransaction is called for all transactions in a freshly
connected block in one go. The call also happens before UpdatedBlockTip is
called, so it's fine with the existing logic.

* Use tx.IsCoinBase() instead of checking index 0

Also check for empty vin.
2019-03-19 13:55:51 +03:00
Alexander Block
5299d39338 Multiple refactorings/fixes for LLMQ bases InstantSend and ChainLocks (#2779)
* Remove unused parameters from CInstantSendManager::ProcessTx

* Pass txHash in CheckCanLock by reference instead of pointer

* Dont' allow locking of TXs without inputs

* Remove unused local variable nInstantSendConfirmationsRequired

* Don't subtract 1 from nInstantSendConfirmationsRequired

This was necessary in the old system but is not necessary in the new system.
It also prevented proper retroactive signing of chained TXs in regtest as
it resulted in child TXs to return true immediately for CheckCanLock when
it should actually have waited for the parent TX to become locked first.

* Access chainActive.Height() while cs_main is locked

* Properly read and write lastChainLockBlock

"pindex" is NOT the chainlocked block after the while loop finishes. We
must use the pindex (renamed to pindexChainLock now) given on method entry.

Also, the GetLastChainLockBlock() result was not assigned to,
lastChainLockBlock which resulted in the while loop to run unnecessarily
long.

* Generalize filtering in NewPoWValidBlock and SyncTransaction

We're actually interested in all TXs that have inputs, so no need to
explicitly check for tx types.

* Use tx.IsCoinBase() instead of checking for index 0

* Handle cases where a TX is not received yet in wait_for_instantlock

* Wait on all nodes for the locks

Otherwise we end up with the sender having it locked but other nodes
not yet, failing the test.

* Fix LogPrintf call in CChainLocksHandler::DoInvalidateBlock
2019-03-19 10:38:16 +03:00
Alexander Block
a69a5cf4a0 Use smaller (3 out of 5) quorums for regtest/Travis (#2774)
* Require only 3 out of 5 signatures for old InstantSend in regtest mode

* Use LLMQs of size 5 with threshold of 3 for regtest

* Fix wrong check for out-of-range bits in CFixedBitSet

* Reduce number of masternodes in masternode/LLMQ tests

* Add missing \n to LogPrintf call

* Use correct indexes for isolated/receiver/sender nodes

The way it was before resulted in nodes 1-3 being unused and 6-8 being used
for these 3 special nodes even though these are masternodes.

* Avoid stopping/starting isolated node in p2p-instantsend.py

It's enough to disable networking for this node.
2019-03-15 11:48:24 +03:00