Commit Graph

388 Commits

Author SHA1 Message Date
PastaPastaPasta
7a5d407563
replace inter-quorum with intra-quorum (#3642)
Inter means between two groups, intra means inside of a group.

Signed-off-by: pasta <pasta@dashboost.org>
2020-08-01 23:08:30 +03:00
UdjinM6
ffc38301f7 More of 13153 (fix LogPrintf-s) -- TODO: backport to 0.16 2020-07-29 10:56:26 -05:00
UdjinM6
d514668f5e More of 13153 ("Continued") 2020-07-29 10:56:10 -05:00
MarcoFalke
db747ea384 Merge #12757: Clarify include guard naming convention
3bcc0059b8 Add lint-include-guards.sh which checks include guard consistency (practicalswift)
8fd6af89a0 Fix missing or inconsistent include guards (practicalswift)
8af65d96f4 Document include guard convention (practicalswift)

Pull request description:

  * **Documentation**: Document include guard convention
  * **Fix**: Fix missing or inconsistent include guards
  * **Regression test**: Add `lint-include-guards.sh` which checks include guard consistency

Tree-SHA512: 8171878f60fd08ccbea943a11e835195750592abb9d7ab74eaa4265ae7fac523b1da9d31ca13d6ab73dd596e49986bfb7593c696e5f39567c93e610165bc2acc
Signed-off-by: pasta <pasta@dashboost.org>

# Conflicts:
#	src/bech32.h
#	src/consensus/merkle.h
#	src/key_io.h
#	src/policy/fees.h
#	src/rpc/server.h
#	src/script/bitcoinconsensus.h
#	src/wallet/coinselection.h
2020-07-19 00:42:54 -05:00
dustinface
d157718243
llmq: Fix thread handling in CDKGSessionManager and CDKGSessionHandler (#3601)
* llqm: Fix thread handling in CDKGSessionManager and CDKGSessionHandler

* llmq: Removed unused thread_pool from CDKGSessionManager

* Tweak `CDKGSessionHandler::StartThread()`

* llmq: Simplify CDKGSessionHandler's thread naming

* llmq: Make sure CDKGSessionHandler uses a valid LLMQ type

Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-07-16 23:30:59 +03:00
dustinface
2b8be8f873
llmq: Remove unused variable in CLLMQUtils::GetQuorumConnections (#3595) 2020-07-09 04:03:20 +03:00
pasta
9d26ad6d8f remove duplicate includes so that ./contrib/devtools/lint-includes.sh is clean
Signed-off-by: pasta <pasta@dashboost.org>
2020-07-01 20:12:03 -05:00
Alexander Block
3f7553a711
Disable new connection handling and concentrated recovery for large LLMQs (#3548)
* Rename CollectSigSharesToSend to CollectSigSharesToSendConcentrated

* Allow to disable concentrated recovery and new connection handling for large LLMQs
2020-06-25 09:47:24 +02:00
UdjinM6
955e3868fb
Fix CSigningManager::VerifyRecoveredSig (#3546) 2020-06-24 10:04:10 +02:00
Alexander Block
de099fe7a3
Use exponential backoff timeouts for recovery (#3535)
Starting with 2 seconds.
2020-06-24 10:02:57 +02:00
PastaPastaPasta
3f8a7b2068
Merge #12193: RPC: Consistently use UniValue.pushKV instead of push_back(Pair()) (karel-3d) (#3532)
* Begin Merge 12193 Squashed 'src/univalue/' changes from 07947ff2da..51d3ab34ba

51d3ab34ba Merge #10: Add pushKV(key, boolean) function (replaces #5)
129bad96d5 [tests] test pushKV for boolean values
b3c44c947f Pushing boolean value to univalue correctly

git-subtree-dir: src/univalue
git-subtree-split: 51d3ab34ba2857f0d03dc07250cb4a2b5e712e67

* scripted-diff: Use UniValue.pushKV instead of push_back(Pair()) (end #12193)

-BEGIN VERIFY SCRIPT-
git grep -l "push_back(Pair" | xargs sed -i "s/push_back(Pair(\(.*\)));/pushKV(\1);/g"
-END VERIFY SCRIPT-

Signed-off-by: pasta <pasta@dashboost.org>

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-06-18 12:17:23 +03:00
Alexander Block
737ed479bc
Don'd send SENDXXX messages to fMasternode connections (#3537)
And respect a received QSENDRECSIGS no matter if fMasternode or not. We
assume that fMasternode peers won't send QSENDRECSIGS from now on.
2020-06-18 11:41:18 +03:00
Alexander Block
0fa058deb5
Only relay DKG messages to intra quorum connection members (#3536)
This reuses the old intra-quorum connections algorithm.
2020-06-18 11:40:33 +03:00
Pasta
ec7c7dfc8d
fix lint failure
Signed-off-by: Pasta <pasta@dashboost.org>
2020-06-12 01:17:37 -05:00
pasta
518ccbf986
scripted-diff: Convert 11 enums into scoped enums (C++11) (merge #10742)
-BEGIN VERIFY SCRIPT-

sed -i 's/enum DBErrors/enum class DBErrors/g' src/wallet/walletdb.h
git grep -l DB_ | xargs sed -i 's/DB_\(LOAD_OK\|CORRUPT\|NONCRITICAL_ERROR\|TOO_NEW\|LOAD_FAIL\|NEED_REWRITE\)/DBErrors::\1/g'
sed -i 's/^    DBErrors::/    /g' src/wallet/walletdb.h

sed -i 's/enum VerifyResult/enum class VerifyResult/g' src/wallet/db.h
sed -i 's/\(VERIFY_OK\|RECOVER_OK\|RECOVER_FAIL\)/VerifyResult::\1/g' src/wallet/db.cpp

sed -i 's/enum ThresholdState/enum class ThresholdState/g' src/versionbits.h
git grep -l THRESHOLD_ | xargs sed -i 's/THRESHOLD_\(DEFINED\|STARTED\|LOCKED_IN\|ACTIVE\|FAILED\)/ThresholdState::\1/g'
sed -i 's/^    ThresholdState::/    /g' src/versionbits.h

sed -i 's/enum SigVersion/enum class SigVersion/g' src/script/interpreter.h
git grep -l SIGVERSION_ | xargs sed -i 's/SIGVERSION_\(BASE\|WITNESS_V0\)/SigVersion::\1/g'
sed -i 's/^    SigVersion::/    /g' src/script/interpreter.h

sed -i 's/enum RetFormat {/enum class RetFormat {/g' src/rest.cpp
sed -i 's/RF_\(UNDEF\|BINARY\|HEX\|JSON\)/RetFormat::\1/g' src/rest.cpp
sed -i 's/^    RetFormat::/    /g' src/rest.cpp

sed -i 's/enum HelpMessageMode {/enum class HelpMessageMode {/g' src/init.h
git grep -l HMM_ | xargs sed -i 's/HMM_BITCOIN/HelpMessageMode::BITCOIN/g'
sed -i 's/^    HelpMessageMode::/    /g' src/init.h

sed -i 's/enum FeeEstimateHorizon/enum class FeeEstimateHorizon/g' src/policy/fees.h

sed -i 's/enum BlockSource {/enum class BlockSource {/g' src/qt/clientmodel.h
git grep -l BLOCK_SOURCE_ | xargs sed -i 's/BLOCK_SOURCE_\(NONE\|REINDEX\|DISK\|NETWORK\)/BlockSource::\1/g'
sed -i 's/^    BlockSource::/    /g' src/qt/clientmodel.h

sed -i 's/enum FlushStateMode {/enum class FlushStateMode {/g' src/validation.cpp
sed -i 's/FLUSH_STATE_\(NONE\|IF_NEEDED\|PERIODIC\|ALWAYS\)/FlushStateMode::\1/g' src/validation.cpp
sed -i 's/^    FlushStateMode::/    /g' src/validation.cpp

-END VERIFY SCRIPT-

Signed-off-by: pasta <pasta@dashboost.org>
2020-06-11 23:20:48 -05:00
UdjinM6
89851d3361
Fix chainlock scheduler handling (#3514)
Reindexing corrupted chain always fails otherwise.
2020-06-11 11:39:23 +03:00
UdjinM6
232430fac9
Fix ProcessNewBlock vs EnforceBestChainLock deadlocks in ActivateBestChain (#3492)
* Drop dead code in DoInvalidateBlock

* Let ActivateBestChain skip SyncWithValidationInterfaceQueue when called from IS or CL threads

* Use CL's own scheduler instead of a global one

* Revert "Let ActivateBestChain skip SyncWithValidationInterfaceQueue when called from IS or CL threads"

This reverts commit 1c9f6da50a.
2020-06-09 06:53:16 +03:00
UdjinM6
ae5faf23da
Better error handling while processing special txes (#3504)
* Return invalid state for GetTxPayload failures instead of asserting

* Throw an error when internal `find` fails instead of asserting

* Unify mnlist checks for lists created from blocks and lists created via updates of older lists with diffs

* Wrap evo block processing in try..catch to make sure all earlier introduced throws are caught

* evo/llmq: Adjust reject reasons for CFinalCommitmentTxPayload

In CheckLLMQCommitment this reject string was used so i adjusted the
other one in CQuorumBlockProcessor::GetCommitmentsFromBlock and the new
introduced in PR #3506

* return state.DoS instead of asserting in cbtx.cpp

Signed-off-by: pasta <pasta@dashboost.org>

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: pasta <pasta@dashboost.org>
2020-06-08 05:57:57 +03:00
thephez
970c23d6e6
Remove logic for forcing chainlocks without DIP8 activation on testnet (#3470) 2020-05-11 15:35:55 +03:00
dustinface
ae15299117
Feature request #3425 - Add information to "protx diff" (#3468)
* llmq: Add more information to the result of CFinalCommitment::ToJson

Example output: 

```
{
      "version": 1,
      "llmqType": 1,
      "quorumHash": 
"000002a20a35757fc467f0ebe8663f76e4c5df74411cc5f1735cfd92278071fe",
      "signersCount": 49,
      "signers": "03fbffffffffff",
      "validMembersCount": 49,
      "validMembers": "03fbffffffffff",
      "quorumPublicKey": 
"8db1108b50dd741076b577f6798f0bf382ca4d11dafd55ed14aaaf613f604aa55bf1da1f0469f75105376e440381ee23",
      "quorumVvecHash": 
"feedb15d5a7ca25a68d9c51036973d37925d8439d4769d7dd7bace7440089071",
      "quorumSig": 
"1879f6260ef427dc9542088b39e02a8e73e34c0484269cbc12108ab991d438f81e5be80a5b4b5595b6634113fdf060060c8fe190677627fb2d25f6e052c7f93598d36dd415f8f84c693c6a5943098616a1e135522852d8f572bb6e1470df4a94",
      "membersSig": 
"8ef7d881fcc1e8111dd82c556ab62dad56ea5ecb47a9ac067eb2057d92d27e39530630a7f77d19adc857d33eb485a1b601b2d78ac4c73742f2036089bfb1f25441f1dfcd3cfd1eed6e7d664c0e97bf75b7d7097ebd647791fbcef070fe3e3e91"
    }
```

* llmq: Changed the byte order in ToHexStr(const std::vector<bool>& vBits)
2020-05-11 15:35:28 +03:00
10xcryptodev
017c4779ca
Fix recovery from coin db crashes (and dbcrash.py test) (#3467)
* 🪲 improve evodb consistency recovering from dbcrash

* Adjust the fix

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* Fix it

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* Disable recovery from a crash during a fork and a corresponding part of dbcrash.py

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* Skip some checks in CQuorumBlockProcessor when replaying blocks after the crash

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* Process special txes in RollforwardBlock

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* Update src/init.cpp

Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
2020-05-11 15:33:21 +03:00
Alexander Block
608aed3d85
Don't try to connect to itself through CLLMQUtils::GetQuorumConnections (#3448) 2020-04-21 16:43:59 +02:00
Alexander Block
96ed9fae39
Merge pull request #3429 from codablock/pr_remove_optimistic_send
Fix flushing of rejects before disconnecting
2020-04-17 21:46:36 +02:00
Alexander Block
d8bbdee7c4 Remove support for optimistic send
This was only used in only one remaining place and only to ensure that
reject messages are sent before closing sockets. This is solved by the
previous commit now.
2020-04-17 11:32:18 +02:00
Alexander Block
6ca78b601e Introduce NETCONN logging category and NET_NETCONN to log in NET and NETCONN
This allows easier debugging of connection issues, as it produces much less
chatter in the logs when only -debug=mnconn is given.
2020-04-17 08:52:57 +02:00
Alexander Block
72afaddf72 Introduce new way of deterministic selection of outbound LLMQ connections
The new way does not require the quorum hash to be included in the
calculation, which allows us to have determinism globally and not only
for single LLMQs.
2020-04-16 13:59:02 +02:00
Alexander Block
26dec64e79
Merge pull request #3397 from codablock/pr_backport_txrequests
Backport bitcoin#14897 and bitcoin#15834 and modify it to work with Dash messages
2020-04-09 00:12:44 +02:00
Alexander Block
6b32192bdc Fix compilation 2020-04-08 14:55:27 +02:00
Alexander Block
26fcd3f0bf Also remove m_tx_announced and m_tx_in_flight entries when EraseObjectRequest is called
Otherwise they'll run into false-positive timeouts.
2020-04-07 13:25:29 +02:00
Alexander Block
4bfc20cb67 Force re-requesting of IS locked TXs 2020-04-07 13:23:29 +02:00
Alexander Block
f142fff881 Skip verification of recovered sigs that were reconstructed in InstantSend
Also don't request them via getdata
2020-04-07 07:30:00 +02:00
Alexander Block
454fae3bda Only process 32 IS locks at a time in ProcessPendingInstantSendLocks 2020-04-07 07:29:51 +02:00
Alexander Block
d856fd1407 Use salted hasher for pendingInstantSendLocks 2020-04-07 07:29:32 +02:00
Alexander Block
465af48e83 Improve logging in LLMQ sig handling 2020-04-07 07:29:23 +02:00
Alexander Block
2443c8f582 Replace uses of CNode::AskFor with RequestObject 2020-04-07 07:14:53 +02:00
Alexander Block
004d9224c4 Replace uses of CConnman::RemoveAskFor with EraseObjectRequest 2020-04-07 07:14:42 +02:00
Alexander Block
c7b6eb851d
Merge pull request #3389 from codablock/pr_concentrated_recovery
Implement "concentrated recovery" of LLMQ signatures
2020-04-02 13:59:05 +02:00
Alexander Block
148bbdd5cf
Use GetTime instead of GetAdjustedTime
Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-04-01 18:05:44 +02:00
Alexander Block
76b6614fe5 Use auto keyword in CollectSigSharesToSend 2020-03-31 19:58:07 +02:00
Alexander Block
900ee0f194 Use range based for loop in SelectMemberForRecovery 2020-03-31 19:58:07 +02:00
Alexander Block
b212f21c15 Implement new way of concentrated signature recovery
Instead of propagating all sig shares to all LLMQ members, this will now
make all members send their individual sig share to a single member, which
is then responsible for the recovery and propagation of the recovered
signature. This process is repeated by all members every second for another
target/recovering member, until a recovered signature appears.
2020-03-31 19:58:07 +02:00
Alexander Block
45064d8dc9 Rename sigSharesToSend to sigShareBatchesToSend 2020-03-31 19:58:07 +02:00
Alexander Block
88d4f551c9 Move spork21 condition into VerifyConnectionAndMinProtoVersions 2020-03-31 07:11:27 +02:00
Alexander Block
32c83b432b Verify min proto version and open ports of LLMQ members and vote on bad ones 2020-03-31 07:07:52 +02:00
Alexander Block
b0668028b6
Implement more randomized behavior in GetQuorumConnections (#3385)
When taking the proTxHash naively, we might end up with a few unlucky MNs
which always have to perform most of the outbound connections while other
unlucky MNs would always have to wait for inbound connections. Hashing
the proTxHash with the quorum hash makes this more random.
2020-03-28 00:59:32 +03:00
Alexander Block
a09e36106e Fix onlyOutbound handling 2020-03-27 15:12:20 +01:00
Alexander Block
f82204db95 Move intra-quorum connection calculation into local func 2020-03-27 15:12:20 +01:00
Alexander Block
dbaf13848d Include inbound connections in output of "quorum dkgstatus" 2020-03-27 15:12:20 +01:00
Alexander Block
8f644d18d7 Implement probing of public ip/port of LLMQ members 2020-03-27 15:12:20 +01:00
Alexander Block
14bb62ac8e Connect all LLMQ members to all other members 2020-03-27 15:12:20 +01:00
Alexander Block
c68b5f68aa
Hold CEvoDB lock while iterating mined commitments (#3379) 2020-03-27 15:11:42 +01:00
Alexander Block
17ece14f40
Better/more logging for DKGs (#3381)
* Better/more logging for DKGs

* Use type name instead of type number in CDKGLogger
2020-03-26 13:25:01 +01:00
Alexander Block
8211ff0a8a
Merge pull request #3368 from codablock/pr_fmasternode
Don't relay anything to inbound/outbound fMasternode connections
2020-03-24 17:40:51 +01:00
Alexander Block
7f1f1d12f5 Make EnsureQuorumConnections re-set connections in every iteration
Instead of only doing it the first time. This is a preparation for
the new intra-quorum connection system (which connects all members to all
other members)
2020-03-24 10:58:29 +01:00
Alexander Block
9ef1e7cb51 Only log new quorum connections when it's actually new 2020-03-24 10:58:16 +01:00
Alexander Block
364d6c37f7 Move and unify logic for quorum connection establishment into CLLMQUtils 2020-03-24 10:58:16 +01:00
Alexander Block
f7ddee13a1
Fix possible segfault (#3365)
When CheckActiveState is called very early, there might be no chain yet.
2020-03-21 13:31:09 +01:00
Alexander Block
31825146a3 Don't relay anything to fMasternode connections
This reduces traffic on these connections to PS and DKG/LLMQ traffic only.
2020-03-20 17:16:49 +01:00
UdjinM6
9efa0e3d82
Merge pull request #3361 from PastaPastaPasta/backports-0.16-pr11
Backports 0.16 pr11
2020-03-20 01:47:53 +03:00
PastaPastaPasta
8a1ec935a0
Backport 11651 (#3358)
* scripted-diff: Replace #include "" with #include <> (ryanofsky)

-BEGIN VERIFY SCRIPT-
for f in \
  src/*.cpp \
  src/*.h \
  src/bench/*.cpp \
  src/bench/*.h \
  src/compat/*.cpp \
  src/compat/*.h \
  src/consensus/*.cpp \
  src/consensus/*.h \
  src/crypto/*.cpp \
  src/crypto/*.h \
  src/crypto/ctaes/*.h \
  src/policy/*.cpp \
  src/policy/*.h \
  src/primitives/*.cpp \
  src/primitives/*.h \
  src/qt/*.cpp \
  src/qt/*.h \
  src/qt/test/*.cpp \
  src/qt/test/*.h \
  src/rpc/*.cpp \
  src/rpc/*.h \
  src/script/*.cpp \
  src/script/*.h \
  src/support/*.cpp \
  src/support/*.h \
  src/support/allocators/*.h \
  src/test/*.cpp \
  src/test/*.h \
  src/wallet/*.cpp \
  src/wallet/*.h \
  src/wallet/test/*.cpp \
  src/wallet/test/*.h \
  src/zmq/*.cpp \
  src/zmq/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* scripted-diff: Replace #include "" with #include <> (Dash Specific)

-BEGIN VERIFY SCRIPT-
for f in \
  src/bls/*.cpp \
  src/bls/*.h \
  src/evo/*.cpp \
  src/evo/*.h \
  src/governance/*.cpp \
  src/governance/*.h \
  src/llmq/*.cpp \
  src/llmq/*.h \
  src/masternode/*.cpp \
  src/masternode/*.h \
  src/privatesend/*.cpp \
  src/privatesend/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* build: Remove -I for everything but project root

Remove -I from build system for everything but the project root,
and built-in dependencies.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/Makefile.test.include

* qt: refactor: Use absolute include paths in .ui files

* qt: refactor: Changes to make include paths absolute

This makes all include paths in the GUI absolute.

Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/qt/dash.cpp
#	src/qt/optionsmodel.cpp
#	src/qt/test/rpcnestedtests.cpp

* test: refactor: Use absolute include paths for test data files

* Recommend #include<> syntax in developer notes

* refactor: Include obj/build.h instead of build.h

* END BACKPORT #11651 Remove trailing whitespace causing travis failure

* fix backport 11651

Signed-off-by: Pasta <pasta@dashboost.org>

* More of 11651

* fix blockchain.cpp

Signed-off-by: pasta <pasta@dashboost.org>

* Add missing "qt/" in includes

* Add missing "test/" in includes

* Fix trailing whitespaces

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-20 01:46:56 +03:00
Pasta
e2c7d1f55a
add include 2020-03-14 00:57:41 -05:00
UdjinM6
af7cfd6a3f
Define constants for keys in CInstantSendDb and use them instead of plain strings (#3352) 2020-03-01 16:25:13 +03:00
UdjinM6
67c8484e20
More of 11583 for Dash-specific code 2020-01-28 15:42:25 -06:00
UdjinM6
e4ef7e8d09
Drop unused invSet in CDKGSession (#3303) 2020-01-24 17:09:25 +03:00
UdjinM6
7d39637b02 Bump copyright year to 2020 (#3290)
* Bump _COPYRIGHT_YEAR

* Run copyright update script

./contrib/devtools/copyright_header.py update .

* Update COPYING

* Bump copyright year in dash-cli/qt/tx and dashd map pages
2020-01-17 15:42:55 +01:00
UdjinM6
5857ec7154
More of 10969 2020-01-10 10:34:03 -06:00
UdjinM6
b56bebc57b
Push islock invs when syncing mempool (#3250)
* Push islock invs when syncing mempool

* Send islock invs right away instead of stacking them for later
2020-01-01 17:12:41 +03:00
UdjinM6
8b0bb9c0e3
Try to avoid being marked as a bad quorum member when we sleep for too long in SleepBeforePhase (#3245)
* Do not sleep at the last block of the phase, it's not safe

* Refactor it a bit to make it clearer what's going on here

* Stop sleeping if blocks came faster than we expected
2019-12-31 13:01:01 +03:00
UdjinM6
630f792b5c
Fix log output in CDKGPendingMessages::PushPendingMessage (#3244) 2019-12-24 22:35:21 +03:00
UdjinM6
91a996e325
Make sure mempool txes are properly processed by CChainLocksHandler despite node restarts (#3226)
* Pass nAcceptTime via TransactionAddedToMempool and use it for ChainLocks

Otherwise the "first seen" time is way off after node restart

* Don't skip TransactionAddedToMempool for chainlocks while blockchain is not synced yet

Otherwise txes from mempool.dat won't be processed there
2019-12-07 13:56:17 +03:00
UdjinM6
2b587f0ebc
Slightly refactor CDKGSessionHandler::SleepBeforePhase (#3224)
This should make code a little bit cleaner, should be no changes in the actual behaviour: non-members do not sleep already due to `sleepTime` being negative for them and `phaseTime = 0` does the same on regtest but for everyone.
2019-12-06 12:10:01 +03:00
Alexander Block
fdb05860e2 Don't join thread in CQuorum::~CQuorum when called from within the thread (#3223) 2019-12-06 12:06:13 +03:00
Alexander Block
4c00d98ea6 Allow re-signing of IS locks when performing retroactive signing (#3219)
* Implement re-signing of InstantSend inputs when TXs come in via blocks

* Use GetAdjustedTime instead of GetTimeMillis in CSigSharesManager

This allows use of mocktime in tests.

* Expose verifiedProRegTxHash in getpeerinfo and implement wait_for_mnauth

* Allow to wait for IS and CL to NOT happen

* Bump timeout for wait_for_instantlock

* Implement tests for retroactive signing of IS and CLs

* Add wait_for_tx function to DashTestFramework

* Add -whitelist=127.0.0.1 to node0

* Use node3 for isolated block generation

* Don't test for non-receival of TXs on node4/node5
2019-12-06 12:05:58 +03:00
UdjinM6
bbd9b10d47
Refactor nonLockedTxsByInputs (#3178) 2019-10-29 21:37:54 +03:00
Alexander Block
3c818e95bc
Only track last seen time instead of first and last seen time (#3165)
This avoids timeouts on parts of the network
2019-10-23 09:55:06 +02:00
Alexander Block
efd8d2c82b Avoid propagating InstantSend related old recovered sigs (#3145)
* More/better logging for InstantSend

* Implement CRecoveredSigsDb::TruncateRecoveredSig

* Truncate recovered sigs for ISLOCKs instead of completely removing them

This makes AlreadyHave() return true even when the recovered sig is deleted
locally. This avoids re-requesting and re-processing of old recovered sigs.

* Also truncate recovered sigs for freshly received ISLOCKs

* Fix comment
2019-10-16 17:10:06 +03:00
PastaPastaPasta
746b5f8cda Remove commented out code (#3117)
* Remove LogPrints which have been commented out.

We have version control systems for a reason, if we want code to not run it should be removed. I personally see no value in keeping these around. I presume at one point they were spamming debug.log so we commented them out, but we really should have just removed them.

I believe all of this is dash specific code but any conflicts this does create are so minor they are not of concern imo.

Signed-off-by: Pasta <pasta@dashboost.org>

* remove a couple of extra comments

Signed-off-by: Pasta <pasta@dashboost.org>

* remove commented out code

Signed-off-by: Pasta <pasta@dashboost.org>
2019-09-30 09:55:26 +02:00
PastaPastaPasta
c5da938518 A couple of minor improvements in IS code (#3114)
* check if we can lock before checking if it is conflicting, it is very rare a tx will actually be conflicting, whereas it is very common that a MN will not be able to sign for a specific Tx

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused variable

Signed-off-by: Pasta <pasta@dashboost.org>

* move sync check higher up

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused/unnecessary variable

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused variable

Signed-off-by: Pasta <pasta@dashboost.org>

* Revert "move sync check higher up"

This reverts commit 77fbe054df78b8bb12f686a627ef618ecff7e7a1.
2019-09-27 20:25:45 +02:00
UdjinM6
9f1ee8c70a
scripted-diff: Refactor llmq type consensus param names (#3093)
-BEGIN VERIFY SCRIPT-
sed -i 's/llmqChainLocks/llmqTypeChainLocks/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/llmqForInstantSend/llmqTypeInstantSend/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
-END VERIFY SCRIPT-
2019-09-24 12:38:50 +03:00
UdjinM6
1c74b668b6
Introduce getbestchainlock rpc and fix llmq-is-cl-conflicts.py (#3094)
* Introduce getbestchainlock rpc and fix llmq-is-cl-conflicts.py

* Add `known_block` field and move `getbestchainlock` to `blockchain` rpc category

* Add CChainLockSig::IsNull() and throw an exception in getbestchainlock if there is no known chainlock yet

* drop blockHash initializer
2019-09-23 21:36:55 +03:00
PastaPastaPasta
35f079cbf1 Remove unused code (#3097)
* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code, maybe need to revert / change the surrounding values

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: Pasta <pasta@dashboost.org>

* Revert "remove unused code"

This reverts commit 8ea0bbbc7b.
2019-09-22 23:45:22 +03:00
UdjinM6
9abc393839
Refactor few things here and there (#3066)
* Refactor `HandleFullyConfirmedBlock()`

* Pass `pindexMined` into `AddNonLockedTx()`

* Unify time import in Dash specific tests

* Refactor UpdateSpork and SetPrivKey
2019-08-28 14:51:59 +03:00
UdjinM6
60cc3c0562
Merge pull request #3049 from PastaPastaPasta/backports-0.15-pr24
Backports 0.15 pr24
2019-08-16 14:47:58 +03:00
Pasta
1a0d52814e
#10483 scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL
-BEGIN VERIFY SCRIPT-
sed -i 's/\<NULL\>/nullptr/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h src/qt/*/*.cpp src/qt/*/*.h src/wallet/*/*.cpp src/wallet/*/*.h src/support/allocators/*.h
sed -i 's/Prefer nullptr, otherwise SAFECOOKIE./Prefer NULL, otherwise SAFECOOKIE./g' src/torcontrol.cpp
sed -i 's/tor: Using nullptr authentication/tor: Using NULL authentication/g' src/torcontrol.cpp
sed -i 's/METHODS=nullptr/METHODS=NULL/g' src/test/torcontrol_tests.cpp src/torcontrol.cpp
sed -i 's/nullptr certificates/NULL certificates/g' src/qt/paymentserver.cpp
sed -i 's/"nullptr"/"NULL"/g' src/torcontrol.cpp src/test/torcontrol_tests.cpp
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>
2019-08-12 09:07:03 -05:00
Alexander Block
dd7873857b
Re-verify invalid IS sigs when the active quorum set rotated (#3052)
* Split ProcessPendingInstantSendLocks into two methods

* Split SelectQuorumForSigning into SelectQuorumForSigning and GetActiveQuorumSet

* Implement retrying of IS lock verification when the LLMQ active set rotates
2019-08-12 09:36:09 +02:00
Alexander Block
13e0235106
Remove recovered sigs from the LLMQ db when corresponding IS locks get confirmed (#3048)
* Remove unused overload of RemoveInstantSendLock

* Move deletion of recovered sigs into own method

* Remove recovered sigs for fully confirmed IS locks

* Also remove rs_t entries when removing recovered sigs from the outside

CleanupOldRecoveredSigs already does this as the last step, but when
RemoveRecoveredSig is called from the outside (e.g. from InstantSend),
these keys are not removed. This PR fixes this by storing the write time
into rs_r and later uses it to remove the rs_t entry.

Old entries will be incompatible with this (1 byte written in the past,
4 bytes written now). This checked by comparing the data size with
sizeof(uint32_t).

* Add TODO
2019-08-12 09:35:44 +02:00
Pasta
4e46f25117
add end commend for all llmq namespace
Signed-off-by: Pasta <pasta@dashboost.org>
2019-07-15 13:55:01 -05:00
Alexander Block
2f21e55514 Remove legacy InstantSend code (#3020)
* Remove ppszTypeName from protocol.cpp and reimplement GetCommand

This removes the need to carefully maintain ppszTypeName, which required
correct order and also did not allow to permanently remove old message
types.

To get the command name for an INV type, GetCommandInternal uses a switch
which needs to be maintained from now on.

The way this is implemented also resembles the way it is implemented in
Bitcoin today, but it's not identical. The original PR that introduced the
switch case in Bitcoin was part of the Segwit changes and thus never got
backported. I decided to implement it in a slightly different way that
avoids throwing exceptions when an unknown INV type is encountered.

IsKnownType will now also leverage GetCommandInternal() to figure out if
the INV type is known locally. This has the side effect of old/legacy
message types to return false from now on. We will depend on this side
effect in later commits when we remove legacy InstantSend code.

* Stop handling/relaying legacy IX messages

When we receive an IX message, we simply treat it as a regular TX and relay
it as such.

We'll however still request IX messages when they are announced to us. We
can't simply revert to requesting TX messages in this case as it might
result in the other peer not answering due to the TX not being in mapRelay
yet. We should at some point in the future completely drop handling of IX
messages instead.

* Remove IsNewInstantSendEnabled() and only use IsInstantSendEnabled()

* Remove legacy InstantSend from GUI

* Remove InstantSend from Bitcoin/Dash URIs

* Remove legacy InstantSend from RPC commands

* Remove legacy InstantSend from wallet

* Remove legacy instantsend.h include

* Remove legacy InstantSend from validation code

* Completely remove remaining legacy InstantSend code

* Remove now unused spork

* Fix InstantSend related test failures

* Remove now obsolete auto IS tests

* Make spork2 and spork3 disabled by default

This should have no influence on mainnet as these sporks are actually set
there. This will however affect regtest, which shouldn't have LLMQ based
InstantSend enabled by default.

* Remove instantsend tests from dip3-deterministicmns.py

These were only testing legacy InstantSend

* Fix .QCheckBox#checkUsePrivateSend styling a bit

* s/TXLEGACYLOCKREQUEST/LEGACYTXLOCKREQUEST/

* Revert "verified via InstantSend" back to "verified via LLMQ based InstantSend"

* Use cmd == nullptr instead of !cmd

* Remove last parameter from AvailableCoins call

This was for fUseInstantSend which is not present anymore since rebase
2019-07-09 17:50:08 +03:00
Alexander Block
7a440d626b Optimize on-disk deterministic masternode storage to reduce size of evodb (#3017)
* Implement CompactFull() in CDBWrapper

This allows to compact the whole DB in one go.

* Implement more compact version of CDeterministicMNListDiff

This introduces CDeterministicMNStateDiff which requires to only store
fields on-disk which actually changed.

* Avoid writing mnUniquePropertyMap to disk when storing snapshots

This map can be rebuilt by simply using AddMN for each deserialized MN.

* Implement Serialize/Unserialize in CScript

This allows us to directly use READWRITE() on scripts and removes the need
for the ugly cast to CScriptBase. This commit also changes all Dash specific
uses of CScript to not use the cast.

* Keep track of registeration counts and introduce internalID for masternodes

The "internalId" is simply the number of MNs registered so far when the
new MN is added. It is deterministic and stays the same forever.

* Use internalId as keys in MN list diffs

This reduces the used size on-disk.

* Two simple speedups in MN list diff handling

1. Avoid full compare if dmn or state pointers match in BuildDiff
2. Use std::move when adding diff to listDiff in GetListForBlock

* Implement upgrade code for old CDeterministicMNListDiff format to new format

* Track tipIndex instead of tipHeight/tipBlockHash

* Store and pass around CBlockIndex* instead of block hash and height

This allows us to switch CDeterministicMNManager::GetListForBlock to work
with CBlockIndex.

* Refactor CDeterministicMNManager::GetListForBlock to require CBlockIndex*

Instead of requiring a block hash. This allows us to remove blockHash and
prevBlockHash from CDeterministicMNListDiff without the use of cs_main
locks in GetListForBlock.

* Remove prevBlockHash, blockHash and nHeight from CDeterministicMNListDiff

* Remove access to determinisitcMNManager in CMasternodeMetaMan::ToString()

The deterministic MN manager is not fully initialized yet at the time this
is called, which results in an empty list being returned everytime.

* Better logic to determine if an upgrade is needed

Reuse the "best block" logic to figure out if an upgrade is needed. Also
use it to ensure that older nodes are unable to start after the upgrade
was performed.

* Return null block hash if it was requested with getmnlistdiff

* bump CGovernanceManager::SERIALIZATION_VERSION_STRING

* Check SERIALIZATION_VERSION_STRING before deserializing anything else

* Invoke Clear() before deserializing just to be sure
2019-07-09 08:59:57 +03:00
Alexander Block
09b017fc53 Only load signingActiveQuorumCount + 1 quorums into cache (#3002)
No need to load 25 quorums when we're scanning for the larger quorums which
only have 4 active quorums. This avoids loading thousands of masternode
lists unnecessarily.
2019-06-27 23:25:02 +03:00
Pasta
6bfbe6053d
Scripted diff find src/ -name "*.cpp" ! -wholename "src/util.h" ! -wholename "src/util.cpp" | xargs perl -i -pe 's/(?<!\.)(ParseParameters|ReadConfigFile|IsArgSet|(Soft|Force)?(Get|Set)(|Bool|)Arg(s)?)\(/gArgs.\1(/g' based on bb81e17355
(#10607)

Signed-off-by: Pasta <pasta@dashboost.org>
2019-06-24 12:03:00 -05:00
Wladimir J. van der Laan
2edd094a21
Merge #10189: devtools/net: add a verifier for scriptable changes. Use it to make CNode::id private.
0f3471f net: make CNode's id private (Cory Fields)
9ff0a51 scripted-diff: net: Use accessor rather than node's id directly (Cory Fields)
e50c33e devtools: add script to verify scriptable changes (Cory Fields)

skipped travis changes

Tree-SHA512: a0ff50f4e1d38a2b63109b4996546c91b3e02e00d92c0bf04f48792948f78b1f6d9227a15d25c823fd4723a0277fc6a32c2c1287c7abbb7e50fd82ffb0f8d994

pnode->id to pnode->GetId()

Signed-off-by: Pasta <pasta@dashboost.org>
2019-06-19 08:20:15 -05:00
UdjinM6
c58184f6c4
Merge pull request #2974 from PastaPastaPasta/backports-0.15-pr14
Backports 0.15 pr14
2019-06-19 14:30:54 +03:00
Alexander Block
2e13d1305c Add some comments to make quorum merkle root calculation more clear+ (#2984)
This should avoid future confusion.
2019-06-18 14:34:16 +03:00
UdjinM6
5152e751cc Notify UI/external scripts about wallet txes locks 2019-06-14 14:50:07 -05:00
Pasta
34a1e93ca2
remove UpdatedTransaction calls in instantsend code 2019-06-14 01:25:59 -05:00
Alexander Block
864856688d Multiple speed optimizations for deterministic MN list handling (#2972)
* Generalize CBLSLazyWrapper so that it can be used of pubkeys and secret keys

* Implement == and != operators for CBLSLazyWrapper

* Implement cached hash for CBLSLazyWrapper

* Use CBLSLazyPublicKey for CDeterministicMNState::pubKeyOperator

* Speed up GetProjectedMNPayees by sorting the MN list by last paid

Instead of updating a temporary list for each projected height and calling
GetMNPayee() on it.

* Cache intermediate lists in GetListForBlock

This avoids re-loading and applying diffs again and again.

* Only update masternode list UI max once every 3 seconds

This avoids updating the UI on every block, which turned out to be very
expensive.

* Fix compilation

* Drop time restrictions for mn list update in ClientModel

They are fully handled by MasternodeList now.
2019-06-13 12:01:26 +03:00
PastaPastaPasta
d931cb723e Update copyright date (2019) (#2970)
Signed-off-by: Pasta <Pasta@dash.org>
2019-06-11 14:46:07 +03:00
PastaPastaPasta
56d1d13c43 Backport 8824 (#2968)
* Merge #8824: Refactor TxToJSON() and ScriptPubKeyToJSON()

0ff9320 refactor TxToJSON() and ScriptPubKeyToJSON() (jonnynewbs)

Tree-SHA512: caf7d590829e221522edd5b1ab8ce67b53a2c6986d3bbe8477eab420b1007bf60f885ed0a25ba9587e468c00768360ddc31db37847e862858573eaed5ed8b0d6

* remove witness and vsize

Signed-off-by: Pasta <Pasta@dash.org>

* Add valueSat

To preserve rpc output format

* Move extrapayload and special tx json output to `TxToUniv`

* Add spent index info
2019-06-11 14:42:17 +03:00
UdjinM6
85c9ea400b
Throw a bit more descriptive error message on UpgradeDB failure on pruned nodes (#2962) 2019-06-08 14:16:41 +03:00
Alexander Block
40fd1b1685 Make Consensus::LLMQType serializable and use it in serialized classes
Instead of using uint8_t and explicitely casting between the enum
and uint8_t.
2019-05-29 08:06:11 +02:00
Alexander Block
715a3e9518 Move updating of txFirstSeenTime into TransactionAddedToMempool and BlockConnected
This removes the need for SyncTransaction
2019-05-27 16:40:05 +02:00
Alexander Block
a61127e1ae Move filling of blockTxs into BlockConnected
Also add code to remove entries in blockTxs when BlockDisconnected is
called.
2019-05-27 16:40:05 +02:00
Alexander Block
d4e75abbfd Call new signals in CChainLocksHandler 2019-05-27 16:40:05 +02:00
Alexander Block
25c40fe409 Fix typo 2019-05-27 16:40:05 +02:00
Alexander Block
5233045373 Don't check for posInBlock when updating pindexMined
ProcessNewTransaction is now only called with pindex != nullptr when a
block is connected but never when it is disconnected, so there is no need
to check for posInBlock anymore (we don't even have it now).
2019-05-27 16:40:05 +02:00
Alexander Block
0b4f7b9b6d Move removal of mined-locks info to BlockDisconnected 2019-05-27 16:40:05 +02:00
Alexander Block
b645757be9 Call RemoveConflictedTx from BlockConnected instead of ProcessNewTransaction 2019-05-27 16:40:05 +02:00
Alexander Block
b36f0a324f Call new signals instead of SyncTransaction on CInstantSendManager
This commit moves all logic of SyncTransaction into ProcessNewTransaction
and then calls it from TransactionAddedToMempool and BlockConnected.

This won't compile/work at first, but the next commits will fix this.
2019-05-27 16:40:05 +02:00
UdjinM6
250195185f
Adjust CDSNotificationInterface to align with CValidationInterface changes 2019-05-27 09:03:19 -05:00
UdjinM6
428f304506
Implement rawchainlocksig and rawtxlocksig (#2930)
* Pass is-lock into `CInstantSendManager::UpdateWalletTransaction()`

* Implement `rawtxlocksig` which pushes tx+islock

* Implement `rawchainlocksig` which pushes block+clsig

* fix log category
2019-05-23 12:13:58 +03:00
PastaPastaPasta
29194b1f5a Backport Bitcoin#9424, Bitcoin#10123 and Bitcoin#10153 (#2918)
* Contains dashification. disables `-debug dash`
Merge #9424: Change LogAcceptCategory to use uint32_t rather than sets of strings.

6b3bb3d Change LogAcceptCategory to use uint32_t rather than sets of strings. (Gregory Maxwell)

Tree-SHA512: ebb5bcf9a7d00a32dd1390b727ff4d29330a038423611da01268d8e1d2c0229e52a1098e751d4e6db73ef4ae862e1e96d38249883fcaf12b68f55ebb01035b34
Signed-off-by: Pasta <Pasta@dash.org>

31 -> 32

Signed-off-by: Pasta <Pasta@dash.org>

* Merge #10123: Allow debug logs to be excluded from specified component

3bde556 Add -debugexclude option to switch off logging for specified components (John Newbery)

Tree-SHA512: 30202e3f2085fc2fc5dd4bedb92988f4cb162c612a42cf8f6395a7da326f34975ddc347f82bc4ddca6c84c438dc0cc6e87869f90c7ff88105dbeaa52a947fa43

* bump to uint64_t due to added Dash codes

Signed-off-by: Pasta <Pasta@dash.org>

* bump to uint64_t due to added Dash codes cont.

Signed-off-by: Pasta <Pasta@dash.org>

* string -> BCLog format

Signed-off-by: Pasta <Pasta@dash.org>

* uint32_t -> uint64_t

Signed-off-by: Pasta <Pasta@dash.org>

* Fix CBatchedLogger

* Fix most fDebug-s

* Fix `debug` rpc

* Fix BENCH and RAND conflicts

* Add ALERT and use it

* Update LogPrint-s in dash-specific code

* Tweak few log categories

Specifically:
- use PRIVATESEND in `CPrivateSendClientManager::GetRandomNotUsedMasternode()`
- use ZMQ in `CZMQPublishRawGovernanceVoteNotifier::NotifyGovernanceVote()` and `CZMQPublishRawGovernanceObjectNotifier::NotifyGovernanceObject()`

* Drop no longer used MASTERNODE category

* Merge #10153: logging: Fix off-by-one for shrinkdebugfile default

faab624 logging: Fix off-by-one for shrinkdebugfile (MarcoFalke)

Tree-SHA512: d6153e06067906172ff0611af9e585a3ecf0a7d56925b6ad7c12e75aa802441047059b9b6f6c78e79916c3f2abc8f1998bfd2d5b84201ec6421f727c08da3c21

* Shift dash-specific log categories to start from `1ul << 32` to avoid potential future conflicts with bitcoin ones

* Fix `dash` category

* remove debugCategories

Signed-off-by: Pasta <Pasta@dash.org>

* Prepend "std::" to find call

* Check for BCLog::PRIVATESEND instead of logCategories != BCLog::NONE

* Use BCLog::MNPAYMENTS category instead of checking for logCategories != BCLog::NONE

* Move "End Dash" comment below "ALERT"

When adding new entries here, we'll otherwise get confused with ordering
and might end up forgetting that adding something Dash specific must
continue with the bit after 43.
2019-05-23 00:51:39 +03:00
Alexander Block
5652ea0237 Show number of InstantSend locks in Debug Console (#2919)
* Implement GetInstantSendLockCount in CInstantSendManager

* Add islockCountChanged signal to client model

* Show number of InstantSend locks in debug console
2019-05-22 09:35:51 +03:00
PastaPastaPasta
a6eee07f29 Merge bitcoin#9963: util: Properly handle errors during log message formatting (#2917)
* Merge #9963: util: Properly handle errors during log message formatting

b651270 util: Throw tinyformat::format_error on formatting error (Wladimir J. van der Laan)
3b092bd util: Properly handle errors during log message formatting (Wladimir J. van der Laan)

Tree-SHA512: 85e3b7afec2255fc88034187f1abd6060e9421de17ed4e3d918416f393429a99cc2c974b362099aaaff6970549df47664bea4c857c4e46acc0789663201dc541

* "cast" debugMsg to a c string

Signed-off-by: Pasta <Pasta@dash.org>

"cast" debugMsg to a c string pt 2

Signed-off-by: Pasta <Pasta@dash.org>
2019-05-21 15:32:33 +02:00
PastaPastaPasta
32aa229c7f Reorganize Dash Specific code into folders (#2753)
* rename instantx.* file to instantsend.*

* Reorganize Dash Specific code into folders

add to privatesend folder the header files

Signed-off-by: Pasta <Pasta@dash.org>

add "masternode" to imports

Signed-off-by: Pasta <Pasta@dash.org>

add "masternode" to imports

Signed-off-by: Pasta <Pasta@dash.org>

add "masternode" to imports pt 2

Signed-off-by: Pasta <Pasta@dash.org>
2019-05-21 15:26:15 +02:00
UdjinM6
84162021a3 Fix duplicate -instantsendnotify invocation (#2925) 2019-05-16 10:13:02 +02:00
Alexander Block
394bec4838 Skip processing in SyncTransaction when chain is not synced yet (#2920)
Applies to CInstantSendManager and CChainLocksHandler. This fixes extremely
high RAM usage on reindex and resync, which happens to many/all transactions
being kept track of non-locked TXs (nonLockedTxs) and TXs per
block (blockTxs).
2019-05-13 13:35:46 +03:00
Alexander Block
240238b190
Fix db leaks in LLMQ db (#2914)
* Store rs_t key time in big endian

Also implement ConvertInvalidTimeKeys to convert old entries. We can remove
this later when we know that most MNs have run this code on testnet.

The way we stored the time field in the past lead to CleanupOldRecoveredSigs
iterating the keys in a strange order, causing no deletion at all and the
LLMQ DB filling up.

* Write batch in CleanupOldRecoveredSigs when it gets too large

This avoids RAM filling up and OOM getting triggered.

* Keep track of when a vote was written to the DB and clean up after week

Instead of only deleting when the corresponding recovered sig is deleted.
It sometimes happens that a masternode votes on something but a recovered
sig is never created, which leaves us with a vote that will never be
deleted.

* Apply suggestions from code review

Co-Authored-By: PastaPastaPasta <pasta@dashboost.org>
2019-05-13 10:41:48 +02:00
Alexander Block
56f31195d2
Fall back to ReadBlockFromDisk when blockTxs is not filled yet (#2908)
* Fall back to ReadBlockFromDisk when blockTxs is not filled yet

This handles the case where a MN is freshly started and SyncTransaction
was not been called for older transactions/blocks.

* Also update txFirstSeenTime

* Properly handle txids == nullptr
2019-05-10 15:54:40 +02:00
Alexander Block
7696a6fc89
Print inputs on which we voted and quorums used for signing (#2907)
* Print inputs on which we voted

Also print the corresponding requestId. This makes debugging InstantSend
issues easier.

* Print quorum hash when signing a share

* Remove unused nodesByAddress map in CSigSharesManager::SendMessages

Not related to this PR, but a simple cleanup that should have no
side effects.
2019-05-10 11:24:04 +02:00
Alexander Block
a173e6836c Implement integration tests for DKG error handling (#2905)
* Allow modifying simulate DKG error rates via RPC

* Don't lie to yourself :)

* Add some missing new-lines in LogPrintf calls

* More fine grained control over which messages to expect in mine_quorum

* Implement llmq-dkgerrors.py integration tests

These test DKG errors and malicious behavior.
2019-05-08 12:13:27 +03:00
Alexander Block
66a2cdeafc Properly handle conflicts between ChainLocks and InstantSend (#2904)
* Move code to write archived ISLOCKs into its own method

We'll need this from another method as well later.

* Return ISLOCK instead of conflicting txid in GetConflictingTx/GetConflictingLock

* Implement GetInstantSendLocksByParent and RemoveChainedInstantSendLocks

These allow to easily delete multiple chains (actually trees) of ISLOCKs
in one go.

* Implement RemoveConflictedTx and call it from RemoveMempoolConflictsForLock

Also add "retryChildren" parameter to RemoveNonLockedTx so that we can
skip retrying of non-locked children TXs.

* Properly handle/remove conflicted TXs (between mempool and new blocks)

* Track non-locked TXs by inputs

* Implement and call ResolveBlockConflicts

* Also call ResolveBlockConflicts from ConnectBlock

But only when a block is known to have a conflict and at the same time is
ChainLocked, which causes the ISLOCK to be pruned.

* Split out RemoveChainLockConflictingLock from ResolveBlockConflicts

* Implement "quorum getrecsig" RPC

* Include decoded TX data in result of create_raw_tx

* Implement support for CLSIG in mininode.py

* Fix condition for update of nonLockedTxs.pindexMined

* Only add entries to nonLockedTxsByInputs when AddNonLockedTx is called for the first time

* Implement support for ISLOCK in mininode.py

* Implement tests for ChainLock vs InstantSend lock conflict resolution

* Handle review comment

Bail out (continue) early
2019-05-07 15:14:33 +03:00
Alexander Block
7fdc66dd86
Ask for locked TXs after removing conflicting TXs (#2898)
* Also test conflicts in mempool instead of only in blocks

* Ask for locked TXs after removing conflicting TXs

When we removed a conflicting TX from the mempool, the correct/locked TX
is not available locally as the first-seen rule would have filtered before.
We need to re-request this TX if any other node announced it before.

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>
2019-05-06 15:26:27 +02:00
Alexander Block
53827a3764 Remove code for QDEBUGSTATUS propagation (#2891)
* Remove code for QDEBUGSTATUS propagation

This turned out to be too expensive and could easily take the network
down by bringing all nodes to 100% CPU usage. Better to fully remove this
functionality.

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>

* Update src/rpc/rpcquorums.cpp

Co-Authored-By: codablock <ablock84@gmail.com>
2019-05-02 02:20:06 +03:00
UdjinM6
fbd244dded
Bail out in few more places when blockchain is not synced yet (#2888)
* Bail out in few more places when blockchain is not synced yet

* Apply review suggestion
2019-04-30 15:55:11 +03:00
UdjinM6
8ffdcbf999
A bunch of mostly trivial tweaks/fixes (#2889)
* Trivial: vout->txout

* Re-use SetHexStr in few more places

* Tweak log output

* fix v13 release notes links

* Drop no longer used stuff

* Few more trivial fixes

* Adjust few rpc help strings

* Apply review suggestions
2019-04-30 15:48:21 +03:00
UdjinM6
03021fa53c
Harden DIP3 activation (#2881)
* Harden DIP3 activation height

Also drop all related but no longer used parts.

* Pass current block index to GetCommitmentsFromBlock

* Allow to change dip3 activation height for tests

And fix them.
2019-04-25 18:39:04 +03:00
UdjinM6
5cfceab860
Refactor IS-lock GUI notification and implement a similar one for ChainLocks (#2875)
* Refactor IS-lock GUI notification and implement a similar one for ChainLocks

* Initialize cachedNumISLocks in TransactionStatus ctor
2019-04-25 18:37:39 +03:00
Alexander Block
22ae0bc212 Archive islock hashes when removing confirmed islocks (#2872)
This allows AlreadyHave to check if an announced (via INV) islock was
already known in the past. This avoids requesting islocks which got
obsolete due to ChainLocks.
2019-04-16 16:40:26 +03:00
Alexander Block
cd94cbe6f0 Track which TXs are not locked yet and use this info in ProcessPendingRetryLockTxs (#2869)
* Track which TXs are not locked yet and use this info in ProcessPendingRetryLockTxs

Instead of relying on ReadBlockFromDisk. This should be less disk+CPU
intensive but require more RAM.

It also fixes a bug in ProcessPendingRetryLockTxs which caused ChainLocked
parents to not be considered for retrying of its children.

* Handle review commments
2019-04-16 16:39:34 +03:00
UdjinM6
cff9f97179
Prefix all bls/quorum threads with dash- (#2865)
Makes it easier to find them. Note: had to s/quorum/q/ to fit into 15 characters.
2019-04-15 17:42:43 +03:00
UdjinM6
5e865f9c8a
Bump mempool counter on each successful IS lock (#2864) 2019-04-15 17:41:51 +03:00
Alexander Block
0c54e41f22 Retry locking of child TXs in batches instead of per locked parent (#2858)
This especially avoids many calls to ReadBlockFromDisk
2019-04-12 14:36:52 +03:00
Alexander Block
7fe1a4a78b Also invoke WriteInstantSendLockMined when IS lock comes after the mined block (#2861) 2019-04-11 23:11:56 +03:00
Alexander Block
f32f9523b9 Use lazy BLS signatures more often and don't always verify self-recovered sigs (#2860)
* Make CBLSLazySignature thread safe

* Perform malleability check in CBLSLazySignature

* Use CBLSLazySignature in CRecoveredSig and CInstantSendLock

* Only sporadically verify self-recovered signatures

* test
2019-04-11 23:11:39 +03:00
Alexander Block
5e8ae2ceb6 Disable optimistic send in PushMessage by default (#2859)
* Automatically wake up select() when optimistic send was not used

But only when we know that we are actually inside select() and that it
currenlty is unlikely for it to have selected the node's socket for
sending. We accept race conditions here as the select() timeout
will ensure that we always send the data.

* Don't manually call WakeSelect() in CSigSharesManager::SendMessages

Not needed anymore

* Disable optimistic send in PushMessage by default
2019-04-11 15:43:22 +03:00
Alexander Block
90b1b71967 Move processing of InstantSend locks into its own worker thread (#2857)
* Let ProcessPendingInstantSendLocks return true when it did some work

* Introduce own worker thread for CInstantSendManager

Instead of using the scheduler.

* Remove scheduler from CInstantSendManager

* Add missing reset() call for workInterrupt
2019-04-11 15:43:00 +03:00
Alexander Block
ae78360e5d Add cache to CQuorumManager::ScanQuorums (#2856) 2019-04-11 15:42:35 +03:00
Alexander Block
82a47f5432 Allow testing ChainLocks enforcement with spork19 == 1 (#2854)
This is only for testnet.
2019-04-11 15:41:51 +03:00
Alexander Block
225c3898d8
Don't skip "safe TX" check when ChainLocks are not enforced yet (#2851) 2019-04-10 18:16:33 +02:00
Alexander Block
b0260e970f Do not maintain CService in masternodeQuorumNodes
Not needed anymore as we retrieve the service/address of a masternode
on-the-fly in ThreadOpenMasternodeConnections.
2019-04-09 12:32:52 +02:00
Alexander Block
b5bc7c9dac Call HandleFullyConfirmedBlock when ChainLocks are enabled but not enforced (#2844)
Otherwise IS locks never get removed before DIP8 activates via BIP9.
2019-04-08 08:07:47 +03:00
Alexander Block
565754e019
Merge pull request #2839 from codablock/pr_llmq_scanquorumsfix
Fix invalid results from ScanQuorum
2019-04-05 14:39:17 +02:00
Alexander Block
f8bedba7c4
Don't retry locks when new IS system is disabled (#2837)
Retrying still failed, but this spammed logs quite a bit.
2019-04-05 14:38:58 +02:00
Alexander Block
206e5a1b47 Use big endian inversed height in CInstantSendDb 2019-04-05 13:40:18 +02:00
Alexander Block
4b9f6cd3a9 Use big endian inversed height in BuildInversedHeightKey
Otherwise keys are not properly sorted.
2019-04-05 13:39:29 +02:00
Alexander Block
d34ec78664 Update wallet transactions when confirmed IS locks are removed 2019-04-05 08:18:56 +02:00
Alexander Block
b897505f85 Remove the need for maintaining the last ChainLocked block in the DB 2019-04-05 08:05:12 +02:00
Alexander Block
8e7083cb89 Use db.RemoveConfirmedISLocks() in NotifyChainLock to remove confirmed locks
Also move the actual logic into HandleFullyConfirmedBlock and call it
from NotifyChainLock and UpdatedBlockTip.
2019-04-05 08:05:12 +02:00
Alexander Block
4577438e87 Implement RemoveConfirmedInstantSendLocks to prune confirmed IS locks from DB 2019-04-05 08:05:12 +02:00