Commit Graph

15988 Commits

Author SHA1 Message Date
Alexander Block
321bbf5af4 Fix excessive memory use when flushing chainstate and EvoDB (#3008)
* Specialize Serialize(CSizeComputer&) for BLS objects

Speeds up size computation for BLS objects.

* Track memory usage in CDBTransaction and CEvoDB

* Take memory used by CEvoDB/CDBTransaction into account when flushing

* Implement specialized SerReadWrite for immer maps

This allows to use READWRITE and fixes compilation errors with CSizeComputer

* Log EvoDB memory usage independently

Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>
2019-07-02 16:25:28 +02:00
Alexander Block
0410259dd5 Fix 2 common Travis failures which happen when Travis has network issues (#3003)
* Retry downloading of depends sources 3 times

* Manually invoke apt-get update and install instead of using addon

This allows us to use travis_retry on these operations.

* Add exception for depends/Makefile in .gitignore

* Remove --retry from curl call

This avoids future confusion as we don't rely on curl anymore to properly
retry downloads. curls does only retry on transient errors, while Travis
might cause all kinds of errors which circumvent the retry logic of curl.
2019-07-02 06:41:10 +02:00
Alexander Block
8d763c1440 Only load signingActiveQuorumCount + 1 quorums into cache (#3002)
No need to load 25 quorums when we're scanning for the larger quorums which
only have 4 active quorums. This avoids loading thousands of masternode
lists unnecessarily.
2019-07-02 06:40:47 +02:00
UdjinM6
2dc1b06ec7 Remove skipped denom from the list on tx commit (#2997)
* Remove skipped denom from the list on tx commit

Regardless of the tx type (PS or not).

* Update src/privatesend/privatesend-client.cpp

Co-Authored-By: Alexander Block <ablock84@gmail.com>
2019-07-02 06:40:28 +02:00
Alexander Block
dff2c851d5
Update manpages for 0.14.0.2 (#2999) 2019-06-26 06:03:05 +02:00
Alexander Block
d7d005270d
Merge pull request #2989 from codablock/pr_v14_backports
[v0.14.0.x] Backport pending PRs
2019-06-24 12:51:03 +02:00
Alexander Block
fb6f0e04df
Bump version to 0.14.0.2 and copy release notes (#2991) 2019-06-24 12:50:35 +02:00
UdjinM6
46c4f58441 Use Travis stages instead of custom timeouts (#2948) 2019-06-21 12:01:36 +02:00
Alexander Block
49c37b82ae Back off for 1m when connecting to quorum masternodes (#2975)
* Implement GetAddressInfo in CAddrMan

* Back off for 1m when connecting to quorum masternodes
2019-06-21 11:24:14 +02:00
Alexander Block
c1f756fd90 Multiple speed optimizations for deterministic MN list handling (#2972)
* Generalize CBLSLazyWrapper so that it can be used of pubkeys and secret keys

* Implement == and != operators for CBLSLazyWrapper

* Implement cached hash for CBLSLazyWrapper

* Use CBLSLazyPublicKey for CDeterministicMNState::pubKeyOperator

* Speed up GetProjectedMNPayees by sorting the MN list by last paid

Instead of updating a temporary list for each projected height and calling
GetMNPayee() on it.

* Cache intermediate lists in GetListForBlock

This avoids re-loading and applying diffs again and again.

* Only update masternode list UI max once every 3 seconds

This avoids updating the UI on every block, which turned out to be very
expensive.

* Fix compilation

* Drop time restrictions for mn list update in ClientModel

They are fully handled by MasternodeList now.
2019-06-21 09:47:13 +02:00
UdjinM6
11699f540f Process/keep messages/connections from PoSe-banned MNs (#2967)
* Process/keep/count votes from PoSe-banned MNs

* Process dstx from PoSe-banned MNs

* Recognize PoSe-banned MNs as MNs
2019-06-21 09:46:24 +02:00
UdjinM6
c5415e746a Fix UI masternode list (#2966)
NotifyMasternodeListChanged is called before the tip is updated which means we can't rely on GetListAtChainTip() and have to pass the list into CClientUIInterface
2019-06-21 09:45:50 +02:00
Alexander Block
05adda99fe Update release notes with latest commits (#2958) 2019-05-30 13:15:37 +03:00
Alexander Block
a2baa93ec5 Only require valid collaterals for votes and triggers (#2947) (#2957)
* Only require valid collaterals for votes and triggers

* Bump proto version

* Only announce votes from banned MNs to >= 70215 proto version nodes
2019-05-30 11:00:49 +03:00
Alexander Block
b293e6ddee [v0.14.0.x] Fix off-by-one error in InstantSend mining info removal when disconnecting blocks (#2951)
* Refactor checks for disconnected blocks and conflicts in CInstantSendManager::SyncTransaction

* Fix off-by-one when calling RemoveInstantSendLockMined
2019-05-29 18:34:26 +03:00
Alexander Block
276b6e3a82 [v0.14.0.x] bump version to 0.14.0.1 and prepare release notes (#2952)
* Bump version to 0.14.0.1

* Run gen-manpages.sh to update versions

* Prepare 0.14.0.1 release notes

Changelog is still missing and will be added later
2019-05-29 18:34:13 +03:00
UdjinM6
8c743968bd
Update release notes v14 (#2927) 2019-05-17 20:03:37 +03:00
UdjinM6
612a90e8c3
Set CLIENT_VERSION_IS_RELEASE to true (#2926) 2019-05-17 12:16:31 +03:00
UdjinM6
9482f77e57
Update help text via gen-manpages.sh (#2929) 2019-05-17 12:16:02 +03:00
UdjinM6
211a39d19b 0.14 release notes draft (#2896)
* move 0.13.3 releas notes

* 0.14 release notes draft

* apply suggestions

* Apply suggestions

* update with latest commits

* Add recent commits

* Add LLMQ-IS description
2019-05-16 10:13:47 +02:00
UdjinM6
84162021a3 Fix duplicate -instantsendnotify invocation (#2925) 2019-05-16 10:13:02 +02:00
Alexander Block
38aab0c5e3
Add blocks conflicting with ChainLocks to block index (#2923)
* Print hash of block which conflicts with a ChainLock

* Allow specifying custom block status in `AddToBlockIndex`

* Add blocks (headers) to block index even if they conflict with chainlocks
2019-05-16 10:12:36 +02:00
Alexander Block
394bec4838 Skip processing in SyncTransaction when chain is not synced yet (#2920)
Applies to CInstantSendManager and CChainLocksHandler. This fixes extremely
high RAM usage on reindex and resync, which happens to many/all transactions
being kept track of non-locked TXs (nonLockedTxs) and TXs per
block (blockTxs).
2019-05-13 13:35:46 +03:00
UdjinM6
c8a80b2792
Set DIP0008 mainnet activation params (#2915) 2019-05-13 13:35:03 +03:00
PastaPastaPasta
040abafe3a [0.14] Bump chainparams (#2910)
* Bump nMinimumChainWork and defaultAssumeValid for mainnet

Signed-off-by: Pasta <Pasta@dash.org>

add back 0x

Signed-off-by: Pasta <Pasta@dash.org>

* Bump nMinimumChainWork and defaultAssumeValid for testnet

Signed-off-by: Pasta <Pasta@dash.org>

add back 0x 2nd time

Signed-off-by: Pasta <Pasta@dash.org>

* Add a checkpoint for 0.14

Signed-off-by: Pasta <Pasta@dash.org>

* add testnet checkpoint 0.14

Signed-off-by: Pasta <Pasta@dash.org>

* update mainnet ChainTxData

Signed-off-by: Pasta <Pasta@dash.org>

formatting fix mainnet chaintxdata

Signed-off-by: Pasta <Pasta@dash.org>

update mainnet ChainTxData to specific block

Signed-off-by: Pasta <Pasta@dash.org>

* update testnet chainTxData

Signed-off-by: Pasta <Pasta@dash.org>

update testnet ChainTxData to specific block

Signed-off-by: Pasta <Pasta@dash.org>
2019-05-13 11:02:30 +02:00
Alexander Block
240238b190
Fix db leaks in LLMQ db (#2914)
* Store rs_t key time in big endian

Also implement ConvertInvalidTimeKeys to convert old entries. We can remove
this later when we know that most MNs have run this code on testnet.

The way we stored the time field in the past lead to CleanupOldRecoveredSigs
iterating the keys in a strange order, causing no deletion at all and the
LLMQ DB filling up.

* Write batch in CleanupOldRecoveredSigs when it gets too large

This avoids RAM filling up and OOM getting triggered.

* Keep track of when a vote was written to the DB and clean up after week

Instead of only deleting when the corresponding recovered sig is deleted.
It sometimes happens that a masternode votes on something but a recovered
sig is never created, which leaves us with a vote that will never be
deleted.

* Apply suggestions from code review

Co-Authored-By: PastaPastaPasta <pasta@dashboost.org>
2019-05-13 10:41:48 +02:00
Alexander Block
56f31195d2
Fall back to ReadBlockFromDisk when blockTxs is not filled yet (#2908)
* Fall back to ReadBlockFromDisk when blockTxs is not filled yet

This handles the case where a MN is freshly started and SyncTransaction
was not been called for older transactions/blocks.

* Also update txFirstSeenTime

* Properly handle txids == nullptr
2019-05-10 15:54:40 +02:00
Alexander Block
4dc5c0e9ed
Bump "keepOldConnections" by one for all LLMQ types (#2909)
And for llmq5_60, set it to 3 instead of 24 (which was incorrect from the
beginning).

Increasing all LLMQs "keepOldConnections" by one is required because we
don't use the tip when scanning active quorums, but a block that is 8 blocks
in the past. This however means that we need to keep connections for one
additional quorum alive.

This fixes an issue with signing sessions randomly failing with timeouts.
2019-05-10 11:26:39 +02:00
Alexander Block
7696a6fc89
Print inputs on which we voted and quorums used for signing (#2907)
* Print inputs on which we voted

Also print the corresponding requestId. This makes debugging InstantSend
issues easier.

* Print quorum hash when signing a share

* Remove unused nodesByAddress map in CSigSharesManager::SendMessages

Not related to this PR, but a simple cleanup that should have no
side effects.
2019-05-10 11:24:04 +02:00
Alexander Block
a173e6836c Implement integration tests for DKG error handling (#2905)
* Allow modifying simulate DKG error rates via RPC

* Don't lie to yourself :)

* Add some missing new-lines in LogPrintf calls

* More fine grained control over which messages to expect in mine_quorum

* Implement llmq-dkgerrors.py integration tests

These test DKG errors and malicious behavior.
2019-05-08 12:13:27 +03:00
UdjinM6
89f6f75910
Implement zmq notifications for chainlocked blocks (#2899)
* Unify zmq message order

* Implement `zmqpubhashchainlock` and `zmqpubrawchainlock`
2019-05-08 12:12:54 +03:00
Alexander Block
66a2cdeafc Properly handle conflicts between ChainLocks and InstantSend (#2904)
* Move code to write archived ISLOCKs into its own method

We'll need this from another method as well later.

* Return ISLOCK instead of conflicting txid in GetConflictingTx/GetConflictingLock

* Implement GetInstantSendLocksByParent and RemoveChainedInstantSendLocks

These allow to easily delete multiple chains (actually trees) of ISLOCKs
in one go.

* Implement RemoveConflictedTx and call it from RemoveMempoolConflictsForLock

Also add "retryChildren" parameter to RemoveNonLockedTx so that we can
skip retrying of non-locked children TXs.

* Properly handle/remove conflicted TXs (between mempool and new blocks)

* Track non-locked TXs by inputs

* Implement and call ResolveBlockConflicts

* Also call ResolveBlockConflicts from ConnectBlock

But only when a block is known to have a conflict and at the same time is
ChainLocked, which causes the ISLOCK to be pruned.

* Split out RemoveChainLockConflictingLock from ResolveBlockConflicts

* Implement "quorum getrecsig" RPC

* Include decoded TX data in result of create_raw_tx

* Implement support for CLSIG in mininode.py

* Fix condition for update of nonLockedTxs.pindexMined

* Only add entries to nonLockedTxsByInputs when AddNonLockedTx is called for the first time

* Implement support for ISLOCK in mininode.py

* Implement tests for ChainLock vs InstantSend lock conflict resolution

* Handle review comment

Bail out (continue) early
2019-05-07 15:14:33 +03:00
UdjinM6
b63f7dd39a
Fix a crash in masternode winners when count is too large (#2902) 2019-05-06 17:58:59 +03:00
UdjinM6
357b7279de
Implement isolate_node/reconnect_isolated_node in tests (#2901) 2019-05-06 17:58:38 +03:00
Alexander Block
7fdc66dd86
Ask for locked TXs after removing conflicting TXs (#2898)
* Also test conflicts in mempool instead of only in blocks

* Ask for locked TXs after removing conflicting TXs

When we removed a conflicting TX from the mempool, the correct/locked TX
is not available locally as the first-seen rule would have filtered before.
We need to re-request this TX if any other node announced it before.

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>
2019-05-06 15:26:27 +02:00
UdjinM6
5d05ab17a9
Fix PrivateSend log (#2892) 2019-05-02 02:20:36 +03:00
Alexander Block
53827a3764 Remove code for QDEBUGSTATUS propagation (#2891)
* Remove code for QDEBUGSTATUS propagation

This turned out to be too expensive and could easily take the network
down by bringing all nodes to 100% CPU usage. Better to fully remove this
functionality.

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>

* Update src/rpc/rpcquorums.cpp

Co-Authored-By: codablock <ablock84@gmail.com>
2019-05-02 02:20:06 +03:00
UdjinM6
783cb9ca69
Skip CheckCbTxMerkleRoots until assumeValid block (#2890) 2019-04-30 16:20:00 +03:00
UdjinM6
4dee7c4a25
Cache heavy parts of CalcCbTxMerkleRoot* (#2885) 2019-04-30 16:19:41 +03:00
UdjinM6
b3ed6410f4
Be more accurate with denom creation/consumption (#2853)
* Be more accurate with denom creation/consumption

- Calculate and pass an actual balance needed to be denominated to CreateDenominated.
- Drop GetNeedsToBeAnonymizedBalance and fix corresponding conditions: do not overshoot, do not check max pool amount - these conditions are handled outside.
- Properly calculate various balance limits.
- Handle edge case for the final denom.

* Add an option to control max number of denoms created and respect it

`-privatesenddenoms`, default is 300

Note: CreateDenominated failure is not an error anymore

* Add few more stats to log in DoAutomaticDenominating
2019-04-30 16:19:23 +03:00
UdjinM6
3d993ee8fa
Translations v14 (#2638)
* `make translate`

* ru

* ar, bg, de, es, fi, fr, it, ja, ko, nl, pl, pt, sk, th, tr, vi, zh_CN, zh_TW

* Update src/qt/locale/dash_ru.ts

Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>

* Update src/qt/locale/dash_ru.ts

* update sk

* update de

* update de
2019-04-30 16:18:37 +03:00
UdjinM6
fbd244dded
Bail out in few more places when blockchain is not synced yet (#2888)
* Bail out in few more places when blockchain is not synced yet

* Apply review suggestion
2019-04-30 15:55:11 +03:00
Daniel Hoffmann
fd6aaae7f0 Add proTxHash to masternode list rpc response (#2887) 2019-04-30 15:54:49 +03:00
UdjinM6
dd3977523a
More "bench" log for Dash specific parts (#2884) 2019-04-30 15:48:49 +03:00
UdjinM6
8ffdcbf999
A bunch of mostly trivial tweaks/fixes (#2889)
* Trivial: vout->txout

* Re-use SetHexStr in few more places

* Tweak log output

* fix v13 release notes links

* Drop no longer used stuff

* Few more trivial fixes

* Adjust few rpc help strings

* Apply review suggestions
2019-04-30 15:48:21 +03:00
UdjinM6
1951001614
New LLMQ-based IS should have no legacy IS strings in UI and no legacy restrictions (#2883)
(max value/too many inputs/additional fees)
2019-04-29 11:32:43 +03:00
UdjinM6
7f419ae7fc
Accept non-spent LLMQ IS locked outpoints from mempool in PS mixing (#2878) 2019-04-29 11:32:08 +03:00
Alexander Block
2652030a2b Use larger nPruneAfterSizeIn parameter for mapAlreadyAskedFor (#2882)
unordered_limitedmap was meant to be used with much larger
nPruneAfterSizeIn values to maintain good performance. The way it is not
will result in pruning too often on high load.
2019-04-25 18:39:18 +03:00
UdjinM6
03021fa53c
Harden DIP3 activation (#2881)
* Harden DIP3 activation height

Also drop all related but no longer used parts.

* Pass current block index to GetCommitmentsFromBlock

* Allow to change dip3 activation height for tests

And fix them.
2019-04-25 18:39:04 +03:00
UdjinM6
dcdb9dba12
Add NotifyMasternodeListChanged signal to CClientUIInterface (#2880)
And use it instead of BlockTipChanged for mn list changes.
2019-04-25 18:38:49 +03:00