Commit Graph

187 Commits

Author SHA1 Message Date
Alexander Block
66a2cdeafc Properly handle conflicts between ChainLocks and InstantSend (#2904)
* Move code to write archived ISLOCKs into its own method

We'll need this from another method as well later.

* Return ISLOCK instead of conflicting txid in GetConflictingTx/GetConflictingLock

* Implement GetInstantSendLocksByParent and RemoveChainedInstantSendLocks

These allow to easily delete multiple chains (actually trees) of ISLOCKs
in one go.

* Implement RemoveConflictedTx and call it from RemoveMempoolConflictsForLock

Also add "retryChildren" parameter to RemoveNonLockedTx so that we can
skip retrying of non-locked children TXs.

* Properly handle/remove conflicted TXs (between mempool and new blocks)

* Track non-locked TXs by inputs

* Implement and call ResolveBlockConflicts

* Also call ResolveBlockConflicts from ConnectBlock

But only when a block is known to have a conflict and at the same time is
ChainLocked, which causes the ISLOCK to be pruned.

* Split out RemoveChainLockConflictingLock from ResolveBlockConflicts

* Implement "quorum getrecsig" RPC

* Include decoded TX data in result of create_raw_tx

* Implement support for CLSIG in mininode.py

* Fix condition for update of nonLockedTxs.pindexMined

* Only add entries to nonLockedTxsByInputs when AddNonLockedTx is called for the first time

* Implement support for ISLOCK in mininode.py

* Implement tests for ChainLock vs InstantSend lock conflict resolution

* Handle review comment

Bail out (continue) early
2019-05-07 15:14:33 +03:00
UdjinM6
357b7279de
Implement isolate_node/reconnect_isolated_node in tests (#2901) 2019-05-06 17:58:38 +03:00
UdjinM6
03021fa53c
Harden DIP3 activation (#2881)
* Harden DIP3 activation height

Also drop all related but no longer used parts.

* Pass current block index to GetCommitmentsFromBlock

* Allow to change dip3 activation height for tests

And fix them.
2019-04-25 18:39:04 +03:00
Alexander Block
282cb697aa Use version 2 CCbTx in create_coinbase to fix test failures 2019-04-04 12:27:35 +02:00
Alexander Block
40ad06e775 Return the fresh quorum hash from mine_quorum 2019-04-04 12:27:35 +02:00
Alexander Block
17b9318a04 Mine SIGN_HEIGHT_OFFSET additional blocks after the quorum commitment
Otherwise the fresh quorum won't be eligable for signing sessions.
2019-04-04 12:27:35 +02:00
Alexander Block
5e832e2fa6 Implement support for CbTx version 2 (with quorum merkle root)
Also implement support for new fields in MNLISTDIFF.
2019-04-04 12:27:35 +02:00
Alexander Block
b1b41f02ae Fix a crash in mininode.py when inventory type is unknown 2019-04-04 12:27:35 +02:00
Alexander Block
ade5760a92 Allow registering MNs without actually starting them
This can be done by calling prepare_masternode() when inside run_test.
Also implement remove_masternode() which spends the collateral.
2019-04-04 10:39:15 +02:00
Alexander Block
ef6b6a1e64 Implement support for GETMNLISTD and MNLISTDIFF P2P message in mininode.py 2019-04-04 10:39:15 +02:00
Alexander Block
585b9c2818 Make CBlock.get_merkle_root static
We'll later need this method to calculate merkle roots outside of CBlock.
I'd like to avoid moving this code outside of CBlock as it might later
conflict with Bitcoin backports.
2019-04-04 10:39:15 +02:00
Alexander Block
1e0bdbc9b7 Implement CPartialMerkleTree and CMerkleBlock in mininode.py
Needed for upcoming MNLISTDIFF message support.
2019-04-04 10:39:15 +02:00
Alexander Block
d8778f5554 Implement CService in mininode.py 2019-04-04 08:24:38 +02:00
Alexander Block
02480402be Implement deser_dyn_bitset and ser_dyn_bitset in mininode.py 2019-04-04 08:24:33 +02:00
Alexander Block
5299d39338 Multiple refactorings/fixes for LLMQ bases InstantSend and ChainLocks (#2779)
* Remove unused parameters from CInstantSendManager::ProcessTx

* Pass txHash in CheckCanLock by reference instead of pointer

* Dont' allow locking of TXs without inputs

* Remove unused local variable nInstantSendConfirmationsRequired

* Don't subtract 1 from nInstantSendConfirmationsRequired

This was necessary in the old system but is not necessary in the new system.
It also prevented proper retroactive signing of chained TXs in regtest as
it resulted in child TXs to return true immediately for CheckCanLock when
it should actually have waited for the parent TX to become locked first.

* Access chainActive.Height() while cs_main is locked

* Properly read and write lastChainLockBlock

"pindex" is NOT the chainlocked block after the while loop finishes. We
must use the pindex (renamed to pindexChainLock now) given on method entry.

Also, the GetLastChainLockBlock() result was not assigned to,
lastChainLockBlock which resulted in the while loop to run unnecessarily
long.

* Generalize filtering in NewPoWValidBlock and SyncTransaction

We're actually interested in all TXs that have inputs, so no need to
explicitly check for tx types.

* Use tx.IsCoinBase() instead of checking for index 0

* Handle cases where a TX is not received yet in wait_for_instantlock

* Wait on all nodes for the locks

Otherwise we end up with the sender having it locked but other nodes
not yet, failing the test.

* Fix LogPrintf call in CChainLocksHandler::DoInvalidateBlock
2019-03-19 10:38:16 +03:00
UdjinM6
e52763d21b Refactor and fix instantsend tests/utils (#2776)
* Unify autoIS/send_smth functions

* Rename autoix-mempool.py -> autois-mempool.py

* Make sure create_raw_trx produces expected results

* Make sure sender has enough inputs and nodes are synced before starting the actual test

* Mine one block to clean mempool up

* 2 blocks is enough for IS on regtest

This also unifies it across different IS tests

* Allow wait_for_instantlock to be called on any node, not only on the one that has the tx in the wallet

* No need to query for tx this often in wait_for_instantlock

* Rename create_raw_trx -> create_raw_tx

* Fund sender with a single TX instead of 30
2019-03-18 14:50:44 +01:00
Alexander Block
a69a5cf4a0 Use smaller (3 out of 5) quorums for regtest/Travis (#2774)
* Require only 3 out of 5 signatures for old InstantSend in regtest mode

* Use LLMQs of size 5 with threshold of 3 for regtest

* Fix wrong check for out-of-range bits in CFixedBitSet

* Reduce number of masternodes in masternode/LLMQ tests

* Add missing \n to LogPrintf call

* Use correct indexes for isolated/receiver/sender nodes

The way it was before resulted in nodes 1-3 being unused and 6-8 being used
for these 3 special nodes even though these are masternodes.

* Avoid stopping/starting isolated node in p2p-instantsend.py

It's enough to disable networking for this node.
2019-03-15 11:48:24 +03:00
PastaPastaPasta
6f90cf7a17 Merge bitcoin#9602: Remove coin age priority and free transactions - implementation (#2768)
b421e6d Update example bitcoin.conf (Alex Morcos)
7d4e950 Allow setting minrelaytxfee to 0 (Alex Morcos)
359e8a0 [cleanup] Remove coin age priority completely. (Alex Morcos)
f9b9371 [rpc] Remove priorityDelta from prioritisetransaction (Alex Morcos)
49be7e1 [rpc] Remove priority information from mempool RPC calls (Alex Morcos)
0315888 [test] Remove priority from tests (Alex Morcos)
f838005 No longer allow "free" transactions (Alex Morcos)
ad727f4 [rpc] sendrawtransaction no longer bypasses minRelayTxFee (Alex Morcos)
fe282ac [cleanup] Remove estimatePriority and estimateSmartPriority (Alex Morcos)
400b151 [debug] Change -printpriority option (Alex Morcos)
272b25a [mining] Remove -blockprioritysize. (Alex Morcos)
12839cd [rpc] Remove estimatepriority and estimatesmartpriority. (Alex Morcos)
ddf58c7 wallet: Remove sendfree (MarcoFalke)

Tree-SHA512: a9a4499405923ce794ef18f9e334dbbd59dfc73a3dc2df6f85cc9c62af6f353ec2eed9c2d5e58e904f918d0d7ab738f403dd4939d9bc2276136864fe63710782
Signed-off-by: Pasta <Pasta@dash.org>

Fix backport and fix dash specific priority code
2019-03-14 17:44:42 +03:00
Wladimir J. van der Laan
d1a6022605 Merge #11113: [net] Ignore getheaders requests for very old side blocks
eff4bd8 [test] P2P functional test for certain fingerprinting protections (Jim Posen)
a2be3b6 [net] Ignore getheaders requests for very old side blocks (Jim Posen)

Pull request description:

  Sending a getheaders message with an empty locator and a stop hash is a request for a single header by hash. The node will respond with headers for blocks not in the main chain as well as those in the main chain. To avoid fingerprinting, the node should, however, ignore requests for headers on side branches that are too old. This replicates the logic that currently exists for `getdata` requests for blocks.

Tree-SHA512: e04ef61e2b73945be6ec5977b3c5680b6dc3667246f8bfb67afae1ecaba900c0b49b18bbbb74869f7a37ef70b6ed99e78ebe0ea0a1569369fad9e447d720ffc4
2019-03-12 10:43:47 +01:00
Alexander Block
cd0f94fb54 Give wait_for_quorum_phase more time 2019-03-12 09:20:17 +01:00
Alexander Block
6488135f46 Track index into self.nodes in mninfo 2019-03-12 07:02:52 +01:00
UdjinM6
9f40afce86
Merge pull request #2727 from PastaPastaPasta/backports-0.15-pr7
Backports 0.15 pr7
2019-03-11 11:15:51 +03:00
Pasta
609114a803 Code review: re-add string cast in mininode.py 2019-03-08 17:05:10 -06:00
Alexander Block
0e91ebcf42 Use logging framework in Dash specific tests 2019-03-08 14:22:40 -06:00
Alexander Block
f8f867a6bb Sync blocks after generating in mine_quorum() 2019-03-07 21:15:09 +01:00
Pasta
0db2d1596a
code review and reset file perms 2019-03-03 20:50:39 -06:00
MarcoFalke
dad8c67d38 Merge #9768: [qa] Add logging to test_framework.py
64c0800 Use logging in individual tests (John Newbery)
38ad281 Use logging in test_framework/comptool.py (John Newbery)
ff19073 Use logging in test_framework/blockstore.py (John Newbery)
2a9c7c7 Use logging in test_framework/util.py (John Newbery)
b0dec4a Remove manual debug settings in qa tests. (John Newbery)
af1363c Always enable debug log and microsecond logging for test nodes. (John Newbery)
6d0e325 Use logging in mininode.py (John Newbery)
553a976 Add logging to p2p-segwit.py (John Newbery)
0e6d23d Add logging to test_framework.py (John Newbery)

Tree-SHA512: 42ee2acbf444ec32d796f930f9f6e272da03c75e93d974a126d4ea9b2dbaa77cc57ab5e63ce3fd33d609049d884eb8d9f65272c08922d10f8db69d4a60ad05a3
2019-02-26 15:54:32 -06:00
Alexander Block
48d92f116e Implement optional pretty printed stacktraces (#2420)
* Add libbacktrace to depends

This is currently only useful to extract symbols. It fails to gather
stacktraces when compiled with MinGW, so we can only use it to get symbol
information from a stack trace which we gathered outside of libbacktrace.

* Add -mbig-obj to CXXFLAGS for MinGW builds

* Implement stacktraces for C++ exceptions

This is a hack and should only be used for debugging. It works by wrapping
the C++ ABI __wrap___cxa_allocate_exception. The wrapper records a backtrace
and stores it in a global map. Later the stacktrace can be retrieved with
GetExceptionStacktraceStr.

This commit also adds handlers to pretty print uncaught exceptions and
signals.

* Use GetPrettyExceptionStr for all unhandled exceptions

* Use --enable-stacktraces in CI for linux32/linux64

* Register exception translators to pretty print exceptions in unit tests

* Catch and print python exceptions when stopping nodes

Otherwise the code at the bottom is never executed when nodes crash,
leading to no output of debug.log files on Travis.

* Remove now unneeded/unused TestCrash methods
2019-02-21 21:37:15 +03:00
UdjinM6
a79b928532
Merge pull request #2682 from PastaPastaPasta/backports-0.15-pr6
Backports 0.15 pr6
2019-02-19 13:04:49 +03:00
UdjinM6
86fc050495
Drop no longer used code and bump min protos (#2697)
* Drop registry deletion of the old key

* Drop no longer used CGovernanceObjectVoteFile::RemoveOldVotes()

* Drop temporary disconnect code and bump min protos to 70213

* drop comment

* fix
2019-02-12 22:51:21 +03:00
Wladimir J. van der Laan
22cda1a92f Merge #9929: tests: Delete unused function _rpchost_to_args
99fecf8 tests: Delete unused function _rpchost_to_args (Wladimir J. van der Laan)

Tree-SHA512: 40911d048d3fd7b3ce83e9b3caf2a409d55b47cbe08ea4450a16ca72264300bb12d5ef7dbcf335885975119b5977f949e6879546840064138fb506e24494d849
2019-02-04 19:58:09 -06:00
UdjinM6
d343bfa520
Merge pull request #2647 from PastaPastaPasta/backports-0.15-pr5
Backports 0.15 pr5
2019-02-04 21:58:02 +03:00
Pasta
071035b93a
Apply code review suggestions #2647 2019-02-01 13:39:23 -06:00
Wladimir J. van der Laan
56890f98f2
Merge #9832: [qa] assert_start_raises_init_error
025dec0 [qa] assert_start_raises_init_error (NicolasDorier)

Tree-SHA512: 0fe3ecbd47625b181aed92f15445ac26993e1a8b9843bbc1088c4adcea774e503b870912a18e13dca3f255c22a9964c1c0ca92c758907538143f316c5272ea4a
2019-02-01 10:50:53 -06:00
Alexander Block
18950f923e Optimize DKG debug message processing for performance and lower bandwidth (#2672)
* Allow sub-batch verification in CBLSInsecureBatchVerifier

* Implement batch verification of CDKGDebugStatus messages

* Use uint8_t for statusBitset in CDKGDebugMemberStatus and CDKGDebugSessionStatus

No need to waste one byte per member and per LLMQ type.

* Reserve 4k of buffer for CSerializedNetMsg buffer

Profiling has shown that a lot of time is spent in resizing the data
vector when large messages are involved.

* Remove nHeight from CDKGDebugStatus

This field changes every block and causes all masternodes to propagate
its status for every block, even if nothing DKG related has changed.

* Leave out session statuses when we're not a member of that session

Otherwise MNs which are not members of DKG sessions will spam the network

* Remove receivedFinalCommitment from CDKGDebugSessionStatus

This is not bound to a session and thus is prone to spam the network when
final commitments are propagated in the finalization phase.

* Add "minableCommitments" to "quorum dkgstatus"

* Hold cs_main while calling GetMinableCommitment

* Abort processing of pending debug messages when spork18 gets disabled

* Don't ask for debug messages when we've already seen them

"statuses" only contains the current messages but none of the old messages,
so nodes kept re-requesting old messages.
2019-02-01 10:49:01 +03:00
UdjinM6
25cb14b615 Fix confusion between dip3 activation and enforcement (#2651)
* Fix confusion between dip3 activation and enforcement

* drop DIP0003ActivationHeight for now
2019-01-29 15:54:38 +01:00
Alexander Block
7ee31cbd65 Speed up integration tests with masternodes (#2642)
* Implement copy_datadir to allow easy copying of state from one node to another

* Instead of starting with a fresh datadir for MNs, reuse a copy of the faucet

* Start masternodes in parallel instead of waiting for the previous to finish

* Allow specifying of window and threshold with -bip9params

* Implement -dip3activationheight for regtests

* Implement fast DIP3 activation in DashTestFramework

* Speed up activation of DIP3 in dip3-deterministicmns.py

* Update qa/rpc-tests/test_framework/test_framework.py

Co-Authored-By: codablock <ablock84@gmail.com>

* Always assign fast_dip3_activation
2019-01-23 19:36:51 +03:00
UdjinM6
fda16f1fea
Fix off-by-1 in phase calculations and the rest of llmq-signing.py issues (#2641)
* Fix off-by-1 in phase calculations

* Fix wait_for_quorum_phase, should look for check_received_messages

* Fix wait_for_quorum_phase for complain phase

* Bump default timeout in wait_for_quorum_phase/wait_for_quorum_commitment to 15
2019-01-23 17:02:56 +03:00
Alexander Block
b595f9e6a1 Fix LLMQ signing integration tests (#2640)
* Fix cleanup of old recovered sigs

When iterating the db, we should also include entries that match exactly
the end time.

* Fix key not found error

* Raise AssertionError in case wait_for_quorum_phase/wait_for_quorum_commitment time out
2019-01-23 11:37:02 +03:00
Alexander Block
4d25148c03 Add llmq-signing.py tests 2019-01-21 14:42:08 +01:00
Alexander Block
d020ffa00d Add wait_for_sporks_same and mine_quorum to DashTestFramework 2019-01-21 14:42:08 +01:00
PastaPastaPasta
07dcddb4ca Backports 0.15 pr2 (#2597)
* Merge #9815: Trivial: use EXIT_ codes instead of magic numbers

a87d02a use EXIT_ codes instead of magic numbers (Marko Bencun)

* Merge #9801: Removed redundant parameter from mempool.PrioritiseTransaction

eaea2bb Removed redundant parameter from mempool.PrioritiseTransaction (gubatron)

* remove extra parameter (see 3a3745bb) in dash specific code

* Merge #9819: Remove harmless read of unusued priority estimates

bc8fd12 Remove harmless read of unusued priority estimates (Alex Morcos)

* Merge #9766: Add --exclude option to rpc-tests.py

c578408 Add exclude option to rpc-tests.py (John Newbery)

* Merge #9577: Fix docstrings in qa tests

3f95a80 Fix docstrings in qa tests (John Newbery)

* Merge #9823: qa: Set correct path for binaries in rpc tests

3333ad0 qa: Set correct path for binaries in rpc tests (MarcoFalke)

* Merge #9833: Trivial: fix comments referencing AppInit2

ef9f495 Trivial: fix comments referencing AppInit2 (Marko Bencun)

* Merge #9612: [trivial] Rephrase the definition of difficulty.

dc222f8 Trivial: Rephrase the definition of difficulty in the code. (Karl-Johan Alm)

* Merge #9847: Extra test vector for BIP32

30aedcb BIP32 extra test vector (Pieter Wuille)

* Merge #9839: [qa] Make import-rescan.py watchonly check reliable

864890a [qa] Make import-rescan.py watchonly check reliable (Russell Yanofsky)

Tree-SHA512: ea0e2b1d4fc8f35174c3d575fb751b428daf6ad3aa944fad4e3ddcc9195e4f17051473acabc54203b1d27cca64cf911b737ab92e986c40ef384410652e2dbea1

* Change back file params
2019-01-07 12:55:35 +03:00
Alexander Block
0648496e21 Fix flaky p2p-fullblocktest (#2605)
* Announce blocks as HEADERS instead of INV in comptool

Announcing as INV results in one GETHEADERS message per INV item with
every GETHEADERS containing the same block locator info. This in turn
results in many (1088 in p2p-fullblocktests) replies with each having i+1
headers. As every message is sent in one go, the message processing queue
gets overloaded, leading to GETDATA timeouts for blocks and thus failing
the tests.

Announcing the blocks directly with HEADERS avoids all this and results in
one GETDATA per announced header.

This will later conflict with backported improvements from Bitcoin. When we
get to this point, we can simply ignore changes on Dash's side and take
the improvements from Bitcoin (which also switch to header announcement,
but after some refactorings)

* Give wait_for_pings more time

Large reorgs as in the p2p-fullblocktest can cause very long long delays
for the ping that we're waiting for. This was not a problem before #2593,
as before this wait_until was waiting forever.
2019-01-03 12:21:03 +03:00
UdjinM6
c58f775cc2
De-dashify env vars and dashify help text in tests instead (#2603) 2019-01-03 12:19:46 +03:00
MarcoFalke
75024e1173 Merge #10365: [tests] increase timeouts in sendheaders test
56befa0 [tests] increase timeouts in sendheaders test (John Newbery)
b0bfa23 [tests] Make wait_until timeout 60 seconds by default (John Newbery)

Tree-SHA512: ca68ad91a7cabb0673f230631a9f2196a4a394cf7cea7a266b3dad3923f7887d2a192ceedd68f7a4a06d105a7d13870e75e971193b93476f8d72b34639d7c3d1
2018-12-31 09:38:08 +01:00
Alexander Block
82745dd076
Use DIP3 MNs in auto-IX tests (#2588)
* Speed up autoix-mempool tests by making the max mempool smaller

With this less TXs have to be created to fill up the mempool.

* Change DashTestFramework to use DIP3 MNs

* Don't assert disabled bip9 in p2p-autoinstantsend.py
2018-12-31 08:12:36 +01:00
Alexander Block
ce5a511349 Speed up stopping of nodes in tests by splitting up stop and wait (#2587)
Especially speed up MN based tests which currently spend multiple seconds
waiting for nodes to stop.
2018-12-28 19:14:27 +03:00
Alexander Block
c60687fe61 Sleep longer between attempts in sync_blocks 2018-11-15 15:25:04 +01:00
Alexander Block
91af72b18f Allow to specify how log to sleep between attempts in wait_until 2018-11-15 09:44:37 +01:00
UdjinM6
bea590958b
Fix auto-IS tests (#2414)
* no `-debug` by default (it's too heavy)

* drop useless `sync_masternodes` calls and add a couple where they could be actually usefull

* change autoix-mempool.py test logic:
1. use separate nodes to send test txes
2. sync nodes while filling the mempool
2018-11-05 12:28:04 +03:00