Commit Graph

800 Commits

Author SHA1 Message Date
Alexander Block
401da32090 More fixes in llmq-is-retroactive tests 2019-12-07 00:35:27 +01:00
Alexander Block
33721eaa11 Make llmq-is-retroactive test compatible with 0.14.0.x 2019-12-07 00:35:27 +01:00
Alexander Block
85bd162a3e Make wait_for_xxx methods compatible with 0.14.0.x
Mostly related to wait_until not asserting in 0.14.0.x
2019-12-07 00:35:27 +01:00
Alexander Block
22cfddaf12 Allow re-signing of IS locks when performing retroactive signing (#3219)
* Implement re-signing of InstantSend inputs when TXs come in via blocks

* Use GetAdjustedTime instead of GetTimeMillis in CSigSharesManager

This allows use of mocktime in tests.

* Expose verifiedProRegTxHash in getpeerinfo and implement wait_for_mnauth

* Allow to wait for IS and CL to NOT happen

* Bump timeout for wait_for_instantlock

* Implement tests for retroactive signing of IS and CLs

* Add wait_for_tx function to DashTestFramework

* Add -whitelist=127.0.0.1 to node0

* Use node3 for isolated block generation

* Don't test for non-receival of TXs on node4/node5
2019-12-07 00:35:27 +01:00
Alexander Block
a8b8891a1d Add wait_for_xxx methods as found in develop
But slightly modified so that they work with wait_until which does not
assert in v0.14.0.x
2019-12-06 22:47:57 +01:00
UdjinM6
fdd19cf667 Tests: Fix the way nodes are connected to each other in setup_network/start_masternodes (#3221)
* Tests: Connect to the control node only in start_masternodes

Masternodes should take care of intra-quorum connections themselves

* Reconnect non-masternodes back to the control node
2019-12-06 22:47:57 +01:00
Alexander Block
41f0e9d028 More fixes related to extra_args 2019-12-06 22:47:57 +01:00
UdjinM6
5213118601 Tests: Allow specifying different cmd-line params for each masternode (#3222) 2019-12-06 22:47:57 +01:00
Alexander Block
c0dda38fea Circumvent BIP69 sorting in fundrawtransaction.py test (#3100)
"subtractFeeFromOutputs" is applied to the ordering of the input
transaction and after that BIP69 sorting is performed. This causes
flakiness in tests.
2019-11-19 12:25:23 +01:00
Nathan Marley
16b6b6f7c2 Update activemn if protx info changed (#3176)
* Update activemn if protx info changed

* Add `==` and `!=` operators to CDeterministicMNState

* Only re-init active MN if its IP changed, changes to payout, voting etc. can be done without it

* Test `masternode status` updates

* Don't track mnListEntry anymore and instead get the DMN on demand

* Revert "Add `==` and `!=` operators to CDeterministicMNState"

This reverts commit fba4687581.
2019-11-19 12:25:23 +01:00
Alexander Block
f2d401aa85 Make orphan TX map limiting dependent on total TX size instead of TX count (#3121) 2019-11-19 12:25:23 +01:00
UdjinM6
2141d5f9d9 Fix bip69 vs change position issue (#3063)
* Fix bip69 vs change position issue

* Drop `setbip69enabled` rpc
2019-11-18 15:29:49 +01:00
UdjinM6
c74f2cd8b2 Fix SelectCoinsMinConf to allow instant respends (#3061)
* Modify tests to check for instant respends

This should fail atm...

* Fix SelectCoinsMinConf to allow instant respends

Now tests should pass again.
2019-11-18 15:28:19 +01:00
Alexander Block
2e7ec23693
[0.14.0.x] Remove check for mempool size in CInstantSendManager::CheckCanLock (#3119)
* Remove check for mempool size in CInstantSendManager::CheckCanLock

This should not have been here at all and is already removed in develop.
Recent InstantSend failures on mainnet were partly related to this check.

* Skip autois tests for new IS system
2019-09-30 09:56:25 +02:00
Alexander Block
a173e6836c Implement integration tests for DKG error handling (#2905)
* Allow modifying simulate DKG error rates via RPC

* Don't lie to yourself :)

* Add some missing new-lines in LogPrintf calls

* More fine grained control over which messages to expect in mine_quorum

* Implement llmq-dkgerrors.py integration tests

These test DKG errors and malicious behavior.
2019-05-08 12:13:27 +03:00
Alexander Block
66a2cdeafc Properly handle conflicts between ChainLocks and InstantSend (#2904)
* Move code to write archived ISLOCKs into its own method

We'll need this from another method as well later.

* Return ISLOCK instead of conflicting txid in GetConflictingTx/GetConflictingLock

* Implement GetInstantSendLocksByParent and RemoveChainedInstantSendLocks

These allow to easily delete multiple chains (actually trees) of ISLOCKs
in one go.

* Implement RemoveConflictedTx and call it from RemoveMempoolConflictsForLock

Also add "retryChildren" parameter to RemoveNonLockedTx so that we can
skip retrying of non-locked children TXs.

* Properly handle/remove conflicted TXs (between mempool and new blocks)

* Track non-locked TXs by inputs

* Implement and call ResolveBlockConflicts

* Also call ResolveBlockConflicts from ConnectBlock

But only when a block is known to have a conflict and at the same time is
ChainLocked, which causes the ISLOCK to be pruned.

* Split out RemoveChainLockConflictingLock from ResolveBlockConflicts

* Implement "quorum getrecsig" RPC

* Include decoded TX data in result of create_raw_tx

* Implement support for CLSIG in mininode.py

* Fix condition for update of nonLockedTxs.pindexMined

* Only add entries to nonLockedTxsByInputs when AddNonLockedTx is called for the first time

* Implement support for ISLOCK in mininode.py

* Implement tests for ChainLock vs InstantSend lock conflict resolution

* Handle review comment

Bail out (continue) early
2019-05-07 15:14:33 +03:00
UdjinM6
357b7279de
Implement isolate_node/reconnect_isolated_node in tests (#2901) 2019-05-06 17:58:38 +03:00
Alexander Block
7fdc66dd86
Ask for locked TXs after removing conflicting TXs (#2898)
* Also test conflicts in mempool instead of only in blocks

* Ask for locked TXs after removing conflicting TXs

When we removed a conflicting TX from the mempool, the correct/locked TX
is not available locally as the first-seen rule would have filtered before.
We need to re-request this TX if any other node announced it before.

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>
2019-05-06 15:26:27 +02:00
UdjinM6
8ffdcbf999
A bunch of mostly trivial tweaks/fixes (#2889)
* Trivial: vout->txout

* Re-use SetHexStr in few more places

* Tweak log output

* fix v13 release notes links

* Drop no longer used stuff

* Few more trivial fixes

* Adjust few rpc help strings

* Apply review suggestions
2019-04-30 15:48:21 +03:00
UdjinM6
03021fa53c
Harden DIP3 activation (#2881)
* Harden DIP3 activation height

Also drop all related but no longer used parts.

* Pass current block index to GetCommitmentsFromBlock

* Allow to change dip3 activation height for tests

And fix them.
2019-04-25 18:39:04 +03:00
Alexander Block
1d8c7226d0 Fix race condition in sendheaders.py (#2862)
generate() will push INV messages to all nodes, including our test_node.
The original check_last_announcement() call done here will then
sporadically return Fals when the INV message is received shortly after
clear_last_announcement()

Solution is to check for the INV announcement first and then continue with
the test.
2019-04-12 14:37:14 +03:00
UdjinM6
1a25c2084c Apply suggestions from code review
Co-Authored-By: codablock <ablock84@gmail.com>
2019-04-04 17:41:30 +02:00
Alexander Block
282cb697aa Use version 2 CCbTx in create_coinbase to fix test failures 2019-04-04 12:27:35 +02:00
Alexander Block
adc101a11a Implement quorum commitment merkle root tests in dip4-coinbasemerkleroots.py 2019-04-04 12:27:35 +02:00
Alexander Block
40ad06e775 Return the fresh quorum hash from mine_quorum 2019-04-04 12:27:35 +02:00
Alexander Block
17b9318a04 Mine SIGN_HEIGHT_OFFSET additional blocks after the quorum commitment
Otherwise the fresh quorum won't be eligable for signing sessions.
2019-04-04 12:27:35 +02:00
Alexander Block
5e832e2fa6 Implement support for CbTx version 2 (with quorum merkle root)
Also implement support for new fields in MNLISTDIFF.
2019-04-04 12:27:35 +02:00
Alexander Block
b1b41f02ae Fix a crash in mininode.py when inventory type is unknown 2019-04-04 12:27:35 +02:00
UdjinM6
c23dfaf573
Update qa/rpc-tests/dip4-coinbasemerkleroots.py
Co-Authored-By: codablock <ablock84@gmail.com>
2019-04-04 11:46:52 +02:00
Alexander Block
9f2e5d0852 Use FromHex to deserialize block header 2019-04-04 10:42:13 +02:00
Alexander Block
999848432a Implement dip4-coinbasemerkleroots.py integration tests 2019-04-04 10:39:15 +02:00
Alexander Block
ade5760a92 Allow registering MNs without actually starting them
This can be done by calling prepare_masternode() when inside run_test.
Also implement remove_masternode() which spends the collateral.
2019-04-04 10:39:15 +02:00
Alexander Block
ef6b6a1e64 Implement support for GETMNLISTD and MNLISTDIFF P2P message in mininode.py 2019-04-04 10:39:15 +02:00
Alexander Block
585b9c2818 Make CBlock.get_merkle_root static
We'll later need this method to calculate merkle roots outside of CBlock.
I'd like to avoid moving this code outside of CBlock as it might later
conflict with Bitcoin backports.
2019-04-04 10:39:15 +02:00
Alexander Block
1e0bdbc9b7 Implement CPartialMerkleTree and CMerkleBlock in mininode.py
Needed for upcoming MNLISTDIFF message support.
2019-04-04 10:39:15 +02:00
Alexander Block
d8778f5554 Implement CService in mininode.py 2019-04-04 08:24:38 +02:00
Alexander Block
02480402be Implement deser_dyn_bitset and ser_dyn_bitset in mininode.py 2019-04-04 08:24:33 +02:00
Alexander Block
7b76e7abb2
Implement BIP9 style deployment for DIP8/ChainLocks and fix a bug with late headers (#2793)
* Also update bestChainLockWithKnownBlock in AcceptedBlockHeader

* Implement BIP9 style DIP8 deployment

* Fix ChainLocks tests

* Apply suggestions from code review

Co-Authored-By: codablock <ablock84@gmail.com>
2019-03-22 11:51:50 +01:00
Alexander Block
c3602372cc Implement retroactive IS locking of transactions first seen in blocks instead of mempool (#2770)
* Don't rely on UTXO set in CheckCanLock

The UTXO set only works for TXs in the mempool and won't work when we try
to retroactively lock unlocked TXs from blocks.

This is safe as ProcessTx is only called when a TX was accepted into the
mempool or connected in a block, which means that all input checks were
good.

* Rename RetryLockMempoolTxs to RetryLockTxs and let it retry connected TXs

* Instead of manually calling ProcessTx, let SyncTransaction handle all cases

SyncTransaction is called from AcceptToMemoryPool and when transactions got
connected in a block. So this is the time we want to run TXs through
ProcessTx. This also enables retroactive signing of TXs that were unknown
before a new block appeared.

* Test retroactive signing and safe TXs in LLMQ ChainLocks tests

* Also test for retroactive signing of chained TXs

* Honor lockedParentTx when looking for TXs to retry signing

* Stop scanning for TXs to retry after a depth of 6

* Generate 6 block to avoid retroactive signing overloading Travis

* Avoid retroactive signing

* Don't rely on NewPoWValidBlock and use SyncTransaction to build blockTxs

NewPoWValidBlock is not guaranteed to be called when blocks come in fast.
When a block is accepted in AcceptBlock, NewPoWValidBlock is only called
when the new block is a successor of the currently active tip. This is not
the case when after the first block a second block is accepted immediately
as the first block is not connected yet.

This might be a bug actually in the handling of NewPoWValidBlock, so we
might need to check/fix this later, but currently I prefer to not touch
that part.

Instead, we now use SyncTransaction to gather TXs for blockTxs. This works
because SyncTransaction is called for all transactions in a freshly
connected block in one go. The call also happens before UpdatedBlockTip is
called, so it's fine with the existing logic.

* Use tx.IsCoinBase() instead of checking index 0

Also check for empty vin.
2019-03-19 13:55:51 +03:00
Alexander Block
5299d39338 Multiple refactorings/fixes for LLMQ bases InstantSend and ChainLocks (#2779)
* Remove unused parameters from CInstantSendManager::ProcessTx

* Pass txHash in CheckCanLock by reference instead of pointer

* Dont' allow locking of TXs without inputs

* Remove unused local variable nInstantSendConfirmationsRequired

* Don't subtract 1 from nInstantSendConfirmationsRequired

This was necessary in the old system but is not necessary in the new system.
It also prevented proper retroactive signing of chained TXs in regtest as
it resulted in child TXs to return true immediately for CheckCanLock when
it should actually have waited for the parent TX to become locked first.

* Access chainActive.Height() while cs_main is locked

* Properly read and write lastChainLockBlock

"pindex" is NOT the chainlocked block after the while loop finishes. We
must use the pindex (renamed to pindexChainLock now) given on method entry.

Also, the GetLastChainLockBlock() result was not assigned to,
lastChainLockBlock which resulted in the while loop to run unnecessarily
long.

* Generalize filtering in NewPoWValidBlock and SyncTransaction

We're actually interested in all TXs that have inputs, so no need to
explicitly check for tx types.

* Use tx.IsCoinBase() instead of checking for index 0

* Handle cases where a TX is not received yet in wait_for_instantlock

* Wait on all nodes for the locks

Otherwise we end up with the sender having it locked but other nodes
not yet, failing the test.

* Fix LogPrintf call in CChainLocksHandler::DoInvalidateBlock
2019-03-19 10:38:16 +03:00
UdjinM6
e52763d21b Refactor and fix instantsend tests/utils (#2776)
* Unify autoIS/send_smth functions

* Rename autoix-mempool.py -> autois-mempool.py

* Make sure create_raw_trx produces expected results

* Make sure sender has enough inputs and nodes are synced before starting the actual test

* Mine one block to clean mempool up

* 2 blocks is enough for IS on regtest

This also unifies it across different IS tests

* Allow wait_for_instantlock to be called on any node, not only on the one that has the tx in the wallet

* No need to query for tx this often in wait_for_instantlock

* Rename create_raw_trx -> create_raw_tx

* Fund sender with a single TX instead of 30
2019-03-18 14:50:44 +01:00
Alexander Block
a69a5cf4a0 Use smaller (3 out of 5) quorums for regtest/Travis (#2774)
* Require only 3 out of 5 signatures for old InstantSend in regtest mode

* Use LLMQs of size 5 with threshold of 3 for regtest

* Fix wrong check for out-of-range bits in CFixedBitSet

* Reduce number of masternodes in masternode/LLMQ tests

* Add missing \n to LogPrintf call

* Use correct indexes for isolated/receiver/sender nodes

The way it was before resulted in nodes 1-3 being unused and 6-8 being used
for these 3 special nodes even though these are masternodes.

* Avoid stopping/starting isolated node in p2p-instantsend.py

It's enough to disable networking for this node.
2019-03-15 11:48:24 +03:00
UdjinM6
396ebc2dc7 Fix tests after 2768 (#2772)
* No free non-prioritised txes can be mined/relayed now

* fix typo
2019-03-15 07:18:16 +01:00
PastaPastaPasta
6f90cf7a17 Merge bitcoin#9602: Remove coin age priority and free transactions - implementation (#2768)
b421e6d Update example bitcoin.conf (Alex Morcos)
7d4e950 Allow setting minrelaytxfee to 0 (Alex Morcos)
359e8a0 [cleanup] Remove coin age priority completely. (Alex Morcos)
f9b9371 [rpc] Remove priorityDelta from prioritisetransaction (Alex Morcos)
49be7e1 [rpc] Remove priority information from mempool RPC calls (Alex Morcos)
0315888 [test] Remove priority from tests (Alex Morcos)
f838005 No longer allow "free" transactions (Alex Morcos)
ad727f4 [rpc] sendrawtransaction no longer bypasses minRelayTxFee (Alex Morcos)
fe282ac [cleanup] Remove estimatePriority and estimateSmartPriority (Alex Morcos)
400b151 [debug] Change -printpriority option (Alex Morcos)
272b25a [mining] Remove -blockprioritysize. (Alex Morcos)
12839cd [rpc] Remove estimatepriority and estimatesmartpriority. (Alex Morcos)
ddf58c7 wallet: Remove sendfree (MarcoFalke)

Tree-SHA512: a9a4499405923ce794ef18f9e334dbbd59dfc73a3dc2df6f85cc9c62af6f353ec2eed9c2d5e58e904f918d0d7ab738f403dd4939d9bc2276136864fe63710782
Signed-off-by: Pasta <Pasta@dash.org>

Fix backport and fix dash specific priority code
2019-03-14 17:44:42 +03:00
Alexander Block
3a1aeb000e Multiple fixes/refactorings for ChainLocks (#2765)
* Print which DKG type aborted

* Don't directly call EnforceBestChainLock and instead schedule the call

Calling EnforceBestChainLock might result in switching chains, which in
turn might end up calling signals, so we get into a recursive call chain.

Better to call EnforceBestChainLock from the scheduler.

* Regularly call EnforceBestChainLock and reset error flags on locked chain

* Don't invalidate blocks from CChainLocksHandler::TrySignChainTip

As the name of this method implies, it's trying to sign something and not
enforce/invalidate chains. Invalidating blocks is the job of
EnforceBestChainLock.

* Only call ActivateBestChain when tip != best CL tip

* Fix unprotected access of bestChainLockBlockIndex and bail out if its null

* Fix ChainLocks tests after changes in enforcement handling

* Only invoke NotifyChainLock signal from EnforceBestChainLock

This ensures that NotifyChainLock is not prematurely called before the
block is fully connected.

* Use a mutex to ensure that only one thread executes ActivateBestChain

It might happen that 2 threads enter ActivateBestChain at the same time
start processing block by block, while randomly switching between threads
so that sometimes one thread processed the block and then another one
processes it. A mutex protects ActivateBestChain now against this race.

* Rename local copy of bestChainLockBlockIndex to currentBestChainLockBlockIndex

* Don't call ActivateBestChain when best CL is part of the main chain
2019-03-13 16:00:54 +03:00
Alexander Block
152a78eab2 Add compatibility code to P2PFingerprintTest until we catch up with backports 2019-03-12 10:43:47 +01:00
Wladimir J. van der Laan
9344dee8aa Merge #11580: Do not send (potentially) invalid headers in response to getheaders
725b79a [test] Verify node doesn't send headers that haven't been fully validated (Russell Yanofsky)
3788a84 Do not send (potentially) invalid headers in response to getheaders (Matt Corallo)

Pull request description:

  Nowhere else in the protocol do we send headers which are for
  blocks we have not fully validated except in response to getheaders
  messages with a null locator. On my public node I have not seen any
  such request (whether for an invalid block or not) in at least two
  years of debug.log output, indicating that this should have minimal
  impact.

Tree-SHA512: c1f6e0cdcdfb78ea577d555f9b3ceb1b4b60eff4f6cf313bfd8b576c9562d797bea73abc23f7011f249ae36dd539c715f3d20487ac03ace60e84e1b77c0c1e1a
2019-03-12 10:43:47 +01:00
Wladimir J. van der Laan
d1a6022605 Merge #11113: [net] Ignore getheaders requests for very old side blocks
eff4bd8 [test] P2P functional test for certain fingerprinting protections (Jim Posen)
a2be3b6 [net] Ignore getheaders requests for very old side blocks (Jim Posen)

Pull request description:

  Sending a getheaders message with an empty locator and a stop hash is a request for a single header by hash. The node will respond with headers for blocks not in the main chain as well as those in the main chain. To avoid fingerprinting, the node should, however, ignore requests for headers on side branches that are too old. This replicates the logic that currently exists for `getdata` requests for blocks.

Tree-SHA512: e04ef61e2b73945be6ec5977b3c5680b6dc3667246f8bfb67afae1ecaba900c0b49b18bbbb74869f7a37ef70b6ed99e78ebe0ea0a1569369fad9e447d720ffc4
2019-03-12 10:43:47 +01:00
Alexander Block
cd0f94fb54 Give wait_for_quorum_phase more time 2019-03-12 09:20:17 +01:00
Alexander Block
e21da2d99b Move simple PoSe tests into llmq-simplepose.py
These tend to fail quite often on Travis due to multiple reasons. One
reason is that establishing intra quorum connections take some time and
the tests in dip3-deterministicmns.py did not sleep long enough. Another
reason is that the individual stages were not really checked for completion
but instead just a hardcoded sleep was used. And another reason was that
with a total of 13 MNs, it's not guaranteed that every DKG results in one
MN to be punished.
2019-03-12 07:02:52 +01:00