* Don't rely on UTXO set in CheckCanLock
The UTXO set only works for TXs in the mempool and won't work when we try
to retroactively lock unlocked TXs from blocks.
This is safe as ProcessTx is only called when a TX was accepted into the
mempool or connected in a block, which means that all input checks were
good.
* Rename RetryLockMempoolTxs to RetryLockTxs and let it retry connected TXs
* Instead of manually calling ProcessTx, let SyncTransaction handle all cases
SyncTransaction is called from AcceptToMemoryPool and when transactions got
connected in a block. So this is the time we want to run TXs through
ProcessTx. This also enables retroactive signing of TXs that were unknown
before a new block appeared.
* Test retroactive signing and safe TXs in LLMQ ChainLocks tests
* Also test for retroactive signing of chained TXs
* Honor lockedParentTx when looking for TXs to retry signing
* Stop scanning for TXs to retry after a depth of 6
* Generate 6 block to avoid retroactive signing overloading Travis
* Avoid retroactive signing
* Don't rely on NewPoWValidBlock and use SyncTransaction to build blockTxs
NewPoWValidBlock is not guaranteed to be called when blocks come in fast.
When a block is accepted in AcceptBlock, NewPoWValidBlock is only called
when the new block is a successor of the currently active tip. This is not
the case when after the first block a second block is accepted immediately
as the first block is not connected yet.
This might be a bug actually in the handling of NewPoWValidBlock, so we
might need to check/fix this later, but currently I prefer to not touch
that part.
Instead, we now use SyncTransaction to gather TXs for blockTxs. This works
because SyncTransaction is called for all transactions in a freshly
connected block in one go. The call also happens before UpdatedBlockTip is
called, so it's fine with the existing logic.
* Use tx.IsCoinBase() instead of checking index 0
Also check for empty vin.
* Remove unused parameters from CInstantSendManager::ProcessTx
* Pass txHash in CheckCanLock by reference instead of pointer
* Dont' allow locking of TXs without inputs
* Remove unused local variable nInstantSendConfirmationsRequired
* Don't subtract 1 from nInstantSendConfirmationsRequired
This was necessary in the old system but is not necessary in the new system.
It also prevented proper retroactive signing of chained TXs in regtest as
it resulted in child TXs to return true immediately for CheckCanLock when
it should actually have waited for the parent TX to become locked first.
* Access chainActive.Height() while cs_main is locked
* Properly read and write lastChainLockBlock
"pindex" is NOT the chainlocked block after the while loop finishes. We
must use the pindex (renamed to pindexChainLock now) given on method entry.
Also, the GetLastChainLockBlock() result was not assigned to,
lastChainLockBlock which resulted in the while loop to run unnecessarily
long.
* Generalize filtering in NewPoWValidBlock and SyncTransaction
We're actually interested in all TXs that have inputs, so no need to
explicitly check for tx types.
* Use tx.IsCoinBase() instead of checking for index 0
* Handle cases where a TX is not received yet in wait_for_instantlock
* Wait on all nodes for the locks
Otherwise we end up with the sender having it locked but other nodes
not yet, failing the test.
* Fix LogPrintf call in CChainLocksHandler::DoInvalidateBlock
* Unify autoIS/send_smth functions
* Rename autoix-mempool.py -> autois-mempool.py
* Make sure create_raw_trx produces expected results
* Make sure sender has enough inputs and nodes are synced before starting the actual test
* Mine one block to clean mempool up
* 2 blocks is enough for IS on regtest
This also unifies it across different IS tests
* Allow wait_for_instantlock to be called on any node, not only on the one that has the tx in the wallet
* No need to query for tx this often in wait_for_instantlock
* Rename create_raw_trx -> create_raw_tx
* Fund sender with a single TX instead of 30
* Require only 3 out of 5 signatures for old InstantSend in regtest mode
* Use LLMQs of size 5 with threshold of 3 for regtest
* Fix wrong check for out-of-range bits in CFixedBitSet
* Reduce number of masternodes in masternode/LLMQ tests
* Add missing \n to LogPrintf call
* Use correct indexes for isolated/receiver/sender nodes
The way it was before resulted in nodes 1-3 being unused and 6-8 being used
for these 3 special nodes even though these are masternodes.
* Avoid stopping/starting isolated node in p2p-instantsend.py
It's enough to disable networking for this node.
* Print which DKG type aborted
* Don't directly call EnforceBestChainLock and instead schedule the call
Calling EnforceBestChainLock might result in switching chains, which in
turn might end up calling signals, so we get into a recursive call chain.
Better to call EnforceBestChainLock from the scheduler.
* Regularly call EnforceBestChainLock and reset error flags on locked chain
* Don't invalidate blocks from CChainLocksHandler::TrySignChainTip
As the name of this method implies, it's trying to sign something and not
enforce/invalidate chains. Invalidating blocks is the job of
EnforceBestChainLock.
* Only call ActivateBestChain when tip != best CL tip
* Fix unprotected access of bestChainLockBlockIndex and bail out if its null
* Fix ChainLocks tests after changes in enforcement handling
* Only invoke NotifyChainLock signal from EnforceBestChainLock
This ensures that NotifyChainLock is not prematurely called before the
block is fully connected.
* Use a mutex to ensure that only one thread executes ActivateBestChain
It might happen that 2 threads enter ActivateBestChain at the same time
start processing block by block, while randomly switching between threads
so that sometimes one thread processed the block and then another one
processes it. A mutex protects ActivateBestChain now against this race.
* Rename local copy of bestChainLockBlockIndex to currentBestChainLockBlockIndex
* Don't call ActivateBestChain when best CL is part of the main chain
725b79a [test] Verify node doesn't send headers that haven't been fully validated (Russell Yanofsky)
3788a84 Do not send (potentially) invalid headers in response to getheaders (Matt Corallo)
Pull request description:
Nowhere else in the protocol do we send headers which are for
blocks we have not fully validated except in response to getheaders
messages with a null locator. On my public node I have not seen any
such request (whether for an invalid block or not) in at least two
years of debug.log output, indicating that this should have minimal
impact.
Tree-SHA512: c1f6e0cdcdfb78ea577d555f9b3ceb1b4b60eff4f6cf313bfd8b576c9562d797bea73abc23f7011f249ae36dd539c715f3d20487ac03ace60e84e1b77c0c1e1a
eff4bd8 [test] P2P functional test for certain fingerprinting protections (Jim Posen)
a2be3b6 [net] Ignore getheaders requests for very old side blocks (Jim Posen)
Pull request description:
Sending a getheaders message with an empty locator and a stop hash is a request for a single header by hash. The node will respond with headers for blocks not in the main chain as well as those in the main chain. To avoid fingerprinting, the node should, however, ignore requests for headers on side branches that are too old. This replicates the logic that currently exists for `getdata` requests for blocks.
Tree-SHA512: e04ef61e2b73945be6ec5977b3c5680b6dc3667246f8bfb67afae1ecaba900c0b49b18bbbb74869f7a37ef70b6ed99e78ebe0ea0a1569369fad9e447d720ffc4
These tend to fail quite often on Travis due to multiple reasons. One
reason is that establishing intra quorum connections take some time and
the tests in dip3-deterministicmns.py did not sleep long enough. Another
reason is that the individual stages were not really checked for completion
but instead just a hardcoded sleep was used. And another reason was that
with a total of 13 MNs, it's not guaranteed that every DKG results in one
MN to be punished.
* Fix remaining `print`s in tests
* use AssertLockHeld(cs) instead of relying on comments
* actually use `clsig` in `EnforceBestChainLock()`
* fix log output in `EnforceBestChainLock()`
* drop comments
dcf2112 Add safe flag to listunspent result (NicolasDorier)
af61d9f Add COutput::fSafe member for safe handling of unconfirmed outputs (Russell Yanofsky)
Tree-SHA512: 311edb6fa8075b3ede5b24cb8c6e5d133ccd8ac9ecafea07b604ffa812ee4f071337e31695e662d8573590a0460af20aaaeb39d49c9ea87924449ea50bdfb0b3
64c0800 Use logging in individual tests (John Newbery)
38ad281 Use logging in test_framework/comptool.py (John Newbery)
ff19073 Use logging in test_framework/blockstore.py (John Newbery)
2a9c7c7 Use logging in test_framework/util.py (John Newbery)
b0dec4a Remove manual debug settings in qa tests. (John Newbery)
af1363c Always enable debug log and microsecond logging for test nodes. (John Newbery)
6d0e325 Use logging in mininode.py (John Newbery)
553a976 Add logging to p2p-segwit.py (John Newbery)
0e6d23d Add logging to test_framework.py (John Newbery)
Tree-SHA512: 42ee2acbf444ec32d796f930f9f6e272da03c75e93d974a126d4ea9b2dbaa77cc57ab5e63ce3fd33d609049d884eb8d9f65272c08922d10f8db69d4a60ad05a3
* Add libbacktrace to depends
This is currently only useful to extract symbols. It fails to gather
stacktraces when compiled with MinGW, so we can only use it to get symbol
information from a stack trace which we gathered outside of libbacktrace.
* Add -mbig-obj to CXXFLAGS for MinGW builds
* Implement stacktraces for C++ exceptions
This is a hack and should only be used for debugging. It works by wrapping
the C++ ABI __wrap___cxa_allocate_exception. The wrapper records a backtrace
and stores it in a global map. Later the stacktrace can be retrieved with
GetExceptionStacktraceStr.
This commit also adds handlers to pretty print uncaught exceptions and
signals.
* Use GetPrettyExceptionStr for all unhandled exceptions
* Use --enable-stacktraces in CI for linux32/linux64
* Register exception translators to pretty print exceptions in unit tests
* Catch and print python exceptions when stopping nodes
Otherwise the code at the bottom is never executed when nodes crash,
leading to no output of debug.log files on Travis.
* Remove now unneeded/unused TestCrash methods
* Drop registry deletion of the old key
* Drop no longer used CGovernanceObjectVoteFile::RemoveOldVotes()
* Drop temporary disconnect code and bump min protos to 70213
* drop comment
* fix
99fecf8 tests: Delete unused function _rpchost_to_args (Wladimir J. van der Laan)
Tree-SHA512: 40911d048d3fd7b3ce83e9b3caf2a409d55b47cbe08ea4450a16ca72264300bb12d5ef7dbcf335885975119b5977f949e6879546840064138fb506e24494d849
* Allow sub-batch verification in CBLSInsecureBatchVerifier
* Implement batch verification of CDKGDebugStatus messages
* Use uint8_t for statusBitset in CDKGDebugMemberStatus and CDKGDebugSessionStatus
No need to waste one byte per member and per LLMQ type.
* Reserve 4k of buffer for CSerializedNetMsg buffer
Profiling has shown that a lot of time is spent in resizing the data
vector when large messages are involved.
* Remove nHeight from CDKGDebugStatus
This field changes every block and causes all masternodes to propagate
its status for every block, even if nothing DKG related has changed.
* Leave out session statuses when we're not a member of that session
Otherwise MNs which are not members of DKG sessions will spam the network
* Remove receivedFinalCommitment from CDKGDebugSessionStatus
This is not bound to a session and thus is prone to spam the network when
final commitments are propagated in the finalization phase.
* Add "minableCommitments" to "quorum dkgstatus"
* Hold cs_main while calling GetMinableCommitment
* Abort processing of pending debug messages when spork18 gets disabled
* Don't ask for debug messages when we've already seen them
"statuses" only contains the current messages but none of the old messages,
so nodes kept re-requesting old messages.
* Use fast_dip3_enforcement instead of fast_dip3_activation
DashTestFramework was refactored before ChainLocks got merged, causing tests
to fail now.
* Move updating of DKG debug status into WaitForNextPhase
Otherwise callers of the RPCs might believe that the next phase has already
started and start producing more blocks, which would then cancel the
current session if it happens faster than the phase handler thread can
progress to the next phase.
