* Make constructors explicit
Signed-off-by: pasta <pasta@dashboost.org>
* static analysis fixes
Signed-off-by: pasta <pasta@dashboost.org>
* Make pFrom nullptr check it's own
Signed-off-by: pasta <pasta@dashboost.org>
* revert std thread changes and hasOperatorKey
Signed-off-by: pasta <pasta@dashboost.org>
* refactor/bls: misc refactoring and spelling/grammar fixes in bls code
* refactor/evo: misc refactoring and spelling/grammar fixes in evo code
* refactor: some include changes
* refactor: remove redundant `public`
* fix linter
Signed-off-by: pasta <pasta@dashboost.org>
* Sort includes
* Move `class CTxDSIn;`
* Drop unused functions in CBLSWorker
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Function CWallet::KeepKey requires locking as it has concurrent access to database and member nKeysLeftSinceAutoBackup.
Avoid data race when reading setInventoryTxToSend size by locking the read. If locking happens after the read, the size may change.
Lock cs_mnauth when reading verifiedProRegTxHash.
Make fRPCRunning atomic as it can be read/written from different threads simultaneously.
Make m_masternode_iqr_connection atomic as it can be read/written from different threads simultaneously.
Use a recursive mutex to synchronize concurrent access to quorumVvec.
Make m_masternode_connection atomic as it can be read/written from different threads simultaneously.
Make m_masternode_probe_connection atomic as it can be read/written from different threads simultaneously.
Use a recursive mutex in order to lock access to activeMasterNode.
Use a recursive mutex to synchronize concurrent access to skShare.
Guarded all mnauth fields of a CNode.
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Removes all (besides one) usages of "Params().GetConsensus().llmqs.at" and instead uses the wrapper in quorum_utils.cpp
Rename all params to llmq_params for consistency and not conflict with non-llmq params
make some llmq_params const where possible
remove unneeded llmq_params variables where it's only used once
Signed-off-by: pasta <pasta@dashboost.org>
Change ctpl implementation to use STL queue & mutex.
Use ctpl synchronized queue instead of boost lockfree queue in bls worker aggregator.
Use smart pointers for memory management of Aggregator and VectorAggregator. With 'delete this;' the objects are prone to data race on the delete operator.
Use smart pointers for memory management of ContributionVerifier.
Pass shared_ptr by value to other threads via worker pool.
1fabd59e7 Break circular dependency: init -> * -> init by extracting shutdown.h (Ben Woosley)
e62fdfeea Drop unused init.h includes (Ben Woosley)
Pull request description:
Most includers just wanted to react to pending shutdown.
This isolates access to `fRequestShutdown` and limits access to the shutdown api functions, including the new `CancelShutdown` for setting it to `false`.
Tree-SHA512: df42f75dfbba163576710e9a67cf1228531fd99d70a2f187bfba0bcc476d6749cf88180a97e66a81bb5b6c3c7f0917de7402d26039ba7b644cb7509b02f7e267
* instantsend: make stuff const where possible
Signed-off-by: pasta <pasta@dashboost.org>
* instantsend: remove unused `params`
Signed-off-by: pasta <pasta@dashboost.org>
* instantsend: combine two nested if's into one
Signed-off-by: pasta <pasta@dashboost.org>
* instantsend: use auto in spots where possible and clear
Signed-off-by: pasta <pasta@dashboost.org>
* coinjoin: make IsValidStructure const
Signed-off-by: pasta <pasta@dashboost.org>
* coinjoin: divide by an integer to avoid double implicit conversions
Signed-off-by: pasta <pasta@dashboost.org>
* coinjoin: make unused parameter unnamed
Signed-off-by: pasta <pasta@dashboost.org>
* quorums.* use const and use references where possible
Signed-off-by: pasta <pasta@dashboost.org>
* quorums.h don't return const when returning by value
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_blockprocessor.cpp remove redundant casts and combine two if statements
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_blockprocessor.cpp make values const
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_chainlocks.cpp access static function statically
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_chainlocks.h remove commented out include
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_commitment.cpp remove redundant casts
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_debug.cpp remove redundant casts and add const
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_dkgsession.cpp use const where possible
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_dkgsessionhandler.cpp use const where possible
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_dkgsessionhandler.cpp don't use std::move which apparently prevents "copy elision" in this instance
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_dkgsessionhandler.cpp use const
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_dkgsessionmgr.cpp misc refactoring
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_signing.* misc refactoring
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_signing_shares.* misc refactoring
Signed-off-by: pasta <pasta@dashboost.org>
* quorums_utils.cpp misc refactoring
Signed-off-by: pasta <pasta@dashboost.org>
232f96f5c8a3920c09db92f4dbac2ad7d10ce8cf doc: Add release notes for -avoidpartialspends (Karl-Johan Alm)
e00b4699cc6d2ee5697d38dd6607eb2631c9b77a clean-up: Remove no longer used ivars from CInputCoin (Karl-Johan Alm)
43e04d13b1ffc02b1082176e87f420198b40c7b1 wallet: Remove deprecated OutputEligibleForSpending (Karl-Johan Alm)
0128121101fb3ee82f3abd3973a967a4226ffe0e test: Add basic testing for wallet groups (Karl-Johan Alm)
59d6f7b4e2f847ec1f2ff46c84e6157655984f85 wallet: Switch to using output groups instead of coins in coin selection (Karl-Johan Alm)
87ebce25d66952f5ce565bb5130dcf5e24049872 wallet: Add output grouping (Karl-Johan Alm)
bb629cb9dc567cc819724d9f4852652926e60cbf Add -avoidpartialspends and m_avoid_partial_spends (Karl-Johan Alm)
65b3eda458221644616d0fdd6ba0fe01bdbce893 wallet: Add input bytes to CInputCoin (Karl-Johan Alm)
a443d7a0ca333b0bae63e04b5d476f9ad9c7aeac moveonly: CoinElegibilityFilter into coinselection.h (Karl-Johan Alm)
173e18a289088c6087ba6fac708e322aa63b7a94 utils: Add insert() convenience templates (Karl-Johan Alm)
Pull request description:
This PR adds an optional (off by default) `-avoidpartialspends` flag, which changes coin select to use output groups rather than outputs, where each output group corresponds to all outputs with the same destination.
It is a privacy improvement, as each time you spend some output, any other output that is publicly associated with the destination (address) will also be spent at the same time, at the cost of fee increase for cases where coin select without group restriction would find a more optimal set of coins (see example below).
For regular use without address reuse, this PR should have no effect on the user experience whatsoever; it only affects users who, for some reason, have multiple outputs with the same destination (i.e. address reuse).
Nodes with this turned off will still try to avoid partial spending, if the fee of the resulting transaction is not greater than the fee of the original transaction.
Example: a node has four outputs linked to two addresses `A` and `B`:
* 1.0 btc to `A`
* 0.5 btc to `A`
* 1.0 btc to `B`
* 0.5 btc to `B`
The node sends 0.2 btc to `C`. Without `-avoidpartialspends`, the following coin selection will occur:
* 0.5 btc to `A` or `B` is picked
* 0.2 btc is output to `C`
* 0.3 - fee is output to (unique change address)
With `-avoidpartialspends`, the following will instead happen:
* Both of (0.5, 1.0) btc to `A` or `B` is picked (one or the other pair)
* 0.2 btc is output to `C`
* 1.3 - fee is output to (unique change address)
As noted, the pro here is that, assuming nobody sends to the address after you spend from it, you will only ever use one address once. The con is that the transaction becomes slightly larger in this case, because it is overpicking outputs to adhere to the no partial spending rule.
This complements #10386, in particular it addresses @luke-jr and @gmaxwell's concerns in https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-300667926 and https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-302361381.
Together with `-avoidreuse`, this fully addresses the concerns in #10065 I believe.
Tree-SHA512: 24687a4490ba59cf4198ed90052944ff4996653a4257833bb52ed24d058b3e924800c9b3790aeb6be6385b653b49e304453e5d7ff960e64c682fc23bfc447621
# Conflicts:
# src/Makefile.am
# src/bench/coin_selection.cpp
# src/wallet/coincontrol.h
# src/wallet/coinselection.cpp
# src/wallet/coinselection.h
# src/wallet/init.cpp
# src/wallet/test/coinselector_tests.cpp
# src/wallet/wallet.cpp
# src/wallet/wallet.h
# test/functional/test_runner.py
* instantsend: remove unused variable sum of input value
Signed-off-by: pasta <pasta@dashboost.org>
* instantsend: Access static member statically
Signed-off-by: pasta <pasta@dashboost.org>
* refac: make functions private as possible
8c2d695c4a util: Store debug log file path in BCLog::Logger member. (Jim Posen)
8e7b961388 scripted-diff: Rename BCLog::Logger member variables. (Jim Posen)
1eac317f25 util: Refactor GetLogCategory. (Jim Posen)
3316a9ebb6 util: Encapsulate logCategories within BCLog::Logger. (Jim Posen)
6a6d764ca5 util: Move debug file management functions into Logger. (Jim Posen)
f55f4fcf05 util: Establish global logger object. (Jim Posen)
Pull request description:
This is purely a refactor with no behavior changes.
This creates a new class `BCLog::Logger` to encapsulate all global logging configuration and state.
Tree-SHA512: b34811f54a53b7375d7b6f84925453c6f2419d21179379ee28b3843d0f4ff8e22020de84a5e783453ea927e9074e32de8ecd05a6fa50d7bb05502001aaed8e53
9b2704777c [doc] Include txindex changes in the release notes. (Jim Posen)
ed77dd6b30 [test] Simple unit test for TxIndex. (Jim Posen)
6d772a3d44 [rpc] Public interfaces to GetTransaction block until synced. (Jim Posen)
a03f804f2a [index] Move disk IO logic from GetTransaction to TxIndex::FindTx. (Jim Posen)
e0a3b80033 [validation] Replace tx index code in validation code with TxIndex. (Jim Posen)
8181db88f6 [init] Initialize and start TxIndex in init code. (Jim Posen)
f90c3a62f5 [index] TxIndex method to wait until caught up. (Jim Posen)
70d510d93c [index] Allow TxIndex sync thread to be interrupted. (Jim Posen)
94b4f8bbb9 [index] TxIndex initial sync thread. (Jim Posen)
34d68bf3a3 [index] Create new TxIndex class. (Jim Posen)
c88bcec93f [db] Migration for txindex data to new, separate database. (Jim Posen)
0cb8303241 [db] Create separate database for txindex. (Jim Posen)
Pull request description:
I'm re-opening #11857 as a new pull request because the last one stopped loading for people
-------------------------------
This refactors the tx index code to be in it's own class and get built concurrently with validation code. The main benefit is decoupling and moving the txindex into a separate DB. The primary motivation is to lay the groundwork for other indexers that might be desired (such as the [compact filters](https://github.com/bitcoin/bips/pull/636)). The basic idea is that the TxIndex spins up its own thread, which first syncs the txindex to the current block index, then once in sync the BlockConnected ValidationInterface hook writes new blocks.
### DB changes
At the suggestion of some other developers, the txindex has been split out into a separate database. A data migration runs at startup on any nodes with a legacy txindex. Currently the migration blocks node initialization until complete.
### Open questions
- Should the migration of txindex data from the old DB to the new DB block in init or should it happen in a background thread? The downside to backgrounding it is that `getrawtransaction` would return an error message saying the txindex is syncing while the migration is running.
### Impact
In a sample size n=1 test where I synced nodes from scratch, the average time [Index writing](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L1903) was 3.36ms in master and 1.72ms in this branch. The average time between `UpdateTip` log lines for sequential blocks between 400,000 and IBD end on mainnet was 0.297204s in master and 0.286134s in this branch. Most likely this is just variance in IBD times, but I can try with some more trials if people want.
Tree-SHA512: 451fd7d95df89dfafceaa723cdf0f7b137615b531cf5c5035cfb54e9ccc2026cec5ac85edbcf71b7f4e2f102e36e9202b8b3a667e1504a9e1a9976ab1f0079c4
* instantsend: Upgrade IS db on dip0020 activation
* instantsend: Do not remove islocks for txes removed from mempool until dip0020 is activated
* refactor: introduce fUpgradedDB to avoid excessive locking and checking the state
* llmq: Decouple `fUpgradedDB` from `cs`
* Update src/llmq/quorums_instantsend.cpp
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Fix deadlock
Upgrade locks cs_main via GetTransaction
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* instantsend: Resolve block conflicts first and take care of mempool ones later
* refactor: Rename RemoveChainLockConflictingLock -> RemoveConflictingLock
* instantsend: Handle transaction removal from mempool (for all reasons besides inclusion in blocks)
* instantsend: Remove old islocks with no known txes from db (once)
* refactor: Replace magic number with CURRENT_VERSION
* fix: Do not remove islocks for (yet) valid orphans
* Apply suggestions from code review
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* tests: Check that CLs override ISes which invalidated non-CLed blocks earlier
* partial revert 3987: Do not mark blocks which conflict with ISes as "conflicting"
1527015 Avoid std::locale/imbue in DateTimeStrFormat (Pieter Wuille)
Pull request description:
And replace them with just hardcoded ISO8601 strings and `gmtime_r`.
Pointed out by @laanwj here: https://github.com/bitcoin/bitcoin/pull/12970#issuecomment-380962488
Tree-SHA512: a459758b42ca56f8462115aefe8e6377c1319fce509ea64dbb767f3f087c9b848335954cb684e5896c38008847684045505a3e1559fb3e83b8e80e10b003d1e7
* Simple changes, mostly just includes
* Continued include changes combined with using pointers to avoid including more than necessary in headers
* remove dup include
* masternode-utils.cpp include net.h in all builds
* resolve linter -7 +11
* drop quorums.h from dkgsessionhandler.cpp
* Add `<utilstrencodings.h>`
* Initialize lastMNListForVotingKeys
* Refactor GetMinedCommitment to return sharedptr
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* remove unused includes in validation.cpp
circular depends -2
* use more specific include in quorums.h
circular depends -1
* Remove unneeded include from quorums.cpp
circular depends -1
* llmq: Return bool in EnsureQuorumConnections
* llmq: Call RemoveMasternodeQuorumNodes directly
Instead of removing the good nodes from the vector in first loop and
call it with the bad ones in the second loop.
* refactor: Introduce and use CLLMQUtils::IsWatchQuorumsEnabled
* llmq: Move IsWatchQuorumsEnabled into EnsureQuorumConnections
* Fix extra line break in CommitTransaction log message
Introduced in #3680ebe7e80a49 (diff-b2bb174788c7409b671c46ccc86034bdR4113)
* doc: Fix `quorum sign` help
* doc: Add `getdata` to quorum commands list help
* doc: Drop "P2WSH" from signrawtransactionwithkey help
* trivial: Replace "push_back(Pair(..))" with "pushKV"
* trivial: Reorder wallet cmd-line options
* git: Add macos debug simbols to .gitignore
* trivial: Fix typos and whitespaces, drop unused stuff
* llmq: Protect nodeStates in CSigSharesManager::Cleanup()
* refactor: Tweak nodeState cleanup in CSigSharesManager::Cleanup()
* llmq: Add a couple of AssertLockHeld-s to ensure nodeStates protection
Instead of calling `CLLMQUtils::GetAllQuorumMembers` before each `CFinalCommitment::Verify` call to get the quorum members, just get them inside `CFinalCommitment::Verify`.
This avoids race condition for "spork19 off, spork 19 on" situations when some nodes might call CheckActiveState already, created new clsig and sent it so that other nodes processed it too already but they will call CheckActiveState afterwards (because it's done every 5 sec) and reset best chainlock to null.
* llmq|init|test: Add "mode" to -llmq-qvvec-sync parameter
This changes the paramter from `-llmq-qvvec-sync=<quorum_name>` to `-llmq-qvvec-sync=<quorum_name:mode>`
With the following definitions:
- `quorum_name`: Internal name of the quorum type
- `mode=0` - Sync always from all quorums of the type defined by `quorum_name`
- `mode=1` - Sync only if member of any from all other quorum of the type defined by `quorum_name`
`-llmq-qvvec-sync=llmq_100_67:0` To always request qvvec's from all `LLMQ_100_67`.
`-llmq-qvvec-sync=llmq_100_67:1` Only request if type member.
This means, if platform enables this on all MNs with `mode=0` we will
have all nodes asking new quorum for their verification vector instead
of only `24*100` at max.
* llmq: Adjust GetQuorumRecoveryStartOffset to use all MNs
* Turn `QvvecSyncMode` into `enum class`
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* llmq: Let return ProcessTx void instead of bool
The returned boolean is never used..
* llmq: Combine some "bail out early" cases
* llmq: Explicitly check for nullptr
* instantsend: refactor input locking into it's own method
Signed-off-by: pasta <pasta@dashboost.org>
* instantsend: introduce spork 24 `SPORK_24_INSTANTSEND_SIGNING_ENABLED`
This spork tells masternodes to refuse to lock transactions in mempool. Only transactions included in a block should be retroactively signed.
Signed-off-by: pasta <pasta@dashboost.org>
add spork defenition
Signed-off-by: pasta <pasta@dashboost.org>
* instantsend: refactor `sed -i 's/allowReSigning/fRetroactive/g' src/llmq/*`
Signed-off-by: pasta <pasta@dashboost.org>
* instantsend: adjust comments
Signed-off-by: pasta <pasta@dashboost.org>
* instantsend/tests: implement Spork 24 support in tests, and test it's usage
Signed-off-by: pasta <pasta@dashboost.org>
* fix feature_llmq_is_retroactive.py
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* drop Spork 24 and use Spork 2 value 1 as being no mempool signing
Signed-off-by: pasta <pasta@dashboost.org>
* fix spork check
Signed-off-by: pasta <pasta@dashboost.org>
* Fix tests
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Change comment
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* IsInstantSendSigningEnabled -> IsInstantSendMempoolSigningEnabled
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* instantsend: keep islocks cache when removing confirmed islocks from db
unordered_lru_cache should truncate it automagically
* instantsend: Batched write/erase for connected/disconnected blocks
* docs: Add help for -recsigsmaxage option
* Make `-llmq-data-recovery` help string dynamic
* Make devnet/regtest specific help strings dynamic
Only show devnet related help strings on devnet.
* Make linter happy
* Skip FindDevNetGenesisBlock when creating devnet params for help purposes only
* llmq: Use thread pool for quorum cache and data recovery threads
Move quorum data and cache thread handling into CQuorumManager.
* llmq: Fix explicit capture list
* llmq: Directly push the lamdas to workerPool
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* instantsend: Bail out early on disabled IS in more places
* instantsend: Disable InstantSend while reindexing and importing blocks
* Drop extra checks in GetInstantSendLockHashByTxid, GetInstantSendLockByInput and GetInstantSendLockByHash
* instantsend: Introduce (and use) KnownInstantSendLock
Check islock hash against both current and archived islocks
* instantsend: Optimize ProcessInstantSendLock, check for known islocks first
Two reasons:
1. GetTransaction is a potentially much heavier one
2. Doesn't make much sense to look for a tx if we have a known islock already
* Drop extra pendingInstantSendLocks.count(hash)
* Check pindexMined != nullptr
* Update src/llmq/quorums_instantsend.cpp
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Update src/llmq/quorums_instantsend.h
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* llmq: Drop InternalHasConflictingChainLock in ProcessNewChainLock
* llmq: Directly use clsig.blockHash instead of copying it into msgHash
* llmq: Reuse CInv(MSG_CLSIG, hash)
* llmq: Add const in two places
* instantsend: Mark a block with IS-locks which conflict with txes in a CL-ed block as conflicting and not as invalid
* tests: Tweak feature_llmq_is_cl_conflicts.py to test CL overriding a block with conflicting IS-locks
* instantsend|sigs: Sleep when there is no more work
Instead of sleeping only when no work has been done.
Avoids useless cycles, improves batching.
* llmq: Add and use nMaxBatchSize
* llmq: Compare to what we got in return, not what we verified at the end
It might happen that we get 32 pending but do only verify less than 32 and in this case we would assume there is no more work but it could still be more in the pipeline from my understanding.
* llmq: Rename more_work -> fMoreWork
* llmq: Be consistent with the other fMoreWork initialization
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* llmq: Avoid writing commitments to evodb and altering caches when all we want is to check block candidate validity
* tests: call `getblocktemplate` to trigger `CreateNewBlock` before quorum commitment is mined
* llmq: Implement automated DKG recovery threads
* llmq: Implement quorum verification vector sync
* init: Validiate quorum data recovery related command line parameter
* test: Add quorum_data_request_timeout_seconds in DashTestFramework
* test: Test quorum data recovery in feature_llmq_data_recovery.py
* test: Add feature_llmq_data_recovery.py to BASE_SCRIPTS
* test: Fix quorum_data_request_expiration_timeout in wait_for_quorum_data
* test: Always test the existence of secretKeyShare in test_mn_quorum_data
With this change it also validates that "secretKeyShare" is not in `quorum_info` if its not expected to be in there. Before this was basically just not tested.
* llmq|test: Use bool as argument type for -llmq-data-recovery
* llmq: Always set nTimeLastSuccess to 0
* test: Set -llmq-data-recovery=0 in p2p_quorum_data.py
* test: Simplify test_mns
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* refactor: pass CQuorumCPtr to StartQuorumDataRecoveryThread
* test: Fix thread name in comment
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* version: Bump PROTOCOL_VERSION and MIN_MASTERNODE_PROTO_VERSION
* version: Introduce LLMQ_DATA_MESSAGES_VERSION for QGETDATA/QDATA support
* test: Bump MY_VERSION to 70219 (LLMQ_DATA_MESSAGES_VERSION)
* llmq: Introduce CQuorumDataRequest as wrapper for QGETDATA requests
* llmq: Implement CQuorum::{SetVerificationVector, SetSecretKeyShare}
* llmq|net|protocol: Implement QGETDATA/QDATA P2P messages
* llmq: Restrict processing QGETDATA/QDATA to masternodes only
* llmq: Implement request limiting for QGETDATA/QDATA
* llmq: Implement CQuorumManger::RequestQuorumData
* rpc: Implement "quorum getdata" as wrapper around QGETDATA
Allows to trigger sending QGETDATA messages to connected peers by RPC.
* test: Handle QGETDATA/QDATA messages in mininode
* test: Add data structures to support QGETDATA/QDATA
* test: Add some helper in test_framework.py
* test: Implement tests for QGETDATA/QDATA in p2p_quorum_data.py
* test: Add p2p_quorum_data.py to BASE_SCRIPTS
* llmq|test: Add QWATCH support for QGETDATA/QDATA
* llmq: Store CQuorumPtr in cache, not CQuorumCPtr
* llmq: Fix cache usage after recent changes
* Use uacomment to create/find specific p2ps
* No need to use network adjusted time here, GetTime should be enough
* rpc: check proTxHash
* minor tweaks
* test: Adjustments after 4e27d6513e
* llmq: Rename and improve error lambda in CQuorumManager::ProcessMessage
* llmq: Process QDATA if -watchquorums is enabled
* test: Handle qwatch messages in mininode
* test: Add test for -watchquorums support
* test: Just some empty lines
* test: Properly stop the p2p network thread at the end of the test
* rpc: Adjust "quorum getdata" parameter descriptions
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* rpc: Fix optionality of proTxHash in "quorum getdata" command
* test: Test optionality of proTxHash for "quorum getdata" command
* test: Be more specific about imports in p2p_quorum_data.py
* llmq|rpc: Add some comments about the request.GetDataMask checks
* test: Some more empty lines
* rpc: One more parameter description
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* test: Unify assert statements / drop parentheses for all of them
* fix typo
Signed-off-by: pasta <pasta@dashboost.org>
* adjust some line wrapping to 80 chars
Signed-off-by: pasta <pasta@dashboost.org>
* tests: Seperate out into dif atomic methods, add logging
Signed-off-by: pasta <pasta@dashboost.org>
* test: Avoid restarting masternodes, just let available requests expire
Just takes a lot time and isn't required imo.
* test: Drop redundant code/tests after separation
This was introduced in 9e224ec2f2
* test: Merge three tests
"test_mnauth_restriction", "test_invalid_messages" and "test_invalid_unexpected_qdata" with the resulting name "test_basics" because i don't feel like DKG recovery thing should be part of a test called "test_invalid_messages" and giving it an own test probably wouldn't make a lot sense because it would still depend on "test_invalid_messages". I also think there is no need for a separated "test_invalid_unexpected_qdata".
* test: Rename test_ratelimiting_banscore -> test_request_limit
* test: Apply python style
* test: Wrap all at 120 characters
Thats the default "draw annoying warnings" setting for PyCharm (and IMO a reasonable line length).
* test: Move some variables
* test: Optimize for speed
* tests: use wait_until in get_mininode_id
* test: Don't use `!=` to check for `None`
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: pasta <pasta@dashboost.org>
* llmq: Detach dash-q-cachepop from caller
There should be no reason to keep this tread attached
to its parent, if so, let me know.
* llmq: Avoid nullptr access for pindexStart in ScanQuorums
* llmq: Add cacheKey in ProcessCommitment
* llmq: Erase minable commitments if they have been processed
* llmq: Add CLLMQUtils::InitQuorumsCache
* llmq: Use unordered_lru_cache for quorumsCache and rename it
* llmq: Use unordered_lru_cache for hasMinedCommitmentCache and rename it
* llmq: Drop redundant check
* llmq: Rename nMaxCount2 -> nScanCommitments
* llmq: Refactor storeCache -> fCacheExists
* llmq: Rename maxCount -> nCountRequested
* llmq: Rename result -> vecResultQuorums
* llmq: Return an empty vector if the are zero elements requested
* unordered_lru_cache: Add max_size()
* llmq: Partially reuse existing cache if more than max is requested
* llmq: std::map<LLMQType, unordered_lru_cache<...>> for scanQuoumsCache
* llmq: Drop params
* llmq: Only emplace to cache if there is something available
* llmq: Add CDKGSessionManager::WriteEncryptedContributions
Allows to store each member's encrypted contributions of the DKG.
* llmq: Store each member's contributions in the llmq database
* llmq: Add CDKGSessionManager::GetEncryptedContributions
I decided to don't cache here since its probably very unlikely this is called twice in a short period with what we have planed for it so far. We can add caching if the requirement for it changes at some point?
* Use single-threaded scheduler for IS, CL and Governance notifications
* Pass shared_ptr-s instead of objects themselves for CL, IS and Governance notifiers in CMainSignals/CValidatibnInterface
* llmq: Create shared_ptr for clsig at the root of its lifetime
* llmq: Create shared_ptr for islock clsig at the root of its lifetime
* llmq: Create shared_ptr for recSig at the root of its lifetime
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* llmq: Refactor CQuorumManager::{BuildQuorumFromCommitment, GetQuorum}
Construct and cache new quorums inside BuildQuorumFromCommitment
* llmq: Make all methods of CQuorumManager const
* More accurate handling of the BLOCK_CONFLICT_CHAINLOCK flag
* Update test/functional/feature_llmq_chainlocks.py
Co-authored-by: thephez <thephez@users.noreply.github.com>
* tests: make sure that previous tip on the reorged node is marked conflicting after chainlock
* Apply suggestions from code review
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: thephez <thephez@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* llmq: Split CSigShare creation/processing in CSigSharesManager
* rpc: Add "submit" parameter to "quorum sign"
* test: Add CSigShare and msg_qsigshare to messages.py
* test: Test the optional "submit" parameter of "quorum sign"
* random: Introduce std::shuffle alternative for FastRandomContext
3db746beb4
* random: change std::random_shuffle calls to std::shuffle
https://en.cppreference.com/w/cpp/algorithm/random_shuffle (deprecated in c++14)
* random: change FastRandomContext std::random_shuffle calls to shuffle
* random: change last std::shuffle calls to Shuffle
std::shuffle doesn't accept only two arguments so we use FastRandomContext()
* llmq: use inherited FastRandomContext
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* llmq: use inherited FastRandomContext
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Make the linter happy :)
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Do not store hashes in ProcessPendingMessageBatch and do not pass them to ReceiveMessage-s
* follow up: refactor/unify code in ProcessPendingMessageBatch
* llmq: Add an optional quorum hash to CSigningManager::AsyncSignIfMember
Allows to select the quorum to sign by its hash.
* rpc: Fix quorum selection of "quorum sign"
* test: Test the optional "quorumHash" parameter of "quorum sign"
* llmq: Move quorum checks up to avoid calling WriteVoteForId if they fail
* llmq: Drop hash parameter in PreVerifyMessage methods
* llmq: Drop some unused variables
* rpc: Drop unused variable
* llmq|net: Drop some unused CConnman parameter
* llmq: Drop some unused quorum parameter
* llmq: Drop some unused nodeId parameter
* Drop unused variables
* llmq: Drop more
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* llmq: Fix GetVerifiedContribution to return false in case of failure
* llmq: Move GetVerifiedContribution into GetVerifiedContributions
* llmq: Drop GetVerifiedContribution
* llmq: Keep cache locked while building GetVerifiedContributions result
* llmq: Read from DB into vvecPtr directly
Rename `sigSharesToAnnounce` class member in `CSigSharesManager` to `sigSharesQueuedToAnnounce` to avoid confusion with local variables/function args with the same name
* llmq: Restrict `ShouldSimulateError` to trigger for LLMQ_TEST only
Current `develop` tests fail. This was basically introduced by dashpay#3844 but it didn't come up before dashpay#3853 because the `v17` fork wasn't activated in `feature_llmq_dkgerrors.py`.
After dashpay#3853 `dip0008` activation takes [200 blocks](b95cf017c3 (diff-4a04bc0b355c780033960e8c261ee9b6d3c452897e1dcd88a15d272512266c76R539)) which was normally activated after [10 blocks](b95cf017c3 (diff-b92fa0fafafa27172736ebc88f9f9b658b1160caca512a318eefb7d93d22bf3cL18)) in `feature_llmq_dkgerrors.py`. Now with the 200 blocks `v17` gets activated during test which then leads to MN1, MN2 banning MN0 because it lies in DKG of `LLMQ_TEST` and `LLMQ_TEST_V17`.
There are other ways to solve it, like enabling `dip0008` earlier or enable `v17` later but IMO its anyway better to restrict `ShouldSimulateError` to only trigger for `LLMQ_TEST`.
* Revert "llmq: Restrict `ShouldSimulateError` to trigger for LLMQ_TEST only"
This reverts commit ec42d86126.
* llmq: Restrict `ShouldSimulateError` to trigger for LLMQ_TEST only (alternative)
Move ShouldSimulateError into CDKGSession
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Add LLMQ_100_67 quorums
* Re-use DEPLOYMENT_V17 bit to activate LLMQ_100_67 quorums
* Add LLMQ_TEST_NEW quorum and test its activation
* Tweak mine_quorum to work correctly with multiple quorum types
And to avoid a potentialy endless "while" loop
* llmq: Rename IsQuorumTypeEnabledAtBlock -> IsQuorumTypeEnabled
* chainparams|test: Rename llmq_test_new -> llmq_test_v17
* chainparams|consensus|llmq: Rename LLMQ_TEST_NEW -> LLMQ_TEST_V17
* Tweak few strings and the name of the test
* llmq: Make GetEnabledQuorumTypes return a vector of LLMQTypes, introduce GetLLMQParams
Signed-off-by: pasta <pasta@dashboost.org>
* Tweak minSize
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* Exclude LLMQ_100_67 from Concentrated Recovery
* Update test/functional/feature_new_quorum_type_activation.py
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: pasta <pasta@dashboost.org>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* Send islock notifications for txes received after their islocks were received
Also drop UpdateWalletTransaction - its name makes no sense and it's only used once.
* tests: early islocks should trigger notifications once a corresponding tx is received
* Tweak tests
- fail if an unexpected islock is received
- drop unused variable
* llmq: Drop `c_str()` in two log statements
* test: Move create_islock to DashTestFramework in test_framework.py
Just because it's used the same way in two files
* test: Simplify send tx in zmq test
* format
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* Fix potential deadlock in `CSporkManager::UpdateSpork()`
* Protect `inputRequestIds` with cs lock
* Protect `curDBTransaction` in `CEvoDB::CommitRootTransaction()`
* Check for `AssertLockNotHeld` in `EnforceBestChainLock()` instead of just having a comment in code
* Protect spork maps on (de)serialization
* Handle situations when we have evodb records but no blocks anymore
* Use IsEmpty
* Apply suggestions from code review
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* Always check for previous quorum set in llmq IS
* Refactor SelectQuorumForSigning and related code
Should have no changes in behaviour
* Do not use SIGN_HEIGHT_OFFSET when checking pending IS locks, use actual chain tip
This commit actually changes the behaviour
It seems like the idea here was to avoid announcing already available sigshares for a signhash to everyone if concentrated recovery is enabled. But by returning there if `IsAllMembersConnectedEnabled` equals to `false` we disable the re-announcements prior to the spork activation, where we want to still have them and re-enable if the spork is enabled where we don't want to have them.
* Rename CollectSigSharesToSend to CollectSigSharesToSendConcentrated
* Allow to disable concentrated recovery and new connection handling for large LLMQs
* Drop dead code in DoInvalidateBlock
* Let ActivateBestChain skip SyncWithValidationInterfaceQueue when called from IS or CL threads
* Use CL's own scheduler instead of a global one
* Revert "Let ActivateBestChain skip SyncWithValidationInterfaceQueue when called from IS or CL threads"
This reverts commit 1c9f6da50a.
* Return invalid state for GetTxPayload failures instead of asserting
* Throw an error when internal `find` fails instead of asserting
* Unify mnlist checks for lists created from blocks and lists created via updates of older lists with diffs
* Wrap evo block processing in try..catch to make sure all earlier introduced throws are caught
* evo/llmq: Adjust reject reasons for CFinalCommitmentTxPayload
In CheckLLMQCommitment this reject string was used so i adjusted the
other one in CQuorumBlockProcessor::GetCommitmentsFromBlock and the new
introduced in PR #3506
* return state.DoS instead of asserting in cbtx.cpp
Signed-off-by: pasta <pasta@dashboost.org>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: pasta <pasta@dashboost.org>
* llmq: Add more information to the result of CFinalCommitment::ToJson
Example output:
```
{
"version": 1,
"llmqType": 1,
"quorumHash":
"000002a20a35757fc467f0ebe8663f76e4c5df74411cc5f1735cfd92278071fe",
"signersCount": 49,
"signers": "03fbffffffffff",
"validMembersCount": 49,
"validMembers": "03fbffffffffff",
"quorumPublicKey":
"8db1108b50dd741076b577f6798f0bf382ca4d11dafd55ed14aaaf613f604aa55bf1da1f0469f75105376e440381ee23",
"quorumVvecHash":
"feedb15d5a7ca25a68d9c51036973d37925d8439d4769d7dd7bace7440089071",
"quorumSig":
"1879f6260ef427dc9542088b39e02a8e73e34c0484269cbc12108ab991d438f81e5be80a5b4b5595b6634113fdf060060c8fe190677627fb2d25f6e052c7f93598d36dd415f8f84c693c6a5943098616a1e135522852d8f572bb6e1470df4a94",
"membersSig":
"8ef7d881fcc1e8111dd82c556ab62dad56ea5ecb47a9ac067eb2057d92d27e39530630a7f77d19adc857d33eb485a1b601b2d78ac4c73742f2036089bfb1f25441f1dfcd3cfd1eed6e7d664c0e97bf75b7d7097ebd647791fbcef070fe3e3e91"
}
```
* llmq: Changed the byte order in ToHexStr(const std::vector<bool>& vBits)
* 🪲 improve evodb consistency recovering from dbcrash
* Adjust the fix
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Fix it
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Disable recovery from a crash during a fork and a corresponding part of dbcrash.py
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Skip some checks in CQuorumBlockProcessor when replaying blocks after the crash
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Process special txes in RollforwardBlock
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Update src/init.cpp
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
This was only used in only one remaining place and only to ensure that
reject messages are sent before closing sockets. This is solved by the
previous commit now.
The new way does not require the quorum hash to be included in the
calculation, which allows us to have determinism globally and not only
for single LLMQs.
Instead of propagating all sig shares to all LLMQ members, this will now
make all members send their individual sig share to a single member, which
is then responsible for the recovery and propagation of the recovered
signature. This process is repeated by all members every second for another
target/recovering member, until a recovered signature appears.
When taking the proTxHash naively, we might end up with a few unlucky MNs
which always have to perform most of the outbound connections while other
unlucky MNs would always have to wait for inbound connections. Hashing
the proTxHash with the quorum hash makes this more random.
Instead of only doing it the first time. This is a preparation for
the new intra-quorum connection system (which connects all members to all
other members)
* scripted-diff: Replace #include "" with #include <> (ryanofsky)
-BEGIN VERIFY SCRIPT-
for f in \
src/*.cpp \
src/*.h \
src/bench/*.cpp \
src/bench/*.h \
src/compat/*.cpp \
src/compat/*.h \
src/consensus/*.cpp \
src/consensus/*.h \
src/crypto/*.cpp \
src/crypto/*.h \
src/crypto/ctaes/*.h \
src/policy/*.cpp \
src/policy/*.h \
src/primitives/*.cpp \
src/primitives/*.h \
src/qt/*.cpp \
src/qt/*.h \
src/qt/test/*.cpp \
src/qt/test/*.h \
src/rpc/*.cpp \
src/rpc/*.h \
src/script/*.cpp \
src/script/*.h \
src/support/*.cpp \
src/support/*.h \
src/support/allocators/*.h \
src/test/*.cpp \
src/test/*.h \
src/wallet/*.cpp \
src/wallet/*.h \
src/wallet/test/*.cpp \
src/wallet/test/*.h \
src/zmq/*.cpp \
src/zmq/*.h
do
base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
Signed-off-by: Pasta <pasta@dashboost.org>
* scripted-diff: Replace #include "" with #include <> (Dash Specific)
-BEGIN VERIFY SCRIPT-
for f in \
src/bls/*.cpp \
src/bls/*.h \
src/evo/*.cpp \
src/evo/*.h \
src/governance/*.cpp \
src/governance/*.h \
src/llmq/*.cpp \
src/llmq/*.h \
src/masternode/*.cpp \
src/masternode/*.h \
src/privatesend/*.cpp \
src/privatesend/*.h
do
base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
Signed-off-by: Pasta <pasta@dashboost.org>
* build: Remove -I for everything but project root
Remove -I from build system for everything but the project root,
and built-in dependencies.
Signed-off-by: Pasta <pasta@dashboost.org>
# Conflicts:
# src/Makefile.test.include
* qt: refactor: Use absolute include paths in .ui files
* qt: refactor: Changes to make include paths absolute
This makes all include paths in the GUI absolute.
Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.
Signed-off-by: Pasta <pasta@dashboost.org>
# Conflicts:
# src/qt/dash.cpp
# src/qt/optionsmodel.cpp
# src/qt/test/rpcnestedtests.cpp
* test: refactor: Use absolute include paths for test data files
* Recommend #include<> syntax in developer notes
* refactor: Include obj/build.h instead of build.h
* END BACKPORT #11651 Remove trailing whitespace causing travis failure
* fix backport 11651
Signed-off-by: Pasta <pasta@dashboost.org>
* More of 11651
* fix blockchain.cpp
Signed-off-by: pasta <pasta@dashboost.org>
* Add missing "qt/" in includes
* Add missing "test/" in includes
* Fix trailing whitespaces
Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Do not sleep at the last block of the phase, it's not safe
* Refactor it a bit to make it clearer what's going on here
* Stop sleeping if blocks came faster than we expected
* Pass nAcceptTime via TransactionAddedToMempool and use it for ChainLocks
Otherwise the "first seen" time is way off after node restart
* Don't skip TransactionAddedToMempool for chainlocks while blockchain is not synced yet
Otherwise txes from mempool.dat won't be processed there
This should make code a little bit cleaner, should be no changes in the actual behaviour: non-members do not sleep already due to `sleepTime` being negative for them and `phaseTime = 0` does the same on regtest but for everyone.
* Implement re-signing of InstantSend inputs when TXs come in via blocks
* Use GetAdjustedTime instead of GetTimeMillis in CSigSharesManager
This allows use of mocktime in tests.
* Expose verifiedProRegTxHash in getpeerinfo and implement wait_for_mnauth
* Allow to wait for IS and CL to NOT happen
* Bump timeout for wait_for_instantlock
* Implement tests for retroactive signing of IS and CLs
* Add wait_for_tx function to DashTestFramework
* Add -whitelist=127.0.0.1 to node0
* Use node3 for isolated block generation
* Don't test for non-receival of TXs on node4/node5
* More/better logging for InstantSend
* Implement CRecoveredSigsDb::TruncateRecoveredSig
* Truncate recovered sigs for ISLOCKs instead of completely removing them
This makes AlreadyHave() return true even when the recovered sig is deleted
locally. This avoids re-requesting and re-processing of old recovered sigs.
* Also truncate recovered sigs for freshly received ISLOCKs
* Fix comment
* Remove LogPrints which have been commented out.
We have version control systems for a reason, if we want code to not run it should be removed. I personally see no value in keeping these around. I presume at one point they were spamming debug.log so we commented them out, but we really should have just removed them.
I believe all of this is dash specific code but any conflicts this does create are so minor they are not of concern imo.
Signed-off-by: Pasta <pasta@dashboost.org>
* remove a couple of extra comments
Signed-off-by: Pasta <pasta@dashboost.org>
* remove commented out code
Signed-off-by: Pasta <pasta@dashboost.org>
* check if we can lock before checking if it is conflicting, it is very rare a tx will actually be conflicting, whereas it is very common that a MN will not be able to sign for a specific Tx
Signed-off-by: Pasta <pasta@dashboost.org>
* remove unused variable
Signed-off-by: Pasta <pasta@dashboost.org>
* move sync check higher up
Signed-off-by: Pasta <pasta@dashboost.org>
* remove unused/unnecessary variable
Signed-off-by: Pasta <pasta@dashboost.org>
* remove unused variable
Signed-off-by: Pasta <pasta@dashboost.org>
* Revert "move sync check higher up"
This reverts commit 77fbe054df78b8bb12f686a627ef618ecff7e7a1.
* Introduce getbestchainlock rpc and fix llmq-is-cl-conflicts.py
* Add `known_block` field and move `getbestchainlock` to `blockchain` rpc category
* Add CChainLockSig::IsNull() and throw an exception in getbestchainlock if there is no known chainlock yet
* drop blockHash initializer
* Refactor `HandleFullyConfirmedBlock()`
* Pass `pindexMined` into `AddNonLockedTx()`
* Unify time import in Dash specific tests
* Refactor UpdateSpork and SetPrivKey
* Split ProcessPendingInstantSendLocks into two methods
* Split SelectQuorumForSigning into SelectQuorumForSigning and GetActiveQuorumSet
* Implement retrying of IS lock verification when the LLMQ active set rotates
* Remove unused overload of RemoveInstantSendLock
* Move deletion of recovered sigs into own method
* Remove recovered sigs for fully confirmed IS locks
* Also remove rs_t entries when removing recovered sigs from the outside
CleanupOldRecoveredSigs already does this as the last step, but when
RemoveRecoveredSig is called from the outside (e.g. from InstantSend),
these keys are not removed. This PR fixes this by storing the write time
into rs_r and later uses it to remove the rs_t entry.
Old entries will be incompatible with this (1 byte written in the past,
4 bytes written now). This checked by comparing the data size with
sizeof(uint32_t).
* Add TODO
* Remove ppszTypeName from protocol.cpp and reimplement GetCommand
This removes the need to carefully maintain ppszTypeName, which required
correct order and also did not allow to permanently remove old message
types.
To get the command name for an INV type, GetCommandInternal uses a switch
which needs to be maintained from now on.
The way this is implemented also resembles the way it is implemented in
Bitcoin today, but it's not identical. The original PR that introduced the
switch case in Bitcoin was part of the Segwit changes and thus never got
backported. I decided to implement it in a slightly different way that
avoids throwing exceptions when an unknown INV type is encountered.
IsKnownType will now also leverage GetCommandInternal() to figure out if
the INV type is known locally. This has the side effect of old/legacy
message types to return false from now on. We will depend on this side
effect in later commits when we remove legacy InstantSend code.
* Stop handling/relaying legacy IX messages
When we receive an IX message, we simply treat it as a regular TX and relay
it as such.
We'll however still request IX messages when they are announced to us. We
can't simply revert to requesting TX messages in this case as it might
result in the other peer not answering due to the TX not being in mapRelay
yet. We should at some point in the future completely drop handling of IX
messages instead.
* Remove IsNewInstantSendEnabled() and only use IsInstantSendEnabled()
* Remove legacy InstantSend from GUI
* Remove InstantSend from Bitcoin/Dash URIs
* Remove legacy InstantSend from RPC commands
* Remove legacy InstantSend from wallet
* Remove legacy instantsend.h include
* Remove legacy InstantSend from validation code
* Completely remove remaining legacy InstantSend code
* Remove now unused spork
* Fix InstantSend related test failures
* Remove now obsolete auto IS tests
* Make spork2 and spork3 disabled by default
This should have no influence on mainnet as these sporks are actually set
there. This will however affect regtest, which shouldn't have LLMQ based
InstantSend enabled by default.
* Remove instantsend tests from dip3-deterministicmns.py
These were only testing legacy InstantSend
* Fix .QCheckBox#checkUsePrivateSend styling a bit
* s/TXLEGACYLOCKREQUEST/LEGACYTXLOCKREQUEST/
* Revert "verified via InstantSend" back to "verified via LLMQ based InstantSend"
* Use cmd == nullptr instead of !cmd
* Remove last parameter from AvailableCoins call
This was for fUseInstantSend which is not present anymore since rebase
* Implement CompactFull() in CDBWrapper
This allows to compact the whole DB in one go.
* Implement more compact version of CDeterministicMNListDiff
This introduces CDeterministicMNStateDiff which requires to only store
fields on-disk which actually changed.
* Avoid writing mnUniquePropertyMap to disk when storing snapshots
This map can be rebuilt by simply using AddMN for each deserialized MN.
* Implement Serialize/Unserialize in CScript
This allows us to directly use READWRITE() on scripts and removes the need
for the ugly cast to CScriptBase. This commit also changes all Dash specific
uses of CScript to not use the cast.
* Keep track of registeration counts and introduce internalID for masternodes
The "internalId" is simply the number of MNs registered so far when the
new MN is added. It is deterministic and stays the same forever.
* Use internalId as keys in MN list diffs
This reduces the used size on-disk.
* Two simple speedups in MN list diff handling
1. Avoid full compare if dmn or state pointers match in BuildDiff
2. Use std::move when adding diff to listDiff in GetListForBlock
* Implement upgrade code for old CDeterministicMNListDiff format to new format
* Track tipIndex instead of tipHeight/tipBlockHash
* Store and pass around CBlockIndex* instead of block hash and height
This allows us to switch CDeterministicMNManager::GetListForBlock to work
with CBlockIndex.
* Refactor CDeterministicMNManager::GetListForBlock to require CBlockIndex*
Instead of requiring a block hash. This allows us to remove blockHash and
prevBlockHash from CDeterministicMNListDiff without the use of cs_main
locks in GetListForBlock.
* Remove prevBlockHash, blockHash and nHeight from CDeterministicMNListDiff
* Remove access to determinisitcMNManager in CMasternodeMetaMan::ToString()
The deterministic MN manager is not fully initialized yet at the time this
is called, which results in an empty list being returned everytime.
* Better logic to determine if an upgrade is needed
Reuse the "best block" logic to figure out if an upgrade is needed. Also
use it to ensure that older nodes are unable to start after the upgrade
was performed.
* Return null block hash if it was requested with getmnlistdiff
* bump CGovernanceManager::SERIALIZATION_VERSION_STRING
* Check SERIALIZATION_VERSION_STRING before deserializing anything else
* Invoke Clear() before deserializing just to be sure
No need to load 25 quorums when we're scanning for the larger quorums which
only have 4 active quorums. This avoids loading thousands of masternode
lists unnecessarily.
0f3471f net: make CNode's id private (Cory Fields)
9ff0a51 scripted-diff: net: Use accessor rather than node's id directly (Cory Fields)
e50c33e devtools: add script to verify scriptable changes (Cory Fields)
skipped travis changes
Tree-SHA512: a0ff50f4e1d38a2b63109b4996546c91b3e02e00d92c0bf04f48792948f78b1f6d9227a15d25c823fd4723a0277fc6a32c2c1287c7abbb7e50fd82ffb0f8d994
pnode->id to pnode->GetId()
Signed-off-by: Pasta <pasta@dashboost.org>
* Generalize CBLSLazyWrapper so that it can be used of pubkeys and secret keys
* Implement == and != operators for CBLSLazyWrapper
* Implement cached hash for CBLSLazyWrapper
* Use CBLSLazyPublicKey for CDeterministicMNState::pubKeyOperator
* Speed up GetProjectedMNPayees by sorting the MN list by last paid
Instead of updating a temporary list for each projected height and calling
GetMNPayee() on it.
* Cache intermediate lists in GetListForBlock
This avoids re-loading and applying diffs again and again.
* Only update masternode list UI max once every 3 seconds
This avoids updating the UI on every block, which turned out to be very
expensive.
* Fix compilation
* Drop time restrictions for mn list update in ClientModel
They are fully handled by MasternodeList now.
* Merge #8824: Refactor TxToJSON() and ScriptPubKeyToJSON()
0ff9320 refactor TxToJSON() and ScriptPubKeyToJSON() (jonnynewbs)
Tree-SHA512: caf7d590829e221522edd5b1ab8ce67b53a2c6986d3bbe8477eab420b1007bf60f885ed0a25ba9587e468c00768360ddc31db37847e862858573eaed5ed8b0d6
* remove witness and vsize
Signed-off-by: Pasta <Pasta@dash.org>
* Add valueSat
To preserve rpc output format
* Move extrapayload and special tx json output to `TxToUniv`
* Add spent index info
ProcessNewTransaction is now only called with pindex != nullptr when a
block is connected but never when it is disconnected, so there is no need
to check for posInBlock anymore (we don't even have it now).
This commit moves all logic of SyncTransaction into ProcessNewTransaction
and then calls it from TransactionAddedToMempool and BlockConnected.
This won't compile/work at first, but the next commits will fix this.
* Contains dashification. disables `-debug dash`
Merge #9424: Change LogAcceptCategory to use uint32_t rather than sets of strings.
6b3bb3d Change LogAcceptCategory to use uint32_t rather than sets of strings. (Gregory Maxwell)
Tree-SHA512: ebb5bcf9a7d00a32dd1390b727ff4d29330a038423611da01268d8e1d2c0229e52a1098e751d4e6db73ef4ae862e1e96d38249883fcaf12b68f55ebb01035b34
Signed-off-by: Pasta <Pasta@dash.org>
31 -> 32
Signed-off-by: Pasta <Pasta@dash.org>
* Merge #10123: Allow debug logs to be excluded from specified component
3bde556 Add -debugexclude option to switch off logging for specified components (John Newbery)
Tree-SHA512: 30202e3f2085fc2fc5dd4bedb92988f4cb162c612a42cf8f6395a7da326f34975ddc347f82bc4ddca6c84c438dc0cc6e87869f90c7ff88105dbeaa52a947fa43
* bump to uint64_t due to added Dash codes
Signed-off-by: Pasta <Pasta@dash.org>
* bump to uint64_t due to added Dash codes cont.
Signed-off-by: Pasta <Pasta@dash.org>
* string -> BCLog format
Signed-off-by: Pasta <Pasta@dash.org>
* uint32_t -> uint64_t
Signed-off-by: Pasta <Pasta@dash.org>
* Fix CBatchedLogger
* Fix most fDebug-s
* Fix `debug` rpc
* Fix BENCH and RAND conflicts
* Add ALERT and use it
* Update LogPrint-s in dash-specific code
* Tweak few log categories
Specifically:
- use PRIVATESEND in `CPrivateSendClientManager::GetRandomNotUsedMasternode()`
- use ZMQ in `CZMQPublishRawGovernanceVoteNotifier::NotifyGovernanceVote()` and `CZMQPublishRawGovernanceObjectNotifier::NotifyGovernanceObject()`
* Drop no longer used MASTERNODE category
* Merge #10153: logging: Fix off-by-one for shrinkdebugfile default
faab624 logging: Fix off-by-one for shrinkdebugfile (MarcoFalke)
Tree-SHA512: d6153e06067906172ff0611af9e585a3ecf0a7d56925b6ad7c12e75aa802441047059b9b6f6c78e79916c3f2abc8f1998bfd2d5b84201ec6421f727c08da3c21
* Shift dash-specific log categories to start from `1ul << 32` to avoid potential future conflicts with bitcoin ones
* Fix `dash` category
* remove debugCategories
Signed-off-by: Pasta <Pasta@dash.org>
* Prepend "std::" to find call
* Check for BCLog::PRIVATESEND instead of logCategories != BCLog::NONE
* Use BCLog::MNPAYMENTS category instead of checking for logCategories != BCLog::NONE
* Move "End Dash" comment below "ALERT"
When adding new entries here, we'll otherwise get confused with ordering
and might end up forgetting that adding something Dash specific must
continue with the bit after 43.
* Implement GetInstantSendLockCount in CInstantSendManager
* Add islockCountChanged signal to client model
* Show number of InstantSend locks in debug console
* Merge #9963: util: Properly handle errors during log message formatting
b651270 util: Throw tinyformat::format_error on formatting error (Wladimir J. van der Laan)
3b092bd util: Properly handle errors during log message formatting (Wladimir J. van der Laan)
Tree-SHA512: 85e3b7afec2255fc88034187f1abd6060e9421de17ed4e3d918416f393429a99cc2c974b362099aaaff6970549df47664bea4c857c4e46acc0789663201dc541
* "cast" debugMsg to a c string
Signed-off-by: Pasta <Pasta@dash.org>
"cast" debugMsg to a c string pt 2
Signed-off-by: Pasta <Pasta@dash.org>
* rename instantx.* file to instantsend.*
* Reorganize Dash Specific code into folders
add to privatesend folder the header files
Signed-off-by: Pasta <Pasta@dash.org>
add "masternode" to imports
Signed-off-by: Pasta <Pasta@dash.org>
add "masternode" to imports
Signed-off-by: Pasta <Pasta@dash.org>
add "masternode" to imports pt 2
Signed-off-by: Pasta <Pasta@dash.org>
Applies to CInstantSendManager and CChainLocksHandler. This fixes extremely
high RAM usage on reindex and resync, which happens to many/all transactions
being kept track of non-locked TXs (nonLockedTxs) and TXs per
block (blockTxs).
* Store rs_t key time in big endian
Also implement ConvertInvalidTimeKeys to convert old entries. We can remove
this later when we know that most MNs have run this code on testnet.
The way we stored the time field in the past lead to CleanupOldRecoveredSigs
iterating the keys in a strange order, causing no deletion at all and the
LLMQ DB filling up.
* Write batch in CleanupOldRecoveredSigs when it gets too large
This avoids RAM filling up and OOM getting triggered.
* Keep track of when a vote was written to the DB and clean up after week
Instead of only deleting when the corresponding recovered sig is deleted.
It sometimes happens that a masternode votes on something but a recovered
sig is never created, which leaves us with a vote that will never be
deleted.
* Apply suggestions from code review
Co-Authored-By: PastaPastaPasta <pasta@dashboost.org>
* Fall back to ReadBlockFromDisk when blockTxs is not filled yet
This handles the case where a MN is freshly started and SyncTransaction
was not been called for older transactions/blocks.
* Also update txFirstSeenTime
* Properly handle txids == nullptr
* Print inputs on which we voted
Also print the corresponding requestId. This makes debugging InstantSend
issues easier.
* Print quorum hash when signing a share
* Remove unused nodesByAddress map in CSigSharesManager::SendMessages
Not related to this PR, but a simple cleanup that should have no
side effects.
* Allow modifying simulate DKG error rates via RPC
* Don't lie to yourself :)
* Add some missing new-lines in LogPrintf calls
* More fine grained control over which messages to expect in mine_quorum
* Implement llmq-dkgerrors.py integration tests
These test DKG errors and malicious behavior.
* Move code to write archived ISLOCKs into its own method
We'll need this from another method as well later.
* Return ISLOCK instead of conflicting txid in GetConflictingTx/GetConflictingLock
* Implement GetInstantSendLocksByParent and RemoveChainedInstantSendLocks
These allow to easily delete multiple chains (actually trees) of ISLOCKs
in one go.
* Implement RemoveConflictedTx and call it from RemoveMempoolConflictsForLock
Also add "retryChildren" parameter to RemoveNonLockedTx so that we can
skip retrying of non-locked children TXs.
* Properly handle/remove conflicted TXs (between mempool and new blocks)
* Track non-locked TXs by inputs
* Implement and call ResolveBlockConflicts
* Also call ResolveBlockConflicts from ConnectBlock
But only when a block is known to have a conflict and at the same time is
ChainLocked, which causes the ISLOCK to be pruned.
* Split out RemoveChainLockConflictingLock from ResolveBlockConflicts
* Implement "quorum getrecsig" RPC
* Include decoded TX data in result of create_raw_tx
* Implement support for CLSIG in mininode.py
* Fix condition for update of nonLockedTxs.pindexMined
* Only add entries to nonLockedTxsByInputs when AddNonLockedTx is called for the first time
* Implement support for ISLOCK in mininode.py
* Implement tests for ChainLock vs InstantSend lock conflict resolution
* Handle review comment
Bail out (continue) early
* Also test conflicts in mempool instead of only in blocks
* Ask for locked TXs after removing conflicting TXs
When we removed a conflicting TX from the mempool, the correct/locked TX
is not available locally as the first-seen rule would have filtered before.
We need to re-request this TX if any other node announced it before.
* Apply suggestions from code review
Co-Authored-By: codablock <ablock84@gmail.com>
* Remove code for QDEBUGSTATUS propagation
This turned out to be too expensive and could easily take the network
down by bringing all nodes to 100% CPU usage. Better to fully remove this
functionality.
* Apply suggestions from code review
Co-Authored-By: codablock <ablock84@gmail.com>
* Update src/rpc/rpcquorums.cpp
Co-Authored-By: codablock <ablock84@gmail.com>
