* refactor: optimizations for processing specialtransaction, specifcally IDB / reindex
GetValidMNsCount -> GetAllMNsCount for reservation as GetValidMNsCount is a slow operation, while GetAllMNsCount is simply .size
Reduce some locking
Pass const Consensus::Params& to CleanupCache. I saw in benchmarks that a substancial
all CheckProRegTx ish methods should take check_sigs
introduce new CDeterministicMNManager thread for handling cleanup
* use main scheduler
* Update src/evo/deterministicmns.cpp
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* refactor: numerous changes to avoid passing around a const ref to shared_ptr of CDeterministicMNC when not needed.
Introduces ForEachMNShared, a version of ForEachMN that uses a shared_ptr, and may extend the lifetime of the underlying shared_ptr. This is not preferred, should prefer ForEachMN. See docs.
Adjusts ForEachMN to pass a reference. This is preferred for use over ForEachMNShared. See docs. A reference should be used since in usage we assume it's non-null anyway. Additionally, it allows us to know that the lifespan of the dmn is not being being extended (if lifespan needs to be extended, should use ForEachMNShared.
IsMNValid, IsMNPoSeBanned, UpdateMN, UpdateMN, AddUniqueProperty, DeleteUniqueProperty, UpdateUniqueProperty now take a const reference to CDeterministicMN instead of a const reference to shared_ptr<CDeterministicMN>. All of these functions previously assumed (or would've crashed) a non-null ptr, and non extended lifetime, as such converting to ref is appropriate.
CompareByLastPaid ptr overload now takes raw ptr instead of a const ref to shared. Since we simply dereference them, a raw ptr makes the most sense. This also avoids a potential expensive and implicit raw ptr -> shared ptr conversion if the function was called with raw ptrs.
rpcevo BuildDMNListEntry now takes a const ref for reasons as stated above
Signed-off-by: Pasta <pasta@dashboost.org>
* make stuff const
Signed-off-by: Pasta <pasta@dashboost.org>
* refactor/llmq: use ranges count_if
Signed-off-by: Pasta <pasta@dashboost.org>
* src/evo/evodb.cpp:57:29: warning: Assert statement calls a function which may have desired side effects: 'IsClean'. [assertWithSideEffect]
* src/llmq/quorums.cpp:635:37: note: Null pointer dereference
src/llmq/quorums.cpp:635:37: warning: Either the condition 'pFrom==nullptr' is redundant or there is possible null pointer dereference: pFrom. [nullPointerRedundantCheck]
src/llmq/quorums.cpp:636:81: note: Assuming that condition 'pFrom==nullptr' is not redundant
* fix a bunch of cppcheck warnings
* cppcheck: run on many more files. Enable all checks except a few ignored ones.
ignored
```
"Consider using std::transform algorithm instead of a raw loop."
"Consider using std::accumulate algorithm instead of a raw loop."
```
* ci: build specific version of cppcheck instead of install from apt
* ci: use cppcheck 2.4, remove commented out line, fix symlink
cppcheck 2.6 is latest, however causes issues
```
src/spork.cpp:135:51: warning: Analysis failed. If the code is valid then please report this failure. [cppcheckError]
```
cppcheck 2.5 appears to get into an infinite loop
* no need to check presence before insertion
* use if-init, remove redundant check
* remove redundant check
* don't remove cmake? fix macOs depends build?
* cppcheck: one per line, alphabetize
* remove duplicate cmake install
bf2e01097 uint256: Remove unnecessary crypto/common.h use (Karl-Johan Alm)
Pull request description:
This is an alternative to #13242 which keeps the `ReadLE64` part, but moves the `crypto/common.h` dependency into `crypto/common.h` as a function outside of `uint256`.
**Reason:** this change will remove dependencies for `uint256` to `crypto/common.h`, `compat/endian.h`, and `compat/byteswap.h`.
This PR removes the need to update tests to be endian-aware/-independent, but keeps the (arguably dubious) `ReadLE64` part (which was only introduced to fix the tests, not for any functionality).
Tree-SHA512: 78b35123cdb185b3b3ec59aba5ca8a5db72624d147f2d6a5484ffa5ce626a72f782a01dc6893fc8f5619b03e2eae7b5a03b0df5d43460f3bda428e719e188aec
* Make constructors explicit
Signed-off-by: pasta <pasta@dashboost.org>
* static analysis fixes
Signed-off-by: pasta <pasta@dashboost.org>
* Make pFrom nullptr check it's own
Signed-off-by: pasta <pasta@dashboost.org>
* revert std thread changes and hasOperatorKey
Signed-off-by: pasta <pasta@dashboost.org>
* refactor/bls: misc refactoring and spelling/grammar fixes in bls code
* refactor/evo: misc refactoring and spelling/grammar fixes in evo code
* refactor: some include changes
* refactor: remove redundant `public`
* fix linter
Signed-off-by: pasta <pasta@dashboost.org>
* Sort includes
* Move `class CTxDSIn;`
* Drop unused functions in CBLSWorker
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* evo: Refactor (Add/Update/Remove)MNs to do the job in one go
Instead of hashing the value and querying `mnUniquePropertyMap` both 2 or 3 times per property.
Also unify their error messages while at it.
* evo: Align nullValue checks with others in {Add|Delete}UniqueProperty
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* Simple changes, mostly just includes
* Continued include changes combined with using pointers to avoid including more than necessary in headers
* remove dup include
* masternode-utils.cpp include net.h in all builds
* resolve linter -7 +11
* drop quorums.h from dkgsessionhandler.cpp
* Add `<utilstrencodings.h>`
* Initialize lastMNListForVotingKeys
* Refactor GetMinedCommitment to return sharedptr
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* evo: Pass CCoinsViewCache instead of relying on pcoinsTip
This ensures that we are on the same page with ConnectBlock etc.
* evo: Process special txes before updating UTXOs
This ensures consistency between the way we do it in blocks and in mempool
* test: Verify db consistency after MN collateral is spent via ProTx that updates the same MN
* Make stuff const
* more constness
Co-authored-by: pasta <pasta@dashboost.org>
499d95e27 Add static_assert to prevent VARINT(<signed value>) (Russell Yanofsky)
Pull request description:
Using VARINT with signed types is dangerous because negative values will appear to serialize correctly, but then deserialize as positive values mod 128.
This commit changes the VARINT macro to trigger a compile error by default if called with an signed value, and it updates existing broken uses of VARINT to pass a special flag that lets them keep working with no changes in behavior.
There is some discussion about this issue here: https://github.com/bitcoin/bitcoin/pull/9693#issuecomment-278701473. I think another good change along these lines would be to make `GetSizeOfVarInt` and `WriteVarInt` throw exceptions if they are passed numbers less than 0 to serialize. But unlike this change, that would be a change in runtime behavior, and need more consideration.
Tree-SHA512: 082c65598cfac6dc1da042bdb47dbc9d5d789fc849fe52921cc238578588f4e5ff976c8b4b2ce42cb75290eb14f3b42ea76e26202c223c5b2aa63ef45c2ea3cc
* Adjust CDeterministicMNState, add helper methods
Changes all setting of `nPoSeBanHeight` into a call to `BanIfNotBanned`
Adds a helper method `IsBanned` that just is `return nPoSeBanHeight != -1`
In my opinion this makes the code generally more readable and easy to understand
Adds `Revive` helper method
I don't feel too strongly about this, because from what I have seen, this revive code is only done in one place,
but I generally think it makes sense to be a helper method of it's own
Signed-off-by: pasta <pasta@dashboost.org>
* Add `!` that was accidentally not added
Signed-off-by: pasta <pasta@dashboost.org>
* Make nPoSeBanHeight private
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
* Make internalId private to make sure it's set/accessed properly
* evo: Make internalId only accessible by the constructor
* Tweak constructors
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* Return invalid state for GetTxPayload failures instead of asserting
* Throw an error when internal `find` fails instead of asserting
* Unify mnlist checks for lists created from blocks and lists created via updates of older lists with diffs
* Wrap evo block processing in try..catch to make sure all earlier introduced throws are caught
* evo/llmq: Adjust reject reasons for CFinalCommitmentTxPayload
In CheckLLMQCommitment this reject string was used so i adjusted the
other one in CQuorumBlockProcessor::GetCommitmentsFromBlock and the new
introduced in PR #3506
* return state.DoS instead of asserting in cbtx.cpp
Signed-off-by: pasta <pasta@dashboost.org>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
Co-authored-by: pasta <pasta@dashboost.org>
* scripted-diff: Replace #include "" with #include <> (ryanofsky)
-BEGIN VERIFY SCRIPT-
for f in \
src/*.cpp \
src/*.h \
src/bench/*.cpp \
src/bench/*.h \
src/compat/*.cpp \
src/compat/*.h \
src/consensus/*.cpp \
src/consensus/*.h \
src/crypto/*.cpp \
src/crypto/*.h \
src/crypto/ctaes/*.h \
src/policy/*.cpp \
src/policy/*.h \
src/primitives/*.cpp \
src/primitives/*.h \
src/qt/*.cpp \
src/qt/*.h \
src/qt/test/*.cpp \
src/qt/test/*.h \
src/rpc/*.cpp \
src/rpc/*.h \
src/script/*.cpp \
src/script/*.h \
src/support/*.cpp \
src/support/*.h \
src/support/allocators/*.h \
src/test/*.cpp \
src/test/*.h \
src/wallet/*.cpp \
src/wallet/*.h \
src/wallet/test/*.cpp \
src/wallet/test/*.h \
src/zmq/*.cpp \
src/zmq/*.h
do
base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
Signed-off-by: Pasta <pasta@dashboost.org>
* scripted-diff: Replace #include "" with #include <> (Dash Specific)
-BEGIN VERIFY SCRIPT-
for f in \
src/bls/*.cpp \
src/bls/*.h \
src/evo/*.cpp \
src/evo/*.h \
src/governance/*.cpp \
src/governance/*.h \
src/llmq/*.cpp \
src/llmq/*.h \
src/masternode/*.cpp \
src/masternode/*.h \
src/privatesend/*.cpp \
src/privatesend/*.h
do
base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
Signed-off-by: Pasta <pasta@dashboost.org>
* build: Remove -I for everything but project root
Remove -I from build system for everything but the project root,
and built-in dependencies.
Signed-off-by: Pasta <pasta@dashboost.org>
# Conflicts:
# src/Makefile.test.include
* qt: refactor: Use absolute include paths in .ui files
* qt: refactor: Changes to make include paths absolute
This makes all include paths in the GUI absolute.
Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.
Signed-off-by: Pasta <pasta@dashboost.org>
# Conflicts:
# src/qt/dash.cpp
# src/qt/optionsmodel.cpp
# src/qt/test/rpcnestedtests.cpp
* test: refactor: Use absolute include paths for test data files
* Recommend #include<> syntax in developer notes
* refactor: Include obj/build.h instead of build.h
* END BACKPORT #11651 Remove trailing whitespace causing travis failure
* fix backport 11651
Signed-off-by: Pasta <pasta@dashboost.org>
* More of 11651
* fix blockchain.cpp
Signed-off-by: pasta <pasta@dashboost.org>
* Add missing "qt/" in includes
* Add missing "test/" in includes
* Fix trailing whitespaces
Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
a357293 Use MakeUnique<Db>(...) (practicalswift)
3e09b39 Use MakeUnique<T>(...) instead of std::unique_ptr<T>(new T(...)) (practicalswift)
8617989 Add MakeUnique (substitute for C++14 std::make_unique) (practicalswift)
d223bc9 Use unique_ptr for pcoinscatcher/pcoinsdbview/pcoinsTip/pblocktree (practicalswift)
b45c597 Use unique_ptr for pdbCopy (Db) and fix potential memory leak (practicalswift)
29ab96d Use unique_ptr for dbenv (DbEnv) (practicalswift)
f72cbf9 Use unique_ptr for pfilter (CBloomFilter) (practicalswift)
8ccf1bb Use unique_ptr for sem{Addnode,Outbound} (CSemaphore) (practicalswift)
73db063 Use unique_ptr for upnp_thread (boost::thread) (practicalswift)
0024531 Use unique_ptr for dbw (CDBWrapper) (practicalswift)
fa6d122 Use unique_ptr:s for {fee,short,long}Stats (TxConfirmStats) (practicalswift)
5a6f768 Use unique_ptr for httpRPCTimerInterface (HTTPRPCTimerInterface) (practicalswift)
860e912 Use unique_ptr for pwalletMain (CWallet) (practicalswift)
Pull request description:
Use `std::unique_ptr` (C++11) where possible.
Rationale:
1. Avoid resource leaks (specifically: forgetting to `delete` an object created using `new`)
2. Avoid undefined behaviour (specifically: double `delete`:s)
**Note to reviewers:** Please let me know if I've missed any obvious `std::unique_ptr` candidates. Hopefully this PR should cover all the trivial cases.
Tree-SHA512: 9fbeb47b800ab8ff4e0be9f2a22ab63c23d5c613a0c6716d9183db8d22ddbbce592fb8384a8b7874bf7375c8161efb13ca2197ad6f24b75967148037f0f7b20c
* Implement CompactFull() in CDBWrapper
This allows to compact the whole DB in one go.
* Implement more compact version of CDeterministicMNListDiff
This introduces CDeterministicMNStateDiff which requires to only store
fields on-disk which actually changed.
* Avoid writing mnUniquePropertyMap to disk when storing snapshots
This map can be rebuilt by simply using AddMN for each deserialized MN.
* Implement Serialize/Unserialize in CScript
This allows us to directly use READWRITE() on scripts and removes the need
for the ugly cast to CScriptBase. This commit also changes all Dash specific
uses of CScript to not use the cast.
* Keep track of registeration counts and introduce internalID for masternodes
The "internalId" is simply the number of MNs registered so far when the
new MN is added. It is deterministic and stays the same forever.
* Use internalId as keys in MN list diffs
This reduces the used size on-disk.
* Two simple speedups in MN list diff handling
1. Avoid full compare if dmn or state pointers match in BuildDiff
2. Use std::move when adding diff to listDiff in GetListForBlock
* Implement upgrade code for old CDeterministicMNListDiff format to new format
* Track tipIndex instead of tipHeight/tipBlockHash
* Store and pass around CBlockIndex* instead of block hash and height
This allows us to switch CDeterministicMNManager::GetListForBlock to work
with CBlockIndex.
* Refactor CDeterministicMNManager::GetListForBlock to require CBlockIndex*
Instead of requiring a block hash. This allows us to remove blockHash and
prevBlockHash from CDeterministicMNListDiff without the use of cs_main
locks in GetListForBlock.
* Remove prevBlockHash, blockHash and nHeight from CDeterministicMNListDiff
* Remove access to determinisitcMNManager in CMasternodeMetaMan::ToString()
The deterministic MN manager is not fully initialized yet at the time this
is called, which results in an empty list being returned everytime.
* Better logic to determine if an upgrade is needed
Reuse the "best block" logic to figure out if an upgrade is needed. Also
use it to ensure that older nodes are unable to start after the upgrade
was performed.
* Return null block hash if it was requested with getmnlistdiff
* bump CGovernanceManager::SERIALIZATION_VERSION_STRING
* Check SERIALIZATION_VERSION_STRING before deserializing anything else
* Invoke Clear() before deserializing just to be sure
* Specialize Serialize(CSizeComputer&) for BLS objects
Speeds up size computation for BLS objects.
* Track memory usage in CDBTransaction and CEvoDB
* Take memory used by CEvoDB/CDBTransaction into account when flushing
* Implement specialized SerReadWrite for immer maps
This allows to use READWRITE and fixes compilation errors with CSizeComputer
* Log EvoDB memory usage independently
Co-Authored-By: UdjinM6 <UdjinM6@users.noreply.github.com>
* Generalize CBLSLazyWrapper so that it can be used of pubkeys and secret keys
* Implement == and != operators for CBLSLazyWrapper
* Implement cached hash for CBLSLazyWrapper
* Use CBLSLazyPublicKey for CDeterministicMNState::pubKeyOperator
* Speed up GetProjectedMNPayees by sorting the MN list by last paid
Instead of updating a temporary list for each projected height and calling
GetMNPayee() on it.
* Cache intermediate lists in GetListForBlock
This avoids re-loading and applying diffs again and again.
* Only update masternode list UI max once every 3 seconds
This avoids updating the UI on every block, which turned out to be very
expensive.
* Fix compilation
* Drop time restrictions for mn list update in ClientModel
They are fully handled by MasternodeList now.
* Replace IsDeterministicMNsSporkActive with IsDIP3Active
IsDIP3Active will now use a fixed parameter from consensus params.
Values for DIP0003Height/DIP0003Hash need to be updated when spork15
activates on mainnet.
Also enforce correct block hash on testnet/mainnet for DIP3 activation
block.
* Remove SPORK_15_DETERMINISTIC_MNS_ENABLED
* Replace all uses of IsDeterministicMNsSporkActive with IsDIP3Active
* Remove DIP3 upgrade-path tests and directly start with DIP3 enabled tests
* Make -masternodeprivkey non-mandatory
This code will vanish later.
* Don't delete/add values to the unique property map when it's null
This happens when revoking operator keys, as the revoked MN will then have
the address and operator keys set to the null representation. We shouldn't
add the null value to the unique property map as otherwise future revokes
crash in an assert.
* Assert that no null values are passed to Add/DeleteUniqueProperty
* Check for null values before calling Add/DeleteUniqueProperty
* Apply suggestions from code review
Co-Authored-By: codablock <ablock84@gmail.com>
* Add user generated default constructors to BLS primitives
Fixes build issues on Mac:
In file included from evo/deterministicmns.cpp:5:
./evo/deterministicmns.h:375:24: error: default initialization of an object of const type 'const CBLSPublicKey' without a user-provided default constructor
static const T nullValue;
^
All logging that happens in BuildNewListFromBlock is currently printed twice.
This is because BuildNewListFromBlock is called while processing the block
and also while calculating the DIP4 MN list commitment.
This commit adds the debugLogs to this method to allow omitting logs while
called from the DIP4 code.
Members which are not in the validMembers bitsets are now PoSe punished.
The maximum PoSe score is dynamic and mostly identical to the number of
registered MNs. Added PoSe scores for failures are then a percentage of
the maximum score, so that we can better control how often failures are
allowed per payment cycle. For LLMQ failures, this is 66% to allow
approximately 2 failures per payment cycle.
* Track confirmedHash of masternodes
* Also include confirmedHash in DIP4
* Implement CalculateQuorum and CalculateScores in CDeterministicMNList
* Use new ProRegTx based scoring function in GetMasternodeScores
* Update test vectors for simplifiedmns_merkleroots tests
