refactor: decouple db hooks from CFlatDB-based C*Manager objects, migrate to *Store structs (#5555)

## Motivation

As highlighted in https://github.com/dashpay/dash-issues/issues/52,
decoupling of `CFlatDB`-interacting components from managers of objects
like `CGovernanceManager` and `CSporkManager` is a key task for
achieving deglobalization of Dash-specific components.

The design of `CFlatDB` as a flat database agent relies on hooking into
the object's state its meant to load and store, using its
(de)serialization routines and other miscellaneous functions (notably,
without defining an interface) to achieve those ends. This approach was
taken predominantly for components that want a single-file cache.

Because of the method it uses to hook into the object (templates and the
use of temporary objects), it explicitly prevented passing arguments
into the object constructor, an explicit requirement for storing
references to other components during construction. This, in turn,
created an explicit dependency on those same components being available
in the global context, which would block the backport of bitcoin#21866,
a requirement for future backports meant to achieve parity in
`assumeutxo` support.

The design of these objects made no separation between persistent (i.e.
cached) and ephemeral (i.e. generated/fetched during initialization or
state transitions) data and the design of `CFlatDB` attempts to "clean"
the database by breaching this separation and attempting to access this
ephemeral data.

This might be acceptable if it is contained within the manager itself,
like `CSporkManager`'s `CheckAndRemove()` but is utterly unacceptable
when it relies on other managers (that, as a reminder, are only
accessible through the global state because of restrictions caused by
existing design), like `CGovernanceManager`'s `UpdateCachesAndClean()`.

This pull request aims to separate the `CFlatDB`-interacting portions of
these managers into a struct, with `CFlatDB` interacting only with this
struct, while the manager inherits the struct and manages
load/store/update of the database through the `CFlatDB` instance
initialized within its scope, though the instance only has knowledge of
what is exposed through the limited parent struct.

## Additional information

* As regards to existing behaviour, `CFlatDB` is written entirely as a
header as it relies on templates to specialize itself for the object it
hooks into. Attempting to split the logic and function definitions into
separate files will require you to explicitly define template
specializations, which is tedious.

* `m_db` is defined as a pointer as you cannot instantiate a
forward-declared template (see [this Stack Overflow
answer](https://stackoverflow.com/a/12797282) for more information),
which is done when defined as a member in the object scope.

* The conditional cache flush predicating on RPC _not_ being in the
warm-up state has been replaced with unconditional flushing of the
database on object destruction (@UdjinM6, is this acceptable?)

## TODOs

This is a list of things that aren't within the scope of this pull
request but should be addressed in subsequent pull requests

* [ ] Definition of an interface that `CFlatDB` stores are expected to
implement
* [ ] Lock annotations for all potential uses of members protected by
the `cs` mutex in each manager object and store
* [ ] Additional comments documenting what each function and member does
* [ ] Deglobalization of affected managers

---------

Co-authored-by: Kittywhiskers Van Gogh <63189531+kittywhiskers@users.noreply.github.com>
This commit is contained in:
Kittywhiskers Van Gogh 2023-09-24 20:20:21 +05:30 committed by GitHub
parent 633cc3260f
commit ee313525ad
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
21 changed files with 444 additions and 338 deletions

View File

@ -106,18 +106,18 @@ void CCoinJoinClientQueueManager::ProcessDSQueue(const CNode& peer, PeerManager&
dmn->pdmnState->addr.ToString());
return;
} else {
int64_t nLastDsq = mmetaman.GetMetaInfo(dmn->proTxHash)->GetLastDsq();
int64_t nDsqThreshold = mmetaman.GetDsqThreshold(dmn->proTxHash, mnList.GetValidMNsCount());
int64_t nLastDsq = mmetaman->GetMetaInfo(dmn->proTxHash)->GetLastDsq();
int64_t nDsqThreshold = mmetaman->GetDsqThreshold(dmn->proTxHash, mnList.GetValidMNsCount());
LogPrint(BCLog::COINJOIN, "DSQUEUE -- nLastDsq: %d nDsqThreshold: %d nDsqCount: %d\n", nLastDsq,
nDsqThreshold, mmetaman.GetDsqCount());
nDsqThreshold, mmetaman->GetDsqCount());
// don't allow a few nodes to dominate the queuing process
if (nLastDsq != 0 && nDsqThreshold > mmetaman.GetDsqCount()) {
if (nLastDsq != 0 && nDsqThreshold > mmetaman->GetDsqCount()) {
LogPrint(BCLog::COINJOIN, "DSQUEUE -- Masternode %s is sending too many dsq messages\n",
dmn->proTxHash.ToString());
return;
}
mmetaman.AllowMixing(dmn->proTxHash);
mmetaman->AllowMixing(dmn->proTxHash);
LogPrint(BCLog::COINJOIN, "DSQUEUE -- new CoinJoin queue (%s) from masternode %s\n", dsq.ToString(),
dmn->pdmnState->addr.ToString());
@ -1144,13 +1144,13 @@ bool CCoinJoinClientSession::StartNewQueue(CAmount nBalanceNeedsAnonymized, CCon
continue;
}
int64_t nLastDsq = mmetaman.GetMetaInfo(dmn->proTxHash)->GetLastDsq();
int64_t nDsqThreshold = mmetaman.GetDsqThreshold(dmn->proTxHash, nMnCount);
if (nLastDsq != 0 && nDsqThreshold > mmetaman.GetDsqCount()) {
int64_t nLastDsq = mmetaman->GetMetaInfo(dmn->proTxHash)->GetLastDsq();
int64_t nDsqThreshold = mmetaman->GetDsqThreshold(dmn->proTxHash, nMnCount);
if (nLastDsq != 0 && nDsqThreshold > mmetaman->GetDsqCount()) {
WalletCJLogPrint(m_wallet, "CCoinJoinClientSession::StartNewQueue -- Too early to mix on this masternode!" /* Continued */
" masternode=%s addr=%s nLastDsq=%d nDsqThreshold=%d nDsqCount=%d\n",
dmn->proTxHash.ToString(), dmn->pdmnState->addr.ToString(), nLastDsq,
nDsqThreshold, mmetaman.GetDsqCount());
nDsqThreshold, mmetaman->GetDsqCount());
nTries++;
continue;
}

View File

@ -78,9 +78,9 @@ void CCoinJoinServer::ProcessDSACCEPT(CNode& peer, CDataStream& vRecv)
}
}
int64_t nLastDsq = mmetaman.GetMetaInfo(dmn->proTxHash)->GetLastDsq();
int64_t nDsqThreshold = mmetaman.GetDsqThreshold(dmn->proTxHash, mnList.GetValidMNsCount());
if (nLastDsq != 0 && nDsqThreshold > mmetaman.GetDsqCount()) {
int64_t nLastDsq = mmetaman->GetMetaInfo(dmn->proTxHash)->GetLastDsq();
int64_t nDsqThreshold = mmetaman->GetDsqThreshold(dmn->proTxHash, mnList.GetValidMNsCount());
if (nLastDsq != 0 && nDsqThreshold > mmetaman->GetDsqCount()) {
if (fLogIPs) {
LogPrint(BCLog::COINJOIN, "DSACCEPT -- last dsq too recent, must wait: peer=%d, addr=%s\n", peer.GetId(), peer.addr.ToString());
} else {
@ -161,15 +161,15 @@ void CCoinJoinServer::ProcessDSQUEUE(const CNode& peer, PeerManager& peerman, CD
}
if (!dsq.fReady) {
int64_t nLastDsq = mmetaman.GetMetaInfo(dmn->proTxHash)->GetLastDsq();
int64_t nDsqThreshold = mmetaman.GetDsqThreshold(dmn->proTxHash, mnList.GetValidMNsCount());
LogPrint(BCLog::COINJOIN, "DSQUEUE -- nLastDsq: %d nDsqThreshold: %d nDsqCount: %d\n", nLastDsq, nDsqThreshold, mmetaman.GetDsqCount());
int64_t nLastDsq = mmetaman->GetMetaInfo(dmn->proTxHash)->GetLastDsq();
int64_t nDsqThreshold = mmetaman->GetDsqThreshold(dmn->proTxHash, mnList.GetValidMNsCount());
LogPrint(BCLog::COINJOIN, "DSQUEUE -- nLastDsq: %d nDsqThreshold: %d nDsqCount: %d\n", nLastDsq, nDsqThreshold, mmetaman->GetDsqCount());
//don't allow a few nodes to dominate the queuing process
if (nLastDsq != 0 && nDsqThreshold > mmetaman.GetDsqCount()) {
if (nLastDsq != 0 && nDsqThreshold > mmetaman->GetDsqCount()) {
LogPrint(BCLog::COINJOIN, "DSQUEUE -- Masternode %s is sending too many dsq messages\n", dmn->pdmnState->addr.ToString());
return;
}
mmetaman.AllowMixing(dmn->proTxHash);
mmetaman->AllowMixing(dmn->proTxHash);
LogPrint(BCLog::COINJOIN, "DSQUEUE -- new CoinJoin queue (%s) from masternode %s\n", dsq.ToString(), dmn->pdmnState->addr.ToString());

View File

@ -123,7 +123,7 @@ void CMNAuth::ProcessMessage(CNode& peer, PeerManager& peerman, CConnman& connma
}
if (!peer.fInbound) {
mmetaman.GetMetaInfo(mnauth.proRegTxHash)->SetLastOutboundSuccess(GetAdjustedTime());
mmetaman->GetMetaInfo(mnauth.proRegTxHash)->SetLastOutboundSuccess(GetAdjustedTime());
if (peer.m_masternode_probe_connection) {
LogPrint(BCLog::NET_NETCONN, "CMNAuth::ProcessMessage -- Masternode probe successful for %s, disconnecting. peer=%d\n",
mnauth.proRegTxHash.ToString(), peer.GetId());

View File

@ -6,6 +6,7 @@
#define BITCOIN_FLAT_DATABASE_H
#include <clientversion.h>
#include <chainparams.h>
#include <fs.h>
#include <hash.h>
#include <streams.h>
@ -35,7 +36,7 @@ private:
std::string strFilename;
std::string strMagicMessage;
bool Write(const T& objToSave)
bool CoreWrite(const T& objToSave)
{
// LOCK(objToSave.cs);
@ -70,7 +71,7 @@ private:
return true;
}
ReadResult Read(T& objToLoad, bool fDryRun = false)
ReadResult CoreRead(T& objToLoad)
{
//LOCK(objToLoad.cs);
@ -151,28 +152,13 @@ private:
LogPrintf("Loaded info from %s %dms\n", strFilename, GetTimeMillis() - nStart);
LogPrintf(" %s\n", objToLoad.ToString());
if(!fDryRun) {
LogPrintf("%s: Cleaning....\n", __func__);
objToLoad.CheckAndRemove();
LogPrintf(" %s\n", objToLoad.ToString());
}
return Ok;
}
public:
CFlatDB(std::string strFilenameIn, std::string strMagicMessageIn)
bool Read(T& objToLoad)
{
pathDB = GetDataDir() / strFilenameIn;
strFilename = strFilenameIn;
strMagicMessage = strMagicMessageIn;
}
bool Load(T& objToLoad)
{
LogPrintf("Reading info from %s...\n", strFilename);
ReadResult readResult = Read(objToLoad);
ReadResult readResult = CoreRead(objToLoad);
if (readResult == FileError)
LogPrintf("Missing file %s, will try to recreate\n", strFilename);
else if (readResult != Ok)
@ -191,36 +177,34 @@ public:
return true;
}
bool Dump(T& objToSave)
public:
CFlatDB(std::string strFilenameIn, std::string strMagicMessageIn)
{
int64_t nStart = GetTimeMillis();
pathDB = GetDataDir() / strFilenameIn;
strFilename = strFilenameIn;
strMagicMessage = strMagicMessageIn;
}
bool Load(T& objToLoad)
{
LogPrintf("Reading info from %s...\n", strFilename);
return Read(objToLoad);
}
bool Store(T& objToSave)
{
LogPrintf("Verifying %s format...\n", strFilename);
T tmpObjToLoad;
ReadResult readResult = Read(tmpObjToLoad, true);
if (!Read(tmpObjToLoad)) return false;
// there was an error and it was not an error on file opening => do not proceed
if (readResult == FileError)
LogPrintf("Missing file %s, will try to recreate\n", strFilename);
else if (readResult != Ok)
{
LogPrintf("Error reading %s: ", strFilename);
if(readResult == IncorrectFormat)
LogPrintf("%s: Magic is ok but data has invalid format, will try to recreate\n", __func__);
else
{
LogPrintf("%s: File format is unknown or invalid, please fix it manually\n", __func__);
return false;
}
}
int64_t nStart = GetTimeMillis();
LogPrintf("Writing info to %s...\n", strFilename);
Write(objToSave);
CoreWrite(objToSave);
LogPrintf("%s dump finished %dms\n", strFilename, GetTimeMillis() - nStart);
return true;
}
};

View File

@ -9,6 +9,7 @@
#include <chainparams.h>
#include <consensus/validation.h>
#include <evo/deterministicmns.h>
#include <flat-database.h>
#include <governance/classes.h>
#include <governance/validators.h>
#include <masternode/meta.h>
@ -26,27 +27,49 @@ std::unique_ptr<CGovernanceManager> governance;
int nSubmittedFinalBudget;
const std::string CGovernanceManager::SERIALIZATION_VERSION_STRING = "CGovernanceManager-Version-16";
const std::string GovernanceStore::SERIALIZATION_VERSION_STRING = "CGovernanceManager-Version-16";
const int CGovernanceManager::MAX_TIME_FUTURE_DEVIATION = 60 * 60;
const int CGovernanceManager::RELIABLE_PROPAGATION_TIME = 60;
CGovernanceManager::CGovernanceManager() :
nTimeLastDiff(0),
nCachedBlockHeight(0),
GovernanceStore::GovernanceStore() :
cs(),
mapObjects(),
mapErasedGovernanceObjects(),
cmapVoteToObject(MAX_CACHE_SIZE),
cmapInvalidVotes(MAX_CACHE_SIZE),
cmmapOrphanVotes(MAX_CACHE_SIZE),
mapLastMasternodeObject(),
lastMNListForVotingKeys(std::make_shared<CDeterministicMNList>())
{
}
CGovernanceManager::CGovernanceManager() :
m_db{std::make_unique<db_type>("governance.dat", "magicGovernanceCache")},
nTimeLastDiff(0),
nCachedBlockHeight(0),
setRequestedObjects(),
fRateChecksEnabled(true),
lastMNListForVotingKeys(std::make_shared<CDeterministicMNList>()),
votedFundingYesTriggerHash(std::nullopt),
cs()
votedFundingYesTriggerHash(std::nullopt)
{
}
CGovernanceManager::~CGovernanceManager()
{
if (!is_valid) return;
m_db->Store(*this);
}
bool CGovernanceManager::LoadCache(bool load_cache)
{
assert(m_db != nullptr);
is_valid = load_cache ? m_db->Load(*this) : m_db->Store(*this);
if (is_valid && load_cache) {
CheckAndRemove();
InitOnLoad();
}
return is_valid;
}
// Accessors for thread-safe access to maps
bool CGovernanceManager::HaveObjectForHash(const uint256& nHash) const
{
@ -318,7 +341,7 @@ void CGovernanceManager::UpdateCachesAndClean()
LogPrint(BCLog::GOBJECT, "CGovernanceManager::UpdateCachesAndClean\n");
std::vector<uint256> vecDirtyHashes = mmetaman.GetAndClearDirtyGovernanceObjectHashes();
std::vector<uint256> vecDirtyHashes = mmetaman->GetAndClearDirtyGovernanceObjectHashes();
LOCK2(cs_main, cs);
@ -368,7 +391,7 @@ void CGovernanceManager::UpdateCachesAndClean()
if ((pObj->IsSetCachedDelete() || pObj->IsSetExpired()) &&
(nTimeSinceDeletion >= GOVERNANCE_DELETION_DELAY)) {
LogPrint(BCLog::GOBJECT, "CGovernanceManager::UpdateCachesAndClean -- erase obj %s\n", (*it).first.ToString());
mmetaman.RemoveGovernanceObject(pObj->GetHash());
mmetaman->RemoveGovernanceObject(pObj->GetHash());
// Remove vote references
const object_ref_cm_t::list_t& listItems = cmapVoteToObject.GetItemList();
@ -850,13 +873,13 @@ void CGovernanceManager::SyncObjects(CNode& peer, PeerManager& peerman, CConnman
// do not provide any data until our node is synced
if (!::masternodeSync->IsSynced()) return;
if (netfulfilledman.HasFulfilledRequest(peer.addr, NetMsgType::MNGOVERNANCESYNC)) {
if (netfulfilledman->HasFulfilledRequest(peer.addr, NetMsgType::MNGOVERNANCESYNC)) {
// Asking for the whole list multiple times in a short period of time is no good
LogPrint(BCLog::GOBJECT, "CGovernanceManager::%s -- peer already asked me for the list\n", __func__);
peerman.Misbehaving(peer.GetId(), 20);
return;
}
netfulfilledman.AddFulfilledRequest(peer.addr, NetMsgType::MNGOVERNANCESYNC);
netfulfilledman->AddFulfilledRequest(peer.addr, NetMsgType::MNGOVERNANCESYNC);
int nObjCount = 0;
@ -1323,7 +1346,7 @@ void CGovernanceManager::InitOnLoad()
LogPrintf(" %s\n", ToString());
}
std::string CGovernanceManager::ToString() const
std::string GovernanceStore::ToString() const
{
LOCK(cs);

View File

@ -13,6 +13,8 @@
class CBloomFilter;
class CBlockIndex;
template<typename T>
class CFlatDB;
class CInv;
class CGovernanceManager;
@ -124,14 +126,9 @@ public:
}
};
//
// Governance Manager : Contains all proposals for the budget
//
class CGovernanceManager
class GovernanceStore
{
friend class CGovernanceObject;
public: // Types
protected:
struct last_object_rec {
explicit last_object_rec(bool fStatusOKIn = true) :
triggerBuffer(),
@ -148,57 +145,95 @@ public: // Types
bool fStatusOK;
};
using object_ref_cm_t = CacheMap<uint256, CGovernanceObject*>;
using txout_m_t = std::map<COutPoint, last_object_rec>;
using vote_cmm_t = CacheMultiMap<uint256, vote_time_pair_t>;
using txout_m_t = std::map<COutPoint, last_object_rec>;
using hash_s_t = std::set<uint256>;
private:
protected:
static constexpr int MAX_CACHE_SIZE = 1000000;
static const std::string SERIALIZATION_VERSION_STRING;
static const int MAX_TIME_FUTURE_DEVIATION;
static const int RELIABLE_PROPAGATION_TIME;
int64_t nTimeLastDiff;
// keep track of current block height
int nCachedBlockHeight;
public:
// critical section to protect the inner data structures
mutable RecursiveMutex cs;
protected:
// keep track of the scanning errors
std::map<uint256, CGovernanceObject> mapObjects GUARDED_BY(cs);
// mapErasedGovernanceObjects contains key-value pairs, where
// key - governance object's hash
// value - expiration time for deleted objects
std::map<uint256, int64_t> mapErasedGovernanceObjects;
std::map<uint256, CGovernanceObject> mapPostponedObjects;
hash_s_t setAdditionalRelayObjects;
object_ref_cm_t cmapVoteToObject;
CacheMap<uint256, CGovernanceVote> cmapInvalidVotes;
vote_cmm_t cmmapOrphanVotes;
txout_m_t mapLastMasternodeObject;
hash_s_t setRequestedObjects;
hash_s_t setRequestedVotes;
bool fRateChecksEnabled;
// used to check for changed voting keys
CDeterministicMNListPtr lastMNListForVotingKeys;
std::optional<uint256> votedFundingYesTriggerHash;
public:
GovernanceStore();
~GovernanceStore() = default;
template<typename Stream>
void Serialize(Stream &s) const
{
LOCK(cs);
s << SERIALIZATION_VERSION_STRING
<< mapErasedGovernanceObjects
<< cmapInvalidVotes
<< cmmapOrphanVotes
<< mapObjects
<< mapLastMasternodeObject
<< *lastMNListForVotingKeys;
}
template<typename Stream>
void Unserialize(Stream &s)
{
Clear();
LOCK(cs);
std::string strVersion;
s >> strVersion;
if (strVersion != SERIALIZATION_VERSION_STRING) {
return;
}
s >> mapErasedGovernanceObjects
>> cmapInvalidVotes
>> cmmapOrphanVotes
>> mapObjects
>> mapLastMasternodeObject
>> *lastMNListForVotingKeys;
}
void Clear()
{
LOCK(cs);
LogPrint(BCLog::GOBJECT, "Governance object manager was cleared\n");
mapObjects.clear();
mapErasedGovernanceObjects.clear();
cmapVoteToObject.Clear();
cmapInvalidVotes.Clear();
cmmapOrphanVotes.Clear();
mapLastMasternodeObject.clear();
}
std::string ToString() const;
};
//
// Governance Manager : Contains all proposals for the budget
//
class CGovernanceManager : public GovernanceStore
{
friend class CGovernanceObject;
private:
using hash_s_t = std::set<uint256>;
using db_type = CFlatDB<GovernanceStore>;
class ScopedLockBool
{
@ -220,13 +255,31 @@ private:
}
};
private:
static const int MAX_TIME_FUTURE_DEVIATION;
static const int RELIABLE_PROPAGATION_TIME;
private:
const std::unique_ptr<db_type> m_db;
bool is_valid{false};
int64_t nTimeLastDiff;
// keep track of current block height
int nCachedBlockHeight;
std::map<uint256, CGovernanceObject> mapPostponedObjects;
hash_s_t setAdditionalRelayObjects;
hash_s_t setRequestedObjects;
hash_s_t setRequestedVotes;
bool fRateChecksEnabled;
std::optional<uint256> votedFundingYesTriggerHash;
public:
// critical section to protect the inner data structures
mutable RecursiveMutex cs;
CGovernanceManager();
~CGovernanceManager();
virtual ~CGovernanceManager() = default;
bool LoadCache(bool load_cache);
bool IsValid() const { return is_valid; }
/**
* This is called by AlreadyHave in net_processing.cpp as part of the inventory
@ -263,55 +316,8 @@ public:
void CheckAndRemove() { UpdateCachesAndClean(); }
void Clear()
{
LOCK(cs);
LogPrint(BCLog::GOBJECT, "Governance object manager was cleared\n");
mapObjects.clear();
mapErasedGovernanceObjects.clear();
cmapVoteToObject.Clear();
cmapInvalidVotes.Clear();
cmmapOrphanVotes.Clear();
mapLastMasternodeObject.clear();
}
std::string ToString() const;
UniValue ToJson() const;
template<typename Stream>
void Serialize(Stream &s) const
{
LOCK(cs);
s << SERIALIZATION_VERSION_STRING
<< mapErasedGovernanceObjects
<< cmapInvalidVotes
<< cmmapOrphanVotes
<< mapObjects
<< mapLastMasternodeObject
<< *lastMNListForVotingKeys;
}
template<typename Stream>
void Unserialize(Stream &s)
{
LOCK(cs);
Clear();
std::string strVersion;
s >> strVersion;
if (strVersion != SERIALIZATION_VERSION_STRING) {
return;
}
s >> mapErasedGovernanceObjects
>> cmapInvalidVotes
>> cmmapOrphanVotes
>> mapObjects
>> mapLastMasternodeObject
>> *lastMNListForVotingKeys;
}
void UpdatedBlockTip(const CBlockIndex* pindex, CConnman& connman);
int64_t GetLastDiffTime() const { return nTimeLastDiff; }
void UpdateLastDiffTime(int64_t nTimeIn) { nTimeLastDiff = nTimeIn; }

View File

@ -199,7 +199,7 @@ bool CGovernanceObject::ProcessVote(const CGovernanceVote& vote, CGovernanceExce
return false;
}
if (!mmetaman.AddGovernanceVote(dmn->proTxHash, vote.GetParentHash())) {
if (!mmetaman->AddGovernanceVote(dmn->proTxHash, vote.GetParentHash())) {
std::ostringstream ostr;
ostr << "CGovernanceObject::ProcessVote -- Unable to add governance vote"
<< ", MN outpoint = " << vote.GetMasternodeOutpoint().ToStringShort()

View File

@ -237,11 +237,6 @@ void PrepareShutdown(NodeContext& node)
StopHTTPServer();
if (node.llmq_ctx) node.llmq_ctx->Stop();
// fRPCInWarmup should be `false` if we completed the loading sequence
// before a shutdown request was received
std::string statusmessage;
bool fRPCInWarmup = RPCIsInWarmup(&statusmessage);
for (const auto& client : node.chain_clients) {
client->flush();
}
@ -277,20 +272,6 @@ void PrepareShutdown(NodeContext& node)
// CValidationInterface callbacks, flush them...
GetMainSignals().FlushBackgroundCallbacks();
if (!fRPCInWarmup) {
// STORE DATA CACHES INTO SERIALIZED DAT FILES
CFlatDB<CMasternodeMetaMan> flatdb1("mncache.dat", "magicMasternodeCache");
flatdb1.Dump(mmetaman);
CFlatDB<CNetFulfilledRequestManager> flatdb4("netfulfilled.dat", "magicFulfilledCache");
flatdb4.Dump(netfulfilledman);
CFlatDB<CSporkManager> flatdb6("sporks.dat", "magicSporkCache");
flatdb6.Dump(*::sporkManager);
if (!fDisableGovernance) {
CFlatDB<CGovernanceManager> flatdb3("governance.dat", "magicGovernanceCache");
flatdb3.Dump(*::governance);
}
}
// After the threads that potentially access these pointers have been stopped,
// destruct and reset all to nullptr.
node.peerman.reset();
@ -324,6 +305,8 @@ void PrepareShutdown(NodeContext& node)
::governance.reset();
::sporkManager.reset();
::masternodeSync.reset();
::netfulfilledman.reset();
::mmetaman.reset();
// Stop and delete all indexes only after flushing background callbacks.
if (g_txindex) {
@ -1696,6 +1679,7 @@ bool AppInitMain(const CoreContext& context, NodeContext& node, interfaces::Bloc
node.chainman = &g_chainman;
ChainstateManager& chainman = *Assert(node.chainman);
assert(!::governance);
::governance = std::make_unique<CGovernanceManager>();
assert(!node.peerman);
@ -1706,7 +1690,6 @@ bool AppInitMain(const CoreContext& context, NodeContext& node, interfaces::Bloc
assert(!::sporkManager);
::sporkManager = std::make_unique<CSporkManager>();
::masternodeSync = std::make_unique<CMasternodeSync>(*node.connman, *::governance);
std::vector<std::string> vSporkAddresses;
if (args.IsArgSet("-sporkaddr")) {
@ -1732,6 +1715,8 @@ bool AppInitMain(const CoreContext& context, NodeContext& node, interfaces::Bloc
}
}
::masternodeSync = std::make_unique<CMasternodeSync>(*node.connman, *::governance);
// sanitize comments per BIP-0014, format user agent and check total size
std::vector<std::string> uacomments;
@ -1852,8 +1837,8 @@ bool AppInitMain(const CoreContext& context, NodeContext& node, interfaces::Bloc
}
#endif
assert(masternodeSync != nullptr);
assert(governance != nullptr);
assert(::governance != nullptr);
assert(::masternodeSync != nullptr);
pdsNotificationInterface = new CDSNotificationInterface(
*node.connman, *::masternodeSync, ::deterministicMNManager, *::governance, node.llmq_ctx, node.cj_ctx
);
@ -1867,10 +1852,9 @@ bool AppInitMain(const CoreContext& context, NodeContext& node, interfaces::Bloc
// ********************************************************* Step 7a: Load sporks
uiInterface.InitMessage(_("Loading sporks cache...").translated);
CFlatDB<CSporkManager> flatdb6("sporks.dat", "magicSporkCache");
if (!flatdb6.Load(*::sporkManager)) {
return InitError(strprintf(_("Failed to load sporks cache from %s"), (GetDataDir() / "sporks.dat").string()));
if (!::sporkManager->LoadCache()) {
auto file_path = (GetDataDir() / "sporks.dat").string();
return InitError(strprintf(_("Failed to load sporks cache from %s"), file_path));
}
// ********************************************************* Step 7b: load block chain
@ -2202,6 +2186,40 @@ bool AppInitMain(const CoreContext& context, NodeContext& node, interfaces::Bloc
g_wallet_init_interface.InitCoinJoinSettings(*node.cj_ctx->clientman);
#endif // ENABLE_WALLET
// ********************************************************* Step 7d: Setup other Dash services
bool fLoadCacheFiles = !(fReindex || fReindexChainState) && (::ChainActive().Tip() != nullptr);
if (!fDisableGovernance) {
if (!::governance->LoadCache(fLoadCacheFiles)) {
auto file_path = (GetDataDir() / "governance.dat").string();
if (fLoadCacheFiles && !fDisableGovernance) {
return InitError(strprintf(_("Failed to load governance cache from %s"), file_path));
}
return InitError(strprintf(_("Failed to clear governance cache at %s"), file_path));
}
}
assert(!::mmetaman);
::mmetaman = std::make_unique<CMasternodeMetaMan>(fLoadCacheFiles);
if (!::mmetaman->IsValid()) {
auto file_path = (GetDataDir() / "mncache.dat").string();
if (fLoadCacheFiles) {
return InitError(strprintf(_("Failed to load masternode cache from %s"), file_path));
}
return InitError(strprintf(_("Failed to clear masternode cache at %s"), file_path));
}
assert(!::netfulfilledman);
::netfulfilledman = std::make_unique<CNetFulfilledRequestManager>(fLoadCacheFiles);
if (!::netfulfilledman->IsValid()) {
auto file_path = (GetDataDir() / "netfulfilled.dat").string();
if (fLoadCacheFiles) {
return InitError(strprintf(_("Failed to load fulfilled requests cache from %s"), file_path));
}
return InitError(strprintf(_("Failed to clear fulfilled requests cache at %s"), file_path));
}
// ********************************************************* Step 8: start indexers
if (args.GetBoolArg("-txindex", DEFAULT_TXINDEX)) {
g_txindex = std::make_unique<TxIndex>(nTxIndexCache, false, fReindex);
@ -2263,60 +2281,9 @@ bool AppInitMain(const CoreContext& context, NodeContext& node, interfaces::Bloc
return false;
}
// ********************************************************* Step 10a: Load cache data
// ********************************************************* Step 10a: schedule Dash-specific tasks
// LOAD SERIALIZED DAT FILES INTO DATA CACHES FOR INTERNAL USE
bool fLoadCacheFiles = !(fReindex || fReindexChainState) && (::ChainActive().Tip() != nullptr);
fs::path pathDB = GetDataDir();
std::string strDBName;
strDBName = "mncache.dat";
uiInterface.InitMessage(_("Loading masternode cache...").translated);
CFlatDB<CMasternodeMetaMan> flatdb1(strDBName, "magicMasternodeCache");
if (fLoadCacheFiles) {
if(!flatdb1.Load(mmetaman)) {
return InitError(strprintf(_("Failed to load masternode cache from %s"), (pathDB / strDBName).string()));
}
} else {
CMasternodeMetaMan mmetamanTmp;
if(!flatdb1.Dump(mmetamanTmp)) {
return InitError(strprintf(_("Failed to clear masternode cache at %s"), (pathDB / strDBName).string()));
}
}
strDBName = "governance.dat";
uiInterface.InitMessage(_("Loading governance cache...").translated);
CFlatDB<CGovernanceManager> flatdb3(strDBName, "magicGovernanceCache");
if (fLoadCacheFiles && !fDisableGovernance) {
if(!flatdb3.Load(*::governance)) {
return InitError(strprintf(_("Failed to load governance cache from %s"), (pathDB / strDBName).string()));
}
::governance->InitOnLoad();
} else {
CGovernanceManager governanceTmp;
if(!flatdb3.Dump(governanceTmp)) {
return InitError(strprintf(_("Failed to clear governance cache at %s"), (pathDB / strDBName).string()));
}
}
strDBName = "netfulfilled.dat";
uiInterface.InitMessage(_("Loading fulfilled requests cache...").translated);
CFlatDB<CNetFulfilledRequestManager> flatdb4(strDBName, "magicFulfilledCache");
if (fLoadCacheFiles) {
if(!flatdb4.Load(netfulfilledman)) {
return InitError(strprintf(_("Failed to load fulfilled requests cache from %s"),(pathDB / strDBName).string()));
}
} else {
CNetFulfilledRequestManager netfulfilledmanTmp;
if(!flatdb4.Dump(netfulfilledmanTmp)) {
return InitError(strprintf(_("Failed to clear fulfilled requests cache at %s"),(pathDB / strDBName).string()));
}
}
// ********************************************************* Step 10b: schedule Dash-specific tasks
node.scheduler->scheduleEvery(std::bind(&CNetFulfilledRequestManager::DoMaintenance, std::ref(netfulfilledman)), std::chrono::minutes{1});
node.scheduler->scheduleEvery(std::bind(&CNetFulfilledRequestManager::DoMaintenance, std::ref(*netfulfilledman)), std::chrono::minutes{1});
node.scheduler->scheduleEvery(std::bind(&CMasternodeSync::DoMaintenance, std::ref(*::masternodeSync)), std::chrono::seconds{1});
node.scheduler->scheduleEvery(std::bind(&CMasternodeUtils::DoMaintenance, std::ref(*node.connman), std::ref(*::masternodeSync), std::ref(*node.cj_ctx)), std::chrono::minutes{1});
node.scheduler->scheduleEvery(std::bind(&CDeterministicMNManager::DoMaintenance, std::ref(*deterministicMNManager)), std::chrono::seconds{10});

View File

@ -463,7 +463,7 @@ void CDKGSession::VerifyConnectionAndMinProtoVersions() const
logger.Batch("%s does not have min proto version %d (has %d)", m->dmn->proTxHash.ToString(), MIN_MASTERNODE_PROTO_VERSION, it->second);
}
if (mmetaman.GetMetaInfo(m->dmn->proTxHash)->OutboundFailedTooManyTimes()) {
if (mmetaman->GetMetaInfo(m->dmn->proTxHash)->OutboundFailedTooManyTimes()) {
m->badConnection = true;
logger.Batch("%s failed to connect to it too many times", m->dmn->proTxHash.ToString());
}

View File

@ -924,7 +924,7 @@ bool EnsureQuorumConnections(const Consensus::LLMQParams& llmqParams, const CBlo
}
void AddQuorumProbeConnections(const Consensus::LLMQParams& llmqParams, const CBlockIndex *pQuorumBaseBlockIndex,
CConnman& connman, const uint256 &myProTxHash)
CConnman& connman, const uint256 &myProTxHash)
{
if (!IsQuorumPoseEnabled(llmqParams.type)) {
return;
@ -938,7 +938,7 @@ void AddQuorumProbeConnections(const Consensus::LLMQParams& llmqParams, const CB
if (dmn->proTxHash == myProTxHash) {
continue;
}
auto lastOutbound = mmetaman.GetMetaInfo(dmn->proTxHash)->GetLastOutboundSuccess();
auto lastOutbound = mmetaman->GetMetaInfo(dmn->proTxHash)->GetLastOutboundSuccess();
if (curTime - lastOutbound < 10 * 60) {
// avoid re-probing nodes too often
continue;

View File

@ -4,13 +4,31 @@
#include <masternode/meta.h>
#include <flat-database.h>
#include <timedata.h>
#include <sstream>
CMasternodeMetaMan mmetaman;
std::unique_ptr<CMasternodeMetaMan> mmetaman;
const std::string CMasternodeMetaMan::SERIALIZATION_VERSION_STRING = "CMasternodeMetaMan-Version-3";
const std::string MasternodeMetaStore::SERIALIZATION_VERSION_STRING = "CMasternodeMetaMan-Version-3";
CMasternodeMetaMan::CMasternodeMetaMan(bool load_cache) :
m_db{std::make_unique<db_type>("mncache.dat", "magicMasternodeCache")},
is_valid{
[&]() -> bool {
assert(m_db != nullptr);
return load_cache ? m_db->Load(*this) : m_db->Store(*this);
}()
}
{
}
CMasternodeMetaMan::~CMasternodeMetaMan()
{
if (!is_valid) return;
m_db->Store(*this);
}
UniValue CMasternodeMetaInfo::ToJson() const
{
@ -110,14 +128,7 @@ std::vector<uint256> CMasternodeMetaMan::GetAndClearDirtyGovernanceObjectHashes(
return vecTmp;
}
void CMasternodeMetaMan::Clear()
{
LOCK(cs);
metaInfos.clear();
vecDirtyGovernanceObjectHashes.clear();
}
std::string CMasternodeMetaMan::ToString() const
std::string MasternodeMetaStore::ToString() const
{
std::ostringstream info;
LOCK(cs);

View File

@ -6,14 +6,17 @@
#define BITCOIN_MASTERNODE_META_H
#include <serialize.h>
#include <sync.h>
#include <uint256.h>
#include <univalue.h>
#include <atomic>
#include <uint256.h>
#include <sync.h>
#include <memory>
class CConnman;
template<typename T>
class CFlatDB;
static constexpr int MASTERNODE_MAX_MIXING_TXES{5};
static constexpr int MASTERNODE_MAX_FAILED_OUTBOUND_ATTEMPTS{5};
@ -89,16 +92,13 @@ public:
};
using CMasternodeMetaInfoPtr = std::shared_ptr<CMasternodeMetaInfo>;
class CMasternodeMetaMan
class MasternodeMetaStore
{
private:
protected:
static const std::string SERIALIZATION_VERSION_STRING;
mutable RecursiveMutex cs;
std::map<uint256, CMasternodeMetaInfoPtr> metaInfos GUARDED_BY(cs);
std::vector<uint256> vecDirtyGovernanceObjectHashes GUARDED_BY(cs);
// keep track of dsq count to prevent masternodes from gaming coinjoin queue
std::atomic<int64_t> nDsqCount{0};
@ -117,8 +117,9 @@ public:
template<typename Stream>
void Unserialize(Stream &s)
{
LOCK(cs);
Clear();
LOCK(cs);
std::string strVersion;
s >> strVersion;
if (strVersion != SERIALIZATION_VERSION_STRING) {
@ -132,7 +133,33 @@ public:
}
}
void Clear()
{
LOCK(cs);
metaInfos.clear();
}
std::string ToString() const;
};
class CMasternodeMetaMan : public MasternodeMetaStore
{
private:
using db_type = CFlatDB<MasternodeMetaStore>;
private:
const std::unique_ptr<db_type> m_db;
const bool is_valid{false};
std::vector<uint256> vecDirtyGovernanceObjectHashes GUARDED_BY(cs);
public:
explicit CMasternodeMetaMan(bool load_cache);
~CMasternodeMetaMan();
bool IsValid() const { return is_valid; }
CMasternodeMetaInfoPtr GetMetaInfo(const uint256& proTxHash, bool fCreate = true);
int64_t GetDsqCount() const { return nDsqCount; }
@ -145,14 +172,8 @@ public:
void RemoveGovernanceObject(const uint256& nGovernanceObjectHash);
std::vector<uint256> GetAndClearDirtyGovernanceObjectHashes();
void Clear();
// Needed to avoid errors in flat-database.h
void CheckAndRemove() const {};
std::string ToString() const;
};
extern CMasternodeMetaMan mmetaman;
extern std::unique_ptr<CMasternodeMetaMan> mmetaman;
#endif // BITCOIN_MASTERNODE_META_H

View File

@ -76,7 +76,7 @@ void CMasternodeSync::SwitchToNextAsset()
uiInterface.NotifyAdditionalDataSyncProgressChanged(1);
connman.ForEachNode(CConnman::AllNodes, [](CNode* pnode) {
netfulfilledman.AddFulfilledRequest(pnode->addr, "full-sync");
netfulfilledman->AddFulfilledRequest(pnode->addr, "full-sync");
});
LogPrintf("CMasternodeSync::SwitchToNextAsset -- Sync has finished\n");
@ -159,13 +159,13 @@ void CMasternodeSync::ProcessTick()
if (!pnode->CanRelay() || (fMasternodeMode && pnode->fInbound)) continue;
{
if ((pnode->HasPermission(PF_NOBAN) || pnode->m_manual_connection) && !netfulfilledman.HasFulfilledRequest(pnode->addr, strAllow)) {
netfulfilledman.RemoveAllFulfilledRequests(pnode->addr);
netfulfilledman.AddFulfilledRequest(pnode->addr, strAllow);
if ((pnode->HasPermission(PF_NOBAN) || pnode->m_manual_connection) && !netfulfilledman->HasFulfilledRequest(pnode->addr, strAllow)) {
netfulfilledman->RemoveAllFulfilledRequests(pnode->addr);
netfulfilledman->AddFulfilledRequest(pnode->addr, strAllow);
LogPrintf("CMasternodeSync::ProcessTick -- skipping mnsync restrictions for peer=%d\n", pnode->GetId());
}
if(netfulfilledman.HasFulfilledRequest(pnode->addr, "full-sync")) {
if(netfulfilledman->HasFulfilledRequest(pnode->addr, "full-sync")) {
// We already fully synced from this node recently,
// disconnect to free this connection slot for another peer.
pnode->fDisconnect = true;
@ -175,9 +175,9 @@ void CMasternodeSync::ProcessTick()
// SPORK : ALWAYS ASK FOR SPORKS AS WE SYNC
if(!netfulfilledman.HasFulfilledRequest(pnode->addr, "spork-sync")) {
if(!netfulfilledman->HasFulfilledRequest(pnode->addr, "spork-sync")) {
// always get sporks first, only request once from each peer
netfulfilledman.AddFulfilledRequest(pnode->addr, "spork-sync");
netfulfilledman->AddFulfilledRequest(pnode->addr, "spork-sync");
// get current network sporks
connman.PushMessage(pnode, msgMaker.Make(NetMsgType::GETSPORKS));
LogPrint(BCLog::MNSYNC, "CMasternodeSync::ProcessTick -- nTick %d nCurrentAsset %d -- requesting sporks from peer=%d\n", nTick, nCurrentAsset, pnode->GetId());
@ -201,9 +201,9 @@ void CMasternodeSync::ProcessTick()
if (gArgs.GetBoolArg("-syncmempool", DEFAULT_SYNC_MEMPOOL)) {
// Now that the blockchain is synced request the mempool from the connected outbound nodes if possible
for (auto pNodeTmp : vNodesCopy) {
bool fRequestedEarlier = netfulfilledman.HasFulfilledRequest(pNodeTmp->addr, "mempool-sync");
bool fRequestedEarlier = netfulfilledman->HasFulfilledRequest(pNodeTmp->addr, "mempool-sync");
if (pNodeTmp->nVersion >= 70216 && !pNodeTmp->fInbound && !fRequestedEarlier && !pNodeTmp->IsBlockRelayOnly()) {
netfulfilledman.AddFulfilledRequest(pNodeTmp->addr, "mempool-sync");
netfulfilledman->AddFulfilledRequest(pNodeTmp->addr, "mempool-sync");
connman.PushMessage(pNodeTmp, msgMaker.Make(NetMsgType::MEMPOOL));
LogPrint(BCLog::MNSYNC, "CMasternodeSync::ProcessTick -- nTick %d nCurrentAsset %d -- syncing mempool from peer=%d\n", nTick, nCurrentAsset, pNodeTmp->GetId());
}
@ -235,12 +235,12 @@ void CMasternodeSync::ProcessTick()
}
// only request obj sync once from each peer
if(netfulfilledman.HasFulfilledRequest(pnode->addr, "governance-sync")) {
if(netfulfilledman->HasFulfilledRequest(pnode->addr, "governance-sync")) {
// will request votes on per-obj basis from each node in a separate loop below
// to avoid deadlocks here
continue;
}
netfulfilledman.AddFulfilledRequest(pnode->addr, "governance-sync");
netfulfilledman->AddFulfilledRequest(pnode->addr, "governance-sync");
nTriedPeerCount++;
@ -260,7 +260,7 @@ void CMasternodeSync::ProcessTick()
// request votes on per-obj basis from each node
for (auto& pnode : vNodesCopy) {
if(!netfulfilledman.HasFulfilledRequest(pnode->addr, "governance-sync")) {
if(!netfulfilledman->HasFulfilledRequest(pnode->addr, "governance-sync")) {
continue; // to early for this node
}
int nObjsLeftToAsk = m_govman.RequestGovernanceObjectVotes(*pnode, connman);

View File

@ -2513,6 +2513,8 @@ void CConnman::ThreadOpenMasternodeConnections()
if (gArgs.IsArgSet("-connect") && gArgs.GetArgs("-connect").size() > 0)
return;
assert(::mmetaman != nullptr);
auto& chainParams = Params();
bool didConnect = false;
@ -2579,7 +2581,7 @@ void CConnman::ThreadOpenMasternodeConnections()
continue;
}
if (!connectedNodes.count(addr2) && !IsMasternodeOrDisconnectRequested(addr2) && !connectedProRegTxHashes.count(proRegTxHash)) {
int64_t lastAttempt = mmetaman.GetMetaInfo(dmn->proTxHash)->GetLastOutboundAttempt();
int64_t lastAttempt = mmetaman->GetMetaInfo(dmn->proTxHash)->GetLastOutboundAttempt();
// back off trying connecting to an address if we already tried recently
if (nANow - lastAttempt < chainParams.LLMQConnectionRetryTimeout()) {
continue;
@ -2603,14 +2605,14 @@ void CConnman::ThreadOpenMasternodeConnections()
bool connectedAndOutbound = connectedProRegTxHashes.count(dmn->proTxHash) && !connectedProRegTxHashes[dmn->proTxHash];
if (connectedAndOutbound) {
// we already have an outbound connection to this MN so there is no theed to probe it again
mmetaman.GetMetaInfo(dmn->proTxHash)->SetLastOutboundSuccess(nANow);
mmetaman->GetMetaInfo(dmn->proTxHash)->SetLastOutboundSuccess(nANow);
it = masternodePendingProbes.erase(it);
continue;
}
++it;
int64_t lastAttempt = mmetaman.GetMetaInfo(dmn->proTxHash)->GetLastOutboundAttempt();
int64_t lastAttempt = mmetaman->GetMetaInfo(dmn->proTxHash)->GetLastOutboundAttempt();
// back off trying connecting to an address if we already tried recently
if (nANow - lastAttempt < chainParams.LLMQConnectionRetryTimeout()) {
continue;
@ -2661,7 +2663,7 @@ void CConnman::ThreadOpenMasternodeConnections()
didConnect = true;
mmetaman.GetMetaInfo(connectToDmn->proTxHash)->SetLastOutboundAttempt(nANow);
mmetaman->GetMetaInfo(connectToDmn->proTxHash)->SetLastOutboundAttempt(nANow);
OpenMasternodeConnection(CAddress(connectToDmn->pdmnState->addr, NODE_NETWORK), isProbe);
// should be in the list now if connection was opened
@ -2674,7 +2676,7 @@ void CConnman::ThreadOpenMasternodeConnections()
if (!connected) {
LogPrint(BCLog::NET_NETCONN, "CConnman::%s -- connection failed for masternode %s, service=%s\n", __func__, connectToDmn->proTxHash.ToString(), connectToDmn->pdmnState->addr.ToString(false));
// Will take a few consequent failed attempts to PoSe-punish a MN.
if (mmetaman.GetMetaInfo(connectToDmn->proTxHash)->OutboundFailedTooManyTimes()) {
if (mmetaman->GetMetaInfo(connectToDmn->proTxHash)->OutboundFailedTooManyTimes()) {
LogPrint(BCLog::NET_NETCONN, "CConnman::%s -- failed to connect to masternode %s too many times\n", __func__, connectToDmn->proTxHash.ToString());
}
}

View File

@ -2731,6 +2731,8 @@ void PeerManagerImpl::ProcessGetCFCheckPt(CNode& peer, CDataStream& vRecv, const
std::pair<bool /*ret*/, bool /*do_return*/> static ValidateDSTX(CTxMemPool& mempool, ChainstateManager& chainman, CCoinJoinBroadcastTx& dstx, uint256 hashTx)
{
assert(::mmetaman != nullptr);
if (!dstx.IsValidStructure()) {
LogPrint(BCLog::COINJOIN, "DSTX -- Invalid DSTX structure: %s\n", hashTx.ToString());
return {false, true};
@ -2773,7 +2775,7 @@ std::pair<bool /*ret*/, bool /*do_return*/> static ValidateDSTX(CTxMemPool& memp
return {false, true};
}
if (!mmetaman.GetMetaInfo(dmn->proTxHash)->IsValidForMixingTxes()) {
if (!mmetaman->GetMetaInfo(dmn->proTxHash)->IsValidForMixingTxes()) {
LogPrint(BCLog::COINJOIN, "DSTX -- Masternode %s is sending too many transactions %s\n", dstx.masternodeOutpoint.ToStringShort(), hashTx.ToString());
return {true, true};
// TODO: Not an error? Could it be that someone is relaying old DSTXes
@ -2787,7 +2789,7 @@ std::pair<bool /*ret*/, bool /*do_return*/> static ValidateDSTX(CTxMemPool& memp
LogPrint(BCLog::COINJOIN, "DSTX -- Got Masternode transaction %s\n", hashTx.ToString());
mempool.PrioritiseTransaction(hashTx, 0.1*COIN);
mmetaman.DisallowMixing(dmn->proTxHash);
mmetaman->DisallowMixing(dmn->proTxHash);
return {true, false};
}

View File

@ -3,11 +3,32 @@
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <chainparams.h>
#include <flat-database.h>
#include <netfulfilledman.h>
#include <shutdown.h>
#include <util/system.h>
CNetFulfilledRequestManager netfulfilledman;
std::unique_ptr<CNetFulfilledRequestManager> netfulfilledman;
CNetFulfilledRequestManager::CNetFulfilledRequestManager(bool load_cache) :
m_db{std::make_unique<db_type>("netfulfilled.dat", "magicFulfilledCache")},
is_valid{
[&]() -> bool {
assert(m_db != nullptr);
return load_cache ? m_db->Load(*this) : m_db->Store(*this);
}()
}
{
if (is_valid && load_cache) {
CheckAndRemove();
}
}
CNetFulfilledRequestManager::~CNetFulfilledRequestManager()
{
if (!is_valid) return;
m_db->Store(*this);
}
void CNetFulfilledRequestManager::AddFulfilledRequest(const CService& addr, const std::string& strRequest)
{
@ -62,13 +83,13 @@ void CNetFulfilledRequestManager::CheckAndRemove()
}
}
void CNetFulfilledRequestManager::Clear()
void NetFulfilledRequestStore::Clear()
{
LOCK(cs_mapFulfilledRequests);
mapFulfilledRequests.clear();
}
std::string CNetFulfilledRequestManager::ToString() const
std::string NetFulfilledRequestStore::ToString() const
{
std::ostringstream info;
info << "Nodes with fulfilled requests: " << (int)mapFulfilledRequests.size();

View File

@ -9,41 +9,61 @@
#include <serialize.h>
#include <sync.h>
class CNetFulfilledRequestManager;
extern CNetFulfilledRequestManager netfulfilledman;
#include <memory>
// Fulfilled requests are used to prevent nodes from asking for the same data on sync
// and from being banned for doing so too often.
class CNetFulfilledRequestManager
template<typename T>
class CFlatDB;
class CNetFulfilledRequestManager;
class NetFulfilledRequestStore
{
private:
protected:
typedef std::map<std::string, int64_t> fulfilledreqmapentry_t;
typedef std::map<CService, fulfilledreqmapentry_t> fulfilledreqmap_t;
protected:
//keep track of what node has/was asked for and when
fulfilledreqmap_t mapFulfilledRequests;
mutable RecursiveMutex cs_mapFulfilledRequests;
public:
CNetFulfilledRequestManager() {}
SERIALIZE_METHODS(CNetFulfilledRequestManager, obj)
SERIALIZE_METHODS(NetFulfilledRequestStore, obj)
{
LOCK(obj.cs_mapFulfilledRequests);
READWRITE(obj.mapFulfilledRequests);
}
void Clear();
std::string ToString() const;
};
// Fulfilled requests are used to prevent nodes from asking for the same data on sync
// and from being banned for doing so too often.
class CNetFulfilledRequestManager : public NetFulfilledRequestStore
{
private:
using db_type = CFlatDB<NetFulfilledRequestStore>;
private:
const std::unique_ptr<db_type> m_db;
const bool is_valid{false};
public:
explicit CNetFulfilledRequestManager(bool load_cache);
~CNetFulfilledRequestManager();
bool IsValid() const { return is_valid; }
void CheckAndRemove();
void AddFulfilledRequest(const CService& addr, const std::string& strRequest);
bool HasFulfilledRequest(const CService& addr, const std::string& strRequest);
void RemoveAllFulfilledRequests(const CService& addr);
void CheckAndRemove();
void Clear();
std::string ToString() const;
void DoMaintenance();
};
extern std::unique_ptr<CNetFulfilledRequestManager> netfulfilledman;
#endif // BITCOIN_NETFULFILLEDMAN_H

View File

@ -1303,7 +1303,8 @@ static UniValue BuildDMNListEntry(CWallet* pwallet, const CDeterministicMN& dmn,
}
#endif
auto metaInfo = mmetaman.GetMetaInfo(dmn.proTxHash);
CHECK_NONFATAL(mmetaman != nullptr);
auto metaInfo = mmetaman->GetMetaInfo(dmn.proTxHash);
o.pushKV("metaInfo", metaInfo->ToJson());
return o;

View File

@ -6,6 +6,7 @@
#include <chainparams.h>
#include <consensus/params.h>
#include <flat-database.h>
#include <key_io.h>
#include <logging.h>
#include <messagesigner.h>
@ -25,6 +26,8 @@
std::unique_ptr<CSporkManager> sporkManager;
const std::string SporkStore::SERIALIZATION_VERSION_STRING = "CSporkManager-Version-2";
std::optional<SporkValue> CSporkManager::SporkValueIfActive(SporkId nSporkID) const
{
AssertLockHeld(cs);
@ -56,7 +59,7 @@ std::optional<SporkValue> CSporkManager::SporkValueIfActive(SporkId nSporkID) co
return std::nullopt;
}
void CSporkManager::Clear()
void SporkStore::Clear()
{
LOCK(cs);
mapSporksActive.clear();
@ -65,10 +68,32 @@ void CSporkManager::Clear()
// we should not alter them here.
}
CSporkManager::CSporkManager() :
m_db{std::make_unique<db_type>("sporks.dat", "magicSporkCache")}
{
}
CSporkManager::~CSporkManager()
{
if (!is_valid) return;
m_db->Store(*this);
}
bool CSporkManager::LoadCache()
{
assert(m_db != nullptr);
is_valid = m_db->Load(*this);
if (is_valid) {
CheckAndRemove();
}
return is_valid;
}
void CSporkManager::CheckAndRemove()
{
LOCK(cs);
assert(!setSporkPubKeyIDs.empty());
if (setSporkPubKeyIDs.empty()) return;
for (auto itActive = mapSporksActive.begin(); itActive != mapSporksActive.end();) {
auto itSignerPair = itActive->second.begin();
@ -327,7 +352,7 @@ bool CSporkManager::SetPrivKey(const std::string& strPrivKey)
return true;
}
std::string CSporkManager::ToString() const
std::string SporkStore::ToString() const
{
LOCK(cs);
return strprintf("Sporks: %llu", mapSporksActive.size());

View File

@ -20,6 +20,8 @@
#include <vector>
class CConnman;
template<typename T>
class CFlatDB;
class CNode;
class CDataStream;
class PeerManager;
@ -155,41 +157,17 @@ public:
void Relay(CConnman& connman) const;
};
/**
* CSporkManager is a higher-level class which manages the node's spork
* messages, rules for which sporks should be considered active/inactive, and
* processing for certain sporks (e.g. spork 12).
*/
class CSporkManager
class SporkStore
{
private:
static constexpr std::string_view SERIALIZATION_VERSION_STRING = "CSporkManager-Version-2";
mutable Mutex cs_mapSporksCachedActive;
mutable std::unordered_map<const SporkId, bool> mapSporksCachedActive GUARDED_BY(cs_mapSporksCachedActive);
mutable Mutex cs_mapSporksCachedValues;
mutable std::unordered_map<SporkId, SporkValue> mapSporksCachedValues GUARDED_BY(cs_mapSporksCachedValues);
protected:
static const std::string SERIALIZATION_VERSION_STRING;
mutable Mutex cs;
std::unordered_map<uint256, CSporkMessage, StaticSaltedHasher> mapSporksByHash GUARDED_BY(cs);
std::unordered_map<SporkId, std::map<CKeyID, CSporkMessage> > mapSporksActive GUARDED_BY(cs);
std::set<CKeyID> setSporkPubKeyIDs GUARDED_BY(cs);
int nMinSporkKeys GUARDED_BY(cs) {std::numeric_limits<int>::max()};
CKey sporkPrivKey GUARDED_BY(cs);
/**
* SporkValueIfActive is used to get the value agreed upon by the majority
* of signed spork messages for a given Spork ID.
*/
std::optional<SporkValue> SporkValueIfActive(SporkId nSporkID) const EXCLUSIVE_LOCKS_REQUIRED(cs);
public:
CSporkManager() = default;
template<typename Stream>
void Serialize(Stream &s) const LOCKS_EXCLUDED(cs)
{
@ -221,6 +199,50 @@ public:
*/
void Clear() LOCKS_EXCLUDED(cs);
/**
* ToString returns the string representation of the SporkManager.
*/
std::string ToString() const LOCKS_EXCLUDED(cs);
};
/**
* CSporkManager is a higher-level class which manages the node's spork
* messages, rules for which sporks should be considered active/inactive, and
* processing for certain sporks (e.g. spork 12).
*/
class CSporkManager : public SporkStore
{
private:
using db_type = CFlatDB<SporkStore>;
private:
const std::unique_ptr<db_type> m_db;
bool is_valid{false};
mutable Mutex cs_mapSporksCachedActive;
mutable std::unordered_map<const SporkId, bool> mapSporksCachedActive GUARDED_BY(cs_mapSporksCachedActive);
mutable Mutex cs_mapSporksCachedValues;
mutable std::unordered_map<SporkId, SporkValue> mapSporksCachedValues GUARDED_BY(cs_mapSporksCachedValues);
std::set<CKeyID> setSporkPubKeyIDs GUARDED_BY(cs);
int nMinSporkKeys GUARDED_BY(cs) {std::numeric_limits<int>::max()};
CKey sporkPrivKey GUARDED_BY(cs);
/**
* SporkValueIfActive is used to get the value agreed upon by the majority
* of signed spork messages for a given Spork ID.
*/
std::optional<SporkValue> SporkValueIfActive(SporkId nSporkID) const EXCLUSIVE_LOCKS_REQUIRED(cs);
public:
CSporkManager();
~CSporkManager();
bool LoadCache();
bool IsValid() const { return is_valid; }
/**
* CheckAndRemove is defined to fulfill an interface as part of the on-disk
* cache used to cache sporks between runs. If sporks that are restored
@ -244,7 +266,6 @@ public:
*/
void ProcessSpork(const CNode& peer, PeerManager& peerman, CConnman& connman, CDataStream& vRecv) LOCKS_EXCLUDED(cs);
/**
* ProcessGetSporks is used to handle the 'getsporks' p2p message.
*
@ -315,11 +336,6 @@ public:
* address in the set of valid spork signers (see SetSporkAddress).
*/
bool SetPrivKey(const std::string& strPrivKey) LOCKS_EXCLUDED(cs);
/**
* ToString returns the string representation of the SporkManager.
*/
std::string ToString() const LOCKS_EXCLUDED(cs);
};
#endif // BITCOIN_SPORK_H

View File

@ -11,10 +11,13 @@
#include <consensus/params.h>
#include <consensus/validation.h>
#include <crypto/sha256.h>
#include <flat-database.h>
#include <governance/governance.h>
#include <index/txindex.h>
#include <init.h>
#include <interfaces/chain.h>
#include <masternode/meta.h>
#include <netfulfilledman.h>
#include <llmq/blockprocessor.h>
#include <llmq/chainlocks.h>
#include <llmq/context.h>
@ -206,6 +209,8 @@ ChainTestingSetup::ChainTestingSetup(const std::string& chainName, const std::ve
::sporkManager = std::make_unique<CSporkManager>();
::governance = std::make_unique<CGovernanceManager>();
::masternodeSync = std::make_unique<CMasternodeSync>(*m_node.connman, *::governance);
::mmetaman = std::make_unique<CMasternodeMetaMan>(/* load_cache */ false);
::netfulfilledman = std::make_unique<CNetFulfilledRequestManager>(/* load_cache */ false);
m_node.creditPoolManager = std::make_unique<CCreditPoolManager>(*m_node.evodb);
@ -222,6 +227,8 @@ ChainTestingSetup::~ChainTestingSetup()
StopScriptCheckWorkerThreads();
GetMainSignals().FlushBackgroundCallbacks();
GetMainSignals().UnregisterBackgroundSignalScheduler();
::netfulfilledman.reset();
::mmetaman.reset();
::masternodeSync.reset();
::governance.reset();
::sporkManager.reset();