* Remove orphan state wipe from UnloadBlockIndex.
As orphan state is now "network state", like in
d6ea737be19a0001e69e4e854eb1cef21523ea7a,
UnloadBlockIndex is only used during init if we end up reindexing
to clear our block state so that we can start over. However, at
that time no connections have been brought up as CConnman hasn't
been started yet, so all of the network processing state logic is
empty when its called.
* Move network-msg-processing code out of main to its own file
* Rename the remaining main.{h,cpp} to validation.{h,cpp}
* serialization: teach serializers variadics
Also add a variadic CDataStream ctor for ease-of-use.
* connman is in charge of pushing messages
The changes here are dense and subtle, but hopefully all is more explicit
than before.
- CConnman is now in charge of sending data rather than the nodes themselves.
This is necessary because many decisions need to be made with all nodes in
mind, and a model that requires the nodes calling up to their manager quickly
turns to spaghetti.
- The per-node-serializer (ssSend) has been replaced with a (quasi-)const
send-version. Since the send version for serialization can only change once
per connection, we now explicitly tag messages with INIT_PROTO_VERSION if
they are sent before the handshake. With this done, there's no need to lock
for access to nSendVersion.
Also, a new stream is used for each message, so there's no need to lock
during the serialization process.
- This takes care of accounting for optimistic sends, so the
nOptimisticBytesWritten hack can be removed.
- -dropmessagestest and -fuzzmessagestest have not been preserved, as I suspect
they haven't been used in years.
* net: switch all callers to connman for pushing messages
Drop all of the old stuff.
* drop the optimistic write counter hack
This is now handled properly in realtime.
* net: remove now-unused ssSend and Fuzz
* net: construct CNodeStates in place
* net: handle version push in InitializeNode
* net: move CBanDB and CAddrDB out of net.h/cpp
This will eventually solve a circular dependency
* net: Create CConnman to encapsulate p2p connections
* net: Move socket binding into CConnman
* net: move OpenNetworkConnection into CConnman
* net: move ban and addrman functions into CConnman
* net: Add oneshot functions to CConnman
* net: move added node functions to CConnman
* net: Add most functions needed for vNodes to CConnman
* net: handle nodesignals in CConnman
* net: Pass CConnection to wallet rather than using the global
* net: Add rpc error for missing/disabled p2p functionality
* net: Pass CConnman around as needed
* gui: add NodeID to the peer table
* net: create generic functor accessors and move vNodes to CConnman
* net: move whitelist functions into CConnman
* net: move nLastNodeId to CConnman
* net: move nLocalHostNonce to CConnman
This behavior seems to have been quite racy and broken.
Move nLocalHostNonce into CNode, and check received nonces against all
non-fully-connected nodes. If there's a match, assume we've connected
to ourself.
* net: move messageHandlerCondition to CConnman
* net: move send/recv statistics to CConnman
* net: move SendBufferSize/ReceiveFloodSize to CConnman
* net: move nLocalServices/nRelevantServices to CConnman
These are in-turn passed to CNode at connection time. This allows us to offer
different services to different peers (or test the effects of doing so).
* net: move semOutbound and semMasternodeOutbound to CConnman
* net: SocketSendData returns written size
* net: move max/max-outbound to CConnman
* net: Pass best block known height into CConnman
CConnman then passes the current best height into CNode at creation time.
This way CConnman/CNode have no dependency on main for height, and the signals
only move in one direction.
This also helps to prevent identity leakage a tiny bit. Before this change, an
attacker could theoretically make 2 connections on different interfaces. They
would connect fully on one, and only establish the initial connection on the
other. Once they receive a new block, they would relay it to your first
connection, and immediately commence the version handshake on the second. Since
the new block height is reflected immediately, they could attempt to learn
whether the two connections were correlated.
This is, of course, incredibly unlikely to work due to the small timings
involved and receipt from other senders. But it doesn't hurt to lock-in
nBestHeight at the time of connection, rather than letting the remote choose
the time.
* net: pass CClientUIInterface into CConnman
* net: Drop StartNode/StopNode and use CConnman directly
* net: Introduce CConnection::Options to avoid passing so many params
* net: add nSendBufferMaxSize/nReceiveFloodSize to CConnection::Options
* net: move vNodesDisconnected into CConnman
* Made the ForEachNode* functions in src/net.cpp more pragmatic and self documenting
* Convert ForEachNode* functions to take a templated function argument rather than a std::function to eliminate std::function overhead
* net: move MAX_FEELER_CONNECTIONS into connman
* fix vulnerability with mapMasternodeOrphanObjects
The vulnerability is that a malicious node can send a lot of NetMsgType::MNGOVERNANCEOBJECT messages which refer to many arbitrary MN's. In this case, mapMasternodeOrphanObjects will grow unrestrictedly.
* MN collateral moved to governance-object.cpp; ban score applied to misbehaving nodes
* recursive locks removed
* check for the mn collateral code segregated to a separate function
* CheckCollateral implementation moved to cpp
* fix issues with mapSeenGovernanceObjects
Removed seen-governance-objects optimization except for deleted objects. Otherwise some nodes can permanently lost proposals if they received them too early.
Beside of that there is a vulnerability with seen-governance-objects mechanism if malicious node send us a lot of invalid governance objects.
* mapSeenGovernanceObjects renamed to mapErasedGovernanceObjects
* current fixes
* use int64_t for expiration timestamp
* process governance objects in CheckMasternodeOrphanObjects as usual
* code refactoring: SetRateChecksHelper class added
* fixed race condition issues with propagation of governance objects
* change GetCollateralConfirmations signature
* code refactoring
* reduced minimum number of collateral confirmations required for relaying proposals
* bug fixes and improvements
* Implement RequestOrphanObjects
* Ensure governance objects are only requested from peers once per call
* Add gobject log messages
* Implemented CleanOrphanObjects
* Move governance maintenance functions from NewBlock to a time-based function
* Remove unused delcaration for mapAskedForGovernanceObject
* Only accept wd's that are more recent or have a higher hash than the current best
* Fix whitespace typo
* Relay current watchdog when lower priority ones are received
* Fix nHashWatchdogCurrent reset conditions
* expire previous current wd when a new one is found in UpdateCurrentWatchdog
* fail to process votes for expired or deleted object
* Add "enough data" stop conditions for gov sync
* fix:
- make sure condition is checked only once per tick
- let condition be fully used on resync (reset nTimeNoObjectsLeft)
* fix watchdogs:
- do not accept if CreationTime is out of bounds (using CreationTime, not local time now)
- do not sync expired
- fix disk serialization
* drop watchdogs early, avoid adding//removing
* clean mapWatchdogObjects when object is deleted via votes
* Few networking fixes:
- skip "masternode"/inbound connections for sync related processes
- do not sync gov data to other nodes until fully synced ourselves
- do not accept incoming connections until fully synced
* inbound connections could be harmful only if our node is a masternode
* same for CGovernanceManager::Sync
* Adjust gov sync:
- simulate mainnet gov obj sync conditions on testnet
- add redundancy: ask up to 3 peers for the same obj
* stop loop if max number of peers per obj was asked
* Remove vote count check from IsValidLocally
* Do not check voted validity flag when syncing
* Do not send objects marked for deletion during syncing
* Remove node penalty for unrequested objects.
We should remove hash from setAskFor when the message corresponding to previous inv arrives, otherwise it's stays there forever and setAskFor overflows (i.e. AskFor returns immediately without processing).
* On gov sync first sync objs, then ask for votes on per-obj basis from different peers.
This should help to sync obj list initially and split the load among many peers. Also adds ability to catch up votes later after the sync.
* ask for all objects, do this in cycles
* Fix Sync() code, better readability
* ask multiple nodes at once when possible, perf boost for large numper of objs
* Addressed comments: pass reference, more peer version check
* Store time we saw mnb last time, bump sync timeout
if we received seen mnb but we are too close to MASTERNODE_NEW_START_REQUIRED_SECONDS
* Reset blockchain sync status if new blocks were accepted during sync
* Add some debug log output
* wait for at least one new block to be accepted
* bump CGovernanceManager-Version
- some were not used, some were included twice, some were in the wrong place, some were missing (but it compiled because some were in the wrong place)
- organized a bit better, grouped dash specific includes in original bitcoin files, should save some time solving conflicts when/if merging patches later
- added description for 'deserialize'
- added 'type' filter to 'list'
- added 'count' command (changed CGovernanceManager::ToString to output a bit more detailed info)
* Change rate check logic to avoid DoS attacks
* Convert rate check to use object timestamp instead of arrival time
* Update cached variables before checking for superblocks
* Ensure that last times are monotonically non-decreasing
* Bump governance manager serialization format
* Improved rate check error reporting
* Vote relaying changes
- Remove vote relaying from ProcessVote
- Remove vote relaying from orphan vote processing
- Relay vote in ProcessMessages (only)
* Do not relay governance objects during orphan processing
* Restore relaying of local votes
* Changed overloaded function name: ProcessVote->ProcessVoteAndRelay
* Added logging to PushInventory
* Fix LogPrint format
* Log errors found during governance syncing
* Turn off rate checks during syncing
* Turn off rate check during maintenance
* locks in PS
* lock in governance
* locks in IS
* lock in ProcessGetData
* locks in CMasternodeSync
* centralize mnodeman.Check call
* locks order in mnpayments
* use current block chainTip when possible (less locks)
* add missing lock in CountInputsWithAmount
* fix deadlock RequestLowDataPaymentBlocks/IsTransactionValid
* LOCK2 in CheckMnbAndUpdateMasternodeList, CheckAndUpdate, SendVerifyRequest
* LOCK(cs) is not needed here
* Decouple governance init actions from serialization
Should fix this:
```
Assertion failed: lock governance.cs not held in governance-classes.cpp:117; locks held:
cs_Shutdown init.cpp:200 (TRY)
cs ./governance.h:195
cs governance.cpp:835
Abort trap: 6
```
* Increase quorum for object deletion to 2/3 of MN network
* Implement expiration of watchdog objects
* Remove objects from the watchdog map itself
* Message fix for invalid objects
* Fix CGovernanceManager initialization problem
* Added logging messages for cases where CGovernanceManager receives a message while not synced
* Prevent potential NULL pointer dereference
* Added net logging messages in main.cpp
* Added logging for trigger removal
* Improved log message for CGovernanceManager::UpdatedBlockTip
* Improved log messages in CGovernanceManager::UpdateCachesAndClean
* Added more logging to CGovernanceTriggerManager
* Check vote validity before pushing inventory during sync
* Add triggers to map after loading governance.dat file
Squashed:
* Replaced unsafe mnodeman.Find function with Get in governance-vote.cpp
* Reject unparsable governance objects
* Implemented sentinel watchdog objects (separated out from locking changes)
* Added WATCHDOG support to rpcgovernance.cpp
* Implemented WATCHDOG_EXPIRED state for masternodes
* Added serialization of watchdog timestamps
* Masternode fixes
- Added version check to CMasternodeMan deserialization
- Added several missing locking calls in CMasternodeMan
* Fixed missing member initialization in CMasternode constructor and added more logging
* Added MASTERNODE_WATCHDOG_MAX_SECONDS to governanceinfo
* Added masternodewatchdogmaxseconds info to getgovernanceinfo help
* Make masternodes remain in WATCHDOG_EXPIRED state unless removed or collateral expires
* Allow watchdog object creation by WATCHDOG_EXPIRED MN
* Fixed MN validation logic for governance object creation
* Count total masternodes instead of enabled masternodes in masternode-sync
* Transition out of WATCHDOG_EXPIRED state if the watchdog is inactive
* Fixed IsWatchdogExpired bug
* Fixed rate check for watchdog objects and no longer check MN state when validating governance objects
* Applied PR #1061 patch
* Ported locking changes from other branch
* Require only 1 block between new watchdog objects
* Accept pings for WATCHDOG_EXPIRED masternodes
* Lock CmasternodeMan::cs in CmasternodeMan::ProcessMessage
* Several governance changes
- Fixed uninitialized value in CGovernancePayment class
- Return an error on submission if any superblock payment cannot be parsed
- Added logging more statements
* Explicitly initialize all governance object members
* Fix deadlock
* Fixed non-threadsafe access to masternode in activemasternode.cpp
* Revert added wallet lock
* Changed CActiveMasternode so that watchdog expired nodes can still send pings
* Modified CActiveMasternode to run pinger regardless of state when MN is in list
* Added voter and time information to getvotes command
* Improved CActiveMasternode state management
* Implemented GetInfo functions for more efficient thread-safe access to masternode information
* Added CActiveMasternode debug logging messages
* Fixed initial type setting and error message for incorrect protocol version
* Changes based on code review comments
* Set active state for local mode
There is a bug AddOrUpdateVote function in CGovernanceManager. If a new vote has been arrived it is checked if a corresponding parent object are present in the mapObjects. If it is not we need to sync the parent object and return false. But the syncing is never performed because the corresponding code is placed after return statement. So we need to sync and then return.
* GetMinCollateralFee should not validate object type, it should be IsValidLocally's job
* Explicitly set 0 fee for known free objects, set fee to MAX_MONEY for unknown one
* Implemented several governance changes
- Limit strData size to avoid propagation of very large messages
- Remove unused CGovernanceObject::SetData method
- Remove CGovernanceObject::strName field to avoid data redundancy
* Fixed parameter count bug in gobject prepare
* Remove unnecessary call to IsCollateralValid and hence allow superblocks
to propagate
* Added CMasternodeMan::Get overload for masternode vin
* Use vin as masternode identifier instead of public key
* Fixed missing member in copy constructor, improved logging
* Added logging for MasternodeRateCheck failures
* Removed pubkeyMasternode field from CGovernanceObject
* Impose MN trigger creation rate limit only when MN info is synced
* Improve rpcgovernance error handling
- Prevent attempts to prepare trigger objects (and waste the collateral)
- Improve clarity of gobject submit error messages
* trivial governance cleanup:
- spaces
- names
- no "using namespace std;"
- few log and rpc messages adjusted
- remove unused
- use defined types
- move few members to private
* fixing after code review
* Added more specific error message about attempts to submit superblocks by
non-masternodes
* Fixed governance object validation bug
* Fixed logic bug in governance object submission
* Improved exception handling
- Removed attempts to catch exceptions in intermediate helper function calls
- Made helper functions for JSON parsing private
* Governance voting fixes
- Converted voting defines to enums for better type safety
- Enabled gobject voteraw rpc command
- Removed unused parameter from gobject getvotes rpc command
* Fixed help message for gobject get
* Improved encapsulation of CGovernanceVote objects
- CGovernanceVote data members are now private
- Necessary to ensure that enum values are used for signals and outcomes since data members remain int's for compatibility with serialiation code
* Removed unused cs_budget mutex (has been replaced with governance.cs)
* Fix code review issues
- Restored early return in ConvertVoteSignal
- Removed special case for "none" string to make clear that NONE is
for invalid strings
- Removed commented defines (informational comments preserved)
* Fixed code review issues
- Fixed error messages for vote-conf, vote-alias and voteraw
- Removed voteraw from gobject command list and help messages because
it is actually a top-level command
- Fixed parameter indices for voteraw
* Implemented different fees for different types of governance objects
* Added fee amounts to object returned by getgovernanceinfo
* Implement new requireents for Superblock creation
- Superblocks creation requires a valid masternode signature
- Superblock creation no longer requires a collateral fee
- Superblock creation rate is limited to roughly 1 per masternode per cycle
* Fixed getgovernanceinfo help message
* Removed old governance fee constant
* Fixed bug in IsSignatureValid and added debugging code
* Fixed parent hash variable index and added debugging code
* Modified GetBudgetSystemCollateralTX to take fee amount parameter
* Changes due to code review comments
- Naming changes
- Removed confusing comment
da406e3 Masternode sync improvements
- add simple helpers for few more sync states (use them where appropriate instead of old code + rpc output)
- use new helpers to avoid meaningless message processing
- actually fail if sync shouldn't continue due to lack of info, make sure Reset is used to quit failed state
* Improve JSON error reporting in CGovernanceObject::LoadData
* Changed JSON parsing to match current version of sentinel which now sends
correct JSON integers instead of quoting them as strings
068c178 Added DBG macro in util.h to facilitate debugging
- This macro allows debugging statements (typically printf's or cout's) to
be activated or deactivated with a single comment. Uncomment the line:
//#define ENABLE_DASH_DEBUG
in util.h to enable debugging statements.
- When commented any code wrapped with the DBG() macro will simply be removed
by the preprocessor. When not commented all such wrapped statements will
be present.
- For maximum effectiveness it is best that util.h be the first effective include
in all source files. It is also possible to enable the macro for a single file
by temporarily adding #define ENABLE_DASH_DEBUG to the top of the file.
- Code committed to non-development branches should always have the define
commented.
d125d9b V0.12.1.x -- merging trigger/generic object/superblock changes for testnet phase II
- This commit contains the core governance system changes for 0.12.1. Any unrelated
changes have either been removed or moved to separate commits.
120724c File mode fixes
- Changed mode 0755->0644 on several source files.
c7f9e11 Updated todo reminders
- Added reminder to revert temporary reduction of number of votes
required to trigger superblock to 1 for testing
92adc98 Made CSuperblockManager::IsValidSuperblockHeight an inline function
- This is for efficiency since this function is called often and is
only 1 line of code.
c050ed7 Added comment explaining rationale for no LOCK(cs) in CSuperblock::IsValid
dc933fe Removed unused CSuperblockManager::IsBlockValid function
decec88 Moved calls to SuperblockManager::IsValidSuperblockHeight into IsSuperblockTriggered.
- Since calls to the later function are always protected by the former there's
no reason to keep these separate and this simplifies the code in
masternode-payments.cpp.
8672885 Reestablished expected value check for non-superblocks in IsBlockValueValid
b01cbe0 Changes to IsBlockValueValid to fix rpc test failure
a937c76 Changed include order to allow per file activation of the DBG macro
d116aa5 Fixed IsValidSuperblockHeight logic
- Note this has an effect on testing because we can now only create
1 superblock per day. Devs may need to temporarily change testnet params
for easier testing.
2d0c2de Convert superblock payments to CAmount
- We assume that payment values in JSON are in units of DASH
for consistency with other RPC functions, such as
createrawtransaction.
376b833 Revert temporary testing value for nAbsVoteReq
- Also ensure that number of votes required is never smaller than 1
8c89f4b Cleaned up CSuperblock error handling
- Exceptions are now thrown consistently rather than using a mix of
exceptions and return code checking. Exceptions are now caught only
in AddNewTrigger when the CSuperblock constructor is called. Unnecessary object
status members have been removed.
d7c8a6b Removed utilstrencodings header
- This appears to help with travis tests, for unknown reasons.
c4dfc7a Fixed some minor code review issues
63c3580 Reverted locking change in miner.
- This should have been done in the original PR but was overlooked.
4ab72de Fixed variable name to match common practice and bracket formatting
886a678 Improvements to vote conversion code
- Replaced redundantly defined function with inclusion of governance-vote.h
- Replaced magic numbers with their corresponding constant symbols
0a37966 Reordered governance message handling
- mn hash compatibility with 70103
- ignore some requests while syncing
- fix locking/initializing in sync
- do not ban for old mnw
- split budget/governance messages/invs
98b7d29 fix invalid cached govobj values
581b46a added information explaining cached variable meanings
390c9c3 add dirty flag for future use
b76304e added other cached flags into output
487674f Governance object/vote syncing fixes
- disable fCached values
- use two maps for storing votes, by hash and parent-hash/type
- disable part of flatdb.dump (still overwriting)
- fixed govobj/votes relay and sync
15821fe various fixes
- Added const where possible
- Uncommented sync block
- Protocol min 70201
- Fixed bug which flags invalid votes incorrectly
- Formatting
aa8fdd7 fix curly braces
d8e39b1 Fix GetTypeHash bug
- Should not collide based on the outcome
732a8a3 fixed mismatched index for vote map
- Terrible bug which caused governance objects to have their hashes corrupted, causing objects to become invalid
- More proposal -> governance name changes
