There is no reason to use g_connman global variable in ThreadSendAlert()
because reference to CConnman instance is already passed to it as argument.
This was overlooked when refactoring sendalert module, it's time to fix it.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
* Add protocol details for mnv, dsf, dsc, govsync, dseg, ssc, and mnget messages
Remove duplicate ix message
* Update markdown table formatting
* Add CTransaction
* Add links from field name to Common Structures for reference
* Add CService structure and update links
* Fix duplicate headers download in initial sync
Now that initial block download is delayed until the headers sync is done,
it was noticed that the initial headers sync may happen multiple times in
parallel in the case new blocks are announced. This happens because for
every block in INV that is received, a getheaders message is immediately
sent out resulting in a full download of the headers chain starting from
the point of where the initial headers sync is currently at. This happens
once for each peer that announces the new block. This slows down the
initial headers sync and increases the chance of another block being
announced before it is finished, probably leading to the same behavior
as already described, slowing down the sync even more...and so on.
This commit delays sending of GETHEADERS to later in case the chain is too
far behind while a new block gets announced. Header chains will still be
downloaded multiple times, but the downloading will start much closer
to the tip of the chain, so the damage is not that bad anymore.
This ensures that we get all headers from all peers, even if any of them
is on another chain. This should avoid what happened in
https://github.com/bitcoin/bitcoin/pull/8054
which needed to be reverted later.
This fixes the Bitcoin issue https://github.com/bitcoin/bitcoin/issues/6755
* Introduce DelayGetHeadersTime chain param and fix tests
The delaying of GETHEADERS in combination with very old block times in
test cases resulted in the delaying being triggered when the first newly
mined block arrives. This results in a completely stalled sync.
This is fixed by avoiding delaying in when running tests.
* Disconnect peers which are not catched up
Peers which stop sending us headers too early are very likely peers which
did not catch up before and stalled for some reason. We should disconnect
these peers and chose another one to continue.
- add `MASTERNODE_SYNC_WAITING` sync state/asset and initial timeout
- wait for block headers to be downloaded (new signal `AcceptedBlockHeader`)
- wait for blocks to reach best headers tip before switching to masternode list sync (new signal `NotifyHeaderTip`)
- switched sync from `UpdatedBlockTip` to a combination of `AcceptedBlockHeader` and `NotifyHeaderTip`
- all blockchain-related bumps should take place only while we are still syncing blockchain, should be no such bumps after that
* net: Split resolving out of CNetAddr
* net: Split resolving out of CService
* net: Split resolving out of CSubNet
* net: move CNetAddr/CService/CSubNet out of netbase
* net: narrow include scope after moving to netaddress
Net functionality is no longer needed for CAddress/CAddrman/etc. now that
CNetAddr/CService/CSubNet are dumb storage classes.
* net: Add direct tests for new CSubNet constructors
* net: Have LookupNumeric return a CService directly
Also fix up a few small issues:
- Lookup with "badip:port" now sets the port to 0
- Don't allow assert to have side-effects
* net: fixup nits
* net: require lookup functions to specify all arguments
To make it clear where DNS resolves are happening
* net: manually resolve dns seed sources
Note: Some seeds aren't actually returning an IP for their name entries, so
they're being added to addrman with a source of [::].
This commit shouldn't change that behavior, for better or worse.
* net: resolve outside of storage structures
Rather than allowing CNetAddr/CService/CSubNet to launch DNS queries, require
that addresses are already resolved.
This greatly simplifies async resolve logic, and makes it harder to
accidentally leak DNS queries.
* net: disable resolving from storage structures
CNetAddr/CService/CSubnet can no longer resolve DNS.
After #8594 the addrFrom sent by a node is not used anymore at all,
so don't bother sending it.
Also mitigates the privacy issue in (#8616). It doesn't completely solve
the issue as GetLocalAddress is also called in AdvertiseLocal, but at
least when advertising addresses it stands out less as *our* address.
We should learn about new peers via address messages.
An inbound peer connecting to us tells us nothing about
its ability to accept incoming connections from us, so
we shouldn't assume that we can connect to it based on
this.
The vast majority of nodes on the network do not accept
incoming connections, adding them will only slow down
the process of making a successful connection in the
future.
Nodes which have configured themselves to not announce would prefer we
not violate their privacy by announcing them in GETADDR responses.
The use of mocktime in test logic means that comparisons between
GetTime() and GetTimeMicros()/1000000 are unreliable since the former
can use mocktime values while the latter always gets the system clock;
this changes the networking code's inactivity checks to consistently
use the system clock for inactivity comparisons.
Also remove some hacks from setmocktime() that are no longer needed,
now that we're using the system clock for nLastSend and nLastRecv.
* fixed an issue with MasternodeRateCheck always returns true
* additioanal fixes and refactoring (rebase)
* slightly improved CRateBuffer synchronization for newly started nodes
* IBD check uses minimumchain work instead of checkpoints.
This introduces a 'minimum chain work' chainparam which is intended
to be the known amount of work in the chain for the network at the
time of software release. If you don't have this much work, you're
not yet caught up.
This is used instead of the count of blocks test from checkpoints.
This criteria is trivial to keep updated as there is no element of
subjectivity, trust, or position dependence to it. It is also a more
reliable metric of sync status than a block count.
* Remove GetTotalBlocksEstimate and checkpoint tests that test nothing.
GetTotalBlocksEstimate is no longer used and it was the only thing
the checkpoint tests were testing.
Since checkpoints are on their way out it makes more sense to remove
the test file than to cook up a new pointless test.
# Conflicts:
# src/Makefile.test.include
# src/test/Checkpoints_tests.cpp
* IsInitialBlockDownload no longer uses header-only timestamps.
This avoids a corner case (mostly visible on testnet) where bogus
headers can keep nodes in IsInitialBlockDownload.
* Delay parallel block download until chain has sufficient work
nMinimumChainWork is an anti-DoS threshold; wait until we have a proposed
tip with more work than that before downloading blocks towards that tip.
* Add timeout for headers sync
At startup, we choose one peer to serve us the headers chain, until
our best header is close to caught up. Disconnect this peer if more
than 15 minutes + 1ms/expected_header passes and our best header
is still more than 1 day away from current time.
* Introduce assumevalid setting to skip presumed valid scripts.
This disentangles the script validation skipping from checkpoints.
A new option is introduced "assumevalid" which specifies a block whos
ancestors we assume all have valid scriptsigs and so we do not check
them when they are also burried under the best header by two weeks
worth of work.
Unlike checkpoints this has no influence on consensus unless you set
it to a block with an invalid history. Because of this it can be
easily be updated without risk of influencing the network consensus.
This results in a massive IBD speedup.
This approach was independently recommended by Peter Todd and Luke-Jr
since POW based signature skipping (see PR#9180) does not have the
verifiable properties of a specific hash and may create bad incentives.
The downside is that, like checkpoints, the defaults bitrot and older
releases will sync slower. On the plus side users can provide their
own value here, and if they set it to something crazy all that will
happen is more time will be spend validating signatures.
Checkblocks and checklevel are also moved to the hidden debug options:
Especially now that checkblocks has a low default there is little need
to change these settings, and users frequently misunderstand them as
influencing security or IBD speed. By hiding them we offset the
space added by this new option.
* Add consensusParams to FindNextBlocksToDownload
* Adjust check in headers timeout logic to align with 144 blocks in Dash
* net: fix typo causing the wrong receive buffer size
Surprisingly this hasn't been causing me any issues while testing, probably
because it requires lots of large blocks to be flying around.
Send/Recv corks need tests!
* net: make vRecvMsg a list so that we can use splice()
* net: make GetReceiveFloodSize public
This will be needed so that the message processor can cork incoming messages
* net: only disconnect if fDisconnect has been set
These conditions are problematic to check without locking, and we shouldn't be
relying on the refcount to disconnect.
* net: wait until the node is destroyed to delete its recv buffer
when vRecvMsg becomes a private buffer, it won't make sense to allow other
threads to mess with it anymore.
* net: set message deserialization version when it's actually time to deserialize
We'll soon no longer have access to vRecvMsg, and this is more intuitive anyway.
* net: handle message accounting in ReceiveMsgBytes
This allows locking to be pushed down to only where it's needed
Also reuse the current time rather than checking multiple times.
* net: record bytes written before notifying the message processor
* net: Add a simple function for waking the message handler
This may be used publicly in the future
* net: remove useless comments
* net: remove redundant max sendbuffer size check
This is left-over from before there was proper accounting. Hitting 2x the
sendbuffer size should not be possible.
* net: rework the way that the messagehandler sleeps
In order to sleep accurately, the message handler needs to know if _any_ node
has more processing that it should do before the entire thread sleeps.
Rather than returning a value that represents whether ProcessMessages
encountered a message that should trigger a disconnnect, interpret the return
value as whether or not that node has more work to do.
Also, use a global fProcessWake value that can be set by other threads,
which takes precedence (for one cycle) over the messagehandler's decision.
Note that the previous behavior was to only process one message per loop
(except in the case of a bad checksum or invalid header). That was changed in
PR #3180.
The only change here in that regard is that the current node now falls to the
back of the processing queue for the bad checksum/invalid header cases.
* net: add a new message queue for the message processor
This separates the storage of messages from the net and queued messages for
processing, allowing the locks to be split.
* net: add a flag to indicate when a node's process queue is full
Messages are dumped very quickly from the socket handler to the processor, so
it's the depth of the processing queue that's interesting.
The socket handler checks the process queue's size during the brief message
hand-off and pauses if necessary, and the processor possibly unpauses each time
a message is popped off of its queue.
* net: add a flag to indicate when a node's send buffer is full
Similar to the recv flag, but this one indicates whether or not the net's send
buffer is full.
The socket handler checks the send queue when a new message is added and pauses
if necessary, and possibly unpauses after each message is drained from its buffer.
* net: remove cs_vRecvMsg
vRecvMsg is now only touched by the socket handler thread.
The accounting vars (nRecvBytes/nLastRecv/mapRecvBytesPerMsgCmd) are also
only used by the socket handler thread, with the exception of queries from
rpc/gui. These accesses are not threadsafe, but they never were. This needs to
be addressed separately.
Also, update comment describing data flow
* Dont deserialize nVersion into CNode, should fix#9212
* net: deserialize the entire version message locally
This avoids having some vars set if the version negotiation fails.
Also copy it all into CNode at the same site. nVersion and
fSuccessfullyConnected are set last, as they are the gates for the other vars.
Make them atomic for that reason.
* net: don't run callbacks on nodes that haven't completed the version handshake
Since ForEach* are can be used to send messages to all nodes, the caller may
end up sending a message before the version handshake is complete. To limit
this, filter out these nodes. While we're at it, may as well filter out
disconnected nodes as well.
Delete unused methods rather than updating them.
* net: Disallow sending messages until the version handshake is complete
This is a change in behavior, though it's much more sane now than before.
* net: log an error rather than asserting if send version is misused
Also cleaned up the comments and moved from the header to the .cpp so that
logging headers aren't needed from net.h
* Implement conditions for ForEachNode() and ForNode() methods of CConnman.
A change making ForEachNode() and ForNode() methods ignore nodes that
have not completed initial handshake have been backported from Bitcoin.
Unfortunately, some Dash-specific code needs to iterate over all nodes.
This change introduces additional condition argument to these methods.
This argument is a functional object that should return true for nodes
that should be taken into account, not ignored.
Two functional objects are provided in CConnman namespace:
* FullyConnectedOnly returns true for nodes that have handshake completed,
* AllNodes returns true for all nodes.
Overloads for ForEachNode() and ForNode() methods without condition argument
are left for compatibility with non-Dash-specific code.
They use FullyConnectedOnly functional object for condition.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
* Iterate over all nodes in Dash-specific code using AllNodes condition.
Use AllNodes functional object as newly introduced condition argument for
ForEachNode() and ForNode() methods of CConnman to iterate over all nodes
where needed in Dash-specific code.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
* Remove un-needed #includes (what is the policy?)
Data was duplicated in masternode_info_t and CMasternode classes:
* CMasternode is changed to inherit from masternode_info_t
so the data members are inherited rather than repeated
(also inherits unrepeated nTimeLastPing and fInfoValid members;
this slight intrusiveness made up for in simplicity).
* Use in-class member initializers (C++11) for defaults,
so only non-default initializers are required in the lists.
Allows to shorten repetitous constructor initializer lists.
This makes checking for uninitialized data simpler.
* Default constructors are defined as "= default;" if possible.
* masternode_info_t is changed to behave like an aggregate
(but requires over-complicated constructors until c++14).
There are pros and cons here - aggregate initialization
is convenient but implicit).
* Removed user-defined swap functions.
They appear to only be used in operator= definitions,
using the copy-in,swap-and-return idiom:
* Default operator=, where possible.
* Move in class `friend bool operator==` out-of-class.
* Change sync process:
- IsBlockchainSynced(): drop CheckNodeHeight() and all complicated code, use fInitialDownload in UpdatedBlockTip() to switch initial states
- ProcessTick(): detect sleep mode like it was in IsBlockchainSynced(), not by number of masternodes
* Changes for sync in governance:
- do not keep sync alive on ConfirmInventoryRequest()
- skip some governance actions until we are synced to some level
* do not run CMasternodeMan::UpdateLastPaid() until winners list is synced
* start syncing mn list on the same node right after requesting sporks
* replace nTimeLast<Asset> with the unified nTimeLastBumped, bump on UpdatedBlockTip
* fix comments and LogPrintf-s
* remove excessive MASTERNODE_SYNC_IBD
* a bit more descriptive BumpAssetLastTime in few cases
* net: a few small cleanups before replacing boost threads
- Drop the interruption point directly after the pnode allocation. This would
be leaky if hit.
- Rearrange thread creation so that the socket handler comes first
* net: add CThreadInterrupt and InterruptibleSleep
* net: make net interruptible
Also now that net threads are interruptible, switch them to use std
threads/binds/mutexes/condvars.
* net: make net processing interruptible
* net: remove thread_interrupted catch
This is now a std::thread, so there's no hope of catching a boost interruption
point.
* net: make proxy receives interruptible
* net: misc header cleanups
* Remove orphan state wipe from UnloadBlockIndex.
As orphan state is now "network state", like in
d6ea737be19a0001e69e4e854eb1cef21523ea7a,
UnloadBlockIndex is only used during init if we end up reindexing
to clear our block state so that we can start over. However, at
that time no connections have been brought up as CConnman hasn't
been started yet, so all of the network processing state logic is
empty when its called.
* Move network-msg-processing code out of main to its own file
* Rename the remaining main.{h,cpp} to validation.{h,cpp}
* net: Consistent checksum handling
In principle, the checksums of P2P packets are simply 4-byte blobs which
are the first four bytes of SHA256(SHA256(payload)).
Currently they are handled as little-endian 32-bit integers half of the
time, as blobs the other half, sometimes copying the one to the other,
resulting in somewhat confused code.
This PR changes the handling to be consistent both at packet creation
and receiving, making it (I think) easier to understand.
* net: Hardcode protocol sizes and use fixed-size types
The P2P network uses a fixed protocol, these sizes shouldn't change
based on what happens to be the architecture.
* Expose AcceptBlockHeader through main.h
* Split ::HEADERS processing into two separate cs_main locks
This will allow NotifyHeaderTip to be called from an
AcceptBlockHeader wrapper function without holding cs_main.
* Use exposed ProcessNewBlockHeaders from ProcessMessages
* Remove pfrom parameter from ProcessNewBlock
This further decouples ProcessNewBlock from networking/peer logic.
* Replace CValidationState param in ProcessNewBlock with BlockChecked
* Move MarkBlockAsReceived out of ProcessNewMessage
* Remove network state wipe from UnloadBlockIndex.
UnloadBlockIndex is only used during init if we end up reindexing
to clear our block state so that we can start over. However, at
that time no connections have been brought up as CConnman hasn't
been started yet, so all of the network processing state logic is
empty when its called.
Additionally, the initialization of the recentRejects set is moved
to InitPeerLogic.
* Move all calls to CheckBlockIndex out of net-processing logic
This will result in many more calls to CheckBlockIndex when
connecting a list of headers (eg in ::HEADERS messages processing)
but its only enabled in debug mode, and that should mostly just be
during IBD, so it should be OK.
* Move FlushStateToDisk call out of ProcessMessages::TX into ATMP
* Move nTimeBestReceived updating into net processing code
* Make validationinterface.UpdatedBlockTip more verbose
In anticipation of making all the callbacks out of block processing
flow through it. Note that vHashes will always have something in it
since pindexFork != pindexNewTip.
* Remove duplicate nBlocksEstimate cmp (we already checked IsIBD())
* Remove CConnman parameter from ProcessNewBlock/ActivateBestChain
* Remove SyncWithWallets wrapper function
* Move net-processing logic definitions together in main.h
* Use CValidationInterface from chain logic to notify peer logic
This adds a new CValidationInterface subclass, defined in main.h,
to receive notifications of UpdatedBlockTip and use that to push
blocks to peers, instead of doing it directly from
ActivateBestChain.
* Always call UpdatedBlockTip, even if blocks were only disconnected
* Use BlockChecked signal to send reject messages from mapBlockSource
In the case of (for example) an already-running bitcoind, the shutdown sequence
begins before CConnman has been created, leading to a null-pointer dereference
when g_connman->Stop() is called.
Instead, Just let the CConnman dtor take care of stopping.
* serialization: teach serializers variadics
Also add a variadic CDataStream ctor for ease-of-use.
* connman is in charge of pushing messages
The changes here are dense and subtle, but hopefully all is more explicit
than before.
- CConnman is now in charge of sending data rather than the nodes themselves.
This is necessary because many decisions need to be made with all nodes in
mind, and a model that requires the nodes calling up to their manager quickly
turns to spaghetti.
- The per-node-serializer (ssSend) has been replaced with a (quasi-)const
send-version. Since the send version for serialization can only change once
per connection, we now explicitly tag messages with INIT_PROTO_VERSION if
they are sent before the handshake. With this done, there's no need to lock
for access to nSendVersion.
Also, a new stream is used for each message, so there's no need to lock
during the serialization process.
- This takes care of accounting for optimistic sends, so the
nOptimisticBytesWritten hack can be removed.
- -dropmessagestest and -fuzzmessagestest have not been preserved, as I suspect
they haven't been used in years.
* net: switch all callers to connman for pushing messages
Drop all of the old stuff.
* drop the optimistic write counter hack
This is now handled properly in realtime.
* net: remove now-unused ssSend and Fuzz
* net: construct CNodeStates in place
* net: handle version push in InitializeNode
* net: Add fRelayTxes flag
Add a fRelayTxes to keep track of the relay transaction flag
we send to other peers.
* rpc: Add `relaytxes` flag to `getnetworkinfo`
Re-work of PR #7841 by dragongem45.
Closes#7771.
