3339ba28e9 Make g_enable_bip61 a member variable of PeerLogicValidation (Jesse Cohen)
6690a28606 Restrict as much as possible in net_processing to translation unit (Jesse Cohen)
1d4df02b7e [move-only] Move things only referenced in net_processing out of header file (Jesse Cohen)
02bbc05310 Rescope g_enable_bip61 to net_processing (Jesse Cohen)
Pull request description:
As part of a larger effort to decouple net_processing and validation a bit, these are a bunch of simple scope cleanups. I've moved things out of the header file that are only referenced in net_processing and added static (or anonymous namespace) modifiers to everything possible in net_processing.
There are a handful of functions which could be static except that they are exposed for the sake of unit testing - these are explicitly commented. There has been some discussion of a compile time annotation, but no conclusion has been reached on that yet.
This is somewhat related to other prs #12934#13413#13407 and will be followed by prs that reduce reliance on cs_main to synchronize data structures which are translation unit local to net_processing
Tree-SHA512: 46c9660ee4e06653feb42ba92189565b0aea17aac2375c20747c0d091054c63829cbf66d2daddf65682b58ce1d6922e23aefea051a7f2c8abbb6db253a609082
Signed-off-by: Pasta <pasta@dashboost.org>
# Conflicts:
# src/init.cpp
# src/net_processing.cpp
# src/net_processing.h
# src/test/test_dash.cpp
* scripted-diff: Replace #include "" with #include <> (ryanofsky)
-BEGIN VERIFY SCRIPT-
for f in \
src/*.cpp \
src/*.h \
src/bench/*.cpp \
src/bench/*.h \
src/compat/*.cpp \
src/compat/*.h \
src/consensus/*.cpp \
src/consensus/*.h \
src/crypto/*.cpp \
src/crypto/*.h \
src/crypto/ctaes/*.h \
src/policy/*.cpp \
src/policy/*.h \
src/primitives/*.cpp \
src/primitives/*.h \
src/qt/*.cpp \
src/qt/*.h \
src/qt/test/*.cpp \
src/qt/test/*.h \
src/rpc/*.cpp \
src/rpc/*.h \
src/script/*.cpp \
src/script/*.h \
src/support/*.cpp \
src/support/*.h \
src/support/allocators/*.h \
src/test/*.cpp \
src/test/*.h \
src/wallet/*.cpp \
src/wallet/*.h \
src/wallet/test/*.cpp \
src/wallet/test/*.h \
src/zmq/*.cpp \
src/zmq/*.h
do
base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
Signed-off-by: Pasta <pasta@dashboost.org>
* scripted-diff: Replace #include "" with #include <> (Dash Specific)
-BEGIN VERIFY SCRIPT-
for f in \
src/bls/*.cpp \
src/bls/*.h \
src/evo/*.cpp \
src/evo/*.h \
src/governance/*.cpp \
src/governance/*.h \
src/llmq/*.cpp \
src/llmq/*.h \
src/masternode/*.cpp \
src/masternode/*.h \
src/privatesend/*.cpp \
src/privatesend/*.h
do
base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
Signed-off-by: Pasta <pasta@dashboost.org>
* build: Remove -I for everything but project root
Remove -I from build system for everything but the project root,
and built-in dependencies.
Signed-off-by: Pasta <pasta@dashboost.org>
# Conflicts:
# src/Makefile.test.include
* qt: refactor: Use absolute include paths in .ui files
* qt: refactor: Changes to make include paths absolute
This makes all include paths in the GUI absolute.
Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.
Signed-off-by: Pasta <pasta@dashboost.org>
# Conflicts:
# src/qt/dash.cpp
# src/qt/optionsmodel.cpp
# src/qt/test/rpcnestedtests.cpp
* test: refactor: Use absolute include paths for test data files
* Recommend #include<> syntax in developer notes
* refactor: Include obj/build.h instead of build.h
* END BACKPORT #11651 Remove trailing whitespace causing travis failure
* fix backport 11651
Signed-off-by: Pasta <pasta@dashboost.org>
* More of 11651
* fix blockchain.cpp
Signed-off-by: pasta <pasta@dashboost.org>
* Add missing "qt/" in includes
* Add missing "test/" in includes
* Fix trailing whitespaces
Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
6262915 Add unit test for stale tip checking (Suhas Daftuar)
83df257 Add CConnmanTest to mutate g_connman in tests (João Barbosa)
ac7b37c Connect to an extra outbound peer if our tip is stale (Suhas Daftuar)
db32a65 Track tip update time and last new block announcement from each peer (Suhas Daftuar)
2d4327d net: Allow connecting to extra outbound peers (Suhas Daftuar)
Pull request description:
This is an alternative approach to #11534. Rather than disconnect an outbound peer when our tip looks stale, instead try to connect to an additional outbound peer.
Periodically, check to see if we have more outbound peers than we target (ie if any extra peers are in use), and if so, disconnect the one that least recently announced a new block (breaking ties by choosing the newest peer that we connected to).
Tree-SHA512: 8f19e910e0bb36867f81783e020af225f356451899adfc7ade1895d6d3bd5afe51c83759610dfd10c62090c4fe404efa0283b2f63fde0bd7da898a1aaa7fb281
e065249 Add unit test for outbound peer eviction (Suhas Daftuar)
5a6d00c Permit disconnection of outbound peers on bad/slow chains (Suhas Daftuar)
c60fd71 Disconnecting from bad outbound peers in IBD (Suhas Daftuar)
Pull request description:
The first commit will disconnect an outbound peer that serves us a headers chain with insufficient work while we're in IBD.
The second commit introduces a way to disconnect outbound peers whose chains fall out of sync with ours:
For a given outbound peer, we check whether their best known block (which is known from the blocks they announce to us) has at least as much work as our tip. If it doesn't, we set a 20 minute timeout, and if we still haven't heard about a block with as much work as our tip had when we set the timeout, then we send a single getheaders message, and wait 2 more minutes. If after two minutes their best known block has insufficient work, we disconnect that peer.
We protect 4 of our outbound peers (who provide some "good" headers chains, ie a chain with at least as much work as our tip at some point) from being subject to this logic, to prevent excessive network topology changes as a result of this algorithm, while still ensuring that we have a reasonable number of nodes not known to be on bogus chains.
We also don't require our peers to be on the same chain as us, to prevent accidental partitioning of the network in the event of a chain split. Note that if our peers are ever on a more work chain than our tip, then we will download and validate it, and then either reorg to it, or learn of a consensus incompatibility with that peer and disconnect. This PR is designed to protect against peers that are on a less work chain which we may never try to download and validate.
Tree-SHA512: 2e0169a1dd8a7fb95980573ac4a201924bffdd724c19afcab5efcef076fdbe1f2cec7dc5f5d7e0a6327216f56d3828884f73642e00c8534b56ec2bb4c854a656
2525b972a net: stop both net/net_processing before destroying them (Cory Fields)
80e2e9d0c net: drop unused connman param (Cory Fields)
8ad663c1f net: use an interface class rather than signals for message processing (Cory Fields)
28f11e940 net: pass CConnman via pointer rather than reference (Cory Fields)
Pull request description:
See individual commits.
Benefits:
- Allows us to begin moving stuff out of CNode and into CNodeState (after #10652 and follow-ups)
- Drops boost dependency and overhead
- Drops global signal registration
- Friendlier backtraces
Tree-SHA512: af2038c959dbec25f0c90c74c88dc6a630e6b9e984adf52aceadd6954aa463b6aadfccf979c2459a9f3354326b5077ee02048128eda2a649236fadb595b66ee3
3fb81a8 Use list initialization (C++11) for maps/vectors instead of boost::assign::map_list_of/list_of (practicalswift)
Tree-SHA512: 63a9ac9ec5799472943dce1cd92a4b14e7f1fe12758a5fc4b1efceaf2c85a4ba71dad5ccc50813527f18b192e7714c076e2478ecd6ca0d452b24e88416f872f7
e94584858 scripted-diff: Use new naming style for insecure_rand* functions (Pieter Wuille)
2fcd9cc86 scripted-diff: Use randbits/bool instead of randrange where possible (Pieter Wuille)
2ada67852 Use randbits instead of ad-hoc emulation in prevector tests (Pieter Wuille)
5f0b04eed Replace rand() & ((1 << N) - 1) with randbits(N) (Pieter Wuille)
3ecabae36 Replace more rand() % NUM by randranges (Pieter Wuille)
efee1db21 scripted-diff: use insecure_rand256/randrange more (Pieter Wuille)
1119927df Add various insecure_rand wrappers for tests (Pieter Wuille)
124d13a58 Merge test_random.h into test_bitcoin.h (Pieter Wuille)
90620d66c scripted-diff: Rename cuckoo tests' local rand context (Pieter Wuille)
37e864eb9 Add FastRandomContext::rand256() and ::randbytes() (Pieter Wuille)
Tree-SHA512: d09705a3ec718ae792f7d66a75401903ba7b9c9d3fc36669d6e3b9242f0194738106be26baefc8a8e3fa6df7c9a35978c71c0c430278a028b331df23a3ea3070
3457331 test: Add test for `getpeerinfo` `bindaddr` field (Wladimir J. van der Laan)
a7e3c28 rpc: Add listen address to incoming connections in `getpeerinfo` (Wladimir J. van der Laan)
Tree-SHA512: bcd58bca2d35fc9698e958e22a7cf8268a6c731a3a309df183f43fc5e725a88ae09f006290fde7aa03cee9a403e2e25772097409677cedbce8f267e01e9040f6
59ac5c5 net: Use deterministic randomness for CNode's nonce, and make it const (Cory Fields)
aff6584 net: constify a few CNode vars to indicate that they're threadsafe (Cory Fields)
54326a6 Increase maximum orphan size to 100,000 bytes. (Gregory Maxwell)
8c99d1b Treat orphans as implicit inv for parents, discard when parents rejected. (Gregory Maxwell)
11cc143 Adds an expiration time for orphan tx. (Gregory Maxwell)
db0ffe8 This eliminates the primary leak that causes the orphan map to always grow to its maximum size. (Gregory Maxwell)
1b0bcc5 Track orphan by prev COutPoint rather than prev hash (Pieter Wuille)
* Dont deserialize nVersion into CNode, should fix#9212
* net: deserialize the entire version message locally
This avoids having some vars set if the version negotiation fails.
Also copy it all into CNode at the same site. nVersion and
fSuccessfullyConnected are set last, as they are the gates for the other vars.
Make them atomic for that reason.
* net: don't run callbacks on nodes that haven't completed the version handshake
Since ForEach* are can be used to send messages to all nodes, the caller may
end up sending a message before the version handshake is complete. To limit
this, filter out these nodes. While we're at it, may as well filter out
disconnected nodes as well.
Delete unused methods rather than updating them.
* net: Disallow sending messages until the version handshake is complete
This is a change in behavior, though it's much more sane now than before.
* net: log an error rather than asserting if send version is misused
Also cleaned up the comments and moved from the header to the .cpp so that
logging headers aren't needed from net.h
* Implement conditions for ForEachNode() and ForNode() methods of CConnman.
A change making ForEachNode() and ForNode() methods ignore nodes that
have not completed initial handshake have been backported from Bitcoin.
Unfortunately, some Dash-specific code needs to iterate over all nodes.
This change introduces additional condition argument to these methods.
This argument is a functional object that should return true for nodes
that should be taken into account, not ignored.
Two functional objects are provided in CConnman namespace:
* FullyConnectedOnly returns true for nodes that have handshake completed,
* AllNodes returns true for all nodes.
Overloads for ForEachNode() and ForNode() methods without condition argument
are left for compatibility with non-Dash-specific code.
They use FullyConnectedOnly functional object for condition.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
* Iterate over all nodes in Dash-specific code using AllNodes condition.
Use AllNodes functional object as newly introduced condition argument for
ForEachNode() and ForNode() methods of CConnman to iterate over all nodes
where needed in Dash-specific code.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
* net: a few small cleanups before replacing boost threads
- Drop the interruption point directly after the pnode allocation. This would
be leaky if hit.
- Rearrange thread creation so that the socket handler comes first
* net: add CThreadInterrupt and InterruptibleSleep
* net: make net interruptible
Also now that net threads are interruptible, switch them to use std
threads/binds/mutexes/condvars.
* net: make net processing interruptible
* net: remove thread_interrupted catch
This is now a std::thread, so there's no hope of catching a boost interruption
point.
* net: make proxy receives interruptible
* net: misc header cleanups
* Remove orphan state wipe from UnloadBlockIndex.
As orphan state is now "network state", like in
d6ea737be1,
UnloadBlockIndex is only used during init if we end up reindexing
to clear our block state so that we can start over. However, at
that time no connections have been brought up as CConnman hasn't
been started yet, so all of the network processing state logic is
empty when its called.
* Move network-msg-processing code out of main to its own file
* Rename the remaining main.{h,cpp} to validation.{h,cpp}
* serialization: teach serializers variadics
Also add a variadic CDataStream ctor for ease-of-use.
* connman is in charge of pushing messages
The changes here are dense and subtle, but hopefully all is more explicit
than before.
- CConnman is now in charge of sending data rather than the nodes themselves.
This is necessary because many decisions need to be made with all nodes in
mind, and a model that requires the nodes calling up to their manager quickly
turns to spaghetti.
- The per-node-serializer (ssSend) has been replaced with a (quasi-)const
send-version. Since the send version for serialization can only change once
per connection, we now explicitly tag messages with INIT_PROTO_VERSION if
they are sent before the handshake. With this done, there's no need to lock
for access to nSendVersion.
Also, a new stream is used for each message, so there's no need to lock
during the serialization process.
- This takes care of accounting for optimistic sends, so the
nOptimisticBytesWritten hack can be removed.
- -dropmessagestest and -fuzzmessagestest have not been preserved, as I suspect
they haven't been used in years.
* net: switch all callers to connman for pushing messages
Drop all of the old stuff.
* drop the optimistic write counter hack
This is now handled properly in realtime.
* net: remove now-unused ssSend and Fuzz
* net: construct CNodeStates in place
* net: handle version push in InitializeNode
* net: move CBanDB and CAddrDB out of net.h/cpp
This will eventually solve a circular dependency
* net: Create CConnman to encapsulate p2p connections
* net: Move socket binding into CConnman
* net: move OpenNetworkConnection into CConnman
* net: move ban and addrman functions into CConnman
* net: Add oneshot functions to CConnman
* net: move added node functions to CConnman
* net: Add most functions needed for vNodes to CConnman
* net: handle nodesignals in CConnman
* net: Pass CConnection to wallet rather than using the global
* net: Add rpc error for missing/disabled p2p functionality
* net: Pass CConnman around as needed
* gui: add NodeID to the peer table
* net: create generic functor accessors and move vNodes to CConnman
* net: move whitelist functions into CConnman
* net: move nLastNodeId to CConnman
* net: move nLocalHostNonce to CConnman
This behavior seems to have been quite racy and broken.
Move nLocalHostNonce into CNode, and check received nonces against all
non-fully-connected nodes. If there's a match, assume we've connected
to ourself.
* net: move messageHandlerCondition to CConnman
* net: move send/recv statistics to CConnman
* net: move SendBufferSize/ReceiveFloodSize to CConnman
* net: move nLocalServices/nRelevantServices to CConnman
These are in-turn passed to CNode at connection time. This allows us to offer
different services to different peers (or test the effects of doing so).
* net: move semOutbound and semMasternodeOutbound to CConnman
* net: SocketSendData returns written size
* net: move max/max-outbound to CConnman
* net: Pass best block known height into CConnman
CConnman then passes the current best height into CNode at creation time.
This way CConnman/CNode have no dependency on main for height, and the signals
only move in one direction.
This also helps to prevent identity leakage a tiny bit. Before this change, an
attacker could theoretically make 2 connections on different interfaces. They
would connect fully on one, and only establish the initial connection on the
other. Once they receive a new block, they would relay it to your first
connection, and immediately commence the version handshake on the second. Since
the new block height is reflected immediately, they could attempt to learn
whether the two connections were correlated.
This is, of course, incredibly unlikely to work due to the small timings
involved and receipt from other senders. But it doesn't hurt to lock-in
nBestHeight at the time of connection, rather than letting the remote choose
the time.
* net: pass CClientUIInterface into CConnman
* net: Drop StartNode/StopNode and use CConnman directly
* net: Introduce CConnection::Options to avoid passing so many params
* net: add nSendBufferMaxSize/nReceiveFloodSize to CConnection::Options
* net: move vNodesDisconnected into CConnman
* Made the ForEachNode* functions in src/net.cpp more pragmatic and self documenting
* Convert ForEachNode* functions to take a templated function argument rather than a std::function to eliminate std::function overhead
* net: move MAX_FEELER_CONNECTIONS into connman
* Only store and connect to NODE_NETWORK nodes
* Keep addrman's nService bits consistent with outbound observations
* Verify that outbound connections have expected services
* Don't require services in -addnode
* Introduce enum ServiceFlags for service flags
* Introduce REQUIRED_SERVICES constant
We used to have a trickle node, a node which was chosen in each iteration of
the send loop that was privileged and allowed to send out queued up non-time
critical messages. Since the removal of the fixed sleeps in the network code,
this resulted in fast and attackable treatment of such broadcasts.
This pull request changes the 3 remaining trickle use cases by random delays:
* Local address broadcast (while also removing the the wiping of the seen filter)
* Address relay
* Inv relay (for transactions; blocks are always relayed immediately)
The code is based on older commits by Patrick Strateman.
Github-Pull: #7125
Rebased-From: 5400ef6bcb
Prevent denial-of-service attacks by banning
peers that send us invalid orphan transactions
and only storing orphan transactions given to
us by a peer while the peer is connected.
Split up util.cpp/h into:
- string utilities (hex, base32, base64): no internal dependencies, no dependency on boost (apart from foreach)
- money utilities (parsesmoney, formatmoney)
- time utilities (gettime*, sleep, format date):
- and the rest (logging, argument parsing, config file parsing)
The latter is basically the environment and OS handling,
and is stripped of all utility functions, so we may want to
rename it to something else than util.cpp/h for clarity (Matt suggested
osinterface).
Breaks dependency of sha256.cpp on all the things pulled in by util.
Two changes:
First removes a unit test that fails in my development environment
(OSX, compiled -g3 with clang).
sipa says that's not terribly surprising; the CMutableTransaction change
makes signing a little more expensive but verification quicker. The unit
test timed sign+verify-uncached versus verify-cached-five-times.
He also says the test will be invalid when libsec256kp1 is integrated
(because validation is super-optimized over signing).
core.h change fixes a compiler warning (clang -Wall : CMutableTransaction defined
as struct, declared as class in script.h).
- Add license headers to source files (years based on commit dates)
in `src/test` as well as `qa`
- Add `README.md` to `src/test/data` specifying MIT license
Fixes#3848
