* serialization: teach serializers variadics
Also add a variadic CDataStream ctor for ease-of-use.
* connman is in charge of pushing messages
The changes here are dense and subtle, but hopefully all is more explicit
than before.
- CConnman is now in charge of sending data rather than the nodes themselves.
This is necessary because many decisions need to be made with all nodes in
mind, and a model that requires the nodes calling up to their manager quickly
turns to spaghetti.
- The per-node-serializer (ssSend) has been replaced with a (quasi-)const
send-version. Since the send version for serialization can only change once
per connection, we now explicitly tag messages with INIT_PROTO_VERSION if
they are sent before the handshake. With this done, there's no need to lock
for access to nSendVersion.
Also, a new stream is used for each message, so there's no need to lock
during the serialization process.
- This takes care of accounting for optimistic sends, so the
nOptimisticBytesWritten hack can be removed.
- -dropmessagestest and -fuzzmessagestest have not been preserved, as I suspect
they haven't been used in years.
* net: switch all callers to connman for pushing messages
Drop all of the old stuff.
* drop the optimistic write counter hack
This is now handled properly in realtime.
* net: remove now-unused ssSend and Fuzz
* net: construct CNodeStates in place
* net: handle version push in InitializeNode
* net: Add fRelayTxes flag
Add a fRelayTxes to keep track of the relay transaction flag
we send to other peers.
* rpc: Add `relaytxes` flag to `getnetworkinfo`
Re-work of PR #7841 by dragongem45.
Closes#7771.
* net: move CBanDB and CAddrDB out of net.h/cpp
This will eventually solve a circular dependency
* net: Create CConnman to encapsulate p2p connections
* net: Move socket binding into CConnman
* net: move OpenNetworkConnection into CConnman
* net: move ban and addrman functions into CConnman
* net: Add oneshot functions to CConnman
* net: move added node functions to CConnman
* net: Add most functions needed for vNodes to CConnman
* net: handle nodesignals in CConnman
* net: Pass CConnection to wallet rather than using the global
* net: Add rpc error for missing/disabled p2p functionality
* net: Pass CConnman around as needed
* gui: add NodeID to the peer table
* net: create generic functor accessors and move vNodes to CConnman
* net: move whitelist functions into CConnman
* net: move nLastNodeId to CConnman
* net: move nLocalHostNonce to CConnman
This behavior seems to have been quite racy and broken.
Move nLocalHostNonce into CNode, and check received nonces against all
non-fully-connected nodes. If there's a match, assume we've connected
to ourself.
* net: move messageHandlerCondition to CConnman
* net: move send/recv statistics to CConnman
* net: move SendBufferSize/ReceiveFloodSize to CConnman
* net: move nLocalServices/nRelevantServices to CConnman
These are in-turn passed to CNode at connection time. This allows us to offer
different services to different peers (or test the effects of doing so).
* net: move semOutbound and semMasternodeOutbound to CConnman
* net: SocketSendData returns written size
* net: move max/max-outbound to CConnman
* net: Pass best block known height into CConnman
CConnman then passes the current best height into CNode at creation time.
This way CConnman/CNode have no dependency on main for height, and the signals
only move in one direction.
This also helps to prevent identity leakage a tiny bit. Before this change, an
attacker could theoretically make 2 connections on different interfaces. They
would connect fully on one, and only establish the initial connection on the
other. Once they receive a new block, they would relay it to your first
connection, and immediately commence the version handshake on the second. Since
the new block height is reflected immediately, they could attempt to learn
whether the two connections were correlated.
This is, of course, incredibly unlikely to work due to the small timings
involved and receipt from other senders. But it doesn't hurt to lock-in
nBestHeight at the time of connection, rather than letting the remote choose
the time.
* net: pass CClientUIInterface into CConnman
* net: Drop StartNode/StopNode and use CConnman directly
* net: Introduce CConnection::Options to avoid passing so many params
* net: add nSendBufferMaxSize/nReceiveFloodSize to CConnection::Options
* net: move vNodesDisconnected into CConnman
* Made the ForEachNode* functions in src/net.cpp more pragmatic and self documenting
* Convert ForEachNode* functions to take a templated function argument rather than a std::function to eliminate std::function overhead
* net: move MAX_FEELER_CONNECTIONS into connman
* lock cs_main for chainActive
ActivateBestChain uses chainActive after releasing the lock; reorder operations
to move all access to synchronized object into existing LOCK(cs_main) block.
* lock cs_main for State/Misbehaving
ProcessMessage calls State(...) and Misbehaving(...) without holding the
required lock; add LOCK(cs_main) blocks.
Tests if addresses are online or offline by briefly connecting to them. These short lived connections are referred to as feeler connections. Feeler connections are designed to increase the number of fresh online addresses in tried by selecting and connecting to addresses in new. One feeler connection is attempted on average once every two minutes.
This change was suggested as Countermeasure 4 in
Eclipse Attacks on Bitcoin’s Peer-to-Peer Network, Ethan Heilman,
Alison Kendler, Aviv Zohar, Sharon Goldberg. ePrint Archive Report
2015/263. March 2015.
As per meeting 2016-03-31
https://bitcoincore.org/en/meetings/2016/03/31/#bad-chain-alerts
The partition checker was producing huge number of false-positives
and was disabled in 0.12.1 on the understanding it would either be
fixed in 0.13 or removed entirely from master if not.
*** Dash specific note: ***
This check was disabled in Dash already.
* Rework addnode behaviour
* Use CNode::addeName to track whether a connection to a name is already open
* A new connection to a previously-connected by-name addednode is only opened when
the previous one closes (even if the name starts resolving to something else)
* At most one connection is opened per addednode (even if the name resolves to multiple)
* Unify the code between ThreadOpenAddedNodeConnections and getaddednodeinfo
* Information about open connections is always returned, and the dns argument becomes a dummy
* An IP address and inbound/outbound is only reported for the (at most 1) open connection
* Prevent duplicate connections where one is by name and another by ip
* Randomize name lookup result in ConnectSocketByName
* fix vulnerability with mapMasternodeOrphanObjects
The vulnerability is that a malicious node can send a lot of NetMsgType::MNGOVERNANCEOBJECT messages which refer to many arbitrary MN's. In this case, mapMasternodeOrphanObjects will grow unrestrictedly.
* MN collateral moved to governance-object.cpp; ban score applied to misbehaving nodes
* recursive locks removed
* check for the mn collateral code segregated to a separate function
* CheckCollateral implementation moved to cpp
This reverts commit 1f828f45ec.
The commit being reverted changed FindNode(const CService& addr)
to make no difference between nodes with the same IP address,
but different ports, but only for regtest network.
As functional tests run several nodes on different ports or the same
IP address (127.0.0.1), this eventually started breaking functional tests.
The only use for regtest network I know is for functional tests,
so it's time to revert that commit.
* fix issues with mapSeenGovernanceObjects
Removed seen-governance-objects optimization except for deleted objects. Otherwise some nodes can permanently lost proposals if they received them too early.
Beside of that there is a vulnerability with seen-governance-objects mechanism if malicious node send us a lot of invalid governance objects.
* mapSeenGovernanceObjects renamed to mapErasedGovernanceObjects
* current fixes
* use int64_t for expiration timestamp
* Add recently accepted blocks and txn to AttemptToEvictConnection.
This protects any not-already-protected peers who were the most
recent four to relay transactions and most recent four to send
blocks to us.
* Allow disconnecting a netgroup with only one member in eviction.
With the latest additions there are enough protective measures that
we can take the training wheels off.
* net: don't import std namespace
This file is about to be broken up into chunks and moved around. Drop the
namespace now rather than requiring other files to use it.
* net: remove unused set
* net: use the exposed GetNodeSignals() rather than g_signals directly
* net: make Ban/Unban/ClearBan functionality consistent
- Ban/Unban/ClearBan call uiInterface.BannedListChanged() as necessary
- Ban/Unban/ClearBan sync to disk if the operation is user-invoked
- Mark node for disconnection automatically when banning
- Lock cs_vNodes while setting disconnected
- Don't spin in a tight loop while setting disconnected
* net: No need to export DumpBanlist
DumpBanList currently does this:
- with lock: take a copy of the banmap
- perform I/O (write out the banmap)
- with lock: mark the banmap non-dirty
If a new ban is added during the I/O operation, it may never be persisted to
disk.
Reorder operations so that the data to be persisted cannot be older than the
time at which the banmap was marked non-dirty.
* Make ProcessNewBlock dbp const and update comment
* Switch reindexing to AcceptBlock in-loop and ActivateBestChain afterwards
* Optimize ActivateBestChain for long chains
* Add -reindex-chainstate that does not rebuild block index
* Report reindexing progress in GUI
* Only store and connect to NODE_NETWORK nodes
* Keep addrman's nService bits consistent with outbound observations
* Verify that outbound connections have expected services
* Don't require services in -addnode
* Introduce enum ServiceFlags for service flags
* Introduce REQUIRED_SERVICES constant
* CAddrDB modified so that when de-serialization code throws an exception Addrman is reset to a clean state
* CAddrDB modified to make unit tests possible
* Regression test created to ensure bug is fixed
* StartNode modifed to clear adrman if CAddrDB::Read returns an error code.
* process governance objects in CheckMasternodeOrphanObjects as usual
* code refactoring: SetRateChecksHelper class added
* fixed race condition issues with propagation of governance objects
* change GetCollateralConfirmations signature
* code refactoring
* reduced minimum number of collateral confirmations required for relaying proposals
* bug fixes and improvements
* banlist: update set dirty to be more fine grained
- move the SetBannedSetDirty(false) call from DumpData() into DumpBanlist()
- ensure we only set false, if the write succeeded
* banlist: better handling of banlist in StartNode()
- only start working on/with banlist data, if reading in the banlist from
disk didn't fail
- as CNode::setBannedIsDirty is false (default) when reading fails, we
don't need to explicitly set it to false to prevent writing
banlist.dat in that case either
* banlist: add more banlist infos to log / add GUI signal
- to match the peers.dat handling also supply a debug.log entry for how
many entries were loaded from banlist.dat and how long it took
- add a GUI init message for loading the banlist (same as with peers.dat)
- move the same message for peers.dat upwards in the code, to be able to
reuse the timing variable nStart and also just log, if our read from
peers.dat didn't fail
* banlist (bugfix): allow CNode::SweepBanned() to run on interval
- allows CNode::SweepBanned() to run, even if !CNode::BannedSetIsDirty(),
because if nBanUntil is over we want the ban to be disabled for these
nodes
* Add hassentinelping to governanceinfo
* sentinelping rpc call
* additional fields in mnp
* sentinel ping implementation
* change sentinel state to byte in mnp
* use adjusted time in sentinel ping
* update nTimeLastWatchdogVote if sentinel ping is actual
* remove unused fields
* bump protocol to 70207
* Fix small issues
- fix the error message text in CActivbeMasternodeUpdateSentinelPing;
- add empty string before public: in CActiveMasternode class declaration;
- rename field sentinelPing in CMasternodePing to sentinelIsActual and change $
- decrease sentinelVersion field size to uint16_t;
* revert proto bump for MIN_... consts
* revert changes in getgovernanceinfo
* Update mn vote time for remote masternodes
- call UpdateWatchdogVoteTime in CMasternodeMan::ProcessMessage
- deserialize masternodeping from the previous version archive without exception
- add ability to set time in UpdateWatchdogVoteTime
- set nTimeLastWatchdogVote to masternode ping sigTime if sentinel is actual
- bump CMasternodeMan::SERIALIZATION_VERSION_STRING
* remove mn state checks and add correct rpc param convertion
* fix var names
* Helper class for version in string and integer form
* String version in sentinel ping
Version format is "x.x.x"
* test for bacward compatibility in serialization
* Change VersionInfo class to convert functions
* Build against system UniValue when available
* doc: Add UniValue to build instructions
* Bugfix: The var is LIBUNIVALUE,not LIBBITCOIN_UNIVALUE
* Change default configure option --with-system-univalue to "no"
* Bugfix: Always include univalue in DIST_SUBDIRS
* LDADD dependency order shuffling
* build-unix: Update UniValue build conditions
ProcessNewBlock would return failure early if CheckBlock failed, before
calling AcceptBlock. AcceptBlock also calls CheckBlock, and upon failure
would update mapBlockIndex to indicate that a block was failed. By returning
early in ProcessNewBlock, we were not marking blocks that fail a check in
CheckBlock as permanently failed, and thus would continue to re-request and
reprocess them.
Adds several unittests for CAddrMan and CAddrInfo.
Increases the accuracy of addrman tests.
Removes non-determinism in tests by overriding the random number generator.
Extracts testing code from addrman class to test class.
* log bytes recv/sent per command
* net: Account for `sendheaders` `verack` messages
Looks like these were forgotten in #6589.
* Backport remaining part of Bitcoin PR bitcoin/bitcoin#7181.
Most of this PR is already merged, but a small part remaining
that makes per-command byte counts in CNode working.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
Fixes: #1345
The actual problem is that GetDataDir has the side effect of creating the
datadir, even if it is not known yet where it really is. This is only
known after reading the config file or when explicitly specified in the
cmd line.
Thus, if GetDataDir gets called before the datadir value from the config
is read, it tries to create it at the default location.
* Implement proposal validation
Includes commits:
Implemented CProposalValidator
Use CProposalValidator to check proposals at prepare and submit stages
Modify proposal validator to support numerical data in string format
Multiple bug fixes in governance-validators.cpp
Fixed bug in CheckURL
Fixed stream state check
Increase strictness of payment address validation for compatibility with sentinel
Improved error reporting
Implemented "check" rpc command to validate proposals
Fixes to RPC check command
Fix error message
Unit test and data files for proposal validator
Added test cases
Removed debugging code
* Fix name validation
* Changes to address code review comments
* HD wallet
Minimal set of changes (no refactoring) backported from Bitcoin upstream to make HD wallets work in Dash 0.12.1.x+
* minimal bip44 (hardcoded account and change)
* minimal bip39
Additional cmd-line options for new wallet:
-mnemonic
-mnemonicpassphrase
* Do not recreate HD wallet on encryption
Adjusted keypool.py test
* Do not store any private keys for hd wallet besides the master one
Derive all keys on the fly.
Original idea/implementation - btc PR9298, backported and improved
* actually use bip39
* pbkdf2 test
* backport wallet-hd.py test
* Allow specifying hd seed, add dumphdseed rpc, fix bugs
- -hdseed cmd-line param to specify HD seed on wallet creation
- dumphdseed rpc to dump HD seed
- allow seed of any size
- fix dumpwallet rpc bug (wasn't decrypting HD seed)
- print HD seed and extended public masterkey on dumpwallet
* top up keypool on HD wallet encryption
* split HD chain: external/internal
* add missing cs_wallet lock in init.cpp
* fix `const char *` issues (use strings)
* default mnemonic passphrase is an empty string in all cases
* store mnemonic/mnemonicpassphrase
replace dumphdseed with dumphdinfo
* Add fCrypted flag to CHDChain
* prepare internal structures for multiple HD accounts
(plus some code cleanup)
* use secure allocator for storing sensitive HD data
* use secure strings for mnemonic(passphrase)
* small fix in GenerateNewHDChain
* use 24 words for mnemonic by default
* make sure mnemonic passphrase provided by user does not exceed 256 symbols
* more usage of secure allocators and memory_cleanse
* code cleanup
* rename: CSecureVector -> SecureVector
* add missing include
* fix warning in rpcdump.cpp
* refactor mnemonic_check (also fix a bug)
* move bip39 functions to CMnemonic
* Few fixes for CMnemonic:
- use `SecureVector` for data, bits, seed
- `Check` should return bool
* init vectors with desired size where possible
* dont waste keys from keypool on failure in CreateDenominated
* bug fix - log actual number of total outputs, comment error
* log number of total outputs as separate value
* add lock so no one can spend outputs used for denominations
* clear trafficgraph on clear button click
* set default sample height
set default sample height so after clearing traffic graph have some
scale
* reduce available traffic graph ranges, add optimized graph data storage
reduce available traffic graph ranges to 10
(5m,10m,15m,30m,1h,2h,3h,6h,12h,24h),
store graph data so range change is possible,
data storage contains only necessary data to create graphs for all
supported ranges
eg. for 10m range storage only half of 10m samples - the second half is
calculated from 5m range samples,
encapsulate all traffic graph related data into one class
* code formatting corrections
The rpcserver.h header file is included twice as a result of
changes merged from Bitcoin 2 years ago
(commit 64eebc3316).
Include this file just once.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
Change the few occurrences of the deprecated `auto_ptr` to c++11 `unique_ptr`.
Silences the deprecation warnings.
Also add a missing `std::` for consistency.
* build: Enable C++11 build, require C++11 compiler
Implements #6211.
* depends: use c++11
* build: update ax_cxx_compile_stdcxx to serial 4
* build: force a c++ standard to be specified
Newer compilers may switch to newer standards by default. For example, gcc6
uses std=gnu++14 by default.
* c++11: fix libbdb build against libc++ in c++11 mode
atomic_init clashes with
* c++11: CAccountingEntry must be defined before use in a list
c++11ism. This fixes builds against libc++.
* PS should limit entry size, not mixing amount
* There should be no fee in mixing tx
* make sure pwalletMain is not null in PrepareDenominate
* no need for "double" in GetAverageAnonymizedRounds, "float" should be enough
* add strErrorRet
All names containing bitcoinconsensus remaned to contain dashconsensus.
This is needed to avoid conflicts with real bitcoinconsensus library
shipped with Bitcoin Core.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
noexcept is default for destructors as of c++11. By throwing in reverselock's
destructor if it's lock has been tampered with, the likely result is
std::terminate being called. Indeed that happened before this change.
Once reverselock has taken another lock (its ctor didn't throw), it makes no
sense to try to grab or lock the parent lock. That is be broken/undefined
behavior depending on the parent lock's implementation, but it shouldn't cause
the reverselock to fail to re-lock when destroyed.
To avoid those problems, simply swap the parent lock's contents with a dummy
for the duration of the lock. That will ensure that any undefined behavior is
caught at the call-site rather than the reverse lock's destruction.
Barring a failed mutex unlock which would be indicative of a larger problem,
the destructor should now never throw.
* Implement BIP 9 GBT changes
- BIP9DeploymentInfo struct for static deployment info
- VersionBitsDeploymentInfo: Avoid C++11ism by commenting parameter names
- getblocktemplate: Make sure to set deployments in the version if it is LOCKED_IN
- In this commit, all rules are considered required for clients to support
* qa/rpc-tests: bip9-softforks: Add tests for getblocktemplate versionbits updates
* getblocktemplate: Explicitly handle the distinction between GBT-affecting softforks vs not
* getblocktemplate: Use version/force mutation to support pre-BIP9 clients
* Don't use floating point
Github-Pull: #8317
Rebased-From: 477777f250
* Send tip change notification from invalidateblock
This change is needed to prevent sync_blocks timeouts in the mempool_reorg
test after the sync_blocks update in the upcoming commit
"[qa] Change sync_blocks to pick smarter maxheight".
This change was initially suggested by Suhas Daftuar <sdaftuar@chaincode.com>
in https://github.com/bitcoin/bitcoin/pull/8680#r78209060
Github-Pull: #9196
Rebased-From: 67c6326abd
* torcontrol: Explicitly request RSA1024 private key
When generating a new service key, explicitly request a RSA1024 one.
The bitcoin P2P protocol has no support for the longer hidden service names
that will come with ed25519 keys, until it does, we depend on the old
hidden service type so make this explicit.
See #9214.
Github-Pull: #9234
Rebased-From: 7d3b627395
* Bugfix: FRT: don't terminate when keypool is empty
Github-Pull: #9295
Rebased-From: c24a4f5981
* add fundrawtransaction test on a locked wallet with empty keypool
Github-Pull: #9295
Rebased-From: 1a6eacbf3b
