Instead of killing a connection when the receive buffer overflows,
just temporarily halt receiving before that happens. Also, no
matter what, always allow at least one full message in the receive
buffer (otherwise blocks larger than the configured buffer size
would pause indefinitely).
This introduces the concept of the 'sync node', which is the one we
asked for missing blocks. In case the sync node goes away, a new one
will be selected.
For now, the heuristic is very simple, but it can easily be extended
later to add better policies.
It seems there were two mechanisms for assessing whether a CNode
was still in use: a refcount and a release timestamp. The latter
seems to have been there for a long time, as a safety mechanism.
However, this timer also keeps CNode objects alive for far longer
than necessary after disconnects, potentially opening up a DoS
window.
This commit removes the timestamp-based mechanism, and replaces
it with an assert(nRefCount >= 0), to verify that the refcounting
is indeed correctly working.
Two reasons for this change:
1. Need to always use boost::thread's sleep, even on Windows, so the
sleeps can be interrupted (prior code used Windows' built-in Sleep).
2. I always forgot what units the old Sleep took.
Create a boost::thread_group object at the qt/bitcoind main-loop level
that will hold pointers to all the main-loop threads.
This will replace the vnThreadsRunning[] array.
For testing, ported the BitcoinMiner threads to use its
own boost::thread_group.
This will result in re-requesting invs if we are under heavy inv
load, however as long as we get no more than 16,000 invs in two
minutes, this should have no effect on runtime behavior.
* Change CNode::vRecvMsg to be a deque instead of a vector (less copying)
* Make sure to acquire cs_vRecvMsg in CNode::CloseSocketDisconnect (as it
may be called without that lock).
1) "optimistic write": Push each message to kernel socket buffer immediately.
2) If there is write data at select time, that implies send() blocked
during optimistic write. Drain write queue, before receiving
any more messages.
This avoids needlessly queueing received data, if the remote peer
is not themselves receiving data.
Result: write buffer (and thus memory usage) is kept small, DoS
potential is slightly lower, and TCP flow control signalling is
properly utilized.
The kernel will queue data into the socket buffer, then signal the
remote peer to stop sending data, until we resume reading again.
Replaces CNode::vRecv buffer with a vector of CNetMessage's. This simplifies
ProcessMessages() and eliminates several redundant data copies.
Overview:
* socket thread now parses incoming message datastream into
header/data components, as encapsulated by CNetMessage
* socket thread adds each CNetMessage to a vector inside CNode
* message thread (ProcessMessages) iterates through CNode's CNetMessage vector
Message parsing is made more strict:
* Socket is disconnected, if message larger than MAX_SIZE
or if CMessageHeader deserialization fails (latter is impossible?).
Previously, code would simply eat garbage data all day long.
* Socket is disconnected, if we fail to find pchMessageStart.
We do not search through garbage, to find pchMessageStart. Each
message must begin precisely after the last message ends.
ProcessMessages() always processes a complete message, and is more efficient:
* buffer is always precisely sized, using CDataStream::resize(),
rather than progressively sized in 64k chunks. More efficient
for large messages like "block".
* whole-buffer memory copy eliminated (vRecv -> vMsg)
* other buffer-shifting memory copies eliminated (vRecv.insert, vRecv.erase)
Note that the default value for fRelayTxes is false, meaning we
now no longer relay tx inv messages before receiving the remote
peer's version message.
Client (SPV) mode never got implemented entirely, and whatever part was already
working, is likely not been tested (or even executed at all) for the past two
years. This removes it entirely.
If we want an SPV implementation, I think we should first get the block chain
data structures to be encapsulated in a class implementing a standard interface,
and then writing an alternate implementation with SPV semantics.
- "ThreadIRCSeed started" was not displayed, even if the thread ran
(although only for a short time as the "do we want this thread?"-checks
happen IN ThreadIRCSeed2())
- the patch ensures we always get that message
- add a "ThreadIRCSeed trying to connect..." message
- add missing "ThreadDumpAddress started" message
- fix#1560 by properly locking proxy related data-structures
- update GetProxy() and introduce GetNameProxy() to be able to use a
thread-safe local copy from proxyInfo and nameproxyInfo
- update usage of GetProxy() all over the source to match the new
behaviour, as it now fills a full proxyType object
- rename GetNameProxy() into HaveNameProxy() to be more clear
These command are a leftover from send-to-IP transactions, which have been
removed a long time ago.
Also removes CNode::mapRequests and CNode::PushRequests, as these were
only used for the mentioned commands.
- I checked every occurance of strprintf() in the code and used %u, where
unsigned vars are used
- the change to GetByte() was made, as ip is an unsigned char
- ensure warnings always start with "Warning:" and that the first
character after ":" is written uppercase
- ensure the first sentence in warnings ends with an "!"
- remove unneeded spaces from Warning-strings
- add missing Warning-string translation
- remove a "\n" and replace with untranslatable "<br><br>"
* Fix wrong thread name for wallet *relocking* thread
- Was named the unlocking thread
* Use consistent naming
Signed-off-by: Giel van Schijndel <me@mortis.eu>
NOTE: These thread names are visible in gdb when using 'info threads'.
Additionally both 'top' and 'ps' show these names *unless* told to
display the command-line instead of task name.
Signed-off-by: Giel van Schijndel <me@mortis.eu>
Because new nodes pull from the first connected node the load
balancing of the first connection is more important than it should
be. This change puts Pieter's seed first, because its probably
the best maintained right now.
Bitcoin will not make an outbound connection to a network group
(/16 for IPv4) that it is already connected to. This means that
if an attacker wants good odds of capturing all a nodes outbound
connections he must have hosts on a a large number of distinct
groups.
Previously both inbound and outbound connections were used to
feed this exclusion. The use of inbound connections, which can be
controlled by the attacker, actually has the potential of making
sibyl attacks _easier_: An attacker can start up hosts in groups
which house many honest nodes and make outbound connections to
the victim to exclude big swaths of honest nodes. Because the
attacker chooses to make the outbound connection he can always
beat out honest nodes for the consumption of inbound slots.
At _best_ the old behavior increases attacker costs by a single
group (e.g. one distinct group to use to fill up all your inbound
slots), but at worst it allows the attacker to select whole
networks you won't connect to.
This commit makes the nodes use only outbound links to exclude
network groups for outbound connections. Fancier things could
be done, like weaker exclusion for inbound groups... but
simplicity is good and I don't believe more complexity is
currently needed.
- Signals now go directly from the core to WalletModel/ClientModel.
- WalletModel subscribes to signals on CWallet: Prepares for multi-wallet support, by no longer assuming an implicit global wallet.
- Gets rid of noui.cpp, the few lines that were left are merged into init.cpp
- Rename wxXXX message flags to MF_XXX, to make them UI indifferent.
- ThreadSafeMessageBox no longer returns the value `4` which was never used, converted to void.
Gets rid of `MainFrameRepaint` in favor of specific update functions that tell the UI exactly what changed.
This improves the efficiency of various handlers. Also fixes problems with mined transactions not showing up until restart.
The following notifications were added:
- `NotifyBlocksChanged`: Block chain changed
- `NotifyKeyStoreStatusChanged`: Wallet status (encrypted, locked) changed.
- `NotifyAddressBookChanged`: Address book entry changed.
- `NotifyTransactionChanged`: Wallet transaction added, removed or updated.
- `NotifyNumConnectionsChanged`: Number of connections changed.
- `NotifyAlertChanged`: New, updated or cancelled alert. As this finally makes it possible for the UI to know when a new alert arrived, it can be shown as OS notification.
These notifications could also be useful for RPC clients. However, currently, they are ignored in bitcoind (in noui.cpp).
Also brings back polling with timer for numBlocks in ClientModel. This value updates so frequently during initial download that the number of signals clogs the UI thread and causes heavy CPU usage. And after initial block download, the value changes so rarely that a delay of half a second until the UI updates is unnoticable.
Introduce a boolean variable for each "network" (ipv4, ipv6, tor, i2p),
and track whether we are likely to able to connect to it. Addresses in
"addr" messages outside of our network get limited relaying and are not
stored in addrman.
This will make bitcoin relay valid routable IPv6 addresses, and when
USE_IPV6 is enabled, listen on IPv6 interfaces and attempt connections
to IPv6 addresses.
Change internal HTTP JSON-RPC server from single-threaded to
thread-per-connection model. The IP filter list is applied prior to starting
the thread, which then processes the RPC.
A mutex covers the entire RPC operation, because not all RPC operations are
thread-safe.
[minor modifications by jgarzik, to make change upstream-ready]
-externalip=<ip> can be used to explicitly set the public IP address
of your node. -discover=0 can be used to disable the automatic public
IP discovery system.
Keep a global counter for nOutbound, protected with its own waitable
critical section, and wait when all outbound slots are filled, rather
than polling.
This removes the (on average) 1 second delay between a lost connection
and a new connection attempt, and may speed up shutdowns.
not process any already received messages.
The primary reason to do this is if a node spams hundreds of messages
and we ban them, we don't want to continue processing the rest of it.
Design goals:
* Only keep a limited number of addresses around, so that addr.dat does not grow without bound.
* Keep the address tables in-memory, and occasionally write the table to addr.dat.
* Make sure no (localized) attacker can fill the entire table with his nodes/addresses.
See comments in addrman.h for more detailed information.
* do not let vnThreadsRunning[1] go negative
* do not perform locking operations while vnThreadsRunning[1] is decreased
* check vnThreadsRunning[1] at exit
This fixes a potential bug where some NATs may replace the node's
interal IP with its external IP in version messages, causing
incorrect checksums when version messages begin being checksummed
on February 14, 2012.
This also avoids flushing setAddrKnown until 24 hours has passed,
and avoids contacting the external IP services when not listening.
Advertising non-listening nodes is just addr message spam.
It doesn't help the network, in fact it hurts the network,
and it also hurts user's privacy.
Advertising far out of sync nodes doesn't help the network—
they can't even forward (most) transactions and wastes nodes
outbound slots.
This turns on most gcc warnings, and removes some unused variables and other code that triggers warnings.
Exceptions are:
-Wno-sign-compare : triggered by lots of comparisons of signed integer to foo.size(), which is unsigned.
-Wno-char-subscripts : triggered by the convert-to-hex functions (I may fix this in a future commit).
This introduces CNetAddr and CService, respectively wrapping an
(IPv6) IP address and an IP+port combination. This functionality used
to be part of CAddress, which also contains network flags and
connection attempt information. These extra fields are however not
always necessary.
These classes, along with logic for creating connections and doing
name lookups, are moved to netbase.{h,cpp}, which does not depend on
headers.h.
Furthermore, CNetAddr is mostly IPv6-ready, though IPv6
functionality is not yet enabled for the application itself.
Made three critical blocks for cs_mapAddresses smaller, and moved
writing to the database out of them. This should also improve the
concurrency of the code.
Replaced all occurrences of #if* __WXMSW__ with WIN32,
and all occurrences of __WXMAC_OSX__ with MAC_OSX, and made
sure those are defined appropriately in the makefile and bitcoin-qt.pro.
Explicitly make these global variables less-global to reduce the maximum
scope of this global state.
In my experience global variables tend to be a major source of bugs. As
such the less accessible they are the less likely they are to be the
source of a bug.
Signed-off-by: Giel van Schijndel <me@mortis.eu>
Don't check for a negative parameter count, because not only will it
never happen, it doesn't make any sense either.
Invalid sockets (as returned by socket(2)) are always exactly -1 (not
just negative as negative file descriptors are technically not
prohibited by POSIX) on POSIX systems. Since we store them in SOCKET
(unsigned int), however, that really is ~0U (or MAX_UINT) which happens
to be what INVALID_SOCKET is already defined to, so an additional check
for being negative is not only unnecessary (unsigned integers aren't
*ever* negative) its redundant as well (the INVALID_SOCKET comparison is
enough).
Signed-off-by: Giel van Schijndel <me@mortis.eu>
Introduce SendBufferSize() and ReceiveBufferSize(), and limit
the blocks sent as response to the "getblocks" message to
half of the active send buffer size.
* A new class CKeyStore manages private keys, and script.cpp depends on access to CKeyStore.
* A new class CWallet extends CKeyStore, and contains all former wallet-specific globals; CWallet depends on script.cpp, not the other way around.
* Wallet-specific functions in CTransaction/CTxIn/CTxOut (GetDebit, GetCredit, GetChange, IsMine, IsFromMe), are moved to CWallet, taking their former 'this' argument as an explicit parameter
* CWalletTx objects know which CWallet they belong to, for convenience, so they have their own direct (and caching) GetDebit/... functions.
* Some code was moved from CWalletDB to CWallet, such as handling of reserve keys.
* Main.cpp keeps a set of all 'registered' wallets, which should be informed about updates to the block chain, and does not have any notion about any 'main' wallet. Function in main.cpp that require a wallet (such as GenerateCoins), take an explicit CWallet* argument.
* The actual CWallet instance used by the application is defined in init.cpp as "CWallet* pwalletMain". rpc.cpp and ui.cpp use this variable.
* Functions in main.cpp and db.cpp that are not used by other modules are marked static.
* The code for handling the 'submitorder' message is removed, as it not really compatible with the idea that a node is independent from the wallet(s) connected to it, and obsolete anyway.
Use non-blocking connects, and a select() call to wait a predefined
time (5s by default, but configurable with -timeout) for either
success or failure. This allows much more connections to be tried
per time unit.
Based on a patch by phantomcircuit.
* A new option -dns is introduced that enables name lookups in
-connect and -addnode, which is not enabled by default,
as it may be considered a security issue.
* A Lookup function is added that supports retrieving one or
more addresses based on a host name
* CAddress constructors (optionally) support name lookups.
* The different places in the source code that did name lookups
are refactored to use NameLookup or CAddress instead (dns seeding,
irc server lookup, getexternalip, ...).
* Removed ToStringLog() from CAddress, and switched to ToString(),
since it was empty.
there is no internal modification of any file in this commit
files are moved into directories according to established standards in
sourcecode distribution; these directories contain:
src - Files that are used in constructing the executable binaries,
but are not installed.
doc - Files in HTML and text format that document usage, quirks of
the implementation, and contributor checklists.
locale - Files that contain human language translation of strings
used in the program
contrib - Files contributed from distributions or other third party
implementing scripts and auxiliary programs
