Due to growing coinsviewcaches, the memory usage with checklevel=3
(and standard settings for dbcache) could be up to 500MiB on a
64-bit system. This is about twice the peak during reindexing,
unnecessarily extending bitcoind's memory envelope.
This commit reduces the maximum total size of the caches used during
verification to just nCoinCacheSize, which should be the limit.
Remove the 'state' and 'exceptmask' from serialize.h's stream implementations,
as well as related methods.
As exceptmask always included 'failbit', and setstate was always called with
bits = failbit, all it did was immediately raise an exception. Get rid of
those variables, and replace the setstate with direct exception throwing
(which also removes some dead code).
As a result, good() is never reached after a failure (there are only 2
calls, one of which is in tests), and can just be replaced by !eof().
fail(), clear(n) and exceptions() are just never called. Delete them.
The only other method of logging remote addresses is via
-logips=1 -debug=net
which increases the logged activity by 100x or more.
Github-Pull: #4608
Amended-By: Wladimir J. van der Laan <laanwj@gmail.com>
Port over https://github.com/chronokings/huntercoin/pull/19 from
Huntercoin: This implements a new RPC command "getchaintips" that can be
used to find all currently active chain heads. This is similar to the
-printblocktree startup option, but it can be used without restarting
just via the RPC interface on a running daemon.
* Replace -benchmark (and the related fBenchmark) with a regular debug option, -debug=bench.
* Increase coverage and granularity of individual block processing steps.
* Add cummulative times.
First and foremost, this defaults to OFF.
This option lets a node consider such transactions non-standard,
meaning they will not be relayed or mined by default, but other miners
are free to mine these as usual.
4eedf4f make RandAddSeed() use OPENSSL_cleanse() (Philip Kaufmann)
6354935 move rand functions from util to new random.h/.cpp (Philip Kaufmann)
001a53d add GetRandBytes() as wrapper for RAND_bytes() (Philip Kaufmann)
This adds a -whitelist option to specify subnet ranges from which peers
that connect are whitelisted. In addition, there is a -whitebind option
which works like -bind, except peers connecting to it are also
whitelisted (allowing a separate listen port for trusted connections).
Being whitelisted has two effects (for now):
* They are immune to DoS disconnection/banning.
* Transactions they broadcast (which are valid) are always relayed,
even if they were already in the mempool. This means that a node
can function as a gateway for a local network, and that rebroadcasts
from the local network will work as expected.
Whitelisting replaces the magic exemption localhost had for DoS
disconnection (local addresses are still never banned, though), which
implied hidden service connects (from a localhost Tor node) were
incorrectly immune to DoS disconnection as well. This old
behaviour is removed for that reason, but can be restored using
-whitelist=127.0.0.1 or -whitelist=::1 can be specified. -whitebind
is safer to use in case non-trusted localhost connections are expected
(like hidden services).
- add a small wrapper in util around RAND_bytes() and replace with
GetRandBytes() in the code to log errors from calling RAND_bytes()
- remove OpenSSL header rand.h where no longer needed
75f51f2a introduced asynchronous processing for blocks, where reject messages
and DoS scoring could be applied outside of ProcessBlock, because block
validation may happen later.
However, some types of errors are still detected immediately (in particular,
CheckBlock violations), which need acting after ProcessBlock returns.
The wallet now uses the mempool fee estimator with a new
command-line option: -txconfirmtarget (default: 1) instead
of using hard-coded fees or priorities.
A new bitcoind that hasn't seen enough transactions to estimate
will fall back to the old hard-coded minimum priority or
transaction fee.
-paytxfee option overrides -txconfirmtarget.
Relaying and mining code isn't changed.
For Qt, the coin control dialog now uses priority estimates to
label transaction priority (instead of hard-coded constants);
unspent outputs were consistently labeled with a much higher
priority than is justified by the free transactions actually
being accepted into blocks.
I did not implement any GUI for setting -txconfirmtarget; I would
suggest getting rid of the "Pay transaction fee" GUI and replace
it with either "target number of confirmations" or maybe
a "faster confirmation <--> lower fee" slider or select box.
The original comment forgets to account for the script push which will
need an OP_PUSHDATA2 + 2-bytes for the 513 script bytes.
props davecgh
fixes#4224
Allows network wallets and other clients to see transactions that respend
a prevout already spent in an unconfirmed transaction in this node's mempool.
Knowledge of an attempted double-spend is of interest to recipients of the
first spend. In some cases, it will allow these recipients to withhold
goods or services upon being alerted of a double-spend that deprives them
of payment.
As before, respends are not added to the mempool.
Anti-Denial-of-Service-Attack provisions:
- Use a bloom filter to relay only one respend per mempool prevout
- Rate-limit respend relays to a default of 100 thousand bytes/minute
- Define tx2.IsEquivalentTo(tx1): equality when scriptSigs are not considered
- Do not relay these equivalent transactions
Remove an unused variable declaration in txmempool.cpp.
Relax the AreInputsStandard() tests for P2SH transactions --
allow any Script in a P2SH transaction to be relayed/mined,
as long as it has 15 or fewer signature operations.
Rationale: https://gist.github.com/gavinandresen/88be40c141bc67acb247
I don't have an easy way to test this, but the code changes are
straightforward and I've updated the AreInputsStandard unit tests.
... instead of after 30 minutes of no sending, for latency measurement
and keep-alive. Also, disconnect if no reply arrives within 20 minutes,
instead of 90 of inactivity (for peers supporting the 'pong' message).
f0a83fc Use Params().NetworkID() instead of TestNet() from the payment protocol (jtimon)
2871889 net.h was using std namespace through chainparams.h included in protocol.h (jtimon)
c8c52de Replace virtual methods with static attributes, chainparams.h depends on protocol.h instead of the other way around (jtimon)
a3d946e Get rid of TestNet() (jtimon)
6fc0fa6 Add RPCisTestNet chain parameter (jtimon)
cfeb823 Add RequireStandard chain parameter (jtimon)
21913a9 Add AllowMinDifficultyBlocks chain parameter (jtimon)
d754f34 Move majority constants to chainparams (jtimon)
8d26721 Get rid of RegTest() (jtimon)
cb9bd83 Add DefaultCheckMemPool chain parameter (jtimon)
2595b9a Add DefaultMinerThreads chain parameter (jtimon)
bfa9a1a Add MineBlocksOnDemand chain parameter (jtimon)
1712adb Add MiningRequiresPeers chain parameter (jtimon)
New RPC methods: return an estimate of the fee (or priority) a
transaction needs to be likely to confirm in a given number of
blocks.
Mike Hearn created the first version of this method for estimating fees.
It works as follows:
For transactions that took 1 to N (I picked N=25) blocks to confirm,
keep N buckets with at most 100 entries in each recording the
fees-per-kilobyte paid by those transactions.
(separate buckets are kept for transactions that confirmed because
they are high-priority)
The buckets are filled as blocks are found, and are saved/restored
in a new fee_estiamtes.dat file in the data directory.
A few variations on Mike's initial scheme:
To estimate the fee needed for a transaction to confirm in X buckets,
all of the samples in all of the buckets are used and a median of
all of the data is used to make the estimate. For example, imagine
25 buckets each containing the full 100 entries. Those 2,500 samples
are sorted, and the estimate of the fee needed to confirm in the very
next block is the 50'th-highest-fee-entry in that sorted list; the
estimate of the fee needed to confirm in the next two blocks is the
150'th-highest-fee-entry, etc.
That algorithm has the nice property that estimates of how much fee
you need to pay to get confirmed in block N will always be greater
than or equal to the estimate for block N+1. It would clearly be wrong
to say "pay 11 uBTC and you'll get confirmed in 3 blocks, but pay
12 uBTC and it will take LONGER".
A single block will not contribute more than 10 entries to any one
bucket, so a single miner and a large block cannot overwhelm
the estimates.
Use CFeeRate instead of an int64_t for quantities that are
fee-per-size.
Helps prevent unit-conversion mismatches between the wallet,
relaying, and mining code.
f40dbee remove CPubKey::VerifyCompact( ) which is never used (Kamil Domanski)
28b6c1d remove GetMedianTime( ) which is never used (Kamil Domanski)
5bd4adc remove LookupHostNumeric( ) which is never used (Kamil Domanski)
595f691 remove LogException( ) which is never used (Kamil Domanski)
f4057cb remove CTransaction::IsNewerThan which is never used (Kamil Domanski)
0e31e56 remove CWallet::AddReserveKey which is never used (Kamil Domanski)
8c93bf4 LoadBlockIndexDB(): Require block db reindex if any blk*.dat files are missing. (Ashley Holman)
7a0e84d ProcessGetData(): abort if a block file is missing from disk (Ashley Holman)
397668e Deduplicate uint* comparison operator logic (Pieter Wuille)
df9eb5e Move {Get,Set}Compact from bignum to uint256 (Pieter Wuille)
a703150 Add multiplication and division to uint160/uint256 (Pieter Wuille)
4d480c8 Exception instead of assigning 0 in case of wrong vector length (Pieter Wuille)
eb2cbd7 Deduplicate shared code between uint160 and uint256 (Pieter Wuille)
787ee0c Check redeemScript size does not exceed 520 byte limit (Peter Todd)
4d79098 Increase IsStandard() scriptSig length (Peter Todd)
f80cffa Do not trigger a DoS ban if SCRIPT_VERIFY_NULLDUMMY fails (Peter Todd)
6380180 Add rejection of non-null CHECKMULTISIG dummy values (Peter Todd)
29c1749 Let tx (in)valid tests use any SCRIPT_VERIFY flag (Peter Todd)
68f7d1d Create (MANDATORY|STANDARD)_SCRIPT_VERIFY_FLAGS constants (Peter Todd)
Removes the limits on number of pubkeys for P2SH CHECKMULTISIG outputs.
Previously with the 500 byte scriptSig limit there were odd restrictions
where even a 1-of-12 P2SH could be spent in a standard transaction(1),
yet multisig scriptPubKey's requiring more signatures quickly ran out of
scriptSig space.
From a "stuff-data-in-the-blockchain" point of view not much has changed
as with the prior commit now only allowing the dummy value to be null
the newly allowed scriptSig space can only be used for signatures. In
any case, just using more outputs is trivial and doesn't cost much.
1) See 779b519480d8c5346de6e635119c7ee772e97ec872240c45e558f582a37b4b73
Mined by BTC Guild.
Size specifiers are no longer needed now that we use typesafe tinyformat
for string formatting, instead of the system's sprintf.
No functional changes.
This continues the work in #3735.
Generally useless information. Only updates on connect time, not after
that. Peers can easily lie and the median filter is not effective in
preventing that.
In the past it was used for progress display in the GUI but
`CheckPoints::guessVerificationProgress` provides a better way that is now used.
It was too easy to mislead it. Peers do lie about it in practice, see issue #4065.
From the RPC, `getpeerinfo` gives the peer raw values, which are more
useful.
- In wallet and GUI code LOCK cs_main as well as cs_wallet when
necessary
- In main.cpp SendMessages move the TRY_LOCK(cs_main) up, to encompass the call
to IsInitialBlockDownload.
- Make ActivateBestChain, AddToBlockIndex, IsInitialBlockDownload,
InitBlockIndex acquire the cs_main lock
Fixes#3997
All functions that use ChainActive but do not aquire the cs_main
lock themselves, need to be called with the cs_main lock held.
This commit adds assertions to all externally callable functions
that use chainActive or chainMostWork.
This will flag usages when built with -DDEBUG_LOCKORDER.
PrintBlockTree output was broken starting from e010af70.
Everything appears on one line.
PrintWallet() added the newline after a block, but this functionality
was removed and no newline was added.
Seemingly, no one noticed. Add a newline after the block information
to fix this.
This resolves a case in which a mismatch could be used to bloat up the
mempool by sending transactions that pay enough fee to relay, but not
to be mined, with the default policies.
Amend to d5f1e72. It turns out that BerkelyDB was including inttypes.h
indirectly, so we cannot fix this with just macros.
Trivial commit: apply the following script to all .cpp and .h files:
# Middle
sed -i 's/"PRIx64"/x/g' "$1"
sed -i 's/"PRIu64"/u/g' "$1"
sed -i 's/"PRId64"/d/g' "$1"
# Initial
sed -i 's/PRIx64"/"x/g' "$1"
sed -i 's/PRIu64"/"u/g' "$1"
sed -i 's/PRId64"/"d/g' "$1"
# Trailing
sed -i 's/"PRIx64/x"/g' "$1"
sed -i 's/"PRIu64/u"/g' "$1"
sed -i 's/"PRId64/d"/g' "$1"
After this commit, `git grep` for PRI.64 should turn up nothing except
the defines in util.h.
As the tinyformat-based formatting system (introduced in b77dfdc) is
type-safe, no special format characters are needed to specify sizes.
Tinyformat can support (ignore) the C99 prefixes such as "ll" but
chokes on MSVC's inttypes.h defines prefixes such as "I64X". So don't
include inttypes.h and define our own for compatibility.
(an alternative would be to sweep the entire codebase using sed -i to
get rid of the size specifiers but this has less diff impact)
5770254 Copyright header updates s/2013/2014 on files whose last git commit was done in 2014. contrib/devtools/fix-copyright-headers.py script to be able to perform this maintenance task with ease during the rest of the year, every year. Modifications to contrib/devtools/README.md to document what fix-copyright-headers.py does. (gubatron)
Extend CMerkleTx::GetDepthInMainChain with the concept of
a "conflicted" transaction-- a transaction generated by the wallet
that is not in the main chain or in the mempool, and, therefore,
will likely never be confirmed.
GetDepthInMainChain() now returns -1 for conflicted transactions
(0 for unconfirmed-but-in-the-mempool, and >1 for confirmed).
This makes getbalance, getbalance '*', and listunspent all agree when there are
mutated transactions in the wallet.
Before:
listunspent: one 49BTC output
getbalance: 96 BTC (change counted twice)
getbalance '*': 46 BTC (spends counted twice)
After: all agree, 49 BTC available to spend.
contrib/devtools/fix-copyright-headers.py script to be able to perform this maintenance task with ease during the rest of the year, every year. Modifications to contrib/devtools/README.md to document what fix-copyright-headers.py does.
Keep track of which block is being requested (and to be requested) from
each peer, and limit the number of blocks in-flight per peer. In addition,
detect stalled downloads, and disconnect if they persist for too long.
This means blocks are never requested twice, and should eliminate duplicate
downloads during synchronization.
In case the total number of orphan blocks in memory exceeds a limit
(currently set to 750), a random orphan block (which is not
depended on by another orphan block) is dropped. This means it will
need to be downloaded again, but it won't consume memory until then.
c117d9e Support for error messages and a few more rejection reasons (Luke Dashjr)
14e7ffc Use standard BIP 22 rejection reasons where applicable (Luke Dashjr)
This changes the block processing logic from "try to atomically switch
to a new block" to a continuous "(dis)connect a block, aiming for the
assumed best chain".
This means the smallest atomic operations on the chainstate become
individual block connections or disconnections, instead of entire
reorganizations. It may mean that we try to reorganize to one block,
fail, and rereorganize again to the old block. This is slower, but
doesn't require unbounded RAM.
It also means that a ConnectBlock which fails may be no longer called
from the ProcessBlock which knows which node sent it. To deal with that,
a mapBlockSource is kept, and invalid blocks cause asynchronous "reject"
messages and banning (if necessary).
Previously CreateNewBlock() didn't take into account the fact that
IsFinalTx() without any arguments tests if the transaction is considered
final in the *current* block, when both those functions really needed to
know if the transaction would be final in the *next* block.
Additionally the UI had a similar misunderstanding.
Also adds some basic tests to check that CreateNewBlock() is in fact
mining nLockTime-using transactions correctly.
Thanks to Wladimir J. van der Laan for rebase.
After the tinyformat switch sprintf() family functions support passing
actual std::string objects.
Remove unnecessary c_str calls (236 of them) in logging and formatting.
012ca1c LoadWallet: acquire cs_wallet mutex before clearing setKeyPool (Wladimir J. van der Laan)
9569168 Document cs_wallet lock and add AssertLockHeld (Wladimir J. van der Laan)
19a5676 Use mutex pointer instead of name for AssertLockHeld (Wladimir J. van der Laan)
There were quite a few places where assert() was used with side effects,
making operation with NDEBUG non-functional. This commit fixes all the
cases I know about, but also adds an #error on NDEBUG because the code
is untested without assertions and may still have vulnerabilities if
used without assert.
This dead code can be resurrected from git history if
transaction replacement is ever implemented. Keeping
dead code in the source is a bad idea, because it implies
it was tested and worked at some point, which is not true.
The last fee drop was by 5x (from 50k satoshis to 10k satoshis)
in the 0.8.2 release which was about 6 months ago.
The current fee is (assuming a $500 exchange rate) about 5 dollar
cents. The new fee after this patch is 0.5 cents.
Miners who prefer the higher fees are obviously still able to
use the command line flags to override this setting. Miners who
choose to create smaller blocks will select the highest-fee paying
transactions anyway.
This would hopefully be the last manual adjustment ever required
before floating fees become normal.
Use misc methods of avoiding unnecesary header includes.
Replace int typedefs with int##_t from stdint.h.
Replace PRI64[xdu] with PRI[xdu]64 from inttypes.h.
Normalize QT_VERSION ifs where possible.
Resolve some indirect dependencies as direct ones.
Remove extern declarations from .cpp files.
caca6aa Make some globals in main non-public. (Pieter Wuille)
85eb2ce Do not use the redundant BestInvalidWork record in the block database. (Pieter Wuille)
As block index entries have a flag for marking invalid blocks, the
'best invalid work' information can be derived from there. In addition,
remove the global from main.h
- re-work -debug help message text
- make -debug log every debugging information again (even all categories)
- remove unneeded fDebug checks in front of LogPrint()/qDebug(), as that
check is done in LogPrintf() when category is != NULL (true for all
LogPrint() calls
- remove fDebug ONLY in code which is NOT performance-critical
- harmonize addrman category name
- deprecate -debugnet usage, should be used via -debug=net and remove the
corresponding global
Instead of explicitly testing for the presence of any output, and
dealing with this case specially, just interpret it as an empty
CCoins.
The case previously caught using the HaveCoins check, is now handled
by the generic outs != outsBlock test.
This required some code movement (what was CWalletTx::AcceptToMemoryPool
doing in main?), and adding a few explicit includes that used to be
implicit through init.h.
INIT_PROTO_VERSION is the initial version, after a succesful version/verack it is increased to a negotiated version.
MIN_PEER_PROTO_VERSION could be a different value to disconnect from peers older than a specified version.
Changes the response to the 'mempool' command so that if
the memory pool has more than MAX_INV_SZ transactions (50,000)
it will respond with multiple 'inv' messages.
SendMessages() tries to acquire a cs_main lock now, but this isn't nessecary
for much of its functionality. Move those parts out of the locked section,
so they can always be performed, and we hold cs_main for a shorter time.
This removes a few unused CBlockLocator methods, and moves the
construction and fork-finding logic to CChain (which can do these
more efficiently, as it has a height-indexable chain available).
It also makes CBlockLocator independent from the validation code.
971bb3e Added ping time measurement. New RPC "ping" command to request ping. Implemented "pong" message handler. New "pingtime" field in getpeerinfo, to provide results to user. New "pingwait" field, to show pings still in flight, to better see newly lagging peers. (Josh Lehan)
Changes the maximum size of a free transaction that will be created
from 10,000 bytes to 1,000 bytes.
The idea behind this change is to make the free transaction area
available to a greater number of people; with the default 27K-per-block,
just three very-large very-high-priority transactions could fill the space.
Remove the (relay/mempool) rule that all outputs of free transactions
must be greater than 0.01 XBT. Dust spam is now taken care of by making
dusty outputs non-standard.
New RPC "ping" command to request ping.
Implemented "pong" message handler.
New "pingtime" field in getpeerinfo, to provide results to user.
New "pingwait" field, to show pings still in flight, to better see newly lagging peers.
- rename URL into URI in paymentserver where correct
- add some missing Qt-coding-stuff in paymentserver
- change QSpinBox to QLineEdit as base for BitcoinAmountField in .ui files
(as this is the result when converting the BAF back into base)
- remove some c_str() and replace with QString::fromStdString()
- remove several new-lines
- remove unneeded spaces
- indentation fixes
This also makes negative transaction versions non-standard.
This avoids an issue triggered in block 256818 where transactions with
negative version numbers were incorrectly serialized into the UTXO set.
On restart nodes detect the inconsistency and refuse to start so long as
a block with these transactions is inside the self-consistency check
window, logging "coin database inconsistencies found". The software
recommends reindexing, but reindexing does not correct the problem.
This should be fixed by changing the chainstate serialization, but
working around it seems harmless for now because the version is not
used by any network rule currently.
A patch free workaround is to start with -checklevel=2 which skips
the consistency checks, but the IsStandard change is important for
miners in order to protect unpatched nodes.
There have been several incidents where mainnet experimentation with
raw transactions resulted in insane fees. This is hard to prevent
in the raw transaction api because the inputs may not be known.
Since sending doesn't work if the inputs aren't known, we can catch
it there.
This rejects fees > than 10000 * nMinRelayTxFee or 1 BTC with the
defaults and can be overridden with a bool at the rpc.
We're not seeing large reorgs that would justify waiting a large
amount past the rule required maturity, and the extra three
hours is just a nuisance. Take one more block to at least give
the 100th block time to propagate.
This reduces a peer's ability to attack network resources by
using a full bloom filter, but without reducing the usability
of bloom filters. It sets a default match everything filter
for peers and it generalizes a prior optimization to
cover more cases.
The length of vectors, maps, sets, etc are serialized using
Write/ReadCompactSize -- which, unfortunately, do not use a
unique encoding.
So deserializing and then re-serializing a transaction (for example)
can give you different bits than you started with. That doesn't
cause any problems that we are aware of, but it is exactly the type
of subtle mismatch that can lead to exploits.
With this pull, reading a non-canonical CompactSize throws an
exception, which means nodes will ignore 'tx' or 'block' or
other messages that are not properly encoded.
Please check my logic... but this change is safe with respect to
causing a network split. Old clients that receive
non-canonically-encoded transactions or blocks deserialize
them into CTransaction/CBlock structures in memory, and then
re-serialize them before relaying them to peers.
And please check my logic with respect to causing a blockchain
split: there are no CompactSize fields in the block header, so
the block hash is always canonical. The merkle root in the block
header is computed on a vector<CTransaction>, so
any non-canonical encoding of the transactions in 'tx' or 'block'
messages is erased as they are read into memory by old clients,
and does not affect the block hash. And, as noted above, old
clients re-serialize (with canonical encoding) 'tx' and 'block'
messages before relaying to peers.
Orphan transactions were stored as a CDataStream pointer;
this changes the mapOrphanTransactions data structures to
store orphans as a CTransaction.
This also fixes CVE-2013-4627 by always re-serializing
transactions before relaying them.
