30fb598737f6efb7802d707a1fa989872e7f8b7b Fix segfault in allocator_tests/arena_tests (Jeffrey Czyz)
15c84f53f47bf6e6a9c4c9dfe50c78d98f7ec07f Define ARENA_DEBUG in Travis test runs (Jeffrey Czyz)
ad715488222f2f2ce2e2cff632eae94fd49ea9c5 Fix compilation errors in support/lockedpool.cpp (Jeffrey Czyz)
Pull request description:
Changes in #12048 cause a compilation error in Arena::walk() when
ARENA_DEBUG is defined. Specifically, Arena's chunks_free map was
changed to have a different value type.
Additionally, missing includes cause other compilation errors when
ARENA_DEBUG is defined.
Reproduced with:
make CPPFLAGS=-DARENA_DEBUG
ACKs for top commit:
laanwj:
ACK 30fb598737f6efb7802d707a1fa989872e7f8b7b
fanquake:
ACK 30fb598737f6efb7802d707a1fa989872e7f8b7b - thanks for following up jkczyz.
Tree-SHA512: 4eec368a4e9c67e4e2a27bc05608a807c2892d50c60d06ed21490cd274c0369f9671bc05b3006acc2a193316caf4896454c9c299603bfed29bd488f1987ec446
f381299 Move CKeyStore::cs_KeyStore to CBasicKeyStore (João Barbosa)
25eb9f5 Inline CKeyStore::AddKey(const CKey &) in CBasicKeyStore (João Barbosa)
Pull request description:
Made these simplifications while reviewing #12714. This aims to make `CKeyStore` a *pure* interface:
- no variable members - the mutex is moved to `CBasicKeyStore` which is where it is used;
- no method implementations - `AddKey(const CKey &)` is moved to `CBasicKeyStore` which is where it is needed.
Tree-SHA512: 84e44f4390c59600e5cefa599b5464e1771c31dd4abc678ef50db8e06ffac778d692860a352918444f8bcd66430634637b6277a818a658721ffc4f381c1c6a90
1ec1602a45 Make FastRandomContext support standard C++11 RNG interface (Pieter Wuille)
Pull request description:
This makes it possible to plug it into the various standard C++11 random distribution algorithms and other functions like `std::shuffle`.
Tree-SHA512: 935eae9c4fae31e1964c16d9cf9d0fcfa899e04567f010d8b3e1ff824e55e2392aa838ba743d03c1b2a5010c5b8da04343f453983dfeed83747d85828a564713
8674e74 Provide relevant error message if datadir is not writable. (murrayn)
Pull request description:
If the --datadir exists, but is not writable, the current error message on startup is 'Cannot obtain a lock on data directory foo. Bitcoin Core is probably already running.' This is misleading.
I believe this PR addresses #11668, although the issue is not Windows-specific.
Tree-SHA512: 10cbbaea433072aee4fb3e8938a72073c7a5c841f7a7685c9e12549c322b2925c7d34bac254ac33021b23132bfc352c058712bc9542298cf86f8fd9757f528b2
5fbf7c4 fix nits: variable naming, typos (Martin Ankerl)
1e0ee90 Use best-fit strategy in Arena, now O(log(n)) instead O(n) (Martin Ankerl)
Pull request description:
This replaces the first-fit algorithm used in the Arena with a best-fit. According to "Dynamic Storage Allocation: A Survey and Critical Review", Wilson et. al. 1995, http://www.scs.stanford.edu/14wi-cs140/sched/readings/wilson.pdf, both startegies work well in practice.
The advantage of using best-fit is that we can switch the O(n) allocation to O(log(n)). Additionally, some previously O(log(n)) operations are now O(1) operations by using hash maps. The end effect is that the benchmark runs about 2.5 times faster on my machine:
# Benchmark, evals, iterations, total, min, max, median
old: BenchLockedPool, 5, 530, 5.25749, 0.00196938, 0.00199755, 0.00198172
new: BenchLockedPool, 5, 1300, 5.11313, 0.000781493, 0.000793314, 0.00078606
I've run all unit tests and benchmarks, and increased the number of iterations so that BenchLockedPool takes about 5 seconds again.
Tree-SHA512: 6551e384671f93f10c60df530a29a1954bd265cc305411f665a8756525e5afe2873a8032c797d00b6e8c07e16d9827465d0b662875433147381474a44119ccce
a7c45bc Add native support for serializing char arrays without FLATDATA (Pieter Wuille)
Pull request description:
Support is added to serialize arrays of type `char` or `unsigned char` directly, without any wrappers. All invocations of the `FLATDATA` wrappers that are obsoleted by this are removed.
This includes a patch by @ryanofsky to make `char` casting type safe.
The serialization of `CSubNet` is changed to serialize a `bool` directly rather than though `FLATDATA`. This makes the serialization independent of the size of the bool type (and will use 1 byte everywhere).
This is a small change taken from #10785.
Tree-SHA512: a41f61ca5fdc2fadb2d0e1702351a58a23841d551f505292a9542602cdb19f90d8944b8df14b872810a56bd201648fa4c0e958f3e9427fe829886284e85b9bfd
Signed-off-by: pasta <pasta@dashboost.org>
# Conflicts:
# src/test/serialize_tests.cpp
cb1e319 Bugfix: RPC: savemempool: Don't save until LoadMempool() is finished (Jorge Timón)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/12142
The tests are a little bit slow, mempool_persist.py goes from about 20 s to about 120 s in my hardware.
Perhaps there's a better way to test this.
Tree-SHA512: 9e6c24b32a9cf3774e8f0bd81c035b0deb53fba5ac3eb2532d85900579d21cef8a1135b75a4fa0a9d883e3822eb35e7d4b47a0838abf99789039205041962629
6feb46c Add --with-sanitizers option to configure (Evan Klitzke)
Pull request description:
This adds configure options for `-fsanitize=address`, `-fsanitize=thread`, and `-fsanitize=undefined` which are all disabled by default. These flags are useful for developers who wish to do additional safety checking. Note that some of these are mutually incompatible, and these may have a large performance overhead.
There's some kind of strange logic required to properly check for the availability of these flags in a way that works on both GCC and Clang, hopefully the comments make it clear what's going on.
Tree-SHA512: 2d6fe402799110e59ee452dddf37f7ca2d26a7fecec50be25c8a134e4a20beb31f1e8f438dffd443641562418075896d1eeb450623425b272d80e05e3027a587
ccedbaf Increase LevelDB max_open_files unless on 32-bit Unix. (Evan Klitzke)
Pull request description:
Currently we set `max_open_files = 64` on all architectures due to concerns about file descriptor exhaustion. This is extremely expensive due to how LevelDB is designed.
When a LevelDB file handle is opened, a bloom filter and block index are decoded, and some CRCs are checked. Bloom filters and block indexes in open table handles can be checked purely in memory. This means that when doing a key lookup, if a given table file may contain a given key, all of the lookup operations can happen completely in RAM until the block itself is fetched. In the common case fetching the block is one disk seek, because the block index stores its physical offset. This is the ideal case, and what we want to happen as often as possible.
If a table file handle is not open in the table cache, then in addition to the regular system calls to open the file, the block index and bloom filter need to be decoded before they can be checked. This is expensive and is something we want to avoid.
The current setting of 64 file handles means that on a synced node, only about 4% of key lookups can be satisifed by table file handles that are actually open and in memory.
The original concerns about file descriptor exhaustion are unwarranted on most systems because:
* On 64-bit POSIX hosts LevelDB will open up to 1000 file descriptors using `mmap()`, and it does not retain an open file descriptor for such files.
* On Windows non-socket files do not interfere with the main network `select()` loop, so the same fd exhaustion issues do not apply there.
This change keeps the default `max_open_files` value (which is 1000) on all systems except 32-bit POSIX hosts (which do not use `mmap()`). Open file handles use about 20 KB of memory (for the block index), so the extra file handles do not cause much memory overhead. At most 1000 will be open, and a fully synced node right now has about 1500 such files.
Profile of `loadblk` thread before changes: https://monad.io/maxopenfiles-master.svg
Profile of `loadblk` thread after changes: https://monad.io/maxopenfiles-increase.svg
Tree-SHA512: de54f77d57e9f8999eaf8d12592aab5b02f5877be8fa727a1f42cf02da2693ce25846445eb19eb138ce4e5045d1c65e14054df72faf3ff32c7655c9cfadd27a9
d40f06a Introduce interface for signing providers (Pieter Wuille)
Pull request description:
`CKeyStore` is a rich interface that provides many features, including knowledge of scripts and pubkeys for solving, private keys for signing, in addition to watch-only keys and scripts, and distinguishing lack of keys from them just being encrypted.
The signing logic in script/sign does not actually need most of these features. Here we introduce a simpler interface (`SigningProvider`) which *only* provides keys and scripts. This is actually sufficient for signing.
In addtion, we swap the dependency between keystore and script/sign (keystore now depends on script/script with `CKeyStore` deriving from `SigningProvider`, rather than `CKeyStore` being the interface that signing relies on).
This is a very early step towards the design in https://gist.github.com/sipa/125cfa1615946d0c3f3eec2ad7f250a2, separating the concern between deciding what outputs are ours and signing.
Tree-SHA512: d511b7b03eec0e513530db1d9ae5aacf6d0bfa1d3e1c03d06c5bde396bafb5824c4491b227d32bcda9288530caf49835da18e846ccf66538d6c0cc6ae27291c9
Signed-off-by: pasta <pasta@dashboost.org>
# Conflicts:
# src/script/sign.cpp
# src/script/sign.h
620361fce8 Fix accidental use of the addition assignment operator ("+="). Remove newlines from error message. (practicalswift)
Pull request description:
Fix accidental use of the addition assignment operator (`+=`).
_Note to reviewers:_ Perhaps the `\n`:s should be removed too?
Tree-SHA512: 4e8c2dfd6025d78ef9d60522297994829dacc447e6b6782e15c0bdd5dd2daa17ca9a8948bfa9a15be57d9286092356381d7e6747980303852d273eb0df0dd76b
a192636 -blocksdir: keep blockindex leveldb database in datadir (Jonas Schnelli)
f38e4fd QA: Add -blocksdir test (Jonas Schnelli)
386a6b6 Allow to optional specify the directory for the blocks storage (Jonas Schnelli)
Pull request description:
Since the actual block files taking up more and more space, it may be desirable to have them stored in a different location then the data directory (use case: SSD for chainstate, etc., HD for blocks).
This PR adds a `-blocksdir` option that allows one to keep the blockfiles and the blockindex external from the data directory (instead of creating symlinks).
I fist had an option to keep the blockindex within the datadir, but seems to make no sense since accessing the index will (always) lead to access (r/w) the block files.
Tree-SHA512: f8b9e1a681679eac25076dc30e45e6e12d4b2d9ac4be907cbea928a75af081dbcb0f1dd3e97169ab975f73d0bd15824c00c2a34638f3b284b39017171fce2409
- Call `statusUpdateNeeded` before `tryGetTxStatus`. This allows to run
the latter only if really required i.e. if `statusUpdateNeeded` returns true.
- Reuse `cachedNumBlocks` from `WalletModel` in `TransactionTablePriv::index()` to _actually_ avoid redundant tx status updates
- Initialize `cachedNumBlocks` and `cachedChainLockHeight` with `-1` to avoid extra `pollBalanceChanged` call
bf36a3ccc212ad4d7c5cb8f26d7a22e279fe3cec gui: Fix race in WalletModel::pollBalanceChanged (Russell Yanofsky)
Pull request description:
Poll function was wrongly setting cached height to the current chain height instead of the chain height at the time of polling.
This bug could cause balances to appear out of date, and was first introduced a0704a8996 (diff-2e3836af182cfb375329c3463ffd91f8L117). Before that commit, there wasn't a problem because cs_main was held during the poll update.
Currently, the problem should be rare. But if 8937d99ce81a27ae5e1012a28323c0e26d89c50b from #17954 were merged, the problem would get worse, because the wrong cachedNumBlocks value would be set if the wallet was polled in the interval between a block being connected and it processing the BlockConnected notification.
MarcoFalke also points out that a0704a8996 could lead to GUI hangs as well, because previously the pollBalanceChanged method, which runs on the GUI thread, would only make a nonblocking TRY_LOCK(cs_main) call, but after could make blocking LOCK(cs_main) calls, potentially locking up the GUI.
Thanks to John Newbery for finding this bug this while reviewing https://github.com/bitcoin/bitcoin/pull/17954.
ACKs for top commit:
Empact:
utACK bf36a3ccc2
jonasschnelli:
utACK bf36a3c
Tree-SHA512: 1f4f229fa70a6d1fcf7be3806dca3252e86bc1755168fb421258389eb95aae67f863cb1216e6dc086b596c33560d1136215a4c87b5ff890abc8baaa3333b47f4
Made node sense to squash it into the related OptionsDialog commit of
the initial PR because this diff uses the wallet interface which gets
introduced one commit after the related commit.
