Commit Graph

13746 Commits

Author SHA1 Message Date
Wladimir J. van der Laan
8780c762e2 Merge #10985: Add undocumented -forcecompactdb to force LevelDB compactions
8842d1a Add undocumented -forcecompactdb to force LevelDB compactions (Pieter Wuille)

Pull request description:

Tree-SHA512: de91f3f574f75248fa6e5091089c840957fae5a972ebcd2b89493f7d777d4658560a6f5a3b43ab0c9b2c333ad98f9f185ae224c9caffc1a5e8df369cc414f123
2017-10-31 21:19:11 +01:00
Wladimir J. van der Laan
4102211a39 Merge #10526: Force on-the-fly compaction during pertxout upgrade
efeb273 Force on-the-fly compaction during pertxout upgrade (Pieter Wuille)

Pull request description:

  It seems that LevelDB tends to leave the old "per txid" UTXO entries in the database lying around for a significant amount of time during and after the per-txout upgrade. This introduces a `CompactRange` function in the database wrapper, and invokes it after every batch of updates in `CCoinsViewDB::Upgrade()`. This lowers temporary disk usage during and after the upgrade.

Tree-SHA512: fbf964c0a33f4e73709c999c8a2bfdef974779c15820907398a2f8828f5fa3e4e153ddd9031d6fc5083be81e22b999b9bd826fd063ad8b88f55c5e8342503290
2017-10-31 21:19:11 +01:00
Wladimir J. van der Laan
1a9add78c9 Merge #10660: Allow to cancel the txdb upgrade via splashscreen keypress 'q'
542ce6e Report [CANCELLED] instead of [DONE] when shut down during txdb upgrade (Jonas Schnelli)
83fbea3 Report txdb upgrade not more often then every 10% (Jonas Schnelli)
06c5b6e Show txdb upgrade progress in debug log (Jonas Schnelli)
316fcb5 Allow to cancel the txdb upgrade via splashscreen callback (Jonas Schnelli)
ae09d45 Allow to shut down during txdb upgrade (Jonas Schnelli)
00cb69b [Qt] allow to execute a callback during splashscreen progress (Jonas Schnelli)

Tree-SHA512: 23190f23f441bfd60821e49f8b3698a6bef97eb0e0ee659328e4a7395769ecd1616420eacc38aa1fa0ff62b9de5f13a0098dc798cdec6bff649575cefebc0db2
2017-10-31 21:19:11 +01:00
Pieter Wuille
5b232161a7 Merge #10558: Address nits from per-utxo change
21d4afa12 Comment clarifications in coins.cpp (Alex Morcos)
3c8a9aeff Add belt-and-suspenders in DisconnectBlock (Alex Morcos)

Tree-SHA512: d83e12ed71674faaaaebc03ffa1e2276984c35a29db419268ac9e14a45b33ccab716e3606dff8cfe1dcee4bec6e4794d2ca90341f10d5684be80e3fee61addf8
2017-10-31 21:19:11 +01:00
Wladimir J. van der Laan
549839a504 Merge #10581: Simplify return values of GetCoin/HaveCoin(InCache)
21180ff Simplify return values of GetCoin/HaveCoin(InCache) (Pieter Wuille)

Tree-SHA512: eae0aa64fa1308191100cdc7cdc790c825f33b066c200a18b5895d7d5806cee1cc4caba1766ef3379a7cf93dde4bbae2bc9be92947935f5741f5c126d3ee991b
2017-10-31 21:19:11 +01:00
Wladimir J. van der Laan
06aa02ff63 Merge #10559: Change semantics of HaveCoinInCache to match HaveCoin
5257698 Change semantics of HaveCoinInCache to match HaveCoin (Alex Morcos)

Tree-SHA512: 397e9ba28646b81fffa53e55064735d4d242aaffdf8484506825f785b0e414f334e4c5cd1e4e1dd9a4b6d1f6954c7ecad15429934a1c4e8d39f596cbd9f5dd80
2017-10-31 21:19:11 +01:00
Pieter Wuille
151c552c71 Merge #10537: Few Minor per-utxo assert-semantics re-adds and tweak
9417d7a33 Be much more agressive in AccessCoin docs. (Matt Corallo)
f58349ca8 Restore some assert semantics in sigop cost calculations (Matt Corallo)
3533fb4d3 Return a bool in SpendCoin to restore pre-per-utxo assert semantics (Matt Corallo)
ec1271f2b Remove useless mapNextTx lookup in CTxMemPool::TrimToSize. (Matt Corallo)

Tree-SHA512: 158a4bce063eac93e1d50709500a10a7cb1fb3271f10ed445d701852fce713e2bf0da3456088e530ab005f194ef4a2adf0c7cb23226b160cecb37a79561f29ca
2017-10-31 21:19:11 +01:00
Wladimir J. van der Laan
4f807422fa Merge #10550: Don't return stale data from CCoinsViewCache::Cursor()
3ff1fa8 Use override keyword on CCoinsView overrides (Russell Yanofsky)
24e44c3 Don't return stale data from CCoinsViewCache::Cursor() (Russell Yanofsky)

Tree-SHA512: 08699dae0925ffb9c018f02612ac6b7eaf73ec331e2f4f934f1fe25a2ce120735fa38596926e924897c203f7470e99f0a99cf70d2ce31ff428b105e16583a861
2017-10-31 21:19:11 +01:00
Alexander Block
9ad56fe18a Dash related fixes for per-utxo DB 2017-10-31 21:19:11 +01:00
Alexander Block
92bb65894b Fix address index to use new per-utxo DB 2017-10-31 21:19:11 +01:00
Alexander Block
d4562b5e51 Fix CCoinsViewCache::GetPriority to use new per-utxo 2017-10-31 21:19:11 +01:00
Pieter Wuille
c81394b975 Merge #10195: Switch chainstate db and cache to per-txout model
589827975 scripted-diff: various renames for per-utxo consistency (Pieter Wuille)
a5e02bc7f Increase travis unit test timeout (Pieter Wuille)
73de2c1ff Rename CCoinsCacheEntry::coins to coin (Pieter Wuille)
119e552f7 Merge CCoinsViewCache's GetOutputFor and AccessCoin (Pieter Wuille)
580b02309 [MOVEONLY] Move old CCoins class to txdb.cpp (Pieter Wuille)
8b25d2c0c Upgrade from per-tx database to per-txout (Pieter Wuille)
b2af357f3 Reduce reserved memory space for flushing (Pieter Wuille)
41aa5b79a Pack Coin more tightly (Pieter Wuille)
97072d668 Remove unused CCoins methods (Pieter Wuille)
ce23efaa5 Extend coins_tests (Pieter Wuille)
508307968 Switch CCoinsView and chainstate db from per-txid to per-txout (Pieter Wuille)
4ec0d9e79 Refactor GetUTXOStats in preparation for per-COutPoint iteration (Pieter Wuille)
13870b56f Replace CCoins-based CTxMemPool::pruneSpent with isSpent (Pieter Wuille)
05293f3cb Remove ModifyCoins/ModifyNewCoins (Pieter Wuille)
961e48397 Switch tests from ModifyCoins to AddCoin/SpendCoin (Pieter Wuille)
8b3868c1b Switch CScriptCheck to use Coin instead of CCoins (Pieter Wuille)
c87b957a3 Only pass things committed to by tx's witness hash to CScriptCheck (Matt Corallo)
f68cdfe92 Switch from per-tx to per-txout CCoinsViewCache methods in some places (Pieter Wuille)
000391132 Introduce new per-txout CCoinsViewCache functions (Pieter Wuille)
bd83111a0 Optimization: Coin&& to ApplyTxInUndo (Pieter Wuille)
cb2c7fdac Replace CTxInUndo with Coin (Pieter Wuille)
422634e2f Introduce Coin, a single unspent output (Pieter Wuille)
7d991b55d Store/allow tx metadata in all undo records (Pieter Wuille)
c3aa0c119 Report on-disk size in gettxoutsetinfo (Pieter Wuille)
d34242430 Remove/ignore tx version in utxo and undo (Pieter Wuille)
7e0032290 Add specialization of SipHash for 256 + 32 bit data (Pieter Wuille)
e484652fc Introduce CHashVerifier to hash read data (Pieter Wuille)
f54580e7e error() in disconnect for disk corruption, not inconsistency (Pieter Wuille)
e66dbde6d Add SizeEstimate to CDBBatch (Pieter Wuille)

Tree-SHA512: ce1fb1e40c77d38915cd02189fab7a8b125c7f44d425c85579d872c3bede3a437760997907c99d7b3017ced1c2de54b2ac7223d99d83a6658fe5ef61edef1de3
2017-10-31 21:19:11 +01:00
Pieter Wuille
afa96b7c12 Merge #10249: Switch CCoinsMap from boost to std unordered_map
e6756ad Switch CCoinsMap from boost to std unordered_map (Pieter Wuille)
344a2c4 Add support for std::unordered_{map,set} to memusage.h (Pieter Wuille)

Tree-SHA512: 51288301e7c0f29ffac8c59f4cc73ddc36b7abeb764009da6543f2eaeeb9f89bd47dde48131a7e0aefad8f7cb0b74b2f33b8be052c8e8a718339c3e6bb963447
2017-10-31 21:19:11 +01:00
Wladimir J. van der Laan
8ed6722196 Merge #10250: Fix some empty vector references
f478d98 Fix some empty vector references (Pieter Wuille)

Tree-SHA512: a22022b9060cd39f8d349e8dd24490614c0028eae2fbc7186d0d26b1d47529049b2daee41f093c0722130274a0b6f7f8671c2295c8cb4a97028771eaff080734
2017-10-31 21:19:11 +01:00
Alexander Block
d23adcc0f1 Merge #10231: [Qt] Reduce a significant cs_main lock freeze (#1704)
4082fb0 Add missing <atomic> header in clientmodel.h (Jonas Schnelli)
928d4a9 Set both time/height header caches at the same time (Jonas Schnelli)
610a917 Declare headers height/time cache mutable, re-set the methods const (Jonas Schnelli)
cf92bce Update the remaining blocks left in modaloverlay at init. (Jonas Schnelli)
7148f5e Reduce cs_main locks during modal overlay by adding an atomic cache (Jonas Schnelli)

Tree-SHA512: a92ca22f90b8b2a5e8eb94fdce531ef44542e21a8dbbb0693f7723d7018592cb68de687a2a0aac91d31cbf019793f8e922550656d2b130ed3d854d60630341db
2017-10-31 23:14:19 +03:00
UdjinM6
14d11e4a8a
Translations 201710 part2 (#1676)
* fix ru

* updating bg, fr, sk, zh_TW

* update fi, pl, zh_TW

* update pl
2017-10-31 17:50:28 +03:00
UdjinM6
470e5435c0
remove InstantSend votes for failed lock attemts after some timeout (#1705) 2017-10-31 17:50:03 +03:00
UdjinM6
dfb8dbbf61
Fix mnp relay bug (#1700)
* relay ping for nodes in ENABLED/EXPIRED/WATCHDOG_EXPIRED state only, skip everyone else

* typo
2017-10-31 17:46:42 +03:00
UdjinM6
673e161d5d
fix trafficgraphdatatests for qt4 (#1699) 2017-10-31 17:46:24 +03:00
UdjinM6
ffbe92bae5 Merge pull request #1697 from codablock/backport_bitcoin_perutxo_0.15_1
Third batch of per-utxo backports (up until 0.15, excluding actual per-utxo)
2017-10-26 23:30:47 +03:00
Alexander Block
b06a6a2e74 Fix use of missing self.log in blockchain.py
self.log will only be available in later backports. Use print until these
are backported.
2017-10-26 16:30:56 +02:00
Kamuela Franco
304b886d08 Update build-osx.md formatting (#1690)
* Update build-osx.md

* Updated based on comments in #1546

* Update grammar
2017-10-25 17:58:28 +03:00
UdjinM6
788ae63ac2 Fix edge case for IS (skip inputs that are too large) (#1695)
* fix edge case for IS (skip inputs that are too large)

* fix
2017-10-25 17:57:56 +03:00
UdjinM6
6f57519c60 allow up to 40 chars in proposal name (#1693)
align with 299035e36d
2017-10-25 17:57:39 +03:00
Matt Corallo
6f1997182a Add COMPACTSIZE wrapper similar to VARINT for serialization 2017-10-24 13:54:10 +02:00
Pieter Wuille
fc5ced317f Merge #10445: Add test for empty chain and reorg consistency for gettxoutsetinfo.
513da90cd Add test for empty chain and reorg consistency for gettxoutsetinfo. (Gregory Maxwell)
822755a42 Fix: make CCoinsViewDbCursor::Seek work for missing keys (Pieter Wuille)

Tree-SHA512: e549921e8b8f599bf61ebe0ee7ef1d2f474043723d633e24665fe434b996a98e039612de8a1c2cd16b63f154943ff5ea1c1935e9561cfb813a00d47d926d0b22
2017-10-24 13:54:07 +02:00
Pieter Wuille
9a266e68d0 Merge #10297: Simplify DisconnectBlock arguments/return value
db994b2 Simplify DisconnectBlock arguments/return value (Pieter Wuille)

Tree-SHA512: 62ce1a85bde2a5baffb9173ed28f2d8008200ecf8b09332122f1516fe68b33b9d7223cc1c2fffe804e38f767874c6353b76bd483e8ad7d48c4a5e80d6b683039
2017-10-24 13:54:04 +02:00
Matt Corallo
567043d360 Make DisconnectBlock and ConnectBlock static in validation.cpp 2017-10-24 13:54:00 +02:00
Wladimir J. van der Laan
ff9b2967af Merge #10133: Clean up calculations of pcoinsTip memory usage
1b55e07 Make threshold for flushing more conservative. (Alex Morcos)
f33afd3 Lower default memory footprint slightly (Alex Morcos)
5b95a19 Make pcoinsTip memory calculations consistent (Alex Morcos)

Tree-SHA512: d0061138596cf89008397b8729d9b25293938b1ad454cc99a6fe2f6210e94f76dfa78a8f0fce4c1ba3efec4e742a9c1a3ab26676a4a8346d3e7c3055d032669b
2017-10-24 13:53:59 +02:00
Wladimir J. van der Laan
cfefd34f45 Merge #10126: Compensate for memory peak at flush time
7228ce8 Compensate for memory peak at flush time (Pieter Wuille)

Tree-SHA512: 97e9848410fab061402c85d8440c54a50dd8a0203b2ea194013ea116700a6dc1b4b26b8c5f9c9c68c1f5c6b935c5d6c737437c1911b003d9ff5445c570cd449d
2017-10-24 13:53:55 +02:00
Wladimir J. van der Laan
249db27761 Merge #9999: [LevelDB] Plug leveldb logs to bitcoin logs
cfce581 [LevelDB] Plug leveldb logs to bitcoin logs (NicolasDorier)

Tree-SHA512: e40a2c2644c269bb2da7be04aec39ff64ad350d508391750a757955ed3f9d96998775d01e04b282a75b36d776c3960a345cc7b6f1466e6ae167d27518bf4baee
2017-10-24 13:53:52 +02:00
UdjinM6
53f9fd9b27 Merge pull request #1696 from codablock/backport_bitcoin_perutxo_0.14
Second batch of per-utxo backports (up until 0.14)
2017-10-24 12:05:40 +03:00
Pieter Wuille
817ecc03d3 Merge #9353: Add data() method to CDataStream (and use it)
5113474 wallet: Use CDataStream.data() (Wladimir J. van der Laan)
e2300ff bench: Use CDataStream.data() (Wladimir J. van der Laan)
adff950 dbwrapper: Use new .data() method of CDataStream (Wladimir J. van der Laan)
a2141e4 streams: Remove special cases for ancient MSVC (Wladimir J. van der Laan)
af4c44c streams: Add data() method to CDataStream (Wladimir J. van der Laan)
2017-10-23 19:07:45 +02:00
Pieter Wuille
ceb64fcd47 Merge #8610: Share unused mempool memory with coincache
ba3cecf Share unused mempool memory with coincache (Pieter Wuille)
2017-10-23 19:07:45 +02:00
Pieter Wuille
09b3e042f4 Merge #9310: Assert FRESH validity in CCoinsViewCache::BatchWrite
dd44ea3 Check FRESH validity in CCoinsViewCache::BatchWrite (Russell Yanofsky)
2017-10-23 19:07:45 +02:00
Pieter Wuille
a589c94a9a Merge #9107: Safer modify new coins
b50cd7a Fix dangerous condition in ModifyNewCoins. (Alex Morcos)
2017-10-23 19:07:45 +02:00
Wladimir J. van der Laan
4b4d222933 Merge #9308: [test] Add CCoinsViewCache Access/Modify/Write tests
07df40b [test] Add CCoinsViewCache Access/Modify/Write tests (Russell Yanofsky)
2017-10-23 19:07:45 +02:00
Pieter Wuille
6974f1723d Merge #9346: Batch construct batches
ed58969 Batch construct batches (Pieter Wuille)
2017-10-23 19:07:45 +02:00
Pieter Wuille
6fbe93aa71 Merge #9307: Remove undefined FetchCoins method declaration
fe41f58 Remove undefined FetchCoins method declaration (Russell Yanofsky)
2017-10-23 19:07:45 +02:00
Pieter Wuille
105fd18157 Use fixed preallocation instead of costly GetSerializeSize
Dbwrapper used GetSerializeSize() to compute the size of the buffer
to preallocate. For some cases (specifically: CCoins) this requires
a costly compression call. Avoid this by just using fixed size
preallocations instead.
2017-10-23 19:07:45 +02:00
Pieter Wuille
94e01eb660 Merge #8467: [Trivial] Do not shadow members in dbwrapper
4a35e0f Do not shadow members in dbwrapper (Pavel Janík)
2017-10-23 19:02:49 +02:00
UdjinM6
0fdd11ced5 Merge pull request #1691 from codablock/backport_bitcoin_perutxo_0.13
First batch of per-utxo backports (up until 0.13)
2017-10-23 19:57:06 +03:00
Wladimir J. van der Laan
e3b7ed449f Merge #8273: Bump -dbcache default to 300MiB
efd1d83 doc: Mention dbcache increase in release notes (Wladimir J. van der Laan)
32cab91 Bump `-dbcache` default to 300MiB (Wladimir J. van der Laan)
2017-10-19 10:51:59 +02:00
Wladimir J. van der Laan
952383e16c Merge #7933: Fix OOM when deserializing UTXO entries with invalid length
1e44169 Add tests for CCoins deserialization (Pieter Wuille)
5d0434d Fix OOM bug: UTXO entries with invalid script length (Pieter Wuille)
4bf631e CDataStream::ignore Throw exception instead of assert on negative nSize. (Patrick Strateman)
4f87af6 Treat overly long scriptPubKeys as unspendable (Pieter Wuille)
f8e6fb1 Introduce constant for maximum CScript length (Pieter Wuille)
2017-10-19 10:51:50 +02:00
Wladimir J. van der Laan
753cb1563b Merge #7874: Improve AlreadyHave
c6cb6f7 Avoid unnecessary database access for unknown transactions (Alex Morcos)
2017-10-19 10:51:45 +02:00
Alexander Block
60e6a602e6 Use C++11 thread-safe static initializers in coins.h/coins.cpp
Taken from Bitcoin #8173/888483098e60f2a944f1d246bbfec4d14a2975f8

Needed to align code for upcoming per-utxo backports.
2017-10-19 10:51:40 +02:00
Pieter Wuille
44526af95a Use SipHash-2-4 for CCoinsCache index
This is ~1.7x slower than the Lookup3-of-Xor-with-salt construct we were
using before, but it is a primitive designed for exactly this.
2017-10-19 10:51:28 +02:00
Pieter Wuille
d92b454a25 Add SipHash-2-4 primitives to hash 2017-10-19 10:51:26 +02:00
Alexander Block
02a6cef94d Move index structures into spentindex.h
txdb.h does not include validation.h anymore, so we need these structs somewhere else
2017-10-19 10:51:23 +02:00
Wladimir J. van der Laan
abaf524f0c Merge #7815: Break circular dependency main ↔ txdb
99e7075 Break circular dependency main ↔ txdb (Wladimir J. van der Laan)
2017-10-19 10:51:12 +02:00