Commit Graph

16828 Commits

Author SHA1 Message Date
MarcoFalke
4bfc6ab303 Merge #13788: Fix --disable-asm for newer assembly checks/code
4207c1b35c configure: Initialise assembly enable_* variables (Luke Dashjr)
afe0875577 configure: Skip assembly support checks, when assembly is disabled (Luke Dashjr)
d8ab8dc12d configure: Invert --enable-asm help string since default is now enabled (Luke Dashjr)

Pull request description:

  Fixes #13759

  Also inverts the help (so it shows `--disable-asm` like other enabled-by-default options, and initialises the flag variables.

ACKs for commit 4207c1:
  laanwj:
    makes sense, utACK 4207c1b35c2e2ee1c9217cc7db3290a24c3b4b52
  achow101:
    utACK 4207c1b35c2e2ee1c9217cc7db3290a24c3b4b52
  ken2812221:
    ACK 4207c1b35c2e2ee1c9217cc7db3290a24c3b4b52
  practicalswift:
    tACK 4207c1b35c2e2ee1c9217cc7db3290a24c3b4b52

Tree-SHA512: a30be1008fd8f019db34073f78e90a3c4ad3767d88d7c20ebb83e99c7abc23552f7da3ac8bd20f727405799aff1ecb6044cf869653f8db70478a074d0b877e0a
2019-10-01 23:20:06 +02:00
Wladimir J. van der Laan
1b2252c28c Merge #13386: SHA256 implementations based on Intel SHA Extensions
66b2cf1ccfad545a8ec3f2a854e23f647322bf30 Use immintrin.h everywhere for intrinsics (Pieter Wuille)
4c935e2eee456ff66cdfb908b0edffdd1e8a6c04 Add SHA256 implementation using using Intel SHA intrinsics (Pieter Wuille)
268400d3188200c9e3dcd3482c4853354388a721 [Refactor] CPU feature detection logic for SHA256 (Pieter Wuille)

Pull request description:

  Based on #13191.

  This adds SHA256 implementations that use Intel's SHA Extension instructions (using intrinsics). This needs GCC 4.9 or Clang 3.4.

  In addition to #13191, two extra implementations are provided:
  * (a) A variable-length SHA256 implementation using SHA extensions.
  * (b) A 2-way 64-byte input double-SHA256 implementation using SHA extensions.

  Benchmarks for 9001-element Merkle tree root computation on an AMD Ryzen 1800X system:
  * Using generic C++ code (pre-#10821): 6.1ms
  * Using SSE4 (master, #10821): 4.6ms
  * Using 4-way SSE4 specialized for 64-byte inputs (#13191): 2.8ms
  * Using 8-way AVX2 specialized for 64-byte inputs (#13191): 2.1ms
  * Using 2-way SHA-NI specialized for 64-byte inputs (this PR): 0.56ms

  Benchmarks for 32-byte SHA256 on the same system:
  * Using SSE4 (master, #10821): 190ns
  * Using SHA-NI (this PR): 53ns

  Benchmarks for 1000000-byte SHA256 on the same system:
  * Using SSE4 (master, #10821): 2.5ms
  * Using SHA-NI (this PR): 0.51ms

Tree-SHA512: 2b319e33b22579f815d91f9daf7994a5e1e799c4f73c13e15070dd54ba71f3f6438ccf77ae9cbd1ce76f972d9cbeb5f0edfea3d86f101bbc1055db70e42743b7
2019-10-01 23:20:06 +02:00
Wladimir J. van der Laan
1df17c02e5 Merge #13393: Enable double-SHA256-for-64-byte code on 32-bit x86
57ba401abcfe564a2c4d259e0f758401ed74616d Enable double-SHA256-for-64-byte code on 32-bit x86 (Pieter Wuille)

Pull request description:

  The SSE4 and AVX2 double-SHA256-for-64-byte input code from #13191 compiles fine on 32-bit x86 systems, but the autodetection logic in sha256.cpp doesn't enable it. Fix this.

  Note that these instruction sets are only available on CPUs that support 64-bit mode as well, so it is only beneficial in the (perhaps unlikely) scenario where a 64-bit CPU is running a 32-bit Bitcoin Core binary.

Tree-SHA512: 39d5963c1ba8c33932549d5fe98bd184932689a40aeba95043eca31dd6824f566197c546b60905555eccaf407408a5f0f200247bb0907450d309b0a70b245102
2019-10-01 23:20:06 +02:00
Wladimir J. van der Laan
d07f54b967 Merge #13471: For AVX2 code, also check for AVX, XSAVE, and OS support
32d153fa360f73b4999701b97d55b12318fd2659 For AVX2 code, also check for AVX, XSAVE, and OS support (Pieter Wuille)

Pull request description:

  Fixes #12903.

Tree-SHA512: 01e71efb5d3a43c49a145a5b1dc4fe7d0a491e1e78479e7df830a2aaac57c3dcfc316e28984c695206c76f93b68e4350fc037ca36756ca579b7070e39c835da2
2019-10-01 23:20:06 +02:00
Wladimir J. van der Laan
e44b6c7e58 Merge #13438: Improve coverage of SHA256 SelfTest code
1e1eb6367f67dcf968bb62993b98b5873b926fc0 Improve coverage of SHA256 SelfTest code (Pieter Wuille)

Pull request description:

  The existing SelfTest code does not cover the specialized double-SHA256-for-64-byte-inputs transforms added in #13191. Fix this.

Tree-SHA512: 593c7ee5dc9e77fc4c89e0a7753a63529b0d3d32ddbc015ae3895b52be77bee8a80bf16b754b30a22c01625a68db83fb77fa945a543143542bebb5b0f017ec5b
2019-10-01 23:20:06 +02:00
Wladimir J. van der Laan
adfd2e1a83 Merge #13408: crypto: cleanup sha256 build
f68049dd879c216d1e98b6635eec488f8e936ed4 crypto: cleanup sha256 build (Cory Fields)

Pull request description:

  Requested by @sipa in #13386.

  Rather than appending all possible cpu variants to all targets, create a convenience variable that encompasses all.

Tree-SHA512: 8e9ab2185515672b79bb7925afa4f3fbfe921bfcbe61456833d15457de4feba95290de17514344ce42ee81cc38b252476cd0c29432ac48c737c2225ed515a4bd
2019-10-01 23:20:06 +02:00
Wladimir J. van der Laan
5a23934df1 Merge #13191: Specialized double-SHA256 with 64 byte inputs with SSE4.1 and AVX2
4defdfab94504018f822dc34a313ad26cedc8255 [MOVEONLY] Move unused Merkle branch code to tests (Pieter Wuille)
4437d6e1f3107a20a8c7b66be8b4b972a82e3b28 8-way AVX2 implementation for double SHA256 on 64-byte inputs (Pieter Wuille)
230294bf5fdeba7213471cd0b795fb7aa36e5717 4-way SSE4.1 implementation for double SHA256 on 64-byte inputs (Pieter Wuille)
1f0e7ca09c9d7c5787c218156fa5096a1bdf2ea8 Use SHA256D64 in Merkle root computation (Pieter Wuille)
d0c96328833127284574bfef26f96aa2e4afc91a Specialized double sha256 for 64 byte inputs (Pieter Wuille)
57f34630fb6c3e218bd19535ac607008cb894173 Refactor SHA256 code (Pieter Wuille)
0df017889b4f61860092e1d54e271092cce55f62 Benchmark Merkle root computation (Pieter Wuille)

Pull request description:

  This introduces a framework for specialized double-SHA256 with 64 byte inputs. 4 different implementations are provided:
  * Generic C++ (reusing the normal SHA256 code)
  * Specialized C++ for 64-byte inputs, but no special instructions
  * 4-way using SSE4.1 intrinsics
  * 8-way using AVX2 intrinsics

  On my own system (AVX2 capable), I get these benchmarks for computing the Merkle root of 9001 leaves (supported lengths / special instructions / parallellism):
  * 7.2 ms with varsize/naive/1way (master, non-SSE4 hardware)
  * 5.8 ms with size64/naive/1way (this PR, non-SSE4 capable systems)
  * 4.8 ms with varsize/SSE4/1way (master, SSE4 hardware)
  * 2.9 ms with size64/SSE4/4way (this PR, SSE4 hardware)
  * 1.1 ms with size64/AVX2/8way (this PR, AVX2 hardware)

Tree-SHA512: efa32d48b32820d9ce788ead4eb583949265be8c2e5f538c94bc914e92d131a57f8c1ee26c6f998e81fb0e30675d4e2eddc3360bcf632676249036018cff343e
2019-10-01 23:20:06 +02:00
Wladimir J. van der Laan
ad55048a68 Merge #11176: build: Rename --enable-experimental-asm to --enable-asm and enable by default
538cc0ca8 build: Mention use of asm in summary (Wladimir J. van der Laan)
ce5381e7f build: Rename --enable-experimental-asm to --enable-asm and enable by default (Wladimir J. van der Laan)

Pull request description:

  Now that 0.15 is branched off, enable assembler SHA256 optimizations by default, but still allow disabling them, for example if something goes wrong with auto-detection on a platform.

  Also add mention of the use of asm in the configure summary.

Tree-SHA512: cd20c497f65edd6b1e8b2cc3dfe82be11fcf4777543c830ccdec6c10f25eab4576b0f2953f3957736d7e04deaa4efca777aa84b12bb1cecb40c258e86c120ec8
2019-10-01 23:20:06 +02:00
Alexander Block
b4aefb513d Also consider txindex for transactions in AlreadyHave() (#3126)
This avoids many false negatives where TXs are announced to us which are
already mined and then were spent later.
2019-10-01 17:24:42 +03:00
UdjinM6
d9e98e31ea
Fix scripted diff check condition (#3128) 2019-10-01 17:16:26 +03:00
UdjinM6
bad3243b8e
Bump mocktime before generating new blocks and generate a few blocks at the end of test_mempool_doublespend in p2p-instantsend.py (#3125)
* Cleanup p2p-instantsend.py

Bump mocktime before generating new blocks and generate a few blocks at the end of test_mempool_doublespend to clean things up

* Update test/functional/p2p-instantsend.py

Co-Authored-By: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
2019-10-01 17:15:28 +03:00
UdjinM6
82ebba18f4
Few fixes for wait_for_instantlock (#3123)
* Fix `wait_for_instantlock` to make it fail if instantlock wasn't aquired, use `wait_until`

Currently it simply returns False if islock failed but that's not the way we use it (we never check results).

* Wait for txes to propagate before checking for instantlock
2019-10-01 17:14:26 +03:00
UdjinM6
7ba50d93f6
Merge pull request #3116 from codablock/pr_backports_v15_v16_4
Backport remaining PRs which were also backported into Bitcoin 0.15
2019-10-01 17:13:15 +03:00
UdjinM6
a2fa9bb7eb Ignore recent rejects filter for locked txes (#3124)
* Ignore recent rejects filter for locked txes

If we had a conflicting tx in the mempool before the locked tx arrived and the locked one arrived before the corresponding islock (i.e. we don't really know it's the one that should be included yet), the locked one is going to be rejected due to a mempool conflict. The old tx is going to be removed from the mempool by an incoming islock a bit later, however, we won't be able to re-request the locked tx until the tip changes because of the recentRejects filter. This patch fixes it.

* Add some explanation
2019-10-01 09:36:33 +02:00
Alexander Block
2ca2138fc8
Whitelist nodes in llmq-dkgerrors.py (#3112)
This avoids banning due to invalid sigs
2019-10-01 09:25:12 +02:00
Alexander Block
a8fa5cff9c
Make orphan TX map limiting dependent on total TX size instead of TX count (#3121) 2019-09-30 15:34:13 +02:00
PastaPastaPasta
746b5f8cda Remove commented out code (#3117)
* Remove LogPrints which have been commented out.

We have version control systems for a reason, if we want code to not run it should be removed. I personally see no value in keeping these around. I presume at one point they were spamming debug.log so we commented them out, but we really should have just removed them.

I believe all of this is dash specific code but any conflicts this does create are so minor they are not of concern imo.

Signed-off-by: Pasta <pasta@dashboost.org>

* remove a couple of extra comments

Signed-off-by: Pasta <pasta@dashboost.org>

* remove commented out code

Signed-off-by: Pasta <pasta@dashboost.org>
2019-09-30 09:55:26 +02:00
Alexander Block
e2b4b205ea Remove now redundant and actually erroneous throwing of "Invalid blockhash"
bitcoin#11565 achieved the same goal and due to incorrect merge conflict
resolution this resulted in always throwing an exception.
2019-09-30 08:33:34 +02:00
MarcoFalke
f38caa9884 Merge #11676: contrib/init: Update openrc-run filename
2f041f0e7 contrib/init: Update openrc-run filename (Luke Dashjr)

Pull request description:

  OpenRC changed their program binary names in 2014 (3 years ago), and using the old names has loud warnings now

Tree-SHA512: 2b81802b21c32b8df6010142f9593c0b6cc814a052f83b7f5654f6885566e8dbcaf4da772145fa2cf5d94c16c2fb488c5d4879f71021407c4d7b3a3b7e7ed21e
2019-09-30 08:33:34 +02:00
Wladimir J. van der Laan
617c886233 Merge #11289: Add wallet backup text to import* and add* RPCs
c098c58 Wrap dumpwallet warning and note scripts aren't dumped (MeshCollider)
a38bfbc Add wallet backup text to import*, add* and dumpwallet RPCs (MeshCollider)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/11243

  Adds "Requires a new wallet backup" text to `addwitnessaddress`, `importprivkey`, `importmulti`, `importaddress`, `importpubkey`, and `addmultisigaddress`. Also adds a warning to `dumpwallet` that backing up the seed alone is not sufficient to back up non-HD addresses

Tree-SHA512: 76d7cdca54d5b458acf479154620322391b889922525fddd6153f4164cfee393ad743757400cb8f6b1b30f24947df68ea9043b4e509f7df77a8fa05dda370933
2019-09-30 08:33:33 +02:00
Wladimir J. van der Laan
d7119e6487 Merge #11277: Fix uninitialized URI in batch RPC requests
4526d21 Add test for multiwallet batch RPC calls (Russell Yanofsky)
74182f2 Add missing batch rpc calls to python coverage logs (Russell Yanofsky)
505530c Add missing multiwallet rpc calls to python coverage logs (Russell Yanofsky)
9f67646 Make AuthServiceProxy._batch method usable (Russell Yanofsky)
e02007a Limit AuthServiceProxyWrapper.__getattr__ wrapping (Russell Yanofsky)
edafc71 Fix uninitialized URI in batch RPC requests (Russell Yanofsky)

Pull request description:

  This fixes "Wallet file not specified" errors when making batch wallet RPC calls with more than one wallet loaded. This issue was reported by @NicolasDorier in https://github.com/bitcoin/bitcoin/issues/11257

  Request URI is not used for anything except multiwallet request dispatching, so this change has no other effect.

Tree-SHA512: b3907af48a6323f864bb045ee2fa56b604188b835025ef82ba3d81673244c04228d796323cec208a676e7cd578a95ec7c7ba1e84d0158b93844d5dda8f6589b9
2019-09-30 08:33:16 +02:00
MarcoFalke
18f104b97e Merge #11590: [Wallet] always show help-line of wallet encryption calls
720d9e8fa [Wallet] always show help-line of wallet encryption calls (Jonas Schnelli)

Pull request description:

  We do currently show/hide the wallet encryption RPC calls from the help if the current wallet.
  In case of an encrypted wallet, `encryptwallet` is hidden and `walletpassphrasechange`, `walletpassphrasechange` and `walletlock` do appear in the help.

  This is no longer ideal in case of multiwallet due to the fact that one may want help infos in order to target a specific wallet.

  IMO its preferable to have a static help screen (show everything always). The currently show/hidden calls do handle the possible invalid encryption-state fine.

  Fixes #11588

Tree-SHA512: 513fecd15248a31361f5143685e8cdeb63dfd3fa7120828917e1db54d936dc3db60d48ce46efa5c3a563a48157fe962689879856eeeed53f904686b12aec204e
2019-09-30 08:33:16 +02:00
MarcoFalke
569a11fef4 Merge #11554: Sanity-check script sizes in bitcoin-tx
a6f33ea77 Sanity-check script sizes in bitcoin-tx (Matt Corallo)

Pull request description:

Tree-SHA512: bb8ecb628763af23816ab085758f6140920a6ff05dcb298129c2bbe584a02a759c700a05740eca77023292c98a5658b2a608fa27d5a948d183f87ed9ab827952
2019-09-30 08:33:16 +02:00
Wladimir J. van der Laan
7919c96818 Merge #11539: [verify-commits] Allow revoked keys to expire
d23be30 [verify-commits] Allow revoked keys to expire (Matt Corallo)

Pull request description:

  This should fix verify-commits on master.

Tree-SHA512: 9bfca41fdfcdb11f6d07fcbc80a7b2de37706051e963292e0fbb4c608f146c87b65ab1e8395792197b4a7099e89fa045f278a60276672f6540b68d5e15b5a4a7
2019-09-30 08:33:16 +02:00
Alexander Block
55550b8dd7 Add missing comment
To align with bitcoin#11538
2019-09-30 08:33:16 +02:00
Wladimir J. van der Laan
46373f5ee2 Merge #11565: Make listsinceblock refuse unknown block hash
659b206 Make listsinceblock refuse unknown block hash (Russell Yanofsky)

Pull request description:

  Change suggested by @theuni  who noticed listsinceblock would ignore invalid block hashes causing it to return a completely unfiltered list of transactions.

Tree-SHA512: 3c8fb160265780d1334e856e853ab48e2e18372b8f1fc71ae480c3f45317048cc1fee0055d5c58031981a91b9c2bdbeb8e49a889d04ecba61729ce8109f2ce3f
2019-09-30 08:33:16 +02:00
Alexander Block
4096433ad9 Update OpenBSD build docs as in bitcoin#11442 2019-09-30 08:33:16 +02:00
Wladimir J. van der Laan
c6c337152f Merge #11530: Add share/rpcuser to dist. source code archive
fa81534 Add share/rpcuser to dist. source code archive (MarcoFalke)

Pull request description:

  As the legacy rpcuser and rpcpassword are deprected since 0.12.0, we should actually include the script to generate the new auth pair in the distributed source code archive.

  Ref: #6753

  (Tagging for backport, since it is a trivial bugfix)

Tree-SHA512: f2737957a92396444573f41071a785be5fb318df9efeb3ade7e56b3b56d512e5f9ca36723365fe5be8aaee69c5e8d8ed1178510bf02186c848b3910ee001ecb9
2019-09-30 08:33:16 +02:00
Wladimir J. van der Laan
f8aca185b1 Merge #11521: travis: move back to the minimal image
3d1c311 Revert "travis: filter out pyenv" (Cory Fields)
a86e81b travis: move back to the minimal image (Cory Fields)

Pull request description:

  The most recent update replaced the minimal image with a large one for the
  'generic' image. Switching back to 'minimal' should reduce dependencies and
  maybe speed us up some.

  It should also eliminiate the need for aa2e0f09e.

Tree-SHA512: 0e5f3e97e8d97add07ea228bc5ce1e51e8e069950dbb2871a7eece297995f20b671afdf1c68211ce404cba3ba393d61dfef30ed54d46d6805fde9388f6b4455e
2019-09-30 08:33:16 +02:00
Alexander Block
b2c6b48031
Merge pull request #3113 from codablock/pr_backports_v15_v16_5
Backport same PRs as done in bitcoin#11592
2019-09-30 08:32:16 +02:00
Alexander Block
424ed32dbb Few assert_raises_jsonrpc -> assert_raises_rpc_error fixes 2019-09-29 12:48:49 +02:00
Alexander Block
60ef97cc41 Adjust STALE_CHECK_INTERVAL to be 2.5 minutes instead of 10 minutes
This value was meant to mimic the block time in Bitcoin, so should be set
to 2.5 minutes in Dash.
2019-09-29 12:42:14 +02:00
Alexander Block
71d39e6a42 Don't disconnect masternodes just because they were slow in block announcement 2019-09-29 12:42:14 +02:00
Alexander Block
438a972a72 Fix minchainwork.py 2019-09-29 12:42:14 +02:00
Alexander Block
9a96c06057 Remove uses of NODE_WITNESS 2019-09-29 12:42:14 +02:00
Wladimir J. van der Laan
3e8962323e Merge #11593: rpc: work-around an upstream libevent bug
97932cd rpc: further constrain the libevent workaround (Cory Fields)
6b58360 rpc: work-around an upstream libevent bug (Cory Fields)

Pull request description:

  A rare race condition may trigger while awaiting the body of a message.

  This may fix some reported rpc hangs/crashes.

  This work-around mimics what libevent does internally once a write has started, which is what usually happens, but not always due to the processing happening on a different thread: e7ff4ef2b4/http.c (L373)

  Fixed upstream at: 5ff8eb2637

Tree-SHA512: b9fa97cae9da2a44101c5faf1e3be0b9cbdf722982d35541cf224be31430779c75e519c8ed18d06ab7487bfb1211069b28f22739f126d6c28ca62d3f73b79a52
2019-09-29 12:42:14 +02:00
Wladimir J. van der Laan
58cb7e38f4 Merge #11560: Connect to a new outbound peer if our tip is stale
6262915 Add unit test for stale tip checking (Suhas Daftuar)
83df257 Add CConnmanTest to mutate g_connman in tests (João Barbosa)
ac7b37c Connect to an extra outbound peer if our tip is stale (Suhas Daftuar)
db32a65 Track tip update time and last new block announcement from each peer (Suhas Daftuar)
2d4327d net: Allow connecting to extra outbound peers (Suhas Daftuar)

Pull request description:

  This is an alternative approach to #11534.  Rather than disconnect an outbound peer when our tip looks stale, instead try to connect to an additional outbound peer.

  Periodically, check to see if we have more outbound peers than we target (ie if any extra peers are in use), and if so, disconnect the one that least recently announced a new block (breaking ties by choosing the newest peer that we connected to).

Tree-SHA512: 8f19e910e0bb36867f81783e020af225f356451899adfc7ade1895d6d3bd5afe51c83759610dfd10c62090c4fe404efa0283b2f63fde0bd7da898a1aaa7fb281
2019-09-29 12:42:14 +02:00
Alexander Block
859b962a29 Move DEFAULT_BLOCK_RECONSTRUCTION_EXTRA_TXN up
This avoids future merge conflicts
2019-09-29 12:42:14 +02:00
Wladimir J. van der Laan
7d21a78fa1 Merge #11531: Check that new headers are not a descendant of an invalid block (more effeciently)
f3d4adf Make p2p-acceptablock not an extended test (Matt Corallo)
00dcda6 [qa] test that invalid blocks on an invalid chain get a disconnect (Matt Corallo)
015a525 Reject headers building on invalid chains by tracking invalidity (Matt Corallo)
932f118 Accept unrequested blocks with work equal to our tip (Matt Corallo)
3d9c70c Stop always storing blocks from whitelisted peers (Matt Corallo)
3b4ac43 Rewrite p2p-acceptblock in preparation for slight behavior changes (Matt Corallo)

Pull request description:

  @sdaftuar pointed out that the version in #11487 was somewhat DoS-able as someone could feed you a valid chain that forked off the the last checkpoint block and force you to do lots of work just walking backwards across blocks for each new block they gave you. We came up with a few proposals but settled on the one implemented here as likely the simplest without obvious DoS issues. It uses our existing on-load mapBlockIndex walk to make sure everything that descends from an invalid block is marked as such, and then simply caches blocks which we attempted to connect but which were found to be invalid. To avoid DoS issues during IBD, this will need to depend on #11458.

  Includes tests from #11487.

Tree-SHA512: 46aff8332908e122dae72ceb5fe8cd241902c2281a87f58a5fb486bf69d46458d84a096fdcb5f3e8e07fbcf7466232b10c429f4d67855425f11b38ac0bf612e1
2019-09-29 12:42:14 +02:00
Wladimir J. van der Laan
c743111c66 Merge #11578: net: Add missing lock in ProcessHeadersMessage(...)
2530bf2 net: Add missing lock in ProcessHeadersMessage(...) (practicalswift)

Pull request description:

  Add missing lock in `ProcessHeadersMessage(...)`.

  Reading the variable `mapBlockIndex` requires holding the mutex `cs_main`.

  The new "Disconnect outbound peers relaying invalid headers" code added in commit 37886d5e2f and merged as part of #11568 two days ago did not lock `cs_main` prior to accessing `mapBlockIndex`.

Tree-SHA512: b799c234be8043d036183a00bc7867bbf3bd7ffe3baa94c88529da3b3cd0571c31ed11dadfaf29c5b8498341d6d0a3c928029a43b69f3267ef263682c91563a3
2019-09-29 12:42:14 +02:00
Pieter Wuille
2e980018cf Merge #11568: Disconnect outbound peers on invalid chains
37886d5e2 Disconnect outbound peers relaying invalid headers (Suhas Daftuar)
4637f1852 moveonly: factor out headers processing into separate function (Suhas Daftuar)

Pull request description:

  Alternate to #11446.

  Disconnect outbound (non-manual) peers that serve us block headers that are already known to be invalid, but exempt compact block announcements from such disconnects.

  We restrict disconnection to outbound peers that are using up an outbound connection slot, because we rely on those peers to give us connectivity to the honest network (our inbound peers are not chosen by us and hence could all be from an attacker/sybil).  Maintaining connectivity to peers that serve us invalid headers is sometimes desirable, eg after a soft-fork, to protect unupgraded software from being partitioned off the honest network, so we prefer to only disconnect when necessary.

  Compact block announcements are exempted from this logic to comply with BIP 152, which explicitly permits nodes to relay compact blocks before fully validating them.

Tree-SHA512: 3ea88e4ccc1184f292a85b17f800d401d2c3806fefc7ad5429d05d6872c53acfa5751e3df83ce6b9c0060ab289511ed70ae1323d140ccc5b12e3c8da6de49936
2019-09-29 12:42:14 +02:00
Wladimir J. van der Laan
b451094e2e Merge #11490: Disconnect from outbound peers with bad headers chains
e065249 Add unit test for outbound peer eviction (Suhas Daftuar)
5a6d00c Permit disconnection of outbound peers on bad/slow chains (Suhas Daftuar)
c60fd71 Disconnecting from bad outbound peers in IBD (Suhas Daftuar)

Pull request description:

  The first commit will disconnect an outbound peer that serves us a headers chain with insufficient work while we're in IBD.

  The second commit introduces a way to disconnect outbound peers whose chains fall out of sync with ours:

  For a given outbound peer, we check whether their best known block (which is known from the blocks they announce to us) has at least as much work as our tip.  If it doesn't, we set a 20 minute timeout, and if we still haven't heard about a block with as much work as our tip had when we set the timeout, then we send a single getheaders message, and wait 2 more minutes.  If after two minutes their best known block has insufficient work, we disconnect that peer.

  We protect 4 of our outbound peers (who provide some "good" headers chains, ie a chain with at least as much work as our tip at some point) from being subject to this logic, to prevent excessive network topology changes as a result of this algorithm, while still ensuring that we have a reasonable number of nodes not known to be on bogus chains.

  We also don't require our peers to be on the same chain as us, to prevent accidental partitioning of the network in the event of a chain split.  Note that if our peers are ever on a more work chain than our tip, then we will download and validate it, and then either reorg to it, or learn of a consensus incompatibility with that peer and disconnect.  This PR is designed to protect against peers that are on a less work chain which we may never try to download and validate.

Tree-SHA512: 2e0169a1dd8a7fb95980573ac4a201924bffdd724c19afcab5efcef076fdbe1f2cec7dc5f5d7e0a6327216f56d3828884f73642e00c8534b56ec2bb4c854a656
2019-09-29 12:42:14 +02:00
MarcoFalke
c35aa663bc Merge #11326: Fix crash on shutdown with invalid wallet
77939f27f Fix uninitialized g_connman crash in Shutdown() (MeshCollider)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/11312

  As @dooglus pointed out, `g_connman` is uninitialized when an invalid wallet path is passed on start up, but then dereferenced in `Shutdown()`, so this tiny PR just fixes that.

Tree-SHA512: 2557133422a6e393017081450a7e6c100fe7d9ce36e628e5f5f479bc07617a7bd9a9ad4d44c0d8abadf2e3eb62a11ce9743abc27b4ae8c20f709e72df4f25a7f
2019-09-29 12:42:14 +02:00
Alexander Block
38a45a53d2 Make CDBEnv::IsMock() const 2019-09-29 12:42:14 +02:00
Alexander Block
bf7485213b More "connman." to "connman->" changes 2019-09-29 12:42:14 +02:00
Wladimir J. van der Laan
2edc29ee32 Merge #10756: net processing: swap out signals for an interface class
2525b972a net: stop both net/net_processing before destroying them (Cory Fields)
80e2e9d0c net: drop unused connman param (Cory Fields)
8ad663c1f net: use an interface class rather than signals for message processing (Cory Fields)
28f11e940 net: pass CConnman via pointer rather than reference (Cory Fields)

Pull request description:

  See individual commits.
  Benefits:
  - Allows us to begin moving stuff out of CNode and into CNodeState (after #10652 and follow-ups)
  - Drops boost dependency and overhead
  - Drops global signal registration
  - Friendlier backtraces

Tree-SHA512: af2038c959dbec25f0c90c74c88dc6a630e6b9e984adf52aceadd6954aa463b6aadfccf979c2459a9f3354326b5077ee02048128eda2a649236fadb595b66ee3
2019-09-29 12:42:14 +02:00
Pieter Wuille
f8c310a974 Merge #11456: Replace relevant services logic with a function suite.
15f5d3b17 Switch DNSSeed-needed metric to any-automatic-nodes, not services (Matt Corallo)
5ee88b4bd Clarify docs for requirements/handling of addnode/connect nodes (Matt Corallo)
57edc0b0c Rename fAddnode to a more-descriptive "manual_connection" (Matt Corallo)
44407100f Replace relevant services logic with a function suite. (Matt Corallo)

Pull request description:

  This was mostly written as a way to clean things up so that the NETWORK_LIMITED PR (#10387) can be simplified a ton, but its also a nice standalone cleanup that will also require a bit of review because it tweaks a lot of stuff across net. The new functions are fine in protocol.h right now since they're straight-forward, but after NETWORK_LIMITED will really want to move elsewhere after @theuni moves the nServices-based selection to addrman from connman.

  Adds HasAllRelevantServices and GetRelevantServices, which check
  for NETWORK|WITNESS.

  This changes the following:
   * Removes nRelevantServices from CConnman, disconnecting it a bit
     more from protocol-level logic.
   * Replaces our sometimes-connect-to-!WITNESS-nodes logic with
     simply always requiring WITNESS|NETWORK for outbound non-feeler
     connections (feelers still only require NETWORK).
   * This has the added benefit of removing nServicesExpected from
     CNode - instead letting net_processing's VERSION message
     handling simply check HasAllRelevantServices.
   * This implies we believe WITNESS nodes to continue to be a
     significant majority of nodes on the network, but also because
     we cannot sync properly from !WITNESS nodes, it is strange to
     continue using our valuable outbound slots on them.
   * In order to prevent this change from preventing connection to
     -connect= nodes which have !WITNESS, -connect nodes are now
     given the "addnode" flag. This also allows outbound connections
     to !NODE_NETWORK nodes for -connect nodes (which was already true
     of addnodes).
   * Has the (somewhat unintended) consequence of changing one of the
     eviction metrics from the same
     sometimes-connect-to-!WITNESS-nodes metric to requiring
     HasRelevantServices.

  This should make NODE_NETWORK_LIMITED much simpler to implement.

Tree-SHA512: 90606896c86cc5da14c77843b16674a6a012065e7b583d76d1c47a18215358abefcbab44ff4fab3fadcd39aa9a42d4740c6dc8874a58033bdfc8ad3fb5c649fc
2019-09-29 12:42:14 +02:00
Wladimir J. van der Laan
35d60de2b5 Merge #11458: Don't process unrequested, low-work blocks
01b52ce Add comment explaining forced processing of compact blocks (Suhas Daftuar)
08fd822 qa: add test for minchainwork use in acceptblock (Suhas Daftuar)
ce8cd7a Don't process unrequested, low-work blocks (Suhas Daftuar)

Pull request description:

  A peer could try to waste our resources by sending us unrequested blocks with
  low work (eg to fill up our disk).  Since e265200 we no longer request blocks until we
  know we're on a chain with more than nMinimumChainWork (our anti-DoS
  threshold), but we would still process unrequested blocks that had more work
  than our tip (which generally has low-work during IBD), even though we may not
  yet have found a headers chain with sufficient work.

  Fix this and add a test.

Tree-SHA512: 1a4fb0bbd78054b84683f995c8c3194dd44fa914dc351ae4379c7c1a6f83224f609f8b9c2d9dde28741426c6af008ffffea836d21aa31a5ebaa00f8e0f81229e
2019-09-29 12:42:14 +02:00
Wladimir J. van der Laan
9938dd83d4 Merge #10357: Allow setting nMinimumChainWork on command line
eac64bb7a [qa] Test nMinimumChainWork (Suhas Daftuar)
0311836f6 Allow setting nMinimumChainWork on command line (Suhas Daftuar)

Pull request description:

  As discussed briefly here: https://botbot.me/freenode/bitcoin-core-dev/2017-02-28/?msg=81712308&page=4

  This adds a hidden command line option for setting `nMinimumChainWork`, which allows us to test this parameter in our functional tests, as well as allowing for niche use cases like syncing nodes that are otherwise disconnected from the network.

  See also #10345, which proposes a new use of `nMinimumChainWork`.

Tree-SHA512: fe4d8f4f289697615c98d8760f1cc74c076110310ea0b5b875fcab78c127da9195b4eb84148aebacc7606c246e5773d3f13bd5d9559d0a8bffac20a3a28c62df
2019-09-29 12:42:14 +02:00
Wladimir J. van der Laan
f18fa576b9 Merge #11472: qa: Make tmpdir option an absolute path, misc cleanup
fafa003 qa: Remove never used return value of sync_with_ping (MarcoFalke)
fa9de37 qa: Make tmpdir option an absolute path (MarcoFalke)

Pull request description:

  This should fix issues with the multiwallet test and its symlinks
  when the tmpdir is a relative path.

  Rather than fixing os.symlink to work with paths relative to a
  directory descriptor, which does not work on Windows, normalize
  the path instead.

Tree-SHA512: 189690f3d065ea2f0f48e06775c86d513d0916c7c86312432e8e16df160e65539e288c2bd53d49a4180735fa940f6fcd52b506ccd7d9815651a9b1a69850dda6
2019-09-29 12:42:14 +02:00