Commit Graph

916 Commits

Author SHA1 Message Date
Wladimir J. van der Laan
909a13082f Merge #16631: net: The default whitelistrelay should be true
3b05f0f70fbaee5b5eaa0d1b6f3b9d32f44410bb Reformat p2p_permissions.py (nicolas.dorier)
ce7eac3cb0e7d301db75de24e9a7b0af93c61311 [Fix] The default whitelistrelay should be true (nicolas.dorier)

Pull request description:

  I thought `whitelistrelay` default was `false` when it is `true`.

  The root of the issue come from the fact that all references to `DEFAULT_` are not in the scope of this file, so hard coding of default values are used everywhere in `net.cpp`. I think that in a separate PR we should fix that more fundamentally everywhere.

ACKs for top commit:
  promag:
    ACK 3b05f0f70fbaee5b5eaa0d1b6f3b9d32f44410bb.
  Sjors:
    re-ACK 3b05f0f70fbaee5b5eaa0d1b6f3b9d32f44410bb

Tree-SHA512: f4a75f986fa2adf1a5f1c91605e0d261f7ac5ac8535fb05437d83b8392dbcf5cc1a47d755adcf8ad8dc67a88de28060187200fd3ce06545261a5c7ec0fea831a
2021-07-19 19:00:04 -05:00
MarcoFalke
ce8b04fc38 Merge #16618: [Fix] Allow connection of a noban banned peer
d117f4541d4717e83c9396273e92960723622030 Add test for setban (nicolas.dorier)
dc7529abf0197dccb876dc4a93cbdd2ad9f03e5c [Fix] Allow connection of a noban banned peer (nicolas.dorier)

Pull request description:

  Reported by @MarcoFalke on https://github.com/bitcoin/bitcoin/pull/16248#discussion_r314026195

  The bug would mean that if the peer connecting to you is banned, but whitelisted without specific permissions, it would not be able to connect to the node.

  The solution is just to move the same line below.

ACKs for top commit:
  Sjors:
    Agree inline is more clear. utACK d117f45
  MarcoFalke:
    ACK d117f4541d4717e83c9396273e92960723622030

Tree-SHA512: 0fed39acb1e8db67bb0bf4c4de3ad034ae776f38d55bd661f1ae0e1a4c6becaf1824ab46ed8279f2f31df3f4b29ff56461d8b167d3e9cece62cfe58b5a912811
2021-07-19 19:00:04 -05:00
Wladimir J. van der Laan
6c75d20277 Merge #16248: Make whitebind/whitelist permissions more flexible
c5b404e8f1973afe071a07c63ba1038eefe13f0f Add functional tests for flexible whitebind/list (nicolas.dorier)
d541fa391844f658bd7035659b5b16695733dd56 Replace the use of fWhitelisted by permission checks (nicolas.dorier)
ecd5cf7ea4c3644a30092100ffc399e30e193275 Do not disconnect peer for asking mempool if it has NO_BAN permission (nicolas.dorier)
e5b26deaaa6842f7dd7c4537ede000f965ea0189 Make whitebind/whitelist permissions more flexible (nicolas.dorier)

Pull request description:

  # Motivation

  In 0.19, bloom filter will be disabled by default. I tried to make [a PR](https://github.com/bitcoin/bitcoin/pull/16176) to enable bloom filter for whitelisted peers regardless of `-peerbloomfilters`.

  Bloom filter have non existent privacy and server can omit filter's matches. However, both problems are completely irrelevant when you connect to your own node. If you connect to your own node, bloom filters are the most bandwidth efficient way to synchronize your light client without the need of some middleware like Electrum.

  It is also a superior alternative to BIP157 as it does not require to maintain an additional index and it would work well on pruned nodes.

  When I attempted to allow bloom filters for whitelisted peer, my proposal has been NACKed in favor of [a more flexible approach](https://github.com/bitcoin/bitcoin/pull/16176#issuecomment-500762907) which should allow node operator to set fine grained permissions instead of a global `whitelisted` attribute.

  Doing so will also make follow up idea very easy to implement in a backward compatible way.

  # Implementation details

  The PR propose a new format for `--white{list,bind}`. I added a way to specify permissions granted to inbound connection matching `white{list,bind}`.

  The following permissions exists:
  * ForceRelay
  * Relay
  * NoBan
  * BloomFilter
  * Mempool

  Example:
  * `-whitelist=bloomfilter@127.0.0.1/32`.
  * `-whitebind=bloomfilter,relay,noban@127.0.0.1:10020`.

  If no permissions are specified, `NoBan | Mempool` is assumed. (making this PR backward compatible)

  When we receive an inbound connection, we calculate the effective permissions for this peer by fetching the permissions granted from `whitelist`  and add to it the permissions granted from `whitebind`.

  To keep backward compatibility, if no permissions are specified in `white{list,bind}` (e.g. `--whitelist=127.0.0.1`) then parameters `-whitelistforcerelay` and `-whiterelay` will add the permissions `ForceRelay` and `Relay` to the inbound node.

  `-whitelistforcerelay` and `-whiterelay` are ignored if the permissions flags are explicitly set in `white{bind,list}`.

  # Follow up idea

  Based on this PR, other changes become quite easy to code in a trivially review-able, backward compatible way:

  * Changing `connect` at rpc and config file level to understand the permissions flags.
  * Changing the permissions of a peer at RPC level.

ACKs for top commit:
  laanwj:
    re-ACK c5b404e8f1973afe071a07c63ba1038eefe13f0f

Tree-SHA512: adfefb373d09e68cae401247c8fc64034e305694cdef104bdcdacb9f1704277bd53b18f52a2427a5cffdbc77bda410d221aed252bc2ece698ffbb9cf1b830577
2021-07-19 19:00:04 -05:00
Jonas Schnelli
134c11e9f1 Merge #14929: net: Allow connections from misbehavior banned peers
0297be61a Allow connections from misbehavior banned peers. (Gregory Maxwell)

Pull request description:

  This allows incoming connections from peers which are only banned
   due to an automatic misbehavior ban if doing so won't fill inbound.

  These peers are preferred for eviction when inbound fills, but may
   still be kept if they fall into the protected classes.  This
   eviction preference lasts the entire life of the connection even
   if the ban expires.

  If they misbehave again they'll still get disconnected.

  The main purpose of banning on misbehavior is to prevent our
   connections from being wasted on unhelpful peers such as ones
   running incompatible consensus rules.  For inbound peers this
   can be better accomplished with eviction preferences.

  A secondary purpose was to reduce resource waste from repeated
   abuse but virtually any attacker can get a nearly unlimited
   supply of addresses, so disconnection is about the best we can
   do.

  This can reduce the potential from negative impact due to incorrect misbehaviour bans.

Tree-SHA512: 03bc8ec8bae365cc437daf70000c8f2edc512e37db821bc4e0fafa6cf56cc185e9ab40453aa02445f48d6a2e3e7268767ca2017655aca5383108416f1e2cf20f
2021-07-19 18:57:23 -05:00
UdjinM6
7ec01fd77a
Merge pull request #4278 from PastaPastaPasta/backports-0.18-pr9
Backports 0.18 pr9
2021-07-20 02:26:47 +03:00
Wladimir J. van der Laan
ff5a94748d
Merge #13249: Make objects in range declarations immutable by default. Avoid unnecessary copying of objects in range declarations.
f34c8c466a0e514edac2e8683127b4176ad5d321 Make objects in range declarations immutable by default. Avoid unnecessary copying of objects in range declarations. (practicalswift)

Pull request description:

  Make objects in range declarations immutable by default.

  Rationale:
  * Immutable objects are easier to reason about.
  * Prevents accidental or hard-to-notice change of value.

Tree-SHA512: cad69d35f0cf8a938b848e65dd537c621d96fe3369be306b65ef0cd1baf6cc0a9f28bc230e1e383d810c555a6743d08cb6b2b0bd51856d4611f537a12e5abb8b
2021-07-19 17:11:18 -05:00
UdjinM6
95f89eec56
Fix more spelling errors 2021-07-19 12:52:03 -05:00
Wladimir J. van der Laan
38ee2a7a94 Merge #14605: Return of the Banman
18185b57c32d0a43afeca4c125b9352c692923e9 scripted-diff: batch-recase BanMan variables (Carl Dong)
c2e04d37f3841d109c1fe60693f9622e2836cc29 banman: Add, use CBanEntry ctor that takes ban reason (Carl Dong)
1ffa4ce27d4ea6c1067d8984455df97994c7713e banman: reformulate nBanUtil calculation (Carl Dong)
daae598feb034f2f56e0b00ecfb4854d693d3641 banman: add thread annotations and mark members const where possible (Cory Fields)
84fc3fbd0304a7d6e660bf783c84bed2dd415141 scripted-diff: batch-rename BanMan members (Cory Fields)
af3503d903b1a608cd212e2d74b274103199078c net: move BanMan to its own files (Cory Fields)
d0469b2e9386a7a4b268cb9725347e7517acace6 banman: pass in default ban time as a parameter (Cory Fields)
2e56702ecedd83c4b7cb8de9de5c437c8c08e645 banman: pass the banfile path in (Cory Fields)
4c0d961eb0d7825a1e6f8389d7f5545114ee18c6 banman: create and split out banman (Cory Fields)
83c1ea2e5e66b8a83072e3d5ad6a4ced406eb1ba net: split up addresses/ban dumps in preparation for moving them (Cory Fields)
136bd7926c72659dd277a7b795ea17f72e523338 tests: remove member connman/peerLogic in TestingSetup (Cory Fields)
7cc2b9f6786f9bc33853220551eed33ca6b7b7b2 net: Break disconnecting out of Ban() (Cory Fields)

Pull request description:

  **Old English à la Beowulf**
  ```
  Banman wæs bréme    --blaéd wíde sprang--
  Connmanes eafera    Coreum in.
  aéglaéca            léodum forstandan
  Swá bealdode        bearn Connmanes
  guma gúðum cúð      gódum daédum·
  dréah æfter dóme·   nealles druncne slóg
  ```

  **Modern English Translation**
  ```
  Banman was famed              --his renown spread wide--
  Conman's hier,                in Core-land.
  against the evil creature     defend the people
  Thus he was bold,             the son of Connman
  man famed in war,             for good deeds;
  he led his life for glory,    never, having drunk, slew
  ```

  --

  With @theuni's blessing, here is Banman, rebased. Original PR: https://github.com/bitcoin/bitcoin/pull/11457

  --

  Followup PRs:
  1. Give `CNode` a `Disconnect` method ([source](https://github.com/bitcoin/bitcoin/pull/14605#discussion_r248065847))
  2. Add a comment to `std::atomic_bool fDisconnect` in `net.h` that setting this to true will cause the node to be disconnected the next time `DisconnectNodes()` runs ([source](https://github.com/bitcoin/bitcoin/pull/14605#discussion_r248384309))

Tree-SHA512: 9c207edbf577415c22c9811113e393322d936a843d4ff265186728152a67c057779ac4d4f27b895de9729f7a53e870f828b9ebc8bcdab757520c2aebe1e9be35
2021-07-17 22:32:12 -05:00
Wladimir J. van der Laan
b8bc9e9643 Merge #15138: Drop IsLimited in favor of IsReachable
d6b076c17bc7d513243711563b262524ef0ba74c Drop IsLimited in favor of IsReachable (Ben Woosley)

Pull request description:

  These two methods have had the same meaning, but inverted, since
  110b62f069. Having one name for a single
  concept simplifies the code.

  This is a follow-up to #15051.
  /cc #7553

Tree-SHA512: 347ceb9e2a55ea06f4c01226411c7bbcade09dd82130e4c59d0824ecefd960875938022edbe5d4bfdf12b0552c9b4cb78b09a688284d707119571daf4eb371b4
2021-07-17 22:32:12 -05:00
UdjinM6
cc601e80c5
Fix CConnman related locks 2021-07-13 19:43:32 +03:00
UdjinM6
bc09637998
Merge pull request #4225 from PastaPastaPasta/backport-triv-pr5
backport: 'trivial' pr5
2021-07-04 00:07:27 +03:00
MarcoFalke
570d573786
Merge #13774: Return void instead of bool for functions that cannot fail
d78a8dc3e8 Return void instead of bool for functions that cannot fail (practicalswift)

Pull request description:

  Return `void` instead of `bool` for functions that cannot fail:
  * `CBlockTreeDB::ReadReindexing(...)`
  * `CChainState::ResetBlockFailureFlags(...)`
  * `CTxMemPool::addUnchecked(...)`
  * `CWallet::CommitTransaction(...)`
  * `CWallet::LoadDestData(...)`
  * `CWallet::LoadKeyMetadata(...)`
  * `CWallet::LoadScriptMetadata(...)`
  * `CWallet::LoadToWallet(...)`
  * `CWallet::SetHDChain(...)`
  * `CWallet::SetHDSeed(...)`
  * `PendingWalletTx::commit(...)`
  * `RemoveLocal(...)`
  * `SetMinVersion(...)`
  * `StartHTTPServer(...)`
  * `StartRPC(...)`
  * `TorControlConnection::Disconnect(...)`

  Some of the functions can fail by throwing.

  Found by manually inspecting the following candidate functions:

  ```
  $ git grep -E '(^((static|virtual|inline|friend)[^a-z])*[^a-z]*bool [^=]*\(|return true|return false)' -- "*.cpp" "*.h"
  ```

Tree-SHA512: c0014e045362dbcd1a0cc8f69844e7b8cbae4f538e7632028daeca3a797ac11d8d3d86ebc480bedcb8626df3e96779d592747d52a12556fc49921b114fa0ccc6
2021-07-02 12:59:28 +03:00
Wladimir J. van der Laan
deab1b0397 Merge #14127: build: avoid getifaddrs when unavailable
9256f7d13f5b68ebc2981e8f45777f4bdc43f1b3 build: avoid getifaddrs when unavailable (Cory Fields)

Pull request description:

  These changes from @theuni help building when targeting platforms that don't always have getifaddrs available like Android < 24

Tree-SHA512: dbfeb83297bd6f00b7991f53eef8a04948d2d739bf47c0524d9ae5335b843b8a5c06ff98c109fe5e6192665e6d0cf58700b8aa7e2a0b410281d3c052881973ff
2021-07-01 16:54:18 -05:00
Wladimir J. van der Laan
8223053a49
Merge #15051: Tests: IsReachable is the inverse of IsLimited (DRY). Includes unit tests
6dc4593db1ccfb8745b2daa42f457981ae08dba9 IsReachable is the inverse of IsLimited (DRY). Includes unit tests (marcaiaf)

Pull request description:

  IsReachable is the inverse of IsLimited, but the implementation is duplicated (DRY)

  - Changed the implementation accordingly.
  - Added unit tests to document behavior and relationship
  - My modification in net.cpp  applies only to IsReachable.
  - Applied clang-format-diffpy

  Created new pull request to avoid the mess with:
  https://github.com/bitcoin/bitcoin/pull/15044

  Checked with supposedly conflicting PRs mentioned in the old PR. No conflicts with the specific changes in this PR.

Tree-SHA512: b132dec6cc2c788ebe4f63f228d78f441614e156743b17adebc990de0180a5872874d2724c86eeaa470b4521918bd137b0e33ebcaae77c5efc1f0d56104f6c87
2021-06-28 13:40:53 -05:00
Kittywhiskers Van Gogh
ef4dfa8524 merge bitcoin#14555: Move util files to directory
(script modified to account for Dash backports, doesn't account for rebasing)

------------- BEGIN SCRIPT ---------------
mkdir -p src/util
git mv src/util.h src/util/system.h
git mv src/util.cpp src/util/system.cpp
git mv src/utilmemory.h src/util/memory.h
git mv src/utilmoneystr.h src/util/moneystr.h
git mv src/utilmoneystr.cpp src/util/moneystr.cpp
git mv src/utilstrencodings.h src/util/strencodings.h
git mv src/utilstrencodings.cpp src/util/strencodings.cpp
git mv src/utiltime.h src/util/time.h
git mv src/utiltime.cpp src/util/time.cpp
git mv src/utilasmap.h src/util/asmap.h
git mv src/utilasmap.cpp src/util/asmap.cpp
git mv src/utilstring.h src/util/string.h
git mv src/utilstring.cpp src/util/string.cpp

gsed -i 's/<util\.h>/<util\/system\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmemory\.h>/<util\/memory\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmoneystr\.h>/<util\/moneystr\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstrencodings\.h>/<util\/strencodings\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utiltime\.h>/<util\/time\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilasmap\.h>/<util\/asmap\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstring\.h>/<util\/string\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')

gsed -i 's/BITCOIN_UTIL_H/BITCOIN_UTIL_SYSTEM_H/g' src/util/system.h
gsed -i 's/BITCOIN_UTILMEMORY_H/BITCOIN_UTIL_MEMORY_H/g' src/util/memory.h
gsed -i 's/BITCOIN_UTILMONEYSTR_H/BITCOIN_UTIL_MONEYSTR_H/g' src/util/moneystr.h
gsed -i 's/BITCOIN_UTILSTRENCODINGS_H/BITCOIN_UTIL_STRENCODINGS_H/g' src/util/strencodings.h
gsed -i 's/BITCOIN_UTILTIME_H/BITCOIN_UTIL_TIME_H/g' src/util/time.h
gsed -i 's/BITCOIN_UTILASMAP_H/BITCOIN_UTIL_ASMAP_H/g' src/util/asmap.h
gsed -i 's/BITCOIN_UTILSTRING_H/BITCOIN_UTIL_STRING_H/g' src/util/string.h

gsed -i 's/ util\.\(h\|cpp\)/ util\/system\.\1/g' src/Makefile.am
gsed -i 's/utilmemory\.\(h\|cpp\)/util\/memory\.\1/g' src/Makefile.am
gsed -i 's/utilmoneystr\.\(h\|cpp\)/util\/moneystr\.\1/g' src/Makefile.am
gsed -i 's/utilstrencodings\.\(h\|cpp\)/util\/strencodings\.\1/g' src/Makefile.am
gsed -i 's/utiltime\.\(h\|cpp\)/util\/time\.\1/g' src/Makefile.am
gsed -i 's/utilasmap\.\(h\|cpp\)/util\/asmap\.\1/g' src/Makefile.am
gsed -i 's/utilstring\.\(h\|cpp\)/util\/string\.\1/g' src/Makefile.am

gsed -i 's/-> util ->/-> util\/system ->/' test/lint/lint-circular-dependencies.sh
gsed -i 's/src\/util\.cpp/src\/util\/system\.cpp/g' test/lint/lint-format-strings.py test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilmoneystr\.cpp/src\/util\/moneystr\.cpp/g' test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilstrencodings\.\(h\|cpp\)/src\/util\/strencodings\.\1/g' test/lint/lint-locale-dependence.sh
------------- END   SCRIPT ---------------
2021-06-27 12:03:13 +05:30
Wladimir J. van der Laan
6e054f897b
Merge #15993: net: Drop support of the insecure miniUPnPc versions
59cb722fd050393a69f1e0df97d857c893d19d80 Update configure to reject unsafe miniUPnPc API ver (Hennadii Stepanov)
ab2190557ec2757fa48b52855b05561854af49af doc: Add release notes for 15993 (Hennadii Stepanov)
02709e95601c6020a87a6a05ee1d00c13fc38f9b Align formatting with clang-format (Hennadii Stepanov)
91a1b8508358d04685391651aea303ebce1c3d05 Use PACKAGE_NAME in UPnP description (Hennadii Stepanov)
9f76e45b9d6671e2074fb7a3885db703045a791f Drop support of insecure miniUPnPc versions (Hennadii Stepanov)

Pull request description:

  1. Minimum supported miniUPnPc API version is set to 10:
  - https://packages.ubuntu.com/xenial/libminiupnpc-dev
  - https://packages.debian.org/jessie/libminiupnpc-dev

  Refs:
  - #6583
  - #6789
  - #10414

  2. The hardcoded "Bitcoin" replaced with `PACKAGE_NAME`:
  ![Screenshot from 2019-05-06 23-10-29](https://user-images.githubusercontent.com/32963518/57253178-afc60780-7056-11e9-83c9-e85670c58c1e.png)

  3. Also style-only commit applied.

  Pardon: could not reopen my previous PR #15966.

ACKs for top commit:
  ryanofsky:
    utACK 59cb722fd050393a69f1e0df97d857c893d19d80. Changes since last review: adding a new commit which updates configure script to fall back to disabling upnp if version is too old, adding a requested comment explaining static_assert condition, and fixing a spelling (jessy/jessie)

Tree-SHA512: 42ed11bc2fb2ec83d5dd58e2383da5444a24fd572707f6cf10b622cb8943e28adfcca4750d06801024c4472625b5ea9279516fbd9d2ccebc9bbaafe1d148e80d
2021-06-26 11:23:44 -05:00
MarcoFalke
3a75bef4af
Merge #15654: net: Remove unused unsanitized user agent string CNode::strSubVer
fa8548c5d1 net: Remove unused unsanitized user agent string CNode::strSubVer (MarcoFalke)

Pull request description:

  I fail to see a use case for this unsanitized byte array. In fact this can easily be confused with `cleanSubVer` and be displayed to the user (or logged) by a simple typo that is hard to find in review.

  Further reading: https://btcinformation.org/en/developer-reference#version

ACKs for commit fa8548:
  promag:
    utACK fa8548c, good catch.
  practicalswift:
    utACK fa8548c5d13957f57f9b1e20e03002600962f7f0
  sipa:
    utACK fa8548c5d13957f57f9b1e20e03002600962f7f0

Tree-SHA512: 3c3ff1504d1583ad099df9a6aa761458a82ec48a58ef7aaa9b5679a5281dd1b59036ba2932ed708488951a565b669a3083ef70be5a58472ff8677b971162ae2f
2021-06-26 11:23:43 -05:00
fanquake
24b24af72b
Merge #15558: Don't query all DNS seeds at once
6170ec5 Do not query all DNS seed at once (Pieter Wuille)

Pull request description:

  Before this PR, when we don't have enough connections after 11 seconds, we proceed to query all DNS seeds in a fixed order, loading responses from all of them.

  Change this to to only query three randomly-selected DNS seed. If 11 seconds later we still don't have enough connections, try again with another one, and so on.

  This reduces the amount of information DNS seeds can observe about the requesters by spreading the load over all of them.

ACKs for top commit:
  Sjors:
    ACK 6170ec5
  sdaftuar:
    ACK 6170ec5
  jonasschnelli:
    utACK 6170ec5 - I think the risk of a single seeder codebase is orthogonal to this PR. Such risks could also be interpreted differently (diversity could also increase the risk based on the threat model).
  fanquake:
    ACK 6170ec5 - Agree with the reasoning behind the change. Did some testing with and without `-forcednsseed` and/or a `peers.dat` and monitored the DNS activity.

Tree-SHA512: 33f6be5f924a85d312303ce272aa8f8d5e04cb616b4b492be98832e3ff37558d13d2b16ede68644ad399aff2bf5ff0ad33844e55eb40b7f8e3fddf9ae43add57
2021-06-26 11:23:42 -05:00
Wladimir J. van der Laan
fff8f97917
Merge #15486: [addrman, net] Ensure tried collisions resolve, and allow feeler connections to existing outbound netgroups
20e6ea259b222b10f066f22695a5f56c52071f63 [addrman] Improve collision logging and address nits (Suhas Daftuar)
f71fdda3bc2e7acd2a8b74e882364866b8b0f55b [addrman] Ensure collisions eventually get resolved (Suhas Daftuar)
4991e3c813c9848d3b3957ea3ad433f02fca9e81 [net] feeler connections can be made to outbound peers in same netgroup (Suhas Daftuar)
4d834018e368c3481a5421891395f64aa9002185 [addrman] Improve tried table collision logging (Suhas Daftuar)

Pull request description:

  The restriction on outbound peers sharing the same network group is not intended to apply to feeler connections, so fix this.

  This fixes an issue where a tried table collision with an entry to a netgroup we already have an outbound connection to could cause feelers to stop working, because the tried collision buffer (`m_tried_collisions`) would never be drained.

  Also, ensure that all entries don't linger in `m_tried_collisions` by evicting an old entry if its collisions is unresolved after 40 minutes.

Tree-SHA512: 553fe2b01b82cd7f0f62f90c6781e373455a45b254e3bec085b5e6b16690aa9f3938e8c50e7136f19dafa250ed4578a26227d944b76daf9ce4ef0c75802389b6
2021-06-26 11:23:42 -05:00
Wladimir J. van der Laan
ae6ce01be6
Merge #15345: net: Correct comparison of addr count
107623c26c2113428446effaa24edb986a3e780c net: Correct comparison of addr count (Carl Dong)

Pull request description:

  `LOCAL_NONE` is supposed to be an enum indicating the `nScore` of a
  `LocalServiceInfo` rather than the count of an addr in `mapLocalHost`.

Tree-SHA512: a47a0859dd11c991d75b54e96b08c502e3d235f7a6522a2355053f377d05e7853483996919292f458d917a561b23951e6945d6bf0ff5a2f29513c477c640bdd2
2021-06-25 18:43:54 -05:00
Kittywhiskers Van Gogh
d559983eeb merge #16127: more thread safety annotation coverage 2021-06-09 17:36:31 +05:30
Kittywhiskers Van Gogh
c5c3dee308 merge #11640: Make LOCK, LOCK2, TRY_LOCK work with CWaitableCriticalSection 2021-06-06 16:22:17 +05:30
MarcoFalke
7475782bc0
Merge #13163: Make it clear which functions that are intended to be translation unit local
c3f34d06be Make it clear which functions that are intended to be translation unit local (practicalswift)

Pull request description:

  Make it clear which functions that are intended to be translation unit local.

  Do not share functions that are meant to be translation unit local with other translation units. Use internal linkage for those consistently.

Tree-SHA512: 05eebd233d5cfbf6116724eec3a99b465bf534ca220f2b6f5e56341a7da41387454d3cb6ceadd8ab6714a5df94069e4ad0dcab8801ccc7e8949be7199a19fb53
2021-05-25 14:09:36 +03:00
PastaPastaPasta
b76e7fec1f
Merge pull request #4164 from kittywhiskers/auxprs
Merge #19660, #19373, #19841, #13862, #13866, #17280, #17682 and partial #19326, #14978: Auxiliary Backports
2021-05-21 14:25:27 -05:00
Kittywhiskers Van Gogh
56f1b2d01c Partial #19326: Simplify hash.h interface using Spans 2021-05-20 10:15:17 +05:30
Kittywhiskers Van Gogh
f3819c4eef Merge #18023: Fix some asmap issues 2021-05-20 00:05:12 +05:30
Kittywhiskers Van Gogh
4b2b5f78d4 Merge #16702: supplying and using asmap to improve IP bucketing 2021-05-20 00:05:09 +05:30
PastaPastaPasta
c5b919d084
Update copyright (#4115)
* run: `python3 contrib/devtools/copyright_header.py update .`

* bump copyright year
2021-04-20 22:33:02 +03:00
UdjinM6
0b7d7a084a
net: Use Intra-Quorum Relay connections for other messages too (#4021)
* net: Use Intra-Quorum Relay connections for other messages too

Make intra-quorum data delivery more robust.

* Apply review suggestions
2021-04-03 20:24:03 +03:00
UdjinM6
bca9577b8f
partial/logic backport #18544: net: limit BIP37 filter lifespan (active between 'filterload'..'filterclear') (#4043)
* partial backport 18544: net: limit BIP37 filter lifespan (active between 'filterload'..'filterclear')

Previously, a default match-everything bloom filter was set for every peer,
i.e. even before receiving a 'filterload' message and after receiving a
'filterclear' message code branches checking for the existence of the filter
by testing the pointer "pfilter" were _always_ executed.

* net: Match the backport PR a bit more

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
2021-03-22 13:29:31 +03:00
dustinface
ae506bae66
refactor: PrivateSend -> CoinJoin + Move the tab (#4038)
* qt|wallet|privatesend: Rename PrivateSend to CoinJoin in GUI strings

* qt: Move CoinJoin next to Transactions

* qt: Adjust status tip of privateSendCoinsMenuAction

Co-authored-by: thephez <thephez@users.noreply.github.com>

* rename: privateSend -> coinJoin

* rename: privatesend -> coinjoin

* rename: PrivateSend -> CoinJoin

* rename: use_ps -> use_cj

* rename: PRIVATESEND -> COINJOIN

* rename: privatesend -> coinjoin for files and folders

* refactor: Re-order coinjoin files in cmake/make files

* refactor: Re-order coinjoin includes where it makes sense

* test: Update lint-circular-dependencies.sh

* Few cleanups

* test: test/coinjoin_tests.cpp -> wallet/test/coinjoin_test.cpp

* s/AdvancedPSUI/AdvancedCJUI/g

* s/privateSentAmountChanged/coinJoinAmountChanged/g

* wallet: Rename "ps_salt" backwards compatible

* Minimal PrivateSend -> CoinJoin migration for settings and cmd-line

* wallet: Fix privatesendrounds -> coinjoinrounds migration

* qt: Migrate nPrivateSendAmount -> nCoinJoinAmount

* `-coinjoindenoms` never existed

* Migrate all PS options/settings

* rpc: Formatting only

* qt: Make Send/CoinJoin tabs a bit more distinguishable

Co-authored-by: thephez <thephez@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-03-17 18:36:11 -04:00
UdjinM6
95e8492f31
llmq: Implement Intra-Quorum Relay connections for qrecsig-s (#4020)
* llmq: Implement Intra-Quorum Relay connections for qrecsig-s

Avoid relaying recsigs to non-masternodes.

* test: Simplify loop

* test: Assert connections are equal to llmq_size instead of static number

* test: Drop redundant brackets

* test: Fix typo

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
2021-03-14 22:49:38 -04:00
dustinface
0c1a02be2c
net|rpc: getpeerinfo - Add verified_pubkey_hash, fix a typo in verified_proregtx_hash docs (#3929)
* net|rpc: Add verified_pubkey_hash in getpeerinfo for MN connections

* Apply suggestions from code review

Co-authored-by: thephez <thephez@users.noreply.github.com>

* rpc: One more typo

Co-authored-by: thephez <thephez@users.noreply.github.com>
2021-01-22 17:12:22 -05:00
UdjinM6
1f1f2d8078
refactor: Rename fMasternode and fMasternodeProbe (#3928)
* fMasternode -> m_masternode_connection

* fMasternodeProbe -> m_probe_connection

* probe_connection -> masternode_probe_connection
2021-01-14 13:59:18 -06:00
UdjinM6
a06eba3eb9
Implement kqueue support (#3892) 2020-12-30 13:34:42 -06:00
UdjinM6
fade3c440a
Fix WakeSelect() (#3895)
Fixes `Syscall param write(buf) points to uninitialised byte(s)`
2020-12-26 19:02:13 -06:00
Wladimir J. van der Laan
a09555c389
Merge #12678: build: Fix a few compilation issues with Clang 7 and -Werror
8ae413235 Remove redundant checks for MSG_* from configure.ac (Vasil Dimov)
71129e026 Do not check for main() in libminiupnpc (Vasil Dimov)
8c632f73c ax_boost_{chrono,unit_test_framework}.m4: take changes from upstream (Vasil Dimov)

Pull request description:

Tree-SHA512: a99ef98c0b94f892eadeda24b3d55c25bedf225b98c6e4178cf6c2d886b44d43e9f75414d0b37db9ac261cec2350666e5e64fab9c104249dd34ff485c51663cb
2020-12-15 17:00:05 -06:00
UdjinM6
b559a8f904
Backporting Statoshi and bitcoin#16728 (#2515)
* Backport Statoshi

This backports some of https://github.com/jlopp/statoshi.

Missing stuff: README.md and client name changes, segwit and fee estimation stats.

Fix RejectCodeToString

Fix copy-paste mistake s/InvalidBlockFound/InvalidChainFound/

* Merge #16728: move-only: move coins statistics utils out of RPC

8a3b2eb17572ca2131778d52cc25ec359470a90f move-only: move coins statistics utils out of RPC (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  In the short-term, this move-only commit will help with fuzzing (https://github.com/bitcoin/bitcoin/pull/15606#issuecomment-524482297). Later, these procedures will be used to compute statistics (particularly a content hash) for UTXO sets coming in from snapshots.

  Most easily reviewed with `git ... --color-moved=dimmed_zebra`. A nice follow-up would be adding unittests, which I'll do if nobody else gets around to it.

ACKs for top commit:
  MarcoFalke:
    ACK 8a3b2eb17572ca2131778d52cc25ec359470a90f, checked --color-moved=dimmed-zebra

Tree-SHA512: a187d2f7590ad2450b8e8fa3d038c80a04fc3d903618c24222d7e3172250ce51badea35860c86101f2ba266eb4354e6efb8d7d508b353f29276e4665a1efdf74

* Fix 16728

* Modernize StatsdClient

- Reuse some functionality from netbase
- Switch from GetRand to FastRandomContext
- Drop `using namespace std` and add `// namespace statsd`

* Introduce PeriodicStats and make StatsdClient configurable via -stats<smth> (enabled/host/port/ns/period)

* Move/rename tip stats from CheckBlock to ConnectBlock

* Add new false positives to lint-format-strings.py

* Add snprintf in statsd_client to the list of known violations in lint-locale-dependence.sh

* Fix incorrect include guard

* Use bracket syntax includes

* Replace magic numbers with defaults

* Move connection stats calculation into its own function

And bail out early if stats are disabled

* assert in PeriodicStats

Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
2020-12-15 10:22:23 -06:00
Wladimir J. van der Laan
0c6ba028c1
Merge #12569: net: Increase signal-to-noise ratio in debug.log by adjusting log level when logging failed non-manual connect():s
cba2800 Increase signal-to-noise ratio in debug.log by adjusting log level when logging failed non-manual connect():s (practicalswift)

Pull request description:

  Increase signal-to-noise ratio in `debug.log` by adjusting log level when logging failed non-manual `connect()`:s.

  Before this patch:

  ```
  $ src/bitcoind -printtoconsole
  …
  2018-02-28 18:42:51 UpdateTip: new best=0000000000005448b10a219683d34b770a28044e1cc421032dea1a79ff548948 height=1286903 version=0x20000000 log2_work=69.791313 tx=17408546 date='2018-02-28 18:42:46' progress=1.000000 cache=0.0MiB(173txo)
  2018-02-28 18:37:52 connect() 10.11.21.34:18333 failed after select(): Connection refused (111)
  2018-02-28 18:43:22 connect() to 10.11.43.14:18333 failed after select(): Network is unreachable (101)
  2018-02-28 18:44:49 UpdateTip: new best=000000000000029a521ff2803e1441b09413b876accff5084a4cccf7747d798b height=1286904 version=0x20000000 log2_work=69.791345 tx=17408559 date='2018-02-28 18:44:51' progress=1.000000 cache=0.1MiB(502txo)
  2018-02-28 18:46:54 connect() to [2001:0:9d38:78ff:1234🔢1234:1234]:18333 failed: Network is unreachable (101)
  2018-02-28 18:48:56 connect() to [2001:0:9d38:6aff:1234🔢1234:1234]:18333 failed: Network is unreachable (101)
  2018-02-28 18:49:11 UpdateTip: new best=000000000000000206b79eb235e5dd907b6369de0e5d764330bf40ec0d460311 height=1286905 version=0x20000000 log2_work=69.791377 tx=17408577 date='2018-02-28 18:49:12' progress=1.000000 cache=1.0MiB(5245txo)
  ```

  After this patch:

  ```
  $ src/bitcoind -printtoconsole
  …
  2018-02-28 18:42:51 UpdateTip: new best=0000000000005448b10a219683d34b770a28044e1cc421032dea1a79ff548948 height=1286903 version=0x20000000 log2_work=69.791313 tx=17408546 date='2018-02-28 18:42:46' progress=1.000000 cache=0.0MiB(173txo)
  2018-02-28 18:44:49 UpdateTip: new best=000000000000029a521ff2803e1441b09413b876accff5084a4cccf7747d798b height=1286904 version=0x20000000 log2_work=69.791345 tx=17408559 date='2018-02-28 18:44:51' progress=1.000000 cache=0.1MiB(502txo)
  2018-02-28 18:49:11 UpdateTip: new best=000000000000000206b79eb235e5dd907b6369de0e5d764330bf40ec0d460311 height=1286905 version=0x20000000 log2_work=69.791377 tx=17408577 date='2018-02-28 18:49:12' progress=1.000000 cache=1.0MiB(5245txo)
  ```

  Please note that "manual `connect()`:s" (invoked via `-connect`, `-proxy` or `addnode`) are still reported at the default log level as these messages are likely to be relevant to end-users:

  ```
  $ src/bitcoind -printtoconsole -connect=127.0.0.1:1234
  …
  2018-02-28 18:31:13 connect() to 127.0.0.1:1234 failed after select(): Connection refused (111)

  $ src/bitcoind -printtoconsole -proxy=127.0.0.1:1234
  …
  2018-02-28 18:32:32 connect() to 127.0.0.1:1234 failed after select(): Connection refused (111)

  $ src/bitcoind -printtoconsole &
  $ src/bitcoin-cli addnode 127.0.0.1:1234 onetry
  …
  2018-02-28 18:33:40 connect() to 127.0.0.1:1234 failed after select(): Connection refused (111)
  ```

Tree-SHA512: 92e3c1e4b54ce8ccdd7ec31de147c8505710cd799ceb2bbc8576a086709967802403c9184df364b3cfa59bd98859f6ac8feb27fb09b9324194c6c47a042fc6d3

fix compilation

Signed-off-by: pasta <pasta@dashboost.org>
2020-11-17 14:32:36 -05:00
dustinface
5176a26007
masternode|net|rpc: Improve masternode sync process (#3690)
* masternode: Replace sync states INITIAL and WAITING with BLOCKCHAIN

* masternode: Peer dependent "assume tip" timeout

I would say its enough to only wait 1 tick if we have more than 3
peers before we move over to governance sync.

* masternode: Notify the UI instantly if switched to governance sync

Without this it takes one iteration more for the UI to receive the
update.

* masternode: Notify the UI about CMasternodeSync::Reset calls

* masternode: Don't instantly reset the sync process

Give it MASTERNODE_SYNC_RESET_SECONDS (600) seconds time after the last
UpdateBlockTip call.

* rpc: Don't switch to next asset in "mnsync reset"

* rpc: Force the reset in "mnsync reset"

* net: Make sure the sync gets a reset if required after network changes

This will reset the sync process if its outdated in the following cases:

- If the connections dropped to zero
- If the connections went from zero to one
- If the network has been enabled or disabled

* Apply suggestions from code review

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* net: Only open masternode connections if the blockchain is synced

In general it doesn't make sense to connect to masternodes before due to 
MNAUTH requires blockchain sync. This could lead to failing quorum 
connections/failing masternode 
probing.. if a just restarted node/a out of sync node 
would hit a dkg block.. Then they would not try to open those 
llmq/probing connections for the next 60s (nLLMQConnectionRetryTimeout). 
Thats basically what happens in tests right now and they fail without 
this commit.

* test: Make sure nodes are synced when they get restored after isolation

Their sync might be out of date otherwise due to bigger mocktime bumps

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-09-11 15:07:34 +03:00
PastaPastaPasta
7725e6fa4b
p2p: remove some old protocol version checks/dead code (#3647)
Signed-off-by: pasta <pasta@dashboost.org>
2020-08-14 14:42:15 +03:00
Wladimir J. van der Laan
2af1c8d6e7 Merge #13153: Add missing newlines to debug logging
4b75dcf devtools: Make linter check LogPrint calls (MarcoFalke)
ff2ad2d Add missing newlines to LogPrint debug logging (Wladimir J. van der Laan)

Pull request description:

  ~~Don't we have a linter that should catch these?~~

Tree-SHA512: 1a58eca01ded9c1719e943c09447deeb59bb06dba00528cf460eefe857fdf95b42671fbdebc87cdd2f51e931e86942d06587ffd097cbb0d8dd9eb7a0ba17a8f0
2020-07-29 10:55:33 -05:00
Wladimir J. van der Laan
86c064bed3
Merge #13824: doc: Remove outdated net comment
fa365021bbb4c0865e6655100c1cbf85e2b3c7fa doc: Remove outdated net comment (MarcoFalke)

Pull request description:

  `mapAddresses` and the corresponding "critsect" has been removed in 5fee401fe1 more than 6 years ago. Now is probably a good time to remove this confusing comment.

Tree-SHA512: 498a403d5703da395c18a7ebb776aa6e693e59fe43a839fefd261e0a5af58621763813979d4cfbd8d1728ce73b325b82002e393cde79bdbff33e0fbf68ab6747
2020-07-17 15:45:43 -05:00
Wladimir J. van der Laan
21d0711316
Merge #13615: net: Remove unused interrupt from SendMessages
4b6ab02122d225818f6c6a4b4ef343d2c90e6d55 Remove unused argument to ProcessGetBlockData(...) (practicalswift)
c469ecf22e8ab55acf2cf8fc6fcf34415b541f76 net: Remove unused interrupt from SendMessages (fanquake)

Pull request description:

  Discussed very briefly with cfields.

  Includes 65b4400 from #13554 as it's a similar refactor.

Tree-SHA512: 45cd64208a5c8164242db74e6687e9344ea592bab5e7f9ba8e1bb449057fc908ec9d8b8523748a68426e4a4304e3388a138cd834698b39837b2149b72beefdc9
2020-07-08 20:03:37 -05:00
MarcoFalke
005d241fea Merge #11878: Add Travis check for duplicate includes
c36b720d00 Add Travis check for duplicate includes (practicalswift)
280023f31d Remove duplicate includes (practicalswift)

Pull request description:

  This enforces parts of the project header include guidelines (added by @sipa in #10575).

  Example run:

  ```
  $ git diff
  diff --git a/src/warnings.cpp b/src/warnings.cpp
  index c52a1fd..d8994dd 100644
  --- a/src/warnings.cpp
  +++ b/src/warnings.cpp
  @@ -5,6 +5,8 @@

   #include <sync.h>
   #include <clientversion.h>
  +#include <string>
   #include <util.h>
   #include <warnings.h>
  +#include <util.h>

  diff --git a/src/warnings.h b/src/warnings.h
  index e8e982c..8d2252e 100644
  --- a/src/warnings.h
  +++ b/src/warnings.h
  @@ -7,6 +7,7 @@
   #define BITCOIN_WARNINGS_H

   #include <stdlib.h>
   #include <string>
  +#include <stdlib.h>

   void SetMiscWarning(const std::string& strWarning);
  $ contrib/devtools/lint-includes.sh
  Duplicate include(s) in src/warnings.h:
  #include <stdlib.h>

  Include(s) from src/warnings.h duplicated in src/warnings.cpp:
  #include <string>

  Duplicate include(s) in src/warnings.cpp:
  #include <util.h>
  $ echo $?
  1
  $ git checkout .
  $ contrib/devtools/lint-includes.sh
  $ echo $?
  0
  ```

Tree-SHA512: f653d23c58ebc024dfc5b1fb8570698fd3c515c75b60b5cabbc43595548c488fca92349fa4c8b64460edbe61c879ff1d24f37f959e18552e202a7342460ddbf1
2020-07-01 20:12:03 -05:00
Alexander Block
c1f889c564
Dont skip sendmessages (#3534)
* Revert "Only call SendMessages when ProcessMessages did some work or when some time passed"

This reverts commit 24ead62905.

* Only skip SendMessages() for fMasternode connections
2020-06-18 21:51:24 +03:00
Wladimir J. van der Laan
8b9012d292 Merge #9037: net: Add test-before-evict discipline to addrman
e68172ed9 Add test-before-evict discipline to addrman (Ethan Heilman)

Pull request description:

  This change implement countermeasures 3 (test-before-evict) suggested in our paper: ["Eclipse Attacks on Bitcoin’s Peer-to-Peer Network"](http://cs-people.bu.edu/heilman/eclipse/).
  # Design:

  A collision occurs when an address, addr1, is being moved to the tried table from the new table, but maps to a position in the tried table which already contains an address (addr2). The current behavior is that addr1 would evict addr2 from the tried table.

  This change ensures that during a collision, addr1 is not inserted into tried but instead inserted into a buffer (setTriedCollisions). The to-be-evicted address, addr2, is then tested by [a feeler connection](https://github.com/bitcoin/bitcoin/pull/8282). If addr2 is found to be online, we remove addr1 from the buffer and addr2 is not evicted, on the other hand if addr2 is found be offline it is replaced by addr1.

  An additional small advantage of this change is that, as no more than ten addresses can be in the test buffer at once, and addresses are only cleared one at a time from the test buffer (at 2 minute intervals), thus an attacker is forced to wait at least two minutes to insert a new address into tried after filling up the test buffer. This rate limits an attacker attempting to launch an eclipse attack.
  # Risk mitigation:
  - To prevent this functionality from being used as a DoS vector, we limit the number of addresses which are to be tested to ten. If we have more than ten addresses to test, we drop new addresses being added to tried if they would evict an address. Since the feeler thread only creates one new connection every 2 minutes the additional network overhead is limited.
  - An address in tried gains immunity from tests for 4 hours after it has been tested or successfully connected to.
  # Tests:

  This change includes additional addrman unittests which test this behavior.

  I ran an instance of this change with a much smaller tried table (2 buckets of 64 addresses) so that collisions were much more likely and observed evictions.

  ```
  2016-10-27 07:20:26 Swapping 208.12.64.252:8333 for 68.62.95.247:8333 in tried table
  2016-10-27 07:20:26 Moving 208.12.64.252:8333 to tried
  ```

  I documented tests we ran against similar earlier versions of this change in #6355.
  # Security Benefit

  This is was originally posted in PR #8282 see [this comment for full details](https://github.com/bitcoin/bitcoin/pull/8282#issuecomment-237255215).

  To determine the security benefit of these larger numbers of IPs in the tried table I modeled the attack presented in [Eclipse Attacks on Bitcoin’s Peer-to-Peer Network](https://eprint.iacr.org/2015/263).

  ![attackergraph40000-10-1000short-line](https://cloud.githubusercontent.com/assets/274814/17366828/372af458-595b-11e6-81e5-2c9f97282305.png)

  **Default node:** 595 attacker IPs for ~50% attack success.
  **Default node + test-before-evict:** 620 attacker IPs for ~50% attack success.
  **Feeler node:** 5540 attacker IPs for ~50% attack success.
  **Feeler node + test-before-evict:** 8600 attacker IPs for ~50% attack success.

  The node running feeler connections has 10 times as many online IP addresses in its tried table making an attack 10 times harder (i.e. requiring the an attacker require 10 times as many IP addresses in different /16s). Adding test-before-evict increases resistance of the node by an additional 3000 attacker IP addresses.

  Below I graph the attack over even greater attacker resources (i.e. more attacker controled IP addresses). Note that test-before-evict maintains some security far longer even against an attacker with 50,000 IPs. If this node had a larger tried table test-before-evict could greatly boost a nodes resistance to eclipse attacks.

  ![attacker graph long view](https://cloud.githubusercontent.com/assets/274814/17367108/96f46d64-595c-11e6-91cd-edba160598e7.png)

Tree-SHA512: fdad4d26aadeaad9bcdc71929b3eb4e1f855b3ee3541fbfbe25dca8d7d0a1667815402db0cb4319db6bd3fcd32d67b5bbc0e12045c4252d62d6239b7d77c4395
2020-06-13 14:50:11 -05:00
PastaPastaPasta
224d0a3fb2
Backport 12381 (#3528)
* Merge #12381: Remove more boost threads

004f999 boost: drop boost threads for [alert|block|wallet]notify (Cory Fields)
0827267 boost: drop boost threads from torcontrol (Cory Fields)
ba91724 boost: remove useless threadGroup parameter from Discover (Cory Fields)
f26866b boost: drop boost threads for upnp (Cory Fields)

Pull request description:

  This doesn't completely get rid of boost::thread, but this batch should be easy to review, and leaves us with only threadGroup (scheduler + scriptcheck) remaining.

  Note to reviewers: The upnp diff changes a bunch of whitespace, it's much more clear with 'git diff -w'

Tree-SHA512: 5a356798d0785f93ed143d1f0afafe890bc82f0d470bc969473da2d2aa78bcb9b096f7ba11b92564d546fb447d4bd0d347e7842994ea0170aafd53fda7e0a66e

* fix using std::thread

Signed-off-by: pasta <pasta@dashboost.org>

* Switch to std::thread in NotifyTransactionLock

* Move StopTorControl call from Shutdown to PrepareShutdown

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-06-13 21:21:30 +03:00
UdjinM6
99ecf1d76c
Merge pull request #3526 from PastaPastaPasta/backport-14733-14024
Backport 14733 14024
2020-06-13 21:16:17 +03:00
Wladimir J. van der Laan
feb0c4949b Merge #14733: P2P: Make peer timeout configurable, speed up very slow test and ensure correct code path tested.
48b37db50 make peertimeout a debug argument, remove error message translation (Zain Iqbal Allarakhia)
8042bbfbf p2p: allow p2ptimeout to be configurable, speed up slow test (Zain Iqbal Allarakhia)

Pull request description:

  **Summary:**

  1. _Primary_: Adds a `debug_only=true` flag for peertimeout, defaults to 60 sec., the current hard-coded setting.
  2. _Secondary_: Drastically speeds up `p2p_timeout.py` test.
  3. _Secondary_: Tests that the correct code path is being tested by adding log assertions to the test.

  **Rationale:**

  - P2P timeout was hard-coded: make it explicitly specified and configurable, instead of a magic number.
  - Addresses #13518; `p2p_timeout.py` takes 4 sec. to run instead of 61 sec.
  - Makes `p2p_timeout.py` more explicit. Previously, we relied on a comment to inform us of the timeout amount being tested. Now it is specified directly in the test via passing in the new arg; `-peertimeout=3`.
  - Opens us up to testing more P2P scenarios; oftentimes slow tests are the reason we don't test.

  **Locally verified changes:**

  _With Proposed Change (4.7 sec.):_
  ```
  $ time ./test/functional/p2p_timeouts.py
  2018-11-19T00:04:19.077000Z TestFramework (INFO): Initializing test directory /tmp/testhja7g2n7
  2018-11-19T00:04:23.479000Z TestFramework (INFO): Stopping nodes
  2018-11-19T00:04:23.683000Z TestFramework (INFO): Cleaning up /tmp/testhja7g2n7 on exit
  2018-11-19T00:04:23.683000Z TestFramework (INFO): Tests successful

  real    0m4.743s
  ```

  _Currently  on master (62.8 sec.):_
  ```
  $ time ./test/functional/p2p_timeouts.py
  2018-11-19T00:06:10.948000Z TestFramework (INFO): Initializing test directory /tmp/test6mo6k21h
  2018-11-19T00:07:13.376000Z TestFramework (INFO): Stopping nodes
  2018-11-19T00:07:13.631000Z TestFramework (INFO): Cleaning up /tmp/test6mo6k21h on exit
  2018-11-19T00:07:13.631000Z TestFramework (INFO): Tests successful

  real    1m2.836s
  ```

  _Error message demonstrated for new argument `-peertimeout`:_
  ```
  $ ./bitcoind -peertimeout=-5
  ...
  Error: peertimeout cannot be configured with a negative value.
  ```

Tree-SHA512: ff7a244ebea54c4059407bf4fb86465714e6a79cef5d2bcaa22cfe831a81761aaf597ba4d5172fc2ec12266f54712216fc41b5d24849e5d9dab39ba6f09e3a2a
2020-06-12 17:03:00 -05:00
MarcoFalke
bb035972ee
Merge #12716: Fix typos and cleanup in various files
4d9b4256d8 Fix typos (Dimitris Apostolou)

Pull request description:

  Unfortunately I messed up my repo while trying to squash #12593 so I created a PR with just the correct fixes.

Tree-SHA512: 295d77b51bd2a9381f1802c263de7ffb2edd670d9647391e32f9a414705b3c8b483bb0e469a9b85ab6a70919ea13397fa8dfda2aea7a398b64b187f178fe6a06
Signed-off-by: pasta <pasta@dashboost.org>
2020-06-11 23:20:48 -05:00
UdjinM6
2b7657d9e4
Merge pull request #3518 from PastaPastaPasta/backports-0.17-pr1
Backports 0.17 pr1
2020-06-11 20:51:58 +03:00
MarcoFalke
26ca61ec21
Merge #10498: Use static_cast instead of C-style casts for non-fundamental types
9ad6746ccd Use static_cast instead of C-style casts for non-fundamental types (practicalswift)

Pull request description:

  A C-style cast is equivalent to try casting in the following order:

  1. `const_cast(...)`
  2. `static_cast(...)`
  3. `const_cast(static_cast(...))`
  4. `reinterpret_cast(...)`
  5. `const_cast(reinterpret_cast(...))`

  By using `static_cast<T>(...)` explicitly we avoid the possibility of an unintentional and dangerous `reinterpret_cast`. Furthermore `static_cast<T>(...)` allows for easier grepping of casts.

  For a more thorough discussion, see ["ES.49: If you must use a cast, use a named cast"](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#es49-if-you-must-use-a-cast-use-a-named-cast) in the C++ Core Guidelines (Stroustrup & Sutter).

Tree-SHA512: bd6349b7ea157da93a47b8cf238932af5dff84731374ccfd69b9f732fabdad1f9b1cdfca67497040f14eaa85346391404f4c0495e22c467f26ca883cd2de4d3c
2020-06-10 18:32:43 -05:00
Wladimir J. van der Laan
90ddef59a6 Merge #14140: [Net] Switch nPrevNodeCount to vNodesSize
83d53058ae240a5c396ac927972ad96d2a11e903 Switch nPrevNodeCount to vNodesSize. (Patrick Strateman)

Pull request description:

  These both have the same value, but the variable naming is confusing.

Tree-SHA512: 4f645e89efdc69884ff4c8bbcf42e2b35d2733687c0fc6ab3f0797e0141fe23ef9cde8bb6ba422f47a88f554e55a099b1f0b3f47cb9fde12db3d46b9a0041bb0
2020-06-08 20:27:14 -05:00
Wladimir J. van der Laan
02ed3cb8ca Merge #13672: Modified in_addr6 cast in CConman class to work with msvc
822a2a33a74c3f997e7982d629c8f6158b80c093 Modified in_addr6 cast in CConman class to work with msvc. (Aaron Clauson)

Pull request description:

  Fix to allow net.cpp to compile with MSVC. Without this fix the `(in6_addr)IN6ADDR_ANY_INIT` implicit cast generates a compilation error.

Tree-SHA512: f21c5002401dc93564dcf8d49fbafe7c03ad4182df1616d2ee201e2e172f1d696ca7982fb5b42a3b7d6878c8649823044a858401b4172239fb4b0cc2a38db282
2020-06-08 20:27:14 -05:00
Alexander Block
911b5580e4
Fix typo in error log when EPOLL_CTL_ADD fails for wakeup pipe (#3451) 2020-04-23 12:33:36 +03:00
Alexander Block
d59deea77b
Merge pull request #3445 from codablock/pr_epoll
Implement epoll support
2020-04-22 17:10:33 +02:00
Alexander Block
d6b69dbf69 Revert "Only run DisconnectNodes once per second"
This reverts commit 30e4afae00.
2020-04-22 16:05:11 +02:00
Alexander Block
e51580f052 Only linger when the other side has not shutdown/closed the socket yet
Otherwise immediately close the socket and remove the node from the list.
2020-04-22 07:50:40 +02:00
Alexander Block
1df71a0f22 Don't consider disconnecting nodes in GetNodeCount and GetNodeStats 2020-04-22 06:32:32 +02:00
Alexander Block
ccb1f84b3a Revert "Invoke DisconnectNodes right before checking for duplicate connections"
This reverts commit 76d7b17dcd.
2020-04-22 06:32:32 +02:00
Alexander Block
33bfaffbea Don't return nodes with fDisconnect=true in FindNode
FindNode is only interested in active connections, especially when called
from OpenNetworkConnection. Connections which are about to get disconnected
and removed should be treated as if they are not existent anymore, as
otherwise there is a small race between disconnecting and reconnecting
nodes, causing OpenNetworkConnection to return early.
2020-04-22 06:32:32 +02:00
Alexander Block
d516e3d1e0 Advance iterator in CConnman::DisconnectNodes when lingering 2020-04-21 08:16:49 +02:00
Alexander Block
3fa94aac56 Implement epoll support 2020-04-20 15:38:19 +02:00
PastaPastaPasta
087d98477b
Merge bitcoin#10387: Eventually connect to NODE_NETWORK_LIMITED peers (#3417)
* Merge #10387: Eventually connect to NODE_NETWORK_LIMITED peers

eb91835 Add setter for g_initial_block_download_completed (Jonas Schnelli)
3f56df5 [QA] add NODE_NETWORK_LIMITED address relay and sync test (Jonas Schnelli)
158e1a6 [QA] fix mininode CAddress ser/deser (Jonas Schnelli)
fa999af [QA] Allow addrman loopback tests (add debug option -addrmantest) (Jonas Schnelli)
6fe57bd Connect to peers signaling NODE_NETWORK_LIMITED when out-of-IBD (Jonas Schnelli)
31c45a9 Accept addresses with NODE_NETWORK_LIMITED flag (Jonas Schnelli)

Pull request description:

  Eventually connect to peers signalling NODE_NETWORK_LIMITED if we are out of IBD.
  Accept and relay NODE_NETWORK_LIMITED peers in addrman.

Tree-SHA512: 8a238fc97f767f81cae1866d6cc061390f23a72af4a711d2f7158c77f876017986abb371d213d1c84019eef7be4ca951e8e6f83fda36769c4e1a1d763f787037
Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/init.cpp
#	src/protocol.h
#	test/functional/node_network_limited.py

* remove witness

Signed-off-by: Pasta <pasta@dashboost.org>

* fix test expecting witness flag

Signed-off-by: Pasta <pasta@dashboost.org>

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
2020-04-19 16:21:47 +03:00
UdjinM6
d663f48085
Lower SELECT_TIMEOUT_MILLISECONDS for USE_WAKEUP_PIPE case (#3444) 2020-04-19 14:48:44 +02:00
Alexander Block
47af42a69c AddRef/Release when adding/erasing CNode* entries to/from mapNodesWithDataToSend 2020-04-18 21:05:42 +02:00
Alexander Block
e4be48bc7e Invoke select/poll with 0 timeout in case we know that there is work 2020-04-18 21:05:42 +02:00
Alexander Block
e263edd573 Still invoke ReleaseNodeVector when interrupted 2020-04-18 21:05:42 +02:00
Alexander Block
5c9f548640 Track which nodes are able to receive/send data
Instead of selecting every socket in every SocketHandler iteration, we will
now track which nodes are known to have pending receivable data and/or
have empty send buffers.

Each time recv fails to fill a whole receive buffer, fHasRecvData is
set to false so that the socket is added to the receive select set
in the next iteration. When that socket is signalled through select/poll,
fHasRecvData is set to true again and remains true until a future recv
fails.

Each time send fails to send a full message, fCanSendData is set to false
so that the socket is added to the send select set in the next iteration.

At the same time, nodes which have pending messages to send are tracked
in mapNodesWithDataToSend, so that SocketHandler knows for which nodes
SocketSendData must be invoked.
2020-04-18 21:05:42 +02:00
Alexander Block
0e8e22aa16 Track SOCKET to CNode* mapping 2020-04-18 21:05:42 +02:00
Alexander Block
94fc4fb027 Track size of vSendMsg in atomic nSendMsgSize
This allows us to check for pending messages without locking cs_vSend
2020-04-18 21:05:15 +02:00
Alexander Block
1621e82e95 Move socket receiving into SocketRecvData 2020-04-18 21:05:15 +02:00
Alexander Block
50fc3c5cce Make SocketSendData non-const 2020-04-18 21:05:15 +02:00
Alexander Block
176d625860 Sleep longer when USE_WAKEUP_PIPE is enabled 2020-04-18 12:41:25 +02:00
Alexander Block
97ee3015e1 Wakeup select/poll when new nodes are added 2020-04-18 12:41:25 +02:00
Alexander Block
6d83b0a053
Merge pull request #3431 from codablock/pr_socketevents_arg
Make socketevents mode (poll vs select) configurable via parameter
2020-04-18 12:34:34 +02:00
UdjinM6
20a0f3a51f
Merge pull request #3407 from PastaPastaPasta/backports-0.16-pr17
Backports 0.16 pr17
2020-04-18 12:57:51 +03:00
Alexander Block
ea81ab5af7 Make socketevents mode (poll vs select) configurable via parameter
Instead of selecting the mode at compile time
2020-04-17 22:09:52 +02:00
Alexander Block
9a8caf0986
Remove fix for fNetworkActive vs OpenNetworkConnection race (#3430)
This is not needed anymore due to bitcoin#13212 being backported
2020-04-17 21:46:58 +02:00
Alexander Block
96ed9fae39
Merge pull request #3429 from codablock/pr_remove_optimistic_send
Fix flushing of rejects before disconnecting
2020-04-17 21:46:36 +02:00
Alexander Block
0cb385c567
Merge pull request #3428 from codablock/pr_better_conn_logging
Improve network connections related logging
2020-04-17 21:46:17 +02:00
Alexander Block
08b57c198b Add some linger time between fDisconnect=true and actually closing the socket 2020-04-17 12:42:30 +02:00
Alexander Block
4117579e8f Honor fLogIPs in OpenNetworkConnection 2020-04-17 12:21:00 +02:00
Alexander Block
d8bbdee7c4 Remove support for optimistic send
This was only used in only one remaining place and only to ensure that
reject messages are sent before closing sockets. This is solved by the
previous commit now.
2020-04-17 11:32:18 +02:00
Alexander Block
926beb3406 Handle sockets one last time before closing sockets 2020-04-17 11:10:08 +02:00
Alexander Block
6ca78b601e Introduce NETCONN logging category and NET_NETCONN to log in NET and NETCONN
This allows easier debugging of connection issues, as it produces much less
chatter in the logs when only -debug=mnconn is given.
2020-04-17 08:52:57 +02:00
Alexander Block
71f1181d21 Improve connection logging in net.cpp 2020-04-17 08:52:48 +02:00
Alexander Block
76d7b17dcd Invoke DisconnectNodes right before checking for duplicate connections 2020-04-17 07:53:07 +02:00
Alexander Block
30e4afae00 Only run DisconnectNodes once per second
This change is quite invasive for integration tests, as they all expect
connections to be dropped ASAP. The next commits all try to fix the upcoming
issues.
2020-04-17 07:53:07 +02:00
Alexander Block
65fb8bc454 Only run InactivityCheck once per second 2020-04-16 18:08:53 +02:00
Alexander Block
24ead62905 Only call SendMessages when ProcessMessages did some work or when some time passed
Invoking SendMessages for hundreds of peers while we know that there is no
work to do give a lot of overhead on the message handler thread.
2020-04-16 18:06:59 +02:00
MarcoFalke
a13e2f4355
Merge #13123: net: Add Clang thread safety annotations for guarded variables in the networking code
4894133dc5 Add missing lock in CNode::copyStats(...) (practicalswift)
b312cd7707 Add missing locking annotations (practicalswift)

Pull request description:

  Add Clang thread safety annotations for variables guarded by:
  * `cs_addrLocal`
  * `cs_addrName`
  * `cs_feeFilter`
  * `cs_filter`
  * `cs_hSocket`
  * `cs_inventory`
  * `cs_mapLocalHost`
  * `cs_most_recent_block`
  * `cs_proxyInfos`
  * `cs_sendProcessing`
  * `cs_setBanned`
  * `cs_SubVer`
  * `cs_vOneShots`
  * `cs_vProcessMsg`
  * `cs_vRecv`
  * `cs_vSend`

  Changed files:
  * `src/net.{cpp,h}`
  * `src/netbase.cpp`

Tree-SHA512: 319a1574a07d766e81fab19b9cfdcf8b5f0b175034ebef220cd406f1672b4ef2c57f5c456c623456ca7a1f96308de69c73535792e9e4c34b848b55fd4f35fc95
2020-04-16 10:24:22 -05:00
Wladimir J. van der Laan
b215a1f005
Merge #12986: Trivial: Corrected comment array name from pnSeeds6 to pnSeed6
d1b828b correcting the array name from pnSeeds6 to pnSeed6 (okayplanet)

Pull request description:

  After reading this comment, I tried to grep the code base for `pnSeeds6` but couldn't find anything.  After some time wasted, I realized the arrays actually start with `pnSeed6`.

Tree-SHA512: 78a66d96128267278c7b02dbc0c966a7447cb258c71b04be5a00de8ed10ee11437550bbbf7dd4208315a59d19408a31b62774ae6dbc521261818dece4970a602
2020-04-16 10:24:22 -05:00
Alexander Block
36d909aa30 Fix compilation error 2020-04-14 13:32:50 +02:00
Pieter Wuille
ea9f97c53b Merge #13298: Net: Bucketing INV delays (1 bucket) for incoming connections to hide tx time
d45b344ffd Bucket for inbound when scheduling invs to hide tx time (Gleb)

Pull request description:

  It has been brought up to my attention that current random delays mechanism (originally intended to obfuscate transaction metadata) allows to easily estimate the time a transaction was received by a node.

  It may be done by connecting multiple observer nodes to the same node. Each of those nodes will generate its own schedule of delays. Combined metadata regarding those events from different sources allows an observer to estimate transaction time.

  After this patch a spy won't gain additional information by just creating multiple connections to a target.

Tree-SHA512: c71dae5ff350b614cb40a8e201fd0562d3e03e3e72a5099718cd451f0d84c66d5e52bbaf0d5b4b75137514c8efdedcc6ef4df90142b360153f04ad0721545ab1
2020-04-14 13:32:50 +02:00
UdjinM6
d2756fb210
Merge pull request #3395 from PastaPastaPasta/backports-0.16-pr16
Backports 0.16 pr16
2020-04-10 13:11:03 +03:00
Alexander Block
26dec64e79
Merge pull request #3397 from codablock/pr_backport_txrequests
Backport bitcoin#14897 and bitcoin#15834 and modify it to work with Dash messages
2020-04-09 00:12:44 +02:00
Alexander Block
9401366e8e Set wakeupSelectNeeded=true while poll() is running 2020-04-08 23:19:17 +02:00
Wladimir J. van der Laan
1524185462 Merge #13212: Net: Fixed a race condition when disabling the network.
793290f940a9af18d4f0292a263d976a066dff65 Net: Fixed a race condition when disabling the network. (lmanners)

Pull request description:

  This change addresses a race condition where setnetworkactive=false wouldn't always disconnect all peers.

  Before this change, the following could happen:
  1. Thread A -- Begins connecting to a node.
  2. Thread B -- Sets kNetworkActive=false and disconnects connected nodes.
  3. Thread A -- Finishes connecting and adds node to list of connected nodes.

  The node that was connected from Thread A remains connected and active,
  even though kNetworkActive=false.

  To fix the race, disconnections when kNetworkActive=false are now handled in the main network loop.

  fixes #13038

Tree-SHA512: 6d0b7a78ae956358e796efcc034cb532c2e0d824a52ae822a3899eefc7df76327519d1f2d77362c9fddf34ef860abd095d7490d7cc02d0ba7543bf1e8c8f2535
2020-04-08 22:46:34 +02:00
Alexander Block
575d89fcf4 Introduce USE_WAKEUP_PIPE
Instead of checking for WIN32
2020-04-08 22:43:56 +02:00
Alexander Block
efec6cc409 Implement wakeup pipe handling in poll based SocketEvents 2020-04-08 22:43:56 +02:00
Patrick Strateman
c0b10497bf Implement poll() on systems which support it properly.
This eliminates the restriction on maximum socket descriptor number.
2020-04-08 22:43:56 +02:00
Patrick Strateman
7c022a20a1 Move SocketEvents logic to private method.
This separates the select() logic from the socket handling logic, setting up
for a switch to poll().
2020-04-08 22:43:56 +02:00
Patrick Strateman
ccc3f267ba Move GenerateSelectSet logic to private method.
This separates the socket event collection logic from the logic
deciding which events we're interested in at all.
2020-04-08 22:43:56 +02:00
Patrick Strateman
7a38df4f76 Introduce and use constant SELECT_TIMEOUT_MILLISECONDS. 2020-04-08 22:43:56 +02:00
Patrick Strateman
4e00e435e9 Move SocketHandler logic to private method. 2020-04-08 22:43:56 +02:00
Patrick Strateman
cbb6c9a4f9 Move InactivityCheck logic to private method. 2020-04-08 22:43:56 +02:00
Patrick Strateman
07f6d92e36 Move DisconnectNodes logic to private method. 2020-04-08 22:43:55 +02:00
Patrick Strateman
6eef194443 Move NotifyNumConnectionsChanged logic to private method. 2020-04-08 22:38:04 +02:00
Alexander Block
e20c63f535
Merge pull request #3398 from codablock/pr_speedups
A few simple/trivial optimizations
2020-04-08 22:19:14 +02:00
Alexander Block
38556a3d49
Don't try to connect to masternodes that we already have a connection to (#3401) 2020-04-08 22:18:45 +02:00
Alexander Block
2dff0501e9
Remove semaphore for masternode connections (#3403)
We now have hundreds of MN connections due to the new intra quorum connections,
which means having a limiting semaphore here doesn't make sense anymore. It
actually causes DKG and signing failures as not all necessary connections
can be established.

Masternode connections are still implicitely limited as there is always
only a limited set of LLMQs active.
2020-04-08 22:18:21 +02:00
Luke Dashjr
38ccf6f628
Bugfix: Include <memory> for std::unique_ptr 2020-04-08 11:46:27 -05:00
Alexander Block
fd1875b61f Reserve vector size in CopyNodeVector 2020-04-07 07:27:23 +02:00
Alexander Block
122b740e2d Optimize deletion of vNodes entries
Iterate through the vectors with iterators and use them for .erase().
This avoids an expensive lookup when erasing.
2020-04-07 07:27:07 +02:00
Alexander Block
481339355d Don't hold cs_vNodes in ReleaseNodeVector
We're not touching this->vNodes here, so there is no need to hold the lock.
2020-04-07 07:25:14 +02:00
Alexander Block
004d9224c4 Replace uses of CConnman::RemoveAskFor with EraseObjectRequest 2020-04-07 07:14:42 +02:00
Pieter Wuille
8c0ff34ccd Merge #14897: randomize GETDATA(tx) request order and introduce bias toward outbound
1cff3d6cb0 Change in transaction pull scheduling to prevent InvBlock-related attacks (Gleb Naumenko)

Pull request description:

  This code makes executing two particular (and potentially other) attacks harder.

  ### InvBlock
  This behavior was described well [here](https://www.cs.umd.edu/projects/coinscope/coinscope.pdf) (page 11).

  Per current implementation, if node A receives _INV_ (tx) from node B, node A sends _GETDATA_ to B and waits for _TX_ message back.

  Node A is likely to receive more _INVs_ (regarding the same tx) from other peers. But node A would not send another _GETDATA_ unless it does not hear _TX_ back from node B for next 2 minutes (to save bandwidth)

  Thus, if B is a malicious node, it can prevent node A from getting the transaction (even if all A’s peers have it) for 2 minutes.

  This behavior seems to be an inherent limitation of the current P2P relay protocol, and I don’t see how it can be fundamentally changed (I can see workarounds which involve rewriting a lot of P2P code though).

  ### What does this PR fix?

  The attacks I’m looking at involve preventing A from learning the transaction for 2*N minutes. To do that, an attacker has to spin up N nodes and send N _INVs_ simultaneously to node A (then InvBlocks will be queued with an interval of 2 minutes according to current implementation)

  More precisely, 2 scenarios I’m looking at are:
  1. An attacker censors a particular transaction. By performing InvBlock from different nodes, an attacker can execute a network-wide censorship of a particular transaction (or all transactions). The earlier an attacker founds the transaction he wants to censor, the easier it is to perform an attack. As it was pointed out by @gwillen, this is even more dangerous in the case of lightning, where transactions are known in advance.
  2. Topology inference described in papers [1](https://www.cs.umd.edu/projects/coinscope/coinscope.pdf), [2](https://arxiv.org/pdf/1812.00942.pdf) involve network-wide InvBlock. This fix would not mitigate this type of inference, but I believe it will make it more expensive to perform (an attacker would have to create more transactions and perform more rounds to learn the topology, the second paper itself notes that InvBlock isolation is important for the attack).

  ### How does it work
  This PR introduces bias toward outbound connections (they have higher priority when a node chooses from whom it should request a transaction) and randomizes the order.
  As per @gmaxwell suggestion, GETDATA requests queue is created after processing all incoming messages from all nodes.

  After this fix, if the incoming messages were [I1, I2, I3, O1, O2, O3, O4], the queue for _GETDATA_ may look like [O2, O1, O3, O4, I1, I3, I2, ….].

  If {I1, I2, I3} were significantly earlier (but the difference is less than TX_TIMEOUT=60 s) than others, the queue for _GETDATA_ may look like [I2, O2, O1, O3, O4, I1, I3, ….].

  ### Other comments:
  1. This mitigation works better if the connectivity is higher (especially outbound, because it would be less likely that 2 _GETDATAs_ for inbound malicious nodes queued together)

Tree-SHA512: 2ad1e80c3c7e16ff0f2d1160aa7d9a5eaae88baa88467f156b987fe2a387f767a41e11507d7f99ea02ab75e89ab93b6a278d138cb1054f1aaa2df336e9b2ca6a
2020-04-07 07:14:00 +02:00
UdjinM6
3e072bed89
Merge pull request #3387 from PastaPastaPasta/backports-0.16-pr13
Backports 0.16 pr13
2020-04-03 02:31:23 +03:00
Wladimir J. van der Laan
fe8c18ca77 Merge #10574: Remove includes in .cpp files for things the corresponding .h file already included
a720b92 Remove includes in .cpp files for things the corresponding .h file already included (practicalswift)

Pull request description:

  Remove includes in .cpp files for things the corresponding .h file already included.

  Example case:
  * `addrdb.cpp` includes `addrdb.h` and `fs.h`
  * `addrdb.h` includes `fs.h`

  Then remove the direct inclusion of `fs.h` in `addrman.cpp` and rely on the indirect inclusion of `fs.h` via the included `addrdb.h`.

  In line with the header include guideline (see #10575).

Tree-SHA512: 8704b9de3011a4c234db336a39f7d2c139e741cf0f7aef08a5d3e05197e1e18286b863fdab25ae9638af4ff86b3d52e5cab9eed66bfa2476063aa5c79f9b0346
2020-04-01 12:43:18 -05:00
MarcoFalke
a98db86ada Merge #10493: Use range-based for loops (C++11) when looping over map elements
680bc2cbb Use range-based for loops (C++11) when looping over map elements (practicalswift)

Pull request description:

  Before this commit:

  ```c++
  for (std::map<T1, T2>::iterator x = y.begin(); x != y.end(); ++x) {
      T1 z = (*x).first;
      …
  }
  ```

  After this commit:

  ```c++
  for (auto& x : y) {
      T1 z = x.first;
      …
  }
  ```

Tree-SHA512: 954b136b7f5e6df09f39248a6b530fd9baa9ab59d7c2c7eb369fd4afbb591b7a52c92ee25f87f1745f47b41d6828b7abfd395b43daf84a55b4e6a3d45015e3a0
2020-04-01 12:43:18 -05:00
Alexander Block
d3586e1df0 Immediately close connections again when fNetworkActive==false 2020-03-31 07:07:52 +02:00
Alexander Block
cace76d07f Actually use LLMQConnectionRetryTimeout for probing 2020-03-31 07:07:52 +02:00
Alexander Block
e8bbbec259 Don't try to open masternode connections when network is disabled 2020-03-27 15:12:20 +01:00
Alexander Block
8f644d18d7 Implement probing of public ip/port of LLMQ members 2020-03-27 15:12:20 +01:00
Alexander Block
b6bdb8be9e
Faster opening of masternode connections (#3375)
Only sleep 100ms when we previously tried to connect a MN. The back-off
logic in ThreadOpenMasternodeConnections will prevent too many unsuccessful
connects to offline/bad nodes.
2020-03-26 13:24:06 +01:00
Alexander Block
8211ff0a8a
Merge pull request #3368 from codablock/pr_fmasternode
Don't relay anything to inbound/outbound fMasternode connections
2020-03-24 17:40:51 +01:00
Alexander Block
7f1f1d12f5 Make EnsureQuorumConnections re-set connections in every iteration
Instead of only doing it the first time. This is a preparation for
the new intra-quorum connection system (which connects all members to all
other members)
2020-03-24 10:58:29 +01:00
Alexander Block
c0bb06e766
Merge pull request #3366 from codablock/pr_fix_mnconns
Multiple fixes for masternode connection handling
2020-03-24 10:57:42 +01:00
Alexander Block
f2ece1031f
Remove logging for waking of select() (#3370)
This was always quite spammy and so far never useful in debugging.
2020-03-24 10:55:24 +01:00
Alexander Block
2a6465a6fb Move LLMQ connection retry timeout into chainparams 2020-03-21 12:21:09 +01:00
Alexander Block
40cdfe8662
Add peer id to "socket send error" logs (#3363) 2020-03-21 11:33:37 +01:00
Alexander Block
71e57a25fa Add masternode flag to result of getpeerinfo 2020-03-20 17:16:49 +01:00
Alexander Block
31825146a3 Don't relay anything to fMasternode connections
This reduces traffic on these connections to PS and DKG/LLMQ traffic only.
2020-03-20 17:16:49 +01:00
Alexander Block
458a63736d Track last outbound connection attempts in CMasternodeMetaMan
Instead of relying on CAddrMan, which only works well for addresses
announced in P2P networking (and not with MNs).
2020-03-20 17:16:24 +01:00
Alexander Block
93ed22b239 Logging for outgoing masternode connections 2020-03-20 17:16:24 +01:00
Alexander Block
35d75b19e6 Make pending masternode queue proTxHash based
Instead of CService
2020-03-20 17:16:24 +01:00
Alexander Block
0adef2cf7a Fix ThreadOpenMasternodeConnections to not drop pending MN connections
The way it was implemented caused vPendingMasternodes entries to be popped
but not necessarily connected to when at the same time quorum connections
were pending.
2020-03-20 17:16:24 +01:00
PastaPastaPasta
8a1ec935a0
Backport 11651 (#3358)
* scripted-diff: Replace #include "" with #include <> (ryanofsky)

-BEGIN VERIFY SCRIPT-
for f in \
  src/*.cpp \
  src/*.h \
  src/bench/*.cpp \
  src/bench/*.h \
  src/compat/*.cpp \
  src/compat/*.h \
  src/consensus/*.cpp \
  src/consensus/*.h \
  src/crypto/*.cpp \
  src/crypto/*.h \
  src/crypto/ctaes/*.h \
  src/policy/*.cpp \
  src/policy/*.h \
  src/primitives/*.cpp \
  src/primitives/*.h \
  src/qt/*.cpp \
  src/qt/*.h \
  src/qt/test/*.cpp \
  src/qt/test/*.h \
  src/rpc/*.cpp \
  src/rpc/*.h \
  src/script/*.cpp \
  src/script/*.h \
  src/support/*.cpp \
  src/support/*.h \
  src/support/allocators/*.h \
  src/test/*.cpp \
  src/test/*.h \
  src/wallet/*.cpp \
  src/wallet/*.h \
  src/wallet/test/*.cpp \
  src/wallet/test/*.h \
  src/zmq/*.cpp \
  src/zmq/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* scripted-diff: Replace #include "" with #include <> (Dash Specific)

-BEGIN VERIFY SCRIPT-
for f in \
  src/bls/*.cpp \
  src/bls/*.h \
  src/evo/*.cpp \
  src/evo/*.h \
  src/governance/*.cpp \
  src/governance/*.h \
  src/llmq/*.cpp \
  src/llmq/*.h \
  src/masternode/*.cpp \
  src/masternode/*.h \
  src/privatesend/*.cpp \
  src/privatesend/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* build: Remove -I for everything but project root

Remove -I from build system for everything but the project root,
and built-in dependencies.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/Makefile.test.include

* qt: refactor: Use absolute include paths in .ui files

* qt: refactor: Changes to make include paths absolute

This makes all include paths in the GUI absolute.

Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/qt/dash.cpp
#	src/qt/optionsmodel.cpp
#	src/qt/test/rpcnestedtests.cpp

* test: refactor: Use absolute include paths for test data files

* Recommend #include<> syntax in developer notes

* refactor: Include obj/build.h instead of build.h

* END BACKPORT #11651 Remove trailing whitespace causing travis failure

* fix backport 11651

Signed-off-by: Pasta <pasta@dashboost.org>

* More of 11651

* fix blockchain.cpp

Signed-off-by: pasta <pasta@dashboost.org>

* Add missing "qt/" in includes

* Add missing "test/" in includes

* Fix trailing whitespaces

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-20 01:46:56 +03:00
Wladimir J. van der Laan
c1897a6f14
Merge #11043: Use std::unique_ptr (C++11) where possible
a357293 Use MakeUnique<Db>(...) (practicalswift)
3e09b39 Use MakeUnique<T>(...) instead of std::unique_ptr<T>(new T(...)) (practicalswift)
8617989 Add MakeUnique (substitute for C++14 std::make_unique) (practicalswift)
d223bc9 Use unique_ptr for pcoinscatcher/pcoinsdbview/pcoinsTip/pblocktree (practicalswift)
b45c597 Use unique_ptr for pdbCopy (Db) and fix potential memory leak (practicalswift)
29ab96d Use unique_ptr for dbenv (DbEnv) (practicalswift)
f72cbf9 Use unique_ptr for pfilter (CBloomFilter) (practicalswift)
8ccf1bb Use unique_ptr for sem{Addnode,Outbound} (CSemaphore) (practicalswift)
73db063 Use unique_ptr for upnp_thread (boost::thread) (practicalswift)
0024531 Use unique_ptr for dbw (CDBWrapper) (practicalswift)
fa6d122 Use unique_ptr:s for {fee,short,long}Stats (TxConfirmStats) (practicalswift)
5a6f768 Use unique_ptr for httpRPCTimerInterface (HTTPRPCTimerInterface) (practicalswift)
860e912 Use unique_ptr for pwalletMain (CWallet) (practicalswift)

Pull request description:

  Use `std::unique_ptr` (C++11) where possible.

  Rationale:
  1. Avoid resource leaks (specifically: forgetting to `delete` an object created using `new`)
  2. Avoid undefined behaviour (specifically: double `delete`:s)

  **Note to reviewers:** Please let me know if I've missed any obvious `std::unique_ptr` candidates. Hopefully this PR should cover all the trivial cases.

Tree-SHA512: 9fbeb47b800ab8ff4e0be9f2a22ab63c23d5c613a0c6716d9183db8d22ddbbce592fb8384a8b7874bf7375c8161efb13ca2197ad6f24b75967148037f0f7b20c
2020-02-29 11:51:03 -06:00
UdjinM6
f2db4e7d66
Merge pull request #3316 from PastaPastaPasta/backports-0.16-pr7
Backports 0.16 pr7
2020-02-04 15:32:11 +03:00
UdjinM6
df73438708
Fix node protection logic false positives (#3314)
We could be reading multiple messages from a socket buffer at once _without actually processing them yet_ which means that `fSuccessfullyConnected` might not be switched to `true` at the time we already parsed `VERACK` message and started to parse the next one. This is basically a false positive and we drop a legit node as a result even though the order of messages sent by this node was completely fine. To fix this I partially reverted #2790 (where the issue was initially introduced) and moved the logic for tracking the first message into ProcessMessage instead.
2020-02-01 04:59:01 +03:00
Wladimir J. van der Laan
b9e3ae689a
Merge #11616: Update ban-state in case of dirty-state during periodic sweep
57ac471 Call BannedListChanged outside of cs_setBanned lock (Jonas Schnelli)
c853812 Update ban-state in case of dirty-state during periodic sweep (Jonas Schnelli)

Pull request description:

  We do currently not update the UI during periodic ban list sweeps (via dump banlist).
  Fixes #11612

Tree-SHA512: bffbdcc03c63042177bdd511b0a9187c211c2b5011178481e8ee3e43a71eef1e4cd6b72f73672babab142b644f62f8b56f0aac1d26d3f19372b1f8644fec9395
2020-01-31 07:43:43 -06:00
UdjinM6
3bcf23fac7
Merge pull request #3305 from PastaPastaPasta/backport-11363
Backport P2P refactor
2020-01-31 13:54:16 +03:00
Wladimir J. van der Laan
e7beff5bcd
Merge #12855: net: Minor accumulated cleanups
2c084a6 net: Minor accumulated cleanups (Thomas Snider)

Pull request description:

  From now-derelict larger changes I had been working on, here are a series of DRY refactors/cleanups.  Net loss of 35 lines of code - a small step in the good fight.

  In particular I think operator!= should only ever be implemented as a negation of operator==.  Lower chance for errors, and removes the possibility of divergent behavior.

Tree-SHA512: 58bf4b542a4e8e5bc465b508aaa16e9ab51448c3f9bee52cd9db0a64a5c6c5a13e4b4286d0a5aa864934fc58064799f6a88a40a87154fd3a4bd731a72e254393
2020-01-26 20:44:58 -06:00
Wladimir J. van der Laan
a3ea0e93ef
Merge #12626: Limit the number of IPs addrman learns from each DNS seeder
46e7f800b Limit the number of IPs we use from each DNS seeder (e0)

Pull request description:

  A risk exists where a malicious DNS seeder eclipses a node by returning an enormous number of IP addresses. In this commit we mitigate this risk by limiting the number of IP addresses addrman learns to 256 per DNS seeder.

  As discussed with @theuni

Tree-SHA512: 949e870765b1470200f2c650341d9e3308a973a7d1a6e557b944b0a2b8ccda49226fc8c4ff7d2a05e5854c4014ec0b67e37a3f2287556fe7dfa2048ede1f2e6f
2020-01-26 20:44:58 -06:00
Wladimir J. van der Laan
12a9809824
Merge #12342: Extend #11583 to include "version handshake timeout" message
c887f87 Extend #11583 to include the most common message generated by non-contributing peers (port scanners?) 37% of the log default log entries for a node that has been up for ~24hrs was "version handshake timeout..." (Clem Taylor)

Pull request description:

  37% of the default log entries for a node that has been up for ~24hrs was "version handshake timeout..."

Tree-SHA512: dceeee5d55a9ff7570174aeb63faac9beda239087220522adefef7ed11e0eeffa008ca28726011247c8834c1a222d37817baf895635ab874a95ebc435959070e
2020-01-26 20:44:58 -06:00
Wladimir J. van der Laan
03576217e5
Merge #12329: net: don't retry failed oneshot connections forever
660f5f1 net: don't retry failed oneshot connections forever (Cory Fields)

Pull request description:

  As introduced by (my suggestion, sorry, in) #11512, failed dns resolves end up as oneshots. But failed oneshots are re-added as oneshots, so we need to make sure that we're not queuing these up forever after failed resolves.

  Rather than trying to differentiate, I think we should just not re-add failed oneshots and be done with it.

  Maybe @sipa can shed a light on what the original intention was.

Tree-SHA512: 2dfe35dabfb6354c315cf6f8ae42971765d36575e685662caae7ed8f9dea9472c6fb1fd5e62ec35301550b74b6613a54265e90fca2a6618544f78dacaac4d4fd

fix 12329 backport

Signed-off-by: Pasta <pasta@dashboost.org>

fix 12329 backport

Signed-off-by: Pasta <pasta@dashboost.org>

fix 12329 backport

Signed-off-by: Pasta <pasta@dashboost.org>
2020-01-26 20:44:58 -06:00
Wladimir J. van der Laan
211c8a3a5b
Merge #12326: net: initialize socket to avoid closing random fd's
96dbd38 net: initialize socket to avoid closing random fd's (Cory Fields)

Pull request description:

  An excellent spot by @david60.

  Even if it isn't causing the fd issue we're looking for, this should be fixed.

Tree-SHA512: 062a8f2cdd39d895213e1263dbd7b8391473ddaea2f93c82c211a9bb6ea6744d48a6c84c8ff804b16b865d14145492635c500a9fd138d0988fee5e4f719ebb91
2020-01-26 20:44:58 -06:00