Commit Graph

93 Commits

Author SHA1 Message Date
MarcoFalke
186c32b3ff
Merge bitcoin/bitcoin#21129: fuzz: check that ser+unser produces the same AddrMan
87651795d8622d354f8e3c481eb868d9433b841c fuzz: check that ser+unser produces the same AddrMan (Vasil Dimov)
6408b24517f3418e2a408071b4c2ce26571f3167 fuzz: move init code to the CAddrManDeterministic constructor (Vasil Dimov)

Pull request description:

  Add a fuzz test that fills addrman with a pile of randomly generated addresses, serializes it to a stream, unserializes the stream to another addrman object and compares the two.

  Some discussion of this already happened at https://github.com/jnewbery/bitcoin/pull/18.

ACKs for top commit:
  practicalswift:
    cr ACK 87651795d8622d354f8e3c481eb868d9433b841c
  jonatack:
    ACK 87651795d8622d354f8e3c481eb868d9433b841c rebased to current master, reviewed, fuzz build, ran `FUZZ=addrman_serdeser src/test/fuzz/fuzz`

Tree-SHA512: 7eda79279f14f2649840bf752e575d7b02cbaad541f74f7254855ebd4a32da988f042d78aa9228983350283bb74dd0c71f51f04c0846889c3ba2f19f01a0c303
2024-01-01 17:48:13 -06:00
fanquake
51c69d1d0a Merge bitcoin/bitcoin#18722: addrman: improve performance by using more suitable containers
a92485b2c250fd18f55d22aa32722bf52ab32bfe addrman: use unordered_map instead of map (Vasil Dimov)

Pull request description:

  `CAddrMan` uses `std::map` internally even though it does not require
  that the map's elements are sorted. `std::map`'s access time is
  `O(log(map size))`. `std::unordered_map` is more suitable as it has a
  `O(1)` access time.

  This patch lowers the execution times of `CAddrMan`'s methods as follows
  (as per `src/bench/addrman.cpp`):

  ```
  AddrMan::Add(): -3.5%
  AddrMan::GetAddr(): -76%
  AddrMan::Good(): -0.38%
  AddrMan::Select(): -45%
  ```

ACKs for top commit:
  jonatack:
    ACK a92485b2c250fd18f55d22aa32722bf52ab32bfe
  achow101:
    ACK a92485b2c250fd18f55d22aa32722bf52ab32bfe
  hebasto:
    re-ACK a92485b2c250fd18f55d22aa32722bf52ab32bfe, only suggested changes and rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/18722#pullrequestreview-666663681) review.

Tree-SHA512: d82959a00e6bd68a6c4c5a265dd08849e6602ac3231293b7a3a3b7bf82ab1d3ba77f8ca682919c15c5d601b13e468b8836fcf19595248116635f7a50d02ed603
2023-12-03 20:32:22 -06:00
fanquake
795fe6a864 Merge bitcoin/bitcoin#22505: addrman: Remove unused test_before_evict argument from Good()
f036dfbb692c4d44d0f59194d089ed0aa1096347 [addrman] Remove unused test_before_evict argument from Good() (John Newbery)

Pull request description:

  This has never been used in the public interface method since it was
  introduced in #9037.

ACKs for top commit:
  lsilva01:
    Tested ACK f036dfbb69 on Ubuntu 20.04.
  theStack:
    Code-review ACK f036dfbb692c4d44d0f59194d089ed0aa1096347

Tree-SHA512: 98145d9596b4ae1f354cfa561be1a54c6b8057c920e0ac3d4c1d42c9326b2dad2d44320f4171bb701d97088b216760cca8017b84c8b5dcd2b1dc8f158f28066d
2023-12-03 20:13:09 -06:00
MarcoFalke
f40cb714e5 (partial) Merge bitcoin/bitcoin#21940: refactor: Mark CAddrMan::Select and GetAddr const
fae108ceb53f61d7338ba205873623ede3c1d3be Fix incorrect whitespace in addrman (MarcoFalke)
fa32024d51c098441623e246f304a80f011e29d1 Add missing GUARDED_BY to CAddrMan::insecure_rand (MarcoFalke)
fab755b77f88873f01cbd988051de7ad3f0150de fuzz: Actually use const addrman (MarcoFalke)
fae0c79351ce34186249d44af0c5c9c7521f4b6c refactor: Mark CAddrMan::GetAddr const (MarcoFalke)
fa02934c8c9d290ea4d12683e8680c70967a4d3a refactor: Mark CAddrMan::Select const (MarcoFalke)

Pull request description:

  To clarify that a call to this only changes the random state and nothing else.

ACKs for top commit:
  jnewbery:
    Code review ACK fae108ceb53f61d7338ba205873623ede3c1d3be
  theStack:
    re-ACK fae108ceb53f61d7338ba205873623ede3c1d3be 🍦

Tree-SHA512: 3ffb211d4715cc3daeb3bfcdb3fcc6b108ca96df5fa565510436fac0e8da86c93b30c9c4aad0563e27d84f615fcd729481072009a4e2360c8b3d40787ab6506a
2023-12-03 20:13:09 -06:00
Konstantin Akimov
ba97f49f2f
refactor: re-order headers and forward declarations to improve compile time (#5693)
## Issue being fixed or feature implemented
Some headers include other heavy headers, such as `logging.h`,
`tinyformat.h`, `iostream`. These headers are heavy and increase
compilation time on scale of whole project drastically because can be
used in many other headers.

## What was done?
Moved many heavy includes from headers to cpp files to optimize
compilation time.
In some places  added forward declarations if it is reasonable.

As side effect removed 2 circular dependencies:
```
"llmq/debug -> llmq/dkgsessionhandler -> llmq/debug"
"llmq/debug -> llmq/dkgsessionhandler -> llmq/dkgsession -> llmq/debug"
```


## How Has This Been Tested?
Run build 2 times before refactoring and after refactoring: `make clean
&& sleep 10s; time make -j18`

Before refactoring:
```
real    5m37,826s
user    77m12,075s
sys     6m20,547s

real    5m32,626s
user    76m51,143s
sys     6m24,511s
```

After refactoring:
```
real    5m18,509s
user    73m32,133s
sys     6m21,590s

real    5m14,466s
user    73m20,942s
sys     6m17,868s
```

~5% of improvement for compilation time. That's not huge, but that's
worth to get merged

There're several more refactorings TODO but better to do them later by
backports:
 - bitcoin/bitcoin#27636
 - bitcoin/bitcoin#26286
 - bitcoin/bitcoin#27238
 - and maybe this one: bitcoin/bitcoin#28200


## Breaking Changes
N/A

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
2023-11-17 10:04:18 -06:00
MarcoFalke
3ff5348d97 Merge bitcoin/bitcoin#19238: refactor: Make CAddrMan::cs non-recursive
ae98aec9c0521cdcec76459c8200bd45ff6a1485 refactor: Make CAddrMan::cs non-recursive (Hennadii Stepanov)
f5d1c7fac70f424114dae3be270fdc31589a8c34 Add AssertLockHeld to CAddrMan private functions (Hennadii Stepanov)
5ef1d0b6982f05f70ff2164ab9af1ac1d2f97f5d Add thread safety annotations to CAddrMan public functions (Hennadii Stepanov)
b138973a8b4bbe061ad97011f278a21e08ea79e6 refactor: Avoid recursive locking in CAddrMan::Clear (Hennadii Stepanov)
f79a664314b88941c1a2796623e846d0a5916c06 refactor: Apply consistent pattern for CAddrMan::Check usage (Hennadii Stepanov)
187b7d2bb36e6de9cd960378021ebe690619a2ef refactor: Avoid recursive locking in CAddrMan::Check (Hennadii Stepanov)
f77d9c79aa41dab4285e95c9432cc6d853be67a3 refactor: Fix CAddrMan::Check style (Hennadii Stepanov)
06703973c758c2c5d0ff916993aa7055f609d2d7 Make CAddrMan::Check private (Hennadii Stepanov)
efc6fac951e75ba913350bb470c3d4e6a4e284b9 refactor: Avoid recursive locking in CAddrMan::size (Hennadii Stepanov)
2da95545ea42f925dbc7703e42e9356908a8c83e test: Drop excessive locking in CAddrManTest::SimConnFail (Hennadii Stepanov)

Pull request description:

  This PR replaces `RecursiveMutex CAddrMan::cs` with `Mutex CAddrMan::cs`.

  All of the related code branches are covered by appropriate lock assertions to insure that the mutex locking policy has not been changed by accident.

  Related to #19303.

  Based on #22025, and first three commits belong to it.

ACKs for top commit:
  vasild:
    ACK ae98aec9c0521cdcec76459c8200bd45ff6a1485

Tree-SHA512: c3a2d3d955a75befd7e497a802b8c10730e393be9111ca263ad0464d32fae6c7edf9bd173ffb6bc9bb61c4b39073a74eba12979d47f26b0b7b4a861d100942df
2023-08-28 11:24:41 -05:00
Kittywhiskers Van Gogh
5088da93db merge bitcoin#22112: Force port 0 in I2P 2023-07-24 20:45:49 +03:00
Kittywhiskers Van Gogh
cf27db8574 partial bitcoin#21843: enable GetAddr, GetAddresses, and getnodeaddresses by network
excludes:
- 6c98c099918bd20e2d3aa123643d6e3594e080e4
- 3f89c0e9902338ad8a507a938dceeeb3191eece6
- ce6bca88e8c685c69686e0b8dc095ffc3e2ac34d
2023-07-24 20:45:49 +03:00
fanquake
a0984bb07c Merge bitcoin/bitcoin#22576: doc: Update high-level addrman description
036d7eadf5dd0e06e0734a5d78dbe28f4bfaa07f doc: Correct description of CAddrMan::Create() (Amiti Uttarwar)
318176aff1ded36d1fbc5977f288ac3bac1d8712 doc: Update high-level addrman description (Martin Zumsande)

Pull request description:

  The high-level description of `addrman` has outdated information with respect to the eviction behavior, both for the New and Tried tables (at least since #5941) - this has confused me in the past.

  This PR corrects this and also adds basic info about the bucket size and position.

ACKs for top commit:
  amitiuttarwar:
    reACK 036d7eadf5dd0e06e0734a5d78dbe28f4bfaa07f
  jnewbery:
    ACK 036d7eadf5dd0e06e0734a5d78dbe28f4bfaa07f

Tree-SHA512: 3f0635d765f5e580a1fae31187742a833cef66ef2286d40eeb28f2253521260038e16e5f1a65741464a2ddfdbeb5c0f1bc38bf73841e600639033d59c3c534e4
2023-06-10 17:40:23 -05:00
Kittywhiskers Van Gogh
99b336deba merge bitcoin#22179: Torv2 removal followups 2023-06-05 10:11:03 -05:00
Kittywhiskers Van Gogh
31acbf1e2d merge bitcoin#22025: Group and re-order CAddrMan members by access type 2023-06-05 10:11:03 -05:00
Kittywhiskers Van Gogh
1ef6322fc2 merge bitcoin#20284: ensure old versions don't parse peers.dat 2023-06-05 10:11:03 -05:00
Kittywhiskers Van Gogh
56cd0dd80a merge bitcoin#19658: Allow RPC to fetch all addrman records and add records to addrman 2023-06-05 10:11:03 -05:00
Kittywhiskers Van Gogh
1244d59522 merge bitcoin#18991: Cache responses to GETADDR to prevent topology leaks 2023-06-05 10:11:03 -05:00
Konstantin Akimov
b8b37f314b Merge #17891: scripted-diff: Replace CCriticalSection with RecursiveMutex
e09c701e0110350f78366fb837308c086b6503c0 scripted-diff: Bump copyright of files changed in 2020 (MarcoFalke)
6cbe6209646db8914b87bf6edbc18c6031a16f1e scripted-diff: Replace CCriticalSection with RecursiveMutex (MarcoFalke)

Pull request description:

  `RecursiveMutex` better clarifies that the mutex is recursive, see also the standard library naming: https://en.cppreference.com/w/cpp/thread/recursive_mutex

  For that reason, and to avoid different people asking me the same question repeatedly (e.g. https://github.com/bitcoin/bitcoin/pull/15932#pullrequestreview-339175124 ), remove the outdated alias `CCriticalSection` with a scripted-diff
2023-05-24 12:43:57 -05:00
Kittywhiskers Van Gogh
3e993abf10 merge bitcoin#20291: Consolidate logic around calling CAddrMan::Connected() 2023-02-28 00:11:11 +03:00
MarcoFalke
de08467941 Merge bitcoin/bitcoin#22455: addrman: detect on-disk corrupted nNew and nTried during unserialization
816f29eab296ebec2da8f8606ad618609e3ba228 addrman: detect on-disk corrupted nNew and nTried during unserialization (Vasil Dimov)

Pull request description:

  Negative `nNew` or `nTried` are not possible during normal operation.
  So, if we read such values during unserialize, report addrman
  corruption.

  Fixes https://github.com/bitcoin/bitcoin/issues/22450

ACKs for top commit:
  MarcoFalke:
    cr ACK 816f29eab296ebec2da8f8606ad618609e3ba228
  jonatack:
    ACK 816f29eab296ebec2da8f8606ad618609e3ba228
  lsilva01:
    Code Review ACK 816f29eab2.  This change provides a more accurate description of the error.

Tree-SHA512: 01bdd72d2d86a0ef770a319fee995fd1e147b24a8db84ddb8cd121688e7f94fed73fddc0084758e7183c4f8d08e971f0b1b224f5adb10928a5aa4dbbc8709d74
2022-03-07 09:42:58 -06:00
Wladimir J. van der Laan
95d588e9b4 Merge #15109: refactor: Use C++11 default member initializers
fa2510d5c1cdf9c2cd5cc9887302ced4378c7202 Use C++11 default member initializers (MarcoFalke)

Pull request description:

  Changes:
  * Remove unused constructors that leave some members uninitialized
  * Remove manual initialization in each constructor and prefer C++11 default member initializers

  This is not a stylistic change, but a change that avoids bugs such as:

  *  fix uninitialized read when stringifying an addrLocal #14728
  *  qt: Initialize members in WalletModel #12426
  *  net: correctly initialize nMinPingUsecTime #6636
  * ...

Tree-SHA512: 0f896f3b9fcc464d5fc7525f7c86343ef9ce9fb13425fbc68e9a9728fd8710c2b4e2fd039ee08279ea41ff20fd92b7185cf5cca95a0bcb6a5340a1e6f03cae6b
2021-08-10 22:20:30 -03:00
UdjinM6
a972afdf9e
More of 13115: addrman: Add Clang thread safety annotations for variables guarded by CAddrMan.cs 2021-07-29 03:33:51 +03:00
Wladimir J. van der Laan
9009f57e27 Merge #14624: Some simple improvements to the RNG code
e414486d56b9f06af7aeb07ce13e3c3780c2b69b Do not permit copying FastRandomContexts (Pieter Wuille)
022cf47dd7ef8f46e32a184e84f94d1e9f3a495c Simplify testing RNG code (Pieter Wuille)
fd3e7973ffaaa15ed32e5aeadcb02956849b8fc7 Make unit tests use the insecure_rand_ctx exclusively (Pieter Wuille)
8d98d426116f0178612f14d1874d331042c4c4b7 Bugfix: randbytes should seed when needed (non reachable issue) (Pieter Wuille)
273d02580aa736b7ccea8fce51d90541665fdbd1 Use a FastRandomContext in LimitOrphanTxSize (Pieter Wuille)
3db746beb407f7cdd9cd6a605a195bef1254b4c0 Introduce a Shuffle for FastRandomContext and use it in wallet and coinselection (Pieter Wuille)
8098379be5465f598220e1d6174fc57c56f9da42 Use a local FastRandomContext in a few more places in net (Pieter Wuille)
9695f31d7544778853aa373f0aeed629fa68d85e Make addrman use its local RNG exclusively (Pieter Wuille)

Pull request description:

  This improves a few minor issues with the RNG code:
  * Avoid calling `GetRand*()` functions (which currently invoke OpenSSL, later may switch to using our own RNG pool) inside loops in addrman, networking code, `KnapsackSolver`, and `LimitOrphanSize`
  * Fix a currently unreachable bug in `FastRandomContext::randbytes`.
  * Make a number of simplifications to the unit tests' randomness code (some tests unnecessarily used their own RNG or the OpenSSL one, instead of using the unit test specific `insecure_rand_ctx`).
  * As a precaution, make it illegal to copy a `FastRandomContext`.

Tree-SHA512: 084c70b533ea68ca7adc0186c39f0b3e0a5c0ae43a12c37286e5d42086e056a8cd026dde61b12c0a296dc80f87fdc87fe303b9e8e6161b460ac2086cf7615f9d
2021-07-19 19:00:04 -05:00
Pasta
0bbe57d0a0
Avoid redundant protected in CAddrMan 2021-07-19 12:28:24 -05:00
MarcoFalke
b968ebf34c Merge #13115: addrman: Add Clang thread safety annotations for variables guarded by CAddrMan.cs
3e9f6c821b Add missing locks and locking annotations for CAddrMan (practicalswift)

Pull request description:

  * Add Clang thread safety annotations for variables guarded by `CAddrMan.cs `
  * Add missing `CAddrMan.cs ` locks

Tree-SHA512: c78d56d56eb63a4469333c04c95317545a8f97d5e3a36ff2699ee4a91a6433d416221eed6c5ff168e1e31f6936c2ae101a4c60b635f2b2309f40e3d66a727322
2021-07-17 13:51:59 -05:00
Wladimir J. van der Laan
cd3bedaf1c Merge #20557: addrman: Fix new table bucketing during unserialization
4676a4fb5be0f6ef0b3f71c1f4361c20f7cb0e0b [addrman] Don't repeat "Bucketing method was updated" log multiple times (John Newbery)
436292367c1d737cf73bd985293539500d1206f5 [addrman] Improve serialization comments (John Newbery)
ac3547eddd8a7d67b4103508f30d5d02a9c1f148 [addrman] Improve variable naming/code style of touched code. (John Newbery)
a5c9b04959f443372400f9a736c6eaf5502284a1 [addrman] Don't rebucket new table entries unnecessarily (John Newbery)
8062d928ce5c495c1b6ecd18e4b30c12da822d90 [addrman] Rename asmap version to asmap checksum (John Newbery)
009b8e0fdf3bfb11668edacced5d8b70726d5d0e [addrman] Improve variable naming/code style of touched code. (John Newbery)
b4c5fda417dd9ff8bf9fe24a87d384a649e3730d [addrman] Fix new table bucketing during unserialization (John Newbery)

Pull request description:

  This fixes three issues in addrman unserialization.

  1. An addrman entry can appear in up to 8 new table buckets. We store this entry->bucket indexing during shutdown so that on restart we can restore the entries to their correct buckets. Commit ec45646de9e62b3d42c85716bfeb06d8f2b507dc broke the deserialization code so that each entry could only be put in up to one new bucket.

  2. Unserialization may result in an entry appearing in a 9th bucket. If the entry already appears in 8 buckets don't try to place it in another bucket.

  3. We unnecessarily rebucket when reading a peers.dat with file version 1. Don't do that.

ACKs for top commit:
  vasild:
    ACK 4676a4fb5be0f6ef0b3f71c1f4361c20f7cb0e0b
  glozow:
    re-ACK 4676a4fb5b, changes were a rename, comments, and removing repeat-logging.
  naumenkogs:
    ACK 4676a4f
  laanwj:
    Code review ACK 4676a4fb5be0f6ef0b3f71c1f4361c20f7cb0e0b
  dhruv:
    ACK 4676a4fb5be0f6ef0b3f71c1f4361c20f7cb0e0b
  ryanofsky:
    Code review ACK 4676a4fb5be0f6ef0b3f71c1f4361c20f7cb0e0b. I'm not previously familiar with this code but all the changes here do make sense and seem like improvements. Left some notes and comments, but they aren't important so feel to ignore.

Tree-SHA512: b228984f6dec5910be23c3740ae20258da33bcf66ceb7edb10e5a53163450f743bab349e47f09808b7e8d40f27143119ec3e0981d7e678aa494d8559a1c99c23
2021-07-16 10:04:09 -05:00
Kittywhiskers Van Gogh
ef4dfa8524 merge bitcoin#14555: Move util files to directory
(script modified to account for Dash backports, doesn't account for rebasing)

------------- BEGIN SCRIPT ---------------
mkdir -p src/util
git mv src/util.h src/util/system.h
git mv src/util.cpp src/util/system.cpp
git mv src/utilmemory.h src/util/memory.h
git mv src/utilmoneystr.h src/util/moneystr.h
git mv src/utilmoneystr.cpp src/util/moneystr.cpp
git mv src/utilstrencodings.h src/util/strencodings.h
git mv src/utilstrencodings.cpp src/util/strencodings.cpp
git mv src/utiltime.h src/util/time.h
git mv src/utiltime.cpp src/util/time.cpp
git mv src/utilasmap.h src/util/asmap.h
git mv src/utilasmap.cpp src/util/asmap.cpp
git mv src/utilstring.h src/util/string.h
git mv src/utilstring.cpp src/util/string.cpp

gsed -i 's/<util\.h>/<util\/system\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmemory\.h>/<util\/memory\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmoneystr\.h>/<util\/moneystr\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstrencodings\.h>/<util\/strencodings\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utiltime\.h>/<util\/time\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilasmap\.h>/<util\/asmap\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstring\.h>/<util\/string\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')

gsed -i 's/BITCOIN_UTIL_H/BITCOIN_UTIL_SYSTEM_H/g' src/util/system.h
gsed -i 's/BITCOIN_UTILMEMORY_H/BITCOIN_UTIL_MEMORY_H/g' src/util/memory.h
gsed -i 's/BITCOIN_UTILMONEYSTR_H/BITCOIN_UTIL_MONEYSTR_H/g' src/util/moneystr.h
gsed -i 's/BITCOIN_UTILSTRENCODINGS_H/BITCOIN_UTIL_STRENCODINGS_H/g' src/util/strencodings.h
gsed -i 's/BITCOIN_UTILTIME_H/BITCOIN_UTIL_TIME_H/g' src/util/time.h
gsed -i 's/BITCOIN_UTILASMAP_H/BITCOIN_UTIL_ASMAP_H/g' src/util/asmap.h
gsed -i 's/BITCOIN_UTILSTRING_H/BITCOIN_UTIL_STRING_H/g' src/util/string.h

gsed -i 's/ util\.\(h\|cpp\)/ util\/system\.\1/g' src/Makefile.am
gsed -i 's/utilmemory\.\(h\|cpp\)/util\/memory\.\1/g' src/Makefile.am
gsed -i 's/utilmoneystr\.\(h\|cpp\)/util\/moneystr\.\1/g' src/Makefile.am
gsed -i 's/utilstrencodings\.\(h\|cpp\)/util\/strencodings\.\1/g' src/Makefile.am
gsed -i 's/utiltime\.\(h\|cpp\)/util\/time\.\1/g' src/Makefile.am
gsed -i 's/utilasmap\.\(h\|cpp\)/util\/asmap\.\1/g' src/Makefile.am
gsed -i 's/utilstring\.\(h\|cpp\)/util\/string\.\1/g' src/Makefile.am

gsed -i 's/-> util ->/-> util\/system ->/' test/lint/lint-circular-dependencies.sh
gsed -i 's/src\/util\.cpp/src\/util\/system\.cpp/g' test/lint/lint-format-strings.py test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilmoneystr\.cpp/src\/util\/moneystr\.cpp/g' test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilstrencodings\.\(h\|cpp\)/src\/util\/strencodings\.\1/g' test/lint/lint-locale-dependence.sh
------------- END   SCRIPT ---------------
2021-06-27 12:03:13 +05:30
Wladimir J. van der Laan
fff8f97917
Merge #15486: [addrman, net] Ensure tried collisions resolve, and allow feeler connections to existing outbound netgroups
20e6ea259b222b10f066f22695a5f56c52071f63 [addrman] Improve collision logging and address nits (Suhas Daftuar)
f71fdda3bc2e7acd2a8b74e882364866b8b0f55b [addrman] Ensure collisions eventually get resolved (Suhas Daftuar)
4991e3c813c9848d3b3957ea3ad433f02fca9e81 [net] feeler connections can be made to outbound peers in same netgroup (Suhas Daftuar)
4d834018e368c3481a5421891395f64aa9002185 [addrman] Improve tried table collision logging (Suhas Daftuar)

Pull request description:

  The restriction on outbound peers sharing the same network group is not intended to apply to feeler connections, so fix this.

  This fixes an issue where a tried table collision with an entry to a netgroup we already have an outbound connection to could cause feelers to stop working, because the tried collision buffer (`m_tried_collisions`) would never be drained.

  Also, ensure that all entries don't linger in `m_tried_collisions` by evicting an old entry if its collisions is unresolved after 40 minutes.

Tree-SHA512: 553fe2b01b82cd7f0f62f90c6781e373455a45b254e3bec085b5e6b16690aa9f3938e8c50e7136f19dafa250ed4578a26227d944b76daf9ce4ef0c75802389b6
2021-06-26 11:23:42 -05:00
Kittywhiskers Van Gogh
680067ce7a
merge #19954: Complete the BIP155 implementation and upgrade to TORv3 2021-05-29 23:24:52 +03:00
Kittywhiskers Van Gogh
401098283e Merge #17812: asmap feature refinements and functional tests 2021-05-20 00:05:12 +05:30
Kittywhiskers Van Gogh
4b2b5f78d4 Merge #16702: supplying and using asmap to improve IP bucketing 2021-05-20 00:05:09 +05:30
Kittywhiskers Van Gogh
e483a689e3
Merge #17850: Convert addrdb/addrman to new serialization
9250a087d2

Partial #17850: Introduce new serialization macros without casts

ca33451535
2021-04-18 17:02:40 -04:00
MarcoFalke
ff7f5d5dea
Merge #12747: Fix typos
d27327c79a Fix typos (practicalswift)

Pull request description:

  Fix typos.

Tree-SHA512: f0d13d991acdec0d3adc2f091cd00ccbdda6da3c7623dfb4cbf698bac9eb6b3d88c8ad121256a96cb130f8e97bf54892f3616da0e8dc833dcf713ca7949e2801
2020-12-15 21:49:46 -06:00
pasta
c49e506176 scripted-diff: Merge #13753: Remove trailing whitespaces
-BEGIN VERIFY SCRIPT-

sed --in-place'' --regexp-extended 's/[[:space:]]+$//g' $(git grep -I --files-with-matches --extended-regexp '[[:space:]]+$' -- src test  ':!*.svg' ':!src/crypto/sha256_sse4*' ':!src/leveldb' ':!src/qt/locale' ':!src/secp256k1' ':!src/univalue')

-END VERIFY SCRIPT-

Signed-off-by: pasta <pasta@dashboost.org>
2020-07-28 20:23:12 -05:00
Wladimir J. van der Laan
8b9012d292 Merge #9037: net: Add test-before-evict discipline to addrman
e68172ed9 Add test-before-evict discipline to addrman (Ethan Heilman)

Pull request description:

  This change implement countermeasures 3 (test-before-evict) suggested in our paper: ["Eclipse Attacks on Bitcoin’s Peer-to-Peer Network"](http://cs-people.bu.edu/heilman/eclipse/).
  # Design:

  A collision occurs when an address, addr1, is being moved to the tried table from the new table, but maps to a position in the tried table which already contains an address (addr2). The current behavior is that addr1 would evict addr2 from the tried table.

  This change ensures that during a collision, addr1 is not inserted into tried but instead inserted into a buffer (setTriedCollisions). The to-be-evicted address, addr2, is then tested by [a feeler connection](https://github.com/bitcoin/bitcoin/pull/8282). If addr2 is found to be online, we remove addr1 from the buffer and addr2 is not evicted, on the other hand if addr2 is found be offline it is replaced by addr1.

  An additional small advantage of this change is that, as no more than ten addresses can be in the test buffer at once, and addresses are only cleared one at a time from the test buffer (at 2 minute intervals), thus an attacker is forced to wait at least two minutes to insert a new address into tried after filling up the test buffer. This rate limits an attacker attempting to launch an eclipse attack.
  # Risk mitigation:
  - To prevent this functionality from being used as a DoS vector, we limit the number of addresses which are to be tested to ten. If we have more than ten addresses to test, we drop new addresses being added to tried if they would evict an address. Since the feeler thread only creates one new connection every 2 minutes the additional network overhead is limited.
  - An address in tried gains immunity from tests for 4 hours after it has been tested or successfully connected to.
  # Tests:

  This change includes additional addrman unittests which test this behavior.

  I ran an instance of this change with a much smaller tried table (2 buckets of 64 addresses) so that collisions were much more likely and observed evictions.

  ```
  2016-10-27 07:20:26 Swapping 208.12.64.252:8333 for 68.62.95.247:8333 in tried table
  2016-10-27 07:20:26 Moving 208.12.64.252:8333 to tried
  ```

  I documented tests we ran against similar earlier versions of this change in #6355.
  # Security Benefit

  This is was originally posted in PR #8282 see [this comment for full details](https://github.com/bitcoin/bitcoin/pull/8282#issuecomment-237255215).

  To determine the security benefit of these larger numbers of IPs in the tried table I modeled the attack presented in [Eclipse Attacks on Bitcoin’s Peer-to-Peer Network](https://eprint.iacr.org/2015/263).

  ![attackergraph40000-10-1000short-line](https://cloud.githubusercontent.com/assets/274814/17366828/372af458-595b-11e6-81e5-2c9f97282305.png)

  **Default node:** 595 attacker IPs for ~50% attack success.
  **Default node + test-before-evict:** 620 attacker IPs for ~50% attack success.
  **Feeler node:** 5540 attacker IPs for ~50% attack success.
  **Feeler node + test-before-evict:** 8600 attacker IPs for ~50% attack success.

  The node running feeler connections has 10 times as many online IP addresses in its tried table making an attack 10 times harder (i.e. requiring the an attacker require 10 times as many IP addresses in different /16s). Adding test-before-evict increases resistance of the node by an additional 3000 attacker IP addresses.

  Below I graph the attack over even greater attacker resources (i.e. more attacker controled IP addresses). Note that test-before-evict maintains some security far longer even against an attacker with 50,000 IPs. If this node had a larger tried table test-before-evict could greatly boost a nodes resistance to eclipse attacks.

  ![attacker graph long view](https://cloud.githubusercontent.com/assets/274814/17367108/96f46d64-595c-11e6-91cd-edba160598e7.png)

Tree-SHA512: fdad4d26aadeaad9bcdc71929b3eb4e1f855b3ee3541fbfbe25dca8d7d0a1667815402db0cb4319db6bd3fcd32d67b5bbc0e12045c4252d62d6239b7d77c4395
2020-06-13 14:50:11 -05:00
Wladimir J. van der Laan
96e2c5a25f
Merge #12731: Support serialization as another type without casting
818dc74 Support serialization as another type without casting (Pieter Wuille)

Pull request description:

  This adds a `READWRITEAS(type, obj)` macro which serializes `obj` as if it were converted to `const type&` when `const`, and to `type&` when non-`const`. No actual cast is involved, so this only works when this conversion can be done automatically.

  This makes it usable in serialization code that uses a single implementation for both serialization and deserializing, which doesn't know the constness of the object involved.

  This is a redo of #12712, using a slightly different interface.

Tree-SHA512: 262f0257284ff99b5ffaec9b997c194e221522ba35c3ac8eaa9bb344449d7ea0a314de254dc77449fa7aaa600f8cd9a24da65aade8c1ec6aa80c6e9a7bba5ca7
2020-06-11 23:20:48 -05:00
MarcoFalke
26ca61ec21
Merge #10498: Use static_cast instead of C-style casts for non-fundamental types
9ad6746ccd Use static_cast instead of C-style casts for non-fundamental types (practicalswift)

Pull request description:

  A C-style cast is equivalent to try casting in the following order:

  1. `const_cast(...)`
  2. `static_cast(...)`
  3. `const_cast(static_cast(...))`
  4. `reinterpret_cast(...)`
  5. `const_cast(reinterpret_cast(...))`

  By using `static_cast<T>(...)` explicitly we avoid the possibility of an unintentional and dangerous `reinterpret_cast`. Furthermore `static_cast<T>(...)` allows for easier grepping of casts.

  For a more thorough discussion, see ["ES.49: If you must use a cast, use a named cast"](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#es49-if-you-must-use-a-cast-use-a-named-cast) in the C++ Core Guidelines (Stroustrup & Sutter).

Tree-SHA512: bd6349b7ea157da93a47b8cf238932af5dff84731374ccfd69b9f732fabdad1f9b1cdfca67497040f14eaa85346391404f4c0495e22c467f26ca883cd2de4d3c
2020-06-10 18:32:43 -05:00
MarcoFalke
a98db86ada Merge #10493: Use range-based for loops (C++11) when looping over map elements
680bc2cbb Use range-based for loops (C++11) when looping over map elements (practicalswift)

Pull request description:

  Before this commit:

  ```c++
  for (std::map<T1, T2>::iterator x = y.begin(); x != y.end(); ++x) {
      T1 z = (*x).first;
      …
  }
  ```

  After this commit:

  ```c++
  for (auto& x : y) {
      T1 z = x.first;
      …
  }
  ```

Tree-SHA512: 954b136b7f5e6df09f39248a6b530fd9baa9ab59d7c2c7eb369fd4afbb591b7a52c92ee25f87f1745f47b41d6828b7abfd395b43daf84a55b4e6a3d45015e3a0
2020-04-01 12:43:18 -05:00
PastaPastaPasta
8a1ec935a0
Backport 11651 (#3358)
* scripted-diff: Replace #include "" with #include <> (ryanofsky)

-BEGIN VERIFY SCRIPT-
for f in \
  src/*.cpp \
  src/*.h \
  src/bench/*.cpp \
  src/bench/*.h \
  src/compat/*.cpp \
  src/compat/*.h \
  src/consensus/*.cpp \
  src/consensus/*.h \
  src/crypto/*.cpp \
  src/crypto/*.h \
  src/crypto/ctaes/*.h \
  src/policy/*.cpp \
  src/policy/*.h \
  src/primitives/*.cpp \
  src/primitives/*.h \
  src/qt/*.cpp \
  src/qt/*.h \
  src/qt/test/*.cpp \
  src/qt/test/*.h \
  src/rpc/*.cpp \
  src/rpc/*.h \
  src/script/*.cpp \
  src/script/*.h \
  src/support/*.cpp \
  src/support/*.h \
  src/support/allocators/*.h \
  src/test/*.cpp \
  src/test/*.h \
  src/wallet/*.cpp \
  src/wallet/*.h \
  src/wallet/test/*.cpp \
  src/wallet/test/*.h \
  src/zmq/*.cpp \
  src/zmq/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* scripted-diff: Replace #include "" with #include <> (Dash Specific)

-BEGIN VERIFY SCRIPT-
for f in \
  src/bls/*.cpp \
  src/bls/*.h \
  src/evo/*.cpp \
  src/evo/*.h \
  src/governance/*.cpp \
  src/governance/*.h \
  src/llmq/*.cpp \
  src/llmq/*.h \
  src/masternode/*.cpp \
  src/masternode/*.h \
  src/privatesend/*.cpp \
  src/privatesend/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* build: Remove -I for everything but project root

Remove -I from build system for everything but the project root,
and built-in dependencies.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/Makefile.test.include

* qt: refactor: Use absolute include paths in .ui files

* qt: refactor: Changes to make include paths absolute

This makes all include paths in the GUI absolute.

Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/qt/dash.cpp
#	src/qt/optionsmodel.cpp
#	src/qt/test/rpcnestedtests.cpp

* test: refactor: Use absolute include paths for test data files

* Recommend #include<> syntax in developer notes

* refactor: Include obj/build.h instead of build.h

* END BACKPORT #11651 Remove trailing whitespace causing travis failure

* fix backport 11651

Signed-off-by: Pasta <pasta@dashboost.org>

* More of 11651

* fix blockchain.cpp

Signed-off-by: pasta <pasta@dashboost.org>

* Add missing "qt/" in includes

* Add missing "test/" in includes

* Fix trailing whitespaces

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-20 01:46:56 +03:00
MarcoFalke
e1cbdef16c
Merge #11585: addrman: Add missing lock in Clear() (CAddrMan)
3ab545d7f addrman: Add missing lock in Clear() (CAddrMan) (practicalswift)

Pull request description:

  Add missing lock in `Clear()` (`CAddrMan`).

  The variable `vRandom` is guarded by the mutex `cs`.

  **Note to reviewers:** Does this look correct? Should the lock cover the entire scope of the method, or should it be limited to cover only `std::vector<int>().swap(vRandom);`?

Tree-SHA512: 8833f31beaed1728fa55b13ddf9e0b8e24e395931497329be2440ce1c5113ff02871707d40830260adabd30c4ea86088f5da5cf8a821150c0d820f50a2ce386a
2020-01-16 09:22:46 -06:00
Alexander Block
5107582f25
Merge #11252: [P2P] When clearing addrman clear mapInfo and mapAddr. (#3171)
b86a42077 when clearing addrman clear mapInfo and mapAddr (Gregory Sanders)

Pull request description:

  Power failure on my machine resulted in a corrupted addrman that would hit bad assertions when trying to serialize the "cleared" addrman to disk: 6866b4912b/src/addrman.h (L320)

Tree-SHA512: 07ca8b6cbd88407e5f3f0dccb346ae31bd1392f4210b2d5c5647c853986bfec95cf70240b92bafdc61b90e452a5d8315962738d10c10c2b53fdabff10503d05a
2019-10-23 09:56:30 +02:00
Pasta
1a0d52814e
#10483 scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL
-BEGIN VERIFY SCRIPT-
sed -i 's/\<NULL\>/nullptr/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h src/qt/*/*.cpp src/qt/*/*.h src/wallet/*/*.cpp src/wallet/*/*.h src/support/allocators/*.h
sed -i 's/Prefer nullptr, otherwise SAFECOOKIE./Prefer NULL, otherwise SAFECOOKIE./g' src/torcontrol.cpp
sed -i 's/tor: Using nullptr authentication/tor: Using NULL authentication/g' src/torcontrol.cpp
sed -i 's/METHODS=nullptr/METHODS=NULL/g' src/test/torcontrol_tests.cpp src/torcontrol.cpp
sed -i 's/nullptr certificates/NULL certificates/g' src/qt/paymentserver.cpp
sed -i 's/"nullptr"/"NULL"/g' src/torcontrol.cpp src/test/torcontrol_tests.cpp
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>
2019-08-12 09:07:03 -05:00
UdjinM6
c58184f6c4
Merge pull request #2974 from PastaPastaPasta/backports-0.15-pr14
Backports 0.15 pr14
2019-06-19 14:30:54 +03:00
Alexander Block
e035387783 Back off for 1m when connecting to quorum masternodes (#2975)
* Implement GetAddressInfo in CAddrMan

* Back off for 1m when connecting to quorum masternodes
2019-06-14 14:04:19 +03:00
Wladimir J. van der Laan
198558a362
Merge #9792: FastRandomContext improvements and switch to ChaCha20
4fd2d2f Add a FastRandomContext::randrange and use it (Pieter Wuille)
1632922 Switch FastRandomContext to ChaCha20 (Pieter Wuille)
e04326f Add ChaCha20 (Pieter Wuille)
663fbae FastRandom benchmark (Pieter Wuille)
c21cbe6 Introduce FastRandomContext::randbool() (Pieter Wuille)

Tree-SHA512: 7fff61e3f6d6dc6ac846ca643d877b377db609646dd401a0e8f50b052c6b9bcd2f5fc34de6bbf28f04afd1724f6279ee163ead5f37d724fb782a00239f35db1d
2019-06-14 01:25:59 -05:00
PastaPastaPasta
29194b1f5a Backport Bitcoin#9424, Bitcoin#10123 and Bitcoin#10153 (#2918)
* Contains dashification. disables `-debug dash`
Merge #9424: Change LogAcceptCategory to use uint32_t rather than sets of strings.

6b3bb3d Change LogAcceptCategory to use uint32_t rather than sets of strings. (Gregory Maxwell)

Tree-SHA512: ebb5bcf9a7d00a32dd1390b727ff4d29330a038423611da01268d8e1d2c0229e52a1098e751d4e6db73ef4ae862e1e96d38249883fcaf12b68f55ebb01035b34
Signed-off-by: Pasta <Pasta@dash.org>

31 -> 32

Signed-off-by: Pasta <Pasta@dash.org>

* Merge #10123: Allow debug logs to be excluded from specified component

3bde556 Add -debugexclude option to switch off logging for specified components (John Newbery)

Tree-SHA512: 30202e3f2085fc2fc5dd4bedb92988f4cb162c612a42cf8f6395a7da326f34975ddc347f82bc4ddca6c84c438dc0cc6e87869f90c7ff88105dbeaa52a947fa43

* bump to uint64_t due to added Dash codes

Signed-off-by: Pasta <Pasta@dash.org>

* bump to uint64_t due to added Dash codes cont.

Signed-off-by: Pasta <Pasta@dash.org>

* string -> BCLog format

Signed-off-by: Pasta <Pasta@dash.org>

* uint32_t -> uint64_t

Signed-off-by: Pasta <Pasta@dash.org>

* Fix CBatchedLogger

* Fix most fDebug-s

* Fix `debug` rpc

* Fix BENCH and RAND conflicts

* Add ALERT and use it

* Update LogPrint-s in dash-specific code

* Tweak few log categories

Specifically:
- use PRIVATESEND in `CPrivateSendClientManager::GetRandomNotUsedMasternode()`
- use ZMQ in `CZMQPublishRawGovernanceVoteNotifier::NotifyGovernanceVote()` and `CZMQPublishRawGovernanceObjectNotifier::NotifyGovernanceObject()`

* Drop no longer used MASTERNODE category

* Merge #10153: logging: Fix off-by-one for shrinkdebugfile default

faab624 logging: Fix off-by-one for shrinkdebugfile (MarcoFalke)

Tree-SHA512: d6153e06067906172ff0611af9e585a3ecf0a7d56925b6ad7c12e75aa802441047059b9b6f6c78e79916c3f2abc8f1998bfd2d5b84201ec6421f727c08da3c21

* Shift dash-specific log categories to start from `1ul << 32` to avoid potential future conflicts with bitcoin ones

* Fix `dash` category

* remove debugCategories

Signed-off-by: Pasta <Pasta@dash.org>

* Prepend "std::" to find call

* Check for BCLog::PRIVATESEND instead of logCategories != BCLog::NONE

* Use BCLog::MNPAYMENTS category instead of checking for logCategories != BCLog::NONE

* Move "End Dash" comment below "ALERT"

When adding new entries here, we'll otherwise get confused with ordering
and might end up forgetting that adding something Dash specific must
continue with the bit after 43.
2019-05-23 00:51:39 +03:00
Alexander Block
2c303cdb11 A few devnet related fixes (#2168)
* Remove testnet seeds from devnet

* Lift multiple ports restriction on devnet when considering new nodes

Allow to connect to multiple nodes behind the same IP

* Don't skip addresses with non-default port if it matches -port

If the user specified -port, he very likely intends to connect to nodes
with the same port.

* Don't pass false to CAddrMan constructor as it is already the default

* Make if statements easier to read
2018-07-08 00:19:33 +03:00
Wladimir J. van der Laan
ab9bce2c3a Merge #9225: Fix some benign races
dfed983 Fix unlocked access to vNodes.size() (Matt Corallo)
3033522 Remove double brackets in addrman (Matt Corallo)
dbfaade Fix AddrMan locking (Matt Corallo)
047ea10 Make fImporting an std::atomic (Matt Corallo)
42071ca Make fDisconnect an std::atomic (Matt Corallo)
2018-01-17 17:27:24 +01:00
Wladimir J. van der Laan
19a2d668cf Merge #9039: Various serialization simplifcations and optimizations
d59a518 Use fixed preallocation instead of costly GetSerializeSize (Pieter Wuille)
25a211a Add optimized CSizeComputer serializers (Pieter Wuille)
a2929a2 Make CSerAction's ForRead() constexpr (Pieter Wuille)
a603925 Avoid -Wshadow errors (Pieter Wuille)
5284721 Get rid of nType and nVersion (Pieter Wuille)
657e05a Make GetSerializeSize a wrapper on top of CSizeComputer (Pieter Wuille)
fad9b66 Make nType and nVersion private and sometimes const (Pieter Wuille)
c2c5d42 Make streams' read and write return void (Pieter Wuille)
50e8a9c Remove unused ReadVersion and WriteVersion (Pieter Wuille)
2018-01-15 06:14:15 +01:00
Wladimir J. van der Laan
239ce534c2 Merge #8914: Kill insecure_random and associated global state
5eaaa83 Kill insecure_random and associated global state (Wladimir J. van der Laan)
2018-01-12 09:58:14 +01:00
Wladimir J. van der Laan
00c84ca53e Merge #8065: Addrman offline attempts
6182d10 Do not increment nAttempts by more than one for every Good connection. (Gregory Maxwell)
c769c4a Avoid counting failed connect attempts when probably offline. (Gregory Maxwell)
2017-12-22 16:33:55 +01:00
Wladimir J. van der Laan
0450dfdeb8 Merge #7932: CAddrMan::Deserialize handle corrupt serializations better.
fb26bf0 CAddrMan::Deserialize handle corrupt serializations better. (Patrick Strateman)
2017-12-21 18:50:13 +01:00
Wladimir J. van der Laan
5323a08afa Merge #7300: [trivial] Add missing copyright headers
fabcee1 Remove copyright header from autogenerated chainparamsseeds.h (MarcoFalke)
fa60d05 Add missing copyright headers (MarcoFalke)
fa7e4c0 Bump copyright headers to 2014 (MarcoFalke)
2017-12-11 08:08:48 +01:00