sikkienl/dash
1
0
Fork 0
mirror of https://github.com/dashpay/dash.git synced 2024-12-24 19:42:46 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge #6043: backport: merge bitcoin#22879, #22762, #23041, #22734, #22950, #23053, #22839, #23140, #23306, #23354, #23380 (addrman backports: part 2)

Browse Source 
a93fec6f2d merge bitcoin#23380: Fix AddrMan::Add() return semantics and logging (Kittywhiskers Van Gogh)
d1a4b14b48 merge bitcoin#23354: Introduce new V4 format addrman (Kittywhiskers Van Gogh)
7a97aabfe0 test: restore pre-bitcoin#23306 tests to validate port distinguishment (Kittywhiskers Van Gogh)
1a050d6cb4 merge bitcoin#23306: Make AddrMan support multiple ports per IP (Kittywhiskers Van Gogh)
d56702aa0c merge bitcoin#23140: Make CAddrman::Select_ select buckets, not positions, first (Kittywhiskers Van Gogh)
19b0145379 merge bitcoin#22839: improve addrman logging (Kittywhiskers Van Gogh)
3910c68028 merge bitcoin#23053: Use public methods in addrman fuzz tests (Kittywhiskers Van Gogh)
b6ec8ab6df merge bitcoin#22950: Pimpl AddrMan to abstract implementation details (Kittywhiskers Van Gogh)
236cf36d88 merge bitcoin#22734: Avoid crash on corrupt data, Force Check after deserialize (Kittywhiskers Van Gogh)
2420ac9e53 merge bitcoin#23041: Add addrman deserialization error tests (Kittywhiskers Van Gogh)
8aefa9b93a merge bitcoin#22762: Raise InitError when peers.dat is invalid or corrupted (Kittywhiskers Van Gogh)
c9a645f814 merge bitcoin#22879: Fix format string in deserialize error (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Dependent on https://github.com/dashpay/dash/pull/6040.
  * Dash already introduced support for storage of address-port pairs (referred to as "port discrimination") and allowed the usage of non-default ports in P2P with [dash#2168](https://github.com/dashpay/dash/pull/2168).
    * Albeit this was only permitted for networks with `fAllowMultiplePorts` enabled (which at the time was `devnet` and as it stands currently, on every network except `mainnet`).
  * Keeping in line with the above policy (discussion on lifting such restrictions on `mainnet` is outside the scope of this PR), when backporting [bitcoin#23306](https://github.com/bitcoin/bitcoin/pull/23306), changes have been made to retain the effects of `discriminate_ports`.
    * This involves appending placing a `!m_discriminate_ports` condition to behaviour that otherwise would be _removed_ entirely.
    * Additionally, in line with upstream backports, changes have been made that render port distinguishment _enabled_ as the new default in `addrman_tests` (the old default was to keep it _disabled_, to mirror `mainnet` and pre-change upstream behaviour).
      * To ensure distinguishment _disabled_ works as expected, affected pre-backport tests were reintroduced with the `_nondiscriminate` suffix.

  ---

  I would propose at some point to rename the flag to `ignore_port`/`suppress_port` (if not remove it altogether should the `mainnet` restriction be lifted) as discriminate (or distinguish) isn't immediately clear with if address entries will be discriminated/distinguished _using_ ports (i.e. considered) or ports will be discriminated _against_ (i.e. ignored).

  ## Breaking Changes

  It's unclear if these backports result in serialization issues for older versions, as Dash Core technically supported address-port pairs since 0.12 and suppressed it on `mainnet` by setting zero-ing out the port ([source](19512988c6/src/addrman.cpp (L135-L138))), meaning even with port discrimination _disabled_, the serialization format should remain the same.

  Regardless, following upstream backports, a new version has been introduced (v4) that marks the AddrMan format incompatible with older versions of Dash Core.

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    utACK a93fec6f2d
  PastaPastaPasta:
    utACK a93fec6f2d

Tree-SHA512: 49b35af3e4eb660249ce9a65d5a539205d852e9c728da22dc88779f6b3b15c13cf91522896a313bfe2a91889fedf3b6b2cebdea12cc2bbe865ec3b85b6a5dfa8
This commit is contained in:
pasta 2024-06-10 23:35:36 -05:00
commit 74fcd026db
No known key found for this signature in database
GPG Key ID: 52527BEDABE87984
28 changed files with 1444 additions and 1042 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
src/Makefile.am
View File

@ -133,6 +133,7 @@ BITCOIN_CORE_H = \
addressindex.h \
spentindex.h \
addrman.h \
addrman_impl.h \
attributes.h \
banman.h \
base58.h \

1
src/Makefile.test.include
View File

@ -263,7 +263,6 @@ test_fuzz_fuzz_SOURCES = \
test/fuzz/crypto_hkdf_hmac_sha256_l32.cpp \
test/fuzz/crypto_poly1305.cpp \
test/fuzz/cuckoocache.cpp \
test/fuzz/data_stream.cpp \
test/fuzz/decode_tx.cpp \
test/fuzz/descriptor_parse.cpp \
test/fuzz/deserialize.cpp \

64
src/addrdb.cpp
View File

@ -17,10 +17,17 @@
#include <univalue.h>
#include <util/settings.h>
#include <util/system.h>
#include <util/translation.h>
#include <cstdint>
namespace {
class DbNotFoundError : public std::exception
{
using std::exception::exception;
};
template <typename Stream, typename Data>
bool SerializeDB(Stream& stream, const Data& data)
{
@ -78,16 +85,16 @@ bool SerializeFileDB(const std::string& prefix, const fs::path& path, const Data
}
template <typename Stream, typename Data>
bool DeserializeDB(Stream& stream, Data& data, bool fCheckSum = true)
void DeserializeDB(Stream& stream, Data& data, bool fCheckSum = true)
{
try {
CHashVerifier<Stream> verifier(&stream);
// de-serialize file header (network specific magic number) and ..
unsigned char pchMsgTmp[4];
verifier >> pchMsgTmp;
// ... verify the network matches ours
if (memcmp(pchMsgTmp, Params().MessageStart(), sizeof(pchMsgTmp)))
return error("%s: Invalid network magic number", __func__);
if (memcmp(pchMsgTmp, Params().MessageStart(), sizeof(pchMsgTmp))) {
throw std::runtime_error{"Invalid network magic number"};
}
// de-serialize data
verifier >> data;
@ -97,28 +104,21 @@ bool DeserializeDB(Stream& stream, Data& data, bool fCheckSum = true)
uint256 hashTmp;
stream >> hashTmp;
if (hashTmp != verifier.GetHash()) {
return error("%s: Checksum mismatch, data corrupted", __func__);
throw std::runtime_error{"Checksum mismatch, data corrupted"};
}
}
}
catch (const std::exception& e) {
return error("%s: Deserialize or I/O error - %s", __func__, e.what());
}
return true;
}
template <typename Data>
bool DeserializeFileDB(const fs::path& path, Data& data, int version)
void DeserializeFileDB(const fs::path& path, Data& data, int version)
{
// open input file, and associate with CAutoFile
FILE* file = fsbridge::fopen(path, "rb");
CAutoFile filein(file, SER_DISK, version);
if (filein.IsNull()) {
LogPrintf("Missing or invalid file %s\n", path.string());
return false;
throw DbNotFoundError{};
}
return DeserializeDB(filein, data);
DeserializeDB(filein, data);
}
} // namespace
@ -171,21 +171,38 @@ bool CBanDB::Read(banmap_t& banSet)
return true;
}
bool DumpPeerAddresses(const ArgsManager& args, const CAddrMan& addr)
bool DumpPeerAddresses(const ArgsManager& args, const AddrMan& addr)
{
const auto pathAddr = GetDataDir() / "peers.dat";
return SerializeFileDB("peers", pathAddr, addr, CLIENT_VERSION);
}
bool ReadPeerAddresses(const ArgsManager& args, CAddrMan& addr)
void ReadFromStream(AddrMan& addr, CDataStream& ssPeers)
{
const auto pathAddr = GetDataDir() / "peers.dat";
return DeserializeFileDB(pathAddr, addr, CLIENT_VERSION);
DeserializeDB(ssPeers, addr, false);
}
bool ReadFromStream(CAddrMan& addr, CDataStream& ssPeers)
std::optional<bilingual_str> LoadAddrman(const std::vector<bool>& asmap, const ArgsManager& args, std::unique_ptr<AddrMan>& addrman)
{
return DeserializeDB(ssPeers, addr, false);
auto check_addrman = std::clamp<int32_t>(args.GetArg("-checkaddrman", DEFAULT_ADDRMAN_CONSISTENCY_CHECKS), 0, 1000000);
addrman = std::make_unique<AddrMan>(asmap, /* deterministic */ false, /* consistency_check_ratio */ check_addrman);
int64_t nStart = GetTimeMillis();
const auto path_addr{GetDataDir() / "peers.dat"};
try {
DeserializeFileDB(path_addr, *addrman, CLIENT_VERSION);
LogPrintf("Loaded %i addresses from peers.dat %dms\n", addrman->size(), GetTimeMillis() - nStart);
} catch (const DbNotFoundError&) {
// Addrman can be in an inconsistent state after failure, reset it
addrman = std::make_unique<AddrMan>(asmap, /* deterministic */ false, /* consistency_check_ratio */ check_addrman);
LogPrintf("Creating peers.dat because the file was not found (%s)\n", path_addr);
DumpPeerAddresses(args, *addrman);
} catch (const std::exception& e) {
addrman = nullptr;
return strprintf(_("Invalid or corrupt peers.dat (%s). If you believe this is a bug, please report it to %s. As a workaround, you can move the file (%s) out of the way (rename, move, or delete) to have a new one created on the next start."),
e.what(), PACKAGE_BUGREPORT, path_addr);
}
return std::nullopt;
}
void DumpAnchors(const fs::path& anchors_db_path, const std::vector<CAddress>& anchors)
@ -197,9 +214,10 @@ void DumpAnchors(const fs::path& anchors_db_path, const std::vector<CAddress>& a
std::vector<CAddress> ReadAnchors(const fs::path& anchors_db_path)
{
std::vector<CAddress> anchors;
if (DeserializeFileDB(anchors_db_path, anchors, CLIENT_VERSION | ADDRV2_FORMAT)) {
try {
DeserializeFileDB(anchors_db_path, anchors, CLIENT_VERSION | ADDRV2_FORMAT);
LogPrintf("Loaded %i addresses from %s\n", anchors.size(), anchors_db_path.filename());
} else {
} catch (const std::exception&) {
anchors.clear();
}

12
src/addrdb.h
View File

@ -10,17 +10,18 @@
#include <net_types.h> // For banmap_t
#include <univalue.h>
#include <optional>
#include <vector>
class ArgsManager;
class CAddrMan;
class AddrMan;
class CAddress;
class CDataStream;
struct bilingual_str;
bool DumpPeerAddresses(const ArgsManager& args, const CAddrMan& addr);
bool ReadPeerAddresses(const ArgsManager& args, CAddrMan& addr);
bool DumpPeerAddresses(const ArgsManager& args, const AddrMan& addr);
/** Only used by tests. */
bool ReadFromStream(CAddrMan& addr, CDataStream& ssPeers);
void ReadFromStream(AddrMan& addr, CDataStream& ssPeers);
/** Access to the banlist database (banlist.json) */
class CBanDB
@ -46,6 +47,9 @@ public:
bool Read(banmap_t& banSet);
};
/** Returns an error string on failure */
std::optional<bilingual_str> LoadAddrman(const std::vector<bool>& asmap, const ArgsManager& args, std::unique_ptr<AddrMan>& addrman);
/**
* Dump the anchor IP address database (anchors.dat)
*

862
src/addrman.cpp
View File

File diff suppressed because it is too large Load Diff

415
src/addrman.h
View File

@ -6,94 +6,23 @@
#ifndef BITCOIN_ADDRMAN_H
#define BITCOIN_ADDRMAN_H
#include <fs.h>
#include <logging.h>
#include <netaddress.h>
#include <protocol.h>
#include <sync.h>
#include <streams.h>
#include <timedata.h>
#include <cstdint>
#include <memory>
#include <optional>
#include <set>
#include <unordered_map>
#include <utility>
#include <vector>
class AddrInfo;
class AddrManImpl;
/** Default for -checkaddrman */
static constexpr int32_t DEFAULT_ADDRMAN_CONSISTENCY_CHECKS{0};
/**
* Extended statistics about a CAddress
*/
class CAddrInfo : public CAddress
{
public:
//! last try whatsoever by us (memory only)
int64_t nLastTry{0};
//! last counted attempt (memory only)
int64_t nLastCountAttempt{0};
private:
//! where knowledge about this address first came from
CNetAddr source;
//! last successful connection by us
int64_t nLastSuccess{0};
//! connection attempts since last successful attempt
int nAttempts{0};
//! reference count in new sets (memory only)
int nRefCount{0};
//! in tried set? (memory only)
bool fInTried{false};
//! position in vRandom
mutable int nRandomPos{-1};
friend class CAddrMan;
friend class CAddrManDeterministic;
public:
SERIALIZE_METHODS(CAddrInfo, obj)
{
READWRITEAS(CAddress, obj);
READWRITE(obj.source, obj.nLastSuccess, obj.nAttempts);
}
CAddrInfo(const CAddress &addrIn, const CNetAddr &addrSource) : CAddress(addrIn), source(addrSource)
{
}
CAddrInfo() : CAddress(), source()
{
}
//! Calculate in which "tried" bucket this entry belongs
int GetTriedBucket(const uint256 &nKey, const std::vector<bool> &asmap) const;
//! Calculate in which "new" bucket this entry belongs, given a certain source
int GetNewBucket(const uint256 &nKey, const CNetAddr& src, const std::vector<bool> &asmap) const;
//! Calculate in which "new" bucket this entry belongs, using its default source
int GetNewBucket(const uint256 &nKey, const std::vector<bool> &asmap) const
{
return GetNewBucket(nKey, source, asmap);
}
//! Calculate in which position of a bucket to store this entry.
int GetBucketPosition(const uint256 &nKey, bool fNew, int nBucket) const;
//! Determine whether the statistics about this entry are bad enough so that it can just be deleted
bool IsTerrible(int64_t nNow = GetAdjustedTime()) const;
//! Calculate the relative chance this entry should be given when selecting nodes to connect to
double GetChance(int64_t nNow = GetAdjustedTime()) const;
};
/** Stochastic address manager
*
* Design goals:
@ -123,115 +52,61 @@ public:
* * Several indexes are kept for high performance. Setting m_consistency_check_ratio with the -checkaddrman
* configuration option will introduce (expensive) consistency checks for the entire data structure.
*/
/** Total number of buckets for tried addresses */
static constexpr int32_t ADDRMAN_TRIED_BUCKET_COUNT_LOG2{8};
static constexpr int ADDRMAN_TRIED_BUCKET_COUNT{1 << ADDRMAN_TRIED_BUCKET_COUNT_LOG2};
/** Total number of buckets for new addresses */
static constexpr int32_t ADDRMAN_NEW_BUCKET_COUNT_LOG2{10};
static constexpr int ADDRMAN_NEW_BUCKET_COUNT{1 << ADDRMAN_NEW_BUCKET_COUNT_LOG2};
/** Maximum allowed number of entries in buckets for new and tried addresses */
static constexpr int32_t ADDRMAN_BUCKET_SIZE_LOG2{6};
static constexpr int ADDRMAN_BUCKET_SIZE{1 << ADDRMAN_BUCKET_SIZE_LOG2};
/**
* Stochastical (IP) address manager
*/
class CAddrMan
class AddrMan
{
const std::unique_ptr<AddrManImpl> m_impl;
public:
template <typename Stream>
void Serialize(Stream& s_) const EXCLUSIVE_LOCKS_REQUIRED(!cs);
explicit AddrMan(std::vector<bool> asmap, bool deterministic, int32_t consistency_check_ratio, bool discriminate_ports = false);
~AddrMan();
template <typename Stream>
void Unserialize(Stream& s_) EXCLUSIVE_LOCKS_REQUIRED(!cs);
void Serialize(Stream& s_) const;
explicit CAddrMan(std::vector<bool> asmap, bool deterministic, int32_t consistency_check_ratio, bool _discriminatePorts = false);
~CAddrMan()
{
nKey.SetNull();
}
template <typename Stream>
void Unserialize(Stream& s_);
//! Return the number of (unique) addresses in all tables.
size_t size() const
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs); // TODO: Cache this in an atomic to avoid this overhead
return vRandom.size();
}
size_t size() const;
//! Add addresses to addrman's new table.
bool Add(const std::vector<CAddress> &vAddr, const CNetAddr& source, int64_t nTimePenalty = 0)
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs);
int nAdd = 0;
Check();
for (std::vector<CAddress>::const_iterator it = vAddr.begin(); it != vAddr.end(); it++)
nAdd += Add_(*it, source, nTimePenalty) ? 1 : 0;
Check();
if (nAdd) {
LogPrint(BCLog::ADDRMAN, "Added %i addresses from %s: %i tried, %i new\n", nAdd, source.ToString(), nTried, nNew);
}
return nAdd > 0;
}
/**
* Attempt to add one or more addresses to addrman's new table.
*
* @param[in] vAddr Address records to attempt to add.
* @param[in] source The address of the node that sent us these addr records.
* @param[in] nTimePenalty A "time penalty" to apply to the address record's nTime. If a peer
* sends us an address record with nTime=n, then we'll add it to our
* addrman with nTime=(n - nTimePenalty).
* @return true if at least one address is successfully added. */
bool Add(const std::vector<CAddress>& vAddr, const CNetAddr& source, int64_t nTimePenalty = 0);
//! Mark an entry as accessible.
void Good(const CService &addr, int64_t nTime = GetAdjustedTime())
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs);
Check();
Good_(addr, /* test_before_evict */ true, nTime);
Check();
}
//! Mark an entry as accessible, possibly moving it from "new" to "tried".
void Good(const CService& addr, int64_t nTime = GetAdjustedTime());
//! Mark an entry as connection attempted to.
void Attempt(const CService &addr, bool fCountFailure, int64_t nTime = GetAdjustedTime())
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs);
Check();
Attempt_(addr, fCountFailure, nTime);
Check();
}
void Attempt(const CService& addr, bool fCountFailure, int64_t nTime = GetAdjustedTime());
//! See if any to-be-evicted tried table entries have been tested and if so resolve the collisions.
void ResolveCollisions()
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs);
Check();
ResolveCollisions_();
Check();
}
void ResolveCollisions();
//! Randomly select an address in tried that another address is attempting to evict.
CAddrInfo SelectTriedCollision()
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs);
Check();
const CAddrInfo ret = SelectTriedCollision_();
Check();
return ret;
}
/**
* Randomly select an address in the tried table that another address is
* attempting to evict.
*
* @return CAddress The record for the selected tried peer.
* int64_t The last time we attempted to connect to that peer.
*/
std::pair<CAddress, int64_t> SelectTriedCollision();
/**
* Choose an address to connect to.
*
* @param[in] newOnly Whether to only select addresses from the new table.
* @return CAddress The record for the selected peer.
* int64_t The last time we attempted to connect to that peer.
*/
CAddrInfo Select(bool newOnly = false) const
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs);
Check();
const CAddrInfo addrRet = Select_(newOnly);
Check();
return addrRet;
}
std::pair<CAddress, int64_t> Select(bool newOnly = false) const;
/**
* Return all or many randomly selected addresses, optionally by network.
@ -239,197 +114,10 @@ public:
* @param[in] max_addresses Maximum number of addresses to return (0 = all).
* @param[in] max_pct Maximum percentage of addresses to return (0 = all).
* @param[in] network Select only addresses of this network (nullopt = all).
*/
std::vector<CAddress> GetAddr(size_t max_addresses, size_t max_pct, std::optional<Network> network) const
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs);
Check();
std::vector<CAddress> vAddr;
GetAddr_(vAddr, max_addresses, max_pct, network);
Check();
return vAddr;
}
//! Outer function for Connected_()
void Connected(const CService &addr, int64_t nTime = GetAdjustedTime())
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs);
Check();
Connected_(addr, nTime);
Check();
}
void SetServices(const CService &addr, ServiceFlags nServices)
EXCLUSIVE_LOCKS_REQUIRED(!cs)
{
LOCK(cs);
Check();
SetServices_(addr, nServices);
Check();
}
const std::vector<bool>& GetAsmap() const { return m_asmap; }
CAddrInfo GetAddressInfo(const CService& addr)
{
CAddrInfo addrRet;
{
LOCK(cs);
Check();
addrRet = GetAddressInfo_(addr);
Check();
}
return addrRet;
}
private:
//! A mutex to protect the inner data structures.
mutable Mutex cs;
//! Source of random numbers for randomization in inner loops
mutable FastRandomContext insecure_rand GUARDED_BY(cs);
//! secret key to randomize bucket select with
uint256 nKey;
//! Serialization versions.
enum Format : uint8_t {
V0_HISTORICAL = 0, //!< historic format, before commit e6b343d88
V1_DETERMINISTIC = 1, //!< for pre-asmap files
V2_ASMAP = 2, //!< for files including asmap version
V3_BIP155 = 3, //!< same as V2_ASMAP plus addresses are in BIP155 format
};
//! The maximum format this software knows it can unserialize. Also, we always serialize
//! in this format.
//! The format (first byte in the serialized stream) can be higher than this and
//! still this software may be able to unserialize the file - if the second byte
//! (see `lowest_compatible` in `Unserialize()`) is less or equal to this.
static constexpr Format FILE_FORMAT = Format::V3_BIP155;
//! The initial value of a field that is incremented every time an incompatible format
//! change is made (such that old software versions would not be able to parse and
//! understand the new file format). This is 32 because we overtook the "key size"
//! field which was 32 historically.
//! @note Don't increment this. Increment `lowest_compatible` in `Serialize()` instead.
static constexpr uint8_t INCOMPATIBILITY_BASE = 32;
//! last used nId
int nIdCount GUARDED_BY(cs){0};
//! table with information about all nIds
std::unordered_map<int, CAddrInfo> mapInfo GUARDED_BY(cs);
//! find an nId based on its network address
std::unordered_map<CNetAddr, int, CNetAddrHash> mapAddr GUARDED_BY(cs);
//! randomly-ordered vector of all nIds
//! This is mutable because it is unobservable outside the class, so any
//! changes to it (even in const methods) are also unobservable.
mutable std::vector<int> vRandom GUARDED_BY(cs);
// number of "tried" entries
int nTried GUARDED_BY(cs){0};
//! list of "tried" buckets
int vvTried[ADDRMAN_TRIED_BUCKET_COUNT][ADDRMAN_BUCKET_SIZE] GUARDED_BY(cs);
//! number of (unique) "new" entries
int nNew GUARDED_BY(cs){0};
//! list of "new" buckets
int vvNew[ADDRMAN_NEW_BUCKET_COUNT][ADDRMAN_BUCKET_SIZE] GUARDED_BY(cs);
//! last time Good was called (memory only). Initially set to 1 so that "never" is strictly worse.
int64_t nLastGood GUARDED_BY(cs){1};
//! Holds addrs inserted into tried table that collide with existing entries. Test-before-evict discipline used to resolve these collisions.
std::set<int> m_tried_collisions;
/** Perform consistency checks every m_consistency_check_ratio operations (if non-zero). */
const int32_t m_consistency_check_ratio;
// Compressed IP->ASN mapping, loaded from a file when a node starts.
// Should be always empty if no file was provided.
// This mapping is then used for bucketing nodes in Addrman.
//
// If asmap is provided, nodes will be bucketed by
// AS they belong to, in order to make impossible for a node
// to connect to several nodes hosted in a single AS.
// This is done in response to Erebus attack, but also to generally
// diversify the connections every node creates,
// especially useful when a large fraction of nodes
// operate under a couple of cloud providers.
//
// If a new asmap was provided, the existing records
// would be re-bucketed accordingly.
const std::vector<bool> m_asmap;
// discriminate entries based on port. Should be false on mainnet/testnet and can be true on devnet/regtest
bool discriminatePorts GUARDED_BY(cs);
//! Find an entry.
CAddrInfo* Find(const CService& addr, int *pnId = nullptr) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Create a new entry and add it to the internal data structures mapInfo, mapAddr and vRandom.
CAddrInfo* Create(const CAddress &addr, const CNetAddr &addrSource, int *pnId = nullptr) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Swap two elements in vRandom.
void SwapRandom(unsigned int nRandomPos1, unsigned int nRandomPos2) const EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Move an entry from the "new" table(s) to the "tried" table
void MakeTried(CAddrInfo& info, int nId) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Delete an entry. It must not be in tried, and have refcount 0.
void Delete(int nId) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Clear a position in a "new" table. This is the only place where entries are actually deleted.
void ClearNew(int nUBucket, int nUBucketPos) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Mark an entry "good", possibly moving it from "new" to "tried".
void Good_(const CService &addr, bool test_before_evict, int64_t time) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Add an entry to the "new" table.
bool Add_(const CAddress &addr, const CNetAddr& source, int64_t nTimePenalty) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Mark an entry as attempted to connect.
void Attempt_(const CService &addr, bool fCountFailure, int64_t nTime) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Select an address to connect to, if newOnly is set to true, only the new table is selected from.
CAddrInfo Select_(bool newOnly) const EXCLUSIVE_LOCKS_REQUIRED(cs);
//! See if any to-be-evicted tried table entries have been tested and if so resolve the collisions.
void ResolveCollisions_() EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Return a random to-be-evicted tried table address.
CAddrInfo SelectTriedCollision_() EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Consistency check
void Check() const EXCLUSIVE_LOCKS_REQUIRED(cs)
{
AssertLockHeld(cs);
const int err = Check_();
if (err) {
LogPrintf("ADDRMAN CONSISTENCY CHECK FAILED!!! err=%i\n", err);
assert(false);
}
}
//! Perform consistency check. Returns an error code or zero.
int Check_() const EXCLUSIVE_LOCKS_REQUIRED(cs);
/**
* Return all or many randomly selected addresses, optionally by network.
*
* @param[out] vAddr Vector of randomly selected addresses from vRandom.
* @param[in] max_addresses Maximum number of addresses to return (0 = all).
* @param[in] max_pct Maximum percentage of addresses to return (0 = all).
* @param[in] network Select only addresses of this network (nullopt = all).
* @return A vector of randomly selected addresses from vRandom.
*/
void GetAddr_(std::vector<CAddress>& vAddr, size_t max_addresses, size_t max_pct, std::optional<Network> network) const EXCLUSIVE_LOCKS_REQUIRED(cs);
std::vector<CAddress> GetAddr(size_t max_addresses, size_t max_pct, std::optional<Network> network) const;
/** We have successfully connected to this peer. Calling this function
* updates the CAddress's nTime, which is used in our IsTerrible()
@ -442,17 +130,18 @@ private:
* @param[in] addr The address of the peer we were connected to
* @param[in] nTime The time that we were last connected to this peer
*/
void Connected_(const CService& addr, int64_t nTime) EXCLUSIVE_LOCKS_REQUIRED(cs);
void Connected(const CService& addr, int64_t nTime = GetAdjustedTime());
//! Update an entry's service bits.
void SetServices_(const CService &addr, ServiceFlags nServices) EXCLUSIVE_LOCKS_REQUIRED(cs);
void SetServices(const CService& addr, ServiceFlags nServices);
//! Get address info for address
CAddrInfo GetAddressInfo_(const CService& addr) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! See if any to-be-evicted tried table entries have been tested and if so resolve the collisions.
AddrInfo GetAddressInfo(const CService& addr);
const std::vector<bool>& GetAsmap() const;
friend class CAddrManTest;
friend class CAddrManDeterministic;
friend class AddrManTest;
friend class AddrManDeterministic;
};
#endif // BITCOIN_ADDRMAN_H

283
src/addrman_impl.h Normal file
View File

@ -0,0 +1,283 @@
// Copyright (c) 2021 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#ifndef BITCOIN_ADDRMAN_IMPL_H
#define BITCOIN_ADDRMAN_IMPL_H
#include <logging.h>
#include <netaddress.h>
#include <protocol.h>
#include <serialize.h>
#include <sync.h>
#include <uint256.h>
#include <cstdint>
#include <optional>
#include <set>
#include <unordered_map>
#include <unordered_set>
#include <utility>
#include <vector>
/** Total number of buckets for tried addresses */
static constexpr int32_t ADDRMAN_TRIED_BUCKET_COUNT_LOG2{8};
static constexpr int ADDRMAN_TRIED_BUCKET_COUNT{1 << ADDRMAN_TRIED_BUCKET_COUNT_LOG2};
/** Total number of buckets for new addresses */
static constexpr int32_t ADDRMAN_NEW_BUCKET_COUNT_LOG2{10};
static constexpr int ADDRMAN_NEW_BUCKET_COUNT{1 << ADDRMAN_NEW_BUCKET_COUNT_LOG2};
/** Maximum allowed number of entries in buckets for new and tried addresses */
static constexpr int32_t ADDRMAN_BUCKET_SIZE_LOG2{6};
static constexpr int ADDRMAN_BUCKET_SIZE{1 << ADDRMAN_BUCKET_SIZE_LOG2};
/**
* Extended statistics about a CAddress
*/
class AddrInfo : public CAddress
{
public:
//! last try whatsoever by us (memory only)
int64_t nLastTry{0};
//! last counted attempt (memory only)
int64_t nLastCountAttempt{0};
//! where knowledge about this address first came from
CNetAddr source;
//! last successful connection by us
int64_t nLastSuccess{0};
//! connection attempts since last successful attempt
int nAttempts{0};
//! reference count in new sets (memory only)
int nRefCount{0};
//! in tried set? (memory only)
bool fInTried{false};
//! position in vRandom
mutable int nRandomPos{-1};
SERIALIZE_METHODS(AddrInfo, obj)
{
READWRITEAS(CAddress, obj);
READWRITE(obj.source, obj.nLastSuccess, obj.nAttempts);
}
AddrInfo(const CAddress &addrIn, const CNetAddr &addrSource) : CAddress(addrIn), source(addrSource)
{
}
AddrInfo() : CAddress(), source()
{
}
//! Calculate in which "tried" bucket this entry belongs
int GetTriedBucket(const uint256 &nKey, const std::vector<bool> &asmap) const;
//! Calculate in which "new" bucket this entry belongs, given a certain source
int GetNewBucket(const uint256 &nKey, const CNetAddr& src, const std::vector<bool> &asmap) const;
//! Calculate in which "new" bucket this entry belongs, using its default source
int GetNewBucket(const uint256 &nKey, const std::vector<bool> &asmap) const
{
return GetNewBucket(nKey, source, asmap);
}
//! Calculate in which position of a bucket to store this entry.
int GetBucketPosition(const uint256 &nKey, bool fNew, int nBucket) const;
//! Determine whether the statistics about this entry are bad enough so that it can just be deleted
bool IsTerrible(int64_t nNow = GetAdjustedTime()) const;
//! Calculate the relative chance this entry should be given when selecting nodes to connect to
double GetChance(int64_t nNow = GetAdjustedTime()) const;
};
class AddrManImpl
{
public:
AddrManImpl(std::vector<bool>&& asmap, bool deterministic, int32_t consistency_check_ratio, bool discriminate_ports);
~AddrManImpl();
template <typename Stream>
void Serialize(Stream& s_) const EXCLUSIVE_LOCKS_REQUIRED(!cs);
template <typename Stream>
void Unserialize(Stream& s_) EXCLUSIVE_LOCKS_REQUIRED(!cs);
size_t size() const EXCLUSIVE_LOCKS_REQUIRED(!cs);
bool Add(const std::vector<CAddress>& vAddr, const CNetAddr& source, int64_t nTimePenalty)
EXCLUSIVE_LOCKS_REQUIRED(!cs);
void Good(const CService& addr, int64_t nTime)
EXCLUSIVE_LOCKS_REQUIRED(!cs);
void Attempt(const CService& addr, bool fCountFailure, int64_t nTime)
EXCLUSIVE_LOCKS_REQUIRED(!cs);
void ResolveCollisions() EXCLUSIVE_LOCKS_REQUIRED(!cs);
std::pair<CAddress, int64_t> SelectTriedCollision() EXCLUSIVE_LOCKS_REQUIRED(!cs);
std::pair<CAddress, int64_t> Select(bool newOnly) const
EXCLUSIVE_LOCKS_REQUIRED(!cs);
std::vector<CAddress> GetAddr(size_t max_addresses, size_t max_pct, std::optional<Network> network) const
EXCLUSIVE_LOCKS_REQUIRED(!cs);
void Connected(const CService& addr, int64_t nTime)
EXCLUSIVE_LOCKS_REQUIRED(!cs);
void SetServices(const CService& addr, ServiceFlags nServices)
EXCLUSIVE_LOCKS_REQUIRED(!cs);
AddrInfo GetAddressInfo(const CService& addr);
const std::vector<bool>& GetAsmap() const;
friend class AddrManTest;
friend class AddrManDeterministic;
private:
//! A mutex to protect the inner data structures.
mutable Mutex cs;
//! Source of random numbers for randomization in inner loops
mutable FastRandomContext insecure_rand GUARDED_BY(cs);
//! secret key to randomize bucket select with
uint256 nKey;
//! Serialization versions.
enum Format : uint8_t {
V0_HISTORICAL = 0, //!< historic format, before commit e6b343d88
V1_DETERMINISTIC = 1, //!< for pre-asmap files
V2_ASMAP = 2, //!< for files including asmap version
V3_BIP155 = 3, //!< same as V2_ASMAP plus addresses are in BIP155 format
V4_MULTIPORT = 4, //!< adds support for multiple ports per IP
};
//! The maximum format this software knows it can unserialize. Also, we always serialize
//! in this format.
//! The format (first byte in the serialized stream) can be higher than this and
//! still this software may be able to unserialize the file - if the second byte
//! (see `lowest_compatible` in `Unserialize()`) is less or equal to this.
static constexpr Format FILE_FORMAT = Format::V4_MULTIPORT;
//! The initial value of a field that is incremented every time an incompatible format
//! change is made (such that old software versions would not be able to parse and
//! understand the new file format). This is 32 because we overtook the "key size"
//! field which was 32 historically.
//! @note Don't increment this. Increment `lowest_compatible` in `Serialize()` instead.
static constexpr uint8_t INCOMPATIBILITY_BASE = 32;
//! last used nId
int nIdCount GUARDED_BY(cs){0};
//! table with information about all nIds
std::unordered_map<int, AddrInfo> mapInfo GUARDED_BY(cs);
//! find an nId based on its network address and port.
std::unordered_map<CService, int, CServiceHash> mapAddr GUARDED_BY(cs);
//! randomly-ordered vector of all nIds
//! This is mutable because it is unobservable outside the class, so any
//! changes to it (even in const methods) are also unobservable.
mutable std::vector<int> vRandom GUARDED_BY(cs);
// number of "tried" entries
int nTried GUARDED_BY(cs){0};
//! list of "tried" buckets
int vvTried[ADDRMAN_TRIED_BUCKET_COUNT][ADDRMAN_BUCKET_SIZE] GUARDED_BY(cs);
//! number of (unique) "new" entries
int nNew GUARDED_BY(cs){0};
//! list of "new" buckets
int vvNew[ADDRMAN_NEW_BUCKET_COUNT][ADDRMAN_BUCKET_SIZE] GUARDED_BY(cs);
//! last time Good was called (memory only). Initially set to 1 so that "never" is strictly worse.
int64_t nLastGood GUARDED_BY(cs){1};
//! Holds addrs inserted into tried table that collide with existing entries. Test-before-evict discipline used to resolve these collisions.
std::set<int> m_tried_collisions;
/** Perform consistency checks every m_consistency_check_ratio operations (if non-zero). */
const int32_t m_consistency_check_ratio;
// Compressed IP->ASN mapping, loaded from a file when a node starts.
// Should be always empty if no file was provided.
// This mapping is then used for bucketing nodes in Addrman.
//
// If asmap is provided, nodes will be bucketed by
// AS they belong to, in order to make impossible for a node
// to connect to several nodes hosted in a single AS.
// This is done in response to Erebus attack, but also to generally
// diversify the connections every node creates,
// especially useful when a large fraction of nodes
// operate under a couple of cloud providers.
//
// If a new asmap was provided, the existing records
// would be re-bucketed accordingly.
const std::vector<bool> m_asmap;
//! Discriminate entries based on port.
bool m_discriminate_ports GUARDED_BY(cs);
//! Find an entry.
AddrInfo* Find(const CService& addr, int* pnId = nullptr) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Create a new entry and add it to the internal data structures mapInfo, mapAddr and vRandom.
AddrInfo* Create(const CAddress& addr, const CNetAddr& addrSource, int* pnId = nullptr) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Swap two elements in vRandom.
void SwapRandom(unsigned int nRandomPos1, unsigned int nRandomPos2) const EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Delete an entry. It must not be in tried, and have refcount 0.
void Delete(int nId) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Clear a position in a "new" table. This is the only place where entries are actually deleted.
void ClearNew(int nUBucket, int nUBucketPos) EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Move an entry from the "new" table(s) to the "tried" table
void MakeTried(AddrInfo& info, int nId) EXCLUSIVE_LOCKS_REQUIRED(cs);
/** Attempt to add a single address to addrman's new table.
* @see AddrMan::Add() for parameters. */
bool AddSingle(const CAddress& addr, const CNetAddr& source, int64_t nTimePenalty) EXCLUSIVE_LOCKS_REQUIRED(cs);
void Good_(const CService& addr, bool test_before_evict, int64_t time) EXCLUSIVE_LOCKS_REQUIRED(cs);
bool Add_(const std::vector<CAddress> &vAddr, const CNetAddr& source, int64_t nTimePenalty) EXCLUSIVE_LOCKS_REQUIRED(cs);
void Attempt_(const CService& addr, bool fCountFailure, int64_t nTime) EXCLUSIVE_LOCKS_REQUIRED(cs);
std::pair<CAddress, int64_t> Select_(bool newOnly) const EXCLUSIVE_LOCKS_REQUIRED(cs);
std::vector<CAddress> GetAddr_(size_t max_addresses, size_t max_pct, std::optional<Network> network) const EXCLUSIVE_LOCKS_REQUIRED(cs);
void Connected_(const CService& addr, int64_t nTime) EXCLUSIVE_LOCKS_REQUIRED(cs);
void SetServices_(const CService& addr, ServiceFlags nServices) EXCLUSIVE_LOCKS_REQUIRED(cs);
AddrInfo GetAddressInfo_(const CService& addr) EXCLUSIVE_LOCKS_REQUIRED(cs);
void ResolveCollisions_() EXCLUSIVE_LOCKS_REQUIRED(cs);
std::pair<CAddress, int64_t> SelectTriedCollision_() EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Consistency check, taking into account m_consistency_check_ratio. Will std::abort if an inconsistency is detected.
void Check() const EXCLUSIVE_LOCKS_REQUIRED(cs);
//! Perform consistency check, regardless of m_consistency_check_ratio.
//! @returns an error code or zero.
int ForceCheckAddrman() const EXCLUSIVE_LOCKS_REQUIRED(cs);
};
#endif // BITCOIN_ADDRMAN_IMPL_H

22
src/bench/addrman.cpp
View File

@ -51,14 +51,14 @@ static void CreateAddresses()
}
}
static void AddAddressesToAddrMan(CAddrMan& addrman)
static void AddAddressesToAddrMan(AddrMan& addrman)
{
for (size_t source_i = 0; source_i < NUM_SOURCES; ++source_i) {
addrman.Add(g_addresses[source_i], g_sources[source_i]);
}
}
static void FillAddrMan(CAddrMan& addrman)
static void FillAddrMan(AddrMan& addrman)
{
CreateAddresses();
@ -72,26 +72,26 @@ static void AddrManAdd(benchmark::Bench& bench)
CreateAddresses();
bench.run([&] {
CAddrMan addrman{/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0};
AddrMan addrman{/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0};
AddAddressesToAddrMan(addrman);
});
}
static void AddrManSelect(benchmark::Bench& bench)
{
CAddrMan addrman(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
AddrMan addrman(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
FillAddrMan(addrman);
bench.run([&] {
const auto& address = addrman.Select();
assert(address.GetPort() > 0);
assert(address.first.GetPort() > 0);
});
}
static void AddrManGetAddr(benchmark::Bench& bench)
{
CAddrMan addrman(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
AddrMan addrman(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
FillAddrMan(addrman);
@ -103,21 +103,21 @@ static void AddrManGetAddr(benchmark::Bench& bench)
static void AddrManGood(benchmark::Bench& bench)
{
/* Create many CAddrMan objects - one to be modified at each loop iteration.
* This is necessary because the CAddrMan::Good() method modifies the
/* Create many AddrMan objects - one to be modified at each loop iteration.
* This is necessary because the AddrMan::Good() method modifies the
* object, affecting the timing of subsequent calls to the same method and
* we want to do the same amount of work in every loop iteration. */
bench.epochs(5).epochIterations(1);
const size_t addrman_count{bench.epochs() * bench.epochIterations()};
std::vector<std::unique_ptr<CAddrMan>> addrmans(addrman_count);
std::vector<std::unique_ptr<AddrMan>> addrmans(addrman_count);
for (size_t i{0}; i < addrman_count; ++i) {
addrmans[i] = std::make_unique<CAddrMan>(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
addrmans[i] = std::make_unique<AddrMan>(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
FillAddrMan(*addrmans[i]);
}
auto markSomeAsGood = [](CAddrMan& addrman) {
auto markSomeAsGood = [](AddrMan& addrman) {
for (size_t source_i = 0; source_i < NUM_SOURCES; ++source_i) {
for (size_t addr_i = 0; addr_i < NUM_ADDRESSES_PER_SOURCE; ++addr_i) {
if (addr_i % 32 == 0) {

14
src/init.cpp
View File

@ -1693,19 +1693,9 @@ bool AppInitMain(const CoreContext& context, NodeContext& node, interfaces::Bloc
LogPrintf("Using /16 prefix for IP bucketing\n");
}
auto check_addrman = std::clamp<int32_t>(args.GetArg("-checkaddrman", DEFAULT_ADDRMAN_CONSISTENCY_CHECKS), 0, 1000000);
node.addrman = std::make_unique<CAddrMan>(asmap, /* deterministic */ false, /* consistency_check_ratio */ check_addrman);
// Load addresses from peers.dat
uiInterface.InitMessage(_("Loading P2P addresses…").translated);
int64_t nStart = GetTimeMillis();
if (ReadPeerAddresses(args, *node.addrman)) {
LogPrintf("Loaded %i addresses from peers.dat %dms\n", node.addrman->size(), GetTimeMillis() - nStart);
} else {
// Addrman can be in an inconsistent state after failure, reset it
node.addrman = std::make_unique<CAddrMan>(asmap, /* deterministic */ false, /* consistency_check_ratio */ check_addrman);
LogPrintf("Recreating peers.dat\n");
DumpPeerAddresses(args, *node.addrman);
if (const auto error{LoadAddrman(asmap, args, node.addrman)}) {
return InitError(*error);
}
}

15
src/net.cpp
View File

@ -2626,17 +2626,18 @@ void CConnman::ThreadOpenConnections(const std::vector<std::string> connect, CDe
if (nTries > 100)
break;
CAddrInfo addr;
CAddress addr;
int64_t addr_last_try{0};
if (fFeeler) {
// First, try to get a tried table collision address. This returns
// an empty (invalid) address if there are no collisions to try.
addr = addrman.SelectTriedCollision();
std::tie(addr, addr_last_try) = addrman.SelectTriedCollision();
if (!addr.IsValid()) {
// No tried table collisions. Select a new table address
// for our feeler.
addr = addrman.Select(true);
std::tie(addr, addr_last_try) = addrman.Select(true);
} else if (AlreadyConnectedToAddress(addr)) {
// If test-before-evict logic would have us connect to a
// peer that we're already connected to, just mark that
@ -2645,11 +2646,11 @@ void CConnman::ThreadOpenConnections(const std::vector<std::string> connect, CDe
// a currently-connected peer.
addrman.Good(addr);
// Select a new table address for our feeler instead.
addr = addrman.Select(true);
std::tie(addr, addr_last_try) = addrman.Select(true);
}
} else {
// Not a feeler
addr = addrman.Select();
std::tie(addr, addr_last_try) = addrman.Select();
}
auto dmn = mnList.GetMNByService(addr);
@ -2678,7 +2679,7 @@ void CConnman::ThreadOpenConnections(const std::vector<std::string> connect, CDe
continue;
// only consider very recently tried nodes after 30 failed attempts
if (nANow - addr.nLastTry < 600 && nTries < 30)
if (nANow - addr_last_try < 600 && nTries < 30)
continue;
// for non-feelers, require all the services we'll want,
@ -3308,7 +3309,7 @@ void CConnman::SetNetworkActive(bool active, CMasternodeSync* const mn_sync)
uiInterface.NotifyNetworkActiveChanged(fNetworkActive);
}
CConnman::CConnman(uint64_t nSeed0In, uint64_t nSeed1In, CAddrMan& addrman_in, bool network_active) :
CConnman::CConnman(uint64_t nSeed0In, uint64_t nSeed1In, AddrMan& addrman_in, bool network_active) :
addrman(addrman_in), nSeed0(nSeed0In), nSeed1(nSeed1In)
{
SetTryNewOutboundPeer(false);

4
src/net.h
View File

@ -893,7 +893,7 @@ public:
m_onion_binds = connOptions.onion_binds;
}
CConnman(uint64_t seed0, uint64_t seed1, CAddrMan& addrman, bool network_active = true);
CConnman(uint64_t seed0, uint64_t seed1, AddrMan& addrman, bool network_active = true);
~CConnman();
bool Start(CDeterministicMNManager& dmnman, CMasternodeMetaMan& mn_metaman, CMasternodeSync& mn_sync,
CScheduler& scheduler, const Options& options)
@ -1404,7 +1404,7 @@ private:
std::vector<ListenSocket> vhListenSocket;
std::atomic<bool> fNetworkActive{true};
bool fAddressesInitialized{false};
CAddrMan& addrman;
AddrMan& addrman;
std::deque<std::string> m_addr_fetches GUARDED_BY(m_addr_fetches_mutex);
Mutex m_addr_fetches_mutex;
std::vector<std::string> m_added_nodes GUARDED_BY(m_added_nodes_mutex);

10
src/net_processing.cpp
View File

@ -363,7 +363,7 @@ using PeerRef = std::shared_ptr<Peer>;
class PeerManagerImpl final : public PeerManager
{
public:
PeerManagerImpl(const CChainParams& chainparams, CConnman& connman, CAddrMan& addrman, BanMan* banman,
PeerManagerImpl(const CChainParams& chainparams, CConnman& connman, AddrMan& addrman, BanMan* banman,
CScheduler &scheduler, ChainstateManager& chainman, CTxMemPool& pool,
CMasternodeMetaMan& mn_metaman, CMasternodeSync& mn_sync,
CGovernanceManager& govman, CSporkManager& sporkman,
@ -497,7 +497,7 @@ private:
const CChainParams& m_chainparams;
CConnman& m_connman;
CAddrMan& m_addrman;
AddrMan& m_addrman;
/** Pointer to this node's banman. May be nullptr - check existence before dereferencing. */
BanMan* const m_banman;
ChainstateManager& m_chainman;
@ -1836,7 +1836,7 @@ bool PeerManagerImpl::BlockRequestAllowed(const CBlockIndex* pindex)
(GetBlockProofEquivalentTime(*pindexBestHeader, *pindex, *pindexBestHeader, m_chainparams.GetConsensus()) < STALE_RELAY_AGE_LIMIT);
}
std::unique_ptr<PeerManager> PeerManager::make(const CChainParams& chainparams, CConnman& connman, CAddrMan& addrman, BanMan* banman,
std::unique_ptr<PeerManager> PeerManager::make(const CChainParams& chainparams, CConnman& connman, AddrMan& addrman, BanMan* banman,
CScheduler &scheduler, ChainstateManager& chainman, CTxMemPool& pool,
CMasternodeMetaMan& mn_metaman, CMasternodeSync& mn_sync,
CGovernanceManager& govman, CSporkManager& sporkman,
@ -1848,7 +1848,7 @@ std::unique_ptr<PeerManager> PeerManager::make(const CChainParams& chainparams,
return std::make_unique<PeerManagerImpl>(chainparams, connman, addrman, banman, scheduler, chainman, pool, mn_metaman, mn_sync, govman, sporkman, mn_activeman, dmnman, cj_ctx, llmq_ctx, ignore_incoming_txs);
}
PeerManagerImpl::PeerManagerImpl(const CChainParams& chainparams, CConnman& connman, CAddrMan& addrman, BanMan* banman,
PeerManagerImpl::PeerManagerImpl(const CChainParams& chainparams, CConnman& connman, AddrMan& addrman, BanMan* banman,
CScheduler &scheduler, ChainstateManager& chainman, CTxMemPool& pool,
CMasternodeMetaMan& mn_metaman, CMasternodeSync& mn_sync,
CGovernanceManager& govman, CSporkManager& sporkman,
@ -3440,7 +3440,7 @@ void PeerManagerImpl::ProcessMessage(
// table is also potentially detrimental because new-table entries
// are subject to eviction in the event of addrman collisions. We
// mitigate the information-leak by never calling
// CAddrMan::Connected() on block-relay-only peers; see
// AddrMan::Connected() on block-relay-only peers; see
// FinalizeNode().
//
// This moves an address from New to Tried table in Addrman,

4
src/net_processing.h
View File

@ -14,7 +14,7 @@
#include <atomic>
class CActiveMasternodeManager;
class CAddrMan;
class AddrMan;
class CTxMemPool;
class CDeterministicMNManager;
class CMasternodeMetaMan;
@ -56,7 +56,7 @@ struct CNodeStateStats {
class PeerManager : public CValidationInterface, public NetEventsInterface
{
public:
static std::unique_ptr<PeerManager> make(const CChainParams& chainparams, CConnman& connman, CAddrMan& addrman,
static std::unique_ptr<PeerManager> make(const CChainParams& chainparams, CConnman& connman, AddrMan& addrman,
BanMan* banman, CScheduler &scheduler, ChainstateManager& chainman,
CTxMemPool& pool, CMasternodeMetaMan& mn_metaman, CMasternodeSync& mn_sync,
CGovernanceManager& govman, CSporkManager& sporkman,

2
src/netaddress.cpp
View File

@ -166,7 +166,7 @@ void CNetAddr::SetLegacyIPv6(Span<const uint8_t> ipv6)
}
/**
* Create an "internal" address that represents a name or FQDN. CAddrMan uses
* Create an "internal" address that represents a name or FQDN. AddrMan uses
* these fake addresses to keep track of which DNS seeds were used.
* @returns Whether or not the operation was successful.
* @see NET_INTERNAL, INTERNAL_IN_IPV6_PREFIX, CNetAddr::IsInternal(), CNetAddr::IsRFC4193()

37
src/netaddress.h
View File

@ -63,7 +63,7 @@ enum Network {
NET_CJDNS,
/// A set of addresses that represent the hash of a string or FQDN. We use
/// them in CAddrMan to keep track of which DNS seeds were used.
/// them in AddrMan to keep track of which DNS seeds were used.
NET_INTERNAL,
/// Dummy value to indicate the number of NET_* constants.
@ -261,7 +261,6 @@ public:
}
}
friend class CNetAddrHash;
friend class CSubNet;
private:
@ -482,22 +481,6 @@ public:
}
};
class CNetAddrHash
{
public:
size_t operator()(const CNetAddr& a) const noexcept
{
CSipHasher hasher(m_salt_k0, m_salt_k1);
hasher.Write(a.m_net);
hasher.Write(a.m_addr.data(), a.m_addr.size());
return static_cast<size_t>(hasher.Finalize());
}
private:
const uint64_t m_salt_k0 = GetRand(std::numeric_limits<uint64_t>::max());
const uint64_t m_salt_k1 = GetRand(std::numeric_limits<uint64_t>::max());
};
class CSubNet
{
protected:
@ -582,7 +565,25 @@ public:
READWRITE(Using<BigEndianFormatter<2>>(obj.port));
}
friend class CServiceHash;
friend CService MaybeFlipIPv6toCJDNS(const CService& service);
};
class CServiceHash
{
public:
size_t operator()(const CService& a) const noexcept
{
CSipHasher hasher(m_salt_k0, m_salt_k1);
hasher.Write(a.m_net);
hasher.Write(a.port);
hasher.Write(a.m_addr.data(), a.m_addr.size());
return static_cast<size_t>(hasher.Finalize());
}
private:
const uint64_t m_salt_k0 = GetRand(std::numeric_limits<uint64_t>::max());
const uint64_t m_salt_k1 = GetRand(std::numeric_limits<uint64_t>::max());
};
#endif // BITCOIN_NETADDRESS_H

4
src/node/context.h
View File

@ -13,7 +13,7 @@
class ArgsManager;
class BanMan;
class CActiveMasternodeManager;
class CAddrMan;
class AddrMan;
class CBlockPolicyEstimator;
class CConnman;
class CCreditPoolManager;
@ -53,7 +53,7 @@ class Loader;
//! any member functions. It should just be a collection of references that can
//! be used without pulling in unwanted dependencies or functionality.
struct NodeContext {
std::unique_ptr<CAddrMan> addrman;
std::unique_ptr<AddrMan> addrman;
std::unique_ptr<CConnman> connman;
std::unique_ptr<CTxMemPool> mempool;
std::unique_ptr<CBlockPolicyEstimator> fee_estimator;

280
src/test/addrman_tests.cpp
View File

@ -4,6 +4,7 @@
#include <addrdb.h>
#include <addrman.h>
#include <addrman_impl.h>
#include <chainparams.h>
#include <clientversion.h>
#include <hash.h>
@ -19,26 +20,26 @@
using namespace std::literals;
class CAddrManSerializationMock : public CAddrMan
class AddrManSerializationMock : public AddrMan
{
public:
virtual void Serialize(CDataStream& s) const = 0;
CAddrManSerializationMock()
: CAddrMan(/* asmap */ std::vector<bool>(), /* deterministic */ true, /* consistency_check_ratio */ 100)
AddrManSerializationMock()
: AddrMan(/* asmap */ std::vector<bool>(), /* deterministic */ true, /* consistency_check_ratio */ 100, /* discriminate_ports */ true)
{}
};
class CAddrManUncorrupted : public CAddrManSerializationMock
class AddrManUncorrupted : public AddrManSerializationMock
{
public:
void Serialize(CDataStream& s) const override
{
CAddrMan::Serialize(s);
AddrMan::Serialize(s);
}
};
class CAddrManCorrupted : public CAddrManSerializationMock
class AddrManCorrupted : public AddrManSerializationMock
{
public:
void Serialize(CDataStream& s) const override
@ -59,12 +60,12 @@ public:
CAddress addr = CAddress(serv, NODE_NONE);
CNetAddr resolved;
BOOST_CHECK(LookupHost("252.2.2.2", resolved, false));
CAddrInfo info = CAddrInfo(addr, resolved);
AddrInfo info = AddrInfo(addr, resolved);
s << info;
}
};
static CDataStream AddrmanToStream(const CAddrManSerializationMock& _addrman)
static CDataStream AddrmanToStream(const AddrManSerializationMock& _addrman)
{
CDataStream ssPeersIn(SER_DISK, CLIENT_VERSION);
ssPeersIn << Params().MessageStart();
@ -74,45 +75,46 @@ static CDataStream AddrmanToStream(const CAddrManSerializationMock& _addrman)
return CDataStream(vchData, SER_DISK, CLIENT_VERSION);
}
class CAddrManTest : public CAddrMan
class AddrManTest : public AddrMan
{
private:
bool deterministic;
public:
explicit CAddrManTest(bool makeDeterministic = true,
std::vector<bool> asmap = std::vector<bool>())
: CAddrMan(asmap, makeDeterministic, /* consistency_check_ratio */ 100)
explicit AddrManTest(bool makeDeterministic = true,
std::vector<bool> asmap = std::vector<bool>(),
bool discriminatePorts = true)
: AddrMan(asmap, makeDeterministic, /* consistency_check_ratio */ 100, discriminatePorts)
{
deterministic = makeDeterministic;
}
CAddrInfo* Find(const CService& addr, int* pnId = nullptr)
AddrInfo* Find(const CService& addr, int* pnId = nullptr)
{
LOCK(cs);
return CAddrMan::Find(addr, pnId);
LOCK(m_impl->cs);
return m_impl->Find(addr, pnId);
}
CAddrInfo* Create(const CAddress& addr, const CNetAddr& addrSource, int* pnId = nullptr)
AddrInfo* Create(const CAddress& addr, const CNetAddr& addrSource, int* pnId = nullptr)
{
LOCK(cs);
return CAddrMan::Create(addr, addrSource, pnId);
LOCK(m_impl->cs);
return m_impl->Create(addr, addrSource, pnId);
}
void Delete(int nId)
{
LOCK(cs);
CAddrMan::Delete(nId);
LOCK(m_impl->cs);
m_impl->Delete(nId);
}
// Used to test deserialization
std::pair<int, int> GetBucketAndEntry(const CAddress& addr)
{
LOCK(cs);
int nId = mapAddr[addr];
LOCK(m_impl->cs);
int nId = m_impl->mapAddr[addr];
for (int bucket = 0; bucket < ADDRMAN_NEW_BUCKET_COUNT; ++bucket) {
for (int entry = 0; entry < ADDRMAN_BUCKET_SIZE; ++entry) {
if (nId == vvNew[bucket][entry]) {
if (nId == m_impl->vvNew[bucket][entry]) {
return std::pair<int, int>(bucket, entry);
}
}
@ -164,20 +166,20 @@ BOOST_FIXTURE_TEST_SUITE(addrman_tests, BasicTestingSetup)
BOOST_AUTO_TEST_CASE(addrman_simple)
{
auto addrman = std::make_unique<CAddrManTest>();
auto addrman = std::make_unique<AddrManTest>();
CNetAddr source = ResolveIP("252.2.2.2");
// Test: Does Addrman respond correctly when empty.
BOOST_CHECK_EQUAL(addrman->size(), 0U);
CAddrInfo addr_null = addrman->Select();
auto addr_null = addrman->Select().first;
BOOST_CHECK_EQUAL(addr_null.ToString(), "[::]:0");
// Test: Does Addrman::Add work as expected.
CService addr1 = ResolveService("250.1.1.1", 8333);
BOOST_CHECK(addrman->Add({CAddress(addr1, NODE_NONE)}, source));
BOOST_CHECK_EQUAL(addrman->size(), 1U);
CAddrInfo addr_ret1 = addrman->Select();
auto addr_ret1 = addrman->Select().first;
BOOST_CHECK_EQUAL(addr_ret1.ToString(), "250.1.1.1:8333");
// Test: Does IP address deduplication work correctly.
@ -198,7 +200,7 @@ BOOST_AUTO_TEST_CASE(addrman_simple)
BOOST_CHECK(addrman->size() >= 1);
// Test: reset addrman and test AddrMan::Add multiple addresses works as expected
addrman = std::make_unique<CAddrManTest>();
addrman = std::make_unique<AddrManTest>();
std::vector<CAddress> vAddr;
vAddr.push_back(CAddress(ResolveService("250.1.1.3", 8333), NODE_NONE));
vAddr.push_back(CAddress(ResolveService("250.1.1.4", 8333), NODE_NONE));
@ -208,7 +210,35 @@ BOOST_AUTO_TEST_CASE(addrman_simple)
BOOST_AUTO_TEST_CASE(addrman_ports)
{
CAddrManTest addrman;
AddrManTest addrman;
CNetAddr source = ResolveIP("252.2.2.2");
BOOST_CHECK_EQUAL(addrman.size(), 0U);
// Test 7; Addr with same IP but diff port does not replace existing addr.
CService addr1 = ResolveService("250.1.1.1", 8333);
BOOST_CHECK(addrman.Add({CAddress(addr1, NODE_NONE)}, source));
BOOST_CHECK_EQUAL(addrman.size(), 1U);
CService addr1_port = ResolveService("250.1.1.1", 8334);
BOOST_CHECK(addrman.Add({CAddress(addr1_port, NODE_NONE)}, source));
BOOST_CHECK_EQUAL(addrman.size(), 2U);
auto addr_ret2 = addrman.Select().first;
BOOST_CHECK(addr_ret2.ToString() == "250.1.1.1:8333" || addr_ret2.ToString() == "250.1.1.1:8334");
// Test: Add same IP but diff port to tried table; this converts the entry with
// the specified port to tried, but not the other.
addrman.Good(CAddress(addr1_port, NODE_NONE));
BOOST_CHECK_EQUAL(addrman.size(), 2U);
bool newOnly = true;
auto addr_ret3 = addrman.Select(newOnly).first;
BOOST_CHECK_EQUAL(addr_ret3.ToString(), "250.1.1.1:8333");
}
BOOST_AUTO_TEST_CASE(addrman_ports_nondiscriminate)
{
AddrManTest addrman(/* deterministic */ true, /* asmap */ std::vector<bool>(), /* discriminate */ false);
CNetAddr source = ResolveIP("252.2.2.2");
@ -222,7 +252,7 @@ BOOST_AUTO_TEST_CASE(addrman_ports)
CService addr1_port = ResolveService("250.1.1.1", 8334);
BOOST_CHECK(!addrman.Add({CAddress(addr1_port, NODE_NONE)}, source));
BOOST_CHECK_EQUAL(addrman.size(), 1U);
CAddrInfo addr_ret2 = addrman.Select();
auto addr_ret2 = addrman.Select().first;
BOOST_CHECK_EQUAL(addr_ret2.ToString(), "250.1.1.1:8333");
// Test: Add same IP but diff port to tried table, it doesn't get added.
@ -230,14 +260,13 @@ BOOST_AUTO_TEST_CASE(addrman_ports)
addrman.Good(CAddress(addr1_port, NODE_NONE));
BOOST_CHECK_EQUAL(addrman.size(), 1U);
bool newOnly = true;
CAddrInfo addr_ret3 = addrman.Select(newOnly);
auto addr_ret3 = addrman.Select(newOnly).first;
BOOST_CHECK_EQUAL(addr_ret3.ToString(), "250.1.1.1:8333");
}
BOOST_AUTO_TEST_CASE(addrman_select)
{
CAddrManTest addrman;
AddrManTest addrman;
CNetAddr source = ResolveIP("252.2.2.2");
@ -247,16 +276,16 @@ BOOST_AUTO_TEST_CASE(addrman_select)
BOOST_CHECK_EQUAL(addrman.size(), 1U);
bool newOnly = true;
CAddrInfo addr_ret1 = addrman.Select(newOnly);
auto addr_ret1 = addrman.Select(newOnly).first;
BOOST_CHECK_EQUAL(addr_ret1.ToString(), "250.1.1.1:8333");
// Test: move addr to tried, select from new expected nothing returned.
addrman.Good(CAddress(addr1, NODE_NONE));
BOOST_CHECK_EQUAL(addrman.size(), 1U);
CAddrInfo addr_ret2 = addrman.Select(newOnly);
auto addr_ret2 = addrman.Select(newOnly).first;
BOOST_CHECK_EQUAL(addr_ret2.ToString(), "[::]:0");
CAddrInfo addr_ret3 = addrman.Select();
auto addr_ret3 = addrman.Select().first;
BOOST_CHECK_EQUAL(addr_ret3.ToString(), "250.1.1.1:8333");
BOOST_CHECK_EQUAL(addrman.size(), 1U);
@ -289,14 +318,14 @@ BOOST_AUTO_TEST_CASE(addrman_select)
// Test: Select pulls from new and tried regardless of port number.
std::set<uint16_t> ports;
for (int i = 0; i < 20; ++i) {
ports.insert(addrman.Select().GetPort());
ports.insert(addrman.Select().first.GetPort());
}
BOOST_CHECK_EQUAL(ports.size(), 3U);
}
BOOST_AUTO_TEST_CASE(addrman_new_collisions)
{
CAddrManTest addrman;
AddrManTest addrman;
CNetAddr source = ResolveIP("252.2.2.2");
@ -325,7 +354,7 @@ BOOST_AUTO_TEST_CASE(addrman_new_collisions)
BOOST_AUTO_TEST_CASE(addrman_tried_collisions)
{
CAddrManTest addrman;
AddrManTest addrman;
CNetAddr source = ResolveIP("252.2.2.2");
@ -345,7 +374,7 @@ BOOST_AUTO_TEST_CASE(addrman_tried_collisions)
//Test: tried table collision!
CService addr1 = ResolveService("250.1.1." + ToString(++num_addrs));
uint32_t collisions{1};
BOOST_CHECK(addrman.Add({CAddress(addr1, NODE_NONE)}, source));
BOOST_CHECK(!addrman.Add({CAddress(addr1, NODE_NONE)}, source));
BOOST_CHECK_EQUAL(addrman.size(), num_addrs - collisions);
CService addr2 = ResolveService("250.1.1." + ToString(++num_addrs));
@ -355,7 +384,40 @@ BOOST_AUTO_TEST_CASE(addrman_tried_collisions)
BOOST_AUTO_TEST_CASE(addrman_find)
{
CAddrManTest addrman;
AddrManTest addrman;
BOOST_CHECK_EQUAL(addrman.size(), 0U);
CAddress addr1 = CAddress(ResolveService("250.1.2.1", 8333), NODE_NONE);
CAddress addr2 = CAddress(ResolveService("250.1.2.1", 9999), NODE_NONE);
CAddress addr3 = CAddress(ResolveService("251.255.2.1", 8333), NODE_NONE);
CNetAddr source1 = ResolveIP("250.1.2.1");
CNetAddr source2 = ResolveIP("250.1.2.2");
BOOST_CHECK(addrman.Add({addr1}, source1));
BOOST_CHECK(addrman.Add({addr2}, source2));
BOOST_CHECK(addrman.Add({addr3}, source1));
// Test: ensure Find returns an IP/port matching what we searched on.
AddrInfo* info1 = addrman.Find(addr1);
BOOST_REQUIRE(info1);
BOOST_CHECK_EQUAL(info1->ToString(), "250.1.2.1:8333");
// Test; Find discriminates by port number.
AddrInfo* info2 = addrman.Find(addr2);
BOOST_REQUIRE(info2);
BOOST_CHECK_EQUAL(info2->ToString(), "250.1.2.1:9999");
// Test: Find returns another IP matching what we searched on.
AddrInfo* info3 = addrman.Find(addr3);
BOOST_REQUIRE(info3);
BOOST_CHECK_EQUAL(info3->ToString(), "251.255.2.1:8333");
}
BOOST_AUTO_TEST_CASE(addrman_find_nondiscriminate)
{
AddrManTest addrman(/* deterministic */ true, /* asmap */ std::vector<bool>(), /* discriminate */ false);
BOOST_CHECK_EQUAL(addrman.size(), 0U);
@ -371,24 +433,24 @@ BOOST_AUTO_TEST_CASE(addrman_find)
BOOST_CHECK(addrman.Add({addr3}, source1));
// Test: ensure Find returns an IP matching what we searched on.
CAddrInfo* info1 = addrman.Find(addr1);
AddrInfo* info1 = addrman.Find(addr1);
BOOST_REQUIRE(info1);
BOOST_CHECK_EQUAL(info1->ToString(), "250.1.2.1:8333");
// Test 18; Find does not discriminate by port number.
CAddrInfo* info2 = addrman.Find(addr2);
AddrInfo* info2 = addrman.Find(addr2);
BOOST_REQUIRE(info2);
BOOST_CHECK_EQUAL(info2->ToString(), info1->ToString());
// Test: Find returns another IP matching what we searched on.
CAddrInfo* info3 = addrman.Find(addr3);
AddrInfo* info3 = addrman.Find(addr3);
BOOST_REQUIRE(info3);
BOOST_CHECK_EQUAL(info3->ToString(), "251.255.2.1:8333");
}
BOOST_AUTO_TEST_CASE(addrman_create)
{
CAddrManTest addrman;
AddrManTest addrman;
BOOST_CHECK_EQUAL(addrman.size(), 0U);
@ -396,19 +458,19 @@ BOOST_AUTO_TEST_CASE(addrman_create)
CNetAddr source1 = ResolveIP("250.1.2.1");
int nId;
CAddrInfo* pinfo = addrman.Create(addr1, source1, &nId);
AddrInfo* pinfo = addrman.Create(addr1, source1, &nId);
// Test: The result should be the same as the input addr.
BOOST_CHECK_EQUAL(pinfo->ToString(), "250.1.2.1:8333");
CAddrInfo* info2 = addrman.Find(addr1);
AddrInfo* info2 = addrman.Find(addr1);
BOOST_CHECK_EQUAL(info2->ToString(), "250.1.2.1:8333");
}
BOOST_AUTO_TEST_CASE(addrman_delete)
{
CAddrManTest addrman;
AddrManTest addrman;
BOOST_CHECK_EQUAL(addrman.size(), 0U);
@ -422,13 +484,13 @@ BOOST_AUTO_TEST_CASE(addrman_delete)
BOOST_CHECK_EQUAL(addrman.size(), 1U);
addrman.Delete(nId);
BOOST_CHECK_EQUAL(addrman.size(), 0U);
CAddrInfo* info2 = addrman.Find(addr1);
AddrInfo* info2 = addrman.Find(addr1);
BOOST_CHECK(info2 == nullptr);
}
BOOST_AUTO_TEST_CASE(addrman_getaddr)
{
CAddrManTest addrman;
AddrManTest addrman;
// Test: Sanity check, GetAddr should never return anything if addrman
// is empty.
@ -488,7 +550,7 @@ BOOST_AUTO_TEST_CASE(addrman_getaddr)
BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket_legacy)
{
CAddrManTest addrman;
AddrManTest addrman;
CAddress addr1 = CAddress(ResolveService("250.1.1.1", 8333), NODE_NONE);
CAddress addr2 = CAddress(ResolveService("250.1.1.1", 9999), NODE_NONE);
@ -496,7 +558,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket_legacy)
CNetAddr source1 = ResolveIP("250.1.1.1");
CAddrInfo info1 = CAddrInfo(addr1, source1);
AddrInfo info1 = AddrInfo(addr1, source1);
uint256 nKey1 = (uint256)(CHashWriter(SER_GETHASH, 0) << 1).GetHash();
uint256 nKey2 = (uint256)(CHashWriter(SER_GETHASH, 0) << 2).GetHash();
@ -511,14 +573,14 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket_legacy)
// Test: Two addresses with same IP but different ports can map to
// different buckets because they have different keys.
CAddrInfo info2 = CAddrInfo(addr2, source1);
AddrInfo info2 = AddrInfo(addr2, source1);
BOOST_CHECK(info1.GetKey() != info2.GetKey());
BOOST_CHECK(info1.GetTriedBucket(nKey1, asmap) != info2.GetTriedBucket(nKey1, asmap));
std::set<int> buckets;
for (int i = 0; i < 255; i++) {
CAddrInfo infoi = CAddrInfo(
AddrInfo infoi = AddrInfo(
CAddress(ResolveService("250.1.1." + ToString(i)), NODE_NONE),
ResolveIP("250.1.1." + ToString(i)));
int bucket = infoi.GetTriedBucket(nKey1, asmap);
@ -530,7 +592,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket_legacy)
buckets.clear();
for (int j = 0; j < 255; j++) {
CAddrInfo infoj = CAddrInfo(
AddrInfo infoj = AddrInfo(
CAddress(ResolveService("250." + ToString(j) + ".1.1"), NODE_NONE),
ResolveIP("250." + ToString(j) + ".1.1"));
int bucket = infoj.GetTriedBucket(nKey1, asmap);
@ -543,14 +605,14 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket_legacy)
BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket_legacy)
{
CAddrManTest addrman;
AddrManTest addrman;
CAddress addr1 = CAddress(ResolveService("250.1.2.1", 8333), NODE_NONE);
CAddress addr2 = CAddress(ResolveService("250.1.2.1", 9999), NODE_NONE);
CNetAddr source1 = ResolveIP("250.1.2.1");
CAddrInfo info1 = CAddrInfo(addr1, source1);
AddrInfo info1 = AddrInfo(addr1, source1);
uint256 nKey1 = (uint256)(CHashWriter(SER_GETHASH, 0) << 1).GetHash();
uint256 nKey2 = (uint256)(CHashWriter(SER_GETHASH, 0) << 2).GetHash();
@ -566,13 +628,13 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket_legacy)
BOOST_CHECK(info1.GetNewBucket(nKey1, asmap) != info1.GetNewBucket(nKey2, asmap));
// Test: Ports should not affect bucket placement in the addr
CAddrInfo info2 = CAddrInfo(addr2, source1);
AddrInfo info2 = AddrInfo(addr2, source1);
BOOST_CHECK(info1.GetKey() != info2.GetKey());
BOOST_CHECK_EQUAL(info1.GetNewBucket(nKey1, asmap), info2.GetNewBucket(nKey1, asmap));
std::set<int> buckets;
for (int i = 0; i < 255; i++) {
CAddrInfo infoi = CAddrInfo(
AddrInfo infoi = AddrInfo(
CAddress(ResolveService("250.1.1." + ToString(i)), NODE_NONE),
ResolveIP("250.1.1." + ToString(i)));
int bucket = infoi.GetNewBucket(nKey1, asmap);
@ -584,7 +646,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket_legacy)
buckets.clear();
for (int j = 0; j < 4 * 255; j++) {
CAddrInfo infoj = CAddrInfo(CAddress(
AddrInfo infoj = AddrInfo(CAddress(
ResolveService(
ToString(250 + (j / 255)) + "." + ToString(j % 256) + ".1.1"), NODE_NONE),
ResolveIP("251.4.1.1"));
@ -597,7 +659,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket_legacy)
buckets.clear();
for (int p = 0; p < 255; p++) {
CAddrInfo infoj = CAddrInfo(
AddrInfo infoj = AddrInfo(
CAddress(ResolveService("250.1.1.1"), NODE_NONE),
ResolveIP("250." + ToString(p) + ".1.1"));
int bucket = infoj.GetNewBucket(nKey1, asmap);
@ -621,7 +683,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket_legacy)
// 101.8.0.0/16 AS8
BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket)
{
CAddrManTest addrman;
AddrManTest addrman;
CAddress addr1 = CAddress(ResolveService("250.1.1.1", 8333), NODE_NONE);
CAddress addr2 = CAddress(ResolveService("250.1.1.1", 9999), NODE_NONE);
@ -629,7 +691,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket)
CNetAddr source1 = ResolveIP("250.1.1.1");
CAddrInfo info1 = CAddrInfo(addr1, source1);
AddrInfo info1 = AddrInfo(addr1, source1);
uint256 nKey1 = (uint256)(CHashWriter(SER_GETHASH, 0) << 1).GetHash();
uint256 nKey2 = (uint256)(CHashWriter(SER_GETHASH, 0) << 2).GetHash();
@ -644,14 +706,14 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket)
// Test: Two addresses with same IP but different ports can map to
// different buckets because they have different keys.
CAddrInfo info2 = CAddrInfo(addr2, source1);
AddrInfo info2 = AddrInfo(addr2, source1);
BOOST_CHECK(info1.GetKey() != info2.GetKey());
BOOST_CHECK(info1.GetTriedBucket(nKey1, asmap) != info2.GetTriedBucket(nKey1, asmap));
std::set<int> buckets;
for (int j = 0; j < 255; j++) {
CAddrInfo infoj = CAddrInfo(
AddrInfo infoj = AddrInfo(
CAddress(ResolveService("101." + ToString(j) + ".1.1"), NODE_NONE),
ResolveIP("101." + ToString(j) + ".1.1"));
int bucket = infoj.GetTriedBucket(nKey1, asmap);
@ -663,7 +725,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket)
buckets.clear();
for (int j = 0; j < 255; j++) {
CAddrInfo infoj = CAddrInfo(
AddrInfo infoj = AddrInfo(
CAddress(ResolveService("250." + ToString(j) + ".1.1"), NODE_NONE),
ResolveIP("250." + ToString(j) + ".1.1"));
int bucket = infoj.GetTriedBucket(nKey1, asmap);
@ -676,14 +738,14 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_tried_bucket)
BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket)
{
CAddrManTest addrman;
AddrManTest addrman;
CAddress addr1 = CAddress(ResolveService("250.1.2.1", 8333), NODE_NONE);
CAddress addr2 = CAddress(ResolveService("250.1.2.1", 9999), NODE_NONE);
CNetAddr source1 = ResolveIP("250.1.2.1");
CAddrInfo info1 = CAddrInfo(addr1, source1);
AddrInfo info1 = AddrInfo(addr1, source1);
uint256 nKey1 = (uint256)(CHashWriter(SER_GETHASH, 0) << 1).GetHash();
uint256 nKey2 = (uint256)(CHashWriter(SER_GETHASH, 0) << 2).GetHash();
@ -699,13 +761,13 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket)
BOOST_CHECK(info1.GetNewBucket(nKey1, asmap) != info1.GetNewBucket(nKey2, asmap));
// Test: Ports should not affect bucket placement in the addr
CAddrInfo info2 = CAddrInfo(addr2, source1);
AddrInfo info2 = AddrInfo(addr2, source1);
BOOST_CHECK(info1.GetKey() != info2.GetKey());
BOOST_CHECK_EQUAL(info1.GetNewBucket(nKey1, asmap), info2.GetNewBucket(nKey1, asmap));
std::set<int> buckets;
for (int i = 0; i < 255; i++) {
CAddrInfo infoi = CAddrInfo(
AddrInfo infoi = AddrInfo(
CAddress(ResolveService("250.1.1." + ToString(i)), NODE_NONE),
ResolveIP("250.1.1." + ToString(i)));
int bucket = infoi.GetNewBucket(nKey1, asmap);
@ -717,7 +779,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket)
buckets.clear();
for (int j = 0; j < 4 * 255; j++) {
CAddrInfo infoj = CAddrInfo(CAddress(
AddrInfo infoj = AddrInfo(CAddress(
ResolveService(
ToString(250 + (j / 255)) + "." + ToString(j % 256) + ".1.1"), NODE_NONE),
ResolveIP("251.4.1.1"));
@ -730,7 +792,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket)
buckets.clear();
for (int p = 0; p < 255; p++) {
CAddrInfo infoj = CAddrInfo(
AddrInfo infoj = AddrInfo(
CAddress(ResolveService("250.1.1.1"), NODE_NONE),
ResolveIP("101." + ToString(p) + ".1.1"));
int bucket = infoj.GetNewBucket(nKey1, asmap);
@ -742,7 +804,7 @@ BOOST_AUTO_TEST_CASE(caddrinfo_get_new_bucket)
buckets.clear();
for (int p = 0; p < 255; p++) {
CAddrInfo infoj = CAddrInfo(
AddrInfo infoj = AddrInfo(
CAddress(ResolveService("250.1.1.1"), NODE_NONE),
ResolveIP("250." + ToString(p) + ".1.1"));
int bucket = infoj.GetNewBucket(nKey1, asmap);
@ -758,9 +820,9 @@ BOOST_AUTO_TEST_CASE(addrman_serialization)
{
std::vector<bool> asmap1 = FromBytes(raw_tests::asmap, sizeof(raw_tests::asmap) * 8);
auto addrman_asmap1 = std::make_unique<CAddrManTest>(true, asmap1);
auto addrman_asmap1_dup = std::make_unique<CAddrManTest>(true, asmap1);
auto addrman_noasmap = std::make_unique<CAddrManTest>();
auto addrman_asmap1 = std::make_unique<AddrManTest>(true, asmap1);
auto addrman_asmap1_dup = std::make_unique<AddrManTest>(true, asmap1);
auto addrman_noasmap = std::make_unique<AddrManTest>();
CDataStream stream(SER_NETWORK, PROTOCOL_VERSION);
CAddress addr = CAddress(ResolveService("250.1.1.1"), NODE_NONE);
@ -790,8 +852,8 @@ BOOST_AUTO_TEST_CASE(addrman_serialization)
BOOST_CHECK(bucketAndEntry_asmap1.second != bucketAndEntry_noasmap.second);
// deserializing non-asmaped peers.dat to asmaped addrman
addrman_asmap1 = std::make_unique<CAddrManTest>(true, asmap1);
addrman_noasmap = std::make_unique<CAddrManTest>();
addrman_asmap1 = std::make_unique<AddrManTest>(true, asmap1);
addrman_noasmap = std::make_unique<AddrManTest>();
addrman_noasmap->Add({addr}, default_source);
stream << *addrman_noasmap;
stream >> *addrman_asmap1;
@ -802,8 +864,8 @@ BOOST_AUTO_TEST_CASE(addrman_serialization)
BOOST_CHECK(bucketAndEntry_asmap1_deser.second == bucketAndEntry_asmap1_dup.second);
// used to map to different buckets, now maps to the same bucket.
addrman_asmap1 = std::make_unique<CAddrManTest>(true, asmap1);
addrman_noasmap = std::make_unique<CAddrManTest>();
addrman_asmap1 = std::make_unique<AddrManTest>(true, asmap1);
addrman_noasmap = std::make_unique<AddrManTest>();
CAddress addr1 = CAddress(ResolveService("250.1.1.1"), NODE_NONE);
CAddress addr2 = CAddress(ResolveService("250.2.1.1"), NODE_NONE);
addrman_noasmap->Add({addr, addr2}, default_source);
@ -823,7 +885,7 @@ BOOST_AUTO_TEST_CASE(remove_invalid)
{
// Confirm that invalid addresses are ignored in unserialization.
auto addrman = std::make_unique<CAddrManTest>();
auto addrman = std::make_unique<AddrManTest>();
CDataStream stream(SER_NETWORK, PROTOCOL_VERSION);
const CAddress new1{ResolveService("5.5.5.5"), NODE_NONE};
@ -855,19 +917,19 @@ BOOST_AUTO_TEST_CASE(remove_invalid)
BOOST_REQUIRE(pos + sizeof(tried2_raw_replacement) <= stream.size());
memcpy(stream.data() + pos, tried2_raw_replacement, sizeof(tried2_raw_replacement));
addrman = std::make_unique<CAddrManTest>();
addrman = std::make_unique<AddrManTest>();
stream >> *addrman;
BOOST_CHECK_EQUAL(addrman->size(), 2);
}
BOOST_AUTO_TEST_CASE(addrman_selecttriedcollision)
{
CAddrManTest addrman;
AddrManTest addrman;
BOOST_CHECK(addrman.size() == 0);
// Empty addrman should return blank addrman info.
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
// Add twenty two addresses.
CNetAddr source = ResolveIP("252.2.2.2");
@ -878,7 +940,7 @@ BOOST_AUTO_TEST_CASE(addrman_selecttriedcollision)
// No collisions yet.
BOOST_CHECK(addrman.size() == i);
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
}
// Ensure Good handles duplicates well.
@ -887,14 +949,14 @@ BOOST_AUTO_TEST_CASE(addrman_selecttriedcollision)
addrman.Good(addr);
BOOST_CHECK(addrman.size() == 22);
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
}
}
BOOST_AUTO_TEST_CASE(addrman_noevict)
{
CAddrManTest addrman;
AddrManTest addrman;
// Add 35 addresses.
CNetAddr source = ResolveIP("252.2.2.2");
@ -905,7 +967,7 @@ BOOST_AUTO_TEST_CASE(addrman_noevict)
// No collision yet.
BOOST_CHECK(addrman.size() == i);
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
}
// Collision between 36 and 19.
@ -914,11 +976,11 @@ BOOST_AUTO_TEST_CASE(addrman_noevict)
addrman.Good(addr36);
BOOST_CHECK(addrman.size() == 36);
BOOST_CHECK_EQUAL(addrman.SelectTriedCollision().ToString(), "250.1.1.19:0");
BOOST_CHECK_EQUAL(addrman.SelectTriedCollision().first.ToString(), "250.1.1.19:0");
// 36 should be discarded and 19 not evicted.
addrman.ResolveCollisions();
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
// Lets create two collisions.
for (unsigned int i = 37; i < 59; i++) {
@ -927,7 +989,7 @@ BOOST_AUTO_TEST_CASE(addrman_noevict)
addrman.Good(addr);
BOOST_CHECK(addrman.size() == i);
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
}
// Cause a collision.
@ -936,26 +998,26 @@ BOOST_AUTO_TEST_CASE(addrman_noevict)
addrman.Good(addr59);
BOOST_CHECK(addrman.size() == 59);
BOOST_CHECK_EQUAL(addrman.SelectTriedCollision().ToString(), "250.1.1.10:0");
BOOST_CHECK_EQUAL(addrman.SelectTriedCollision().first.ToString(), "250.1.1.10:0");
// Cause a second collision.
BOOST_CHECK(!addrman.Add({CAddress(addr36, NODE_NONE)}, source));
addrman.Good(addr36);
BOOST_CHECK(addrman.size() == 59);
BOOST_CHECK(addrman.SelectTriedCollision().ToString() != "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() != "[::]:0");
addrman.ResolveCollisions();
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
}
BOOST_AUTO_TEST_CASE(addrman_evictionworks)
{
CAddrManTest addrman;
AddrManTest addrman;
BOOST_CHECK(addrman.size() == 0);
// Empty addrman should return blank addrman info.
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
// Add 35 addresses
CNetAddr source = ResolveIP("252.2.2.2");
@ -966,7 +1028,7 @@ BOOST_AUTO_TEST_CASE(addrman_evictionworks)
// No collision yet.
BOOST_CHECK(addrman.size() == i);
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
}
// Collision between 36 and 19.
@ -975,7 +1037,7 @@ BOOST_AUTO_TEST_CASE(addrman_evictionworks)
addrman.Good(addr);
BOOST_CHECK_EQUAL(addrman.size(), 36);
CAddrInfo info = addrman.SelectTriedCollision();
auto info = addrman.SelectTriedCollision().first;
BOOST_CHECK_EQUAL(info.ToString(), "250.1.1.19:0");
// Ensure test of address fails, so that it is evicted.
@ -983,28 +1045,28 @@ BOOST_AUTO_TEST_CASE(addrman_evictionworks)
// Should swap 36 for 19.
addrman.ResolveCollisions();
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
// If 36 was swapped for 19, then this should cause no collisions.
BOOST_CHECK(!addrman.Add({CAddress(addr, NODE_NONE)}, source));
addrman.Good(addr);
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
// If we insert 19 it should collide with 36
CService addr19 = ResolveService("250.1.1.19");
BOOST_CHECK(!addrman.Add({CAddress(addr19, NODE_NONE)}, source));
addrman.Good(addr19);
BOOST_CHECK_EQUAL(addrman.SelectTriedCollision().ToString(), "250.1.1.36:0");
BOOST_CHECK_EQUAL(addrman.SelectTriedCollision().first.ToString(), "250.1.1.36:0");
addrman.ResolveCollisions();
BOOST_CHECK(addrman.SelectTriedCollision().ToString() == "[::]:0");
BOOST_CHECK(addrman.SelectTriedCollision().first.ToString() == "[::]:0");
}
BOOST_AUTO_TEST_CASE(load_addrman)
{
CAddrManUncorrupted addrmanUncorrupted;
AddrManUncorrupted addrmanUncorrupted;
CService addr1, addr2, addr3;
BOOST_CHECK(Lookup("250.7.1.1", addr1, 8333, false));
@ -1023,7 +1085,7 @@ BOOST_AUTO_TEST_CASE(load_addrman)
// Test that the de-serialization does not throw an exception.
CDataStream ssPeers1 = AddrmanToStream(addrmanUncorrupted);
bool exceptionThrown = false;
CAddrMan addrman1(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 100);
AddrMan addrman1(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 100);
BOOST_CHECK(addrman1.size() == 0);
try {
@ -1040,21 +1102,21 @@ BOOST_AUTO_TEST_CASE(load_addrman)
// Test that ReadFromStream creates an addrman with the correct number of addrs.
CDataStream ssPeers2 = AddrmanToStream(addrmanUncorrupted);
CAddrMan addrman2(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 100);
AddrMan addrman2(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 100);
BOOST_CHECK(addrman2.size() == 0);
BOOST_CHECK(ReadFromStream(addrman2, ssPeers2));
ReadFromStream(addrman2, ssPeers2);
BOOST_CHECK(addrman2.size() == 3);
}
BOOST_AUTO_TEST_CASE(load_addrman_corrupted)
{
CAddrManCorrupted addrmanCorrupted;
AddrManCorrupted addrmanCorrupted;
// Test that the de-serialization of corrupted addrman throws an exception.
CDataStream ssPeers1 = AddrmanToStream(addrmanCorrupted);
bool exceptionThrown = false;
CAddrMan addrman1(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 100);
AddrMan addrman1(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 100);
BOOST_CHECK(addrman1.size() == 0);
try {
unsigned char pchMsgTmp[4];
@ -1070,9 +1132,9 @@ BOOST_AUTO_TEST_CASE(load_addrman_corrupted)
// Test that ReadFromStream fails if peers.dat is corrupt
CDataStream ssPeers2 = AddrmanToStream(addrmanCorrupted);
CAddrMan addrman2(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 100);
AddrMan addrman2(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 100);
BOOST_CHECK(addrman2.size() == 0);
BOOST_CHECK(!ReadFromStream(addrman2, ssPeers2));
BOOST_CHECK_THROW(ReadFromStream(addrman2, ssPeers2), std::ios_base::failure);
}
BOOST_AUTO_TEST_SUITE_END()

144
src/test/fuzz/addrman.cpp
View File

@ -4,8 +4,10 @@
#include <addrdb.h>
#include <addrman.h>
#include <addrman_impl.h>
#include <chainparams.h>
#include <merkleblock.h>
#include <random.h>
#include <test/fuzz/FuzzedDataProvider.h>
#include <test/fuzz/fuzz.h>
#include <test/fuzz/util.h>
@ -23,26 +25,25 @@ void initialize_addrman()
SelectParams(CBaseChainParams::REGTEST);
}
class CAddrManDeterministic : public CAddrMan
FUZZ_TARGET_INIT(data_stream_addr_man, initialize_addrman)
{
public:
FuzzedDataProvider& m_fuzzed_data_provider;
explicit CAddrManDeterministic(std::vector<bool> asmap, FuzzedDataProvider& fuzzed_data_provider)
: CAddrMan(std::move(asmap), /* deterministic */ true, /* consistency_check_ratio */ 0)
, m_fuzzed_data_provider(fuzzed_data_provider)
{
WITH_LOCK(cs, insecure_rand = FastRandomContext{ConsumeUInt256(fuzzed_data_provider)});
FuzzedDataProvider fuzzed_data_provider{buffer.data(), buffer.size()};
CDataStream data_stream = ConsumeDataStream(fuzzed_data_provider);
AddrMan addr_man(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
try {
ReadFromStream(addr_man, data_stream);
} catch (const std::exception&) {
}
}
/**
* Generate a random address. Always returns a valid address.
*/
CNetAddr RandAddr() EXCLUSIVE_LOCKS_REQUIRED(cs)
CNetAddr RandAddr(FuzzedDataProvider& fuzzed_data_provider, FastRandomContext& fast_random_context)
{
CNetAddr addr;
if (m_fuzzed_data_provider.remaining_bytes() > 1 && m_fuzzed_data_provider.ConsumeBool()) {
addr = ConsumeNetAddr(m_fuzzed_data_provider);
if (fuzzed_data_provider.remaining_bytes() > 1 && fuzzed_data_provider.ConsumeBool()) {
addr = ConsumeNetAddr(fuzzed_data_provider);
} else {
// The networks [1..6] correspond to CNetAddr::BIP155Network (private).
static const std::map<uint8_t, uint8_t> net_len_map = {{1, ADDR_IPV4_SIZE},
@ -50,7 +51,7 @@ public:
{4, ADDR_TORV3_SIZE},
{5, ADDR_I2P_SIZE},
{6, ADDR_CJDNS_SIZE}};
uint8_t net = insecure_rand.randrange(5) + 1; // [1..5]
uint8_t net = fast_random_context.randrange(5) + 1; // [1..5]
if (net == 3) {
net = 6;
}
@ -58,7 +59,7 @@ public:
CDataStream s(SER_NETWORK, PROTOCOL_VERSION | ADDRV2_FORMAT);
s << net;
s << insecure_rand.randbytes(net_len_map.at(net));
s << fast_random_context.randbytes(net_len_map.at(net));
s >> addr;
}
@ -73,44 +74,50 @@ public:
return addr;
}
/**
* Fill this addrman with lots of addresses from lots of sources.
*/
void Fill()
/** Fill addrman with lots of addresses from lots of sources. */
void FillAddrman(AddrMan& addrman, FuzzedDataProvider& fuzzed_data_provider)
{
LOCK(cs);
// Add some of the addresses directly to the "tried" table.
// Add a fraction of the addresses to the "tried" table.
// 0, 1, 2, 3 corresponding to 0%, 100%, 50%, 33%
const size_t n = m_fuzzed_data_provider.ConsumeIntegralInRange<size_t>(0, 3);
const size_t n = fuzzed_data_provider.ConsumeIntegralInRange<size_t>(0, 3);
const size_t num_sources = m_fuzzed_data_provider.ConsumeIntegralInRange<size_t>(1, 50);
const size_t num_sources = fuzzed_data_provider.ConsumeIntegralInRange<size_t>(1, 50);
CNetAddr prev_source;
// Use insecure_rand inside the loops instead of m_fuzzed_data_provider because when
// the latter is exhausted it just returns 0.
// Generate a FastRandomContext seed to use inside the loops instead of
// fuzzed_data_provider. When fuzzed_data_provider is exhausted it
// just returns 0.
FastRandomContext fast_random_context{ConsumeUInt256(fuzzed_data_provider)};
for (size_t i = 0; i < num_sources; ++i) {
const auto source = RandAddr();
const size_t num_addresses = insecure_rand.randrange(500) + 1; // [1..500]
const auto source = RandAddr(fuzzed_data_provider, fast_random_context);
const size_t num_addresses = fast_random_context.randrange(500) + 1; // [1..500]
for (size_t j = 0; j < num_addresses; ++j) {
const auto addr = CAddress{CService{RandAddr(), 8333}, NODE_NETWORK};
const auto time_penalty = insecure_rand.randrange(100000001);
Add_(addr, source, time_penalty);
const auto addr = CAddress{CService{RandAddr(fuzzed_data_provider, fast_random_context), 8333}, NODE_NETWORK};
const auto time_penalty = fast_random_context.randrange(100000001);
addrman.Add({addr}, source, time_penalty);
if (n > 0 && mapInfo.size() % n == 0) {
Good_(addr, false, GetTime());
if (n > 0 && addrman.size() % n == 0) {
addrman.Good(addr, GetTime());
}
// Add 10% of the addresses from more than one source.
if (insecure_rand.randrange(10) == 0 && prev_source.IsValid()) {
Add_(addr, prev_source, time_penalty);
if (fast_random_context.randrange(10) == 0 && prev_source.IsValid()) {
addrman.Add({addr}, prev_source, time_penalty);
}
}
prev_source = source;
}
}
class AddrManDeterministic : public AddrMan
{
public:
explicit AddrManDeterministic(std::vector<bool> asmap, FuzzedDataProvider& fuzzed_data_provider)
: AddrMan(std::move(asmap), /* deterministic */ true, /* consistency_check_ratio */ 0)
{
WITH_LOCK(m_impl->cs, m_impl->insecure_rand = FastRandomContext{ConsumeUInt256(fuzzed_data_provider)});
}
/**
* Compare with another AddrMan.
* This compares:
@ -118,46 +125,51 @@ public:
* - vvNew entries refer to the same addresses
* - vvTried entries refer to the same addresses
*/
bool operator==(const CAddrManDeterministic& other) const
bool operator==(const AddrManDeterministic& other) const
{
LOCK2(cs, other.cs);
LOCK2(m_impl->cs, other.m_impl->cs);
if (mapInfo.size() != other.mapInfo.size() || nNew != other.nNew ||
nTried != other.nTried) {
if (m_impl->mapInfo.size() != other.m_impl->mapInfo.size() || m_impl->nNew != other.m_impl->nNew ||
m_impl->nTried != other.m_impl->nTried) {
return false;
}
// Check that all values in `mapInfo` are equal to all values in `other.mapInfo`.
// Keys may be different.
using CAddrInfoHasher = std::function<size_t(const CAddrInfo&)>;
using CAddrInfoEq = std::function<bool(const CAddrInfo&, const CAddrInfo&)>;
CNetAddrHash netaddr_hasher;
CAddrInfoHasher addrinfo_hasher = [&netaddr_hasher](const CAddrInfo& a) {
return netaddr_hasher(static_cast<CNetAddr>(a)) ^ netaddr_hasher(a.source) ^
a.nLastSuccess ^ a.nAttempts ^ a.nRefCount ^ a.fInTried;
auto addrinfo_hasher = [](const AddrInfo& a) {
CSipHasher hasher(0, 0);
auto addr_key = a.GetKey();
auto source_key = a.source.GetAddrBytes();
hasher.Write(a.nLastSuccess);
hasher.Write(a.nAttempts);
hasher.Write(a.nRefCount);
hasher.Write(a.fInTried);
hasher.Write(a.GetNetwork());
hasher.Write(a.source.GetNetwork());
hasher.Write(addr_key.size());
hasher.Write(source_key.size());
hasher.Write(addr_key.data(), addr_key.size());
hasher.Write(source_key.data(), source_key.size());
return (size_t)hasher.Finalize();
};
CAddrInfoEq addrinfo_eq = [](const CAddrInfo& lhs, const CAddrInfo& rhs) {
return static_cast<CNetAddr>(lhs) == static_cast<CNetAddr>(rhs) &&
lhs.source == rhs.source && lhs.nLastSuccess == rhs.nLastSuccess &&
lhs.nAttempts == rhs.nAttempts && lhs.nRefCount == rhs.nRefCount &&
lhs.fInTried == rhs.fInTried;
auto addrinfo_eq = [](const AddrInfo& lhs, const AddrInfo& rhs) {
return std::tie(static_cast<const CService&>(lhs), lhs.source, lhs.nLastSuccess, lhs.nAttempts, lhs.nRefCount, lhs.fInTried) ==
std::tie(static_cast<const CService&>(rhs), rhs.source, rhs.nLastSuccess, rhs.nAttempts, rhs.nRefCount, rhs.fInTried);
};
using Addresses = std::unordered_set<CAddrInfo, CAddrInfoHasher, CAddrInfoEq>;
using Addresses = std::unordered_set<AddrInfo, decltype(addrinfo_hasher), decltype(addrinfo_eq)>;
const size_t num_addresses{mapInfo.size()};
const size_t num_addresses{m_impl->mapInfo.size()};
Addresses addresses{num_addresses, addrinfo_hasher, addrinfo_eq};
for (const auto& [id, addr] : mapInfo) {
for (const auto& [id, addr] : m_impl->mapInfo) {
addresses.insert(addr);
}
Addresses other_addresses{num_addresses, addrinfo_hasher, addrinfo_eq};
for (const auto& [id, addr] : other.mapInfo) {
for (const auto& [id, addr] : other.m_impl->mapInfo) {
other_addresses.insert(addr);
}
@ -165,14 +177,14 @@ public:
return false;
}
auto IdsReferToSameAddress = [&](int id, int other_id) EXCLUSIVE_LOCKS_REQUIRED(cs, other.cs) {
auto IdsReferToSameAddress = [&](int id, int other_id) EXCLUSIVE_LOCKS_REQUIRED(m_impl->cs, other.m_impl->cs) {
if (id == -1 && other_id == -1) {
return true;
}
if ((id == -1 && other_id != -1) || (id != -1 && other_id == -1)) {
return false;
}
return mapInfo.at(id) == other.mapInfo.at(other_id);
return m_impl->mapInfo.at(id) == other.m_impl->mapInfo.at(other_id);
};
// Check that `vvNew` contains the same addresses as `other.vvNew`. Notice - `vvNew[i][j]`
@ -180,7 +192,7 @@ public:
// themselves may differ between `vvNew` and `other.vvNew`.
for (size_t i = 0; i < ADDRMAN_NEW_BUCKET_COUNT; ++i) {
for (size_t j = 0; j < ADDRMAN_BUCKET_SIZE; ++j) {
if (!IdsReferToSameAddress(vvNew[i][j], other.vvNew[i][j])) {
if (!IdsReferToSameAddress(m_impl->vvNew[i][j], other.m_impl->vvNew[i][j])) {
return false;
}
}
@ -189,7 +201,7 @@ public:
// Same for `vvTried`.
for (size_t i = 0; i < ADDRMAN_TRIED_BUCKET_COUNT; ++i) {
for (size_t j = 0; j < ADDRMAN_BUCKET_SIZE; ++j) {
if (!IdsReferToSameAddress(vvTried[i][j], other.vvTried[i][j])) {
if (!IdsReferToSameAddress(m_impl->vvTried[i][j], other.m_impl->vvTried[i][j])) {
return false;
}
}
@ -211,7 +223,7 @@ FUZZ_TARGET_INIT(addrman, initialize_addrman)
FuzzedDataProvider fuzzed_data_provider(buffer.data(), buffer.size());
SetMockTime(ConsumeTime(fuzzed_data_provider));
std::vector<bool> asmap = ConsumeAsmap(fuzzed_data_provider);
auto addr_man_ptr = std::make_unique<CAddrManDeterministic>(asmap, fuzzed_data_provider);
auto addr_man_ptr = std::make_unique<AddrManDeterministic>(asmap, fuzzed_data_provider);
if (fuzzed_data_provider.ConsumeBool()) {
const std::vector<uint8_t> serialized_data{ConsumeRandomLengthByteVector(fuzzed_data_provider)};
CDataStream ds(serialized_data, SER_DISK, INIT_PROTO_VERSION);
@ -220,10 +232,10 @@ FUZZ_TARGET_INIT(addrman, initialize_addrman)
try {
ds >> *addr_man_ptr;
} catch (const std::ios_base::failure&) {
addr_man_ptr = std::make_unique<CAddrManDeterministic>(asmap, fuzzed_data_provider);
addr_man_ptr = std::make_unique<AddrManDeterministic>(asmap, fuzzed_data_provider);
}
}
CAddrManDeterministic& addr_man = *addr_man_ptr;
AddrManDeterministic& addr_man = *addr_man_ptr;
while (fuzzed_data_provider.ConsumeBool()) {
CallOneOf(
fuzzed_data_provider,
@ -293,12 +305,12 @@ FUZZ_TARGET_INIT(addrman_serdeser, initialize_addrman)
SetMockTime(ConsumeTime(fuzzed_data_provider));
std::vector<bool> asmap = ConsumeAsmap(fuzzed_data_provider);
CAddrManDeterministic addr_man1{asmap, fuzzed_data_provider};
CAddrManDeterministic addr_man2{asmap, fuzzed_data_provider};
AddrManDeterministic addr_man1{asmap, fuzzed_data_provider};
AddrManDeterministic addr_man2{asmap, fuzzed_data_provider};
CDataStream data_stream(SER_NETWORK, PROTOCOL_VERSION);
addr_man1.Fill();
FillAddrman(addr_man1, fuzzed_data_provider);
data_stream << addr_man1;
data_stream >> addr_man2;
assert(addr_man1 == addr_man2);

2
src/test/fuzz/connman.cpp
View File

@ -25,7 +25,7 @@ FUZZ_TARGET_INIT(connman, initialize_connman)
{
FuzzedDataProvider fuzzed_data_provider{buffer.data(), buffer.size()};
SetMockTime(ConsumeTime(fuzzed_data_provider));
CAddrMan addrman(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
AddrMan addrman(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
CConnman connman{fuzzed_data_provider.ConsumeIntegral<uint64_t>(), fuzzed_data_provider.ConsumeIntegral<uint64_t>(), addrman};
CNetAddr random_netaddr;
CNode random_node = ConsumeNode(fuzzed_data_provider);

27
src/test/fuzz/data_stream.cpp
View File

@ -1,27 +0,0 @@
// Copyright (c) 2020 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <addrdb.h>
#include <addrman.h>
#include <net.h>
#include <test/fuzz/FuzzedDataProvider.h>
#include <test/fuzz/fuzz.h>
#include <test/fuzz/util.h>
#include <test/util/setup_common.h>
#include <cstdint>
#include <vector>
void initialize_data_stream_addr_man()
{
static const auto testing_setup = MakeNoLogFileContext<>();
}
FUZZ_TARGET_INIT(data_stream_addr_man, initialize_data_stream_addr_man)
{
FuzzedDataProvider fuzzed_data_provider{buffer.data(), buffer.size()};
CDataStream data_stream = ConsumeDataStream(fuzzed_data_provider);
CAddrMan addr_man(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
ReadFromStream(addr_man, data_stream);
}

5
src/test/fuzz/deserialize.cpp
View File

@ -4,6 +4,7 @@
#include <addrdb.h>
#include <addrman.h>
#include <addrman_impl.h>
#include <blockencodings.h>
#include <blockfilter.h>
#include <chain.h>
@ -103,7 +104,7 @@ FUZZ_TARGET_DESERIALIZE(block_filter_deserialize, {
})
*/
FUZZ_TARGET_DESERIALIZE(addr_info_deserialize, {
CAddrInfo addr_info;
AddrInfo addr_info;
DeserializeFromFuzzingInput(buffer, addr_info);
})
FUZZ_TARGET_DESERIALIZE(block_file_info_deserialize, {
@ -189,7 +190,7 @@ FUZZ_TARGET_DESERIALIZE(blockmerkleroot, {
BlockMerkleRoot(block, &mutated);
})
FUZZ_TARGET_DESERIALIZE(addrman_deserialize, {
CAddrMan am(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
AddrMan am(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
DeserializeFromFuzzingInput(buffer, am);
})
FUZZ_TARGET_DESERIALIZE(blockheader_deserialize, {

2
src/test/fuzz/net.cpp
View File

@ -37,7 +37,7 @@ FUZZ_TARGET_INIT(net, initialize_net)
CallOneOf(
fuzzed_data_provider,
[&] {
CAddrMan addrman(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
AddrMan addrman(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
CConnman connman{fuzzed_data_provider.ConsumeIntegral<uint64_t>(), fuzzed_data_provider.ConsumeIntegral<uint64_t>(), addrman};
node.CloseSocketDisconnect(&connman);
},

2
src/test/util/setup_common.cpp
View File

@ -180,7 +180,7 @@ BasicTestingSetup::BasicTestingSetup(const std::string& chainName, const std::ve
SetupNetworking();
InitSignatureCache();
InitScriptExecutionCache();
m_node.addrman = std::make_unique<CAddrMan>(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
m_node.addrman = std::make_unique<AddrMan>(/* asmap */ std::vector<bool>(), /* deterministic */ false, /* consistency_check_ratio */ 0);
m_node.chain = interfaces::MakeChain(m_node);
// while g_wallet_init_interface is init here at very early stage
// we can't get rid of unique_ptr from wallet/contex.h

142
test/functional/feature_addrman.py Executable file
View File

@ -0,0 +1,142 @@
#!/usr/bin/env python3
# Copyright (c) 2021 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test addrman functionality"""
import os
import struct
from test_framework.messages import ser_uint256, hash256
from test_framework.p2p import MAGIC_BYTES
from test_framework.test_framework import BitcoinTestFramework
from test_framework.test_node import ErrorMatch
from test_framework.util import assert_equal
def serialize_addrman(
*,
format=1,
lowest_compatible=4,
net_magic="regtest",
bucket_key=1,
len_new=None,
len_tried=None,
mock_checksum=None,
):
new = []
tried = []
INCOMPATIBILITY_BASE = 32
r = MAGIC_BYTES[net_magic]
r += struct.pack("B", format)
r += struct.pack("B", INCOMPATIBILITY_BASE + lowest_compatible)
r += ser_uint256(bucket_key)
r += struct.pack("i", len_new or len(new))
r += struct.pack("i", len_tried or len(tried))
ADDRMAN_NEW_BUCKET_COUNT = 1 << 10
r += struct.pack("i", ADDRMAN_NEW_BUCKET_COUNT ^ (1 << 30))
for _ in range(ADDRMAN_NEW_BUCKET_COUNT):
r += struct.pack("i", 0)
checksum = hash256(r)
r += mock_checksum or checksum
return r
def write_addrman(peers_dat, **kwargs):
with open(peers_dat, "wb") as f:
f.write(serialize_addrman(**kwargs))
class AddrmanTest(BitcoinTestFramework):
def set_test_params(self):
self.num_nodes = 1
def run_test(self):
peers_dat = os.path.join(self.nodes[0].datadir, self.chain, "peers.dat")
init_error = lambda reason: (
f"Error: Invalid or corrupt peers.dat \\({reason}\\). If you believe this "
f"is a bug, please report it to {self.config['environment']['PACKAGE_BUGREPORT']}. "
f'As a workaround, you can move the file \\("{peers_dat}"\\) out of the way \\(rename, '
"move, or delete\\) to have a new one created on the next start."
)
self.log.info("Check that mocked addrman is valid")
self.stop_node(0)
write_addrman(peers_dat)
with self.nodes[0].assert_debug_log(["Loaded 0 addresses from peers.dat"]):
self.start_node(0, extra_args=["-checkaddrman=1"])
assert_equal(self.nodes[0].getnodeaddresses(), [])
self.log.info("Check that addrman from future cannot be read")
self.stop_node(0)
write_addrman(peers_dat, lowest_compatible=111)
self.nodes[0].assert_start_raises_init_error(
expected_msg=init_error(
"Unsupported format of addrman database: 1. It is compatible with "
"formats >=111, but the maximum supported by this version of "
f"{self.config['environment']['PACKAGE_NAME']} is 4.: (.+)"
),
match=ErrorMatch.FULL_REGEX,
)
self.log.info("Check that corrupt addrman cannot be read (EOF)")
self.stop_node(0)
with open(peers_dat, "wb") as f:
f.write(serialize_addrman()[:-1])
self.nodes[0].assert_start_raises_init_error(
expected_msg=init_error("CAutoFile::read: end of file.*"),
match=ErrorMatch.FULL_REGEX,
)
self.log.info("Check that corrupt addrman cannot be read (magic)")
self.stop_node(0)
write_addrman(peers_dat, net_magic="devnet")
self.nodes[0].assert_start_raises_init_error(
expected_msg=init_error("Invalid network magic number"),
match=ErrorMatch.FULL_REGEX,
)
self.log.info("Check that corrupt addrman cannot be read (checksum)")
self.stop_node(0)
write_addrman(peers_dat, mock_checksum=b"ab" * 32)
self.nodes[0].assert_start_raises_init_error(
expected_msg=init_error("Checksum mismatch, data corrupted"),
match=ErrorMatch.FULL_REGEX,
)
self.log.info("Check that corrupt addrman cannot be read (len_tried)")
self.stop_node(0)
write_addrman(peers_dat, len_tried=-1)
self.nodes[0].assert_start_raises_init_error(
expected_msg=init_error("Corrupt AddrMan serialization: nTried=-1, should be in \\[0, 16384\\]:.*"),
match=ErrorMatch.FULL_REGEX,
)
self.log.info("Check that corrupt addrman cannot be read (len_new)")
self.stop_node(0)
write_addrman(peers_dat, len_new=-1)
self.nodes[0].assert_start_raises_init_error(
expected_msg=init_error("Corrupt AddrMan serialization: nNew=-1, should be in \\[0, 65536\\]:.*"),
match=ErrorMatch.FULL_REGEX,
)
self.log.info("Check that corrupt addrman cannot be read (failed check)")
self.stop_node(0)
write_addrman(peers_dat, bucket_key=0)
self.nodes[0].assert_start_raises_init_error(
expected_msg=init_error("Corrupt data. Consistency check failed with code -16: .*"),
match=ErrorMatch.FULL_REGEX,
)
self.log.info("Check that missing addrman is recreated")
self.stop_node(0)
os.remove(peers_dat)
with self.nodes[0].assert_debug_log([
f'Creating peers.dat because the file was not found ("{peers_dat}")',
]):
self.start_node(0)
assert_equal(self.nodes[0].getnodeaddresses(), [])
if __name__ == "__main__":
AddrmanTest().main()

3
test/functional/feature_anchors.py
View File

@ -24,9 +24,6 @@ class AnchorsTest(BitcoinTestFramework):
def set_test_params(self):
self.num_nodes = 1
def setup_network(self):
self.setup_nodes()
def run_test(self):
node_anchors_path = os.path.join(
self.nodes[0].datadir, "regtest", "anchors.dat"

1
test/functional/p2p_addr_relay.py
View File

@ -148,7 +148,6 @@ class AddrTest(BitcoinTestFramework):
msg = self.setup_addr_msg(num_ipv4_addrs)
with self.nodes[0].assert_debug_log(
[
'Added {} addresses from 127.0.0.1: 0 tried'.format(num_ipv4_addrs),
'received: addr (301 bytes) peer=1',
]
):

3
test/functional/p2p_addrv2_relay.py
View File

@ -74,9 +74,6 @@ class AddrTest(BitcoinTestFramework):
addr_receiver = self.nodes[0].add_p2p_connection(AddrReceiver())
msg.addrs = ADDRS
with self.nodes[0].assert_debug_log([
# The I2P address is not added to node's own addrman because it has no
# I2P reachability (thus 10 - 1 = 9).
'Added 9 addresses from 127.0.0.1: 0 tried',
'received: addrv2 (159 bytes) peer=1',
]):
addr_source.send_and_ping(msg)

1
test/functional/test_runner.py
View File

@ -316,6 +316,7 @@ BASE_SCRIPTS = [
'p2p_add_connections.py',
'p2p_blockfilters.py',
'p2p_message_capture.py',
'feature_addrman.py',
'feature_asmap.py',
'feature_includeconf.py',
'mempool_unbroadcast.py',