Commit Graph

547 Commits

Author SHA1 Message Date
fanquake
c0f7ffd27c
Merge #19109: Only allow getdata of recently announced invs
f32c408f3a0b7e597977df2bc2cdc4ae298586e5 Make sure unconfirmed parents are requestable (Pieter Wuille)
c4626bcd211af08c85b6567ef07eeae333edba47 Drop setInventoryTxToSend based filtering (Pieter Wuille)
43f02ccbff9b137d59458da7a8afdb0bf80e127f Only respond to requests for recently announced transactions (Pieter Wuille)
b24a17f03982c9cd8fd6ec665b16e022374c96f0 Introduce constant for mempool-based relay separate from mapRelay caching (Pieter Wuille)
a9bc5638031a29abaa40284273a3507b345c31e9 Swap relay pool and mempool lookup (Pieter Wuille)

Pull request description:

  This implements the follow-up suggested here: https://github.com/bitcoin/bitcoin/pull/18861#issuecomment-627630111 . Instead of checking `setInventoryTxToSend`, maintain an explicit bloom filter with the 3500 most recently announced invs, and permit fetching any of these as long as they're in the relay pool or the mempool. In addition, permit relay from the mempool after just 2 minutes instead of 15.

  This:

  * Fixes the brief opportunity an attacker has to request unannounced invs just after the connection is established (pointed out by naumenkogs, see https://github.com/bitcoin/bitcoin/pull/18861#issuecomment-627627010).
  * Guarantees that locally resubmitted invs after `filterInventoryKnown` rolls over can still be requested (pointed out by luke-jr, see https://github.com/bitcoin/bitcoin/pull/18861#discussion_r419695831).

  It adds 37 KiB of filter per peer.

  This is also a step towards dropping the relay pool entirely and always relaying from the mempool directly (see #17303), but that is still blocked by dealing properly with NOTFOUNDs (see #18238).

ACKs for top commit:
  jnewbery:
    reACK f32c408f3
  jonatack:
    re-ACK f32c408 per `git range-diff f7c19e8 2da7ee3 f32c408` and redid the following: code review, thought about motivation, DoS and privacy aspects, debug build to check for warnings after updating Clang from 6 to 11 since last review.
  ajtowns:
    re-ACK f32c408f3a0b7e597977df2bc2cdc4ae298586e5

Tree-SHA512: aa05b9fd01bad59581c4ec91836a52d7415dc933fa49d4c4adced79aa25aaad51e11166357e8c8b29fbf6021a7401b98c21b850b5d8e8ad773fdb5d6608e1e85
2024-01-27 22:55:30 -06:00
MarcoFalke
3f97a3e212
Merge #20027: Use mockable time everywhere in net_processing
b6834e312a6a7bb395ec7266bc9469384639df96 Avoid 'timing mishap' warnings when mocking (Pieter Wuille)
ec3916f40a3fc644ecbbaaddef6258937c7fcfbc Use mockable time everywhere in net_processing (Pieter Wuille)

Pull request description:

  The fact that net_processing uses a mix of mockable tand non-mockable time functions made it hard to write functional tests for #19988.

  I'm opening this as a separate PR as I believe it's independently useful. In some ways this doesn't go quite as far as it could, as there are now several data structures that could be converted to `std::chrono` types as well now. I haven't done that here, but I'm happy to reconsider that.

ACKs for top commit:
  MarcoFalke:
    ACK b6834e312a 🌶
  jnewbery:
    utACK b6834e312a6a7bb395ec7266bc9469384639df96
  naumenkogs:
    utACK b6834e3

Tree-SHA512: 6528a167c57926ca12894e0c476826411baf5de2f7b01c2125b97e5f710e620f427bbb13f72bdfc3de59072e56a9c1447bce832f41c725e00e81fea019518f0e
2024-01-27 22:55:29 -06:00
Hennadii Stepanov
91e6817a18
Merge #17785: p2p: Unify Send and Receive protocol versions
ddefb5c0b759950942ac03f28c43b548af7b4033 p2p: Use the greatest common version in peer logic (Hennadii Stepanov)
e084d45562b94827b3a7873895882fcaae9f4d48 p2p: Remove SetCommonVersion() from VERACK handler (Hennadii Stepanov)
8d2026796a6f7add0c2cda9806e759817d1eae6f refactor: Rename local variable nSendVersion (Hennadii Stepanov)
e9a6d8b13b0558b17cdafbd32fd2663b4138ff11 p2p: Unify Send and Receive protocol versions (Hennadii Stepanov)

Pull request description:

  On master (6fef85bfa3cd7f76e83b8b57f9e4acd63eb664ec) `CNode` has two members to keep protocol version:
  - `nRecvVersion` for received messages
  - `nSendVersion` for messages to send

  After exchanging with `VERSION` and `VERACK` messages via protocol version `INIT_PROTO_VERSION`, both nodes set `nRecvVersion` _and_ `nSendVersion` to _the same_ value which is the greatest common protocol version.

  This PR:
  - replaces two `CNode` members, `nRecvVersion` `nSendVersion`, with `m_greatest_common_version`
  - removes duplicated getter and setter

  There is no change in behavior on the P2P network.

ACKs for top commit:
  jnewbery:
    ACK ddefb5c0b759950942ac03f28c43b548af7b4033
  naumenkogs:
    ACK ddefb5c0b759950942ac03f28c43b548af7b4033
  fjahr:
    Code review ACK ddefb5c0b759950942ac03f28c43b548af7b4033
  amitiuttarwar:
    code review but untested ACK ddefb5c0b7
  benthecarman:
    utACK `ddefb5c`

Tree-SHA512: 5305538dbaa5426b923b0afd20bdef4f248d310855d1d78427210c00716c67b7cb691515c421716b6157913e453076e293b10ff5fd2cd26a8e5375d42da7809d
2024-01-27 22:55:28 -06:00
fanquake
0c28db72c2
Merge #19879: [p2p] miscellaneous wtxid followups
a8a64acaf32ac21feeb885671772282b531ef9a2 [BroadcastTransaction] Remove unsafe move operator (Amiti Uttarwar)
125c0381266e0e05a408f8e1818501ab73d29110 [p2p] Remove dead code (Amiti Uttarwar)
fc66d0a65cdc52a3b259effe0c29b5eafb1b5ff5 [p2p] Check for nullptr before dereferencing pointer (Adam Jonas)
cb79b9dbf4cd06e17c8c65b36bf15c3ea2641de4 [mempool] Revert unbroadcast set to tracking just txid (Amiti Uttarwar)

Pull request description:

  Addresses some outstanding review comments from #18044

  - reverts unbroadcast txids to a set instead of a map (simpler, communicates intent better, takes less space, no efficiency advantages of map)
  - adds safety around two touchpoints (check for nullptr before dereferencing pointer, remove an inaccurate std::move operator)
  - removes some dead code

  Links to comments on wtxid PR: [1](https://github.com/bitcoin/bitcoin/pull/18044#discussion_r460495254) [2](https://github.com/bitcoin/bitcoin/pull/18044#discussion_r460496023) [3](https://github.com/bitcoin/bitcoin/pull/18044#discussion_r463532611)

  thanks to jnewbery & adamjonas for flagging these ! !

ACKs for top commit:
  sdaftuar:
    utACK a8a64acaf32ac21feeb885671772282b531ef9a2
  naumenkogs:
    utACK a8a64acaf32ac21feeb885671772282b531ef9a2
  jnewbery:
    utACK a8a64acaf32ac21feeb885671772282b531ef9a2

Tree-SHA512: 7be669cb30cc17fb9e06b50e636ef7887c6a27354697987e4e4d38dba4b8f50e175647587430cd9bc3295bec01ce8b1e6639a50a4249d8fff9b1ca1b9ead3277
2024-01-27 22:55:28 -06:00
Wladimir J. van der Laan
11d166d609
Merge #19610: p2p: refactor AlreadyHave(), CInv::type, INV/TX processing
fb56d37612dea6666e7da73d671311a697570dae p2p: ensure inv is GenMsgTx before ToGenTxid in inv processing (John Newbery)
aa3621385ee66c9dde5c632c0a79fba3a6ea2d62 test: use CInv::MSG_WITNESS_TX flag in p2p_segwit (Jon Atack)
24ee4f01eadb870435712950a1364cf0def06e9f p2p: make gtxid(.hash) and fAlreadyHave localvars const (Jon Atack)
b1c855453bf2634e7fd9b53c4a76a8536fc9865d p2p: use CInv block message helpers in net_processing.cpp (Jon Atack)
acd66421671e42a58e8e067868e1ab86268e3231 [net processing] Change AlreadyHaveTx() to take a GenTxid (John Newbery)
5fdfb80b861e0de3fcf8a57163b3f52af4b2df3b [net processing] Change AlreadyHaveBlock() to take block_hash argument (John Newbery)
430e183b89d00b4148f0b77a6fcacca2cd948202 [net processing] Remove mempool argument from AlreadyHaveBlock() (John Newbery)
42ca5618cae0fd9ef97d2006b17d896bc58cc17c [net processing] Split AlreadyHave() into separate block and tx functions (John Newbery)
39f1dc944554218911b0945fff7e6d06f3dab284 p2p: remove nFetchFlags from NetMsgType TX and INV processing (Jon Atack)
471714e1f024fb3b4892a7a8b34a76b83a13fa19 p2p: add CInv block message helper methods (Jon Atack)

Pull request description:

  Building on #19590 and the recent `wtxid` and `GenTxid` changes, this is a refactoring and cleanup PR to simplify and improve some of the net processing code.

  Some of the diffs are best reviewed with `-w` to ignore spacing.

  Co-authored by John Newbery.

ACKs for top commit:
  laanwj:
    Code review ACK fb56d37612dea6666e7da73d671311a697570dae
  jnewbery:
    utACK fb56d37612dea6666e7da73d671311a697570dae
  vasild:
    ACK fb56d3761

Tree-SHA512: ba39b58e6aaf850880a842fe5f6295e9f1870906ef690206acfc17140aae2ac854981e1066dbcd4238062478762fbd040ef772fdc2c50eea6869997c583e6a6d
2024-01-27 22:55:26 -06:00
MarcoFalke
d47344abe8
Merge #19514: [net/net processing] check banman pointer before dereferencing
ca3585a483ca5f6fc4cc54fd1530f89d13e5b7b0 [net/net processing] check banman pointer before dereferencing (John Newbery)

Pull request description:

  Although we currently don't do this, it should be possible to create a
  CConnman or PeerLogicValidation without a Banman instance. Therefore
  always check that banman exists before dereferencing the pointer.

  Also add comments to the m_banman members of CConnman and
  PeerLogicValidation to document that these may be nullptr.

ACKs for top commit:
  jonatack:
    ACK ca3585a
  theStack:
    ACK ca3585a483

Tree-SHA512: 726401c8921b9a502029ead34ae797473a1bc359d6e4e58dcbe3e25b70dde40bb100723be467fd3e2bf418892c493911998226de19c9d529d72034e3be26be48
2024-01-23 22:14:15 -06:00
MarcoFalke
3bb9504652
Merge #19347: [net] Make cs_inventory nonrecursive
e8a2822119233ade0de84f791a9e92918a3d6896 [net] Don't try to take cs_inventory before deleting CNode (John Newbery)
3556227ddd3365cfac43b307204d73058b2943f0 [net] Make cs_inventory a non-recursive mutex (John Newbery)
344e831de54f7b864f03a90f6cb19692eafcd463 [net processing] Remove PushBlockInventory and PushBlockHash (John Newbery)

Pull request description:

  - Remove PushBlockInventory() and PushBlockHash(). These are one-line functions that can easy be inlined into the calling code. Doing so also allows us to eliminate the one place that cs_inventory is recursively locked.
  - Make cs_inventory a nonrecursive mutex
  - Remove a redundant TRY_LOCK of cs_inventory when deleting CNode.

ACKs for top commit:
  sipa:
    utACK e8a2822119233ade0de84f791a9e92918a3d6896
  MarcoFalke:
    ACK e8a2822119233ade0de84f791a9e92918a3d6896 🍬
  hebasto:
    re-ACK e8a2822119233ade0de84f791a9e92918a3d6896

Tree-SHA512: dbc721d102cdef7b5827a8f2549daf8b54f543050266999a7ea56c9f36618565b71e31ce0beb1209ba2db43d15388be173355a03fb6db8ad24e2475b145050bd
2024-01-23 22:14:14 -06:00
fanquake
ddb14feb83
Merge #20119: BIP155 follow-ups
56f9dba015c592b8925795012e3061a710070a27 Only relay IPv4, IPv6, Tor addresses (Pieter Wuille)
79f3d9b932bf62b90995bce1cf4b0b1f0152d26d Mention BIP155 in doc/bips.md (Pieter Wuille)

Pull request description:

  This:
  * Documents BIP155 support in doc/bips.md
  * Restricts addrv2 relay to IPv4, IPv6, and Tor addresses. Relaying addresses in ranges that no network software has support for seems like a gratuitous spam vector.

ACKs for top commit:
  jonatack:
    ACK 56f9dba015c592b8925795012e3061a710070a27
  naumenkogs:
    ACK 56f9dba
  hebasto:
    ACK 56f9dba015c592b8925795012e3061a710070a27, verified both links.

Tree-SHA512: f0a2072b3d84a05cdbc7b961c18d7322a2e7260517f5306599ff52d8c728f9167de0a59a6d66cb95d84d69f3028680ce8bd05dab0db8c4f97938a287e5ce9631
2024-01-22 19:47:13 -06:00
MarcoFalke
48016a3fba
Merge #19914: refactor: Do not pass chain params to CheckForStaleTipAndEvictPeers twice
fa7e407b504bc60c77341f02636ed9d6a4b53d79 Do not pass chain params to CheckForStaleTipAndEvictPeers twice (MarcoFalke)

Pull request description:

  `PeerManager` already keeps a reference to the chain params as a member variable. No need to pass it in once again as a function parameter.

ACKs for top commit:
  naumenkogs:
    utACK fa7e407b504bc60c77341f02636ed9d6a4b53d79
  jnewbery:
    code review ACK fa7e407b504bc60c77341f02636ed9d6a4b53d79
  epson121:
    Code review ACK fa7e407b504bc60c77341f02636ed9d6a4b53d79

Tree-SHA512: 640c2d8adf9f1d54d0bfbdf81989064be2f5ba4b534d07d42258b372dc130f7b9c3fd087c7d28f0439678d124127f5d6f82f3139b1766f59f5ed661e7ac2a923
2024-01-22 19:47:12 -06:00
fanquake
0a27a2af73
Merge #19569: Enable fetching of orphan parents from wtxid peers
10b7a6d532148f880568c529e61a6d7edc7c91a9 refactor: make txmempool interface use GenTxid (Pieter Wuille)
5c124e17407a5b5824fec062b73a03a1030fa28c refactor: make FindTxForGetData use GenTxid (Pieter Wuille)
a2bfac893549e2d62708d8cda7071b4fe9750a2d refactor: use GenTxid in tx request functions (Pieter Wuille)
e65d115b725640eefb3bfa09786447816f7ca9cc test: request parents of orphan from wtxid relay peer (Anthony Towns)
900d7f6c075fd78e63503f31d267dbc16b3983d9 p2p: enable fetching of orphans from wtxid peers (Pieter Wuille)
9efd86a908cf09d9ddbadd3195f202635117d505 refactor: add GenTxid (=txid or wtxid) type and use it for tx request logic (Pieter Wuille)
d362f19355b36531a4a82094e0259f7f3db500a7 doc: list support for BIP 339 in doc/bips.md (Pieter Wuille)

Pull request description:

  This is based on https://github.com/bitcoin/bitcoin/pull/18044#discussion_r450687076.

  A new type `GenTxid` is added to protocol.h, which represents a tagged txid-or-wtxid. The tx request logic is updated to use these instead of uint256s, permitting per-announcement distinguishing of txid/wtxid (instead of assuming that everything we want to request from a wtxid peer is wtx). Then the restriction of orphan-parent requesting to non-wtxid peers is lifted.

  Also document BIP339 in doc/bips.md.

ACKs for top commit:
  jnewbery:
    Code review ACK 10b7a6d532148f880568c529e61a6d7edc7c91a9
  jonatack:
    ACK 10b7a6d532148f880568c529e61a6d7edc7c91a9
  ajtowns:
    ACK 10b7a6d532148f880568c529e61a6d7edc7c91a9 -- code review. Using gtxid to replace the is_txid_or_wtxid flag for the mempool functions is nice.
  naumenkogs:
    utACK 10b7a6d

Tree-SHA512: d518d13ffd71f8d2b3c175dc905362a7259689e6022a97a0b4f14f1f9fdd87475cf5af70cb12338d1e5d31b52c12e4faaea436114056a2ae9669cb506240758b
2024-01-22 19:44:36 -06:00
Konstantin Akimov
9b3d2e0c17
Merge bitcoin#18044: Use wtxid for transaction relay
This backport is marked as full, not partial, but it has only refactorings
and non-witness related changes.

Included commits are:
 - test: Update test framework p2p protocol version to 70016
 - Rename AddInventoryKnown() to AddKnownTx()
 - Add support for tx-relay via wtxid

    This adds a field to CNodeState that tracks whether to relay transactions with
    that peer via wtxid, instead of txid. As of this commit the field will always
    be false, but in a later commit we will add a way to negotiate turning this on
    via p2p messages exchanged with the peer.

 - Just pass a hash to AddInventoryKnown

    Since it's only used for transactions, there's no need to pass in an inv type.

 - Add wtxid to mempool unbroadcast tracking
2024-01-22 19:44:33 -06:00
fanquake
8ef196d9b8
Merge #19620: Add txids with non-standard inputs to reject filter
9f88ded82b2898ca63d44c08072f1ba52f0e18d7 test addition of unknown segwit spends to txid reject filter (Gregory Sanders)
7989901c7eb62ca28b3d1e5d5831041a7267e495 Add txids with non-standard inputs to reject filter (Suhas Daftuar)

Pull request description:

  Our policy checks for non-standard inputs depend only on the non-witness
  portion of a transaction: we look up the scriptPubKey of the input being
  spent from our UTXO set (which is covered by the input txid), and the p2sh
  checks only rely on the scriptSig portion of the input.

  Consequently it's safe to add txids of transactions that fail these checks to
  the reject filter, as the witness is irrelevant to the failure. This is helpful
  for any situation where we might request the transaction again via txid (either
  from txid-relay peers, or if we might fetch the transaction via txid due to
  parent-fetching of orphans).

  Further, in preparation for future witness versions being deployed on the
  network, ensure that WITNESS_UNKNOWN transactions are rejected in
  AreInputsStandard(), so that transactions spending v1 (or greater) witness
  outputs will fall into this category of having their txid added to the reject
  filter.

ACKs for top commit:
  ajtowns:
    ACK 9f88ded82b2898ca63d44c08072f1ba52f0e18d7 - code review
  jnewbery:
    Code review ACK 9f88ded82b2898ca63d44c08072f1ba52f0e18d7
  ariard:
    Code Review/Tested ACK 9f88ded
  naumenkogs:
    utACK 9f88ded82b2898ca63d44c08072f1ba52f0e18d7
  jonatack:
    ACK 9f88ded82b2

Tree-SHA512: 1e93c0a5b68cb432524780ffc0093db893911fdfed9e2ed17f888e59114cc75d2a07062aefad4e5ce2e87c9270886117a8abb3c78fb889c9b9f31967f1777148
2024-01-19 10:34:32 -06:00
Wladimir J. van der Laan
09f98c5880
Merge #19590: p2p, refactor: add CInv transaction message helpers; use in net processing
c251d710a4c2981c6d52362a9a89db84da3d4a67 p2p, refactoring: use CInv helpers in net_processing.cpp (Jon Atack)
4254cd9f8f2437a916b06db4d925ce4eff8c94b9 p2p: add CInv transaction message helper methods (Jon Atack)

Pull request description:

  Following the merge of wtxid relay in #18044, this is the first of three refactoring PRs (this one, #19610, and #19611) with no change in behavior, tightly scoped to ease review, to simplify the net processing code and improve encapsulation:

  - add `CInv` transaction message helper methods, defined in the class

  - use the new helpers in `net_processing.cpp` to simplify the code and improve encapsulation

  Test coverage is provided by the functional p2p tests, notably (from seeing which tests failed when breaking things to test coverage) `p2p_segwit`, `p2p_tx_download`, `p2p_feefilter`, and `p2p_permissions`.

ACKs for top commit:
  fjahr:
    Code review ACK c251d710a4c2981c6d52362a9a89db84da3d4a67
  laanwj:
    Code review ACK c251d710a4c2981c6d52362a9a89db84da3d4a67
  vasild:
    ACK c251d71
  theStack:
    Code-Review ACK c251d710a4c2981c6d52362a9a89db84da3d4a67
  hebasto:
    ACK c251d710a4c2981c6d52362a9a89db84da3d4a67, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: ead034b3c9e438909b4c5010c570d7930e69063c114290b051b7cebfd9bd5b19f573218bebe8a521256d32e830797f997adad3d85b4539c64ac5762b698e656d
2024-01-19 10:34:31 -06:00
Konstantin Akimov
852adb56ae
refactor: split llmq/utils to Quorum Calculation and llmq/options (#5790)
## Issue being fixed or feature implemented
`llmq/utils` has simple util code that used all over code base and also
have too heavy code for calculation quorums such as:
`GetAllQuorumMembers`, `EnsureQuorumConnections` and other.

These helpers for calculation quorums are used only by
evo/deterministicmns, evo/simplifiedmns and llmq/* modules, but
llmq/utils is included in many other modules for various trivial
helpers.



## What was done?
Prior work:
 - https://github.com/dashpay/dash/pull/5753
 - #5486
 See also #4798

This PR remove all non-quorum calculation code from llmq/utils.
Eventually it happens that easier to take everything out rather than
move Quorum Calculation to new place atm:
- new module llmq/options have a code related to various params, command
line options, spork-related etc
- llmq/utils is not included in various files which do not use any
llmq/utils code
 - helper `BuildCommitmentHash` goes to llmq/commitment
 - helper `BuildSignHash` goes to llmq/signing
- helper `GetLLMQParam` inlined since it's trivial (it has not been
trivial when introduced ages ago)
- removed dependency of `IsQuorumEnabled` on CQuorumManager which means
`quorumManager` deglobalization is done for 90%


## How Has This Been Tested?
 - Run unit functional tests
- updated circular dependencies
`test/lint/lint-circular-dependencies.sh`
- check that llmq/utils is not included without needs to calculate
Quorums Members
```
$ grep -r include src/ 2> /dev/null | grep -v .Po: | grep -vE 'llmq/utils.(h|cpp)': | grep llmq/utils  
src/evo/mnauth.cpp:#include <llmq/utils.h>
src/evo/deterministicmns.cpp:#include <llmq/utils.h>
src/llmq/quorums.cpp:#include <llmq/utils.h>
src/llmq/blockprocessor.cpp:#include <llmq/utils.h>
src/llmq/commitment.cpp:#include <llmq/utils.h>
src/llmq/debug.cpp:#include <llmq/utils.h>
src/llmq/dkgsessionhandler.cpp:#include <llmq/utils.h>
src/llmq/dkgsession.cpp:#include <llmq/utils.h>
src/llmq/dkgsessionmgr.cpp:#include <llmq/utils.h>
src/rpc/quorums.cpp:#include <llmq/utils.h>
```


## Breaking Changes
N/A

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
2024-01-17 19:56:41 -06:00
fanquake
65217a74a3
Merge #19260: p2p: disconnect peers that send filterclear + update existing filter msg disconnect logic
3a10d935ac8ebabdfd336569d943f042ff84b13e [p2p/refactor] move disconnect logic and remove misbehaving (gzhao408)
ff8c430c6589ea72b9e169455cf6437c8623cc52 [test] test disconnect for filterclear (gzhao408)
1c6b787e0319c44f0e0bede3f4a77ac7c2089db2 [netprocessing] disconnect node that sends filterclear (gzhao408)

Pull request description:

  Nodes that don't have bloomfilters turned on (i.e. no `NODE_BLOOM` service) should disconnect peers that send them `filterclear` P2P messages.

  Non-bloomfilter nodes already disconnect peers for [`filteradd` and `filterload`](19e919217e/src/net_processing.cpp (L2218)), but #8709 removed `filterclear` so it could be used to reset tx relay. This isn't needed now because using `feefilter` message is much better for this purpose (See #19204).

  Also refactors existing disconnect logic for `filteradd` and `filterload` into respective message handlers and removes banning for them.

ACKs for top commit:
  jnewbery:
    Code review ACK 3a10d935ac8ebabdfd336569d943f042ff84b13e
  naumenkogs:
    utACK 3a10d93
  gillichu:
    tested ACK: quick test_runner on macOS [`3a10d93`](3a10d935ac)
  MarcoFalke:
    re-ACK 3a10d935ac only change is replacing false with true 🚝

Tree-SHA512: 7aad8b3c0b0e776a47ad52544f0c1250feb242320f9a2962542f5905042f77e297a1486f8cdc3bf0fb93cd00c1ab66a67b2ec426eb6da3fe4cda56b5e623620f
2024-01-16 15:05:10 -06:00
UdjinM6
84657305c4
fix: bitcoin#18808 follow-up
The bug was introduced in v19 via 5118
2024-01-16 15:05:09 -06:00
Konstantin Akimov
f4fd1436a9
refactor: llmq/quorums no more depends on net_processing 2024-01-10 15:12:08 -06:00
Konstantin Akimov
c6a21bb1fb
refactor: llmq/blockprocessor no more depends on net_processing 2024-01-10 15:12:08 -06:00
Konstantin Akimov
f613f34343
refactor: llmq/chainlocks no more depends on net_processing 2024-01-10 15:12:07 -06:00
Konstantin Akimov
6ffc7451e3
refactor: evo/mnauth no more depends on net_processing 2024-01-10 15:12:07 -06:00
Konstantin Akimov
3e31f29c9d
refactor: governance/governance no more depends on net_processing 2024-01-10 15:12:07 -06:00
Konstantin Akimov
08d1b35ee4
refactor: spork no more depends on net_processing 2024-01-10 15:12:06 -06:00
Konstantin Akimov
91eca516e2
refactor: coinjoin/server no more depends on net_processing 2024-01-10 15:12:06 -06:00
Konstantin Akimov
1681eb8f3a
refactor: coinjoin/client no more depends on net_processing 2024-01-10 15:12:06 -06:00
Konstantin Akimov
051cdb3cae
refactor: new helpers in net_processing for external handlers
That's a prior work for removing circular dependencies over net_processing
2024-01-10 15:12:05 -06:00
Konstantin Akimov
d3da62fe6d
refactor: removed unused PeerMan from several classes 2024-01-10 15:12:03 -06:00
UdjinM6
b208e911c6
refactor: rename CJClientManager 2024-01-10 12:06:01 -06:00
Konstantin Akimov
e23b07bfa4
refactor: create a new global object dstxManager
Its class CDSTXManager has a lot of ex-static functions and data that
are actually meant to be an object
2024-01-10 12:06:00 -06:00
Konstantin Akimov
ad48a74a69
refactor: rename namespace CCoinJoin to CoinJoin 2024-01-10 12:05:59 -06:00
Wladimir J. van der Laan
796353adaa
Merge #19724: [net] Cleanup connection types- followups
eb1c5d090f4ef844dabc1e8bf3864c1afe1c791c [doc] Follow developer notes, add comment about missing default. (Amiti Uttarwar)
d5a57cef62ee9e9d30f7e3b80e178149ceeef67c [doc] Describe connection types in more depth. (Amiti Uttarwar)
4829b6fcc6489b445f80689af6c2a1a919f176b1 [refactor] Simplify connection type logic in ThreadOpenConnections (Amiti Uttarwar)
1e563aed785565af6b7aed7f7399c52205d8f19c [refactor] Simplify check for block-relay-only connection. (Amiti Uttarwar)
da3a0be61b025224231206cb4656e420453bfdeb [test] Add explicit tests that connection types get set correctly (Amiti Uttarwar)
1d74fc7df621b31d1b8adc9d7f53e7478d6e40b5 [trivial] Small style updates (Amiti Uttarwar)
ff6b9081add3a40d53b1cc1352b57eeb46e41d45 [doc] Explain address handling logic in process messages (Amiti Uttarwar)
dff16b184b1428a068d144e5e4dde7595b4729c0 [refactor] Restructure logic to check for addr relay. (Amiti Uttarwar)
a6ab1e81f964df131cfa0e11e07bedb3283b823f [net] Remove unnecessary default args on OpenNetworkConnection (Amiti Uttarwar)
8d6ff46f55f373e344278ab3f1ac27b1dba36623 scripted-diff: Rename `OUTBOUND` ConnectionType to `OUTBOUND_FULL_RELAY` (Amiti Uttarwar)

Pull request description:

  This PR addresses outstanding review comments from #19316. It further simplifies `net.cpp` complexity and adds documentation about design goals about different connection types.

ACKs for top commit:
  naumenkogs:
    ACK eb1c5d090f4ef844dabc1e8bf3864c1afe1c791c
  laanwj:
    Code review ACK eb1c5d090f4ef844dabc1e8bf3864c1afe1c791c

Tree-SHA512: 2fe14e428404c95661e5518c8c90db07ab5b9ebb1bac921b3bdf82b181f444fae379f8fc0a2b619e6b4693f01c55bd246fbd8c8eedadd96849a30de3161afee5
2024-01-09 08:15:36 -06:00
MarcoFalke
4143e359f7
(Partial) Merge #19725: [RPC] Add connection type to getpeerinfo, improve logs
a512925e19a70d7f6b80ac530a169f45ffaafa1c [doc] Release notes (Amiti Uttarwar)
50f94b34a33c954f6e207f509c93d33267a5c3e2 [rpc] Deprecate getpeerinfo addnode field (Amiti Uttarwar)
df091b9b509f0b10e4315c0bfa2da0cc0c31c22f [refactor] Rename test file to allow any getpeerinfo deprecations. (Amiti Uttarwar)
395acfa83a5436790c1a722a5609ac9d48df235f [rpc] Add connection type to getpeerinfo RPC, update tests (Amiti Uttarwar)
49c10a9ca40967d28ae16dfea9cccc6f3a6624a1 [log] Add connection type to log statement (Amiti Uttarwar)

Pull request description:

  After #19316, we can more directly expose information about the connection type on the `getpeerinfo` RPC. Doing so also makes the existing addnode field redundant, so this PR begins the process of deprecating this field.

  This PR also includes one commit that improves a log message, as both use a shared function to return the connection type as a string.

  Suggested by sdaftuar- https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468001604 & https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468018093

ACKs for top commit:
  jnewbery:
    Code review ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c.
  sipa:
    utACK a512925e19a70d7f6b80ac530a169f45ffaafa1c
  guggero:
    Tested and code review ACK a512925e.
  MarcoFalke:
    cr ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c 🌇
  promag:
    Code review ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c.

Tree-SHA512: 601a7a38aee235ee59aca690784f886dc2ae4e418b2e6422c4b58cd597376c00f74910f66920b08a08a0bec28bf8022e71a1435785ff6ba8a188954261aba78e
2024-01-09 08:15:36 -06:00
fanquake
2865a2d142
Merge #19316: [net] Cleanup logic around connection types
01e283068b9e6214f2d77a2f772a4244ebfe2274 [net] Remove unnecessary default args on CNode constructor (Amiti Uttarwar)
bc5d65b3ca41eebb1738fdda4451d1466e77772e [refactor] Remove IsOutboundDisconnectionCandidate (Amiti Uttarwar)
2f2e13b6c2c8741ca9d825eaaef736ede484bc85 [net/refactor] Simplify multiple-connection checks (Amiti Uttarwar)
7f7b83deb2427599c129f4ff581d4d045461e459 [net/refactor] Rework ThreadOpenConnections logic (Amiti Uttarwar)
35839e963bf61d2da0d12f5b8cea74ac0e0fbd7b [net] Fix bug where AddrFetch connections would be counted as outbound full relay (Amiti Uttarwar)
4972c21b671ff73f13a1b5053338b6abbdb471b5 [net/refactor] Clarify logic for selecting connections in ThreadOpenConnections (Amiti Uttarwar)
60156f5fc40d56bb532278f16ce632c5a8b8035e [net/refactor] Remove fInbound flag from CNode (Amiti Uttarwar)
7b322df6296609570e368e5f326979279041c11f [net/refactor] Remove m_addr_fetch member var from CNode (Amiti Uttarwar)
14923422b08ac4b21b35c426bf0e1b9e7c97983b [net/refactor] Remove fFeeler flag from CNode (Amiti Uttarwar)
49efac5cae7333c6700d9b737d09fae0f3f4d7fa [net/refactor] Remove m_manual_connection flag from CNode (Amiti Uttarwar)
d3698b5ee309cf0f0cdfb286d6b30a256d7deae5 [net/refactor] Add connection type as a member var to CNode (Amiti Uttarwar)
46578c03e92a55925308363ccdad04dcfc820d96 [doc] Describe different connection types (Amiti Uttarwar)
442abae2bac7bff85886143df01e14215532b974 [net/refactor] Add AddrFetch connections to ConnectionType enum (Amiti Uttarwar)
af59feb05235ecb85ec9d75b09c66e71268c9889 [net/refactor] Extract m_addr_known logic from initializer list (Amiti Uttarwar)
e1bc29812ddf1d946bc5acca406a7ed2dca064a6 [net/refactor] Add block relay only connections to ConnectionType enum (Amiti Uttarwar)
0e52a659a2de915fc3dce37fc8fac39be1c8b6fa [net/refactor] Add feeler connections to ConnectionType enum (Amiti Uttarwar)
1521c47438537e192230486dffcec0228a53878d [net/refactor] Add manual connections to ConnectionType enum (Amiti Uttarwar)
26304b4100201754fb32440bec3e3b78cd3f0e6d [net/refactor] Introduce an enum to distinguish type of connection (Amiti Uttarwar)
3f1b7140e95d0f8f958cb35f31c3d964c57e484d scripted-diff: Rename OneShot to AddrFetch (Amiti Uttarwar)

Pull request description:

  **This is part 1 of #19315, which enables the ability to test `outbound` and `block-relay-only` connections from the functional tests.** Please see that PR for more information of overall functionality.

  **This PR simplifies how we manage different connection types.** It introduces an enum with the various types of connections so we can explicitly define the connection type. The existing system relies on a series of independent flags, then has asserts scattered around to ensure that conflicting flags are not enabled at the same time. I find this approach to be both brittle and confusing. While making these changes, I found a small bug due to the silent assumptions.

  This PR also proposes a rename from `OneShot` to `AddrFetch`. I find the name `OneShot` to be very confusing, especially when we also have `onetry` manual connections. Everyone I've talked to offline has agreed that the name is confusing, so I propose a potential alternative. I think this is a good opportunity for a rename since I'm creating an enum to explicitly define the connection types.
  (some context for the unfamiliar: `oneshot` or `addrfetch` connections are short-lived connections created on startup. They connect to the seed peers, send a `getaddr` to solicit addresses, then close the connection.)

  Overview of this PR:
  * rename `oneshot` to `addrfetch`
  * introduce `ConnectionType` enum
  * one by one, add different connection types to the enum
  * expose the `conn_type` on CNode, and use this to reduce reliance on flags (& asserts)
  * fix the bug in counting different type of connections
  * some additional cleanup to simplify logic and make expectations explicit/inclusive rather than implicit/exclusive.

ACKs for top commit:
  jnewbery:
    utACK 01e283068b9e6214f2d77a2f772a4244ebfe2274
  laanwj:
    Code review ACK 01e283068b9e6214f2d77a2f772a4244ebfe2274, the commits are pretty straightforward to follow, and I think this is a move in the right direction overall
  vasild:
    ACK 01e283068
  sdaftuar:
    ACK 01e283068b9e6214f2d77a2f772a4244ebfe2274.
  fanquake:
    ACK 01e283068b9e6214f2d77a2f772a4244ebfe2274 - I don't have as much experience with the networking code but these changes look fairly straight forward, the new code seems more robust/understandable and the additional documentation is great. I'm glad that a followup branch is already underway. There might be some more review comments here later today, so keep an eye on the discussion, however I'm going to merge this now.
  jb55:
    wow this code was messy before... ACK 01e283068b9e6214f2d77a2f772a4244ebfe2274

Tree-SHA512: 7bb644a6ed5849913d777ebc2ff89133ca0fbef680355a9a344e07496a979e6f9ff21a958e8eea93dcd7d5c343682b0c7174b1a3de380a4247eaae73da436e15
2024-01-09 08:15:35 -06:00
MarcoFalke
cff1c7b2c3 Merge #21043: net: Avoid UBSan warning in ProcessMessage(...)
3ddbf22ed179a2db733af4b521bec5d2b13ebf4b util: Disallow negative mocktime (MarcoFalke)
f5f2f9716885e7548809e77f46b493c896a019bf net: Avoid UBSan warning in ProcessMessage(...) (practicalswift)

Pull request description:

  Avoid UBSan warning in `ProcessMessage(...)`.

  Context: https://github.com/bitcoin/bitcoin/pull/20380#issuecomment-770427182 (thanks Crypt-iQ!)

ACKs for top commit:
  MarcoFalke:
    re-ACK 3ddbf22ed179a2db733af4b521bec5d2b13ebf4b only change is adding patch written by me
  ajtowns:
    ACK 3ddbf22ed179a2db733af4b521bec5d2b13ebf4b -- code review only

Tree-SHA512: e8d7af0457ca86872b75a4e406c0a93aafd841c2962e244e147e748cc7ca118c56be0fdafe53765f4b291410030b2c3cc8f76f733b37a955d34fc885ab6037b9
2023-12-08 21:16:00 +03:00
fanquake
c7a6f378f5 Merge #21162: Net Processing: Move RelayTransaction() into PeerManager
680eb56d828ce358b4e000c140f5b247ff5e6179 [net processing] Don't pass CConnman to RelayTransactions (John Newbery)
a38a4e8f039dfabfd9435f3a63f1a9b56de086d6 [net processing] Move RelayTransaction into PeerManager (John Newbery)

Pull request description:

  This is the first part of #21160. It moves the RelayTransaction() function to be a member function of the PeerManager class. This is required in order to move the transaction inventory data into the Peer object, since Peer objects are only accessible from within PeerManager.

ACKs for top commit:
  ajtowns:
    ACK 680eb56d828ce358b4e000c140f5b247ff5e6179

Tree-SHA512: 8c93491a4392b6369bb7f090de326a63cd62a088de59026e202f226f64ded50a0cf1a95ed703328860f02a9d2f64d3a87ca1bca9a6075b978bd111d384766235
2023-12-08 21:16:00 +03:00
fanquake
2680860707
Merge #21728: remove executable flag for src/net_processing.cpp
f2f2541ee7ad36191515ff351b667fe12a2ab871 remove executable flag for src/net_processing.cpp (Sebastian Falbesoner)

Pull request description:

  The file permissions for `src/net_processing.cpp` have been changed in #21713, as discovered by fanquake (https://github.com/bitcoin/bitcoin/pull/21713#issuecomment-822245960). This PR removes the executable flag again.

ACKs for top commit:
  kiminuo:
    ACK f2f2541ee7ad36191515ff351b667fe12a2ab871 :)
  jnewbery:
    ACK f2f2541ee7ad36191515ff351b667fe12a2ab871
  promag:
    ACK f2f2541ee7ad36191515ff351b667fe12a2ab871.

Tree-SHA512: 1d5a62afb1152029e69fccea2ae53dcb262a91724a5c03dfc4de8c409b280814d0c211c2f9a71f1a6e927f4ed571ba4ac311de9de8ebb797eaf1051674241bdb
2023-12-03 20:44:58 -06:00
MarcoFalke
f4ea109e65 (partial) Merge bitcoin/bitcoin#21562: [net processing] Various tidying up of PeerManagerImpl ctor
fde1bf4f6136638e84cdf9806eedaae08e841bbf [net processing] Default initialize m_recent_confirmed_transactions (John Newbery)
37dcd12d539e4a875581fa049aa0f7fafeb932a4 scripted-diff: Rename recentRejects (John Newbery)
cd9902ac5054c01228d52616bf85f7196364d4ff [net processing] Default initialize recentRejects (John Newbery)
a28bfd1d4cfa523a6abf3832dbfd6183cd546944 [net processing] Default initialize m_stale_tip_check_time (John Newbery)
9190b01d8dcf03b74e9b9e1653688a97ac171b37 [net processing] Add Orphanage empty consistency check (John Newbery)

Pull request description:

  - Use default initialization of PeerManagerImpl members where possible
  - Remove unique_ptr indirection where it's not needed

ACKs for top commit:
  MarcoFalke:
    ACK fde1bf4f6136638e84cdf9806eedaae08e841bbf 👞
  theStack:
    re-ACK fde1bf4f6136638e84cdf9806eedaae08e841bbf

Tree-SHA512: 7ddedcc972df8e933e1fbe5c88b8ea17df89e1e58fc769518512c5540e49dc8eddb3f47e78d1329a6fc5644d2c1d11c981f681fd633f5218bfa4b3e6a86f3d7b
2023-12-03 20:25:16 -06:00
fanquake
25eb5cbb4b Merge #18861: Do not answer GETDATA for to-be-announced tx
2896c412fadbc03916a33028f4f50fd87ac48edb Do not answer GETDATA for to-be-announced tx (Pieter Wuille)
f2f32a3dee9a965c8198f9ddd3aaebc627c273e4 Push down use of cs_main into FindTxForGetData (Pieter Wuille)
c6131bf407c1ada78a0e5509a702bc7da0bfd57d Abstract logic to determine whether to answer tx GETDATA (Pieter Wuille)

Pull request description:

  This PR intends to improve transaction-origin privacy.

  In general, we should try to not leak information about what transactions we have (recently) learned about before deciding to announce them to our peers. There is a controlled transaction dissemination process that reveals our transactions to peers that has various safeguards for privacy (it's rate-limited, delayed & batched, deterministically sorted, ...), and ideally there is no way to test which transactions we have before that controlled process reveals them. The handling of the `mempool` BIP35 message has protections in this regard as well, as it would be an obvious way to bypass these protections (handled asynchronously after a delay, also deterministically sorted).

  However, currently, if we receive a GETDATA for a transaction that we have not yet announced to the requester, we will still respond to it if it was announced to *some* other peer already (because it needs to be in `mapRelay`, which only happens on the first announcement). This is a slight privacy leak.

  Thankfully, this seems easy to solve: `setInventontoryTxToSend` keeps track of the txids we have yet to announce to a peer - which almost(*) exactly corresponds to the transactions we know of that we haven't revealed to that peer. By checking whether a txid is in that set before responding to a GETDATA, we can filter these out.

  (*) Locally resubmitted or rebroadcasted transactions may end up in setInventoryTxToSend while the peer already knows we have them, which could result in us incorrectly claiming we don't have such transactions if coincidentally requested right after we schedule reannouncing them, but before they're actually INVed. This is made even harder by the fact that filterInventoryKnown will generally keep known reannouncements out of setInventoryTxToSend unless it overflows (which needs 50000 INVs in either direction before it happens).

  The condition for responding now becomes:

  ```
    (not in setInventoryTxToSend) AND
    (
      (in relay map) OR
      (
        (in mempool) AND
        (old enough that it could have expired from relay map) AND
        (older than our last getmempool response)
      )
    )
  ```

ACKs for top commit:
  naumenkogs:
    utACK 2896c41
  ajtowns:
    ACK 2896c412fadbc03916a33028f4f50fd87ac48edb
  amitiuttarwar:
    code review ACK 2896c412fa
  jonatack:
    ACK 2896c412fadbc03916 per `git diff 2b3f101 2896c41` only change since previous review is moving the recency check up to be verified first in `FindTxForGetData`, as it was originally in 353a391 (good catch), before looking up the transaction in the relay pool.
  jnewbery:
    code review ACK 2896c412fadbc03916a33028f4f50fd87ac48edb

Tree-SHA512: e7d5bc006e626f60a2c108a9334f3bbb67205ace04a7450a1e4d4db1d85922a7589e0524500b7b4953762cf70554c4a08eec62c7b38b486cbca3d86321600868
2023-12-03 20:01:26 -06:00
Wladimir J. van der Laan
ec11695199 Merge #18962: net processing: Only send a getheaders for one block in an INV
746736639e6d05acdb85c866d4c605c947d4c500 [net processing] Only send a getheaders for one block in an INV (John Newbery)

Pull request description:

  Headers-first is the primary method of announcement on the network. If a node fell back sending blocks by inv, it's probably for a re-org. The final block hash provided should be the highest, so send a getheaders and then fetch the blocks we need to catch up.

  Sending many GETHEADERS messages to the peer would cause them to send a large number of potentially large HEADERS messages with redundant data, which is a waste of bandwidth.

ACKs for top commit:
  sipa:
    utACK 746736639e6d05acdb85c866d4c605c947d4c500
  mzumsande:
    utACK 746736639e6d05acdb85c866d4c605c947d4c500 as per ajtowns' reasoning.
  naumenkogs:
    utACK 7467366
  ajtowns:
    ACK 746736639e6d05acdb85c866d4c605c947d4c500
  jonatack:
    ACK 746736639e6d05acdb85c866d4c605c947d4c500

Tree-SHA512: 59e243b80d3f0873709dfacb2e4ffba34689aad7de31ec7f69a64e0e3a0756235a0150e4082ff5de823949ba4411ee1aed2344b4749b62e0eb1ea906e41f5ea9
2023-12-03 20:01:26 -06:00
Odysseas Gabrielides
112564974d
refactor: deprecate non-deterministic IS support (#5553)
## Issue being fixed or feature implemented
Non-deterministic IS locks aren't used anymore since v18 dip24.
We should drop that support to make code simpler.

## What was done?
Dropped non-deterministic IS code, `evo_instantsend_tests` and
`feature_llmq_is_migration.py` (don't need it anymore), adjusted func
tests.

## How Has This Been Tested?
all tests, synced Testnet

## Breaking Changes

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_

---------

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: Konstantin Akimov <545784+knst@users.noreply.github.com>
2023-11-20 10:17:04 -06:00
Kittywhiskers Van Gogh
ee313525ad
refactor: decouple db hooks from CFlatDB-based C*Manager objects, migrate to *Store structs (#5555)
## Motivation

As highlighted in https://github.com/dashpay/dash-issues/issues/52,
decoupling of `CFlatDB`-interacting components from managers of objects
like `CGovernanceManager` and `CSporkManager` is a key task for
achieving deglobalization of Dash-specific components.

The design of `CFlatDB` as a flat database agent relies on hooking into
the object's state its meant to load and store, using its
(de)serialization routines and other miscellaneous functions (notably,
without defining an interface) to achieve those ends. This approach was
taken predominantly for components that want a single-file cache.

Because of the method it uses to hook into the object (templates and the
use of temporary objects), it explicitly prevented passing arguments
into the object constructor, an explicit requirement for storing
references to other components during construction. This, in turn,
created an explicit dependency on those same components being available
in the global context, which would block the backport of bitcoin#21866,
a requirement for future backports meant to achieve parity in
`assumeutxo` support.

The design of these objects made no separation between persistent (i.e.
cached) and ephemeral (i.e. generated/fetched during initialization or
state transitions) data and the design of `CFlatDB` attempts to "clean"
the database by breaching this separation and attempting to access this
ephemeral data.

This might be acceptable if it is contained within the manager itself,
like `CSporkManager`'s `CheckAndRemove()` but is utterly unacceptable
when it relies on other managers (that, as a reminder, are only
accessible through the global state because of restrictions caused by
existing design), like `CGovernanceManager`'s `UpdateCachesAndClean()`.

This pull request aims to separate the `CFlatDB`-interacting portions of
these managers into a struct, with `CFlatDB` interacting only with this
struct, while the manager inherits the struct and manages
load/store/update of the database through the `CFlatDB` instance
initialized within its scope, though the instance only has knowledge of
what is exposed through the limited parent struct.

## Additional information

* As regards to existing behaviour, `CFlatDB` is written entirely as a
header as it relies on templates to specialize itself for the object it
hooks into. Attempting to split the logic and function definitions into
separate files will require you to explicitly define template
specializations, which is tedious.

* `m_db` is defined as a pointer as you cannot instantiate a
forward-declared template (see [this Stack Overflow
answer](https://stackoverflow.com/a/12797282) for more information),
which is done when defined as a member in the object scope.

* The conditional cache flush predicating on RPC _not_ being in the
warm-up state has been replaced with unconditional flushing of the
database on object destruction (@UdjinM6, is this acceptable?)

## TODOs

This is a list of things that aren't within the scope of this pull
request but should be addressed in subsequent pull requests

* [ ] Definition of an interface that `CFlatDB` stores are expected to
implement
* [ ] Lock annotations for all potential uses of members protected by
the `cs` mutex in each manager object and store
* [ ] Additional comments documenting what each function and member does
* [ ] Deglobalization of affected managers

---------

Co-authored-by: Kittywhiskers Van Gogh <63189531+kittywhiskers@users.noreply.github.com>
2023-09-24 09:50:21 -05:00
Kittywhiskers Van Gogh
41a6613fba
refactor: subsume CoinJoin objects under CJContext, deglobalize coinJoin{ClientQueueManager,Server} (#5337)
## Motivation

CoinJoin's subsystems are initialized by variables and managers that
occupy the global context. The _extent_ to which these subsystems
entrench themselves into the codebase is difficult to assess and moving
them out of the global context forces us to enumerate the subsystems in
the codebase that rely on CoinJoin logic and enumerate the order in
which components are initialized and destroyed.

Keeping this in mind, the scope of this pull request aims to:

* Reduce the amount of CoinJoin-specific entities present in the global
scope
* Make the remaining usage of these entities in the global scope
explicit and easily searchable

## Additional Information

* The initialization of `CCoinJoinClientQueueManager` is dependent on
blocks-only mode being disabled (which can be alternatively interpreted
as enabling the relay of transactions). The same applies to
`CBlockPolicyEstimator`, which `CCoinJoinClientQueueManager` depends.

Therefore, `CCoinJoinClientQueueManager` is only initialized if
transaction relaying is enabled and so is its scheduled maintenance
task. This can be found by looking at `init.cpp`
[here](93f8df1c31/src/init.cpp (L1681-L1683)),
[here](93f8df1c31/src/init.cpp (L2253-L2255))
and
[here](93f8df1c31/src/init.cpp (L2326-L2327)).
  
For this reason, `CBlockPolicyEstimator` is not a member of `CJContext`
and its usage is fulfilled by passing it as a reference when
initializing the scheduling task.

* `CJClientManager` has not used `CConnman` or `CTxMemPool` as `const`
as existing code that is outside the scope of this PR would cast away
constness, which would be unacceptable. Furthermore, some logical paths
are taken that will grind to a halt if they are stored as `const`.

  Examples of such a call chains would be:

* `CJClientManager::DoMaintenance >
CCoinJoinClientManager::DoMaintenance > DoAutomaticDenominating >
CCoinJoinClientSession::DoAutomaticDenominating >
CCoinJoinClientSession::StartNewQueue > CConnman::AddPendingMasternode`
which modifies `CConnman::vPendingMasternodes`, which is non-const
behaviour

* `CJClientManager::DoMaintenance >
CCoinJoinClientManager::DoMaintenance > DoAutomaticDenominating >
CCoinJoin::IsCollateralValid > AcceptToMemoryPool` which adds a
transaction to the memory pool, which is non-const behaviour

* There were cppcheck [linter
failures](https://github.com/dashpay/dash/pull/5337#issuecomment-1685084688)
that seemed to be caused by the usage of `Assert` in
`coinjoin/client.h`. This seems to be resolved by backporting
[bitcoin#24714](https://github.com/bitcoin/bitcoin/pull/24714). (Thanks
@knst!)
    * Depends on #5546

---------

Co-authored-by: Kittywhiskers Van Gogh <63189531+kittywhiskers@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
2023-09-13 12:52:38 -05:00
PastaPastaPasta
38e70430b9
refactor: governance constification and deglobalization (#5572)
## Issue being fixed or feature implemented
Some relatively simple refactoring; inspired by reviewing #5569; adds
some constification and some deglobalization

## What was done?
Partial deglobalization and constification

## How Has This Been Tested?
Building

## Breaking Changes
None

## Checklist:
_Go over all the following points, and put an `x` in all the boxes that
apply._
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-09-10 14:05:49 -05:00
Konstantin Akimov
f8befc811c
fix: add missing includes and remove obsolete includes (#5562)
## Issue being fixed or feature implemented
Some headers or modules are used objects from STL without including it
directly, it cause compilation failures on some platforms for some
specific compilers such as #5554

## What was done?
Added missing includes and removed obsolete includes for `optional`,
`deque`, `tuple`, `unordered_set`, `unordered_map`, `set` and `atomic`.

Please, note, that this PR doesn't cover all cases, only cases when it
is obviously missing or obviously obsolete.

Also most of changes belongs to to dash specific code; but for cases of
original bitcoin code I keep it untouched, such as missing <map> in
`src/psbt.h`

I used this script to get a list of files/headers which looks suspicious
`./headers-scanner.sh std::optional optional`:
```bash
#!/bin/bash

set -e

function check_includes() {
    obj=$1
    header=$2
    file=$3

    used=0
    included=0

    grep "$obj" "$file" >/dev/null 2>/dev/null && used=1
    grep "include <$header>" $file >/dev/null 2>/dev/null && included=1
    if [ $used == 1 ] && [ $included == 0 ]
        then echo "missing <$header> in $file"
    fi
    if [ $used == 0 ] && [ $included == 1 ]
        then echo "obsolete <$header> in $file"
    fi
}
export -f check_includes

obj=$1
header=$2

find src \( -name '*.h' -or -name '*.cpp' -or -name '*.hpp' \) -exec bash -c 'check_includes "$0" "$1" "$2"'  "$obj" "$header"  {} \;
```

## How Has This Been Tested?
Built code locally

## Breaking Changes
n/a


## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
2023-09-07 09:07:02 -05:00
fanquake
7a2d07c683 Merge #21394: [doc] Improve comment about protected peers
ebde946a527e50630df180c6565ea5bf8d2ab5aa [doc] Improve comment about protected peers (Amiti Uttarwar)

Pull request description:

  The comment currently suggests a long-standing node would infrequently protect peers under normal circumstances. Clarify that we also protect peers that are synced to the same work as our chain tip. [Relevant check here](ee0dc02c6f/src/net_processing.cpp (L1997)).

ACKs for top commit:
  Empact:
    ACK ebde946a52
  jnewbery:
    ACK ebde946a527e50630df180c6565ea5bf8d2ab5aa

Tree-SHA512: 3692f4098e95f935d801e0ee6bbd3a7c9480e66ca070a7c68ba79c4fc2e62377f5d37080c7b6a7d15ab617aaf4d3df9b26abc4f1b090d572ba46fdd092a6a64a
2023-08-28 11:31:55 -05:00
MarcoFalke
30b57e072f Merge #21187: Net processing: Only call PushAddress() from net_processing
3e68efa615968e0c9d68a7f197c7852478f6be78 [net] Move checks from GetLocalAddrForPeer to caller (John Newbery)
d21d2b264cd77c027a06f68289cf4c3f177d1ed0 [net] Change AdvertiseLocal to GetLocalAddrForPeer (John Newbery)

Pull request description:

  This is the first part of #21186. It slightly disentangles addr handling in net/net_processing by making it explicit that net_processing is responsible for pushing addr records into `vAddrToSend`.

ACKs for top commit:
  MarcoFalke:
    re-ACK 3e68efa615968e0c9d68a7f197c7852478f6be78 🍅

Tree-SHA512: 9af50c41f5a977e2e277f24a589db38e2980b353401def5e74b108ac5f493d9b5d6b1b8bf15323a4d66321495f04bc271450fcef7aa7d1c095f051a4f8e9b15f
2023-08-28 11:24:41 -05:00
Konstantin Akimov
3443630a8c
ci: adds flag -Werror=reorder for arm target (#5540)
## Issue being fixed or feature implemented
The order of members in a class/struct definition and the order of their
initialization should match. This ensures that the code is more
error-proof in cases where the order of member initializations is
important, as they may depend on each other.


Instead manual checking of member initialization better let CI handle
it.
Last PR where it's noticed:
https://github.com/dashpay/dash/pull/5531#discussion_r1299404387

## What was done?
New flag "-Werror=reorder" for `configure.ac` and fixes existing code.

## How Has This Been Tested?
Build code with `--enable-werror`



## Breaking Changes
N/A

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
2023-08-22 23:19:48 +03:00
PastaPastaPasta
690f47c493
Merge pull request #5490 from vijaydasmp/bp22_2
backport: Merge bitcoin#20023, 21713, 20575, 21989, 20971, 20964, 20497, 20425, 19980, (partial) 20125
2023-08-20 23:39:50 -05:00
fanquake
c32857c05b Merge #20747: net processing: Remove dropmessagestest
176325a5a47befe32d480b3dc206dd0e64e04b21 [net processing] Remove dropmessagestest (John Newbery)

Pull request description:

  -dropmessagestest is a command line option that causes 1 in n received
  messages to be dropped. The Bitcoin P2P protocol is stateful and in
  general cannot handle messages being dropped. Dropped
  version/verack/ping/pong messages will cause the connection to time out
  and be torn down. Other dropped messages may also cause the peer to
  believe that the peer has stalled and tear down the connection.

  It seems difficult to uncover any actual issues with -dropmessagestest,
  and any coverage that could be generated would probably be easier to
  trigger with fuzz testing.

ACKs for top commit:
  MarcoFalke:
    cr ACK 176325a5a47befe32d480b3dc206dd0e64e04b21
  practicalswift:
    cr ACK 176325a5a47befe32d480b3dc206dd0e64e04b21
  dhruv:
    cr ACK 176325a
  amitiuttarwar:
    ACK 176325a5a47befe32d480b3dc206dd0e64e04b21

Tree-SHA512: bd582e5e8c9eb272a5d8ec01ff07c36c0033fbb84c30d1c72c87a7a6c7290021dcaf7bf549179a8b95aeb4f7243158d5593bc7fcf1ec16213782e470fe36bb89
2023-08-08 06:33:29 -05:00
fanquake
2111c7727b Merge #20617: p2p: Remove m_is_manual_connection from CNodeState
a33442fdc73eabd1c5596ab92954344edc9517e6 Remove m_is_manual_connection from CNodeState (Antoine Riard)

Pull request description:

  Currently, this member is only used to exclude MANUAL peers from discouragement
  in MaybePunishNodeForBlock(). Manual connections are already protected in
  MaybeDiscourageAndDisconnect(), independently from their network
  processing behaviors.

ACKs for top commit:
  MarcoFalke:
    cr ACK a33442fdc73eabd1c5596ab92954344edc9517e6
  promag:
    Code review ACK a33442fdc73eabd1c5596ab92954344edc9517e6.
  jnewbery:
    utACK a33442fdc73eabd1c5596ab92954344edc9517e6
  amitiuttarwar:
    code review ACK a33442fdc73eabd1c5596ab92954344edc9517e6

Tree-SHA512: cfe3f3dfa131373e3299002d34ae9e22ca6e1a966831bab32fcf06ff1d08f06095b4ab020cc4d267f3ec05ae23fbdc22373382ab828b999c0db11b8c842a4f0c
2023-08-08 06:26:09 -05:00
MarcoFalke
2c9d41d073 Merge #18454: net: Make addr relay mockable, add test
NOTE: There is slight difference with original backport due to future changes
in bitcoin#19272, bitcoin#19763 - otherwise functional test p2p_addr_relay.py fails

fa1da3d4bfc0511a89f5b19d5a4d89e55ff7ccde test: Add basic addr relay test (MarcoFalke)
fa1793c1c44a3f75a09f9c636467b8274c541bdd net: Pass connman const when relaying address (MarcoFalke)
fa47a0b003f53708b6d5df1ed4e7f8a7c68aa3ac net: Make addr relay mockable (MarcoFalke)

Pull request description:

  As usual:

  * Switch to std::chrono time to be type-safe and mockable
  * Add basic test that relies on mocktime to add code coverage

ACKs for top commit:
  naumenkogs:
    utACK  fa1da3d
  promag:
    ACK fa1da3d4bfc0511a89f5b19d5a4d89e55ff7ccde (fabe56e44b6f683e24e37246a7a8851190947cb3 before https://github.com/bitcoin/bitcoin/pull/18454#issuecomment-607866453), fa5bf23d527a450e72c2bf13d013e5393b664ca3 was dropped since last review.

Tree-SHA512: 0552bf8fcbe375baa3cab62acd8c23b2994efa47daff818ad1116d0ffaa0b9e520dc1bca2bbc68369b25584e85e54861fe6fd0968de4f503b95439c099df9bd7

fixup - see #19272, #19763
2023-08-03 11:16:41 -05:00