02320032c7 merge bitcoin#25060: add LIFETIMEBOUND to GetFirstStoredBlock()::start_time (Kittywhiskers Van Gogh)
cfda9e5eb5 merge bitcoin#25016: GetFirstStoredBlock() and getblockchaininfo follow-ups (Kittywhiskers Van Gogh)
99d028e9e1 merge bitcoin#24956: Call CHECK_NONFATAL only once where needed (Kittywhiskers Van Gogh)
87b556d322 refactor: use loop to iterate through `SoftForkDescPushBack` args (Kittywhiskers Van Gogh)
80615b8c01 merge bitcoin#24784: deduplicate integer serialization in RollingBloom benchmark (Kittywhiskers Van Gogh)
092a11e2d1 merge bitcoin#15423: Query Tor for correct -onion configuration (Kittywhiskers Van Gogh)
74dceb3c5c merge bitcoin#24486: dedup sqlite blob binding (Kittywhiskers Van Gogh)
aa945c2c1d merge bitcoin#23595: Add ParseHex<std::byte>() helper (Kittywhiskers Van Gogh)
c2defe7210 merge bitcoin#23819: don't serialize block hash twice (Kittywhiskers Van Gogh)
9a8d6e149f merge bitcoin#24265: Drop StripRedundantLastElementsOfPath() function (Kittywhiskers Van Gogh)
2d299f128a merge bitcoin#24136: Extract CTxIn::MAX_SEQUENCE_NONFINAL constant, rework BIP 65/68/112 docs (Kittywhiskers Van Gogh)
9aacee78ae merge bitcoin#23832: Changes time variables from int to chrono (Kittywhiskers Van Gogh)
06bf649e1b merge bitcoin#23499: Add interfaces::Node::broadCastTransaction method (Kittywhiskers Van Gogh)
Pull request description:
## Breaking Changes
None expected.
## Checklist
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation **(note: N/A)**
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 02320032c7
Tree-SHA512: b633e1caac95e565d43e28cef7c915f65952fbabdf55624090bc587c75b84fc04474bb6cd8625219ba828da172391c3d2d1aa6db998927312055c82caaad7ba0
09504bdd1f merge bitcoin#28895: do not make automatic outbound connections to addnode peers (Kittywhiskers Van Gogh)
6cf206ca0e merge bitcoin#28155: improves addnode / m_added_nodes logic (Kittywhiskers Van Gogh)
11d654af19 merge bitcoin#28189: diversify network outbounds release note (Kittywhiskers Van Gogh)
5dc52b3b6f merge bitcoin#27213: Diversify automatic outbound connections with respect to networks (Kittywhiskers Van Gogh)
291305b4d2 merge bitcoin#26497: Make ConsumeNetAddr always produce valid onion addresses (Kittywhiskers Van Gogh)
6a37934af4 partial bitcoin#26396: Avoid SetTxRelay for feeler connections (Kittywhiskers Van Gogh)
221a78ea84 merge bitcoin#25156: Introduce PeerManagerImpl::RejectIncomingTxs (Kittywhiskers Van Gogh)
cc694c2e5b merge bitcoin#22778: Reduce resource usage for inbound block-relay-only connections (Kittywhiskers Van Gogh)
6e6de54e5e net: use `Peer::m_can_relay_tx` when we mean `m_tx_relay != nullptr` (Kittywhiskers Van Gogh)
77526d2129 net: rename `Peer::m_block_relay_only` to `Peer::m_can_tx_relay` (Kittywhiskers Van Gogh)
Pull request description:
## Additional Information
* Dependency for https://github.com/dashpay/dash/pull/6333
* When backporting [bitcoin#22778](https://github.com/bitcoin/bitcoin/pull/22778), the `m_tx_inventory_known_filter.insert()` call in `queueAndMaybePushInv()` had to be moved out as `EXCLUSIVE_LOCKS_REQUIRED` does not seem to work with lambda parameters list (though it does work with capture list, [source](4b5e39290c/src/net_processing.cpp (L5781))), see error below
<details>
<summary>Compiler error:</summary>
```
net_processing.cpp:5895:21: note: found near match 'tx_relay->m_tx_inventory_mutex'
net_processing.cpp:5955:21: error: calling function 'operator()' requires holding mutex 'queueAndMaybePushInv.m_tx_inventory_mutex' exclusively [-Werror,-Wthread-safety-precise]
queueAndMaybePushInv(tx_relay, CInv(nInvType, hash));
^
net_processing.cpp:5955:21: note: found near match 'tx_relay->m_tx_inventory_mutex'
net_processing.cpp:5977:17: error: calling function 'operator()' requires holding mutex 'queueAndMaybePushInv.m_tx_inventory_mutex' exclusively [-Werror,-Wthread-safety-precise]
queueAndMaybePushInv(inv_relay, inv);
^
```
</details>
* Attempting to remove the `EXCLUSIVE_LOCKS_REQUIRED` or the `AssertLockHeld` are not options due to stricter thread sanitization checks being applied since [dash#6319](https://github.com/dashpay/dash/pull/6319) (and in general, removing annotations being inadvisable regardless)
* We cannot simply lock `peer->GetInvRelay()->m_tx_inventory_mutex` as a) the caller already has a copy of the relay pointer (which means that `m_tx_relay_mutex` was already held) that b) they used to lock `m_tx_inventory_mutex` (which we were already asserting) so c) we cannot simply re-lock `m_tx_inventory_mutex` as it's already already held, just through a less circuitous path.
* The reason locking is mentioned instead of asserting is that the compiler treats `peer->GetInvRelay()->m_tx_inventory_mutex` and `tx_relay->m_tx_inventory_mutex` as separate locks (despite being different ways of accessing the same thing) and would complain similarly to the error above if attempting to assert the former while holding the latter.
* As `m_tx_relay` is always initialized for Dash, to mimic the behaviour _expected_ upstream, in [bitcoin#22778](https://github.com/bitcoin/bitcoin/pull/22778), `SetTxRelay()` will _allow_ `GetTxRelay()` to return `m_can_tx_relay` (while Dash code that demands unconditional access can use `GetInvRelay()` instead) with the lack of `SetTxRelay()` resulting in `GetTxRelay()` feigning the absence of `m_can_tx_relay`.
This allows us to retain the same style of checks used upstream instead of using proxies like `!CNode::IsBlockOnlyConn()` to determined if we _should_ use `m_tx_relay`.
* Speaking of proxies, being a block-only connection is now no longer the only reason not to relay transactions
In preparation for [bitcoin#26396](https://github.com/bitcoin/bitcoin/pull/26396), proxy usage of `!CNode::IsBlockOnlyConn()` and `!Peer::m_block_relay_only` was replaced with the more explicit `Peer::m_can_tx_relay` before being used to gate the result of `GetTxRelay()`, to help it mimic upstream semantics.
## Breaking Changes
None expected
## Checklist
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 09504bdd1f
Tree-SHA512: f4f36f12f749b697dd4ad5521ed15f862c93ed4492a047759554aa80a3ce00dbd1bdc0242f7a4468f41f25925d5b79c8ab774d8489317437b1983f0a1277eecb
85b0b78d47 merge bitcoin#24565: Remove LOCKTIME_MEDIAN_TIME_PAST constant (Kittywhiskers Van Gogh)
7068abd796 merge bitcoin#25485: Use enum instead of string for `filtertype_name` (Kittywhiskers Van Gogh)
1c1fcc60a8 merge bitcoin#24080: Remove unused locktime flags (Kittywhiskers Van Gogh)
536c4e1b27 merge bitcoin#23637: Remove uncompiled MTP code (Kittywhiskers Van Gogh)
0b65f1d241 merge bitcoin#23517: Move miner to src/node (Kittywhiskers Van Gogh)
b7db4cbab0 merge bitcoin#23152: add `--enable-lto` configuration option (Kittywhiskers Van Gogh)
4a46391791 merge bitcoin#23439: Open streams_test_tmp file in temporary folder (Kittywhiskers Van Gogh)
e3e225f59d merge bitcoin#23293: Add comment to COIN constant (Kittywhiskers Van Gogh)
5fb2cc8bb6 merge bitcoin#23137: move-only bloom to src/common (Kittywhiskers Van Gogh)
20d15056f1 merge bitcoin#22951: move amount.h into consensus (Kittywhiskers Van Gogh)
Pull request description:
## Breaking Changes
None expected.
## Checklist
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 85b0b78d47
Tree-SHA512: 2bd31056291f86e8f33f6b8ca5a8236a5f876d6811f852d13ac0d6d818013a62af3cfdd05893f16f7476338ac93655e150fbbe651b0b8cbf414afc7f82b05664
Right now, it's not immediately obvious which checks are to see if we
are looking for block-relay-only status or are using that status as a
proxy to see if we can relay transactions. Let's fix that.
The true purpose of `m_block_relay_only` is to use it in place of
`m_tx_relay == nullptr`, used to indicate if the node is permitted
to relay transactions.
In upcoming commits, being a block-relay-only node will not be the only
reason to not relay transactions, so let's rename the variable to what
we actually use it for.
37389c7d38 Merge bitcoin/bitcoin#28781: depends: latest config.guess & config.sub (fanquake)
3239f1525d Merge bitcoin/bitcoin#28479: build: use _LIBCPP_ENABLE_DEBUG_MODE over ENABLE_ASSERTIONS (fanquake)
45cc44bcf9 Merge bitcoin/bitcoin#27628: build: Fix shared lib linking for darwin with lld (fanquake)
b8ddcd937c Merge bitcoin/bitcoin#27575: Introduce platform-agnostic `ALWAYS_INLINE` macro (fanquake)
71c6d7f6ca Merge bitcoin/bitcoin#26653: test, init: perturb file to ensure failure instead of only deleting them (fanquake)
417f71a587 Merge bitcoin/bitcoin#27422: test: add coverage to rpc_scantxoutset.py (fanquake)
898dcbdc4f Merge bitcoin/bitcoin#27559: doc: clarify processing of mempool-msgs when NODE_BLOOM (glozow)
a4e429cb5a Merge bitcoin/bitcoin#26953: contrib: add ELF OS ABI check to symbol-check.py (fanquake)
deb7de26dd Merge bitcoin/bitcoin#26604: test: add coverage for `-bantime` (fanquake)
f725ed509a Merge bitcoin/bitcoin#26314: test: perturb anchors.dat to test error during initialization (fanquake)
3306f96d80 Merge bitcoin/bitcoin#25937: test: add coverage for rpc error when trying to rescan beyond pruned data (fanquake)
712dcaf86b Merge bitcoin/bitcoin#27516: test: simplify uint256 (de)serialization routines (fanquake)
90d65f25e1 Merge bitcoin/bitcoin#27508: build: use latest config.{guess,sub} in depends (fanquake)
df7be026e4 Merge bitcoin/bitcoin#27506: test: prevent intermittent failures (fanquake)
9b58b2d97b Merge bitcoin/bitcoin#27447: depends: Remove `_LIBCPP_DEBUG` from depends DEBUG mode (fanquake)
07770b77a1 Merge bitcoin/bitcoin#26741: doc: FreeBSD DataDirectoryGroupReadable Setting (fanquake)
5645362f11 Merge bitcoin/bitcoin#27362: test: remove `GetRNGState` lsan suppression (fanquake)
671e8e6851 Merge bitcoin/bitcoin#27368: refactor: Drop no longer used `CNetMsgMaker` instances (fanquake)
a11690bf62 Merge bitcoin/bitcoin#27301: depends: make fontconfig build under clang-16 (fanquake)
0a94b3f27b Merge bitcoin/bitcoin#27328: depends: fix osx build with clang 16 (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 37389c7d38
kwvg:
utACK 37389c7d38
Tree-SHA512: 1ea075a58361f57e037febcf003d380ab845b6c8e1c62d9fcf8954d46cd006046d1951f15a41a5deb9ab7af734df9dafcf89f33d115d78246752f7e2cd13f4ee
750447e345 Merge bitcoin/bitcoin#20586: Fix Windows build with --enable-werror (W. J. van der Laan)
368a6ef512 Merge bitcoin-core/gui#390: Add SubFeeFromAmount to options (Hennadii Stepanov)
7df9788c85 Merge bitcoin-core/gui#381: refactor: Make BitcoinCore class reusable (W. J. van der Laan)
40a8b925db Merge bitcoin/bitcoin#22688: contrib: use `keys.openpgp.org` to retrieve builder keys (fanquake)
62b5358a9c Merge #11909: contrib: Replace developer keys with list of pgp fingerprints (Wladimir J. van der Laan)
1ff42b40e3 Merge bitcoin/bitcoin#21500: wallet, rpc: add an option to list private descriptors (Samuel Dobson)
5a803ae765 Merge bitcoin/bitcoin#22378: test: remove confusing `MAX_BLOCK_BASE_SIZE` (MarcoFalke)
42a0cf0709 Merge bitcoin/bitcoin#21562: [net processing] Various tidying up of PeerManagerImpl ctor (MarcoFalke)
e3c69da4f2 Merge bitcoin/bitcoin#22232: refactor: Pass interpreter flags as uint32_t instead of signed int (MarcoFalke)
Pull request description:
## What was done?
Regular batch of backports from Bitcoin v23
## How Has This Been Tested?
Run unit and functional tests
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
utACK 750447e345
PastaPastaPasta:
utACK 750447e345
Tree-SHA512: 46e0de22798937b6b0a228ea9ca39b465cd9e8b822f973f23af80fccb1fe2d733716930d3f3843aaf41a58225aa98e18cd5d5e361245f64e146ff1be118f91ca
4581a682d2d1fdd0e56fb4a56e6228be878a04a3 clarify processing of mempool-msgs when NODE_BLOOM (0xb10c)
Pull request description:
Under which circumstances we process received 'mempool' P2P messages caused confusion in #27426. Rather than bike-shedding the formulation of the IF-statement, this adds a comment clarifying when we process the message. Also, correcting the `m_send_mempool` description.
ACKs for top commit:
dergoegge:
ACK 4581a682d2d1fdd0e56fb4a56e6228be878a04a3
willcl-ark:
ACK 4581a682d2
glozow:
ACK 4581a682d2d1fdd0e56fb4a56e6228be878a04a3
Tree-SHA512: 51ec673c3446b67c26f6c715430d0708b998b256260f5f5d0c034f271be8447d0bb8540dfd3879aa51904512fb26c9411766786c86287acff62d037a1df88855
83003ffe049a432f6fa4127e054f073127e70b90 refactor: replace RecursiveMutex `m_most_recent_block_mutex` with Mutex (Sebastian Falbesoner)
8edd0d31ac683378135a9839e5d4172b82f8f5b8 refactor: reduce scope of lock `m_most_recent_block_mutex` (Sebastian Falbesoner)
Pull request description:
This PR is related to #19303 and gets rid of the RecursiveMutex `m_most_recent_block_mutex`. All of the critical sections (5 in total) only directly access the guarded elements, i.e. it is not possible that within one section another one is called, and we can use a regular Mutex:
b019cdc036/src/net_processing.cpp (L1650-L1655)b019cdc036/src/net_processing.cpp (L1861-L1865)b019cdc036/src/net_processing.cpp (L3149-L3152)b019cdc036/src/net_processing.cpp (L3201-L3206)b019cdc036/src/net_processing.cpp (L4763-L4769)
The scope of the last critical section is reduced in the first commit, in order to avoid calling the non-trivial method `CConnman::PushMessage` while the lock is held.
ACKs for top commit:
furszy:
Code ACK 83003ffe with a small comment.
hebasto:
ACK 83003ffe049a432f6fa4127e054f073127e70b90
w0xlt:
ACK 83003ffe04
Tree-SHA512: 3df290cafd2f6c4d40afb9f14e822a77d9c1828e66f5e2233f3ac1deccc2b0a8290bc5fb8eb992f49d39e887b50bc0e9aad63e05db2d870791a8d409fb95695f
ea7ec7808745805c0a18513d7da271dedb2de3f1 refactor: Drop no longer used `CNetMsgMaker` instances (Hennadii Stepanov)
Pull request description:
The removed lines have been unused since the abf5d16c24 commit from https://github.com/bitcoin/bitcoin/pull/25454.
ACKs for top commit:
dergoegge:
utACK ea7ec7808745805c0a18513d7da271dedb2de3f1
Sjors:
ACK ea7ec7808745805c0a18513d7da271dedb2de3f1
TheCharlatan:
ACK ea7ec7808745805c0a18513d7da271dedb2de3f1
Tree-SHA512: 9a2a9ff3f124b68a8cd20a637e90885096996c3aa354a4d8adbec98f5761e9e826c1c064ccd90aaf6d72beac61dd9e22c8b76d089e18bba6e0ad51e59a9c7df8
1db9e6ac76 Merge #19979: Replace LockAssertion with AssertLockHeld, remove LockAssertion (MarcoFalke)
d0a4198166 Merge #21598: refactor: Remove negative lock annotations from globals (MarcoFalke)
55114a682e Merge #19668: Do not hide compile-time thread safety warnings (MarcoFalke)
898282d620 Merge bitcoin/bitcoin#28774: wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it (Ava Chow)
6d452845dc fix: add missing lock annotation for ContextualCheckBlock and related TODO (Konstantin Akimov)
846ebab6e0 fix: fixes for orphange's locks and cs_main (Konstantin Akimov)
002580c42d fix: add ignoring safety-annotation for llmq/signing_shares due to template lambdas (Konstantin Akimov)
502d6ae8ef fix: add multiple missing annotation about locks for dash specific code (Konstantin Akimov)
a219a8be15 partial Merge #29040: refactor: Remove pre-C++20 code, fs::path cleanup (Konstantin Akimov)
042e8a4101 fix: drop unneded lock assert in net.cpp for m_nodes_mutex (Konstantin Akimov)
Pull request description:
## What was done?
These 4 backports improve noticeable implementation of thread-safety analysis by moving many possible warnings to compilation level.
There's a lot of fixes for annotations after that!
## How Has This Been Tested?
Build with clang:
```
CC=clang CXX=clang++ ./configure --prefix=$(pwd)/depends/x86_64-pc-linux-gnu --enable-suppress-external-warnings --enable-debug --enable-stacktraces --enable-werror --enable-crash-hooks --enable-maintainer-mode
```
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
utACK 1db9e6ac76
PastaPastaPasta:
utACK 1db9e6ac76
Tree-SHA512: 42a73e423f311689babc366b58a92f024556cf63af93357cfa0715c471efc1491d02ed9e7ba19b23efcdf423ab5bd051a24da2ad64c32e4b6db132a05f6669c1
0bd1184adf6610c0bd14f4e9a25c0a200e65ae25 Remove unused LockAssertion struct (Hennadii Stepanov)
ab2a44297fd0796bf5797ae2a477e8e56d9c3c12 Replace LockAssertion with a proper thread safety annotations (Hennadii Stepanov)
73f71e19965e07534eb47701f2b23c9ed59ef475 refactor: Use explicit function type instead of template (Hennadii Stepanov)
Pull request description:
This PR replaces `LockAssertion` with `AssertLockHeld`, and removes `LockAssertion`.
This PR is compared with alternatives in https://github.com/bitcoin-core/bitcoin-devwiki/wiki/AssertLockHeld-PRs
ACKs for top commit:
MarcoFalke:
ACK 0bd1184adf6610c0bd14f4e9a25c0a200e65ae25
ajtowns:
ACK 0bd1184adf6610c0bd14f4e9a25c0a200e65ae25
vasild:
ACK 0bd1184ad
Tree-SHA512: ef7780dd689faf0bb479fdb97c49bc652e2dd10c148234bb95502dfbb676442d8565ee37864d923ca21a25f9dc2a335bf46ee82c095e387b59a664ab05c0ae41
fa5eabe72117f6e3704858e8d5b2c57a120258ed refactor: Remove negative lock annotations from globals (MarcoFalke)
Pull request description:
They only make sense for mutexes that are private members. Until cs_main is a private member the negative annotations should be replaced by excluded annotations, which are optional.
ACKs for top commit:
sipa:
utACK fa5eabe72117f6e3704858e8d5b2c57a120258ed
ajtowns:
ACK fa5eabe72117f6e3704858e8d5b2c57a120258ed
hebasto:
ACK fa5eabe72117f6e3704858e8d5b2c57a120258ed
vasild:
ACK fa5eabe72117f6e3704858e8d5b2c57a120258ed
Tree-SHA512: 06f8a200304f81533010efcc42d9f59b8c4d0ae355920c0a28efb6fa161a3e3e68f2dfffb0c009afd9c2501e6a293c6e5a419a64d718f1f4e79668ab2ab1fcdc
We need to extend `bypass_limits` to `ProcessTransaction()` because
Dash allows the `sendrawtransaction` RPC to bypass limits with the
optional `bypasslimits` boolean (introduced in dash#2110).
The bool arguments are not in alphabetical order to prevent breakage
with Bitcoin code that expects `bypass_limits` to always be false.
c77216ea80 docs: explain meaning of MessageProcessingResult's members (Konstantin Akimov)
d0f17788fc refactor: drop dependency of EhfSignals on PeerManager (Konstantin Akimov)
1d13f010d0 refactor: remove dependency of QuorumBlockProcessor on PeerManager (Konstantin Akimov)
f1c6d17879 refactor: remove dependency of chainlocks on PeerManager (Konstantin Akimov)
538342138c refactor: HandleNewRecoveredSig return PostProcessingMessage (Konstantin Akimov)
09565fe6cc refactor: new type of message processing result for resolving circular dependency over PeerManager (Konstantin Akimov)
d54b3eeb7b refactor: move ChainLocksSigningEnabled from header to cpp file as static function (Konstantin Akimov)
cab700aae3 refactor: remove unused include spork.h from validation.cpp (Konstantin Akimov)
Pull request description:
## Issue being fixed or feature implemented
It reduces circular dependencies and simplify code.
## What was done?
Removed circular dependency over PeerManager for llmq::QuorumBlockProcessor, llmq::CEhfSignals, llmq::ChainLocks, and several extra useful refactorings: see commits.
## How Has This Been Tested?
Run `test/lint/lint-circular-dependencies.sh`
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
utACK c77216ea80
PastaPastaPasta:
utACK c77216ea80
Tree-SHA512: e01829a694c9bfbbe70bc346ef5949b5e9e4532560f4c40ee292952f05f0fd23ecf4bd978e918f74dd3422c1b90231fd7d9984f491f3ab8f7eb08072540406b4
e458adb61c merge bitcoin#30118: improve robustness of connect_nodes() (UdjinM6)
ac94de23ae merge bitcoin#28287: add `sendmsgtopeer` rpc and a test for net-level deadlock situation (Kittywhiskers Van Gogh)
d1fce0b7ca fix: ensure that deadlocks are actually resolved (Kittywhiskers Van Gogh)
19e7bf64c8 merge bitcoin#27863: do not break when addr is not from a distinct network group (Kittywhiskers Van Gogh)
1adb9a232c merge bitcoin#27761: Log addresses of stalling peers (Kittywhiskers Van Gogh)
2854a6aa5a merge bitcoin#27128: fix intermittent issue in `p2p_disconnect_ban` (Kittywhiskers Van Gogh)
d4b0faeae1 merge bitcoin#26854: Fix intermittent timeout in p2p_permissions.py (Kittywhiskers Van Gogh)
892e329ada merge bitcoin#26138: Avoid race in disconnect_nodes helper (Kittywhiskers Van Gogh)
d6ce037814 merge bitcoin#25443: Fail if connect_nodes fails (Kittywhiskers Van Gogh)
60b5392d92 partial bitcoin#22778: Reduce resource usage for inbound block-relay-only connections (Kittywhiskers Van Gogh)
85c4aef9cb merge bitcoin#23774: Add missing assert_equal import to p2p_add_connections.py (Kittywhiskers Van Gogh)
03544175d9 merge bitcoin#22777: don't request tx relay on feeler connections (Kittywhiskers Van Gogh)
7229eb0ae2 merge bitcoin#23042: Avoid logging AlreadyHaveTx when disconnecting misbehaving peer (Kittywhiskers Van Gogh)
05395ff37b merge bitcoin#22817: Avoid race after connect_nodes (Kittywhiskers Van Gogh)
Pull request description:
## Additional Information
* Depends on https://github.com/dashpay/dash/pull/6286
* Depends on https://github.com/dashpay/dash/pull/6287
* Depends on https://github.com/dashpay/dash/pull/6289
* When backporting [bitcoin#28287](https://github.com/bitcoin/bitcoin/pull/28287), `p2p_net_deadlock.py` relies on the function, `random_bytes()`, that is introduced in [bitcoin#25625](https://github.com/bitcoin/bitcoin/pull/25625). Backporting [bitcoin#25625](https://github.com/bitcoin/bitcoin/pull/25625) would attract changes outside the scope of this PR.
In the interest of brevity, the changes that introduce `random_bytes()` have been included in [bitcoin#28287](https://github.com/bitcoin/bitcoin/pull/28287) instead.
## Breaking Changes
None expected.
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation **(note: N/A)**
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK e458adb61c
PastaPastaPasta:
utACK e458adb61c
Tree-SHA512: 48494004dddecb31c53f5e19ab0114b92ed7b4381c7977800fd49b7403222badbfdcfe46241e854f5b086c6f54a35f6483f91c6f047b7ac9b1e88e35bb32ad02
cc1a75ab3a docs: add release notes (Kittywhiskers Van Gogh)
39625f16f0 stats: drop copyright notice from `stats/client.cpp` (Kittywhiskers Van Gogh)
18a2e48eb9 stats: rename `statsns` to clearer `statsprefix` (Kittywhiskers Van Gogh)
42918c2cdc stats: rename `statshostname` to more appropriate `statssuffix` (Kittywhiskers Van Gogh)
f3a4844b0a stats: implicitly treat stats as enabled if `statshost` is specified (Kittywhiskers Van Gogh)
69603a83fa stats: miscellaneous changes and housekeeping (Kittywhiskers Van Gogh)
3e12ac0e09 stats: deduplicate `send` and `sendDouble` logic (Kittywhiskers Van Gogh)
bf44fc3bf6 feat(stats): introduce support for batching messages (Kittywhiskers Van Gogh)
38b1643fe6 feat(stats): introduce support for queuing messages (Kittywhiskers Van Gogh)
fc4a736e2a stats: move message sending logic to `RawSender` (Kittywhiskers Van Gogh)
92690685be stats: move `statsd_client` to `stats` directory (Kittywhiskers Van Gogh)
f782dfd562 stats: remove double indentation in header file (Kittywhiskers Van Gogh)
Pull request description:
## Motivation
This pull request achieves the goal originally set out in [dash#5167](https://github.com/dashpay/dash/pull/5167), to migrate the base of our Statsd client implementation to one that is actively maintained. Statoshi ([source](54c3ffdcf0/src/statsd_client.cpp)) utilizes [talebook/statsd-client-cpp](https://github.com/talebook/statsd-client-cpp), which in turn is inherited by Dash.
As Statsd is the only cross-platform reporting mechanism available (USDT requires a Linux host with superuser privileges and RPCs don't provide as much flexibility as desired), emphasis is placed on using Statsd as the primary way for the Dash daemon to report metrics related to node and network health.
As part of maintaining our Statsd client, this PR aims to migrate the base of our implementation to [vthiery/cpp-statsd-client](https://github.com/vthiery/cpp-statsd-client), a streamlined implementation that embraces C++11 with a thread-safe implementation of queueing and batching, which reduces the number of packets used to transmit stats.
These capabilities are optional and users can opt not to use them by setting `-statsduration=0` to disable queueing (which will also disable batching) or `-statsbatchsize=0`, which will disable batching (but will not disable queueing unless requested explicitly).
## Additional Information
* Dependent on https://github.com/dashpay/dash/pull/5167
* `RawSender` (and by extension, `RawMessage`) strive to remain as unopinionated as possible, moving the responsibility to construct valid Statsd messages onto `StatsdClient`. This is to ensure that `RawSender` can be reused down the line or independently extended without impacting the Statsd client.
* `RawMessage` exists to provide extensions to `std::vector<uint8_t>` that make it easier to abstract away strings while also implementing some of its semantics like `append()` (and its alias, `+=`).
* `InitStatsClient()` was introduced to keep `StatsdClient` indifferent to _how_ arguments are obtained and sanitized before they're supplied to the constructor. This is to keep it indifferent to all the backwards-compatibility code for deprecated arguments still work.
* When constructing the Statsd message, we can use `%f` without having to specify a precision as tinyformat automatically assumes a precision of 6 ([source](17110f50b3/src/tinyformat.h (L673))) and problems don't seem to be observed when using `%f` with integers ([source](https://github.com/dashpay/dash/pull/6267#issuecomment-2345592051)).
* As a guardrail, there is a `static_assert` to ensure that a specialization of `send()` involving a non-arithmetic type will raise alarm when compiling ([source](a0ce720207/src/stats/client.cpp (L145))).
## Breaking changes
* `-statsenabled` (replaced with specifying `-statshost`), `-statshostname` (replaced by `-statssuffix`) and `-statsns` (replaced by `-statsprefix`) have been deprecated and will be removed in a future release.
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
knst:
utACK cc1a75ab3a
UdjinM6:
utACK cc1a75ab3a
Tree-SHA512: b038419f2b6d807dac40a04d23c5046fbaa95beedb88f5a9e4c06a7042c2f5da7e01c72c4a2744bce10878cafc747136d6599dcd86ae1be0782ad4194d5b7bec
8f06ac9dfa Merge bitcoin/bitcoin#22172: doc: update tor.md, release notes with removal of tor v2 support (W. J. van der Laan)
9b22501a4d Merge bitcoin/bitcoin#22122: ci: Bump macOS image to big-sur-xcode-12.5 (MarcoFalke)
3b05a99b50 Merge bitcoin/bitcoin#22106: refactor: address ProcessNewBlock comments from #21713 (fanquake)
c8725560c9 Merge bitcoin/bitcoin#21856: doc: add OSS-Fuzz section to fuzzing.md doc (MarcoFalke)
facf685285 Merge bitcoin/bitcoin#22261: [p2p/mempool] Two small fixes to node broadcast logic (fanquake)
1430897fc4 Merge bitcoin/bitcoin#22445: fuzz: Move implementations of non-template fuzz helpers from util.h to util.cpp (MarcoFalke)
f0c62d50a5 Merge bitcoin/bitcoin#22447: test: whitelist rpc_rawtransaction peers to speed up tests (fanquake)
b609514142 Merge #22381: guix: Test security-check sanity before performing them (Carl Dong)
9ef68d1905 Merge bitcoin/bitcoin#22061: ci: Bump multiprocess memory (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Regular backports from bitcoin v22
## What was done?
See commits
## How Has This Been Tested?
Run unit and functional tests
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
utACK 8f06ac9dfa
PastaPastaPasta:
utACK 8f06ac9dfa
Tree-SHA512: f800b7ca8d357f2d02ce5cb1fc4951c2765242676c5494efd5e22e8f6d41d889e1fa2f888930f72aded75813413c6488d8a7d96baa2cf4820e3461464708658e
e12f287498e5836bb5e32de5abaef02f3d20d868 net: cleanup newly added PeerManagerImpl::ProcessNewBlock (fanquake)
610151f5b076d4b1ab90c0dd2717e5410aba6b19 validation: change ProcessNewBlock() to take a CBlock reference (fanquake)
Pull request description:
Addresses some [post-merge comments](https://github.com/bitcoin/bitcoin/pull/21713#pullrequestreview-638777410) from #21713. Also makes `ChainstateManager::ProcessNewBlock` take a const reference argument, as it [was asked](https://github.com/bitcoin/bitcoin/pull/21713#discussion_r615229548) why it was not the case in that PR.
ACKs for top commit:
jnewbery:
Code review ACK e12f287498e5836bb5e32de5abaef02f3d20d868
MarcoFalke:
review ACK e12f287498e5836bb5e32de5abaef02f3d20d868 🚚
Tree-SHA512: 9c3e7353240c862d50bce2a0f58741c109dd628040b56ed46250103f8ebe9009238b131da710486791e28e3a83c985057b7be0a32aed1a929269b43097c7425b
74a5a9f984 style: apply clang-format (Konstantin Akimov)
db9798f5e4 refactor: move call GetQuorumMembers inside Init() (Konstantin Akimov)
9593566802 refactor: move pQuorumBaseBlockIndex from Init() to constructor in CDKGSession (Konstantin Akimov)
9f3eb6bbb8 perf: check DIP0003 before CDKGSession initialization (Konstantin Akimov)
7f815cb501 refactor: remove unused constructor of CDKGLogger (Konstantin Akimov)
c82672af94 refactor: remove dependency of CDKGSession on PeerManager (Konstantin Akimov)
fb78b0cc94 refactor: remove retBan flag from ReceiveMessage (Konstantin Akimov)
d26d4ab0bc refactor: remove dependency of dkgsessionmgr on dkgsession (Konstantin Akimov)
d361b11e5b refactor: moved including llmq/dkgsession.h from dkgsessionmgr.h to cpp file (Konstantin Akimov)
e77aeb321b refactor: removed including quorums.h from chainlocks.h (Konstantin Akimov)
6f7068ef42 refactor: remove exceeding evodb.h from headers (Konstantin Akimov)
Pull request description:
## Issue being fixed or feature implemented
We have 72 circular dependencies of dash specific code. This PR removes 2 of them, over dkgsession.
## What was done?
Refactor dkgsession initialization, message processing, dropped unused arguments, re-distributed code between functions and modules... See each commit.
Also optimized headers: excluded evo/evodb.h and llmq/quorums.h from the headers where they are not needed.
## How Has This Been Tested?
Run `test/lint/lint-circular-dependencies.sh`
Run unit/functional tests
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
utACK 74a5a9f984
PastaPastaPasta:
utACK 74a5a9f984
Tree-SHA512: f3adabe6a7bc6c4dcae6430be6857b31a0722d7f605d6651f3ff93c6fcb350a7312a9a0ecbda8ac131ffef6fbf2499700112de3990c7512ddb057bde7cc42665
71689fe6dc (partial) Merge bitcoin/bitcoin#22981: doc: Fix incorrect C++ named args (fanquake)
2b71a9b030 Merge bitcoin/bitcoin#23755: rpc: Quote user supplied strings in error messages (MarcoFalke)
5a441b38de (partial) Merge bitcoin-core/gui#409: Fix window title of wallet loading window (Hennadii Stepanov)
49c87e93a6 Merge bitcoin/bitcoin#23142: Return false on corrupt tx rather than asserting (W. J. van der Laan)
Pull request description:
backports from bitcoin
ACKs for top commit:
UdjinM6:
utACK 71689fe6dc
knst:
utACK 71689fe6dc
Tree-SHA512: c68e2a1be5669f4fd8b02001ea81310b41fcac2cc5cc660e67b5140b334669c9a071a4bd5b33232580215607f323af5f87218a3465493675a633e112984296eb
63b13aa519 merge bitcoin#28525: Drop v2 garbage authentication packet (Kittywhiskers Van Gogh)
662394cd49 merge bitcoin#28489: fix incorrect assumption in v2transport_test (Kittywhiskers Van Gogh)
98782c62df merge bitcoin#28433: Follow-up to BIP324 connection support (Kittywhiskers Van Gogh)
f9825168fb merge bitcoin#28196: BIP324 connection support (Kittywhiskers Van Gogh)
3087275039 merge bitcoin#28419: introduce and use `ConsumePrivateKey` helper (Kittywhiskers Van Gogh)
dccd395a4e merge bitcoin#27577: give seednodes time before falling back to fixed seeds (Kittywhiskers Van Gogh)
eb4f01f931 merge bitcoin#26584: include local ("unreachable") peers in -netinfo table (Kittywhiskers Van Gogh)
10dc874136 merge bitcoin#26982: bitcoin#25880 fixups (Kittywhiskers Van Gogh)
a36f8f2a1a merge bitcoin#25880: Make stalling timeout adaptive during IBD (Kittywhiskers Van Gogh)
1d77f3ff55 merge bitcoin#26519: Add getpeerinfo test for missing version message (Kittywhiskers Van Gogh)
Pull request description:
## Additional Information
* When backporting [bitcoin#25880](https://github.com/bitcoin/bitcoin/pull/25880), changes needed to be made to `p2p_ibd_stalling.py` to help it pass.
* DIP3 activation had to be delayed to a block beyond the range of the test. This is to prevent block rejection arising from a missing DIP3-compliant coinbase (done with `-dip3params=2000:2000`)
* Mock time was disabled to ensure nodes in the test do not resort to direct fetching (with mock time enabled, nodes would be considered close apart in time, which would prevent the primary node from fetching in parallel, which is behavior this test relies on) (done with `self.disable_mocktime = True`)
* The nodes connected do not report compressed headers support (the test relies on sending `headers` messages and reworking it to use compressed headers has little benefit) (done with `services = NODE_NETWORK | NODE_BLOOM`)
* When backporting [bitcoin#28196](https://github.com/bitcoin/bitcoin/pull/28196), in the `v2transport_test` unit test, references to `4000000` were substituted with `MAX_PROTOCOL_MESSAGE_LENGTH` as Dash messages have a protocol limit of 3MiB ([source](d754799580/src/net.h (L79-L80))) while Bitcoin messages have a protocol limit of 4MB ([source](225718eda8/src/net.h (L62-L63))).
* Additionally note that short message IDs as defined in the BIP324 spec ([source](22660ad307/bip-0324.mediawiki (v2-bitcoin-p2p-message-structure))) have not been changed to include Dash-specific messages, meaning, Dash-specific messages will always take 13 bytes.
* As `FEEFILTER` is not supported by Dash, it has been replaced with a blank string in the short IDs table. It was not removed as doing so would disturb the table's arrangement as specified in spec and require readjustment of tests to account for the change in layout.
## Breaking Changes
None expected.
## Checklist
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation **(note: N/A)**
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 63b13aa519
knst:
utACK 63b13aa519
Tree-SHA512: c41d2c6b1f145be513e285c8f91d00ac31fff4e4d24f611a1fdab24c8740f453b0bb28912021cdf8be4f5ce93dcff8579864727ee14b8e45894b56df524ab48d
48c7f98b1a doc: drop trailing whitespace (pasta)
697743d77b test: add missing import (UdjinM6)
cfe99fd289 docs: add release notes for 6239 (pasta)
a6bbaacfaa fix: GetHeadersLimit is used for getheaders(2) and headers(2), refactor it to accept `compressed` instead of `msg_type` (UdjinM6)
b224f3f6ca bump p2p_version in tests (PastaPastaPasta)
b423f42aae refactor: sort imports (UdjinM6)
f6c68ba71b refactor: simplify _compute_requested_block_headers (UdjinM6)
07876b2c4a use `MAX_HEADERS_UNCOMPRESSED_RESULT` not `MAX_HEADERS_UNCOMPRESSED_RESULTS` ; use `MAX_HEADERS_UNCOMPRESSED_RESULT` in RPC to avoid breaking changes (pasta)
b137280df4 change to _COMPRESSED or _UNCOMPRESSED (pasta)
303bc7af99 fix: increase it for headers2 only (UdjinM6)
e23410ffdd trivial: rename `MAX_HEADERS_RESULTS_NEW` to `MAX_HEADERS_RESULTS` (Kittywhiskers Van Gogh)
bcf0320691 trivial: move the headers limit determination to `GetHeadersLimit()` (Kittywhiskers Van Gogh)
993c7c0f90 feat: increase the number of block headers able to be downloaded at once to 8000 in protocol version `70234` (pasta)
Pull request description:
## Issue being fixed or feature implemented
We did some testing quite a while ago that found that sending 8000 headers at a time could speed stuff up. But we wanted to wait until compressed headers were implemented. Well, they've been implemented!
## What was done?
Bump 2000 -> 8000 triggered by protocol version
## How Has This Been Tested?
Hasn't, we should setup a few nodes running this and sync them from each other
## Breaking Changes
New protocol version, not breaking but should add notes? I should probably add release notes
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
light ACK 48c7f98b1a
knst:
utACK 48c7f98b1a
Tree-SHA512: 54c68b9496131ab7f32504d44398d776a151df809d0120d093bbabb18904a783bd9b58796820209f5d75552df5476e30eaa09d68f7c5057882f94b5766a64f4c
We need to disable mocktime so that the node doesn't resort to direct
fetching. We also need to delay DIP3's activation so that blocks don't
get rejected for not having a valid DIP3-compliant coinbase.
The old `GetLocal()` has been moved to `masternode/node.cpp` due to its
use in determining a node's external address. We don't want the old
variant to be used otherwise so we'll move it out of `net.{cpp,h}` for
good measure.
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
In upcoming commits, message sending will be split off into a separate
file and stats capabilities will be fleshed out. Prepare for that by
giving it its own directory.
Also get rid of `statsd` namespace, it is entirely unnecessary.
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
