09504bdd1f merge bitcoin#28895: do not make automatic outbound connections to addnode peers (Kittywhiskers Van Gogh)
6cf206ca0e merge bitcoin#28155: improves addnode / m_added_nodes logic (Kittywhiskers Van Gogh)
11d654af19 merge bitcoin#28189: diversify network outbounds release note (Kittywhiskers Van Gogh)
5dc52b3b6f merge bitcoin#27213: Diversify automatic outbound connections with respect to networks (Kittywhiskers Van Gogh)
291305b4d2 merge bitcoin#26497: Make ConsumeNetAddr always produce valid onion addresses (Kittywhiskers Van Gogh)
6a37934af4 partial bitcoin#26396: Avoid SetTxRelay for feeler connections (Kittywhiskers Van Gogh)
221a78ea84 merge bitcoin#25156: Introduce PeerManagerImpl::RejectIncomingTxs (Kittywhiskers Van Gogh)
cc694c2e5b merge bitcoin#22778: Reduce resource usage for inbound block-relay-only connections (Kittywhiskers Van Gogh)
6e6de54e5e net: use `Peer::m_can_relay_tx` when we mean `m_tx_relay != nullptr` (Kittywhiskers Van Gogh)
77526d2129 net: rename `Peer::m_block_relay_only` to `Peer::m_can_tx_relay` (Kittywhiskers Van Gogh)
Pull request description:
## Additional Information
* Dependency for https://github.com/dashpay/dash/pull/6333
* When backporting [bitcoin#22778](https://github.com/bitcoin/bitcoin/pull/22778), the `m_tx_inventory_known_filter.insert()` call in `queueAndMaybePushInv()` had to be moved out as `EXCLUSIVE_LOCKS_REQUIRED` does not seem to work with lambda parameters list (though it does work with capture list, [source](4b5e39290c/src/net_processing.cpp (L5781))), see error below
<details>
<summary>Compiler error:</summary>
```
net_processing.cpp:5895:21: note: found near match 'tx_relay->m_tx_inventory_mutex'
net_processing.cpp:5955:21: error: calling function 'operator()' requires holding mutex 'queueAndMaybePushInv.m_tx_inventory_mutex' exclusively [-Werror,-Wthread-safety-precise]
queueAndMaybePushInv(tx_relay, CInv(nInvType, hash));
^
net_processing.cpp:5955:21: note: found near match 'tx_relay->m_tx_inventory_mutex'
net_processing.cpp:5977:17: error: calling function 'operator()' requires holding mutex 'queueAndMaybePushInv.m_tx_inventory_mutex' exclusively [-Werror,-Wthread-safety-precise]
queueAndMaybePushInv(inv_relay, inv);
^
```
</details>
* Attempting to remove the `EXCLUSIVE_LOCKS_REQUIRED` or the `AssertLockHeld` are not options due to stricter thread sanitization checks being applied since [dash#6319](https://github.com/dashpay/dash/pull/6319) (and in general, removing annotations being inadvisable regardless)
* We cannot simply lock `peer->GetInvRelay()->m_tx_inventory_mutex` as a) the caller already has a copy of the relay pointer (which means that `m_tx_relay_mutex` was already held) that b) they used to lock `m_tx_inventory_mutex` (which we were already asserting) so c) we cannot simply re-lock `m_tx_inventory_mutex` as it's already already held, just through a less circuitous path.
* The reason locking is mentioned instead of asserting is that the compiler treats `peer->GetInvRelay()->m_tx_inventory_mutex` and `tx_relay->m_tx_inventory_mutex` as separate locks (despite being different ways of accessing the same thing) and would complain similarly to the error above if attempting to assert the former while holding the latter.
* As `m_tx_relay` is always initialized for Dash, to mimic the behaviour _expected_ upstream, in [bitcoin#22778](https://github.com/bitcoin/bitcoin/pull/22778), `SetTxRelay()` will _allow_ `GetTxRelay()` to return `m_can_tx_relay` (while Dash code that demands unconditional access can use `GetInvRelay()` instead) with the lack of `SetTxRelay()` resulting in `GetTxRelay()` feigning the absence of `m_can_tx_relay`.
This allows us to retain the same style of checks used upstream instead of using proxies like `!CNode::IsBlockOnlyConn()` to determined if we _should_ use `m_tx_relay`.
* Speaking of proxies, being a block-only connection is now no longer the only reason not to relay transactions
In preparation for [bitcoin#26396](https://github.com/bitcoin/bitcoin/pull/26396), proxy usage of `!CNode::IsBlockOnlyConn()` and `!Peer::m_block_relay_only` was replaced with the more explicit `Peer::m_can_tx_relay` before being used to gate the result of `GetTxRelay()`, to help it mimic upstream semantics.
## Breaking Changes
None expected
## Checklist
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 09504bdd1f
Tree-SHA512: f4f36f12f749b697dd4ad5521ed15f862c93ed4492a047759554aa80a3ce00dbd1bdc0242f7a4468f41f25925d5b79c8ab774d8489317437b1983f0a1277eecb
b6544791b4 Merge bitcoin/bitcoin#30705: test: Avoid intermittent block download timeout in p2p_ibd_stalling (merge-script)
745a819683 Merge bitcoin/bitcoin#30690: devtools, utxo-snapshot: Fix block height out of range in script (Ava Chow)
01b570e9e2 Merge bitcoin/bitcoin#29999: guix: fix suggested fake date for openssl-1.1.1l (Ava Chow)
432f352719 Merge bitcoin/bitcoin#30580: doc: Add note about distro's `g++-mingw-w64-x86-64-posix` version (merge-script)
1bd090e7fb Merge bitcoin/bitcoin#30597: doc: Drop no longer needed workaround for WSL (merge-script)
8a12237745 Merge bitcoin/bitcoin#30630: doc: Update ccache website link (merge-script)
f66547f84e Merge bitcoin/bitcoin#30588: depends: fix ZMQ CMake getcachesize check (merge-script)
ddaec964c0 Merge bitcoin/bitcoin#30565: depends: Fix `zeromq` build on OpenBSD (merge-script)
e4e5605ef8 Merge bitcoin/bitcoin#30552: test: fix constructor of msg_tx (merge-script)
df3c2392ca Merge bitcoin/bitcoin#26950: cleanse: switch to SecureZeroMemory for Windows cross-compile (merge-script)
57945ce337 Merge bitcoin/bitcoin#30506: depends: Cleanup postprocess commands after switching to CMake (merge-script)
e016ffada1 Merge bitcoin/bitcoin#29878: depends: build expat with CMake (merge-script)
62dcd43b58 Merge bitcoin/bitcoin#29880: depends: build FreeType with CMake (merge-script)
745addf6a7 Merge bitcoin/bitcoin#30245: net: Allow -proxy=[::1] on nodes with IPV6 lo only (Ava Chow)
4e144be0dd Merge bitcoin-core/gui#795: Keep focus on "Hide" while ModalOverlay is visible (Hennadii Stepanov)
69c04b2c48 Merge bitcoin/bitcoin#30372: util: Use SteadyClock in RandAddSeedPerfmon (merge-script)
ebed8af1d0 Merge bitcoin/bitcoin#30336: depends: update doc in Qt pwd patch (merge-script)
9793fb1a87 Merge bitcoin/bitcoin#30340: test: Added coverage to Block not found error using gettxoutsetinfo (Ava Chow)
479cb8bcb8 Merge bitcoin/bitcoin#30312: contrib: add R(UN)PATH check to ELF symbol-check (merge-script)
ca83773854 Merge bitcoin/bitcoin#30283: upnp: fix build with miniupnpc 2.2.8 (merge-script)
63e139d11b Merge bitcoin/bitcoin#30185: guix: show `*_FLAGS` variables in pre-build output (merge-script)
3be0d3e5cd Merge bitcoin/bitcoin#30097: crypto: disable asan for sha256_sse4 with clang and -O0 (merge-script)
3070c3e388 Merge bitcoin/bitcoin#30078: depends: set AR & RANLIB for CMake (merge-script)
Pull request description:
## Issue being fixed or feature implemented
Trivial backports
## What was done?
## How Has This Been Tested?
built locally
## Breaking Changes
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK b6544791b4
kwvg:
utACK b6544791b4
Tree-SHA512: 10b5af4e92c83fa9d6764b20bf066bba8e4c600402966fd5c1d6dad07b0549d8a42151a33f21e2f8263336c12a810a6f3fc2828d90bc98153e09c165d9e5b043
700b8c5ac5 Merge bitcoin/bitcoin#29658: Bugfix: GUI: Help messages already have a trailing newline, so don't add an extra one (merge-script)
a0cd305a7c Merge bitcoin/bitcoin#29948: test: add missing comparison of node1's mempool in MempoolPackagesTest (Ava Chow)
f1907ea997 Merge bitcoin/bitcoin#29984: net: Replace ifname check with IFF_LOOPBACK in Discover (merge-script)
daa6eeed5f Merge bitcoin/bitcoin#29960: depends: pass verbose through to cmake based makefiles (merge-script)
61a5832a6a Merge bitcoin/bitcoin#29907: test: Fix `test/streams_tests.cpp` compilation on SunOS / illumos (merge-script)
23f25a94fa Merge bitcoin/bitcoin#29872: test: Add missing Assert(mock_time_in >= 0s) to SetMockTime (merge-script)
a7daee71da Merge bitcoin/bitcoin#29689: lint: scripted-diff verification also requires GNU grep (Ava Chow)
3df1ca102b Merge bitcoin/bitcoin#29953: doc: Bash is needed in gen_id and is not installed on FreeBSD by default (merge-script)
b53b85409f Merge bitcoin/bitcoin#29850: net: Decrease nMaxIPs when learning from DNS seeds (Ava Chow)
c4a147cfea Merge bitcoin/bitcoin#28340: security: restrict abis in bitcoind.service (Ryan Ofsky)
acfdf9e438 Merge bitcoin/bitcoin#28373: doc: Add example of mixing private and public keys in descriptors (Ava Chow)
51bc8bdcd6 Merge bitcoin/bitcoin#29859: build: Fix false positive `CHECK_ATOMIC` test (merge-script)
bb4102c590 Merge bitcoin/bitcoin#29893: test: fix intermittent failure in p2p_compactblocks_hb.py (glozow)
4ecb76104e Merge bitcoin/bitcoin#29786: Drop Windows Socket dependency for `randomenv.cpp` (fanquake)
1a8e805aab Merge bitcoin/bitcoin#29498: test: Update --tmpdir doc string to say directory must not exist (fanquake)
81ca71c266 Merge bitcoin/bitcoin#29781: depends: add new LLVM debug macro (fanquake)
5ce92ca9ea Merge bitcoin/bitcoin#29527: depends: add -g to DEBUG=1 flags (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Trivial backports
## What was done?
## How Has This Been Tested?
built locally
## Breaking Changes
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 700b8c5ac5 but pls confirm Guix is happy now before merging
kwvg:
utACK 700b8c5ac5
Tree-SHA512: 9722979c4f0589cb02bfeaf39373713372a4aa1c8c5a55aa5b4d33388f73ef19231de3963b80ffaad25fbe0db90a133de5080baac691daed41b86a762b867b2d
fa5b58ea01fac1adb6336b8b6b5217193295c695 test: Avoid intermittent block download timeout in p2p_ibd_stalling (MarcoFalke)
Pull request description:
Fixes#30704
The goal of the test is to check the stalling timeout, not the block download timeout.
On extremely slow hardware (for example qemu virtual hardware), downloading the 1023 blocks may take longer than the block download timeout.
Fix it by pinning the time using mocktime, and only advance it when testing the stalling timeout.
ACKs for top commit:
tdb3:
CR ACK fa5b58ea01fac1adb6336b8b6b5217193295c695
brunoerg:
utACK fa5b58ea01fac1adb6336b8b6b5217193295c695
Tree-SHA512: 9a9221f264bea52be5e9fe81fd319f5a6970cd315cc5e9f5e2e049c5d84619b19b9f6f075cda8d34565c2d6c17a88fb57e195c66c271e40f73119a77caecb6d7
ec5e294e4b830766dcc4a80add0613d3705c1794 test: fix constructor of msg_tx (Martin Zumsande)
Pull request description:
In python, if the default value is a mutable object (here: a class) it is shared over all instances, so that one instance being changed would affect others to be changed as well.
This was the source of #30543, and possibly various other intermittent bugs in the functional tests, see
https://github.com/bitcoin/bitcoin/issues/29621#issuecomment-1999298224.
Fixes#30543Fixes#29621Fixes#25128
ACKs for top commit:
sipa:
utACK ec5e294e4b830766dcc4a80add0613d3705c1794. I believe some linters even warn about doing this.
maflcko:
ACK ec5e294e4b830766dcc4a80add0613d3705c1794
vasild:
ACK ec5e294e4b830766dcc4a80add0613d3705c1794 ❤️
theStack:
ACK ec5e294e4b830766dcc4a80add0613d3705c1794
Tree-SHA512: a6204fb1a326de3f9aa965f345fd658f6a4dcf78731db25cc905ff6eb8d4eeb65d14cc316305eebd89387aec8748c57c3a4f4ca62408f8e5ee53f535b88b1411
8ec24bdad89e2a72c394060ba5661a91f374b874 test: Added coverage to Block not found error using gettxoutsetinfo (kevkevinpal)
Pull request description:
#### Description
There were no tests that checked for the `Block not found` error called in `ParseHashOrHeight` when using `gettxoutsetinfo`, this change adds coverage to it.
You can see there are no tests that do the following by doing the below
`grep -nri "Block not found.*gettxoutsetinfo" ./test/functional/`
which leads to no results
ACKs for top commit:
achow101:
ACK 8ec24bdad89e2a72c394060ba5661a91f374b874
tdb3:
ACK 8ec24bdad89e2a72c394060ba5661a91f374b874
kristapsk:
ACK 8ec24bdad89e2a72c394060ba5661a91f374b874
brunoerg:
crACK 8ec24bdad89e2a72c394060ba5661a91f374b874
alfonsoromanz:
Re ACK 8ec24bdad89e2a72c394060ba5661a91f374b874
Tree-SHA512: 2c61c681e7304c679cc3d7dd13af1b795780e85716c25c7423d68104e253d01271e048e21bc21be35dbc7ec1a4fde94e439542f3cfd669fe5a16478c5fa982ab
e912717ff63f111d8f1cd7ed1fcf054e28f36409 test: add missing comparison of node1's mempool in MempoolPackagesTest (umiumi)
Pull request description:
#29941 Recreated a pull request because there was a conflict. Trying to resolve the conflict but the old one automatically closed.
Add missing comparison for TODO comments in `mempool_packages.py`
Also, notice that the ancestor size limits and descendant size limits actually implemented in #21800 , so I removed the todo for those two size limits.
ACKs for top commit:
kevkevinpal:
ACK [e912717](e912717ff6)
achow101:
ACK e912717ff63f111d8f1cd7ed1fcf054e28f36409
alfonsoromanz:
Tested ACK e912717ff63f111d8f1cd7ed1fcf054e28f36409. The code looks good to me and the test execution is successful.
rkrux:
tACK [e912717](e912717ff6)
Tree-SHA512: 8cb51746b0547369344c9ceef59599bfe9c91d424687af5e24dc6641f9e99fb433515d79c724e71fd3d5e02994f0cef623d3674367b8296b05c3c6fcdde282ef
1ae5b208d339fa984d9caf4fab89b0b2ba9cc197 test: fix intermittent failure in p2p_compactblocks_hb.py (Martin Zumsande)
Pull request description:
Fixes#29860
As a result of node1 receiving a block, it sends out SENDCMPCT messages to some of its peers to update the high-bandwidth status. We need to wait until those are received and processed by the peers to avoid intermittent failures. Before, we'd only wait until all peers have synced with the new block (within `generate`) which is not sufficient.
I could reproduce the failure by adding a `std::this_thread::sleep_for(std::chrono::milliseconds(1000));` sleep to the [net_processing code](c7567d9223/src/net_processing.cpp (L3763)) that processes `NetMsgType::SENDCMPCT`.
ACKs for top commit:
instagibbs:
ACK 1ae5b208d339fa984d9caf4fab89b0b2ba9cc197
alfonsoromanz:
Tested ACK 1ae5b208d339fa984d9caf4fab89b0b2ba9cc197
glozow:
ACK 1ae5b208d339fa984d9caf4fab89b0b2ba9cc197
Tree-SHA512: 47c29616e73a5e0ff966fc231e4f672c1a6892511e5c10a3905b30ad6b2a3d1267fa0a88bd8f64b523fe580199d22a43545c84e361879e5096483152065c4b9a
d4e36ae80d4b3f03647fd9057461edf7ecd794a3 test: Update --tmpdir doc string to say directory must not exist (kevkevin)
Pull request description:
The error message given if passing an existing dir to --tmpdir is confusing so this makes it clear that the directory must not already exist
This change is motivated by this comment https://github.com/bitcoin/bitcoin/pull/29335#issuecomment-1960913020
ACKs for top commit:
maflcko:
lgtm ACK d4e36ae80d4b3f03647fd9057461edf7ecd794a3
davidgumberg:
ACK d4e36ae80d
Tree-SHA512: fb31fd079767abbf94076615817943f35f5c9262fc97e65c631a18d33b3a343fe6a2d151613256e632d2b372ab2de0435f4712309b4a77ed3c663fd93a7dcdd1
7c6c93d201 fix: remove missing comment to follow-up for bitcoin#15864 (Konstantin Akimov)
65226da849 Merge bitcoin/bitcoin#22229: test: consolidate to f-strings (part 1) (MarcoFalke)
ad2c5a53ee refactor: unify feature_notifications.py after #5522 with bitcoin's codebase (Konstantin Akimov)
Pull request description:
## Issue being fixed or feature implemented
Just one backport bitcoin#22229 because it is a big size, even though there's nothing non-trivial in it.
Though, even it is called as part I, there has not been part II yet.
## What was done?
Some preparation, code unifications to make bitcoin#22229 with less conflicts and finally backport of itselfl.
## How Has This Been Tested?
Run unit & functional test
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
utACK 7c6c93d201
PastaPastaPasta:
utACK 7c6c93d201
Tree-SHA512: fe296e3255d45a7a1924bd1e5e21634b3cd36ea3f71cf5e8684b54336771665ea7758de7bfc78721669a928f967e7d4db7b1da0a5cd275feb1a2ec0df841ad5c
37389c7d38 Merge bitcoin/bitcoin#28781: depends: latest config.guess & config.sub (fanquake)
3239f1525d Merge bitcoin/bitcoin#28479: build: use _LIBCPP_ENABLE_DEBUG_MODE over ENABLE_ASSERTIONS (fanquake)
45cc44bcf9 Merge bitcoin/bitcoin#27628: build: Fix shared lib linking for darwin with lld (fanquake)
b8ddcd937c Merge bitcoin/bitcoin#27575: Introduce platform-agnostic `ALWAYS_INLINE` macro (fanquake)
71c6d7f6ca Merge bitcoin/bitcoin#26653: test, init: perturb file to ensure failure instead of only deleting them (fanquake)
417f71a587 Merge bitcoin/bitcoin#27422: test: add coverage to rpc_scantxoutset.py (fanquake)
898dcbdc4f Merge bitcoin/bitcoin#27559: doc: clarify processing of mempool-msgs when NODE_BLOOM (glozow)
a4e429cb5a Merge bitcoin/bitcoin#26953: contrib: add ELF OS ABI check to symbol-check.py (fanquake)
deb7de26dd Merge bitcoin/bitcoin#26604: test: add coverage for `-bantime` (fanquake)
f725ed509a Merge bitcoin/bitcoin#26314: test: perturb anchors.dat to test error during initialization (fanquake)
3306f96d80 Merge bitcoin/bitcoin#25937: test: add coverage for rpc error when trying to rescan beyond pruned data (fanquake)
712dcaf86b Merge bitcoin/bitcoin#27516: test: simplify uint256 (de)serialization routines (fanquake)
90d65f25e1 Merge bitcoin/bitcoin#27508: build: use latest config.{guess,sub} in depends (fanquake)
df7be026e4 Merge bitcoin/bitcoin#27506: test: prevent intermittent failures (fanquake)
9b58b2d97b Merge bitcoin/bitcoin#27447: depends: Remove `_LIBCPP_DEBUG` from depends DEBUG mode (fanquake)
07770b77a1 Merge bitcoin/bitcoin#26741: doc: FreeBSD DataDirectoryGroupReadable Setting (fanquake)
5645362f11 Merge bitcoin/bitcoin#27362: test: remove `GetRNGState` lsan suppression (fanquake)
671e8e6851 Merge bitcoin/bitcoin#27368: refactor: Drop no longer used `CNetMsgMaker` instances (fanquake)
a11690bf62 Merge bitcoin/bitcoin#27301: depends: make fontconfig build under clang-16 (fanquake)
0a94b3f27b Merge bitcoin/bitcoin#27328: depends: fix osx build with clang 16 (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 37389c7d38
kwvg:
utACK 37389c7d38
Tree-SHA512: 1ea075a58361f57e037febcf003d380ab845b6c8e1c62d9fcf8954d46cd006046d1951f15a41a5deb9ab7af734df9dafcf89f33d115d78246752f7e2cd13f4ee
168e5e4a50 Merge bitcoin/bitcoin#28877: bench: Update nanobench to 4.3.11 (fanquake)
417c86b949 Merge bitcoin/bitcoin#28105: doc: Clarify that -fstack-reuse=all bugs exist on all versions of GCC (fanquake)
a620cccd81 Merge bitcoin/bitcoin#26970: test: fix immediate tx relay in wallet_groups.py (merge-script)
f115d9c27f Merge bitcoin/bitcoin#27061: doc: Document affected gcc versions for -fstack-reuse=none workaround (fanquake)
6889a8db29 Merge bitcoin/bitcoin#27056: doc: use arch agnostic clang path in fuzzing doc (macOS) (MarcoFalke)
97858384ec Merge bitcoin/bitcoin#21995: build: Make dependency package archive timestamps deterministic (fanquake)
c4760bb32e Merge bitcoin/bitcoin#27030: Update nanobench to version v4.3.10 (fanquake)
a7e3c2c916 Merge bitcoin-core/gui#705: doc: Fix comment about how wallet txs are sorted (Hennadii Stepanov)
44e6c9e902 Merge bitcoin/bitcoin#27004: test: Use std::unique_ptr over manual delete in coins_tests (fanquake)
2ab1989a39 Merge bitcoin/bitcoin#27010: refactor: use `Hash` helpers for double-SHA256 calculations (MarcoFalke)
c681aaad30 Merge bitcoin/bitcoin#22811: build: Fix depends build system when working with subtargets (fanquake)
d1b7386374 Merge bitcoin/bitcoin#26930: fuzz: Actually use mocked mempool in tx_pool target (MarcoFalke)
cd53a195a6 Merge bitcoin/bitcoin#26873: doc: add databases/py-sqlite3 to FreeBSD test suite deps (fanquake)
8cc5f11a2f Merge bitcoin/bitcoin#26506: refactor: rpc: use convenience fn to auto parse non-string parameters (MarcoFalke)
662302c42b Merge bitcoin/bitcoin#26805: tests: Use unique port for ZMQ tests to allow for multiple test instances (MarcoFalke)
66a3981a7a Merge bitcoin/bitcoin#24279: build: Make `$(package)_*_env` available to all `$(package)_*_cmds` (fanquake)
3261092f85 Merge bitcoin/bitcoin#26520: doc: test: update/fix TestShell example instructions (fanquake)
5f78859562 Merge bitcoin/bitcoin#25248: refactor: Add LIFETIMEBOUND / -Wdangling-gsl to Assert() (fanquake)
459425776c Merge bitcoin/bitcoin#26229: test: Use proper Boost macros instead of assertions (MacroFake)
3be81a2d4c Merge bitcoin/bitcoin#25915: test: Fix wallet_balance intermittent issue (Andrew Chow)
da1d3f2654 Merge bitcoin/bitcoin#25663: tracing: do not use `coin` after move in `CCoinsViewCache::AddCoin` (MacroFake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 168e5e4a50
Tree-SHA512: 3099e09bc500a86bffafea0db136e3213f69b69e7af74304c171780e56ff1ff4c973a228962cf80aec62158ded19365d6f8506ef202a15751a43851574f082e2
750447e345 Merge bitcoin/bitcoin#20586: Fix Windows build with --enable-werror (W. J. van der Laan)
368a6ef512 Merge bitcoin-core/gui#390: Add SubFeeFromAmount to options (Hennadii Stepanov)
7df9788c85 Merge bitcoin-core/gui#381: refactor: Make BitcoinCore class reusable (W. J. van der Laan)
40a8b925db Merge bitcoin/bitcoin#22688: contrib: use `keys.openpgp.org` to retrieve builder keys (fanquake)
62b5358a9c Merge #11909: contrib: Replace developer keys with list of pgp fingerprints (Wladimir J. van der Laan)
1ff42b40e3 Merge bitcoin/bitcoin#21500: wallet, rpc: add an option to list private descriptors (Samuel Dobson)
5a803ae765 Merge bitcoin/bitcoin#22378: test: remove confusing `MAX_BLOCK_BASE_SIZE` (MarcoFalke)
42a0cf0709 Merge bitcoin/bitcoin#21562: [net processing] Various tidying up of PeerManagerImpl ctor (MarcoFalke)
e3c69da4f2 Merge bitcoin/bitcoin#22232: refactor: Pass interpreter flags as uint32_t instead of signed int (MarcoFalke)
Pull request description:
## What was done?
Regular batch of backports from Bitcoin v23
## How Has This Been Tested?
Run unit and functional tests
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
utACK 750447e345
PastaPastaPasta:
utACK 750447e345
Tree-SHA512: 46e0de22798937b6b0a228ea9ca39b465cd9e8b822f973f23af80fccb1fe2d733716930d3f3843aaf41a58225aa98e18cd5d5e361245f64e146ff1be118f91ca
c371cae07a7ba045130568b6abc470eaa4f95ef4 test, init: perturb file to ensure failure instead of only deleting them (brunoerg)
Pull request description:
In `feature_init.py` there is a TODO about perturbing the files instead of only testing by deleting them.
```py
# TODO: at some point, we should test perturbing the files instead of removing
# them, e.g.
#
# contents = target_file.read_bytes()
# tweaked_contents = bytearray(contents)
# tweaked_contents[50:250] = b'1' * 200
# target_file.write_bytes(bytes(tweaked_contents))
#
# At the moment I can't get this to work (bitcoind loads successfully?) so
# investigate doing this later.
```
This PR adds it by writing into the file random bytes and checking whether it throws an error when starting.
ACKs for top commit:
MarcoFalke:
lgtm ACK c371cae07a7ba045130568b6abc470eaa4f95ef4
Tree-SHA512: d691eee60b91dd9d1b200588608f56b0a10dccd9761a75254b69e0ba5e5866cae14d2f90cb2bd7ec0f95b0617c2562cd33f20892ffd16355b6df770d3806a0ff
7e3d4f8e86e86f32d8911abd458b9e7c939ef3d5 test: add coverage to ensure the first arg of scantxoutset is needed (ismaelsadeeq)
Pull request description:
Include a test that checks whether the first argument of scantxoutset RPC call "start" is required.
The rpc call should fail if the "start" argument is not provided.
ACKs for top commit:
MarcoFalke:
lgtm ACK 7e3d4f8e86e86f32d8911abd458b9e7c939ef3d5
Tree-SHA512: 6a456af9f3ccd5437be2edcd61936eb9f9c21ab926a6056c2c11b6b5121d1caca4e1f2ffd09015f9414af152c635a20e1da041eefdef980afbe8a0e8ccce07bd
9c18992bbaf649f8c5461d5e4dc39eb1a07ffc77 test: add coverage for `-bantime` (brunoerg)
Pull request description:
This PR adds test coverage for `-bantime`. This flag sets the time in seconds how long the IP is banned (in the case you don't explicitly set `bantime` when using `setban`).
ACKs for top commit:
MarcoFalke:
lgtm ACK 9c18992bbaf649f8c5461d5e4dc39eb1a07ffc77
Tree-SHA512: e95f8608aa5df9b09cc5577daae662ed79ef5d5c69ee5e704d7c69520b9b51cc142e9e6be69d80356eda25a5215c4770b1a208638560c48cd3bc8f6d195a371f
33fdfc7986455191df8ce339261bc0561115cf7f test: perturb anchors.dat to test it doesn't throw an error during initialization (brunoerg)
Pull request description:
Got some inspiration from `feature_init`. This PR tests whether perturbing `anchors.dat` doesn't throw any error during initialization.
3f1f5f6f1e/src/addrdb.cpp (L223-L235)
ACKs for top commit:
MarcoFalke:
lgtm ACK 33fdfc7986455191df8ce339261bc0561115cf7f
Tree-SHA512: e6584debb37647677581fda08366b45b42803022cc4c4f1d5a7bd5e9e04d64da77656dad2b804855337487bdcfc891f300a2e03668d6122de769dd14f39af9ed
cca4f82b828669ae23f6ac64fb83e068b81ae189 test: add coverage for rpc error when trying to rescan beyond pruned data (brunoerg)
Pull request description:
This PR adds test coverage for the following rpc error:
15692e2641/src/wallet/rpc/transactions.cpp (L896-L899)
ACKs for top commit:
MarcoFalke:
lgtm ACK cca4f82b828669ae23f6ac64fb83e068b81ae189
aureleoules:
ACK cca4f82b828669ae23f6ac64fb83e068b81ae189
Tree-SHA512: 724a055e9f6cddf1935699e8769015115f24f6485a0bd87e8660072ee44a15c1bddfdda848acc101ea7184b7e65a33b5b0d80b563d2ba3ecdab7a631378d6476
96bf0bca4a0e3aa0b7c07d8c225861e72f970fa9 test: simplify uint256 (de)serialization routines (Sebastian Falbesoner)
Pull request description:
These routines look fancy, but do nothing more than converting between byte objects of length 32 to/from integers in little endian byte order and can be replaced by simple one-liners, using the `int.{from,to}_bytes` methods (available since Python 3.2).
ACKs for top commit:
MarcoFalke:
lgtm ACK 96bf0bca4a0e3aa0b7c07d8c225861e72f970fa9
brunoerg:
crACK 96bf0bca4a0e3aa0b7c07d8c225861e72f970fa9
Tree-SHA512: f3031502d61a936147867ad8a0efa841a9bbdd2acf8781653036889a38524f4f1a5c86b1e07157bf2d9663097e7b84be6846678d0883d2a334beafd87e9510f0
d573e4ff86 Merge bitcoin/bitcoin#28084: doc: update windows `-fstack-clash-protection` doc (fanquake)
e3bbd1a46e Merge bitcoin-core/gui#740: Show own outputs on PSBT signing window (Hennadii Stepanov)
70cbd3f8a2 Merge bitcoin/bitcoin#28044: test: indexes, fix on error infinite loop (Ryan Ofsky)
551109105a Merge bitcoin/bitcoin#28036: test: Restore unlimited timeout in IndexWaitSynced (fanquake)
2767a13268 Merge bitcoin/bitcoin#28021: docs: fixup honggfuzz fuzz patch (fanquake)
af944b7c8d Merge bitcoin/bitcoin#28013: doc: Fix verify-binaries link in contrib README (fanquake)
79a20f96a5 Merge bitcoin/bitcoin#27929: Added static_assert to check that base_blob is using whole bytes. (fanquake)
ece625c754 Merge bitcoin/bitcoin#27914: feerate: For GetFeePerK() return nSatoshisPerK instead of round trip through GetFee (fanquake)
ba5f4c0332 Merge bitcoin/bitcoin#27906: doc: test: update TestShell instructions (fanquake)
fbc6c6e644 Merge bitcoin/bitcoin#27875: build: make sure we can overwrite config.{guess,sub} before doing so (fanquake)
e2fcd1d947 Merge bitcoin/bitcoin#27225: doc: document json rpc endpoints (fanquake)
324db8bb31 Merge bitcoin/bitcoin#27603: test: added coverage to mining_basic.py (glozow)
a21b4b16f7 Merge bitcoin/bitcoin#27802: Update .style.yapf (fanquake)
ee6b7d66f1 Merge bitcoin/bitcoin#27721: depends: remove redundant stdlib option (fanquake)
f27778abe7 Merge bitcoin/bitcoin#27561: test: Explicitly specify directory where to search tests for (fanquake)
105442f8cb Merge bitcoin/bitcoin#26422: build: Use newest `config.{guess,sub}` available (fanquake)
bef9631e99 Merge bitcoin/bitcoin#27661: doc, test: Document steps to reproduce TSan warning for `libdb` (fanquake)
277766fcef Merge bitcoin/bitcoin#27493: depends: no-longer nuke libc++abi.so* in native_clang package (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK d573e4ff86
knst:
utACK d573e4ff86
Tree-SHA512: c94f55888de1758457a617253ddd13013141398438a32343ee10ee170d76b7e091ef4479e96f7983cb7661ceb99984a60a668ee142a9aea52b8c5305738398cc
4101fea620 Merge bitcoin/bitcoin#28304: doc: Remove confusing assert linter (fanquake)
c59cb158e5 Merge bitcoin/bitcoin#26282: wallet: have prune error take precedence over assumedvalid (fanquake)
e2e8598c5a Merge bitcoin/bitcoin#23997: wallet: avoid rescans under assumed-valid blocks (Andrew Chow)
b66eebe64d Merge bitcoin/bitcoin#25599: build: Check for std::atomic::exchange rather than std::atomic_exchange (fanquake)
1204dc0f83 Merge bitcoin/bitcoin#25486: test: fix failing test `interface_usdt_utxocache.py` (MacroFake)
de17997621 Merge bitcoin/bitcoin#24062: refactor: replace RecursiveMutex `m_most_recent_block_mutex` with Mutex (MacroFake)
c91f010e0e Merge bitcoin/bitcoin#25092: doc: various developer notes updates (MacroFake)
f39fcd1402 Merge bitcoin/bitcoin#24988: lint: Mention NONFATAL_UNREACHABLE in lint-assertions.py (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 4101fea620
kwvg:
utACK 4101fea620
Tree-SHA512: e948ff58b256f2ecb9611681f773570d233985f1470e3eaa6899f3b7e53701c06f56ed5b965d250e22764938b0afebc8d85f92879ba111a0e20127cd63e99809
68faa87881f5334b2528db4adc72ec19d94316a3 test: use f-strings in mining_*.py tests (fanquake)
c2a5d560df2824df5731100c2584e8ad7a3d7bc2 test: use f-strings in interface_*.py tests (fanquake)
86d958262dff43002820d58ccb8958e2dbfb9d5b test: use f-strings in feature_proxy.py (fanquake)
31bdb33dcb8345df1bb94b28e811252a918d7dcb test: use f-strings in feature_segwit.py (fanquake)
b166d54c3cbb0c028210cee977b3dcde5ac5474f test: use f-strings in feature_versionbits_warning.py (fanquake)
cf6d66bf941d946600047d712c7cd15d7605322e test: use f-strings in feature_settings.py (fanquake)
6651d77f22862716f5bd7d0b31cfbd3937ab7b1d test: use f-strings in feature_pruning.py (fanquake)
961f5813ba65b6a601081912c4ece96c2679794d test: use f-strings in feature_notifications.py (fanquake)
1a546e6f6ca95772f0d7dbc2792477becbb8ea63 test: use f-strings in feature_minchainwork.py (fanquake)
6679eceacc915a8ea7cd7063f103ffc5eb9da884 test: use f-strings in feature_logging.py (fanquake)
fb633933ab570e945d2a366f37eeff39f516c613 test: use f-strings in feature_loadblock.py (fanquake)
e9ca8b254d4b9567831c0e113ce1c0a2b4795a95 test: use f-strings in feature_help.py (fanquake)
ff7e3309995a8960ac371741b2b00c6da40f7490 test: use f-strings in feature_filelock.py (fanquake)
d5a6adc5e478fa5c6e562377eea873dc38e66578 test: use f-strings in feature_fee_estimation.py (fanquake)
a2de33cbdc79202bccddb4beadfde88266ac979f test: use f-strings in feature_dersig.py (fanquake)
a2502cc63fd308be8af840962da9c53339433fa6 test: use f-strings in feature_dbcrash.py (fanquake)
3e2f84e7a96cb4b97b609ac853f78edd0ed43f82 test: use f-strings in feature_csv_activation.py (fanquake)
e2f1fd8ee92fa421b6d293169044d6ddd5a9b8df test: use f-strings in feature_config_args.py (fanquake)
36d33d32b1b498b61f56d552f6e2c1d064f978c3 test: use f-strings in feature_cltv.py (fanquake)
dca173cc044270b30782b1e3355e9dcb8c534295 test: use f-strings in feature_blocksdir.py (fanquake)
5453e8706278918ac51a725e81599cfa18c8cdbc test: use f-strings in feature_backwards_compatibility.py (fanquake)
6f3d5ad67ac8e7b50abae1a2949898d858e38106 test: use f-strings in feature_asmap.py (fanquake)
Pull request description:
Rather than using 3 different ways to build/format strings (sometimes all in the same test, i.e [`feature_config_args.py`](https://github.com/bitcoin/bitcoin/blob/master/test/functional/feature_config_args.py)), consolidate to using [f-strings (3.6+)](https://docs.python.org/3/reference/lexical_analysis.html#f-strings), which are generally more concise / readable, as well as more performant than existing methods.
This deals with the `feature_*.py`, `interface_*.py` and `mining_*.py` tests.
See also: [PEP 498](https://www.python.org/dev/peps/pep-0498/)
ACKs for top commit:
mjdietzx:
reACK 68faa87881f5334b2528db4adc72ec19d94316a3
Zero-1729:
crACK 68faa87881f5334b2528db4adc72ec19d94316a3
Tree-SHA512: d4e1a42e07d96d2c552387a46da1534223c4ce408703d7568ad2ef580797dd68d9695b8d19666b567af37f44de6e430e8be5db5d5404ba8fcecf9f5b026a6efb
f665c6ecda854adaaa629e961b1912847b42fd34 test: fix failing test interface_usdt_utxocache.py (Sebastian Falbesoner)
Pull request description:
The `from_node` argument doesn't exist anymore for `MiniWallet.create_self_transfer` since PR #25435 (commit fa8421bc5bcd483a9501257073db17ff2e76eb46), leading to an error on master:
```
$ sudo ./test/functional/interface_usdt_utxocache.py
2022-06-27T17:45:35.585000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_7s1djjo1
2022-06-27T17:45:36.515000Z TestFramework (INFO): testing the utxocache:uncache tracepoint API
2022-06-27T17:45:36.517000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
File "/home/honeybadger/bitcoin/test/functional/test_framework/test_framework.py", line 133, in main
self.run_test()
File "/home/honeybadger/bitcoin/./test/functional/interface_usdt_utxocache.py", line 149, in run_test
self.test_uncache()
File "/home/honeybadger/bitcoin/./test/functional/interface_usdt_utxocache.py", line 172, in test_uncache
invalid_tx = self.wallet.create_self_transfer(
TypeError: create_self_transfer() got an unexpected keyword argument 'from_node'
2022-06-27T17:45:36.568000Z TestFramework (INFO): Stopping nodes
[...]
```
Fix this by removing the argument. (Unfortunately, the USDT tests don't seem to run on any CI target, I guess that's due to missing permissions to hook into the kernel.)
ACKs for top commit:
MarcoFalke:
cr ACK f665c6ecda854adaaa629e961b1912847b42fd34
Tree-SHA512: 74f8e398739a25ab5518ff71b998d03d4e529a786ba5b424509de81a511ad3e2e1cd38a5b7bb9f1f5a21340391d6807f4951ff39fa3a2ad65a3b11b989eebea6
14405e8d4d259c18a21fc006d0a27550be3171f8 doc: test: update TestShell instructions (ismaelsadeeq)
Pull request description:
Fixes #27904
From #27904 and IRC.
Update [Testshell instructions ](https://github.com/bitcoin/bitcoin/blob/master/test/functional/test-shell.md#2-importing-testshell-from-the-bitcoin-core-repository)
E.g `TestShell.setup()` throws
```
AttributeError: type object 'TestShell' has no attribute 'setup'
```
Parentheses are missing, it should be `TestShell().setup()`
ACKs for top commit:
Sjors:
utACK 14405e8d4d259c18a21fc006d0a27550be3171f8
brunoerg:
crACK 14405e8d4d259c18a21fc006d0a27550be3171f8
hernanmarino:
utACK 14405e8d4d259c18a21fc006d0a27550be3171f8
Tree-SHA512: ffe5fa1103a3b00ef0ee99879adae967b0da07cb8f8451c4c261b0a70b3b666af7aeaacd6f46f85a84ee5e9c7c7ed49700209b5b1f124d7a76efc420ad5c9cd9
a7b46a1feae60e38fe4bdcacf5034f44cae49222 test: added coverage to mining_basic.py (kevkevin)
Pull request description:
Included a test that checks if we call submitblock with block.vtx.empty() then it throws an rpc deserialization error, currently we only test if !block.vtx->IsCoinBase() throws an rpc deserialization error
I've tested to make sure this actually doing what I intended by breaking up this if block into two if blocks with different error messages and running the functional test
322ec63b01/src/rpc/mining.cpp (L963)
This change should increase the test coverage for the `submitblock()` rpc in `./src/rpc/mining.cpp`
ACKs for top commit:
theStack:
ACK a7b46a1feae60e38fe4bdcacf5034f44cae49222
Tree-SHA512: 4078cb1fa879cc9e34438319f73085b521b90a5a95348b23e494cf8e5ac792ec426bc0e1a63e949645e16afebe54c5f35a194f02e20b7273871163d89a5c44e6
c44f3f231988dc05c4c7a8a96bc2e7b1a54da277 test: Explicitly specify directory where to search tests for (Hennadii Stepanov)
Pull request description:
For out-of-source builds, the `test/functional/test_runner.py` is supposed to be run from the build directory which allows it to pick the `test/config.ini` file generated by the build system. Currently, it works accidently for the following reasons:
- on POSIX systems, when running a created by Autoconf symlink to the `test/functional/test_runner.py` in the source directory, it actually has the source directory location in the `sys.path`.
- on Windows (the `build_msvc` directory) VS project puts and copies every build artifact into the source tree (which is wrong and ugly).
This PR makes `test/functional/test_runner.py` work from a build directory in any form (a symbolic link, a hard link, a copy) on _all_ supported platforms, which is highly desirable in the upcoming [CMake-based build system](https://github.com/bitcoin/bitcoin/pull/25797).
For the current master branch, this PR has no behaviour change.
Required for https://github.com/hebasto/bitcoin/pull/15.
---
**Steps to reproduce the issue**
While the issue is mostly specific to Windows and CMake builds, it is still possible to reproduce it on the current master branch.
1. Make an out-of-source build:
```
$ ./autogen.sh
$ mkdir ../build && cd ../build
$ ../bitcoin/configure
$ make
```
2. Note that Autoconf created a symbolic link `test/functional/test_runner.py` in the `../build` directory:
```
$ ls -l test/functional/test_runner.py
lrwxrwxrwx 1 hebasto hebasto 47 May 5 17:40 test/functional/test_runner.py -> ../../../bitcoin/test/functional/test_runner.py
```
which works flawlessly.
3. However, replacing this symbolic link with a hard link or a copy of `test/functional/test_runner.py` from the source tree will cause the following error:
```
$ cp ../bitcoin/test/functional/test_runner.py test/functional/test_runner.py
$ ls -l test/functional/test_runner.py
$ ./test/functional/test_runner.py
Temporary test directory at /tmp/test_runner_₿_🏃_20230505_175104
Running Unit Tests for Test Framework Modules
E
======================================================================
ERROR: test_framework (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_framework
Traceback (most recent call last):
File "/usr/lib/python3.10/unittest/loader.py", line 154, in loadTestsFromName
module = __import__(module_name)
ModuleNotFoundError: No module named 'test_framework'
----------------------------------------------------------------------
Ran 1 test in 0.000s
FAILED (errors=1)
Early exiting after failure in TestFramework unit tests
```
ACKs for top commit:
stickies-v:
re-ACK c44f3f2319
MarcoFalke:
lgtm ACK c44f3f231988dc05c4c7a8a96bc2e7b1a54da277 💸
Tree-SHA512: 622ff629080a55f76dd4c1dab6699de0e9f06b75da3315cd3b31b972ef4bde746458bf3e8a95e879b3c6a63be2368af70005a83f6a3c85c4f1ba5be51e91a61d
ab4efad51b9ba276ffeb6871931e13772493f7cc test: fix immediate tx relay in wallet_groups.py (Sebastian Falbesoner)
Pull request description:
In the functional test wallet_groups.py we whitelist peers on all nodes (`-whitelist=noban@127.0.0.1`) to enable immediate tx relay for fast mempool synchronization. However, considering that this setting only applies to inbound peers and the default test topology looks like this:
```
node0 <--- node1 <---- node2 <--- ... <-- nodeN
```
txs propagate fast only from lower- to higher-numbered nodes (i.e. "left to right" in the above diagram) and take long from higher- to lower-numbered nodes ("right to left") since in the latter direction we only have outbound peers, where the trickle relay is still active. As a consequence, if a tx is submitted from any node other than node0, the mempool synchronization can take quite long.
This PR fixes this by simply adding another connection from node0 to the last node, leading to a ~2-3x speedup (5 runs measured via `time ./test/functional/wallet_groups.py` are shown):
```
master:
0m53.31s real 0m08.22s user 0m05.60s system
0m32.85s real 0m07.44s user 0m04.08s system
0m46.40s real 0m09.18s user 0m04.23s system
0m46.96s real 0m11.10s user 0m05.74s system
0m57.23s real 0m10.53s user 0m05.59s system
PR:
0m19.64s real 0m09.58s user 0m05.50s system
0m18.05s real 0m07.77s user 0m04.03s system
0m18.99s real 0m07.90s user 0m04.25s system
0m17.49s real 0m07.56s user 0m03.92s system
0m18.11s real 0m07.74s user 0m03.88s system
```
Note that in most tests this is not a problem since txs very often originate from node0.
ACKs for top commit:
brunoerg:
utACK ab4efad51b9ba276ffeb6871931e13772493f7cc
Tree-SHA512: 12675357e6eb5a18383f2bfe719a184c0790863b37a98749d8e757dd5dc3a36212e16a81f0a192340c11b793eda00db359c7011f46f7c27e3a093af4f5b62147
c6119f478876f245ca5c65dd05da4cdc5be0e91f tests: Use unique port for ZMQ tests (Andrew Chow)
Pull request description:
The ZMQ interface tests should use unique ports as we do for the p2p and rpc ports so that multiple instances of the test can be run at the same time.
Without this, the test may hang until killed, or fail.
ACKs for top commit:
MarcoFalke:
ACK c6119f478876f245ca5c65dd05da4cdc5be0e91f
Tree-SHA512: 2ca3ed2f35e5a83d7ab83740674fed362a8d146dc751156cfe100133a591347cd1ac9d164046f1744d65451a57c52cb22d3bb2161105f421f8f655c4a2512c59
31d0067f8bccd6b090bf88bad8472bb5cb1f20a4 doc: test: update/fix TestShell example instructions (Sebastian Falbesoner)
Pull request description:
This PR tackles two issues in the TestShell documentation:
- add missing instruction for creating a wallet prior to the `getnewaddress` call (needed as there is no default wallet created anymore since v0.21)
- fix `generatetoaddress` call syntax (the scripted-diff in commit fa0b916971e5bc23ad6396831940a2899ca05402 only worked for tests using `BitcoinTestFramework`)
ACKs for top commit:
fanquake:
ACK 31d0067f8bccd6b090bf88bad8472bb5cb1f20a4 - current instructions don't work. These do.
Tree-SHA512: d2b7808a06892ad16728cb2b6d4a72b255ad711d27fe98b1de562f80444e7bb25d73296abdde4308162fe3be702864e2f7b7dbbbb000fe54c709951c09e6c730
0dbafcee46 Merge bitcoin/bitcoin#27289: Refactor: Remove unused FlatFilePos::SetNull (fanquake)
dbe2e04d62 Merge bitcoin/bitcoin#27212: test: Make the unlikely race in p2p_invalid_messages impossible (fanquake)
6f6b718f78 Merge bitcoin/bitcoin#27236: util: fix argsman dupe key error (fanquake)
74c6e38530 Merge bitcoin/bitcoin#27205: doc: Show how less noisy clang-tidy output can be achieved (fanquake)
9e552f0293 Merge bitcoin/bitcoin#27232: Use string interpolation for default value of -listen (fanquake)
2a39b93233 Merge bitcoin/bitcoin#27226: test: Use self.wait_until over wait_until_helper (fanquake)
be2e16f33a Merge bitcoin/bitcoin#27192: util: add missing include and fix function signature (fanquake)
176a4a60d2 Merge bitcoin/bitcoin#27173: valgrind: remove libsecp256k1 suppression (fanquake)
d2fc8be331 Merge bitcoin/bitcoin#27154: doc: mention sanitizer suppressions in developer docs (glozow)
f5b4cc7e32 Merge bitcoin/bitcoin#16195: util: Use void* throughout support/lockedpool.h (Andrew Chow)
c66c0fdbf8 Merge bitcoin/bitcoin#27137: test: Raise PRNG seed log to INFO (fanquake)
bba215031b Merge bitcoin/bitcoin#25950: test: fix test abort for high timeout values (and `--timeout-factor 0`) (fanquake)
6751add2ea Merge bitcoin/bitcoin#27107: doc: remove mention of "proper signing key" (merge-script)
34c895a542 Merge bitcoin/bitcoin#26997: psbt: s/transcation/transaction/ (fanquake)
befdbeddf9 Merge bitcoin/bitcoin#27097: descriptors: fix docstring (param [in] vs [out]) (fanquake)
c98dd824b6 Merge bitcoin/bitcoin#27080: Wallet: Zero out wallet master key upon locking so it doesn't persist in memory (Andrew Chow)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
knst:
utACK 0dbafcee46
UdjinM6:
utACK 0dbafcee46
Tree-SHA512: e93a1136e848aa6c6f3d9fb0567b3e284975d35e82bbc1d9a8cd908067df4bf1257c939882abcaca6820360a4982b991f505a1165d95e1a8b52c9b181b7026b7
bb822a7af86897a9b6a5d616f193c258e8e76729 wallet, rpc: add listdescriptors private option (S3RK)
Pull request description:
Rationale: make it possible to backup your wallet with `listdescriptors` command
* The default behaviour is still to show public version
* For private version only the root xprv is returned
Example use-case:
```
> bitcoin-cli -regtest -named createwallet wallet_name=old descriptors=true
> bitcoin-cli -regtest -rpcwallet=old listdescriptors true | jq '.descriptors' > descriptors.txt
> bitcoin-cli -regtest -named createwallet wallet_name=new descriptors=true blank=true
> bitcoin-cli -regtest -rpcwallet=new importdescriptors "$(cat descriptors.txt)"
```
In case of watch-only wallet without private keys there will be following output:
```
error code: -4
error message:
Can't get descriptor string.
```
ACKs for top commit:
achow101:
re-ACK bb822a7af86897a9b6a5d616f193c258e8e76729
Rspigler:
tACK bb822a7af86897a9b6a5d616f193c258e8e76729
jonatack:
ACK bb822a7af86897a9b6a5d616f193c258e8e76729 per `git diff 2854ddc bb822a7`
prayank23:
tACK bb822a7af8
meshcollider:
Code review ACK bb822a7af86897a9b6a5d616f193c258e8e76729
Tree-SHA512: f6dddc72a74e5667071ccd77f8dce578382e8e29e7ed6a0834ac2e114a6d3918b59c2f194f4079b3259e13d9ba3b4f405619940c3ecb7a1a0344615aed47c43d
607076d01bf23c69ac21950c17b01fb4e1130774 test: remove confusing `MAX_BLOCK_BASE_SIZE` (Sebastian Falbesoner)
4af97c74edcda56cd15523bf3a335adea2bad14a test: introduce `get_weight()` helper for CBlock (Sebastian Falbesoner)
a084ebe1330bcec15715e08b0f65319142927ad1 test: introduce `get_weight()` helper for CTransaction (Sebastian Falbesoner)
Pull request description:
This is a very late follow-up PR to #10618, which removed the constant `MAX_BLOCK_BASE_SIZE` from the core implementation about four years ago (see also #10608 in why it was considered confusing and superfluous).
Since there is also no point in still keeping it in the functional test framework, the PR switches to weight-based accounting on the relevant test code parts and use `MAX_BLOCK_WEIGHT` instead for the block limit checks. To prepare that, the first two commits introduce `get_weight()` helpers for the classes CTransaction and CBlock, respectively.
ACKs for top commit:
MarcoFalke:
review ACK 607076d01bf23c69ac21950c17b01fb4e1130774 🚴
Tree-SHA512: d59aa0b6b3dfd0a849b8063e66de275d252f705f99e25cd3bf6daec028b47d946777ee5b42a060f5283cb18e917ac073119c2c0e11bbc21211f69ef0a6ed335a
8bf1d06599 Merge bitcoin/bitcoin#29308: doc: update `BroadcastTransaction` comment (glozow)
2a77808596 Merge bitcoin-core/gui#789: Avoid non-self-contained Windows header (Hennadii Stepanov)
da371b830d Merge bitcoin/bitcoin#28870: depends: Include `config.guess` and `config.sub` into `meta_depends` (fanquake)
2e41562d81 Merge bitcoin/bitcoin#29219: fuzz: Improve fuzzing stability for ellswift_roundtrip harness (fanquake)
b091329599 Merge bitcoin/bitcoin#29211: fuzz: fix `connman` initialization (Ava Chow)
df42d41060 Merge bitcoin/bitcoin#29200: net: create I2P sessions using both ECIES-X25519 and ElGamal encryption (fanquake)
4cdd1a8a5d Merge bitcoin/bitcoin#29172: fuzz: set `nMaxOutboundLimit` in connman target (fanquake)
97012ea522 Merge bitcoin/bitcoin#28962: doc: Rework guix docs after 1.4 release (fanquake)
c70ff5d702 Merge bitcoin/bitcoin#28844: contrib: drop GCC MAX_VERSION to 4.3.0 in symbol-check (fanquake)
e6f19e7760 Merge bitcoin/bitcoin#29068: test: Actually fail when a python unit test fails (fanquake)
75e0334866 Merge bitcoin/bitcoin#28989: test: Fix test by checking the actual exception instance (Andrew Chow)
8cd85d311f Merge bitcoin/bitcoin#28852: script, assumeutxo: Enhance validations in utxo_snapshot.sh (Ryan Ofsky)
fd2e88d6f3 Merge bitcoin/bitcoin#26077: guix: switch from `guix environment` to `guix shell` (fanquake)
02741a7706 Merge bitcoin/bitcoin#28913: coins: make sure PoolAllocator uses the correct alignment (fanquake)
dfd53dabed Merge bitcoin/bitcoin#28902: doc: Simplify guix install doc, after 1.4 release (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 8bf1d06599
knst:
utACK 8bf1d06599
Tree-SHA512: 506273e5a188f9ca74edf656e3cd338992192e6e97f68c89fc43e34be20fb7f211b48e4dfa8693727839a7920da8284509413c722f55774a428939c296dad517
9a79217756 Merge bitcoin/bitcoin#28227: test: check for specific bip157 disconnect reasons, add test coverage (fanquake)
a7a4603b8e Merge bitcoin-core/gui#755: Silence `-Wcast-function-type` warning (Hennadii Stepanov)
e216d0851d Merge bitcoin/bitcoin#27934: test: added coverage to estimatefee (fanquake)
34f0f56582 Merge bitcoin/bitcoin#28506: fuzz: Add missing PROVIDE_FUZZ_MAIN_FUNCTION guard to __AFL_FUZZ_INIT (fanquake)
ca0225c0fd Merge bitcoin/bitcoin#28480: fuzz: Don't use afl++ deferred forkserver mode (fanquake)
2b236ad07b Merge bitcoin/bitcoin#28460: fuzz: Use afl++ shared-memory fuzzing (fanquake)
52f036b316 Merge bitcoin/bitcoin#28427: index: coinstats reorg, fail when block cannot be reversed (fanquake)
6ad6f2f28d Merge bitcoin/bitcoin#28412: test: remove unused variables in `p2p_invalid_block` (fanquake)
43b88315e1 Merge bitcoin/bitcoin#28426: doc: s/--no-substitute/--no-substitutes in guix/INSTALL (fanquake)
1730a267ba Merge bitcoin/bitcoin#28386: test: remove fixed timeouts from feature_config_args (fanquake)
7f83db0d0c Merge bitcoin/bitcoin#28332: test: previous releases: speed up fetching sources with shallow clone (fanquake)
8490bf4b03 Merge bitcoin/bitcoin#28288: test: fix 'unknown named parameter' test in `wallet_basic` (fanquake)
8b8ff1c7d5 Merge bitcoin/bitcoin#28215: fuzz: fix a couple incorrect assertions in the `coins_view` target (fanquake)
c36f7d93fa Merge bitcoin/bitcoin#27401: tracepoints: Disables `-Wgnu-zero-variadic-macro-arguments` to compile without warnings (fanquake)
163020ef92 Merge bitcoin/bitcoin#28203: refactor: serialization simplifications (fanquake)
24e57da770 Merge bitcoin/bitcoin#28181: qa, doc: Fix comment (fanquake)
933a63e8fc Merge bitcoin/bitcoin#28145: valgrind: add suppression for bug 472219 (fanquake)
33766805eb Merge bitcoin/bitcoin#28124: fuzz: Re-enable symbolize=1 in ASAN_OPTIONS (fanquake)
621061459a Merge bitcoin/bitcoin#28099: contrib: move user32.dll from bitcoind.exe libs (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
knst:
utACK 9a79217756
UdjinM6:
utACK 9a79217756
Tree-SHA512: 12d18abf28e3140bbb488fe912cf8e5c18a38aebaf09caad3120dc556116f348c4bbb40131f72c89edfbf48eb980c6947dde155e58e47c783f1d5b125aa6524b
c96f9e0285 feat: only require reindexing when the index was off going to off (pasta)
Pull request description:
## What was done?
It does not seem reasonable that we need to reindex when turning indexes off. this logic was introduced in e0781095f0
## How Has This Been Tested?
Hasn't
## Breaking Changes
None, basically
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
knst:
utACK c96f9e0285
Tree-SHA512: a5b634b9fbb6632b35286055a8e5b8db340738b7327a7d860e645ec1f6ef5b5cddc140a3e1c51b09c2f47db375c83e7c53fe5eaf221ab8df1d173caf50fa862d
fa1eb0ecaef14d428812f956082d29ab134fc728 test: Make the unlikely race in p2p_invalid_messages impossible (MarcoFalke)
Pull request description:
After `add_p2p_connection` both sides have the verack processed.
However the pong from conn in reply to the ping from the node has not
been processed and recorded in totalbytesrecv.
Flush the pong from conn by sending a ping from conn.
This should make the unlikely race impossible.
ACKs for top commit:
mzumsande:
ACK fa1eb0ecaef14d428812f956082d29ab134fc728
pinheadmz:
ACK fa1eb0ecaef14d428812f956082d29ab134fc728
Tree-SHA512: 44166587572e8c0c758cac460fcfd5cf403b2883880128b13dc62e7f74ca5cb8f145bb68a903df177ff0e62faa360f913fd409b009d4cd1360f1f4403ade39ae
faa671591f9c83ef0fb5afea151a1907c28f024b test: Use self.wait_until over wait_until_helper (MarcoFalke)
Pull request description:
`wait_until_helper` is a "private" helper, not intended to be used directly, because it doesn't scale the timeout with the timeout factor. Fix this by replacing it with a call to `self.wait_until`, which does the scaling.
ACKs for top commit:
theStack:
Code-review ACK faa671591f9c83ef0fb5afea151a1907c28f024b
Tree-SHA512: 70705f309f83ffd6ea5d090218195d05b868624d909106863372f861138b5a70887070b25beb25044ae1b44250345e45c9cc11191ae7aeca2ad37801a0f62f61
fae5bd920007c33de0b794bf2b2d698bfccc12ee test: Fix wallet_balance intermittent issue (MacroFake)
Pull request description:
Diff to reproduce:
```diff
index d2ed97ca76..25cc2d5734 100755
--- a/test/functional/wallet_balance.py
+++ b/test/functional/wallet_balance.py
@@ -265,7 +265,7 @@ class WalletTest(BitcoinTestFramework):
self.nodes[0].invalidateblock(block_reorg)
self.nodes[1].invalidateblock(block_reorg)
assert_equal(self.nodes[0].getbalance(minconf=0), 0) # wallet txs not in the mempool are untrusted
- self.generatetoaddress(self.nodes[0], 1, ADDRESS_WATCHONLY, sync_fun=self.no_op)
+ self.generatetoaddress(self.nodes[0], 1, ADDRESS_WATCHONLY)
assert_equal(self.nodes[0].getbalance(minconf=0), 0) # wallet txs not in the mempool are untrusted
# Now confirm tx_orig
```
Example in CI:
```
test 2022-08-24T10:09:22.486000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 133, in main
self.run_test()
File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/wallet_balance.py", line 269, in run_test
assert_equal(self.nodes[0].getbalance(minconf=0), 0) # wallet txs not in the mempool are untrusted
File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/util.py", line 56, in assert_equal
raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(98.85983340 == 0)
```
https://cirrus-ci.com/task/4981266251513856?logs=ci#L3269
ACKs for top commit:
achow101:
ACK fae5bd920007c33de0b794bf2b2d698bfccc12ee
w0xlt:
ACK fae5bd9200
Tree-SHA512: 470f366720615c4a9326ec4c581fff569ecce9877f9134bb1975ec3d6f1d13a6403051418a91a80b2a86de617f43e539ec11bbf4f1713d0354d5b0ab98d22437
2ab7952bda8d15e91b03f8307839030cbb55614e test: add bip157 coverage for (start height > stop height) disconnect (Sebastian Falbesoner)
63e90e1d3f5ed08f9871f07667d389ec66aa621c test: check for specific disconnect reasons in p2p_blockfilters.py (Sebastian Falbesoner)
Pull request description:
This PR checks for specific disconnect reasons using `assert_debug_log` in the functional test `p2p_blockfilters.py`. With that we ensure that the disconnect happens for the expected reason and also makes it easier to navigate between implementation and test code, i.e. both the questions "do we have test coverage for this disconnect cause?" (from an implementation reader's perspective) and "where is the code handling this disconnect cause?" (from a test reader's perspective) can be answered simply by grep-ping the corresponding debug message.
Also, based on that, missing coverage for the (start height > stop height) disconnect case is added:
b7138252ac/src/net_processing.cpp (L3050-L3056)
ACKs for top commit:
MarcoFalke:
lgtm ACK 2ab7952bda8d15e91b03f8307839030cbb55614e
furszy:
Looks good, code ACK 2ab7952b
Tree-SHA512: 0581cb569d5935aaa004a95a6f16eeafe628b9d816ebb89232f2832e377049df878a1e74c369fb46931b94e1a3a5e3f4aaa21a007c0a488f4ad2cda0919c605d
