Commit Graph

286 Commits

Author SHA1 Message Date
pasta
d7413ffbf7
Merge #6047: backport: trivial 2024 06 05
76279c1a37 Merge bitcoin/bitcoin#25149: refactor: Add thread safety annotation to `BanMan::SweepBanned()` (MacroFake)
6269c6f1db Merge bitcoin/bitcoin#25053: Guard `#include <config/bitcoin-config.h>` (fanquake)
d50f0b016f Merge bitcoin/bitcoin#24977: rpc: Explain active and internal in listdescriptors (fanquake)
3c44399d55 Merge bitcoin/bitcoin#24856: lint: Converting lint-assertions.sh to lint-assertions.py (laanwj)
e9f5b4b735 Merge bitcoin/bitcoin#24213: refactor: use Span in random.* (laanwj)
7e0474ac1c Merge bitcoin/bitcoin#24632: add `(none)` in -getinfo `Warnings:` if no warning returned (laanwj)
57e9b56bad Merge bitcoin/bitcoin#24145: mempool: Clear vTxHashes when mapTx is cleared (laanwj)
fe56d9b994 Merge bitcoin/bitcoin#24698: test: -peerblockfilters without -blockfilterindex raises an error (MarcoFalke)
3cabce645e Merge bitcoin/bitcoin#24472: fuzz: execute each file in dir without fuzz engine (MarcoFalke)
f5116a7d31 Merge bitcoin-core/gui#549: refactor: use std::chrono for formatDurationStr() helper (Hennadii Stepanov)
3fa8158510 Merge bitcoin/bitcoin#22317: doc: Highlight DNS requests part in tor.md (Andrew Chow)
72b62edd5a Merge bitcoin/bitcoin#23834: wallettool: Check that the dumpfile checksum is the correct size (laanwj)
ee9b3cdb0e Merge bitcoin/bitcoin#23979: test: wait for rather than assert presence of file in startupnotify test (MarcoFalke)
2ec5940399 Merge bitcoin/bitcoin#23532: test: add functional test for -startupnotify (MarcoFalke)
5a31be9608 Merge bitcoin/bitcoin#23812: test: fix intermittent failures in p2p_timeouts.py (MarcoFalke)
10828f5b3d Merge bitcoin/bitcoin#23733: fuzz: Move ISO8601 to one place (MarcoFalke)
7f39b5af41 Merge bitcoin/bitcoin#23635: test: Bump shellcheck version to 0.8.0 (fanquake)

Pull request description:

  ## Issue being fixed or feature implemented
  Trivial batch of backports

  ## What was done?
  trivial backports

  ## How Has This Been Tested?
  Unit tests ran; waiting on CI

  ## Breaking Changes

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

Top commit has no ACKs.

Tree-SHA512: a3f97003e6441468951b827b2c3ea607740e5b9d36b96c2f93e45e7fb4088ecf4d0a2b7038de050ca0e7d61379c364969f4a8caff98ec1cc69016f4114e64c6a
2024-06-07 09:33:44 -05:00
MarcoFalke
2ec5940399
Merge bitcoin/bitcoin#23532: test: add functional test for -startupnotify
126853214a490ee840e83ca17c717c40cfbe6837 test: add functional test for -startupnotify (Bruno Garcia)

Pull request description:

  This PR adds a functional test for -startupnotify. It basically starts the node passing a command on -startupnotify to create a file on tmp and then, we check if the file has been successfully created.

ACKs for top commit:
  theStack:
    Tested ACK 126853214a490ee840e83ca17c717c40cfbe6837
  kristapsk:
    re-ACK 126853214a490ee840e83ca17c717c40cfbe6837

Tree-SHA512: 5bf3e46124ee5c9d609c9993e6465d5a71a8d2275dcf07c8ce0549f013f7f8863d483b46b7164152f566468a689371ccb87f01cf118c3c9cac5b2be673b61a5c
2024-06-06 22:57:57 -05:00
Kittywhiskers Van Gogh
f619f8ff80
merge bitcoin#20234: don't bind on 0.0.0.0 if binds are restricted to Tor 2024-06-04 13:32:27 +00:00
pasta
7596a7320a
Merge #6033: backport: bitcoin#18202, #19202, #19501, #19725, #19770, #19877, #20043, partial #18878
34c80473a8 Merge #19877: [test] clarify rpc_net & p2p_disconnect_ban functional tests (Wladimir J. van der Laan)
e42412924f Merge #19770: RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions") (MarcoFalke)
f96966b7ea Merge #20043: doc: Add 19501 release notes (fanquake)
6a164eaea9 Merge #19501: send* RPCs in the wallet returns the "fee reason" (MarcoFalke)
b6c8d852e3 Merge #19725: [RPC] Add connection type to getpeerinfo, improve logs (MarcoFalke)
f86263b180 Merge #18202: refactor: consolidate sendmany and sendtoaddress code (Samuel Dobson)
fab41fd3c5 partial Merge #18878: test: Add test for conflicted wallet tx notifications (Wladimir J. van der Laan)
db5bd34ee8 Merge #19202: log: remove deprecated `db` log category (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Regular backports from bitcoin v21

  ## What was done?
   - bitcoin/bitcoin#19202
   - partial bitcoin/bitcoin#18878
   - bitcoin/bitcoin#18202
   - bitcoin/bitcoin#19725
   - bitcoin/bitcoin#19501
   - bitcoin/bitcoin#20043
   - bitcoin/bitcoin#19770
   - bitcoin/bitcoin#19877

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  - (RPC) The `getpeerinfo` RPC no longer returns the `addnode` field by default. This
    field will be fully removed in the next major release.  It can be accessed
    with the configuration option `-deprecatedrpc=getpeerinfo_addnode`. However,
    it is recommended to instead use the `connection_type` field (it will return
    `manual` when addnode is true)
  - (Settings) The `sendtoaddress` and `sendmany` RPCs accept an optional `verbose=True`
    argument to also return the fee reason about the sent tx.
  - (Settings) The `-debug=db` logging category, which was deprecated in v0.18 and replaced by
    `-debug=walletdb` to distinguish it from `coindb`, has been removed.
  - (RPC)  To make RPC `sendtoaddress` more consistent with `sendmany` the following error
      `sendtoaddress` codes were changed from `-4` to `-6`:
    - Insufficient funds
    - Fee estimation failed
    - Transaction has too long of a mempool chain

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 34c80473a8

Tree-SHA512: 725a103e04c9c7d44a79da6f3f54e7745c7fb98ec906e7228ae16f7662d568e48c015c855902ff8485f2908f0f71815e769ca394cf6c3ca2e5fd920dd39cca74
2024-05-29 12:04:08 -05:00
Kittywhiskers Van Gogh
9bf3829558
merge bitcoin#25355: add support for transient addresses for outbound connections 2024-05-29 11:48:37 -05:00
MarcoFalke
b6c8d852e3
Merge #19725: [RPC] Add connection type to getpeerinfo, improve logs
a512925e19a70d7f6b80ac530a169f45ffaafa1c [doc] Release notes (Amiti Uttarwar)
50f94b34a33c954f6e207f509c93d33267a5c3e2 [rpc] Deprecate getpeerinfo addnode field (Amiti Uttarwar)
df091b9b509f0b10e4315c0bfa2da0cc0c31c22f [refactor] Rename test file to allow any getpeerinfo deprecations. (Amiti Uttarwar)
395acfa83a5436790c1a722a5609ac9d48df235f [rpc] Add connection type to getpeerinfo RPC, update tests (Amiti Uttarwar)
49c10a9ca40967d28ae16dfea9cccc6f3a6624a1 [log] Add connection type to log statement (Amiti Uttarwar)

Pull request description:

  After #19316, we can more directly expose information about the connection type on the `getpeerinfo` RPC. Doing so also makes the existing addnode field redundant, so this PR begins the process of deprecating this field.

  This PR also includes one commit that improves a log message, as both use a shared function to return the connection type as a string.

  Suggested by sdaftuar- https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468001604 & https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468018093

ACKs for top commit:
  jnewbery:
    Code review ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c.
  sipa:
    utACK a512925e19a70d7f6b80ac530a169f45ffaafa1c
  guggero:
    Tested and code review ACK a512925e.
  MarcoFalke:
    cr ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c 🌇
  promag:
    Code review ACK a512925e19a70d7f6b80ac530a169f45ffaafa1c.

Tree-SHA512: 601a7a38aee235ee59aca690784f886dc2ae4e418b2e6422c4b58cd597376c00f74910f66920b08a08a0bec28bf8022e71a1435785ff6ba8a188954261aba78e
2024-05-29 14:03:55 +07:00
fanquake
6d44f36afd
Merge bitcoin/bitcoin#22096: p2p: AddrFetch - don't disconnect on self-announcements
5730a43703f7e5a5ca26245ba3b55fbdd027d0b6 test: Add functional test for AddrFetch connections (Martin Zumsande)
c34ad3309f93979b274a37de013502b05d25fad8 net, rpc: Enable AddrFetch connections for functional testing (Martin Zumsande)
533500d9072b7d5a36a6491784bdeb9247e91fb0 p2p: Add timeout for AddrFetch peers (Martin Zumsande)
b6c5d1e450dde6a54bd785504c923adfb45c7060 p2p: AddrFetch - don't disconnect on self-announcements (Martin Zumsande)

Pull request description:

  AddrFetch connections (old name: oneshots) are intended to be short-lived connections on which we ask a peer for addresses via `getaddr` and disconnect after receiving them.

  This is done by disconnecting after receiving the first `addr`. However, it is no longer working as intended, because nowadays, the first `addr` a typical bitcoin core node sends is its self-announcement.
  So we'll disconnect before the peer gets a chance to answer our `getaddr`.

  I checked that this affects both `-seednode` peers specified manually, and DNS seeds when AddrFetch is used as a fallback if DNS doesn't work for us.

  The current behavior of getting peers via AddrFetch when starting with an empty addrman would be to connect to the peer, receive its self-announcement and add it to addrman, disconnect, reconnect to the same peer again as a full outbound (no other addresses in addrman) and then receive more `addr`. This is silly and not in line with AddrFetch peer being intended to be short-lived peers. 

  Fix this by only disconnecting after receiving an `addr` message of size > 1.

  [Edit] As per review discussion, this PR now also adds a timeout after which we disconnect if we haven't received any suitable `addr`, and a functional test.

ACKs for top commit:
  amitiuttarwar:
    reACK 5730a43703f7e5a5ca26245ba3b55fbdd027d0b6
  naumenkogs:
    ACK 5730a43703f7e5a5ca26245ba3b55fbdd027d0b6
  jnewbery:
    ACK 5730a43703

Tree-SHA512: 8a81234f37e827705138eb254223f7f3b3bf44a06cb02126fc7990b0d231b9bd8f07d38d185cc30d55bf35548a6fdc286b69602498d875b937e7c58332158bf9
2024-05-27 12:18:23 -05:00
MarcoFalke
61f9d96f38
Merge bitcoin/bitcoin#22423: test: wallet_listtransactions improvements (speedup, cleanup, logging)
a006d7d73019b8cf4d68626c019c3d69729dda69 test: add logging to wallet_listtransactions (Sebastian Falbesoner)
47915b118720c6e2b2ec9f599f25848041b42b99 test: remove unneeded/redundant code in wallet_listtransactions (Sebastian Falbesoner)
fb6c6a7938cb7c4808ad88d23bfc2b7408407b12 test: speedup wallet_listtransactions by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)

Pull request description:

  This PR improves the test `wallet_listtransactions.py` in three ways:
  * speeds up runtime by a factor of 2-3x by using the good ol' immediate tx relay trick (`-whitelist=noban@127.0.0.1`)
  * removes unneeded/redundant code
  * adds log messages, mostly by turning comments into `self.log.info(...)` calls

ACKs for top commit:
  jonatack:
    ACK a006d7d73019b8cf4d68626c019c3d69729dda69
  kristapsk:
    ACK a006d7d73019b8cf4d68626c019c3d69729dda69

Tree-SHA512: a91a19f5ebc4d05f0b96c5419683c4c57ac0ef44b64eeb8dd550bd72296fd3a2857a3ba83f755fe4b0b3bd06439973f226070b5d0ce2dee58344dae78cb50290
2024-05-18 17:54:16 -05:00
Konstantin Akimov
b2ede8bfee
feat: update list of tests that still doesn't support descriptor wallets
That are:
 - feature_dip3_deterministicmns.py
 - interface_zmq_dash.py
 - feature_governance.py
 - wallet_upgradetohd.py (as expected to be implemented for legacy-only wallets)
 - p2p_timeouts.py (why? can not understand it)

This partially reverts commit b20f812674.
2024-05-07 00:17:18 +07:00
Konstantin Akimov
838d06f2fd
feat: enable descriptor wallets for more tests
Enables for rpc_quorum.py, feature_notifications.py

see #5981, it partial revert of b20f812674
2024-05-07 00:17:18 +07:00
pasta
544d33309a
Merge #5981: backport: bitcoin#19136, #21063, #21277, #21302, partial #20267 - descriptor wallets part IV
ceefab5226 fix: feature_backwards compatible works now with as expected if no bdb compiled (Konstantin Akimov)
b20f812674 fix: follow-up fixes for functional tests used protx (Konstantin Akimov)
655146d5e7 Merge #21302: wallet: createwallet examples for descriptor wallets (W. J. van der Laan)
99a8b60393 Merge #21063: wallet, rpc: update listdescriptors response format (fanquake)
6ee2c7cc59 Merge #21277: wallet: listdescriptors uses normalized descriptor form (Wladimir J. van der Laan)
8bacdbf71f Merge #19136: wallet: add parent_desc to getaddressinfo (Samuel Dobson)
f567de007a chore: release notes for 5965 with wallet tool improvements (Konstantin Akimov)
0daf360edf chore: add TODO to implement mnemonic for descriptor wallets (Konstantin Akimov)
5016294307 chore: move functional test wallet_multiwallet from category "slow 5 minutes" to "fast test" (Konstantin Akimov)
ef7ce87c1b fix: remove workarounds introduced due to missing bitcoin#20267 (bdb is not compiled) (Konstantin Akimov)
06b2d85bb4 partial Merge #20267: Disable and fix tests for when BDB is not compiled (Wladimir J. van der Laan)

Pull request description:

  ## Issue being fixed or feature implemented
  https://github.com/dashpay/dash-issues/issues/59

  ## Extra notes
  This commit `chore: move functional test wallet_multiwallet from category "slow 5 minutes" to "fast test"` is not directly connected to descriptor wallets, but added to this PR due to conflicts with 20267

  ## What was done?
  It steadily improves support of descriptor wallets in Dash core.

  Done backports and related fixes:
   - partial bitcoin/bitcoin#20267
   - bitcoin/bitcoin#19136
   - bitcoin/bitcoin#21277
   - bitcoin/bitcoin#21063
   - bitcoin/bitcoin#21302

  Beside backports and related fixes, this PR includes release notes for previous batch of backports for descriptor wallets support #5965

  ## How Has This Been Tested?
  Run unit functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

Top commit has no ACKs.

Tree-SHA512: f4b2033f8c4fa1d0f72cfc31378909703b3ae8f44748989ff00c3e71311ac80ac37837137133c7e4a166823a941ed7df10efa09c89f5b213f3c8ede7d3d6e8f4
2024-04-16 08:56:59 -05:00
Konstantin Akimov
b20f812674
fix: follow-up fixes for functional tests used protx
Since bitcoin#20267 changes default wallet in functional tests from legacy
wallets to descriptor wallets, we need to enforce --legacy-wallets for
functional tests that used protx which doesn't work yet for descriptor wallets
2024-04-12 17:31:57 +07:00
Konstantin Akimov
0daf360edf
chore: add TODO to implement mnemonic for descriptor wallets 2024-04-11 02:37:04 +07:00
Konstantin Akimov
5016294307
chore: move functional test wallet_multiwallet from category "slow 5 minutes" to "fast test" 2024-04-11 02:37:03 +07:00
Wladimir J. van der Laan
06b2d85bb4
partial Merge #20267: Disable and fix tests for when BDB is not compiled
Backport notice: changes in feature_notification.py are missing due to #18878 is not done yet

49797c3ccfbb9f7ac9c1fbb574d35b315c103805 tests: Disable bdb dump test when no bdb (Andrew Chow)
1194cf9269e6c4bf67b09c4766f42bf173d12c0a Fix wallet_send.py wallet setup to work with descriptors (Andrew Chow)
fbaea7bfe44822710a36601c6b0febbd5e33dfbd Require legacy wallet for wallet_upgradewallet.py (Andrew Chow)
b1b679e0ab7a9981e3e78424fe8836edd59abf6f Explicitly mark legacy wallet tests as such (Andrew Chow)
09514e1bef46444a67cde9ff13e76bd4b9f8c7ac Setup wallets for interface_zmq.py (Andrew Chow)
4d03ef9a73ceb5ef4e9d184a135bca6bdeb8c311 Use MiniWallet in rpc_net.py (Andrew Chow)
4de23824b0c711ece68f9fc007ffac12126710aa Setup wallets for interface_bitcoin_cli.py (Andrew Chow)
7c71c627d28f0cddaf2349a55336278a681c27c2 Setup wallets with descriptors for feature_notifications (Andrew Chow)
1f1bef8dbab7225884d769a45477ee11d0ebf654 Have feature_filelock.py test both bdb and sqlite, depending on compiled (Andrew Chow)
c77975abc0123b29b0eb3481b8916e7c025b7c4c Disable upgrades tests that require BDB if BDB is not compiled (Andrew Chow)
1f20cac9d41e507901a2811d6db7147d7ab0321b Disable wallet_descriptor.py bdb format check if BDB is not compiled (Andrew Chow)
3641597d7ef6f5097a9e93cab3ef7e0f9c820296 tests: Don't make any wallets unless wallet is required (Andrew Chow)
b9b88f57a9b9a28e0f0614c12ae3012cf5050b10 Skip legacy wallet reliant tests if BDB is not compiled (Andrew Chow)
6f36242389bd3e7eacf594ce90491e8ccca70f3a tests: Set descriptors default based on compilation (Andrew Chow)

Pull request description:

  This PR fixes tests for when BDB is not compiled. Tests which rely on or test legacy wallet behavior are disabled and skipped when BDB is not compiled. For the components of some tests that are for legacy wallet things, those parts of the tests are skipped.

  For the majority of tests, changes are made so that they can be run with either legacy wallets or descriptor wallets without materially effecting the test. Most tests only need the wallet for balance and transactions, so the type of wallet is not an important part of those tests. Additionally, some tests are wallet agnostic and modified to instead use the test framework's MiniWallet.

ACKs for top commit:
  laanwj:
    ACK 49797c3ccfbb9f7ac9c1fbb574d35b315c103805
  ryanofsky:
    Code review ACK 49797c3ccfbb9f7ac9c1fbb574d35b315c103805. Only change since last review is dropping last commit. Previous review w/ suggestions for future followup is https://github.com/bitcoin/bitcoin/pull/20267#pullrequestreview-581508843

Tree-SHA512: 69659f8a81fb437ecbca962f4082c12835282dbf1fba7d9952f727a49e01981d749af9b09feda1c8ca737516c7d7a08ef17e782795df3fa69892d5021b41c1ed
2024-04-11 02:37:03 +07:00
MarcoFalke
e10eec249b
Merge #21338: test: add functional test for anchors.dat
581791c62067403fbeb4e1fd88c1d80549627c94 test: add functional test for anchors.dat (bruno)

Pull request description:

  This PR adds a functional test for anchors.dat.

  It creates a node and adds 2 outbound block-relay-only connections and 5 inbound connections.
  When the node is down, anchors.dat should contain the 2 addresses from the outbound block-relay-only connections.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 581791c62067403fbeb4e1fd88c1d80549627c94
  hebasto:
    ACK 581791c62067403fbeb4e1fd88c1d80549627c94

Tree-SHA512: 77038b09e36ee5ae473a26d6f566c0ed283af258c34df8486706a24f72b05abab621a293ac886d03849bc45bc28be7336137252225b25aff393baa6b5238688c
2024-04-11 02:26:01 +07:00
Kittywhiskers Van Gogh
1d4f10a378
merge bitcoin#19315: Allow outbound & block-relay-only connections in functional tests 2024-04-03 16:06:40 +00:00
MarcoFalke
eba325d7a2
Merge #16551: test: Test that low difficulty chain fork is rejected
Backport notice:
 - data have real blocks from testnet
 - due to big gap in blocks before checkpoints (half-year) for sack of this test is added one more checkpoint, otherwise blocks are not accepted due to non-mockable time for other chains except regtest
 - data for 2 blocks in split chain are generated locally for testnet
--------------
333317ce6b67aa92f7363d48cd750712190b4b6b test: Test that low difficulty chain fork is rejected (MarcoFalke)
fa31dc1bf4ee471c4641eef8de02702ba0619ae7 test: Pass down correct chain name in tests (MarcoFalke)

Pull request description:

  To prevent OOM, Bitcoin Core will reject chain forks at low difficulty by default. This is the only use-case of checkpoints, so add a test for it to make sure the feature works as expected. If it didn't work, checkpoints would have no use-case and we might as well remove them

ACKs for top commit:
  Sjors:
    Thanks for adding the node 1 example. Code review ACK 333317c

Tree-SHA512: 90dffa540d0904f3cffb61d2382b1a26f84fe9560b7013e4461546383add31a8757b350616a6d43217c59ef7b8b2a1b62bb3bab582c679cbb2c660a782ce7be1
2024-04-03 14:16:43 +07:00
fanquake
c0c5d05419
Merge #19469: rpc: deprecate banscore field in getpeerinfo
41d55d30579358c805036201664ad6a1c1d48681 doc: getpeerinfo banscore deprecation release note (Jon Atack)
dd54e3796e633cfdf6954af306afd26eadc25116 test: getpeerinfo banscore deprecation test (Jon Atack)
8c7647b3fbbab03ea84071cf3cd2d0d2bf8be255 rpc: deprecate banscore field in rpc getpeerinfo (Jon Atack)

Pull request description:

  Per https://github.com/bitcoin/bitcoin/pull/19219#discussion_r443074487 and https://github.com/bitcoin/bitcoin/pull/19219#issuecomment-652699592, this PR deprecates returning the `banscore` field in the `getpeerinfo` RPC, updates the help, adds a test, and updates the release notes. Related to #19464.

ACKs for top commit:
  fanquake:
    ACK 41d55d30579358c805036201664ad6a1c1d48681

Tree-SHA512: 8eca08332581e2fe191a2aafff6ba89ce39413f0491ed0de8b86577739f0ec430b1a8fbff2914b0f3138a229563dfcc1981c0cf5b7dd6061b5c48680a28423bc
2024-03-22 11:08:10 -05:00
pasta
c90fd21378
Merge #5930: backport: bitcoin#14582, #16404, #17445, #19507, #19538, #19743, #19756, #19773, #19830, partial bitcoin#27053
5a6b8b6b1f partial Merge bitcoin/bitcoin#27053: wallet: reuse change dest when re-creating TX with avoidpartialspends (fanquake)
2f788aa76d fix: change port to use for zmq in interface_zmq_dash.py (Konstantin Akimov)
0ce66fd477 Merge #19507: Expand functional zmq transaction tests (Wladimir J. van der Laan)
a1c2386153 Merge #17445: zmq: Fix due to invalid argument and multiple notifiers (Wladimir J. van der Laan)
44929bad82 Merge #16404: qa: Test ZMQ notification after chain reorg (MarcoFalke)
1707f01309 fix: follow-up changes from bitcoin/bitcoin#22220 for maxapsfee (Konstantin Akimov)
eb4270deae Merge #19743: -maxapsfee follow-up (Samuel Dobson)
6a6d379711 Merge #19756: tests: add sync_all to fix race condition in wallet groups test (MarcoFalke)
5821a1d23a Merge #14582: wallet: always do avoid partial spends if fees are within a specified range (Samuel Dobson)
59d5a4ef39 Merge #19773: wallet: Avoid recursive lock in IsTrusted (Samuel Dobson)
2489f29f0e Merge #19830: test: Add tsan supp for leveldb::DBImpl::DeleteObsoleteFiles (fanquake)
10fa7a66b6 Merge #19538: ci: Add tsan suppression for race in DatabaseBatch (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Regular backports from bitcoin v21

  ## What was done?
   - bitcoin/bitcoin#19538
   - bitcoin/bitcoin#19830
   - bitcoin/bitcoin#19773
   - bitcoin/bitcoin#14582
   - bitcoin/bitcoin#19756
   - bitcoin/bitcoin#19743
   - bitcoin/bitcoin#16404
   - bitcoin/bitcoin#17445
   - bitcoin/bitcoin#19507
   - partial bitcoin/bitcoin#27053

  +some extra fixes and missing changes from bitcoin/bitcoin#22220 for `maxapsfee`

  +changed port for zmq in `interface_zmq_dash.py` to prevent intermittent error in `interface_zmq.py`

  ## How Has This Been Tested?
  Run unit & functional tests

  ## Breaking Changes
  `CreateTransaction` now uses sometime 2 private keys for one transaction instead one

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 5a6b8b6b1f

Tree-SHA512: 7efd8a31808f155c08035d0fb7ceaac369e3e44e68d2c91a88e52815a60efba5fe9458f41d93352627c2c062d414fb0207dcf216fa75b54af210b503f9123de6
2024-03-19 10:12:39 -05:00
fanquake
5a6b8b6b1f
partial Merge bitcoin/bitcoin#27053: wallet: reuse change dest when re-creating TX with avoidpartialspends
14b4921a91920df25b19ff420bfe2bff8c56f71e wallet: reuse change dest when recreating TX with avoidpartialspends (Matthew Zipkin)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/27051

  When the wallet creates a transaction internally, it will also create an alternative that spends using destination groups and see if the fee difference is negligible. If it costs the user the same to send the grouped version, we send it (even if the user has `avoidpartialspends` set to `false` which is default). This patch ensures that the second transaction creation attempt re-uses the change destination selected by the first attempt. Otherwise, the first change address remains reserved, will not be used in the second attempt, and then will never be used by the wallet, leaving gaps in the BIP44 chain.

  If the user had `avoidpartialspends` set to true, there is no second version of the created transaction and the change addresses are not affected.

  I believe this behavior was introduced in https://github.com/bitcoin/bitcoin/pull/14582

ACKs for top commit:
  achow101:
    ACK 14b4921a91920df25b19ff420bfe2bff8c56f71e

Tree-SHA512: a3d56f251ff4b333fc11325f30d05513e34ab0a2eb703fadd0ad98d167ae074493df1a24068298336c6ed2da6b31aa2befa490bc790bbc260ed357c8f2397659
2024-03-18 16:30:45 +07:00
Wladimir J. van der Laan
860d31f504
Merge #19455: rpc generate: print useful help and error message
f0aa8aeea5a183ea44a877255d12db7732f2e0a8 test: add rpc_generate functional test (Jon Atack)
92d94ffb8d07cc0d2665c901de5903a3a90d5fd0 rpc: print useful help and error message for generate (Jon Atack)
8d32d2011d3f4e1d9e587d6f80dfa4a3e9f9393d test: consider generate covered in _get_uncovered_rpc_commands() (Jon Atack)

Pull request description:

  This was a requested follow-up to #19133 and #17700 to alleviate confusion and head-scratching by people following tutorials that use `generate`. See https://github.com/bitcoin/bitcoin/pull/19455#issuecomment-668172916 below, https://github.com/bitcoin/bitcoin/pull/19133#issuecomment-636860943 and https://github.com/bitcoin/bitcoin/pull/17700#issuecomment-566159096.

  before
  ```
  $ bitcoin-cli help generate
  help: unknown command: generate

  $ bitcoin-cli generate
  error code: -32601
  error message:
  Method not found
  ```

  after
  ```
  $ bitcoin-cli help generate
  generate ( nblocks maxtries ) has been replaced by the -generate cli option. Refer to -help for more information.

  $ bitcoin-cli generate
  error code: -32601
  error message:
  generate ( nblocks maxtries ) has been replaced by the -generate cli option. Refer to -help for more information.
  ```

  In the general help it remains hidden, as requested by laanwj.
  ```
  $ bitcoin-cli help

  == Generating ==
  generateblock "output" ["rawtx/txid",...]
  generatetoaddress nblocks "address" ( maxtries )
  generatetodescriptor num_blocks "descriptor" ( maxtries )
  ```

ACKs for top commit:
  adamjonas:
    utACK f0aa8aeea5a183ea44a877255d12db7732f2e0a8
  pinheadmz:
    ACK f0aa8aeea5a183ea44a877255d12db7732f2e0a8

Tree-SHA512: d083652589ad3e8228c733455245001db22397559c3946e7e573cf9bd01c46e9e88b72d934728ec7f4361436ae4c74adb8f579670b09f479011924357e729af5
2024-03-17 13:02:57 -05:00
pasta
c23514dc49
Merge #5823: backport: bitcoin#19200, #19405, #20282
9c54cb16de Merge #19405: rpc, cli: add network in/out connections to `getnetworkinfo` and `-getinfo` (Wladimir J. van der Laan)
19aba38cab Merge #19200: rpc: remove deprecated getaddressinfo fields (Samuel Dobson)
f5642281cc Merge #20282: wallet: change upgradewallet return type to be an object (Samuel Dobson)

Pull request description:

  ## Issue being fixed or feature implemented
  Bitcoin backports with breaking changes

  ## What was done?
   - bitcoin/bitcoin#20282
   - bitcoin/bitcoin#19200
   - bitcoin/bitcoin#19405

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  ### RPC:
  - `upgradewallet` now returns object for future extensibility (#20282)

  - `getnetworkinfo` now returns fields `connections_in`, `connections_out`,
    `connections_mn_in`, `connections_mn_out`, `connections_mn`
    that provide the number of inbound and outbound peer
    connections. These new fields are in addition to the existing `connections`
    field, which returns the total number of peer connections. Old fields
    `inboundconnections`, `outboundconnections`, `inboundmnconnections`,
    `outboundmnconnections` and `mnconnections` are removed (#19405)

  - Backwards compatibility has been dropped for two `getaddressinfo` RPC
    deprecations, as notified in the 19.1.0 and 19.2.0 release notes.
    The deprecated `label` field has been removed as well as the deprecated `labels` behavior of
    returning a JSON object containing `name` and `purpose` key-value pairs. Since
    20.1, the `labels` field returns a JSON array of label names. (#19200)

  ### CLI

  - The `connections` field of `bitcoin-cli -getinfo` is expanded to return a JSON
    object with `in`, `out` and `total` numbers of peer connections and `mn_in`,
    `mn_out` and `mn_total` numbers of verified mn connections. It previously
    returned a single integer value for the total number of peer connections. (#19405)

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 9c54cb16de

Tree-SHA512: 9710cdd062d02d64e2eebcecca1b5c2e6ccda5ca6d9bd6d1833700f4273fcfb206ae99134f71bbc8b0843cb8ebba208c72139f5a624d79ec7362bd73b117bfb2
2024-03-17 12:58:46 -05:00
Samuel Dobson
19aba38cab
Merge #19200: rpc: remove deprecated getaddressinfo fields
BACKPORT NOTICE:
These backports #17578 and #17585 are included to 19.1 and 19.2. That's long enough!
------------------------------------------
bc01f7ae0538d3c647ce8dfbc29f7914d5df3fbb doc: release note for rpc getaddressinfo removals (Jon Atack)
90e989390ee50633fff0e4f210a1ea23ff00e012 rpc: getaddressinfo RPCResult fixup (Jon Atack)
a8507c99da10791aa69ca277128e06753942e976 rpc: remove deprecated getaddressinfo `labels: purpose` (Jon Atack)
645a8653c895e4fc7717e9e5ac045612b5deaa60 rpc: remove deprecated getaddressinfo `label` field (Jon Atack)

Pull request description:

  These were deprecated in #17578 and #17585, with expected 0.21 removal notified in the 0.20 release notes.
  ```
  - The `getaddressinfo` RPC has had its `label` field deprecated
    (re-enable for this release using the configuration parameter
    `-deprecatedrpc=label`).  The `labels` field is altered from returning
    JSON objects to returning a JSON array of label names (re-enable
    previous behavior for this release using the configuration parameter
    `-deprecatedrpc=labelspurpose`).  Backwards compatibility using the
    deprecated configuration parameters is expected to be dropped in the
    0.21 release.  (#17585, #17578)
  ```

ACKs for top commit:
  Sjors:
    utACK bc01f7a
  adamjonas:
    utACK bc01f7a
  meshcollider:
    utACK bc01f7ae0538d3c647ce8dfbc29f7914d5df3fbb

Tree-SHA512: ae1af381e32c4c3bde8b061a56382838513a9a82c88767843cdeae3a2ab8aa7d8c2e66e106d2b31ea07d74bb80c191a2f842c9aaecc7c5438ad9a9bc66d1b251
2024-03-16 02:39:42 +07:00
Wladimir J. van der Laan
953b6706f4
Merge #18788: tests: Update more tests to work with descriptor wallets
c7b7e0a69265946aecc885be911c7650911ba2e3 tests: Make only desc wallets for wallet_multwallet.py --descriptors (Andrew Chow)
d4b67ad214ada7645c4ce2d5ec336fe5c3f7f7ca Avoid creating legacy wallets in wallet_importdescriptors.py (Andrew Chow)
6c9c12bf87f95066acc28ea2270a00196eb77703 Update feature_backwards_compatibility for descriptor wallets (Andrew Chow)
9a4c631e1c00eb1661c000978b133d7aa0226290 Update wallet_labels.py to not require descriptors=False (Andrew Chow)
242aed7cc1d003e8fed574bbebd19c7e54e23402 tests: Add a --legacy-wallet that is mutually exclusive with --descriptors (Andrew Chow)
388053e1722632c2e485c56a444bc75cf0152188 Disable some tests for tool_wallet when descriptors (Andrew Chow)
47d3243160fdec7e464cfb8f869be7f5d4ee25fe Make raw multisig tests legacy wallet only in rpc_rawtransaction.py (Andrew Chow)
59d3da5bce4ebd9c2291d8f201a53ee087938b21 Do addmultisigaddress tests in legacy wallet mode in wallet_address_types.py (Andrew Chow)
25bc5dccbfd52691adca6edd418dd54290300c28 Use importdescriptors when in descriptor wallet mode in wallet_createwallet.py (Andrew Chow)
0bd1860300b13b12a25d330ba3a93ff2d13aa379 Avoid dumpprivkey and watchonly behavior in rpc_signrawtransaction.py (Andrew Chow)
08067aebfd7e838e6ce6b030c31a69422260fc6f Add script equivalent of functions in address.py (Andrew Chow)
86968882a8a26312a7af29c572313c4aff488c11 Add descriptor wallet output to tool_wallet.py (Andrew Chow)
3457679870e8eff2a7d14fe59a479692738c48b6 Use separate watchonly wallet for multisig in feature_nulldummy.py (Andrew Chow)
a42652ec10c733a5bf37e418e45d4841f54331b4 Move import and watchonly tests to be legacy wallet only in wallet_balance.py (Andrew Chow)
4b871909d6e4a51888e062d322bf53263deda15e Use importdescriptors for descriptor wallets in wallet_bumpfee.py (Andrew Chow)
c2711e4230d9a423ead24f6609691fb338b1d26b Avoid dumpprivkey in wallet_listsinceblock.py (Andrew Chow)
553dbf9af4dea96e6a3e79bba9607003342029bd Make import tests in wallet_listtransactions.py legacy wallet only (Andrew Chow)
dc81418fd01021070f3f66bab5fee1484456691a Use a separate watchonly wallet in rpc_fundrawtransaction.py (Andrew Chow)
a357111047411f18c156cd34a002a38430f2901c Update wallet_importprunedfunds to avoid dumpprivkey (Andrew Chow)

Pull request description:

  I went through all the tests and checked whether they passed with descriptor wallets. This partially informed some changes in #16528. Some tests needed changes to work with descriptor wallets. These were primarily due to import and watchonly behavior. There are some tests and test cases that only test legacy wallet behavior so those tests won't be run with descriptor wallets.

  This PR updates more tests to have to the `--descriptors` switch in `test_runner.py`. Additionally a mutually exclusive `--legacy-wallet` option has been added to force legacy wallets. This does nothing currently but will be useful in the future when descriptor wallets are the default. For the tests that rely on legacy wallet behavior, this option is being set so that we don't forget in the future. Those tests are `feature_segwit.py`, `wallet_watchonly.py`, `wallet_implicitsegwit.py`, `wallet_import_with_label.py`, and `wallet_import_with_label.py`.

  If you invert the `--descriptors`/`--legacy-wallet` default so that descriptor wallets are the default, all tests (besides the legacy wallet specific ones) will pass.

ACKs for top commit:
  MarcoFalke:
    review ACK c7b7e0a69265946aecc885be911c7650911ba2e3 🎿
  laanwj:
    ACK c7b7e0a69265946aecc885be911c7650911ba2e3

Tree-SHA512: 2f4e87815005d1d0a2543ea7947f7cd7593d8cf5312228ef85f8e096f19739b225769961943049cb44f6f07a35b8de988e2246ab9aca5bb5a0b2e62694d5637d
2024-03-09 03:01:24 +07:00
Samuel Dobson
56d9fe9510
Merge #20226: wallet, rpc: add listdescriptors command
647b81b70938dc4dbcf32399c56f78be395c721a wallet, rpc: add listdescriptors command (Ivan Metlushko)

Pull request description:

  Looking for concept ACKs

  **Rationale**: allow users to inspect the contents of their newly created descriptor wallets.

  Currently the command only returns xpubs which is not very useful in itself, but there are multiples ways to extend it:
   * add an option to export xprv
   * with #19136 it'll be possible to return normalised descriptors suitable for a watch-only purposes

  The output is compatible with `importdescriptors` command so it could be easily used for backup/recover purposes.

  **Output example:**
  ```json
  [
    {
      "desc": "wpkh(tpubD6NzVbkrYhZ4WW6E2ZETFyNfq2hfF23SKxqSGFvUpPAY58jmmuBybwqwFihAyQPk9KnwTt5516NDZRJ7k5QPeKjy7wuVd5WvXNxwwAs5tUD/*)#nhavpr5h",
      "timestamp": 1296688602,
      "active": false,
      "range": [
        0,
        999
      ],
      "next": 0
    }
  ]
  ```

ACKs for top commit:
  jonatack:
    re-ACK 647b81b70938dc4dbcf32399c56f78be395c721a rebased to master, debug builds cleanly, reviewed diff since last review, tested with a descriptor wallet (and with a legacy wallet)
  achow101:
    re-ACK 647b81b

Tree-SHA512: 51a3620bb17c836c52cecb066d4fa9d5ff418af56809046eaee0528c4dc240a4e90fff5711ba96e399c6664e00b9ee8194e33852b1b9e75af18061296e19a8a7
2024-03-09 03:00:29 +07:00
MarcoFalke
a1e3885a68
Merge #19239: tests: move generate_wif_key to wallet_util.py
3a83a01694160f2e722e1bc90a328bd569b8e109 [tests] move generate_wif_key to wallet_util.py (John Newbery)
b216b0b71f7853d747af8b712fc250c699f3c320 [tests] sort imports in rpc_createmultisig.py (John Newbery)
e38081846d889fcbbbc6ca4a4d3bca26807fde2f Revert "[TESTS] Move base58 to own module to break circular dependency" (John Newbery)

Pull request description:

  generate_wif_key is a wallet utility function. Move it from the EC key module to the wallet util module.

  This fixes the circular dependency issue in #17977

ACKs for top commit:
  MarcoFalke:
    ACK 3a83a01694160f2e722e1bc90a328bd569b8e109 🍪

Tree-SHA512: 24985dffb75202721ccc0c6c5b52f1fa5d1ce7963bccde24389feb913cab4dad0c265274ca67892c46c8b64e6a065a0f23263a89be4fb9134dfefbdbe5c7238a
2024-03-09 03:00:27 +07:00
MarcoFalke
eaf31ad2be
Merge #19230: [TESTS] Move base58 to own module to break circular dependency
c75de5da5fdd3a304f9da3d8a2e0370d1723ddd0 [TESTS] Move base58 to own module to break circular dependency (Pieter Wuille)

Pull request description:

  I encountered difficulties with the test framework in #17977. This fixes them, and I think the change is generally useful.

ACKs for top commit:
  laanwj:
    Code review ACK c75de5da5fdd3a304f9da3d8a2e0370d1723ddd0
  MarcoFalke:
    ACK c75de5da5fdd3a304f9da3d8a2e0370d1723ddd0 according to --color-moved=dimmed-zebra this is a move-only apart from the imports 👒

Tree-SHA512: 9e0493de3e279074f0c70e92c959b73ae30479ad6f2083a3c6bbf4b0191d65ef94854559a5b7c904f5dadc5e93129ed00f6dc0a8ccce6ba7921cd45f7119f74b
2024-03-09 03:00:27 +07:00
Kittywhiskers Van Gogh
c0f6b55f76
merge bitcoin#16378: The ultimate send RPC 2024-03-07 09:29:09 +00:00
Samuel Dobson
c172605cd7
Merge #19077: wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets
c4a29d0a90b821c443c10891d9326c534d15cf97 Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky)
310b0fde04639b7446efd5c1d2701caa4b991b86 Run dumpwallet for legacy wallets only in  wallet_backup.py (Andrew Chow)
6c6639ac9f6e1677da066cf809f9e3fa4d2e7c32 Include sqlite3 in documentation (Andrew Chow)
f023b7cac0eb16d3c1bf40f1f7898b290de4cc73 wallet: Enforce sqlite serialized threading mode (Andrew Chow)
6173269866306058fcb1cc825b9eb681838678ca Set and check the sqlite user version (Andrew Chow)
9d3d2d263c331e3c77b8f0d01ecc9fea0407dd17 Use network magic as sqlite wallet application ID (Andrew Chow)
9af5de3798c49f86f27bb79396e075fb8c1b2381 Use SQLite for descriptor wallets (Andrew Chow)
9b78f3ce8ed1867c37f6b9fff98f74582d44b789 walletutil: Wallets can also be sqlite (Andrew Chow)
ac38a87225be0f1103ff9629d63980550d2f372b Determine wallet file type based on file magic (Andrew Chow)
6045f77003f167bee9a85e2d53f8fc6ff2e297d8 Implement SQLiteDatabase::MakeBatch (Andrew Chow)
727e6b2a4ee5abb7f2dcbc9f7778291908dc28ad Implement SQLiteDatabase::Verify (Andrew Chow)
b4df8fdb19fcded7e6d491ecf0b705cac0ec76a1 Implement SQLiteDatabase::Rewrite (Andrew Chow)
010e3659069e6f97dd7b24483f50ed71042b84b0 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow)
ac5c1617e7f4273daf24c24da1f6bc5ef5ab2d2b Implement SQLiteDatabase::Backup (Andrew Chow)
f6f9cd6a64842ef23777312f2465e826ca04b886 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow)
bf90e033f4fe86cfb90492c7e0962278ea3a146d Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow)
7aa45620e2f2178145a2eca58ccbab3cecff08fb Add SetupSQLStatements (Andrew Chow)
6636a2608a4e5906ee8092d5731595542261e0ad Implement SQLiteBatch::Close (Andrew Chow)
93825352a36456283bf87e39b5888363ee242f21 Implement SQLiteDatabase::Close (Andrew Chow)
a0de83372be83f59015cd3d61af2303b74fb64b5 Implement SQLiteDatabase::Open (Andrew Chow)
3bfa0fe1259280f8c32b41a798c9453b73f89b02 Initialize and Shutdown sqlite3 globals (Andrew Chow)
5a488b3d77326a0d957c1233493061da1b6ec207 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow)
ca8b7e04ab89f99075b093fa248919fd10acbdf7 Implement SQLiteDatabaseVersion (Andrew Chow)
7577b6e1c88a1a7b45ecf5c7f1735bae6f5a82bf Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow)
e87df8258090138d5c22ac46b8602b618620e8a1 Add sqlite to travis and depends (Andrew Chow)
54729f3f4e6765dfded590af5fb28c88331685f8 Add libsqlite3 (Andrew Chow)

Pull request description:

  This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`.

  For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite.

  We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use.

  I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier.

ACKs for top commit:
  Sjors:
    re-utACK c4a29d0a90b821c443c10891d9326c534d15cf97
  promag:
    Tested ACK c4a29d0a90b821c443c10891d9326c534d15cf97.
  fjahr:
    reACK c4a29d0a90b821c443c10891d9326c534d15cf97
  S3RK:
    Re-review ACK c4a29d0a90b821c443c10891d9326c534d15cf97
  meshcollider:
    re-utACK c4a29d0a90b821c443c10891d9326c534d15cf97
  hebasto:
    re-ACK c4a29d0a90b821c443c10891d9326c534d15cf97, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19077#pullrequestreview-507743699) review, verified with `git range-diff master d18892dcc c4a29d0a9`.
  ryanofsky:
    Code review ACK c4a29d0a90b821c443c10891d9326c534d15cf97. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction.
  jonatack:
    ACK c4a29d0a90b821c443c10891d9326c534d15cf97, debug builds and test runs after rebase to latest master @ c2c4dbaebd9, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns.

Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c
2024-03-07 01:23:21 +07:00
Samuel Dobson
a340ad641e
Merge #20262: tests: Skip --descriptor tests if sqlite is not compiled
7411876c75471a45ad40c38c668db3a4b9413fb9 Ensure a legacy wallet for BDB format check (Andrew Chow)
586640381a2c379ce3d6366b1b4534ccc4e8ccf2 Skip --descriptor tests if sqlite is not compiled (Andrew Chow)

Pull request description:

  #20156 allows sqlite to not be compiled by configuring `--without-sqlite`. However doing so and then running the test runner will result in all of the `--descriptor` tests to fail. We should be skipping those tests if sqlite was not compiled.

ACKs for top commit:
  practicalswift:
    ACK 7411876c75471a45ad40c38c668db3a4b9413fb9: patch looks correct
  Sjors:
    tACK 7411876c75471a45ad40c38c668db3a4b9413fb9
  ryanofsky:
    Code review ACK 7411876c75471a45ad40c38c668db3a4b9413fb9
  hebasto:
    ACK 7411876c75471a45ad40c38c668db3a4b9413fb9, tested on Linux Mint 20 (x86_64), tests pass for binaries compiled with:

Tree-SHA512: 1d635a66d2b7bb865300144dfefcfdaf86133aaaa020c8f440a471476ac1205d32f2df704906ce6c2ea48ddf791c3c95055f6291340b4f7b353c1b02cab5cabe
2024-03-07 01:23:20 +07:00
Andrew Chow
f293c046f4
Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow)
869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow)
886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow)
f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow)
82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow)
953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)

Pull request description:

  Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.

  Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.

  Descriptors can also be imported with a new `importdescriptors` RPC.

  Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.

  A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).

ACKs for top commit:
  Sjors:
    utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed)
  jonatack:
    Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82.
  fjahr:
    re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
  instagibbs:
    light re-ACK 223588b
  meshcollider:
    Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82

Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986

Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it

Introduce DescriptorScriptPubKeyMan as a dummy class

Add WALLET_FLAG_DESCRIPTORS

Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet

Create LegacyScriptPubKeyMan when not a descriptor wallet

Introduce WalletDescriptor class

WalletDescriptor is a Descriptor with other wallet metadata

Add a lock cs_desc_man for DescriptorScriptPubKeyMan

Store WalletDescriptor in DescriptorScriptPubKeyMan

Implement SetType in DescriptorScriptPubKeyMan

Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet

Implement IsMine for DescriptorScriptPubKeyMan

Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE

Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan

Implement IsHDEnabled in DescriptorScriptPubKeyMan

Implement GetID for DescriptorScriptPubKeyMan

Load the descriptor cache from the wallet file

Implement loading of keys for DescriptorScriptPubKeyMan

Add IsSingleType to Descriptors

IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys

Implement several simple functions in DescriptorScriptPubKeyMan

Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB

Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file

Implement SetupGeneration for DescriptorScriptPubKeyMan

Implement TopUp in DescriptorScriptPubKeyMan

Implement GetNewDestination for DescriptorScriptPubKeyMan

Implement Unlock and Encrypt in DescriptorScriptPubKeyMan

Implement GetReservedDestination in DescriptorScriptPubKeyMan

Implement ReturnDestination in DescriptorScriptPubKeyMan

Implement GetKeypoolOldestTime and only display it if greater than 0

Implement GetSolvingProvider for DescriptorScriptPubKeyMan

Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.

Implement SignTransaction in DescriptorScriptPubKeyMan

Implement SignMessage for descriptor wallets

Implement FillPSBT in DescriptorScriptPubKeyMan

FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.

Change GetMetadata to use unique_ptr<CKeyMetadata>

Implement GetMetadata in DescriptorScriptPubKeyMan

Be able to create new wallets with DescriptorScriptPubKeyMans as backing

Generate new descriptors when encrypting

Add IsLegacy to CWallet so that the GUI knows whether to show watchonly

add importdescriptors RPC and tests for native descriptor wallets

Co-authored-by: Andrew Chow <achow101-github@achow101.com>

Functional tests for descriptor wallets

Change wallet_encryption.py to use signmessage instead of dumpprivkey

Return error when no ScriptPubKeyMan is available for specified type

When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.

Implement CWallet::IsSpentKey for non-LegacySPKMans

tests: Add RPCOverloadWrapper which overloads some disabled RPCs

RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.

Add a --descriptors option to various tests

Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.

Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py <---- wallet_keypool_hd.py actually
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
2024-03-07 01:23:15 +07:00
Kittywhiskers Van Gogh
c2aa01cf1d
merge bitcoin#28374: python cryptography required for BIP 324 functional tests 2024-03-05 21:43:22 +00:00
MarcoFalke
ad3f424b4d
partial Merge #18638: net: Use mockable time for ping/pong, add tests
fa3365430c5fb57d7c0b5f2bce9fbbe290be93c3 net: Use mockable time for ping/pong, add tests (MarcoFalke)
faab4aaf2fa1153c6d76efc8113fa01b06943ece util: Add count_microseconds helper (MarcoFalke)

Pull request description:

  Switch `CNode::m_ping_start` and `CNetMessage::m_time` to mockable time, so that tests can be added.

  Mockable time is also type-safe, since it uses `std::chrono`

ACKs for top commit:
  jonatack:
    Code review re-ACK fa33654 re-read code, verified rebase per `git range-diff 4b5c919 fa94d6f fa33654`, previous tested ACKs still valid
  troygiorshev:
    ACK fa3365430c5fb57d7c0b5f2bce9fbbe290be93c3

Tree-SHA512: 7d632bd6019ce7c882029e71b667a61517e783af82755a85dd979ef09380934e172dec8b8f91d57b200a30a6e096aeaf01f19fee7f3aed0e0e871c72eb44d70e
2024-03-06 02:00:30 +07:00
Konstantin Akimov
a392be6cf0
fix: actually run rpc_fundrawtransaction with and without HD feature 2024-03-03 23:39:11 -06:00
fanquake
a91512e1d4
Merge bitcoin/bitcoin#24574: test: Actually print TSan tracebacks
fa76d8d4d71d844e217686881d4f630eac3a8e10 test: Actually print TSan tracebacks (MarcoFalke)

Pull request description:

  Commit 5e5138a721738f47053d915e4c65f925838ad5b4 made the TSan logs to be printed before returning an error from the ci script.

  However, it seems that on Cirrus CI, the `--failfast` option will kill not only all python process and bitcoind child process, but also the parent CI bash script, rendering the `trap` inefficient. I believe this bug was introduced in commit 451b96f7d2796d00eabaec56d831f9e9b1a569cc.

ACKs for top commit:
  fanquake:
    utACK fa76d8d4d71d844e217686881d4f630eac3a8e10

Tree-SHA512: 686f889d38a343882cb62ad6e0c2080196330e7cc7086891a7ff66d9443b455c82ba8d7e4a5cc42daa0513b0ad2743055bfe90e2f6ac88a910ee3b663fabddcd
2024-02-22 20:58:43 -06:00
Kittywhiskers Van Gogh
3710439adf
merge bitcoin#24005: add python implementation of Elligator swift 2024-02-19 10:17:13 -06:00
Kittywhiskers Van Gogh
74925f94c2
merge bitcoin#27542: add ripemd160 to test framework modules list 2024-02-19 10:17:12 -06:00
Kittywhiskers Van Gogh
13c8dc535c
partial bitcoin#19953: Implement BIP 340-342 validation
includes:
- 3c226639eb134314a0640d34e4ccb6148dbde22f
- f06e6d03452cf5e0b1a0863afb08c9e6d3ef452e (only changes to test/
  functional/test_framework/key.py)
2024-02-19 10:17:10 -06:00
MarcoFalke
9a92452a5c
Merge #19509: Per-Peer Message Capture
bff7c66e67aa2f18ef70139338643656a54444fe Add documentation to contrib folder (Troy Giorshev)
381f77be858d7417209b6de0b7cd23cb7eb99261 Add Message Capture Test (Troy Giorshev)
e4f378a505922c0f544b4cfbfdb169e884e02be9 Add capture parser (Troy Giorshev)
4d1a582549bc982d55e24585b0ba06f92f21e9da Call CaptureMessage at appropriate locations (Troy Giorshev)
f2a77ff97bec09dd5fcc043d8659d8ec5dfb87c2 Add CaptureMessage (Troy Giorshev)
dbf779d5deb04f55c6e8493ce4e12ed4628638f3 Clean PushMessage and ProcessMessages (Troy Giorshev)

Pull request description:

  This PR introduces per-peer message capture into Bitcoin Core.  📓

  ## Purpose

  The purpose and scope of this feature is intentionally limited.  It answers a question anyone new to Bitcoin's P2P protocol has had: "Can I see what messages my node is sending and receiving?".

  ## Functionality

  When a new debug-only command line argument `capturemessages` is set, any message that the node receives or sends is captured.  The capture occurs in the MessageHandler thread.  When receiving a message, it is captured as soon as the MessageHandler thread takes the message off of the vProcessMsg queue.  When sending, the message is captured just before the message is pushed onto the vSendMsg queue.

  The message capture is as minimal as possible to reduce the performance impact on the node.  Messages are captured to a new `message_capture` folder in the datadir.  Each node has their own subfolder named with their IP address and port.  Inside, received and sent messages are captured into two binary files, msgs_recv.dat and msgs_sent.dat, like so:

  ```
  message_capture/203.0.113.7:56072/msgs_recv.dat
  message_capture/203.0.113.7:56072/msgs_sent.dat
  ```

  Because the messages are raw binary dumps, included in this PR is a Python parsing tool to convert the binary files into human-readable JSON.  This script has been placed on its own and out of the way in the new `contrib/message-capture` folder.  Its usage is simple and easily discovered by the autogenerated `-h` option.

  ## Future Maintenance

  I sympathize greatly with anyone who says "the best code is no code".

  The future maintenance of this feature will be minimal.  The logic to deserialize the payload of the p2p messages exists in our testing framework.  As long as our testing framework works, so will this tool.

  Additionally, I hope that the simplicity of this tool will mean that it gets used frequently, so that problems will be discovered and solved when they are small.

  ## FAQ

  "Why not just use Wireshark"

  Yes, Wireshark has the ability to filter and decode Bitcoin messages.  However, the purpose of the message capture added in this PR is to assist with debugging, primarily for new developers looking to improve their knowledge of the Bitcoin Protocol.  This drives the design in a different direction than Wireshark, in two different ways.  First, this tool must be convenient and simple to use.  Using an external tool, like Wireshark, requires setup and interpretation of the results.  To a new user who doesn't necessarily know what to expect, this is unnecessary difficulty.  This tool, on the other hand, "just works".  Turn on the command line flag, run your node, run the script, read the JSON.  Second, because this tool is being used for debugging, we want it to be as close to the true behavior of the node as possible.  A lot can happen in the SocketHandler thread that would be missed by Wireshark.

  Additionally, if we are to use Wireshark, we are at the mercy of whoever it maintaining the protocol in Wireshark, both as to it being accurate and recent.  As can be seen by the **many** previous attempts to include Bitcoin in Wireshark (google "bitcoin dissector") this is easier said than done.

  Lastly, I truly believe that this tool will be used significantly more by being included in the codebase.  It's just that much more discoverable.

ACKs for top commit:
  MarcoFalke:
    re-ACK bff7c66e67aa2f18ef70139338643656a54444fe only some minor changes: 👚
  jnewbery:
    utACK bff7c66e67aa2f18ef70139338643656a54444fe
  theStack:
    re-ACK bff7c66e67aa2f18ef70139338643656a54444fe

Tree-SHA512: e59e3160422269221f70f98720b47842775781c247c064071d546c24fa7a35a0e5534e8baa4b4591a750d7eb16de6b4ecf54cbee6d193b261f4f104e28c15f47
2024-02-14 10:34:10 -06:00
Kittywhiskers Van Gogh
792b430547
partial bitcoin#20833: enable packages through testmempoolaccept
excludes:
- c9e1a26d1f17c8b98632b7796ffa8f8788b5a83c (will be added in future fuzzing PR)

inapplicable:
- 249f43f3cc52b0ffdf2c47aad95ba9d195f6a45e (we don't have RBF)
2024-02-02 23:14:06 -06:00
Samuel Dobson
c893da457f
Merge #20832: rpc: Better error messages for invalid addresses
8f0b64fb513e8c6cdd1f115856100a4ef5afe23e Better error messages for invalid addresses (Bezdrighin)

Pull request description:

  This PR addresses #20809.

  We add more detailed error messages in case an invalid address is provided inside the 'validateaddress' and 'getaddressinfo' RPC calls. This also covers the case when a user provides an address from a wrong network.

  We also add a functional test to test the new error messages.

ACKs for top commit:
  kristapsk:
    ACK 8f0b64fb513e8c6cdd1f115856100a4ef5afe23e
  meshcollider:
    Code review ACK 8f0b64fb513e8c6cdd1f115856100a4ef5afe23e

Tree-SHA512: ca0f806ab573e96b79e98d9f8c810b81fa99c638d9b5e4d99dc18c8bd2568e6a802ec305fdfb2983574a97a19a46fd53b77645f8078fb77e9deb24ad2a22cf93
2024-02-01 11:09:04 -06:00
MarcoFalke
cc6ebdb8da
Merge #19083: test: msg_mempool, fRelay, and other bloomfilter tests
(BACKPORT NOTICE: p2p_filter.py has also fixes for d5fbd4a92a3e7c1f8266d4cb4b639a0fe4c4c61f:test/functional/p2p_filter.py)
+    Merge #18726: test: check misbehavior more independently in p2p_filter.py
---------------------

dca73941eb0f0a4c9b68efed3870b536f7dd6cfe scripted-diff: rename node to peer for mininodes (gzhao408)
0474ea25afc65546cbfe5f822c0212bf3e211023 [test] fix race conditions and test in p2p_filter (gzhao408)
4ef80f0827392a1310ca5a29cc1f8f5ca5d16f95 [test] sending invalid msgs to node with bloomfilters=0 causes disconnect (gzhao408)
497a619386008dfaec0db15ecaebcdfaf75f5011 [test] add BIP 37 test for node with fRelay=false (gzhao408)
e8acc6015695c8439fc971a12709468995b96dcf [test] add mempool msg test for node with bloomfilter enabled (gzhao408)

Pull request description:

  This PR adds a few tests that are bloomfilter-related, including behavior for when bloomfilters are turned _off_:
  1. Tests p2p message `msg_mempool`: a node that has `peerbloomfilters` enabled should send its mempool (disabled behavior already tested [here](https://github.com/bitcoin/bitcoin/blob/master/test/functional/p2p_mempool.py)).
  2. Tests that bloomfilter peers with [`fRelay=False`](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki#extensions-to-existing-messages) in the `version` message should not receive any invs until they set the filter. The rest is the same as what’s already tested in `p2p_filter.py`.
  3. Tests that peers get disconnected if they send `filterload` or `filteradd` p2p messages to a node with bloom filters disabled.
  4. Refactor: renames p2p_mempool.py to p2p_nobloomfilter_messages.py.
  5. Fixes race conditions in p2p_filter.py

ACKs for top commit:
  MarcoFalke:
    ACK dca73941eb only changes is restoring accidentally deleted test 🍮
  jonatack:
    ACK dca73941eb0f0a4c9b68efed3870b536f7dd6cfe modulo a few nits if you retouch, happy to re-ACK if you take any of them but don't feel obliged to.

Tree-SHA512: 442aeab0755cb8b830251ea170d1d5e6da8ac9029b3276d407a20ee3d588cc61b77b8842368de18c244056316b8c63b911776d6e106bc7c023439ab915b27ad3
2024-01-16 15:05:09 -06:00
laanwj
a245e52f55
Merge bitcoin/bitcoin#24793: test: Change color of skipped functional tests
3258bad996262792ba77573d6080dafa3952929c changes color of skipped functional tests (Jacob P. Fickes)

Pull request description:

  changes the color of skipped functional tests (currently grey and can be hard to read/invisible on dark backgrounds) to yellow.

  resolves #24791

ACKs for top commit:
  theStack:
    Tested ACK 3258bad996262792ba77573d6080dafa3952929c
  jarolrod:
    Tested ACK 3258bad996

Tree-SHA512: 3fe5ae0d3b4902b2b6bda6e89ab780feb8bf4b7cb1ce7e8467057b94a1e0a26ddeaf3cac0bc19b06ef10d8bccaac9c495029d42740fbedab8fb0d5fdd7d02eaf
2024-01-14 11:05:35 -06:00
fanquake
271acac3a7
Merge #18210: test: type hints in Python tests
bd7e530f010d43816bb05d6f1590d1cd36cdaa2c This PR adds initial support for type hints checking in python scripts. (Kiminuo)

Pull request description:

  This PR adds initial support for type hints checking in python scripts.

  Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention."

  [Mypy](https://mypy.readthedocs.io/en/latest/index.html) is used in `lint-python.sh` to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error.

  **Notes:**

  * [--ignore-missing-imports](https://mypy.readthedocs.io/en/latest/command_line.html#cmdoption-mypy-ignore-missing-imports) switch is passed on to `mypy` checker for now. The effect of this is that one does not need `# type: ignore` for `import zmq`. More information about import processing can be found [here](https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports). This can be changed in a follow-up PR, if it is deemed useful.
  * We are stuck with Python 3.5 until 04/2021 (see https://packages.ubuntu.com/xenial/python3). When Python version is bumped to 3.6+, one can change:
      ```python
      _opcode_instances = []  # type: List[CScriptOp]
      ```
      to
      ```python
      _opcode_instances:List[CScriptOp] = []
      ```
      for type hints that are **not** function parameters and function return types.

  **Useful resources:**

  * https://docs.python.org/3.5/library/typing.html
  * https://www.python.org/dev/peps/pep-0484/

ACKs for top commit:
  fanquake:
    ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c - the type checking is not the most robust (there are things it fails to detect), but I think this is worth adopting (in a limited capacity while we maintain 3.5 compat).
  MarcoFalke:
    ACK bd7e530f010d43816bb05d6f1590d1cd36cdaa2c fine with me

Tree-SHA512: 21ef213915fb1dec6012f59ef17484e6c9e0abf542a316b63d5f21a7778ad5ebabf8961ef5fc8e5414726c2ee9c6ae07c7353fb4dd337f8fcef5791199c8987a
2024-01-10 12:07:54 -06:00
Wladimir J. van der Laan
ae97ed8a6f
Merge #19368: test: improve functional tests compatibility with BSD/macOS
3a7e79478ab41af7c53ce14d9fca9815bffe1f73 test: retry when write to a socket fails on macOS (Ivan Metlushko)
8cf9d15b823d91d2a74fc83832fccca2219342c9 test: use pgrep for better compatibility (Ivan Metlushko)

Pull request description:

  Rationale: a few minor changes to make experience of running tests on macOS a bit better
  1.`pidof` is not available on BSD/macOS, while `pgrep` is present on BSD, Linux and macOS
  2. Add retry as a workaround for a weird behavior when writing to a socket (https://bugs.python.org/issue33450). Stacktrace attached

  Man pages:
  https://www.freebsd.org/cgi/man.cgi?query=pgrep&apropos=0&sektion=1&manpath=FreeBSD+6.0-RELEASE&arch=default&format=html
  https://man7.org/linux/man-pages/man1/pgrep.1.html

  Related to #19281

  Stacktrace example:
  ```
  ...
  33/161 - feature_abortnode.py failed, Duration: 63 s

  stdout:
  2020-06-11T10:46:43.947000Z TestFramework (INFO): Initializing test directory /var/folders/2q/d5w9zh614r7g5c8r74ln3g400000gq/T/test_runner_₿_🏃_20200611_174102/feature_abortnode_128
  2020-06-11T10:46:45.199000Z TestFramework (INFO): Waiting for crash
  2020-06-11T10:47:15.921000Z TestFramework (INFO): Node crashed - now verifying restart fails
  2020-06-11T10:47:47.068000Z TestFramework (INFO): Stopping nodes
  [node 1] Cleaning up leftover process

  stderr:
  Traceback (most recent call last):
    File "/Users/xxx/Projects/bitcoin/test/functional/feature_abortnode.py", line 50, in <module>
      AbortNodeTest().main()
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_framework.py", line 142, in main
      exit_code = self.shutdown()
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_framework.py", line 266, in shutdown
      self.stop_nodes()
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_framework.py", line 515, in stop_nodes
      node.stop_node(wait=wait)
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_node.py", line 318, in stop_node
      self.stop(wait=wait)
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/authproxy.py", line 142, in __call__
      response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/authproxy.py", line 107, in _request
      self.__conn.request(method, path, postdata, headers)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 1107, in request
      self._send_request(method, url, body, headers)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 1152, in _send_request
      self.endheaders(body)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 1103, in endheaders
      self._send_output(message_body)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 936, in _send_output
      self.send(message_body)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 908, in send
      self.sock.sendall(data)
  OSError: [Errno 41] Protocol wrong type for socket
  ```

ACKs for top commit:
  laanwj:
    ACK 3a7e79478ab41af7c53ce14d9fca9815bffe1f73

Tree-SHA512: fefbe40ce94ab29f18bbbed2a434194b1384ffa5279b1d04db7a3708e3dd422bd9e450f1db3f95a1a851fac5a626ab533c6ebcfd7ede96f8ccae9e6f3e9fff92
2024-01-06 19:30:15 -06:00
MarcoFalke
8b98e1ab7b Merge bitcoin/bitcoin#18795: Test: wallet issue with orphaned rewards
e4356f6a6c18e5027a064a4d3a5deda27985f584 Testcase for wallet issue with orphaned rewards. (Daniel Kraft)

Pull request description:

  This adds a new test case demonstrating the wallet issue when block rewards are orphaned (#14148).

ACKs for top commit:
  LarryRuane:
    ACK e4356f6a6c18e5027a064a4d3a5deda27985f584
  leonardojobim:
    reACK e4356f6a6c .

Tree-SHA512: e9a2310ee1b3d52cfa302f431ed3d272bbc1b9195439ff318d9eb1006c0b28968dbe840e1600b6ff185e5d7ea57e4dcc837cef16051b5537445e10bc363b8c22
2023-12-08 21:16:00 +03:00
MarcoFalke
a800821e9f Merge #18965: tests: implement base58_decode
60ed33904cf974e8f3c1b95392a23db1fe2d4a98 tests: implement base58_decode (10xcryptodev)

Pull request description:

  implements TODO: def base58_decode

ACKs for top commit:
  ryanofsky:
    Code review ACK 60ed33904cf974e8f3c1b95392a23db1fe2d4a98. Just suggested changes since last review. Thank you for taking suggestions!

Tree-SHA512: b3c06b4df041a6d88033cd077a093813a688e42d0b9aa777c715e5fd69cfba7b1bf984428bd98417d3c15232d3d48bc9c163317564f9e1d562db6611c21e2c10
2023-12-06 12:33:15 -06:00
Odysseas Gabrielides
112564974d
refactor: deprecate non-deterministic IS support (#5553)
## Issue being fixed or feature implemented
Non-deterministic IS locks aren't used anymore since v18 dip24.
We should drop that support to make code simpler.

## What was done?
Dropped non-deterministic IS code, `evo_instantsend_tests` and
`feature_llmq_is_migration.py` (don't need it anymore), adjusted func
tests.

## How Has This Been Tested?
all tests, synced Testnet

## Breaking Changes

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_

---------

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: Konstantin Akimov <545784+knst@users.noreply.github.com>
2023-11-20 10:17:04 -06:00
Konstantin Akimov
5e31bd5545 refactor: multiple fixes, cleanups, improvements and refactorings 2023-10-06 11:02:15 -05:00