Commit Graph

2384 Commits

Author SHA1 Message Date
pasta
1fc62c81a0
Merge #6020: backport: bitcoin#21053, #22082, #22118, #22292, #22308, #22334, #22358, #22388, bitcoin-core/gui#271, #311
0bed7b4702 Merge bitcoin/bitcoin#22292: bench, doc: benchmarking updates and fixups (fanquake)
c95df68637 Merge bitcoin/bitcoin#22388: ci: use Ubuntu 20.04 as the default Docker container (MarcoFalke)
c586ca5b56 Merge bitcoin/bitcoin#22334: wallet: do not spam about non-existent spk managers (fanquake)
62f9394374 Merge bitcoin/bitcoin#22308: wallet: Add missing BlockUntilSyncedToCurrentChain (MarcoFalke)
240d8efb82 Merge bitcoin/bitcoin#22358: Remove unused wallet pointer from wallet signals (fanquake)
9a1500ab47 Merge bitcoin/bitcoin#22118: test: check anchors.dat when node starts for the first time (MarcoFalke)
262c8b6f44 Merge bitcoin/bitcoin#22082: test: update nanobench from release 4.0.0 to 4.3.4 (MarcoFalke)
3d2cea667b Merge bitcoin-core/gui#311: Peers Window rename 'Peer id' to 'Peer' (Hennadii Stepanov)
dc498be3be Merge bitcoin-core/gui#271: Don't clear console prompt when font resizing (W. J. van der Laan)
1ed2d2d891 Merge #21053: rpc, test: document {previous,next}blockhash as optional (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Regular backports from bitcoin v22

  ## What was done?
   - bitcoin/bitcoin#21053
   - bitcoin-core/gui#271
   - bitcoin-core/gui#311
   - bitcoin/bitcoin#22082
   - bitcoin/bitcoin#22118
   - bitcoin/bitcoin#22358
   - bitcoin/bitcoin#22308
   - bitcoin/bitcoin#22334
   - bitcoin/bitcoin#22388
   - bitcoin/bitcoin#22292

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 0bed7b4702

Tree-SHA512: 8354297857516ddf94b242f2e50e1a28d999e613da2a7eb90c603e1fee7212e46d6e8a20ad42aa2945b48137e98fc7f589c9c77469c71cc01d032a33fa6da517
2024-05-18 21:08:06 -05:00
MarcoFalke
38649da984
Merge bitcoin/bitcoin#22520: test: improve rpc_blockchain.py tests and assert on time and mediantime
ef5e9304cd407adab1563f24215da1b582274c20 test: update logging and docstring in rpc_blockchain.py (Jon Atack)
d548dc71e4849f638fccaea6be86ac4fa5304f01 test: replace magic values by constants in rpc_blockchain.py (Jon Atack)
78c361086fc0bf27612e8142bd33e05e37a36af6 test: assert on mediantime in getblockheader and getblockchaininfo (Jon Atack)
0a9129c588ab016eb0453b40a0cae918ca4aa6a2 test: assert on the value of getblockchaininfo#time (Jon Atack)

Pull request description:

  Follow-up to #22407 improving test coverage per https://github.com/bitcoin/bitcoin/pull/22407#pullrequestreview-702077013.

ACKs for top commit:
  tryphe:
    untested ACK ef5e9304cd407adab1563f24215da1b582274c20

Tree-SHA512: f746d56f430331bc6a2ea7ecd27b21b06275927966aacf1f1127d8d5fdfd930583cabe72e23df3adb2e005da904fc05dc573b8e5eaa2f86e0e193b89a17a5734
2024-05-18 17:54:16 -05:00
MarcoFalke
61f9d96f38
Merge bitcoin/bitcoin#22423: test: wallet_listtransactions improvements (speedup, cleanup, logging)
a006d7d73019b8cf4d68626c019c3d69729dda69 test: add logging to wallet_listtransactions (Sebastian Falbesoner)
47915b118720c6e2b2ec9f599f25848041b42b99 test: remove unneeded/redundant code in wallet_listtransactions (Sebastian Falbesoner)
fb6c6a7938cb7c4808ad88d23bfc2b7408407b12 test: speedup wallet_listtransactions by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)

Pull request description:

  This PR improves the test `wallet_listtransactions.py` in three ways:
  * speeds up runtime by a factor of 2-3x by using the good ol' immediate tx relay trick (`-whitelist=noban@127.0.0.1`)
  * removes unneeded/redundant code
  * adds log messages, mostly by turning comments into `self.log.info(...)` calls

ACKs for top commit:
  jonatack:
    ACK a006d7d73019b8cf4d68626c019c3d69729dda69
  kristapsk:
    ACK a006d7d73019b8cf4d68626c019c3d69729dda69

Tree-SHA512: a91a19f5ebc4d05f0b96c5419683c4c57ac0ef44b64eeb8dd550bd72296fd3a2857a3ba83f755fe4b0b3bd06439973f226070b5d0ce2dee58344dae78cb50290
2024-05-18 17:54:16 -05:00
Konstantin Akimov
e0ad143e08
chore: dashify file list exception for liner 2024-05-16 02:10:16 +07:00
Konstantin Akimov
98a2dad78c
fix: wrong permission for various files accordingly new linter 2024-05-16 02:09:48 +07:00
MacroFake
c8c58a1810
Merge bitcoin/bitcoin#25015: test: Use permissions from git in lint-files.py
908fb7e2ec37fe68675d38dbfee4df9f861bb2b5 test: Use permissions from git in `lint-files.py` (laanwj)
48d2e80a7479a44b0ab09e87542c8cb7a8f72223 test: Don't use shell=True in `lint-files.py` (laanwj)

Pull request description:

  Improvements to the `lint-files.py` script:

  - Avoid use of `shell=True`.
  - Check the permissions in git's metadata instead of in the filesystem. This stops the umask or filesystem from interfering. It's also more efficient as it only needs a single call to `git ls-files`.

  (what triggered this change was `File "..." contains a shebang line, but has the file permission 775 instead of the expected executable permission 755.` errors running the script locally).

ACKs for top commit:
  vincenzopalazzo:
    re-tACK 908fb7e2ec

Tree-SHA512: 2eaf868c55a9c3508b12658a5b3ac429963fd0551e645332d0ac54be56fefccee95115e4667386df24b46b545593cb0d0bf8c6cecab73f9cb19d37ddf704c614
2024-05-16 02:09:38 +07:00
MarcoFalke
f226e8dc1f
Merge bitcoin/bitcoin#24762: lint: Start to use py lint scripts
fae211c0ae0dd90876a3390eb21449b7b0bb45c4 lint: Start to use py lint scripts (MarcoFalke)
fa82e890e7950fe5ba6d4fa88fcd922cc929dc47 Move lint script and data file to avoid lint- prefix (MarcoFalke)

Pull request description:

ACKs for top commit:
  fjahr:
    tACK fae211c0ae0dd90876a3390eb21449b7b0bb45c4

Tree-SHA512: f8272a1bab9efb8203cac121710baae68f01f79e520ad71ff15aa516d19763d61c088b411b019de105a6a30e7ee3c274814d59963f6ac22ba1084560fb601f45
2024-05-16 02:09:38 +07:00
MarcoFalke
85013e99d6
Merge bitcoin/bitcoin#21873: test: minor fixes & improvements for files linter test
2227fc4e6203064b14e99bcf453601bd263a0196 test: minor fixes & improvements for files linter test (windsok)

Pull request description:

  Couple of minor fixes & improvements for files linter test added in #21740

  - Use a context manager when opening files, so that files are closed are we are done with them

  - Use the `-z` flag when shelling out to `git ls-files` so that we can catch newlines and other weird control characters in filenames.

  From the `git ls-files` manpage:
  ```
  -z \0 line termination on output and do not quote filenames. See OUTPUT below for more information.

  Without the -z option, pathnames with "unusual" characters are quoted as explained for the configuration variable
  core.quotePath (see git-config(1)). Using -z the filename is output verbatim and the line is terminated by a NUL byte.
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK 2227fc4e6203064b14e99bcf453601bd263a0196
  practicalswift:
    cr ACK 2227fc4e6203064b14e99bcf453601bd263a0196: patch looks correct

Tree-SHA512: af059a805f4a7614162de85dea856052a45ab531895cb0431087e7fc9e037513fa7501bb5eb2fe43238adf5f09e77712ebdbb15b1486983359ad3661a3da0c60
2024-05-16 02:09:38 +07:00
W. J. van der Laan
dce79f5c8e
Merge bitcoin/bitcoin#21740: test: add new python linter to check file names and permissions
46b025e00df40724175735eb5606ac73067cb3b8 test: add new python linter to check file names and permissions (windsok)
6f6bb3ebc7cb8e17a5dfc8ef55aa2d3f2dc6bdea test: fix file permissions on various scripts (windsok)

Pull request description:

  Adds a new python linter test which tests for correct filenames and file permissions in the repository.

  Replaces the existing tests in the `test/lint/lint-filenames.sh` and `test/lint/lint-shebang.sh` linter tests, as well as adding some new and increased testing. This increased coverage is intended to catch issues such as in #21728 and https://github.com/bitcoin/bitcoin/pull/16807/files#r345547050

  Summary of tests:
  * Checks every file in the repository against an allowed regexp to make sure only lowercase or uppercase alphanumerics (a-zA-Z0-9), underscores (_), hyphens (-), at (@) and dots (.) are used in repository filenames.

  * Checks only source files (*.cpp, *.h, *.py, *.sh) against a stricter allowed regexp to make sure only lowercase alphanumerics (a-z0-9), underscores (_), hyphens (-) and dots (.) are used in source code filenames. Additionally there is an exception regexp for directories or files which are excepted from matching this regexp (This should replicate the existing `test/lint/lint-filenames.sh` test)

  * Checks all files in the repository match an allowed executable or non-executable file permission octal. Additionally checks that for executable files, the file contains a shebang line.

  * Checks that for executable `.py` and `.sh` files, the shebang line used matches an allowable list of shebangs (This should replicate the existing `test/lint/lint-shebang.sh` test)

  * Checks every file that contains a shebang line to ensure it has an executable permission

  Additionally updates the permissions on various files to comply with the new tests.

  Fixes #21729

ACKs for top commit:
  practicalswift:
    cr re-ACK 46b025e00df40724175735eb5606ac73067cb3b8: patch still looks correct
  kiminuo:
    code review ACK 46b025e00df40724175735eb5606ac73067cb3b8 if `contrib/gitian-descriptors/assign_DISTNAME` permission change is deemed OK.
  laanwj:
    Code review ACK 46b025e00df40724175735eb5606ac73067cb3b8

Tree-SHA512: 1c8201a2cee0d9cbce15652b68cec9a6458a8b493fcd5392f98560aca0b1a12e668baab65a47100f116f626dadc3f591deb47f7368468c6a46c6c712c2533455
2024-05-16 02:09:37 +07:00
MarcoFalke
62f9394374
Merge bitcoin/bitcoin#22308: wallet: Add missing BlockUntilSyncedToCurrentChain
fa27baa9c8a13239625e5a7b6c472d236fe5b9fa Revert "test: Add temporary logging to debug #20975" (MarcoFalke)
fadb55085a02c9e355617bcb5f84b6335e4f8c9d wallet: Add missing BlockUntilSyncedToCurrentChain (MarcoFalke)

Pull request description:

  Fixes #20975

  Also replace the wallet pointer by a reference

ACKs for top commit:
  achow101:
    ACK fa27baa9c8a13239625e5a7b6c472d236fe5b9fa

Tree-SHA512: 79047a30998104a12c2ff84a8e3cc5207151410bbe92b74cfedbe1c1aca3ffa5909391607fc597f3a3cf0725fa827528a4c57edaeacc8360536b1965e166be6a
2024-05-15 03:03:17 +07:00
MarcoFalke
9a1500ab47
Merge bitcoin/bitcoin#22118: test: check anchors.dat when node starts for the first time
ef99d03c2bbb6b5fa5ff3d3d3cb9c5da7d471133 test: check anchors.dat when node starts for the first time (bruno)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/21338#discussion_r598406712, https://github.com/bitcoin/bitcoin/pull/21338#discussion_r598406187, https://github.com/bitcoin/bitcoin/pull/21338#discussion_r598405613.

ACKs for top commit:
  laanwj:
    Code review ACK ef99d03c2bbb6b5fa5ff3d3d3cb9c5da7d471133

Tree-SHA512: 505f1f34fbc0c72a92968883be0f1c5f169a4ba3aa8a56e1ce8bc5e514f49e3a17ce51fd40be0073dc4bc06eaeda36dfe90ace843c181170e34643226afd78ef
2024-05-15 03:03:17 +07:00
MarcoFalke
1ed2d2d891
Merge #21053: rpc, test: document {previous,next}blockhash as optional
ba7e17e073f833eccd4c7c111ae9058c3f123371 rpc, test: document {previous,next}blockhash as optional (Sebastian Falbesoner)

Pull request description:

  This PR updates the result help of the following RPCs w.r.t. the `previousblockhash` and `nextblockhash` fields:
  - getblockheader
  - getblock

  Also adds trivial tests on genesis block (should not contain "previousblockhash") and best block (should not contain "nextblockhash").

Top commit has no ACKs.

Tree-SHA512: ef42c5c773fc436e1b4a67be14e2532e800e1e30e45e54a57431c6abb714d2c069c70d40ea4012d549293b823a1973b3f569484b3273679683b28ed40abf46bb
2024-05-15 03:02:45 +07:00
pasta
146be9f0d8
Merge #6003: feat: support rpc protx-register for descriptor wallets - part VI
a33dcb3283 fix: CheckWalletOwnsScript/CheckWalletOwnsKey to use wallet instead of SPK (Konstantin Akimov)
b2ede8bfee feat: update list of tests that still doesn't support descriptor wallets (Konstantin Akimov)
838d06f2fd feat: enable descriptor wallets for more tests (Konstantin Akimov)
5ab108c982 feat: implementation of RPC 'protx register' for descriptor wallets (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  Many rpc such as `protx register` uses forcely LegacyScriptPubKeyMan instead using CWallet's interface.
  It causes a failures such as
  ```
  test_framework.authproxy.JSONRPCException: This type of wallet does not support this command (-4)
  ```
  for all functional tests that uses Masternodes/evo nodes.

  See https://github.com/dashpay/dash-issues/issues/59 to track progress

  ## What was done?
  Some direct usages of LegacyScriptPubKeyMan refactored to use CWallet's functionality.
  There are still 4 functional tests that doesn't work for descriptor wallets:
   - feature_dip3_deterministicmns.py (no rpc `protx updateregistar`)
   - feature_governance.py: no rpc for `governance votemany` and `governance votealias`
   - interface_zmq_dash.py (see governance)

  That's part I of changes, other changes are not PR-ready yet, WIP.

  ## How Has This Been Tested?
  Firstly, the flag `--legacy-wallets` are removed for many functional tests.
  Secondly, the flag `--descriptors` is inverted in default value:
  ```
  diff --git a/test/functional/test_framework/test_framework.py b/test/functional/test_framework/test_framework.py
  index 585a6a74d6..9ad5fd1daa 100755
  --- a/test/functional/test_framework/test_framework.py
  +++ b/test/functional/test_framework/test_framework.py
  @@ -242,10 +242,10 @@ class BitcoinTestFramework(metaclass=BitcoinTestMetaClass):

           if self.options.descriptors is None:
               # Prefer BDB unless it isn't available
  -            if self.is_bdb_compiled():
  -                self.options.descriptors = False
  -            elif self.is_sqlite_compiled():
  +            if self.is_sqlite_compiled():
                   self.options.descriptors = True
  +            elif self.is_bdb_compiled():
  +                self.options.descriptors = False
               else:
                   # If neither are compiled, tests requiring a wallet will be skipped and the value of self.options.descriptors won't matter
                   # It still needs to exist and be None in order for tests to work however.
  ```

  ## Breaking Changes
  N/A, descriptor wallets have not been publicly released yet

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK a33dcb3283

Tree-SHA512: 24e22ac91e30a3804d1ac9af864eb9d073208bbb6d1f3326c7f0438f3ccce2b553aa46450989e48cb5c6e0d838ff1c88c6522f195e7aa2bd89342710f3ecef77
2024-05-14 09:17:00 -05:00
Samuel Dobson
ad25d54300
Merge bitcoin/bitcoin#21329: descriptor wallet: Cache last hardened xpub and use in normalized descriptors
e6cf0ed92de31a5ac35a271b0da8f0a8364d1175 wallet, rpc: listdescriptors does not need unlocked (Andrew Chow)
3280704886b60644d103a5eb310691c003a39328 Pass in DescriptorCache to ToNormalizedString (Andrew Chow)
7a26ff10c2f2e139fbc63e2f37fb33ea4efae088 Change DescriptorImpl::ToStringHelper to use an enum (Andrew Chow)
75530c93a83f3e94bcb78b6aa463c5570c1e737e Remove priv option for ToNormalizedString (Andrew Chow)
74fede3b8ba69e2cc82c617cdf406ab79df58825 wallet: Upgrade existing descriptor caches (Andrew Chow)
432ba9e5434da90d2cf680f23e8c7b7164c9f945 wallet: Store last hardened xpub cache (Andrew Chow)
d87b544b834077f102724415e0fada6ee8b2def2 descriptors: Cache last hardened xpub (Andrew Chow)
cacc3910989c4f3d7afa530dbab042461426abce Move DescriptorCache writing to WalletBatch (Andrew Chow)
0b4c8ef75cd03c8f0a8cfadb47e0fbcabe3c5e59 Refactor Cache merging and writing (Andrew Chow)
976b53b085d681645fd3a008fe382de85647e29f Revert "Cache parent xpub inside of BIP32PubkeyProvider" (Andrew Chow)

Pull request description:

  Currently fetching a normalized descriptor requires the wallet to be unlocked as it needs the private keys to derive the last hardened xpub. This is not very user friendly as normalized descriptors shouldn't require and don't involve the private keys except for derivation. We solve this problem by caching the last hardened xpub (which has to be derived at some point when generating the address pool).

  However the last hardened xpub was not already being cached. We only cached the immediate parent xpub and derived child keys. For example, with a descriptor derivation path of `/84'/0'/0'/0/*`, the parent xpub that is cached is `m/84'/0'/0'/0`, and the child keys of `m/84'/0'/0'/0/i` (note that child keys would not be cached in this case). This parent xpub is not suitable for the normalized descriptor form as we want the key at `m/84'/0'/0'`. So this PR adds another field to `DescriptorCache` to cache the last hardened xpub so that we can use them for normalized descriptors.

  Since `DescriptorCache` is changing, existing descriptor wallets need to be upgraded to use this new cache. The upgrade will occur in the background either at loading time (if the wallet is not encrypted) or at unlocking time in the same manner that `UpgradeKeyMetadata` operates. It will use a new wallet flag `WALLET_FLAG_LAST_HARDENED_XPUB_CACHED` to indicate whether the descriptor wallet has the last hardened xpub cache.

  Lastly `listdescriptors` will not require the wallet to be locked and `getaddressinfo`'s `parent_desc` will always be output (assuming the upgrade has occurred).

ACKs for top commit:
  fjahr:
    tACK e6cf0ed92de31a5ac35a271b0da8f0a8364d1175
  S3RK:
    reACK e6cf0ed
  jonatack:
    Semi ACK e6cf0ed92de31a5ac35a271b0da8f0a8364d1175 reviewed, debug-built and ran unit tests and some of the descriptor functional tests at each commit. I'm not very familiar with this code and it could be clearer to the uninitiated IMHO, so I'm not confident enough to give a full ACK. Various minor suggestions follow, most of them for readability, feel free to pick and choose.
  meshcollider:
    Code review + functional test run ACK e6cf0ed92de31a5ac35a271b0da8f0a8364d1175

Tree-SHA512: ac27aade8644525cd65bfcaf27ff32afb974085b1451faf4ff68c6671a690bd6a41d4f39a33cbf461ae0fbe85995c0a4c08dbd36171da1c1d2a1d00053ad298d
2024-05-10 14:02:01 +07:00
MarcoFalke
19b2b27785
Merge #20403: wallet: upgradewallet fixes, improvements, test coverage
3eb6f8b2e61c24a22ea9396d86672307845f35eb wallet (not for backport): improve upgradewallet error messages (Jon Atack)
ca8cd893bb56bf5d455154b0498b1f58f77d20ed wallet: fix and improve upgradewallet error responses (Jon Atack)
99d56e357159c7154f69f28cb5587c5ca20d6594 wallet: fix and improve upgradewallet result responses (Jon Atack)
2498b04ce88696a3216fc38b7d393906b733e8b1 Don't upgrade to HD split if it is already supported (Andrew Chow)
c46c18b788cb0862aafbb116fd37936cbed6a431 wallet: refactor GetClosestWalletFeature() (Jon Atack)

Pull request description:

  This follows up on #18836 and #20282 to fix and improve the as-yet unreleased `upgradewallet` feature and also implement review follow-up in https://github.com/bitcoin/bitcoin/pull/18836#discussion_r519328607.

  This PR fixes 4 upgradewallet issues:

  - this bug: https://github.com/bitcoin/bitcoin/pull/20403#discussion_r526063920
  - it returns nothing in the absence of an RPC error, which isn't reassuring for users
  - it returns the same thing both in the case of a successful upgrade and when no upgrade took place
  - the error message object is currently dead code

  This PR fixes the above and provides:

  ...user feedback to not silently return without upgrading
  ```
  {
    "wallet_name": "disable private keys",
    "previous_version": 169900,
    "current_version": 169900,
    "result": "Already at latest version. Wallet version unchanged."
  }
  ```
  ...better feedback after successfully upgrading
  ```
  {
    "wallet_name": "watch-only",
    "previous_version": 159900,
    "current_version": 169900,
    "result": "Wallet upgraded successfully from version 159900 to version 169900."
  }
  ```
  ...helpful error responses
  ```
  {
    "wallet_name": "blank",
    "previous_version": 169900,
    "current_version": 169900,
    "error": "Cannot downgrade wallet from version 169900 to version 159900. Wallet version unchanged."
  }
  {
    "wallet_name": "blank",
    "previous_version": 130000,
    "current_version": 130000,
    "error": "Cannot upgrade a non HD split wallet from version 130000 to version 169899 without upgrading to support pre-split keypool. Please use version 169900 or no version specified."
  }
  ```
  updated help:
  ```
  upgradewallet ( version )

  Upgrade the wallet. Upgrades to the latest version if no version number is specified.
  New keys may be generated and a new wallet backup will need to be made.
  Arguments:
  1. version    (numeric, optional, default=169900) The version number to upgrade to. Default is the latest wallet version.

  Result:
  {                            (json object)
    "wallet_name" : "str",     (string) Name of wallet this operation was performed on
    "previous_version" : n,    (numeric) Version of wallet before this operation
    "current_version" : n,     (numeric) Version of wallet after this operation
    "result" : "str",          (string, optional) Description of result, if no error
    "error" : "str"            (string, optional) Error message (if there is one)
  }
  ```

ACKs for top commit:
  achow101:
    ACK  3eb6f8b
  MarcoFalke:
    review ACK 3eb6f8b2e61c24a22ea9396d86672307845f35eb 🛡

Tree-SHA512: b767314069e26b5933b123acfea6aa40708507f504bdb22884da020a4ca1332af38a7072b061e36281533af9f4e236d94d3c129daf6fe5b55241127537038eed
2024-05-10 13:59:59 +07:00
Andrew Chow
708586c77e
Merge #18836: wallet: upgradewallet fixes and additional tests
5f9c0b6360215636cfa62a70d3a70f1feb3977ab wallet: Remove -upgradewallet from dummywallet (MarcoFalke)
a314271f08215feba53ead27096ac7fda34acb3c test: Remove unused wallet.dat (MarcoFalke)
bf7635963c03203e7189ddaa56c6b086a0108cbf tests: Test specific upgradewallet scenarios and that upgrades work (Andrew Chow)
4b418a9decc3e855ee4b0bbf9e61121c8e9904e5 test: Add test_framework/bdb.py module for inspecting bdb files (Andrew Chow)
092fc434854f881330771a93a1280ac67b1d3549 tests: Add a sha256sum_file function to util (Andrew Chow)
0bd995aa19be65b0dd23df1df571c71428c2bc32 wallet: upgrade the CHDChain version number when upgrading to split hd (Andrew Chow)
8e32e1c41c995e832e643f605d35a7aa112837e6 wallet: remove nWalletMaxVersion (Andrew Chow)
bd7398cc6258c258e9f4411c50630ec4a552341b wallet: have ScriptPubKeyMan::Upgrade check against the new version (Andrew Chow)
5f720544f34dedf75b063b962845fa8eca604514 wallet: Add GetClosestWalletFeature function (Andrew Chow)
842ae3842df489f1b8d68e67a234788966218184 wallet: Add utility method for CanSupportFeature (Andrew Chow)

Pull request description:

  This PR cleans up the wallet upgrade mechanism a bit, fixes some probably bugs, and adds more test cases.

  The `nWalletMaxVersion` member variable has been removed as it made `CanSupportFeature` unintuitive and was causing a couple of bugs. The reason this was introduced originally was to allow a wallet upgrade to only occur when the new feature is first used. While this makes sense for the old `-upgradewallet` option, for an RPC, this does not quite make sense. It's more intuitive for an upgrade to occur if possible if the `upgradewallet` RPC is used as that's an explicit request to upgrade a particular wallet to a newer version. `nWalletMaxVersion` was only relevant for upgrades to `FEATURE_WALLETCRYPT` and `FEATURE_COMPRPUBKEY` both of which are incredibly old features. So for such wallets, the behavior of `upgradewallet` will be that the feature is enabled immediately without the wallet needing to be encrypted at that time (note that `FEATURE_WALLETCRYPT` indicates support for encryption, not that the wallet is encrypted) or for a new key to be generated.

  `CanSupportFeature` would previously indicate whether we could upgrade to `nWalletMaxVersion` not just whether the current wallet version supported a feature. While this property was being used to determine whether we should upgrade to HD and HD chain split, it was also causing a few bugs. Determining whether we should upgrade to HD or HD chain split is resolved by passing into `ScriptPubKeyMan::Upgrade` the version we are upgrading to and checking against that. By removing `nWalletMaxVersion` we also fix a bug where you could upgrade to HD chain split without the pre-split keypool.

  `nWalletMaxVersion` was also the version that was being reported by `getwalletinfo` which meant that the version reported was not always consistent across restarts as it depended on whether `upgradewallet` was used. Additionally to make the wallet versions consistent with actually supported versions, instead of just setting the wallet version to whatever is given to `upgradewallet`, we normalize the version number to the closest supported version number. For example, if given 150000, we would store and report 139900.

  Another bug where CHDChain was not being upgraded to the version supporting HD chain split is also fixed by this PR.

  Lastly several more tests have been added. Some refactoring to the test was made to make these tests easier. These tests check specific upgrading scenarios, such as from non-HD (version 60000) to HD to pre-split keypool. Although not specifically related to `upgradewallet`, `UpgradeKeyMetadata` is now being tested too.

  Part of the new tests is checking that the wallet files are identical before and after failed upgrades. To facilitate this, a utility function `sha256sum_file` has been added. Another part of the tests is to examine the wallet file itself to ensure that the records in the wallet.dat file have been correctly modified. So a new `bdb.py` module has been added to deserialize the BDB db of the wallet.dat file. This format isn't explicitly documented anywhere, but the code and comments in BDB's source code in file `dbinc/db_page.h` describe it. This module just dumps all of the fields into a dict.

ACKs for top commit:
  MarcoFalke:
    approach ACK 5f9c0b6360
  laanwj:
    Code review ACK 5f9c0b6360215636cfa62a70d3a70f1feb3977ab
  jonatack:
    ACK 5f9c0b6360215636cfa62a70d3a70f1feb3977ab, approach seems fine, code review, only skimmed the test changes but they look well done, rebased on current master, debug built and verified the `wallet_upgradewallet.py` test runs green both before and after running `test/get_previous_releases.py -b v0.19.1 v0.18.1 v0.17.2 v0.16.3 v0.15.2`

Tree-SHA512: 7c4ebf420850d596a586cb6dd7f2ef39c6477847d12d105fcd362abb07f2a8aa4f7afc5bfd36cbc8b8c72fcdd1de8d2d3f16ad8e8ba736b6f4f31f133fe5feba
2024-05-10 13:59:59 +07:00
Konstantin Akimov
266aefc544
feat: sethdseed rpc added. Based on bitcoin#12560 and the newest related changes
The key difference between bitcoin's and dash's implementation that sethdseed
does not update existing seed for wallet. Seed can be set only once.
It behave similarly to `upgradetohd` rpc, but since v20.1 all wallets are HD and
the name `upgradetohd` is not relevant more.
2024-05-10 13:59:44 +07:00
Konstantin Akimov
b2ede8bfee
feat: update list of tests that still doesn't support descriptor wallets
That are:
 - feature_dip3_deterministicmns.py
 - interface_zmq_dash.py
 - feature_governance.py
 - wallet_upgradetohd.py (as expected to be implemented for legacy-only wallets)
 - p2p_timeouts.py (why? can not understand it)

This partially reverts commit b20f812674.
2024-05-07 00:17:18 +07:00
Konstantin Akimov
838d06f2fd
feat: enable descriptor wallets for more tests
Enables for rpc_quorum.py, feature_notifications.py

see #5981, it partial revert of b20f812674
2024-05-07 00:17:18 +07:00
Kittywhiskers Van Gogh
33098aefff
merge bitcoin#21160: Move tx inventory into net_processing 2024-04-26 20:25:55 +00:00
Kittywhiskers Van Gogh
24205d94fe
partial bitcoin#20196: fix GetListenPort() to derive the proper port
excludes:
- 0cfc0cd32239d3c08d2121e028b297022450b320
- 7d64ea4a01920bb55bc6de0de6766712ec792a11
2024-04-26 20:25:31 +00:00
Kittywhiskers Van Gogh
6f8c730f35
merge bitcoin#19499: Make timeout mockable and type safe, speed up test 2024-04-26 20:25:31 +00:00
Kittywhiskers Van Gogh
b0216ac8a6
refactor: remove fMasternodeMode usage in rpc logic 2024-04-24 18:46:34 +00:00
pasta
27b2e5cd5d
Merge #5985: backport: bitcoin#21373, #21377, #21681, #21714, #21749, #21792, #21814, #21874, #21884, bitcoin-core/gui#284
7cc77f3a30 Merge #21373: test: generate fewer blocks in feature_nulldummy to fix timeouts, speed up (MarcoFalke)
a933a60b1a feat: new command line argument -bip147height for bitcoin#21373 (Konstantin Akimov)
51911388f2 Merge #21377: Speedy trial support for versionbits (fanquake)
ecade9bc39 Merge bitcoin/bitcoin#21749: test: Bump shellcheck version (W. J. van der Laan)
eeec2f2799 Merge bitcoin/bitcoin#21884: fuzz: Remove unused --enable-danger-fuzz-link-all option (fanquake)
51633d70ea Merge bitcoin/bitcoin#21874: fuzz: Add WRITE_ALL_FUZZ_TARGETS_AND_ABORT (MarcoFalke)
a02a2c0322 Merge bitcoin/bitcoin#21681: validation: fix ActivateSnapshot to use hardcoded nChainTx (MarcoFalke)
71f23d6e33 Merge bitcoin/bitcoin#21814: test: Fix feature_config_args.py intermittent issue (MarcoFalke)
de4d2a839d Merge bitcoin/bitcoin#21714: refactor: Drop CCoinControl::SetNull (MarcoFalke)
a63f9c31cc Merge bitcoin-core/gui#284: refactor: Simplify SendCoinsDialog::updateCoinControlState (Hennadii Stepanov)
b2d889380c Merge bitcoin/bitcoin#21792: test: Fix intermittent issue in p2p_segwit.py (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Regular batch of backports from bitcoin v22

  ## What was done?
  Implemented new commandline argument `-bip147height` for RegTest.
  Backports:

   - bitcoin/bitcoin#21377
   - bitcoin/bitcoin#21792
   - bitcoin-core/gui#284
   - bitcoin/bitcoin#21714
   - bitcoin/bitcoin#21373
   - bitcoin/bitcoin#21814
   - bitcoin/bitcoin#21681
   - bitcoin/bitcoin#21874
   - bitcoin/bitcoin#21884
   - bitcoin/bitcoin#21749

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  n/a

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 7cc77f3a30

Tree-SHA512: d46218667430af19c445d67d6411b1e7f19920e85f86e6e74ae7b9062aeb3763637a8613587c203ba8d285ccc3ee755f936141010944cfae8627397e8b8584d3
2024-04-24 10:17:59 -05:00
Kittywhiskers Van Gogh
bfd33cd2b4
net: move CConnman::RelayInv{Filtered} into PeerManager 2024-04-23 16:08:10 +00:00
Kittywhiskers Van Gogh
313a7e9a50
trivial: cleanup unnecessary headers in context files 2024-04-23 16:06:41 +00:00
Kittywhiskers Van Gogh
c3f1ac2291
net: retire CConnman::RelayTransaction, use PeerManager::RelayTransaction 2024-04-23 16:06:41 +00:00
MarcoFalke
7cc77f3a30
Merge #21373: test: generate fewer blocks in feature_nulldummy to fix timeouts, speed up
ccd976dd3dbb8f991dc1203ada2043f1736be5a4 test: use 327 fewer blocks in feature_nulldummy (Jon Atack)
68c280f19732fb96bc29113ce9c8007d0101868c test, refactor: abstract the feature_nulldummy blockheight values (Jon Atack)

Pull request description:

  The resolved timeout issue seen in the CI can be reproduced locally by running `test/functional/feature_nulldummy.py --valgrind --loglevel=debug`

  Speeds up the normal test runtime for me from 3.8 to 2.2 seconds (debug build). Thanks to Marco Falke for the approach suggestion.

ACKs for top commit:
  AnthonyRonning:
    reACK ccd976dd3dbb8f991dc1203ada2043f1736be5a4 - ran a few times with the rest of the tests and still passing for me with just the fewer block change.
  MarcoFalke:
    review ACK ccd976dd3dbb8f991dc1203ada2043f1736be5a4 🏝

Tree-SHA512: 38339dca4276d1972e3a5a5ee436da64e9e58fd3b50b186e34b07ade9523ac4c03f6c3869c5f2a59c23b43c44f87e712f8297a01a8d83202049c081e3eeb4445
2024-04-23 22:41:11 +07:00
fanquake
51911388f2
Merge #21377: Speedy trial support for versionbits
ffe33dfbd4c3b11e3475b022b6c1dd077613de79 chainparams: drop versionbits threshold to 90% for mainnnet and signet (Anthony Towns)
f054f6bcd2c2ce5fea84cf8681013f85a444e7ea versionbits: simplify state transitions (Anthony Towns)
55ac5f568a3b73d6f1ef4654617fb76e8bcbccdf versionbits: Add explicit NEVER_ACTIVE deployments (Anthony Towns)
dd07e6da48040dc7eae46bc7941db48d98a669fd fuzz: test versionbits delayed activation (Anthony Towns)
dd85d5411c1702c8ae259610fe55050ba212e21e tests: test versionbits delayed activation (Anthony Towns)
73d4a706393e6dbd6b6d6b6428f8d3233ac0a2d8 versionbits: Add support for delayed activation (Anthony Towns)
9e6b65f6fa205eee5c3b99343988adcb8d320460 tests: clean up versionbits test (Anthony Towns)
593274445004506c921d5d851361aefb3434d744 tests: test ComputeBlockVersion for all deployments (Anthony Towns)
63879f0a4760c0c0f784029849cb5d21ee088abb tests: pull ComputeBlockVersion test into its own function (Anthony Towns)

Pull request description:

  BIP9-based implementation of "speedy trial" activation specification, see https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-March/018583.html

  Edge cases are tested by fuzzing added in #21380.

ACKs for top commit:
  instagibbs:
    tACK ffe33dfbd4
  jnewbery:
    utACK ffe33dfbd4c3b11e3475b022b6c1dd077613de79
  MarcoFalke:
    review ACK ffe33dfbd4c3b11e3475b022b6c1dd077613de79 💈
  achow101:
    re-ACK ffe33dfbd4c3b11e3475b022b6c1dd077613de79
  gmaxwell:
    ACK ffe33dfbd4c3b11e3475b022b6c1dd077613de79
  benthecarman:
    ACK ffe33dfbd4c3b11e3475b022b6c1dd077613de79
  Sjors:
    ACK ffe33dfbd4c3b11e3475b022b6c1dd077613de79
  jonatack:
    Initial approach ACK ffe33dfbd4c3b11e3475b022b6c1dd077613de79 after a first pass of review, building and testing each commit, mostly looking at the changes and diffs. Will do a more high-level review iteration. A few minor comments follow to pick/choose/ignore.
  ariard:
    Code Review ACK ffe33df

Tree-SHA512: f79a7146b2450057ee92155cbbbcec12cd64334236d9239c6bd7d31b32eec145a9781c320f178da7b44ababdb8808b84d9d22a40e0851e229ba6d224e3be747c
2024-04-23 22:41:10 +07:00
W. J. van der Laan
ecade9bc39
Merge bitcoin/bitcoin#21749: test: Bump shellcheck version
08f3dbb1b0cd5ca01d87e488a2fa905adf7df057 test: Bump shellcheck version (Hennadii Stepanov)

Pull request description:

  The changelog for v0.7.2 is available [here](https://github.com/koalaman/shellcheck/blob/v0.7.2/CHANGELOG.md).

  Only [SC2268](https://github.com/koalaman/shellcheck/wiki/SC2268) requires to update our code.

ACKs for top commit:
  jarolrod:
    ACK  08f3dbb1b0cd5ca01d87e488a2fa905adf7df057

Tree-SHA512: 4585cd1f4d9def2fbaafe5a2a57761288d432781eb8c6c6d37064727d7ca8fc3f35c552e6a2ffdf0820d753d4bde2c8e43e5f3f57d242f5f57591a9b1b03558d
2024-04-23 22:41:10 +07:00
MarcoFalke
a02a2c0322
Merge bitcoin/bitcoin#21681: validation: fix ActivateSnapshot to use hardcoded nChainTx
91d93aac4e3fe6fff5ef492ed152c4d8fa6f2672 validation: remove nchaintx from assumeutxo metadata (James O'Beirne)
931684b24a89aba884cb18c13fa67ccca339ee8c validation: fix ActivateSnapshot to use hardcoded nChainTx (James O'Beirne)

Pull request description:

  This fixes an oversight from the move of nChainTx from the user-supplied
  snapshot metadata into the hardcoded assumeutxo chainparams.

  Since the nChainTx is now unused in the metadata, it should be removed
  in a future commit.

  See: https://github.com/bitcoin/bitcoin/pull/19806#discussion_r612165410

ACKs for top commit:
  Sjors:
    utACK 91d93aac4e3fe6fff5ef492ed152c4d8fa6f2672
  ryanofsky:
    Code review ACK 91d93aac4e3fe6fff5ef492ed152c4d8fa6f2672. No change to previous commit, just new commit removing now unused utxo snapshot field and updating tests.

Tree-SHA512: 445bdd738faf007451f40bbcf360dd1fb4675e17a4c96546e6818c12e33dd336dadd95cf8d4b5f8df1d6ccfbc4bf5496864bb5528e416cea894857b6b732140c
2024-04-23 22:41:09 +07:00
MarcoFalke
71f23d6e33
Merge bitcoin/bitcoin#21814: test: Fix feature_config_args.py intermittent issue
fab1eb65b196d62466fdc2ed319ffa19d3560a0c test: Fix feature_config_args.py intermittent issue (MarcoFalke)

Pull request description:

  Fix #21448

ACKs for top commit:
  laanwj:
    Code review ACK fab1eb65b196d62466fdc2ed319ffa19d3560a0c

Tree-SHA512: cad9f684f43aa801d0c1cb5f1684ffa624df1216be225cea46b1389ba2b67cbd6159ffb786fe144bf1ca865623dd9a10289d4293cfabb678bdb243d4ea00734d
2024-04-23 22:41:08 +07:00
MarcoFalke
b2d889380c
Merge bitcoin/bitcoin#21792: test: Fix intermittent issue in p2p_segwit.py
fad6269916dbf8adc14d757a18f19c74e95cf659 test: Assert that exit code indicates failure (MarcoFalke)
faecb72c3ca744f1adb77bd910c643cedec3b445 test: Fix intermittent issue in p2p_segwit.py (MarcoFalke)

Pull request description:

  Calling `start_node` might call `wait_for_rpc_connection`, which will fail.

  https://cirrus-ci.com/task/5669555591708672?logs=ci#L3504

  ```
    File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/p2p_segwit.py", line 1974, in test_upgrade_after_activation
      self.start_node(2, extra_args=["-reindex", f"-segwitheight={SEGWIT_HEIGHT}"])
    File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 508, in start_node
      node.wait_for_rpc_connection()
    File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_node.py", line 224, in wait_for_rpc_connection
      raise FailedToStartError(self._node_msg(
  test_framework.test_node.FailedToStartError: [node 2] bitcoind exited with status 1 during initialization

ACKs for top commit:
  jnewbery:
    ACK fad6269916dbf8adc14d757a18f19c74e95cf659
  dhruv:
    ACK fad6269

Tree-SHA512: 4c5e39ce25e135717ea433258518f93f09d1c528c4538a8627d3da13bc0c0ba4b45911703c26392ff0f5e0cb7831a6c7cc53e6e29102d3da9c8cfce7cef333cc
2024-04-23 22:41:07 +07:00
MarcoFalke
7bcc56c9b6
Merge bitcoin/bitcoin#22138: script: fix spelling linter raising spuriously on "invokable"
8050eb43bf15501e33ec5312918d926e47e4fc8d script: fix spelling linter raising spuriously on "invokable" (Jon Atack)

Pull request description:

  "invokable" is a valid word that means to be callable, but the linter is raising on it:
  ```
  $ test/lint/lint-spelling.sh
  contrib/guix/guix-attest:18: invokable ==> invocable
  contrib/guix/guix-clean:18: invokable ==> invocable
  contrib/guix/guix-verify:18: invokable ==> invocable
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK 8050eb43bf15501e33ec5312918d926e47e4fc8d

Tree-SHA512: 43f8dc7b7adb00ae563ccfe04a64a7ceb50237f24ff87209062bf57b2564b4d38a409df80e0183aa4f40ab306b5e07d7a5fad1600d41705bd3c443ed66a6d1c1
2024-04-23 09:53:04 -05:00
Kittywhiskers Van Gogh
9ae39f9ba7
qt: drop leftover boost::function usage in Qt, drop header from list 2024-04-23 15:34:48 +00:00
Kittywhiskers Van Gogh
c47e9e78ed
bench: drop leftover boost::lexical_cast benches, drop header from list
`boost::lexical_cast` isn't used anywhere in Dash Core, the sole remaining
use being in a benchmark, despite it no longer being used in Dash Core.
Let's drop the benchmark and drop `boost/lexical_cast.hpp` from allowed
Boost headers
2024-04-23 15:34:48 +00:00
pasta
0bb188a077
Merge #5988: test: disable ipv6 tests for now
10a006e626 test: disable ipv6 tests for now (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  Our CI nodes on Amazon have issues running tests using ipv6.

  ## What was done?
  Pretend we don't have ipv6 when we run tests for now

  ## How Has This Been Tested?
  See CI results for this PR.

  ## Breaking Changes
  n/a but we'll have ipv6 not being tested for some time.

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  kwvg:
    ACK 10a006e626
  PastaPastaPasta:
    utACK 10a006e

Tree-SHA512: 75da180b916a5a99b1363f152494cbdd1031c7daf2d5eb370ca74547dee694ad04db3e9c677f0f738b9dbb15760d685a6b43b032dde642a4c1d49fff671e2aac
2024-04-23 09:30:24 -05:00
Vijay
2996daad4c
(followup) bitcoin#19940 2024-04-23 09:15:20 -05:00
fanquake
b6dbd8bd7d
Merge bitcoin/bitcoin#22092: test: convert documentation into type annotations
68ace23fa3bc01baa734ddf2c0963acae1c75ed1 test: convert docs into type annotations in test_framework/test_node.py (fanquake)
8bfcba36db974326d258c610456bd55cf5818b1e test: convert docs into type annotations in test_framework/wallet.py (fanquake)
b043ca8e8b65199061ebe4bbed2200504dfc6ce9 test: convert docs into type annotations in test_framework/util.py (fanquake)

Pull request description:

  Rather than having function types exist as documentation, make them type annotations, which enables more `mypy` checking.

ACKs for top commit:
  instagibbs:
    utACK 68ace23fa3

Tree-SHA512: b705f26b48baabde07b9b2c0a8d547b4dcca291b16eaf5ac70462bb3a1f9e9c2783d93a9d4290889d0cbb3f7db3671446754411a1f498b265d336f6ff33478df
2024-04-23 09:15:20 -05:00
MarcoFalke
f3bc9535da
Merge bitcoin/bitcoin#22187: test: Add sync_blocks in wallet_orphanedreward.py
7a681d61b0d98a310fbb1b8e095ab8fbc5d5741c Add sync_blocks in wallet_orphanedreward.py. (Daniel Kraft)

Pull request description:

  Add an explicit `sync_blocks` call in `wallet_orphanedreward.py`, which was missing and could lead to intermittent failures of the test due to race conditions.

  This will presumably fix #22181.

ACKs for top commit:
  MarcoFalke:
    review ACK 7a681d61b0d98a310fbb1b8e095ab8fbc5d5741c

Tree-SHA512: bb226c31bf3f2e7c52beb829d7b67496e5b38781245db5f9184e3f28c93ac3aa4d21fcf5bf3055e79d384cfd0ed916e79dccb3d77486e86fe1fedb5e35f894ad
2024-04-23 09:15:19 -05:00
UdjinM6
10a006e626
test: disable ipv6 tests for now 2024-04-21 15:12:18 +03:00
MarcoFalke
e22ebca746
Merge #21712: qa: Test default include_mempool value of gettxout
44dab423eb88dbf854d22f2991e79c828ffac0f2 qa: Test default include_mempool value of gettxout (João Barbosa)

Pull request description:

  With the following diff the functional test would pass. Fix by testing the default value.

  ```diff
  --- a/src/rpc/blockchain.cpp
  +++ b/src/rpc/blockchain.cpp
  @@ -1142,7 +1142,7 @@ static RPCHelpMan gettxout()
       uint256 hash(ParseHashV(request.params[0], "txid"));
       int n = request.params[1].get_int();
       COutPoint out(hash, n);
  -    bool fMempool = true;
  +    bool fMempool = false;
       if (!request.params[2].isNull())
           fMempool = request.params[2].get_bool();
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK 44dab423eb88dbf854d22f2991e79c828ffac0f2

Tree-SHA512: 14db21b29d6b2c01d1d1278e18a0cf35d6ae566e33e45515d1fe2983dda94ad1ff6065c217601d283f9515cae39b57e981b62ac71ec2002de5359bd8a9e3efa9
2024-04-16 09:20:29 -05:00
pasta
544d33309a
Merge #5981: backport: bitcoin#19136, #21063, #21277, #21302, partial #20267 - descriptor wallets part IV
ceefab5226 fix: feature_backwards compatible works now with as expected if no bdb compiled (Konstantin Akimov)
b20f812674 fix: follow-up fixes for functional tests used protx (Konstantin Akimov)
655146d5e7 Merge #21302: wallet: createwallet examples for descriptor wallets (W. J. van der Laan)
99a8b60393 Merge #21063: wallet, rpc: update listdescriptors response format (fanquake)
6ee2c7cc59 Merge #21277: wallet: listdescriptors uses normalized descriptor form (Wladimir J. van der Laan)
8bacdbf71f Merge #19136: wallet: add parent_desc to getaddressinfo (Samuel Dobson)
f567de007a chore: release notes for 5965 with wallet tool improvements (Konstantin Akimov)
0daf360edf chore: add TODO to implement mnemonic for descriptor wallets (Konstantin Akimov)
5016294307 chore: move functional test wallet_multiwallet from category "slow 5 minutes" to "fast test" (Konstantin Akimov)
ef7ce87c1b fix: remove workarounds introduced due to missing bitcoin#20267 (bdb is not compiled) (Konstantin Akimov)
06b2d85bb4 partial Merge #20267: Disable and fix tests for when BDB is not compiled (Wladimir J. van der Laan)

Pull request description:

  ## Issue being fixed or feature implemented
  https://github.com/dashpay/dash-issues/issues/59

  ## Extra notes
  This commit `chore: move functional test wallet_multiwallet from category "slow 5 minutes" to "fast test"` is not directly connected to descriptor wallets, but added to this PR due to conflicts with 20267

  ## What was done?
  It steadily improves support of descriptor wallets in Dash core.

  Done backports and related fixes:
   - partial bitcoin/bitcoin#20267
   - bitcoin/bitcoin#19136
   - bitcoin/bitcoin#21277
   - bitcoin/bitcoin#21063
   - bitcoin/bitcoin#21302

  Beside backports and related fixes, this PR includes release notes for previous batch of backports for descriptor wallets support #5965

  ## How Has This Been Tested?
  Run unit functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

Top commit has no ACKs.

Tree-SHA512: f4b2033f8c4fa1d0f72cfc31378909703b3ae8f44748989ff00c3e71311ac80ac37837137133c7e4a166823a941ed7df10efa09c89f5b213f3c8ede7d3d6e8f4
2024-04-16 08:56:59 -05:00
pasta
92409675e6
Merge #5978: backport: merge bitcoin#21594, #21843, #22306, #22211, #22387, #21528, #22616, #22604, #22960, #23218 (networking backports: part 3)
1fedf470cd test: add type annotation for `ADDRS` in `p2p_addrv2_relay` (Kittywhiskers Van Gogh)
022b76f20b merge bitcoin#23218: Use mocktime for ping timeout (Kittywhiskers Van Gogh)
45d9e58023 merge bitcoin#22960: Set peertimeout in write_config (Kittywhiskers Van Gogh)
06e909b737 merge bitcoin#22604: address rate-limiting follow-ups (Kittywhiskers Van Gogh)
60b3e08ed1 merge bitcoin#22616: address relay fixups (Kittywhiskers Van Gogh)
8b8fbc5226 merge bitcoin#22618: Small follow-ups to 21528 (Kittywhiskers Van Gogh)
18fe765988 merge bitcoin#21528: Reduce addr blackholes (Kittywhiskers Van Gogh)
c1874c6615 net_processing: gate `m_tx_relay` access behind `!IsBlockOnlyConn()` (Kittywhiskers Van Gogh)
602d13d2a2 merge bitcoin#22387: Rate limit the processing of rumoured addresses (Kittywhiskers Van Gogh)
fe66202c05 merge bitcoin#22211: relay I2P addresses even if not reachable (by us) (Kittywhiskers Van Gogh)
7e08db55fe merge bitcoin#22306: Improvements to p2p_addr_relay.py (Kittywhiskers Van Gogh)
ff3497c18b merge bitcoin#21843: enable GetAddr, GetAddresses, and getnodeaddresses by network (Kittywhiskers Van Gogh)
51edeb082c merge bitcoin#21594: add network field to getnodeaddresses (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Dependency for https://github.com/dashpay/dash/pull/5982
  * Population of `ADDRS` in `p2p_addr`(`v2`)`_relay` in Dash is done in the test object ([source](0a62b9f985/test/functional/p2p_addrv2_relay.py (L42-L49))) as opposed to upstream, where it is done in the global state ([source](d930c7f5b0/test/functional/p2p_addrv2_relay.py (L23-L35))). This is because Dash specifically relies on `self.mocktime` instead of Bitcoin, which will work with simply sampling current time (`time.time()`).
    * [bitcoin#22211](https://github.com/bitcoin/bitcoin/pull/22211) adds changes ([source](https://github.com/bitcoin/bitcoin/pull/22211/files#diff-d3d7b1bb23f25a96c9c7444a79159ad1799895565f99efebf1618e41e886bd53R44-R46)) that add usage of `ADDRS` outside the test object. That, alongside with other considerations, resulted in [dash#5967](https://github.com/dashpay/dash/pull/5967) and a discussion ([source](https://github.com/dashpay/dash/pull/5967/files#r1548101561))
    * Eventually, following the footsteps of [dash#5967](https://github.com/dashpay/dash/pull/5967), `ADDRS` was defined outside but setup within the test object. This worked just fine ([build](https://gitlab.com/dashpay/dash/-/jobs/6594036014)) but displeased the linter ([build](https://gitlab.com/dashpay/dash/-/jobs/6594035886)) because `ADDRS` type could not be implicitly determined solely on usage in the global scope.
    * An attempt to correct this was done by realignment with upstream ([commit](262d00682c)), which pleased the linter ([build](https://gitlab.com/dashpay/dash/-/jobs/6597322521)) but broken the test ([build](https://gitlab.com/dashpay/dash/-/jobs/6597322548)) for the reasons as mentioned above.
    * Therefore, to keep the linter happy, `ADDRS` has been annotated as a `List[CAddress]` (which involved importing `List` but that's fine) ([commit](cb6d36df7d))
  * Working on [bitcoin#21528](https://github.com/bitcoin/bitcoin/pull/21528) proved challenging due to differences in Dash's and Bitcoin's approach to relaying and the workarounds used to accommodate for that.
    * Bitcoin conditionally initializes `m_tx_relay` ([source](3f7250b328/src/net.cpp (L2989-L2991))) and can always check if transaction relaying is permitted by checking if it's initialized ([source](3f7250b328/src/net_processing.cpp (L1820-L1826))).
    * Dash unconditionally initializes it ([source](0a62b9f985/src/net.h (L605-L607))). Earlier, Dash used to check if it's _appropriate_ to relay transactions by checking if it can relay addresses ([source](dc6f52ac99/src/net_processing.cpp (L2134-L2140))), which at the time, simply meant, it wasn't a block-only connection ([source](dc6f52ac99/src/net.h (L568-L572))).
    * This mutual exclusivity no longer held true in [dash#5964](https://github.com/dashpay/dash/pull/5964) and therefore, some transaction relay decisions were bound to **not** being a block-only connection ([commit](26c39f5b92)) but some were left behind, adopting `RelayAddrsWithPeer()` ([source](0a62b9f985/src/net_processing.cpp (L2215-L2221))), which, to be noted, is determined by the initialization status of `Peer::m_addr_known` ([source](0a62b9f985/src/net_processing.cpp (L839-L842))), which, so far, was pegged to **not** block-relay connection status ([source](0a62b9f985/src/net_processing.cpp (L1319))).
    * [bitcoin#21528](https://github.com/bitcoin/bitcoin/pull/21528) got rid of `RelayAddrsWithPeer()` and replaced it with `Peer::m_addr_relay_enabled` ([source](3f7250b328/src/net_processing.cpp (L237-L251))), which is setup using `Peer::SetupAddressRelay()` ([source](3f7250b328/src/net_processing.cpp (L637-L643))). This means, rather than defining the address relay status during construction, it is setup during the first address-related message (i.e. `ADDR`, `ADDRV2`, `GETADDR`) ([source](3f7250b328/src/net_processing.cpp (L227-L236))).
      * Meaning, until the first addr-related message happens, the state is has not been determined and defaults to `false`. Because some `m_tx_relay` usage still piggybacked on addr-relay permission to determine tx-relay, if a transaction message is processed before an address message is processed, there will be a false-negative condition.

        The transaction relay logic won't run since it's expecting that if transactions can be relayed, so can addresses and checks for address relaying but believes that it cannot do address relaying, borrowing that state for transaction relaying, despite address relaying permissions actually being indeterminate since it hasn't had a chance to validate its eligibility.
      * There were two approaches, run `SetupAddressRelay()` as early in the connection as possible to substitute for the "determine at construction" behaviour and change no other conditional statements... and break address-related tests _or_ move the remaining conditional transaction relay logic to use **not** block-only connection checks instead.
      * We've gone with the latter, resulting in some changes where the condition only changes form but is the same (`RelayAddrsWithPeer()` > `Peer::m_addr_relay_enabled`) ([source](109c5a9383 (diff-6875de769e90cec84d2e8a9c1b962cdbcda44d870d42e4215827e599e11e90e3L2131-L2134))) but other changes where the condition itself has been changed (`RelayAddrsWithPeer()` > `!CNode::IsBlockOnlyConn()`) ([source](109c5a9383 (diff-6875de769e90cec84d2e8a9c1b962cdbcda44d870d42e4215827e599e11e90e3R2256-R2259)))
    * This does mean that in [dash#5982](https://github.com/dashpay/dash/pull/5982), `Peer::m_block_relay_only` is introduced to be the counterpart to `Peer::m_addr_relay_enabled` ([source](45b48dae0a/src/net_processing.cpp (L321-L322))) to account for some `CConnman` logic being moved into `PeerManager` ([source](45b48dae0a/src/net_processing.cpp (L2186-L2195))), which, in a way, reverts [dash#5339](https://github.com/dashpay/dash/pull/5339) but also, doesn't, since it moves the information into `Peer` instead of reinstating it into `CNode`.
      * This was eventual since the underlying presumption that `CNode::IsAddrRelayPeer() == !CNode::IsBlockOnlyConn()` no longer holds true (also because `CNode::IsAddrRelayPeer()` doesn't exist anymore).

  Special thanks to @UdjinM6 for help with understanding Dash-specifics with respect to functional tests through help on [dash#5964](https://github.com/dashpay/dash/pull/5964) and [dash#5967](https://github.com/dashpay/dash/pull/5967)

  ## Breaking Changes

  None expected.

  RPC changes have been introduced in `getnodeaddresses`, where a new input `network`, can filter addresses based on desired network and a new output, also `network`, will associate the address with the origin network. This change is expected to be backwards-compatible.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK 1fedf470cd

Tree-SHA512: 533d33f79a0d9fd730073b3b9a58baf1dd3b0c95823e765c88a43cc974970ed3609bf1863c63ac7fc5586d1437e5250b0a2d3005468da09e407110a412bd0264
2024-04-15 10:49:14 -05:00
Kittywhiskers Van Gogh
1fedf470cd
test: add type annotation for ADDRS in p2p_addrv2_relay
Required to avoid unhappy python linter[1] result. Have to use annotation
instead of re-aligning with upstream (where ADDRS is populated in the
global state) due to reliance on `self.mocktime`, without which, the test
fails[2]

[1] - https://gitlab.com/dashpay/dash/-/jobs/6594035886
[2] - https://gitlab.com/dashpay/dash/-/jobs/6597322548
2024-04-12 16:55:07 +00:00
Kittywhiskers Van Gogh
022b76f20b
merge bitcoin#23218: Use mocktime for ping timeout 2024-04-12 16:55:07 +00:00
Kittywhiskers Van Gogh
45d9e58023
merge bitcoin#22960: Set peertimeout in write_config 2024-04-12 16:55:06 +00:00
Kittywhiskers Van Gogh
06e909b737
merge bitcoin#22604: address rate-limiting follow-ups 2024-04-12 16:55:06 +00:00
Kittywhiskers Van Gogh
8b8fbc5226
merge bitcoin#22618: Small follow-ups to 21528 2024-04-12 16:55:05 +00:00
Kittywhiskers Van Gogh
18fe765988
merge bitcoin#21528: Reduce addr blackholes 2024-04-12 16:55:05 +00:00
Kittywhiskers Van Gogh
602d13d2a2
merge bitcoin#22387: Rate limit the processing of rumoured addresses 2024-04-12 16:40:58 +00:00
Kittywhiskers Van Gogh
fe66202c05
merge bitcoin#22211: relay I2P addresses even if not reachable (by us) 2024-04-12 16:38:36 +00:00
Kittywhiskers Van Gogh
7e08db55fe
merge bitcoin#22306: Improvements to p2p_addr_relay.py 2024-04-12 16:38:35 +00:00
Kittywhiskers Van Gogh
ff3497c18b
merge bitcoin#21843: enable GetAddr, GetAddresses, and getnodeaddresses by network
continuation of cf27db8574 from dash#5491

includes:
- 6c98c09
- 3f89c0e
- ce6bca8
2024-04-12 16:38:34 +00:00
Kittywhiskers Van Gogh
51edeb082c
merge bitcoin#21594: add network field to getnodeaddresses 2024-04-12 16:37:49 +00:00
pasta
7aa8f54c0f
Merge #5976: backport: bitcoin#17934, #21338, #21390, #21445, #21602, #21606, #21609, #21676, bitcoin-core/gui#260,
21ad71c578 Merge #21676: test: Use mocktime to avoid intermittent failure in rpc_tests (MarcoFalke)
76a41eb245 Merge #21602: rpc: add additional ban time fields to listbanned (MarcoFalke)
adea52a5fe Merge bitcoin-core/gui#260: Handle exceptions instead of crash (W. J. van der Laan)
7e023c394f Merge #17934: doc: Use CONFIG_SITE variable instead of --prefix option (fanquake)
bc6e3ed6e4 Merge #21606: fuzz: Extend psbt fuzz target a bit (MarcoFalke)
233fb245f7 Merge #21445: cirrus: Use SSD cluster for speedup (fanquake)
a224b800e4 Merge #21609: ci: increase CPU count of sanitizer job to increase memory limit (MarcoFalke)
ad947099a0 test: remove exception for util::Ref which doesn't exist more (Konstantin Akimov)
6674ee85ab Merge #21390: test: Test improvements for UTXO set hash tests (MarcoFalke)
e10eec249b Merge #21338: test: add functional test for anchors.dat (MarcoFalke)
d9c31d6817 Merge #21411: test: add logging, reduce blocks, move sync_all in wallet_ groups (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Regular backports from bitcoin v22

  ## Note for reviewers:
  PRs bitcoin#17934 and bitcoin#21606 have been backported partially in past.

  ## What was done?
  Removed unused sanitizer rules (see bitcoin#21366 and dashpay/dash#5055)
   - bitcoin/bitcoin#21338
   - bitcoin/bitcoin#21390
   - bitcoin/bitcoin#21609
   - bitcoin/bitcoin#21445
   - bitcoin/bitcoin#21606
   - bitcoin/bitcoin#17934
   - bitcoin-core/gui#260
   - bitcoin/bitcoin#21602
   - bitcoin/bitcoin#21676

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 21ad71c578

Tree-SHA512: 94276d56255300d7d8c056d15b468720ba028d83cc585b16396e8bad90157e9e010490be239e09cccd4362f575f8df2d56dde3fb505745376c7790d70e3635cd
2024-04-12 10:30:27 -05:00
pasta
54ea9260d4
Merge #5974: backport: bitcoin#19522, #19809, #20993, #21075, #21126, #21138, #21221, #21354, #21542
005a6b104a fix: format string in llmq/commitment - mismatched arguments (Konstantin Akimov)
4774e1e8f6 Merge #19809: log: Prefix log messages with function name and source code location if -logsourcelocations is set (Wladimir J. van der Laan)
43a94f0580 fix: adjust functional tests due to dash's support of thread name after v0.12 (Konstantin Akimov)
085120d9f9 Merge #20993: test: store subversion (user agent) as string in msg_version (MarcoFalke)
e866b43160 Merge #21542: ci: Bump macOS VM image to the latest version (fanquake)
a3702534e5 Merge #21354: build, doc: Drop no longer required packages from macOS cross-compiling dependencies (fanquake)
6bcc86ad3b Merge #21221: [tools] Allow argument/parameter bin packing in clang-format (MarcoFalke)
318c7263d0 Merge #19522: build: fix building libconsensus with reduced exports for Darwin targets (Wladimir J. van der Laan)
88a45d4a9a Merge #21138: ci: Re-run wine tests once if they fail (fanquake)
4abb768456 Merge #21126: ci: Properly bump to focal for win cross build (fanquake)
f254f77d75 Merge #21075: doc: Fix markdown formatting (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Regular backports from bitcoin v22 and related fixes

  ## What was done?
  Follow-up fixes for  bitcoin#19809
  Backports:
   - bitcoin/bitcoin#21075
   - bitcoin/bitcoin#21126
   - bitcoin/bitcoin#21138
   - bitcoin/bitcoin#19522
   - bitcoin/bitcoin#21221
   - bitcoin/bitcoin#21354
   - bitcoin/bitcoin#21542
   - bitcoin/bitcoin#19809
   - bitcoin/bitcoin#20993

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  N/A
  Notice, that function name is included now by default to logs with `-logfunctionnames` and many logs have function name twice now such as:
  ```
   node0 2024-04-06T20:13:56.564123Z (mocktime: 2014-12-04T17:15:38Z) [httpworker.3] [masternode/sync.cpp:331] [NotifyHeaderTip] CMasternodeSync::NotifyHeaderTip -- pindexNew->nHeight: 5 fInitialDownload=0
  ```
  For further development need to take it in account and do not use more direct calls `__func__` from code as well as reduce usages in codebase.

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 005a6b104a

Tree-SHA512: f98949c4605dda7d6dfe790554e1d31a8c8178b57520578fcd58178c68fe7af33c0d66a79865683c1357de9a73fa4f484eb828b28e11ca110b8e1915c1fcf9b3
2024-04-12 10:20:15 -05:00
pasta
938fd23447
Merge #5973: backport: bitcoin#18772, #20690, #20789, #20813, #21531, bitcoin-core/gui#13, #72, 115, #139, #171
a2f190deff Merge bitcoin-core/gui#115: Replace "Hide tray icon" option with positive "Show tray icon" one (Jonas Schnelli)
65b80e76b7 Merge #21531: test: remove qt byteswap compattests (MarcoFalke)
ba883c5da2 Merge bitcoin-core/gui#139: doc: Improve gui/src/qt README.md (MarcoFalke)
368c65d050 Merge bitcoin-core/gui#72: util: Log static plugins meta data and used style (Jonas Schnelli)
317777e7d7 Merge bitcoin-core/gui#171: Use layout manager for Create Wallet dialog (MarcoFalke)
83313a5603 Merge bitcoin#20789: Rework strong and weak net enum fuzzing (MarcoFalke)
4a3e3af6e7 Merge #20813: scripted-diff: Bump copyright headers (MarcoFalke)
e36eacd868 Merge #18772: rpc: calculate fees in getblock using BlockUndo data (MarcoFalke)
41a1e10954 Merge #20690: Clean up logging of outbound connection type (MarcoFalke)
648d6f04fb Merge bitcoin-core/gui#13: Hide peer detail view if multiple are selected (Jonas Schnelli)

Pull request description:

  ## Issue being fixed or feature implemented
  Regular backports from bitcoin v22

  ## What was done?
   - bitcoin-core/gui#13
   - bitcoin-core/gui#115
   - bitcoin/bitcoin#20690
   - bitcoin/bitcoin#18772
   - bitcoin/bitcoin#20813
   - bitcoin/bitcoin#20789
   - bitcoin-core/gui#171
   - bitcoin-core/gui#72
   - bitcoin-core/gui#139
   - bitcoin/bitcoin#21531

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK a2f190deff

Tree-SHA512: 29421e7ca38583c47f49c2605775f34b64ae2fb6aeb45ac42941fbc78598fc26e7f7a248b40fcc2c9fd21154b0a6f2aed64287a8b7cca43de1b99ae3dccd990f
2024-04-12 10:16:32 -05:00
Konstantin Akimov
ceefab5226
fix: feature_backwards compatible works now with as expected if no bdb compiled
It is follow-up fixes for bitcoin#20267
2024-04-12 17:34:03 +07:00
Konstantin Akimov
b20f812674
fix: follow-up fixes for functional tests used protx
Since bitcoin#20267 changes default wallet in functional tests from legacy
wallets to descriptor wallets, we need to enforce --legacy-wallets for
functional tests that used protx which doesn't work yet for descriptor wallets
2024-04-12 17:31:57 +07:00
fanquake
99a8b60393
Merge #21063: wallet, rpc: update listdescriptors response format
2e5f7def22e1b212fbd69e9147145d9d1f408aaf wallet, rpc: update listdescriptors response format (Ivan Metlushko)

Pull request description:

  Update `listdescriptors` response format according to [RPC interface guidelines](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#rpc-interface-guidelines).

  This is a follow up for #20226

  **Before:**
  ```
  Result:
  [                               (json array) Response is an array of descriptor objects
    {                             (json object)
      "desc" : "str",             (string) Descriptor string representation
      "timestamp" : n,            (numeric) The creation time of the descriptor
      "active" : true|false,      (boolean) Activeness flag
      "internal" : true|false,    (boolean, optional) Whether this is internal or external descriptor; defined only for active descriptors
      "range" : [                 (json array, optional) Defined only for ranged descriptors
        n,                        (numeric) Range start inclusive
        n                         (numeric) Range end inclusive
      ],
      "next" : n                  (numeric, optional) The next index to generate addresses from; defined only for ranged descriptors
    },
    ...
  ]
  ```

  **After:**
  ```
  Result:
  {                                 (json object)
    "wallet_name" : "str",          (string) Name of wallet this operation was performed on
    "descriptors" : [               (json array) Array of descriptor objects
      {                             (json object)
        "desc" : "str",             (string) Descriptor string representation
        "timestamp" : n,            (numeric) The creation time of the descriptor
        "active" : true|false,      (boolean) Activeness flag
        "internal" : true|false,    (boolean, optional) Whether this is internal or external descriptor; defined only for active descriptors
        "range" : [                 (json array, optional) Defined only for ranged descriptors
          n,                        (numeric) Range start inclusive
          n                         (numeric) Range end inclusive
        ],
        "next" : n                  (numeric, optional) The next index to generate addresses from; defined only for ranged descriptors
      },
      ...
    ]
  }
  ```

ACKs for top commit:
  achow101:
    re-ACK 2e5f7def22e1b212fbd69e9147145d9d1f408aaf
  meshcollider:
    utACK 2e5f7def22e1b212fbd69e9147145d9d1f408aaf
  jonatack:
    re-ACK 2e5f7def22e1b212fbd69e9147145d9d1f408aaf

Tree-SHA512: 49bf73e46e2a61003ce594a4bfc506eb9592ccb799c2909c43a1a527490a4b4009f78dc09f3d47b4e945d3d7bb3cd2632cf48c5ace5feed5066158cc010dddc1
2024-04-12 12:31:49 +07:00
Wladimir J. van der Laan
6ee2c7cc59
Merge #21277: wallet: listdescriptors uses normalized descriptor form
a69c3b35f8974b378a87a3e42d331bd4147e07df wallet: listdescriptors uses normalized descriptor form (Ivan Metlushko)

Pull request description:

  Rationale: show importable descriptors with `listdescriptors` RPC

  It uses #19136 to derive xpub at the last hardened step.

  **Before**:
  ```
  [
      {
        "desc": "wpkh(tpubD6NzVbkrYhZ4YUQRJL49TWw1VR5v3QKUNYaGGMUfJUm19x5ZqQ2hEiPiYbAQvD2nHoPGQGPg3snLPM8sjmYpvx7XQhkmyfk8xhsUwKbXzyh/84'/1'/0'/0/*)#p4cn3erf",
        "timestamp": 1613982591,
        "active": true,
        "internal": false,
        "range": [
          0,
          999
        ],
        "next": 0
      },
      ...
  ]
  ```

  **After**:
  ```
  [
    {
      "desc": "wpkh([d4ade89c/84'/1'/0']tpubDDUEYcVXy6Vh5meHvcXN3sAr4k3fWwLZGpAHbkAHL8EnkDxp4d99CjNhJHfM2fUJicANvAKnCZS6XaVAgwAeKYc1KesGCN5qbQ25qQHrRxM/0/*)#8wq8rcft",
      "timestamp": 1613982591,
      "active": true,
      "internal": false,
      "range": [
        0,
        999
      ],
      "next": 0
    },
    ...
  ]
  ```

ACKs for top commit:
  achow101:
    ACK a69c3b35f8974b378a87a3e42d331bd4147e07df

Tree-SHA512: 4f92e726cb8245aa0b520729cfd272945f0c66830f0555544e0618883aca516318543fa6ab1862058c64b4e4ed54ad1da953e032f4846eef7454122031c1b005
2024-04-12 12:31:49 +07:00
Samuel Dobson
8bacdbf71f
Merge #19136: wallet: add parent_desc to getaddressinfo
de6b389d5db7b8426313c5be6fbd290f992c5aa8 tests: Test getaddressinfo parent_desc (Andrew Chow)
e4ac869a0a0083e2e3af3b56301bd5c8e0cf650b rpc: Add parent descriptor to getaddressinfo output (Andrew Chow)
bbe4a36152fb8d9c8c3682ca2380f1c88cca61cb wallet: Add GetDescriptorString to DescriptorScriptPubKeyMan (Andrew Chow)
9be1437c49f986e8ed964d5f863b4bbcec340751 descriptors: Add ToNormalizedString and tests (Andrew Chow)

Pull request description:

  Adds `parent_desc` field to the `getaddressinfo` RPC to export a public descriptor. Using the given address, `getaddressinfo` will look up which `DescriptorScriptPubKeyMan` can be used to produce that address. It will then return the descriptor for that `DescriptorScriptPubKeyMan` in the `parent_desc` field. The descriptor will be in a normalized form where the xpub at the last hardened step is derived so that the descriptor can be imported to other wallets. Tests are added to check that the correct descriptor is being returned for the wallet's addresses and that these descriptors can be imported and used in other wallets.

  As part of this PR, a `ToNormalizedString` function is added to the descriptor classes. This really only has an effect on `BIP32PubkeyProvider`s that have hardened derivation steps. Tests are added to check that normalized descriptors are returned.

ACKs for top commit:
  Sjors:
    utACK de6b389d5db7b8426313c5be6fbd290f992c5aa8
  S3RK:
    Tested ACK de6b389
  jonatack:
    Tested ACK de6b389d5db7b8426313c5be6fbd290f992c5aa8 modulo a few minor comments
  fjahr:
    Code review ACK de6b389d5db7b8426313c5be6fbd290f992c5aa8
  meshcollider:
    Tested ACK de6b389d5db7b8426313c5be6fbd290f992c5aa8

Tree-SHA512: a633e4a39f2abbd95afd7488484cfa66fdd2651dac59fe59f2b80a0940a2a4a13acf889c534a6948903d701484a2ba1218e3081feafe0b9a720dccfa9e43ca2b
2024-04-12 12:31:49 +07:00
Konstantin Akimov
0daf360edf
chore: add TODO to implement mnemonic for descriptor wallets 2024-04-11 02:37:04 +07:00
Konstantin Akimov
5016294307
chore: move functional test wallet_multiwallet from category "slow 5 minutes" to "fast test" 2024-04-11 02:37:03 +07:00
Konstantin Akimov
ef7ce87c1b
fix: remove workarounds introduced due to missing bitcoin#20267 (bdb is not compiled)
This partially reverts commit da8e5639ee.
Also it adds missing changed from bitcoin#16404
2024-04-11 02:37:03 +07:00
Wladimir J. van der Laan
06b2d85bb4
partial Merge #20267: Disable and fix tests for when BDB is not compiled
Backport notice: changes in feature_notification.py are missing due to #18878 is not done yet

49797c3ccfbb9f7ac9c1fbb574d35b315c103805 tests: Disable bdb dump test when no bdb (Andrew Chow)
1194cf9269e6c4bf67b09c4766f42bf173d12c0a Fix wallet_send.py wallet setup to work with descriptors (Andrew Chow)
fbaea7bfe44822710a36601c6b0febbd5e33dfbd Require legacy wallet for wallet_upgradewallet.py (Andrew Chow)
b1b679e0ab7a9981e3e78424fe8836edd59abf6f Explicitly mark legacy wallet tests as such (Andrew Chow)
09514e1bef46444a67cde9ff13e76bd4b9f8c7ac Setup wallets for interface_zmq.py (Andrew Chow)
4d03ef9a73ceb5ef4e9d184a135bca6bdeb8c311 Use MiniWallet in rpc_net.py (Andrew Chow)
4de23824b0c711ece68f9fc007ffac12126710aa Setup wallets for interface_bitcoin_cli.py (Andrew Chow)
7c71c627d28f0cddaf2349a55336278a681c27c2 Setup wallets with descriptors for feature_notifications (Andrew Chow)
1f1bef8dbab7225884d769a45477ee11d0ebf654 Have feature_filelock.py test both bdb and sqlite, depending on compiled (Andrew Chow)
c77975abc0123b29b0eb3481b8916e7c025b7c4c Disable upgrades tests that require BDB if BDB is not compiled (Andrew Chow)
1f20cac9d41e507901a2811d6db7147d7ab0321b Disable wallet_descriptor.py bdb format check if BDB is not compiled (Andrew Chow)
3641597d7ef6f5097a9e93cab3ef7e0f9c820296 tests: Don't make any wallets unless wallet is required (Andrew Chow)
b9b88f57a9b9a28e0f0614c12ae3012cf5050b10 Skip legacy wallet reliant tests if BDB is not compiled (Andrew Chow)
6f36242389bd3e7eacf594ce90491e8ccca70f3a tests: Set descriptors default based on compilation (Andrew Chow)

Pull request description:

  This PR fixes tests for when BDB is not compiled. Tests which rely on or test legacy wallet behavior are disabled and skipped when BDB is not compiled. For the components of some tests that are for legacy wallet things, those parts of the tests are skipped.

  For the majority of tests, changes are made so that they can be run with either legacy wallets or descriptor wallets without materially effecting the test. Most tests only need the wallet for balance and transactions, so the type of wallet is not an important part of those tests. Additionally, some tests are wallet agnostic and modified to instead use the test framework's MiniWallet.

ACKs for top commit:
  laanwj:
    ACK 49797c3ccfbb9f7ac9c1fbb574d35b315c103805
  ryanofsky:
    Code review ACK 49797c3ccfbb9f7ac9c1fbb574d35b315c103805. Only change since last review is dropping last commit. Previous review w/ suggestions for future followup is https://github.com/bitcoin/bitcoin/pull/20267#pullrequestreview-581508843

Tree-SHA512: 69659f8a81fb437ecbca962f4082c12835282dbf1fba7d9952f727a49e01981d749af9b09feda1c8ca737516c7d7a08ef17e782795df3fa69892d5021b41c1ed
2024-04-11 02:37:03 +07:00
Konstantin Akimov
ad947099a0
test: remove exception for util::Ref which doesn't exist more
This PR is follow-up for dash#5055 and based on bitcoin#21366 which is DNM
2024-04-11 02:26:02 +07:00
MarcoFalke
6674ee85ab
Merge #21390: test: Test improvements for UTXO set hash tests
4f2653a89018fa4d24bd2a551832a7410b682600 test: Use deterministic chain in utxo set hash test (Fabian Jahr)
4973c5175c5fd1f4791ea26e8ddefd6fb11ac1c3 test: Remove wallet dependency of utxo set hash test (Fabian Jahr)
1a27af1d7b5ec18b4248ead1eaf0f381047b4b24 rpc: Improve gettxoutsetinfo help (Fabian Jahr)

Pull request description:

  Follow-ups to #19145:
  - Small improvement on the help text of RPC gettxoutsetinfo
  - Using deterministic blockchain in the test `functional/feature_utxo_set_hash.py`
  - Removing wallet dependency in the test `functional/feature_utxo_set_hash.py`

  Split out of #19521.

ACKs for top commit:
  MarcoFalke:
    review ACK 4f2653a89018fa4d24bd2a551832a7410b682600 👲

Tree-SHA512: 92927b3aa22b6324eb4fc9d346755313dec44d973aa69a0ebf80a8569b5f3a7cf3539721ebdba183737534b9e29b3e33f412515890f0d0b819878032a3bba8f9
2024-04-11 02:26:01 +07:00
MarcoFalke
e10eec249b
Merge #21338: test: add functional test for anchors.dat
581791c62067403fbeb4e1fd88c1d80549627c94 test: add functional test for anchors.dat (bruno)

Pull request description:

  This PR adds a functional test for anchors.dat.

  It creates a node and adds 2 outbound block-relay-only connections and 5 inbound connections.
  When the node is down, anchors.dat should contain the 2 addresses from the outbound block-relay-only connections.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 581791c62067403fbeb4e1fd88c1d80549627c94
  hebasto:
    ACK 581791c62067403fbeb4e1fd88c1d80549627c94

Tree-SHA512: 77038b09e36ee5ae473a26d6f566c0ed283af258c34df8486706a24f72b05abab621a293ac886d03849bc45bc28be7336137252225b25aff393baa6b5238688c
2024-04-11 02:26:01 +07:00
MarcoFalke
d9c31d6817
Merge #21411: test: add logging, reduce blocks, move sync_all in wallet_ groups
c62f9bc0e931f65eef63041d2c53f9a294c0e8d6 test: use fewer blocks in wallet_groups and move sync call (Jon Atack)
3a16b5ef95c1c25f8b78e591f985e80b41a6dbdd test: add missing logging to wallet_groups.py (Jon Atack)

Pull request description:

  - add logging (particularly useful as the tests are somewhat slow)
  - generate 101 blocks instead of 110
  - move `sync_all` call into the loop, so fewer blocks are synced on each call, to hopefully see fewer CI timeouts as in https://bitcoinbuilds.org/index.php?ansilog=88eee99e-1727-44ed-b778-3b9c75c33928.log

  ```
  L2742     File "/home/ubuntu/src/test/functional/wallet_groups.py", line 162, in run_test
  L2743       self.sync_all()
  test_framework.authproxy.JSONRPCException: 'syncwithvalidationinterfacequeue' RPC took longer than 960.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK c62f9bc0e931f65eef63041d2c53f9a294c0e8d6

Tree-SHA512: 711deafcd589cb8196cb207ff882e0f2ab6b70828a6abad91f83f535974cc430a56b9e8a960fd233d31d610932a0d48b49ee681aae564d145a3040288ecda8f8
2024-04-11 02:26:01 +07:00
Wladimir J. van der Laan
4774e1e8f6
Merge #19809: log: Prefix log messages with function name and source code location if -logsourcelocations is set
b4511e2e2ed1a6077ae6826a9ee6b7a311293d08 log: Prefix log messages with function name if -logsourcelocations is set (practicalswift)

Pull request description:

  Prefix log messages with function name if `-logfunctionnames` is set.

  Yes, exactly like `-logthreadnames` but for function names instead of thread names :)

  This is a small developer ergonomics improvement: I've found this to be a cheap/simple way to correlate log output and originating function.

  For me it beats the ordinary cycle of 1.) try to figure out a regexp matching the static part of the dynamic log message, 2.) `git grep -E 'Using .* MiB out of .* requested for signature cache'`, 3.) `mcedit filename.cpp` (`openemacs filename.cpp` works too!) and 4.) search for log message and scroll up to find the function name :)

  Without any logging parameters:

  ```
  $ src/bitcoind -regtest
  2020-08-25T03:29:04Z Using RdRand as an additional entropy source
  2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
  2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
  2020-08-25T03:29:04Z Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000
  2020-08-25T03:29:04Z block tree size = 1
  2020-08-25T03:29:04Z nBestHeight = 0
  2020-08-25T03:29:04Z Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
  2020-08-25T03:29:04Z 0 addresses found from DNS seeds
  ```

  With `-logthreadnames` and `-logfunctionnames`:

  ```
  $ src/bitcoind -regtest -logthreadnames -logfunctionnames
  2020-08-25T03:29:04Z [init] [ReportHardwareRand] Using RdRand as an additional entropy source
  2020-08-25T03:29:04Z [init] [InitSignatureCache] Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
  2020-08-25T03:29:04Z [init] [InitScriptExecutionCache] Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
  2020-08-25T03:29:04Z [init] [LoadChainTip] Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000
  2020-08-25T03:29:04Z [init] [AppInitMain] block tree size = 1
  2020-08-25T03:29:04Z [init] [AppInitMain] nBestHeight = 0
  2020-08-25T03:29:04Z [loadblk] [LoadMempool] Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
  2020-08-25T03:29:04Z [dnsseed] [ThreadDNSAddressSeed] 0 addresses found from DNS seeds
  ```

ACKs for top commit:
  laanwj:
    Code review ACK b4511e2e2ed1a6077ae6826a9ee6b7a311293d08
  MarcoFalke:
    review ACK b4511e2e2ed1a6077ae6826a9ee6b7a311293d08 🌃

Tree-SHA512: d100f5364630c323f31d275259864c597f7725e462d5f4bdedcc7033ea616d7fc0d16ef1b2af557e692f4deea73c6773ccfc681589e7bf6ba970b9ec169040c7
2024-04-11 02:25:08 +07:00
Konstantin Akimov
43a94f0580
fix: adjust functional tests due to dash's support of thread name after v0.12 2024-04-11 02:25:08 +07:00
MarcoFalke
085120d9f9
Merge #20993: test: store subversion (user agent) as string in msg_version
de85af5cce727981383ac0fe81f635451b331f23 test: store subversion (user agent) as string in msg_version (Sebastian Falbesoner)

Pull request description:

  It seems more natural to treat the "subversion" field (=user agent string, see [BIP 14](https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki#Proposal)) of a node as pure string rather than a bytestring within the test framework. This is also suggested with the naming prefix in `msg_version.strSubVer`: one probably wouldn't expect a field starting with "str" to be a bytestring that needs further decoding to be useful. This PR moves the encoding/decoding parts to the serialization/deserialization routines so that the user doesn't have to bother with that anymore.

  Note that currently, in the master branch the `msg_version.strSubVer` is never read (only in `msg_version.__repr__`); However, one issue that is solved by this PR came up while testing #19509 (not merged yet): A decoding script for binary message capture files takes use of the functional test framework convert it into JSON format. Bytestrings will be convered to hexstrings, while pure strings will (surprise surprise) end up without modification in the file.

  So without this patch, we get:

  ```
  $ jq . out.json | grep -m5 strSubVer
        "strSubVer": "2f5361746f7368693a32312e39392e302f"
        "strSubVer": "2f5361746f7368693a302e32302e312f"
        "strSubVer": "2f5361746f7368693a32312e39392e302f"
        "strSubVer": "2f5361746f7368693a302e32302e312f"
        "strSubVer": "2f5361746f7368693a32312e39392e302f"
  ```

  After this patch:

  ```
  $ jq . out2.json | grep -m5 strSubVer
        "strSubVer": "/Satoshi:21.99.0/"
        "strSubVer": "/Satoshi:0.20.1/"
        "strSubVer": "/Satoshi:21.99.0/"
        "strSubVer": "/Satoshi:0.20.1/"
        "strSubVer": "/Satoshi:21.99.0/"
  ```

ACKs for top commit:
  jnewbery:
    utACK de85af5cce727981383ac0fe81f635451b331f23

Tree-SHA512: ff23642705c858e8387a625537dfec82e6b8a15da6d99b8d12152560e52d243ba17431b602b26f60996d897e00e3f37dcf8dc8a303ffb1d544df29a5937080f9
2024-04-11 02:25:08 +07:00
MarcoFalke
4a3e3af6e7
Merge #20813: scripted-diff: Bump copyright headers
fa0074e2d82928016a43ca408717154a1c70a4db scripted-diff: Bump copyright headers (MarcoFalke)

Pull request description:

  Needs to be done because no one has removed the years yet

ACKs for top commit:
  practicalswift:
    ACK fa0074e2d82928016a43ca408717154a1c70a4db

Tree-SHA512: 210e92acd7d400b556cf8259c3ec9967797420cfd19f0c2a4fa54cb2b3d32ad9ae27e771269201e7d554c0f4cd73a8b1c1a42c9f65d8685ca4d52e5134b071a3
2024-04-10 03:19:34 +07:00
Konstantin Akimov
31ffb78ced
feat: add option -usehd to wallettool to let create non-hd wallets 2024-04-10 01:59:00 +07:00
MarcoFalke
3f4b42caa4
Merge #20687: wallet: Add missing check for -descriptors wallet tool option
fae32f295cc5b57c1cb95090bb60cddb42f9778a wallet: Add missing check for -descriptors wallet tool option (MarcoFalke)
faf8f61368696b9cbbea55ead30d6a48203235ff test: Add missing check for is_sqlite_compiled (MarcoFalke)
fa7dde1c418e2e700853bd30cc9e012c4e4c5ef2 wallet: Pass ArgsManager into ExecuteWalletToolFunc instead of using global (MarcoFalke)

Pull request description:

  Also, fix a test failure when compiled without sqlite

ACKs for top commit:
  ryanofsky:
    Code review ACK fae32f295cc5b57c1cb95090bb60cddb42f9778a. Thanks for implementing the -descriptors check and dealing with the test failure!
  jonatack:
    Code review utACK fae32f295cc5b57c1cb95090bb60cddb42f9778a

Tree-SHA512: 3d7710694085822739a8316e4abc6db270799ca6ff6b0f9e5563ae240da65ae6a9cab7ba2647feae6ba540dac40b55b38ed41c8f6ed0bf02a3d1536284448927
2024-04-10 01:59:00 +07:00
MarcoFalke
2978c452cd
Merge #19137: wallettool: Add dump and createfromdump commands
23cac24dd3f2aaf88aab978e7ef4905772815cd2 tests: Test bitcoin-wallet dump and createfromdump (Andrew Chow)
a88c320041bd1cd1786b2dfd9ab698a67c2a57c6 wallettool: Add createfromdump command (Andrew Chow)
e1e7a90d5f0616a46ffadd62a9f1c65406cca6b4 wallettool: Add dump command (Andrew Chow)

Pull request description:

  Adds two commands to the `bitcoin-wallet` tool: `dump` and `createfromdump`. These commands will be useful for a wallet storage migration in the future. It is also generally useful to have a storage agnostic dump like this. These commands are similar to BDB's `db_dump` and `db_load` tools. This can also be useful for manual construction of a wallet file for tests.

  `dump` outputs every key-value pair from the wallet as comma separated hex. Each key-value pair is on its own line with the key and value in hex separated by a comma. This is output to the file specified by the new `-dumpfile` option.

  `createfromdump` takes a file produced by `dump` and creates a new wallet file with exactly the records specified in that file.

  A new option `-dumpfile` is added to the wallet tool. When used with `dump`, the records will be written to the specified file. When used with `createfromdump`, the file is read and the key-value pairs constructed from it. `createfromdump` requires `-dumpfile`.

  A simple round-trip test is added to the `tool_wallet.py`.

  This PR is based on #19334,

ACKs for top commit:
  Sjors:
    re-utACK 23cac24
  MarcoFalke:
    re review ACK 23cac24dd3f2aaf88aab978e7ef4905772815cd2 only change is rebase and removing useless shared_ptr wrapper 🎼
  ryanofsky:
    Code review ACK 23cac24dd3f2aaf88aab978e7ef4905772815cd2. Only changes since last review rebase and changing a pointer to a reference

Tree-SHA512: 2d63cf62baca3d16495aa698dc02f7d889c81b41015e9c92c23c275bb4a690fc176d351c3fd7f310bd6b17f5a936cc9be694cbecd702af741b96c0f530e72fa2
2024-04-10 01:58:59 +07:00
MarcoFalke
99dec80fbb
Merge #19253: Tests: tidy up address.py and segwit_addr.py
825fcae484f31182041dfacbf820e818d759b130 [tests] Replace bytes literals with hex literals (John Newbery)
64eca45100536579a3849631e59d4277bbc25be1 [tests] Fix pep8 style violations in address.py (John Newbery)
b230f8b3f3adcb1e2ae299094f9ae0a8bc7cc3d0 [tests] Correct docstring for address.py (John Newbery)
ea70e6a2ca0e183ef40cdb9b3b86f39e94366015 [tests] Tidy up imports in address.py (John Newbery)
7f639df0b8a15aaeccedab00b634925f568c2c9a [tests] Remove unused optional verify_checksum parameter (John Newbery)
011e784f74411bd5d5dbccfd3af39e0937fd8933 [tests] Rename segwit encode and decode functions (John Newbery)
e4557133f595f357df5e16ae4f2f19c579631396 [tests] Move bech32 unit tests to test framework (John Newbery)

Pull request description:

  Lots of small fixes:

  - moving unit tests to test_framework implementation files
  - renaming functions to be clearer
  - removing multiple imports
  - removing unreadable byte literals from the code
  - fixing pep8 violations
  - correcting out-of-date docstring

ACKs for top commit:
  jonatack:
    re-ACK 825fcae484f31182041dfacbf820e818d759b130 per `git range-diff a0a422c 7edcdcd 825fcae` and verified `wallet_address_types.py` and `wallet_basic.py --descriptors` (the failure on one travis job) are green locally.
  MarcoFalke:
    ACK 825fcae484f31182041dfacbf820e818d759b130
  fanquake:
    ACK 825fcae484f31182041dfacbf820e818d759b130 - looks ok to me.

Tree-SHA512: aea509c27c1bcb94bef11205b6a79836c39c62249672815efc9822f411bc2e2336ceb3d72b3b861c3f4054a08e16edb28c6edd3aa5eff72eec1d60ea6ca82dc4
2024-04-10 01:58:59 +07:00
MarcoFalke
25248f9cb7
Merge #20365: wallettool: add parameter to create descriptors wallet
173cc9b7be335b5dd2cc1bb112dfa6ec5c13ec12 test: walettool create descriptors (Ivan Metlushko)
345e88eecf1b28607d5da3af38e19794a8a115ce wallettool: add param to create descriptors wallet (Ivan Metlushko)
6d3af3ab627096a824cb6a7ca1ebeddc7530361c wallettool: pass in DatabaseOptions into MakeWallet (Ivan Metlushko)

Pull request description:

  Rationale: expose and promote descriptor wallets in more places; make cli tool more consistent with `createwallet` rpc.

  Add `-descriptors` parameter which is off by default. When specified it will create a new descriptors wallet with sqlite backend, which is consistent with `createwallet` rpc.

  This PR is based on a suggestion from **ryanofsky** https://github.com/bitcoin/bitcoin/pull/19137#discussion_r516779603

  Example:
  ```
  $ ./src/bitcoin-wallet  -wallet=fewty -descriptors create
  Topping up keypool...
  Wallet info
  ===========
  Name: fewty
  Format: sqlite
  Descriptors: yes
  Encrypted: no
  HD (hd seed available): yes
  Keypool Size: 6000
  Transactions: 0
  Address Book: 0
  ```
  ```
  $ ./src/bitcoin-wallet  -wallet=fewty create
  Topping up keypool...
  Wallet info
  ===========
  Name: fewty
  Format: bdb
  Descriptors: no
  Encrypted: no
  HD (hd seed available): yes
  Keypool Size: 2000
  Transactions: 0
  Address Book: 0
  ```

ACKs for top commit:
  achow101:
    ACK 173cc9b7be335b5dd2cc1bb112dfa6ec5c13ec12
  ryanofsky:
    Code review ACK 173cc9b7be335b5dd2cc1bb112dfa6ec5c13ec12. This seems pretty nicely implemented now, with opportunities to clean up more and dedup later
  MarcoFalke:
    Concept ACK 173cc9b7be335b5dd2cc1bb112dfa6ec5c13ec12 🌠

Tree-SHA512: cc32ba336ff709de2707ee15f495b4617908e8700ede8401a58e894f44cda485c544d644023c9a6604d88a62db9d92152383ee2e8abf691688c25cf6e222c622
2024-04-10 01:58:58 +07:00
Konstantin Akimov
31040abae6
fix: isHDenabled is true only when chain is generated; before that's always false 2024-04-10 01:58:57 +07:00
MarcoFalke
e36eacd868
Merge #18772: rpc: calculate fees in getblock using BlockUndo data
66d012ad7f9381bacfc9b8388fa2ebf82cb86c9e test: RPC: getblock fee calculations (Elliott Jin)
bf7d6e31b1062ab5f90e14e83c56309f499fa2e9 RPC: getblock: tx fee calculation for verbosity 2 via Undo data (Elliott Jin)

Pull request description:

  This change is progress towards #18771 .  It adapts the fee calculation part of #16083 and addresses some feedback.  The additional "verbosity level 3" features are planned for a future PR.

  **Original PR description:**

  > Using block undo data (like in #14802) we can now show fee information for each transaction in a block without the need for additional -txindex and/or a ton of costly lookups. For a start we'll add transaction fee information to getblock verbosity level 2. This comes at a negligible speed penalty (<1%).

ACKs for top commit:
  luke-jr:
    tACK 66d012ad7f9381bacfc9b8388fa2ebf82cb86c9e
  fjahr:
    tACK 66d012ad7f9381bacfc9b8388fa2ebf82cb86c9e
  MarcoFalke:
    review ACK 66d012ad7f9381bacfc9b8388fa2ebf82cb86c9e 🗜

Tree-SHA512: be1fe4b866946a8dc36427f7dc72a20e10860e320a28fa49bc85bd2a93a0d699768179be29fa52e18b2ed8505d3ec272e586753ef2239b4230e0aefd233acaa2
2024-04-09 22:34:38 +07:00
Kittywhiskers Van Gogh
03ab144b8f
merge bitcoin#21785: Fix intermittent issue in p2p_addr_relay.py 2024-04-03 16:10:16 +00:00
Kittywhiskers Van Gogh
6d27db58d1
merge bitcoin#21707: Extend functional tests for addr relay 2024-04-03 16:10:16 +00:00
Kittywhiskers Van Gogh
1d4f10a378
merge bitcoin#19315: Allow outbound & block-relay-only connections in functional tests 2024-04-03 16:06:40 +00:00
Kittywhiskers Van Gogh
017d1b40e3
merge bitcoin#19763: don't relay to the address' originator 2024-04-03 16:05:30 +00:00
Wladimir J. van der Laan
1995d2e7ca
Merge #20451: lint: run mypy over contrib/devtools
1ef2138c0db3bd4f9332c777fa3fb2770dc1b08c lint: run mypy over contrib/devtools (fanquake)

Pull request description:

  wumpus mentioned on IRC that we don't currently run `mypy` over the `contrib/devtools` directory, and that it would likely be worthwhile given #20434. This just adds that dir to the linter, as well as some missing annotations to fix existing errors. Note that now we require Python 3.6 we can make use of variable annotations.

  master (patched to check contrib devtools):
  ```bash
  test/lint/lint-python.sh
  contrib/devtools/symbol-check.py:154: error: Incompatible types in assignment (expression has type "List[str]", variable has type "str")
  contrib/devtools/circular-dependencies.py:35: error: Need type annotation for 'deps' (hint: "deps: Dict[<type>, <type>] = ...")
  contrib/devtools/circular-dependencies.py:67: error: Need type annotation for 'closure' (hint: "closure: Dict[<type>, <type>] = ...")
  Found 4 errors in 3 files (checked 187 source files)
  ```

  I haven't quite gone as far as to add annotations like
  ```python
  CHECKS: Dict[str, List[Tuple[str, Callable[[Any], bool]]]] = {...
  ```
  to `symbol-check.py`.

ACKs for top commit:
  laanwj:
    ACK 1ef2138c0db3bd4f9332c777fa3fb2770dc1b08c

Tree-SHA512: a58c2ece588c640289dc1d35dad5b1b8732788272daa0965d6bf44ee8a7f7c8e8585f94d233ac41c84b9ffcfc97841a00fe2c9acba41f58fd164f01de4b6512b
2024-04-03 14:16:43 +07:00
MarcoFalke
eba325d7a2
Merge #16551: test: Test that low difficulty chain fork is rejected
Backport notice:
 - data have real blocks from testnet
 - due to big gap in blocks before checkpoints (half-year) for sack of this test is added one more checkpoint, otherwise blocks are not accepted due to non-mockable time for other chains except regtest
 - data for 2 blocks in split chain are generated locally for testnet
--------------
333317ce6b67aa92f7363d48cd750712190b4b6b test: Test that low difficulty chain fork is rejected (MarcoFalke)
fa31dc1bf4ee471c4641eef8de02702ba0619ae7 test: Pass down correct chain name in tests (MarcoFalke)

Pull request description:

  To prevent OOM, Bitcoin Core will reject chain forks at low difficulty by default. This is the only use-case of checkpoints, so add a test for it to make sure the feature works as expected. If it didn't work, checkpoints would have no use-case and we might as well remove them

ACKs for top commit:
  Sjors:
    Thanks for adding the node 1 example. Code review ACK 333317c

Tree-SHA512: 90dffa540d0904f3cffb61d2382b1a26f84fe9560b7013e4461546383add31a8757b350616a6d43217c59ef7b8b2a1b62bb3bab582c679cbb2c660a782ce7be1
2024-04-03 14:16:43 +07:00
MarcoFalke
436a5783c7
Merge bitcoin/bitcoin#22736: log, sync: change lock contention from preprocessor directive to log category
7e698732836121912f179b7c743a72dd6fdffa72 sync: remove DEBUG_LOCKCONTENTION preprocessor directives (Jon Atack)
9b08006bc502e67956d6ab518388fad6397cac8d log, sync: improve lock contention logging and add time duration (Jon Atack)
3f4c6b87f1098436693c4990f2082515ec0ece26 log, timer: add timing macro in usec LOG_TIME_MICROS_WITH_CATEGORY (Jon Atack)
b7a17444e0746c562ae97b26eba431577947b06a log, sync: add LOCK logging category, apply it to lock contention (Jon Atack)

Pull request description:

  To enable lock contention logging, `DEBUG_LOCKCONTENTION` has to be defined at compilation. Once built, the logging is not limited to a category and is high frequency, verbose and in all-caps. With these factors combined, it seems likely to be rarely used.

  This patch:
  - adds a `lock` logging category
  - adds a timing macro in microseconds, `LOG_TIME_MICROS_WITH_CATEGORY`
  - updates `BCLog::LogMsg()` to omit irrelevant decimals for microseconds and skip unneeded code and math
  - improves the lock contention logging, drops the all-caps, and displays the duration in microseconds
  - removes the conditional compilation directives
  - allows lock contentions to be logged on startup with `-debug=lock` or at run time with `bitcoin-cli logging '["lock"]'`

  ```
  $ bitcoind -signet -debug=lock
  2021-09-01T12:40:01Z LockContention: cs_vNodes, net.cpp:1920 started
  2021-09-01T12:40:01Z LockContention: cs_vNodes, net.cpp:1920 completed (4μs)
  2021-09-01T12:40:01Z LockContention: cs_vNodes, net.cpp:1302 started
  2021-09-01T12:40:01Z LockContention: cs_vNodes, net.cpp:1302 completed (4μs)
  2021-09-01T12:40:02Z LockContention: cs_vNodes, net.cpp:2242 started
  2021-09-01T12:40:02Z LockContention: cs_vNodes, net.cpp:2242 completed (20μs)
  2021-09-01T12:43:04Z LockContention: ::cs_main, validation.cpp:4980 started
  2021-09-01T12:43:04Z LockContention: ::cs_main, validation.cpp:4980 completed (3μs)

  $ bitcoin-cli -signet logging
    "lock": true,

  $ bitcoin-cli -signet logging [] '["lock"]'
    "lock": false,

  $ bitcoin-cli -signet logging '["lock"]'
    "lock": true,
  ```

  I've tested this with Clang 13 and GCC 10.2.1, on Debian, with and without `--enable-debug`.

ACKs for top commit:
  hebasto:
    re-ACK 7e698732836121912f179b7c743a72dd6fdffa72, added a contention duration to the log message since my [previous](https://github.com/bitcoin/bitcoin/pull/22736#pullrequestreview-743764606) review.
  theStack:
    re-ACK 7e698732836121912f179b7c743a72dd6fdffa72 🔏 ⏲️

Tree-SHA512: c4b5eb88d3a2c051acaa842b3055ce30efde1f114f61da6e55fcaa27476c1c33a60bc419f7f5ccda532e1bdbe70815222ec2b2b6d9226f29c8e94e598aacfee7
2024-04-02 12:09:35 -05:00
pasta
14a923a7e9
Merge #5967: test: Fix p2p_addr_relay.py
c79f8b5ea2 fix: keep ADDRS outside (UdjinM6)
a39065be88 test: fix incorrect nServices assertion, use NODE_NETWORK value (Kittywhiskers Van Gogh)
0e78555a5b fix: add missing check for sending addrs (UdjinM6)
3cd69377b6 fix: test nodes should use mocktime (UdjinM6)
acbbe8c9a2 fix: relayed addresses should use mocktime (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  `p2p_addr_relay.py` is broken but we don't see it in CI because it doesn't test everything it should atm. This weird behaviour was discovered by @kwvg while preparing #5964.

  ## What was done?
  Bring back the missing check and fix the test. Borrowed one fix from #5964 to make this PR complete.

  ## How Has This Been Tested?
  Run `p2p_addr_relay.py`

  ## Breaking Changes
  n/a

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  kwvg:
    ACK c79f8b5ea2

Tree-SHA512: a88f9c3563205b092ebd9ab7e8f0c034b6ef5bef2adbf57c269c444ef334e519e30d028325e73e99de025654a0dbd94baad033fb1538896e85572d4db2a00b23
2024-04-02 12:03:24 -05:00
UdjinM6
c79f8b5ea2
fix: keep ADDRS outside 2024-04-02 18:28:22 +03:00
pasta
85caf8aa34
Merge #5949: chore: apply CI for clang-format only for dash files
adc0e4b382 fix: apply changes for .clang-format to make it matched with our code style (Konstantin Akimov)
0c884f9740 chore: narrow score of clang-diff-format for dash specific files only (Konstantin Akimov)
4bc0e1f697 chore: intentionally introducing wrong formatting to bip39.cpp to trigger CI (Konstantin Akimov)
2c74ad427d fix: adjust wallet/bip39 accordingly linter comments (Konstantin Akimov)
d3faa8522c refactor: use better masks for list of files; add missing bip39.{h,cpp} (Konstantin Akimov)
7788f1db0e refactor: move list of non backported files o test/util/data/non-backported.txt (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  **Note**: should be this PR either https://github.com/dashpay/dash/pull/5942 be merged, not both

  CI clang-format triggers to non-dash files + clang format is differ from out current formatting.

  ## What was done?
  See each commits

  ## How Has This Been Tested?
  See CI result
  To test locally how new style will look, just run this command:
  ```
  diff -u <(cat {coinjoin,governance,llmq,evo,masternode}/*.{h,cpp}) <(clang-format-16 {coinjoin,governance,llmq,evo,masternode}/*.{h,cpp} )
  ```

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

Top commit has no ACKs.

Tree-SHA512: d87f30ba78e04f886d7eb2b6b235c20a966bc4438e6428a83ecff5c795d72777516d4270eb9769ffebef9f06e9509acf3c535b4c87b1be6c8a5aef7e2b7efecb
2024-04-02 09:40:33 -05:00
Kittywhiskers Van Gogh
a39065be88
test: fix incorrect nServices assertion, use NODE_NETWORK value
The value of nServices for `NODE_NETWORK | NODE_WITNESS`, the default for
p2p_addr{v2}_relay.py in Bitcoin Core, is 9. Dash doesn't implement SegWit
and so the corresponding nServices value is `NODE_NETWORK`, which is 1.
2024-04-02 17:40:19 +03:00
UdjinM6
0e78555a5b
fix: add missing check for sending addrs 2024-04-02 17:38:58 +03:00
UdjinM6
3cd69377b6
fix: test nodes should use mocktime 2024-04-02 17:38:35 +03:00
UdjinM6
acbbe8c9a2
fix: relayed addresses should use mocktime 2024-04-02 17:37:37 +03:00
fanquake
ae74ad09fb
Merge #20817: lint: update list of spelling linter false positives, bump to codespell 2.0.0
f3ba916e8b5b5ee2a381cef38882671eadb231df lint: ignore gitian keys file for spelling linter (Sebastian Falbesoner)
da289a6c4a0a5e110e301f34f1db57b6d31bcdcc lint: update list of spelling linter false positives (Sebastian Falbesoner)
a0022f1cfbb3d8f1f8f3ff135f854be0cb89643f test: bump codespell linter version to 2.0.0 (Sebastian Falbesoner)

Pull request description:

  This small patch updates the ignore list for the spelling linter script (which uses `codespell`), both removing false-positives that are not relevant anymore and adding new ones. As [suggested by jonatack](https://github.com/bitcoin/bitcoin/pull/20762#issuecomment-750889701)~~, whose last name is now also part of the list :)~~. Also changed the linter script to not check the gitian keys file, as [suggested by hebasto](https://github.com/bitcoin/bitcoin/pull/20817#discussion_r550763409). The codespell version used is bumped to most recent version 2.0.0, which is more aware of some terms that were previously needed in the ignorelist for v1.17.1, see https://github.com/bitcoin/bitcoin/pull/20817#issuecomment-753428669.

  Running spelling linter on master branch (repeated findings in the same file are removed to keep the output short):
  ```
  $ ./test/lint/lint-spelling.sh
  contrib/gitian-keys/keys.txt:16: Atack ==> Attack
  doc/developer-notes.md:1284: inout ==> input, in out
  doc/psbt.md:122: Asend ==> Ascend, as end
  src/bench/verify_script.cpp:27: Keypair ==> Key pair
  src/blockencodings.h:30: Unser ==> Under, unset, unsure, user
  src/compressor.h:65: Unser ==> Under, unset, unsure, user
  src/core_read.cpp:131: presense ==> presence
  src/index/disktxpos.h:21: blockIn ==> blocking
  src/net_processing.h:67: anounce ==> announce
  src/netaddress.h:486: compatiblity ==> compatibility
  src/primitives/transaction.h:35: nIn ==> inn, min, bin, nine
  src/qt/bitcoinunits.cpp:101: nIn ==> inn, min, bin, nine
  src/rpc/blockchain.cpp:2150: nIn ==> inn, min, bin, nine
  src/rpc/misc.cpp:198: nIn ==> inn, min, bin, nine
  src/script/bitcoinconsensus.cpp:81: nIn ==> inn, min, bin, nine
  src/script/bitcoinconsensus.h:63: nIn ==> inn, min, bin, nine
  src/script/interpreter.cpp:1279: nIn ==> inn, min, bin, nine
  src/script/interpreter.h:222: nIn ==> inn, min, bin, nine
  src/script/sign.cpp:17: nIn ==> inn, min, bin, nine
  src/script/sign.h:39: nIn ==> inn, min, bin, nine
  src/serialize.h:181: Unser ==> Under, unset, unsure, user
  src/signet.cpp:142: nIn ==> inn, min, bin, nine
  src/test/base32_tests.cpp:17: fo ==> of, for
  src/test/base64_tests.cpp:17: fo ==> of, for
  src/test/script_tests.cpp:1509: nIn ==> inn, min, bin, nine
  src/test/sighash_tests.cpp:27: nIn ==> inn, min, bin, nine
  src/test/validation_tests.cpp:78: excercise ==> exercise
  src/undo.h:36: Unser ==> Under, unset, unsure, user
  src/validation.cpp:1403: nIn ==> inn, min, bin, nine
  src/validation.h:255: nIn ==> inn, min, bin, nine
  src/wallet/wallet.cpp:1532: nIn ==> inn, min, bin, nine
  src/wallet/walletdb.cpp:429: Crypted ==> Encrypted
  test/functional/feature_nulldummy.py:63: unnecssary ==> unnecessary
  test/functional/wallet_encryption.py:81: crypted ==> encrypted
  test/functional/wallet_upgradewallet.py:36: fpr ==> for, far, fps
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```

  Running spelling linter on PR branch:
  ```
  $ ./test/lint/lint-spelling.sh
  src/core_read.cpp:131: presense ==> presence
  src/net_processing.h:67: anounce ==> announce
  src/netaddress.h:486: compatiblity ==> compatibility
  src/test/validation_tests.cpp:78: excercise ==> exercise
  src/wallet/walletdb.cpp:429: Crypted ==> Encrypted
  test/functional/feature_nulldummy.py:63: unnecssary ==> unnecessary
  test/functional/wallet_encryption.py:81: crypted ==> encrypted
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```
  This list of remaining findings doesn't contain false positives anymore -- the typos are fixed in PR https://github.com/bitcoin/bitcoin/pull/20762.
  Happy new year! 🍾

ACKs for top commit:
  hebasto:
    re-ACK f3ba916e8b5b5ee2a381cef38882671eadb231df, only suggested changes since my [previous](https://github.com/bitcoin/bitcoin/pull/20817#pullrequestreview-560632881) review.
  jonatack:
    ACK f3ba916e8b5b5ee2a381cef38882671eadb231df I don't know if there are any particular issues with bumping codespell to v2.0.0, but locally running the spelling linter and the cirrus job at https://cirrus-ci.com/task/5004066998714368 both LGTM. Thanks for also verifying and removing the unused words from the ignore list.

Tree-SHA512: e92ae6f16c01d4ff3d54f8c3a0ee95e12741f7bfe031d307a785f5cfd8a80525b16b34275f413b914c4a318f5166f9887399c21f2dad9cc7e9be41647042ef37
2024-03-27 00:48:28 +07:00
MarcoFalke
3231ad2255
Merge #19983: Drop some TSan suppressions
3e1571285f4a0edf59d51bbdeee028be3038b6dc Update TSan suppressions (Hennadii Stepanov)

Pull request description:

  It seems possible now to drop some TSan suppressions.

Top commit has no ACKs.

Tree-SHA512: 94518fd2f3a7168b2989424de0696e42c8f509b833aafbc7e75f4c1180a0b8d9a47f43c50d06b03b26a924643afe86274b2062c9d456c17a68576d19566ed66f
2024-03-27 00:48:28 +07:00
MarcoFalke
802cb9521f
Merge #20697: ci: Fix COMMIT_RANGE variable value for PRs
3c2478c38522c176e81befd4d991a259b09be063 ci: Print COMMIT_RANGE to the log as it was in Travis CI (Hennadii Stepanov)
c123892c2e47e3706f06820aba2454d494a39564 ci: Drop Travis-specific workaround for shellcheck (Hennadii Stepanov)
10af252d97532843b26505d215f6e975f4b21672 ci: Drop Travis-specific way to set COMMIT_RANGE variable (Hennadii Stepanov)
93504da3a932f33126545ebc9383f695a6efe51e ci: Fix COMMIT_RANGE variable value for PRs (Hennadii Stepanov)

Pull request description:

  This PR:
  - is a #20658 and #20682  followup
  - set the `COMMIT_RANGE` variable correctly for PRs
  - cleans up Travis-specific code
  - prints COMMIT_RANGE value to the log for convenience as it was in Travis CI

ACKs for top commit:
  MarcoFalke:
    ACK 3c2478c38522c176e81befd4d991a259b09be063

Tree-SHA512: beb933352b10fd5eb3e66373ddb62439e4f3a03b50fb037ee89fa92c0706cec41d05f2d307f15bb18d1e634e6464f4e123b7e2f88703c8edfd145d8d6eff0b1a
2024-03-27 00:48:27 +07:00
MarcoFalke
8daef64f04
Merge #20691: ci, doc: Travis CI features and mentions cleanup
95487b055328b590ba83f258de9637ab0f9a2f17 doc: Drop mentions of Travis CI as it is no longer used (Hennadii Stepanov)
09d105ef0f8b4b06bf248721a1209c9e16e9db75 ci: Drop travis_fold feature as Travis CI is no longer used (Hennadii Stepanov)

Pull request description:

  As Travis CI is no longer used, this PR:
  - drops `travis_fold` feature
  - drops mentions of Travis CI in docs

ACKs for top commit:
  MarcoFalke:
    ACK 95487b055328b590ba83f258de9637ab0f9a2f17

Tree-SHA512: 2e259bb8b1e37bcefc1251737bb2716f06ddb57c490010b373825c4e70f42ca38efae69a2f63f21f577d7cee3725b94097bdddbd313f8ebf499281cf97c53cef
2024-03-27 00:48:26 +07:00
pasta
f217e0ae7b
Merge #5940: refactor: consolidate activeMasternodeInfo{Cs} into CActiveMasternodeManager, create NodeContext alias, reduce globals usage
815e4f8026 masternode: protect m_{error,state} with cs (pasta)
136e445abc refactor: pass CActiveMasternodeManager as pointer arg to LLMQContext (Kittywhiskers Van Gogh)
5e0f77747a refactor: pass CActiveMasternodeManager as pointer arg to CJContext (Kittywhiskers Van Gogh)
f171c24a29 refactor: add CActiveMasternodeManager NodeContext alias, use in RPC (Kittywhiskers Van Gogh)
44beb941cb refactor: prefix member variable names with m_ (Kittywhiskers Van Gogh)
73cef4f5f9 refactor: make bls{Pub}KeyOperator member variables instead of pointers (Kittywhiskers Van Gogh)
fbc783635a refactor: make m_info private, get const refs (or copies) from Get*() functions (Kittywhiskers Van Gogh)
1b516ce4ed refactor: use signing helper function instead of passing blsKeyOperator (Kittywhiskers Van Gogh)
33702aca39 refactor: add helper function to decrypt messages with blsKeyOperator (Kittywhiskers Van Gogh)
3eb931b596 refactor: add helper function to sign messages with blsKeyOperator (Kittywhiskers Van Gogh)
3827355cce refactor: move key initialization to InitKeys, define destructor (Kittywhiskers Van Gogh)
e5295dec1f refactor: move activeMasternodeInfo{Cs} into CActiveMasternodeManager (Kittywhiskers Van Gogh)
b8c1f010e7 refactor: avoid accessing active masternode info if not in masternode mode (Kittywhiskers Van Gogh)
9a3c5a3c48 trivial: access activeMasternodeInfo when lock is in scope (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * `CActiveMasternodeManager`, unlike other managers, is _conditionally_ initialized (specifically, when the node is hosting a masternode). This means that checks need to be made to ensure that the conditions needed to initialize the manager are true or that the pointer leads to a valid manager instance.

    As the codebase currently checks (and fast-fails) based on the node being in "masternode mode" (`fMasternodeMode`) or not, we will continue with this approach, but with additional assertions _after_ the masternode mode check if the manager exists.

  * Though, since `activeMasternodeInfo`(`Cs`) are global variables, they can be accessed _regardless_ of whether the corresponding manager exists. This means some parts of the codebase attempt to fetch information about the (nonexistent) active masternode _before_ determining if it should use the masternode mode path or not (looking at you, `CMNAuth::ProcessMessage`)

    Moving them into `CActiveMasternodeManager` meant adding checks _before_ attempting to access information about the masternode, as they would no longer be accessible with dummy values ([here](2110c0c309/src/init.cpp (L1633-L1635))) on account of being part of the conditionally initialized manager.
    * In an attempt to opportunistically dereference the manager, `CDKGSessionManager` (accepting a pointer) was dereferencing the manager before passing it to `CDKGSessionHandler`. This was done under the assumption that  `CDKGSessionManager` would only ever be initialized in masternode mode.

      This is not true. I can confirm that because I spent a few days trying to debug test failures. `CDKGSessionHandler` is initialized in two scenarios:

      * In masternode mode
      * If the `-watchquorums` flag is enabled

      The latter scenario doesn't initialize `CActiveMasternodeManager`.

      Furthermore, the DKG round thread is started unconditionally ([here](2110c0c309/src/llmq/context.cpp (L79))) and the `CDKGSessionHandler::StartThreads` > `CDKGSessionHandler::StartThread` > `CDKGSessionHandler::PhaseHandlerThread` > `CDKGSessionHandler::HandleDKGRound` > `CDKGSessionHandler::InitNewQuorum` > `CActiveMasternodeManager::GetProTxHash` call chain reveals an attempt to fetch active masternode information without any masternode mode checks.

      This behaviour has now been changed and the thread will only be spun up if in masternode mode.

    * Dereferencing so far has been limited to objects that primarily hold data (like `CCoinJoinBroadcastTx` or `CGovernanceObject`) as they should not have knowledge of node's state (that responsibility lies with whatever manager manipulates those objects), perform one-off operations and static functions.

  * `activeMasternodeInfo` allowed its members to be read-write accessible to anybody who asked. Additionally, signing and decrypting involved borrowing the operator secret key from the active masternode state to perform those operations.

     This behaviour has now been changed. The internal state is now private and accessible read-only as a const ref (or copy) and `Decrypt`/`Sign` functions have been implemented to allow those operations to happen without having another manager access the operator private key in order to do so.

  * You cannot combine a `WITH_LOCK` and an `Assert` (in either mutex or accessed value), doing so will cause errors if `-Werror=thread-safety` is enabled. This is why `assert`s are added even when it would intuitively seem that `Assert` would've been more appropriate to use.

  ## Future Considerations

  Currently there are no unit tests that test the functionality of `CActiveMasternodeManager` as it's never initialized in test contexts, breakage had to be found using functional tests. Perhaps some (rudimentary) tests for `CActiveMasternodeManager` may prove to be valuable.

  ## Breaking Changes

  Not _really_. Some behaviour has been modified but nothing that should necessitate updates or upgrades.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK 815e4f8026

Tree-SHA512: cbe49ea9e1c35df514e1b40869ee271baef1c348c9d09e4b356e5fc8fe5449cbbe66569258f2d664029faa9a46f711df9bf9e41eb8734c3aefc6cd8e94378948
2024-03-26 08:43:54 -05:00
pasta
f2a42a01b1
Merge #5951: backport: trivial 2024 03 22
d5d1a714fb Merge bitcoin/bitcoin#24390: test: Remove suppression no longer needed with headers-only Boost.Test (fanquake)
51630d2e5e Merge bitcoin/bitcoin#22824: refactor: remove RecursiveMutex cs_nBlockSequenceId (MarcoFalke)
a9b1575fe8 Merge bitcoin/bitcoin#22781: wallet: fix the behavior of IsHDEnabled, return false in case of a blank hd wallet. (Samuel Dobson)
0505229c89 Merge bitcoin/bitcoin#22327: cli: Avoid truncating -rpcwaittimeout (MarcoFalke)
1dc97c7679 Merge bitcoin/bitcoin#22149: test: Add temporary logging to debug #20975 (W. J. van der Laan)
44f91cbc9a Merge #21597: test: Document race:validation_chainstatemanager_tests suppression (fanquake)
c326830f48 Merge bitcoin-core/gui#243: fix issue when disabling the auto-enabled blank wallet checkbox (MarcoFalke)
267f42fd6a Merge #21382: build: Clean remnants of QTBUG-34748 fix (fanquake)
1fcc5f1101 Merge #20540: test: Fix wallet_multiwallet issue on windows (MarcoFalke)
4afbaf2ea1 Merge #20322: test: Fix intermittent issue in wallet_listsinceblock (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Batch of backports

  ## What was done?
  Trivial batch of backports

  ## How Has This Been Tested?
  CI looks good

  ## Breaking Changes
  None

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

Top commit has no ACKs.

Tree-SHA512: 8eeac54f011eb1111888c745dd56184ac9601de290f2b0f7b7ad02240e8dc1cab5a47fed26bfed2bd6f1066e0710827a3e5b2426f0bf66821cf1cd09099d5160
2024-03-25 22:46:15 -05:00
fanquake
d5d1a714fb
Merge bitcoin/bitcoin#24390: test: Remove suppression no longer needed with headers-only Boost.Test
81738d2881253f28b69666ada2a01ebb353f503a test: Remove suppression no longer needed with headers-only Boost.Test (Hennadii Stepanov)

Pull request description:

  It appears, that moving to [headers-only](https://github.com/bitcoin/bitcoin/pull/24301) Boost.Test makes the removed suppression unneeded even without [bumping](https://github.com/bitcoin/bitcoin/pull/24383) boost version.

ACKs for top commit:
  MarcoFalke:
    cr ACK 81738d2881253f28b69666ada2a01ebb353f503a

Tree-SHA512: e60443f79a2e38cc78fceeff5c2956d622e8a10730129f9c27c14aef59bc6fa0894b8011e6191530443bf3165f78da978bc08ad04248ddb65e2da373264afa6a
2024-03-25 11:21:49 -05:00
W. J. van der Laan
1dc97c7679
Merge bitcoin/bitcoin#22149: test: Add temporary logging to debug #20975
faa94961d6e38392ba068381726ed4e033367b03 test: Add temporary logging to debug #20975 (MarcoFalke)

Pull request description:

  to be reverted after a fix

ACKs for top commit:
  laanwj:
    Code review ACK faa94961d6

Tree-SHA512: 1f3103fcf4cad0af54e26c4d257bd824b128b5f2d2b81c302e861a829fd55d6a099fa476b79b30a71fe98975ae604b9e3ff31fd48a51d442389a9bd515e60ba0
2024-03-25 11:21:48 -05:00
fanquake
44f91cbc9a
Merge #21597: test: Document race:validation_chainstatemanager_tests suppression
fab19871bad1cbe15ec2193f01152eacbf14aeb1 test: Document race:validation_chainstatemanager_tests suppression (MarcoFalke)

Pull request description:

ACKs for top commit:
  jamesob:
    ACK fab19871ba
  practicalswift:
    ACK fab19871bad1cbe15ec2193f01152eacbf14aeb1

Tree-SHA512: 3f1838b4cf11eba768ce06826cd4b57c9065669b61a5530af44216fc96535ebf37124b47a8de8f72aedf32345157a72d2208cd63214481a9cb56c063f05db5dd
2024-03-25 11:21:48 -05:00
Kittywhiskers Van Gogh
0d9046586b
merge bitcoin#21254: Avoid connecting to real network when running tests 2024-03-25 11:55:07 +00:00
Kittywhiskers Van Gogh
0d46acbfef
merge bitcoin#21165: Use mocktime in test_seed_peers 2024-03-25 11:55:07 +00:00
Kittywhiskers Van Gogh
8f40769385
merge bitcoin#19884: No delay in adding fixed seeds if -dnsseed=0 and peers.dat is empty 2024-03-25 11:55:06 +00:00
Kittywhiskers Van Gogh
446076d094
test: add missing dnsseed=0 in configuration
line added in fa31dc1bf4 (diff-e20e1f68486e5c096fdc11bca1cda063aacef524411f011d9c3497eb650c5eafR301)
2024-03-25 11:55:06 +00:00
Kittywhiskers Van Gogh
081d8db4d5
mempool: remove stray boost::optional usage 2024-03-25 11:55:06 +00:00
Kittywhiskers Van Gogh
bcd383c2d6
merge bitcoin#20724: Cleanup of -debug=net log messages 2024-03-25 11:55:05 +00:00
Kittywhiskers Van Gogh
1b516ce4ed
refactor: use signing helper function instead of passing blsKeyOperator 2024-03-24 07:20:58 +00:00
Konstantin Akimov
2c74ad427d
fix: adjust wallet/bip39 accordingly linter comments 2024-03-24 00:41:23 +07:00
Konstantin Akimov
d3faa8522c
refactor: use better masks for list of files; add missing bip39.{h,cpp} 2024-03-24 00:41:23 +07:00
Konstantin Akimov
7788f1db0e
refactor: move list of non backported files o test/util/data/non-backported.txt
This files is refactored out from 'lint-cppcheck-dash' to dedicated file.
It is supposed to be used by clang formatter also in CI
2024-03-24 00:41:22 +07:00
MarcoFalke
1fcc5f1101
Merge #20540: test: Fix wallet_multiwallet issue on windows
fada2dfcac1c4b47ee76b877d91d515cf1d36410 test: Fix wallet_multiwallet issue on windows (MarcoFalke)

Pull request description:

  The error message on windows:

  > 2020-11-30T18:10:47.536032Z ListWalletDir: Error scanning C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20201130_181042\wallet_multiwallet_0\node0\regtest\wallets\self_walletdat_symlink: boost::filesystem::status: The name of the file cannot be resolved by the system: "C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20201130_181042\wallet_multiwallet_0\node0\regtest\wallets\self_walletdat_symlink\wallet.dat"

ACKs for top commit:
  promag:
    Code review ACK fada2dfcac1c4b47ee76b877d91d515cf1d36410. Although it could ignore (don't log) directories that lead to no permission error.
  fanquake:
    ACK fada2dfcac1c4b47ee76b877d91d515cf1d36410

Tree-SHA512: b475162cc3cd1574209d916605b229a79c8089714295f5e16569b71f958f0007d54dc76833938492d931387784588b11b73e3ef00f963540af42c079417f8d72
2024-03-22 11:20:57 -05:00
MarcoFalke
4afbaf2ea1
Merge #20322: test: Fix intermittent issue in wallet_listsinceblock
444412821e5349ce0e0b039d884583edb70c4399 test: Fix intermittent issue in wallet_listsinceblock (MarcoFalke)

Pull request description:

ACKs for top commit:
  Empact:
    Code Review ACK 444412821e

Tree-SHA512: 86d47b1e3c8681dd479654589c894016ac81a3c96a34c3b4a75278b2af85054ea8c6f768e518a5322a4928d82d5e99105bbce0f4fa6a7a18c40e3e0799f9ab54
2024-03-22 11:20:51 -05:00
Wladimir J. van der Laan
d7985aa36a
Merge #19512: p2p: banscore updates to gui, tests, release notes
fa108d6a757838225179a8df942cfb6d99c98c90 test: update tests for peer discouragement (Jon Atack)
1a9f462caa63fa16d7b4415312d2032a42b3fe0b gui, doc: rm Ban Score in GUI Peers window/release notes updates (Jon Atack)

Pull request description:

  This is the third `-banscore` PR in the mini-series described in #19464. See that PR for the intention and reasoning.

  - no longer display "Ban Score" in the GUI peers window and add a release note, plus release note fixups per https://github.com/bitcoin/bitcoin/pull/19464#pullrequestreview-447452052
  - update tests (`src/test/denialofservice_tests.cpp` and `test/functional/p2p_leak.py`) from banning to discouragement and per https://github.com/bitcoin/bitcoin/pull/19464#issuecomment-658052518

ACKs for top commit:
  jnewbery:
    ACK fa108d6a757838225179a8df942cfb6d99c98c90
  laanwj:
    ACK fa108d6a757838225179a8df942cfb6d99c98c90

Tree-SHA512: 58a449b3f47b8cb5490b34e4442ee8675bfad1ce48af4e4fd5c67715b0c1a596fb8e731d42e576b4c3b64627f76e0a68cbb1da9ea9f588a5932fe119baf40d50
2024-03-22 11:08:11 -05:00
MarcoFalke
b9799de985
Merge #19464: net: remove -banscore configuration option
06059b0c2a6c2db70c87a7715f8a344a13400fa1 net: rename DEFAULT_BANSCORE_THRESHOLD to DISCOURAGEMENT_THRESHOLD (Jon Atack)
1d4024bca8086cceff7539dd8c15e0b7fe1cc5ea net: remove -banscore configuration option (Jon Atack)

Pull request description:

  per https://github.com/bitcoin/bitcoin/pull/19219#issuecomment-652684340, https://github.com/bitcoin/bitcoin/pull/19219#discussion_r443074487 and https://github.com/bitcoin/bitcoin/pull/19219#issuecomment-652699592. Edit: now split into 3 straightforward PRs:
  - net: remove -banscore configuration option (this PR)
  - rpc: deprecate banscore field in getpeerinfo (#19469, *merged*)
  - gui: no longer display banscores (TBA in the gui repo)

ACKs for top commit:
  MarcoFalke:
    review ACK 06059b0c2a6c2db70c87a7715f8a344a13400fa1 📙
  vasild:
    ACK 06059b0c

Tree-SHA512: 03fad249986e0896697033fbb8ba2cbfaae7d7603b1fb2a38b3d41db697630d238623f4d732b9098c82af249ce5a1767dd432b7ca0fec10544e23d24fbd57c50
2024-03-22 11:08:10 -05:00
fanquake
c0c5d05419
Merge #19469: rpc: deprecate banscore field in getpeerinfo
41d55d30579358c805036201664ad6a1c1d48681 doc: getpeerinfo banscore deprecation release note (Jon Atack)
dd54e3796e633cfdf6954af306afd26eadc25116 test: getpeerinfo banscore deprecation test (Jon Atack)
8c7647b3fbbab03ea84071cf3cd2d0d2bf8be255 rpc: deprecate banscore field in rpc getpeerinfo (Jon Atack)

Pull request description:

  Per https://github.com/bitcoin/bitcoin/pull/19219#discussion_r443074487 and https://github.com/bitcoin/bitcoin/pull/19219#issuecomment-652699592, this PR deprecates returning the `banscore` field in the `getpeerinfo` RPC, updates the help, adds a test, and updates the release notes. Related to #19464.

ACKs for top commit:
  fanquake:
    ACK 41d55d30579358c805036201664ad6a1c1d48681

Tree-SHA512: 8eca08332581e2fe191a2aafff6ba89ce39413f0491ed0de8b86577739f0ec430b1a8fbff2914b0f3138a229563dfcc1981c0cf5b7dd6061b5c48680a28423bc
2024-03-22 11:08:10 -05:00
MarcoFalke
57b6fe7327
Merge bitcoin/bitcoin#22257: test: refactor: various (de)serialization helpers cleanups/improvements
bdb8b9a347e68f80a2e8d44ce5590a2e8214b6bb test: doc: improve doc for `from_hex` helper (mention `to_hex` alternative) (Sebastian Falbesoner)
191405420815d49ab50184513717a303fc2744d6 scripted-diff: test: rename `FromHex` to `from_hex` (Sebastian Falbesoner)
a79396fe5f8f81c78cf84117a87074c6ff6c9d95 test: remove `ToHex` helper, use .serialize().hex() instead (Sebastian Falbesoner)
2ce7b47958c4a10ba20dc86c011d71cda4b070a5 test: introduce `tx_from_hex` helper for tx deserialization (Sebastian Falbesoner)

Pull request description:

  There are still many functional tests that perform conversions from a hex-string to a message object (deserialization) manually. This PR identifies all those instances and replaces them with a newly introduced helper `tx_from_hex`.

  Instances were found via
  * `git grep "deserialize.*BytesIO"`

  and some of them manually, when it were not one-liners.

  Further, the helper `ToHex` was removed and simply replaced by `.serialize().hex()`, since now both variants are in use (sometimes even within the same test) and using the helper doesn't really have an advantage in readability. (see discussion https://github.com/bitcoin/bitcoin/pull/22257#discussion_r652404782)

ACKs for top commit:
  MarcoFalke:
    review re-ACK bdb8b9a347e68f80a2e8d44ce5590a2e8214b6bb 😁

Tree-SHA512: e25d7dc85918de1d6755a5cea65471b07a743204c20ad1c2f71ff07ef48cc1b9ad3fe5f515c1efaba2b2e3d89384e7980380c5d81895f9826e2046808cd3266e
2024-03-22 10:29:15 -05:00
W. J. van der Laan
5c85b7dc14
Merge bitcoin/bitcoin#21056: rpc: Add a -rpcwaittimeout parameter to limit time spent waiting
b9e76f1bf08c52fcd402b2314e00db4ad247ebc8 rpc: Add test for -rpcwaittimeout (Christian Decker)
f76cb10d7dc9a7b0c55d28011161606399417664 rpc: Prefix rpcwaittimeout error with details on its nature (Christian Decker)
c490e17ef698a1695050f82ef6567b3b87a21861 doc: Add release notes for the `-rpcwaittimeout` cli parameter (Christian Decker)
a7fcc8eb59fe51473571661316214156fbdbdcae rpc: Add a `-rpcwaittimeout` parameter to limit time spent waiting (Christian Decker)

Pull request description:

  Adds a new numeric `-rpcwaittimeout` that can be used to limit the
  time we spend waiting on the RPC server to appear. This is used by
  downstream projects to provide a bit of slack when `bitcoind`s RPC
  interface is not available right away.

  This makes the `-rpcwait` argument more useful, since we can now limit
  how long we'll ultimately wait, before potentially giving up and reporting
  an error to the caller. It was discussed in the context of the BTCPayServer
  wanting to have c-lightning wait for the RPC interface to become available
  but still have the option of giving up eventually ([4355]).

  I checked with laanwj whether this is already possible ([comment]), and
  whether this would be a welcome change. Initially I intended to repurpose
  the (optional) argument to `-rpcwait`, however I decided against it since it
  would potentially break existing configurations, using things like `rpcwait=1`,
  or `rpcwait=true` (the former would have an unintended short timeout, when
  old behavior was to wait indefinitely).

  ~Due to its simplicity I didn't implement a test for it yet, but if that's desired I
  can provide one.~ Test was added during reviews.

  [4355]: https://github.com/ElementsProject/lightning/issues/4355
  [comment]: https://github.com/ElementsProject/lightning/issues/4355#issuecomment-768288261

ACKs for top commit:
  laanwj:
    Code review ACK b9e76f1bf08c52fcd402b2314e00db4ad247ebc8
  promag:
    ACK b9e76f1bf08c52fcd402b2314e00db4ad247ebc8.

Tree-SHA512: 3cd6728038ec7ca7c35c2e7ccb213bfbe963f99a49bb48bbc1e511c4dd23d9957c04f9af1f8ec57120e47b26eaf580b46817b099d5fc5083c98da7aa92db8638

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2024-03-22 10:29:15 -05:00
MarcoFalke
17598db793
Merge bitcoin/bitcoin#22408: test: add tests for bad-txns-prevout-null reject reason
1f449586a9e39bc4fb53cb5c7a31362e47aea19b test: add `bad-txns-prevout-null` test to mempool_accept.py (Sebastian Falbesoner)
aa0a5bb70d77739d43d5a9ceae78fb0c6fafd435 test: add `bad-txns-prevout-null` test case to invalid_txs.py (Sebastian Falbesoner)

Pull request description:

  This simple PR adds missing tests for the reject reason `bad-txns-prevout-null`, which is thrown in the function `CheckTransaction()`: a62fc35a15/src/consensus/tx_check.cpp (L52-L54)

  Basically this condition is met for non-coinbase transactions (the code snippet above only hits if `!tx.IsCoinBase()`) with coinbase-like outpoints, i.e. hash=0, n=0xffffffff.

  Can be tested by running the functional tests `feature_block.py`, `p2p_invalid_tx.py` and `mempool_accept.py`. Not sure if the redundancy in the tests is desired (I guess it would make sense if the mempool acceptance test also makes use of the invalid_txs templates?).

ACKs for top commit:
  rajarshimaitra:
    tACK 1f449586a9
  brunoerg:
    tACK 1f449586a9e39bc4fb53cb5c7a31362e47aea19b
  kristapsk:
    ACK 1f449586a9e39bc4fb53cb5c7a31362e47aea19b, code looks correct and all tests pass.

Tree-SHA512: 2d4f940a6ac8e0d80d2670c9e1111cbf43ae6ac62809a2ccf17cffee9a41d387ea4d889ee300eb4a407c055b13bfa5d37102a32ed59964a9b6950bd907ba7204
2024-03-22 10:29:12 -05:00
Kittywhiskers Van Gogh
419a8af274
fix: revert "drop circular dependencies over deterministicmns in validationinterface"
This reverts commit fa6847d2fb.
2024-03-21 12:24:38 +00:00
pasta
c90fd21378
Merge #5930: backport: bitcoin#14582, #16404, #17445, #19507, #19538, #19743, #19756, #19773, #19830, partial bitcoin#27053
5a6b8b6b1f partial Merge bitcoin/bitcoin#27053: wallet: reuse change dest when re-creating TX with avoidpartialspends (fanquake)
2f788aa76d fix: change port to use for zmq in interface_zmq_dash.py (Konstantin Akimov)
0ce66fd477 Merge #19507: Expand functional zmq transaction tests (Wladimir J. van der Laan)
a1c2386153 Merge #17445: zmq: Fix due to invalid argument and multiple notifiers (Wladimir J. van der Laan)
44929bad82 Merge #16404: qa: Test ZMQ notification after chain reorg (MarcoFalke)
1707f01309 fix: follow-up changes from bitcoin/bitcoin#22220 for maxapsfee (Konstantin Akimov)
eb4270deae Merge #19743: -maxapsfee follow-up (Samuel Dobson)
6a6d379711 Merge #19756: tests: add sync_all to fix race condition in wallet groups test (MarcoFalke)
5821a1d23a Merge #14582: wallet: always do avoid partial spends if fees are within a specified range (Samuel Dobson)
59d5a4ef39 Merge #19773: wallet: Avoid recursive lock in IsTrusted (Samuel Dobson)
2489f29f0e Merge #19830: test: Add tsan supp for leveldb::DBImpl::DeleteObsoleteFiles (fanquake)
10fa7a66b6 Merge #19538: ci: Add tsan suppression for race in DatabaseBatch (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Regular backports from bitcoin v21

  ## What was done?
   - bitcoin/bitcoin#19538
   - bitcoin/bitcoin#19830
   - bitcoin/bitcoin#19773
   - bitcoin/bitcoin#14582
   - bitcoin/bitcoin#19756
   - bitcoin/bitcoin#19743
   - bitcoin/bitcoin#16404
   - bitcoin/bitcoin#17445
   - bitcoin/bitcoin#19507
   - partial bitcoin/bitcoin#27053

  +some extra fixes and missing changes from bitcoin/bitcoin#22220 for `maxapsfee`

  +changed port for zmq in `interface_zmq_dash.py` to prevent intermittent error in `interface_zmq.py`

  ## How Has This Been Tested?
  Run unit & functional tests

  ## Breaking Changes
  `CreateTransaction` now uses sometime 2 private keys for one transaction instead one

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 5a6b8b6b1f

Tree-SHA512: 7efd8a31808f155c08035d0fb7ceaac369e3e44e68d2c91a88e52815a60efba5fe9458f41d93352627c2c062d414fb0207dcf216fa75b54af210b503f9123de6
2024-03-19 10:12:39 -05:00
pasta
1c20180271
Merge #5936: backport: bitcoin#19528, #19455, #19717, #19849, #19994 - assert for RPCArg names
c5031685bc fix: rename arguments for 'voteraw' (Konstantin Akimov)
3621966f12 feat: add todo to drop Throw() from rpc/util.h (Konstantin Akimov)
b54f03a0c1 fix: wrong name of argument for coinjoin (Konstantin Akimov)
d0163543d9 refactor: use new format CPCCommand for rpc/coinjoin (Konstantin Akimov)
0e1a31159f Merge #19994: Assert that RPCArg names are equal to CRPCCommand ones (net, rpcwallet) (MarcoFalke)
af9eb81e56 fix: wrong name of arguments for RPC (Konstantin Akimov)
c30c8f22dd Merge #19849: Assert that RPCArg names are equal to CRPCCommand ones (blockchain,rawtransaction) (MarcoFalke)
f525f574b0 fix: follow-up missing changes from Merge #18607: rpc: Fix named arguments in documentation (Konstantin Akimov)
7ac1ee0fb4 Merge #19717: rpc: Assert that RPCArg names are equal to CRPCCommand ones (mining,zmq,rpcdump) (MarcoFalke)
860d31f504 Merge #19455: rpc generate: print useful help and error message (Wladimir J. van der Laan)
41c35fd8dc fix: adjust missing arguments and help for misc rpc: debug, echo, mnsync (Konstantin Akimov)
58d923cd5b Merge #19528: rpc: Assert that RPCArg names are equal to CRPCCommand ones (misc) (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  This batch of backports asserts that RPCArg names are equal to CRPCCommand ones.

  ## What was done?
  done backports:
   - bitcoin/bitcoin#19994
   - bitcoin/bitcoin#19849
   - bitcoin/bitcoin#19717
   - bitcoin/bitcoin#19455
   - bitcoin/bitcoin#19528

  Beside that same changes are applied for src/coinjoin's rpc.

  There's also applied multiple fixes for various rpcs for cases when RPCArg names are mismatched with CPCCommand

  **Please, note, that this PR is not final fix for all RPCArgs**. There's a lot of dash's rpc that is not refactored that.
  That it is not easy to implement for `quorum command` because the list of arguments (and even their numbers) are different for each sub-command. This fixes are out-of scope of this PR and should be done before bitcoin#18531 is backported.

  See also relevant bitcoin#21035.

  ## How Has This Been Tested?
  I used this helper to see which exactly args are specified wrongly:
  ```cpp
  diff --git a/src/rpc/server.h b/src/rpc/server.h
  index d4a7ba60eb..cdfd741f54 100644
  --- a/src/rpc/server.h
  +++ b/src/rpc/server.h
  @@ -16,6 +16,7 @@
   #include <string>

   #include <univalue.h>
  +#include <logging.h>

   class CRPCCommand;

  @@ -110,6 +111,19 @@ public:
                 fn().GetArgNames(),
                 intptr_t(fn))
       {
  +        if (fn().m_name != name_in || fn().GetArgNames() != args_in) {
  +            std::cerr << "names:  " << fn().m_name << ' ' << name_in << std::endl;
  +            std::cerr << "arg names: " << fn().GetArgNames().size() << std::endl;
  +            for (const auto& i : fn().GetArgNames()) {
  +                std::cerr << "arg: " << i << std::endl;
  +            }
  +            std::cerr << "FIASCO FIASCO FIASCO FIASCO FIASCO FIASCO" << std::endl;
  +        }
           CHECK_NONFATAL(fn().m_name == name_in);
           CHECK_NONFATAL(fn().GetArgNames() == args_in);
       }
  ```

  ## Breaking Changes
  N/A

  Some arguments are renamed in RPC but they have been broken (used incorrect name not same as in docs)

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    (self)utACK c5031685bc

Tree-SHA512: e885a8f8fa8bc282dae092fe8df65a37e2ab6ca559cd0598d54bfc06cacddb3bd6f3c74fa2d9c1551f8a4fbdfdeabb8d065649df66d5809e792aec6f51d0df14
2024-03-18 15:12:34 -05:00
fanquake
5a6b8b6b1f
partial Merge bitcoin/bitcoin#27053: wallet: reuse change dest when re-creating TX with avoidpartialspends
14b4921a91920df25b19ff420bfe2bff8c56f71e wallet: reuse change dest when recreating TX with avoidpartialspends (Matthew Zipkin)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/27051

  When the wallet creates a transaction internally, it will also create an alternative that spends using destination groups and see if the fee difference is negligible. If it costs the user the same to send the grouped version, we send it (even if the user has `avoidpartialspends` set to `false` which is default). This patch ensures that the second transaction creation attempt re-uses the change destination selected by the first attempt. Otherwise, the first change address remains reserved, will not be used in the second attempt, and then will never be used by the wallet, leaving gaps in the BIP44 chain.

  If the user had `avoidpartialspends` set to true, there is no second version of the created transaction and the change addresses are not affected.

  I believe this behavior was introduced in https://github.com/bitcoin/bitcoin/pull/14582

ACKs for top commit:
  achow101:
    ACK 14b4921a91920df25b19ff420bfe2bff8c56f71e

Tree-SHA512: a3d56f251ff4b333fc11325f30d05513e34ab0a2eb703fadd0ad98d167ae074493df1a24068298336c6ed2da6b31aa2befa490bc790bbc260ed357c8f2397659
2024-03-18 16:30:45 +07:00
Kittywhiskers Van Gogh
dfddfd09a4
trivial: remove unneeded header, enumerate circular dependencies
As found in a review comment in #5908[1], validation.h is not needed for
specialtxman.cpp and removing the header uncovers other circular
dependencies that were obscured by the shortest circular path[2].

[1] - https://github.com/dashpay/dash/pull/5929#discussion_r1526663050
[2] - https://github.com/dashpay/dash/pull/5929#discussion_r1527594636
2024-03-18 09:09:53 +00:00
Konstantin Akimov
2f788aa76d
fix: change port to use for zmq in interface_zmq_dash.py
It resolves a conflict with interface_zmq.py and intermittent failure
2024-03-18 16:01:41 +07:00
Wladimir J. van der Laan
0ce66fd477
Merge #19507: Expand functional zmq transaction tests
7356292e1d7a44da8a2bd31c02c58d550bf38009 Have zmq reorg test cover mempool txns (Gregory Sanders)
a0f4f9c983e57cc97ecbc56d0177eaf1854c842c Add zmq test for transaction pub during reorg (Gregory Sanders)
2399a0600ca9c4b676fa2f97520b8ecc44642246 Add test case for mempool->block zmq notification (Gregory Sanders)
e70512a83c69bc85e96b08ade725594eda3e230f Make ordering of zmq consumption irrelevant to functional test (Gregory Sanders)

Pull request description:

  Tests written to better define what messages are sent when. Also did a bit of refactoring to make sure the exact notification channel ordering doesn't matter.

  Confusions below aside, I believe having these more descriptive tests helps describe what behavior we expect from ZMQ notificaitons.

  Remaining confusion:
  1) Notification patterns seem to vary wildly with the inclusion of mempool transactions being reorg'ed. See difference between "Add zmq test for transaction pub during reorg" and "Have zmq reorg test cover mempool txns" commits for specifics.
  2) Why does a reorg'ed transaction get announced 3 times? From what I understand it can get announced once for disconnected block, once for mempool entry. What's the third? It occurs a 4th time when included in a block(not added in test)

ACKs for top commit:
  laanwj:
    code review ACK 7356292e1d7a44da8a2bd31c02c58d550bf38009
  promag:
    Code review ACK 7356292e1d7a44da8a2bd31c02c58d550bf38009.

Tree-SHA512: 573662429523fd6a1af23dd907117320bc68cb51a93fba9483c9a2160bdce51fb590fcd97bcd2b2751d543d5c1148efa4e22e1c3901144f882b990ed2b450038
2024-03-18 16:01:41 +07:00
Wladimir J. van der Laan
a1c2386153
Merge #17445: zmq: Fix due to invalid argument and multiple notifiers
3e730bf90aaf53c41ff3a778f6aac15d163d1c0c zmq: Fix due to invalid argument and multiple notifiers (João Barbosa)

Pull request description:

  ZMQ initialization is interrupted if any notifier fails, and in that case all notifiers are destroyed. The notifier shutdown assumes that the initialization had occurred. This is not valid when there are multiple notifiers and any except the last fails to initialize.

  Can be tested by running test/functional/interface_zmq.py from this branch with bitcoind from master.

  Closes #17185.

ACKs for top commit:
  laanwj:
    Code review ACK 3e730bf90aaf53c41ff3a778f6aac15d163d1c0c, thanks for adding a test

Tree-SHA512: 5da710e97dcbaa94896d019e75162d470f6d381ee07c60e5b3e9db93d11e8f7ca9bf2c509efa4486199e88c96c3e720cc96b4e35b62725d4c7db8e8e9bf6e09d
2024-03-18 16:01:40 +07:00
MarcoFalke
44929bad82
Merge #16404: qa: Test ZMQ notification after chain reorg
abdfc5e89b687f73de4ab97e924c29cc27e71c15 qa: Test ZMQ notification after chain reorg (João Barbosa)
aa2622a726bc0f02152d79c888a332694678a989 qa: Refactor ZMQ test (João Barbosa)
6bc1ff915dd495f05985d3402a34dbfc3b6a08b4 doc: Add note regarding ZMQ block notification (João Barbosa)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: b93237adc8c84b3aa72ccc28097090eabcb006cf408083218bebf6fec703bd0de2ded80b6879e77096872e14ba9402a6d3f923b146a54d4c4e41dcb862c3e765
2024-03-18 16:01:40 +07:00
Samuel Dobson
eb4270deae
Merge #19743: -maxapsfee follow-up
7e31ea9fa0a59ced2293057acb14c71ec97db689 -maxapsfee: follow-up fixes (Karl-Johan Alm)
9f77b821764dcaebc97a5ae76c3052936418308d doc: release notes for -maxapsfee (Karl-Johan Alm)

Pull request description:

  Addresses feedback from jonatack and meshcollider in #14582.

ACKs for top commit:
  jonatack:
    ACK 7e31ea9fa0a
  meshcollider:
    re-ACK 7e31ea9fa0a59ced2293057acb14c71ec97db689

Tree-SHA512: 5d159d78e917e41140e1394bef05e821430dbeac585e9bd708f897041dd7104c2a6b563bfab8b2c85e6d923441160a3880d7864d768aa8e0e66860e0a2c4f56b
2024-03-18 16:01:39 +07:00
MarcoFalke
6a6d379711
Merge #19756: tests: add sync_all to fix race condition in wallet groups test
72ae20fc142457a200278cb2fedc5e32a3766b58 tests: add sync_all to fix race condition in wallet groups test (Karl-Johan Alm)

Pull request description:

  This most likely fixes #19749, the intermittent CI issues with wallet_groups.

  This fix is also included in #19743.

Top commit has no ACKs.

Tree-SHA512: dd6ef7f89829483e2278191c21fe0912b51fd2187c10a0fa158339c5ab9f22d93b733ae10f17ef25d8b64f44e596e66dba8d7db5c009343472f422ce4cd67d8f
2024-03-18 16:01:39 +07:00
Samuel Dobson
5821a1d23a
Merge #14582: wallet: always do avoid partial spends if fees are within a specified range
7f13dfb587dd6a7a5b7dfbfe689ae0ce818fe5c9 test: test the implicit avoid partial spends functionality (Karl-Johan Alm)
b82067bf696c53f22536f9ca2e51949c164f6b06 wallet: try -avoidpartialspends mode and use its result if fees are below threshold (Karl-Johan Alm)

Pull request description:

  The `-avoidpartialspends` feature is normally disabled, as it may affect the optimal fee for payments. This PR introduces a new parameter `-maxapsfee` (max avoid partial spends fee) which acts on the following values:
  * -1: disable partial spend avoidance completely (do not even try it)
  * 0: only do partial spend avoidance if fees are the same or better as the regular coin selection
  * 1..∞: use APS variant if the absolute fee difference is less than or equal to the max APS fee

  For values other than -1, the code will now try partial spend avoidance once, and if that gives a value within the accepted range, it will use that.

  Example: -maxapsfee=0.00001000 means the wallet will do regular coin select, APS coin select, and then pick AKS iff the absolute fee difference is <= 1000 satoshi.

  Edit: updated this to reflect the fact we are now using a max fee.

ACKs for top commit:
  fjahr:
    tested ACK 7f13dfb587dd6a7a5b7dfbfe689ae0ce818fe5c9
  achow101:
    ACK 7f13dfb587dd6a7a5b7dfbfe689ae0ce818fe5c9
  jonatack:
    ACK 7f13dfb58, code review, debug build, verified the test fails with `AssertionError: not(2 == 1)` for the number of vouts when `-maxapsfee=0.0001` is changed to 0, and verified the new logging with an added assertion.
  meshcollider:
    Code review ACK 7f13dfb587dd6a7a5b7dfbfe689ae0ce818fe5c9

Tree-SHA512: 475929df57f6191bb4e36bfbcad5a280a64bb0ecd8767b76cb2e44e2301235d0eb294a3f2fac5bbf15d35d7ecfba47acb2285feadb883c9ce31c08377e3afb3c
2024-03-18 16:01:38 +07:00
fanquake
2489f29f0e
Merge #19830: test: Add tsan supp for leveldb::DBImpl::DeleteObsoleteFiles
fa1fc536bb26471fd2a6fe8d12f98cf53c646306 test: Add tsan supp for leveldb::DBImpl::DeleteObsoleteFiles (MarcoFalke)

Pull request description:

  Fixes #19712

ACKs for top commit:
  practicalswift:
    ACK fa1fc536bb26471fd2a6fe8d12f98cf53c646306 -- patch looks correct
  hebasto:
    ACK fa1fc536bb26471fd2a6fe8d12f98cf53c646306

Tree-SHA512: 24d6a4e871fda11196a9f88e2ddbd1c1461d895c503a04b103791233e46638421836200eaaa7d70689564e51dee0d68d32b880dd90a5c259fb6a906f21d07853
2024-03-18 16:01:38 +07:00
MarcoFalke
10fa7a66b6
Merge #19538: ci: Add tsan suppression for race in DatabaseBatch
0cdf2a77ddfa1d53c6fbd830d557a3f20d7fc365 ci: add tsan debug symbols option (Russell Yanofsky)
9a2f12680b3f00a207f1cdd4e0c50a3c7613aefc ci: Add tsan suppression for race in DatabaseBatch (Hennadii Stepanov)

Pull request description:

  Since #19325 was merged, the corresponding change in TSan suppression file gets required.

  This PR is:
  - an analogous to #19226 and #19450, and
  - a temporary workaround for CI fail like https://cirrus-ci.com/task/5741795508224000?command=ci#L4993

ACKs for top commit:
  MarcoFalke:
    ACK 0cdf2a77ddfa1d53c6fbd830d557a3f20d7fc365

Tree-SHA512: 7832f143887c8a0df99dea03e00694621710378fbe923e3592185fcd3658546a590693b513abffc5ab96e9ef76c9c4bff3330eeee69a0c5dbe7574f34c417220
2024-03-18 16:01:37 +07:00
Konstantin Akimov
af9eb81e56
fix: wrong name of arguments for RPC 2024-03-17 13:02:59 -05:00
MarcoFalke
7ac1ee0fb4
Merge #19717: rpc: Assert that RPCArg names are equal to CRPCCommand ones (mining,zmq,rpcdump)
fa3d9ce3254882c545d700990fe8e9a678f31eed rpc: Assert that RPCArg names are equal to CRPCCommand ones (rpcdump) (MarcoFalke)
fa32c1d5ec25bc53bf989a8ae68e688593d2859d rpc: Assert that RPCArg names are equal to CRPCCommand ones (zmq) (MarcoFalke)
faaa46dc204d6d714f71dbc6f0bf02215dba0f0f rpc: Assert that RPCArg names are equal to CRPCCommand ones (mining) (MarcoFalke)
fa93bc14c7411a108dd024d391344fabf0f76369 rpc: Remove unused return type from appendCommand (MarcoFalke)

Pull request description:

  This is split out from #18531 to just touch the RPC methods in misc. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  fjahr:
    tested ACK fa3d9ce3254882c545d700990fe8e9a678f31eed
  promag:
    Code review ACK fa3d9ce3254882c545d700990fe8e9a678f31eed.

Tree-SHA512: 068ade4b55cc195868d53b7f9a27151d45b440857bb069e261a49d102a49a38fdba5d68868516a1d66a54a73ba34681362f934ded7349e894042bde873b75719
2024-03-17 13:02:58 -05:00
Wladimir J. van der Laan
860d31f504
Merge #19455: rpc generate: print useful help and error message
f0aa8aeea5a183ea44a877255d12db7732f2e0a8 test: add rpc_generate functional test (Jon Atack)
92d94ffb8d07cc0d2665c901de5903a3a90d5fd0 rpc: print useful help and error message for generate (Jon Atack)
8d32d2011d3f4e1d9e587d6f80dfa4a3e9f9393d test: consider generate covered in _get_uncovered_rpc_commands() (Jon Atack)

Pull request description:

  This was a requested follow-up to #19133 and #17700 to alleviate confusion and head-scratching by people following tutorials that use `generate`. See https://github.com/bitcoin/bitcoin/pull/19455#issuecomment-668172916 below, https://github.com/bitcoin/bitcoin/pull/19133#issuecomment-636860943 and https://github.com/bitcoin/bitcoin/pull/17700#issuecomment-566159096.

  before
  ```
  $ bitcoin-cli help generate
  help: unknown command: generate

  $ bitcoin-cli generate
  error code: -32601
  error message:
  Method not found
  ```

  after
  ```
  $ bitcoin-cli help generate
  generate ( nblocks maxtries ) has been replaced by the -generate cli option. Refer to -help for more information.

  $ bitcoin-cli generate
  error code: -32601
  error message:
  generate ( nblocks maxtries ) has been replaced by the -generate cli option. Refer to -help for more information.
  ```

  In the general help it remains hidden, as requested by laanwj.
  ```
  $ bitcoin-cli help

  == Generating ==
  generateblock "output" ["rawtx/txid",...]
  generatetoaddress nblocks "address" ( maxtries )
  generatetodescriptor num_blocks "descriptor" ( maxtries )
  ```

ACKs for top commit:
  adamjonas:
    utACK f0aa8aeea5a183ea44a877255d12db7732f2e0a8
  pinheadmz:
    ACK f0aa8aeea5a183ea44a877255d12db7732f2e0a8

Tree-SHA512: d083652589ad3e8228c733455245001db22397559c3946e7e573cf9bd01c46e9e88b72d934728ec7f4361436ae4c74adb8f579670b09f479011924357e729af5
2024-03-17 13:02:57 -05:00
MarcoFalke
58d923cd5b
Merge #19528: rpc: Assert that RPCArg names are equal to CRPCCommand ones (misc)
fa77de2baa40ee828c850ef4068c76cc3619e87b rpc: Assert that RPCArg names are equal to CRPCCommand ones (misc) (MarcoFalke)
fa50bdc755489b2e291ea5ba0e39e44a20c6c6de rpc: Limit echo to 10 args (MarcoFalke)
fa89ca9b5bd334813fd7e7edb202c56b35076e8d refactor: Use C++11 range based for loops to simplify rpc code (MarcoFalke)
fa459bdc87bbb050ca1c8d469023a96ed798540e rpc: Treat all args after a hidden arg as hidden as well (MarcoFalke)

Pull request description:

  This is split out from #18531 to just touch the RPC methods in misc. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  laanwj:
    Code review ACK fa77de2baa40ee828c850ef4068c76cc3619e87b
  fjahr:
    tested ACK fa77de2baa40ee828c850ef4068c76cc3619e87b
  theStack:
    ACK fa77de2baa
  ryanofsky:
    Code review ACK fa77de2baa40ee828c850ef4068c76cc3619e87b. Pretty straightfoward changes

Tree-SHA512: badae1606518c0b55ce2c0bb9025d14f05556532375eb20fd6f3bfadae1e5e6568860bff8599d037e655bf1d23f1f464ca17f4db10a6ab3d502b6e9e61c7b3d3
2024-03-17 13:02:57 -05:00
pasta
c23514dc49
Merge #5823: backport: bitcoin#19200, #19405, #20282
9c54cb16de Merge #19405: rpc, cli: add network in/out connections to `getnetworkinfo` and `-getinfo` (Wladimir J. van der Laan)
19aba38cab Merge #19200: rpc: remove deprecated getaddressinfo fields (Samuel Dobson)
f5642281cc Merge #20282: wallet: change upgradewallet return type to be an object (Samuel Dobson)

Pull request description:

  ## Issue being fixed or feature implemented
  Bitcoin backports with breaking changes

  ## What was done?
   - bitcoin/bitcoin#20282
   - bitcoin/bitcoin#19200
   - bitcoin/bitcoin#19405

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  ### RPC:
  - `upgradewallet` now returns object for future extensibility (#20282)

  - `getnetworkinfo` now returns fields `connections_in`, `connections_out`,
    `connections_mn_in`, `connections_mn_out`, `connections_mn`
    that provide the number of inbound and outbound peer
    connections. These new fields are in addition to the existing `connections`
    field, which returns the total number of peer connections. Old fields
    `inboundconnections`, `outboundconnections`, `inboundmnconnections`,
    `outboundmnconnections` and `mnconnections` are removed (#19405)

  - Backwards compatibility has been dropped for two `getaddressinfo` RPC
    deprecations, as notified in the 19.1.0 and 19.2.0 release notes.
    The deprecated `label` field has been removed as well as the deprecated `labels` behavior of
    returning a JSON object containing `name` and `purpose` key-value pairs. Since
    20.1, the `labels` field returns a JSON array of label names. (#19200)

  ### CLI

  - The `connections` field of `bitcoin-cli -getinfo` is expanded to return a JSON
    object with `in`, `out` and `total` numbers of peer connections and `mn_in`,
    `mn_out` and `mn_total` numbers of verified mn connections. It previously
    returned a single integer value for the total number of peer connections. (#19405)

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 9c54cb16de

Tree-SHA512: 9710cdd062d02d64e2eebcecca1b5c2e6ccda5ca6d9bd6d1833700f4273fcfb206ae99134f71bbc8b0843cb8ebba208c72139f5a624d79ec7362bd73b117bfb2
2024-03-17 12:58:46 -05:00
Wladimir J. van der Laan
9c54cb16de
Merge #19405: rpc, cli: add network in/out connections to getnetworkinfo and -getinfo
581b343d5bf517510ab0236583ca96628751177d Add in/out connections to cli -getinfo (Jon Atack)
d9cc13e88d096c1a171159c01cbb96444f7f8d7f UNIX_EPOCH_TIME fixup in rpc getnettotals (Jon Atack)
1ab49b81cf32b6ef9e312a0a8ac45c68a3262f0d Add in/out connections to rpc getnetworkinfo (Jon Atack)

Pull request description:

  This is basic info that is present in the GUI that I've been wishing to have exposed via the RPC and CLI without needing a bash workaround or script. For human users it would also be useful to have it in `-getinfo`.

  `bitcoin-cli getnetworkinfo`
  ```
    "connections": 15,
    "connections_in": 6,
    "connections_out": 9,
  ```

  `bitcoin-cli -getinfo`
  ```
    "connections": {
      "in": 6,
      "out": 9,
      "total": 15
    },
  ```

  Update the tests, RPC help, and release notes for the changes. Also fixup the `getnettotals` timemillis help while touching `rpc/net.cpp`.

  -----

  Reviewers can manually test this PR by [building from source](https://jonatack.github.io/articles/how-to-compile-bitcoin-core-and-run-the-tests), launching bitcoind, and then running `bitcoin-cli -getinfo`, `bitcoin-cli getnetworkinfo`, `bitcoin-cli help getnetworkinfo`, and `bitcoin-cli help getnettotals` (for the UNIX epoch time change).

ACKs for top commit:
  eriknylund:
    > tACK [581b343](581b343d5b) on master at [a0a422c](a0a422c34c), ran unit & functional tests and and confirmed changes on an existing datadir ✌️
  benthecarman:
    tACK `581b343`
  willcl-ark:
    tACK for 581b343d5bf517510ab0236583ca96628751177d, this time rebased onto master at 862fde88be706adb20a211178253636442c3ae00.
  shesek:
    tACK `581b343`. This provides what I needed, thanks!
  n-thumann:
    tACK 581b343 on master at a0a422c, ran unit & functional tests and and confirmed changes on an existing datadir ✌️

Tree-SHA512: 08dd3ac8fefae401bd8253ff3ac027603c528eeccba53cedcb127771316173a7052fce44af8fa33ac98ebc4cf2a2b11cdefd949995d55e9b9a5942b876d00dc5
2024-03-16 02:39:45 +07:00
Samuel Dobson
19aba38cab
Merge #19200: rpc: remove deprecated getaddressinfo fields
BACKPORT NOTICE:
These backports #17578 and #17585 are included to 19.1 and 19.2. That's long enough!
------------------------------------------
bc01f7ae0538d3c647ce8dfbc29f7914d5df3fbb doc: release note for rpc getaddressinfo removals (Jon Atack)
90e989390ee50633fff0e4f210a1ea23ff00e012 rpc: getaddressinfo RPCResult fixup (Jon Atack)
a8507c99da10791aa69ca277128e06753942e976 rpc: remove deprecated getaddressinfo `labels: purpose` (Jon Atack)
645a8653c895e4fc7717e9e5ac045612b5deaa60 rpc: remove deprecated getaddressinfo `label` field (Jon Atack)

Pull request description:

  These were deprecated in #17578 and #17585, with expected 0.21 removal notified in the 0.20 release notes.
  ```
  - The `getaddressinfo` RPC has had its `label` field deprecated
    (re-enable for this release using the configuration parameter
    `-deprecatedrpc=label`).  The `labels` field is altered from returning
    JSON objects to returning a JSON array of label names (re-enable
    previous behavior for this release using the configuration parameter
    `-deprecatedrpc=labelspurpose`).  Backwards compatibility using the
    deprecated configuration parameters is expected to be dropped in the
    0.21 release.  (#17585, #17578)
  ```

ACKs for top commit:
  Sjors:
    utACK bc01f7a
  adamjonas:
    utACK bc01f7a
  meshcollider:
    utACK bc01f7ae0538d3c647ce8dfbc29f7914d5df3fbb

Tree-SHA512: ae1af381e32c4c3bde8b061a56382838513a9a82c88767843cdeae3a2ab8aa7d8c2e66e106d2b31ea07d74bb80c191a2f842c9aaecc7c5438ad9a9bc66d1b251
2024-03-16 02:39:42 +07:00
pasta
2110c0c309
Merge #5908: refactor: move masternode payments processing to helper class, move C{Governance,Spork}Manager to NodeContext
a93de8690b refactor: s/governanceManager/govman/g (Kittywhiskers Van Gogh)
0aa08ba80d refactor: remove CGovernanceManager global, move to NodeContext (Kittywhiskers Van Gogh)
405b8c669a refactor: s/sporkManager/sporkman/g (Kittywhiskers Van Gogh)
60fd1aa774 refactor: remove CSporkManager global, move to NodeContext (Kittywhiskers Van Gogh)
24ba2f027c refactor: remove redundant condition check in `IsOldBudgetBlockValueValid` (Kittywhiskers Van Gogh)
e2405e67fb refactor: move MasternodePayments::* functions into helper class (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * `PeerManager`'s initialization has been moved downwards (it used to be initialized _after_ `CGovernanceManager`) to now _after_ `CMasternodeSync`. This is to avoid having to pass `CSporkManager`'s `unique_ptr` container and instead pass the its dereferenced pointer.
  * `CChainstateHelper` is just proxy for helper classes meant to hold references to managers that would be needed by functions that are called by `CChainState`. It's the alternative to passing every single manager into `CChainState` through `ChainstateManager`.

    Instead, they're all bunched up via `CChainstateHelper` and is accessible to `CChainState` through passing it as an argument. For this reason, it should ideally initialized _after_ all relevant managers are setup but _before_ the chain is validated. We would want to avoid deferred dereferencing if we can help it.
    * Internal/private functions have been marked as `[[nodiscard]]`.

  ## Breaking Changes

  None. Changes are limited to refactoring, no logical changes have been made.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK a93de8690b

Tree-SHA512: b576ca61a94b86b8a6fa8909379d156ff902198b5824fcfb4665e5eb9d1d5e250db737f84a877de634a490146b82759c2350370903f430997423fd71142106d1
2024-03-15 12:21:26 -05:00
Kittywhiskers Van Gogh
e2405e67fb
refactor: move MasternodePayments::* functions into helper class
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
2024-03-14 03:29:04 +00:00
fanquake
85fa6e4585
Merge bitcoin/bitcoin#22446: test: Fix wallet_listdescriptors.py if bdb is not compiled
0c845e3f8995eb8dc543a63899e5633a46091b4e test: Fix wallet_listdescriptors.py if bdb is not compiled (Hennadii Stepanov)

Pull request description:

  If build system is configured `--without-bdb`, the `wallet_listdescriptors.py` fails:
  ```
  $ test/functional/wallet_listdescriptors.py --descriptors
  2021-07-14T13:20:52.931000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_02p7o1c9
  2021-07-14T13:21:23.377000Z TestFramework (INFO): Test that the command is not available for legacy wallets.
  2021-07-14T13:21:23.381000Z TestFramework (ERROR): JSONRPC error
  Traceback (most recent call last):
    File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/test_framework.py", line 128, in main
      self.run_test()
    File "test/functional/wallet_listdescriptors.py", line 34, in run_test
      node.createwallet(wallet_name='w1', descriptors=False)
    File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/test_node.py", line 685, in createwallet
      return self.__getattr__('createwallet')(wallet_name, disable_private_keys, blank, passphrase, avoid_reuse, descriptors, load_on_startup, external_signer)
    File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__
      raise JSONRPCException(response['error'], status)
  test_framework.authproxy.JSONRPCException: Compiled without bdb support (required for legacy wallets) (-4)
  2021-07-14T13:21:23.436000Z TestFramework (INFO): Stopping nodes
  2021-07-14T13:21:24.092000Z TestFramework (WARNING): Not cleaning up dir /tmp/bitcoin_func_test_02p7o1c9
  2021-07-14T13:21:24.092000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/bitcoin_func_test_02p7o1c9/test_framework.log
  2021-07-14T13:21:24.092000Z TestFramework (ERROR):
  2021-07-14T13:21:24.092000Z TestFramework (ERROR): Hint: Call /home/hebasto/GitHub/bitcoin/test/functional/combine_logs.py '/tmp/bitcoin_func_test_02p7o1c9' to consolidate all logs
  2021-07-14T13:21:24.092000Z TestFramework (ERROR):
  2021-07-14T13:21:24.092000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
  2021-07-14T13:21:24.092000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
  2021-07-14T13:21:24.092000Z TestFramework (ERROR):
  ```

  This PR fixes this issue.

  Also see #20267.

ACKs for top commit:
  achow101:
    ACK 0c845e3f8995eb8dc543a63899e5633a46091b4e

Tree-SHA512: d7116a9ae30c7b7e3f55f55d2eea66f9e293c38d6757ed66d0477e4256ff5fedca5ddedafa0ef0c09f4dc1f7f973163e5a46090da26b067fdddbd9ea2ee76633
2024-03-09 03:01:27 +07:00
Wladimir J. van der Laan
953b6706f4
Merge #18788: tests: Update more tests to work with descriptor wallets
c7b7e0a69265946aecc885be911c7650911ba2e3 tests: Make only desc wallets for wallet_multwallet.py --descriptors (Andrew Chow)
d4b67ad214ada7645c4ce2d5ec336fe5c3f7f7ca Avoid creating legacy wallets in wallet_importdescriptors.py (Andrew Chow)
6c9c12bf87f95066acc28ea2270a00196eb77703 Update feature_backwards_compatibility for descriptor wallets (Andrew Chow)
9a4c631e1c00eb1661c000978b133d7aa0226290 Update wallet_labels.py to not require descriptors=False (Andrew Chow)
242aed7cc1d003e8fed574bbebd19c7e54e23402 tests: Add a --legacy-wallet that is mutually exclusive with --descriptors (Andrew Chow)
388053e1722632c2e485c56a444bc75cf0152188 Disable some tests for tool_wallet when descriptors (Andrew Chow)
47d3243160fdec7e464cfb8f869be7f5d4ee25fe Make raw multisig tests legacy wallet only in rpc_rawtransaction.py (Andrew Chow)
59d3da5bce4ebd9c2291d8f201a53ee087938b21 Do addmultisigaddress tests in legacy wallet mode in wallet_address_types.py (Andrew Chow)
25bc5dccbfd52691adca6edd418dd54290300c28 Use importdescriptors when in descriptor wallet mode in wallet_createwallet.py (Andrew Chow)
0bd1860300b13b12a25d330ba3a93ff2d13aa379 Avoid dumpprivkey and watchonly behavior in rpc_signrawtransaction.py (Andrew Chow)
08067aebfd7e838e6ce6b030c31a69422260fc6f Add script equivalent of functions in address.py (Andrew Chow)
86968882a8a26312a7af29c572313c4aff488c11 Add descriptor wallet output to tool_wallet.py (Andrew Chow)
3457679870e8eff2a7d14fe59a479692738c48b6 Use separate watchonly wallet for multisig in feature_nulldummy.py (Andrew Chow)
a42652ec10c733a5bf37e418e45d4841f54331b4 Move import and watchonly tests to be legacy wallet only in wallet_balance.py (Andrew Chow)
4b871909d6e4a51888e062d322bf53263deda15e Use importdescriptors for descriptor wallets in wallet_bumpfee.py (Andrew Chow)
c2711e4230d9a423ead24f6609691fb338b1d26b Avoid dumpprivkey in wallet_listsinceblock.py (Andrew Chow)
553dbf9af4dea96e6a3e79bba9607003342029bd Make import tests in wallet_listtransactions.py legacy wallet only (Andrew Chow)
dc81418fd01021070f3f66bab5fee1484456691a Use a separate watchonly wallet in rpc_fundrawtransaction.py (Andrew Chow)
a357111047411f18c156cd34a002a38430f2901c Update wallet_importprunedfunds to avoid dumpprivkey (Andrew Chow)

Pull request description:

  I went through all the tests and checked whether they passed with descriptor wallets. This partially informed some changes in #16528. Some tests needed changes to work with descriptor wallets. These were primarily due to import and watchonly behavior. There are some tests and test cases that only test legacy wallet behavior so those tests won't be run with descriptor wallets.

  This PR updates more tests to have to the `--descriptors` switch in `test_runner.py`. Additionally a mutually exclusive `--legacy-wallet` option has been added to force legacy wallets. This does nothing currently but will be useful in the future when descriptor wallets are the default. For the tests that rely on legacy wallet behavior, this option is being set so that we don't forget in the future. Those tests are `feature_segwit.py`, `wallet_watchonly.py`, `wallet_implicitsegwit.py`, `wallet_import_with_label.py`, and `wallet_import_with_label.py`.

  If you invert the `--descriptors`/`--legacy-wallet` default so that descriptor wallets are the default, all tests (besides the legacy wallet specific ones) will pass.

ACKs for top commit:
  MarcoFalke:
    review ACK c7b7e0a69265946aecc885be911c7650911ba2e3 🎿
  laanwj:
    ACK c7b7e0a69265946aecc885be911c7650911ba2e3

Tree-SHA512: 2f4e87815005d1d0a2543ea7947f7cd7593d8cf5312228ef85f8e096f19739b225769961943049cb44f6f07a35b8de988e2246ab9aca5bb5a0b2e62694d5637d
2024-03-09 03:01:24 +07:00
Samuel Dobson
56d9fe9510
Merge #20226: wallet, rpc: add listdescriptors command
647b81b70938dc4dbcf32399c56f78be395c721a wallet, rpc: add listdescriptors command (Ivan Metlushko)

Pull request description:

  Looking for concept ACKs

  **Rationale**: allow users to inspect the contents of their newly created descriptor wallets.

  Currently the command only returns xpubs which is not very useful in itself, but there are multiples ways to extend it:
   * add an option to export xprv
   * with #19136 it'll be possible to return normalised descriptors suitable for a watch-only purposes

  The output is compatible with `importdescriptors` command so it could be easily used for backup/recover purposes.

  **Output example:**
  ```json
  [
    {
      "desc": "wpkh(tpubD6NzVbkrYhZ4WW6E2ZETFyNfq2hfF23SKxqSGFvUpPAY58jmmuBybwqwFihAyQPk9KnwTt5516NDZRJ7k5QPeKjy7wuVd5WvXNxwwAs5tUD/*)#nhavpr5h",
      "timestamp": 1296688602,
      "active": false,
      "range": [
        0,
        999
      ],
      "next": 0
    }
  ]
  ```

ACKs for top commit:
  jonatack:
    re-ACK 647b81b70938dc4dbcf32399c56f78be395c721a rebased to master, debug builds cleanly, reviewed diff since last review, tested with a descriptor wallet (and with a legacy wallet)
  achow101:
    re-ACK 647b81b

Tree-SHA512: 51a3620bb17c836c52cecb066d4fa9d5ff418af56809046eaee0528c4dc240a4e90fff5711ba96e399c6664e00b9ee8194e33852b1b9e75af18061296e19a8a7
2024-03-09 03:00:29 +07:00
Samuel Dobson
9600f9ca35
Merge #19441: walletdb: don't reinitialize desc cache with multiple cache entries
a66a7a1a7060bb422eba3b8c214852416c4280d1 walletdb: don't reinitialize desc cache with multiple cache entries (Andrew Chow)

Pull request description:

  When loading descriptor caches, we would accidentally reinitialize the descriptor cache when seeing that one already exists. This should have only been initializing the cache when one does not exist. However this code itself is unnecessary as the act of looking up the cache to add to it will initialize it if it didn't already exist.

  This issue could be hit by trying to load a wallet that had imported a multisig descriptor. The wallet would fail to load.

  A test has been added to wallet_importdescriptors.py to catch this case. Another test case has also been added to check that loading a wallet with only single key descriptors works.

ACKs for top commit:
  hugohn:
    tACK [a66a7a1](a66a7a1a70)
  jonatack:
    ACK a66a7a1a706
  meshcollider:
    Code review ACK a66a7a1a7060bb422eba3b8c214852416c4280d1

Tree-SHA512: 3df746421a008708eaa3bbbdd12b9ddd3e2ec111d54625a212dca7414b971cc1f6e2b1757b3232c31a2f637d1b1ef43bf3ffa4ac4216646cf1e92db5f79954f1
2024-03-09 03:00:28 +07:00