Commit Graph

26538 Commits

Author SHA1 Message Date
pasta
d16e95352c
Merge #6233: fix: release unused memory in CNetMsgMaker::Make()
c2c4b2b794 fix: release unused memory in `CNetMsgMaker::Make()` (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  We reserve capacity for large messages in `CNetMsgMaker::Make()` but most messages are small yet we never release unused memory here. Discovered while debugging 28165 backport issues.

  ## What was done?

  ## How Has This Been Tested?

  ## Breaking Changes
  n/a

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone

ACKs for top commit:
  kwvg:
    ACK c2c4b2b794
  PastaPastaPasta:
    utACK c2c4b2b794
  knst:
    utACK c2c4b2b794

Tree-SHA512: 72a3728e316fb76ca135fbfff4f15b80c60b048d7e916d2f5dbcde4b64dce7177af80f8f24153636ccfab2f5d5c7d9edc5b3bdba121f64c8da03117a98fd7411
2024-08-30 22:08:23 -05:00
pasta
cddbc2a87b
Merge #6240: backport: partial bitcoin#27920, #28799, #29007 - to improve CI experience
2eadcf2f68 partial Merge bitcoin/bitcoin#29007: test: create deterministic addrman in the functional tests (stratospher)
f95ca4ed3e fix: unify with bitcoin: removed requirement of txindex=0 (Konstantin Akimov)
bf46e7a8e1 partial Merge bitcoin/bitcoin#28799: wallet: cache descriptor ID to avoid repeated descriptor string creation (Andrew Chow)
05e5966199 partial Merge bitcoin/bitcoin#27920: wallet: bugfix, always use apostrophe for spkm descriptor ID (furszy)

Pull request description:

  ## Issue being fixed or feature implemented
  Lately our CI for tsan is flapping many functional tests and take long times.

  This PR has several important changes backported from the latest bitcoin's version to improve CI experience

  ## What was done?
  This PR has several backports that improved CI experience drastically.

  **Firstly, it aims to fix flapping test p2p_node_network_limited.py**

  For example: https://gitlab.com/dashpay/dash/-/jobs/7692635307

  <details>

  <summary>p2p_node_network_limited.py                        | ✖ Failed  | 28 s</summary>

  ```
   test  2024-08-29T02:50:53.929000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
  File "/builds/dashpay/dash/build-ci/dashcore-linux64_tsan/test/functional/test_framework/test_framework.py", line 158, in main
    self.run_test()
  File "/builds/dashpay/dash/build-ci/dashcore-linux64_tsan/test/functional/p2p_node_network_limited.py", line 79, in run_test
    self.nodes[0].disconnect_p2ps()
  File "/builds/dashpay/dash/build-ci/dashcore-linux64_tsan/test/functional/test_framework/test_node.py", line 611, in disconnect_p2ps
    wait_until_helper(check_peers, timeout=5)
  File "/builds/dashpay/dash/build-ci/dashcore-linux64_tsan/test/functional/test_framework/util.py", line 262, in wait_until_helper
    raise AssertionError("Predicate {} not true after {} seconds".format(predicate_source, timeout))
    AssertionError: Predicate ''''
         def check_peers():
             for p in self.getpeerinfo():
                 for p2p in self.p2ps:
                     if p['subver'] == p2p.strSubVer:
                         return False
             return True
  ''' not true after 5.0 seconds
  ```
  </details>

  **Secondly, it improves performance of Descriptor wallets significantly for case of `tsan` CI**. It is tiny improvement for Release build and local runs, but some fucnctional tests run as fast as twice:

  https://gitlab.com/dashpay/dash/-/jobs/7694458953
  https://gitlab.com/dashpay/dash/-/jobs/7665132625

  ```
  wallet_create_tx.py --descriptors                  | ✓ Passed  | 236 s <-- new version
  wallet_create_tx.py --legacy-wallet                | ✓ Passed  | 108 s
  wallet_basic.py --descriptors                      | ✓ Passed  | 135 s <---- new version
  wallet_basic.py --legacy-wallet                    | ✓ Passed  | 97 s

  wallet_create_tx.py --descriptors                  | ✓ Passed  | 456 s <-- old version
  wallet_create_tx.py --legacy-wallet                | ✓ Passed  | 98 s
  wallet_basic.py --descriptors                      | ✓ Passed  | 189 s <--- old version
  wallet_basic.py --legacy-wallet                    | ✓ Passed  | 131 s
  ```
  See performance investigation here: https://github.com/dashpay/dash/pull/6226

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 2eadcf2f68
  UdjinM6:
    utACK 2eadcf2f68

Tree-SHA512: 127fbaa65c160aa95e2145a6b40d3811f7c42e36fbee9ce98a9ac021abd9cbe6edc7791870b331a54855ba891e3804885db7936ef212647b693f50f79a60d232
2024-08-30 10:32:20 -05:00
pasta
e3a5afd6a6
Merge #6064: backport: Merge bitcoin#22568, 21800, (partial)22707
bda74458b5 (partial) Merge bitcoin/bitcoin#22707: test: refactor use of getrawmempool in functional tests for efficiency (MarcoFalke)
1a8268770c Merge bitcoin/bitcoin#21800: mempool/validation: mempool ancestor/descendant limits for packages (fanquake)
97fd2b2842 Merge bitcoin/bitcoin#22568: test: add addr-fetch peer connection state and timeout coverage (MarcoFalke)

Pull request description:

  bitcoin backports

ACKs for top commit:
  UdjinM6:
    utACK bda74458b5
  PastaPastaPasta:
    utACK bda74458b5

Tree-SHA512: a001f790b36df8e6c640c383a302ce47a6add63f39561596a670fd8261b9a461d71b31b6457a78706a2f596134705879f5f4ba142a79ee587eba02302f57be71
2024-08-29 12:44:50 -05:00
stratospher
2eadcf2f68
partial Merge bitcoin/bitcoin#29007: test: create deterministic addrman in the functional tests
BACKPORT NOTICE
It includes only this commit: be25ac3092b7755e26e1ec6c33a27cd0e3dd9eac

[init] Remove -addrmantest command line arg

-addrmantest is only used in `p2p_node_network_limited.py` test to
test if the node self-advertises a hard-coded local address
(which wouldn't be advertised in the tests because it's unroutable
without the test-only code path) to check pruning-related services
are correct in that addr.

Remove -addrmantest because the self advertisement happens because
of hard coded test path logic, and expected services are nominal
due to how easily the test-only code could diverge from mainnet
logic. It's also being used only in 1 test.
2024-08-29 17:23:40 +07:00
Konstantin Akimov
f95ca4ed3e
fix: unify with bitcoin: removed requirement of txindex=0 2024-08-29 17:23:40 +07:00
Andrew Chow
bf46e7a8e1
partial Merge bitcoin/bitcoin#28799: wallet: cache descriptor ID to avoid repeated descriptor string creation
BACKPORT NOTICE:
It doesn't include changes related to wallet_miniscript.py

5e6bc6d830664a5afeb5d5bd7e7b3818a01376b7 test: remove custom rpc timeout for `wallet_miniscript.py`, reorder in test_runner (Sebastian Falbesoner)
f811a24421fe102a96ab8f75427cc6a3c5503dc3 wallet: cache descriptor ID to avoid repeated descriptor string creation (Sebastian Falbesoner)

Pull request description:

  Right now a wallet descriptor is converted to its string representation (via `Descriptor::ToString`) repeatedly at different instances:
  - on finding a `DescriptorScriptPubKeyMan` for a given descriptor (`CWallet::GetDescriptorScriptPubKeyMan`, e.g. used by the `importdescriptors` RPC); the string representation is created once for each spkm in the wallet and at each iteration again for the searched descriptor (`DescriptorScriptPubKeyMan::HasWalletDescriptor`)
  - whenever `DescriptorScriptPubKeyMan::GetID()` is called, e.g. in `TopUp` or any instances where a descriptor is written to the DB to determine the database key, also at less obvious places like `FastWalletRescanFilter` etc.

  As there is no good reason to calculate a fixed descriptor's string/ID more than once, add the ID as a field to `WalletDescriptor` and calculate it immediately at initialization (or deserialization). `HasWalletDescriptor` is changed to compare the spkm's and searched descriptor's ID instead of the string to take use of that.

  This speeds up the functional test `wallet_miniscript.py` by a factor of 5-6x on my machine (3m30.95s on master vs. 0m38.02s on PR). The recently introduced "max-size TapMiniscript" test-case introduced a descriptor that takes 2-3 seconds to create a string representation, so the repeated calls to that were significantly hurting the performance.

  Fixes https://github.com/bitcoin/bitcoin/issues/28800.

ACKs for top commit:
  Sjors:
    ACK 5e6bc6d830664a5afeb5d5bd7e7b3818a01376b7
  S3RK:
    Code Review ACK 5e6bc6d830664a5afeb5d5bd7e7b3818a01376b7
  achow101:
    ACK 5e6bc6d830664a5afeb5d5bd7e7b3818a01376b7
  BrandonOdiwuor:
    ACK 5e6bc6d830664a5afeb5d5bd7e7b3818a01376b7

Tree-SHA512: 98b43963a5dde6055bb26cecd3b878dadd837d6226af4c84142383310495da80b3c4bd552e73b9107f2f2ff1c11f5e18060c6fd3d9e44bbd5224114c4d245c1c
2024-08-29 17:23:40 +07:00
furszy
05e5966199
partial Merge bitcoin/bitcoin#27920: wallet: bugfix, always use apostrophe for spkm descriptor ID
BACKPORT NOTICE

It includes only this commit: 97a965d98f1582ea1b1377bd258c9088380e1b8b
refactor: extract descriptor ID calculation from spkm GetID()

This allows us to verify the descriptor ID on the descriptors
unit tests in different software versions without requiring to
use the entire DescriptorScriptPubKeyMan machinery.

Note:
The unit test changes are introduced after the bugfix commit
but this commit + the unit test commit can be cherry-picked
on top of the v25 branch to verify IDs correctness. IDs must
be the same for v25 and after the bugfix commit.
2024-08-29 17:23:40 +07:00
MarcoFalke
bda74458b5
(partial) Merge bitcoin/bitcoin#22707: test: refactor use of getrawmempool in functional tests for efficiency
47c48b5f35b4910fcf87caa6e37407e67d879c80 test: only use verbose for getrawmempool when necessary in functional tests (Michael Dietz)
77349713b189e80f2c140db4df50177353a1cb83 test: use getmempoolentry instead of getrawmempool in functional tests when appropriate (Michael Dietz)
86dbd54ae8a8f9c693c0ea67114bbff24a0754df test: improve mempool_updatefrom efficiency by using getmempoolentry for specific txns (Michael Dietz)

Pull request description:

  I don't think this changes the intention of the test. But it does shave ~30 seconds off the time it takes to run. From what I've seen our CI `macOS 11 native [gui] [no depends]` runs `mempool_updatefrom.py` in ~135 seconds. After this PR it should run in ~105 seconds

  I noticed this improvement should probably be made when testing performance/runtimes of https://github.com/bitcoin/bitcoin/pull/22698. But I wanted to separate this out from that PR so the affects of each is decoupled

  Edit: The major change in this PR is improving mempool_updatefrom.py's runtime as this is a very long running test. Then made the same efficiency improvements across all the functional tests as it made since to do that here

ACKs for top commit:
  theStack:
    Tested ACK 47c48b5f35b4910fcf87caa6e37407e67d879c80

Tree-SHA512: 40f553715f3d4649dc18c2738554eafaca9ea800c4b028c099217896cc1c466ff457ae814d59cf8564c782a8964d8fac3eda60c1b6ffb08bbee1439b2d34434b
Signed-off-by: Vijay <vijaydas.mp@gmail.com>
2024-08-29 10:01:34 +05:30
fanquake
1a8268770c
Merge bitcoin/bitcoin#21800: mempool/validation: mempool ancestor/descendant limits for packages
accf3d5868460b4b14ab607fd66ac985b086fbb3 [test] mempool package ancestor/descendant limits (glozow)
2b6b26e57c24d2f0abd442c1c33098e3121572ce [test] parameterizable fee for make_chain and create_child_with_parents (glozow)
313c09f7b7beddfdb74c284720d209c81dfdb94f [test] helper function to increase transaction weight (glozow)
f8253d69d6f02850995a11eeb71fedc22e6f6575 extract/rename helper functions from rpc_packages.py (glozow)
3cd663a5d33aa7ef87994e452bced7f192d021a0 [policy] ancestor/descendant limits for packages (glozow)
c6e016aa139c8363e9b38bbc1ba0dca55700b8a7 [mempool] check ancestor/descendant limits for packages (glozow)
f551841d3ec080a2d7a7988c7b35088dff6c5830 [refactor] pass size/count instead of entry to CalculateAncestorsAndCheckLimits (glozow)
97dd1c729d2bbedf9527b914c0cc8267b8a7c21b MOVEONLY: add helper function for calculating ancestors and checking limits (glozow)
f95bbf58aaf72aab8a9c5827b1f162f3b8ac38f4 misc package validation doc improvements (glozow)

Pull request description:

  This PR implements a function to calculate mempool ancestors for a package and enforces ancestor/descendant limits on them as a whole. It reuses a portion of `CalculateMemPoolAncestors()`; there's also a small refactor to move the reused code into a generic helper function. Instead of calculating ancestors and descendants on every single transaction in the package and their ancestors, we use a "worst case" heuristic, treating every transaction in the package as each other's ancestor and descendant. This may overestimate everyone's counts, but is still pretty accurate in the our main package use cases, in which at least one of the transactions in the package is directly related to all the others (e.g. 1 parent + 1 child, multiple parents with 1 child, or chains).

  Note on Terminology: While "package" is often used to describe groups of related transactions _within_ the mempool, here, I only use package to mean the group of not-in-mempool transactions we are currently validating.

  #### Motivation

  It would be a potential DoS vector to allow submission of packages to mempool without a proper guard for mempool ancestors/descendants. In general, the purpose of mempool ancestor/descendant limits is to limit the computational complexity of dealing with families during removals and additions. We want to be able to validate multiple transactions on top of the mempool, but also avoid these scenarios:

  - We underestimate the ancestors/descendants during package validation and end up with extremely complex families in our mempool (potentially a DoS vector).
  - We expend an unreasonable amount of resources calculating everyone's ancestors and descendants during package validation.

ACKs for top commit:
  JeremyRubin:
    utACK accf3d5
  ariard:
    ACK accf3d5.

Tree-SHA512: 0d18ce4b77398fe872e0b7c2cc66d3aac2135e561b64029584339e1f4de2a6a16ebab3dd5784f376e119cbafc4d50168b28d3bd95d0b3d01158714ade2e3624d
Signed-off-by: Vijay <vijaydas.mp@gmail.com>
2024-08-29 10:01:28 +05:30
MarcoFalke
97fd2b2842
Merge bitcoin/bitcoin#22568: test: add addr-fetch peer connection state and timeout coverage
f8d8eb5fdaa93b6e5b77fd901b94927dc3a0473e test: add addr-fetch timeout connection coverage in p2p_addrfetch.py (Jon Atack)
9321086af7b75be536767d25abef4d7e02ca416a test: add assert_getpeerinfo method and coverage in p2p_addrfetch.py (Jon Atack)

Pull request description:

  This patch adds additional addr-fetch peer connection state and timeout coverage as a follow-up to #22096.

ACKs for top commit:
  Saviour1001:
    Tested ACK <code>[f8d8eb5](f8d8eb5fda)</code>
  mzumsande:
    Code review ACK f8d8eb5fdaa93b6e5b77fd901b94927dc3a0473e

Tree-SHA512: 9a13a705d1da6b308d6dcbc6930575205e2e88bfe9f2e7cb4e0c4c40d26538430e6b02c6c772d0cee64e534777348291469a139f99afbf9d4f93f31b9e7b0818
Signed-off-by: Vijay <vijaydas.mp@gmail.com>
2024-08-29 08:07:41 +05:30
pasta
8d3f31306e
Merge #6039: fix: optimize compilation time of gsl library
dc59cbc315 fix: optimize includes in gsl/pointer.h to speed up compile time (Konstantin Akimov)
d5e32e1b79 fix: disable gsl iostream feature for faster compile time (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  This header `gsl/pointers.h` is included in multiple other headers all over codebase.
  Any extra line of code inside gsl/pointers.h makes all project to compile slower.

  ## What was done?
  Removed headers `<algorithm>`, `<system_error>`, `<iosfwd>` from gsl/pointers.h

  ## How Has This Been Tested?
  Run command 5 times, takes minimum time:

  - baseline
  ```
  $ time g++ -o /tmp/a.out gsl/pointers.h -O2 -g -I.
  real    0m0,572s
  user    0m0,461s
  sys     0m0,108s
  ```

   - removed algorithm:
  ```
  $ time g++ -o /tmp/a.out gsl/pointers.h -O2 -g -I.
  real    0m0,505s
  user    0m0,398s
  sys     0m0,107s
  ```

   - removed algorithm and system_error:
  ```
  real    0m0,332s
  user    0m0,265s
  sys     0m0,067s
  ```

   - disabled iostream:
  ```
  $ time g++ -o /tmp/a.out gsl/pointers.h -O2 -g -I. -D GSL_NO_IOSTREAMS
  real    0m0,316s
  user    0m0,256s
  sys     0m0,060s
  ```

  as a result, overall project compilation time is also improved: `make clean ; sleep 3s ; time make -j20`
  ```
  real    5m42,934s
  user    80m35,127s
  sys     6m40,735s
  ```
  ```
  real    5m28,862s
  user    75m31,931s
  sys     6m32,591s
  ```

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  UdjinM6:
    utACK dc59cbc315

Tree-SHA512: 353d79c0e297c92e823972b53daaaed42494554ce550ea800ce8aa6990c704cedfbcb922e1d735eb8fc01ad14ffa873d86cdbc4d3731d841496392bb74286d33
2024-08-28 19:31:55 -05:00
pasta
a40802b1fc
Merge #6156: fix: let internal govobj/vote inv request trackers expire after 60 sec
1f4e1a17ed test: add test for governance inv expiration (UdjinM6)
06b4dba0d3 feat: make CInv python implementation aware of governance invs (UdjinM6)
c7c930ece6 fix: use correct condition in logs (UdjinM6)
d41d87a5be fix: use `std::chrono::seconds` (UdjinM6)
d6fe7146ff chore: make clang-format and linter happy (UdjinM6)
b57a9220c1 refactor: sqash 2 hash maps into one, use proper naming (UdjinM6)
4116ba3253 fix: let internal govobj/vote inv request trackers expire after 60 sec (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  Another issue noticed during recent sb voting period. Lots of inv are coming from peers that never send the object/vote they announced. Let's not wait for these forever, 60 seconds should be enough.

  ## What was done?
  Add a "timer" to each request.

  ## How Has This Been Tested?
  Run tests, run a MN on mainnet and check logs.

  ## Breaking Changes
  n/a

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  knst:
    utACK 1f4e1a17ed

Tree-SHA512: f5c5a896f61b5aed27c6f42c926254156c604edb4efd2a3cd3738a7e8d1ed7bafffadc584148e4a5c1a172c2b3d61a06884a1f6d56eb4ebe37a4b0dbc050edd6
2024-08-28 19:29:36 -05:00
pasta
6dbed2ae75
Merge #6192: fix: protect CoinJoinWalletManager::m_wallet_manager_map with Mutex, avoid data race, partial bitcoin#22868
87d775d27c partial bitcoin#22868: Call load handlers without cs_wallet locked (Kittywhiskers Van Gogh)
c602ca15e1 coinjoin: protect `m_wallet_manager_map` with `cs_wallet_manager_map` (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  This pull request aims to deal with regressions ([build](https://gitlab.com/dashpay/dash/-/jobs/7550859495)) spotted in `develop` after the merger of [dash#6143](https://github.com/dashpay/dash/pull/6143), namely, a failure to build the multiprocess variant and two data races, one involving `ChainstateManager::m_best_header` and another involving `CoinJoinWalletManager::m_wallet_manager_map`.

  * Fixes for the build failure and the first data race (b136742f98c6792735232cc170123f849fd13bad and 9e7c6850139e69cb64cfe89813e6f0e4b1e626e1), have been spun off into [dash#6199](https://github.com/dashpay/dash/pull/6199) and merged

  * The second data race is being avoided by protected `m_wallet_manager_map` with a new `RecursiveMutex`, `cs_wallet_manager_map` (the contents of this PR). ~~A version of these changes are available using a regular `Mutex` but prove far more cumbersome ([source](b89457dc9a)) when taking practicalities into account ([comment](https://github.com/dashpay/dash/pull/6192#discussion_r1713207664)).~~ A `Mutex` version is now available courtesy of UdjinM6 (see [patch](ecf5c1ba76)), thanks!

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    light-ACK 87d775d27c

Tree-SHA512: 52545a1547357a45fb6bf4d3948fc75a5e88f1d86e9809b1060007cd74dd383249691d8d9777f647c7534d458fe126ae818fa2b47f0e5b0cee78a469af1ed0c9
2024-08-28 19:28:22 -05:00
pasta
9bcda44b2e
Merge #6130: backport: Merge bitcoin/bitcoin#22337
1973532372 Merge bitcoin/bitcoin#22337: wallet: Use bilingual_str for errors (Samuel Dobson)

Pull request description:

  bitcoin backport

Top commit has no ACKs.

Tree-SHA512: ec638bde047788824961c230804461cee96ddcc0dbe1928bfb8c675c7f7f8c0c104850676a70fb8c9380401ead6d5986022ae3698a2bbc84f679531b11349580
2024-08-28 12:20:51 -05:00
pasta
b9a2f08338
Merge #6071: backport: Merge bitcoin#22619, 22593
ad840ec31a Merge bitcoin#22593: remove `hex_str_to_bytes` helper (Kittywhiskers Van Gogh)
cbc4c63f58 Merge bitcoin/bitcoin#22619: test: refactor: use consistent bytes <-> hex-string conversion in functional test framework (MarcoFalke)

Pull request description:

  bitcoin backports

ACKs for top commit:
  knst:
    utACK ad840ec31a
  kwvg:
    utACK ad840ec31a

Tree-SHA512: 48b4137683daab01a1bf51493d082ec359f80be7a9930b3423476e9ac5f4e73035d0f64d0f8e9b6b0c61b3e06efb648f9cc6bd620088c8cb5f27830157440adb
2024-08-28 12:17:01 -05:00
UdjinM6
c2c4b2b794
fix: release unused memory in CNetMsgMaker::Make() 2024-08-27 20:45:36 +03:00
Samuel Dobson
1973532372
Merge bitcoin/bitcoin#22337: wallet: Use bilingual_str for errors
92993aa5cf37995e65e68dfd6f129ecaf418e01c Change SignTransaction's input_errors to use bilingual_str (Andrew Chow)
171366e89b828a557f8262d9dc14ff7a03f813f7 Use bilingual_str for address fetching functions (Andrew Chow)
9571c69b51115454c6a699be9492024f7b46c2b4 Add bilingual_str::clear() (Andrew Chow)

Pull request description:

  In a couple of places in the wallet, errors are `std::string`. In order for these errors to be translated, change them to use `bilingual_str`.

ACKs for top commit:
  hebasto:
    re-ACK 92993aa5cf37995e65e68dfd6f129ecaf418e01c, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/22337#pullrequestreview-694542729) review, verified with
  klementtan:
    Code review ACK 92993aa5cf37995e65e68dfd6f129ecaf418e01c
  meshcollider:
    Code review ACK 92993aa5cf37995e65e68dfd6f129ecaf418e01c

Tree-SHA512: 5400e419dd87db8c49b67ed0964de2d44b58010a566ca246f2f0760ed9ef6a9b6f6df7a6adcb211b315b74c727bfe8c7d07eb5690b5922fa5828ceef4c83461f
Signed-off-by: Vijay <vijaydas.mp@gmail.com>
2024-08-27 20:08:25 +05:30
Kittywhiskers Van Gogh
87d775d27c
partial bitcoin#22868: Call load handlers without cs_wallet locked
excludes changes in `wallet/context.h` due to `::vpwallets` (and thus,
`cs_wallets`) not being deglobalized yet
2024-08-26 15:55:38 +00:00
Kittywhiskers Van Gogh
c602ca15e1
coinjoin: protect m_wallet_manager_map with cs_wallet_manager_map
Avoid TSan-reported data race

```
WARNING: ThreadSanitizer: data race (pid=374820)
  Read of size 8 at 0x7b140002ce10 by thread T12:
    #0 _M_ptr /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:154:42 (dashd+0xb58e08) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
    #1 get /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:361:21 (dashd+0xb58e08)
    #2 operator-> /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/unique_ptr.h:355:9 (dashd+0xb58e08)
    #3 CoinJoinWalletManager::DoMaintenance() /src/dash/src/coinjoin/client.cpp:1907:9 (dashd+0xb58e08)
    [...]
  Previous write of size 8 at 0x7b140002ce10 by thread T17 (mutexes: write M0):
    #0 operator new(unsigned long) <null> (dashd+0x162657) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
    #1 allocate /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/ext/new_allocator.h:114:27 (dashd+0xb772b4) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
    #2 allocate /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/alloc_traits.h:443:20 (dashd+0xb772b4)
    #3 _M_get_node /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_tree.h:580:16 (dashd+0xb772b4)
    [...]
    #8 CoinJoinWalletManager::Add(CWallet&) /src/dash/src/coinjoin/client.cpp:1898:26 (dashd+0xb58c73) (BuildId: c3fdce9f7e778985a4fb0968ff4506d9ad24d408)
[...]
SUMMARY: ThreadSanitizer: data race [...]
```
2024-08-26 15:55:37 +00:00
Konstantin Akimov
dc59cbc315
fix: optimize includes in gsl/pointer.h to speed up compile time
it removes:
 - algorithm: `forward` is a part of utility not algorithm
 - system_error: `hash` is already declared in memory
2024-08-26 16:16:31 +07:00
Konstantin Akimov
d5e32e1b79
fix: disable gsl iostream feature for faster compile time
We do not use this feature and implementation is trivial: just call `get()`
2024-08-26 16:16:31 +07:00
pasta
f675fcbd25
Merge #6230: fix: resolve GitHub workflow warnings
88bf7a8cb4 fix: resolve GitHub workflow warnings (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

  ## What was done?

  ## How Has This Been Tested?
  Check results for CI and Guix actions (in future PRs? 🤷‍♂️ )

  ## Breaking Changes
  n/a

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

Top commit has no ACKs.

Tree-SHA512: 91464dbb85767e27bd26484cf597f7794e450d56901a407864ce9d3880fb15118047ee040f4e4ab19ea2a0a6dc8aaa6239353427f35f9c7500a6df0fdff6fb6f
2024-08-24 21:28:44 -05:00
Kittywhiskers Van Gogh
ad840ec31a
Merge bitcoin#22593: remove hex_str_to_bytes helper
Signed-off-by: Vijay <vijaydas.mp@gmail.com>
2024-08-25 07:55:20 +05:30
MarcoFalke
cbc4c63f58
Merge bitcoin/bitcoin#22619: test: refactor: use consistent bytes <-> hex-string conversion in functional test framework
5a1bef60a03b57de708a1a751bd90b8245fd8b83 test: refactor: remove binascii from test_framework (Zero-1729)

Pull request description:

  This PR continues the work started in PR #22593, regarding using the `bytes` built-in module. In this PR specifically, instances of `binascii`'s methods `hexlify`, `unhexlify`,  and `a2b_hex` have been replaced with the build-in `bytes` module's `hex` and `fromhex` methods where appropriate to make bytes <-> hex-string conversions consistent across the functional test files and test_framework.

  Additionally, certain changes made are based on the following assumption:

  ```
  bytes.hex(data) == binascii.hexlify(data).decode()
  bytes.hex(data).encode() == binascii.hexlify(data)
  ```

  Ran the functional tests to ensure behaviour is still consistent and changes didn't break existing tests.

  closes #22605

ACKs for top commit:
  theStack:
    Code-review ACK 5a1bef60a03b57de708a1a751bd90b8245fd8b83 🔢

Tree-SHA512: 8f28076cf0580a0d02a156f3e1e94c9badd3d41c3fbdfb2b87cd8a761dde2c94faa5f4c448d6747b1ccc9111c3ef1a1d7b42a11c806b241fa0410b7529e2445f
Signed-off-by: Vijay <vijaydas.mp@gmail.com>
2024-08-25 07:55:13 +05:30
pasta
114d7877d3
Merge #6098: backport: merge bitcoin#22278, #24103, #24235, #24177, #24299, #24917, #22564, #25349, #25571, #17487, #26999, #27011 (blockstorage backports: part 2)
b65863783f merge bitcoin#27011: Add simulation-based `CCoinsViewCache` fuzzer (Kittywhiskers Van Gogh)
1d0e410280 merge bitcoin#26999: A few follow-ups to bitcoin#17487 (Kittywhiskers Van Gogh)
7d837ea5c5 merge bitcoin#17487: allow write to disk without cache drop (Kittywhiskers Van Gogh)
2c758f4ba2 merge bitcoin#25571: Make mapBlocksUnknownParent local, and rename it (Kittywhiskers Van Gogh)
70a91e1d3f merge bitcoin#25349: CBlockIndex/CDiskBlockIndex improvements for safety, consistent behavior (Kittywhiskers Van Gogh)
eca0a64ea1 merge bitcoin#22564: Move mutable globals cleared in ::UnloadBlockIndex to BlockManager (Kittywhiskers Van Gogh)
916b3f0041 merge bitcoin#24917: Make BlockManager::LoadBlockIndex private (Kittywhiskers Van Gogh)
e10ca27fa6 merge bitcoin#24299: UnloadBlockIndex and ChainstateManager::Reset thread safety cleanups (Kittywhiskers Van Gogh)
18aa55be1e merge bitcoin#24177: add missing thread safety lock assertions (Kittywhiskers Van Gogh)
678e67c505 merge bitcoin#24235: use stronger EXCLUSIVE_LOCKS_REQUIRED() (Kittywhiskers Van Gogh)
edc665c80d merge bitcoin#24103: Replace RecursiveMutex m_cs_chainstate with Mutex, and rename it (Kittywhiskers Van Gogh)
d19ffd613a merge bitcoin#22278: Add LIFETIMEBOUND to CScript where needed (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Dependent on https://github.com/dashpay/dash/pull/6097
  * Dependency for https://github.com/dashpay/dash/pull/6067
  * Test failures in `linux64_multiprocess-build` ([build](https://gitlab.com/dashpay/dash/-/jobs/7550924662)) and `linux64_tsan-test` ([build](https://gitlab.com/dashpay/dash/-/jobs/7550924666)) do not stem from this PR but are pre-existing failures in `develop` ([build](https://gitlab.com/dashpay/dash/-/jobs/7550859495), [build](https://gitlab.com/dashpay/dash/-/jobs/7550859499)). A fix for the build failures has been opened as a separate PR.

  ## Breaking Changes

  None observed.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    LGTM, utACK b65863783f
  PastaPastaPasta:
    utACK b65863783f

Tree-SHA512: 1a9c3a41617af274db169db47a9c9fce7ba7ce0b2d68aa75617640a55da11a0fa095cb25ce6a2d38f06d3f6a6cc4c08cb4cf82dca4bdc74192e8882fd5f7052f
2024-08-24 21:12:29 -05:00
UdjinM6
88bf7a8cb4
fix: resolve GitHub workflow warnings 2024-08-24 01:15:31 +03:00
Kittywhiskers Van Gogh
b65863783f
merge bitcoin#27011: Add simulation-based CCoinsViewCache fuzzer 2024-08-23 07:00:39 +00:00
Kittywhiskers Van Gogh
1d0e410280
merge bitcoin#26999: A few follow-ups to bitcoin#17487 2024-08-23 06:57:38 +00:00
Kittywhiskers Van Gogh
7d837ea5c5
merge bitcoin#17487: allow write to disk without cache drop 2024-08-23 06:57:37 +00:00
Kittywhiskers Van Gogh
2c758f4ba2
merge bitcoin#25571: Make mapBlocksUnknownParent local, and rename it 2024-08-23 06:57:37 +00:00
Kittywhiskers Van Gogh
70a91e1d3f
merge bitcoin#25349: CBlockIndex/CDiskBlockIndex improvements for safety, consistent behavior 2024-08-23 06:57:37 +00:00
Kittywhiskers Van Gogh
eca0a64ea1
merge bitcoin#22564: Move mutable globals cleared in ::UnloadBlockIndex to BlockManager
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2024-08-23 06:57:37 +00:00
Kittywhiskers Van Gogh
916b3f0041
merge bitcoin#24917: Make BlockManager::LoadBlockIndex private 2024-08-23 06:57:37 +00:00
Kittywhiskers Van Gogh
e10ca27fa6
merge bitcoin#24299: UnloadBlockIndex and ChainstateManager::Reset thread safety cleanups 2024-08-23 06:57:37 +00:00
Kittywhiskers Van Gogh
18aa55be1e
merge bitcoin#24177: add missing thread safety lock assertions 2024-08-23 06:57:37 +00:00
Kittywhiskers Van Gogh
678e67c505
merge bitcoin#24235: use stronger EXCLUSIVE_LOCKS_REQUIRED() 2024-08-23 06:57:37 +00:00
Kittywhiskers Van Gogh
edc665c80d
merge bitcoin#24103: Replace RecursiveMutex m_cs_chainstate with Mutex, and rename it 2024-08-23 06:57:36 +00:00
Kittywhiskers Van Gogh
d19ffd613a
merge bitcoin#22278: Add LIFETIMEBOUND to CScript where needed 2024-08-23 06:57:36 +00:00
UdjinM6
1f4e1a17ed
test: add test for governance inv expiration 2024-08-22 22:03:48 +03:00
UdjinM6
06b4dba0d3
feat: make CInv python implementation aware of governance invs 2024-08-22 22:01:07 +03:00
UdjinM6
c7c930ece6
fix: use correct condition in logs 2024-08-22 21:30:26 +03:00
pasta
045e178730
Merge #6134: backport: Merge bitcoin#22860
b9213fa976 Merge bitcoin/bitcoin#22860: test: Always clear reject reason in IsStandard tx test (MarcoFalke)

Pull request description:

  bitcoin backports

Top commit has no ACKs.

Tree-SHA512: 7257e9e7bdc6681fcb810f3b4c4034e8afcccdd6f3a6afe63e329e9d211833d55ce87d41c9c4e06af76d04e2ea234fb9d1ffa98108b94e1eddec43fd027bbadc
2024-08-22 10:00:40 -05:00
MarcoFalke
b9213fa976
Merge bitcoin/bitcoin#22860: test: Always clear reject reason in IsStandard tx test
fa1b08eb1413d547b5e322f20e6907b2f827a162 test: Always clear reject reason in IsStandard tx test (MarcoFalke)

Pull request description:

  For some tests the reject reason wasn't cleared between runs and thus subsequent tests might (theoretically) fail to verify the correct reject reason.

ACKs for top commit:
  benthecarman:
    ACK fa1b08eb1413d547b5e322f20e6907b2f827a162
  theStack:
    Code-review ACK fa1b08eb1413d547b5e322f20e6907b2f827a162

Tree-SHA512: fcb727a690f92a4cf06127c302ba464f1e8cb997498e4f7fd9e210d193559b07e6efdb9d5c8a0bef3fe643bdfd5fedd431aaace20978dd49e56b8e770cb9f930
2024-08-22 10:00:01 -05:00
pasta
3f4b50a794
Merge #6222: fix: adjust payee predictions after mn_rr activation, add tests
715bc1af66 test: check `masternode winners` before and after mn_rr (UdjinM6)
9d47cd2226 fix: adjust payee predictions after mn_rr activation (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  Payment predictions in GUI are wrong when mn_rr is active, `masternode winners` RPC is affected by the same issue too. Actual payments aren't affected.

  ## What was done?
  Adjust calculations, add tests for `masternode winners`.

  ## How Has This Been Tested?
  Run dash-qt on testnet, check "Next Payment" on "Masternode" tab. Run tests.

  ## Breaking Changes
  n/a

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK 715bc1a

Tree-SHA512: 293c77974bcb50c6f9c51449d7bb12f89ad8db5871cad3a6083fe1951fe77e0deba8de7688b2f600fabe977bdc7390a66a984a6a076be19183c23742e00e27bf
2024-08-22 08:21:00 -05:00
UdjinM6
d41d87a5be
fix: use std::chrono::seconds 2024-08-22 15:03:13 +03:00
UdjinM6
d6fe7146ff
chore: make clang-format and linter happy 2024-08-22 15:03:13 +03:00
UdjinM6
b57a9220c1
refactor: sqash 2 hash maps into one, use proper naming 2024-08-22 15:03:13 +03:00
UdjinM6
4116ba3253
fix: let internal govobj/vote inv request trackers expire after 60 sec 2024-08-22 15:03:10 +03:00
pasta
f2940c4e9c
Merge #6168: backport: bitcoin-core#gui18, #121, #257, #263, #281, #335, #362, #828, bitcoin#21912, #21942, #21988
c7d3161b3b Merge bitcoin-core/gui#362: Add keyboard shortcuts to context menus (Hennadii Stepanov)
25f87b9434 Merge bitcoin-core/gui#121: Early subscribe core signals in transaction table model (Hennadii Stepanov)
ed56e28a7c Merge bitcoin-core/gui#335: test: Use QSignalSpy instead of QEventLoop (Hennadii Stepanov)
c52b75609b Merge bitcoin-core/gui#281: set shortcuts for console's resize buttons (W. J. van der Laan)
b442a59b6d Merge bitcoin/bitcoin#21988: doc: note that brew installed qt is not supported (W. J. van der Laan)
0e2e315fcc Merge bitcoin/bitcoin#21942: docs: improve make with parallel jobs description. (MarcoFalke)
c2735a8a67 Merge bitcoin/bitcoin#21912: doc: Remove mention of priority estimation (W. J. van der Laan)
1d56d207cd Merge bitcoin-core/gui#257: refactor: Use template function qOverload in signal-slot connections (Hennadii Stepanov)
b5fb559706 Merge bitcoin-core/gui#18: Add peertablesortproxy module (Hennadii Stepanov)
1cdd9fbdf5 refactor: use new QAction style for governance list and masternode list (Konstantin Akimov)
4f89c98dc4 Merge bitcoin-core/gui#263: Revamp context menus (Hennadii Stepanov)
c36bb8e6fb fix: use && in governance urls instead & (Konstantin Akimov)
1e585b1987 Merge bitcoin-core/gui#828: Rendering an amp characters in the wallet name for QMenu (Hennadii Stepanov)

Pull request description:

  ## Issue being fixed or feature implemented
  Just regular backports from bitcoin v22, mostly Qt related

  ## What was done?
  See commits for a list of backports.

  This PR also fixes a rendering an url on Governance tab if it has any '&' inside. see screenshots.

  Original url: `https://example.com/?test=nothing&to=see&&lol` - yes, double '&&' just for test even if url is silly.
  Failed behaviour:
  ![image](https://github.com/user-attachments/assets/ac45c192-7d0e-4cd2-97f8-060af8f3911b)
  Correctly rendered:
  ![image](https://github.com/user-attachments/assets/5e345197-776a-4bb8-9476-cab4aba3429e)

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK c7d3161b3b
  UdjinM6:
    utACK c7d3161b3b

Tree-SHA512: 67e7e8e0ec1a768d1f13baa48c123e4a415d3f32177a427d8117339a5eacf70864ebf46e9f1165bb8a3bf9c231f7929d33ac6aa19742e06a4e19d2f86dda6dc3
2024-08-21 09:11:32 -05:00
pasta
e4e6d9bdf3
Merge #6213: backport: trivial 2024 08 14
c8092fb7fc Merge bitcoin/bitcoin#25214: multiprocess build fix: ipc/capnp/init.capnp.h: No such file or directory (fanquake)
1c40b5d161 Merge bitcoin/bitcoin#25088: Wallet: Ensure m_attaching_chain is set before registering for signals (Andrew Chow)
247fe418d6 Merge bitcoin/bitcoin#24984: wallet: ignore chainStateFlushed notifications while attaching chain (Andrew Chow)
93e7c38788 Merge bitcoin/bitcoin#24338: util: Work around libstdc++ create_directories issue (fanquake)
e80c2a383a Merge bitcoin/bitcoin#24308: util: use stronger-guarantee rename method (MarcoFalke)
f3775d96b0 Merge bitcoin/bitcoin#24251: Re-enable windows path tests disabled by #20744 (fanquake)
f2f298286e Merge bitcoin/bitcoin#23707: fuzz: Fix RPC internal bug detection (MarcoFalke)
d06b693864 Merge bitcoin/bitcoin#23654: fuzz: Rework rpc fuzz target (MarcoFalke)
e45229de76 Merge bitcoin/bitcoin#23385: refactor: get wallet path relative to wallet_dir (fanquake)
9d9d97d4fd Merge bitcoin/bitcoin#22218: multiprocess: Add ipc::Context and ipc::capnp::Context structs (MarcoFalke)

Pull request description:

  ## Issue being fixed or feature implemented
  Batch of trivial backports

  ## What was done?

  ## How Has This Been Tested?
  Built and tests locally

  ## Breaking Changes

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  knst:
    utACK c8092fb7fc
  UdjinM6:
    utACK c8092fb7fc

Tree-SHA512: 7e3f95737b8c945e737396f130cddca24f71aa4d7f5ccb8ead62b8ba90e187af7036c1be0984e7f000a058606cc74e5381768dc2aad6e56d07a8238934198dcc
2024-08-21 09:00:07 -05:00