Allow `upgradetohd` in IBD, better errors, no GUI lock-up
Pls see individual commits. Most of it is changes in whitespaces, might
want to use ?w=1 to review i.e.
https://github.com/dashpay/dash/pull/5455/files?w=1
run tests, try `upgradetohd` on testnet
n/a
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
TL;DR: Should hopefully fix crashes like
https://gitlab.com/dashpay/dash/-/jobs/4522256293
In dashd we flush all callbacks first and then destroy `g_txindex`. In
tests we had to move `g_txindex` to `TestChainSetup` and its dtor is
executed first, so the order is broken. It also explains why this crash
happens so rare. In most cases tx index is up to date and you need some
kind of a hiccup for scheduler to lag behind a bit. Basically, between
`g_txindex.reset()` and `FlushBackgroundCallbacks`
`BaseIndex::BlockConnected` finally arrives. But it’s processed on a
(now) null instance hence a crash. If it’s earlier - it’s processed
normally, if it’s later - it’s flushed without execution, so there is a
tiny window to catch this crash.
## What was done?
Give tx index a bit of time to process everything
## How Has This Been Tested?
run tests (but this crash is rare 🤷♂️ )
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
It's super slow for wallets with 100.000s of txes to process lots of
notifications produced by rescan. Skip them all and simply refresh the
whole wallet instead. In my case (500k+ txes testnet wallet) gui update
after `rescanblockchain` time is down from _forever_ to ~30 seconds.
Same for `wipewallettxes true` (#5451 ). Gui update after
`wipewallettxes`/`wipewallettxes false` is instant (cause there are no
txes anymore) vs _forever_ before the patch.
## What was done?
refresh the whole wallet when notification queue is above 10K operations
actual changes (ignoring whitespaces):
d013cb4f5c
## How Has This Been Tested?
running on top of #5451 and #5452 , wiping and rescanning w/ and w/out
this patch.
## Breaking Changes
should be none
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
Given the hard fork that happened on testnet, there is now lots of the
transactions that were made on the fork that is no longer valid. Some
transactions could be relayed and mined again but some like coinjoin
mixing won't be relayed because of 0 fee and transactions spending
coinbases from the forked branch are no longer valid at all.
## What was done?
Introduce `wipewallettxes` RPC and `wipetxes` command for `dash-wallet`
tool to be able to get rid of some/all txes in the wallet.
## How Has This Been Tested?
run tests, use rpc/command on testnet wallet
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
Added the filter `hpmn` for both `masternodelist` and `protx list` rpcs.
## What was done?
## How Has This Been Tested?
Calling this RPC on Testnet.
## Breaking Changes
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
---------
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
## Issue being fixed or feature implemented
@ogabrielides @kittywhiskers I somehow failed to add you guys to the
list of v19.2 contributors 🙈 sorry!
## What was done?
## How Has This Been Tested?
## Breaking Changes
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
before 12%
<img width="1538" alt="image"
src="https://github.com/dashpay/dash/assets/6443210/fa5043fb-4e48-4728-bfaf-8636d5c20a8c">
after 10%
<img width="1544" alt="image"
src="https://github.com/dashpay/dash/assets/6443210/1df6aff4-2901-4af1-b421-3604f54df157">
Redundant rehash
Avoid redundant rehash
Reindexed 0-500000 on testnet
None
_Go over all the following points, and put an `x` in all the boxes that
apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
Mining blocks with a specific version can be useful on testnet and
devnets too
## What was done?
lift restrictions for `-blockversion`
## How Has This Been Tested?
it should just work :)
## Breaking Changes
n//a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
f827e151a2ce96e14aadb9e7d25045fe0a8afbd2 refactor: remove straggling boost::mutex usage (fanquake)
Pull request description:
After the merge of #18710, the linter is warning:
```bash
A new Boost dependency in the form of "boost/thread/mutex.hpp" appears to have been introduced:
src/sync.cpp:#include <boost/thread/mutex.hpp>
src/test/sync_tests.cpp:#include <boost/thread/mutex.hpp>
^---- failure generated from test/lint/lint-includes.sh
```
#18710 removed `boost/thread/mutex.hpp` from lint-includes, however in the interim #19337 was merged, which introduced more `boost::mutex` usage.
Given we no longer use `boost::mutex`, just remove the double lock test and remaining includes.
ACKs for top commit:
laanwj:
Code review ACK f827e151a2ce96e14aadb9e7d25045fe0a8afbd2
hebasto:
ACK f827e151a2ce96e14aadb9e7d25045fe0a8afbd2
Tree-SHA512: f738b12189fe5b39db3e8f8231e9002714413a962eaf98adc84a6614fa474df5616358cfb1c89b92a2b0564efa9b704a774c49d4a25dca18a0ccc3cd9eabfc0a
89bdad5b25ae4ac03a486f729a5b58ae6f21946d RPC/Wallet: unloadwallet: Allow specifying wallet_name param matching RPC endpoint (Luke Dashjr)
Pull request description:
Allow specifying the `wallet_name` param to `unloadwallet` on RPC wallet endpoints, so long as it matches the endpoint wallet.
ACKs for top commit:
jonatack:
ACK 89bdad5b25ae4ac03a486f729a5b58ae6f21946d
MarcoFalke:
review ACK 89bdad5b25ae4ac03a486f729a5b58ae6f21946d
Tree-SHA512: efb399c33f7b5596870a26a8680f453ca47aa7a6db4e550f9435d13044f1c4bad0ae11e8f0205213409d08b75c4188c3be782e54aafab1f65b97eb8cf5c252a9
95975dd08d8fdaaeaf28e0d06b861ce2748c17b6 sync: detect double lock from the same thread (Vasil Dimov)
4df6567e4cbb4677e8048de2f8008612e1b860b9 sync: make EnterCritical() & push_lock() type safe (Vasil Dimov)
Pull request description:
Double lock of the same (non-recursive) mutex from the same thread would produce an undefined behavior. Detect this from `DEBUG_LOCKORDER` and react similarly to the deadlock detection.
This came up during discussion in another, related PR: https://github.com/bitcoin/bitcoin/pull/19238#discussion_r442394521.
ACKs for top commit:
laanwj:
code review ACK 95975dd08d8fdaaeaf28e0d06b861ce2748c17b6
hebasto:
re-ACK 95975dd08d8fdaaeaf28e0d06b861ce2748c17b6
Tree-SHA512: 375c62db7819e348bfaecc3bd82a7907fcd8f5af24f7d637ac82f3f16789da9fc127dbd0e37158a08e0dcbba01a55c6635caf1d8e9e827cf5a3747f7690a498e
fabecce71909c984504c21fa05f91d5f1b471e8c net: Treat raw message bytes as uint8_t (MarcoFalke)
Pull request description:
Using `uint8_t` from the beginning when messages are `recv`ed has two style benefits:
* The signedness is clear from reading the code, as it does not depend on the architecture
* When passing the bytes on, the need for static signedness casts is dropped, making the code a bit less verbose and more coherent
ACKs for top commit:
laanwj:
Code review ACK fabecce71909c984504c21fa05f91d5f1b471e8c
theStack:
Code Review ACK fabecce71909c984504c21fa05f91d5f1b471e8c
jonatack:
Tested ACK fabecce71909c984504c21fa05f91d5f1b471e8c
Tree-SHA512: e6d9803c78633fde3304faf592afa961ff9462a7912d1da97a24720265274aa10ab4168d71b6ec2756b7448dd42585321afee0e5c889e705be778ce9a330d145
fa5ed3b4ca609426b2622cad235e107d33db7b30 net: Use Span in ReceiveMsgBytes (MarcoFalke)
Pull request description:
Pass a data pointer and a size as span in `ReceiveMsgBytes` to get the benefits of a span
ACKs for top commit:
jonatack:
ACK fa5ed3b4ca609426b2622cad235e107d33db7b30 code review, rebased to current master 12a1c3ad1a43634, debug build, unit tests, ran bitcoind/-netinfo/getpeerinfo
theStack:
ACK fa5ed3b4ca609426b2622cad235e107d33db7b30
Tree-SHA512: 89bf111323148d6e6e50185ad20ab39f73ab3a58a27e46319e3a08bcf5dcf9d6aa84faff0fd6afb90cb892ac2f557a237c144560986063bc736a69ace353ab9d
6927933782acb9b158787e6f35debb916793f6b1 [net processing] Add ChainSyncTimeoutState default initializers (John Newbery)
55966e0cc03f0e380d21a9434b048d4d515b6729 [net processing] Remove CNodeState ctor body (John Newbery)
Pull request description:
This addresses the two outstanding review comments from #21370.
ACKs for top commit:
practicalswift:
cr ACK 6927933782acb9b158787e6f35debb916793f6b1: patch looks correct
hebasto:
ACK 6927933782acb9b158787e6f35debb916793f6b1, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: b3ef5c8a096e447887df255406b3a760f01c73e2b942374595416b4b4031fc69b89cd93168c45040489d581f340b2a62d3fbabd207d4307f587c00a7a7daacd1
c92387232f750397da7d131f262c150a608408c2 refactor: Extract ParseOpCode from ParseScript (João Barbosa)
Pull request description:
Seems more natural to have `mapOpNames` "hidden" in `ParseOpCode` than in `ParseScript`.
A second lookup in `mapOpNames` is also removed.
ACKs for top commit:
laanwj:
ACK c92387232f750397da7d131f262c150a608408c2
theStack:
re-ACK c92387232f750397da7d131f262c150a608408c2
Tree-SHA512: d59d1964760622cf365479d44e3e676aa0bf46b60e77160140d967e012042df92121d3224c7551dc96eff5ff3294598cc6bade82adb3f60d28810e18e60e1257
33330778230961cfbf2a24de36b5877e395cc596 rpc: Adjust witness-tx deserialize error message (MarcoFalke)
cccc7525697e7b8d99b545e34f0f504c78ffdb94 rpc: Properly deserialize txs with witness before signing (MarcoFalke)
Pull request description:
Signing a transaction can only happen when the transaction has inputs. A transaction with inputs can always be deserialized as witness-transaction. If `try_no_witness` decoding is attempted, this will lead to rare intermittent failures.
Fixes#18803
ACKs for top commit:
achow101:
ACK 33330778230961cfbf2a24de36b5877e395cc596
ajtowns:
ACK 33330778230961cfbf2a24de36b5877e395cc596
Tree-SHA512: 73f8a5cdfe03fb0e68908d2fa09752c346406f455694a020ec0dd1267ef8f0a583b8e84063ea74aac127106dd193b72623ca6d81469a94b3f5b3c766ebf2c42b
9d09132be4ff99f98ca905c342347d5f35f13350 CConnman: initialise at declaration rather than in Start() (Anthony Towns)
Pull request description:
Ensure nMaxOutboundTotalBytesSentInCycle and nMaxOutboundCycleStartTime are initialized even if CConnman::Start() is not called. Prevents failures in test/fuzz/connman when run under valgrind.
ACKs for top commit:
practicalswift:
ACK 9d09132be4ff99f98ca905c342347d5f35f13350: patch looks correct!
MarcoFalke:
review ACK 9d09132be4ff99f98ca905c342347d5f35f13350 , checked that we call Start only once and in the same scope where connman is constructed (AppInitMain) 💸
jnewbery:
Code review ACK 9d09132be4
Tree-SHA512: 1c6c893e8c616a91947a8cc295b0ba508af3ecfcdcd94cdc5f95d808cc93c6d1a71fd24dcc194dc583854e9889fb522ca8523043367fb0263370fbcab08c6aaa
## Issue being fixed or feature implemented
Fixes issue #5497.
## What was done?
Checks if settings file is empty, and deletes it if that's the case.
It will will be generated with default value `{}` afterwards.
## How Has This Been Tested?
Running Dash Qt on regtest masternode with `--nocleanup` and
`./src/qt/dash-qt --regtest --datadir=`
## Breaking Changes
No
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
The logic for additional indexes is incomplete, handling of P2PK on
block disconnect is broken (luckily no one is using P2PK and reorgs are
rare) and there are a few other small issues that would be nice to have
fixed.
## What was done?
Pls see individual commits
## How Has This Been Tested?
Run `feature_dbcrash.py`, it should succeed (NOTE: it takes ~30 minutes
to complete, that's normal).
Run `feature_addressindex.py`, `feature_timestampindex.py` and
`feature_spentindex.py` (and other tests) should still succeed too.
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
8f5dc8800aeb524eee2fa2451cd22883b7b2bfec test: display command line options passed to send_cli() in debug log (Jon Atack)
Pull request description:
as per https://github.com/bitcoin/bitcoin/pull/18691#discussion_r411382589, and revert two cli calls changed in #18691 from rpc commands back to command line options (these were the only occurrences).
ACKs for top commit:
MarcoFalke:
ACK 8f5dc8800aeb524eee2fa2451cd22883b7b2bfec
Tree-SHA512: fcb3eca00aa4099066028c90d5e50a02e074366e09a17f5f5b937d9f7562dd054ff65681aa0ad4c94f6de1e98b1e2b9ac4cd084ddc297010253989a80483b1b9
## Issue being fixed or feature implemented
This is an implementation of DIP0027 "Credit Asset Locks".
It's a mechanism to fluidly exchange between Dash and credits.
## What was done?
This pull request includes:
- Asset Lock transaction
- Asset Unlock transaction (withdrawal)
- Credit Pool in coinbase
- Unit tests for Asset Lock/Unlock tx
- New functional test `feature_asset_locks.py`
RPC: currently locked amount (credit pool) is available through rpc call
`getblock`.
## How Has This Been Tested?
There added new unit tests for basic checks of transaction validity
(asset lock/unlock).
Also added new functional test "feature_asset_locks.py" that cover
typical cases, but not all corner cases yet.
## Breaking Changes
This feature should be activated as hard-fork because:
- It adds 2 new special transaction and one of them [asset unlock tx]
requires update consensus rulels
- It adds new data in coinbase tx (credit pool)
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
**To release DIP 0027**
- [x] I have assigned this pull request to a milestone
---------
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
## Issue being fixed or feature implemented
Refusing to process `dsq` will result in node not being able to process
`dstx`es later.
## What was done?
## How Has This Been Tested?
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
V19 is active on mainnet/testnet now, no need to check activation bits
anymore. This PR also bumps `MinBIP9WarningHeight` to
post-v19-activation height which should stop `unknown new rules
activated (versionbit 8)` warning from appearing.
## What was done?
Bury v19, bump `MinBIP9WarningHeight`
## How Has This Been Tested?
Run tests, reindex on mainnet/testnet.
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
5aadd4be1883386a04bef6a04e9a1142601ef7a7 Convert amounts from float to decimal (Prayank)
Pull request description:
> decimal is preferred in accounting applications
https://docs.python.org/3.8/library/decimal.html
Decimal type saves an exact value so better than using float.
~~3 variables declared with type as 'Decimal' in [test/functional/mempool_accept.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/mempool_accept.py): fee, fee_expected, output_amount~~
~~Not required to convert to string anymore for using the above variables as decimal~~
+ fee, fee_expected, output_amount
~~+ 8 decimal places~~
+ Using value of coin['amount'] as decimal and removed 'int'
+ Removed unnecessary parentheses
+ Remove str() and use quotes
Fixes https://github.com/bitcoin/bitcoin/issues/20011
ACKs for top commit:
guggero:
ACK 5aadd4be1883386a04bef6a04e9a1142601ef7a7
Tree-SHA512: 5877cf3837e5b65bec0fc8909de141a720bfa02a747513e21d20f3c41ec0cfecc524d2c347a96596b0a1a97900da2acf08b799f26b11d537e4dcddc6ce45f38e
d76925478efd35e6fd835370639f2139b28381e4 [doc] Clarify semantic of peer's m_protect w.r.t to outbound eviction logics (Antoine Riard)
ac71fe936da290adf5a3155fe8db5f78b485f1f1 [doc] Clarify scope of eviction protection of outbound block-relay peers (Antoine Riard)
Pull request description:
Block-relay-only peers were introduced by #15759. According to its
author, it was intented to make them only immune to outbound peer
rotation-based eviction and not from all eviction as modified comment
leans to think of.
Clearly indicate that outbound block-relay peers aren't protected
from eviction by the bad/lagging chain logic.
Fix#19863
ACKs for top commit:
naumenkogs:
ACK d76925478efd35e6fd835370639f2139b28381e4
jonatack:
ACK d76925478efd35e6fd835370639f2139b28381e4
Tree-SHA512: 597fbd62838a6e39276024165b11514cad20a2e9d33cf9202d261cbadcb62b2df427c858e0cb57e585840d4c1d4600104aa53916bb868541f2580e4eed9b4b52
fab94534b64593be1620c989bf69eb02e1be9b1b doc: Document that wallet salvage is experimental (MarcoFalke)
Pull request description:
See #20151
ACKs for top commit:
practicalswift:
ACK fab94534b64593be1620c989bf69eb02e1be9b1b: user safety first
hebasto:
ACK fab94534b64593be1620c989bf69eb02e1be9b1b, maybe capitalize into "WARNING"?
meshcollider:
Trivial ACK fab94534b64593be1620c989bf69eb02e1be9b1b
Tree-SHA512: 94912c491facc485293e4333066057933d706d84c7172f615296e7ba998c583c8bd07e751e6f00cd6576e7791007ace321f959181f7bf6a4e15e10d7ec8a1b7e
0956e46bff7f0b6da65a4de6d4f8261fe9d7055c test: use zero-argument super() shortcut (Python 3.0+) (Sebastian Falbesoner)
Pull request description:
This mini-PR replaces all calls to `super(...)` with arguments with the zero-argument shortcut `super()` where applicable. See [PEP 3135](https://www.python.org/dev/peps/pep-3135/#specification):
> The new syntax:
>
> super()
>
> is equivalent to:
>
> super(__class__, <firstarg>)
>
> where __class__ is the class that the method was defined in, and <firstarg> is
> the first parameter of the method (normally self for instance methods, and cls
> for class methods).
ACKs for top commit:
fanquake:
ACK 0956e46bff7f0b6da65a4de6d4f8261fe9d7055c
Tree-SHA512: 4ac66fe7ab2be2e8a514e5fcfc41dbb298f21b23ebb7b7b0310d704b0b3cef8adf287a8d80346d1ea9418998c597b4f0ff1f66148d0d806bb43db6607e0fe1cf
fc289b7898fb90d4800675b69c0bb9b42df5599f wallet: Refactor WalletRescanReserver to use wallet reference (João Barbosa)
Pull request description:
Simple refactor to `WalletRescanReserver` to use wallet reference instead of pointer.
Complements #18259.
ACKs for top commit:
MarcoFalke:
ACK fc289b7898fb90d4800675b69c0bb9b42df5599f
Tree-SHA512: b03e33f2d9df2870436aa3284137fd022dd89ea96a1b170fa27f8685ad4f986e6c4ba5975a84966c30d18430a4014d7d8740a1dff2f985c9ef8226ed18e69db9
